experimental changes for referrers API, ORAS artifact manifest support #5
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| name: Release oras-project/registry container image | ||
|
|
||
| on: | ||
| push: | ||
| tags: | ||
| - "v[0-9]+.[0-9]+.[0-9]+-alpha" | ||
|
|
||
| jobs: | ||
| publish: | ||
| name: Build and publish container image | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: read | ||
| packages: write | ||
| env: | ||
| REGISTRY: ghcr.io | ||
| REPOSITORY: ${{ format('{0}/registry', github.repository_owner) }} | ||
| DOCKER_BUILDTAGS: "include_oss include_gcs" | ||
| CGO_ENABLED: 1 | ||
| GO111MODULE: "auto" | ||
| GOPATH: ${{ github.workspace }} | ||
| GOOS: linux | ||
| COMMIT_RANGE: ${{ github.event_name == 'pull_request' && format('{0}..{1}',github.event.pull_request.base.sha, github.event.pull_request.head.sha) || format('{0}..{1}', github.event.before, github.event.after) }} | ||
|
|
||
| steps: | ||
| - name: Get git tag | ||
| id: get_git_tag | ||
| run: echo ::set-output name=git_tag::${GITHUB_REF#refs/tags/} | ||
|
|
||
| - name: Check out source code | ||
| if: ${{ success() }} | ||
| uses: actions/checkout@v2 | ||
| with: | ||
| ref: ${{ steps.get_git_tag.outputs.git_tag }} | ||
|
|
||
| - name: Set docker image tag | ||
| env: | ||
| GIT_TAG: ${{ steps.get_git_tag.outputs.git_tag }} | ||
| id: get_image_tag | ||
| run: echo ::set-output name=docker_tag::${GIT_TAG} | ||
|
|
||
| - name: Login to GitHub Container Registry | ||
| uses: docker/login-action@v1 | ||
| with: | ||
| registry: ${{ env.REGISTRY }} | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Build and push | ||
| if: ${{ success() }} | ||
| uses: docker/build-push-action@v2 | ||
| with: | ||
| context: . | ||
| file: ./Dockerfile | ||
| platforms: linux/amd64 | ||
| push: true | ||
| tags: | | ||
| ${{ env.REGISTRY }}/${{ env.REPOSITORY }}:${{ steps.get_image_tag.outputs.docker_tag }} | ||
| ${{ env.REGISTRY }}/${{ env.REPOSITORY }}:latest |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,73 +1,160 @@ | ||
| # Distribution | ||
|
|
||
| This fork of [distribution/distribution](distribution-distribution) provides | ||
| an experimental implementation of [reference types](reference-types). | ||
|
|
||
| Features supported: | ||
|
|
||
| - :heavy_check_mark: PUT ORAS Artifact Manifest | ||
| - :heavy_check_mark: GET ORAS Artifact Manifest | ||
| - :heavy_check_mark: LIST referrers | ||
| - [ ] Pagination support | ||
| - [ ] Garbage Collection of reference types | ||
|
|
||
| To power the `/referrers` API, the implementation creates and uses an index. | ||
| See [referrers.md](docs/referrers.md) for details. | ||
|
|
||
| ## Usage - Push, Discover, Pull | ||
|
|
||
| The following steps illustrate how ORAS artifacts can be stored and retrieved | ||
| from a registry. The artifact in this example is a Notary V2 | ||
| [signature](signature). | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| - Local registry prototype instance | ||
| - [docker-generate](https://github.com/shizhMSFT/docker-generate) | ||
| - [nv2](https://github.com/notaryproject/nv2) | ||
| - `curl` | ||
| - `jq` | ||
|
|
||
| ### Push an image to your registry | ||
|
|
||
| ```shell | ||
| # Initialize local registry variables | ||
| regIp="127.0.0.1" && \ | ||
| regPort="5000" && \ | ||
| registry="$regIp:$regPort" && \ | ||
| repo="busybox" && \ | ||
| tag="latest" && \ | ||
| image="$repo:$tag" && \ | ||
| reference="$registry/$image" | ||
|
|
||
| # Pull an image from docker hub and push to local registry | ||
| docker pull $image && \ | ||
| docker tag $image $reference && \ | ||
| docker push $reference | ||
| ``` | ||
|
|
||
| ### Generate image manifest and sign it | ||
|
|
||
| ```shell | ||
| # Generate self-signed certificates | ||
| openssl req \ | ||
| -x509 \ | ||
| -sha256 \ | ||
| -nodes \ | ||
| -newkey rsa:2048 \ | ||
| -days 365 \ | ||
| -subj "/CN=$regIp/O=example inc/C=IN/ST=Haryana/L=Gurgaon" \ | ||
| -addext "subjectAltName=IP:$regIp" \ | ||
| -keyout example.key \ | ||
| -out example.crt | ||
|
|
||
| # Generate image manifest | ||
| manifestFile="manifest-to-sign.json" && \ | ||
| docker generate manifest $image > $manifestFile | ||
|
|
||
| # Sign manifest | ||
| signatureFile="manifest-signature.jwt" && \ | ||
| nv2 sign --method x509 \ | ||
| -k example.key \ | ||
| -c example.crt \ | ||
| -r $reference \ | ||
| -o $signatureFile \ | ||
| file:$manifestFile | ||
| ``` | ||
|
|
||
| ### Obtain manifest and signature digests | ||
|
|
||
| ```shell | ||
| manifestDigest="sha256:$(sha256sum $manifestFile | cut -d " " -f 1)" && \ | ||
| signatureDigest="sha256:$(sha256sum $signatureFile | cut -d " " -f 1)" | ||
| ``` | ||
|
|
||
| ### Create an Artifact file referencing the manifest that was signed and its signature as blob | ||
|
|
||
| ```shell | ||
| artifactFile="artifact.json" && \ | ||
| artifactMediaType="application/vnd.cncf.oras.artifact.manifest.v1+json" && \ | ||
| artifactType="application/vnd.cncf.notary.v2" && \ | ||
| signatureMediaType="application/vnd.cncf.notary.signature.v2+jwt" && \ | ||
| signatureFileSize=`wc -c < $signatureFile` && \ | ||
| manifestMediaType="$(cat $manifestFile | jq -r '.mediaType')" && \ | ||
| manifestFileSize=`wc -c < $manifestFile` | ||
|
|
||
| cat <<EOF > $artifactFile | ||
| { | ||
| "mediaType": "$artifactMediaType", | ||
| "artifactType": "$artifactType", | ||
| "blobs": [ | ||
| { | ||
| "mediaType": "$signatureMediaType", | ||
| "digest": "$signatureDigest", | ||
| "size": $signatureFileSize | ||
| } | ||
| ], | ||
| "subject": { | ||
| "mediaType": "$manifestMediaType", | ||
| "digest": "$manifestDigest", | ||
| "size": $manifestFileSize | ||
| } | ||
| } | ||
| EOF | ||
| ``` | ||
|
|
||
| ### Obtain artifact digest | ||
|
|
||
| ```shell | ||
| artifactDigest="sha256:$(sha256sum $artifactFile | cut -d " " -f 1)" | ||
| ``` | ||
|
|
||
| ### Push signature and artifact | ||
|
|
||
| ```shell | ||
| # Initiate blob upload and obtain PUT location | ||
| blobPutLocation=`curl -I -X POST -s http://$registry/v2/$repo/blobs/uploads/ | grep "Location: " | sed -e "s/Location: //;s/$/\&digest=$signatureDigest/;s/\r//"` | ||
|
|
||
| # Push signature blob | ||
| curl -X PUT -H "Content-Type: application/octet-stream" --data-binary @"$signatureFile" $blobPutLocation | ||
|
|
||
| # Push artifact | ||
| curl -X PUT --data-binary @"$artifactFile" -H "Content-Type: $artifactMediaType" "http://$registry/v2/$repo/manifests/$artifactDigest" | ||
| ``` | ||
|
|
||
| ### List referrers | ||
|
|
||
| ```shell | ||
| # Retrieve referrers | ||
| curl -s "http://$registry/oras/artifacts/v1/$repo/manifests/$manifestDigest/referrers?artifactType=$artifactType" | jq | ||
| ``` | ||
|
|
||
| ### Verify signature | ||
|
|
||
| ```shell | ||
| # Retrieve signature | ||
| artifactDigest=`curl -s "http://$registry/oras/artifacts/v1/$repo/manifests/$manifestDigest/referrers?artifactType=$artifactType" | jq -r '.references[0].digest'` && \ | ||
| signatureDigest=`curl -s "http://$registry/oras/artifacts/v1/$repo/manifests/$artifactDigest" | jq -r '.blobs[0].digest'` && \ | ||
| retrievedSignatureFile="retrieved-signature.json" && \ | ||
| curl -s http://$registry/v2/$repo/blobs/$signatureDigest > $retrievedSignatureFile | ||
|
|
||
| # Verify signature | ||
| nv2 verify \ | ||
| -f $retrievedSignatureFile \ | ||
| -c example.crt \ | ||
| file:$manifestFile | ||
| ``` | ||
|
|
||
| [distribution-distribution]: https://github.com/distribution/distribution | ||
| [reference-types]: https://github.com/oras-project/artifacts-spec | ||
| [signature]: https://github.com/notaryproject/nv2/tree/prototype-2/docs/nv2 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| [[__TOC__]] | ||
|
|
||
| # ORAS Artifacts Distribution | ||
|
|
||
| This document describes an experimental prototype that implements the | ||
| [ORAS Artifact Manifest](https://github.com/oras-project/artifacts-spec) spec. | ||
|
|
||
| ## Implementation | ||
|
|
||
| To power the [/referrers](https://github.com/oras-project/artifacts-spec/blob/main/manifest-referrers-api.md) API, the | ||
| referrers of a manifest are indexed in the repository store. The following example illustrates the creation of this | ||
| index. | ||
|
|
||
| The `nginx:v1` image is already persisted: | ||
|
|
||
| - repository: `nginx` | ||
| - digest: `sha256:111ma2d22ae5ef400769fa51c84717264cd1520ac8d93dc071374c1be49a111m` | ||
| - tag: `v1.0` | ||
|
|
||
| The repository store layout is represented as: | ||
|
|
||
| ```bash | ||
| <root> | ||
| └── v2 | ||
| └── repositories | ||
| └── nginx | ||
| └── _manifests | ||
| └── revisions | ||
| └── sha256 | ||
| └── 111ma2d22ae5ef400769fa51c84717264cd1520ac8d93dc071374c1be49a111m | ||
| └── link | ||
| ``` | ||
|
|
||
| Push a signature as blob and an ORAS Artifact that contains a blobs property referencing the signature, with the | ||
| following properties: | ||
|
|
||
| - digest: `sha256:222ibbf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cb222i` | ||
| - `subjectManifest` digest: `sha256:111ma2d22ae5ef400769fa51c84717264cd1520ac8d93dc071374c1be49a111m` | ||
| - `artifactType`: `application/vnd.example.artifact` | ||
|
|
||
| On `PUT`, the artifact appears as a manifest revision. Additionally, an index entry is created under | ||
| the subject to facilitate a lookup to the referrer. The index path where the entry is added is | ||
| `/ref/<artifactType>`, as shown below. | ||
|
|
||
| ``` | ||
| <root> | ||
| └── v2 | ||
| └── repositories | ||
| └── nginx | ||
| └── _manifests | ||
| └── revisions | ||
| └── sha256 | ||
| ├── 111ma2d22ae5ef400769fa51c84717264cd1520ac8d93dc071374c1be49a111m | ||
| │ ├── link | ||
| │ └── ref | ||
| │ └── digest(application/vnd.example.artifact) | ||
| │ └── sha256 | ||
| │ └── 222ibbf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cb222i | ||
| │ └── link | ||
| └── 222ibbf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cb222i | ||
| └── link | ||
| ``` | ||
|
|
||
| Push another ORAS artifact with the following properties: | ||
|
|
||
| - digest: `sha256:333ic0c33ebc4a74a0a554c86ac2b28ddf3454a5ad9cf90ea8cea9f9e75c333i` | ||
| - `subjectManifest` digest: `sha256:111ma2d22ae5ef400769fa51c84717264cd1520ac8d93dc071374c1be49a111m` | ||
| - `artifactType`: `application/vnd.another.example.artifact` | ||
|
|
||
| This results in an addition to the index as shown below. | ||
|
|
||
| ``` | ||
| <root> | ||
| └── v2 | ||
| └── repositories | ||
| └── nginx | ||
| └── _manifests | ||
| └── revisions | ||
| └── sha256 | ||
| ├── 111ma2d22ae5ef400769fa51c84717264cd1520ac8d93dc071374c1be49a111m | ||
| │ ├── link | ||
| │ └── ref | ||
| │ ├── digest(application/vnd.example.artifact) | ||
| │ │ └── sha256 | ||
| │ │ └── 222ibbf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cb222i | ||
| │ │ └── link | ||
|
There was a problem hiding this comment. Another thought is to have a separate folder for references Couple of benefits are, it aligns with the current model where separate folder exists for entity type like manifests, tags, layers etc. In the similar fashion |
||
| │ └── digest(application/vnd.another.example.artifact) | ||
| │ └── sha256 | ||
| │ └── 333ic0c33ebc4a74a0a554c86ac2b28ddf3454a5ad9cf90ea8cea9f9e75c333i | ||
| │ └── link | ||
| ├── 222ibbf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cb222i | ||
| │ └── link | ||
| └── 333ic0c33ebc4a74a0a554c86ac2b28ddf3454a5ad9cf90ea8cea9f9e75c333i | ||
| └── link | ||
| ``` | ||
|
There was a problem hiding this comment. Can we support up a configuration that would enable this API? |
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.