Support for importing layers in the block CIM format. #2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Henry Wang <henwang@amazon.com>
Signed-off-by: Henry Wang <henwang@amazon.com>
Signed-off-by: Henry Wang <henwang@amazon.com>
Signed-off-by: Henry Wang <henwang@amazon.com>
…ntries Signed-off-by: Henry Wang <henwang@amazon.com>
Signed-off-by: Henry Wang <henwang@amazon.com>
The default transport are used in 3 places: 1. `ConfigureDefaultRegistries` (no `hosts_dir` is set) 2. `ConfigureHosts` (when `hosts_dir` is set) 3. in cri service 2 and 3 use/duplicate the same default transport, whereas 1 uses go's default Client/Transport This PR moves the default transport to a common funcion (can pass in tls config). Signed-off-by: Jin Dong <djdongjin95@gmail.com>
ambarve
force-pushed
the
cimfs_layer_refactor
branch
from
6c2b23d to
098d303
Compare
ambarve
force-pushed
the
blocked_cim
branch
2 times, most recently
from
e6566fd to
3d8b95d
Compare
This is a planned follow-on from containerd#10721 primarily at the request of @fuweid, exchanging MNT_DETACH at unmount time for MOUNT_ATTR_RDONLY at mount time. The effect is to increase risk of unmount failure due to EBUSY (as observed in the wild) but add an additional protection that the then-leaked bind mount does not act as a conduit for inadvertent modification of the underlying data, including our own efforts to clean up the mountpoint. Tests covering the lifecycle of the temporary idmap mounts and integrity of the underlying lower layer data is also included in the normal and failed-unmount case. Fixes containerd#10704 Signed-off-by: Mike Baynton <mike@mbaynton.com>
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
ambarve
force-pushed
the
cimfs_layer_refactor
branch
2 times, most recently
from
bc5d8ad to
19ef8aa
Compare
ambarve
force-pushed
the
blocked_cim
branch
2 times, most recently
from
5e91e62 to
a9250ea
Compare
Signed-off-by: ningmingxiao <ning.mingxiao@zte.com.cn>
ambarve
force-pushed
the
cimfs_layer_refactor
branch
from
19ef8aa to
ad15e70
Compare
ambarve
force-pushed
the
cimfs_layer_refactor
branch
from
ad15e70 to
b81ace8
Compare
Clarify Go client API guidance
…nsport Unify default transport in docker resolver
Otherwise its a matter of luck that the man directory is created before man dir generation. Bug: https://bugs.gentoo.org/880057 Signed-off-by: Alfred Wingate <parona@protonmail.com>
Fuzz integration tests on Github Action panics because it cannot find the containerd PATH. ``` failed to start daemon: failed to start daemon: exec: "containerd": executable file not found in $PATH: panic: fatal [recovered] panic: fatal ``` It's because in Action the `OUT` env var (/github/workspace/build-out) is different compared to oss-fuzz. Signed-off-by: Jin Dong <djdongjin95@gmail.com>
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
The EROFS differ only applies to EROFS layers which are marked by a special file `.erofslayer` generated by the EROFS snapshotter. Why it's needed? Since we'd like to parse []mount.Mount directly without actual mounting and convert OCI layers into EROFS blobs, `.erofslayer` gives a hint that the active snapshotter supports the output blob generated by the EROFS differ. I'd suggest it could be read together with the next commit. Signed-off-by: cardy.tang <zuniorone@gmail.com> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
It allows us to mount each EROFS blob layer (generated by the EROFS differ) independently, or use the "unpacked" fs/ directories (if some other differ is used.) Currently, it's somewhat like the overlay snapshotter, but I tend to separate the new EROFS logic into a self-contained component, rather than keeping it tangled in the very beginning. Existing users who use the overlay snapshotter won't be impacted at all but they have a chance to use this new snapshotter to leverage the EROFS filesystem. Signed-off-by: cardy.tang <zuniorone@gmail.com> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
…alOpts client: add WithExtraDialOpts option
…_actions/lycheeverse/lychee-action-2.2.0 build(deps): bump lycheeverse/lychee-action from 2.1.0 to 2.2.0
Signed-off-by: Derek McGowan <derek@mcg.dev>
update to go1.23.5 / go1.22.11
Update cimfs snapshotter & differ for new hcsshim interface
…ules/github.com/tchap/go-patricia/v2-2.3.2 build(deps): bump github.com/tchap/go-patricia/v2 from 2.3.1 to 2.3.2
Adds new criteria and schedule for time based releases. Adds more ownership and roles for the different phases of the release process. Signed-off-by: Derek McGowan <derek@mcg.dev>
Signed-off-by: Derek McGowan <derek@mcg.dev>
…ules/otel-e4b1f3184c build(deps): bump the otel group across 1 directory with 8 updates
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.1 to 3.28.6. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@b6a472f...17a820b) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/stale](https://github.com/actions/stale) from 9.0.0 to 9.1.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@28ca103...5bef64f) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/attest-build-provenance](https://github.com/actions/attest-build-provenance) from 2.1.0 to 2.2.0. - [Release notes](https://github.com/actions/attest-build-provenance/releases) - [Changelog](https://github.com/actions/attest-build-provenance/blob/main/RELEASE.md) - [Commits](actions/attest-build-provenance@7668571...520d128) --- updated-dependencies: - dependency-name: actions/attest-build-provenance dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
…_actions/actions/stale-9.1.0 build(deps): bump actions/stale from 9.0.0 to 9.1.0
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
The NRI plugins define hooks on the Pod and the Container lifecycle and provide contextual information for each of them in the corresponding hooks. The StopPodSandbox hook already has the namespaces created and the plugins using that hook may require that information, so we should be able to do a best effort to pass it down to the NRI plugins. Signed-off-by: Antonio Ojea <aojea@google.com>
…_actions/actions/attest-build-provenance-2.2.0 build(deps): bump actions/attest-build-provenance from 2.1.0 to 2.2.0
…_actions/github/codeql-action-3.28.6 build(deps): bump github/codeql-action from 3.28.1 to 3.28.6
Bumps [google-github-actions/auth](https://github.com/google-github-actions/auth) from 2.1.7 to 2.1.8. - [Release notes](https://github.com/google-github-actions/auth/releases) - [Changelog](https://github.com/google-github-actions/auth/blob/main/CHANGELOG.md) - [Commits](google-github-actions/auth@6fc4af4...71f9864) --- updated-dependencies: - dependency-name: google-github-actions/auth dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.6 to 3.28.8. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@17a820b...dd74661) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [google-github-actions/upload-cloud-storage](https://github.com/google-github-actions/upload-cloud-storage) from 2.2.1 to 2.2.2. - [Release notes](https://github.com/google-github-actions/upload-cloud-storage/releases) - [Changelog](https://github.com/google-github-actions/upload-cloud-storage/blob/main/CHANGELOG.md) - [Commits](google-github-actions/upload-cloud-storage@386ab77...7c6e11c) --- updated-dependencies: - dependency-name: google-github-actions/upload-cloud-storage dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
…m-controller Fix state/root bug in shim sandbox controller
…_actions/github/codeql-action-3.28.8 build(deps): bump github/codeql-action from 3.28.6 to 3.28.8
…_actions/google-github-actions/upload-cloud-storage-2.2.2 build(deps): bump google-github-actions/upload-cloud-storage from 2.2.1 to 2.2.2
…_actions/google-github-actions/auth-2.1.8 build(deps): bump google-github-actions/auth from 2.1.7 to 2.1.8
nri: make OCI spec available on StopPodSandbox
Update RELEASES.md for new release schedule and LTS policy
[Feat] erofs snapshotter and differ
Adds a new diff plugin that can import image layers in the block CIM format using the new block CIM layer writer added in hcsshim repo. This commit also makes another important change in the way a diff is applied when using CimFS based layer writers. Currently, the diff plugins call archive.Apply to apply a diff and pass a function (that can actually apply the diff) as an argument (via archive.ApplyOptions). This allows the callers to call archive.Apply with either a custom applier function or if the caller doesn't pass such a function archive.Apply uses the default naive diff applier. However, there is drawback to this approach. The applier function passed to the `archive.Apply` call needs to follow a specific signature. This signature expects it that all parent layers are represented as an array of strings. In cases like CimFS, we can't easily represent a set of layers as strings (unless we encode extra data in those strings in a hacky way). To get around this problem, the diff plugins for CimFS based layers, skip the archive.Apply call and directly call the layer writer instead. Signed-off-by: Amit Barve <ambarve@microsoft.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a new diff plugin that can import image layers in the block CIM format using the new block CIM layer writer added in hcsshim repo.
This commit also makes another important change in the way a diff is applied when using CimFS based layer writers. Currently, the diff plugins call archive.Apply to apply a diff and pass a function (that can actually apply the diff) as an argument (via archive.ApplyOptions). This allows the callers to call archive.Apply with either a custom applier function or if the caller doesn't pass such a function archive.Apply uses the default naive diff applier.
However, there is drawback to this approach. The applier function passed to the
archive.Applycall needs to follow a specific signature. This signature expects it that all parent layers are represented as an array of strings. In cases like CimFS, we can't easily represent a set of layers as strings (unless we encode extra data in those strings in a hacky way). To get around this problem, the diff plugins for CimFS based layers, skip the archive.Apply call and directly call the layer writer instead.