feat: add support to Debian kernel source builds and kernel patches#97
feat: add support to Debian kernel source builds and kernel patches#97
Conversation
There was a problem hiding this comment.
@bakhtin Are these three files things we want to include on all of our debian-based kernels?
There was a problem hiding this comment.
if they work for all our projects then yes. We can also merge all 3 fragments into one.
There was a problem hiding this comment.
@alexhulbert should I move into kernel and merge these files? For now they are under buildernet/kernel/config.d/
There was a problem hiding this comment.
Yes, they should be moved to the kernel directory and added to the defaults. The buildernet folder is deleted by my PR anyway, since we're going to swap it out with the real one from the buildernet branch.
| CONFIG_CRYPTO_SHA256_SSSE3=y | ||
| CONFIG_CRYPTO_SHA512_SSSE3=y | ||
| CONFIG_CRYPTO_SIMD=y | ||
| # CONFIG_DEFAULT_CUBIC is not set |
There was a problem hiding this comment.
@bakhtin I'm worried that these are just comments and they won't actually override anything. Perhaps we should set them to =n manually?
There was a problem hiding this comment.
I haven't edited any of the kernel configs manually, they are all produced with the in-tree helper scripts after tweaking the parameters with menuconfig. I don't fully understand how these parameters are later parsed by the kernel build scripts but I can confirm that final image has a default congestion algorithm set to bbr.
I also discourage to edit kernel configs manually as you don't know the full dependency chain of every parameter. The order the parameters appear in the config also seem to matter.
There was a problem hiding this comment.
According to https://www.spinics.net/lists/kernel/msg4259558.html:
"Kconfig accepts both '# CONFIG_FOO is not set' and 'CONFIG_FOO=n' as a valid input."
So these do correctly disable the options.
I can't find a good docs about this, but e.g. this: zephyrproject-rtos/zephyr#5443 (comment).
And I agree with @bakhtin to not edit generated config fragments manually. @alexhulbert can you confirm you're good with this?
There was a problem hiding this comment.
I'm still not 100% convinced. Could you explicitly test that adding a "# XYZ is not set" line after a XYZ=y line disables the option on a parameter whose default is false?
bakhtin
left a comment
There was a problem hiding this comment.
Good start. But so far this looks a bit bloated and too verbose.
| CONFIG_CRYPTO_SHA256_SSSE3=y | ||
| CONFIG_CRYPTO_SHA512_SSSE3=y | ||
| CONFIG_CRYPTO_SIMD=y | ||
| # CONFIG_DEFAULT_CUBIC is not set |
There was a problem hiding this comment.
I haven't edited any of the kernel configs manually, they are all produced with the in-tree helper scripts after tweaking the parameters with menuconfig. I don't fully understand how these parameters are later parsed by the kernel build scripts but I can confirm that final image has a default congestion algorithm set to bbr.
I also discourage to edit kernel configs manually as you don't know the full dependency chain of every parameter. The order the parameters appear in the config also seem to matter.
There was a problem hiding this comment.
if they work for all our projects then yes. We can also merge all 3 fragments into one.
kernel/mkosi.build
Outdated
| rm -f "$build_dir/.version" | ||
|
|
||
| mkosi-chroot --chdir "$chroot_build_dir" make olddefconfig | ||
| mkosi-chroot --chdir "$chroot_build_dir" make -j "$(nproc 2>/dev/null || echo 2)" bzImage ARCH=x86_64 CONFIG_EFI_STUB=y |
There was a problem hiding this comment.
@alexhulbert I remember we agreed to build a DEB package with the kernel and install it normally in the mkosi.conf file as a normal package.
@fkondej If you're gonna do this make sure to not build unnecessary artifacts. In trunk/buildernet I have this: export DEB_BUILD_PROFILES='nodoc noudeb pkg.linux.nosource pkg.linux.notools pkg.linux.nokerneldbg pkg.linux.nokerneldbginfo pkg.linux.nometa' to avoid building extra dependencies.
There was a problem hiding this comment.
Agreed on DEB package
There was a problem hiding this comment.
Done. building .deb with DEB_BUILD_PROFILES. The .deb is copied to $PACKAGEDIR and installed via VolatilePackages.
There was a problem hiding this comment.
@bakhtin I'm pretty sure the DEB_BUILD_PROFILES line is doing nothing. Those build profiles are hardcoded directly into the debian control file, so they're ignored if you call make bindeb-pkg directly. Also, when calling that make target directly, KDEB_SOURCENAME defaults to linux-upstream. So I think the correct line here is DEB_BUILD_PROFILES='pkg.linux-upstream.nokernelheaders pkg.linux-upstream.nokerneldbg'. The other ones aren't relevant anyway since the bindeb-pkg target doesn't make doc/udeb/etc deb files.
buildernet/mkosi.conf
Outdated
| [Build] | ||
| Environment=LIGHTHOUSE_BINARY RETH_BINARY RBUILDER_BINARY | ||
| KERNEL_VERSION_DEBIAN=6.16 | ||
| KERNEL_CONFIG_SNIPPETS_BUILDERNET=kernel/configs/debian/01-sane-defaults,kernel/configs/debian/02-disable-unneeded,kernel/configs/debian/03-bbr-net-fq |
There was a problem hiding this comment.
If would be easier to just have a script to find all the snippets in some well-known directory to avoid adding a new one here every time.
There was a problem hiding this comment.
Done: env vars KERNEL_CONFIG_SNIPPETS* and KERNEL_PATCHES* points now to directories */kernel/config.d and */kernel/patches, and the build script auto-discovers all files in there.
@bakhtin @alexhulbert does this match what you wanted?
| # KERNEL_VERSION must match a linux-source package available in this snapshot | ||
| Mirror=https://snapshot.debian.org/archive/debian/20251113T083151Z/ |
There was a problem hiding this comment.
This isn't accurate since the base module doesn't prescribe an explicit timestamp. This comment should probably be removed to avoid confusion.
|
|
||
| CleanPackageMetadata=true | ||
| # Kernel .deb is built by kernel/mkosi.build and placed in $PACKAGEDIR | ||
| VolatilePackages=linux-image-6.16.3-mkosi-cloud |
| dpkg-dev | ||
| debhelper | ||
| libdw-dev | ||
| python3 |
There was a problem hiding this comment.
Are you sure we need python3 to build the Debian kernel?
| [Build] | ||
| Environment=KERNEL_CONFIG_SNIPPETS=kernel/snippets/ubuntu.config KERNEL_CONFIG_SNIPPETS_BOB=bob-common/kernel.config | ||
| Environment=KERNEL_CONFIG_SNIPPETS_BOB=bob-common/kernel/config.d | ||
| KERNEL_PATCHES_BOB=bob-common/kernel/patches |
There was a problem hiding this comment.
We don't have bob kernel patches
|
|
||
| [Distribution] | ||
| Mirror=https://snapshot.debian.org/archive/debian/20251113T083151Z/ | ||
|
|
||
| [Build] | ||
| ToolsTreeMirror=https://snapshot.debian.org/archive/debian/20251113T083151Z/ |
There was a problem hiding this comment.
These shouldn't be removed. This snapshot should have new enough kernels.
| export LOCALVERSION # e.g. -mkosi-cloud → kernel version becomes 6.16.3-mkosi-cloud | ||
| # TODO: Explore whether these profiles apply to upstream bindeb-pkg: | ||
| # https://github.com/torvalds/linux/blob/master/scripts/package/mkdebian | ||
| export DEB_BUILD_PROFILES='nodoc noudeb pkg.linux.nosource pkg.linux.notools pkg.linux.nokerneldbg pkg.linux.nokerneldbginfo pkg.linux.nometa' |
There was a problem hiding this comment.
Replace with pkg.linux-upstream.nokernelheaders pkg.linux-upstream.nokerneldbg (see my comment above in the thread with Artyom)
kernel/mkosi.build
Outdated
| rm -f "$build_dir/.version" | ||
|
|
||
| mkosi-chroot --chdir "$chroot_build_dir" make olddefconfig | ||
| mkosi-chroot --chdir "$chroot_build_dir" make -j "$(nproc 2>/dev/null || echo 2)" bzImage ARCH=x86_64 CONFIG_EFI_STUB=y |
There was a problem hiding this comment.
@bakhtin I'm pretty sure the DEB_BUILD_PROFILES line is doing nothing. Those build profiles are hardcoded directly into the debian control file, so they're ignored if you call make bindeb-pkg directly. Also, when calling that make target directly, KDEB_SOURCENAME defaults to linux-upstream. So I think the correct line here is DEB_BUILD_PROFILES='pkg.linux-upstream.nokernelheaders pkg.linux-upstream.nokerneldbg'. The other ones aren't relevant anyway since the bindeb-pkg target doesn't make doc/udeb/etc deb files.
| @@ -0,0 +1 @@ | |||
| # CONFIG_MODULES is not set No newline at end of file | |||
There was a problem hiding this comment.
Missing trailing newline. This is very important for the merge script so that the first line of one file isn't combined with the last line of another file
| echo "ERROR: merge_config.sh not found in kernel source" >&2 | ||
| exit 1 | ||
| fi | ||
| cloud_config_xz="$BUILDROOT/usr/src/linux-config-${KERNEL_VERSION}/config.amd64_none_cloud-amd64.xz" |
There was a problem hiding this comment.
Even though we're going to hardcode the cloud flavor into an env var at the top of this file, this should still reference the flavor and not separately hardcode the word cloud
| cp "$kernel_file" "$DESTDIR/usr/lib/modules/$KERNEL_VERSION/vmlinuz" | ||
| rm -f "$config_file" | ||
| # Clean up unwanted build artifacts | ||
| rm -f "$BUILDROOT/build"/*.deb "$BUILDROOT/build"/*.buildinfo "$BUILDROOT/build"/*.changes |
There was a problem hiding this comment.
This could potentially delete artifacts written by other modules. The kernel should be built in a subdirectory of /build and the whole subdirectory should be cleaned up when its done
No description provided.