flatpak again

[drahnr]

No description provided.

[drahnr]

@tinywrkb take a look at https://ci.spearow.io/teams/main/pipelines/oregano/jobs/build-pkg-flatpak/builds/22 - it's still not happy but I am not sure the fuse calls can be avoided.

[tinywrkb]

Try disabling rofiles-fuse by running Flatpak builder with --disable-rofiles-fuse.

--disable-rofiles-fuse
Disable the use of rofiles-fuse to optimize the cache use via hardlink checkouts.

rofiles-fuse is being used because it's recommended with ostree overlays see here for more details.

[drahnr]

Even that is not enough:

Starting build of com.github.drahnr.oregano
========================================================================
Building module intltool in /flatpak-builder/build/intltool-1
========================================================================
bwrap: Creating new namespace failed: Operation not permitted
Error: module intltool: Child process exited with code 1

since the whole thing already runs in an unprivileged container

[tinywrkb]

I believe Flatpak is using user namespaces so should be enabled in the CI container.

[drahnr]

Even with kernel commandline systemd.unified_cgroup_hierarchy=1 namespace.unpriv_enable=1 flatpak is unhappy. I did not plan on putting this in an unprivileged container. In a perfect world I would like to just cut out bubblewrap or pass sufficient debug infos to it.

Feel free to inspect it via making changes in a PR to https://github.com/drahnr/oregano/blob/master/.concourse/tasks/flatpak.yml

[tinywrkb]

Are you sure that user namespaces are enabled?

Flatpak set the sandbox with Bubblewrap so the following should work in the container because we don't use namespaces:

bwrap \
  --bind /usr /usr \
  --dev /dev \
  --proc /proc \
  --symlink usr/bin /bin \
  --symlink usr/bin /sbin \
  --symlink usr/lib /lib \
  --symlink usr/lib /lib64 \
  --tmpfs /etc \
  --tmpfs /tmp \
  --tmpfs /var \
  -- cat /proc/self/stat

On the other hand, we can create a PID user namespace by adding --unshare-pid:

bwrap \
  --bind /usr /usr \
  --dev /dev \
  --proc /proc \
  --symlink usr/bin /bin \
  --symlink usr/bin /sbin \
  --symlink usr/lib /lib \
  --symlink usr/lib /lib64 \
  --tmpfs /etc \
  --tmpfs /tmp \
  --tmpfs /var \
  --unshare-pid \
  -- cat /proc/self/stat

You should be able to confirm that the issue is with user namespaces by running bwrap with strace and looking for a failed unshare or clone calls with CLONE_NEWUSER flag. See user_namespaces.7 man page.

With --unshare-all options I'm seeing something like this.

clone(child_stack=NULL, flags=CLONE_NEWNS|CLONE_NEWCGROUP|CLONE_NEWUTS|CLONE_NEWIPC|CLONE_NEWUSER|CLONE_NEWPID|CLONE_NEWNET|SIGCHLDstrace: Process 2379502 attached
) = 2379502

You need to confirm that user namespaces is actually enabled in the container's running kernel. This is what I'm seeing kernel 5.11.7.

$ zgrep CONFIG_USER_NS /proc/config.gz

CONFIG_USER_NS=y
CONFIG_USER_NS_UNPRIVILEGED=y

Check also unprivileged_userns_clone value

$ cat /proc/sys/kernel/unprivileged_userns_clone

1

[drahnr]

So it turns out my host kernel does not have CONFIG_USER_NS_UNPRIVILEGED in the config (as in does not exist).

This seems to be related: https://bugs.archlinux.org/task/62990

[drahnr]

I think the easiest way would be to use --with-priv-mode=none with bubblewrap as used with flatpack-builder - Q: how do I inject this? I did not see any options in the manual?

[drahnr]

$ bwrap \
>   --bind /usr /usr \
>   --dev /dev \
>   --proc /proc \
>   --symlink usr/bin /bin \
>   --symlink usr/bin /sbin \
>   --symlink usr/lib /lib \
>   --symlink usr/lib /lib64 \
>   --tmpfs /etc \
>   --tmpfs /tmp \
>   --tmpfs /var \
>   -- cat /proc/self/stat
bwrap: Creating new namespace failed: Operation not permitted

with and without --unshare-pid.

[tinywrkb]

Bubblewrap is probably still setting a mount namespace and I don't believe there's a way around this.

I think the easiest way would be to use --with-priv-mode=none with bubblewrap as used with flatpack-builder - Q: how do I inject this? I did not see any options in the manual?

I don't think it's possible. flatpak-builder is wrapping flatpak build calls while processing the manifest file (yaml or json file), and user namespaces sandboxing is hardcoded in Flatpak.

The easy way to solve this is to use a CI with user namespaces working.

The more complex solution is (which I haven't confirmed is working):

• Enter a chroot following how flatpak run or flatpak build is setting up the sandbox. flatpak run and I believe also flatpak build will show you the complete bubblewrap command when running with -vv command-line switch.
• Build the app and modules in the chroot and install to a /app as the prefix.
• Create a target directory to export to a repo. It gonna have files folder that is /app from the chroot and the rest will be created by running flatpak build-finish. I don't see a reason why a build-finish call would need to run in a sandbox using namespaces. If it does need user namespaces then it might be possible to complete this step manually.
• Export the app from the target directory to a Flatpak repo with flatpak build-export. There's a --disable-sandbox option so it's possible that this command can run without user namespaces.
• Create a bundle from the repo with flatpak build-bundle. There's again this question if a sandbox is needed and used by this command.

[tinywrkb]

p.s. A good example of how to build a Flatpak manually is the Firefox packaging.
edit: This example is missing the compilation step where I suggested building in a chroot, making use of the Flatpak Sdk mounted to /usr.