Kernel: Multiverse not usable in loadable kernel modules

[florommel]

Multiverse holds a list of all multiversed functions at runtime. The list is populated during startup by the generated __multiverse_init constructor functions. The function multiverse_init gathers the additional runtime information (called manually once at startup).

This approach is problematic when using multiverse in loadable kernel modules as they can be loaded at any time (after multiverse_init) and unloaded (leaving an invalid entry in the mv_information list, corrupting it).

[florommel]

A similar problem may also arise in userspace when a program using multiverse includes a (static) library that also uses the plugin..

[vrothberg]

Let's forget how the multiverse code looks like at the moment - it's not carved into stone.

module support
If you want to support kernel modules, you must extract the multiverse data from a module. You could hook or extend the kernel's loader, but I can't imagine that this is a change kernel guys would accept. Or you add an extra multiverse_load_module that is called at module_init and module_unload at module_exit respectively. In both cases, multiverse data must be managed dynamically, which is also required when removing specific functions. Given the fact that a huge amount of code in distributions resides in modules, it would be nice to support kernel modules. But it seems prone to race conditions. In this case, a big multiverse lock could help or if we use nice kernel lists, we could use RCU. When going down this path, more and more functionality becomes explicit, so a user must really know what she's doing.

IMHO: Multiverse is a (really exotic) compiler feature and it sounds unnatural to me to optimize code I don't know :-)

@mcgrof , @agraf - What's your opinion? Do you see an immediate benefit of using multiverse in modules?

static libraries
Could you explain what could break for static libraries? I don't see a real difference to multiple object files using multiverse.

[florommel]

Yes, some kind of module_init and module_unload functionality would solve the problem. The difficult part will be to collect all the multiversed functions and variables in the module on init or unload.

Okay, static libraries are probably not a real problem ;) The scenario I thought of: Program A uses Library B. Both use Multiverse. Where is multiverse_init called? Should the library do that in an init function?

[vrothberg]

A library should not call multiverse_init as this would remove all explicit control from the user of this library, and that's really bad. That's similar to multiple object files calling multiverse_init.

[stettberger]

As loading a module does call the constructors of the module (at least if CONFIG_CONSTRUCTORS is enabled) adds the descriptor list to internal list, it would corrupt the current state. If we head for module and shared library support, we should make the preprocessing done in multiverse_init() idempotent: It should be callable any number of times. In that scenario, a library could call (an we cannot prevent it from doing it, we only can discourage it) multiverse_init. For this, we should use a second list to hold all the initialized data, multiverse_init() then transfers all descriptors from the incoming queue (filled by the constructors) to the actual working list.

For now I would suggest, the following: step 1) use multiverse in a kernel without module support to get it working. In this stage, we can and should do first measurements. 2) make multiverse_init() idempotent and call it from the module (or mix it into the module loading code). Ignore unloading of modules for the sake of simplicity as this could become a little bit more ugly.

[mcgrof]

Module support should be hard and a selling point should be ease of integration, so the more complex that becomes the harder I think folks will like it.

I've had to also think about module support for the alternative to compiler multiverse support which I had worked on: linker tables. Upon review it would seem that other than addressing things on module init a typical solution to hard module problems which also rely on some kernel internals is to address hacks on find_module_sections() and use a special ELF section. Refer to find_module_sections() on kernel/module.c. This is for instance how jump labels are supported on modules, refer to references with "__jump_table". This is also how we support constructors on modules -- refer to ".ctors" references. That said, using custom ELF sections is a pain in the ass currently, but linker tables actually helps address these shortcomings by enabling custom ELF sections to be declared only using C code. My current linker-table implementation does not yet support modules but this is how I'd go about it. We'd add parsing custom module linker table sections once so custom linker tables would then always be parsed.

I've rebased my latest code to next-20170620 but I haven't build tested this yet. I will have to respin soon based on the last feedback I got back in January, I have not gotten a chance since then to go back to it.

https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux-next.git/log/?h=20170620-linker-tables-v8

If this seems like a viable strategy let me know. I should also note that linker tables can also supplement multiverse when compiler support is not present, however it would require code specific binary patching.

Concurrency is certainly an issue that would need to be addressed, but perhaps jump labels are an example of a model to follow.

What's the status of this git tree BTW -- could it soon be made public?

[vrothberg]

Module support should be hard and a selling point should be ease of integration, so the more complex that becomes the harder I think folks will like it.

👍

What's the status of this git tree BTW -- could it soon be made public?

It's already public and has moved to luhsra, my former group in Hanover.