Static dtors are not run on DSO unload

[raziel-]

GCC11.3.0

Unloading a DSO that registers destructors/functions to be run on exit, will always crash when the main program exits

static dtors are not run on DSO unload, but at program exit. Since the SO is already unloaded, this causes a crash.

A simplified version of the repro code is at
https://github.com/PushmePullyu/sotest/tree/repro-dlclose-dtors
which removes the SDL dependency

See here for the full discussion:
https://bugs.scummvm.org/ticket/15015

[salass00]

Constructor/destructor functions that are added in the .ctors/.dtors section should already be called at dlclose() for DSOs loaded by dlopen() as this is handled by libdl/elf.library.

From what I understand however gcc does not use that method, but instead __cxa_atexit() and if this is not available (as in our case) atexit(). As atexit() is used explicitly to add callbacks that are to be called on program exit (it has no knowledge of DSOs) this cannot work when called from a DSO loaded by dlopen().

Adding __cxa_atexit() support to newlib should be possible (in fact the framework for this already exists there), but in addition to this newlib specific support will need to be added to libdl or libdl functions will need to be integrated into newlib (basically dlclose() will need to call __call_exitprocs() with __dso_handle as the 2nd parameter before unloading the DSO). An alternative solution to modifying libdl would be to call __call_exitprocs() from __shlib_call_destructors() but since IIRC newer elf.library versions bypass the __shlib_call_constructors()/__shlib_call_destructors() functions entirely (not sure why this change was made) this might not be possible any more.

I'm still not quite sure where the __dso_handle comes from and how the compiler gets access to it but I assume it's a symbol that's supposed to be in the DSO somewhere.

[raziel-]

@salass00

anything i can do (with my little to non existing knowledge) to help?

btw, the original sotest branch might vanish, so i forked it here:
https://github.com/raziel-/dlclose-dtors-sotest

[salass00]

After some more googling I have found that there is a function __cxa_finalize() for calling atexit functions registered by a specific DSO (using this is preferable to using the newlib private __call_exitprocs() function). Both __cxa_atexit() and __cxa_finalize() need to be added to the INewlib interface and to libc.

[raziel-]

so, the ball is back in the field of the newlib devs?

[afxgroup]

I know one of them!! :D

[raziel-]

@afxgroup

would you alert him about this thread?
or are you referring to yourself? 😀

[afxgroup]

He is @salass00 :)

[raziel-]

oh 😀

[salass00]

The __cxa_atexit() and __cxa_finalize() functions are now exported from the newlib library and from the libc. I also added the __dso_handle hidden symbol to shcrtbegin.o and __shlib_call_destructors() calls __cxa_finalize(&__dso_handle) before iterating over the .dtors array as usual.

Compiling the libfoo.so with -fuse-cxa-atexit a call to __cxa_atexit() is generated by gcc instead of atexit() however sotest still crashes because (as I suspected) elf.library 53.37+ is bypassing the __shlib_call_destructors() mechanism.

[raziel-]

wow, thanks a lot :-)

is a new newlib needed, or is a recompiled gcc enough?

[salass00]

It will require an updated newlib.library and associated developer files. The current gcc should be fine though as long as you use -fuse-cxa-atexit when compiling DSOs.

An updated elf.library will likely be needed as well because of the problem I described earlier. Adding the __cxa_finalize() call directly to libdl.so is more complicated and does nothing for programs that do not use libdl.so but instead use the elf.library directly to open and close DSOs (they would need similar fixes as well).

[raziel-]

may i be so bold to ask you to relay those informations to the people in charge of newlib, elf.library and the other parts that needs updating?

im afraid im just a small unimportant user

[salass00]

I already wrote a mail to the os4-developer mailing list. A good suggestion that I got was to use a .dtors destructor function to call __cxa_finalize() however implementing it is turning out to be problematic still as I only have the shcrtbegin.o and shcrtend.o files to work with.

[raziel-]

thank you very much

[salass00]

My initial thought was that the .dtors section was being badly generated because when I tried to dump the contents of it with objdump it outputted the first sentinel entry (0xffffffff) and after that just "...", however this turned out to be a red herring. The real reason that it wasn't working was that IElf->DLClose() didn't have any code to call destructors when a DSO is unloaded (open count goes to zero).

After adding the missing functionality to elf.library the sotest program works as it's supposed to.