Reworked Gitlab-CI

[pearzt]

With this PR, I propose a new, revised Gitlab-CI. These are the key changes:

• Environment is managed using Spack
• Designed to run on Lichtenberg's RHEL9 nodes
• Extended matrix of tested GCC and LLVM versions
• Improved CI runtimes, as dependencies (cxxopts, etc.) are managed by Spack and do not need to be rebuilt. Currently, a CI run takes around 6 minutes
• Removed some CI jobs (stripped builds, mostly)
• PGIS is no longer tested in the CI, as it is deprecated anyway

[TimHeldmann]

I going through this PR together with @pearzt, I think it looks good to me.

There are two things that remain after this PR is merged:
One thing that still irks me, but should not be part of this PR, is how many different testing scripts we now have.
In the future I hope we could get some of these scripts either unified, or ported to ctest (or whatever we come up with).

Secondly, it seems there have been problems with compiling if SPD_LOG tries to compile with exceptions.
I have no idea why this would occur. I think this needs some further investigation.

[jplehr]

I personally disagree with removing PGIS from the build. I understand that it is deprecated. The reason I disagree is because it is the one (only one?) tool that acts as a full end-to-end test for the library and package components.

I simply do not trust the rest of our testing enough.

If this is what we want to do, I won't get in the way, but I want to raise my concern.

[pearzt]

@jplehr

I personally disagree with removing PGIS from the build. I understand that it is deprecated.

I excluded PGIS mostly due to it's dependence on a very old version of Extra-P which we would need to make available on Lichtenberg or build via build_submodules.sh (and sacrifice the short CI runtime). If someone wants to put in the work to make this work, I have nothing against it.

The reason I disagree is because it is the one (only one?) tool that acts as a full end-to-end test for the library and package components.

IMHO, PGIS's contribution to the overall MetaCG testing is small. This is kind of anecdotal, but I can't remember a CI failure over the last months that was caused by PGIS catching an error in the library.

[jplehr]

IMHO, PGIS's contribution to the overall MetaCG testing is small. This is kind of anecdotal, but I can't remember a CI failure over the last months that was caused by PGIS catching an error in the library.

This is less about PGI's ability to uncover / detect bugs in the library and more to have something non-trivial that actually uses the library API. I don't want us to work on an API that is only covered with unit tests as I have seen an API deteriorate from usability despite best intentions, simply because no one actually used it.

So, if we think that with the increase in use of the API through the additional tools and downstream / out of tree consumers, I am less concerned about this point.

Note, however, that at least the out of tree consumers won't "catch" any weird decisions/directions we take/implement here. I am aware that having a larger consumer of the API does not guarantee a good API design, as I know of quirks we had in the graph lib in the presence of PGIS.

[jplehr]

What are chances that tracking devel here breaks us for no reason?
At least that is my understanding from glimpsing at the CMake for this dependency.

[pearzt]

I think that's indeed a risk. Maybe be should talk to @ahueck about defining a metavirt release that we could then reference. This is unrelated to the CI though, right?

[jplehr]

yes in the sense of the technical changes

[jplehr]

Does this set the compilers for all the jobs? Wouldn't that ruin the matrix job purpose?

[pearzt]

It does set the compiler, but clang and clang++ refer to binaries in different LLVM installations, depending on which environment is active.

[pearzt]

I've just added another variant that uses GCC to build MetaCG

[jplehr]

I also think we should build some different configurations, i.e., enable / disable some components to see if that still works as intended.