CI: Explore consolidating Linux/Windows test workflows?

[leofang]

#600 opens up the door toward a single (Bash-based) implementation of the test workflow, which was previously not possible due to lack of Git Bash on the GH-hosted Windows GPU runner. It was the main reason that we had to re-implement test-wheel-windows.yml in Powershell in the first place (#444 (comment)). Note that our build workflow is already cross-platform (and Bash-based, since all of our build runners have Bash pre-installed -- trivial on Linux, and thanks to GH-hosted runner images on Windows).

There might still be a few kinks, so a naive unification likely won't work (ex: container vs VM image, gcc vs msvc, ...). But I think most job steps are enough identical that we could probably extract out a reusable workflow, e.g. test-wheel-common.yml, for both OS to call.

[leofang]

(Assigning to Marcus to triage/evaluate/decide in light of the ongoing CI refactoring #576.)

[leofang]

also cc @shwina for vis

[leofang]

#638 is one example showing the drawback of the status quo: Instead of just updating one file, we need to update two.

[cryos]

Yes moving all the Windows stuff to bash, and unifying on some common scripts would help a lot.

[leofang]

There might still be a few kinks, so a naive unification likely won't work (ex: container vs VM image, gcc vs msvc, ...).

As a reminder to myself why an indirection might still be needed: actions/runner#904 (comment)

[cryos]

There might still be a few kinks, so a naive unification likely won't work (ex: container vs VM image, gcc vs msvc, ...).

As a reminder to myself why an indirection might still be needed: actions/runner#904 (comment)

I was more thinking at the bash script level for more reuse/central place to manage install, build, etc. Not container or action level where I don't see a full merge being feasible.

[leofang]

I guess you're right about this. The reusable bits will likely have to happen at the bash script level, not at the action or workflow level, because this is another thing that cannot be expressed today 😞 (jobs.<job_id>.container is not allowed when invoking a reusable workflow):

  test-linux:
    # use container 
    container:
      image: ubuntu:24.04
    uses:
      ./.github/workflows/test-wheel-common.yml
    ...

  test-windows:
    # don't use container 
    uses:
      ./.github/workflows/test-wheel-common.yml
    ...

However, I really love the capability of accessing the workflow yaml variables inside a bash snippet, like

          if [[ "${{ inputs.host-platform }}" == linux* ]]; then
            # do this
          elif [[ "${{ inputs.host-platform }}" == win* ]]; then
            # do that
          fi

It is so very neat (ex: the variables are statically expanded). Sounds like we'll be losing this 🥲

[cryos]

We can just add these things to the env though, and then use them offering more standardization. It means that the bash scripts can be more reusable, for example reproducing pieces of CI locally in docker containers etc is more approachable. I don't think losing YAML variables is a huge thing, and in many workflows I tend to add them to the environment to avoid this.