[WIP] Make the nightly job use TheRock instead of ROCm

[charleshofer]

Motivation

TheRock is the future of ROCm. We're going to start getting our ROCm installations from TheRock instead of the old ROCm release pipeline. Eventually, we want to be in a place where we can install ROCm through the Python wheels in TheRock, but for now, our build scripts and plugin wheels only work with TheRock's tarballs unzipped into /opt/rocm. This changes the nightly workflow to pull a nightly tarball from TheRock instead of using the old ROCm release and adds TheRock builds to the base docker image build workflow.

Technical Details

Adds build options for therock-path in our GHA workflows. Much like how we used to pin the nightly build to specific build numbers of ROCm, we "pin" to a specific nightly build of TheRock by passing in a URL.

Test Plan

• Run the base docker image workflow
• Run the nightly workflow

TODO

• Typing in a full URL to an artifact of TheRock is pretty unwieldy. We could split this into two parameters; one for the TheRock's build number (which always corresponds to the day it was built), and one for the base URL of where to get TheRock's tarball. There are several directories with different types of builds for TheRock.
• Figure out if there's builds of TheRock that target more than just one specific GFX. We want to build for many relevant GFXs, not just, say, 940.
• Use TheRock's Python wheels instead of the tarball: https://github.com/ROCm/TheRock/blob/main/RELEASES.md#python-packages-release-status

[alekstheod]

Not sure if you are interested but we could do a hermetic build with therock. Check openxla rocm_ci_hermetic config for more details.

[alekstheod]

Can we also create manylinux images so we can reuse them on PR checks?

[charleshofer]

Can we also create manylinux images so we can reuse them on PR checks?

Yes. I need to rebase this now that I merged the manylinux PR. I'll add those when I rebase.