Set up daily CI jobs to validate XGBoost against latest CCCL and RMM

[hcho3]

Currently, XGBoost is not being tested against latest CCCL and RMM. Breaking changes are not found until much later in the development cycle. For example: conda-forge/xgboost-feedstock#259 was blocked because XGBoost was found to be incompatible with CCCL 3.1. The particular incompatibility was due to an upstream bug (NVIDIA/cccl#6419). Had we found about this bug earlier, we could have addressed it sooner in the development cycle.

Proposal: Set a nightly CI job on the dmlc/xgboost repo to validate XGBoost against latest CCCL and RMM.

The CI job should be nightly so that it doesn't block pull requests.

For this effort to be successful, I or someone else should regularly monitor the status of the nightly CI.

Discussion: Which version of CCCL and RMM should we test with XGBoost?

• Latest stable?
• Latest nightly version?

Sometimes, nightly versions of RMM and CCCL may not be compatible with each other. Given their relative position in the software stack, I propose:

• Use latest stable version of CCCL
• Use nightly version of RMM

[bdice]

I would like to ensure we test rapidsai/xgboost-feedstock in particular. It is not always in sync with upstream, and that is where we ultimately make releases from for RAPIDS.

[jakirkham]

Think CCCL also has a job that tests RAPIDS. Maybe it could make sense to include XGBoost in that job. Though this might be in addition to as opposed to substitute for an XGBoost nightly job

cc-ing a few folks who may be interested in this: @bdice @trxcllnt @jrhemstad

[jakirkham]

I would like to ensure we test rapidsai/xgboost-feedstock in particular. It is not always in sync with upstream, and that is where we ultimately make releases from for RAPIDS.

Think Hyunsu's point about testing upstream XGBoost with RMM + CCCL covers this

We are not really doing anything to different from upstream in either of the Conda feedstocks

[hcho3]

Update: We now have a working nightly CI pipeline to validate XGBoost against latest CCCL and RMM: https://github.com/dmlc/xgboost/actions/workflows/cccl_nightly.yml