Incompatibility between cuEquivariance and torch<=2.5.0

[MicPellegrino]

Hi,

According to the documentation in here: https://mace-docs.readthedocs.io/en/latest/guide/lammps_mliap.htm, one should install cuEquivariance and torch<=2.5.0 in order for the ML-IAP model generation to work.

However, new versions of cuEquivariance are incompatible with torch<=2.5.0:

cuequivariance-ops-cu12 0.7.0 requires nvidia-cublas-cu12>=12.5.0, but you have nvidia-cublas-cu12 12.4.5.8 which is incompatible

I am trying with a downgraded version:

cuequivariance==0.6.1
cuequivariance-torch==0.6.1
cuequivariance-ops-torch-cu12==0.6.0

and it seems to work.

Is there a way to use a newer version of cuEquivariance?

In general, I think it would be great if you could specify in the documentation the specific versions of the dependencies, or at least the ones that are compatible with the torch 2.5.0.

P.s. e3nn and should probably be added in the dependency list, in my experience it never installs automagically...

[bernstei]

Pretty sure you can use newer versions of torch, it'll just give warnings about loading parameters

[bernstei]

With kokkos (lmp -k on g 1 -sf kk -pk kokkos newton on neigh half -in lammps.in) it's fine, without the kokkos flags I get the forward_exchange error.

[MicPellegrino]

BTW, going back to my old post #945, I initially tough that the runtime error I was facing was due to not being the correct LAMMPS branch, but now even if I am on the develop branch, I get the same error with torch 2.5.0 and cuequivariance 0.4:

Traceback (most recent call last):
  File "mliap_unified_couple_kokkos.pyx", line 418, in mliap_unified_couple_kokkos.compute_forces_python_kokkos
  File "mliap_unified_couple_kokkos.pyx", line 400, in mliap_unified_couple_kokkos.MLIAPUnifiedInterfaceKokkos.compute_forces
  File "/home/michele/mambaforge/envs/benchmark-mace/lib/python3.12/site-packages/mace/calculators/lammps_mliap_mace.py", line 160, in compute_forces
    _, atom_energies, pair_forces = self.model(batch)
                                    ^^^^^^^^^^^^^^^^^
  File "/home/michele/mambaforge/envs/benchmark-mace/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/michele/mambaforge/envs/benchmark-mace/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/michele/mambaforge/envs/benchmark-mace/lib/python3.12/site-packages/mace/calculators/lammps_mliap_mace.py", line 84, in forward
    out = self.model(
          ^^^^^^^^^^^
  File "/home/michele/mambaforge/envs/benchmark-mace/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/michele/mambaforge/envs/benchmark-mace/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/michele/mambaforge/envs/benchmark-mace/lib/python3.12/site-packages/mace/modules/models.py", line 418, in forward
    node_feats, sc = interaction(
                     ^^^^^^^^^^^^
  File "/home/michele/mambaforge/envs/benchmark-mace/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/michele/mambaforge/envs/benchmark-mace/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/michele/mambaforge/envs/benchmark-mace/lib/python3.12/site-packages/mace/modules/blocks.py", line 545, in forward
    node_feats = self.handle_lammps(
                 ^^^^^^^^^^^^^^^^^^^
  File "/home/michele/mambaforge/envs/benchmark-mace/lib/python3.12/site-packages/mace/modules/blocks.py", line 355, in handle_lammps
    node_feats = LAMMPS_MP.apply(node_feats, lammps_class)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/michele/mambaforge/envs/benchmark-mace/lib/python3.12/site-packages/torch/autograd/function.py", line 575, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/michele/mambaforge/envs/benchmark-mace/lib/python3.12/site-packages/mace/tools/utils.py", line 158, in forward
    data.forward_exchange(feats, out, ctx.vec_len)
    ^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'mliap_unified_couple_kokkos.MLIAPDataPy' object has no attribute 'forward_exchange'
ERROR: Running mliappy unified compute_forces failure. (src/KOKKOS/mliap_unified_kokkos.cpp:82)

So either something has been rolled-back on the LAMMPS side (I made sure to be up to date with their current develop branch), or I was just extremely lucky (apparently is a matter of luck nowadays).

I am going to give one last try with the most recent release (which should support the ML-IAP interface).

[MicPellegrino]

Works fine for me with cuequivariance 0.7.0 and pytorch 2.8. I just needed to run with kokkos, and not have any typos in my input structure.

Of course I am trying this combination of torch and cueq, too. Can you please confirm that you are using MACE 0.3.14 to create the model and you are building LAMMPS from their develop branch?

[bernstei]

I'm using MACE develop branch as of today, and LAMMPS develop branch from Sep (commit 0d479cae29).

You keep on not commenting on kokkos and GPU usage. Are you compiling LAMMPS with kokkos, running with a GPU, and passing the kokkos flags to the lammps executable? The forward_exchange error is specifically a kokkos vs. non-kokkos mliap implementation issue, I believe, and maybe more specifically kokkos-CUDA.

[MicPellegrino]

@bernstei I keep "not commenting" about Kokkos simpy because I didnt even know one could use the MACE via the ML-IAP interface without Kokkos. I have always run with Kokkos.
The fact that the interface is usuable, but doesn't work, without Kokkos is not really reassuring, tough.

[MicPellegrino]

OK! cuequivariance 0.7.0 and pytorch 2.8 works with LAMMPS latest release (22Jul2025)!
It does not work with LAMMPS develop branch.

[bernstei]

So are you saying that with the latest lammps develop, with kokkos enabled (including the flags), you get the forward_exchange error, but you don't get it with the 22Jul2025 version? Can you try the commit I mention above, which is (I believe) the develop branch as of Sep 12th 2025

[MicPellegrino]

Hi, I reset to 0d479cae29, and it works. So it works with Kokkos (which I've always been using, with all the flags as reported on the MACE documentation page for ML-IAP) both on the 22Jul2025 release and on the develop branch, but possibly not the latest commit (I am pulled everything from develop before doing my tests yesterday).
I am going to stick with release + torch2.8 + cueq0.7 then, as it seems the most "stable" setup.

[ilyes319]

i see thank you @MicPellegrino !

[bernstei]

Once I get my GPU nodes back, I can try to reproduce the issue with the latest LAMMPS develop version, and perhaps investigate what LAMMPS commit broke it.

[bernstei]

I cannot reproduce the problem with the latest lammps develop branch. I just recompiled it from lammps commit ac5940, develop branch as of Nov 24 2025, and it ran fine.

I did have to make a small compatibility patch to symmetrix, wcwitt/symmetrix#55, which is presumably irrelevant to this issue