Fix for ASE MACE calculator failures with TorchScript on ROCm #1044
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi all,
Not sure if others have run into this issue on Archer2 or elsewhere, but in case this fix is useful:
Context
My collaborator's ASE MD runs with MACE
0.3.13/0.3.14failed when using a ROCm PyTorch build on Archer2. TorchScript was enforcing strict schema checks, silently ignoring unknown kwargs and omitting optional outputs, causing runtime errors and deadlocks.This PR
Improves robustness of the ASE MACE calculator to handle these scenarios:
Dynamic kwarg gating
Inspect
model.forwardat runtime and passcompute_edge_forces/compute_atomic_stressesonly if supported, eliminating unknown‑kwarg errors on TorchScripted models on ROCm builds.Safe output access
Replace
out["..."]without.get("...")plus null checks foratomic_stressesandatomic_virials, preventingKeyErrors or hangs when keys are absent.Empty‑list stacking guard
Before aggregating per‑model tensors with
torch.stack(), verify that the corresponding list is non‑empty, avoiding deadlocks.All changes are in
mace/calculators/mace.py.Tested on CUDA, ROCm, and CPUs on the following machines:
The overheads should be (and appear to be) negligible.