Fine-tuning MH-1

[matr1x-1]

I am trying to fine-tune a MH-1 foundation model with the omat_pbe head on the GAP-Carbon-20 dataset. In order to extract the necessary E0 value, I started a fresh MACE model and used the value there. The problem is that when I run the fine-tuning using replay the inital errors are very high - far above the 0.4 eV/atom mentioned in the documentation as a warning signal that E0 might be wrong.
This is not the only case where this has happened. I have other datasets that I created myself with CP2K and where I calculated E0 directly using DFT. The problem was exactly the same.
This is the way I call the training:
 

srun --cpu-bin=socket ./mps-wrapper.sh python -m mace.cli.run_train   \
        --name="mymodel_finetuned"   \
        --train_file="carbon_MACE_READY.xyz" \
        --valid_fraction=0.05 \
        --foundation_model "mh-1" \
        --downloads_dir "../mace_downloads" \
        --foundation_head="omat_pbe"   \
        --pt_train_file="replay-small-2000_train.xyz" \
        --pt_valid_file="replay-small-2000_valid.xyz" \
        --weight_pt_head=5.0 \
        --multiheads_finetuning=True   \
        --device=cuda     \
        --atomic_numbers="[6]" \
        --energy_key="REF_energy" \
        --batch_size=1 \
        --E0s "{6: -6.554891353818097}" \
        --num_workers=1 \
        --default_dtype="float64" \
        --forces_key="REF_forces" \
        --lr 1e-4 \
        --stress_weight=0.0 \
        --loss="ef" \
        --compute_stress=False


Is there anything that I might have missed? Thank you!