About finetuning on mace-mp-0b3-medium.model

[Asif-Iqbal-Bhatti]

Hello,

I am fine-tuning on mace-mp-0b3-medium.model, as this one is a good choice by your current review paper on this potential. As I have been working on fine-tuning on grain boundary data, it appears that it struggles to converge, and the path obtained from DFT is not the same as with this MLIP or FT model. I need some advice on what to do. To fix this, what I did in my fine-tuned data was obtain a few dozen grain boundaries and relax them with VASP and get the intermediate ionic steps and the initial and the final so the PES is described. In this way I have 500 configurations, and then I shuffle them and fine-tune them. I also included the AIMD for a few of the grain boundaries and incorporated the NEB path from DFT despite the fact that mace-mp-0b3-medium.model-FT struggled to obtain the path similar to DFT (I know it can never be the same but should produce the same trend). Could you provide guidance on the process and duration of fine-tuning? I am using the REPLAY FT Head. How many epochs should one go for? When using a large number of epochs, the pt_head starts to overfit. I am fine-tuning on EFS. Do you think stress might be causing this problem?
'''
--name MACE_from_scratch
--model="MACE"
--multiheads_finetuning False
--foundation_model ../mace-mp-0b3-medium.model
--hidden_irreps='128x0e + 128x1o'
--r_max 6
--E0s='{17:-0.25125556, 3:-0.29658376, 15:-1.884791, 16:-0.88440305}'
--atomic_numbers "[3, 15, 16, 17]"
--eval_interval=1
--device cuda
--train_file train_data.extxyz
--valid_file valid_data.extxyz
--test_file test_data.extxyz
--compute_stress True
--energy_weight 1.0
--forces_weight 100.0
--stress_weight 1.0
--swa_energy_weight 100.0
--swa_forces_weight 10.0
--swa_stress_weight 10.0
--num_workers 1
--loss universal
--stage_two
--max_num_epochs 350
--batch_size 1
--plot_frequency 10
--ema
--ema_decay 0.99
--seed 123
--default_dtype float64
--scheduler_patience 5
'''

[gabor1]

It's better to not fit on close to equilibrium data, and there are better models now than mp0b3. so I think you'd be better start with MPA0 or MATPES-PBE for example. And I would relax the grain boundaries with the model NOT WITH VASP, and then rattle the atoms a bit, then use that as training data. You can then do the NEB with the FT model, take some images from the path, do single point DFT on them, put them in the training set (maybe the highest energy image is enough), you shouldn't need many of these.

[Asif-Iqbal-Bhatti]

Okay, i will follow and will get back to you with this strategy.

[gabor1]

if you send us scatter plots of energy forces and stress components for your FT set before and after FT, it might help.

[Asif-Iqbal-Bhatti]

Thank you for your prompt reply. Here is the result for Naive Fine-tuning for now.

As I explained in this, I have included the relax data in my training file.

[Asif-Iqbal-Bhatti]

I also want to share that when I fine-tune the mp0b3 mode on single-point energy data of GB, the parity plot on test data is fine but the forces range is high

When I use this fine-tuned model, the NEB path is wrong like this, compared with dft

This is witht he model

This is my concern; that is why I thought to ask this question.

[ilyes319]

please do try to finetune the most recent foundation models including the omat or mh-1 models. if you want to finetune the mh-1 models, you need to use the latest main branch.

[Asif-Iqbal-Bhatti]

Okay thanks I will try this and will get back to you soon.