Does it work if I put the selekt method on lora tensor ?

[shiwanghua]

Direct abs(lora_tensor), sort it and call topk()

No needs for loading the base model and calculate the diff.

[shiwanghua]

No, the loss has soared after every selekt operation although alpha=0.999 (keep 99.9% of the parameters and halve the value of the remaining 0.1% of the parameters)

[shiwanghua]

Maybe we should only linearly selekt loraA rather than both of loraA and loraB.