This repository was archived by the owner on Oct 31, 2023. It is now read-only.
This repository was archived by the owner on Oct 31, 2023. It is now read-only.
FP16/AMP training #15
Description
Hello, I have noticed that all scripts use full precision calculations. Are there reasons for that? Does it make substantial difference in terms of accuracy on downstream tasks? Were there some ablation studies regarding that?
I am wondering because it's easier to fit model with fp16 for university project.
Sorry if this question was answered elsewhere.