I have successed in running the code with 4 a800 80g

[xiangliustc]

--num_generations 4 \
--per_device_train_batch_size 8 \

and it takes 48hours ,I will report if this change will down the performance

[xiangliustc]

--num_generations 4
--per_device_train_batch_size 8 \means not 8 pictures but 2 pictures ,while means it may decrease to dpo ,but with 4 a800 80g, it is my only way to run the code, i think this may hurt performence badly, i will continue focusing the performance

[TianheWu]

Thanks. Maybe there are some settings in your code are different. 4 A800 80G can run 8 images with num_generation 6 in my settings.