SingleWorker and horovod

[vlimant]

the combination --np N --n-processes N does not function while --np 1 --n-processes 1 would

[abidmalikwaterloo]

@vlimant Does it mean that it only works with a ring having one GPU and one communicator?

[vlimant]

nope. --np 1 --n-processes 1 is making only one process that uses the MPISingleWorker, --np N --n-processes N should actually do something similar, but put the N process in a single ring under N MPISingleWorker ...