Comparison with other Engines (like Simbody, PhysX, etc)

[acxz]

I'd like to preface this by saying that this benchmark software is an amazing tool for the simulation community. As the robotics community is growing and many different tools are popping up, having an understanding of what is the state of the art is very relevant. This tool can also help development of engines, as mentioned in previous comments. For an end user, options may be overwhelming and this helps in realizing which engine is appropriate for their own tasks. There have been comparisons of engines previously, however, this is the first time (to my very limited knowledge) that a benchmark tool has been publicly available. This benchmark tool makes me very happy.

Having said that, are there plans for expanding the number of engines compared? It would be nice to compare Simbody, Drake, and the now open-source PhysX. I am sure there are others that can be compared as well.

[jhwangbo]

Thank you for positive feedback. We chose a set of engines for robotics applications, especially for DRL research. We suspect that the engines you mentioned would not be great for this purpose. In addition, it is quite some effort to get it right. So I expect other engine developers to contribute to this repo as well. A few reasons why the engines you mentioned are not being selected for the tests are:

Simbody, as far as I know, is not being actively developed. I heard that the developer is now working on Drake. In addition, from our experience from one of the RL locomotion challenges, Simbody was not suitable for DRL research.

Drake is, from my experience, is also not competitive in terms of simulation speed. It is designed for very different purposes.

When we started this project, physX4 was not available. It didn't make much sense to use older versions. Now, physX4 seems very decent and can utilize GPU's.

[acxz]

Thank you for your comments, it makes sense to focus on a particular set of engines given the constraints you mentioned.

I did not know about the above mentioned statements about Simbody and Drake, but in the end I would trust a benchmark test over personal experiences. But as you mentioned, it does take effort. I think I'll try asking developers of both of these projects to take a look at this and try to contribute. It would be wonderful if they agree.

I think benchmarking PhysX4 would be very beneficial, especially since it utilizes GPUs. Once this semester ends, I may get around to playing with it and trying to benchmark it using this repo.

I am looking forward to seeing this expand!

[acxz]

Following up with this here are links to relevant issues on other physics engine repos.

Simbody: simbody/simbody#665
Drake: RobotLocomotion/drake#11100
PhysX: NVIDIAGameWorks/PhysX#91
CHRONO: projectchrono/chrono#147

[duburcqa]

Following up on this discussion, I would like to introduce you with Jiminy. I'm the author of Jiminy, which is aiming to do basically the same as Bullet and Dart. It is a Physics Engine tailored for Robotics and Machine Learning. In particular, ANYmal and Atlas environments are provided. They are ready to use just doing:

import gym
env = gym.make('gym_jiminy:jiminy-anymal-v0')

It is as simple as that ! Here is a rendering using gepetto-gui, but meshcat backend is also available.

Hope it piques your curiosity 😄
Cheers!

[EricVoll]

Just to start the conversation again: PhysX just became much more interesting a few months ago with the release of isaacgym and RSL's corresponding RL framework legged_gym. Would be great to have issacgym compared to the other alternatives.

It is probably clear that IsaacGym with the PhysX backend will be quicker, but it might be interesting to compare precision etc.

[jhwangbo]

From our tests, we found that raisim is still faster on the ANYmal B locomotion task. But we used 3950x and rtx3070. It highly depends on what hardware you use. The number of environments and the simulation setup were tuned for both simulators. So it is hard to tell which one wins in the end. If you have a super-powerful GPU, you can go with IssacGym. My guess is that IssacGym uses single floating operations and it might have issues with accuracies.