What is Sim-to-real gap?

The sim-to-real gap is the difference between a policy’s performance in the simulator it was trained in and in the real environment. It is largest for policies that exploit what the simulator gets wrong, which is what an optimiser does with any error that pays.