General Suggestions

• The environments heavily use JIT compilation for speed ups. But it might return NaN gradients in some rare situations. The training will explicitly fail in such conditions. In these situations use PYTORCH_JIT=0.

• A minor bottleneck might be horovod caching. Disable caching with HOROVOD_CACHE_CAPACITY=0.

• By default we simulate the environment on CPU, this is performant for low nagents due to the high kernel launch overhead. In case you want to use our tsim and agents modules for simulating a large number of vehicles, uncomment this line or the corresponding line in other trainers.