PyTorchRL: Modular and Distributed Reinforcement Learning in PyTorch

Deep reinforcement learning (RL) has proved successful at solving challenging environments but often requires scaling to large sampling and computing resources. Furthermore, advancing RL requires tools that are flexible enough to easily prototype new methods, yet avoiding impractically slow experimental turnaround times. To this end, we present PyTorchRL, a PyTorch-based library for RL with a modular design that allows composing agents from a set of reusable and easily extendable modules. Additionally, PyTorchRL permits the definition of distributed training architectures with flexibility and independence of the Agent components. In combination, these two features can accelerate the pace at which ideas are implemented and tested, simplifying research and enabling to tackle more challenging RL problems. We present several interesting use-cases of PyTorchRL and showcase the library by obtaining the highest to-date test performance on the Obstacle Tower Unity3D challenge environment.