As perception models continue to develop, the need for large-scale datasets increases. However, data annotation remains far too expensive to effectively scale and meet the demand. Synthetic datasets provide a solution to boost model performance with substantially reduced costs. However, current synthetic datasets remain limited in their scope, realism, and are designed for specific tasks and applications. In this work, we present RealDriveSim, a realistic multi-modal synthetic dataset for autonomous driving that not only supports popular 2D computer vision applications but also their LiDAR counterparts, providing fine-grained annotations for up to 64 classes. We extensively evaluate our dataset for a wide range of applications and domains, demonstrating state-of-the-art results compared to existing synthetic benchmarks. The dataset is publicly available atthis https URL.
View on arXiv@article{jadon2025_2506.16319,
title={ RealDriveSim: A Realistic Multi-Modal Multi-Task Synthetic Dataset for Autonomous Driving },
author={ Arpit Jadon and Haoran Wang and Phillip Thomas and Michael Stanley and S. Nathaniel Cibik and Rachel Laurat and Omar Maher and Lukas Hoyer and Ozan Unal and Dengxin Dai },
journal={arXiv preprint arXiv:2506.16319},
year={ 2025 }
}