RV-FuseNet: Range View Based Fusion of Time-Series LiDAR Data for Joint 3D Object Detection and Motion Forecasting

Robust real-time detection and motion forecasting of traffic participants are necessary for autonomous vehicles to safely navigate urban environments. We present RV-FuseNet, a novel end-to-end approach for joint detection and trajectory estimation using raw time-series LiDAR data. Instead of the widely used bird's eye view (BEV) representation, we utilize the native range view (RV) representation of LiDAR data. RV preserves the full resolution of the raw sensor data by avoiding the voxelization used in BEV. Furthermore, RV can be processed efficiently due to its compactness. However, for time-series fusion, the data is projected to a common viewpoint, and often this viewpoint is different from where it was captured. This can lead to loss of data and structure in RV which has an adverse impact on performance. To address this challenge we propose a novel architecture that sequentially projects each RV sweep into the viewpoint of the next sweep in time. We demonstrate that our sequential fusion approach is superior to directly projecting all the data into the most recent viewpoint. Furthermore, we show that our approach significantly improves motion forecasting accuracy over the existing state-of-the-art.