ENFT: Efficient Non-Consecutive Feature Tracking for Robust Structure-from-Motion

Structure-from-motion (SfM) largely relies on the quality of feature tracking. In image sequences, if disjointed tracks caused by objects moving in and out of the view, occasional occlusion, or image noise, are not handled well, the corresponding SfM could be significantly affected. This problem becomes more serious for accurate SfM of large-scale scenes, which typically requires to capture multiple sequences to cover the whole scene. In this paper, we propose an efficient non-consecutive feature tracking (ENFT) framework to match the interrupted tracks distributed in different subsequences or even in different videos. Our framework consists of steps of solving the feature `dropout' problem when indistinctive structures, noise or even large image distortion exist, and of rapidly recognizing and joining common features located in different subsequences. In addition, we contribute an effective segment-based coarse-to-fine SfM estimation algorithm for efficiently and robustly handling large datasets. Experimental results on several challenging and large video datasets demonstrate the effectiveness of the proposed system.