CNN based Patch Matching for Optical Flow with Thresholded Hinge Loss

Learning based approaches have not yet achieved their full potential in optical flow estimation, where their performance still trails heuristic approaches. In this paper, we present a novel optical flow pipeline that uses patch-matching with CNN trained features at multiple scales. We show a novel way for calculating CNN based features for different scales, which performs better than existing methods. Furthermore, we introduce a new thresholded loss for Siamese networks and demonstrate that our novel loss performs clearly better than existing losses. It also allows to speed up training by a factor of 2 in our tests. Moreover, we discuss new ways of evaluating the robustness of trained features for the application of patch matching for optical flow. An interesting discovery in our paper is that low pass filtering feature maps can increase the robustness of features created by CNNs. We prove competitive performance of our approach by submitting it to the KITTI 2012 and KITTI 2015 evaluation portals and obtaining the best results on KITTI 2012 and the best for foreground objects in KITTI 2015.