FG-DFPN: Flow Guided Deformable Frame Prediction Network

Video frame prediction remains a fundamental challenge in computer vision with direct implications for autonomous systems, video compression, and media synthesis. We present FG-DFPN, a novel architecture that harnesses the synergy between optical flow estimation and deformable convolutions to model complex spatio-temporal dynamics. By guiding deformable sampling with motion cues, our approach addresses the limitations of fixed-kernel networks when handling diverse motion patterns. The multi-scale design enables FG-DFPN to simultaneously capture global scene transformations and local object movements with remarkable precision. Our experiments demonstrate that FG-DFPN achieves state-of-the-art performance on eight diverse MPEG test sequences, outperforming existing methods by 1dB PSNR while maintaining competitive inference speeds. The integration of motion cues with adaptive geometric transformations makes FG-DFPN a promising solution for next-generation video processing systems that require high-fidelity temporal predictions. The model and instructions to reproduce our results will be released at:this https URLGroup/frame-prediction

@article{yılmaz2025_2503.11343,
  title={ FG-DFPN: Flow Guided Deformable Frame Prediction Network },
  author={ M. Akın Yılmaz and Ahmet Bilican and A. Murat Tekalp },
  journal={arXiv preprint arXiv:2503.11343},
  year={ 2025 }
}