VIP: Video Inpainting Pipeline for Real World Human Removal

Inpainting for real-world human and pedestrian removal in high-resolution video clips presents significant challenges, particularly in achieving high-quality outcomes, ensuring temporal consistency, and managing complex object interactions that involve humans, their belongings, and their shadows. In this paper, we introduce VIP (Video Inpainting Pipeline), a novel promptless video inpainting framework for real-world human removal applications. VIP enhances a state-of-the-art text-to-video model with a motion module and employs a Variational Autoencoder (VAE) for progressive denoising in the latent space. Additionally, we implement an efficient human-and-belongings segmentation for precise mask generation. Sufficient experimental results demonstrate that VIP achieves superior temporal consistency and visual fidelity across diverse real-world scenarios, surpassing state-of-the-art methods on challenging datasets. Our key contributions include the development of the VIP pipeline, a reference frame integration technique, and the Dual-Fusion Latent Segment Refinement method, all of which address the complexities of inpainting in long, high-resolution video sequences.

@article{sun2025_2504.03041,
  title={ VIP: Video Inpainting Pipeline for Real World Human Removal },
  author={ Huiming Sun and Yikang Li and Kangning Yang and Ruineng Li and Daitao Xing and Yangbo Xie and Lan Fu and Kaiyu Zhang and Ming Chen and Jiaming Ding and Jiang Geng and Jie Cai and Zibo Meng and Chiuman Ho },
  journal={arXiv preprint arXiv:2504.03041},
  year={ 2025 }
}