v1v2 (latest)

Generative Omnimatte: Learning to Decompose Video into Layers

25 November 2024

Papers citing "Generative Omnimatte: Learning to Decompose Video into Layers"

50 / 56 papers shown

Title
PSDiffusion: Harmonized Multi-Layer Image Generation via Layout and Appearance Alignment Dingbang Huang Wenbo Li Yifei Zhao Xinyu Pan Yanhong Zeng Bo Dai DiffM 51 0 0 16 May 2025
OmnimatteZero: Fast Training-free Omnimatte with Pre-trained Video Diffusion Models Dvir Samuel Matan Levy N. Darshan Gal Chechik Rami Ben-Ari DiffM 99 0 0 23 Mar 2025
Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Model Yingying Fan Quanwei Yang Kaisiyuan Wang Hang Zhou Yingying Li Haocheng Feng Errui Ding Y. Wu Jiadong Wang DiffM 85 3 0 21 Mar 2025
Generative AI for Cel-Animation: A Survey Yunlong Tang Junjia Guo Pinxin Liu Zhiyuan Wang Hang Hua ... Jing Bi Mingqian Feng Xuzhao Li Zeliang Zhang Chenliang Xu VGen 133 7 0 08 Jan 2025
VidPanos: Generative Panoramic Videos from Casual Panning Videos Jingwei Ma Erika Lu Roni Paiss Shiran Zada Aleksander Holynski Tali Dekel Brian L. Curless Michael Rubinstein Forrester Cole VGen 55 3 0 17 Oct 2024
CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer Zhuoyi Yang Jiayan Teng Wendi Zheng Ming Ding Shiyu Huang ... Weihan Wang Yean Cheng Xiaotao Gu Yuxiao Dong Jie Tang DiffM VGen 226 512 0 12 Aug 2024
SAM 2: Segment Anything in Images and Videos Nikhila Ravi Valentin Gabeur Yuan-Ting Hu Ronghang Hu Chaitanya K. Ryali ... Nicolas Carion Chao-Yuan Wu Ross B. Girshick Piotr Dollár Christoph Feichtenhofer VLM MLLM 143 876 0 01 Aug 2024
Matting by Generation Zhixiang Wang Baiang Li Jian Wang Yu-Lun Liu Jinwei Gu Yung-Yu Chuang Shiníchi Satoh DiffM 76 1 0 30 Jul 2024
Learning Temporally Consistent Video Depth from Video Diffusion Priors Jiahao Shao Yuanbo Yang Hongyu Zhou Youmin Zhang Yujun Shen Vitor Campagnolo Guizilini Yue Wang Matteo Poggi Yiyi Liao VGen DiffM MDE 83 41 0 03 Jun 2024
ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertion Daniel Winter Matan Cohen Shlomi Fruchter Yael Pritch Alex Rav-Acha Yedid Hoshen DiffM 77 32 0 27 Mar 2024
Magic Fixup: Streamlining Photo Editing by Watching Dynamic Videos Hadi Alzayer Zhihao Xia Xuaner Zhang Eli Shechtman Jia-Bin Huang Michael Gharbi DiffM VGen 53 20 0 19 Mar 2024
CoCoCo: Improving Text-Guided Video Inpainting for Better Consistency, Controllability and Compatibility Bojia Zi Shihao Zhao Xianbiao Qi Jianan Wang Yukai Shi Qianyu Chen Bin Liang Kam-Fai Wong Lei Zhang DiffM VGen 74 22 0 18 Mar 2024
Transparent Image Layer Diffusion using Latent Transparency Lvmin Zhang Maneesh Agrawala 83 50 0 27 Feb 2024
Lumiere: A Space-Time Diffusion Model for Video Generation Omer Bar-Tal Hila Chefer Omer Tov Charles Herrmann Roni Paiss ... T. Michaeli Oliver Wang Deqing Sun Tali Dekel Inbar Mosseri VGen 190 242 0 23 Jan 2024
Towards Language-Driven Video Inpainting via Multimodal Large Language Models Jianzong Wu Xiangtai Li Chenyang Si Shangchen Zhou Jingkang Yang ... Yining Li Kai Chen Yunhai Tong Ziwei Liu Chen Change Loy VGen DiffM MLLM 97 17 0 18 Jan 2024
VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models Haoxin Chen Yong Zhang Xiaodong Cun Menghan Xia Xintao Wang Chao-Liang Weng Ying Shan VGen DiffM 209 308 0 17 Jan 2024
AVID: Any-Length Video Inpainting with Diffusion Model Zhixing Zhang Bichen Wu Xiaoyan Wang Yaqiao Luo Luxin Zhang Yinan Zhao Peter Vajda Dimitris N. Metaxas Licheng Yu VGen DiffM 79 41 0 06 Dec 2023
Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets A. Blattmann Tim Dockhorn Sumith Kulal Daniel Mendelevitch Maciej Kilian ... Zion English Vikram S. Voleti Adam Letts Varun Jampani Robin Rombach VGen 263 1,170 0 25 Nov 2023
Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioning Rohit Girdhar Mannat Singh Andrew Brown Quentin Duval S. Azadi Sai Saketh Rambhatla Akbar Shah Xi Yin Devi Parikh Ishan Misra DiffM VGen 103 206 0 17 Nov 2023
DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors Jinbo Xing Menghan Xia Yong Zhang Haoxin Chen Wangbo Yu Hanyuan Liu Xintao Wang Tien-Tsin Wong Ying Shan VGen 103 246 0 18 Oct 2023
Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation David Junhao Zhang Jay Zhangjie Wu Jia-Wei Liu Rui Zhao L. Ran Yuchao Gu Difei Gao Mike Zheng Shou DiffM VGen 86 220 0 27 Sep 2023
OmnimatteRF: Robust Omnimatte with 3D Background Modeling Geng Lin Chen Gao Jia-Bin Huang Changil Kim Yipeng Wang Matthias Zwicker Ayush Saraf 56 7 0 14 Sep 2023
ProPainter: Improving Propagation and Transformer for Video Inpainting Shangchen Zhou Chongyi Li Kelvin C. K. Chan Chen Change Loy ViT 90 104 0 07 Sep 2023
Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models Songwei Ge Seungjun Nah Guilin Liu Tyler Poon Andrew Tao Bryan Catanzaro David Jacobs Jia-Bin Huang Ming-Yuan Liu Yogesh Balaji DiffM VGen 95 259 0 17 May 2023
Learning Physical-Spatio-Temporal Features for Video Shadow Removal Zhihao Chen Liang Wan Yefan Xiao Lei Zhu Huazhu Fu 68 7 0 16 Mar 2023
MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation Omer Bar-Tal Lior Yariv Y. Lipman Tali Dekel 78 383 1 16 Feb 2023
Shape-aware Text-driven Layered Video Editing Yao-Chih Lee Ji-Ze Jang Yi-Ting Chen Elizabeth Qiu Jia-Bin Huang VGen DiffM 61 54 0 30 Jan 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 426 4,563 0 30 Jan 2023
FactorMatte: Redefining Video Matting for Re-Composition Tasks Zeqi Gu Wenqi Xian Noah Snavely Abe Davis 69 12 0 03 Nov 2022
Imagen Video: High Definition Video Generation with Diffusion Models Jonathan Ho William Chan Chitwan Saharia Jay Whang Ruiqi Gao ... Diederik P. Kingma Ben Poole Mohammad Norouzi David J. Fleet Tim Salimans VGen 162 1,528 0 05 Oct 2022
Flow-Guided Transformer for Video Inpainting Kaiwen Zhang Jingjing Fu Dong Liu ViT 66 72 0 14 Aug 2022
D $^2$ NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Video Tianhao Wu Fangcheng Zhong Andrea Tagliasacchi Forrester Cole Cengiz Öztireli 60 143 0 31 May 2022
Deformable Sprites for Unsupervised Video Decomposition Vickie Ye Zhengqi Li Richard Tucker Angjoo Kanazawa Noah Snavely OCL 63 67 0 14 Apr 2022
Towards An End-to-End Framework for Flow-Guided Video Inpainting Zerui Li Cheng Lu Jia Qin Chunle Guo Mingg-Ming Cheng 91 153 0 06 Apr 2022
Text2LIVE: Text-Driven Layered Image and Video Editing Omer Bar-Tal Dolev Ofri-Amar Rafail Fridman Yoni Kasten Tali Dekel VGen DiffM 89 314 0 05 Apr 2022
Kubric: A scalable dataset generator Klaus Greff Francois Belletti Lucas Beyer Carl Doersch Yilun Du ... Ziyu Wang Tianhao Wu K. M. Yi Fangcheng Zhong Andrea Tagliasacchi 102 266 0 07 Mar 2022
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 460 15,665 0 20 Dec 2021
Layered Neural Atlases for Consistent Video Editing Yoni Kasten Dolev Ofri-Amar Oliver Wang Tali Dekel VGen 243 163 0 23 Sep 2021
FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting R. Liu Hanming Deng Yangyi Huang Xiaoyu Shi Lewei Lu Wenxiu Sun Xiaogang Wang Jifeng Dai Hongsheng Li ViT 73 128 0 07 Sep 2021
SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations Chenlin Meng Yutong He Yang Song Jiaming Song Jiajun Wu Jun-Yan Zhu Stefano Ermon DiffM 147 1,492 0 02 Aug 2021
Omnimatte: Associating Objects and Their Effects in Video Erika Lu Forrester Cole Tali Dekel Andrew Zisserman William T. Freeman Michael Rubinstein DiffM VOS 95 54 0 14 May 2021
MarioNette: Self-Supervised Sprite Learning Dmitriy Smirnov Michael Gharbi Matthew Fisher Vitor Campagnolo Guizilini Alexei A. Efros Justin Solomon SSL OCL 119 37 0 29 Apr 2021
Privacy-Preserving Portrait Matting Jizhizi Li Sihan Ma Jing Zhang Dacheng Tao PICV 77 61 0 29 Apr 2021
Real-Time High-Resolution Background Matting Shanchuan Lin Andrey Ryabtsev Soumyadip Sengupta Brian L. Curless S. M. Seitz Ira Kemelmacher-Shlizerman 3DH 101 222 0 14 Dec 2020
MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition Zhanghan Ke Jiayu Sun Kaican Li Qiong Yan Rynson W. H. Lau 58 161 0 24 Nov 2020
Layered Neural Rendering for Retiming People in Video Erika Lu Forrester Cole Tali Dekel Weidi Xie Andrew Zisserman D. Salesin William T. Freeman Michael Rubinstein 3DH 34 71 0 16 Sep 2020
Flow-edge Guided Video Completion Chen Gao Ayush Saraf Jia-Bin Huang Johannes Kopf 62 166 0 03 Sep 2020
Denoising Diffusion Probabilistic Models Jonathan Ho Ajay Jain Pieter Abbeel DiffM 650 18,096 0 19 Jun 2020
Learning to See Through Obstructions Yu-Lun Liu Wei-Sheng Lai Ming-Hsuan Yang Yung-Yu Chuang Jia-Bin Huang 37 60 0 02 Apr 2020
Controllable Attention for Structured Layered Video Decomposition Jean-Baptiste Alayrac João Carreira Relja Arandjelović Andrew Zisserman 38 10 0 24 Oct 2019