v1v2v3 (latest)

Speech Driven Video Editing via an Audio-Conditioned Diffusion Model

10 January 2023

Papers citing "Speech Driven Video Editing via an Audio-Conditioned Diffusion Model"

50 / 57 papers shown

Title
DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework for Talking Head Video Generation Hanbo Cheng Limin Lin Chenyu Liu Pengcheng Xia Pengfei Hu Jiefeng Ma Jun Du Jia Pan DiffM VGen 413 0 0 17 Oct 2024
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model Mingyuan Zhang Zhongang Cai Liang Pan Fangzhou Hong Xinying Guo Lei Yang Ziwei Liu DiffM VGen 103 571 0 31 Aug 2022
Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis Wanshu Fan Yen-Chun Chen Dongdong Chen Yu Cheng Lu Yuan Yu-Chiang Frank Wang DiffM 76 95 0 29 Aug 2022
Diffsound: Discrete Diffusion Model for Text-to-sound Generation Dongchao Yang Jianwei Yu Helin Wang Wen Wang Chao Weng Yuexian Zou Dong Yu DiffM 90 304 0 20 Jul 2022
ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech Rongjie Huang Zhou Zhao Huadai Liu Jinglin Liu Chenye Cui Yi Ren DiffM 87 198 0 13 Jul 2022
Zero-Shot Voice Conditioning for Denoising Diffusion TTS Models Alon Levkovitch Eliya Nachmani Lior Wolf DiffM 63 29 0 05 Jun 2022
Guided-TTS 2: A Diffusion Model for High-quality Adaptive Text-to-Speech with Untranscribed Data Sungwon Kim Heeseung Kim Sung-Hoon Yoon DiffM 219 53 0 30 May 2022
EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model Xinya Ji Hang Zhou Kaisiyuan Wang Qianyi Wu Wayne Wu Feng Xu Xun Cao CVBM 94 164 0 30 May 2022
Flexible Diffusion Modeling of Long Videos William Harvey Saeid Naderiparizi Vaden Masrani Christian D. Weilbach Frank Wood DiffM BDL VGen 222 295 0 23 May 2022
Talking Head Generation Driven by Speech-Related Facial Action Units and Audio- Based on Multimodal Representation Fusion Sen Chen Zhilei Liu Jiaxing Liu Longbiao Wang 64 6 0 27 Apr 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 404 6,866 0 13 Apr 2022
Learning Trajectory-Aware Transformer for Video Super-Resolution Chengxu Liu Huan Yang Jianlong Fu Xueming Qian ViT 73 86 0 08 Apr 2022
Video Diffusion Models Jonathan Ho Tim Salimans Alexey A. Gritsenko William Chan Mohammad Norouzi David J. Fleet DiffM VGen 204 1,626 0 07 Apr 2022
Diffusion Probabilistic Modeling for Video Generation Ruihan Yang Prakhar Srivastava Stephan Mandt DiffM VGen 128 266 0 16 Mar 2022
Generative Adversarial Networks Gilad Cohen Raja Giryes GAN 280 30,103 0 01 Mar 2022
RePaint: Inpainting using Denoising Diffusion Probabilistic Models Andreas Lugmayr Martin Danelljan Andrés Romero Feng Yu Radu Timofte Luc Van Gool DiffM 349 1,409 0 24 Jan 2022
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models Alex Nichol Prafulla Dhariwal Aditya A. Ramesh Pranav Shyam Pamela Mishkin Bob McGrew Ilya Sutskever Mark Chen 353 3,605 0 20 Dec 2021
Tackling the Generative Learning Trilemma with Denoising Diffusion GANs Zhisheng Xiao Karsten Kreis Arash Vahdat DiffM 98 555 0 15 Dec 2021
Diffusion Autoencoders: Toward a Meaningful and Decodable Representation Konpat Preechakul Nattanat Chatthee Suttisak Wizadwongsa Supasorn Suwajanakorn SyDa DiffM 119 431 0 30 Nov 2021
Blended Diffusion for Text-driven Editing of Natural Images Omri Avrahami Dani Lischinski Ohad Fried DiffM 121 947 0 29 Nov 2021
Conditional Image Generation with Score-Based Diffusion Models Georgios Batzolis Jan Stanczuk Carola-Bibiane Schönlieb Christian Etmann DiffM 43 192 0 26 Nov 2021
Palette: Image-to-Image Diffusion Models Chitwan Saharia William Chan Huiwen Chang Chris A. Lee Jonathan Ho Tim Salimans David J. Fleet Mohammad Norouzi DiffM VLM 484 1,640 0 10 Nov 2021
Talking Head Generation with Audio and Speech Related Facial Action Units Sen Chen Zhilei Liu Jiaxing Liu Zhengxiang Yan Longbiao Wang CVBM 46 16 0 19 Oct 2021
EdiTTS: Score-based Editing for Controllable Text-to-Speech Jaesung Tae Hyeongju Kim Taesu Kim DiffM 226 40 0 06 Oct 2021
FACIAL: Synthesizing Dynamic Talking Face with Implicit Attribute Learning Chenxu Zhang Yifan Zhao Yifei Huang Ming Zeng Saifeng Ni M. Budagavi Xiaohu Guo CVBM 68 125 0 18 Aug 2021
Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion Suzhe Wang Lincheng Li Yu-qiong Ding Changjie Fan Xin Yu VGen 94 163 0 20 Jul 2021
LipSync3D: Data-Efficient Learning of Personalized 3D Talking Faces from Video using Pose and Lighting Normalization A. Lahiri Vivek Kwatra C. Frueh J. P. Lewis C. Bregler 3DH 73 101 0 08 Jun 2021
Cascaded Diffusion Models for High Fidelity Image Generation Jonathan Ho Chitwan Saharia William Chan David J. Fleet Mohammad Norouzi Tim Salimans 161 1,222 0 30 May 2021
Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech Vadim Popov Ivan Vovk Vladimir Gogoryan Tasnima Sadekova Mikhail Kudinov DiffM 97 533 0 13 May 2021
Diffusion Models Beat GANs on Image Synthesis Prafulla Dhariwal Alex Nichol 230 7,857 0 11 May 2021
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation Hang Zhou Yasheng Sun Wayne Wu Chen Change Loy Xiaogang Wang Ziwei Liu CVBM 107 366 0 22 Apr 2021
MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement Alexander Richard Michael Zollhoefer Yandong Wen Fernando de la Torre Yaser Sheikh CVBM 66 200 0 16 Apr 2021
Improved Denoising Diffusion Probabilistic Models Alex Nichol Prafulla Dhariwal DiffM 340 3,702 0 18 Feb 2021
Robust One Shot Audio to Video Generation Neeraj Kumar Srishti Goel Ankur Narang H. Mujtaba VGen 36 13 0 14 Dec 2020
DiffWave: A Versatile Diffusion Model for Audio Synthesis Zhifeng Kong Ming-Yu Liu Jiaji Huang Kexin Zhao Bryan Catanzaro DiffM BDL 155 1,457 0 21 Sep 2020
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild Prajwal K R Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar EGVM 101 779 0 23 Aug 2020
Talking-head Generation with Rhythmic Head Motion Lele Chen Guofeng Cui Celong Liu Zhong Li Ziyi Kou Yi Tian Xu Chenliang Xu 57 182 0 16 Jul 2020
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations Alexei Baevski Henry Zhou Abdel-rahman Mohamed Michael Auli SSL 285 5,801 0 20 Jun 2020
MakeItTalk: Speaker-Aware Talking-Head Animation Yang Zhou Xintong Han Eli Shechtman J. Echevarria E. Kalogerakis Dingzeyu Li 63 421 0 27 Apr 2020
Audio-driven Talking Face Video Generation with Learning-based Personalized Head Pose Ran Yi Zipeng Ye Juyong Zhang Hujun Bao Yong Liu CVBM 72 123 0 24 Feb 2020
Everybody's Talkin': Let Me Talk as You Want Linsen Song Wayne Wu Chao Qian Ran He Chen Change Loy DiffM VGen 77 145 0 15 Jan 2020
DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection Ruben Tolosana R. Vera-Rodríguez Julian Fierrez Aythami Morales J. Ortega-Garcia 3DPC CVBM 93 792 0 01 Jan 2020
Animating Face using Disentangled Audio Representations Gaurav Mittal Baoyuan Wang CVBM 81 39 0 02 Oct 2019
Generative Modeling by Estimating Gradients of the Data Distribution Yang Song Stefano Ermon SyDa DiffM 258 3,916 0 12 Jul 2019
Realistic Speech-Driven Facial Animation with GANs Konstantinos Vougioukas Stavros Petridis Maja Pantic 127 294 0 14 Jun 2019
MediaPipe: A Framework for Building Perception Pipelines C. Lugaresi Jiuqiang Tang Hadon Nash C. McClanahan Esha Uboweja ... Juhyun Lee W. Chang Wei Hua Manfred Georg Matthias Grundmann MoE 72 1,301 0 14 Jun 2019
Capture, Learning, and Synthesis of 3D Speaking Styles Daniel Cudeiro Timo Bolkart Cassidy Laidlaw Anurag Ranjan Michael J. Black CVBM 3DH 93 343 0 08 May 2019
Arbitrary Talking Face Generation via Attentional Audio-Visual Coherence Learning Hao Zhu Huaibo Huang Yi Li Aihua Zheng Ran He CVBM 56 58 0 17 Dec 2018
Talking Face Generation by Adversarially Disentangled Audio-Visual Representation Hang Zhou Yu Liu Ziwei Liu Ping Luo Xiaogang Wang CVBM 92 441 0 20 Jul 2018
End-to-End Speech-Driven Facial Animation with Temporal GANs Konstantinos Vougioukas Stavros Petridis Maja Pantic CVBM 65 103 0 23 May 2018