DAE-Talker: High Fidelity Speech-Driven Talking Face Generation with Diffusion Autoencoder

30 March 2023

Chenpeng Du

Qi Chen

Xie Chen

K. Yu

DiffM

ArXiv PDF HTML

Papers citing "DAE-Talker: High Fidelity Speech-Driven Talking Face Generation with Diffusion Autoencoder"

41 / 41 papers shown

Title
KeySync: A Robust Approach for Leakage-free Lip Synchronization in High Resolution Antoni Bigata Rodrigo Mira Stella Bounareli Michał Stypułkowski Konstantinos Vougioukas Stavros Petridis Maja Pantic 52 0 0 01 May 2025
Revisiting Diffusion Autoencoder Training for Image Reconstruction Quality Pramook Khungurn Sukit Seripanitkarn Phonphrm Thawatdamrongkit Supasorn Suwajanakorn DiffM 77 0 0 30 Apr 2025
MagicPortrait: Temporally Consistent Face Reenactment with 3D Geometric Guidance Mengting Wei Yante Li Tuomas Varanka Yan Jiang Guoying Zhao DiffM VGen 74 0 0 30 Apr 2025
3D Engine-ready Photorealistic Avatars via Dynamic Textures Yifan Wang Ivan Molodetskikh Ondrej Texler Dimitar Dinev 45 0 0 19 Mar 2025
SyncDiff: Diffusion-based Talking Head Synthesis with Bottlenecked Temporal Visual Prior for Improved Synchronization Xulin Fan Heting Gao Ziyi Chen Peng Chang Mei Han Mark Hasegawa-Johnson DiffM 62 0 0 17 Mar 2025
FREAK: Frequency-modulated High-fidelity and Real-time Audio-driven Talking Portrait Synthesis Ziqi Ni Ao Fu Yi Zhou 61 0 0 06 Mar 2025
KeyFace: Expressive Audio-Driven Facial Animation for Long Sequences via KeyFrame Interpolation Antoni Bigata Michał Stypułkowski Rodrigo Mira Stella Bounareli Konstantinos Vougioukas Zoe Landgraf Nikita Drobyshev Maciej Ziȩba Stavros Petridis M. Pantic DiffM VGen 70 2 0 03 Mar 2025
Dimitra: Audio-driven Diffusion model for Expressive Talking Head Generation Baptiste Chopin Tashvik Dhamija P. Balaji Yaohui Wang A. Dantcheva DiffM VGen 49 0 0 24 Feb 2025
Takin-ADA: Emotion Controllable Audio-Driven Animation with Canonical and Landmark Loss Optimization Bin Lin Yanzhen Yu Jianhao Ye Ruitao Lv Yuqing Yang Ruoye Xie Pan Yu Hongbin Zhou VGen 35 1 0 18 Oct 2024
DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework for Talking Head Video Generation Hanbo Cheng Limin Lin Chenyu Liu Pengcheng Xia Pengfei Hu Jiefeng Ma Jun Du Jia Pan DiffM VGen 153 0 0 17 Oct 2024
MIMAFace: Face Animation via Motion-Identity Modulated Appearance Feature Learning Yue Han Junwei Zhu Yuxiang Feng Xiaozhong Ji Keke He Xiangtai Li Zhucun Xue Yong Liu 26 0 0 23 Sep 2024
DreamHead: Learning Spatial-Temporal Correspondence via Hierarchical Diffusion for Audio-driven Talking Head Synthesis Fa-Ting Hong Yunfei Liu Yu Li Changyin Zhou Fei Yu D. Xu DiffM 35 0 0 16 Sep 2024
FD2Talk: Towards Generalized Talking Head Generation with Facial Decoupled Diffusion Model Ziyu Yao Xuxin Cheng Zhiqi Huang DiffM 21 3 0 18 Aug 2024
High-fidelity and Lip-synced Talking Face Synthesis via Landmark-based Diffusion Model Weizhi Zhong Junfan Lin Peixin Chen Liang Lin Guanbin Li 39 1 0 10 Aug 2024
Style-Preserving Lip Sync via Audio-Aware Style Reference Weizhi Zhong Jichang Li Yinqi Cai Liang Lin Guanbin Li 35 2 0 10 Aug 2024
JambaTalk: Speech-Driven 3D Talking Head Generation Based on Hybrid Transformer-Mamba Language Model Farzaneh Jafari Stefano Berretti Anup Basu Mamba 39 1 0 03 Aug 2024
Text-based Talking Video Editing with Cascaded Conditional Diffusion Bo Han Heqing Zou Haoyang Li Guangcong Wang Chng Eng Siong VGen DiffM 37 2 0 20 Jul 2024
Make Your Actor Talk: Generalizable and High-Fidelity Lip Sync with Motion and Appearance Disentanglement Runyi Yu Tianyu He Ailing Zhang Yuchi Wang Junliang Guo Xu Tan Chang Liu Jie Chen Jiang Bian VGen 34 4 0 12 Jun 2024
Emotional Conversation: Empowering Talking Faces with Cohesive Expression, Gaze and Pose Generation Jiadong Liang Feng Lu CVBM 34 0 0 12 Jun 2024
OpFlowTalker: Realistic and Natural Talking Face Generation via Optical Flow Guidance Shuheng Ge Haoyu Xing Li Zhang Xiangqian Wu 39 0 0 23 May 2024
GSTalker: Real-time Audio-Driven Talking Face Generation via Deformable Gaussian Splatting Bo Chen Shoukang Hu Qi Chen Chenpeng Du Ran Yi Yanmin Qian Xie Chen 3DGS 15 8 0 29 Apr 2024
VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time Sicheng Xu Guojun Chen Yu-Xiao Guo Jiaolong Yang Chong Li Zhenyu Zang Yizhong Zhang Xin Tong Baining Guo 48 87 0 16 Apr 2024
Superior and Pragmatic Talking Face Generation with Teacher-Student Framework Chao Liang Jianwen Jiang Tianyun Zhong Gaojie Lin Zhengkun Rong Jiaqi Yang Yongming Zhu 45 1 0 26 Mar 2024
Deepfake Generation and Detection: A Benchmark and Survey Gan Pei Jiangning Zhang Menghan Hu Zhenyu Zhang Chengjie Wang Yunsheng Wu Guangtao Zhai Jian Yang Chunhua Shen Dacheng Tao 52 25 0 26 Mar 2024
DiffusionAct: Controllable Diffusion Autoencoder for One-shot Face Reenactment Stella Bounareli Christos Tzelepis Vasileios Argyriou Ioannis Patras Georgios Tzimiropoulos DiffM 45 7 0 25 Mar 2024
Model Will Tell: Training Membership Inference for Diffusion Models Xiaomeng Fu Xi Wang Qiao Li Jin Liu Jiao Dai Jizhong Han 52 5 0 13 Mar 2024
FaceChain-ImagineID: Freely Crafting High-Fidelity Diverse Talking Faces from Disentangled Audio Chao Xu Yang Liu Jiazheng Xing Weida Wang Mingze Sun ... Tianxin Huang Siyuan Li Zhi-Qi Cheng Ying Tai Baigui Sun CVBM 54 11 0 04 Mar 2024
G4G:A Generic Framework for High Fidelity Talking Face Generation with Fine-grained Intra-modal Alignment Juan Zhang Jiahao Chen Cheng Wang Zhi-Yang Yu Tangquan Qi Di Wu CVBM 41 0 0 28 Feb 2024
Dubbing for Everyone: Data-Efficient Visual Dubbing using Neural Rendering Priors Jack D. Saunders Vinay P. Namboodiri VGen DiffM 31 1 0 11 Jan 2024
DreamTalk: When Expressive Talking Head Generation Meets Diffusion Probabilistic Models Yifeng Ma Shiwei Zhang Jiayu Wang Xiang Wang Yingya Zhang Zhidong Deng DiffM 47 2 0 15 Dec 2023
GAIA: Zero-shot Talking Avatar Generation Tianyu He Junliang Guo Runyi Yu Yuchi Wang Jialiang Zhu ... Chunyu Wang Han Hu HsiangTao Wu Sheng Zhao Jiang Bian 31 25 0 26 Nov 2023
DiffDub: Person-generic Visual Dubbing Using Inpainting Renderer with Diffusion Auto-encoder Tao Liu Chenpeng Du Shuai Fan Feilong Chen Kai Yu DiffM VGen 14 6 0 03 Nov 2023
State of the Art on Diffusion Models for Visual Computing Ryan Po Wang Yifan Vladislav Golyanik Kfir Aberman Jonathan T. Barron ... Matthias Nießner Bjorn Ommer Christian Theobalt Peter Wonka Gordon Wetzstein 33 102 0 11 Oct 2023
FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusion Stefan Stan Kazi Injamamul Haque Zerrin Yumak DiffM 31 54 0 20 Sep 2023
Towards the generation of synchronized and believable non-verbal facial behaviors of a talking virtual agent Alice Delbosc M. Ochs Nicolas Sabouret Brian Ravenet Stéphane Ayache 40 7 0 15 Sep 2023
DiffTalker: Co-driven audio-image diffusion for talking faces via intermediate landmarks Zipeng Qi Xulong Zhang Ning Cheng Jing Xiao Jianzong Wang 22 7 0 14 Sep 2023
DF-3DFace: One-to-Many Speech Synchronized 3D Face Animation with Diffusion Se Jin Park Joanna Hong Minsu Kim Y. Ro 37 4 0 23 Aug 2023
High-Fidelity Eye Animatable Neural Radiance Fields for Human Face Hengfei Wang Zhongqun Zhang Yihua Cheng H. Chang 3DH CVBM 14 5 0 01 Aug 2023
Learning and Evaluating Human Preferences for Conversational Head Generation Mohan Zhou Yalong Bai Wei Zhang Ting Yao Tiejun Zhao Tao Mei 32 2 0 20 Jul 2023
A Comprehensive Multi-scale Approach for Speech and Dynamics Synchrony in Talking Head Generation Louis Airale Dominique Vaufreydaz Xavier Alameda-Pineda 23 1 0 04 Jul 2023
Talking Head from Speech Audio using a Pre-trained Image Generator M. M. Alghamdi He Wang A. Bulpitt David C. Hogg 75 21 0 09 Sep 2022