VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild

27 November 2022

K. Cheng

Xiaodong Cun

Yong Zhang

Mingrui Zhu

Papers citing "VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild"

50 / 61 papers shown

Title
KeySync: A Robust Approach for Leakage-free Lip Synchronization in High Resolution Antoni Bigata Rodrigo Mira Stella Bounareli Michał Stypułkowski Konstantinos Vougioukas Stavros Petridis Maja Pantic 52 0 0 01 May 2025
Efficient Listener: Dyadic Facial Motion Synthesis via Action Diffusion Z. Wang Alexandre Bruckert P. Le Callet Guangtao Zhai VGen 32 0 0 29 Apr 2025
Audio-Driven Talking Face Video Generation with Joint Uncertainty Learning Yifan Xie Fei Ma Yi Bin Ying He Fei Richard Yu 57 0 0 26 Apr 2025
Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generation Weipeng Tan Chuming Lin Chengming Xu F. Xu Xiaobin Hu Xiaozhong Ji Junwei Zhu Chengjie Wang Yanwei Fu 44 0 0 25 Apr 2025
Detecting Lip-Syncing Deepfakes: Vision Temporal Transformer for Analyzing Mouth Inconsistencies Soumyya Kanti Datta Shan Jia Siwei Lyu 39 0 0 02 Apr 2025
STSA: Spatial-Temporal Semantic Alignment for Visual Dubbing Zijun Ding Mingdie Xiong Congcong Zhu Jingrun Chen DiffM 56 0 0 29 Mar 2025
HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animation Zunnan Xu Zhentao Yu Zixiang Zhou Jun Zhou Xiaoyu Jin ... Chengfei Cai Shiyu Tang Qin Lin Xiu Li Qinglin Lu DiffM VGen 91 7 0 24 Mar 2025
TruthLens: Explainable DeepFake Detection for Face Manipulated and Fully Synthetic Data Rohit Kundu Athula Balachandran A. Roy-Chowdhury 40 0 0 20 Mar 2025
PC-Talk: Precise Facial Animation Control for Audio-Driven Talking Face Generation Baiqin Wang Xiangyu Zhu Fan Shen Hao-Xuan Xu Zhen Lei 55 0 0 18 Mar 2025
Unlock Pose Diversity: Accurate and Efficient Implicit Keypoint-based Spatiotemporal Diffusion for Audio-driven Talking Portrait Chaolong Yang Kai Yao Yuyao Yan Chenru Jiang Weiguang Zhao Jie Sun Guangliang Cheng Yifei Zhang Bin Dong K. Huang DiffM 67 0 0 17 Mar 2025
RASA: Replace Anyone, Say Anything -- A Training-Free Framework for Audio-Driven and Universal Portrait Video Editing Tianrui Pan Lin Liu Jie Liu X. Zhang J. Tang Gangshan Wu Q. Tian DiffM VGen 46 0 0 14 Mar 2025
Cosh-DiT: Co-Speech Gesture Video Synthesis via Hybrid Audio-Visual Diffusion Transformers Yasheng Sun Zhiliang Xu Hang Zhou Jiazhi Guan Quanwei Yang ... Yingying Li Haocheng Feng J. Wang Ziwei Liu Koike Hideki VGen 56 0 0 13 Mar 2025
Removing Averaging: Personalized Lip-Sync Driven Characters Based on Identity Adapter Yanyu Zhu Licheng Bai Jintao Xu Jiwei Tang Hai-tao Zheng 33 0 0 09 Mar 2025
SayAnything: Audio-Driven Lip Synchronization with Conditional Video Diffusion Junxian Ma Shiwen Wang Jian Yang Junyi Hu Jian Liang Guosheng Lin Jingbo Chen Kai Li Yu Meng DiffM VGen 61 3 0 17 Feb 2025
Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook Florinel-Alin Croitoru Andrei Iulian Hiji Vlad Hondru Nicolae-Cătălin Ristea Paul Irofti Marius Popescu Cristian Rusu Radu Tudor Ionescu F. Khan Mubarak Shah 84 2 0 29 Nov 2024
Stereo-Talker: Audio-driven 3D Human Synthesis with Prior-Guided Mixture-of-Experts Xiang Deng Youxin Pang Xiaochen Zhao Chao Xu Lizhen Wang Hongjiang Xiao Shi Yan Hongwen Zhang Yebin Liu DiffM VGen 38 1 0 31 Oct 2024
Titanic Calling: Low Bandwidth Video Conference from the Titanic Wreck Fevziye Irem Eyiokur Christian Huber Thai-Binh Nguyen T. Nguyen Fabian Retkowski Enes Yavuz Ugan Dogucan Yaman Alexander Waibel 27 0 0 15 Oct 2024
Separation of Neural Drives to Muscles from Transferred Polyfunctional Nerves using Implanted Micro-electrode Arrays Laura Ferrante Anna Boesendorfer D. Barsakcioglu Benedikt Baumgartner Yazan Al-Ajam Alex Woollard Norbert Venantius Kang Oskar Aszmann D. Farina 36 0 0 14 Oct 2024
MuseTalk: Real-Time High-Fidelity Video Dubbing via Spatio-Temporal Sampling Yue Zhang Minhao Liu Zhaokang Chen Bin Wu Yubin Zeng Chao Zhan Yingjie He Junxin Huang Wenjiang Zhou Wenjiang Zhou 34 6 0 14 Oct 2024
EmoGene: Audio-Driven Emotional 3D Talking-Head Generation Wenqing Wang Yun Fu VGen 74 0 0 07 Oct 2024
PersonaTalk: Bring Attention to Your Persona in Visual Dubbing Longhao Zhang Shuang Liang Zhipeng Ge Tianshu Hu DiffM VGen 26 5 0 09 Sep 2024
KAN-Based Fusion of Dual-Domain for Audio-Driven Facial Landmarks Generation Hoang-Son Vo-Thanh Quang Vinh Nguyen Soo-Hyung Kim CVBM 24 0 0 09 Sep 2024
SegTalker: Segmentation-based Talking Face Generation with Mask-guided Local Editing Lingyu Xiong Xize Cheng Jintao Tan Xianjia Wu Xiandong Li Lei Zhu Fei Ma Minglei Li Huang Xu Zhihu Hu 29 3 0 05 Sep 2024
SVP: Style-Enhanced Vivid Portrait Talking Head Diffusion Model Weipeng Tan Chuming Lin Chengming Xu Xiaozhong Ji Junwei Zhu Chengjie Wang Yanwei Fu DiffM 41 0 0 05 Sep 2024
Digital Avatars: Framework Development and Their Evaluation Timothy Rupprecht Sung-En Chang Yushu Wu Lei Lu Enfu Nan ... Zhimin Li Zhijun Hu Yumei He David Kaeli Yanzhi Wang 18 0 0 07 Aug 2024
ReSyncer: Rewiring Style-based Generator for Unified Audio-Visually Synced Facial Performer Jiazhi Guan Zhiliang Xu Hang Zhou Kaisiyuan Wang Shengyi He ... Errui Ding Jingtuo Liu Jingdong Wang Youjian Zhao Ziwei Liu VGen 46 2 0 06 Aug 2024
Audio-driven High-resolution Seamless Talking Head Video Editing via StyleGAN Jiacheng Su Kunhong Liu Liyan Chen Junfeng Yao Qingsong Liu Dongdong Lv VGen 43 3 0 08 Jul 2024
RealTalk: Real-time and Realistic Audio-driven Face Generation with 3D Facial Prior-guided Identity Alignment Network Xiaozhong Ji Chuming Lin Zhonggan Ding Ying Tai Junwei Zhu Xiaobin Hu Donghao Luo Yanhao Ge Chengjie Wang CVBM 24 2 0 26 Jun 2024
A Comprehensive Taxonomy and Analysis of Talking Head Synthesis: Techniques for Portrait Generation, Driving Mechanisms, and Editing Ming Meng Yufei Zhao Bo Zhang Yonggui Zhu Weimin Shi Maxwell Wen Zhaoxin Fan VGen 34 1 0 15 Jun 2024
Make Your Actor Talk: Generalizable and High-Fidelity Lip Sync with Motion and Appearance Disentanglement Runyi Yu Tianyu He Ailing Zhang Yuchi Wang Junliang Guo Xu Tan Chang Liu Jie Chen Jiang Bian VGen 26 4 0 12 Jun 2024
Controllable Talking Face Generation by Implicit Facial Keypoints Editing Dong Zhao Jiaying Shi Wenjun Li Shudong Wang Shenghui Xu Zhaoming Pan CVBM 41 0 0 05 Jun 2024
OpFlowTalker: Realistic and Natural Talking Face Generation via Optical Flow Guidance Shuheng Ge Haoyu Xing Li Zhang Xiangqian Wu 31 0 0 23 May 2024
PolyGlotFake: A Novel Multilingual and Multimodal DeepFake Dataset Yang Hou Haitao Fu Chuankai Chen Zida Li Haoyu Zhang Jianjun Zhao 24 3 0 14 May 2024
NeRFFaceSpeech: One-shot Audio-driven 3D Talking Head Synthesis via Generative Prior Gihoon Kim Kwanggyoon Seo Sihun Cha Junyong Noh DiffM 3DH 34 1 0 09 May 2024
Audio-Visual Speech Representation Expert for Enhanced Talking Face Video Generation and Evaluation Dogucan Yaman Fevziye Irem Eyiokur Leonard Barmann Seymanur Akti H. K. Ekenel Alexander H. Waibel EGVM 25 9 0 07 May 2024
Explicit Correlation Learning for Generalizable Cross-Modal Deepfake Detection Cai Yu Shan Jia Xiaomeng Fu Jin Liu Jiahe Tian Jiao Dai Xi Wang Siwei Lyu Jizhong Han 34 5 0 30 Apr 2024
VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time Sicheng Xu Guojun Chen Yu-Xiao Guo Jiaolong Yang Chong Li Zhenyu Zang Yizhong Zhang Xin Tong Baining Guo 40 86 0 16 Apr 2024
THQA: A Perceptual Quality Assessment Database for Talking Heads Yingjie Zhou Zicheng Zhang Wei Sun Xiaohong Liu Xiongkuo Min Zhihua Wang Xiao-Ping Zhang Guangtao Zhai EGVM 27 10 0 13 Apr 2024
Learning to Generate Conditional Tri-plane for 3D-aware Expression Controllable Portrait Animation Taekyung Ki Dongchan Min Gyeongsu Chae 3DH 30 5 0 31 Mar 2024
Adaptive Super Resolution For One-Shot Talking-Head Generation Luchuan Song Pinxin Liu Guojun Yin Chenliang Xu 14 7 0 23 Mar 2024
G4G:A Generic Framework for High Fidelity Talking Face Generation with Fine-grained Intra-modal Alignment Juan Zhang Jiahao Chen Cheng Wang Zhi-Yang Yu Tangquan Qi Di Wu CVBM 30 0 0 28 Feb 2024
GMTalker: Gaussian Mixture-based Audio-Driven Emotional Talking Video Portraits Yibo Xia Lizhen Wang Xiang Deng Xiaoyan Luo Yunhong Wang Yebin Liu VGen 33 1 0 12 Dec 2023
Neural Text to Articulate Talk: Deep Text to Audiovisual Speech Synthesis achieving both Auditory and Photo-realism Georgios Milis P. Filntisis A. Roussos Petros Maragos CVBM 24 2 0 11 Dec 2023
SyncTalk: The Devil is in the Synchronization for Talking Head Synthesis Ziqiao Peng Wentao Hu Yue Shi Xiangyu Zhu Xiaomei Zhang Hao Zhao Jun He Hongyan Liu Zhaoxin Fan 33 39 0 29 Nov 2023
MFR-Net: Multi-faceted Responsive Listening Head Generation via Denoising Diffusion Model Jin Liu Xi Wang Xiaomeng Fu Yesheng Chai Cai Yu Jiao Dai Jizhong Han DiffM 21 10 0 31 Aug 2023
FaceChain: A Playground for Human-centric Artificial Intelligence Generated Content Yang Liu Cheng Yu Lei Shang Yongyi He Ziheng Wu ... Jiaqi Xu Qiang Wang Yingda Chen Xuansong Xie Baigui Sun 33 5 0 28 Aug 2023
Text-to-Video: a Two-stage Framework for Zero-shot Identity-agnostic Talking-head Generation Zhichao Wang M. Dai Keld Lundgaard VGen DiffM 43 2 0 12 Aug 2023
A Unified Framework for Modality-Agnostic Deepfakes Detection Cai Yu Peng-Wen Chen Jiahe Tian Jin Liu Jiao Dai Xi Wang Yesheng Chai Shan Jia Siwei Lyu Jizhong Han 24 0 0 26 Jul 2023
Audio-driven Talking Face Generation with Stabilized Synchronization Loss Dogucan Yaman Fevziye Irem Eyiokur Leonard Barmann H. K. Ekenel Alexander Waibel CVBM 27 3 0 18 Jul 2023
My3DGen: A Scalable Personalized 3D Generative Model Luchao Qi Jiaye Wu Annie N. Wang Sheng-Yu Wang Roni Sengupta 3DH 34 3 0 11 Jul 2023