A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild

23 August 2020

Prajwal K R

Rudrabha Mukhopadhyay

Papers citing "A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild"

50 / 410 papers shown

Title
Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text Modalities Rebecca Mobbs Dimitrios Makris Vasileios Argyriou 43 0 0 02 Feb 2025
Towards Dynamic Neural Communication and Speech Neuroprosthesis Based on Viseme Decoding Ji-Ha Park Seo-Hyun Lee Soowon Kim Seong-Whan Lee 43 0 0 28 Jan 2025
DEGSTalk: Decomposed Per-Embedding Gaussian Fields for Hair-Preserving Talking Face Synthesis Kaijun Deng Dezhi Zheng Jindong Xie Jinbao Wang Weicheng Xie L. Shen Siyang Song 3DGS 37 0 0 31 Dec 2024
HELPNet: Hierarchical Perturbations Consistency and Entropy-guided Ensemble for Scribble Supervised Medical Image Segmentation Xiao Zhang Shaoxuan Wu Peilin Zhang Zhuo Jin Xiaosong Xiong Qirong Bu Jingkun Chen Jun Feng 94 0 0 25 Dec 2024
FADA: Fast Diffusion Avatar Synthesis with Mixed-Supervised Multi-CFG Distillation Tianyun Zhong Chao Liang Jianwen Jiang Gaojie Lin Jiaqi Yang Zhou Zhao DiffM 102 1 0 22 Dec 2024
Improving Lip-synchrony in Direct Audio-Visual Speech-to-Speech Translation Lucas Goncalves Prashant Mathur Xing Niu Brady Houston Chandrashekhar Lavania Srikanth Vishnubhotla Lijia Sun Anthony Ferritto 77 0 0 21 Dec 2024
Real-time One-Step Diffusion-based Expressive Portrait Videos Generation Hanzhong Guo Hongwei Yi Daquan Zhou Alexander William Bergman Michael Lingelbach Yizhou Yu DiffM 85 1 0 18 Dec 2024
EmoDubber: Towards High Quality and Emotion Controllable Movie Dubbing Gaoxiang Cong Jiadong Pan Liang-Sheng Li Yuankai Qi Yuxin Peng Anton Van Den Hengel Jian Yang Qingming Huang 92 6 0 12 Dec 2024
Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer Jiahao Cui Hui Li Yun Zhan Hanlin Shang K. Cheng Yuqi Ma Shan Mu Hang Zhou Jingdong Wang Siyu Zhu ViT VGen 104 6 0 01 Dec 2024
Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learning Dragos-Alexandru Boldisor Stefan Smeu Dan Oneaţă Elisabeta Oneata 103 1 0 29 Nov 2024
Passive Deepfake Detection Across Multi-modalities: A Comprehensive Survey Hong-Hanh Nguyen-Le Van-Tuan Tran Dinh-Thuc Nguyen Nhien-An Le-Khac AAML 110 1 0 26 Nov 2024
Sonic: Shifting Focus to Global Audio Perception in Portrait Animation Xiaozhong Ji Xiaobin Hu Zhihong Xu Junwei Zhu Chuming Lin ... Donghao Luo Yi Chen Qin Lin Qinglin Lu Chengjie Wang VGen 81 4 0 25 Nov 2024
Design-o-meter: Towards Evaluating and Refining Graphic Designs Sahil Goyal Abhinav Mahajan Swasti Mishra Prateksha Udhayanan Tripti Shukla K. J. Joseph Balaji Vasan Srinivasan 80 1 0 22 Nov 2024
EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation Rang Meng Xingyu Zhang Yuming Li Chenguang Ma 33 5 0 15 Nov 2024
How Good is ChatGPT at Audiovisual Deepfake Detection: A Comparative Study of ChatGPT, AI Models and Human Perception Sahibzada Adil Shahzad Ammarah Hashmi Yan-Tsung Peng Yu Tsao H. Wang 39 1 0 14 Nov 2024
Dynamic Neural Communication: Convergence of Computer Vision and Brain-Computer Interface Ji-Ha Park Seo-Hyun Lee Soowon Kim Seong-Whan Lee 29 0 0 14 Nov 2024
JoyVASA: Portrait and Animal Image Animation with Diffusion-Based Audio-Driven Facial Dynamics and Head Motion Generation Xuyang Cao Guoxin Wang Sheng Shi Jun Zhao Yang Yao Jintao Fei Minyu Gao VGen 44 1 0 14 Nov 2024
Stereo-Talker: Audio-driven 3D Human Synthesis with Prior-Guided Mixture-of-Experts Xiang Deng Youxin Pang Xiaochen Zhao Chao Xu Lizhen Wang Hongjiang Xiao Shi Yan Hongwen Zhang Yebin Liu DiffM VGen 40 1 0 31 Oct 2024
Takin-ADA: Emotion Controllable Audio-Driven Animation with Canonical and Landmark Loss Optimization Bin Lin Yanzhen Yu Jianhao Ye Ruitao Lv Yuqing Yang Ruoye Xie Pan Yu Hongbin Zhou VGen 35 1 0 18 Oct 2024
DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework for Talking Head Video Generation Hanbo Cheng Limin Lin Chenyu Liu Pengcheng Xia Pengfei Hu Jiefeng Ma Jun Du Jia Pan DiffM VGen 165 0 0 17 Oct 2024
Titanic Calling: Low Bandwidth Video Conference from the Titanic Wreck Fevziye Irem Eyiokur Christian Huber Thai-Binh Nguyen T. Nguyen Fabian Retkowski Enes Yavuz Ugan Dogucan Yaman Alexander Waibel 29 0 0 15 Oct 2024
Separation of Neural Drives to Muscles from Transferred Polyfunctional Nerves using Implanted Micro-electrode Arrays Laura Ferrante Anna Boesendorfer D. Barsakcioglu Benedikt Baumgartner Yazan Al-Ajam Alex Woollard Norbert Venantius Kang Oskar Aszmann D. Farina 44 7 0 14 Oct 2024
Beyond Fixed Topologies: Unregistered Training and Comprehensive Evaluation Metrics for 3D Talking Heads Federico Nocentini T. Besnier Claudio Ferrari Sylvain Arguillere Stefano Berretti Mohamed Daoudi 59 1 0 14 Oct 2024
MuseTalk: Real-Time High-Fidelity Video Dubbing via Spatio-Temporal Sampling Yue Zhang Minhao Liu Zhaokang Chen Bin Wu Yubin Zeng Chao Zhan Yingjie He Junxin Huang Wenjiang Zhou Wenjiang Zhou 36 6 0 14 Oct 2024
Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animation Jiahao Cui Hui Li Yao Yao Hao Zhu Hanlin Shang Kaihui Cheng Hang Zhou Siyu Zhu Jingdong Wang DiffM VGen 46 22 0 10 Oct 2024
MimicTalk: Mimicking a personalized and expressive 3D talking face in minutes Zhenhui Ye Tianyun Zhong Yi Ren Ziyue Jiang Jiawei Huang ... Chen Zhang Zehan Wang Xize Chen Xiang Yin Zhou Zhao VGen 39 3 0 09 Oct 2024
EmoGene: Audio-Driven Emotional 3D Talking-Head Generation Wenqing Wang Yun Fu VGen 79 0 0 07 Oct 2024
TANGO: Co-Speech Gesture Video Reenactment with Hierarchical Audio Motion Embedding and Diffusion Interpolation Haiyang Liu Xingchao Yang Tomoya Akiyama Yuantian Huang Qiaoge Li Shigeru Kuriyama Takafumi Taketomi VGen SLR 22 7 0 05 Oct 2024
LaDTalk: Latent Denoising for Synthesizing Talking Head Videos with High Frequency Details Jian Yang Xukun Wang Wentao Wang Guoming Li Qihang Fang Ruihong Yuan Tianyang Wang Jason Zhaoxin Fan Yeying Jin Zhaoxin Fan VGen 47 1 0 01 Oct 2024
Learning Frame-Wise Emotion Intensity for Audio-Driven Talking-Head Generation Jingyi Xu Hieu Le Zhixin Shu Yang Wang Yi-Hsuan Tsai Dimitris Samaras 34 0 0 29 Sep 2024
Face Forgery Detection with Elaborate Backbone Zonghui Guo Y. Liu Jie Zhang Haiyong Zheng Shiguang Shan AAML CVBM 28 1 0 25 Sep 2024
TalkinNeRF: Animatable Neural Fields for Full-Body Talking Humans Aggelina Chatziagapi Bindita Chaudhuri Amit Kumar Rakesh Ranjan Dimitris Samaras N. Sarafianos 3DH AI4CE 22 1 0 25 Sep 2024
MIMAFace: Face Animation via Motion-Identity Modulated Appearance Feature Learning Yue Han Junwei Zhu Yuxiang Feng Xiaozhong Ji Keke He Xiangtai Li Zhucun Xue Yong Liu 26 0 0 23 Sep 2024
A Comprehensive Survey with Critical Analysis for Deepfake Speech Detection Lam Pham Phat Lam Dat Tran Hieu Tang Tin Nguyen Alexander Schindler Canh Vu Alexander Polonsky Canh Vu 56 3 0 23 Sep 2024
DreamHead: Learning Spatial-Temporal Correspondence via Hierarchical Diffusion for Audio-driven Talking Head Synthesis Fa-Ting Hong Yunfei Liu Yu Li Changyin Zhou Fei Yu D. Xu DiffM 35 0 0 16 Sep 2024
LawDNet: Enhanced Audio-Driven Lip Synthesis via Local Affine Warping Deformation Deng Junli Luo Yihao Yang Xueting Li Siyou Wang Wei Guo Jinyang Shi Ping 26 0 0 14 Sep 2024
StyleTalk++: A Unified Framework for Controlling the Speaking Styles of Talking Heads Suzhen Wang Yifeng Ma Yu Ding Zhipeng Hu Changjie Fan Tangjie Lv Zhidong Deng Xin Yu 46 9 0 14 Sep 2024
Multi-modal Speech Transformer Decoders: When Do Multiple Modalities Improve Accuracy? Yiwen Guan V. Trinh Vivek Voleti Jacob Whitehill 42 1 0 13 Sep 2024
PersonaTalk: Bring Attention to Your Persona in Visual Dubbing Longhao Zhang Shuang Liang Zhipeng Ge Tianshu Hu DiffM VGen 29 5 0 09 Sep 2024
KAN-Based Fusion of Dual-Domain for Audio-Driven Facial Landmarks Generation Hoang-Son Vo-Thanh Quang Vinh Nguyen Soo-Hyung Kim CVBM 29 0 0 09 Sep 2024
SegTalker: Segmentation-based Talking Face Generation with Mask-guided Local Editing Lingyu Xiong Xize Cheng Jintao Tan Wenxiong Kang Xiandong Li Lei Zhu Fei Ma Minglei Li Huang Xu Zhihu Hu 34 3 0 05 Sep 2024
SVP: Style-Enhanced Vivid Portrait Talking Head Diffusion Model Weipeng Tan Chuming Lin Chengming Xu Xiaozhong Ji Junwei Zhu Chengjie Wang Yanwei Fu DiffM 41 0 0 05 Sep 2024
CyberHost: Taming Audio-driven Avatar Diffusion Model with Region Codebook Attention Gaojie Lin Jianwen Jiang Chao Liang Tianyun Zhong Jiaqi Yang Yanbo Zheng VGen DiffM 69 13 0 03 Sep 2024
Interpretable Convolutional SyncNet Sungjoon Park Jaesub Yun Donggeon Lee Minsik Park 59 0 0 02 Sep 2024
Empowering Sign Language Communication: Integrating Sentiment and Semantics for Facial Expression Synthesis Rafael Azevedo Thiago M. Coutinho Joao Klock Ferreira Thiago L. Gomes Erickson R. Nascimento SLR 43 5 0 27 Aug 2024
FD2Talk: Towards Generalized Talking Head Generation with Facial Decoupled Diffusion Model Ziyu Yao Xuxin Cheng Zhiqi Huang DiffM 26 3 0 18 Aug 2024
S^3D-NeRF: Single-Shot Speech-Driven Neural Radiance Field for High Fidelity Talking Head Synthesis Dongze Li Kang Zhao Wei Wang Yifeng Ma Bo Peng Yingya Zhang Jing Dong 3DH CVBM 35 2 0 18 Aug 2024
Content and Style Aware Audio-Driven Facial Animation Qingju Liu Hyeongwoo Kim Gaurav Bharaj DiffM 43 1 0 13 Aug 2024
High-fidelity and Lip-synced Talking Face Synthesis via Landmark-based Diffusion Model Weizhi Zhong Junfan Lin Peixin Chen Liang Lin Guanbin Li 39 1 0 10 Aug 2024
Style-Preserving Lip Sync via Audio-Aware Style Reference Weizhi Zhong Jichang Li Yinqi Cai Liang Lin Guanbin Li 35 2 0 10 Aug 2024