A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild

23 August 2020

Prajwal K R

Rudrabha Mukhopadhyay

Papers citing "A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild"

50 / 410 papers shown

Title
Synchronous Multi-modal Semantic Communication System with Packet-level Coding Yun Tian Jingkai Ying Zhijin Qin Ye Jin Xiaoming Tao 37 3 0 08 Aug 2024
Digital Avatars: Framework Development and Their Evaluation Timothy Rupprecht Sung-En Chang Yushu Wu Lei Lu Enfu Nan ... Zhimin Li Zhijun Hu Yumei He David Kaeli Yanzhi Wang 31 0 0 07 Aug 2024
ReSyncer: Rewiring Style-based Generator for Unified Audio-Visually Synced Facial Performer Jiazhi Guan Zhiliang Xu Hang Zhou Kaisiyuan Wang Shengyi He ... Errui Ding Jingtuo Liu Jingdong Wang Youjian Zhao Ziwei Liu VGen 54 2 0 06 Aug 2024
Landmark-guided Diffusion Model for High-fidelity and Temporally Coherent Talking Head Generation Jintao Tan Xize Cheng Lingyu Xiong Lei Zhu Xiandong Li Wenxiong Kang Kai Gong Minglei Li Yi Cai DiffM 28 2 0 03 Aug 2024
EmoTalk3D: High-Fidelity Free-View Synthesis of Emotional 3D Talking Head Qianyun He Xinya Ji Ruichen Zheng Yuanxun Lu Zhengyu Diao ... Songcen Xu Xiaofei Wu Zixiao Zhang Xun Cao Hao Zhu 34 4 0 01 Aug 2024
Text-based Talking Video Editing with Cascaded Conditional Diffusion Bo Han Heqing Zou Haoyang Li Guangcong Wang Chng Eng Siong VGen DiffM 37 2 0 20 Jul 2024
EmoFace: Audio-driven Emotional 3D Face Animation Chang Liu Qunfen Lin Zijiao Zeng Ye Pan CVBM 44 4 0 17 Jul 2024
Learning Online Scale Transformation for Talking Head Video Generation Fa-Ting Hong Dan Xu 60 1 0 13 Jul 2024
One-Shot Pose-Driving Face Animation Platform He Feng Donglin Di Yongjia Ma Wei Chen Tonghua Su CVBM 28 1 0 12 Jul 2024
EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions Zhiyuan Chen Jiajiong Cao Zhiquan Chen Yuming Li Chenguang Ma VGen 40 49 0 11 Jul 2024
The Tug-of-War Between Deepfake Generation and Detection Hannah Lee Changyeon Lee Kevin Farhat Lin Qiu Steve Geluso Aerin Kim O. Etzioni 34 1 0 08 Jul 2024
Audio-driven High-resolution Seamless Talking Head Video Editing via StyleGAN Jiacheng Su Kunhong Liu Liyan Chen Junfeng Yao Qingsong Liu Dongdong Lv VGen 48 3 0 08 Jul 2024
MobilePortrait: Real-Time One-Shot Neural Head Avatars on Mobile Devices Jianwen Jiang Gaojie Lin Zhengkun Rong Chao Liang Yongming Zhu Jiaqi Yang Tianyun Zhong 3DH 90 8 0 08 Jul 2024
VCoME: Verbal Video Composition with Multimodal Editing Effects Weibo Gong Xiaojie Jin Xin Li Dongliang He Xinglong Wu 43 0 0 05 Jul 2024
Towards Attention-based Contrastive Learning for Audio Spoof Detection C. Goel Surya Koppisetti Ben Colman Ali Shahriyari Gaurav Bharaj 60 5 0 03 Jul 2024
RealTalk: Real-time and Realistic Audio-driven Face Generation with 3D Facial Prior-guided Identity Alignment Network Xiaozhong Ji Chuming Lin Zhonggan Ding Ying Tai Junwei Zhu Xiaobin Hu Donghao Luo Yanhao Ge Chengjie Wang CVBM 32 2 0 26 Jun 2024
A Multi-Stream Fusion Approach with One-Class Learning for Audio-Visual Deepfake Detection Kyungbok Lee You Zhang Zhiyao Duan 27 0 0 20 Jun 2024
DF40: Toward Next-Generation Deepfake Detection Zhiyuan Yan Taiping Yao Shen Chen Yandan Zhao Xinghe Fu ... Donghao Luo Li Yuan Chengjie Wang Shouhong Ding Yunsheng Wu 34 29 0 19 Jun 2024
Straight Through Gumbel Softmax Estimator based Bimodal Neural Architecture Search for Audio-Visual Deepfake Detection Aravinda Reddy PN Raghavendra Ramachandra K. S. Rao Pabitra Mitra Vinod Rathod 42 2 0 19 Jun 2024
RITA: A Real-time Interactive Talking Avatars Framework Wuxinlin Cheng Cheng Wan Yupeng Cao Sihan Chen 43 0 0 18 Jun 2024
NLDF: Neural Light Dynamic Fields for Efficient 3D Talking Head Generation Niu Guanchen 3DH 47 0 0 17 Jun 2024
A Comprehensive Taxonomy and Analysis of Talking Head Synthesis: Techniques for Portrait Generation, Driving Mechanisms, and Editing Ming Meng Yufei Zhao Bo Zhang Yonggui Zhu Weimin Shi Maxwell Wen Zhaoxin Fan VGen 46 1 0 15 Jun 2024
DubWise: Video-Guided Speech Duration Control in Multimodal LLM-based Text-to-Speech for Dubbing Neha Sahipjohn Ashishkumar Gudmalwar Nirmesh Shah Pankaj Wasnik R. Shah 43 5 0 13 Jun 2024
Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation Mingwang Xu Hui Li Qingkun Su Hanlin Shang Liwei Zhang Ce Liu Jingdong Wang Yao Yao Siyu Zhu VGen 40 68 0 13 Jun 2024
Make Your Actor Talk: Generalizable and High-Fidelity Lip Sync with Motion and Appearance Disentanglement Runyi Yu Tianyu He Ailing Zhang Yuchi Wang Junliang Guo Xu Tan Chang Liu Jie Chen Jiang Bian VGen 34 4 0 12 Jun 2024
Emotional Conversation: Empowering Talking Faces with Cohesive Expression, Gaze and Pose Generation Jiadong Liang Feng Lu CVBM 34 0 0 12 Jun 2024
Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversation Se Jin Park Chae Won Kim Hyeongseop Rha Minsu Kim Joanna Hong Jeong Hun Yeo Yong Man Ro CVBM AuLLM 48 6 0 12 Jun 2024
Zero-Shot Fake Video Detection by Audio-Visual Consistency Xiaolou Li Zehua Liu Chen Chen Lantian Li Li Guo D. Wang 63 4 0 12 Jun 2024
Principles of Designing Robust Remote Face Anti-Spoofing Systems Xiang Xu Tianchen Zhao Zheng Zhang Zhihua Li Jon Wu Alessandro Achille Mani Srivastava AAML 40 3 0 06 Jun 2024
AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection Trevine Oorloff Surya Koppisetti Nicolo Bonettini Divyaraj Solanki Ben Colman Yaser Yacoob Ali Shahriyari Gaurav Bharaj 42 21 0 05 Jun 2024
Controllable Talking Face Generation by Implicit Facial Keypoints Editing Dong Zhao Jiaying Shi Wenjun Li Shudong Wang Shenghui Xu Zhaoming Pan CVBM 49 0 0 05 Jun 2024
V-Express: Conditional Dropout for Progressive Training of Portrait Video Generation Cong Wang Kuan Tian Jun Zhang Yonghang Guan Feng Luo Fei Shen Zhiwei Jiang Qing Gu Xiao Han Wei Yang 55 38 0 04 Jun 2024
OpFlowTalker: Realistic and Natural Talking Face Generation via Optical Flow Guidance Shuheng Ge Haoyu Xing Li Zhang Xiangqian Wu 39 0 0 23 May 2024
Generative Artificial Intelligence: A Systematic Review and Applications S. S. Sengar Affan Bin Hasan Sanjay Kumar Fiona Carroll MedIm 36 52 0 17 May 2024
Faces that Speak: Jointly Synthesising Talking Face and Speech from Text Youngjoon Jang Ji-Hoon Kim Junseok Ahn Doyeop Kwak Hong-Sun Yang Yooncheol Ju Il-Hwan Kim Byeong-Yeol Kim Joon Son Chung CVBM 31 9 0 16 May 2024
Task-adaptive Q-Face Haomiao Sun Mingjie He Shiguang Shan Hu Han Xilin Chen CVBM 43 4 0 15 May 2024
PolyGlotFake: A Novel Multilingual and Multimodal DeepFake Dataset Yang Hou Haitao Fu Chuankai Chen Zida Li Haoyu Zhang Jianjun Zhao 32 3 0 14 May 2024
NeRFFaceSpeech: One-shot Audio-driven 3D Talking Head Synthesis via Generative Prior Gihoon Kim Kwanggyoon Seo Sihun Cha Junyong Noh DiffM 3DH 42 1 0 09 May 2024
SwapTalk: Audio-Driven Talking Face Generation with One-Shot Customization in Latent Space Zeren Zhang Haibo Qin Jiayu Huang Yixin Li Hui Lin Yitao Duan Jinwen Ma 35 0 0 09 May 2024
Audio-Visual Speech Representation Expert for Enhanced Talking Face Video Generation and Evaluation Dogucan Yaman Fevziye Irem Eyiokur Leonard Barmann Seymanur Akti H. K. Ekenel Alexander H. Waibel EGVM 33 9 0 07 May 2024
Unmasking Illusions: Understanding Human Perception of Audiovisual Deepfakes Ammarah Hashmi Sahibzada Adil Shahzad Chia-Wen Lin Yu Tsao Hsin-Min Wang 46 3 0 07 May 2024
Explicit Correlation Learning for Generalizable Cross-Modal Deepfake Detection Cai Yu Shan Jia Xiaomeng Fu Jin Liu Jiahe Tian Jiao Dai Xi Wang Siwei Lyu Jizhong Han 42 5 0 30 Apr 2024
EMOPortraits: Emotion-enhanced Multimodal One-shot Head Avatars Nikita Drobyshev Antoni Bigata Casademunt Konstantinos Vougioukas Zoe Landgraf Stavros Petridis Maja Pantic 46 23 0 29 Apr 2024
GSTalker: Real-time Audio-Driven Talking Face Generation via Deformable Gaussian Splatting Bo Chen Shoukang Hu Qi Chen Chenpeng Du Ran Yi Yanmin Qian Xie Chen 3DGS 20 8 0 29 Apr 2024
Embedded Representation Learning Network for Animating Styled Video Portrait Tianyong Wang Xiangyu Liang Wangguandong Zheng Dan Niu Haifeng Xia Siyu Xia 3DH 29 0 0 29 Apr 2024
Compressed Deepfake Video Detection Based on 3D Spatiotemporal Trajectories Zongmei Chen Xin Liao Xiaoshuai Wu Yanxiang Chen 34 3 0 28 Apr 2024
TalkingGaussian: Structure-Persistent 3D Talking Head Synthesis via Gaussian Splatting Jiahe Li Jiawei Zhang Xiao Bai Jin Zheng Xin Ning Jun Zhou Lin Gu 3DGS 48 15 0 23 Apr 2024
GaussianTalker: Speaker-specific Talking Head Synthesis via 3D Gaussian Splatting Hongyun Yu Zhan Qu Qihang Yu Jianchuan Chen Zhonghua Jiang ... Shengyu Zhang Jimin Xu Fei Wu Chengfei Lv Gang Yu 3DGS 38 12 0 22 Apr 2024
Learn2Talk: 3D Talking Face Learns from 2D Talking Face Yixiang Zhuang Baoping Cheng Yao Cheng Yuntao Jin Renshuai Liu Chengyang Li Xuan Cheng Jing Liao Juncong Lin CVBM 3DH 37 6 0 19 Apr 2024
VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time Sicheng Xu Guojun Chen Yu-Xiao Guo Jiaolong Yang Chong Li Zhenyu Zang Yizhong Zhang Xin Tong Baining Guo 48 87 0 16 Apr 2024