FaceFormer: Speech-Driven 3D Facial Animation with Transformers

10 December 2021

Yingruo Fan

Zhaojiang Lin

Jun Saito

Wenping Wang

Taku Komura

CVBM

ArXiv PDF HTML

Papers citing "FaceFormer: Speech-Driven 3D Facial Animation with Transformers"

50 / 132 papers shown

Title
M3G: Multi-Granular Gesture Generator for Audio-Driven Full-Body Human Motion Synthesis Zhizhuo Yin Yuk Hang Tsui Pan Hui SLR VGen 19 0 0 13 May 2025
OT-Talk: Animating 3D Talking Head with Optimal Transportation Xinmu Wang Xiang Gao Xiyun Song Heather Yu Zongfang Lin Liang Peng Xianfeng Gu 22 0 0 03 May 2025
Model See Model Do: Speech-Driven Facial Animation with Style Control Yifang Pan Karan Singh Luiz Gustavo Hafemann DiffM 50 0 0 02 May 2025
Efficient Listener: Dyadic Facial Motion Synthesis via Action Diffusion Z. Wang Alexandre Bruckert P. Le Callet Guangtao Zhai VGen 32 0 0 29 Apr 2025
Supervising 3D Talking Head Avatars with Analysis-by-Audio-Synthesis Radek Daněček Carolin Schmitt Senya Polikovsky Michael J. Black 36 0 0 18 Apr 2025
EchoMask: Speech-Queried Attention-based Mask Modeling for Holistic Co-Speech Motion Generation Xiangyue Zhang Jianfang Li Jiaxu Zhang Jianqiang Ren Liefeng Bo Zhigang Tu 30 0 0 12 Apr 2025
EasyGenNet: An Efficient Framework for Audio-Driven Gesture Video Generation Based on Diffusion Model Renda Li Xiaohua Qi Q. Ling Jun Yu Ziyi Chen Peng Chang Mei HanJing Xiao DiffM VGen 45 0 0 11 Apr 2025
Perceptually Accurate 3D Talking Head Generation: New Definitions, Speech-Mesh Representation, and Evaluation Metrics Lee Chae-Yeon Oh Hyun-Bin Han EunGi Kim Sung-Bin Suekyeong Nam Tae-Hyun Oh EGVM 3DH 87 0 1 26 Mar 2025
DiffusionTalker: Efficient and Compact Speech-Driven 3D Talking Head via Personalizer-Guided Distillation Peng Chen Xiaobao Wei Ming Lu Hui Chen Feng Tian 39 1 0 23 Mar 2025
PC-Talk: Precise Facial Animation Control for Audio-Driven Talking Face Generation Baiqin Wang Xiangyu Zhu Fan Shen Hao-Xuan Xu Zhen Lei 60 0 0 18 Mar 2025
EmoDiffusion: Enhancing Emotional 3D Facial Animation with Latent Diffusion Models Yixuan Zhang Qing Chang Yuxi Wang Guang Chen Z. Zhang Junran Peng 42 0 0 14 Mar 2025
Cafe-Talk: Generating 3D Talking Face Animation with Multimodal Coarse- and Fine-grained Control Hejia Chen Haoxian Zhang Shoulong Zhang Xiaoqiang Liu Sisi Zhuang Yuan Zhang Pengfei Wan Di Zhang Shuai Li 54 1 0 14 Mar 2025
Towards High-fidelity 3D Talking Avatar with Personalized Dynamic Texture X. Li Jianyu Wang Yuhao Cheng Yikun Zeng X. Ren W. Zhu Weiming Zhao Yichao Yan 31 0 0 01 Mar 2025
Skinned Motion Retargeting with Dense Geometric Interaction Perception Zijie Ye Jia-Wei Liu Jia Jia Shikun Sun Mike Zheng Shou 39 2 0 17 Jan 2025
Joint Co-Speech Gesture and Expressive Talking Face Generation using Diffusion with Adapters S. Hogue Chenxu Zhang Yapeng Tian Xiaohu Guo DiffM 69 0 0 18 Dec 2024
D $^2$ -World: An Efficient World Model through Decoupled Dynamic Flow Haiming Zhang Xu Yan Ying Xue Zixuan Guo Shuguang Cui Z. Li Bingbing Liu 64 0 0 26 Nov 2024
Takin-ADA: Emotion Controllable Audio-Driven Animation with Canonical and Landmark Loss Optimization Bin Lin Yanzhen Yu Jianhao Ye Ruitao Lv Y. Yang Ruoye Xie Pan Yu Hongbin Zhou VGen 30 1 0 18 Oct 2024
Beyond Fixed Topologies: Unregistered Training and Comprehensive Evaluation Metrics for 3D Talking Heads Federico Nocentini T. Besnier Claudio Ferrari Sylvain Arguillere Stefano Berretti Mohamed Daoudi 59 1 0 14 Oct 2024
MMHead: Towards Fine-grained Multi-modal 3D Facial Animation Sijing Wu Yunhao Li Yichao Yan Huiyu Duan Ziwei Liu Guangtao Zhai 3DH VGen 34 4 0 10 Oct 2024
Diverse Code Query Learning for Speech-Driven Facial Animation Chunzhi Gu Shigeru Kuriyama Katsuya Hotta DiffM 26 0 0 27 Sep 2024
FastTalker: Jointly Generating Speech and Conversational Gestures from Text Zixin Guo Jian Zhang 29 1 0 24 Sep 2024
MIMAFace: Face Animation via Motion-Identity Modulated Appearance Feature Learning Yue Han Junwei Zhu Yuxiang Feng Xiaozhong Ji Keke He Xiangtai Li Zhucun Xue Yong Liu 26 0 0 23 Sep 2024
PainDiffusion: Learning to Express Pain Quang Tien Dam Tri Tung Nguyen Nguyen D. Tran Joo-Ho Lee Joo-Ho Lee VGen 30 0 0 18 Sep 2024
3DFacePolicy: Speech-Driven 3D Facial Animation with Diffusion Policy Xuanmeng Sha Liyun Zhang Tomohiro Mashita Yuki Uranishi VGen 25 0 0 17 Sep 2024
ProbTalk3D: Non-Deterministic Emotion Controllable Speech-Driven 3D Facial Animation Synthesis Using VQ-VAE Sichun Wu Kazi Injamamul Haque Zerrin Yumak VGen 28 2 0 12 Sep 2024
TalkLoRA: Low-Rank Adaptation for Speech-Driven Animation Jack D. Saunders Vinay P. Namboodiri 45 1 0 25 Aug 2024
T3M: Text Guided 3D Human Motion Synthesis from Speech Wenshuo Peng Kaipeng Zhang Sai Qian Zhang 22 0 0 23 Aug 2024
DEGAS: Detailed Expressions on Full-Body Gaussian Avatars Zhijing Shao D. B. Wang Qing-Yao Tian Yao-Dong Yang Hengyu Meng Zeyu Cai Bo Dong Yu Zhang Kang Zhang Z. Wang 3DGS 42 3 0 20 Aug 2024
Combo: Co-speech holistic 3D human motion generation and efficient customizable adaptation in harmony Chao Xu Mingze Sun Zhi-Qi Cheng Fei-Yue Wang Yang Liu Baigui Sun Ruqi Huang Alexander G. Hauptmann VGen 37 2 0 18 Aug 2024
FD2Talk: Towards Generalized Talking Head Generation with Facial Decoupled Diffusion Model Ziyu Yao Xuxin Cheng Zhiqi Huang DiffM 19 3 0 18 Aug 2024
Content and Style Aware Audio-Driven Facial Animation Qingju Liu Hyeongwoo Kim Gaurav Bharaj DiffM 32 1 0 13 Aug 2024
ReSyncer: Rewiring Style-based Generator for Unified Audio-Visually Synced Facial Performer Jiazhi Guan Zhiliang Xu Hang Zhou Kaisiyuan Wang Shengyi He ... Errui Ding Jingtuo Liu Jingdong Wang Youjian Zhao Ziwei Liu VGen 51 2 0 06 Aug 2024
JambaTalk: Speech-Driven 3D Talking Head Generation Based on Hybrid Transformer-Mamba Language Model Farzaneh Jafari Stefano Berretti Anup Basu Mamba 31 1 0 03 Aug 2024
GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer Yihong Lin Zhaoxin Fan Lingyu Xiong Liang Peng Xiandong Li Wenxiong Kang Xianjia Wu Songju Lei Huang Xu 39 3 0 03 Aug 2024
UniTalker: Scaling up Audio-Driven 3D Facial Animation through A Unified Model Xiangyu Fan Jiaqi Li Zhiqian Lin Weiye Xiao Lei Yang CVBM VGen 31 3 0 01 Aug 2024
The impact of differences in facial features between real speakers and 3D face models on synthesized lip motions Rabab Algadhy Yoshihiko Gotoh Steve Maddock CVBM 25 0 0 24 Jul 2024
EmoFace: Audio-driven Emotional 3D Face Animation Chang Liu Qunfen Lin Zijiao Zeng Ye Pan CVBM 44 4 0 17 Jul 2024
LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control Jianzhu Guo Dingyun Zhang Xiaoqiang Liu Zhizhou Zhong Yuan Zhang Pengfei Wan Di Zhang VGen 61 53 0 03 Jul 2024
Enhancing Speech-Driven 3D Facial Animation with Audio-Visual Guidance from Lip Reading Expert Han EunGi Oh Hyun-Bin Kim Sung-Bin Corentin Nivelet Etcheberry Suekyeong Nam Janghoon Joo Tae-Hyun Oh 23 5 0 01 Jul 2024
MultiTalk: Enhancing 3D Talking Head Generation Across Languages with Multilingual Video Dataset Kim Sung-Bin Lee Chae-Yeon Gihun Son Oh Hyun-Bin Janghoon Ju Suekyeong Nam Tae-Hyun Oh 34 11 0 20 Jun 2024
A Comprehensive Taxonomy and Analysis of Talking Head Synthesis: Techniques for Portrait Generation, Driving Mechanisms, and Editing Ming Meng Yufei Zhao Bo Zhang Yonggui Zhu Weimin Shi Maxwell Wen Zhaoxin Fan VGen 39 1 0 15 Jun 2024
PianoMotion10M: Dataset and Benchmark for Hand Motion Generation in Piano Performance Qijun Gan Song Wang Shengtao Wu Jianke Zhu 54 1 0 13 Jun 2024
Emotional Conversation: Empowering Talking Faces with Cohesive Expression, Gaze and Pose Generation Jiadong Liang Feng Lu CVBM 34 0 0 12 Jun 2024
DNPM: A Neural Parametric Model for the Synthesis of Facial Geometric Details Haitao Cao Baoping Cheng Qiran Pu Haocheng Zhang Bin Luo Yixiang Zhuang Juncong Lin Liyan Chen Xuan Cheng 3DH 38 0 0 30 May 2024
InterAct: Capture and Modelling of Realistic, Expressive and Interactive Activities between Two Persons in Daily Scenarios Yinghao Huang Leo Ho Dafei Qin Mingyi Shi Taku Komura VGen 39 1 0 19 May 2024
AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encoding Tao Liu Feilong Chen Shuai Fan Chenpeng Du Qi Chen Xie Chen Kai Yu DiffM PINN 36 25 0 06 May 2024
Embedded Representation Learning Network for Animating Styled Video Portrait Tianyong Wang Xiangyu Liang Wangguandong Zheng Dan Niu Haifeng Xia Siyu Xia 3DH 21 0 0 29 Apr 2024
CSTalk: Correlation Supervised Speech-driven 3D Emotional Facial Animation Generation Xiangyu Liang Wenlin Zhuang Tianyong Wang Guangxing Geng Guangyue Geng Haifeng Xia Siyu Xia CVBM 33 1 0 29 Apr 2024
Learn2Talk: 3D Talking Face Learns from 2D Talking Face Yixiang Zhuang Baoping Cheng Yao Cheng Yuntao Jin Renshuai Liu Chengyang Li Xuan Cheng Jing Liao Juncong Lin CVBM 3DH 31 6 0 19 Apr 2024
VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time Sicheng Xu Guojun Chen Yu-Xiao Guo Jiaolong Yang Chong Li Zhenyu Zang Yizhong Zhang Xin Tong Baining Guo 45 86 0 16 Apr 2024