MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement

16 April 2021

Papers citing "MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement"

41 / 41 papers shown

Title
OT-Talk: Animating 3D Talking Head with Optimal Transportation Xinmu Wang Xiang Gao Xiyun Song Heather Yu Zongfang Lin Liang Peng Xianfeng Gu 27 0 0 03 May 2025
Model See Model Do: Speech-Driven Facial Animation with Style Control Yifang Pan Karan Singh Luiz Gustavo Hafemann DiffM 55 0 0 02 May 2025
VividListener: Expressive and Controllable Listener Dynamics Modeling for Multi-Modal Responsive Interaction Shiying Li Xingqun Qi Bingkun Yang Chen Weile Zezhao Tian Muyi Sun Qifeng Liu Man Zhang Zhenan Sun 64 0 0 30 Apr 2025
ARTalk: Speech-Driven 3D Head Animation via Autoregressive Model Xuangeng Chu Nabarun Goswami Ziteng Cui Hanqin Wang Tatsuya Harada DiffM 80 0 0 27 Feb 2025
Beyond Fixed Topologies: Unregistered Training and Comprehensive Evaluation Metrics for 3D Talking Heads Federico Nocentini T. Besnier Claudio Ferrari Sylvain Arguillere Stefano Berretti Mohamed Daoudi 59 1 0 14 Oct 2024
3DFacePolicy: Speech-Driven 3D Facial Animation with Diffusion Policy Xuanmeng Sha Liyun Zhang Tomohiro Mashita Yuki Uranishi VGen 27 0 0 17 Sep 2024
ProbTalk3D: Non-Deterministic Emotion Controllable Speech-Driven 3D Facial Animation Synthesis Using VQ-VAE Sichun Wu Kazi Injamamul Haque Zerrin Yumak VGen 30 2 0 12 Sep 2024
DEEPTalk: Dynamic Emotion Embedding for Probabilistic Speech-Driven 3D Face Animation Jisoo Kim Jungbin Cho Joonho Park Soonmin Hwang Da Eun Kim Geon Kim Youngjae Yu 57 1 0 12 Aug 2024
GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer Yihong Lin Zhaoxin Fan Lingyu Xiong Liang Peng Xiandong Li Xiandong Li Wenxiong Kang Xiandong Li Huang Xu 42 3 0 03 Aug 2024
InterAct: Capture and Modelling of Realistic, Expressive and Interactive Activities between Two Persons in Daily Scenarios Yinghao Huang Leo Ho Dafei Qin Mingyi Shi Taku Komura VGen 50 1 0 19 May 2024
Embedded Representation Learning Network for Animating Styled Video Portrait Tianyong Wang Xiangyu Liang Wangguandong Zheng Dan Niu Haifeng Xia Siyu Xia 3DH 29 0 0 29 Apr 2024
Learn2Talk: 3D Talking Face Learns from 2D Talking Face Yixiang Zhuang Baoping Cheng Yao Cheng Yuntao Jin Renshuai Liu Chengyang Li Xuan Cheng Jing Liao Juncong Lin CVBM 3DH 37 6 0 19 Apr 2024
FSRT: Facial Scene Representation Transformer for Face Reenactment from Factorized Appearance, Head-pose, and Facial Expression Features Andre Rochow Max Schwarz Sven Behnke ViT 48 6 0 15 Apr 2024
SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesis George Retsinas P. Filntisis Radek Daněček Victoria Fernandez-Abrevaya A. Roussos Timo Bolkart Petros Maragos 3DH 67 30 0 05 Apr 2024
VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis Enric Corona Andrei Zanfir Eduard Gabriel Bazavan Nikos Kolotouros Thiemo Alldieck C. Sminchisescu VGen DiffM 43 26 0 13 Mar 2024
CustomListener: Text-guided Responsive Interaction for User-friendly Listening Head Generation Xi Liu Ying Guo Cheng Zhen Tong Li Yingying Ao Pengfei Yan DiffM 34 3 0 01 Mar 2024
DiffSpeaker: Speech-Driven 3D Facial Animation with Diffusion Transformer Zhiyuan Ma Xiangyu Zhu Guojun Qi Chen Qian Zhaoxiang Zhang Zhen Lei 24 9 0 08 Feb 2024
GMTalker: Gaussian Mixture-based Audio-Driven Emotional Talking Video Portraits Yibo Xia Lizhen Wang Xiang Deng Xiaoyan Luo Yunhong Wang Yebin Liu VGen 45 1 0 12 Dec 2023
AdaMesh: Personalized Facial Expressions and Head Poses for Adaptive Speech-Driven 3D Facial Animation Liyang Chen Weihong Bao Shunwei Lei Boshi Tang Zhiyong Wu Shiyin Kang Haozhi Huang Helen M. Meng 42 1 0 11 Oct 2023
ExpCLIP: Bridging Text and Facial Expressions via Semantic Alignment Yicheng Zhong Huawei Wei Pei-Yin Yang Zhisheng Wang CLIP 37 6 0 28 Aug 2023
UniBriVL: Robust Universal Representation and Generation of Audio Driven Diffusion Models Sen Fang Bowen Gao Yangjian Wu T. Teoh DiffM 34 1 0 29 Jul 2023
Audio-Driven 3D Facial Animation from In-the-Wild Videos Liying Lu Tianke Zhang Yunfei Liu Xuangeng Chu Yu Li VGen 47 3 0 20 Jun 2023
Emotional Speech-Driven Animation with Content-Emotion Disentanglement Radek Danvevcek Kiran Chhatre Shashank Tripathi Yandong Wen Michael J. Black Timo Bolkart 16 66 0 15 Jun 2023
An Android Robot Head as Embodied Conversational Agent Marcel Heisler C. Becker-Asano LM&Ro LLMAG 29 0 0 18 May 2023
EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation Ziqiao Peng Hao Wu Zhenbo Song Hao-Xuan Xu Xiangyu Zhu Jun He Hongyan Liu Zhaoxin Fan CVBM 33 99 0 20 Mar 2023
Exploring Efficient-Tuned Learning Audio Representation Method from BriVL Sen Fang Yang Wu Bowen Gao Jingwen Cai T. Teoh DiffM 29 1 0 08 Mar 2023
Expressive Speech-driven Facial Animation with controllable emotions Yutong Chen Junhong Zhao Weiqiang Zhang 33 8 0 05 Jan 2023
Imitator: Personalized Speech-driven 3D Facial Animation Balamurugan Thambiraja I. Habibie S. Aliakbarian Darren Cosker Christian Theobalt Justus Thies CVBM 47 49 0 30 Dec 2022
Generating Holistic 3D Human Motion from Speech Hongwei Yi Hualin Liang Yifei Liu Qiong Cao Yandong Wen Timo Bolkart Dacheng Tao Michael J. Black SLR 31 143 0 08 Dec 2022
Audio-Driven Co-Speech Gesture Video Generation Xian Liu Qianyi Wu Hang Zhou Yuanqi Du Wayne Wu Dahua Lin Ziwei Liu SLR VGen 36 47 0 05 Dec 2022
Rhythmic Gesticulator: Rhythm-Aware Co-Speech Gesture Synthesis with Hierarchical Neural Embeddings Tenglong Ao Qingzhe Gao Yuke Lou Baoquan Chen Libin Liu SLR 27 59 0 04 Oct 2022
Multiface: A Dataset for Neural Face Rendering Cheng-hsin Wuu N. Zheng Scott Ardisson Rohan Bali Danielle Belko ... Xinshuo Weng David Whitewolf Chenglei Wu Shoou-I Yu Yaser Sheikh 3DH CVBM 39 77 0 22 Jul 2022
u-HuBERT: Unified Mixed-Modal Speech Pretraining And Zero-Shot Transfer to Unlabeled Modality Wei-Ning Hsu Bowen Shi SSL VLM 27 41 0 14 Jul 2022
TEMOS: Generating diverse human motions from textual descriptions Mathis Petrovich Michael J. Black Gül Varol 40 372 0 25 Apr 2022
Audio-Visual Speech Codecs: Rethinking Audio-Visual Speech Enhancement by Re-Synthesis Karren D. Yang Dejan Marković Steven Krenn Vasu Agrawal Alexander Richard VGen 16 32 0 31 Mar 2022
FLAG: Flow-based 3D Avatar Generation from Sparse Observations S. Aliakbarian Pashmina Cameron Federica Bogo Andrew Fitzgibbon T. Cashman 3DH 20 55 0 11 Mar 2022
Responsive Listening Head Generation: A Benchmark Dataset and Baseline Mohan Zhou Yalong Bai Wei Zhang Ting Yao T. Zhao Tao Mei EGVM 30 45 0 27 Dec 2021
FaceFormer: Speech-Driven 3D Facial Animation with Transformers Yingruo Fan Zhaojiang Lin Jun Saito Wenping Wang Taku Komura CVBM 49 195 0 10 Dec 2021
Joint Audio-Text Model for Expressive Speech-Driven 3D Facial Animation Yingruo Fan Zhaojiang Lin Jun Saito Wenping Wang Taku Komura 31 21 0 04 Dec 2021
Neural Dubber: Dubbing for Videos According to Scripts Chenxu Hu Qiao Tian Tingle Li Yuping Wang Yuxuan Wang Hang Zhao DiffM VGen 36 39 0 15 Oct 2021
Learning an Animatable Detailed 3D Face Model from In-The-Wild Images Yao Feng Haiwen Feng Michael J. Black Timo Bolkart CVBM 3DH 41 569 0 07 Dec 2020