Speech Gesture Generation from the Trimodal Context of Text, Audio, and Speaker Identity

4 September 2020

Papers citing "Speech Gesture Generation from the Trimodal Context of Text, Audio, and Speaker Identity"

50 / 62 papers shown

Title
AsynFusion: Towards Asynchronous Latent Consistency Models for Decoupled Whole-Body Audio-Driven Avatars T. Zhang Jian Zhao Yuer Li Zheng Zhu Ping Hu Zhaoxin Fan Wenjun Wu Xuelong Li 21 0 0 21 May 2025
M3G: Multi-Granular Gesture Generator for Audio-Driven Full-Body Human Motion Synthesis Zhizhuo Yin Yuk Hang Tsui Pan Hui SLR VGen 24 0 0 13 May 2025
Inter-Diffusion Generation Model of Speakers and Listeners for Effective Communication Jinhe Huang Yongkang Cheng Yuming Hang Gaoge Han Jiajian Li Jing Zhang Xingjian Gu 53 0 0 08 May 2025
$Co$^{3}$Gesture: Towards Coherent Concurrent Co-speech 3D Gesture Generation with Interactive Diffusion$ Co $^{3}$ Gesture: Towards Coherent Concurrent Co-speech 3D Gesture Generation with Interactive Diffusion Xingqun Qi Yatian Wang Hengyuan Zhang J. Pan Wei Xue Shanghang Zhang Wenhan Luo Qifeng Liu Yike Guo SLR 66 0 0 03 May 2025
EchoMask: Speech-Queried Attention-based Mask Modeling for Holistic Co-Speech Motion Generation Xiangyue Zhang Jianfang Li Jiaxu Zhang Jianqiang Ren Liefeng Bo Zhigang Tu 37 0 0 12 Apr 2025
ExGes: Expressive Human Motion Retrieval and Modulation for Audio-Driven Gesture Synthesis Xukun Zhou Fengxin Li Ming Chen Yan Zhou Pengfei Wan Di Zhang Yeying Jin Zhaoxin Fan Hongyan Liu Jun He DiffM VGen 56 0 0 09 Mar 2025
Gesture Generation from Trimodal Context for Humanoid Robots Shiyi Tang Christian Dondrup 32 0 0 08 Sep 2024
DEEPTalk: Dynamic Emotion Embedding for Probabilistic Speech-Driven 3D Face Animation Jisoo Kim Jungbin Cho Joonho Park Soonmin Hwang Da Eun Kim Geon Kim Youngjae Yu 62 1 0 12 Aug 2024
Investigating the impact of 2D gesture representation on co-speech gesture generation Teo Guichoux Laure Soulier Nicolas Obin Catherine Pelachaud SLR 21 0 0 21 Jun 2024
PianoMotion10M: Dataset and Benchmark for Hand Motion Generation in Piano Performance Qijun Gan Song Wang Shengtao Wu Jianke Zhu 62 1 0 13 Jun 2024
CoCoGesture: Toward Coherent Co-speech 3D Gesture Generation in the Wild Xingqun Qi Hengyuan Zhang Yatian Wang J. Pan Chen Liu ... Qixun Zhang Shanghang Zhang Wenhan Luo Qifeng Liu Qi-fei Liu DiffM SLR 115 5 0 27 May 2024
LLAniMAtion: LLAMA Driven Gesture Animation John T. Windle Iain Matthews Sarah Taylor 43 0 0 13 May 2024
Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesis Shivam Mehta Anna Deichler Jim O'Regan Birger Moëll Jonas Beskow G. Henter Simon Alexanderson 51 4 0 30 Apr 2024
Bridge to Non-Barrier Communication: Gloss-Prompted Fine-grained Cued Speech Gesture Generation with Diffusion Model Wen-Ling Lei Li Liu Jun Wang DiffM 43 2 0 30 Apr 2024
MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models Zunnan Xu Yukang Lin Haonan Han Sicheng Yang Ronghui Li Yachao Zhang Xiu Li Mamba 51 25 0 14 Mar 2024
Cascaded Cross-Modal Transformer for Audio-Textual Classification Nicolae-Cătălin Ristea Andrei Anghel Radu Tudor Ionescu 36 2 0 15 Jan 2024
Freetalker: Controllable Speech and Text-Driven Gesture Generation Based on Diffusion Models for Enhanced Speaker Naturalness Sicheng Yang Zunnan Xu Haiwei Xue Yongkang Cheng Shaoli Huang Biwei Huang Zhiyong Wu DiffM VGen 45 11 0 07 Jan 2024
Weakly-Supervised Emotion Transition Learning for Diverse 3D Co-speech Gesture Generation Xingqun Qi Jiahao Pan Peng Li Ruibin Yuan Xiaowei Chi ... Wenhan Luo Wei Xue Shanghang Zhang Qi-fei Liu Yi-Ting Guo SLR 39 11 0 29 Nov 2023
SpeechAct: Towards Generating Whole-body Motion from Speech Jinsong Zhang Minjie Zhu Yuxiang Zhang Yebin Liu Kun Li 45 0 0 29 Nov 2023
Large language models in textual analysis for gesture selection Laura Birka Hensel Nutchanon Yongsatianchot P. Torshizi E. Minucci Stacy Marsella SLR 38 7 0 04 Oct 2023
Autoregressive Sign Language Production: A Gloss-Free Approach with Discrete Representations Eui Jun Hwang Huije Lee Jong C. Park SLR 47 6 0 21 Sep 2023
UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons Sicheng Yang Zehao Wang Zhiyong Wu Minglei Li Zhensong Zhang ... Lei Hao Songcen Xu Xiaofei Wu Changpeng Yang Zonghong Dai DiffM 54 14 0 13 Sep 2023
BodyFormer: Semantics-guided 3D Body Gesture Synthesis with Transformer Kunkun Pang Dafei Qin Yingruo Fan Julian Habekost Takaaki Shiratori Junichi Yamagishi Taku Komura SLR ViT 28 19 0 07 Sep 2023
AI-Generated Content (AIGC) for Various Data Modalities: A Survey Lin Geng Foo Hossein Rahmani Jing Liu 78 31 0 27 Aug 2023
TranSTYLer: Multimodal Behavioral Style Transfer for Facial and Body Gestures Generation Mireille Fares Catherine Pelachaud Nicolas Obin 47 1 0 08 Aug 2023
Human Motion Generation: A Survey Wentao Zhu Xiaoxuan Ma Dongwoo Ro Hai Ci Jinlu Zhang Jiaxin Shi Feng Gao Qi Tian Yizhou Wang VGen 52 53 0 20 Jul 2023
Augmented Co-Speech Gesture Generation: Including Form and Meaning Features to Guide Learning-Based Gesture Synthesis Hendric Voss S. Kopp SLR 54 4 0 13 Jul 2023
EMoG: Synthesizing Emotive Co-speech 3D Gesture with Diffusion Model Li-Ping Yin Yijun Wang Tianyu He Jinming Liu Wei Zhao Bohan Li Xin Jin Jianxin Lin DiffM 37 14 0 20 Jun 2023
MPE4G: Multimodal Pretrained Encoder for Co-Speech Gesture Generation Gwantae Kim Seong-Soo Noh Insung Ham Hanseok Ko SLR 22 7 0 25 May 2023
QPGesture: Quantization-Based and Phase-Guided Motion Matching for Natural Speech-Driven Gesture Generation Sicheng Yang Zhiyong Wu Minglei Li Zhensong Zhang Lei Hao Weihong Bao Hao-Wen Zhuang SLR 26 41 0 18 May 2023
GesGPT: Speech Gesture Synthesis With Text Parsing from ChatGPT Nan Gao Zeyu Zhao Zhi Zeng Shuwu Zhang Dongdong Weng Yihua Bao 45 8 0 23 Mar 2023
Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generation Lingting Zhu Xian Liu Xuanyu Liu Rui Qian Ziwei Liu Lequan Yu 36 115 0 16 Mar 2023
Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022 Taras Kucherenko Pieter Wolfert Youngwoo Yoon Carla Viegas Teodor Nikolov Mihail Tsakov G. Henter 37 24 0 15 Mar 2023
A Comprehensive Review of Data-Driven Co-Speech Gesture Generation Simbarashe Nyatsanga Taras Kucherenko Chaitanya Ahuja G. Henter Michael Neff SLR 44 90 0 13 Jan 2023
Generating Holistic 3D Human Motion from Speech Hongwei Yi Hualin Liang Yifei Liu Qiong Cao Yandong Wen Timo Bolkart Dacheng Tao Michael J. Black SLR 34 145 0 08 Dec 2022
Audio-Driven Co-Speech Gesture Video Generation Xian Liu Qianyi Wu Hang Zhou Yuanqi Du Wayne Wu Dahua Lin Ziwei Liu SLR VGen 39 49 0 05 Dec 2022
Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models Simon Alexanderson Rajmund Nagy Jonas Beskow G. Henter DiffM VGen 29 166 0 17 Nov 2022
Deep Gesture Generation for Social Robots Using Type-Specific Libraries Hitoshi Teshima Naoki Wake Diego Thomas Yuta Nakashima Hiroshi Kawasaki Katsushi Ikeuchi SLR 36 7 0 13 Oct 2022
Rhythmic Gesticulator: Rhythm-Aware Co-Speech Gesture Synthesis with Hierarchical Neural Embeddings Tenglong Ao Qingzhe Gao Yuke Lou Baoquan Chen Libin Liu SLR 34 59 0 04 Oct 2022
NEURAL MARIONETTE: A Transformer-based Multi-action Human Motion Synthesis System Weiqiang Wang Xuefei Zhe Huan Chen Di Kang Tingguang Li Ruizhi Chen Linchao Bao 56 5 0 27 Sep 2022
ZeroEGGS: Zero-shot Example-based Gesture Generation from Speech Saeed Ghorbani Ylva Ferstl Daniel Holden N. Troje M. Carbonneau 41 79 0 15 Sep 2022
The ReprGesture entry to the GENEA Challenge 2022 Sicheng Yang Zhiyong Wu Minglei Li Mengchen Zhao Jiuxin Lin Liyang Chen Weihong Bao 33 11 0 25 Aug 2022
The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation Youngwoo Yoon Pieter Wolfert Taras Kucherenko Carla Viegas Teodor Nikolov Mihail Tsakov G. Henter VGen 37 81 0 22 Aug 2022
Audio-driven Neural Gesture Reenactment with Video Motion Graphs Yang Zhou Jimei Yang Dingzeyu Li Jun Saito Deepali Aneja E. Kalogerakis DiffM SLR 42 20 0 23 Jul 2022
Interaction Transformer for Human Reaction Generation Baptiste Chopin Hao Tang N. Otberdout Mohamed Daoudi N. Sebe ViT 38 27 0 04 Jul 2022
Analysis of Co-Laughter Gesture Relationship on RGB videos in Dyadic Conversation Contex Hugo Bohy Ahmad Hammoudeh Antoine Maiorca Stéphane Dupont Thierry Dutoit 19 2 0 20 May 2022
Evaluating the Quality of a Synthesized Motion with the Fréchet Motion Distance Antoine Maiorca Youngwoo Yoon Thierry Dutoit 14 9 0 26 Apr 2022
Speaker Extraction with Co-Speech Gestures Cue Zexu Pan Xinyuan Qian Haizhou Li SLR 31 27 0 31 Mar 2022
Freeform Body Motion Generation from Speech Jing-Fen Xu Wei Zhang Yalong Bai Qi-Biao Sun Tao Mei SLR 41 18 0 04 Mar 2022
Joint Audio-Text Model for Expressive Speech-Driven 3D Facial Animation Yingruo Fan Zhaojiang Lin Jun Saito Wenping Wang Taku Komura 36 21 0 04 Dec 2021