Analyzing Input and Output Representations for Speech-Driven Gesture Generation

8 March 2019

Papers citing "Analyzing Input and Output Representations for Speech-Driven Gesture Generation"

50 / 60 papers shown

Title
ReCoM: Realistic Co-Speech Motion Generation with Recurrent Embedded Transformer Yong Xie Yunlian Sun Hongwen Zhang Y. Liu Jinhui Tang VGen 103 0 0 27 Mar 2025
Cosh-DiT: Co-Speech Gesture Video Synthesis via Hybrid Audio-Visual Diffusion Transformers Yasheng Sun Zhiliang Xu Hang Zhou Jiazhi Guan Quanwei Yang ... Yingying Li Haocheng Feng Jie Wang Ziwei Liu Koike Hideki VGen 61 0 0 13 Mar 2025
Maximizing Signal in Human-Model Preference Alignment Kelsey Kraus Margaret Kroll ALM 55 0 0 06 Mar 2025
SyncViolinist: Music-Oriented Violin Motion Generation Based on Bowing and Fingering Hiroki Nishizawa Keitaro Tanaka Asuka Hirata Shugo Yamaguchi Qi Feng Masatoshi Hamanaka Shigeo Morishima 72 0 0 11 Dec 2024
Multi-Resolution Generative Modeling of Human Motion from Limited Data David Eduardo Moreno-Villamarín Anna Hilsmann Peter Eisert DiffM 3DH 91 0 0 25 Nov 2024
Allo-AVA: A Large-Scale Multimodal Conversational AI Dataset for Allocentric Avatar Gesture Animation Saif Punjwani Larry Heck SLR VGen 35 0 0 21 Oct 2024
LLM Gesticulator: Leveraging Large Language Models for Scalable and Controllable Co-Speech Gesture Synthesis Haozhou Pang Tianwei Ding Lanshan He Ming Tao Lu Zhang Qi Gan 31 1 0 06 Oct 2024
Learning Co-Speech Gesture Representations in Dialogue through Contrastive Learning: An Intrinsic Evaluation E. Ghaleb Bulat Khaertdinov Wim Pouw Marlou Rasenberg Judith Holler Aslı Özyürek Raquel Fernández SSL 33 1 0 31 Aug 2024
M2D2M: Multi-Motion Generation from Text with Discrete Diffusion Models Seung-geun Chi Hyung-Gun Chi Hengbo Ma Nakul Agarwal Faizan Siddiqui Karthik Ramani Kwonjoon Lee DiffM 46 11 0 19 Jul 2024
Aligning Human Motion Generation with Human Perceptions Haoru Wang Wentao Zhu Luyi Miao Yishu Xu Feng Gao Qi Tian Yizhou Wang EGVM 73 1 0 02 Jul 2024
CLHOP: Combined Audio-Video Learning for Horse 3D Pose and Shape Estimation Ci Li Elin Hernlund Hedvig Kjellström Silvia Zuffi 3DH 42 2 0 01 Jul 2024
LLAniMAtion: LLAMA Driven Gesture Animation John T. Windle Iain Matthews Sarah Taylor 43 0 0 13 May 2024
Leveraging Speech for Gesture Detection in Multimodal Communication E. Ghaleb I. Burenko Marlou Rasenberg Wim Pouw Ivan Toni Peter Uhrig Anna Wilson Judith Holler Asli Ozyurek Raquel Fernández SLR 30 4 0 23 Apr 2024
Large Motion Model for Unified Multi-Modal Motion Generation Mingyuan Zhang Daisheng Jin Chenyang Gu Fangzhou Hong Zhongang Cai ... Chongzhi Zhang Xinying Guo Lei Yang Ying He Ziwei Liu VGen 60 25 0 01 Apr 2024
Towards Variable and Coordinated Holistic Co-Speech Motion Generation Yifei Liu Qiong Cao Yandong Wen Huaiguang Jiang Changxing Ding SLR 71 14 0 30 Mar 2024
Freetalker: Controllable Speech and Text-Driven Gesture Generation Based on Diffusion Models for Enhanced Speaker Naturalness Sicheng Yang Zunnan Xu Haiwei Xue Yongkang Cheng Shaoli Huang Biwei Huang Zhiyong Wu DiffM VGen 45 11 0 07 Jan 2024
Pose2Gaze: Eye-body Coordination during Daily Activities for Gaze Prediction from Full-body Poses Zhiming Hu Jiahui Xu Syn Schmitt Andreas Bulling CVBM 20 6 0 19 Dec 2023
SpeechAct: Towards Generating Whole-body Motion from Speech Jinsong Zhang Minjie Zhu Yuxiang Zhang Yebin Liu Kun Li 40 0 0 29 Nov 2023
ACT2G: Attention-based Contrastive Learning for Text-to-Gesture Generation Hitoshi Teshima Naoki Wake Diego Thomas Yuta Nakashima Hiroshi Kawasaki Katsushi Ikeuchi 32 0 0 28 Sep 2023
Speech-Gesture GAN: Gesture Generation for Robots and Embodied Agents Carson Yu Liu Gelareh Mohammadi Yang Song W. Johal 15 2 0 17 Sep 2023
Towards the generation of synchronized and believable non-verbal facial behaviors of a talking virtual agent Alice Delbosc M. Ochs Nicolas Sabouret Brian Ravenet Stéphane Ayache 40 7 0 15 Sep 2023
UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons Sicheng Yang Zehao Wang Zhiyong Wu Minglei Li Zhensong Zhang ... Lei Hao Songcen Xu Xiaofei Wu Changpeng Yang Zonghong Dai DiffM 54 14 0 13 Sep 2023
Diffusion-Based Co-Speech Gesture Generation Using Joint Text and Audio Representation Anna Deichler Shivam Mehta Simon Alexanderson Jonas Beskow DiffM 23 23 0 11 Sep 2023
BodyFormer: Semantics-guided 3D Body Gesture Synthesis with Transformer Kunkun Pang Dafei Qin Yingruo Fan Julian Habekost Takaaki Shiratori Junichi Yamagishi Taku Komura SLR ViT 26 19 0 07 Sep 2023
C2G2: Controllable Co-speech Gesture Generation with Latent Diffusion Model Longbin Ji Pengfei Wei Yi Ren Jinglin Liu Chen Zhang Xiang Yin DiffM 42 3 0 29 Aug 2023
The DiffuseStyleGesture+ entry to the GENEA Challenge 2023 Sicheng Yang Haiwei Xue Zhensong Zhang Minglei Li Zhiyong Wu Xiaofei Wu Songcen Xu Zonghong Dai DiffM 37 15 0 26 Aug 2023
Human Motion Generation: A Survey Wentao Zhu Xiaoxuan Ma Dongwoo Ro Hai Ci Jinlu Zhang Jiaxin Shi Feng Gao Qi Tian Yizhou Wang VGen 47 53 0 20 Jul 2023
ZS-MSTM: Zero-Shot Style Transfer for Gesture Animation driven by Text and Speech using Adversarial Disentanglement of Multimodal Style Encoding Mireille Fares Catherine Pelachaud Nicolas Obin 22 0 0 22 May 2023
QPGesture: Quantization-Based and Phase-Guided Motion Matching for Natural Speech-Driven Gesture Generation Sicheng Yang Zhiyong Wu Minglei Li Zhensong Zhang Lei Hao Weihong Bao Hao-Wen Zhuang SLR 26 41 0 18 May 2023
Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022 Taras Kucherenko Pieter Wolfert Youngwoo Yoon Carla Viegas Teodor Nikolov Mihail Tsakov G. Henter 37 24 0 15 Mar 2023
A Comprehensive Review of Data-Driven Co-Speech Gesture Generation Simbarashe Nyatsanga Taras Kucherenko Chaitanya Ahuja G. Henter Michael Neff SLR 44 90 0 13 Jan 2023
Generating Holistic 3D Human Motion from Speech Hongwei Yi Hualin Liang Yifei Liu Qiong Cao Yandong Wen Timo Bolkart Dacheng Tao Michael J. Black SLR 31 144 0 08 Dec 2022
Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models Simon Alexanderson Rajmund Nagy Jonas Beskow G. Henter DiffM VGen 24 166 0 17 Nov 2022
Deep Gesture Generation for Social Robots Using Type-Specific Libraries Hitoshi Teshima Naoki Wake Diego Thomas Yuta Nakashima Hiroshi Kawasaki Katsushi Ikeuchi SLR 25 7 0 13 Oct 2022
Rhythmic Gesticulator: Rhythm-Aware Co-Speech Gesture Synthesis with Hierarchical Neural Embeddings Tenglong Ao Qingzhe Gao Yuke Lou Baoquan Chen Libin Liu SLR 32 59 0 04 Oct 2022
The ReprGesture entry to the GENEA Challenge 2022 Sicheng Yang Zhiyong Wu Minglei Li Mengchen Zhao Jiuxin Lin Liyang Chen Weihong Bao 33 11 0 25 Aug 2022
The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation Youngwoo Yoon Pieter Wolfert Taras Kucherenko Carla Viegas Teodor Nikolov Mihail Tsakov G. Henter VGen 37 81 0 22 Aug 2022
Learning in Audio-visual Context: A Review, Analysis, and New Perspective Yake Wei Di Hu Yapeng Tian Xuelong Li 46 55 0 20 Aug 2022
Zero-Shot Style Transfer for Gesture Animation driven by Text and Speech using Adversarial Disentanglement of Multimodal Style Encoding Mireille Fares Michele Grimaldi Catherine Pelachaud Nicolas Obin 32 17 0 03 Aug 2022
A Probabilistic Model Of Interaction Dynamics for Dyadic Face-to-Face Settings Renke Wang Ifeoma Nwogu CVBM 17 0 0 10 Jul 2022
Rhythm is a Dancer: Music-Driven Motion Synthesis with Global Structure A. Aristidou Anastasios Yiannakidis Kfir Aberman Daniel Cohen-Or Ariel Shamir Y. Chrysanthou 40 74 0 23 Nov 2021
Speech Drives Templates: Co-Speech Gesture Synthesis with Learned Templates Shenhan Qian Zhi Tu Yihao Zhi Wen Liu Shenghua Gao SLR 18 71 0 18 Aug 2021
SGToolkit: An Interactive Gesture Authoring Toolkit for Embodied Conversational Agents Youngwoo Yoon Keunwoo Park Minsu Jang Jaehong Kim Geehyuk Lee VGen SLR 39 19 0 10 Aug 2021
Speech2AffectiveGestures: Synthesizing Co-Speech Gestures with Generative Adversarial Affective Expression Learning Uttaran Bhattacharya Elizabeth Childs Nicholas Rewkowski Tianyi Zhou SLR GAN 18 80 0 31 Jul 2021
Improving Ultrasound Tongue Image Reconstruction from Lip Images Using Self-supervised Learning and Attention Mechanism Haiyang Liu Jihang Zhang 21 4 0 20 Jun 2021
A large, crowdsourced evaluation of gesture generation systems on common data: The GENEA Challenge 2020 Taras Kucherenko Patrik Jonell Youngwoo Yoon Pieter Wolfert G. Henter 24 74 0 23 Feb 2021
Learning Speech-driven 3D Conversational Gestures from Video I. Habibie Weipeng Xu Dushyant Mehta Lingjie Liu Hans-Peter Seidel Gerard Pons-Moll Mohamed A. Elgharib Christian Theobalt SLR CVBM 3DH 40 108 0 13 Feb 2021
HEMVIP: Human Evaluation of Multiple Videos in Parallel Patrik Jonell Youngwoo Yoon Pieter Wolfert Taras Kucherenko G. Henter 15 21 0 28 Jan 2021
Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agents Uttaran Bhattacharya Nicholas Rewkowski A. Banerjee P. Guhan Aniket Bera Tianyi Zhou LM&Ro 18 149 0 26 Jan 2021
Generating coherent spontaneous speech and gesture from text Simon Alexanderson Éva Székely G. Henter Taras Kucherenko Jonas Beskow SLR 37 22 0 14 Jan 2021