Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robots

30 October 2018

Papers citing "Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robots"

44 / 44 papers shown

Title
CoCoGesture: Toward Coherent Co-speech 3D Gesture Generation in the Wild Xingqun Qi Hengyuan Zhang Yatian Wang J. Pan Chen Liu ... Qixun Zhang Shanghang Zhang Wenhan Luo Qifeng Liu Qi-fei Liu DiffM SLR 115 5 0 27 May 2024
Bridge to Non-Barrier Communication: Gloss-Prompted Fine-grained Cued Speech Gesture Generation with Diffusion Model Wen-Ling Lei Li Liu Jun Wang DiffM 38 2 0 30 Apr 2024
MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models Zunnan Xu Yukang Lin Haonan Han Sicheng Yang Ronghui Li Yachao Zhang Xiu Li Mamba 46 25 0 14 Mar 2024
PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interaction Yinhuai Wang Jing Lin Ailing Zeng Zhengyi Luo Jian Zhang Lei Zhang 32 36 0 07 Dec 2023
Weakly-Supervised Emotion Transition Learning for Diverse 3D Co-speech Gesture Generation Xingqun Qi Jiahao Pan Peng Li Ruibin Yuan Xiaowei Chi ... Wenhan Luo Wei Xue Shanghang Zhang Qi-fei Liu Yi-Ting Guo SLR 37 11 0 29 Nov 2023
SpeechAct: Towards Generating Whole-body Motion from Speech Jinsong Zhang Minjie Zhu Yuxiang Zhang Yebin Liu Kun Li 33 0 0 29 Nov 2023
Large language models in textual analysis for gesture selection Laura Birka Hensel Nutchanon Yongsatianchot P. Torshizi E. Minucci Stacy Marsella SLR 35 7 0 04 Oct 2023
Autoregressive Sign Language Production: A Gloss-Free Approach with Discrete Representations Eui Jun Hwang Huije Lee Jong C. Park SLR 42 6 0 21 Sep 2023
UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons Sicheng Yang Zehao Wang Zhiyong Wu Minglei Li Zhensong Zhang ... Lei Hao Songcen Xu Xiaofei Wu Changpeng Yang Zonghong Dai DiffM 49 14 0 13 Sep 2023
Diffusion-Based Co-Speech Gesture Generation Using Joint Text and Audio Representation Anna Deichler Shivam Mehta Simon Alexanderson Jonas Beskow DiffM 20 23 0 11 Sep 2023
BodyFormer: Semantics-guided 3D Body Gesture Synthesis with Transformer Kunkun Pang Dafei Qin Yingruo Fan Julian Habekost Takaaki Shiratori Junichi Yamagishi Taku Komura SLR ViT 21 19 0 07 Sep 2023
Audio is all in one: speech-driven gesture synthetics using WavLM pre-trained model Fan Zhang Naye Ji Fuxing Gao Siyuan Zhao Zhaohan Wang Shunman Li 32 0 0 11 Aug 2023
Human Motion Generation: A Survey Wentao Zhu Xiaoxuan Ma Dongwoo Ro Hai Ci Jinlu Zhang Jiaxin Shi Feng Gao Qi Tian Yizhou Wang VGen 47 53 0 20 Jul 2023
EMoG: Synthesizing Emotive Co-speech 3D Gesture with Diffusion Model Li-Ping Yin Yijun Wang Tianyu He Jinming Liu Wei Zhao Bohan Li Xin Jin Jianxin Lin DiffM 29 14 0 20 Jun 2023
MPE4G: Multimodal Pretrained Encoder for Co-Speech Gesture Generation Gwantae Kim Seong-Soo Noh Insung Ham Hanseok Ko SLR 17 7 0 25 May 2023
QPGesture: Quantization-Based and Phase-Guided Motion Matching for Natural Speech-Driven Gesture Generation Sicheng Yang Zhiyong Wu Minglei Li Zhensong Zhang Lei Hao Weihong Bao Hao-Wen Zhuang SLR 26 41 0 18 May 2023
Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generation Lingting Zhu Xian Liu Xuanyu Liu Rui Qian Ziwei Liu Lequan Yu 31 114 0 16 Mar 2023
Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022 Taras Kucherenko Pieter Wolfert Youngwoo Yoon Carla Viegas Teodor Nikolov Mihail Tsakov G. Henter 32 24 0 15 Mar 2023
A Comprehensive Review of Data-Driven Co-Speech Gesture Generation Simbarashe Nyatsanga Taras Kucherenko Chaitanya Ahuja G. Henter Michael Neff SLR 44 90 0 13 Jan 2023
Generating Holistic 3D Human Motion from Speech Hongwei Yi Hualin Liang Yifei Liu Qiong Cao Yandong Wen Timo Bolkart Dacheng Tao Michael J. Black SLR 31 143 0 08 Dec 2022
Audio-Driven Co-Speech Gesture Video Generation Xian Liu Qianyi Wu Hang Zhou Yuanqi Du Wayne Wu Dahua Lin Ziwei Liu SLR VGen 36 47 0 05 Dec 2022
Evaluating Data-Driven Co-Speech Gestures of Embodied Conversational Agents through Real-Time Interaction Yuan He André Pereira Taras Kucherenko 30 10 0 13 Oct 2022
SwarMan: Anthropomorphic Swarm of Drones Avatar with Body Tracking and Deep Learning-Based Gesture Recognition A. Baza Ayush Gupta Ekaterina Dorzhieva A. Fedoseev Dzmitry Tsetserukou 43 4 0 04 Oct 2022
Rhythmic Gesticulator: Rhythm-Aware Co-Speech Gesture Synthesis with Hierarchical Neural Embeddings Tenglong Ao Qingzhe Gao Yuke Lou Baoquan Chen Libin Liu SLR 27 59 0 04 Oct 2022
The ReprGesture entry to the GENEA Challenge 2022 Sicheng Yang Zhiyong Wu Minglei Li Mengchen Zhao Jiuxin Lin Liyang Chen Weihong Bao 33 11 0 25 Aug 2022
Learning in Audio-visual Context: A Review, Analysis, and New Perspective Yake Wei Di Hu Yapeng Tian Xuelong Li 46 55 0 20 Aug 2022
Speaker Extraction with Co-Speech Gestures Cue Zexu Pan Xinyuan Qian Haizhou Li SLR 21 27 0 31 Mar 2022
Freeform Body Motion Generation from Speech Jing-Fen Xu Wei Zhang Yalong Bai Qi-Biao Sun Tao Mei SLR 39 18 0 04 Mar 2022
Integrated Speech and Gesture Synthesis Siyang Wang Simon Alexanderson Joakim Gustafson Jonas Beskow G. Henter Éva Székely 37 19 0 25 Aug 2021
Multimodal analysis of the predictability of hand-gesture properties Taras Kucherenko Rajmund Nagy Michael Neff Hedvig Kjellström G. Henter 34 22 0 12 Aug 2021
SGToolkit: An Interactive Gesture Authoring Toolkit for Embodied Conversational Agents Youngwoo Yoon Keunwoo Park Minsu Jang Jaehong Kim Geehyuk Lee VGen SLR 39 19 0 10 Aug 2021
Speech2AffectiveGestures: Synthesizing Co-Speech Gestures with Generative Adversarial Affective Expression Learning Uttaran Bhattacharya Elizabeth Childs Nicholas Rewkowski Tianyi Zhou SLR GAN 16 80 0 31 Jul 2021
Speech2Properties2Gestures: Gesture-Property Prediction as a Tool for Generating Representational Gestures from Speech Taras Kucherenko Rajmund Nagy Patrik Jonell Michael Neff Hedvig Kjellström G. Henter 22 18 0 28 Jun 2021
Transflower: probabilistic autoregressive dance generation with multimodal attention Guillermo Valle Pérez G. Henter Jonas Beskow A. Holzapfel Pierre-Yves Oudeyer Simon Alexanderson 30 42 0 25 Jun 2021
Toward Automated Generation of Affective Gestures from Text:A Theory-Driven Approach Micol Spitale Maja J. Matarić 25 1 0 04 Mar 2021
A Framework for Integrating Gesture Generation Models into Interactive Conversational Agents Rajmund Nagy Taras Kucherenko Birger Moell André Pereira Hedvig Kjellström Ulysses Bernardet 34 12 0 24 Feb 2021
Learning Speech-driven 3D Conversational Gestures from Video I. Habibie Weipeng Xu Dushyant Mehta Lingjie Liu Hans-Peter Seidel Gerard Pons-Moll Mohamed A. Elgharib Christian Theobalt SLR CVBM 3DH 40 107 0 13 Feb 2021
Generating coherent spontaneous speech and gesture from text Simon Alexanderson Éva Székely G. Henter Taras Kucherenko Jonas Beskow SLR 37 22 0 14 Jan 2021
A Review of Evaluation Practices of Gesture Generation in Embodied Conversational Agents Pieter Wolfert Nicole L. Robinson Tony Belpaeme 49 50 0 11 Jan 2021
Temporally Guided Music-to-Body-Movement Generation Hsuan-Kai Kao Li Su 44 42 0 17 Sep 2020
Body2Hands: Learning to Infer 3D Hands from Conversational Gesture Body Dynamics Evonne Ng Shiry Ginosar Trevor Darrell Hanbyul Joo SLR 3DH 23 45 0 23 Jul 2020
Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation Taras Kucherenko Dai Hasegawa Naoshi Kaneko G. Henter Hedvig Kjellström 13 41 0 16 Jul 2020
Let's Face It: Probabilistic Multi-modal Interlocutor-aware Generation of Facial Gestures in Dyadic Settings Patrik Jonell Taras Kucherenko G. Henter Jonas Beskow CVBM 17 61 0 11 Jun 2020
Analyzing Input and Output Representations for Speech-Driven Gesture Generation Taras Kucherenko Dai Hasegawa G. Henter Naoshi Kaneko Hedvig Kjellström 9 150 0 08 Mar 2019