Diffusion-Based Co-Speech Gesture Generation Using Joint Text and Audio
Representation

Diffusion-Based Co-Speech Gesture Generation Using Joint Text and Audio Representation

11 September 2023

Simon Alexanderson

Papers citing "Diffusion-Based Co-Speech Gesture Generation Using Joint Text and Audio Representation"

18 / 18 papers shown

Title
Joint Co-Speech Gesture and Expressive Talking Face Generation using Diffusion with Adapters S. Hogue Chenxu Zhang Yapeng Tian Xiaohu Guo DiffM 76 0 0 18 Dec 2024
Towards a GENEA Leaderboard -- an Extended, Living Benchmark for Evaluating and Advancing Conversational Motion Synthesis Rajmund Nagy Hendric Voss Youngwoo Yoon Taras Kucherenko Teodor Nikolov Thanh Hoang-Minh R. Mcdonnell Stefan Kopp Michael Neff G. Henter 34 1 0 08 Oct 2024
MM-Conv: A Multi-modal Conversational Dataset for Virtual Humans Anna Deichler Jim O'Regan Jonas Beskow 29 0 0 30 Sep 2024
2D or not 2D: How Does the Dimensionality of Gesture Representation Affect 3D Co-Speech Gesture Generation? Teo Guichoux Laure Soulier Nicolas Obin Catherine Pelachaud SLR 32 0 0 16 Sep 2024
DiffTED: One-shot Audio-driven TED Talk Video Generation with Diffusion-based Co-speech Gestures S. Hogue Chenxu Zhang Hamza Daruger Yapeng Tian Xiaohu Guo VGen 46 10 0 11 Sep 2024
Incorporating Spatial Awareness in Data-Driven Gesture Generation for Virtual Agents Anna Deichler Simon Alexanderson Jonas Beskow 44 0 0 07 Aug 2024
Investigating the impact of 2D gesture representation on co-speech gesture generation Teo Guichoux Laure Soulier Nicolas Obin Catherine Pelachaud SLR 19 0 0 21 Jun 2024
Semantic Gesticulator: Semantics-Aware Co-Speech Gesture Synthesis Zeyi Zhang Tenglong Ao Yuyao Zhang Qingzhe Gao Chuan Lin Baoquan Chen Libin Liu SLR 37 15 0 16 May 2024
LLAniMAtion: LLAMA Driven Gesture Animation John T. Windle Iain Matthews Sarah Taylor 43 0 0 13 May 2024
Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesis Shivam Mehta Anna Deichler Jim O'Regan Birger Moëll Jonas Beskow G. Henter Simon Alexanderson 46 4 0 30 Apr 2024
MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models Zunnan Xu Yukang Lin Haonan Han Sicheng Yang Ronghui Li Yachao Zhang Xiu Li Mamba 46 25 0 14 Mar 2024
LM2D: Lyrics- and Music-Driven Dance Synthesis Wenjie Yin Xuejiao Zhao Yi Yu Hang Yin Danica Kragic Mårten Björkman 28 1 0 14 Mar 2024
MotionScript: Natural Language Descriptions for Expressive 3D Human Motions Payam Jome Yazdian Eric Liu Li Cheng Angelica Lim Li Cheng Angelica Lim 35 7 0 19 Dec 2023
State of the Art on Diffusion Models for Visual Computing Ryan Po Wang Yifan Vladislav Golyanik Kfir Aberman Jonathan T. Barron ... Matthias Nießner Bjorn Ommer Christian Theobalt Peter Wonka Gordon Wetzstein 33 102 0 11 Oct 2023
Unified speech and gesture synthesis using flow matching Shivam Mehta Ruibo Tu Simon Alexanderson Jonas Beskow Éva Székely G. Henter 30 3 0 08 Oct 2023
The GENEA Challenge 2023: A large scale evaluation of gesture generation models in monadic and dyadic settings Taras Kucherenko Rajmund Nagy Youngwoo Yoon Jieyeon Woo Teodor Nikolov Mihail Tsakov G. Henter VGen 32 40 0 24 Aug 2023
GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents Tenglong Ao Zeyi Zhang Libin Liu DiffM VGen 72 145 0 26 Mar 2023
Human Motion Diffusion Model Guy Tevet Sigal Raab Brian Gordon Yonatan Shafir Daniel Cohen-Or Amit H. Bermano DiffM VGen 223 724 0 29 Sep 2022