Streaming Generation of Co-Speech Gestures via Accelerated Rolling Diffusion

13 March 2025

Papers citing "Streaming Generation of Co-Speech Gestures via Accelerated Rolling Diffusion"

37 / 37 papers shown

Title
Dynamical Diffusion: Learning Temporal Dynamics with Diffusion Models Xingzhuo Guo Yu Zhang Baixu Chen Haoran Xu Jianmin Wang Mingsheng Long DiffM AI4TS 93 2 0 02 Mar 2025
DiffTED: One-shot Audio-driven TED Talk Video Generation with Diffusion-based Co-speech Gestures S. Hogue Chenxu Zhang Hamza Daruger Yapeng Tian Xiaohu Guo VGen 61 11 0 11 Sep 2024
Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion Boyuan Chen Diego Marti Monso Yilun Du Max Simchowitz Russ Tedrake Vincent Sitzmann DiffM 77 92 0 01 Jul 2024
FIFO-Diffusion: Generating Infinite Videos from Text without Training Jihwan Kim Junoh Kang Jinyoung Choi Bohyung Han DiffM VGen 89 31 0 19 May 2024
Speech-driven Personalized Gesture Synthetics: Harnessing Automatic Fuzzy Feature Inference Fan Zhang Zhaohan Wang Xin Lyu Siyuan Zhao Mengjian Li ... Naye Ji Hui Du Fuxing Gao Hao Wu Shunman Li VGen 59 4 0 16 Mar 2024
Motion Mamba: Efficient and Long Sequence Motion Generation with Hierarchical and Bidirectional Selective SSM Zeyu Zhang Akide Liu Ian Reid Richard Hartley Bohan Zhuang Hao Tang Mamba 72 70 0 12 Mar 2024
Seamless Human Motion Composition with Blended Positional Encodings German Barquero Sergio Escalera Cristina Palmero DiffM 70 33 0 23 Feb 2024
Rolling Diffusion Models David Ruhe Jonathan Heek Tim Salimans Emiel Hoogeboom DiffM 64 38 0 12 Feb 2024
Lumiere: A Space-Time Diffusion Model for Video Generation Omer Bar-Tal Hila Chefer Omer Tov Charles Herrmann Roni Paiss ... T. Michaeli Oliver Wang Deqing Sun Tali Dekel Inbar Mosseri VGen 166 237 0 23 Jan 2024
Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation Mathis Petrovich Or Litany Umar Iqbal Michael J. Black Gül Varol Xue Bin Peng Davis Rempe DiffM VGen 66 44 0 16 Jan 2024
DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generation Junming Chen Yunfei Liu Jianan Wang Ailing Zeng Yu Li Qifeng Chen VGen 55 32 0 09 Jan 2024
Freetalker: Controllable Speech and Text-Driven Gesture Generation Based on Diffusion Models for Enhanced Speaker Naturalness Sicheng Yang Zunnan Xu Haiwei Xue Yongkang Cheng Shaoli Huang Biwei Huang Zhiyong Wu DiffM VGen 47 11 0 07 Jan 2024
From Audio to Photoreal Embodiment: Synthesizing Humans in Conversations Evonne Ng Javier Romero Timur M. Bagautdinov Shaojie Bai Trevor Darrell Angjoo Kanazawa Alexander Richard VGen 43 41 0 03 Jan 2024
EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling Haiyang Liu Zihao Zhu Giorgio Becherini Yichen Peng Mingyang Su You Zhou Xuefei Zhe Naoya Iwamoto Bo Zheng Michael J. Black SLR 77 35 0 31 Dec 2023
FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editing Mingyuan Zhang Huirong Li Zhongang Cai Jiawei Ren Lei Yang Ziwei Liu VGen DiffM 40 47 0 22 Dec 2023
Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets A. Blattmann Tim Dockhorn Sumith Kulal Daniel Mendelevitch Maciej Kilian ... Zion English Vikram S. Voleti Adam Letts Varun Jampani Robin Rombach VGen 228 1,143 0 25 Nov 2023
VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation Xin Li Wenqing Chu Ye Wu Weihang Yuan Fanglong Liu Qi Zhang Fu Li Haocheng Feng Errui Ding Jingdong Wang VGen 73 53 0 01 Sep 2023
MotionGPT: Human Motion as a Foreign Language Biao Jiang Xin Chen Wen Liu Jingyi Yu Gang Yu Tao Chen MLLM 64 288 0 26 Jun 2023
Common Diffusion Noise Schedules and Sample Steps are Flawed Shanchuan Lin Bingchen Liu Jiashi Li Xiao Yang DiffM 63 215 0 15 May 2023
DiffuseStyleGesture: Stylized Audio-Driven Co-Speech Gesture Generation with Diffusion Models Sicheng Yang Zhiyong Wu Minglei Li Zhensong Zhang Lei Hao Weihong Bao Ming Cheng Long Xiao 42 70 0 08 May 2023
Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generation Lingting Zhu Xian Liu Xuanyu Liu Rui Qian Ziwei Liu Lequan Yu 59 118 0 16 Mar 2023
Human Motion Diffusion as a Generative Prior Yonatan Shafir Guy Tevet Roy Kapon Amit H. Bermano DiffM VGen 50 227 0 02 Mar 2023
A Comprehensive Review of Data-Driven Co-Speech Gesture Generation Simbarashe Nyatsanga Taras Kucherenko Chaitanya Ahuja G. Henter Michael Neff SLR 54 92 0 13 Jan 2023
MultiAct: Long-Term 3D Human Motion Generation from Multiple Action Labels T. Lee Gyeongsik Moon Kyoung Mu Lee 50 45 0 12 Dec 2022
Generating Holistic 3D Human Motion from Speech Hongwei Yi Hualin Liang Yifei Liu Qiong Cao Yandong Wen Timo Bolkart Dacheng Tao Michael J. Black SLR 63 148 0 08 Dec 2022
EDGE: Editable Dance Generation From Music Jo-Han Tseng Rodrigo Castellon Chenxi Liu 69 235 0 19 Nov 2022
Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models Simon Alexanderson Rajmund Nagy Jonas Beskow G. Henter DiffM VGen 63 170 0 17 Nov 2022
ZeroEGGS: Zero-shot Example-based Gesture Generation from Speech Saeed Ghorbani Ylva Ferstl Daniel Holden N. Troje M. Carbonneau 62 82 0 15 Sep 2022
TEACH: Temporal Action Composition for 3D Humans Nikos Athanasiou Mathis Petrovich Michael J. Black Gül Varol 122 147 0 09 Sep 2022
Weakly-supervised Action Transition Learning for Stochastic Human Motion Prediction Wei Mao Miaomiao Liu Mathieu Salzmann 58 34 0 31 May 2022
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding Chitwan Saharia William Chan Saurabh Saxena Lala Li Jay Whang ... Raphael Gontijo-Lopes Tim Salimans Jonathan Ho David J Fleet Mohammad Norouzi VLM 362 5,978 0 23 May 2022
Video Diffusion Models Jonathan Ho Tim Salimans Alexey A. Gritsenko William Chan Mohammad Norouzi David J. Fleet DiffM VGen 163 1,608 0 07 Apr 2022
BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis Haiyang Liu Zihao Zhu Naoya Iwamoto Yichen Peng Zhengqing Li You Zhou E. Bozkurt Bo Zheng SLR CVBM 47 140 0 10 Mar 2022
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 367 15,454 0 20 Dec 2021
Denoising Diffusion Implicit Models Jiaming Song Chenlin Meng Stefano Ermon VLM DiffM 213 7,294 0 06 Oct 2020
Denoising Diffusion Probabilistic Models Jonathan Ho Ajay Jain Pieter Abbeel DiffM 512 17,888 0 19 Jun 2020
Generative Modeling by Estimating Gradients of the Data Distribution Yang Song Stefano Ermon SyDa DiffM 213 3,870 0 12 Jul 2019