Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation

2 January 2024

Papers citing "Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation"

34 / 34 papers shown

Title
Editing Music with Melody and Text: Using ControlNet for Diffusion Transformer Siyuan Hou Shansong Liu Ruibin Yuan Wei Xue Ying Shan Mangsuo Zhao Chao Zhang 131 5 0 17 Jan 2025
Audio-Language Datasets of Scenes and Events: A Survey Gijs Wijngaard Elia Formisano Michele Esposito M. Dumontier 151 2 0 10 Jan 2025
TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization Chia-Yu Hung Navonil Majumder Zhifeng Kong Ambuj Mehrish Rafael Valle Bryan Catanzaro Soujanya Poria Bryan Catanzaro Soujanya Poria 114 9 0 30 Dec 2024
MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis Ho Kei Cheng Masato Ishii Akio Hayakawa Takashi Shibuya Alex Schwing Yuki Mitsufuji VGen 261 17 0 19 Dec 2024
Audio-Agent: Leveraging LLMs For Audio Generation, Editing and Composition Zixuan Wang Chi-Keung Tang Chi-Keung Tang DiffM VGen LLMAG 93 4 0 04 Oct 2024
Images that Sound: Composing Images and Sounds on a Single Canvas Ziyang Chen Daniel Geng Andrew Owens DiffM 102 9 0 20 May 2024
Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation Jia-Bin Huang Yi Ren Rongjie Huang Dongchao Yang Zhenhui Ye Chen Zhang Jinglin Liu Xiang Yin Zejun Ma Zhou Zhao DiffM 86 64 0 29 May 2023
WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research Xinhao Mei Chutong Meng Haohe Liu Qiuqiang Kong Tom Ko Chengqi Zhao Mark D. Plumbley Yuexian Zou Wenwu Wang 117 216 0 30 Mar 2023
Adding Conditional Control to Text-to-Image Diffusion Models Lvmin Zhang Anyi Rao Maneesh Agrawala AI4CE 173 4,146 1 10 Feb 2023
eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers Yogesh Balaji Seungjun Nah Xun Huang Arash Vahdat Jiaming Song ... Timo Aila S. Laine Bryan Catanzaro Tero Karras Xuan Li VLM MoE 174 827 0 02 Nov 2022
AudioGen: Textually Guided Audio Generation Felix Kreuk Gabriel Synnaeve Adam Polyak Uriel Singer Alexandre Défossez Jade Copet Devi Parikh Yaniv Taigman Yossi Adi DiffM 82 309 0 30 Sep 2022
Prompt-to-Prompt Image Editing with Cross Attention Control Amir Hertz Ron Mokady J. Tenenbaum Kfir Aberman Yael Pritch Daniel Cohen-Or DiffM 200 1,773 0 02 Aug 2022
Classifier-Free Diffusion Guidance Jonathan Ho Tim Salimans FaML 193 3,898 0 26 Jul 2022
Diffsound: Discrete Diffusion Model for Text-to-sound Generation Dongchao Yang Jianwei Yu Helin Wang Wen Wang Chao Weng Yuexian Zou Dong Yu DiffM 90 305 0 20 Jul 2022
Masked Autoencoders that Listen Po-Yao (Bernie) Huang Hu Xu Juncheng Billy Li Alexei Baevski Michael Auli Wojciech Galuba Florian Metze Christoph Feichtenhofer 89 282 0 13 Jul 2022
BigVGAN: A Universal Neural Vocoder with Large-Scale Training Sang-gil Lee Ming-Yu Liu Boris Ginsburg Bryan Catanzaro Sung-Hoon Yoon 94 251 0 09 Jun 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 410 6,897 0 13 Apr 2022
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 460 15,665 0 20 Dec 2021
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models Alex Nichol Prafulla Dhariwal Aditya A. Ramesh Pranav Shyam Pamela Mishkin Bob McGrew Ilya Sutskever Mark Chen 364 3,605 0 20 Dec 2021
Palette: Image-to-Image Diffusion Models Chitwan Saharia William Chan Huiwen Chang Chris A. Lee Jonathan Ho Tim Salimans David J. Fleet Mohammad Norouzi DiffM VLM 484 1,645 0 10 Nov 2021
Resolution-robust Large Mask Inpainting with Fourier Convolutions Roman Suvorov Elizaveta Logacheva Anton Mashikhin Anastasia Remizova Arsenii Ashukha Aleksei Silvestrov Naejin Kong Harshith Goka Kiwoong Park Victor Lempitsky 106 862 0 15 Sep 2021
Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering Chenyu You Nuo Chen Yuexian Zou SSL 71 63 0 08 Sep 2021
SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations Chenlin Meng Yutong He Yang Song Jiaming Song Jiajun Wu Jun-Yan Zhu Stefano Ermon DiffM 147 1,492 0 02 Aug 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 418 4,987 0 24 Feb 2021
Knowledge Distillation for Improved Accuracy in Spoken Question Answering Chenyu You Nuo Chen Yuexian Zou 65 52 0 21 Oct 2020
Towards Data Distillation for End-to-end Spoken Conversational Question Answering Chenyu You Nuo Chen Fenglin Liu Dongchao Yang Yuexian Zou 56 48 0 18 Oct 2020
HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis Jungil Kong Jaehyeon Kim Jaekyoung Bae 177 1,936 0 12 Oct 2020
PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition Qiuqiang Kong Yin Cao Turab Iqbal Yuxuan Wang Wenwu Wang Mark D. Plumbley VLM SSL 192 1,082 0 21 Dec 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 445 20,298 0 23 Oct 2019
Clotho: An Audio Captioning Dataset Konstantinos Drossos Samuel Lipping Tuomas Virtanen 98 393 0 21 Oct 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 665 24,528 0 26 Jul 2019
Neural Discrete Representation Learning Aaron van den Oord Oriol Vinyals Koray Kavukcuoglu BDL SSL OCL 228 5,061 0 02 Nov 2017
CNN Architectures for Large-Scale Audio Classification Shawn Hershey Sourish Chaudhuri D. Ellis J. Gemmeke A. Jansen ... Rif A. Saurous Bryan Seybold M. Slaney Ron J. Weiss K. Wilson 123 2,506 0 29 Sep 2016
Deep Unsupervised Learning using Nonequilibrium Thermodynamics Jascha Narain Sohl-Dickstein Eric A. Weiss Niru Maheswaranathan Surya Ganguli SyDa DiffM 306 7,005 0 12 Mar 2015