AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining

10 August 2023

Yuxuan Wang

Papers citing "AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining"

50 / 168 papers shown

Title
FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio Generation Huadai Liu Jialei Wang Rongjie Huang Yang Liu H. Lu Wei Xue Zhou Zhao 13 3 0 16 Oct 2024
Enhancing Robustness in Deep Reinforcement Learning: A Lyapunov Exponent Approach Rory Young Nicolas Pugeault AAML 65 3 0 14 Oct 2024
Language-Guided Joint Audio-Visual Editing via One-Shot Adaptation Susan Liang Chao Huang Yapeng Tian Anurag Kumar Chenliang Xu DiffM 34 7 0 09 Oct 2024
SRC-gAudio: Sampling-Rate-Controlled Audio Generation Chenxing Li Manjie Xu Dong Yu DiffM 33 0 0 09 Oct 2024
Art2Mus: Bridging Visual Arts and Music through Cross-Modal Generation Ivan Rinaldi Nicola Fanelli Giovanna Castellano G. Vessio 31 2 0 07 Oct 2024
Presto! Distilling Steps and Layers for Accelerating Music Generation Zachary Novack Ge Zhu Jonah Casebeer Julian McAuley Taylor Berg-Kirkpatrick Nicholas J. Bryan 45 5 0 07 Oct 2024
Did You Hear That? Introducing AADG: A Framework for Generating Benchmark Data in Audio Anomaly Detection Ksheeraja Raghavan Samiran Gode Ankit Parag Shah Surabhi Raghavan Wolfram Burgard Bhiksha Raj Rita Singh 25 0 0 04 Oct 2024
Audio-Agent: Leveraging LLMs For Audio Generation, Editing and Composition Zixuan Wang Chi-Keung Tang Chi-Keung Tang DiffM VGen LLMAG 49 4 0 04 Oct 2024
SoundMorpher: Perceptually-Uniform Sound Morphing with Diffusion Model Xinlei Niu Jing Zhang Charles Patrick Martin 25 1 0 03 Oct 2024
MIMII-Gen: Generative Modeling Approach for Simulated Evaluation of Anomalous Sound Detection System Harsh Purohit Tomoya Nishida Kota Dohi Takashi Endo Y. Kawaguchi DiffM 46 0 0 27 Sep 2024
Gradient-free Decoder Inversion in Latent Diffusion Models Seongmin Hong Suh Yoon Jeon Kyeonghyun Lee Ernest K. Ryu S. Chun 26 0 0 27 Sep 2024
MuCodec: Ultra Low-Bitrate Music Codec Yaoxun Xu Hangting Chen Jianwei Yu Wei Tan Rongzhi Gu Shun Lei Zhiwei Lin Zhiyong Wu 32 1 0 20 Sep 2024
AudioEditor: A Training-Free Diffusion-Based Audio Editing Framework Yuhang Jia Yang Chen Jinghua Zhao Shiwan Zhao Wenjia Zeng Yong Chen Yong Qin DiffM 36 1 0 19 Sep 2024
AudioComposer: Towards Fine-grained Audio Generation with Natural Language Descriptions Yishuo Wang Hangting Chen Dongchao Yang Zhiyong Wu Xixin Wu DiffM 45 2 0 19 Sep 2024
High-Resolution Speech Restoration with Latent Diffusion Model Tushar Dhyani Florian Lux Michele Mancusi Giorgio Fabbro Fritz Hohl Ngoc Thang Vu DiffM 37 0 0 17 Sep 2024
FakeMusicCaps: a Dataset for Detection and Attribution of Synthetic Music Generated via Text-to-Music Models Luca Comanducci Paolo Bestagini Stefano Tubaro 35 7 0 16 Sep 2024
Towards Diverse and Efficient Audio Captioning via Diffusion Models Manjie Xu Chenxing Li Xinyi Tu Yong Ren Ruibo Fu Wei Liang Dong Yu DiffM 49 1 0 14 Sep 2024
Joint Semantic Knowledge Distillation and Masked Acoustic Modeling for Full-band Speech Restoration with Improved Intelligibility Xiaoyu Liu Xu Li Joan Serrà Santiago Pascual 31 3 0 14 Sep 2024
Rhythmic Foley: A Framework For Seamless Audio-Visual Alignment In Video-to-Audio Synthesis Zhiqi Huang Dan Luo Jun Wang Huan Liao Zhiheng Li Zhiyong Wu VGen 53 4 0 13 Sep 2024
MambaFoley: Foley Sound Generation using Selective State-Space Models Marco Furio Colombo Francesca Ronchini Luca Comanducci Fabio Antonacci Mamba 25 1 0 13 Sep 2024
STA-V2A: Video-to-Audio Generation with Semantic and Temporal Alignment Yong Ren Chenxing Li Manjie Xu Wei Liang Yu Gu Rilin Chen Dong Yu VGen DiffM 48 7 0 13 Sep 2024
Bridging Paintings and Music -- Exploring Emotion based Music Generation through Paintings Tanisha Hisariya Huan Zhang Jinhua Liang 29 3 0 12 Sep 2024
VMAS: Video-to-Music Generation via Semantic Alignment in Web Music Videos Yan-Bo Lin Yu Tian L. Yang Gedas Bertasius Heng Wang VGen 34 7 0 11 Sep 2024
Enhancing Emotional Text-to-Speech Controllability with Natural Language Guidance through Contrastive Learning and Diffusion Models Xin Jing Kun Zhou Andreas Triantafyllopoulos Björn W. Schuller DiffM 42 3 0 10 Sep 2024
Multi-Source Music Generation with Latent Diffusion Zhongweiyang Xu Debottam Dutta Yu-Lin Wei Romit Roy Choudhury DiffM 45 1 0 10 Sep 2024
Draw an Audio: Leveraging Multi-Instruction for Video-to-Audio Synthesis Qi Yang Binjie Mao Zili Wang Xing Nie Pengfei Gao Ying Guo Cheng Zhen Pengfei Yan Shiming Xiang VGen DiffM 46 5 0 10 Sep 2024
SongCreator: Lyrics-based Universal Song Generation Shun Lei Yixuan Zhou Boshi Tang Max W. Y. Lam Feng Liu Hangyu Liu Jingcheng Wu Shiyin Kang Zhiyong Wu Helen Meng 52 5 0 09 Sep 2024
MetaBGM: Dynamic Soundtrack Transformation For Continuous Multi-Scene Experiences With Ambient Awareness And Personalization Haoxuan Liu Zihao Wang HaoRong Hong Youwei Feng Jiaxin Yu Han Diao Yunfei Xu Kaipeng Zhang 36 0 0 05 Sep 2024
FLUX that Plays Music Zhengcong Fei Mingyuan Fan Changqian Yu Junshi Huang 84 7 0 01 Sep 2024
DisMix: Disentangling Mixtures of Musical Instruments for Source-level Pitch and Timbre Manipulation Yin-Jyun Luo K. Cheuk Woosung Choi Toshimitsu Uesaka Keisuke Toyama ... Chieh-Hsin Lai Yuhta Takida Wei-Hsiang Liao Simon Dixon Yuki Mitsufuji CoGe 49 2 0 20 Aug 2024
TEAdapter: Supply abundant guidance for controllable text-to-music generation Jialing Zou Jiahao Mei Xudong Nan Jinghua Li Daoguo Dong Liang He 36 0 0 09 Aug 2024
MusiConGen: Rhythm and Chord Control for Transformer-Based Text-to-Music Generation Yun-Han Lan Wen-Yi Hsiao Hao-Chung Cheng Yi-Hsuan Yang 53 7 0 21 Jul 2024
Stable Audio Open Zach Evans Julian Parker CJ Carr Zack Zukowski Josiah Taylor Jordi Pons 75 38 0 19 Jul 2024
MEDIC: Zero-shot Music Editing with Disentangled Inversion Control Huadai Liu Jialei Wang Rongjie Huang Yang Liu Jiayang Xu Zhou Zhao 31 4 0 18 Jul 2024
Audio Conditioning for Music Generation via Discrete Bottleneck Features Simon Rouard Yossi Adi Jade Copet Axel Roebel Alexandre Défossez MGen 57 1 0 17 Jul 2024
LiteFocus: Accelerated Diffusion Inference for Long Audio Synthesis Zhenxiong Tan Xinyin Ma Gongfan Fang Xinchao Wang 38 3 0 15 Jul 2024
Masked Generative Video-to-Audio Transformers with Enhanced Synchronicity Santiago Pascual Chunghsin Yeh Ioannis Tsiamas Joan Serrà DiffM VGen 47 15 0 15 Jul 2024
Video-to-Audio Generation with Hidden Alignment Manjie Xu Chenxing Li Yong Ren Rilin Chen Yu Gu Yu Gu Dong Yu Dong Yu DiffM VGen 43 12 0 10 Jul 2024
Read, Watch and Scream! Sound Generation from Text and Video Yujin Jeong Yunji Kim Sanghyuk Chun Jiyoung Lee VGen DiffM 31 12 0 08 Jul 2024
A Reference-free Metric for Language-Queried Audio Source Separation using Contrastive Language-Audio Pretraining Feiyang Xiao Jian Guan Qiaoxi Zhu Xubo Liu Wenbo Wang Shuhan Qi Kejia Zhang Jianyuan Sun Wenwu Wang 30 4 0 06 Jul 2024
PAGURI: a user experience study of creative interaction with text-to-music models Francesca Ronchini Luca Comanducci Gabriele Perego Fabio Antonacci 35 3 0 05 Jul 2024
PicoAudio: Enabling Precise Timestamp and Frequency Controllability of Audio Events in Text-to-audio Generation Zeyu Xie Xuenan Xu Zhizheng Wu Mengyue Wu 40 8 0 03 Jul 2024
AudioTime: A Temporally-aligned Audio-text Benchmark Dataset Zeyu Xie Xuenan Xu Zhizheng Wu Mengyue Wu AuLLM 51 5 0 03 Jul 2024
FoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized Sounds Yiming Zhang Yicheng Gu Yanhong Zeng Zhening Xing Yuancheng Wang Zhizheng Wu Kai Chen VGen 29 37 0 01 Jul 2024
SpecMaskGIT: Masked Generative Modeling of Audio Spectrograms for Efficient Audio Synthesis and Beyond Marco Comunità Zhi-Wei Zhong Akira Takahashi Shiqi Yang Mengjie Zhao Koichi Saito Yukara Ikemiya Takashi Shibuya Shusuke Takahashi Yuki Mitsufuji 71 2 0 25 Jun 2024
Exploring compressibility of transformer based text-to-music (TTM) models Vasileios Moschopoulos Thanasis Kotsiopoulos Pablo Peso Parada Konstantinos Nikiforidis Alexandros Stergiadis Gerasimos Papakostas Md. Asif Jalal Jisi Zhang Anastasios Drosou Karthikeyan P. Saravanan 25 0 0 24 Jun 2024
Improving Text-To-Audio Models with Synthetic Captions Zhifeng Kong Sang-gil Lee Deepanway Ghosal Navonil Majumder Ambuj Mehrish Rafael Valle Soujanya Poria Bryan Catanzaro 53 11 0 18 Jun 2024
MusicScore: A Dataset for Music Score Modeling and Generation Yuheng Lin Zheqi Dai Qiuqiang Kong VLM 37 2 0 17 Jun 2024
MINT: a Multi-modal Image and Narrative Text Dubbing Dataset for Foley Audio Content Planning and Generation Ruibo Fu Shuchen Shi Hongming Guo Tao Wang Chunyu Qiang ... Zhiyong Wang Yukun Liu Xuefei Liu Shuai Zhang Guanjun Li VGen 30 0 0 15 Jun 2024
Diffusion Synthesizer for Efficient Multilingual Speech to Speech Translation Nameer Hirschkind Xiao Yu Mahesh Kumar Nandwana Joseph Liu Eloi DuBois ... Colin Sinclair Kyle Spence Charles Shang Zoë Abrams Morgan McGuire 35 0 0 14 Jun 2024