v1v2 (latest)

MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis

19 December 2024

Papers citing "MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis"

22 / 72 papers shown

Title
Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models Rongjie Huang Jia-Bin Huang Dongchao Yang Yi Ren Luping Liu Mingze Li Zhenhui Ye Jinglin Liu Xiaoyue Yin Zhou Zhao DiffM 243 344 0 30 Jan 2023
MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation Ludan Ruan Yi Ma Huan Yang Huiguo He Bei Liu Jianlong Fu Nicholas Jing Yuan Qin Jin B. Guo DiffM VGen 149 193 0 19 Dec 2022
Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation Yusong Wu Kai Chen Tianyu Zhang Yuchen Hui Marianna Nezhurina Taylor Berg-Kirkpatrick Shlomo Dubnov CLIP 192 546 0 12 Nov 2022
Flow Matching for Generative Modeling Y. Lipman Ricky T. Q. Chen Heli Ben-Hamu Maximilian Nickel Matt Le OOD 318 1,393 0 06 Oct 2022
Classifier-Free Diffusion Guidance Jonathan Ho Tim Salimans FaML 213 3,983 0 26 Jul 2022
BigVGAN: A Universal Neural Vocoder with Large-Scale Training Sang-gil Lee Ming-Yu Liu Boris Ginsburg Bryan Catanzaro Sung-Hoon Yoon 171 255 0 09 Jun 2022
Taming Visually Guided Sound Generation Vladimir E. Iashin Esa Rahtu VLM 133 128 0 17 Oct 2021
Efficient Training of Audio Transformers with Patchout Khaled Koutini Jan Schluter Hamid Eghbalzadeh Gerhard Widmer ViT 187 263 0 11 Oct 2021
RoFormer: Enhanced Transformer with Rotary Position Embedding Jianlin Su Yu Lu Shengfeng Pan Ahmed Murtadha Bo Wen Yunfeng Liu 385 2,555 0 20 Apr 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 1.1K 30,116 0 26 Feb 2021
Generating Visually Aligned Sound from Videos Peihao Chen Yang Zhang Mingkui Tan Hongdong Xiao Deng Huang Chuang Gan VGen 114 97 0 14 Jul 2020
VGGSound: A Large-scale Audio-Visual Dataset Honglie Chen Weidi Xie Andrea Vedaldi Andrew Zisserman 110 583 0 29 Apr 2020
Decision-Making with Auto-Encoding Variational Bayes Romain Lopez Pierre Boyeau Nir Yosef Michael I. Jordan Jeffrey Regier BDL 839 10,591 0 17 Feb 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 679 4,948 0 23 Jan 2020
PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition Qiuqiang Kong Yin Cao Turab Iqbal Yuxuan Wang Wenwu Wang Mark D. Plumbley VLM SSL 265 1,091 0 21 Dec 2019
Clotho: An Audio Captioning Dataset Konstantinos Drossos Samuel Lipping Tuomas Virtanen 136 396 0 21 Oct 2019
Fréchet Audio Distance: A Metric for Evaluating Music Enhancement Algorithms Kevin Kilgour Mauricio Zuluaga Dominik Roblek Matthew Sharifi 149 200 0 20 Dec 2018
FiLM: Visual Reasoning with a General Conditioning Layer Ethan Perez Florian Strub H. D. Vries Vincent Dumoulin Aaron Courville FAtt AIMat OffRL AI4CE 516 2,250 0 22 Sep 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 1.0K 133,599 0 12 Jun 2017
Self-Normalizing Neural Networks Günter Klambauer Thomas Unterthiner Andreas Mayr Sepp Hochreiter 618 2,528 0 08 Jun 2017
Improved Techniques for Training GANs Tim Salimans Ian Goodfellow Wojciech Zaremba Vicki Cheung Alec Radford Xi Chen GAN 650 9,092 0 10 Jun 2016
Visually Indicated Sounds Andrew Owens Phillip Isola Josh H. McDermott Antonio Torralba Edward H. Adelson William T. Freeman 136 386 0 28 Dec 2015