v1v2 (latest)

I Hear Your True Colors: Image Guided Audio Generation

6 November 2022

Roy Sheffer

Yossi Adi

VLM

ArXiv (abs)PDF HTML

Papers citing "I Hear Your True Colors: Image Guided Audio Generation"

24 / 24 papers shown

Title
DiffGAP: A Lightweight Diffusion Module in Contrastive Space for Bridging Cross-Model Gap Shentong Mo Zehua Chen Fan Bao Jun-Jie Zhu DiffM 97 1 0 15 Mar 2025
AudioX: Diffusion Transformer for Anything-to-Audio Generation Zeyue Tian Yizhu Jin Zhaoyang Liu Ruibin Yuan Xu Tan Qifeng Chen Wei Xue Yu Guo 88 6 0 13 Mar 2025
LoVA: Long-form Video-to-Audio Generation Xin Cheng Xihua Wang Yihan Wu Yuyue Wang Ruihua Song VGen DiffM 97 3 0 31 Dec 2024
Gotta Hear Them All: Sound Source Aware Vision to Audio Generation Wei Guo Heng Wang Jianbo Ma Weidong Cai DiffM 156 5 0 23 Nov 2024
Enhancing Robustness in Deep Reinforcement Learning: A Lyapunov Exponent Approach Rory Young Nicolas Pugeault AAML 118 4 0 14 Oct 2024
MDSGen: Fast and Efficient Masked Diffusion Temporal-Aware Transformers for Open-Domain Sound Generation T. Pham Tri Ton Chang D. Yoo 82 3 0 03 Oct 2024
STA-V2A: Video-to-Audio Generation with Semantic and Temporal Alignment Yong Ren Chenxing Li Manjie Xu Wei Liang Yu Gu Rilin Chen Dong Yu VGen DiffM 85 9 0 13 Sep 2024
Video-to-Audio Generation with Hidden Alignment Manjie Xu Chenxing Li Yong Ren Rilin Chen Yu Gu Yu Gu Dong Yu Dong Yu DiffM VGen 81 12 0 10 Jul 2024
AudioGen: Textually Guided Audio Generation Felix Kreuk Gabriel Synnaeve Adam Polyak Uriel Singer Alexandre Défossez Jade Copet Devi Parikh Yaniv Taigman Yossi Adi DiffM 104 309 0 30 Sep 2022
Classifier-Free Diffusion Guidance Jonathan Ho Tim Salimans FaML 196 3,963 0 26 Jul 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 413 6,916 0 13 Apr 2022
Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors Oran Gafni Adam Polyak Oron Ashual Shelly Sheynin Devi Parikh Yaniv Taigman DiffM 89 524 0 24 Mar 2022
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models Alex Nichol Prafulla Dhariwal Aditya A. Ramesh Pranav Shyam Pamela Mishkin Bob McGrew Ilya Sutskever Mark Chen 364 3,627 0 20 Dec 2021
Wav2CLIP: Learning Robust Audio Representations From CLIP Ho-Hsiang Wu Prem Seetharaman Kundan Kumar J. P. Bello CLIP VLM 145 273 0 21 Oct 2021
Taming Visually Guided Sound Generation Vladimir E. Iashin Esa Rahtu VLM 97 128 0 17 Oct 2021
Efficient Training of Audio Transformers with Patchout Khaled Koutini Jan Schluter Hamid Eghbalzadeh Gerhard Widmer ViT 136 261 0 11 Oct 2021
CLIPScore: A Reference-free Evaluation Metric for Image Captioning Jack Hessel Ari Holtzman Maxwell Forbes Ronan Le Bras Yejin Choi CLIP 169 1,588 0 18 Apr 2021
Jukebox: A Generative Model for Music Prafulla Dhariwal Heewoo Jun Christine Payne Jong Wook Kim Alec Radford Ilya Sutskever VLM 128 756 0 30 Apr 2020
VGGSound: A Large-scale Audio-Visual Dataset Honglie Chen Weidi Xie Andrea Vedaldi Andrew Zisserman 89 583 0 29 Apr 2020
Generating Diverse High-Fidelity Images with VQ-VAE-2 Ali Razavi Aaron van den Oord Oriol Vinyals DRL BDL 149 1,828 0 02 Jun 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 129 1,916 0 23 Apr 2019
A Style-Based Generator Architecture for Generative Adversarial Networks Tero Karras S. Laine Timo Aila 619 10,590 0 12 Dec 2018
CNN Architectures for Large-Scale Audio Classification Shawn Hershey Sourish Chaudhuri D. Ellis J. Gemmeke A. Jansen ... Rif A. Saurous Bryan Seybold M. Slaney Ron J. Weiss K. Wilson 130 2,509 0 29 Sep 2016
WaveNet: A Generative Model for Raw Audio Aaron van den Oord Sander Dieleman Heiga Zen Karen Simonyan Oriol Vinyals Alex Graves Nal Kalchbrenner A. Senior Koray Kavukcuoglu DiffM 406 7,421 0 12 Sep 2016