Visual to Sound: Generating Natural Sound for Videos in the Wild

4 December 2017

Papers citing "Visual to Sound: Generating Natural Sound for Videos in the Wild"

43 / 43 papers shown

Title
MM-NeRF: Multimodal-Guided 3D Multi-Style Transfer of Neural Radiance Field Zijian Győző Yang Zhongwei Qiu Chang Xu Dongmei Fu 50 2 0 28 Jan 2025
Gotta Hear Them All: Sound Source Aware Vision to Audio Generation Wei Guo Heng Wang Jianbo Ma Weidong Cai DiffM 93 3 0 23 Nov 2024
Audio-Agent: Leveraging LLMs For Audio Generation, Editing and Composition Zixuan Wang Chi-Keung Tang Chi-Keung Tang DiffM VGen LLMAG 49 4 0 04 Oct 2024
Tarsier: Recipes for Training and Evaluating Large Video Description Models Jiawei Wang Liping Yuan Yuchen Zhang 47 52 0 30 Jun 2024
VideoAdviser: Video Knowledge Distillation for Multimodal Transfer Learning Yanan Wang Donghuo Zeng Shinya Wada Satoshi Kurihara 32 6 0 27 Sep 2023
UniBriVL: Robust Universal Representation and Generation of Audio Driven Diffusion Models Sen Fang Bowen Gao Yangjian Wu T. Teoh DiffM 34 1 0 29 Jul 2023
Exploring Efficient-Tuned Learning Audio Representation Method from BriVL Sen Fang Yang Wu Bowen Gao Jingwen Cai T. Teoh DiffM 29 1 0 08 Mar 2023
iQuery: Instruments as Queries for Audio-Visual Sound Separation Jiaben Chen Renrui Zhang Dongze Lian Jiaqi Yang Ziyao Zeng Jianbo Shi 34 27 0 07 Dec 2022
VarietySound: Timbre-Controllable Video to Sound Generation via Unsupervised Information Disentanglement Chenye Cui Yi Ren Jinglin Liu Rongjie Huang Zhou Zhao VGen 38 14 0 19 Nov 2022
I Hear Your True Colors: Image Guided Audio Generation Roy Sheffer Yossi Adi VLM 18 74 0 06 Nov 2022
Visual onoma-to-wave: environmental sound synthesis from visual onomatopoeias and sound-source images Hien Ohnaka Shinnosuke Takamichi Keisuke Imoto Yuki Okamoto Kazuki Fujii Hiroshi Saruwatari DiffM 24 8 0 17 Oct 2022
Learning in Audio-visual Context: A Review, Analysis, and New Perspective Yake Wei Di Hu Yapeng Tian Xuelong Li 46 55 0 20 Aug 2022
How Should We Evaluate Synthesized Environmental Sounds Yuki Okamoto Keisuke Imoto Shinnosuke Takamichi Takahiro Fukumori Y. Yamashita 15 0 0 16 Aug 2022
Let the paintings play P. Gervasio A. Quarteroni D. Cassani 19 0 0 26 May 2022
GWA: A Large High-Quality Acoustic Dataset for Audio Processing Zhenyu Tang R. Aralikatti Anton Ratnarajah Tianyi Zhou 35 31 0 04 Apr 2022
Sound and Visual Representation Learning with Multiple Pretraining Tasks A. Vasudevan Dengxin Dai Luc Van Gool SSL 38 6 0 04 Jan 2022
Soundify: Matching Sound Effects to Video David Chuan-En Lin Anastasis Germanidis Cristobal Valenzuela Yining Shi Nikolas Martelaro 30 16 0 17 Dec 2021
Everything at Once -- Multi-modal Fusion Transformer for Video Retrieval Nina Shvetsova Brian Chen Andrew Rouditchenko Samuel Thomas Brian Kingsbury Rogerio Feris David Harwath James R. Glass Hilde Kuehne ViT 34 128 0 08 Dec 2021
Geometry-Aware Multi-Task Learning for Binaural Audio Generation from Video Rishabh Garg Ruohan Gao Kristen Grauman 15 28 0 21 Nov 2021
Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances Shibo Zhang Yaxuan Li Shen Zhang Farzad Shahabi S. Xia Yuanbei Deng N. Alshurafa BDL 23 295 0 31 Oct 2021
Taming Visually Guided Sound Generation Vladimir E. Iashin Esa Rahtu VLM 32 122 0 17 Oct 2021
Visual Scene Graphs for Audio Source Separation Moitreya Chatterjee Jonathan Le Roux Narendra Ahuja A. Cherian 26 36 0 24 Sep 2021
FoleyGAN: Visually Guided Generative Adversarial Network-Based Synchronous Sound Generation in Silent Videos Sanchita Ghose John J. Prevost GAN 27 26 0 20 Jul 2021
End-to-End Video-To-Speech Synthesis using Generative Adversarial Networks Rodrigo Mira Konstantinos Vougioukas Pingchuan Ma Stavros Petridis Björn W. Schuller M. Pantic 29 43 0 27 Apr 2021
Can audio-visual integration strengthen robustness under multimodal attacks? Yapeng Tian Chenliang Xu AAML 36 37 0 05 Apr 2021
Sound Synthesis, Propagation, and Rendering: A Survey Shiguang Liu Tianyi Zhou 27 26 0 11 Nov 2020
Cross-modal Center Loss Longlong Jing Elahe Vahdani Jiaxing Tan Yingli Tian 3DPC 12 4 0 08 Aug 2020
Generating Visually Aligned Sound from Videos Peihao Chen Yang Zhang Mingkui Tan Hongdong Xiao Deng Huang Chuang Gan VGen 24 95 0 14 Jul 2020
AVLnet: Learning Audio-Visual Language Representations from Instructional Videos Andrew Rouditchenko Angie Boggust David Harwath Brian Chen D. Joshi ... Rogerio Feris Brian Kingsbury M. Picheny Antonio Torralba James R. Glass SSL 22 141 0 16 Jun 2020
VisualEchoes: Spatial Image Representation Learning through Echolocation Ruohan Gao Changan Chen Ziad Al-Halah Carl Schissler Kristen Grauman MDE SSL 171 84 0 04 May 2020
Vocoder-Based Speech Synthesis from Silent Videos Daniel Michelsanti Olga Slizovskaia G. Haro Emilia Gómez Zheng-Hua Tan Jesper Jensen 31 31 0 06 Apr 2020
Deep Audio-Visual Learning: A Survey Hao Zhu Mandi Luo Rui Wang A. Zheng Ran He 31 156 0 14 Jan 2020
Listen to Look: Action Recognition by Previewing Audio Ruohan Gao Tae-Hyun Oh Kristen Grauman Lorenzo Torresani VLM 29 251 0 10 Dec 2019
Learning to Localize Sound Sources in Visual Scenes: Analysis and Applications Arda Senocak Tae-Hyun Oh Junsik Kim Ming-Hsuan Yang In So Kweon SSL 33 52 0 20 Nov 2019
Vision-Infused Deep Audio Inpainting Hang Zhou Ziwei Liu Lingfeng Guo Ping Luo Dahua Lin 35 88 0 24 Oct 2019
Learning to Have an Ear for Face Super-Resolution Givi Meishvili Simon Jenni Paolo Favaro SupR CVBM 33 23 0 27 Sep 2019
Recursive Visual Sound Separation Using Minus-Plus Net Xudong Xu Bo Dai Dahua Lin 35 91 0 30 Aug 2019
Unpaired Image-to-Speech Synthesis with Multimodal Information Bottleneck Shuang Ma Daniel J. McDuff Yale Song 25 22 0 19 Aug 2019
Co-Separating Sounds of Visual Objects Ruohan Gao Kristen Grauman 33 206 0 16 Apr 2019
The Sound of Motions Hang Zhao Chuang Gan Wei-Chiu Ma Antonio Torralba 17 251 0 11 Apr 2019
2.5D Visual Sound Ruohan Gao Kristen Grauman VGen 24 130 0 11 Dec 2018
The Sound of Pixels Hang Zhao Chuang Gan Andrew Rouditchenko Carl Vondrick Josh H. McDermott Antonio Torralba VLM 22 529 0 09 Apr 2018
Audio-Visual Event Localization in Unconstrained Videos Yapeng Tian Jing Shi Bochen Li Zhiyao Duan Chenliang Xu 36 426 0 23 Mar 2018