v1v2 (latest)

Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation

20 March 2023

Ziyang Chen

Shengyi Qian

Andrew Owens

ArXiv (abs)PDF HTML

Papers citing "Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation"

45 / 45 papers shown

Title
NeRAF: 3D Scene Infused Neural Radiance and Acoustic Fields Amandine Brunetto Sascha Hornauer Fabien Moutarde 118 2 0 28 May 2024
Images that Sound: Composing Images and Sounds on a Single Canvas Ziyang Chen Daniel Geng Andrew Owens DiffM 138 9 0 20 May 2024
Novel-View Acoustic Synthesis Changan Chen Alexander Richard Roman Shapovalov V. Ithapu Natalia Neverova Kristen Grauman Andrea Vedaldi 72 37 0 20 Jan 2023
Perspective Fields for Single Image Camera Calibration Linyi Jin Jianming Zhang Yannick Hold-Geoffroy Oliver Wang Kevin Blackburn-Matzen Matthew Sticha David Fouhey 73 50 0 06 Dec 2022
AudioGen: Textually Guided Audio Generation Felix Kreuk Gabriel Synnaeve Adam Polyak Uriel Singer Alexandre Défossez Jade Copet Devi Parikh Yaniv Taigman Yossi Adi DiffM 104 309 0 30 Sep 2022
Learning State-Aware Visual Representations from Audible Interactions Himangi Mittal Pedro Morgado Unnat Jain Abhinav Gupta 118 24 0 27 Sep 2022
A Closer Look at Weakly-Supervised Audio-Visual Source Localization Shentong Mo Pedro Morgado 152 66 0 30 Aug 2022
AudioScopeV2: Audio-Visual Attention Architectures for Calibrated Open-Domain On-Screen Sound Separation Efthymios Tzinis Scott Wisdom Tal Remez J. Hershey 103 30 0 20 Jul 2022
Diffsound: Discrete Diffusion Model for Text-to-sound Generation Dongchao Yang Jianwei Yu Helin Wang Wen Wang Chao Weng Yuexian Zou Dong Yu DiffM 92 306 0 20 Jul 2022
End-to-End Binaural Speech Synthesis Wen-Chin Huang Dejan Marković Alexander Richard I. D. Gebru Anjali Menon 59 9 0 08 Jul 2022
NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors Jiepeng Wang Peng Wang Xiaoxiao Long Christian Theobalt Taku Komura Lingjie Liu Wenping Wang 3DV 50 146 0 27 Jun 2022
Virtual Correspondence: Humans as a Cue for Extreme-View Geometry Wei-Chiu Ma A. Yang Shenlong Wang R. Urtasun Antonio Torralba 112 27 0 16 Jun 2022
SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learning Changan Chen Carl Schissler Sanchit Garg Philip Kobernik Alexander Clegg P. Calamia Dhruv Batra Philip Robinson Kristen Grauman 3DGS 102 86 0 16 Jun 2022
Few-Shot Audio-Visual Learning of Environment Acoustics Sagnik Majumder Changan Chen Ziad Al-Halah Kristen Grauman 72 55 0 08 Jun 2022
Sound Localization by Self-Supervised Time Delay Estimation Ziyang Chen David Fouhey Andrew Owens SSL 78 19 0 26 Apr 2022
Active Audio-Visual Separation of Dynamic Sound Sources Sagnik Majumder Kristen Grauman 93 21 0 02 Feb 2022
Taming Visually Guided Sound Generation Vladimir E. Iashin Esa Rahtu VLM 100 128 0 17 Oct 2021
Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI Santhosh Kumar Ramakrishnan Aaron Gokaslan Erik Wijmans Oleksandr Maksymets Alexander Clegg ... Andrew Westbury Angel X. Chang Manolis Savva Yili Zhao Dhruv Batra 90 393 0 16 Sep 2021
Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection Ruijie Tao Zexu Pan Rohan Kumar Das Xinyuan Qian Mike Zheng Shou Haizhou Li 79 181 0 14 Jul 2021
Bootstrap Your Own Correspondences Mohamed El Banani Justin Johnson 3DPC SSL 68 33 0 01 Jun 2021
Visually Informed Binaural Audio Generation without Binaural Audios Xudong Xu Hang Zhou Ziwei Liu Bo Dai Xiaogang Wang Dahua Lin DiffM 46 59 0 13 Apr 2021
Localizing Visual Sounds the Hard Way Honglie Chen Weidi Xie Triantafyllos Afouras Arsha Nagrani Andrea Vedaldi Andrew Zisserman ObjD 88 191 0 06 Apr 2021
In-Place Scene Labelling and Understanding with Implicit Scene Representation Shuaifeng Zhi Tristan Laidlow Stefan Leutenegger Andrew J. Davison 156 421 0 29 Mar 2021
Planar Surface Reconstruction from Sparse Views Linyi Jin Shengyi Qian Andrew Owens David Fouhey 3DV 48 39 0 26 Mar 2021
Image2Reverb: Cross-Modal Reverb Impulse Response Synthesis Nikhil Singh Jeff Mentch Jerry Ng Matthew Beveridge Iddo Drori 60 47 0 26 Mar 2021
Beyond Image to Depth: Improving Depth Prediction using Echoes Kranti K. Parida Siddharth Srivastava Gaurav Sharma MDE 86 38 0 15 Mar 2021
UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering Mohamed El Banani Luya Gao Justin Johnson 3DPC 47 59 0 23 Feb 2021
MAAS: Multi-modal Assignation for Active Speaker Detection Juan Carlos León Alcázar Fabian Caba Heilbron Ali K. Thabet Guohao Li 116 52 0 11 Jan 2021
VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistency Ruohan Gao Kristen Grauman CVBM 237 202 0 08 Jan 2021
Audio-Visual Floorplan Reconstruction Senthil Purushwalkam S. V. A. Garí V. Ithapu Carl Schissler Philip Robinson Abhinav Gupta Kristen Grauman VGen 3DV 132 41 0 31 Dec 2020
Semantic Audio-Visual Navigation Changan Chen Ziad Al-Halah Kristen Grauman 96 106 0 21 Dec 2020
Learning Representations from Audio-Visual Spatial Alignment Pedro Morgado Yi Li Nuno Vasconcelos SSL 79 124 0 03 Nov 2020
Associative3D: Volumetric Reconstruction from Sparse Views Shengyi Qian Linyi Jin David Fouhey 109 20 0 27 Jul 2020
Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling Yuliang Zou Pan Ji Quoc-Huy Tran Jia-Bin Huang Manmohan Chandraker SSL 122 68 0 21 Jul 2020
Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation Hang Zhou Xudong Xu Dahua Lin Xiaogang Wang Ziwei Liu DiffM 80 84 0 20 Jul 2020
VisualEchoes: Spatial Image Representation Learning through Echolocation Ruohan Gao Changan Chen Ziad Al-Halah Carl Schissler Kristen Grauman MDE SSL 229 84 0 04 May 2020
Audio-Visual Instance Discrimination with Cross-Modal Agreement Pedro Morgado Nuno Vasconcelos Ishan Misra SSL 82 276 0 27 Apr 2020
Audiovisual SlowFast Networks for Video Recognition Fanyi Xiao Yong Jae Lee Kristen Grauman Jitendra Malik Christoph Feichtenhofer 265 209 0 23 Jan 2020
SuperGlue: Learning Feature Matching with Graph Neural Networks Paul-Edouard Sarlin Daniel DeTone Tomasz Malisiewicz Andrew Rabinovich 3DPC OffRL 143 1,953 0 26 Nov 2019
End-to-end Binaural Sound Localisation from the Raw Waveform Paolo Vecchiotti Ning Ma S. Squartini Guy J. Brown 45 59 0 03 Apr 2019
Audio-Visual Scene Analysis with Self-Supervised Multisensory Features Andrew Owens Alexei A. Efros SSL 100 754 0 10 Apr 2018
Direction of arrival estimation for multiple sound sources using convolutional recurrent neural network Sharath Adavanne Archontis Politis Tuomas Virtanen 57 238 0 27 Oct 2017
Relative Camera Pose Estimation Using Convolutional Neural Networks Iaroslav Melekhov Juha Ylioinas Arno Solin Esa Rahtu 134 198 0 05 Feb 2017
FMA: A Dataset For Music Analysis M. Defferrard Kirell Benzi P. Vandergheynst Xavier Bresson 125 441 0 06 Dec 2016
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 355 18,661 0 06 Feb 2015