Music Gesture for Visual Sound Separation

20 April 2020

Chuang Gan

Deng Huang

Hang Zhao

J. Tenenbaum

Antonio Torralba

ArXiv PDF HTML

Papers citing "Music Gesture for Visual Sound Separation"

40 / 40 papers shown

Title
Reading to Listen at the Cocktail Party: Multi-Modal Speech Separation Akam Rahimi Triantafyllos Afouras Andrew Zisserman 40 28 0 02 Jan 2025
Aligning Sight and Sound: Advanced Sound Source Localization Through Audio-Visual Alignment Arda Senocak H. Ryu Junsik Kim Tae-Hyun Oh Hanspeter Pfister Joon Son Chung 38 3 0 18 Jul 2024
Robust Active Speaker Detection in Noisy Environments Siva Sai Nagender Vasireddy Chenxu Zhang Xiaohu Guo Yapeng Tian 40 0 0 27 Mar 2024
Sound Source Localization is All about Cross-Modal Alignment Arda Senocak H. Ryu Junsik Kim Tae-Hyun Oh Hanspeter Pfister Joon Son Chung 33 18 0 19 Sep 2023
ProgSG: Cross-Modality Representation Learning for Programs in Electronic Design Automation Yunsheng Bai Atefeh Sohrabizadeh Zongyue Qin Ziniu Hu Yizhou Sun Jason Cong 18 1 0 18 May 2023
A vector quantized masked autoencoder for audiovisual speech emotion recognition Samir Sadok Simon Leglaive Renaud Séguier SSL 79 6 0 05 May 2023
A Closer Look at Weakly-Supervised Audio-Visual Source Localization Shentong Mo Pedro Morgado 83 64 0 30 Aug 2022
Learning in Audio-visual Context: A Review, Analysis, and New Perspective Yake Wei Di Hu Yapeng Tian Xuelong Li 46 55 0 20 Aug 2022
AudioScopeV2: Audio-Visual Attention Architectures for Calibrated Open-Domain On-Screen Sound Separation Efthymios Tzinis Scott Wisdom Tal Remez J. Hershey 36 29 0 20 Jul 2022
Learning Music-Dance Representations through Explicit-Implicit Rhythm Synchronization Jiashuo Yu Junfu Pu Ying Cheng Rui Feng Ying Shan 21 5 0 07 Jul 2022
Learning Neural Acoustic Fields Andrew F. Luo Yilun Du Michael J. Tarr J. Tenenbaum Antonio Torralba Chuang Gan AI4CE 20 76 0 04 Apr 2022
Audio-Visual Speech Codecs: Rethinking Audio-Visual Speech Enhancement by Re-Synthesis Karren D. Yang Dejan Marković Steven Krenn Vasu Agrawal Alexander Richard VGen 16 32 0 31 Mar 2022
Speaker Extraction with Co-Speech Gestures Cue Zexu Pan Xinyuan Qian Haizhou Li SLR 21 26 0 31 Mar 2022
The Sound of Bounding-Boxes Takashi Oya Shohei Iwase Shigeo Morishima 19 2 0 30 Mar 2022
Learning to Answer Questions in Dynamic Audio-Visual Scenarios Guangyao Li Yake Wei Yapeng Tian Chenliang Xu Ji-Rong Wen Di Hu 29 136 0 26 Mar 2022
Sound Adversarial Audio-Visual Navigation Yinfeng Yu Wenbing Huang Gang Hua Changan Chen Yikai Wang Xiaohong Liu AAML 19 29 0 22 Feb 2022
One-shot Scene Graph Generation Yuyu Guo Jingkuan Song Lianli Gao Heng Tao Shen 25 29 0 22 Feb 2022
Visual Sound Localization in the Wild by Cross-Modal Interference Erasing Xian Liu Rui Qian Hang Zhou Di Hu Weiyao Lin Ziwei Liu Bolei Zhou Xiaowei Zhou 18 25 0 13 Feb 2022
Active Audio-Visual Separation of Dynamic Sound Sources Sagnik Majumder Kristen Grauman 19 21 0 02 Feb 2022
Geometry-Aware Multi-Task Learning for Binaural Audio Generation from Video Rishabh Garg Ruohan Gao Kristen Grauman 15 27 0 21 Nov 2021
Beyond Mono to Binaural: Generating Binaural Audio from Mono Audio with Depth and Cross Modal Attention Kranti K. Parida Siddharth Srivastava Gaurav Sharma MDE 36 20 0 15 Nov 2021
TriBERT: Full-body Human-centric Audio-visual Representation Learning for Visual Sound Separation Tanzila Rahman Mengyu Yang Leonid Sigal ViT 29 8 0 26 Oct 2021
Taming Visually Guided Sound Generation Vladimir E. Iashin Esa Rahtu VLM 28 121 0 17 Oct 2021
Audio-Visual Transformer Based Crowd Counting Usman Sajid Xiangyu Chen Hasan Sajid Taejoon Kim Guanghui Wang ViT 35 22 0 04 Sep 2021
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation Hang Zhou Yasheng Sun Wayne Wu Chen Change Loy Xiaogang Wang Ziwei Liu CVBM 28 360 0 22 Apr 2021
A cappella: Audio-visual Singing Voice Separation Juan F. Montesinos V. S. Kadandale G. Haro 38 16 0 20 Apr 2021
Visually Informed Binaural Audio Generation without Binaural Audios Xudong Xu Hang Zhou Ziwei Liu Bo Dai Xiaogang Wang Dahua Lin DiffM 13 53 0 13 Apr 2021
Unsupervised Sound Localization via Iterative Contrastive Learning Yan-Bo Lin Hung-Yu Tseng Hsin-Ying Lee Yen-Yu Lin Ming-Hsuan Yang SSL 19 34 0 01 Apr 2021
TransCenter: Transformers with Dense Representations for Multiple-Object Tracking Yihong Xu Yutong Ban Guillaume Delorme Chuang Gan Daniela Rus Xavier Alameda-Pineda VOT 25 92 0 28 Mar 2021
Beyond Image to Depth: Improving Depth Prediction using Echoes Kranti K. Parida Siddharth Srivastava Gaurav Sharma MDE 42 37 0 15 Mar 2021
Learning Audio-Visual Correlations from Variational Cross-Modal Generation Ye Zhu Yu Wu Hugo Latapie Yi Yang Yan Yan SSL 16 20 0 05 Feb 2021
Music source separation conditioned on 3D point clouds Francesc Lluís V. Chatziioannou A. Hofmann 3DPC 24 5 0 03 Feb 2021
Learning Representations from Audio-Visual Spatial Alignment Pedro Morgado Yi Li Nuno Vasconcelos SSL 27 121 0 03 Nov 2020
Into the Wild with AudioScope: Unsupervised Audio-Visual Separation of On-Screen Sounds Efthymios Tzinis Scott Wisdom A. Jansen Shawn Hershey Tal Remez D. Ellis J. Hershey 28 68 0 02 Nov 2020
Multiple Sound Sources Localization from Coarse to Fine Rui Qian Di Hu Heinrich Dinkel Mengyue Wu N. Xu Weiyao Lin 28 153 0 13 Jul 2020
ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation Chuang Gan Jeremy Schwartz S. Alter Damian Mrowca Martin Schrimpf ... Antonio Torralba J. DiCarlo J. Tenenbaum Josh H. McDermott Daniel L. K. Yamins VGen 53 303 0 09 Jul 2020
Visually Guided Sound Source Separation using Cascaded Opponent Filter Network Lingyu Zhu Esa Rahtu 19 23 0 04 Jun 2020
VisualEchoes: Spatial Image Representation Learning through Echolocation Ruohan Gao Changan Chen Ziad Al-Halah Carl Schissler Kristen Grauman MDE SSL 171 83 0 04 May 2020
On the Role of Visual Cues in Audiovisual Speech Enhancement Zakaria Aldeneh Anushree Prasanna Kumar B. Theobald Erik Marchi S. Kajarekar Devang Naik Ahmed Hussen Abdelaziz 20 6 0 25 Apr 2020
Lip Reading Sentences in the Wild Joon Son Chung A. Senior Oriol Vinyals Andrew Zisserman 162 784 0 16 Nov 2016