Language-Guided Audio-Visual Source Separation via Trimodal Consistency

28 March 2023

Papers citing "Language-Guided Audio-Visual Source Separation via Trimodal Consistency"

32 / 32 papers shown

Title
Reading to Listen at the Cocktail Party: Multi-Modal Speech Separation Akam Rahimi Triantafyllos Afouras Andrew Zisserman 104 29 0 02 Jan 2025
A Reference-free Metric for Language-Queried Audio Source Separation using Contrastive Language-Audio Pretraining Feiyang Xiao Jian Guan Qiaoxi Zhu Xubo Liu Wenbo Wang Shuhan Qi Kejia Zhang Jianyuan Sun Wenwu Wang 49 6 0 06 Jul 2024
Mix and Localize: Localizing Sound Sources in Mixtures Xixi Hu Ziyang Chen Andrew Owens 61 52 0 28 Nov 2022
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion Rinon Gal Yuval Alaluf Yuval Atzmon Or Patashnik Amit H. Bermano Gal Chechik Daniel Cohen-Or 109 1,873 0 02 Aug 2022
Text-Driven Separation of Arbitrary Sounds Kevin Kilgour Beat Gfeller Qingqing Huang A. Jansen Scott Wisdom Marco Tagliasacchi 60 34 0 12 Apr 2022
"This is my unicorn, Fluffy": Personalizing frozen vision-language representations Niv Cohen Rinon Gal E. Meirom Gal Chechik Yuval Atzmon VLM MLLM 72 86 0 04 Apr 2022
VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training Zhan Tong Yibing Song Jue Wang Limin Wang ViT 201 1,185 0 23 Mar 2022
End-to-end Generative Pretraining for Multimodal Video Captioning Paul Hongsuck Seo Arsha Nagrani Anurag Arnab Cordelia Schmid 66 168 0 20 Jan 2022
MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound Rowan Zellers Jiasen Lu Ximing Lu Youngjae Yu Yanpeng Zhao Mohammadreza Salehi Aditya Kusupati Jack Hessel Ali Farhadi Yejin Choi 89 212 0 07 Jan 2022
DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting Yongming Rao Wenliang Zhao Guangyi Chen Yansong Tang Zheng Zhu Guan Huang Jie Zhou Jiwen Lu VLM CLIP 187 574 0 02 Dec 2021
Wav2CLIP: Learning Robust Audio Representations From CLIP Ho-Hsiang Wu Prem Seetharaman Kundan Kumar J. P. Bello CLIP VLM 108 269 0 21 Oct 2021
Look at What I'm Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videos Reuben Tan Bryan A. Plummer Kate Saenko Hailin Jin Bryan C. Russell SSL 61 27 0 20 Oct 2021
Visual Scene Graphs for Audio Source Separation Moitreya Chatterjee Jonathan Le Roux Narendra Ahuja A. Cherian 60 37 0 24 Sep 2021
StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators Rinon Gal Or Patashnik Haggai Maron Gal Chechik Daniel Cohen-Or CLIP VLM 72 226 0 02 Aug 2021
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation Xiuye Gu Nayeon Lee Weicheng Kuo Huayu Chen VLM ObjD 263 915 0 28 Apr 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Huayu Chen Boqing Gong ViT 300 588 0 22 Apr 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 826 29,341 0 26 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 422 3,839 0 11 Feb 2021
Memory-augmented Dense Predictive Coding for Video Representation Learning Tengda Han Weidi Xie Andrew Zisserman SSL 77 241 0 03 Aug 2020
AVLnet: Learning Audio-Visual Language Representations from Instructional Videos Andrew Rouditchenko Angie Boggust David Harwath Brian Chen D. Joshi ... Rogerio Feris Brian Kingsbury M. Picheny Antonio Torralba James R. Glass SSL 57 142 0 16 Jun 2020
Solos: A Dataset for Audio-Visual Music Analysis Juan F. Montesinos Olga Slizovskaia G. Haro 49 11 0 14 Jun 2020
Evolving Losses for Unsupervised Video Representation Learning A. Piergiovanni A. Angelova Michael S. Ryoo SSL 62 140 0 26 Feb 2020
SoundSpaces: Audio-Visual Navigation in 3D Environments Changan Chen Unnat Jain Carl Schissler S. V. A. Garí Ziad Al-Halah V. Ithapu Philip Robinson Kristen Grauman 51 26 0 24 Dec 2019
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 401 42,393 0 03 Dec 2019
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips Antoine Miech Dimitri Zhukov Jean-Baptiste Alayrac Makarand Tapaswi Ivan Laptev Josef Sivic VGen 105 1,199 0 07 Jun 2019
Co-Separating Sounds of Visual Objects Ruohan Gao Kristen Grauman 115 208 0 16 Apr 2019
The Sound of Motions Hang Zhao Chuang Gan Wei-Chiu Ma Antonio Torralba 66 254 0 11 Apr 2019
A Short Note about Kinetics-600 João Carreira Eric Noland Andras Banki-Horvath Chloe Hillier Andrew Zisserman 82 527 0 03 Aug 2018
The Sound of Pixels Hang Zhao Chuang Gan Andrew Rouditchenko Carl Vondrick Josh H. McDermott Antonio Torralba VLM 88 535 0 09 Apr 2018
Localizing Moments in Video with Natural Language Lisa Anne Hendricks Oliver Wang Eli Shechtman Josef Sivic Trevor Darrell Bryan C. Russell 110 946 0 04 Aug 2017
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.1K 193,814 0 10 Dec 2015
U-Net: Convolutional Networks for Biomedical Image Segmentation Olaf Ronneberger Philipp Fischer Thomas Brox SSeg 3DV 1.8K 77,099 0 18 May 2015