UAVM: Towards Unifying Audio and Visual Models

UAVM: Towards Unifying Audio and Visual Models

29 July 2022

Alexander H. Liu

Andrew Rouditchenko

Papers citing "UAVM: Towards Unifying Audio and Visual Models"

15 / 15 papers shown

Title
A Survey of Recent Advances and Challenges in Deep Audio-Visual Correlation Learning Luis Vilaca Yi Yu Paula Vinan 75 0 0 24 Nov 2024
USpeech: Ultrasound-Enhanced Speech with Minimal Human Effort via Cross-Modal Synthesis Luca Jiang-Tao Yu Running Zhao Sijie Ji Edith C. H. Ngai Chenshu Wu 30 0 0 29 Oct 2024
Generalized Multimodal Fusion via Poisson-Nernst-Planck Equation Jiayu Xiong Jing Wang Hengjing Xiang Jun Xue Chen Xu Zhouqiang Jiang 32 0 0 20 Oct 2024
Audio Mamba: Bidirectional State Space Model for Audio Representation Learning Mehmet Hamza Erol Arda Senocak Jiu Feng Joon Son Chung Mamba 73 19 0 05 Jun 2024
Detail-Enhanced Intra- and Inter-modal Interaction for Audio-Visual Emotion Recognition Tong Shi Xuri Ge Joemon M. Jose Nicolas Pugeault Paul Henderson 36 0 0 26 May 2024
Triple Disentangled Representation Learning for Multimodal Affective Analysis Ying Zhou Xuefeng Liang Han Chen Yin Zhao Xin Chen Lida Yu 52 3 0 29 Jan 2024
Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities A. Piergiovanni Isaac Noble Dahun Kim Michael S. Ryoo Victor Gomes A. Angelova 36 19 0 09 Nov 2023
OmniVec: Learning robust representations with cross modal sharing Siddharth Srivastava Gaurav Sharma SSL 27 64 0 07 Nov 2023
Multimodal Fish Feeding Intensity Assessment in Aquaculture Meng Cui Xubo Liu Haohe Liu Zhuangzhuang Du Tao Chen Guoping Lian Daoliang Li Wenwu Wang 28 5 0 10 Sep 2023
AudioFormer: Audio Transformer learns audio feature representations from discrete acoustic codes Zhaohui Li Haitao Wang Xinghua Jiang 40 1 0 14 Aug 2023
Looking Similar, Sounding Different: Leveraging Counterfactual Cross-Modal Pairs for Audiovisual Representation Learning Nikhil Singh Chih-Wei Wu Iroro Orife Mahdi M. Kalayeh 23 2 0 12 Apr 2023
Self-Supervised Multimodal Learning: A Survey Yongshuo Zong Oisin Mac Aodha Timothy M. Hospedales SSL 24 43 0 31 Mar 2023
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing Junyi Ao Rui Wang Long Zhou Chengyi Wang Shuo Ren ... Yu Zhang Zhihua Wei Yao Qian Jinyu Li Furu Wei 118 193 0 14 Oct 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Huayu Chen Boqing Gong ViT 248 577 0 22 Apr 2021
PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation Yuan Gong Yu-An Chung James R. Glass VLM 104 144 0 02 Feb 2021