v1v2 (latest)

Anchors Aweigh! Sail for Optimal Unified Multi-Modal Representations

2 October 2024

Dongyeop Kang

Papers citing "Anchors Aweigh! Sail for Optimal Unified Multi-Modal Representations"

50 / 59 papers shown

Title
NeuroBind: Towards Unified Multimodal Representations for Neural Signals Fengyu Yang Chao Feng Daniel Wang Tianye Wang Ziyao Zeng ... Hyoungseob Park Pengliang Ji Han Zhao Yuanning Li Alex Wong 96 9 0 19 Jul 2024
OmniBind: Large-scale Omni Multimodal Representation via Binding Spaces Zehan Wang Ziang Zhang Hang Zhang Luping Liu Rongjie Huang Xize Cheng Hengshuang Zhao Zhou Zhao 96 13 0 16 Jul 2024
HEMM: Holistic Evaluation of Multimodal Foundation Models Paul Pu Liang Akshay Goindani Talha Chafekar Leena Mathur Haofei Yu Ruslan Salakhutdinov Louis-Philippe Morency 73 15 0 03 Jul 2024
MVBIND: Self-Supervised Music Recommendation For Videos Via Embedding Space Binding Jiajie Teng Huiyu Duan Yucheng Zhu Sijing Wu Guangtao Zhai 52 2 0 15 May 2024
FreeBind: Free Lunch in Unified Multimodal Space via Knowledge Fusion Zehan Wang Ziang Zhang Xize Cheng Rongjie Huang Luping Liu ... Haifeng Huang Yang Zhao Tao Jin Peng Gao Zhou Zhao 72 10 0 08 May 2024
GEOBIND: Binding Text, Image, and Audio through Satellite Images Aayush Dhakal Subash Khanal Srikumar Sastry Adeel Ahmad Nathan Jacobs 65 3 0 17 Apr 2024
UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them All Yuanhuiyi Lyu Xueye Zheng Jiazhou Zhou Lin Wang 84 25 0 19 Mar 2024
Binding Touch to Everything: Learning Unified Multimodal Tactile Representations Fengyu Yang Chao Feng Ziyang Chen Hyoungseob Park Daniel Wang ... Ziyao Zeng Xien Chen Rit Gangopadhyay Andrew Owens Alex Wong 109 71 0 31 Jan 2024
ViT-Lens: Towards Omni-modal Representations Weixian Lei Yixiao Ge Kun Yi Jianfeng Zhang Difei Gao Dylan Sun Yuying Ge Ying Shan Mike Zheng Shou 79 20 0 27 Nov 2023
Multimodal Representation Learning by Alternating Unimodal Adaptation Xiaohui Zhang Jaehong Yoon Mohit Bansal Huaxiu Yao 105 26 0 17 Nov 2023
SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalization Hao Dong Ismail Nejjar Han Sun Eleni Chatzi Olga Fink 92 25 0 30 Oct 2023
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment Bin Zhu Bin Lin Munan Ning Yang Yan Jiaxi Cui ... Zongwei Li Wancai Zhang Zhifeng Li Wei Liu Liejie Yuan VLM MLLM 143 229 0 03 Oct 2023
Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following Ziyu Guo Renrui Zhang Xiangyang Zhu Yiwen Tang Xianzheng Ma ... Ke Chen Peng Gao Xianzhi Li Hongsheng Li Pheng-Ann Heng MLLM 93 144 0 01 Sep 2023
Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining? Fei Wang Liang Ding Jun Rao Ye Liu Li Shen Changxing Ding 80 15 0 24 Aug 2023
Factorized Contrastive Learning: Going Beyond Multi-view Redundancy Paul Pu Liang Zihao Deng Martin Q. Ma James Zou Louis-Philippe Morency Ruslan Salakhutdinov SSL 83 54 0 08 Jun 2023
ImageBind: One Embedding Space To Bind Them All Rohit Girdhar Alaaeldin El-Nouby Zhuang Liu Mannat Singh Kalyan Vasudev Alwala Armand Joulin Ishan Misra VLM 176 941 0 09 May 2023
On Uni-Modal Feature Learning in Supervised Multi-Modal Learning Chenzhuang Du Jiaye Teng Tingle Li Yichen Liu Tianyuan Yuan Yue Wang Yang Yuan Hang Zhao 172 45 0 02 May 2023
MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action Zhengyuan Yang Linjie Li Jianfeng Wang Kevin Qinghong Lin E. Azarnasab Faisal Ahmed Zicheng Liu Ce Liu Michael Zeng Lijuan Wang ReLM KELM LRM 111 394 0 20 Mar 2023
Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework Paul Pu Liang Yun Cheng Xiang Fan Chun Kai Ling Suzanne Nie ... Nicholas B. Allen Randy P. Auerbach Faisal Mahmood Ruslan Salakhutdinov Louis-Philippe Morency 84 35 0 23 Feb 2023
Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions Paul Pu Liang Amir Zadeh Louis-Philippe Morency 98 78 0 07 Sep 2022
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation Nataniel Ruiz Yuanzhen Li Varun Jampani Yael Pritch Michael Rubinstein Kfir Aberman 282 2,895 0 25 Aug 2022
Face-to-Face Contrastive Learning for Social Intelligence Question-Answering Alex Wilf Qianli Ma Paul Pu Liang Amir Zadeh Louis-Philippe Morency 73 10 0 29 Jul 2022
Clover: Towards A Unified Video-Language Alignment and Fusion Model Jingjia Huang Yinan Li Jiashi Feng Xinglong Wu Xiaoshuai Sun Rongrong Ji VLM 101 50 0 16 Jul 2022
VLMixer: Unpaired Vision-Language Pre-training via Cross-Modal CutMix Teng Wang Wenhao Jiang Zhichao Lu Feng Zheng Ran Cheng Chengguo Yin Ping Luo VLM 72 44 0 17 Jun 2022
K-LITE: Learning Transferable Visual Models with External Knowledge Sheng Shen Chunyuan Li Xiaowei Hu Jianwei Yang Yujia Xie ... Ce Liu Kurt Keutzer Trevor Darrell Anna Rohrbach Jianfeng Gao CLIP VLM 60 85 0 20 Apr 2022
Balanced Multimodal Learning via On-the-fly Gradient Modulation Xiaokang Peng Yake Wei Andong Deng Dong Wang Di Hu 80 214 0 29 Mar 2022
VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training Zhan Tong Yibing Song Jue Wang Limin Wang ViT 230 1,207 0 23 Mar 2022
Rethinking Minimal Sufficient Representation in Contrastive Learning Haoqing Wang Xun Guo Zhiwei Deng Yan Lu SSL 73 76 0 14 Mar 2022
Multi-modal Alignment using Representation Codebook Jiali Duan Liqun Chen Son Tran Jinyu Yang Yi Xu Belinda Zeng Trishul Chilimbi 88 68 0 28 Feb 2022
Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networks Nan Wu Stanislaw Jastrzebski Kyunghyun Cho Krzysztof J. Geras 47 76 0 10 Feb 2022
PointCLIP: Point Cloud Understanding by CLIP Renrui Zhang Ziyu Guo Wei Zhang Kunchang Li Xupeng Miao Tengjiao Wang Yu Qiao Peng Gao Hongsheng Li VLM 3DPC 262 452 0 04 Dec 2021
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing Sanyuan Chen Chengyi Wang Zhengyang Chen Yu-Huan Wu Shujie Liu ... Yao Qian Jian Wu Micheal Zeng Xiangzhan Yu Furu Wei SSL 267 1,905 0 26 Oct 2021
V-SlowFast Network for Efficient Visual Sound Separation Lingyu Zhu Esa Rahtu 110 10 0 18 Sep 2021
Attention Bottlenecks for Multimodal Fusion Arsha Nagrani Shan Yang Anurag Arnab A. Jansen Cordelia Schmid Chen Sun 106 569 0 30 Jun 2021
AudioCLIP: Extending CLIP to Image, Text and Audio A. Guzhov Federico Raue Jörn Hees Andreas Dengel CLIP VLM 127 370 0 24 Jun 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 502 10,526 0 17 Jun 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 1.0K 29,926 0 26 Feb 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir Fahad Shahbaz Khan M. Shah ViT 334 2,537 0 04 Jan 2021
Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis Daniel Seichter Mona Köhler Benjamin Lewandowski Tim Wengefeld H. Groß 100 223 0 13 Nov 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 684 41,563 0 22 Oct 2020
What Makes for Good Views for Contrastive Learning? Yonglong Tian Chen Sun Ben Poole Dilip Krishnan Cordelia Schmid Phillip Isola SSL 118 1,338 0 20 May 2020
Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere Tongzhou Wang Phillip Isola SSL 164 1,860 0 20 May 2020
Urban2Vec: Incorporating Street View Imagery and POIs for Multi-Modal Urban Neighborhood Embedding Zhecheng Wang Haoyuan Li Ram Rajagopal 81 82 0 29 Jan 2020
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 252 2,493 0 20 Aug 2019
Bilinear Graph Networks for Visual Question Answering Dalu Guo Chang Xu Dacheng Tao GNN 75 52 0 23 Jul 2019
Contrastive Multiview Coding Yonglong Tian Dilip Krishnan Phillip Isola SSL 182 2,412 0 13 Jun 2019
Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper) Santiago Castro Devamanyu Hazarika Verónica Pérez-Rosas Roger Zimmermann Rada Mihalcea Soujanya Poria 66 258 0 05 Jun 2019
Multimodal Transformer for Unaligned Multimodal Language Sequences Yao-Hung Hubert Tsai Shaojie Bai Paul Pu Liang J. Zico Kolter Louis-Philippe Morency Ruslan Salakhutdinov 97 1,311 0 01 Jun 2019
What Makes Training Multi-Modal Classification Networks Hard? Weiyao Wang Du Tran Matt Feiszli 154 453 0 29 May 2019
Adaptive Cross-Modal Few-Shot Learning Chen Xing Negar Rostamzadeh Boris N. Oreshkin Pedro H. O. Pinheiro 148 274 0 19 Feb 2019