ImageBind: One Embedding Space To Bind Them All

9 May 2023

Kalyan Vasudev Alwala

Papers citing "ImageBind: One Embedding Space To Bind Them All"

50 / 172 papers shown

Title
Human-inspired Perspectives: A Survey on AI Long-term Memory Zihong He Weizhe Lin Hao Zheng Fan Zhang Matt Jones Laurence Aitchison X. Xu Miao Liu Per Ola Kristensson Junxiao Shen 77 2 0 01 Nov 2024
GiVE: Guiding Visual Encoder to Perceive Overlooked Information Junjie Li Jianghong Ma Xiaofeng Zhang Yuhang Li Jianyang Shi 43 0 0 26 Oct 2024
AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models Kim Sung-Bin Oh Hyun-Bin JungMok Lee Arda Senocak Joon Son Chung Tae-Hyun Oh MLLM VLM 48 3 0 23 Oct 2024
CLaMP 2: Multimodal Music Information Retrieval Across 101 Languages Using Large Language Models Shangda Wu Yashan Wang Ruibin Yuan Zhancheng Guo Xu Tan ... Yuanliang Dong Jiafeng Liu Xiaobing Li Feng Yu Maosong Sun 36 4 0 17 Oct 2024
MultiVENT 2.0: A Massive Multilingual Benchmark for Event-Centric Video Retrieval Reno Kriz Kate Sanders David Etter Kenton W. Murray Cameron Carpenter ... Alexander Martin Ronald Colaianni Nolan King Eugene Yang Benjamin Van Durme VGen 45 2 0 15 Oct 2024
X-Fi: A Modality-Invariant Foundation Model for Multimodal Human Sensing Xinyan Chen Jianfei Yang 48 1 0 14 Oct 2024
An Eye for an Ear: Zero-shot Audio Description Leveraging an Image Captioner using Audiovisual Distribution Alignment Hugo Malard Michel Olvera Stéphane Lathuilière S. Essid VLM 39 0 0 08 Oct 2024
Art2Mus: Bridging Visual Arts and Music through Cross-Modal Generation Ivan Rinaldi Nicola Fanelli Giovanna Castellano G. Vessio 31 2 0 07 Oct 2024
MDSGen: Fast and Efficient Masked Diffusion Temporal-Aware Transformers for Open-Domain Sound Generation T. Pham Tri Ton Chang D. Yoo 49 3 0 03 Oct 2024
Anchors Aweigh! Sail for Optimal Unified Multi-Modal Representations Minoh Jeong Min Namgung Zae Myung Kim Dongyeop Kang Yao-Yi Chiang Alfred Hero 30 0 0 02 Oct 2024
Image-guided topic modeling for interpretable privacy classification Alina Elena Baia Andrea Cavallaro 42 0 0 27 Sep 2024
MIO: A Foundation Model on Multimodal Tokens Zekun Wang King Zhu Chunpu Xu Wangchunshu Zhou Jiaheng Liu ... Yuanxing Zhang Ge Zhang Ke Xu Jie Fu Wenhao Huang MLLM AuLLM 60 11 0 26 Sep 2024
Robotic Environmental State Recognition with Pre-Trained Vision-Language Models and Black-Box Optimization Kento Kawaharazuka Yoshiki Obinata Naoaki Kanazawa Kei Okada Masayuki Inaba LM&Ro 30 0 0 26 Sep 2024
A Simple but Strong Baseline for Sounding Video Generation: Effective Adaptation of Audio and Video Diffusion Models for Joint Generation Masato Ishii Akio Hayakawa Takashi Shibuya Yuki Mitsufuji VGen DiffM 68 4 0 26 Sep 2024
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions Kai Chen Yunhao Gou Runhui Huang Zhili Liu Daxin Tan ... Qun Liu Jun Yao Lu Hou Hang Xu Hang Xu AuLLM MLLM VLM 82 22 0 26 Sep 2024
Measuring Sound Symbolism in Audio-visual Models Wei-Cheng Tseng Yi-Jen Shih David Harwath Raymond Mooney 37 0 0 18 Sep 2024
One missing piece in Vision and Language: A Survey on Comics Understanding Emanuele Vivoli Andrey Barsky Mohamed Ali Souibgui Artemis LLabres Marco Bertini Dimosthenis Karatzas 42 4 0 14 Sep 2024
Enhancing Long Video Understanding via Hierarchical Event-Based Memory Dingxin Cheng Mingda Li Jingyu Liu Yongxin Guo Bin Jiang Qingbin Liu Xi Chen Bo Zhao 38 4 0 10 Sep 2024
Multi-Modal Adapter for Vision-Language Models Dominykas Seputis Serghei Mihailov Soham Chatterjee Zehao Xiao VLM 36 1 0 03 Sep 2024
YOLOO: You Only Learn from Others Once Lipeng Gu Mingqiang Wei Xuefeng Yan Dingkun Zhu Wei Zhao H. Xie Yang Liu 3DPC 44 0 0 01 Sep 2024
From Latent to Engine Manifolds: Analyzing ImageBind's Multimodal Embedding Space Andrew Hamara Pablo Rivas 18 1 0 30 Aug 2024
D&M: Enriching E-commerce Videos with Sound Effects by Key Moment Detection and SFX Matching Jingyu Liu Minquan Wang Ye Ma Bo Wang Aozhu Chen Quan Chen Peng Jiang Xirong Li 48 1 0 23 Aug 2024
Audio Match Cutting: Finding and Creating Matching Audio Transitions in Movies and Videos Dennis Fedorishin Lie Lu S. Setlur Venu Govindaraju VGen 52 3 0 20 Aug 2024
PhishAgent: A Robust Multimodal Agent for Phishing Webpage Detection Tri Cao Chengyu Huang Yuexin Li Huilin Wang Amy He Nay Oo Bryan Hooi LLMAG OffRL 83 5 0 20 Aug 2024
A Markov Random Field Multi-Modal Variational AutoEncoder Fouad Oubari M. Baha Raphael Meunier Rodrigue Décatoire Mathilde Mougeot 36 0 0 18 Aug 2024
End-to-end Semantic-centric Video-based Multimodal Affective Computing Ronghao Lin Ying Zeng Sijie Mai Haifeng Hu VGen 48 0 0 14 Aug 2024
How Does Audio Influence Visual Attention in Omnidirectional Videos? Database and Model Yuxin Zhu Huiyu Duan Kaiwei Zhang Yucheng Zhu Xilei Zhu Long Teng Xiongkuo Min Guangtao Zhai 72 2 0 10 Aug 2024
Audio-visual training for improved grounding in video-text LLMs Shivprasad Sagare Hemachandran S Kinshuk Sarabhai Prashant Ullegaddi SA Rajeshkumar 30 0 0 21 Jul 2024
Aligning Sight and Sound: Advanced Sound Source Localization Through Audio-Visual Alignment Arda Senocak H. Ryu Junsik Kim Tae-Hyun Oh Hanspeter Pfister Joon Son Chung 38 3 0 18 Jul 2024
Large Visual-Language Models Are Also Good Classifiers: A Study of In-Context Multimodal Fake News Detection Ye Jiang Yimin Wang MLLM 38 1 0 16 Jul 2024
FabGPT: An Efficient Large Multimodal Model for Complex Wafer Defect Knowledge Queries Yuqi Jiang Xudong Lu Qian Jin Qi Sun Hanming Wu Cheng Zhuo 48 6 0 15 Jul 2024
Read, Watch and Scream! Sound Generation from Text and Video Yujin Jeong Yunji Kim Sanghyuk Chun Jiyoung Lee VGen DiffM 34 12 0 08 Jul 2024
Sequential Contrastive Audio-Visual Learning Ioannis Tsiamas Santiago Pascual Chunghsin Yeh Joan Serrà 47 2 0 08 Jul 2024
GUIDE: A Guideline-Guided Dataset for Instructional Video Comprehension Jiafeng Liang Shixin Jiang Zekun Wang Haojie Pan Zerui Chen Zheng Chu Ming Liu Ruiji Fu Zhongyuan Wang Bing Qin 34 2 0 26 Jun 2024
Sports Intelligence: Assessing the Sports Understanding Capabilities of Language Models through Question Answering from Text to Video Zhengbang Yang Haotian Xia Jingxi Li Zezhi Chen Zhuangdi Zhu Weining Shen ELM LRM 47 1 0 21 Jun 2024
AnoPatch: Towards Better Consistency in Machine Anomalous Sound Detection Anbai Jiang Bing Han Zhiqiang Lv Yufeng Deng Wei-Qiang Zhang Xie Chen Yanmin Qian Jia Liu Pingyi Fan 40 3 0 17 Jun 2024
VideoVista: A Versatile Benchmark for Video Understanding and Reasoning Yunxin Li Xinyu Chen Baotian Hu Longyue Wang Haoyuan Shi Min-Ling Zhang MLLM LRM 56 26 0 17 Jun 2024
Exploring Self-Supervised Multi-view Contrastive Learning for Speech Emotion Recognition with Limited Annotations Bulat Khaertdinov Pedro Jeuris Annanda Sousa Enrique Hortal 38 1 0 12 Jun 2024
Benchmarking Vision-Language Contrastive Methods for Medical Representation Learning Shuvendu Roy Yasaman Parhizkar Franklin Ogidi Vahid Reza Khazaie Michael Colacci Ali Etemad Elham Dolatabadi Arash Afkanpour VLM 52 1 0 11 Jun 2024
FaceGPT: Self-supervised Learning to Chat about 3D Human Faces Haoran Wang Mohit Mendiratta Christian Theobalt Adam Kortylewski 3DH CVBM 31 3 0 11 Jun 2024
RU-AI: A Large Multimodal Dataset for Machine Generated Content Detection Liting Huang Zhihao Zhang Yiran Zhang Xiyue Zhou Shoujin Wang NoLa 46 2 0 07 Jun 2024
MuJo: Multimodal Joint Feature Space Learning for Human Activity Recognition Stefan Gerd Fritsch Cennet Oğuz Vitor Fortes Rey L. Ray Maximilian Kiefer-Emmanouilidis Paul Lukowicz HAI 53 0 0 06 Jun 2024
VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling Zeyue Tian Zhaoyang Liu Ruibin Yuan Jiahao Pan Xiaoqiang Huang Xu Tan Xu Tan Qifeng Chen Yu Guo VGen 104 16 0 06 Jun 2024
BeFA: A General Behavior-driven Feature Adapter for Multimedia Recommendation Qile Fan Penghang Yu Zhiyi Tan Bing-Kun Bao Guanming Lu 37 1 0 01 Jun 2024
X-VILA: Cross-Modality Alignment for Large Language Model Hanrong Ye De-An Huang Yao Lu Zhiding Yu Ming-Yu Liu ... Jan Kautz Song Han Dan Xu Pavlo Molchanov Hongxu Yin MLLM VLM 49 30 0 29 May 2024
OmniBind: Teach to Build Unequal-Scale Modality Interaction for Omni-Bind of All Yuanhuiyi Lyu Xueye Zheng Dahun Kim Lin Wang 51 14 0 25 May 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 82 45 0 23 May 2024
Images that Sound: Composing Images and Sounds on a Single Canvas Ziyang Chen Daniel Geng Andrew Owens DiffM 50 9 0 20 May 2024
Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts Yunxin Li Shenyuan Jiang Baotian Hu Longyue Wang Wanqi Zhong Wenhan Luo Lin Ma Min-Ling Zhang MoE 46 30 0 18 May 2024
EALD-MLLM: Emotion Analysis in Long-sequential and De-identity videos with Multi-modal Large Language Model Deng Li Xin Liu Bohao Xing Baiqiang Xia Yuan Zong Bihan Wen Heikki Kälviäinen 42 3 0 01 May 2024