OneLLM: One Framework to Align All Modalities with Language

10 January 2025

Papers citing "OneLLM: One Framework to Align All Modalities with Language"

50 / 168 papers shown

Title
HoloLLM: Multisensory Foundation Model for Language-Grounded Human Sensing and Reasoning Chuhao Zhou Jianfei Yang VLM 221 0 0 23 May 2025
Learning Interpretable Representations Leads to Semantically Faithful EEG-to-Text Generation Xiaozhao Liu Dinggang Shen Xihui Liu 56 0 0 21 May 2025
Exploring The Visual Feature Space for Multimodal Neural Decoding Weihao Xia Cengiz Öztireli 60 0 0 21 May 2025
Multi-Modality Expansion and Retention for LLMs through Parameter Merging and Decoupling Junlin Li Guodong DU Jing Li Sim Kuan Goh Wenya Wang ... Fangming Liu Jing Li Saleh Alharbi Daojing He Min Zhang MoMe CLL 99 1 0 21 May 2025
Coherent Language Reconstruction from Brain Recordings with Flexible Multi-Modal Input Stimuli Chunyu Ye Shaonan Wang AI4CE 71 0 0 15 May 2025
Token Communication-Driven Multimodal Large Models in Resource-Constrained Multiuser Networks Junhe Zhang Wanli Ni Pengwei Wang Dongyu Wang 45 0 0 06 May 2025
TxP: Reciprocal Generation of Ground Pressure Dynamics and Activity Descriptions for Improving Human Activity Recognition L. Ray Lars Krupp Vitor Fortes Rey Bo Zhou Sungho Suh Paul Lukowicz AI4CE 274 0 0 04 May 2025
An LLM-Empowered Low-Resolution Vision System for On-Device Human Behavior Understanding Siyang Jiang Bufang Yang Lilin Xu Mu Yuan Yeerzhati Abudunuer ... Liekang Zeng Hongkai Chen Zhenyu Yan Xiaofan Jiang Guoliang Xing VLM 298 0 0 03 May 2025
CMT: A Cascade MAR with Topology Predictor for Multimodal Conditional CAD Generation Jianyu Wu Yizhou Wang Xiangyu Yue Xinzhu Ma Jinpei Guo Dongzhan Zhou Wanli Ouyang Shixiang Tang 114 0 0 29 Apr 2025
Multimodal Long Video Modeling Based on Temporal Dynamic Context Haoran Hao Jiaming Han Yiyuan Zhang Xiangyu Yue 92 0 0 14 Apr 2025
PACT: Pruning and Clustering-Based Token Reduction for Faster Visual Language Models M. Dhouib Davide Buscaldi Sonia Vanier A. Shabou VLM 84 1 0 11 Apr 2025
Aligned Better, Listen Better for Audio-Visual Large Language Models Yuxin Guo Shuailei Ma Shijie Ma Xiaoyi Bao Chen-Wei Xie Kecheng Zheng Tingyu Weng Siyang Sun Yun Zheng Wei Zou MLLM AuLLM 97 2 0 02 Apr 2025
Aurelia: Test-time Reasoning Distillation in Audio-Visual LLMs Sanjoy Chowdhury Hanan Gani Nishit Anand Sayan Nag Ruohan Gao Mohamed Elhoseiny Salman Khan Dinesh Manocha LRM 129 1 0 29 Mar 2025
Tokenization of Gaze Data Tim Rolff Jurik Karimian Niklas Hypki S. Schmidt Markus Lappe Frank Steinicke 73 0 0 28 Mar 2025
UniSTD: Towards Unified Spatio-Temporal Learning across Diverse Disciplines Chen Tang Xinzhu Ma Encheng Su Xiufeng Song Xiaohong Liu Wei-Hong Li Lei Bai Wanli Ouyang Xiangyu Yue 3DGS AI4TS 93 0 0 26 Mar 2025
ACVUBench: Audio-Centric Video Understanding Benchmark Yue Yang Jimin Zhuang Guangzhi Sun Changli Tang Yongqian Li P. Li Yifan Jiang W. Li Zejun Ma Chao Zhang AuLLM CoGe 93 0 0 25 Mar 2025
LangBridge: Interpreting Image as a Combination of Language Embeddings Jiaqi Liao Yuwei Niu Fanqing Meng Hao Li Changyao Tian ... Dianqi Li X. Zhu Li Yuan Jifeng Dai Yu Cheng MLLM 101 1 0 25 Mar 2025
LLaVAction: evaluating and training multi-modal large language models for action recognition Shaokai Ye Haozhe Qi Alexander Mathis Mackenzie W. Mathis 106 1 0 24 Mar 2025
CountLLM: Towards Generalizable Repetitive Action Counting via Large Language Model Ziyu Yao Xuxin Cheng Zhiqi Huang Lei Li 123 1 0 22 Mar 2025
From Eye to Mind: brain2text Decoding Reveals the Neural Mechanisms of Visual Semantic Processing Feihan Feng Jingxin Nie 100 0 0 15 Mar 2025
DAVE: Diagnostic benchmark for Audio Visual Evaluation Gorjan Radevski Teodora Popordanoska Matthew B. Blaschko Tinne Tuytelaars 75 0 0 12 Mar 2025
Optimal Transport for Brain-Image Alignment: Unveiling Redundancy and Synergy in Neural Information Processing Yang Xiao Wang Lu Jie Ji Ruimeng Ye Gen Li Xiaolong Ma Bo Hui OT 72 0 0 09 Mar 2025
Merge then Realign: Simple and Effective Modality-Incremental Continual Learning for Multimodal LLMs Dingkun Zhang Shuhan Qi Xinyu Xiao Kehai Chen Xuan Wang CLL MoMe 82 0 0 08 Mar 2025
MM-OR: A Large Multimodal Operating Room Dataset for Semantic Understanding of High-Intensity Surgical Environments Ege Özsoy Chantal Pellegrini Tobias Czempiel Felix Tristram Kun Yuan David Bani-Harouni U. Eck Benjamin Busam Matthias Keicher Nassir Navab 104 4 0 04 Mar 2025
Omni-SILA: Towards Omni-scene Driven Visual Sentiment Identifying, Locating and Attributing in Videos Jiamin Luo Jingjing Wang Junxiao Ma Yujie Jin Shoushan Li Guodong Zhou 61 0 0 26 Feb 2025
Sherlock: Towards Multi-scene Video Abnormal Event Extraction and Localization via a Global-local Spatial-sensitive LLM Junxiao Ma Jingjing Wang Jiamin Luo Peiying Yu Guodong Zhou 85 1 0 26 Feb 2025
Mojito: LLM-Aided Motion Instructor with Jitter-Reduced Inertial Tokens Ziwei Shan Yaoyu He Chengfeng Zhao Jiashen Du Jingyan Zhang Qixuan Zhang Jingyi Yu Lan Xu 80 1 0 22 Feb 2025
MindLLM: A Subject-Agnostic and Versatile Model for fMRI-to-Text Decoding Weikang Qiu Zheng Huang Haoyu Hu Aosong Feng Yujun Yan Rex Ying 75 0 0 18 Feb 2025
3UR-LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understanding Haomiao Xiong Yunzhi Zhuge Jiawen Zhu Lu Zhang Huchuan Lu 65 2 0 14 Jan 2025
OpenOmni: Advancing Open-Source Omnimodal Large Language Models with Progressive Multimodal Alignment and Real-Time Self-Aware Emotional Speech Synthesis Run Luo Ting-En Lin Jun Wang Yuchuan Wu Xiong Liu ... Lei Zhang Yushen Chen Xiaobo Xia Hamid Alinejad-Rokny Fei Huang VLM AuLLM 112 0 0 08 Jan 2025
AllSpark: A Multimodal Spatio-Temporal General Intelligence Model with Ten Modalities via Language as a Reference Framework Run Shao Cheng Yang Qiujun Li Qing Zhu Yongjun Zhang ... Yu Liu Yong Tang Dapeng Liu Shizhong Yang Haifeng Li 120 0 0 08 Jan 2025
A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine Hanguang Xiao Feizhong Zhou Xianglong Liu Tianqi Liu Zhipeng Li Xin Liu Xiaoxuan Huang AILaw LM&MA LRM 95 25 0 31 Dec 2024
Bag of Tricks for Multimodal AutoML with Image, Text, and Tabular Data Zhiqiang Tang Zihan Zhong Tong He Gerald Friedland 128 1 0 19 Dec 2024
Deploying Foundation Model Powered Agent Services: A Survey Wenchao Xu Jinyu Chen Peirong Zheng Xiaoquan Yi Tianyi Tian ... Quan Wan Yining Qi Yunfeng Fan Qinliang Su Xuemin Shen AI4CE 156 2 0 18 Dec 2024
GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training Renqiu Xia Mingxing Li Hancheng Ye Wenjie Wu Hongbin Zhou ... Zeang Sheng Botian Shi Tao Chen Junchi Yan Bo Zhang 128 10 0 16 Dec 2024
ViSymRe: Vision-guided Multimodal Symbolic Regression Da Li Junping Yin Jin Xu Xinxin Li Juan Zhang 102 1 0 15 Dec 2024
InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions Pan Zhang Xiaoyi Dong Yuhang Cao Yuhang Zang Rui Qian ... Xinsong Zhang Kai Chen Yu Qiao Dahua Lin Jiaqi Wang KELM 131 15 0 12 Dec 2024
AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information? Kaixiong Gong Kaituo Feng Yangqiu Song Yibing Wang Mofan Cheng ... Jiaming Han Benyou Wang Yutong Bai Zhiyong Yang Xiangyu Yue MLLM AuLLM VLM 107 11 0 03 Dec 2024
Enhancing Perception Capabilities of Multimodal LLMs with Training-Free Fusion Zhuokun Chen Jinwu Hu Zeshuai Deng Yufeng Wang Bohan Zhuang Mingkui Tan 100 0 0 02 Dec 2024
PSA-VLM: Enhancing Vision-Language Model Safety through Progressive Concept-Bottleneck-Driven Alignment Zhendong Liu Yuanbi Nie Yingshui Tan Xiangyu Yue Qiushi Cui Chongjun Wang Xiaoyong Zhu Jian Xu Bo Zheng 101 0 0 18 Nov 2024
Spider: Any-to-Many Multimodal LLM Jinxiang Lai Jie Zhang Jun Liu Jian Li Xiaocheng Lu Song Guo MLLM 122 2 0 14 Nov 2024
Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Engines Zhixin Zhang Yiyuan Zhang Xiaohan Ding Xiangyu Yue 53 4 0 28 Oct 2024
AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models Kim Sung-Bin Oh Hyun-Bin JungMok Lee Arda Senocak Joon Son Chung Tae-Hyun Oh MLLM VLM 89 5 0 23 Oct 2024
RAP: Retrieval-Augmented Personalization for Multimodal Large Language Models Haoran Hao Jiaming Han Changsheng Li Yu-Feng Li Xiangyu Yue RALM 72 1 0 17 Oct 2024
X-Fi: A Modality-Invariant Foundation Model for Multimodal Human Sensing Xinyan Chen Jianfei Yang 78 2 0 14 Oct 2024
Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulation Qingwen Bu Hongyang Li Li Chen Jisong Cai Jia Zeng Heming Cui Maoqing Yao Yu Qiao 92 8 0 10 Oct 2024
Open-vocabulary Multimodal Emotion Recognition: Dataset, Metric, and Benchmark Zheng Lian Haiyang Sun Guoying Zhao Lan Chen Haoyu Chen ... Rui Liu Shan Liang Ya Li Jiangyan Yi Jianhua Tao VLM 91 0 0 02 Oct 2024
OneEncoder: A Lightweight Framework for Progressive Alignment of Modalities Bilal Faye Hanane Azzag M. Lebbah ObjD 80 0 0 17 Sep 2024
IVGF: The Fusion-Guided Infrared and Visible General Framework Fangcen Liu Chenqiang Gao Fang Chen Pengcheng Li Junjie Guo Deyu Meng 102 0 0 02 Sep 2024
More Text, Less Point: Towards 3D Data-Efficient Point-Language Understanding Yuan Tang Xu Han Xianzhi Li Qiao Yu Jinfeng Xu Yixue Hao Long Hu Min Chen 109 2 0 28 Aug 2024