LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention

28 March 2023

Yu Qiao

Papers citing "LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention"

50 / 588 papers shown

Title
ChartLlama: A Multimodal LLM for Chart Understanding and Generation Yucheng Han C. Zhang Xin Chen Xu Yang Zhibin Wang Gang Yu Bin-Bin Fu Hanwang Zhang MLLM 22 94 0 27 Nov 2023
Boosting the Power of Small Multimodal Reasoning Models to Match Larger Models with Self-Consistency Training Cheng Tan Jingxuan Wei Zhangyang Gao Linzhuang Sun Siyuan Li Ruifeng Guo Xihong Yang Stan Z. Li LRM 31 7 0 23 Nov 2023
Physical Reasoning and Object Planning for Household Embodied Agents Ayush Agrawal Raghav Prabhakar Anirudh Goyal Dianbo Liu LM&Ro LRM 13 0 0 22 Nov 2023
Vamos: Versatile Action Models for Video Understanding Shijie Wang Qi Zhao Minh Quan Do Nakul Agarwal Kwonjoon Lee Chen Sun 27 19 0 22 Nov 2023
ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization Prateek Yadav Leshem Choshen Colin Raffel Mohit Bansal 32 13 0 22 Nov 2023
Multimodal Large Language Models: A Survey Jiayang Wu Wensheng Gan Zefeng Chen Shicheng Wan Philip S. Yu 36 169 0 22 Nov 2023
Towards Better Parameter-Efficient Fine-Tuning for Large Language Models: A Position Paper Chengyu Wang Junbing Yan Wei Zhang Jun Huang ALM 42 3 0 22 Nov 2023
ShareGPT4V: Improving Large Multi-Modal Models with Better Captions Lin Chen Jinsong Li Xiao-wen Dong Pan Zhang Conghui He Jiaqi Wang Feng Zhao Dahua Lin MLLM VLM 58 583 0 21 Nov 2023
A Survey of Graph Meets Large Language Model: Progress and Future Directions Yuhan Li Zhixun Li Peisong Wang Jia Li Xiangguo Sun Hongtao Cheng Jeffrey Xu Yu 40 56 0 21 Nov 2023
LION : Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge Gongwei Chen Leyang Shen Rui Shao Xiang Deng Liqiang Nie VLM MLLM 70 42 0 20 Nov 2023
Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents Zhuosheng Zhang Yao Yao Aston Zhang Xiangru Tang Xinbei Ma ... Yiming Wang Mark B. Gerstein Rui Wang Gongshen Liu Hai Zhao LLMAG LM&Ro LRM 42 53 0 20 Nov 2023
Token-Level Adaptation of LoRA Adapters for Downstream Task Generalization Joshua Belofsky MoMe 19 13 0 17 Nov 2023
Modality-invariant and Specific Prompting for Multimodal Human Perception Understanding Hao Sun Ziwei Niu Xinyao Yu Jiaqing Liu Yen-Wei Chen Lanfen Lin 29 0 0 17 Nov 2023
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection Bin Lin Yang Ye Bin Zhu Jiaxi Cui Munan Ning Peng Jin Li-ming Yuan VLM MLLM 197 595 0 16 Nov 2023
SiRA: Sparse Mixture of Low Rank Adaptation Yun Zhu Nevan Wichers Chu-Cheng Lin Xinyi Wang Tianlong Chen ... Han Lu Canoee Liu Liangchen Luo Jindong Chen Lei Meng MoE 25 27 0 15 Nov 2023
Unlock the Power: Competitive Distillation for Multi-Modal Large Language Models Xinwei Li Li Lin Shuai Wang Chen Qian 9 3 0 14 Nov 2023
Vision-Language Instruction Tuning: A Review and Analysis Chen Li Yixiao Ge Dian Li Ying Shan VLM 39 12 0 14 Nov 2023
Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding Peng Jin Ryuichi Takanobu Caiwan Zhang Xiaochun Cao Li-ming Yuan MLLM 36 226 0 14 Nov 2023
SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models Ziyi Lin Chris Liu Renrui Zhang Peng Gao Longtian Qiu ... Siyuan Huang Yichi Zhang Xuming He Hongsheng Li Yu Qiao MLLM VLM 33 210 0 13 Nov 2023
A Comprehensive Evaluation of GPT-4V on Knowledge-Intensive Visual Question Answering Yunxin Li Longyue Wang Baotian Hu Xinyu Chen Wanqi Zhong Chenyang Lyu Wei Wang Min Zhang ELM 32 21 0 13 Nov 2023
Speech-based Slot Filling using Large Language Models Guangzhi Sun Shutong Feng Dongcheng Jiang Chao Zhang Milica Gasic P. Woodland 31 1 0 13 Nov 2023
Detecting and Correcting Hate Speech in Multimodal Memes with Large Visual Language Model Minh-Hao Van Xintao Wu VLM MLLM 33 10 0 12 Nov 2023
On the steerability of large language models toward data-driven personas Junyi Li Ninareh Mehrabi Charith Peris Palash Goyal Kai-Wei Chang Aram Galstyan Richard Zemel Rahul Gupta 31 29 0 08 Nov 2023
TEAL: Tokenize and Embed ALL for Multi-modal Large Language Models Zhen Yang Yingxue Zhang Fandong Meng Jie Zhou VLM MLLM 47 3 0 08 Nov 2023
Meta-Adapter: An Online Few-shot Learner for Vision-Language Model Cheng Cheng Lin Song Ruoyi Xue Hang Wang Hongbin Sun Yixiao Ge Ying Shan VLM ObjD 39 19 0 07 Nov 2023
GLaMM: Pixel Grounding Large Multimodal Model H. Rasheed Muhammad Maaz Sahal Shaji Mullappilly Abdelrahman M. Shaker Salman Khan Hisham Cholakkal Rao M. Anwer Erix Xing Ming-Hsuan Yang Fahad S. Khan MLLM VLM 47 203 0 06 Nov 2023
Octavius: Mitigating Task Interference in MLLMs via LoRA-MoE Zeren Chen Ziqin Wang Zhen Wang Huayang Liu Zhen-fei Yin Si Liu Lu Sheng Wanli Ouyang Yu Qiao Jing Shao MoE 36 7 0 05 Nov 2023
Multimodal Foundation Models for Zero-shot Animal Species Recognition in Camera Trap Images Zalan Fabian Zhongqi Miao Chunyuan Li Yuanhan Zhang Ziwei Liu ... Laura Siabatto Andrés Link Pablo Arbelaez Rahul Dodhia J. L. Ferres 44 10 0 02 Nov 2023
From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities Md Farhan Ishmam Md Sakib Hossain Shovon M. F. Mridha Nilanjan Dey 43 36 0 01 Nov 2023
Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning Ruizhe Shi Yuyao Liu Yanjie Ze Simon S. Du Huazhe Xu OffRL RALM 34 18 0 31 Oct 2023
ControlLLM: Augment Language Models with Tools by Searching on Graphs Zhaoyang Liu Zeqiang Lai Zhangwei Gao Erfei Cui Ziheng Li ... Lewei Lu Qifeng Chen Yu Qiao Jifeng Dai Wenhai Wang MLLM 128 30 0 26 Oct 2023
Content-based Controls For Music Large Language Modeling Liwei Lin Gus Xia Junyan Jiang Yixiao Zhang 18 14 0 26 Oct 2023
JudgeLM: Fine-tuned Large Language Models are Scalable Judges Lianghui Zhu Xinggang Wang Xinlong Wang ELM ALM 59 110 0 26 Oct 2023
DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Models Ge Zheng Bin Yang Jiajin Tang Hong-Yu Zhou Sibei Yang LRM MLLM 35 93 0 25 Oct 2023
Large Language Models are Temporal and Causal Reasoners for Video Question Answering Dohwan Ko Ji Soo Lee Wooyoung Kang Byungseok Roh Hyunwoo J. Kim LRM 33 31 0 24 Oct 2023
Frozen Transformers in Language Models Are Effective Visual Encoder Layers Ziqi Pang Ziyang Xie Yunze Man Yu-xiong Wang 53 25 0 19 Oct 2023
Know Where to Go: Make LLM a Relevant, Responsible, and Trustworthy Searcher Xiang Shi Jiawei Liu Yinpeng Liu Qikai Cheng Wei Lu RALM HILM KELM 27 6 0 19 Oct 2023
Non-Intrusive Adaptation: Input-Centric Parameter-efficient Fine-Tuning for Versatile Multimodal Modeling Yaqing Wang Jialin Wu T. Dabral Jiageng Zhang Geoff Brown ... Frederick Liu Yi Liang Bo Pang Michael Bendersky Radu Soricut VLM 25 14 0 18 Oct 2023
Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V Jianwei Yang Hao Zhang Feng Li Xueyan Zou Chun-yue Li Jianfeng Gao MLLM VLM 35 159 0 17 Oct 2023
Towards Training-free Open-world Segmentation via Image Prompt Foundation Models Lv Tang Peng-Tao Jiang Haoke Xiao Bo Li VLM 18 8 0 17 Oct 2023
Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks Erfan Shayegani Md Abdullah Al Mamun Yu Fu Pedram Zaree Yue Dong Nael B. Abu-Ghazaleh AAML 147 148 0 16 Oct 2023
JM3D & JM3D-LLM: Elevating 3D Understanding with Joint Multi-modal Cues Jiayi Ji Haowei Wang Changli Wu Yiwei Ma Xiaoshuai Sun Rongrong Ji 49 1 0 14 Oct 2023
EasyGen: Easing Multimodal Generation with BiDiffuser and LLMs Xiangyu Zhao Bo Liu Qijiong Liu Guangyuan Shi Xiao-Ming Wu VLM DiffM 26 7 0 13 Oct 2023
Ziya-Visual: Bilingual Large Vision-Language Model via Multi-Task Instruction Tuning Junyu Lu Di Zhang Xiaojun Wu Xinyu Gao Ruyi Gan Jiaxing Zhang Yan Song Pingjian Zhang VLM MLLM 17 7 0 12 Oct 2023
Improving Compositional Text-to-image Generation with Large Vision-Language Models Song Wen Guian Fang Renrui Zhang Peng Gao Hao Dong Dimitris N. Metaxas 25 17 0 10 Oct 2023
MuseChat: A Conversational Music Recommendation System for Videos Zhikang Dong Bin Chen Xiulong Liu Paweł Polak Peng Zhang LRM 45 26 0 10 Oct 2023
Transformers and Large Language Models for Chemistry and Drug Discovery Andres M Bran Philippe Schwaller LM&MA MedIm AI4CE 38 14 0 09 Oct 2023
SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF Yi Dong Zhilin Wang Makesh Narsimhan Sreedhar Xianchao Wu Oleksii Kuchaiev ALM LLMSV 34 65 0 09 Oct 2023
Toolink: Linking Toolkit Creation and Using through Chain-of-Solving on Open-Source Model Cheng Qian Chenyan Xiong Zhenghao Liu Zhiyuan Liu LRM 29 12 0 08 Oct 2023
AvalonBench: Evaluating LLMs Playing the Game of Avalon Jonathan Light Min Cai Sheng Shen Ziniu Hu LLMAG 17 0 0 08 Oct 2023