Mixture-of-Subspaces in Low-Rank Adaptation

Mixture-of-Subspaces in Low-Rank Adaptation

16 June 2024

Ngai Wong

Papers citing "Mixture-of-Subspaces in Low-Rank Adaptation"

12 / 12 papers shown

Title
PaCA: Partial Connection Adaptation for Efficient Fine-Tuning Sunghyeon Woo Sol Namkung Sunwoo Lee Inho Jeong Beomseok Kim Dongsuk Jeon 39 0 0 28 Feb 2025
K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs Ziheng Ouyang Zhen Li Qibin Hou MoMe OffRL 130 2 0 25 Feb 2025
Parameter Efficient Merging for Multimodal Large Language Models with Complementary Parameter Adaptation Fanhu Zeng Haiyang Guo Fei Zhu Li Shen Hao Tang MoMe 54 1 0 24 Feb 2025
Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning Zhen Li Yupeng Su Runming Yang C. Xie Z. Wang Zhongwei Xie Ngai Wong Hongxia Yang MQ LRM 44 3 0 06 Jan 2025
Adaptive Rank, Reduced Forgetting: Knowledge Retention in Continual Learning Vision-Language Models with Dynamic Rank-Selective LoRA Haodong Lu Chongyang Zhao Jason Xue Lina Yao Kristen Moore Dong Gong VLM KELM CLL 85 3 0 01 Dec 2024
LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models Runming Yang Taiqiang Wu Jiahao Wang Pengfei Hu Ngai Wong Yujiu Yang Yujiu Yang 134 0 0 11 Nov 2024
Low-Rank Interconnected Adaptation Across Layers Yibo Zhong Yao Zhou OffRL MoE 40 1 0 13 Jul 2024
Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences Nikolaos Dimitriadis Pascal Frossard F. Fleuret MoE 61 6 0 10 Jul 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 144 308 0 21 Mar 2024
Weight-Inherited Distillation for Task-Agnostic BERT Compression Taiqiang Wu Cheng-An Hou Shanshan Lao Jiayi Li Ngai Wong Zhe Zhao Yujiu Yang 60 10 0 16 May 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 211 1,105 0 20 Sep 2022
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,844 0 18 Apr 2021