OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for
Memory-Efficient LLM Fine-tuning

OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuning

28 May 2024

Lu Yin

Papers citing "OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuning"

7 / 7 papers shown

Title
The Curse of Depth in Large Language Models Wenfang Sun Xinyuan Song Pengxiang Li Lu Yin Yefeng Zheng Shiwei Liu 75 4 0 09 Feb 2025
SmartFRZ: An Efficient Training Framework using Attention-Based Layer Freezing Sheng Li Geng Yuan Yuezhen Dai Youtao Zhang Yanzhi Wang Xulong Tang 31 18 0 30 Jan 2024
Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning Wenhan Xia Chengwei Qin Elad Hazan 60 55 0 08 Jan 2024
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 162 585 0 06 Apr 2023
Outliers Dimensions that Disrupt Transformers Are Driven by Frequency Giovanni Puccetti Anna Rogers Aleksandr Drozd F. Dell’Orletta 79 42 0 23 May 2022
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,858 0 18 Apr 2021
WARP: Word-level Adversarial ReProgramming Karen Hambardzumyan Hrant Khachatrian Jonathan May AAML 254 342 0 01 Jan 2021