OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for
Memory-Efficient LLM Fine-tuning

OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuning

28 May 2024

Lu Yin

Papers citing "OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuning"

6 / 6 papers shown

Title
SmartFRZ: An Efficient Training Framework using Attention-Based Layer Freezing Sheng Li Geng Yuan Yuezhen Dai Youtao Zhang Yanzhi Wang Xulong Tang 31 18 0 30 Jan 2024
Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning Wenhan Xia Chengwei Qin Elad Hazan 57 52 0 08 Jan 2024
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 159 579 0 06 Apr 2023
Outliers Dimensions that Disrupt Transformers Are Driven by Frequency Giovanni Puccetti Anna Rogers Aleksandr Drozd F. Dell’Orletta 76 42 0 23 May 2022
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
WARP: Word-level Adversarial ReProgramming Karen Hambardzumyan Hrant Khachatrian Jonathan May AAML 254 342 0 01 Jan 2021