Beyond Zero Initialization: Investigating the Impact of Non-Zero Initialization on LoRA Fine-Tuning Dynamics

29 May 2025

Papers citing "Beyond Zero Initialization: Investigating the Impact of Non-Zero Initialization on LoRA Fine-Tuning Dynamics"

2 / 2 papers shown

Title
Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning Kaustubh Ponkshe Raghav Singhal Eduard A. Gorbunov Alexey Tumanov Samuel Horváth Praneeth Vepakomma 273 7 0 29 Nov 2024
LoRA-Pro: Are Low-Rank Adapters Properly Optimized? Zhengbo Wang Jian Liang Ran He Zilei Wang Tieniu Tan 172 29 0 25 Jul 2024