Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN

18 December 2024

Papers citing "Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN"

3 / 3 papers shown

Title
HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization Zhijian Zhuo Yutao Zeng Ya Wang Sijun Zhang Jian Yang Xiaoqing Li Xun Zhou Jinwen Ma 51 0 0 06 Mar 2025
The Curse of Depth in Large Language Models Wenfang Sun Xinyuan Song Pengxiang Li Lu Yin Yefeng Zheng Shiwei Liu 75 4 0 09 Feb 2025
SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training Tianjin Huang Ziquan Zhu Gaojie Jin Lu Liu Zhangyang Wang Shiwei Liu 47 1 0 12 Jan 2025