ResiDual: Transformer with Dual Residual Connections

28 April 2023

Junliang Guo

Xu Tan

Jiang Bian

Rui Yan

Papers citing "ResiDual: Transformer with Dual Residual Connections"

4 / 4 papers shown

Title
HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization Zhijian Zhuo Yutao Zeng Ya Wang Sijun Zhang Jian Yang Xiaoqing Li Xun Zhou Jinwen Ma 51 0 0 06 Mar 2025
More Expressive Attention with Negative Weights Ang Lv Ruobing Xie Shuaipeng Li Jiayi Liao Xingchen Sun Zhanhui Kang Di Wang Rui Yan 39 0 0 11 Nov 2024
Hyper-Connections Defa Zhu Hongzhi Huang Zihao Huang Yutao Zeng Yunyao Mao Banggu Wu Qiyang Min Xun Zhou 35 3 0 29 Sep 2024
Centered Self-Attention Layers Ameen Ali Tomer Galanti Lior Wolf 28 6 0 02 Jun 2023