UnitNorm: Rethinking Normalization for Transformers in Time Series

UnitNorm: Rethinking Normalization for Transformers in Time Series

24 May 2024

Xiang Zhang

Papers citing "UnitNorm: Rethinking Normalization for Transformers in Time Series"

6 / 6 papers shown

Title
TOKON: TOKenization-Optimized Normalization for time series analysis with a large language model Janghoon Yang AI4TS 77 0 0 08 Feb 2025
RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation Songming Liu Lingxuan Wu Bangguo Li Hengkai Tan Huayu Chen Zhengyi Wang Ke Xu Hang Su Jun Zhu 34 77 0 10 Oct 2024
On the Expressivity Role of LayerNorm in Transformers' Attention Shaked Brody Shiyu Jin Xinghao Zhu MoE 63 30 0 04 May 2023
Stabilizing Transformer Training by Preventing Attention Entropy Collapse Shuangfei Zhai Tatiana Likhomanenko Etai Littwin Dan Busbridge Jason Ramapuram Yizhe Zhang Jiatao Gu J. Susskind AAML 46 64 0 11 Mar 2023
Incorporating Residual and Normalization Layers into Analysis of Masked Language Models Goro Kobayashi Tatsuki Kuribayashi Sho Yokoi Kentaro Inui 160 46 0 15 Sep 2021
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting Haoyi Zhou Shanghang Zhang J. Peng Shuai Zhang Jianxin Li Hui Xiong Wan Zhang AI4TS 169 3,885 0 14 Dec 2020