Root Mean Square Layer Normalization

16 October 2019

Papers citing "Root Mean Square Layer Normalization"

47 / 97 papers shown

Title
Chain-of-Thought Prompting for Speech Translation Ke Hu Zhehuai Chen Chao-Han Huck Yang Piotr Żelasko Oleksii Hrinchuk Vitaly Lavrukhin Jagadeesh Balam Boris Ginsburg LRM 84 6 0 17 Sep 2024
Flash STU: Fast Spectral Transform Units Y. Isabel Liu Windsor Nguyen Yagiz Devre Evan Dogariu Anirudha Majumdar Elad Hazan AI4TS 86 1 0 16 Sep 2024
MambaFoley: Foley Sound Generation using Selective State-Space Models Marco Furio Colombo Francesca Ronchini Luca Comanducci Fabio Antonacci Mamba 46 1 0 13 Sep 2024
$u-$\mu$P: The Unit-Scaled Maximal Update Parametrization$ u- $\mu$ P: The Unit-Scaled Maximal Update Parametrization Charlie Blake C. Eichenberg Josef Dean Lukas Balles Luke Y. Prince Bjorn Deiseroth Andres Felipe Cruz Salinas Carlo Luschi Samuel Weinbach Douglas Orr 68 10 0 24 Jul 2024
Controlling Space and Time with Diffusion Models Daniel Watson Saurabh Saxena Lala Li Andrea Tagliasacchi David J. Fleet VGen 105 28 0 10 Jul 2024
Video In-context Learning: Autoregressive Transformers are Zero-Shot Video Imitators Wentao Zhang Junliang Guo Tianyu He Li Zhao Linli Xu Jiang Bian 61 3 0 10 Jul 2024
Learning to (Learn at Test Time): RNNs with Expressive Hidden States Yu Sun Xinhao Li Karan Dalal Jiarui Xu Arjun Vikram ... Xinlei Chen Xiaolong Wang Sanmi Koyejo Tatsunori Hashimoto Carlos Guestrin 106 99 0 05 Jul 2024
RouteFinder: Towards Foundation Models for Vehicle Routing Problems Federico Berto Chuanbo Hua Nayeli Gast Zepeda André Hottung N. Wouda Leon Lan Kevin Tierney J. Park Jinkyoo Park 75 13 0 21 Jun 2024
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling Liliang Ren Yang Liu Yadong Lu Yelong Shen Chen Liang Weizhu Chen Mamba 95 63 0 11 Jun 2024
TerDiT: Ternary Diffusion Models with Transformers Xudong Lu Aojun Zhou Ziyi Lin Qi Liu Yuhui Xu Renrui Zhang Yafei Wen Shuai Ren Peng Gao Junchi Yan MQ 71 3 0 23 May 2024
Chameleon: Mixed-Modal Early-Fusion Foundation Models Chameleon Team MLLM 111 290 0 16 May 2024
On the Challenges and Opportunities in Generative AI Laura Manduchi Kushagra Pandey Robert Bamler Ryan Cotterell Sina Daubener ... F. Wenzel Frank Wood Stephan Mandt Vincent Fortuin Vincent Fortuin 128 18 0 28 Feb 2024
FLTrojan: Privacy Leakage Attacks against Federated Language Models Through Selective Weight Tampering Md Rafi Ur Rashid Vishnu Asutosh Dasu Kang Gu Najrin Sultana Shagufta Mehnaz AAML FedML 71 11 0 24 Oct 2023
Baichuan 2: Open Large-scale Language Models Ai Ming Yang Bin Xiao Bingning Wang Borong Zhang Ce Bian ... Youxin Jiang Yuchen Gao Yupeng Zhang Guosheng Dong Zhiying Wu ELM LRM 129 731 0 19 Sep 2023
PowerNorm: Rethinking Batch Normalization in Transformers Sheng Shen Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer BDL 54 16 0 17 Mar 2020
A Lightweight Recurrent Network for Sequence Modeling Biao Zhang Rico Sennrich 29 7 0 30 May 2019
Understanding Batch Normalization Johan Bjorck Carla P. Gomes B. Selman Kilian Q. Weinberger 86 603 0 01 Jun 2018
How Does Batch Normalization Help Optimization? Shibani Santurkar Dimitris Tsipras Andrew Ilyas Aleksander Madry ODL 65 1,531 0 29 May 2018
Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese Shiyu Zhou Linhao Dong Shuang Xu Bo Xu 42 117 0 28 Apr 2018
The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation Mengzhao Chen Orhan Firat Ankur Bapna Melvin Johnson Wolfgang Macherey ... Niki Parmar M. Schuster Zhifeng Chen Yonghui Wu Macduff Hughes AIMat 40 457 0 26 Apr 2018
A Call for Clarity in Reporting BLEU Scores Matt Post 81 2,941 0 23 Apr 2018
Group Normalization Yuxin Wu Kaiming He 129 3,626 0 22 Mar 2018
Norm matters: efficient and accurate normalization schemes in deep networks Elad Hoffer Ron Banner Itay Golan Daniel Soudry OffRL 35 179 0 05 Mar 2018
L1-Norm Batch Normalization for Efficient Training of Deep Neural Networks Shuang Wu Guoqi Li Lei Deng Liu Liu Yuan Xie Luping Shi 32 117 0 27 Feb 2018
Image Transformer Niki Parmar Ashish Vaswani Jakob Uszkoreit Lukasz Kaiser Noam M. Shazeer Alexander Ku Dustin Tran ViT 80 1,671 0 15 Feb 2018
Improving Lexical Choice in Neural Machine Translation Toan Q. Nguyen David Chiang 43 86 0 03 Oct 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 368 129,831 0 12 Jun 2017
Nematus: a Toolkit for Neural Machine Translation Rico Sennrich Orhan Firat Kyunghyun Cho Alexandra Birch Barry Haddow ... Marcin Junczys-Dowmunt Samuel Läubli Antonio Valerio Miceli Barone Jozef Mokry Maria Nadejde 53 407 0 13 Mar 2017
Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models Sergey Ioffe BDL 45 538 0 10 Feb 2017
Instance Normalization: The Missing Ingredient for Fast Stylization Dmitry Ulyanov Andrea Vedaldi Victor Lempitsky OOD 116 3,689 0 27 Jul 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 227 10,412 0 21 Jul 2016
TensorFlow: A system for large-scale machine learning Martín Abadi P. Barham Jianmin Chen Zhiwen Chen Andy Davis ... Vijay Vasudevan Pete Warden Martin Wicke Yuan Yu Xiaoqiang Zhang GNN AI4CE 317 18,300 0 27 May 2016
Theano: A Python framework for fast computation of mathematical expressions The Theano Development Team Rami Al-Rfou Guillaume Alain Amjad Almahairi Christof Angermüller ... Kelvin Xu Lijun Xue Li Yao Saizheng Zhang Ying Zhang 117 2,338 0 09 May 2016
Recurrent Batch Normalization Tim Cooijmans Nicolas Ballas César Laurent Çağlar Gülçehre Aaron Courville ODL 32 410 0 30 Mar 2016
Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks Devansh Arpit Yingbo Zhou Bhargava U. Kota V. Govindaraju 41 127 0 04 Mar 2016
Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks Tim Salimans Diederik P. Kingma ODL 139 1,933 0 25 Feb 2016
Order-Embeddings of Images and Language Ivan Vendrov Ryan Kiros Sanja Fidler R. Urtasun 62 544 0 19 Nov 2015
Batch Normalized Recurrent Neural Networks César Laurent Gabriel Pereyra Philemon Brakel Yanzhe Zhang Yoshua Bengio 47 213 0 05 Oct 2015
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 149 7,683 0 31 Aug 2015
Teaching Machines to Read and Comprehend Karl Moritz Hermann Tomás Kociský Edward Grefenstette L. Espeholt W. Kay Mustafa Suleyman Phil Blunsom 261 3,527 0 10 Jun 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 274 43,154 0 11 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 675 149,474 0 22 Dec 2014
Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models Ryan Kiros Ruslan Salakhutdinov R. Zemel VLM 75 1,395 0 10 Nov 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 786 99,991 0 04 Sep 2014
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 369 27,205 0 01 Sep 2014
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation Kyunghyun Cho B. V. Merrienboer Çağlar Gülçehre Dzmitry Bahdanau Fethi Bougares Holger Schwenk Yoshua Bengio AIMat 578 23,235 0 03 Jun 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 226 43,290 0 01 May 2014