Empirical Study on Updating Key-Value Memories in Transformer Feed-forward Layers

19 February 2024

Papers citing "Empirical Study on Updating Key-Value Memories in Transformer Feed-forward Layers"

8 / 8 papers shown

Title
Human-inspired Perspectives: A Survey on AI Long-term Memory Zihong He Weizhe Lin Hao Zheng Fan Zhang Matt Jones Laurence Aitchison X. Xu Miao Liu Per Ola Kristensson Junxiao Shen 177 3 0 01 Nov 2024
Layerwise Recurrent Router for Mixture-of-Experts Zihan Qiu Zeyu Huang Shuang Cheng Yizhi Zhou Zili Wang Ivan Titov Jie Fu MoE 117 2 0 13 Aug 2024
Transformer-Patcher: One Mistake worth One Neuron Zeyu Huang Songlin Yang Xiaofeng Zhang Jie Zhou Wenge Rong Zhang Xiong KELM 86 175 0 24 Jan 2023
Mass-Editing Memory in a Transformer Kevin Meng Arnab Sen Sharma A. Andonian Yonatan Belinkov David Bau KELM VLM 121 584 0 13 Oct 2022
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning V. Aribandi Yi Tay Tal Schuster J. Rao H. Zheng ... Jianmo Ni Jai Gupta Kai Hui Sebastian Ruder Donald Metzler MoE 77 215 0 22 Nov 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 184 3,743 0 03 Sep 2021
Editing Factual Knowledge in Language Models Nicola De Cao Wilker Aziz Ivan Titov KELM 112 504 0 16 Apr 2021
Transformer Feed-Forward Layers Are Key-Value Memories Mor Geva R. Schuster Jonathan Berant Omer Levy KELM 149 828 0 29 Dec 2020