Recurrent Memory Transformer

14 July 2022

Papers citing "Recurrent Memory Transformer"

36 / 36 papers shown

Title
Towards LifeSpan Cognitive Systems Yu Wang Chi Han Tongtong Wu Xiaoxin He Wangchunshu Zhou ... Zexue He Wei Wang Gholamreza Haffari Heng Ji Julian McAuley KELM CLL 386 2 0 20 Sep 2024
MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory Ali Modarressi Abdullatif Köksal Ayyoob Imani Mohsen Fayyaz Hinrich Schütze KELM 137 10 0 17 Apr 2024
Investigating Recurrent Transformers with Dynamic Halt Jishnu Ray Chowdhury Cornelia Caragea 67 1 0 01 Feb 2024
ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attention Yang Liu Jiaxiang Liu L. Chen Yuxiang Lu Shi Feng Zhida Feng Yu Sun Hao Tian Huancheng Wu Hai-feng Wang 49 9 0 23 Mar 2022
Contrastive Document Representation Learning with Graph Attention Networks Peng Xu Xinchi Chen Xiaofei Ma Zhiheng Huang Bing Xiang 32 9 0 20 Oct 2021
Perceiver IO: A General Architecture for Structured Inputs & Outputs Andrew Jaegle Sebastian Borgeaud Jean-Baptiste Alayrac Carl Doersch Catalin Ionescu ... Olivier J. Hénaff M. Botvinick Andrew Zisserman Oriol Vinyals João Carreira MLLM VLM GNN 48 572 0 30 Jul 2021
Staircase Attention for Recurrent Processing of Sequences Da Ju Stephen Roller Sainbayar Sukhbaatar Jason Weston 34 11 0 08 Jun 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 460 3,952 0 18 Apr 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 327 4,873 0 24 Feb 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 174 4,209 0 01 Jan 2021
ERNIE-Doc: A Retrospective Long-Document Modeling Transformer Siyu Ding Junyuan Shang Shuohuan Wang Yu Sun Hao Tian Hua Wu Haifeng Wang 90 54 0 31 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 403 40,217 0 22 Oct 2020
Rethinking Attention with Performers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Afroz Mohiuddin Lukasz Kaiser David Belanger Lucy J. Colwell Adrian Weller 144 1,548 0 30 Sep 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 484 2,051 0 28 Jul 2020
Memory Transformer Andrey Kravchenko Yuri Kuratov Anton Peganov Grigory V. Sapunov RALM 41 70 0 20 Jun 2020
Linformer: Self-Attention with Linear Complexity Sinong Wang Belinda Z. Li Madian Khabsa Han Fang Hao Ma 170 1,678 0 08 Jun 2020
GMAT: Global Memory Augmentation for Transformers Ankit Gupta Jonathan Berant RALM 26 50 0 05 Jun 2020
MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioning Jie Lei Liwei Wang Yelong Shen Dong Yu Tamara L. Berg Joey Tianyi Zhou 47 189 0 11 May 2020
ETC: Encoding Long and Structured Inputs in Transformers Joshua Ainslie Santiago Ontanon Chris Alberti Vaclav Cvicek Zachary Kenneth Fisher Philip Pham Anirudh Ravula Sumit Sanghai Qifan Wang Li Yang 45 54 0 17 Apr 2020
Longformer: The Long-Document Transformer Iz Beltagy Matthew E. Peters Arman Cohan RALM VLM 100 3,996 0 10 Apr 2020
Compressive Transformers for Long-Range Sequence Modelling Jack W. Rae Anna Potapenko Siddhant M. Jayakumar Timothy Lillicrap RALM VLM KELM 55 636 0 13 Nov 2019
Large Memory Layers with Product Keys Guillaume Lample Alexandre Sablayrolles MarcÁurelio Ranzato Ludovic Denoyer Hervé Jégou MoE 44 131 0 10 Jul 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 76 1,880 0 23 Apr 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 140 3,714 0 09 Jan 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 969 93,936 0 11 Oct 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 453 129,831 0 12 Jun 2017
Memory Augmented Neural Networks with Wormhole Connections Çağlar Gülçehre A. Chandar Yoshua Bengio 68 63 0 30 Jan 2017
Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes Jack W. Rae Jonathan J. Hunt Tim Harley Ivo Danihelka A. Senior Greg Wayne Alex Graves Timothy Lillicrap 35 158 0 27 Oct 2016
Using Fast Weights to Attend to the Recent Past Jimmy Ba Geoffrey E. Hinton Volodymyr Mnih Joel Z Leibo Catalin Ionescu 45 267 0 20 Oct 2016
Pointer Sentinel Mixture Models Stephen Merity Caiming Xiong James Bradbury R. Socher RALM 168 2,814 0 26 Sep 2016
Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes Çağlar Gülçehre A. Chandar Kyunghyun Cho Yoshua Bengio 94 64 0 30 Jun 2016
Learning to Transduce with Unbounded Memory Edward Grefenstette Karl Moritz Hermann Mustafa Suleyman Phil Blunsom 65 297 0 08 Jun 2015
Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets Armand Joulin Tomas Mikolov TPM 103 411 0 03 Mar 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 859 149,474 0 22 Dec 2014
Neural Turing Machines Alex Graves Greg Wayne Ivo Danihelka 78 2,318 0 20 Oct 2014
On the Properties of Neural Machine Translation: Encoder-Decoder Approaches Kyunghyun Cho B. V. Merrienboer Dzmitry Bahdanau Yoshua Bengio AI4CE AIMat 162 6,760 0 03 Sep 2014