Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability

8 November 2023

Jishnu Ray Chowdhury

Cornelia Caragea

Papers citing "Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability"

50 / 51 papers shown

Title
Investigating Recurrent Transformers with Dynamic Halt Jishnu Ray Chowdhury Cornelia Caragea 142 1 0 01 Feb 2024
Transformer Working Memory Enables Regular Language Reasoning and Natural Language Length Extrapolation Ta-Chung Chi Ting-Han Fan Alexander I. Rudnicky Peter J. Ramadge LRM 62 13 0 05 May 2023
Resurrecting Recurrent Neural Networks for Long Sequences Antonio Orvieto Samuel L. Smith Albert Gu Anushan Fernando Çağlar Gülçehre Razvan Pascanu Soham De 328 297 0 11 Mar 2023
What Makes Convolutional Models Great on Long Sequence Modeling? Yuhong Li Tianle Cai Yi Zhang De-huai Chen Debadeepta Dey VLM 94 98 0 17 Oct 2022
Liquid Structural State-Space Models Ramin Hasani Mathias Lechner Tsun-Hsuan Wang Makram Chahine Alexander Amini Daniela Rus AI4TS 147 105 0 26 Sep 2022
Recurrent Memory Transformer Aydar Bulatov Yuri Kuratov Andrey Kravchenko CLL 47 111 0 14 Jul 2022
How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections Albert Gu Isys Johnson Aman Timalsina Atri Rudra Christopher Ré Mamba 166 101 0 24 Jun 2022
On the Parameterization and Initialization of Diagonal State Space Models Albert Gu Ankit Gupta Karan Goel Christopher Ré 91 324 0 23 Jun 2022
ChordMixer: A Scalable Neural Attention Model for Sequences with Different Lengths Ruslan Khalitov Tong Yu Lei Cheng Zhirong Yang 81 12 0 12 Jun 2022
Diagonal State Spaces are as Effective as Structured State Spaces Ankit Gupta Albert Gu Jonathan Berant 122 312 0 27 Mar 2022
Block-Recurrent Transformers DeLesley S. Hutchins Imanol Schlag Yuhuai Wu Ethan Dyer Behnam Neyshabur 87 100 0 11 Mar 2022
Fast-R2D2: A Pretrained Recursive Neural Network based on Pruned CKY for Grammar Induction and Text Representation Xiang Hu Haitao Mi Liang Li Gerard de Melo 82 14 0 01 Mar 2022
HiP: Hierarchical Perceiver João Carreira Skanda Koppula Daniel Zoran Adrià Recasens Catalin Ionescu ... M. Botvinick Oriol Vinyals Karen Simonyan Andrew Zisserman Andrew Jaegle VLM 98 14 0 22 Feb 2022
Transformer Quality in Linear Time Weizhe Hua Zihang Dai Hanxiao Liu Quoc V. Le 126 231 0 21 Feb 2022
Sliced Recursive Transformer Zhiqiang Shen Zechun Liu Eric P. Xing ViT 51 27 0 09 Nov 2021
Hierarchical Transformers Are More Efficient Language Models Piotr Nawrot Szymon Tworkowski Michał Tyrolski Lukasz Kaiser Yuhuai Wu Christian Szegedy Henryk Michalewski 73 69 0 26 Oct 2021
Learning Hierarchical Structures with Differentiable Nondeterministic Stacks Brian DuSell David Chiang BDL 53 15 0 05 Sep 2021
Combiner: Full Attention Transformer with Sparse Computation Cost Hongyu Ren H. Dai Zihang Dai Mengjiao Yang J. Leskovec Dale Schuurmans Bo Dai 146 80 0 12 Jul 2021
R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling Xiang Hu Haitao Mi Zujie Wen Yafang Wang Yi Su Jing Zheng Gerard de Melo 51 23 0 02 Jul 2021
Staircase Attention for Recurrent Processing of Sequences Da Ju Stephen Roller Sainbayar Sukhbaatar Jason Weston 74 11 0 08 Jun 2021
Hi-Transformer: Hierarchical Interactive Transformer for Efficient and Effective Long Document Modeling Chuhan Wu Fangzhao Wu Tao Qi Yongfeng Huang 109 68 0 02 Jun 2021
Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding Zizhao Zhang Han Zhang Long Zhao Ting Chen Sercan O. Arik Tomas Pfister ViT 66 173 0 26 May 2021
Do Transformer Modifications Transfer Across Implementations and Applications? Sharan Narang Hyung Won Chung Yi Tay W. Fedus Thibault Févry ... Wei Li Nan Ding Jake Marcus Adam Roberts Colin Raffel 80 127 0 23 Feb 2021
Long Range Arena: A Benchmark for Efficient Transformers Yi Tay Mostafa Dehghani Samira Abnar Songlin Yang Dara Bahri Philip Pham J. Rao Liu Yang Sebastian Ruder Donald Metzler 160 730 0 08 Nov 2020
ConjNLI: Natural Language Inference Over Conjunctive Sentences Swarnadeep Saha Yixin Nie Joey Tianyi Zhou 77 36 0 20 Oct 2020
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing Zihang Dai Guokun Lai Yiming Yang Quoc V. Le 90 235 0 05 Jun 2020
Learning the Difference that Makes a Difference with Counterfactually-Augmented Data Divyansh Kaushik Eduard H. Hovy Zachary Chase Lipton CML 96 570 0 26 Sep 2019
Theoretical Limitations of Self-Attention in Neural Sequence Models Michael Hahn 70 275 0 16 Jun 2019
Hierarchical Transformers for Multi-Document Summarization Yang Liu Mirella Lapata 116 298 0 30 May 2019
PAWS: Paraphrase Adversaries from Word Scrambling Yuan Zhang Jason Baldridge Luheng He 77 544 0 01 Apr 2019
Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement W. Kool H. V. Hoof Max Welling 121 220 0 14 Mar 2019
Cooperative Learning of Disjoint Syntax and Semantics Serhii Havrylov Germán Kruszewski Armand Joulin 55 48 0 25 Feb 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 260 3,747 0 09 Jan 2019
Sliced Recurrent Neural Networks Zeping Yu Gongshen Liu 45 41 0 06 Jul 2018
Breaking NLI Systems with Sentences that Require Simple Lexical Inferences Max Glockner Vered Shwartz Yoav Goldberg NAI 93 366 0 06 May 2018
ListOps: A Diagnostic Dataset for Latent Tree Learning Nikita Nangia Samuel R. Bowman 68 138 0 17 Apr 2018
Annotation Artifacts in Natural Language Inference Data Suchin Gururangan Swabha Swayamdipta Omer Levy Roy Schwartz Samuel R. Bowman Noah A. Smith 155 1,180 0 06 Mar 2018
Learning to Compose Task-Specific Tree Structures Jihun Choi Kang Min Yoo Sang-goo Lee 84 189 0 10 Jul 2017
Depthwise Separable Convolutions for Neural Machine Translation Lukasz Kaiser Aidan Gomez François Chollet 74 279 0 09 Jun 2017
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 524 4,497 0 18 Apr 2017
Language Modeling with Gated Convolutional Networks Yann N. Dauphin Angela Fan Michael Auli David Grangier 245 2,408 0 23 Dec 2016
Categorical Reparameterization with Gumbel-Softmax Eric Jang S. Gu Ben Poole BDL 363 5,388 0 03 Nov 2016
The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables Chris J. Maddison A. Mnih Yee Whye Teh BDL 200 2,541 0 02 Nov 2016
Neural Tree Indexers for Text Understanding Tsendsuren Munkhdalai Hong-ye Yu 69 104 0 15 Jul 2016
A large annotated corpus for learning natural language inference Samuel R. Bowman Gabor Angeli Christopher Potts Christopher D. Manning 329 4,297 0 21 Aug 2015
Tree-structured composition in neural networks without tree-structured architectures Samuel R. Bowman Christopher D. Manning Christopher Potts 65 75 0 16 Jun 2015
Compositional Distributional Semantics with Long Short Term Memory Phong Le Willem H. Zuidema 102 104 0 09 Mar 2015
Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks Kai Sheng Tai R. Socher Christopher D. Manning AIMat 144 3,122 0 28 Feb 2015
Convolutional Neural Networks for Sentence Classification Yoon Kim AILaw VLM 641 13,432 0 25 Aug 2014
A Convolutional Neural Network for Modelling Sentences Nal Kalchbrenner Edward Grefenstette Phil Blunsom 109 3,559 0 08 Apr 2014