Does syntax need to grow on trees? Sources of hierarchical inductive bias in sequence-to-sequence networks

10 January 2020

Papers citing "Does syntax need to grow on trees? Sources of hierarchical inductive bias in sequence-to-sequence networks"

23 / 23 papers shown

Title
Scaling Laws and Representation Learning in Simple Hierarchical Languages: Transformers vs. Convolutional Architectures Francesco Cagnetta Alessandro Favero Antonio Sclocchi M. Wyart 26 0 0 11 May 2025
Learning curves theory for hierarchically compositional data with power-law distributed features Francesco Cagnetta Hyunmo Kang M. Wyart 36 0 0 11 May 2025
Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora Alex Warstadt Aaron Mueller Leshem Choshen E. Wilcox Chengxu Zhuang ... Rafael Mosquera Bhargavi Paranjape Adina Williams Tal Linzen Ryan Cotterell 38 108 0 10 Apr 2025
Sneaking Syntax into Transformer Language Models with Tree Regularization Ananjan Nandi Christopher D. Manning Shikhar Murty 74 0 0 28 Nov 2024
Beyond Scaling Laws: Understanding Transformer Performance with Associative Memory Xueyan Niu Bo Bai Lei Deng Wei Han 36 6 0 14 May 2024
Learned feature representations are biased by complexity, learning order, position, and more Andrew Kyle Lampinen Stephanie C. Y. Chan Katherine Hermann AI4CE FaML SSL OOD 34 6 0 09 May 2024
On the Challenges and Opportunities in Generative AI Laura Manduchi Kushagra Pandey Robert Bamler Ryan Cotterell Sina Daubener ... F. Wenzel Frank Wood Stephan Mandt Vincent Fortuin Vincent Fortuin 56 17 0 28 Feb 2024
Punctuation Restoration Improves Structure Understanding Without Supervision Junghyun Min Minho Lee Woochul Lee Yeonsoo Lee 59 1 0 13 Feb 2024
Language acquisition: do children and language models follow similar learning stages? Linnea Evanson Yair Lakretz J. King 27 26 0 06 Jun 2023
Dissociating language and thought in large language models Kyle Mahowald Anna A. Ivanova I. Blank Nancy Kanwisher J. Tenenbaum Evelina Fedorenko ELM ReLM 29 209 0 16 Jan 2023
Structural generalization is hard for sequence-to-sequence models Yuekun Yao Alexander Koller 27 21 0 24 Oct 2022
State-of-the-art generalisation research in NLP: A taxonomy and review Dieuwke Hupkes Mario Giulianelli Verna Dankers Mikel Artetxe Yanai Elazar ... Leila Khalatbari Maria Ryskina Rita Frieske Ryan Cotterell Zhijing Jin 114 93 0 06 Oct 2022
Exploring Length Generalization in Large Language Models Cem Anil Yuhuai Wu Anders Andreassen Aitor Lewkowycz Vedant Misra V. Ramasesh Ambrose Slone Guy Gur-Ari Ethan Dyer Behnam Neyshabur ReLM LRM 33 158 0 11 Jul 2022
A Property Induction Framework for Neural Language Models Kanishka Misra Julia Taylor Rayz Allyson Ettinger 21 12 0 13 May 2022
Coloring the Blank Slate: Pre-training Imparts a Hierarchical Inductive Bias to Sequence-to-sequence Models Aaron Mueller Robert Frank Tal Linzen Luheng Wang Sebastian Schuster AIMat 19 33 0 17 Mar 2022
How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN R. Thomas McCoy P. Smolensky Tal Linzen Jianfeng Gao Asli Celikyilmaz SyDa 25 119 0 18 Nov 2021
How Do Neural Sequence Models Generalize? Local and Global Context Cues for Out-of-Distribution Prediction Anthony Bau Jacob Andreas 19 3 0 04 Nov 2021
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey Bonan Min Hayley L Ross Elior Sulem Amir Pouran Ben Veyseh Thien Huu Nguyen Oscar Sainz Eneko Agirre Ilana Heinz Dan Roth LM&MA VLM AI4CE 74 1,030 0 01 Nov 2021
Monolingual and Cross-Lingual Acceptability Judgments with the Italian CoLA corpus Daniela Trotta R. Guarasci Elisa Leonardelli Sara Tonelli 42 30 0 24 Sep 2021
Sequence-to-Sequence Learning with Latent Neural Grammars Yoon Kim 33 40 0 02 Sep 2021
Can Transformers Jump Around Right in Natural Language? Assessing Performance Transfer from SCAN Rahma Chaabouni Roberto Dessì Eugene Kharitonov 21 20 0 03 Jul 2021
Can neural networks acquire a structural bias from raw linguistic data? Alex Warstadt Samuel R. Bowman AI4CE 20 53 0 14 Jul 2020
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 218 7,926 0 17 Aug 2015