Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora

10 April 2025

Papers citing "Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora"

50 / 73 papers shown

Title
Parameterized Synthetic Text Generation with SimpleStories Lennart Finke Chandan Sreedhara Thomas Dooms Mat Allen Emerald Zhang Juan Diego Rodriguez Noa Nabeshima Thomas Marshall Dan Braun SyDa 32 0 0 12 Apr 2025
Pretraining Language Models for Diachronic Linguistic Change Discovery Elisabeth Fittschen Sabrina Li Tom Lippincott Leshem Choshen Craig Messner 26 0 0 07 Apr 2025
Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Models Qing Yao Kanishka Misra Leonie Weissweiler Kyle Mahowald 47 0 0 26 Mar 2025
Do Construction Distributions Shape Formal Language Learning In German BabyLMs? Bastian Bunzeck Daniel Duran Sina Zarrieß 48 0 0 14 Mar 2025
BAMBI: Developing Baby Language Models for Italian Alice Suozzi Luca Capone Gianluca E. Lebani Alessandro Lenci 60 0 0 12 Mar 2025
Language Models Fail to Introspect About Their Knowledge of Language Siyuan Song Jennifer Hu Kyle Mahowald LRM KELM HILM ELM 84 2 0 10 Mar 2025
Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases Michael Y. Hu Jackson Petty Chuan Shi William Merrill Tal Linzen AI4CE 66 1 0 26 Feb 2025
Can Language Models Learn Typologically Implausible Languages? Tianyang Xu Tatsuki Kuribayashi Yohei Oseki Ryan Cotterell Alex Warstadt 75 1 0 17 Feb 2025
Language Models Largely Exhibit Human-like Constituent Ordering Preferences Ada Defne Tur Gaurav Kamath Siva Reddy 61 0 0 08 Feb 2025
BabyLMs for isiXhosa: Data-Efficient Language Modelling in a Low-Resource Context Alexis Matzopoulos Charl Hendriks Hishaam Mahomed Francois Meyer 30 0 0 08 Jan 2025
GPT or BERT: why not both? Lucas Georges Gabriel Charpentier David Samuel 55 5 0 31 Dec 2024
Learning from Impairment: Leveraging Insights from Clinical Linguistics in Language Modelling Research Dominique Brunato 74 0 0 20 Dec 2024
Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora Michael Y. Hu Aaron Mueller Candace Ross Adina Williams Tal Linzen Chengxu Zhuang Ryan Cotterell Leshem Choshen Alex Warstadt Ethan Gotlieb Wilcox 99 7 0 06 Dec 2024
When Babies Teach Babies: Can student knowledge sharing outperform Teacher-Guided Distillation on small datasets? Srikrishna Iyer FedML 82 0 0 25 Nov 2024
BudgetMLAgent: A Cost-Effective LLM Multi-Agent system for Automating Machine Learning Tasks Shubham Gandhi Manasi S. Patwardhan L. Vig Gautam M. Shroff LLMAG 47 0 0 12 Nov 2024
ZhoBLiMP: a Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese Yikang Liu Yeting Shen Hongao Zhu Lilong Xu Zhiheng Qian ... Jialong Tang Pei Zhang Baosong Yang Rui-cang Wang Hai Hu 45 2 0 09 Nov 2024
From Babble to Words: Pre-Training Language Models on Continuous Streams of Phonemes Zébulon Goriely Richard Diehl Martinez Andrew Caines Lisa Beinborn P. Buttery CLL 50 5 0 30 Oct 2024
Choosy Babies Need One Coach: Inducing Mode-Seeking Behavior in BabyLlama with Reverse KL Divergence Shaozhen Shi Yevgen Matusevych Malvina Nissim 39 0 0 29 Oct 2024
Dreaming Out Loud: A Self-Synthesis Approach For Training Vision-Language Models With Developmentally Plausible Data Badr AlKhamissi Yingtian Tang Abdülkadir Gökce Johannes Mehrer Martin Schrimpf VLM 49 0 0 29 Oct 2024
Are BabyLMs Second Language Learners? Lukas Edman Lisa Bylinina Faeze Ghorbanpour Alexander Fraser 22 0 0 28 Oct 2024
Team Ryu's Submission to SIGMORPHON 2024 Shared Task on Subword Tokenization Zilong Li 30 0 0 19 Oct 2024
A Hitchhiker's Guide to Scaling Law Estimation Leshem Choshen Yang Zhang Jacob Andreas 43 6 0 15 Oct 2024
Can Language Models Induce Grammatical Knowledge from Indirect Evidence? Miyu Oba Yohei Oseki Akiyo Fukatsu Akari Haga Hiroki Ouchi Taro Watanabe Saku Sugawara 39 1 0 08 Oct 2024
BabyLlama-2: Ensemble-Distilled Models Consistently Outperform Teachers With Limited Data J. Tastet I. Timiryasov 40 4 0 25 Sep 2024
What is the Role of Small Models in the LLM Era: A Survey Lihu Chen Gaël Varoquaux ALM 63 23 0 10 Sep 2024
Generating novel experimental hypotheses from language models: A case study on cross-dative generalization Kanishka Misra Najoung Kim 29 3 0 09 Aug 2024
Is Child-Directed Speech Effective Training Data for Language Models? Steven Y. Feng Noah D. Goodman Michael C. Frank 40 9 0 07 Aug 2024
Social Learning through Interactions with Other Agents: A Survey Dylan Hillier Cheston Tan Jing Jiang 40 0 0 31 Jul 2024
HVM-1: Large-scale video models pretrained with nearly 5000 hours of human-like video data Emin Orhan VLM SyDa 38 1 0 25 Jul 2024
Testing learning hypotheses using neural networks by manipulating learning data Cara Su-Yi Leong Tal Linzen 31 4 0 05 Jul 2024
Efficient Training of Language Models with Compact and Consistent Next Token Distributions Ashutosh Sathe Sunita Sarawagi 40 0 0 03 Jul 2024
Black Big Boxes: Do Language Models Hide a Theory of Adjective Order? Jaap Jumelet Lisa Bylinina Willem H. Zuidema Jakub Szymanik 75 4 0 02 Jul 2024
BAMBINO-LM: (Bilingual-)Human-Inspired Continual Pretraining of BabyLM Zhewen Shen Aditya Joshi Ruey-Cheng Chen CLL 52 2 0 17 Jun 2024
The BabyView dataset: High-resolution egocentric videos of infants' and young children's everyday experiences Bria Long Violet Xiang Stefan Stojanov Robert Z. Sparks Zi Yin ... Steven Y. Feng Chengxu Zhuang V. Marchman Daniel L. K. Yamins Michael C. Frank VGen EgoV 30 2 0 14 Jun 2024
DevBench: A multimodal developmental benchmark for language learning A. W. M. Tan Sunny Yu Bria Long Wanjing Anya Ma Tonya Murray Rebecca D. Silverman Jason D. Yeatman Michael C. Frank 39 3 0 14 Jun 2024
From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks Jacob Russin Sam Whitman McGrath Danielle J. Williams Lotem Elber-Dorozko AI4CE 75 3 0 24 May 2024
Super Tiny Language Models Dylan Hillier Leon Guertler Cheston Tan Palaash Agrawal Ruirui Chen Bobby Cheng 58 4 0 23 May 2024
Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations Ziqiao Ma Zekun Wang Joyce Chai 60 2 0 22 May 2024
Age-Dependent Analysis and Stochastic Generation of Child-Directed Speech Okko Rasanen Daniil Kocharov 25 0 0 13 May 2024
Natural Language Processing RELIES on Linguistics Juri Opitz Shira Wein Nathan Schneider AI4CE 55 7 0 09 May 2024
[Call for Papers] The 2nd BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus Leshem Choshen Ryan Cotterell Michael Y. Hu Tal Linzen Aaron Mueller Candace Ross Alex Warstadt Ethan Gotlieb Wilcox Adina Williams Chengxu Zhuang 42 22 0 09 Apr 2024
NumeroLogic: Number Encoding for Enhanced LLMs' Numerical Reasoning Eli Schwartz Leshem Choshen J. Shtok Sivan Doveh Leonid Karlinsky Assaf Arbelle 28 13 0 30 Mar 2024
Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs Kanishka Misra Kyle Mahowald 43 23 0 28 Mar 2024
Lexicon-Level Contrastive Visual-Grounding Improves Language Modeling Chengxu Zhuang Evelina Fedorenko Jacob Andreas 40 2 0 21 Mar 2024
Do Not Worry if You Do Not Have Data: Building Pretrained Language Models Using Translationese Meet Doshi Raj Dabre Pushpak Bhattacharyya SyDa 36 2 0 20 Mar 2024
Acquiring Linguistic Knowledge from Multimodal Input Theodor Amariucai Alexander Scott Warstadt CLL 34 2 0 27 Feb 2024
Tree-Planted Transformers: Unidirectional Transformer Language Models with Implicit Syntactic Supervision Ryosuke Yoshida Taiga Someya Yohei Oseki 42 0 0 20 Feb 2024
A systematic investigation of learnability from single child linguistic input Yulu Qin Wentao Wang Brenden M. Lake 29 4 0 12 Feb 2024
A Philosophical Introduction to Language Models -- Part I: Continuity With Classic Debates Raphael Milliere Cameron Buckner LRM ELM 41 20 0 08 Jan 2024
WhisBERT: Multimodal Text-Audio Language Modeling on 100M Words Lukas Wolf Greta Tuckute Klemen Kotar Eghbal Hosseini Tamar I. Regev Ethan Gotlieb Wilcox Alex Warstadt 45 3 0 05 Dec 2023