Generalization on the Unseen, Logic Reasoning and Degree Curriculum

30 January 2023

Papers citing "Generalization on the Unseen, Logic Reasoning and Degree Curriculum"

42 / 42 papers shown

Title
Geometric Reasoning in the Embedding Space Jan Hůla David Mojžíšek Jiří Janeček David Herel Mikoláš Janota 41 0 0 02 Apr 2025
Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More Arvid Frydenlund LRM 48 0 0 13 Mar 2025
The Role of Sparsity for Length Generalization in Transformers Noah Golowich Samy Jelassi David Brandfonbrener Sham Kakade Eran Malach 37 0 0 24 Feb 2025
Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers Alireza Amiri Xinting Huang Mark Rofin Michael Hahn LRM 174 0 0 04 Feb 2025
Out-of-distribution generalization via composition: a lens through induction heads in Transformers Jiajun Song Zhuoyan Xu Yiqiao Zhong 85 4 0 31 Dec 2024
Selective Attention: Enhancing Transformer through Principled Context Control Xuechen Zhang Xiangyu Chang Mingchen Li A. Roy-Chowdhury J. Chen Samet Oymak 78 3 0 19 Nov 2024
Number Cookbook: Number Understanding of Language Models and How to Improve It Haotong Yang Yi Hu Shijia Kang Zhouchen Lin Muhan Zhang LRM 46 2 0 06 Nov 2024
Provable Length Generalization in Sequence Prediction via Spectral Filtering Annie Marsden Evan Dogariu Naman Agarwal Xinyi Chen Daniel Suo Elad Hazan 34 1 0 01 Nov 2024
Transformation-Invariant Learning and Theoretical Guarantees for OOD Generalization Omar Montasser Han Shao Emmanuel Abbe OOD 44 1 0 30 Oct 2024
Interchangeable Token Embeddings for Extendable Vocabulary and Alpha-Equivalence İlker Işık R. G. Cinbis Ebru Aydin Gol 28 0 0 22 Oct 2024
Low-Dimension-to-High-Dimension Generalization And Its Implications for Length Generalization Yang Chen Yitao Liang Zhouchen Lin 32 1 0 11 Oct 2024
Visual Scratchpads: Enabling Global Reasoning in Vision Aryo Lotfi Enrico Fini Samy Bengio Moin Nabi Emmanuel Abbe LRM 37 0 0 10 Oct 2024
Relating the Seemingly Unrelated: Principled Understanding of Generalization for Generative Models in Arithmetic Reasoning Tasks Xingcheng Xu Zibo Zhao Haipeng Zhang Yanqing Yang LRM 36 0 0 25 Jul 2024
Universal Length Generalization with Turing Programs Kaiying Hou David Brandfonbrener Sham Kakade Samy Jelassi Eran Malach 44 7 0 03 Jul 2024
VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbation Kun Qian Shunji Wan Claudia Tang Youzhi Wang Xuanming Zhang Maximillian Chen Zhou Yu AAML 42 8 0 25 Jun 2024
Transformers meet Neural Algorithmic Reasoners Wilfried Bounsi Borja Ibarz Andrew Dudzik Jessica B. Hamrick Larisa Markeeva Alex Vitvitskyi Razvan Pascanu Petar Veličković NAI AI4CE LRM 35 5 0 13 Jun 2024
On the Minimal Degree Bias in Generalization on the Unseen for non-Boolean Functions Denys Pushkin Raphael Berthier Emmanuel Abbe 32 0 0 10 Jun 2024
Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers Brian K Chen Tianyang Hu Hui Jin Hwee Kuan Lee Kenji Kawaguchi 45 0 0 05 Jun 2024
Feature contamination: Neural networks learn uncorrelated features and fail to generalize Tianren Zhang Chujie Zhao Guanyu Chen Yizhou Jiang Feng Chen OOD MLT OODD 77 3 0 05 Jun 2024
Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks Mahdi Sabbaghi George Pappas Hamed Hassani Surbhi Goel 36 4 0 04 Jun 2024
Language Models Need Inductive Biases to Count Inductively Yingshan Chang Yonatan Bisk LRM 32 5 0 30 May 2024
When does compositional structure yield compositional generalization? A kernel theory Samuel Lippl Kim Stachenfeld NAI CoGe 73 5 0 26 May 2024
Theoretical Analysis of Weak-to-Strong Generalization Hunter Lang David Sontag Aravindan Vijayaraghavan 25 19 0 25 May 2024
Learning Invariant Causal Mechanism from Vision-Language Models Zeen Song Siyu Zhao Xingyu Zhang Jiangmeng Li Changwen Zheng Wenwen Qiang CML BDL VLM 39 0 0 24 May 2024
A Theory for Length Generalization in Learning to Reason Changnan Xiao Bing Liu LRM 39 8 0 31 Mar 2024
Transfer Learning Beyond Bounded Density Ratios Alkis Kalavasis Ilias Zadik Manolis Zampetakis 38 4 0 18 Mar 2024
Neural Redshift: Random Networks are not Random Functions Damien Teney A. Nicolicioiu Valentin Hartmann Ehsan Abbasnejad 94 18 0 04 Mar 2024
Why are Sensitive Functions Hard for Transformers? Michael Hahn Mark Rofin 29 23 0 15 Feb 2024
Transformers Can Achieve Length Generalization But Not Robustly Yongchao Zhou Uri Alon Xinyun Chen Xuezhi Wang Rishabh Agarwal Denny Zhou 46 36 0 14 Feb 2024
On Provable Length and Compositional Generalization Kartik Ahuja Amin Mansouri OODD 38 7 0 07 Feb 2024
Overcoming the Pitfalls of Vision-Language Model Finetuning for OOD Generalization Yuhang Zang Hanlin Goh Josh Susskind Chen Huang VLM 34 12 0 29 Jan 2024
Unsupervised Multi-modal Feature Alignment for Time Series Representation Learning Cheng Liang Donghua Yang Zhiyu Liang Hongzhi Wang Zheng Liang Xiyang Zhang Jianfeng Huang AI4TS 149 1 0 09 Dec 2023
When can transformers reason with abstract symbols? Enric Boix-Adserà Omid Saremi Emmanuel Abbe Samy Bengio Etai Littwin Josh Susskind LRM NAI 31 12 0 15 Oct 2023
Adaptivity and Modularity for Efficient Generalization Over Task Complexity Samira Abnar Omid Saremi Laurent Dinh Shantel Wilson Miguel Angel Bautista ... Vimal Thilak Etai Littwin Jiatao Gu Josh Susskind Samy Bengio 34 5 0 13 Oct 2023
Improving Length-Generalization in Transformers via Task Hinting Pranjal Awasthi Anupam Gupta 21 7 0 01 Oct 2023
Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputs Emmanuel Abbe Elisabetta Cornacchia Aryo Lotfi 28 11 0 29 Jun 2023
A Mathematical Model for Curriculum Learning for Parities Elisabetta Cornacchia Elchanan Mossel 34 10 0 31 Jan 2023
Towards Better Out-of-Distribution Generalization of Neural Algorithmic Reasoning Tasks Sadegh Mahdavi Kevin Swersky Thomas Kipf Milad Hashemi Christos Thrampoulidis Renjie Liao LRM OOD NAI 45 25 0 01 Nov 2022
Revisiting Neural Scaling Laws in Language and Vision Ibrahim M. Alabdulmohsin Behnam Neyshabur Xiaohua Zhai 156 102 0 13 Sep 2022
The CLRS Algorithmic Reasoning Benchmark Petar Velivcković Adria Puigdomenech Badia David Budden Razvan Pascanu Andrea Banino Mikhail Dashevskiy R. Hadsell Charles Blundell 161 87 0 31 May 2022
A Fine-Grained Analysis on Distribution Shift Olivia Wiles Sven Gowal Florian Stimberg Sylvestre-Alvise Rebuffi Ira Ktena Krishnamurthy Dvijotham A. Cemgil OOD 225 201 0 21 Oct 2021
Domain Adaptation: Learning Bounds and Algorithms Yishay Mansour M. Mohri Afshin Rostamizadeh 179 789 0 19 Feb 2009