How Well Can Transformers Emulate In-context Newton's Method?

How Well Can Transformers Emulate In-context Newton's Method?

5 March 2024

Angeliki Giannou

Dimitris Papailiopoulos

Papers citing "How Well Can Transformers Emulate In-context Newton's Method?"

14 / 14 papers shown

Title
Adversarially Pretrained Transformers may be Universally Robust In-Context Learners Soichiro Kumano Hiroshi Kera Toshihiko Yamasaki AAML 91 0 0 20 May 2025
Vector-ICL: In-context Learning with Continuous Vector Representations Yufan Zhuang Chandan Singh Liyuan Liu Jingbo Shang Jianfeng Gao 114 6 0 21 Feb 2025
On the Training Convergence of Transformers for In-Context Classification of Gaussian Mixtures Wei Shen Ruida Zhou Jing Yang Cong Shen 55 3 0 15 Oct 2024
Transformers Handle Endogeneity in In-Context Linear Regression Haodong Liang Krishnakumar Balasubramanian Lifeng Lai 73 1 0 02 Oct 2024
On Expressive Power of Looped Transformers: Theoretical Analysis and Enhancement via Timestep Encoding Kevin Xu Issei Sato 79 4 0 02 Oct 2024
Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection Yu Bai Fan Chen Haiquan Wang Caiming Xiong Song Mei 44 191 0 07 Jun 2023
Transformers learn in-context by gradient descent J. Oswald Eyvind Niklasson E. Randazzo João Sacramento A. Mordvintsev A. Zhmoginov Max Vladymyrov MLT 91 487 0 15 Dec 2022
Teaching Algorithmic Reasoning via In-context Learning Hattie Zhou Azade Nova Hugo Larochelle Rameswar Panda Behnam Neyshabur Hanie Sedghi LRM ReLM 62 114 0 15 Nov 2022
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes Shivam Garg Dimitris Tsipras Percy Liang Gregory Valiant 116 504 0 01 Aug 2022
Language models show human-like content effects on reasoning tasks Ishita Dasgupta Andrew Kyle Lampinen Stephanie C. Y. Chan Hannah R. Sheahan Antonia Creswell D. Kumaran James L. McClelland Felix Hill ReLM LRM 109 186 0 14 Jul 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 263 2,462 0 15 Jun 2022
GPT-NeoX-20B: An Open-Source Autoregressive Language Model Sid Black Stella Biderman Eric Hallahan Quentin G. Anthony Leo Gao ... Shivanshu Purohit Laria Reynolds J. Tow Benqi Wang Samuel Weinbach 157 825 0 14 Apr 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 742 9,330 0 28 Jan 2022
Show Your Work: Scratchpads for Intermediate Computation with Language Models Maxwell Nye Anders Andreassen Guy Gur-Ari Henryk Michalewski Jacob Austin ... Aitor Lewkowycz Maarten Bosma D. Luan Charles Sutton Augustus Odena ReLM LRM 159 737 0 30 Nov 2021