Compositional Capabilities of Autoregressive Transformers: A Study on
Synthetic, Interpretable Tasks

Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks

21 November 2023

Ekdeep Singh Lubana

Hidenori Tanaka

Papers citing "Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks"

19 / 19 papers shown

Title
In-Context Learning with Hypothesis-Class Guidance Ziqian Lin Shubham Kumar Bharti Kangwook Lee 76 0 0 27 Feb 2025
ICLR: In-Context Learning of Representations Core Francisco Park Andrew Lee Ekdeep Singh Lubana Yongyi Yang Maya Okawa Kento Nishi Martin Wattenberg Hidenori Tanaka AIFin 118 3 0 29 Dec 2024
Diffusion Beats Autoregressive: An Evaluation of Compositional Generation in Text-to-Image Models Arash Marioriyad Parham Rezaei M. Baghshah M. Rohban CoGe 142 0 0 30 Oct 2024
Rule Extrapolation in Language Models: A Study of Compositional Generalization on OOD Prompts Anna Mészáros Szilvia Ujváry Wieland Brendel Patrik Reizinger Ferenc Huszár 29 0 0 09 Sep 2024
Position: Understanding LLMs Requires More Than Statistical Generalization Patrik Reizinger Szilvia Ujváry Anna Mészáros A. Kerekes Wieland Brendel Ferenc Huszár 36 12 0 03 May 2024
Holistic Evaluation of Text-To-Image Models Tony Lee Michihiro Yasunaga Chenlin Meng Yifan Mai Joon Sung Park ... Jun-Yan Zhu Fei-Fei Li Jiajun Wu Stefano Ermon Percy Liang 149 126 0 07 Nov 2023
Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task Maya Okawa Ekdeep Singh Lubana Robert P. Dick Hidenori Tanaka CoGe DiffM 37 44 0 13 Oct 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 298 3,007 0 22 Mar 2023
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Kevin Wang Alexandre Variengien Arthur Conmy Buck Shlegeris Jacob Steinhardt 212 496 0 01 Nov 2022
Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought Abulhair Saparov He He ELM LRM ReLM 121 275 0 03 Oct 2022
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 250 460 0 24 Sep 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 322 4,077 0 24 May 2022
Instruction Induction: From Few Examples to Natural Language Task Descriptions Or Honovich Uri Shaham Samuel R. Bowman Omer Levy ELM LRM 120 136 0 22 May 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 367 8,495 0 28 Jan 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,657 0 15 Oct 2021
BBQ: A Hand-Built Bias Benchmark for Question Answering Alicia Parrish Angelica Chen Nikita Nangia Vishakh Padmakumar Jason Phang Jana Thompson Phu Mon Htut Sam Bowman 217 367 0 15 Oct 2021
Can Machines Learn Morality? The Delphi Experiment Liwei Jiang Jena D. Hwang Chandra Bhagavatula Ronan Le Bras Jenny T Liang ... Yulia Tsvetkov Oren Etzioni Maarten Sap Regina A. Rini Yejin Choi FaML 127 111 0 14 Oct 2021
Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models Alex Tamkin Miles Brundage Jack Clark Deep Ganguli AILaw ELM 200 259 0 04 Feb 2021
The Woman Worked as a Babysitter: On Biases in Language Generation Emily Sheng Kai-Wei Chang Premkumar Natarajan Nanyun Peng 214 616 0 03 Sep 2019