Towards a Theoretical Understanding of the 'Reversal Curse' via Training
Dynamics

Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics

7 May 2024

Michael I. Jordan

Papers citing "Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics"

12 / 12 papers shown

Title
Open Problems and a Hypothetical Path Forward in LLM Knowledge Paradigms Xiaotian Ye M. Zhang Shu Wu KELM ELM 41 0 0 09 Apr 2025
Is the Reversal Curse a Binding Problem? Uncovering Limitations of Transformers from a Basic Generalization Failure Boshi Wang Huan Sun 34 2 0 02 Apr 2025
Effective LLM Knowledge Learning via Model Generalization Mingkang Zhu Xi Chen Zhilin Wang Bei Yu Hengshuang Zhao Jiaya Jia 62 0 0 05 Mar 2025
$The Two-Hop Curse: LLMs trained on A$\rightarrow$B, B$\rightarrow$C fail to learn A$\rightarrow$C$ The Two-Hop Curse: LLMs trained on A $\rightarrow$ B, B $\rightarrow$ C fail to learn A $\rightarrow$ C Mikita Balesni Tomek Korbak Owain Evans ReLM LRM 79 0 0 25 Nov 2024
Delving into the Reversal Curse: How Far Can Large Language Models Generalize? Zhengkai Lin Z. Fu Kai Liu Liang Xie Binbin Lin Wenxiao Wang D. Cai Yue Wu Jieping Ye LRM 25 3 0 24 Oct 2024
Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs Tianyu Guo Druv Pai Yu Bai Jiantao Jiao Michael I. Jordan Song Mei 29 10 0 17 Oct 2024
Reverse Modeling in Large Language Models S. Yu Yuanchen Xu Cunxiao Du Yanying Zhou Minghui Qiu Q. Sun Hao Zhang Jiawei Wu 36 2 0 13 Oct 2024
Do Transformers Parse while Predicting the Masked Word? Haoyu Zhao A. Panigrahi Rong Ge Sanjeev Arora 76 31 0 14 Mar 2023
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 328 4,106 0 24 May 2022
Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations Jaehun Jung Lianhui Qin Sean Welleck Faeze Brahman Chandra Bhagavatula Ronan Le Bras Yejin Choi ReLM LRM 226 190 0 24 May 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 314 3,248 0 21 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 379 8,495 0 28 Jan 2022