The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"

21 September 2023

Papers citing "The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A""

50 / 56 papers shown

Title
Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for Real-world Knowledge Injection Yuwei Zhang Wenhao Yu Shangbin Feng Yifan Zhu Letian Peng Jayanth Srinivasa Gaowen Liu Jingbo Shang KELM 12 0 0 18 May 2025
Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis Akarsh Kumar Jeff Clune Joel Lehman Kenneth O. Stanley OOD 21 0 0 16 May 2025
Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs Jingcheng Niu Xingdi Yuan Tong Wang Hamidreza Saghir Amir H. Abdi 27 0 0 14 May 2025
Memorization-Compression Cycles Improve Generalization Fangyuan Yu 39 0 0 13 May 2025
Consistency in Language Models: Current Landscape, Challenges, and Future Directions Jekaterina Novikova Carol Anderson Borhane Blili-Hamelin Subhabrata Majumdar HILM 73 0 0 01 May 2025
On the generalization of language models from in-context learning and finetuning: a controlled study Andrew Kyle Lampinen Arslan Chaudhry Stephanie Chan Cody Wild Diane Wan Alex Ku Jorg Bornschein Razvan Pascanu Murray Shanahan James L. McClelland 57 1 0 01 May 2025
Memorization and Knowledge Injection in Gated LLMs Xu Pan Ely Hahami Zechen Zhang H. Sompolinsky KELM CLL RALM 108 1 0 30 Apr 2025
Benchmarking Systematic Relational Reasoning with Large Language and Reasoning Models Irtaza Khalid Amir Masoud Nourollah Steven Schockaert LRM 52 0 0 30 Mar 2025
Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More Arvid Frydenlund LRM 63 0 0 13 Mar 2025
Training Plug-n-Play Knowledge Modules with Deep Context Distillation Lucas Caccia Alan Ansell Edoardo Ponti Ivan Vulić Alessandro Sordoni SyDa 265 0 0 11 Mar 2025
Reversal Blessing: Thinking Backward May Outpace Thinking Forward in Multi-choice Questions Yizhe Zhang Richard He Bai Zijin Gu Ruixiang Zhang Jiatao Gu Emmanuel Abbe Samy Bengio Navdeep Jaitly LRM BDL 72 1 0 25 Feb 2025
Order Matters: Investigate the Position Bias in Multi-constraint Instruction Following Jie Zeng Qianyu He Qingyu Ren Jiaqing Liang Yanghua Xiao Weikang Zhou Zeye Sun Fei Yu 86 1 0 24 Feb 2025
Large Language Diffusion Models Shen Nie Fengqi Zhu Zebin You Xiaolu Zhang Jingyang Ou Jun Hu Jun Zhou Yankai Lin Zhicheng Dou Chongxuan Li 114 19 0 14 Feb 2025
The Stochastic Parrot on LLM's Shoulder: A Summative Assessment of Physical Concept Understanding Mo Yu Lemao Liu J. Wu Tsz Ting Chung Shunchi Zhang JiangNan Li Dit-Yan Yeung Jie Zhou 87 1 0 13 Feb 2025
Evolution and The Knightian Blindspot of Machine Learning Joel Lehman Elliot Meyerson Tarek El-Gaaly Kenneth O. Stanley Tarin Ziyaee 96 2 0 22 Jan 2025
Time-Reversal Provides Unsupervised Feedback to LLMs Yerram Varun Rahul Madhavan Sravanti Addepalli A. Suggala Karthikeyan Shanmugam Prateek Jain LRM SyDa 71 0 0 03 Dec 2024
Sneaking Syntax into Transformer Language Models with Tree Regularization Ananjan Nandi Christopher D. Manning Shikhar Murty 75 0 0 28 Nov 2024
Latent Paraphrasing: Perturbation on Layers Improves Knowledge Injection in Language Models Minki Kang Sung Ju Hwang Gibbeum Lee Jaewoong Cho KELM 43 0 0 01 Nov 2024
Constraint Back-translation Improves Complex Instruction Following of Large Language Models Y. Qi Hao Peng Xinyu Wang Bin Xu Lei Hou Juanzi Li 64 1 0 31 Oct 2024
Take Caution in Using LLMs as Human Surrogates: Scylla Ex Machina Yuan Gao Dokyun Lee Gordon Burtch Sina Fazelpour LRM 59 7 0 25 Oct 2024
Scaling up Masked Diffusion Models on Text Shen Nie Fengqi Zhu Chao Du Tianyu Pang Qian Liu Guangtao Zeng Min Lin Chongxuan Li AI4CE 63 14 0 24 Oct 2024
LLM The Genius Paradox: A Linguistic and Math Expert's Struggle with Simple Word-based Counting Problems Nan Xu Xuezhe Ma LRM 59 4 0 18 Oct 2024
The Mystery of the Pathological Path-star Task for Language Models Arvid Frydenlund LRM 27 4 0 17 Oct 2024
Reverse Modeling in Large Language Models S. Yu Yuanchen Xu Cunxiao Du Yanying Zhou Minghui Qiu Q. Sun Hao Zhang Jiawei Wu 41 2 0 13 Oct 2024
Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely Siyun Zhao Yuqing Yang Zilong Wang Zhiyuan He Luna Qiu Lili Qiu SyDa RALM 3DV 46 36 0 23 Sep 2024
Co-occurrence is not Factual Association in Language Models Xiao Zhang Miao Li Ji Wu KELM 75 2 0 21 Sep 2024
Revisiting the Graph Reasoning Ability of Large Language Models: Case Studies in Translation, Connectivity and Shortest Path Xinnan Dai Qihao Wen Yifei Shen Hongzhi Wen Dongsheng Li Jiliang Tang Caihua Shan LRM 57 3 0 18 Aug 2024
Does Refusal Training in LLMs Generalize to the Past Tense? Maksym Andriushchenko Nicolas Flammarion 50 28 0 16 Jul 2024
Teaching Transformers Causal Reasoning through Axiomatic Training Aniket Vashishtha Abhinav Kumar Abbavaram Gowtham Reddy Vineeth N. Balasubramanian Amit Sharma Vineeth N Balasubramanian Amit Sharma 42 2 0 10 Jul 2024
ConCodeEval: Evaluating Large Language Models for Code Constraints in Domain-Specific Languages Mehant Kammakomati Sameer Pimparkhede Srikanth G. Tamilselvam Prince Kumar Pushpak Bhattacharyya ALM 40 0 0 03 Jul 2024
From Distributional to Overton Pluralism: Investigating Large Language Model Alignment Thom Lake Eunsol Choi Greg Durrett 46 9 0 25 Jun 2024
Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models Lynn Chua Badih Ghazi Yangsibo Huang Pritish Kamath Ravi Kumar Pasin Manurangsi Amer Sinha Chulin Xie Chiyuan Zhang 69 1 0 23 Jun 2024
Cognitively Inspired Energy-Based World Models Alexi Gladstone Ganesh Nanduru Md. Mofijul Islam Aman Chadha Jundong Li Tariq Iqbal 49 0 0 13 Jun 2024
The CLRS-Text Algorithmic Reasoning Language Benchmark Larisa Markeeva Sean McLeish Borja Ibarz Wilfried Bounsi Olga Kozlova Alex Vitvitskyi Charles Blundell Tom Goldstein Avi Schwarzschild Petar Veličković LRM 40 12 0 06 Jun 2024
Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data Jingyang Ou Shen Nie Kaiwen Xue Fengqi Zhu Jiacheng Sun Zhenguo Li Chongxuan Li DiffM 41 29 0 06 Jun 2024
Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Models Marianna Nezhurina Lucia Cipolina-Kun Mehdi Cherti J. Jitsev LLMAG LRM ELM ReLM 66 28 0 04 Jun 2024
Evaluating the External and Parametric Knowledge Fusion of Large Language Models Hao Zhang Yuyang Zhang Xiaoguang Li Wenxuan Shi Haonan Xu ... Yasheng Wang Lifeng Shang Qun Liu Yong-jin Liu Ruiming Tang KELM 45 4 0 29 May 2024
Knowledge Circuits in Pretrained Transformers Yunzhi Yao Ningyu Zhang Zekun Xi Meng Wang Ziwen Xu Shumin Deng Huajun Chen KELM 74 20 0 28 May 2024
Can LLMs Solve longer Math Word Problems Better? Xin Xu Tong Xiao Zitong Chao Zhenya Huang Can Yang Yang Wang 70 12 0 23 May 2024
RAR-b: Reasoning as Retrieval Benchmark Chenghao Xiao G. Thomas Al Moubayed LRM RALM 43 8 0 09 Apr 2024
FreeEval: A Modular Framework for Trustworthy and Efficient Evaluation of Large Language Models Zhuohao Yu Chang Gao Wenjin Yao Yidong Wang Zhengran Zeng Wei Ye Jindong Wang Yue Zhang Shikun Zhang 46 1 0 09 Apr 2024
Are large language models superhuman chemists? Adrian Mirza Nawaf Alampara Sreekanth Kunchapu Benedict Emoekabu Aswanth Krishnan ... Leanne M. Stafast Dinga Wonanke Michael Pieler P. Schwaller Kevin Maik Jablonka ELM AI4MH LRM LM&MA 36 5 0 01 Apr 2024
Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs Xiaoze Liu Feijie Wu Tianyang Xu Zhuo Chen Yichi Zhang Xiaoqian Wang Jing Gao HILM 54 8 0 01 Apr 2024
Punctuation Restoration Improves Structure Understanding Without Supervision Junghyun Min Minho Lee Woochul Lee Yeonsoo Lee 62 1 0 13 Feb 2024
Improving Sequential Recommendations with LLMs Artun Boz Wouter Zorgdrager Zoe Kotti Jesse Harte Panos Louridas Dietmar Jannach Vassilios Karakoidas Marios Fragkoulis KELM LRM 72 4 0 02 Feb 2024
Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion Ruilin Luo Tianle Gu Haoling Li Junzhe Li Zicheng Lin Jiayi Li Yujiu Yang AI4CE 36 8 0 11 Jan 2024
MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluation Zhongshen Zeng Pengguang Chen Shu Liu Haiyun Jiang Jiaya Jia ReLM ELM LRM 41 18 0 28 Dec 2023
Hazards from Increasingly Accessible Fine-Tuning of Downloadable Foundation Models Alan Chan Ben Bucknall Herbie Bradley David M. Krueger 16 6 0 22 Dec 2023
Concise and Organized Perception Facilitates Reasoning in Large Language Models Junjie Liu Shaotian Yan Chen Shen Zhengdong Xiao Wenxiao Wang Jieping Ye Jieping Ye LRM 26 1 0 05 Oct 2023
Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning Jiasheng Ye Zaixiang Zheng Yu Bao Lihua Qian Quanquan Gu DiffM 57 14 0 23 Aug 2023