System 2 Attention (is something you might need too)

20 November 2023

Jason Weston

Papers citing "System 2 Attention (is something you might need too)"

42 / 42 papers shown

Title
On the Thinking-Language Modeling Gap in Large Language Models Chenxi Liu Yongqiang Chen Tongliang Liu James Cheng Bo Han Kun Zhang LRM AI4CE 9 0 0 19 May 2025
Exploring and Controlling Diversity in LLM-Agent Conversation Kuanchao Chu Yi-Pei Chen Hideki Nakayama LLMAG 50 1 0 24 Feb 2025
An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problems Hashmath Shaik Alex Doboli OffRL ELM 226 0 0 31 Dec 2024
System-2 Mathematical Reasoning via Enriched Instruction Tuning Huanqia Cai Yijun Yang Zhifeng Li LRM 81 0 0 22 Dec 2024
Can We Afford The Perfect Prompt? Balancing Cost and Accuracy with the Economical Prompting Index Tyler McDonald Anthony Colosimo Yifeng Li Ali Emami 80 1 0 02 Dec 2024
Sycophancy in Large Language Models: Causes and Mitigations Lars Malmqvist 83 9 0 22 Nov 2024
Explaining GPT-4's Schema of Depression Using Machine Behavior Analysis Adithya V Ganesan Vasudha Varadarajan Yash Kumar Lal Veerle C. Eijsbroek Katarina Kjell ... Elizabeth C. Stade J. Eichstaedt Ryan L. Boyd H. Andrew Schwartz Lucie Flek AI4MH 77 0 0 21 Nov 2024
Thinking LLMs: General Instruction Following with Thought Generation Tianhao Wu Janice Lan Weizhe Yuan Jiantao Jiao Jason Weston Sainbayar Sukhbaatar LRM 30 16 0 14 Oct 2024
LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints Thomas Palmeira Ferraz Kartik Mehta Yu-Hsiang Lin Haw-Shiuan Chang Shereen Oraby Sijia Liu Vivek Subramanian Tagyoung Chung Mohit Bansal Nanyun Peng 56 8 0 09 Oct 2024
ALR $^2$ : A Retrieve-then-Reason Framework for Long-context Question Answering Huayang Li Pat Verga Priyanka Sen Bowen Yang Vijay Viswanathan Patrick Lewis Taro Watanabe Yixuan Su RALM LRM 51 8 0 04 Oct 2024
Verbalized Graph Representation Learning: A Fully Interpretable Graph Model Based on Large Language Models Throughout the Entire Process Xingyu Ji Jiale Liu Lu Li Maojun Wang Zeyu Zhang 36 1 0 02 Oct 2024
Distilling System 2 into System 1 Ping Yu Jing Xu Jason Weston Ilia Kulikov OffRL LRM 54 62 0 08 Jul 2024
Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models Chani Jung Dongkwan Kim Jiho Jin Jiseon Kim Yeon Seonwoo Yejin Choi Alice Oh Hyunwoo Kim LRM 58 7 0 08 Jul 2024
Re-Tuning: Overcoming the Compositionality Limits of Large Language Models with Recursive Tuning Eric Pasewark Kyle Montgomery Kefei Duan Dawn Song Chenguang Wang LRM CLL ReLM 44 1 0 05 Jul 2024
CItruS: Chunked Instruction-aware State Eviction for Long Sequence Modeling Yu Bai Xiyuan Zou Heyan Huang Sanxing Chen Marc-Antoine Rondeau Yang Gao Jackie Chi Kit Cheung 39 4 0 17 Jun 2024
MultiMax: Sparse and Multi-Modal Attention Learning Yuxuan Zhou Mario Fritz M. Keuper 45 1 0 03 Jun 2024
LOGIN: A Large Language Model Consulted Graph Neural Network Training Framework Yiran Qiao Xiang Ao Yang Liu Jiarong Xu Xiaoqian Sun Qing He 45 4 0 22 May 2024
Pixels and Predictions: Potential of GPT-4V in Meteorological Imagery Analysis and Forecast Communication John R. Lawson Montgomery Flora Kevin H. Goebbert Seth N. Lyman Corey K. Potvin David M. Schultz Adam J. Stepanek Joseph E. Trujillo-Falcón MLLM 58 1 0 22 Apr 2024
Social Skill Training with Large Language Models Diyi Yang Caleb Ziems William B. Held Omar Shaikh Michael S. Bernstein John C. Mitchell LLMAG 51 8 0 05 Apr 2024
Conceptual and Unbiased Reasoning in Language Models Ben Zhou Hongming Zhang Sihao Chen Dian Yu Hongwei Wang Baolin Peng Dan Roth Dong Yu ReLM LRM ELM 39 12 0 30 Mar 2024
RAFT: Adapting Language Model to Domain Specific RAG Tianjun Zhang Shishir G. Patil Naman Jain Sheng Shen Matei A. Zaharia Ion Stoica Joseph E. Gonzalez RALM 37 182 0 15 Mar 2024
Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought James Chua Edward Rees Hunar Batra Samuel R. Bowman Julian Michael Ethan Perez Miles Turpin LRM 50 13 0 08 Mar 2024
Can't Remember Details in Long Documents? You Need Some R&R Devanshu Agrawal Shang Gao Martin Gajek RALM 197 6 0 08 Mar 2024
CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following Kaiyan Zhang Jianyu Wang Ermo Hua Biqing Qi Ning Ding Bowen Zhou SyDa 38 20 0 05 Mar 2024
LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History Akash Gupta Ivaxi Sheth Vyas Raina Mark Gales Mario Fritz 48 4 0 28 Feb 2024
Are LLMs Rational Investors? A Study on Detecting and Reducing the Financial Bias in LLMs Yuhang Zhou Yuchen Ni Yu Gan Zhangyue Yin Xiang Liu Jian Zhang Sen Liu Xipeng Qiu Guangnan Ye Hongfeng Chai AIFin 49 4 0 20 Feb 2024
Rethinking Human-like Translation Strategy: Integrating Drift-Diffusion Model with Large Language Models for Machine Translation Hongbin Na Zimu Wang M. Maimaiti Tong Chen Wei Wang Tao Shen Ling Chen LRM 27 5 0 16 Feb 2024
Chain of Logic: Rule-Based Reasoning with Large Language Models Sergio Servantez Joe Barrow Kristian J. Hammond R. Jain ReLM ELM AILaw LRM AI4CE 37 2 0 16 Feb 2024
A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts Kuang-Huei Lee Xinyun Chen Hiroki Furuta John F. Canny Ian S. Fischer RALM 55 30 0 15 Feb 2024
Measuring and Controlling Instruction (In)Stability in Language Model Dialogs Kenneth Li Tianle Liu Naomi Bashkansky David Bau Fernanda Viégas Hanspeter Pfister Martin Wattenberg 18 6 0 13 Feb 2024
Refined Direct Preference Optimization with Synthetic Data for Behavioral Alignment of LLMs Víctor Gallego SyDa 35 6 0 12 Feb 2024
A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications Pranab Sahoo Ayush Kumar Singh Sriparna Saha Vinija Jain S. Mondal Aman Chadha 69 280 0 05 Feb 2024
Demystifying Chains, Trees, and Graphs of Thoughts Maciej Besta Florim Memedi Zhenyu Zhang Robert Gerstenberger Guangyuan Piao ... Aleš Kubíček H. Niewiadomski Aidan O'Mahony Onur Mutlu Torsten Hoefler AI4CE LRM 75 27 0 25 Jan 2024
DocFinQA: A Long-Context Financial Reasoning Dataset Varshini Reddy Rik Koncel-Kedziorski Viet Dac Lai Michael Krumdick Charles Lovering Chris Tanner RALM 35 16 0 12 Jan 2024
Large Language Models for Social Networks: Applications, Challenges, and Solutions Jingying Zeng Richard Huang Waleed Malik Langxuan Yin Bojan Babic Danny Shacham Xiao Yan Jaewon Yang Qi He 22 7 0 04 Jan 2024
A Survey of Reasoning with Foundation Models Jiankai Sun Chuanyang Zheng Enze Xie Zhengying Liu Ruihang Chu ... Xipeng Qiu Yi-Chen Guo Hui Xiong Qun Liu Zhenguo Li ReLM LRM AI4CE 30 77 0 17 Dec 2023
Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves Yihe Deng Weitong Zhang Zixiang Chen Quanquan Gu LRM 40 73 0 07 Nov 2023
Towards Understanding Sycophancy in Language Models Mrinank Sharma Meg Tong Tomasz Korbak David Duvenaud Amanda Askell ... Oliver Rausch Nicholas Schiefer Da Yan Miranda Zhang Ethan Perez 227 197 0 20 Oct 2023
Concise and Organized Perception Facilitates Reasoning in Large Language Models Junjie Liu Shaotian Yan Chen Shen Zhengdong Xiao Wenxiao Wang Jieping Ye Jieping Ye LRM 26 1 0 05 Oct 2023
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 328 4,142 0 24 May 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 447 8,699 0 28 Jan 2022
UnNatural Language Inference Koustuv Sinha Prasanna Parthasarathi Joelle Pineau Adina Williams 226 80 0 30 Dec 2020