Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective

24 May 2023

Papers citing "Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective"

50 / 166 papers shown

Title
RATT: A Thought Structure for Coherent and Correct LLM Reasoning Jinghan Zhang Xiting Wang Weijieying Ren Lu Jiang Dongjie Wang Kunpeng Liu LRM 36 9 0 04 Jun 2024
HoneyGPT: Breaking the Trilemma in Terminal Honeypots with Large Language Model Ziyang Wang Jianzhou You Haining Wang Tianwei Yuan Shichao Lv Yang Wang Limin Sun 39 2 0 04 Jun 2024
Rethinking Transformers in Solving POMDPs Chenhao Lu Ruizhe Shi Yuyao Liu Kaizhe Hu Simon S. Du Huazhe Xu AI4CE 35 3 0 27 May 2024
Dissociation of Faithful and Unfaithful Reasoning in LLMs Evelyn Yee Alice Li Chenyu Tang Yeon Ho Jung R. Paturi Leon Bergen LRM 32 4 0 23 May 2024
Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization Boshi Wang Xiang Yue Yu-Chuan Su Huan Sun LRM 29 42 0 23 May 2024
WorldAfford: Affordance Grounding based on Natural Language Instructions Changmao Chen Yuren Cong Zhen Kan 24 4 0 21 May 2024
ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models Siwei Wang Yifei Shen Shi Feng Haoran Sun Shang-Hua Teng Wei Chen 41 4 0 15 May 2024
Chain of Thoughtlessness? An Analysis of CoT in Planning Kaya Stechly Karthik Valmeekam Subbarao Kambhampati LRM LM&Ro 75 40 0 08 May 2024
Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics Hanlin Zhu Baihe Huang Shaolun Zhang Michael I. Jordan Jiantao Jiao Yuandong Tian Stuart Russell LRM AI4CE 55 13 0 07 May 2024
A Transformer with Stack Attention Jiaoda Li Jennifer C. White Mrinmaya Sachan Ryan Cotterell 30 2 0 07 May 2024
Let's Think Dot by Dot: Hidden Computation in Transformer Language Models Jacob Pfau William Merrill Samuel R. Bowman LRM 31 61 0 24 Apr 2024
UrbanCross: Enhancing Satellite Image-Text Retrieval with Cross-Domain Adaptation Siru Zhong Xixuan Hao Yibo Yan Ying Zhang Yangqiu Song Keli Zhang 48 8 0 22 Apr 2024
On the Empirical Complexity of Reasoning and Planning in LLMs Liwei Kang Zirui Zhao David Hsu Wee Sun Lee LRM 30 5 0 17 Apr 2024
The Illusion of State in State-Space Models William Merrill Jackson Petty Ashish Sabharwal 54 44 0 12 Apr 2024
Transformers as Transducers Lena Strobl Dana Angluin David Chiang Jonathan Rawski Ashish Sabharwal 31 5 0 02 Apr 2024
What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks Xingwu Chen Difan Zou ViT 26 12 0 02 Apr 2024
A Theory for Length Generalization in Learning to Reason Changnan Xiao Bing Liu LRM 47 9 0 31 Mar 2024
MSCoTDet: Language-driven Multi-modal Fusion for Improved Multispectral Pedestrian Detection Taeheon Kim Sangyun Chung Damin Yeom Youngjoon Yu Hak Gu Kim Y. Ro 38 2 0 22 Mar 2024
NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokens Cunxiang Wang Ruoxi Ning Boqi Pan Tonghui Wu Qipeng Guo ... Guangsheng Bao Xiangkun Hu Zheng Zhang Qian Wang Yue Zhang RALM 106 4 0 18 Mar 2024
The pitfalls of next-token prediction Gregor Bachmann Vaishnavh Nagarajan 37 63 0 11 Mar 2024
GPT as Psychologist? Preliminary Evaluations for GPT-4V on Visual Affective Computing Hao Lu Xuesong Niu Jiyao Wang Yin Wang Qingyong Hu ... Dengbo He Shuiguang Deng Hao Chen Ying-Cong Chen Shiguang Shan MLLM 54 11 0 09 Mar 2024
DiaHalu: A Dialogue-level Hallucination Evaluation Benchmark for Large Language Models Kedi Chen Qin Chen Jie Zhou Yishen He Liang He HILM 38 1 0 01 Mar 2024
RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval Kaiyue Wen Xingyu Dang Kaifeng Lyu 52 24 0 28 Feb 2024
How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning Subhabrata Dutta Joykirat Singh Soumen Chakrabarti Tanmoy Chakraborty LRM 45 23 0 28 Feb 2024
Divide-or-Conquer? Which Part Should You Distill Your LLM? Zhuofeng Wu Richard He Bai Aonan Zhang Jiatao Gu V. Vydiswaran Navdeep Jaitly Yizhe Zhang LRM 40 6 0 22 Feb 2024
Do Efficient Transformers Really Save Computation? Kai-Bo Yang Jan Ackermann Zhenyu He Guhao Feng Bohang Zhang Yunzhen Feng Qiwei Ye Di He Liwei Wang 42 8 0 21 Feb 2024
BBA: Bi-Modal Behavioral Alignment for Reasoning with Large Vision-Language Models Xueliang Zhao Xinting Huang Tingchen Fu Qintong Li Shansan Gong Lemao Liu Wei Bi Lingpeng Kong LRM 37 1 0 21 Feb 2024
Chain of Thought Empowers Transformers to Solve Inherently Serial Problems Zhiyuan Li Hong Liu Denny Zhou Tengyu Ma LRM AI4CE 30 101 0 20 Feb 2024
Chain-of-Thought Reasoning Without Prompting Xuezhi Wang Denny Zhou ReLM LRM 152 102 0 15 Feb 2024
Why are Sensitive Functions Hard for Transformers? Michael Hahn Mark Rofin 41 25 0 15 Feb 2024
Toward a Team of AI-made Scientists for Scientific Discovery from Gene Expression Data Haoyang Liu Yijiang Li Jinglin Jian Yuxuan Cheng Jianrong Lu Shuyi Guo Jinglei Zhu Mianchen Zhang Miantong Zhang Haohan Wang 19 4 0 15 Feb 2024
On Limitations of the Transformer Architecture Binghui Peng Srini Narayanan Christos H. Papadimitriou 32 32 0 13 Feb 2024
Towards an Understanding of Stepwise Inference in Transformers: A Synthetic Graph Navigation Model Mikail Khona Maya Okawa Jan Hula Rahul Ramesh Kento Nishi Robert P. Dick Ekdeep Singh Lubana Hidenori Tanaka 46 5 0 12 Feb 2024
Limits of Transformer Language Models on Learning to Compose Algorithms Jonathan Thomm Aleksandar Terzić Giacomo Camposampiero Michael Hersche Bernhard Schölkopf Abbas Rahimi 39 3 0 08 Feb 2024
An Examination on the Effectiveness of Divide-and-Conquer Prompting in Large Language Models Yizhou Zhang Lun Du Defu Cao Qiang Fu Yan Liu LRM 25 7 0 08 Feb 2024
Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation Xinyi Wang Alfonso Amayuelas Kexun Zhang Liangming Pan Wenhu Chen Luu Anh Tuan LRM 40 12 0 05 Feb 2024
Investigating Recurrent Transformers with Dynamic Halt Jishnu Ray Chowdhury Cornelia Caragea 39 1 0 01 Feb 2024
Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation Zhenyu He Guhao Feng Shengjie Luo Kai-Bo Yang Liwei Wang Jingjing Xu Zhi Zhang Hongxia Yang Di He 32 14 0 29 Jan 2024
Enhancing Emotional Generation Capability of Large Language Models via Emotional Chain-of-Thought Zaijing Li Gongwei Chen Rui Shao Dongmei Jiang Liqiang Nie 34 12 0 12 Jan 2024
Rethinking Model-based, Policy-based, and Value-based Reinforcement Learning via the Lens of Representation Complexity Guhao Feng Han Zhong OffRL 76 2 0 28 Dec 2023
Conditions for Length Generalization in Learning Reasoning Skills Changnan Xiao Bing Liu LRM 40 7 0 22 Nov 2023
Meta Prompting for AI Systems Yifan Zhang Yang Yuan Andrew Chi-Chih Yao LLMAG LRM 29 5 0 20 Nov 2023
Contrastive Chain-of-Thought Prompting Yew Ken Chia Guizhen Chen Anh Tuan Luu Soujanya Poria Lidong Bing LRM AI4CE 65 31 0 15 Nov 2023
How are Prompts Different in Terms of Sensitivity? Sheng Lu Hendrik Schuff Iryna Gurevych 40 18 0 13 Nov 2023
The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis Yuxiang Zhou Jiazheng Li Yanzheng Xiang Hanqi Yan Lin Gui Yulan He 24 14 0 01 Nov 2023
What Formal Languages Can Transformers Express? A Survey Lena Strobl William Merrill Gail Weiss David Chiang Dana Angluin AI4CE 20 48 0 01 Nov 2023
Defining a New NLP Playground Sha Li Chi Han Pengfei Yu Carl Edwards Manling Li ... Yi R. Fung Charles Yu Joel R. Tetreault Eduard H. Hovy Heng Ji 41 5 0 31 Oct 2023
Small Language Models Fine-tuned to Coordinate Larger Language Models improve Complex Reasoning Gurusha Juneja Subhabrata Dutta Soumen Chakrabarti Sunny Manchanda Tanmoy Chakraborty LRM ReLM 16 15 0 21 Oct 2023
The Expressive Power of Transformers with Chain of Thought William Merrill Ashish Sabharwal LRM AI4CE ReLM 27 41 0 11 Oct 2023
Guiding Language Model Math Reasoning with Planning Tokens Xinyi Wang Lucas Caccia O. Ostapenko Xingdi Yuan William Yang Wang Alessandro Sordoni LRM 33 2 0 09 Oct 2023