Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters

20 December 2022

Luke Zettlemoyer

Huan Sun

LRM

ReLM

ArXiv PDF HTML

Papers citing "Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters"

50 / 58 papers shown

Title
Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models Zhanke Zhou Zhaocheng Zhu Xuan Li Mikhail Galkin Xiao Feng Sanmi Koyejo Jian Tang Bo Han LRM 56 0 0 28 Mar 2025
Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More Arvid Frydenlund LRM 52 0 0 13 Mar 2025
How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach Ayeong Lee Ethan Che Tianyi Peng LRM 47 12 0 03 Mar 2025
A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition Zihan Wang Ziqi Zhao Yougang Lyu Ziyang Chen Maarten de Rijke Z. Z. Ren 66 3 0 25 Feb 2025
BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning Ercong Nie Bo Shao Zifeng Ding Mingyang Wang Helmut Schmid Hinrich Schütze KELM 137 7 0 21 Feb 2025
Towards Sustainable NLP: Insights from Benchmarking Inference Energy in Large Language Models S. Poddar Paramita Koley Janardan Misra Niloy Ganguly Saptarshi Ghosh Saptarshi Ghosh 64 0 0 08 Feb 2025
PICLe: Pseudo-Annotations for In-Context Learning in Low-Resource Named Entity Detection Sepideh Mamooler Syrielle Montariol Alexander Mathis Antoine Bosselut 92 1 0 16 Dec 2024
PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generation Qiyao Xue Xiangyu Yin Boyuan Yang Wei Gao DiffM VGen 80 9 0 30 Nov 2024
Uncovering Autoregressive LLM Knowledge of Thematic Fit in Event Representation Safeyah Khaled Alshemali Daniel Bauer Yuval Marton BDL 155 0 0 19 Oct 2024
Sensitivity of Generative VLMs to Semantically and Lexically Altered Prompts Sri Harsha Dumpala Aman Jaiswal Chandramouli Shama Sastry E. Milios Sageev Oore Hassan Sajjad VLM 26 2 0 16 Oct 2024
To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning Zayne Sprague Fangcong Yin Juan Diego Rodriguez Dongwei Jiang Manya Wadhwa Prasann Singhal Xinyu Zhao Xi Ye Kyle Mahowald Greg Durrett ReLM LRM 119 88 0 18 Sep 2024
Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction Melkamu Mersha Khang Lam Joseph Wood Ali AlShami Jugal Kalita XAI AI4TS 69 28 0 30 Aug 2024
Visual Agents as Fast and Slow Thinkers Guangyan Sun Mingyu Jin Zhenting Wang Cheng-Long Wang Siqi Ma Qifan Wang Ying Nian Wu Ying Nian Wu Dongfang Liu Dongfang Liu LLMAG LRM 79 13 0 16 Aug 2024
Prompting Medical Large Vision-Language Models to Diagnose Pathologies by Visual Question Answering Danfeng Guo Sumitaka Honji LRM 64 0 0 31 Jul 2024
DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusion Huiguo He Huan Yang Zixi Tuo Yuan Zhou Qiuyue Wang Yuhang Zhang Zeyu Liu Wenhao Huang Hongyang Chao Jian Yin DiffM VGen 62 12 0 17 Jul 2024
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Daking Rai Yilun Zhou Shi Feng Abulhair Saparov Ziyu Yao 82 19 0 02 Jul 2024
Leveraging LLMs for Dialogue Quality Measurement Jinghan Jia A. Komma Timothy Leffel Xujun Peng Ajay Nagesh Tamer Soliman Aram Galstyan Anoop Kumar 39 5 0 25 Jun 2024
Chain-of-Probe: Examining the Necessity and Accuracy of CoT Step-by-Step Zezhong Wang Xingshan Zeng Weiwen Liu Yufei Wang Liangyou Li Yasheng Wang Lifeng Shang Xin Jiang Qun Liu Kam-Fai Wong LRM 64 3 0 23 Jun 2024
People will agree what I think: Investigating LLM's False Consensus Effect Junhyuk Choi Yeseon Hong Bugeun Kim 54 0 0 16 Jun 2024
Break the Chain: Large Language Models Can be Shortcut Reasoners Mengru Ding Hanmeng Liu Zhizhang Fu Jian Song Wenbo Xie Yue Zhang KELM LRM 36 7 0 04 Jun 2024
ACCORD: Closing the Commonsense Measurability Gap François Roewer-Després Jinyue Feng Zining Zhu Frank Rudzicz LRM 48 0 0 04 Jun 2024
DaVinci at SemEval-2024 Task 9: Few-shot prompting GPT-3.5 for Unconventional Reasoning Suyash Vardhan Mathur Akshett Rai Jindal Manish Shrivastava LRM 39 1 0 19 May 2024
RDRec: Rationale Distillation for LLM-based Recommendation Xinfeng Wang Jin Cui Yoshimi Suzuki Fumiyo Fukumoto LRM 39 11 0 17 May 2024
Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs Yu Xia Rui Wang Xu Liu Mingyan Li Tong Yu Xiang Chen Julian McAuley Shuai Li LRM 53 19 0 24 Apr 2024
Learning to Maximize Mutual Information for Chain-of-Thought Distillation Xin Chen Hanxian Huang Yanjun Gao Yi Wang Jishen Zhao Ke Ding 35 11 0 05 Mar 2024
GATE X-E : A Challenge Set for Gender-Fair Translations from Weakly-Gendered Languages Spencer Rarrick Ranjita Naik Sundar Poudel Vishal Chowdhary 36 1 0 22 Feb 2024
Chain of Thought Empowers Transformers to Solve Inherently Serial Problems Zhiyuan Li Hong Liu Denny Zhou Tengyu Ma LRM AI4CE 28 96 0 20 Feb 2024
RE-GAINS & EnChAnT: Intelligent Tool Manipulation Systems For Enhanced Query Responses Sahil Girhepuje Siva Sankar Sajeev Purvam Jain Arya Sikder Adithya Rama Varma Ryan George Akshay Govind Srinivasan Mahendra Kurup Ashmit Sinha Sudip Mondal RALM 37 0 0 28 Jan 2024
Gradable ChatGPT Translation Evaluation Hui Jiao Bei Peng Lu Zong Xiaojun Zhang Xinwei Li 33 2 0 18 Jan 2024
Retrieval-augmented Multi-modal Chain-of-Thoughts Reasoning for Large Language Models Bingshuai Liu Chenyang Lyu Zijun Min Zhanyu Wang Jinsong Su Longyue Wang LRM 36 7 0 04 Dec 2023
FinanceBench: A New Benchmark for Financial Question Answering Pranab Islam Anand Kannappan Douwe Kiela Rebecca Qian Nino Scherrer Bertie Vidgen RALM 24 72 0 20 Nov 2023
Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents Zhuosheng Zhang Yao Yao Aston Zhang Xiangru Tang Xinbei Ma ... Yiming Wang Mark B. Gerstein Rui Wang Gongshen Liu Hai Zhao LLMAG LM&Ro LRM 42 53 0 20 Nov 2023
R-Tuning: Instructing Large Language Models to Say `I Don't Know' Hanning Zhang Shizhe Diao Yong Lin Yi R. Fung Qing Lian Xingyao Wang Yangyi Chen Heng Ji Tong Zhang UQLM 39 37 0 16 Nov 2023
Self-Contradictory Reasoning Evaluation and Detection Ziyi Liu Isabelle G. Lee Yongkang Du Soumya Sanyal Jieyu Zhao LRM 30 2 0 16 Nov 2023
Towards A Unified View of Answer Calibration for Multi-Step Reasoning Shumin Deng Ningyu Zhang Nay Oo Bryan Hooi LRM 48 1 0 15 Nov 2023
Empirical Study of Zero-Shot NER with ChatGPT Tingyu Xie Qi Li Jian Zhang Yan Zhang Zuozhu Liu Hongwei Wang LRM ReLM 24 63 0 16 Oct 2023
Selective Demonstrations for Cross-domain Text-to-SQL Shuaichen Chang Eric Fosler-Lussier 20 19 0 10 Oct 2023
Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models Yangyi Chen Karan Sikka Michael Cogswell Heng Ji Ajay Divakaran LRM 36 24 0 08 Sep 2023
PromptNER: Prompting For Named Entity Recognition D. Ashok Zachary Chase Lipton 27 37 0 24 May 2023
Active Learning Principles for In-Context Learning with Large Language Models Katerina Margatina Timo Schick Nikolaos Aletras Jane Dwivedi-Yu 30 39 0 23 May 2023
Dr.ICL: Demonstration-Retrieved In-context Learning Man Luo Xin Xu Zhuyun Dai Panupong Pasupat Mehran Kazemi Chitta Baral Vaiva Imbrasaite Vincent Zhao RALM 26 48 0 23 May 2023
Concept-aware Training Improves In-context Learning Ability of Language Models Michal Štefánik Marek Kadlcík KELM LRM 38 0 0 23 May 2023
Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate Boshi Wang Xiang Yue Huan Sun ELM LRM 46 60 0 22 May 2023
Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach Masahiro Kaneko Graham Neubig Naoaki Okazaki 39 6 0 19 May 2023
How Many Demonstrations Do You Need for In-context Learning? Jiuhai Chen Lichang Chen Chen Zhu Dinesh Manocha LRM 24 39 0 14 Mar 2023
ICL-D3IE: In-Context Learning with Diverse Demonstrations Updating for Document Information Extraction Jiabang He Lei Wang Yingpeng Hu Ning Liu Hui-juan Liu Xingdong Xu Hengtao Shen MLLM 6 47 0 09 Mar 2023
Larger language models do in-context learning differently Jerry W. Wei Jason W. Wei Yi Tay Dustin Tran Albert Webson ... Xinyun Chen Hanxiao Liu Da Huang Denny Zhou Tengyu Ma ReLM LRM 49 353 0 07 Mar 2023
Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Following Seonghyeon Ye Hyeonbin Hwang Sohee Yang Hyeongu Yun Yireun Kim Minjoon Seo LRM 32 34 0 28 Feb 2023
Large Language Models Can Be Easily Distracted by Irrelevant Context Freda Shi Xinyun Chen Kanishka Misra Nathan Scales David Dohan Ed H. Chi Nathanael Scharli Denny Zhou ReLM RALM LRM 30 530 0 31 Jan 2023
Towards Reasoning in Large Language Models: A Survey Jie Huang Kevin Chen-Chuan Chang LM&MA ELM LRM 29 583 0 20 Dec 2022