Teaching Small Language Models to Reason

16 December 2022

Lucie Charlotte Magister

Papers citing "Teaching Small Language Models to Reason"

41 / 191 papers shown

Title
Answering Unseen Questions With Smaller Language Models Using Rationale Generation and Dense Retrieval Tim Hartill Diana Benavides-Prado Michael Witbrock Patricia J. Riddle ReLM LRM 28 1 0 09 Aug 2023
Sci-CoT: Leveraging Large Language Models for Enhanced Knowledge Distillation in Small Models for Scientific QA Yuhan Ma Haiqi Jiang Chenyou Fan LRM 34 14 0 09 Aug 2023
Learning Evaluation Models from Large Language Models for Sequence Generation Chenglong Wang Hang Zhou Kai-Chun Chang Tongran Liu Chunliang Zhang Quan Du Tong Xiao Yue Zhang Jingbo Zhu ELM 46 3 0 08 Aug 2023
Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models Jiaao Chen Xiaoman Pan Dian Yu Kaiqiang Song Xiaoyang Wang Dong Yu Jianshu Chen ReLM LRM 21 24 0 01 Aug 2023
MinT: Boosting Generalization in Mathematical Reasoning via Multi-View Fine-Tuning Zhenwen Liang Dian Yu Xiaoman Pan Wenlin Yao Qingkai Zeng Xiangliang Zhang Dong Yu ALM LRM 46 13 0 16 Jul 2023
OntoChatGPT Information System: Ontology-Driven Structured Prompts for ChatGPT Meta-Learning O. Palagin Vladislav Kaverinskiy Anna Litvin Kyrylo S. Malakhov KELM 11 24 0 11 Jul 2023
Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step Liunian Harold Li Jack Hessel Youngjae Yu Xiang Ren Kai-Wei Chang Yejin Choi LRM AI4CE ReLM 22 129 0 24 Jun 2023
On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes Rishabh Agarwal Nino Vieillard Yongchao Zhou Piotr Stańczyk Sabela Ramos Matthieu Geist Olivier Bachem 40 4 0 23 Jun 2023
Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Personalization Swarnadeep Saha Peter Hase Mohit Bansal LRM 27 10 0 15 Jun 2023
Leveraging Training Data in Few-Shot Prompting for Numerical Reasoning Zhanming Jie Wei Lu LRM ReLM 35 15 0 29 May 2023
Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks Minki Kang Seanie Lee Jinheon Baek Kenji Kawaguchi Sung Ju Hwang ALM LRM 52 56 0 28 May 2023
Learning from Mistakes via Cooperative Study Assistant for Large Language Models Danqing Wang Lei Li 34 6 0 23 May 2023
Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generation Zhenwen Liang W. Yu Tanmay Rajpurohit Peter Clark Xiangliang Zhang Ashwin Kaylan 32 37 0 22 May 2023
Enhance Reasoning Ability of Visual-Language Models via Large Language Models Yueting Yang Xintong Zhang Wenjuan Han VLM ReLM LRM 28 1 0 22 May 2023
Evaluation of medium-large Language Models at zero-shot closed book generative question answering René Peinl Johannes Wirth ELM 26 7 0 19 May 2023
CCGen: Explainable Complementary Concept Generation in E-Commerce Jie Huang Yifan Gao Zheng Li Jingfeng Yang Yangqiu Song Chao Zhang Zining Zhu Haoming Jiang Kevin Chen-Chuan Chang Bing Yin 3DV LRM 18 5 0 19 May 2023
MoT: Memory-of-Thought Enables ChatGPT to Self-Improve Xiaonan Li Xipeng Qiu ReLM KELM LRM AI4MH 26 32 0 09 May 2023
T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question Answering Lei Wang Yilang Hu Jiabang He Xingdong Xu Ning Liu Hui-juan Liu Hengtao Shen LRM MLLM 31 41 0 05 May 2023
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes Lokesh Nagalapatti Chun-Liang Li Chih-Kuan Yeh Hootan Nakhost Yasuhisa Fujii Alexander Ratner Ranjay Krishna Chen-Yu Lee Tomas Pfister ALM 220 502 0 03 May 2023
Distill or Annotate? Cost-Efficient Fine-Tuning of Compact Models Junmo Kang Wei-ping Xu Alan Ritter 47 15 0 02 May 2023
CONSCENDI: A Contrastive and Scenario-Guided Distillation Approach to Guardrail Models for Virtual Assistants A. Sun Varun Nair Elliot Schumacher Anitha Kannan 32 3 0 27 Apr 2023
Boosting Theory-of-Mind Performance in Large Language Models via Prompting Shima Rahimi Moghaddam C. Honey LLMAG LRM AI4CE 16 78 0 22 Apr 2023
Natural Language Reasoning, A Survey Fei Yu Hongbo Zhang Prayag Tiwari Benyou Wang ReLM LRM 49 51 0 26 Mar 2023
What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring Yonadav Shavit 31 22 0 20 Mar 2023
Is ChatGPT a General-Purpose Natural Language Processing Task Solver? Chengwei Qin Aston Zhang Zhuosheng Zhang Jiaao Chen Michihiro Yasunaga Diyi Yang LM&MA AI4MH LRM ELM 34 670 0 08 Feb 2023
Multimodal Chain-of-Thought Reasoning in Language Models Zhuosheng Zhang Aston Zhang Mu Li Hai Zhao George Karypis Alexander J. Smola LRM 30 409 0 02 Feb 2023
Specializing Smaller Language Models towards Multi-Step Reasoning Yao Fu Hao-Chun Peng Litu Ou Ashish Sabharwal Tushar Khot ReLM LRM 42 242 0 30 Jan 2023
ExaRanker: Explanation-Augmented Neural Ranker Fernando Ferraretto Thiago Laitz R. Lotufo Rodrigo Nogueira ELM LRM 36 7 0 25 Jan 2023
A Survey on In-context Learning Qingxiu Dong Lei Li Damai Dai Ce Zheng Jingyuan Ma ... Zhiyong Wu Baobao Chang Xu Sun Lei Li Zhifang Sui ReLM AIMat 20 464 0 31 Dec 2022
Self-Instruct: Aligning Language Models with Self-Generated Instructions Yizhong Wang Yeganeh Kordi Swaroop Mishra Alisa Liu Noah A. Smith Daniel Khashabi Hannaneh Hajishirzi ALM SyDa LRM 39 2,079 0 20 Dec 2022
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions H. Trivedi Niranjan Balasubramanian Tushar Khot Ashish Sabharwal KELM RALM LRM 37 388 0 20 Dec 2022
Towards Reasoning in Large Language Models: A Survey Jie Huang Kevin Chen-Chuan Chang LM&MA ELM LRM 29 583 0 20 Dec 2022
Large Language Models Are Reasoning Teachers Namgyu Ho Laura Schmid Se-Young Yun ReLM ELM LRM 37 317 0 20 Dec 2022
KNIFE: Distilling Reasoning Knowledge From Free-Text Rationales Aaron Chan Zhiyuan Zeng Wyatt Lake Brihi Joshi Hanjie Chen Xiang Ren ReLM LRM 37 1 0 19 Dec 2022
Reasoning with Language Model Prompting: A Survey Shuofei Qiao Yixin Ou Ningyu Zhang Xiang Chen Yunzhi Yao Shumin Deng Chuanqi Tan Fei Huang Huajun Chen ReLM ELM LRM 71 311 0 19 Dec 2022
Large Language Models Can Self-Improve Jiaxin Huang S. Gu Le Hou Yuexin Wu Xuezhi Wang Hongkun Yu Jiawei Han ReLM AI4MH LRM 47 564 0 20 Oct 2022
Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model Jacob Eisenstein D. Andor Bernd Bohnet Michael Collins David M. Mimno LRM 191 24 0 05 Oct 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 314 3,273 0 21 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 395 8,559 0 28 Jan 2022
Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics Artidoro Pagnoni Vidhisha Balachandran Yulia Tsvetkov HILM 231 306 0 27 Apr 2021
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies Mor Geva Daniel Khashabi Elad Segal Tushar Khot Dan Roth Jonathan Berant RALM 250 677 0 06 Jan 2021