PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuning

23 May 2023

Papers citing "PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuning"

21 / 21 papers shown

Title
Retrieval-Augmented Visual Question Answering via Built-in Autoregressive Search Engines Xinwei Long Zhiyuan Ma Ermo Hua Kaiyan Zhang Biqing Qi Bowen Zhou RALM 48 0 0 23 Feb 2025
Position: AI Scaling: From Up to Down and Out Yunke Wang Yanxi Li Chang Xu HAI 88 1 0 02 Feb 2025
RAT: Adversarial Attacks on Deep Reinforcement Agents for Targeted Behaviors Fengshuo Bai Runze Liu Yali Du Ying Wen Yaodong Yang AAML 88 3 0 14 Dec 2024
Dynamic Self-Distillation via Previous Mini-batches for Fine-tuning Small Language Models Y. Fu Yin Yu Xiaotian Han Runchao Li Xianxuan Long Haotian Yu Pan Li SyDa 67 0 0 25 Nov 2024
ToW: Thoughts of Words Improve Reasoning in Large Language Models Zhikun Xu Ming shen Jacob Dineen Zhaonan Li Xiao Ye Shijie Lu Aswin Rrv Chitta Baral Ben Zhou LRM 149 1 0 21 Oct 2024
Mentor-KD: Making Small Language Models Better Multi-step Reasoners Hojae Lee Junho Kim SangKeun Lee LRM 34 1 0 11 Oct 2024
Inference Optimizations for Large Language Models: Effects, Challenges, and Practical Considerations Leo Donisch Sigurd Schacht Carsten Lanquillon 30 2 0 06 Aug 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 41 48 0 15 Feb 2024
Mixed Distillation Helps Smaller Language Model Better Reasoning Chenglin Li Qianglong Chen Liangyue Li Wang Caiyu Yicheng Li Zhang Yin Yin Zhang LRM 41 11 0 17 Dec 2023
Grounding Foundation Models through Federated Transfer Learning: A General Framework Yan Kang Tao Fan Hanlin Gu Xiaojin Zhang Lixin Fan Qiang Yang AI4CE 68 19 0 29 Nov 2023
Large Language Models are Zero Shot Hypothesis Proposers Biqing Qi Kaiyan Zhang Haoxiang Li Kai Tian Sihang Zeng Zhang-Ren Chen Bowen Zhou 26 27 0 10 Nov 2023
FIAT: Fusing learning paradigms with Instruction-Accelerated Tuning Xinyi Wang John Wieting J. Clark CLL ALM 26 1 0 09 Sep 2023
A Survey on Model Compression for Large Language Models Xunyu Zhu Jian Li Yong Liu Can Ma Weiping Wang 36 193 0 15 Aug 2023
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes Lokesh Nagalapatti Chun-Liang Li Chih-Kuan Yeh Hootan Nakhost Yasuhisa Fujii Alexander Ratner Ranjay Krishna Chen-Yu Lee Tomas Pfister ALM 220 502 0 03 May 2023
SCOTT: Self-Consistent Chain-of-Thought Distillation Jamie Yap Zhengyang Wang Zheng Li K. Lynch Bing Yin Xiang Ren LRM 64 93 0 03 May 2023
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 328 4,077 0 24 May 2022
Teaching language models to support answers with verified quotes Jacob Menick Maja Trebacz Vladimir Mikulik John Aslanides Francis Song ... Mia Glaese Susannah Young Lucy Campbell-Gillingham G. Irving Nat McAleese ELM RALM 246 259 0 21 Mar 2022
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation Yue Wang Weishi Wang Shafiq R. Joty S. Hoi 238 1,489 0 02 Sep 2021
Neurosymbolic AI: The 3rd Wave Artur Garcez Luís C. Lamb NAI 65 292 0 10 Dec 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 261 4,489 0 23 Jan 2020
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 417 2,588 0 03 Sep 2019