MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models

2 February 2024

Justin Chih-Yao Chen

Swarnadeep Saha

Elias Stengel-Eskin

Mohit Bansal

LRM

LLMAG

ArXiv PDF HTML

Papers citing "MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models"

21 / 21 papers shown

Title
Spatial Audio Processing with Large Language Model on Wearable Devices Ayushi Mishra Yang Bai Priyadarshan Narayanasamy Nakul Garg Nirupam Roy 65 1 0 11 Apr 2025
Don't lie to your friends: Learning what you know from collaborative self-play Jacob Eisenstein Reza Aghajani Adam Fisch Dheeru Dua Fantine Huot Mirella Lapata Vicky Zayats Jonathan Berant 96 0 0 18 Mar 2025
DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback Zaid Khan Elias Stengel-Eskin Jaemin Cho Joey Tianyi Zhou VGen 121 3 0 08 Oct 2024
Orca 2: Teaching Small Language Models How to Reason Arindam Mitra Luciano Del Corro Shweti Mahajan Andres Codas Clarisse Simoes ... Hamid Palangi Guoqing Zheng Corby Rosset Hamed Khanpour Ahmed Hassan Awadallah ReLM LRM 72 141 0 18 Nov 2023
Implicit Chain of Thought Reasoning via Knowledge Distillation Yuntian Deng Kiran Prasad Roland Fernandez P. Smolensky Vishrav Chaudhary Stuart M. Shieber ReLM LRM 49 51 0 02 Nov 2023
Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step Liunian Harold Li Jack Hessel Youngjae Yu Xiang Ren Kai-Wei Chang Yejin Choi LRM AI4CE ReLM 57 141 0 24 Jun 2023
Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Personalization Swarnadeep Saha Peter Hase Mohit Bansal LRM 44 11 0 15 Jun 2023
Large Language Models Are Reasoning Teachers Namgyu Ho Laura Schmid Se-Young Yun ReLM ELM LRM 93 344 0 20 Dec 2022
Teaching Small Language Models to Reason Lucie Charlotte Magister Jonathan Mallinson Jakub Adamek Eric Malmi Aliaksei Severyn LRM AI4CE ReLM 158 268 0 16 Dec 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 763 9,330 0 28 Jan 2022
Ethical and social risks of harm from Language Models Laura Weidinger John F. J. Mellor Maribeth Rauh Conor Griffin J. Uesato ... Lisa Anne Hendricks William S. Isaac Sean Legassick G. Irving Iason Gabriel PILM 97 1,030 0 08 Dec 2021
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 236 4,392 0 27 Oct 2021
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models Peter West Chandrasekhar Bhagavatula Jack Hessel Jena D. Hwang Liwei Jiang Ronan Le Bras Ximing Lu Sean Welleck Yejin Choi SyDa 97 333 0 14 Oct 2021
Directed Acyclic Graph Network for Conversational Emotion Recognition Weizhou Shen Siyue Wu Yunyi Yang Xiaojun Quan 65 241 0 27 May 2021
Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs Jiaao Chen Diyi Yang 57 98 0 16 Apr 2021
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies Mor Geva Daniel Khashabi Elad Segal Tushar Khot Dan Roth Jonathan Berant RALM 330 718 0 06 Jan 2021
Co-GAT: A Co-Interactive Graph Attention Network for Joint Dialog Act Recognition and Sentiment Classification Libo Qin Zhouyang Li Wanxiang Che Minheng Ni Ting Liu 67 66 0 24 Dec 2020
DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation Deepanway Ghosal Navonil Majumder Soujanya Poria Niyati Chhaya Alexander Gelbukh 77 513 0 30 Aug 2019
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions Christopher Clark Kenton Lee Ming-Wei Chang Tom Kwiatkowski Michael Collins Kristina Toutanova 213 1,516 0 24 May 2019
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge Alon Talmor Jonathan Herzig Nicholas Lourie Jonathan Berant RALM 140 1,725 0 02 Nov 2018
Semi-Supervised Classification with Graph Convolutional Networks Thomas Kipf Max Welling GNN SSL 602 29,032 0 09 Sep 2016