Muppet: Massive Multi-task Representations with Pre-Finetuning

26 January 2021

Luke Zettlemoyer

Papers citing "Muppet: Massive Multi-task Representations with Pre-Finetuning"

21 / 171 papers shown

Title
Pruning Pretrained Encoders with a Multitask Objective Patrick Xia Richard Shin 74 0 0 10 Dec 2021
UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Faisal Ahmed Zicheng Liu Yumao Lu Lijuan Wang 149 117 0 23 Nov 2021
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning V. Aribandi Yi Tay Tal Schuster J. Rao H. Zheng ... Jianmo Ni Jai Gupta Kai Hui Sebastian Ruder Donald Metzler MoE 131 216 0 22 Nov 2021
Learning to Generalize Compositionally by Transferring Across Semantic Parsing Tasks Wang Zhu Peter Shaw Tal Linzen Fei Sha 108 7 0 09 Nov 2021
MetaICL: Learning to Learn In Context Sewon Min M. Lewis Luke Zettlemoyer Hannaneh Hajishirzi LRM 273 493 0 29 Oct 2021
Deep Transfer Learning & Beyond: Transformer Language Models in Information Systems Research Ross Gruetzemacher D. Paradice 97 35 0 18 Oct 2021
Meta-learning via Language Model In-context Tuning Yanda Chen Ruiqi Zhong Sheng Zha George Karypis He He 334 163 0 15 Oct 2021
CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training Patrick Huber Armen Aghajanyan Barlas Oğuz Dmytro Okhonko Wen-tau Yih Sonal Gupta Xilun Chen 92 15 0 14 Oct 2021
Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System Yixuan Su Lei Shu Elman Mansimov Arshit Gupta Deng Cai Yi-An Lai Yi Zhang 229 195 0 29 Sep 2021
Cross-lingual Intermediate Fine-tuning improves Dialogue State Tracking Nikita Moghe Mark Steedman Alexandra Birch 107 13 0 28 Sep 2021
Should We Be Pre-training? An Argument for End-task Aware Training as an Alternative Lucio Dery Paul Michel Ameet Talwalkar Graham Neubig CLL 106 35 0 15 Sep 2021
STraTA: Self-Training with Task Augmentation for Better Few-shot Learning Tu Vu Minh-Thang Luong Quoc V. Le Grady Simon Mohit Iyyer 208 61 0 13 Sep 2021
CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation Yunfan Shao Zhichao Geng Yitao Liu Junqi Dai Hang Yan Fei Yang Li Zhe Hujun Bao Xipeng Qiu MedIm 161 152 0 13 Sep 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 427 3,819 0 03 Sep 2021
Domain-matched Pre-training Tasks for Dense Retrieval Barlas Oğuz Kushal Lakhotia Anchit Gupta Patrick Lewis Vladimir Karpukhin ... Xilun Chen Sebastian Riedel Wen-tau Yih Sonal Gupta Yashar Mehdad RALM 87 67 0 28 Jul 2021
DaCy: A Unified Framework for Danish NLP Kenneth Enevoldsen Lasse Hansen Kristoffer Nielbo 63 13 0 12 Jul 2021
Question Answering Infused Pre-training of General-Purpose Contextualized Representations Robin Jia M. Lewis Luke Zettlemoyer 82 29 0 15 Jun 2021
Cross-Task Generalization via Natural Language Crowdsourcing Instructions Swaroop Mishra Daniel Khashabi Chitta Baral Hannaneh Hajishirzi LRM 210 756 0 18 Apr 2021
Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections Ruiqi Zhong Kristy Lee Zheng Zhang Dan Klein 196 173 0 10 Apr 2021
Multi-task Retrieval for Knowledge-Intensive Tasks Jean Maillard Vladimir Karpukhin Fabio Petroni Wen-tau Yih Barlas Oğuz Veselin Stoyanov Gargi Ghosh 271 67 0 01 Jan 2021
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning Armen Aghajanyan Luke Zettlemoyer Sonal Gupta 124 582 1 22 Dec 2020