Muppet: Massive Multi-task Representations with Pre-Finetuning

26 January 2021

Luke Zettlemoyer

Papers citing "Muppet: Massive Multi-task Representations with Pre-Finetuning"

50 / 171 papers shown

Title
Unveiling the Black Box of PLMs with Semantic Anchors: Towards Interpretable Neural Semantic Parsing L. Nie Jiu Sun Yanlin Wang Lun Du Lei Hou Juanzi Li Shi Han Dongmei Zhang Jidong Zhai 95 6 0 04 Oct 2022
Code Compliance Assessment as a Learning Problem Neela Sawant Srinivasan H. Sengamedu 76 1 0 10 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 179 114 0 31 Aug 2022
Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization Pengcheng He Baolin Peng Liyang Lu Song Wang Jie Mei ... Chenguang Zhu Wayne Xiong Michael Zeng Jianfeng Gao Xuedong Huang 119 47 0 21 Aug 2022
Coarse-to-Fine: Hierarchical Multi-task Learning for Natural Language Understanding Zhaoye Fei Yu Tian Yongkang Wu Xinyu Zhang Yutao Zhu ... Dejiang Kong Ruofei Lai Bo Zhao Zhicheng Dou Xipeng Qiu 292 1 0 19 Aug 2022
Low-Resource Dense Retrieval for Open-Domain Question Answering: A Comprehensive Survey Xiaoyu Shen Svitlana Vakulenko Marco Del Tredici Gianni Barlacchi Bill Byrne Adria de Gispert RALM VLM 82 20 0 05 Aug 2022
Few-shot Adaptation Works with UnpredicTable Data Jun Shern Chan Michael Pieler Jonathan Jao Jérémy Scheurer Ethan Perez 110 5 0 01 Aug 2022
STT: Soft Template Tuning for Few-Shot Adaptation Ping Yu Wei Wang Chunyuan Li Ruiyi Zhang Zhanpeng Jin Changyou Chen VLM 41 0 0 18 Jul 2022
Big Learning Yulai Cong Miaoyun Zhao AI4CE 108 0 0 08 Jul 2022
Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation Zejiang Hou Julian Salazar George Polovets 79 15 0 07 Jul 2022
BigBIO: A Framework for Data-Centric Biomedical Natural Language Processing Jason Alan Fries Leon Weber Natasha Seelam Gabriel Altay Debajyoti Datta ... Minh Chien Vu Trishala Neeraj Jonas Golde Albert Villanova del Moral Benjamin Beilharz LM&MA 158 50 0 30 Jun 2022
MVP: Multi-task Supervised Pre-training for Natural Language Generation Tianyi Tang Junyi Li Wayne Xin Zhao Ji-Rong Wen 137 24 0 24 Jun 2022
Unified BERT for Few-shot Natural Language Understanding Junyu Lu Ping Yang Ruyi Gan Jing Yang Jiaxing Zhang 81 2 0 24 Jun 2022
KnowDA: All-in-One Knowledge Mixture Model for Data Augmentation in Low-Resource NLP Yufei Wang Jiayi Zheng Can Xu Xiubo Geng Tao Shen Chongyang Tao Daxin Jiang VLM MoE 92 2 0 21 Jun 2022
Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems Jack G. M. FitzGerald Shankar Ananthakrishnan Konstantine Arkoudas Davide Bernardi Abhishek Bhagia ... Pan Wei Haiyang Yu Shuai Zheng Gokhan Tur Premkumar Natarajan ELM 53 30 0 15 Jun 2022
Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization Puyuan Liu Chenyang Huang Lili Mou 102 20 0 28 May 2022
AANG: Automating Auxiliary Learning Lucio Dery Paul Michel M. Khodak Graham Neubig Ameet Talwalkar 122 9 0 27 May 2022
Eliciting and Understanding Cross-Task Skills with Task-Level Mixture-of-Experts Qinyuan Ye Juan Zha Xiang Ren MoE 90 14 0 25 May 2022
DFM: Dialogue Foundation Model for Universal Large-Scale Dialogue-Oriented Task Learning Zhi Chen Jijia Bao Lu Chen Yuncong Liu Da Ma ... Xinhsuai Dong Fujiang Ge Qingliang Miao Jian-Guang Lou Kai Yu ALM AI4CE 88 3 0 25 May 2022
Leveraging QA Datasets to Improve Generative Data Augmentation Dheeraj Mekala Tu Vu Timo Schick Jingbo Shang 104 18 0 25 May 2022
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 219 102 0 24 May 2022
Let the Model Decide its Curriculum for Multitask Learning Neeraj Varshney Swaroop Mishra Chitta Baral 79 8 0 19 May 2022
PreQuEL: Quality Estimation of Machine Translation Outputs in Advance Shachar Don-Yehiya Leshem Choshen Omri Abend 96 11 0 18 May 2022
Few-shot Mining of Naturally Occurring Inputs and Outputs Mandar Joshi Terra Blevins M. Lewis Daniel S. Weld Luke Zettlemoyer 101 1 0 09 May 2022
P^3 Ranker: Mitigating the Gaps between Pre-training and Ranking Fine-tuning with Prompt-based Learning and Pre-finetuning Xiaomeng Hu S. Yu Chenyan Xiong Zhenghao Liu Zhiyuan Liu Geoffrey X. Yu 47 11 0 04 May 2022
Improving In-Context Few-Shot Learning via Self-Supervised Training Mingda Chen Jingfei Du Ramakanth Pasunuru Todor Mihaylov Srini Iyer Ves Stoyanov Zornitsa Kozareva SSL AI4MH 115 67 0 03 May 2022
Exploring the Role of Task Transferability in Large-Scale Multi-Task Learning Vishakh Padmakumar Leonard Lausen Miguel Ballesteros Sheng Zha He He George Karypis 115 20 0 23 Apr 2022
Locally Aggregated Feature Attribution on Natural Language Model Understanding Shenmin Zhang Jin Wang Haitao Jiang Rui Song FAtt 97 4 0 22 Apr 2022
Sparse and Dense Approaches for the Full-rank Retrieval of Responses for Dialogues Gustavo Penha C. Hauff RALM 89 0 0 22 Apr 2022
IndicXNLI: Evaluating Multilingual Inference for Indian Languages Divyanshu Aggarwal V. Gupta Anoop Kunchukuttan 87 28 0 19 Apr 2022
Bridging Cross-Lingual Gaps During Leveraging the Multilingual Sequence-to-Sequence Pretraining for Text Generation and Understanding Changtong Zan Liang Ding Li Shen Yu Cao Weifeng Liu Dacheng Tao LRM 103 8 0 16 Apr 2022
Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners Shashank Gupta Subhabrata Mukherjee K. Subudhi Eduardo Gonzalez Damien Jose Ahmed Hassan Awadallah Jianfeng Gao MoE 84 50 0 16 Apr 2022
In-BoXBART: Get Instructions into Biomedical Multi-Task Learning Mihir Parmar Swaroop Mishra Mirali Purohit Man Luo M. H. Murad Chitta Baral 103 23 0 15 Apr 2022
CLUES: A Benchmark for Learning Classifiers using Natural Language Explanations Rakesh R Menon Sayan Ghosh Shashank Srivastava LRM ELM 110 11 0 14 Apr 2022
NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks Swaroop Mishra Arindam Mitra Neeraj Varshney Bhavdeep Singh Sachdeva Peter Clark Chitta Baral Ashwin Kalyan AIMat ReLM ELM LRM 98 110 0 12 Apr 2022
Fusing finetuned models for better pretraining Leshem Choshen Elad Venezian Noam Slonim Yoav Katz FedML AI4CE MoMe 159 96 0 06 Apr 2022
Task-guided Disentangled Tuning for Pretrained Language Models Jiali Zeng Yu Jiang Shuangzhi Wu Yongjing Yin Mu Li DRL 155 3 0 22 Mar 2022
Label Semantics for Few Shot Named Entity Recognition Jie Ma Miguel Ballesteros Srikanth Doss Kadarundalagi Raghuram Doss Rishita Anubhai S. Mallya Yaser Al-Onaizan Dan Roth 75 68 0 16 Mar 2022
Geographic Adaptation of Pretrained Language Models Valentin Hofmann Goran Glavaš Nikola Ljubevsić J. Pierrehumbert Hinrich Schütze VLM 134 18 0 16 Mar 2022
Hyperdecoders: Instance-specific decoders for multi-task NLP Hamish Ivison Matthew E. Peters AI4CE 121 22 0 15 Mar 2022
SkillNet-NLU: A Sparsely Activated Model for General-Purpose Natural Language Understanding Fan Zhang Duyu Tang Yong Dai Cong Zhou Shuangzhi Wu Shuming Shi CLL MoE 143 12 0 07 Mar 2022
HyperPrompt: Prompt-based Task-Conditioning of Transformers Yun He H. Zheng Yi Tay Jai Gupta Yu Du ... Yaguang Li Zhaoji Chen Donald Metzler Heng-Tze Cheng Ed H. Chi LRM VLM 132 93 0 01 Mar 2022
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? Sewon Min Xinxi Lyu Ari Holtzman Mikel Artetxe M. Lewis Hannaneh Hajishirzi Luke Zettlemoyer LLMAG LRM 202 1,507 0 25 Feb 2022
Using natural language prompts for machine translation Xavier Garcia Orhan Firat AI4CE 98 33 0 23 Feb 2022
UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training Daniel Khashabi Yeganeh Kordi Hannaneh Hajishirzi 111 68 0 23 Feb 2022
Generative multitask learning mitigates target-causing confounding Taro Makino Krzysztof J. Geras Kyunghyun Cho OOD 81 6 0 08 Feb 2022
Artefact Retrieval: Overview of NLP Models with Knowledge Base Access Vilém Zouhar Marius Mosbach Debanjali Biswas Dietrich Klakow KELM 100 4 0 24 Jan 2022
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models Tianbao Xie Chen Henry Wu Peng Shi Ruiqi Zhong Torsten Scholak ... Lingpeng Kong Rui Zhang Noah A. Smith Luke Zettlemoyer Tao Yu LMTD 160 304 0 16 Jan 2022
Assemble Foundation Models for Automatic Code Summarization Jian Gu P. Salza H. Gall 101 36 0 13 Jan 2022
Know Thy Strengths: Comprehensive Dialogue State Tracking Diagnostics Hyundong Justin Cho Chinnadhurai Sankar Christopher Lin Kaushik Ram Sadagopan Shahin Shayandeh Asli Celikyilmaz Jonathan May Ahmad Beirami 130 10 0 15 Dec 2021