v1v2v3 (latest)

Sparse*BERT: Sparse Models Generalize To New tasks and Domains

25 May 2022

Daniel Fernando Campos

Papers citing "Sparse*BERT: Sparse Models Generalize To New tasks and Domains"

20 / 20 papers shown

Title
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models Eldar Kurtic Daniel Fernando Campos Tuan Nguyen Elias Frantar Mark Kurtz Ben Fineran Michael Goin Dan Alistarh VLM MQ MedIm 92 126 0 14 Mar 2022
Prune Once for All: Sparse Pre-Trained Language Models Ofir Zafrir Ariel Larey Guy Boudoukh Haihao Shen Moshe Wasserblat VLM 55 85 0 10 Nov 2021
BERT Busters: Outlier Dimensions that Disrupt Transformers Olga Kovaleva Saurabh Kulshreshtha Anna Rogers Anna Rumshisky 86 92 0 14 May 2021
ViViT: A Video Vision Transformer Anurag Arnab Mostafa Dehghani G. Heigold Chen Sun Mario Lucic Cordelia Schmid ViT 225 2,163 0 29 Mar 2021
CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification Chun-Fu Chen Quanfu Fan Yikang Shen ViT 71 1,482 0 27 Mar 2021
Random Feature Attention Hao Peng Nikolaos Pappas Dani Yogatama Roy Schwartz Noah A. Smith Lingpeng Kong 107 362 0 03 Mar 2021
Undivided Attention: Are Intermediate Layers Necessary for BERT? S. N. Sridhar Anthony Sarah 50 15 0 22 Dec 2020
LEGAL-BERT: The Muppets straight out of Law School Ilias Chalkidis Manos Fergadiotis Prodromos Malakasiotis Nikolaos Aletras Ion Androutsopoulos AILaw 54 259 0 06 Oct 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 841 42,332 0 28 May 2020
Movement Pruning: Adaptive Sparsity by Fine-Tuning Victor Sanh Thomas Wolf Alexander M. Rush 73 486 0 15 May 2020
Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learning Mitchell A. Gordon Kevin Duh Nicholas Andrews VLM 59 342 0 19 Feb 2020
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 234 7,547 0 02 Oct 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 109 1,869 0 23 Sep 2019
PubMedQA: A Dataset for Biomedical Research Question Answering Qiao Jin Bhuwan Dhingra Zhengping Liu William W. Cohen Xinghua Lu 387 910 0 13 Sep 2019
SciBERT: A Pretrained Language Model for Scientific Text Iz Beltagy Kyle Lo Arman Cohan 160 2,983 0 26 Mar 2019
BioBERT: a pre-trained biomedical language representation model for biomedical text mining Jinhyuk Lee Wonjin Yoon Sungdong Kim Donghyeon Kim Sunkyu Kim Chan Ho So Jaewoo Kang OOD 175 5,667 0 25 Jan 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,114 0 11 Oct 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 728 132,199 0 12 Jun 2017
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 312 8,160 0 16 Jun 2016
Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books Yukun Zhu Ryan Kiros R. Zemel Ruslan Salakhutdinov R. Urtasun Antonio Torralba Sanja Fidler 127 2,554 0 22 Jun 2015