Title
Improving Model Evaluation using SMART Filtering of Benchmark Datasets Vipul Gupta Candace Ross David Pantoja R. Passonneau Megan Ung Adina Williams 280 2 0 26 Oct 2024
Self-calibration for Language Model Quantization and Pruning Miles Williams G. Chrysostomou Nikolaos Aletras MQ 457 0 0 22 Oct 2024
ActiveLLM: Large Language Model-based Active Learning for Textual Few-Shot Scenarios Markus Bayer Justin Lutz Christian A. Reuter 121 7 0 17 May 2024
Natural Language Processing RELIES on Linguistics Juri Opitz Shira Wein Nathan Schneider AI4CE 94 8 0 09 May 2024
Only Send What You Need: Learning to Communicate Efficiently in Federated Multilingual Machine Translation Yun-Wei Chu Dong-Jun Han Christopher G. Brinton 112 4 0 15 Jan 2024
The Forward-Forward Algorithm: Some Preliminary Investigations Geoffrey E. Hinton 110 275 0 27 Dec 2022
Bridging Fairness and Environmental Sustainability in Natural Language Processing Marius Hessenthaler Emma Strubell Dirk Hovy Anne Lauscher 68 8 0 08 Nov 2022
How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers Michael Hassid Hao Peng Daniel Rotem Jungo Kasai Ivan Montero Noah A. Smith Roy Schwartz 81 26 0 07 Nov 2022
Intriguing Properties of Compression on Multilingual Models Kelechi Ogueji Orevaoghene Ahia Gbemileke Onilude Sebastian Gehrmann Sara Hooker Julia Kreutzer 56 14 0 04 Nov 2022
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning Yaqing Wang Sahaj Agarwal Subhabrata Mukherjee Xiaodong Liu Jing Gao Ahmed Hassan Awadallah Jianfeng Gao MoE 91 133 0 31 Oct 2022
What Makes Convolutional Models Great on Long Sequence Modeling? Yuhong Li Tianle Cai Yi Zhang De-huai Chen Debadeepta Dey VLM 83 98 0 17 Oct 2022
DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation Mojtaba Valipour Mehdi Rezagholizadeh I. Kobyzev A. Ghodsi 88 182 0 14 Oct 2022
Mega: Moving Average Equipped Gated Attention Xuezhe Ma Chunting Zhou Xiang Kong Junxian He Liangke Gui Graham Neubig Jonathan May Luke Zettlemoyer 89 184 0 21 Sep 2022
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics Shoaib Ahmed Siddiqui Nitarshan Rajkumar Tegan Maharaj David M. Krueger Sara Hooker 105 28 0 20 Sep 2022
A Review of Sparse Expert Models in Deep Learning W. Fedus J. Dean Barret Zoph MoE 119 153 0 04 Sep 2022
FP8 Quantization: The Power of the Exponent Andrey Kuzmin M. V. Baalen Yuwei Ren Markus Nagel Jorn W. T. Peters Tijmen Blankevoort MQ 64 85 0 19 Aug 2022
Long Range Language Modeling via Gated State Spaces Harsh Mehta Ankit Gupta Ashok Cutkosky Behnam Neyshabur Mamba 101 242 0 27 Jun 2022
On the Parameterization and Initialization of Diagonal State Space Models Albert Gu Ankit Gupta Karan Goel Christopher Ré 86 324 0 23 Jun 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 286 2,507 0 15 Jun 2022
Modeling the Machine Learning Multiverse Samuel J. Bell Onno P. Kampman Jesse Dodge Neil D. Lawrence 68 18 0 13 Jun 2022
Measuring the Carbon Intensity of AI in Cloud Instances Jesse Dodge Taylor Prewitt Rémi Tachet des Combes Erika Odmark Roy Schwartz Emma Strubell A. Luccioni Noah A. Smith Nicole DeCario Will Buchanan 71 192 0 10 Jun 2022
8-bit Numerical Formats for Deep Neural Networks Badreddine Noune Philip Jones Daniel Justus Dominic Masters Carlo Luschi MQ 72 35 0 06 Jun 2022
Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts Basil Mustafa C. Riquelme J. Puigcerver Rodolphe Jenatton N. Houlsby VLM MoE 165 202 0 06 Jun 2022
Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future Jan-Christoph Klie Bonnie Webber Iryna Gurevych 82 46 0 05 Jun 2022
Extreme Compression for Pre-trained Transformers Made Simple and Efficient Xiaoxia Wu Z. Yao Minjia Zhang Conglong Li Yuxiong He MQ 36 31 0 04 Jun 2022
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Tri Dao Daniel Y. Fu Stefano Ermon Atri Rudra Christopher Ré VLM 237 2,272 0 27 May 2022
Chunk-based Nearest Neighbor Machine Translation Pedro Henrique Martins Zita Marinho André F.T. Martins RALM 124 28 0 24 May 2022
What Do Compressed Multilingual Machine Translation Models Forget? Alireza Mohammadshahi Vassilina Nikoulina Alexandre Berard Caroline Brun James Henderson Laurent Besacier AI4CE 71 9 0 22 May 2022
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning Haokun Liu Derek Tam Mohammed Muqeeth Jay Mohta Tenghao Huang Joey Tianyi Zhou Colin Raffel 104 921 0 11 May 2022
Towards Climate Awareness in NLP Research Daniel Hershcovich Nicolas Webersinke Mathias Kraus J. Bingler Markus Leippold 91 33 0 10 May 2022
Adaptable Adapters N. Moosavi Quentin Delfosse Kristian Kersting Iryna Gurevych 86 21 0 03 May 2022
OPT: Open Pre-trained Transformer Language Models Susan Zhang Stephen Roller Naman Goyal Mikel Artetxe Moya Chen ... Daniel Simig Punit Singh Koura Anjali Sridhar Tianlu Wang Luke Zettlemoyer VLM OSLM AI4CE 344 3,685 0 02 May 2022
Efficient Machine Translation Domain Adaptation Pedro Henrique Martins Zita Marinho André F.T. Martins 133 14 0 26 Apr 2022
Sources of Irreproducibility in Machine Learning: A Review Odd Erik Gundersen Kevin Coakley Christine R. Kirkpatrick Yolanda Gil SyDa 118 34 0 15 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 515 6,279 0 05 Apr 2022
PERFECT: Prompt-free and Efficient Few-shot Learning with Language Models Rabeeh Karimi Mahabadi Luke Zettlemoyer James Henderson Marzieh Saeidi Lambert Mathias Ves Stoyanov Majid Yazdani VLM 72 71 0 03 Apr 2022
Monarch: Expressive Structured Matrices for Efficient and Accurate Training Tri Dao Beidi Chen N. Sohoni Arjun D Desai Michael Poli Jessica Grogan Alexander Liu Aniruddh Rao Atri Rudra Christopher Ré 101 95 0 01 Apr 2022
Structured Pruning Learns Compact and Accurate Models Mengzhou Xia Zexuan Zhong Danqi Chen VLM 66 187 0 01 Apr 2022
Diagonal State Spaces are as Effective as Structured State Spaces Ankit Gupta Albert Gu Jonathan Berant 119 311 0 27 Mar 2022
Mokey: Enabling Narrow Fixed-Point Inference for Out-of-the-Box Floating-Point Transformer Models Ali Hadi Zadeh Mostafa Mahmoud Ameer Abdelhadi Andreas Moshovos MQ 62 33 0 23 Mar 2022
Pathways: Asynchronous Distributed Dataflow for ML P. Barham Aakanksha Chowdhery J. Dean Sanjay Ghemawat Steven Hand ... Parker Schuh Ryan Sepassi Laurent El Shafey C. A. Thekkath Yonghui Wu GNN MoE 115 131 0 23 Mar 2022
How does the pre-training objective affect what large language models learn about linguistic properties? Ahmed Alajrami Nikolaos Aletras 62 20 0 20 Mar 2022
Combining Modular Skills in Multitask Learning Edoardo Ponti Alessandro Sordoni Yoshua Bengio Siva Reddy MoE 60 38 0 28 Feb 2022
Should You Mask 15% in Masked Language Modeling? Alexander Wettig Tianyu Gao Zexuan Zhong Danqi Chen CVBM 67 166 0 16 Feb 2022
EdgeFormer: A Parameter-Efficient Transformer for On-Device Seq2seq Generation Tao Ge Si-Qing Chen Furu Wei MoE 79 23 0 16 Feb 2022
A Survey on Dynamic Neural Networks for Natural Language Processing Canwen Xu Julian McAuley AI4CE 88 29 0 15 Feb 2022
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts Stephen H. Bach Victor Sanh Zheng-Xin Yong Albert Webson Colin Raffel ... Khalid Almubarak Xiangru Tang Dragomir R. Radev Mike Tian-Jian Jiang Alexander M. Rush VLM 322 349 0 02 Feb 2022
A Survey on Retrieval-Augmented Text Generation Huayang Li Yixuan Su Deng Cai Yan Wang Lemao Liu RALM 116 209 0 02 Feb 2022
Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval Uri Alon Frank F. Xu Junxian He Sudipta Sengupta Dan Roth Graham Neubig RALM 116 63 0 28 Jan 2022
Can Model Compression Improve NLP Fairness Guangxuan Xu Qingyuan Hu 52 28 0 21 Jan 2022