Title
Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding Yu Zhang Hao Cheng Zhihong Shen Xiaodong Liu Yejiang Wang Jianfeng Gao 32 14 0 23 May 2023
Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model Leo Liu Tim Dettmers Xi Lin Ves Stoyanov Xian Li MoE 26 9 0 23 May 2023
Lifting the Curse of Capacity Gap in Distilling Language Models Chen Zhang Yang Yang Jiahao Liu Jingang Wang Yunsen Xian Benyou Wang Dawei Song MoE 32 19 0 20 May 2023
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Xiaowei Huang Wenjie Ruan Wei Huang Gao Jin Yizhen Dong ... Sihao Wu Peipei Xu Dengyu Wu André Freitas Mustafa A. Mustafa ALM 45 83 0 19 May 2023
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining Sang Michael Xie Hieu H. Pham Xuanyi Dong Nan Du Hanxiao Liu Yifeng Lu Percy Liang Quoc V. Le Tengyu Ma Adams Wei Yu MoMe MoE 56 178 0 17 May 2023
PaLM 2 Technical Report Rohan Anil Andrew M. Dai Orhan Firat Melvin Johnson Dmitry Lepikhin ... Ce Zheng Wei Zhou Denny Zhou Slav Petrov Yonghui Wu ReLM LRM 128 1,152 0 17 May 2023
SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification Xupeng Miao Gabriele Oliaro Zhihao Zhang Xinhao Cheng Zeyu Wang ... Chunan Shi Zhuoming Chen Daiyaan Arfeen Reyna Abhyankar Zhihao Jia LRM 65 120 0 16 May 2023
Towards Being Parameter-Efficient: A Stratified Sparsely Activated Transformer with Dynamic Capacity Da Xu Maha Elbayad Kenton W. Murray Jean Maillard Vedanuj Goswami MoE 47 3 0 03 May 2023
Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference Tao Lei Junwen Bai Siddhartha Brahma Joshua Ainslie Kenton Lee ... Vincent Zhao Yuexin Wu Bo-wen Li Yu Zhang Ming-Wei Chang BDL AI4CE 30 55 0 11 Apr 2023
Language Models can Solve Computer Tasks Geunwoo Kim Pierre Baldi Stephen Marcus McAleer LLMAG LM&Ro 43 342 0 30 Mar 2023
MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks Weicheng Kuo A. Piergiovanni Dahun Kim Xiyang Luo Benjamin Caine ... Luowei Zhou Andrew M. Dai Zhifeng Chen Claire Cui A. Angelova MLLM VLM 34 23 0 29 Mar 2023
On the Creativity of Large Language Models Giorgio Franceschelli Mirco Musolesi 72 54 0 27 Mar 2023
Personalizing Task-oriented Dialog Systems via Zero-shot Generalizable Reward Function A. B. Siddique M. H. Maqbool Kshitija Taywade H. Foroosh 24 12 0 24 Mar 2023
The Contribution of Knowledge in Visiolinguistic Learning: A Survey on Tasks and Challenges Maria Lymperaiou Giorgos Stamou VLM 32 4 0 04 Mar 2023
Full Stack Optimization of Transformer Inference: a Survey Sehoon Kim Coleman Hooper Thanakul Wattanawong Minwoo Kang Ruohan Yan ... Qijing Huang Kurt Keutzer Michael W. Mahoney Y. Shao A. Gholami MQ 36 101 0 27 Feb 2023
Modular Deep Learning Jonas Pfeiffer Sebastian Ruder Ivan Vulić Edoardo Ponti MoMe OOD 32 73 0 22 Feb 2023
Massively Multilingual Shallow Fusion with Large Language Models Ke Hu Tara N. Sainath Bo-wen Li Nan Du Yanping Huang Andrew M. Dai Yu Zhang Rodrigo Cabrera Z. Chen Trevor Strohman 35 13 0 17 Feb 2023
Auto-Parallelizing Large Models with Rhino: A Systematic Approach on Production AI Platform Shiwei Zhang Lansong Diao Siyu Wang Zongyan Cao Yiliang Gu Chang Si Ziji Shi Zhen Zheng Chuan Wu W. Lin AI4CE 32 4 0 16 Feb 2023
Symbolic Discovery of Optimization Algorithms Xiangning Chen Chen Liang Da Huang Esteban Real Kaiyuan Wang ... Xuanyi Dong Thang Luong Cho-Jui Hsieh Yifeng Lu Quoc V. Le 67 353 0 13 Feb 2023
Multipath agents for modular multitask ML systems Andrea Gesmundo 28 1 0 06 Feb 2023
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers K. Choromanski Shanda Li Valerii Likhosherstov Kumar Avinava Dubey Shengjie Luo Di He Yiming Yang Tamás Sarlós Thomas Weingarten Adrian Weller 37 8 0 03 Feb 2023
A Survey of Methods, Challenges and Perspectives in Causality Gaël Gendron Michael Witbrock Gillian Dobbie OOD AI4CE CML 39 13 0 01 Feb 2023
Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation Fei Yuan Yinquan Lu Wenhao Zhu Lingpeng Kong Lei Li Yu Qiao Jingjing Xu MoE 38 22 0 20 Dec 2022
Go-tuning: Improving Zero-shot Learning Abilities of Smaller Language Models Jingjing Xu Qingxiu Dong Hongyi Liu Lei Li ALM LRM 33 1 0 20 Dec 2022
Memory-efficient NLLB-200: Language-specific Expert Pruning of a Massively Multilingual Machine Translation Model Yeskendir Koishekenov Alexandre Berard Vassilina Nikoulina MoE 35 29 0 19 Dec 2022
Fixing MoE Over-Fitting on Low-Resource Languages in Multilingual Machine Translation Maha Elbayad Anna Y. Sun Shruti Bhosale MoE 54 9 0 15 Dec 2022
DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing Conglong Li Z. Yao Xiaoxia Wu Minjia Zhang Connor Holmes Cheng Li Yuxiong He 27 24 0 07 Dec 2022
SPARTAN: Sparse Hierarchical Memory for Parameter-Efficient Transformers Ameet Deshpande Md Arafat Sultan Anthony Ferritto Ashwin Kalyan Karthik R. Narasimhan Avirup Sil MoE 43 1 0 29 Nov 2022
MegaBlocks: Efficient Sparse Training with Mixture-of-Experts Trevor Gale Deepak Narayanan C. Young Matei A. Zaharia MoE 25 103 0 29 Nov 2022
Spatial Mixture-of-Experts Nikoli Dryden Torsten Hoefler MoE 36 9 0 24 Nov 2022
Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks Wenhu Chen Xueguang Ma Xinyi Wang William W. Cohen ReLM ReCod LRM 101 741 0 22 Nov 2022
Coreference Resolution through a seq2seq Transition-Based System Bernd Bohnet Chris Alberti Michael Collins 28 39 0 22 Nov 2022
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models Guangxuan Xiao Ji Lin Mickael Seznec Hao Wu Julien Demouth Song Han MQ 74 741 0 18 Nov 2022
HMOE: Hypernetwork-based Mixture of Experts for Domain Generalization Jingang Qu T. Faney Zehao Wang Patrick Gallinari Soleiman Yousef J. D. Hemptinne OOD 24 7 0 15 Nov 2022
A Universal Discriminator for Zero-Shot Generalization Haike Xu Zongyu Lin Jing Zhou Yanan Zheng Zhilin Yang AI4CE 21 14 0 15 Nov 2022
Piloting Copilot and Codex: Hot Temperature, Cold Prompts, or Black Magic? Jean-Baptiste Döderlein M. Acher D. Khelladi B. Combemale 34 33 0 26 Oct 2022
Will we run out of data? Limits of LLM scaling based on human-generated data Pablo Villalobos A. Ho J. Sevilla T. Besiroglu Lennart Heim Marius Hobbhahn ALM 44 111 0 26 Oct 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 100 2,999 0 20 Oct 2022
On the Adversarial Robustness of Mixture of Experts J. Puigcerver Rodolphe Jenatton C. Riquelme Pranjal Awasthi Srinadh Bhojanapalli OOD AAML MoE 45 18 0 19 Oct 2022
Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective Ping Yang Junjie Wang Ruyi Gan Xinyu Zhu Lin Zhang Ziwei Wu Xinyu Gao Jiaxing Zhang Tetsuya Sakai BDL 22 25 0 16 Oct 2022
Spontaneous Emerging Preference in Two-tower Language Model Zhengqi He Taro Toyoizumi LRM 21 1 0 13 Oct 2022
Large Language Models are few(1)-shot Table Reasoners Wenhu Chen LMTD ReLM LRM 22 138 0 13 Oct 2022
Mind's Eye: Grounded Language Model Reasoning through Simulation Ruibo Liu Jason W. Wei S. Gu Te-Yen Wu Soroush Vosoughi Claire Cui Denny Zhou Andrew M. Dai ReLM LRM 118 79 0 11 Oct 2022
Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context Experts Nghia T. Le Fan Bai Alan Ritter 37 12 0 07 Oct 2022
Generate rather than Retrieve: Large Language Models are Strong Context Generators Wenhao Yu Dan Iter Shuohang Wang Yichong Xu Mingxuan Ju Soumya Sanyal Chenguang Zhu Michael Zeng Meng Jiang RALM AIMat 237 322 0 21 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 33 109 0 31 Aug 2022
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model Saleh Soltan Shankar Ananthakrishnan Jack G. M. FitzGerald Rahul Gupta Wael Hamza ... Mukund Sridhar Fabian Triefenbach Apurv Verma Gokhan Tur Premkumar Natarajan 56 82 0 02 Aug 2022
Scaling Autoregressive Models for Content-Rich Text-to-Image Generation Jiahui Yu Yuanzhong Xu Jing Yu Koh Thang Luong Gunjan Baid ... Zarana Parekh Xin Li Han Zhang Jason Baldridge Yonghui Wu EGVM 125 1,066 0 22 Jun 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 84 2,354 0 15 Jun 2022
Language Models are General-Purpose Interfaces Y. Hao Haoyu Song Li Dong Shaohan Huang Zewen Chi Wenhui Wang Shuming Ma Furu Wei MLLM 30 96 0 13 Jun 2022