v1v2 (latest)

Mobile Edge Intelligence for Large Language Models: A Contemporary Survey

9 July 2024

Guanqiao Qu

Qiyuan Chen

Wei Wei

Zheng Lin

Xianhao Chen

Kaibin Huang

ArXiv (abs)PDF HTML

Papers citing "Mobile Edge Intelligence for Large Language Models: A Contemporary Survey"

50 / 211 papers shown

Title
Token Merging: Your ViT But Faster Daniel Bolya Cheng-Yang Fu Xiaoliang Dai Peizhao Zhang Christoph Feichtenhofer Judy Hoffman MoMe 132 474 0 17 Oct 2022
In-situ Model Downloading to Realize Versatile Edge AI in 6G Mobile Networks Kaibin Huang Hai Wu Zhiyan Liu Xiaojuan Qi 65 10 0 07 Oct 2022
A Review of Sparse Expert Models in Deep Learning W. Fedus J. Dean Barret Zoph MoE 129 154 0 04 Sep 2022
Confident Adaptive Language Modeling Tal Schuster Adam Fisch Jai Gupta Mostafa Dehghani Dara Bahri Vinh Q. Tran Yi Tay Donald Metzler 153 170 0 14 Jul 2022
OPT: Open Pre-trained Transformer Language Models Susan Zhang Stephen Roller Naman Goyal Mikel Artetxe Moya Chen ... Daniel Simig Punit Singh Koura Anjali Sridhar Tianlu Wang Luke Zettlemoyer VLM OSLM AI4CE 394 3,707 0 02 May 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 572 6,320 0 05 Apr 2022
Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizations Youwei Liang Chongjian Ge Zhan Tong Yibing Song Jue Wang P. Xie ViT 81 255 0 16 Feb 2022
Block-Skim: Efficient Question Answering for Transformer Yue Guan Zhengyi Li Jingwen Leng Zhouhan Lin Minyi Guo Yuhao Zhu 87 32 0 16 Dec 2021
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 301 863 0 14 Oct 2021
Secure Aerial Surveillance using Split Learning Yoo Jeong Ha M. Yoo Soohyun Park Soyi Jung Joongheon Kim 38 9 0 19 Jul 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 699 10,631 0 17 Jun 2021
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units Wei-Ning Hsu Benjamin Bolte Yao-Hung Hubert Tsai Kushal Lakhotia Ruslan Salakhutdinov Abdel-rahman Mohamed SSL 190 3,013 0 14 Jun 2021
Accelerating BERT Inference for Sequence Labeling via Early-Exit Xiaonan Li Yunfan Shao Tianxiang Sun Hang Yan Xipeng Qiu Xuanjing Huang 84 41 0 28 May 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 345 688 0 21 Apr 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 694 4,119 0 18 Apr 2021
ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning Samyam Rajbhandari Olatunji Ruwase Jeff Rasley Shaden Smith Yuxiong He GNN 101 393 0 16 Apr 2021
Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM Deepak Narayanan Mohammad Shoeybi Jared Casper P. LeGresley M. Patwary ... Prethvi Kashinkunti J. Bernauer Bryan Catanzaro Amar Phanishayee Matei A. Zaharia MoE 197 712 0 09 Apr 2021
Attention, please! A survey of Neural Attention Models in Deep Learning Alana de Santana Correia Esther Luna Colombini HAI 128 197 0 31 Mar 2021
Hierarchical Federated Learning with Quantization: Convergence Analysis and System Design Lumin Liu Jun Zhang Shenghui Song Khaled B. Letaief FedML 70 87 0 26 Mar 2021
Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges Yoshitomo Matsubara Marco Levorato Francesco Restuccia 120 215 0 08 Mar 2021
Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach Jiawei Shao Yuyi Mao Jun Zhang 100 222 0 08 Feb 2021
ZeRO-Offload: Democratizing Billion-Scale Model Training Jie Ren Samyam Rajbhandari Reza Yazdani Aminabadi Olatunji Ruwase Shuangyang Yang Minjia Zhang Dong Li Yuxiong He MoE 289 434 0 18 Jan 2021
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity W. Fedus Barret Zoph Noam M. Shazeer MoE 133 2,247 0 11 Jan 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 254 4,330 0 01 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 402 6,848 0 23 Dec 2020
Parameter-Efficient Transfer Learning with Diff Pruning Demi Guo Alexander M. Rush Yoon Kim 92 406 0 14 Dec 2020
Advancements of federated learning towards privacy preservation: from federated learning to split learning Chandra Thapa Pathum Chamikara Mahawaga Arachchige S. Çamtepe FedML 75 87 0 25 Nov 2020
mT5: A massively multilingual pre-trained text-to-text transformer Linting Xue Noah Constant Adam Roberts Mihir Kale Rami Al-Rfou Aditya Siddhant Aditya Barua Colin Raffel 182 2,569 0 22 Oct 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 750 41,796 0 22 Oct 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Zhiwen Chen MoE 180 1,198 0 30 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 1.1K 42,651 0 28 May 2020
End-to-End Object Detection with Transformers Nicolas Carion Francisco Massa Gabriel Synnaeve Nicolas Usunier Alexander Kirillov Sergey Zagoruyko ViT 3DV PINN 500 13,213 0 26 May 2020
The Cascade Transformer: an Application for Efficient Answer Sentence Selection Luca Soldaini Alessandro Moschitti 90 44 0 05 May 2020
Automatic Cross-Replica Sharding of Weight Update in Data-Parallel Training Yuanzhong Xu HyoukJoong Lee Dehao Chen Hongjun Choi Blake A. Hechtman Shibo Wang 82 42 0 28 Apr 2020
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference Ji Xin Raphael Tang Jaejun Lee Yaoliang Yu Jimmy J. Lin 65 377 0 27 Apr 2020
SplitFed: When Federated Learning Meets Split Learning Chandra Thapa Pathum Chamikara Mahawaga Arachchige S. Çamtepe Lichao Sun FedML 107 596 0 25 Apr 2020
ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data Di Qi Lin Su Jianwei Song Edward Cui Taroon Bharti Arun Sacheti VLM 116 263 0 22 Jan 2020
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 268 10,913 0 29 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 739 20,418 0 23 Oct 2019
ZeRO: Memory Optimizations Toward Training Trillion Parameter Models Samyam Rajbhandari Jeff Rasley Olatunji Ruwase Yuxiong He ALM AI4CE 90 923 0 04 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 534 6,482 0 26 Sep 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Mohammad Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 363 1,926 0 17 Sep 2019
Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence Shuiguang Deng Hailiang Zhao Weijia Fang Yuxiang Cai Schahram Dustdar Albert Y. Zomaya 106 618 0 02 Sep 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 829 24,615 0 26 Jul 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 302 8,462 0 19 Jun 2019
OpenEI: An Open Framework for Edge Intelligence Xingzhou Zhang Yifan Wang Sidi Lu Liangkai Liu Lanyu Xu Weisong Shi 79 101 0 05 Jun 2019
Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing Zhi Zhou Xu Chen En Li Liekang Zeng Ke Luo Junshan Zhang 114 1,450 0 24 May 2019
Client-Edge-Cloud Hierarchical Federated Learning Lumin Liu Jun Zhang S. H. Song Khaled B. Letaief FedML 93 758 0 16 May 2019
Cross-Modal Self-Attention Network for Referring Image Segmentation Linwei Ye Mrigank Rochan Zhi Liu Yang Wang EgoV 73 478 0 09 Apr 2019
An Attentive Survey of Attention Models S. Chaudhari Varun Mithal Gungor Polatkan R. Ramanath 194 666 0 05 Apr 2019