Distributed Inference and Fine-tuning of Large Language Models Over The Internet

13 December 2023

Tim Dettmers

Papers citing "Distributed Inference and Fine-tuning of Large Language Models Over The Internet"

28 / 28 papers shown

Title
Taming the Titans: A Survey of Efficient LLM Inference Serving Ranran Zhen J. Li Yixin Ji Zhengyuan Yang Tong Liu Qingrong Xia Xinyu Duan Zehao Wang Baoxing Huai Hao Fei LLMAG 77 0 0 28 Apr 2025
Prompt Inference Attack on Distributed Large Language Model Inference Frameworks Xinjian Luo Ting Yu X. Xiao AAML SILM 88 1 0 12 Mar 2025
Prompt Inversion Attack against Collaborative Inference of Large Language Models Wenjie Qu Yuguang Zhou Yongji Wu Tingsong Xiao Binhang Yuan Heng Chang Jiaheng Zhang 76 0 0 12 Mar 2025
Can Generative AI be Egalitarian? Philip G. Feldman James R. Foulds Shimei Pan 65 0 0 20 Jan 2025
Deploying Foundation Model Powered Agent Services: A Survey Wenchao Xu Jinyu Chen Peirong Zheng Xiaoquan Yi Tianyi Tian ... Quan Wan Yining Qi Yunfeng Fan Qinliang Su Xuemin Shen AI4CE 119 1 0 18 Dec 2024
Photon: Federated LLM Pre-Training Lorenzo Sani Alex Iacob Zeyu Cao Royson Lee Bill Marino ... Dongqi Cai Zexi Li Wanru Zhao Xinchi Qiu Nicholas D. Lane AI4CE 36 7 0 05 Nov 2024
Meta-Learning for Speeding Up Large Model Inference in Decentralized Environments Yuzhe Yang Yipeng Du Ahmad Farhan Claudio Angione Yue Zhao Harry Yang Fielding Johnston James Buban Patrick Colangelo 29 0 0 28 Oct 2024
TPI-LLM: Serving 70B-scale LLMs Efficiently on Low-resource Edge Devices Zonghang Li Wenjiao Feng Mohsen Guizani Hongfang Yu 50 3 0 01 Oct 2024
AnyMatch -- Efficient Zero-Shot Entity Matching with a Small Language Model Zeyu Zhang Paul Groth Iacer Calixto Sebastian Schelter 41 1 0 06 Sep 2024
Decentralized LLM Inference over Edge Networks with Energy Harvesting A. Khoshsirat Giovanni Perin Michele Rossi 18 2 0 28 Aug 2024
Recognizing Emotion Regulation Strategies from Human Behavior with Large Language Models Philipp Müller Alexander Heimerl Sayed Muddashir Hossain Lea Siegel Jan Alexandersson Patrick Gebhard Elisabeth André T. Schneeberger 34 0 0 08 Aug 2024
LocalValueBench: A Collaboratively Built and Extensible Benchmark for Evaluating Localized Value Alignment and Ethical Safety in Large Language Models Achintya Gopal Nicholas Wai Long Lau Eva Adelina Susanto Chi Lok Yu Aditya Paul ELM 25 7 0 27 Jul 2024
Mobile Edge Intelligence for Large Language Models: A Contemporary Survey Guanqiao Qu Qiyuan Chen Wei Wei Zheng Lin Xianhao Chen Kaibin Huang 42 43 0 09 Jul 2024
Thinking Forward: Memory-Efficient Federated Finetuning of Language Models Kunjal Panchal Nisarg Parikh Sunav Choudhary Lijun Zhang Yuriy Brun Hui Guan 61 3 0 24 May 2024
VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks Yang Li Shaobo Han Shihao Ji MoE 27 11 0 24 May 2024
PerLLM: Personalized Inference Scheduling with Edge-Cloud Collaboration for Diverse LLM Services Zheming Yang Yuanhao Yang Chang Zhao Qi Guo Wenkai He Wen Ji 50 13 0 23 May 2024
The Future of Large Language Model Pre-training is Federated Lorenzo Sani Alexandru Iacob Zeyu Cao Bill Marino Yan Gao ... Wanru Zhao William F. Shen Preslav Aleksandrov Xinchi Qiu Nicholas D. Lane AI4CE 35 13 0 17 May 2024
$$\texttt{MiniMol}$: A Parameter-Efficient Foundation Model for Molecular Learning$ $\texttt{MiniMol}$ : A Parameter-Efficient Foundation Model for Molecular Learning Kerstin Klaser Bla.zej Banaszewski S. Maddrell-Mander Callum McLean Luis Muller Alipanah Parviz Shenyang Huang Andrew Fitzgibbon AI4CE 48 5 0 23 Apr 2024
HLAT: High-quality Large Language Model Pre-trained on AWS Trainium Haozheng Fan Hao Zhou Guangtai Huang Parameswaran Raman Xinwei Fu Gaurav Gupta Dhananjay Ram Yida Wang Jun Huan 48 5 0 16 Apr 2024
Is Your AI Truly Yours? Leveraging Blockchain for Copyrights, Provenance, and Lineage Yilin Sai Qin Wang Guangsheng Yu H.M.N. Dilum Bandara Shiping Chen 46 1 0 09 Apr 2024
Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems Xupeng Miao Gabriele Oliaro Zhihao Zhang Xinhao Cheng Hongyi Jin Tianqi Chen Zhihao Jia 67 76 0 23 Dec 2023
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes Zhen Qin Daoyuan Chen Bingchen Qian Bolin Ding Yaliang Li Shuiguang Deng FedML 40 32 0 11 Dec 2023
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 253 1,073 0 05 Oct 2022
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 238 808 0 14 Oct 2021
What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers Boseop Kim Hyoungseok Kim Sang-Woo Lee Gichang Lee Donghyun Kwak ... Jaewook Kang Inho Kang Jung-Woo Ha W. Park Nako Sung VLM 249 121 0 10 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,858 0 18 Apr 2021
ZeRO-Offload: Democratizing Billion-Scale Model Training Jie Ren Samyam Rajbhandari Reza Yazdani Aminabadi Olatunji Ruwase Shuangyang Yang Minjia Zhang Dong Li Yuxiong He MoE 177 416 0 18 Jan 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 264 4,489 0 23 Jan 2020