Small Language Models: Survey, Measurements, and Insights

24 September 2024

Zhenyan Lu

Xiang Li

Dongqi Cai

Rongjie Yi

Fangming Liu

Xiwen Zhang

Nicholas D. Lane

Mengwei Xu

ObjD

LRM

ArXiv PDF HTML

Papers citing "Small Language Models: Survey, Measurements, and Insights"

26 / 26 papers shown

Title
Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inference Haolin Zhang Jeff Huang 35 0 0 09 May 2025
LSRP: A Leader-Subordinate Retrieval Framework for Privacy-Preserving Cloud-Device Collaboration Yuhang Zhang Pengyue Jia Xin Li Derong Xu Maolin Wang ... Zhaocheng Du Huifeng Guo Y. Liu Ruiming Tang Xiangyu Zhao 44 0 0 08 May 2025
RobotxR1: Enabling Embodied Robotic Intelligence on Large Language Models through Closed-Loop Reinforcement Learning Liam Boyle Nicolas Baumann Paviththiren Sivasothilingam Michele Magno Luca Benini LM&Ro LRM 51 0 0 06 May 2025
Position: Enough of Scaling LLMs! Lets Focus on Downscaling Ayan Sengupta Yash Goel Tanmoy Chakraborty 34 0 0 02 May 2025
The Rise of Small Language Models in Healthcare: A Comprehensive Survey Muskan Garg Shaina Raza Shebuti Rayana Xingyi Liu Sunghwan Sohn LM&MA AILaw 92 0 0 23 Apr 2025
Enhancing Autonomous Driving Systems with On-Board Deployed Large Language Models Nicolas Baumann Cheng Hu Paviththiren Sivasothilingam Haotong Qin Lei Xie Michele Magno Luca Benini 27 1 0 15 Apr 2025
Cat, Rat, Meow: On the Alignment of Language Model and Human Term-Similarity Judgments Lorenz Linhardt Tom Neuhäuser Lenka Tětková Oliver Eberle ALM AI4TS 42 0 0 10 Apr 2025
T1: Tool-integrated Self-verification for Test-time Compute Scaling in Small Language Models Minki Kang Jongwon Jeong Jaewoong Cho ALM LRM 46 2 0 07 Apr 2025
Does Chain-of-Thought Reasoning Help Mobile GUI Agent? An Empirical Study Li Lyna Zhang Longxi Gao Mengwei Xu LRM 37 1 0 21 Mar 2025
PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing Cheng Deng Luoyang Sun Jiwen Jiang Yongcheng Zeng Xinjian Wu ... Haoyang Li Lei Chen Lionel M. Ni Hongzhi Zhang Jun Wang 171 0 0 15 Mar 2025
Evaluating Mathematical Reasoning Across Large Language Models: A Fine-Grained Approach Afrar Jahin Arif Hassan Zidan Yu Bao Shizhe Liang T. Liu LRM 73 1 0 13 Mar 2025
Cost-Optimal Grouped-Query Attention for Long-Context Modeling Y. Chen Yutong Wu Chenyang Song Zhiyuan Liu Maosong Sun Xu Han Zhiyuan Liu Maosong Sun 61 0 0 12 Mar 2025
Can Small Language Models Reliably Resist Jailbreak Attacks? A Comprehensive Evaluation Wenhui Zhang Huiyu Xu Zhibo Wang Zeqing He Ziqi Zhu Kui Ren AAML PILM 72 0 0 09 Mar 2025
GenieBlue: Integrating both Linguistic and Multimodal Capabilities for Large Language Models on Mobile Devices Xudong Lu Yinghao Chen Renshou Wu Haohao Gao Xi Chen ... Fangyuan Li Yafei Wen Xiaoxin Chen Shuai Ren Hongsheng Li 75 0 0 08 Mar 2025
GPIoT: Tailoring Small Language Models for IoT Program Synthesis and Development Leming Shen Qiang Yang Xinyu Huang Zijing Ma Yuanqing Zheng 27 1 0 02 Mar 2025
From Small to Large Language Models: Revisiting the Federalist Papers So Won Jeong Veronika Rockova 39 0 0 25 Feb 2025
Towards Reasoning Ability of Small Language Models Gaurav Srivastava Shuxiang Cao Xuan Wang ReLM LRM 60 4 0 17 Feb 2025
A Survey of Personalized Large Language Models: Progress and Future Directions Jiahong Liu Zexuan Qiu Zhongyang Li Quanyu Dai Jieming Zhu Minda Hu Menglin Yang Irwin King LM&MA 54 2 0 17 Feb 2025
Division-of-Thoughts: Harnessing Hybrid Language Model Synergy for Efficient On-Device Agents Chenyang Shao Xinyuan Hu Yutang Lin Fengli Xu LLMAG LRM 67 4 0 06 Feb 2025
Evaluating Small Language Models for News Summarization: Implications and Factors Influencing Performance Borui Xu Yao Chen Zeyi Wen Weiguo Liu Bingsheng He 77 1 0 02 Feb 2025
Language and Planning in Robotic Navigation: A Multilingual Evaluation of State-of-the-Art Models Malak Mansour Ahmed Aly Bahey Tharwat Sarim Hashmi Dong An Ian Reid LM&Ro ELM LRM 56 0 0 07 Jan 2025
Fineweb-Edu-Ar: Machine-translated Corpus to Support Arabic Small Language Models Sultan Alrashed Dmitrii Khizbullin David R. Pugh 36 0 0 10 Nov 2024
MiniPLM: Knowledge Distillation for Pre-Training Language Models Yuxian Gu Hao Zhou Fandong Meng Jie Zhou Minlie Huang 70 5 0 22 Oct 2024
Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices Yuxiang Huang Binhang Yuan Xu Han Chaojun Xiao Zhiyuan Liu RALM 81 1 0 02 Oct 2024
What is the Role of Small Models in the LLM Era: A Survey Lihu Chen Gaël Varoquaux ALM 63 23 0 10 Sep 2024
Requirements are All You Need: The Final Frontier for End-User Software Engineering Diana Robinson Christian Cabrera Andrew D. Gordon Neil D. Lawrence Lars Mennen 37 4 0 22 May 2024