v1v2 (latest)

TinyLlama: An Open-Source Small Language Model

4 January 2024

Wei Lu

ArXiv (abs)PDF HTML Github (8509★)

Papers citing "TinyLlama: An Open-Source Small Language Model"

50 / 287 papers shown

Title
EvoLM: In Search of Lost Language Model Training Dynamics Zhenting Qi Fan Nie Alexandre Alahi James Zou Himabindu Lakkaraju Yilun Du Eric P. Xing Sham Kakade Hanlin Zhang 49 1 0 19 Jun 2025
Modeling the One-to-Many Property in Open-Domain Dialogue with LLMs Jing Yang Lee Kong-Aik Lee Woon-Seng Gan 40 0 0 18 Jun 2025
S $^4$ C: Speculative Sampling with Syntactic and Semantic Coherence for Efficient Inference of Large Language Models Tao He Guang Huang Yu Yang Tianshi Xu Sicheng Zhao Guiguang Ding Pengyang Wang Feng Tian LRM 30 0 0 17 Jun 2025
Mixture of Weight-shared Heterogeneous Group Attention Experts for Dynamic Token-wise KV Optimization Guanghui Song Dongping Liao Yiren Zhao Kejiang Ye Cheng-zhong Xu X. Gao MoE 21 0 0 16 Jun 2025
Attribution-guided Pruning for Compression, Circuit Discovery, and Targeted Correction in LLMs Sayed Mohammad Vakilzadeh Hatefi Maximilian Dreyer Reduan Achtibat Patrick Kahardipraja Thomas Wiegand Wojciech Samek Sebastian Lapuschkin 31 0 0 16 Jun 2025
Just Go Parallel: Improving the Multilingual Capabilities of Large Language Models Muhammad Reza Qorib Junyi Li Hwee Tou Ng LRM 25 0 0 16 Jun 2025
Bridging the Digital Divide: Small Language Models as a Pathway for Physics and Photonics Education in Underdeveloped Regions Asghar Ghorbani Hanieh Fattahi 34 0 0 14 Jun 2025
DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts Yuchen Feng Bowen Shen Naibin Gu Jiaxuan Zhao Peng Fu Zheng Lin Weiping Wang MoMe MoE 57 0 0 11 Jun 2025
FLoRIST: Singular Value Thresholding for Efficient and Accurate Federated Fine-Tuning of Large Language Models Hariharan Ramesh Jyotikrishna Dass 33 0 0 10 Jun 2025
Basis Transformers for Multi-Task Tabular Regression Wei Min Loh Jiaqi Shang Pascal Poupart LMTD 20 0 0 07 Jun 2025
Distillation Robustifies Unlearning Bruce W. Lee Addie Foote Alex Infanger Leni Shor Harish Kamath Jacob Goldman-Wetzler Bryce Woodworth Alex Cloud Alexander Matt Turner MU 75 0 0 06 Jun 2025
TokAlign: Efficient Vocabulary Adaptation via Token Alignment Chong Li Jiajun Zhang Chengqing Zong VLM 61 0 0 04 Jun 2025
Revisiting LRP: Positional Attribution as the Missing Ingredient for Transformer Explainability Yarden Bakish Itamar Zimerman Hila Chefer Lior Wolf 24 0 0 02 Jun 2025
STORM-BORN: A Challenging Mathematical Derivations Dataset Curated via a Human-in-the-Loop Multi-Agent Framework Wenhao Liu Zhenyi Lu Xinyu Hu Jierui Zhang Dailin Li ... Pei Zhang Chengbo Zhang Yuxiang Ren Xiaohong Huang Yan Ma OffRL 96 1 0 02 Jun 2025
One for All: Update Parameterized Knowledge Across Multiple Models Weitao Ma Xiyuan Du Xiaocheng Feng L. Huang Yichong Huang ... Xiaoliang Yang Baohang Li Xiachong Feng Ting Liu Bing Qin KELM 63 0 0 01 Jun 2025
ConsRec: Denoising Sequential Recommendation through User-Consistent Preference Modeling Haidong Xin Qiushi Xiong Zhenghao Liu Sen Mei Yukun Yan Shi Yu Shuo Wang Yu Gu Ge Yu Chenyan Xiong HAI 55 0 0 28 May 2025
From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications Feibo Jiang Cunhua Pan Li Dong Kezhi Wang O. Dobre Mérouane Debbah LLMAG AI4TS 175 1 0 28 May 2025
Dissecting Physics Reasoning in Small Language Models: A Multi-Dimensional Analysis from an Educational Perspective Nicy Scaria Silvester John Joseph Kennedy Diksha Seth Deepak N. Subramani LRM 50 0 0 27 May 2025
Pretraining Language Models to Ponder in Continuous Space Boyi Zeng Shixiang Song Siyuan Huang Yixuan Wang He Li Ziwei He Xinbing Wang Zhiyu Li Zhouhan Lin LRM 98 0 0 27 May 2025
Small Language Models: Architectures, Techniques, Evaluation, Problems and Future Adaptation Tanjil Hasan Sakib Md. Tanzib Hosain Md. Kishor Morol ALM 45 0 0 26 May 2025
Efficient and Direct Duplex Modeling for Speech-to-Speech Language Model Ke Hu Ehsan Hosseini-Asl Chen Chen Edresson Casanova Subhankar Ghosh Piotr .Zelasko Zhiwen Chen Jia-Nan Li Jagadeesh Balam Boris Ginsburg AuLLM 134 0 0 21 May 2025
Effective and Efficient Schema-aware Information Extraction Using On-Device Large Language Models Zhihao Wen Sheng Liang Yaxiong Wu Yongyue Zhang Yang Liu 53 0 0 21 May 2025
Communication-Efficient Hybrid Language Model via Uncertainty-Aware Opportunistic and Compressed Transmission Seungeun Oh Jinhyuk Kim Jihong Park Seung-Woo Ko Jinho Choi Tony Q. S. Quek Seong-Lyun Kim 67 0 0 17 May 2025
Chain-of-Model Learning for Language Model Kaitao Song Xiaohua Wang Xu Tan Huiqiang Jiang Chengruidong Zhang ... Xiaoqing Zheng Tao Qin Yuqing Yang Dongsheng Li Lili Qiu LRM AI4CE 193 1 0 17 May 2025
The Ripple Effect: On Unforeseen Complications of Backdoor Attacks Rui Zhang Yun Shen Hongwei Li Wenbo Jiang Hanxiao Chen Yuan Zhang Guowen Xu Yang Zhang SILM AAML 85 0 0 16 May 2025
SpecEdge: Scalable Edge-Assisted Serving Framework for Interactive LLMs Jinwoo Park Seunggeun Cho Dongsu Han 84 0 0 16 May 2025
Model-Distributed Inference for Large Language Models at the Edge Davide Macario H. Seferoglu Erdem Koyuncu 73 0 0 13 May 2025
LM-Scout: Analyzing the Security of Language Model Integration in Android Apps Muhammad Ibrahim Gűliz Seray Tuncay Z. Berkay Celik Aravind Machiry Antonio Bianchi 71 0 0 13 May 2025
Towards Artificial General or Personalized Intelligence? A Survey on Foundation Models for Personalized Federated Intelligence Yu Qiao Huy Q. Le Avi Deb Raha Phuong-Nam Tran Apurba Adhikary Mengchun Zhang Loc X. Nguyen Eui-nam Huh Dusit Niyato Choong Seon Hong AI4CE 163 1 0 11 May 2025
Camera Control at the Edge with Language Models for Scene Understanding Alexiy Buynitsky Sina Ehsani Bhanu Pallakonda Pragyana Mishra VLM 102 0 0 09 May 2025
Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inference Haolin Zhang Jeff Huang 73 0 0 09 May 2025
PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model Baijiong Lin Weisen Jiang Yuancheng Xu Hao Chen Ying-Cong Chen 88 1 0 06 May 2025
Invoke Interfaces Only When Needed: Adaptive Invocation for Large Language Models in Question Answering Jihao Zhao Chunlai Zhou Biao Qin 117 0 0 05 May 2025
Position: Enough of Scaling LLMs! Lets Focus on Downscaling Ayan Sengupta Ayan Sengupta Tanmoy Chakraborty 112 0 0 02 May 2025
When Reasoning Beats Scale: A 1.5B Reasoning Model Outranks 13B LLMs as Discriminator Md Fahim Anjum LRM 130 0 0 30 Apr 2025
Combatting Dimensional Collapse in LLM Pre-Training Data via Diversified File Selection Ziqing Fan Siyuan Du Shengchao Hu Pingjie Wang Li Shen Yanzhe Zhang Dacheng Tao Yucheng Wang 94 2 0 29 Apr 2025
A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning Jieming Bian Yuanzhe Peng Lei Wang Yin Huang Jie Xu FedML 117 1 0 29 Apr 2025
On-Device Qwen2.5: Efficient LLM Inference with Model Compression and Hardware Acceleration Maoyang Xiang Ramesh Fernando Bo Wang MQ 117 1 0 24 Apr 2025
Synergistic Weak-Strong Collaboration by Aligning Preferences Yizhu Jiao Xuchao Zhang Zhaoyang Wang Yubo Ma Zhun Deng Rujia Wang Chetan Bansal Saravan Rajmohan Jiawei Han Huaxiu Yao 481 0 0 21 Apr 2025
Kuwain 1.5B: An Arabic SLM via Language Injection Khalil Hennara Sara Chrouf Mohamed Motaism Hamed Zeina Aldallal Omar Hadid Safwan AlModhayan 96 2 0 21 Apr 2025
A Dual-Space Framework for General Knowledge Distillation of Large Language Models Wei Wei Songming Zhang Yunlong Liang Fandong Meng Yufeng Chen Jinan Xu Jie Zhou 127 0 0 15 Apr 2025
AccLLM: Accelerating Long-Context LLM Inference Via Algorithm-Hardware Co-Design Yanbiao Liang Huihong Shi Haikuo Shao Zhongfeng Wang 97 0 0 07 Apr 2025
Thanos: A Block-wise Pruning Algorithm for Efficient Large Language Model Compression Ivan Ilin Peter Richtárik 50 0 0 06 Apr 2025
MegaMath: Pushing the Limits of Open Math Corpora Fan Zhou Zengzhi Wang Nikhil Ranjan Zhoujun Cheng Liping Tang Guowei He Zhengzhong Liu Eric P. Xing LRM 137 3 0 03 Apr 2025
Mobile-VideoGPT: Fast and Accurate Video Understanding Language Model Abdelrahman M. Shaker Muhammad Maaz Chenhui Gou Hamid Rezatofighi Salman Khan Fahad Shahbaz Khan 428 0 0 27 Mar 2025
UGen: Unified Autoregressive Multimodal Model with Progressive Vocabulary Learning Hongxuan Tang Hao Liu Xinyan Xiao 79 2 0 27 Mar 2025
Overcoming Vocabulary Mismatch: Vocabulary-agnostic Teacher Guided Language Modeling Haebin Shin Lei Ji Xiao Liu Yeyun Gong 119 0 0 24 Mar 2025
SkyLadder: Better and Faster Pretraining via Context Window Scheduling Tongyao Zhu Qian Liu Haonan Wang Shiqi Chen Xiangming Gu Tianyu Pang Min-Yen Kan 102 0 0 19 Mar 2025
MASS: Mathematical Data Selection via Skill Graphs for Pretraining Large Language Models Jia-Nan Li Lu Yu Daixin Wang Qing Cui Jun Zhou Yanfang Ye Chuxu Zhang 122 0 0 19 Mar 2025
ConSCompF: Consistency-focused Similarity Comparison Framework for Generative Large Language Models Alexey Karev Dong Xu 150 0 0 18 Mar 2025