v1v2 (latest)

TinyLlama: An Open-Source Small Language Model

4 January 2024

Wei Lu

ArXiv (abs)PDF HTML Github (8509★)

Papers citing "TinyLlama: An Open-Source Small Language Model"

50 / 287 papers shown

Title
Probing Language Models for Pre-training Data Detection Zhenhua Liu Tong Zhu Chuanyuan Tan Haonan Lu Bing Liu Wenliang Chen 85 13 0 03 Jun 2024
Joint Embeddings for Graph Instruction Tuning Vlad Argatu Aaron Haag Oliver Lohse 91 0 0 31 May 2024
Shotluck Holmes: A Family of Efficient Small-Scale Large Language Vision Models For Video Captioning and Summarization Richard Luo Austin Peng Adithya Vasudev Rishabh Jain 44 2 0 31 May 2024
Would I Lie To You? Inference Time Alignment of Language Models using Direct Preference Heads Avelina Asada Hadji-Kyriacou Ognjen Arandjelović 37 1 0 30 May 2024
Improve Student's Reasoning Generalizability through Cascading Decomposed CoTs Distillation Chengwei Dai Kun Li Wei Zhou Song Hu LRM 89 5 0 30 May 2024
Beyond Imitation: Learning Key Reasoning Steps from Dual Chain-of-Thoughts in Reasoning Distillation Chengwei Dai Kun Li Wei Zhou Song Hu LRM 98 7 0 30 May 2024
Large Language Model Pruning Hanjuan Huang Hao-Jia Song H. Pao 117 0 0 24 May 2024
A Comparative Analysis of Distributed Training Strategies for GPT-2 Ishan Patwardhan Shubham Gandhi Om M. Khare Amit Joshi Suraj Sawant 100 2 0 24 May 2024
Bayesian WeakS-to-Strong from Text Classification to Generation Ziyun Cui Ziyang Zhang Wen Wu Wen Wu Chao Zhang 127 3 0 24 May 2024
AstroPT: Scaling Large Observation Models for Astronomy Michael J. Smith Ryan J. Roberts E. Angeloudi M. Huertas-Company 76 2 0 23 May 2024
Super Tiny Language Models Dylan Hillier Leon Guertler Cheston Tan Palaash Agrawal Ruirui Chen Bobby Cheng 113 6 0 23 May 2024
Dense Connector for MLLMs Huanjin Yao Wenhao Wu Taojiannan Yang Yuxin Song Mengxi Zhang Haocheng Feng Yifan Sun Zhiheng Li Wanli Ouyang Jingdong Wang MLLM VLM 102 25 0 22 May 2024
Disperse-Then-Merge: Pushing the Limits of Instruction Tuning via Alignment Tax Reduction Tingchen Fu Deng Cai Lemao Liu Shuming Shi Rui Yan MoMe 166 13 0 22 May 2024
Imp: Highly Capable Large Multimodal Models for Mobile Devices Zhenwei Shao Zhou Yu Jun Yu Xuecheng Ouyang Lihao Zheng Zhenbiao Gai Mingyang Wang Jiajun Ding 67 11 0 20 May 2024
TinyLLaVA Factory: A Modularized Codebase for Small-scale Large Multimodal Models Junlong Jia Ying Hu Xi Weng Yiming Shi Miao Li ... Baichuan Zhou Ziyu Liu Jie Luo Lei Huang Ji Wu 97 9 0 20 May 2024
Layer-Condensed KV Cache for Efficient Inference of Large Language Models Haoyi Wu Kewei Tu MQ 130 19 0 17 May 2024
Thinking Fair and Slow: On the Efficacy of Structured Prompts for Debiasing Language Models Shaz Furniturewala Surgan Jandial Abhinav Java Pragyan Banerjee Simra Shahid Sumita Bhatia Kokil Jaidka 107 11 0 16 May 2024
OpenBA-V2: Reaching 77.3% High Compression Ratio with Fast Multi-Stage Pruning Dan Qiao Yi Su Pinzheng Wang Jing Ye Wen Xie ... Wenliang Chen Guohong Fu Guodong Zhou Qiaoming Zhu Min Zhang MQ 60 0 0 09 May 2024
MiniGPT-3D: Efficiently Aligning 3D Point Clouds with Large Language Models using 2D Priors Yuan Tang Xu Han Xianzhi Li Qiao Yu Yixue Hao Long Hu Min Chen 86 20 0 02 May 2024
HLSTransform: Energy-Efficient Llama 2 Inference on FPGAs Via High Level Synthesis Andy He Darren Key Mason Bulling Andrew Chang Skyler Shapiro Everett Lee 68 1 0 29 Apr 2024
HateTinyLLM : Hate Speech Detection Using Tiny Large Language Models Tanmay Sen Ansuman Das Mrinmay Sen 75 4 0 26 Apr 2024
AI Coders Are Among Us: Rethinking Programming Language Grammar Towards Efficient Code Generation Zhensu Sun Xiaoning Du Zhou Yang Li Li David Lo 97 9 0 25 Apr 2024
XFT: Unlocking the Power of Code Instruction Tuning by Simply Merging Upcycled Mixture-of-Experts Yifeng Ding Jiawei Liu Yuxiang Wei Terry Yue Zhuo Lingming Zhang ALM MoE 99 3 0 23 Apr 2024
Automated Multi-Language to English Machine Translation Using Generative Pre-Trained Transformers Elijah Pelofske Vincent Urias L. Liebrock 87 0 0 23 Apr 2024
OpenELM: An Efficient Language Model Family with Open Training and Inference Framework Sachin Mehta Mohammad Hossein Sekhavat Qingqing Cao Maxwell Horton Yanzi Jin ... Iman Mirzadeh Mahyar Najibi Dmitry Belenko Peter Zatloukal Mohammad Rastegari OSLM AIFin 108 61 0 22 Apr 2024
Graphic Design with Large Multimodal Model Yutao Cheng Zhao Zhang Maoke Yang Hui Nie Chunyuan Li Xinglong Wu Jie Shao 98 15 0 22 Apr 2024
A Survey on Efficient Inference for Large Language Models Zixuan Zhou Xuefei Ning Ke Hong Tianyu Fu Jiaming Xu ... Shengen Yan Guohao Dai Xiao-Ping Zhang Yuhan Dong Yu Wang 171 98 0 22 Apr 2024
When Life gives you LLMs, make LLM-ADE: Large Language Models with Adaptive Data Engineering Stephen Choi William Gazeley KELM 46 2 0 19 Apr 2024
Which questions should I answer? Salience Prediction of Inquisitive Questions Yating Wu Ritika Mangla A. Dimakis Greg Durrett Junyi Jessy Li 73 5 0 16 Apr 2024
Resilience of Large Language Models for Noisy Instructions Bin Wang Chengwei Wei Zhengyuan Liu Geyu Lin Nancy F. Chen 142 15 0 15 Apr 2024
NoticIA: A Clickbait Article Summarization Dataset in Spanish Iker García-Ferrero Begoña Altuna 93 2 0 11 Apr 2024
pfl-research: simulation framework for accelerating research in Private Federated Learning Filip Granqvist Congzheng Song Áine Cahill Rogier van Dalen Martin Pelikan Yi Sheng Chan Xiaojun Feng Natarajan Krishnaswami Vojta Jina Mona Chitnis FedML 88 6 0 09 Apr 2024
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies Shengding Hu Yuge Tu Xu Han Chaoqun He Ganqu Cui ... Chaochao Jia Guoyang Zeng Dahai Li Zhiyuan Liu Maosong Sun MoE 131 347 0 09 Apr 2024
Enhancing Clinical Efficiency through LLM: Discharge Note Generation for Cardiac Patients Hyoje Jung Yunha Kim Heejung Choi Hyeram Seo Minkyoung Kim ... Soyoung Ko Byeolhee Kim Suyeon Kim Tae Joon Jun Young-Hak Kim 90 18 0 08 Apr 2024
Chinese Tiny LLM: Pretraining a Chinese-Centric Large Language Model Xinrun Du Zhouliang Yu Songyang Gao Ding Pan Yuyang Cheng ... Tianyu Zheng Xinchen Luo Guorui Zhou Wenhu Chen Ge Zhang 130 20 0 05 Apr 2024
Sailor: Open Language Models for South-East Asia Longxu Dou Qian Liu Guangtao Zeng Jia Guo Jiahui Zhou Wei Lu Min Lin LRM 106 9 0 04 Apr 2024
Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models Taiqiang Wu Chaofan Tao Jiahao Wang Zhe Zhao Ngai Wong ALM 99 18 0 03 Apr 2024
Exploring Backdoor Vulnerabilities of Chat Models Yunzhuo Hao Wenkai Yang Yankai Lin SILM KELM 64 11 0 03 Apr 2024
CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models Xuechen Liang Meiling Tao Yinghui Xia Yiting Xie Jun Wang JingSong Yang LLMAG 172 14 0 02 Apr 2024
Source-Aware Training Enables Knowledge Attribution in Language Models Muhammad Khalifa David Wadden Emma Strubell Honglak Lee Lu Wang Iz Beltagy Hao Peng HILM 137 14 0 01 Apr 2024
Capability-aware Prompt Reformulation Learning for Text-to-Image Generation Jingtao Zhan Qingyao Ai Yiqun Liu Jia Chen Shaoping Ma DiffM 81 5 0 27 Mar 2024
LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning Boyao Wang Xiang Liu Shizhe Diao Renjie Pi Jipeng Zhang Chi Han Tong Zhang 106 55 0 26 Mar 2024
MetaAligner: Towards Generalizable Multi-Objective Alignment of Language Models Kailai Yang Zhiwei Liu Qianqian Xie Jimin Huang Tianlin Zhang Sophia Ananiadou 86 18 0 25 Mar 2024
Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference Han Zhao Min Zhang Wei Zhao Pengxiang Ding Siteng Huang Donglin Wang Mamba 123 74 0 21 Mar 2024
Arcee's MergeKit: A Toolkit for Merging Large Language Models Charles Goddard Shamane Siriwardhana Malikeh Ehghaghi Luke Meyers Vladimir Karpukhin Brian Benedict Mark McQuade Jacob Solawetz MoMe KELM 178 103 0 20 Mar 2024
Semiparametric Token-Sequence Co-Supervision Hyunji Lee Doyoung Kim Jihoon Jun Se June Joo Joel Jang Kyoung-Woon On Minjoon Seo 114 1 0 14 Mar 2024
Mipha: A Comprehensive Overhaul of Multimodal Assistant with Small Language Models Minjie Zhu Yichen Zhu Xin Liu Ning Liu Zhiyuan Xu Yaxin Peng Chaomin Shen Zhicai Ou Feifei Feng Jian Tang VLM 100 22 0 10 Mar 2024
ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment Xiwei Hu Rui Wang Yixiao Fang Bin-Bin Fu Pei Cheng Gang Yu VLM 124 103 0 08 Mar 2024
Embodied Understanding of Driving Scenarios Yunsong Zhou Linyan Huang Qingwen Bu Jia Zeng Tianyu Li Hang Qiu Hongzi Zhu Minyi Guo Yu Qiao Hongyang Li LM&Ro 103 33 0 07 Mar 2024
SaulLM-7B: A pioneering Large Language Model for Law Pierre Colombo T. Pires Malik Boudiaf Dominic Culver Rui Melo ... Andre F. T. Martins Fabrizio Esposito Vera Lúcia Raposo Sofia Morgado Michael Desa ELM AILaw 114 75 0 06 Mar 2024