v1v2 (latest)

TinyLlama: An Open-Source Small Language Model

4 January 2024

Wei Lu

ArXiv (abs)PDF HTML Github (8509★)

Papers citing "TinyLlama: An Open-Source Small Language Model"

37 / 287 papers shown

Title
CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following Kaiyan Zhang Jianyu Wang Ermo Hua Biqing Qi Ning Ding Bowen Zhou SyDa 121 22 0 05 Mar 2024
Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge Heydar Soudani Evangelos Kanoulas Faegheh Hasibi 81 39 0 03 Mar 2024
Accelerating Greedy Coordinate Gradient via Probe Sampling Yiran Zhao Wenyue Zheng Tianle Cai Xuan Long Do Kenji Kawaguchi Anirudh Goyal Michael Shieh 96 13 0 02 Mar 2024
Large Language Models and Games: A Survey and Roadmap Roberto Gallotta Graham Todd Marvin Zammit Sam Earle Antonios Liapis Julian Togelius Georgios N. Yannakakis LLMAG LM&MA AI4CE LRM 131 86 0 28 Feb 2024
MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT Omkar Thawakar Ashmal Vayani Salman Khan Hisham Cholakal Rao M. Anwer Michael Felsberg Timothy Baldwin Eric P. Xing Fahad Shahbaz Khan 115 35 0 26 Feb 2024
RetrievalQA: Assessing Adaptive Retrieval-Augmented Generation for Short-form Open-Domain Question Answering Zihan Zhang Meng Fang Ling-Hao Chen RALM 96 14 0 26 Feb 2024
MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases Zechun Liu Changsheng Zhao Forrest N. Iandola Chen Lai Yuandong Tian ... Ernie Chang Yangyang Shi Raghuraman Krishnamoorthi Liangzhen Lai Vikas Chandra ALM 137 102 0 22 Feb 2024
Analysing The Impact of Sequence Composition on Language Model Pre-Training Yu Zhao Yuanbin Qu Konrad Staniszewski Szymon Tworkowski Wei Liu Piotr Milo's Yuxiang Wu Pasquale Minervini 83 15 0 21 Feb 2024
Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding Weilin Zhao Yuxiang Huang Xu Han Wang Xu Chaojun Xiao Xinrong Zhang Yewei Fang Kaihuo Zhang Zhiyuan Liu Maosong Sun 125 12 0 21 Feb 2024
Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs Jiejun Tan Zhicheng Dou Yutao Zhu Peidong Guo Kun Fang Ji-Rong Wen 129 30 0 19 Feb 2024
Self-AMPLIFY: Improving Small Language Models with Self Post Hoc Explanations Milan Bhan Jean-Noel Vittaut Nicolas Chesneau Marie-Jeanne Lesot ReLM LRM 117 6 0 19 Feb 2024
Revisiting Knowledge Distillation for Autoregressive Language Models Qihuang Zhong Liang Ding Li Shen Juhua Liu Bo Du Dacheng Tao KELM 117 19 0 19 Feb 2024
ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao LM&MA 100 23 0 19 Feb 2024
Efficient Multimodal Learning from Data-centric Perspective Muyang He Yexin Liu Boya Wu Jianhao Yuan Yueze Wang Tiejun Huang Bo Zhao MLLM 90 88 0 18 Feb 2024
OneBit: Towards Extremely Low-bit Large Language Models Yuzhuang Xu Xu Han Zonghan Yang Shuo Wang Qingfu Zhu Zhiyuan Liu Weidong Liu Wanxiang Che MQ 117 46 0 17 Feb 2024
DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows Ajay Patel Colin Raffel Chris Callison-Burch SyDa AI4CE 77 27 0 16 Feb 2024
Into the Unknown: Self-Learning Large Language Models Teddy Ferdinan Jan Kocoñ P. Kazienko 79 3 0 14 Feb 2024
An Embarrassingly Simple Approach for LLM with Strong ASR Capacity Ziyang Ma Guanrou Yang Yifan Yang Zhifu Gao Jiaming Wang ... Fan Yu Qian Chen Siqi Zheng Shiliang Zhang Xie Chen AuLLM 96 60 0 13 Feb 2024
Large Language Models: A Survey Shervin Minaee Tomas Mikolov Narjes Nikzad M. Asgari-Chenaghlu R. Socher Xavier Amatriain Jianfeng Gao ALM LM&MA ELM 248 425 0 09 Feb 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 241 116 0 08 Feb 2024
RevOrder: A Novel Method for Enhanced Arithmetic in Language Models Si Shen Peijun Shen Danhao Zhu MU 63 1 0 06 Feb 2024
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding Yichao Fu Peter Bailis Ion Stoica Hao Zhang 204 164 0 03 Feb 2024
Tiny Titans: Can Smaller Large Language Models Punch Above Their Weight in the Real World for Meeting Summarization? Xue-Yong Fu Md Tahmid Rahman Laskar Elena Khasanova Cheng-Hsiung Chen TN ShashiBhushan ALM 98 23 0 01 Feb 2024
CroissantLLM: A Truly Bilingual French-English Language Model Manuel Faysse Patrick Fernandes Nuno M. Guerreiro António Loison Duarte M. Alves ... François Yvon André F.T. Martins Gautier Viaud C´eline Hudelot Pierre Colombo 165 37 0 01 Feb 2024
H2O-Danube-1.8B Technical Report Philipp Singer Pascal Pfeiffer Yauhen Babakhin Maximilian Jeblick Nischay Dhankhar Gabor Fodor SriSatish Ambati VLM 60 8 0 30 Jan 2024
TeenyTinyLlama: open-source tiny language models trained in Brazilian Portuguese N. Corrêa Sophia Falk Shiza Fatimah Aniket Sen N. D. Oliveira 89 9 0 30 Jan 2024
Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling Pratyush Maini Skyler Seto Richard He Bai David Grangier Yizhe Zhang Navdeep Jaitly SyDa 89 67 0 29 Jan 2024
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models Fuzhao Xue Zian Zheng Yao Fu Jinjie Ni Zangwei Zheng Wangchunshu Zhou Yang You MoE 97 104 0 29 Jan 2024
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty Yuhui Li Fangyun Wei Chao Zhang Hongyang R. Zhang 191 165 0 26 Jan 2024
Wordflow: Social Prompt Engineering for Large Language Models Zijie J. Wang Aishwarya Chakravarthy David Munechika Duen Horng Chau 80 14 0 25 Jan 2024
Metacognition is all you need? Using Introspection in Generative Agents to Improve Goal-directed Behavior Jason Toy Josh MacAdam Phil Tabor LLMAG LRM AI4CE 82 4 0 09 Jan 2024
Mitigating Open-Vocabulary Caption Hallucinations Assaf Ben-Kish Moran Yanuka Morris Alper Raja Giryes Hadar Averbuch-Elor MLLM VLM 123 6 0 06 Dec 2023
AdaLomo: Low-memory Optimization with Adaptive Learning Rate Kai Lv Hang Yan Qipeng Guo Haijun Lv Xipeng Qiu ODL 88 23 0 16 Oct 2023
Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning Mengzhou Xia Tianyu Gao Zhiyuan Zeng Danqi Chen 125 311 0 10 Oct 2023
Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity Lu Yin You Wu Zhenyu Zhang Cheng-Yu Hsieh Yaqing Wang ... Mykola Pechenizkiy Yi Liang Michael Bendersky Zhangyang Wang Shiwei Liu 137 103 0 08 Oct 2023
SlimPajama-DC: Understanding Data Combinations for LLM Training Zhiqiang Shen Tianhua Tao Liqun Ma Willie Neiswanger Zhengzhong Liu ... Bowen Tan Joel Hestness Natalia Vassilieva Daria Soboleva Eric Xing 109 50 0 19 Sep 2023
Understanding Performance of Long-Document Ranking Models through Comprehensive Evaluation and Leaderboarding Leonid Boytsov David Akinpelu Tianyi Lin Fangwei Gao Yutian Zhao Jeffrey Huang Nipun Katyal Eric Nyberg 182 9 0 04 Jul 2022