v1v2 (latest)

TinyLlama: An Open-Source Small Language Model

4 January 2024

Wei Lu

ArXiv (abs)PDF HTML Github (8509★)

Papers citing "TinyLlama: An Open-Source Small Language Model"

50 / 287 papers shown

Title
Selective Preference Optimization via Token-Level Reward Function Estimation Kailai Yang Zhiwei Liu Qianqian Xie Jimin Huang Erxue Min Sophia Ananiadou 87 13 0 24 Aug 2024
Predicting Rewards Alongside Tokens: Non-disruptive Parameter Insertion for Efficient Inference Intervention in Large Language Model Chenhan Yuan Fei Huang Ru Peng Keming Lu Bowen Yu Chang Zhou Jingren Zhou KELM 78 0 0 20 Aug 2024
Edge-Cloud Collaborative Motion Planning for Autonomous Driving with Large Language Models Jiao Chen Suyan Dai Fangfang Chen Zuohong Lv Jianhua Tang 77 6 0 19 Aug 2024
The advantages of context specific language models: the case of the Erasmian Language Model João Gonçalves Nick Jelicic Michele Murgia Evert Stamhuis 99 0 0 13 Aug 2024
Deeploy: Enabling Energy-Efficient Deployment of Small Language Models On Heterogeneous Microcontrollers Moritz Scherer Luka Macan Victor J. B. Jung Philip Wiese Luca Bompani Luca Bompani Francesco Conti Luca Benini MoE 73 12 0 08 Aug 2024
Designing Efficient LLM Accelerators for Edge Devices Jude Haris Rappy Saha Wenhao Hu José Cano 106 7 0 01 Aug 2024
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen Senmiao Wang Zhihang Lin Zhihang Lin Yushun Zhang Tian Ding Ruoyu Sun Ruoyu Sun CLL 177 5 0 30 Jul 2024
SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domain Pierre Colombo T. Pires Malik Boudiaf Rui Melo Dominic Culver Sofia Morgado Etienne Malaboeuf Gabriel Hautreux Johanne Charpentier Michael Desa ELM AILaw ALM 98 17 0 28 Jul 2024
Towards Effective and Efficient Continual Pre-training of Large Language Models Jie Chen Zhipeng Chen Jiapeng Wang Kun Zhou Yutao Zhu ... Rui Yan Zhewei Wei Di Hu Wenbing Huang Ji-Rong Wen KELM ALM CLL ELM LRM 337 6 0 26 Jul 2024
Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models Haoyu Tang Ye Liu Xukai Liu Xukai Liu Yanghai Zhang Kai Zhang Xiaofang Zhou Enhong Chen MU 160 3 0 25 Jul 2024
Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning Yeongbin Seo Dongha Lee Jinyoung Yeo CLL KELM 163 2 0 24 Jul 2024
LLaST: Improved End-to-end Speech Translation System Leveraged by Large Language Models Xi Chen Songyang Zhang Qibing Bai Kai-xiang Chen Satoshi Nakamura AuLLM 106 7 0 22 Jul 2024
Watermark Smoothing Attacks against Language Models Hongyan Chang Hamed Hassani Reza Shokri WaLM 141 3 0 19 Jul 2024
Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies Chaofan Tao Qian Liu Longxu Dou Niklas Muennighoff Zhongwei Wan Ping Luo Min Lin Ngai Wong PILM 130 54 0 18 Jul 2024
PFPs: Prompt-guided Flexible Pathological Segmentation for Diverse Potential Outcomes Using Large Vision and Language Models Can Cui Ruining Deng Junlin Guo Quan Liu Tianyuan Yao Haichun Yang Yuankai Huo VLM MedIm 73 1 0 13 Jul 2024
H2O-Danube3 Technical Report Pascal Pfeiffer Philipp Singer Yauhen Babakhin Gabor Fodor Nischay Dhankhar Sri Satish Ambati 57 4 0 12 Jul 2024
FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation Liqun Ma Mingjie Sun Zhiqiang Shen 86 9 0 09 Jul 2024
Statistical investigations into the geometry and homology of random programs Jon Sporring Ken Friis Larsen 32 0 0 05 Jul 2024
LoCo: Low-Bit Communication Adaptor for Large-scale Model Training Xingyu Xie Zhijie Lin Kim-Chuan Toh Pan Zhou 96 3 0 05 Jul 2024
LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models Hayder Elesedy Pedro M. Esperança Silviu Vlad Oprea Mete Ozay KELM 94 4 0 03 Jul 2024
RegMix: Data Mixture as Regression for Language Model Pre-training Qian Liu Xiaosen Zheng Niklas Muennighoff Guangtao Zeng Longxu Dou Tianyu Pang Jing Jiang Min Lin MoE 172 54 1 01 Jul 2024
BERGEN: A Benchmarking Library for Retrieval-Augmented Generation David Rau Hervé Déjean Nadezhda Chirkova Thibault Formal Shuai Wang Vassilina Nikoulina Stéphane Clinchant 86 14 0 01 Jul 2024
FoldGPT: Simple and Effective Large Language Model Compression Scheme Songwei Liu Chao Zeng Lianqiang Li Chenqian Yan Lean Fu Xing Mei Fangmin Chen 86 5 0 01 Jul 2024
Can Small Language Models Learn, Unlearn, and Retain Noise Patterns? Nicy Scaria Silvester John Joseph Kennedy Deepak N. Subramani MU 124 2 0 01 Jul 2024
Single Parent Family: A Spectrum of Family Members from a Single Pre-Trained Foundation Model Habib Hajimolahoseini Mohammad Hassanpour Foozhan Ataiefard Boxing Chen Yang Liu 83 2 0 28 Jun 2024
BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5 Zhehuai Chen He Huang Oleksii Hrinchuk Krishna Puvvada Nithin Rao Koluguri Piotr Żelasko Jagadeesh Balam Boris Ginsburg AuLLM RALM 92 11 0 28 Jun 2024
Dual-Space Knowledge Distillation for Large Language Models Songming Zhang Xue Zhang Zengkui Sun Yufeng Chen Jinan Xu 90 9 0 25 Jun 2024
Efficient Continual Pre-training by Mitigating the Stability Gap Yiduo Guo Jie Fu Huishuai Zhang Dongyan Zhao Songlin Yang 79 15 0 21 Jun 2024
Instruction Data Generation and Unsupervised Adaptation for Speech Language Models Vahid Noroozi Zhehuai Chen Somshubra Majumdar Steve Huang Jagadeesh Balam Boris Ginsburg SyDa 139 5 0 18 Jun 2024
Unveiling and Mitigating Bias in Mental Health Analysis with Large Language Models Yuqing Wang Yun Zhao Sara Alessandra Keller A. D. Hond Marieke M. van Buchem Malvika Pillai Tina Hernandez-Boussard AI4MH 65 4 0 17 Jun 2024
HARE: HumAn pRiors, a key to small language model Efficiency Lingyun Zhang Bin jin Gaojian Ge Lunhui Liu Xuewen Shen Mingyong Wu Houqian Zhang Yongneng Jiang Shiqi Chen Shi Pu ALM 68 0 0 17 Jun 2024
Self-training Large Language Models through Knowledge Detection Wei Jie Yeo Teddy Ferdinan Przemyslaw Kazienko Ranjan Satapathy Erik Cambria 104 10 0 17 Jun 2024
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion Chenghao Fan Zhenyi Lu Wei Wei Jie Tian Xiaoye Qu Dangyang Chen Yu Cheng MoMe 105 6 0 17 Jun 2024
GEB-1.3B: Open Lightweight Large Language Model Jie Wu Yufeng Zhu Lei Shen Xuqing Lu ALM 44 0 0 14 Jun 2024
MobileAIBench: Benchmarking LLMs and LMMs for On-Device Use Cases Rithesh Murthy Liangwei Yang Juntao Tan Tulika Awalgaonkar Yilun Zhou ... Zuxin Liu Ming Zhu Huan Wang Caiming Xiong Silvio Savarese 81 6 0 12 Jun 2024
The Impact of Initialization on LoRA Finetuning Dynamics Soufiane Hayou Nikhil Ghosh Bin Yu AI4CE 105 22 0 12 Jun 2024
Prompt-Based Length Controlled Generation with Multiple Control Types Renlong Jie Xiaojun Meng Lifeng Shang Xin Jiang Qun Liu 89 8 0 12 Jun 2024
OLMES: A Standard for Language Model Evaluations Yuling Gu Oyvind Tafjord Bailey Kuehl Dany Haddad Jesse Dodge Hannaneh Hajishirzi ELM 131 20 0 12 Jun 2024
Open-LLM-Leaderboard: From Multi-choice to Open-style Questions for LLMs Evaluation, Benchmark, and Arena Aidar Myrzakhan Sondos Mahmoud Bsharat Zhiqiang Shen ELM 75 39 0 11 Jun 2024
Language Models Resist Alignment: Evidence From Data Compression Yalan Qin Kaile Wang Tianyi Qiu Boyuan Chen Jiayi Zhou Changye Li Hantao Lou Juntao Dai Y. Liu Yaodong Yang 119 3 0 10 Jun 2024
Revisiting Catastrophic Forgetting in Large Language Model Tuning Hongyu Li Liang Ding Meng Fang Dacheng Tao CLL KELM 84 19 0 07 Jun 2024
LipGER: Visually-Conditioned Generative Error Correction for Robust Automatic Speech Recognition Sreyan Ghosh Sonal Kumar Ashish Seth Purva Chiniya Utkarsh Tyagi R. Duraiswami Dinesh Manocha 99 0 0 06 Jun 2024
RoboMamba: Multimodal State Space Model for Efficient Robot Reasoning and Manipulation Jiaming Liu Mengzhen Liu Zhenyu Wang Lily Lee Kaichen Zhou Pengju An Senqiao Yang Renrui Zhang Yandong Guo Shanghang Zhang LM&Ro LRM Mamba 111 19 0 06 Jun 2024
DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMs Lingchen Meng Jianwei Yang Rui Tian Xiyang Dai Zuxuan Wu Jianfeng Gao Yu-Gang Jiang VLM 90 9 0 06 Jun 2024
LLMEmbed: Rethinking Lightweight LLM's Genuine Function in Text Classification Chun Liu Hongguang Zhang Kainan Zhao Xinghai Ju Lin Yang 82 4 0 06 Jun 2024
Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models Peijie Dong Lujun Li Zhenheng Tang Xiang Liu Xinglin Pan Qiang-qiang Wang Xiaowen Chu 153 33 0 05 Jun 2024
Defending Large Language Models Against Attacks With Residual Stream Activation Analysis Amelia Kawasaki Andrew Davis Houssam Abbas AAML KELM 98 3 0 05 Jun 2024
Loki: Low-Rank Keys for Efficient Sparse Attention Prajwal Singhania Siddharth Singh Shwai He Soheil Feizi A. Bhatele 110 22 0 04 Jun 2024
SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices Ruslan Svirschevski Avner May Zhuoming Chen Beidi Chen Zhihao Jia Max Ryabinin 140 21 0 04 Jun 2024
Sparsity-Accelerated Training for Large Language Models Da Ma Lu Chen Pengyu Wang Hongshen Xu Hanqi Li Liangtai Sun Su Zhu Shuai Fan Kai Yu LRM 60 1 0 03 Jun 2024