The False Promise of Imitating Proprietary LLMs

25 May 2023

Pieter Abbeel

Papers citing "The False Promise of Imitating Proprietary LLMs"

50 / 178 papers shown

Title
Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer Hele-Andra Kuulmets Taido Purason Agnes Luhtaru Mark Fishel 21 17 0 05 Apr 2024
Emergent Abilities in Reduced-Scale Generative Language Models Sherin Muckatira Vijeta Deshpande Vladislav Lialin Anna Rumshisky ReLM ELM LRM 27 4 0 02 Apr 2024
GOLD: Generalized Knowledge Distillation via Out-of-Distribution-Guided Language Data Generation Mohsen Gholami Mohammad Akbari Cindy Hu Vaden Masrani Z. J. Wang Yong Zhang 41 4 0 28 Mar 2024
Qibo: A Large Language Model for Traditional Chinese Medicine Heyi Zhang Xin Wang Zhaopeng Meng Zhe Chen Pengwei Zhuang Yongzhe Jia Dawei Xu Wenbin Guo LM&MA 26 9 0 24 Mar 2024
Information-Theoretic Distillation for Reference-less Summarization Jaehun Jung Ximing Lu Liwei Jiang Faeze Brahman Peter West Pang Wei Koh Yejin Choi 38 3 0 20 Mar 2024
Securing Large Language Models: Threats, Vulnerabilities and Responsible Practices Sara Abdali Richard Anarfi C. Barberan Jia He PILM 67 24 0 19 Mar 2024
Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision Zhiqing Sun Longhui Yu Yikang Shen Weiyang Liu Yiming Yang Sean Welleck Chuang Gan 30 52 0 14 Mar 2024
Unveiling the Generalization Power of Fine-Tuned Large Language Models Haoran Yang Yumeng Zhang Jiaqi Xu Hongyuan Lu Pheng Ann Heng Wai Lam 43 28 0 14 Mar 2024
Large Language Models and Causal Inference in Collaboration: A Survey Xiaoyu Liu Paiheng Xu Junda Wu Jiaxin Yuan Yifan Yang ... Haoliang Wang Tong Yu Julian McAuley Wei Ai Furong Huang ELM LRM 77 34 0 14 Mar 2024
HaluEval-Wild: Evaluating Hallucinations of Language Models in the Wild Zhiying Zhu Yiming Yang Zhiqing Sun HILM VLM 41 14 0 07 Mar 2024
Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs Aly M. Kassem Omar Mahmoud Niloofar Mireshghallah Hyunwoo J. Kim Yulia Tsvetkov Yejin Choi Sherif Saad Santu Rana 50 18 0 05 Mar 2024
A Survey of AI-generated Text Forensic Systems: Detection, Attribution, and Characterization Tharindu Kumarage Garima Agrawal Paras Sheth Raha Moraffah Amanat Chadha Joshua Garland Huan Liu DeLMO 34 11 0 02 Mar 2024
LAB: Large-Scale Alignment for ChatBots Shivchander Sudalairaj Abhishek Bhandwaldar Aldo Pareja Kai Xu David D. Cox Akash Srivastava OSLM 33 28 0 02 Mar 2024
FAC $^2$ E: Better Understanding Large Language Model Capabilities by Dissociating Language and Cognition Xiaoqiang Wang Bang Liu Lingfei Wu 35 0 0 29 Feb 2024
Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptation Nihal V. Nayak Yiyang Nan Avi Trost Stephen H. Bach SyDa 35 13 0 28 Feb 2024
Learning or Self-aligning? Rethinking Instruction Fine-tuning Mengjie Ren Boxi Cao Hongyu Lin Liu Cao Xianpei Han Ke Zeng Guanglu Wan Xunliang Cai Le Sun 30 24 0 28 Feb 2024
MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues Ge Bai Jie Liu Xingyuan Bu Yancheng He Jiaheng Liu ... Zhuoran Lin Wenbo Su Tiezheng Ge Bo Zheng Wanli Ouyang ELM LM&MA 32 71 0 22 Feb 2024
Kuaiji: the First Chinese Accounting Large Language Model Jiayuan Luo Songhua Yang Xiaoling Qiu Panyu Chen Yufei Nai Wenxuan Zeng Wentao Zhang Xinke Jiang RALM ALM 38 1 0 21 Feb 2024
Retrieval-Augmented Data Augmentation for Low-Resource Domain Tasks Minju Seo Jinheon Baek James Thorne Sung Ju Hwang RALM 34 9 0 21 Feb 2024
Large Language Models for Data Annotation: A Survey Zhen Tan Dawei Li Song Wang Alimohammad Beigi Bohan Jiang Amrita Bhattacharjee Mansooreh Karami Jundong Li Lu Cheng Huan Liu SyDa 54 49 0 21 Feb 2024
A Survey on Knowledge Distillation of Large Language Models Xiaohan Xu Ming Li Chongyang Tao Tao Shen Reynold Cheng Jinyang Li Can Xu Dacheng Tao Tianyi Zhou KELM VLM 44 100 0 20 Feb 2024
Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models Haoran Li Qingxiu Dong Zhengyang Tang Chaojun Wang Xingxing Zhang ... Wei Lu Zhifang Sui Benyou Wang Wai Lam Furu Wei SyDa 56 50 0 20 Feb 2024
Reformatted Alignment Run-Ze Fan Xuefeng Li Haoyang Zou Junlong Li Shwai He Ethan Chern Jiewen Hu Pengfei Liu 57 8 0 19 Feb 2024
Revisiting Knowledge Distillation for Autoregressive Language Models Qihuang Zhong Liang Ding Li Shen Juhua Liu Bo Du Dacheng Tao KELM 39 16 0 19 Feb 2024
Dissecting Human and LLM Preferences Junlong Li Fan Zhou Shichao Sun Yikai Zhang Hai Zhao Pengfei Liu ALM 21 5 0 17 Feb 2024
Aligning Crowd Feedback via Distributional Preference Reward Modeling Dexun Li Cong Zhang Kuicai Dong Derrick-Goh-Xin Deik Ruiming Tang Yong Liu 16 14 0 15 Feb 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 41 47 0 15 Feb 2024
Rethinking Data Selection for Supervised Fine-Tuning Ming Shen 26 16 0 08 Feb 2024
Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning Hao Zhao Maksym Andriushchenko Francesco Croce Nicolas Flammarion ALM 89 44 0 07 Feb 2024
Position: What Can Large Language Models Tell Us about Time Series Analysis Ming Jin Yifan Zhang Wei Chen Kexin Zhang Yuxuan Liang Bin Yang Jindong Wang Shirui Pan Qingsong Wen AI4TS 29 16 0 05 Feb 2024
A Closer Look at the Limitations of Instruction Tuning Sreyan Ghosh Chandra Kiran Reddy Evuru Sonal Kumar Reddy Evuru Deepali Aneja Zeyu Jin R. Duraiswami Dinesh Manocha ALM 75 28 0 03 Feb 2024
Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment Keming Lu Bowen Yu Chang Zhou Jingren Zhou 45 56 0 23 Jan 2024
Tuning Language Models by Proxy Alisa Liu Xiaochuang Han Yizhong Wang Yulia Tsvetkov Yejin Choi Noah A. Smith ALM 38 44 0 16 Jan 2024
Multilingual Instruction Tuning With Just a Pinch of Multilinguality Uri Shaham Jonathan Herzig Roee Aharoni Idan Szpektor Reut Tsarfaty Matan Eyal LRM 37 41 0 03 Jan 2024
A Reliable Knowledge Processing Framework for Combustion Science using Foundation Models Vansh Sharma Venkat Raman 21 7 0 31 Dec 2023
Hazards from Increasingly Accessible Fine-Tuning of Downloadable Foundation Models Alan Chan Ben Bucknall Herbie Bradley David M. Krueger 14 6 0 22 Dec 2023
A Comparative Analysis of Large Language Models for Code Documentation Generation Shubhang Shekhar Dvivedi Vyshnav Vijay Sai Leela Rahul Pujari Shoumik Lodh Dhruv Kumar ELM 25 12 0 16 Dec 2023
TinyGSM: achieving >80% on GSM8k with small language models Bingbin Liu Sébastien Bubeck Ronen Eldan Janardhan Kulkarni Yuanzhi Li Anh Nguyen Rachel A. Ward Yi Zhang ALM 19 47 0 14 Dec 2023
AI capabilities can be significantly improved without expensive retraining Tom Davidson Jean-Stanislas Denain Pablo Villalobos Guillem Bas OffRL VLM 24 26 0 12 Dec 2023
The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning Bill Yuchen Lin Abhilasha Ravichander Ximing Lu Nouha Dziri Melanie Sclar Khyathi Raghavi Chandu Chandra Bhagavatula Yejin Choi 22 164 0 04 Dec 2023
The Cost of Compression: Investigating the Impact of Compression on Parametric Knowledge in Language Models Srinath Namburi Makesh Narsimhan Sreedhar Srinath Srinivasan Frederic Sala MQ 26 8 0 01 Dec 2023
The Philosopher's Stone: Trojaning Plugins of Large Language Models Tian Dong Minhui Xue Guoxing Chen Rayne Holland Shaofeng Li Yan Meng Zhen Liu Haojin Zhu AAML 18 9 0 01 Dec 2023
AviationGPT: A Large Language Model for the Aviation Domain Liya Wang Jason Chou Xin Zhou A. Tien Diane M. Baumgartner 19 7 0 29 Nov 2023
LIMIT: Less Is More for Instruction Tuning Across Evaluation Paradigms Aditi Jha Sam Havens Jeremey Dohmann Alex Trott Jacob P. Portes ALM 19 11 0 22 Nov 2023
Data Diversity Matters for Robust Instruction Tuning Alexander Bukharin Tuo Zhao 77 35 0 21 Nov 2023
Exploring the Relationship between In-Context Learning and Instruction Tuning Hanyu Duan Yixuan Tang Yi Yang Ahmed Abbasi K. Tam 37 11 0 17 Nov 2023
Mind's Mirror: Distilling Self-Evaluation Capability and Comprehensive Thinking from Large Language Models Weize Liu Guocong Li Kai Zhang Bang Du Qiyuan Chen Xuming Hu Hongxia Xu Jintai Chen Jian Wu LRM 18 6 0 15 Nov 2023
Post Turing: Mapping the landscape of LLM Evaluation Alexey Tikhonov Ivan P. Yamshchikov ELM 46 4 0 03 Nov 2023
Market Concentration Implications of Foundation Models Jai Vipra Anton Korinek ELM 32 16 0 02 Nov 2023
Defining a New NLP Playground Sha Li Chi Han Pengfei Yu Carl N. Edwards Manling Li ... Yi Ren Fung Charles Yu Joel R. Tetreault Eduard H. Hovy Heng Ji 33 5 0 31 Oct 2023