The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only

1 June 2023

Guilherme Penedo

Quentin Malartic

Daniel Hesslow

Ruxandra-Aimée Cojocaru

Papers citing "The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only"

31 / 131 papers shown

Title
Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on Understanding Bram van Dijk Tom Kouwenhoven M. Spruit Max J. van Duijn 100 21 0 30 Oct 2023
Efficient kernel surrogates for neural network-based regression S. Qadeer A. Engel Amanda A. Howard Adam Tsou Max Vargas P. Stinis Tony Chiang 105 5 0 28 Oct 2023
LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers Theo X. Olausson Alex Gu Benjamin Lipkin Cedegao E. Zhang Armando Solar-Lezama Josh Tenenbaum Roger Levy LRM AI4CE ReLM 160 119 0 23 Oct 2023
Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluation M. Boubdir Edward Kim Beyza Ermis Marzieh Fadaee Sara Hooker ALM 83 19 0 22 Oct 2023
Enhancing Zero-Shot Crypto Sentiment with Fine-tuned Language Model and Prompt Engineering Rahman S. M. Wahidur Ishmam Tashdeed Manjit Kaur Heung-No Lee ALM 89 17 0 20 Oct 2023
NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails Traian Rebedea R. Dinu Makesh Narsimhan Sreedhar Christopher Parisien Jonathan Cohen KELM 99 152 0 16 Oct 2023
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning Jun Chen Deyao Zhu Xiaoqian Shen Xiang Li Zechun Liu Pengchuan Zhang Raghuraman Krishnamoorthi Vikas Chandra Yunyang Xiong Mohamed Elhoseiny MLLM 249 473 0 14 Oct 2023
Lemur: Harmonizing Natural Language and Code for Language Agents Yiheng Xu Hongjin Su Chen Xing Boyu Mi Qian Liu ... Siheng Zhao Lingpeng Kong Bailin Wang Caiming Xiong Tao Yu 93 74 0 10 Oct 2023
SCAR: Power Side-Channel Analysis at RTL-Level Amisha Srivastava Sanjay Das Navnil Choudhury Rafail Psiakis Pedro Henrique Silva Debjit Pal Kanad Basu 70 11 0 10 Oct 2023
GPT-who: An Information Density-based Machine-Generated Text Detector Saranya Venkatraman Adaku Uchendu Dongwon Lee DeLMO 90 35 0 09 Oct 2023
MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning Ke Wang Houxing Ren Aojun Zhou Zimu Lu Sichun Luo Weikang Shi Renrui Zhang Linqi Song Mingjie Zhan Hongsheng Li ReLM LRM SyDa 114 106 0 05 Oct 2023
JsonTuning: Towards Generalizable, Robust, and Controllable Instruction Tuning Chang Gao Wenxuan Zhang Guizhen Chen Wai Lam 214 6 0 04 Oct 2023
From Text to Source: Results in Detecting Large Language Model-Generated Content Wissam Antoun Benoît Sagot Djamé Seddah DeLMO 85 13 0 23 Sep 2023
GlotScript: A Resource and Tool for Low Resource Writing System Identification Amir Hossein Kargaran François Yvon Hinrich Schütze 42 11 0 23 Sep 2023
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models L. Yu Weisen Jiang Han Shi Jincheng Yu Zhengying Liu Yu Zhang James T. Kwok Zheng Li Adrian Weller Weiyang Liu OSLM LRM 114 395 0 21 Sep 2023
BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model Nolan Dey Daria Soboleva Faisal Al-Khateeb Bowen Yang Ribhu Pathria ... Robert Myers Jacob Robert Steeves Natalia Vassilieva Marvin Tom Joel Hestness MoE 85 16 0 20 Sep 2023
Baichuan 2: Open Large-scale Language Models Ai Ming Yang Bin Xiao Bingning Wang Borong Zhang Ce Bian ... Youxin Jiang Yuchen Gao Yupeng Zhang Guosheng Dong Zhiying Wu ELM LRM 320 755 0 19 Sep 2023
MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning Xiang Yue Xingwei Qu Ge Zhang Yao Fu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen AIMat LRM 183 404 0 11 Sep 2023
Understanding the Impact of Post-Training Quantization on Large Language Models Somnath Roy MQ 71 5 0 11 Sep 2023
No Train Still Gain. Unleash Mathematical Reasoning of Large Language Models with Monte Carlo Tree Search Guided by Energy Function Haotian Xu LRM 87 14 0 01 Sep 2023
WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct Haipeng Luo Qingfeng Sun Can Xu Pu Zhao Jian-Guang Lou ... Xiubo Geng Qingwei Lin Shifeng Chen Yansong Tang Dongmei Zhang LRM OSLM 299 468 0 18 Aug 2023
A Comparative Study of Open-Source Large Language Models, GPT-4 and Claude 2: Multiple-Choice Test Taking in Nephrology Sean Wu Michael Koo L. Blum A. Black Liyo Kao Fabien Scalzo Ira Kurtz LM&MA ELM AI4MH 64 49 0 09 Aug 2023
General Purpose Artificial Intelligence Systems (GPAIS): Properties, Definition, Taxonomy, Societal Implications and Responsible Governance I. Triguero Daniel Molina Javier Poyatos Javier Del Ser Francisco Herrera AI4TS AI4MH 122 5 0 26 Jul 2023
CMMLU: Measuring massive multitask language understanding in Chinese Haonan Li Yixuan Zhang Fajri Koto Yifei Yang Hai Zhao Yeyun Gong Nan Duan Tim Baldwin ALM ELM 116 273 0 15 Jun 2023
WizardCoder: Empowering Code Large Language Models with Evol-Instruct Ziyang Luo Can Xu Pu Zhao Qingfeng Sun Xiubo Geng Wenxiang Hu Chongyang Tao Jing Ma Qingwei Lin Daxin Jiang ELM SyDa ALM 178 697 0 14 Jun 2023
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration Ji Lin Jiaming Tang Haotian Tang Shang Yang Wei-Ming Chen Wei-Chen Wang Guangxuan Xiao Xingyu Dang Chuang Gan Song Han EDL MQ 156 587 0 01 Jun 2023
Scaling Data-Constrained Language Models Niklas Muennighoff Alexander M. Rush Boaz Barak Teven Le Scao Aleksandra Piktus Nouamane Tazi S. Pyysalo Thomas Wolf Colin Raffel ALM 176 226 0 25 May 2023
Polyglot or Not? Measuring Multilingual Encyclopedic Knowledge in Foundation Models Tim Schott Daniel Furman Shreshta Bhat ELM 69 4 0 23 May 2023
A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity Shayne Longpre Gregory Yauney Emily Reif Katherine Lee Adam Roberts ... Denny Zhou Jason W. Wei Kevin Robinson David M. Mimno Daphne Ippolito 117 168 0 22 May 2023
Prophet: Prompting Large Language Models with Complementary Answer Heuristics for Knowledge-based Visual Question Answering Zhou Yu Xuecheng Ouyang Zhenwei Shao Mei Wang Jun Yu MLLM 184 11 0 03 Mar 2023
JASMINE: Arabic GPT Models for Few-Shot Learning El Moatez Billah Nagoudi Muhammad Abdul-Mageed AbdelRahim Elmadany Alcides Alcoba Inciarte Md. Tawkat Islam Khondaker 69 8 0 21 Dec 2022