The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only

1 June 2023

Guilherme Penedo

Quentin Malartic

Daniel Hesslow

Ruxandra-Aimée Cojocaru

Papers citing "The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only"

50 / 587 papers shown

Title
LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers Theo X. Olausson Alex Gu Benjamin Lipkin Cedegao E. Zhang Armando Solar-Lezama Josh Tenenbaum Roger Levy LRM AI4CE ReLM 49 95 0 23 Oct 2023
S3Eval: A Synthetic, Scalable, Systematic Evaluation Suite for Large Language Models Fangyu Lei Qian Liu Yiming Huang Shizhu He Jun Zhao Kang Liu ELM LRM 25 12 0 23 Oct 2023
Domain Terminology Integration into Machine Translation: Leveraging Large Language Models Yasmin Moslem Gianfranco Romani Mahdi Molaei Rejwanul Haque John D. Kelleher Andy Way 33 16 0 22 Oct 2023
Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluation M. Boubdir Edward Kim Beyza Ermis Marzieh Fadaee Sara Hooker ALM 33 18 0 22 Oct 2023
CLMSM: A Multi-Task Learning Framework for Pre-training on Procedural Text Abhilash Nandy M. Kapadnis Pawan Goyal Niloy Ganguly 13 1 0 22 Oct 2023
From Static to Dynamic: A Continual Learning Framework for Large Language Models Mingzhe Du A. Luu Bin Ji See-kiong Ng 11 2 0 22 Oct 2023
Revisiting Instruction Fine-tuned Model Evaluation to Guide Industrial Applications Manuel Faysse Gautier Viaud C´eline Hudelot Pierre Colombo 32 9 0 21 Oct 2023
Ensemble-Instruct: Generating Instruction-Tuning Data with a Heterogeneous Mixture of LMs Young-Suk Lee Md Arafat Sultan Yousef El-Kurdi Tahira Naseem Asim Munawar Radu Florian Salim Roukos Ramón Fernández Astudillo SyDa 27 6 0 21 Oct 2023
Foundation Model's Embedded Representations May Detect Distribution Shift Max Vargas Adam Tsou A. Engel Tony Chiang 28 1 0 20 Oct 2023
Copyright Violations and Large Language Models Antonia Karamolegkou Jiaang Li Li Zhou Anders Sogaard 17 54 0 20 Oct 2023
Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking Shengyao Zhuang Bing Liu Bevan Koopman Guido Zuccon RALM 32 46 0 20 Oct 2023
Enhancing Zero-Shot Crypto Sentiment with Fine-tuned Language Model and Prompt Engineering Rahman S. M. Wahidur Ishmam Tashdeed Manjit Kaur Heung-No Lee ALM 33 17 0 20 Oct 2023
DialogueLLM: Context and Emotion Knowledge-Tuned Large Language Models for Emotion Recognition in Conversations Yazhou Zhang Mengyao Wang Youxi Wu Prayag Tiwari Qiuchi Li Benyou Wang Jing Qin 37 22 0 17 Oct 2023
H2O Open Ecosystem for State-of-the-art Large Language Models Arno Candel Jon McKinney Philipp Singer Pascal Pfeiffer Maximilian Jeblick Chun Ming Lee Marcos V. Conde VLM 30 4 0 17 Oct 2023
On Context Utilization in Summarization with Large Language Models Mathieu Ravaut Aixin Sun Nancy F. Chen Chenyu You 39 13 0 16 Oct 2023
NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails Traian Rebedea R. Dinu Makesh Narsimhan Sreedhar Christopher Parisien Jonathan Cohen KELM 19 133 0 16 Oct 2023
Let's reward step by step: Step-Level reward model as the Navigators for Reasoning Qianli Ma Haotian Zhou Tingkai Liu Jianbo Yuan Pengfei Liu Yang You Hongxia Yang LRM 35 43 0 16 Oct 2023
Self-Detoxifying Language Models via Toxification Reversal Chak Tou Leong Yi Cheng Jiashuo Wang Jian Wang Wenjie Li MU 24 30 0 14 Oct 2023
Can Large Language Model Comprehend Ancient Chinese? A Preliminary Test on ACLUE Yixuan Zhang Haonan Li LRM ELM 16 9 0 14 Oct 2023
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning Jun Chen Deyao Zhu Xiaoqian Shen Xiang Li Zechun Liu Pengchuan Zhang Raghuraman Krishnamoorthi Vikas Chandra Yunyang Xiong Mohamed Elhoseiny MLLM 160 443 0 14 Oct 2023
Learning To Teach Large Language Models Logical Reasoning Meiqi Chen Yubo Ma Kaitao Song Yixin Cao Yan Zhang Dongsheng Li ELM LRM 28 14 0 13 Oct 2023
xDial-Eval: A Multilingual Open-Domain Dialogue Evaluation Benchmark Chen Zhang L. F. D’Haro Chengguang Tang Ke Shi Guohua Tang Haizhou Li ELM 46 9 0 13 Oct 2023
CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Models Sreyan Ghosh Ashish Seth Sonal Kumar Utkarsh Tyagi Chandra Kiran Reddy Evuru S. Ramaneswaran S. Sakshi Oriol Nieto R. Duraiswami Dinesh Manocha AuLLM VLM CoGe 43 23 0 12 Oct 2023
Lemur: Harmonizing Natural Language and Code for Language Agents Yiheng Xu Hongjin Su Chen Xing Boyu Mi Qian Liu ... Siheng Zhao Lingpeng Kong Bailin Wang Caiming Xiong Tao Yu 32 68 0 10 Oct 2023
OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text Keiran Paster Marco Dos Santos Zhangir Azerbayev Jimmy Ba LRM 22 79 0 10 Oct 2023
Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning Mengzhou Xia Tianyu Gao Zhiyuan Zeng Danqi Chen 40 268 0 10 Oct 2023
Let Models Speak Ciphers: Multiagent Debate through Embeddings Chau Pham Boyi Liu Yingxiang Yang Zhengyu Chen Tianyi Liu Jianbo Yuan Bryan A. Plummer Zhaoran Wang Hongxia Yang LLMAG 39 15 0 10 Oct 2023
SCAR: Power Side-Channel Analysis at RTL-Level Amisha Srivastava Sanjay Das Navnil Choudhury Rafail Psiakis Pedro Henrique Silva Debjit Pal Kanad Basu 32 8 0 10 Oct 2023
GPT-who: An Information Density-based Machine-Generated Text Detector Saranya Venkatraman Adaku Uchendu Dongwon Lee DeLMO 32 34 0 09 Oct 2023
LLM for SoC Security: A Paradigm Shift Dipayan Saha Shams Tarek Katayoon Yahyaei S. Saha Jingbo Zhou M. Tehranipoor Farimah Farahmandi 63 46 0 09 Oct 2023
MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning Chengpeng Li Zheng Yuan Hongyi Yuan Guanting Dong Keming Lu Jiancan Wu Chuanqi Tan Xiang Wang Chang Zhou LRM 20 22 0 09 Oct 2023
ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learning Wang Lu Hao Yu Jindong Wang Damien Teney Haohan Wang Yiqiang Chen Qiang Yang Xing Xie Xiangyang Ji 70 8 0 08 Oct 2023
Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU Fajri Koto Nurul Aisyah Haonan Li Timothy Baldwin AI4Ed LRM ELM 32 37 0 07 Oct 2023
ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models Iman Mirzadeh Keivan Alizadeh-Vahid Sachin Mehta C. C. D. Mundo Oncel Tuzel Golnoosh Samei Mohammad Rastegari Mehrdad Farajtabar 126 60 0 06 Oct 2023
MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning Ke Wang Houxing Ren Aojun Zhou Zimu Lu Sichun Luo Weikang Shi Renrui Zhang Linqi Song Mingjie Zhan Hongsheng Li ReLM LRM SyDa 30 95 0 05 Oct 2023
GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction Oscar Sainz Iker García-Ferrero Rodrigo Agerri Oier López de Lacalle German Rigau Eneko Agirre 38 76 0 05 Oct 2023
LLM Based Multi-Document Summarization Exploiting Main-Event Biased Monotone Submodular Content Extraction Litton J. Kurisinkel Nancy F. Chen 21 5 0 05 Oct 2023
JsonTuning: Towards Generalizable, Robust, and Controllable Instruction Tuning Chang Gao Wenxuan Zhang Guizhen Chen Wai Lam 55 5 0 04 Oct 2023
Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models Xianjun Yang Xiao Wang Qi Zhang Linda R. Petzold William Y. Wang Xun Zhao Dahua Lin 26 163 0 04 Oct 2023
Fool Your (Vision and) Language Model With Embarrassingly Simple Permutations Yongshuo Zong Tingyang Yu Ruchika Chavhan Bingchen Zhao Timothy M. Hospedales MLLM AAML LRM 32 18 0 02 Oct 2023
GrowLength: Accelerating LLMs Pretraining by Progressively Growing Training Length Hongye Jin Xiaotian Han Jingfeng Yang Zhimeng Jiang Chia-Yuan Chang Xia Hu 33 11 0 01 Oct 2023
ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving Zhibin Gou Zhihong Shao Yeyun Gong Yelong Shen Yujiu Yang Minlie Huang Nan Duan Weizhu Chen LRM AI4CE LLMAG 61 145 0 29 Sep 2023
Curriculum-Driven Edubot: A Framework for Developing Language Learning Chatbots Through Synthesizing Conversational Data Yu Li Shang Qu Jili Shen Shangchao Min Zhou Yu 52 16 0 28 Sep 2023
Chatmap : Large Language Model Interaction with Cartographic Data Eren Unlu KELM 28 4 0 28 Sep 2023
LORD: Low Rank Decomposition Of Monolingual Code LLMs For One-Shot Compression Ayush Kaushal Tejas Vaidhya Irina Rish 64 15 0 25 Sep 2023
Can LLM-Generated Misinformation Be Detected? Canyu Chen Kai Shu DeLMO 39 158 0 25 Sep 2023
Towards LLM-guided Causal Explainability for Black-box Text Classifiers Amrita Bhattacharjee Raha Moraffah Joshua Garland Huan Liu 32 33 0 23 Sep 2023
From Text to Source: Results in Detecting Large Language Model-Generated Content Wissam Antoun Benoît Sagot Djamé Seddah DeLMO 33 11 0 23 Sep 2023
GlotScript: A Resource and Tool for Low Resource Writing System Identification Amir Hossein Kargaran François Yvon Hinrich Schütze 13 10 0 23 Sep 2023
HANS, are you clever? Clever Hans Effect Analysis of Neural Systems Leonardo Ranaldi Fabio Massimo Zanzotto 34 2 0 21 Sep 2023