The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only

1 June 2023

Guilherme Penedo

Quentin Malartic

Daniel Hesslow

Ruxandra-Aimée Cojocaru

Papers citing "The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only"

50 / 587 papers shown

Title
R-Tuning: Instructing Large Language Models to Say `I Don't Know' Hanning Zhang Shizhe Diao Yong Lin Yi R. Fung Qing Lian Xingyao Wang Yangyi Chen Heng Ji Tong Zhang UQLM 42 38 0 16 Nov 2023
Assessing Translation capabilities of Large Language Models involving English and Indian Languages Vandan Mujadia Ashok Urlana Yash Bhaskar Penumalla Aditya Pavani Kukkapalli Shravya Parameswari Krishnamurthy D. Sharma ELM 169 7 0 15 Nov 2023
AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph Zhaowei Wang Haochen Shi Weiqi Wang Tianqing Fang Hongming Zhang Sehyun Choi Xin Liu Yangqiu Song 23 19 0 15 Nov 2023
How Well Do Large Language Models Truly Ground? Hyunji Lee Se June Joo Chaeeun Kim Joel Jang Doyoung Kim Kyoung-Woon On Minjoon Seo HILM 33 6 0 15 Nov 2023
HeLM: Highlighted Evidence augmented Language Model for Enhanced Table-to-Text Generation Junyi Bian Xiaolei Qin Wuhe Zou Mengzuo Huang Congyi Luo Ke Zhang Weidong Zhang LMTD 34 2 0 15 Nov 2023
Large Language Models are legal but they are not: Making the case for a powerful LegalLLM Thanmay Jayakumar Fauzan Farooqui Luqman Farooqui ELM AILaw ALM 30 16 0 15 Nov 2023
Can Large Language Models Follow Concept Annotation Guidelines? A Case Study on Scientific and Financial Domains Marcio Fonseca Shay B. Cohen ALM 27 6 0 15 Nov 2023
Toucan: Token-Aware Character Level Language Modeling William Fleshman Benjamin Van Durme 20 3 0 15 Nov 2023
Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment Philippe Laban Lidiya Murakhovs'ka Caiming Xiong Chien-Sheng Wu LRM 26 19 0 14 Nov 2023
SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models Bertie Vidgen Nino Scherrer Hannah Rose Kirk Rebecca Qian Anand Kannappan Scott A. Hale Paul Röttger ALM ELM 35 27 0 14 Nov 2023
MC $^2$ : Towards Transparent and Culturally-Aware NLP for Minority Languages in China Chen Zhang Mingxu Tao Quzhe Huang Jiuheng Lin Zhibin Chen Yansong Feng 30 2 0 14 Nov 2023
A Survey of Confidence Estimation and Calibration in Large Language Models Jiahui Geng Fengyu Cai Yuxia Wang Heinz Koeppl Preslav Nakov Iryna Gurevych UQCV 41 56 0 14 Nov 2023
SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models Ziyi Lin Chris Liu Renrui Zhang Peng Gao Longtian Qiu ... Siyuan Huang Yichi Zhang Xuming He Hongsheng Li Yu Qiao MLLM VLM 33 210 0 13 Nov 2023
It's Not Easy Being Wrong: Large Language Models Struggle with Process of Elimination Reasoning Nishant Balepur Shramay Palta Rachel Rudinger LRM 28 7 0 13 Nov 2023
On Measuring Faithfulness or Self-consistency of Natural Language Explanations Letitia Parcalabescu Anette Frank LRM 74 22 0 13 Nov 2023
Towards the Law of Capacity Gap in Distilling Language Models Chen Zhang Dawei Song Zheyu Ye Yan Gao ELM 38 20 0 13 Nov 2023
In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering Sheng Liu Haotian Ye Lei Xing James Y. Zou 26 86 0 11 Nov 2023
Intentional Biases in LLM Responses Nicklaus Badyal Derek Jacoby Yvonne Coady 22 4 0 11 Nov 2023
BizBench: A Quantitative Reasoning Benchmark for Business and Finance Rik Koncel-Kedziorski Michael Krumdick Viet Dac Lai Varshini Reddy Charles Lovering Chris Tanner AIMat 35 4 0 11 Nov 2023
Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Models Shahriar Golchin Mihai Surdeanu 29 24 0 10 Nov 2023
How to Bridge the Gap between Modalities: Survey on Multimodal Large Language Model Shezheng Song Xiaopeng Li Shasha Li Shan Zhao Jie Yu Jun Ma Xiaoguang Mao Weimin Zhang 71 4 0 10 Nov 2023
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions Lei Huang Weijiang Yu Weitao Ma Weihong Zhong Zhangyin Feng ... Qianglong Chen Weihua Peng Xiaocheng Feng Bing Qin Ting Liu LRM HILM 47 732 0 09 Nov 2023
Conic10K: A Challenging Math Problem Understanding and Reasoning Dataset Haoyi Wu Wenyang Hui Yezeng Chen Weiqi Wu Kewei Tu Yi Zhou LRM 45 4 0 09 Nov 2023
Prompt Cache: Modular Attention Reuse for Low-Latency Inference In Gim Guojun Chen Seung-seob Lee Nikhil Sarda Anurag Khandelwal Lin Zhong 42 76 0 07 Nov 2023
A Survey of Large Language Models Attribution Dongfang Li Zetian Sun Xinshuo Hu Zhenyu Liu Ziyang Chen Baotian Hu Aiguo Wu Min Zhang HILM 21 49 0 07 Nov 2023
Ziya2: Data-centric Learning is All LLMs Need Ruyi Gan Ziwei Wu Renliang Sun Junyu Lu Xiaojun Wu ... Ping Yang Qi Yang Hao Wang Jiaxing Zhang Yan Song VLM ALM 23 16 0 06 Nov 2023
DeepInception: Hypnotize Large Language Model to Be Jailbreaker Xuan Li Zhanke Zhou Jianing Zhu Jiangchao Yao Tongliang Liu Bo Han 50 152 0 06 Nov 2023
AI-TA: Towards an Intelligent Question-Answer Teaching Assistant using Open-Source LLMs Yann Hicke Anmol Agarwal Qianou Ma Paul Denny AI4Ed 42 24 0 05 Nov 2023
Citance-Contextualized Summarization of Scientific Papers S. Syed Ahmad Dawar Hakimi Khalid Al Khatib Martin Potthast 27 5 0 04 Nov 2023
Indicative Summarization of Long Discussions S. Syed Dominik Schwabe Khalid Al Khatib Martin Potthast 27 1 0 03 Nov 2023
SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency Jiaxin Zhang Zhuohang Li Kamalika Das Bradley Malin Kumar Sricharan HILM LRM 24 56 0 03 Nov 2023
Market Concentration Implications of Foundation Models Jai Vipra Anton Korinek ELM 40 16 0 02 Nov 2023
ChineseWebText: Large-scale High-quality Chinese Web Text Extracted with Effective Evaluation Model Jianghao Chen Pu Jian Tengxiao Xi Yidong Yi Qianlong Du Chenglin Ding Guibo Zhu Chengqing Zong Jinqiao Wang Jiajun Zhang 40 7 0 02 Nov 2023
Learn to Refuse: Making Large Language Models More Controllable and Reliable through Knowledge Scope Limitation and Refusal Mechanism Lang Cao 26 13 0 02 Nov 2023
Attention Alignment and Flexible Positional Embeddings Improve Transformer Length Extrapolation Ta-Chung Chi Ting-Han Fan Alexander I. Rudnicky 22 4 0 01 Nov 2023
Theory of Mind in Large Language Models: Examining Performance of 11 State-of-the-Art models vs. Children Aged 7-10 on Advanced Tests Max J. van Duijn Bram van Dijk Tom Kouwenhoven Werner de Valk M. Spruit P. V. D. Putten ELM LRM 28 28 0 31 Oct 2023
Which Examples to Annotate for In-Context Learning? Towards Effective and Efficient Selection Costas Mavromatis Balasubramaniam Srinivasan Zhengyuan Shen Jiani Zhang Huzefa Rangwala Christos Faloutsos George Karypis 19 21 0 30 Oct 2023
Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on Understanding Bram van Dijk Tom Kouwenhoven M. Spruit Max J. van Duijn 32 19 0 30 Oct 2023
Skywork: A More Open Bilingual Foundation Model Tianwen Wei Liang Zhao Lichang Zhang Bo Zhu Lijie Wang ... Yongyi Peng Xiaojuan Liang Shuicheng Yan Han Fang Yahui Zhou 38 93 0 30 Oct 2023
Efficient kernel surrogates for neural network-based regression S. Qadeer A. Engel Amanda A. Howard Adam Tsou Max Vargas P. Stinis Tony Chiang 21 5 0 28 Oct 2023
T5 meets Tybalt: Author Attribution in Early Modern English Drama Using Large Language Models Rebecca M. M. Hicke David M. Mimno 29 3 0 27 Oct 2023
A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications Ahmed Magooda Alec Helyar Kyle Jackson David Sullivan Chad Atalla ... Eslam Kamal Federico Zarfati Hanna Wallach Sarah Bird Mei Chen 16 12 0 26 Oct 2023
Exploring Question Decomposition for Zero-Shot VQA Zaid Khan B. Vijaykumar S. Schulter Manmohan Chandraker Yun Fu ReLM 17 10 0 25 Oct 2023
The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI Shayne Longpre Robert Mahari Anthony Chen Naana Obeng-Marnu Damien Sileo ... K. Bollacker Tongshuang Wu Luis Villa Sandy Pentland Sara Hooker 32 56 0 25 Oct 2023
Exploring Large Language Models for Code Explanation Paheli Bhattacharya Manojit Chakraborty Kartheek N S N Palepu Vikas Pandey Ishan Dindorkar Rakesh Rajpurohit Rishabh Gupta 32 10 0 25 Oct 2023
Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models Paul Youssef Osman Alperen Koracs Meijie Li Jorg Schlotterer Christin Seifert KELM 24 16 0 25 Oct 2023
MindLLM: Pre-training Lightweight Large Language Model from Scratch, Evaluations and Domain Applications Yizhe Yang Huashan Sun Jiawei Li Runheng Liu Yinghao Li Yuhang Liu Heyan Huang Yang Gao ALM LRM 16 8 0 24 Oct 2023
Fighting Fire with Fire: The Dual Role of LLMs in Crafting and Detecting Elusive Disinformation Jason Samuel Lucas Adaku Uchendu Michiharu Yamashita Jooyoung Lee Shaurya Rohatgi Dongwon Lee 32 42 0 24 Oct 2023
FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions Hyunwoo J. Kim Melanie Sclar Xuhui Zhou Ronan Le Bras Gunhee Kim Yejin Choi Maarten Sap LLMAG 29 80 0 24 Oct 2023
DoGE: Domain Reweighting with Generalization Estimation Simin Fan Matteo Pagliardini Martin Jaggi 32 33 0 23 Oct 2023