The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only

1 June 2023

Guilherme Penedo

Quentin Malartic

Daniel Hesslow

Ruxandra-Aimée Cojocaru

Papers citing "The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only"

50 / 587 papers shown

Title
High-quality Data-to-Text Generation for Severely Under-Resourced Languages with Out-of-the-box Large Language Models Michela Lorandi Anya Belz 6 5 0 19 Feb 2024
Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs Jiejun Tan Zhicheng Dou Yutao Zhu Peidong Guo Kun Fang Ji-Rong Wen 49 25 0 19 Feb 2024
Unveiling the Magic: Investigating Attention Distillation in Retrieval-augmented Generation Zizhong Li Haopeng Zhang Jiawei Zhang RALM 53 1 0 19 Feb 2024
Structured Chain-of-Thought Prompting for Few-Shot Generation of Content-Grounded QA Conversations M. Sultan Jatin Ganhotra Ramón Fernández Astudillo LRM 32 3 0 19 Feb 2024
Large Language Models for Stemming: Promises, Pitfalls and Failures Shuai Wang Shengyao Zhuang Guido Zuccon 41 1 0 19 Feb 2024
BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocation Peng Xu Wenqi Shao Yonghong Tian Shitao Tang Kai-Chuang Zhang Peng Gao Fengwei An Yu Qiao Ping Luo MoE 35 27 0 18 Feb 2024
Can Deception Detection Go Deeper? Dataset, Evaluation, and Benchmark for Deception Reasoning Kang Chen Zheng Lian Haiyang Sun Bin Liu Jianhua Tao 42 0 0 18 Feb 2024
AbsInstruct: Eliciting Abstraction Ability from LLMs through Explanation Tuning with Plausibility Estimation Zhaowei Wang Wei Fan Qing Zong Hongming Zhang Sehyun Choi Tianqing Fang Xin Liu Yangqiu Song Ginny Wong Simon See 54 13 0 16 Feb 2024
Where is the answer? Investigating Positional Bias in Language Model Knowledge Extraction Kuniaki Saito Kihyuk Sohn Chen-Yu Lee Yoshitaka Ushiku 66 2 0 16 Feb 2024
PAL: Proxy-Guided Black-Box Attack on Large Language Models Chawin Sitawarin Norman Mu David Wagner Alexandre Araujo ELM 32 29 0 15 Feb 2024
Copyright Traps for Large Language Models Matthieu Meeus Igor Shilov Manuel Faysse Yves-Alexandre de Montjoye 36 18 0 14 Feb 2024
LLMAuditor: A Framework for Auditing Large Language Models Using Human-in-the-Loop Maryam Amirizaniani Jihan Yao Adrian Lavergne Elizabeth Snell Okada Aman Chadha Tanya Roosta Chirag Shah HILM 34 2 0 14 Feb 2024
AuditLLM: A Tool for Auditing Large Language Models Using Multiprobe Approach Maryam Amirizaniani Elias Martin Tanya Roosta Aman Chadha Chirag Shah 26 2 0 14 Feb 2024
Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues Zhiyuan Chang Mingyang Li Yi Liu Junjie Wang Qing Wang Yang Liu 94 38 0 14 Feb 2024
SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding Zhangchen Xu Fengqing Jiang Luyao Niu Jinyuan Jia Bill Yuchen Lin Radha Poovendran AAML 131 88 0 14 Feb 2024
Eliciting Personality Traits in Large Language Models Airlie Hilliard Cristian Muñoz Zekun Wu Adriano Soares Koshiyama 11 7 0 13 Feb 2024
Pushing The Limit of LLM Capacity for Text Classification Yazhou Zhang Mengyao Wang Chenyu Ren Qiuchi Li Prayag Tiwari Benyou Wang Jing Qin VLM AI4TS 70 23 0 12 Feb 2024
Sentinels of the Stream: Unleashing Large Language Models for Dynamic Packet Classification in Software Defined Networks -- Position Paper Shariq Murtuza 29 1 0 10 Feb 2024
Large Language Models: A Survey Shervin Minaee Tomáš Mikolov Narjes Nikzad M. Asgari-Chenaghlu R. Socher Xavier Amatriain Jianfeng Gao ALM LM&MA ELM 134 375 0 09 Feb 2024
Large Language Models to Enhance Bayesian Optimization Tennison Liu Nicolás Astorga Nabeel Seedat M. Schaar 66 48 0 06 Feb 2024
ReLU $^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs Zhengyan Zhang Yixin Song Guanghui Yu Xu Han Yankai Lin Chaojun Xiao Chenyang Song Zhiyuan Liu Zeyu Mi Maosong Sun 22 31 0 06 Feb 2024
A Survey on Transformer Compression Yehui Tang Yunhe Wang Jianyuan Guo Zhijun Tu Kai Han Hailin Hu Dacheng Tao 41 29 0 05 Feb 2024
Evading Data Contamination Detection for Language Models is (too) Easy Jasper Dekoninck Mark Niklas Muller Maximilian Baader Marc Fischer Martin Vechev 99 18 0 05 Feb 2024
DeAL: Decoding-time Alignment for Large Language Models James Y. Huang Sailik Sengupta Daniele Bonadiman Yi-An Lai Arshit Gupta Nikolaos Pappas Saab Mansour Katrin Kirchoff Dan Roth 64 29 0 05 Feb 2024
KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache Zirui Liu Jiayi Yuan Hongye Jin Shaochen Zhong Zhaozhuo Xu Vladimir Braverman Beidi Chen Xia Hu MQ 36 163 0 05 Feb 2024
DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averaging Matteo Pagliardini Amirkeivan Mohtashami F. Fleuret Martin Jaggi 40 6 0 04 Feb 2024
NetLLM: Adapting Large Language Models for Networking Duo Wu Xianda Wang Yaqi Qiao Zhi Wang Junchen Jiang Shuguang Cui Fangxin Wang 40 30 0 04 Feb 2024
Zero-shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Sentiment Lexicon Fajri Koto Tilman Beck Zeerak Talat Iryna Gurevych Timothy Baldwin 60 7 0 03 Feb 2024
On Catastrophic Inheritance of Large Foundation Models Hao Chen Bhiksha Raj Xing Xie Jindong Wang AI4CE 56 12 0 02 Feb 2024
OLMo: Accelerating the Science of Language Models Dirk Groeneveld Iz Beltagy Pete Walsh Akshita Bhagia Rodney Michael Kinney ... Jesse Dodge Kyle Lo Luca Soldaini Noah A. Smith Hanna Hajishirzi OSLM 141 363 0 01 Feb 2024
Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning Ming Li Yong Zhang Shwai He Zhitao Li Hongyu Zhao Jianzong Wang Ning Cheng Dinesh Manocha 35 69 0 01 Feb 2024
CroissantLLM: A Truly Bilingual French-English Language Model Manuel Faysse Patrick Fernandes Nuno M. Guerreiro António Loison Duarte M. Alves ... François Yvon André F.T. Martins Gautier Viaud C´eline Hudelot Pierre Colombo 58 32 0 01 Feb 2024
When Large Language Models Meet Vector Databases: A Survey Zhi Jing Yongye Su Yikun Han Bo Yuan Haiyun Xu Chunjiang Liu Kehai Chen Min Zhang 61 36 0 30 Jan 2024
H2O-Danube-1.8B Technical Report Philipp Singer Pascal Pfeiffer Yauhen Babakhin Maximilian Jeblick Nischay Dhankhar Gabor Fodor SriSatish Ambati VLM 29 8 0 30 Jan 2024
InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model Xiao-wen Dong Pan Zhang Yuhang Zang Yuhang Cao Bin Wang ... Conghui He Xingcheng Zhang Yu Qiao Dahua Lin Jiaqi Wang VLM MLLM 89 245 0 29 Jan 2024
Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling Pratyush Maini Skyler Seto Richard He Bai David Grangier Yizhe Zhang Navdeep Jaitly SyDa 46 55 0 29 Jan 2024
LLaMandement: Large Language Models for Summarization of French Legislative Proposals Joseph Gesnouin Yannis Tannier Christophe Gomes Da Silva Hatim Tapory Camille Brier ... Emmanuel Cortes Pierre-Etienne Devineau Ulrich Tan Esther Mac Namara Su Yang AILaw 44 8 0 29 Jan 2024
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models Bin Lin Zhenyu Tang Yang Ye Jiaxi Cui Bin Zhu ... Jinfa Huang Junwu Zhang Yatian Pang Munan Ning Li-ming Yuan VLM MLLM MoE 43 153 0 29 Jan 2024
Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting Masahiro Kaneko Danushka Bollegala Naoaki Okazaki Timothy Baldwin LRM 37 27 0 28 Jan 2024
Genie: Achieving Human Parity in Content-Grounded Datasets Generation Asaf Yehudai Boaz Carmeli Y. Mass Ofir Arviv Nathaniel Mills Assaf Toledo Eyal Shnarch Leshem Choshen 45 22 0 25 Jan 2024
Large Language Models are Efficient Learners of Noise-Robust Speech Recognition Yuchen Hu Chen Chen Chao-Han Huck Yang Ruizhe Li Chao Zhang Pin-Yu Chen Ensiong Chng 27 20 0 19 Jan 2024
MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learning Chenyu Wang Weixin Luo Qianyu Chen Haonan Mai Jindi Guo Sixun Dong Xiaohua Xuan MLLM LLMAG 52 19 0 19 Jan 2024
EmoLLMs: A Series of Emotional Large Language Models and Annotation Tools for Comprehensive Affective Analysis Zhiwei Liu Kailai Yang Tianlin Zhang Qianqian Xie Sophia Ananiadou 28 39 0 16 Jan 2024
A Study on Large Language Models' Limitations in Multiple-Choice Question Answering Aisha Khatun Daniel G. Brown ELM 35 13 0 15 Jan 2024
Are self-explanations from Large Language Models faithful? Andreas Madsen Sarath Chandar Siva Reddy LRM 30 25 0 15 Jan 2024
DocFinQA: A Long-Context Financial Reasoning Dataset Varshini Reddy Rik Koncel-Kedziorski Viet Dac Lai Michael Krumdick Charles Lovering Chris Tanner RALM 29 16 0 12 Jan 2024
INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning Yutao Zhu Peitian Zhang Chenghao Zhang Yifei Chen Binyu Xie Zheng Liu Ji-Rong Wen Zhicheng Dou 21 15 0 12 Jan 2024
PersianMind: A Cross-Lingual Persian-English Large Language Model Pedram Rostami Ali Salemi M. Dousti CLL LRM 32 5 0 12 Jan 2024
AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters L. Lucy Suchin Gururangan Luca Soldaini Emma Strubell David Bamman Lauren Klein Jesse Dodge 28 15 0 12 Jan 2024
Zero-shot Generative Large Language Models for Systematic Review Screening Automation Shuai Wang Harrisen Scells Shengyao Zhuang Martin Potthast Bevan Koopman Guido Zuccon 33 12 0 12 Jan 2024