BudgetLongformer: Can we Cheaply Pretrain a SotA Legal Language Model From Scratch?

30 November 2022

Papers citing "BudgetLongformer: Can we Cheaply Pretrain a SotA Legal Language Model From Scratch?"

17 / 17 papers shown

Title
Unlocking Legal Knowledge: A Multilingual Dataset for Judicial Summarization in Switzerland Luca Rolshoven Vishvaksenan Rasiah Srinanda Brügger Bose Matthias Sturmer Joel Niklaus ELM AILaw 38 1 0 17 Oct 2024
SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domain Pierre Colombo T. Pires Malik Boudiaf Rui Melo Dominic Culver Sofia Morgado Etienne Malaboeuf Gabriel Hautreux Johanne Charpentier Michael Desa ELM AILaw ALM 35 12 0 28 Jul 2024
SaulLM-7B: A pioneering Large Language Model for Law Pierre Colombo T. Pires Malik Boudiaf Dominic Culver Rui Melo ... Andre F. T. Martins Fabrizio Esposito Vera Lúcia Raposo Sofia Morgado Michael Desa ELM AILaw 41 66 0 06 Mar 2024
Improving Legal Judgement Prediction in Romanian with Long Text Encoders Mihai Masala Traian Rebedea Horia Velicu AILaw 43 2 0 29 Feb 2024
Towards Explainability and Fairness in Swiss Judgement Prediction: Benchmarking on a Multilingual Dataset Santosh T.Y.S.S Nina Baumgartner Matthias Sturmer Matthias Grabmair Joel Niklaus ELM AILaw 36 6 0 26 Feb 2024
Legal-HNet: Mixing Legal Long-Context Tokens with Hartley Transform Daniele Giofré Sneha Ghantasala AILaw 29 0 0 09 Nov 2023
Automatic Anonymization of Swiss Federal Supreme Court Rulings Joel Niklaus Robin Mamié Matthias Sturmer Daniel Brunner Marcel Gygli 31 0 0 07 Oct 2023
MultiLegalPile: A 689GB Multilingual Legal Corpus Joel Niklaus Veton Matoshi Matthias Sturmer Ilias Chalkidis Daniel E. Ho AILaw ELM 23 40 0 03 Jun 2023
MultiLegalSBD: A Multilingual Legal Sentence Boundary Detection Dataset Tobias Brugger Matthias Sturmer Joel Niklaus AILaw 36 8 0 02 May 2023
LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain Joel Niklaus Veton Matoshi Pooja Rani Andrea Galassi Matthias Sturmer Ilias Chalkidis ELM AILaw 19 55 0 30 Jan 2023
Clinical-Longformer and Clinical-BigBird: Transformers for long clinical sequences Yikuan Li R. M. Wehbe F. Ahmad Hanyin Wang Yuan Luo VLM MedIm 147 85 0 27 Jan 2022
LexGLUE: A Benchmark Dataset for Legal Language Understanding in English Ilias Chalkidis Abhik Jana D. Hartung M. Bommarito Ion Androutsopoulos Daniel Martin Katz Nikolaos Aletras AILaw ELM 130 248 0 03 Oct 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 244 644 0 21 Apr 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 280 2,015 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 243 580 0 12 Mar 2020
Neural Legal Judgment Prediction in English Ilias Chalkidis Ion Androutsopoulos Nikolaos Aletras AILaw ELM 112 325 0 05 Jun 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018