Title
MAKIEval: A Multilingual Automatic WiKidata-based Framework for Cultural Awareness Evaluation for LLMs Raoyuan Zhao Beiduo Chen Barbara Plank Michael A. Hedderich 17 0 0 27 May 2025
Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Ahead Jesujoba Oluwadara Alabi Michael A. Hedderich David Ifeoluwa Adelani Dietrich Klakow 51 0 0 27 May 2025
CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesis Ruixiang Feng Shen Gao Xiuying Chen Lisi Chen Shuo Shang 7 0 0 26 May 2025
Evaluating Large Language Model with Knowledge Oriented Language Specific Simple Question Answering Bowen Jiang Runchuan Zhu Jiang Wu Zinco Jiang Yifan He ... Haote Yang Songyang Zhang Dahua Lin Lijun Wu Conghui He ELM 35 0 0 22 May 2025
From Surveys to Narratives: Rethinking Cultural Value Adaptation in LLMs Muhammad Farid Adilazuarda Chen Cecilia Liu Iryna Gurevych Alham Fikri Aji 127 0 0 22 May 2025
WorldView-Bench: A Benchmark for Evaluating Global Cultural Perspectives in Large Language Models Abdullah Mushtaq Imran Taj Rafay Naeem Ibrahim Ghaznavi Junaid Qadir 43 0 0 14 May 2025
An Evaluation of Cultural Value Alignment in LLM Nicholas Sukiennik Chen Gao Fengli Xu Yongqian Li 48 1 0 11 Apr 2025
Surveying Professional Writers on AI: Limitations, Expectations, and Fears Anastasiia Ivanova Natalia Fedorova Sergey Tilga Ekaterina Artemova 49 0 0 07 Apr 2025
Can LLMs Grasp Implicit Cultural Values? Benchmarking LLMs' Metacognitive Cultural Intelligence with CQ-Bench Ziyi Liu Priyanka Dey Zhenyu Zhao Jen-tse Huang Rahul Gupta Yang Liu Jieyu Zhao 58 2 0 01 Apr 2025
The Case for "Thick Evaluations" of Cultural Representation in AI Rida Qadri Mark Díaz Ding Wang Michael Madaio 74 3 0 24 Mar 2025
SaudiCulture: A Benchmark for Evaluating Large Language Models Cultural Competence within Saudi Arabia Lama Ayash Hassan Alhuzali Ashwag Alasmari Sultan Aloufi 69 0 0 21 Mar 2025
LLM Alignment for the Arabs: A Homogenous Culture or Diverse Ones? Amr Keleg 83 1 0 19 Mar 2025
Empowering Smaller Models: Tuning LLaMA and Gemma with Chain-of-Thought for Ukrainian Exam Tasks Mykyta Syromiatnikov Victoria Ruvinskaya Nataliia Komleva ALM LRM 79 0 0 18 Mar 2025
Towards properties of adversarial image perturbations Egor Kuznetsov Kirill Aistov Maxim Koroteev 124 0 0 18 Mar 2025
Uncovering inequalities in new knowledge learning by large language models across different languages Chenglong Wang Haoyu Tang Xiyuan Yang Yueqi Xie Jina Suh ... Junming Huang Yu Xie Zhaoya Gong Xing Xie Fangzhao Wu 85 0 0 06 Mar 2025
Evaluating Polish linguistic and cultural competency in large language models Sławomir Dadas Małgorzata Grębowiec Michał Perełkiewicz Rafał Poświata ELM 68 2 0 02 Mar 2025
Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs Fakhraddin Alwajih Abdellah El Mekki Samar Magdy AbdelRahim Elmadany Omer Nacar ... Anis Koubaa Ismail Berrada Mustafa Jarrar Shady Shehata Muhammad Abdul-Mageed 85 2 0 28 Feb 2025
Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs Longxu Dou Qian Liu Fan Zhou Changyu Chen Zili Wang ... Tianyu Pang Chao Du Xinyi Wan Wei Lu Min Lin 170 2 0 18 Feb 2025
Diffusion Models Through a Global Lens: Are They Culturally Inclusive? Zahra Bayramli Ayhan Suleymanzade Na Min An Huzama Ahmad Eunsu Kim Junyeong Park James Thorne Alice Oh 105 1 0 13 Feb 2025
INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge Angelika Romanou Negar Foroutan Anna Sotnikova Zeming Chen Sree Harsha Nelaturu ... Mike Zhang Imanol Schlag Marzieh Fadaee Sara Hooker Antoine Bosselut ELM 145 8 0 29 Nov 2024
MM-Eval: A Multilingual Meta-Evaluation Benchmark for LLM-as-a-Judge and Reward Models Guijin Son Dongkeun Yoon Juyoung Suk Javier Aula-Blasco Mano Aslan Vu Trong Kim Shayekh Bin Islam Jaume Prats-Cristià Lucía Tormo-Bañuelos Seungone Kim ELM LRM 45 0 0 23 Oct 2024
Can Language Models Reason about Individualistic Human Values and Preferences? Liwei Jiang Taylor Sorensen Sydney Levine Yejin Choi 68 12 0 04 Oct 2024
CulturalBench: A Robust, Diverse, and Challenging Cultural Benchmark by Human-AI CulturalTeaming Yu Ying Chiu Liwei Jiang Bill Yuchen Lin Chan Young Park Shuyue Stella Li ... Mehar Bhatia Maria Antoniak Yulia Tsvetkov Vered Shwartz Yejin Choi ELM ALM 83 22 0 03 Oct 2024
AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMs Basel Mousi Nadir Durrani Fatema Ahmad Md. Arid Hasan Maram Hasanain Tameem Kabbani Fahim Dalvi Shammur A. Chowdhury Firoj Alam 70 8 0 17 Sep 2024
Evaluating Cultural Awareness of LLMs for Yoruba, Malayalam, and English Fiifi Dawson Zainab Mosunmola Sahil Pocker Raj Abhijit Dandekar Rajat Dandekar Sreedath Panat 53 3 0 14 Sep 2024
NativQA: Multilingual Culturally-Aligned Natural Query for LLMs Md. Arid Hasan Maram Hasanain Fatema Ahmad Sahinur Rahman Laskar Sunaya Upadhyay Vrunda N. Sukhadia Mucahid Kutlu Shammur A. Chowdhury Firoj Alam 98 6 0 13 Jul 2024
CulturePark: Boosting Cross-cultural Understanding in Large Language Models Cheng-rong Li Damien Teney Linyi Yang Qingsong Wen Xing Xie Jindong Wang 63 4 0 24 May 2024
Aya 23: Open Weight Releases to Further Multilingual Progress Viraat Aryabumi John Dang Dwarak Talupuru Saurabh Dash David Cairuz ... Aidan Gomez Phil Blunsom Marzieh Fadaee Ahmet Üstün Sara Hooker OSLM 69 82 0 23 May 2024
CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies Weiyan Shi Ryan Li Yutong Zhang Caleb Ziems Chunhua yu R. Horesh Rogerio Abreu de Paula Diyi Yang 63 30 0 23 Apr 2024
IndoCulture: Exploring Geographically-Influenced Cultural Commonsense Reasoning Across Eleven Indonesian Provinces Fajri Koto Rahmad Mahendra Nurul Aisyah Timothy Baldwin LRM 109 16 0 02 Apr 2024
CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean Eunsu Kim Juyoung Suk Philhoon Oh Haneul Yoo James Thorne Alice Oh ELM 100 18 0 11 Mar 2024
Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanese Rifki Afina Putri Faiz Ghifari Haznitrama Dea Adhista Alice Oh 64 18 0 27 Feb 2024
Massively Multi-Cultural Knowledge Acquisition & LM Benchmarking Yi R. Fung Ruining Zhao Jae Doo Chenkai Sun Heng Ji 51 31 0 14 Feb 2024
Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model Ahmet Üstün Viraat Aryabumi Zheng-Xin Yong Wei-Yin Ko Daniel D'souza ... Shayne Longpre Niklas Muennighoff Marzieh Fadaee Julia Kreutzer Sara Hooker ALM ELM SyDa LRM 60 211 0 12 Feb 2024
CultureLLM: Incorporating Cultural Differences into Large Language Models Cheng-rong Li Mengzhou Chen Jindong Wang Sunayana Sitaram Xing Xie VLM 85 19 0 09 Feb 2024
Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting Masahiro Kaneko Danushka Bollegala Naoaki Okazaki Timothy Baldwin LRM 51 30 0 28 Jan 2024
SeaLLMs -- Large Language Models for Southeast Asia Xuan-Phi Nguyen Wenxuan Zhang Xin Li Mahani Aljunied Zhiqiang Hu ... Yue Deng Sen Yang Chaoqun Liu Hang Zhang Li Bing LRM 52 78 0 01 Dec 2023
A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models Yi Zhou Jose Camacho-Collados Danushka Bollegala 109 6 0 19 Oct 2023
Qwen Technical Report Jinze Bai Shuai Bai Yunfei Chu Zeyu Cui Kai Dang ... Zhenru Zhang Chang Zhou Jingren Zhou Xiaohuan Zhou Tianhang Zhu OSLM 174 1,756 0 28 Sep 2023
HAE-RAE Bench: Evaluation of Korean Knowledge in Language Models Guijin Son Hanwool Albert Lee Suwan Kim Huiseo Kim Jaecheol Lee Je Won Yeom Jihyu Jung Jung Woo Kim Songseong Kim RALM ELM 59 22 0 06 Sep 2023
Exploring Cross-Cultural Differences in English Hate Speech Annotations: From Dataset Construction to Analysis Nayeon Lee Chani Jung Jun-Hee Myung Jiho Jin Jose Camacho-Collados Juho Kim Alice Oh 69 17 0 31 Aug 2023
Towards Measuring the Representation of Subjective Global Opinions in Language Models Esin Durmus Karina Nyugen Thomas I. Liao Nicholas Schiefer Amanda Askell ... Alex Tamkin Janel Thamkul Jared Kaplan Jack Clark Deep Ganguli 59 225 0 28 Jun 2023
Having Beer after Prayer? Measuring Cultural Bias in Large Language Models Tarek Naous Michael Joseph Ryan Alan Ritter Wei Xu 60 89 0 23 May 2023
Extracting Cultural Commonsense Knowledge at Scale Shrestha Ghosh Simon Razniewski A. Varde Gerhard Weikum 60 62 0 14 Oct 2022
StereoKG: Data-Driven Knowledge Graph Construction for Cultural Knowledge and Stereotypes Awantee V. Deshpande Dana Ruiter Marius Mosbach Dietrich Klakow 20 11 0 27 May 2022
Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation Kushal Arora Layla El Asri Hareesh Bahuleyan Jackie C.K. Cheung 52 81 0 03 Apr 2022
Challenges and Strategies in Cross-Cultural NLP Daniel Hershcovich Stella Frank Heather Lent Miryam de Lhoneux Mostafa Abdou ... Ruixiang Cui Constanza Fierro Katerina Margatina Phillip Rust Anders Søgaard 74 171 0 18 Mar 2022
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models Nikita Nangia Clara Vania Rasika Bhalerao Samuel R. Bowman 92 667 0 30 Sep 2020
StereoSet: Measuring stereotypical bias in pretrained language models Moin Nadeem Anna Bethke Siva Reddy 71 979 0 20 Apr 2020
The State and Fate of Linguistic Diversity and Inclusion in the NLP World Pratik M. Joshi Sebastin Santy A. Budhiraja Kalika Bali Monojit Choudhury LMTD 81 822 0 20 Apr 2020