LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning

16 July 2020

Leyang Cui

Yue Zhang

Papers citing "LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning"

50 / 76 papers shown

Title
SATBench: Benchmarking LLMs' Logical Reasoning via Automated Puzzle Generation from SAT Formulas Anjiang Wei Yuheng Wu Yingjia Wan Tarun Suresh Huanmi Tan Zhanke Zhou Sanmi Koyejo Ke Wang Alex Aiken ReLM LRM 7 0 0 20 May 2025
RICo: Refined In-Context Contribution for Automatic Instruction-Tuning Data Selection Yixin Yang Qingxiu Dong Linli Yao Fangwei Zhu Zhifang Sui 48 0 0 08 May 2025
QualBench: Benchmarking Chinese LLMs with Localized Professional Qualifications for Vertical Domain Evaluation Mengze Hong Wailing Ng Di Jiang Chen Zhang ELM 55 0 0 08 May 2025
Computational Reasoning of Large Language Models Haitao Wu Zongbo Han Joey Tianyi Zhou Huaxi Huang Changqing Zhang ELM LRM 62 0 0 29 Apr 2025
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks Yixin Cao Shibo Hong Xuzhao Li Jiahao Ying Yubo Ma ... Juanzi Li Aixin Sun Xuanjing Huang Tat-Seng Chua Tianwei Zhang ALM ELM 96 2 0 26 Apr 2025
LLMSR@XLLM25: Less is More: Enhancing Structured Multi-Agent Reasoning via Quality-Guided Distillation Jiahao Yuan Xingzhe Sun Xing Yu Jingwen Wang Dehui Du Zhiqing Cui Zixiang Di LRM 38 0 0 23 Apr 2025
DONOD: Robust and Generalizable Instruction Fine-Tuning for LLMs via Model-Intrinsic Dataset Pruning Jucheng Hu Steve Yang Dongzhan Zhou Lijun Wu 34 0 0 21 Apr 2025
Generative Evaluation of Complex Reasoning in Large Language Models Haowei Lin Xinbing Wang Ruilin Yan Baizhou Huang Haotian Ye Jianhua Zhu Zihao Wang James Zou Jianzhu Ma Yitao Liang ReLM ELM LRM 216 0 0 03 Apr 2025
Advancing MoE Efficiency: A Collaboration-Constrained Routing (C2R) Strategy for Better Expert Parallelism Design Mohan Zhang Pingzhi Li Jie Peng Mufan Qiu Tianlong Chen MoE 50 0 0 02 Apr 2025
MastermindEval: A Simple But Scalable Reasoning Benchmark Jonas Golde Patrick Haller Fabio Barth Alan Akbik LRM ReLM ELM 58 2 0 07 Mar 2025
Predictive Data Selection: The Data That Predicts Is the Data That Teaches Kashun Shum Yuanmin Huang Hongjian Zou Qi Ding Yixuan Liao Xiao Chen Qian Liu Junxian He 67 2 0 02 Mar 2025
ANPMI: Assessing the True Comprehension Capabilities of LLMs for Multiple Choice Questions Gyeongje Cho Yeonkyoung So Jaejin Lee ELM 62 0 0 26 Feb 2025
Reversal Blessing: Thinking Backward May Outpace Thinking Forward in Multi-choice Questions Yizhe Zhang Richard He Bai Zijin Gu Ruixiang Zhang Jiatao Gu Emmanuel Abbe Samy Bengio Navdeep Jaitly LRM BDL 70 1 0 25 Feb 2025
Can LLMs Reason About Program Semantics? A Comprehensive Evaluation of LLMs on Formal Specification Inference Thanh Le-Cong Bach Le Toby Murray LRM 55 1 0 22 Feb 2025
Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale Fan Zhou Zengzhi Wang Qian Liu Junlong Li Pengfei Liu ALM 106 15 0 17 Feb 2025
Logical forms complement probability in understanding language model (and human) performance Yixuan Wang Freda Shi ReLM LRM 79 2 0 13 Feb 2025
MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections Da Xiao Qingye Meng Shengping Li Xingyuan Yuan MoE AI4CE 66 1 0 13 Feb 2025
Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search Maohao Shen Guangtao Zeng Zhenting Qi Zhang-Wei Hong Zhenfang Chen Wei Lu G. Wornell Subhro Das David D. Cox Chuang Gan LLMAG LRM 216 7 0 04 Feb 2025
SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domains Ran Xu Hui Liu Sreyashi Nag Zhenwei Dai Yaochen Xie ... Chen Luo Yang Li Joyce C. Ho Carl Yang Qi He RALM 78 8 0 28 Jan 2025
Merino: Entropy-driven Design for Generative Language Models on IoT Devices Youpeng Zhao Ming Lin Huadong Tang Qiang Wu Jun Wang 83 0 0 28 Jan 2025
Large Language Models as an Indirect Reasoner: Contrapositive and Contradiction for Automated Reasoning Yanfang Zhang Yiliu Sun Yibing Zhan Dapeng Tao Dacheng Tao Chen Gong LRM AI4CE LLMAG 102 2 0 28 Jan 2025
Assessing and Enhancing the Robustness of Large Language Models with Task Structure Variations for Logical Reasoning Qiming Bao Gaël Gendron A. Peng Wanjun Zhong N. Tan Yang Chen Michael Witbrock Qingbin Liu LRM ELM 73 2 0 20 Jan 2025
In-Context Learning with Iterative Demonstration Selection Chengwei Qin Aston Zhang Chong Chen Anirudh Dagar Wenming Ye LRM 70 40 0 31 Dec 2024
MoH: Multi-Head Attention as Mixture-of-Head Attention Peng Jin Bo Zhu Li Yuan Shuicheng Yan MoE 34 13 0 15 Oct 2024
Narrative-of-Thought: Improving Temporal Reasoning of Large Language Models via Recounted Narratives Xinliang Frederick Zhang Nick Beauchamp Lu Wang LRM AI4CE 32 3 0 07 Oct 2024
TypedThinker: Diversify Large Language Model Reasoning with Typed Thinking Danqing Wang Jianxin Ma Fei Fang Lei Li LLMAG LRM 208 0 0 02 Oct 2024
Geometric Signatures of Compositionality Across a Language Model's Lifetime Jin Hwa Lee Thomas Jiralerspong Lei Yu Yoshua Bengio Emily Cheng CoGe 87 1 0 02 Oct 2024
Logic-of-Thought: Injecting Logic into Contexts for Full Reasoning in Large Language Models Tongxuan Liu Wenjiang Xu Weizhe Huang Yuting Zeng Jiaxing Wang Hailong Yang Hailong Yang Jing Li LRM ReLM 52 6 0 26 Sep 2024
Thought-Path Contrastive Learning via Premise-Oriented Data Augmentation for Logical Reading Comprehension Chenxu Wang Ping Jian Zhen Yang LRM 27 0 0 22 Sep 2024
Order Matters in Hallucination: Reasoning Order as Benchmark and Reflexive Prompting for Large-Language-Models Zikai Xie HILM LRM 61 5 0 09 Aug 2024
Learning to Refuse: Towards Mitigating Privacy Risks in LLMs Zhenhua Liu Tong Zhu Chuanyuan Tan Wenliang Chen PILM MU 56 8 0 14 Jul 2024
Retrieved In-Context Principles from Previous Mistakes Hao Sun Yong-jia Jiang Bo Wang Yingyan Hou Yan Zhang Pengjun Xie Fei Huang 63 1 0 08 Jul 2024
RegMix: Data Mixture as Regression for Language Model Pre-training Qian Liu Xiaosen Zheng Niklas Muennighoff Guangtao Zeng Longxu Dou Tianyu Pang Jing Jiang Min Lin MoE 74 41 1 01 Jul 2024
On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models Sree Harsha Tanneru Dan Ley Chirag Agarwal Himabindu Lakkaraju LRM 31 4 0 15 Jun 2024
The CLRS-Text Algorithmic Reasoning Language Benchmark Larisa Markeeva Sean McLeish Borja Ibarz Wilfried Bounsi Olga Kozlova Alex Vitvitskyi Charles Blundell Tom Goldstein Avi Schwarzschild Petar Veličković LRM 38 12 0 06 Jun 2024
Break the Chain: Large Language Models Can be Shortcut Reasoners Mengru Ding Hanmeng Liu Zhizhang Fu Jian Song Wenbo Xie Yue Zhang KELM LRM 36 7 0 04 Jun 2024
Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Models Zachary Ankner Cody Blakeney Kartik K. Sreenivasan Max Marion Matthew L. Leavitt Mansheej Paul 43 24 0 30 May 2024
DGRC: An Effective Fine-tuning Framework for Distractor Generation in Chinese Multi-choice Reading Comprehension Runfeng Lin Dacheng Xu Huijiang Wang Zebiao Chen Yating Wang Shouqiang Liu 44 0 0 29 May 2024
Quantifying In-Context Reasoning Effects and Memorization Effects in LLMs Siyu Lou Yuntian Chen Xiaodan Liang Liang Lin Quanshi Zhang 42 2 0 20 May 2024
HumanRankEval: Automatic Evaluation of LMs as Conversational Assistants Milan Gritta Gerasimos Lampouras Ignacio Iacobacci ALM 32 1 0 15 May 2024
Optimizing Language Model's Reasoning Abilities with Weak Supervision Yongqi Tong Sizhe Wang Dawei Li Yifan Wang Simeng Han Zi Lin Chengsong Huang Jiaxin Huang Jingbo Shang LRM ReLM 42 8 0 07 May 2024
Self-Reflection in LLM Agents: Effects on Problem-Solving Performance Matthew Renze Erhan Guven LRM LLMAG 44 39 0 05 May 2024
PECC: Problem Extraction and Coding Challenges Patrick Haller Jonas Golde Alan Akbik ReLM 40 5 0 29 Apr 2024
LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models Mihir Parmar Nisarg Patel Neeraj Varshney Mutsumi Nakamura Man Luo Santosh Mashetty Arindam Mitra Chitta Baral LRM ReLM ELM 38 24 0 23 Apr 2024
Self-playing Adversarial Language Game Enhances LLM Reasoning Pengyu Cheng Tianhao Hu Han Xu Zhisong Zhang Yong Dai Lei Han Nan Du Nan Du Xiaolong Li SyDa LRM ReLM 98 29 0 16 Apr 2024
DiJiang: Efficient Large Language Models through Compact Kernelization Hanting Chen Zhicheng Liu Xutao Wang Yuchuan Tian Yunhe Wang VLM 31 5 0 29 Mar 2024
The Effect of Sampling Temperature on Problem Solving in Large Language Models Matthew Renze Erhan Guven 58 74 0 07 Feb 2024
Demystifying Chains, Trees, and Graphs of Thoughts Maciej Besta Florim Memedi Zhenyu Zhang Robert Gerstenberger Guangyuan Piao ... Aleš Kubíček H. Niewiadomski Aidan O'Mahony Onur Mutlu Torsten Hoefler AI4CE LRM 75 27 0 25 Jan 2024
Large Scale Foundation Models for Intelligent Manufacturing Applications: A Survey Haotian Zhang S. D. Semujju Zhicheng Wang Xianwei Lv Kang Xu ... Jing Wu Zhuo Long Wensheng Liang Xiaoguang Ma Ruiyan Zhuang UQCV AI4TS AI4CE 29 4 0 11 Dec 2023
GLoRE: Evaluating Logical Reasoning of Large Language Models Hanmeng Liu Zhiyang Teng Ruoxi Ning Jian Liu Qiji Zhou Yuexin Zhang Yue Zhang ReLM ELM LRM 70 8 0 13 Oct 2023