MING-MOE: Enhancing Medical Multi-Task Learning in Large Language Models with Sparse Mixture of Low-Rank Adapter Experts

13 April 2024

Papers citing "MING-MOE: Enhancing Medical Multi-Task Learning in Large Language Models with Sparse Mixture of Low-Rank Adapter Experts"

25 / 25 papers shown

Title
Towards Building Multilingual Language Model for Medicine Pengcheng Qiu Chaoyi Wu Xiaoman Zhang Weixiong Lin Haicheng Wang Ya Zhang Yanfeng Wang Weidi Xie LM&MA ELM 62 83 0 21 Feb 2024
HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs Junying Chen Xidong Wang Anningzhe Gao Feng Jiang Shunian Chen ... Chuyi Kong Jianquan Li Xiang Wan Haizhou Li Benyou Wang LM&MA 67 66 0 16 Nov 2023
When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications Qidong Liu Xian Wu Xiangyu Zhao Yuanshao Zhu Derong Xu Feng Tian Yefeng Zheng MoE 70 67 0 21 Oct 2023
Qwen Technical Report Jinze Bai Shuai Bai Yunfei Chu Zeyu Cui Kai Dang ... Zhenru Zhang Chang Zhou Jingren Zhou Xiaohuan Zhou Tianhang Zhu OSLM 251 1,805 0 28 Sep 2023
Baichuan 2: Open Large-scale Language Models Ai Ming Yang Bin Xiao Bingning Wang Borong Zhang Ce Bian ... Youxin Jiang Yuchen Gao Yupeng Zhang Guosheng Dong Zhiying Wu ELM LRM 160 743 0 19 Sep 2023
CMMLU: Measuring massive multitask language understanding in Chinese Haonan Li Yixuan Zhang Fajri Koto Yifei Yang Hai Zhao Yeyun Gong Nan Duan Tim Baldwin ALM ELM 89 259 0 15 Jun 2023
K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization Cheng Deng Tianhang Zhang Zhongmou He Yi Xu Qiyuan Chen ... Weinan Zhang Xinbing Wang Cheng Zhou Zhouhan Lin Junxian He ALM 65 63 0 08 Jun 2023
Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Dataset Junling Liu Peilin Zhou Yining Hua Dading Chong Zhongyu Tian ... Helin Wang Chenyu You Zhenhua Guo Lei Zhu Michael Lingzhi Li LM&MA ELM 76 77 0 05 Jun 2023
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 181 3,117 0 20 Oct 2022
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng Zhang Yuxiao Dong Jie Tang BDL LRM 344 1,091 0 05 Oct 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 471 6,231 0 05 Apr 2022
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts Nan Du Yanping Huang Andrew M. Dai Simon Tong Dmitry Lepikhin ... Kun Zhang Quoc V. Le Yonghui Wu Zhiwen Chen Claire Cui ALM MoE 209 812 0 13 Dec 2021
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 342 1,701 0 15 Oct 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 178 3,743 0 03 Sep 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 398 10,301 0 17 Jun 2021
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark Ningyu Zhang Mosha Chen Zhen Bi Xiaozhuan Liang Lei Li ... Jun Yan Hongying Zan Kunli Zhang Buzhou Tang Qingcai Chen LM&MA ELM 64 185 0 15 Jun 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 540 4,036 0 18 Apr 2021
GPT Understands, Too Xiao Liu Yanan Zheng Zhengxiao Du Ming Ding Yujie Qian Zhilin Yang Jie Tang VLM 160 1,173 0 18 Mar 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 223 4,247 0 01 Jan 2021
Transformer Feed-Forward Layers Are Key-Value Memories Mor Geva R. Schuster Jonathan Berant Omer Levy KELM 149 828 0 29 Dec 2020
MedDG: An Entity-Centric Medical Consultation Dataset for Entity-Aware Medical Dialogue Generation Wenge Liu Jianheng Tang Yi Cheng Wenjie Li Yefeng Zheng Xiaodan Liang LM&MA MedIm 49 41 0 15 Oct 2020
AdapterFusion: Non-Destructive Task Composition for Transfer Learning Jonas Pfeiffer Aishwarya Kamath Andreas Rucklé Kyunghyun Cho Iryna Gurevych CLL MoMe 129 848 0 01 May 2020
Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning Zhaojiang Lin Andrea Madotto Pascale Fung 94 160 0 08 Apr 2020
Medical Exam Question Answering with Large-scale Reading Comprehension Xiao Zhang Ji Wu Zhiyang He Xien Liu Ying Su 50 67 0 28 Feb 2018
Learning multiple visual domains with residual adapters Sylvestre-Alvise Rebuffi Hakan Bilen Andrea Vedaldi OOD 160 933 0 22 May 2017