About CMB

CMB is a comprehensive, multi-level Medical Benchmark in Chinese. It encompasses 280,839 multiple-choice questions and 74 complex case consultation questions, covering all clinical medical specialties and various professional levels. The platform aims to holistically evaluate a model's medical knowledge and clinical consultation capabilities. The specific components are as follows. You can view and download our dataset on HuggingFace.

Components

• CMB-Exam: Comprehensive multi-level assessment for medical knowledge

  • Structure: 6 major categories and 28 subcategories, View Catalog
  • CMB-test: 400 questions per subcategories, 11200 questions in total
  • CMB-val: 280 questions with solutions and explanations; used as source for CoT and few-shot
  • CMB-train: 269359 questions for medical knowledge injection

• CMB-Clin: 74 cases of complex medical inquires

Data

Our data can be directly downloaded from Huggingface. Please refer to our GitHub for instructions on how to access and utilize the data.

Citation

@article{wang2023cmb,
            title={CMB: A Comprehensive Medical Benchmark in Chinese},
            author={Wang, Xidong and Chen, Guiming Hardy and Song, Dingjie and Zhang, Zhiyi and Chen, Zhihong and Xiao, Qingying and Jiang, Feng and Li, Jianquan and Wan, Xiang and Wang, Benyou and others},
            journal={arXiv preprint arXiv:2308.08833},
            year={2023}
}

@misc{cmedbenchmark,
            title={CMB: Chinese Medical Benchmark},
            author={Xidong Wang*, Guiming Hardy Chen*, Dingjie Song*, Zhiyi Zhang*, Qingying Xiao, Xiangbo Wu, Feng Jiang, Jianquan Li, Benyou Wang},
            note={Xidong Wang, Guiming Hardy Chen, Dingjie Song, and Zhiyi Zhang contributed equally to this github repo.},
            year = {2023},
            publisher = {GitHub},
            journal = {GitHub repository},
            howpublished = {\url{https://github.com/FreedomIntelligence/CMB}},
}

Contact US

For inquiries regarding CMB, please contact us at cmedbenchmark@163.com or create an issue on Github. If you are interested in potential collaboration, please reach out to wangbenyou@cuhk.edu.cn.