Pathological diagnosis remains the definitive standard for identifying tumors. The rise of multimodal large models has simplified the process of integrating image analysis with textual descriptions. Despite this advancement, the substantial costs associated with training and deploying these complex multimodal models, together with a scarcity of high-quality training datasets, create a significant divide between cutting-edge technology and its application in the clinical setting. We had meticulously compiled a dataset of approximately 45,000 cases, covering over 6 different tasks, including the classification of organ tissues, generating pathology report descriptions, and addressing pathology-related questions and answers. We have fine-tuned multimodal large models, specifically LLaVA, Qwen-VL, InternLM, with this dataset to enhance instruction-based performance. We conducted a qualitative assessment of the capabilities of the base model and the fine-tuned model in performing image captioning and classification tasks on the specific dataset. The evaluation results demonstrate that the fine-tuned model exhibits proficiency in addressing typical pathological questions. We hope that by making both our models and datasets publicly available, they can be valuable to the medical and research communities.
DepViT-CAD: Deployable Vision Transformer-Based Cancer Diagnosis in Histopathology
University of Padova, Italy(意大利帕多瓦大学);Unit of Surgical Pathology and Cytopathology, Ospedale dell’Angelo, Italy(意大利奥斯佩达莱-安吉洛外科病理学与细胞病理学单位);Pathological Anatomy Unit, Padova University-Hospital, Italy(意大利帕多瓦大学医院病理学解剖单位);Department of Medicine, University of Padova, Italy(意大利帕多瓦大学医学系);Department of Integrated Diagnostics, Azienda Ospedale-Università Padova, Italy(意大利帕多瓦医院-大学整合诊断系)
GasHisSDB: A New Gastric Histopathology Image Dataset for Computer Aided Diagnosis of Gastric Cancer
Northeastern University(东北大学);Cancer Hospital, China Medical University(中国医科大学肿瘤医院);Liaoning Cancer Hospital and Institute(辽宁省肿瘤医院和研究所);Heilongjiang University(黑龙江大学);Stevens Institute of Technology(斯蒂文斯理工学院);Shengjing Hospital, China Medical University(中国医科大学盛京医院);University of Luebeck(吕贝克大学)
Reduced Deep Convolutional Activation Features (R-DeCAF) in Histopathology Images to Improve the Classification Performance for Breast Cancer Diagnosis
Institute of Medical Science and Technology, Shahid Beheshti University(沙希德·贝赫什提大学医学科学与技术研究所)
AWGUNET: Attention-Aided Wavelet Guided U-Net for Nuclei Segmentation in Histopathology Images
Jadavpur University(jadavpur大学);Artificial Intelligence Research Institute, China University of Mining and Technology(中国矿业大学人工智能研究院);Saint Petersburg Electrotechnical University "LETI"(圣彼得堡国立电工技术大学"LETI")
SMILE: a Scale-aware Multiple Instance Learning Method for Multicenter STAS Lung Cancer Histopathology Diagnosis
College of Computer Science and Electronic Engineering, Hunan University(湖南大学计算机与通信工程学院);College of Information Science and Engineering, Hunan Normal University(湖南师范大学信息科学与工程学院);The Second Xiangya Hospital, Central South University(中南大学湘雅二医院)
A High Magnifications Histopathology Image Dataset for Oral Squamous Cell Carcinoma Diagnosis and Prognosis
School of Software Engineering, South China University of Technology(华南理工大学软件工程学院);Department of Oral and Maxillofacial Surgery, Sun Yat-sen Memorial Hospital, Sun Yat-sen University(中山大学口腔颌面外科医院);Mohamed Bin Zayed University of Artificial Intelligence(马尔代夫人工智能大学);School of Computer Science, University of Adelaide(阿德莱德大学计算机科学学院)
Performance of externally validated machine learning models based on histopathology images for the diagnosis, classification, prognosis, or treatment outcome prediction in female breast cancer: A systematic review
McMaster University(麦克马斯特大学);Mayo Clinic(梅奥诊所);William Osler Health System(威廉·奥斯勒医疗系统)
Spatial Transcriptomics Expression Prediction from Histopathology Based on Cross-Modal Mask Reconstruction and Contrastive Learning
University of Regensburg(雷根斯堡大学);Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)(埃尔朗根-纽伦堡弗里德里希-亚历山大大学);Beijing Friendship Hospital, Capital Medical University(首都医科大学附属北京友谊医院);Hanover Medical School(汉诺威医学院);University Clinic Freiburg(弗莱堡大学附属医院);Fraunhofer Institute for Digital Medicine MEVIS(弗劳恩霍夫数字医学研究所)
A Lung Nodule Dataset with Histopathology-based Cancer Type Annotation
School of Computer Science and Technology, Shandong University of Finance and Economics(山东财经大学计算机科学与技术学院);School of Information Science and Technology, Linyi University(临沂大学信息科学与技术学院);Thoracic Surgery Department of Linyi Central Hospital(临沂市中心医院胸外科);Personnel Department of Linyi Central Hospital(临沂市中心医院人事科)
Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosis
Broad Institute of Harvard and MIT(哈佛大学与麻省理工学院博德研究所);Dana-Farber Cancer Institute(丹娜法伯癌症研究院);Department of Biomedical Informatics, Harvard Medical School(哈佛医学院生物医学信息学系)
A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Images
University of British Columbia(不列颠哥伦比亚大学);Eindhoven University of Technology(埃因霍温理工大学);Massachusetts General Hospital(马萨诸塞州综合医院);Harvard Medical School(哈佛医学院);BC Cancer Agency(不列颠哥伦比亚癌症机构)
MM-NeuroOnco: A Multimodal Benchmark and Instruction Dataset for MRI-Based Brain Tumor Diagnosis
Guangdong Institute of Intelligence Science and Technology(广东智能科学与技术研究院);Center for Brain-Inspired Computing Research (CBICR), Department of Precision Instrument, Tsinghua University(脑启发计算研究中心(CBICR)、精密仪器系,清华大学)
Prompt-Free Lightweight SAM Adaptation for Histopathology Nuclei Segmentation with Strong Cross-Dataset Generalization
School of Information and Communication Technology, Griffith University, QLD, Australia(信息与通信技术学院,格里菲斯大学,昆士兰,澳大利亚);School of MDP -- Clinical Medicine, Griffith University, QLD, Australia(MDP临床医学学院,格里菲斯大学,昆士兰,澳大利亚)
PathFinder: A Multi-Modal Multi-Agent System for Medical Diagnostic Decision-Making Applied to Histopathology
University of Washington(华盛顿大学);Microsoft Research(微软研究院);David Geffen School of Medicine, UCLA(加州大学洛杉矶分校大卫·格芬医学院);Allen Institute for AI (AI2)(艾伦人工智能研究所)
Clinically-Informed Modeling for Pediatric Brain Tumor Classification from Whole-Slide Histopathology Images
Dept. of Computer Science(计算机科学系);University of Texas at Austin(德克萨斯大学奥斯汀分校);School of Information(信息学院);Dell Children's Medical Center(德尔儿童医学中心);Dept. of OREI(OREI部门);University of Nevada, Reno(内华达大学里诺分校)
LDCSF: Local depth convolution-based Swim framework for classifying multi-label histopathology images
College of Computer Science and Electronic Engineering(计算机科学与电子工程学院);Third Xiangya Hospital(湘雅三医院);Hunan University(湖南大学);Central South University(中南大学)
A Chain-of-thought Reasoning Breast Ultrasound Dataset Covering All Histopathology Categories
Peking University(北京大学);Peking Union Medical College Hospital(北京大学人民医院);Peking University Cancer Hospital & Institute(北京大学肿瘤医院);National Cancer Center / National Clinical Research Center for Cancer / Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College(中国医学科学院国家癌症中心/国家癌症临床科研中心/癌症医院);Shenzhen Maternity & Child Health Care Hospital(深圳妇幼保健医院);Xi’an Jiaotong University(西安交通大学);Yizhun Medical AI Co., Ltd(亿众医疗人工智能有限公司)
Multispectral Spatial Characterization: Application to Mitosis Detection in Breast Cancer Histopathology
University Joseph Fourier(约瑟夫·傅里叶大学);IPAL, CNRS UMI 2955(IPAL,法国国家科学研究中心UMI 2955);CoSMo Software(CoSMo软件公司);Temasys Communications(Temasys通信公司);University Pierre and Marie Curie(皮埃尔和玛丽·居里大学)
Feature-interactive Siamese graph encoder-based image analysis to predict STAS from histopathology images in lung cancer
Hunan University(湖南大学);The Second Xiangya Hospital, Central South University(中南大学湘雅二医院);Central South University(中南大学);Hunan Clinical Medical Research Center for Cancer Pathogenic Genes Testing and Diagnosis(湖南省肿瘤致病基因检测与诊断临床医学研究中心);Shanghai Cancer Institute(上海市肿瘤研究所);Renji Hospital, School of Medicine, Shanghai Jiaotong University(上海交通大学医学院附属仁济医院);Shanghai Jiaotong University(上海交通大学)
A Hierarchical Conditional Random Field-based Attention Mechanism Approach for Gastric Histopathology Image Classification
Northeastern University(东北大学);Shenyang Institute of Automation, Chinese Academy of Sciences(中国科学院沈阳自动化研究所);Stevens Institute of Technology(史蒂文斯理工学院);China Medical University(中国医科大学);Liaoning Cancer Hospital and Institute(辽宁省肿瘤医院暨研究所);Chengdu University of Information Technology(成都信息工程大学)
Institute of AI for Health, Helmholtz Munich, Germany International School of Medicine, Istanbul Medipol University, T\"urkiye Munich Leukemia Laboratory, Germany Department of Medicine III, Ludwig-Maximilian-University Hospital, Germany Department of Physics, University of Munich, Germany Munich Center for Machine Learning (MCML), Germany DKTK, German Cancer Consortium, Germany
Learning Disentangled Stain and Structural Representations for Semi-Supervised Histopathology Segmentation
University of Science, VNU-HCM(越南胡志明市科学大学);Ho Chi Minh University of Technology(胡志明技术大学);Vietnam National University(越南国家大学);Carnegie Mellon University(卡内基梅隆大学);Northwestern University(西北大学);College of Engineering & Computer Science, VinUniversity(Vin大学工程与计算机科学学院);VinUni-Illinois Smart Health Center, VinUniversity(Vin大学伊利诺伊智能健康中心)