Efficient and Comprehensive Feature Extraction in Large Vision-Language Model for Pathology Analysis

Pathological diagnosis is vital for determining disease characteristics, guiding treatment, and assessing prognosis, relying heavily on detailed, multi-scale analysis of high-resolution whole slide images (WSI). However, existing large vision-language models (LVLMs) are limited by input resolution constraints, hindering their efficiency and accuracy in pathology image analysis. To overcome these issues, we propose two innovative strategies: the mixed task-guided feature enhancement, which directs feature extraction toward lesion-related details across scales, and the prompt-guided detail feature completion, which integrates coarse- and fine-grained features from WSI based on specific prompts without compromising inference speed. Leveraging a comprehensive dataset of 490K samples from diverse pathology tasks, we trained the pathology-specialized LVLM, OmniPath. Extensive experiments demonstrate that this model significantly outperforms existing methods in diagnostic accuracy and efficiency, providing an interactive, clinically aligned approach for auxiliary diagnosis in a wide range of pathology applications.

@article{zhang2025_2412.09521,
  title={ Efficient and Comprehensive Feature Extraction in Large Vision-Language Model for Pathology Analysis },
  author={ Shengxuming Zhang and Weihan Li and Tianhong Gao and Jiacong Hu and Haoming Luo and Xiuming Zhang and Jing Zhang and Mingli Song and Zunlei Feng },
  journal={arXiv preprint arXiv:2412.09521},
  year={ 2025 }
}