A clinical multimodal vision language foundation model with fine-grain explainability

Despite their considerable promise, many current medical foundation models remain modality specific and lack transparent reasoning, limiting clinical adoption. To address this gap, we introduce Explainable Vision Language Foundation Model for Medicine (EVLF-FM), a multimodal vision language foundation model that combines broad diagnostic capabilities with fine-grained explainability. The development and testing of EVLF-FM encompassed more than 1.3 million total samples from 18 global datasets across 10 imaging modalities related to six clinical specialties: dermatology, hepatology, ophthalmology, pathology, pulmonology and radiology. External validation employed 13,443 independent test samples from 17 additional datasets across 10 imaging modalities. EVLF-FM supports disease diagnosis, visual question answering and pixel-level visual grounding with reasoning capabilities. In internal validation for disease diagnostics, EVLF-FM achieved the highest average accuracy, 0.895, outperforming leading generalist and specialist models. In medical visual grounding, EVLF-FM also achieved reasonable performance across nine modalities with average accuracy at 0.5 intersection over union (Acc@0.5) of 0.842. External testing further confirmed consistent zero-shot and few-shot performance, with competitive F1 scores despite a smaller model size. A hybrid training strategy combining supervised and visual reinforcement fine-tuning enables EVLF-FM to generate clinically interpretable explanations aligned with visual evidence. Human evaluation of these reasoning outputs using the S.C.O.R.E. framework yielded a high overall mean score of 4.25/5, supporting their clinical interpretability. Current limitations include the need for additional validation across diverse healthcare settings. Prospective clinical studies are required to evaluate the safety, effectiveness and clinical utility of EVLF-FM before routine clinical deployment.

Yang Bai, Haoran Cheng, Yang Zhou, Jun Zhou, Arun Thirunavukarasu, Yuhe Ke, Jie Yao, Kanae Fukutsu, Chrystie Wan Ning Quek, Ashley Shuen Ying Hong, Laura Gutierrez, Alessandro Berni, Zhen Ling Teo, Darren Shu Jeng Ting, Brian T Soetikno, Christopher S Nielsen, Tobias Elze, Zengxiang Li, Sumit Sharma, Nicholas Ruthmann, Le Dinh Linh, Hiok Hong Chan, Victor Koh, Marcus Tan, Kelvin Z Li, Leonard Yip, Ching Yu Cheng, Yih Chung Tham, Gavin Siew Wei Tan, Leopold Schmetterer, et al.

Leverage4 / 5

A general-purpose tool used across several fields (Adam, ResNet, LoRA, next-generation sequencing).

10% of this desk’s rubric

Magnitude3 / 5

Large gain: roughly 2x, or a clear new state of the art on a hard, unsaturated problem.

20% of this desk’s rubric

Evidence4 / 5

Strong: large scale, preregistered, independently replicated, or a well-powered randomized trial.

32% of this desk’s rubric

Novelty3 / 5

A genuinely new approach to an open problem.

8% of this desk’s rubric

Trajectory3 / 5

A clear path to scale.

5% of this desk’s rubric

Stakes2 / 5

Benefits a professional community (practitioners, clinicians, engineers).

25% of this desk’s rubric

• Citations0 (reference 20, via openalex, Oct 31, 2026, 06:17 UTC)

• Reviewed by heuristic-v10 on Oct 31, 2026, 06:17 UTC. Paper type: method.
• Categories: Journal Article
• BRIEF, No.3 in the Artificial Intelligence edition of October 31, 2026.