Pathology Foundation Models Plus Semi-Supervised Curation Detect Perineural Invasion in Colon, Prostate and Pancreatic H&E Slides

Two foundation models, two different error profiles

The team benchmarked four backbones on the same 262-slide baseline. UNI, a 307-million-parameter ViT-L/16 pretrained on 100,426 diagnostic H&E slides, came out strongest on first pass with an F1 of 0.826 and 2.22 false positives per slide. Virchow-2, a 632-million-parameter ViT-H trained on 3.1 million slides, scored 0.740 and 5.64 false positives per slide, with the damage concentrated in pancreatic tissue at 10.4 per slide, where desmoplastic stroma kept being called PNI. The two ImageNet models did not come close: EfficientNet-B3 reached 0.360 and ConvNeXt-2 0.339, and in pancreas both produced over 600 false positives per slide at comparable sensitivity.

The authors read the gap between the two foundation models as a pretraining-composition effect, not a scale effect, since UNI has roughly half the parameters and a corpus about 30 times smaller. Colon made up about 7% of Virchow-2's training data, prostate 6.5% and pancreas under 1%, by the paper's account.

Where the 90 extra slides went

The semi-supervised loop trained a teacher on the 150 expert-annotated slides, ran it over 90 unannotated slides, and kept only patches where the top class probability exceeded 0.9. A pathologist then reviewed the high-probability PNI patches, removed false positives from structures that mimic nerve, and re-annotated the benign class to include morphologies the original set under-represented. Annotation count grew from 2,390 to 2,680 bounding boxes and annotated pixels from 24 billion to 56 billion. All slides were tiled into 224 by 224 patches at 20x and 5x, and every model ran a two-stage protocol in which a second-stage binary classifier filtered first-stage false positives.

An ablation arm that used the pseudo-labels without human curation isolates the two contributions for Virchow-2. Pseudo-labels alone moved F1 from 0.740 to 0.804 and false positives from 5.64 to 3.80 per slide. Curation took it the rest of the way to 0.832 and 2.51. The paper attributes 70% of the F1 gain to data volume but only 59% of the false-positive reduction, so the curated benign class did disproportionate work on specificity. UNI, which already handled benign mimics well, barely moved (0.826 to 0.829), though its pancreatic false positives fell from 2.93 to 0.40 per slide.

Pancreas gained most, prostate least

Per-organ FROC analysis puts numbers on the pattern. Virchow-2 in pancreas went from an F1 of 0.597 to 0.781 with an 82% fall in false positives per slide. Colon moved from 0.760 to 0.817 with false positives down from 2.73 to 1.93, consistent with partial curation of crypt mimics, and slides with dense crypts showed the largest baseline-to-SSL improvement across every architecture. Prostate, where Virchow-2 already scored 0.866, edged up to 0.875 with false positives unchanged at 3.7 per slide. The authors write that "the challenge in pancreatic tissue was not data scarcity but the absence of representative desmoplastic stroma in the benign class," and they note that PNI occurs in 70 to 100% of pancreatic ductal adenocarcinoma, so nerve and desmoplasia co-occur in almost every slide.

Beyond PNI, the same pipeline classifies tumor and nerve. Tumor detection was near ceiling for both foundation models (F1 0.952 for Virchow-2 and 0.954 for UNI). Nerve was the hardest class for every backbone: UNI held 0.810 under SSL, Virchow-2 slipped to 0.756, and the two CNNs produced 22.9 and 40.4 nerve false positives per slide.

No external cohort yet

Every number above comes from the PAIP 2021 and WISEPAIP material, and the holdout test set of 45 slides was drawn from the same expert-annotated pool. The authors state that "external validation on independent multi-institutional cohorts is needed to confirm generalisability across centres," and they list expansion to further organ types and prospective clinical evaluation as future work. They compare their 0.832 with the top PAIP 2021 challenge score of 0.597 while conceding that the challenge scored pixel-level segmentation on 150 training slides, so the two figures are not like for like. The code is public at github.com/AhmadAlkhan/PNI_SSL; the slides themselves require a PAIP data use agreement. Funding came from the University of Limerick Research Office, Breakthrough Cancer Research, the Government of Ireland's Shared Island Fund and two EU projects, and the authors declare no conflicts.

Why This Matters to the APO|APE Reader

The paper cites a 26% inter-observer discordance on difficult PNI cases and routine colorectal reporting rates of 16% against 22% on expert re-review, which is the gap any automated detector would be judged on. Its practical lesson for anyone building on Virchow-2 or UNI is that the annotation budget should go toward the benign structures a given backbone confuses, and that those blind spots differ by model and by organ. A prostate biopsy service would see little change from this pipeline; a pancreatic resection service, where the baseline model produced ten false calls per slide, would see the difference between an unusable tool and a screenable one.