Introducing PLUTO-4: PathAI’s latest State-of-the-Art Foundation Model for Digital Pathology

What is a foundation model, and why does it matter?

Machine learning models work by performing a series of mathematical operations that transform an input (such as an image of a biopsy) to an output (such as whether the biopsy is predicted to contain cancer). Modern digital pathology tools have found great success by breaking this process into a two-step approach:

1. A general-purpose ‘encoder’ pre-digests the input image, distilling its data down into an abstract sequence of numbers known as an embedding.
2. These embeddings can then be fed to more specialized downstream models, such as ones for classifying tissue or identifying different cell types within the image.

The advantage of this approach is efficiency and flexibility. Because the encoder has done the heavy lifting in pre-digesting the input image, downstream models can be more easily trained - requiring less specially-labelled data - compared to the traditional approach of training task-specific models from scratch.

Encoder models trained on massive, diverse datasets can generate embeddings that work well across many different applications. These versatile encoders are known as foundation models.

Meet the PLUTO-4 series

At PathAI, we have built our own pathology-specific foundation model, PLUTO. Our latest version, PLUTO-4, was trained on an even larger and more diverse dataset than our previous versions, encompassing slides stained with hematoxylin and eosin (H&E) – the routine stain of diagnostic pathology – as well as more specialized immunohistochemistry (IHC) and special stains. The training dataset for PLUTO-4 consisted of 551,000 slides covering over 60 disease indications from over 50 source labs.

For additional versatility, we developed two variants: PLUTO-4S and PLUTO-4G. PLUTO-4S is a lightweight, flexible, and fast version with 22 million parameters, ideal for cost-efficient applications. PLUTO-4G is a 1.1 billion parameter model designed for use-cases requiring the best performance.

Benchmarks

We evaluated PLUTO-4 on a range of digital pathology tasks from public benchmarks plus our own test data. These tasks included identification of cell and tissue substances within slides, prediction of spatial transcriptomic signals within slides, and slide-level classification.

PLUTO-4G achieved state-of-the-art performance compared to other recent models. In particular, it shows first-in-class performance in cell segmentation, spatial transcriptomics, and a complex 17-class dermatopathology internal benchmark. PLUTO-4S, despite its much smaller size, also achieved impressive performance results. Read more about PLUTO-4 here.

What’s next for PLUTO?

Over the next few months, we’ll be integrating PLUTO-4 throughout our portfolio to give our customers the best-performing digital pathology products:

• Explore product line for biomarker discovery
• Detect and PathAssist product lines aimed at improving workflow efficiencies
• AIM product line for automated, reproducible biomarker quantification

For instance, the strong performance PLUTO-4 in the dermatopathology benchmark (up to an 11% boost over PLUTO-3) is promising for real-world performance improvements in PathAssist Derm*, our solution for automated prioritization of dermatopathology cases. These exciting results showcase how our PLUTO-4 foundation model can enhance our AI-pathology products in digital diagnostics and translational research.

In follow-up posts, we’ll discuss ways we’ve been using PLUTO internally to accelerate model development.

Table references

1. Benchmarks from eva

2. HEST benchmark

3. PathAI internal dermatopathology benchmark with >2,000 slides. Classes include: actinic keratosis, basal cell carcinoma, benign nevus, cyst, dermatitis, dysplastic nevus, invasive melanoma, lichenoid keratosis, melanoma in situ, scar, seborrheic keratosis, squamous cell carcinoma, squamous cell carcinoma in situ, vascular lesion, verruca vulgaris, other benign non-melanocytic lesion, normal skin. Performance measured by macro F1 score.

4. Atlas foundation model

5. H-optimus-1 foundation model

*PathAssist Derm is For Research Use Only. Not for use in diagnostic procedures.