AI-powered segmentation and analysis of nuclear morphology predicts genomic and clinical biomarkers in multiple cancer types

Distortion to the nuclear envelope, such as altered size, shape, and morphology, is a common feature of cancer reflective of the underlying hallmark genomic instability. Nuclear morphology is a common visual aid to diagnostic and prognostic pathology. Nuclei can be well-established markers of specific cancers; for example, a clear nucleus (“Orphan Annie Eye”) is a known indication for papillary thyroid carcinoma. Nuclear structure changes
during mitosis, and distorted nuclei can indicate dysregulated replication processes, genetic mutations that affect stability and function of the nuclear
envelope, aneuploidy, and genome instability. Nuclear features have been found to correlate with prognosis in several cancer subtypes.

To enable the use of nuclear morphology in digital pathology, we developed a pan-tissue, deep-learning-based digital pathology pipeline for exhaustive nucleus detection, instance segmentation, and classification on whole-slide hematoxylin and eosin (H&E)-stained pathology images.