Chapter 32: Segmentation

Segmentation extracts the region of interest from an image. It separates bone, organs, tumors, or defects from the background and other tissues, labeling each pixel with "what it is." It underpins many applications: measuring volume and area, 3D display (the isosurfaces of Chapter 28 are often built from a segmentation), and target definition for radiotherapy.

Thresholding and Otsu's method

When values differ clearly from tissue to tissue, as with CT numbers, the simplest segmentation is thresholding: above a value is foreground, below is background. The question is how to choose the threshold. When the histogram splits into two humps (foreground and background), Otsu's method automatically picks the threshold that maximizes the variance between the two classes (the between-class variance). That corresponds to the boundary that best separates foreground from background.

Cleaning up with morphology

A mask made from a threshold alone is usually rough. Noise leaves isolated specks in the foreground, and small holes open up in regions that should be connected. Morphology operations clean these up. On a binary mask, using a structuring element (a small window), they perform:

• Erosion: keep foreground only where the whole window is foreground. It thins objects and removes thin spurs and isolated specks.
• Dilation: set foreground if any pixel in the window is foreground. It fattens objects and fills small holes and gaps.
• Opening: erode then dilate. Removes specks and thin spurs while roughly keeping the size.
• Closing: dilate then erode. Fills small holes and gaps while roughly keeping the size.

The cleaned mask can be measured by connected-component labeling: "how many blobs" and "how large is each." This is the same flow as counting low-density regions above a threshold as defects in the non-destructive testing of Chapter 7.

Simulation: extract and clean

A phantom with an organ, bone, noise, and salt-and-pepper specks is thresholded to a binary mask. The "Otsu" button picks the threshold automatically. The threshold-only mask (middle) still has noise specks, but applying morphological "opening" (right) removes the specks and keeps only the bone blobs. Watch the number of connected components and the foreground area. Moving the threshold changes which tissue is captured.

When thresholding is not enough

Thresholding is powerful when tissue values do not overlap, but it cannot separate tissues of similar value (soft tissue from soft tissue). For those, other methods are used: region growing, which grows a connected region of similar intensity from a seed pixel; active contours (snakes, level sets), which evolve a contour along edges; graph cuts, which treat pixels as graph nodes and find a minimum cut; and, in recent years, deep learning (U-Net and the like), which learns how to distinguish regions from large amounts of hand-labeled data. U-Net is the segmentation version of the convolutional network of Chapter 12 and has become the standard for medical image segmentation. Classical methods are transparent and fast; learned methods are strong on complex boundaries.

Key points

Segmentation labels an image with "what each region is." Tissues of clearly different value can be extracted by thresholding, and Otsu's method automatically picks the threshold that maximizes between-class variance. A mask left rough by thresholding is cleaned by erosion, dilation, opening, and closing, and connected components measure the number and area of blobs. For tissues of similar value, more advanced methods are used: region growing, active contours, graph cuts, and deep learning (U-Net).

References

• Otsu N. A Threshold Selection Method from Gray-Level Histograms. IEEE Transactions on Systems, Man, and Cybernetics 9, 62–66 (1979).
• Serra J. Image Analysis and Mathematical Morphology. Academic Press (1982) — the framework of morphology.
• Boykov Y, Jolly MP. Interactive Graph Cuts for Optimal Boundary & Region Segmentation. Proc. ICCV 105–112 (2001).
• Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI 234–241 (2015).