Chapter 33: Registration

Registration is the process of bringing two or more images into the same coordinate system. It is used to compare images taken at different times (follow-up), to fuse images from different scanners (function and structure in PET/CT, soft tissue and bone in MR/CT), and to align a patient to a standard brain. The PET/CT attenuation correction of Chapter 19 also assumed PET and CT overlapped at the same position.

The registration loop

Registration is posed as an optimization. Apply a transform $T$ to the moving image, measure the similarity $S$ with the fixed image, and iterate, updating $T$ to maximize $S$. There are levels according to the degrees of freedom of the transform. Rigid (rotation and translation, 3 degrees of freedom in 2D) aligns the hard structures of the same patient (the skull, say). Affine (adding scaling and shear) has more freedom, and deformable registration allows motion to vary from place to place, aligning even breathing and organ deformation.

Similarity: what counts as "similar"

The success of registration is decided by the choice of similarity metric.

• SSD (sum of squared differences): sums the squared differences of pixel values; smaller is more similar. It is straightforward and fast for images from the same scanner on the same intensity scale, but useless when the intensity relationship differs.
• NCC (normalized cross-correlation): correlation normalized by mean and variance. Robust to linear intensity differences (brightness and contrast offsets), but weak to nonlinear relationships.
• Mutual information (MI): measures the strength of statistical dependence from the joint histogram of the two images. It asks "once one value is fixed, how much is the other narrowed down." Even with a nonlinear intensity relationship, when the position is aligned the joint histogram concentrates and MI grows large.

Simulation: aligning a shifted image

A fixed image (CT-like) is overlaid with a moving image that has a different-modality look and an unknown offset. Move $t_x, t_y$, and rotation to align them. In the checkerboard on the right, alignment shows as boundaries that continue without a step across tiles. Switch the similarity between SSD, NCC, and MI, and use "auto-align" to search for the transform that maximizes that metric. Because the intensity relationship is nonlinear, confirm that SSD and NCC do not align well while MI aligns cleanly.

Registration in practice

In real clinics, combined scanners such as PET/CT and PET/MR that acquire pre-aligned images are widespread (hardware registration). Even so, software registration remains essential: fusing separately acquired images, differencing follow-ups, normalizing to an atlas, and matching radiotherapy planning to delivery. Deformable registration, and in recent years fast and accurate deep-learning registration, are also in practical use.

Key points

Registration is the optimization that maximizes the similarity between a fixed image and a moving image under a transform $T$. The transform grows in freedom from rigid to affine to deformable. For same-scanner images SSD or NCC suffice, but for multimodal cases with a nonlinear intensity relationship (CT/MR, PET/CT), mutual information (MI), which measures statistical dependence, is needed. PET/CT attenuation correction and the fusion of function and structure rest on this alignment.

References

• Maes F et al. Multimodality Image Registration by Maximization of Mutual Information. IEEE Transactions on Medical Imaging 16, 187–198 (1997).
• Viola P, Wells WM. Alignment by Maximization of Mutual Information. International Journal of Computer Vision 24, 137–154 (1997).
• Maintz JBA, Viergever MA. A survey of medical image registration. Medical Image Analysis 2, 1–36 (1998).
• Sotiras A, Davatzikos C, Paragios N. Deformable Medical Image Registration: A Survey. IEEE Transactions on Medical Imaging 32, 1153–1190 (2013).