Chapter 28: 3D Visualization

There are two broad approaches to drawing volume data itself on screen as a solid. One is surface rendering: extract a surface of a given value (an isosurface) from the volume as a triangle mesh and draw it with shading. The other is volume rendering: without extracting any surface, treat the volume as a translucent cloud and integrate along the line of sight. Where the previous chapter's MPR and MIP flattened the volume onto a plane, both of these draw its shape as a solid directly. We take them in turn.

Marching cubes: extracting an isosurface

The first step of surface rendering is to pull an isosurface out of the volume as a set of triangles. The standard way to do this is marching cubes. Its principle is easiest to see in the 2D version, marching squares. Divide the image into grid cells and, for each cell, check whether the four corners are above or below the threshold. Wherever a corner flips from inside to outside, the contour crosses that edge, so you find the crossing point by linear interpolation and join them. Only edges that straddle the threshold get a crossing point, so the combination of corner states ($2^4=16$ of them) decides how the line segments are drawn.

Three dimensions is just this extended to a cube. A cell has 8 corners and 12 edges, so the corner combinations number $2^8 = 256$. A table tells you, for each case, which edges to cross to form triangles. The crossing points come from linear interpolation on the edges, just as in 2D, and neighboring cells share crossing points, so the result is a closed triangle mesh overall.

Simulation: moving the isocontour

On the central slice of a volume, the isocontour at threshold level is overlaid via marching squares. Move level and the "isosurface" (here a line) you extract changes, switching from the outline around the bone shell to inner structures. The number of line segments is a measure of the surface's complexity.

Surface rendering

The extracted isosurface is a triangle mesh with vertices and normals. Light it and add shading, and it appears as a solid. Because the surface is explicit, rotation and zoom are fast, and measuring area and volume is easy. On the other hand, since it cuts out a surface at one chosen threshold, structures that do not appear at that value (the gentle gradations of soft tissue, for example) are discarded. It suits structures with high contrast against their surroundings, such as bone.

Below, a head-like volume phantom is turned into an isosurface with three.js marching cubes and drawn with shading. Drag to rotate. Raise the threshold and only the bone shell remains; lower it and inner structures are included in the surface.

Volume rendering

Drawing the volume itself without extracting a surface is volume rendering. For each pixel, cast a ray along the line of sight, sample the volume finely, map each sample's value to a color and opacity through a transfer function, and composite from front to back. By designing the transfer function you can choose what to show: bone opaque white, soft tissue faintly translucent, and so on. Its strength is representing the continuum of intensities directly, rather than a single surface at one threshold.

The maximum intensity projection (MIP) of the previous chapter can be seen as a kind of volume rendering that takes the maximum along the ray instead of compositing. The digitally reconstructed radiograph (DRR), which integrates attenuation like an X-ray film, is a relative with a different transfer function and compositing rule.

Surface or volume?

The two approaches are chosen by purpose. Surface rendering yields an explicit mesh, so it is fast and suits uses that need geometry: measurement, 3D printing, surgical simulation. But it discards information by cutting at one threshold. Volume rendering shows the whole continuum of intensities through the transfer function and suits observing soft tissue and faint structures, but it is computationally heavy and its image quality depends strongly on the transfer-function design. Clinical workstations carry both and switch between them by the task.

Key points

Methods for seeing volume data as a solid divide into surface rendering, which extracts an isosurface into a mesh and shades it, and volume rendering, which maps the volume to color and opacity through a transfer function and integrates. Isosurface extraction uses marching cubes, whose principle is the 2D marching squares (linear-interpolated edge crossings from corner states) extended to a cube. Surface rendering is fast and good for measurement; volume rendering shows the whole continuum but is heavy and transfer-function dependent. MIP and DRR are special cases of volume rendering.

References

• Lorensen WE, Cline HE. Marching Cubes: A High Resolution 3D Surface Construction Algorithm. ACM SIGGRAPH Computer Graphics 21, 163–169 (1987) — the original marching cubes paper.
• Levoy M. Display of Surfaces from Volume Data. IEEE Computer Graphics and Applications 8, 29–37 (1988) — the basis of volume rendering.
• Drebin RA, Carpenter L, Hanrahan P. Volume Rendering. ACM SIGGRAPH Computer Graphics 22, 65–74 (1988).
• Fishman EK et al. Volume Rendering versus Maximum Intensity Projection in CT Angiography. RadioGraphics 26, 905–922 (2006).