The Digital Humans CD-ROM is a multimedia exploration of the Visible Human Project intended for the general public. An important part of the CD-ROM was a variety of 3D anatomy reconstructions using both the cryosection and computerized tomography (CT) cross-sectional images. This abstract describes the basic process used in segmenting the raw datasets and the graphics algorithms used in generating the 3D anatomy.

The male dataset was obtained from the NLM ftp site over the course of several weeks, while the female dataset was purchased directly from the University of Colorado in the form of fourteen CD-ROMs. In the case of the male images, reconstruction of coronal and sagittal slices showed registration errors (i.e. offsets in the axial plane) at several points. The raw male images, besides offset problems, also contained background imagery such as the ruler and color bar. In contrast, the female images were obtained after conservative background removal by the Univ. of Colorado group and did not suffer from registration problems on coronal/sagittal reconstruction.

As a first step, the male images were aligned visually using coronal and sagittal reconstructions as guides. In order to remove all background material from the cryosections as fully as possible without removing actual tissue, our algorithms employed color thresholding, mathematical morphology operations (opening/closing) and connected components analysis.

Cryosection images were preprocessed by removing all pixels with green and blue values greater than or equal to the red component. After this initial thresholding, the largest connected component at the center of head images was chosen as our initial region of interest. In order to automatically detect the entire body in 3D, the algorithm iterated from the head cross-sectional images down to the toes, using the region of interest calculated from superior slices to provide seed points for later slices.

Each cross-sectional image was also "cleaned" by application of an opening operation [1] to all pixels which were within 10 pixels of the removed background. This opening operation removed small areas of gelatin which adhered to the skin after thresholding. In the case of the male cadaver, a dynamic programming (DP) boundary detection algorithm (similar to that used in [2]) was used to remove gelatin from the skin for the areas above the chest. While the DP algorithm proved more accurate than simple thresholding/opening procedures in separating gelatin from real tissue in most cases, it was perhaps too sensitive to variations in human tissue color and would remove darker (perhaps necrotic) tissue. The differences between DP boundary detection and thresholding/opening segmentation can be seen in the 3D male cadaver reconstruction (Figure 1).

The brain of the male cadaver in the color cryosection images was segmented using interactive tracing and slice-to-slice propagation and editing of defined regions. While we initially contemplated using multimodal classification methods to semi-automate segmentation of various structures within the body, it became apparent that registration of the MRI, CT, and cryosection images was extremely difficult at the single voxel resolution level.

Skeletal anatomy was segmented using the frozen CT images of the man and the fresh CT images of the woman. In both cases, a simple threshold of 1200 was applied to determine bone. However, there was a fair amount of artifacts, particularly about the rib cage, that manifested itself as high-intensity streaks. Post-processing in the form of an opening operation (using a small element oriented orthogonally to the streaking) was used to reduce the artifacts.

After removal of the background from the cryosections and segmentation of the bone from the CT images, a voxel-based projection algorithm was used to visualize the data [3]. Essentially, three voxel lists were constructed in order to represent the surfaces of the skin, bone, and tissue along cut surfaces. Since only surfaces were used, the visualization system required access to only a small portion of the entire dataset.

Each voxel on a surface had an associated color (derived from either the cryosection color or an ad-hoc designation like white for bone) and a surface normal calculated from its 3D neighborhood. The surface normals were used to calculate the intensity of the associated color using a Phong shading model [4].

Final images were rendered by simply compositing the images of the surfaces using a z-buffer and user-defined alpha values for each surface. The resulting images can show semi-transparent skin derived from the cryosections with registered bone models derived from the CT images (Figure 2).

The 3D models and approximately 10,000 3D images were generated within a three month period using the described segmentation and visualization approach. Most of the 3D images were then processed into a digital movie using the Intel Indeo 3.2 video codec (Figure 3). By linking the multimedia interface of the CD-ROM to specific frames in the digital videos, the Digital Humans interface allows the user to rotate the 3D models in real-time on a household PC. These 3D models were also rendered as stereoscopic views using red and non-red encoding for the left and right eyes, respectively. The combination of stereoscopic viewing and real-time interactive rotation allows users a better appreciation of the 3D nature of the datasets.

FIGURES

Figure 1. Three-dimensional reconstruction of the color cryosection images from the male dataset. The arrow marks the level at which slices were processed using thresholding and opening rather than the dynamic programming boundary detection algorithm.

Figure 2. Image showing three surfaces: a bone model derived from CT images and tissue along cut planes and semi-transparent skin derived from the color cryosectional images.

Figure 3. Spinning 3D man in Intel Indeo 3.2 video format

1. Dougherty ER, An Introduction to Morphological Image Processsing. Bellingham: SPIE, 1992.

2. Geiger D. and Gupta A., "Detecting and Tracking the Left and Right Heart Ventricles via Dynamic Programming", SPIE Image Processing Conference Proceedings, Vol. 2167, 1994; pages 391-402.

3. Katz WT. Semiautomated Segmentation and Display of Three-Dimensional Images of the Head. Ph.D. Dissertation, University of Virginia, May 1994.

4. Rogers DF. Procedural Elements for Computer Graphics. New York: McGraw-Hill, 1985.