Collecting large numbers of images from the transmission electron microscope (TEM) can be an arduous task when manual data collection procedures are employed, and operator presence is required for setting key parameters such as beam intensity, eucentric height, defocus, and exposure time for each area imaged in each specimen. Over the last fifteen years, the availability of computerized microscopes has significantly enhanced the level of automation in data collection that is possible once a specimen has been inserted into the microscope. A variety of software packages have been developed for these computerized microscopes which allow automated data collection at both room and cryogenic temperatures (Kisseberth et al., 1997; Oostergetel et al., 1998; Potter et al., 1999; Carragher et al., 2000; Zhang et al., 2001; Lei and Frank, 2005; Suloway et al., 2005). Automated data collection has been especially critical for driving advances in electron tomography, which requires the collection of a series of images at different tilts from the same region of the specimen under well-defined electron optical conditions (Koster et al., 1992; Mastronarde, 2005; Nickell et al., 2005).

As compared to automation of data collection, automated delivery of multiple specimens into an electron microscope has been a more challenging enterprise. An early step in this direction was the development of multi-specimen holders, which allowed as many as 3 specimens to be loaded at one time, but specimen exchange still remained a manual task. A further advance came from the concept of loading specimens at cryogenic temperatures using a cartridge-based system as implemented in the “Polara” microscope manufactured by FEI Company. This construction allows the pre-loading of 6 specimen cartridges into the chamber, which could be maintained at temperatures below –180° C. Cartridges are delivered into the column using a clamp mechanism, and picked up inside the column by a specimen rod-like assembly, into which the cartridge is threaded. Although specimen insertion is still a manual process in these microscopes, the novel aspect of using this cartridge system is the ability to load specimens into the column without operation of the airlock mechanism as is required in the loading of side-entry specimen holders in conventional microscopes. An alternative approach for automated specimen delivery has been reported (Potter et al., 2004) that uses a robotic arm to mimic the movements of a human operator to load specimens into the microscope, similar in concept to the kind of robots used in automobile assembly plants. In this approach, specimen grids are picked up from a 96 position tray using a mechanical caliper, mounted on a specimen holder, and delivered robotically into the microscope column by repetitively carrying out the same sequence of operations carried out by a user.

Here, we report the design, development and implementation of an imaging system that allows fully automated delivery of as many as 100 specimens into the microscope column, and includes fully automated data acquisition from user-identified regions of interest in each specimen. We refer to the overall specimen delivery and imaging system as the “Gatling” because of similarities of the mechanical components in this device to its historical namesake. Operation of the Gatling is based on the use of specimen-containing cartridges designed to accept standard 3 mm grids. The cartridges are housed on a transfer cylinder inside a compact vacuum chamber. The delivery and retrieval of cartridges, as well as all steps in data acquisition are controlled using a modified version of our in-house AutoEM software package that was originally developed for automated data acquisition from single specimens maintained at liquid nitrogen temperatures (Zhang et al., 2001).

The mechanical construction of the Gatling system was carried out at Gatan UK (Abingdon), and most of the software development was carried out at NCI/NIH, Bethesda, MD. Asymmetrically patterned grids (H7 and H4) were obtained from SPI Supplies, West Chester, PA. Operations for cartridge loading and unloading are constructed in a multithreaded program implemented in Microsoft Visual Basic 6.0 (Microsoft, Inc. Redland, WA). A graphical user interface (GUI) or Transmission Control Protocol / Internet Protocol (TCP/IP) server are interfaces to receive input to initiate the loading or unloading operations. The microscope stage is remotely repositioned by sending commands to the microscope computer that is running a TCP/IP server interface to the Tecnai Stage Object, which is implemented in Microsoft Visual Basic 6.0. All of the data presented here was recorded with the Gatling system mounted on a Tecnai12 electron microscope from FEI Company (Portland, OR) equipped with a LaB6 filament operating at 120 kV. Electron micrographs were recorded on a Gatan Ultrascan 4000SP CCD 4kx4k camera. The algorithms for operation of the Gatling and data acquisition were built on the AutoEM platform, while all of the data acquisition routines programs were implemented in DMScript and executed within the context of Digital Micrograph (Gatan Inc., Pleasanton, CA). The complete Gatling system and associated software is available commercially from Gatan Inc.

Figure 1 Design overview of the Gatling. The 3 main components of the Gatling are the linear drive (LD), transfer cylinder (TC), and the motorized cartridge holder (MH). The magnified view of the transfer cylinder shows the spiral arrangement of the cartridge-holding (more ...)

Figure 2 Gatling mounted on TEM. (a) A schematic view of a Tecnai 12 transmission electron microscope that has been equipped with the Gatling, mounted on a side entry port on the left side. No modifications were made to the microscope other than opening this side (more ...)

Inserting a cartridge into the microscope column requires coordinated operation of the linear drive, pick-up tweezers, transfer cylinder, and the motorized cartridge holder. First, the transfer cylinder motor is switched on, rotating the transfer cylinder into the path of the linear drive. The rotations of the transfer cylinder are measured by polling an optical encoder to ensure that the blades of the pick-up “tweezers” are perfectly aligned with the profile of the desired cartridge. When this point is reached, the motor driving the transfer cylinder is halted, leaving the cartridge ready to be retrieved by the pick-up tweezers. Subsequently, the linear drive pushes the tweezers over the top of the cartridge and reverses, extracting the cartridge from the transfer cylinder. An optical switch checks for a cartridge in the tweezers and the transfer cylinder is then driven away from the path of the linear drive. The linear drive pushes the cartridge forward into the microscope column towards the cartridge holder, which is set to be co-axial with the cartridge holder prior to each insertion. A threaded rod at the end of cartridge holder rotates automatically to screw into the back of the cartridge. Once the cartridge is secured, the linear drive is retracted from the column. The same operations are carried out in reverse order to unload the cartridge from the holder and to return it to the Gatling chamber.

We find specimen delivery to be adequately reproducible for most experiments. Translational displacements measured from the images recorded after repeated insertion of the same cartridge show that loading is reproducible to within ~ 1.1 μm for both x- and y-axes regardless of the specimen or cartridge. Larger displacements were observed sometimes, but were almost always the result of operator error in securely mounting the grid on the cartridge. Loosely mounted grids can move randomly during cartridge delivery and retraction, thus resulting in unpredictable translations and rotations. Nevertheless, our software can correctly take these movements into account, and uses an algorithm that relies on computational determination of the absolute orientation of the grid.

Knowledge of the absolute grid orientation is central to our approach for locating regions of interest. Thus, determination of the rotational and translational offsets of the specimen relative to the center of the coordinate system allows each region of interest to be relocated independently of the position of the specimen within the cartridge. Offsets are determined by cross-correlating a low magnification image of the specimen with a reference image that has no rotational and translational offset (figure 3). We chose to use asymmetrically-patterned grids for our experiments because they contain features that dominate the cross-correlation signal over regularly spaced grid bars. The reference is a low magnification (60 X) image of a clean patterned grid that is centered and non-rotated relative to the CCD image frame. Offsets are determined in two steps. First the central feature of the grid is found by using the darkest pixel of a low magnification image of the specimen and the stage is adjusted so that the feature is approximately centered in the CCD image frame. A second low magnification image is then acquired. The image size is reduced and then used for a set of 360 cross-correlations with the reference image at 1° rotational increments, which takes ~ 27 seconds for completion. The highest cross-correlation value is recalled, which corresponds to the rotational offset relative to the centered reference. A final cross-correlation of a full-sized, rotated image of the specimen at low magnification provides the translational offset. This algorithm can determine the rotation of the specimen grid to an accuracy that is within ± 1° and is robust enough to use with specimens that contain dense aggregates. Because these steps are done using low magnification images, translations are accurate to within about 1.4 μm in the x-direction and 3.9 μm in y-direction. The reasons for the different accuracies in x- and y-directions are not yet understood. A second pass of this algorithm at a higher magnification may further reduce these errors, but they have proved to be adequate for our present applications. We have not yet modified the algorithm to manage situations where the grid has been loaded upside-down, is occluded, or warped. In these instances, image acquisition would continue as in the normal case, but the offsets from the reference would not be properly determined. Serial sections and other non-homogenous specimens that require different types of grids (i.e. slotted grid) would require modified algorithms to determine offsets in coordinates starting from the initial low-magnification image.

Figure 3 Determining coordinates using a patterned grid. A low magnification image is acquired after the specimen is delivered to the microscope (top panel). Next, image size is reduced for speed, and rotated by one degree increments for a total of 360 cross-correlations (more ...)

Figure 5 Automated image acquisition using the Gatling and AutoEM. 95 specimen-loaded cartridges were sequentially delivered into the microscope. a) 76 specimens were successfully aligned with the reference image. b) Low magnification images were acquired at 60X (more ...)

Four low magnification images are acquired from each specimen, usually at 60X, for the low magnification atlas. Physical offsets between images are set by augmenting the stage by the length of the CCD image frame so that imaged areas do not overlap, resulting in a 2 x 2 montage of the specimen grid, which in our current implementation covers an area of 1.53 mm x 1.53 mm. Image data is stored along with a log file that uniquely identifies the specimen and provides its horizontal and vertical distances from central feature of the grid. This information is important for recalling a particular position on the specimen in subsequent steps. Image rotations and offsets due to changes in electron optical conditions are handled independently, and do not affect the origin of the coordinate system.

The set of overview images and accompanying log files serve as a map of the specimen used for identifying specific regions of the grid. Using Digital Micrograph, which does not need to reside on the microscope computer, the “Grid Map” script first scans the file system directory given by the user for image and log files that describe each specimen. The set of overview images is displayed so that “region of interest” markers can be drawn on each image and then be rewritten to the directory along with the regions of interest data. We estimate that it takes a novice operator less than 10 seconds to place one marker per image, which for 100 specimens would take just over an hour, as there are 4 images per specimen. If there are no markers in an image file, then this region on the specimen is not revisited later. Upon completing region of interest identification, marker coordinates are written to a text file and are organized so that each is associated with a particular specimen including the distance from center of the reference coordinate axes. The coordinates of the markers are read from the image files and are written into the text file, resulting in less than 225 bytes per marker.

Acquiring data from regions of interest requires that each region can be relocated after the specimen has been redelivered into the microscope. The rotational and translational offsets are determined as described above and are applied to all stage displacements. Initializing the “AutoEM GG” program is similar to “LowMag Survey” except that the operator inputs the desired number of images per region of interest and the defocus used for image acquisition. To start the acquisition process, regions of interest are visited in a specific sequence, beginning from the most negative in the horizontal and vertical axes. Each region of interest has a maximum number of images that may be acquired depending upon the field of view at a particular magnification and the area of the region of interest. Comparing the maximum number of images that can be acquired for the region of interest with the number of images provided by the operator gives the linear distance between images. The incremental distance is calculated by dividing the length and width of the region of interest by number of the desired images, thus achieving even sampling of the region of interest. As each image is acquired, a log file identifying the specimen and the coordinates of acquisition on the specimen grid is written. Multiple iterations using the “Grid Map” and “AutoEM GG” scripts can acquire data from the same location using various magnifications.

As an example of typical use of the Gatling system in our laboratory, we show the results of automated delivery and imaging of 95 specimens. Five positions on the transfer cylinder were intentionally left blank to serve as backup positions. The time required for loading 95 specimens and pumping down the chamber to prepare for specimen delivery was about 3 hours, while the time taken to obtain a low magnification atlas for the entire set of 95 cartridges was ~ 13 hours. The approximate time taken for each of step involved in specimen loading and data acquisition is summarized in Table 1. We tested the delivery and imaging of the same 95 specimens on two separate occasions using identical microscope settings. Out of 190 delivery attempts, all were successfully delivered except for one instance where the pick-up tweezers was unable to pull a cartridge from the transfer cylinder. This failure was detected by the loading system and imaging was able to continue without interruption. In both trials, out of the 95 specimen-loaded cartridges, 76 were successfully aligned to the reference by the automated algorithm (fig. 5a). 19 specimen images were not aligned correctly due to failure in detection of the grid’s central feature. This can happen if the motorized holder is not completely threaded into the back of the cartridge or if the specimen grid is not well-centered in the cartridge. Some of these deficiencies are related to manufacturing defects in individual cartridges, and are easily rectified by replacing the cartridge. The other failures mainly arise from errors in specimen loading and/or wear in cartridge screw threads that result in incorrect positioning of the specimen relative to the center of the cartridge. Once specimens were loaded into cartridges and placed into the transfer cylinder, all subsequent operations, including image viewing and selection of regions of interest were carried out remotely. Several regions were selected from the low magnification atlas (fig. 6b) for automated delivery and higher magnification imaging by the “AutoEM GG” script (fig. 6c).

Table 1 Specimen loading and image acquisition times

While electron microscopy is well recognized as a powerful analytical tool in materials science and in biology, until recently electron microscopes have remained instruments with low data throughput involving substantial user involvement. Modern computerized microscopes allow a high level of automation in data collection from single specimen grids both for 2D and 3D (i.e. tomographic) imaging applications. Here we have demonstrated that a high level of automation can be obtained using the Gatling for imaging multiple specimens. Because the Gatling has been designed to be fully integrated into a commercially available data acquisition package (DigitalMicrograph), data acquisition procedures from as many as 100 specimens are a simple extension of procedures used for data collection from a single specimen. In addition, steps such as cartridge delivery into the microscope and other aspects of microscope operation are also controlled from the same user interface.

A unique feature of the Gatling is that the system is largely independent of microscope brand, since it can be mounted on essentially any modern TEM capable of computerized operation. The presence of the Gatling also does not affect the normal functionality of the microscope on its operation in the single-specimen mode, since the motorized specimen holder is easily exchanged with the conventional side entry holder. Because automated specimen delivery of cartridges avoids operation of the airlock mechanism involved in inserting a specimen rod into the microscope, a better vacuum is maintained in the column throughout the duration of the experiment. Further, since the motorized holder does not need to be removed and re-inserted for each specimen, there is less wear on the O-rings that make contact with the holder during insertion.

The Gatling achieves a result that is similar in some respects to that reported recently (Potter et al., 2004) with the attachment of a robotic loading arm to a conventional TEM. With the robotic loading arm, grid pick-up is automated, while in our case cartridge loading is manual. However, the real differences between multi-specimen imaging based on the robotic arm vs. the Gatling are in the mechanism of specimen delivery into the column. With the robotic arm, the delivery each specimen grid involves an operation that mimics the steps that would be manually executed by a user. In contrast, specimen delivery using the Gatling is done by internal movements within the column vacuum. Because all of the grids are stored in the controlled environment of the chamber, and at column vacuum, any of the hundred specimens can be rapidly retrieved for imaging without concerns related to changes in environmental conditions in the room. The length of time taken for specimen delivery using the Gatling is shorter than that reported with the robotic arm loading system. Thus it takes ~ 16 hours for loading and acquisition of a set of overview images from 95 grids using the Gatling as compared to 28.5 hours for acquiring overview images from 96 specimens using the robotic arm (Potter et al., 2004). We note that image acquisition itself is not a rate-limiting step for obtaining overview maps, since the majority of the time is spent loading and unloading specimens. However, the most distinctive advantage of the Gatling is the automation that is enabled after recording of the overview images, since any of the 100 specimens can be inserted into the microscope for recording an unlimited number of images without any further user intervention.

The availability of the Gatling system provides opportunities for rapid data acquisition is especially valuable in applications that require extensive screening. Applications where we are taking advantage of the Gatling system include screening for two-dimensional crystals of membrane proteins, for imaging immunolabeled sections of cells and tissues, and for imaging a variety of nanoparticles. The increases in data throughput enabled by the Gatling can now be expected to drive further advances in the speed of image processing to take full advantage of the large amounts of data that can be collected. Increasing the linear drive speed and the strength of the transfer cylinder motor can be expected to result in further increases in the speed of specimen loading. Although the data collection scheme we have implemented is for 2D imaging, this approach can be extended easily to record a tilt series at each of the chosen regions of interest, and thus obtain 3D images (i.e. tomograms) from each of these regions. Finally, we anticipate that it will be possible to make changes in the design to accommodate imaging multiple specimens at cryogenic temperatures.

We thank Robin Harmon, Nanda Menon, Paul Miller, and Jacob Wilbrink for valuable assistance with programming and software support; John Henry Scott, Patricia Zerfas, and Chad Smith for providing some of the specimens imaged; Lydia Kibiuk for assistance with illustration; David Simons for assisting in development of the alignment algorithm; and other members of our laboratory for advice and discussions. This work was supported by funds from the intramural program of the National Cancer Institute and the IATAP program, NIH, Bethesda.