Proceedings of the International Federation of Automatic Control
June 27-29, 1995, Boston, MA
The Relationship of Binocular Convergence and Errors in Judged Distance to Virtual Objects
Stephen R. Ellis, Urs J. Bucher and Brian M. Menges
NASA Ames Research Center
Moffett Field CA, 94035
silly@eos.arc.nasa.gov
The full paper may be viewed in PostScript format.
Abstract: Errors in judged depth of nearby virtual objects presented via see-through helmet mounted displays are shown to be linked to changes in binocular vergence. This effect varies measurably across the subjects examined and correlates with the magnitude of the subjects' individual depth judgment errors. The relationship is demonstrated by visually superimposing computer-generated virtual objects on physical backgrounds in a manner similar to that suggested for some practical applications. Suggestions for improved control of virtual objects under these viewing conditions are briefly discussed.
Keywords: see-through display, virtual objects, stereo vision, computer-aided work, binocular convergence
1. Judged Distance to Virtual Objects in the Near Visual Field
The perceptual cues to space have been classically divided into either static or dynamic or classified either as monocular or binocular information sources. More recent analyses of depth perception focusing on the behavioral affordances of vision have usefully reclassified the classical depth cues into three categories, those important with respect to personal space (2m ~ 1-2 eye heights), those relevant for action space (3-30 m ~ 2-20 eye heights), and those relevant for vista space (>30m ~ >20 eye heights). (Cutting and Vishton, in press).
This reclassification of sources of information concerning the spatial layout surrounding a viewer is particularly useful since it focuses attention on what vision is to be used for in each of these distinct regions. Depending on the category of relevance, different cues to depth have varying importance. In particular binocular convergence and accommodation play roles mainly relevant for personal space associated with coordinated, manipulative activity. Understanding the interaction of these physiological responses and depth perception will have growing importance as head mounted displays of virtual objects are introduced into the workplace.
Head or panel mounted see-through displays of conformal, computer generated imagery have been used in aircraft cockpits for many years as heads-up displays (Weintraub & Ensing, 1992) or as helmet mounted sights, i.e. the Honeywell sights. But these applications have almost universally presented users with virtual images at the far end of action space or into vista space. More recent applications of such displays are designed to present to their users spatially conformal computer generated virtual objects for medical and manufacturing applications (Rolland, 1994; Janin, Mizell & Caudell, 1993; Azuma & Bishop, 1994). This work has focused attention on precise calibration of the displays and also on perceptual phenomena that might degrade performance even in well calibrated systems.
Previous reports have indicated that indeed such phenomena are observable. The present two experiments are designed to investigate their causes and practical implications. In particular, previous observations (Ellis & Bucher, 1994) have shown that optical superposition of a virtual object on a physical backdrop changes its judged position. In particular, if a physical surface is introduced at the judged depth of the stereoscopic virtual image constituting the virtual object, the virtual object is judged t o be closer to the observer. This effect is enhanced by slowly moving the physical surface. Since a change in rendering to make the virtual image completely occlude the backdrop did not affect its judged depth and since the motion of the backdrop is like ly to have attracted visual attention and binocular convergence (Ellis & Bucher, 1994), it was concluded that the change in judged depth was not due to the occlusion. Rather, it was suspected that the effect was due to an increase in binocular convergen ce associated with the physical object. The following experiment is a direct test of this hypothesis using an unobtrusive, nonius technique to detect convergence.
2. Measurment of Changes in Static Convergence During Viewing of Virtual Objects
2.1 Methods
Stimuli
A stereoscopic virtual image of an upside down, axially rotating (2 rpm) tetrahedron was presented at a distance of 108 cm away from the subjects' eyes by a head-mounted see-through display. This display was operated under normal room illumination and wa s an improved, previously described device (Ellis & Bucher, 1994). One diopter accommodative relief was provided for the virtual image only. Display resolution was better than 5 arcmin.
The depicted size of the presented virtual object was randomly scaled from 70 to 130% of its nominal size for each trial preventing subjects' use of angular size as a depth cue. The wire-frame tetrahedron had a nominal 10 cm base and 5 cm height. The width of the wire frame lines was about 9 arcmin. The lines of the wire frame an all other computer generated lines had a luminance of about 65 cd/m2. and were seen against a 2.9 cd/m2 gray cloth background placed 2.2 m from the subject. Occasional varia tions in the depicted depth of the tetrahedron which were not analyzed further were also introduced to insure that the subjects did not notice that the same depicted depth was repeated. But these additions proved to be unnecessary due to large and variable, experimentally induced perceptual effects that influenced the judged depth of the target.
A physical surface that was introduced along the line of sight to the tetrahedron was provided by a rotating checkerboard made for foamcore. The checkerboard was a disk 29 cm in diameter with 5 cm black and white checks having either 1.3 cd/m2 or 17.8 cd/m2 luminance.
Subjects
Five men and one woman with stereo resolution of better than 1 arcmin as measured with the stereo vision test on an Orthorater participated in the experiment. Some subjects had vision corrected by contact lenses or glasses and were able to wear their corrections during the experiment. Subjects' ages ranged from late teens to late '40's and included laboratory personnel as well as paid subjects recruited by a contractor at Ames.
Task
The first part of the subject's task was to mechanically place a yellow-green LED (about 20 cd/m2) pointer under the nadir of the slowly rotating, wire- frame tetrahedron, which had a size randomly selected for each trail. After aligning the pointer, the subjects were presented with two sets of nonius lines just flanking the tetrahedron. These lines were then adjusted to equal visual directions on each side of the tetrahedron by moving the lower left and right segments (see Figure 1). The second part of the task involved a second adjustment of the pointer to the tetrahedron's depth after a slowly, irregularly rotating (~2 rpm) opaque checkerboard was introduced along the line of sight to the tetrahedron. The tetrahedron was presented a second time at the same depicted depth in this new configuration but the experimental variations generally concealed this fact from the subjects so that they believed each trial, with or without the checkerboard, involved a potentially different depicted depth.
After the second judgment of the tetrahedron's depth, the nonius lines were flashed briefly (ca 250 msec) next to the tetrahedron while the subjects fixated it. Then the subjects made a forced choice indicating whether the upper or lower pair of the f lashed nonius lines were closer. The eye assignments of each segment of the nonius lines were randomly selected so the that meaning of the alternative possibilities in terms of convergence or divergence varied randomly across the trials. The assignment of the lower part of the left nonius line and the upper part of the right line to one eye and the other upper-lower pair to the other eye, produced a differential effect doubling the relative misalignment for any given vergence change. The subject reported whether the upper or lower segments of the paired nonius lines were closer.
Three different experimental conditions were used. In the "on" condition the checkerboard was mechanically introduced at the judged depth of the virtual tetrahedron object. For the "in front" condition the checker- board was introduced 30 cm in front of the judged depth. In the control condition the second judgment was a replication of the first judgment in that the subject made a second judgment of the depth of the virtual image. But this time instead of aligning the nonius lines, the subject made the forced-choice judgment of the nonius lines configuration without the addition of the checkerboard. The control thus was identical to the experimental conditions except the checkerboard was not introduced into the line of sight. This control, thus, provides a check on the subjects' judgment bias or changes of their convergence during the experiment. Each condition was repeated 15 times for each subject in a randomized block design in which blocks of 5 replications of each condition were repeated. The 6 p ossible orders of the 3 conditions were distributed randomly across the 6 subjects in the experiment.
The change in judged distance of the virtual object, the tetrahedron, was analyzed in a single factor repeated measures ANOVA. Chi-square analyzes were conducted on each individual subject's distribution of judgments of convergence/divergence for each of the 3 experimental conditions. Taking the control condition as a baseline, the relative strength of convergence could be measured by a likelihood ratio of the probability of convergence in each experimental condition divided by the probability of convergence in the control.
2.2 Results
Single factor repeated measures analysis of the effect of superposition of the checkerboard and virtual images replicated the previous observations that the virtual object was moved closer to the viewer (F(2,10)= 7.549 p < 0.01). Individual data are shown in Figure 2. This effect was somewhat stronger for the "on" condition than for the "in front" case and varied in strength across the 6 subjects. One subject interestingly showed essentially no effect.
| Table 1. Frequency of convergence and divergence during depth judgments of virtual objects | |||
|---|---|---|---|
| Converg. | Diverg. | Likelihood ratio of convergence to divergence | |
| On | 84 | 21 | 1.58 |
| In front | 70 | 35 | 1.32 |
| Control | 53 | 52 | -- |
The cause of this individual subject's variation is iluminated by considering all subjects' tendency to reltively converge during judgment of the depth of the virtual object in the presence of the checkerboard. This tendency is summarized for the experime nt in Table 1 which displays the frequency of convergence or divergence indicated by the nonius judgments for all subjects in the three experimental conditions (Chi- square = 20.37, df=2, p< .001). The control case shows an almost perfect 50:50 break wh ile the other two conditions show decided convergence, the "on" condition being somewhat stronger.
Each subject's individual likelihood ratio of converence was computed as the ratio of the probability of convergence in an experimental condition to the probbility of convergence in the control. These ratios are plotted in Figure 2 for each subject. A 2X2 Chi square contingency was computed to compare the distribution of convergence and divergence for each experimental condition to that of the control condiion. This was done separately for each subject. Statistically significant distributions are indicated by asterisks in Figure 2.
2.3 Discussion
The individual subject's results in Figure 2 are sorted by the size of the change in the judged position of the virtual object for the "on" condition. These results can then be compared with the likelihood ratio of relative convergence . As is clear from the figure, the two measurements are almost perfectly correlated across the subjects. The only subject not to show a displacement of the virtual object caused by the checkerboard, also is the only one to show essentially no relative con vergence. The results for the "in front" condition show a weaker apparent displacement of the virtual image but also show a correlation convergence. The correlation of relative convergence with magnitude of displacement for the "on" and "in front" condit ions is across subjects and conditions, r= 0.894, df=10; (t=6.31, p < .002). The results generally support the supposition that the change in judged depth could be due to a change in convergence, but the mechanism underlying this change remains to be clarified.
One possibility is that the effect observed here is the reverse of the inward shift of accommodation with infinity-collimated virtual image displays discussed at length by Roscoe (eg. 1991). In his case users of these displays do not fully accommodate to infinity, but remain focused somewhat closer. The reflex coupling of accommodation and vergence would be expected to change the vergence as well. In the cases we investigate in the present experiment, subjects might be accommodating beyond 1 diopter when the small, relatively poor quality, ~20/100 acuity, virtual object is presented. Their accommodation and vergence could be then brought closer by the insertion of the real surface which provides a much bigger and sharper stimulus to accommodation and to v ergence through the accommodation vergence reflex. Another view of this change may be that a fixation disparity present during viewing of the virtual object is removed by the physical target.
Further experimentation will test these alternatives and examine possible relationships to perspective vergence (Enright, 1991) or proximal vergence (Cuiffreda, 1992) which appear to indicate that higher-level spatial interpretations of the visual image c an simulate the vergence system. The results from the present experiment, while indicating that there is a clear oculomotor response associated with the error in judged depth, do not resolve the question of causation since the vergence change could produc e the change in judged depth through disparity reinterpretation just as easily as a perceptual interpretation of occlusion could produce a vergence response through proximal vergence. This question can only be experimentally resolved if a viewing conditio n can be found that would differentiate the lower level from higher level convergence cues.
The present results were observed with a static eye point and can be expected to change when significant head-movement coupled motion parallax is introduced. But it is important to realize that many of the possible new applications of head-mounted see- th rough displays involve situations in which viewing will be relatively static and for which weight considerations might suggest the use of monocular displays. The results of this experiment indicate that such displays should have a variable focus control a nd probably should be used with a bore-sighting procedure in which focus is adjusted to a reference target so as to correct for any errors in depth due to inappropriate vergence. Alternatively, the computer generated targets used with such displays could be binocularly generated with distorted disparity to correct their spatial perception, pushing them away from the viewer.
References
Azuma, R. and Bishop, G., 1994, Improving static and dynamic registration in an optical see-through HMD, Proceedings of SIGGRAPH '94, July 24-29, Orlando, FL.
Cuiffreda, Kenneth J., 1992, Components of clinicalnear vergence testing, Journal of Behavioral Optometry, 3, 1, 3-13.
Cutting, J. E. & Vishton, P. M., 1995, Perceiving layout and knowing distances: the integration, relative potency an d contextual use of different information about depth, in W. Epstein and S. Rogers (Eds), Handbook of Perception and Cognition, Vol. 5., Academic Press.
Ellis, S. R., 1991, Nature and origin of virtual environments: a bibliographical essay, Computer Systems in Engineering, 2, 4, 321-327.
Ellis, S. R., 1994, What are virtual environments? Computer Graphics and Applications, 14, 1, 17-22.
Ellis, Stephen R. and Bucher, Urs J., 1994, Distance perception of stereoscopically presented virtual objects superimposed by a head mounted see through display, Proceedings, 38th Annual Meeting of the Human Factors and Ergonomics Society, Santa Monica CA.
Enright, J. T., 1991, Paradoxical monocular stereopsis and perspective vergence, in Ellis, S. R., Kaiser, M. K., and Grunwald, A. J. (Eds), Pictorial Communication in Virtual and Real Environments, London: Taylor & Francis.
Foley, J. M., 1993, Stereoscopic distance perception, in Ellis, S. R., Kaiser, M. K., & Grunwald, A. J. (Eds), Pictorial Communication in Virtual and Real Environments, London: Taylor & Francis.
Janin, A.L., Mizell, D.W. and Caudell, T.P., 1993, Calibration of head-mounted displays for augmented reality applications, in Proceedings of IEEE VRAIS '93, Seattle, WA.
Ritter, M., 1977, Effect of disparity and viewing distance on perceived depth, Perception and Psychophysics, 22, 4, 400-407.
Rolland, J. P., Ariely, D. and Gibson, W., 1994. Towards quantifying depth and size perception in 3D virtual environments, to appear in Presence.
Roscoe, S., 1991, The eyes prefer real images, in Ellis, S. R., Kaiser, M. K., & Grunwald, A. J. (Eds), Pictorial Communication in Virtual and Real Environments, London: Taylor & Francis.
Weintraub, D. J. and Ensing, M., 1992, Human Factors Issues in Head-up Display Design: the Book of HUD, Wright Patterson AFB, OH: CSERIAC.
The head mounted display used in this experiment was based on a mechanical design by Ramon Alarcon.
Figure 1. Experimental procedure illustration. Top: alignment, magnification, and interpupilary adjustment, Middle: Pointer and nonius lines adjustment, Bottom: A variety of testing conditions are shown, the "on" or "in front" placement of the rotating checkerboard and the flashed nonius lines.
Figure 2. Results for 6 subjects sorted for magnitude of change in judged distance in the "on" condition.