Adapting to Virtual Environments and Teleoperator Devices

Principal Investigator: Robert B. Welch, Ph.D.

Co-investigators:

Stephen Ellis, Ph.D. NASA Ames Research Center
Bernard Adelstein, Ph.D. NASA Ames Research Center
Anthony C. Sampanes, Doctoral Candidate, U.C., Santa Cruz

NASA is committed to the use of virtual environments (VEs) and teleoperator devices (e.g., a robotic arm) for training, simulation, and other tasks necessary in aerospace environments. Unfortunately, much of this “interactive technology” suffers from defects that can cause misperception, performance errors, and motion sickness-like symptoms. Many of these flaws and their perceptual and behavioral consequences will be with us for a long time and some may never be resolved. Thus, they represent an obstacle which operators must learn to overcome or circumvent in order to make the most efficient use of these devices. Two such problems are (1) a mismatch between control actions and their visual consequences and (2) a reduced visual field of view.

1. Adapting to an altered relationship between stylus and cursor movements

This research project is currently being funded by a Code UL grant (#131-20-30). Our general goal is to examine human adaptability to an analogue of a VE or teleoperator device in which there is a transformation between bodily movements and visual feedback A “real world” example would be if an astronaut’s turn of a joystick to the right caused the robotic arm to move off into space at an oblique angle. In our laboratory analogue of such a situation, in-out movements of a stylus on a horizontal digitizing pad cause a cursor, viewed on a monitor, to move off to one side (e.g., 40 deg clockwise), rather than in its usual up-and-down direction.

As of August 2002, we have completed four studies examining the ability of human subjects to adapt to this sensorimotor “rearrangement.” These experiments demonstrated the following:

1. Experiment 1: A “kinesthetic cue” orienting the subject’s unseen left hand in the direction of the imposed rotation (45 deg clockwise or counterclockwise) facilitated adaptation of the right hand with respect to the reduction of errors, but not for the postexposure negative aftereffect (see Figure 1)
2. Experiments 2-3: Attempts to increase the size of the negative aftereffect by simplifying the sensory rearrangement and providing terminal visual feedback failed. This result led us to conclude that our situation was producing visual-motor skill acquisition, rather than perceptual recalibration, the latter being subject to negative aftereffects in contrast to the former.
3. Experiment 4: Reducing the size of the rotation from 45 deg to 15 deg and placing the monitor parallel (rather than orthogonal) to the digitizing tablet, produced apparent perceptual recalibration, as demonstrated by substantial negative aftereffects. Figure 2 shows the difference in negative aftereffect between Experiments 1-3 and Experiment 4. Once again, the kinesthetic cue had no effect on the negative aftereffect.
4. Future experiments: Our next experiments will examine (a) the role of the kinesthetic cue in visual-motor skill acquisition versus perceptual recalibration, (b) confirmation of the distinction between visual-motor skill acquisition versus perceptual recalibration according to the criteria of the presence versus absence of negative aftereffects, shifts in felt limb position, and intermanual transfer, and (c) the role of the “unity assumption” in perceptual recalibration.

2. Identifying and adapting to the visual and visual-motor effects of a reduced field of view

The use of VE technology for space flight training and other purposes should, ideally, involve lightweight, energy-efficient head-mounted displays (HMDs). However, in an attempt to preserve pixel resolution, current commercial HMDs must entail a relatively small field of view (FOV). Whereas the normal FOV approaches 200 deg, that presented by the typical VR system is about 75 deg or less. This visual restriction is known to disrupt visual perception and visual-motor coordination, with potentially serious consequences for task performance. The goals of this research program are to (1) systematically quantify the deleterious effects of reduced FOV on vision and performance in the context of the FOVs that characterize currently available HMDs, (2) determine the role of adaptation as a response to--and countermeasure for--these effects, and (3) identify the underlying basis for this adaptation and its response to variables known to influence adaptation to visual distortions in general. The results should indicate whether the problems caused by a reduced FOV warrant its increase in future HMDs and provide the necessary guidelines and specifications for such a redesign.

So far, two studies have been completed on this currently unfunded project. In the first, we demonstrated that viewing objects by means of a very small FOV (14 deg of visual angle) reduces their apparent size (see Figure 3). In a second study, we found that the small FOV increased errors in hand-eye coordination (see Figure 4).

Future research will examine the role of head orientation and movement on hand-eye coordination errors in the presence of various sized FOVs and the use of error-corrective feedback as a means of overcoming these errors by means of adaptation.