US 7,379,077 B2
Augmented and virtual reality guided instrument positioning using along-the-line-of-sight alignment
Ali Bani-Hashemi, Walnut Creek, Calif. (US); Nassir Navab, Plainsboro, N.J. (US); and Frank Sauer, Princeton, N.J. (US)
Assigned to Siemens Corporate Research, Inc., Princeton, N.J. (US)
Filed on Mar. 27, 2001, as Appl. No. 9/818,427.
Prior Publication US 2003/0037449 A1, Feb. 27, 2003
Int. Cl. G90G 5/00 (2006.01); G06T 15/00 (2006.01)
U.S. Cl. 345—633  [345/632] 23 Claims
OG exemplary drawing
 
1. A method for augmented reality guided instrument positioning, comprising the steps of:
defining a target point on or within a real object;
defining a path to reach the target point with a real instrument;
rendering a graphical representation to mark the path in the form of at least one graphical axis marker to be viewed in the direction of the path towards the target on a display overlaid onto a direct optical or video view of a real scene that comprises said real instrument, and said real object, so that a user can perceive said real instrument and said at least one graphical axis marker and their spatial relationship in an augmented reality view, said user performing the following steps:
aligning said user's augmented reality line-of-sight with the at least one graphical axis marker so that said path to reach said target point with the real instrument is aligned along said augmented reality line-of-sight;
aligning the real instrument to the path by aligning the real instrument with the augmented reality line-of-sight towards the target point; and
moving the real instrument along the path towards the actual target point while keeping the real instrument aligned with the augmented reality line-of-sight.