Augmented Reality Basics
Augmented Reality is used to project virtual projects into the real world in contrast to Virtual Reality which build an entirely virtual space. Therefore it consists of the following components:
- Real and virtual data source (Camera, Models)
- Registration of virtual environments with real world spaces (Hand-Eye-Calibration)
- Display technology to combine the virtual and the real environments (Video-See-Through HMD)
Neuronavigation
A key component in neurosurgery is to properly navigate inside the brain. Therefore previously acquired data is fused with operative imaging data to provide orientation and track the surgical instruments. The problem thus far was that the surgeon had to look away from the operation to inspect the image data und then transfer this information back to the patient. In neurosurgery augmented reality neuronavigation may present a significant advantage, providing a real-time updated 3D virtual model of anatomical details, overlaid on the real surgical field. This chapter tries to give an overviewof augmented reality applications in neurosurgery. More applications can be found in [1].
Microscope AR
According to 1) most of the time the additional information is a surgical microscope where the 3D projections are overlayed into the bilateral eyepieces of the binocular optics. A paper that investigates the effect of glioma surgery with AR is 2). After registering the microscope a reasonable incision line was delineated.
[2]
Trajectory Planning
An important aspect for minimally invasive image guided keyhole surgery is the correct planning of the trajectory, which currently is done from 2D Slices. Navab et. al. developed an AR system that displays multiple trajectories computed from algorithms onto an head through Video-See-Through HMD . The surgeon can then display blood vessels and other risk structures to pick the optimal trajectory. Furthermore the selected trajectory can then be checked in the corresponding 2D Slice. This approach was evaluated by two senior surgeons and showed significant benefits with respect to the usual method.
[3]
Augmented Reality Goggles
Another Approach for AR with Optical-See-Through Goggles is described by 4). They are using a headmounted tracker thus getting rid of external cameras. Their focus was to investigate the feasibility of implementing a head-mounted tracking system with an augmented reality environment to provide the surgeon
[4]
VR and AR Simulators for neurosurgical Training
Surgery is more and more switching from an Halsted approach of training ("see one, do one, teach one") to VR and AR applications to better prepare young surgeons without risking a patients life. 5) developed a simulator for placing external ventricular drains, which are often misplaced by young surgeons. Again this is most of the times connected to wrong trajectory planning. Their system uses the NeuroTouch simulator combined with a tablet that overlays specific renderings onto the head. After performing the trajectory a feedback is rendered comparing the users touch to the ideal one.
[5]
Bibliography
- Meola, Antonio, et al. "Augmented reality in neurosurgery: a systematic review." Neurosurgical review (2016): 1-12.
- Guo-chen Sun et. al. "Impact of Virtual and Augmented Reality Based on Intraoperative Magnetic Resonance Imaging and Functional Neuronavigation in Glioma Surgery Involving Eloquent Areas"
- Nassab et. al. "Trajectory planning with Augmented Reality for improved risk assessment in image-guided keyhole neurosurgery"
- Azimi et. al. "Augmented Reality Goggles with an Integrated Tracking System for Navigation in Neurosurgery"
- Ribaupierre et.al. "VR and AR Simulator for Neurosurgical Training"