4) Augmented Reality

Introduction

In the past 5 years, Augmented Reality (AR) and Virtual Reality (VR) have both started appearing regularly in the media on a regular basis regarding its potential benefits to improve quality of life ranging from shopping to surgery. However, most people still do not know what the difference really is between AR and VR and thus assume it is the same thing. So what is the difference between these two?

Virtual Reality

Virtual Reality is computer generated graphics that is used to simulate any sort of real life environment or situation. What a user sees through VR is completely simulated and does not take into consideration their actual physical location. This is done by simulating not just the visual aspect but also the vocal aspect to generate a "real life like experience". VR is typically achieved by wearing a head mounted device that the user looks into and the user can look into any direction as if they were inside that environment. [1]

Augmented Reality

Augmented Reality uses existing reality and lays over computer generated enhancement graphics on-top of it in order to make it more meaningful for users. Augmented Reality can be implemented in various ways such as inside a mobile device app where the camera can relays the "real world" and a software adds an overlay of computer generated graphics on top of it to make a more meaningful video. The applications of Augmented Reality are becoming more and more popular and one can see them in every day life such as on american football games where the 1st yellow line is painted onto the field and players can walk over it. [1]

Virtual reality offers a digital recreation of a real life setting, while augmented reality delivers virtual elements as an overlay to the real world. [1]

Augmented Reality Applications In Medicine

Augmented reality is not just for catching Pokemon any more. It can be seen in various applications including military, industrial, commercial, entertainment and most importantly medical field. It is used in hospitals around to world to both treat patients and lower the amount of X-Ray exposure required.

Anatomical Evaluation

Modeling structures in the real world using augmented reality is becoming a more common application in the last several years, particularly to educate students in anatomy that are in the medical field. This is done by using clothing that has pre-patterned codes on it using a camera. When the camera software detects the codes, and then super imposes animated anatomical images of the internal organs on the person. This is a relatively new method and a whole new way of teaching and interaction with the only constraints being the computing power that is currently available. Augmented reality can also be used to create virtual vascular endoscopys to generate pre-operative vascular imaging for planning purposes. There are also similar benefits in the visualization of complex hand and lower limb fractures following 3D CT or MRI. [3]

Recently, people have also been using augmented reality to evaluate dynamic anatomy in real time by using ultrasound. This is done by taking a reference points on the Doppler probe which is then superimposed onto the subject using a head mounted display to view it. This process is particularly beneficial for the visualization of structures and blood flow whilst performing invasive procedures. [6]

Augmented reality can also help aid students to learn by superimposing CT and MRI images on to the human body thus creating a view of the spatial anatomy. Additionally, the use of touch sense technology along with augmented reality provides users touch feedback in order to be able to differentiate different tissues. This new touch sense technology is a very exciting step forward in the development of anatomical education. [6]

Curiscope T-shirt with visualization of animated internal organs [2]

Operative Benefits

Augmented Reality has also been used extensively in neurosurgical procedures in the last decade. The use of pre-operative imaging to detect suitable vessels for extracranial-intracranial bypass allows for image injection into the operator’s microscope to guide intra-operative dissection. [5] .

CaptiView Is a recently released technology from Leica Micro-Systems that uses image injection to display virtual overlays from data taken from a microscope in order to help guide Neurosurgeons during a surgery. This system is able to link image-guided surgery software to the hardware (such as a microscope) that is being used. This allows it to lay down critical visual information such as images of a brain scan right on top of a patient's actual brain that is able to help the surgeon. Using this method, both 2-D and 3-D images are able to be injected. Captiview is also capable of tracking both the microscope as well as the focal points of the eyepieces while updating it in real time. It is able to provide real time navigation to the surgeon that will make these complicated procedures safer and more efficient. The surgeons/operators are also able to switch in real time between live feed coming from the patients as well as the preoperative images. [7]

Virtual Reality Operating Room [2]

Limitations Of Augmented Reality

Technological Limitations

With advances seen in augmented reality in the last several years, it is pretty evident that AR will play an important role in the surgical environment in the upcoming years. As the functionalities and image details keep increasing, the amount of computational power required to render these images and videos have also increased.

In order for the AR devices to be a natural extension of the doctor rather than bulky equipment that needs to be lugged around during a surgery, the limitations in battery life and large cables also need to be addressed. For this reason, as technology advances and these issues are able to be addressed, AR tools will be more common in hospitals and clinical facilities. [2]

Ethical Issues

Confidentiality in the medical field has been a big concern for a long time as the information on record is usually very sensitive. With the introduction of electronic patient records and now the integration of recordable head mounted displays in medical practice, people in the medical field have larger obligations to protect the data of patients while reviewing, storing, and transferring patient data.

Although there exists lots of encryption methods to prevent patient records from being hacked, it is not possible to guarantee that it will not happen. This is why it is important that whatever system or solution it developed, it should meet all the requirements for health care information governance. Since there are lots of different healthcare systems, there will be many different types of augmented reality system that are being used, all with different degrees of compatibility. For this reason, the healthcare market will benefit from developing both software and hardware that is price sensitive. [2]

Tracking, Calibration, Registration,and Synchronization

There are 4 main issues associated with taking data from a patient and augmenting it onto an endoscopic view.

A tracking method is required in order to be able to get the precise location of the endoscope
Since the projection geometry is important, endoscopes need to be calibrated in order avoid distortion caused by ultra wide angle lenses used on endoscopes
Image registration requires that all patient data is transformed into one common world coordinate system
Time synchronization is important to avoid visualizing data from different points in time

By ensuring that these 4 issues are dealt with, you can obtain high quality and smooth images that can really help improve medical procedures. [8]