The process of registering and combining multiple images from a single or multiple imaging modalities is called image fusion. This is done to improve the image quality, to use the advantages of two different imaging modalities, but also to monitor deformations in the brain during the surgery compared to preoperative images.
Fusion of Images from Different Modalities
Particularly interesting is the fusion of anatomical imaging and functional imaging, as shown in the pictures below. The advantages of both can be used. Anatomical imaging such as MRI and CT shows the anatomy. It is optimal for detecting structural abnormalities, but it lacks specifity for further characterizing this abnormalities. Functional imaging (e.g. PET, SPECT) uses tracers to track certain functions and activities e.g. increased metabolism due to tumors. Anatomical imaging often has a higher resolution, while functional imaging has a much higher sensitivity. So both methods can be combined in order to determine precise positions and size of lesions.
figure: T1-weighted MRI (left), PET (right), fusion image (center). MRI and PET datasets were simultaneously acquired at coronal (cor), sagittal (sag), and transverse (tra) sections. [1]
figure: Top: CT (left), PET (right), merged image (center). Bottom: T2-weighted MRI (left), PET (right), merged image (center). [1]
Image registration
Registration is the process of bringing two or more images into spatial alignment. Therefore usually one image is "fixed" and the other one is translated and transformed until it "fits" onto the first one.
Transformation and warping
- Rigid: translation and rotation
- Affine: translation, rotation and shearing
- Projective: translation, rotation, shearing and moving point of infinity
- Deformable: all changes allowed - cannot be characterized by a matrix multiplication
figure: Transformations and their transformation matrices. [2]
Extrinsic registration methods
Extrinsic registration is based on external markers attached to the patient. These markers can be skin-affixed, frame-attached or bone-implanted. The most accurate markers are the bone-implanted ones, but they are also the most invasive.
Intrinsic registration methods
Intrinsic registration is based only on image content generated from the patient. It can be feature-, intensity- or gradient-based.
Feature-based registration
In feature-based registration the registration is done based on geometrical entities like isolated points or point sets, representing a curve, contour or surface ( = features/landmarks). The features are usually extracted by segmentation. For the registration, the distance between the features in the two or more images is minimized. Therefore often rigid and affine transformation is used.
Intensity-based registration
Intensity-based registration is based on the optimization of similarity between the grey values of the two or more pictures. Unlike feature-based registration, as described before, the whole image content is analysed. Different similarity measures (sum of squared differences, sum of absolute differences, normalized cross correlation, ...) and optimization approaches (gradient/steepest descent, Newton's method, least squares, ...) can be used.
Gradient-based registration
As the name induces, gradient-based registration is based on gradients. The basic concept is that strong intensity gradients correspond to boundaries (volume gradients). Thus the x-rays pointing to edges are tangent to surfaces. For the registration the similarity between the gradient vectors is optimized. [3]
Bibliography
1) Simultaneous PET/MRI creates diagnostic-quality images, http://www.auntminnie.com/index.aspx?sec=ser&sub=def&pag=dis&ItemID=91503 (access 05/06/17)
2) Prof. N. Navab (2017) Lecture Computer Aided Medical Procedures, Rigid Image Registration
3) Markelj P. (2012) A review of 3D/2D registration methods for image-guided interventions