Author: Christoph Thees

Supervisor: Dr. Shahrooz Faghihroohi

Coronary artery disease (CAD), also referred to as coronary heart disease (CHD), is one of the most prevalent heart diseases and one of the leading causes of death attributed to cardiovascular diseases worldwide [1]. CAD results from the buildup of plaques along the coronary arteries, which narrows the vessels (stenosis) and reduces or obstructs blood and oxygen flow to the heart. Symptoms can vary widely, from chest pain to heart attacks. Early detection and accurate diagnosis of coronary stenosis are crucial for applying appropriate treatments and preventing fatalities.
CAD can be diagnosed using medical imaging techniques such as Coronary Computed Tomography Angiography (CCTA) or Coronary Angiography (CAG) [2]. However, these scans are challenging to interpret and require highly skilled experts for accurate diagnosis. This blog post demonstrates how deep learning models can be utilized to detect and classify coronary artery stenosis. It provides an overview of existing models, with a detailed discussion of three specific models. The blog post concludes with a summary and discussion of the application of deep learning models in CAD classification.

Motivation

Coronary artery disease (CAD) is a leading cause of death, making early and accurate detection of stenosis crucial for preventing severe cardiac events. However, the diagnosis and assessment process are constrained by the limited availability of trained experts qualified to evaluate CCTA and CAG scans. Even for these experts, the assessment is time-consuming. To alleviate the workload and resource demands for diagnosing CAD, deep learning models should be employed to analyze these scans.
These deep learning models should be capable of taking CCTA or CAG data as input, classifying the degree of stenosis, identifying the type of plaque, and localizing the stenosis along the coronary arteries with minimal human involvement. Additionally, the analysis speed should be significantly faster than that of human assessment while maintaining accuracy.
If successfully implemented, these models would offer substantial clinical benefits by enabling faster and more accessible diagnostic options, reducing dependency on specific experts, and potentially decreasing the fatalities and deaths associated with CAD.

Backround Knowledge

Coronary Artery Disease (CAD)

Coronary artery disease (CAD) is a cardiovascular condition caused by the buildup of atherosclerotic plaques within the arteries. These plaques, often referred to as high-risk plaques (HRP), result from the deposition of lipids, cholesterol, or calcium. HRPs are typically classified into four categories: Positive Remodeling Index, Low-attenuation, "Napkin-ring" sign, and Spotty Calcification [3], as illustrated in Figure 1.
As these plaques grow, the diameter of the vessel progressively decreases until total occlusion occurs, preventing blood and oxygen from reaching the heart. Stenosis, the narrowing of the blood vessel, is classified by the Coronary Artery Disease Reporting and Data System (CAD-RADS). Figure 2 displays the six CAD-RADS categories, which classify the degree of stenosis based on the percentage of artery blockage, ranging from no stenosis (CAD-RADS level 0) to total occlusion (CAD-RADS level 5) [4]. These classifications form the basis for diagnosis and subsequent treatment decisions.

Figure 1. High-Risk Plaque Categories [3]                                  Figure 2. CAD-RADS classification level [4]

Coronary Computed Tomography Angiography (CCTA) & Coronary Angiography (CAD)

Coronary Computed Tomography Angiography (CCTA) and Coronary Angiography (CAG) are the most effective imaging techniques for detecting coronary artery stenosis.
CAG is the leading standard for CAD diagnosis. This invasive procedure involves inserting a catheter into the coronary arteries and injecting a contrast dye. X-rays are then used to produce 2D videos from various angles, providing a comprehensive view of the blood vessels. The resulting high-resolution image sequences (videos) offer detailed anatomical information, enabling accurate stenosis detection [6].
In contrast, CCTA is a non-invasive imaging technique commonly used in the early stages of CAD assessment. CT scans of the heart vessels are performed, and the resulting 2D cross-sectional images are reconstructed into a 3D image. While the image quality of CCTA is lower compared to CAG, it allows the detection and classification of the HRPs leading to the stenosis, as well as the stenosis itself. [5]

Related Work

In recent years, numerous studies have aimed to develop deep learning models for coronary artery stenosis detection and classification. The most promising approaches utilize deep convolutional neural networks (CNNs) for these tasks. As depicted in Table 1, these approaches can be categorized based on the type of input data they process: CCTA-based or CAG-based models.
The CCTA-based models mainly differ in treating the resulting CT scan input data. While some models directly use the 3D image as an input for a CNN [11][13], others use the 2D cross-sectional slices to classify stenosis by, e.g., applying a Long Short-Term Memory (LSTM) network to them [8] or simply classifying each slice using a CNN [4]. Most of these approaches use CAD-RADS for stenosis classification, and some are additionally able to classify the type of plaque [13][8].
CAG-based models perform classification on single frames of the input videos. Some models can autonomously select suitable frames for detection [5], while most still require human involvement for frame selection [10][12]. Furthermore, in addition to the classification, most CAG-based models can localize the stenosis along the coronary arteries [12][5][10]. However, classification is often only performed on a binary level, setting a stenosis threshold and classifying stenosis above or below this threshold [5][10].
The following section will present three papers with particularly interesting architectures or results in more detail.

Table 1. Related work overview

CCTA-Based Models

CCTA-Based Model 1 [5] 

The first paper, called 'Lesion Classification of Coronary Artery CTA Images Based on CBAM and Transfer Learning', presents a deep learning model that uses 2D CCTA slices as input to classify the degree of stenosis. This model employs a ResNet18 architecture based on a convolutional block attention module (CBAM) and transfer learning, achieving particularly high metric scores.
In medical use cases, obtaining large sets of training data can be challenging due to the sensitive nature of the information. Therefore, transfer learning is utilized to pre-train most of the model, using the actual CCTA data only to fine-tune the CNN's final layers.

Segmentation

Before classification, segmentation is performed using an Otsu-region growing joint segmentation algorithm. Initially, the Otsu algorithm divides the image into target and background areas based on the pixel gray values. However, the initial segmentation results need to be improved because some other vessels or tissues may share a similar gray value as the artery. Therefore, starting from seed points inside the coronary arteries, the region-growing algorithm iteratively selects all neighboring pixels with similar gray values. An exemplary result of this combined segmentation approach is shown in Figure 3.

Figure 3. Segmentation results of the Otsu-region growing algorithm. (a) and (c) Original Image. (b) and (d) Segmentation result [5]

Classification

After segmentation, the binary input images are fed into the neural network. The structure of the entire pipeline is depicted in Figure 4. The main component, ResNet18, is pre-trained on 1000 ImageNet images. A channel attention module (CAM) is placed before ResNet18, and together with a spatial attention module (SAM) at the end, it forms the CBAM. Medical images often present difficulties in focusing on crucial features and key areas for classification, which CBAM addresses with CAM and SAM. CAM calculates feature-wise weights to emphasize relevant features, while SAM calculates spatial weights to focus on key areas in the image. The structures of CAM and SAM are detailed in Figure 5. The final layer is a SoftMax, performing binary classification using a 50% stenosis threshold.

Figure 4. Overall network structure [5]                 Figure 5. Structure Diagram of CBAM [5]

Results

Apart from ResNet18, ResNet34 and ResNet50 were also evaluated as possible candidates for the classification task. While they achieved 1% and 3.02% higher results in classification accuracy, respectively, the runtime increased almost linearly with the additional number of layers. As time-efficient classification is one of the main goals for the developed models, the increase in classification performance is too low to make up for the increased runtime. There ResNet18 was chosen as the final choice.
The model is trained on 399 images, with 99 additional images for validation and 126 for testing. The final results, shown in Figure 6 and 7, indicate 98.9% accuracy, 98.1% precision, and a recall rate of 100%.

Figure 6. Training (a) and validation (b) accuracy of ResNet18 with transfer learning and different attention modules [5]

Figure 7. Evaluation results of ResNet18 with transfer learning and different attention modules [5]

Discussion

While the classification results show high accuracy and strong performance across all metrics, the paper still has limitations.
CAD-RADS classification is the standard approach for classifying stenosis. However, the model only distinguishes between stenosis, which is more significant or less than 50%, without any plaque classification.
While transfer learning significantly increased classification accuracy, CBAM has almost no effect, especially considering the small test set size of 126 images.

CCTA-Based Model 2 [13]

The second CCTA-Based model, based on the paper 'Coronary Artery Stenosis and High-Risk Plaque Assessed with an Unsupervised Fully Automated Deep Learning Technique', stands out through its ability not only to classify stenosis based on the CAD-RADS levels, but also being able to classify the type of plaque.
The model takes 3D CCTA data as input and performs vessel-wall and lumen segmentation using a 3D-CNN based on a U-Net structure. After that, two classifiers are employed: one for the stenosis classification and one for the plaque classification.

Results

Figure 8 displays the confusion matrix comparing the predicted CAD-RADS levels by the deep learning model to the levels annotated by expert readers per vessel and patient. The model was trained on 313 scans, validated on 257, and tested on 769 scans. The categorical agreement per vessel was 71.8%, and per patient was 55.5%. The categorical agreement within one category was 93.5% and 89.2%, respectively. 
If the model performed the same binary classification as CCTA-Based Model 1, the accuracy per patient would be 93.6%, and per vessel, it would be 87.1%.
The "napkin-ring" sign type was excluded from plaque classification due to insufficient samples in the dataset. The model was trained on 239 CCTA scans, validated on 196 scans, and tested on 45 scans. Figure 9 shows the classification results per vessel and patient. For low-attenuation plaque, positive remodeling, and spotty calcification, the model achieved accuracy scores of 90%, 84.3%, and 84.3% per patient, respectively.

Figure 8. Tables of level 3 expert read vs deep learning model CAD-RADS score on a per vessel (above) and per patient (below) basis [13]

Figure 9. Diagnosic performance for High-Risk Plaque Features [13]

Discussion

The primary drawback of this paper is the need for more technical detail. While the results for stenosis and plaque classification are impressive, there is insufficient explanation or visualization of the model beyond naming the model type. This information would have been valuable for understanding the training process, and classification results more thoroughly. The paper's title states that the model is based on unsupervised learning. However, the description of the training process suggests that supervised learning was utilized, especially for training the stenosis and plaque classification models. Nevertheless, at this point, the technical depth needs to be added to understand the employed training process.
Another limitation is the small test set size of 45 images for plaque classification. To assess the accuracy of a deep learning model effectively, a larger dataset is necessary to provide meaningful statistics.

CAG-Based Models

CAG-Based Model [6]

The third model, called 'Deep learning-based end-to-end automated stenosis classification and localization on catheter coronary angiograph', takes CAG data as input. What distinguishes this model from other CAG-based models is the end-to-end approach. While most other approaches require manual frame selection for stenosis classification, this model automates the frame selection process. Furthermore, it performs not only stenosis classification but also localization.

Frame Selection

The model aims to categorize all frames of the CAG input videos into candidate and redundancy frames. Candidate frames should exhibit good image quality, high contrast, and clear vessel borders. An InceptionV3 model with a bidirectional LSTM is used for this purpose, as shown in Figure 10. InceptionV3 first extracts relevant features, which are fed into a bidirectional LSTM. Finally, a multilayer perceptron (MLP) with sigmoid activation decides whether a frame is labeled as a candidate or not based on the LSTM outputs and InceptionV3 features. Typically, 5-10 frames are selected per CAG input video.

Figure 10. Frame selection network overview [6]

Classification

As a first step of classification, a separate binary classifier distinguishes candidate frames from the left or right coronary artery (LCA and RCA). This distinction is necessary as LCA and RCA have significantly different features and separate models are trained for each. The classification between left and right is achieved with 100% accuracy.
Next, CAG scans from all views of the RCA are classified with one InceptionV3 model, while each of the four views of LCA employs a different model due to each view's complex and divergent features. Two classification approaches are analyzed: binary classification using a 25% threshold (2-CAT) and three categories (<25%, 25%-99%, and total occlusion) (3-CAT). As shown in Figure 11, classification is performed first at the image level, then aggregated at the coronary and patient levels using max pooling on the previous output features.
However, the classifier sometimes overfitted on background structures or learned invalid features. To address this, redundancy training was introduced. Frames not classified as candidate frames were added to the training set as background data with no stenosis in a 1:1 ratio with candidate frames. The classification results improved significantly, and the influence of redundancy learning is visualized in Figure 12. With the original input frames at the top and arrows pointing at the stenosis, the heat maps show where the model detected relevant features for the classification. Redundancy training helped to get rid of overfitting in the background and focus on actual relevant features.

Figure 11. The architecture of the output of the stenosis classification inception model. LCA, left coronary artery; LAO, left anterior oblique; RAO, right anterior oblique; RCA, right coronary artery [6]

Figure 12. Representative image of the effect of the redundancy training demonstrated in a heatmap style [6]

Localization

In addition to classification, the model also performs stenosis localization using a RetinaNet with an anchor-based feature pyramid network (FPN). The InceptionV3 models used for classification serve as the backbone, with outputs from each pooling layer used to build the FPN, as visualized in Figure 13.
The ground truth data also contained expert annotated bounding boxes around the stenosis, which were used to build seven different groups of height and width as anchors using k-means clustering. These anchors were then utilized to localize the stenosis based on the different layers of the FPN.

Figure 13. Localization network overview [6]

Results

To assess the results of the candidate frame selection, mean error and standard deviations between the begin of contrasting frame (BCF) and end of contrasting frame (ECF) have been calculated. For BCF and ECF, the model achieved a mean error of 2.05 and 2.27, respectively.
The overall classification results for 2-CAT and 3-CAT on an image level can be seen in Figure 14. One can observe that the results for 2-CAT are generally better than those for 3-CAT. Furthermore, the results for RCA are better than the ones for LCA, which is most likely due to the more variable anatomy of LCA, which makes it harder to detect artery narrowing.
Figure 15 shows the ROC curves for the image and the coronary and patient level classification. Again, the graphs for the image-level classification show an improved AUC value for the models with redundancy learning (2R-CAT and 3R-CAT).
The localization model was able to achieve a mean squared error of 39.3 and 37.6 for LCA and RCA, respectively. Some qualitative localization results can be seen in Figure 16.

Figure 14. Image level stenosis classification performance for the 2-category and 3-category severy levels [6]

Figure 15. Performance of coronary stenosis classifications in image, coronary artery, and patient levels [6]

Figure 16. Representative images of stenosis position localization [6]

Discussion

This paper is one of the first to present an end-to-end solution for stenosis classification and localization. However, the main limitation of this model is that the classification is based on only two or three categories instead of using CAD-RADS.
The localization achieved good results for the main LCA and RCA arteries but struggled with additional stenoses located in distal coronary arteries or branches.
Especially for LCA, the classification results for 3-CAT are worse compared to 2-CAT, most likely due to the imbalanced dataset. Only about 8% of the data showed vessels with total occlusion, making it a strongly underrepresented category.
A similar issue occurs for the candidate frame selection. The model struggles to detect very short-duration contrasting frames (<5 frames) as they were rarely present in the training data set.

Summary & Discussion

Deep learning models have shown significant promise in detecting coronary artery stenosis from CCTA and CAG inputs, with some models even capable of localization. However, several challenges must be addressed for these models to achieve clinical utility.
One of the primary challenges is the limited availability of training data. Due to the sensitive nature of medical data, most studies only contain small datasets. Furthermore, it requires trained experts to annotate the ground truth data for the training process. This limitation often results in imbalanced classes within the datasets, particularly for high levels of stenosis, where data is even more scarce or sometimes nonexistent. The result are often models that perform well on the small amount of training data but cannot generalize well on new unseen data. The diversity of the input data increases this effect. CAG and CCTA results can each vary significantly based on the patient, conditions, and equipment used. Therefore, many papers break down the complicated task of detecting stenosis in all coronary arteries to focus only on the main LCA and RCA.
Additionally, models are rarely compared in detail to each other due to the use of different datasets and classification methods. While CAD-RADS is the standard among experts, many models perform only binary classification, with varying stenosis thresholds ranging from 25% to 75%. This difference in classification makes it hard to impossible to compare them with each other.
Another common shortcoming of papers on stenosis classification is the lack of technical depth. While the names of the models and statistical results are typically provided, explanations of technical details, such as the hyperparameters, training process, or loss functions used, are often missing. These details are important for understanding the approach in greater detail and using these papers for further research.
In summary, deep learning models hold significant potential for enhancing the detection and diagnosis of coronary artery stenosis. However, current challenges and limitations need to be addressed in order to employ these models for the actual diagnosis of patients. Continued research and innovation are essential to develop fast, accurate, and generalizable models that can be integrated into clinical practice, ultimately speeding up and improving diagnosis results and advancing the field of cardiac imaging.

References

[1] Seth S. Martin, Aaron W. Aday, et al., Latha P. Palaniappan "2024 heart disease and stroke statistics: a report of US and global data from the American Heart Association." Circulation 149.8 (2024): e347-e913

[2] Jacqueline Latina, Mahsima Shabani, et al., Armin Arbab-Zadeh "Ultra-high-resolution coronary CT angiography for assessment of patients with severe coronary artery calcification: initial experience." Radiology:Cardiothoracic Imaging 3.4 (2021): e210053.

[3] Christopher D. Maroules, Christian Hamilton-Craig, et al., Ricardo C. Cury "Coronary artery disease reporting and data system (CAD-RADSTM): inter-observer agreement for assessment categories and modifiers." Journal ofcardiovascular computed tomography 12.2 (2018): 125-130.

[4] Arzu Canan, Praveen Ranganath, et al., Prabhakar Rajiah "CAD-RADS: pushing the limits." Radiographics 40.3 (2020): 629-652.

[5] Yongze Jin, Xin Ye, et al., Yankai Li "Lesion Classification of Coronary Artery CTA Images Based on CBAM and Transfer Learning." IEEE Transactions on Instrumentation and Measurement (2024)

[6] Chao Cong, Yoko Kato, et al., Bharath Ambale-Venkatesh "Deep learning-based end-to-end automated stenosisclassification and localization on catheter coronary angiography." Frontiers in Cardiovascular Medicine 10 (2023): 944135.

[7] Rigatelli, Gianluca, Filippo Gianese, Marco Zuin. "Modern atlas of invasive coronary angiography views: a practicalapproach for fellows and young interventionalists." The International Journal of Cardiovascular Imaging 38.5 (2022): 919-926.

[8] Andrew Lin, Nipun Manral, et al., Damini Dey "Deep learning-enabled coronary CT angiography for plaque and stenosis quantification and cardiac risk prediction: an international multicentre study." The Lancet Digital Health 4.4 (2022): e256-e265.

[9] Yaofang Liu, Xinyue Zhang, et al., Qing Zhang "Two new stenosis detection methods of coronaryangiograms." International Journal of Computer Assisted Radiology and Surgery 17.3 (2022): 521-530.

[10] Emmanuel Ovalle-Magallanes, Juan G. Avina-Cervantes, Ivan Cruz-Aceves, Jose Ruiz-Pinales "Transfer learningfor stenosis detection in X-ray coronary angiography." Mathematics8.9 (2020): 1510.

[11] Majd Zreik,  Robbert W. van Hamersvelt, et al., Ivana Išgum "A recurrent CNN for automatic detection and classification of coronary artery plaque and stenosis in coronary CT angiography." IEEE transactions on medicalimaging 38.7 (2018): 1588-1598.

[12] Hao Ling, Biqian Chen, et al., Chunli Song "Deep learning model for coronary angiography." Journal ofCardiovascular Translational Research16.4 (2023): 896-904.

[13] Abdul Rahman Ihdayhid, Amro Sehly, et al., Girish Dwivedi "Coronary artery stenosis and high-risk plaque assessedwith an unsupervised fully automated deep learning technique." JACC: Advances (2024): 100861