OUTLINE

1. Introduction

1.1. Motivation

The application of deep learning (DL) in healthcare is rising tremendously, with an increase in the number of publications per year. However, these DL models rely heavily on a large amount of ground truth data, which is not easy to obtain, especially in the case of healthcare data. As per different regulatory body guidelines, such as the FDA, medical data should be highly confidential and collected with every patient’s approval. This process of collecting and approving data at the individual level is quite challenging, burdensome, and infeasible for the agencies. Therefore, there is a high demand for new learning models that can work on less ground-truth data [1].  

1.2. What are Physics-Inspired Neural Networks?

Many new deep learning methods are being developed to solve the problem of large data, with physics-inspired neural networks (PINNs) being one of them. They reduce the dependency on large training data by introducing the system's physics into the neural network training. PINNs have two types of loss functions: unsupervised, which is the partial differential equations (PDEs) describing the physics of the system, and supervised, which fits the available data using standard loss function. They are considered to be an unsupervised learning strategy as they do not solely rely on previously labeled large ground truth data. They can learn patterns in a small amount of data just by considering the initial and boundary conditions of the system and solving the PDEs at those conditions. If the ground truth data is available, these networks can use it to fit and further improve the predictions [2].

1.3. Advantages of PINNs

PINNs have multiple advantages over conventional methods:

1. PINNs can solve forward and inverse problems using the same optimization method with minimal modifications. Solving forward problems refers to predicting the behavior or outcomes of a system based on known input conditions and the underlying physical laws governing that system, whereas inverse problems involve deducing the causes or inputs of a system based on observed outputs. PINNs are so flexible that even the same code can be used with minimal modifications to solve both forward and inverse problems [2]. 
2. Standard PDE solution methods involve the creation of a mesh, where an object’s surface is divided into multiple points, as shown in Figure 1. After mesh formation, the PDEs are solved at each point by giving the values for the initial state and conditions at the boundary, which then numerically calculate values at further time points. This process of simulation is computationally very heavy, requiring high computing resources and time. PINNs, on the other hand, do not require any kind of mesh formation and directly solve PDEs with initial values [2].
3. They can address any complicated geometry of higher dimensions, which is otherwise difficult to simulate numerically. For example, it is numerically quite easy to simulate fluid flow inside a straight pipe as the mesh formation on it is numerically not intense. However, in case of an irregular blood vessel geometry or an aneurysm (Figure 2), the mesh becomes complicated and hence takes more time and computation [3].

Figure 1. Mesh formation in a Y-tubeFigure 2. Complexity of forming a mesh in aneurysm

Therefore, considering these advantages of PINNs over conventional methods, this article discusses the application of PINNs to different medical domains or clinical settings.

2. PINNs for Brain Hemodynamic Predictions Using Medical Imaging [4]

2.1. Clinical Motivation

Some diseases are characterized by complex and dynamic interactions of internal physiology and thus need to be monitored quite frequently. They need a high temporal resolution and a high spatial resolution to pinpoint the exact location of the cause in the body. For these reasons, the demand for high spatiotemporal resolution data is increasing. One such commonly occurring disease is stroke. According to the World Health Organization (WHO), there are 12 million new stroke cases every year [5]. A stroke is a sudden burst of a blood vessel in a specific area of the brain, causing neuronal death in that region and finally leading to disability of the corresponding body part that is controlled by that brain area. The burst can be of the artery, or an outgrowth of the artery called an aneurysm, classifying strokes into different categories. Subarachnoid Haemorrhage (SAH) is a type of stroke typically due to intracranial aneurysm rupture. After SAH, many patients show focal or diffused transient narrowing of large intracranial arteries, known as cerebral vasospasm (CVS), as shown in Figure 3. This focal narrowing can further cause complications and therefore needs to be diagnosed at the earliest [6].

Figure 3. Constriction of vessel after SAH leading to CVS

There are different methods for diagnosing CVS:

1. Time of Flight (TOF) MR angiography - This method provides the spatial information of blood vessels in the form of images. It helps in studying the vascular architecture of the brain by providing high-contrast blood vessel images and thus helps identify CVS. However, it only provides the location of blood vessels and does not give any information about the velocity of blood inside those vessels. For accurate diagnosis of CVS, temporal information on blood flow is required, which is provided by the next method.
2. Transcranial Doppler (TCD) Ultrasound - This method of diagnosis provides quantitative blood flow velocity measurements at different locations in the vascular architecture. It provides the change in velocity of blood with time i.e., velocity-time series data. However, it does not provide the location of blood vessels, which is necessary for accurate diagnosis. Also, since this method uses ultrasound which has less penetration capacity, it is limited to diagnosing only those areas of the brain where the skull is thin. 
3. 4D Flow MRI - This method provides both the spatial and temporal information of blood flow i.e., it has a good spatiotemporal resolution. The spatial information comes from the MR images of blood vessels, whereas the temporal information of blood flow velocity comes from the 4D flow component. Apart from that, it can provide in vivo values of velocity flow. However, this method is not used in clinical settings because of large scan time to obtain high spatial resolution. It is mainly limited to fundamental research questions. 

With these methods of diagnosis, it is clearly visible that we need a method that can provide the same information as TOF MR angiography and TCD ultrasound combined together. Therefore, Sarabian et. al. proposed this new PINN model, which combines these two inputs and predicts the values which can be validated with 4D flow MRI. 

2.2. Input and Output of PINN

The input for the PINN model consists of the information from the TOF MR angiography and TCD ultrasound. The first method provides spatial information in the form of a cross-sectional area of blood vessels. The second method provides the velocity-time data of some specific locations in the blood vessel architecture. Both these values are initial values of area and velocity. The PINN model aims to predict the velocity, area, and pressure at some time point "t" which is validated by using the 4D flow MRI data. 

2.3. PDEs of PINN 

The PDEs used for this PINN model are the 1D reduced-order model (ROM) equations, modified versions of the Navier-Stokes equation for blood flow dynamics. These equations represent the elastic vessel wall pressure-area relationship.

where Q is the volume flow rate, P is the average internal pressure, A is the cross-sectional area, α is a non-dimensional momentum flux correction factor, ρ is the constant mass density of blood, and f is the frictional force per unit length. 

2.4. PINN architecture

The network takes the input of the initial area from TOF MR angiography images and velocity-time series data from the TCD ultrasound. With the initial area values, it calculates the initial velocity values at the same locations using the standard continuity equation, which gives the relationship between area and velocity in a fluid-carrying pipe. As the area of a pipe changes, the velocity of fluid inside it also changes, which is represented by:

The velocity-time series data from TCD ultrasound is also fed into another PINN model known as Area-Surrogate PINN (ASPINN), which converts the velocity-time data into area-time data. All these four inputs - initial area, initial velocity, velocity-time series, and area-time series are then fed to the main PINN network for training. A simple network architecture is shown in Figure 4.

Figure 4. Overview of PINN architecture with input and output

The PINN architecture has two transformer networks to map the input variables to the high-dimensional feature space and update the hidden layers using a point-wise multiplication operation. Consequently, this architecture enhances the hidden states through the residual connections. ASPINN model has two networks, each with seven hidden layers and 100 hidden units per hidden layer. 

The loss function used in training the network is composed of four different losses:

The first is measurement loss which corresponds to the fitting of clinical data. Second is residual loss which satisfies the PDEs at different points of arteries. Third is interface loss, which satisfies mass conservation and momentum conservation at artery junctions. Last is slope recovery loss, accelerating the convergence and reducing the training cost. 

2.5. Results

The authors tested the model at different locations in the vascular architecture and found reasonable agreement between the PINN model predictions and the 4D flow MRI data, as shown in Figure 5.

Figure 5. Comparison of blood flow velocity at different locations in the vascular architecture. PINN prediction (Green) and 4D Flow MRI (Red)

3. PINNs for modeling physiological time series for cuffless blood pressure estimation [7]

3.1. Clinical Motivation

Blood pressure is an important physiological parameter that provides valuable insights into various diseases. It is normally measured using a medically-graded device called a sphygmomanometer which requires the patient to wrap a cuff above the elbow and measure the blood pressure by squeezing the cuff. Although this measurement method provides very accurate results, is not always feasible, especially while performing an activity such as running. To counter such problems, wearable devices such as smartwatches have been installed with blood pressure measuring systems that indirectly measure blood pressure using bioimpedance signals from the arterial blood flow. However, these bioimpedance signals do not represent the actual arterial expansion during a cardiac cycle and thus can provide inaccurate data. Hence, there is a need to establish a relationship between the two data types and thus convert bioimpedance to blood pressure values more effectively.

3.2. Input and Output of PINN

The bioimpedance signal measured from wearable sensors acts as the PINN input and predicts the blood pressure of the systole (left heart ventricle contraction). Sel et al. extracted three features from the bioimpedance signal to improve the network performance and fed them along with the bioimpedance signal. For feature extraction, the bioimpedance signal is segmented into multiple cardiac cycles, which are repeated over time to make the complete signal. Three features are then extracted from each cardiac cycle:

1. u1 - This feature represents the amount of arterial expansion during the cardiac cycle
2. u2 - This feature estimates the pulse wave velocity
3. u3 - This feature represents the beat-to-beat heart rate (Figure 6)

The output of the network is validated with ground truth blood pressure values measured using a medically-graded sphygmomanometer.

Figure 6. Bioimpedance signal (left) and its extracted features (right)

3.3. PDEs of PINN

The PDE used for this PINN is Taylor's approximation polynomial, which relates the input sensor measurements to output blood pressure values.

where P represents this Taylor polynomial approximated based on the i^th segment, and f_NN is the approximated function by the neural network.

3.4. PINN architecture 

The PINN architecture takes in the segmented bioimpedance signal and combines the corresponding three features into the final dense layer of the network (Figure 7). The 1D-CNN network has two layers, with the first and second layers having 32 and 64 filters and kernel sizes of 5 and 3, respectively. Both have RELU activation functions with max pooling applied to the output, followed by flattening.  

Figure 7. PINN architecture for cuffless blood pressure estimation

3.5. Results

The results obtained were better than traditional CNNs, as seen in Figure 8.

Figure 8. PINN BP performs better than CNN BP

4. WarpPINN: Cine-MR image registration with PINNs [8]

4.1. Clinical Motivation

Heart activity is often measured using the Ejection Fraction (EF) metric.

Stroke volume represents the volume of blood pumped from the left ventricle per beat, while the end-diastolic volume represents the volume of blood remaining in the ventricle at the end of the diastole (relaxing ventricle). Normally, if the value of ejection fraction for the left ventricle is above 50%, then the heart is considered to be normally functioning. However, in some cases, the patients still sometimes present with heart failure. Therefore, new metrics need to be devised for measuring the health of the heart [9].

A new metric called cardiac strain is currently being used for the same purpose. It represents the relative displacement of cardiac tissue through the whole contraction. The cardiac tissue can deform in different directions - circumferential, radial, and longitudinal. It provides better insights into heart health and is good in reproducibility and sensitivity [10]. 

There are different methods of measuring cardiac strain, out of which cine steady-state free precession (SSFP) MRI is the most commonly used method (Figure 9). It is considered the gold standard for heart imaging and thus is routinely used in clinical imaging. However, it does not contain direct information about the cardiac strain, and thus, there is a need for a better strain estimate method from SSFP MRI [10]. 

Figure 9. Cine steady-state free precession (SSFP) MRI for cardiac strain imaging

4.2. Input and Output of PINN

The input to the PINN is the SSFP MR image. However, when given normally, the network was learning the low-frequency components first and then learning the high-frequency components which ultimately led to inaccurate predictions. Therefore, the authors modified the input by performing a Fourier transform to feed the high-frequency and low-frequency components independently, as shown in Figure 10. The output of the network is the displacement field Φ that represents the difference between the template and reference image. This displacement field is used to register the two images and then is used for cardiac strain assessment. For validation, manually tracked landmarks in SSFP MRI are used. 

Figure 10. Input to network was spectrally biased, so Fourier transform is performed.

4.3. PDE of PINN

In these neural networks, the strain energy function is used as a regularizer for the displacement predicted by the neural network, and it is based on a neo-Hookean hyperelastic function. The strain energy function W is given by:

where Φ is the displacement field predicted by the neural network, λ is a parameter associated with the hyperelastic function, C is the Cauchy-Green deformation tensor, and J is the determinant of the deformation gradient.

4.4. Results

The results of PINN prediction are shown in Figure 11. On the left is the ground truth image, in the middle is the PINN prediction without the Fourier transform, and on the right is the PINN prediction with the Fourier transform. The prediction with the Fourier transform has much better boundary differentiation because the higher frequencies included in the input allow the neural network to better capture the sharp changes in the deformation field at the boundaries, which are difficult to represent with low-frequency components alone.

Figure 11. WarpPINN prediction with the Fourier transform is better than without

5. Conclusion

This blog introduces physics-inspired neural networks (PINNs) to solve dynamic and complex medical systems. Three papers with different clinical motivations are compared based on their input, output, underlying physics and network architecture (Table 1). 

Table 1. Comparison of PINN models based on different clinical needs

Based on these discussions, it can be concluded, in my opinion, that PINNs can solve those physical problems which can be described in terms of PDEs. The data that cannot be described by physics-based PDEs is not solvable by PINNs. Also, since the knowledge from PDEs is being used to solve and fit the data, the need for a large amount of training data is also reduced. However, the explainability of PINNs is one challenge that hasn't been discussed here and can be explored further. 

6. References

1. Sel, K., Mohammadi, A., Pettigrew, R.I. et al. Physics-informed neural networks for modeling physiological time series for cuffless blood pressure estimation. npj Digit. Med. 6, 110 (2023).
2. Cuomo, S., Di Cola, V.S., Giampaolo, F. et al. Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next. J Sci Comput 92, 88 (2022).
3. Moser P, Fenz W, Thumfart S, Ganitzer I, Giretzlehner M. Modeling of 3D Blood Flows with Physics-Informed Neural Networks: Comparison of Network Architectures. Fluids. 2023 Jan 27;8(2):46.
4. Sarabian, H. Babaee and K. Laksari, "Physics-Informed Neural Networks for Brain Hemodynamic Predictions Using Medical Imaging," in IEEE Transactions on Medical Imaging, vol. 41, no. 9, pp. 2285-2303, Sept. 2022.
5. World Stroke Organization (WSO): Global Stroke Fact Sheet 2022.
6. Lucke-Wold, B., Dodd, W., Motwani, K. et al. Investigation and modulation of interleukin-6 following subarachnoid hemorrhage: targeting inflammatory activation for cerebral vasospasm. J Neuroinflammation 19, 228 (2022).

7. Sel, K., Mohammadi, A., Pettigrew, R.I. et al. Physics-informed neural networks for modeling physiological time series for cuffless blood pressure estimation. npj Digit. Med. 6, 110 (2023).

8. López, P. A., Mella, H., Uribe, S., Hurtado, D. E., & Costabal, F. S. (2023). "WarpPINN: Cine-MR image registration with physics-informed neural networks." Medical Image Analysis, 89, 102925.

9. Zhou Q, Li P, Zhao H, Xu X, Li S, Zhao J, Xu D, Zeng Q. Heart failure with mid-range ejection fraction: a distinctive subtype or a transitional stage?. Frontiers in Cardiovascular Medicine. 2021 May 25;8:678121.

10. Scatteia, A., Baritussio, A. & Bucciarelli-Ducci, C. Strain imaging using cardiac magnetic resonance. Heart Fail Rev 22, 465–476 (2017).