• Erstellt von Unbekannter Benutzer (ga74few), zuletzt geändert von Unbekannter Benutzer (ge23kop) am 22.Juni 2021

This is a review paper based on the research paper Recovering Brain Structural Connectivity from Functional Connectivity via Multi-GCN Based Generative Adversarial Network conducted by Lu Zhang, Li Wang and Dajiang Zhu, here on referred to as authors[1].




1       Introduction

Throughout human history, we humans have been perplexed by the brain and many theories have been put forth to try to explain this complex phenomenon. The brain was once thought to be a secondary organ for the heart, while others suggested it was where mental activity took place, or proposed that parts of the brain served as tether to the immaterial [2]. There have been many guesses on the function of the brain throughout history but it is only now that we can finally take a proper look at it. With new imaging technologies and methods, we can see the brain in action and actively map it. However, understanding the brain is still a work in progress and many details still elude us, new functions and mechanisms are being discovered regularly.

            In the past scientists have considered the structure of the brain to be an indicator of the functionality of the brain, e.g., the more folds (gyri) and creases (sulci) the higher the consciousness and intelligence. Recent neuroscientific findings showcase that this might not be the case as seen in contemporary animal studies. In the past the avian brain was considered primitive, due to the lack of surface texture, and the consciousness of birds barely that of an automaton. But with the assistance from new technologies and more focus on these creatures reveal that this is a far cry from reality and that some species of birds have almost the equivalent cognitive abilities to that of some species of great apes [3]. This is just a single example out of many where scientist misinterpret the functionality of the brain based on its structure, another fascinating example is the octopus. The octopus has a brain that extends though out the body which is completely at odds with mammalian brains. Octopi have regardless been documented doing complex behaviors including the behavior of tool usage which we usually associate with intelligence [4]. These findings suggest that, that functionality of the brain is not as tied to the structure of the brain as originally thought.

            Another aspect that needs to be considered is the variability of brains. Brains are very flexible and through processes such as neural plasticity and pruning it can adapt to suit the needs of most environments. Neural plasticity and pruning refer to processes in the brain that happen throughout life. These processes help us to adapt to ever changing requirements in our environment by rewiring and strengthening connections that are frequently used while removing connections that are rarely or ever used. These processes can be seen quite clearly in hemispherectomy patients [5].  

            With the assistance of modern technologies and methods we are able to uncover and solve more complex questions about the brain than ever before. In recent times however brain research has staggered slightly due to big data processing [6]. This corundum has been and can be further solved with the aid of powerful computers and complex algorithms, commonly referred to as A.Is or neural network algorithms.

            This paper suggests a new approach to understanding the relation between brain structural connectivity, here on referred to as SC, and functional connectivity, here on referred to as FC, using a technique based on graph convolution- and generative adversarial networks. Understanding this relation is critical for revealing the organizational principles of the brain. But this is easier said than done, since the brain does not function on linear principles but rather operates in parallel and many-to-one function structure mode [7]. The brain FC can be defined via BOLD signal correlations using a fMRI and the SC is represented as the count of diffusion-MRI fibers connecting different regions in the brain. fMRI is a non-invasive technique to measure brain activity via BOLD signals. BOLD, i.e., Blood-oxygen level dependent, signals are used to measure neuronal oxygen uptake in the brain. This allows us to infer which areas of the brain are reacting to stimuli at the moment of imaging. Diffusion MRI is related to fMRI but instead of measuring BOLD signals it measures Random Brownian motion of water molecules in the brain. With this technique we can see the connections between regions and infer on the connectivity between regions [8,9].

            Generative adversarial networks, here on referred to as GAN, is a class of machine learning that takes in input data, analyses it, and learns how to generate data similar to the original input data [10]. Whereas a Graph convolution network, here on referred to as GCN, is a type of neural network that learns the structural information in graphs [11]. The proposed technique multi-GCN-GAN, here on referred to as MGCN-GAN, is composed of two components: A generator and a discriminator. The generator learns how to generate new data by the input of raw data and then by learning the structural principles. The discriminator is trained by the generator to discriminate real data from generated fake data on an adversarial training scheme. This is done with the help of a loss function. The main point of a loss function is to help guide the generator to learn the intrinsic SC patterns in graphs more effectively and prevent the discriminator from easily classifying the generator output.

1.1      Motivation 

A big challenge in Neuroscience is understanding the relationship between the structure of brains and to their function, therefore the authors of this paper suggest using a new technique which could help medical professionals and other neuro-based scientists navigate through the complexity of the brain by distinguishing the variability of individual brains.

1.2      Contribution

In this paper the authors introduce a new mapping technique that is supposed to be able to map brains more efficiently. To do this they propose using a GAN based on multiple GCNs.

2       Methodology 

2.1      Data collection and preprocessing

They used the human connectome project for data, using in total 300 subjects split into two groups, 180 subjects were used as the training datasheet and 120 subjects were used for testing. To partition the brain the authors referred to the Destrieux atlas along with dMRI and rs-fMRI data. The total number of regions after removing two unknown regions and two empty spaces was 148. They applied standard processing procedures to reach this conclusion, these procedures include skull removal, spatial smoothing, slice time correction, temporal pre-whitening, global drift removal and band pass filtering (0.01-0.1Hz) for the rs-fMRI, eddy current correction and fiber tracking via MedINRIA for the diffusion MRI, registering rsfMRI to dMRI space using FLIRT.

2.2      Problem description

The Authors propose a multi-GCN-GAN method to uncover the relationship between individual SC and the corresponding FC. To achieve this the whole brain was partitioned into 148 brain regions represented as nodes with connections between nodes as edges. To represent the SC the authors used the ratio of the number of fibers connecting two ROIs, this is denoted as A ∈ RN ×N. For the functional connectivity the Pearson’s correlation coefficient between two averaged fMRI signals of two ROIs was used, this is denoted as P ∈ RN ×N. Further details about the model are discussed in the next section.

2.2.1      The Model

To model the brain the authors suggest using a graph based on binary matrices where a connection between two points can be weighted using a value between 0 and 1. The proposed model is similar to traditional GAN models in the fact that it consists of two components: A generator and a discriminator, multi-GCN based generator and a single-GCN based discriminator respectively. The generator aims to generate real-like individual SC from raw input data and by competing with the discriminator. It is also worthwhile to note that the generator is composed of k different GCNs, this enables the generator to model complex relationship between the FC and the SC. The discriminator is then trained by the generated SC to be able to distinguish the generated data from real data using a standard cross-entropy loss. Standard cross-entropy loss is a function that measures the divergence between the real and the predicted label. 



Figure 1: the model

         The FC is used as both features associated with nodes and as the initialization of the brain network topology, which is based on the current topology. FC is then mapped to different feature spaces by each multi-layer GCN component of the generator to explore the latent relationship between SC and FC. This is done to obtain multiple output feature matrices. These matrices are then combined by learnable coefficients to generate the predicted SC. This new predicted SC is then used to update the topology, this can be seen in figure 1.

 

Equation 1-2.


         The generator generates these feature matrices by taking in real- and by learning the intrinsic rules of the data, this can be seen in the equation above, where P represents the functional connectivity, e.g. the real input/features, T = Topology, f represents the nonlinear activation function, H = input features which get updated after each iteration,  θ = learnable coefficient and W = Weight. The second equation shows then how these matrices are combined into a single weighted graph, Gk = Graph to create a predicted structural connectivity.

 

Equation 3-4.

        The discriminator may outperform the generator after only a few training iterations which results in zero-back propagated gradients in the generator, this happens when the loss reaches 0, i.e., there is no error, and the discriminator can perfectly discriminate between real and fake data. When this happens the generator cannot be further optimized and will continue to create invalid data. To combat this, the authors introduce their own loss function based on the combination of three other loss functions: The mean square error, Pearsons’s correlation coefficient and the GAN loss, this can be seen in equation 5. With this loss function the authors hope to avoid the zero backpropagated gradients and maintain the balance between the generator and the discriminator.

 

SPLoss = GAN + αMSE + βPCC

Equation 5.

3       Results

3.1      Predicted structural connectivity

Figure 2. Real and predicted model comparison

The new technique was able to map the individual brains from the Human Connectome Project reliably, this can be seen in Figure 2. In figure 2 we are shown the brains of five randomly selected subjects. As shown the overall similarity between the real SC and the corresponding SC is very high. This is demonstrated in figure 2a. where the authors extract two patches from the real SC and the predicted SC at the same location. figure 2b. shows in brain-space the top 15 strongest correlating connectivity, i.e. brain areas, of the real and the predicted SC for the same five subjects. Due to the individual variability the top 15. SC within subjects is different.

3.2      Comparison within the model

Figure 3. Comparison between generators

To justify the usage of multiple-graphs the authors compared the performance of their technique using a single-GCN and a multiple-GCN technique. As seen in the graphs the using multiple GCNs is better at distinguishing between the individual variability between brains, this can be seen in figure 3a. In the other three remaining graphs you can see that without using the weighted graph the discriminator can reliably distinguish between real data and generated data which hampers the learning process of the generator itself.

3.3      Loss function comparison 

Figure 4. Results with different loss functions

To check the excellence of this new generator the authors compared it with the individual components of the SP Loss while using the same dataset. This was done to see if any component could be removed from the SP loss function while still retaining the same or similar integrity. 

            MSE (Real, gen) is the average MSE between the real SC and the predicted SC within the same subject. This measures the similarity between the real and the corresponding predicted SC. Smaller MSE corresponds to higher similarity and thus the MSE should keep decreasing before converging after each iteration. The MSE (other-real, gen) is the average MSE between predicted SC and real SC of other participants. The authors expected the MSE(other-reals, gen) to keep increasing since they didn’t want to capture the common structure of the brain but rather the individual differences between each brain. As seen in figure 4. the predicted SC generated from the single-GCN generators shows worse performance compared to the MGCN generator. 

             The authors conclude that their SP loss outperforms the individual components. The reasons being that the PCC, though being able to capture the connectivity patterns, might not capture the connection weight between nodes. MSE on the other hand only focuses on the similarity of the elements and ignores the overall patterns. By combining these into the SP loss the authors have a clearer picture of the connectivity and the weight. The multiple GCNs in the generator are combined with learnable coefficients. To initialize these coefficients the authors tried using different values and discovered that the coefficients will converge to a consistent ratio that all the GCN components seem to contribute equally and so each component was indispensable.

4       Conclusion and Discussion

The authors conclude that the new technique can reliably predict the subtle differences in the individual brains from the human connectome data. The authors also conclude that similarities in FC and SC across individuals could indicate a common regulation between specific brain structural and functional architectures.

5       Student Review 

This technique shows a lot of promise, but it has a long journey ahead before it can be used in a clinical setting. As mentioned, the functionality is not dictated by the structure of the brain and this can be found prominently in people that have for example a lesioned brain, are blind or deaf. Contemporary research indicates that the brain functionality alters to compensate for these different experiences, for example areas of the brain that would normally handle visual input for blind people will be pruned and repurposed[12]. Due to the nature of the training data, which is based on a pre-concieved model of the brain, the SP-loss function will induce a bias. This technique will most likely struggle with statistical outliers that don’t conform to common architecture since this technique was only used on data gotten from the Human Connectome. The Human Connectome Project is a a comprehensive brain research project on normal healthy people and by extension; brains[13]. 

            There is also a problem of potential cultural differences affecting the brain structure, e.g., our brain is experience dependent and is shaped in part by our environment. I would be interested in seeing this technique exercised on a more diversified population, for example people born with blindness, people that acquire blindness later in life, people from various cultural backgrounds or environments that elicit different requirements for survival [14,15].

            As touched upon in the introduction, I believe that data processing algorithms has the potential to render neuroscientists, medical professionals, and psychologists alike a much-needed assistance with data processing. The data that comes from brain imaging is often enormous and shifting through it can take a lot of time, time that for example medical professionals might not necessarily have. Therefore, I find these results very promising and look forward to seeing further development in this field.

6       References

  1. Zhang, L., Wang, L., & Zhu, D. (2020, October). Recovering Brain Structural Connectivity from Functional Connectivity via Multi-GCN Based Generative Adversarial Network. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 53-61). Springer, Cham.
  2. Gross, C. G. (1987). Neuroscience, early history of. Journal of Enclyclopedia of Neuroscience pp. 843, 847.
  3. Olkowicz, S., Kocourek, M., Lučan, R. K., Porteš, M., Fitch, W. T., Herculano-Houzel, S., & Němec, P. (2016). Birds have primate-like numbers of neurons in the forebrain. Proceedings of the National Academy of Sciences, 113(26), 7255-7260. Chicago
  4. Hochner, B., Brown, E. R., Langella, M., Shomrat, T., & Fiorito, G. (2003). A learning and memory area in the octopus brain manifests a vertebrate-like long-term potentiation. Journal of Neurophysiology, 90(5), 3547-3554. Markram, H. (2013). Seven challenges for neuroscience. Functional neurology, 28(3), 145.
  5. Althausen, A., Gleissner, U., Hoppe, C., Sassen, R., Buddewig, S., von Lehe, M., ... & Helmstaedter, C. (2013). Long-term outcome of hemispheric surgery at different ages in 61 epilepsy patients. Journal of Neurology, Neurosurgery & Psychiatry, 84(5), 529-536.
  6. Landhuis, E. (2017). Neuroscience: Big brain, big data. Nature, 541(7638), 559-561.
  7. Sigman, M., & Dehaene, S. (2008). Brain mechanisms of serial and parallel processing during dual-task performance. Journal of Neuroscience, 28(30), 7585-7598. Chicago
  8. Detre, J. A., & Wang, J. (2002). Technical aspects and utility of fMRI using BOLD and ASL. Clinical Neurophysiology, 113(5), 621-634.
  9. Le Bihan, D. (2006). From Brownian motion to mind imaging: diffusion MRI. Bulletin de l'Academie nationale de medecine, 190(8), 1605-27.
  10. Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). Generative adversarial networks. arXiv preprint arXiv:1406.2661. Zhang, S., Tong, H., Xu, J., & Maciejewski, R. (2019). Graph convolutional networks: a comprehensive review. Computational Social Networks, 6(1), 1-23.
  11. Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T., & Weinberger, K. (2019, May). Simplifying graph convolutional networks. In International conference on machine learning (pp. 6861-6871). PMLR.
  12. Amedi, A., Merabet, L. B., Bermpohl, F., & Pascual-Leone, A. (2005). The occipital cortex in the blind: Lessons about plasticity and vision. Current Directions in Psychological Science, 14(6), 306-311.
  13. Van Essen, D. C., Smith, S. M., Barch, D. M., Behrens, T. E., Yacoub, E., Ugurbil, K., & Wu-Minn HCP Consortium. (2013). The WU-Minn human connectome project: an overview. Neuroimage, 80, 62-79.
  14. Han, S., & Ma, Y. (2014). Cultural differences in human brain activity: a quantitative meta-analysis. NeuroImage, 99, 293-300.
  15. Martínez-Méndez, R., Martínez-Martínez, E., Villafán-Monroy, H., Guzmán-López, J. A., Fuentes-Farías, A. L., Romo-González, T., ... & Gutiérrez-Ospina, G. (2013). 4. Body and brain plasticity: Unraveling its principles through blindness. Research Signpost India Recent Res. Devel. Neurosci, 37661(4), 89-107.