Neural Networks for NDT of CFRP Plates Using Lock-In Thermography

Osama Atwi, summer semester 2023

Applying neural networks in engineering provides various possibilities to enhance and optimize the performed tasks. However, using neural network techniques comes with difficulties and challenges. This article discusses the usage of neural networks to improve the predictions of failure types in carbon fiber reinforced plastics (CFRP) plates using lock-in thermography non-destructive testing techniques. The article discusses the technology used to test the CFRP plates and how to employ neural networks.

Introduction

Non-destructive testing (NDT) is pivotal in modern engineering, allowing for material and structural assessments without causing harm to the components^[1]. Industries like aerospace, manufacturing, civil engineering, and healthcare rely on NDT for ensuring safety and reliability^{[1] [2]}. Among the array of NDT techniques, lock-in thermography stands out as a powerful and versatile method, offering unique capabilities in detecting and characterizing subsurface defects, irregularities, and structural anomalies^[3]. CFRP plates, widely used in aerospace, automotive, and construction industries for their exceptional strength-to-weight ratio, demand rigorous inspection to ensure structural integrity. However, their complex composite structure poses challenges for traditional inspection methods^[4]. Lock-in thermography, when enhanced with deep learning, provides a possible solution to the downsides of the current methods^[5].

The images obtained from past lock-in thermography tests can be employed to train a neural network for predicting the specific type of failure in a recently conducted imaging of a CFRP plate. This capability stems from the innate learning ability of neural networks to categorize new images following training on a well-labeled data-set. This data-set is composed of images of CFRP plates exhibiting a range of documented failure types, ensuring comprehensive coverage of all possible failure types as well as CFRP plates without damage to be used as a control^[5].

Description of the Lock-in Thermography Examination Process for CFRP Plates

Working Principle of Lock-In Thermography

Lock-in thermography is a powerful non-destructive testing technique used to detect and characterize subsurface defects or irregularities in materials^[5]. It operates on the principle of controlled thermal excitation and phase-sensitive detection as presented in Figure 1^[5].

First, a periodic and controlled thermal excitation, typically in the form of a modulated heat source, is applied to the material under examination. This stimulus induces thermal waves that propagate through the material. These waves interact with any subsurface defects, causing local variations in temperature^[6].

Next, a highly sensitive infrared camera captures the surface temperature variations of the material over time. The camera records not only the thermal waves generated by the excitation, but also their phase information. The key aspect of lock-in thermography lies in its phase-sensitive detection^[7][8].

It utilizes a reference signal that is synchronized with the thermal excitation. By analyzing the phase relationship between the recorded thermal waves and the reference signal, lock-in thermography can extract subtle temperature differences caused by subsurface anomalies^[6][7].

This phase-sensitive approach enables lock-in thermography to distinguish between normal and defective regions within the material. Defects, such as cracks, delaminations, or voids, alter the thermal response, allowing them to be visualized and characterized^[4]. This makes lock-in thermography an invaluable tool for quality control, structural integrity assessment, and materials characterization across various industries^[6].

Another method to perform the excitation of the tested part is by using mechanical excitation methods, such as ultrasound waves. The mechanical waves propagate through the tested part and cause vibrations. These vibrations are profound on the edges of a failure position within the part and leave a different thermal footprint that can be detected and analysed by the examiner^[9].

Characteristics of CFRP Plates

Carbon Fiber Reinforced Polymer (CFRP) plates represent a breakthrough in materials engineering, offering exceptional strength-to-weight ratios and corrosion resistance^[10]. Comprising a matrix of polymer reinforced with carbon fibers, CFRP plates find extensive applications in aerospace, automotive, construction, and other high-performance industries^[2]. However, examining CFRP plates for potential defects and anomalies presents unique challenges for non-destructive testing (NDT) techniques^[11]. The following are a set of some of the difficulties presented while applying traditional NDT methods to examine CFRP Plates:

• Anisotropic Material Properties: CFRP’s mechanical properties vary with fiber orientation, leading to anisotropy. This means that a defect’s detectability can change based on its orientation relative to the fibers^[12][13].
• Heterogeneity due to Manufacturing: Manufacturing of CFRP plates introduces a great amount of heterogeneity due to the nature of the manufacturing methods. This disparity increases the difficulty of localising and assessing failure types of CFRP plates^[14].
• Ultrasound Attenuation: Ultrasound, a common NDT method, experiences significant attenuation in CFRP due to the material’s high density and complex structure. This reduces the penetration depth and limits the ability to detect subsurface defects^[15].

Due to the shortcomming of other NDT methods and the given nature of identifying failure types in Carbon Fiber Reinforced Polymer (CFRP) plates, there is a compelling proposition to incorporate technologies like neural networks. These advanced approaches, when applied to data derived from Non-Destructive Testing (NDT) techniques tailored for CFRP plates, such as Lock-in thermography, can significantly enhance the examiner’s ability to identify failure modes with elevated precision and confidence^[5].

 Neural Networks for NDT

Basics of a Feed Forward Neural Network

A neural network is a collection of interconnected nodes or neurons arranged in layers. Each neuron processes information and passes it to the next layer, mimicking the biological neurons’ behavior. This process is illustrated in Figure 2, in which the feedforward method is shown along side the input and out parameters. The following are high-level descriptions of the neural networks’ architecture^[16]:

• Input Layer: The input layer represents the first layer of the NN and it takes parameters originating from the processed data-set. In the case of image classification, the input parameters consists of pixel values. Each pixel acts as an input feature with a certain value, and its intensity; the value associated with the pixel; contributes to the representation of the image^[16].
• Hidden Layers: The intermediate layers between the input and output layers are called hidden layers. These layers perform complex computations to transform the input data. The dimensions of the hidden layers; number of layers and number of neurons in each layer; plays a pivotal role in affecting the quality of the neural network^[16].
• Output Layer: The final layer provides the network’s prediction or classification. For image classification tasks, the output layer typically has neurons equal to the number of classes. Each neuron corresponds to a class, and their values represent the confidence scores^[16].

Convolutional Neural Networks

In Order to process grid-like data; like images; convolutional Neural Networks were developed. CNNs are a specific type of NNs the uses convolutional layers to scan the initial input data for patterns, enabling them to recognize features like edges, textures, and shapes. The following are descriptions of the main components and functions for CNNs^[17]:

• Convolutional Layers: Applying filter to small regions of the input is part of the convolutional layers. This is done to detect specific features within the said layers. Multiple filters are used in parallel to capture various aspects of the image^[17].
• Pooling Layers: Pooling layers reduce the spatial dimensions of the data while retaining essential information. They aggregate neighboring pixels, reducing computational complexity and making the network more robust^[17].
• Activation Functions: Activation functions introduce non-linearity into the model, allowing neural networks to approximate complex relationships between features. Common activation functions include ReLU (Rectified Linear Unit), Sigmoid, and Tanh^[17].
• Training and Backpropagation: Fine-tuning the weights is crucial while training a neural network to minimize a predefined cost function. The weights are the parameters that are multiplied with the values of each neuron while moving from one neuron to another. Backpropagation is a crucial algorithm that computes gradients for adjusting these weights^[17].
• Loss Functions: The loss function quantifies the discrepancy between predicted and actual outputs. For classification tasks, cross-entropy loss is often used as it penalizes confident but incorrect predictions more severely^[17].
• Optimization Algorithms: Optimization algorithms, like Stochastic Gradient Descent (SGD) and Adam, determine how the model’s weights are updated during training. They aim to find the global minimum of the loss function^[17].
• Regularization Techniques: Regularization methods like dropout and L2 regularization help prevent overfitting by reducing the dependency of the NN on certain parameters and neurons^[17].
• Transfer Learning: Pretrained models; trained on large data-sets; are used for transfer learning. The models are then fine-tuned for certain predefined tasks. This approach can significantly reduce training time and data requirements^[17].

Examining CFRP Plates using Lock-In Thermography and NNs

This integration of Lock-In Thermography and neural networks not only improves the detection accuracy but also establishes a robust foundation for automated or semi-automated evaluation processes. This, in turn, leads to accelerated and more reliable assessments of CFRP plate integrity. By harnessing the potential of these combined techniques, industries relying on CFRP components can substantially elevate the level of quality assurance and safety in their operations^[18][19].

Dataset Preparation

At the heart of any good neural network model is the dataset used to train the model. A dataset should fullfill various requirements in order to be suitabel for usage in a neural network model. For the discussed case the following are the main requirements to be fullfilled by any dataset to be valid for usage^[16]:

• Sufficient Size: The dataset should be of an adequate size to capture the complexity of the underlying patterns in the data. Larger datasets often lead to more robust and generalizable models^[16]. In the case of Infrared Thermogprahy test of CFRP plates this corresponds to having a set of images conducted previously that is of an adequate size to cover all the possible failure types.
• Relevance to the Problem: The data-set should be directly related to the problem the neural network is intended to solve^[16]. It should contain information that is meaningful and pertinent to the task at hand. It should therefore be clear that the images used in the data-set depict a clear and direct representation of the failure types that can be present in CFRP plates.
• Diverse and Varied: The data-set should encompass a wide range of scenarios, variations, and edge cases that the model might encounter in real-world applications^[16]. This helps the model generalize well. In order to be certain that the neural network is trained for all possible failure types, all the mentioned types should be present in the data-set used.
• Clean and Preprocessed: Data should be cleaned to remove any noise, outliers, or irrelevant information. It should also be preprocessed appropriately (e.g., normalization, scaling) to ensure consistency and compatibility with the model^[16]. It is therefore important to have images that were taken using the same scales, color representations and boundary conditions.
• Labeled: In supervised learning, the data-set should have clear and accurate labels associated with each data point^[16]. These labels serve as the ground truth for training the model and to identify the failure types afterwards.

The data-set is comprised of images collected from previous tests. The images and the failure types present in them are known. With this knowledge the data-set is created as seen in Figure 3. The Labels used in the last step of the preparation of the data-set is are called classes. These classes are the output of the neural network as mentioned previously. The final data-set contains; for each image; a set of pixels representing the pixels of the collected image. Each pixel posses a certain number that is co-related to the intensity of the said pixel. The major set of pixels with the corresponding values represents the input of the neural network.

Case Studies for using Machine Learning in Lock-in Thermography Using Deep Learning

Various papers have examined the possibilities of implementing NNs for NDTs^{[20][21][22][23][19]}. Diverse approaches where adopted from using numerical models to train the NNs^[22], to using real data from images and Data-sets conducted^[23]. Various NDT methods were used to perform the latter studies, the focus of this study is using NNs in Thermographic nondestructive testing. Therefore, the case studies considered are bounded by this topic.

Neural Network Based Nondestructive Evaluation of Sandwich Composites

General Method

Just Et al.^[19] analyzed the possibility of integrating neural networks to improve failure prediction in sandwich structures made out of CFRP. The study investigated a novel approach employing a neural network that combines vibration and thermographic techniques, which had not been previously explored in existing literature. When utilizing only vibration-based modal curvature response, the model successfully identified various damage scenarios affecting the face sheet, interface, and core. A numerical approach was validated using experimental results. After that the simulation was used to generate the data-set to train the neural network. This allowed for the generation of a data-set that fulfill the requirements necessary for a proper data-set mentioned before^[19].

Applying Neural Networks

The study showcased the recognition of features extracted from Non-Destructive Evaluation (NDE) damage detection techniques, employing a neural network. Diverse formats of information representation (vectors and bitmaps) were considered, accounting for potential data contamination. To mitigate these issues, a statistical-based approach employing a Bayesian classifier was implemented. Specifically, the Probabilistic Neural Network (PNN), a subclass of Bayesian classifiers, was applied for pattern recognition^[19].

The comprehensive neural network architecture, comprising six PNN subsystems along with pre and post-processing algorithms, was described. Notably, training data for the PNN was sourced from numerical simulations, rather than empirical data, allowing for a wide array of scenarios for training purposes. Preprocessing algorithms rooted in digital signal processing were integrated to refine PNN performance, including techniques tailored to thermographic images^[19].

The technique was rigorously tested with single and multiple damage scenarios utilizing curvaturebased and thermographic analysis. The results showcased the NN’s adeptness in identifying debonding and small face sheet perforations, highlighting its capacity to discern damage scenarios beyond the scope of the training data. The combined utilization of curvature-based and thermographic analysis effectively minimized spurious diagnostics, ensuring a robust diagnostic outcome. When both techniques were employed in conjunction with a Bayesian probabilistic neural network, the curvature and thermal analyses complemented one another, enhancing the overall damage detection capabilities for sandwich composite materials. This combined approach facilitated the identification of damage type, localisation, and extent. While other methods could also have been employed, the chosen techniques were deemed applicable to various sandwich configurations commonly encountered in ship hull construction^[19].

while the presented study did not use solely thermographic NDT to train the neural network to predict the presence of failure in CFRP sandwich plates, it did show the possibilities and strength of applying NNs as a supporting tool to NDTs.

Infrared Thermal Imaging-Based Turbine Blade Crack Classification

General Method

Jaeger Et al.^[20] presented a more thorough analysis of the possibilities of NNs in NDTs focusing on an application in the aerospace sector. The goal was ensuring the safety and optimal operational performance of aircraft engines, with a specific focus on turbine blades. These components, subjected to extreme conditions and chemical impurities, demand advanced NDT methods for their production. The trailing edge of turbine blades, notably thinner than the leading edge, is particularly susceptible to incipient crack propagation. Conventional eddy current testing can yield inaccurate results in this area due to lift-off signals^[20].

Applying Neural Networks

The data-set was obtained through the application of pulsed induction thermography in conjunction with an infrared camera. Subsequently, NDT inspection specialists classified the images into two distinct categories: crack-free and images containing cracks, which may comprise one or multiple defective spots. These images, presented in 16-bit grayscale format, possess dimensions of 512×640 pixels, representing phase images ^[20].

The data-sets for both models were partitioned into three segments: the training set, validation set, and test data-set. The training set is employed for model training, while the validation set serves to enhance the model’s performance throughout the training process. This methodology, known as crossvalidation (CV), provides a statistical framework for evaluating machine learning models. Utilizing CV results, including metric values and visualized learning curves, enables adjustments to the model’s hyperparameters or modifications to the model architecture. Subsequent to model development, a conclusive evaluation is conducted on the remaining test data-set, which remained entirely distinct from the training process and was never utilized in the model’s training^[20].

The study employed a ResNet-18 convolutional neural network (CNN) in two models: one trained on large images (Large Image Model—LIM) and the other on small image patches (Small Image Model—SIM) to classify turbine blade images as either crack or crack-free. Notably, this research represents a pioneering application of deep learning for defect classification in thermographic images of aircraft engine turbine blades^[20].

Several challenges were encountered. The large dimensions of the images (512 × 640 pixels) and the small size of cracks in the turbine blade trailing edges (ranging from approximately 8 × 6 to 8 × 34 pixels) posed significant difficulties. These cracks occupy less than 0.1% of the total pixel count of an image. Additionally, the identification of cracks requires specialized training. The need to convert 16-bit grayscale images into a false-color representation for effective analysis further complicated the process. Moreover, the availability of a relatively small data-set, compounded by a data imbalance, presented challenges in model training. Furthermore, the knowledge of crack positions was only available for 40% of the total crack images, particularly limiting when cropping smaller partial image patches from the original high-resolution images due to potential loss of context. This scarcity of openly available data sets, attributed to the protection of original equipment manufacturers’ intellectual property, further highlighted the complexity of the task at hand^[20].

The resulting performance of the LIM, as assessed on the validation set, showed only a slight improvement in accurately predicting crack images compared to random guessing (with a recall and precision of 0.60). Conversely, the SIM, operating with significantly smaller image patches, achieved a notably higher recall of 0.93. The precision value obtained was 0.62, with a greater emphasis placed on achieving a strong recall. Although this precision value may appear relatively low, it is not a significant concern, as prioritizing fewer false negatives (high recall) over fewer false positives (high precision) is more crucial. It is important to note that the SIM does not have the capacity to employ kfold cross-validation, a technique that can substantially enhance a model’s generalization performance, particularly for small data-sets^[20].

This Study showed in more details the possibilities of applying machine learning algorithms in NDT. At first the developed framework can be seen as a supportive tool to the examiner with further development promising more automated procedures for NDT methods.

Conclusion

The combination of lock-in thermography and deep learning represents a new method in non-destructive testing for CFRP plates. This integrated approach offers a powerful and efficient means of detecting and characterizing subsurface defects. As industries continue to rely on CFRP materials for their lightweight, high-strength properties, the utilization of deep learning in lock-in thermography presents a method of precise and reliable structural integrity assessment. With its potential to revolutionize inspection practices, this combination holds promise for enhancing the safety and performance of critical engineering components in various industries.

Future Developments

Establishing a functional pipeline based on the aforementioned steps serves as an excellent initial step in implementing neural networks within the realm of Non-Destructive Testing (NDT). However, to further enhance the possibilities, a range of additional measures can be undertaken. These include:

• Developing a Graphical User Interface (GUI): Creating a user-friendly GUI can greatly facilitate interactions with neural networks. This interface would support scientists and engineers who lack prior familiarity with neural networks to harness their capabilities effectively.
• Expanding Data Integration for Neural Networks: Beyond the current scope, there is potential to broaden the neural network’s data sources to encompass inputs from other NDT techniques, such as ultrasound testing. This expansion could amplify the network’s versatility.
• Integration with Predictive Maintenance Techniques: Integrating the outcomes generated by neural networks into predictive maintenance methodologies offers the potential to determine the eligibility of a component for continued use or identify the specific maintenance actions required. This fusion of AI-driven insights with established maintenance practices can be a pivotal step in optimizing asset management and operational efficiency.

Literature

1. Baldev Raj, Tammana Jayakumar, and M Thavasimuthu. Practical non-destructive testing. Woodhead Publishing, 2002.
2. Sandeep Kumar Dwivedi, Manish Vishwakarma, and Akhilesh Soni. Advances and researches on non destructive testing: A review. Materials Today: Proceedings, 5(2):3690–3698, 2018.
3. Nabil A Fouad and Torsten Richter. Leitfaden Thermografie im Bauwesen: Theorie, Anwendungsgebiete, praktische Umsetzung. Fraunhofer IRB Verlag, 2007.
4. Jianguang Guo, Xiaorong Gao, Ewald Toma, and Udo Netzelmann. Anisotropy in carbon fiber reinforced polymer (cfrp) and its effect on induction thermography. Ndt & E International, 91:1–8, 2017.
5. BCF Oliveira, AA Seibert, VK Borges, A Albertazzi, and RH Schmitt. Employing a u-net convolutional neural network for segmenting impact damages in optical lock-in thermography images of cfrp plates. Nondestructive Testing and Evaluation, 36(4):440–458, 2021.
6. Clemente Ibarra-Castanedo and Xavier PV Maldague. Infrared thermography. In Handbook of Technical Diagnostics: Fundamentals and Application to Structures and Systems, pages 175–220. Springer, 2013.
7. Junyan Liu, Wang Yang, and Jingmin Dai. Research on thermal wave processing of lock-in thermography based on analyzing image sequences for ndt. Infrared Physics & Technology, 53(5):348– 357, 2010.
8. Takahide Sakagami and Shiro Kubo. Applications of pulse heating thermography and lock-in thermography to quantitative nondestructive evaluations. Infrared Physics & Technology, 43(3- 5):211–218, 2002.
9. Alexander Dillenz, Gerhard Busse, and Datong Wu. Ultrasound lock-in thermography: feasibilities and limitations. In Diagnostic Imaging Technologies and Industrial Applications, volume 3827, pages 10–15. SPIE, 1999.
10. Faizul M Mohee, Adil Al-Mayah, and Alan Plumtree. Anchors for cfrp plates: State-of-the-art review and future potential. Composites Part B: Engineering, 90:432–442, 2016.
11. Gerhard Mook, Juergen Pohl, and Fritz Michel. Non-destructive characterization of smart cfrp structures. Smart Materials and Structures, 12(6):997, 2003.
12. O Putkis, RP Dalton, and AJ Croxford. The influence of temperature variations on ultrasonic guided waves in anisotropic cfrp plates. Ultrasonics, 60:109–116, 2015.
13. Wenyuan Li, Yu Huang, Long Chen, Xinghua Chen, Guojun Zhang, and Youmin Rong. Effect of anisotropy on the quality of laser cutting corner of cfrp plate. Journal of Manufacturing Science and Engineering, 144(11):111003, 2022.
14. Stefan B Lindstr¨om, Hannes Wemming, Zlatan Kapidˇzi´c, Mohamed S Loukil, and Mikael Segers¨all. Integrated digital image correlation for mechanical characterization of carbon fiber-reinforced polymer plates. Composite structures, 305:116501, 2023.
15. RD Adams and PDRD Cawley. A review of defect types and nondestructive testing techniques for composites and bonded joints. NDT international, 21(4):208–222, 1988.
16. Coursera. Supervised machine learning: Regression and classification. https://www.coursera.org/learn/machine-learning. Accessed: 2023-09-12.
17. Coursera. Convolutional neural networks. https://www.coursera.org/learn/convolutional-neural-networks. Accessed: 2023-09-12.
18. Muhammad Saleem and Hector Gutierrez. Using artificial neural network and non-destructive test for crack detection in concrete surrounding the embedded steel reinforcement. Structural Concrete, 22(5):2849–2867, 2021.
19. Frederick Just-Agosto, David Serrano, Basir Shafiq, and Andres Cecchini. Neural network based nondestructive evaluation of sandwich composites. Composites Part B: Engineering, 39(1):217– 225, 2008.
20. Benedict E Jaeger, Simon Schmid, Christian U Grosse, Anian G¨ogelein, and Frederik Elischberger. Infrared thermal imaging-based turbine blade crack classification using deep learning. Journal of Nondestructive Evaluation, 41(4):74, 2022.
21. Mahtab Mohtasham Khani, Sahand Vahidnia, Leila Ghasemzadeh, Y Eren Ozturk, Mustafa Yuvalaklioglu, Selim Akin, and Nazim Kemal Ure. Deep-learning-based crack detection with applications for the structural health monitoring of gas turbines. Structural Health Monitoring, 19(5):1440–1452, 2020.
22. David Serrano, Frederick A Just-Agosto, Basir Shafiq, and Andres Cecchini. The use of neural networks to detect damage in sandwich composites. Major Accomplishments in Composite Materials and Sandwich Structures, pages 407–429, 2009.
23. Liming Jiang, Yuanchang Xie, and Tianzhu Ren. A deep neural networks approach for pixel-level runway pavement crack segmentation using drone-captured images. arXiv preprint arXiv:2001.03257, 2020.