Optical Motion and Deformation Tracking

Marvin Hofmann, winter semester 2021/2022

Optical motion and deformation tracking describes an approach to gain objective data analog to manual visual inspection. Camera based tracking systems are the focus of this article. Laser recording and optical fiber are also optical tracking approaches and will be compared with the properties of camera based systems. Optical flow visualization is a specialized technique. As a conclusion chances for NDT application will be shown. 

1 Camera based tracking systems

The most important optical tracking systems are digital image correlation (DIC) and motion capturing systems. The technologies are based on optical measurement techniques. DIC focuses on the comparison of images to detect and measure deformations. In contrast motion capturing systems focus on the tracking of moving objects. This separation is based on the original purpose and application of the techniques. Due to the increasing number of application and the continuous improvement technical possibilities, a rigid differentiation between these systems is not possible.

1.1 Physical principle

Both systems use at least one camera. Depending on the technology different types of cameras can be used such as standard video, high-speed, infrared cameras. One camera is sufficient for 2D measurements but for 3D application at least two cameras are required. The first images are captured to document the origin. More images are taken at a different time step in order to capture movements, deformation and further changes of the object. The basic principle is rather simple, but it depends on many properties of the technical equipment and the parameter of the specific application.

Table 1: overview of properties and output data

[1], [2], [3]

1.2 Digital image correlation

Digital image correlation can be used for measurement of displacements and strains. The basic idea is comparing two images of an object at different time steps. Therefore, the object is tracked by cameras. The pictures are analyzed by an algorithm which compares the pictures due image texture structure. This algorithm needs to fulfill the specific requirements of the task. In many cases no special light conditions or surface preparations are sufficient for good measuring results. The basic principle is visualized in figure 1. The captured object is represented by the blue square. The surface texture has specific properties shown by the black dots. The object is captured at the original time step t₀ and and one time step further t_1. The postprocessing algorithm can track the movement or deformation by comparing the position of the black dots which represent the surface texture of the object.  [3], [4]

Figure 1: priciple of digital image correlation inspired by [3]

1.3 Motion capture systems

Motion capture (mocap) is the process of recording the movement of objects or people. It involves measuring the position as well as the orientation. In general, it can be distinguished between optical active and passive in combination with infrared cameras and video or non-optical. Optical active describes the use of active markers on the surface of the object which send signals. In contrast optical passive markers just respond to the stimulus of the environment. Markerless video systems tend to have less accuracy, less reliability and require advanced software methods for the analysis of the data. In practice marker free methods are sufficient for many applications and the usage of markers is not always possible. Figure 2 shows a square as a model of an arbitrary object. This object has three markers on its upper surface. The object is moving and tracked by two cameras. These cameras capture the movement of the markers. The results are analyzed on a computer. [5], [6]

Figure 2: Principle of motion capturing

1.4 Advantages of camera based optical tracking systems  

Even a simple optical-passive motion capture system supplies reliable and accurate data. Optitrack offers a positional accuracy of 0.2 mm and a system latency of 10 ms. The calibration is automated. In practice this helps to efficiently gain reliable and reproducible data. The cost for a system starts around 2500 $ which makes it affordable for a broad range of application. Even complex movements can be captured efficiently. [7], [8]

1.5 Application of camera based optical tracking systems

Optical motion capturing systems were used for virtual production such as video games and animated movies. Data for virtual reality application are also captured by motion capture systems. Movement and life sciences also use camera-based motion tracking. Analog to the entertainment applications the technology helps to analyze complex movements and build models of the reality. Another important broad field for application is medical technology. For example, optical tracking systems are used to support cardiac surgery by virtual stabilization. The heartbeat causes complex relative movements between the blood vessel and the surgery instruments. The principle of DIC can be used to analyze the development of a disease like cancer by comparing pictures at different times.

Optical tracking is also interesting for a lot of automatization approaches and due to the low latency, the technology can also be used for critical real time applications such as autonomous driving or drones. The data are an essential part of the input data for artificial intelligences in mobility. Another economically important application is in logistics. The InfraLok project of the Frauenhofer institute uses the data for optimization logistic systems based on localization and tracking of vehicles. [8], [9], [10], [11], [12], [13]

1.6 Commercially available devices

Table 2: expandable list of commercially available devices based on [8], [12], [14], [15], [16], [17], [18] 

2 Comparison with laser recording

Laser measurements are another state-of-the-art measuring technique with broad range of application possibilities based on the principle of laser-based triangulation measurement. The distance between the laser and the object is one important parameter of the measurement. Together with the angle of the scattered laser beam hitting the detector and the parameter of the equipment, changes can be detected, and the position can be evaluated. The object is scanned by a concrete number of focused laser beams. This results in a high dependency on the result of this ultimate amount of laser beams which is a significant difference to the DIC and mocap. Therefor one drawback is the dependency on surface quality. Additional preparation such as surface preparation might be necessary. DIC and mocap systems are also more cost efficient and easy to use in outdoor environment. [19], [20], [21], [22].

3 Comparison with optical fibers

The principle of fibre-bragg-grid-sensors is an optical way of non-destructive testing and deformation tracking. The optical fibers are implemented into the part in order to measure one dimensional strains and deformation. The wavelength increases with deformation or higher temperature. This change can be detected. In contrast to the first methods this technique needs to be considered during manufacturing for an easy implementation or requires a reasonable set-up effort. The fiber is small, light, resistant to environmental circumstances and cheap. Analytic devices are required for measuring results and they cause significant cost. Optical fibers can also be used for vibrational analysis. Incorporated as sensor device the optical fibers can be used to measure deformation in difficult accessible areas, for example in minimal invasive surgery. [23], [24], [25]

4 Optical Flow Visualization

Flow vis is a technique used in formula one to capture aerodynamic flow phenomena. This technique is useful for validation of complex aerodynamic racecars in comparison with fluid dynamic simulation in addition to wind channel testing. A paint like substance is applied to aerodynamic surfaces and gets its final shape during a testing session on the racetrack. As the solvent of the paint evaporates, it takes a snapshot of the flow condition during a specific situation and therefore tracks the aerodynamic flow [26]. 

This visualization techniques gets interesting if you consider the aerodynamic flow reacts sensitive to changes of the structural properties of its surface. In aviation oscillations of functional surfaces due to damage could be detected at an early stage by flow visualization. The main drawback is the high knowledge you need about the desired flow conditions and the complex application of the flow vis paint.

5 Application in non-destructive testing

The broad field of application of optical motion and deformation tracking shows the chances for non-destructive-testing approaches. The physical principle is purely non-destructive. The technology of optical motion and deformation tracking therefor can be used for the inspection of buildings, automotive, aerospace environment and many other engineering disciplines such as visual analysis of vibrating structures.  

5.1 NDT for buildings and bridges

Optical tracking technology is robust against rough environmental conditions such as wind, gusts or earthquakes as well as fatigue behavior. Additionally, they are easy to use so that no highly educated personal is required for measurements. In combination with the comparable affordable equipment this results in a cost-efficient approach for the inspection of buildings and bridges. Therefore, it should be considered when choosing an approach for structural health monitoring of buildings and bridges

One possible issue might be the position of the camera. As it will be mounted on the bridge itself the camera will also move. This movement needs to be considered in order to get accurate and reliable results. Camera based methods come along with the challenge of different illumination conditions. Nevertheless, the technology has the potential to substitute the current state of the art technologies of wired measuring technology and broad application in commercial health monitoring Laser Tracking is an alternative optical approach to monitor buildings. [27], [28], [29], [30]

5.2 Aerospace and wind turbine in life testing

Aerospace and wind energy need to monitor the structural health during application. Due to the damage tolerant design approach monitoring is part of the design philosophy. There is a huge interest as the structures could be used longer and at a higher load level if the monitoring reliability can be increased. Computer vision-based approaches track the surface during crack opening and can therefore capture fatigue cracks by discontinuities in the surface. Furthermore, it is possible to use DIC for inspection, structural health monitoring and full-field vibration testing for wind turbine blades. [30], [31]

5.3 Vibration analysis

New technical possibilities make high speed deformation analysis possible. The high resolution of the deformation measurements in space and time by high-speed cameras allows high frequency vibrational analysis for harmonic and transient vibrations. Conventional measuring techniques require special equipment. Vibrational analysis can therefore be used as a non-destructive method for any system that is vibrating. Combustion engines, electrical machines, turbines can be analyzed by optical motion tracking. Any damage will lead to deviation of the vibrational properties and can therefore be captured by the system. A precise knowledge of the desired state is mandatory to analyze any possible deviation from this state. The system is still easy to use and comparatively cost efficient which makes it interesting compared to the conventional monitoring approaches. [2], [32]

References

1. Mayinger, F., Optical measurements: techniques and applications. 2013: Springer Science & Business Media.
2. Siebert, T., R. Wood, and K. Splitthof. High speed image correlation for vibration analysis. in Journal of Physics: Conference Series. 2009. IOP Publishing.
3. Hannover, G.W.L.U. Digital Image Correlation (DIC). 2021. Available from: https://www.tfd.uni-hannover.de/de/forschung/leistungsspektrum/messtechnik/digital-image-correlation/ (11.03.2022).
4. McCormick, N. and J. Lord, Digital image correlation. Materials today, 2010. 13(12): p. 52-54.
5. Janabi-Sharifi, F. and W. Melek, Advances in Motion Sensing and Control for Robotic Applications. 2019: Springer.
6. Ong, A., I.S. Harris, and J. Hamill, The efficacy of a video-based marker-less tracking system for gait analysis.Computer methods in biomechanics and biomedical engineering, 2017. 20(10): p. 1089-1095.
7. Aurand, A.M., J.S. Dufour, and W.S. Marras, Accuracy map of an optical motion capture system with 42 or 21 cameras in a large measurement volume. Journal of biomechanics, 2017. 58: p. 237-240.
8. Optitrack. Optitrack. 2022. Available from: https://optitrack.com (11.03.2022).
9. Lau, W.W., et al. Stereo-based endoscopic tracking of cardiac surface deformation. in International Conference on Medical Image Computing and Computer-Assisted Intervention. 2004. Springer.
10. Becker, M., et al., Analysis of myocardial deformation based on pixel tracking in two dimensional echocardiographic images enables quantitative assessment of regional left ventricular function. Heart, 2006. 92(8): p. 1102-1108.
11. Optical Tracking - InfraLok. 2022. Available from: https://www.iis.fraunhofer.de/de/ff/lv/lok/opt.html(11.03.2022).
12. Vicon Motion Systems. Vicon. 2022. Available from: https://www.vicon.com (11.03.2022).
13. Trilion Quality Systems. Aramis. 2022. Available from: https://www.trilion.com/aramis (11.03.2022).
14. Qualisys. Applications. 2022. Available from: https://www.qualisys.com/applications/ (16.03.2022).
15. Mo-Sys. StarTracker. 2022. Available from: https://www.mo-sys.com/product/camera-tracking/startracker/(16.03.2022).
16. DantecDynamics. Solid Mechanics DIC. 2022. Available from: https://www.dantecdynamics.com/solutions-applications/solutions/stress-strain-espi-dic/digital-image-correlation-dic/ (16.03.2022).
17. Isi-Sys. Vic-3D Educational System. 2022. Available from: http://www.isi-sys.com/category/products/dic-systems/ (16.03.2022).
18. Lavision. Cameras for DIC. 2022. Available from: https://www.lavision.de/en/products/cameras/cameras-for-dic/index.php (16.03.2022).
19. AG, L.G. Laser Tracker Systeme. 2022. Available from: https://leica-geosystems.com/de-de/products/laser-tracker-systems (11.03.2022).
20. Hexagon. Laser Tracker Systems. 2022. Available from: https://www.hexagonmi.com/products/laser-tracker-systems (11.03.2022).
21. Hexagon. Aircraft Structure. 2022. Available from: https://www.hexagonmi.com/solutions/industries/aerospace/aircraft-structure (11.03.2022).
22. Reihl, C. Laser-based triangulation measurement. 2019. Available from: https://wiki.tum.de/display/zfp/Laser-based+triangulation+measurement (11.03.2022).
23. Garcia, Y.R., J.M. Corres, and J. Goicoechea, Vibration detection using optical fiber sensors. Journal of Sensors, 2010.
24. Krapp, M. Faser-Bragg-Gitter Sensoren. 2011. Available from: https://wiki.tum.de/display/zfp/Faser-Bragg-Gitter+Sensoren (11.03.2022).
25. Marchi, G., et al., Fiberoptic microindentation technique for early osteoarthritis diagnosis: An in vitro study on human cartilage. Biomedical Microdevices, 2019. 21(1): p. 1-9.
26. McLaren. Flow-Vis. 2022. Available from: https://www.mclaren.com/racing/f1-playbook/flow-vis/(11.03.2022).
27. Rommel, M. Structural health monitoring of bridges - general strategies and measurands. 2013. Available from: https://wiki.tum.de/display/zfp/Structural+health+monitoring+of+bridges+-+general+strategies+and+measurands (11.03.2022).
28. Abolhasannejad, V., X. Huang, and N. Namazi, Developing an optical image-based method for bridge deformation measurement considering camera motion. Sensors, 2018. 18(9): p. 2754.
29. Brownjohn, J.M.W., Y. Xu, and D. Hester, Vision-based bridge deformation monitoring. Frontiers in Built Environment, 2017. 3: p. 23.
30. Attanayake, U., A. Servi, and H. Aktan. Noncontact bridge deformation monitoring using laser tracking technology–technology evaluation and field implementation. in Structural Materials Technology 2012. 2012.
31. Carr, J., et al., Dynamic stress–strain on turbine blade using digital image correlation techniques part 1: static load and calibration, in Topics in Experimental Dynamics Substructuring and Wind Turbine Dynamics, Volume 2. 2012, Springer. p. 215-220.
32. Huňady, R., P. Pavelka, and P. Lengvarský, Vibration and modal analysis of a rotating disc using high-speed 3D digital image correlation. Mechanical Systems and Signal Processing, 2019. 121: p. 201-214.