[18WS - BA] Development of a Game with Physiotherapical use with help of AR and VR Technologies

Abstract

A physiotherapy treatment usually involves multiple sessions and repeating same set of exercises depending on the type and severity of the injury. Due to repetitiveness of the treatment, patient motivation and interest in the therapy decreases over time. As a result effectiveness of the treatment also declines. A solution to this problem is using motivational and behavioural effects of games to improve the therapy.By turning the therapy session into a fun experience patient engagement to the session can be enhanced. More motivated and engaged patients also promote the efficiency of the therapy. With the help of the technological developments in the fields of AR, VR and motion tracking, physical aspects of gameplay can be turned into rehabilitative exercises. With this concept in mind aim of this project is to create a game that is usable as a therapy tool. The game focuses on upper body and hand exercises. In the game the player is in a kitchen and has to grab ingredients from the shelves or from the counter and put them into the bowl in front of him/her. By using the kinect camera to track the movements of the player, the game allows the player to exercise stretching and grabbing movements.

Development

In the development of the game as game engine the Unity game engine, as motion capture camera Kinect 2.0 and as VR headset Oculus Rift has been used. Goal was to correctly recognize the hand and arm movements and gestures of the player by using the Kinect camera. To fix different position values from Kinect and Oculus sensors, Sensors set as close as possible and only head rotation values from Oculus sensor has been used. The body movements are solely calculated by Kinect sensor. By correctly recognizing the gestures such as grab and reach, the game can be used in treatment of patient with upper extremity problems. The game involves moving the ingredients, that are spawned around the kitchen, to the cooking pot which is directly in front of the player by grabbing them and carrying them around. Throughout testing there were a lot of false negatives from Kinect sensor when the object is carried in front of the player. Sensor returned the hand state as unknown even though the correct grabbing movement has been done. That is why the planned patient testing has been delayed until these problems are solved. Game is still being developed and the goal is to create an end product that can be used as a valid therapy tool.

Thesis Download