A Human Machine Interface for Serious Games applied to the Rehabilitation of Individuals with Parkinson’s Disease

This work presents a research in progress, which proposes to develop a low-cost wearable device based on Inertial Measurement Units (IMUs) to be used as an interface to the control of a serious games used for wrist rehabilitation of individuals with Parkinson’s disease. The device must track the movement of the hand and send that information to a computer, running a serious game, to control a virtual object. The device uses IMUs to estimate the orientation of the hand. The results obtained in laboratory are very promising, as they show, in comparation to the goniometer (standard reference), good precision (error SD < 2o) and moderate accuracy (error between ±5o). As well as the human machine interface (HMI) performance on track movement showed to be reliable. Keywords— Parkinson’s disease, serious games, inertial measurement units , human machine interface, rehabilitation.


I. INTRODUCTION
Parkinson's disease (PD) is a neurodegenerative disorder that causes neurological symptoms such as bradykinesia, rigidity and tremor [1]. PD is chronic, progressive [1] and still uncurable. The treatments of PD have different approaches like functional rehabilitation (e.g., physiotherapy), medication and surgery.
Rehabilitation can be seen as an adjustment to pharmacological and surgical treatment for PD to maximize functional ability and minimize secondary complications, leading to a better quality of life [2].
In this context, serious games are used for education and training. When employed for rehabilitation they aims to motivate patients to perform their rehabilitation exercises [3].
Whether the rehabilitation is based on serious game a HMI (Human Machine Interface) appears as an important part of this process, that is, the user must interact with the game in a properly way, for the proposed rehabilitation program to be successful.
Depending on the objectives of the rehabilitation and the serious game, a different kind of HMI could be chosen. Although, there are commercial off-the-shelf game sensors, in a specific case of PD rehabilitation with a customized serious game, there are some advantages in develop a customized device to capture human motion. One of them is cost, commercial gaming system are expensive and it is possible to avoid expenses developing an own HMI. Another one, is that with a proprietary technology is easier to achieve a system without limitations as experienced by commercial ones, for instance: do not always require the full extension of movement or can induce frustration as game systems may be too difficult for a population needing rehabilitation [4].
In this work we propose a human machine interface to measure the wrist joint angles created by the hand movements such as pronation/supination, extension/flexion and adduction/abduction. The hand movements for human computer interaction were chosen for a specific treatment of individuals with PD and are based on Unified Disorder Society-Unified Parkinson's Disease Rating Scale(MDS-UPDRS) [26].

II. MATERIAL AND METHODS
The HMI is a wearable device, i.e., a glove, developed specifically to be used by a serious game for wrist rehabilitation, which will control a virtual object (bee) by hand's movements. In this context, the HMI used in this project is a glove that will be used to get the wrist movements.
The IMU (Inertial Measurement Unit) is attached on the back of the hand, so the movement of the hand is captured by the sensors which yield signals for the estimate of the joint wrist angle pitch, roll and yaw. These angles will be used in a serious game to control a virtual object. Figure 1 shows the prototype implemented up to now.
The proposed strategy to develop the HMI employs IMUs that give the orientation information of a rigid body.

A. Orientation of a Rigid Body
A rigid body, free on a 3D space, can be considered as positioned in a coordinate system with 3 axis (x, y, z). A Reference Frame (Fr), which is the position (x0, y0, z0) must be set for the correct estimate of the position of object ( Figure 2a).
The orientation of a rigid body can be represented as a rotation of the body from its Fr ( Figure 2b) to a new frame (Body Frame) about an angle θ. For instance, a rotation over the vector (1, 0, 0) as showed in Figure 2b.
There are several forms to represent rotation such as Euler angles, pitch/roll/yaw angles, direct cosine matrix (DCM), and quaternions. In this work we use pitch, roll, and yaw angle estimated from quaternions.

B. Human Machine Interface
The developed HMI has a microprocessor (Esp32 Wroom module) that communicates with IMU (MPU6050) via I2C protocol. Esp32 reads the quaternion information processed by a Digital Motion Processing (DMP) from the MPU6050 FIFO, and an external magnetometer (QMC5883L).
MPU6050 has a 3-axis accelerometer and a 3-axis gyroscope. Data fusion is processed by DMP that fuses the accelerometer and gyroscope data to yield the orientation in quaternion format.
An external magnetometer (QMC5883L) was incorporated in the system. At first, it was thought to be sufficient only the MPU6050, however an error estimate of rotation angles over the z axis proved to be necessary to implement an electronic compass that fuse accelerometer and magnetometer values to yield head (north), i.e., a reference angle based on the geomagnetic field.
Therefore, this head is used to correct the error on estimated angle over the z axis.
The configuration of DMP (on MPU6050) was set with sensitivity ±2g (accelerometer) and ±2000dps (gyroscope), Digital Low Pass Filter 184Hz (accelerometer) and 188Hz (gyroscope). The Digital Low Pass Filter is intrinsic to the DMP and the highest band was chosen for this first approach. For external magnetometer sensitivity ±8G. The sample rate was set to 100Hz.
Quaternion is used to obtain the yaw, pitch, and roll angles, and this information is packed in a User Datagram Protocol (UDP) messages and send from the HMI to a computer executing the serious game via Wi-Fi link.

C. Estimating Pitch and Roll angles from Accelerometers
Suppose the MPU6050 is in industry standard "NED" (North, East, Down) coordinate system as shown in Figure  3, positioned in a surface without any external acceleration, the accelerometer measures the gravity acceleration. The readings, Gr, of an accelerometer in this start position is shown in Equation 1: Where g is the acceleration due to gravity, i.e., g = 9.81 ms -2 [5].
The readings, Gp, of an accelerometer subjected rotations Rz(ψ), Ry(θ)and Rx(ϕ) can be estimated as in Equation 5 [5]: The pitch (θ) and roll (ϕ) angles can be estimate multiplying Gp obtained in Equation 5 by the inverse rotation matrixes of the Equations 3 and 1, respectively, as shown in Equation 6: where the vector ( ) contains the three components of gravity measured by the accelerometer [5].
Expanding the Equation 6 we obtain Equation 7: The roll angle ϕ is defined by the "y" component of the Equation 7 [5] -Equation 8: The pitch angle θ is defined by the "x" component of the Equation 7 [5] -Equation 9:

D. Estimate Yaw angle from a Gyroscope
Basically yaw angle (ψ) is estimated from gyroscope integrating once the angular velocity measured by the sensor [6] as shown in Equation 10: where ψ0 is the yaw angle in the initial position.

E. Quaternion
As mentioned before, DMP on the MPU6050 gives the orientation of the IMU in a quaternion format.
Quaternions are an extension of complex numbers that can be regarded as an element of ℜ 4 [7] represented by Equation 11: where w is the scalar component and {a, b, c} forms the vector part [8]. A unit quaternion can be used to represent a rotation of an angle θ about an unit axis n in three dimensional space [9] -Equation 12: Quaternions have been used in many scientific fields, for instance, in computer graphics, and it is a compact way of representing an orientation of an object [10].

G. Electronic Compass
As previously presented, the yaw angle (ψ) is obtained from the integration of gyroscope measurement values. The integration causes an accumulative gyroscope measuring error in time. This phenomenon known as "drift" makes impossible to estimate the rotation angle over z axis, in long term.
To avoid this kind of error an "Ecompass" was implemented fusing the values measured from accelerometer and magnetometer to get a head (north). Figure 4 shows the drift on the yaw angle, estimated just from gyroscope during time, and the yaw angle with the error compensated by the "Ecompass". Figure 5 shows the diagram of the algorithm implemented to correct the drift error. The component "YawE(0)" is the initial value of the "Ecompass" in time t0.

H. HMI Calibration
The device was attached on a structure, mounted on laboratory, that rotated the system. For the x and y axis, the rotation range was from -90º to +90º and for the z axis, the rotation range was from -50º to +50º.
Those ranges were defined based on the maximum wrist's angles in the movements of flexion/extension, supination/pronation and abduction/abduction defined on goniometry practice [11].
The angles estimated from the HMI were contrasted with angles measured by a goniometer (standard). A linear regression was performed to fit a linear model for studying the relationship between measurements.

I. HMI Validation
To validate the system, we used a test bench structure that rotated the HMI on axis x, y and z. This structure was mounted to get better precision on rotation movements. Two tests were carried out , one to get accuracy and precision compared to a standard and other to define a performance of the system.  For the accuracy and precision test, the HMI was rotated first over the x axis, second, over the y axis, and finally over the z axis. In each rotation the device covers the range from -90º to +90º (-50º to +50º for axis z) by 10º step (one trial).
Ten trials were performed for each axis, so we obtained 190 measures (scores) for x and y axis, and 110 measures for z axis. The angles measured by the system was compared to the goniometer's angles. The error (system angle -standard angle ) between goniometer's angles and HMI's angles were calculated . A Shapiro Wilk test was performed to confirm a normal distribution of data, and after a probability distribution statistic was calculated to determine the accuracy and precision of the device.
For performance test, the HMI was rotated from about 0º (zero degrees) to about +90º (+50º for axis z) more than thirty times for each axis. The aim of this test was evaluating if the device would be able to reliably track the movement.

A. Accuracy and Precision
Data analysis was performed in R. The Shapiro Wilk test confirmed the normality of the data (x axis: p-value = 0.1929; y-axis: p-value = 0.5161; z-axis: p-value = 0.3770) at α = 0.05.
The errors calculated compared with the reference standard reported a good precision, standard deviation (SD < 2º). A normal distribution of probability was calculated. The probability of an error equal or greater than +5º or -5º showed to be very low, for each axis, reporting a moderate accuracy (error between ±5º). Table 1 presents the mean, SD and probability of errors on axis x, y and z.

B. Performance
The Figure 6 shows the signal measured when the HMI was moving from 0º through +90º (+50º for axis z) more than thirty times, for x, y and z axis.
For the first movement the maximum angle was recorded. For x and y axis a threshold was defined at 95% of this maximum angle and for z axis at 85%. Those thresholds were determined by signal observation. Thus, it was possible evaluate the performance of the system while tracking the movement. All angles at the end of the movement were estimated above the threshold showing that the device has a good performance.
For the axis z the performance (threshold at 85% instead 95%) was worse, probably due to noise presence on the signal.

IV. DISCUSSION
Orientation of a rigid body is important in many areas, such as aerospace engineering, robotics kinematics, computer graphics. This information is also useful in Biomedical Engineering. So joint angle can be measured, and this information can be used to assess, and monitor motor signals in individuals with PD.
Hence, an HMI especially designed to be used with a serious game in a PD rehabilitation task is interesting, because it provides more usability for the game.  This work showed that a wearable device to measure the orientation of the hand can be developed using IMU, with a satisfactory precision and reliability.
A normal distribution of probability of error showed that a probability of an error great than ± 5.0º is almost zero and this performance was confirmed to be sufficient to control the virtual object in game with satisfactory results.
This results showed up accordingly to that reported in a system relying on estimation knee angle using inertial sensors [12]. Nevertheless, algorithms such as Kalman filter can be used to decrease errors in data outputted by the system.
However, depending on the accuracy needed, perhaps the use of more accurate sensors (when compared to MPU6050 and QMC5883L) could be chosen.
The limitation of this research is that until now the HMI was tested only by health individuals and in laboratory, so the performance with individuals with PD must be performed to confirm the usability of the system in a rehabilitation scenario.

V. CONCLUSION
This study proposed a glove design for a serious game rehabilitation. The initial results of data show that our proposed device has good accuracy and stability In that way a low-cost wearable device based on IMUs could be developed to be used as serious games control (HMI) for rehabilitation in PD.
In the future, clinical tests will be conducted, in patients with PD.