sign in sign up
<<
Home , Gyroscope

Personalised stroke rehabilitation intervention using open source 3D software and the Remote Plus

E Tsekleves1, D Skordoulis1, I Paraskevopoulos1, C Kilbride2, A Warland2

1School of Engineering and Design, 2School of Health Sciences and Social Care, Brunel University, Kingston Lane, Uxbridge, UK

[email protected], [email protected]

www.brunel.ac.uk

ABSTRACT The research presented in this paper proposes a novel low-cost customised (VR) based, stroke rehabilitation system for the delivery of motivating rehabilitation sessions and evaluation of performance. The described system is designed to capture and monitor human upper limb motion using a low cost and commercially available and gyroscope device, the and open source 3D software. This is the first project to successfully fuse the Nintendo Wii remote acceleration and gyroscope data to offer a real-time one-to-one representation of the controller in a VR environment. A pilot study established a high degree of user acceptability and high levels enjoyment using the tailor made games and personalised exercises in a chronic stroke survivor. Moreover, positive changes were demonstrated in all four outcome measures employed; of particular note were improved wrist control and greater functional use of the hand.

1. INTRODUCTION Stroke is the largest single cause of long term disability in the United Kingdom (UK). It is estimated that 110,000 new strokes occur each year leading to an annual economic burden of more than £2.8billion (Department of Health, 2010). Upper limb problems are particularly common post stroke with 70% of stroke survivors experiencing significant problems. Treatment interventions effective at improving upper limb movement are by nature intense, highly repetitive and functional. Boredom with such exercises prevails and adherence is problematic (Tijou et al, 2010); with 60-80% of people failing to comply with their prescribed exercise programme (Van Dulmen et al, 2008). This is exacerbated in the community where there are fewer therapists to monitor exercises and people describe feeling unsupported. It can therefore be seen that interventions with the ability to achieve the intensity and repetition levels to promote/drive upper limb recovery which can be monitored and adjusted remotely by specialists could potentially be highly clinically useful. ReWiiRE (Research for Wii Rehabilitation) was an 18-month NHS (National Health Service) funded project (www.rewiire.org.uk). The project aim was twofold. Firstly to explore the current use of the Nintendo Wii console in physical rehabilitation programmes. Secondly to develop a personalised Nintendo Wii stroke rehabilitation intervention together with tailor-made games and therapeutic, functional upper limb exercises in partnership with stroke survivors.

2. RELATED WORK Attempts have been made to use commercial accelerometer devices, such as the Nintendo Wii remote (also known as the “Wiimote”) for rehabilitation purposes. The vast majority of studies have taken place in therapy clinics testing the use of the Nintendo Wii console and the Wii sports game with a limited number of stroke patients (Yong Joo et al, 2010; Saposnik et al, 2010; Williams et al, 2010). These studies along with ours showed that patients found the Wii intervention enjoyable and motivating but described limitations of use, particularly in terms of the duration of the exercises, the speed, a lack of registering the full range of movements (i.e. small movements made by stroke survivors) and the inability to hold the Wii remote. Several studies have developed customised Wii-based interventions. These studies aimed at developing a VR-based system that captures the patient upper limb motion in a more accurate way compared to the

Proc. 9th Intl Conf. Disability, Virtual Reality & Associated 515 Laval, France, 10–12 Sept. 2012 2012 ICDVRAT; ISBN 978-0-7049-1545-9 conventional Wii remote. These can be categorised into three main distinct groups depending on the technical methodology employed for customising the Wii remote technology, namely acceleration data from the Wiimote only (Shih et al, 2008; Leder et al, 2008; Palmke et al 2009; Matamoros et al, 2010; Alankus et al, 2011), the Wiimote as an IR (Infra Red) camera with the aid of LEDs (light-emitting diodes) (Attygalle et al, 2008; Jovanov et al, 2008; Scherfgen et al, 2009; Decker et al, 2009) and a hybrid of the two (Wilson et al, 2007; Martin-Moreno et al, 2008). The first aforementioned group of customised Wii interventions employ one or two Wiimotes with a reverse engineered Application Programming Interface (API) to capture the patient motion by reading the Wiimote accelerometer data. Since these approaches obtain the position of the Wiimote in space using the acceleration data (that is the change in the linear acceleration as the patient moves the Wii remote in space) they suffer from a DoF (Degree of Freedom) limitation. More precisely these solutions offer accuracy only in 2-DoF as the acceleration data can only determine the pitch and roll movement. The second group of customised Wii interventions employ a pair of Wiimotes, which are used as IR cameras detecting motion through the Wii’s sensor bar or some custom LEDs, to build a low-cost motion capture system. The LEDs are usually attached to the patient (by means of a strap, thereby negating the problem of stroke survivor’s ability to hold and operate the Wiimote) and as the exercises are executed, the range of motion is captured and mapped onto the system display. The limitation in this approach is that each Wiimote can detect up to four LEDs in space, thus restricting the range of movement and set of exercises that the patient can perform. Also care must be placed on the view angle of the Wiimotes’ IR camera in order to reduce occlusion of LEDs as they patient moves his/her upper limb. To the best of our knowledge the work presented in this paper is the first to propose the fusion of the Nintendo Wii remote acceleration and gyroscope data for the provision of a real-time personalised stroke rehabilitation intervention.

3. PROPOSED INTERVENTION The developed system as depicted in Figure 1 comprises of three bottom-to- layers, namely data collection, data fusion and motion tracking algorithm, and the 3D visualisation. The data access operation mainly includes functions that allow the application to acquire sensor data sent from multiple Wii remote plus to a personal computer via Bluetooth (in our study two Wiimotes were used per participant but more can be added). In the middle layer, the received information from the Wii remote sensors undergo a process of smoothing and multiplexing using a data fusion algorithm in order to achieve higher accuracy and precision. The end results are mapped into quaternion forms that translate the orientation of a constructed 3D body model and also form the data structure for the top layer. The top layer embraces the dynamic physical simulation of a 3D avatar animation in real-time. For the purpose of the animation, we employ Blender which is an open source 3D application. The angular rate measurements captured by the gyroscope sensor can be used to distinguish true linear motion from the accelerometer readings. The gyroscope is not free from noise, but since the measured is less sensitive to linear mechanical movements and without amplifying hand jitter, both of which suffers from, it allows capturing more complex orientation with a relatively better estimate than we would obtain by using accelerometers alone. A sensible approach for maximising efficiency is to average or concatenate the data that comes from the accelerometer and gyroscope by using a data fusion algorithm and simultaneously, we have been able to employ a smoothing algorithm to remove any excessive noise from the signals while still retaining the useful information. Filtering out and removing as much random noise as possible from the sensors’ output raw information whilst retaining quality data is of fundamental importance. There are various methods for achieving this. Some of them involve very complex computations and offer impressive results whereas others use simple computational methods but lead to initial data being distorted. The choice of selecting the right trade-off depends on the characteristics of the signal (peak position, height, width, area, etc.) and system requirements. For smoothing the fluctuating acceleration values, we utilise a moving average function and for the data from the gyroscope, a five-point window Savitzky-Golay (1964) smoothing filter is used. The Wiimote sensors are very responsive, but they cannot respond to the linear movement accelerometers specialise in. Yet, as described in the above section, when a gyroscope and an accelerometer are combined, the pairing of sensors facilitates a highly accurate one-to-one representation of the control device in 3D space.

516 Proc. 9th Intl Conf. Disability, Virtual Reality & Associated Technologies Laval, France, 10–12 Sept. 2012 2012 ICDVRAT; ISBN 978-0-7049-1545-9 The rationale for the use of an Open Source 3D authoring tool is twofold. On one hand, it provides instant visual feedback of the patient movement that both the patient and therapist can view, store, evaluate and monitor. On the other hand, it creates opportunities for the development of high quality 3D graphical exercises, games and virtual environments that can act as an additional motivational tool. For an exercise treatment to be successful it must be engaging and participative as compliance with on-going physical activity and home exercise programmes are often poor and require behavioural change.

Figure 1. Customised Wii Intervention System Architecture.

4. PRELIMINARY RESULTS The system was incrementally developed and tested by stroke survivors participating in the project. In addition, a single-case, feasibility before and after study was undertaken to field test the system, with a 32 year-old, right- handed, female, who had suffered a right sided stroke 12 years previoously resulting in a left sided hemiplegia. The personalised treatment intervention was undertaken three times a week over a two week period with individual sessions lasting between 30-45 minutes inclusive of rest periods. Clinical change was evaluated using standardised ouuttcome measures: the MAS (Modified Ashworth Scale), MAL (Motor Activity Log, Amount of Use subscale), FMA upper limb section (Fugl-Meyer Assessment) and NHPT (Nine Hole Peg Test). Despite the short duration of the study, siignificant improvements were demonstrrated particularly in the NHPT and MAL and concomitant self-reported functional improvements in everyday activities were noted. The stroke survivor described the activities practised through the personalised Wii system as fun and motivating. The ability to customise the exercises for example being able to alter the the speed and level of difficulty of the games kept her motivated as her capability improved.

5. CONCLUSIONS This paper has presented the development of an innovative system which offers a customised stroke VR- based rehabilitation intervention by using low-cost and off-the-shelf game sensors,, i.e. the Nintendo Wii remote combined with open source 3D softwaare. This is the first project that fuses the Nintendo Wii remote acceleration and gyroscope data together to track the controller movements. The proposed system provides accurate and real-time one-to-one representatiion of the control device in VR space. We are encouraged by the preliminary results and plan to extend the study in terms of both duration and sample size. Furthermore we wish to test the feasibility and acceptability of using the novel intervention for delivering stroke telerehabilitation for community dwelling stroke survivors.

Proc. 9th Intl Conf. Disability, Virtual Reality & Associated Technologies 517 Laval, FFrance, 10–12 Sept. 2012 2012 ICDVRAT; ISBN 978-0-7049-1545-9 Acknowledgements: The authors of the paper would like to acknowledge the NHS London Regional Innovation Fund, which funded the ReWiiRe project and made this work possible. The authors would also like to thank all the stroke survivors who took place in the system evaluation and development as well as our NHS project partner (Mrs Dee Hackett from the Central London Community Healthcare NHS Trust).

5. REFERENCES G. Alankus, R. Proffitt, C. Kelleher and J. Engsberg (2011), Stroke Therapy through Motion-Based Games: A Case Study, ACM Transactions on Accessible Computing (TACCESS), 4, 1, pp. 1-35. S. Attygalle, M. Duff, T. Rikakis and J. He (2008), Low-cost, at-home assessment system with Wii Remote based motion capture, in Proc. Virtual Rehabilitation, Vancouver, BC, pp. 168-174. J. Decker, H. Li and V. Prakash (2009), Wiihabilitation: Rehabilitation of Wrist Flexion and Extension Using a Wiimote-Based Game System, Governor's School of Engineering and Technology, pp. 17. Department of Health (2010), Progress in improving stroke care, National Audit Office, London, 2010. E. Jovanov, N. Hanish, V. Courson, J. Stidham, H. Stinson, C. Webb and K. Denn (2008), Avatar - a multi- sensory system for real time body position monitoring, in Proc. IEEE Engineering in Medicine and Biology Society, Vancouver, pp. 2462-2465. R. Leder, G. Azcarate, R. Savage, S. Savage, L. Sucar, D. Reinkensmeyer, C. Toxtli, E. Roth and A. Molina (2008), Nintendo Wii remote for computer simulated arm and wrist therapy in stroke survivors with upper extremity hemipariesis, In Proc. Virtual Rehabilitation, Vancouver, BC, pp. 74. J. Martin-Moreno, D. Ruiz-Fernandez, A. Soriano-Paya and V. Jesus Berenguer-Miralles (2008), Monitoring 3D movements for the rehabilitation of joints in physiotherapy, in Proc. IEEE Engineering in Medicine and Biology Society, Vancouver, BC, pp. 4836-4839. M. Matamoros, M. Negrete, Y. Haddad and R. Leder (2010), Nintendo WII remote and Nunchuck as a wireless data subsystem for digital acquisition of analog physiologic data relevant to motor rehabilitation after stroke, in Proc. Health Care Exchange (PAHCE), Lima, pp. 196. M. Palmke, H. von Piekartz, C. Zalpour, T. Schuler and K. Morisse (2009), A new perspective for Virtual Mirror Therapy Developing a low-cost-high-convenient environment utilising the Wiimote, in Proc. Virtual Rehabilitation International Conference, Haifa, pp. 197. G. Saposnik, R. Teasell, M. Mamdani, J. Hal, W. McIlroy, D. Cheung, K. Thorpe, L. Cohen and M. Bayley (2010), Effectiveness of virtual reality using Wii gaming technology in stroke rehabilitation: a pilot randomized, Stroke, 7, 41, pp. 1477-1484. A. Savitzky and M. J. E. Golay (1964), Smoothing and differentiation of data by simplified least squares procedures, Analytical Chemistry, 36, 8, pp. 1627–1639. D. Scherfgen and R. Herpers (2009), 3D tracking using multiple Nintendo Wii Remotes: a simple consumer hardware tracking approach, in Proc Future Play, Vancouver, BC, pp. 31-32. C. Shih, M. Chang and C. Shih (2010), A limb action detector enabling people with multiple disabilities to control environmental stimulation through limb action with a Nintendo Wii Remote Controller., Research in Developmental Disabilities, 31, 5, pp. 1047-53. I Tijou, L Yardley, C Sedikides (2010), Understanding adherence to physiotherapy: Findings from an experimental simulation and an observational clinical study. Psychology and Health, 25, pp. 231-247. S Van Dulmen, E Sluijs, L Van Dijk, D Ridder, R Heerdink, J Bensing (2008), Furthering patient adherence: A position paper of the international expert forum on patient adherence based on an internet forum discussion. BMC Health Services Research, 8,47. doi:10.1186/1472-6963-8-47 M. A. Williams, R. L. Soiza, A. M. Jenkinson and A. Stewart (2010), EXercising with Computers in Later Life (EXCELL) - pilot and feasibility study of the acceptability of the Nintendo® WiiFit in community- dwelling fallers, BMC Research Notes 2010, 3, 238, pp. 1-8. P. H. Wilson, J. Duckworth, N. Mumford, R. Eldridge, M. Guglielmetti, P. Thomas, D. Shum and H. Rudolph (2007), A virtual tabletop workspace for the assessment of upper limb function in Traumatic Brain Injury (TBI), in Proc. Virtual Rehabilitation, Venice, pp. 14-19. L. Yong Joo, T. Soon Yin, D. Xu, E. Thia, C. Pei Fen, C. Kuah and K. Kong (2010) A feasibility study using interactive commercial off-the-shelf computer gaming in upper limb rehabilitation in patients after stroke, Journal of Rehabilitation Medicine, 42, 5, pp. 437-41.

518 Proc. 9th Intl Conf. Disability, Virtual Reality & Associated Technologies Laval, France, 10–12 Sept. 2012 2012 ICDVRAT; ISBN 978-0-7049-1545-9