Rehabilitation with virtual trees

Many people lose their independence after a stroke, often being unable to walk, grasp anything or even bring a fork up to their mouth. The reason for this is a hemiparesis or hemiplegia and is particularly evident in the arms. A team led by Denise Baumann from the HLS Institute for Medical Engineering and Medical Informatics, along with partners from industry and research, have been working to help patients recover hand and arm function.

The success of the Innosuisse- funded ISEAR project (Integrated System for the Encouragement of Arm Rehabilitation) is based on a special fitness wristband and mobile phone app. Medical technology engineer Baumann explains: “In the first phase of rehabilitation, patients realise that their physical abilities do not recover spontaneously; in the subsequent chronic phase, some abilities even decline further from lack of use of the impaired body part. That’s where we come in.”

Patients need to be encouraged to move their arms, because even uncoordinated movements help create new nerve cell connections to take on lost functions. Experts call this adaptability of the brain ‘neuroplasticity’. The effect is intensified by repeated controlled movements, so after a stroke, patients receive intensive physiotherapy at the clinic and an exercise programme at home. “Two factors are important for successful home training: intensity and motivation,” says Baumann.

Even healthy people know the positive effect of a wrist pedometer. The arm rehabilitation tracker, developed by the firm yband therapy AG, is based on the same principle. Like a pedometer, it has acceleration and position sensors, the data from which enable it to calculate the number of steps using an algorithm developed at the HLS. The partner team at the ETH Zurich developed an algorithm for detecting arm activity and both algorithms are stored in the wristband’s microcontroller, along with the parameters recorded. They are synchronised via Bluetooth and evaluated with the associated ‘ArysMe’ app, which Baumann and her team at the HLS have programmed for iOS and Android. “The greatest challenge was to recognise and quantify the movements correctly: which signals or signal sequences count as real movements, what patterns do they form and how often are they repeated? These data are the basis of the reward system,” reports Baumann.

The ‘Tree of Recovery’ virtual reward system is based on an idea by two undergraduates at Zurich University of the Arts (ZHdK). The Tree of Recovery is visible as a small tree in the app and only grows if someone has been active. As soon as predefined daily goals have been achieved or exercises have been performed correctly, the system gives virtual badges as a reward. These can be converted into virtual water for your Tree of Recovery so you can see your progress.

At the beginning the app shows only a few green hills and a little stream. The Recovery Tree starts as a seedling which gets bigger and bigger and, if it is watered regularly, eventually fills the whole screen. With more progress, leaves and blossom grow and animals appear; however, if the patient neglects their training, the tree stays the same size but dries out. This visual feedback is designed to motivate patients to continue working to improve their motor skills and achieve their individual daily goal, calculated by the tracker based on average arm activity. There are also other tasks that have to be done in order to water the tree. “Three minutes a day of intensive arm activity is such a task,” says Baumann. The tracker recognises the movements and classes them as intensive or less intensive. Stored exercises can be practised; for example, the physiotherapist reaches for an object or brings it accurately up to the mouth and the patient imitates this as well as they can. The exercise is stored as a motion template in the tracker; the app monitors the real movement patterns and compares them with the recorded template; if an exercise is performed correctly, the virtual reward helps the recovery tree to grow. In addition, physiotherapy specialists can use the professional ‘Arys pro’ app to compare arm movement data and better assess individual progress. To further motivate patients, a vibrating reminder function is built into the wristband which prompts them to train.

These and other functions are currently being investigated in a clinical study with patients at the University Hospital Zurich. The researchers are interested in whether patients cope well with the app and tracker, whether arm function is regained and if the app lacks any functions. They hope their work will help as many patients as possible find their way back to independence.