Serious games and dementia

Some novel approaches are being explored in the European funded project ISISEMB – Intelligent system for independent living and self-care of seniors with cognitive problems or mild dementia [81]. This project aims to enhance the quality of life of both carers and PwD by providing

"innovative intelligent custom services" in private housing through information and communication technology (ICT) such as visual fall detecting systems [82]. On the other hand, the usage of serious games (SG) (or even transformational games [83]) has gained much attention in recent years.

According to Zyda, SG can be defined as "A mental contest, played with a computer in accordance with specific rules, that uses entertainment to further government or corporate training, education, health, public policy, and strategic communication objectives" [84]. Although more studies are required to assess the therapeutic outcomes of serious games [85]–[87], a variety of SG have been developed and used for a variety of medical purposes such as training and simulation [88], diagnosis and therapy [89]–[91], and education [92]. Several games with therapeutical outcomes have been designed for patients with dementia, as well [93]. Such therapeutic games were designed to stimulate not only cognitive and motor capabilities, mood, balance among other related capacities, but also to help relearn competences that might have been lost due to the long term consequences of the disease [7], [94].

Healthcare professionals at the Madeiran delegation of Alzheimer Portugal use technologies such as Nintendo Wii, tablets, and web applications to stimulate PwD. For example, when using the Nintendo Wii, PwD play Wii Sport and perform activities such as boxing, tennis, and bowling. However, we were told that PwD had difficulties in handling Nintendo’s Wii controls, especially, participants who had physical and cognitive limitations (see supplementary material interview 1).

Thus, the inability to interact with digital games can lead to frustration and confusion. An interesting observation has been made by healthcare professionals when PwD were playing the tennis game. Some players were confused and did not understand that the Wii controls work as tennis rackets.

Health professionals believe that if Wii controllers could be customized to look like real tennis rackets, it would be more intuitive for PwD as these are familiar objects. Also, PwD play “Wii Party,” which has

13

activities such as dancing. Here, healthcare professionals noticed more motivation as the interaction appeared to be more intuitive for PwD and ended up having fun playing.

As for the tablet, healthcare professionals used a “piano” app for a participant that likes to

“imagine” playing the piano on the table when listening to music. Healthcare professionals noticed that the user was more attentive and focused while performing the task. PwD also used the tablet to watch Youtube videos.

PwD also performed activities using Lumosity, an application consisting of a set of cognitive exercises, such as, for example, numerical reasoning. Healthcare professionals noticed that participants were motivated to solve the problems and suggested that performance was better than paper-based exercises. Although it is vital to monitor the level of frustration of PwD when playing digital activities, healthcare professionals recognize the benefits that digital games can bring to PwD in terms of cognition, motivation, and physical skills.

From a technological point of view, many platform strategies have been taken into consideration to develop dementia-related therapeutic SG applications. Two main types of interaction can be used to play (and control) these games. Indirect interaction technologies require an intermediate device to translate human action into interaction with the virtual environment. For example, technologies such as PCs [94], or conventional entertainment systems like the Nintendo Wii system [95] require an intermediate device such as a keyboard or gamepads with buttons and analog sticks.

On the other hand, using direct interaction technologies, participants do not have an intermediary device to interact with the virtual environment; participants interact directly with the machines with their bodies [96]. Examples of direct technology devices are touchscreen technology [87], [97] (see [98] for review), and gesture recognition systems, for example, Leap Motion, Kinect, and Bracelet Myo [99] and augmented reality (AR) [100]. Recently, AR technologies have gained much attention to address dementia-related issues [101]. Work has been done using AR in a variety of contexts for PwD, such as in a cognitive screening tool [102] or to provide reminiscence related experiences augmented with multimedia content such as photos and videos. For instance, this technology allows the blending of real objects with virtual content on a mobile device when walking near them [100].

On a cognitive level, indirect interaction devices use more cognitive resources as it involves conscious spatial and mental translations to convert real-world movements into virtual actions [96], while direct interaction devices require less cognitive resources as there is no movement translation between the real and virtual world as opposed to indirect interaction devices [96].

A significant advantage that serious games have over traditional methods is that serious games can enhance motivation among PwD [103], which is an essential factor, especially during clinical

14

interventions. If PwD are engaged in game-like tasks, they will focus less on the clinical aspects, which are a source of emotional stress.

However, the success of using SG by patients in general greatly depends on the resulting game experience [104]. Brian Winn developed the design, player, experience (DPE) framework (see Figure 2.2), which depicts the relationship between the designer and player's experience. The framework is quite straightforward: the designer designs the game, which is played by the player according to that player's experience. According to this framework, Play is mediated by experience. Thus, player's experience (social, cultural, cognitive, and experimental background) influences, consequently, the design of the game.

Figure 2.2.The Design, Play, and Experience Framework. It illustrates the interaction between game designers and players.

Adapted From [104].

Therefore, during the prototype playtest phase, it becomes essential to validate, together with health professionals, the efficacy and effectiveness of digital systems explicitly designed for stimulation purposes with PwD (see Figure 2.3). Also, it is vital to involve PwD in the process, especially if they are the end-user. Unfortunately, many PwD are not involved in the design processes [105]; this may be because co-designing with PwD can be challenging as it requires certain levels of sensory, cognitive, and motor abilities that are affected in such population [106]. Nevertheless, PwD are still capable of sharing and expressing their needs to other people [107].

Figure 2.3. The iterative Design Process illustrates the design process of serious games. All games should be prototyped and play-tested with end-users, in order to guarantee that the design is central to the users' expected play experience. Adapted From [104].

The DPE framework also considers the learning process in using technological devices to play games as it can also influence the users' game experience. For example, in a recent study, R. Muri et al.

evaluated the performance of elderly individuals on a set of technologies while performing two different

15

tasks [108]. The study concluded that interaction with technology is dependent on the task, the user's experience, and motivation.

Moreover, the interaction with technology also depends on how intuitive both hardware and software interfaces are for PwD [103]; many high-tech technologies can overwhelm PwD on a cognitive level, which can affect the learning curve of handling technology [103], [109]. As a result, PwD will require more assistance to use technology to perform tasks [97]. One of the main issues of the SG that are currently available for PwD is that they do not follow a patient-centered design in terms of technology and digital content [101]. Also, first-time contact with novel technologies can lead to anxious behaviors [110] or even undesired side-effects such as cybersickness [111].

Another limitation of the usage of SG is the lack of interoperable systems, that is, be able to have the same game running on different technologies [103]; some technologies may be more appropriate than others for PwD. Thus, to enhance the user experience for PwD while using novel technology, additional guidelines have been suggested to help developers in designing technologies while addressing the need for PwD, such as the REAFF framework [112], [113]. The framework focus in four principles: (1) Responding (technologies should respond to the needs of PwD), (2) Enabling (technologies should improve the quality of life of PwD), (3) Augmenting (Technologies should be able to adapt the reserved skills of people with dementia) and (4) Failure Free (Technologies should be as easy to use as possible without discouraging PwD).

Another issue that needs to be considered by developers regarding the development of SG’s are the costs. Developments of technology and software solutions are costly [114]. Thus, many healthcare professionals and informal caregivers may be reluctant to use SG, as these may only be useful at certain stages of dementia [115].

As stated earlier, Alzheimer’s patients start to lose individual competencies as the disease progresses; their understanding of play will change as well. Therefore, the "play" experiences of SG should match the experiences that are still understandable for patients throughout the different stages of dementia [39]. Table 2.1 summarizes the different kinds of play experiences that are suitable in each stage of the disease. Although it is vital to match the play experience to the current stage of dementia, the ability to learn new information always depends upon the proper functioning of the working memory system. Thus, it is essential to be aware of the current understanding of the working memory system.