JOAQUIM NUNO
SALGUEIRO DOS
SANTOS
DESENVOLVIMENTO DE UM PROTÓTIPO DE
ARTEFACTO TANGÍVEL PARA TRATAR CRIANÇAS
COM PERTURBAÇÕES DOS SONS DA FALA
DEVELOPMENT OF A TANGIBLE ARTEFACT
PROTOTYPE FOR TREATING CHILDREN WITH
SPEECH SOUND DISORDERS
JOAQUIM NUNO
SALGUEIRO DOS
SANTOS
DESENVOLVIMENTO DE UM PROTÓTIPO DE
ARTEFACTO TANGÍVEL PARA TRATAR CRIANÇAS
COM PERTURBAÇÕES DOS SONS DA FALA
DEVELOPMENT OF A TANGIBLE ARTEFACT
PROTOTYPE FOR TREATING CHILDREN WITH SPEECH
SOUND DISORDERS
Dissertação apresentada à Universidade de Aveiro para cumprimento dos requisitos necessários à obtenção do grau de Mestre em Comunicação Multimédia, realizada sob a orientação científica do Doutor Mário Jorge Rodrigues Martins Vairinhos, Professor Auxiliar do Departamento de
Comunicação e Arte da Universidade de Aveiro e do Doutor Luis Miguel Teixeira de Jesus, Professor Coordenador, da Escola Superior de Saúde da Universidade de Aveiro.
Dedico este trabalho à minha esposa Ana, pelo seu constante apoio e auxílio. Por estar presente e me trazer de volta à realidade. Pelas incontáveis horas de que abdicou para que eu pudesse estar sossegado a produzir. E dedico também ao meu filho Gonçalo, que com a sua ajuda constante desarrumando tudo e sendo o que é, uma criança, tantas vezes sem o saber ajudou a quebrar a sensação de pressão e sturm und drang sentido. Ana e Gonçalo, vocês são mais que as palavras aqui vertidas. Esta dissertação é tanto vossa quanto minha.
o júri
presidente Prof. Doutor Carlos Manuel das Neves Santos
professor auxiliar do Departamento de Comunicação e Arte da Universidade de Aveiro
Prof. Doutora Carla Sofia Costa Freire
professora adjunta do Instituto Politécnico de Leiria
Prof. Doutor Mário Jorge Rodrigues Martins Vairinhos
agradecimentos Agradeço ao Sr. Vice-Reitor, Professor Doutor Gonçalo Paiva Dias, por ter aceite o requerimento de extensão de prazo de entrega e defesa de Dissertação. Aos meus orientadores, Professor Doutor Mário Vairinhos e Professor Doutor Luis Jesus, pela sua constante ajuda, apoio e preocupação. Pelas suas ideias, sugestões e críticas, tentando sempre levar-me e a esta Dissertação ainda mais longe e melhor. Qualquer falha será minha, sem dúvida e não da excelente equipa de orientação que tive.
A toda a minha família, por estarem sempre presentes e por me “aturarem” não só ao longo destes longos meses da Dissertação mas ao longo de toda a vida. Aos meus amigos, por aceitarem a minha longa ausência e silêncio como algo necessário e a respeitarem.
Aos meus colegas de trabalho, passados e presentes, por me terem ouvido quando falei e por terem dado sugestões e críticas. Elas foram ouvidas e integradas sempre que possível. Obrigado Joana, Rita e Ricardo Saramago. Uma palavra especial de apreço ao Ricardo Mendes, pelo seu apoio constante e valiosas sugestões, bem como a sua disponibilidade. Muito obrigado pela ajuda com os WebSockets, não o faria sem ti!
Às crianças, educadoras e terapeutas da Fala pela disponibilidade para participar no teste exploratório e pelo feedback dado.
palavras-chave Crianças; Perturbações dos Sons da Fala; Artefacto Tangível; Interação; Pais/Educadores de Infância e Assistentes;
resumo Na primeira parte desta Dissertação são apresentadas diferentes abordagens – paradigmas de interação considerados relevantes – que poderão ser usadas para enriquecer um material de intervenção tradicionalmente físico e como tornar tudo num todo coerente, num artefacto tangível. As vantagens que um artefacto tangível poderá deter sobre um objeto tradicional, bem como o seu enquadramento na forma como as crianças aprendem, são analisados. São ainda descritos três casos considerados de sucesso e deles é produzida uma breve reflexão sobre a criação de artefactos tangíveis..
Na segunda parte da Dissertação é apresentado o jogo escolhido concebido para ser transformado num artefacto tangível – o “Jogo da Pesca”. São abordados aspetos da mecânica de jogo - na versão tradicional e também na versão tangível/digital - e o porquê e as vantagens percecionadas aquando da sua transformação. As tecnologias usadas e os vários momentos e iterações que, tanto o protótipo quanto o software sofreram, são descritos e explicados os motivos e o fio condutor por trás das várias decisões tomadas.
A fim de obter sugestões e verificar se o protótipo estava a ser desenvolvido de acordo com as necessidades dos públicos-alvo identificados, realizou-se um teste exploratório, com uma amostra de 10 elementos. Durante esse teste foi utilizado o método de observação direta e os seguintes mecanismos de recolha de dados: grelha de observação e questionário/entrevista semi-estruturada. Isto permitiu a recolha de dados quantitativos e qualitativos, que nos ajudaram a concluir que o protótipo respondia às necessidades existentes, tem uma elevada capacidade de motivar o voltar a jogar e favorece a imersão. Por fim são apresentadas as conclusões e os resultados obtidos, bem como uma lista exaustiva de sugestões, comentários e alterações a realizar para criar um artefacto tangível, com características de produto final.
O artefacto produzido pode ser extremamente modular e versátil e existe uma clara necessidade e interesse em objetos similares por parte de terapeutas da fala, educadoras e auxiliares. Há no entanto aspetos a melhorar. O processo
keywords Children; Speech Sound Disorders; Tangible Artefact; Interaction; Parents/Caregivers;
abstract In the first part of this Dissertation different approaches - interaction paradigms considered relevant - that can be used to enrich a traditionally physical intervention material and how to turn everything into a coherent whole, a tangible artefact. The advantages that a tangible artefact may hold over a traditional object, as well as its role in children’s learning, are analysed. Three best practices cases are described and from them lessons are drawn for the creation of tangible artefacts.
In the second part of the Dissertation the game selected to be transformed into a tangible artefact - the game of Fishing - is presented and described. Aspects of game mechanics, both in the traditional version and in the tangible/digital version, are discussed. The reasons and advantages perceived in the transformation into a tangible artefact are reviewed. The technologies used and the various stages and iterations that both the prototype and the software suffered are described. The reasons and the motivation behind the various decisions made are explained.
In order to obtain suggestions and to verify if the prototype was being developed according to the needs of the identified target users, an exploratory test was prepared and carried out, with 10 participants. During this test we used the direct observation method and the following data gathering mechanisms: observation grid and semi-structured questionnaire / interview. This enabled the collection of quantitative and qualitative data, which allowed us to conclude that the prototype addresses the existing needs, has a high replay value and favours immersion. Finally, we present the conclusions and results obtained, as well as an exhaustive list of suggestions, comments and changes to be made to create a tangible artefact with final product characteristics.
The artifact produced can be extremely modular and versatile and there is a clear need and interest in similar objects from speech therapists, educators and auxiliaries. However, there are aspects to improve. The process should be even more iterative, with a multidisciplinary team and all end-users able to participate as co-designers.
List of Figures ...xix
List of Tables ...xxi
List of Acronyms ...xxiii
Introduction ... 1
Problem ...2
Relevance ...2
Research question ...2
Objectives ...3
Dissertation structure ...3
Keywords...4
Part I – Theoretical Background... 5
1.
Speech Sound Disorders ... 7
1.1.
Introduction ...7
1.2.
Common Phonological Processes in European Portuguese in children ...7
1.3.
Parents and KTAs role ...8
1.4.
Summary...9
2.
Tangible artefact ... 11
2.1.
Main paradigms of digital interaction with the physical world ... 11
2.1.1.
Ubiquitous Computing/Pervasive Computing ... 11
2.1.2.
Augmented Reality/Mixed Reality ... 16
2.1.3.
Tangible User Interfaces (TUI) ... 20
2.1.4.
Internet of Things (IoT) ... 25
2.2.
Tangible artefacts in education and health ... 28
2.3.
Designing Interaction for and with children ... 29
2.4.
Summary... 30
3.
Seamlessly combining TUI and Interaction for children... 31
3.1.
How children learn ... 31
3.1.1.
Social and cultural contexts in learning – The sense of community
and how it can influence learning ... 32
3.3.
Towards Tangible Interaction ... 34
3.4.
Summary... 34
4.
Technological mediated best practices ... 35
4.1.
Selection criteria... 35
4.2.
Portugal – T2T ... 36
4.2.1.
Table to Tablet development... 36
4.2.2.
Table to Tablet results ... 37
4.3.
Netherlands – LinguaBytes ... 38
4.3.1.
LinguaBytes results ... 40
4.3.2.
LinguaBytes shortcomings ... 40
4.4.
USA (MIT Media Group) – Jabberstamp ... 40
4.4.1.
Jabberstamp lessons to heed... 42
4.4.2.
Jabberstamp guidelines and development strong points ... 43
4.5.
Summary... 43
Part II – Empirical work ... 45
5.
Method ... 47
5.1.
Methodology used ... 47
5.2.
Exploratory test... 48
5.2.1.
Sample definition... 48
5.2.2.
Data gathering ... 49
5.2.3.
Exploratory test – Observation form and open questions ... 49
5.2.4.
Exploratory test – Location and setting up... 50
5.2.5.
Exploratory test – Four possible use cases tested ... 51
5.3.
Summary... 51
6.
The Fishing game ... 53
6.1.
The Fishing Game ... 53
6.1.1.
Tangible User Interface Fishing Game rules ... 54
6.2.
Why make it a TUI? ... 55
6.3.
Conceptual model ... 56
7.
The Technologies ... 63
7.1.
The hardware ... 63
7.2.
The software ... 66
7.2.1.
The Activity ... 67
7.2.2.
The Website/app ... 73
7.2.3.
On the Arduino ... 75
7.3.
Building the prototype – Iterations and final artefact ... 76
7.3.1.
Treasure Chest ... 77
7.3.2.
Fish (and other Figures) ... 81
7.3.3.
Fishing Poles ... 83
7.3.4.
Fishing Basket ... 84
7.3.5.
Speakers – Rocks... 90
7.4.
Summary... 91
8.
Results... 93
8.1.
Observation Form... 93
8.1.1.
Game/Prototype usability ... 93
8.1.2.
Game/Prototype (physical) characteristics ... 95
8.1.3.
Gamification ... 96
8.2.
Open questions questionnaire ... 96
8.2.1.
Children ... 96
8.2.2.
Speech and Language Therapist and Kindergarten Teachers ... 100
8.2.3.
SLTs and the Web/App ... 102
8.3.
Summary... 102
9.
Conclusions and Future Work ... 103
9.1.
Observation Form... 103
9.1.1.
Regarding Game/Prototype usability ... 103
9.1.2.
Regarding Game/Prototype (physical) characteristics; ... 104
9.1.3.
Regarding Gamification ... 105
Exploratory Test ... 106
9.4.
General conclusions ... 107
9.5.
Future Work... 107
9.5.1.
Taking the Fishing Game into a bigger pond ... 108
9.5.2.
Strategies to publicize/advertise the project ... 108
9.5.3.
Steps towards a final product ... 109
References ... 113
Attachments ... 123
A1 – Informed consent declaration ... 124
A2 – Parents text informing about the exploratory test goals... 125
A3 – Observation Grid ... 126
A4 – Open questions questionnaire... 127
A5 – Rules of the Fishing Game ... 128
A6 – Fishing Game diagram ... 129
Figure 1 - Virtual reality versus Embodied (ubiquitous) virtuality (Milgram,
Takemura, Utsumi, & Kishino, 1994) ... 14
Figure 2 - Smartphone sales in 2013 and expected sales until 2017(Richter, 2013) 15
Figure 3 - The reality-virtuality continuum, according to Milgram (“Virtual Reality
vs. Ubiquitous Computing, in cartoons.,” n.d.) ... 17
Figure 4 - How interaction is with GUI and how it could be with TUI (Ishii & Ullmer,
1997) ... 20
Figure 5 - metaDESK Tangible UI versus classic GUI (Hiroshi Ishii & Ullmer, 1997) . 21
Figure 6 - Tangible Interaction framework (Hornecker, n.d.) ... 22
Figure 7 - IoT key technological developments roadmap (Gubbi et al., 2013) ... 27
Figure 8 - An example of an Hearing and Discriminating activity: on the left the
digital and on the right the physical counterpart (T2T, 2017) ... 36
Figure 9 - "Smileyometer" scale used to assess T2T during design and
development phases ... 37
Figure 10 - LinguaBytes' interface consists of output and input modules (top left).
Input modules are especially designed for language exercises (top right) or
interactive story reading (middle right) through the use of RFID tags. (B.
Hengeveld, n.d.) ... 39
Figure 11 - Jabberstamp in use, with the recorder stamp and playback trumpet
tangible interface (Hayes Raffle, 2006) ... 41
Figure 12 - An annotated and sound rich drawing made with Jabberstamp (Hayes
Raffle, 2006) ... 42
Figure 13 - The expected users of the Fishing Game TUI, their intended use and
possible actions. ... 47
Figure 14 - Generic DBR model (De Villiers & Harpur, 2013) ... 48
Figure 15 - Game of Fish Pond from the McLoughlin Brothers – image ©The
Strong - Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
International License ... 53
Figure 16 - Modern day Fish Pond game and its contents... 54
Figure 17 - Set of minimal-pairs and their physical placement in the Fishing Basket
... 59
Figure 18 - Technological (hardware and software) requirements for the Fishing
Game artefact ... 63
Figure 19 - The seal Life sprite sheet. It has a companion json file. ... 68
Figure 20 - The activity screens. From left to right, top to bottom: Main/welcome
Screen, level choice, 1st level – sun, white clouds and seagulls squawking , 2nd
level – dark clouds, rain, mist and wind sounds... 69
Figure 21 - Small scale image of the Activity diagram complete with physical
Figure 23 - The Fishing Game web/app... 74
Figure 24 - The Fishing Basket in use. ... 76
Figure 25 - The complete prototype prior to the Exploratory Testing. The Fish
(figures) are already placed into the board area, inside the chest. ... 77
Figure 26 - The three Treasure Chests side by side. ... 78
Figure 27 - First build of the Treasure Chest. Larger and deeper than the final
versions. ... 78
Figure 28 - A sturdier Treasure Chest, built of wood. Notice the LCD emplacement
waiting for a next revision. ... 80
Figure 29 - The initial build of the "Fish" and the materials used in it. ... 81
Figure 30 - Fish being laser cut from the plywood and the end result... 82
Figure 31 - The fishing poles and some build details. ... 84
Figure 32 - Solution with 1 RFID and 2 slots (not visible). Note the hard cardboard
with the wires where the LDR and LED are housed, opposite facing. ... 85
Figure 33 - Fish Basket Rev.01 - better dimensions, buttons and just one slot. ... 86
Figure 34 - Rev. 01 circuit detail. ... 87
Figure 35 - Circuit board diagram before changes done in Rev.03 and still being
powered by a USB charger with a 3.5mm jack. ... 88
Figure 36 - The final revision, with a slight difference: another white LED to show
which player turn is and the graphical markings to help associate the button and
slot to their respective role. ... 89
Figure 37 - On-screen player 01 is highlighted and the physical counterpart has the
LED 01 lit up (left image). The right image is the same but for player 2... 89
Figure 38 - The evolution/iteration of the prototype Fish Basket latest version: from
breadboard to usable prototype... 90
Figure 39 - Creating the rocks out of blue insulation foam. ... 91
Figure 40 - Smileyometer scale with five values. ... 97
Table 1 - Common phonological processes in EP (Jesus, Lousada, Domingues, Hall,
& Tomé, 2015) ...8
Table 2 - The traditional and the new side by side - a fishing game comparison .... 57
CSS – Cascading Style Sheets DBR – Design Based Research EP – European Portuguese FCP – Family-Centred Practice GUI – Graphical User Interface HCI – Human-computer Interaction HTML – Hyper Text Markup Language IoT – Internet of Things
JSON – JavaScript Object Notation
KTA – Kindergarten Teachers and Assistants LDR – Light Dependent Resistor
LED – Light-Emitting Diode MR – Mixed Reality
NPM – Node Package Manager Rev – Revision
RFID – Radio Frequency Identifier SLT – Speech and Language Therapist SSD – Speech Sound Disorders T2T – Table to Tablet
TC – Treasure Chest
TUI – Tangible User Interface
UbiComp – Ubiquitous Computing / Pervasive Computing VR – Virtual Reality
Introduction
There are thousands of children that need speech therapy in Portugal (Schreck, 2009). According to data from 2009, between 20 to 30% of Portuguese children of pre-school age have, to some degree, Speech Sound Disorders (SSD) (Schreck, 2009). Schools have fewer professionals capable of treating these children and so the role of the parents and Kindergarten Teachers and Assistants (KTA) is more than ever very important.
At the same time, the proliferation of handheld devices has brought a myriad of applications to help children with SSD. However, their validity is, in a majority of them, scientifically unproven (Bowen, 2015). To address the lack of validated intervention material, from 2014 to September 2017 the School of Health Sciences of the University of Aveiro and the Institute of Electronics and Informatics Engineering of Aveiro developed a scientifically validated intervention program. The program, named Table to Tablet (T2), is comprised of a physical material, as well as the digital counterpart application. Both of these materials have already been developed and scientifically validated. They present a statistically significant efficacy.
This led to the idea behind this dissertation: 'A physical material has intrinsic qualities like weight, smell or texture, to name a few, and these can be used to stimulate speech production and thus help the speech and language therapists’ (SLTs) intervention. A digital application, especially if tactile/mobile based, has the appeal of being a well-known device by many children and produce a sense of engagement through the use of sound, animation and colour. So, what if both could be combined into a tangible artefact capable of being used by children, parents and KTA, and also while being understood as a game and not a chore by the children?'
Problem
The two problems mentioned previously in the Introduction – the high percentage of Portuguese children with SSDs, that need specialized intervention and the lack of scientifically validated material in European Portuguese (EP) – open a new and underdeveloped field of research for which this Dissertation aims to contribute.
Nowadays SLTs use physical media (boardgames, teddy bears, other assorted toys and games) to stimulate speech and so help children to overcome SSD. As stated above there are some mobile or computer applications that can be used albeit with scientifically unproven efficacy (Bowen, 2015). However, both physical and digital approaches have their own limitations.
What is proposed in this Dissertation is the creation of a “hybrid” artefact, that is physical but also digital and that allows maximizing an Speech and Language Therapy intervention – a tangible artefact.
Relevance
Due to budget cuts (ASHA, 2015) schools and parents have fewer resources to try and help children with SSD. But the number of children with this pathology and the effect that it can cause later in life is well documented (ASHA, 2015; Lancaster, 2008; McCormack, McLeod, McAllister, & Harrison, 2009). A possible answer to this problem is presented in this Dissertation, with the creation and test of a tangible artefact, capable of being used by SLTs, KTAs and parents, in a one-on-one session or as a group activity, with children with SSD and typically developing children. The use of this artefact, the socialization that it implies and the opportunity to gain a reward, gamifies the experience and makes the children perceive the time as a moment of fun rather than therapy.
Research question
How can tangible user interfaces be used as a tool to help in the intervention in children with Speech Sound Disorders?
Objectives
The work described in this Dissertation aimed to: Design and develop a tangible artefact, to be used for intervention in children with SSD; design and develop an website/application for the Fishing Game prototype to be primarily used by the SLTs; gather data from the target/intended users, such as SLTs, KTAs, Parents and Children; analyse the available applications and the games on sale with wider use and acceptance from the SLTs; analyse low-cost solutions and electronic equipment to create tangible artefacts for children with SSD.
Dissertation structure
This Dissertation is divided into two main components: the theoretical and the empiric parts. In the theoretical part, the literature review is conducted and the problem presented. The following themes, divided into chapters, are approached:
(i) Chapter One – Method;
(ii) Chapter Two – Speech Sound Disorders: what they are; the most common in EP; the SLT approach and the role of parents and KTA;
(iii) Chapter Three – Tangible artefact: what it is; main paradigms of digital interaction with the physical world; the use of these artefacts in Health and Education; Interaction – how to design for children;
(iv) Chapter Four – How to develop a tangible artefact that addresses the problem: how children learn; the sense of community and rewards and how they influence the learning process;
(v) Chapter Five – Best practices use cases: the selection criteria; the Portuguese T2T; the LinguaBytes project from the Netherlands and the MIT Jabberstamp.
In the empiric part questions like the game origins, the technology behind the prototype, its iterations, testing and results were presented.
(vi) Chapter Six – The Fishing Game: the methodology used to approach development of it; the history and rules of the game; why transform the original game into a tangible user interface game; the conceptual model; a default therapeutic intervention with the fishing game prototype;
(vii) Chapter Seven – The Technologies: The hardware used and why; the Software used and why; the activity explained; playing the activity; the website/application; building the prototype (iterations and final build);
(viii) Chapter Eight – Results: data gathered from the observation form and the grid;
The necessary evaluations/assessment and data gathering was conducted and from them sprung the final conclusions and future work.
(ix) Chapter Nine – Conclusions and Future Work;
Keywords
Children; Speech Sound Disorders; Tangible Artefact; Interaction; Parents; Kindergarten Teachers and Assistants; Invisible User Interface;
Part I – Theoretical Background
“To know the world, one must construct it.” Cesare Pavese
1. Speech Sound Disorders
This chapter aims to familiarize the reader with Speech Sound Disorders. A description of the most Common SSD in EP in children is included. On the third part, the Assessment and
Therapeutic Approach to SSD will be analysed. On the fourth part the Parents / KTA role: perceived and real will be discussed.
1.1. Introduction
Children with SSD represent 40% to 90% of paediatric caseloads (Joffe & Pring, 2008; McLeod & Harrison, 2009; Oliveira, Lousada, & Jesus, 2015). Speech Sound Disorders take the form of gaps in their speech sound systems that can cause difficulties in producing or understanding phonemes (Bowen, 2015; Dodd, Holm, Hua, & Crosbie, 2003). They also have speech patterns and structures that should not be present in typically developing children of their age (Bowen, 2015). Children with phonologically-based SSD normally present difficulties in phonology, which can be observed by the number of phonological processes1 in their speech (Orsolini et al., 2001). Phonological awareness, i.e. being able to discern and produce the correct phonemes, has an important role in reading acquisition (Felton & Brown, 1990), and the improvement of these skills prior to starting school diminishes the risk of future academic and socio-emotional difficulties (McCormack et al., 2009). So it is extremely important that these children’s expressive phonological skills and phonological awareness are developed by the SLTs, parents and KTA, in order to support the underlying skills for literacy in children with phonologically based SSD (Gillon, 2004).
1.2. Common Phonological Processes in European Portuguese in children
According to ( Jesus, Lousada, Domingues, Hall, & Tomé, 2015), there are nine phonological processes that can be considered more common in EP. In Table 01 below they will be summarised, using both phonetic alphabet and standard alphabet in the examples column for legibility purposes.Table 1 - Common phonological processes in EP (Jesus, Lousada, Domingues, Hall, & Tomé, 2015)
Phonological Process What it is Example
Final consonant deletion
Children omit a consonant in the final position of the syllable or final position in the word.
The word “porco” is produced as ['poku] (“poco”).
Weak syllable deletion The weak syllable is omitted. The word “chapéu” is produced as ['pɛw] (“ ’péu”).
Cluster reduction An element of the cluster is omitted. The word “zebra” is produced as ['zebɐ] (“zeba”).
Gliding of liquids
The consonants /l/ (e.g., bola) and /r/ (e.g., maré) are substituted by a glide.
The word “bola” is produced as ['bɔwɐ] (“bowa”).
Depalatalization
Replacement of a post-alveolar fricative consonant (e.g., ch or j) by an alveolar fricative (s or z).
The word “chapéu” is produced as [sɐ'pɛw] (“sapéu”). Palatalization Replacement of an alveolar fricative consonant (s or z) by a post-alveolar fricative (e.g., ch or j).
The word “vassoura” is produced as [vɐ'ʃoɾɐ] (“vachora”).
Devoicing
Replacement of a voiced consonant by a devoiced consonant.
The word “mesa” is produced as ['mesɐ] (“messa”).
Stopping Replacement of a fricative (e.g., s, z, ch, j, f, v) by a stop consonant (e.g., p, k, b, d, g).
The word “faca” is produced as ['pakɐ] (“paca”).
Fronting
Replacement of a back consonant (e.g., g or k) by a dental/alveolar (t or d).
The word “cabelo” is produced as [tɐ'belu] (“tabelu”).
1.3. Parents and KTAs role
Speech and language therapists have a central role in SSD intervention but the current recommended practices point to the importance of a family-centred practice (FCP). This promotes not only the parents involvement in the sessions and in homework activities but also in planning a session (setting goals) (Pappas, Mcallister, & Mcleod, 2016).
Family-centred practice follows the following guidelines (but may include more components): (Pappas et al., 2016)
» Whole family as client;
» Positive family/professional relationships; » Empowerment and enablement of families; » Parental decision-making;
According to some studies (Mcleod & Baker, 2014; Pappas, McLeod, Mcallister, & McKinnon, 2008; Ruggero, McCabe, Ballard, & Munro, 2012), most of the SLTs already try and involve parents in some way and believe it to be fundamental for the success of the intervention. However, parents of children with SSD have a different way of perceiving their role and the SLTs’ role.
Despite considering their involvement as a positive thing, parents tend to expect the session to be much more an interaction between SLT and child and see themselves more as spectators and less as decision-makers (Pappas et al., 2016).
Of similar importance to a child development due to the time spent with and nature of the relationship is the KTA. They are part of a child innermost circle (Sharynne McLeod, Daniel, & Barr, 2013). Kindergarten teachers and assistants can help in the detection and report the possible cases of SSD and in the implementation of specific activities with the child, as long as proper training, support and tools are provided by the SLT.
Kindergarten teachers and assistants are well aware of the cognitive and social impact of SSD in children and the negative attitudes people tend to have (McLeod et al., 2013). However, a caregiver has to attend the needs of several children and, at least in the Portuguese reality, cannot do one activity with just one child. Activities have to be able to be done in a group and benefit all.
1.4. Summary
In this chapter, SSDs were described as an inappropriate age level omission/replacement of sounds, perceived by the number of phonological processes in their speech (or, the
simplification of adult speech by a child with SSD) for the EP children. The common approach to assess and intervene in children with SSD and also the parents and caregiver’s role was discussed.
2. Tangible artefact
Beyond conventional typical interaction paradigms, such as Graphical User Interfaces (GUIs) and Command Line Interfaces, there are several interaction paradigms – like Natural Interaction, Ubiquitous Computing, Pervasive Computing, Mixed Realities or Wearable computing, etc. - and their value and importance in better understanding interaction cannot be underrated. However, for this dissertation and the proposed objectives, mainly the development of a tangible artefact, a chronological order of four related interaction paradigms was chosen. In this chapter the concepts of Ubiquitous Computing/Pervasive
Computing, Augmented Reality/Mixed Reality, Tangible User Interfaces and Internet of Things
and their importance to this dissertation will be discussed. The role that tangible artefacts can play in education and health is appraised. The concept of Interaction and how it differs from adults to children will be explained and some psychological aspects that affect how children learn are explored.
2.1. Main paradigms of digital interaction with the physical world
For the scope of this dissertation, in this section and subsequent sub-sections, the concepts and meaning of Ubiquitous Computing/Pervasive Computing, Augmented Reality, Tangible
User Interfaces and Internet of Things are made aware.
2.1.1. Ubiquitous Computing/Pervasive Computing
The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it. Mark Weiser, The computer for the 21st Century, 1991 Ubiquitous Computing (UbiComp), also called Pervasive Computing or, as Weiser, the father of the term preferred, Calm Technology (Hiroshi Ishii, 2004; Weiser & Brown, 1996) goal is to increase the use of computers (Weiser, 1993). To make life both more comfortable and productive (Ebling & Baker, 2012), by making many computers available through the physical environment – hundreds of computers per room (Weiser, 1991). However, these
hundreds of computers and the use of them are not really what Weiser meant (Hiroshi Ishii, 2004). Those were means to achieve the goal of “transparency” - a moment when people apprehend something so well they discontinue their awareness of it (Weiser, 1991). The necessary steps to achieve that vision will be discussed in the following paragraphs.
Weiser, in “The computer for the 21st Century” (1991), argued that personal computers, despite the large numbers of units sold, were still something obscure and demanding of a complex jargon to be able to be used (Weiser, 1991). However, for Weiser, the problem was not one of user interface but one born out of a misconception: a personal computer should be seen as an instrument used to achieve a goal, i.e. tapping into the real potential of information technology (Weiser, 1991), and not an end itself. In order to achieve its full potential, the computer should disappear into the background, no more being an object that attracts attention and needs special skills, but something so well learned that it would, akin to reading or writing, be used without the user even noticing or having to think about it (Weiser, 1991).
Weiser and its colleagues concluded that, despite the already common use of controllers or computers in some appliances that “activate the world” (Weiser, 1991), to be considered a UbiComp device it had to be capable of transmitting and displaying information more directly. Those devices had two major issues that needed addressing in order to achieve Weiser vision: location and scale (Weiser, 1991).
Location
Any UbiComp device needs to know its location in order to be useful and transmit relevant data. By knowing the correct location, a computer can adapt its actions, even without any sort of Artificial Intelligence programmed into it.
Scale
Scale is related to the main concept of having multiple machines in the same place but their area of actuation would be dictated by its size. Weiser envisioned three different sizes and called them Tabs, Pads and Boards. Tabs would be small, card size surfaces (for example driver’s license card), Pads would be paper/magazine size and Boards would be equivalent,
in size and use to a blackboard. They would be used as replacements for today's myriad of papers, cards, posters and publicity boards and they should be picked up or dropped as the user needed, like a “scrap computer” (as in scrap paper) (Weiser, 1991) and the real power behind the vision would be the devices interconnectedness.
The technology to support this concept should be, according to Weiser, inexpensive, with low power computers. The software should allow ubiquitous applications and the networks should be capable of connecting them all together (Weiser, 1991).
The main technical challenges and problems identified by Weiser were software and network related. Questions like being able to seamlessly migrate the contents of a Pad into several smaller Tabs that could be re-arranged by the user as a real “desktop” metaphor, or the content and user interface be able to follow the user into another division are still difficult due to several constraints, for example vendor related (Ebling & Baker, 2012). In terms of networks, Weiser identified the need for three different kinds of network connections in the devices: long range wireless (kilometres), tiny range wireless (centimetres) and very high speed wired, due to the need for faster data transfer rates (Weiser, 1991).
UbiComp is not virtual reality
Weiser was adamant in saying that UbiComp was not virtual reality (Weiser, 1993) and that the aims of both concepts were in fact opposite (Weiser, 1991), as we can see in figure 1. UbiComp aims to have the real world full of connected devices, cheap wireless networks, with carefree users, that do not need to carry around devices because they are everywhere, to be accessible by all. UbiComp wants to augment the World.
Virtual reality, on the other hand, wants to create a virtual world inside the computer and bring into it all the elements of reality, by simulating it. It uses special equipment to allow the user to interact and immerse himself/herself in it. However, that causes more awareness
Figure 1 - Virtual reality versus Embodied (ubiquitous) virtuality (Milgram, Takemura, Utsumi, & Kishino, 1994)
towards the computer. It makes the user even more computer-centric, and, impacts on socialization that becomes mediated by the machine. Weiser argued that UbiComp would remove the unhealthy relationship that computers introduced in society and would eliminate the computer addict (Weiser, 1991).
UbiComp and ethics
Even in 1991, Weiser was aware that privacy would be a concern and a possible pitfall for his vision. The ability to be location-aware and transmit that information across a network, at high-speed, could potentiate a “Big Brother” society with unimaginable reach - “the potential to make totalitarianism up to now seem like sheerest anarchy” (Weiser, 1991). The information could also be used in an abusive way by employers leading to the surveillance of employees and to the blurring of boundaries between private and professional life (Hilty, 2014). In fact recently (in the beginning of 2017) France approved a law to allow employees to turn off or not respond to digital enquiries/mails from their employers (DN, 2017) facilitated by the widespread use (Richter, 2013) (see figure 2) of the most ubiquitous of all devices (even if it hasn’t really faded into the background) and the one that most closely adheres to Weiser idea of a Tab: the smartphone (Ebling & Baker, 2012; Ferreira, Orvalho, & Boavida, 2007).
Weiser was also aware that the amount of information collected by the UbiComp devices on the users could be used by marketing firms in unpleasant ways (Weiser, 1991). This can be seen today with targeted ads or content and can have implications in terms of user autonomy and self-determination (Hilty, 2014).
UbiComp legacy and or conclusion
Weiser vision of a UbiComp “world” is yet to be fully implemented, with UbiComp devices being disconnected from one another and seldom interoperable (Ebling & Baker, 2012). The missing interconnectedness, key to Weiser vision and the fact that users tend to be possessive of their devices and use them more for social practices than for a productive life undermine key concepts in UbiComp. But Weiser’s vision (still valid and followed today) laid the groundwork for other visions that aim incorporating reality and ways of augmenting it, always towards an invisible interface and technology that fades away.
2.1.2. Augmented Reality/Mixed Reality
Augmented Reality is about augmenting people's skills and senses... in Augmented reality technologies, systems and applications (Carmigniani et al., 2011)
Augmented Reality (AR) can enrich our interaction with the real world by creating an extra layer – a virtual world of graphical images – that instead of replacing reality annotates and complements it with various kinds of information (Feiner, Macintyre, & Seligmann, 1993): digital objects, texts and information (Liberati, 2016), that appear to coexist in the same space as the real, physical world objects (Azuma et al., 2001). AR can be seen as a direct extension of Weiser’s UbiComp vision that adds to the Tabs, Pads and Boards small, lightweight see-through devices with world enriching capabilities not restricted to the displays and the handheld interaction envisioned by Weiser (Feiner et al., 1993). AR is also not restricted to vision, it can potentially be applied to all senses and besides adding digital objects to the real world it can also remove real objects from the environment (Azuma et al., 2001).
Milgran, in “Augmented Reality: A class of displays on the reality-virtuality continuum” article, proposes that VR and AR are part, albeit opposites, of a Mixed Reality (MR) continuum as one can see in figure 3 below (Milgram, Takemura, Utsumi, & Kishino, 1994). In it, AR is just one step after the Real Environment and one step closer to a full immersion world where what is real and what is not would be nigh indiscernible – the exact middle point of the chart (Milgram, Takemura, Utsumi, & Kishino, 1994).
There are three different AR displays: head-worn (Azuma et al., 2001) or head- mounted (Carmigniani et al., 2011) displays, handheld displays and projection displays (Azuma et al., 2001)/spatial(Carmigniani et al., 2011). The head-mounted displays can be further divided into optical see-through or video see-through, each with its own strengths and weaknesses, requirements and implementations. The handheld displays act as a window and use their camera and array of sensors to correctly produce and place the augmentations in the real world. The displays that rely on projections tend to be fixed into a location and used in teleconference systems (Carmigniani et al., 2011).
Also noteworthy is the research being done into a virtual retina display by MicroVision2 that could address several issues on use and content privacy (Azuma et al., 2001).
Each allows for different immersion states and different advantages and disadvantages. They all give to the user the computer-generated graphical overlay layer while allowing for a real-world presence (Milgram et al., 1994).
Figure 3 - The reality-virtuality continuum, according to Milgram (“Virtual Reality vs. Ubiquitous Computing, in cartoons.,” n.d.)
Two recent examples of two of the AR displays mentioned above, with varying degrees of success, are the Google Glass3 project and the Pokémon Go4 game. The first one is clearly a manifestation of the head-mounted see-through device. The second is a handheld monitor based AR display that uses the ubiquitous smartphone to juxtapose game elements to the reality surrounding us, while using the smartphone sensors to be location-aware, among other things.
Despite the above examples and reports that AR is on the rise and will be, by 2021 a 5.7 billion dollars industry (Anderton, 2016), this is a technology still in its infancy and as such with still fluid definitions and the idea of what AR really is (Liberati, 2016).
UbiComp and AR are seemingly the same since their goal is to enrich user’s life through the merging of digital information with the real world. They both try to escape the virtuality of a simulated world and aim to bring computational power to the real world and environment. This poses difficulties in terms of defining exactly the focus of these technologies (Liberati, 2016).
In UbiComp, the computer is ubiquitous being perceived by the users as transparent, while still lending its computational power to the betterment of the user's life by displaying information and adapting to the user's actions (Liberati, 2016) and location. AR goal, on the other hand, is to enhance the user skills through a device (Liberati, 2016) without replacing them (Carmigniani et al., 2011).
However, simply displaying information about a real object is not enough to be considered a pure AR device. UbiComp goal is also to present information, through a device. The act of creating a new digital object is also debatable if the only objective is to use it as a form of label (Liberati, 2016) of something that exists in the real world.
AR should introduce new digital objects into our world, shaping and enhancing it with a different kind of digital materiality (Liberati, 2016). The digital object is autonomous, it does
3 Available at https://www.google.com/glass/start/ and more information on
https://www.technologyreview.com/s/532691/google-glass-is-dead-long-live-smart-glasses/ 4 Available at http://www.pokemongo.com/
not exist because or around the “real” object in the world simply to provide information about it (Liberati, 2016). It has a gravitas of its own.
AR is interactive and happens in real time, even if mediated through video and it registers (aligns) real and virtual elements with each other (Azuma et al., 2001).
One key aspect to AR is how the user can interact with it. There are tangible interfaces, collaborative interfaces, hybrid interfaces and multimodal interfaces. The AR interface needs to address social acceptance, by being subtle, discrete and unobtrusive (Carmigniani et al., 2011); it should allow natural interaction through the use of natural hand gestures so it would not be awkward to use in public and it needs to be fashionable (Carmigniani et al., 2011).
AR conclusions
Independently of the device used, the infusing of digital autonomous objects, throughout the real world augments the user's experiences and interactions, while possessing unique affordances and unexpected interaction outcomes or overheads (Dunleavy, Dede, & Mitchell, 2009; Raj, Karlin, & Backstrom, 2016).
AR actual and potential uses range from medical visualization to the gaming industry. It allows to maintain or repair complex equipment however problems such as registration errors, system lag (Azuma et al., 2001) or, more prosaically, social acceptance over fashion issues (Carmigniani et al., 2011) need to be fully understood and solved beforehand, if AR is to become part of everyday life.
AR has greater fidelity to the real world environments than VR, at least for now. But as technology evolves AR will get closer to the centre of Milgram's Reality-Virtuality Continuum and this will fade into a seamless real-virtual world. AR allows multi-dimension contact between its user's and promotes engaging sensory experiences (Dunleavy et al., 2009) that mix the physical sense of touch coupled with visual and auditory generated contexts. From this augmenting of reality and transparency achieved through symbiosis between subject and device (Liberati, 2016) this dissertation will move into the use of a minimal
interface. This interface is so natural to the user since it is based on understood physical objects (Hiroshi Ishii, 2004), inherently simple, that “fits the task so well that learning the task is to learn the appliance” (Norman, 1998, p.86).
2.1.3. Tangible User Interfaces (TUI)
Graphical User Interfaces fall short of embracing the richness of human senses and skills people have developed through a lifetime of interaction with the real world. In (Hiroshi Ishii & Ullmer, 1997) Form (ever) follows function by Louis Sullivan
In 1997 Hiroshi Ishii and Brygg Ulmer presented a paper titled “Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms”. In it, they argued that to fully achieve Weiser’s UbiComp goal of transparency and invisible computing, the interaction had to move away from GUIs and establish a new Human-Computer Interaction paradigm, the TUI (Hiroshi Ishii & Ullmer, 1997). As figure 4 illustrates, TUI would move away from the typical and generic combination of screen, mouse and keyboard interaction and would transform the world itself into an interface (Ishii & Ullmer, 1997). This would be achieved through augmentation of real-world objects and environments with digital information (Ishii & Ullmer, 1997).
TUI conceptual model
Tangible Interfaces can be defined as those that support the user direct interaction with the digital world or digital device by use of real-world objects or tools (Azuma et al., 2001). They use physical forms designed and improved over the millennia to fit the specific task (H. Ishii, 2008) thus facilitating the user legibility and direct manipulation of the interface through the user's peripheral senses (e.g. touch or vision) due to its physical embodiment (H. Ishii, 2008; Hiroshi Ishii, 2008; Hiroshi Ishii & Ullmer, 1997).
As TUI employs the user’s experience of interaction with real objects to use with its digital representations, this alleviates the keyboard bottleneck (Stanton et al., 2001), and the user can focus its attention and consciousness on the task at hand and not on the interface (Hiroshi Ishii, 2004). In figure 5 below this method can be seen in a comparison of a TUI and a GUI for a desktop metaphor. This was used in the metaDESK project (Ishii & Ullmer, 1997).
TUI started by using certain objects or shapes with which the user would interact and that had an actual perceived meaning, for example, a clock object would serve to control time – and time dictates sun position, like in the Urban Planning Workbench project (Underkoffler & Ishii, 1999). This can be understood as a rigid discrete interface (H. Ishii, 2008). However, one shortcoming of this approach is the specificity of objects and projects, making this interaction “tailor-made”. This greatly contrasts with the generic mouse and keyboard controllers (H. Ishii, 2008) that simply “work” for all kinds of different projects. The generic manipulation costs are a certain learning curve and a gap in immediate understanding of what is the manipulation of an object and the manipulation of a graphical representation of
it (Hiroshi Ishii, 2004). When creating a TUI all physical objects have to be pre-determined and there are a finite set of objects and possible interactions (H. Ishii, 2008).
This limitation originated the next evolutionary step, a more “organic” and material malleable TUI (H. Ishii, 2008). This approach takes advantage of new digital and physical materials that can seamlessly pair sensing and displaying capabilities (H. Ishii, 2008). The use of continuous tangible materials like clay or sand allows for quick sculpting and display and together with emerging new materials that integrate fully flexible sensors and displays, TUI shows great potential to break the boundaries of pre-determined interactions (H. Ishii, 2008).
However, there are claims that this definition of TUI and its evolution is very limited (Hornecker & Buur, 2006). They argue that more than classic computer science TUI definition it should be discussed a broader space and interdisciplinary field within what they call Tangible Interaction (Hornecker & Buur, 2006). This broader field encompasses embodied interaction, tangible manipulation, a physical representation of data and embeddedness in real space and it can be approached from a Human-computer Interaction (HCI) perspective,computer science, product design and interactive arts point of view (Hornecker & Buur, 2006). This approach gives rise to a framework of Tangible Interaction, as shown in figure 6.
In this framework, the Tangible Manipulation refers to the use of material objects with distinct tactile (haptic) qualities. It is best applied to systems usually named TUI and tangible appliances (Hornecker & Buur, 2006). Spatial Interaction and Embodied Facilitation are related to the movement in space and the physical configuration of the computing resources are key to them. They affect and direct emergent group behaviour. Since tangible interaction is embedded in real space, interaction stems from a movement in space and the body itself is an input device (Hornecker & Buur, 2006). The Expressive Representation focus is on the material and digital representations used by Tangible Interaction systems, how they are understood (legibility) and how well they represent reality (expressiveness) (Hornecker & Buur, 2006).
Other authors (Alissa Antle, 2007; Hengeveld, Hummels, et al., 2008; Hengeveld, 2011; Hayes Raffle, Vaucelle, Wang, & Ishii, 2007b; Spermon, Schouten, & Hoven, 2014; Suárez, Marco, Baldassarri, & Cerezo, 2011; Xie, Antle, & Motamedi, 2008; Xu, Read, & Mazzone, 2007; Zaman & Abeele, 2007; Zaman, Abeele, Markopoulos, & Marshall, 2009) give relevance to the role of TUI involving children and the benefits they can offer, related to the ease of use while supporting learning and development processes (Almukadi & Stephane, 2017; Zaman et al., 2009). Young children (and even more if they have some form of disability) do not grasp menu structures, layered interfaces or icons. This is especially true when learning a language due to its increasing symbolisation (the word dog has the same meaning/relation to an image or drawing or actual dog). Children natural interaction style is exploratory and multi-sensory (Hengeveld, Hummels, et al., 2008). Furthermore, TUI require less interpretation, allow for a more flexible interaction and collaborative use, permit sensory experiences and are persistent (even powered off a clock is still a clock and allows interpretations while a mouse, despite being physical, when off holds little representation) (Hengeveld, Hummels, et al., 2008; Hiroshi Ishii, 2008).
Some authors suggest that TUI should be reactive, in order to solve inconsistencies and provide additional feedback (Pangaro, Maynes-Aminzade, & Ishii, 2003; Poupyrev, Nashida, & Okabe, 2007) or in order to be easier to use by children with disabilities (Hengeveld, Hummels, et al., 2008).
For that TUI should use actuators, electromagnets, microprocessors, smart materials and others to help children achieve a goal while being able to reduce or completely turn off this behaviour as the child progresses or is used by children without disabilities (Hengeveld et al., 2008). This actuated TUI would overcome the static and rigid nature of a physical object and give to it the malleability of a digital object: easy to create, change in shape, colour, position or speed (Poupyrev et al., 2007). This allows for an interface that communicates to the user changes in the system by re-arranging the user interface or even adjust the display to reflect changing information or user interaction (Pangaro et al., 2003; Poupyrev et al., 2007).
TUI is not AR or UbiComp
Despite being stimulated by the UbiComp concept TUI does not assume itself as being UbiComp due to a different vision in how to make technology invisible. TUI is interested in looking into the richly-afforded physical devices created by Human technological evolution from the last millennia prior to the computer era (Hiroshi Ishii & Ullmer, 1997) and borrow from them the interaction they afford, constrain and convene and make them into a or part of a TUI.
The challenge is in being able to map a physical object and its manipulation to digital computation and feedback in a meaningful way, maintaining its physical affordances and bringing them into the digital realm (Hiroshi Ishii, 2008).
TUI diverges from AR because unlike it, TUI places a strong emphasis on graspable physical objects as input and in a combination of ambient media and said objects, rather than relying on pure visual augmentations (Hiroshi Ishii & Ullmer, 1997).
TUI conclusions
TUI is a natural next step or should be said a return to the roots of Human (natural) Interaction, away from the static screens that permeate our lives, be it in a desk, in one's hand or embed in our everyday environment (Ullmer & Ishii, 2001). TUI assumes itself as an extension of Weiser's vision of UbiComp and one that can really implement the idea of transparent technology, so well learned that it fades into the background.
2.1.4. Internet of Things (IoT)
If we had computers that knew everything there is to know about things – using data they gathered without any help from us – we would be able to track and count everything, and greatly reduce waste, loss and cost. Kevin Ashton in RFID Journal (22 June 2009)
In Weiser's UbiComp concept the interconnectedness and interoperability between devices were key to bring to fruition his vision. However, those concepts are far from solved and it is IoT that aims to solve that (Ebling & Baker, 2012).
The term Internet of Things was first coined by Kevin Ashton in 1999 regarding a specific context: supply chain management with items with tags (things) that would allow for a more efficient, less costly and less wasteful business (Caceres & Friday, 2012; Gubbi, Buyya, Marusic, & Palaniswami, 2013). Since than the definition evolved to cover more areas and today IoT can be understood as the possibility of sensing and actuating objects to communicate between themselves (machine to machine, also known as M2M), utilizing secure network connections and cloud services (unified framework) to transform the information that they obtain from that communication into useful information for people and enterprises to use (Gubbi et al., 2013; Verizon, 2015).
To be able to achieve a seamless UbiComp vision, IoT needs three main components (Gubbi et al., 2013):
1 – Hardware: the sensors, actuators and embedded communication hardware; 2 – Middleware: on-demand storage and computing tools for data analytics;
3 – Presentation: new tools to visually present and design data in order for it to be visualized, understood and accessed on different platforms and different applications. IoT main goal is to make a computer sense information without human intervention and is on the brink of evolving Internet from static into a fully integrated and pervasive Future Internet (Gubbi et al., 2013).
Broadly speaking IoT is gaining momentum, with business, governments and academia investing in it (Gubbi et al., 2013; Verizon, 2015). Due to government policies, industry practices and even homemade “hacks” the world we live in is becoming electronic populated: more and more sensors gather data, control appliances and even send emails to the owners or engineering support (Caceres & Friday, 2012). This can be used to IoT advantage as long as the infrastructures are open to being used and the necessary coupling or interconnectedness by the things with the embedded infrastructures and between themselves is made possible (Caceres & Friday, 2012). The drop in hardware and cloud space costs (with the price per GB lowering) is also an important factor in making IoT accessible to the general public or small business (Verizon, 2015).
What are “Things”?
The Things in IoT are smart objects. For a device to be a Thing/smart object, it needs to: Physically exist and possess physical features, such as size, weight and so on; It needs to be able to communicate, even if in a basic level: it needs to be discoverable and needs to accept and respond to incoming messages; It needs to have a unique identifier; It needs one name (human readable) and an address (machine readable); It needs (at least) basic computing capabilities; it needs to be able to sense physical phenomena or actuate (causing effects) on the physical world (Miorandi, Sicari, De Pellegrini, & Chlamtac, 2012).
The characteristics of the Things presented above support the three main pillars in IoT: anything identifies itself; anything communicates; anything interacts (Miorandi et al., 2012).
IoT in the near future: needs and challenges
Internet transformed the world into a small village, where everyone knows everyone and all are connected. The next similar event has already started, with smart environments thanks to 9 billion interconnected devices (Gubbi et al., 2013)– 24 billion plus expected by 2020 (SBI, 2016). With this growth, it is possible to predict that within a decade, the Internet will be a seamless union of classic networks and networked devices (Miorandi et al., 2012). In figure. 7 the technologies that are already contributing to that scenario and the expected key developments and applications to bring IoT into its full potential are laid out. IoT also
has some specific challenges to solve, such as privacy, participatory sensing, data analytics, geographic information services based visualization and Cloud Computing besides the more standard Wireless Sensors Networks challenges: architecture, energy efficiency, security protocols and Quality of service (Gubbi et al., 2013).
Another challenge faced by IoT is its fragmented research community, most of the times focused around a single application domain (i.e. transportation or Health, for example) or a single technology (Radio Frequency Identifier (RFID) being one example) (Miorandi et al., 2012). While the fragmented research can be understood due to the fact that different countries, institutions or industries have different goals, the single application domain mindset is obtuse.
IoT objects will need to be able to support a wide range of very different devices with different capabilities and accommodate for all of them by being scalable, energy efficient, with self-organizing capabilities and able to analyse massive amounts of data, all the while with embedded security and privacy-preserving mechanisms (Miorandi et al., 2012).
IoT and ethics
IoT faces similar ethical issues when compared to UbiComp. The amount and type of data gathered from users raise alerts to the potential loss of privacy, user autonomy and self-determination (Hilty, 2014). Not all data gathered should be open due to the potential it has for cyberterrorism and abusive surveillance (Caceres & Friday, 2012).
Security is a cornerstone that can hinder IoT adoption commercially and give rise to ethical questions. How confidential is the data gathered and who has access (and how) to it? How private is the data? And if it is a domain such as health, can the patient access its own data? Or, by denying access to it, are the medical staff incurring in a case of technology paternalism in a classic dependency relationship (Hilty, 2014)?
IoT conclusions
A fully functional IoT will represent a kick-off for innovative services that will improve the user's experience and quality of life, due to the responsive nature and ability to anticipate user needs according to the situation and or location the user is in (Miorandi et al., 2012). It will also be a truly UbiComp world, with computing and information flowing (and available) anywhere.
However, there are several interdisciplinary challenges and infrastructure needs, as well as security and ethical issues to solve before being able to deploy a network of things.
2.2. Tangible artefacts in education and health
Despite the copious amounts of educational applications available in mobile applications stores, physically manipulating an object requires a different set of skills (Barredo & Garaizar, 2015). A physical, tangible artefact, enables or rather facilitates self-learning, requires autonomy while stimulating social interactions (i.e. working in groups) (Almukadi & Boy, 2016; Xie et al., 2008) while providing the necessary enjoyment and engagement (Xie et al., 2008), important in learning.
Tangible artefacts used in interactive games for a therapeutic context is nothing new, especially in fields of physical therapy or post-stroke recovery (Li, Fontijn, & Markopoulos,
2008; Vonach, Ternek, Gerstweiler, & Kaufmann, 2016). They also allow assessing several physiological parameters, without any stress associated with a visit to a doctor’s office, relieving an anxiety felt by many children and some adults alike (Vonach et al., 2016). In education both the needs of the teachers as well as the needs and curricula of the students have to be fully understood and satisfied (Bruckman, Bandlow, & Forte, 2007). And serious games and tangible interaction for learning and problem-solving satisfy both teachers and children alike (Almukadi & Boy, 2016). Children like to play and through a playful approach, learning is facilitated (Almukadi & Boy, 2016). Tangible artefacts by nature invite collaboration, allowing several users to interact with the artefact and themselves (Almukadi & Boy, 2016), thus increasing productivity levels (Fails et al., 2005).
In both cases, the artefacts should be cheap and robust to allow to be handled without care (Barredo & Garaizar, 2015). It is important to know that it is not enough to produce an object with an array of sensors in a seemingly playful way. The way in which the user interacts has to be driven through affordances, mappings and game logic to ensure reliability and to take full advantage of the potential of the artefact (A. Antle, 2007; Vonach et al., 2016).
2.3. Designing Interaction for and with children
Designing for interaction is all about how to design for people, their needs, emotions and intellect (Verplank, 2009). This makes it imperative to be extremely aware of what to expect from those who will interact with the final interactive product (Verplank, 2009).
When designing for children, children are presumed to be creative, intelligent and, if given the correct tools, capable of anything (Bruckman et al., 2007). With the shift towards participatory and ethnographic methods, those designing interactive applications or objects have to try and fully understand the how’s and why’s of people using the new creations (Bruckman et al., 2007; Verplank, 2009). However, when designing for children, one cannot expect to “know” or “remember” what a child wants or likes. As people grow and age both physical and cognitive abilities – motor skills, the speed of processing, amount of data processed - increase over time (Kail, 1991; Miller & Vernon, 1997; Thomas, 1980).
Only interactive experiences that include pertinent aspects of easy control and social experiences can be meaningful and playful for children (Marco, Cerezo, & Baldassarri, 2009). The children (ages 3 to 6 or seven) intended as a target for the artefact created alongside this dissertation are still –mostly – preliterate, with short attention spans, difficulty with abstractions and their fine motor skills are not yet fully developed (Bruckman et al., 2007; Thomas, 1980). Nonetheless, designing too childish can be perceived as boring or rude (Bruckman et al., 2007), since children are acutely aware of their capabilities (Nielsen, 2006, 2010).
A workaround is to embrace designing with children, as co-designers or evaluators/subjects, or a mixture of these (Bruckman et al., 2007). This has its own possible pitfalls and both adults and children need to learn to work together, but in the end assures that the design is made according to the needs and specificities of children (Bruckman et al., 2007).
2.4. Summary
In this chapter the interaction paradigms considered as more relevant to this dissertation - Ubiquitous Computing/Pervasive Computing, Augmented Reality/Mixed Reality, Tangible User Interfaces and Internet of Things - were summarised and discussed. The role that tangible artefacts can play in education and health was evaluated. The concept of Interaction and more importantly how it differs from adults to children was addressed.
