[PENDING] Interaction Design and Evaluation of Mobile Guides for the Museum Visit: A Case Study in Multimedia and

(1)

CONSERVATOIRE NATIONAL DES ARTS ET MÉTIERS Centre d'Étude et de Recherche en Informatique

Ecole Doctorale EDITE, Paris

THÈSE DE DOCTORAT PhD Thesis

présentée par

Areti DAMALA

en vue d'obtenir le titre de

Docteur du Conservatoire National des Arts et Métiers discipline Informatique

Interaction Design and Evaluation of Mobile Guides for the Museum Visit: A Case Study in Multimedia and

Mobile Augmented Reality

Soutenue le 30 juin 2009 devant le jury composé de

ZACKLAD, Manuel, Professeur, CNAM-Paris, Chaire Expressions et Cultures au Travail, Président CUBAUD Pierre, Professeur, CNAM-Paris, Laboratoire CEDRIC, Directeur

PELE Danielle, Responsable Unité de R&D DRC à Orange Labs, Co-Directeur

JACOBI Daniel, Professeur, Université d’Avignon, Laboratoire Culture & Communication, Rapporteur ZREIK Khaldoun, Professeur, Université de Paris 8, Laboratoire Paragraphe, Rapporteur GRESSIER - SOUDAN Eric, Professeur, CNAM-Paris, Laboratoire CEDRIC, Examinateur

PARIS

Juin 2009

(2)

(3)

Interaction Design and Evaluation of Mobile Guides for the Museum Visit: A Case Study in Multimedia and Mobile Augmented Reality

Abstract: This thesis examines the context of use of mobile multimedia handheld guides and applications as an alternative interpretation medium in the museum environment. Its main research hypothesis is that the use of mobile Augmented Reality (AR) technologies and the AR metaphor as a principal component for the conceptual and interaction design of mobile museum guide applications could greatly facilitate interaction and navigation, both in the mobile multimedia application and in the sensitive museum ecology.

After examining the context of use of mobile multimedia museum guides, proposing a set of classification criteria, redefining the notion of interactivity in the specified research context and examining how AR technologies can shape already existing or future functional requirements, the main research hypothesis is tested through the design, implementation and evaluation of a mobile AR museum guide, conceived for a state Museum of Fine Arts (Musée des Beaux Arts), in Rennes, France. The evaluation methodology is then exposed, before carrying on to the evaluation of the AR guide. The latter, mainly qualitative in nature, included the use of direct and indirect observations, semi-structured interviews, questionnaires and two focus group sessions and shed light on several issues related with the potential but also the constraints of using AR technologies as an intuitive and easy to understand alternative for geolocalization and orientation in the museum space and the mobile multimedia application.

The topic treated appertains in the wider domain of Human-Computer Interaction (HCI) with mobile devices and services or Mobile Human Computer Interaction (MHCI). However, because of the specificity of the examined research hypothesis, the thesis also aims to contribute methodologically, but also empirically, in the current state of the art regarding the interaction, interaction design and evaluation of mobile AR applications. Finally, due to the nature of the domain-space in which the experimental interventions took place -the museum environment- the thesis also sheds light on the ways by which using mobile multimedia guides can alter, transform or enhance the museum visiting experience and the relations of contemporary museums with their public.

Keywords: mobile human computer interaction (MHCI), interaction design, Augmented Reality (AR), mobile museum guidance systems, museum, museum visit, interpretation media, multimedia, evaluation, museum handheld devices, mobile learning, visitor studies, evaluation, participatory design, edutainment

(4)

(5)

Etude d’Intéraction et Evaluation de Guides Portables Multimedia pour la visite culturelle: Des Multimedia à la Réalité Augmentée Mobile

Résumé: Cette thèse examine le contexte d’utilisation des guides portables multimédia comme aide alternative à la visite culturelle. Son hypothèse de recherche principal est que l’utilisation des technologies de Réalité Augmente (RA) mobile et de la métaphore introduit par la RA en tant que component principal de conception et d’interaction concernant les guides multimedia portables, pourrait énormément faciliter l’interaction et la navigation, tant dans l’application interactif mobile que dans le contexte sensible du musée.

Le domaine traité appartienne plus largement dans le contexte plus large de l’interaction Homme- Machine Mobile. Cependant, à cause de la nature du sujet traite, la thèse apporte aussi des éléments qui enrichissent l’état de l’art concernant l’interaction et évaluation des applications de RA mobile. Enfin, a cause de la nature de l’environnement ou l’expérience a eu lieu –le musée- la thèse apporte aussi des contributions sur les façons avec lesquelles l’utilisation des guides portables multimedia pour la visite culturelle peut altérer, transformer ou promouvoir l’expérience de la visite culturelle et les relations du musée contemporain avec ses publics.

Après avoir examiné le contexte d’utilisation des guides portables multimedia pour la visite culturelle, proposé un set de critères de classification, réexaminé la notion d’interaction dans notre contexte de recherche et investigué de quelle manière les technologies de RA pourraient former l’analyse conceptionelle et fonctionnelle des guides portables multimedia, l’hypothèse de recherche principal est testé a travers la conception, l’implémentation et l’évaluation d’un guide de RA mobile, créé pour et avec le Musée des Beaux Arts de Rennes. La méthodologie et le protocole d’évaluation sont ensuite exposés avant de venir aux résultats des sessions d’évaluation qui ont eu lieu dans l’environnement du Musée. Des observations ont été combinée avec des entretiens semi-structurées, suivis ensuite d’un questionnaire et deux sessions des entretiens de group. L’analyse de donnes a démontré plusieurs avantages mais aussi inconvénients lies avec l’utilisation de la RA comme un alternative intuitive et facile a utiliser pour la geolocalisation, l’orientation, la navigation et l’interaction tant dans l’environnement du musée que dans l’environnement de l’application interactive.

Mots-clés: guides portables multimedia pour la visite culturelle, interaction homme-machine (IHM) mobile, interaction, interactivité, Réalité Augmentée (RA), évaluation, multimedia, musée, visite culturelle, médiation culturelle, évaluation, apprentissage assistée par ordinateur, design participatif

(6)

(7)

Acknowledgements

This work benefited from the continuous support of many “families” of individuals that I would like to thank here.

Starting by the Orange Labs Research and Development division, in which I performed my first steps in a Research and Development environment, I would particularly like to thank Christian Bouville and Danielle Pélé, heads of my research unit for all their encouragement, personal interest and help in many critical occasions but also for doing their best in providing me, as well as other PhD interns, with all the necessary material and immaterial “equipment” in order to carry out this work. I also owe to particularly thank Pascal Houlier and Isabelle Marchal, research engineers in the IAM laboratory, with which I had the immense pleasure of collaborating for three years. Pascal, Isabelle, I will never be able to thank you enough for being constantly there, with all possible ways, in touch not only with the practical aspects of the work undertaken, but also morally and psychologically. I also feel the urge to thank some other individuals that exercised a great influence in my work: Christian Lecoq, research engineer, for sharing and publishing with me valuable data from the Mobivisit project as well as Alexandre Cotarmanac’h for allowing me to take actively part in the DANAE project. With this occasion, I would also like to thank the DANAE family, for their warm welcome in the team, and more in particular Valerie Ledunois, Marc Brelot and Hub Kockelkorn, curator at the Museon Museum in the Netherlands for all the wonderful time we spent while working together. Anne Bationo, ergonomist in Orange Labs, is also to be warmly thanked, for her advice and interest in the experimentations held in the Museum of Fine Arts in Rennes.

The second family of professionals and individuals that I would warmly like to thank are all located in the Museum of Fine Arts in Rennes. I am particularly grateful to Laurence Imbernon, museum curator, for all the time she took to work with me on the content of the Augmented Reality guide prototype, as well as to the team of the museum educators and more in particular, Odile Hays, Carole Marsac, Anne - Sophie Guerrier. Without their valuable help, their personal investment, their trust, their passion and enthusiasm, the work related with the main Augmented Reality prototype would have literally been impossible.

The third family is my academic laboratory, “cedric”. Starting from Professors Stephane Natkin and Pierre Cubaud, with whom I had my first meetings, I would like from this position to particularly thank my academic supervisor, Pierre Cubaud, for all the time he consecrated in my work, the passionate discussions (well…sometimes even too passionate!) we have had and the valuable directions he gave me.

However, joining the “cedric group” gave me also the opportunity to create bonds with many other individuals that I would like to mention here, starting from my colleagues and PhD candidates, Fatima- Zahra Kaghat, Pedro Alessio, Rodrigo Almeida and Shuo-Hsu-Sian. I am also more than happy for having met or/and worked with Eric Gressier-Soudan, Jerome Dupire, Françoise Shailan, Cecile Le Prado, Isabelle Astic, Colline Aunis, and Vivian Gal. Having all met you makes me feel particularly happy, lucky and privileged for joining the lab; this is the reason for which I am sure that many of the moments we spent together will be engraved in my memory as some of the happiest of my life.

The fourth family that I have to thank is the family I joined as soon as I moved from Rennes to Paris:

these are the people I met while living in the Cite Internationale de Paris and more in particular in the Hellenic Foundation (Fondation Hellenique). To all of my friends and neighbors as well as to the direction, administration, maintenance and security agents of this historical institution I owe a great thank.

Finally, I also want to thank my friend Manolis Delakis for substantially helping me out with the final proofreading of this PhD report.

Last, but certainly not least, comes my own family for its continuous and unreserved support. It is to my mother, my father, my brother and my beloved grandmother that I co-dedicate this work.

(8)

(9)

BRIEF CONTENTS

INTRODUCTION... 25

1.1 SCOPE OF THE THESIS... 25

1.2 RESEARCH MOTIVATIONS... 25

1.3 THE RESEARCH SETTING... 29

1.4 RESEARCH QUESTIONS... 30

1.5 RESEARCH APPROACH AND METHODOLOGICAL CONSIDERATIONS... 31

1.6 COLLABORATION STATEMENT... 31

1.7 STRUCTURE OF THE THESIS... 32

1.8 RELATED PUBLICATIONS... 35

MOBILE MULTIMEDIA GUIDES IN THE MUSEUM SETTING ... 37

2.1 INTRODUCTION... 37

2.2 A CROSSROAD BETWEEN TRADITION AND INNOVATION... 37

2.3 CRITERIA OF CLASSIFICATION FOR MOBILE GUIDES IN THE MUSEUM SETTING... 39

2.4 THE MOBIVISIT PROJECT... 62

2.5 THE DANAE PROJECT... 71

2.6 BARRIERS TO OVERCOME AND OPEN ISSUES... 80

2.7 DISCUSSION... 84

INTRODUCING AUGMENTED REALITY FOR MOBILE MULTIMEDIA MUSEUM GUIDES ... 86

3.2 INTERACTION WITH MOBILE GUIDES IN THE MUSEUM SETTING... 86

3.3 DEFINING AUGMENTED REALITY... 91

3.4 APPLICATION REQUIREMENTS FOR MOBILE AR SYSTEMS... 95

3.5 MOBILE AR APPLICATIONS... 100

3.6 AUGMENTED REALITY APPLICATIONS AND CULTURAL HERITAGE... 103

3.7 LIMITATIONS OF THE AR APPROACH... 112

DESIGN AND IMPLEMENTATION OF THE AR GUIDE ... 117

4.2 A COMPREHENSIVE FUNCTIONS LIST... 117

4.3 DESCRIPTION OF THE 1^ST MUSEUM GUIDE AR PROTOTYPE... 127

4.4 MAIN STAKEHOLDERS MOTIVATIONS AND NEEDS... 134

4.5 THE SETTING OF THE MAIN STUDY... 136

4.6 LIFE CYCLE OF THE MAIN AUGMENTED REALITY PROTOTYPE... 137

4.7 DESCRIPTION OF THE 2^ND MUSEUM GUIDE AR PROTOTYPE... 141

4.8 PREPARING THE CONTENT... 146

4.9 A WALKTHROUGH IN THE FINAL AR APPLICATION... 153

4.9 SUMMARY-CONCLUSIONS... 154

METHODOLOGY FOR EVALUATION AND DATA COLLECTION... 155

5.2 EVALUATION FOR MOBILE GUIDES IN THE MUSEUM SETTING... 155

5.3 PLANNING THE AR GUIDE EVALUATION... 165

5.4 TASK AND EXPERIMENTAL SETUP... 179

DATA ANALYSIS AND RESULTS ... 189

6.2 SYNCHRONOUS AND ASYNCHRONOUS OBSERVATION... 189

6.3 THE SURVEY... 198

6.4 SEMI-STRUCTURED INTERVIEWS... 213

6.5 THE FOCUS GROUPS... 221

6.6 SYNTHESIS OF THE MAIN FINDINGS AND RESULTS... 240

(10)

CONCLUSIONS AND FUTURE WORK ... 254

7.1 SUMMARY... 254

7.2 CONTRIBUTIONS... 255

7.3 A CRITICAL APPRAISAL OF THE AR APPROACH... 259

7.4 FUTURE WORK WITHIN A SHORT-TERM HORIZON... 260

7.5 FUTURE WORK WITHIN A LONG-TERM HORIZON... 263

7.6 ON THE FINISHING LINE... 267 BIBLIOGRAPHY ... 269-289 APPENDICES………..291-367

(11)

Figure 2.8a-2.8d: Existing alternative (other than PDAs) mobile platforms for the delivery of multimedia guided visits. From left to right: Use of a mobile phone (Pian, Traverso et al., 2004), a UMPC (Damala, Cubaud et al., 2008), a laptop (Lefftz, d'Hoedt et al., 2003) and a Tablet PC (DANAE project)……….61

Figure 2.9: Mobivisit project, Screenshots of the mobile museum guide, Museum of Fine Arts, Lyon (Damala, Le Coq et al., 2005)………64

Figure 2.10: Mobivisit project, Examples of exhibit presentations (Damala, Le Coq et al., 2005)………...65

Figure 2.11: Mobivisit project, Navigation flowchart……….66

Figure 2.12: Mobivisit project, the fill-in form used for geolocalization and snapshots of the interactive museum plan (Damala, Le Coq et al., 2005)………67

Figure 2.13: Interactives used in the permanent “Your World, my world exhibition”, Museon museum, Netherlands………...73

Figure 2.14: DANAE project, Simplified adaptation architecture (Brelot, Cotarmanach et al., 2005)……….74

Figure 2.15a- 2.15b: DANAE project, session mobility function (A) and one of the DANAE avatars (B) (to the left the original design by Jean-Marie Boomputte, to the right two screen shots of the 3D-rendered avatar)…………....76

Figure 2.16: DANAE project, the content representation graph………....77

Figure 2.17a-2.17b: DANAE project: Plan of the new exhibitions area (A) and the corresponding user interfaces created for the TabletPC delivery (B) (Brelot, Cotarmanach et al., 2005)………...79

Figure 3.1: Contributing academic disciplines and interdisciplinary fields concerned with interaction design according to (Preece, Sharp et al., 2007)………87

Figure 3.2: Co-existence of interaction entities in the museum setting………...89

Figure 3.3: The first AR display (to the left, the mechanism used to maintain the display in place)………91

Figure 3.4: Milgram's et al. Mixed Reality Continuum (Milgram et al., 1994)………...92

Figure 3.5: Basic components of mobile AR systems………95

Figure 3.6: Examples of fiducials/markers used for pose estimation in AR applications………....96

Figure 3.7: Pilot application running on Nintendo DS, Walt Disney World Resort, USA………..98

(17)

Figure 3.8a-3.8b: A. Mockup of lightweight optical display (Low October 27th, 2008) and B. The Sony Glasstron

video see-through display (CNN February 2, 2000)………..99

Figure 3.9a-3.9f: Future AR displays. A. An AR magnifying lens prototype (Greene, 2008), B. “microvision” augmented car windshield (Microvision October 27th, 2008), C. “A-Rage” outdoor AR gaming prototype (A- Rage.Com October 27th, 2008), D. Funamizu’s mockup of the ultimate AR display(Funamizu October 27th, 2008), E. “microvision” eyewear for AR (Microvision October 27th, 2008) and F. the AR contact lens under deployment at the University of Washington……… ………100

Figure 3.10a-3.10c: A. The virtual showcase, B. Ename 974 project and C. Y-Dreams AR outdoor kiosk… ……..104

Figure 3.11a-3.11d: Wearable AR systems applied in the domain of Cultural Heritage. Illustrations extracted from (Gleue and Daehne, 2001; Anastopoulou and Sotiriou, 2005)………..104

Figure 3.12: The Archeoguide project, non-augmented and augmented view of the Hera temple in Olympia (illustrations extracted by (Vlahakis, Ioannidis et al., 2002)……… 105

Figure 3.13: Virtual humans acting in Pompeii archeological site, LIFEPLUS project (images source: Miralab-Geneva University)………...106

Figure 3.14a-3.14d: The four exhibits used in connect project (Anastopoulou and Sotiriou, 2005)………..107

Figure 3.15: VITA system: Real view of the excavated site and its MR lab reconstruction (Benko, Ishak et al., 2004)………...108

Figure 3.16: AR applications used as substitutes of real objects. Illustrations extracted from (Woods, Billinghurst et al., 2004, Liu, Cheok et al. 2007)………..109

Figure 3.17: The AR guide implemented for the Museum of Natural History, Tokyo, Japan (Kondo et al., 2007)...111

Figure 4.1: Front and back view of the 2nd candidate device equipped with a standard webcam………129

Figure 4.2: The 2nd candidate device considered the delivery of the application ………129

Figure 4.3: Mockup created prior to the implementation of the 1st AR prototype………...131

Figure 4.4: Mockup created prior to the implementation of the 1st AR prototype and detail………...132

Figure 4.5: Snapshot of the 1st AR prototype………..133

Figure 4.6: Testing of the 1st AR prototype in the lab……….135

Figure 4.7a-4.7b: a. Guided visit in the museum of Fine Arts in Rennes. b. Hands-on educational activities in the museum………..137

Figure 4.8a-4.8b: a.CD’s used for signposting the existence of mp3 commented works, b. screenshot of the interactive game “The room of wonders” ………...138

Figure 4.9: Initial list of paintings………139

Figure 4.10: The four selected paintings………..140

Figure 4.11: Pipeline of the Museum of Fine Arts Augmented Reality Guide………..142

Figure 4.12: Architecture of the 2nd AR prototype (illustration extracted from (Jouvin, 2007)………144

Figure 4.13: The physical archive of the Museum of Fine Arts and example of the folders used for the content creation (courtesy of the Museum of Fine Arts in Rennes)………..148

Figure 4.14: example of resources “discovered” in the museum archives, providing scenario ideas for the application content………....149

(18)

Figure 4.15: To the right, the iterative design process followed and relations between all three AR prototypes, to the

left graph of the ISO 13407 standard…………...………....150

Figure 4.16: Navigation scheme of the mobile AR guide……….151

Figure 4.17: Mockups demonstrating the navigation scheme adopted for all paintings………152

Figure 4.18: Examples of the navigation scheme employed for the paintings………...153

Figure 5.1: A graph illustrating the three main phases of an ideal exhibition planning process……….158

Figure 5.2a-5.2b: A. Positioning of the main actors during the observation, B. Experimental conditions, plan of one of the museum galleries………...181

Figure 5.3a-5.3b: A. a student wearing the waist-bag containing the ARCHOS player B. a student wearing the web camera for the recording on the on-screen action………....182

Figure 5.4a-5.4b: Set-up of the settings in which the two focus groups took place………...186

Figure 6.1a-6.1d: Visitor attitudes using the mobile guide during the visit………....192

Figure 6.2a-6.2b: Visitor change of focus of attention, as recorded by the ARCHOS multimedia recorder……..…193

Figure 6.3a-6.3b: Visitors trying to compensate the lack of headphones in a noisy environment………..194

Figure 6.4: Young visitors manifesting their curiosity for the multimedia guide………...194

Figure 6.5: Example of the optimal but also uncomfortable position required for the ARCHOS recordings………...197

Figure 6.6: Preparation, analysis and reporting of the semi-structured interviews………215

Figure 6.7: A Fine Arts student mimicking the way audio guides are sometimes used in museums (”They are almost scotched in front of it”)………...226

Figure 6.8: Reactions provoked by the AR posters presented ………230

Figure 6.9: Participants imitating uncomfortable positions while using the guide……….234

Figure 6.10: Participant demonstrating a possible way of gesture interaction with a mobile AR museum application………...238

Figure 6.11: The poster of “must-have” and “must-have-not characteristics” as created by the participants………241

(19)

LIST OF CHARTS

Charts Graph 1: Visitors’ profiles and museum visiting habits………200

Charts Graph 2: Usability of the AR guide………..201

Charts Graph 3: Content Effectiveness………...203

Charts Graph 4: Comparative Use of Interpretation Media……….203

LIST OF TABLES

Table 2.1: Advantages and disadvantages of using museum provided and personal terminals (Damala, Le Coq et al. 2005)……….70

Table 4.1: Inventory of Mobile Museum Guides Functions and Interrelations with AR………....120-125 Table 5.1: Comparative table of evaluation methodologies applied on mobile museum guide projects………160

Table 5.2: The initial evaluation key-points classification grid………..162

Table 5.3: Correlating AR research questions with mobile museum guides’ evaluation key points………...177

Table 5.4: Taxonomy of evaluation key points for the assessment of mobile multimedia museum guides………....188

Table 6.1: Average scores for the section “Usability of the guide”………...202

Table 6.2: Answers obtained for the open-ended question “Is there anything that you would wish the guide to do?”………202

Table 6.3: Answers obtained for the open-ended question regarding the usability of the guide………202

Table 6.4: Average scores obtained for the section of “Content Effectiveness”………...204

Table 6.5: The Object-Guide correlation score………206

Table 6.6: Appreciated aspects of using the guide………....206

Table 6.7: Difficulties encountered while using the guide………206

Table 6.8: Participants expressing prerequisites for a hypothetical future use of the guide………....207

(20)

(21)

Famous Blue Raincoat It’s four in the morning, the end of December I’m writing you now just to see if you’re better New York is cold, but I like where I’m living There’s music on Clinton street all through the evening.

I hear that you’re building your little house deep in the desert You’re living for nothing now, I hope you’re keeping some kind of record.

Yes, and Jane came by with a lock of your hair She said that you gave it to her That night that you planned to go clear Did you ever go clear?

Ah, the last time we saw you, you looked so much older Your famous blue raincoat was torn at the shoulder You’d been to the station to meet every train And you came home without Lily Marlene And you treated my woman to a flake of your life And when she came back she was nobody’s wife.

Well I see you there with the rose in your teeth One more thin gypsy thief Well I see Jane’s awake -- She sends her regards.

And what can I tell you my brother, my killer What can I possibly say?

I guess that I miss you, I guess I forgive you, I’m glad you stood in my way.

If you ever come by here, for Jane or for me Your enemy is sleeping, and his woman is free.

Yes, and thanks, for the trouble you took from her eyes I thought it was there for good so I never tried.

And Jane came by with a lock of your hair She said that you gave it to her That night that you planned to go clear

Sincerely, L. Cohen

(22)

(23)

To “Jane”

(24)

(25)

PART A

«Calme, l’esprit lucide, je regarde le monde et je dis : Tout ce que je vois, entends, goute, sens et touche, est création de mon entendement. Le soleil monte et descends dans mon crane. Dans une de mes tempes se lève le soleil ; dans l’autre le soleil se couche»

Nikos Kazantzakis, Ascèse, Salvatores Dei

“It is only with the heart that one can see rightly; what is essential is invisible to the eye."

Antoine de Saint Exupéry, The Little Prince

(26)

(27)

CHAPTER 1

INTRODUCTION 1.1 SCOPE OF THE THESIS

This thesis examines the context of use of mobile multimedia handheld guides and applications as an alternative interpretation medium in the museum environment. Its main research hypothesis is that the use of mobile Augmented Reality (AR) technologies and the AR metaphor as a principal component for the conceptual and interaction design of mobile museum guide applications could greatly facilitate interaction and navigation both in the mobile multimedia application and in the sensitive museum ecology.

The topic treated appertains in the wider domain of Human-Computer Interaction (HCI) with mobile devices and services or Mobile Human Computer Interaction (MHCI). However, because of the specificity of the examined research hypothesis, the thesis additionally aims to contribute methodologically but also empirically in the current state of the art regarding the interaction, interaction design and evaluation of mobile AR applications. Finally, due to the nature of the domain-space in which the experimental interventions took place, the museum environment, the thesis also sheds light on the ways by which using mobile multimedia guides can alter, transform or enhance the museum visiting experience and the relations of contemporary museums with their public.

1.2 RESEARCH MOTIVATIONS

1.2.1 UNDERSTANDING MOBILE CONTEXTS

Mobile information and communication devices and services are spreading globally with a pace that has never been seen before, especially if one compares with the slower pace of adoption of office information systems. Initially conceived at the late 1970s and early 1980s in Japan and Scandinavia, mobile phones and telephony have undergone a considerable transition from a technology-focused professional tool to a show-off gadget and lately a mass-market consumer product adopted by hundreds of millions of people globally (Kiljander, 2004). Indisputably, currently, owning a mobile phone has largely become a norm as well as a social necessity, at least in western and developed societies.

(28)

The spread of mobile telephony and the change that everyday communications underwent in everyday life is not the only important change witnessed in the course of the last years. Together with the possibility of communicating with anyone, anywhere, anytime, other forms of communication and use of mobile devices and services started to emerge. The most illustrative example is provided by the largely unpredicted success of the short message service (SMS), currently one of the most widely used data applications on the planet (Baron et al., 2006), representing a massive industry reported to worth globally over 80 billion dollars in 2006 (ITU, 2006).

As storage and processing power capacities do not cease to evolve and the population gets more and more accustomed in owning, using and manipulating mobile devices, a large variety of other multimedia applications became available to such a degree that we are now facing the

“convergence” phenomenon: the transformation of mobile phones to devices that can read and send mail, play music, store photographs, be used as playing platforms, provide navigation and way-finding assistance, act as video and audio recorders or even as e-wallets. For countries in the avant-garde of new technologies appropriation, like Japan, it has been argued that mobiles have become more important than house keys, serving not only for the consultation of digital content but also as a remote control for many other devices (Sukemoto, 2004).

A common parameter of all these developments is that, in most of the cases, it is the user needs as well as the users’ patterns of behaviors that shape future directions for services, devices and products and not just simply the industry (Harper, 2003). The possibilities seem still very rich and diverse as, despite the remarkable advances in mobile computing, there are still target groups (like for example senior citizens (Love, 2005) as well as types and contexts of applications that just start to get explored.

At a time where our contemporary knowledge society sets new requirements for training and education with an accentuated emphasis on lifelong learning, one of the least explored and most promising challenges regarding mobile computing is to understand how mobile and tangible devices and technologies can be used in order to better support learning (Vavoula and Karagiannidis, 2005). Mobile learning has the potential to adapt to the learner’s preferences and capabilities and to accompany learning not only in formal but also in informal learning environments seamlessly and discretely, featuring new, engaging and interactive activities (Naismith et al., 2006). The New Media Consortium (NMC), an international not-for-profit consortium regrouping nearly 300 learning-focused organizations has since 2006 consecrating parts of its annual “Horizon” report in the ways mobile technologies can be used to enhance on-

(29)

site, collaborative and interactive learning in formal and informal learning environments (NMC, 2009).

Within this scope, the introduction and use of mobile audio and multimedia guides in the museum environment can be considered as a distinct category of mobile, interactive and multimedia edutainment application. The next section focuses on the particularities of the context in which the PhD Thesis interventions and experimentations took place, the museum environment.

1.2.2 MUSEUMS AND THE INTERPRETATION OF CULTURAL HERITAGE

The last decades have seen museums all over the globe literally re-inventing themselves (Valance, 2007). Since the 1970s, the western industrialized world has been witnessing a “boom” both in relative museum numbers but also in terms of role and influence exercised in our contemporary societies. In Europe, it is estimated that for each museum that existed in the 1950s, four exist today (Ambrose and Paine, 2006), while in the United States attendance of museums doubled from 200 million in 1965 to nearly 400 million in 1984 and 500 million some years later, rendering thus museum visiting as the one of the most popular family activities in America (Falk and Dierking, 1992). And though this trend cannot but be viewed positively, adverse consequences can also be observed. In Germany, for example, the number of museums increased by 30% in between 1991-1996 but the public’s attendance only by 5% (Burton and Scott, 2007).

Museums are thus competing not only with each other but also with other venues and attractions for a public that has less time available than ever before.

However, the most decisive change of the last decades is not the increase in terms of existing museum numbers but the increasing “opening up” of the museums and their collections to their public. In other words, museums passed from a state of “being about something” to “being about somebody” (Weil, 2007); and while they have been characterized at their first state as

“elitist” (Schubert, 2000) or “static” storehouses, mainly focused on the acquisition, preservation and study of their collection that was subsequently presented as something objective and rational without further discussion or explanation (Hooper-Greenhill, 1994), today museums seem to understand that public exhibition and interpretation is at least equally if not more important than collection and research. Reflective of this tendency is that during the 1990s in Australia and the USA many museums devoted more resources to visitor and audience research than to any other activity (Griffin and Abraham, 2007).

(30)

At a time when we observe a developing emphasis on lifelong learning regarding both formal and informal learning environments, the museum setting -defined by the International Council of Museums (ICOM) as “a non-profit, permanent institution in the service of society and its development, open to the public, which acquires, conserves, researches, communicates and exhibits the tangible and intangible heritage of humanity and its environment for the purposes of education, study and enjoyment” (ICOM, 2007)- seems to represent an ideal, multisensory, informal learning environment, open to and able to satisfy a wide public, of different ages, abilities and backgrounds (Dierking, 2007).

The importance of this particular character of a museum visit is also strengthened by the common assumption that museums and other cultural heritage related institutions incorporate not simply objects, but the intellectual values, history and tradition of the society, thus emphasizing the continuity between the previous and the upcoming generations. This is also the reason for which the public sees in museums a leisure activity, closely connected with learning, and differentiated from other “short-term thrills” attractions (Hooper-Greenhill, 1994).

As the change in terms of focus and mission of museums is changing, museum visitors are valued more and more as active meaning makers with a right to a meaningful and informative experience when engaged in a museum visit (Berger, 2004, Doering, 2007, Falk and Dierking, 1992, Schubert, 2000). Museums therefore are now more than ever before experimenting with different strategies and different media not only in order to assist their visitors in getting the most out of a museum visit but also in order to create long lasting relationships with their public. Educational programs for families and schools, guided visits for individuals or handicapped people, audio guides, hands-on workshops and multimedia applications represent now some of the standard interpretation media visitors are provided with in many museums and cultural institutions.

Under this scope, the introduction of lightweight handheld multimedia guides as personalized assistants for interpretation purposes can be seen as one of the latest trends regarding museum educational and interpretation policies, providing a spectrum of functions that will be further examined in the next chapters (particularly in Chapter 2 and 4). On the other hand, examined under the scope of the MHCI community, museums, because of their public and educational character, seem to represent an ideal testbed for experimentations with mobile computing, mobile learning and entertainment applications.

(31)

1.3 THE RESEARCH SETTING

This study has been conducted in an industrial and business-driven Research and Development environment, France Telecom R&D (currently Orange Labs), in Cesson Sevigné, France. France Telecom is primarily a telecommunications provider, though lately a number of other services are also proposed to companies and the large public in several countries in Europe, Asia (Peking, Japan) and USA (San Francisco, Boston). The R&D division employs approximately 3000 engineers who deploy their research in the fields of signal processing and telecommunications, but also in other computer science disciplines, ranging from image, video and audio coding, to networks, information systems, multimedia, and interaction design.

The France Telecom R&D laboratory that hosted the internship, IRIS (Image, Rich Media, New Interactions and Hyperlanguages), has been particularly active in Virtual (VR) and Augmented Reality (AR) environments including intelligent, 3D avatars and agents. Some example applications is the 3D visualization of the city of Rennes for which a 3D interactive visualisation tool was developed allowing to access information on public services (location, opening hours, etc.), public transportation (location, timetable) or sport installations. Virtual Collaborative Environments is another axis of research and experimentations aiming to explore and investigate novel interface solutions. The work performed on intelligent avatars on the other hand, aims to develop a more user-friendly and entertaining access to distant communication services.

Augmented Reality (AR) has been lately representing another important sector of research for the IRIS lab. The Augmented Reality (AR) concept aims to enhance our real world perception, combining it with fictitious, computer generated objects. Some of the related topics of research are the development of computer vision algorithms enabling robust and accurate bindings between real and virtual worlds, the definition of new metaphors in order to control the interactions between the virtual world and the coexisting real world, and hardware developments for appropriate visualization systems (for instance see-through glasses). The possible market for AR applications can be industrial (maintenance, assembly), personal (gaming, support for disabled and elderly people, online shopping), mobile (geolocalized services, gaming), artistic, medical or educational. However, as AR is a relatively young and discipline, AR applications have been so far mostly addressed to a very marginal public related with specific disciplines (medical and military training, architecture, urban planning, industrial maintenance etc). Within this scope, the public and rich in connotations character of the museum environment renders museums very interesting as experimentation environments for mobile AR applications.

(32)

The PhD thesis research also benefited from a very close collaboration with the research team Médias Interactifs et Mobilité (Interactive Media and Mobility) of the CEDRIC laboratory in the Conservatoire National des Arts et Métiers, Paris, France. Some of the main research activities are related with the conception of multimodal, interactive 2D and 3D interfaces (visual and auditory) and the development of tools and environments for the authoring of multimedia applications. Particular emphasis is given on the techniques of conception and development of video games as well as on mobile multimedia applications and digital libraries. The laboratory maintains close relations with the Musée des Arts et Métiers and has been actively involved in the digitisation of the virtual library of the Conservatoire (cnum.cnam.fr), featuring more than 600 000 digitised pages, among which some very important 19^th century journals, of particular interest for the scientific community of the History of Science.

1.4 RESEARCH QUESTIONS

In the previous sections the amalgamation of the PhD topic, as a result of both the author’s motivations and objectives and the research motivations and activities of the settings in which this study was undertaken was examined. More in particular, and with a chronological order, the research questions that were formed are the following:

- What is the context of use of mobile multimedia museum guides and how do they differ or complement other already existing interpretation means? Which are the current challenges and barriers involved in the conception, implementation and integration of this interpretation medium in the museum environment?

- In which ways interaction while using a mobile multimedia museum guide differs from other cases of mobile human computer interaction?

- Can mobile AR technologies and the AR metaphor constitute an interesting alternative for interaction and navigation in a mobile multimedia museum guide application but also in the sensitive museum ecology? Can AR facilitate orientation in navigation in the museum space and the interactive application? Is mobile AR easy and intuitive to understand even by non experienced IT users? Are there any new emerging ideas regarding possible ways of interaction with mobile AR applications?

The research and methodological approach adopted for answering these questions is visited in the next section.

(33)

1.5 RESEARCH APPROACH AND METHODOLOGICAL CONSIDERATIONS The nature of the treated topic implied from the very beginning an interdisciplinary approach. As a consequence the research and methodology employed associated resources from the domains of human computer interaction (HCI), augmented reality (AR), interaction design, museum and visitor studies and evaluation. The study also benefited by the ongoing progress and research in the field of cultural informatics, a concept broad enough to accommodate work carried out in diverse fields such as databases, digitization, computer animation, virtual and augmented reality and interaction design and evaluation, employed for the documentation and interpretation of cultural heritage and cultural heritage related applications.

In more practical terms the literature review was combined with hands-on experience in three projects (DANAE, Mobivisit and Museum of Fine Arts in Rennes projects), numerous on-site museum visits and multifaceted fieldwork undertaken in order to conceive, design, author and evaluate a mobile multimedia guide using mobile AR technologies and the AR metaphor as the predominant geolocalization, orientation and navigation component. Finally, the main AR prototype, built in order to explore the main research hypothesis through the integration of the AR metaphor in a mobile multimedia museum guide, was largely conceived and evaluated using participatory design practices which resulted in informative feedback that was taken under consideration in the next- financed by the French National Agency for Research (Agence National de Recherche) - phase of the mobile AR guide of the Museum of Fine Arts in Rennes.

1.6 COLLABORATION STATEMENT

The following entities and people deserve a specific mention, since a substantial part of the thesis would not have been possible without them. Following a chronological order:

- Christian Lecoq, research engineer, France Telecom R&D, for sharing important information regarding a more ancient mobile museum guide prototype developed by France Telecom R&D.

- The DANAE consortium and partners who contributed to the overall planning of the design and implementation of the Museon museum mobile museum guide application.

- Hub Kockelkorn, museum curator and head of the New Technologies department of Museon museum in Den Hague, Netherlands, who worked jointly with the author for the initial and already published version of the mobile museum guides evaluation taxonomy.