Table 2.8: Grand Challenges in HCI [Shn+16;Ste+19]. Cells highlighted in grey corre-spond to challenges associated toXRtechnologies.
Source Challenges
Grand Challenges for HCI Researchers [Shn+16]
Develop a handbook of human needs
Shift from user experience to community experience Refine theories of persuasion
Encourage resource conservation Shape the learning health system Advance the design of medical devices Support successful aging strategies Promote lifelong learning
Stimulate rapid interface learning Engineer new business models
Design novel input and output devices Accelerate analytic clarity
Amplify empathy, compassion, and caring Secure cyberspace
Encourage reflection, calmness, and mindfulness Clarify responsibility and accountability
Seven Grand HCI Challenges [Ste+19]
Human-Technology Symbiosis Human-Environment Interactions Ethics, Privacy and Security
Well-being, Health and Eudaimonia Accessibility and Universal Access Learning and Creativity
Social Organization and Democracy
Table 2.9: Key Barriers in AuthoringAR/VRApplications (adapted from [Ash+20])
Barriers Examples problems
Difficult to know where to start
Problems related to understanding theAR/VRlandscape and se-lecting tools.
Difficult to make use of online learning resources
Problems related to understanding nomenclature, formulating search queries and finding relevant and up-to-date information.
Lack of concrete design guidelines and examples
Problems related to knowing what is good design inAR/VR rel-ative to good design in other types of development and lack of example projects.
Difficult to design for the physical aspect of immersive experiences
Problems related to the design of natural VR experiences and un-derstanding of human motion, gesture, and audio design.
Difficult to plan and simulate motion inAR
Problems with planning targeted experiences inARand forecast-ing users’ actions/movements.
Difficult to design story-driven immersive experiences
Problems related to providing a compelling, distraction-freeAR experience.
Too many unknowns in development, testing, and debugging
Problems related to constant changes in AR/VR technology and viable debugging strategies.
User testing and evaluation challenges
Problems related to understanding of viable testing methods, users’ knowledge of tool usage and accessibility toAR/VRdevices.
2 . 4 . C O N C LU S I O N
applying trade-offs when needed, as to maintain the illusion of the virtual world. For this mimicking to happen,XRsystems depend onperceptual systemsand the ability of tech-nology to create artificial stimuli, as detailed in section2.2. Trajectories[Ben+09;BG08;
BG11] with a focus on the creator’s intention and performance (through the mapping between space, time, role, and interface) can also be applied toVR. In particular, Jaller and Serafin [JS20] argue for its application to360º video.
Narrowing the focus toVR, Steinicke [Ste16b] identified scientific, technological, and social challenges needing further attention:
• Health and Cybersickness- Steinicke identifies cybersickness/VIMSas a "crucial prob-lem with currentVRtechnology" [Ste16b, p. 48], highlighting the need for long-term studies on the possible health impact ofVRheadsets.
• Unlimited Locomotion- Walking is a fundamental interaction in exploring virtual environments. While devices with positional tracking have allowed more freedom in moving, the limited tracking space still restricts "real walking".
• Missing Realistic Visual-Haptic Interaction- Another base interaction in exploring virtual environments is touching, grasping, lifting, or deforming objects. This is a hard challenge forVR systems due to the complex integration of physical and perceptual systemsneeded to achieve a realistic solution.
• Inadequate Self-representation - user’s self-representation contributes to a strong sense of agency and illusions of embodiment. At the moment, self-representation in mostVRsystems is limited to the position and orientation of the head and hands.
• Isolated Social User Experience- most research focuses on single-user systems due to complexity. AsVR becomes more mainstream, the need to understand social interactions and collaboration is expected to grow.
Intersecting challenges inHCI(table2.8), the barriers affectingXRpractitioners (table 2.9), challenges in VR [Ste16b], and the prevalence, potential and research directions for 360º video (as detailed in subsection 2.3.3), two areas of interest in user experience in 360º video(see fig. 2.5) emerge as relevant for this thesis work:
• Attention Guidance
– Relation to HCI - Guiding users’s attention when interacting with interfaces has been a focus of research since earlyHCI. This focused concern withUXD [Ben19b, p. xv] is present in post-WIMPframeworks: inTrajectories[Ben+09;
BG08; BG11] where there is a concerned mapping with what the creator ex-pects and how theparticipantinteracts, or inRBI[Gir+19;Jac+08] where we risk participants miss details of the story versus introducing an interaction that allows for theparticipantto accomplish the task.
– Relation to Perceptual Systems-Attention guidanceis dependent onperceptual systemslike the visual and auditory system, described in section2.2.
– Relation to XR/VR/360º video- Previous works have explored the use of differ-ent techniques forattention guidanceinVRand360º video[DGD17;Lin+17a;
Lin+17b;Liu+19;PHA17;She+16;Wal+20;Wan+20]. This is fueled by a need to understand how practitioners can design for these immersive mediums and maintain Presence.
– Related work- Chapter3explores related works in using sound forattention guidance, by first looking at the role of sound inHCIand how sound is spa-tialized.
• Visually Induced Motion Sickness (VIMS)
– Relation to HCI-VIMSis a main usability issue when experiencingVRand360º videousingHMDs. ConsideringRBI[Gir+19;Jac+08], solving this involves a trade-offby introducing an interaction that allows for the performance of the task without physical injury, fatigue, or discomfort.
– Relation to Perceptual Systems - VIMSis caused by a mismatch between (or within) the visual, vestibular, and somatosensory senses, so techniques to miti-gate it require an understanding of perceptual science, specifically on integra-tion and misalignment of stimuli (see subsecintegra-tion2.2.4).
– Relation to XR/VR/360º video- Due to the constructive nature ofVR[BCK17], a large body of works exists involving usability and ergonomics issues such as VIMS.
– Related work- Chapter4explores different strategies in addressing and miti-gatingVIMS.
2 . 4 . C O N C LU S I O N
XR
Perceptual Systems
Human Computer Interaction VR 360º
Video
Auditory System
Visual System
Vestibular System
Sound in HCI Post-WIMP
Theory
Spatial Auditory Interfaces Computer
Vision
Attention Guidance XR VIMS
HCI
Perceptual Systems VIMS
Mitigation
User Experience in 360º Video
Foundations (Chapter 2)
Related Work (Chapter 3 & 4)
Narrowing research scope
Figure 2.5: Narrowing of research scope intoAttention Guidance(chapter3) andVIMS (chapter4)
3
S o u n d f o r A t t e n t i o n G u i d a n c e
This chapter is focused onattention guidancefor360º video. Firstly, the chapter looks at the role of sound inHCIand how it is used in various interfaces. Secondly, the chapter describes what isattention guidanceand how it is used inXRsystems. The chapter concludes with identifying research opportunities intersecting the use of sound and the use ofattention guidancein360º video.a
aConspicuous parts of the text below have appeared in co-authored publications [Bal+18b;Bal+19]