Medical UI/UX
Daniel Simões Lopes, PhD
INESC-ID Lisboa
Daniel Simões Lopes
Instituto Superior Técnico UT Austin / Portugal Post-Doc at VIMMI Heads Biomedical Research Line PI of project IT-MEDEX
http://web.ist.utl.pt/daniel.s.lopes
Research Interests:
Computer Graphics VR/AR
Medical UI/UX
Computer-Aided Design
Fundamental Research
Human Computer Interaction
Computer Graphics
Applied Computing
CAD for Engineering CAD for Architecture
Oil & Gas
Molecular Biology Accessibility
Teaching
Cultural Heritage etc
Medical UI/UX
http://jnm.snmjournals.org/content/45/2/279/F2.expansion.html http://jnm.snmjournals.org/content/52/10/1501/F1.expansion.html http://jnm.snmjournals.org/content/51/8/1198/F3.expansion.html
Motivation
Motivation
Motivation
Motivation
Source: OsiriX
Motivation
Motivation
Motivation
Motivation
Motivation
Motivation
Motivation
Medical UI/UX
Interactive Technologies
Interactive surfaces (large displays)
Interactive surfaces (mobile devices)
Head Mounted Displays
Virtual Reality
Augmented Reality
Motion capture & motion tracking (body and hands)
Reality–Virtuality Continuum
Real Reality Immersive VR
Mixed Reality
Augmented Reality
Extended Reality
Problematic
Problematic
Healthcare Scenarios
Scenario: Learning
Scenario: Radiodiagnostics
Scenario: Surgery
Scenario: Communication
Scenario: Image-Based Anatomical Modeling
Medical UI/UX
Medical Education
Patient-Clinician Communication
Clinical and Surgical Scenarios
Radiology
Rehabilitation
Surgical Planning Surgical Navigation
Challenges
Building a network of healthcare professionals
Identifying current practices
Linguistic Barrier
Big Picture but no Specs
“Automagic worshippers”
Published Work
&
Ongoing Projects
Interactive
Rehabilitation
Easing the patient’s patience
Motivation
REPETITION – FEEDBACK – MOTIVATION
Motivation
$
COST WEARABLES
Motivation
COMPENSATORY MOVEMENTS
COLLABORATION
Motivation
Motivation
Motivation
Motivation
Augmented Reality
Motivation
SleeveAR
Augmented Rehabilitation
SleeveAR
SleeveAR
SleeveAR
SleeveAR
Video at https://vimeo.com/144008920
BROTHERS-IN-ARMS
Upper Limb
Rehabilitation
BROTHERS-IN-ARMS
BROTHERS-IN-ARMS
BROTHERS-IN-ARMS
FRONT RAISE LATERAL RAISE HAND TO MOUTH
COMB HAIR FILL THE CUP
Exercise
Immersive Radiology
Getting closer to your voxels
VR for Radiologists in the Reading Room
VR for Radiologists in the Reading Room
Ambient Factors
Display Quality Ambient Light
Glare
Point-of-view variations
VR for Radiologists in the Reading Room
VR for Radiologists in the Reading Room
Video at https://vimeo.com/197920563
Evaluation
Radiology Neuroradiology Gynecology
General Surgery Dental Implantology Obstetrics Medical Specialities
5 with no previous experience in VR
VR for Radiologists in the Reading Room
CAVE COLON: Immersive Virtual Colonography
CAVE COLON: Immersive Virtual Colonography
CAVE COLON: Immersive Virtual Colonography
Interactive Medical Visualization
Touching is seeing
VOXEL TIPs: Coloring 3D Medical Images
VOXEL TIPs: Coloring 3D Medical Images
Video at https://vimeo.com/239797237
Contactless Surgical Navigation
Germ-free interfaces
VOXEL Explorer: Simple Gestures for Exploring 3D Images
Video at https://vimeo.com/226416353
?
Conventional Systems: Dimensional Gap
Source: O’Hara et al., 2014
VOXEL Explorer: Asepsis
VOXEL Explorer: Simple Gestures
Novel Spatial Interaction Techniques for
Exploring 3D Medical Images 65
VOXEL Explorer: Track your Body
VOXEL Explorer: Gameboy Metaphor
Fraunhofer Portugal Challenge 2016 selects the best scientific ideas
1 st PRIZE: Pedro Parreira (IST-Univ Lisboa)
"Novel Spatial Interaction Techniques for Exploring 3D Medical Images"
TAP TAP
SWIVEL
a b c
FEET Input for Contactless Hand Gesture Interaction
Augmented Surgery
More than meets the scalpel
The Promise of an “Augmented Reality”
Motivation
Laparoscopic HUD
Augmented Laparoscopic
Navigation
Motivation
Motivation
Motivation
Laparoscopic Head Mounted Display
ImplantAR
Augmented Implant
Planning
Motivation
Motivation
Motivation
+
Motivation
ImplantAR
ImplantAR
Biokinematics
The Skeleton Dance
Plausible Skeletons
Tracking Body Segments
Depth Sensor
Master Thesis available at
https://fenix.tecnico.ulisboa.pt/cursos/mebio m/dissertacao/1972678479053137
Depth Sensor
Imperceptible Breathing Movements
Imperceptible Breathing Movements
Imperceptible Breathing Movements
Imperceptible Breathing Movements
SleeveAR (João Vieira, Maurício Sousa)
BROTHERS-IN-ARMS (André Domingues, Soraia Paulo)
Voxel TIPs (Rita Mendes, Pedro Parreira, Vasco Pires, SPaulo) VRRRRoom (Maurício Sousa, Daniel Mendes, Soraia Paulo) CAVE COLON (Pedro Borges, Daniel Medeiros, Soraia Paulo) Voxel Explorer (Pedro Parreira, Soraia Paulo)
FEETICHE (Filipe Relvas, Soraia Paulo)
Laparoscopic HUD (Miguel Belo, Soraia Paulo) IMPLANTAR (Tiago J. Silva, Miguel Belo)
Plausible Skeletons (Nuno Matias)
Imperceptible Breathing (Filipe Marques)
Acknowledgments
Champalimaud Foundation, Lisboa
Chaire UNESCO en enseignement et recherche en anatomie numérique Université Paris Descartes
Hospital Prof. Dr. Fernando Fonseca, E.P.E., Amadora
CiiEM, Centro de Investigação Interdisciplinar Egas Moniz, Monte da Caparica Clínica Dentária Egas Moniz, Monte da Caparica
Escola Superior de Saúde Egas Moniz, Monte da Caparica
Serviço de Neurorradiologia - Centro Hospitalar de Lisboa Central, E.P.E.
Centro de Imagiologia, Hospital da Luz, Lisboa
Instituto da Próstata e Incontinência Urinária, Lisboa
Unidade Funcional de Patologia Mamária - Centro Hospitalar de Lisboa Central, E.P.E.
Ortopedia da Criança e do Adolescente, CUF Descobertas, Lisboa Serviço de Ortopedia, Hospital Beatriz Ângelo, Loures
Clinical Partners
Acknowledgments
IT-MEDEX - Interactive Tablets for Collaborative Scenarios Related to 3D Medical Image Exploration
PTDC/EEI-SII/6038/2014 (€173.882,00) http://it-medex.inesc-id.pt/home
Contacts
Joaquim Jorge (VIMMI Group Leader) jorgej@acm.org
http://web.ist.utl.pt/jorgej/
Daniel Simões Lopes (Principal Investigator) daniel.lopes@inesc-id.pt
http://web.tecnico.ulisboa.pt/daniel.s.lopes/
