Automatic Prediction of Health Literacy through an eye tracker

(1)

F

ACULDADE DE

E

NGENHARIA DA

U

NIVERSIDADE DO

P

ORTO

Automatic Prediction of Health

Literacy through an eye tracker

Edgar Duarte Ramos

D

ISSERTATION

Mestrado Integrado em Engenharia Informática e Computação Supervisor: Carla Alexandra Teixeira Lopes

(2)

The present thesis has been developed within the project "NORTE-01-0145-FEDER-000016", financed by the North Portugal Regional Operational Programme (NORTE 2020),

under the PORTUGAL 2020 Partnership Agreement, and through the European Regional Development Fund (ERDF).

(3)

Automatic Prediction of Health Literacy through an eye

tracker

Edgar Duarte Ramos

Mestrado Integrado em Engenharia Informática e Computação

(4)

(5)

Abstract

The Web is one of the main sources of information. Health-related topics are one of the most searched topics on Web, fact corroborated by a study made in 2013 which concluded that 72% of the web users in the USA use it to search for information about health. Statistics gathered in Europe show that almost every country had percentages of individuals searching online for health information above 50% in 2017. With this high percentage of people searching for health topics online, it is noticeable that their diversity is great, which makes difficult the return of the most appropriate results to everyone.

Health literacy is the degree to which an individual has the capacity to obtain, communicate, process and understand basic health information and services to make appropriate health decisions. It is important because, among other things, it determines the capacity individuals have to manage their health. Regarding health information retrieval, it influences the way people understand the documents retrieved from the Web and by understanding individual’s health literacy it may be possible to customize the results returned on the search engine results page (SERP).

Health Literacy can be assessed using instruments that were already validated by other studies, being the same based on questionnaires that are applied to people. That way, this study is important because we’re trying to predict health literacy without being intrusive through an eye-tracker.

Eye tracking is the process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head. Also, as eye tracking technology allows us to assess how long users spend looking at specific components of one object, we can determine the important aspects of one web page to the user. Eye tracking technology is used in many different areas of research being very important in every one of them. Furthermore, one of the largest eye-tracker systems vendors (SensoMotoric Instruments) was acquired by Apple which indicates that this technology will be integrated into Apple products, which will, on one hand, give even more visibility to eye tracking technology and, on the other hand, will make this study have a short-term impact.

This work aims to understand if eye movements vary according to the person’s health literacy during the search for health related topics on the web, through the use of an eye tracker. This study consisted of an experiment with users in a controlled environment. Also, there were created work tasks situations in order to motivate the research on health related topics, registering and comparing the ocular movements of the two groups of users, of high (adequate) and low (inadequate) health literacy.

During the study itself, analyzing the points where one person is looking on a specific web page, or a predefined set of pages, we’ll try to see if there is a pattern between participants’ health literacy and their eye movement patterns regarding those pages. Then, the data will be analyzed trying to understand if there is a relation between health literacy and how individuals view information related to health on the Web.

Finally, this study lead to conclude that participants with higher health literacy were more careful when trying to retrieve information about health conditions, spending more time in SERPs and giving more importance to the source of the content presented to them.

(6)

(7)

Resumo

A Web é uma das principais fontes de informação. Os tópicos relacionados com a saúde são um dos tópicos mais pesquisados na Web, facto esse corroborado por um estudo realizado em 2013 que concluiu que 72% dos utilizadores da web nos EUA usam a mesma para pesquisar informações sobre saúde. Estatísticas obtidas na Europa mostram também que quase todos os países apresentaram percentagens de pesquisa online sobre tópicos de saúde acima de 50% em 2017. Com esta elevada percentagem de pessoas que pesquisam sobre tópicos de saúde online, é percetível que a diversidade das mesmo é grande, o que faz com que seja devolver os resultados mais adequados a todos.

Literacia em Saúde pode ser descrita como o “grau com que um indivíduo tem a capacidade de obter, comunicar, processar e compreender informações e serviços básicos de saúde para tomar decisões de saúde apropriadas”. A literacia em saúde é importante, entre outras coisas, porque determina a capacidade que as pessoas têm de gerir a sua saúde. Relativamente à obtenção de informação sobre saúde, a literacia em saúde influencia a maneira como as pessoas compreendem os documentos recuperados da Web e percebendo a literacia em saúde de um indivíduo poderá ser possível personalizar os resultados retornados na página de resultados do motor de busca.

A literacia em saúde pode ser avaliada usando instrumentos que já foram validados por outros estudos, sendo os mesmos baseados em questionários aplicados às pessoas. Assim, este estudo é importante na medida em que estamos a tentar induzir a literacia em saúde de um indivíduo de forma não intrusiva através de um eye tracker.

Eye trackingé o processo de medir o ponto onde o olhar se está a fixar (referindo-se ao ponto para o qual o indivíduo está a olhar) ou o movimento de um olho em relação à cabeça. Além disso, como a tecnologia de eye tracking nos permite avaliar quanto tempo os utilizadores gastam a olhar para componentes específicos de um objeto, é possível determinar os aspetos importantes de uma página da web para um indivíduo em específico. A tecnologia de eye tracking é usada em muitas áreas de pesquisa, sendo importante em todas elas. Além disso, a Apple adquiriu um dos maiores fornecedores de sistemas de eye tracking (SensoMotoric Instruments), facto esse que, por um lado, dará ainda mais visibilidade à tecnologia de eye tracking e, por outro, fará com que este estudo tenha impacto a curto prazo.

Este trabalho visa perceber se o os padrões oculares durantes as pesquisas de informação de saúde na web mudam consoante a literacia em saúde das pessoas, através do uso de um eye tracker. Este estudo consistiu na realização de uma experiência com utilizadores em ambiente controlado. Foram criadas work tasks situations de forma a motivar a realização da pesquisa sobre tópicos de saúde, registando-se e compararando-se os movimentos oculares dos dois grupos de utilizadores, de alta (adequada) e baixa (inadequada) literacia em saúde.

O primeiro passo será formar dois grupos, com 15 pessoas cada, agrupando indivíduos com alta literacia em saúde num grupo e indivíduos com baixa literacia em saúde no outro grupo. For-mados este dois grupos distintos, o próximo passo será a analise da literacia em saúde de cada um dos individuos constituintes, recorrendo, para tal, ao NVS e ao METER, na expectativa de que o

(8)

uso de dois instrumentos em detrimento de apenas um, torne os resultados mais precisos e, con-sequentemente, facilite a tarefa de agrupar os participantes no respetivo grupo, determinado pelo teste, ou seja, adequado ou inadequado nivel de literacia em saúde. Durante o estudo, analisando os pontos para onde o indivíduo está a olhar enquanto pesquisa numa página da web específica, ou um conjunto de páginas predefinidas, tentaremos ver se existe um padrão entre a literacia em saúde dos participantes e os seus padrões de movimentos oculares, em relação a essas páginas. Os dados serão depois analisados tentando entender se existe uma relação entre literacia em saúde e a forma como os indivíduos observam informações relacionadas com saúde na Web.

Por último, este estudo permitiu concluir que os participantes com maior literacia em saúde tiveram mais cuidado na obtenção da informação sobre as condições de saúde que estavam a pesquisar, passando mais tempo na SERP e dando mais importância à fonte do conteúdo que lhes estava a ser apresentado.

(9)

Acknowledgements

I would like to thank everyone involved in this dissertation. It was a long period of development and I thank to professor Carla Lopes with all the help and guidance help me a lot, motivating me when I needed, so thank you.

Also, I’m truly thankful to my girlfriend which gave me unconditional support not only during the development of this dissertation but during the years of this challenging master’s degree. To my family, my parents and my sister, special thanks for allowing me to end this stage of my life with success, helping me to grow professionally and personally. To my close friends that helped me throughout this journey, giving me good advices on the most opportune moments, I thank Bruna due to all the explaining and support, Soraia for the unconditional support, my group of “Boleias da Maia”, André, Luís, Leonardo e Gustavo for sharing this journey with me, Ricardo for being a good friend and giving good advices and my friends known during the realization of this Master’s, Ivo, Marta, Daniel, thank you for all the support!

Finally, I thank Doctor Dagmara Paiva and all people belonging to SexLab for their help throughout this process because without them none of this would be possible.

(10)

(11)

“The science of today is the technology of tomorrow”

(12)

(13)

List of Figures

2.1 Conceptual Model of Health Literacy created by Mancuso [Man08] . . . 6

2.2 NVS nutrition label . . . 8

2.3 Types of devices of an eye tracker . . . 11

2.4 Heatmap and Scanpath Visualization Example . . . 14

2.5 Areas Of Interest and the amount of time looked at . . . 14

2.6 F pattern when looking for information online . . . 18

4.1 Determining the health literacy group of the participants . . . 25

4.2 Flow of the experiment . . . 28

4.3 Gaze Replay on SMI BeGaze . . . 29

4.4 Heat Map on SMI BeGaze . . . 30

4.6 Examples of a Result Page - being the left part the original page and the right part of the figure the manipulated page . . . 30

4.5 Examples of a SERP Page - being the left part the original page and the right part of the figure the manipulated page . . . 31

4.7 AOI Editor on SMI BeGaze . . . 33

5.1 Scanpath of both participants visualization pattern on the first query (on the left the image from participant 30 and on the right the figure relative to participant 2) 38 5.2 Heatmap of Participant with ID 30 Visualization pattern on the first query . . . . 38

5.3 Scanpath of both participants visualization pattern on the third query (on the left the image from participant 30 and on the right the figure relative to participant 2) 39 5.4 Heatmap of both participants visualization pattern on the third query (on the left the image from participant 1 and on the right the figure relative to participant 30) 40 5.5 Scanpath of both participants visualization pattern on the third query (on the left the image from participant 1 and on the right the figure relative to participant 30) 40 A.1 Focus Map statistics . . . 61

A.2 KPI statistics . . . 62

A.3 Scan path statistics . . . 62

A.4 AOI Sequence Chart statistics . . . 62

A.5 Gridded AOIs statistics . . . 63

B.1 Questionnaire applied to participants . . . 66

D.1 METER . . . 74

D.2 NVS . . . 75

D.3 Informed consent . . . 75

(16)

LIST OF FIGURES

D.5 Task 2 presented to the participants . . . 76 D.6 Task 3 presented to the participants . . . 77

(17)

List of Tables

2.1 Main Eye Movements . . . 11

2.2 More Eye Movements . . . 12

2.3 Research areas using eye-tracking - Data retrieved from https://www.tobiipro.com/ 15 2.4 Variables analyzed by other authors . . . 20

4.1 Work tasks situations . . . 26

4.2 Queries . . . 26

4.3 Latin Square Design for the first 6 participants . . . 27

4.4 SERP AOIs . . . 32

4.5 Results Pages AOIs . . . 32

5.1 Comparison of SERP Behavior Analysis by health literacy group . . . 43

5.2 Comparison of SERP Behavior Analysis by health literacy group . . . 44

5.3 Result Pages Behavior Analysis - Total duration of fixations (in seconds) for each AOI per group . . . 45

5.4 Comparison of Results Pages Behavior Analysis by health literacy group . . . 46

C.1 Results seen before first click . . . 67

C.2 Clicks on SERP Results (results seen by session) . . . 68

C.3 Number of fixations per participant for each SERP . . . 69

C.4 Time spent per participant in each SERP . . . 70

C.5 Number of fixations in Title, Link and Snippet in all 3 tasks’ SERP . . . 71

(18)

(19)

Abbreviations

AOI Area of Interest

HIR Health Information Retrieval HL High Literacy

IR Information Retrieval LL Low Literacy

METER Medical Term Recognition Test NVS Newest Vital Sign

REALM Rapid Estimate of Adult Literacy in Medicine SERP Search Engine Results Page

TOFHLA The Test of Functional Health Literacy in Adults UK United Kingdom

UP University Of Porto URL Uniform Resource Locator USA United States of America USB Universal Serial Bus

(20)

(21)

Chapter 1

Introduction

1.1 Context

Online health searches are becoming very popular, as corroborated by a study that showed that around 72% of adult Internet users on US use the web to search for health-related topics, being the most common topics diseases and treatment information, as Susannah Fox documented [Fox14]. Furthermore, statistics retrieved from Eurostat [Eur18] show the evolution of individuals using the Internet to seek for health-related information. Those statistics are related to people aging from 16 to 74 and show the percentage of people using the Internet for seeking health-related information. The statistics related to the year 2018 show that Netherlands has 72%, being the country with the highest proportion among those, and that only 18 countries have less than 50% of individuals using the Internet for seeking health-related information, and that from that 18 countries 9 have between 45% and 50%. On the other hand, statistics from from the same source regarding the year of 2010 reveal a percentage of 58% from Luxembourg and that only 4 of those countries have percentages over 50%, corroborating the idea stated previously that health related searches are increasing.

With the increase of online searches about health, it is possible to perceive that different indi-viduals may experience the same information retrieved in a different way. At this point the topic of health literacy is introduced to the degree that it influences how each user perceives health information.

Health literacy is the “degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions.” [PR10,PSS+14]. So, health literacy can be interpreted as an individual characteristic in the way that each person has her own literacy and therefore, different people might have different under-standings of the same health information. Health literacy can also be a collective concept being qualified as distributed health literacy [EWDE], however, the focus of this study was the indi-vidual health literacy. That way, health literacy has a huge impact when searching online for health-related topics because individuals with limited health literacy may have more difficulties to understand the information available on the web.

(22)

Introduction

content, an eye tracker allows the study of eye-movement in different areas of research. There are many different examples of this, like the study of the professional performance - it’s interesting for them to understand their work environment and which human factors may be influencing their performance - or in the marketing area and consumer research where the eye tracking technology allows to measure consumers’ attention and responses to marketing strategies, which allows the marketers to design effectively to catch the shoppers’ eye. Thus, and since eye tracking technology has become increasingly popular over time, we realize that it plays an important role in several areas of study allowing researchers to understand interactions that people have (consciously or even unconsciously) that without this technology would not be possible to perceive.

1.2 Problem Definition

Given the diversity of people searching on the web, we believe that search engines could provide better service if they adapt the information given to users according to their health literacy. With this in mind, we want to see if this can be done in a non-intrusive way through their eye movements, due to the fact that the instruments that exist nowadays to measure health literacy are intrusive.

Nowadays, health literacy is measured through the application of instruments that usually take the form of questionnaires. However, when searching online it is very intrusive to use those methods which lead to the need to find new methods. Like this, through the use of an eye tracker, we can see if there is a pattern between where one is looking on the web page and the person’s health literacy. We believe that through eye movements patterns, when searching online, it is possible to detect one’s health literacy.

1.3 Motivation and Goals

This project intends to study if eye movements vary according to individual’s health literacy when searching online for health information. If we are able to prove this is possible, this conclusion will allow to raise this notion in order to later, associating this study with other studies, try to customize the information retrieved from the Web to every person, adapting it in the best way possible to the person’s health literacy. If there is a pattern on how people look in the Web page according to people’s health literacy the results of searches can be more effective giving people information more adjusted to their needs.

As explained before, when searching online detecting individual’s health literacy is important because it may lead to a personalization of results given to the user. If the user has high health literacy, the search engine detects it and the results can be personalized - For example they could be scientific articles, that are more complex but as the user has high health literacy can more easily perceive the information present on the same. Also, for the users with low health literacy the same personalization is possible to be made, for example, the same articles may not have use to him (it may be more complicated for the users with lower health literacy to understand the information

(23)

Introduction

present on those articles), and the search engine could present to the same links with more direct and simple information.

This study may have, even more, a short-term impact with the acquisition of SensoMotoric Instruments by Apple, which points to the fact that eye tracking technology is evolving very fast, being Apple one of the biggest companies nowadays.

Furthermore, this project had different phases, which included the preparation of the study, when it was necessary to recruit 30 individuals and do the setup of all the software involved. This phase was followed by the study itself, when the participants had to perform the search about three different health topics. Finally, all the recorded sessions were analyzed in order to generate statistics and, at the end, understand if health literacy influences the eye movements of people searching online for health related information.

1.4 Report Structure

This report will be structured in seven chapters, being the following six chapters briefly explained below.

After this chapter comes the Chapter 2 which describes the state of the art of the project. There are projects related to Health Literacy trying to understand its evolution and importance which will be analyzed on the chapter mentioned. Also instruments, previously validated, to assess health literacy will be analyzed and all the eye tracking technology and its evolution will be object of analysis during the same chapter. Finally, Health Information Retrieval and Seeking will be addressed along with studies using Eye trackers in Research and Information Retrieval. Throughout this chapter, all the important information already known about those subjects will be deepened.

Chapter 3purpose is to explain the issues that this project is facing and the proposed solutions. On Chapter4it is possible to see the implementation of this project. As it is a study, all the preparation of the same and software used is analyzed during this chapter.

Then, Chapter5 shows all the results. It’s described how participants of the study are dis-tributed and the results retrieved from the experience made.

After describing the results obtained, there is the need to analyze and draw conclusions from them, which is made in Chapter6.

At last, the Chapter 7will focus on the future work associated with this project regarding the conclusions revealed from the investigation.

(24)

(25)

Chapter 2

State of the Art

2.1 Health Literacy

The definition of Health Literacy appears in multiple articles, having as example the definition created by Ratzan and Parker in 2000, reinforced in 2010 [PR10], as being the “degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions.”

However, in 2010, Berkman N., Davis T. and McCormack L. [BDM10] also claimed that this concept continues to evolve, and reaching a consensus is hard due to the amount of different skills, which have been increasing, identified as necessary to have health literacy. The article suggests modifying minor things on the definition stated previously, resulting on “The degree to which individuals can obtain, process, understand, and communicate about health-related information needed to make informed health decisions.” This new definition substitutes “have the capacity to” with “can” and adds the skill “communicate about”, for example, separating even more health literacy from intelligence. It became even more clear that health literacy is necessary, and essential, in society, being important all those skills associated to it.

Also, Sørensen [SVF+12] refers that “Health literacy concerns the knowledge and compe-tences of persons to meet the complex demands of health in modern society”.

This subject has been studied for a long time and various definitions have been created. In fact, 250 definitions appear in different articles according to Malloy-Weir [MW16]. One of the main points present on every definition is that health literacy involve competencies to access, understand and use information about health.

Furthermore, health literacy is considered, in many studies, as a concept that is becoming increasingly popular over time due to the fact that health literacy skills are so important in health care. As Parker [Par00] had stated in 2000, a misunderstanding of a disease or treatment may be harmful and lead to an even worse disease.

Regarding health literacy, limited literacy is often associated with misunderstanding (or com-promised understanding) of health-related information, and it’s also related with the inability to comply to medical prescriptions [The12]. The impact that limited literacy can have is to be noted

(26)

State of the Art

because it is more likely for those people to be hospitalized more often as Safeer and Keenen [SK15] referred in 2015.

Beyond that, it remains unclear what influences individuals health literacy, for example, in-dividuals on the same sociodemographic conditions and educational attainment have different literacy (ranging from very low to very high). The reason for such variability is still unclear [WMM+05].

Health literacy is also defined like an “involved” process, in the way that it requires skills and abilities to navigate through information properly [LMB+12]. Literacy refers to a dynamic process referring to the ability to use skills to interpret the information when facing the need of getting that type of information. Loureiro L., Mendes A., Barroso T. et al. [LMB+12] also claim that health literacy should become a fundamental principle in schools’ health programs, because it promote and enhances the adoption of healthier behavior. So, literacy viewed in this way, allows the development of skills necessary for the empowerment of individuals.

Moreover, in 2008, a conceptual model was created by Mancuso J. [Man08], trying to define health literacy. In this model the author recognizes the presence of antecedents (competences, attributes and individual and social consequences) having as background the relation between the individual and society. Also, there is the concept of competence (antecedent) as a set of knowledge, skills and attitudes implying that an individual is able to apply his knowledge in an appropriate way, given an adjust response to the situation. It’s possible to see that the central attributes of health literacy are capacity, comprehension and communication, which are preceded by the skills necessary to achieve health literacy, presented as competences as shown in Figure 2.1.

(27)

State of the Art

Finally, it is possible to understand that health literacy has a huge impact on everyone because it affects the way that people interact with health systems. In Portugal, this topic has also been increasingly studied, culminating in the study made by Rita Espanha [E6] that positioned Portugal within the Europe dynamics related to this topic. Furthermore, the results of the same study showed that Portugal is situated below the average for the countries in the European study.

2.1.1 Health Literacy Assessment

Paiva D., Silva S., Severo M. et al. [PSS+14] claimed that “Health literacy is measured us-ing instruments based on word recognition/pronunciation”. However, some of the instruments also require some other skills, like calculation and interpretation. In this section it will be ana-lyzed some instruments that can measure health literacy, like Rapid Estimate of Adult Literacy in Medicine (REALM) understood in Section2.1.1.1, The Test of Functional Health Literacy in Adults (TOFHLA) presented in Section2.1.1.2, Newest Vital Sign (NVS) deepened in Section 2.1.1.3and Medical Term Recognition Test (METER) presented in Section2.1.1.4.

2.1.1.1 Rapid Estimate of Adult Literacy in Medicine

REALM is the abbreviation of Rapid Estimate of Adult Literacy in Medicine. It was designed to be used in public health and primary care settings to identify patients with low reading levels, according to Davis T., Long S. et al. [DLJ+93].

In 1993, it was performed a study in order to “abbreviate” REALM. In this study, Davis [DLJ+93] came to the conclusion that REALM assess patients reading ability. Furthermore, REALM was considered to be very practical when considering primary care.

Later in 2006, Wallace L., Rogers E., Roskos S et al. [WRR+06] determined patients health literacy. The subjects of the study were English-speaking patients, over 18 years old, attending a university-based primary care clinic. Based on REALM score, “participants were classified as having limited (minus or equal to 6th grade reading level; REALM=0 to 44), marginal (7th- to 8th-grade reading level; REALM=45 to 60), or adequate (greater or equal to 9th grade reading level; REALM=61 to 66) health literacy skills”. At the end, the results revealed that 17.7% of the 305 participants appeared to have limited health literacy, 17.1% marginal and 65.2% high health literacy.

2.1.1.2 The Test of Functional Health Literacy in Adults

TOFHLA, short for “The Test of Functional Health Literacy in Adults”, was developed in 1995 as part of the study whose authors are Parker R., Baker D., Williams M. et al. [PBWN95]. The main goal of the study was to create a reliable instrument that was able to determine the functional health literacy of patients accurately, consisting in a 50-item reading comprehension, along with 17-item numerical ability test, taking about 22 minutes to administrate. In this study were approached 505 patients, including 256 patients speaking English and 249 patients that spoke Spanish. After the patients took the test, TOFHLA showed good correlation with REALM, being 0.84. At the end,

(28)

State of the Art

this study revealed that TOFHLA had fulfill its initial goal revealing itself reliable when it comes to measure functional literacy in patients. Although, the data that they retrieved suggested that many of the patients cannot read properly.

In 1999 [BWP+99], an abbreviated version of TOFHLA was made and entitled S-TOFHLA. The TOFHLA was reduced from 17 to 4 numeracy items, being administered in 12 minutes in-stead of 22. Also, the correlation between S-TOFHLA and REALM stayed very high (around 0.80), however the two tests revealed disagreements that should be considered. As mentioned in the study, S-TOFHLA is very practical when it comes to asses functional health literacy. This instrument demonstrates good reliability and it can be used to assess when patients need special assistance.

Afterwards, in 2009, Oliveira M. et al. [OPMB09] analyzed this tool in relation to the Alzheimer’s disease and Mild Cognitive Impairment concluding that the S-TOFHLA seemed to be a useful measure for determining the level of literacy in Mild Cognitive Impairment (MCI) patients, but not in Alzheimer’s disease (AD) patients.

2.1.1.3 Newest Vital Sign

The Newest Vital Sign, or NVS, is a quiz with six orally administered questions regarding an ice cream nutrition label, illustrated in Figure2.2, which the individual has to view and answer accordingly. The questionnaire includes questions involving finding information in the label, for example, identifying the presence of an ingredient, and other questions involving the execution of numerical calculations (e.g.: calculate the amount of calories ingested when more than one portion is consumed).

Figure 2.2: NVS nutrition label

Proposed on an article elaborated by Weiss B., Mays M., Martz W. et al. [WMM+05], NVS is one way to measure someone’s literacy. In 2005, and correlating with TOFHLA, NVS was created to be an efficient and quick instrument to measure health literacy, requiring only 3 minutes for administration.

NVS was created from different scenarios, developed by health literacy experts, including health-related information in each scenario. In turn, patients would have to answer questions

(29)

State of the Art

about the scenarios. Later, each answer was scored which lead to a final score for each person. With the help of the patients and interviewers the scenarios were refined gradually.

In 2007, Osborn C and Weiss B [OWD+07] evaluated the performance of NVS comparing its performance with existing literacy measures. That way, and confronting NVS against REALM and S-TOFHLA, “the NVS demonstrated high sensitivity for detecting limited literacy and moderate specificity”, although it was not as effective as S-TOFHLA when predicting health outcomes.

Further, a study [PJR+11] performed in 2011 showed that NVS may not be very effective determining health literacy in older African-American people, which indicates the need of chose the right circumstances for the patients, so that we have the best possible result.

Although many studies showed that testing health literacy could lead to embarrassment or stigmatization on the part of the patients, in 2010 a study whose authors are VanGeest J., Welch V. and Weiner S. [VWW10], referred that not only NVS was a good way to test someone’s health literacy but also did not cause the patients to feel shameful in any way. Going even further, 97% of the patients said that they would even recommend the test.

Furthermore, in 2013, Rowlands G., Khazaezadeh N., Oteng-Ntim E. et al. [RKON+13], modifying NVS and applying it to the UK, also performed a study that led to the conclusion that NVS had showed high internal consistency, authenticating it as a valid instrument to assess Health Literacy, being one of the main pros the fact that it is so easy to use. Finally, they referred that some characteristics of NVS-UK, such as its speed, simplicity and validity, made it ideal to investigate the influence of health literacy not only in health but also in disease.

A year before, in 2012, NVS was studied among Portuguese adolescents [PS12], published in [Beh12]. NVS score had a significant positive correlation with Portuguese and mathematics school grades. Despite these results, the overall accuracy of the Portuguese version was lower than the existing versions of English and Spanish. In a different article Santos O., Azevedo A., Oliveira A. et al. [SAO+12], performed another study on the Portuguese population but it was done in a different age group, focusing this time on the young adults/adults. NVS showed a reliability score (0.6 - Kuder-Richardson coefficient) that was decent considering that these instruments consists in a small quiz, with only 6 questions.

2.1.1.4 Medical Term Recognition Test

In 2010, Rawson et al. [RGH+10] developed an instrument of health literacy, that was brief and simple, resulting in the Medical Term Recognition Test or, in short, METER. The study was made using as object of study 155 patients of cardiology at an urban hospital. METER was used together with REALM (in which it has shown to have a correlation of 0.74), and with self-report questionnaires about health behaviors. The internal consistency of the instrument was very high (reaching almost 1, with a score of 0.93) and it demonstrated to be very fast as it took 2 minutes to complete. The test included 80 total words being 40 of those medical terms and another 40 non-medical terms. During the study, the patient should mark only those words that he recognizes as being related to medical terms. At the end they concluded that “METER is a quick and practical measure of health literacy for use in clinical settings”.

(30)

State of the Art

Applying METER to the Portuguese population, trying to adapt the same to its culture, Paiva et al. [PSS+14] in 2014, performed a study with 249 participants and, as in the study analyzed previously, “METER showed a high degree of internal consistency, and acceptable test–retest reliability”. For this cross-cultural adaptation, the METER was translated to Portuguese so that the language was not an obstacle to the performance of the test. As authors concluded, this instrument looks beyond vocabulary knowledge, contrary to other instruments, differentiating individuals by their health knowledge too. This study demonstrates itself important because it highlights the issue that is the presence of limited health literacy, raising awareness in general Portuguese community and its politics.

2.2 Eye tracking technology

Eye tracking is the technique of analyzing the movement of the eyes and the response of the eyes to various stimulus [Sen10,YS75]. The eye tracker technology is used on many fields and it was pioneered in reading research [IPPR89].

The gaze position is one of the most important aspects when researchers are studying the eye movement. Gaze is the externally-observable indicator of human visual attention, being the gaze position the point where the eye is actually looking, being this fact studied since the late eighteenth century [Hue08].

Formerly, techniques like Electro-oculography technique (consisting in the placement of elec-trodes around the eye to detect eye movement), or methods requiring the use of (huge) contact lenses having on the edge a metal coil, revealed themselves invasive [AGPK13]. To overcome this, modern eye tracking systems use video to determine where someone is looking, the point-of-regard. To infer the point-of-regard it can be used the pupil shape and corneal reflections, for example.

2.2.1 Types of eye-tracker

In terms of eye trackers it should be highlighted the existence of two types: those in which the person has to use them, like glasses, as shown at the left of Figure2.3, and those that record the ocular movement at a distance, normally associated with a monitor, present in the middle of Figure 2.3or a loose device, like the figure present on the right side of the figure present below, which are less intrusive, as we can see on the respective figures [GW02]. Although, the capacity of each eye tracker may be different, for example, the sampling rate, which is the number of samples retrieved per second, may differ from device to device (some have 60Hz, some have 120Hz, ...). Summing up, software embedded in these types of software may differ from one another (depending on its range, as cheaper one’s may have less quality and functionalities).

(31)

State of the Art

Figure 2.3: Types of devices of an eye tracker

2.2.2 Types of Eye Movements

To understand this technology it is also important to understand the movements of the eye, which can be very diversified. There are many patterns of ocular movement [Sen10,YS75] and so the most common eye movements can be grouped into the eye movements present in Table2.1.

Fixations Fixations are very low-velocity movements that correspond to the test person staring at a particular point. Fixation “is a variety of motions which are gener-ally less than 1 degree in amplitude and occur during attempted steady fixation on a target”, according to Young and Sheena [YS75].

Pursuit This movement can be perceived when the eyes follow a moving target in the environment, trying to fix that target on the retina. This movement is done smoothly, however when the eye “loses” the target they will perform “catch-up saccades”, which are rapid eye movements in order to reacquire the target. This movements are not generally under voluntary control, requiring the existence of a moving visual field for their execution, according to Robinson [Rob65]. Saccades Fast eye movements that the eye makes while jumping from point to point in

the stimulus, being triggered by displaying fixation targets at defined times within the stimulus. They are voluntary and are characterized by very high initial acceleration and final deceleration.

Gaze Path It is the path that the eye takes while studying a stimulus image. Gaze path can be thought of as the chronological ordering of fixations and saccades when analyzing an image.

Table 2.1: Main Eye Movements

Those four types of eye movements are the most important types and the ones that eye trackers can detect. However, there are more five types of eye movements, presented briefly in Table2.2.

(32)

State of the Art

Compensatory eye movements

Movements closely related to pursuit movements due to its smoothness. They compensate for active or passive motion of the head, stabilizing the retinal image of fixed objects during head motion.

Vergence eye movements

Movements of the two eyes in opposite directions in order to fuse the image of near or far objects. They are smooth and slow (reaching maximum velocities around 10 deg/sec, with a maximum range of 15 degrees).

Optokinetic nystagmus

Known as “train nystagmus”, consists of a slow phase in which the eye fixates on a portion of the moving field, pursuing it briefly and then executing a fast phase which can have a saccade jump, fixating on a new portion of the field. Vestibular

nystag-mus

It is defined by an oscillatory motion of the eye, being similar to Optokinetic nystagmus having a slow and a fast phase.

Torsional eye movements

Those movements are rotation movements of the eye about the line of gaze (limited usually to 10 degree angles).

Table 2.2: More Eye Movements

When recording studies in computers the cursor is an important factor because it can be the target of users attention. In a study made by Smith and others [SHAZ00] it is revealed that partici-pants visual attention is very correlated with the cursor position. Also, some participartici-pants exhibited switches between focusing on cursor and on the target of the page.

2.2.3 Companies that manufacture eye trackers

The companies that manufacture eye trackers are becoming more and more popular over time. One of the same, SensoMotoric Instruments or SMI, which was founded in 1991 by Winfried Teiwes, was recently purchased by Apple. This acquisition, being Apple one of the largest companies today, may indicate that eye tracking technology will have even more visibility from now on. As it is described later on Chapter4, the eye tracker used in this study was the SMI RED250mobile Eye tracker.

That way, the most important companies manufacturing eye trackers are:

1. Tobii1- Tobii was founded in Sweden in 2001 by John Elvesjö, Mårten Skogö and Henrik Eskilsson and has become one of the main companies in manufacturing eye trackers. That is corroborated by the fact that a huge amount of articles use their tools in their studies. Tobii provides eye tracking units for assist in technologies, research, and gaming.

2. EyeLink 2 - EyeLink eye tracking units are made by the company SR Research. Its eye trackers are cited in over 3290 peer-reviewed publications which reveals that they are very important on the eye tracking market.

1_{https://www.tobii.com/group/about/history-of-tobii/} 2_{http://www.eyelinkinfo.com/publications.html}

(33)

State of the Art

3. LC Technologies3- LC Technologies was founded in 1986 in Virginia. Their products have reputation for being accurate, reliable and easy to use. LC Technologies as their products being use in research, national defense, gaming, virtual worlds, hospitals, and so on. 4. Smart Eye4- Smart Eye was founded in 1999, offering eye tracking units. The company

has as partners big automotive companies (BMW, Mercedes and Audi for example) and major agencies like NASA.

5. The Eye Tribe 5 - The Eye Tribe was founded in Copenhagen in 2007. The goal of the products was to be cheap enough so that private households could afford it. Nowaday, Eye Tribe Software enables eye control on mobile devices, allowing to control it only with the eyes. It allows hands-free navigation of websites and apps, enhanced gaming experiences and among other things.

6. EyeTech6 - Eyetech Digital Systems was created in 1996 in Arizona. The company was the first to create an USB-connected eye tracking unit and a an “eye mouse” for Windows, which meant that users could work on their computers using their eyes to navigate.

7. ISCAN 7 - ISCAN was founded in 1980 and have sold thousands of eye tracking units, being referenced in thousands of publications. Their products are very fast reaching speeds up to 1000Hz.

8. Ergoneers8- Ergoneers was founded in 2005 in Munich, having now a worldwide presence. The company provides eye tracking solutions and market and vehicle research, for example.

2.2.4 Representing data retrieved from eye-tracker studies

After analyzing fixations, saccades, dilation of the pupil, among others, its important for the re-searchers to analyze if there is a pattern that could be found or, at least, what are the sequence of events. Usually researchers collapse information into graphic representations. Those representa-tions are [BKR+14,Bar12]:

1. Heatmaps - Heatmaps are usually semi-transparent (although they can be elaborated with opaque coloring), multi-colored layers over the image analyzed, pointing out with warmer colors the areas of higher attention and with cooler colors the areas that the users gave less attention [Bli10]. However, although this is the most traditional way to represent heat maps, there are some variations obtained by shading the image which only shows without shadow the areas of higher attention [ŠM07]. However it is important to create a correct heat maps

3_{http://www.eyegaze.com/our-story/} 4_{http://smarteye.se/about-us/} 5 http://aws-website-theeyetribe-lbmoo.s3-website-us-east-1.amazonaws.com/theeyetribe.com/the-eye-tribe-story/index.html 6_{https://www.eyetechds.com/about-eyetech.html} 7_{http://www.iscaninc.com/} 8_{http://www.ergoneers.com/en/ergoneers-group/mission/}

(34)

State of the Art

because they are very often used and for wrong reasons [Boj09]. It is important to have caution using them like Bojko refers, directing the appropriate use of heat maps [Boj09]. This author also highlights the importance of the algorithm used to generate heat maps. An example of that is presented at the left of Figure2.49.

Figure 2.4: Heatmap and Scanpath Visualization Example

2. Scanpath Visualizations [NS71] [BKR+14] - representing the points corresponding to fix-ations at each moment as well as a small trait with the line shape, indicating the saccadic movements. Otherwise, scanpath is a fixation path of one subject when viewing a specific pattern [NS71]. The right part of Figure2.4, retrieved from [BKR+14], illustrates the same.

3. Areas of Interest - Areas of Interest (AOI) are areas elaborated by the analysis of the more important parts for the user, or in other words, the parts shown to the individual when performing an experiment using an eye tracker. In Figure 2.5 10 we can see an example of definition of areas of interest in one page and respective results to those areas (present on the chart presented on the right of the figure).

Figure 2.5: Areas Of Interest and the amount of time looked at

9_{https://eyetracking.ch/improve-user-interface-designs/} 10_{http://www.usability.at/e/services/eyetracking-evaluation.html}

(35)

State of the Art

2.2.5 Research areas using eye-tracking

Eye tracking technology is used in many different areas. Due to the fact that eye trackers aren’t in-trusive and some very portable, they can be used in different scenarios with efficiency and without affecting the activity performance. These areas are presented in Table2.3.

Psychology and Neuro-science

Eye trackers are used in this area to understand the connection between what people see and react based on information that they process.

Infant and Child Re-search

Uses eye tracking technology so that is easier to understand the development of a child until early adulthood

Immersive VR Research With a fully-controlled environment (virtual reality) it’s possible to record gaze data.

Marketing and Con-sumer Research

Eye tracking technology is the best way to measure consumers’ attention to marketing strategies, allowing to understand if a prod-uct, that is supposed to have attention from the consumer, is actu-ally receiving it.

Professional Perfor-mance

Eye trackers allow companies to understand their employees ac-tions and if work environment is being favorable to their perfor-mance.

User Experience and In-teraction

Eye tracking provides data about human behavior and how hu-mans interacts with different scenarios and the problems brought by them.

Sports Performance and Research

Eye tracking technology is used in sports trying to understand in-dividual performance of athletes, analyzing their focusing, hand-eye coordination and how they analyze all the components of the game/sport.

Education Eye trackers are used to study education and learning processes, allowing to improve teaching methods.

Clinical Research Eye trackers are an obvious benefit when it comes to detect ocular diseases, and also to identify neural diseases - like autism, among others - and try to learn about them.

Table 2.3: Research areas using eye-tracking - Data retrieved from https://www.tobiipro.com/

2.3 Health Information Retrieval and Seeking

Information Retrieval (IR) was characterized in 1968 [Sal68] as an area concerned with the struc-ture, analysis, organization, storage, searching and retrieval of information. In 1999 a study pre-sented IR like the representation, storage, access and organization of information items [BYRN99]. In other words, IR is focused on indexing and retrieval of information from heterogeneous and mostly textual information resources.

Health Information Retrieval (HIR) focuses on the concepts of IR applied to the health-care domain. With the increase of searches on the Web about health, Health Information Retrieval is gaining attention, not only because it allows the patients to have an easy access to a large amount

(36)

State of the Art

of information, but also because it allows health professionals to easily expand their knowledge. According to Adam Bosworth [Bos07], in a good health system, patients should be able to retrieve the most relevant information related to the search topic, learning and educating people in similar circumstances.

When searching online for health information, the time it takes to load results in the search en-gine results page (SERP) is relevant because at different stages of health/illness individuals search information differently. That fact allowed to the elaboration of eight stages which encompass the different types of users searching health information on the web: “when healthy, when they think they might be ill, before getting a medical test, when diagnosed as ill, before treatment, when receiving the treatment, after treatment/surgery, and when being chronically ill” [Zha10].

For health-related professionals, the need of retrieving valid and correct information is present in everyday life. That way, a study performed by Cheryl Dee [DS05] tried to understand the use of health resources by clinical and nursing students, comprehending if they obtain relevant and reliable information through electronic resources taking advantage of all available resources. The results showed that they were most likely to rely on colleagues and books for medical information and showed that both groups lacked database searching skills.

Health-related information, besides being present online in the form of text, is also in illustra-tions, for example. Can illustrations facilitate the way that people understand health information? Is it easier for everyone to perceive a topic in the best way through illustrations or images? For younger people illustrations may be easier to understand, however to older people they are not, spending more time looking at illustration-related phrases and having poorer comprehension of illustrations than younger people [jLKM09].

Not only age is important when it comes to online searches, also the search of correct infor-mation and the quality of it is very important, having as example the situation of a patient that can easily retrieve wrong information which may lead to a bad treatment of symptoms. That way the overall quality of information seeking and retrieval (along with health information seeking and retrieval) is crucial. Therefore, professionals should recommend websites to their patients so, at the end, the quality of health information retrieval is better [MM04].

Pew Research Center has been studying this topic and analyzing how many people use on-line resources either to search for health topics or to share information they know about health conditions, having as example one study performed by Fox and Jones in 2009 [FJ09].

When analyzing a survey that Pew Research Center did in 2012 on the US, it was found that 72% of adult Internet users used it to search for health-related topics, being the most common topics diseases and treatment information. Also, one-in-four (26%) say they read or watched someone else’s experience or medical issues in the past 12 months. Caregivers and those having a chronic disease are more likely to search on web about health. Although online information is not the main source of information for users, it is an important supplement for those who want to know more about some specific topics.

To corroborate even more the idea that people need to search for health-related information on the Web, previously in 2006 Susannah Fox [Fox06a] had done a study that showed that about 8

(37)

State of the Art

million American adults searched for at least one health topic on a typical day in August 2006. According to other study, performed also by Fox [Fox06b], 66% of health seekers pointed at search engines like Google or Yahoo as their start point with that percentage being even higher re-garding younger people (around 74% of health seekers aged between 18 and 29 start their research on search engines). Also, the great majority of health seekers visit, at least, two websites (around 80%).

Although online information about health can help in different situations, it’s important to make sure that information revealed is correct so that people don’t feel confused or frightened, and so that they don’t put their life at risk with the retrieval of wrong information. On the previsouly study, it was shown that more than half of health seekers (56%) said that they felt confident to raise new questions about the health issue with their doctor and/or felt relieved/comforted by the information they found online. On the other hand, 18% of the health seekers studied said that they felt confused by the information they found online and another 10% were scared due to the seriousness of the information present on the web.

Despite the growing/increasing demand for online health information and guidance, a study [Fox07] shows that women are more likely than men to search online, being that information for own use or someone else’s.

Reinforcing the notion that health-related searches on the Web are increasing, a daily mail notice revealed that three quarters of unwell British people now research their symptoms online, treating themselves at home, naming the search engine Google as “Dr. Google” [Tan17].

Based on the statistics previously shown, as web searches are becoming increasingly impor-tant, it is important to make sure that the information retrieved is as accurate as possible.

2.4 Use of Eye tracking in Information Retrieval Research

The eye-tracker technology is used in many studies in information retrieval and research. One of the reasons why this happens is because it is an easy and non-intrusive way of getting the information that the participants of the study found relevant.

A study made by Soussan Djamasbi in 2011 [DST11], put in evidence that users often favors the top left side of web pages (missing a lot of relevant information), tried to understand through the use of an eye tracker how users view a page. Despite previous studies have revealed great importance given by users to the top left of pages, the study’s results showed that users’ viewing pattern was more scattered, probably because, in the mentioned study, the web pages presented were more complex regarding information itself. A different study, performed by Shrestha in 2007 [SL07], having as base a study made by Nielsen in 2006 [Nie06] revealed that users tend to view information on the web in a “F” pattern, like it is possible to see in Figure2.6. Although, the study made by Soussan Djamasbi [DST11] also leaned over this topic, trying to analyze what is the impact of information retrieval and browsing for information on the web, having as results that the F shape viewing is present, but it can be drastically affected by the visual component of a webpage. Furthermore, this study contributes to design, allowing to a better understanding of the

(38)

State of the Art

best design to a webpage to catch individuals’ attention, and contributes to the area of research of human computer interaction, providing the users’ fixation points when browsing for information.

Figure 2.6: F pattern when looking for information online

Also, in online searches, it is important to know if users analyze a great amount of data before clicking on a specific link. It is important because, regarding to a search engine results page, we can understand if titles and description are important to users or if they click a random link looking for further information. A study made in 2004 [GJG], revealed that, in average, participants took almost 8 seconds to select a document. After analyzing the results of this study, they show that the most important links to the users are the first two links and that fixation drops significantly after the second link.

Recently, in 2017, a study made by Pian W. [PKC17] tried to analyze if it’s possible to iden-tify if people are searching online for themselves, searching for others, or browsing with no health issue in mind. This study used eye-tracking, demographic and urgency (how urgent information retrieval is) information to retrieve data. There were 74 participants analyzed interpreting their eye fixation and saccades in different scenarios. As result, the study revealed that users’ browsing du-rations were significantly different for the three different contexts, so the context can be identified with decent accuracy by analyzing users’ mouse clicks which can be detected by web application in an easy way. On the other hand, understanding charts about health can be difficult for people with low graph literacy. A study made by Yasmina Okan [OGGR16] used an eye-tracker to un-derstand if the literacy on graphs affects the way people interpret the graph. Concluding that most people having low health literacy didn’t have a correct interpretation of the graphs (unlike high literacy individuals).

In 2013, Mackert et al. [MCPW13] published an article of a study that used the eye-tracking technology to explore how people with different health literacy look at health-related

(39)

informa-State of the Art

tion. Participant’s health literacy was measured with assistance of Newest Vital Sign (NVS). The session begins when the NVS nutrition label is presented on the screen. Then the eye-tracker records the subject’s eye movements by noting the individual’s eye gaze path and how long they spent looking at the various points of the screen (how long they look at some point of the label). There were encoded areas containing “relevant” or “nonrelevant” pieces of information based on the information needed to answer the six questions of the NVS health literacy assessment. Re-garding the results, 65.3% of the participants had an NVS score higher or equal to four correct answers, indicating appropriate health literacy, 16.3% had a score of 2-3 indicating possibility of limited literacy, and 18.4% had an NVS score of 0-1, indicating high likelihood of limited literacy. Regarding eye tracking analysis, on average, greater time and number of fixations were given to “relevant” information than to “nonrelevant” information. Associating NVS score with the results retrieved from the eye-tracker, the score was not significantly associated with fixation duration or fixation count for relevant information or for fixation count on nonrelevant information. At the end, the study lead to conclude that people with limited health literacy skills did not spend a lot of time looking at relevant information, spending more time viewing nonrelevant information, having as solution to the problem creating nutrition labels layouts “that force the eye toward the most important and relevant content” which “may make it easier and quicker for patients to find out what they need to know.”

So, is easy to understand that analyzing health literacy through an eye tracker, the data given to health information seekers can be personalized, appropriating the information to the user’s knowledge and capability to understand it.

2.4.1 Variables studied by others authors when using an eye tracker in information retrieval

During our literature review we noticed several variables are considered while using an eye tracker. In Table2.4 those variables analyzed by other authors are presented, being also highlighted the variables in which we were inspired during the analysis of the data obtained in the study, being those variables chosen as a matter of opportunity and feasibility.

(40)

State of the Art

Author Variables analyzed

Cutrell [CG07] Gaze fixations in areas of interest Click accuracy

Mean fixation duration

How many results they look before click on one Total summed duration of fixations on titles Total summed duration of fixations on snippets Total summed duration of fixations on URLs Goldberg [GSL+02] Fixation dwell times within each AOI

Number of fixations Mean fixation durations Mean saccade durations

Total scanpath lengths within each screen Sequence of AOI visits within each screen Dwell times within each screen

An AOI transition matrix, showing the number of transition to and from each AOI on each screen.

Pan [PHG+04] Mean fixation duration

Józsa [JH12] Fixation count and fixation duration Total visit duration

Lorigo [LHB+08] Number of fixations per page Fixation time per page in seconds Duration of fixations in seconds Number of abstracts seen per page Number of abstracts clicked per page Marcos [MGGBP13] Reading patterns by country

Dwell time on SERP

Nr of scanned results by country Success rate

Kammerer [KG12] Total fixation time on search results Nr of selected search results

(41)

Chapter 3

Problem Statement

3.1 Context

With the increase of searches on the web about health topics, it is, more than ever, crucial to provide the most adequate information to patients so that all the search process can be as efficient as possible.

Sometimes the first results of the search engine results page (SERP) may not be the most appropriate for the person who is searching for the information. This is a risk factor taking into account that the misunderstanding of the information obtained is directly harmful to the health of those who are searching and, like we’ve seen, the first two links on the SERP are the most viewed, usually. Along with it, as there are no solutions for automatically detect one’s health literacy, it may be very laborious, complicated and too intrusive to measure health literacy through the existent instruments, like NVS or METER, in order to personalize the retrieved results to the user profile.

As stated previously, this project aims to analyze if eye movements and eye patterns differ from person to person according to their health literacy, when searching online for topics related to health.

With that in mind, we performed a study to understand how different participants, with dif-ferent levels of health literacy, view web pages containing information about health. With that in mind, this study presents itself as a starting point to understand if detecting a person’s health liter-acy could be induced by his eye movements, which, if it does occur, could guide to a customization of the results given to each person, according to their health literacy.

That will allow to a personalization of the contents obtained on the web making the search much more personal and, hopefully to a better lifestyle and better treatment of individual health in a generalized way.

(42)

Problem Statement

3.2 Solution Proposal

This study is a preliminary work to present an introductory study to the topic of automatic detection of health literacy, using an eye tracker. This is a solution that helps significantly the obtaining of correct information leading to a more effective retrieval of health information, leading to an overall better understanding of the contents obtained.

In a first phase, this solution consisted on an experiment, that had as target audience 15 people with low health literacy and 15 people with high health literacy.

Therefore, before carrying out the study itself, it was necessary to recruit users that satisfied our requirements. After that, the study itself was divided on these parts:

1. Preparation of the pages that were seen by the participants (areas of interest were defined so that when participants looked at those areas that could be recorded)

2. The execution of the study itself, presenting the pages to the participants and recording, through an eye tracker, where are they looking.

3. Analysis of the data obtained previously, trying to find a pattern between participant’s health literacy and where they look when searching for information on the web.

(43)

Chapter 4

Methodology

As stated previously, this study, through the elaboration of two groups with different health liter-acy, analyzes whether there is a different pattern in participants’ eye movements when searching the web about health topics.

Furthermore, in order to be valid and respect the moral conditions involved in a study with users, this study was approved by the Ethics Committee of the University of Porto.

4.1 Research question

As mentioned earlier, the goal of our study is to understand if health literacy influences eye move-ments when searching online for health information. Nowadays, health literacy is usually mea-sured using instruments that require the intervention of the users. As this may not be feasible in an online environment, we are working towards the detection of health literacy through other means. That way, the research question of this study is “The health literacy of users affects their health information seeking behaviour, namely their eye movements while interacting with Search Engine Results Pages and Results Pages".

4.2 Participants

Our study had 30 participants with different educational levels (from high school level to a PhD) and with a range of ages between 18 and 59 years. About 53.3% of the participants were men and 46.6% were women. The high health literacy group was constituted by 15 participants being the same aged between 21 and 49 years old, having graduation levels between Graduated and PhD and being 33.3% men and 66.6% women. On the other hand, the 15 participants belonging to the low health literacy group were aged between 18 and 59, had levels of graduation from secondary level to master’s degree and were 73.3% men and 26.6% women.

(44)

Methodology

4.3 Recruitment Procedure

The study’s participants were recruited, first, by email. An email was sent to students from FEUP (Faculty of Engineering from University of Porto), FMUP (Faculty of Medicine from University of Porto) and FPCEUP (Faculty of Psychology and Education Sciences from University of Porto). This email was sent to students from these institutions due to the proximity of each one with the place where the study would have to be carried out (FPCEUP), which facilitated the displacement of those for the study.

From those emails we obtained 10 responses, being those from people that later participated in the study.

Since the responses to the email were not enough to the realization of the study, it was neces-sary to recruit people by approaching them in person, asking if they would like to participate in the study explaining all the steps of the same. Even then, and for some time, it was also difficult to find people who were eligible for the study and people who volunteer to participate in the study.

After the first contact with people, there was a need to understand whether they would be eligible for the study or not. To ensure that people with lower health literacy do not be ashamed, as explained by Parikh [PPN+96], the necessary precautions were taken when approaching each person and when forming the two groups, without stereotyping or being offensive. With that in mind, people were asked to perform the NVS test and Meter test (the Portuguese versions of the tests referred). According to Weiss [WMM+05], a patient that has equal or more than 4 correct answers is unlikely to have low health literacy. Also, according to Paiva and others [PSS+14] on METER a patient has high health literacy if he scores at least the cut-off value in both words and non-words, i.e. 35/40 and 18/30, respectively. Presenting the two tests to people who were interested in carrying out the study, if the result of both tests showed that the person had low health literacy the person would be included on the respective group, being the same applied to people with high health literacy. In other words, a person was only accepted to participate in the study if both NVS and METER determined that the person had low health literacy or if both determined that the person had high health literacy. With this restriction, we hope to have participants assigned to the most appropriate group for them, analyzing not only the result of one test, but of two tests to the participants’ literacy. Another criterion of inclusion in the study was whether participants had already used a computer and searched on Google. Thus, if any of the people approached did not have some of these two requirements they would not be included in the study. This is summed on the scheme present on Figure4.1

That way, the two tests were presented like it is possible to see on FigureD.1andD.2, METER and NVS respectively.

After the recruitment of the participants, and assuming that they verified all the conditions stated previously, each participant was given an ID, and no personal information was used during the study and analysis of the results.

Automatic Prediction of Health Literacy through an eye tracker

F

E

U

P

Automatic Prediction of Health

Literacy through an eye tracker

Edgar Duarte Ramos

D

Automatic Prediction of Health Literacy through an eye

tracker

Edgar Duarte Ramos

Mestrado Integrado em Engenharia Informática e Computação

Abstract

Resumo

Acknowledgements

Contents

List of Figures

List of Tables

Abbreviations

Chapter 1

Introduction

1.1

Context

1.2

Problem Definition

1.3

Motivation and Goals

1.4

Report Structure

Chapter 2

State of the Art

2.1

Health Literacy

2.2

Eye tracking technology

2.3

Health Information Retrieval and Seeking

2.4

Use of Eye tracking in Information Retrieval Research

Chapter 3

Problem Statement

3.1

Context

3.2

Solution Proposal

Chapter 4

Methodology

4.1

Research question

4.2

Participants

4.3

Recruitment Procedure