Team-Based Learning Impact on College Students Anxiety

i

Team-Based Learning Impact on College Students’

Anxiety

Andrea Alexandra Gomes Marques

Dissertation presented as partial requirement for obtaining the Master’s degree in Knowledge Management and

Business Intelligence

i

NOVA Information Management School

Instituto Superior de Estatística e Gestão de Informação

TEAM-BASED LEARNING IMPACT ON COLLEGE STUDENTS’ ANXIETY

by

Andrea Alexandra Gomes Marques

Dissertation presented as partial requirement for obtaining the Master’s degree in Information Management with a specialization in Knowledge Management and Business Intelligence

Advisor: Roberto Henriques Co Advisor: Maria Anastasiadou

November 2022

ii

ACKNOWLEDGEMENTS

First and foremost, I would like to thank all my family members especially my dad, mum, and sister for investing in my education, teaching me its importance, and giving me the best conditions to conclude my academic journey. For all the support, and motivation and for always believing in my capabilities.

I am also grateful to all my friends that become a part of the journey for always helping me with my difficulties.

I would also like to emphasize and express my gratitude to Professor Roberto Henriques and Professor Maria Anastasiadou for all the support, availability, and guidance during this process.

This research development would not have been possible without the assistance and resources provided by Professor Lara Oliveira and all the participants.

Finally, I would also like to thank Catarina for being always available to help me with my doubts.

iii

ABSTRACT

Over the years, the anxiety issue gained more prominence affecting multiple age groups. Young people, such as college students are no exception. They are faced many situations that affect their well-being causing them anxiety and putting at risk their academic performance. Those circumstances lead students to disengagement attitudes toward school and consequently poor evaluation performances. Having those issues in mind, several institutions are trying to adapt their way of teaching through the transition from traditional ways, where student’s knowledge is only supplied by lecturers, to student-centered approaches, namely, active learning where students can participate in their own learning process being that process a collaboration between the teacher and the learner. So far, active learning has proved that in some situations can be useful in anxiety reduction. Therefore, aiming to understand and explore in a broader way what is the impact of active learning practices, namely, team-based learning on NOVA IMS university students’ anxiety, from several study areas, this study focuses its research on the implementation of that approach during one semester. Intending to accomplish those objectives, qualitative and quantitative analyses were carried out through the use of surveys and individual interviews, allowing the further implementation of exploratory, statistical, and cluster analysis supported by Jupyter Notebook, SPSS Statistics, and Excel. Those analyses allowed not only the discovery of the student's opinions on the practice applied as well as the respective relationship between active learning and anxiety levels felt in those situations and the specific semester period being complemented by students’ demographic and non-demographic characteristics.

KEYWORDS

Team-Based Learning; Students Anxiety; Exploratory Analysis; Statistics; Clusters

iv

INDEX

1. Introduction ... 1

2. Literature Review ... 4

2.1. Active Learning ... 4

2.2. Team-Based Learning ... 5

2.3. Active Learning and Anxiety ... 6

2.4. Team-Based Learning and Anxiety ... 7

3. Methodology ... 10

3.1. Context Understanding ... 11

3.2. Survey A

Characteristics ... 12

3.2.1. Survey A – Data Collection ... 12

3.2.2. Survey A – Exploratory Analysis ... 12

3.3. Semi-Structured Individual Interviews – Evaluation of Students Anxiety, TBL Practice Opinions, and Personal Characteristics ... 12

3.3.1. Semi-Structured Individual Interviews – Data Collection ... 12

3.3.2. Semi-Structured Individual Interviews – Exploratory Analysis ... 13

3.4. Survey B – Evaluation of Students Post TBL Classes Anxiety, TBL and Anxiety Relation, and Personal Characteristics ... 17

3.4.1. Survey B – Data Collection ... 17

3.4.2. Survey B – Exploratory Analysis ... 17

3.5. Surveys A and B Matching Students – Evaluation of Pre and Post TBL Classes-Anxiety, TBL Opinions and Personal Characteristics ... 17

3.5.1. Surveys A and B Matching Students – Exploratory Analysis ... 17

3.5.2. Surveys A and B Matching Students – Quantitative Analysis ... 18

3.5.3. Surveys A and B Matching Students – Results Evaluation ... 23

4. Results... 25

4.1. Survey A – Exploratory Analysis ... 25

4.2. Survey B – Exploratory Analysis ... 29

4.3. Surveys A and B – Exploratory Analysis ... 37

4.4. Surveys A and B – Quantitative Analysis Results Evaluation: Shapiro-Wilk Test 49 4.5. Surveys A and B

Test 50 4.6. Surveys A and B – Quantitative Analysis Results Evaluation: Cluster Analysis ... 51

5. Discussion ... 54

v

5.1. Differences between Pre and Post Team-Based Learning Classes-Anxiety Scores54

5.2. Team-Based Learning Impact on Student's Anxiety ... 54

5.3. Demographic and Non-Demographic Variables Relationship with Anxiety ... 56

5.4. Future Adaptations to Reduce Team-Based Learning Anxiety ... 57

6. Conclusions ... 58

7. Limitations and Recommendations for Future Works ... 60

Bibliographical References ... 61

Appendix... 76

Appendix A – Survey A Questions ... 76

Appendix B – Semi-Structured Interviews Questions ... 76

Appendix C – Survey B Questions ... 79

Appendix D – Survey B Additional Graphs ... 81

Appendix E – Surveys A and B Additional Graphs ... 93

Appendix F – Correlation and Regressions Results ... 114

Annexes ... 117

vi

LIST OF FIGURES

Figure 1-Methodology Applied ... 10

Figure 2-Box-plot for Outliers Identification ... 18

Figure 3-Survey A Grades Box-Plots ... 28

Figure 4-Survey B Grades Box-Plots ... 33

Figure 5-Survey B TBL Lecture and Anxiety Opinions ... 34

Figure 6-Survey B TBL and Traditional Classes and Anxiety Opinions ... 34

Figure 7-Survey B Preparation and Anxiety Opinions ... 35

Figure 8-Survey B iRAT and Anxiety Opinions ... 35

Figure 9-Survey B tRAT and Anxiety Opinions ... 36

Figure 10-Survey B Application Exercise and Anxiety Opinions ... 36

Figure 11-Survey B Handout and Anxiety Opinions ... 36

Figure 12-Surveys A and B Number of Students per Anxiety Type ... 39

Figure 13-Surveys A and B TBL Lecture and Anxiety Opinions ... 42

Figure 14-Surveys A and B TBL and Traditional Classes and Anxiety Opinions ... 43

Figure 15-Surveys A and B Preparation and Anxiety Opinions ... 43

Figure 16-Surveys A and B iRAT and Anxiety Opinions ... 44

Figure 17-Surveys A and B tRAT and Anxiety Opinions ... 44

Figure 18-Surveys A and B Application Exercise and Anxiety Opinions ... 44

Figure 19-Surveys A and B Handout and Anxiety Opinions ... 45

Figure 20-Normal Q-Q Plot of Pre Classes Anxiety (Survey A) ... 49

Figure 21-Normal Q-Q Plot of Post Classes Anxiety (Survey B) ... 50

Figure 22-Inertia plot... 51

Figure 23-K-Means Dendrogram ... 52

Figure 24-Survey B Fear of the Unknown and Anxiety Opinions ... 81

Figure 25-Survey B Conciliate TBL with Other Classes and Anxiety Opinions ... 82

Figure 26-Survey B Low-weighted Evaluations and Anxiety Opinions... 82

Figure 27-Survey B TBL and Other Examinations Opinions ... 83

Figure 28-Survey B Constant Evaluations and Anxiety Opinions ... 83

Figure 29-Survey B Doubts Clarification and Anxiety Opinions ... 84

Figure 30-Survey B Teachers Explanations and Anxiety Opinions ... 84

Figure 31-Survey B Amount of Information to Study and Anxiety Opinions ... 85

Figure 32-Survey B Lack of Teachers Explanations During the Preparation and Anxiety

Opinions ... 85

vii

Figure 33-Survey B Lack of Contents Understanding During the Preparation and Anxiety

Opinions ... 86

Figure 34-Survey B Contents Awareness and Anxiety Opinions ... 86

Figure 35-Survey B Not Knowing the Answer in iRAT and Anxiety Opinions... 87

Figure 36-Survey B Team Contents Awareness and Anxiety Opinions ... 87

Figure 37-Survey B Team Incorrect Answers and Anxiety Opinions ... 88

Figure 38-Survey B Incorrect Answers in iRAT and Anxiety Opinions ... 88

Figure 39-Survey B Team Tests/Exercises and Anxiety Opinions ... 89

Figure 40-Survey B Peer Learning and Anxiety Opinions ... 89

Figure 41-Survey B Team Discussion and Anxiety Opinions ... 90

Figure 42-Survey B Team Consensus and Anxiety Opinions ... 90

Figure 43-Survey B Answer Impact on Team and Anxiety Opinions ... 91

Figure 44-Survey B Team Impact on Grade and Anxiety Opinions ... 91

Figure 45-Survey B Application Exercise/Handout Time and Anxiety Opinions ... 92

Figure 46-Survey B Not Understanding an Application Exercise and Anxiety Opinions ... 92

Figure 47-Surveys A and B Conciliate TBL with Other Classes and Anxiety Opinions... 99

Figure 48-Surveys A and B Fear of the Unknown and Anxiety Opinions ... 99

Figure 49-Surveys A and B Low-weighted Evaluations and Anxiety Opinions ... 100

Figure 50-Surveys A and B Constant Evaluations and Anxiety Opinions ... 100

Figure 51-Surveys A and B TBL and Other Examinations Opinions ... 101

Figure 52-Surveys A and B Doubts Clarification and Anxiety Opinions ... 101

Figure 53-Surveys A and B Teachers Explanations and Anxiety Opinions ... 102

Figure 54-Surveys A and B Amount of Information to Study and Anxiety Opinions ... 102

Figure 55-Surveys A and B Lack of Teachers Explanations During the Preparation and Anxiety Opinions ... 103

Figure 56-Surveys A and B Alone Contents Preparation and Anxiety Opinions ... 103

Figure 57-Surveys A and B Lack of Contents Understanding During the Preparation and Anxiety Opinions ... 104

Figure 58-Surveys A and B Contents Awareness and Anxiety Opinions ... 104

Figure 59-Surveys A and B Doubts Clarification Before Classes and Anxiety Opinions ... 105

Figure 60-Surveys A and B Not Knowing the Answer in iRAT and Anxiety Opinions ... 105

Figure 61-Surveys A and B Incorrect Answers in iRAT and Anxiety Opinions ... 106

Figure 62-Surveys A and B iRAT Responsibility and Anxiety Opinions ... 106

Figure 63-Surveys A and B Team Tests/Exercises and Anxiety Opinions ... 107

Figure 64-Surveys A and B Peer Learning and Anxiety Opinions ... 107

Figure 65-Surveys A and B Team Contents Awareness and Anxiety Opinions ... 108

viii

Figure 66-Surveys A and B Team Incorrect Answers and Anxiety Opinions ... 108

Figure 67-Surveys A and B Team Discussion and Anxiety Opinions ... 109

Figure 68-Surveys A and B Team Consensus and Anxiety Opinions ... 109

Figure 69-Surveys A and B Answer Impact on Team and Anxiety Opinions ... 110

Figure 70-Surveys A and B Team Impact on Grade and Anxiety Opinions ... 110

Figure 71-Surveys A and B Algorithm Team Choice and Anxiety Opinions ... 111

Figure 72-Surveys A and B Application Exercise/Handout Time and Anxiety Opinions ... 111

Figure 73-Surveys A and B Not Understanding an Application Exercise and Anxiety Opinions ... 112

Figure 74- Surveys A and B Handout Difficulty and Anxiety Opinions ... 112

Figure 75- Surveys A and B Python Exercises and Anxiety Opinions ... 113

Figure 76-Surveys A and B Faster Exercises Resolution and Anxiety Opinions ... 113

Figure 77-Surveys A and B Others Seeing the iRAT Answers and Anxiety Opinions ... 114

ix

LIST OF TABLES

Table 1-Method by Research Objective ... 3

Table 2- Studies relating Active Learning and Anxiety ... 8

Table 3-Evaluation Elements Frequency and Contribution ... 11

Table 4-Structural Coding Categories and Anxiety Examples identified by the interviewees . 14 Table 5-Cluster Analysis Variables ... 22

Table 6-Survey A Number of Students per Answer ... 25

Table 7-Survey A Statistics ... 26

Table 8-Survey A GAD-7 Results ... 27

Table 9-Survey A Statistics per Anxiety Type ... 27

Table 10-Survey A Statistics and Daily Life Difficulties Type ... 27

Table 11-Survey A Grades Average per Anxiety Type ... 29

Table 12-Survey B Number of Students per Answer ... 29

Table 13-Survey B Statistics ... 31

Table 14-Survey B GAD-7 Results ... 32

Table 15-Survey B Statistics per Anxiety Type ... 32

Table 16-Survey B Statistics and Daily Life Difficulties Type ... 32

Table 17-Survey B Grades Average per Anxiety Type ... 33

Table 18-Surveys A and B Number of Students per Answer ... 38

Table 19-Surveys A and B Statistics ... 40

Table 20-Survey A Anxiety Type Statistics ... 40

Table 21-Survey B Anxiety Type Statistics ... 40

Table 22-Survey A Daily Life Difficulties Statistics... 41

Table 23-Survey B Daily Life Difficulties Type Statistics ... 41

Table 24-Survey A Anxiety Type Evolution... 46

Table 25-Survey A Daily Life Difficulties Type Evolution ... 47

Table 26-Survey A Anxiety Type Evolution and Final Grade Average ... 47

Table 27-TBL Evaluation Elements Surveys A and B Grades per Anxiety Type ... 48

Table 28-Non-TBL Evaluation Elements Surveys A and B Grades per Anxiety Type ... 49

Table 29-Shapiro-Wilk Test Results ... 49

Table 30-Wilcoxon Signed-Rank Test Descriptive Statistics... 50

Table 31-Wilcoxon Signed-Rank Test Results ... 50

Table 32-Wilcoxon Signed-Rank Test Ranks ... 51

Table 33-K-Means Average Silhouette Score by Cluster Number ... 52

Table 34-K-Means Clusters Robust Scaled Means by Variable ... 53

x

Table 35-Survey A Anxiety Type and Gender ... 93

Table 36- Survey A Anxiety Type and Professional Status ... 93

Table 37- Survey A Anxiety Type and Nationality ... 94

Table 38- Survey A Anxiety Type and Master Program ... 94

Table 39- Survey A Anxiety Type and Classes Format ... 95

Table 40- Survey A Anxiety Type and Daytime and Nighttime Classes ... 95

Table 41-Survey B Anxiety Type and Gender ... 96

Table 42- Survey B Anxiety Type and Professional Status ... 96

Table 43- Survey B Anxiety Type and Nationality ... 97

Table 44- Survey B Anxiety Type and Master Program ... 97

Table 45- Survey B Anxiety Type and Classes Format ... 98

Table 46- Survey B Anxiety Type and Daytime and Nighttime Classes ... 98

Table 47-Correlation and Regressions Most Relevant Values ... 114

xi

LIST OF EQUATIONS

Equation 1-Z-Score ... 18

Equation 2-Min-Max Scaler ... 19

Equation 3-Standard Scaler ... 19

Equation 4-Robust Scaler ... 19

Equation 5-Shapiro-Wilk Test Statistic ... 20

Equation 6-Shapiro-Wilk Test Constants... 20

Equation 7-Wilcoxon Signed-Rank Test Statistic... 21

Equation 8-Wilcoxon Signed-Rank Test Hypothesis ... 21

Equation 9-Silhouette Score ... 23

xii

LIST OF ABBREVIATIONS AND ACRONYMS

GAD Generalized Anxiety Disorders

iRAT Individual Readiness Assurance Process Test Min. Minimum

Max. Maximum

PBL Problem-Based Learning Std. Dev. Standard Deviation TBL Team-Based Learning

tRAT Team Readiness Assurance Process Test WHO World Health Organization

1

1. INTRODUCTION

Mental health is considered an essential component of health by the World Health Organization (WHO, 2018). In 2016, around 84 million people in European Union countries had a mental health problem, being anxiety the most common mental disorder, with 25 million people affected (OECD, 2018).

Sigmund Freud was the first responsible for explaining anxiety meaning (Esteves, 2011; Santos &

Silva, 1997). He defined it as something unpleasant involving physiological and behavioral components (Esteves, 2011; Santos & Silva, 1997). Since 1960, emotions have gained increasing prominence, and nowadays, they are seen as specific feelings and physical changes linked to the autonomous nervous system (Santos & Silva, 1997). Afterward, Spielberger et al. (1983) characterized anxiety as a psychological and physical response provoked by a feeling of tension, apprehension, nervousness, and worry (Esteves, 2011).

Within the anxiety topic, another critical issue affecting around 264 million people worldwide are Generalized Anxiety Disorders. Those are characterized by extreme worry and anxiety toward daily activities, including academic performance (Association, 2013; Organization, 2017). College students are no exception. These students are exposed to numerous problematic situations and contexts leading them to poor academic performances, namely, lack of interest, poor assignments and exam performances, and difficulties in concentration, affecting their reasoning (Aronen et al., 2005; Cooper et al., 2018; Vitasari et al., 2010). This scenario was aggravated in January 2020, when WHO declared Covid-19 as a pandemic. According to Bickel (2021) and Cao et al. (2020) during this period, students reported the highest levels of depression and anxiety compared with other years contributing to the aggravation of their overall mental health. The identified causes were the disruption in the academic and social panoramas, such as concerns with technology, social isolation, and lower household incomes and employment (Wu et al., 2020).

The Portuguese perspective goes along with the previous ones. Correia (2003) and Marques (2011) mentioned that Portugal's higher education failure rate is exceptionally high. Considering this context, only 31.4% of the students complete their courses in the expected time, while 25.8% need three or more additional years than expected (P. C. Oliveira & Oliveira, 2014). To aggravate the situation, 30.5% of psychiatric disorders occurring in the country corresponded to people between eighteen and thirty-four years old, where the majority of college students ages fall (Sousa et al., 2018).

Higher education and active learning approaches have been strengthened by several researchers that place their attention on the active development and practical knowledge scenarios by agreeing that students should adopt a more active role in their learning progress (P. Oliveira et al., 2006; P. C.

Oliveira & Oliveira, 2014). Until now, most of the studies developed in this area that related active learning practices with anxiety focused on implementing others apart from TBL (Team-Based Learning). Cooper et al. (2018) and England et al. (2017), for instance, carried out two different analyses one based on semi-structured interviews and statistical methods where clicker questions, group works, cold or volunteering call, and group or individual worksheets were implemented. Both of those analyses allowed the identification of the different perceptions and opinions regarding the implementation of those practices and anxiety as well as the discovery that all of those have the

2 potential to increase or decrease anxiety depending on how they are implemented. The two authors mentioned that clicker questions had proved their efficiency in anxiety reduction having England et al. also referred that it was the practice with the lowest anxiety average values when compared with the others. Cooper et al. (2018) added that group work can in some situations cause fear of negative evaluation or fear of being negatively affected by the peers’ choices, however, in this research the practice was also mentioned as anxiety decreasing because students had the opportunity to discuss and share knowledge with the other members. On the other hand, cold/random call questions made by the teachers during classes were indicated as the most anxiety-provoking practice as students were afraid to show their knowledge weaknesses in front of the class.

When confronting TBL and anxiety although the studies did not focus on anxiety as a primary issue, they oriented their attention to the students or faculty members' opinions about the practice implementation and in some, it was possible to retrieve some anxiety-related reflections. Rezaee et al. (2016), Al Kawas & Hamdy (2017) and Feingold et al. (2008) analyzed those students' perspectives and discovered that the discussions among group members could positively impact anxiety as well as promote an environment that allowed the discussion of the questions while having the peers support. When relating this practice with grades Al Kawas & Hamdy (2017) indicated that TBL could take away this pressure, however, Feingold et al. (2008) disagree with that opinion as he mentions that especially when students work with a group its influence on the grade can negatively impact anxiety. Tweddell et al. (2016) explored the faculty member's point of view in which those reported anxiety associated with the practice logistics, fear of the unknown, and fear of not having the necessary skills.

As described before, the studies conducted so far in this field focused mainly on exploratory interview-based or statistical analysis with specific degrees applying just a few active learning practices, for instance, clicker questions, cold/random calls, and group work (Cooper et al., 2018;

England et al., 2017). Despite that, they were not interested in exploring the differences in how active learning affected students’ anxiety levels (Cooper et al., 2018) or did not integrate the grades into the analysis (England et al., 2017) which can be influenced by those levels. Concerning the data collection, others relied only on a single time-point in the semester (Brigati et al., 2020; Miller et al., 2015).

Nevertheless, it is essential to explore, considering a different university context and degrees, what is the impact of active learning on students’ anxiety levels and understand its extension. Aiming to achieve that, this study will focus on the combination of team-based learning activities performed in the Descriptive Methods of Data Mining master subject classes with students’ anxiety levels felt during those to identify possible relations between them as well as the association between anxiety and student’s demographic and non-demographic characteristics. This research does not only comprehend the discovery of a relation between the TBL practice and students’ anxiety, which has not been prove yet, through exploratory and statistical analysis, but it also adds to the previous literature on the implementation of a cluster analysis. Those three different methods allowed the performance of an empirical analysis that intends to answer the main research question of “What is the impact of team-based learning on a Higher Education Institution students’ anxiety levels?”. To answer this question, the following research objectives are defined:

3 Table 1-Method by Research Objective

The results obtained through the implementation of those methods allowed the discovery of different anxiety levels felt at the beginning and at the final of the semester being the second ones those where the highest values were registered. When relating academic characteristics and anxiety it was verified with the highest anxiety levels were associated with the students with the highest grades on average. It was also observed that some TBL activities performed during the classes caused anxiety, especially the individual tests, application exercises, and handouts. Despite that, the overall students preferred this practice instead of a traditional lecture-based class.

This study complies with the scientific research structure incorporating in chapter 1, the introduction, in chapter 2 the respective literature review, in chapter 3 the methods employed, in chapter 4 the results, in chapters 5 and 6 discussion and main conclusions, and finally in chapter 7 the limitations and future works recommendations.

Research Objective Method

1: Identify the student's anxiety levels differences at the beginning and the final of the semester, namely before and after the implementation of team-based learning

GAD-7 Scale (Surveys A and B Matching Students) / Wilcoxon Signed-Rank Test 2: Verify the impact of team-based learning on

students' anxiety levels

Individual Interviews Exploratory Data Analysis /

Survey B 3: Perform a cluster analysis to segment the students

and identify similarities between them according to anxiety levels, team-based learning activities, demographic (i.e.: gender, age), and non- demographic variables (i.e.: master program, grades).

Exploratory Data Analysis from Surveys A and B /

Cluster Analysis

4

2. LITERATURE REVIEW

2.1. A

L

At the beginning of the 1990s, Bonwell and Eison (1991) compiled several techniques and activities that encouraged students to think about what they were doing and not only about doing those. Active learning theoretical foundations contemplate the constructivist theory, where learners are responsible for their knowledge-building and understanding based on different ideas and experiences they associate with existing ones to extend their expertise (Bransford et al., 2000). These approaches try to broaden the students’ knowledge by providing them with the support of their instructor or peers while solving different problems they are used to (Vygotsky & Cole, 1978). Their primary focus is based on stimulating students’ skills through discussion, reading, writing, or other learning activities while emphasizing their attitudes and values by using, for instance, problem- solving, cooperative group work, breaks during class, small tests, and exams, audiovisual supported classes, computer aid learning, games, and discussions among students and teachers, allowing them to, not only gain new knowledge but also to confront misconceptions (P. C. Oliveira & Oliveira, 2014).

During the last few years, several authors have been developing and applying active learning methodologies to students with different backgrounds, giving specific guidelines towards those approaches' implementation (Downing et al., 2020).

Several pedagogical approaches have been explored in this scope, such as Problem-Based Learning, a technique applied to individuals or groups allowing them to learn while engaging with significant problems (Yew & Goh, 2016). This approach empowers the students with indispensable tools to seek the necessary knowledge as self-directed learners instead of the knowledge supplied by an instructor (Harun et al., 2012; Mayer, 2013; Schmidt et al., 2011). PBL is aligned with monitoring and formative assessment from the instructor, who desires to inform students on how they are progressing, allowing them to take action to improve their performance (Biggs & Tang, 2007). On the other hand, the effectiveness of the feedback provided is dependent on the student’s awareness of what they have learned, what they must achieve, and the expectations from instructors, becoming this process a collaboration between the teacher and the students (Biggs & Tang, 2007; Loacker &

And Others, 1985; Nicol & Macfarlane‐Dick, 2006; P. C. Oliveira & Oliveira, 2014). That technique led to improvements in the satisfaction and academic performance of the individuals, bringing some advantages to those, namely, the enhancement of the ability to carry out a critical analysis and synthesis of the information read (Armbruster et al., 2009; Harun et al., 2012; Lake, 2001).

In addition, another active learning practice explored by Cooper et al. (2018) was clicker questions in which students used devices to answer in real-time and anonymously the questions posed by the instructor during the classes. Besides clicker questions, another approach mentioned was group work, which is commonly introduced as an active learning technique. Group work proved to increase student achievement (Cooper et al., 2018; D. W. Johnson & Johnson, 2009; Springer et al., 1999, p. 11; Steele-Johnson & Kalinoski, 2014; Tanner et al., 2003), enabling those to share concepts, discuss ideas, and hear different opinions (Cooper et al., 2017, 2018; Lamm et al., 2012) while engaging with others and preparing them to their professional careers (P. C. Oliveira & Oliveira, 2014; Poole, 2003). Furthermore, it is easily integrated into an active learning class as it can be

5 associated with clicker questions, written assessments, open questions posed to the students, or any previous approach mentioned (Brigati et al., 2020; Cooper et al., 2018).

Apart from those, a learner-centered environment also suggests that students should be integrated into their evaluation. This can be done by conceding them the ability to openly discuss, review and negotiate the course syllabus (Downing et al., 2020).

Some perspectives focus more on individuals by identifying different types of learners, such as auditory, visual, or kinesthetic, for those who prefer a nonverbal approach, for instance, studying an image or a video. The students conduct a conceptual understanding and benefit from retaining knowledge in a particular area (Clark & Paivio, 1991; Downing et al., 2020).

In the Portuguese context, P. C. Oliveira & Oliveira (2014) presented an integrator element to promote active learning in an engineering course that allowed the use of several active learning strategies, namely group work and collaborative learning, problem-solving and formative assessment. The experiment was characterized by a real-world problem presented weekly as a task.

Each depended on the previous one and was executed in small groups of students with the teacher's support.

2.2. T

-B

L

Following the previous studies on the active learning environment, various researchers center their attention on Team-Based Learning, an approach developed by Larry Michaelsen in the ’80s (Michaelsen, 1983; Miller et al., 2015). TBL is similar to PBL, as students have the opportunity to work in groups. However, TBL requires prior-to-class preparation and a brief discussion between the instructor and the students. This approach is composed of pre-class individual learning where the student gets familiarized with the learning concepts through videos, articles, or other documents.

The second phase includes a readiness assurance part where they must complete a short individual test, or iRAT (individual readiness assessment test), and then repeat the same test within a team, tRAT (team readiness assessment test). In the latter one, the team can write an appeal on those questions they had incorrect to expose their point of view deeper. After those tests, some additional practical exercises are presented as a way to apply the previous knowledge acquired. In the last step, the instructor intervenes to briefly discuss the subjects and give feedback on the questions that most teams got wrong (Ofstad & Brunner, 2013; Perumal, 2020).

From the TBL approach point of view, several studies (Lochner et al., 2020; Rezaee et al., 2016;

Tweddell et al., 2016) have applied it to different scopes relating it to academic performance, learners’ satisfaction and perceptions, assessment of personal and team skills, or even comparing it with the traditional classes approach. Rezaee et al. (2016) concluded that, with TBL, students improved their knowledge retention. This retention positively associated with the discussion between peers also led to improvements concerning personal and team skills, namely the ability to ask, explain and discuss ideas with others, problem-solving, and communication. The participants also mentioned that they were more satisfied with TBL than traditional lectures since this approach encouraged independent study and provided better content coverage. So, students could participate more during classes leading to an overall motivation increment (Rezaee et al., 2016). Another study conducted by Lochner et al. (2020) enhanced the positive outcomes of group work where students reported that the learning was fostered by the group where peers benefit from each other

6 knowledge, especially to clarify doubts. It also mentioned the benefit of communication and teamwork to promote more didactic classes and active engagement with colleagues. Additionally, Tweddell et al. (2016) referred the different TBL benefits including the development of essential skills for future workplaces such as teamwork, the ability to encourage, listen, give support to others, and manage conflicts.

2.3. A

L

A

Cooper et al. (2018) emphasise two approaches when relating active learning with low student anxiety levels: clicker questions and group works. Clicker questions proved not only their efficiency in anxiety reduction but also in student’s perceptions of their knowledge and difficulties, contributing to the concept’s clarification and deepening their understanding of particular topics (Cooper et al., 2018; Knight et al., 2013; M. K. Smith et al., 2009; M. k. Smith et al., 2011). Despite that, showing the results of the questions to the class may be a factor of anxiety increasing for those who answer them incorrectly. Besides clickers, group work can also be a good option for implementing active learning.

Cooper et al. (2018) suggest that the students should be able to choose their group as a way to promote anxiety reduction. In some cases, this approach can also generate some fear regarding negative peer evaluation.

Another research conducted in three biology classrooms by England et al. (2017) compared five approaches, including cold call, when a teacher asks a question directly to a student, volunteering to answer questions, worksheets, group works, and clickers by performing quantitative and qualitative analysis. From those, cold call was the one that presented the highest anxiety ratings, followed by volunteering answering. On the other hand, as the previous authors stated, clicker questions and group work were the ones where students felt less anxious. During student interviews, some mentioned that when the instructor asks them a question directly, it provokes more anxiety as they are afraid not to know the answers and demonstrate it in front of a huge lecture. Worksheets were also an issue regarding anxiety. Firstly, if students were completing a worksheet with a group, they might feel anxious because they did not know each other and if they should trust the other's opinions. However, when completing it individually, if they were not sure of the answer, they could not ask someone else. Group work and worksheets caused students anxiety because some had difficulties finding colleagues to work with. In contrast, some were afraid that they could misguide others or be misguided by those who were not prepared or did not care about the assignment and lost points in their evaluations. Although most students mentioned clicker questions as the less anxiety-provoking approach, some felt that it was unfair to be tested on not previously discussed concepts, becoming afraid of the impact on their grades.

In the PBL approach perspective, Chen et al. (2008) compared Taiwanese with Asian and American students’ anxiety in that specific pedagogy. In this case, students reported anxiety when giving a report or when being the focus of attention in the PBL group, so activities involving public speaking and social interaction. It was perceived that mild to moderate anxiety occurred, especially when speaking in English during group tutorials.

7

2.4. T

-B

L

A

Even though we did not find studies focusing only on the analysis of Team-Based Learning implementation and students’ anxiety, several authors have assessed students’ perceptions or evaluated faculty members' points of view. Ofstad & Brunner (2013) and Tweddell et al. (2016) referred that the latter expressed, in the initial phase, anxiety, lack of understanding of the logistics, and fear of the unknown, namely fear of not having the necessary skills to facilitate the learning process. Besides that, it was also reported a considerable workload compared to the traditional classes, as they had to develop practical exercises and take control during the classes. Despite those initial challenges, some benefits were also identified for students, such as 1) student engagement and interactivity; 2) maximization of students’ attendance and participation; 3) encouragement of their learning responsibilities; 4) enhancement of critical thinking, and 5) improvement of communication skills.

Considering the TBL impact on students’ satisfaction, Rezaee et al. (2016) discovered that overall, students felt more encouraged to study regularly. At the same time, they could learn from others and teach them. Those discussions will positively impact anxiety reduction due to the increase of their learning process awareness. Also, Al Kawas & Hamdy (2017) performed a study based on second-year dental students’ opinions. Those indicated that TBL was stress-free as it took away the pressure associated with time, marks, and the pressure of being judged for their mistakes or asking questions. It promoted a good environment where there were more opportunities for them to express, question, and make errors feel more supported by the other members, reduce their overall anxiety, and improve their teaching, communication, and social skills.

In contrast with the previous studies, Feingold et al. (2008) assessed 48 nursing students’

perceptions of Team-Based Learning, and those stated that although TBL was effective in terms of learning, it was also stressful mainly due to the grade impact associated with it. For instance, students mentioned that the fact that the discussions impact their grades makes it stressful because other members can influence it. Deardorff et al. (2014) also found that if group application exercises on TBL were not graded, students would feel less anxious and stressed.

Table 1 presents the studies that provide some type of relation between active learning and anxiety even if in the majority the identification of that relation is not the main scope as they mostly explored the student’s perceptions or satisfaction with the practices. Through the analysis of the table, it is possible to conclude that most of the studies performed qualitative and quantitative methods based on individual interviews or statistical methods by combining different students’

characteristics and anxiety scales. From those, some instruments and methodologies were also adapted to this research, namely, the GAD-7 (7-item Generalized Anxiety Disorder) scale and the semi-structured interviews approach applied by Cooper et al. (2018), the mixed-methods implementation from England et al. (2017) and the statistical approaches to compare two different points in time or groups as also implemented by Tweddell et al. (2016). The ones in which the main scope was the relation between anxiety, identified by an asterisk, were not applying TBL and relied mostly on the same active learning practices, namely clicker questions, group works, and cold call. In general, the studies gathered the personal student’s characteristics such as age or gender and in the academic field, for instance, their course, prior experience with the practice or skills obtained rather than the grades obtained in each of the activities.

8 Table 2- Studies relating Active Learning and Anxiety

Author,

Year Pedagogical Approaches Instruments Methods Data Variables

Brigati et al.,2020

Clicker Questions, Group Works, Cold/Random Call

Online Survey, Own Anxiety Scale

Quantitative (Frequencies, Means,

Cohen’s Kappa, Bar

charts)

880 Biology College Students

Year in School, Gender, Ethnicity, Anxiety Distribution, Coping

Categories Cooper et

al.,2018*

Clicker Questions, Group Works, Cold/Random Call

GAD-7 Scale, Semi- structured Interviews

Qualitative 52 Biology College Students

GAD-7 score, Class, Gender, Ethnicity, First-Generation College

Going, Anxiety Effect England et

al.,2017*

Clicker Questions, Worksheets, Volunteering

Answer, Cold Call, Group Works

Online Survey (Adapted Research

Anxiety Scale), Individual Interviews

Mixed-Methods (interviews, one-way ANOVA, Tukey’s post- hoc analysis, two-tailed

T-tests, Cohen’s F)

327 Biology College Students

Course, Year, Gender, Race, Anxiety Levels

Al Kawas &

Hamdy, 2017

Peer-Assisted Learning/Team-Based

Learning

Questionnaire/Focus Group Interviews

Quantitative (Frequencies)

38 Dental Students (Questionnaire), 16 Dental Students

(Focus Group)

The effect of PAL/TBL on learning, teaching, communication, and social

skills Rezaee et

al., 2016 Team-Based Learning Questionnaire Quasi-experimental (Paired T-tests)

25 Management Students

Pre and Post Test Knowledge Retention, Final Exam, TBL, and

Lecture Satisfaction Tweddell et

al., 2016 Team-Based Learning Semi-Structured

Interviews Qualitative 19 Pharmacy

Faculty Members

Discipline, Number of Years Teaching, Experience using other

methods before TBL, Number of

Years Using TBL, TBL perceptions

Deardorff et Team-Based Learning Survey Quantitative (Chi-square 175 Medical TBL Survey Domains (General,

9

al., 2014 tests) Students Participation and Communication,

Intra-Team Discussion, Perceived Effort, Teamwork Skills Ofstad &

Brunner, 2013

Team-Based Learning Literature Qualitative

Nursing, Medical, Pharmacy,

Literature

-

Chen et al.

2008* Problem-Based Learning

Liebowitz Social Anxiety Scale (LSAS)

modified to fit PBL

Quantitative (Frequencies, Means, Standard-Deviations)

23 Medical Students (Taiwanese, American, Asian)

Age, Gender, Prior Experience of PBL, 8-item LSAS

Feingold et

al., 2008 Team-Based Learning Observation During

Classes, Interviews Qualitative 48 Nursing

Students

Age, Gender, Academic Characteristics

*Studies in which the main objective was to relate active learning and anxiety

10

3. METHODOLOGY

We followed a positivist developmental mixed-methods research that uses both qualitative and quantitative research methods sequentially to develop a deep understanding of the phenomena (Venkatesh et al., 2013). This research is also based on natural science that considers the explanation of reality by developing sets of concepts to characterize the phenomena (March & Smith, 1995). The methodology applied is presented in the figure below.

Figure 1-Methodology Applied

11

3.1. C

U

In this research, the first step mainly consisted into get to know the context and the problem.

This phase called context understanding focused on assessing and defining the research objectives and the study context by analysing the introduction, and literature review and comparing them to the context in which we were inserted. As mentioned previously, especially through the literature review it was possible to identify some context similarities explored by other authors (Cooper et al., 2018; England et al., 2017; Tweddell et al., 2016) and adapt their methods to this study, which were the cases of the anxiety scale, the semi-structured interviews, the mixed-methods, and the statistical approaches.

Having defined those approaches, the second step focused on the definition of our study context, considering the previously defined Team-Based Learning practice. The TBL was conducted with the help of LAMS (B. Dalziel et al., 2019; J. Dalziel, 2006), an online TBL platform, at NOVA IMS university in the 2021/2022 fall semester in a Descriptive Methods of Data Mining course in the first semester of the first year. The course included several students from different master courses and class formats, daytime, and nighttime, as both students were allowed to attend the classes in a daytime format. It consisted of theoretical classes, where the iRAT (individual tests), tRAT (group tests), application exercises, and tutor/class feedback were performed, interspersed with practical classes where more profound application exercises were proposed. The team members were chosen by an algorithm based on students' personalities. During the theoretical classes, the students started by solving the individual test which had the same questions as the team test performed after that. In the team tests, in each class, a group member was defined as the leader, so, the person responsible to select the answers and deliver the group works. Additionally, during the theoretical classes, the students were allowed to refute the questions and clarify their doubts through an instrument called burning questions. In the practical classes, there were also graded exercises, called handouts.

Moreover, several evaluation elements, divided into the continuous or 1^st call evaluation and 2^nd call evaluation, contributed to the final grade including some of the team-based and non-team-based learning items, as seen in table 3.

Table 3-Evaluation Elements Frequency and Contribution

Evaluation

Element

Number of Times Implemented

Contribution to the Final Grade (%)

Quiz Python

1 5

Individual Readiness Assurance

Test (iRAT)

6 2.5

Team Readiness Assurance

Test (tRAT)

6 2.5

Handout 1

1 5

Handout 2

1 5

Final Project*

1 35

Exam*

1 20

*If the student chooses the 2^nd call period, the exam has a contribution of 65% and the final project 35%.

12 This research comprised two surveys and several individual interviews applied during one semester that covered students' characteristics and anxiety levels evolution as well as their perceptions regarding the TBL impact on those. Both surveys and interviews were implemented considering the subject of the class. Consequently, the sample was chosen by convenience and from the snowball effect resulting from that choice.

3.2. S

A – E

S

P

TBL C

-A

P

C

3.2.1. Survey A – Data Collection

During the first class, the students were introduced to the research's topic and scope. Of 79 students enrolled in the course, 73 agreed to answer the first survey in Google Forms, survey A, which aimed to perceive their anxiety levels at the beginning of the semester and get to know their personal and academic characteristics. To attain those objectives, this survey combined a 7-item Generalized Anxiety Disorder scale (GAD-7) (Spitzer et al., 2006) and additional questions such as age, gender, master's program, course format, namely if the student is enrolled in the daytime or nighttime master program, and student number (See more on Appendix A). The GAD-7 is a self- reported instrument that uses a four Likert-scale (Cooper et al., 2018; Spitzer et al., 2006) ranging from “Not at all” to “Nearly every day” answer choices. This instrument measures anxiety on a continuum (Cooper et al., 2018; Spitzer et al., 2006) by assigning a score between 0 and 21 corresponding to minimal, mild, moderate, or severe anxiety according to the respondent's answer.

(Consult Annexes) It is one of the most used to detect anxiety disorders due to its reliability and efficiency (Cao et al., 2020; S. U. Johnson et al., 2019), and one of the advantages is the ease of score and the lower number of questions when compared with others (Budikayanti et al., 2019).

3.2.2. Survey A – Exploratory Analysis

Secondly, an exploratory analysis was carried out. This type of analysis aims to explore and examine data, its distribution, outliers, coherence, or simply visualize it through graphical tools to understand better and recognize patterns (MIT Critical Data, 2016). In this survey, several steps were performed in the Jupyter Notebook and Excel, such as the datatypes, valid and missing values identification, and the coherence of each variable. The descriptive statistics were also assessed to retrieve the means, maximum and minimum values, standard deviations, modes, medians, and quartiles of the variables, such as the pre-classes anxiety scores, students' grades, and students' ages.

To identify the absolute and relative frequencies, several variables were combined into tables with the different anxiety levels as well as through bar plots and box plots to have a deeper overview of the data, for instance, how many students got a specific anxiety type or identify the average grade values per anxiety type.

3.3. S

-S

I

I

– E

S

A

, TBL P

O

,

P

C

3.3.1. Semi-Structured Individual Interviews – Data Collection

After the first survey, a qualitative exploratory study was conducted in the middle of the semester to understand better the students’ opinions about team-based learning pedagogy and the

13 possible impacts on their anxiety. For that purpose, in the third TBL class, all students were invited to participate in semi-structured individual interviews which included the GAD-7 scale questions and questions related to the different phases of a TBL class and teachers as well as personal questions (See Appendix B). From these, we obtained 15 valid interviews conducted through an online platform. All the interviews allowed the confirmation or denial of the information gathered from the literature in this context and the development of new constructs that could support the design of the quantitative analysis (Brod et al., 2009; Venkatesh et al., 2013).

3.3.2. Semi-Structured Individual Interviews – Exploratory Analysis

After collecting all the information, the interviews were coded to move from the raw data to the findings through the use of Excel. Coding is a simple operation of identifying segments and categories from the data collected in the interviews, that allows the identification of several similar concepts by assigning a code (Linneberg & Korsgaard, 2019). We employed structural coding to characterize a segment of data retrieved from the semi-structured interviews based on its content. Structural coding is a question-based code that relates a research question that framed the interviews with their content, so each topic is related to a question. (MacQueen et al., 2008, p. 124). Through the exploration of the second research objective defined and the embedded research question of “What are the situations/activities in the TBL class that impact students’ anxiety levels?” there were defined 16 main issues that affected students’ anxiety positively or negatively. After that, for each of those, it was determined the number of participants that mentioned that particular theme to identify which themes, ideas, or domains were more common and which were able to increase or decrease anxiety (See Table 4) (Namey et al., 2008; Saldaña, 2013).

14 Table 4-Structural Coding Categories and Anxiety Examples identified by the interviewees

Issues (Number of

Participants Mentioning): Examples of Anxiety Increase: Examples of Anxiety Decrease:

Pre-Class Study (12)

Getting the materials studied on time; A lot of information to study; Difficulties in absorbing all the contents; Contents Format (Big pdf’s); Amount of study time

Having the possibility to consult the materials at any time

Evaluations (11) Constant evaluations during the semester and classes

Shared test responsibility; Split evaluations;

Future evaluations readiness (Exam, Project)

Grades (7)

iRAT and tRAT grades; The iRAT grade depends only on one person; Losing points on questions that they knew but the group did not agree with

Exams and projects have less weight on the final grade; Better grades on the tRAT

Fear of Unknown (6)

Fear of the unknown on the iRAT and tRAT questions; Not having the notion if the contents are sufficient for the exam;

Afraid of not having studied enough New Method Transition (4)

Sudden method change; Not having slides support; Amount of tasks in a class

Other Curricular Units (4)

Other curricular units evaluations and study; Difficulties in reconciling with PBL classes applied in another curricular unit

Team Dynamics (9)

Reach consensus; Debate and debate conflicts; Difficulties with certain personalities and working methodologies;

Having a non-flexible leader; Having more work as a leader;

Join everyone after the class when having to finish tasks after

Peer learning; Team mutual aid

15

it

Autonomous Learning (5)

Preparing alone; Not having the teacher mentioning the most important things

Influence on/of the Team (6)

Influencing the team answers choice; Teams influence on grades; Seeing others solving the exercises faster; Having the responsibility to get the answers right as the team can see the answers; Not having the same answers as the colleagues in the handout

Class Activities Duration (6) Short time to solve the application exercise, handout, and feedback

Doubts Clarification (3) Not having the opportunity to clarify doubts before the evaluation

Clarify doubts in a faster way and more

comfortably with the team and through burning questions

Answers/Contents Uncertainty (9)

Getting answers wrong (in the iRAT or the tRAT); Not knowing the answers in the iRAT; Do not understand the contents in the preparation; Being the team leader when the team is not 100% sure of the answer; Reading the same materials as the colleagues and do not know the contents

Practical Exercises (5)

Handout/Application Exercises; Handout difficulty; Not being capable to solve the python exercises

Solving practical exercises in the class similar to the project delivery

Teachers (9)

Less interaction with the teacher; Not having the teacher explaining the contents

Teacher’s explanations; Teacher’s flexibility and

understanding

16

Contents Awareness (9)

Knowing what the classes contents will be;

Allowing a better content understanding and in- depth learning; Following the contents every week; Easier exam study; Being more attentive during the classes (theoretical and practical) when compared with the traditional methods;

Team member's contents awareness; All the group members are prepared with the same contents; Increased class participation Teams Choice by Algorithm

(4)

Not knowing if the team members have the same academic objectives; Working with unknown people in the evaluations

Teams choice based on personality; Not having

the preoccupation of finding a team

17

3.4. S

B – E

S

P

TBL C

A

, TBL

A

R

,

P

C

3.4.1. Survey B – Data Collection

This second survey construction was possible due to the defined issues from the semi-structured individual interviews and the existing literature. This survey was disseminated online, through Google Forms, at the semester’s end. It aimed to deepen the opinion on the issues defined and collect the overall student’s anxiety levels at the end of the semester and their views on TBL impact. It incorporated the opinion of 64 students and its questions were divided into three main categories, the GAD-7 scale ones, the personal information questions, e.g.: age, gender, nationality, home city, working or not, master program, classes attendance format (online, presential), student number, and the ones related to the TBL classes. The latter was also divided into five sub-categories containing the TBL classes' general questions, pre-class preparation, iRAT, tRAT, Application Exercise/Handout (See Appendix C). All the GAD-7 followed the four Likert-scale previously mentioned from “Not at All” to

“Nearly Every Day”, and the others followed a five Likert-scale from “Strongly Disagree” to “Strongly Agree”, which allows for neutrality (Livingston et al., 2014; H. A. Mennenga, 2012). Posteriorly, those results were combined with the student’s descriptive methods of data mining grades obtained in all the TBL and non-TBL activities as well as the final grades.

3.4.2. Survey B – Exploratory Analysis

Similarly, to the exploratory analysis performed to survey A, the datatypes, variables coherence, valid values, and missing values were verified as well as the descriptive statistics and variables frequencies. In this survey, the specific questions regarding TBL classes were raised and passed through additional analysis of frequencies, by the use of Excel bar plots and pie charts, allowing the discovery of patterns, such as, if most of the females agree that the team random choice was a factor that contributed to their anxiety increment.

3.5. S

A

B M

S

– E

P

P

TBL C

- A

, TBL O

P

C

3.5.1. Surveys A and B Matching Students – Exploratory Analysis

This survey's analyses included the same analysis performed for both of them individually although in this case only the students that had answered the two surveys were considered. Firstly, several tables and line charts were added to make a comparison between the pre and post-classes anxiety levels, such as the number of students that got a specific level of anxiety at the beginning of the classes and how many got that same level on the final, the average GAD-7 anxiety scores, the grades obtained by each anxiety level as well as the comparison between other variables and the average score obtained, for instance, the comparison of the scores obtained between the non- working and working students.

18

3.5.2. Surveys A and B Matching Students – Quantitative Analysis

Survey A results along with survey B, enabled us to perform three different quantitative analyses: exploratory, statistical, and cluster analysis.

After obtaining a first sight of the data the next steps comprised the preparation of the data set from the data retrieved, including, data cleaning, and anomaly elimination (Abbott, 2014) for the statistical and cluster analysis. First, for the statistical part, all the columns were excluded except the ones that pursued the student number, and the surveys A and B GAD-7 anxiety scores to allow the comparison. Secondly, in the cluster analysis, further steps were taken in this phase, the columns were renamed for some of the variables, and the index set, so the student number was considered as the identifier of each observation, and the outliers were checked for all the variables in both analyses. Outliers are the observations that pursue a considerable deviation from the others (Aguinis et al., 2013) so they can lead to important changes when using statistical methods (Aguinis et al., 2013; Cohen et al., 2002; Hunter & Schmidt, 2004; Kutner, 2005) namely the false acceptance or rejection of hypothesis (Aguinis et al., 2013; Bollen & Jackman, 2013). In clustering, outliers are perceived observations that do not fit the overall clustering pattern (V. Patel & Mehta, 2011) affecting their reliability and the algorithm's performance (Chawla & Gionis, 2013). In this research, outliers were identified graphically through box plots and numerically through z-score (Hemmati- Sarapardeh et al., 2020). Box plots pursue a lower extreme, a first quartile, a median, a third quartile, and an upper extreme where the first quartile represents the 25^th percentile, the third quartile the 75^th percentile, the lower and the upper extremes represent the maximum, and the minimum values, respectively (Aguinis et al., 2013; Kwak & Kim, 2017). In this case, an outlier is an observation that lies beyond the upper or lower extremes (Aguinis et al., 2013).

Figure 2-Box-plot for Outliers Identification Own Source

On the other hand, the z-score is a standardized score that computes the number of standard deviations varying from the mean (Grove et al., 2012; Mowbray et al., 2019; Polit, 2010; Salgado et al., 2016) by applying the following formula:

𝑧_𝑖 = 𝑥𝑖− 𝑥̅

𝑠

Equation 1-Z-Score (Salgado et al., 2016)

Where 𝑥̅ is the sample mean and 𝑠 the standard deviation. In this case, outliers are identified by contemplating a score limit. If |𝑧_𝑖|>=3, that is, the values that are above or below this score are

19 labelled as extremes (Mowbray et al., 2019; Salgado et al., 2016). By analysing the outliers obtained using both methods no observations were deleted because no justification was founded for those to be identified as outliers. For instance, in some observations the variables responsible did not match in both methods, all the values were inside the normal variables range having other observations with similar values in that same variable and not being identified as outliers or simply the variables indicated as outliers were related with personal information, such as students with high values on age and for the research, it did not make sense to delete that observation because there were no value limitations considering that variable.

After the outliers checking, two more steps were carried out, firstly the categorical variables were transformed into dummies, and secondly, all the variables were scaled based on three scalers, Min-Max, Standard, and Robust. Focusing on the dummy variables, those are dichotomous variables that derive from the original ones and where the number of dichotomies equals G-1, being G the original categories number (Hardy, 1993). Focusing on the scaling, this step appears as an essential one as variables tend to have different ranges, magnitudes, or scales and so, a variable with a greater range could have a larger impact on the model's results or overpower other ones (Mohamad &

Usman, 2013; Singh et al., 2015). To avoid that, firstly min-max scaler was applied. This scaler applies a range of [0;1] by default to the data (Dey et al., 2018) by applying the following transformation:

𝑥´ = 𝑥 − min (𝑥) max(𝑥) − min (𝑥) Equation 2-Min-Max Scaler

(D.k. et al., 2019)

Where x is the original value, min(x) and max(x) ate the minimum and maximum values of the feature x. After Min-Max, a Standard scaler was performed, in this case, the dataset was transformed to obtain a distribution mean value of zero and a standard deviation of one by applying a Standard scaler. The transformed value arose by subtracting the mean from the original value and dividing this subtraction by the standard deviation as it is represented in the formula below.

𝑍 =𝑥 − 𝜇 𝜎

Equation 3-Standard Scaler (D.k. et al., 2019)

Where z represents the transformed value of the feature, x is the original value, 𝜇 is the mean, and 𝜎 is the standard deviation. Finally, the Robust scaler was implemented. This scaler uses the quartiles to transform the data as it removes the median and scales the data according to the quartile ranges going from the 25^th quartile to the 75^th quartile.

𝑡 = 𝑥_𝑖− 𝑄₁(𝑥) 𝑄₃(𝑥) − 𝑄₁(𝑥) Equation 4-Robust Scaler

(D.k. et al., 2019)