Usability in a virtual learnig environement: a study at IFRN

Texto

(1)UNIVERSIDADE FEDERAL DO RIO GRANDE DO NORTE CENTRO DE CIÊNCIAS HUMANAS, LETRAS E ARTES DEPARTAMENTO DE ARTES PROGRAMA DE PÓS-GRADUAÇÃO EM DESIGN MESTRADO PROFISSIONAL EM DESIGN. USABILITY IN A VIRTUAL LEARNING ENVIRONMENT: A STUDY AT IFRN. FRANCISCO MONTEIRO DE SALES JÚNIOR. NATAL – RN 2016.

(2) FRANCISCO MONTEIRO DE SALES JÚNIOR. USABILITY IN A VIRTUAL LEARNING ENVIRONMENT: A STUDY AT IFRN. Dissertação apresentada ao programa de PósGraduação em Design da Universidade Federal do Rio Grande do Norte como requisito parcial para a obtenção do título de Mestre em Design. Orientador: Prof. Dr. André Luís Santos de Pinho. Coorientador: Prof. Dr. José Guilherme da Silva Santa Rosa. Área de concentração: Interação HumanoComputador e Ergonomia Informacional. .. NATAL – RN 2016.

(3) Universidade Federal do Rio Grande do Norte - UFRN Sistema de Bibliotecas - SISBI Catalogação de Publicação na Fonte. UFRN - Biblioteca Setorial do Departamento de Artes - DEART. Sales Júnior, Francisco Monteiro de. Usability in a virtual learnig environement: a study at IFRN / Francisco Monteiro de Sales Júnior. - 2016. 166 f.: il. Orientador: Prof. Dr. André Luís Santos de Pinho. Coorientador: Prof. Dr. José Guilherme da Silva Santa Rosa. Dissertação (mestrado) - Universidade Federal do Rio Grande do Norte, Centro de Ciências Humanas, Letras e Artes, Departamento de Arte, Programa de pós-graduação em Design.. 1. Interação homem-máquina - Dissertação. 2. Ensino à distância - Dissertação. 3. Moodle (Programa de computador) Dissertação. 4. E-learning - Dissertação. 5. Educação Dissertação. 6. Design centrado no usuário - Dissertação. 7. Instituto Federal de Educação, Ciência e Tecnologia do Rio Grande do Norte - Educação à distância - Dissertação. I. Pinho, André Luís Santos de. II. Santa Rosa, José Guilherme da Silva. III. Título. RN/UF/BS-DEART. CDU 004.5(043.3).

(4) FRANCISCO MONTEIRO DE SALES JÚNIOR. USABILITY IN A VIRTUAL LEARNING ENVIRONMENT: A STUDY AT IFRN. Dissertação apresentada ao programa de PósGraduação em Design da Universidade Federal do Rio Grande do Norte como requisito para a obtenção do título de Mestre em Design.. Aprovada em: __/__/__. BANCA EXAMINADORA. _____________________________________________________ Prof. Dr. André Luís Santos de Pinho Orientador Universidade Federal do Rio Grande do Norte. _____________________________________________________ Prof. Dr. José Guilherme da Silva Santa Rosa Coorientador Universidade Federal do Rio Grande do Norte. ______________________________________________________ Profa. Dra. Maria Altina da Silva Ramos Membro Externo Universidade do Minho - Portugal.

(5) DEDICATÓRIA Dedico ao meu avô materno, homem que conheceu a essência do Evangelho simples, e assim trilhou caminhos retos, de bondade e de justiça. Ele, que nunca teve a oportunidade de estudo formal, e que ainda assim ensinou tanto a todos da família, certamente teria o coração alegre por ver mais um neto seguindo adiante..

(6) AGRADECIMENTOS Eu tenho em mim um canto de vitória e estou grato pela oportunidade de aprendizagem nesta jornada de formação acadêmica e profissional. Tenho sido agraciado por ter ao meu redor pessoas de bem, que contribuem continuamente para eu ser um educador melhor. Dentre elas está o professor André Pinho, homem educado, atencioso, e de um valor interior raro nos dias atuais. O seu respeito, disponibilidade e humildade (que naturalmente dispensam a necessidade de externalidades e aparências) foram essenciais para o bom andamento do trabalho. Também o professor José Guilherme Santa Rosa, que a partir do conhecimento partilhado e orientações adequadas, trouxe segurança e incentivo. Estou realmente grato por depositarem em mim total confiança, dando-me liberdade e autonomia. Agradeço ainda à professora Altina, que com palavras de motivação e apoio, participou da banca e deixou uma valiosa contribuição. Deixo a todos os três o meu sincero agradecimento pela dedicação nos ensinamentos e correções. Agradeço a todos os que fazem o Programa, como os professores que se mostraram empenhados nas aulas, ao pessoal da secretaria pelo contínuo suporte, e aos colegas da turma, sempre gentis e encorajadores, nomeadamente, Aniolly, Gusmão, João, Charles, Caio, Nivaldo e Moally. Obrigado especial a Marcia, exemplo de garra no curso, pela força ao final da dissertação. A todos a minha gratidão pela excelente companhia, aprendizagem e risadas, mesmo nos momentos de maior adversidade. Agradeço pela oportunidade de trabalhar no IFRN, organização que possui uma missão social tão relevante. Desejo que todos os que fazem esta instituição possam sempre reavaliar os processos educacionais e permitir que a vida seja respeitada. Agradeço aos colegas de trabalho, educadores, servidores públicos dedicados, que assoviam canções de esperança e paixão, fazendo florescer zelo e vida em seus dias, rompendo o duro concreto da burocracia e do conformismo com a força do novo e da vontade de servir. Agradeço pelas portas abertas para a minha formação acadêmica. À minha mãe, que nunca deixou de orar pelos filhos. A Gabriel, Davi, Betânia, Luciana, Jamilly, Jonatas, Jamilson, Duda, Davi Duda, Roberto Júnior e a Elizângela, obrigado pelo carinho e apoio. A Meire, que nunca poupou apoio a todo o momento, minha gratidão. Obrigado a todos vocês por acreditarem que nossa vida.

(7) é uma estrada, que caminho se conhece andando, e que a nossa realização enquanto sujeitos somente acontece na presença do outro. Obrigado por compreenderem as minhas inúmeras ausências. Ademais, tenho (ou tive) ao meu redor muita gente boa de Deus, pessoas importantes e especiais. Obrigado a Idalina, que foi excelente e muito importante na reta final, a Katiana, or seu incentivo tantas vezes, a Avelino, mais amigo a cada dia. Ainda, a Assis, Riba, Natália, Marconi, Adriana, Xistus, Doc, Urka, Anita, Alpha,... e tantos outros. Agradeço a todos a amizade e carinho sempre presentes. Trago em mim, por todos vocês, um brado de gratidão e esperança por dias bons. Deixo a todos o meu testemunho de gratidão, não apenas de palavras, mas de vida e amor, enquanto homem que procura acertar e prosseguir a jornada desta vida com sinceridade, equidade e verdade. Obrigado por fazerem parte da minha formação.. O amor não se alegra com a injustiça, mas se alegra com a verdade..

(8) RESUMO A investigação ocorre no âmbito dos estudos sobre ensino a distância no Brasil, especificamente no modelo e-learning e a partir do uso de ambientes virtuais de aprendizagem (AVA), considerando as diretrizes e avanços desta forma educacional no Brasil. Deseja-se contribuir para o aprofundamento do conhecimento sobre a relação existente entre as áreas da usabilidade e da pedagogia. O estudo apresenta uma abordagem essencialmente qualitativa, centrada tanto na percepção do estudante quanto no contexto de professor reflexivo, em que os docentes investigam as próprias práticas. O objeto de investigação é a identificação e adoção de dimensões e princípios de usabilidade em práticas pedagógicas realizadas no Moodle, AVA utilizado na forma educacional a distância do Instituto Federal de Educação, Ciência e Tecnologia do Rio Grande do Norte (IFRN), onde ocorreu o estudo de caso. O estudo se mostra relevante por se propor a investigar como se encontra a relação entre as práticas pedagógicas e a adoção de usabilidade, garantindo um cenário de reflexão sobre até que ponto os esforços e investimentos no ensino a distância na instituição têm obtido êxito, notadamente, a aprendizagem com qualidade. Os resultados e conclusões apontam elementos norteadores para a correção e ajustes das práticas docentes em busca de melhorias na forma educacional a distância da Instituição. Palavras-chave: Moodle. Ensino a Distância. Educação..

(9) ABSTRACT The research takes place in the context of studies on distance learning in Brazil, specifically in the model e-learning and from the use of virtual learning environments (VLE), considering the guidelines and progress of this educational form in Brazil. This work wants to contribute to the deepening of knowledge about the relationship between the areas of usability and pedagogy. The study presents an essentially qualitative approach, focusing both on the perception of the student and the teacher reflective context, where teachers investigate the practices. The object of research is the identification and adoption of the dimensions and the usability principles on pedagogical practices carried out in the Moodle, VLE used in the educational way considering distance learning of the Federal Institute of Education, Science and Technology of Rio Grande do Norte (IFRN) where the case study occurred. The study shows relevant by proposing to investigate how is the relationship between pedagogical practices and the adoption of usability, ensuring a reflection scenario to what extent the efforts and investments in distance learning at the institution have obtained success, notably the learning quality. The results and conclusions point to guiding elements for the correction and adjustment of teaching practices in pursuit of improvements in the educational way to the distance learning at the institution. Keywords: Moodle. E-learning. Education..

(10) SUMÁRIO 1. INTRODUCTION ..................................................................................................................................... 10 1.1 1.2 1.3. 2. PROBLEM AND INVESTIGATION FRAMEWORK ..................................................................................................... 10 DISSERTATION STRUCTURE AND ARTICLES CORRELATION ....................................................................................... 11 IFRN AND DISTANCE LEARNING ....................................................................................................................... 16. OVERALL AND WEBSITES USABILITY EVALUATION IN LEARNING VIRTUAL ENVIRONMENTS.................. 19 2.1 INTRODUCTION ............................................................................................................................................ 21 2.2 THEORICAL FRAMEWORK................................................................................................................................ 22 2.3 METHODOLOGY ........................................................................................................................................... 23 2.3.1 Description ..................................................................................................................................... 23 2.3.2 Scenarios, delimitation and research subjects ............................................................................... 23 2.3.3 Evaluation method and data collection ......................................................................................... 25 2.3.4 Data processing, analysis and interpretation ................................................................................ 26 2.4 RESULTS ..................................................................................................................................................... 27 2.4.1 System status visibility ................................................................................................................... 27 2.4.1.1 Absence of alerts and feedback ................................................................................................ 27 2.4.2 Consistency and student model ..................................................................................................... 30 2.4.2.1 Inadequate language ................................................................................................................ 30 2.4.2.2 Format inconsistency ................................................................................................................ 31 2.4.2.3 Labeling inconsistency............................................................................................................... 32 2.4.3 Help, use flexibility and efficiency .................................................................................................. 33 2.4.3.1 Message and search limited structure ...................................................................................... 33 2.4.4 Minimalist design, organization, navigation and structure ........................................................... 34 2.4.4.1 Non-minimalist design .............................................................................................................. 34 2.4.4.2 Excess of instructions, explanations and details ....................................................................... 34 2.4.4.3 Excess of forum ......................................................................................................................... 35 2.4.4.4 Navigation inconsistency........................................................................................................... 36 2.5 CONCLUSION ............................................................................................................................................... 38. 3. PEDAGOGICAL USABILITY: A THEORETICAL ESSAY FOR E-LEARNING ...................................................... 45 3.1 INTRODUCTION ....................................................................................................................................... 47 3.2 PEDAGOGICAL USABILITY ........................................................................................................................ 49 3.3 DIMENSIONS OF PEDAGOGICAL USABILITY ............................................................................................. 54 3.3.1 Student Control .............................................................................................................................. 55 3.3.2 Student Activity .............................................................................................................................. 55 3.3.3 Collaborative and cooperative learning ......................................................................................... 56 3.3.4 Guidance to purposes .................................................................................................................... 57 3.3.5 Applicability ................................................................................................................................... 57 3.3.6 Value-added ................................................................................................................................... 58 3.3.7 Motivation ..................................................................................................................................... 58 3.3.8 Value of prior knowledge ............................................................................................................... 59 3.3.9 Flexibility ........................................................................................................................................ 59 3.3.10 Feedback ................................................................................................................................... 59 3.4 FINAL CONSIDERATIONS .......................................................................................................................... 60. 4 TECHNICAL AND PEDAGOGICAL USABILITY IN A VIRTUAL LEARNING ENVIRONMENT: A CASE STUDY AT THE FEDERAL INSTITUTE OF RIO GRANDE DO NORTE – BRAZIL ....................................................................... 64 4.1 INTRODUCTION ............................................................................................................................................ 66 4.2 THEORETICAL FRAMEWORK ............................................................................................................................ 67 4.2.1 Technical usability .......................................................................................................................... 67 4.2.2 Pedagogical usability ..................................................................................................................... 68 4.3 METHODOLOGICAL PROCEDURES ..................................................................................................................... 71 4.4 PRELIMINARY RESULTS ................................................................................................................................... 72 4.5 FINAL CONSIDERATIONS ................................................................................................................................. 73.

(11) 5. CONCLUSION ......................................................................................................................................... 77 5.1 5.2. RECOMMENDATIONS FOR THE INSTITUTION ....................................................................................................... 77 SUGGESTIONS FOR FUTURE RESEARCH............................................................................................................... 79. APPENDIX A - LEARNING VIRTUAL ENVIRONMENT EVALUATION: A COMPARATIVE STUDY BETWEEN MOODLE FORMATS FROM A UNIVERSITY ...................................................................................................................... 81 APPENDIX B - INFORMATION ARCHITECTURE AND USABILITY: ANALYSIS OF THE AUTHOR REGISTRATION AND SUBMISSION PROCESS OF AN ARTICLE IN THE ELETRONIC PERIODICAL BIBLIOCANTO OF UFRN ................... 102 APPENDIX C - EVALUATION OF THE HISTORICAL CENTER OF BRAGA - PORTUGAL FROM THE UNIVERSAL DESIGN PRINCIPLES ...................................................................................................................................... 119 APPENDIX D - TOOL OF SUPPORT FOR BOOK’S COVER DESIGN BASED ON ERGONOMIC PRINCIPLES ........... 146.

(12) 10 1 Introduction Federal University of Rio Grande do Norte (IFRN) graduate’s rules, governed by Article 42 of the Resolution 197/2013-CONSEPE dated 10th December 2013, ensure that the final work can be presented in a specific format, according to the knowledge field and the purpose of the course. Thus, one was adopted a modality whose structure forms an incremental project built by a compendium of seven scientific articles, submitted to events or published in journals; two of them for chapter books, following an order and correlation allowing both logical theme sequence and chronological research. Each article can be read independently. The format of each one follows the orientations and standards required by Congress or Journals, which they were submitted to or published in. In the dissertation, they all follow the same structure and text font. Figures and tables contents were withdrawn in order to avoid identification inconsistency and repetition. 1.1. Problem and Investigation framework This research takes place in the context of distance learning (DL) studies in. Brazil, namely e-learning, and in the use of learning virtual environments (LVE), taking into consideration the guidelines and the progress of this educational modality in Brazil. One hopes to contribute to deepening the knowledge about the connection between usability and pedagogy fields. The study presents an essentially qualitative approach, focused both on students’ perception and in teachers’ reflexive context, in which teachers research their own practices. Therefore, the object of this investigation is to identify and adopt usability dimensions and principles in pedagogical practices that carried out in Moodle (Modular Object-Oriented Dynamic Learning Environment), a LVE used in a DL modality in the Federal Institute of Rio Grande do Norte (IFRN), where this case study took place. The relevance of this study is to investigate how is the relationship between pedagogical practices and usability adoption in IFRN’s LVE. This will enable a reflexion about how successfully have been the efforts and investments made by this institution in DL, specially in terms of learning quality..

(13) 11 Exploratory diagnosis can be of crucial importance to evaluating and changing, if necessary, the guidelines currently adopted, as it identifies negative aspects of pedagogical practices concerning the absence of both overall and websites usability, revealing the degree of students’ dissatisfaction, showing that their wishes and expectations are not always being met. The new guidelines base emerged from this research, based on the relationship between usability and pedagogy, can contribute to minimize the identified problems, such as those connected to LVE’S configurations and practices. The results and the conclusions can be considered as guidelines for future projects in this field, such as those addressed to the configuration of the actual design structures; to the training and support to teachers, as well as to those connected to adjustments in educational materials and virtual learning objects. 1.2. Dissertation structure and articles correlation The first chapter presents the problem and the investigation framework, the. parts of the dissertation, publications that compose it, as well as the scenario in which it took place, its questions and goals. Still, it explains the correlations between themes addressed in each article in relation to the main object. It concludes with recommendations and suggestions for further research. The following three chapters are composed of articles whose topics are directly connected to the object of study. They correspond to Usability Evaluation, Theoretical Proposal and Pilot Study stages. Next, a conclusion is presented with discussions and suggestions for further research, implementation. Appendix to this study, there are four articles that contribute to the object of this investigation by using methodologies and theoretical frameworks linked to the main study. These articles have been produced in cooperation with other authors who are not supervisors. For this reason, they are placed in this part of the dissertation. Figure 1 visualizes the structure, showing the stages, the context, the themes, the methodologies and the implications in the object of study in each production..

(14) 12 Figure 1 - Consolidated visual format compendium. Source: created by the autor (2016).. Figure 2 presents chapter sequence and appendices forming dissertation structure and research development. Figure 2 - Chapters and Appendices sequence. Source: created by the autor (2016)..

(15) 13 Chapter 2, being of both qualitative and exploratory nature, covers the initial stage, consolidated as an usability evaluation in Moodle’s environment, which were set by teachers for free e-learning educational modality. One adopted the empirical method based on field observations and virtual ethnography. Students centered approach was addopted in order to analyse if pedagogical practices were been set taking into consideration both overall and websites usability’s principles. The results were presented based on the identified problems, which were categorized according to the criteria present in the literarure. The analysis has also considered limitations’ causes and consequences. The article of this chapter was submitted to IFRN’s Holos Paper in August 2016. Chapter 3 consists in a theoretical essay about the concept of pedagogical usability. It reflects upon the adopted practices in LVE and discusses about what dimensions and criteria presented in the literature can be implemented in order to minimize limitations observed in online modality. Although this dissertation’s references are spread across all its scientific production, this chapter has a particulary important theoretical foundation for this study. The article of this chapter was published in IFRN’s Holos Paper in August, first edition of 2016. Chapter 4 presents the pilot study carried out in this research, whose goal was to performing a pretest to implementing the strategies related to tecnhical usability concept and, mainly, to the pedagogical one in LVE, according to discussions presented in the theoretical essay in chapter 3. From this study on, which took place during IFRN’s free b-learning courses, virtual learning objects, data collection tools and methological strategy, as a whole, were adjusted and improved for the next stage. Failures were identified both by researchers and by tudents, who verbalized their impressions regarding the proposed implementation. These initial and preliminary data was enough to confirming the research viability in reaching the final educational objective, namely, learning. The article of this chapter was published in the minutes of CISTI 2015 – 10th Iberian Conference on Information Systems and Technolofies in June 2015 in Águeda, Aveiro, Portugal. Chapter 5 presents the research conclusions in the form of recommendations to the institution and suggestions for future research. Appendix A is connected to the main object of study, considering that user’s perceptions about LVE, namely Moodle (Modular Object-Oriented Dynamic Learning.

(16) 14 Environment), was investigated in DL context. Different Moodle formats evaluated served for checking the alignment of the above-mentioned LVE to the proposed technical and pedagogical usability dimensions in this study. Several actions converged to assuring that both theoretical and methodological aspects were connected to the goals of the proposed main research, namely, using the usercentred design (UCD) approach, detecting the importance of creating a suitable information architecture project, using the comparative methodological strategy and appropriated data collection techniques, which include in-depth interviews and questionnaires, as well as making data quality analysis and statistical treatment. The article of this appendix was published in the minutes of the IX International Conference on ICT in Education – Challenges 2015, Braga, Portugal. Appendix B is connected to the research object as far as usability is concerned, as well as to the reasons presented for Information Architecture (AI) and Human-Computer Interaction (HCI) topic. Fundamental principles to set up and adjust an LVE were discussed and methods were adopted in order to obtain user’s perception, such as interviews and contextual analysis, providing knowledge and experience to the researcher. The article of this appendix is also correlated with chapter 5 proposal and was published in the minutes of the 15th Ergodesign & USIHC, June 2015, Recife, Brazil. Appendix C focuses on the observation of universal design principles. These are closed connected to usability and accessibility concepts and, far beyond the physical environments, they can be implemented in the configurations of LVE interfaces. Similarly, this theme is linked to various criteria of pedagogical usability concept. Further, it presents a human-centred approach. Still, the methods connected with field observation , such as open enterview, descriptive and qualitative analysis, were useful as a methodological exercise for this work, especially for chapters 3 and 4. Finally, Appendix D concentrates on providing guidelines and some information design concepts, graphic and visual language, linguistic components structure and text joint stategies, while international ergonomics contents, in order to provide easy reading and interpretation, comunicating information. This is a relevant theme when designing educational material for DL, which is related to the object of this investigation. The article is intrinsically linked to chapters 4 and 5..

(17) 15 The articles in Appendices C and D were published in the minutes of the IV Ergotrip Design – Design, Ergonomics and Human-Computer Interaction 2015, in Aveiro, Portugal and selected as a book chapter for Ergotrip Design – text collection selected by the Scientific Committee of International Studies Meetings Luso-Brazilian in Design and Ergonomics. Aveiro, UA Publisher ISBN 978-972-789-485-7. Table 1 presents information about the chapters and appendices, the stages they are linked to and the place the articles were published in or submitted to. Table 1 - Compilation of chapters/articles, stage and submission places Chapter / Appendix. Title. Stage. Published in or Submitted to -. Chapter 1. Introduction. Framework, Problem, Objectives and Reasons. Chapter 2. Overall and websites usability evaluation in learning virtual environments. Usability Evaluation. Submission to Journal HOLOS - IFRN. Chapter 3. Pedagogical usability: a theoretical essay for e-learning. Theoretical Proposal. Journal HOLOS IFRN, V. 1, 2016. Chapter 4. Technical and Pedagogical Usability in a virtual learning environment: a case study at the Federal Institute of Rio Grande do Norte – Brazil. Pilot Study. 10ª CISTI'2015 – Portugal. Chapter 5. Conclusion. Future Research Suggestions. Appendix A. Learning virtual environment evaluation: a comparative study between Moodle formats from a university. Contribution in Chapters 2, 4 e 5. IX International Conference on ICT in Education – Challenges, 2015, Braga, Portugal. Appendix B. Information architecture and usability: analysis of the author registration and submission process of an article in the eletronic periodical Bibliocanto of UFRN. Contribution in Chapters 4 e 5. 15th ERGODESIGN & USIHC, 2015, Brazil. Appendix C. Evaluation of the historical center of Braga - Portugal from the universal design principles. Appendix D. Tool of support for book’s cover design based on ergonomic principles. Source: created by the autor (2016).. Contribution in Chapters 3, 4 e 5. Contribution in Chapters 4 e 5. -. Conference IV Ergotrip Design – Design, Ergonomy and Human- Computer Interaction, 2015, in Aveiro, Portugal + chapter book Conference IV Ergotrip Design – Design, Ergonomy and Human- Computer Interaction, 2015, in Aveiro, Portugal + chapter book.

(18) 16 1.3. IFRN and distance learning IFRN was founded in 1909 as Escola de Aprendizes Artífices1, changing its. name several times over decades. It has always been a teaching technological reference in this state. Having benefited from the Federal Network Expansion Plan of Professional Education, Science and Technology of the Ministry of Education, the Institute increased its offers places and nowadays serves a diversified profile audience, providing courses from integrated technical into the high school up to graduate programs (DANTAS e COSTA, 2012). Distance Learning Campi was created in January 2011, equipped with data center, library, exclusive building and staff for academic and/or administrative tasks, such as tenured and temporary teachers, tutors, trainees and coordinators (IFRNEAD, 2016a). Campi uses Moodle platform in three versions: Academic’s, the current project; 1.9’s, in discontinuity and the Presential’s, supporting activities for classroom lessons. Figure 3 presents the access área. Figure 3 – Area in IFRN-DL portal, which allows students to choose Moodle version to access. Source: IFRN-EAD (2016d).. 1. School for craftsmen.

(19) 17. The offer of DL courses has started with the top level and has bee expanded to meet the needs of the institution different levels. Later, were offered courses from Brazil Technical School Network (Brazil e-tec Network) as well as from Brazil Open University System. Nowadays, there are several free courses: Technical Subsequent (3), Graduation (2), Improvement (2), Specialization (5), Initial and Continuous Training (5) and Profuncionário (4)2 (IFRN-EAD, 2016c). Campi has 29 support centers for students classroom, spread out in 21 municipalities of Rio Grande do Norte, 13 of them based in other IFRN’s Campi, as shown in Figure 4. Figure 4 – IFRN’s support centers. Source: IFRN-EAD (2016b).. Due to geographic coverage within Rio Grande do Norte, being this a sample of what has been implemented in Brazil, this study is, potentially, extremely important from a pedagogical point of view.. 2. Técnico Subsequente (corresponding to secondary school), Graduação (university degree), Aperfeiçoamento (between university degree and specialization), Specialization, Formação Inicial e Continuada and Profuncionário (specially oriented to public teaching), respectively..

(20) 18 References DANTAS, Ana Catharina da Costa; COSTA, Nadja Maria de Lima, Projeto PolíticoPedagógico do IFRN: Uma Construção Coletiva: Documento–Base/ Instituto Federal de Educação Ciência e Tecnologia do Rio Grande do Norte, Natal: IFRN Ed.,2012 INSTITUTO FEDERAL DE EDUCAÇÃO CIÊNCIA E TECNOLOGIA DO RIO GRANDE DO NORTE.IFRN-EAD. 2016ª. Available from: <https://ead.ifrn.edu.br/portal/sobre-o-campus/>. Access on: 13 ago. 2016. ______. IFRN-EAD. 2016b. Available from: <https://ead.ifrn.edu.br/portal/polos-de-apoio/>. Access on: 13 ago. 2016. ______. IFRN-EAD. 2016c. Available from: <https://ead.ifrn.edu.br/portal/cursos-ofertados/>. Access on: 13 ago. 2016. ______. IFRN-EAD. 2016d. Available from: <https://ead.ifrn.edu.br/ava/>. Access on: 13 ago. 2016..

(21) 19 2 OVERALL AND WEBSITES USABILITY EVALUATION IN LEARNING VIRTUAL ENVIRONMENTS. Francisco Monteiro de Sales Júnior André Luís Santos de Pinho José Guilherme da Silva Santa Rosa Maria Altina da Silva Ramos. Submitted to Periodic HOLOS - an online publication of the Federal Institute of Rio Grande do Norte (IFRN) in August 2016.. 2016.

(22) 20 Overall and websites usability evaluation in learning virtual environments Avaliação de usabilidade geral e de websites em ambientes virtuais de aprendizagem Abstract: The goal of this research was to evaluate the usability in Moodle’s environment configurated by teachers for free e-learning courses provided by IFRN (Federal University of Rio Grande do Norte). Being of exploratory and qualitative nature, the empirical method adopted was based on direct observation and virtual ethnography. The approach was focused on students in order to analyse if the pedagogical practices were set up taking into consideration usability principles. This study was conducted during thirty-two months and thirty-two subjects in eight different courses of different levels were evaluated. The results point to nine groups of main problems. These were categorized based on five usability criteria present in the literature. One concludes that it is teacher’s responsibility to act as an educator and as an information architect, aligning pedagogical mediation with usability principles. Keywords:. educational. technologies,. pedagogical. mediation,. information. architecture. Resumo: O objetivo da investigação foi o de realizar uma avaliação de usabilidade nos ambientes do Moodle (Modular Object-Oriented Dynamic Learning Environment) configurados por docentes em cursos ofertados na forma educacional e-learning do Instituto Federal de Educação, Ciência e Tecnologia do Rio Grande do Norte (IFRN). De cunho exploratório e qualitativo, foi adotado o método empírico baseado em observação de campo e etnografia virtual, com abordagem centrada nos estudantes, para verificar se as práticas pedagógicas eram concebidas considerando princípios de usabilidade. Sendo utilizado um recorte temporal de 32 meses, foram avaliadas 32 disciplinas em 8 cursos distintos, de diferentes níveis. Os resultados apontam para a identificação de 9 grupos de problemas principais que foram categorizados a partir de 5 critérios de usabilidade geral e de websites previstos na literatura. Concluiu-se que cabe ao professor atuar como educador e como arquiteto de informação, alinhando a mediação pedagógica a princípios de usabilidade. Palavras-chave: tecnologias educativas, mediação pedagógica, arquitetura da informação..

(23) 21 2.1. Introduction Due to recent public policies to encourage education access (LEMGRUBER,. 2012), it has been growing the number of distante learning (DL) free courses in Brazil. In this expanding scenario, main learning virtual environmentd (LVE) used in schools, including IFRN (Federal Institute of Rio Grande do Norte), has been Moodle (Modular. Object-Oriented. Dynamic. Learning. Environment).. This. worldwide. recognized educational platform has Brazil as its third largest user, with 4.303 registered websites, only behind Spain and United States (MOODLE, 2016). This stucture has educacional resources that can be exploited in different learning levels (SALES JÚNIOR et al., 2015). The whole system is designed to valorizing usability aspects that guarantee students ease of use, which has been widely confirmed in many published research. However, this acknoledgment only provides capacity to create learning ambiances. It is only after teachers’ intervention (not the existence of technological resources) that the virtual environment can de transformed into a fruitful and meaningful space where human beings and technical objects Interact, building learning and knowledge (SANTOS e OKADA, 2003). It is teachers' responsibility, of those who work in the e-learning educational modality, the mission to align pedagogical mediation with virtual elements, available or integrated ones (LEMGRUBER, 2012), observing a comprehensive and intuitive use of the platform. When this challenge is not reached, students are the most affected ones by the negative consequences coming from an environment with usability problems, which causes anxiety, stress, tension, difficulty in memorizing, irritability and abandonment (REIS, 2007). As usability should be evaluated on the platform use context, it is important to observe final users’ records, written or verbalized, checking absence of satisfaction, efficiency and effectiveness (NIELSEN, 1993), as it is them who get the information and Interact with the available interfaces (CYBIS; BETIOL; FAUST, 2010), It is relevant to highlight that setting up the system and respective adopted resources has to respect not only usability guidelines but also to teach online modality assumptions and requirements, in order not to lose the ultimate goal, which is learning. This analysis needs to be done in a continuous way in order to correct the existing barriers and to improve the subsequent educational processes..

(24) 22 In this context, the research is consistent with usability evaluation in Moddle environments, configurated by teachers for IFRN’s free e-learning educational courses. The empirical method adopted, of exploratory and qualitative nature, with approach focused on the students, was based on direct observation and virtual ethnography in order to analyse if pedagogical practices are designed taking into consideration both overall and websites usability principles. The results were presented based on the identified problems and were categorized according to the literature, also considering its causes and consequences. 2.2. Theorical framework The theoretical framework used for the proposed diagnosis of this research. permeates overall and websites usability classical principles connected to effectiveness, efficiency and the satisfaction of specific users, who have specific objectives, in specific interaction environments and contexts, assigning a quality attribute to the interface, as a NBR 9241-11 11 (ASSOCIAÇÃO BRASILEIRA DE NORMAS TÉCNICAS, 2002) and Santa Rosa and Moraes (2012). Nielsen (1990) and Rolf Molich’s heuristics for usability evaluation, as well as the components that have to be observed for an adequate interaction, and later defined by Nielsen (1994), were carefully examined. Guidelines of ISO 9241-11 (2011), dealing with the importance of the interface being self explanatory and providing ease of use were considered. Bastien and Scapin’s recommendations and presents in Ergolist3, were also taken into account. Although lists with ergonomic criteria for inspection were not directly applied, Ergolist was consulted for a deeper qualitative analysis, reducing subjectivity, normaly associated to assessement processes, as usability problems have origin in interface ergonomy (CYBIS; BETIOL; FAUST, 2010). Similarly, authors who propose different methods and techniques for usability evaluation, such as predictive, forward-looking and objectives, based on tests with users or models (PRATES e BARBOSA, 2003; CYBIS, 2003; PREECE, 2005; DIAS, 2006; NASCIMENTO, 2010; SANTA ROSA e MORAES, 2012), were also considered to support methologic strategy.. 3. A support tool that identifies possible interface ergonomic flaws in Evaluation of Human-Computer Interfaces (LABUTIL, 2016), provided by Santa Catarina Federal University’s (UFSC) Usability Lab..

(25) 23 Studies about the relevance of Information Architecture (IA) based on authors as Wurman (1991), Rosenfeld and Morville (2006), Agner (2009) and Christina Wodtke (2009) were also mentioned. The essence of their approaches points to the the fact that if an environment is designed according to IA guidelines, its structure will considerably contribute to users information access, although this is not the unique factor defining usability (MAIA, 2015). Moodle’s studies, LVE adopted in the institution under research, were consulted only to confim that the platform is tested and validated, presenting a dynamic structure, enabling proper management of digital contents based on current computational innovation. However, one considers that this functional system is only an educational tool that needs to be properly fed and to have its pedagogival resources adequately positioned and used, in order to avoid loosing the foreseen benefits (Sales Júnior et al., 2015) Researched object context was analysed according to the main interface evaluation rules. Also specific e-learning studies, with a pedagogical focus, such as those belonging to Ssemugabi (2006), Nokelainen (2006) and Reitz (2009) were mentioned. In the same educational context, were taken into account guidelines for DL, as its own language needed, among others mentioned in Quality References for Distance Higher Education4 (MEC/SEED, 2007). 2.3. Methodology 2.3.1 Description The research is stated as basic and descriptive, essentially of qualitative. nature, which contributes to deepening the theme. It is also characterized as exploratory (Gil, 2008), ensuring greater familiarity with the problem, making it clearer and opening options for future research. Results can be used for practical applied purposes, considering that identification and presentation of usability problems may involve institutional measures aiming to minimize barriers in similar contexts. 2.3.2 Scenarios, delimitation and research subjects. 4. Referenciais de Qualidade para a Educação Superior a Distância..

(26) 24 The case study was carried out based on LVE with disciplines of free courses, in DL modality, in different levels and took place from January 2014 to August 2016. Disciplines involved had different timetables, some of them had distance tutor support. Courses intrinsic aspects were not considered, as they were not objects of the research. The same goes for the contents data class, the delivery or not of the printed education material and the abandonment, estimated at 50%, as institutional average. Research subjects, students enrolled and active in the evaluated courses, did not have their personal profiles collected. For larger support of the results founded, one states that what motivates these students to enrol in these courses is knowledge updating or acquiring new skills and they already have some degree of knowledge and work experience. Only students working in the discipline, and authors of records connected with overall and websites usability topic, were considered for analysis. The Institute offers three Moodle versions, being the “Academic”, the current customized design; the “1.9”, the one that is being discontinued and the “Presencial”, that supports activities for classroom lessons. The access to the platform environment by researchers took place in two distinct and unique ways: a) as teachers of the courses, enabling them to have access not only to specif disciplines under their responsibility but to the others making part of course program; b) as students, regularly enrolled in the courses. In order to keep a neutral environment, researchers in both scenarios adopted a spontaneous attitude in order to ensure naturalness and impartiality. Courses in which students participate as researchers, these were included in digital social networks created by students for better communication among peers. Table 1 presents data consolidation characterizing the research, its scenarios, Moodle version and the sample containing level and quantity of the courses, disciplines, enrolments and tutors as well as researchers positioning while teachers or students. Table 1 – Sample data and research characterization Course level Course. Quantity Discipline Enrolment. Tutor. Researcher positioning. Moodle version. Initial and Continuous Training. 3. 11. 48. 1. Students. Technical Subsequent. 3 1. 14 6. 130 113. 6 3. Students/Teachers Teachers. 1.9. Students. 1.9. Specialization Master’s. 1. 1. 20. 0. Total. 8. 32. 311. 10. Source: research data (2016).. 1.9 / Acadêmico 1.9 / Acadêmico / Presencial.

(27) 25 2.3.3 Evaluation method and data collection For adjustment of usability evaluation techniques, was considered the context in question (PRATES e BARBOSA, 2003), specially the educational one in virtual environment. Students’ real and natural desires were also valuated (CYBIS, 2003; PREECE, 2005) in order to identifying problems hindering learning. Rubin (1994) and Jordan (1998) considerations, dealing with the importance of user’s participation in identifying. existing. barriers,. were. respected. while. observing. difficulties. externalizations, dissatisfactions, preferences and performances. The empirical method is called by Jordan (1998) “Field Observations” and involves participants’ observation using an interface in real context in normal use conditions and without researchers presence, assuring an ambience validity level. The same technique is defined by Preece (2005) as “Field Technique” consisting in researching subjects’ participation, although indirectly. Dias (2006) classifies the method, equally empirical, as “Tests with Users”, stating that these users contribute to find usability problems in real situations. On the other hand, Cybis (2003) fits the procedure as “Objective” as seeking to find problems from users’observations interacting with the real system. In this context, one opted for adapting the essence of the above-mentioned techniques, observing studients’ quotidian and, indirectly, tracking their explicit records on the virtual platform and their externalizations in external informal environment. This continuous involvement in student’s life during the use of the platform in realistic situations took place in two phases, as explained below. In the first phase, researchers used observation techniques based on their experience in students’ virtual communities in digital social networks, external to the platform, as well as exchange of personal messagens, featuring a virtual ethnografy, as it took time and involvement in students’ routines (MACLARAN e CATTERALL, 2002). Experiences external to the platform also included participation in classroom activities in order to fulfil courses timetable and to conduct evaluative activities. The behaviours observed were spontaneous and clearly showed dissatisfaction, insecurities, incertainties, concerns and complaints, among others, sometimes verbalized. in. an. informal manner.. These. reactions. were. understood. as. consequences of potential usability problems and were only used as guiding elements of environment analysis. Researchs’ anonymity and records’ secrecy were preserved..

(28) 26 In the second phase, researchers followed the interactions observing textual discourse, as index of communicational acts in an online community (KOZINETS, 2002), Moodle platform. Direct observation was made to student’s explicit records, specially the asynchronous tool, “forum”, in the course of participations, interactions, interventions and verbalizations in different contexts and during the implementation of online pedagogical practices. In order to carry out the analysis and to detect a potencial problem connected to lack of usability, one only used records that pointed to the difficulties in achieving effectivess and efficiency in interacting with the interface, as well as to low the level of satisfaction (DIAS, 2006). 2.3.4 Data processing, analysis and interpretation Based on the records collected inside and outsite the platform, it was researchers’ responsibility to verify, using a qualitative analysis, the existence (or not) of usability problems and the respective causes and consequences. In fact, one of the main tasks of ethonography virtual use is not only paying attention to participant’s reports but also to the meanings assigned to them. This enables the researchers to know how to decode them, how to understand and decipher participants’ behaviours, feelings and intentions in order to deduct logical implications (MARIAMPOLSKI, 1999). Meanwhile, some reports were not interpreted as connected to overall and websites usability but to external factors, as noise and other interferences. Data was filtered out and tabulated in a Microsoft Office Excel 2007 spreadsheet for further analysis and systematization. Afterwards, using the triangulation technique, they were re-evaluated, contributing to the exclusion of repeated records and to the possibility of calculating data correlation of the remaining records, which resulted in 232 valid unities for the whole period and courses. Among the different possibilities of presenting usability problems identified in the analysis, researchers opted for a systematization based on Reitz (2009), who organizes a base of technical and pedagogical usability issues in his research. This author’s studies are based on Nielsen’s (1994) as well as on Nokelainen’s (2006) and on Ssemugabi’s (2006), who adjuste guidelines to the e-learning context. Thus, problems were classified in two groups: “interface overall usability” and “specific technical usability for webbites”. “Pedagogical usability” was not considered in this research. Some critera were adapted and aggregated among them given the.

(29) 27 proximity and correlation among the problems found, since, in some situations, the evidence of a same problem was categorized in different criteria. That is, in some cases, the same problem was identified in different records; a unique record implied connection to different problemas. This is the reason why the quantitative dimension does not apply for results and conclusion purposes but for the qualitative essence that data reflect. 2.4 Results Table 2 presents the main problems found and the respective quantities of records, which were caregorized for overall and website usability based on Reitz’s (2009) criteria. Table 2 – Problems categorized based on Reitz (2009) Criteria. System status visibility Consistency and student model. Help, use flexibility and efficiency Minimalist design, organization, navigation and structure. Problem. Absence of alerts and feedback Inadequate language Format inconsistency. Record. 11 14 13. Labelling inconsistency. 6. Message and search limited structure. 6. Non-minimalist design Excess of instructions, explanations and details. 10 14. Excess of forum. 4. Navigation inconsistency. 8. Source: Created by the authors (2016).. 2.4.1 System status visibility 2.4.1.1. Absence of alerts and feedback. The system has no resources (or it is not configurated) for sending authomatic messages to the students to informing them about news tasks inserted or schoolworks’ deadlines. Implementing or configurating this functionality would minimize students’ absences in the proposed tasks. These automatic alerts should be suitable to the educational modality. Educational tecnologias resources should meet this need, although autonomy and attention is expected from a DL student (MEC/SEED, 2007; LEMGRUBER, 2012). Figure 1 presents a record of this problem..

(30) 28 Figure 1 – Absence of alerts. Source: Research data (2016).. Some evaluative tasks sent by students are not configurated to feedback reception confirmation, which leaves them suspicious about delivery effectiveness, generating overwork, as presented in Figure 2. Figure 2 – Doubt about feedback absence. Source: Research data (2014).. Confirmation by e-mail is not always activated and messages are timidly displayed on the screen. Repeatedly, and in different courses, when submitting tasks online, students understand that the file was sent. However, afterwards they observe that status activity is under “draft”, as shown in Figure 3. Figure 3 – Missed Feedback. Source: Research data (2015)..

(31) 29 Similarly, the feedback that occurs after the submission of the questionnaires is weak, which leaves the student doubting about the interaction success, closed/sent. “Status” label is small and does not highlight the information for the returning action, which leaves the student insecure, even excluded from the process as shown in Figure 4. Figure 4 – Worry, doubt and uncertainty due to feedback absence. Source: Research data (2014).. In some scenarios, questionnaires are configurated to send or reveal the mark assigned to an activity at a later date, which is corrected by the system (multiplechoice tests) or by the teacher (open questions tests). Feedback system concerning the correction status of the submitted activities is dubious, which leaves students doubt about marks correction and feelings of injustice or, still, that they did not deserved the assigned mark (a non-existent one), demanding explanations. This problem can be minimized if teachers create an adequated communication about the stages and evaluative processes, which need to be improved with simple explanations and with configurations adjusted to the system, avoiding mistakes. Record presented in Figure 5. Figure 5 – Misunserstanding between given marks and configuration error. Source: Research data (2014)..

(32) 30 Informational environments need to be built with features helping user cognition. According to Christina Wodtke (2009), user needs to have feedback on their actions, preferably from existing standards, in order to provide the proper expectation, the right support in case of errors and to foresee, to protect and to inform. 2.4.2 Consistency and student model 2.4.2.1. Inadequate language. The use of inadequated language causes students lack of understanding as well as misinterpretation. It is considered inadequated language the one that does not clarify its primary purpose by using excess details, extreme objectivity, symbols or technical jargon. Students’ misinterpretation to teachers’ orientations causes confusion between activities delivery and platform basic resources. Questions in evaluative activities are built in a way that causes indecisions, leading students to misunderstand what it is expected to as an answer, which it is not acceptable in classroom environment, even less in DL’s. On the other hand, some teachers exaggerate in the comments, explaining what should be intuitive. The use of technical jargon, either coming from the teachers or from system configurations, which leaves students insecure, can be seen in Figures 6, 7 and 8. Figure 6 – Difficulties in understanding the task (language). Source: Research data (2014). Figure 7 – Difficulties in understanding technical jargon. Source: Research data (2015)..

(33) 31 Figure 8 – Misunderstanding between assessment and marks. Source: Research data (2014).. 2.4.2.2. Format inconsistency. The absence of format patterns and of colours inapprotiate use prevents reading, as it is shown in the templates linking to the institutional digital social network, which cover or overlap fixed labels. This problem occurs in different situations, as in the absence of colours contrast between user name and forums background. The disordly combination of bold, italic, underline, continuous text, body font, capital usage, inadequate typology, excess spacing, diverse alignments and other similar formats hinder reading or labels visualization. Copying texts previously configurated in other environments, as those coming from websites and in PDFs, bring native styles and are visually inconsistent. Similarly, course structure adequacy based on the import from other lessons or courses, without being adjusted, have information and discontextualized labels, as unknown teachers names and inadequated terms as “…under construction”. Students identify and interpret these findings as a dearth of interest and amateurism, undermining the credibility of the institution. Some records are presented in Figure 9. Figure 9 – Different format problems and others related. Source: Research data (2015)..

(34) 32 2.4.2.3. Labeling inconsistency. The tool available to control students’ activities progress is not understood, which leaves them confused and anxious. There are doubts about checkbox functionality purpose: if it is to remind stages they have gone through, if it is or not of mandatory use, if the action validates the participation with the teacher, if it interfers with progress activities; if what was done can be reviewed, if the checkbox is authomatically selected according to the natural progress and, even worse, if there is delayed in relation to the classmates who, eventually, might be at an advanced stage in their future activities. Teachers’ negligent tools adoption causes expectation and anxiety and, moreover, causes delay in performing the tasks. This functionality is shown in Figure 10. Figure 10 – Functionaly (checkbox) causing doubts. Source: Research data (2015).. Some hyperlinks labels are presented in an ambiguous format, as the one that suggests that LVE is integrated to digital social networks, namely facebook and twiter, but, in fact, are only institutional profile accesses in those networks (Figure 9) leaving students frustrated. The way messages are structured in forums, Moodle native resource, is not noticeable to students. Although it is possible to organize posted messages in different ways, namely “show nested answers”, “arrange by oldest on top”, “arrange by newest on top”, “list of answers”, student does not understand the possibility of customization and show difficulties in finding the information needed, demanding more effort. Rosenfeld and Morville (2006) discuss the importance of labelling and define a unique system to deal with this issue in their IA proposals..

(35) 33 2.4.3 Help, use flexibility and efficiency 2.4.3.1. Message and search limited structure. Students report their unmet needs during the use of Moodle messaging system as, for instance, the impossibility of sending attachments. Still, they consider the platform system is not user friendly for not keeping the received messages with clear access for a new reading. The system hides them after reading, only enabling the search, which demands memorizing some words, requiring greater cognitive effort. Moreover, when replying to a message, the submission button screen overlaps the original text, which was reported as a nuisance. Figure 11 shows an exemple reported. It also presents a record of exchanded messages among colleages, leaving clear doubts and concerns about conducting activities processes. Figure 11 – Submit botton in the messaging system. Source: Research data (2015).. Educational environments must be designed providing architecture for students of different experience levels, even anticipating by suggesting a contextualized help (WODTKE, 2009). In the forums, message search is not well accepted, considering that the system returns the results from all other qualified forums of the discipline and, sometimes, they are not a few. The difficulty some students have in finding tearchers’ e-mail address shows a system limitation, although it is clearly stated in each participant’s profile..

(36) 34 2.4.4 Minimalist design, organization, navigation and structure 2.4.4.1. Non-minimalist design. Spontaneous comments from students point to the ongoing identification of visual pollution and excessive contents on pages designed for different purposes, such as instructions, hyperlinks, videos, administrative reports, news, notes, educational material and proposed tasks, among others, without systematization and organization. Some functionalities mentioned were considered as elements not adding any value, only contributing to the absence of aesthetic design, as the introductory images taking up space unnecessarily and that could be presented in small sizes. Information organization could be arranged using files, tables, categories, topics or hierarchies in order to facilitate understanding, and not in the body pages. Specific disorder disciplines’ main page hinders the needed orientation for taking steps in a systematic way towards knowledge construction process. These out of alignment features contribute to a faulty organization system, which causes confusion and difficulty in understanding procedures to be performed. Students’ records as “Saw two links for sending (…). an essay activity and a final work. Was there an essay activity?” point to overload information, confusing the user. Disorganization is worsened by mixture of colours and by non-uniformity of educational material format. Informational spaces configurated by teachers do not delimit what is relevant, by choosing what is important to students in order to optimize students’ time and attention, as recommended by por Wurman (1991) and Christina Wodtke (2009). The ease of understanding and learning, the degree of achieved productivity, low need to relearn interaction elements, reduced errors rate and the subjective satisfaction must be adapted to the educational environment, as interface components discussed by Jakob Nielsen (1993) to define usability. Figure 12 shows a non-minimalist design record. 2.4.4.2. Excess of instructions, explanations and details. Some elements that contribute to visual pollution, memory overload and students’ disorientation, such is the case of excess of instructions for carrying out tasks and the complex method of giving marks. Although teachers’ motivation is clearifying the processes, students feel themselves confused with long and detailed instructions only focusing on instrumental and technical aspects, emphasizing.

(37) 35 instructional materials at the expenses of pedagogical mediation (LEMGRUBER, 2012). By not considering simplicity of the rules and procedures, teachers impair understanding. This group of problems is closely related to one that deals with used language. Giving to much information causes distance between what it is understood and what should be understood (REIS, 2007). Figure 13 shows a record of excess of instructions, explanations and details. Figure 12 – Overload information in a non-minimalist design. Source: Research data (2014). Figure 13 – Example of instructions for performing tasks. Source: Research data (2015).. 2.4.4.3. Excess of forum. Similarly to excessive explanations, the amounts of forums created do not please the students, although they are intended to sistematize educational processes stages and studied contents. In some disciplines were used up to five different forums for a 30 days time frame, only to help answering questions. Students feel.

(38) 36 confused about which environment shoud address the issues, posting messages in the wrong forums or not finding information they need. Forum labelling problems are connected to this limitation. Figures 14 and 15 show examples of this problem. Figure 14 – Post in the inadequated forum. Source: Research data (2014). Figure 15 – Forum created and not not used. Source: Research data (2016).. 2.4.4.4. Navigation inconsistency. The navigation system in environments configurated by teachers is fragil, not allowing students to find what they look for and making them lose orientation due to the volume of information available. Demanding scroll bar continuous use in order to get the desired content is a negative aspect. Excess of options creates confusion and students feel lost with such information or need greater cognitive effort and double reading for a better understanding. Labels defined are not functional as students ask for help when have to take the necessary steps to reach the desired features or contents, whereas this process should be intuitive. This can be seen in this record “Hello, (tutor)! I am not finding the activity. Could you help me?” and the answer is “Activity is in the menu ‘Actividad’ (4th menu of our page group). For you to compare, the menu where is this forum is the menu ‘Momento Presencial’ (2nd menu of the course page)”. To this problem contributes the breaking of the navigation pattern that leads to the current lessons, leaving the structure commonly used (vertical) to the menus in horizontal shape, features that can be configurated in the system. This problem is shown in Figure 16..

(39) 37 Figure 16 – Horizontal navigation menu, not commonly adopted. Source: Research data (2014).. As the systems are integrated, some students find it difficult to understand the label system and this disrupts the navigation process. One exemple is misunderstanding the icons available for moving the template screen up and down and vice-versa (Figures 3 and 7). The absence of simplicity and consistency in the navigation occurs from incorrectly positioned or non-existing hyperlinks, such as the summarization in large pages. As the labeling problems, some hyperlinks do not show themselves as standard connections, requiring checking with the mouse, either by students or by specific textual instrucitions, as presented in Figure 17. Figure 17 – Navigation instructions and non-standard hyperlinks. Source: Research data (2015). Paradoxically, some elements suggest hyperlinks presence, when this does not exist. Hyperlink opening does not occurs in a standard way, being observed different strategies, such as direct download files, pop-up use, new browser sessions or overlap the existing one, forcing the student getting out of the environment and demanding the return using the browser, not the system resources. Although redundancy of access to educational material might be positive when properly adopted, excessive repetition for hyperlinks for downloading the same files from different places in the platform implies double work and repeated files in students’ repository, given their profile to navigate exploring, not reading. Some teachers are not qualified for the correct use of text editing features, among which is the one that allows configurating the access to hyperlinks on forums. In this case, teacher “paste” the reference web address, demanding student to access it and to copy it to a browser session, which imples greater work. Hyperlinks discontinued presence, such as those pointing to campi portal or to the old institutional academic system, disclose.

(40) 38 the need to review the connections. Christina Wodtke (2009) states that an environment must have clear and ergonomic design, favouring user understanding of their location and the necessary paths to be able to find what is being looked for. LVE navigation must be simple and easily accessible, integrating media resources (SANTOS and OKADA, 2003). A record of navigation inconsistency can be seen in Figure 18. Figure 18 – Pop-up use when activing a hyperlink. Source: Research data (2015). 2.5 Conclusion Usability problems identified that are related to Moodle native features were minimal compared to other barriers. Both in terms of identified records as in their impact, one cannot say they are enough serious to derailing the learning processes, confirming the premise that it is an established and tested environment. One registers, however, students dissatisfaction with the messasing system that automatically hides the messages received and read, demanding a search using some parameters stored in the memory, in case one needs to perform a new reading. Additionally, it was noticed the inability to attach files to send messages, limiting information sharing. One observes that these questions results from students’.

(41) 39 native experience with Internet webmails and cannot be disregarded. Given students interaction in environments external to the platform, the messaging system is not considered a troubling problem. Neverthless, one emphasizes the importance of being aware of the fact that user experience needs to be considered during the configuration process, include those depending on the teacher. Serious problems arise when the platform is configurated and fed inadequately by teachers, who do not consider strong principles of accessibility and information architecture. One understands that pedagogical support required to teaching in DL method oblige not only knowledge of appropriated mediation strategies but also instrumental ability to use virtual platform resources. Further, knowing IA principles is needed in order to construct and display an informational and interative envinronment, which goes beyond technical issues when configurating the system. Thus, emerge that usability in e-learning systems depends essentially on teacher’s role. Modality demands that the professional is not only an educator but also an information architect. In this sense, teacher must have knowledge in organization systems, navigation, search, labeling and representation in order to meet student’s learning needs. In some scenarios, the problems come from well-intentioned attempts to adopt good learning practices. In this context, is the considerable amount of educational material available and detailed explanations to carry out an activity. Teacher intentions are not to leave any doubt and, without realizing it, confuses students even more, without understanding that a simplified language could be used. The large number of forums available were also identified as a problematic issue, as they long for simplicity. Teachers, however, multiply the number asynchronous sessions, believing the interactions will improve. Although some problems may arise from noble intentions, mistakes encourage judgement. Considering the average profile of institution current students and their maturity degree, finding constant and recurrent errors results in criticism (in an informal setting), risking labeling the present design as amateur, undermining the credibility of the institution. It is in this context that teachers need to be aware to assessing student prior knowledge (REITZ, 2009) and to adapt their practices to the realities. Although the categorized problems provide barriers in learning, one can state, based on the observations provided by virtual ethnography, that the severity of an.

(42) 40 usability limitation identified based on student dissatisfaction, is minimized when the teacher responsible for interface configuration (meaning, the author of the obstacle) is accessible and adopts dialogic communication practices, listening to student. In this scenario, one found that students contribute spontaneously to identify the problem, facilitating teacher to correct the environment. Therefore, usability problems are worsened or mitigated depending on the pedagogical approach during the interactions between teacher and student, on or off the platform, which requires dialog. One example of how usability limitations are naturally overcome with a suitable pedagogical stance is observed when students complain that the system does not provide alerts for schoolwork deadlines. In some disciplines, teachers come forward and, on their own initiative, notify students about their actions and the time required to do the tasks, remaining available. At the same time, other teachers do not do it or leave the communication under tutors’ responsibility, who do not have their roles understood as being teachers (SILVA, 2006). This situation affects students’ expectations, causes natural comparisons and, consequently, favours proactive teachers, not occurring dissatisfaction about the lack of warning or feedback system. Thus, problems occur not only due to technical usability issues present in the platform but by inappropriated use of strategies and resources, having teacher as protagonist. Therefore, one can state that the most serious barriers identified for learning processes do not apply to “overall” or “websites” categories but to those related to the teaching behaviour, being much closer to the usability pedagogical dimension. The recent concept of “pedagogical usability” presents dimensions and critera that can be adopted in DL context in order to combine technological facilities with the nature of pedagogical dimension, improving the learning process. Gradually, interactivity has been occupying the space of one-way actions. Criteria presented by different authors support educational materials development, how its pedagogical preparations was conducted, virtual environment architecture, approach strategies, mediation, feedbacks and evaluation, network, cooperative and collaborative way, interactivity, motivations, creativity, active learning in continuous compliance with educational aspects towards the ultimate goal, which is learning (VETROMILLECASTRO, 2003; NOKELAINEN, 2006; OLIVEIRA, 2011; DIAS, 2012)..

(43) 41 For future research, one recommends deepening Pedagogical Usability theme as well as hindrances evaluation based on the dimensions of this concept in order to know students’ perception and to build environments that provide improvements in the teaching-learning process..