The construction and evaluation of new educational software for nursing
diagnoses: a randomized controlled trial
Vanessa E.C. Sousa
⁎
, Marcos V.O. Lopes
1, Gabriele L. Ferreira
2, Camila M. Diniz
3,
Nathaly B.M. Froes
4, Bianca A. Sobreira
5Nursing Department, Federal University of Ceará, 1115 Alexandre Barauna St., Fortaleza, CE, Brazil
s u m m a r y
a r t i c l e
i n f o
Article history:
Accepted 26 October 2015
Keywords: Nursing students Problem-based learning Nursing informatics Nursing diagnosis
Background:Student nurses often have difficulties with diagnostic inferences. To achieve high accuracy nursing diagnoses, effective learning strategies are required.
Objective:To describe the construction and evaluation of new educational software called Wise Nurse, which was developed to increase the capacity of student nurses to identify nursing diagnoses (NDx) and to establish relationships between NDx, defining characteristics (DC), and related factors (RF).
Design:Randomized controlled trial.
Setting and Participants:Participants were 2nd to 4th year student nurses from an undergraduate program at a university in Brazil. Of the 47 recruited students, 37 completed the survey.
Methods:Students were randomly assigned to test the software (experimental group) and to solve printed clin-ical cases (comparison group). A pretest and post-test were applied before and after the experiment. Statistclin-ical analyses of the students' performance in the tests were conducted. The primary outcome was the students' prog-ress in solving questions and clinical cases regarding NDx. The System Usability Scale was used to measure the software's ease of use.
Results:No significant difference was found between the experimental and comparison groups before and after the experiment. The average students' performance in identifying RF and NDx was higher than in identifying DC. The post-test score was higher than the pretest score in both groups (P = 0.022). The usability score was good (average score 83.75, N = 20).
Conclusion:The use of Wise Nurse supported an improvement in student diagnostic reasoning equivalent to that of the traditional NDx training, but the software stands out as an innovative teaching tool.
© 2015 Elsevier Ltd. All rights reserved.
Introduction
In clinical practice, nurses address the responses of individuals, fam-ilies, groups and communities to health problems and life processes. These responses are called Nursing Diagnoses (NDx), and they occupy a central position in nursing care and must be focused on the patient and family. A nursing diagnosis can be defined as a judgment based on a comprehensive nursing assessment. Therefore, an accurate nursing
diagnosis is essential to ensure more effective results and safer patient care (Herdman, Internet source). There are various classifications of NDx. NANDA-I classification is used internationally and includes 13 do-mains, 47 classes, and 235 current diagnoses (Herdman and Kamitsuru, 2014).
Clinical reasoning is the basis of a NDx. Clinical reasoning is needed to distinguish normal and abnormal conditions, group related data, rec-ognize missing data, identify data inconsistencies and make inferences (Alfaro-Lefebre, 2004).
Student nurses face many challenges in performing the clinical rea-soning to identify NDx. These difficulties include the following: the complexity of clinical reasoning, the difficulty/impossibility of measur-ing some human responses, the fact that many events are not presented as listed in books or in nursing classifications, and the fact that some diagnoses share symptoms or defining characteristics (DC) (Cruz and Pimenta, 2005).
Clinical reasoning difficulties lead student nurses and nurses to have poor accuracy of diagnosis, which compromises the quality of nursing care and patient outcomes (Herdman and Kamitsuru, 2014). Thus, ⁎ Corresponding author at: 370 Carnaubas St., Fortaleza, CE 60743-780, Brazil. Tel.: +55
85 3085 4797.
E-mail addresses:vanessaemille@gmail.com(V.E.C. Sousa),marcos@ufc.br (M.V.O. Lopes),gabrielelima@msn.com(G.L. Ferreira),camiladiniz.enf@gmail.com (C.M. Diniz),nathaly.bmf@hotmail.com(N.B.M. Froes),bianka_alves@hotmail.com (B.A. Sobreira).
1Tel.: +55 85 3366 8459. 2Tel.: +55 85 9627 8780. 3Tel.: +55 85 9629 0278. 4Tel.: +55 85 9638 0979. 5Tel.: +55 85 9770 3145.
http://dx.doi.org/10.1016/j.nedt.2015.10.027 0260-6917/© 2015 Elsevier Ltd. All rights reserved.
Contents lists available atScienceDirect
Nurse Education Today
strategies that contribute to the increased accuracy of nurses' diagnostic inferences are required.
The accuracy of a NANDA-I diagnosis is validated when the nurse is able to clearly identify DC and related or risk factors (RF) found in the patient's evaluation and to connect them to the diagnostic inference (Herdman and Kamitsuru, 2014). In this study, we present new educa-tional software, called Wise Nurse, which was developed to help student nurses improve their clinical reasoning as they prepare for clinical practice. This study's hypothesis is that using Wise Nurse may increase the knowledge of student nurses about NDx and their capacity to correctly identify NDx and its elements.
Wise Nurse Program
Software Development
Based on a literature review, we found some computerized educa-tional systems pertaining to NDx. These systems address various topics and stimulate the development of student nurses' diagnostic reasoning through various interfaces and content. Most systems have focused on solving clinical cases in specific clinical scenarios, such as those of neo-natal care (Góes et al., 2011), or in a group of specific disorders such as cardiopulmonary diseases (Lopes et al., 2013) and urinary dysfunc-tion (Martins, 2011). In Brazil, there is no system involving clinical cases covering various scenarios of nursing care. Wise Nurse is an inno-vative educational solution that addresses the need to improve student nurses' performance to infer NDx.
Wise Nurse development began after the proposed project was granted by the university at which the study was conducted. The soft-ware content consists of informative text regarding NDx and an exercise containing 13 clinical cases, which were developed from accuracy and validation studies of specific NANDA-I diagnoses. Each clinical case refers to a single nursing diagnosis, and all NANDA-I domains were cov-ered by the exercise. Thus, to solve the clinical cases, the students need the NANDA-I textbook.
The clinical cases refer tofictitious situations in several settings (including home care, emergency, medical clinic, pediatric care and out-patient care). Each scenario is followed by multiple-choice questions about what defining characteristics, related factors, and nursing diagno-ses are present in thefictitious situations. In total, the clinical cases involve the identification of 39 DC, 15 RF, and 13 NDx.
Prior to system development, a content validation was performed. A panel of 13 Brazilian expert nurses (four masters and nine doctors in nursing practice) was created based on their experience with specific NANDA-I diagnoses. Each expert evaluated one clinical case according to the expert's area of expertise. The experts' individual perceptions about the appropriateness of the DC, RF and NDx in each clinical case were collected using an open questionnaire. Writing and content ad-justments were made based on the experts' suggestions. Additionally, one of the experts proposed the replacement of one of the 13 nursing diagnoses with another belonging to the same NANDA-I domain.
The target users of Wise Nurse were identified as 2nd to 4th year student nurses who were involved in learning activities covering NDx identification. The difficulty level of the clinical cases is intermediate, as since the beginning, the purpose was to involve students enrolled in the intermediate levels of the nursing program. After content valida-tion, a software prototype was built, and a pilot study was conducted in a convenience sample of 17 volunteer student nurses from a university in Northeast Brazil.
The prototype consisted of an electronic form hosted by the Google Docs platform. Students involved in the research accessed the electronic form easily on their own computers through a web link provided by the main researcher. All responses were automatically recorded in a spread-sheet and stored in the principal researcher's Google Drive cloud. Stu-dents' overall hit rate (arithmetic mean of correct responses regarding the identification of DC, RF and NDx) was 75%, which indicates that
the goal of creating an activity with a low to intermediate difficulty level was reached.
After prototype testing, the software specifications were written, and thefinal version of the software wasfirst released in December 2014. Prior to software evaluation by users, a new expert validation phase was conducted to verify content and technical adequacy. This phase occurred from January to February 2015, using a convenience sample of 12 experts in NDx (content experts) and 12 experts in soft-ware development (technical experts) from various regions of Brazil; these experts were volunteers.
Experts gained access to the tool by receiving a password protected link. After walking through the system following written guidelines available in the systems' main menu, experts were instructed to end the application and respond the validation survey.
Based on scale items used in other studies (Lopes and Araujo, 2005; Góes et al., 2011; Freitas et al., 2012), the authors developed two sur-veys covering the content and technical aspects of the software. Ques-tionnaire A comprised 21 items and included the following content aspects: objectives, specific content, relevance, and environment. Ques-tionnaire B comprised 16 items and included the following technical as-pects: ergonomics, functionality, usability, and efficiency. Each survey item response was evaluated using a Likert scale ranging from 1 to 5, with 1 = totally inadequate, 2 = inadequate, 3 = partially inadequate, 4 = adequate, and 5 = completely adequate. As a next step, the authors plan on conducting a reliability study of the questionnaires.
The results from the validation phase demonstrated proportions equal or greater than 90% and 77%, at levels 4 and 5 on the Likert scale, respectively, indicating that the software was suitable forfinal testing by users (Table 1).
Software Description
Wise Nurse is an educational software that focuses on clinical cases, using NANDA-I diagnoses as a strategy to improve student nurses' diag-nostic reasoning. The program was built in Java and can be installed and run on any computer running Java 7.0 or later. The specifications of Wise Nurse are listed as below:
1. The program will provide fundamental concepts of NDx and infor-mation about the diagnostic reasoning process using the NANDA-I classification.
2. The program will provide 13 clinical cases to be read and solved by the user through steps displayed on the screen.Fig. 1presents exam-ples of the features and platform of Wise Nurse.
3. The program focuses on clinical cases that are solved using multiple-choice questions (Fig. 2).
4. The program has the ability to efficiently recover and retrieve data. 5. The program runs on Microsoft Windows and Mac operating system
platforms.
6. The program allows the user to check his/her answers at the end of the exercise (Fig. 3).
7. The program allows the user to read the answer discussions at the end of the exercise (Fig. 4).
Wise Nurse was developed specifically for this study and is not cur-rently available for use or distribution. It is expected that this study will contribute to the improvement of Wise Nurse and other educational tools that intend to improve student nurses' diagnostic reasoning.
Method
Study Design
(CONSORT) statement. The CONSORT statement was developed to im-prove the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results (Schulz et al., 2010). The RCT involved two groups and two measurements before and after the intervention. The experimental group completed a system usability scale.
Procedure
Data collection was conducted in March 2015 with prior approval of the Institutional Review Board. Eligibility criteria included being a stu-dent at the university, being enrolled in the 2nd to 4th year of the nurs-ing program, and benurs-ing older than 18 years. Exclusion criteria included participation in the previously conducted pilot study.
Sample size was determined using the following formula for testing two proportions in clinical trials: n = (σ12+σ22) · (Z∝/2+ Z1−β)2/ (μ2−μ1)2, in which“σ1”refers to the standard deviation of the scores
obtained in the comparison group;“σ2”refers to the standard deviation of the scores obtained in the experimental group;“Z∝/2”represents the
confidence level of 95%;“Z1−β”represents the power, set at 80%; and “μ2−μ1”represents the average score difference expected to be found between the groups, set at 1.5. To determineσ1, a normal distribution of the students' scores was assumed. Considering a total amplitude of 10 points and a half distribution corresponding to 3σ, thefinalσ1
value was (5/3) 1.66 points. Standard deviation of the experimental group was set as being equal to the comparison group. After calcula-tions, we defined a sample size of 20 individuals in each group.
Forty-seven students volunteered. Then, participants were num-bered and randomly assigned by the principal investigator to two groups (experimental and comparison groups) using a random number generator. The two groups were separately addressed. The participants were informed of the meeting dates/times and venue by mobile phone and SMS. Ten of the participants withdrew from the study because of scheduling conflicts. The experimental group comprised 20 participants, and the comparison group comprised 17 participants. Consent forms were read and signed by all the participants prior to addressing the group. The variable used to measure learning outcomes was NDx knowl-edge. To measure NDx knowledge, a pretest was taken by students who had consented to participate in thefirst meeting. This pretest was developed by the principal investigator and included 5 questions, described as follows: 1) a close-ended question about the purpose of nursing diagnoses, 2) a close-ended question to identify the constitutive elements of an example of nursing diagnosis, 3) a close-ended question to differentiate examples of actual, risk, and health promotion diagno-ses, and 4–5) two clinical cases to identify the appropriate nursing diag-noses, defining characteristics, and related factors. Pre-test and post-test were similar in content, but including different examples of nursing di-agnoses. Both tests contained 17 scores each, with afinal grade ranging from 0 to 10. Grading was made by consensus between two evaluators. To ensure that both the pretest and the post-test had the same level of difficulty, a pilot test was conducted with 12 student nurses who did not participate in the survey. Student responses to this pilot test allowed for the questions to be categorized by the level of difficulty and homo-geneously distributed in the pre- and post-test.
At the second meeting, which lasted 60 min, the groups were sepa-rately addressed by a group of trained researchers. The participants in the experimental group were asked to use the software to read about concepts and information pertaining to NDx and to solve the 13 clinical cases. This experiment was performed in a computer lab, such that each user could individually access the system. Students' answers to the clinical cases were automatically stored by the system. Students in the comparison group were asked to read a printed text and to answer 13 printed clinical cases with exactly the same content displayed on the Wise Nurse screen. In both groups, students were asked to consult the NANDA-I textbook to resolve the clinical cases. At the end of the second meeting, the students of both groups took the post-test. The trial ended when all of the participants had participated in all of the study steps.
After interacting with the system, the System Usability Scale (SUS) was used to measure the software's ease of use. This scale consists of a simple subjective evaluation with ten items measured on a Likert scale from 1 (strongly disagree) to 5 (strongly agree).
To calculate the SUS score, the score contributions from each item arefirst summed. Each item's score contribution ranges from 0 to 4; Table 1
The questionnaire and scoring analysis (N = 24).
Subject Validity
indexa
Content validation I. Objectives
1. The software's objectives are consistent with the nursing practice. 0.95 2. The software facilitates learning in the nursing diagnoses subject. 0.93 3. The proposed objectives can be effectively achieved. 0.92 II. Specific content
4. The software content correspond to the proposed objectives. 0.98 5. The software content is sufficient to achieve the proposed objectives. 0.92 6. The software content reaches the scope of the subject accurately. 0.92 7. The information presented in the software content is correct. 0.95 8. The information presented in the software content is well-structured. 0.93 9. The writing style corresponds to the level of knowledge of the target
audience.
0.93
10. The software displays an appropriate number of clinical cases. 0.93 11. The clinical cases' difficulty level corresponds to the level of
knowledge of the target audience.
0.90
12. The contents facilitate learning with regard to the identification of defining characteristics.
0.97
13. The contents facilitate learning with regard to the identification of related factors.
0.93
14. The contents facilitate learning with regard to the identification of nursing diagnoses.
0.98
15. The contents facilitate the development of the ability to relate defining characteristics, related factors and nursing diagnoses.
0.95
III. Relevance
16. The software contents illustrate key aspects that need to be strengthened to nursing practice.
0.98
17. The software contents are important for learning in the nursing diagnoses subject.
1.00
18. Software contents are relevant to develop the ability to identify nursing diagnoses and their components accurately.
0.98
IV. Environment
19. The virtual environment is suitable for the content presentation. 0.92 20. The virtual environment is suitable for learning about nursing
diagnoses.
0.97
21. The virtual environment proposes situations with different levels of complexity.
0.90
Technical validation I. Ergonomics
1. The user can move from one screen to the other quickly. 0.82 2. Data location is maintained consistently from one screen to the other. 0.95 3. Texts and style features (e.g.: bold) are used properly. 0.85 4. Controls and commands are visually distinguished from information
on the screens.
0.93
5. Action buttons are presented in a different graphical style. 0.93 6. Error messages are concise and objective. 0.90 II. Functionality
7. The software is suitable for the intended proposals. 0.90 8. The software does what has been proposed correctly. 0.93 9. The software enables generating positive results. 0.90 III. Usability
10. The software is easy to use. 0.83
11. It's easy to learn the software concepts and applications. 0.82 12. The software allows the control of the activities, favoring the
navigation of content.
0.77
13. The software allows the user to have ease in applying main concepts. 0.87 IV. Efficiency
14. The software response time is suitable for the user to perform activities.
0.98
15. Computer resources are used efficiently. 0.90 16. The organization of thematic topics is appropriate for the good
understanding of the contents and easy location of the desired topics. 0.93
for odd items, the score contribution is the scale position minus 1, and for even items, the contribution is 5 minus the scale position. The sum of the scores is multiplied by 2.5 to obtain the overall value of the SUS, and the total score ranges from 0 to 100% (Brooke, 1996).
Statistical Analysis
Data were analyzed using SPSS version 20.0. Descriptive statistics, including frequency distribution, mean, standard deviation (SD) and
range, were used to describe the participants' profiles. The Shapiro– Wilk test was used to check for normality between the values. The groups were compared using Student'sttest, the Mann–WhitneyU test, Fisher's exact test and the Chi-square test. Odds ratios (OR) and their confidence intervals (CI) were used to evaluate the differences be-tween the two groups in terms of students' performance. Wilcoxon's signed rank test was used to compare students' grades before and after the experiment. All tests were conducted using a 0.05 significance level.
Fig. 1.Example of the features of the Wise Nurse program.
Results
Sample Characteristics
Participants' age ranged from 20 to 26, with a median age of 22 (SD = 1.8). Most participants were female. The two groups were homo-geneous in terms of gender, age, course level, and knowledge level, as measured during the pretest. Characteristic variables of the two partic-ipant groups are listed inTable 2.
Students' Performances Before, During, and After the Experiment
Students' grades in the pre- and the post-test were measured. As shown inTable 3, no significant difference was found between the two groups in both the pre- and the post-test (pN0.05 by Fisher's exact test/Chi-square test and Mann–WhitneyUtest). Homogeneity be-tween the two groups reduced the risk of bias during the experiment.
During the clinical case resolution, students from the entire sample better identified RF and NDx (hit rate: 79.3% and 79.4%, respectively) than they identified DC (hit rate: 73.2%). Students' best performance oc-curred with the clinical cases belonging to the nutrition and perception/ cognition domains (hit rate: 87.2% in both domains), and the worst performance was found in the clinical cases belonging to the growth/ development domain (hit rate: 51.7%).
A significant difference was found between the two groups in solv-ing the clinical cases in only two situations: 1) students in the experi-mental group were 10 times more likely to hit the inference related to one DC of the clinical case regarding the nutrition domain (p = 0.033; OR = 10.417; CI = 1.09–100.0); and 2) the chance of students in the ex-perimental group hitting the inference related to one DC of the clinical case regarding the elimination and exchange domain was decreased by 78% (p = 0.040; OR = 0.214; CI = 0.047–0.984).
As shown inTable 3, students' pre- and post-test grades were also compared in each group. Both groups showed an improved perfor-mance based on their grades and hit rates before and after the experi-ment (pb0.05 by Wilcoxon's signed rank test).
In both groups, 3 students had decreased performance (post-test gradebpretest grade) and 10 students had increased performance
after the experiment. Seven students in the experimental group and 4 students in the comparison group maintained their grades after the experiment.
Usability Testing
As shown inTable 4, from the students' perspective, the usability of Wise Nurse can be considered good (Final SUS score: 83.75). Students assigned values nearfive to the SUS odd items and values near one to the even items.
Participants found that the system was easy to use; however, the results revealed some limitations, which are outlined below:
• The program does not have the capability to e-mail the students' results. The results can only be stored in the computer hard drive in PDF format.
• The program can only provide 13 clinical case studies. Once the student has solved these 13 clinical cases and accessed the correct answers, the system utility is substantially lowered.
• The 13 clinical cases do not include various difficulty levels.
• The user cannot stop the exercise, save data to a database or use it in a different session.
Discussion
The present study shows that Wise Nurse has some advantages and limitations. The software has successfully contributed to the
improvement of knowledge and the ability of student nurses to infer NANDA-I diagnoses.
The experiment also revealed that the students had more difficulty to correctly identify DC than RF or NDx. Lopes et al. found that 3rd and 4th year student nurses were better at identifying NDx than DC and RF when solving three clinical case studies (Lopes et al., 2013). An accu-rate diagnostic inference is directly related to the ability to identify good clinical indicators (i.e., signs and symptoms that are strongly related to a specific diagnosis) (Lunney, 1990). Ourfindings show that student nurses have difficulty identifying relevant clues in solving clinical cases, which requires clinical experience and taxonomy-related knowledge.
It was noted that the hit rates were higher in the clinical case belong-ing to the nutrition and perception/cognition domains. Previous studies also found that student nurses' performed better in clinical cases regarding Nutrition diagnoses compared with diagnoses in other NANDA-I domains (Türk et al., 2013; Yont et al., 2009).
However, we observed poor student performance in the clinical case belonging to the growth/development domain. This may be because the clinical case addressed a risk diagnosis. Risk diagnoses are supported by risk factors that contribute to increased vulnerability, and no defining characteristics are present because the patient has not yet developed them. It is evident that in their eagerness to identify a real problem, many students selected incorrect choices regarding the presence of DC in this clinical case, although the description of the clinical case did not indicate any DC. In another study that intended to determine 1st year student nurses' diagnostic ability, risk diagnosis was less favored by the students than the problem-focused diagnosis (Yont et al., 2009). It is demanding to develop adequate multiple-choice test items, and distractors are the most difficult part of the test item to write. The key to developing adequate distractors is plausibility—the idea that an item would be correctly answered by those with a high degree of knowledge and incorrectly answered by those with a low degree of knowledge (Haladyna, 2004).
The current study found that students' performance in the experi-mental and comparison groups was similar, except in two clinical cases. Although the reasons for thisfinding were not identified, it is known that the ability to correctly infer nursing diagnoses is related to various cognitive factors that are complex and difficult to explain. Ac-cessibility and structuring of prior knowledge are related to high quality diagnostic reasoning. However, it is almost impossible to measure how accessible or well-structured thought processes are used by an individ-ual (Cholowski and Chan, 2004).
A majorfinding was the significant improvement in the students' performance in both groups after the experiment involving the clinical cases (Wilcoxon's signed rank test: p = 0.022). Thisfinding indicates the system's efficacy in improving nurse students' diagnostic reasoning and NDx-related knowledge at a similar level as using printed clinical cases.
Investments in educational computer technology must be based on ideas that improve students' performance and learning. The exercises provided by the Wise Nurse interface stimulated the metacognition skills of the students (knowledge that an individual has over his own cognition) and provided opportunities for them to consider the steps that led to particular diagnostic inferences. Feedback from responses given to the students may also have contributed to the reflection on their own performance, motivating students to pursue increased ability to correctly identify NDx.
Our students are part of what is called“net generation”or“digital native”, groups that did not have to learn how to use electronics and consider computer technology to be something natural (Barak et al., 2014). Thus, learning can be optimized with the use of information technology in teaching and learning in nursing as well as in other areas. In this study, the usability of Wise Nurse was greatly based on the subjective assessment of the participants using the SUS scale. SUS scores above 70% indicate adequate usability and high probability of acceptance of the system (Bangor et al., 2009).
Information technology integration in the curriculum requires significant time and support from both users and administrators (Kowitlawakul et al., 2014). After software testing, some limitations of Wise Nurse were identified and should be addressed in further itera-tions. The development of thefirst version of Wise Nurse is thefirst step before integrating this tool into curricula.
Difficulties in the diagnostic reasoning process are common, both during training and in the professional practice of nurses, due to the
complexity of this task. The accurate identification of a nursing diagno-sis is related to the ability to make responsible clinical judgements, and many students feel unprepared for this responsibility. In a study con-ducted to assess student nurses' knowledge and use of NANDA-I, it was found that the lack of knowledge and support are the main barriers Fig. 4.Example of answers discussion feature of the Wise Nurse program.
Table 2
Descriptive statistics for sample characteristics (N = 37).
Variable Participant (N = 37)
Experimental group (N = 20)
Comparison group (N = 17)
p-Value
Mean or Num (SD or %)
Mean or Num (SD or %)
Mean or Num (SD or %)
Age 0.95
Mean (SD) 22 (1.8) 22 (1.9) 22 (1.6)
Maximum 26 26 25
Minimum 20 20 20
Gender 0.61
Male 4 (10.8) 3 (15) 1 (5.9)
Female 33 (89.2) 17 (85) 16 (94.1)
Nursing course level 0.99
2nd year 7 (18.9) 4 (20) 3 (17.6)
3rd year 13 (35.2) 7 (35) 6 (35.3) 4th year 17 (45.9) 9 (45) 8 (47.1) Pretest of the outcome
variables
0.54
Knowledge 8.2 (1.4) 8.3 (1.3) 7.9 (1.4)
to the use of this classification (Ogunfowokan et al., 2013). Strategies to guide students' focus as they are learning about NDx can produce posi-tive curricular results (Carpenito‐Moyet, 2010). It is believed that Wise Nurse can meet these demands by supporting the acquisition of new knowledge of NDx and the development of diagnostic ability.
Study Limitations
The main limitation of this study is that it was developed at one university in northeastern Brazil, which limits the generalizability of the results. Moreover, trialing a tool that has been internally developed within a particular institution can be considered a limitation of this study. An English version of the system is under development and is expected to be applied soon in the United States, which may increase thefindings' representativeness and may contribute to a wider geo-graphical distribution of this tool in the future. Furthermore, although improvement in the students' performance has been achieved, a longi-tudinal study is needed to measure long-term knowledge effects of using the software.
Another limitation of the present study is that we did not measure factors like students' grade point average, intelligence quotient, or clinical experience because of time constraints. These variables could
potentially affect the results. However, a randomized recruitment strategy was adopted to minimize selection bias.
To make Wise Nurse more consistent with the educational needs observed in teaching NDx and to overcome the system limitations, improvements need to be made in further iterations.
Conclusion
Wise Nurse is an innovative teaching tool that can contribute to improving student nurses' diagnostic ability and facilitate the use of the NANDA-I taxonomy. Although Wise Nurse has some limitations, its use has advantages, such as simplicity, time efficiency, adequate usability perceived by users and content coverage, considering that it covers all NANDA-I domains. The program's limitations need to be addressed in future studies, and a software update is needed to refine the tool.
Acknowledgments
We thankfinancial support from the Coordination for the Improve-ment of Higher Education Personnel—CAPES (Grant No. BEX14504/13-8).
References
Alfaro-Lefebre, R., 2004.Critical Thinking and Clinical Judgment: A Practical Approach to Outcome-Focused Thinking. 4th ed. Elsevier Saunders, St Louis, MO.
Bangor, A., Kortum, P., Miller, J., 2009.Determining what individual SUS scores mean: adding an adjective rating scale. J. Usability Stud. 4, 114–123.
Barak, M., Morad, S., Ragonis, N., 2014.Students' innovative thinking and their percep-tions about the ideal learning environment. The 8th International Conference on Knowledge Management in Organizations. Springer, Netherlands, pp. 111–125. Bell, B.S., Federman, J.E., 2013.E-learning in postsecondary education. Future Child. 23
(1), 165–185.
Bloomfield, J., Roberts, J., While, A., 2010.The effect of computer-assisted learning versus conventional teaching methods on the acquisition and retention of handwashing the-ory and skills in pre-qualification nursing students: a randomised controlled trial. Int. J. Nurs. Stud. 47 (3), 287–294.
Brooke, J., 1996.SUS: a quick and dirty usability scale. Usability Eval. Ind. 189 (194), 4–7. Carpenito‐Moyet, L.J., 2010.Invited paper: teaching nursing diagnosis to increase
utiliza-tion after graduautiliza-tion. Int. J. Nurs. Terminol. Classif. 21 (3), 124–133.
Cholowski, K.M., Chan, L.K.S., 2004.Cognitive factors in student nurses' clinical problem solving. J. Eval. Clin. Pract. 10 (1), 85–95.
Cruz, D.A.L.M., Pimenta, C.A.M., 2005.Evidence-based practice applied to diagnostic reasoning. Rev. Lat. Am. Enfermagem 13 (3), 415–422.
De Caro, W., Maruccim, A.R., Giordani, M., Sansoni, J., 2014.E-learning and university nursing education: an overview of reviews. Prof. Inferm. 67 (2), 107–116. Freitas, L.V., Teles, L.M.R., Lima, T.M., Vieira, N.F.C., Barbosa, R.C.M., Pinheiro, A.K.B.,
Damasceno, A.K.D.C., 2012.Physical examination during prenatal care: construction and validation of educational hypermedia for nursing. Acta Paul. Enferm. 25 (4), 581–588.
Góes, F.S.N., Fonseca, L.M.M., Furtado, M.C.C., Leite, A.M., Scochi, C.G.S., 2011.Evaluation of the virtual learning object“diagnostic reasoning in nursing applied to preterm newborns”. Rev. Lat. Am. Enfermagem 19 (4), 894–901.
Table 3
Students' hit rates and grades in the pre-test and post-test (N = 37).
Variables Experimental Comparision p Value⁎
I. Pre-test
Item 1 14 13 0.725
Item 2 15 08 0.081
Item 3 19 15 0.584
Item 4 17 11 0.251
Item 5 11 07 0.402
Item 6 17 16 0.609
Item 7 13 10 0.699
Item 8 13 13 0.447
Item 9 17 17 0.234
Item 10 19 17 1.000
Item 11 18 16 1.000
Item 12 20 16 0.459
Item 13 20 16 0.459
Item 14 17 15 1.000
Item 15 18 13 0.383
Item 16 18 14 0.644
Item 17 19 13 0.159
II. Post-test
Item 1 14 09 0.286
Item 2 20 17 –
Item 3 19 15 0.584
Item 4 16 14 1.000
Item 5 19 14 0.315
Item 6 17 15 1.000
Item 7 18 15 1.000
Item 8 20 17 –
Item 9 20 17 –
Item 10 20 17 –
Item 11 20 17 –
Item 12 20 17 –
Item 13 18 17 0.489
Item 14 16 15 0.667
Item 15 16 16 0.348
Item 16 19 14 0.315
Item 17 16 10 0.160
Pre-test hit ratea 15 13.47 0.218c
Post-test hit ratea 16 15 0.542c
Pre-test gradea 8.82 7.92 0.218c
Post-test gradea 9.41 8.82 0.542c
p Valueb 0.022 0.022 –
⁎ Fisher's exact test/Chi-square. Notice that p value cannot be calculated for 100% cor-rect answers.
aArithmetic mean.
b Wilcoxon test for intra-group comparisons. c Mann–WhitneyUtest.
Table 4
System Usability Scale responses (N = 20).
Scale item Mean SD Score
range
1. I think that I would like to use this system frequently. 4 1.02 2–5 2. I found the system unnecessarily complex. 2 0.51 1–2 3. I thought the system was easy to use. 4.5 0.61 3–5 4. I think that I would need the support of a technical
person to be able to use this system.
1 0.69 1–3
5. I found the various functions in this system were well integrated.
4 0.91 1–5
6. I thought there was too much inconsistency in this system.
1.5 0.98 1–5
7. I would imagine that most people would learn to use this system very quickly.
4.5 0.81 2–5
8. I found the system very cumbersome to use. 1.5 0.97 1–4 9. I felt very confident using the system. 4 0.72 3–5 10. I needed to learn a lot of things before I could get
going with this system.
1 0.83 1–4
Haladyna, T.M., 2004.Developing and Validating Multiple-Choice Test Items. 3rd ed. Lawrence Erlbaum Associates, Mahwah.
Herdman, T.H., a. What is Nursing Diagnosis and why Should I Care? Retrieved from http://www.nanda.org/What-is-Nursing-Diagnosis-And-Why-Should-I-Care_b_2. html(accessed 16 October 2015).
Herdman, T.H., Kamitsuru, S., 2014.NANDA International Nursing Diagnoses: Definitions & Classification, 2015–2017. Wiley-Blackwell, Oxford.
Jawaid, M., Moosa, F.A., Jaleel, F., Ashraf, J., 2014.Computer based assessment (CBA): perception of residents at Dow University of Health Sciences. Pak. J. Med. Sci. 30 (4), 688–691.
Koch, J., Andrew, S., Salamonson, Y., Everett, B., Davidson, P.M., 2010.Nursing students' perception of a web-based intervention to support learning. Nurse Educ. Today 30 (6), 584–590.
Kowitlawakul, Y., Wang, L., Chan, S.W., 2013.Development of the electronic health records for nursing education (EHRNE) software program. Nurse Educ. Today 33 (12), 1529–1535.
Kowitlawakul, Y., Chan, S.W., Wang, L., Wang, W., 2014.Exploring faculty perceptions to-wards electronic health records for nursing education. Int. Nurs. Rev. 61 (4), 499–506. Lopes, M.V.O., Araujo, T.L., 2005.Software educativo en la enseñanza de enfermería.
Metas Enferm. 8 (2), 23–26.
Lopes, M.H.B.M., Jensen, R., Cruz, D.A.L.M., Matos, F.G.O.A., Silveira, R.S.P., Ortega, N.R.S., 2013.Application of a model based on fuzzy logic for evaluating nursing diagnostic accuracy of students. Int. J. Med. Inform. 82 (9), 875–881.
Lunney, M., 1990.Accuracy of nursing diagnoses: concept development. Int. J. Nurs. Terminol. Classif. 1 (1), 12–17.
Martins, A.C.F., 2011.Development and Evaluation of a Software Control and Decision Support Calls for Differential Diagnosis Lower Urinary Tract Dysfunction, Based on Fuzzy Logic. Universidade Estadual de Campinas, Campinas, SP, Brazil.
Ogunfowokan, A.A., Oluwatosin, A.O., Olajubu, A.O., Alao, O.A., Faremi, A.F., 2013.Student nurses' perceived use of NANDA nursing diagnoses in the community setting. Int. J. Nurs. Knowl. 24 (1), 37–43.
Rogan, F., Miguel, C.S., 2013.Improving clinical communication of students with English as a second language (ESL) using online technology: a small scale evaluation study. Nurse Educ. Pract. 13 (5), 400–406.
Schulz, K.F., Altman, D.G., Moher, D., 2010.CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMC Med. 8 (1), 18.
Türk, G., Tuğrul, E.,Şahbaz, M., 2013.Determination of nursing diagnoses used by stu-dents in thefirst clinical practice. Int. J. Nurs. Knowl. 24 (3), 129–133.
