Translation and validation of the Manchester Clinical Supervision Scale©: Effective clinical supervision evaluation

(1)

Procedia - Social and Behavioral Sciences 29 ( 2011 ) 51 – 56

International Conference on Education and Educational Psychology (ICEEPSY 2011)

Translation and validation of the Manchester Clinical

Supervision Scale

©

: effective clinical supervision evaluation

Sandra Sílvia Silva Monteiro Santos Cruz *

Escola Superior de Enfermagem do Porto, Rua Dr. António Bernardino de Almeida s/n, 4200-072, Porto, Portugal

Abstract

In Portugal, we are at the beginning of clinical supervision in nursing. We carried out a research to translate and validate the Manchester Clinical Supervision Scale© (MCSS) into Portuguese language from Portugal. Thus, we can assess the clinical supervision process. We applied the methods of translation and back – translation and experts analyzed translations. MCSS and the back translation were compared by collaborative parties. An empirical study using a test – retest design was made to estimate cross-cultural relevance. Cronbach’s alpha value for the total score was 0,923 in both periods. The Portuguese version is culturally acceptable and consistent with the original.

Keywords: Translation; Validation of Translation; Clinical Supervision; Nursing; Manchester Clinical Supervision Scale©_.

Introduction

Nurses need to have a great flexibility and be prepared to complex and demanding clinical situations because as Buerhaus et al (2007) stated taking care of hospitalized patients is difficult and requires several efforts from the health team. Clinical supervision is essential for the quality of the nursing care, it is a mechanism to support nurses in their professional practice (Clough, 2003; Walsh et al, 2003; Edwards et al, 2005). In Portugal, we are at the very beginning of clinical supervision in nursing so we need an accurate instrument to assess its process. “A suitable instrument, and the knowledge it would generate, could initiate development and improve the quality of clinical supervision in practice” (Hyrkäs, Appelqvist-Schmidlechner & Paunonen-Ilmonen, 2003, p.358).

This paper aims to describe the research focused on the translation and validation of the Manchester Clinical Supervision Scale© from English into Portuguese language from Portugal.

This article is divided into four main sections: the first one is related to the original Manchester Clinical Supervision Scale©; in the second, the methodology and the study design are explained; in the third one and the others, we presented the results followed by the discussion and the conclusion of the article.

* Sandra Cruz Tel.: +351225073500 Email address: [email protected]

Selection and/or peer-review under responsibility of Dr Zafer Bekirogullari. doi: 10.1016/j.sbspro.2011.11.205

(2)

1. The Manchester Clinical Supervision Scale©

Effective evaluation of clinical supervision is a tool for excellence in nursing. The Manchester Clinical Supervision Scale© evaluates the quality and effectiveness of the supervision provided and the supervisees’ opinion of the effect of clinical supervision in their professional development, improvement in skills, time for reflection and the quality of the supervisory relationship.

The scale is a 36 – item questionnaire with a Likert – type scale (1-5) ranging between “strongly disagree” to “strongly agree”. It is comprised by seven sub-scales: trust/rapport, supervisor advice/support, improved care/skills, importance /value of clinical supervision, finding time, personal issues and reflection.

The scale has been developed in the United Kingdom, tested in Australia (Hyrkäs, Appelqvist-Schmidlechner & Paunonen-Ilmonen, 2003) and used as an “(…) outcome measure in more than 80 clinical supervision evaluation studies, in 12 countries worldwide, and has been translated into four languages other than English” (White & Winstanley, 2010, p.153). The Manchester Clinical Supervision Scale© was designed to measure these aspects of the clinical supervision process.

2. Methodology and study design

The translation process was conducted by applying the methods of translation, back – translation and comparison proposed by Fortin (2000) followed by the empirical study. “It is important that the translator is fluent in both the source language and target language and is knowledgeable about the both cultures” (Chen & Boore, 2009, p.234). A professional translator translated the Manchester Clinical Supervision Scale© into Portuguese followed by a blind back – translation into English by another professional translator. The back translator also needs to be fluent in both languages and know the cultures involved. We tested the quality of translation; established semantic equivalence of the Manchester Clinical Supervision Scale© translated into Portuguese and estimated the cross-cultural relevance of the scale.

The translations were compared and analyzed by three experts in clinical supervision in nursing (two coordinators of the Post-Graduation in Clinical Supervision in Nursing at the Escola Superior de Enfermagem do Porto (Oporto Nursing School) and the coordinator of the Escola Superior de Enfermagem do Porto Research Unit. The scale and its back translation were compared by the researcher, the experts and the translator and as well as the author of the scale.

An empirical study using a test – retest design was made to estimate cross-cultural relevance. The study was placed within the paradigm of quantitative research and grew in several care units at the Unidade Local de Saúde de Matosinhos EPE (Matosinhos Local Health Unit EPE) between July - August 2008. This Local Health Unit is composed by a hospital and several health care centers. Nurses answered the questionnaire twice (the second time was a week after the first and the nurses were not allowed to look at the first questionnaire when they were filling the second one). A total of 230 sets of paired questionnaires were received from respondents. Statistical Package for Social Sciences version 17.0 was used for data analysis.

We requested to the original author the authorization for the translation and validation of the Manchester Clinical Supervision Scale©. For the empirical study, we obtained permission from the Unidade Local de Saúde de Matosinhos EPE. The questionnaire with the scale had an introductory part where we explained the study and the ethical issues we were going to respect like the anonymity and confidentiality of the collected information. We also outlined the voluntary nature of the nurses’ participation.

3. Results

We achieved the semantic equivalence of the translated version and the original Manchester Clinical Supervision Scale© through the back translation. The translation process always requires professional translators who must know the version of the original language of the built instrument because there are several versions of the same language (Hill & Hill, 2005).

(3)

The face validity was tested after the translation process by the three experts in clinical supervision in nursing. They compared and analyzed several times the Portuguese version and the original scale. They were asked to assess the content of the scale and to examine the words and sentences to see if they were equivalents to the ones in the English version and if they were easily understood by the Portuguese nurses. When the experts reached to a consensus about the Portuguese version of the Manchester Clinical Supervision Scale©, it was pre-tested and then we finally conducted an empirical study using a test-retest design.

A total of 230 paired questionnaires were obtained. The response rate was 57% (n=401 nurses). We have had a convenience sample from the Local Health Unit which the relevant socio demographic data are shown in table 1.

Table 1 – Socio-demographic data from the paired sample (n=230)

Variable 1st Data n=230 % 2nd Data n=230 % Sex Female 171 77 171 77 Male 52 23 52 23 Professional Category Nurse 194 84 193 83 Specialized Nurse 36 16 37 17 Care Unit Health Center 29 13 29 13 Hospital 201 87 201 87

In this sample, the majority of the respondents were female, nurses and they worked in the hospital setting. Appropriated statistical tests were used to find the equivalence between the Portuguese version and the original. Table 2 – Results of the data collected with the Portuguese version of the Manchester Clinical Supervision Scale©

1st Data 2nd Data 1st Data 2nd Data Sub-scales n Mean (Sd)** _{Mean (Sd)}** _ICC***

Trust/rapport 219 26,49 (4,23) 26,10 (4,33) 0,833 0,819 0,843 Supervisor advice/support 220 22,47 (4,01) 22,50 (3,91) 0,852 0,893 0,910 Improve care/skils 223 24,29 (4,60) 24,45 (4,33) 0,843 0,865 0,864 Importance/value of CS* 219 23,82 (3,09) 23,99 (2,95) 0,706 0,648 0,672 Finding time 223 12,30 (2,72) 12,83 (2,65) 0,705 0,653 0,683 Personal Issues 221 8,02 (2,05) 8,24 (1,97) 0,643 0,413 0,453 Reflection 224 11,71 (2,01) 11,61 (1,95) 0,709 0,820 0,790 Total evaluation score 212 129,11 (16,47) 129,72 (15,72) 0,885 0,923 0,923

(4)

The construct validity was tested by examining the internal consistency of the items. The reliability coefficient for the total translated scale was 0,923 in both periods and for each sub-scale is shown in table 2.

The lowest coefficient was 0,413 in the first data and 0,453 in the second one for the sub-scale “personal issues” and the highest coefficient was 0,893 in the first data and 0,910 in the second data for the sub-scale “supervisor advice/support”.

We calculated the intra-class correlation coefficient and it ranges between 0,643 in the sub-scale “personal issues” and 0,852 in the sub-scale “supervisor advice/support” (table 2).

To estimate the relevance of each item, we calculated the kappa statistics (table 3).

Table 3 – Results of the strength agreement in the data collected with the Portuguese version of the Manchester Clinical Supervision Scale©

Variable WK* _(Se)** _{95% CI}*** Item 1 0,528 0,046 0,438; 1,386 Item 2 0,519 0,046 0,429; 1,360 Item 3 0,465 0,051 0,365; 1,180 Item 4 0,469 0,048 0,375; 1,204 Item 5 0,452 0,050 0,354; 1,146 Item 6 0,403 0,051 0,303; 0,997 Item 7 0,432 0,050 0,334; 1,087 Item 8 0,585 0,044 0,499; 1,563 Item 9 0,511 0,046 0,421; 1,336 Item 10 0,525 0,047 0,433; 1,373 Item 11 0,453 0,056 0,343; 1,126 Item 12 0,557 0,057 0,445; 1,430 Item 13 0,569 0,055 0,461; 1,473 Item 14 0,551 0,045 0,463; 1,458 Item 15 0,523 0,046 0,433; 1,371 Item 16 0,621 0,046 0,531; 1,661 Item 17 0,486 0,050 0,388; 1,246 Item 18 0,526 0,049 0,430; 1,369 Item 19 0,501 0,049 0,405; 1,295 Item 20 0,545 0,052 0,443; 1,413 Item 21 0,534 0,054 0,428; 1,373 Item 22 0,644 0,045 0,556; 1,733 Item 23 0,634 0,042 0,552; 1,715 Item 24 0,643 0,048 0,549; 1,719 Item 25 0,446 0,050 0,348; 1,128 Item 26 0,542 0,048 0,448; 1,420 Item 27 0,586 0,046 0,496; 1,558 Item 28 0,445 0,053 0,341; 1,114 Item 29 0,594 0,049 0,498; 1,570 Item 30 0,362 0,050 0,264; 0,879 Item 31 0,627 0,044 0,541; 1,687 Item 32 0,568 0,047 0,476; 1,501 Item 33 0,584 0,048 0,490; 1,544 Item 34 0,644 0,049 0,548; 1,718 Item 35 0,620 0,048 0,526; 1,651 Item 36 0,630 0,049 0,534; 1,677

(5)

A value of 1 indicates perfect agreement although in our sample the strength of agreement (Altman, 1991) ranges between 0,362 in the item 30 (fair agreement) and 0,644 in the items 22 and 34 (good agreement).

4. Discussion

This research focused on the translation and validation of the Manchester Clinical Supervision Scale© from English into the Portuguese language from Portugal. The translation process is much more difficult than it seems (Hill & Hill, 2005).

We applied to the methods of translation and back-translation; the versions were compared by the researcher, the experts, the professional translators and by the author of the scale. We reached to the final version of the translated instrument which was approved by the experts. We pre-tested it and then we carried out an empirical study using a test-retest design to estimate cross-cultural relevance.

In our sample (n=230), Cronbach’s alpha value for the total score was 0,923 in both periods and in the sub-scales it was 0,413 (“personal issues”) – 0,893 (“supervisor advice/support”) the first time, and 0,453 (“personal issues”) – 0,910 (“supervisor advice/support”) the second time. According to Hyrkäs, Appelqvist-Schmidlechner & Paunonen-Ilmonen (2003) in the Finnish version, Cronbach’s alpha value for the total score was 0, 9227 and the lowest Cronbach’s alpha value found was 0, 6393 in the same sub-scale (“personal issues”).

In their professional practice, nurses don’t usually discuss personal issues and they try to focus their attention and their expertise on issues related to their professional activities.

An interesting finding is that in the Portuguese version the highest Cronbach’s alpha value found was for the sub-scale “supervisor advice/support” while in the original one and in the Finnish version it was in the sub-sub-scale “improve care/skills” (0, 8838) (Hyrkäs, Appelqvist-Schmidlechner & Paunonen-Ilmonen, 2003).

The major target of the clinical supervision in nursing is the supervisee’s needs and in our sample the supervisors work in the same care unit as the supervisees. Therefore, the contact and the guidance they provide was very close and they make efforts to enable the supervisees to understand and to have proper emotions and feelings in the clinical practice and this could be a justification for the difference.

Winstanley (2000) reported that: “A high score for any sub-scale reflects a high degree of effectiveness for that aspect of the clinical supervision process. A high total evaluation score reflects a high level of overall effectiveness of the clinical supervision process” (p. 9).

Conclusion

The translation and validation of an instrument into another language is a demanding methodological work and it comprises several phases.

The Portuguese version of the Manchester Clinical Supervision Scale© is culturally acceptable and consistent with the original. Nevertheless, further studies should address other psychometrics characteristics of the Portuguese version.

This instrument evaluates effectively aspects of the clinical supervision process. Thus, supervisors and supervisees can adjust strategies to improve outcomes not only in clinical supervision in nursing, but also in the quality and safety of the care provided to patients.

Acknowledgments

We gratefully acknowledge the contribution and comments of Julie Winstanley PhD, MSc, BSc, CStat, CSci (Director, Osman Consulting Pty Ltd, Associate Professor in Biostatistics and Head of Research and Statistics, Melanoma Institute Australia, University of Sydney, Australia) in the several phases of the study.

We also acknowledge the Unidade Local de Saúde de Matosinhos EPE to allow this research and to collaborate with us.

(6)

References

Altman, D. (1991). Practical statistics for medical research. London: Chapman and Hall

Buerhaus, P. et al (2007). Impact of the nurse shortage on hospital patient care: comparative perspectives. Health

Affairs, 26, pp. 853-862.

Chen, H-Y & Boore, J. (2009). Translation and back-translation in qualitative nursing research: methodological review. Journal of Clinical Nursing, 19, pp. 234-239.

Clough, A. (2003). Clinical supervision in primary care. Primary Health Care, 13, pp. 15-18.

Edwards, D. et al (2005). Factors influencing the effectiveness of clinical supervision. Journal of Psychiatric and

Mental Health Nursing, 12, pp. 405-414.

Fortin, M-F. (2000). O processo de investigação: da concepção à realização. Loures: Lusociência. Hill, M. & Hill A. (2005). Investigação por questionário. Lisboa: Edições Sílabo.

Hyrkäs, K., Appelqvist-Schmidlechner, K. & Paunonen-Ilmonen, M. (2003). Translating and validating the Finnish version of the Manchester Clinical Supervision Scale. Scand J Caring Sci, 17, pp. 358-364.

Walsh, K. et al (2003). Development of a group model of clinical supervision to meet the needs of a community mental health nursing team. International Journal of Nursing Practice, 9, pp. 33-39.

Winstanley, J. (2000). Manchester Clinical Supervision Scale user guide. United Kingdom: Julie Winstanley. White, E. & Winstanley, J. (2010). A randomised controlled trial of clinical supervision: selected findings from a

novel Australian attempt to establish the evidence base for causal relationships with quality of care and patient outcomes, as an informed contribution to mental health nursing practice development. Journal of Research in