Construction of a tool to measure perceptions about the use of the World Health Organization Safe Surgery Checklist Program

REVISTA

BRASILEIRA

DE

ANESTESIOLOGIA

www.sba.com.br

SCIENTIFIC

ARTICLE

Construction

of

a

tool

to

measure

perceptions

about

the

use

of

the

World

Health

Organization

Safe

Surgery

Checklist

Program

Luis

Antonio

dos

Santos

Diego

_,

_Fabiane

_Cardia

_Salman

_,

_João

_Henrique

_Silva

_,

Julio

Cezar

Brandão

_,

_Getúlio

_de

_Oliveira

_Filho

_,

Antonio

Fernando

Carneiro

_,

_Airton

_Bagatini

_,

_José

_Mariano

_de

_Moraes

a_{SociedadeBrasileiradeAnestesiologia(SBA),RiodeJaneiro,RJ,Brazil}

b_{FaculdadedeMedicina,UniversidadeFederalFluminense(UFF),Niterói,RJ,Brazil} c_{Fundac¸ãoOswaldoCruz,RiodeJaneiro,RJ,Brazil}

d_{HospitalMoinhosdeVento,PortoAlegre,RS,Brazil}

e_{UniversidadeFederaldeSergipe(UFS),SãoCristóvão,SE,Brazil}

f_{TranslationalMedicine,UniversidadeFederaldeSãoPaulo(Unifesp),SãoPaulo,SP,Brazil} g_{MassachusettsGeneralHospital,Harvard,USA}

h_{UniversidadeFederaldeSantaCatarina(UFSC),Florianópolis,SC,Brazil}

Received2September2014;accepted11November2014 Availableonline1May2016

KEYWORDS Tool; Checklist; Safesurgery; Adverseevents; WorldHealth Organization Abstract

Background: The World Health Organization (WHO) hasrecommended greater attention to

patientsafety,particularlyregardingpreventableadverseevents.TheSafeSurgerySavesLives

(CSSV)programwasreleasedrecommendingtheapplicationofasurgicalchecklistforitemson

thesafetyofprocedures.Thechecklistimplementationreducedthehospitalmortalityratein

thefirst30days.InBrazil,wefoundnostudiesofanesthesiologists’adherencetothepractice

ofthechecklist.

Objective: Themainobjectivewastodevelopatooltomeasuretheattitudeof

anesthesiolo-gistsandresidentsregardingtheuseofchecklistintheperioperativeperiod.

Method: This was across-sectional study performedduring the59th CBA inBH/MG, whose

participants wereenrolledphysicianswho respondedtothequestionnairewith quantitative

epidemiologicalapproach.

Results:Fromthesampleof459participantswhoansweredthequestionnaire,55%weremale,

44.2%under10yearsofpractice,and15.5%withover30yearsofmedicalschoolcompletion.

∗_{Correspondingauthor.}

E-mail:juliobrand@hotmail.com(J.C.Brandão). http://dx.doi.org/10.1016/j.bjane.2014.11.011

0104-0014/©2015SociedadeBrasileiradeAnestesiologia.PublishedbyElsevierEditoraLtda.ThisisanopenaccessarticleundertheCC BY-NC-NDlicense(http://creativecommons.org/licenses/by-nc-nd/4.0/).

Sevenitemswith78%reliabilitycoefficientwereselected.Therewasastatisticallysignificant

differencebetweenthegroupsofanesthesiologistswhoreportedusingtheinstrumentinless

ormorethan70%ofpatients,indicatingthattheattitudequestionnairediscriminatesbetween

thesetwogroupsofprofessionals.

Conclusions:Thesevenitemsquestionnaireshowedadequateinternalconsistencyanda

well-definedfactorstructure,andcanbeusedasatooltomeasuretheanesthesiologists’perceptions

aboutthechecklistusefulnessandapplicability.

©2015SociedadeBrasileiradeAnestesiologia.PublishedbyElsevierEditoraLtda.Thisisan

openaccessarticleundertheCCBY-NC-NDlicense(

http://creativecommons.org/licenses/by-nc-nd/4.0/). PALAVRAS-CHAVE Ferramenta; Checklist; Cirurgiasegura; Eventosadversos; Organizac¸ãoMundial daSaúde

Construc¸ãodeumaferramentaparamedidadepercepc¸õessobreousodochecklist doProgramadeCirurgiaSeguradaOrganizac¸ãoMundialdaSaúde

Resumo

Introduc¸ão:AOrganizac¸ãoMundialdaSaúde(OMS)temrecomendadoumamaioratenc¸ãocom

aseguranc¸adopaciente,maisespecificamenteemrelac¸ãoaoseventosadversosevitáveis.Foi

lanc¸adooprograma‘‘CirurgiaSeguraSalvaVidas(CSSV)’’,querecomendaaaplicac¸ãodalista

deverificac¸ãocirúrgica(checklist)para aconferênciadeitensrelacionadosàseguranc¸ado

procedimento.A implantac¸ãodochecklistreduziuamortalidadehospitalarnosprimeiros30

dias.NoBrasil,nãoforamidentificadosestudossobreadesãodosanestesiologistasàpráticado

checklist.

Objetivo:Desenvolvimentodeumaferramentaparamensurac¸ãodaatitudedos

anestesiologis-taseresidentesemrelac¸ãoaousodochecklistnoperíodoperioperatório.

Método: Estudotransversalfeitoduranteo59◦ CongressoBrasileirodeAnestesiologia(CBA),

em BeloHorizonte (MG),cujosparticipantes forammédicos inscritosequeresponderam ao

questionáriocomabordagemepidemiológicaquantitativa.

Resultados: A amostraconstou de459 participantesque responderamao questionário, 55%

dosexomasculino, 44,2%commenosde10 anose15,5%acimade30anosdeconclusãodo

cursomédico.Foramselecionadosseteitenscomcoeficientedeconfiabilidadede78%.Houve

diferenc¸aestatisticamentesignificativaentreosgruposdeanestesiologistasquereferiramusar

oinstrumentoemmenosoumaisde70%dospacientesassistidos.Issoindicaqueoquestionário

deatitudesdiscriminaentreessesdoisgruposdeprofissionais.

Conclusões:Oquestionáriodeseteitensmostrouadequadaconsistênciainternaeuma

estru-turafatorialbemdelimitada.Podeserusadocomoferramentaparamedidadaspercepc¸õesde

anestesiologistasquantoàutilidadeeaaplicabilidadedochecklist.

©2015SociedadeBrasileiradeAnestesiologia.PublicadoporElsevierEditoraLtda.Este ´eum

artigoOpen Accesssobumalicenc¸aCCBY-NC-ND(

http://creativecommons.org/licenses/by-nc-nd/4.0/).

Introduction

The 55th World Health Assembly, World Health Organiza-tion(WHO),throughtheResolution55.18hasrecommended toitsmemberstatesincreasedattentiontopatientsafety, particularly regarding preventable adverse events. So, in October 2005 it was introduced the first Global Patient SafetyChallengewiththetheme:CleanCareisSaferCare, whilethesecondGlobalChallengeaddressedthe fundamen-talsandpracticesofsurgicalsafetyfocusingonprevention ofsurgicalsiteinfections,safeanesthesia,surgicalteams, anduseofindicatorsforsurgicalcare.

In2009,theWHOlaunchedtheSafeSurgerySavesLives (SSSL)program,1_{whichrecommendstheapplicationofthe}

surgicalsafetychecklist,whichdeterminesthreebreaksin theperioperative periodfor checkingthe itemsregarding procedure security. The checklist and its implementation manualwere translatedintoPortuguese and publishedby theMinistryofHealth/ANVISA/PAHO(Fig.1).

Themethodchosen2_{fortheproject‘‘SafeSurgery’’was}

based on the tool created at the Johns Hopkins Medical Institutionforimprovedcommunicationamongprofessionals participatingintheoperatingroomteam.Themain objec-tive wasfor all the operation planning tobeshared and, thus,facilitatetheintegrationintheprocedure implemen-tation.

Thisstructuredcommunicationtool,inspiredbyaviation, fragmentscomplextasksinmorewatertightsteps,inorder toreducethechancesofforgettingakeyitemforthe qual-ityandsafetyofthewholecare.Checkingtheitemslistedin apreviouslymadechecklistspeedstheprocessand,atthe same time,creates barriers toany process failures.3 _One

of thedecisivefactorsfor successis theunderstanding of itsusefulnessbyallthosewhomakeuptheteamsinvolved and,perhaps,thisis thegreatestobstacletotheprogram implementation.The perceptionofallshouldbethatit is notacontrolmethod,butausefultoolforreducingflawsin theprocess,asshowninamulticenterstudycoordinatedby

Patient has confirmed

• Identity • Site • Procedure • Consent

Site marked/not applicable Anaesthesia safety check completed Pulse oximeter on patient and functioning Does patient have A:

Known allergy?

No Yes

Difficult airway/aspiration risk?

No

Yes, and equipment/assistance available

Risk of >500ml blood loss

(7ml/kg in children)?

No

Yes, and adequate intravenous access and fluids planned

Nurse verbally confirms with the team:

The name of the procedure recorded

That instrument, sponge and needle

counts are correct (or not

applicable)

How the specimen is labelled

(Including patient name)

Whether there are any equipment

problems to be addressed

Surgeon, anaesthesia professional and nurse review the key concerns for recovery and management of this patient

Sign in

Confirm all team members have introduced themselves by name and role

Surgeon, anaesthesia professional and nurse verbally confirm

• Patient • Site • Procedure

Anticipated critical events

Surgeon reviews: What are the

critical or unexpected steps, operative duration, anticipated blood loss?

Anaesthesia team reviews: Are there

any patient-specific concerns?

Nursing team reviews: Has sterility

(including indicator results) been confirmed? Are there equipment issues or any concerns?

Has antibiotic prophylaxis been given within the last 60 minutes?

Yes Not applicable

Is essential imaging displayed?

Yes Not applicable

Time out Sign out

Before induction of anesthesia Before skin incision Before patient leaves operating room

Surgical safety checklist (first edition)

This checklist is not intended to be comprehensive. Additions and modifications to fit local practice are encouraged.

Figure1 Surgicalsafetychecklist.

Haynesetal.4_{The useofchecklisthasbeen stimulatedby}

accreditingorganizationsoperatingin Brazil thatconsider itan instrumentof perioperativebestpractices,5 _{but one}

cannotsay thatit hasbecome awidespread andcommon practice.Newprocedureslikethisamonghealth profession-als arenotalwaysreceivedpositivelyat first,particularly whentheeffectivenessofitsresults,thoughperemptory,is noteasilydemonstratedintheshortterm.

Thus,adherencetochecklistdependsonhavingpositive attitudesandperceptionsregardingitsusefulnessand appli-cability,inserted,preferably,ininstitutionsthatemphasize safety culture, because it is believed to be an essential requirement to reduce the occurrence of adverse events affecting rates security and entail reduction of patients’ morbidityandmortality.6---12_{TheimplementationoftheSafe}

SurgeryChecklistProgramreducedhospitalmortalityinthe first30days.Althoughtheimpactontheoutcomewaslower than previously reported,the effectdepends crucially on theadherencetotheroutineuseofthechecklist.12

InBrazil,wefoundnostudiesofanesthesiologists’ adher-enceto theWHO ‘‘Surgical Safety Checklist’’norreports onadherence totypeof subgroupsof healthcare institu-tion,departmentofanesthesiologypracticeandprocedure complexity. Thus, assessing the attitude of profession-als regarding adherence to checklist is very important, particularly in the development of educational activities thatencouragethesafetyculture,asanesthesiologistsare present in all threestages of its implementation,aswell asnursingstaff.Thechecklistintroductioninseveral Euro-peancentersalsofacedbarriersandbiasesandledtosome

difficultiesinitsimplementation,butwiththesafetyculture inanesthesiatherewasabetterstandardofacceptanceand modificationof thestandard practiceof suchcentersand adequacyforthechecklistregularuse.12---14_Themain

objec-tiveofthisstudywasthedevelopmentofatooltomeasure theattitudeofanesthesiologistsandresidentsregardingthe useofthechecklistintheperioperativeperiod.

Method

Cross-sectional study performed during de 59th Brazilian AnesthesiologyCongressinBeloHorizonte(MG),whose par-ticipantswereenrolledanesthesiologistsandresidentswho went to the stand of the Brazilian Society of Anesthesi-ology (SBA) and voluntarily responded to the web-based self-administered electronic questionnaire with quantita-tiveepidemiologicalapproach.Thestudywasapprovedby theResearchEthicsCommitteeoftheNationalInstituteof Cardiology,andwritteninformedconsentwasobtainedfrom allparticipants----anonymitywasguaranteed.

The instrument design prioritized the inclusion of questions that could contribute to the understanding of perceptions and attitudes of the profession being stud-ied. Responses were obtained on a 5-point Likert scale7

(strongly disagree, partially disagree, neither agree nor disagree, partially agree,and strongly agree) tomeasure attitudesofrespondentsonvariousaspectsofthechecklist applicabilityandusefulness.Weacceptaspositiveanswers thosein which participants marked options 4 and 5 (par-tially agree/strongly agree) to the sentences that have

Table1 Meandifference between total andfactorial scores of questionnairebetween anesthesiologists groups using the

checklistinlessormorethan70%ofpatients.

Meandifference 95%MeandifferenceCI p

Inferior Superior

F1 _{−0.31 −0.42 −0.20} 0.00

F2 −0.15 −0.29 −0.02 0.03

Total _{−0.25 −0.35 −0.14} 0.00

been formulated in a positive way, or 1 and 2 (strongly disagree/partially disagree) to the questions formulated negatively.Inthisprocess,thescoresassignedtoitemswith negativeconnotations(1and2tothequestionsformulated positivelyand4and5tothoseformulatednegatively)were reversed,soastoproduceaninstrumentwithmean maxi-mumrangeequalto5points,representingthebestattitudes andperceptionsregardingthechecklist.

The questionnaire reliability was evaluated through internalconsistency,estimatedbytheCronbach’salpha reli-abilitycoefficient,relevantbecausethequestionnairewas appliedonceinthesample.Cronbach’salphavaluesrange from0 to1and thelowerlimitusuallyacceptedis 0.60.8

Afterselecting theitems usingthe correlation coefficient withtotalscore≥0.3,theCronbach’salphacoefficientwas calculated.Theselecteditemsweresubjectedtoprincipal componentsfactor analysiswithorthogonalrotation (Vari-max)toidentifythefactorialstructureoftheinstrument.

Discriminantvalidityoftheinstrumentscoreswastested bycalculatingthedifferencebetweenthefactorandtotal scoresofthequestionnaireamonganesthesiologistsgroups whoreportedusingthechecklistinlessormorethan70%of patients(Table1).

Predictivevaliditywasassessed(a)usingthecalculation ofSpearman’scorrelation coefficientsbetween thescores andthebinaryvariableofchecklistuseinlessormorethan 70%ofcases(theoutcomevariable)and(b)theanalysisof theROCcurveparametersbetweenthetotalscoreandthe outcomevariable.

Results

The total sample of 459 participants who completed the questionnaireinfourdaysoftheaforementionedCongress, 55% male and 44.2% with less than 10 years and 15.5% over30yearsofmedicalschoolcompletion.Only2.2%said theyhavedoneorbedoingaspecializationcourseor resi-dencyinanesthesiology.Theaverageageofparticipantswas 40.7years.

Seven items with 78% reliability coefficient were selected.Factoranalysisidentifiedtwofactors:F1,related totheperceptionoftheutility,andF2relatedtothe per-ceptionoftheapplicabilityofthechecklist(Table2).These twofactorsexplained58%ofthevarianceinscores.

Therewasastatisticallysignificant differencebetween the groups of anesthesiologists who reported using the instrumentinlessor morethan70%of patients.This indi-catesthattheattitudequestionnairediscriminatesbetween thesetwogroupsofprofessionals.

Table2 Perceivedutilityfactorsandapplicability.

Questionsregardingtheperceptionfactoronutility(F1)

•Areyoufamiliarwiththe‘‘SafeSurgerySavesLives’’

project(Checklist)?

•Itisusefulandpreventserrorsduringsurgery

•Itisusefulandpreventserrorsduringanesthesia

•Checklistshouldbemandatory

Questionsregardingtheperceptionfactoronapplicability

(F2)

•ShouldbeusedonlyininstitutionswithHospital

Accreditation

•Shouldbeappliedsolelybynurses

•Checklistisnotapplicabletoinvasiveproceduresinthe

DiagnosticCenter,suchasspinalinfiltration

Thecorrelationcoefficientsbetweenthefactorandtotal scoresandtheoutcomevariablewererho=0.32,p<0.01for F1scores;rho=0.14,p<0.02forF2scores;andrho=0.28, p<0.01fortotalscore.Thesevaluesaresignificant,butthe predictivevalidityof thequestionnaireis low.These find-ingsaresubstantiatedbythepercentageofareaunderthe curve=0.66(0.61---0.71).Thisindicates lowsensitivityand specificity of the instrument asa predictor of the use or non-useofthechecklistinmorethan70%ofpatientsseen byanesthesiologists.

Discussion

TheWHOSafeSurgerySavesLivesprogramaimstoincrease thequalityandsafetystandardsinhealthcare;contemplate theprevention ofsentinelevents, surgicalsite infections, safeanesthesia,safesurgicalteams,andindicatorsof sur-gicalcare.Itwasdecidedthattheinstrumentwouldserve acore set ofsafetystandardsthatcouldbe applied glob-allyand in differentscenarios. Inthis set, itwas obvious themultidisciplinarycharacter,includingallwhoworkand contributetotheexcellenceofpatientcare.

Checklistsarequickandsimpletoolsthatshouldbeused byallprofessionals,frombeginnerstoexperts.Itisnotlike an‘‘instructionmanual’’oreventhesameasthestandard operatingprocedures(SOP).Checklists,asproposedinthe WHO ‘‘Safe Surgery Saves Lives’’ project for application before induction of anesthesia, before skin incision and before the patient leaves the operating room, should be brief,butwithoutcompromisingtheeffectivenessregarding theobjectivestowhichitisproposed.

The use of checklist is being monitored by the WHO,9

andcurrentlyabout1800healthcarefacilitiesreporttheir systematicuse,particularlyinthecountriesofNorth Amer-icaandEurope,inadditiontoanincreasingimplementation inAsia andMiddleEast.Brazil, sofar,does notappearas oneofthecountriesthatexpresstheuseofchecklisttoits fullest.

TheSBApresentedtheWHO‘‘SafeSurgerySavesLives’’ project to Brazilian anesthesiologists for the first timein 200910 _{and,sincethen,theQualityandSafetyCommission}

in Anesthesia (CQSA) hasbeen promoting theproject dis-seminationactionsinallofficialeventsoftheSociety and many ofits regionalfacilities nationwide.However,there wasnoway tohave an estimate of anesthesiologistswho workinthecountryandusethechecklist.Thus,theBoard ofSBAin2012acceptedtheproposalofCQSAtoperforma surveyduringthe59thCBAinBH/MGandprovidedresources forthesurveyapplicationpresentedhere.

The choice of the event was due primarily to allow that the largest number of participants could respond to the questionnaire, as the number of subscribers, around 2500professionals, allowit toreacha highpercentageof response, which is essential for successful studies using questionnaires as a tool, as well as being a more homo-geneous sample due tothe participation of professionals fromaroundthecountry,eventhoughthesampledoesnot includeanesthetistswhodonotusuallyattendtheCongress ofAnesthesiology.

Conclusions

The7-itemquestionnaireshowedadequateinternal consis-tency(Cronbach’salphacoefficient>0.7)andawell-defined factorial structure, it can be used as a tool to measure anesthesiologists’ perceptions about the usefulness and applicability of the WHO Safe Surgery Saves Lives check-list. However, although the perceptions captured by the questionnaireareabletodiscriminateamong anesthesiolo-gistswithhighestandlowestpercentageofusechecklistin practice,thescoresresultingfromthequestionnaires can-notpredicttheuseoftheinstrument.Thatis,despitethe moreorlesspositiveperceptionsaboutthechecklist,there isnoassociationbetweentheprobabilityofusingornotthe instrumentinpractice.

Conflicts

of

interest

Theauthorsdeclarenoconflictsofinterest.

References

1.Organizac¸ão Mundial da Saúde. Segundo desafio global para a seguranc¸a do paciente: Cirurgias seguras salvam vidas (orientac¸ões para cirurgia segura da OMS). Rio de Janeiro: Organizac¸ão Pan-Americana da Saúde, Ministério da Saúde, AgênciaNacionaldeVigilânciaSanitária;2009.

2.MakaryMA,HolzmuellerCG,ThompsonD,etal.Operatingroom briefings:workingonthesamepage.JtCommJQualPatient Saf.2006;32:351---5.

3.ManserT,FosterS,FlinR,etal.Teamcommunicationduring patienthandoverfromtheoperatingroom:morethanfactsand figures.HumFactors.2013;55:138---56.

4.HaynesAB,WeiserTG,BerryWR,etal.Asurgicalsafety check-listtoreducemorbidityandmortalityinaglobalpopulation.N EnglJMed.2009;360:491---9.

5.Joint Commission International. https://manual. jointcommission.org/Manual/WebHome[accessed02.08.14]. 6.Handler SM, Castle NG, Studentski AS, et al. Patient safety

cultureassessmentinthenursinghome.QualSafHealthCare. 2006;15:400---4.

7.WolfleD,LikertR,MarquisDG,etal.Standardsforappraising psychologicalresearch.AmPsychol.1949;4:320---8.

8.HairJF,BlackWC,BabinBJ,etal.AnáliseMultivariadadeDados. 6a_{ed.PortoAlegre:Bookman;2009.}

9.Surgical Safety Web Map. http://maps.cga.harvard.edu: 8080/Hospital/[accessed22.09.13].

10.Correa RT. Guia da Organizac¸ão Mundial da Saúde para a seguranc¸a doato anestésico-cirúrgico.Anestesiaem Revista. 2009;2:16---7.

11.Mastracci TM. Whatare the effects ofintroducing theWHO ‘‘surgicalsafetychecklist’’onin-hospitalmortality?JAmColl Surg.2013;217:1151---3.

12.RussSJ.Aqualitativeevaluationofthebarriersandfacilitators towardimplementation oftheWHO surgical safetychecklist acrosshospitalsinEngland:lessonsfromthe‘‘SurgicalChecklist ImplementationProject’’.AnnSurg.2015;261:81---91. 13.World Health Organization. Safe surgery saves lives:

sec-ond global patient safety challenge; 2008. Available at: http://www.who.int/patientsafety/safesurgery/knowledge

base/SSSLBrochurefinalJun08.pdf[accessed13.04.14]. 14.Soria-AledoV.Dificultadesenlaimplantaciondelchecklisten