H A RTZ, Z. M. A. 2 4 6
Cad . Saúde Púb lic a, Rio de Janeiro , 15(2):229-259, ab r-jun, 1999
In th is in t ere stin g ar t i c l e, Zu l m i ra Ha rt z t ra c e s t h e h ist o ry o f th e in st it u t io n a liza tion o f p olicy a n d p r o g ra m e va lu a t io n in Fra n c e a s a re f e r -e n c-e to r-efl-ect o n p o ssib l-e p a th s t o b -e fo llow-e d in Bra zil. Th e d efin itio n of in st it u t io n a liza tio n a d o p t e d b y t h e au t h o r (in t e grat io n o f e va l u a t io n in t o t h e o rga n iza t io n a l sys t e m o n e is a t t e m p t in g t o in flu e n ce ) e xp re ss e s t h e e va l u a -t io n a p p ro a ch sh e a ssu m es, -t h a -t is, a n ac-t ivi-t y n ot lim ited t o re s e a rch p ra c t i c e, b u t an in t egra l p a r t o f p o licy- m a kin g a n d im p le m e n t a tio n o f a c t ivit ie s in t h e se p o lic ie s an d p ro g ra m s. In t h is a p p ro a ch , t h e ro le o f e va lu a t o rs a n d / o r e va lu a tio n age n cie s is e xp a n d e d b y a s su m in g t h e a d d it io n a l fu n c t io n o f fa c ilit a t in g q u alit y i m p rovem en t p ro c e s s e s. Th e a u t h or th u s d ra w s t h e co n ce p t s o f evalu at io n an d re gu la tion clo se r t o gse t h se r. Un d se r a b ro a d a p p ro ac h , rse g u l a -t io n is s e e n n o -t o n ly a s a n o rm a -t i ve re s o u rc e, b u t a ls o a s a n a ct io n fa cilit a t in g gove rn a n c e (Alm e id a e t a l., 1998). Ac c o rd in g t o t h is a p -p ro a ch , t h e e ffe ctive n e ss of e va lu at ion im -p lie s t h e u se o f less co e rc i ve (an d th u s m o re p a rt i c i-p a t o ry) st ra t e g i e s, in a d d it ion t o crea t in g fle xi-b l e, d ece n t ra l i ze d e valu at ion st ru c t u re s.
In t h e sp e cific fie ld o f e va lu a t in g q u a lit y o f h e alt h ca re se r v i c e s, a n in t e gra l p ar t o f e va l u -a t in g h e -a lt h p r o g r-a m s -an d p olicie s, o n e -a ls o o b s e r ve s, in m a n y co u n t r i e s, re l e va n t ch a n ge s in q u a lit y e va lu a t io n re la t in g t o t h e ch a n ge s id en t ifie d b y Ha rtz v i s - à - v i sp ro g ra m a n d p o li-cy evalu a tion . Su ch ch a n ge s re su lt fro m t h e re-a liz re-at io n t h re-a t c u rre n t h e re-a lt h cre-a re p r o d u c t i o n re q u i re s q u ite com p lex syst em s an d p ro c e s s e s, a n d t h u s t h a t t h e re su lt s o f p at ie n t ca re a re l a r ge ly e xp la in a b le b y p ro b le m s in t h e s e s ys-t em s an d p ro ce sse s a n d n oys-t m e re ly b y ys-t h e p er-f o r m a n ce o er-f a s p e cier-fic h e a lt h c a re p rov i d e r (p h ys ic ia n o r n u r se ). Sin c e im p r ove d p e r f o r-m a n c e b y h e a lt h ca re o rga n iza t io n s d e p e n d s fu n d a m e n t ally on a ction s d e ve lo pe d by t h e organ izatio n it se lf, re g u l a t o r y a n d q u alit y eva l u a t io n a gen cies sh o u ld a ct to m o t iva te o rg a n i z a -t io n s -t o im p ro ve -t h e ir p e rf o rm a n c e , le a vin g sa n ct io n s fo r t h o se ca se s in wh ich t h e ir is e vi-d en t risk t o t h e p u b lic. An e xa m p le o f t h is n e w a p p ro ach wa s le d in t h e ea rly 1990s b y th e Jo i n t Com m i ssion f or Ac c red i t a t ion of Hea lt h Ca re Organ iz a t ion s ( J C A H CO ), a tra d it io n a l Am
eri-Debat e sobre o artigo de Zulmira M . A. Hart z
De b ate o n the pap e r by Zulmira M. A. Hart z
Escol a N acion a l d e Sa ú d e Pú bl ica, Fu n d ação O sw al d o Cru z , Ri o d e Ja n e i r o, Bra s i l .
Clau d i a Tra va s s o s
ca n in stit u tion in th is fie ld . Th e JCAH CO re d e-fin e d it s e va lu a t io n focu s, co n ce n t ra t in g m o re o n se e kin g h e a lt h c a re s er v i c e s’ con for m it y t o s t a n d a rd s t a rge t in g p ro c e sse s a n d a c t ivit ie s wit h a d ire ct o r in d ire ct e ffe ct o n p a t ie n t ca re, i m p rovin g t h e ir co m m u n ica t io n s wit h h e a lt h s e r vic e s an d e m p h a siz in g t h e ir e d u c a t io n a l ro l e, in a d d it io n t o t ran sm itt in g e va lu a tio n re -su lts to t h e p u b lic.
Desp it e agre ein g with th e co n cep t o f eva l u at ion ad op t ed b y th e au th o r, I wo u ld like to in -t ro d u ce an a l-t e rn a -t i ve -t o -t h e em p h a sis p la c ed o n in s t it u t io n aliza t io n in h e r a rt i c l e . In e x-p a n d in g th e co n cex-p t of eva lu at io n , I wish t o d o so with ca u t io n in o rd e r n o t t o u n d e r ra t e t h e i m p o rta n ce o f re s e a rch fo r th e effe ctive n e ss of t h e e valu a t io n p ro c e s s. In t h e sp e cific ca se o f h ea lt h care se rv i c e s, e va lu atin g q u alit y o f ca re is an e xtrem ely d ifficu lt task, given cert ain c h a r-a c t e r ist ics o f p h ysicir-a n p r r-a c t i c e, s u ch r-a s t h e fa ct t h a t it is b a s e d o n sp e cia lize d kn ow l e d g e , p e rm e a t e d b y u n ce r t a in t ie s w it h re g a rd t o a m a jo r p o r t io n o f t h e a vaila b le d ia gn o st ic a n d t h e ra p e u t ic p ro c e d u re s , a n d t h e fa ct t h a t p a -t ie n -t s p re s e n -t b r o a d va r ia b ili-t y in -t h e ris ks o f d e ve lo p in g a d ver se re s u l t s. Re s e a rch in h e a lt h c a re se rvice s is a re l a t i ve ly n e w in t e rd i s c i p l i-n a ry fie ld st ill d ealii-n g wit h im p orta i-n t t ech i-n ical a n d m e t h o d o lo gica l lim it a t io n s, d e sp it e a d -van ce a ch ieve d in th e last two d eca d es (Bro o k , 1996). In a d d it io n , in t h is n e w p a ra d igm , t h e wa y t h e re su lt s a re s u m m a r i ze d a n d p u b lic i zed gain s re l e va n lic e, wh ilich im p lies lico n stitu t -in g e va lu a t io n t e am s wit h n e w co m p e t e n cie s, su ch a s com m u n ica t io n s, te ach in g, an d p olicy-m akin g.
T h u s , fr o m m y p o in t o f vie w, in s t it u t io n -a liz -a t io n o f e v-a lu -a t io n in t h e h e -a lt h s e c t o r sh o u ld fo c u s n o t o n ly o n im p r o vin g in t e r a c -tio n b e t wee n d ecisio n -m a ke rs, eva l u a t o r s, a n d h e a lth ca re m a n a ge rs a n d p rov i d e r s. It sh o u ld a lso co n sid er st ra t e gie s a n d re s o u rc es n e e d e d to e n cou ra ge t h e p ro d u ction of kn owled ge an d t ra in in g sp e c ia lize d p e r so n n e l. Re s e a rch p ro -d u c in g im p re cis e re su lt s le a -d s t o co n flict in g i n t e r p re t a t io n s a n d fa ils t o p ro d u ce d isc e r n -m e n t o f th e fa ct s an a lyze d . It can th u s gen e ra t e d i s c re d it over t h e u s efu ln e ss o f e va lu a t ion a c-t i v i c-t i e s, wic-t h a n egac-t ive im p acc-t o n c-th e valu e a s-c rib ed to th em by d es-cision -m akers, h ea lt h s-ca re m an age rs a n d p rov i d e r s, a n d u se rs.
AVALIAÇÃO DE PRO GRAMAS E PO LÍTICAS DE SAÚDE NA FRANÇA 2 4 7
Cad . Saúde Pública, Rio d e Jane iro , 15(2):229-259, ab r-jun, 1999
p o lit ica l cu lt u re o r ie n t e d t ow a rd s e va l u a t i o n , a n d I t h in k t h a t d e s p it e re c e n t e ffo rt s, we s t ill e x p e rie n ce a ch ron ic a n d se ve re la ck of a ca d e m ic a n d te ch n ical sp e cialists in q u ality eva l u a -t io n , q u a li-t y m an a ge m e n -t , p ro d u c -t io n , a n d an a lys is o f h e a lt h d a t a , m e d ic a l d o cu m e n t a -t io n , a n d s o fo r-t h . T h u s, a s p o in -t e d o u -t b y -t h e a u t h o r in q u o t in g Gér a rd d e Po u rvo u rv i l l e, we sh o u ld se a rch fo r s h o rt c u t s, le a rn in g fr o m th e e x p e rien ce of ot he r cou n tri e s, b u t sh a p in g it to ou r m e s u re s, wh ich u n fo rt u n a t ely st ill e xp re s s a m u lt ip licit y o f d eficie n cies.
ALM EID A, C.; T R AVASSOS, C.; PORTO, S. & BAPTIS-TA, T., 198 8. A Reform a Sa n it ári a Bra s i l e i ra : e m Bu sca d a Eq ü i d a d e .Re l a t ó r io fi n a l d e p e sq u i sa , Rio d e Ja n e i ro : Esco la Na cio n a l d e Sa ú d e Pú b lica, Fu n d a ção Oswa ld o Cru z .
B ROOK, R.; Mc G LYN , E. & CLEARY, P., 1996. Qu alit y o f c a re . Pa r t 2: Me a s u rin g q u a lit y of ca re . N ew En g-lan d Jou rn al of Me d i c i n e ,1 3 : 9 6 6 - 9 7 0 .
Th e fi e ld o f e valu a ti o n a nd th e “s u r m es u re ” s t ra t e gy
Zu l m i ra Ha rtz h as lau n ch e d a tim e ly d eb a t e on t h e in st it u tion a liza tion of eva lu atio n fo r h ealth p o licie s a n d p ro g ra m s. Th e a u t h or p rovid e s a n e x t e n s i ve re vie w o f in t e r n a t io n a l e xp e ri e n c e s a n d p a rt ic u la rly fo c u se s o n t h e Fre n ch ca s e, ra isin g p r im e iss u e s fo r t h e d e b a t e o ve r t h e c u r re n t Bra zilia n h e a lt h a ge n d a : t h e u se o f e va lu a tio n to b a ck d e cisio n m akin g a n d it s in -c o r p o ra t io n in t o h e a lt h re f o rm e xp e r i e n -c e s, t h e re la t ion sh ip b e t we e n p o lic ie s a n d p ro -g ra m s, an d e sp e c ia lly t h e fie ld ’s cu r re n t t re n d t ow a rd s m et h o d o lo gica l p lu ralism .
I w o u ld st a r t b y re fle ct in g o n t h e fie ld ’s s p e cificit y a n d t h e o p p o sit ion b e t we e n t h e s t ru c t u red o r p r ê t - à - p o rt e ra n d n on -stru c t u re d o r su r m esu rea p p ro a c h e s. De sp it e t h e va ri o u s lim it a t io n s p o s e d b y e xp e r im e n t al d e sign s, m ain ly w it h re g a rd t o e t h ica l a n d o p e ra t i o n a l p ro b l e m s, t h e y h a ve b e e n u se d t o su p p o rt h e alt h syst e m s a n d s e r vice s m a n a ge m e n t (an asp ect o f th e in st it u t io n a lizat ion of e va l u a t i o n ) In st it u t o d e Sa ú d e
C o l e t i va , U n i ve r s i d a d e Fe d e ra l d a Ba h i a , S a l va d o r, Bra s i l .
Li gi a M a ria Vi e i ra d a Si l v a
p a r t ic u la rly in re la t io n t o t h e e ffica cy o f t e ch -n o l o g i e s. I-n a d d itio-n , b u ild i-n g i-n form a tio-n sys-t e m s sys-t o m o n isys-t o r h e a lsys-t h s isys-t u a sys-t io n s re q u i re s d e fin in g p ro b l e m s, cr i t e ria , a n d p a t t e rn s on a n a t io n al a n d in t e r n a t ion a l sca le , a n a p p ro a c h t h a t h as m a d e it p o ssib le t o c o n t ro l so m e d is-e a s is-e s in t h is-e p a s t . If wis-e d is-e fin is-e, m is-e as u ris-e, a n d e va lu a t e p ro b le m s o n ly o n t h e b a s is o f loc a l c ri t e ria an d p at ter n s, n ot on ly com p ar ison s b eca m e im p o ssib le, b u t t h e p o ssib ilit y o f a rt i c u -la t in g c o n t ro l m e a su re s su ch a s t h o se le a d in g t o t h e e ra d ica tio n of s m a llp ox wo rld wid e a n d p o lio in t h e Am e r i c a s. Th is d o e s n o t m e a n t o d e n y t h e so cia l a n d cu lt u ra l n a t u re o f t h e h e a lt h / d ise a se p h e n om e n o n , se ve ra l a sp e c t s o f w h ic h re q u i r e a lo ca l a n d d e ce n t ra l i ze d f o c u s fo r d ia gn o sis an d in t e r ve n t io n , in ad d i-t io n i-t o n e go i-t ia i-t e d e va lu ai-t io n . Eva lu a i-t ion o f p ro g ra m cove ra ge ca n on ly b e p e r f o rm e d in a q u a n tifie d , st ru c t u re d way. Ye t t h e m e a n in g o f t h is c ove ra ge wit h re g a rd t o t h e d e gre e o f im -p le m e n t a t io n a n d t h e t e ch n ic al a n d scie n t ific q u a lit y is re ve a le d m o r e ac c u ra t e ly t h ro u g h loo sely st ru c t u re d a p p ro a c h e s, ta kin g re c o u r s e t o q u a lit at ive t e ch n iq u e s t o o b t a in in fo rm a -t io n . Like w is e , e va lu a -t io n o f e ffe c-t ive n e s s, wh ich u n t il recen tly re q u i red an exc l u s i vely ex-p e rim e n ta l d esign , can n ow b e co n d u cte d wit h lo ose ly st ru c t u re d st ra t e g i e s.
