AVALIAÇÃO DE PRO GRAMAS E PO LÍTICAS DE SAÚDE NA FRANÇA 2 4 7
Cad . Saúde Pública, Rio d e Jane iro , 15(2):229-259, ab r-jun, 1999 p o lit ica l cu lt u re o r ie n t e d t ow a rd s e va l u a t i o n ,
a n d I t h in k t h a t d e s p it e re c e n t e ffo rt s, we s t ill e x p e rie n ce a ch ron ic a n d se ve re la ck of a ca d e m ic a n d te ch n ical sp e cialists in q u ality eva l u a -t io n , q u a li-t y m an a ge m e n -t , p ro d u c -t io n , a n d an a lys is o f h e a lt h d a t a , m e d ic a l d o cu m e n t a -t io n , a n d s o fo r-t h . T h u s, a s p o in -t e d o u -t b y -t h e a u t h o r in q u o t in g Gér a rd d e Po u rvo u rv i l l e, we sh o u ld se a rch fo r s h o rt c u t s, le a rn in g fr o m th e e x p e rien ce of ot he r cou n tri e s, b u t sh a p in g it to ou r m e s u re s, wh ich u n fo rt u n a t ely st ill e xp re s s a m u lt ip licit y o f d eficie n cies.
ALM EID A, C.; T R AVASSOS, C.; PORTO, S. & BAPTIS-TA, T., 198 8. A Reform a Sa n it ária Bra s i l e i ra : e m Bu sca d a Eq ü i d a d e .Re l a t ó r io fi n a l d e p e sq u i sa , Rio d e Ja n e i ro : Esco la Na cio n a l d e Sa ú d e Pú b lica, Fu n d a ção Oswa ld o Cru z .
B ROOK, R.; Mc G LYN , E. & CLEARY, P., 1996. Qu alit y o f c a re . Pa r t 2: Me a s u rin g q u a lit y of ca re . New En g-lan d Jou rn al of Me d i c i n e ,1 3 : 9 6 6 - 9 7 0 .
Th e fi e ld o f e valu a ti o n a nd th e “s u r m es u re ”
s t ra t e gy
Zu l m i ra Ha rtz h as lau n ch e d a tim e ly d eb a t e on t h e in st it u tion a liza tion of eva lu atio n fo r h ealth p o licie s a n d p ro g ra m s. Th e a u t h or p rovid e s a n e x t e n s i ve re vie w o f in t e r n a t io n a l e xp e ri e n c e s a n d p a rt ic u la rly fo c u se s o n t h e Fre n ch ca s e, ra isin g p r im e iss u e s fo r t h e d e b a t e o ve r t h e c u r re n t Bra zilia n h e a lt h a ge n d a : t h e u se o f e va lu a tio n to b a ck d e cisio n m akin g a n d it s in -c o r p o ra t io n in t o h e a lt h re f o rm e xp e r i e n -c e s, t h e re la t ion sh ip b e t we e n p o lic ie s a n d p ro -g ra m s, an d e sp e c ia lly t h e fie ld ’s cu r re n t t re n d t ow a rd s m et h o d o lo gica l p lu ralism .
I w o u ld st a r t b y re fle ct in g o n t h e fie ld ’s s p e cificit y a n d t h e o p p o sit ion b e t we e n t h e s t ru c t u red o r p r ê t - à - p o rt e ra n d n on -stru c t u re d o r su r m esu rea p p ro a c h e s. De sp it e t h e va ri o u s lim it a t io n s p o s e d b y e xp e r im e n t al d e sign s, m ain ly w it h re g a rd t o e t h ica l a n d o p e ra t i o n a l p ro b l e m s, t h e y h a ve b e e n u se d t o su p p o rt h e alt h syst e m s a n d s e r vice s m a n a ge m e n t (an asp ect o f th e in st it u t io n a lizat ion of e va l u a t i o n ) In stitu t o d e Sa ú d e
C o l e t i va , Un i ve r s i d a d e Fe d e ra l d a Ba h i a , S a l va d o r, Bra s i l . Ligia M aria Vi e i ra d a Si l va
p a r t ic u la rly in re la t io n t o t h e e ffica cy o f t e ch -n o l o g i e s. I-n a d d itio-n , b u ild i-n g i-n form a tio-n sys-t e m s sys-t o m o n isys-t o r h e a lsys-t h s isys-t u a sys-t io n s re q u i re s d e fin in g p ro b l e m s, cr i t e ria , a n d p a t t e rn s on a n a t io n al a n d in t e r n a t ion a l sca le , a n a p p ro a c h t h a t h as m a d e it p o ssib le t o c o n t ro l so m e d is-e a s is-e s in t h is-e p a s t . If wis-e d is-e fin is-e, m is-e as u ris-e, a n d e va lu a t e p ro b le m s o n ly o n t h e b a s is o f loc a l c ri t e ria an d p at ter n s, n ot on ly com p ar ison s b eca m e im p o ssib le, b u t t h e p o ssib ilit y o f a rt i c u -la t in g c o n t ro l m e a su re s su ch a s t h o se le a d in g t o t h e e ra d ica tio n of s m a llp ox wo rld wid e a n d p o lio in t h e Am e r i c a s. Th is d o e s n o t m e a n t o d e n y t h e so cia l a n d cu lt u ra l n a t u re o f t h e h e a lt h / d ise a se p h e n om e n o n , se ve ra l a sp e c t s o f w h ic h re q u i r e a lo ca l a n d d e ce n t ra l i ze d f o c u s fo r d ia gn o sis an d in t e r ve n t io n , in ad d i-t io n i-t o n e go i-t ia i-t e d e va lu ai-t io n . Eva lu a i-t ion o f p ro g ra m cove ra ge ca n on ly b e p e r f o rm e d in a q u a n tifie d , st ru c t u re d way. Ye t t h e m e a n in g o f t h is c ove ra ge wit h re g a rd t o t h e d e gre e o f im -p le m e n t a t io n a n d t h e t e ch n ic al a n d scie n t ific q u a lit y is re ve a le d m o r e ac c u ra t e ly t h ro u g h loo sely st ru c t u re d a p p ro a c h e s, ta kin g re c o u r s e t o q u a lit at ive t e ch n iq u e s t o o b t a in in fo rm a -t io n . Like w is e , e va lu a -t io n o f e ffe c-t ive n e s s, wh ich u n t il recen tly re q u i red an exc l u s i vely ex-p e rim e n ta l d esign , can n ow b e co n d u cte d wit h lo ose ly st ru c t u re d st ra t e g i e s.
H A RTZ, Z. M. A. 2 4 8
Cad. Saúde Pública, Rio de Jane iro , 15(2):229-259, abr-jun, 1999
Eva l u a t i o n : th e Fren ch che fs are still s e a rching for “la n ou v elle cu isi n e”
I wish t o b e gin b y c o m p lim e n t in g Zu l m i ra Ha rtz fo r h e r a rt icle o n t h e in stit u tion a lizatio n of e valu a t io n in t h e Fre n ch co n te xt , s in ce in a few p a ges sh e p rovid es a b rillian t an a lysis o f th e h i s t o ry o f p r o g ra m e va lu a t ion in Fra n c e . Se c-o n d , sh e ra ise s a n u m b e r c-o f im p c-o r t a n t issu e s on wh ich I wo u ld like to co m m en t b rie fly h e re.
Th e first p o in t con cer n s t h e role o f e va l u a -tion in o u r p a rliam en t a ry d e m o cra c i e s. In d e e d , in s t it u t io n a liza t io n o f e va lu a t io n s h o u ld b e se en as an at te m p t to re s t o re to elect ed officia ls t h e p ow e r t o co n t ro l th e fu n c tio n in g o f p u b lic a d m i n i s t ra t io n , t h e so ca lled t ech n ocra c y. Be
-G é ra rd de Po u vo u rv i l l e
Grou p e Im a g e , École N a tion a le d e la San té Pu b l i q u e S a i n t - Ma u r i c e , Fra n c e . e va lu at ion in vo l ves in terests lin ke d to t h e
power sp h powere, espe cia lly wh e n it is a m a tt power of p u b -lic p o -licy e va lu a t ion . A p o -licy’s su cce ss o r fa il-u re m ea n s t h e a ccil-u m il-u lat io n or lo ss of p o litica l a n d sym b olic ca p it a l e it h e r b y t h o se wh o go v-e rn o r b y t h o s v-e in o p p o sit io n . Eva lu at io n ca n a lso b e u sed by m a n a ge rs in sid e in stitu tion s as a t o ol t o con tro l su b o rd i n a t e s. T h u s, re s i s t a n c e t o t h e o u t sid e e va l u a t o r, q u a n t ifica t io n , a n d o b jec t ifica t io n ca n b e gre a t e r, wh ile st ra t e g i e s t o e xp a n d in d ivid u a l p owe r (e m p o we rm e n t ) m ay b e m o re read ily accep te d a n d m o re o rg a n -ic wh e n t h e re is a t re n d t ow a rd s d e c e n t r a l i z a-t io n a n d d e m o cra a-t iz aa-t io n o f d e c isio n -m akin g p ro c e s s e s.
A n a l y ze d fro m a d iffe r e n t a n gle , t h e s u r m e s u rea p p ro ach m ay b e se e n a s a d ju st in g t h e e va l u a t i o n’s m e t h od ological st ra t e gy t o it s ob -je c t . In t h is se n se, it sh o u ld also b e p re f e r re d , wh ile th e m a in p rob le m s in vo l ve m ore th e con -s t r u ct io n o f t h e o b je c t (Bo u r d ie u , 1989) a n d t re a t m e n t o f t h e o ry a s a gu id e fo r e va l u a t i o n (Ch e n , 1990) t h a n t h e o p p os it io n b e t we e n q u a l i t a t i ve a n d q u an t it at ive t e ch n iq u e s, wh ich (a s Ha r t z p o in t s o u t q u it e a p p ro p ria t e ly) ca n b e ar ticu la t ed in act u a l st u d ies.
An o t h e r im p o rt a n t p o in t ra ise d b y t h e a u t ho r is t h e d istin ct io n b e twee n e va lu a tion p ro -g ram s a n d p o licies. Alth o u -gh th e b ib lio -gra p h y she q u ote s d o es n ot m ake t h is d istin ction , I b e-l i e ve it is n ecessa ry from b o th a th eo ret ica e-l an d m e t h o d o lo gica l p o in t o f vie w. Pu b lic p o lic ie s re la te to t h e st a te, i.e., th e p ower fie ld ; eva l u a t-in g p o lic ie s t-in vo l ve s n o t o n ly ju d gt-in g t h e a d e q u a c y, pe rt i n e n c e, e ffect ive n e s s, efficien cy, a n d legitim acy of gove rn m en t al in t en tio n s a n d a c t i o n s, b u t e sp e cia lly a n a lyz in g t h e n a t u re o f t h e st a t e a n d t h e p o lit ic al p owe r in vo l ve d in d ra ft in g t h e m . Me a n w h i l e , p ro g ra m s re l a t e m o re t o a p o licy’s tech n ical an d op erat io n al d im e n sion , i.e., its im at e ria l im a n ife stat ion a s o b -j e c t i ve s, go a ls, re s o u rc e s, a n d a c t ivit ie s, a n d t h e ir e va lu a t ion re q u i re s a set o f m e t h o d s a n d t e ch n iq u e s t h a t are d iffere n t fro m t h o se n e ed -e d t o a n a lyz-e p o lic i-e s. By c o m b in in g p o lici-e s a n d p ro g ra m s in t h e sam e o b ject , o n e ru n s t h e risk o f red ucin g p olit ics to te ch n iq u e or e ven t o p la n n e d p o licy, o r (in ve rse ly) focu sin g on ly o n t h e p o litic a l s id e o f t e ch n iq u e. I d o n o t m e a n t o sa y (e sp e cia lly a t t h e lo ca l gove r n m e n t a l or e ve n in s t it u t io n a l le ve l) t h a t p r o g ram e va l u a -t io n is a m e re ly -t e ch n ic a l issu e . Fo r e xa m p le, d e pe n d in g o n t h e o b je ct or issu e t o b e e va l u a t -e d , con t-e xtu a l a n alysis ca n b -e a n in v-e s t i g a t i o n s t ra t e gy fo r e xp la in in g t h e p roc e sse s in vo l ve d in im p le m en t in g a p ro g ra m .
La st co m es t h e relat io n sh ip b et ween e va l u -a t io n -an d t h e d e cis io n - m -a kin g p ro c e s s. Mo re
t h a n a t h e o re t ical, t e ch n ica l, or m e t h od o logi-ca l q u e st io n , t h is is a p o litilogi-ca l an d eth ilogi-cal issu e, i n vo lvin g c h o ic e s. Th a t is, fa ce d wit h va ri o u s ra t io n a le s t h at in te rf e re with t h e m an a ge m e n t p ro c e s s, in st it u tio n aliz at ion of e valu at io n fo r a p u b lic h e alt h syst e m m e a n s se e kin g t o e n su re t h e h e ge m o n y of th e te ch n ica l/ he alth ra t i o n a l e in t h e d e c isio n -m akin g p ro c e s s , i.e., p ri o ri t i z-in g h e a lt h n e ed s ove r z-in st it u t io n a l cor p ora t i s t o r e ve n e xt e rn a l p re s s e s. It m e an s d e ve l o p i n g m a n a ge m e n t b a se d o n id e n t ificat io n of p ro b -l e m s, o rga n izin g su p p -ly t h ro u gh p ro g ra m m e d a c t i o n s, a n d e m p h as izin g co n t r o l o f ris ks a n d c a u se s t h ro u gh a t e rri t o r ial fo cu s, wit h so cial p a rt icip a t io n . In o t h e r wo rd s, it m e a n s ch a n gin g t h e cu rre n t h e a lt h c a re m o d e l. Th is d e m a n d s n o t o n ly e va lu a t in g, b u t e sp e c ia lly in -t e r ven in g -to ch a n ge -th e cou n -tr y ’s h e a l-th re a l i -t y
su r m esu re.
BOURDIEU, P., 1989. O Poder Si m b ó l i c o. Rio d e Ja n e i-ro : Be rt ra n d Bra sil/ Lisb o a : D i f e l .
BOURDIEU , P. & WACQUA N T, L. J. D., 1992. R é p o n s e s . Pou r Un e An t h ropologie Réflex i ve. Pa r is: Se u il. C H E L I M S K Y, E., 1997. Th e Co m in g Tra n s f o rm a t i o n s
In Eva lu at io n . In : Evaluation for the 21st Ce n t u ry : A Ha n d b o o k(E. Ch e lim sky & W. R. Sh a d ish , e d s. ) , p p . 1-26, Th o u sa n d Oa ks: Sa ge Pu b l i c a t i o n s. CH EN , H ., 1990. T h e o ry - D r i ven Ev a l u a t i o n s. Ne w
