An impact evaluation of the conditional cash transfers to education in praf: an experimental approach

Escola de Pós-Graduação em Economia - EPGE

Fundação Getúlio Vargas - FGV

AN IMPACT EVALUATION OF THE CONDITIONAL

CASH TRANSFERS TO EDUCATION UNDER PRAF:

AN EXPERIMENTAL APPROACH

Dissertação submetida à Escola de Pós-Graduação em Economia

da Fundação Getúlio Vargas como requisito parcial

para obtenção do Título de Mestre em Economia

Aluna: Priscila Zeraik de Souza

Orientador: Luis H. B. Braido

Co-orientador: Pedro Olinto

ii

Escola de Pós-Graduação em Economia - EPGE

Fundação Getúlio Vargas - FGV

AN IMPACT EVALUATION OF THE CONDITIONAL

CASH TRANSFERS TO EDUCATION UNDER PRAF:

AN EXPERIMENTAL APPROACH

Dissertação submetida à Escola de Pós-Graduação em Economia

da Fundação Getúlio Vargas como requisito parcial

para obtenção do Título de Mestre em Economia

Aluna: Priscila Zeraik de Souza

Banca Examinadora:

Luis H. B. Braido (Orientador, EPGE/FGV)

Pedro Olinto (Co-orientador, World Bank)

Marcelo Neri (EPGE/FGV)

iii

iv

Contents

List of Tables, v

List of Figures, xii

Acknowledgements, xiii

1. Introduction, 1

2. Description of PRAF II and of the data, 6

2.1 The PRAF II program, 6

2.2 Data, 9

3. Methodology, 10

4. The 2000 data: Educational results, 20

5. Analysis of the results, 24

5.1 Results of the regression analyses, 24

5.2 Results of the Markov Matrices, 26

5.2.1 Comparison of the treatment and control groups, 27

5.2.2 Comparison between boys and girls, 32

5.2.3 Comparison between the poor and the not poor, 35

5.2.4 Comparison between rural and urban, 39

5.3 Problems and Extensions, 43

6. Conclusion, 45

References, 47

Tables, 50

v

List of Tables

Table 1: Interview month in 2000, by Random Treatment Group, 50

Table 2: Sample Evolution from 2000 to 2002, by Random Treatment Group, 50

Table 3: Percentage that had Started Primary School by 2000, by Age and Sex, 51

Table 4: Percentage Enrolled at School in 2000, by Age and Sex, 51

Table 5: Percentage Attending School on the Survey Date in 2000, by Age and Sex, 52

Table 6: Years of Schooling Completed at Age 16, by Sex, 52

Table 7: Percentage that had Started Primary School, Enrolled and were Attending

School on the Survey Date in 2000, by per capita income quartile, 53

Table 8: Percentage Enrolled and Attending School on the Survey Date in 2000, by

those that Work and those that Do Not Work, 53

Table 9: Percentage that had Started Primary School, Enrolled and were Attending

School on the Survey Date in 2000, by Mother’s Education level, 54

Table 10: Percentage who had Started Primary School, Enrolled and were Attending

School on the Survey Date in 2000, by Distance to School, 54

Table 11: Estimated Impact of Demand-side and Supply-side Interventions on

Schooling Indicators, 55

Table M1a: Demand Transition Matrix, 56

Table M1b: Control Transition Matrix, 56

Table M1c: Differences Demand – Control, 56

Table M2a: Demand Transition Matrix, 57

Table M2b: Control Transition Matrix, 57

Table M2c: Differences Demand – Control, 57

Table M3a: Demand Transition Matrix, 58

Table M3b: Control Transition Matrix, 58

Table M3c: Differences Demand – Control, 58

Table M4a: Demand Transition Matrix, 59

Table M4b: Control Transition Matrix, 59

Table M4c: Differences Demand – Control, 59

vi

Table M6a: Demand Transition Matrix, 61

Table M6b: Control Transition Matrix, 61

Table M6c: Differences Demand – Control, 61

Table M7a: Demand Transition Matrix, 62

Table M7b: Control Transition Matrix, 62

Table M7c: Differences Demand – Control, 62

Table M8a: Control Transition Matrix for Boys, 63

Table M8b: Control Transition Matrix for Girls, 63

Table M8c: Differences Boys – Girls, 63

Table M9a: Control Transition Matrix for Boys, 64

Table M9b: Control Transition Matrix for Girls, 64

Table M9c: Differences Boys – Girls, 64

Table M10a: Control Transition Matrix for Boys, 65

Table M10b: Control Transition Matrix for Girls, 65

Table M10c: Differences Boys – Girls, 65

Table M11a: Control Transition Matrix for Boys, 66

Table M11b: Control Transition Matrix for Girls, 66

Table M11c: Differences Boys – Girls, 66

Table M12a: Control Transition Matrix for Boys, 67

Table M12b: Control Transition Matrix for Girls, 67

Table M12c: Differences Boys – Girls, 67

Table M13a: Control Transition Matrix for Boys, 68

Table M13b: Control Transition Matrix for Girls, 68

Table M13c: Differences Boys – Girls, 68

Table M14a: Control Transition Matrix for Boys, 69

Table M14b: Control Transition Matrix for Girls, 69

Table M14c: Differences Boys – Girls, 69

Table M15a: Demand Transition Matrix for Boys, 70

Table M15b: Control Transition Matrix for Boys, 70

Table M15c: Differences Demand – Control for Boys, 70

Table M16a: Demand Transition Matrix for Boys, 71

Table M16b: Control Transition Matrix for Boys, 71

Table M16c: Differences Demand – Control for Boys, 71

Table M17a: Demand Transition Matrix for Boys, 72

Table M17b: Control Transition Matrix for Boys, 72

Table M17c: Differences Demand – Control for Boys, 72

vii

Table M18c: Differences Demand – Control for Boys, 73

Table M19a: Demand Transition Matrix for Boys, 74

Table M19b: Control Transition Matrix for Boys, 74

Table M19c: Differences Demand – Control for Boys, 74

Table M20a: Demand Transition Matrix for Boys, 75

Table M20b: Control Transition Matrix for Boys, 75

Table M20c: Differences Demand – Control for Boys, 75

Table M21a: Demand Transition Matrix for Boys, 76

Table M21b: Control Transition Matrix for Boys, 76

Table M21c: Differences Demand – Control for Boys, 76

Table M22a: Demand Transition Matrix for Girls, 77

Table M22b: Control Transition Matrix for Girls, 77

Table M22c: Differences Demand – Control for Girls, 77

Table M23a: Demand Transition Matrix for Girls, 78

Table M23b: Control Transition Matrix for Girls, 78

Table M23c: Differences Demand – Control for Girls, 78

Table M24a: Demand Transition Matrix for Girls, 79

Table M24b: Control Transition Matrix for Girls, 79

Table M24c: Differences Demand – Control for Girls, 79

Table M25a: Demand Transition Matrix for Girls, 80

Table M25b: Control Transition Matrix for Girls, 80

Table M25c: Differences Demand – Control for Girls, 80

Table M26a: Demand Transition Matrix for Girls, 81

Table M26b: Control Transition Matrix for Girls, 81

Table M26c: Differences Demand – Control for Girls, 81

Table M27a: Demand Transition Matrix for Girls, 82

Table M27b: Control Transition Matrix for Girls, 82

Table M27c: Differences Demand – Control for Girls, 82

Table M28a: Demand Transition Matrix for Girls, 83

Table M28b: Control Transition Matrix for Girls, 83

Table M28c: Differences Demand – Control for Girls, 83

Table M29a: Control Transition Matrix for the Poor, 84

Table M29b: Control Transition Matrix for the Not-Poor, 84

Table M29c: Differences Poor – Not-Poor, 84

viii

Table M31a: Control Transition Matrix for the Poor, 86

Table M31b: Control Transition Matrix for the Not-Poor, 86

Table M31c: Differences Poor – Not-Poor, 86

Table M32a: Control Transition Matrix for the Poor, 87

Table M32b: Control Transition Matrix for the Not-Poor, 87

Table M32c: Differences Poor – Not-Poor, 87

Table M33a: Control Transition Matrix for the Poor, 88

Table M33b: Control Transition Matrix for the Not-Poor, 88

Table M33c: Differences Poor – Not-Poor, 88

Table M34a: Control Transition Matrix for the Poor, 89

Table M34b: Control Transition Matrix for the Not-Poor, 89

Table M34c: Differences Poor – Not-Poor, 89

Table M35a: Control Transition Matrix for the Poor, 90

Table M35b: Control Transition Matrix for the Not-Poor, 90

Table M35c: Differences Poor – Not-Poor, 90

Table M36a: Demand Transition Matrix for the Poor, 91

Table M36b: Control Transition Matrix for the Poor, 91

Table M36c: Differences Demand – Control for the Poor, 91

Table M37a: Demand Transition Matrix for the Poor, 92

Table M37b: Control Transition Matrix for the Poor, 92

Table M37c: Differences Demand – Control for the Poor, 92

Table M38a: Demand Transition Matrix for the Poor, 93

Table M38b: Control Transition Matrix for the Poor, 93

Table M38c: Differences Demand – Control for the Poor, 93

Table M39a: Demand Transition Matrix for the Poor, 94

Table M39b: Control Transition Matrix for the Poor, 94

Table M39c: Differences Demand – Control for the Poor, 94

Table M40a: Demand Transition Matrix for the Poor, 95

Table M40b: Control Transition Matrix for the Poor, 95

Table M40c: Differences Demand – Control for the Poor, 95

Table M41a: Demand Transition Matrix for the Poor, 96

Table M41b: Control Transition Matrix for the Poor, 96

Table M41c: Differences Demand – Control for the Poor, 96

Table M42a: Demand Transition Matrix for the Poor, 97

Table M42b: Control Transition Matrix for the Poor, 97

Table M42c: Differences Demand – Control for the Poor, 97

ix

Table M43c: Differences Demand – Control for the Not-Poor, 98

Table M44a: Demand Transition Matrix for the Not-Poor, 99

Table M44b: Control Transition Matrix for the Not-Poor, 99

Table M44c: Differences Demand – Control for the Not-Poor, 99

Table M45a: Demand Transition Matrix for the Not-Poor, 100

Table M45b: Control Transition Matrix for the Not-Poor, 100

Table M45c: Differences Demand – Control for the Not-Poor, 100

Table M46a: Demand Transition Matrix for the Not-Poor, 101

Table M46b: Control Transition Matrix for the Not-Poor, 101

Table M46c: Differences Demand – Control for the Not-Poor, 101

Table M47a: Demand Transition Matrix for the Not-Poor, 102

Table M47b: Control Transition Matrix for the Not-Poor, 102

Table M47c: Differences Demand – Control for the Not-Poor, 102

Table M48a: Demand Transition Matrix for the Not-Poor, 103

Table M48b: Control Transition Matrix for the Not-Poor, 103

Table M48c: Differences Demand – Control for the Not-Poor, 103

Table M49a: Demand Transition Matrix for the Not-Poor, 104

Table M49b: Control Transition Matrix for the Not-Poor, 104

Table M49c: Differences Demand – Control for the Not-Poor, 104

Table M50a: Control Transition Matrix for Rural Environment, 105

Table M50b: Control Transition Matrix for Urban Environment, 105

Table M50c: Differences Rural – Urban, 105

Table M51a: Control Transition Matrix for Rural Environment, 106

Table M51b: Control Transition Matrix for Urban Environment, 106

Table M51c: Differences Rural – Urban, 106

Table M52a: Control Transition Matrix for Rural Environment, 107

Table M52b: Control Transition Matrix for Urban Environment, 107

Table M52c: Differences Rural – Urban, 107

Table M53a: Control Transition Matrix for Rural Environment, 108

Table M53b: Control Transition Matrix for Urban Environment, 108

Table M53c: Differences Rural – Urban, 108

Table M54a: Control Transition Matrix for Rural Environment, 109

Table M54b: Control Transition Matrix for Urban Environment, 109

Table M54c: Differences Rural – Urban, 109

x

Table M56a: Control Transition Matrix for Rural Environment, 111

Table M56b: Control Transition Matrix for Urban Environment, 111

Table M56c: Differences Rural – Urban, 111

Table M57a: Demand Transition Matrix for Rural Environment, 112

Table M57b: Control Transition Matrix for Rural Environment, 112

Table M57c: Differences Demand – Control for Rural Environment, 112

Table M58a: Demand Transition Matrix for Rural Environment, 113

Table M58b: Control Transition Matrix for Rural Environment, 113

Table M58c: Differences Demand – Control for Rural Environment, 113

Table M59a: Demand Transition Matrix for Rural Environment, 114

Table M59b: Control Transition Matrix for Rural Environment, 114

Table M59c: Differences Demand – Control for Rural Environment, 114

Table M60a: Demand Transition Matrix for Rural Environment, 115

Table M60b: Control Transition Matrix for Rural Environment, 115

Table M60c: Differences Demand – Control for Rural Environment, 115

Table M61a: Demand Transition Matrix for Rural Environment, 116

Table M61b: Control Transition Matrix for Rural Environment, 116

Table M61c: Differences Demand – Control for Rural Environment, 116

Table M62a: Demand Transition Matrix for Rural Environment, 117

Table M62b: Control Transition Matrix for Rural Environment, 117

Table M62c: Differences Demand – Control for Rural Environment, 117

Table M63a: Demand Transition Matrix for Rural Environment, 118

Table M63b: Control Transition Matrix for Rural Environment, 118

Table M63c: Differences Demand – Control for Rural Environment, 118

Table M64a: Demand Transition Matrix for Urban Environment, 119

Table M64b: Control Transition Matrix for Urban Environment, 119

Table M64c: Differences Demand – Control for Urban Environment, 119

Table M65a: Demand Transition Matrix for Urban Environment, 120

Table M65b: Control Transition Matrix for Urban Environment, 120

Table M65c: Differences Demand – Control for Urban Environment, 120

Table M66a: Demand Transition Matrix for Urban Environment, 121

Table M66b: Control Transition Matrix for Urban Environment, 121

Table M66c: Differences Demand – Control for Urban Environment, 121

Table M67a: Demand Transition Matrix for Urban Environment, 122

Table M67b: Control Transition Matrix for Urban Environment, 122

Table M67c: Differences Demand – Control for Urban Environment, 122

xi

Table M68c: Differences Demand – Control for Urban Environment, 123

Table M69a: Demand Transition Matrix for Urban Environment, 124

Table M69b: Control Transition Matrix for Urban Environment, 124

Table M69c: Differences Demand – Control for Urban Environment, 124

Table M70a: Demand Transition Matrix for Urban Environment, 125

Table M70b: Control Transition Matrix for Urban Environment, 125

Table M70c: Differences Demand – Control for Urban Environment, 125

Table D1a: Years of Schooling Completed at Age 13, by Random Treatment Group

(pdf), 126

Table D1b: Years of Schooling Completed at Age 13, by Random Treatment Group

(cdf), 126

Table D2a: Years of Schooling Completed at Age 13, by Sex (pdf), 127

Table D2b: Years of Schooling Completed at Age 13, by Sex (cdf), 127

Table D3a: Years of Schooling Completed at Age 13, by Poor and Not-Poor (pdf),

128

Table D3b: Years of Schooling Completed at Age 13, by Poor and Not-Poor (cdf),

128

Table D4a: Years of Schooling Completed at Age 13, by Rural and Urban Environment

(pdf), 129

Table D4b: Years of Schooling Completed at Age 13, by Rural and Urban Environment

(cdf), 129

Table W1: Work done by the children, 130

Table W2: Work done by the children, by Sex, 130

Table W3: Work done by the children, by Poor and Not-Poor, 131

xii

List of Figures

Figure 1: Simulated Effects of Demand Treatment on Education Distribution by Age 13,

132

Figure 2: Simulated Education Distribution by Age 13 in the Control Group, by Sex,

133

Figure 3: Simulated Effects of Demand Treatment on Education Distribution by Age 13,

Boys, 134

Figure 4: Simulated Effects of Demand Treatment on Education Distribution by Age 13,

Girls, 135

Figure 5: Simulated Education Distribution by Age 13 in the Control Group, by Poor

and Not Poor, 136

Figure 6: Simulated Effects of Demand Treatment on Education Distribution by Age 13,

Poor, 137

Figure 7: Simulated Effects of Demand Treatment on Education Distribution by Age 13,

Not Poor, 138

Figure 8: Simulated Education Distribution by Age 13 in the Control Group, by Rural

and Urban Environment, 139

Figure 9: Simulated Effects of Demand Treatment on Education Distribution by Age 13,

Rural Environment, 140

xiii

Acknowledgments

In order to be able to make contributions to the body of knowledge in economic

science, it is fundamental to have contact with the best researchers. So, I wish to thank

the Graduate School of Economics at the Getúlio Vargas Foundation (EPGE/FGV), for

having provided me with a broad and solid conceptual base in economics and for having

allowed me to immerse myself in such a rich and stimulating academic environment.

This dissertation counted on the able orientation of Pedro Olinto, who always

combined rigorous theory, creativity and his vast experience with empirical works.

Research Fellow of the International Food Policy Research Institute (IFPRI), he made

available the data used in this study. His assistance was fundamental in the preparation

of this dissertation, and I am grateful for his dedication and effort throughout this

process.

I must also give credit here to the important influence of Marcos Lisboa, the first

professor at the EPGE with whom I carried out research activities. He also accompanied

the initial phase of the process of preparing this dissertation. The conversations about

economics that we had during this period were invaluable and contributed much to my

learning.

I am also enormously grateful to professors Marcelo Néri and Aloisio Araújo for

the opportunity to carry out through research on education alongside them. The further

my studies of the vast and rich literature on Labor Economics and Econometrics

progressed, the more I was impressed by the power of its analytical instruments to

understand a great variety of social questions.

xiv

For their comments and suggestions, I would like to thank the participants in

seminars at the Getúlio Vargas Foundation, and at the XXVI Brazilian Econometric

Meeting of the Brazilian Society for Econometrics in João Pessoa.

I am especially grateful to Lucas, my companion for all time. He not only

affectively supported and encouraged me, but participated in daily discussions about

economics. Moreover, he accompanied the entire process of preparing the dissertation

and made useful observations on important topics of the research.

My family is my greatest blessing. I am so grateful to my father for his continual

encouragement and for having done his utmost for my professional and personal

development. My special gratitude goes to my brothers for all I have learned throughout

the years of growing up together. I am also thankful to my grandmothers and to my

grandfather for their attention and affection.

AN IMPACT EVALUATION OF THE CONDITIONAL

CASH TRANSFERS TO EDUCATION UNDER PRAF:

AN EXPERIMENTAL APPROACH

Pedro Olinto

*

Priscila Z. de Souza

**

World Bank EPGE-FGV

_____________________________________

*

E-mail: polinto@worldbank.org

**

AN IMPACT EVALUATION OF THE CONDITIONAL

CASH TRANSFERS TO EDUCATION UNDER PRAF:

AN EXPERIMENTAL APPROACH

Abstract

This paper uses data from a social experiment in Honduras to estimate the educational impact on children participating in PRAF II, a social program that has two components: demand-side intervention – conditional cash transfers to families whose children are attending school on a regular basis – and supply-side intervention - cash transfers to improve the quality of the schools.

AN IMPACT EVALUATION OF THE CONDITIONAL CASH

TRANSFERS TO EDUCATION UNDER PRAF:

AN EXPERIMENTAL APPROACH

*

Pedro Olinto and Priscila Z. de Souza

Abstract

This paper uses data from a social experiment in Honduras to estimate the educational impact on children participating in PRAF II, a social program that has two components: demand-side intervention – conditional cash transfers to families whose children are attending school on a regular basis – and supply-side intervention - cash transfers to improve the quality of the schools.

The results of the difference-in-differences and cross-sectional regressions indicate that the conditional cash transfers increase school attendance and reduce drop out rates; but they show no impact whatsoever from the supply-side intervention. The results of a Markov schooling transition model, used to assess the impact of the demand-side intervention, show that the program effectively facilitates progression through the grades. When we used a simulation method to evaluate the long-term impact of exposure to the program and a bootstrap method to test the statistical significance of our estimations, we found that if children were to participate in the program between the ages of 6 and 12, they would experience an improvement equivalent to 0.76 years in their average education level, and 35% more children would finish primary school. One notable result is that the conditional transfers have a significant impact on the poor, but have no impact on those who are not poor. Furthermore, the program was seen to reduce educational inequalities between rural and urban areas.

1

1. Introduction

The Family Allowance Program (PRAF)

1

is one of the largest social programs of

the Honduran government, the 116

th

country in the world ranking of the Human

Development Index (HDI)

2

_{. The PRAF was initiated in 1990 and was originally}

composed of cash transfers, distributing grants to schools and health centers. It was

restructured in 1998 and now includes a project known as the Family Allowance

Program/Inter-American Development Bank – Phase II (henceforward, referred to as

PRAF II). Instead of merely alleviating poverty, PRAF II should also contribute to

eradicating the roots of poverty, the principal cause of which was considered to be the

low level of human capital. Hence, the program should provide a human capital

increment to the families, particularly an improvement in the educational levels and

health conditions of the children. As a consequence, PRAF II sought to act in two ways:

1) providing incentives to increase enrollment and attendance in primary schools, as

well as the utilization of preventive health care services; and 2) improving the quality of

the education and health services.

The allocation of the time of the economic agents between work, school and

leisure can be studied using the instruments of microeconomic theory proposed by

Becker (1962, 1965, 1981 and 1993). The basic model presents the family utility as a

function of consumption, work, number of children and school. Moreover, the

maximization problem has a budget restriction and a time restriction. Leisure is

preferred to work, but this leads to a lower consumption of market goods and goods

produced at home. The education of the families’ descendants involves costs in the

present, since the children do not work and there are the costs of their schooling. On the

other hand, it brings future benefits, in the form of an additional gain, due to the higher

educational level attained. Hence, the schooling is a human capital investment.

Therefore, the microeconomic models of family choices try to take account of the

decisions about consumption, child labor and – even – fertility

3

.

1 _{“Programa de Asignación Familiar”} 2 _{Human Development Report (2002)}

3 _{For examples, see Tomes (1981), Iglesias and Riboud (1988), Hill and O´Neill (1994) and Baland and}

2

A social policy can be justified by the existence of market failures, which

generate undesired incentives for the agents. When there are market failures that

produce inefficient allocations, the public intervention has the objective of correcting

the existing incentives. We are going to point out two market failures that can be related

to the inefficiency of individual decisions on education: 1) Imperfections in the credit

market; 2) Externalities of education.

The first market failure that can result in a low level of education being chosen

by the family is the imperfections in the credit market. Investment in human capital

should occur when the expected rate of return on this investment compensates for the

opportunity cost of the invested resources and the credit market does not prevent these

opportunities from being exploited. Many of the credit market imperfections are a result

of information asymmetries among the agents. Normally, the solution would be the

utilization of collateral as a guarantee to the creditors. However, investment in human

capital has no counterpart in assets that can be used as collateral (unlike the case of

physical capital). The consequence is greater difficulty in obtaining credit to finance the

children’s education.

The model in Baland & Robinson (2000) assumes a trade-off between child

labor and human capital accumulation. Even when the parents are altruists and the child

labor is socially inefficient, an equilibrium whereby the children work is possible

because the parents do not internalize all the negative effects of not sending the child to

school. This happens when the parents do not give bequests to their children or when

the capital markets are imperfect. In these circumstances, the parents to not make the

choice between child labor and future gains potential in a socially efficient way.

3

all agents benefit from a higher average level of human capital, no individual decision

on education has an appreciable effect on the average level of human capital in a

society, so nobody takes it into account when deciding on the allocation of his/her time.

Many effects can be internalized within small groups (firms or families) and can be

dealt with at a non-market level without creating gaps between the private and social

returns. On the other hand, basic discoveries that immediately become common

property seem exogenous to the majority of the agents

4

.

The rate of return on each year of education is related to the quality of the

school. A child in an environment where the students are weak produces less than in an

environment where the students are brighter. So, we can find externalities within

educational production: the level of a pupil in his/her class may be relevant. In this way,

the agents at each level of ability are more productive in an environment with a higher

level of human capital. Furthermore, students may have aid from other members of the

community, which reduces their learning costs. This assistance is likely to be greater,

the higher the schooling level and the volume of community resources. However, the

private decision of the agent does not incorporate the impact that the higher education of

each individual can have on the learning capacity of future generations of other families.

Thus, due to market failures, it is possible that the private decision on

investment in education occurs at a level below the optimal. Under these circumstances,

conditional cash transfer programs can be implemented due to the perception that there

are incentive problems regarding the families’ rational decision about the schooling of

their children. As a result, state intervention generating specific incentives for the agents

can help to raise the level of income and welfare. Brazil, Argentina, Chile, Colombia,

Bangladesh, Mexico, Pakistan, Nicaragua and Honduras are all countries that have

4 _{Lucas adds human capital in his model following the technology of the models of Arrow(1962),}

4

introduced programs that provide financial incentives to the economic agents with the

purpose of raising the investment in human capital.

Evaluation of the impact of a social program, that is, of the increment in

well-being related to the program, is important, especially when the objective is to qualify the

participants to achieve higher future incomes. This is the case of programs of incentives

to education and to the generation of income and employment, where the impact

analysis is essential to assess whether the expected benefits were in fact attained. The

results of the program evaluations provide a basis for any increments in the

interventions, generating better instruments and cost efficiency and effectiveness criteria

for the programs in countries with varying social and economic circumstances, technical

capabilities and budget restrictions. This research is fundamental to designing effective

social policy.

Several impact evaluation studies of social policies have been performed for

Mexico’s Education, Health, and Nutrition Program (Progresa)

5

. Skoufias and Parker

(2001) assess the impact of the conditional cash transfers of Progresa on child labor and

school attendance. Schultz (2000) focuses on the impact of Progresa on schooling and

on permanency in higher school grades. This study uses a binary indicator for the child

being or not being at school and does not consider child labor. Behrman, Sengupta and

Todd (2001) use a Markov schooling transition model and study the impact of the

conditional cash transfers of Progresa on school entry ages, drop-out rates, repetition

rates and the rates of school re-entry after dropping-out. Coady (2000) evaluates the

efficacy of costs in the program’s impact on attendance. A related study by

Demombynes (2001) investigates school attendance, considering the children’s

allocation of time between work and school. Dubois, Janvry and Sadoulet (2001)

estimate empirically the effects of the conditional cash transfers of Progresa on

students’ enrollment and performance in Mexico. The authors developed a dynamic

model of educational demand incorporating the effects of educational system incentives

to evaluate the program’s impact on children’s behavior in relation to education.

5

The recent work of Ferreira, Bourguignon and Leite (2003) makes an evaluation

of the School Grant

6

program in Brazil in an ex-ante formulation via micro-simulation.

Ex-ante evaluation methods simulate the effects of the program based on a model of

family choice. The authors simulate the effects of alternative designs for the program

and find a strong conditionality effect on school attendance, but little impact by the

transfers on the reduction of current poverty or on inequality. An ex-post evaluation, to

identify the real effects of the program through direct observation of the characteristics

of the individuals participating in the program, was not carried-out.

This paper studies the impact of PRAF II on the education of poor communities

in Honduras

7

. In the empirical work, we used difference-in-differences and

cross-sectional regressions to estimate the impact of the conditional cash transfers on the

families and of the transfers to Parent-Teacher Associations to improve the quality of

the schools, in terms of the educational indicators for enrollment, drop-out, attendance

and pass rates, as well as of child labor. While the conditional cash transfers tend to

raise class attendance and reduce drop-out, the supply-side interventions seem to have

no impact whatsoever. Hence, we concentrated on the impact of the demand-side

interventions and used a Markov schooling transition model to assess the impact of

these interventions on the number of years of schooling completed by children between

the ages of 6 and 12. The results show that the program effectively improves the

children’s progression through the grades in the short-term. After that, we used the

Markov matrices to simulate the long-term effects of exposure to the program on

13-year-old children, and the bootstrap procedure to verify the statistical significance of the

results of our simulation. We found a substantial impact by conditional cash transfers on

the educational distribution attained by 13-year-old children who remain exposed to the

program for seven years. With the program, these children would experience a 19%

increase in the average education level attained. Furthermore, the percentage of children

finishing primary school would rise by 35%. We found that girls achieve a slightly

higher average schooling than boys when we looked at the control group, and the

program accentuates this difference, making it statistically significant. As was to be

expected, poor children show a lower level of schooling than those who are not poor.

6 _{“Bolsa Escola” program.}

7_{The PRAF II project is continuously monitored and evaluated by the International Food Police Research}

6

Nevertheless, the impact of the demand-side interventions impact on the poor is

significantly greater than on those who are not poor, causing the difference in years of

schooling between these two social groups to be reduced by more than a half. Finally,

urban children have more education than rural ones, but we found that the treatment

reduces the existing differences between the children of the two areas.

The next section explains the PRAF II program and the data that had been

collected up until 2002. Section 3 describes the methodology used in this study. Section

4 presents some descriptive statistics of the data collected in 2000. Analysis of the

program’s impact on the educational results is conducted in section 5. Finally, section 6

presents the conclusion.

2. Description of PRAF II and of the data

2.1 The PRAF II program

In a broad sense, the objective of PRAF II is to improve the education and health

of the children in poor rural communities in Honduras. In the sphere of education, the

specific objectives are to stimulate the demand for educational services, to encourage

the “educational community” to participate in the children’s learning process and to

improve the quality of the schools.

The education and health interventions were conducted in two distinctive forms:

1) demand-side incentives, in the form of cash transfers conditioned on school

attendance and on visits to health centers by the program’s beneficiaries; and 2)

supply-side investments, aimed at improving the quality of the educational and health services

in poor rural areas.

7

attending school. A maximum of three children per family were eligible. Approximately

US$ 5 per month was paid for each child; therefore, a maximum of US$ 15 per month

might be given to each family. To be eligible, a child would have to be enrolled at

school by the end of March and would need to maintain an attendance rate of 85%. On

the other hand, the supply-side intervention in the field of education consisted of

payments to Parent-Teacher Associations (PTAs) organized around each primary

school. These associations needed to have legal status and draw up plans to improve the

quality of the education offered by their respective schools. Each PTA received an

average of US$ 4,000 per year, the amount varying between US$ 1,600 and

US$ 23,000, depending on the size of the school. The interventions in health followed a

format similar to those in education

8

.

PRAF II was implemented in 70 municipalities that were among the poorest in

Honduras. The municipalities were selected in October 1999, and the interventions

began in 2000. Evaluation of the program is based on randomized treatment and control

groups. To assess the impact of the demand-side and supply-side interventions, an

evaluation procedure was designed in which the 70 municipalities were randomly

distributed in four different groups, designated G

1

- G

4

:

G

1

= Demand-side intervention only (20 municipalities)

G

2

= Supply-side intervention only (10 municipalities)

G

3

= Demand-side and supply-side interventions (20 municipalities)

G

4

= Control group, without intervention (20 municipalities)

In each municipality of the groups G

1

, G

2

and G

3

, both the educational and

health projects were implemented. This design allows the measurement of the impact of

8 _{The demand-side interventions in the health area consisted of monetary transfers to pregnant women}

8

the demand-side intervention, the supply-side intervention, and of both types of

intervention.

There was no attempt to conceal the allocation after the day of the randomization

and, thus, the experiment was not blind. So, the individuals in our sample could know if

they would receive the cash transfers and whether there would be any improvements in

the services in their municipalities from the allotment day on. The administrators and

evaluators of the program also knew the allocation of the treatment. Due to the

requirement of residency in 2000, no family could become eligible for the cash transfers

by moving home after the randomization. On the other hand, it was not possible to

restrict the use of the improved educational and health services to the residents of a

specific municipality.

In the 70 municipalities in our experiment, 87% of the families had total

expenditures of less than US$2

per capita

/day, a standard that is frequently used to

characterize poverty at a global level (IFPRI, 2000). Moreover, fewer than 2% of the

families had expenditures of more than US$ 5

per capita

/day. This indicates that the

adoption of all the families residing in a municipality as beneficiaries of PRAF II would

not result in serious inclusion problems

9

. On the other hand, the implementation of a

selection system at the family level would have important costs, such as: 1) The

exclusion of a few families within a community where everyone is considered poor

could cause conflicts; 2) The financial and logistical costs of the training, supervision

and data generation; 3) The technical inefficiency of the predictive algorithms, because

there is no way to avoid inclusion and exclusion errors in the selection procedure; and

4) The application of a selection system creates incentives for the people to distort the

information about their current living standards. As a consequence, there was no

9_{The focalization problem is an immediate consequence of the implementation of a social policy: one}

9

restriction on access to the program, nor variation in the value of the PRAF II benefit as

a function of the

per capita

income of the children’s families. Also, the value of the

benefit, in the municipalities where the conditional cash transfers occurred, did not vary

between boys and girls, nor between rural and urban areas.

2.2 Data

Once the 70 municipalities had been chosen for the experiment, the baseline data

(year 2000) were collected before PRAF II was implemented in groups G

1

, G

2

and G

3

.

Eight communities were randomly selected in each municipality; and ten dwellings

were randomly selected in each community. Assuming one family per dwelling, this

implies a total baseline sample of 5,600 families. In fact, some of these dwellings

housed more than one family and, thus, the total number of selected families in the

baseline was 5,784. These families had a total of 30,588 members.

The baseline surveys took place between mid-August and mid-December in the

year 2000

10

. The educational and child labor data in the baseline were affected by

seasonal factors: the coffee harvest in Honduras begins in October and extends to

March, and many children work, harvesting coffee, during these months. Table 1

presents the interview month in 2000 according to the random treatment group. The

majority of the interviews for group G

1

(Demand) and group G

3

(Demand + Supply)

were carried out in the months of August, September and October. However, almost all

the interviews for group G

2

(Supply) and group G

4

(Control) were conducted in the

months of November and December, so these data were more affected by the impact of

the harvest.

10_{The family questionnaire collected data regarding: 1) Housing, education and employment of all family}

10

The next survey was conducted in 2002, between mid-May and mid-August. In

this second survey, an attempt was made to interview the families that had moved from

the dwellings where they were first interviewed in 2000. So, interviews were sought

with new families derived from the ones in the original baseline, whenever they

contained members who were in the initial survey and who belonged to the program’s

target population (pregnant women, breastfeeding mothers and children between 0 and

16 years old)

11

. According to Table 2, 87% of the children between 6 and 16 years of

age from the families interviewed in 2000 were found again in 2002 in their original

residences. Observing the four treatment groups, we verify that the percentages of

children interviewed in their original residences showed little variation. Approximately

7% of the children from the original families were now part of a new family and were

found in derived residences in 2002. Furthermore, 2% of the children from the survey in

2000 were not interviewed in 2002, due to absence or refusal, and 3% of the interviews

failed because the dwellings were not occupied or could not be found. Again, these

percentages are very similar across the groups G

1

-G

4

. The Pearson chi-squared test does

not reject the null hypothesis that the proportions of situations found in 2002 are

equivalent among these four treatment groups.

Besides the family questionna ire, data about the community characteristics were

collected

12

and questionnaires were administered in primary schools

13

.

3. Methodology

The central problem in the evaluation of social programs is the fact that the

individuals participating in the progr am can not be simultaneously observed in the

11_{Each municipality was surveyed before and after the introduction of the interventions. Following the}

same individuals, the sample remains representative of the municipality as it was constituted in the baseline. This ensures that it is possible to conduct an analysis of “intention to treat”.

12 _{The following data were collected in the 560 communities: 1) If the community has a primary school, a}

public hospital and public transportation; 2) Local daily wage rates for work in agricultural and non-agricultural activities; 3) The possibility of working outside the community; 4) Information about local crime; 5) Prices of a great number of food items.

13 _{Three schools were randomly selected in each municipality, making a total of 210 schools. The school}

11

alternative state of non treatment. To illustrate, let

Y

1

be the result of a given individual

or family in the treatment state (that is, during or after participation in the program) and

Y

0

be the result in the non treatment state (that is, without participating in the program).

Thus, the gain for any given individual or family from being treated by the program is

)

(

Y

₁

−

Y

₀

=

∆

. However, at a given point in time, a person is either in the treatment state

– where

Y

1

is observed and

Y

0

is not observed – or in the non treated state – where

Y

1

is

not observed and

Y

0

is observed. As the lack of

Y

1

or

Y

0

data prevents the measurement

of this gain for any individual, we require statistical methods to solve the problem.

The majority of studies about the evaluation of social programs attempt to verify

whether there is a change in the mean value of a variable among the participants,

compared to what they would experience if they were not participating in the program.

The answer to this question is summarized in a parameter called “the mean direct effect

of the treatment on the treated”. Formally, the mean effect (denoted by the expectancy

operator

E

) of the treatment on the treated (denoted by

T

=1) with characteristics

X

can

be expressed by:

(

T

X

) (

E

Y

Y

T

X

)

E

(

Y

T

X

) (

E

Y

T

X

)

E

∆

|

=

1

,

=

₁

−

₀

|

=

1

,

=

₁

|

=

1

,

−

₀

|

=

1

,

.

(1)

The term

E

(

Y

₁

|

T

=

1

,

X

)

can be reliably estimated from the experience of the

program’s participants. However, we are lacking the contra-factual term

(

Y

T

X

)

12

designs allow information about individuals in the control group to be used to construct

an estimation of the term

E

(

Y

₀

|

T

=

1

,

X

)

14

.

The empirical structure adopted in PRAF II provides a very flexible approach to

solving the evaluation problem. PRAF II is an experimental design with municipality

level randomization, instead of family or ind ividual level, in treatment and control

groups. The data were collected from all the families in the treatment and control groups

both before and after the introduction of the interventions. Given this experimental

design, it is possible to evaluate “the mean direct effect of the treatment on the treated”

using any of the estimators available in the literature, including the before-after

estimator (BADIF), the cross-sectional or first-difference estimator (CSDIF) and the

difference-in-differences or double-difference estimator (2DIF).

The BADIF compares the difference in the mean of the variable Y of a group A

of eligible individuals between the periods before and after the implementation of the

program (that is,

t

=0 and

t

=1,2,3,...)

(

)

(

)

[

=

|

=

1

,

=

1

]

−

[

(

(

=

0

)

|

=

1

,

=

1

)

]

=

E

Y

t

T

e

E

Y

t

T

e

BADIF

τ

for

τ

=1,2,3,…

(2)

where

e

assumes the value 1 if the individual is eligible for the program and zero

otherwise.

The CSDIF compares the differences in the mean of the variable Y between the

groups A and B during the periods after the implementation of the program (that is,

t

=1,2,3,…)

(

)

(

)

[

=

|

=

1

,

=

1

]

−

[

(

(

=

)

|

=

0

,

=

1

)

]

=

E

Y

t

T

e

E

Y

t

T

e

CSDIF

τ

τ

for

τ

=1,2,3,…

(3)

The 2DIF estimator compares the differences in the means of the results between

the groups A and B in the post-intervention survey with the differences in the means of

the results of the groups A and B in the pre-intervention survey:

14 _{For a more extensive discussion on the various solutions to the evaluation problem, see Heckman,}

13

(

)

(

)

[

]

[

(

(

)

)

]

(

)

(

)

[

0

|

1

,

1

]

[

(

(

0

)

|

0

,

1

)

]

1

,

0

|

1

,

1

|

2

=

=

=

−

=

=

=

−

=

=

=

−

=

=

=

=

e

T

t

Y

E

e

T

t

Y

E

e

T

t

Y

E

e

T

t

Y

E

DIF

τ

τ

for

τ

=1,2,3,…

(4)

Each of these estimators has advantages and deficiencies. However, for the

evaluation of programs, the 2DIF estimator is preferred to the CSDIF estimator. The

major advantage is that the former controls for any pre-existing differences in the

expected value of Y for families in the treatment and control groups. Measurement of

the program’s impact based exclusively on the post-program difference in the mean

level of an indicator between the treatment and control groups, as performed by the

CSDIF estimator, can result in potentially misleading conclusions. The extent to which

the CSDIF estimator can lead to biased results essentially depends on whether the

selection of the treatment and control groups was indeed random. Suitable

randomization in the selection of the municipalities ensures the absence of significant

pre-program differences in the results of the variables between the treatment and control

groups, that is:

(

)

(

)

[

E

Y

t

=

0

|

T

=

1

,

e

=

1

]

=

[

E

(

Y

(

t

=

0

)

|

T

=

0

,

e

=

1

)

]

.

(5)

The satisfaction of condition (5) also ensures that CSDIF=2DIF. That is, suitable

randomization implies that focalizing exclusively on the post-program comparisons

between treatment and control leads to unbiased conclusions about the program’s

impact.

14

Evaluation of the Program using a regression framework

Restricting the sample only to eligible families, the various estimators for

evaluating programs that control for the observed characteristics of the individuals,

families and municipalities can be obtained by estimating the regression equation in the

form:

( )

i

t

T

( )

i

R

T

( )

i

R

X

( )

i

t

Y

j j j TR

R

T

(

*

)

,

,

=

α

+

β

+

β

+

β

+

∑

θ

+

η

,

(6)

where

Y(i,t)

denotes the value of the result of the indicator for the family (or individual)

i

in the period

t

;

α

,

β

and

θ

are fixed parameters to be estimated;

T(i)

is a binary

variable that takes the value 1 if the family lives in a municipality of the treatment

group, and zero otherwise;

R

is a binary variable equal to 1 for the survey after the

introduction of the program and equal to zero for the survey before the introduction of

the program;

X

is a vector of characteristics of the individual, of the family and of the

municipality and

η

is an error term summarizing the influence of random disturbances.

The coefficient

β

_T

allows the conditional mean of the indicator to be different

between the eligible families in the treatment and control municipalities before the

introduction of the program. The combination of parameters

β

R

and

β

TR

allows for

differences between eligible families in the treatment and control municipalities after

the introduction of the program. An advantage of this specification is that the values of

the t-student statistics associated with some of these parameters provide direct tests for

some hypotheses that are of interest. For example, the t-student value associated with

the estimated value for

β

T

provides an equality test of the conditional mean of Y

15

Given the above specification, the mean conditional values of the indicators for

the treatment and control groups before and after the introduction of the program are the

following:

(

)

[

E

Y

|

T

=

1

,

R

=

1

,

X

]

=

α

+

β

T

+

β

R

+

β

TR

+

_∑

_j

θ

j

X

j

(7a)

(

)

[

E

Y

|

T

=

1

,

R

=

0

,

X

]

=

α

+

β

T

+

_∑

_j

θ

j

X

j

(7b)

(

)

[

E

Y

|

T

=

0

,

R

=

1

,

X

]

=

α

+

β

R

+

_∑

_j

θ

j

X

j

(7c)

(

)

[

E

Y

|

T

=

0

,

R

=

0

,

X

]

=

α

+

_∑

_j

θ

j

X

j

(7d)

Thus, the BADIF is given by the expression:

(

) (

)

[

E

Y

T

R

E

Y

T

R

]

R TR

BADIF

=

|

=

1

,

=

1

,

X

−

|

=

1

,

=

0

,

X

=

β +

β

.

(8)

while the CSDIF is given by:

(

) (

)

[

E

Y

T

R

E

Y

T

R

]

T TR

CSDIF

=

|

=

1

,

=

1

,

X

−

|

=

0

,

=

1

,

X

=

β +

β

.

(9)

Expression (8) indicates that the BADIF estimator includes any trend or

aggregated effects in the changes in indicator Y (summarized by the presence of the

term

β

R

). Similarly, expression (9) shows that the impact of the program estimated by

the CSDIF estimator includes any pre-program differences between the treatment and

control groups (summarized by the presence of the term

β

_T

).

16

(

)

[

]

[

(

)

]

(

)

[

E

Y

T

R

]

[

E

(

Y

T

R

)

]

TR

R

T

Y

E

R

T

Y

E

DIF

β

=

=

=

−

=

=

−

=

=

−

=

=

=

X

X

X

X

,

0

,

0

|

,

0

,

1

|

,

1

,

0

|

,

1

,

1

|

2

(10)

Using the terminology of Heckman et al. (1999), the parameter

β

TR

provides an

estimate of “the mean direct effect of the treatment on the treated”. It should be

emphasized that the effect of the program summarized by the parameter

β

_TR

includes

the effects of the operationa l efficiency or inefficiency of the program. It is probable

that the persistent delays in processing the forms in some municipalities and other

administrative obstacles could have led to weaker program impacts on the families in

these municipalities in relation to families from municipalities where the program

operated more efficiently.

Estimating the impacts using a Markov Model

Several schooling indicators can be affected by the program, including

enrollment, school attendance, initial age of school entry, drop-out rate, grade repetition

and school re-entry among those who dropped-out. To analyze the results of the

program’s impact in these areas, we adopted a Markov schooling transition model. We

made some modifications in the procedure suggested by Behrman, Sengupta and Todd

(2002) to estimate the transition probability matrices for children aged 6 to 12 and to

simulate the educational distribution of the 13-year-olds. This transition model provides

a convenient structure for studying the short-term and long-term impacts of

participation in the program on the level of education attained.

Let

f

g,a

be the proportion of children of age

a

that completed grade

g

. For

17

[

] [

]

          =

p

p

p

p

p

f

f

f

f

f

6 , 12 6 , 11 6 , 02 6 , 01 6 , 00 6 , 1 6 , 0 7 , 2 7 , 1 7 , 0

...

(11)

where the cell placed without value imposes the restriction that the students cannot

regress in their grades and, once a certain grade has been completed, there cannot be a

diminishing of the schooling level acquired. Denoting the transition matrix from age

a

to age

a+1

as

A

a

and the frequency vector at age

a

as

f

a

, we have

f

a+1

=

f

a

A

a

.

Short -term Impact

The program’s impact from one year to the next on children of a given age can

be obtained by comparing the estimated transition matrices for treatment and control:

A

A

T a T a

ˆ

ˆ

=1 =0

−

This comparison tells us how short-term participation in the program affects

progression through the grades at each age. The Pearson chi-squared tests can be used to

test whether the differences observed between treatment and control are statistically

significant. In the empirical work, we performed two types of test: 1) Tests of

equivalence between the treatment and the control matrices; 2) Tests of equivalence

between the individual lines of these matrices. The test statistic for testing the

equivalence between two matrices (for

T=1

and for

T=0

) is given by:

) ( ~ ) ( } 1 , 0 { , , 2 , , ,

ˆ

ˆ

ˆ

N

N

N

N

p

p

p

z r r c

T rc

a c r a c r a c r

T − _{− −}

∑ ∑

∈

χ

where r and c denote the r

-th

line and the c

-th

column.

N

r

and

N

c

denote, respectively, the

number of lines and columns of the transition matrices.

N

z

denotes the number of

elements set equal to zero.

p

ˆ

aT,r,c

are the estimated transition probabilities conditioned to

18

combining the treatment and control data). The Pearson test compares the conditioned

and unconditioned transition probabilities, which will be equal if the treatment has no

effect.

Long-term impact

To simulate the program’s impact for a synthetic cohort of the data, we need to

make two assumptions: 1) The transition probability matrices at each age level depend

only on the current level of education and on the current participation of the children in

the program; 2) The transition matrices for each age level do not change over time.

Let

s

a

be the level of schooling at age

a

,

T

the binary variable that indicates

whether the child participates in the program, and

H

a vector of treatment and history of

the level of schooling before age

a

. Formally, assumption (1) can be expressed as:

P(s

a+1

/s

a

,T

a

,H

a

)=P(s

a+1

/s

a

,T

a

)

(12)

Under assumptions (1) and (2), and given an initial vector of the proportion of

the states at a certain age, the estimated schooling state distribution at a higher age can

be obtained by:













=

∏

− = = = = 1

ˆ

ˆ

ˆ

a a s s t T a t T a t T s s

A

f

f

(13)

where

a

is a higher age than

a

s

.

Obtaining the confidence intervals using the

Bootstrap

method