Expanding vaccine efficacy estimation with dynamic models fitted to cross-sectional prevalence data post-licensure

ContentslistsavailableatScienceDirect

Epidemics

jo u r n al h om ep ag e :w w w . e l s e v i e r . c o m / l o c a t e / e p i d e m i c s

Expanding

vaccine

efficacy

estimation

with

dynamic

models

fitted

to

cross-sectional

prevalence

data

post-licensure

Erida

Gjini

a,∗

_,

_M.

_Gabriela

_M.

_Gomes

b,c,d

a_{InstitutoGulbenkiandeCiência,Apartado14,2781-901Oeiras,Portugal}

b_{CIBIO-InBIO,CentrodeInvestigac¸ãoemBiodiversidadeeRecursosGenéticos,UniversidadedePorto,Portugal}

c_{InstitutodeMatemáticaeEstatística,UniversidadedeSãoPaulo,Brazil}

d_{LiverpoolSchoolofTropicalMedicine,Liverpool,UnitedKingdom}

a

r

t

i

c

l

e

i

n

f

o

Articlehistory:

Received8May2015

Receivedinrevisedform2November2015

Accepted25November2015

Availableonline9December2015

Keywords:

Vaccinationmodel

Strainreplacement

Co-infection Competition

ODEparameterinference

a

b

s

t

r

a

c

t

The efficacy of vaccinesis typically estimated prior toimplementation, on the basisof

random-izedcontrolledtrials.Thisdoesnotpreclude,however,subsequentassessmentpost-licensure,while

mass-immunizationandnonlineartransmissionfeedbacksareinplace.Inthispaperweshowhow

cross-sectionalprevalencedatapost-vaccinationcanbeinterpretedintermsofpathogen

transmis-sionprocessesandvaccineparameters,usingadynamicepidemiologicalmodel.Weadvocatetheuseof

suchframeworksformodel-basedvaccineevaluationinthefield,fittingtrajectoriesofcross-sectional

prevalenceofpathogenstrainsbeforeandafterintervention.UsingSIandSISmodels,weillustratehow

prevalenceratiosinvaccinatedandnon-vaccinatedhostsdependontruevaccineefficacy,theabsolute

andrelativestrengthofcompetitionbetweentargetandnon-targetstrains,thetimepostfollow-up,

andtransmissionintensity.Wearguethatamechanisticapproachshouldbeaddedtovaccineefficacy

estimationagainstmulti-typepathogens,becauseitnaturallyaccountsforinter-straincompetitionand

indirecteffects,leadingtoarobustmeasureofindividualprotectionpercontact.Ourstudycallsfor

sys-tematicattentiontoepidemiologicalfeedbackswheninterpretingpopulationlevelimpact.Atabroader

level,ourparameterestimationprocedureprovidesapromisingproofofprincipleforageneralizable

frameworktoinfervaccineefficacypost-licensure.

©2015TheAuthors.PublishedbyElsevierB.V.ThisisanopenaccessarticleundertheCCBY-NC-ND

license(http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction

Mathematical epidemiological models for the dynamics of microparasiteinfectionshavealonghistoryofdevelopmentand usein thedesign andoptimizationofinterventionprogrammes (Andersonetal.,1992).Yet,manychallengesremaininapplying suchmodelsretrospectivelytointerpret andquantify interven-tion effects in host–pathogen systems (Keeling, 2005; O’Hagan etal.,2014;Wikramaratnaetal.,2014;Goeyvaertsetal.,2015). It is of public interest to quantify the relative effectiveness of differentcontrolstrategies,assesstheongoingchangesin trans-missiondynamicsfollowingsuchinterventions,andoptimizetheir designthroughacost–benefitanalysisforthefuture.Inthispaper, ourfocusisonvaccinationasatransmission-reducing interven-tion,andmorespecifically,inthecontextofendemicpathogens. Althoughtheamountofdataavailablefromepidemiologicaltrials,

∗ Correspondingauthor.

E-mailaddress:egjini@igc.gulbenkian.pt(E.Gjini).

cross-sectionalandlongitudinalsurveysisvastandrapidly increas-ing, our understanding and interpretation of such data on the basisoftransmissionmechanismsandepidemiologicalfeedbacks is limited.Thisisapparentfor manypathogensystems, includ-ingStreptococcuspneumoniae bacteria,humanpapillomaviruses, dengue,malaria,influenzaandrotaviruses.Currentlyseveral vac-cinesarebeingusedorcontemplatedtocontrolthesepathogens aroundtheworld(Comanduccietal.,2002;Insingaetal.,2007;del AngelandReyes-delValle,2013;Sabchareonetal.,2012;Agnandji et al., 2011; Black et al., 2000), and assessing theirefficacy is crucial.

Conceptualmodelscanplayakeyroleinthisassessment,firstby clearlydefiningthemeasuresofinterest,secondly,by distinguish-ingindividualfrompopulationindicators,andthirdly,byenabling ustoanticipatefutureoutcomesofvaccinationprogrammes.An importantvaccineparameterisefficacyagainstpathogen acquisi-tion,definedasreductionintheprobabilityofinfectionpercontact ofeachvaccinatedindividual(Haberetal.,1991).Beforeavaccine isintroduced,vaccineefficacyestimationistypically performed throughrandomizedcontrolledtrials,involvingasubsetofagiven

http://dx.doi.org/10.1016/j.epidem.2015.11.001

1755-4365/©2015TheAuthors.PublishedbyElsevierB.V.ThisisanopenaccessarticleundertheCCBY-NC-NDlicense(http://creativecommons.org/licenses/by-nc-nd/4.

population(Halloranetal.,2010).Suchvaccineevaluationstudies use1−RR(1minusriskratio),asameasureofefficacy,whereRRis someestimateofrelativeriskinvaccinatedvs.non-vaccinated indi-viduals.Thistendstoignoreindirecteffects(Halloranetal.,1991), suchasthechangesintransmissionmediatedbytheintervention, whichwhileinthetimeandcoverageoftrialsareindeedexpected tobenegligible,arenotquitenegligiblewhenmass-immunization isinplace(ShimandGalvani,2012).

Theassessmentofvaccinespost-licensureisalsoofinterest,and hereiswheredynamicmathematicalmodelscanbeuseful, along-sidestatisticalapproaches(BiondiandWeiss,2015;Croweetal., 2014;Andrewsetal.,2014).Thereareseveralreasonsforwhysuch a-posterioriassessmentisimportant.First,onlyadynamicmodel canproperlylinkpre-licensurevaccineexpectationsandobserved outcomesinapopulationundergoingimmunization,thereby pro-vidingavaliditytestforthenumericalestimatesofvaccineefficacy obtainedfromtrials,andavaliditytestforthepublic-health pro-jectionsmadeaprioriregardingeffectiveness,orpopulationlevel impact.Second,onlyadynamicmodelcantakeintoaccountina mechanisticmannerthetimesincetheonsetofthevaccination programme,regardlessofequilibriumrequirements(Rinta-Kokko etal.,2009),andconsidertheactualvaccinecoverageinagiven setting.Third,inthecontextofmulti-strainpathogens,where mul-tivalentvaccinestargetasubsetofpathogentypes,onlyadynamic modelcanproperlyimplementthenonlinearinteractionsbetween pathogentypes(Lipsitch,1997; Martchevaet al.,2008), arising throughdirectcompetition,cross-immunityorasymmetricvaccine protection.

Although there has been recognition of the importance of dynamictransmissionmodelsforvaccineassessment(Shimand Galvani,2012),fewstudiessofarhaveattemptedtoinfervaccine efficacyfittingdynamicmodelstotemporalprevalence trajecto-riespost-vaccination(Choietal.,2011;Gjinietal.,2016).Other approaches havesuggested thatprevalenceodds ratiosmaybe moresuitablethanprevalenceratiostodeterminevaccineefficacy, andthatspecialattentionmustbegiventothetimeofsampling post-vaccination(Scottetal.,2014).AnotherstudybyOmorietal. (2012)hasuseddynamic models(SISand SIR)toillustratethe biasinodds-ratioestimatorsofvaccineefficacyfortwo compet-ingpathogentypes,buttheirestimationwasbasedonprevalences atendemicsteadystateonly,posinga strongrestrictiononthe method.A recent study by van Boven et al. (2013) dealswith vaccineefficacyestimationinanepidemicscenario,andapplies adynamicmodellingframeworktomumpsoutbreakdatainthe Netherlands.

Here, we advocate a similar dynamic spirit in the context of endemic diseases. We propose a novel approach to vaccine efficacy estimation using cross-sectional prevalence data inte-gratedwithindynamicmathematicalmodels.Thisenablesadeeper understandingofvaccineperformanceinthefield,asmediatedby transmissionintensity,competitionbetweenpathogensubtypes andhost factors.Whenvaccinecoverage is high, the transmis-sioncycleencompassesvaccinatedandnon-vaccinatedindividuals interactingthroughcontact,thus affectingand beingsubjectto adynamicforceofinfection.Withagraduallydiminishing expo-suretovaccinetypes,in polymorphicsystems, subtyperelative frequenciescanchangeinthepopulationfromthecombinedeffects of vaccination and interactions between target and non-target pathogentypes.Ifavaccineinducesareplacementphenomenon, asithasbeenarguedforpneumococcus(Weinbergeretal.,2011) andHPV(BiondiandWeiss,2015),vaccineefficacyagainsttargeted pathogenstrains,canbeestimatedwhilethesestrainsarestillin circulation,namelywhiletypereplacementisnotyetcomplete, andsufficientinformationcanbeextracted.Itispreciselyinthis intermediatedynamicphasethatmostvaccineobservational stud-iesareconducted,andwhereepidemiologicalfeedbacks,including

changesinexposureandinteractionbetweenmultiplestrains,are mostlikelytoplayarole.

Tocorrectlycapturealltheseprocesses,more refined math-ematical frameworks are needed. This requires going beyond direct statistical comparisons, based on static data, e.g. snap-shotprevalenceoddsratiosfromobservationalstudies(Thompson etal.,1998),ortheindirectcohortmethodforcase–controldata (Andrewsetal.,2011),whichneglectspathogensubtype interfer-encealtogether.Evenmoreimportantly,thecohortmethodfails toacknowledgethattheprobabilityofinfectionofanindividual dependsontheinfectionprevalenceinthepopulation,i.e.onthe infectionstatusofothers.

Withadynamicmodellingapproach,instead,theproblemof constanthazardratios(Hernán,2010)canbecircumvented,ascan limitationsoftheindirectcohortmethod(MoberleyandAndrews, 2014)forpurposes ofvaccine efficacyestimation.Furthermore, datacanbeinterpretedrelaxingthestationarityrequirementand accountingforpathogentypereplacement.Otherstatistical esti-mationmethodssuchasincidencedensitysampling(Richardson, 2004),mightalsonotrequiretheassumptionofstationarity,but theydonotdealwithcompetitioninmulti-strainpathogen sys-tems.

Thedefinitionofvaccineefficacythatweconsiderinthispaper hasa clear biological meaning: reduction of the probability of pathogen acquisition per contact, which enables extrapolation beyondasinglestudypopulation.Thiscontrastsclassicalestimates ofvaccineefficacythatarebasedoncomparingattackratesin vac-cinatedandunvaccinatedindividuals(thecohortmethod),orthose thatusethevaccinationstatusoftheinfectedindividualsrelativeto thepopulationvaccinationcoverage(thescreeningmethod).Such vaccineefficacyindicatorslackaclearbiologicalmeaning,which makesinterpretationproblematic,andpreventsanticipationofthe criticalvaccinationcoveragesneededtoreachcertaindesired out-comes.

Inthisstudy,wearguethattemporaleffectsofvaccination pro-grammescanbeaddressedthroughdynamicmathematicalmodels, where parameters of efficacyare explicitlydefined in terms of underlyingtransmission mechanisms,and where epidemiologi-calfeedbacksamongimmunizedandnon-immunizedindividuals, andbetweenpathogenstrainsarecorrectlyaccountedfor.Inthe interestofsimplicityandclarity,weonlyconsiderminimal epi-demiological modelsto illustrate vaccine effects on single and multipleinfectionwithdifferentpathogentypes,buttheuncovered trendsshouldapplyinsimilarveintomorecomplexvaccination scenarios (Halloran et al., 1991, 2010). We delineate a proof-of-concept inferenceprocedure, based onODEmodelfitting,to cross-sectionaldatacollectedoverdifferenttimepointsafter vac-cineimplementation.

2. Materialsandmethods

Tobuildintuitioninourreader,initiallywepresent susceptible-infected (SI) model frameworks accounting for one and two pathogen types, while the susceptible-infected-recovered (SIR) analogues are elaborated in the Supplementary Text S2. Then weproceedtosusceptible-infected-susceptible(SIS)modelswith many-typepathogens,groupedaccordingtowhethertheyare tar-getedbyapolyvalentvaccineornot.Wealwaysassumethatthe vaccineis effective againsttype 1 pathogen(SI/SIR models), or againstpathogensubtypesingroup1(SISsetting).Themodeof actionofthevaccineweconsiderisleaky(Halloranetal.,1991), andthevaccineefficacyisdefinedasthereductionin probabil-ityof infection/pathogenacquisition percontact. Noticethat in thispaper,wewillusetheterms‘infection’and‘carriage’ inter-changeably.Asthesourceofprevalencedata,weconsideractive

surveillanceprogrammespre-andpost-vaccinationina popula-tion,whereby thecarriage statusand pathogentype(s) of each screenedindividualaredetermined.Thebasicstructureofthe mod-elsintheabsenceofaninterventionisgiveninFig.1.

2.1. SImodel–1pathogentype(n=1)

The first model we consider for illustration is a simple susceptible-infectedmodelwithonepathogentypethatis directly-transmitted(SI-1).Withacontinuousvaccinationprogrammein place,theproportionsofhostsindifferentcompartmentsaregiven by:

⎧

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎩

Non-vaccinated hosts dS0 dt =(1−)−ˇS 0_(I0_+I1_)−S0 dI0 dt =ˇS 0_(I0_+I1_)−I0

⎧

⎪

⎪

⎨

⎪

⎪

⎩

Vaccinated hosts dS1 dt =−ˇwS 1_(I0_+I1_)−S1 dI1 dt =ˇwS 1_(I0_+I1_)−I1

wheresubscripts0and1indicatenon-vaccinatedandvaccinated hoststatus,respectively.Theparameterˇistheper-capita trans-missioncoefficientandthebirthrate(equaltothedeathrate). Hostsarebornwithalifeexpectancyof1/.Thevaccination cov-erageatbirthisandvaccineefficacyisgivenby1−w,assuming a homogeneouseffect(leaky vaccine). In theabsenceof a vac-cine,forsuchpathogentopersist,thebasicreproductionnumber (Heesterbeek,2000)R0=ˇ/mustexceed1,andthehigherR0is,

thehigherthepathogenprevalence.

2.2. SImodelwith2competingpathogentypes(n=2)

Extendingtheabovemodeltotwopathogentypes(SI-2),we have:

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

Non-vaccinated hosts dS0 dt =(1−)−(1+2)S 0_−S0 dI01 dt =1S 0_−I0 112−I01 dI02 dt =2S 0_−I0 221−I02 dI0 12 dt =12I 0 1+21I20−I120

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

Vaccinated hosts dS1 dt =−(w1+2)S 1_−S1 dI11 dt =w1S 1_−I1 112−I11 dI12 dt =2S 1_−I1 22w1−I12 dI1 12 dt =12I 1 1+2w1I21−I112,

where1=ˇ(I01+I012/2+I11+I121 /2)and2=ˇ(I20+I012/2+I21+

I1

12/2). The above equations track the proportions of

non-vaccinatedandvaccinatedhostsin4classes:susceptibles,S,hosts carryingpathogentype1,I1,hostscarrying pathogentype2,I2,

andco-infectedhostsI12,whereS0+



I0_=1−andS1₊



_I1_=

respectively for non-vaccinated and vaccinated host compart-ments. Overall we have: S+



I=1. Upon primary pathogen exposure,asusceptiblehostcanacquirepathogentype1ortype 2.Theforcesofinfection(FOI)dependexplicitlyonprevalence: 1(t)fortype 1,and 2(t)fortype 2.Singlecarriers block

sub-sequentacquisitionof thesame typebut canacquiretheother pathogen type witha reduced rate i, this due to competition

betweentheresidentandthenewcomerstrain.Assumingno clear-ance,thereisanendemicpersistenceequilibriumwheneverˇ> intheabsenceofintervention(=0).Stablecoexistencebetween types requires further constraints on 1 and 2, as shown in

Fig.S1.

2.3. SISmodelfor2groupsofpathogentypes(n2)

For pathogens with larger antigenic diversity, many types can circulate simultaneously in a host population. The con-stituentpathogensubtypescanbeequivalentinmostlife-history traits,includingbasictransmissionandclearancepotential. How-ever,whenvaccinationwithpolyvalentvaccinesisconsidered,it becomespracticaltoaggregatethemintotypesofgroup1, (vac-cinetypes,i.e.thosethatwillbetargetedbyanintervention),and group2(remainingones,ornon-vaccinetypes).Typicallyahost mayacquireanypathogentypeofgroup1,orgroup2,orbea dou-ble carrieroftwotypes:fromgroup1,fromgroup2,oroneof each.

Acquisitionof a second pathogentype generally occursat a reducedratecomparedtosinglecarriage,duetodirectcompetition betweentheresidentandnewcomerpathogentypes.Competition betweenanytwopathogentypesisrepresentedbyparameter<1. Whentheybelongtothesamegroupweapplyanextrareduction factortoaccountforthefactthatthereisonelesstypeavailable forcolonizationwithinthesamegroup.Inthissenseshouldbe seenasafactorrepresentingdepletionofavailabletypes,which canexertasmallorlargeeffectoncoinfectionbytypeswithinthe samegroup,dependingonwhetherthegrouphasmanytypesor justafew.

Althoughinprincipleanyindividualcarriageepisodecaninduce sometype-specificimmunity,weconsiderthemagnitudeofsuch immunity negligible when assessed on the entire pool of all pathogentypesfromtheparentgroup.Thereis,however, noth-ingtoprecludetheinclusion ofcumulative immunityinmodel extensionsinformedbydata onspecific pathogens.Meanwhile, weareeffectivelyworkingwithSISepidemiologicaldynamicsat thelevel of groupsof subtypes.The systemwithvaccinationis givenby:

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

Non-vaccinated hosts dS0 dt =(1−)−(1+2+)S 0_+(1−−S0₎ dI01 dt =1S 0_−I0 1(2+1)−(+)I10 dI20 dt =2S 0₋ I20(1+2)−(+)I20 dI011 dt =1I 0 1−(+)I 0 11 dI022 dt =2I 0 2−(+)I220 dI012 dt =(2I 0 1+1I20)−(+)I120

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

Vaccinated hosts dS1 dt =−(w1+2)S 1_−S1_+(−S1₎ dI11 dt =w1S 1₋ I11(2+w1)−(+)I11 dI12 dt =2S 1₋ I21(w1+2)−(+)I12 dI111 dt =w1I 1 1−(+)I111 dI122 dt =2I 1 2−(+)I221 dI112 dt =(2I 1 1+w1I12)−(+)I121 ,

Fig.1.Modeldiagramsintheabsenceofvaccination.Thedirectcompetitionphenomenonbetweenpathogensubtypesisrepresentedbythegreyarrows,modifyingthe

rateoftransitionfromsingletodualcolonization.

wheretheparameterdenotestheclearancerateofeachcarriage episode,assumedequalfor singleand dualcarriage,asinsome pneumococcusmodels(Colijnetal.,2009;Mitchelletal.,2015).The forces of infection are: 1=ˇ(I10+I110 +I120 /2+I11+I111 +I112/2)

and2=ˇ(I20+I220 +I120 /2+I21+I221 +I121 /2).Inthisformulation,

aswellasintheSI-2formulation,itisassumedthatanindividual carryingtwopathogensubtypesisonly50%infectiousforeither oneofthem,aspecialcaseofneutralnullmodels(Lipsitchetal., 2009).

Stabilityoftheendemicequilibriumintheabsenceof interven-tionrequiresR0=ˇ/(+)>1.Aconditionforstablecoexistenceof

thetwogroupsofpathogensubtypesintheabsenceofintervention isthat<1,whichisalwaystruegiventhemeaningof.Asinthe previousmodels,weassumethatafractionofthepopulationis vaccinated,withavaccinethatreducessusceptibilitytotargeted pathogentypes(heregroup1)byafactorw(0≤w≤1).Themodel canbegeneralizedtoincludefinerscalesofcompetitionbetween pathogentypes,ifimportantasymmetrieshappentobeimpliedby specificpathogendata.

2.4. Inferringvaccineefficacy

In all our transmission models, vaccineefficacy is given by VE=1−w, varying between 0 and 1, denoting the reduction in probability of pathogen acquisition per contact in vacci-natedindividuals relativetothosenon-vaccinated(Haberetal., 1991). We explore these models, with relation to retrospec-tiveanalysesofcross-sectionalprevalencedatapost-vaccination. Such data may be available through active sampling in obser-vational studies of populations subject to mass-immunization, and we assume they reflect pathogen carriage irrespective of symptoms.

We initially generate cross-sectional prevalence data post-vaccinationthroughnumericalsimulationofmodelsystemswith fixedparameters.Wegoontocomparetheratiooftargetpathogen prevalence(type1orgroup1)invaccinatedvs.non-vaccinated hosts,withthetruerelativeriskw,andwesystematicallyshow howthediscrepancybetweenthetwovarieswithtimeof obser-vation,withtransmissionintensityand competitionparameters betweenpathogensubtypes.ThisisperformedfortheSI-2and SIS-2models(butseeSupplementTextS2foranSIR-2formulation). Whatweproposeasasolutionistofitthefulldynamicmodelwith vaccinationtoprevalencetrajectories,andinferinthisway vac-cineefficacy,simultaneouslywithotherparameters.Noticethat thisapproachisdifferentfromtheonesuggestedbyScottetal. (2014),wheretheyadvocatethatsnapshotprevalenceratioitself, orprevalence-oddsratiobeused,onlyatappropriatetimes post-vaccination.Herewearenotproposingadirectuseofanystatic observationbutratheradynamicmodelfittomultipletemporal observationsofrelativeandabsoluteprevalenceofcarriage post-vaccination.

To motivate this approach, we illustrate the discrepancy betweenprevalenceratioandvaccineefficacy.Thus,webeginby providinganalyticalinsightusingthesimplerSI-1model,where itiseasytoderiveendemicprevalenceequilibriapre-and post-vaccination.Subsequentlyweexplorenumericallythemodelswith straincompetition.

3. Results

3.1. VaccineefficacyandendemicequilibriaoftheSI-1model Pathogencarriageprevalenceatthestableendemicequilibrium intheabsenceofvaccination(Section2.1,=0)isgivenbyIpre∗ =

1−1/R0,whereR0=ˇ/.Inthepresenceofvaccination(>0),by

settingthedifferentialequationstozero,wecancomputethenew endemicequilibriumpost-vaccine:

I∗=w(R0−1)−1+



4(1−)R0(1−w)w+[w(R0+1)−1]2

2R0w .

(1) Prevalencepost-vaccinebecomesclearlyanonlinearfunctionof basicreproductivenumberR0,vaccineefficacy(1−w),and

cov-erage  (Fig.2a).When thecoverage is perfect =1, we have I∗=1−(1/R0w).Onecanseefor examplethat toeliminatethe

pathogen,withperfectcoverage,wneedstobehigherthan1/R0,

orwithimperfectcoverage,thefractionvaccinatedandindividual vaccineprotectionwcanbejointlytraded-offagainstoneanother indifferentcriticalcombinations(i.e.thosethatsatisfyI*_=0,inEq.

(1)).

These analytical expressions also illustratethat if we know theendemicprevalenceequilibriapre-andpost-vaccine,andthe vaccinationcoverage ,we caninfer simultaneouslyR0 and w,

hence vaccine efficacy (1−w), by comparingpre-vaccine with post-vaccine cross-sectional prevalence of carriage. The above expressioncanbeusedtoassesthe‘overall’effectivenessofa vac-cinationprogram,definedasthereductioninthetransmissionrate foranaverageindividualinapopulationwithavaccination pro-gramatagivenlevelofcoveragecomparedtoanaverageindividual inacomparablepopulationwithnovaccinationprogram(Halloran etal.,1991;Halloran,2006).

Similarly,ifweareinterestedintherelativeequilibrium preva-lence ratio (PR*_{) of infection in vaccinated and non-vaccinated}

hosts,wecanalsoobtainitanalyticallyfromthemodel:

PR∗= I1∗/ I0∗_/(1−) = −1+w−R0w(1−2)+



4(1−)R0(1−w)w+[w(R0+1)−1]2 2R0w , (2)

0 5 10 15 20 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 R₀ Equilibrium prevalence I *

A

0 5 10 15 20 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 w=0.5 R₀

Relative prevalence ratio PR

*

B

w=0.1

w=0.5 w=0.1

Fig.2. Infectionprevalenceatpost-vaccinationequilibriumvs.transmissionintensity(SI-1model).(A)Absoluteprevalenceofcarriage.(B)Relativeprevalenceratioin

vaccinatedvs.non-vaccinatedhosts.Thedifferentlinesfromtoptobottomcorrespondtodifferentvaluesassumedforvaccineefficacy(VE=1−w=50%,90%)while

keepingfixedcoverage=0.5.Theratioofrelativeinfectionprevalenceinvaccinatedvs.non-vaccinatedindividualsatequilibriumdoesnotreflectthesamevalueofrelative

risk(w=1−VE)astransmissionintensityR0changes.Whileinthisfigure,allthepointsalongeachlinereflectscenarioswiththesamevaccineefficacy,theprevalenceratio

observedisdifferentforeachtransmissionintensityR0.Thisshowsthatonecannotsimplytranslate1−prevalenceratioatequilibriumtoadirectlyinterpretablemeasure

ofvaccineefficacy,asdefinedbythe1−wparameter,encapsulatingtheinstantaneousper-unitexposureprotectionfactorexperiencedbyeachvaccinatedindividual.We

omitanalysisofthecorrespondingSIS-1typesystem,astheresultsareidenticaltotheSI-1model.

whichrevealsthatthisquantityvariesnotonlywiththevaccine parametersandw,butalsowithtransmissionintensity,here rep-resentedbyR0 (Fig.2b).Thisanalyticalresultclearlystatesthat

prevalenceratio,evenatequilibrium,isunsuitableasadirect indi-catorofvaccineefficacy,aspreviouslynoted(Haberetal.,1991; ShimandGalvani,2012).Infact,PR*_{iscommonlygreaterthanw}

exceptforspecialtripartitecombinationsof(R0,,w).

Nonethe-less,itispreciselysuchnonlinearrelationshipbetweenR0,vaccine

efficacy,andprevalence,thatliesattheheartoftypicalvaccination programmesagainstchildhooddiseases(typicallycharacterizedby SIRdynamics),wherethecriticalvaccinationcoverageneededto eliminateapathogenhasbeendeterminedthrough mathemati-calmodels,andappliedtocontrolmeasles,smallpox,mumps,and rubella.

Interestingly, when considering another ratio in our epi-demiologicalmodel, namely theprevalence odds ratio (POR),at equilibriumpost-vaccineweget:

POR∗= I1∗/S1∗

I0∗/S0∗ =w, (3)

confirmingtheclassicalresultthattheprevalence-odds-ratio,at leastinasimpleSI-1setting,isaperfectestimatoroftruerelative risk,providedweareatequilibrium.Thishasbeenshown previ-ouslybystudiescomparingprevalenceratiosandprevalence-odds ratios(Greenland,1987;Strömberg,1994;Thompsonetal.,1998). Next,webrieflyaddresshowinpolymorphicpathogensystems, therelationshipbetweenwandprevalenceratiodepends system-atically also onthe strength of competition betweenpathogen types,andfinalizewithourproposaltoinferwasafundamental parameterofadynamicmodelthatcanbefittedtocross-sectional datapre-andpost-vaccination.

3.2. Whenprevalenceratiopost-vaccinationismodulatedby straincompetitionandreplacement

Hereweconsidertheeffectsofsubtypecompetitioninpathogen systemswithmanystrains.Wesimulatehypotheticalvaccination scenariosusingthedynamicSI-2andSIS-2models(Sections 2.2 and2.3)withtwointeractingstrainsorgroupsofstrainsforTtime

unitspost-vaccination.Timeismeasuredinsameunitsashostlife expectancy1/.Focusingonthepathogentypestargetedbythe vaccine,andneglectingthecontributionofdualcarriersofvaccine andnon-vaccinetypes,theinstantaneousprevalenceratiooftype 1orgroup1pathogen,inthetwomodelsisdefinedas:

PR(t)=I 1 1(t) I0 1(t) ×1−  (SI-2model) PR(t)=I 1 1(t)+I111 (t) I0 1(t)+I110 (t) ×1−  (SIS-2model) (4)

IntheSI-2model,weconsidertwopossiblescenarios:(i)vaccine targetstype1,whentype1isdominant,and(ii)vaccinetargetstype 1,whentype2isdominantpriortointervention.Thesescenarios aredeterminedbytheratioofcompetitioncoefficientsı=1/2:

withtype 1dominanceifı<1andtype 2dominanceviceversa. When1=2,thetwopathogentypesstablycoexistatequal

abun-dances.Toexploretype1dominance,inFig.3,weset2=1and

consider1between0and1.Toexploretype2dominancebyan

equalamount,weset1=1andvary2.

InagreementwiththeSI-1analysis(Fig.2),theprevalenceratio fromtheSI-2model(Eq.(4))isalsomostcommonlyhigherthan thetruerelativeriskw(Fig.3).Inaddition,herewecanseethat prevalenceratioisverysensitivetotherelativemagnitudesof com-petitioncoefficients,especiallywhenthetargetstrainisdominant. Forvaccinesthattargetthedominanttypeweexpectwtobe over-estimated(vaccineefficacyunder-estimated),byalargeramount thanforvaccinesthattargetthenon-dominanttype.Thisindicates theimportanceofusingflexiblemodelformalismsthat accommo-datecompetitionparameterstobeestimatedsimultaneouslywith vaccineefficacy.Fromtheorderingofthecurvesinthisfigure,we canexpectthatusingamodelthatignorescompetition(1=2=1)

wouldgenerallyleadtoanoverestimationofw(underestimation ofvaccineefficacy)iftype1isdominantintherealsystem,andthe oppositeiftype2isdominant.

Intuitively,thiscanbeunderstoodbyseeingthevaccineand theintrinsiccompetitiveinteractionsinthesystemastwoforces thataffecttheconstituentstraindynamics.Whentype1is dom-inant,anda vaccinetargetstype1,thenaturaltendencyof the

0 20 40 0 0.2 0.4 0.6 0.8 1

Time post−vaccination (years)

PR(t), type 1 β =0.032 w 0 20 40 0 0.2 0.4 0.6 0.8 1

Time post−vaccination (years)

PR(t), type 1 β =0.32 w 0 20 40 0 0.2 0.4 0.6 0.8 1

Time post−vaccination (years)

PR(t), type 1 β =0.032 w 0 20 40 0 0.2 0.4 0.6 0.8 1

Time post−vaccination (years)

PR(t), type 1

β =0.32

w

Fig.3.Prevalenceratiooftargettypeinvaccinatedandnon-vaccinatedhostsvs.truerelativeriskintheSI-2model.Type1canbedominantpriortovaccination

(0.2≤1/2≤1),oralternatively,type2canbedominant(0.2≤2/1≤1).Thecolouredlinesfrombluetoredcorrespondtoincreasingvaluesofthecompetitionratio,

1/2(abovetheredcurve)and2/1from0.2to1(belowtheredcurve).Althoughastimeincreasestheprevalenceratiotendstowardstheparameterw,thevalueremains

biasedaboveduetocompetitionbetweentypesinbothcases,moresoiftype1isthebettercompetitor.Otherparametervalues:=0.0167,=0.5.Vaccinetargetstype

1.Differentvaluesofwareassumedintop/bottompanels:w=0.5andw=0.3respectively.Initialconditionsatendemicequilibrium.Timeisinunitsofyears,wherethe

averagelifeexpectancyisequalto60years.Thelowtransmissioncases(ˇ=0.032),correspondtoR0=1.9.Whilethehightransmissioncases(ˇ=0.32)correspondtoR0=19.

FortheanalogousfigurewithPR(t)takingintoaccountmixedcarriageI12seeFig.S4.(Forinterpretationofthereferencestocolourinthisfigurelegend,thereaderisreferred

tothewebversionofthisarticle.)

systemand thevaccinegoin oppositedirections, thusthe sys-temrespondsmoreslowly.Whentype2isdominantinstead,the vaccinetargeting1anddirectcompetitionactinthesame direc-tion,andweexpect fasterpropagationofvaccineeffects inthe entiresystem,whereby also thedifferencebetween vaccinated andnon-vaccinatedhostsemergesearlier(PR(t)→wfasterpost vaccination).

Inthismodel,competitionaffectsalsotheconvergenceofPOR (prevalenceoddsratio)oftype1pathogentow.Weexpectfromthe equilibriumanalysisoftheSI-1model,thatPOR(t)shouldbecloser tothetruerelativeriskwthanPR(t).Thisiswhatwefind.POR(t) tendsfastertowardwaftervaccinationisinplaceintheSI-2model. However,inthetwo-strainsystemwithdirectcompetition,POR(t) isalsoaffectedbyrelativestraindominance(Fig.S2),althoughto alesserextentthanPR(t),anddependingon1/2,italsomaynot

alwaysreachasymptoticallythetruew.

IntheSIS-2modelwith2groupsofpathogentypes,weuncover aregimewheretheprevalenceratioofaggregatedpathogentypes targetedbythevaccine(insingleandmultiplecarriage:I1+I11)

invaccinatedvs.non-vaccinatedhostsmaybebelowrelativerisk, andthusyieldanover-estimationofvaccineefficacyifitwereto beusedforthispurpose.Thisoccursforlargetransmission inten-sityˇ,andcloseto1(Fig.4).Whenissmallinstead,thepattern PR(t)>wcouldpersistindefinitely.Asincreases,thereisinitially adownwardbiasifobservationsaremadetoosoonafter interven-tiononset,andonlyaftersometime doesPR(t)tendtothetrue valueofrelativerisk.Dependingontheexactmagnitudeofˇand w,thedeviationofrelativeprevalenceratiofromwcouldpersistfor alongtimeafterthestartofvaccination(Fig.S3).Indeed,as trans-missionintensityincreases,theprevalenceofmultiplecarriagein thehostpopulationincreases,thusamplifyinganyindirecteffects ofcompetitionbetweentypes.

Noticethattheimpactofcompetitionhierarchiesonprevalence ratioPR(t)isverysensitivetohowthisprevalenceratioisdefined: whetherittakesintoaccountornot,multiplemixedcarriageof 1and2:I12.InFigs.S4andS5,weshowtheanalogousscenarios

ofFigs.3and4,foraprevalenceratiothattakesintoaccountthe globalFOIoftargettype(s),summingalsothecontributionI12/2of

themixedcarriagehostclass.ThesensitivityofPR(t)tocompetition hierarchies,inthiscasedecreasesintheSI-2model,andincreases drasticallyintheSIS-2model,especiallyforhighˇ,withparallel exacerbateddeviationfromw,indicatingthatmixedcarriageof targetandnon-targettypescontributesmoreconfoundingfrom indirectvaccineeffects.

3.3. Usingadynamicmodeltoinfervaccineefficacy

Asbrieflyillustratedabove,theuseofprevalenceratiostoassess vaccineefficacypresentsfourmainproblems:

(i)eveninthebestcaseofexplicitlymatchingpre-vaccineand post-vaccineequilibria(SI-1model),thusevenwhensatisfying thestationarityassumption,theindirecttransmissioneffects mediatedbythevaccinearetypicallyconfoundedinthe preva-lenceratiooftargettypesinvaccinatedandnon-vaccinated individuals(subjecttotheinterplaybetweenR0 andvaccine

coverage);

(ii)in generalfor observationalstudies,the prevalenceratiois dependentonthetimeofthesurveypost-vaccination,which makesitnon-robustandhardtocompareorinterpretacross settings,withoutmechanisticallyembeddingtimeinthe anal-ysis;

(iii)in polymorphicpathogensystems,prevalence-based indica-torscanbebiaseddependingonthemagnitudeanddirection

0 10 20 30 40 0.4 0.6 0.8 1 Time (months) PR(t), group 1 w σ =1 β =2 0 10 20 30 40 0.4 0.6 0.8 1 Time (months) PR(t), group 1 w σ =1 β =6 0 10 20 30 40 0.4 0.6 0.8 1 Time (months) PR(t), group 1 w σ =0.5 β =2 0 10 20 30 40 0.4 0.6 0.8 1 Time (months) PR(t), group 1 w σ =0.5 β =6

Fig.4. Deviationoftheprevalenceratiofromtruevaccine-inducedprotectionintheSIS-2modeldependsontransmissionrateandthemagnitudeofcompetition.Higher

transmissionintensity,leadstolargerdiscrepancyintheimmediatetimescaleaftervaccineintroduction.Fixedparametervalues=0.02,=0.5,=0.6andcompetition

coefficient=1and0.5respectivelyinthetop,andbottompanels.Initialconditionsatendemicequilibriumwherebothgroupsoftypescoexistatequalabundance.VE=50%

(dashedlinedepictsw=1−VE).Thedepletionfactoreffectingwithin-groupcoinfectionisvariedbetween0.1and0.9(colouredlinesfrombluetored).Increasingoverall

competitioninthesystemmakesindirecteffectsweaker,thusprevalenceratiosreflectmoreaccuratelytruerelativeriskw.Timeunitsaremonths,ashereweconsidera

childhooddisease,andthemeanageofhostsis50months.Bytype1werefertoallpathogentypesingroup1targetedbythepolyvalentvaccine.Fortheanalogousfigure

withPR(t)takingintoaccountmixedcarriageI12seeFig.S5.(Forinterpretationofthereferencestocolourinthisfigurelegend,thereaderisreferredtothewebversionof

thisarticle.)

of pre-existing competitiveinteractions between strainsor groupsofstrains,whosevaluesarehardtoknowandfactor outapriori.

(iv)in addition,inpolymorphicsystemsshapedbycompetition among strains, how the prevalenceratio of target types is definedintermsofsingleandmultiplecarriageofpathogentype combinationsisacriticaldeterminantofthediscrepancywith truerelativerisk,asitispreciselythedetailsofcompetition at co-colonizationthatdriveindirecteffects ofthevaccine, especiallyathightransmissionintensities.

Evenifweweretouseprevalence-odds-ratios(POR),instead ofprevalenceratios(PR),analysiscouldbeinaccuratewithregards toestimatingtruerelativerisk(andhencevaccineefficacy),asthe matchbetweenthesequantitiesgenerallyappliesonlyat equilib-riumpost-vaccine,anditisnotexemptfrombiasintroducedby samplingtimeandcompetitionhierarchiesbetweenstrains.

Individual vaccine protection, in the dynamic transmission models(0≤w≤1)multipliestransmissionrate(ˇ)andpathogen prevalence,thusitisnaturallydefinedperinfectiouscontact.In contrast, thesnapshot prevalence ratio, often reportedin field studies,missestheexposuredimension,beinganoutputofthe overalldynamicswithnonlinearandoftennon-trivialrelationto theoriginalinputparameter.Toresolvetheseproblems,a produc-tivealternativeistousethefulldynamicmodel,fitittoprevalence data before and after vaccination, as obtained through cross-sectionalobservationalstudies,thusestimatingtogetherseveral key epidemiologicalparameters. Based onpathogen prevalence observations, analysis of epidemiological studies post-licensure couldfactoroutsimultaneouslytheeffectsofmultipleparameters: ˇ,1,2(withintheSI-2model)andˇ,,(intheSIS-2model),

inordertoextractthetruevalueofvaccineefficacy(VE=1−w) againstthetargetedtypes.Frequently,epidemiologistswillhave enoughinformationaboutthesystemtodefinetheequationsthat governitsbehaviour,ortestsimultaneouslycompeting appropri-ateformulations.Inourcase,theparametersand(andinthe SIS-2model)areassumedknown.

3.4. NumericalprocedureforODEmodelfittingandparameter inference

Asaproofofconcept,weapplythefollowingprocedure.We generate hypothetical data performing model simulations with different parameter values, fixing initialconditions at the pre-vaccinationendemicstate.Observingthestateofthesystemat ti time units post-vaccination, subsequently we apply

nonlin-earleast-squaresoptimization(routinelsqnonlin inMATLAB) to model-generatedtrajectories,inordertorecovertheunderlying parameters.Similarmethodologicalapproachesexistandare rou-tinelyappliedtoepidemiological data(e.g.inthecontext ofR0

estimationCintrón-Ariasetal.(2009)).Theerrorfunction(objective functiontobeminimized)isgivenby:

Error=



ti



S,I1,I2...

(Theoretical model proportions(ti)−Data prevalences(ti))2,

(5) wheredatamayberealorsyntheticallygenerated,asinourcase here.Thisestimationdependsnotonlyonfittingtheprevalence of thepathogentypes targetedby thevaccine, butalso onthe prevalenceofsusceptibles,andofhostsinfectedwithtype2,at spe-cifictimepointspost-vaccinationinvaccinatedandnon-vaccinated

Fig.5.Biasinparameterestimationusingthefullepidemiologicalmodellingframework.Dynamicswith100randomparametercombinationsweresimulatedforeach

modelandthenonlinearleast-squaresoptimizationwasperformedonthereducedSI-2typeandSIS-2systemataspecifictimepost-vaccination.Fixedparametervalues

were:=0.5,=0.0167(SI-2)and=0.3,=0.02,=0.57(SIS-2).Initialconditionswerefixedatendemiccoexistenceequilibriumineachmodel.Assumingimperfect

observationsandafinitepopulationsizeN=1000,stochasticitywasimplementedintheobservationprocessbysamplinghostsfromamultinomialdistributionatt=0andat

3time-pointspost-vaccination(t1=12,t2=24,t3=36)whichintheSI-2modelreflect‘years’andintheSIS-2modelreflect‘months’.Themodelswerefittedtotheresulting

‘synthetic’sampleproportions.Inthebiasboxplots(bias=1−(ˆ)/),thecentralmarkisthemedian,theedgesoftheboxarethe25thand75thpercentiles,thewhiskers

extendtothemostextremebiaspointsnotconsideredoutliers,andoutliersareplottedindividually.ˇdenotesthetransmissionrateperunitoftime,1and2denotethe

directcompetitioncoefficientsintheSI-2model,whiledenotesthecompetitioncoefficientandthecompetitioncoefficientcorrectedforwithin-groupcoinfectionin

theSIS-2model.

hosts.Theerrorsareassumedtobenormallydistributedinthis framework,butthisassumptioncanbechangedinmore sophisti-catedparameterestimationprocedures.

Allowingforimperfectobservations(inthesyntheticdata), real-isticsamplingerrorcanbeaddedtodeterministicsimulations.For this,wedrawhostnumbersindifferentcompartmentsaccording toamultinomialprobabilitydistribution,withprobabilitiesfrom trajectoriesoftheODEmodel(SI-2,SIS-2),andagivensamplesize N.Byapplyingnonlinearleastsquaresoptimizationtointerpolate suchsyntheticsampleprevalencesgeneratedaftervaccination,we canrecoveralltheparametersofinterestwithverygoodaccuracy andinanunbiasedmanner(Fig.5).

Acritical requirementfor thesamplingof prevalences post-vaccinationisthatthesamplingtime-pointsaresufficientlyspread overtheinterval[0,1/],whichisintuitiveforavaccine admin-istered at birth, unless its implementation is preceded by a high-coveragevaccination campaign. In order to retrieve suffi-cientvaccineinformation,onehastofollowthedynamicsfromthe onsetofvaccinationatleastovertheentirelife-spanofatypical individual,i.e.overonegeneration.Providedthisminimalrange requirement,thebiasdecreasesfurtherwithsamplesizeandwith

thenumberoftime-pointsusedtosamplethepopulationinthe post-vaccineera(Fig.6).

Ingeneral,otheradvantagesofdeployingdynamicmodelsover moredirectstatisticaldescriptionsofdata,arethattheycanfillthe gapsforunobservedintervalsofsystembehaviour(Fig.7),generate newhypotheses,andyieldmoreaccuratepredictionsforthefuture. 3.5. Theroleoftype-specificimmunityandinferenceinaSIR model

Inthescenariosabove,werestrictedourattentiontoinfections withnoimmunity. Ina separatemodel,we extendedthebasic SI-2framework,toallowfortype-specificimmunityupon recov-ery,allelsekeptequal,includingvaccinationagainsttype1(see

SupplementaryTextS2).WhenchangingtoanSIR-2framework, thebaseline prevalenceof infection inthe pre-vaccine popula-tiondecreasesforthesameR0,andcompetitionbetweentypesis

reduced.Wenotethatprevalence-ratiodeviationsfromtrue vac-cineefficacypersistintheSIR-2model,andgetexacerbatedbythe oscillatorynatureofepidemiologicaldynamicsofinfectionswith immunity(Fig.S6).

2 3 4 5 6 −0.05 0 0.05 0.1 0.15 0.2

Number of time points post−vaccination

Mean bias over all parameters

N = 1000 0 500 1000 1500 2000 −0.05 0 0.05 0.1 0.15 0.2 Sample size N

Mean bias over all parameters

t_points = 3

Fig.6.BiasdecreaseswithnumberofsamplingpointsandsamplesizeN.Here,weshowfortheSI-2model,howthemean(absolute)biasinparameterestimatesacross

differentvaluesfittedthroughthedynamicmodel,varieswithnumberofsamplingtimepointsandsamplesize.Thefirsttime-pointistakenat12timeunitspost-vaccination,

andthesubsequenttimepointsaretakeninstepsof12.Weassume=0.0167,correspondingtoalifeexpectancyof60yearsandforvaccinationcoverage=0.5.Thevalues

ofˇand1,2aredrawnrandomlyintherange[0.06,0.32]and[0,1],respectively,subjecttotheconstraintofstableendemiccoexistencepriortovaccination.Aslongas

thesamplingtimescovertheinterval[0,1/],informationaboutthevaccinewhichisadministeredatbirthcanbeextracted.

Fig.7. IllustrationofdynamicmodelfittingtoprevalencedataintheSI-2model.Aftervaccinationagainsttype1pathogen,cross-sectionalprevalencedata(circles)canbe

integratedwithinadynamicmathematicalmodel(lines)toestimateepidemiologicalparameters,suchasvaccineefficacyandcompetitionparametersbetweendifferent

pathogentypes.Hereonlyoverallprevalenceoftype1(blueline,filledcircles)andprevalenceoftype2(redline,emptycircles)areshown.Parametersusedinthissimulation:

ˇ=0.3,w=0.2,1=0.1,2=0.5and=0.5,=0.0167arefixed.N=1000in(a)andN=100in(b)hasbeenusedinthemultinomialsamplingschemetoaddsampling

errortosyntheticdata.Applyingthenonlinearleastsquaresroutine,intheseparticularinstances,wehaveobtainedthesepointestimates:(a)ˇ=0.297,w=0.215,1=

0.108,2=0.441and(b)ˇ=0.416,w=0.184,1=0.059,2=0.283.(Forinterpretationofthereferencestocolourinthisfigurelegend,thereaderisreferredtotheweb

versionofthisarticle.)

Whenapplyingdynamicmodelfitting,wewereabletoinfer theparametersofinterestˇ,w,1,2,alsofortheSIR-2model

fromprevalenceobservationspost-vaccination.Wegrouped‘data’ into:pathogen-freehosts(susceptibleandimmune),those carry-ingtype1,type2,andthosecarryingboth1and2.Underrandom samplingeffects(usingthemultinomialframework),eventhough transmissionrateandvaccineefficacycouldbeinferredaccurately, thequantificationofthedirectcompetitioncoefficients1and2

washarder(Fig.S7).Thisisunsurprising,giventhelowerpathogen prevalenceexpectedintheSIR-2model,asalargeproportionof hostseventuallybecomeimmunetobothtypesandcompetition informationislost.Infact,weexpectthatestimationofdirect com-petitionparametersinaSIRframeworkrequireslargersamplesizes andmoretime-points,oracompleteresolutionofODEvariables withregardstohostimmunestatusandserologicalhistory.

Taken together, our simulations convey that in principle, dynamicmodelfittingthroughinterpolatingpre-vaccineand post-vaccine pathogen prevalences can be used to retrospectively assessvaccineeffectsindifferenthostpopulations,under differ-entcompetitionscenariosbetweentargetandnon-targetstrains, accountingforessentialnonlinearitiesintransmission.

4. Discussion

In this study, we have suggested toclose the gap between mechanisticmodelsofvaccineeffectsandstatisticalapproaches for estimationof vaccineefficacy,addressing inaddition multi-typepathogens characterizedbydirectcompetition.Onereason for why mechanistic models have been somewhat neglected in vaccine assessment studies, is that for validation, dynamic

epidemiologicalmodelshavetorelyonspatiotemporallyresolved data,andtodate,epidemiologicaldatatendtobestatic,and there-foremoreamenabletothetoolsofstatistics.Yet,withincreasing infrastructuresandeffortstomonitorhealthinpopulationsover time,comesanopportunitytoapplymoredynamicframeworks in the future, and match them with dynamic vaccine study designs.

4.1. Theimportanceandchallengesofdynamicmodelling

Here,wedrawattentiononanewapplicationof mathemat-ical models: namely, in the inference of vaccine efficacy from cross-sectionalprevalencedata,accountingforothercritical deter-minantsofpathogendynamicssuchastransmissionrate,vaccine coverage,and competition betweenmultipletypes (Choiet al., 2011;Gjinietal.,2016).Thesedatacanbegatheredfromactive samplingand cross-sectional populationsurveyspre- and post-vaccination,nationalscreeningprogramsaftermassimmunization, orreportedpathogenprevalenceindifferentcountriesunder spe-cificvaccinationcoveragerates.

We focused on scenarios of treatment effect homogeneity (leakyvaccine)andbaselinehomogeneity(nostructureinthehost population),assumingrandomizationacrossvaccinatedand non-vaccinated hosts.Of coursethemodels canbeshaped soasto incorporatealternativescenarios,butthiswasbeyondourscope here.Depending on thetype of data that are available (e.g. if prevalenceinformationisstratifiedbyage,geographyetc.), epi-demiologicalmodelscanincludetransmissionbetweenagegroups ordifferentspatiallocations(e.g.asin(Goeyvaertsetal.,2015)).The spiritofinferenceremainsthesame;onlyinthosecasesonewould havetospecifycontactparametersandfeedbacksbetween differ-entsub-populations.In thecurrentpaper, themainstructuring dimensionwashostvaccinationstatus.

Theneed to modify current vaccine evaluation practices by including theexposure dimension in vaccine studies has been recently pointed out by Gomes et al. (2014), and additional computationalframeworksarealreadyemergingtoestimate per-exposurevaccineeffects at theindividuallevel (O’Haganet al., 2014).Interpretation of vaccineefficacyis inevitably entangled withcontext-specificepidemiologicalinteractionsandfeedbacks. Thus,accuratequantificationofsuchindividualprotection param-eter,whichisrobust,identifiableandcomparableacrosssettings, canbenefitfrominclusionofmoremathematicalapproaches in vaccinestudies.Anylengthofpost-vaccinefollow-upperiodcan beinterpretedwithindynamicmodels,aslongasthetime-scale ofobservationscoversthetypicalhostlife-spaninterval[0,1/], especiallyinthecaseofSIandSIRdynamics,andaslongas approx-imationsunderlyingmodelstructure(e.g.homogeneousmixing) arevalid.

Notice,thattheshortcomingofhavingtosampleoveran inter-valof1/time-unitstoextractepidemiologicalparametersand vaccineefficacy,ifnecessary,canbedealtwiththrough parame-terinferenceproceduresthatextractbaselineparameterssuchas transmissionrateandcompetitioncoefficientsfromfitting analyt-icalequilibriumexpressions toprevalencespre-vaccination,and thenusingthoseestimatestoprojectdynamicsforwardinthe vac-cineera.Inthatcase,withalreadyknowncoveragerateandknown baselineparameters,inferenceofvaccineefficacyalonethrough dynamicfittingofprevalenceobservationsovershortertime-scales shouldproveeasier.

Dependingonthetransmissionmodel,caremustbetakenwith regardstocorrelationbetweenparameters.Some epidemiologi-calquantities, e.g.total prevalenceoftarget types,maydisplay thesameglobalmagnitudeatagiventimepost-vaccination,for verydifferentparametercombinations(e.g.andVEintheSIS-2 model,showninFig.S8).However,iffiner-scaledataareavailable,

including stratification with respect to host vaccination status and single/multiplecarriage,theapparently correlated parame-terscanthenbeseparated.Thishighlightstheimportanceofhigh resolution and highquality epidemiological datafor parameter estimation.

Here we show that transmission rate and vaccine efficacy against targeted pathogen types can be robustly estimated by dynamicmodelfitting,eveninthepresenceofobservationerror. Yet,wealsoshowthatourpowertoidentifydirectcompetition coefficientsfrompost-vaccineprevalencedatamaydependonthe overalllevelofpathogenprevalenceinthepopulation.Infectious agentsthatdisplaylowerprevalencearelikelytorequirelarger samplesizesinprevalencestudies,foracorrectresolutionoftheir diversityandinferenceofunderlyingsubtypecompetition.Thisis especiallyrelevant insystemswhere hostsrecoverwith immu-nity.Lackofestimabilityofinteractioncoefficientshoweverdoes not necessarily meanthemodel assumingthose interactions is invalid;ratherit mayhint that thoseparticular parametersare notasimportantfortheoveralldynamicsatthepopulationscale considered(Wikramaratna etal.,2014), orthata better resolu-tionofhostimmunologicalstatus,besidessimplycarriagestatus,is needed.

4.2. Vaccineefficacyandeffectiveness:linkingtrialsand populationobservations

Noticethatwhileefficacyisadirectparameter,thatiseasily understoodand can be averagedacross settings,vaccine effec-tivenessis a somewhatmore subjective criterion: (i) it canbe definedoveraspecifictimepost-vaccination,e.g.reductionin over-allprevalenceinthe3yearsfollowingvaccination;(ii)inthecase ofmulti-typesystems,effectivenesscanbedefinedwithregards tocarriageofthetargetedpathogentypes,ortotalcarriage;(iii) itcanbedefinedwithregardstothevaccinatedsub-population only,orasanoverallmeasurefortheentirepopulation irrespec-tiveofvaccinationstatus;(iv)itcanreflectdiseasemanifestations ofthepathogenatthepopulationlevelorasymptomaticcarriage states.Allthese‘effectiveness’ indicatorsare importantoutputs oftheepidemiologicaldynamics,thatcanbecalculatedtomeet thedemands of various policy-makers,and while theymaybe different,theintrinsicvaccineprotectionVEperindividualisthe same.

When linking vaccine efficacy against acquisition obtained throughrandomizedcontrolledtrialspre-licensure,andthe vac-cineefficacyestimateddynamicallypost-licensure,weexpectin principlethetwoquantitiesshouldmatch.Thatiswhythe frame-work we propose couldalso serve as a validation test. Notice howeverthat oftenvaccinetrialsare conductedinone popula-tion,whilevaccinesareimplementedinanother.Thus,differences couldarise.For example,interferencewithmaternalantibodies againstlocalpathogensmayalterthevaccineprotectionagainst otherpathogensinanewvaccinationprogramme.Immunological responsesmayalsobeinfluencedbynutritionstatusandother fac-tors,thusthereisnothingtoprecludepost-licensureassessmentof VEinpopulationswheretrialswerenotconductedinthefirstplace. Alltheseissuescanbeexploredusingdynamicmodelsof transmis-siontointerpretvaccination-inducedchangesinpopulations,and extractthebasicprotectionparameter.Evenifthetwoquantitiesdo notmatch,onecouldarguethatstudyingtheirdiscrepancywould openthewayforbetterdatacollection,modelimprovement, fine-tuningofvaccinationcoverage,inclusionofhostheterogeneities, andultimatelyabetterunderstandingofthedynamics.Ideally,one shouldworktogetherwiththetwoapproaches:bothpre-licensure andpost-licensure,andtrytomatchpredictionswith retrospec-tiveanalyses.Althoughnotprimarilyintendedtoaddresswaningof immunity,adynamicframeworkisidealtoexplorealsotheissueof

thewaningofvaccine-mediatedprotection,especiallywhen inter-polatingprevalenceobservationsoverlongintervalssincetheonset ofavaccinationprogramme.

A dynamic model, byits natureof mechanistically connect-ingmulti-dimensionalepidemiologicalobservationsovertime,can explainandlinkalsomorekindsofdata,includingtheprevalenceof multiplecarriage,ofcompetingpathogentypes,andalterationsin prevalenceoftotalcarriageovertime.Allthesequantitiescould have implications for disease, for the evolutionarypotential of thepathogen,andinteractionswithotherpathogens.Adynamic modelthusoffersabroaderviewof‘effectiveness’.Predictionsfor pathogendynamicsinthepopulationcanbefurtherintegratedinto mechanisticmodelsforspecificsymptoms,asshownby Rodriguez-Barraqueretal.(2013).

4.3. Methodologicalprospects

Our aimsin this paperwereillustrative and conceptual.For thisreason,wedidnotelaborateextensivelyonmethodological aspects.ApplyingdynamicmodelfittingwithinaBayesian frame-work,toaccountforparameteruncertainty,isalsopossible.One coulduse thesame objective function asin Section 3.4,or an explicitmultinomiallikelihoodfunction,fortheexactnumbersof hostsobservedindifferentepidemiologicalclasses.Inthelatter case,thesamplesizeNandexpectedprobabilityvectorcoming fromnumericalintegrationoftheODEsystemwithitssetof param-eters,wouldenterexplicitlythelikelihoodterm.Moreover,prior informationcouldbeaddedaboutspecificparameters,ifavailable. OnesuchBayesianprocedureisexplainedindetail,andapplied toaspecificdatasetonpneumococcusvaccinationinPortugal,ina relatedpaperbytheauthors(Gjinietal.,2016),whereinadditionan analysisofcompetingmodelformulationsisundertaken.Another importantfactorisprocessnoise andstochasticity(Ellneretal., 1998;Shresthaetal.,2011).Parameterinferenceinmechanistic modelsbasedonsystemsofcoupledODEsisatimelyand compu-tationallychallengingproblem.Manyadvancesinthisrespectare beingmadeacrossfieldssuchasstatisticsandcomputing(Haario etal.,2006;Calderheadetal.,2009;GirolamiandCalderhead,2011; Dondelingeretal.,2013;Kingetal.,2014),andappliedsuccessfully acrossdomainsofscience,includingsystemsbiology,astrophysics andclimatestudies.Refinement ofsuchmethodologiesfor epi-demiologicalmodelswithvaccinationshouldbestraightforward giventhemultitudeoftoolsthatalreadyexistorarein develop-ment.Theconceptualbackboneofdynamicinferenceislikelyto applyacrosssystems,however,challengesinspecificdiseaseswill inevitablyinvolveadaptationofmodelstructureandrefinementof theinferenceprocedure.

Finally,therearemanywaysinwhichtwopathogenstrainscan competewitheachother,mostnotablyviacross-immunity,which wedidnotconsiderhere.Theexistenceofnaturalcross-immunity, suchasamongdengue(Adamsetal.,2006;WearingandRohani, 2006), orHuman Papilloma Virus strains(Elbasha and Galvani, 2005; Durham et al., 2012), poses the issueof eventual cross-immunityalsointhevaccine-inducedprotectionagainstparticular pathogen types.To account for immune-mediated interactions, modelsneedtospecifyhownaturallyacquiredtype-specific immu-nitymightinterferewithvaccine-inducedimmunity(Gomesetal., 2004)andvaccineeffectiveness(Omorietal.,2012).Robust infer-enceofvaccineprotectioninthesecasesremainsanopenareafor futureresearch,callingforsophisticatedvaccinestudydesigns,as dootherfactors,suchashostpopulationstructure,seculartrends andstochasticity.Anaccurateunderstandingofintervention effec-tivenessinpopulationsundermassimmunizationwillincreasingly requireintegratedmodellingframeworksforpre-andpost-vaccine dynamics,accountingfortheinterplayofallthesefactors.

AppendixA. Supplementarydata

Supplementarydataassociatedwiththisarticlecanbefound,in theonlineversion,athttp://dx.doi.org/10.1016/j.epidem.2015.11. 001.

References

Adams,B.,Holmes,E.,Zhang,C.,Mammen,M.,Nimmannitya,S.,Kalayanarooj,S.,

Boots,M.,2006.Cross-protectiveimmunitycanaccountforthealternating

epi-demicpatternofdenguevirusserotypescirculatinginBangkok.Proc.Natl.Acad. Sci.U.S.A.103(38),14234–14239.

Agnandji,S.T.,Lell,B.,Soulanoudjingar,S.S.,Fernandes,J.F.,Abossolo,B.P.,

Conzel-mann,C.,Methogo,B.,Doucka,Y.,Flamen,A.,Mordmüller,B.,etal.,2011.First

resultsofphase3trialofRTS,S/AS01malariavaccineinAfricanchildren.N.Engl. J.Med.365(20),1863–1875.

Anderson,R.M.,May,R.M.,Anderson,B.,1992.InfectiousDiseasesofHumans:

DynamicsandControl,vol.28.WileyOnlineLibrary.

Andrews,N.,Waight,P.a.,Borrow,R.,Ladhani,S.,George,R.C.,Slack,M.P.E.,Miller,

E.,2011.Usingtheindirectcohortdesigntoestimatetheeffectivenessofthe

sevenvalentpneumococcalconjugatevaccineinEnglandandWales.PLoSONE 6(12),e28435.

Andrews,N.J.,Waight,P.A.,Burbidge,P.,Pearce,E.,Roalfe,L.,Zancolli,M.,Slack,M.,

Ladhani,S.N.,Miller,E.,Goldblatt,D.,2014.Serotype-specificeffectivenessand

correlatesofprotectionforthe13-valentpneumococcalconjugatevaccine:a postlicensureindirectcohortstudy.LancetInfect.Dis.14(9),839–846.

Biondi,B.E.,Weiss,S.H.,2015.Impactofbivalenthumanpapillomavirus(HPV)

vac-cinationupontheriskofacquisitionofotherHPVtypes.Med.Res.Arch.1(1).

Black,S.,Shinefield,H.,Fireman,B.,Lewis,E.,Ray,P.,Hansen,J.R.,Elvin,L.,Ensor,

K.M.,Hackell,J.,Siber,G.,etal.,2000.Efficacy,safetyandimmunogenicityof

heptavalentpneumococcalconjugatevaccineinchildren.Pediatricinfect.Dis.J. 19(3),187–195.

Calderhead,B.,Girolami,M.,Lawrence,N.D.,2009.AcceleratingBayesianinference

overnonlineardifferentialequationswithGaussianprocesses.In:Advancesin NeuralInformationProcessingSystems,pp.217–224.

Choi,Y.H.,Jit,M.,Gay,N.,Andrews,N.,Waight,P.A.,Melegaro,A.,George,R.,Miller,

E.,2011.7-ValentpneumococcalconjugatevaccinationinEnglandandWales:

isitstillbeneficialdespitehighlevelsofserotypereplacement.PLoSONE6(10), e26190.

Cintrón-Arias,A.,Castillo-Chávez,C.,Bettencourt,L.M.,Lloyd,A.L.,Banks,H.,2009.

Theestimationoftheeffectivereproductivenumberfromdiseaseoutbreakdata. Math.Biosci.Eng.6(2),261–282.

Colijn,C.,Cohen,T.,Fraser,C.,Hanage,W.,Goldstein,E.,Givon-Lavi,N.,Dagan,R.,

Lipsitch,M.,2009.Whatisthemechanismforpersistentcoexistenceof

drug-susceptibleanddrug-resistantstrainsofstreptococcuspneumoniae?J.R.Soc. Interface,rsif20090400.

Comanducci,M.,Bambini,S.,Brunelli,B.,Adu-Bobie,J.,Aricò,B.,Capecchi,B.,Giuliani,

M.M.,Masignani,V.,Santini,L.,Savino,S.,etal.,2002.NadA,anovelvaccine

candidateofNeisseriameningitidis.J.Exp.Med.195(11),1445–1454.

Crowe,E.,Pandeya,N.,Brotherton,J.M.,Dobson,A.J.,Kisely,S.,Lambert,S.B.,

White-man,D.C.,etal.,2014.Effectivenessofquadrivalenthumanpapillomavirus

vaccineforthepreventionofcervicalabnormalities:case–controlstudynested withinapopulationbasedscreeningprogrammeinAustralia.BMJ348,g1458.

delAngel,R.M.,Reyes-delValle,J.,2013.Denguevaccines:stronglysoughtbutnot

arealityjustyet.PLoSPathog.9(10),e1003551.

Dondelinger,F.,Filippone,M.,Rogers,S.,Husmeier,D.,2013.ODEParameter

Infer-enceUsingAdaptiveGradientMatchingwithGaussianProcesses.

Durham, D.P., Poolman, E.M., Ibuka, Y., Townsend, J.P., Galvani, A.P., 2012.

Reevaluationofepidemiologicaldatademonstratesthatitisconsistentwith cross-immunityamonghumanpapillomavirustypes.J.Infect.Dis.206(8), 1291–1298.

Elbasha,E.H.,Galvani,A.P.,2005.VaccinationagainstmultipleHPVtypes.Math.

Biosci.197(1),88–117.

Ellner,S.,Bailey,B.,Bobashev,G.,Gallant,A.,Grenfell,B.,Nychka,D.,1998.Noise

andnonlinearityinmeaslesepidemics:combiningmechanisticandstatistical approachestopopulationmodeling.Am.Nat.151(5),425–440.

Girolami,M.,Calderhead,B.,2011.RiemannmanifoldLangevinandHamiltonian

MonteCarlomethods.J.R.Stat.Soc.:Ser.BStat.Methodol.73(2),123–214.

Gjini,E.,Valente,C.,Sa-Leao,R.,Gomes,M.,2016.Howdirectcompetitionshapes

coexistenceandvaccineeffectsinmulti-strainpathogensystems.J.Theoret. Biol.388,50–60.

Goeyvaerts,N.,Willem,L.,VanKerckhove,K.,Vandendijck,Y.,Hanquet,G.,

Beu-tels,P.,Hens,N.,2015.Estimatingdynamictransmissionmodelparametersfor

seasonalinfluenzabyfittingtoageandseason-specificinfluenza-likeillness incidence.Epidemics13,1–9.

Gomes,M.G.M., Lipsitch,M.,Wargo, A.R.,Kurath,G.,Rebelo, C.,Medley,G.F.,

Coutinho,A.,2014.Amissingdimensioninmeasuresofvaccinationimpacts.

PLoSPathog.10(3).

Gomes,M.G.M.,White,L.J.,Medley,G.F.,2004.Infection,reinfection,andvaccination

undersuboptimalimmuneprotection:epidemiologicalperspectives.J.Theoret. Biol.228(4),539–549.

Greenland,S.,1987.Interpretationandchoiceofeffectmeasuresinepidemiologic

Haario,H.,Laine,M.,Mira,A.,Saksman,E.,2006.DRAM:efficientadaptiveMCMC. Stat.Comput.16(4),339–354.

Haber,M.,Longini,I.M.,Halloran,M.E.,1991.Measuresoftheeffectsofvaccination

inarandomlymixingpopulation.Int.J.Epidemiol.20(1),300–310.

Halloran,M.,2006.Overviewofvaccinefieldstudies:typesofeffectsanddesigns.J.

Biopharmaceut.Stat.16(4),415–427.

Halloran,M.E.,Haber,M.,Longini, I.M.,Struchiner,C.J., 1991.Directand

indi-recteffectsinvaccineefficacyandeffectiveness.Am.J.Epidemiol.133(4), 323–331.

Halloran,M.E.,Longini,I.M.,Struchiner,C.J.,Longini,I.M.,Struchiner,C.J.,2010.

DesignandAnalysisofVaccineStudies.Springer.

Heesterbeek,J.,2000.MathematicalEpidemiologyofInfectiousDiseases:Model

Building,AnalysisandInterpretation,vol.5.Wiley.

Hernán,M.A.,2010.Thehazardsofhazardratios.Epidemiology(Cambridge,Mass.)

21(1),13.

Insinga,R.P.,Dasbach,E.J.,Elbasha,E.H.,Puig,A.,Reynales-Shigematsu,L.M.,2007.

Cost-effectivenessofquadrivalenthumanpapillomavirus(HPV)vaccination inMexico:atransmissiondynamicmodel-basedevaluation.Vaccine26(1), 128–139.

Keeling,M.J.,2005.Modelsoffoot-and-mouthdisease.Proc.R.Soc.Lond.B:Biol.Sci.

272(1569),1195–1202.

King,A.A.,Nguyen,D.,Ionides,E.L.,2014.Statisticalinferenceforpartiallyobserved

Markovprocessesviatherpackagepomp.J.Stat.Softw.55(2).

Lipsitch,M.,1997.Vaccinationagainstcolonizingbacteriawithmultipleserotypes.

Proc.Natl.Acad.Sci.U.S.A.94(12),6571–6576.

Lipsitch,M.,Colijn,C.,Cohen,T.,Hanage,W.P.,Fraser,C.,2009.Nocoexistencefor

free:neutralnullmodelsformultistrainpathogens.Epidemics1(1),2–13.

Martcheva,M.,Bolker,B.M., Holt,R.D.,2008.Vaccine-inducedpathogen strain

replacement:whatarethemechanisms?J.R.Soc.Interface5(18),3–13.

Mitchell,P.K.,Lipsitch,M.,Hanage,W.P.,2015.Carriageburden,multiple

coloniza-tionandantibioticpressurepromoteemergenceofresistantvaccineescape pneumococci.Philos.Trans.R.Soc.Lond.B:Biol.Sci.370(1670),20140342.

Moberley,S.,Andrews,R.,2014.Anevaluationoftheindirectcohortmethodto

esti-matetheeffectivenessofthepneumococcalpolysaccharidevaccine.J.Vaccines Immunol.2,4–6.

O’Hagan,J.J.,Lipsitch,M.,Hernán,M.A.,2014.Estimatingtheper-exposureeffectof

infectiousdiseaseinterventions.Epidemiology25(1),134–138.

Omori,R.,Cowling,B.J.,Nishiura,H.,2012.Howisvaccineeffectivenessscaledby

thetransmissiondynamicsofinteractingpathogenstrainswithcross-protective immunity?PLoSONE7(11),e50751.

Richardson,D.,2004.Anincidencedensitysamplingprogramfornestedcase-control

analyses.Occup.Environ.Med.61(12),e59.

Rinta-Kokko,H.,Dagan,R.,Givon-Lavi,N.,Auranen,A.,2009.Estimationof

vac-cineefficacyagainstacquisitionofpneumococcalcarriage.Vaccine27(29), 3831–3837.

Rodriguez-Barraquer,I.,Mier-yTeran-Romero,L.,Burke,D.S.,Cummings,D.A.,2013.

Challengesintheinterpretationofdenguevaccinetrialresults.PLoSNeglect. Trop.Dis.7(8),e2126.

Sabchareon,A.,Wallace,D.,Sirivichayakul,C.,Limkittikul,K.,Chanthavanich,P.,

Suvannadabba,S.,Jiwariyavej,V.,Dulyachai,W.,Pengsaa,K.,Wartel,T.A.,etal.,

2012.Protectiveefficacyoftherecombinant,live-attenuated,CYDtetravalent

denguevaccineinThaischoolchildren:arandomised,controlledphase2btrial. Lancet380(9853),1559–1567.

Scott,P.,Herzog,S.A.,Auranen,K.,Dagan,R.,Low,N.,Egger,M.,Heijne,J.C.,2014.

Tim-ingofbacterialcarriagesamplinginvaccinetrials:amodellingstudy.Epidemics 9,8–17.

Shim,E.,Galvani,A.P.,2012.Distinguishingvaccineefficacyandeffectiveness.

Vac-cine30(47),6700–6705.

Shrestha,S.,King,A.A.,Rohani,P.,2011.Statisticalinferenceformulti-pathogen

systems.PLoSComput.Biol.7(8),e1002135.

Strömberg,U.,1994.Prevalenceoddsratiovsprevalenceratio.Occup.Environ.Med.

51(2),143.

Thompson,M.L.,Myers,J.,Kriebel,D.,1998.Prevalenceoddsratioorprevalenceratio

intheanalysisofcrosssectionaldata:whatistobedone?Occup.Environ.Med. 55(4),272–277.

vanBoven,M.,Ruijs,W.L.,Wallinga,J.,O’Neill,P.D.,Hahne,S.,2013.Estimationof

VaccineEfficacyandCriticalVaccinationCoverageinPartiallyObserved Out-breaks.

Wearing,H.J.,Rohani,P., 2006.Ecologicalandimmunologicaldeterminantsof

dengueepidemics.Proc.Natl.Acad.Sci.U.S.A.103(31),11802–11807.

Weinberger,D.M.,Malley,R.,Lipsitch,M.,2011.Serotypereplacementindisease

afterpneumococcalvaccination.Lancet378(9807),1962–1973.

Wikramaratna,P.S.,Kucharski,A.,Gupta,S.,Andreasen,V.,McLean,A.R.,Gog,J.R.,