衡量最重要的福利

PublicDisclosureAuthorizedPublicDisclosureAuthorizedPublicDisclosureAuthorizedPublicDisclosureAuthorized

r-tm

Measuringwelfare

whenitmattersmost

WORLDBANKGROUP

Atypologyofapproachesforreal-timemonitoring

Contents

Introduction7

1MethodsforNowcastingWelfare—WithaFocusonMonetaryPoverty17

1.1NowcastingWelfareUsingSurveyandOtherNon-surveyCovariates18

ConsiderationsRegardingReliableSurvey-and

Non-survey-basedImputation21

SurveyImputationMethodscanbeComplementedwith

DataCollectiontoDealwithMissingAuxiliaryorBaselineData24

LessonsLearntandResources25

1.2NowcastingWelfareUsingGDPGrowth27

ConsiderationsRegardingGDP-basedNowcasting30

Resources33

1.3NowcastingWelfareUsingMicrosimulationsandGeneral

EquilibriumModels33

ConsiderationsRegardingMicrosimulationandGeneralEquilibrium

Models36

Resources37

2HarnessingDataforReal-timeWelfareMonitoring39

2.1RapidSurveyDataCollection40

High-frequencyPhoneSurveys40

RapidFace-to-faceSurveys46

OnlineandMessaging-basedSurveys49

FurtherResources50

2.2GeospatialData52

MainCharacteristicsandExamples52

CaveatsforUsingGeospatialData56

LessonsLearntandResources59

2.3DigitalTraceData62

MainCharacteristicsandExamples62

CaveatsforUsingDigitalTraceData64

LessonsLearntandResources65

3

2.4AdministrativeData67

MainCharacteristicsandExamples67

CaveatsforAdministrativeDataforReal-timeWelfareMonitoring69

LessonsLearntandResources70

3MovingForward:IdentifyingAreasforAdvancement71

References73

Annex1.SummaryofModelsUsedtoUpdatePovertyEstimates95

Annex2.CommonlyUsedMachineLearning(ML)Modelsfor

EstimatingPoverty97

Annex3.SummaryofAllDataSources100

Annex4.NowcastingImpactsofShocks(Vulnerabilityand

DamageFunctions)103

ConsiderationsRegardingDamageFunctions104

Resources105

4MEASURINGWELFAREWHENITMATTERSMOST—ATypologyofApproachesforReal-timeMonitoring

Acknowledgments

ThisdraftwaspreparedbyateamfromtheWorldBankPovertyandEquityGlobalPracticeconsistingofKimberlyBolch,MariaEugeniaGenoni,andHenryStemmler.CarlosSabatinoalsoprovidedexcellentinputstothedocument.TheworkwasconductedunderthesupervisionofLuisF.López-Calva(GlobalDirector,PovertyandEquityGP)andBenuBidani(PracticeManager,PovertyandEquityGP).ThisdocumentbenefitedfromconsultationswithmanymembersofthePovertyandEquityGlobalPracticeaswellasotherWorldBankteamswholedthedevelopmentandimplementationofmanyoftheinitiativesreferencedhere.TheteamisparticularlygratefultoMaurizioBussolo,PaulCorral,XimenaDelCarpio,DanielGerszonMahler,CraigHammer,RuthHill,DeanJolliffe,WalkerKosmidou-Bradley,LauraMorenoHerrera,SergioOlivieri,andNobuoYoshidafortheircommentsandadvice.DesignandtypesettingbyReyesWork.

5

Introduction

TimelyInformationonWelfareisCriticalforEffectivePolicymaking

AstheWorldDevelopmentReport2021:DataforBetterLiveshighlights,dataisafoundationalinputforimprovingdevelopmentoutcomesthroughenhancingtheeffectivenessofpolicymaking.However,thedegreetowhichdatacangener-atevaluefordevelopmentdependsonitsquality(WorldBank,2021b).Onecriticalaspectofdataqualityistimeliness.Havingup-to-dateinformationisessentialforpolicymakerstofinetunepoliciesasconditionschange.Inacontemporaryglobalenvironmentmarkedbyheighteneduncertaintyinthefaceofchallengessuchasclimatechange,conflict,andpandemics—theneedformoretimelysourcesofdatatoinformpolicyisparticularlypressing.

Inthecontextofpoliciestoreducepovertyandvulnerability,moretimelyinformationonhouseholdwelfareisneeded.Traditionalmethodsproducemeasuresofhouseholdwelfaretooinfrequentlytomeettheneedsofmanypolicymakers.Officialmeasuresofpovertyarederivedfromhouseholdsurveys,which(eveninidealsettings)areonlyconductedeveryfewyears—giventhefinancialandadministrativecostsinvolved.Inmanysettings,andparticularlyinlow-incomeandfragilecountries,thesesurveysareconductedwithmuchgreaterlags.1However,bycombiningtraditionalsurveys(“baselinedata”)withdifferentmodellingapproachesandalternativesourcesoffrequentlycol-lecteddata(“auxiliarydata”)—itispossibletodevelopmonitoringsystemsthat

1Onaverage,themostrecenthouseholdsurveyintheWorldBank’sPovertyandInequalityPlatform(PIP)isoversixyearsold.Ofthe168countriesinPIP,37percenthavedatathatismorethanfiveyearsoutofdate;ofthe56IDAcountriesinPIP,52percenthavedatathatismorethanfiveyearsoutofdate(September2023PIPUpdate).

7

provideup-to-dateestimatesontheevolutionandstatusofhouseholdwelfare.Investinginthiscapacitytomonitorwelfarein“realtime”isessentialtoboth(i)informnewpolicyactioninthewakeofshocksand(ii)enhancetheadap-tivecapacityofexistingpoliciesascircumstanceschange.Inadditiontoservingasinputstoeffectivepolicymaking,manyoftheapproachesdiscussedcanbeappliedinthecontextofprojectmonitoring.SeeBox1foradiscussiononhowwedefinemonitoringof"welfare"in"realtime".

Methodologicalandtechnologicaladvanceshaveexpandedourabilitytomonitorwelfareinrealtime

Inrecentyears,theWorldBank’sPovertyandEquityGlobalPractice(GP)hasincreaseditscapacitytoprovidemoretimelyinformationonwelfare.Inclosecollaborationwithinternalandexternalpartners,wehaveledeffortsatthecountry(respondingtocontextspecificneeds)andcorporatelevels(relatedtotheglobalmonitoringofpoverty)toimplementabroadrangeofmodellingapproachesandleverageorcollectnewsourcesofhigh-frequencyauxiliarydata.Moreover,thisworkhasincreasinglybenefitedfromfrontiermethodologicalapproaches(forexample,machinelearning)anddatasources(forexample,bigdata)thatcanenhancetheperformanceofexistingmethods.Whileongoingforsometime,theworkwasgreatlyscaledupinthecontextofrecentcrisessuchastheCOVID-19pandemicandclimate-relateddisasters.

Thistypologytakesstockofthegrowingbodyofworkonreal-timewelfaremonitoring,bringingtogetherexistingresourcesandlessonslearnedinoneplace.Itaimstoofferanoverarchingroadmaptohelpteamsnavigatediffer-entapproachesandidentifythebestfitforansweringaspecificquestioninagivencontext.The“bestfit”approachmaydifferacrosssettingsdependingonacountry’sdataecosystemandimplementationconstraints.Thistypologysys-tematizesthedecision-makingprocessbylayingoutthevariousadvantages,disadvantages,underlyingdatarequirements,andassumptionsofdifferentapproaches.WhileprimarilydrawingthePovertyandEquityGP’swork,thetypologyaimstocontextualizereal-timemonitoringwithinabroaderbodyofresearchandtowardsrecentinnovationsinthefield.TheresearchthistypologyhasproducedispartofabroaderglobalinitiativeoftheGPonmoving“TowardsReal-TimeMonitoringofWelfare”andwillbecomplementedbyamoredetailedtechnicalhandbook(forthcoming).

8MEASURINGWELFAREWHENITMATTERSMOST—ATypologyofApproachesforReal-timeMonitoring

Box1DefiningReal-timeWelfareMonitoring

Whatdowemeanby“realtime”?Thistypologyusestheterm“realtime”torefertoinformationproducedwithashorterlagthantradi-tionalhouseholdsurveysallow.Forwelfaremonitoring,wheresurveygapsoftenspanmultipleyears,dataproducedwithweekly,monthly,orevenyearlyperiodicitymaybeconsidered“realtime.”Thegoalofthevariousapproachesdescribedinthistypologyistoprovidethemostup-to-datewelfareinformationpossible,giventhefeasibilityconstraintsfordoingsoreliably.Itdoesnotnecessarilyimplyinstan-taneousupdates.

Howdowedefine“welfare”?Weusetheterm“welfare”broadlytoencompassmultipledimensionsofwell-being.Thetypologyhigh-lightsexamplesofwelfaremonitoringacrossarangeofdimensionswithafocusonmonitoringmonetarypovertyatthenationallevel,reflectingtheextensiveworkproducedbythePovertyandEquityGPonthisaspectofwell-being.

Monetarypovertyisastateofdeprivationcharacterizedbyalackofsufficientincomeorfinancialresourcestomeetbasicneeds,suchasfood,shelter,clothing,andhealthcare.Itistypicallymeasuredbycom-paringanindividual’sorhousehold’sincomeorconsumptionagainstadefinedpovertythresholdorpovertyline,withthosebelowthethresholdconsideredmonetarilypoor.

Monetarypovertymeasurementisdataintensiveandchallengingindata-deprivedcontexts.Insomecases,directlymeasuringotherdimensionsofwelfare(forexample,foodsecurity,employment,hous-ing,education)maybeeasierandequallyinsightfulforunderstandingchangesinindividualwell-being.

9|IntroductIon

Part1:Methods

Analyticalmodelsto

leveragemicro,macro,andbigdatatoupdatepovertyandotherwelfaremeasures

ATypologyinTwoParts:MethodsandData

Thistypologyisorganizedintwoparts.Thefirstpartfocusesonmethods,map-pingoutanalyticalmodelsthatleveragemicro,macro,andbigdatatonowcastpovertyandotherwelfaremeasures.Thesecondpartfocusesondata,listingoptionstocollecthigh-frequencydataorbetterharnessexistingsources.Mostapproachesrequireastrategiccombinationofboth—withmodelsrequiringhigh-frequencydataasakeyinput(Figure1).

Figure1Real-timewelfaremonitoringrequiresacombinationofmodelingandhigh-frequencydata

Part2:Data

Effortsforthecollectionofnewdataandbetterharnessingofexistingdata

Notably,mostapproachesrelyonhavingrecentbaselinedataasaprecondi-tion(Box2).Inthissense,theseapproachesarenotmeanttobeasubstituteforinvestingintraditionalsurveys(suchashouseholdbudgetsurveysorcensuses);infact,havingarelativelyrecentbaselinesurveyisacriticalinputtoensurethequalityandaccuracyofthemodelinganddatacollectionmethodscoveredinthistypology.Whenthisisnotthecase,thefeasibilityofreal-timemonitoringmaybelimited,andthecollectionofnewbaselinedatamayberequired.

10MEASURINGWELFAREWHENITMATTERSMOST—ATypologyofApproachesforReal-timeMonitoring

Box2BuildingonaStrongFoundation:BaselineDataisaPrerequisiteforReal-timeMonitoring

Methodsforimputingpovertyandhigh-frequency“auxiliarydata”arenotyetsubstitutesfortraditionalhouseholdsurveys,whichremainthefoundationofreliablewelfareestimates.Afullsurveywithcom-prehensivewelfareinformation(suchasahouseholdbudgetsurvey)orpopulationinformation(suchasacensus)isoftenaprerequisitetoeffectivelyapplytheapproachesdescribedhere.

Figure2showshowtothinkaboutthesedifferentdatasetsandhowtheytogetherfeedintomodelstomonitorwelfareinreal-time.Thistypologyreferstothisfoundationaldataas“baselinedata.”However,inthelanguageofmachinelearningitcanalsobethoughtofas“train-ingdata.”Trainingdataservesapivotalrole,providingtheunderlyinginformationnecessaryformodelstolearnpatterns,classifydata,andmakepredictions.Thequalityandquantityoftrainingdatasignifi-cantlyimpactstheperformanceandaccuracyofthealgorithm.Iftrain-ingdataonwelfareisnon-existentortoooutofdate,thesemethodswillbeunreliable.

Themethodsanddatasourcesdiscussedinthistypologyshouldbeseenascomplementarytoratherthansubstitutesfortraditionalsur-veys.Assuch,effortstoadvancethereal-timemonitoringofwelfaregreatlydependoncontinuedinvestmentsinclosingfoundationaldatagaps.TheWorldBankhaslongbeenworkingwithcountrypartnerstoinvestinthemodernizationofnationalstatisticalsystems.Atthegloballevel,thisworkisbeingledbytheGlobalSolutionsGrouponDataforPolicy.Thisincludesanimportantefforttoclosepoverty-relateddatagaps,includingthroughtheimplementationofmorefrequenthouse-holdsurveys.Whilemuchprogresshasbeenmadeinrecentyears,thereisstillalongwaytogo.

11|IntroductIon

Figure2Theingredientsforreal-timewelfaremonitoring

Surveyornon-surveyimputation

Anothermicrosurvey

(LFS,DHS,specially

collectedsurvey)

Macrodata(e.g.,GDP)

Bigdata(e.g.,geospatial,admin,digitaltrace)

collecteddatawithwelfareinformation)

Baselinedata

Datawithwelfare

information(e.g.,budgetsurveyorspecially

GDP-growthmodelsMicrosimulations

Auxiliarydata

Model

PartI:Methods

Thisportionofthetypologyprovidesanoverviewofvarioustypesofmethodsthatcanbeusedtoimputeorpredictwelfarein“realtime”.Thesemethodsuti-lizetimelyinformationfrom“auxiliarydata”sources(suchasmicrosurveys,mac-roeconomicstatistics,orotherbigdatasources)andmodelrelationshipswithvariablesinolderbaselinedatatoestimatemissingdatapoints.

Figure3providesanoverviewofmethodsofreal-timemonitoringfordiffer-entusecases.Themaintypesofmethodsdiscussedinthistypologyarecovari-ate-basednowcasting,GDP-basednowcasting,andmicrosimulationmodels.Researchershaveallthesemethodsattheirdisposalwhentheobjectiveistoobtainanupdatedpoverty-ratenowcast.GDP-basednowcastingneedstobemodifiedtocapturedifferencesacrossincomedistribution,whiletheothermethodsincorporatedistributionsensitivenowcasts.Whenresearchersaimtoincorporatedifferentmechanismsandindirecteffects,theyneedtorelyonmicrosimulationmodels.Finally,microsimulationmodelsandrelatedvulner-abilityfunctionsareusefulforupdatingestimatestoaccountfortheimpactsofshocks.Covariate-basednowcastingcanalsoprovideestimatesofshock

12MEASURINGWELFAREWHENITMATTERSMOST—ATypologyofApproachesforReal-timeMonitoring

impacts,buttypicallyonlywhencombinedwithdatacollectionefforts,whicharediscussedinfurtherdetailinpart2ofthistypology.

Figure3Methodsforreal-timemonitoringfordifferentusecases

UsecaseMethods

Distribution-

Poverty-ratenowcast

GDP-poverty

elasticity

(section1.2)

Covariate-basednowcasting(section1.1)

scaling

(section1.2)

Micro-simulation

(section1.3)

Estimatesalongtheincomedistribution

Canincorporateassumptionsaboutdistributionalchanges

Incorporateand

understandmecha-

nismsorindirecte?ects

Vulnerability

analysis

(Appendix4)

Collectionofex-postdata (section2.1)

Nowcastingchangesinwelfareaftershocks

Harnessingdata(section2)

Monitorproxyorleadingindicatorsforwelfare

Ultimately,choosingbetweenthevariousmethodswilldependontheusecaseandtoalargeextentontheunderlyingdatarequirementsandthescaleofanalysis(forexample,subnational,national,regional,global).Moreover,implementingthesemethodsrequiresdifferentinputsintermsofskills,time,andfinancialresources.Dependingontheconstraintsthatateamfacesinagivencontext,differentapproachesmaybebettersuitedtotherealitiesontheground.Thistypologyfeaturesseveraldecisiontreestohelpusersthinkthroughwhichmethod(s)arebettersuitedtodifferentcontextsandobjectives.

PartII:Data

Thetimelinessofwelfareestimatesproducedbythemethodsdependsentirelyonthetimelinessoftheauxiliarydatainputs.Reliable,high-frequencyandup-to-datedatasourcesarecriticalforanyapproachtomonitorwelfareinrealtime.PartIIofthistypologyfocusesontwokeyefforts:(i)collectingnewhigh-fre-quencydata,and(ii)betterharnessingexistingsourcesofhigh-frequencydata(Figure4illustratesafewexamples).

13|IntroductIon

Figure4Dataforreal-timemonitoring:Collectingandharnessinghigh-frequencydata

CollectingNewData

ExistingData

Sources

Rapid

surveys

Geospatialdata

Digital

tracedata

Administrativedata

?Phone

?Face-to-Face

?Onlineand

messaging-based

?Satelliteimagery

?Nighttimelights

?Vegetationindices

?Calldetailrecords

?Socialmediadata

?Taxdata

?Barcodescannerdata

?Socialregistries

Thesemorefrontiertypesofdatasourcescanbeleveragedinseveralwaysforreal-timemonitoring.First,theycancomplementexistingbaselinesurveydataasaninputtoimprovethenowcastingmethodsdiscussedabove.Thiscanbepar-ticularlyusefulwhenexistingsurveydataisnotrecent,doesnotcoverthewholepopulation,orlacksspecificdimensionsthatarerelevantforwelfareestimation.Second,theycanofferabroaderpictureonwelfarewhendataconstraintslimitthefeasibilityofestimatingmonetarywelfare.Inmanycases,other(non-mon-etary)measuresareveryinformativeindepictingwelfaretrendsordifferencesbetweenpopulations.Variablessuchasemployment,foodsecurity,orsubjectivewell-beingmaybeavailablefromothersourcesorcanbecollectedmoreeasilythanfullinformationonincomeorconsumption.Third,leadingindicators,suchaspredictionsofdroughtsorfloodsorinflationdata,canprovideimportantsig-nalsofchangesinwelfare,beforetheseareobservableinsurveydata.TheselasttwousecasesaresummarizedbythelastrowofFigure3.

Selectingthebest-fitapproach

Thistypologyisnotmeanttobeprescriptivenordoesitrankapproaches.Rather,itseekstoprovideamorestructuredwaytohelpusersidentifyacoresetofavailableoptionsandsystematicallythinkthroughthetrade-offs

14MEASURINGWELFAREWHENITMATTERSMOST—ATypologyofApproachesforReal-timeMonitoring

betweenthem.Eachapproachcoveredinthetypologyincludesadiscussiononthemaincharacteristics,caveats,andlessonslearned—alongsideacollectionofresources.Thebest-fitapproachinonecontextmaynotalwaystranslatesuccess-fullyinanother.Inallcases,itwillbecriticaltokeepinmindthecorepolicyques-tiondrivingtheanalysisaswellasthebroadrangeofdataecosystemsinwhichuserswillbeseekingtoapplythesemethods,rangingfromstablesettingsrichinfrequentbaselineandauxiliarydatatofragileandconflict-affectedsettingswithverylimiteddatainputsandhighimplementationconstraints.

15|IntroductIon

1.

MethodsforNowcastingWelfare—WithaFocus

onMonetaryPoverty

Nowcastingandimputationmethodsleveragebaselinedatathatcontainsadirectmeasureofwelfareandmorerecentauxiliarydatasourceswithwhichwelfareisimputed.Thebaselinedataprovidesthefoundationoftheanalysis,con-tainingvariableswithwhichwelfarecanbeestimated(forexample,fromahouse-holdsurvey).Auxiliarydatasourcesvary;somemodelsmakeuseofhouseholdmicrodatasuchaslaborforce,census,demographicandhealth,orspeciallycol-lectedhouseholdsurveys;othersrelyonmoreeconomy-widedatasuchascurrentGDPorFinalConsumptionExpenditurenumbers.Somemethodsalsousebigdatasourcessuchasgeospatialorcalldetailrecorddata.Still,almostallthesemethodsneedthebaselineinformationtounderstandhowtheseauxiliaryvariablesrelatetowelfareorrequirebaselineincomedistributionstomakeinferencesaboutchangesinwelfare.

Inthefollowing,severaldifferentmethodsofestimatingwelfareandpovertyaredescribedinmoredetail,withspecificguidanceonadvantagesanddisad-vantages,andexampleusecasesandlinkstofurtherresourcesareprovided.

Annex1providesasummaryofthedifferentmethods,whicharediscussedinthistypology,includingrequirementsforthemethodtoaccuratelyestimatewelfareindicatorsandwhatlimitationsthemethodhas.

Beforewemoveon,itisimportanttonotethatallmodelsdescribednextrelyonimportantassumptionsthatneedtobeassessedandpossiblyvalidatedineachcontext.Whenfeasible,itisrecommendedtorundifferentoptionstocom-pareresults.Triangulationoffindingswithotherexternalsourcesofinformationisalsoadvisable.Finally,allmethodshaveerrors,andwhereverpossible,confi-denceintervalsshouldbereportedwiththeresults.

17

1.1NowcastingWelfareUsingSurveyandOtherNon-surveyCovariates

Surveyandnon-survey-basedimputationmethods(covariate-basednowcasting)modeltherelationshipbetweenconsumptionorincomeandothercovariatestonowcastpoverty.Survey-to-surveyimputationmethodsdrawupondistributionsofconsumption(orincome)variablesandothercovari-atesfromabaselinesurveytonowcastconsumption(orincome)levelsusingarecentauxiliarysurvey,whichitselfdoesnotholdconsumptionvariables.Non-survey-basedimputationdrawsuponinformationfromnon-surveyauxil-iarydata,suchasremotely-sensedgeospatialdata.Thesevariablescaneitherbeusedtoimprovesurvey-basedmodelsortoindependentlyformimputationmodels.

Whileimputationacrossspacehasreceivedconsiderableattention,advance-mentsinsurvey-basedimputationofwelfareovertimearestillrecent.Hentscheletal.(1998)andElbers,Lanjouw,andLanjouw(2003)initiatedawaveofresearchwithinandoutsideoftheWorldBanktoadaptimputationmethodstoestimatemonetarypovertyforpovertymapping.Theseimputationmodelshavebeenwidelyusedtogeneratespatiallydisaggregatedwelfareinformation.23Morerecentworkisexploringwaystoadaptthesemodelstoupdatewelfareacrosstime.

Mostcommonly,linearregressionmodelsareusedtoimputeconsumptionandexpenditurevariables.Surveyandnon-survey-basedimputationmodelsalsoinvolvestatisticalapproacheslikehot-deckimputationandmultipleimpu-tation(MI),whichaimtoreducenonresponsebiasandimprovetheoverallrep-resentativenessandqualityofsurveydata.4Somestudiesestimateapooror

2Formoreinformationaboutimputationmethodsacrossspace,seeforinstance

Corraletal.(2022)

,StifelandChristiaensen(2007),Tarozzi(2007),Christiaensen(2012),Mathiassen(2013),orthereport“

MoreThanaPrettyPicture:UsingPovertyMapstoDesignBetterPoliciesandInterventions

.”

3Survey-to-surveyimputationcanalsobeusefulforotherapplicationsbeyondupdatingormappingmonetarypoverty,suchasensuringcomparabilityofconsumptionovertimeorimputingnon-mone-tarywelfaremetrics.Evenwhensurveysareavailable,changesinpovertylinesorconsumptionmod-ulescanhindercomparisonsofpovertyovertime.Survey-to-surveyimputationhasalsobeenusedtorestorecomparabilityin