計量經濟學課件3

Chapter3

MultipleRegression

Analysis:EstimationWooldridge:IntroductoryEconometrics:AModernApproach,5eMultipleRegression

Analysis:EstimationDefinitionofthemultiplelinearregressionmodelDependentvariable,explainedvariable,responsevariable,…Independentvariables,explanatoryvariables,regressors,…Errorterm,disturbance,unobservables,…InterceptSlopeparameters?Explainsvariableintermsofvariables“MotivationformultipleregressionIncorporatemoreexplanatoryfactorsintothemodelExplicitlyholdfixedotherfactorsthatotherwisewouldbeinAllowformoreflexiblefunctionalformsExample:WageequationHourlywageYearsofeducationLabormarketexperienceAllotherfactors…NowmeasureseffectofeducationexplicitlyholdingexperiencefixedMultipleRegression

Analysis:EstimationExample:AveragetestscoresandperstudentspendingPerstudentspendingislikelytobecorrelatedwithaveragefamilyincomeatagivenhighschoolbecauseofschoolfinancingOmittingaveragefamilyincomeinregressionwouldleadtobiasedestimateoftheeffectofspendingonaveragetestscoresInasimpleregressionmodel,effectofperstudentspendingwouldpartlyincludetheeffectoffamilyincomeontestscoresAveragestandardizedtestscoreofschoolOtherfactorsPerstudentspendingatthisschoolAveragefamilyincomeofstudentsatthisschoolMultipleRegression

Analysis:EstimationExample:FamilyincomeandfamilyconsumptionModelhastwoexplanatoryvariables:inomeandincomesquaredConsumptionisexplainedasaquadraticfunctionofincomeOnehastobeverycarefulwheninterpretingthecoefficients:FamilyconsumptionOtherfactorsFamilyincomeFamilyincomesquaredByhowmuchdoesconsumptionincreaseifincomeisincreasedbyoneunit?DependsonhowmuchincomeisalreadythereMultipleRegression

Analysis:EstimationExample:CEOsalary,salesandCEOtenureModelassumesaconstantelasticityrelationshipbetweenCEOsalaryandthesalesofhisorherfirmModelassumesaquadraticrelationshipbetweenCEOsalaryandhisorhertenurewiththefirmMeaningof?linear“regressionThemodelhastobelinearintheparameters(notinthevariables)LogofCEOsalaryLogsalesQuadraticfunctionofCEOtenurewithfirmMultipleRegression

Analysis:EstimationOLSEstimationofthemultipleregressionmodelRandomsampleRegressionresidualsMinimizesumofsquaredresidualsMinimizationwillbecarriedoutbycomputerMultipleRegression

Analysis:EstimationInterpretationofthemultipleregressionmodelThemultiplelinearregressionmodelmanagestoholdthevaluesofotherexplanatoryvariablesfixedevenif,inreality,theyarecorrelatedwiththeexplanatoryvariableunderconsideration?Ceterisparibus“-interpretationIthasstilltobeassumedthatunobservedfactorsdonotchangeiftheexplanatoryvariablesarechangedByhowmuchdoesthedependentvariablechangeifthej-thindependentvariableisincreasedbyoneunit,holdingallotherindependentvariablesandtheerrortermconstantMultipleRegression

Analysis:EstimationExample:DeterminantsofcollegeGPAInterpretationHoldingACTfixed,anotherpointonhighschoolgradepointaverageisassociatedwithanother.453pointscollegegradepointaverageOr:IfwecomparetwostudentswiththesameACT,butthehsGPAofstudentAisonepointhigher,wepredictstudentAtohaveacolGPAthatis.453higherthanthatofstudentBHoldinghighschoolgradepointaveragefixed,another10pointsonACTareassociatedwithlessthanonepointoncollegeGPAGradepointaverageatcollegeHighschoolgradepointaverageAchievementtestscoreMultipleRegression

Analysis:Estimation?Partiallingout“interpretationofmultipleregressionOnecanshowthattheestimatedcoefficientofanexplanatoryvariableinamultipleregressioncanbeobtainedintwosteps:1)Regresstheexplanatoryvariableonallotherexplanatoryvariables2)RegressontheresidualsfromthisregressionWhydoesthisprocedurework?TheresidualsfromthefirstregressionisthepartoftheexplanatoryvariablethatisuncorrelatedwiththeotherexplanatoryvariablesTheslopecoefficientofthesecondregressionthereforerepresentstheisolatedeffectoftheexplanatoryvariableonthedep.variableMultipleRegression

Analysis:EstimationPropertiesofOLSonanysampleofdataFittedvaluesandresidualsAlgebraicpropertiesofOLSregressionFittedorpredictedvaluesResidualsDeviationsfromregressionlinesumuptozeroCorrelationsbetweendeviationsandregressorsarezeroSampleaveragesofyandoftheregressorslieonregressionlineMultipleRegression

Analysis:EstimationGoodness-of-FitDecompositionoftotalvariationR-squaredAlternativeexpressionforR-squaredNoticethatR-squaredcanonlyincreaseifanotherexplanatoryvariableisaddedtotheregressionR-squaredisequaltothesquaredcorrelationcoefficientbetweentheactualandthepredictedvalueofthedependentvariableMultipleRegression

Analysis:EstimationExample:ExplainingarrestrecordsInterpretation:Proportionpriorarrests+0.5!-.075=-7.5arrestsper100menMonthsinprison+12!-.034(12)=-0.408arrestsforgivenmanQuartersemployed+1!-.104=-10.4arrestsper100menNumberoftimesarrested1986ProportionpriorarreststhatledtoconvictionMonthsinprison1986Quartersemployed1986MultipleRegression

Analysis:EstimationExample:Explainingarrestrecords(cont.)Anadditionalexplanatoryvariableisadded:Interpretation:Averagepriorsentenceincreasesnumberofarrests(?)LimitedadditionalexplanatorypowerasR-squaredincreasesbylittleGeneralremarkonR-squaredEvenifR-squaredissmall(asinthegivenexample),regressionmaystillprovidegoodestimatesofceterisparibuseffectsAveragesentenceinpriorconvictionsR-squaredincreasesonlyslightlyMultipleRegression

Analysis:EstimationStandardassumptionsforthemultipleregressionmodelAssumptionMLR.1(Linearinparameters)AssumptionMLR.2(Randomsampling)Inthepopulation,therelation-shipbetweenyandtheexpla-natoryvariablesislinearThedataisarandomsampledrawnfromthepopulationEachdatapointthereforefollowsthepopulationequationMultipleRegression

Analysis:EstimationStandardassumptionsforthemultipleregressionmodel(cont.)AssumptionMLR.3(Noperfectcollinearity)RemarksonMLR.3Theassumptiononlyrulesoutperfectcollinearity/correlationbet-weenexplanatoryvariables;imperfectcorrelationisallowedIfanexplanatoryvariableisaperfectlinearcombinationofotherexplanatoryvariablesitissuperfluousandmaybeeliminatedConstantvariablesarealsoruledout(collinearwithintercept)?Inthesample(andthereforeinthepopulation),noneoftheindependentvariablesisconstantandtherearenoexactrelationshipsamongtheindependentvariables“MultipleRegression

Analysis:EstimationExampleforperfectcollinearity:smallsampleExampleforperfectcollinearity:relationshipsbetweenregressorsInasmallsample,avgincmayaccidentallybeanexactmultipleofexpend;itwillnotbepossibletodisentangletheirseparateeffectsbecausethereisexactcovariationEithershareAorshareBwillhavetobedroppedfromtheregressionbecausethereisanexactlinearrelationshipbetweenthem:shareA+shareB=1MultipleRegression

Analysis:EstimationStandardassumptionsforthemultipleregressionmodel(cont.)AssumptionMLR.4(Zeroconditionalmean)Inamultipleregressionmodel,thezeroconditionalmeanassumptionismuchmorelikelytoholdbecausefewerthingsendupintheerrorExample:AveragetestscoresThevalueoftheexplanatoryvariablesmustcontainnoinformationaboutthemeanoftheunobservedfactorsIfavgincwasnotincludedintheregression,itwouldendupintheerrorterm;itwouldthenbehardtodefendthatexpendisuncorrelatedwiththeerrorMultipleRegression

Analysis:EstimationDiscussionofthezeromeanconditionalassumptionExplanatoryvariablesthatarecorrelatedwiththeerrortermarecalledendogenous;endogeneityisaviolationofassumptionMLR.4Explanatoryvariablesthatareuncorrelatedwiththeerrortermarecalledexogenous;MLR.4holdsifallexplanat.var.areexogenousExogeneityisthekeyassumptionforacausalinterpretationoftheregression,andforunbiasednessoftheOLSestimatorsTheorem3.1(UnbiasednessofOLS)Unbiasednessisanaveragepropertyinrepeatedsamples;inagivensample,theestimatesmaystillbefarawayfromthetruevaluesMultipleRegression

Analysis:EstimationIncludingirrelevantvariablesinaregressionmodelOmittingrelevantvariables:thesimplecase=0inthepopulationNoproblembecause.However,includingirrevelantvariablesmayincreasesamplingvariance.Truemodel(containsx1andx2)Estimatedmodel(x2isomitted)MultipleRegression

Analysis:EstimationOmittedvariablebiasConclusion:AllestimatedcoefficientswillbebiasedIfx1andx2arecorrelated,assumealinearregressionrelationshipbetweenthemIfyisonlyregressedonx1thiswillbetheestimatedinterceptIfyisonlyregressedonx1,thiswillbetheestimatedslopeonx1errortermMultipleRegression

Analysis:EstimationExample:OmittingabilityinawageequationWhenistherenoomittedvariablebias?IftheomittedvariableisirrelevantoruncorrelatedWillbothbepositiveThereturntoeducationwillbeoverestimatedbecause.Itwilllookasifpeoplewithmanyyearsofeducationearnveryhighwages,butthisispartlyduetothefactthatpeoplewithmoreeducationarealsomoreableonaverage.MultipleRegression

Analysis:EstimationOmittedvariablebias:moregeneralcasesNogeneralstatementspossibleaboutdirectionofbiasAnalysisasinsimplecaseifoneregressoruncorrelatedwithothersExample:OmittingabilityinawageequationTruemodel(containsx1,x2andx3)Estimatedmodel(x3isomitted)Ifexperisapproximatelyuncorrelatedwitheducandabil,thenthedirectionoftheomittedvariablebiascanbeasanalyzedinthesimpletwovariablecase.MultipleRegression

Analysis:EstimationStandardassumptionsforthemultipleregressionmodel(cont.)AssumptionMLR.5(Homoscedasticity)Example:WageequationShorthandnotationThevalueoftheexplanatoryvariablesmustcontainnoinformationaboutthevarianceoftheunobservedfactorsThisassumptionmayalsobehardtojustifyinmanycaseswithAllexplanatoryvariablesarecollectedinarandomvectorMultipleRegression

Analysis:EstimationTheorem3.2(SamplingvariancesofOLSslopeestimators)UnderassumptionsMLR.1–MLR.5:VarianceoftheerrortermTotalsamplevariationinexplanatoryvariablexj:R-squaredfromaregressionofexplanatoryvariablexjonallotherindependentvariables(includingaconstant)MultipleRegression

Analysis:EstimationComponentsofOLSVariances:1)TheerrorvarianceAhigherrorvarianceincreasesthesamplingvariancebecausethereismore?noise“intheequationAlargeerrorvariancenecessarilymakesestimatesimpreciseTheerrorvariancedoesnotdecreasewithsamplesize2)ThetotalsamplevariationintheexplanatoryvariableMoresamplevariationleadstomorepreciseestimatesTotalsamplevariationautomaticallyincreaseswiththesamplesizeIncreasingthesamplesizeisthusawaytogetmorepreciseestimatesMultipleRegression

Analysis:Estimation3)LinearrelationshipsamongtheindependentvariablesSamplingvarianceofwillbethehigherthebetterexplanatoryvariablecanbelinearlyexplainedbyotherindependentvariablesTheproblemofalmostlinearlydependentexplanatoryvariablesiscalledmulticollinearity(i.e.forsome)Regressonallotherindependentvariables(includingaconstant)TheR-squaredofthisregressionwillbethehigherthebetterxjcanbelinearlyexplainedbytheotherindependentvariablesMultipleRegression

Analysis:EstimationAnexampleformulticollinearityAveragestandardizedtestscoreofschoolExpendituresforteachersExpendituresforin-structionalmaterialsOtherex-pendituresThedifferentexpenditurecategorieswillbestronglycorrelatedbecauseifaschoolhasalotofresourcesitwillspendalotoneverything.Itwillbehardtoestimatethedifferentialeffectsofdifferentexpenditurecategoriesbecauseallexpendituresareeitherhighorlow.Forpreciseestimatesofthedifferentialeffects,onewouldneedinformationaboutsituationswhereexpenditurecategorieschangedifferentially.Asaconsequence,samplingvarianceoftheestimatedeffectswillbelarge.MultipleRegression

Analysis:EstimationDiscussionofthemulticollinearityproblemIntheaboveexample,itwouldprobablybebettertolumpallexpen-diturecategoriestogetherbecauseeffectscannotbedisentangledInothercases,droppingsomeindependentvariablesmayreducemulticollinearity(butthismayleadtoomittedvariablebias)Onlythesamplingvarianceofthevariablesinvolvedinmulticollinearitywillbeinflated;theestimatesofothereffectsmaybeverypreciseNotethatmulticollinearityisnotaviolationofMLR.3inthestrictsenseMulticollinearitymaybedetectedthrough?varianceinflationfactors“Asan(arbitrary)ruleofthumb,thevarianceinflationfactorshouldnotbelargerthan10MultipleRegression

Analysis:EstimationVariancesinmisspecifiedmodelsThechoiceofwhethertoincludeaparticularvariableinaregressioncanbemadebyanalyzingthetradeoffbetweenbiasandvarianceItmightbethecasethatthelikelyomittedvariablebiasinthemisspecifiedmodel2isovercompensatedbyasmallervarianceTruepopulationmodelEstimatedmodel1Estimatedmodel2MultipleRegression

Analysis:EstimationVariancesinmisspecifiedmodels(cont.)Case1:Case2:Conditionalonx1andx2,thevarianceinmodel2isalwayssmallerthanthatinmodel1Conclusion:DonotincludeirrelevantregressorsTradeoffbiasandvariance;Caution:biaswillnotvanisheveninlargesamplesMultipleRegression

Analysis:EstimationEstimatingtheerrorvarianceTheorem3.3(Unbiasedestimatoroftheerrorvariance)Anunbiasedestimateoftheerrorvariancecanbeobtainedbysubstractingthenumberofestimatedregressioncoefficientsfromthenumberofobservations.