BP神經(jīng)網(wǎng)絡(luò)模型推導(dǎo)

1、Theprojectdescribesteachingprocessofmulti-layerneuralnetworkemployingbackpropagationalgoritlmi.Toillustratethisprocessthethreelayerneuralnetworkwithtwoinputsandoneoutput.whichisshowninthepicturebelow,isused:Eachneuroniscomposedoftwounits.Fiistunitaddsproductsofweightscoefficientsandinputsignals.Thes

2、econdunitrealisenonlinearfiinction,calledneuronactivationfunction.Signaleisadderoutputsignal,andy=f(e)isoutputsignalofnonlinearelement.SignalyisalsooutputsignalofneuronX1X1X2Toteachtheneuralnetworkweneedtrainingdataset.Thetrainingdatasetconsistsofinputsignals(x/andX2)assignedwithcorrespondingtarget(

3、deskedoutput)乙ThenetworktrainingisaniterativeprocessIneachiterationweightscoefficientsofnodesaremodifiedusingnewdatafromtrainingdataset.Modificationiscalculatedusingalgoritlmidescribedbelow:Eachteachingstepstartswithforcingbothinputsignalsfromtrainingset.Afterthisstagewecandetermineoutputsignalsvalu

4、esforeachneuronineachnetworklayerPicturesbelowillustratehowsignalispropagatingthroughthenetwork,Symbolsw(xm)nrepresentweightsofcoimectionsbetweennetworkinputxmandneuron/?ininputlayer.SymbolsynrepresentsoutputsignalofneuronXXPropagationofsignalsthroughthehiddenlayerSymbolswmnrepresentweightsofcomiect

5、ionsbetweenoutputofneuronmandinputofneuronninthenextlaver丁5=/5（15丁1十”25丁2十比5）勺）Propagationofsignalsthroughtheoutputlayer.Inthenextalgoritlmisteptheoutputsignalofthenetworkyiscomparedwiththedesiiedoutputvalue(thetarget),whichisfoundilltrainingdataset.Thedifferenceiscallederrorsignal3ofoutputlayerneur

6、onZyItisimpossibletocomputeerrorsignalforinternalneuronsdirectly,becauseoutputvaluesoftheseneuronsareuiikiiown.Formanyyearstheeffectivemethodfortrainingmultiplayernetworkshasbeenunknown.Onlyinthemiddleeightiesthebackpropagationalgoritlmihasbeenworkedout.TheideaistopropagateerrorsignalS(computedinsin

7、gleteachingstep)backtoallneurons,whichoutputsignalswereinputfordiscussedneuron.Theweights1coefficientswmnusedtopropagateerrorsbackareequaltothisusedduringcomputingoutputvalue.Onlythediiectionofdataflowischanged(signalsarepropagatedfromoutputtoinputsoneaftertheother).Thisteclmiqueisusedforallnetworkl

8、ayers.Ifpropagatederrorscamefromfewneuronstheyareadded.Theillustrationisbelow:JXXX1X2X1二叫46+255yWhentheerrorsignalforeachneuroniscomputed,theweightscoefficientsofeachneuroninputnodemaybemodified.Informulasbelowdf(e)/derepresentsderivativeofneuronactivationfunction(whichweightsaremodified)dfi(e)/dw(x

9、i)i=dfi(e)/de*de/dw(xi)i=df(e)/de*Xi閥4=眄4+帀習(xí)卑旳用24=“24+叭纓12cte卅15=嗎5十爲(wèi)聞25二叱5+帀$葺丁2川46二企耳aeCoefficient77affectsnetworkteachingspeed.ThereareafewteclmiquestoselectthisparameterThefustmethodistostartteachingprocesswithlargevalueoftheparameterWhileweightscoefficientsarebeingestablishedtheparameterisbeingdecreasedgraduallyThesecond,morecomplicated,methodstartsteachingwithsmallparametervalue.Duringtheteachingprocesstheparameterisbeingincreasedwhentheteachingisadvancedandthendecreasedagaininthef