人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值

人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第1頁2第四章

函數(shù)、導(dǎo)數(shù)、最值1.導(dǎo)數(shù)和方向?qū)?shù)

2.導(dǎo)數(shù)幾何意義3.函數(shù)最值4.梯度下降法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第2頁3導(dǎo)數(shù)概念函數(shù)f(x)在x=x0處瞬時(shí)改變率為：

咱們稱它為函數(shù)f(x)在x=x0處導(dǎo)數(shù)．記作：人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第3頁4導(dǎo)數(shù)概念xoyx0x0+xx0+xyx<0x>0y=f(x)比如,y=f(x),如圖人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第4頁5方向?qū)?shù)概念表示在x0處沿x

軸正方向改變率.表示在x0處沿x

軸負(fù)方向改變率.人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第5頁6方向?qū)?shù)概念又比如,z=f(x,y),偏導(dǎo)數(shù)分別表示函數(shù)在點(diǎn)(x0,y0)沿x軸方向,沿y軸方向改變率.人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第6頁7方向?qū)?shù)概念如圖xoyzx0(x0,y0)y人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第7頁8方向?qū)?shù)概念表示在(x0,y0)處沿y

軸正方向改變率.表示在(x0,y0)處沿y

軸負(fù)方向改變率.人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第8頁9方向?qū)?shù)概念yxzoz=f(x,y)X0M0即f'x

(x0,y0)表示y=y0與z=f(x,y)交線在M0處切線對(duì)x斜率.T11

:z=f(x,y0)1y0把偏導(dǎo)數(shù)概念略加推廣即可得到方向?qū)?shù)概念.人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第9頁10方向?qū)?shù)概念yxzoz=f(x,y)M0X022

:z=f(x0,y)即f'y

(x0,y0)表示x=x0與z=f(x,y)交線在M0處切線對(duì)y斜率.x0T2人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第10頁11方向?qū)?shù)概念如圖xoyzM0lX0=(x0,y0)X=(x0+x,

y0+y)MN人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第11頁12方向?qū)?shù)概念若z=f(X)=f(x,y)在X0=(x0,y0)處偏導(dǎo)存在.則在X0處沿x

軸正向方向?qū)?shù),人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第12頁13方向?qū)?shù)概念在X0處沿x

軸負(fù)方向方向?qū)?shù),一樣可得沿y

軸正向方向?qū)?shù)為f'y(x0,y0),而沿y

軸負(fù)方向方向?qū)?shù)為–f'y(x0,y0).人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第13頁14導(dǎo)數(shù)概念設(shè)求例：解：人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第14頁導(dǎo)數(shù)幾何意義βy=f(x)PQMΔxΔyOxyβPy=f(x)QMΔxΔyOxy

如圖,曲線C是函數(shù)y=f(x)圖象,P(x0,y0)是曲線C上任意一點(diǎn),Q(x0+Δx,y0+Δy)為P鄰近一點(diǎn),PQ為C割線,PM//x軸,QM//y軸,β為PQ傾斜角.斜率!則人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第15頁導(dǎo)數(shù)幾何意義PQoxyy=f(x)割線切線T請(qǐng)看當(dāng)點(diǎn)Q沿著曲線逐步向點(diǎn)P靠近時(shí),割線PQ繞著點(diǎn)P逐步轉(zhuǎn)動(dòng)情況.人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第16頁導(dǎo)數(shù)幾何意義

設(shè)切線傾斜角為α,那么當(dāng)Δx→0時(shí),割線PQ斜率,稱為曲線在點(diǎn)P處切線斜率.①提供了求曲線上某點(diǎn)切線斜率一個(gè)方法;

②切線斜率本質(zhì)——函數(shù)在x=x0處導(dǎo)數(shù).人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第17頁(3)當(dāng)t=t2

時(shí),曲線h(t)

在t2處切線l2斜率h’(t2)<0.故在t=t2

附近曲線下降,即函數(shù)h(t)

在t=t2

附近也單調(diào)遞減.導(dǎo)數(shù)幾何意義例曲線h(t)在t0,t1,t2附近改變情況.解:可用曲線h(t)

在

t0,t1,t2

處切線刻畫曲線h(t)

在上述三個(gè)時(shí)刻附近改變情況.(1)當(dāng)t=t0

時(shí),曲線h(t)

在t0

處切線l0

平行于x軸.故在t=t0

附近曲線比較平坦,幾乎沒有升降.(2)當(dāng)t=t1

時(shí),曲線h(t)

在t1

處切線l1斜率h’(t1)<0.故在t=t1

附近曲線下降,即函數(shù)h(t)

在t=t1

附近單調(diào)遞減.tohl0t0t1l1t2l2t4t3

從圖能夠看出，直線

l1

傾斜程度小于直線l2

傾斜程度，這說明h(t)曲線在l1

附近比在l2

附近下降得遲緩人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第18頁aby=f(x)xoyy=f(x)xoyabf'(x)>0f'(x)<0設(shè)函數(shù)y=f(x)在某個(gè)區(qū)間內(nèi)可導(dǎo)，函數(shù)單調(diào)性與導(dǎo)數(shù)關(guān)系為增函數(shù)為減函數(shù)為常數(shù)人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第19頁凹凸曲線凹凸性人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第20頁定理:設(shè)f(x)在[a,b]上連續(xù)，在(a,b)內(nèi)含有一階和二階導(dǎo)數(shù)，那么例1解所以曲線是凸。曲線凹凸性人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第21頁例2解曲線凹凸性人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第22頁拐點(diǎn)定義：曲線凹凸性普通地，設(shè)y=f(x)在I上連續(xù)，x0是I內(nèi)點(diǎn)，假如曲線y=f(x)在經(jīng)過點(diǎn)(x0，f(x0))時(shí)，曲線凹凸性改變了，那么稱點(diǎn)(x0，f(x0))為這曲線拐點(diǎn)。人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第23頁解凹凸凹拐點(diǎn)拐點(diǎn)曲線凹凸性人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第24頁25

下列圖是導(dǎo)函數(shù)圖象,試找出函數(shù)極值點(diǎn),并指出哪些是極大值點(diǎn),哪些是極小值點(diǎn).abxyx1Ox2x3x4x5x6人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第25頁函數(shù)極值定義設(shè)函數(shù)f(x)在點(diǎn)x0附近有定義，假如對(duì)x0附近全部點(diǎn)，都有f(x)<f(x0),則f(x0)是函數(shù)f(x)一個(gè)極大值，記作y極大值=f(x0)；假如對(duì)x0附近全部點(diǎn)，都有f(x)>f(x0),則f(x0)是函數(shù)f(x)一個(gè)極小值，記作y極小值=f(x0)；函數(shù)極大值與極小值統(tǒng)稱為極值

使函數(shù)取得極值點(diǎn)x0稱為極值點(diǎn)人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第26頁用導(dǎo)數(shù)法求解函數(shù)極值步驟(1)求導(dǎo)函數(shù)f'(x)；(2)求解方程f'(x)=0；(3)檢驗(yàn)f'(x)在方程f'(x)=0根左右符號(hào),并依據(jù)符號(hào)確定極大值與極小值.口訣：左負(fù)右正為極小，左正右負(fù)為極大。人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第27頁函數(shù)極值判定定理假如函數(shù)f(x)在x0附近有連續(xù)二階導(dǎo)數(shù)f"(x)，f'(x0)＝0，f"(x)≠0，那么⑴若f"(x0)＜0，則函數(shù)f(x)在點(diǎn)x0處取得極大值⑵若f"(x0)＞0，則函數(shù)f(x)在點(diǎn)x0處取得極小值人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第28頁例：求以下函數(shù)極值f(x)＝2x3－3x2解：f'(x)＝6x2－6x，f"(x)＝12x－6令6x2－6x＝0，得駐點(diǎn)為x1＝1，x2＝0∵f"(1)＝6＞0，f"(0)＝－6＜0把x1＝1，x2＝0代入原函數(shù)計(jì)算得f(1)＝－1、

f(0)＝0∴當(dāng)x＝1時(shí)，y極?。剑?，x＝0時(shí)，y極大＝0函數(shù)極值判定人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第29頁例：求以下函數(shù)極值f(x)＝sinx＋cosx，x∈[0,2π]解：f'(x)＝cosx－sinx，令cosx－sinx＝0，得駐點(diǎn)為x1＝，x2＝，又f"(x)＝－sinx－cosx，把x1＝，x2＝代入原函數(shù)計(jì)算得f()＝、f()＝－。所以當(dāng)x＝時(shí)，y極大＝，x＝時(shí)，y極小＝－[注意]假如f'(x0)＝0，f"(x0)＝0或不存在，本定理無效，則需要考查點(diǎn)x0兩邊f(xié)'(x0)符號(hào)來判定是否為函數(shù)極值點(diǎn)。函數(shù)極值判定人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第30頁實(shí)例問題實(shí)質(zhì)：?jiǎn)栴}：一塊長(zhǎng)方形金屬板，四個(gè)頂點(diǎn)坐標(biāo)是(1,1)，(5,1)，(1,3)，(5,3)．在坐標(biāo)原點(diǎn)處有一個(gè)火焰，它使金屬板受熱．假定板上任意一點(diǎn)處溫度與該點(diǎn)到原點(diǎn)距離成反比．在(3,2)處有一個(gè)螞蟻，問這只螞蟻應(yīng)沿什么方向爬行才能最快抵達(dá)較涼快地點(diǎn)？應(yīng)沿由熱變冷改變最驟烈方向（即梯度方向）爬行．人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第31頁梯度概念人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第32頁在幾何上表示一個(gè)曲面曲面被平面所截得所得曲線在xoy面上投影如圖等高線梯度為等高線上法向量梯度概念人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第33頁等高線畫法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第34頁等高線畫法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第35頁等高線畫法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第36頁等高線畫法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第37頁等高線畫法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第38頁等高線畫法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第39頁等高線畫法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第40頁等高線畫法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第41頁等高線畫法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第42頁等高線畫法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第43頁比如,等高線畫法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第44頁梯度與等高線關(guān)系：人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第45頁類似于二元函數(shù)，此梯度也是一個(gè)向量，其方向與取得最大方向?qū)?shù)方向一致，其模為方向?qū)?shù)最大值.梯度概念能夠推廣到三元函數(shù)人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第46頁梯度概念能夠推廣到三元函數(shù)人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第47頁在處梯度解由梯度計(jì)算公式得故梯度概念能夠推廣到三元函數(shù)人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第48頁49梯度下降法又稱最速下降法。函數(shù)J(a)在某點(diǎn)ak梯度是一個(gè)向量，其方向是J(a)增加最快方向。顯然，負(fù)梯度方向是J(a)降低最快方向。在梯度下降法中，求某函數(shù)極大值時(shí)，沿著梯度方向走，能夠最快到達(dá)極大點(diǎn)；反之，沿著負(fù)梯度方向走，則最快地到達(dá)極小點(diǎn)。梯度下降法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第49頁梯度下降法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第50頁51求函數(shù)J(a)極小值問題，能夠選擇任意初始點(diǎn)a0，從a0出發(fā)沿著負(fù)梯度方向走，可使得J(a)下降最快。s(0)：點(diǎn)a0搜索方向。梯度下降法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第51頁52對(duì)于任意點(diǎn)ak，能夠定義ak點(diǎn)負(fù)梯度搜索方向單位向量為:從ak點(diǎn)出發(fā)，沿著方向走一步，步長(zhǎng)為,得到新點(diǎn)ak+1,表示為：梯度下降法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第52頁53梯度下降法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第53頁54所以，在新點(diǎn)ak+1，函數(shù)J(a)函數(shù)值為：全部ak組成一個(gè)序列，該序列由迭代算法生成

a0,a1,a2,….,ak,ak+1,...該序列在一定條件下收斂于使得J(a)最小解a*迭代算法公式：梯度下降法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第54頁55迭代算法公式：關(guān)鍵問題：怎樣設(shè)計(jì)步長(zhǎng)假如選得太小，則算法收斂慢，假如選得太大，

可能會(huì)造成發(fā)散。

梯度下降法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第55頁56梯度法迭代過程：選取初始點(diǎn)a0，給定允許誤差α>0,β>0,并令k=0。計(jì)算負(fù)梯度及其單位向量。檢驗(yàn)是否滿足條件，若滿足則轉(zhuǎn)8，不然繼續(xù)。計(jì)算最正確步長(zhǎng)。令：計(jì)算并檢驗(yàn)另一判據(jù)：，滿足轉(zhuǎn)8，不然繼續(xù)。令k=k+1,轉(zhuǎn)2。輸出結(jié)果，結(jié)束。梯度下降法人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第56頁57目標(biāo)函數(shù)曲面J(W)人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第57頁58目標(biāo)函數(shù)曲面J(W)--連續(xù)、可微人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第58頁59目標(biāo)函數(shù)曲面J(W)--連續(xù)、可微人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第59頁60全局極小點(diǎn)人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第60頁61局部極小點(diǎn)1人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第61頁62局部極小點(diǎn)2人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第62頁63目標(biāo)函數(shù)曲面J(W)人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第63頁64目標(biāo)函數(shù)曲面J(W)--連續(xù)人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第64頁65目標(biāo)函數(shù)曲面J(W)--連續(xù)、可微人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第65頁66初始狀態(tài)1人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第66頁67搜索尋優(yōu)－－梯度下降人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第67頁68搜索尋優(yōu)－－梯度下降人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第68頁69搜索尋優(yōu)－－梯度下降人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第69頁70搜索尋優(yōu)－－梯度下降人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第70頁71搜索尋優(yōu)－－梯度下降人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第71頁72搜索尋優(yōu)－－梯度下降人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第72頁73搜索尋優(yōu)－－梯度下降人工神經(jīng)網(wǎng)絡(luò)函數(shù)導(dǎo)數(shù)最值第73頁74搜索尋優(yōu)－