機(jī)器學(xué)習(xí)實(shí)驗(yàn)報(bào)告

1、曲美邦野大拳X rAN UNI VE RSfTY OF 戶 8T && TEL ECOMMUNICATIONS機(jī)器學(xué)習(xí)課內(nèi)實(shí)驗(yàn)報(bào)告(1) m算法實(shí)現(xiàn)決策樹2015 - 2016學(xué)年第2學(xué)期專業(yè):智能科學(xué)與技術(shù)班級(jí):智能1301班學(xué)號(hào):06133029姓名:張爭(zhēng)輝一、實(shí)驗(yàn)?zāi)康模豪斫?D 3算法的基本原理，并且編程實(shí)現(xiàn)。二、實(shí)驗(yàn)要求：使用C/C+4ATLAB實(shí)現(xiàn)D 3算法。輸入：若干行，每行5個(gè)字符串，表示0 utbok T em perature Humility W hd P lay ball 如上表。輸出：決策樹。實(shí)驗(yàn)結(jié)果如下: 輸入：SunnySunny0 veicastH

2、 ot H fehH ot H fehH ot H ighW eak N oS tong N oW eak YesRah M ild HighW eak YesR ahC ool N oim alR ahC ool N oim alW eak YesStrong N o0 veicast C ool N om alS tong Y esSunny M iH H 適h W eak No SunnyC ool Nom alW eak YesR ah M iH N onn al Weak Yes SunnyMild N om alS tong Y es0 veicast M fld High S t

3、ong Y es 0 veicast H ot N om al W eak Y esRah M iH HighStrong N o輸出：0 utbokRah W hdStrongN oW eakYes0 veicastYesSunnyHum idityN oim alYesHfeh No三、具體實(shí)現(xiàn):實(shí)現(xiàn)算法如下： tthclide <bstream > tthclide <fettBam > ffhchde <m a1h.h> tthclide <string> using nam espace std;ftdefhe ROW 14ttdefh

4、e COL 5ftdefhe bg2 0.69314718055typedef stmctTN ode(char data 15;charw efeht15;TN ode * firstchild,和extsiblhg;typedef stmctLN ode(char 0 utLook15;char Tem pemtLiiE 15;char H um idity 15;char W hd15;char P byTennis后;LN ode 的lext;*lhk;typedef stmctA tttN ode(char attributes 15;/® t生ht atu” um 性的個(gè)

5、數(shù)A tttN ode 和ext;M ttrbutes;char*Exam plesR0 W C 0 L = /'0 vetC ast,C ool；、翦h,S ta)ng/N o, / ，ZR ah，,zzH of；' oral al；”S tong，z/7 es，“Sunny”,H of；、igh，細(xì) eak, o，“Sunny”,、of；? igh,S tongz，,/zN o，0 veiC as，、eak,Y es，Ratr31垣h，9 eak/Yes1R ah，z,T ooY；N otm aT,/ZV eak,乍 es，R ah，z,T ooT/'N oun a

6、l'/S trong/N o，0 veiC as，C oof/ otm aT/'S tongz/7 es，SunnyM州 eakWo，S tinny”,C ooI'JN otm aT,eak，77 es，ZR a h，z,，XM ild/，,，/N om aT/V eak,Yes，Sunny/M fld,Nom al'/S trong,Yes，0 veiC as，'加 ildt'W oral al；S tong/z/7 es，0 veiC as，？ otm aT,?AV eak,Y es, R an , M in , H jgh , S tron

7、g , N o);char*Attributes_khd4 = "0 ulLook/Teni peiature",?um Hity,lY hd/z;ht Atttikhd4= 3,3,2,2;char*0 utLook_khd 3="Sunny”,0 veiC asf',R ail;char* T em peratLiw_kiid 3 =Hd,C oof; char*Hum idity_khd 2 = fH oim ap;char*W hd_khd 2=ea'k,S trong?;Aiiti_Exam pple 145 = 0,0,0,0,1,0,

8、0,0,1,1,1,0,0,1,0,24,0,0,0,2,2,1,0,0,l,2,l,l,0,04,0,0,1,0,2,1,0,0,24,1,0,0,0,1,1,1,0,14,1,1,0,14,1,0,0,24,0,0,1 ;*/void tree lists (tree T);void hitAttr(Attributes &attiilhk,char*Attributes_khdD,htAttckhdD)；void InitLhkQhk &L,char*Exam pies DIG 0 LD;void ID 3 (tree &T,lhk L,lhk TaigelA t

9、ti;Attributes attr);void PN_Num Ohk L,ht&positve,ht&negative);double G ah (htpositi/e,htnegati/e,char*atrrbute,lrik L,A ttributes attiL);void m ah 0Ihk LL,p;Attributes attiiL,q;tree T;T = new TN ode;T - firBtchfU = T>nextsi)lhg = NULL;stepy(Tw eighth;sttcpy (rdata/z/);attiiL = new A ttiN

10、ode ;atttiL-next二 NULL;LL = new LN ode ;LL少next= NULL;彼功建立兩個(gè)鏈表hiLink(LL,Exam pies);IhitA ttr(atttiL,A ttributes_khd,A ttiikhd);D3(r,LL,NULL,attiLL);couK決策樹以廣義表形式輸出如下：(endl; tree lists (T)；/1、廣義表的形式輸出樹/ cout<<G ah(9,5,/zO utLook,LL,attiiL)Vendl;cout<<endl;)力以廣義表的形式輸出樹void tree lists (tree

11、 T)(tree p;if(FT)return;couK<(< T-> w e 這hK'?； cout<<TH>data;p = Tfctehfld;if 3)cout(；while 6)treefets G);p = p>nextsi)lhg;if 3)couK','；)couKV'Y;)void hitA ttr(A ttributes &attihk,char* A ttributes_khd D,htA ttr_khd 口) (Attributes p;for (hti=0 ;i< 4廿+)p =

12、new A tttN ode ;p>next= NU LL;s trcp y 33attrib u te s, A ttrib u te s_k ii d ；pH>at&iN um = Attiikhdi；pH>next= attclhk-next;attiilhk-next二 p;)void InitLhkQhk &LL,char*Exam pJesDlC 0 L)(link p;for (ht i= 0 ;i< R 0 W ;i<-+)p 二 new LN ode ;pH>next= NU LL;strcpy630 utLook,Exam

13、 pesi0D;strcpy )S>Tem petature,Exam piesil);stepy)>Hum ility,Exam plesi2);stepy(p>W hd,Exam pbs|i3);strcpy (pH>P byTenn is,Exam pies 田口 );p>next= LL-next;LL>next= p;)void PN_Num (hk L,ht&positve,ht&negative)positve = 0;negative = 0;link p;p = Lnext;while (p)if fetrcm p 39P

14、JayTennis,，zN o'0二二 0) negative+;eke if (stem p 33P JayT ennis/Y esQ = 0) positve+;p 二 pH>next;/M算信息增益#lhk L :樣本集合S/a-ttr_L ：屬性集合double G ah (htpositive,htnega1i/e,char*atirbute,lhk L,A ttributes atttL) (htataikhds;/每個(gè)屬性中的值的個(gè)數(shù)Attributes p = a.t&iL9next;link q 二 Lnext;htattclh = 0;4第幾個(gè)屬性wh

15、ile 6)if fetem p 39attributes,attribute) = 0)atoikhds = pattiiNum ;break;p = pH>next;atliilh+;)double entopy,gah=0;double pl = 1.0和ositive/3ositive + negative);double p2 = 1.0的iegative/3ositive + negative);entopy 二 0l*bg &l)/bg2 -p2*bg 2)/bg2 ;集合烙gain = entopy;%軼取每個(gè)屬性值在訓(xùn)練樣本中出現(xiàn)的個(gè)數(shù)娥取每個(gè)屬性值所對(duì)應(yīng)的正例

16、和反例的個(gè)數(shù)明一個(gè)3*ataikhds的數(shù)組ht*khds= new ht* 3;for (ht j=0 ;j< 3 ;jF+) kinds同=new intM/taikiids;保存每個(gè)屬性值在訓(xùn)練樣本中出現(xiàn)的個(gè)數(shù) 闌始化for (ht j= O;jC 3 才+)fcr (hti=0 ;i< ataikiids;H+)(khds|jil= 0;)while if fetem p CO utLook/z,attribute) = 0)for (ht i= 0 ;i< atnikhds ;升+)iffetem p ©90 ulLook,0 utLook_khd =0)

17、 khds0i+;if (stem p (q3P JayT ennis/V esO = 0) kinds li+ ；ekekinds 口田+;)else if fctrcm p CTem peratureattribute) = 0)(fcr (ht i= 0 ;i< ateikhds ;升+)iffetem p (q-Tem perature,Tem perature_khd =0)kinds K)加 + ;iffctrcm p byTennis/V esO 二二 0)khds lil+ ；elsekhds 2i+;)else if fetrcm p OH um attribute)二

18、二 0)(fcr (inti= 0 ;i< atoikiids;升+)(if (stem p G3 H um idity,H um iiity_kiid =0)khds0il+;iffettcm p(a->P JayTennis/V esQ 二二 0)kinds 1擊+ ;elsekhds2i+;)else if fctrcm p QM hdz，,attribute) = 0)(fcr (ht i= 0 ;i< atnikrids ;"+)iffetem p ->W hd,W ind_kiid 二二 0)kinds 0ik+ ；iffctrcm p(a->

19、;P byTennis乍 esO 二二 0) khdsli+;elsekhds 團(tuán) Cfl+;)q 二 q9next;)%計(jì)算信息增益double * gah_khd = new double ataikhds;htpositive_khd = 0,negative_khd = 0;for (ht j= 0 ;j< atrr_khds：jF+)if(khdsOj != 0 && khdslj != 0 && kinds2j != 0) pl = l.Ohds ljl/khds 0j;p2 = 1.0根hds 團(tuán)U/kiidsi)U;gah_khd m=R*

20、bg (pl)/bg2p2*bg (p2)/bg2 ;gah = gah 一 (l.Ohds Oj/ositive + negative)詫aii_kiid m;) " ekegah_khd l=0 ;)return gah;)MD3算法中的訓(xùn)練樣本子集合與屬性子集合的鏈表需要進(jìn)行清空 void FweLhkQhk &Lhk) (Ink p,q;p = Lhk>next;Link-next二 NULL;while (p)Q = P；p = p-next;fee (q);)void 3D 3 (tree &T,lhk L,lhk TatgeO ttt;A ttri

21、butes attr) (Attributes p,m ax,attachiH,pl;link q,lhk_child,ql;tree i;tree_p;iitpositive =0,negative =0 ;PN_Num (L,positive,negative);/初始化兩個(gè)子集合atttichild = new A ttiN ode ;atttichild>next= NU LL;lhk_child 二 new LN ode ;lhk_chill少next= NULL;if (positive = 0)全是反例strcpy (Tdata/ o');return;)else

22、if(negative 二二 0)全是正例stepy (Tdata/V esO;return;)p = atti->next;勿屬T生鏈表double gah,g = 0;#建立屬性子集合與訓(xùn)練樣本子集合有兩個(gè)方案：一：在原來(lái)鏈表的基礎(chǔ)上進(jìn)行刪除；二：另外申請(qǐng)空間進(jìn)行存儲(chǔ)子集合；采用第二種方法雖然浪費(fèi)了空間，但也省了很多事情，避免了變量之間 的應(yīng)用混亂*/般)while (p)gah = G ah positive,negative,p-attributes,L,attr);couK pS attribute s< z，<<gah<<endl;if(gah

23、> g)g = gain;max=p /尋找信息增益最大的屬性)p = p>next;stepy (Tdata,m ax3attributes) 曾加決策樹的節(jié)點(diǎn)couK信 息增益 最大的 屬性：m ax->attributes = z，<<m ax- >attributes<<endK<endl;/門面開始建立決策樹的u建屬性子集合p = attH>next;while (p)if fetrcm p 3-attributes,m ax>attributes) != 0)pl 二 new A ttiN ode ;strcpy (

24、pl attrbutes,p attributes);plattiiNum = pattiiNum ;pl9next= NULL;pl9next= attiichiHnext;atttichild9next二 pl;) p = p>next;)/需要區(qū)分出是哪一種屬性雁立每一層的第一個(gè)節(jié)點(diǎn)if fetrcm p CO utLook,m axattributes) = 0)r- new TN ode;r>firstehild = rnextsblhg = NU LL ;strcpy(H>weight,0 ulLook_khd0);Tfirstehild = r;彼取與屬性值相關(guān)

25、的訓(xùn)練樣例Exam pfe (vi),建立一個(gè)新的訓(xùn)練樣本鏈 表 lhk_childq = L>next;while (q)if (stem p G90 utLook,0 ulLook_kind 0) = 0)ql = new LN ode ;stepy ulLook,q30 utLook);stepy (q1>H um jdity,qH um ility);stepy(q1H>Tem perature,q>Tem perature);strcpy1>W hd,q>W hd);stepy (q1>P JayTennis,q-P byTennfe);ql

26、H>next= NU LL;qlH>next= lhk_child- next;lhk_chHd>next= ql;)q = q 9 next;)else if fcttcm p CTem peiature/z,m ax>attributes) = 0) (r- new TN ode;iS>tetehild = rnextsblhg = NU LL ;stepy(H>w eightTem peiatuiB_kind 0);Tfirstehild = r;力獲取與屬性值相關(guān)的訓(xùn)練樣例Exam pL(vD,建立一個(gè)新的訓(xùn)練樣本鏈 表 lhk_childq 二 L

27、>next;while (q)if (stem p G-Tem perature,?em peiatuiB_kiid 0) = 0) ql = new LN ode;strcpyQl - O ulLook,q30 utLook);stepy(qlH>Hum idily,qHum idity);stepyem perature,q>Tem perature);stepyhd,qH>W hd);stepy (qlH>P JayTennis,qS>P byTennis);qlnext= NU LL;ql>next= lhk_child- next;hk_ch

28、Hdnext= ql;)q = q 9 next;)else if fetrcm p (、um idilym ax-attributes)=二 0)r= new TN ode;r>firstehild = rnextsblhg = NULL;strcpy(H>weightHum ilily_khd0D;Tfirstehild = r;力獲取與屬性值相關(guān)的訓(xùn)練樣例Exam pL(vD,建立一個(gè)新的訓(xùn)練樣本鏈 表 lhk_childq 二 L>next;while (q)if fcttcm p (qH um idity,H um idity_krid 0) = 0)ql = ne

29、w LN ode;stepy (ql>0 ulLook,q0 utLook);stepy(qlHum jdity,qHum ility);stepy(q1H>Tem perature,q>Tem perature);strcpy1>W hd,q>W hd);stepy (q1>P JayTennis,q-P byTennfe);ql>next= NU LL;qlnext= lhk_chill- next;hk_childnext= ql;)q = q 9 next;)else if fctrcm p GV hdz，,m ax>attributes

30、) = 0)r- new TN ode;iS>tetehild = rnextsblhg = NU LL ;strcpy (H>w eight,W hd_khd0);T - ffetchAd = r;獲取與屬性值相關(guān)的訓(xùn)練樣例Exam pfe (vi),建立一個(gè)新的訓(xùn)練樣本鏈 表 lhk_childq = L>next;while (q)if (stem p (qW hd,W hd_kind 0) = 0)(ql = new LN ode;stepy (ql>0 ulLook,q0 utLook);stepyum idity,qHum ility);stepy(q1H&

31、gt;Tem perature,qH>Tem peratuiB);strcpy1>W hd,q>W hd);stepy 1S>P JayTennis,q>P byTennis);qlH>next= NU LL;qlH>next= lhk_childnext;lhk_child-next= ql; )q = q 9 next;)htp = 0,n = 0;PN_N um (lhk_child,p,n);if 30 && n != 0)(ID 3(IH>fetchiH,lnk_chLld,TaigeO tti;attcchid);Fr

32、eeLink(lhk_child);else if(p = 0)stepy (Tfetchilddatao');FreeLhk(lhk_chfld);/stepy (TfinstehilddatajqlP byTennis);/Z一此處應(yīng)該需要修改：)else 跖-=0)(stepyfetch ild data, esO;FreeLhk(lhk_chfld);)力建立每一層上的其他節(jié)點(diǎn)tree_p = Tfctehfld;fcr (ht i= 1 ;i< m ax>atttiN um 廿+)/需要區(qū)分出是哪一種屬性if fetrcm p (0 utLook,m ax>

33、attributes) = 0) r= new TN ode;rfinstehild = r> nexts ib ling = NULL;stepy (H>wefeht,0 ulLook_krid國(guó))； tme_pS>nextsibliig = r;4缺取與屬性值相關(guān)的訓(xùn)練樣例Exam pie (vi),建立一個(gè)新的訓(xùn)練樣本 鏈表 link_childq = L 9 next;while (q)if fetrcm p (qO utLook,0 utLook_khd =0) ql = new LN ode ;sttcpy(fllO utLook,q>0 utLook);s

34、ttcpyQl-Hum idily,q>Hum ility);sttcpyGl-Tem peiatUiB,QTem perature);sttcpy(qIW hd,qW hd);sttcpyQl - P JayTennis,qP byTennis);ql-next= NU LL;ql - next= lhk_chiHnext;lhk_childSnext= ql;)q = q>next;)else if fetrcm p (Tem peiatuie,ni axattributes) = 0)r= new TN ode;rfinstehild = r> nexts ib lin

35、g = NULL;sttcpy (H>w e jght,Tem pera.ture_khd ；tme_pS>nextsibliig = r;4缺取與屬性值相關(guān)的訓(xùn)練樣例Exam pie (vi),建立一個(gè)新的訓(xùn)練樣本 鏈表 link_childq = L 9 next;while (q)if fetrcm p (q-Tem peiature,Tem pera.ture_khd =0)ql = new LN ode ;sttcpyQl90 ulLook,q->0 utLook);strcpy(q1S>H um idily,q>H um idity);stepy1&g

36、t;Tem peiature,qTem peiatutB);sttcpy(q1S>W hd,q>W hd);strcpy (fllP hyTennis,qP byTennis);ql-next= NU LL;ql - next= Ihk_Child-next;Ihk_ChiHSnext= ql;)q = q-next;)else if fctrcm p CH um ax>attr±)utes)=二 0)r= new TN ode;rfinstehild = nextsibling 二 NULL;strcpy (H>w efeht,H um idity_khd

37、1D;tree_pH>nextsibliig 二 r;娥取與屬性值相關(guān)的訓(xùn)練樣例Exam pfeH,建立一個(gè)新的訓(xùn)練樣本 鏈表 lhk_chiHq = L 9 next;while if (stem p G-H um um idity_khd =二 0)ql = new LN ode ;sttcpyQl90 utiLook,q>0 utLook);stepyum idity,qH>Hum ility);sttcpy(qlS>Tem peiature,qTem petature);sttcpyGl W hd,qW hd);sttcpyQl - P JayTennis,qP

38、 byTennis);ql - next= NU LL;ql - next= lhk_chiHnext;lhk_childSnext= ql;)q = q>next;)else if fctrcm p (IV ind,ni axattributes) = 0)r= new TN ode;rffcstehild = nextsibling = NULL;sttcpy (H>w efeht,W hd_khd 1);tree_p->nextsiblhg = r;娥取與屬性值相關(guān)的訓(xùn)練樣例Exam pfe (vi),建立一個(gè)新的訓(xùn)練樣本 鏈表 lhk_chiHq = L 9 next;while if fetem p (q>W hd,W ind_khd =0)ql = new LN ode ;sttcpy(fllO utLook,q>0