信息論與編碼第二章答案解析

1、_ 3 u ,u ,upu )12u，231231111( ) 0p u1p u()0u ，2pu )pu )pu )uu，u，u，2332131122322( ) 0p upu )pu )uu，u 31323332 1/2 0p(s |s ) 3 0 2/3jiW ，W ，W 123111W = W + W + W ，W = W + W , W = W122W +W +W 233233112321332123 296，W =1W =2W =352525 由 符 號 集 ， 組 成 的 二 階 馬 爾 可 夫 鏈 ， 其 轉(zhuǎn) 移 概 率 為 ：00ji00w、w 、w 、w 1234_w w

2、p w p w p w p 0.8w 0.5w111122133144113w w p w p w p w p 0.2w 0.5w211222233244213w w p w p w p w p 0.5w 0.2w311322333344324w w p w p w p w p 0.5w 0.8w411422433444424且 1w w w w；123451415w p(00)p(01)p(10)1411w 71712w 75753 (11)pw141441 6 和5 111 11 和5p(x )= 26 61811I(x )=-lb 4.17bit1811 1 11p(x )= 6 6 3

3、621I(x )=-lb 5.17bit362 _ 1 111 ( )15H X64.337bitlblb36 3618 18213243546578594321x6p ( ( )p x xHiii2111 ( )p x36311I(x )=-lb 1.17bit363 11212( )( )， p xx x p x12121 1 1 1 ( )(H X)lb lb2 2 2 2 ( ) 0.99 ( )0.01p x, p x;12H(X)(0.99lb0.01lb14p x p x p x( ) ( ) ( ) ( )x x x x p x123412341 1 ( )4(H X)bitl

4、b4 4_ ppp(AB) p()P(B| ) 0.250.75 3=p(B)P(B)0.58 37 3 1,2）和（ 36，即概率為1p(x3) 181I(x3)lbp(x3)lb 4.17bit18 7 11p(x7) I(x7)lb bit66X x 0 x 1 x 3 x 31234 P 3 81 41 41 8202 120 130 213 001 203 210 110 321 010 021 032 011 223 2103I(x )=-lb bitx 1811x I(x )=-lb bit422_1x I(x )=-lb bit4331x I(x )=-lb bit844x x

5、 x x 61234I（x x x x x ）i123419.8187.81I 87.81/ 451.95 比特/符號2H(X) p(x )log p(x ) p(x )log p(x ) p(x )log p(x ) p(x )log p(x )112233443111 1.415 2 2 38448bit111I (X)lb bit I (X)lb bit I (X)lb bit248248 5XYXY_13( )x x p x，1212p(x ) 321 2H(X) H( , )( log log )0.5280.390bit3 3 3 3 31 1 223x x 122757p y x

6、()(、 p y x)。11212 2 5 5HY x ) p(y x )log p(y x )( log log )7 7 7 71i1i1i0.5160.3470.863bitYy y 1259 ()(、 p y x)p y x1。141422255 99HY x ) p(y x )log p(y x ) ( log log )14 14 14 142i2i2i0.5300.4100.940bit2HY | X) p(x )HY |x )=p(x )HY x ) p(x )HY x ) =0.91bit/ii1122i025 B ，2121592121225555992121 2121 2

7、121 2121_ 有2 XY(xy) p(x)Y是X pi ji2218321832 3232111.5798 38380.3010X和 , , x x x ，Y , , ( , ) y y y r x y r 為X123123ijijr r r0 7/24 1/24 r r r 1/24 1/4 1/24 01/24 7/24r r r X和YYY33 ( , )H X Y( , ) ( , )p x yp x yijiji1 j1_7711 12 log 4 log log 2.3bit/24 24 24 24 4141p y p y( ) ( ) ( )p y31231 1 1113H

8、Y) p y ( ( ) ( , , )3 1.58bit/p yH3 3 333iii1(X |Y) H(X,Y)HY)0.72bit/ Hp x yH(X Y) p(x ,y )log ()ijijijp(x ,y ) p(x ,y )logijp(y )ijijj7124131714 log241244(2 log log )13142430.1120.50.104bitX和并 定 義 另 一 隨 機 變 量 （ 一 般 乘 積 ） 。 H(X) HY) H(Z) H(X,Z) HY,Z) H(X,Y,Z)， X ) H(X Z) H(Z X) HY Z) H(Z Y)，H(X Y,Z)

9、，HY X,Z)，H(Z X,Y)(X;Y) I(X;Z) IY;Z)( ; )( ;I X Z Y( ; ) I，I X Y Z I Y Z X，1 3 1p(x )=p(xy )+p(xy )= 8 8 211 11 23 1 1p(x )=p(x y )+p(x y )= 8 8 222 12 2( ) ( )log ( ) 1 /H X p xp x bit symboliii_1 3 1p(y )=p(xy )+p(x y )=8 8 23 1 111 12 1p(y )=p(xy )+p(x y )=8 8 221 22 2H ) ) jjjz 0z 1Z127818P)77 18

10、 812kH )p ) ( 8 ) 8k ) z) z )z) ) 0.5z ) 011 11 21 111 2738) z) z) z) ) z)0.5811 12 12 111 11 ) z ) p z ) z ) )821 2222 22H )z )logpz )i ki kik1 3 1H ( ,0, , ) ( log2 8 811 33 18 81loglog ) 22 8811 32 8log3 1log ) 1H )p(y z )logp(y z ) ( log28 88j kj kjk 1 3 3 1 y z y z ) H ( , , , )8 8 8 8H )i j k2

11、i j kijk1 1 3 3 3 3 1 1( log log log log ) 8 8 8 8 8 8 8 811 38 83（2）H ) 2 ( loglog ) 88_H(X,Y)H(X)HY X)HY)H(X Y)H(X Y) H(X,Y)HY)；HY X)H(X,Y)H(X)H(X Y)1.811HY X)1.8110.81bitH(X Z)H(X,Z)H(Z)H(Z X)H(X,Z)H(X)HY Z)HY,Z)H(Z)H(Z Y)HY,Z)HY)1.410.540.87bit1.4110.41bit1.410.540.87bit1.4110.41bitH(X Y,Z)H(X,Y

12、,Z)HY,Z)1.811.41bitHY X,Z)H(X,Y,Z)H(X,Z)1.81H(Z X,Y)H(X,Y,Z)H(X,Y)1.81I(X;Y)H(X)H(X Y)1I(X;Z)H(X)H(X Z)1IY;Z)HY)HY Z)1I(X;Y,Z)I(X;Z)I(X;Y Z)則I(X;Y Z) I(X;Y,Z)I(X;Z) H(X)H(X Y,Z)H(X)H(X Z)H(X Z)H(X Y,Z) 0.870.4bitIY;Z X)HY X)HY X,Z)0.81I(X;Z Y)H(X Y)H(X Y,Z)0.81 _103107 ( )0.3logH X0.7log0.8813220.70

13、.30.7黑白0.3 時1H (X) H(X | X ) p(x,y)log21ij2( , )p x yijij1110.91430.7log0.08570.7log0.20.3log20.914320.085720.210.80.3log20.81 X ( )p x exx,2_1H (X) p (x)log p (x)dx p (x)log e xdxcxxx211212 p (x)log dx p x x( )( )logedxlogloge ( )ex x dxx2x12121e0log log ( )e x dxlog ( )ex x dxx201212112elog 2logex

14、e dxlog log (1 ) e x e elog log log2xx22002E(X)0,D(X) 212 1e22 e2 eeH(X) 2 e H(X),222 X和Y11Np(x,y) x2 )2xyy22NS2 ( )H X ,H Y ,HY X) I(X;Y)求( ),ccc11( )( , )NP xX p x y dy ex2 )2xyy2dy2NS2 SN111111Ne (xy)2 x2dy ex2e(xy)2dy2NS2S2NS NSN11x2e2S2 S11NSP (y) p ,ydxY(xex2)2xyy2dx2N2 SNS NSS(xy)2y2edx2SN(SN

15、)2(S N) SN1S NSN1S NSe y2ey) dx(x2SN)2SN(S N)11e y2SN)(S N)_11( )Xex2P xX 0，2S2 S11y2( )eY2(SN)P yY2 (S N)1111H (X) P (x)log P (x)dxex2 loge dxx22S2SSScXX1111 2E (loge) log2Sloge( )E x2x2SS22Sxx1S 1 log2 S logelog2eS22S 2112(SN)12(SN)1ey2logey2dyH Y) P (y)log P Y)dy (S N)(S N)cYY1111y2E (logee) log2

16、 (S N)logeE (y )2(SN)2(S N)22(S N)yY1S N1 log2 S N)log(log2 ( )e S N22(S N) 2H (X,Y) P(x,y)log P(X,Y)dxdyc1111NNe x2 )2xyy2log x2 )2xyy2dxdye2NS2NS SN SN11NE ex2 )2xyy2)2NS2 1Nlog2 SN logeE (x (1 )2xy y )222N2NSlog2 SN logelog2 e SN2N121H Y X)H (X,Y)H (X) log2 e SN log2eS log2eN2ccc11212SI(X;Y)H Y)H

17、 Y X) log2e(S N )log2eN )2Ncc _ m 個 0 和 個 11 33 ( ) log log 0.50.310.81bitH X44 441 3 ( )log( ) ( 100m 200(100 )log3 1.59 41I X m mm4 4 (H X bit)100 ( )81H X100I(X)H(X)2m/100)log3100 X ， X X 121( )X D，E，F(xiàn)，G p x 為2i111p() , p(B) , pC) p(x x )2362j iH(X ,X ).12(X ,X )H(X )H(X X )H1212111 11 11H(X ) lo

18、g log log 0.50.5280.4311.459bit22 33 661H(X X )211 111 111 331 11 1 114 log 2 log 2 log 3 log log 2 443 553 10 106 66 6 2210.30940.34720.21530.0833bitH(X ,X )1.4591.955 3.414bit/序列12 , ,X , X X,各X A a a a 1,2rr 3p p 4 123r_j123i12300(X ,X ,X ) 求123 ,H ,H 求H012H(X ,X ,X ) H(X )H(X | X )H(X | X X )12312131 H(X )H(X | X )H(X | X )1213211 11 11H(X ) log log log bit/122 44 44X 12(x ) p(x x )123p122 ji 2 jiji1230X的2012311113 113 1H(X | X ) log4 log4 log4 log log3 log log32148862 1262 12X X 231232i 3j1_230077153 553 5H(X | X ) log2 log4 log4 log lo