信息論與編碼-習題解答

1、.2.5 居住某地區(qū)的女孩子有 25%是大學生，在女大學生中有 75%是身高 160 厘米以上的，而女孩子中身高 160 厘米以上的占總數(shù)的一半。假如我們得知“身高 160厘米以上的某女孩是大學生”的消息，問獲得多少信息量？XXxx12P(X)YYyy12P(Y)p(y /x ) 11p(x )p(y /x )0.25I(x /y ) p(x /y )111(y )0.51111p1X x 0 x 1 x 2 x 3 1234 P3/81/41/41/8 18I(x )loglog bit122p(x)31 ( )2 , ( )2 , ( )3I x2 I x3 I x3 ( ) ( ) (

2、)6 ( )II 1 I x2I x3I x487.81451.95X Y2.13 有兩個二元隨機變量 和 ，它們的聯(lián)合概率為Xx =0 x =1Y12.w.1/83/83/81/81y =12并定義另一隨機變量Z=XYH(X) H(Y), H(Z), H(XZ), H(YZ) H(XYZ)；(1)(2),和H(X/Y), H(Y/X), H(X/Z), H(Z/X), H(Y/Z), H(Z/Y), H(X/YZ), 和H(Z/XY)；I(X;Y), I(X;Z), I(Y;Z), I(X;Y/Z), I(Y;Z/X) I(X;Z/Y)。(3)和1 3 1p(x ) p(x y ) p(x

3、y ) 8 8 211 1123 1 1p(x ) p(x y ) p(x y ) 8 8 222122H(X) p(x )logp(x )1 bit/symboliii1 3 1p(y ) p(x y ) p(x y ) 8 8 211 12 13 1 1p(y ) p(x y ) p(x y ) 8 8 221222HY) p(y )logp(y )1 bit/symboljjjZ = XY z 0 z 1Z1217P(Z)8 877 1182H(Z) p z ( ) log log 0.544bit symbol/88 8kk.w.p(x ) p(x z ) p(x z )11 11 2

4、p(x z )01 2p(x z ) p(x )0.51 11p(z ) p(x z ) p(x z )11 12 1738p(x z ) p(z ) p(x z ) 0.582 111 1p(z ) p(x z ) p(x z )21 22 218p(x z ) p(z )22211 33 118H(XZ) p x z(p x z)log () log log log 1.406 /bit symbol22 88 8iki kikp(y ) p(y z ) p(y z )11 11 2p(y z )012p(y z ) p(y )1 11p(z ) p(y z ) p(y z )11 12

5、1738p(y z ) p(z ) p(y z ) 82 111 1p(z ) p(y z ) p(y z )2122 218p(y z ) p(z )22211 33 18 818HYZ) p y z(p y z)log () log log log 1.406bit symbol/22 8jkjkjk.w.p(x y z )01 1 2p(x y z )012 2p(x y z )021 2p(x y z ) p(x y z ) p(x y )1 1 11 1 21 1p(x y z ) p(x y )1/81 1 11 1p(x y z ) p(x y z ) p(x z )12 11

6、1 11 11 1 3p(x y z ) p(x z ) p(x y z ) 2 8 812 11 11 1 1p(x y z ) p(x y z ) p(x y )21 121 22 138p(x y z ) p(x y )21 12 1p(x y z )022 1p(x y z ) p(x y z ) p(x y )22 12222218p(x y z ) p(x y )22222H(XYZ)(p x y z)log ( )p x y zijk2ij kijk11 33 33 118 log log log log 1.811 bit/symbol88 88 88 811 33 38 83

7、 18 818H(XY) p x y(p x y)log () log log log log 1.811 /bit symbol88 8ij2ijijH(X /Y) H(XY)HY)1.81110.811 bit/symbolHY/X) H(XY)H(X)1.81110.811 bit/symbolH(X /Z) H(XZ)H(Z)1.4060.5440.862 bit/symbolH(Z/X) H(XZ)H(X)1.40610.406 bit/symbolHY/Z) HYZ)H(Z)1.4060.5440.862 bit/symbolH(Z/Y) HYZ)HY)1.40610.406 bi

8、t/symbolH(X /YZ) H(XYZ)HYZ)1.8111.4060.405 bit/symbolHY/XZ) H(XYZ)H(XZ)1.8111.4060.405 bit/symbolH(Z/XY) H(XYZ)H(XY)1.8111.8110 bit/symbolI(X;Y) H(X)H(X /Y)10.8110.189 bit/symbolI(X;Z) H(X)H(X /Z)10.8620.138 bit/symbolIY;Z) HY)HY/Z)10.8620.138 bit/symbolI(X;Y/Z) H(X /Z)H(X /YZ)0.8620.4050.457 bit/sy

9、mbolIY;Z/X) HY /X)HY/XZ)0.8620.4050.457 bit/symbolI(X;Z/Y) H(X /Y)H(X /YZ)0.8110.4050.406 bit/symbol.w. 322H(X ) NH(X) 310N56H(X) log n log 1000013.288 bit/symbol22NN6N CCCCC1234560001123456C :62 13163123456642636431244135236C 不是唯一可譯碼，而C ：54.，C ，CC136.w.2 13 31 23.1 設二元對稱信道的傳遞矩陣為3 3P(0)= 3/4, P(1)=

10、1/4，求H(X), H(X/Y), H(Y/X) I(X;Y)和 ；(1) 若(2) 求該信道的信道容量及其達到信道容量時的輸入概率分布；33 11H(X) p(x ) ( log log ) 0.811 bit/symbol44 44i22iHY / X) p(x )p(y /x )log p(y /x )ijijiij3 2 2 3 1 1 1 1 1 1 2 2 ( lg lg lg lg )log 104 3 3 4 3 3 4 3 3 4 3 320.918 bit/symbol3 2 1 1p(y ) p(x y ) p(x y ) p(x )p(y /x ) p(x )p(y

11、/x ) 0.58334 3 4 3111211112123 1 1 2p(y ) p(x y ) p(x y ) p(x )p(y /x ) p(x )p(y /x ) 0.41674 3 4 321222121222HY) p(y ) (0.5833log 0.58330.4167log 0.4167) 0.980 bit/symbolj22jI(X;Y) H(X)H(X /Y) HY)HY / X)H(X /Y) H(X)HY)HY / X) 0.8110.9800.918 0.749 bit/symbolI(X;Y) H(X)H(X /Y) 0.8110.7490.062 bit/sy

12、mbol1 1 2 2C maxI(X;Y)log mH log 2( lg lg )log 100.082 bit/symbol3 3 3 32mi2212p(x )i 2i( )， 3xip xai1/ 2 1/ 2 0。j， yP1/2 1/4 1/4i ( | )；H Y X 1/2 1/2 0P 1/ 2 1/ 4 1/ 4( , )Xp x yijYyyy123.w.a/2a)/2 a)/4 a)/4x01x2則YYyyy1231/ 2(1a)/4 a)/411+a4 1a4HY) log2loglog241a41a1116a1a log2 log log214 1a 411a21

13、1a1a log2 log16 log log23414 1a 4 1a21a1a1a log2 log log24 1a 42取23 11a1a1aHY)( log log1a 4)bit2 42221a1 a1 1a1 1a 1 1a HY | X) 22 22224444a)alog23alog22log22取23aHY | X)bit2a111a 4a1a1acI(X;Y) HY)HY | X) 2 242p(x )p(x )p(x )iiia11a 1a( ln2 ln ln)24 1a 4 1a2取ea11 2a1 1a a11 ln2 ln ()241a 4 1a 4 1a 1a

14、21a1 1a a 2 ln2 ln212(1a ) 4 1a 41a1 1a22 ln2 ln24 1a=01a 11a 43a 51 3c 2 2 5111 3 4 514419.w.3125 314 log2 log log1034116 205 3 log2 log log21024 10154 log23.3 在有擾離散信道上傳輸符號 0和 100個符號發(fā)生一個錯誤，已知 P(0)=P(1)=1/2，信源每秒發(fā)出 1000 個符號，求此信道的信道容量。0.99 0.01P 0.01 0.9912C logsH(P)log2 plog 0.92 /bit signpiii11C C C

15、 bit/sectt C W log (1SNR)2110 log 10)623.159C W log (15)3.459Mbps2223.159MW 1.338 622C W )3.4593323.459log SNR)0.52.w.w.X 01 23P(X)Y4.3 一個四元對稱信源 = 0, 1, 2, 3，1/4 1/4 1/4 1/4 0 1 1 11 0 1 11 1 0 1DDR(D)函數(shù)，并畫出其曲線其失真矩陣為 1 1 1 0 ，求和及信源的（取 4至 5maxmin111134D minD min p(x )d(x ,y ) 1 1 1 04444maxjiijji1111

16、D p(x )mind(x ,y ) 0 0 0 004444miniijjni DD D D1 ln1 aR(D)lnn lna n1 a a = = anD R(D)ln4D 1D 1D3R(D)ln4D01/41/23/4DR(0)ln4 /1 1D ,R(D)ln4 ln/412 31D ,R(D)ln4 ln12 /232D ,R(D)0 /4.w.X 01a 0D P(X)D4.2 某二元信源其失真矩陣為求這信源的0 a1/2 1/2 maxDR(D)和和函數(shù)。min11aD minD min p(x )d(x ,y ) a 0222maxjiijji11D p(x )mind(x ,y ) 0 0022miniijjDR(D)nH a n = D DDDR(D)ln2ln 1 ln1 a a a a X 01 23P(X)Y4.3 一個四元對稱信源 = 0, 1, 2, 3，1/4 1/4 1/4 1/4 0 1 1 11 0 1 11 1 0 1DDR(D)函數(shù)，并畫出其曲線其失真矩陣為 1 1 1 0 ，求和及信源的（取 4至 5maxmin111134D minD