第2章1信息論與編碼

1、2022-4-712022-4-722022-4-732022-4-74信源無記憶信源有記憶信源發(fā)出單個符號的無記憶信源發(fā)出符號序列的無記憶信源發(fā)出符號序列的有記憶信源發(fā)出符號序列的馬爾可夫信源2022-4-752022-4-7612,nXa aa12,nPp ap ap a1212nnaaaXp ap ap aP10,1niiip ap a2022-4-772022-4-780( )()(/)(/)()1ijjiijijp xp yp yxp xyp x y，11111111()1,()1,(/)1,(/)1,()1,()(),()()nmnijijijimmnjiijjjinmijjiji

2、ijp xp yp xyp yxp x yp x yp yp x yp x2022-4-79()() (/)() (/)ijijijijp x yp x p yxp yp xy()( ) ()(/)()(/)( )ijijjijijiXYp x yp x p yp yxp yp xyp x當 與 相互獨立時，11()()(/)(/)()()ijijijjinmijijijp x yp x yp xyp yxp x yp x y，2022-4-710123456111111666666aaaaaaXP610,1iiip ap a2022-4-7111212nnaaaXp ap ap aP10,1

3、niiip ap a2022-4-712( , )( )Xa bXpxP( )0( )1XbXapxpx dx 2022-4-71312,lLXXXXX12,1,2,lnXa aalL001010011 100101 110111111666666XP123,3,0,1lXXXXLX2022-4-7141211121112,nnnnXXa aa aa aXp a ap a ap a aPX ,/ijijiijp a ap ap aap ap a2022-4-7150101XppP0001101100011011XppppP000001010011100101110111000001010011

4、100101110111XppppppppP000102101112202122000102101112202122XpppppppppP2022-4-716LLnnLpppPX212112iiiiLX XX2022-4-7172022-4-7182022-4-719 12112121/LLLLXXXXpXXXpXXXp 1212211221/LLLLLXXXXpXXXXpXXXp 121121/LLp X p XXp XX XX 2022-4-7202022-4-7211211(/)(/)LLLL mLp XX XXp XXX123121312122112132121()() (/) (/)

5、(/)() (/) (/)(/) (/)LLLLLLLLp X X XXp X p XX p XX Xp XX XXXp X p XX p XXp XXp XX2022-4-72212lLX XXXX2022-4-7232022-4-7242022-4-7252022-4-7262022-4-727( ) ( )iiI xf p x 1ixp ( )0if p x0ip x ( )if p x 2022-4-728 iiixpxpxIlog1log2022-4-7292022-4-730mxxnnmlogloglog1021010loglog3.322loglog2xxx2022-4-7312

6、022-4-732o 例：例：英文字母中英文字母中“e” 出現(xiàn)的概率為出現(xiàn)的概率為0.105，“c”出現(xiàn)的概率為出現(xiàn)的概率為0.023，“o”出現(xiàn)的概率為出現(xiàn)的概率為0.001。分別計算它們的自信息量。分別計算它們的自信息量。o 解：解：“e”的自信息量的自信息量 I(e) =log20.105=3.25 bit “c”的自信息量的自信息量 I(c) =log20.023=5.44 bit “o”的自信息量的自信息量 I(o) =log20.001=9.97 bit2022-4-7332022-4-734)(log)(jijiyxpyxI2022-4-735(/)log(/)ijijI xyp

7、 xy 22()log() (/)()(/)log() (/)()(/)ijijiijijijjijI x yp x p yxI xI yxp yp xyI yI xy 2022-4-736 bit322. 08 . 0loglog2121xpxI120.80.2XxxP2222loglog 0.22.322bitI xp x 2022-4-737 2211xIxnpxIxnp 11221212222211logloglog0.722 /iiiH Xnp xI xnp xI xnp xp xp xp xp xp x 比特 次2022-4-738()()( ) ( )( )log( )iiiii

8、iH XE I Xp x I xp xp x 2022-4-739120.990.01XxxP120.50.5YyyP( )0.99log0.99 0.01log0.01 0.08 /H X 比特 符號( )0.5log0.50.5log0.51 /H Y 比特 符號2022-4-74053 1010n5500 6003 10 53 1022156()()log()log 10 =3.322 3 1010 /niiiH Xp xp x 比特 畫面2022-4-741342()log3.322 4 101.3 10 /H XN 比特 千字文123111244xxxXP222111111loglo

9、glog1.5 /224444H X 比特 符號2022-4-74201XPpq()- log- log - log-(1- )log(1- ) ( )H Xpp qqppppH p2022-4-7432022-4-7442,/() (/)()log(/)ijjiijjii ji jH YXp x yI yxp x yp yx (),2,/()(/)() (/) (/) () (/)()log(/)jjjijijji jijijijiji ji jH X Yp yH Xyp yp xyI xyp x yI xyp x yp xy ()=2(/)(/) (/)(/)log(/)jijijijij

10、iiH Xyp xy I xyp xyp xy 2022-4-745 112,0,0/log/ijijijiji ji jH X Yp x yI xyp x yp xy 130,01,1,0,11,088p XYp XYp XYp XY10/ijijijip x yp xyp x y0000010110101111log/log/log/log/p x yp xyp x yp xyp x yp xyp x yp xy 0112p yp y222211333311loglogloglog84848484 0.811 /比特 符號,/()(/)() (/) (/)jjjijijji jH X Yp

11、 yH Xyp yp xyI xy()=2022-4-7462,()() ()()log()ijijijiji ji jH XYp x yI x yp x yp x y /H XYH XH YXH XYH YH X Y2022-4-747/H XYH XH YX,()() ()()log()()log( ) (/)ijijijiji ji jijijii jH XYp x yI x yp x yp x yp x yp x p yx ,()log( )()log(/)( )log( )()log(/)()( /)ijiijjii ji jiiijjiii jp x yp xp x yp yxp

12、xp xp x yp yxH XH YX 2022-4-748p(y1)p(x4)111111117loglogloglog224488884H X 114log244H Y 1111111127log2log6log4log4488161632328H XY 2022-4-7492022-4-750000002001021022( /)( /)log( /)( /)log( /)( /)log( /)3311loglog0.82/4444iiiH V up v up v up v up v up v up v ubit 符號2022-4-75111002222(/)()log(/)33111

13、111loglogloglog848444440.91 /ijjiijH V Up u vp vu 比特 符號1100()()log()1.91 /ijijijH UVp u vp u v 比特 符號 1111loglog1 /2222/1.91 /H UH UVH UH V U 比特 符號比特 符號2022-4-7523 5( )(,)0.96/8 8(/)()( )1.91 0.960.95/H VHbitH U VH UVH Vbit符號符號1100(/)()log(/)0.95/ijijijH U Vp u vp uvbit 符號2022-4-7532022-4-7542211()lo

14、glog0.92/3333H Xbit 符號)/()()/()()(jijijijiyxpypxypxpyxp( / )()log(/ )13111111loglogloglog0.88/24646262ijjiijH Y Xp x yp y xbit 符號2022-4-755(/ )()log (/)0.33/ijijijH X Yp x yp xybit符號10100()(/)0()p x yp xyp y00000()1/ 2(/)1()1/ 2p x yp xyp y1()()nijjip x yp y111111( )logloglog1.47/223366H Ybit 符號2022

15、-4-7562022-4-757有擾信道干擾源信源X信宿Y)()()(2121nnxpxpxpxxxPX)()()(2121nnypypypyyyPY2022-4-7582022-4-7592(/)( ;)log( )(/)( )ijijiijip xyI x yI xI xyp x2022-4-7601/ 21/ 41/81/8XP晴陰雨雪2022-4-7612022-4-762421/2;01log2 1/8I xbit421/4;0log1 1/8I xbit421;011log3 1/8I xbit2022-4-763( /)( )( ;) loglog0( )( )ijiijiip

16、x yp xI x yp xp x( ;)(;)ijjiI x yI yx(/)(/)loglog()( )()( )( )()jijjiijijip xyp yxp xp yp yp xp yp x2022-4-764(/)(/) ( ;)(/)log(;)jijijijijiiip xyp xyI x yI X yp xyp x,(/)() (;)() (/)log( )(/)()log( )(; )ijjjjijji jiijiji jip xyp y I X yp yp xyp xp xyp x yp xI X Y,logl/og/iiijijii jp xp xp x ypHXxYy

17、XH 2022-4-765 ;/;/I X YH XH X YI Y XH YH YX2022-4-7662022-4-767( ;);( ;/)ijiijkkkI x y zI x zI x yz(/)log( )( ;)ijkiijkp xy zp xI x y z(/)log(/)( ;/)ijkikijkp xy zp xzI x yz(/)(/)ikikp xzp xz2022-4-768(; )(;/)(;)I X YI X Z YI X YZ(;)(;/)(;)I X ZI X Y ZI X YZ( ;)( ;/)(;)I Y XI Z X YI YZ X2022-4-769XY

18、Z(; )(;/)(;)I X YI X Z YI X YZ(;)(;/)(;)I X ZI X Y ZI X YZ2022-4-770ZYXIYZXIYXIZXI/;/;( ;)( ;/)(;)I Y XI Z X YI YZ X(;)( ;/)(; )I X ZI Y Z XI XY Z( ;)(;/)(; )I Y ZI X Z YI XY Z;/;/I X ZI Y ZI X Z YI Y Z X;I X ZI X Y;I X ZI Y Z2022-4-771;I U VI X Y;I U VI X VI X VI X Y2022-4-772123111326xxxXP123111362yyyYP123111236zzzZP H XH YH Z2022-4-7731