西安工業(yè)信號檢測與估計SDE_04 ch3b CRLB

1、2021-7-9SDE_04 (b)1 Ch3 CRLB Chapter 3 Cramer-Rao Lower Bound (B) Signal Detection and Estimation 信號檢測與估值信號檢測與估值 第四講第四講 Spring 2016 lecture 4 雷斌雷斌2021-7-9SDE_04 (b)2 對于所有的 定理定理3.1 標量情況下的標量情況下的CRLB Theorem 3.1 CRLB for Scalar Parameter 假設(shè) PDF p(X; ) 滿足“正則”條件，既 0 );(lnxp E Assume “regular

2、ity” condition is met: 則有 Then 2 2 2 );(ln 1 var xp E 其中 while dxxp xpxp E);( );(ln);(ln 2 2 2 2 CRLB 3.4 Cramer-Rao Lower Bound 2021-7-9SDE_04 (b)3 進一步 Since no unbiased estimator can Since no unbiased estimator can do better. this is the MVU do better. this is the MVU estimate!estimate! This gives

3、 a possible way to find the MVU: Compute ln p(x; )/ (need to anyway) Check to see if it can be put in form like (!) If so. then g(x) is the MVU esimator 2021-7-9SDE_04 (b)4 Some Uses of the CRLB 1. 可行性研究 Feasibility studies ( 如傳感器有效性等 e.g. Sensor usefulness, etc.) Can we meet our specifications? 2.

4、評價估計算法Judgment of proposed estimators Estimators that don.t achieve CRLB are looked down upon in the technical literature 3. 有時用以得到MVU估計算法 Can sometimes provide form for MVU est. 4. 評估某些參數(shù)對估計的影響 Demonstrates importance of physical and/or signal parameters to the estimation problem e.g. We.ll see tha

5、t a signal.s BW determines delay est. accuracy Radars should use wide BW signals 2021-7-9SDE_04 (b)5 CRLB: Ex. 3.2 : PDF Dependence for DC Level in Noise 2 2 2 22 );(ln 1 var xp E CRLB 2 2 2 0 2 1 exp 2 1 );0(AxAxp i i x Ex. 3.1: PDF Dependence for DC Level in Noise Then the parameter-dependent PDF

6、of the data point x0 is: x0 = A + w0 w0 N(0,2) 2 )var(A 2 2 2 0 2 1 2ln);0(lnAxAxp )2 . 3(0 1);0(ln 2 Ax A Axp 22 2 1);0(ln A Axp 0)0(xxgA LLF 2021-7-9SDE_04 (b)6 Example 3.3 CRLB for DC in AWGN xn = A + wn, n=0,1,2,N-1 wn= WGN N(0,2) 單獨觀測的概率密度函數(shù) PDF of an individual data sample: 無關(guān)的高斯隨機變量是統(tǒng)計獨立的 Unc

7、orrelated G- RVs are Independent 聯(lián)合概率密度函數(shù)是各自PDF相乘 Joint PDF= product of the individual PDFs: Ax N Anx A Ap N n 2 1 0 2 1);(ln x xgA)(x N A Ap E CRLB 2 2 2 );(ln 1 x )()( );(ln x x gI p 通過多次觀測 降低估計噪聲 2021-7-9SDE_04 (b)7 Fisher信息信息 Fisher Information 1.1.由（由（3.113.11），），它它是非是非負負的；的； I( ) 0 (easy to se

8、e using the alternate form of CRLB) 2.2.對對于于獨獨立立觀測觀測，具有可加性。，具有可加性。 I( ) is additive for independent observations follows from: 信息越多，估信息越多，估計計的方差越小，估的方差越小，估計計越準越準 If each In ( ) is the same: I( ) = NI( ) 2021-7-9SDE_04 (b)8 3.5 高斯白噪聲中信號的一般高斯白噪聲中信號的一般CRLB CRLB for Signals in AWGN Signal Model: x x n n

9、 = = s s n n; ; + + w w n n, , n n = 0, 1, 2, . , = 0, 1, 2, . , N N-1-1 When we have the case that our data is .signal + AWGN. When we have the case that our data is .signal + AWGN. then we get a simple form for the CRLB:then we get a simple form for the CRLB: wn, White,Gaussian, Zero Mean 似然函數(shù) lik

10、elihood function: Differentiate Log LF twice to get: 2021-7-9SDE_04 (b)9 未知參量與信號的依賴程度:Signal in AWGN AWGN 信號對參數(shù)變化很敏感，則信號對參數(shù)變化很敏感，則CRLB較小，估計很準較小，估計很準 If signal is very sensitive to parameter change. then CRLB is small . can get very accurate estimate! x x n n = = s s n n; ; + + w w n n, , 似然函數(shù) likeli

11、hood function: 未知參量與信號的依賴程度 2021-7-9SDE_04 (b)10 3.6 參數(shù)的變換參數(shù)的變換 Transformation of Parameters 已知 的CRLB，但實際關(guān)心的 ，它是 的函數(shù) Say there is a parameter with known CRLB But imagine that we instead are interested in estimating some other parameter that is a function of : 重要！ 2021-7-9SDE_04 (b)11 Example: Speed

12、of Vehicle From Elapsed Time Test T, But. really want to measure speed V = d/T 2021-7-9SDE_04 (b)12 變化對估計的影響：線性變換變化對估計的影響：線性變換 Effect of Transformation on Efficiency Suppose you have an efficient estimator of But. you are really interested in estimating = g( ) 是否對 的有效估計，通過公式就是對 的有效估計？ Is this an eff

13、icient estimator of ? 2021-7-9SDE_04 (b)13 變化對估計的影響：非線性變換變化對估計的影響：非線性變換 Effect of Transformation on Efficiency 幅度估幅度估計計A A 功率估功率估計計 A A2 2 2 2 2 );(ln )( x E g aVar N A N A AVar 22 2 2 2 4 / 2 )( 2 2 222 )var()()(A N AxxExE 非非線線性性變換變換破壞了估破壞了估計計的有效性的有效性 2021-7-9SDE_04 (b)14 漸近有效漸近有效 Asymptotic Effici

14、ency Under Transformation 若變換為非線性，則結(jié)果不成立 If the mapping = g( ) is not affine仿射的. this result does NOT hold 但若采但若采樣數(shù)樣數(shù)據(jù)很多，據(jù)很多，則則估估計結(jié)計結(jié)果果漸漸近有效近有效 ButBut. if the number of data samples used is large, then the estimator is approximately efficient (.Asymptotically EfficientAsymptotically Efficient.) 2021

15、-7-9SDE_04 (b)15 3.7 擴展到矢量參數(shù)擴展到矢量參數(shù) CRLB for Vector Parameter Case Vector Parameter: = 1 2 . p T 標量參數(shù)我們看其方差 For a scalar parameter we looked at its variance. 矢量參數(shù)我們看其協(xié)方差陣 for a vector parameter we look at its covariance matrix: 估計是無偏的estimate is unbiased: 2021-7-9SDE_04 (b)16 3.7 CRLB 矢量情況 0 );(ln X

16、p E 0 11 a 假設(shè) PDF p(X X; ) 滿足“正則”條件，既 對于所有的 其中數(shù)學期望是對p( X;X; )求出的。那么， 任何無偏估計 的協(xié)方差矩陣滿足 0)( 1 IC ji ij p E );(ln )( 2 X I Fisher信息矩陣 半正定矩陣 positive semi-definite 引申到得到MVU估計 0 21122211 2221 1211 aaaa aa aa 2021-7-9SDE_04 (b)17 半正定半正定 positive semi-definite 是半正定矩陣 is positive semi-definite ij aaa aaa aaa

17、 iji21 i j22221 j11211 0 11 a 條件 0 21122211 2221 1211 aaaa aa aa 2021-7-9SDE_04 (b)18 The CRLB Matrix Then, under the same kind of regularity conditions, the CRLB matrix CRLB matrix is the inverse of the FIM: 2021-7-9SDE_04 (b)19 Fisher Information Matrix For the vector parameter case. Fisher Info b

18、ecomes the Fisher Info Matrix (FIM) Fisher Info Matrix (FIM) I I() whose mnth element is given by: Diagonal elements of Inverse FIM bound the parameter variances, which are the diagonal elements of the parameter covariance matrix 2021-7-9SDE_04 (b)20 Ex.3.6:高斯白噪聲中的DC電平 A 均未知；參數(shù)矢 量 = A T 1 0 2 2 2 )(

19、 2 1 ln 2 2ln 2 );(ln N n Anx NN Xp 4 2 2 0 0 )( N N I N AVar 2 )( N Var 4 2 2 )( 2 2 2021-7-9SDE_04 (b)21 Ex.3.7:直線擬合 1, 1 , 0NnnwBnAnx 斜率B 截距A 參數(shù)矢量 = A B T 似然函數(shù)如下 1 0 2 2 2 2 2 1 exp )2( 1 ),( N n N BnAnxp x 計算推導(dǎo)結(jié)果： p35 ) 1( ) 12(2 )var( 2 NN N A ) 1( 12 )var( 2 2 NN B 觀測對參數(shù)變化的 敏感程度：無噪聲 2021-7-9SD

20、E_04 (b)22 Ex.3.7:直線擬合的引申 BB AA NNNNN NNN nBnAnx BnAnx B p A p p N n N n 22 22 1 0 2 1 0 2 6 ) 12)(1( 2 ) 1( 2 ) 1( )( 1 )( 1 );(ln );(ln );(ln X X X 1 0 1 0 ) 1( 6 ) 1( ) 12(2 N n N n nnx NN nx NN N A 1 0 2 1 0 ) 1( 12 ) 1( 6 N n N n nnx NN nx NN B 1 0 2 2 2 2 2 1 exp )2( 1 ),( N n N BnAnxp x g(x)

21、is the MVU esimator 2021-7-9SDE_04 (b)23 3.8 Vector Transformations 矢量參數(shù)變換的CRLB 將前述標量變換的結(jié)論推廣到矢量，則當)(ga 可以證明： )30. 3 (0 )( )( )( 1 T gg I Ca 其中 p rrr p p g g g g g g g g g g )()()( )()()( )()()( )( 21 2 2 2 1 2 1 2 1 1 1 雅克比 矩陣 Jacobian Matrix 半正定矩陣 positive semi-definite 2021-7-9SDE_04 (b)24 參數(shù)的變換參數(shù)

22、的變換 Transformation of Parameters BAa)(g其中A是rxp階矩陣，B是rx1矢量 BAa aBAa)(E T TT a gg I AAIAACC )( )( )( )( 11 已知 的CRLB，但實際關(guān)心的a ，它是 的函數(shù) 對于非線性變換，有效性只有當N趨近無窮大時保持 因為那時，因為那時， 的估計集中在真值附近的估計集中在真值附近 統(tǒng)計線性統(tǒng)計線性 2021-7-9SDE_04 (b)25 Example 3.8：DCinWGN 信噪比的估計 繼續(xù)P19例3.6 Example 3.8 2 2 )( A g 4 2 2 0 0 )( N N I T A 2

23、 NN A N A A A N N AAgg T 2 4 4 2 2 4 2 2 4 2 4 2 2 1 2424 2 2 0 0 2)( )( )( I N 2 24 )var( 2021-7-9SDE_04 (b)26 Example2 通過方位和距離估計XY坐標 Example2 : Usually can estimate Range (R) and Bearing (?) directly, But might really want emitter (x, y) 在基于雷達和光電經(jīng)緯儀的彈道測試系統(tǒng)中，通過方位和距離估計XY坐標 2021-7-9SDE_04 (b)27 2 21

24、2 2 2 1 2 1 2 1 1 1 /1 )()()( )()()( )()()( )( TdT g g g g g g g g g g p rrr p p Example: Speed of Vehicle From Elapsed Time Td /)(ga 2 2 2 /*/1*TdTVarTdVarVVar V = d/T d=100mm0.5mm T=20ms0.01ms V=5m/s ？ 2 2 1 01. 00 05 . 0 )(I 0 )( )( )( 1 T gg I Ca dVar TVar 2021-7-9SDE_04 (b)28 3.9 CRLB for Gener

25、al Gaussian Case 一般高斯情況的CRLB 推廣到以下的情況推廣到以下的情況 Now generalize to the case where: 數(shù)據(jù)仍然是高斯分布的 Data is still Gaussian, but 均值不一定為0 Parameter-Dependence not restricted to Mean 0 噪聲非白 Noise not restricted to White 協(xié)方差不一定為對角陣 Cov not necessarily diagonal In Sect. 3.5 we saw the CRLB for “signal + AWGN” 確定信

26、號Deterministic Signal w/ 標量確定參數(shù) Scalar Deterministic Parameter One way to get this case: “signal + AGN” Random GaussianSignal w/ Vector Deterministic Parameter Non-WhiteNoise 均值和方差均與待估參量有關(guān) 2021-7-9SDE_04 (b)29 Gen. Gauss. Ex.: Time-Difference-of-Arrival 一般高斯情況舉例：到達時間差異一般高斯情況舉例：到達時間差異 2 2021-7-9SDE_0

27、4 (b)30 Fisher信息 jii T i ij tr )( )( )( )( 2 1)( )( )( )( 111 C C C C C I 方陣 的跡 對角線 元素之和 i N i i i )( )( )( )( 2 1 i NN i N i N i N ii i N ii i )()()( )()()( )()()( )( 21 22221 11211 CCC CCC CCC C 待估參數(shù)與信號（均值）有關(guān) 待估參數(shù)與方差有關(guān) This shows the impact of signal model assumptions 信號模式假設(shè)的影響 deterministic signa

28、l + AGN 確定性信號加平穩(wěn)高斯噪聲 random Gaussian signal + AGN 隨機高斯信號加平穩(wěn)高斯噪聲 2021-7-9SDE_04 (b)31 方陣的跡 定定義義A A( (a aij ij) )n n n n， ，則則A A的跡的跡為為對對角角線線元素之和元素之和 tr (tr (A A) ) = a= a11 11+a +a22 22+a +ann nn 性性質(zhì)質(zhì) tr (A+B) = tr (A) + tr (B) tr (kA) = k tr(A) tr (AB) = tr (BA) tr (P1AP) = tr (A) 2021-7-9SDE_04 (b)3

29、2 協(xié)方差矩陣與 無關(guān)，即下式0 Example3.9 WGN中信號參數(shù)中信號參數(shù) j N n i i T i jii T i ij ns ns tr ,1 )(1)( )( )( )( )( 2 1)( )( )( )( 1 0 2 2 111 I C C C C C I Signal Model: x x n n = = s s n n; ; + + w w n n, , n n = 0, 1, 2, . , = 0, 1, 2, . , N N-1-1 例中，wn獨立同分布， IC 2 與 無關(guān) 與3.5的結(jié)果一致 2021-7-9SDE_04 (b)33 Example3.10 噪聲參

30、數(shù)噪聲參數(shù) 4 2 2 2 2 2 21 1112 2 1 2 1 )( )( 2 1 )( )( )( )( 2 1)( )( )( )( N tr tr tr jii T i I C C C C C C C I 與例3.6的結(jié)果一致 nnx WGNn的是具有待估參量其中 2 IC 22 )( 2021-7-9SDE_04 (b)34 待估參量 Example3.11 WGN中的隨機中的隨機DC電平電平 2 22 2 22 1112 2 1 1111 1 2 1 )( )( )( )( 2 1)( )( )( )( A TT A jii T i A N N N tr tr u u C C C C C I nwAnx 2 A A為高斯隨機變量， 與wn獨立 所以，CRLB是 ijAijA jwAiwAEjxixE 222