機器視覺三維測量技術

1、Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China合肥工業(yè)大學 儀器科學與光電工程學院 光機電工程研究所盧盧 榮榮 勝勝Tel: 2905872 Email: 機器視覺三維測量技術機器視覺三維測量技術3D Machine Vision Measurement TechniquesInstitute of Opto-mechatronics Engineering, Hefei University of Tech

2、nology, Hefei, Anhui, 230009, China大 綱1. 機器視覺的概念2. 攝像機模型3. 三維視覺傳感器的類型4. 多視覺傳感器測量系統(tǒng)的組成5. 攝像機與視覺傳感器的標定6. 測量系統(tǒng)的全局標定7. 應用舉例Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China機器視覺技術的概念機器視覺技術的概念測量測量檢測檢測識別識別 引導引導光源傳感器光學鏡組傳感單元信號處理單元控制單元 采用光學成像技術（通常使用攝像機）獲取

3、被測目標的圖像，再經過快速圖像處理與圖形識別算法，從攝取圖像中獲取目標的尺寸、位置、方向、光譜特征、結構以及缺陷等信息，從而可以執(zhí)行產品檢驗、分類與分組，裝配線上的機械手運動引導、零部件的識別與定位，生產過程中質量監(jiān)控與過程控制反饋等等任務。Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China應用領域舉例應用領域舉例 1. IC及一般電子業(yè)：PCB、BGA、LCD屏幕、被動組件形狀腳位及定位、生產插件、晶元(Wafer)鏡面研磨、生產組裝、被

4、動組件辨識2. MEMS 組件：微小器件的尺寸、形貌、表面缺陷的檢測，組件動態(tài)特性的測量等。3. 機械工具/自動化機械：零件尺寸、外形、瑕疵檢測、零件分類比對、裝配定位、加工定位、熔焊檢測4. 電機工業(yè)：控制器紅外線熱像儀檢測、電線瑕疵、裂縫檢測、纜線配置檢測5. 金屬鋼鐵業(yè)：鋼板尺寸檢測、表面瑕疵檢測、鑄件瑕疵檢測、材料金像檢測6. 橡膠/塑料制品：保特瓶口尺寸檢測、制品顏色分類檢測7. 食品加工/包裝業(yè)：瓶內液位高度、異物或灰塵檢測、包裝印刷辨識、打印字形及零件編號檢測與識別8. 紡織皮革工業(yè)：表面針織紋路檢測、色差檢測、皮革表面特性檢測9. 汽車工業(yè)：陶軸裁切定位、零件涂黃油檢測、白車身

5、檢驗10. 家電/辦公：產品外殼印刷檢測、1維/2維條形碼辨識11. 保全/監(jiān)視系統(tǒng)：人像特征辨識、指紋辨識等Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China機器視覺在精密測量中的應用照明光源 圖像傳感器成像內部三維表面表面輪廓表面輪廓粗糙度粗糙度缺陷缺陷紋理紋理內部結構內部結構應力應力變形變形缺陷缺陷被測場景方位方位三維形貌三維形貌尺寸尺寸坐標坐標Institute of Opto-mechatronics Engineering, H

6、efei University of Technology, Hefei, Anhui, 230009, China二、攝像機的建模1.影射幾何與坐標變換2.攝像機建模3.鏡頭畸變4.攝像機內部參數的修正5.攝像機的精確模型6.攝像機的簡化模型Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China歐式幾何與影射幾何的概念 xyzxyz不變量不變量: Distance and angle不變量不變量: Cross ratiobacdbdadbca

7、cd , c;b ,aRInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China齊次矩陣fzYyXx101000000001zyxffYXzyxffzYX00001zyfYzxfXInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China坐標變換xyz110001333231232221131211w

8、wwzyxzyxtrrrtrrrtrrrzyxwxwywzzyxwwwtttzyxrrrrrrrrrzyx333231232221131211Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China眼睛的光學模型眼睛的光學模型像平面物體透鏡fImage planeobjectlensInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei,

9、Anhui, 230009, China攝像機的光學投影變換攝像機的光學投影變換Image planeobjectlensfzYyXxxyXYzImage planeFocus planefz , y, xInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China攝像機的光學投影變換攝像機的光學投影變換 （續(xù)）（續(xù)）fzYyXxfzYyXxxyXYzImage planefY,Xz,y,xxXYzImage planeFocus planefz ,

10、y, xy101000000001zyxffYXInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China像平面與幀存體的對應關系110000)(10101YXvdudsvuyxxxyXYzImage planefY,Xz,y,xuvO00,vuv ,uComputer framexdyd)( )(00uudYuudsXyxxInstitute of Opto-mechatronics Engineering, Hefei University o

11、f Technology, Hefei, Anhui, 230009, China幀存體與攝像機坐標的關系1010000000010000)(10101zyxffvdudsvuyxxxyXYzImage planefY,Xz,y,xuvO00,vuv ,uComputer frameInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China2.4 攝像機建模xyXYzImage planefY,Xz,y,xuvO00,vuv ,uComputer

12、framewywxwz110001333231232221131211wwwzyxzyxtrrrtrrrtrrrzyx11000010000000010000)(13332312322211312110101wwwzyxyxxzyxtrrrtrrrtrrrffvdudsvuInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China攝像機建模（續(xù)）110000)(13332312322211312110101wwwzyxyxxzyxtrrrtrrrt

13、rrrvfdufdsvuMtRAm| 未知參數：00,vusddfxyx(1) Intrinsic parameters:(2) Extrinsic parameters333231232221131211r ,r ,r ,r ,r ,r ,r ,r ,rOrientation: Translation:zyxt ,t ,t69, independent 33Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China鏡頭畸變的概念鏡頭畸變的概念xy

14、XYzImage planefddY,Xz,y,xRadial distortionTangential distortionDecentering distortionThin prism distortionInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China徑向畸變5231rkrkr22ddYXrsinrYcosrXdd 22222212222221ddddddyrddddddxrYXYkYXYkYXXkYXXkInstitute of

15、 Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China偏心畸變0012011cossinjpjp偏心畸變既含有徑向畸變又有切向畸變04221042213cosrjrjsinrjrjtdrdtdrdydxdcossinsincos最大切向畸變與 x 軸的夾角, 設在直角坐標系下22212221223ddddydddddxdYXpYXpYXpYXpInstitute of Opto-mechatronics Engineering, Hefei University of

16、 Technology, Hefei, Anhui, 230009, China薄棱鏡畸變1422114221cosririsinriritprp11211cosissinis薄棱鏡畸變是由鏡頭與攝像機面有小的傾角造成的1最大切向畸變與 x 軸的夾角, 設221221ddypddxpYXsYXsInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China鏡頭總畸變鏡頭總畸變 ypydyrdxpxdxrdYYXX,s ,p,p,k ,k ,Y ,XYY

17、,s ,p,p,k ,k ,Y ,XXXddydddxd121211212111ydxdYYXX11)()(2)1 ()(2)3 ()1 (2222221422122122214221dddddddyddddddddxdYXsYXpYXprkrkYYXsYXpYXprkrkX若考慮分項的前兩項化簡Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China攝像機內部參數的修正101000000001)1 ()1 (zyxffYXydxd1010000

18、)1/(0000)1/(1zyxffYXyxdd1001/001/)(A0101vfdufdsyyxxx,1212100sppkkvusddfxyx Intrinsic parameters7Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China攝像機的精確模型計算機屏幕uvO00,vuv ,uwywxwzm11001/001/)(13332312322211312110101wwwzyxyyxxxzyxtrrrtrrrtrrrvfdufdsv

19、uxyXYzImage planefddY,Xz,y,xInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China攝像機的精確模型 （續(xù)）)1 ()1 ()()(98765498732100ydzwwwywwwxdzwwwxwwwydxxdYtzryrxrtzryrxrfXtzryrxrtzryrxrfvvdYuudsX計算機屏幕uvO00,vuv ,uwywxwzmdydddxXprkXYXprk12121212/3一般情況下，取徑向與切向畸變的

20、的第一項就可以達到滿意的效果xyXYzImage planefddY,Xz,y,xInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China6 攝像機的簡化模型1000A00vfusfvuMtRAm| 1vum1wwwzyxM333231232221131211rrrrrrrrrRzyxttttxyXYz像平面fY,Xz,y,x計算機屏幕uvO00,vuv ,uwywxwzMmInstitute of Opto-mechatronics Engin

21、eering, Hefei University of Technology, Hefei, Anhui, 230009, China三、視覺傳感器的類型1.主動與被動2.單目與雙目3.點、線、面結構光4.編碼結構光Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China三角法測量原理激光器鏡頭CCD 鏡頭xCCD 激光器鏡頭鏡頭Institute of Opto-mechatronics Engineering, Hefei Universit

22、y of Technology, Hefei, Anhui, 230009, China視覺傳感器的坐標系統(tǒng)zwxwywTyTxTzxw0yw0 xsyszsosPointOXYOzxyLaserImage planeOrigin Sensor coordinate system (SCS)Global Coordinate system (GCS)Camera coordinate system (CCS)Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 2300

23、09, China單一攝像機二維傳感器單一攝像機二維傳感器Image planesssz ,y,xPscyyscxxscooY ,XPXYfMeasured plane(CCS)SCSsczzInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China點結構光視覺傳感器syszsxcocxcyXYZpqPQOCCSSCSLaserInstitute of Opto-mechatronics Engineering, Hefei University

24、of Technology, Hefei, Anhui, 230009, China線結構光視覺傳感器Laser Light planepPXYcxcyImage planesxsyszCCSSCSczInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China條紋結構光視覺傳感器條紋結構光視覺傳感器The k-th light stripe planeImage planeProjector patternkxzyxzywFvuCFoknkF3In

25、stitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, ChinaXYcxcyImage planesyszCCSSCScz00110010011001sxProjector條紋編碼三維視覺傳感器Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China彩色編碼視覺傳感器彩色編碼視覺傳感器cXcYcxcyspyyp

26、XpYczspxxprojectorcameraCCSSCSobjectspzzInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China被動雙目視覺傳感器syy11X2Ysxx1lamp2X2Y2x2y1zcamera 2CCS 2(CCS1) SCSobjectcamera 12zInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei,

27、Anhui, 230009, China編碼照明雙目視覺傳感器 syy11X2Ysxx1projector2X2Y2x2yszz1camera 2CCS 2(CCS 1) SCSobjectcamera 12zInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China多視覺傳感器測量系統(tǒng)的組成Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hef

28、ei, Anhui, 230009, ChinaCAN總線網絡控制視覺測量系統(tǒng)總線網絡控制視覺測量系統(tǒng)圖像采集卡圖像采集卡計算機計算機多路視頻切換開關多路視頻切換開關CAN 接口卡接口卡CAN 網絡控制總線網絡控制總線傳感器控制單元傳感器控制單元傳感器傳感器視頻信號數據總線視頻信號數據總線 Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China1394總線并行圖像處理系統(tǒng)總線并行圖像處理系統(tǒng)傳感器控制單元傳感器控制單元傳感器傳感器計算機計算機接

29、口卡接口卡數據與控制總線數據與控制總線 Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China基于工控PC機的多計算機分布式并行圖像處理架構PC控制器運動驅動機構可編程光源可編程光源多PC并行處理器HUB生產與管理系統(tǒng)主機辦公電腦遠端大型資料庫視覺系統(tǒng)主控計算機PC三軸控制器攝像機Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei,

30、Anhui, 230009, China常用總線的比較 1.RS232 小于15m傳輸速率25kb/s2.RS485 最大傳輸速度10Mb/s3.CAN 小于40m傳輸速率1Mb/s4.USB 最高速率480Mb/s，可掛接127個外設5.CameraLink 最高4.8Gbps，最遠可傳輸10米。6.Gigabit ethernet 帶寬可達到1000 Mbps， 最遠可傳輸100米。7. Firewire，最遠可傳輸100米，帶寬3.2GInstitute of Opto-mechatronics Engineering, Hefei University of Technology, H

31、efei, Anhui, 230009, China雙總線架構總線接口PC多路視頻切換器視覺傳感器1視覺傳感器N視覺傳感器2控制總線視頻圖像圖像采集處理平臺數據總線Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China單總線架構單總線架構計算機總線接口視覺傳感器 1總線總線視覺傳感器 2視覺傳感器 N 傳感器控制、圖像采集與并行處理模塊 傳感器控制、圖像采集與并行處理模塊 傳感器控制、圖像采集與并行處理模塊Institute of Opto-m

32、echatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China攝像機內部參數的標定Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China1 攝像機攝標定靶標攝像機攝標定靶標Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, An

33、hui, 230009, China標定靶標的控制點類型標定靶標的控制點類型幾何中心與投影中心的偏移Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China2 直接線性變換法(DLT)1 用一個齊次矩陣建立場景與控制點的對應關系11343332312423222114131211wiwiwiiiizyxaaaaaaaaaaaavuw2 從上式中解出DLT矩陣參數3411aa0La nnwnnwnnwnwnwnwnnnwnnwnnwnwnwnwni

34、iwiiwiiwiwiwiwiiiwiiwiiwiwiwiwiwwwwwwwwwwwwvvzvyvxzyxuuzuyuxzyxvvzvyvxzyxuuzuyuxzyxvvzvyvxzyxuuzuyuxzyxLaaaaaaaaa1000000001100000000110000000010,11111111111111111111333231232221131211aInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China直接線性變換法(DLT)在

35、最小二乘意義下可求出參數 ，為了避免解很小的 ，必須使用合適的正則化方法，一種方法是令 然后用偽逆法求解，但當 ， 這種正則化可能產生奇異性，為了克服這個缺點，可以令3411aa3411aa034a134a1234232231aaa3 矩陣參數分解：RQ 方法FMTBVA11100100100vuV10001011221bbbbB1000000ffF矩陣M和T分別表示靶標世界坐標系相對于攝像機坐標系的旋轉矢量與平移矢量（b1 b2) 為仿射失真系數Institute of Opto-mechatronics Engineering, Hefei University of Technology

36、, Hefei, Anhui, 230009, China3 RAC 兩步法徑向準直約束：1）徑向畸變不影響矢量 的方向2）有效焦距不影響矢量 的方向3）光軸沿Z軸平移不影響矢量 的方向uOPuOPuOP該方法第一步采用最小二乘方法求解超定方程組求出外部參數，第二步求解內部參數 , 91rrxyx, s ,tt1k , f , zInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China 計算：0PPOP0d0yxYXddyxYXddywwwxww

37、wtzryrxrytzryrxrx654321)()(00vvdYYuudXsXyddxdxddndidyyyyxxyxyxyxdnwndnwndnwndndnwndnwndnwndiwidiwidiwididiwidiwidwidwdwdwddwdwdwXXXtrtrtrttstrstrstrsYzYyYxYYzYyYxYzYyYxYYzYyYxYzYyYxYYzYyYx165432111111111111111/, 91rrxyx, s ,tt（1）（2）yddwdwdwxddwdwdwTXrYzrYyrYxtYrYzrYyrYx654321（3）令把（2）代入（1）Institute o

38、f Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China傳感器坐標系與攝像機外部參數的標定Laser Light planepPXYcxcyImage planesxsyszCCSSCSczuizwwwywwwuizwwwxwwwydxxdYtzryrxrtzryrxrfXtzryrxrtzryrxrfuudYuudsX98765498732100)()()1 ()1 ()()(00yduixduiiydixxdiYYXXvvdYuudsX00011198653297

39、6431875421292623282522272421rrrrrrrrrrrrrrrrrrrrrrrrrrrInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China 測量系統(tǒng)的全局標定Institute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, ChinaTarget1 標定的基本原理標定的基本原理TCSTheo

40、dolitesGCSMeasured object SCSVisual sensorInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, ChinaTarget標定的基本原理標定的基本原理TCSLaser TrackerGCSMeasured object SCSVisual sensorInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China經緯儀的標定經緯儀的標定abRulerTCSABSensor 2Sensor 1Sensor 4Sensor 3Sensor 6Sensor 5SCS (WCS)targetlight planeGCSInstitute of Opto-mechatronics Engineering, Hefei University of Technology, Hefei, Anhui, 230009, China直接變換標定法直接變換標定法 sgssggTPRP987