張浩-機(jī)器人操控中的計(jì)算機(jī)視覺

機(jī)器人操控中的計(jì)算機(jī)視覺ComputerVisioninRobotManipulationDorabotInc.藍(lán)胖子機(jī)器人CTO張浩TableofContentsWhat’sthedifference?Typicaltasks:ObjectposeestimationToyproblems:RobotcompetitionReal-worldproblems加入“知識(shí)星球行業(yè)與管理資源”庫(kù)，每日免費(fèi)獲取報(bào)告1、每月分享1000+份最新行業(yè)報(bào)告（涵蓋科技、金融、教育、互聯(lián)網(wǎng)、房地產(chǎn)、生物制藥、醫(yī)療健康等最新行業(yè)）2、每月分享500+企業(yè)咨詢管理文件（涵蓋國(guó)內(nèi)外著名咨詢公司相關(guān)管理方案，企業(yè)運(yùn)營(yíng)制度等）3、每月分享500+科技類論文或者大師課件、筆記。4、與高端人士交流企業(yè)管理思想與方法。加入微信群，每日獲取免費(fèi)3+份報(bào)告1、掃一掃二維碼；或者添加微信（微信號(hào)：Teamkon）2、加入報(bào)告分享群，備注：姓名+單位+業(yè)務(wù)領(lǐng)域

微信掃碼加入“知識(shí)星球行業(yè)與管理資源”，獲取更多行業(yè)報(bào)告、管理文案、大師筆記微信掃碼二維碼，免費(fèi)報(bào)告輕松領(lǐng)What’sthedifference?2DV.S.3D2DV.S.3DTaskFocusGeneralmethodv.s.TailoredmethodComputerVisiontriestosolveproblemsinitsmostgeneralform.DifferentrobotmanipulationtaskshavedifferentassumptionsthatsuitfordifferentcombinationsofCVmethods,mostprobablywithsomesmallinnovationshereandthere.Atailoredsolutionisusuallyrequired.Howdoesrobotinteractwiththephysicalworld?PerceptionPlanningSceneDecisionMotionPlanningforallDOFMotionExecutionforallDOFPlanningSceneTypicalVisionTasksinRobotManipulationEnvironmentreconstruction:buildplanningsceneSelf-localization:localizerobotinsuchsceneObjectposeestimation:supplyinformationinsceneVisualservoing:copewitherrorinvision&tasksrequirereal-timereactionRGBD-SLAMMappingandlocalizingdonothavetohappensimultaneously,buttheyhavetobe3D.ObjectPoseEstimationGoal:Full6DobjectposeApproach1:Align3DCADmodelto3Dpointcloud.ICP(iterativeclosestpoint)isfrequentlyused.Approach2:2Ddescriptors(SIFT,SURFetc.),3Ddescriptors(FPFH),orsimpler2Dor3Dfeatures(colorgradient,edge,normal)basedcorrespondences.MOPED,LINEMODroughlyfallsintothiscategory.ICPMOPEDLINEMODLINEMODLINEMODLINEMODNoneofthemethodsareuniversal!GoodToyProblems:RobotCompetitionsAmazonPickingChallenge:Fullyautonomous,focusonpick&placeDARPARoboticChallenge:RemotecontrolwithunstablenetworkconnectionRobocup@Home:Fullyautonomous,withvariouskindsoftasksincludingmanipulationRobocup@Work:Fullyautonomous,focusonmobilemanipulationofsimpleshapedworkpiecesRoboticGraspingandManipulationCompetition:Differenttrackswithdifferentautomationlevel,focusingonmanipulationcapabilityofrobotichandAmazonPickingChallengeAmazonPickingChallengeAmazonPickingChallengeClutteredenvironment:multipleobjectsinnarrowspacesOcclusion:LimitedcamerapositionMissingdata:reflective/transparent/meshedsurfacesSmallobjects:FewdatapointsDeformableobjects:modelalignmentdoesn’tworkSpeed:TimelimitforoveralltaskUncontrolledlightingProbabilisticMulti-ClassSegmentationProbabilisticMulti-ClassSegmentationJonschkowski,R.,Eppner,C.,H?fer,S.,Martín-Martín,R.,&Brock,O.(2016).ProbabilisticMulti-ClassSegmentationfortheAmazonPickingChallenge.InIEEE/RSJInternationalConferenceonIntelligentRobotsandSystems.ProbabilisticMulti-ClassSegmentationMulti-viewSelf-supervisedDeepLearningfor6DPoseEstimationMulti-viewSelf-supervisedDeepLearningfor6DPoseEstimationZeng,A.,Yu,K.-T.,Song,S.,Suo,D.,Walker,E.,Rodriguez,A.,&Xiao,J.(2016).Multi-viewSelf-supervisedDeepLearningfor6DPoseEstimationintheAmazonPickingChallenge.Retrievedfrom/abs/1609.09475Multi-viewSelf-supervisedDeepLearningfor6DPoseEstimationMulti-viewSelf-supervisedDeepLearningfor6DPoseEstimationDoraPickerDoraPickerZhang,H.,Long,P.,Zhou,D.,Qian,Z.,Wang,Z.,Wan,W.,…Pan,J.(2016).DoraPicker:AnAutonomousPickingSystemforGeneralObjects,10./10.1109/COASE.2016.7743473DoraPicker(LINEMOD+ICP)RoboticGraspingandManipulationCompetitionBookinpreparation:RoboticGraspingandManipulationCompetition,Springer,2017.DataSets/bigbird//rutgers_apc_rgbd_dataset/~andyz/apc2016Enoughtoyproblem,howaboutreal-worldproblem?DorabotMobileManipulator:WarehousedemoLookingforchallenges?Joinustoworkonreal-worldproblems!dream@關(guān)注藍(lán)胖子公眾賬號(hào)加入“知識(shí)星球行業(yè)與管理資源”庫(kù)，每日免費(fèi)獲取報(bào)告1、每月分享1000+份最新行業(yè)報(bào)告（涵蓋科技、金融、教育、互聯(lián)網(wǎng)、房地產(chǎn)、生物制藥、醫(yī)療健康等最新行業(yè)）2、每月分享500+企業(yè)咨詢管理文件（涵蓋國(guó)內(nèi)外著名咨詢公司相關(guān)管理方案，企業(yè)