Intel RealSense D400 Series User Manual Download

Page: 1 / 49

Intel

RealSense™ Depth Module

D400 Series Custom Calibration

Revision 1.1.0

January 2018

Summary of Contents for RealSense D400 Series

Page 1: ...Intel RealSense Depth Module D400 Series Custom Calibration Revision 1 1 0 January 2018 ...

Page 2: ...s and product descriptions at any time without notice Designers must not rely on the absence or characteristics of any features or instructions marked reserved or undefined Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them The information here is subject to change without notice Do not final...

Page 3: ...Calibration Sample Application 21 4 1 Process Overview 21 4 2 Connect Device to Computer 21 4 3 Running Custom Calibration Sample Application 21 4 3 1 Starting Application 21 4 3 2 Capturing Images from 6 Viewpoints 22 4 4 Calibration Result 35 4 5 Updating Results to Device 37 4 5 1 Depth Quality Check before Updating Calibration 38 4 5 2 Writing Optimized Calibration to Device 38 4 5 3 Depth Qua...

Page 4: ...Calculating Depth Camera Calibration with OpenCV 46 5 5 Calculating RGB Camera Calibration with OpenCV 47 5 6 Calculating RGB Camera Calibration Extrinsics with OpenCV 47 5 7 Writing Calibration Parameters 48 ...

Page 5: ...mats Used in Custom Calibration 11 Table 3 1 Intel RealSense Calibration API Resources 18 Table 3 2 Intel RealSense SDK Resources 19 Table 3 3 OpenCV 3 3 0 Resources 20 Table 3 4 OpenCV 3 3 0 Libraries Required for the Example 20 ...

Page 6: ...int 1 26 Figure 4 3 Center Right 27 Figure 4 4 Viewpoint 2 28 Figure 4 5 Left 29 Figure 4 6 Viewpoint 3 29 Figure 4 7 Right 30 Figure 4 8 Viewpoint 4 31 Figure 4 9 Top Looking Down 32 Figure 4 10 Viewpoint 5 33 Figure 4 11 Bottom Looking Up 34 Figure 4 12 Viewpoint 6 34 Figure 4 13 Depth Quality before Updating Calibration 38 Figure 4 14 Updating Calibration Parameters to Device 39 Figure 4 15 Cal...

Page 7: ...7 Revision History Document Number Revision Number Description Revision Date XXXXXX 1 0 Initial Release 01 2018 1 1 Update with Calibration API 2 5 2 0 release 01 2018 ...

Page 8: ...In addition a simple sample application based on OpenCV algorithm is provided as an example It is not in the scope of this document to discuss details of calibration algorithm or accuracy Developing custom calibration requires knowledge in computer vision as well as good understanding of the RealSense device operating details It is a complex topic and intended only for those with expert level know...

Page 9: ...ters to device ReadCalibrationParameters read parameters from device Figure 2 1 Software Stack with Dynamic Calibration API and Calibration Apps An example tool CustomRW is also included in Calibration API to read calibration parameters from XML file and write them to the device A user custom calibration app can choose one of the two approaches to update the results to the device To link to the Wr...

Page 10: ... model k1 k2 p1 p2 k3 for left right and RGB cameras Extrinsic includes RotationLeftRight rotation from right camera coordinate system to left camera coordinate system specified as a 3x3 rotation matrix TranslationLeftRight translation from right camera coordinate system to left camera coordinate system specified as a 3x1 vector in millimeters RotationLeftRGB rotation from RGB camera coordinate sy...

Page 11: ...el RealSense Camera D430 D420 D435 D430 YUY2 D415 RGB Sensor 1920x1080 15 FPS Intel RealSense Camera D415 D435 D435 2 4 Frame Sync The calibration frames are not synced To avoid possible motion blur calibrate the device with images captured at static positions In this sample a tripod is used to keep the camera device at fixed positions while capturing the images for calibration This is same if you...

Page 12: ...Overview 12 also not be accurate if the camera or board is handheld or not completely still as there is no framesync in calibration mode ...

Page 13: ...ice and developing a Custom Calibration Application 3 1 Hardware The hardware required including a calibration target D400 series device to be calibrated USB cable and a Windows 10 computer To ensure the images are synced pictures are captured at static device positions a tripod is used to support the device during calibration Figure 3 1 Hardware Setup ...

Page 14: ...tion example is an 8x7 checkerboard with a 60x60mm checker size The target image pdf 540x480_60mm pdf is included in the software package described in 3 2 1 Custom Calibration Sample Application The target pdf is located under the target directory Users must ensure that when printing the target the target is not scaled A developer may choose to use a different target but will need to modify the sa...

Page 15: ...m Checker Calibration Target 3 1 3 Tripod Any medium sized tripod should be sufficient In this example setup we used a Manfrotto Compact Light Aluminum Tripod with ball head The ball head makes adjustment to device orientation easier ...

Page 16: ...Setup 16 Figure 3 4 Tripod ...

Page 17: ...ion Sample Application It includes the binary executable and a calibration target file The source code is included in Intel RealSense Calibration API package Unzip the package CustomCalibrationSample Bin CustomCalibrationSampleApp exe Target 540x480_60mm pdf CustomResult Bin folder contains the executable CustomCalibrationSampleApp exe is the executable for the Custom Calibration Sample Applicatio...

Page 18: ...tation Table 3 1 Intel RealSense Calibration API Resources Resource URL Intel RealSense Depth Module D400 Series Dynamic Calibration Software Download https downloadcenter intel com download 27415 v t Intel RealSense Depth Module D400 Series Dynamic Calibration User Guide https cdrd intel com v1 dl getContent 574999 htm Intel RealSense Depth Module D400 Series Dynamic Calibration Programmer Guide ...

Page 19: ...libration application can use LibRealSense to conveniently open and access Intel RealSense D400 series modules Table 3 2 contains pointers to the SDK homepage GitHub repository where you can download the latest release and the SDK documentation Table 3 2 Intel RealSense SDK Resources Resource URL Intel RealSense SDK Home Page https software intel com en us realsense sdk LibRealSense GitHub https g...

Page 20: ...ample OpenCV 3 3 0 Library ittnotify lib opencv_calib3d330 lib opencv_core330 lib opencv_features2d330 lib opencv_flann330 lib opencv_imgproc330 lib zlib lib 3 2 5 Glut Library The FreeGLUT package is a convenient open source alternative to the OpenGL Utility Toolkit for displaying graphical output to the operator of the custom calibration application You can download and install the FreeGLUT pack...

Page 21: ... parameters based on the captured images and writes the results to a XML file The user then run CustomRW calibration data R W tool to read the parameters from the XML file and write them to the device 4 2 Connect Device to Computer Connect the device through the USC cable to the PC where Custom Calibration Sample App locates 4 3 Running Custom Calibration Sample Application 4 3 1 Starting Applicat...

Page 22: ...m right corner it displays a progress counter in the form of x 6 where x is the number of images captured and accepted out of the total 6 images required In the middle of the Window is a green help message to instruct the user to position the camera and press the enter key to capture image In the top middle portion of the Window is an area where error message will display in red when the image is ...

Page 23: ...lts A general rule is that the target in these 6 images combined should cover as much as possible of the entire field of view and from different angles and distances from the target Cover whole target images from any of the viewpoints should cover the entire target in all imagers Left Right and RGB Good example cover the entire target in all three imagers ...

Page 24: ...mple Application 24 Bad example cover only portion of the target in any of the three imagers Now let s capture the images 4 3 2 1 Viewpoint 1 Center Right Adjust the tripod and position the device on the center right facing directly to the target ...

Page 25: ...Calibrating Device with Custom Calibration Sample Application 25 Figure 4 1 Center Right Position The viewpoint should looks like below with the target cover much of the right side the FOV ...

Page 26: ... frame counter on the lower right corner will increase to 1 6 meaning 1 out of 6 images are captured If it failed a red error message will appear and user is directed to retake a viewpoint 4 3 2 2 Viewpoint 2 Center Left Move the device to the center left directly facing the target so the target covers much of the left portion of the FOV ...

Page 27: ...Calibrating Device with Custom Calibration Sample Application 27 Figure 4 3 Center Right ...

Page 28: ...sful the frame counter on the lower right corner will increase to 2 6 meaning 2 out of 6 images are captured If it failed a red error message will appear and user is directed to retake a viewpoint 4 3 2 3 Viewpoint 3 Left Move the device to the left of the camera facing the device at an angle Use the viewpoint below for reference ...

Page 29: ...Calibrating Device with Custom Calibration Sample Application 29 Figure 4 5 Left Figure 4 6 Viewpoint 3 ...

Page 30: ... counter on the lower right corner will increase to 3 6 meaning 3 out of 6 images are captured If it failed a red error message will appear and user is directed to retake a viewpoint 4 3 2 4 Viewpoint 4 Right Move the device to the right facing the target at an angle Use the viewpoint below for reference Figure 4 7 Right ...

Page 31: ...rame counter on the lower right corner will increase to 4 6 meaning 4 out of 6 images are captured If it failed a red error message will appear and user is directed to retake a viewpoint 4 3 2 5 Viewpoint 5 Top Looking Down Adjust the tripod so the device high enough to look down the target at an angle Use the viewpoint below for reference ...

Page 32: ...Calibrating Device with Custom Calibration Sample Application 32 Figure 4 9 Top Looking Down ...

Page 33: ...unter on the lower right corner will increase to 5 6 meaning 5 out of 6 images are captured If it failed a red error message will appear and user is directed to retake a viewpoint 4 3 2 6 Viewpoint 6 Bottom Looking Up Folding the tripod so that the device will be in a very low position looking up to the target Use the viewpoint below for reference ...

Page 34: ...Calibrating Device with Custom Calibration Sample Application 34 Figure 4 11 Bottom Looking Up Figure 4 12 Viewpoint 6 ...

Page 35: ...rial number as the folder name For example CustomCalibrationSample CustomResult 725112060578 DC1 6 RGB images colorImage001 png colorImage002 png colorImage003 png colorImage004 png colorImage005 png colorImage006 png 6 left images leftImage001 png leftImage002 png leftImage003 png leftImage004 png leftImage005 png leftImage006 png Right RGB Images rightImage001 png rightImage002 png rightImage003...

Page 36: ...ame DistortionRight value 0 123289 value value 0 290911 value value 0 000149083 value value 0 000723527 value value 0 0992178 value param param name RotationLeftRight value 0 999983 value value 0 00150008 value value 0 00558928 value value 0 00150249 value value 0 999999 value value 0 000428313 value value 0 00558863 value value 0 000436704 value value 0 999984 value param param name TranslationLe...

Page 37: ...4 value value 0 00346622 value value 0 000876088 value value 0 00346642 value value 0 999994 value value 0 000222113 value value 0 000875313 value value 0 000225148 value value 1 value param param name TranslationLeftRGB value 14 6987 value value 0 0644999 value value 0 448085 value param Config 4 5 Updating Results to Device We will use Intel Realsense CustomRW to read the CalibrationParameters x...

Page 38: ...ting Calibration 4 5 2 Writing Optimized Calibration to Device Change directory to Calibration Tool API bin Cd C CalibrationToolAPI 2 5 2 0 bin Dump the current parameters on the device into before txt Intel Realsense CustomRW exe r before txt Writing the parameters in the XML result file into device Intel Realsense CustomRW exe w CalibrationParameters xml ...

Page 39: ...ce with Custom Calibration Sample Application 39 Figure 4 14 Updating Calibration Parameters to Device Dump the parameters from device into after txt Intel Realsense CustomRW r after txt Compare the parameters changed ...

Page 40: ...Calibrating Device with Custom Calibration Sample Application 40 Figure 4 15 Calibration Parameter Change 4 5 3 Depth Quality Check after Updating Calibration ...

Page 41: ...Calibrating Device with Custom Calibration Sample Application 41 ...

Page 42: ...re images from the camera module perform the desired calibration computations with OpenCV and to write the new calibration parameters back to the cameras using routines from the Intel RealSense calibration library Figure 5 1 General Process for custom Calibration 5 1 Sample Application Source Code and Compile The source code for Custom Calibration Sample Application is included in the Calibration ...

Page 43: ...rapper h Rs400Dev cpp Rs400Dev h FileOp h Main cpp CustomCalibrationTest vcxproj CustomCalibrationTest sln Common CalibParamXmlWrite h ThirdParty glut The sample app depends on a few third party libraries including GLUT LibRealSense and OpenCV 3 3 GLUT is included under examples ThirdParty glut LibRealSense is included under librealsense bin contains the runtime libraries for Calibration API GLUT ...

Page 44: ...n with Visual Studio 2015 and build Release x64 5 2 Calibration Mode Camera Configuration For calibration the camera device needs to be configured to capture calibration images for both depth and RGB camera using LibRealSense Basic device operations are defined in Rs400Dev h and Rs400Dev cpp The Configurations for the device is defined in CalibrationManager h and CalibrationManager cpp 5 2 1 Emitt...

Page 45: ...t void rightImage const void colorImage const uint64_t timeStamp 5 2 4 Image Captures The sample app captures 6 images More images from different positions may improve accurate The number of images is defined in CalibrationManager h const int NUM_SHOTS 6 5 2 5 Demosaic Left Right Images for ASR PSR SKUs For ASR PSR SKUs left and right images need to be demosaiced 5 3 Detecting the Chessboard in an...

Page 46: ...for int j 0 j chessboardSize width j corners3D 0 i chessboardSize width j Point3f j checkerSize i checkerSize 0 0f for size_t i 1 i numImages i corners3D i corners3D 0 double CalibrateDepthCamera const vector vector Point2f cornersLeft const vector vector Point2f cornersRight const Size chessboardSize float checkerSize const Size imageSizeLR Mat Kl Mat Dl Mat Kr Mat Dr Mat Rlr Mat Tlr CV_Assert co...

Page 47: ...D chessboardSize checkerSize cornersLeft size corners3D Calibrate RGB camera calibrateCamera corners3D cornersRGB imageSizeRGB Kc Dc noArray noArray CV_CALIB_FIX_ASPECT_RATIO TermCriteria TermCriteria COUNT TermCriteria EPS 60 DBL_EPSILON Calibrate the extrinsics between them return stereoCalibrate corners3D cornersLeft cornersRGB Kl Dl Kc Dc Size 1 1 Rlc Tlc noArray noArray CV_CALIB_FIX_INTRINSIC...

Page 48: ... link to the WriteCustomCalibrationParameters and ReadCalibrationParameters and write directly to the device through the APIs To write the results into a parameter XML file and then use CustomRW to read write the parameters to the device Custom Calibration Sample Application write the results into a parameter XML file and use CustomRW to update the calibration parameters to the device ...

Page 49: ...Developing Custom Calibration Application 49 ...

Search results

Summary of Contents for RealSense D400 Series

Reviews:

Related manuals for RealSense D400 Series

Brands by name

Popular brands

Intel RealSense D400 Series, User Manual

Search results

Summary of Contents for RealSense D400 Series

Reviews:

Related manuals for RealSense D400 Series

Brands by name

Popular brands