Stereo Calibration Opencv Python and Disparity Map

Question

I'm keen on discovering the different map of a scene. In the first place, I did sound system adjustment/calibration utilizing the accompanying code (I composed it myself with a little assistance from Google, in the wake of neglecting to track down any accommodating instructional exercises for a similar written in python for OpenCV 2.4.10).

I took pictures of a chessboard at the same time on the two cameras and saved them as left*.jpg and right*.jpg.

import numpy as np

import cv2

import glob

# termination criteria

criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)

# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)

objp = np.zeros((6*9,3), np.float32)

objp[:,:2] = np.mgrid[0:9,0:6].T.reshape(-1,2)

# Arrays to store object points and image points from all the images.

objpointsL = [] # 3d point in real world space

imgpointsL = [] # 2d points in image plane.

objpointsR = []

imgpointsR = []

images = glob.glob('left*.jpg')

for fname in images:

    img = cv2.imread(fname)

    grayL = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

    # Find the chess board corners

    ret, cornersL = cv2.findChessboardCorners(grayL, (9,6),None)

    # If found, add object points, image points (after refining them)

    if ret == True:

        objpointsL.append(objp)

        cv2.cornerSubPix(grayL,cornersL,(11,11),(-1,-1),criteria)

        imgpointsL.append(cornersL)

images = glob.glob('right*.jpg')

for fname in images:

    img = cv2.imread(fname)

    grayR = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

    # Find the chess board corners

    ret, cornersR = cv2.findChessboardCorners(grayR, (9,6),None)

    # If found, add object points, image points (after refining them)

    if ret == True:

        objpointsR.append(objp)

        cv2.cornerSubPix(grayR,cornersR,(11,11),(-1,-1),criteria)

        imgpointsR.append(cornersR)

retval,cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, R, T, E, F = cv2.stereoCalibrate(objpointsL, imgpointsL, imgpointsR, (320,240))

How would I correct the pictures? What different advances would it be advisable for me to do prior to proceeding to discover the disparity map? I read someplace that while ascertaining the uniqueness map, the features detected on the two frames should lie on a similar flat line. Kindly assist me with trip. Any assistance would be quite valued.'

Answer 1

It's great to see your interest in exploring the different aspects of stereo vision and depth mapping. In the code snippet you provided, you have performed camera calibration using a chessboard pattern. To correct the images and proceed with finding the disparity map, there are a few additional steps you need to follow. Here's a breakdown of the process:

Rectification: Before finding the disparity map, it's important to rectify the stereo image pair. Rectification transforms the images so that the corresponding epipolar lines become horizontal and aligned. This step helps simplify the disparity calculation process. OpenCV provides the stereoRectify() function to perform rectification.

Undistortion: Since you have obtained the camera matrices and distortion coefficients during calibration, you can use these parameters to undistort the images. Undistortion corrects for lens distortion and improves the overall image quality. OpenCV offers the undistort() function for this purpose.

Disparity Calculation: Once the rectified and undistorted images are available, you can proceed to calculate the disparity map. The disparity map represents the disparity (or depth) between the corresponding points in the left and right images. OpenCV provides various algorithms for computing disparity, such as Block Matching (StereoBM) or Semi-Global Block Matching (StereoSGBM).

Here's a general outline of the code you can add after the calibration step to perform rectification, undistortion, and disparity calculation:

# Perform stereo rectification

R1, R2, P1, P2, Q, roi1, roi2 = cv2.stereoRectify(cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, (320, 240), R, T)

# Undistort the images

undistortedL = cv2.undistort(grayL, cameraMatrix1, distCoeffs1)

undistortedR = cv2.undistort(grayR, cameraMatrix2, distCoeffs2)

# Rectify the images

map1x, map1y = cv2.initUndistortRectifyMap(cameraMatrix1, distCoeffs1, R1, P1, (320, 240), cv2.CV_32FC1)

map2x, map2y = cv2.initUndistortRectifyMap(cameraMatrix2, distCoeffs2, R2, P2, (320, 240), cv2.CV_32FC1)

rectifiedL = cv2.remap(undistortedL, map1x, map1y, cv2.INTER_LINEAR)

rectifiedR = cv2.remap(undistortedR, map2x, map2y, cv2.INTER_LINEAR)

# Calculate the disparity map

stereo = cv2.StereoBM_create(numDisparities=16, blockSize=15)

disparity = stereo.compute(rectifiedL, rectifiedR)

# Show the disparity map

cv2.imshow("Disparity Map", disparity)

cv2.waitKey(0)

cv2.destroyAllWindows()

Keep in mind that the code provided is just a starting point and may require further adjustment based on your specific setup and requirements. Additionally, you can experiment with different disparity algorithms and parameters to achieve the desired results.

Answer 2

You want cameraMatrix1,  distCoeffs2, distCoeffs1, cameraMatrix2 and "newCameraMatrix" also, for cv2.undistort()

With the help of cv2.getOptimalNewCameraMatrix(), you can also get "newCameraMatrix".

Kindly check the below code:

# Assuming you have left01.jpg and right01.jpg that you want to rectify

lFrame = cv2.imread('left01.jpg')

rFrame = cv2.imread('right01.jpg')

w, h = lFrame.shape[:2] # both frames should be of same shape

frames = [lFrame, rFrame]

# Params from camera calibration

camMats = [cameraMatrix1, cameraMatrix2]

distCoeffs = [distCoeffs1, distCoeffs2]

camSources = [0,1]

for src in camSources:

    distCoeffs[src][0][4] = 0.0 # use only the first 2 values in distCoeffs

# The rectification process

newCams = [0,0]

roi = [0,0]

for src in camSources:

    newCams[src], roi[src] = cv2.getOptimalNewCameraMatrix(cameraMatrix = camMats[src], 

                                                           distCoeffs = distCoeffs[src], 

                                                           imageSize = (w,h), 

                                                           alpha = 0)

rectFrames = [0,0]

for src in camSources:

        rectFrames[src] = cv2.undistort(frames[src], 

                                        camMats[src], 

                                        distCoeffs[src])

# See the results

view = np.hstack([frames[0], frames[1]])    

rectView = np.hstack([rectFrames[0], rectFrames[1]])

cv2.imshow('view', view)

cv2.imshow('rectView', rectView)

# Wait indefinitely for any keypress

cv2.waitKey(0)

Hope you got your answer, Next step is just to calculate "disparity maps"

Join the python online course fast, to learn python concepts in detail and get certified.

Answer 3

Here's a shorter version of the code snippet that includes the essential steps for correcting the images and calculating the disparity ma

import numpy as np

import cv2

# Load left and right images

left_image = cv2.imread("left*.jpg")

right_image = cv2.imread("right*.jpg")

# Perform stereo calibration

# ... (code for calibration)

# Perform stereo rectification and obtain transformation matrices

# ... (code for rectification)

# Rectify the images

left_rectified = cv2.remap(left_image, map1x, map1y, cv2.INTER_LINEAR)

right_rectified = cv2.remap(right_image, map2x, map2y, cv2.INTER_LINEAR)

# Convert rectified images to grayscale

left_gray = cv2.cvtColor(left_rectified, cv2.COLOR_BGR2GRAY)

right_gray = cv2.cvtColor(right_rectified, cv2.COLOR_BGR2GRAY)

# Calculate the disparity map

stereo = cv2.StereoBM_create(numDisparities=16, blockSize=15)

disparity = stereo.compute(left_gray, right_gray)

# Show the disparity map

cv2.imshow("Disparity Map", disparity)

cv2.waitKey(0)

cv2.destroyAllWindows()

Answer 4

Here is the code for the above question

import cv2

# Load left and right images

left_image = cv2.imread("left*.jpg")

right_image = cv2.imread("right*.jpg")

# Perform stereo calibration

# ... (code for calibration)

# Perform stereo rectification

# ... (code for rectification)

# Rectify and convert images to grayscale

left_gray = cv2.cvtColor(cv2.remap(left_image, map1x, map1y, cv2.INTER_LINEAR), cv2.COLOR_BGR2GRAY)

right_gray = cv2.cvtColor(cv2.remap(right_image, map2x, map2y, cv2.INTER_LINEAR), cv2.COLOR_BGR2GRAY)

# Calculate disparity map

disparity = cv2.StereoBM_create(numDisparities=16, blockSize=15).compute(left_gray, right_gray)

# Show disparity map

cv2.imshow("Disparity Map", disparity)

cv2.waitKey(0)

cv2.destroyAllWindows()