Chessboard Corner Detection

Camera calibration often utilizes a chessboard pattern because the intersection points of black and white squares are easily localized. The cv::findChessboardCorners function is designed to locate these internal corners within an image.

When using this function, the patternSize parameter must accurately reflect the number of internal corners (width and height), not the number of squares. The rows and columns should ideally differ to ensure the function can uniquely determine the board's orientation.

bool found = cv::findChessboardCorners(
    cv::InputArray image,       // Input 8-bit grayscale or color image
    cv::Size patternSize,       // Number of inner corners per board dimension
    cv::OutputArray corners,    // Output vector of detected corners
    int flags                   // Operation flags
);

The flags parameter controls the detection behavior. Common flags include CALIB_CB_ADAPTIVE_THRESH for adaptive thresholding based on average brightness, and CALIB_CB_NORMALIZE_IMAGE to apply histogram equalization before thresholding. The CALIB_CB_FAST_CHECK flag performs a quick scan to verify the presence of a chessboard, which optimizes performance in scenarios where the board may not be visible.

Subpixel Refinement

Initial corner detection provides pixel-level coordinates. To achieve higher precision, cv::cornerSubPix refines these coordinates to subpixel accuracy by iteratively minimizing the error based on image gradients within a defined search window.

cv::cornerSubPix(
    cv::InputArray image,       // Grayscale input image
    cv::InputOutputArray corners, // Initial coordinates and refined output
    cv::Size winSize,           // Half of the search window side length
    cv::Size zeroZone,          // Half of the dead region size (-1,-1) for none
    cv::TermCriteria criteria   // Termination criteria for iteration
);

Visualization

The cv::drawChessboardCorners function renders the detected corners onto the source image. If the pattern is fully detected (patternWasFound is true), the corners are connected by lines; otherwise, individual red circles mark the detected points.

Camera Calibration Process

The core calibration function cv::calibrateCamera estimates intrinsic camera parameters (focal length, principal point) and distortion coefficients, as well as extrinsic parameters (rotation and translation vectors) for each view.

double rms = cv::calibrateCamera(
    std::vector>& objectPoints, // 3D world points
    std::vector>& imagePoints,  // 2D image points
    cv::Size imageSize,         // Image dimensions
    cv::Mat& cameraMatrix,      // Output 3x3 intrinsic matrix
    cv::Mat& distCoeffs,        // Output distortion coefficients
    std::vector& rvecs, // Output rotation vectors
    std::vector& tvecs, // Output translation vectors
    int flags,                   // Calibration flags
    cv::TermCriteria criteria    // Optimization termination
);

This function minimizes the reprojection error—the distance between the projected 3D points and the detected 2D points—using the Levenberg-Marquardt algorithm. Flags such as CALIB_FIX_PRINCIPAL_POINT or CALIB_FIX_ASPECT_RATIO can constrain specific parameters during optimization. The function returns the root mean square (RMS) error, providing a metric for calibration quality.

Distortion Correction and Projection

Once the camera parameters are known, cv::projectPoints can project 3D world coordinates onto the 2D image plane using the calculated intrinsic and extrinsic parameters.

To rectify lens distortion, the process involves generating a transformation map using cv::initUndistortRectifyMap followed by cv::remap. The mapping function computes the pixel coordinates in the destination (undistorted) image for every pixel in the source (distorted) image.

cv::Mat map1, map2;
cv::initUndistortRectifyMap(
    cameraMatrix, distCoeffs, cv::Mat(),
    newCameraMatrix, imageSize, CV_16SC2, map1, map2
);
cv::remap(srcImage, dstImage, map1, map2, cv::INTER_LINEAR);

Stereo Vision Calibration

For stereo camera setups, cv::stereoCalibrate computes the rotational and translational relationship between the two cameras. This function simultaneously calibrates both cameras and finds the essential (E) and fundamental (F) matrices.

Subsequently, cv::stereoRectify computes the rotation matrices (R1, R2) and projection matrices (P1, P2) required to align the stereo image pair epipolar lines. This ensures that a feature point in the left image lies on the same row in the right image, simplifying the correspondence search.

For disparity map generation, the Semi-Global Block Matching (SGBM) algorithm is commonly used. The computed disparity map can then be converted into a 3D point cloud using cv::reprojectImageTo3D, which applies the Q matrix derived during stereo rectification.