Data Storage in OpenCV

OpenCV provides flexible data storage through the InputArrayOfArrays interface, which can be implemented by various types: Mat, Mat_<T>, Mat_<T, m, n>, vector<T>, vector<vector<T>>, and vector<Mat>.

Consider vector<vector<T>> as an example: the outer vector holds elements (inner vectors) each representing one image, while the inner vector stores the coordinates of corner points for that image. If Mat is used, each row could represent a single image, with columns corresponding to the number of images.

Storing World Coordinates

/****************************************************************************************************************************
 * Function:    static void calcChessboardCorners(Size boardSize, float squareSize, vector<Point3f>& corners, Pattern patternType = CHESSBOARD)
 * Purpose:     Compute all possible world coordinates for chessboard corners and store them in a container.
 * Parameters:
 *      1. Size boardSize          - Dimensions of the chessboard (number of inner corners per row and column).
 *      2. float squareSize        - Distance between adjacent corners on the chessboard.
 *      3. vector<Point3f>& corners - Container to store the 3D world coordinates of corners.
 *      4. Pattern patternType     - Type of calibration pattern (default is CHESSBOARD).
 * Return:
 *      void
 ****************************************************************************************************************************/

static void calcChessboardCorners(Size boardSize, float squareSize, vector<Point3f>& corners, Pattern patternType = CHESSBOARD)
{
    corners.resize(0);

    switch (patternType)
    {
    case CHESSBOARD:
    case CIRCLES_GRID:
        for (int i = 0; i < boardSize.height; i++)
            for (int j = 0; j < boardSize.width; j++)
                // typedef Point3_<float> Point3f;
                // Each call creates a 3D point using the Point3_<float> constructor.
                // Assumes that the first inner corner (bottom-left in world) is the origin.
                corners.push_back(Point3f(float(j * squareSize),
                                          float(i * squareSize), 0));
        break;

    case ASYMMETRIC_CIRCLES_GRID:
        for (int i = 0; i < boardSize.height; i++)
            for (int j = 0; j < boardSize.width; j++)
                corners.push_back(Point3f(float((2 * j + i % 2) * squareSize),
                                          float(i * squareSize), 0));
        break;

    default:
        CV_Error(Error::StsBadArg, "Unknown pattern type\n");
    }
}

Implementation Logic for World Coordinates

The core idea is that during image capture, the chessboard's world coordinate system is fixed with its origin at the bottom-left inner corner. Regardless of how the chessboard is moved or the camera is repositioned, the world coordinates of all chessboard corners remain constant. Only the image (pixel) coordinates of these corners change. Therefore, the world coordinates are identical for every image of the chessboard.

To match each image with its corresponding world coordinates, we store all world coordinates across all images. The storage structure is a container of containers: the outer container holds one element per image, and each inner container holds the world coordinates of all corners for that image. This forms an N×(H×W) matrix, where N is the total number of images, H is the number of inner corner rows, and W is the number of inner corner columns.

// Create an outer container with a single inner container for the first image.
vector<vector<Point3f>> objectPoints(1);

// Compute world coordinates for the first image.
calcChessboardCorners(boardSize, squareSize, objectPoints[0], patternType);

// Adjust the last corner's x-coordinate if needed (e.g., to set a known physical width).
objectPoints[0][boardSize.width - 1].x = objectPoints[0][0].x + grid_width;

// Keep a copy for later use.
newObjPoints = objectPoints[0];

// 'imagePoints.size()' gives the number of images.
// Resize the outer container to match the number of images, filling new entries with a copy of the first set of world coordinates.
objectPoints.resize(imagePoints.size(), objectPoints[0]);

This method ensures that for every image, the world coordinates are consistent and correspond to the same physical grid.