Plate Calibration User Guide

This guide walks you through the camera calibration process using a calibration plate (dot grid) in OpenLPT. The process consists of two main stages: Point Detection and Calibration, with an optional Pinhole+Refraction calibration workflow for refractive setups.

Hardware Setup Recommendation:

Non-coplanar Views: It is highly recommended that the viewing axes of your cameras are NOT in the same plane. An angular difference of at least 5 degrees relative to the central plane significantly improves 3D reconstruction accuracy.

1. Point Detection

First, select the Plate Calibration tab and ensure you are on the Point Detection sub-tab.

Step 1: Configure Camera Settings

Set the camera parameters in the Camera Images panel:

Num Cameras: Enter the number of cameras in your setup (e.g., 3).
Target Camera: Select the camera you want to calibrate (e.g., Camera 1).

Step 2: Load Calibration Images

Click Open Files to select one or more calibration plate images for the target camera.
Images will appear in the Frame List below.
Click on any frame to preview it in the left panel.

Step 3: Configure Detection Settings

In the Detection Settings panel:

Point Color: Choose Bright (Standard) for bright dots on dark background, or Dark for dark dots on bright background.
Select Template: Click this button, then draw a rectangle around ONE dot in the image. This template will be used for template matching.
Select Search ROI (Optional): Define a region of interest to limit the search area.
Match Threshold: Adjust the slider (default 0.70). Lower values detect more points but may include false positives.

Step 4: Detect Points

Click Detect to run template matching on the current image.
Verification: Look at the image view on the left. You should see green crosses marking all detected dots.
If detection is incorrect:
- Click Remove to clear all detected points.
- Adjust the Match Threshold or select a better template.
Click Add to manually add individual points by clicking on the image.

Step 5: Index Points (Assign 3D Coordinates)

In the Indexing panel, assign world coordinates to detected points:

Set Origin: Click this button, then click on a detected point to mark it as the origin (0, 0, 0).
Fixed Axis: Select which axis is fixed for this plane:
- Z fixed (default): For a horizontal plate at a known Z-plane.
- X or Y fixed: For vertical plates.
Plane: Enter the value of the fixed axis (e.g., 0 for Z=0 plane).
Set Axis Directions: Click this button, then click two points to define the positive X and Y directions. A hint will follow your mouse showing which axis to click.
Index Points: Click to automatically assign indices to all detected points based on the origin and axis directions.

Step 6: Configure Physical Spacing

Set the physical spacing between dots:

dx: Spacing in X direction (e.g., 10.00 mm).
dy: Spacing in Y direction (e.g., 10.00 mm).
dz: Spacing in Z direction (usually 0.00 mm for flat plates).

Click Check Position to verify the 3D world coordinates are correct by clicking on a point.

Step 7: Add to Calibration Data

When satisfied with the indexing, click Add to Calibration Data.
Repeat Steps 2-7 for all calibration images and all cameras.

                Important: Each camera can have multiple images at different planes.
                The more images you add, the more robust the calibration will be.
            

2. Calibration

After adding points for all cameras, switch to the Calibration sub-tab.

Pinhole Model: Note on Refracted Interfaces:

The pinhole calibration model assumes a homogeneous medium without refractive interfaces. For experimental setups involving observation windows (e.g., glass or acrylic), it is critical that the cameras are oriented as close to the surface normal (orthogonal) as possible. Furthermore, it is assumed that the paraxial approximation is valid within the reconstructed volume (i.e., small-angle observation) to maintain geometric accuracy.

Step 8: Configure Camera Settings

In the Camera Settings panel:

Target Camera: Select the camera to calibrate.
Model: Choose Pinhole (standard camera model).
Image Width/Height (px): Verify the image resolution (e.g., 1280 x 800).
Sensor Width (mm/px): Enter the sensor pixel pitch (e.g., 0.0200 mm/px).
Focal Length (mm): Enter the lens focal length (e.g., 180.00 mm).
Distortion Coeffs Num: Number of distortion coefficients to optimize (0 for no distortion, 5 for full model).

Step 9: Run Calibration

Click Run Calibration.
The system will perform a two-stage optimization:
- Stage 1: Estimate camera pose using solvePnP.
- Stage 2: Refine intrinsics and extrinsics using bundle adjustment.
The 3D View on the left will visualize the camera positions and calibration points.
Check the Mean RMS Error in the Calibration Results section (should be < 1.0 px for good calibration).

Step 10: Calibrate All Cameras

Change the Target Camera to the next camera.
Click Run Calibration again.
Repeat for all cameras.
The 3D View will accumulate all calibrated cameras, showing their relative positions.

Step 11: Save Results

When all cameras are calibrated, click Save All Camera Parameters.
Select an output directory.
A camFile folder will be created containing:
- cam0.txt - Camera 0 parameters
- cam1.txt - Camera 1 parameters
- cam2.txt - Camera 2 parameters
- ...

                Parameter File Format: The saved files are compatible with OpenLPT's tracking module
                and contain camera matrix, distortion coefficients, rotation matrix, and translation vector.
            

3. Calibration Model: Pinhole + Refraction

Use this model when refractive interfaces are non-negligible and you need to model camera-side medium, window material/thickness, and object-side medium explicitly.

Refraction Model Notes:

It is highly recommended to use 2 or more cameras per refraction plate/window for better calibration robustness. Multiple plates are recommended for better calibration results. If your camera viewing angle is nearly normal to the refraction plate, it is still recommended to use the Pinhole model.

$Refraction Settings Panel$

Step 12: Select the Refraction Model

In Camera Settings, set Model to Pinhole+Refraction.
Verify image size, sensor pitch, and focal-length initialization for each camera.

Step 13: Configure Refraction Settings

Set Number of Windows.
In Camera-Window Mapping, assign each camera to its corresponding window.
In Window Configuration, set media/material parameters for each layer:
- Camera side (e.g., Air, refractive index)
- Window (material index and thickness)
- Object side (e.g., Water, refractive index)

Step 14: Run and Refine Calibration

Click Run Calibration and review residual errors.
Calibrate all cameras after verifying each camera's fit quality.
Save all camera parameters when results are stable.

Troubleshooting

High RMS Error

Check that all points are correctly indexed (no reversed axes).
Verify the physical spacing (dx, dy) is accurate.
Ensure the template matched the correct dots (not reflections or noise).
Add more images from different angles to improve calibration.

Points Not Detected

Adjust the Match Threshold slider (lower = more detections).
Select a clearer template from a well-lit area of the image.
Change Point Color if dots are dark on bright background.