Introduction

5.1. Introduction#

For this course, we have chosen the camera as a sensor, because it is often used in robotics applications and most people have hands on experience in using cameras.

5.1.1. What is camera calibration?#

Camera calibration is the process of estimating unknown parameters of a camera. There are internal parameters of a camera, which are called intrinsic parameters. Moreover, there is the position/orientation of the camera in the world, which is called extrinsic parameters. The problem of camera calibration addresses how these internal and external parameters can be estimated.

Intrinsic parameters describes the camera’s internal optics and sensor geometry (e.g. focal lengths, principal point, lens distortion) which allows for instance to undistort the raw camera images, as shown in Fig. 5.1.

Extrinsic parameters describe the camera pose in world coordinates, meaning that it describes the position and orientation of the camera in some global 3d Euclidian reference frame. This allows you to interpret pixel measurements in metric 3D space.

geometric_calibration

Fig. 5.1 Distortion by a fish eye lens. (CC BY 2.0: https://commons.wikimedia.org/wiki/File:Oslo_City_buildings_as_seen_by_a_fisheye_lens.jpg)#

Camera calibration typically involves the camera observing a known planar pattern, such as a checkerboard with known dimensions. The pattern is observed at a few different orientations, where either the camera or the pattern can be moved. We then detect feature points (e.g. corners) on the plane and estimate the intrinsic and extrinsic parameters. We can do this by minimizing an error metric (the re-projection error) of the known feature positions of our planar pattern. This process will be addressed in this course.

Based on this initial estimation, we then estimate the distortion parameters, which in turn will be used to refine the intrinsic parameters. However, in this course we won’t go into this part. Thus, we assume that the lens is not distorting. In a practical situation, lenses are distorting and this distortion also needs to be calibrated.

5.1.2. Why should you care?#

In robotics, calibration is essential because cameras serve as a primary sensing modality for tasks such as navigation, object detection, and manipulation. An uncalibrated camera produces images that do not faithfully correspond to the geometry of the real world, making metric measurements (e.g., distances, sizes, depth estimation) unreliable. In Fig. 5.1 you see how a checkerboard can be distorted. We know that the checkerboard has straight lines. However, in the image we see that the lines are not straight anymore.

In practice, poor calibration can lead to:

  • Navigation errors: a robot may misjudge distances to obstacles, causing collisions or overly conservative path planning.

  • Pose estimation drift: if we use a camera to estimate the position or the speed of robot, (e.g. in methods like visual odometry or SLAM), even small errors in intrinsics can accumulate, leading to distorted maps or incorrect localization.

  • Faulty object interaction: a manipulator relying on vision may miss its grasp target or apply force in the wrong place.

  • Sensor fusion issues: if a camera is mis-calibrated relative to another sensor, such as an Inertial Measurement Unit (IMU) or LiDAR sensor, fused estimates become inconsistent, reducing reliability.

  • Distorted reconstructions: 3D maps and point clouds generated from images appear warped or scaled incorrectly.

With good calibration you can:

  • Undistort images to remove lens-induced warping, so straight lines in the scene look straight in the image.

  • Project image points in world coordinates (or the other way around: reproject 3D scene points into pixels).

  • Fuse camera data with IMU/LiDAR/robot kinematics reliably, because you know the camera’s pose and scale relative to those sensors.

5.1.3. What can you expect from this module?#

  • Learn why calibration is necessary in robotics

  • How to design, execute and evaluate a camera calibration procedure

  • The mathematical tools required to formulate and solve the calibration problem

  • Practical tips on camera calibration

Contents