鱼眼镜头相机标定：理论与实践

精确的相机标定是任何3D视觉系统的基础。对于鱼眼相机——尤其是安装在天花板上俯视的相机——标准标定方法会失效。本文涵盖从内参估计到多相机外参对齐的完整标定流程，包括数学基础和实际实现细节。

为什么鱼眼标定不同

标准针孔相机模型假设视场角较小，径向畸变可以用多项式项近似。鱼眼镜头（FOV > 180°）需要根本不同的投影模型。

投影模型对比

带径向畸变的针孔模型：

$r_{distorted} = r(1 + k_1 r^2 + k_2 r^4 + k_3 r^6)$

这种多项式展开在大角度时发散，不适用于鱼眼。

等距（f-theta）模型：

$r = f \cdot \theta$

图像半径与入射角之间的线性关系。最常用于鱼眼标定。

Kannala-Brandt 通用模型：

$r = k_1 \theta + k_2 \theta^3 + k_3 \theta^5 + k_4 \theta^7 + k_5 \theta^9$

关于 $\theta$ （而非 $r$ ）的灵活多项式，可处理各种鱼眼设计。

OpenCV 鱼眼模型

OpenCV 实现了修改后的等距模型：

$\theta_d = \theta(1 + k_1\theta^2 + k_2\theta^4 + k_3\theta^6 + k_4\theta^8)$

其中 $\theta = \arctan(r/f)$ 是与光轴的夹角。

投影方程：

$x' = \frac{x}{r} \cdot \theta_d, \quad y' = \frac{y}{r} \cdot \theta_d$

$u = f_x \cdot x' + c_x, \quad v = f_y \cdot y' + c_y$

内参标定流程

步骤 1：标定靶设计

对于天花板鱼眼相机，我们使用放置在地板上的大型 AprilTag 网格（2m × 2m）：

import cv2
import numpy as np
from cv2 import aruco

def create_calibration_board(
    rows=6, cols=8,
    square_size=0.15,  # 米
    marker_size=0.11,
    dictionary=aruco.DICT_4X4_100
):
    """生成用于打印的 ChArUco 板。"""
    aruco_dict = aruco.getPredefinedDictionary(dictionary)
    board = aruco.CharucoBoard(
        (cols, rows),
        squareLength=square_size,
        markerLength=marker_size,
        dictionary=aruco_dict
    )
    
    # 生成用于打印的高分辨率图像
    img = board.generateImage((4000, 3000))
    return board, img

为什么选择 ChArUco 而非棋盘格？

部分可见性处理：只需要标记的子集
唯一角点识别：每个角点有已知的 ID
对遮挡的鲁棒性：对于有人走动的大板很关键

步骤 2：多姿态图像采集

采集 50-100 张标定板位于不同位置的图像：

class CalibrationImageCollector:
    def __init__(self, camera, board, output_dir):
        self.camera = camera
        self.board = board
        self.output_dir = output_dir
        self.detector = aruco.CharucoDetector(board)
        self.images = []
        self.corners_list = []
        self.ids_list = []
    
    def capture_and_detect(self):
        """捕获帧并检测 ChArUco 角点。"""
        frame = self.camera.capture()
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        
        # 检测 ChArUco
        charuco_corners, charuco_ids, marker_corners, marker_ids = \
            self.detector.detectBoard(gray)
        
        if charuco_corners is not None and len(charuco_corners) >= 6:
            # 需要最少角点数以获得可靠姿态
            self.corners_list.append(charuco_corners)
            self.ids_list.append(charuco_ids)
            self.images.append(frame.copy())
            
            # 可视化
            annotated = aruco.drawDetectedCornersCharuco(
                frame.copy(), charuco_corners, charuco_ids
            )
            return True, annotated
        
        return False, frame
    
    def get_coverage_map(self, img_size):
        """可视化角点在图像中的分布。"""
        coverage = np.zeros(img_size[:2], dtype=np.float32)
        
        for corners in self.corners_list:
            for corner in corners:
                x, y = corner.ravel().astype(int)
                cv2.circle(coverage, (x, y), 20, 1.0, -1)
        
        return coverage

覆盖要求：

角点分布在整个图像中，尤其是边缘
多种距相机的距离
各种标定板方向

步骤 3：鱼眼标定

def calibrate_fisheye(corners_list, ids_list, board, image_size):
    """
    使用 OpenCV 标定鱼眼相机。
    返回：K (3x3), D (4x1), rvecs, tvecs, reproj_error
    """
    # 为每个检测准备物点
    obj_points = []
    img_points = []
    
    for corners, ids in zip(corners_list, ids_list):
        obj_pts, img_pts = board.matchImagePoints(corners, ids)
        if obj_pts is not None and len(obj_pts) >= 6:
            obj_points.append(obj_pts.astype(np.float32))
            img_points.append(img_pts.astype(np.float32))
    
    # 内参初始猜测
    K = np.array([
        [image_size[0] * 0.5, 0, image_size[0] * 0.5],
        [0, image_size[1] * 0.5, image_size[1] * 0.5],
        [0, 0, 1]
    ], dtype=np.float64)
    
    D = np.zeros((4, 1), dtype=np.float64)
    
    # 标定标志
    flags = (
        cv2.fisheye.CALIB_RECOMPUTE_EXTRINSIC |
        cv2.fisheye.CALIB_CHECK_COND |
        cv2.fisheye.CALIB_FIX_SKEW
    )
    
    # 运行标定
    criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 100, 1e-6)
    
    ret, K, D, rvecs, tvecs = cv2.fisheye.calibrate(
        obj_points,
        img_points,
        image_size,
        K,
        D,
        flags=flags,
        criteria=criteria
    )
    
    return K, D, rvecs, tvecs, ret

# 使用
K, D, rvecs, tvecs, reproj_error = calibrate_fisheye(
    corners_list, ids_list, board, (1920, 1080)
)
print(f"重投影误差：{reproj_error:.4f} 像素")

步骤 4：验证

可视化重投影误差以识别异常值：

def compute_per_image_error(obj_points, img_points, K, D, rvecs, tvecs):
    """计算每张标定图像的重投影误差。"""
    errors = []
    
    for obj_pts, img_pts, rvec, tvec in zip(obj_points, img_points, rvecs, tvecs):
        projected, _ = cv2.fisheye.projectPoints(
            obj_pts.reshape(-1, 1, 3), rvec, tvec, K, D
        )
        
        error = np.sqrt(np.sum((img_pts - projected.reshape(-1, 2))**2, axis=1))
        errors.append({
            'mean': np.mean(error),
            'max': np.max(error),
            'std': np.std(error),
            'per_point': error
        })
    
    return errors

def filter_outlier_images(errors, threshold_factor=2.0):
    """移除重投影误差高的图像。"""
    mean_errors = [e['mean'] for e in errors]
    median = np.median(mean_errors)
    mad = np.median(np.abs(mean_errors - median))
    
    threshold = median + threshold_factor * mad * 1.4826
    
    valid_indices = [i for i, e in enumerate(errors) if e['mean'] < threshold]
    return valid_indices

外参标定：相机到世界

一旦内参已知，我们需要相机在世界坐标中的位姿。

地面平面标定

对于天花板相机，我们标定到地面水平的坐标系统：

class FloorCalibrator:
    def __init__(self, K, D):
        self.K = K
        self.D = D
        self.floor_points_3d = []  # 已知世界坐标
        self.floor_points_2d = []  # 对应图像点
    
    def add_reference_point(self, world_xyz, image_uv):
        """添加已知地面点。"""
        self.floor_points_3d.append(world_xyz)
        self.floor_points_2d.append(image_uv)
    
    def calibrate(self, method='pnp'):
        """从地面参考点计算相机外参。"""
        obj_pts = np.array(self.floor_points_3d, dtype=np.float32)
        img_pts = np.array(self.floor_points_2d, dtype=np.float32)
        
        # 去畸变图像点
        img_pts_undist = cv2.fisheye.undistortPoints(
            img_pts.reshape(-1, 1, 2), self.K, self.D, P=self.K
        ).reshape(-1, 2)
        
        if method == 'pnp':
            success, rvec, tvec = cv2.solvePnP(
                obj_pts, img_pts_undist, self.K, None,
                flags=cv2.SOLVEPNP_ITERATIVE
            )
        elif method == 'ransac':
            success, rvec, tvec, inliers = cv2.solvePnPRansac(
                obj_pts, img_pts_undist, self.K, None,
                reprojectionError=5.0
            )
        
        R, _ = cv2.Rodrigues(rvec)
        
        # 相机在世界坐标中的位置
        camera_position = -R.T @ tvec
        
        return R, tvec, camera_position

多相机外参对齐

对于多个重叠的鱼眼相机，我们需要一致的世界坐标：

class MultiCameraCalibrator:
    def __init__(self, cameras):
        self.cameras = cameras
        self.shared_points = {}  # point_id -> {cam_id: image_coords}
    
    def add_shared_observation(self, point_id, cam_id, image_coords):
        """记录从某相机观察到的共享点。"""
        if point_id not in self.shared_points:
            self.shared_points[point_id] = {}
        self.shared_points[point_id][cam_id] = image_coords
    
    def triangulate_point(self, point_id):
        """从多个视角三角化 3D 位置。"""
        observations = self.shared_points[point_id]
        
        if len(observations) < 2:
            return None
        
        # 构建投影矩阵
        A = []
        for cam_id, img_pt in observations.items():
            cam = self.cameras[cam_id]
            P = cam.K @ np.hstack([cam.R, cam.t])
            
            # 去畸变点
            pt_undist = cv2.fisheye.undistortPoints(
                np.array([[img_pt]], dtype=np.float32),
                cam.K, cam.D, P=cam.K
            ).ravel()
            
            x, y = pt_undist
            A.append(x * P[2] - P[0])
            A.append(y * P[2] - P[1])
        
        A = np.array(A)
        _, _, Vt = np.linalg.svd(A)
        X = Vt[-1]
        X = X[:3] / X[3]
        
        return X
    
    def optimize_extrinsics(self, initial_poses, shared_points_3d):
        """光束法平差以优化所有相机位姿。"""
        from scipy.optimize import least_squares
        
        def residuals(params):
            # 解包参数
            poses = params.reshape(-1, 6)  # 每个相机 [rx, ry, rz, tx, ty, tz]
            
            errors = []
            for point_id, point_3d in shared_points_3d.items():
                for cam_id, img_pt in self.shared_points[point_id].items():
                    cam_idx = list(self.cameras.keys()).index(cam_id)
                    rvec = poses[cam_idx, :3]
                    tvec = poses[cam_idx, 3:]
                    
                    cam = self.cameras[cam_id]
                    projected, _ = cv2.fisheye.projectPoints(
                        point_3d.reshape(1, 1, 3),
                        rvec, tvec, cam.K, cam.D
                    )
                    
                    errors.extend((projected.ravel() - np.array(img_pt)).tolist())
            
            return np.array(errors)
        
        # 初始参数向量
        x0 = np.array([
            np.hstack([cv2.Rodrigues(p['R'])[0].ravel(), p['t'].ravel()])
            for p in initial_poses
        ]).ravel()
        
        result = least_squares(residuals, x0, method='lm')
        
        return result.x.reshape(-1, 6)

实用标定技巧

1. 处理边缘的极端畸变

角点检测在图像边缘会失败。使用大搜索窗口的亚像素细化：

def refine_corners_fisheye(gray, corners, win_size=11):
    """鱼眼图像的亚像素角点细化。"""
    criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 100, 0.001)
    
    refined = cv2.cornerSubPix(
        gray, 
        corners.astype(np.float32),
        (win_size, win_size),
        (-1, -1),
        criteria
    )
    
    return refined

2. 温度补偿

镜头焦距随温度变化：

def temperature_compensate_intrinsics(K, D, temp_celsius, ref_temp=25):
    """
    调整内参以补偿温度漂移。
    典型系数：玻璃镜头约 20 ppm/°C
    """
    temp_coeff = 20e-6  # 每度 ppm
    delta_temp = temp_celsius - ref_temp
    
    scale = 1 + temp_coeff * delta_temp
    
    K_adjusted = K.copy()
    K_adjusted[0, 0] *= scale  # fx
    K_adjusted[1, 1] *= scale  # fy
    
    return K_adjusted

3. 使用已知距离验证

始终使用独立测量进行验证：

def validate_calibration(K, D, R, t, known_distances):
    """
    使用已知地面距离验证标定。
    known_distances: ((u1,v1), (u2,v2), ground_truth_meters) 的列表
    """
    errors = []
    
    for (pt1_img, pt2_img, gt_distance) in known_distances:
        # 将点投影到地面平面（世界坐标 Z=0）
        pt1_world = image_to_floor(pt1_img, K, D, R, t)
        pt2_world = image_to_floor(pt2_img, K, D, R, t)
        
        measured = np.linalg.norm(pt1_world - pt2_world)
        error_percent = abs(measured - gt_distance) / gt_distance * 100
        errors.append(error_percent)
    
    print(f"距离验证：")
    print(f"  平均误差：{np.mean(errors):.2f}%")
    print(f"  最大误差：{np.max(errors):.2f}%")
    print(f"  标准差：{np.std(errors):.2f}%")
    
    return errors

完整标定配置

# calibration_config.yaml
intrinsics:
  board:
    type: "charuco"
    rows: 6
    cols: 8
    square_size: 0.15  # 米
    marker_size: 0.11
    dictionary: "DICT_4X4_100"
  
  collection:
    min_images: 50
    max_images: 100
    min_corners_per_image: 10
    coverage_threshold: 0.8  # 图像区域的 80%
  
  optimization:
    fix_skew: true
    fix_principal_point: false
    recompute_extrinsic: true
    max_iterations: 100
    epsilon: 1e-6
  
  validation:
    max_reproj_error: 0.5  # 像素
    outlier_threshold: 2.0  # MAD 乘数

extrinsics:
  method: "floor_calibration"
  min_reference_points: 6
  ransac_threshold: 5.0  # 像素
  
  floor_markers:
    - id: "A1"
      world_position: [0.0, 0.0, 0.0]
    - id: "A2"
      world_position: [3.0, 0.0, 0.0]
    - id: "B1"
      world_position: [0.0, 4.0, 0.0]
    # ... 更多标记

validation:
  known_distances:
    - points: ["A1", "A2"]
      distance: 3.0
    - points: ["A1", "B1"]
      distance: 4.0
  max_distance_error_percent: 1.0

结论

鱼眼相机标定需要：

正确的投影模型：使用 OpenCV 的鱼眼模块，而非标准标定
适当的标定靶：天花板安装相机使用大型 ChArUco 板
完整覆盖：角点遍布整个图像，尤其是边缘
迭代细化：多相机设置使用光束法平差
独立验证：始终使用已知物理测量进行验证

标定良好的系统可达到 < 0.5 像素重投影误差和 < 1% 地面平面距离测量误差——这对于精确的人体定位和空间分析至关重要。