What is this?
An interactive playground for the pinhole camera model — the basic building block of multi-view geometry, SfM, SLAM and neural rendering (NeRF, Gaussian Splatting). Every slider changes one number of the camera; the 3D view shows the camera in the world, and the image panel shows what that camera sees. Hover any symbol in the Lab to get a short explanation.
1. From a 3D point to a pixel
A world point Xw becomes a pixel in three steps:
In homogeneous coordinates the whole chain is one matrix:
Step ② is the only non-linear part, and it is where depth is lost: every point on the ray through the camera center C lands on the same pixel. That is why one image cannot give you 3D — you need a second view (next chapter: epipolar geometry).
2. Intrinsics K — the camera itself
- fx, fy — focal length measured in pixels (fx = f / pixel width). They set the field of view: FOVx = 2·atan(W / 2fx).
- cx, cy — the principal point, where the optical axis hits the sensor. Usually close to the image center.
- s — skew, practically always 0 for modern sensors (fixed to 0 here).
K does not depend on where the camera is. You estimate it once with calibration (e.g. OpenCV calibrateCamera on a checkerboard).
3. Extrinsics R, t — where the camera is
- R (3×3 rotation, RᵀR = I, det R = 1) rotates world coordinates into the camera frame. Its rows are the camera's x, y, z axes written in world coordinates.
- t is the world origin expressed in the camera frame. It is not the camera position.
- The camera center in the world is C = −Rᵀt; equivalently t = −RC. Pose has 6 degrees of freedom (3 rotation + 3 translation).
Here R is parameterized by Euler angles R = Rz(rz) Ry(ry) Rx(rx). Euler angles are easy to read but suffer from gimbal lock (at ry = ±90°); real systems use quaternions or axis-angle (Rodrigues) vectors.
4. Conventions (the #1 source of bugs)
| x | y | z (forward) | Used by | |
|---|---|---|---|---|
| OpenCV / COLMAP | right | down | +z | this lab, COLMAP, 3DGS |
| OpenGL / Blender | right | up | −z | NeRF (original), three.js |
Converting between them is a flip of the y and z axes: diag(1, −1, −1). Also check whether a file stores world-to-camera [R | t] (COLMAP) or camera-to-world [Rᵀ | C] (NeRF transforms.json).
5. Things to try
- Increase fx only (unlock fx = fy): the image stretches horizontally — non-square pixels.
- Run Zoom then Dolly zoom. Zoom changes the house size but not the perspective; dolly zoom keeps the house size but changes the perspective (the background moves relative to the house).
- Run Shift (cx, cy) and watch the frustum in 3D become skewed (off-axis) while the camera does not move.
- Move tz until a point goes behind the camera — it disappears from the image (Zc ≤ 0).
- Hover a point and check the numbers in the formula against the matrices.
Roadmap
- Pinhole camera — K, R, t, frustum, projection this page
- Two views — essential & fundamental matrix, epipolar lines, epipoles
- Three views — triangulation, noise and baseline, epipolar transfer
- 3D Gaussian Splatting — projecting 3D Gaussians to 2D splats