The question
A pixel x₁ in view 1 tells us the 3D point lies somewhere on a ray from camera center C₁ — but not at which depth. Where can its match x₂ be in view 2? Not anywhere: the ray seen from camera 2 is a line. That line is the epipolar line, and finding matches becomes a 1D search instead of a 2D one.
1. The objects
- Baseline — the segment joining the two camera centers C₁C₂.
- Epipoles e₁, e₂ — where the baseline pierces each image; e₂ is the image of C₁ in view 2, and vice versa. They can lie outside the image, or at infinity.
- Epipolar plane — the plane through C₁, C₂ and the 3D point X. Every 3D point gives one; they all contain the baseline (a "book" of pages hinged on the baseline).
- Epipolar lines — each epipolar plane cuts each image in a line. All epipolar lines of one image pass through its epipole.
2. The epipolar constraint
Let camera 2 be related to camera 1 by X₂ = R X₁ + t. The vectors X₂, t and R X₁ lie in the epipolar plane, so
X₂ · (t × R X₁) = 0 ⇔ x̂₂ᵀ [t]× R x̂₁ = 0
with normalized coordinates x̂ = K⁻¹ x. This gives the two matrices:
Essential matrixE = [t]× R, x̂₂ᵀ E x̂₁ = 0
Fundamental matrixF = K₂⁻ᵀ E K₁⁻¹, x₂ᵀ F x₁ = 0
Epipolar linesl₂ = F x₁, l₁ = Fᵀ x₂
EpipolesF e₁ = 0, Fᵀ e₂ = 0
- E needs calibrated cameras (known K). 5 degrees of freedom: 3 rotation + 2 for the translation direction — the scale of t cannot be recovered from images alone.
- F works directly in pixels. 7 degrees of freedom: 3×3 up to scale (8) and det F = 0 (rank 2).
3. Special configurations
- Rectified stereo — same orientation, baseline along camera x. Epipoles at infinity, epipolar lines are horizontal rows: matching is a search along the same row (this is what stereo depth uses).
- Forward motion — camera moves along its optical axis. The epipole is the principal point ("focus of expansion") and epipolar lines radiate from it.
- Pure rotation (t = 0) — E = 0: no epipolar geometry, no depth. The two images are related by a homography instead. Try putting both camera centers at the same place.
4. Things to try
- Hover a house corner in view 1. Watch the white point slide along the ray in 3D and along the epipolar line in view 2. The true match sits on the line.
- Turn on all epipolar lines: they meet at the epipoles. Then switch between the setups.
- Look at the 3D view: the purple epipolar plane cuts the image plane of camera 2 exactly along the epipolar line drawn on it.
- Run Orbit camera 2 and watch the epipoles move. When the baseline becomes parallel to an image plane, that image's epipole runs off to infinity and its epipolar lines become parallel.
Next
F (or E) is estimated from matches with the 8-point algorithm (5-point for E), inside RANSAC. Once the poses are known, matched rays can be intersected — triangulation, in chapter 3.