Image rectification

Image rectification is a transformation process used to project images onto a common image plane. This process has several degrees of freedom and there are many strategies for transforming images to the common plane. Image rectification is used in computer stereo vision to simplify the problem of finding matching points between images (i.e. the correspondence problem), and in geographic information systems (GIS) to merge images taken from multiple perspectives into a common map coordinate system.

Image rectification is a transformation process used to project images onto a common image plane. This process has several degrees of freedom and there are many strategies for transforming images to the common plane. Image rectification is used in computer stereo vision to simplify the problem of finding matching points between images (i.e. the correspondence problem), and in geographic information systems (GIS) to merge images taken from multiple perspectives into a common map coordinate system. == In computer vision== Computer stereo vision takes two or more images with known relative camera positions that show an object from different viewpoints. For each pixel it then determines the corresponding scene point's depth (i.e. distance from the camera) by first finding matching pixels (i.e. pixels showing the same scene point) in the other image(s) and then applying triangulation to the found matches to determine their depth. Finding matches in stereo vision is restricted by epipolar geometry: Each pixel's match in another image can only be found on a line called the epipolar line. If two images are coplanar, i.e. they were taken such that the right camera is only offset horizontally compared to the left camera (not being moved towards the object or rotated), then each pixel's epipolar line is horizontal and at the same vertical position as that pixel. However, in general settings (the camera does move towards the object or rotate) the epipolar lines are slanted. Image rectification warps both images such that they appear as if they have been taken with only a horizontal displacement and as a consequence all epipolar lines are horizontal, which slightly simplifies the stereo matching process. Note however, that rectification does not fundamentally change the stereo matching process: It searches on lines, slanted ones before and horizontal ones after rectification. Image rectification is also an equivalent (and more often used ) alternative to perfect camera coplanarity. Even with high-precision equipment, image rectification is usually performed because it may be impractical to maintain perfect coplanarity between cameras. Image rectification can only be performed with two images at a time and simultaneous rectification of more than two images is generally impossible. Our model for this example is based on a pair of images that observe a 3D point P, which corresponds to p and p' in the pixel coordinates of each image. O and O' represent the optical centers of each camera, with known camera matrices M=K[I~ 0] and M'=K'[R~ T] (we assume the world origin is at the first camera). We will briefly outline and depict the results for a simple approach to find a H and H' projective transformation that rectify the image pair from the example scene. First, we compute the epipoles, e and e' in each image: e=M \begin{bmatrix} O' \\ 1 \end{bmatrix} =M\begin{bmatrix} -R^T T \\ 1 \end{bmatrix} = K[I~ 0]\begin{bmatrix} -R^T T \\ 1 \end{bmatrix} = -KR^T T e'=M'\begin{bmatrix} O \\ 1 \end{bmatrix} = M'\begin{bmatrix} 0 \\ 1 \end{bmatrix} = K'[R~T]\begin{bmatrix} 0 \\ 1 \end{bmatrix} = K'T Second, we find a projective transformation H 1 that rotates our first image to be parallel to the baseline connecting O and O' (row 2, column 1 of 2D image set). This rotation can be found by using the cross product between the original and the desired optical axes. Next, we find the projective transformation H 2 that takes the rotated image and twists it so that the horizontal axis aligns with the baseline. If calculated correctly, this second transformation should map the e to infinity on the x axis (row 3, column 1 of 2D image set). Finally, define H=H_2H_1 as the projective transformation for rectifying the first image. Third, through an equivalent operation, we can find H' to rectify the second image (column 2 of 2D image set). Note that H' 1 should rotate the second image's optical axis to be parallel with the transformed optical axis of the first image. One strategy is to pick a plane parallel to the line where the two original optical axes intersect to minimize distortion from the reprojection process. In this example, we simply define H' using the rotation matrix R and initial projective transformation H as H' = HR^T . Finally, we scale both images to the same approximate resolution and align the now horizontal epipoles for easier horizontal scanning for correspondences (row 4 of 2D image set). Note that it is possible to perform this and similar algorithms without having the camera parameter matrices M and M' . All that is required is a set of seven or more image to image correspondences to compute the fundamental matrices and epipoles. ==See also== Binocular disparity Correspondence problem Epipolar geometry Geographic information system Georeferencing Homography Image registration Photogrammetry Point set registration Rubbersheeting Software correction of lens distortion Stereo camera Stereo vision Structure from motion ==References== ==Further reading== Computing Rectifying Homographies for Stereo Vision by Charles Loop and Zhengyou Zhang (April 8, 1999) Microsoft Research Computer Vision: Algorithms and Applications, Section 11.1.1 "Rectification" by Richard Szeliski (September 3, 2010) Springer

Text derived from https://en.wikipedia.org/w/index.php?oldid=1376892664 under CC BY-SA 4.0; modifications have been made.