I have described a stereo matching algorithm that uses (i) edges as features, (ii) dynamic programming technique and (iii) three camera model. Edges are powerful abstractions of image content, and are used to greatly reduces the combinatorics of the correspondence process. They provide higher precision disparity measures than intensity matching techniques \cite{baker}, and through their mutual connectivity, enable explicit use of global information for reducing the ambiguity at matching level.
The line-by-line stereo correspondence problem has been solved by Viterbi technique which successively partitions the problem of searching for the best set of matches into two subproblems, each of which can be processed to yield a global optimum. The correspondence process relies on {\it strong} matches which in turn constrain and simplify computation of {\it weak} matches. The algorithm is shown to be efficient in solving a coarse-to-fine multi-stage problem in a single step using dynamic programming. I have used a new cooperative procedure to enforce edge-connectivity and to detect and correct edge correspondences which violate surface continuity.
The trinocular confirmation algorithm has been defined and shown to be successful in overcoming many of the problems in binocular stereo matching though the use of vertical and horizontal matching with both binocular and trinocular constraints. The algorithm does not rely on an edge being present in all three images in order to match it. This allows the algorithm to match edges using simple local information and still overcome the difficulty of matching edges oriented along epipolar lines. Regions of the base image that are occluded in the right image are matched vertically solely through information from base and vertical images. This provides accurate disparities in partially occluded regions. Similarly disparities are calculated for regions missing in the right image but present in the top image.
The number of unmatched edges in trinocular algorithm is considerably lower than the number for binocular algorithm. The computation time however is more. Dhond and Agrawal, in \cite{dhond}, have indicated that, on average use of a third camera reduces errors by half and increases computation by a fourth. For the example discussed, the computation time was observed to be 50\% higher and the number of unmatched edges to be 80\% lower.
The trinocular support algorithm has been described and it seems to avoid bias towards a horizontal or vertical matching solution. A use of large 4D array makes this algorithm difficult to implement. However, it should be possible to reduce the memory requirement by utilizing the trinocular uniqueness constraint in a more effective way. This algorithm also introduces interline dependencies and thus reduces the potential degree of parallel realization. Future work should focus on finding a solution to these two problems.
The output of an edge-based matcher is a sparse disparity map because the computations are performed only at edge pixels. The performance of the algorithm is also highly dependent on existence of well defined edges in the input images. To improve in these two areas, Baker \cite{baker} and others, e.g. \cite{ohata-yeuchi} have used intensity based matching of edge-delimited intervals on scanline pair. Future work in trinocular algorithm will also involve some type of intensity based matching. However, in case of three camera model, an edge-delimited interval on a horizontal scanline has no obvious corresponding interval in the top image. This makes the job of creating a dense map using confirmation algorithm extremely difficult. The support algorithm, on the other hand keeps track of {\it all} image pixels but computes solution at only edge pixels. The support algorithm should be modified so that the results of edge-based matching can be later used to guide the solution for edge-constrained intervals.
The connectivity constraint is used {\it after} line-by-line correspondence between edges has been established. The cooperative process simply detects and corrects errors in the solution generated by intra-scanline matching. Ohta and Kanade \cite{ohata-yeuchi} indicate that, if the inter-scanline connectivity is not enforced {\it during} matching, the result from the cooperative process is not guaranteed to be optimal. For example, detection and correction of disparities may create new values which violate monotonicity or match edges with opposite contrast. An improved trinocular algorithm should atleast attempt a global left-bottom to top-right ordering of contours (which do not intersect scanlines) in the base image, so that monotonicity constraints will not be violated. Ideally, the algorithm should combine matching and connectivity enforcement into one step \cite{ohata-yeuchi} \cite{raju-binford}. The improvements suggested by Ohta and Kanade \cite{ohata-yeuchi} and Raju and Binford \cite{raju-binford} are applicable only to binocular matching. Future modifications to the proposed trinocular algorithm should try to use such global constraints simultaneously with epipolar, monotonicity and similarity constraints.