學習筆記之基於代價聚合算法的分類 ，主要針對cost aggregration分類，2008 CVPR
1. Introduction
經典的全局算法有：
BP[31]
DP[28]
SO[20]
本文主要內容有：從精度的角度對比各個算法，主要基於文獻【23】給出的評估方法，同時也在計算複雜度上進行了比較，最後綜合這兩方面提出一個trade-off的比較。2 Classification of cost aggregation strategies
主要分爲兩種：
1)The former generalizes the concept of variable support by allowing the support to have any shape instead of being built upon rectangular windows only.
2) The latter assigns adaptive - rather than fixed - weights to the points belonging to the support.
大部分的代價聚合都是採用symmetric方案，也就是綜合兩幅圖的信息。（實際上在後面的博客中也可以發現，不一定要採用symmetric的形式，而可以採用asymmetric+TAC的形式，效果反而更好）。
採用的匹配函數爲（matching (or error) function ）：Lp distance between two vectors
包括 SAD、Truncated SAD [30,25]、SSD、M-estimator [12]、similarity function based on point distinctiveness[32]
最後要指出的是，本文基於平行平面（fronto-parallel）support。
2.1 基於矩形窗的代價聚合（Cost aggregation based on rectangular windows）
Let Ir and It be respectively the reference and target image of a rectified stereo pair.
2.1.1 varying window size
and/or offset
1) shiftable windows [23]
窗定義爲：
n：窗size
在depth borders 邊界處有效
變形：【4,10】只選9個窗
2） vary the size of the window rather than its displacement by properly selectig n between Nmin ~Nmax
This allows, e.g., to employ bigger windows within low-textured regions.
[17] the best displacement is selected by means of a shiftable window
approach, while, to determine the size of the support, starting from n = Nmin the window is iteratively enlarged until a given minimum variance of the error function is reached.
[27] which selects as support the window minimizing a matching cost over a set of windows
[7] the best displacement is selected out of 9 using the shiftable windows approach
matching cost 選擇的3個標準：
- error function 的最小化
- error function 方差的最小化
- 加偏移的biased weights 以支持在弱紋理區域的更大窗口的選擇
2.1.2 selecting more than
one window
選擇一系列的窗
[22] a variable support strategy
[15] S(p, q) is defined as a set of 5 squared windows
每次選擇3個最佳的，其中一個是以p爲中心，另外兩個是around p的4個窗裏面選出2個。
類似的有 13選5,25選9
不限於矩形窗
2.1.3 Associating different weights to window points
concerns the explicit assignment of different weights to the points of each window belonging to S(p, q).
【18】the aggregation stage defines S(p, q) as a set of 108 rod-shaped windows. Each point is then classified as homogeneous or heterogeneous based on the outcome of the application of a LoG filter.
【13】S(p, q) is defined as a set of 5 × 5 window pairs centered on (p, q), each window point being characterized by a weight belonging to the set {0, 1, 2, 4}.
2.2 Cost aggregation based on unconstrained shapes
不再限於矩形窗，而可以是任意形狀。
首先提出這個思想的是【5】
[26] the support shape at each correspondence is represented by a polygonal line around p, which is extracted by applying the minimum ratio cycle technique.
[12] support shape is represented by the intersection between the segment on which p lies, Gp, and a squared window centered on p, wrn (x, y).
2.3 Use
of adaptive weights
the assignment of different and variable weights to the points surrounding p and q.
只包括only a subset of points represented by a squared or a round window centered on p and q
【29】3 different cues are deployed to determine the support weights
for points belonging to the reference view Ir. The first one (the certainty) is based on the variance
of the error function: With increasing variances, the
assigned weight is lower since it corresponds to a low certainty. The two other cues are color and disparity distribution
correlation: the weight assigned to a point pi increases as the difference in the color space between pi and p decreases and
as the correlation between pi and p disparity distribution increases. Each cue is weighted by means of a gaussian function in the
final weight formulation, the 3 gaussian variances being 3 parameters of the method.
Evaluation
1) approaches based on a selection over a set of windows
Shiftable window [4],(1999)
Reliability [17] 2001
Variable windows [27] 2003
Recursive adaptive [7] 2003
Multiple adaptive [9] 2005
Multiple windows [15] 2002
Oriented rod [18] 2005
Gradient guided [13]. 2005
2) unconstrained support shapes
Max connected [5] 1998
Segmentation based [12].2006
Segmentation based [12].2006
3) adaptive weights
Radial adaptive [29]2002
Adaptive weight [30] code
2006
Segment support [25]. 2007
Reference
[4] A. Bobick and S. Intille. Large occlusion stereo. IJCV,
33(3):181–200, 1999.
[5] Y. Boykov, O. Veksler, and R. Zabih. A variable window
approach to early vision. IEEE Trans. PAMI, 20(12):1283–
1294, 1998.
[6] M. Z. Brown, D. Burschka, and G. D. Hager. Advances in
computational stereo. IEEE Trans. PAMI, 25(8):993–1008,
2003.
[7] S. Chan, Y. Wong, and J. Daniel. Dense stereo correspondence
based on recursive adaptive size multi-windowing. In
Proc. Image and Vision Computing New Zealand, volume 1,
pages 256–260, 2003.
[8] T. Darrel. A radial cumulative similarity transform for robust
image correspondence. In Proc. Conf. Computer Vision and
Pattern Recognition, pages 656–662, 1998.
[9] C. Demoulin and M. Van Droogenbroeck. A method based
on multiple adaptive windows to improve the determination
of disparity maps. In Proc. IEEE Workshop on Circuit, Systems
and Signal Processing, pages 615–618, 2005.
[10]A. Fusiello, V. Roberto, and E. Trucco. Symmetric stereo
with multiple windowing. Int. Journal of Pattern Recognition
and Artificial Intelligence, 14(8):1053–1066, 2000. 2
[11] D. Geiger, B. Ladendorf, and A. Yuille. Occlusions and
binocular stereo. IJCV, 14(3):211–226, 1995. 1
[12] M. Gerrits and P. Bekaert. Local stereo matching with
segmentation-based outlier rejection. In Proc. Conf. Computer
and Robot Vision, pages 66–66, 2006.
[13] M. Gong and R. Yang. Image-gradient-guided real-time
stereo on graphics hardware. In Proc. Conf. 3D Digital Imaging
and Modeling (3DIM), pages 548–555, 2005.
[14] M. Gong, R. Yang, W. Liang, and M. Gong. A performance
study on different cost aggregation approaches used in realtime
stereo matching. IJCV, 75(2):283–296, 2007.
[15] H. Hirschmuller, P. Innocent, and J. Garibaldi. Real-time
correlation-based stereo vision with reduced border errors.
IJCV, 47:1–3, 2002.
[16] H. Hirschmuller and D. Scharstein. Evaluation of cost functions
for stereo matching. In Proc. Conf. Computer Vision
and Pattern Recognition, volume 1, pages 1–8, 2007. 1
[17] S. Kang, R. Szeliski, and J. Chai. Handling occlusions in
dense multi-view stereo. In Proc. Conf. Computer Vision
and Pattern Recognition, pages 103–110, 2001. 2, 4, 6
[18] J. Kim, K. Lee, B. Choi, and S. Lee. A dense stereo matching
using two-pass dynamic programming with generalized
ground control points. In Proc. Conf. Computer Vision and
Pattern Recognition, pages 1075–1082, 2005. 3, 4, 6
[19] M. Levine, D. O’Handley, and G. Yagi. Computer determination
of depth maps. Computer Graphics and Image Processing,
2:131–150, 1973. 1
[20] S. Mattoccia, F. Tombari, and L. Di Stefano. Stereo vision
enabling precise border localization within a scanline optimization
framework. In Proc. Asian Conf. on Computer Vision,
pages 517–527, 2007. 1
[21] M. Okutomi and T. Kanade. A locally adaptive window for
signal matching. IJCV, 7(2):143–162, 1992. 1, 2
[22] M. Okutomi, Y. Katayama, and S. Oka. A simple stereo
algorithm to recover precise object boundaries and smooth
surfaces. IJCV, 47(1-3):261–273, 2002. 3, 7
D. Scharstein and R. Szeliski. A taxonomy and evaluation of
dense two-frame stereo correspondence algorithms. IJCV,
47(1/2/3):7–42, 2002. 1, 2, 4
[24] H. Tao, H. Sawheny, and R. Kumar. A global matching
framework for stereo computation. In Proc. Int. Conf. Computer
Vision, volume 1, pages 532–539, 2001. 3
[25] F. Tombari, S. Mattoccia, and L. Di Stefano. Segmentationbased
adaptive support for accurate stereo correspondence.
In Proc. Pacific-Rim Symposium on Image and Video Technology,
2007. 2, 4, 5, 6
[26] O. Veksler. Stereo matching by compact windows via minimum
ratio cycle. In Proc. Int. Conf. Computer Vision, volume
1, pages 540–547, 2001. 3, 7
[27] O. Veksler. Fast variable window for stereo correspondence
using integral images. In Proc. Conf. Computer Vision and
Pattern Recognition, pages 556–561, 2003. 2, 4, 5, 6
[28] L. Wang, M. Liao, M. Gong, R. Yang, and D. Nister. Highquality
real-time stereo using adaptive cost aggregation and
dynamic programming. In Proc. Int. Symp. 3D Data Proc.,
Vis. and Transm. (3DPVT), pages 798–805, 2006. 1, 4
[29] Y. Xu, D. Wang, T. Feng, and H. Shum. Stereo computation
using radial adaptive windows. In Int. Conf. Pattern Recognition,
volume 3, pages 595– 598, 2002. 4, 5, 6
[30] K. Yoon and I. Kweon. Adaptive support-weight approach
for correspondence search. IEEE Trans. PAMI, 28(4):650–
656, 2006. 2, 4, 5, 6 (code!!!)
[31] K. Yoon and I. Kweon. Stereo matching with symmetric
cost functions. In Proc. Conf. Computer Vision and Pattern
Recognition, volume 2, pages 2371 – 2377, 2006. 1
[32] K. Yoon and I. Kweon. Stereo matching with the distinctive
similarity measure. In Proc. Int. Conf. Computer Vision
(ICCV’07), 2007. 2
[33] C. Zitnick and T. Kanade. A cooperative algorithm for
stereo matching and occlusion detection. IEEE Trans. PAMI,
22(7):675–684, 2000. 2
