5.4 Experimental results
5.4.1 Comparing the performance of the two normal models
In order to compare the two algorithms for normal estimation we use two example images. The first example is the corridorand the second one, the headimages.
We first begin our discussion with thecorridorimage (courtesy University of Bonn) (fig- ure5.6(a)). This stereo pair consists of images of the size256with the disparity range[0,11].
While the ground-truth disparity is available (figure 5.6(b)), the normal “ground-truth” is generated using thesurfnormfunction in MATLAB. Thesurfnormfunction computes sur- face normals for the surface defined by any X,Y, and Z, in our case these correspond the pixel coordinates forXandY, andZcorresponds to the ground-truth disparity values. This function determines the surface normals based on a cross product of the tangent vectors.
The tangent vectors are determined using the first difference between the neighbouring posi- tions. In other words, tangent vectors are the gradients computed in thexandy-directions.
As these gradients are not defined at discontinuities, we slightly smooth the ground-truth disparities with a Gaussian filter (with a standard deviation of 1.0). We use the normals, shown in figure 5.6(e), computed using this function as groundtruth for surface normals.
In figures. 5.6(c) and 5.6(d) we show the disparity maps obtained using Mean Field optimization and each of the two methods of normal estimation within alternation, namely ICM and BP respectively. We see that the disparity map obtained alternating with ICM- based normal estimation is much smoother compared to the one obtained using BP-based normal estimation. We further show the results of the normals obtained using ICM and BP as an arrow diagram in the the two figures 5.6(f)and 5.6(g). We see that the normals obtained using the ICM procedure are noisier compared to that of BP. However, it is difficult to compare the arrow diagrams as we do not completely see the difference in two results.
We therefore convert the two normals into colour coded normal images.
We use an HSV colour-code4 to represent the normals. The colour coding is done as follows: Letnx= (nu, nv, nd) for x= (u, v) and disparity at the location dx=d. Now the colour code is obtained by mapping the azimuthθ and elevationφof each normal to hue H and saturation S, respectively, and setting the value V to1. The azimuthθand elevationφ are computed in the standard way:
θ=π+ arctannv
nu
φ= arccosnd (5.34)
The range colours obtained using such a technique can be visualized by normals on a sphere. figure 5.7(a) shows spherical disparity surface. The normals obtained using surfnormfunction in MATLAB is shown in figure5.7(b)as an arrow diagram. The different directions associated with the sphere can be seen better on the flattened arrow map in figure 5.8(a). The figure5.8(b) shows the range of colours obtained using the equation (5.34). It can be seen that the fronto-parallel normal is represented as white (middle of the sphere).
4. Note that the same colour code is used for all the other normal-maps
(a) Original Image of size 256× 256
(b) Disparity Groundtruth (c) Disparity estimated using Normals in figure5.6(f)(below)
(d) Disparity estimated using Normals in figure5.6(g)(below)
(e) Normals Groundtruth (f) Normal Estimation using ICM
(g) Normal Estimation using BP
Figure 5.6: Disparity and Normals obtained for thecorridorimage using the ICM and BP for Normal estimation
0 10
20 30
40 50
60
0 10 20 30 40 50 60 40 20 30
ima ge x-axisu imag e y-axisv
disparityd
(a) Disparity Surface
0 10
20 30
40 50
60
0 10 20 30 40 50 60 40 20 30
image x-axisu image y-axisv
disparityd
(b) Normals as arrow map
Figure 5.7: Example showing the normals for a spherical surface
(a) Flattened arrow map of figure5.7(b)
(b) Colour Coded normals for figure5.7(b)
Figure 5.8: Example showing the normals as colour for the spherical surface
(a) Normals Groundtruth (b) Normals Initialization through plane-fitting
(c) ICM Normals (d) BP Normals
Figure 5.9: Comparing the normals obtained for the corridor image using the two ap- proaches to groundtruth. The normals are colour coded into HSV colours using (5.34). The highlighted regions in5.9(d) shows how the normals obtained using BP follow the plane-fit solution in 5.9(b).
Now, using such a colour code for the normals in figures 5.6(e), 5.6(g) and 5.6(f), we compare the results obtained by BP and ICM procedures. We see that even though the normals obtained using the BP procedure are not noisy they have large errors, especially at the highlighted regions of the figure5.9(d), as compared to the ground-truth in figure5.6(e).
Comparing the results of the normals obtained using BP figure 5.9(d) to its initialization obtained using segmentation/plane-fit figure5.9(b), we see that the bias towards the plane fit solution is quite large. But this is natural because as the image is mainly made of planar surfaces the final solution tends to stay close to the plane-fit normals. We see that the normals obtained using ICM based approach (figure 5.9(c)) better approximates the groundtruth (figure 5.9(a)) as compared to the BP based solution.
We now compare results obtained on much smoother surface like theheadimage shown in
(a) Original Image of size219×255 (b) Disparity estimated using Nor- mals in figure5.11(c)
(c) Disparity estimated using Nor- mals in figure5.11(d)
Figure 5.10: Disparity estimated using Normals from BP and ICM
figure5.10(a)(courtesy of University of Manchester and University of Sheffield ). This image is of size219×255and has a disparity range of[−30,10]. The figure5.10shows the results obtained using the normals obtained from ICM and BP optimizations. In the absence of groundtruth, disparity results obtained using the two solutions seems satisfactory. However, comparing the results of the normals using ICM and BP (figure5.11(c)and5.11(d)), we see that the results obtained using the BP approach do not conform to the surface variations.
This discrepancy is mainly due to the discretization of the normal space. In absence of dense discretization the final solution either tends to the closest normal in discrete-normal space.
As a result, if the surface is close to being fronto-parallel most solutions will tend to fronto- parallel solutions, nx= (0,0,1)indicated by white in the figure5.11(d). But increasing the discretization of the normal space makes the procedure computationally inefficient. Already, with K = 162 levels of discretization the BP procedure takes 60 seconds for a 32×32 image, as compared to ICM procedure which takes1 second. Furthermore, this estimation procedure also depends on the size of the regions obtained during segmentation. This is because a certain minimum region size is required in order for the RANSAC procedure (used for plane-fitting in our approach) to provide a good plane representation of the region, and thus the initialization for the normal optimization. Due to these limitations of the discrete BP approach, we now concentrate on the ICM-based continuous normal estimation procedure.