ansaurus

Question

Answer 1

+1 A:

It may be that your points are in a degenerate configuration. Try to add a couple of points from the scene that don't belong to the cube and see how it goes.

jmbr 2010-03-31 12:22:24

Added about another 9 points not on the cube. Still the cube looks flat, although the newly added points vary in their Z values relative to the cube. However, the Z values themselves do not scale with the X and Y values (z values are about 10^3 smaller than X or Y values).

srand 2010-03-31 17:49:07

Have you tried using the new points (and these points only) to compute the fundamental matrix? If that works then you could replace non-cube points with cube points, recompute F, and see how this affects the reconstruction. I insist on this approach because I've been bitten in the past by the degeneracy of points on the cube while debugging calibration algorithms. Hope this helps.

jmbr 2010-03-31 19:07:02

Hey jmbr, what do you think about subpixel correspondence errors in triangulation?

Jacob 2010-03-31 19:10:08

Hi Jacob, see my comment in the thread corresponding to your answer.The high skewness bothers me too.

jmbr 2010-03-31 19:15:00

Answer 2

+1 A:

More information required:

What is t? The baseline might be too small for parallax.
What is the disparity between x1 and x2?
Are you confident about the accuracy of the calibration (I'm assuming you used the Stereo part of the Bouguet Toolbox)?
When you say the correspondences are hand-picked, do you mean you selected the corresponding points on the image or did you use an interest point detector on the two images are then set the correspondences?

I'm sure we can resolve this problem :)

Jacob 2010-03-31 18:00:12

t = [-0.8754; 0.2249; -0.4279]. image points differ an average of: x 65px, y 89px. Fairly confident, updated the question with my calibration information, error is about 0.2 pixels.

srand 2010-03-31 18:22:58

What units is that? Could you post *K*?

Jacob 2010-03-31 18:23:53

In fact, could you post *K*,*R*,*t* and as many `x1` and `x2`?

Jacob 2010-03-31 18:24:43

Ah thanks, didn't see the calib info while commenting!

Jacob 2010-03-31 18:26:24

*Why* is the skew 1.2584? It's normally (and is often assumed to be) **0**. Are you using a normal camera?

Jacob 2010-03-31 18:30:48

Poor quality camera? :) Using Microsoft LifeCam NX-6000 Webcam, K = [fc(1), alpha_c*fc(1), cc(1); 0, fc(2), cc(2); 0, 0, 1]. I tried it with alpha_c = 0... still the same results

srand 2010-03-31 18:33:22

Hmm .. I see your problem, are these faces supposed to be orthogonal?

Jacob 2010-03-31 18:44:51

Yes, the faces should be orthogonal

srand 2010-03-31 18:45:52

What's your baseline, i .e. |t|? I see you've normalized it so what's the actual length?

Jacob 2010-03-31 18:48:17

I'm not sure. How do I compute the baseline? And I didn't know it was normalized :)

srand 2010-03-31 18:50:31

What's the distance between the two cameras in your stereo images?

Jacob 2010-03-31 18:51:36

They were moved by hand (two snapshots), so I don't have their exact distance of change

srand 2010-03-31 18:53:47

Try increasing the baseline, I'll keep you updated if I think of anything

Jacob 2010-03-31 18:54:56

Doesn't K*[R,t] take care of the normalization?

srand 2010-03-31 18:55:26

I was referring to *t* being normalized thus not giving me an idea of the length of the baseline.

Jacob 2010-03-31 19:01:16

Also, how did you get these correspondences?

Jacob 2010-03-31 19:04:02

The correspondences were hand picked

srand 2010-03-31 19:04:43

As in, you clicked them on the screen? Hmmm .. maybe you should use an interest point detector like SIFT (try the VL-feat toolbox) - subpixel estimation is important in triangulation.

Jacob 2010-03-31 19:07:53

If he clicked them on the screen using some tool like cpselect (which is part of the Image Processing Toolbox) using sub-pixel accuracy then it's fine.

jmbr 2010-03-31 19:14:05

I've tried VLFeat's SIFT library but there are some false positives that get through my RANSAC algorithm, seems more trouble than its work for now. I do use the optimal triangulation method (12.1 in Hartley/Zisserman's book), so that might compensate for some inaccuracies in my hand picked results..

srand 2010-03-31 19:14:46

@jmbr, I just read them off the screen using a paint application. If accuracy is that important, I'll take a look at cpselect.

srand 2010-03-31 19:16:27

If you want to handpick them, do zoom up the section so you can click them as accurately as possible.

Jacob 2010-03-31 19:16:48

I've done manual correspondence matching in the past and it's OK as long as you're careful and use optimization afterwards.

jmbr 2010-03-31 19:16:52

Zooming on the image, which is something that cpselect allows you to easily do.

jmbr 2010-03-31 19:17:13

I would suggest using VL-feat to generate the interest points and then hand-picking the correspondences. Then you get the benefit of subpixel corners! But if not, do zoom on the patches in question.

Jacob 2010-03-31 19:18:30

So did subpixel corners and/or increasing the baseline work?

Jacob 2010-04-01 14:26:55

So, I made an observation. The Microsoft LifeCam NX-6000 Webcam has a 2 megapixel (1.3 effective??) CMOS sensor. My calibration images are 800 x 600 pixels. However, a 2MP camera can take 1600 x 1200 pixel images. Does this affect the calibration results if everything is kept at 800x600 ?

srand 2010-04-02 19:31:00

Also, I gather a new set of points from new images. This time, I picked points on 3 sides of the cube (top, left and right). Again, X and Y look good but the Z is near zero (thus making the 3d pts look flat). Maybe the calibration is bad?

srand 2010-04-02 19:39:29

Also, sub-pixel accuracy shouldn't be necessary because the X and Y are obviously (visually) accurate. The lack of accuracy shouldn't only affect the Z axis

srand 2010-04-02 19:54:08

Setting the skew to zero doesn't affect the reconstruction by much and still exhibits the same problem

srand 2010-04-05 17:42:57

Did you obtain points with subpixel accuracy using `cpselect`?

Jacob 2010-04-05 18:06:46

I used Paint and the zoom tool :) cpselect doesn't seem to guide the matching, unless I missed something

srand 2010-04-05 19:16:43

`cpselect` allows you to zoom and click points on two images resulting in vectors representing the correspondences- you have to decide the matching.

Jacob 2010-04-05 19:41:10

I'm still getting the same results, is there anything else that could be the culprit ?

srand 2010-04-06 15:23:09

Could you post the new points selected with cpselect?

Jacob 2010-04-06 17:22:49

Updated with the new cpselect points

srand 2010-04-06 19:10:39

So, when I triangulate I use the essential matrix rather than the fundamental matrix. Also I multiply every image correspondences in the triangulation algorithm by K^-1. My projection matrix for P2 is [R, t];. Now I get a varying z value, and the reconstruction looks semi-decent. What is going on? I thought the optimal triangulation method (hartley/zisserman 12.1) required the F not E, additional has no mention of multiplying the image correspondences by K^-1.

srand 2010-04-14 18:18:04

Answer 3

A:

** Note all reference are to Multiple View Geometry in Computer Vision by Hartley and Zisserman.

OK, so there were a couple bugs:

When computing the essential matrix (p. 257-259) the author mentions the correct R,t pair from the set of four R,t (Result 9.19) is the one where the 3D points lay in front of both cameras (Fig. 9.12, a) but doesn't mention how one computes this. By chance I was re-reading chapter 6 and discovered that 6.2.3 (p.162) discusses depth of points and Result 6.1 is the equation needed to be applied to get the correct R and t.
In my implementation of the optimal triangulation method (Algorithm 12.1 (p.318)) in step 2 I had T2^-1' * F * T1^-1 where I needed to have (T2^-1)' * F * T1^-1. The former translates the -1.I wanted, and in the latter, to translate the inverted the T2 matrix (foiled again by MATLAB!).
Finally, I wasn't computing P1 correctly, it should have been P1 = K * [eye(3),zeros(3,1)];. I forgot to multiple by the calibration matrix K.

Hope this helps future passerby's !

srand 2010-04-16 17:34:19

Also thanks Jacob and jmbr :)

srand 2010-04-16 17:37:08

ansaurus

tags:

views:

answers:

Metric 3d reconstruction

related questions