I think you meant the goal to be to detect lines in an image not comparing two images??
Anyway, to find the maximum intensities in the Hough transform matrix generated by the hough function, we use the houghpeaks function, and pass it the desired number of peaks to detect.
I figured I add an example to show the procedure:
%# load image, process it, find edges
I = rgb2gray( imread('pillsetc.png') );
I = imcrop(I, [30 30 450 350]);
J = imfilter(I, fspecial('gaussian', [17 17], 5), 'symmetric');
BW = edge(J, 'canny');
%# Hough Transform and show matrix
[H T R] = hough(BW);
imshow(imadjust(mat2gray(H)), [], 'XData',T, 'YData',R, ...
'InitialMagnification','fit')
xlabel('\theta (degrees)'), ylabel('\rho')
axis on, axis normal, hold on
colormap(hot), colorbar
%# detect peaks
P = houghpeaks(H, 4);
plot(T(P(:,2)), R(P(:,1)), 'gs', 'LineWidth',2);
%# detect lines and overlay on top of image
lines = houghlines(BW, T, R, P);
figure, imshow(I), hold on
for k = 1:length(lines)
xy = [lines(k).point1; lines(k).point2];
plot(xy(:,1), xy(:,2), 'g.-', 'LineWidth',2);
end
hold off
EDIT2:
Following your recent update, I managed to detect the lines by only making a few changes to the same above code:
- I cropped the region:
[200 70 160 140]
- used an 11x11 gaussian filter with sigma=3
note you will have to add the offset to get the position of the lines in the original image uncropped. Also if you want more accurate results, you might want to detect 4 lines and get the lines in the middle as shown below: