img=imread('img27.jpg');
%function rectangle=rect_test(img)
% edge detection
[gx,gy]=gradient(img); 
gx=abs(gx);
 gy=abs(gy);
g=gx+gy;
g=abs(g);
% make it 300x300
img=zeros(300);
s=size(g);
img(1:s(1),1:s(2))=g;
figure; 
imagesc((img));
 colormap(gray);
 title('Edge detection')
% take the dct of the image
IM=abs(dct2(img));
figure;
 imagesc((IM));
 colormap(gray); 
title('DCT')
% normalize
m=max(max(IM)); IM2=IM./m*100;
% get rid of the peak
size_IM2=size(IM2);
IM2(1:round(.05*size_IM2(1)),1:round(.05*size_IM2(2))) = 0;
% threshold
L=length( find(IM2>20)) ;
if( L > 60 )
   ret = 1;
else
   ret = 0;
end
views:
51answers:
1
                +2 
                A: 
                
                
              I guess that the function doesn't work if you uncomment the line function rectangle=rect_test(img). That is most likely because in your function, the variable rectangle is never defined. Probably you want to write ret=rect_test(img) instead.
                  Jonas
                   2010-06-08 03:18:27
                
              That would probably do it. When debugging, try not to turn functions into scripts. That can become wildly confusing.
                  Steve
                   2010-06-08 03:30:53