Performance of find() vs. for loop

Question

I've got a large (4000 values) set of unsorted, normally distributed points. I'm taking each of these data points into bins whose limits are in the BinLimit array. Then I'm tabulating the number of values in each bin.

X is the array of raw data, and N is the number of data points. The number of bins desired (TotalBins) is specified by the user.

Method #1

for i=1:TotalBins
    Freq(i) = length(find(X >= BinLimit(j) & X <= BinLimit(j+1)));
    j = j + 1;
end

Method #2:

sort(X);

for i=1:N
    if (X(i) >= BinLimit(j) && X(i) <= BinLimit(j+1))
        Freq(j) = freq(j) + 1;
    elseif (j < TotalBins)
        Freq(j+1) = freq(j+1) + 1;
        j = j + 1;
    end
end

Now, I know that the first solution is slower - for normal values of Total Bins (25-50) it's about 8 times slower but I'm wondering if there's a faster, more efficient solution than what I'm doing in Method #2.

Also, this is homework, but this is really just out of curiosity as either of the above solutions do the job - it's a chem class.

Answer 1

Use histc.

N = HISTC(X,EDGES), for vector X, counts the number of values in X that fall between the elements in the EDGES vector (which must contain monotonically non-decreasing values). N is a LENGTH(EDGES) vector containing these counts.

N(k) will count the value X(i) if EDGES(k) <= X(i) < EDGES(k+1). The last bin will count any values of X that match EDGES(end). Values outside the values in EDGES are not counted. Use -inf and inf in EDGES to include all non-NaN values.

E.g.

BinLimit = sort(rand(50,1));
X = rand(4000,1);
Freq = histc(X,BinLimit);

For fun:

%%# Generating data
X = rand(1000000,1);
BinLimit = sort(rand(50,1));

%%# OP's method
disp('Method #1');
disp('=========');
tic;
j =1;
TotalBins = numel(BinLimit)-1;
for i=1:TotalBins
    Freq(i) = length(find(X >= BinLimit(j) & X <= BinLimit(j+1)));
    j = j + 1;
end
toc

%%# histc
disp('histc');
disp('=====');
tic;
histc(X,BinLimit);
toc

%%# My method
disp('Alternative');
disp('===========');
tic;
TotalBins = numel(BinLimit)-1;
Freq = zeros(TotalBins,1);
for i = 1:TotalBins
    Freq(i) = sum(X >= BinLimit(i) & X <= BinLimit(i+1));
end
toc

After a few runs to warm it up:

Method #1
=========
Elapsed time is 0.803120 seconds.
histc
=====
Elapsed time is 0.030633 seconds.
Alternative
===========
Elapsed time is 0.704808 seconds.

Answer 2

Apart from using HISTC, here's a vectorized one-line solution:

X = randn(4000,1);                %# column vector
BinLimits = linspace(-4,4,10);    %# row vector

Freq = sum( bsxfun(@ge, X, BinLimits(1:end-1)) & bsxfun(@le, X, BinLimits(2:end)) )

Note that its not space-efficient though, as it create a matrix of size: length(X) by length(BinLimits)-1