views:

224

answers:

8

is there an algorithm that is faster than binary search, for searching in sorted values of array?

in my case, I have a sorted values (could be any type values) in an A array, I need to return n if the value I was looking is in range of A[n] and A[n+1]

+1  A: 

Yes and no. Yes there are searches that are faster, on average, than a bisection search. But I believe that they are still O(lg N), just with a lower constant.

You want to minimize the time taken to find your element. Generally it is desirable to use fewer steps, and one way to approach this is to maximize the expected number of elements that will be eliminated at each step. With bisection, always exactly half the elements are eliminated. You can do better than this, IF you know something about the distribution of the elements. But, the algorithm for choosing the partition element is generally more complicated than choosing the midpoint, and this extra complexity may overwhelm any time savings you expected to get from using fewer steps.

Really, in a problem like this it's better to attack second-order effects like cache locality, than the search algorithm. For example, when doing a repeated binary search, the same few elements (first, second, and third quartiles) are used VERY frequently, so putting them in a single cache line could be far superior to random access into the list.

Dividing each level into say 4 or 8 equal sections (instead of 2) and doing a linear search through those could also be quicker than the bisection search, because a linear search doesn't require calculating the partition and also has fewer data dependencies that can cause cache stalls.

But all of these are still O(lg N).

Ben Voigt
On a single ordered list, no. But there are much faster searches; you just need a different data structure than an ordered list. A hash would be virtually constant in lookup time, at a cost of a great deal more memory. A hybrid approach could take the approach of a dictionary.
tchrist
@tchrist: The problem requires finding the pair of elements that tightly bound a sought entry that is not in the list at all. Hashing only finds exact matches.
Ben Voigt
Whoops, you're right. Somehow I only read the first sentence, not the second one.
tchrist
A: 

You can always put them in a hash table, then search will be O(1). It will be memory intensive though and if you keep adding items, the hash table might need to be re-bucketed. Re-bucketing is O(n) but it will get amortized to O(1). It essentially depends on whether you can afford that space and the potential cache misses.

srean
It's possible that his array does not contain the value n, but does contain two values that bracket n. It's not obvious that hashing is applicable here.
xscott
Oh I missed that. But you could still hash first and fall back to binary search if the value is not in the key set. But this is an added complexity. In general you cannot do better than the entropy of the distribution of the values. If you knew the distribution you can use a Huffman tree to decide where you partition.
srean
+3  A: 

If the values in the list are evenly distributed then you could try a weighted split instead of a binary split, e.g. if the desired value is a third of the way from the current lower limit to the current value then you could try the element that is also a third of the way. This could suffer badly on lists where values are bunched up though.

Ignacio Vazquez-Abrams
Some more optimization is necessary. You don't want to choose the element closest to where you guess the answer to be, you want to test a point between the guessed location and the center of list, so that with p > .5 you eliminate more than half the list. The exact optimal partition point depends on the distribution of values in the list.
Ben Voigt
@Ignacio What you describe is exactly interpolation search. @Ben an efficient way to implement your suggestion is through a Huffman tree
srean
+1  A: 

One possibility is to treat it like finding the roots of a function. Basically, finding:

a[i] <= i <= a[i + 1]

Is equivalent to:

a[i] - i <= 0 <= a[i + 1] - i

Then you could try something like Newton's method and so on. These kinds of algorithms frequently converge faster than a binary search when they work, but I don't know of one that is guaranteed to converge for all input.

http://en.wikipedia.org/wiki/Root-finding_algorithm

xscott
Newton's method requires a differentiable function, so one would have to fit an interpolating spline first. If the values are uni-modal its quite well behaved else it might diverge and act totally bizarre.
srean
Yes. You can use a linear spline, and the derivative at any point is: f'(i) = a[i+1] - a[i]
xscott
Linear splines are piecewise linear, so its derivative wont be continuous. One has to go for quadratic atleast. Which is no biggie. This will turn out to be similar to [http://en.wikipedia.org/wiki/Interpolation_search]
srean
I don't believe the derivative needs to be continuous in Newton's method. Thanks for the link on interpolation search.
xscott
@xscott Just to calrify by "this" I meant your suggestion of using linear interpolation.
srean
A: 

In binary search you split the list into two "sublists" and you only search the sublist that may contain the value. Depending on how large your array is, you could see a speedup if you split the array into more than two splices.

You can determine which region of the array you have to search, by keeping an index, that you search first. Like in a telephone book of a large city, where you can see from the outside, where you have to start to search. (I have trouble expressing my idea in text, and I did not find an english link yet that explains it better).

bjoernz
+4  A: 

You can do better than O(log n) if the values are integers, in which case the best worst-case running time you can achieve, in terms of n, is O(sqrt(log n)). Otherwise, there is no way to beat O(log n) unless there are patterns in the input sequence. There are two approaches used to beat O(log n) in the case of integers.

First, you can use y-fast trees which work by storing in a hash table all prefixes for which you are storing at least one integer with that prefix. This enables you to perform a binary search to find the length of the longest matching prefix. This enables you to find the successor of an element for which you are searching in time O(log w) where w is the number of bits in a word. There are some details to work though to make this work and use only linear space, but they aren't too bad (see the link below).

Second, you can use fusion trees, which use bit tricks to enable you to perform w^O(1) comparisons in just a constant number of instructions, yielding a running time of O(log n / log w).

The optimum tradeoff between these two data structures occurs when log w = sqrt(log n), giving a running time of O(sqrt(log n)).

For details on the above, see lectures 12 and 13 of Erik Demaine's course: http://courses.csail.mit.edu/6.851/spring07/lec.html

jonderry
I'd like to know more about fusion trees. Maybe you'd be willing to explaing them: http://stackoverflow.com/questions/3878320/understanding-fusion-trees
xscott
+1  A: 

First of all, measure before doing optimization.

Do you really need to optimize that search?

If so, then secondly, think about algorithmic complexity first. E.g. can you use a tree (like a std::map, say) instead of an array? If so then it depends on the relative frequency of insertions/deletions versus searches, but the premise of having a sorted array at hand indicates that searches are frequent compared to data set changes, so that it would make sense to do some little additional work for insertions/deletions, making each search much faster -- namely logarithmic time.

If you find that indeed the search times are a bottleneck that needs addressing, and no, no change of data representation is possible, and the list is short, then a linear search will generally be faster because it does less work per comparision.

Otherwise, if the list is longer, and no particular distribution of values is known or assumed, and the values can't be treated as numerical, and memory consumption should be constant (ruling out constructing a hash table, say), then binary search produces 1 bit of information per comparision and is probably the best you can do for the first search.

Cheers & hth.

Alf P. Steinbach
A: 

If you have a huge amount of numbers to find, and by some fluke they are ALSO sorted, you could do it in O(n + m) where m is the number of numbers to find. Basically just your typical merge algorithm, with slight modification to record which value each checked number would be inserted before, if it was to be inserted into the array.

You can always trade off space... And time of other operations. Assuming all your elements are constant size p bits, you can make a massive array which stores, for each possible value you could look up, the index of the next bigger value currently stored. This array needs to be 2^p*lg(n) bits, where n is the number values stored. Each insertion or deletion is O(2^p) but typically around 2^p/n, because you have to go through updating all those indices.

But your lookup is now O(1)!

OK, OK, it's not really practical. But dividing the input into blocks in a similar fashion could possibly reduce the constant in front of your log. Possibly.

David