tags:

views:

796

answers:

9

I had a painful experience with the "Analysis of Algorithms" classes back in college but have recently found a need for it in the real world. -- Anyway, I'm looking for a simple-yet-effective crash course. Any ideas?

Related Sidenote: It sure would be nice if there were a "Cartoon Guide to Algorithm Analysis", taught by Dilbert.

UPDATE: A very similar question can be found at: How to get started on ALGORITHMS?

+10  A: 

There are a lot of good books on the subject. I like An Introduction to the Analysis of Algorithms. Also check out the algorithms course on MIT OpenCourseWare (using CLRS as the course text). It's a little bit deep, but having it online allows you to go at your own pace.

A couple of other books that I've started reading recently are Algorithms in a Nutshell and the Algorithm Design Manual. They both take a lighter approach than most algorithms books. Instead of heavy math and formal proofs these books give you realistic problem statements and show you the steps taken to refine an algorithm. They also show you how to estimate and measure the complexity of a solution. I would highly recommend either book.

Bill the Lizard
+1. for the MIT OCW and Skiena recommendations
Mitch Wheat
...oh and the sedgewick book (that was the one I started with)
Mitch Wheat
+3  A: 

I like Introduction to Algorithms by Cormen, Leiserson, Rivest, and Stein. It's a bit heavy, but I found it to be a decent reference text.

Elie
Wow. At 1056 pages, I'm sure it is heavy.
Paul Croarkin
you can curl that! (no... not the analog to wget...)
warren
@Paul: it's not a book you'll want to read from cover to cover. However, it is one of the #1 books on this topic.
Jasper Bekkers
+4  A: 

Likewise, UC Berkeley has myriad podcasts that you may find helpful.

http://webcast.berkeley.edu/course_feeds.php

warren
In particular, look here http://www.cs.berkeley.edu/~vazirani/algorithms.html for a draft of the textbook used at Berkeley. There are no podcasts or webcasts that I know of for a class on Algorithms at Berkeley.
Kaushik
+4  A: 

Google Code University offers a page on Algorithms that points to some slides offered by Stanford and Princeton.

Nate
+1  A: 

Most algorithm analysis courses at Universities are only open to upper level undergraduates and graduate students. Why would you expect this topic to be easy to learn? There are algorithms that are simple to analyze and the Wikipedia article on Big O notation is probably adequate to understand how and perform an analysis on them, but doing an analysis on any reasonably complex algorithm is non-trivial. The Cormen book is probably the most widely used book on algorithms, but I wouldn't consider learning algorithm analysis from it or any other book painless.

tvanfosson
+3  A: 

You don't say a lot about the remainder of you background. For straight out analysis of algorithms, the methods by which you evaluate an algorithm to find its order statistics and behavior, If you're comfortable with mathematics in general -- say you've had two years of calculus, or a good abstract algebra course -- then you can't really do much better than to read Knuth Volume One.

The usual "Analysis of Algorithms" course is also a data structures course, so a data structures text might be better if you also need to learn about lists, trees, etc. My favorite in graduate school was Aho, Hopcroft and Ullman.

I like Cormen as a reference; it also serves as an admirable doorstop, for the execution of large icky bugs, for clamping small glue joints (the slick cover releases most wood glues), and as an end book because it holds a bookend in place. Wouldn't recommend it as an intro text.

Charlie Martin
+3  A: 

Erik Öjebo posted a great answer to a nearly identical question, asked shortly after yours.

Quoting his answer:

MIT has a course on algorithms in their Open Courseware Program with video, audio and PDF lectures.

There is also an online course, also with video lectures, at ArsDigita University.

At University of Florida there is the course Data Structures and Algorithms in Java, and just as the above it has video lectures available online.

At freescienceonline.blogspot.com you can find a whole lot of video lectures on algorithms, as well as a lot of other interesting videos.

+1  A: 

The single most helpful tool I've had for algorithms is Introduction to Algorithms.

It's the single best resource I know of for algorithms. It covers so many topics in good depth and has examples throughout. I still refer to it quite frequently.

Without that book my algorithms analysis class would have been a pain.

Ben S
+1  A: 

There is a simple shortcut to understanding the performance of searching and sorting algorithms, if that is what you're looking for.

First, sorting is basically repeated search. For each of N items, you are searching for where to insert it in the list, and then inserting it, so it takes N times the big-O of the search procedure.

To understand big-O of search, a simple way is to think of it as a series of decisions (usually binary) with a certain probability of taking each branch.

Suppose your table has N = 1024 entries. That means it takes 10 bits to index the table, because log(1024) = 10 (base 2). Searching is a process of learning those 10 bits.

If a decision point has roughly equal probability of going either way, then it has an entropy of -0.5 log(0.5) - 0.5 log(0.5) (base 2) which is 1 bit, so it learns 1 bit of information on each decision. Voila! It takes roughly 10 decisions, or log(N). So the sorting is O(N log(N)). All the NlogN sorting algorithms are based on binary decisions having roughly equally likely outcomes.

Suppose you are doing linear search (as in bubble sort). On the first decision, the chance of being right is 1/1024 vs 1023/1024. The entropy is 1/1024 * log(1024/1) + 1023/1024 * log(1024/1023) or roughly 10/1024 + 0 (i.e. about .01). So on the first decision you only learn about .01 bit because the outcomes are so skewed. That is why linear search is inefficient. It takes on the order of N operations, so sorting takes O(N*N).

(Aside: linear search is actually exponential. If you define the information content of a problem as n = log(N), then linear search takes O(2^n) steps. That's why things like unguided game-tree search are exponential in number of moves.)

On the other hand, suppose rather than making binary decisions, you are indexing. You take some or all of the bits of the word that you're looking for, and use them as an index into an array, where you've pre-stored answers. If that indexing operation has 1024 equally likely outcomes, then it learns 10 bits, so it only takes roughly 1 operation to get the answer. That is the basic idea behind hash coding. If your hash function happens to perserve order, it can be used to make an O(N) sorting algorithm.

There are fancy ways to analyze algorithms, and maybe that's what you need, but looking at them as learning bits of information serves 99% of my needs.

Mike Dunlavey