Delegates provide a way for you to pass behavior as a parameter.
Common examples are Events and asynchronous programming where something other than your object is responsible for calling into your object. You provide that event with a delegate, and now it's able to run the behavior associated with that delegate.
This can also be a useful technique when implementing a general algorithm. There've been times where I'm writing multiple methods that are very similar. Perhaps they loop over the same set of data, but they perform slightly different tasks. I can pass in a delegate to a single function to perform that task, then call the delegate from inside the loop over the data. This way I don't have to implement the loop over the data multiple times, I only have to write new code that does new things-- the common behavior is all captured in a common way.
In response to the comment:
The difference between calling the delegate from within the loop and calling the method from within the loop is that the delegate is a parameter to the function that contains the loop. This means that that function could do anything, not just what's defined within a particular method. That flexibility can and has been leveraged to supply generic algorithms in libraries completely independent of the specifics of what the algorithms are working with. Linq is a fine example of the generality that is allowed via the flexibility of delegates.