A common use of continuations is in the implementation of control structures for: returning from a function, breaking from a loop, exception handling etc. Most languages (like Java, C++ etc) provide these features as part of the core language. Some languages don't (e.g: Scheme). Instead, these languages expose continuatiions as first class objects and let the programmer define new control structures. Thus Scheme should be looked upon as a programming language toolkit, not a complete language in itself.
In Clojure, we almost never need to use continuations directly, because almost all the control structures are provided by the language/VM combination. Still, first class continuations can be a powerful tool in the hands of the competent programmer. Especially in Scheme, continuations are better than the equivalent counterparts in other languages (like the setjmp/longjmp pair in C). This article has more details on this.
BTW, it will be interesting to know how Rich Hickey justifies his opinion about continuations. Any links for that?