I've been developing a Java applet in which everything is written in Clojure except the applet code, which is written in Java. The applet invokes the Clojure code's callbacks of init, paint, etc from java's hooks for those methods that are defined by the applet model. So the code ends up being 99.999 percent Clojure and you don't have to think about the tiny Java piece at all for the most part.
There are some drawbacks to this approach, which I hope to discuss in more detail on the Clojure Google Group. I think the Clojure developers should include a native way of building applications. Presently you can do whatever GUI stuff you like from the REPL, but if you want a deliverable GUI application, it is necessary to write some Java to call the Clojure code. Also, it seems like the architecture of a Java Applet kind of forces you outside of Clojure's more idiomatic best practices, requiring you to use mutable state, etc.
But also, I am not very far along with Clojure yet and it might be the case that it is possible and I just haven't discovered how to do it correctly yet.