How about not loosing the reference, then? When you say your code crashes, I assume you get an exception somewhere. Where exactly? So someone capable of intercepting this exception obviously needs to also have a reference to the HttpServer, which you might have to pass around yourself.
Edit: Oh. In that case if you don't want to kill the entire JVM with the HttpServer in it, then you will need to offer some form of IPC to the environment, e.g. a command channel via RMI that can be invoked from a Java program (and hence Ant).
Another solution would be to have the server listen for some "secret" cookie query, where you e.g. print/save the cookie on startup so that the Ant script can retrieve the cookie, and you can fire off a query to your "secret" URL upon which the server will exit itself gracefully.
I'd go with a quick RMI solution.