How do I fake an specific browser client when using Java's Net library?

Question

A small program of mine just broke because, it seems, the site I was programmatically browsing now assumes a Java request comes from a mobile phone, and the link I was looking for is not on their mobile page.

So I want to fake an Internet Explorer access. How do I do that with java.net?

Answer 1

You need to set the User-Agent header in the HTTP request to a value used by Internet Explorer.

I recommend using the Jakarta HttpClient library to make the request as it provides a higher level API for manipulating the request.

Answer 2

Assuming you're using java.net.URLConnection, then call setRequestProperty(String,String) to set the request header to a value that IE would use. For example, to fake IE6:

URL url = new URL("http://google.com");
URLConnection connection = url.openConnection();
connection.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 1.2.30703)");

and then use the connection object as before.

But java.net is horrible. Use Apache Commons HttpClient instead, it's much nicer.

Even better, use a framework designed for navigating websites, like HtmlUnit

Answer 3

IIRC, set "http.agent" system property through System, -D on the command line, in your JNLP file or elsewhere.