square brackets in URLs

Question

Are square brackets in URLs allowed?

I noticed that Apache commons HttpClient (3.0.1) throws an IOException, wget and Firefox however accept square brackets.

URL example: http://example.com/path/to/file[3].html

My HTTP client encounters such URLs but I'm not sure whether to patch the code or to throw an exception (as it actually should be).

Answer 1

Best to URL encode those, as they are clearly not supported in all web servers. Sometimes, even when there is a standard, not everyone follows it.

Answer 2

Pretty much the only characters not allowed in pathnames are # and ? as they signify the end of the path.

The uri rfc will have the definative answer:

http://www.ietf.org/rfc/rfc1738.txt

Unsafe:

Characters can be unsafe for a number of reasons. The space character is unsafe because significant spaces may disappear and insignificant spaces may be introduced when URLs are transcribed or typeset or subjected to the treatment of word-processing programs. The characters "<" and ">" are unsafe because they are used as the delimiters around URLs in free text; the quote mark (""") is used to delimit URLs in some systems. The character "#" is unsafe and should always be encoded because it is used in World Wide Web and in other systems to delimit a URL from a fragment/anchor identifier that might follow it. The character "%" is unsafe because it is used for encodings of other characters. Other characters are unsafe because gateways and other transport agents are known to sometimes modify such characters. These characters are "{", "}", "|", "\", "^", "~", "[", "]", and "`".

All unsafe characters must always be encoded within a URL. For example, the character "#" must be encoded within URLs even in systems that do not normally deal with fragment or anchor identifiers, so that if the URL is copied into another system that does use them, it will not be necessary to change the URL encoding.

The answer is that they should be hex encoded, but knowing postel's law, most things will accept them verbatim.

Answer 3

According to the URL specification, the square brackets are not valid URL characters.

Here's the relevant snippets:

The "national" and "punctuation" characters do not appear in any productions and therefore may not appear in URLs.
national { | } | vline | [ | ] | \ | ^ | ~
punctuation < | >

Answer 4

Any browser or web-enabled software that accepts URLs and is not throwing an exception when special characters are introduced is almost guaranteed to be encoding the special characters behind the scenes. Curly brackets, square brackets, spaces, etc all have special encoded ways of representing them so as not to produce conflicts. As per the previous answers, the safest way to deal with these is to URL-encode them before handing them off to something that will try to resolve the URL.

Answer 5

For using the HttpClient commons class, you want to look into the org.apache.commons.httpclient.util.URIUtil class, specifically the encode() method. Use it to URI-encode the URL before trying to fetch it.

Answer 6

[RFC 3986][1] states

A host identified by an Internet Protocol literal address, version 6 [RFC3513] or later, is distinguished by enclosing the IP literal within square brackets ("[" and "]"). This is the only place where square bracket characters are allowed in the URI syntax.

So you should not be seeing such URI's in the wild in theory, as they should arrive encoded.

[1]: http://www.ietf.org/rfc/rfc3986.txt RFC 3986

Answer 7

I know this question is a bit old, but I just wanted to note that PHP uses brackets to pass arrays in a URL.

http://www.example.com/foo.php?bar[]=1&bar[]=2&bar[]=3

In this case $_GET['bar'] will contain array(1, 2, 3).

Answer 8

StackOverflow seems to not encode them:

http://stackoverflow.com/search?q=square+brackets+[url]