Which encoding is used by the HTTP protocol?

Question

When a browser sends an HTTP request to a web server, what encoding is used to encode the HTTP protocol on the wire? Is it ASCII? UTF8? or UTF16? Or does it specify which encoding it uses in a predefined format (before any decoding takes place?)

P.S I'm not asking about the actual payload (e.g. HTML) of the request/response. I'm asking about the request line (i.e. GET /index.html HTTP/1.1) and headers (i.e. Host: google.com)

Answer 1

The raw text is in ASCII I believe.

2.2 Basic Rules

The following rules are used throughout this specification to describe basic parsing constructs. The US-ASCII coded character set is defined by ANSI X3.4-1986 [21].

From RFC 2616

Looking further, unicode is supported after reading the Accept-Charset header.

Answer 2

RFC 2616 includes this:

OCTET          = <any 8-bit sequence of data>
CHAR           = <any US-ASCII character (octets 0 - 127)>
UPALPHA        = <any US-ASCII uppercase letter "A".."Z">
LOALPHA        = <any US-ASCII lowercase letter "a".."z">
ALPHA          = UPALPHA | LOALPHA
DIGIT          = <any US-ASCII digit "0".."9">
CTL            = <any US-ASCII control character
                  (octets 0 - 31) and DEL (127)>
CR             = <US-ASCII CR, carriage return (13)>
LF             = <US-ASCII LF, linefeed (10)>
SP             = <US-ASCII SP, space (32)>
HT             = <US-ASCII HT, horizontal-tab (9)>
<">            = <US-ASCII double-quote mark (34)>

And then pretty much everything else in the document is defined in terms of those entities (OCTET, CHAR, etc.). So you could look through the RFC to find out which parts of an HTTP request/response can include OCTETs; all other parts must be ASCII. (I'd do it myself, but it'd take a long time)

For the request line specifically, the method name and HTTP version are going to be ASCII characters only, but it's possible that the URL itself could include non-ASCII characters. But if you look at RFC 2396, it says that.

A URI is a sequence of characters from a very limited set, i.e. the letters of the basic Latin alphabet, digits, and a few special characters.

Which I guess means that it'll consist of ASCII characters as well.

Answer 3

HTTP 1.1 uses US-ASCII as basic character set for the request line in requests, the status line in responses (expect the reason phrase) and the field names but allows any octet in the field values and the message body.