Amazon ec2 API version 2 signature encoding with c#

Question

I am having a problem with encoding the hash for the Version 2 Signature of the ec2 API.

Note my Version 1 Signature hashing works fine, but this is depreciated and I will need to move to version 2. So firstly here is the code that works...

parameters is just a dictionary, what I have to do is simply sort the parameters by key and append each value-pair with no delimeters, then hash that string against my key. (again, note this works fine)

private string GetVersion1Sig()
{
  string sig = string.Join(string.Empty, parameters.OrderBy(vp => vp.Key).Select(p => string.Format("{0}{1}", p.Key, p.Value)).ToArray());
  UTF8Encoding encoding = new UTF8Encoding();
  HMACSHA256 signature = new HMACSHA256(encoding.GetBytes(_secretAccessKey));
  byte[] hash = signature.ComputeHash(encoding.GetBytes(sig));
  string result = Convert.ToBase64String(hash);
  return result;
}

Now, with Version 2 there are some changes, here is the doco from the API developers guide...

1. Create the canonicalized query string that you need later in this procedure: a. Sort the UTF-8 query string components by parameter name with natural byte ordering. The parameters can come from the GET URI or from the POST body (when Content-Type is application/x-www-form-urlencoded). b. URL encode the parameter name and values according to the following rules: • Do not URL encode any of the unreserved characters that RFC 3986 defines. These unreserved characters are A-Z, a-z, 0-9, hyphen ( - ), underscore ( _ ), period ( . ), and tilde ( ~ ). • Percent encode all other characters with %XY, where X and Y are hex characters 0-9 and uppercase A-F. • Percent encode extended UTF-8 characters in the form %XY%ZA.... • Percent encode the space character as %20 (and not +, as common encoding schemes do). Note Currently all AWS service parameter names use unreserved characters, so you don't need to encode them. However, you might want to include code to handle parameter names that use reserved characters, for possible future use. c. Separate the encoded parameter names from their encoded values with the equals sign ( = ) (ASCII character 61), even if the parameter value is empty. d. Separate the name-value pairs with an ampersand ( & ) (ASCII code 38).
2. Create the string to sign according to the following pseudo-grammar (the "\n" represents an ASCII newline). StringToSign = HTTPVerb + "\n" + ValueOfHostHeaderInLowercase + "\n" + HTTPRequestURI + "\n" +
   CanonicalizedQueryString The HTTPRequestURI component is the HTTP absolute path component of the URI up to, but not including, the query string. If the HTTPRequestURI is empty, use a forward slash ( / ).
3. Calculate an RFC 2104-compliant HMAC with the string you just created, your Secret Access Key as the key, and SHA256 or SHA1 as the hash algorithm. For more information, go to http://www.rfc.net/rfc2104.html.
4. Convert the resulting value to base64.
5. Use the resulting value as the value of the Signature request parameter.

So what I have is....

private string GetSignature()
{
  StringBuilder sb = new StringBuilder();
  sb.Append("GET\n");
  sb.Append("ec2.amazonaws.com\n");
  sb.Append("/\n");
  sb.Append(string.Join("&", parameters.OrderBy(vp => vp.Key, new CanonicalizedDictCompare()).Select(p => string.Format("{0}={1}", HttpUtility.UrlEncode(p.Key), HttpUtility.UrlEncode(p.Value))).ToArray()));
  UTF8Encoding encoding = new UTF8Encoding();
  HMACSHA256 signature = new HMACSHA256(encoding.GetBytes(_secretAccessKey));
  byte[] hash = signature.ComputeHash(encoding.GetBytes(sb.ToString()));
  string result = Convert.ToBase64String(hash);
  return result;
}

for completeness here is the IComparer implementation....

  internal class CanonicalizedDictCompare : IComparer<string>
  {
    #region IComparer<string> Members

    public int Compare(string x, string y)
    {
      return string.CompareOrdinal(x, y);
    }

    #endregion
  }

As far as I can tell I have done everything I need to do for this hash, but I keep getting an error from the server telling me that my signature is incorrect. Help...

Answer 1

Ok, I figured it out....The UrlEncoding in the HttpUtility class does not conform to the Amazon encoding scheme....grrr (specifically the hex value after the % in the .NET utility is lowercase, not uppercase)

b. URL encode the parameter name and values according to the following rules:

• Do not URL encode any of the unreserved characters that RFC 3986 defines. These unreserved characters are A-Z, a-z, 0-9, hyphen ( - ), underscore ( _ ), period ( . ), and tilde ( ~ ).

• Percent encode all other characters with %XY, where X and Y are hex characters 0-9 and uppercase A-F.

• Percent encode extended UTF-8 characters in the form %XY%ZA....

• Percent encode the space character as %20 (and not +, as common encoding schemes do).

So after writing a quick method that encodes to this scheme, it works fine.