Using CryptoAPI to generate ascii cipher text.

Question

Specifically what i'm trying to do is Generate a PassStub field for a Remote Assistance ticket. The problem is that my results look like binary data but somehow Microsoft generates printable characters.

In [MS-RAI]: Remote Assistance Initiation Protocol Specification <16> Section 6: Microsoft says that the "PassStub" field "is encrypted using PROV_RSA_FULL predefined Cryptographic provider with MD5 hashing and CALG_RC4, the RC4 stream encryption algorithm."

There is a data flow diagram here: http://msdn.microsoft.com/en-us/library/cc240189(PROT.10).aspx#id16

The diagram shows the hashed password being encrypted with a "RA SessionID" which looks like this: u0RIQibSMntm0wAHQZ2mhatI63sjMjX15kh/vnciytOix8z6w+36B01OiJoB5uYe

When I call CryptEncrypt the result is binary data about the length of the SessionID. Microsoft somehow gets something that looks like this: "Po^1BiNrHBvHGP"

Here is the code i'm trying to use to do this:

HCRYPTPROV hCryptProv;
HCRYPTKEY hKey;
HCRYPTHASH hHash;
BOOL bret=0;

passwordlen = SysStringByteLen(L"password");
    char RASessionID[] = "u0RIQibSMntm0wAHQZ2mhatI63sjMjX15kh/vnciytOix8z6w+36B01OiJoB5uYe";

//----------------------------------------------------------------
// Acquire a cryptographic provider context handle.
if(!CryptAcquireContext(&hCryptProv, NULL, MS_DEF_PROV, PROV_RSA_FULL, 0))
{
    return FALSE;
}
//----------------------------------------------------------------
// Create an empty hash object.
if(!CryptCreateHash(hCryptProv, CALG_MD5, 0, 0, &hHash))
{
    return FALSE;
}
if(!CryptHashData(hHash, (BYTE *)bpassword, passwordlen, 0))
{
    return FALSE;
}

//----------------------------------------------------------------
// Create a session key based on the hash of the password.
if(!CryptDeriveKey(hCryptProv, CALG_RC4, hHash, CRYPT_EXPORTABLE, &hKey))
{
    return FALSE;
}

DWORD rasessionidlen = strlen(rasessionid);
char* proxystub = (char*)malloc(rasessionidlen*2);
strcpy(proxystub, rasessionid);
bret = CryptEncrypt(hKey, NULL, TRUE, 0, (BYTE*)proxystub, &rasessionidlen, rasessionidlen*2);
return bret;

Answer 1

The "RA SessionID" looks like it is base64-encoded. My guess would be that the pass-stub is base64-encoded too - except that your example: "Po^1BiNrHBvHGP" is too short and contains a ^. Is that a real example?

You might also need to base64-decode the RA Session ID before feeding it to CryptEncrypt.