EncryptMessage returns SEC_E_INVALID_TOKEN

Question

Hi

When using the EncryptMessage (SChannel) from the win32 API with a valid context, I am supplying the four buffers in the correct order I get the SEC_E_INVALID_TOKEN response which according to the documentation is No SECBUFFER_DATA type buffer was found. I know that the set of pvBuffers should be allocated from contiguous memory for speed but for simplicity I have made it obvious what is what. Can anyone see what the problem could be?

Thanks, Bruce

The code is the following;

procedure TTCPSocket.SSPEncryptBuffer(SSPCtx: PCtxtHandle; InData: PAnsiChar; InDataLength: Cardinal);
var
  SecStatus: TSecurityStatus;

  SecBufDesc: TSecBufferDesc;
  SecBufs: packed array [0 .. 3] of TSecBuffer;
begin
  SecBufs[0].BufferType := SECBUFFER_STREAM_HEADER;
  SecBufs[0].cbBuffer := FSecPkgSizes.cbHeader;
  SecBufs[0].pvBuffer := AllocMem(FSecPkgSizes.cbHeader); 

  SecBufs[1].BufferType := SECBUFFER_DATA;
  SecBufs[1].cbBuffer := InDataLength; 
  SecBufs[1].pvBuffer := InData; 

  SecBufs[2].BufferType := SECBUFFER_STREAM_TRAILER;
  SecBufs[2].cbBuffer := FSecPkgSizes.cbTrailer;
  SecBufs[2].pvBuffer := AllocMem(FSecPkgSizes.cbTrailer);

  SecBufs[3].BufferType := SECBUFFER_EMPTY;
  SecBufs[3].cbBuffer := 0;
  SecBufs[3].pvBuffer := nil;

  SecBufDesc.ulVersion := SECBUFFER_VERSION;
  SecBufDesc.cBuffers := 4;
  SecBufDesc.pBuffers := @SecBufs[0];

  SecStatus := EncryptMessage(SSPCtx, 0, @SecBufDesc, 0);

  if SecStatus <> SEC_E_OK then
  begin
    // Error code..
  end;
end;

I used STRACE injected into the executable and this line looks interesting;

12/07/2009 23:10:30:635 - SecBuffer #0 BufferType:0x00000007 cbBuffer:5
12/07/2009 23:10:30:636 - SecBuffer #1 BufferType:0x00000001 cbBuffer:13
12/07/2009 23:10:30:636 - SECBUFFER_DATA - 13 byte(s) / EncryptMessage - INPUT 
=====================================================
      00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f   0123456789abcdef

0000: 68 65 6c 6c 6f 20 77 6f 72 6c 64 21 00            hello world!.
=====================================================
12/07/2009 23:10:30:636 - SecBuffer #2 BufferType:0x00000006 cbBuffer:36
12/07/2009 23:10:30:636 - SecBuffer #3 BufferType:0x00000000 cbBuffer:0
12/07/2009 23:10:30:636 - 

 *** WARNING : EncryptMessage failed (80090308) ***

Which looks as though the OS is getting the correct information.

I have searched a bit and found that 80090308 usually means something wrong with the certificate in that the full name of the server should be in the subject, CN=www.foobar.com but this didn't fix the problem either, the certificate and CA are generated with OpenSSL.

Answer 1

Somewhat of a guess here since this seems like a common problem people run into when trying to do anything with asymmetric crypto on Windows.

If it is a self-signed CA and your cert is signed with the self signed CA, you need to import the CA into your computer's trusted CA store.

To do this, run MMC and do the following:

1. File->Add/Remove Snap In

2. Add the "Certificates" snap in and choose "Computer Account" if you want it to apply for all users on the computer.

3. Open the "Trusted Root Certificate Authorities->Certificates" tree node.

4. Right click on "Certificates" and choose import.

5. Import the CA certificate file. (It should accept the PEM encoded version without a problem.)

While the CN will need to match the name of the machine in your machine certificate in most cases, the CA validation will fail if the CA is not imported into the Windows trust storage.

I hope it helps.

Answer 2

When you set up your context on both client and server using;

InitializeSecurityContext // Schannel Client
AcceptSecurityContext // Schannel Server

pay very special care to the flags you pass in for the type of context you want as that determines the types of SecBuffer(s) that you need for both EncryptMessage and DecryptMessage. For instance I am currently using;

  FClientContextFlags :=
    ISC_REQ_CONFIDENTIALITY or
    ISC_REQ_STREAM or
    ISC_REQ_ALLOCATE_MEMORY;

  FServerContextFlags :=
    ASC_REQ_CONFIDENTIALITY or
    ASC_REQ_STREAM or
    ASC_REQ_ALLOCATE_MEMORY;

which means that for EncryptMessage you need four SecBuffers of;

SECBUFFER_STREAM_HEADER
SECBUFFER_DATA
SECBUFFER_STREAM_TRAILER
SECBUFFER_EMPTY

and for DecryptMessage you also need four SecBuffers of;

SECBUFFER_DATA
SECBUFFER_EMPTY
SECBUFFER_EMPTY
SECBUFFER_EMPTY

My problem was that I had *_REQ_STREAM and *_REQ_CONNECTION which on closer reading of the documentation are essentially incompatible. This is on top of making sure you have a valid certificate/trusted etc.

I hope this helps someone.

Bruce