Decrypt file using XOR

Question

First I'm using MASM, I'm opening an encrypted file and putting it's contents into a buffer and exporting it to a new file. I have everything working except the decrypting portion.

I'm not sure if I need to XOR the buffer itself or do I reference edx (where I store the buffer), or do I need to XOR the bytes read which I put in the eax register.

Answer 1

By XOR, I assume you mean inverting all the bits in the buffer, i.e. XOR with all 1's.

I'm rusty on ASM, but I think the thing to do would be to load $FFFF into some register and then loop over your buffer, loading each byte or word or dword into AX, XORing with that other register and storing the value back into the buffer.

Answer 2

If you are using simple XOR function for decryption and encryption then function is the same for boath calls. XOR binary function is reversible!

Answer 3

off the top of my head (no testing) ...

mov esi,dword ptr [buffer]
mov ecx,dword ptr [bufsize]
shr ecx,2
jz startloop1
toploop4:
mov eax,dword ptr [esi]
xor eax,ffffffffh
mov dword ptr [esi],eax
dec ecx
jnz toploop4
startloop1:
mov ecx,dword ptr [bufsize]
and ecx,3
jz end
toploop1:
mov al,byte [esi]
xor al,ffh
mov dword ptr [esi],al
dec ecx
jnz toploop1
end: