views:

254

answers:

8

My client asked me to write an custom encrypted executable to prevent easy cracking of the licensing system. Now, I understand that this is a false sense of security, but despite this he insisted on it.

So, I dug up my knowledge of portable executables and came up with this idea:

  • Encrypt the executable
  • Stick this to the end of a loader executable along with it's size
  • The loader decrypts the data
  • It copies the code to a page allocated with VirtualAlloc that has executable permissions
  • It finds the entry point of the application
  • Jumps there and we are all set.

I have a problem with the jumping there part. How can I do that? If I were to set a function pointer to it, what would be the signature? The signature of the loaded executable's main() function? Or do I need to resort to assembly?

I understand that it might be needed to correct absolute addresses after loading the code. How do I check if I need to, and how do I actually perform this?

Edit: Working on windows and compiling with GCC. I can switch the Microsoft compiler if necessary.

Edit2: To clarify: I KNOW it's mostly pointless. I believe that stands for any kind of DRM. It's up to my client to decide, and he still wants it despite me warning him about this.

Thanks in advance.

A: 

You should be able to c-style cast your address to a function pointer, and call it:

typedef void (*MyPtr)();

MyPtr p = (MyPtr)1234;
p();
Jesse Collins
+10  A: 

Unfortunately, jumping to the entry point of your code is the least of your worries. A Portable Executable (PE) file (which is the file format used for EXE and DLL files on Windows) is not something you can just load into a single memory block and then run. Your custom PE loader would have to take care of the following jobs:

  • Load the various code and data sections in the PE file into separate memory blocks.

  • Resolve dependencies from the import table to load DLLs that your EXE depends on.

  • Perform relocations.

Getting all of the details right would probably be a pretty involved job. I would suggest that you look for a tool you can buy that does this kind of EXE encryption.

Edit: A quick Google suggests you might want to take a look at the following:

  • EXECryptor (proprietary)

  • RLPack (proprietary)

  • UPX (GPL) This one only does compression as far as I understand, but you could Use The Source to add encryption (if the GPL is compatible with your needs).

There are bound to be more tools like these -- this is just the result of a quick search.

Another edit:

MSDN magazine carried an article by Matt Pietrek called "An In-Depth Look into the Win32 Portable Executable File Format" (Part 1, Part 2). It contains a lot of information on the PE file format that should be useful to you. One interesting bit of information: Recent versions of the Microsoft linker appear to leave out base relocations for EXEs by default. You'll probably want to instruct the linker to put these back in because it's likely that your wrapper EXE is already loaded at your payload EXE's preferred load address. Alternatively, you could try and give your wrapper EXE an exotic preferred load address where it hopefully won't interfere with the payload EXE.

I've also found a page that discusses a rudimentary PE file compressor. It doesn't appear to be fully general though and would probably require some additional work before you can use it.

Martin B
I know some of these tools, and suggested them for the client, but he googled unpackers for every single one of them and feels he needs a custom one. Even if I can't undertake the task, I'd like to learn about it as I find it a very interesting topic. Could you suggest me some material on those points you mentioned? Especially relocations.
Tamás Szelei
@sztomi: I've added some links to more information. For more on relocations, take a look at Part 2 of Matt Pietrek's article.
Martin B
@sztime: "WINE" ( http://www.winehq.org/ ) project has implemented PE loader for linux, so you could probably study the source at least. MPlayer ( http://www.mplayerhq.hu/design7/news.html ) has used some kind of code (probably from wine) to load windows dll on linux platform as well. Anyway, this can be done, but if this is a "copy protection defense", it can be cracked, and it WILL be cracked if product is good enough. Anybody that has access to your exe can (try to) crack it by bypassing protection, or unpacking it.
SigTerm
+1. UPX is not going to work, since you either have to use unmodified UPX or distribute programs under the GPL. I don't think there's any advantage to rolling your own provided you can add your own encryption to a (reasonably) free compressor. Ultimately, though, the simple decrypt-into-memory approach is easily defeated by a memory dump.
tc.
+3  A: 

In addition to all of the fuss and bother of correctly loading a PE image, you will also need to worry about Data Execution Protection which is intended to prevent doing this very thing.

Don't forget that this also can appear like malware behavior to some anitivirus and antimalware tools.

RBerteig
Data Execution Protection only applies to data pages; if you alloc a page using VirtualAllocEx() with one of the PAGE_EXECUTE_* flags, you'll be able to execute code from it.
Martin B
To expand on this: DEP is not intended to prevent intentional loading and execution of code; it's intended to prevent accidental (or more likely malicious) execution of code from data pages -- e.g. a buffer overflow attack that executes code from the stack. Creating and executing your own code on the fly is legitimate behaviour -- JIT compilers are a prominent example.
Martin B
You are right, Martin, I looked up the DEP documentation before the question.
Tamás Szelei
@Martin, right. But you do have to be aware of it and correctly allocate the memory to be executable or it will stand up and swat you at some inopportune moment.
RBerteig
+1  A: 

Your typed solution in my point of view is little buggy, what this your functions doing jumps to other functions? They are fixed other addresses and i see many many problems with this solution like where is the static linked libraries of the encrypted code.

I think you can do something like:

  1. Transport a encypted dll file with your secret assembly
  2. Encrypt her in a tmp directory
  3. do LoadLibrary on this
  4. Unlock DLL file and remove from system?
Svisstack
Yes, this is how I've seen it done. The encrypted DLL can be stored in your main .exe's resources even. You might also want to prevent the extracted DLL operating independently by making it call through to functions in your main .exe through a jump table passed in to do most operations. But you're probably better off buying this sort of thing in anyway.
Rup
A: 

You can't execute data like code in most operating system, thanks to gdt, the global descriptors table that divides data from code. If you try to execute data an exception is given by the processor.

Charlie
Until very recently, read permission and execute permission were the same thing on x86.
tc.
Only because operating system didn't use GDT as they should it doesn't mean that they where the same thing. Please check yourself on wikipedia or on Intel datasheets
Charlie
@Charlie: The GDT can't be used for this purpose in the flat memory model (used by all common x86 OSs) because there is only one segment that contains the whole of the flat address space. The NX bit (http://en.wikipedia.org/wiki/NX_bit) fulfills the same purpose on a page table level but was only introduced with the 5x0J series of Prescott (http://en.wikipedia.org/wiki/Pentium_4) in 2004.
Martin B
If an osdev designer decides to use the flat memory model is a decision he can do. It's the easier I know but the gtd exist for something, not to be hidden and use memory as a long flat block. From 2004 up to today there are 6 years of difference, I think nobody uses Pentium 3 today.
Charlie
Only on Harvard architecture you could not execute data, but since we're talking here about Windows..Also, even on Harvard the code has to be loaded somehow in the memory in order to be executed, so the difference between "code" and "data" is even more fuzzy (only on simple controllers when the code is executed from ROM there is real difference).
ruslik
+1  A: 

The loader decrypts the data

This says it all really.

For the loader to be able to decrypt the data it must also contain (or at least know about) the decryption key.

Therefore if you distribute this loader physically to your users they will have full access to both the loader code and the key used.

Maybe this is what you mean by "false sense of security"?

As your client seems to be either stubborn or just ignorant, why not build a simple little decryption demo that clearly shows the flaws in their thinking?

Ash
Code obfuscation *does* "work" though, assuming that preventing *some* cracking makes some people buy the full app - it's a matter of guessing what the curve looks like and putting the appropriate amount of effort in.
tc.
+3  A: 

As others have mentioned, simply loading the entire EXE up into a data section and linking it at runtime is a difficult task; however, here's another option.

Take your input EXE; find its code and initialized data (including constant) sections. Rename these sections and convert them all to read-write initialized data sections; encrypt the contents. Now add a new code segment containing your decryption stub, and change the entry point there. This stub should decrypt the segments in-place, then change their protection to whatever is appropriate for their type, and jump to the original entry point.

This avoids having to implement all the functions of a full PE loader, as the imports tables are not encrypted, and thus the normal windows loader will take care of them for you.

It should be noted, of course, that this kind of naive approach will not survive a concerted attack for any time at all - an attacker can simply dump the process's memory to get the original, decrypted code. To avoid this one would likely need to encrypt and decrypt code on a continual basis, at which point PE handling is the least of your concerns.

bdonlan
That makes sense. So if I stick a new section in the exe and update the section tables and the entry point I can simply jump to the decrypted code because the OS loader took care of the imports and relocations (if needed)?
Tamás Szelei
Basically, yeah - except relocations _won't_ be handled if they're in the section in question. So make sure it's a non-relocatable EXE. Imports have a separate section of stubs in PE files, I believe... (not sure, you should probably check this :)
bdonlan
+1 memory dump attack ;)
tc.
+1  A: 

Also see the accepted answer to What is your favourite anti-debugging trick for a good solution to make cracking of the executable harder.

Jonas Gulle