Since each of the answers & comments is giving useful information. I just compiled, all the answers & comments into a single post.
I was just wondering if it is possible
to replace Loader (executable program
loader not the boot loader) of an
Operating System (Windows is my
choice).
No, in windows process creation and the user-mode loader in ntdll are tied together (PsCreateProcess will directly map in ntdll and jump to it so that it can finish resolving modules and setting up the process), you cannot replace it.
but there are resources availbable describing the format and loading of processes.
Here is a quite old but still uptodate MSDN article regarding PE files ( exe + dll )
- Part I. An In-Depth Look into the Win32 Portable Executable File
Format by Matt Pietrek (MSDN
Magazine, February 2002)
- Part II. An In-Depth Look into the Win32 Portable Executable File
Format by Matt Pietrek (MSDN
Magazine, March 2002)
You can use this information to write an app that starts a given executable.
If you are more interested in linux and the elf format you will find all you need in google.
Is there any way through which I can
obtain the control over the OS Loader?
I mean, I want things it is doing to
be visible to me(each and every step).
On Windows, you can get some visibility into the loader at work by enabling Loader Snaps. You do this with gflags.exe
(part of Debugging Tools for Windows). There's a nice gflags.exe
reference http://www.osronline.com/DDKx/ddtools/gflags_4n77.htm . With Show Loader Snaps enabled, you can see loader trace messages by starting the application under a debugger (WinDBG).
If you want to play with this sort of thing then Linux is the best way to go.
The loader is part of the kernal -- but as you have access to all the kernal source you can play with it to your hearts content.
The loaders for various binary formats are in fs/binfmt_*.c
in the Linux source (fs/binfmt_elf.c
is the loader used for executables in ELF format - ie. the vast majority).
The dynamic loader /lib{,64}/ld-linux.so.2
is also used for dynamically linked binaries - it's an example of an "interpreter" as referenced by the code in binfmt_elf.c.
Linux has pluggable executable file formats, so it is possible to add an extra program loader which will do its own custom stuff with executable files, rather than the standard ones (ELF, shell scripts, binfmt_misc).
The binfmt_misc
module allows you to write custom loaders for executable programs entirely in userspace; this is commonly used to execute non-native binaries or interpreted binaries such as Java, CLR executables etc.
On the other hand if you wanted to replace the ELF loader with something else you can make a binfmt module directly in the kernel. Look at fs/binfmt_*
for examples. The ELF loader itself is in there.