Is it the program or the OS that is responsible for setting up the stack

Question

This question is asked because the author is building a compiler (src -> asm) and is in the process of writing code for generating assembly code.

One of the first things that happens when a program executes on a Linux (any OS really) based operating system is that the stack gets set up and the SP register is assigned with the memory address of the beginning of the stack.

I was curious about if the responsibility of doing the above lies with the program, or the OS itself. If it is the case that the responsibility lies with the program, how is this accomplished by the program (specifically, in a Linux base environment)?

What other actions must an individual program do before it can start executing its main function?

Examples with commented i386 NASM assembly code is appreciated. Also, any web resource that could help the author in his goal would also be very much appreciated.

Answer 1

The operating system's program loader is responsible for allocating stack space to executable processes. The stack is just one memory segment (look at your link map) that the loader fixes up when loading the program into RAM for execution. Other segments include uninitialized memory and initialized memory. The loader is also called the "relocatable" loader to indicate that it takes care of loading the program into a convenient place in memory.

When cross-compiling/linking for an embedded system, the link spec contains heap and stack information that is used at load time.

Answer 2

You may go search about ELF format for linux. And PE-COFF for windows.

Those are the way executable are format and it is something that the OS know about. Under Linux you have modules responsible of loading program and there use executable format in other to load correctly the program.

EDIT: A clue from the glibc give the answer:

/*
[snip]
%esp        The stack contains the arguments and environment:
        0(%esp)         argc
        4(%esp)         argv[0]
        ...
        (4*argc)(%esp)      NULL
        (4*(argc+1))(%esp)  envp[0]
        ...
                    NULL
[snip]
*/
[snip]
_start:
    /* Clear the frame pointer.  The ABI suggests this be done, to mark
       the outermost frame obviously.  */
    xorl %ebp, %ebp

    /* Extract the arguments as encoded on the stack and set up
       the arguments for `main': argc, argv.  envp will be determined
       later in __libc_start_main.  */
    popl %esi       /* Pop the argument count.  */
    movl %esp, %ecx     /* argv starts just at the current stack top.*/

    /* Before pushing the arguments align the stack to a 16-byte
    (SSE needs 16-byte alignment) boundary to avoid penalties from
    misaligned accesses.  Thanks to Edward Seidl <[email protected]>
    for pointing this out.  */
    andl $0xfffffff0, %esp
    pushl %eax      /* Push garbage because we allocate
                   28 more bytes.  */

    /* Provide the highest stack address to the user code (for stacks
       which grow downwards).  */
    pushl %esp

    pushl %edx      /* Push address of the shared library
                   termination function.  */

[snip]

Answer 3

The OS will set up the SP for you and place the program arguments on the stack. There is nothing special your program needs to set up in order to run.

As for the actual stack layout on Linux when your program starts, you could check this document for details.