In Delphi the Image Base linker option defaults to 00400000.
Per the help:
Specifies the preferred load address of the compiled image. This value is typically only changed when compiling DLLs. Default = 400000
Is there no effect for changing it on EXE's? What would the effect be? Is the address relative to each process?
Change EXE's image base is almost useless unless you're doing some very low-level dirty hack.
Is the address relative to each process?
Yes, each process has its own address space.
Executable images (EXEs and DLLs, and other things that are DLLs in disguise, like BPLs and OCXs) are loaded by the OS loader at their preferred load address (Image Base) if possible; if that area of the virtual address space is reserved for some other purpose (another image, a thread stack, heap allocation), then the OS loader will relocate the image. Relocating the image involves putting it somewhere else in the address space, then taking the difference between the new load address and the preferred load address and adding this difference to every relocation fixup inside the image. Relocation fixups point to all the places in the executable image where the code or data refers to itself, such as code loading values from global variables, or making absolute jumps to other routines.
Because relocation involves the OS modifying the in-memory version of the image data, it takes longer, it takes up more I/O and commits more pages (the entire image with relocations needs to be paged in), and the OS virtual memory subsystem won't be able to share the loaded image with other processes that have loaded the same executable image (since it will be different in-memory). Thus, it's desirable to avoid relocation upon loading.
The preferred address for executable images is $00400000 by convention on 32-bit Windows, and other DLLs (including OS DLLs) rely on this convention by not having default load addresses that are likely to coincide with the main executable. Thus they avoid relocation. In fact, relocating an EXE image is so infrequently done that the relocation data can often be stripped from EXE images without harm.
Changing it for DLLs makes sense to avoid conflicting with any of the default OS DLLs and any other DLLs that normally ship with the DLL / EXE. Since changing it for an EXE increases the chances that the OS will need to relocate a DLL, it's not recommended that the EXE load address be changed.
Executable image compactors like UPX are not recommended for DLLs in particular, and for executables that may have many instances running, because the in-memory decompression acts like relocation in preventing the in-memory image from being shared between multiple processes.