In the context of compilers, binding is the phase where the address references are changed into actual absolute addresses.
When the program is compiled, the address of each symbol (variable, function) is stored in symbol table for example as a relative offset from the beginning of the object module, together with the symbol name. The symbol name is needed since the symbols may be called from another object module.
When the program is linked, the object modules are combined into a single program file and the symbol names are no more needed.
If the program is linked into known absolute address, all the address references can be bound into absolute addresses already at this phase.
However, in workstations such as PC:s, the program can be loaded to any address, so the address is not known at link time.
Therefore, additional relocating information is stored in the program file so that the loader can bind the addresses at load time.
Binding is done at load time using the relocation information. When the address where the program is going to be run is known, the loader replaces the relative addresses with absolute addresses using the relocation information that tells where in the code the changes need to be done.
For dynamic objects/variables, binding can be done at runtime. (I think this is what is usually called late binding.)
Usually you do not need to care much about binding, at least not when early binding is used. (However, binding at runtime may have negative impact to performance and security.)