One example of how a bug can create an opportunity for exploit:
Let's say you have a subroutine in a program that modifies data in an array of characters. Let's say it also contains a bug that when the array is a particular size or the array contains particular characters, the subroutine inadvertantly writes past the end of the array of characters.
This by itself doesn't seem like much of an opportunity, but depending on how execution reaches the subroutine and other artifacts of how it is implemented and compiled, it could be used as a springboard to executing arbitrary code.
In traditional programming (C, C++), character arrays (buffers) are often stored on the program stack. The stack is very fast and easy memory allocation for smallish temporary data.
Another thing that is stored on the stack is the function call return address - what code address to return to when this function exits.
Now you have all the pieces needed to create disaster: If you can pass just the right data to this subroutine to make it overwrite the stack, and overwrite it enough to overwrite the function return address that is also on the stack not far from the data buffer, then you have the potential to alter where program execution will return to when the function exits. Instead of returning to the caller, it could be made to "return" (jump, really) to Halt() or Format() or PhoneHome(). Any function in any library or DLL referenced by the current process is accessible at this point.
This is just one example of an arbitrary execution exploit. There are dozens of such patterns.
The easiest way to thwart this particular exploit is to ensure that your code respects the bounds of your data buffers. For most compilers, this means turning on range checking or similar runtime checks. The compiler will emit code to validate that an array index value is in range before accessing the memory location in the array.