Nope, it's not that simple. :)
Modern operating systems use virtual memory.
Every process is provided with a full virtual address space.
Every process is given its own "view" of all addresses (from 0x00000000 to 0xffffffff on a 32-bit system). Processes A and B can both write to the same address, without affecting each others, because they're not accessing physical memory addresses, but virtual addresses. When a process tries to access a virtual address, the OS translates that into some other physical address to avoid collisions.
Essentially, the OS keeps track of a table of allocate memory pages for every process. It tracks which address ranges have been allocated to a process, and which physical addresses they're mapped to. If a process tries to access an address not allocated to it, you get an access violation/segmentation fault. And if you try to access an address that is allocated to your process, you get your own data. So there is no way to read other processes data just by typing in the "wrong" address.