I haven't tried compiling anything from source but I'd bet that it just compiles all to one directory then I run it from that directory.
Sadly, no.
Most traditional apps and libraries based on the configure-build-install model also spew themselves over ‘lib’, ‘bin’, ‘include’ (and so on) directories.
The trick is you can change the ‘prefix’, usually by passing it to configure:
./configure --prefix=/anywhere
and the app will plonk itself into ‘/anywhere/lib’, ‘/anywhere/bin’ and so on. Prefix defaults to ‘/usr/local’, which is left as a place for you-as-admin to install any programs you want all users to have access to, but which are not part of the OS/distribution.
Dumping all your local programs into /usr/local is reasonable where there are only a few of them, but starts to get unmanageable when there are lots, or some of them start dropping insane amounts of files. (For example, MySQL and Apache both dump a horrible mess of little tools into .../bin.)
In this case you can make a single prefix for each app, such as ‘/usr/local/myapp’ or, commonly, ‘/opt/myapp’. This allows you to deal with the application as a single unit, so you can get rid of it by deleting it. But it's not a standalone unit you can rename or move, because the linker likes to know full pathnames.
You can also, as an unprivileged user, drop into a prefix in your home directory such as ‘/home/me/.local/myapp’. You can then add ‘/home/me/.local/myapp/bin’ to your PATH environment variable to be able to run it just by typing its command. If you are building a library other applications are going to have to link to, things are more complicated still, as you will have to tell them where to find it so they can get the ‘lib’ and ‘include’ files they need.
Distro-owned packages are (usually) compiled into the prefix ‘/usr’ — ‘/usr/bin’ contains all the commands provided by all the programs on the system, ‘/usr/lib’ contains all the libraries, and so on, each folder containing a thousand programs mixed together in a confusing heap.
Mess with these directories at your peril. Managing this maze is what package managers (such as apt) are for. It's a hard job, and sometimes the package manager gets confused and screws up leaving you in dependency hell, Linux's analogue to DLL hell. It's fun!
Of course this is all absolutely unsatisfactory, but you can't criticise it because It's The Unix Way™ and if you don't like it you should go back to Windows, you luser™. There are some efforts to improve the Unix packaging experience, such as GoboLinux and 0install, but they're still pretty much fringe activities at the moment. Most distros think that their package manager is the only tool you'll ever need, and can't imagine that you might want to install something that doesn't come from their repositories.