This is more general than just building websites. This is good practice for any large set of files you have to organise. http://stackoverflow.com/ is the root of the web server, the root of the filesystem as far as the web server knows (not over-complicating with virtual hosts e.t.c.). stackoverflow.com/questions/ will go into the directory called questions. Usually these directories will contain index.html or index.php and a bunch of other files|scripts|pages that are required for the main page to run properly.
I don't think it affects your ranking. It only impacts your surfing if you often type exactly what you want in the URL bar rather than clicking on links from the main site. Even then, the impact is minimal. For the administrator/maintainer/developer of the website this system of building using folders and sub-folders is indispensable for any modern website.