IMHO depending on your document management system the choice of structure for your documents may not be an issue. When looking at the problems project related documents are trying to solve you typically come to the conclusion that documents are about communication.
Different documents attempt to communicate different things (or contexts); test plans discuss how testing should/has been executed, requirements specifications discuss how the business rules should be applied, architecture documents discuss the technical components and so forth. Each of these documents might have the need for its own unique structure. For example the structure you choose for your test plans may be vastly different from the structure you need for your architecture documents.
When keeping the communication issue and the document context in mind I generally come back to these 2 key aspects.
- Searchability – What is the easiest way to find the document I am looking for?
- Versioning – How do I know that the document I am looking for is the most recent one?
I feel searchability is the most important thing to remember because different people call the same document by different names. For example some people call Business Requirements documents Functional Specifications. Some people call Functional Specifications use case documents. As you cannot always govern the naming convention of documents I feel finding the right document to be far more important than the folder or place in which it is stored.
So to answer your question I would simply answer by saying it doesn’t really matter which structure you use, just that you should use some form of document management system (SharePoint, Documentum, Trim, etc). The benefits are simply too great to work without one :)