No, view selection formulas cannot contain LotusScript
Not with a view, but you're onto something there. I've done that in the past using folders. Views contain a set of documents that match a view selection formula. Folders, on the other hand, can contain a collection of documents placed there by a user or by code. In the Lotusscript world, you access a folder the same way as a view, using the NotesView class. Then you can call the document's PutInFolder method or a documentcollection's PutAllInFolder method to add documents to that folder.
No. You can programmatically specify the formula used by views, though, via the SelectionFormula property. But that property expects a string written in formula language.
I'm not very familiar with the C API, but my bet is you can only do the same as #3 - specify the formula used as the selection formula. It won't help you select documents based on LotusScript code.
I've been down this road before in a few projects, and the best solution I found was what I suggest in answer #2. That is to say you can clear and then populate a folder programmatically, and then take the user to that folder as the final step in your code. Using that method you are free to use LotusScript to filter the documents and build your "query results" view.
Another less ideal, but functional method you can use is to stamp one item within all the documents with a special value, then filter a view to show only documents with items equal to that special value. For example, you can run a search in LotusScript to build a documentcollection using the db.Search method. Then use the documentcollection.StampAll method to set the "SHOWME" item to "YES". Then your view will just have to be set to show only documents where SHOWME = "YES". Of course, you'll need to remove that SHOWME item from all documents in the database as a first step in this code, so you're starting with a clean slate each time. This method would get increasingly slow the more documents you have, but if you only have a few hundred docs it would work fine.
Hope this helps!