What was Tim Sweeney thinking? (How does this C++ parser work?)

Answer 1

Can't tell for sure, but the C++ code kinda sorta looks like Spirit, a C++ parser generator that makes extensive use of templates. Test.ae looks like it's metaprogramming (defining language details in the language itself), which is harder to do in C++ (templates are a start, but is error prone and ugly) than it would be in some other target language (e.g., UnrealScript, which is what I assume test.ae is written in).

So - it looks like Parser.cpp defines the base grammar for UnrealScript (using Spirit), and Test.ae is defining extensions to UnrealScript.

Answer 2

I don't know what Sweeney did, and I'll assume that other answers about using Spirit are in, uh, the right spirit. I have no experience with Spirit templates, but my understanding is that if you define a complex grammar with it, it becomes pretty difficult to handle (as well as slow to compile). Other people's actual experience should be used to guage the truth of this.

There are other ways to implement extensions to C++, e.g., using program transformations and extendible grammars. See this SO answer on augmenting the C++ grammar itself with arbitrary extensions, where very complex extensions are possible and were in fact used.

Template metaprogramming generates interesting code where the templates are specifically invoked. Using program transformations you can generate arbitrarily interesting code at any point in the program, e.g, its as if the "templates" (additional syntax) changes the semantics any way you think is useful.

Answer 3

The screenshot is obviously from an MSVC 6.0 or earlier time frame, which did not really appreciate complex templates (and certainly did not support partial template specialization). I've not used spirit. From these screenshots, it's impossible to tell what Sweeney is truly doing beyond defining what looks to be a DSL in Test.ae.

The only full C++ statements you can see are in Parser.cpp - and they don't tell you much of anything except he's declaring 3 types. You really can't tell much of anything - too much is obscured by the 'Test.ae' window.