How to write a programming language ?

Question

Hi I would like to know which books to recommend writing a programming language.

Someone has already developed a programming language ?

Which were the biggest problems or issues unresolved ?

---

Related SO question

SO - How to write a compiler

Answer 1

Learning Assembly can be a first good step if you want to build something really near of the computer. The link in this post will give you free reference to start with.

You can always build something easier by calling C/C++ code by creating an Interpreter. You have many direction you can take.

Answer 2

"Compilers: Principles, Techniques, and Tools" is pretty popular for writing compilers and goes into detail for language design. It's commonly referred to as "The Dragon Book"

Answer 3

Actually I think parsing the language is the biggest challenge, but with the new dynamic languages that are coming up, where you can pretty much create a new language out of an existing language, all the rules are changed.

Answer 4

"Writing a programming language" can include everything from designing the language, including developing the grammar, parsing that grammar (including lexical analysis, tokenization, and constructing abstract syntax trees (ASTs)), and then either compiling the AST to another laguage (e.g., bytecode) or interpreting it and executing it at runtime (or both). You can also create your own execution environment (e.g., a stack-based virtual machine).

You don't have to do all these steps, but at a minumum, I guess you want to:

1. write a grammar for the language
2. use the grammar to parse input text into a parse tree
3. convert the parse tree to an executable form
4. run the program

Answer 5

I would learn flex and yacc, I took a Programming Languages class in school a few years ago and we wrote a small language using flex yacc and SML. It was kind of a weird toolchain but all in all we implemented a small functional programming language (with static type checking) in just a few hundred lines.

Answer 6

Great exercise for learning but I still think modifying an existing language is a better idea. Unless you really need to do something completely different.

Answer 7

Types and Programming Languages is a good start along with Programming Language Pragmatics.

Maybe a compiler -- Dragon Book

Answer 8

Your biggest hassle is going to be parsing, type checking, and correct output.

Answer 9

The Dragon Book has been mentioned, but that goes over more of how to write a compiler, which I think is a slightly different question. Anyway, it's bordering on obsolete, even for that purpose (unless it has gotten a significant update in the last few years?).

If you are interested in a book that covers designing a language (separate from actually writing its compiler), the only one I ever came across is Nicklaus Wirth's Project Oberon. Wirth has created several programming languages, including the very influential Pascal, so he does know what he's talking about.

He also covers his design of a small Operating System, for those who are interested in that kind of thing. However, there are other good books covering that topic too.

Answer 10

This question is somewhat related.

Answer 11

The language I wrote was in old school 'C' (about 8 years ago) which was an interpreter (actually more like a pre-compiler but it could be used as an interpreter), and used Flex and Bison (Lex and Yacc). The Lex and Yacc book is an excellent companion to the Dragon book (which I agree was getting long int the tooth 8 years ago) if you want to go the old school root.

If I had to do it again, I would definitely write it in Antlr in your language of choice (as long as that language is once of Java, C#, C, Python, ActionScript, Javascript -- it supports more but they are not shall we say, as supported). The author of Antlr has written an excellent book on Antlr, which is available from the Pragmatic Programmer guys. It also has an excellent IDE which really helps in the understanding of a how the parser/lexer and grammar works. I wish I had a tool like it when I was slogging through thousands of lines of debug output from Bison.

Answer 12

Define what "writing a programming language" means to you. Just some syntax parsing? Will it generate directly executable code (compiled) or will be be executed by some intermediate layer (interpreted)?

I wrote my own programming language before, I wanted it to be compiled (not interpreted), but I didn't want to write my own compiler (which is really cumbersome if you want to support more than just one architecture, that is more than one operating system and more than one kind of CPU). Actually even worse than a compiler seemed the idea to write my own linker. So I end up doing it the cheap way :-P My language code is translated by a translator (a syntax parser and reformater one could say) into plain, simple C code and this code is then passed to GCC to compile and link it. That way my language can immediately generate code for any platform GCC supports (and there are plenty of these).

Answer 13

I'm going to assume that the question means "how does one design a programming language"?

The first step is knowing what problem you want to solve. The second step is to understand why no existing language solves this problem. The third step is usually to identify an existing language that's closest to solving your problem, and what needs to be changed about that language to make it suitable.

Obviously, the more different languages you know about -- and more importantly, the more different kinds of language and language features you know about -- the better off you are.

An invaluable resource in this kind of endeavor is set of papers published in the proceedings of the three ACM/SIGPLAN History of Programming Languages conferences. These papers are usually by the authors of the language, and the languages are generally the important ones. (For instance, Dennis Ritchie wrote about the history of C.) These are lengthy articles that explore the motivation, implementation, evolution, and evaluation of a programming language.

As one online example, you might want to take a look at "The Evolution of Lua", by the authors of that language.

Answer 14

Yet another way: you can use the power of Lisp metaprogramming and create your own unique "sublanguage" :)