What exactly is metaprogramming?

Question

I was reading an article on TheServerSide on ployglot programming on the Java platform. Some comments in the article refer to metaprogramming as the ability to generate code (perhaps on the fly).

Is metaprogramming the ability to generate code on the fly or is it the ability to inject methods and attributes into existing objects at runtime (like what some dynamic languages like Python, Ruby, and Groovy allow).

edited--- Thanks to everyone who answered. Reading all the answers have helped clarify what metaprogramming means. Since the answer emerged after reading several answers, it is hard to select an answer as the 'selected one'.

Answer 1

Metaprogramming is the writing of computer programs that write or manipulate other programs (or themselves) as their data, or that do part of the work at runtime that would otherwise be done at compile time. In many cases, this allows programmers to get more done in the same amount of time as they would take to write all the code manually, or it gives programs greater flexibility to efficiently handle new situations without recompilation. (Source.)

Basically, it's writing code that outputs more code, which is run to accomplish some goal. This is usually done either within the same language (using javascript to create a javascript string, then eval it) or to emit another language (using .NET to create a windows batch file).

Answer 2

wikipedia has a nice article on the topic. One does not have to do runtime modifications for something to qualify as metaprogramming. For example, many people use C++ templates to do metaprogramming at compile time.

Answer 3

Metaprogramming is writing a program which outputs another program. This is something languages like Lisp are really good at. It is much easier to do in a language that supports real macros (not C++ macros, but rather ones that can manipulate the code they output) such as Ruby, Lisp, Scheme, etc. than in a language like Java.

One implementation is to create a "domain specific language" which is a way of enhancing a programming language to accomplish a specific task. It can be incredibly powerful if done correctly. Ruby on Rails is a good example of this sort of programming.

If you interested in exploring this method, check out the Structure and Interpretation of Computer Programs which is one of the seminal books covering the subject.

Answer 4

Metaprogramming refers to a variety of ways a program has knowledge of itself or can manipulate itself.

In languages like C#, reflection is a form of metaprogramming since the program can examine information about itself. For example returning a list of all the properties of an object.

In languages like ActionScript, you can evaluate functions at runtime to create new programs such as eval("x" + i).DoSomething() would affect an object called x1 when i is 1 and x2 when i is 2.

Finally, another common form of metaprogramming is when the program can change itself in non-trivial fashions. LISP is well known for this and is something Paul Graham championed about a decade ago. I'll have to look up some of his specific essays. But the idea is that the program would change another part of the program based on its state. This allows a level of flexibility to make decisions at runtime that is very difficult in most popular languages today.

It is also worth noting that back in the good ol' days of programming in straight assembly, programs that altered themselves at runtime were necessary and very commonplace.

From Paul Graham's essay "What Made Lisp Different":

Many languages have something called a macro. But Lisp macros are unique. And believe it or not, what they do is related to the parentheses. The designers of Lisp didn't put all those parentheses in the language just to be different. To the Blub programmer, Lisp code looks weird. But those parentheses are there for a reason. They are the outward evidence of a fundamental difference between Lisp and other languages.

Lisp code is made out of Lisp data objects. And not in the trivial sense that the source files contain characters, and strings are one of the data types supported by the language. Lisp code, after it's read by the parser, is made of data structures that you can traverse.

If you understand how compilers work, what's really going on is not so much that Lisp has a strange syntax as that Lisp has no syntax. You write programs in the parse trees that get generated within the compiler when other languages are parsed. But these parse trees are fully accessible to your programs. You can write programs that manipulate them. In Lisp, these programs are called macros. They are programs that write programs.

Programs that write programs? When would you ever want to do that? Not very often, if you think in Cobol. All the time, if you think in Lisp. It would be convenient here if I could give an example of a powerful macro, and say there! how about that? But if I did, it would just look like gibberish to someone who didn't know Lisp; there isn't room here to explain everything you'd need to know to understand what it meant. In Ansi Common Lisp I tried to move things along as fast as I could, and even so I didn't get to macros until page 160.

But I think I can give a kind of argument that might be convincing. The source code of the Viaweb editor was probably about 20-25% macros. Macros are harder to write than ordinary Lisp functions, and it's considered to be bad style to use them when they're not necessary. So every macro in that code is there because it has to be. What that means is that at least 20-25% of the code in this program is doing things that you can't easily do in any other language. However skeptical the Blub programmer might be about my claims for the mysterious powers of Lisp, this ought to make him curious. We weren't writing this code for our own amusement. We were a tiny startup, programming as hard as we could in order to put technical barriers between us and our competitors.

A suspicious person might begin to wonder if there was some correlation here. A big chunk of our code was doing things that are very hard to do in other languages. The resulting software did things our competitors' software couldn't do. Maybe there was some kind of connection. I encourage you to follow that thread. There may be more to that old man hobbling along on his crutches than meets the eye.

Answer 5

Well, metaprogramming is just programming, but it's basically "writing code that writes code".

The ability that you mention, when a program can observe and modify its own structure and behaviour is called reflection and it's a type of metaprogramming.

Dynamically typed languages, have powerful runtime reflection features, made possible by the interpreted nature of these languages...

Static typed languages also have powerful metaprogramming techniques, for example the C++ template metaprogramming...

Answer 6

This is just my personal opinion. Which is probably the most liberal definition of metaprogramming.

I think it includes:
1) Compile code generation or Runtime code generation (or both)
2) Aspect-Oriented Thinking or Aspect Oriented Programming
3) DRY Thinking

I think you can get there by using any of these and in combination:
1) Reflection
2) DSLs (Domain Specific Languages)
3) Attributes (.NET) or Annotations (Java)
4) Generics (.NET/Java)
5) Templates (C++)
6) method_missing (Ruby)
7) closures / first class functions / delegates
8) AOP - Aspect Oriented Programming

I like some of the answers also. Just thought I would throw in the 10,000 foot view.