When talking about programming languages, what is the definition of Magic?

Question

The word "magic" gets thrown around a lot here in contexts like "language X just has too much magic", or "platform Y generally avoids magic". However, it seems the term is pretty poorly defined, something people know when they see it. For example, Java is reputed to contain very little magic, but its garbage collector hides a lot from the programmer. If magic simply means abstraction that hides details, then why is it considered a bad thing, given that nobody writes large programs in assembly language anymore? If magic means something more, then what does it mean?

Answer 1

For Java

• GC - IMHO not magic, it's just like having a house cleaner
• Auto-boxing - magic - invisible transmogrification of a thing into something else

Answer 2

Something is magic while you ignore it / you don't know how it works.

Answer 3

"any technology sufficiently advanced is indistinguishable from magic" - Arthur C. Clarke.

Actually not quite that - magic is used for complicated and hidden rather than advanced (though presumably the designers thought they were being advanced), such as systems which require specific states before procedure calls (COM threading models) as well as 'automagic' type conversions (VB variants, Javascript ==, Java autoboxing).

Once systems hide details from the programmer that they are no longer predictable, they become magic. You're stuck with repeating an invocation in a language you don't understand because it's had the right effect in the past. That's bad magic, or voodoo.

There's also good magic - http://aggregate.org/MAGIC/

Answer 4

Magic has the same meaning than in other context : something with hidden complexity and logic. It's not always bad, but for programmers (that are magicians for non-techie people) it is. Magic is not magic when you (can) know how it work the way it look.

For example, magic constants : constants with a hidden meaning.

When we talk about "black magic" occurring in some obfuscated code, we mean that something happen in this code that we don't understand; or that something non-written happen but is hidden from the code reader.

In some cases, you talk about magic in a positive way : when you don't see the real complexity of a simple to understand how to use thing. For example, some really complex code can be hidden in a tiny and simple to use interface class. That seems magic because it does complex thing with simple usage. That is good magic (but rare).

Answer 5

Ruby on Rails, for example, is Magic.

At the basic level, especially when approached naively, there's a big thick layer between your minimalist code and the database, and then Magic Happens, and you get crud functionality and all the rest.

Some folks can live with that very high level experience, but eventually you'll start running in to barriers where you need to understand the "secret" of the "trick", so you can manipulate it betters towards your end.

Languages themselves don't have a lot of Magic (well, maybe Prolog -- it's core is somewhat magical), but the modern frameworks and middleware can be chock full of it.

Frankly, anytime you encounter a system or component that does "something amazing" and you wonder "how do they do that", it's magic.

Answer 6

Unknown behavior is very often spoken of as magic. Some things happen for no apparent reason.

But there are tho kings of unknown:

• The lack of knowledge on the developers side on how things work. (For example GC, it's not magic, just too few developers actually have bothered to find out and understand it's underpinnings)

• Undefined behavior magic, where things happen behind the curtains and there's no public knowledge on how they actually work. This is the true magic in programming.

For example Rails, it's not magic and Ruby is not magic, but it certainly may seem so for some of people if they never bothered to understand the internals (which are public knowledge and available).

Answer 7

At Microsoft this tends to be used two ways

1) In a negative way (probably the most common usage). For example, someone might say the following "I don't understand; first you will refactor the API, next you will keep the old API and have a layer between them that performs magic?"

The implication is that the magic component is too complex, too undefined, or too 'something'.

The other negative context is when something is defined naively: This often happens when someone assumes there is a simple solution to a complex problem because they do not realize that the problem is really complex. Raymond Chen's blog is full of these kinds of examples.

2) Its less often that the term 'magic' is used positively. But it does happen. Usefully to refer to things that perform complex tasks that just work. To me NTFS is that kind of magic. NTFS is really, really mature and hides a lot of complexity behind some pretty simple APIs. Compiler's can be seen as 'magic' in this way as well - do you really understand, deeply, how a C++ compiler works? I don't - I just trust that they do.

Answer 8

Magic is very relative to your understanding.

I wrote a fair amount of C and C++ before I started writing in Java. I had to allocate and deallocate all of my memory. It wasn't a lot of fun once I started getting memory leaks. Once I started writing in Java, I knew (basically) what was going on behind the scenes in order to make all my instance objects. Thus, to me, it's not near as much magic (I've done allocation but how it actually stores objects and knows when to deallocate them is a little more "magical" to me) as it would be to someone who had started on Java (who has never had to manually handle memory.)

Another example is ORM. I rolled my own temporary Object Relational Mapping structure early in a project until I had time to really buckle down and learn Hibernate or something similar. I had to write in things like ResultSet mappers, Lazy Loading and some basic caching. Now, when I go back and replace this stuff with more mature Hibernate or JPA mapped entities, I'll still have a basic understanding of what's going on. If you've never used JBDC or anything similar, all you're going to really know is that the data goes from the database to the object and back again.

Everyone deals with a certain level of magic in their careers. We can't know everything. Magic really can be synonomous with Abstraction. When it becomes "too magic" is when the Abstraction begins to cover up things that you need to be able to control. An example the other day was that in a JSF program, I didn't know how to start a JSF context. JSF just does this when you navigate to a JSF page for the first time. I needed to start the context from a regular servlet. I ended up having to make a dummy JSF page to handle it instead. I simply didn't have time in this project to learn the "magic" of JSF context management.

Answer 9

An interesting aside: the term magic is linked to formal programming languages, which are one of the theoretical foundations of computer science. In Morgan's book "Programming from Specifications", we have this definition of magic:

w:[true, false]

This notation is of the form:

w:[P, Q]

and represents a program that, when executed in a state where P is true, will terminate in a state where Q is true, whilst changing only the variables in w.

In this context, then, "magic", is an infeasable program that can take a program from any state (since true is always true) into one that is impossible (since false can never be true).

A much easier program is :

w:[true, true]

which is the program that always terminates, but guarantees no particular result. This can be implemented with a simple skip.

Obviously, this technical definition has been abused when people talk of magic programs, since this ought to mean a program that is provably infeasible. Nevertheless, the term remains colloquially, since it's nice to imagine that something magical happens behind the scenes when we don't know, or understand how something works.

Answer 10

One context where I use the term "black magic" is bytecode manipulation (in Java) - transforming the program code at runtime so that it does something different than what the plain code would do. Often times bytecode manipulation is very useful, but it's also very easy to shoot yourself in the foot with it.

Many frameworks and middleware use AOP and bytecode manipulation to implement cross-cutting concerns. This reduces the programmers work, but in some situations it may make debugging the behaviour of the system hard. And it's easy to have some subtle bugs remain in the bytecode transformations.

I have done some bytecode manipulation myself, so basically I know how the bytecode instructions can be changed and it's not "magic" to me in that sense, that I would not understand how it works. But still I refer to it as "magic", because it changes the way the code normally works and is not visible in the source code of the classes being changed.

Answer 11

You call something "magic" when you don't understand it.

People were used to be burned in middle ages when they where doing / knowing / saying something, that the rest of men coundn't understand. They were accused of doing "magic".

You call code / framework / language a "magic" when you don't understand what it does. It's easier to call something "magic" than to spend time to understand it.

Magic Does Not Exist!

Instead of calling something "magic" get your hands dirty and understand what that thing is doing... Or call it "unexplored part of (your) knowledge".

Answer 12

For some nice examples of black magic, look at the documentation in the perl source code ;-)

Answer 13

The classic magic story from the jargon file appendix.

http://catb.org/jargon/html/magic-story.html

go read it.