Guys... Girls, I'm working on a project which I think could be enhanced by implementing a Domain Specific Language for defining a set of rules and/or conditions for some type of work-flow.
I want to get a firm grasp on the subject, fundamentals, best practices, etc. specially how to implement them somehow with Java.
What do you suggest?
As others have commented, Java really isn't a great choice for creating a DSL. Scala, Clojure, Groovy, Ruby/JRuby would all be great choices. However, considering you were thinking about using Java, I think Groovy or Scala seem like the most natural choices. The learning curve for java developers is quite gradual for both languages. Here are some links that will get you started:
First I would recommend reading chapter 9 (Notation) of The Practice of Programming by Kernighan and Pike.
When you have done that, come back here with specific questions on how to map the concepts in that chapter to specific designs for the problems you want to solve.
The basic pattern is to write an interpreter that is passed a 'command' argument, and possibly an 'environment' argument and executes the command (in the environment). You then have the option of writing a parser, that takes a 'script' string and converts it into a valid 'command' object (ie. an external-DSL); or you provide a library to help users build the 'command' object explicitly in the same language you are using (internal-DSL).
Kernighan and Pike do a good job of showing both how trivial and how complex an interpreter can be. If you want more depth, then I would suggest reading The Essentials of Programming Languages by Daniel Friedman et al. Which builds at least one different interpreter per chapter, and demonstrates how to implement features such as variables, functions, scopes, objects, classes, static-typing, and continuations.
However I would suggest trying your hand at a trivial DSL first, otherwise it's all just theory—a book is much more interesting when it is made relevant and practical by your previous experience.
Once I used openArchitectureware to define and use a DSL. oAW is a plugin to eclipse and now part of the eclipse modeling framework, but of course it can be used outside EMF too.
I liked it because it was fairly easy to define a DSL and oAW will automatically generate an editor with syntacx highlighting and error checking for the DSL.
And it provides a template engine that is pretty comfortable if you plan to use documents written in your DSL to autogenerate Java, XML or other files.
(I've linked the old oAW URL because the page still provides some details and all links to the eclipse project pages)