Language Workbenches

In the article “Language Workbenches: The Killer-App for Domain Specific Languages,” Martin Fowler introduces the concept of “language workbenches.” This is an attempt at a summary.

A typical enterprise software project includes various languages. Besides the programming language—Java, C#, or whatever—there is usually some kind of file mapping a relational database to code, there exists some kind of mapping between code and user interface, whether that is in HTML or a GUI. In addition, there are portions of code that maps incoming or outgoing files to internal data structures, and these also qualify as languages, as Fowler shows in his article. And finally there are sometimes, as advocated by the book The Pragmatic Programmer, domain-specific languages that non-programmers can read and write.

Fowler suggests that all these files in different languages together describe one larger structure. These various languages can be viewed as concrete syntaxes for one abstract syntax that is particular to the project in question. So in an enterprise software project, there’s one large structure, described by database mapping files, UI mapping files, files mapping input and output formats, and so on—besides the regular source code files.

This model already exists in modern IDEs, what Fowler calls the “post-IntelliJ IDEs.” These, he writes, create

an abstract representation in-memory, which [they use] to help you edit the [source code files]. [...] This abstract representation helps with simple things like method name completion and sophisticated things like refactoring (automated refactoring is a transform on the abstract representation).

Eclipse does this, to name one popular example.

Language workbenches would take this model further. Or turn it around 180 degrees. Instead of creating a transient abstract representation from the source files, as IDEs do today, a language workbench would treat the abstract structure itself as the source, and use it generate the source code files (or “project into” as Fowler says). Java files would be transient and only exist as long as the programmer needs to make modifications to them.

To give a sense of what this could be like, see the following passage from Fowler’s article:

My colleague Matt Foemmel described how this struck him one time while working in IntelliJ. He made a change that was strongly assisted by these features, and suddenly realized that he wasn’t typing text—instead he was running commands against the abstract representation. Although the IDE translated these changes in abstract representation back into the text—it was really the abstract representation he was manipulating.

But the interesting thing with making the abstract representation be “the source” is that this allows you to project the abstract structure into any form of editable representation. Some manipulations of the abstract representation might work better in graphical form, for instance, and in those cases the language workbench would project parts of the abstract structure into some form of graphical file format, that can be parsed back after the modifications are done. And projections could be invented that non-developers can understand and manipulate. So “langauge workbench” means a tool that is capable of translating the single abstract structure into several different types of languages.

There are two principal advantages offered by this approach, Fowler writes:

One is improved programmer productivity by giving them better tools for the job. The other is improving the productivity of development by forging a closer relationship with domain experts by allowing domain experts more opportunity to contribute directly to the development base.

Of these two, Fowler believes that improved programmer productivity is more likely to happen, although it “carries less impact.” What would really improve things is to enable developers and domain experts to work more closely together.

If this sounds interesting, go ahead and read the article. Also see my previous post on Software Factories, which offers one form of “language-oriented programming.”

(This post was cross-posted to our company weblog.)