Language and Dictatorship
The Arc forum has become my favorite list lately. There’s the usual stuff going on: feature requests, requests for help with specific coding problems, code examples, polls, challenges, and so forth. In this sense, it’s more or less like any other hacker forum. But there’s something else going on that’s very exciting.
Arc isn’t the first attempt to re-imagine Lisp outside of the Scheme and Common Lisp standards, but because Paul Graham — a very important voice in the Lisp community — is behind it, I think there’s an unusual amount of attention being paid to it. It’s also not a finished product (far from it), and so I think everyone is having a lot of fun trying to imagine what it might be. Graham’s writing the code, obviously, but the forum is full of bold thinking about where it might go, and I think that many of these ideas will come to influence the future of Arc.
One of the more interesting subjects to come up lately has been the issue of “dictatorship” versus what we might call “pluralism.” Now, when hackers use the word “dictatorship,” they don’t mean that in a bad way. When we say that a language is run by a dictatorship, we mean that its canonical form is determined by an individual or by a core group of developers. Perl is run this way, as is Ruby and Python (by Larry Wall, Yukihiro “Matz” Matsumoto, and Guido van Rossum, respectively). There aren’t dozens of implementations of these languages; there’s really only one of each. Few such languages are formally standardized, because there isn’t really a need to do so. If there were one relational database implementation in the world, I doubt very much that anyone would feel the need for an sql standard. In a sense, standardization is a bit like radical Athenian democracy. We standardize not because everyone agrees, but because everyone does not agree. Various parties compromise in order to prevent any one faction from overwhelming the others. (This is in part why I find the impulse within the xml community toward creating “standards” for things that are not widely used or are brand new to be entirely baffling. Now that’s dictatorship!)
Until I started writing Lisp, I hadn’t really encountered anything other than dictatorship. One might argue C is run through pluralism, but the platforms I’ve worked on basically have one canonical implementation of the C compiler (gcc), so the effect is pretty much the same. But languages like Scheme and Common Lisp are quite different. There are standards for both languages, but there are dozens of implementations. And in the case of Scheme, the various implementations can differ wildly while still adhering to the (deliberately minimal) r5rs standard. Common Lisp has a much broader and more comprehensive standard, but even there, if you want to write code that can run under any implementation, you’re probably going to have to write some code that will check to see how things work in each of the major implementations.
Some people find this quite intolerable. They’d really like to be able to write code in cl or Scheme and have it run on any compliant implementation. Such people find the situation of Scheme to be particularly onerous, since the diversity of implementations really means that we need to speak not of Scheme, but “plt Scheme scheme,” “Chicken Scheme scheme,” and “Bigloo Scheme scheme.” For these people, Arc feels like a chance to stop the chaos.
I’m personally not bothered by this, simply because I find that the kinds of things I write don’t need to run on multiple implementations, and the implementations I use tend to have everything I need. Most of the coding I do is intended for my own use, and while it would be nice not to have to ask my users to install a particular implementation, I don’t think many people find that qualification onerous. However, I can well understand why others would find this very frustrating. And everyone (including me) finds the situation at least mildly frustrating when it comes to third-party modules. If you write a module for doing, say, xml, shouldn’t it run on any Scheme interpreter/compiler out there?
I honestly don’t think Arc will achieve this, and I think the reasons involve an interesting mixture of the cultural and the technical.
The diversity of Lisp implementations is breathtaking. Some run as interpreters, some compile to C, some compile to native code, and some do all three. Some are designed to be embedded, others more or less ignore the os. Some are designed to be “small and pure,” while others try to build into the language everything that would normally be in third party libraries. Some emphasize the ability to work with C libraries, while others are focused on the Web. And of course, there are versions not only for the major operating systems, but for handheld devices, the Java Virtual Machine, and microcontrollers. Incompatibilities abound, and yet if you’re, say, a Scheme programmer, you can probably find an implementation that seems highly optimized for whatever you’re trying to do.
All of this is facilitated by the most salient aspect of Lisp itself. Lisp has been aptly called “a programmable programming language,” because unlike the descendants of Algol, Lisp offers the programmer the ability to alter the syntax of the language itself with very little effort. In his book, Peter Siebel talks about how with most languages, the implementation of a new language feature almost always involves a drawn out process (which, in the case of a dictatorial language) can always be vetoed. And it’s easy to see why. A new language feature is almost certainly going to involve changing the compiler or the interpreter itself (and perhaps the external libraries as well). With Lisp, you just add it. This puts Lisp in the enviable position of being able to tack on — in a kind of borg-like fashion — any new trick that comes along. If you like Ruby’s for-each loop, and would like to add it to Scheme, you can do so in about a five minutes. But the same goes for object-oriented programming, aspect-oriented programming, meta programming, annotations, or whatever else is the flavor of the month. You don’t even have to settle on a particular style of, say, oo. You could write six different styles of oo and let the programmer just choose one that strikes his or her fancy. (One recalls Alan Kay’s famous quip, “I invented the term Object-Oriented, and I can tell you I did not have C++ in mind”). In a sense, every non-trivial Lisp program is a fork of the language.
Because Lisp is this way, it’s very hard to get people to settle on an implementation. If you don’t like the Perl module system, you have a couple of choices. You can either re-implement Perl or you can just suffer with what you have. With Lisp, it’s just too easy to change something like that, because it’s easy to change pretty much anything. And Lisps are (comparatively) easy to implement, so the low barriers to change extend even to matters that relate to the compiler itself. In fact, one could argue that this is precisely what Graham is doing — using Lisp to create a new Lisp that he likes. I suspect I’ll like it too. And so will lots of other people. But there will be many people who sorta like it. And they’re going to turn it into something they love.
Graham’s going to implement Arc in a way that makes sense to him. Being Paul Graham, he’s probably going to implement Arc in Arc (this, after all, is the guy who named his company after the Y combinator). I predict that people will then complain that it’s too slow — or too big, or too minimal, or too maximal, or too terse, or something else. But there just won’t be any strong disinsentive to go do something about it. And so a thousand flowers will bloom.
Of course, there are lots of things Graham and others could do to ensure that code that runs on one implementation runs on all the others, but I’m not sure that dictatorship, per se, is going to be a workable answer to the problem. Lisp programmers, perhaps more than any other kinds of hackers, really don’t like being told what to do. And they have the language to back them up.