What can I say? The Greatest Language Ever and the Greatest Language Possible are both doomed propositions. The former ignores the infinite variation of use cases, platforms, coding styles, and the sociology of individuals and groups. The latter does all of that and goes still further by arbitrarily choosing language features for which the “best” is likely to be a highly debatable matter. Like all good philosophical questions, these one’s don’t have concrete answers. We pose them not to arrive somewhere, but to stimulate thought and further innovation.
But Lisp is still the Greatest Language Ever.
I only learned Lisp a couple of years ago. I’m not sure how I got there, really. I think I might have stumbled on one of Paul Graham’s essays (perhaps from a Slashdot interview Kent Pitman gave some time ago). I was thoroughly intrigued by what Graham had to say, and decided to give it a shot. That, of course, led me into the Scheme vs. Common Lisp debate, as well as to the question of which implementation to use.
I think it took me about an hour to decide against Scheme. Nearly everything I read about it suggested that it was a highly minimalistic version of Lisp intended for introductory computer science courses and ponderous experiments in the study of languages and algorithms. In other words, it wasn’t meant for serious application development at all. In fact, the very first Q from the Scheme faq sealed it:
Scheme is often used in computer science curricula and programming language research, due to its ability to represent many programming abstractions with its simple primitives. Common Lisp is often used for real world programming because of its large library of utility functions, a standard object-oriented programming facility (clos), and a sophisticated condition handling system.
I started this recent series of blog posts, because I wanted to talk about why I’m both dazzled and frustrated by certain aspects of Common Lisp. None of my objections involved the language itself (I’m not even sure I follow some of the critiques of cl that show up on the newsgroups). My problems were the garden variety ones. cl folks use Emacs/Slime. I use Vim (which has very weak support for ide-style development with Lisp). And anyway, I hate ides. I want the compiler to compile (or the interpreter to interpret), the debugger to debug, the editor to edit, and I want to switch tools whenever I want. I also don’t want anything abstracting me away from the details of the tools themselves. That’s just me, but there it is.
I loved having a repl, but I hated having just a repl, and while it’s possible to get sbcl to work with a shebang, it involves a lot of hacking of configuration files. Package installation was often complex and tedious, and the library I was looking for usually wasn’t available (or if it was, it had absolutely no documentation whatsoever). I fully understood the arguments about why it’s hard to create standalone “binaries” for cl, but I wondered why there wasn’t something like a Java jar file — some easy way to launch an app with minimal fuss.
As I’ve said before, there is a way to do all of this if you’re patient. But when you read the posts about these matters, you witness a very peculiar rhetorical trend. Someone posts asking about, say, how you go about creating an “executable” with CLISP or sbcl. Someone might respond with a howto, but the person asking is much more likely to get a sermon about how they need to free their mind. They’re told about how the line between development and use is blurred in Lisp, about how “standalone programs” are actually a myth, since the operating system ships with the C runtime and is therefore providing you with mountains of scaffolding (no different from Lisp!). They’re told that the repl is a perfectly legitimate “shell” for ordinary users. And anyway, the unix shell is a horrific hack from which we should all attempt to extricate ourselves. In fact, it’s really much better to have Lisp insulated from the fallen world of shells, C libraries, conventional interfaces, etc. You just need to embrace the Lambda Nature.
I get it! I really do. I understand these arguments. In fact, some of my objections are completely unfair (No documentation? No libraries? Well surely you can do something about that!). Other points are absolutely dead on (like the one about the C runtime). It’s just that I’m trying to get my work done and make my users happy. They will not be happy if they have to fiddle with the common-lisp-controller, set up aliases for asdf, recompile fasls, and then learn Lisp syntax so they can use my program. And I’m going to be unhappy if I have to wade through source code for lack of documentation, struggle with the ffi, and spend an hour setting up my system so I can use a particular package. It may be — in fact it is — the Best Language Ever. But it’s not the Best Userspace Tool Ever. Not by a long mile.
So I started looking around for other Lisp alternatives, and this led me to a long, second look at Scheme.
I discovered that the opening salvo of the faq is absolutely true. It is a very small language, and is therefore ideal for teaching and language research. But what the faq was leaving out, I think, is the startling implications of a language that is trivially easy to implement (relatively speaking). If you’re going to implement cl, you better lock and load. It’s going to be a major job. Scheme? Well that’s a different story.
The fact that Scheme is easy to implement has led to dozens and dozens of implementations. Some of them are “pure,” in the sense that they provide the core language and don’t concern themselves with os integration. That’s perfect if you’re teaching Lisp. But what if you want to target the jvm (“for shame!” say the cl folks)? Or compile to C? Or run it on a Palm? Or use it as a systems programming language? Or embed it in another program? Or embed it in another programming language? Or put in on a cell phone? Or use it as the scripting language for your editor?
And we can go further. What about a Scheme that is designed to work seamlessly with unix? Or a Scheme that makes it trivially easy to call native libraries? Or a Scheme that can be used as a shell? Or a scheme that acts like an ordinary “scripting language?”
What I discovered, is that there are Scheme implementations for all of that. They’re all Scheme (and therefore all Lisp), but none of them try to be all things to all users (or worse, impose philosophical restrictions on how the language is used).
Imagine my delight when I stumbled across Chicken Scheme (one of several that try to fill in a particular programming niche). This is a Scheme implementation that compiles to C (using a very clever method), which means that you can very easily create the elusive “executable” with a single command, and it will behave like any other native app. Its package system is brainlessly easy to use, and it has lots of libraries to do good-old-fashioned every day stuff (talk to databases, munge xml, export to csv, etc.). In fact, it really feels like Ruby or Perl or Python. But it’s fast! And it’s Lisp!
That was enough to make me fall in love, but my real moment of epiphany came when I discovered Easyffi.
All of the cl implementations allow you call native functions through a foreign function interface. I read over some of the specifications for doing that (cffi and uffi). “Yeah, it’s doable,” I thought, “But it’s going to require some serious effort.”
Here’s how you call native C libraries from Chicken using Easyffi:
- You reference the C header in your source file.
- You call the native function (in Scheme).
Sorry, did I miss something? I mean, it’s a little more complicated than that, but not much. I sat down to write an interface to the native GraphViz library. I had it working in about a half an hour.
The mind reels. I’m speechless. So, I can just create interfaces to any C library I want (making them as Lispy as I care to with macros and whatnot)? Just by calling them? And then I can compile the programs that call those interfaces into fast C code that will behave like an ordinary posix program?
And I can do all this in the Greatest Language Ever!?
I’m a convert. I’ve known for awhile that Lisp is the red pill. Now it seems that, for me at least, Chicken Scheme (and Scheme in general) might be the little blue one.