For the longest time, I wanted to start learning about issues related to concurrency, and how to handle them in Lisp. I kept reading around to learn about different approaches to writing concurrent code, and studied up on things like Software Transactional Memory, futures/promises, and Erlang-style actors. After boggling to understand all these different approaches, I was left with a bit of disappointment: None of them seemed to be the right combination of generalized+clean+easy.
STM caught my eye for a while, but it still has its issues, and I’m still not sure how I feel about the whole “thrash until you can agree on something” issue — it seems to me like it’s just the same as wrapping code in (with-lock-held …). Futures/promises looked easy enough for simple one-shot tasks, but that’s not necessarily what you want to do when you write heavily-parallel code: Sometimes you want threads constantly yielding values. Erlang-style actors are, of course, one of the big success stories when it comes to people trying to write heavily-parallel code. It just seems so damn ugly — and it -is- nice to be able to share data sometimes.
Then, magic happened: I came across something called Communicating Sequential Processes, through Rob Pike’s Google Tech Talk on Newsqueak. It was really amazing, it seemed to be just what I needed: a relatively low-level synchronisation mechanism that works naturally with the concept of multiple parallel threads/processes. I’m totally sold on Rob’s point of view here: don’t try to pretend you’re not parallelizing code (hear that, STM? That’s right). Instead, make the parallelization part of your interface. Deadlocks are a bug — find them.
I spent a good while thinking about CSP, wondering why the heck everything else is the talk of the town. The more I thought about it, the more I was amazed that CSP wasn’t everyone’s obvious choice when it came to parallelization. I kept wondering why no one seemed to even know about it. Then again, it’s not entirely obscure — Stackless Python uses this exact same concept. Clearly, the only way I could really get a good idea of what CSP is all about, and what its advantages/disadvantages are was by actually using it.
As it turns out, there was already an existing library for CL that implemented these: csp. The API, unfortunately, was a bit weird, and I couldn’t get myself into really using. I just didn’t get it — is CSP a red herring meant to distract me from some other Right Thing? I figured the only way to find out, then, would be to implement my own library.
Enter ChanL, my attempt at implementing channels in CL. I’ve been working with Adlai on these for a couple of weeks, and we just tagged v0.4. As the library developed, and we started writing examples, things started becoming more and more clear. Channels, as it turns out, are just high-level enough to make a lot of tasks extremely easy that would otherwise be a bit obnoxious to write with locks.Well, sort of: it turns out to be a little tricky to convert existing, very-linear code into channels unless you’re used to them (STM certainly wins in that front, I have to say)… but once you’ve embraced the concurrent nature of your algorithm, channels become a powerful beast.
The most interesting part, I think, is that channels turned out to be extremely useful for implementing -other- sorts of (sometimes simpler) concurrency mechanisms. Futures/promises, for example, were implemented in about 20 lines of code. The whole chanl/examples/futures.lisp file is 93 lines long, including comments, several convenience macros for creating futures, a select function, and a parallelized version of LET, as well as a fully-functional cross-thread condition-handling mechanism. That’s a lot of stuff, and it was made simple because of how nice channels are. Actors can be implemented similarly, and, in fact, I’m working on them already, although the code isn’t online yet.
All-in-all, after working with channels for a few weeks, my impression is that they’re much closer to a generalizable solution to handling concurrency than all these other constructs that keep popping up (again, perhaps with the exception of STM, which has different issues). I’m still honestly perplexed as to why channels aren’t more widely accepted and used. Perhaps it’s high time that changed.
If you ever feel like messing around with channels, go ahead and try it out. It’s asdf-installable with (asdf-install:install :chanl), or you can download it at Github. It’s got a fully-documented API, and you can even write your own unique channel styles fairly easily.
Next time: ChanL code samples/examples.