For the longest time, I wanted to start learning about issues related to concurrency, and how to handle them in Lisp. I kept reading around to learn about different approaches to writing concurrent code, and studied up on things like Software Transactional Memory, futures/promises, and Erlang-style actors. After boggling to understand all these different approaches, I was left with a bit of disappointment: None of them seemed to be the right combination of generalized+clean+easy.
STM caught my eye for a while, but it still has its issues, and I’m still not sure how I feel about the whole “thrash until you can agree on something” issue — it seems to me like it’s just the same as wrapping code in (with-lock-held …). Futures/promises looked easy enough for simple one-shot tasks, but that’s not necessarily what you want to do when you write heavily-parallel code: Sometimes you want threads constantly yielding values. Erlang-style actors are, of course, one of the big success stories when it comes to people trying to write heavily-parallel code. It just seems so damn ugly — and it -is- nice to be able to share data sometimes.
Then, magic happened: I came across something called Communicating Sequential Processes, through Rob Pike’s Google Tech Talk on Newsqueak. It was really amazing, it seemed to be just what I needed: a relatively low-level synchronisation mechanism that works naturally with the concept of multiple parallel threads/processes. I’m totally sold on Rob’s point of view here: don’t try to pretend you’re not parallelizing code (hear that, STM? That’s right). Instead, make the parallelization part of your interface. Deadlocks are a bug — find them.
Continue reading