Posts Tagged ‘lisp’

Sheeple 3.0.2

Tuesday, December 15th, 2009

It’s taken a while to get this release out — we kept adding stuff on — but it’s finally out. Even though it’s a minor version bump, there’s some important API changes with this new version, including the complete removal of ADD-PROPERTY. Originally, I thought it would be a good idea to force users to add a property manually before being allowed to set a value. Then I realized that the Python community has had this ability for a very long time. The freenode Pythonistas seemed to pretty consistently agree that it had never caused serious problems. Thus, A-P is gone, and (SETF PROPERTY-VALUE) is now used to both add a property, and to set a value on existing ones.

There is also now a manual! A pretty one in both PDF and HTML form, which can be found at our new project page on common-lisp.net. There’s also some mailing lists on that page you might want to sign up for if you feel like tracking Sheeple development.

Things have slowly been ramping up. Ever since I gave that talk at the TC Lispers meeting, a few other people have given Sheeple a whirl. I’m pleased that their experiences have been good, even though Sheeple is still a bit unstable. Of particular note is Patrick Stein’s Wooly, a GL-based GUI toolkit using Sheeple objects.

Development has slowed down a bit since I started working at Clockwork, but I’ve been working with Adlai lately towards a native-Sheeple MOP. So far, the basics of object creation have been taken care of, and a property MOP should be up and running soon. We’ve also decided to rename persistent-sheeple, but that will get announced once we have enough of a MOP to hook up our objects to CouchDB.

Release: Sheeple 3.0

Wednesday, October 21st, 2009

Sheeple 3.0 is finally out!

After months in development, it’s finally done. A major rewrite of the entire Sheeple system, this new version boasts a completely new, cleaned up, and serious’d interface, reworked, lightweight objects, and efficient property access (right now, matching CCL’s slot-value performance.) You can download a tarball here, or use asdf-install to grab the latest version.

I’m really excited about this release. I would write more about it and all the goodies it has, but I have a presentation to finish. Expect more in a later post.

Introducing: ChanL

Wednesday, October 14th, 2009

For the longest time, I wanted to start learning about issues related to concurrency, and how to handle them in Lisp. I kept reading around to learn about different approaches to writing concurrent code, and studied up on things like Software Transactional Memory, futures/promises, and Erlang-style actors. After boggling to understand all these different approaches, I was left with a bit of disappointment: None of them seemed to be the right combination of generalized+clean+easy.

STM caught my eye for a while, but it still has its issues, and I’m still not sure how I feel about the whole “thrash until you can agree on something” issue — it seems to me like it’s just the same as wrapping code in (with-lock-held …). Futures/promises looked easy enough for simple one-shot tasks, but that’s not necessarily what you want to do when you write heavily-parallel code: Sometimes you want threads constantly yielding values. Erlang-style actors are, of course, one of the big success stories when it comes to people trying to write heavily-parallel code. It just seems so damn ugly — and it -is- nice to be able to share data sometimes.

Then, magic happened: I came across something called Communicating Sequential Processes, through Rob Pike’s Google Tech Talk on Newsqueak. It was really amazing, it seemed to be just what I needed: a relatively low-level synchronisation mechanism that works naturally with the concept of multiple parallel threads/processes. I’m totally sold on Rob’s point of view here: don’t try to pretend you’re not parallelizing code (hear that, STM? That’s right). Instead, make the parallelization part of your interface. Deadlocks are a bug — find them.

I spent a good while thinking about CSP, wondering why the heck everything else is the talk of the town. The more I thought about it, the more I was amazed that CSP wasn’t everyone’s obvious choice when it came to parallelization. I kept wondering why no one seemed to even know about it. Then again, it’s not entirely obscure — Stackless Python uses this exact same concept. Clearly, the only way I could really get a good idea of what CSP is all about, and what its advantages/disadvantages are was by actually using it.

As it turns out, there was already an existing library for CL that implemented these: csp. The API, unfortunately, was a bit weird, and I couldn’t get myself into really using. I just didn’t get it — is CSP a red herring meant to distract me from some other Right Thing? I figured the only way to find out, then, would be to implement my own library.

Enter ChanL, my attempt at implementing channels in CL. I’ve been working with Adlai on these for a couple of weeks, and we just tagged v0.4. As the library developed, and we started writing examples, things started becoming more and more clear. Channels, as it turns out, are just high-level enough to make a lot of tasks extremely easy that would otherwise be a bit obnoxious to write with locks.Well, sort of: it turns out to be a little tricky to convert existing, very-linear code into channels unless you’re used to them (STM certainly wins in that front, I have to say)… but once you’ve embraced the concurrent nature of your algorithm, channels become a powerful beast.

The most interesting part, I think, is that channels turned out to be extremely useful for implementing -other- sorts of (sometimes simpler) concurrency mechanisms. Futures/promises, for example, were implemented in about 20 lines of code. The whole chanl/examples/futures.lisp file is 93 lines long, including comments, several convenience macros for creating futures, a select function, and a parallelized version of LET, as well as a fully-functional cross-thread condition-handling mechanism. That’s a lot of stuff, and it was made simple because of how nice channels are. Actors can be implemented similarly, and, in fact, I’m working on them already, although the code isn’t online yet.

All-in-all, after working with channels for a few weeks, my impression is that they’re much closer to a generalizable solution to handling concurrency than all these other constructs that keep popping up (again, perhaps with the exception of STM, which has different issues). I’m still honestly perplexed as to why channels aren’t more widely accepted and used. Perhaps it’s high time that changed.

If you ever feel like messing around with channels, go ahead and try it out. It’s asdf-installable with (asdf-install:install :chanl), or you can download it at Github. It’s got a fully-documented API, and you can even write your own unique channel styles fairly easily.

Next time: ChanL code samples/examples.

Sheeple continued: Dispatch

Monday, July 20th, 2009

In the last entry, I wrote a bit about the basics of how object instantiation and property/slot access works in Sheeple. In this entry, I’ll introduce a whole different realm of Sheeple: the method dispatch system. If you’re familiar with Javascript, it might already be obvious to you how polymorphic method dispatch can be implemented with the small bit of Sheeple we know so far:

SHEEPLE-USER> (add-property *sheep* 'foo
                            (lambda ()
                              (print "Method called!")))
#<Sheep #x15031356>
SHEEPLE-USER> (foo *sheep*)
#<Anonymous Function #x1506C446>

For those confused: the above equivalent in javascript would be:

sheep.foo = function () { alert("Method called!"); }
sheep.foo -> function object

And there you have it. Actually calling the function is just a matter of using Lisp’s version of adding () at the end of a variable:

SHEEPLE-USER> (funcall (foo *sheep*))
"Method called!"

In JS:

sheep.foo(); => an alert pops up. The method was called.

Nothing new here. It’s cute and all, but Sheeple has a much nicer facility for defining ‘methods’. I’ve opted to call these “Messages/Replies.” For those familiar with CLOS, the basic concept is exactly like Generic Functions/Methods. In fact, I would have named them the same thing, but I wanted to be able to operate in parallel with CLOS — which I do regularly. The big deal about these message/replies is that, instead of dispatching on a single value (whichever object holds the function that will be called), method lookup involves 0 or more arguments. The semantics, of course, end up being more complex than the singly-dispatched approach, but it opens up a world of possibilities and expressive power (not to mention completely obsoletes the Visitor pattern).

The concept of messages/replies is as follows: You have something called a ‘Message’ that works like a generic dispatcher. It defines the basic idea of what arguments to this function should look like, and it defines which arguments will be dispatched on. It does some other things, but the important part here is that Messages work like interfaces (no-implementation definitions) for replies that will be defined.

In Smalltalk terms, a Message is passed to not necessarily just one object, but to multiple objects, and the code that is run is whatever the collective Reply they agree on to that message is.

SHEEPLE-USER> (defmessage synergize (a b))
#<Message: SYNERGIZE #x150A23DE>

This gives us an empty message. As far as Lisp is concerned, it can be called just like any function.Right now, though, it won’t be able to do anything, (It will signal an error saying there are no applicable replies) because we haven’t defined any replies for it, so let’s define one.

SHEEPLE-USER> (defreply synergize ((a (proto 'number))
                                   (b (proto 'number)))
                (+ a b))
#<Reply: SYNERGIZE #x150433F6>

This reply specializes the arguments A and B on (proto ‘number), which is an abstract object that represents all numbers. This means that if SYNERGIZE is called with its first and second arguments being numbers, the body of this reply will execute.

SHEEPLE-USER> (synergize 5 2)
7

We did numbers, but what does it mean to ’synergize’ strings?

SHEEPLE-USER> (defreply synergize ((a (proto 'string))
                                   (b (proto 'string)))
                (concatenate 'string a b))
#<Reply: SYNERGIZE #x150B4A76>
SHEEPLE-USER> (synergize "foo" "bar")
"foobar"

So far so good. Now we can start mixing and matching and seeing what kind of behavior we end up with. A more

SHEEPLE-USER> (defreply synergize ((a (proto 'string)) b)
                (concatenate 'string a
                             (format nil "+object: ~A" b)))
#<Reply: SYNERGIZE #x150B49F6>
SHEEPLE-USER> (synergize "foo" 45)
"foo+object: 45"

By not wrapping our parameter in parens, we default to the toplevel object in the Sheeple hierarchy: (proto ‘t). This means that this reply will be applicable when -any- lisp object is given as the second argument to SYNERGIZE. This is essentially single-dispatch, except our old string-string reply will still apply, since it’s more specific than this general one:

SHEEPLE-USER> (synergize "foo" "bar")
"foobar"

As a note: dispatch is left-weighted, so higher specificity to the left of the argument list (lambda list in lisp terms) will outweigh higher specificity in arguments to its right. The algorithm itself is based on Slate’s description of Prototype Multiple Dispatch (PMD, take a look at Fundamentals). Sheeple supports much more than what I’ve shown. It has :before, :around, and :after replies (which have identical semantics to the way CLOS handles them), and has a mechanism for handling &key, &rest, &optional, and company (which, again, works identical to CLOS’ facilities).

The big difference, in the end, between CLOS and Sheeple is that Sheeple dispatches everything on actual objects. It’s like defining all CLOS methods with EQL specializers in a world where the objects themselves hold the hierarchy. The biggest implication of this, I think, is that the hierarchy never bottoms out — and that’s quite useful when you’re developing.

Next time, I’ll start talking about how development with Sheeple actually works. I’ve written a couple of applications using it, and there’s been some good lessons learned from those — which have then reshaped the way Sheeple works. Questions so far? Leave a comment! Or just check out Sheeple’s documentation.

Sheeple

Sunday, July 19th, 2009

For a few months now, I’ve been working on something I call Sheeple. What is Sheeple? From the
official description:

Sheeple is a Dynamic, CLOS-like, Delegative Prototype-based Object-Oriented Programming
Framework (or “POOP Framework”) that strives to optimize application flexibility, minimize cost
while increasing value, maximize programmer resources, and empower application implementers to
better assist them in leveraging modern paradigms in order to proactively achieve next-generation
synergy in tomorrow’s web 3.0 world. It is implemented in (mostly) ANSI Common Lisp. Sheeple is
fully buzzword compliant.

Now that doesn’t really say much of anything, but it’s
amusing, specially since some actual (professional!) projects actually describe things in such a
way.

What does it actually do, though?

Well, Sheeple is a library for Common Lisp that implements a Prototype-based object-oriented language very similar to Common Lisp’s standard CLOS. It does things like multiple dispatch, multiple inheritance and such.

One goal I had while designing it was to have it be as practical as possible. Javascript is making waves around the internet these days — not just because it’s the de-facto standard browser-scripting language, but because it’s prototype-based.

The idea of having a prototype-based language instead of using classes for OO is nothing new. Even so, it seems like there’s still debate about the merits of using prototypes instead of classes. In fact, some prototype-based languages, specifically Javascript, have libraries that implement classes!

Even though there are various languages to choose from, there are still few languages that provide what I would consider full facilities. Cecil seems to be one of those ‘full’ languages, but I really ended up disliking its approach — it was too complicated, and there were too many obvious minefields. Sheeple is an attempt at providing the kind of massive framework CLOS provides for class-based OO, but with prototypes, while remaining easy-to-use, straightforward, and most importantly, convenient. Whether it’s successful at this right now (it’s still in development after all), and whether it will ever be is a different question. ;)

Here’s some basic sample code, from the docs:

The CLONE function called with no args creates a regular sheep object with DOLLY as its only parent object.

SHEEPLE-USER> (defparameter *my-sheep* (clone))
*MY-SHEEP*

Sheeple (I refer to single objects as ‘a sheep’, and multiple objects as ’sheeple’) are, at the basic level, simply glorified hash tables: key/value stores that can be changed at runtime. Every property a sheep object has is named by a symbol.

SHEEPLE-USER> (add-property *my-sheep* 'var "value")
#<Sheep #x1508F91E>
SHEEPLE-USER> (var *my-sheep*)
"value"

A relationship hierarchy in the style of super/subclasses is created by simply cloning an existing object.

SHEEPLE-USER> (defparameter *child* (clone *my-sheep*))
*CHILD*

You can think of the way this new object works as “It’s just like its parent, but X, Y, and Z are different.”
This is called differential inheritance. Sheeple, unlike other languages that follow the prototype model for OO, such as Slate or Self, does differential inheritance by default, making it more like Io in that sense.
To understand how differential inheritance works, look at this example:

SHEEPLE-USER> (var *child*)
"value"

Here’ we can see that the value of child’s VAR property is the same value that *my-sheep* had in that property.
The funky awesome stuff comes when we change the parent object in some way:

SHEEPLE-USER> (setf (var *my-sheep*) "new-value")
"new-value"
SHEEPLE-USER> (var *my-sheep*)
"new-value"
SHEEPLE-USER> (var *child*)
"new-value"

As you can see, changing the value of one of the parent object’s properties changes the value *child* sees whenever it tries to access that property. This maintains the “*child* is just like *parent* relationship, although it has some other benefits that I like (which I’ll discuss in a later post about differential inheritance).

But what if we never intended for *child*’s var property to change?
In that case, all we would have to do is set that property in *child*, and the change in *my-sheep* would never have affected it.

SHEEPLE-USER> (setf (var *child*) "my-own-value"
"my-own-value"
SHEEPLE-USER> (var *child*)
"my-own-value"

In general, the design I’m aiming for is “parents can change their own behavior, which can affect children, but they cannot directly touch the children, nor can children touch them”… or something like that. The truth is that differential inheritance can be a bit tricky to understand, and there’s some major caveats. Once you really get why it’s useful, and know how to avoid pitfalls, though, I think it can turn into a genuinely useful tool.

In a later post, I’ll write about Sheeple’s method dispatch mechanism, which is based on Slate’s PMD, and why you want it. If you’re too impatient, just check out Sheeple’s code on github, and skim through doc/user-guide.org, which is a bit of a working spec of a good chunk of the language.