April 25, 2005

On Lisp

[Hacking_ Books_] Last weekend, I had a chance to browse Paul Graham's website, read his wonderful book "On Lisp", and learn about his new language idea "Arc". This is more of my ramblings inspired by his work rather than commentary on the book.

Lately, most of my programming projects seem to go through
the same cycle. I first write a program in Perl. Eventually I
run into some sort of memory related performance bottleneck. (My
programs typically work with large datasets and/or build large
statistical models.) Then I rewrite the whole program painfully
in C. All the while, I actually want to be using Lisp. What I
need is a Lisp variant that offers the advantages of Perl and C.

Why do I start with Perl? In one word: "conciseness". It takes
me five minutes to perform some task in Perl that would take me an
hour in C. Perl programs are concise. The operator names are
concise. There are concise ways to express things you most need:
hashes, extensible arrays, regular expressions, file I/O. I like
$a{color} better than (gethash 'color a).

Why do I end up going to C? In one word: "overhead".
Implementors of high level programming languages like Java and
Perl do not worry about memory as much as they worry about speed.
Soon you find out that a short string actually costs 100 extra
bytes and since nobody expected you to create 100,000 hash tables
they crammed all sort of extra information into the data structure
that they could have kept outside. Any speed gain a smart
compiler can give you is worthless once your disk starts
thrashing. C gives you the control to spend as many bytes as you
want on whatever data structure you need.

Why do I want to use Lisp? If you are asking this you should read SICP and On Lisp or take a look at Paul Graham's articles on the subject.

So all I need is a "concise" Lisp with "no overhead"!


  • SICP: Abelson and Sussman's book.
  • On Lisp: Paul Graham's book. (Here is a blog pointing to on-line copies)
  • Arc: Paul Graham's new Lisp variant.

Related link

No comments: