Archives

Tags

Posts with tag "lisp"

Racket is an acceptable Python

By Christopher Lemmer Webber on Tue 09 July 2019

A little over a decade ago, there were some popular blogposts about whether Ruby was an acceptable Lisp or whether even Lisp was an acceptable Lisp. Peter Norvig was also writing at the time introducing Python to Lisp programmers. Lisp, those in the know knew, was the right thing to strive for, and yet seemed unattainable for anything aimed for production since the AI Winter shattered Lisp's popularity in the 80s/early 90s. If you can't get Lisp, what's closest thing you can get?

This was around the time I was starting to program; I had spent some time configuring my editor with Emacs Lisp and loved every moment I got to do it; I read some Lisp books and longed for more. And yet when I tried to "get things done" in the language, I just couldn't make as much headway as I could with my preferred language for practical projects at the time: Python.

Python was great... mostly. It was easy to read, it was easy to write, it was easy-ish to teach to newcomers. (Python's intro material is better than most, but my spouse has talked before about some major pitfalls that the Python documentation has which make getting started unnecessarily hard. You can hear her talk about that at this talk we co-presented on at last year's RacketCon.) I ran a large free software project on a Python codebase, and it was easy to get new contributors; the barrier to entry to becoming a programmer with Python was low. I consider that to be a feature, and it certainly helped me bootstrap my career.

Most importantly of all though, Python was easy to pick up and run with because no matter what you wanted to do, either the tools came built in or the Python ecosystem had enough of the pieces nearby that building what you wanted was usually fairly trivial.

But Python has its limitations, and I always longed for a lisp. For a brief time, I thought I could get there by contributing to the Hy project, which was a lisp that transformed itself into the Python AST. "Why write Python in a syntax that's easy to read when you could add a bunch of parentheses to it instead?" I would joke when I talked about it. Believe it or not though, I do consider lisps easier to read, once you are comfortable to understand their syntax. I certainly find them easier to write and modify. And I longed for the metaprogramming aspects of Lisp.

Alas, Hy didn't really reach my dream. That macro expansion made debugging a nightmare as Hy would lose track of where the line numbers are; it wasn't until that when I really realized that without line numbers, you're just lost in terms of debugging in Python-land. That and Python didn't really have the right primitives; immutable datastructures for whatever reason never became first class, meaning that functional programming was hard, "cons" didn't really exist (actually this doesn't matter as much as people might think), recursive programming isn't really as possible without tail call elimination, etc etc etc.

But I missed parentheses. I longed for parentheses. I dreamed in parentheses. I'm not kidding, the only dreams I've ever had in code were in lisp, and it's happened multiple times, programs unfolding before me. The structure of lisp makes the flow of code so clear, and there's simply nothing like the comfort of developing in front of a lisp REPL.

Yet to choose to use a lisp seemed to mean opening myself up to eternal yak-shaving of developing packages that were already available on the Python Package Index or limiting my development community an elite group of Emacs users. When I was in Python, I longed for the beauty of a Lisp; when I was in a Lisp, I longed for the ease of Python.

All this changed when I discovered Racket:

  • Racket comes with a full-featured editor named DrRacket built-in that's damn nice to use. It has all the features that make lisp hacking comfortable previously mostly only to Emacs users: parenthesis balancing, comfortable REPL integration, etc etc. But if you want to use Emacs, you can use racket-mode. Win-win.
  • Racket has intentionally been built as an educational language, not unlike Python. One of the core audiences of Racket is middle schoolers, and it even comes with a built-in game engine for kids. (The How to Design Programs prologue might give you an introductory taste, and Realm of Racket is a good book all about learning to program by building Racket games.)
  • My spouse and I even taught classes about how to learn to program for humanities academics using Racket. We found the age-old belief that "lisp syntax is just too hard" is simply false; the main thing that most people lack is decent lisp-friendly tooling with a low barrier to entry, and DrRacket provides that. The only people who were afraid of the parentheses turned out to be people who already knew how to program. Those who didn't even praised the syntax for its clarity and the way the editor could help show you when you made a syntax error (DrRacket is very good at that). "Lisp is too hard to learn" is a lie; if middle schoolers can learn it, so can more seasoned programmers.
  • Racket might even be more batteries included than Python. At least all the batteries that come included are generally nicer; Racket's GUI library is the only time I've ever had fun in my life writing GUI programs (and they're cross platform too). Constructing pictures with its pict library is a delight. Plotting graphs with plot is an incredible experience. Writing documentation with Scribble is the best non-org-mode experience I've ever had, but has the advantage over org-mode in that your document is just inverted code. I could go on. And these are just some packages bundled with Racket; the Package repository contains much more.
  • Racket's documentation is, in my experience, unparalleled. The Racket Guide walks you through all the key concepts, and the Racket Reference has everything else you need.
  • The tutorials are also wonderful; the introductory tutorial gets your feet wet not through composing numbers or strings but by building up pictures. Want to learn more? The next two tutorials show you how to build web applications and then build your own web server.
  • Like Python, even though Racket has its roots in education, it is more than ready for serious practical use. These days, when I want to build something and get it done quickly and efficiently, I reach for Racket first.

Racket is a great Lisp, but it's also an acceptable Python. Sometimes you really can have it all.

Phyllis Fox, documenting Lisp History

By Christopher Lemmer Webber on Wed 08 March 2017

In honor of International Womens' Day, let's celebrate Phyllis Fox, who may have saved Lisp from the dustbin of history... by documenting it. From her oral history:

HAIGH: So you say that you wrote the first LISP manual?

FOX: Now, this was not because I was a great LISP programmer, but they never documented or wrote down anything, especially McCarthy. Nobody in that group ever wrote down anything. McCarthy was furious that they didn’t document the code, but he wouldn’t do it, either. So I learned enough LISP that I could write it and ask them questions and write some more. One of the people in the group was a student named Jim Slagel, who was blind. He learned LISP sort of from me, because I would read him what I had written and he would tell me about LISP and I would write some more. His mind was incredible. He could give lectures. Have you ever seen a blind person lecture?

HAIGH: No.

FOX: They write on a black (or white) board, and then they put a finger on the board at the point they have stopped to keep the place. Then they talk some more and then they go on writing. His mind was remarkable. He was very helpful to me. But I wrote those manuals. I would ask questions from Minsky or McCarthy, and I got it done. I think it was helpful for people to have it. I guess, essentially I’m a documenter. If you’re looking for it, that’s what I am.

Phyllis Fox did a lot more than that, but as a Lisp enthusiast, thank you to Dr. Fox for preserving our programming knowledge!

Wisp: Lisp, minus the parentheses

By Christopher Lemmer Webber on Wed 23 September 2015

Arne Babenhauserheide has built a really cool syntax alternative for Scheme, Wisp (not to be confused with a different lisp-related-wisp), or in standards version, SRFI 119. It looks pretty nice:

;; hello world example
display                             ;    (display
  string-append "Hello " "World!"   ;      (string-append "Hello " "World!"))
display "Hello Again!"              ;    (display "Hello Again!")

;; hello world function
define : hello who                  ;    (define (hello who)
  display                           ;      (display 
    string-append "Hello " who "!"  ;        (string-append "Hello " who "!")))

Actually, let's see that in emacs, just to be sure.

Wisp and hello world

How about something slightly more substantial? How about a real life Guix package for GNU Grep:

Wisp, Emacs, Guix and Grep

Wow, not bad... not bad at all! I'd say that's quite readable! (Too bad the lines don't line up exactly in that screenshot; that's not the code but rather my emacs theme bolding the wisp code.)

What's nice is that unlike most s-expression alternatives, it doesn't lack any of the power of Lisp; it's "just lisp" with the parentheses hidden by vaguely pythonesque indentation, which means even macros work.

Now me personally? I've learned to love the parens, and there's nothing that beats an editor that knows how to do cool structural s-expression editing and navigation. But I admit that learning to read through all the parentheses was a tough thing for me initally, and certainly for many others. Maybe this can help boil the lisp frog for some.

Now what would really be hylarious would be to port this to Hy...

Javascript beyond Javascript (with Guile?)

By Christopher Lemmer Webber on Fri 15 August 2014

I'm learning more in my spare time / off normal hours about compilers and graph theory and reading various books on lisp. I have an agenda here, no idea if it'll happen; at the very worst I put a lot of tools in my toolkit that I should have had. But there is another reason... Well, this one's easiest to lay out point for point, so here goes:

  • Python is still my favorite language to write in day to day, but I guess that I keep feeling that any language that doesn't have a way to transcompile nicely to Javascript (or isn't Javascript itself) is unideal for writing really great web applications, for a simple reason: modern web applications are highly interactive, and I feel that a really stellar web application needs to share some code between the backend and the frontend. This becomes more obvious when writing dynamic templates or forms.

    While Python probably has hands-down the nicest asynchronous library in current development with asyncio, the above problem makes it feel like you can only do so much in the web world with it. (The Javascript Problem is a good page on the Haskell wiki which puts this all pretty nicely.)

    There's a lot lost by not being able to interweave applications between the frontend and the backend. Sadly, the best option in Python right now is to just give up and write entirely separate codebases in javascript that interact with your main codebase. But clients on the web are becoming thicker and thicker these days (even more so with technology like websockets and webrtc becoming prevalent), so this feels both weak and redundant.

  • You could say "just write javascript!" then, but let's face it, I don't really like Javascript. The language is getting better by leaps and bounds as time goes on, but a lot of the enthusiasm for the language still feels like Stockholm syndrome to me. But we're still left with Javascript. So what, then?

  • Transcompiling actually is becoming an increasingly appealing target. With asm.js this might not only be possible but actually very optimal. One only need look at the Unreal demo to see just how feasible transcompilation has become.

  • So okay, let's say we're doing some kind of transcompiling. We have two options, "transcompile wholesale", or "transcompile a subset that's acceptable for javascript and local evaluation". Obviously the former is desirable if possible, though the latter is a lot easier.

  • Indeed, transcompiling a subset has already been done, and well, by Clojurescript. I'm not really excited by anything too tied to the JVM, but still, pretty cool. Maybe good enough to get me into a hybrid Clojure/Clojurescript solution. But still... there's no asm.js target (maybe in the future?), and it feels like you're tossing out so much with Clojurescript, it really isn't the same language too much, just a very similarly overlapping language. I haven't tried it though... this is just from reading docs and watching talks :)

  • We could say maybe we could transcompile something like Python, but assuming we really want to have something like a template engine, we probably want to be writing in a full fledged version of the language, not a restricted subset. Python feels way too huge to expect for people to load in their browsers. So, what's more lightweight?

  • Let's continue in the lisp direction. Is there a way we can get a complete implementation in the browser, but of a language that's a bit more lightweight? How about Scheme? Scheme is notoriously simple to implement, maybe even too simple... and Javascript shares a lot of overlap with Scheme in design...

  • But which Scheme? How about Guile? Yes, Guile, GNU's extension language based on scheme.

Okay, wait, digression before we continue... I want to talk a bit more about Guile, because up until a few months ago my impression of it was still the somewhat-dismissive impression I had a few years ago. But I've come back to looking at Guile again. Largely this is thanks to Dave Thompson's Sly project. There is some seriously cool stuff going on with Sly, and that made me want to look into what's happening with Guile.

What I found is that Guile isn't the same thing it was a few years ago. For evidence of all the cool things that are happening under the hood, I point you to wingolog (warning: danger of being a real time sink; linking to Andy Wingo's blog has been described as "wingorolling" by a friend of mine). Guile isn't just an interpreter anymore, it has a sophisticated "compiler tower" since Guile 2.0. With the levels of abstraction that exist in Guile, could compiling to javascript/asm.js be a target? (What makes this feel extra appealing/feasible: Andy Wingo also works at his day job on improving javascript virtual machines, so maybe...?)

Well, better to ask Guile hackers themselves if it's possible... and I've pestered a number of them. The responses seem to be (and I may be representing incorrectly):

  • Getting the core language of Guile (which is of course scheme) ported over is not the hard part.
  • Making a subset of the language that works in Javascript is not too hard, but isn't too interesting. It would be much better to have the whole language work.
  • Overall, the most tedious part of porting guile to target any non-C target will be all of the library procedures currently implemented in C... however, gradually the Guile team is reducing the amount of C code in Guile and replacing it with Scheme code.
  • The problem is the runtime; you probably would want to re-use some higher level things like the allocator and garbage collector. Emscripten might do garbage collection correctly (guile relies on Boehm GC to do garbage collection); emscripten seems to provide a similar garbage collector? Tail calls are also a problem until at least es6 is common, and even then until they are optimized.
  • Maybe someone could try transcompiling Guile with emscripten as a test, but that probably wouldn't provide the right integration, and regardless, this would probably be a bit uncomfortable because emscripten relies on llvm, not gcc.
  • Targeting asm.js brings things more low-level, but the asm.js typedarray heap may not be appropriate; the guile-on-js heap and the javascript heap should really be the same.

Okay, so, that's a good number of problems that need to be overcome. Certainly it makes it feel like things are a ways off, but then again, it seems like there's strong interest. And it feels feasible... as an outsider to development. :)

But even assuming all the above are resolved, there are some other problems that I think scheme will always face adoption-wise:

  • I like lisp, and I like parentheses. I have emacs set up with rainbow-delimiters, smartparens, and highlight-parentheses (set up just to highlight the background, not the foreground, so it doesn't clobber rainbow-delimiters). Lisp's greatest enemy has never been its functional capability, but a general parenthesitis in the general population. This the big one, so let's come back to it in a moment.
  • Lots of scheme (lisp generally, but especially scheme) feels like it just is too in love with ideas that make it hard to approach. "car" and "cdr" are not good names but form the very basis of the language for historical reasons where even prominent lispers have had to backtrack to remember why. Linked lists as the core of lisp is great, but overuse of them is encouraged when other data types are much more appropriate: see teaching associative lists before hashmaps. While iteration tools are provided, there's simply too much emphasis on recursion. Recursion is powerful and awesome and necessary to solving many important computer science problems, but rarely needed by say, a web developer, and significantly harder to read and wrap one's mind around than iteration.
  • That said, these could be overcome with better introductory material. There seems to be interest in the guile world for this, thankfully!
  • Module names often look like robot serial numbers ("srfi-13") than anything human-readable and feel like they really need aliases.
  • The Guile community is very small, and does not seem to be very diverse. Outreach is greatly needed.

Returning to "parenthesitis", while I love lisp and all its parentheses, admittedly it's not the most readable language ever. I can wake up groggily at 5AM, be tossed a block of Python code, and even through blurry eyes, I can get a feel for what's happening in the code just by its structure. I can't say the same of any lisp.

But there's a way out of that situation! Guile's compiler stack is nicely set up so that another syntax can be laid on top of that. Guile's VM has semi-complete language implementations of ecmascript and some other languages on top of it. Recently Arne Babenhauserheide has written a whitespace to lisp preprocessor named wisp; I don't feel it really solves the problem by making things so much more readable, but it's a nice demonstration.

The most promising resolution I think would be to implement a syntax akin to Julia on top of Guile. If you haven't looked at Julia, it's a cool project: it has a python-like syntax with lisp-like power, and many other interesting features, including macros(!) in a language that is certified safe for those with parenthesitis. In fact, the language is largely built on top of scheme! So it might be nice to have a "sugarcoat" language that sits on top of Guile.

If all the above were achieved and a well developed web framework were written on top of Guile, it could be well positioned for writing web applications that are a joy to write, both on the backend and frontend.

One more thing: the complaint about "don't use emscripten" because it's built on top of llvm is an indication of how sorely needed a working javascript/asm.js compiler target is in GCC. ARM, X86, SPARC... these are all important compiler targets to have, but to advance user freedom where the user is today, the browser is the most important target of all.