Right now I'm reading The Little Learner. Like how The Little Schemer introduces fundamental concepts of how computing and programming languages work, and The Reasoned Schemer teaches logic programming, and in each you build your own implementation of the language in question at the end, The Little Learner is like that but for deep learning neural networks.

I tend to be vocally skeptical of the hype around LLMs right now, so it may seem strange that I'm working through this book, but the truth is that I mostly think that LLMs are insufficient in many ways, and that there's more to the puzzle, but not having the complete puzzle is a dangerous situation for society. More on this in an upcoming blogpost.

But I wanted to really actually understand things on an algorithm level before I said the things I am thinking, and the Little Books have introduced me to multiple deep subjects, and I thought I'd give this one a try. It seemed pretty difficult to believe that they'd be able to explain something that's much more statistical and what I largely thought of as a big ol' number soup, but they've done it quite well.

I have my MNT Pocket Reform out and I've been working on it through that. It's nice insofar as, well, with my normal laptop out it's easy enough to get distracted, but the Pocket feels focused.

Originally I was using Racket and DrRacket, I figured I'd try to go through the book using the Racket package malt. But for whatever reason, DrRacket is terribly slow on this device despite actually having a reasonably powerful board on it, and the load time for setting up Malt seems to be huge. Maybe Racket performs slowly on ARM64? I'm not actually sure. But anyway, eventually I decided, they have the implementation at the back of the book in an appendix (well actually they have two versions, Appendix A for the simpler but slower version, and Appendix B for the faster parallelizable one). So I started typing in the implementation in the appendix into my own little Guile modules, and that turned out to be a good idea, because I am already getting a better sense of what's going on.

And to my delight I realized that their neural network kernel is a metacircular evaluator! (If you don't know what this is but are now very curious, you might try reading A Scheme Primer, which I think does a good job of introducing the ideas towards the end, but hey I'm biased.) There's the equivalent of eval and apply in there, but the arguments to functions are basically matrices of floating point numbers, but that idea is there! And as the metacircular evaluator evaluates things, it "updates its intuitions" about its statistics. Well, anyway, that's how I see it.

The point is: eval/apply will never die.

Anyway, the book is helping me understand, respect, and also better see the limitations of these systems. My opinions on a social level haven't changed much, but my vision for how to forge a better future than the present are congealing.

And in the meanwhile, I'm in a beautiful city, and I suppose I should be out exploring more. Instead I've locked myself mostly in this hotel room, venturing out occasionally for boba tea to fuel my hacking sessions, locking myself in again. It's not even a great hotel experience, I'm staying in hotel dorms (and the University wifi's nanny firewall, hilariously, blocks this very website, who knows why).

But it's so hard to find hack time on things like this anymore. I've spent many a vacation with some time head down, doing the kind of programming and research I wouldn't be able to do day to day. I guess it feels silly. But I'm still venturing out, wandering around, just a little bit.

But I'm here for a wedding, one of a good friend, who loves this kind of stuff also. That makes it feel a little bit more appropriate.

Anyway, rambly post. Something more of substance about these topics is coming soon. I should finish getting ready! The wedding is tomorrow, but people are gathering tonight, and I should leave soon. As excited as I am to hack, I am also excited to step out and spend time with friends. Off I go!