Back Original

How magic is too magic? Personalities of programming languages

The languages I know best are Zig, Lua and Rust, each of which comes with very different goals and assumptions, lending to a different style in terms of what feels fun to do.

Of these three, I think it’s obvious that Zig is the most serious programming language. It prides itself on having “no hidden control flow”, for example, hands you control (if you want it) over the memory layout of types, and requires you to manage memory correctly. That’s not to say Zig isn’t fun: when I implemented the Uxn virtual machine, I was able to define my own type which takes up one byte of space, but treats each bit of that byte correctly as either a “mode flag” or as part of a list of instructions. Whereas in C one has to do bitwise arithmetic directly, in Zig you can ask whether a bit is set as if it were a regular boolean, or switch on only the enum field of the byte-sized struct. Other classic metaprogramming tasks are also a joy in Zig, simply because they realized that programming Zig should be done in Zig, even if the purpose of that code is to construct a generic type and so the code runs at compile time instead of run time.

Rust, by comparison, is positively loosey-goosey. I’ve read too many complaints that Rust is hard because of the borrow checker and having to think about ownership. I could be wrong, but I think the reason for these complaints is because Rust was born as a Haskell that realized it could come for C++’s crown, but has morphed instead into a JavaScript run on that most virtual of machines, the actual CPU.

When you think about it using these frames, Rust is a joy to write. The borrow checker makes powerful, “zero-cost” abstractions possible, like copy-on-write structures, reference counting, Impl Drop for T running arbitrary code, closures, iterators, etc. Combine this with implicit returns and chainable methods, and you get the ability to write these beautifully terse koans that through the magic of LLVM are somehow also fast too. I love writing Rust; it makes me want to give it a noogie. Is your object on the stack or the heap? You actually probably don’t get to know, but as long as you and the borrow checker agree, you don’t really need to, either.

Lua is interesting. On the one hand, somehow the do and the then and the function spelled all the way out like that give it a certain amount of clunk. On the other hand, if you’re willing to venture into the corners of the language, magic is truly possible. I want to spend the rest of this post describing a use of the Lua API that I’m pretty proud of.

Lua has very few types; this is to facilitate holding the entire language in your head as much as possible. Here is the full list: nil, boolean, number (integers, interestingly, exist as a bit of an odd subtype), string, table, userdata, “light userdata”, thread (really “coroutine” or “stack”) and function. Light userdata differ from “full” userdata in two ways: from C’s (or Zig’s) perspective, light userdata are just pointers; the object they point to is not in Lua’s universe so is not subject to garbage collection and mostly just sleets through the Lua environment undisturbed; their purpose is largely to enable Lua functions implemented in C to get handles to relevant data from the host environment. That’s one way; the other is that light userdata cannot be operated on—you can ask whether they’re equal (that’s just whether the addresses are equal), but you can’t index into them like you can a table.

Userdata and metatables

Full userdata, on the other hand, are also “just pointers” from C’s perspective, but the memory they point to is owned by the Lua environment, so is subject to garbage collection. In contrast to light userdata, full userdata may be made to behave more or less in the way that Lua tables do by providing a metatable for them.

A metatable is what it sounds like. I’m so sorry for writing that, but it’s true; at least in the common sense of “meta” as “referring to its own category”. functionally in Lua, the point of tables is that you can index into them. (They’re an interesting mix between an array and a hash map.) When a table t is missing a key k, but t has a metatable mt, the Lua virtual machine will query mt for its __index metamethod. If __index is a table, what you get will be mt.__index[k]. Otherwise if __index is a function, what you get is the result of mt.__index(t, k). There is also a __newindex metamethod. Metatables provide other functionality too: for example, if you want to be able to add two tables together, you can do that with an __add metamethod. Or you want to __call a table as if it were a function? That’s allowed!

The interesting thing about this is that because functions are code, this means you can really go wild. An initial draft of this post intended to go further into this observation. Given that that post stalled out for a month, let me close by offering the following summary of it:

In my v2.0.0 rewrite of seamstress, an "art engine" and batteries-included Lua runtime, I have a Timer object type. From the perspective of the Zig code that implements it, a Timer is a little struct whose purpose is to sit on the seamstress event loop. From the perspective of Lua, a Timer has several builtin fields, perhaps the most interesting from today's perspective of which is named running. If t is an object of type seamstress.Timer, altering t.running from true to false executes code (like all code...) with the goal of removing the Timer from the event loop. In a language like Zig I would be aghast at this, since calling several functions as a result of setting a boolean feels like the definition of hidden control flow. Even as a writer of Rust I would probably balk at this. But in Lua? Bring it on.