Since last time, I've been continuing to rewrite my janky little text editor from scratch with a lot of ai assistance. This started out as a learning project, but recently I switched to actually using the new editor because it's less janky and more featureful than the old version.
intelligent associates
Contrary to last time, I'm now often just committing ai-generated code after a little manual testing and a glance over the diff. It's not clear how much this is due to having the architecture and testing strategy already in place vs the models getting better.
They definitely have gotten better though. I don't have to handhold nearly as much, they actually follow instructions about how to write tests, and they often catch edge cases that I wouldn't have thought of.
Sometimes they even venture to push back against bad instructions. Opus wrote something along the lines of "I didn't implement your instructions literally because that would have caused this obvious issue", and was sadly correct.
That's rare though. Most often they're still in evil genie mode and will just attempt to bludgeon through all obstacles rather than point out that I made a mistake. I've found it very useful to stop writing "Do X" and instead write "I want to do X. Any questions?" to catch them before they go haring off.
The majority of the recent work was done by opus 5 within claude code on a $20 subscription. I still like pi better, but the models are increasingly trained against their own harnesses and I spend most of my time interacting with text files anyway and avoid spending much time in the harness itself.
deterministic simulations
The biggest new feature is tests.
The code is split into two crates - focus-core and focus. As much as possible of the logic is in focus-core. It takes input events and returns a list of characters/rects to render, using a &mut dyn IO to talk to the outside world. The focus crate contains the real implementation of the IO trait, the cli interface, and daemonization logic.
> scc focus-core/src
───────────────────────────────────────────────────────────────────────────────
Language Files Lines Blanks Comments Code Complexity
───────────────────────────────────────────────────────────────────────────────
Rust 34 13831 1067 1856 10908 1202
───────────────────────────────────────────────────────────────────────────────
Total 34 13831 1067 1856 10908 1202
───────────────────────────────────────────────────────────────────────────────
> scc focus/src
───────────────────────────────────────────────────────────────────────────────
Language Files Lines Blanks Comments Code Complexity
───────────────────────────────────────────────────────────────────────────────
Rust 8 3661 288 720 2653 257
───────────────────────────────────────────────────────────────────────────────
Total 8 3661 288 720 2653 257
───────────────────────────────────────────────────────────────────────────────
The payoff is that it's trivial, although tedious, to write end-to-end tests for focus-core. The models desperately want to write unit tests so if you don't give them a stable public interface to test against they'll go poke holes in all your boundaries.
#[test]
fn dir_picker_ctrl_enter_descends_into_selected_dir() {
let (mut app, mut io, window_id) = common::scratch_app();
insert_file(&mut io, "/proj/src/main.rs", "");
common::control_key(&mut app, &mut io, window_id, Key::Character("m"));
common::tick(&mut app, &mut io);
// ctrl+enter descends into the first (and only) listed dir (proj/).
common::control_key(&mut app, &mut io, window_id, Key::Named(NamedKey::Enter));
common::tick(&mut app, &mut io);
// The path editor now shows /proj/.
assert_eq!(buffer_text(&app, DIR_PATH), "/proj/");
// The list now shows src/.
assert!(buffer_text(&app, DIR_LIST).contains("src/"));
app.assert_invariants();
}
The tests are largely slop. I rarely even look at them. But I do see them catching regressions and prompting the robots to notice that they broke something.
There is also a fuzzer that effectively just opens the app, sends thousands of random keypresses, and then calls app.assert_invariants(). It's slop and has poor coverage, but still flushes out bugs.
Even though I don't actually look at them, the combination of e2e tests and fuzzing actually seems to be good enough to produce an editor that is basically usable. I haven't seen any bugs or crashes after a few weeks of use.
Obviously testing effort has to be driven by risk and impact. For a project with only one user and which doesn't risk losing important data, it's often more time-efficient to just rely on the slop and fix bugs only if I actually run into them. I don't yet know what would change in my approach if I was actually shipping something important.
The IO trait is a cheap crappy solution. I wish there were better options for simulation. There is no reasonable way in rust to inject mocks for the stdlib and most libraries are not written in sans-io style, so I can't make them use my own mockable IO trait. That means a decent chunk of library calls are forced to live outside of focus-core where they are harder to test. Eg daemonization is a pain to test and sometimes regresses.
I have some vague thoughts about using WASI or some hypervisor to do a kind of antithesis-lite and bring everything inside the simulatable boundary, but it's not immediately on the roadmap.
avoiding decay
I'm trying to figure out how little I can review code. It depends a lot on what kind of boundaries I can draw around the behaviour.
Eg a robot wrote a manual tokenizer for python and I barely need to look at it. It's obeying an existing highlighting interface, the control flow is one big loop that looks guaranteed to be linear in buffer size, and all the code is contained in a single module. The fallout is limited. I open some python code to check the highlighting seems reasonable, and then commit and move on.
But most changes aren't as easy. I'm wary of vibecoding myself into a mess. Often the mess happens a little at a time, and each tiny increment isn't totally obvious when looking at a big commit. So I've been playing with producing different views of the code as snapshot tests.
The api snapshots parse all the code and produce snapshots of the public and internal interfaces. Here is a chunk of the public interface:
## pub mod focus_core::buffer
#[derive(PartialEq, Eq, PartialOrd, Ord, Hash, Clone, Copy, Debug)]
pub struct BufferId(/* private fields */);
pub struct Buffers {
/* private fields */
}
impl Buffers {
pub fn keys(&self) -> impl Iterator<Item = BufferId> + '_;
}
pub fn from_file(app: &mut App, io: &mut dyn IO, absolute_path: PathBuf) -> BufferId;
impl BufferId {
pub fn text(self, app: &App) -> &BStr;
}
Even if I only glance over a commit, I read changes to the interface snapshots closely to catch the introduction of leaky abstractions.
I also have a cackle test that looks for accidental calls to IO functions from inside focus-core. It's pretty adhoc, probably easy to defeat, and I have to disable some crates whose build scripts cause issues and allowlist the hashmap seeding from /dev/random, but it's much better than nothing.
I even have assertions in the fuzzer that assert that fuzzer input never blows the frame budget, and some very rough unit tests for asymptotics eg:
#[test]
fn rust_highlighting_is_linear() {
check("speed.rs", include_str!("fixtures/indent.rs"));
}
/// Open a file of `sample` repeated to two sizes, and require that the
/// bigger one costs about what its size says it should.
fn check(name: &str, sample: &str) {
let small = time_open(name, &repeat(sample, SMALL));
let large = time_open(name, &repeat(sample, LARGE));
let growth = large.as_secs_f64() / small.as_secs_f64();
let rate = (LARGE as f64 / 1e6) / large.as_secs_f64();
let report =
format!("{name}: {SMALL} bytes in {small:.2?}, {LARGE} in {large:.2?} ({rate:.1} MB/s)");
assert!(
growth <= MAX_GROWTH,
"{report}\n{}x the text took {growth:.1}x the time, which is not linear",
LARGE / SMALL,
);
assert!(
rate >= MIN_MB_PER_SECOND,
"{report}\nslower than {MIN_MB_PER_SECOND} MB/s, which is an order of magnitude off",
);
}
These are sadly necessary because even the latest models still often write accidentally quadratic code. I'm interested in how to make these tests more robust. We have a lot of theory and practice about how to specify the logical behaviour of code, but much less about the performance characteristics. The model I fuzz against currently is "with <=16 windows open and all files <=200kb, no single input should take more than 16ms to process and render on this laptop", but I would like something much finer-grained.
pointer-free functions
This is only tangentially relevant to ai, but I ended up with a weird relational/DoD architecture where everything is stored in separate collections and indexed by handle eg:
pub struct Buffers {
pub(crate) buffer_count: usize,
source: Map<BufferId, Source>,
text: Map<BufferId, BString>,
highlight: Map<BufferId, Highlight>,
newlines: Map<BufferId, Vec<usize>>,
last_modified_time: Map<BufferId, Duration>,
undos: Map<BufferId, Vec<Vec<Vec<Edit>>>>,
doing: Map<BufferId, Vec<Vec<Edit>>>,
redos: Map<BufferId, Vec<Vec<Vec<Edit>>>>,
// Can be set by editor.
pub(crate) last_center_offset: Map<BufferId, usize>,
}
Map is just a fancy wrapper around a Vec that uses typed keys (eg BufferId) instead of integers.
I tried a lot of other ways of organizing this and all of them led to tricky ownership/lifetime situations that required judgement to solve ie would cause the robots to go down wild rabbit holes and write crazy code. Working with handles instead makes life much easier, and within a function rust is smart enough to recognize that buffers.text and buffers.source aren't conflicting borrows.
single-threaded lifestyle
There are two popular approaches to email:
- Check your email every five minutes, or whenever you feel anxious, and reply to each email with the minimal amount of text that will get it off your plate.
- Check your email once a day, and reply to each email with sufficient detail and foresight that the sender doesn't need to reply.
I've seen discussion of agentic workflows with lots of git worktrees and parallel agents and merges. I tried it and it was an exhausting mess. So I'm still mostly single-threaded. I try to write a really thorough design for a large batch of work, do a synchronous pass for questions from the model, and then I go for a walk or read a paper or do some more design work until I get a notification on my phone saying it's time to come back and look at the results.
This seems plenty productive and is much more enjoyable. Plus I'm getting a lot more sunlight than I usually do.
If I was working on multiple projects at once, or had experiments that were actually parallelizable, then I might try to mix some workstreams. But I would still try to do as much work as possible on each one at a time so that I don't fry my brain with context switching.
This does mean that my rhythm of work has changed. I'm listening to much less death metal and high bpm electronic, and much more grime and acid rock. Less intensely focused.
directed laughs
I feel uncalibrated now about how much code is too much. This editor is starting to be bigger than I would previously have wanted for a spare time project, but I don't have to maintain it by hand now. So far it's manageable.
More than manageable actually - it took so much less time than the previous version and has so much more functionality. I feel like I can make steady daily progress with much less mental energy. I can spend 15 minutes designing a feature, go to the climbing gym for a few hours, and then spend 15 minutes reviewing/testing while waiting for dinner to cook. A few rounds of that was enough to add a major feature - vcs integration - that I never got around to in the original.
I think I do still need some dense concentration periods from time to time to figure out harder problems and to clean up accumulated architectural fluff. But much less than before.
For a while I thought that I would give up on projects like zest because it's such a huge amount of work to get to a usable point and I have no idea what the world will look like by the time I finish. Will I even still be writing code?
But this project has totally changed my mind. It's much easier now to go from design to implementation, at a quality plenty good enough for a research project. And even as models continue to advance we're going to be increasingly faced with problems of specification, understanding, and control that lend themselves to systems programming solutions.
What I want from a programming system will change a lot. But there is still a lot of really interesting work to be done in figuring out how to actually take advantage of ai-generated code. You can only go faster to the extent that you can figure out how to either generate trust or mitigate damage. This is a great time to be pushing on capabilities/effects, formal methods, model checking, fuzzing, sandboxing etc.
mass produced personalization
The whole quality-vs-effort frontier can shift. At one end, teams that really care about quality can put a lot more eyeballs on bugs and can build a lot more tooling. At the other end, an effort that previously wouldn't have been enough to ship at all can now produce something mediocre.
I expect an onslaught of low-quality software, but it's a mistake to bemoan the quality without realizing that the previous alternative was not high-quality software but no software at all.
Two years ago essentially no individual could afford a programmer. Now anyone with $20 can have a personal programmer working for them. Today it's still very genie-like - you'll only get a good result if you make a good wish, which requires knowing what's even reasonable to ask for. But it continues to improve.
And even with robot butlers it's still pretty hard for a non-technical person to deploy and maintain any kind of stateful app. But if you pair today's models with a pre-existing system like notion that already handles state, versioning, collaboration, and UI, then the long dream of end-user programming might finally be upon us.
The robots certainly devalue many of my skills, but rather than mourning my high status I'm excited to see a world where everyone can design software.
I know that only artisanal hand-crafted goods truly have a soul, but I still love to ride my factory-produced bike along smooth factory-produced asphalt while using my factory-produced phone to take photos of murals painted with factory-produced spray-cans. And it blows my mind that while I'm sitting on a bench by the ocean I can call a friend on the other side of the world, and I don't think even the steadiest craftsman could have etched those chips by hand.
signs of stopping
The improvements must surely tail off at some point, but I see no reason why that point has to come any time soon.
It's hard to imagine what the world would look like even if model progress stopped today and we had time to figure out how to integrate them. Even harder to imagine what the world will look like after a few more years at this rate of progress.
I wouldn't mind slowing down.