- 8 mins read
One of my colleagues through the Recurse Center community posted their thoughts on using AI to contribute to open source, and it got me thinking a bit and goaded me into writing this post.
It feels like open-source projects are losing the plot–for both good and bad reasons–on AI contributions, and it really sucks to see it.
In January of this year, Daniel Stenberg announced that the curl project would no longer be running a bug-bounty program, citing AI slop. Never one to mince words:
The never-ending slop submissions take a serious mental toll to manage and sometimes also a long time to debunk. Time and energy that is completely wasted while also hampering our will to live.
As a man of taste and talent, he actually shows his work so we can get a feel for what he’s facing. Some of those show drive-by bug reports dropped like cinderblocks from a highway overpass, some show reports filed and then poorly-defended as the originating human takes over and gets in over their head almost immediately, others even find a real bug but wrap it in slop in such a way that it enrages and alienates the maintainers. I’d suggest skimming a few of those reports, they’re informative to get a feel for the problem.
Other projects have gone on to even more extreme solutions–ghostty introduced a vouch/web-of-trust system (which of course will never be abused for political infighting later in that or other projects–I am assured of this).
Open source has always worked on a system of trust and verify.
Historically, the effort required to understand a codebase, implement a change, and submit that change for review was high enough that it naturally filtered out many low quality contributions from unqualified people. For over 20 years of my life, this was enough for my projects as well as enough for most others.
Unfortunately, the landscape has changed particularly with the advent of AI tools that allow people to trivially create plausible-looking but extremely low-quality contributions with little to no true understanding. Contributors can no longer be trusted based on the minimal barrier to entry to simply submit a change.
Zig has setup a strict no-LLM policy:
No LLMs for issues.
No LLMs for pull requests.
No LLMs for comments on the bug tracker, including translation. English is encouraged, but not required. You are welcome to post in your native language and rely on others to have their own translation tools of choice to interpret your words.
Anyways, those are some examples of anti-AI sentiment in projects in the wild.
The blog post I mentioned at the very beginning suggested this a spectrum for evaluating AI usage:
Level 0: Human did not use AI at all
Level 1: Human asked chatbot for ideas
Level 2: Human coded with minor assists
Level 3: Human coded, bots assisted non-trivially
Level 4: Human coded, bots helped significantly
Level 5: Bots coded, human understands completely
Level 6: Bots coded, human understands mostly
Level 7: Human specced, bots coded
Level 8: Bots planned, human approved
Level 9: Human fired-and-forgot
Level 10: Rogue bots, zero human attention
As spectrums go, I think this is as reasonable as any–we start with pathetic creatures of meat and bone on one end and matrices that absolutely will not stop on the other, and we get intermediate points along that line.
The thing that I notice muddies the waters is that in those intermediate steps we get conflated concerns:
…and underlying this all:
These are not the same things, and putting them on a single line does us all a disservice.
If the problem is that the planning for some code change was ill-conceived, that’s a problem whether it came from an LLM or a human. If the problem is that the human doesn’t understand the code, that exists irrespective of LLM usage (the old thing we used to complain about, years ago, was people blindly copy-and-pasting things off Stackoverflow without mentioning that’s where they took it from). If the problem is that a contributor passed off thinking as their own that wasn’t, that’s the case regardless of where the ideas actually came from.
We don’t need to additionally pull in LLMs (along with their considerable political and ideological baggage) to try and wrestle with these topics–and when we do pull in the LLM stuff, we rapidly find ourselves dealing with statements that are dumb, ignorant, or extremely motivated. The moving goalposts I’ve seen around “No LLMs can’t do that/will never do that/rarely do that/maybe can do that/can do that but I’m mad about it/it doesn’t matter that they can do that anyways” are incredibly tiresome wherever they pop up, and take very real attention economy from those questions I’ve mentioned above.
I mentioned this on a podcast, but the nasty take I’ll have is:
The way you treat LLMs tells me a lot about how you’re gonna treat juniors.
The way projects treat contributors they suspect of using LLMs tells me a lot about how they’ll treat newcomers more generally and how they’ll treat new technology.
If they reject LLM contributions because they’re failing a quality bar around documentation, testing, or whatever–that’s good news! That means the project has quality standards they can enforce!
If they reject LLM contributions because they don’t have the bandwidth to handle them–okay, that’s a data point about how much workload the maintainers have, and that is almost always a reflection of the health and efficiency of their processes more broadly. It’s not an immediate flag for me, but it is data. When Daniel explained curl’s withdrawal of the bug bounty program, he did a really good job of explaining the numbers around it and giving some historical context as to why that was a problem for the project–they’d previously had issues around bad vulnerability reports and the new tooling made (and their problem domain) makes that even worse. Useful data.
If they reject an LLM contribution that is correct and works, that tells me that they value something other than shipping value to users (the difference between quality, value, and craftsmanship is something I’ll write about some other day).
Basically, if it feels like a project would accept the bespoke, artisanal, human-made xz backdoor and summarily close a Claude-generated patch to fix it, that’s no-go for me…and right now, that’s more projects than I think is healthy and with more cheering than I think is rational.
Also–if you’re telling me that it’s okay to be an asshole to somebody using an LLM (or even just to the LLM itself), you’re telling me that your project culture is such that it’s okay to be an asshole under specific circumstances to certain groups. I have been around too long and been in too many groups that suddenly found themselves on the outside to be comfortable when I observe that behavior. To paraphrase an ancient joke: we’ve already established that you’re an intolerant asshole, and now we’re just haggling over price.
It’s simple: projects need to identify the real problems they’re worried about that LLMs are surfacing, and solve those.
If you’re worried about maintainer bandwidth, you’d have the same problem if it weren’t for LLMs and your project got popular (as we’ve already seen with Hacktober).
If you’re worried about code quality, you’d have the same problems if you don’t have good CI and automated testing and linters.
If you’re worried about poor design, you’d have the same problems if you don’t have good documentation and architecture.
If you’re worried about copyright and IP, you’d have the same problem if you don’t have users sign a contributor license agreement or use a license that reflects your values.
If you’re worried about getting too many people who don’t understand the code, you’d have the same problem if you write in obscure languages and don’t document patterns and give no guidance for newcomers.
Don’t use LLMs as scapegoats. If you can’t handle the volume of activity LLMs can create, you couldn’t handle human growth and attention either–it’ll be hard and annoying, but I guarantee your project will be healthier by addressing the problems the LLMs are surfacing rather than trying to ban the LLMs themselves.
Want to discuss this post? | Discuss via email | Hacker News | Lobsters |