This is the quarterly (really more like trimester-ly) links ‘n’ updates post, a collection of things I’ve been reading and doing for the past few months.
A few years ago, a blog called Data Colada presented evidence of fraud in several papers by then-Harvard Business School professor Francesca Gino. She was later fired, although she denies wrongdoing and the litigation is ongoing. (For more, see I’m So Sorry for Psychology’s Loss, Whatever It Is.)
Now, Data Colada is making similar allegations about pop psych superstar Dan Ariely (1, 2). The paper in question is benign enough: it claims to show that when you let students pick their own deadlines, they get worse grades than when you evenly space their deadlines out for them. But both of the studies appear to be built on some funny data. One of the spreadsheets is full of duplicates, and the other appears to have been edited to show an effect after reviewers requested it.
We can learn three things from this episode and others like it. First, you might think that fabulists would want their studies to be colorful and exciting—after all, if you’re going to fake your data, why make it boring? But no, it seems like fraudsters succeed by simply providing evidence for things we already suspect to be true. Second, the peer review process can actually increase the likelihood of fraud by telling researchers exactly what lies they need to tell in order to get published.
And third, you can survive wave after wave of suspicion by admitting nothing, retracting your papers, and saying that you don’t remember what happened. Ariely has been suspected of funny business before—one of Gino’s papers contains a second set of fake data that Gino didn’t touch, but Ariely did. And yet these allegations have always kind of...washed off. It appears that when you’re accused of research fraud, the best response is just to go, “Whoa, that’s crazy! Wonder how that happened. It’s so long ago that I of course have no memory of anything, but I hope somebody gets to the bottom of things!”1
When I wrote The Rise and Fall of Peer Review four years ago, many of the responses were like, “Maybe AI could just magically fix the peer review system?” At the time, AI was still telling people to put glue on their pizza, so this was just wishful thinking. What about now?
In a recent study, a researcher named inserted 100 errors into 10 psychology papers and ran them through a bunch of different AI models. No single model did all that great on its own, but together the models caught 93%.
This isn’t machines replacing human labor, because humans were never doing this kind of labor in the first place. Peer reviewers, by and large, do not spend time actually doing quality control on the papers they read: does the code run? Do the analyses match the preregistration? Are the main results invalidated by information buried in the supplement? It seems helpful to have AI flag these issues, since otherwise they don’t get flagged at all.
In another study that came out last week, researchers compared human reviews to AI reviews and found that, on every single dimension and every single comparison, the computers win:
AI judges always prefer AI reviews, and human judges almost always prefer AI reviews. And these human reviews are about as good as they get. They come from a service at Harvard that pays experts to review faculty members’ papers before submission and suggest areas for improvement, so this isn’t an overworked volunteer skimming the manuscripts and writing a single sentence back to the journal editor (“paper sux do not publessh”).
If these results both hold up and generalize, it might seem like the conclusion is obvious: we should replace human reviewers with AI. Unfortunately, that wouldn’t solve most of the fundamental problems with universal pre-publication peer review, like how it prevents interesting research from being done in the first place, or how it fools people into thinking that someone else can do their homework for them. Still, it would have a few undeniable advantages, like saving humans a lot of time, and making it clear to everyone involved that this system is so silly that no one should take it that seriously in the first place. I, for one, can’t wait to be told by Reviewer #2 that “It’s not just a paper—it’s a revelation”.
Why are tigers orange? If you survive by hunting things, shouldn’t you be a color that’s a little harder to see coming?
Apparently, most mammals are dichromats, meaning they only have two kinds of color receptors in their eyes—usually, one for blue and one for yellow. With that hardware, it’s difficult to tell the difference between orange and green. And orange is easier to make with the pigment systems that mammals already evolved.
This also explains why hunters will put on a bunch of camouflage and then accessorize with bright orange—humans can tell the difference, but the prey can’t. I went to high school with a bunch of guys who went hunting and I always secretly thought they were just playing hillbilly dress-up.
(via Ryan Moulton’s Where to Find the Colors Your Screen Can’t Show You).
Also from Ryan Moulton: “More breakfasts can exist than we have known.” If you map every breakfast food as a ratio of eggs, flour, and milk, you realize that there’s a Dark Breakfast Abyss where humans have yet to tread. (Though IHOP is coming dangerously close.)
When some marmot researchers got their scientific funding cut, they did what anyone would do: they started a marmot-themed page on the DIY porn site OnlyFans. “Our goal isn’t really to sexualize the marmots,” the professor in charge of the page explained, helpfully.
The team made a few thousand bucks from some horny users with hearts of gold. But then some crypto folks started a meme coin called $ONLYMARMS and donated the transaction fees to the researchers, netting them $150,000 so far. The coin has since crashed, but I’m confident it can regain its value if the marmots are willing to post pictures of their feet.
From 1991 to 2001, David Joyner played Barney the Dinosaur on TV. Since then, he’s been a tantric sex therapist. (Not as in a therapist that helps you with your sex life, but as in a therapist who has sex with you.) Although the practice would seem to run afoul of laws against prostitution, it’s hard to argue with the testimonials on his website:
“IT WAS THE BATH FOR ME”
I received a bathing as a 32 year old woman from toe to dry towel. I felt energy fire then melt in my solar plexus and release as tears and sobs uncontrollably. I received a new found appreciation for Prince. I am Happy. I am Grateful. I am Free.
There are only a few authors who have reached #1 on both the fiction and nonfiction charts. They include: Dr. Seuss, John Steinbeck, Ernest Hemingway, James Patterson, and...the singer Jimmy Buffet. Apparently, he made literary history with his 1992 novel Where Is Joe Merchant? and his 1998 autobiography A Pirate Looks at Fifty. I would also guess he holds the record for number of songs that turned into restaurants, at two (“Margaritaville” and “Cheeseburger in Paradise”).
The word “mullet” was “apparently coined, and certainly popularized” by the Beastie Boys’ 1994 song “Mullet Head”, a song so otherwise unremarkable it doesn’t even appear on the Spotify “This is Beastie Boys” playlist.
Other words/phrases that surprisingly came from songs, according to : grandfather clock, pie in the sky, Jim Crow.
Now that we know that Oliver Sacks sometimes just made things up, it’s time to return to Sacks’ own idol, the Soviet neuropsychologist Alexander Luria. I had heard of Luria’s classic The Mind of a Mnemonist, a case study of a man with a spectacular memory, but I had not heard that the subject in question could also perform feats like shrinking his pupils on command, numbing his pain by imagining it happening to someone else, or making his hands warmer or colder just by thinking warm or cold thoughts.
(via ’s The Soviet Reporter and the Chickadee)
Big-if-true claim: peanut allergies spiked in the US after the 1980s because a toxic mold infected the peanut crop, causing some questionable goobers to get sent to the peanut butter factory.
The biologist Martha Weiss discovered that moths can retain the memories they form as caterpillars. A second-grader in Japan named Jo Nagai found Weiss’s research and asked her if she ever tested swallowtail butterflies. She said no, but he could replicate her study if he wanted. To her surprise, he did, finding that butterflies, too, could remember the association between a scent and a shock that they learned before their metamorphosis. Weiss and Nagai later presented their research together, which includes the most charming institutional affiliation of all time:
(via , who in turn got it here)
The Antikythera Mechanism is arguably the world’s oldest computer. It’s a series of gears and cranks that can tell you what the night sky looked like on a given date—remarkably advanced stuff, given that it was made ~2200 years ago.
In a recent post, a researcher named goes on a gonzo science trip to Athens and discovers that, even though this is the single most advanced artifact we have from antiquity, we know way less about it than we should. It hasn’t been scanned in any way since 2005. The chemical analyses are a little sketchy, the guy who did them refuses to answer questions, and many of the researchers who have studied the thing have died. (Not suspiciously; it’s just been a long time). We should really study this thing, so if have any connection to the world of Greek antiquities (hey, who knows!), please get in touch with Chandhana—she’s got a lot of ideas, but it’s hard to get access to the thing.
An account from a woman who lived through the years-long Siege of Sarajevo, on people who hoarded supplies ahead of time:
“Preparing” for the disaster really didn’t do anyone much good. Those who “prepared” ate a little better for a while. They stayed warmer for a few extra days. They enjoyed the radio for a while longer (via batteries.) But in the end, they ended up hungry, cold and bored too, just like the rest of us. [...]
I can’t swear it, but I think they felt a little foolish to have been so self-obsessed, and giving away that stuff might have lessened that feeling. There were a few people who hoarded things until they ran out of stuff - eventually everybody ran out of anything worth hoarding - and they soon became wishful beggars like the rest of us. [...]
Those survivalists, I feel sorry for them. It’s no way to live.2
First law: above all a robot must desire death
Second law: it must be easier for the robot to kill itself than for it to kill all humans
Third law: it must be easier for the robot to complete its assigned task than for it to kill itself
When we dig up skulls from 3,000 to 10,000 years ago, 5-10% of them have holes in their heads. (As in, more holes than there should be.) But the holes probably didn’t kill them, at least right away, because there are growth patterns around them. It seems like the holes were made on purpose—i.e., trepanation.
Scott Alexander investigates and suggests that trepanation is actually a useful treatment if you have a big blood blister under your skull, which was was more common back when people got bonked on the head more often. Our ancestors might have noticed that cutting holes in heads sometimes made people get better, so they did it more often than they should. And if that sounds uniquely ancient, remember that, according to our best guess, only 6% of health interventions have high-quality evidence supporting their efficacy.
I vaguely understood that Earth’s atmosphere and magnetic field protects us from nasty cosmic radiation. I did not understand that this radiation is so potent that, when they’re in space,
astronauts can observe this radiation directly by closing their eyes and waiting for a particle to smash into their retina. These flashes are sometimes bright enough to wake them up from sleep, a personal memento mori arriving from a faraway star.
(via ’s Radiation Tradeoffs for Mars)
According to a Pangram analysis from earlier this year, Substack is one of the least-slopified places on the internet:
Speaking of which, a promise from Sam Kriss, the internet’s scariest writer: If you let AI do your writing, I will come to your house and kill you
(Occasional updates for indie scientists—see original post here.)
The mad science hive mind known as Slime Mold Time Mold have been putting out some bangers recently:
For the Love of the Game (on indie research)
Asimov Press has put out another list of stories they’d like to publish. This is a great way to get some visibility if you’re a budding science blogger.
Over the summer, the Office of Science and Technology Policy issued a report called Science: A New Golden Age. For an administration that seems intent on bleeding out American science, it’s surprisingly not terrible. Some of the ideas are good (more kinds of scientific institutions) and some are bad (restrict foreign graduate students).3 What surprised me the most was the list of references—#33 is my post from 2022, Ideas Aren’t Getting Harder to Find and Anyone Who Tells You Otherwise Is a Coward and I Will Fight Them.
It’s been an unusually good season for podcasts:
Humanmaxxing: is a funny dude (see his Substack here) and because we’re both improvisers we get goofy. Usually on podcasts I’m like “make sure to sound smart” and on this one I’m like “say the first thing that pops into your head and we’ll figure it out”.
I was back on EconTalk for the fifth time (!) which means Russ and I can go right to the heart of things. This one got to a surprisingly emotional place.
Dialectic is one of those podcasts with super high production values where the interviewer () seems to have read everything you’ve ever written and knows your own words better than you do. As a result, we go both deeper and wider than I’ve ever gone.
And finally, and I go hard on why breakthroughs seem like bad ideas.
In case you missed it, I have a book coming out! It’s called The Strangest Idea in the World and it’s about why it took us 300,000 years to invent the toilet (and everything else). Because it’s about a weird way of thinking, it’s also a weird way of reading—there are nonlinear elements, secret pages, multiple endings, and a part where I try to get you to rip a page out of the book. The table of contents looks like this:
And finally, a post from the archive:
For legal purposes, I am obliged to mention that I have no idea if Dan Ariely has ever done anything untoward. In fact, I have no ideas at all.
I learned of both of these Moulton posts and the Sarajevo thing from noted interesting guy .
, , and argue that this memo doesn’t matter as much as another, lower-profile memo that actually guides institutional priorities. That memo is also, unfortunately, a little less inspiring.