Back Original

The Hallway Track No. 027: AGMAI advisory group, Millennium Problems for biology, autonomous research pipelines

When an AI system produces a scientific result, someone still has to decide whether it holds up, and three of this week's items answer that in different ways. In mathematics, nine researchers including Timothy Gowers and Edward Witten have formed an advisory group at the Institute for Advanced Study to help OpenAI decide how to release a batch of machine-produced results it has not yet published, which comes down to setting the terms on which those results enter the literature. ScientistTwo, a multi-agent system from Google, is judged the other way around: its papers are graded by AI reviewers and score higher than human papers accepted at ICLR, ICML, and NeurIPS, so the reviewer is itself a model. The twelve Millennium Problems for biology were chosen so that a human can check each answer in a standard wet lab within days to weeks, which keeps the test in a person's hands.

Autonomous research pipelines

Benchmarks and the volume problem

Institutions, governance, and the debate