Some scientists have “magic hands” in the lab. This AI learns why.

On a recent afternoon, scientists moved around a laboratory in Cambridge, Massachusetts, conducting experiments. Their equipment was standard: a laboratory hood for working with dangerous chemicals, an incubator for growing cells. Their procedures were everywhere the same as in biological laboratories.

However, a closer look revealed some peculiarities. Along with lab coats and gloves, most scientists wore miniature cameras on their headbands. Each workstation was watched from a shelf by three additional cameras.

Lab notebooks were strangely absent. Instead, the scientists quietly recounted their work, muttering into microphones. The team huddled at one end of the lab, watching videos of the experiments.

That was the real research going on in this lab. Using a system of sensors and software designed to capture science as it happens down to the millisecond, the scientists trained artificial intelligence models to recognize every object the scientists used and every action they took.

The AI ​​even created its own story, describing the videos every few seconds with sentences like, “The operator resuspends the pellet by pipetting it up and down 10 times.”

The technology is the brainchild of a startup called Transfyr, which emerged from stealth mode this week with $25 million in seed funding. Society is grappling with an age-old challenge in science: the hidden factors that make some experiments succeed and others fail.

Failure can take many forms. Researchers can spend months preparing a series of artificially engineered cells, only to have them mysteriously die along the way. A government test for Covid appears to work in one lab but fails to detect the virus when used by others.

A biotech company creates a promising new drug; when he passes the protocol for mass production, suddenly it just doesn’t work.

“It’s an incredibly painful problem,” said Anna Marie Wagner, co-founder of Transfyr. “There’s pointing back and forth. Was your protocol wrong? Or did you screw something up? These are very, very expensive mistakes in terms of time, money and lives.”