The key word in data science is still science

My last post before starting my lab went up in October 2022. It was a revisit of a famous commentary by Jeff Leek, so now that I am writing again, it seems fitting to pick up where I left off.

In 2013, Jeff Leek wrote a now-famous blog post arguing that the key word in “data science” is science. In that 2022 post, I revisited it with a simple 2x2 table: near-perfect or imperfect data on one side, near-perfect or imperfect science on the other. Good data with good science is a win-win, bad data with bad science is a lose-lose, and the two cells in between are where a data scientist earns their keep.

I wrote that a few weeks before large language models became everyday tools. Four years on, an AI assistant can clean a dataset, fit a dozen models, and write up the results in the time it takes to make a coffee. So it is fair to ask whether the 2x2 table still holds. I think it does, and I think its off-diagonal cells matter more than they did.

What got cheaper

The mechanics of analysis: code, pipelines, standard models, the first draft of a report. In the “near-perfect data” column this is all gain. A simple model was already enough there, and now it arrives faster.

What did not get cheaper

Knowing whether the answer is right. An AI system that is handed imperfect data still returns a fluent, well-formatted, confident result. “Garbage in, garbage out” has acquired a more dangerous cousin: garbage in, polished prose out. The lose-lose cell of the table now looks like a win on the surface, which makes the combination of imperfect data and imperfect science harder to spot than it was in 2013 or in 2022.

An example from the microbiome field

Microbiome research makes the point concrete. In a recent essay, Ran Blekhman reviews why AI keeps failing at microbiome prediction. In the largest benchmark he discusses, 156 modeling pipelines were compared across 83 gut microbiome cohorts and 20 diseases, and ridge regression and random forest came out on top, ahead of neural networks. In another study, a transformer trained on American Gut Project data was outperformed by a plain random forest on nearly every task. The reasons are scientific ones: the typical study has about 160 samples, more than half of the entries in an abundance table are zeros, and the signal linking microbes to disease is modest. In Blekhman’s words, “the bottleneck is biological, not algorithmic.”

The same essay shows where deep learning does help: generalizing across studies, and integrating the microbiome with metabolomics, host gene expression or clinical data. Knowing which of the two situations you are in is the science.

The 2026 edition of the table

• Near-perfect data + AI: the win-win, only faster.
• Imperfect data + AI without scientific direction: the new lose-lose, and a well-disguised one.
• Imperfect data + AI directed by science: still the real test of a data scientist, and now within reach of many more people.

Thirteen years after Jeff Leek’s post, the key word in data science is still science.