# The Promise and Peril of Foundation Models in Biology and Medicine
## Introduction
In recent years, foundation models—large-scale artificial intelligence systems pre-trained on broad datasets and then adapted to a wide range of downstream tasks—have emerged as a transformative force across the biological and medical sciences. From predicting the three-dimensional structures of proteins to interpreting electronic health records and analyzing single-cell genomic data, these models have generated tremendous excitement and significant investment. Yet alongside this excitement, a growing body of critical research has raised important questions about the reliability, generalizability, and scientific rigor of these approaches.
This article explores the landscape of foundation models in biology and medicine, examining both their remarkable achievements and the serious challenges that researchers must confront if these tools are to fulfill their potential.
## The Rise of Foundation Models in the Life Sciences
The concept of a foundation model in biology draws inspiration from general-purpose AI systems that have demonstrated extraordinary capabilities across diverse domains. In computational biology, foundation models are typically trained on vast corpora of biological data—protein sequences, genomic text, medical images, or electronic health records—and are then fine-tuned or prompted for specific scientific tasks.
The protein structure prediction field provides perhaps the most celebrated success story. Systems like AlphaFold and its successors have achieved near-experimental accuracy in predicting the three-dimensional structures of individual proteins, a feat that had eluded computational biologists for decades. Independent community-wide assessments, such as the Critical Assessment of protein Structure Prediction (CASP) competitions, have provided rigorous external validation of these breakthroughs, demonstrating that community-driven evaluation can establish genuine methodological advances. More recently, newer models have extended this capability to predict not only protein structures but also the interactions between biomolecules, including protein-ligand binding affinities, with increasing speed and accuracy.
Beyond proteins, foundation models are being developed for an expanding array of biological modalities. Genomic language models trained on the DNA sequences of diverse organisms can now predict regulatory elements, model evolutionary constraints, and even design novel genomes across all domains of life. In single-cell biology, transformer-based models trained on millions of individual cell transcriptomes are being used to predict the effects of genetic perturbations, annotate cell types, and integrate heterogeneous datasets. Meanwhile, multimodal models that combine information from RNA, protein, and other molecular layers are beginning to approach the vision of a “virtual cell”—a computational representation capable of simulating the behavior of living systems from first principles.
In the clinical domain, foundation models trained on electronic health records are being explored for tasks such as disease prediction, patient risk stratification, and clinical decision support. Medical imaging models are similarly advancing, with applications ranging from radiology report generation to pathology image analysis.
## The Benchmarking Crisis
Despite these advances, the scientific community has increasingly recognized a troubling problem: many of the benchmarks used to evaluate foundation models in biology may not measure what they claim to measure. A pattern of critical re-examination has revealed that commonly used evaluation frameworks can systematically overestimate model performance or fail to detect meaningful limitations.
One significant concern is the “self-assessment trap,” where models appear to outperform benchmarks simply because those benchmarks lack sufficiently strong baselines. Several studies have demonstrated that simple linear models or uninformative baselines can match or exceed the performance of deep neural networks on certain perturbation-response prediction tasks when appropriate controls are applied. This finding underscores the importance of rigorous experimental design in model evaluation.
The “Clever Hans effect”—named after a horse that appeared to perform arithmetic by responding to unconscious cues from its handler—has also been identified as a pervasive problem in biomedical AI. Models can learn to exploit spurious correlations in datasets, such as batch effects or technical artifacts, rather than genuine biological signals. In medical imaging, for example, models trained on data from one hospital may perform poorly when applied to data from another institution because they have learned to associate imaging equipment differences with disease labels rather than learning medically meaningful features.
Furthermore, widely used metrics for evaluating single-cell data integration—such as silhouette scores—have been shown to have significant shortcomings. These metrics can fail to detect when model-generated embeddings distort biologically meaningful relationships between cell states, leading researchers to mistakenly believe they have achieved high-quality integration. Cell embedding evaluation frameworks need to be more sensitive to the preservation of biological structure, not just statistical properties of the data.
## The Reproducibility Challenge
The reliability concerns surrounding foundation models in biology intersect with a broader reproducibility crisis that has long affected the biomedical sciences. As machine-learning-based methods become more central to scientific discovery, the risk that poorly validated models will propagate errors through the research literature grows ever larger.
Data leakage represents one of the most insidious forms of methodological failure. When information from the test set inadvertently leaks into the training process—through shared patient samples, overlapping temporal windows, or pre-processing steps that use global dataset statistics—models can achieve deceptively impressive performance that does not generalize to new data. In single-cell genomics, for instance, private patient information can potentially be inferred from model outputs trained on count matrices, raising both privacy and validity concerns.
The problem is compounded by the fact that foundation models are often trained on proprietary or inaccessible datasets, making it difficult for independent researchers to verify claims or reproduce results. This opacity is particularly concerning in clinical applications, where a widely deployed sepsis prediction model, for example, was found to fail dramatically when externally validated on patient populations from different hospital systems.
## Toward More Rigorous Science
Addressing these challenges requires a multifaceted approach. Community-driven benchmarking initiatives, modeled on the success of CASP in protein structure prediction, can provide independent and transparent evaluation of model capabilities. Initiatives calling for industry-led efforts to critically assess machine learning for real-world drug discovery have highlighted the importance of out-of-distribution testing and the inclusion of properly calibrated baselines.
Standardized evaluation frameworks are also emerging for specific domains. In single-cell biology, several comprehensive benchmarks have been proposed for evaluating transcriptomic foundation models on tasks such as perturbation response prediction, gene function inference, and data integration. These benchmarks increasingly emphasize the importance of testing models on truly unseen perturbations and biological contexts, rather than allowing models to exploit dataset-specific patterns.
Transparency and reproducibility standards for machine learning in the life sciences have been formally articulated, calling for practices such as code and data sharing, detailed documentation of preprocessing pipelines, and clear reporting of both positive and negative results. The development of open-source evaluation tools and publicly accessible datasets—such as freely accessible electronic health record repositories—has been an important step toward making the field more accountable.
## Realistic Expectations and Future Directions
It is important to recognize that foundation models, for all their power, have fundamental limitations. Recent work has shown that representations from current genomic foundation models provide little to no advantage over simple one-hot sequence encodings for certain regulatory genomics tasks, suggesting that generic genome-scale pretraining does not necessarily yield transferable biological understanding. Similarly, some deep-learning-based perturbation prediction models have been found not to outperform simple linear baselines when appropriate comparisons are made, highlighting that the gap between foundation models and traditional methods may be narrower than initially claimed for many biological tasks.
Nevertheless, the trajectory of progress is promising. Parameter-efficient fine-tuning techniques are making it possible to adapt large models to specific biological applications without requiring massive computational resources. Multimodal models that integrate diverse data types—from genomic sequences to medical images to clinical records—are beginning to capture the complexity of biological systems in ways that single-modality approaches cannot. And the growing emphasis on rigorous evaluation and transparent methodology promises to produce foundation models that are not only powerful but also trustworthy.
The path forward will require sustained collaboration between AI researchers, domain scientists, clinicians, and the broader scientific community. Establishing norms of reproducibility, demanding strong baselines in model evaluation, and maintaining healthy skepticism about claims of generalizability will be essential to ensuring that foundation models deliver genuine scientific progress rather than a cycle of overhyped but unreliable results.
## Frequently Asked Questions
**What exactly is a foundation model in biology?**
A foundation model in biology is a large-scale artificial intelligence system that is pre-trained on broad biological datasets—such as protein sequences, DNA, RNA transcripts, or medical records—and can then be adapted to many different downstream tasks. Rather than being built for a single narrow problem, these models aim to capture general principles of biology that can be useful across applications.
**Why are foundation models controversial in the life sciences?**
While foundation models have achieved impressive results, they have also been criticized for unreliable performance when applied to new data, susceptibility to spurious correlations, and a tendency to produce results that cannot be easily reproduced. Several studies have shown that models trained on biomedical data can fail to generalize across institutions, populations, or experimental conditions, raising concerns about their reliability for real-world scientific and clinical use.
**How do benchmarking problems affect the evaluation of biological AI models?**
Benchmarking problems arise when the test datasets or evaluation metrics used to judge a model’s performance contain biases, artifacts, or insufficiently strong comparison baselines. This can lead to inflated performance estimates that do not reflect the model’s true capabilities. Common issues include data leakage, where information from the test set contaminates the training process, and the failure to include proper controls that match the complexity of the model being evaluated.
**What is the relationship between foundation models and the broader reproducibility crisis in science?**
Foundation models amplify many of the challenges already present in the reproducibility crisis. Because these models are often trained on large, opaque datasets using complex and poorly documented methods, it can be difficult for other researchers to verify results or reproduce findings. The stakes are particularly high in biomedical research, where flawed models could mislead drug discovery efforts, clinical decision-making, or scientific conclusions.
**Can foundation models replace traditional scientific methods?**
Foundation models are best understood as powerful tools that complement, rather than replace, traditional scientific approaches. They can accelerate hypothesis generation, identify patterns in large datasets, and reduce the cost of certain computations. However, they remain dependent on the quality of the data they are trained on, the rigor of their evaluation, and the biological expertise of the scientists who interpret their outputs. Robust scientific methodology—including careful experimental design, independent validation, and mechanistic understanding—remains essential.
**What steps can improve the reliability of foundation models in biology?**
Key steps include conducting independent external validation of models on diverse datasets, using strong and properly chosen baselines in all benchmarking studies, implementing standardized transparency and reporting requirements, sharing code and data whenever possible, and adopting community-driven evaluation frameworks modeled on successful precedents like the CASP protein structure prediction competitions.
## Conclusion
Foundation models represent one of the most exciting developments in computational biology and medicine, offering unprecedented capabilities for understanding complex biological systems and improving healthcare outcomes. However, the gap between the promise and the reality of these models is significant, and closing that gap will require a committed effort to improve the rigor, transparency, and reproducibility of the research that underlies them. The scientific community must resist the temptation to celebrate impressive benchmark scores without scrutinizing whether those scores reflect genuine biological insight or merely the exploitation of dataset artifacts. By embracing the principles of careful evaluation, open science, and healthy skepticism, researchers can ensure that foundation models fulfill their extraordinary potential as tools for advancing human health and understanding the fundamental mechanisms of life.
Thank you for reading
