How AI in Drug Discovery Is Critically Transforming Clinical Trials, Multi-Omics, and the Road Ahead — Part 3: Systems Biology, Patient Stratification, and Open Challenges

This is the final part of a three-part series on AI in drug discovery. Part 1 covered molecular representations, graph neural networks, and transfer learning for QSAR modelling. Part 2 covered protein structure prediction with AlphaFold, generative molecular design, and deep learning virtual screening. Part 3 examines how AI is being applied beyond the molecule: to systems-level disease biology, clinical trial optimisation, and the open theoretical and practical challenges that remain.

Beyond the Molecule

Parts 1 and 2 of this series focused on AI in drug discovery at the molecular scale: representing chemical structures, predicting binding affinities, generating candidate molecules, and screening compound libraries computationally. These approaches operate primarily on the drug-target interaction as an isolated system. But disease biology is not isolated. A drug candidate that binds its intended target with nanomolar affinity may fail in clinical trials because the disease it is meant to treat is driven by a complex network of interacting molecular processes, only one node of which is the chosen target.

The next frontier in AI in drug discovery is the integration of this systems-level complexity into the computational pipeline. This requires moving from molecular representations of individual compounds to multi-modal representations of biological systems: gene expression profiles, protein interaction networks, genomic variants, epigenetic modifications, metabolite concentrations, and clinical phenotypes, simultaneously. It also requires applying AI to the later stages of the pipeline where most drug failures actually occur: clinical trial design, patient stratification, and the prediction of clinical outcomes from pre-clinical data.

Multi-Omics Integration and Disease Biology

The term “omics” refers to the large-scale measurement of biological molecules at a systems level. Genomics measures DNA sequence variants. Transcriptomics measures gene expression levels across the genome. Proteomics measures protein abundances and modifications. Metabolomics measures the concentrations of small-molecule metabolites. Epigenomics measures chemical modifications to DNA and histones that regulate gene expression without changing the sequence. Each of these data modalities provides a partial view of the molecular state of a cell, tissue, or organism. Integrating them provides a far richer picture of disease biology than any single modality can offer.

Multi-omics data integration presents substantial machine learning challenges. The datasets are high-dimensional: a transcriptomics dataset may have expression measurements for 20,000 genes across thousands of patient samples. They are heterogeneous: different modalities have different scales, noise characteristics, and missing data patterns. They are multi-scale: genomic variants act through intermediate molecular mechanisms to produce phenotypic consequences, and causal relationships must be traced across these scales. And they are confounded: patient samples differ in age, sex, tissue type, disease stage, and treatment history in ways that must be accounted for before meaningful biological signals can be extracted.

The dominant deep learning approach for multi-omics integration is multi-modal representation learning. A separate encoder network is trained for each data modality, projecting high-dimensional omics measurements into a shared low-dimensional embedding space. The encoders are trained jointly using a contrastive objective that brings the embeddings of matched samples (different modalities measured from the same patient) close together while pushing the embeddings of unmatched samples apart:$$\mathcal{L}_{\text{contrastive}} = -\frac{1}{N}\sum_{i=1}^{N} \log \frac{\exp(\text{sim}(\mathbf{z}_i^A, \mathbf{z}_i^B) / \tau)}{\sum_{j=1}^{N} \exp(\text{sim}(\mathbf{z}_i^A, \mathbf{z}_j^B) / \tau)}$$

where $\mathbf{z}_i^A$ and $\mathbf{z}_i^B$​ are the embeddings of sample $i$ in modalities $A$ and $B$, $\text{sim}(\cdot, \cdot)$ is cosine similarity, and $\tau$ is a temperature parameter. This objective, directly analogous to the CLIP contrastive loss used in vision-language models, produces a shared embedding space in which biological similarity is encoded as geometric proximity regardless of which combination of modalities was measured for a given sample.

The shared embeddings produced by multi-omics integration models have been used for disease subtype discovery, biomarker identification, and drug repurposing, finding new therapeutic applications for existing approved drugs. In the context of AI in drug discovery, multi-omics integration is most powerful when it is used to identify the molecular signatures that distinguish patients who respond to a drug from those who do not, which is the patient stratification problem at the heart of clinical trial design.

Knowledge Graphs and Biological Network Reasoning

A complementary approach to multi-omics integration for AI in drug discovery is the use of biological knowledge graphs: large heterogeneous graphs that encode known relationships between genes, proteins, diseases, drugs, pathways, and phenotypes, extracted from databases such as UniProt, DrugBank, DisGeNET, and the Gene Ontology. A node in a biological knowledge graph might represent a protein, a disease, a drug, or a biological pathway, and edges encode relationships such as “drug inhibits protein,” “gene is associated with disease,” or “protein participates in pathway.”

Graph neural networks applied to biological knowledge graphs can predict new edges: new drug-target interactions, new gene-disease associations, or new drug repurposing opportunities. The theoretical basis for these predictions is the relational inductive bias of graph neural networks: patterns of connectivity in the known graph carry information about the probability of unknown connections. A drug that inhibits several targets known to be involved in Alzheimer’s disease pathology is more likely to be therapeutically relevant to Alzheimer’s than a drug with no such connections, even if this association was never explicitly entered into the knowledge base.

Relational graph convolutional networks (R-GCNs) extend the message-passing framework of standard GNNs to heterogeneous graphs with multiple edge types. For a node $v$v with neighbours of relation type $r$r:$$\mathbf{h}_v^{(l+1)} = \sigma\!\left(\sum_{r \in \mathcal{R}} \sum_{u \in \mathcal{N}_r(v)} \frac{1}{c_{v,r}} W_r^{(l)} \mathbf{h}_u^{(l)} + W_0^{(l)} \mathbf{h}_v^{(l)}\right)$$

where $W_r^{(l)}$​ is a relation-specific weight matrix, $\mathcal{N}_r(v)$ is the set of neighbours of $v$ under relation $r$, and $c_{v,r}$ is a normalisation constant. This architecture allows the model to learn distinct aggregation functions for different biological relationship types, which is essential for reasoning over the heterogeneous entity types in a biological knowledge graph.

The knowledge graph approach to AI in drug discovery has produced notable results in drug repurposing. During the COVID-19 pandemic, knowledge graph models trained on pre-pandemic biological databases predicted several drug candidates that were subsequently validated in clinical studies, demonstrating that the relational information encoded in known biology carries genuine predictive signal for novel therapeutic questions.

Clinical Trial Optimisation and Patient Stratification

The majority of drug failures in AI in drug discovery occur not in the laboratory but in clinical trials, and the majority of those failures are attributable to efficacy failures rather than safety. A drug that works in the average patient population may fail in a heterogeneous trial cohort because the biological mechanism it targets is only active in a specific patient subgroup. Identifying that subgroup prospectively, before the trial begins, is the patient stratification problem.

AI approaches to patient stratification use the multi-omics representations described above to cluster patients by molecular subtype, identify biomarkers that predict treatment response, and design enriched trial cohorts that are more likely to show a statistically detectable treatment effect. The theoretical framework is that of heterogeneous treatment effect estimation: the goal is not to estimate the average treatment effect across the population but to estimate the conditional average treatment effect for each patient as a function of their molecular and clinical features:$$\tau(\mathbf{x}) = \mathbb{E}\!\left[Y^{(1)} – Y^{(0)} \mid \mathbf{X} = \mathbf{x}\right]$$

where $Y^{(1)}$ and $Y^{(0)}$ are the potential outcomes under treatment and control respectively, and $\mathbf{X}$ is the patient feature vector. Causal forest models and their neural network extensions, including the TARNet and DragonNet architectures, estimate $\tau(\mathbf{x})$ from observational or randomised trial data while controlling for confounding between the patient features and treatment assignment.

Beyond patient stratification, AI in drug discovery is being applied to trial design itself. Bayesian adaptive trial designs use AI models to update the trial protocol in response to accumulating data, adjusting dose levels, sample sizes, and patient inclusion criteria based on interim results. Synthetic control arms, generated by matching trial patients to historical patient records using deep learning-based propensity models, can reduce the size of placebo arms and accelerate trial timelines. And natural language processing models applied to electronic health records can identify eligible patients for recruitment significantly faster than manual chart review, which is one of the principal bottlenecks in trial execution.

Foundation Models for Biology

The most significant recent development in AI in drug discovery is the emergence of large foundation models pre-trained on biological sequence data at a scale comparable to the language model pre-training described in the LLM series on this blog. ESM-3, released by EvolutionaryScale in 2024, is a 98-billion-parameter model jointly trained on protein sequences, structures, and functional annotations. Like a language model predicting the next token in a text sequence, ESM-3 learns to predict masked amino acid residues, masked structural tokens, and masked functional labels simultaneously, producing a unified representation of protein biology across sequence, structure, and function.

For AI in drug discovery, foundation models for biology offer the same advantages that LLM pre-training offers for natural language tasks: rich general-purpose representations that can be fine-tuned on small labelled datasets for specific prediction tasks, dramatically reducing the data requirements for new applications. A foundation model pre-trained on hundreds of millions of protein sequences and structures can be fine-tuned to predict the effect of a specific mutation on drug binding affinity using only a few hundred experimental measurements, a capability that classical QSAR methods could not approach at this sample size.

The logical extension of sequence-level foundation models is multi-modal biological foundation models that integrate molecular, cellular, tissue, and organismal data simultaneously. Projects including Geneformer, scGPT, and the Biological Foundation Model consortium are pursuing this vision, training transformer architectures on single-cell RNA sequencing data from tens of millions of cells to learn general-purpose cellular representations.

The theoretical aspiration of this line of research is a model that can answer arbitrary questions about biological systems: what is the effect of inhibiting this protein in this cell type in this disease context? This is the generalised inverse problem of systems biology, and it is the horizon toward which the most ambitious applications of AI in drug discovery are oriented.

Open Challenges and Theoretical Limitations

An honest assessment of AI in drug discovery requires acknowledging the substantial theoretical and practical challenges that remain, and that partially explain why the transformation of the drug development pipeline has been slower than the most optimistic early predictions suggested.

The data quality problem is perhaps the most fundamental. Machine learning models are only as good as their training data, and biological activity data is notoriously noisy, heterogeneous, and difficult to compare across experimental protocols. IC50 measurements, the most common metric of binding affinity in drug discovery datasets, can vary by an order of magnitude between laboratories for the same compound and target, depending on assay conditions, cell lines, and measurement protocols. Models trained on these heterogeneous datasets learn to fit the noise as well as the signal, producing predictions that generalise poorly to new experimental settings.

The distribution shift problem is closely related. Drug discovery models are trained on historical datasets of compounds that have been prioritised by human medicinal chemists using their own intuitions about what makes a good drug candidate. The training distribution is therefore heavily biased toward certain chemical scaffolds, certain target classes, and certain disease areas. Models trained on these datasets may perform well in regions of chemical space near the training distribution but fail catastrophically when applied to genuinely novel scaffolds or target classes. This is particularly concerning for AI in drug discovery, because the most valuable applications are precisely those that require exploring regions of chemical space far from what has been studied before.

The synthesis and experimental validation bottleneck constrains the practical impact of even the most accurate computational models. A generative model can propose millions of candidate molecules in hours, but each candidate must still be physically synthesised and experimentally tested before it can progress. Synthesis is slow, expensive, and frequently fails for complex or novel structures. The gap between computational proposal and experimental validation remains the primary rate-limiting step in AI in drug discovery pipelines, and closing it requires advances in automated synthesis and high-throughput experimental biology that are progressing but not yet at the scale that would fully exploit the computational capabilities described in this series.

The causal inference problem is perhaps the deepest theoretical challenge. Predicting that a molecule will bind to a target is a correlation problem. Predicting that inhibiting a target will produce a therapeutic benefit in patients is a causal problem. The distinction matters enormously: many targets that are statistically associated with disease in genomic studies turn out not to be causal drivers of disease, and drugs that inhibit them fail in clinical trials despite performing well in pre-clinical models.

AI models trained on correlational data cannot reliably distinguish causal from spurious associations without additional structure, either in the form of experimental interventional data (which is expensive) or causal modelling assumptions (which may not hold). Incorporating causal reasoning into AI in drug discovery is an active research frontier at the intersection of machine learning, statistics, and molecular biology.

Conclusion: What AI in Drug Discovery Can and Cannot Yet Do

AI in drug discovery has already produced tangible contributions at every stage of the pipeline this series has examined. AlphaFold has made protein structure prediction routine, enabling structure-based drug design for targets that were previously inaccessible. Deep learning scoring functions have accelerated virtual screening by orders of magnitude. Generative models are proposing molecules in previously unexplored regions of chemical space. Multi-omics integration is enabling patient stratification approaches that were not possible with classical biostatistics. And foundation models for biology are beginning to provide the kind of general-purpose biological reasoning that could eventually make the idealised version of AI-driven drug discovery a practical reality.

What AI in drug discovery cannot yet do is reliably translate these molecular-level capabilities into clinical success. The 90% clinical trial failure rate has not yet moved significantly, and the gap between pre-clinical AI performance and clinical outcomes remains the field’s defining open problem. Closing that gap will require not just better models but better data, better experimental feedback loops, better causal reasoning, and a deeper integration of AI tools with the biological and clinical expertise of the humans who understand disease in its full complexity.

The molecules are becoming easier to find. Making them into medicines remains hard. That is where the most important work in AI in drug discovery lies, and it is where the field will be judged over the decade ahead.

This concludes the three-part series on AI in drug discovery. Recommended further reading includes the AlphaFold 2 paper in Nature (Jumper et al., 2021), the REINVENT paper from AstraZeneca, and the ESM-3 technical report from EvolutionaryScale.