How AI in Drug Discovery Is Powerfully Transforming the Search for New Medicines — Part 1: The Molecular Foundations

This is Part 1 of a three-part series on AI in drug discovery. Part 1 covers the theoretical foundations: the drug discovery pipeline, molecular representation, and how machine learning models learn to reason about chemical space. Part 2 will cover protein structure prediction, generative molecular design, and virtual screening. Part 3 will examine clinical trial optimisation, multi-omics integration, and the open challenges facing the field.

A Pipeline in Crisis

The pharmaceutical industry operates under a brutal set of statistics. It takes an average of 12 to 15 years and over $2 billion to bring a single new drug from initial discovery to regulatory approval. Roughly 90% of drug candidates that enter clinical trials fail before reaching patients. The attrition is highest at the transition from Phase II to Phase III trials, where drugs that appeared promising in smaller studies fail to demonstrate efficacy or safety at scale. The consequence is that the patients who need new medicines most urgently wait the longest, and the cost of failure is embedded in the price of the drugs that do eventually succeed.

AI in drug discovery is not a single technology applied to a single problem. It is a collection of machine learning, deep learning, and generative modelling approaches applied across every stage of a pipeline that was, until recently, dominated by slow, expensive, and failure-prone experimental methods. Understanding what AI is actually doing in this pipeline, and why it has the potential to change these statistics, requires starting at the molecular level: with how drugs work, how chemical space is structured, and how machine learning models can be made to reason meaningfully about both.

What a Drug Actually Does

A drug is, at its most fundamental level, a molecule that binds to a biological target and modulates its activity in a therapeutically useful way. The target is usually a protein: an enzyme whose activity needs to be inhibited, a receptor whose signalling needs to be blocked or activated, or a transport protein whose function needs to be altered. The drug molecule must bind to the target with sufficient affinity to produce a biological effect, with sufficient selectivity to avoid binding other proteins and causing side effects, with sufficient stability to survive the journey from administration to target site, and with sufficient safety to be tolerable in a living organism.

These four requirements, potency, selectivity, pharmacokinetics, and safety, collectively define what chemists call the multi-parameter optimisation problem of AI in drug discovery. Optimising a molecule for one parameter frequently degrades another. Increasing a molecule’s binding affinity to its target often increases its tendency to bind other proteins. Improving its stability in the body often reduces its ability to cross cell membranes. The search for a molecule that satisfies all constraints simultaneously, within the enormous space of possible drug-like molecules, is the core challenge that AI in drug discovery is being applied to solve.

The Scale of Chemical Space

The number of drug-like small molecules that could theoretically exist is estimated at between $10^{23}$ and $10^{60}$. This range, known as chemical space, is so vast that it dwarfs the number of atoms in the observable universe at its upper bound. The entire historical output of medicinal chemistry, every compound ever synthesised and tested, represents an infinitesimally small sample of this space. Traditional drug discovery navigates this space through a combination of chemical intuition, high-throughput screening, and iterative medicinal chemistry optimisation. High-throughput screening tests libraries of hundreds of thousands of compounds against a target and identifies those with measurable activity. Medicinal chemistry then iteratively modifies the most promising hits to improve their properties.

This approach has two fundamental limitations. First, the compound libraries used in high-throughput screening are biased toward previously synthesised chemical scaffolds, meaning that large regions of potentially valuable chemical space are never explored. Second, the iterative optimisation process is slow and expensive, typically requiring dozens to hundreds of synthesise-test-analyse cycles to progress a hit compound into a viable drug candidate.

AI in drug discovery addresses both limitations directly. Machine learning models can learn the relationship between molecular structure and biological activity from historical data, allowing them to predict the activity of compounds that have never been synthesised. Generative models can propose entirely new molecules in previously unexplored regions of chemical space. And virtual screening using deep learning can evaluate millions of candidate molecules computationally in the time it would take a laboratory to test a few thousand experimentally.

Representing Molecules for Machine Learning

Before any machine learning model can reason about molecules, those molecules must be converted into a numerical representation that the model can process. This is a non-trivial problem, because molecular structure encodes information at multiple levels simultaneously: the identity and connectivity of atoms, the three-dimensional geometry of the molecule, the distribution of electrons across its surface, and the conformational flexibility that determines how it will interact with a protein binding site. Different representations capture different subsets of this information, and the choice of representation significantly affects model performance.

SMILES strings (Simplified Molecular Input Line Entry System) are the most widely used text-based representation of molecular structure. A SMILES string encodes the atoms and bonds of a molecule as a sequence of characters: for example, the SMILES for aspirin is CC(=O)Oc1ccccc1C(=O)O. The simplicity of SMILES makes them compatible with language model architectures: a transformer trained on SMILES strings can learn the grammar of chemical space in much the same way that a language model learns the grammar of English. Models including ChemBERTa and MolGPT use this approach.

Molecular fingerprints are fixed-length binary or count vectors that encode the presence or absence of specific structural features, called substructures, within a molecule. The Morgan fingerprint algorithm, also known as ECFP (Extended Connectivity Fingerprints), generates circular fingerprints by iteratively encoding each atom’s chemical environment to a specified radius. The resulting bit vector can be used directly as input to traditional machine learning models including random forests, support vector machines, and gradient boosting, and forms the basis of many quantitative structure-activity relationship (QSAR) models.

Molecular graphs represent molecules as graphs in which nodes correspond to atoms and edges correspond to bonds, with both nodes and edges carrying feature vectors encoding chemical properties such as atomic number, hybridisation state, formal charge, and bond order. Graph Neural Networks (GNNs) are particularly well-suited to molecular graph representations because they can learn representations that are invariant to the arbitrary numbering of atoms in a molecule, which has no chemical meaning.

3D conformer representations encode the three-dimensional geometry of a molecule, including the coordinates of each atom in space. These representations are essential for modelling protein-ligand interactions, where the shape complementarity between the drug molecule and the protein binding site is a primary determinant of binding affinity. Equivariant neural networks, including SE(3)-Transformers and DiffSBDD, are designed to process 3D molecular representations while respecting the physical symmetries of three-dimensional space: rotation, reflection, and translation of the entire molecule should not change the predicted properties.

Learning Structure-Activity Relationships

The central task of computational drug discovery is learning the relationship between molecular structure and biological activity: given a molecule’s structure, predict whether and how strongly it will bind to a target protein, and with what selectivity over other proteins. This is the quantitative structure-activity relationship (QSAR) modelling problem, which has a history stretching back to the 1960s but has been transformed by deep learning in the past decade.

Classical QSAR models used linear regression and later support vector machines, applied to handcrafted molecular descriptors such as molecular weight, lipophilicity, and hydrogen bond donor count. These models worked reasonably well within narrow chemical series but generalised poorly to structurally diverse compounds, because the descriptors failed to capture the full complexity of molecular structure.

Deep learning models for AI in drug discovery learn their own representations from molecular data rather than relying on handcrafted descriptors. A graph neural network trained on a dataset of measured binding affinities learns to associate specific structural patterns with activity by propagating information across the molecular graph through successive layers of message passing. At each layer, each atom’s representation is updated by aggregating the representations of its bonded neighbours, weighted by learned parameters:$$\mathbf{h}_v^{(l+1)} = \sigma\!\left(W^{(l)} \cdot \text{AGG}\!\left(\left\{\mathbf{h}_u^{(l)} : u \in \mathcal{N}(v)\right\}\right) + B^{(l)} \mathbf{h}_v^{(l)}\right)$$

where $\mathbf{h}_v^{(l)}$​ is the representation of atom $v$v at layer $l$, $\mathcal{N}(v)$ is the set of atoms bonded to $v$, AGG is an aggregation function such as sum or mean, and $\sigma$ is a non-linear activation function. After $L$ layers of message passing, the atom representations encode information about the chemical environment within $L$ bonds of each atom. A global readout function then aggregates the atom representations into a molecular representation, which is passed to a prediction head that outputs the predicted property value.

This architecture has two important theoretical properties for the AI in drug discovery process. First, it is permutation-invariant: the predicted property is independent of the order in which atoms are numbered, which matches the physical reality that molecular identity does not depend on atom numbering. Second, it can generalise across chemical series in a way that fingerprint-based models cannot, because the message-passing mechanism learns structural patterns at multiple length scales simultaneously.

Transfer Learning and Pre-Training on Molecular Data

A significant advance in AI in drug discovery has been the application of transfer learning: pre-training a model on a large dataset of molecular data using a self-supervised objective, then fine-tuning it on a smaller labelled dataset for a specific prediction task. This approach is directly analogous to the pre-training and fine-tuning paradigm that transformed natural language processing, and it addresses one of the central data challenges in drug discovery: labelled biological activity data is expensive to generate and often available only in small quantities for any given target.

Models including MolBERT, Uni-Mol, and GraphMVP are pre-trained on tens of millions of unlabelled molecular structures from databases such as PubChem, ChEMBL, and ZINC, using objectives such as masked atom prediction (analogous to masked language modelling in BERT), 3D geometry prediction, and contrastive learning across multiple molecular representations. The pre-trained model learns a rich, general-purpose representation of chemical space that can be fine-tuned to predict activity against a specific target using as few as a few hundred labelled data points, a regime where classical QSAR models perform poorly.

The theoretical justification for this approach rests on the assumption that the structural patterns relevant to biological activity across different targets share substantial common features: aromatic rings, hydrogen bond donors and acceptors, hydrophobic cores, and stereochemical configurations recur across drug-target interactions in ways that a sufficiently large and diverse pre-training corpus can capture. Empirical evidence strongly supports this assumption, with pre-trained models consistently outperforming models trained from scratch on the same labelled datasets across a wide range of benchmarks.

Conclusion

AI in drug discovery begins at the level of molecular representation and structure-activity relationship modelling. The choice between SMILES strings, molecular fingerprints, molecular graphs, and 3D conformer representations determines what information a model has access to and what architectural choices are appropriate. Graph neural networks with message-passing architectures provide a theoretically principled approach to learning permutation-invariant molecular representations, and transfer learning from large unlabelled molecular databases has addressed the data scarcity problem that limited earlier computational approaches.

These foundations set the stage for the more ambitious applications of AI in drug discovery covered in Part 2: protein structure prediction with AlphaFold, generative molecular design in chemical space, and deep learning-powered virtual screening at scale.

Part 2: Protein Structure Prediction, Generative Design, and Virtual Screening — coming next in the AI Theory series.