What Can Machine Learning Tell Us About Noncanonical Peptides?

Suppose we already have a peptide that binds its target. We want to change one residue to improve another property, while keeping that binding intact. Which chemical change should we try first?

This is the design problem I have in mind when reading about machine learning for noncanonical amino acids (NCAAs). Once we move beyond the usual amino acid alphabet, we need a model to describe the actual chemistry of a residue. But describing a molecule is only the beginning. The useful prediction is what happens when we change it.

The progress in protein structure prediction makes this an exciting question. AlphaFold 2 changed what we can infer from a sequence,¹ and AlphaFold 3 extended prediction to a much wider range of molecular interactions.² Crucially, newer models already support modified chemistry: AlphaFold 3 handles modified residues, and Boltz and OpenFold3 also provide ways to specify noncanonical residues.³ Their support for such inputs gives us a starting point. It does not, by itself, establish how reliably they can predict the effect of a new substitution on binding or function.

For me, that distinction connects several otherwise quite different papers. A useful representation, a plausible generated peptide, and a reliable prediction of experimental activity are separate achievements. The limitations of each approach matter because they determine which design decisions its results can support.

Learning the chemistry

A peptide representation needs to distinguish chemical changes that a conventional sequence alphabet cannot express. PepLand tackles this through molecular graphs, learning from canonical peptides before further pretraining on noncanonical chemistry. Its benchmarks cover properties such as permeability and binding, bringing modified peptides into the same evaluation framework as canonical ones.⁴

The reported Spearman correlation of 0.768 on its nc-Binding benchmark is encouraging. The supplement describes linear probing: a supervised predictor is trained on features extracted from the pretrained model. That is evidence that the learned representation contains useful information, and the pretraining ablations help assess how its training stages contribute.⁴

My main reservation about PepLand for substitution design is its lack of explicit conditioning on a particular bound conformation. A molecular graph specifies atoms and bonds, and a model can learn correlations with conformation-dependent properties. But the benchmark score alone does not settle whether it can distinguish subtle chemical changes at a specific interface. I would want evaluations that hold out peptide scaffolds or residue families, because the split determines how strongly we can claim generalization to unfamiliar chemistry.

What does a better generated peptide mean?

That question becomes more consequential when the model starts proposing molecules. We can check whether a generated peptide is chemically valid, novel, or diverse. These are useful checks: an invalid molecule cannot become an experimental candidate. The harder step is showing that a better computational score leads to a better peptide.

PepTune makes this step especially visible. It generates peptide SMILES through masked diffusion, then uses Monte Carlo Tree Guidance to search for candidates with several desirable predicted properties. The paper evaluates noncanonical residue usage and cyclic peptide generation as well as validity and diversity, so its contribution goes beyond generating ordinary peptide sequences.⁵

The limitation I see is the dependence of that search on learned property predictors: errors in a predictor can affect which molecules the search recommends. How much confidence to place in those candidates depends on the predictors’ training coverage and evaluation splits. I would especially want to see whether they remain reliable on the unfamiliar chemistry that generation is meant to explore.

Its GLP-1R example also illustrates the limits of computational evaluation. The generated candidates receive more favorable docking scores than the semaglutide and liraglutide controls used in the study. Those scores help formulate a hypothesis about binding. They do not establish stronger binding experimentally, and they cannot establish receptor agonism. The paper reports no wet-lab assays of these candidates.⁵

HELM-GPT illustrates the same dependence on predicted properties, using HELM to specify monomers and their connections. It uses learned predictors to guide macrocyclic peptides toward improved permeability and KRAS binding scores. The study reports no experimental validation of those generated candidates, so the proposed improvements remain computational predictions.⁶ My confidence in them therefore depends on how well the property predictors generalize to the chemistry that optimization explores.

GPepT provides a small experimental bridge. After generating peptidomimetics from a model fine-tuned for antimicrobial activity, the authors selected five candidates using availability and synthesis criteria. Three were synthesized and tested; one showed activity against E. coli.⁷ The small, selected set limits what we can conclude: it does not establish a prospective synthesis success rate, an experimental activity distribution, or a hit rate across model outputs.

These results make generation interesting to me as a way to choose experiments. A prospective test would follow a stated candidate-selection rule and report the outcomes across the tested set, including unsuccessful candidates. Stating the rule in advance would help us judge how much the model improves candidate selection, alongside the chemist’s judgment.

Bringing the structure back in

For a substitution in an existing binder, there is another source of information: the complex itself. A bound structure gives us a way to examine whether a new side chain might fit, make a contact, or disrupt an interaction. The challenge is representing unfamiliar residues and scoring their effects accurately.

Holden and colleagues used AutoRotLib to parameterize NCAAs for Rosetta calculations on the PUMA–MCL-1 and CP2–KDM4 systems. They compared computed substitution effects with experimental data, including classifications of stabilizing and destabilizing changes. Agreement varied across positions, and canonical substitutions generally performed better than noncanonical ones.⁸ This is a valuable test because the prediction is tied directly to the substitution decision. Its limitation is the accuracy of the resulting Rosetta calculation: providing residue parameters and rotamer libraries does not guarantee that the energy model and sampled conformations capture the experimental effect.

FakeRotLib makes NCAA parameterization faster, with publicly available code. Its published benchmarks chiefly assess rotamer and sequence recovery using canonical residues under alternative parameterization schemes.⁹ That evaluation is its limitation for the question here: it demonstrates a way to prepare calculations more efficiently, but does not establish improved prediction of experimental NCAA substitution effects.

The evidence I would like to see

The most useful evaluation depends on the decision we want a model to support. If I am choosing substitutions for an existing binder, I care about whether it ranks those changes correctly. If I am generating a new scaffold, I care about whether the selected candidates can be synthesized and show the intended activity.

For substitution prediction, the nonproteinogenic deep mutational scanning work of Rogers and colleagues is particularly relevant: it measures the effects of many chemical changes within the same peptide systems.¹⁰ I would like to see more models evaluated on such experimentally grounded comparisons, with training overlap checked and performance reported for unfamiliar scaffolds and residue types.

I am optimistic about tools that help us explore more of peptide chemistry. My strongest interest is in connecting that exploration to a concrete experiment: a substitution worth making, a candidate worth synthesizing, or a prediction we can test. That connection is where I would look for the next convincing advance.

References