How to Evaluate AI-Designed Proteins: Structure, Stability, and Experimental Validation

A generative model can propose thousands of protein sequences in an afternoon. Deciding which of them deserve to be synthesised is the harder problem. Evaluation works best as three layers, each answering a different question: does the design look like it will fold the way intended, is it actually stable, and does it do what it was designed to do. Each layer filters out failures before the next, more expensive one.

Layer 1: Does the predicted structure match the design?

The standard computational check is self-consistency. An independent structure predictor is run on the designed sequence, and its output is compared with the intended backbone. In the RFdiffusion paper, a design counted as an in silico success when AlphaFold2, predicting from a single sequence, returned a mean predicted aligned error (pAE) below 5, a global backbone RMSD within 2 Å of the design, and an RMSD within 1 Å on any scaffolded functional site (Watson et al., 2023).

For binders, confidence at the interface matters more than confidence in the chain as a whole. Bennett and colleagues used the AF2 interaction pAE, the average predicted error over residue pairs across the two chains. A cut-off of 10 was extremely effective at identifying experimentally confirmed binders in retrospective data, and filtering on AF2 or RoseTTAFold confidence increased design success rates nearly 10-fold. The authors also note that success rates remained low (below 1%) for the targets they tested (Bennett et al., 2023).

Newer interface metrics are being compared at scale. A 2025 bioRxiv meta-analysis, a preprint, gathered 3,766 binder designs across 15 targets; 436 of them (11.6%) bound in vitro. AlphaFold3-derived ipSAE_min was the best single metric, with a 1.4-fold gain in average precision over AF2 ipAE (bioRxiv, 2025). The authors flag important caveats: assay formats and binding definitions differed between studies, some of the data had been pre-selected using ipAE (which may inflate its apparent performance), and the best threshold varied from target to target.

In our view, these scores are best used to rank candidates, not to declare them correct. A high predicted confidence is a statement about the model's own output, not independent evidence that the protein behaves as predicted.

Layer 2: Is the protein stable?

A design can be confidently predicted and still unfold on the bench. Stability needs its own measurement, and high-throughput assays make it one of the cheaper experimental filters.

The most influential high-throughput version is yeast display with protease challenge. Rocklin and colleagues defined a stability score as the difference between a sequence's measured protease EC50 and the EC50 predicted for its unfolded state, so that a score of 1 means a 10-fold higher EC50 than unfolded. They tested over 15,000 designed miniproteins, and iterating the design rules raised the design success rate from 6% to 47% (Rocklin et al., 2017). The informative part was what predicted failure: buried nonpolar surface area dominated, none of the 95 designs below 32 Ų per residue were stable, and total Rosetta energy was only weakly correlated with success.

The limits are stated plainly in that paper. The designs were 40 to 43 residues long, the stability score is not a direct analogue of a thermodynamic parameter, and a stable design is not guaranteed to fold as intended.

For quantitative free energies, cDNA-display proteolysis extends the idea. In its preprint version, the method measured 851,552 single and double mutants from 354 natural domains and 188 designs, with Pearson correlations above 0.7 against earlier purified-protein measurements on 1,143 variants of ten proteins. Its dynamic range is about 5 kcal/mol, and the authors caution that aggregation or non-cooperative unfolding can produce spurious values (Tsuboyama et al., preprint).

For larger designs, lower-throughput checks do the same job. In the RFdiffusion study, circular dichroism spectra of six 300-residue and three 200-residue unconditional monomers matched the intended mixed α/β designs, and the proteins were extremely thermostable (Watson et al., 2023).

Layer 3: Does it do the job, and does the structure match?

Experimental validation is a ladder, and each rung tests something different.

The first rung is whether the protein expresses and adopts the intended assembly. For symmetric oligomers in the RFdiffusion study, 608 designs were selected and size-exclusion chromatography showed that at least 87 matched the intended oligomeric state. Among 48 icosahedral designs, only one was observed by negative-stain EM to form homogeneous particles (Watson et al., 2023). Pass rates for complex architectures can be low even for a leading method.

The second rung is function, most often binding. Across five targets with 95 designs each, 19% of RFdiffusion binders passed a threshold of 50% of the positive-control signal at 10 µM by biolayer interferometry, which the authors describe as conservative. For p53/MDM2, 55 of 96 designs showed detectable binding at 10 µM, and the best reached 0.5 nM against 600 nM for the native p53 peptide (Watson et al., 2023). Using two assays is good practice: in Bennett et al., hits were called by yeast display (SC50 better than 4 µM), and every yeast-display binder subsequently tested by biolayer interferometry also bound. Initial binders were in the high-nanomolar range (Bennett et al., 2023).

The final rung is structural. A 2.9 Å cryo-EM reconstruction of the HA/HA_20 complex matched the design model with an RMSD of 0.63 Å (Watson et al., 2023). An experimental structure is the most direct check that a binder engages its target the way it was designed to. It also calibrates the predictors: AF2 reproduced five experimentally determined minibinder interfaces with binder Cα RMSDs of 1.0 to 2.0 Å (Bennett et al., 2023).

Putting the layers together

In practice, the order follows cost. Filter computationally on self-consistency and interface confidence, then test stability at scale, then spend on binding assays and structures for the few designs that survive. Report the denominator at every stage: how many designs were generated, how many passed each filter, and how many were tested. A 19% hit rate and a sub-1% hit rate can both be accurate descriptions of the same field, depending on the target and the threshold.

Two cautions recur across these papers. Benchmarks can flatter the metrics they test: the meta-analysis above notes that some of its data were pre-selected with the very metric being evaluated. And stability or binding in a single assay is not proof of the designed mechanism, which is why orthogonal assays and, where possible, an experimental structure carry so much weight.

The field is moving quickly, and the metrics above will keep changing. The structure of the argument should not: predict, measure stability, then show function and structure independently.