Genome Language Models: What an Engineer Should Actually Take From Evo 2

I am not a biologist. I write industrial and manufacturing software and I have spent the last few years in applied AI, so I read genome language models the way I read any other system: what goes in, what comes out, where it breaks under load. Saying that up front matters, because the field borrowed our vocabulary and then quietly changed what the words mean. Tokens. Context window. Pretraining. Zero-shot. All present. All subtly different.

The pitch is clean enough that you can see why it spread. DNA is a sequence over a four-letter alphabet. Sequence models are good at sequences. Point the second thing at the first thing. That is genuinely the shape of the idea, it has produced real results and the interesting part is the list of places where the analogy stops paying.

Four symbols, no word boundaries and a runtime you cannot see

Start with the alphabet. Four symbols means a token carries at most two bits. A byte-pair vocabulary of 100,000 entries tops out near 16.6 bits. So a one-million-token DNA context and a one-million-token text context are not comparable quantities of information, and whatever intuition you hold about what a model can keep in a million tokens needs dividing by something like eight before it means anything here.

Then the structure. No word boundaries. No punctuation. No sentences. A protein-coding stretch is read in triplets, the same stretch read one base to the left is a different protein or nonsense and the reverse complement is a third reading. Six frames over the same characters.

The break that actually matters is the runtime. A genome is not a program. It is closer to a program plus a runtime whose configuration lives outside the file. Every cell in your body carries the same sequence and a neuron is not a hepatocyte. The difference is methylation, chromatin state and transcription factor concentration, none of which is in the string. A sequence model sees the source and never the environment.

The corpus is not what a text corpus is either. Text is written by many independent authors. Every genome in a genomic corpus is a sample from one phylogeny, so near-duplicates are everywhere by descent. Split at random and you have leaked. In the industrial data I work with the leak is usually one machine appearing in train and test under two asset IDs. In genomics the leak is evolution, and it is the most sophisticated leak I have met, because homology is exactly the signal the model is meant to learn and exactly the thing that inflates your test score.

Data filtration is a build-time control applied to an artifact that ships with the ability to be rebuilt.

Evo 2 in numbers you can check

Evo 2 comes from the Arc Institute with NVIDIA, released as a preprint in February 2025 and published in Nature in March 2026. Two headline sizes, 7B and 40B parameters. The 40B was trained on 9.3 trillion nucleotide tokens, the 7B on 2.4 trillion. The corpus, OpenGenome2, holds over 8.8 trillion nucleotides drawn from more than 128,000 genomes across bacteria, archaea, eukarya and phage, spanning over 100,000 species.

Run the ratio. 9.3 trillion tokens over 40 billion parameters is about 233 tokens per parameter. Heavily overtrained by the usual scaling conventions, which is what you do when data is cheap relative to inference and you want the smaller checkpoints to be worth using.

The architecture is not a Transformer. StripedHyena 2 is a convolutional multi-hybrid: short, medium and long input-dependent convolution operators interleaved with a minority of attention layers. The paper reports roughly a threefold speedup over Transformers at one million context. That choice is the reason the context target was reachable at all.

Compute was over 2,000 NVIDIA H100 GPUs on DGX Cloud via AWS, running for months. Weights and code are Apache 2.0 and OpenGenome2 is published. That last fact carries more weight in the rest of this piece than any benchmark does.

One practical note. The 40B and 20B checkpoints need Transformer Engine and FP8 on Hopper-class hardware, while the 7B runs in bfloat16 on ordinary GPUs. This is not an Ollama pull. If you are renting on something like Modal, budget for a multi-GPU node rather than a card.

One megabase is both enormous and a keyhole

One million base pairs at single-nucleotide resolution is the number that gets quoted. Put it next to the human genome, which the Telomere-to-Telomere consortium completed at 3.055 billion base pairs. The largest context window in this class covers about three hundredths of one percent of one human genome. Enormous by sequence-model standards. A keyhole by genome standards.

The reason to chase it is concrete. The regulatory elements that decide whether a gene is expressed are frequently nowhere near the gene. At 8,192 bases you are reading the coding sequence and almost nothing that controls it. The convergence is telling: DeepMind's AlphaGenome, a completely different kind of model, also takes a one-megabase input. Two groups with different objectives landed on the same input scale, which suggests the scale is set by the biology rather than by the architecture.

The engineering underneath is familiar. They did not train at one million from scratch. Pretraining ran at 8,192 tokens, a long-context midtraining phase extended it and the 8,192 base checkpoints still ship separately. Anyone who has watched a language model's advertised context outrun what it actually uses knows the question to ask, and it is the same question here. Accepting a million tokens and reasoning over a million tokens are two different claims.

Variant ranking is the product, genome design is the demo

The dominant real use is variant effect prediction and the mechanism is almost embarrassingly simple. Score the reference window under the model. Change one base. Score it again. Subtract. The delta in log-likelihood is your variant score. No labels, no task head, nothing fine-tuned. That is what zero-shot means here.

It works, with a shape worth reading carefully. On BRCA1, the strong figures in the paper, AUROC 0.94 on coding single-nucleotide variants and 0.95 across all SNVs, come from a supervised classifier built on top of Evo 2 likelihoods rather than from the raw subtraction. On ClinVar coding SNVs, Evo 2 sat fourth or fifth, behind AlphaMissense and GPN-MSA. Where it led was noncoding variants, and on SpliceVarDB it had the highest zero-shot performance for exonic and intronic variants.

That is the honest summary and it is more interesting than the headline. A genome language model wins where labelled data is thin and specialised supervised models do not reach, which is the noncoding genome and splicing. It loses where a specialist already has good labels. Anyone who has built ML systems in a domain with expensive labels has seen this exact shape.

Sequence design is the other use and it is further along than I expected. Evo 2 generated mitochondrial genomes with plausible gene content, and for Mycoplasma genitalium nearly 70% of predicted genes in generated sequences had significant Pfam hits against 18% for the previous model. Then the wet-lab result. A Stanford and Arc team published the generative design of bacteriophages in Science in August 2026, producing roughly 302 candidate phiX174 genomes of which 16 were viable, replicating and lysing E. coli, with a cocktail of the designs overcoming phiX174-resistant strains.

Sixteen out of about three hundred, on a template that is one of the smallest genomes anyone works with. A genuine result. Also not what "AI designs genomes" implies to a general reader.

Sequence modelling is not understanding

Here is the objection I find hardest to argue with, and I do not think it can be dismissed.

The likelihood a genome language model assigns to a sequence measures one thing: how much this looks like something evolution kept. Purifying selection removes variants that break function, so conservation correlates with function, so a model that has read a hundred thousand species is a very good learned substitution matrix. It is not a model of mechanism. It cannot tell you which protein, which pathway, which cell type or which developmental window.

This is not a hypothetical complaint. A 2025 Nature Methods benchmarking study found genomic language models consistently underperforming well-established supervised models on biologically meaningful tasks, and related work has reported gLMs failing to beat one-hot encodings and randomly initialised baselines in low-homology settings. AlphaGenome, which is supervised on thousands of measured functional genomic tracks rather than trained on likelihood alone, matched or exceeded the strongest available external models on 24 of 26 variant effect evaluations. The models winning at regulatory variant effect prediction right now are the ones that were shown experimental measurements.

There is also a limit that follows directly from what the score is. A conservation-derived score inherits evolution's objective, and evolution does not optimise for anything that happens after reproduction. Harm that arrives late sits under weaker selection and a likelihood model has no way to represent that asymmetry.

Compare the adjacent case. Protein structure prediction, where open models like Boltz operate, has a well-defined target: sequence in, coordinates out, measurable against a solved structure. Genome-scale function has no equivalent ground truth to regress against, which is exactly why likelihood became the training signal.

The rebuttal, which is also fair, is that nobody deploying this asks the model to understand. Clinical genetics is a triage problem. Three thousand variants of uncertain significance and budget to assay fifty. A well-ranked list with no mechanism attached is worth a great deal. The complaint is mostly about how the output gets described on the way to the press release, not about whether the tool earns its place. Both are true at once and I have not seen anyone resolve it.

Why the release conversation is not a chatbot release conversation

A chatbot release argues about information. The model tells a person how to do something and the harm is bounded by that person's ability to actually do it. Genome models change the geometry, because the output is not advice. It is an order. A generated sequence is a file you send to a synthesis vendor, and the distance between model output and physical material is a purchase order and a screening step.

That screening step is the control surface and in 2025 it was shown to be softer than assumed. A team led by Eric Horvitz at Microsoft published work in Science on 2 October 2025 in which three open-source protein design tools generated 76,080 synthetic sequences encoding variants of 72 proteins of concern. Existing biosecurity screening software missed hundreds of them. The group worked confidentially with vendors for ten months and the patches that followed cut the misses to a few percent per tool. That was protein design rather than genome modelling. The control surface is the same one.

Arc's mitigation for Evo 2 was to exclude viruses that infect eukaryotic hosts from the training data, then verify that the model shows high perplexity on those sequences. As a design decision it is defensible and it was published openly, which is more than most releases manage.

On open weights it is also a default rather than a boundary. An October 2025 evaluation of biorisk on open-weight bio-foundation models fine-tuned Evo 2 7B on related viral species and found perplexity on held-out excluded species dropping to unfiltered levels within 50 fine-tuning steps, which the authors measured at 0.72 H100 GPU hours. Two thousand steps, 28.9 GPU hours, lifted mean correlation on mutational effect prediction from 0.034 to 0.164. Linear probes on the pretrained hidden states were already predictive for virulence with no fine-tuning at all.

Read that as an engineer. Data filtration is a build-time control applied to an artifact that ships with the ability to be rebuilt. Under an hour of one GPU undid the filter for the species tested, and the probe result suggests the information was never fully absent from the representation, only from the decoder's habits.

No rollback primitive

Software has a rollback primitive. The deploy is bad, you revert, you restore the snapshot, you write the postmortem. Every professional risk calculation I have made has that escape hatch quietly priced in, and it is the reason we tolerate shipping things we are not certain about.

Two things in this stack have no such primitive. Released weights cannot be unreleased. Once an Apache 2.0 checkpoint is mirrored, no decision reverses it. And a self-replicating construct has no snapshot to restore to. The phage work is a therapeutic effort against antibiotic resistance, run by careful people on a well-characterised host, and it remains the case that the artifact reproduces.

I am not arguing for closure. The openness is why the benchmarks in this piece exist, why an independent group could run the fine-tuning attack and publish the exact GPU-hour cost, and why every number here could be checked against a paper instead of a press release. Closed weights would have given us a safety claim with no way to test it.

The tension I cannot get past is narrower. The safety case is stated as a property of a checkpoint, its training data. The threat model is a property of the ecosystem around the checkpoint, its fine-tuning. Those are different objects and only one of them was ever under anyone's control.

What unsettles me is not the possibility that these models understand biology. It is how little understanding the useful parts seem to require.

Tools referenced

Boltz, reviewed here: Boltz review.

Modal, reviewed here: Modal review.

Ollama, reviewed here: Ollama review.

Sources

Genome modeling and design across all domains of life with Evo 2 (Arc Institute preprint, full text): https://www.biorxiv.org/content/10.1101/2025.02.18.638918v1.full

Arc Institute: AI can now model and design the genetic code for all domains of life with Evo 2: https://arcinstitute.org/news/evo2

ArcInstitute/evo2 on GitHub (Apache 2.0 code and checkpoints): https://github.com/ArcInstitute/evo2

Benchmarking genomic language models (Nature Methods, 2025): https://www.nature.com/articles/s41592-025-02829-6

AlphaGenome: advancing regulatory variant effect prediction with a unified DNA sequence model: https://www.biorxiv.org/content/10.1101/2025.06.25.661532v1

Generative design of bacteriophages with genome language models (Science, 2026): https://www.science.org/doi/10.1126/science.aec2657

Strengthening nucleic acid biosecurity screening against generative protein design tools (Microsoft Research, Science 2025): https://www.microsoft.com/en-us/research/publication/strengthening-nucleic-acid-biosecurity-screening-against-generative-protein-design-tools/

Best Practices for Biorisk Evaluations on Open-Weight Bio-Foundation Models (arXiv 2510.27629): https://arxiv.org/abs/2510.27629