GenNA
Deciphering how nucleotide sequences encode biological function remains a central challenge in genomics and synthetic biology. Recent foundation models have shown that DNA and RNA can be treated as learnable languages, but most generative nucleotide models are conditioned mainly on local sequence context, prefixes, or a limited set of structured labels. Such inputs are useful for continuation and reconstruction, yet they provide only a restricted interface for expressing complex biological intentions. Natural language offers a more flexible way to specify gene-associated function, species context, and design goals. We therefore developed GenNA, a generative nucleotide foundation model that directly integrates natural-language annotations with nucleotide sequence generation.
GenNA is a decoder-only Transformer with approximately 3.6 billion parameters. It was pretrained from scratch on a multimodal corpus derived from NCBI RefSeq, covering 2,221 eukaryotic species and approximately 416 billion characters. Each training example combines nucleotide sequence, natural-language functional descriptions, species information, gene identifiers, and XML-style structural tags. A unified cross-modal byte-pair encoding tokenizer maps all of these elements into one shared vocabulary, allowing the model to learn their relationships through a single causal language-modeling objective. In this framework, text is not an external label added after generation; it is part of the same autoregressive context used to predict every subsequent token.
We first tested whether GenNA had genuinely learned relationships between biological annotations and nucleotide sequences. When the nucleotide sequence was held constant but its functional annotation was replaced with an incompatible annotation, the median perplexity increased from 6.68 for native sequence-annotation pairs to 331.24 for mismatched pairs. Attention analysis further showed that functional prompts continued to receive substantial attention during coding-sequence generation, even when the generated position was far from the prompt. These results indicate that the model uses natural-language information as an active conditioning signal rather than as superficial metadata.
GenNA also showed zero-shot sensitivity to biological constraints. In silico mutational scanning revealed that synonymous substitutions and changes in noncoding regions generally produced modest changes in perplexity, whereas mutations that altered or disrupted the encoded protein caused progressively stronger rejection signals. Missense mutations produced a median change in perplexity approximately four times larger than synonymous mutations, and frameshift deletions produced the strongest effect. In a complementary species-label perturbation analysis, labels from closely related species caused smaller perplexity increases than labels from distant species. Intermediate-layer representations also formed partially separable clusters corresponding to broad biological groups. Together, these observations suggest that GenNA captures statistical signatures associated with coding integrity, species context, and evolutionary relatedness without task-specific fine-tuning.
To strengthen benchmarking, we designed a sequence-recovery experiment using 2,000 human mRNA sequences and 1,000 human genomic DNA sequences from non-RefSeq GenBank records. Models received nucleotide prefixes of 10, 50, 100, or 200 nucleotides and generated the following 200 nucleotides. GenNA was evaluated under three conditions: full annotation, gene identity only, and no annotation. We compared it with Evo2-7B, GENERator-v2, and HyenaDNA. GenNA showed its clearest advantage when nucleotide context was limited. With a 10-nucleotide prefix, full-annotation GenNA achieved mean alignment identities of 0.465 for RNA and 0.455 for genomic DNA, exceeding Evo2-7B and GENERator-v2. GenNA also achieved the highest mean identity at the 50-nucleotide prefix for both modalities and remained comparable to Evo2-7B for RNA at 100 nucleotides. At longer prefixes, Evo2-7B and GENERator-v2 benefited more strongly from additional sequence context. Annotation ablation consistently reduced GenNA performance, demonstrating that natural-language conditioning contributes measurable information beyond the nucleotide prefix itself. These findings position language guidance as complementary to model scale, long-context architectures, and nucleotide-level supervision rather than as a replacement for them.
We next evaluated GenNA in open-ended and targeted generation tasks. In species-conditioned generation across human, zebrafish, fruit fly, Arabidopsis, yeast, and maize, generated coding sequences reproduced broad species-associated patterns in GC content, GC3 content, codon usage, and amino-acid composition. For a concrete targeted-design example, we prompted GenNA to generate transfer RNAs across 61 distinct anticodon categories. Among 30,495 valid outputs, the sequences collapsed to 5,318 globally distinct candidates. Only 61 of these distinct sequences, or 1.15%, exactly matched a sequence in the pretraining corpus. The non-exact-match candidates nevertheless retained strong tRNAscan-SE scores and frequently exhibited the requested anticodon or a biologically compatible wobble pairing. Thus, the tRNA results cannot be explained solely by retrieval of memorized training sequences and instead illustrate the generation of diverse candidates that preserve prompt-associated structural and recognition features.
GenNA also generated major ribosomal RNA classes and protein-coding sequences for the five major histone families. To examine protein-level plausibility more directly, we generated nucleotide sequences conditioned on descriptions of RNA-recognition motifs, EF-hand domains, SH3 domains, PDZ domains, protein kinase domains, and ankyrin repeats. The resulting coding sequences were translated and analyzed with InterProScan, and representative positive cases were examined using AlphaFold 3. Detected domains were consistent with the conditioning descriptions and mapped to spatially localized regions with plausible local folds in the predicted structures. Independently generated samples varied in protein length, domain position, flanking regions, and repeat number, indicating that the model did not collapse to a single fixed architecture.
This protein-domain analysis also clarifies the relationship between nucleotide and protein generation. Direct protein language models operate in amino-acid space and are naturally suited to optimizing protein sequence, structure, and activity. GenNA instead generates the underlying nucleotide sequence and therefore retains information that disappears after translation, including synonymous codon choices, species-specific codon usage, untranslated regions, transcript organization, and other nucleotide-level constraints. The two approaches are therefore complementary: GenNA can propose annotation-guided nucleotide candidates, which can then be translated and ranked or refined using specialized protein sequence and structure models.
Across more than 3,000 real benchmark sequences and over 70,000 generated sequences, GenNA demonstrates a unified capacity for sequence understanding, conditional generation, and biologically informed evaluation. Its strongest comparative advantage appears when local nucleotide context is short or absent, where natural-language annotations provide information unavailable from the sequence prefix alone. The sequence-recovery benchmark, non-memorized tRNA candidates, and protein-domain case studies provide concrete evidence of this capability. Current evaluations remain computational, and systematic wet-lab validation will be required to establish biochemical activity and in vivo function. Nevertheless, GenNA offers a practical interface for translating high-level biological descriptions into nucleotide-level design candidates and provides a foundation for future workflows that jointly optimize nucleotide and protein representations.