A long-context language model for deciphering and generating bacteriophage genomes

TY - JOUR

T1 - A long-context language model for deciphering and generating bacteriophage genomes

AU - Shao, Bin

AU - Yan, Jiawei

N1 - Publisher Copyright: © The Author(s) 2024.

PY - 2024/12

Y1 - 2024/12

N2 - Inspired by the success of large language models (LLMs), we develop a long-context generative model for genomes. Our multiscale transformer model, megaDNA, is pre-trained on unannotated bacteriophage genomes with nucleotide-level tokenization. We demonstrate the foundational capabilities of our model including the prediction of essential genes, genetic variant effects, regulatory element activity and taxonomy of unannotated sequences. Furthermore, it generates de novo sequences up to 96 K base pairs, which contain potential regulatory elements and annotated proteins with phage-related functions.

AB - Inspired by the success of large language models (LLMs), we develop a long-context generative model for genomes. Our multiscale transformer model, megaDNA, is pre-trained on unannotated bacteriophage genomes with nucleotide-level tokenization. We demonstrate the foundational capabilities of our model including the prediction of essential genes, genetic variant effects, regulatory element activity and taxonomy of unannotated sequences. Furthermore, it generates de novo sequences up to 96 K base pairs, which contain potential regulatory elements and annotated proteins with phage-related functions.

UR - https://www.scopus.com/pages/publications/85208162723

U2 - 10.1038/s41467-024-53759-4

DO - 10.1038/s41467-024-53759-4

M3 - Article

C2 - 39477977

AN - SCOPUS:85208162723

SN - 2041-1723

VL - 15

JO - Nature Communications

JF - Nature Communications

IS - 1

M1 - 9392

ER -