About
Meta ESM (Evolutionary Scale Modeling) is a suite of open-source pretrained language models developed by Meta AI (Facebook Research) specifically for protein sequence modeling. Analogous to how large language models process text, ESM models are trained on vast databases of protein sequences to learn evolutionary and structural patterns embedded in amino acid sequences. The repository provides pre-trained model weights and code that researchers can use for a wide variety of protein science applications, including protein structure prediction, function annotation, and generative protein design. Notable capabilities include zero-shot prediction of mutational effects, unsupervised contact map prediction, and high-level programmatic control for generative protein design. Two landmark research directions are included: "Language models generalize beyond natural proteins" (for exploring sequence space beyond natural evolution) and "A high-level programming language for generative protein design" (enabling complex, conditional protein generation). ESM models are built on the transformer architecture and can be fine-tuned or used as feature extractors for downstream tasks in computational biology. Meta ESM is aimed at computational biologists, structural biologists, ML researchers, and bioinformaticians working on drug discovery, enzyme engineering, and protein design. The library is implemented in PyTorch and integrates with standard ML tooling. Note that the original ESM repository was archived in August 2024; the successor project is ESM3 under EvolutionaryScale.
Key Features
- Pretrained Protein Language Models: Transformer-based models trained on millions of protein sequences, capturing evolutionary and structural information in learned representations.
- Zero-Shot Mutational Effect Prediction: Predict the functional impact of amino acid mutations without task-specific training data, useful for protein engineering and variant analysis.
- Generative Protein Design: Includes code for designing novel proteins beyond natural sequence space, with a high-level programming language interface for conditional generation.
- Unsupervised Contact Map Prediction: Extract structural contact information directly from model attention maps without supervision, aiding 3D structure inference.
- Pre-trained Weights & PyTorch Integration: Ready-to-use model weights accessible via PyTorch Hub, enabling easy fine-tuning or feature extraction for downstream computational biology tasks.
Use Cases
- Predicting the functional impact of protein point mutations for drug target validation and variant interpretation.
- Generating embeddings for large-scale protein sequence datasets to power downstream classification, clustering, or retrieval models.
- Designing novel enzymes or therapeutic proteins beyond natural sequence space using ESM-based generative models.
- Unsupervised prediction of residue-residue contacts to support computational protein structure determination pipelines.
- Fine-tuning pretrained ESM representations on domain-specific protein datasets for tasks like binding affinity prediction or subcellular localization.
Pros
- Backed by Meta AI Research: Developed and validated by a world-class research team, with multiple peer-reviewed publications supporting its scientific credibility.
- Versatile for Protein Science: Covers a broad range of applications from structure prediction and variant effect scoring to de novo generative protein design.
- Open Source with Pre-trained Weights: Fully open-source under MIT license with downloadable pre-trained weights, lowering the barrier to entry for protein ML research.
- Seamless PyTorch Integration: Built on PyTorch and loadable via torch.hub, making it straightforward to integrate into existing ML research pipelines.
Cons
- Repository Archived: The original ESM GitHub repository was archived in August 2024 and is no longer actively maintained; users should migrate to ESM3 via EvolutionaryScale.
- Requires Significant Compute: Larger ESM models demand substantial GPU memory and compute resources, which may be a barrier for researchers without access to high-performance hardware.
- Steep Learning Curve for Non-ML Biologists: Effective use requires familiarity with Python, PyTorch, and machine learning concepts, which can be challenging for biologists without a computational background.
Frequently Asked Questions
What is Meta ESM used for?
Meta ESM is used for protein sequence modeling tasks including predicting the effect of mutations, inferring structural contacts, embedding protein sequences for downstream ML tasks, and designing novel proteins via generative models.
Is Meta ESM free to use?
Yes, Meta ESM is fully open-source under the MIT license. Pre-trained model weights and all code are freely available on GitHub.
What happened to the ESM repository?
The original facebookresearch/esm repository was archived on August 1, 2024, and is now read-only. The next generation model, ESM3, has been released by EvolutionaryScale (a spin-off from Meta AI) in a separate repository.
What models are included in Meta ESM?
The repository includes a family of ESM models of varying sizes (ESM-1b, ESM-2, etc.), as well as ESMFold for end-to-end protein structure prediction. Specific model sizes range from 8M to 15B parameters.
How do I load an ESM model in Python?
ESM models can be loaded via PyTorch Hub using `torch.hub.load('facebookresearch/esm:main', 'esm2_t33_650M_UR50D')`, or installed as a Python package via pip and imported directly in your scripts.
