---
license: mit
language:
- en
library_name: transformers
pipeline_tag: text-classification
base_model: ctheodoris/Geneformer-V2-316M
datasets:
- norman-2019-k562
metrics:
- accuracy
- f1
- pearson_r_delta
- direction_score
tags:
- single-cell
- perturbation-prediction
- crispr
- crispra
- geneformer
- vcbench
- foundation-model
- benchmark
- norman-2019
- bert-classifier
model-index:
- name: vcbench-geneformer-perturbation
  results:
  - task:
      type: text-classification
      name: Perturbation classification (Norman 2019 K562, GEARS test split)
    dataset:
      type: norman-2019-k562
      name: Norman 2019 K562 (107 GEARS test perturbations, seed=1 simulation split)
    metrics:
    - type: pearson_r_delta
      value: 0.627
      name: PRR (Pearson r on perturbation deltas)
    - type: direction_score
      value: 0.8778
      name: DES (top-20 DEG sign agreement)
---

# vcbench-geneformer-perturbation — Geneformer V2-316M fine-tuned on Norman 2019

Geneformer V2-316M fine-tuned for Norman 2019 K562 perturbation
classification. The fine-tuning uses
`BertForSequenceClassification` over 247 perturbation classes; PRR is
recovered downstream by passing the predicted class probabilities through
the canonical perturbation-mean predictor for the corresponding class.

This is a **VCBench Dim A foundation-model checkpoint** — see
[`AppliedScientific/VCBench`](https://github.com/AppliedScientific/VCBench)
for the evaluation code.

## Headline results (Dim A, Norman 2019 K562, GEARS test split)

| Regime | PRR | DES |
|---|---|---|
| FT+D (fine-tuned + decoder) | **0.627** | 0.878 |

Geneformer V2-316M scores **VC Level 1** under VCBench v1.0
(exceeds no-change baseline 0.000 on Dim A; fails to exceed
mean-prediction baseline 0.579).

## Files

```
model.safetensors                        # Fine-tuned classifier weights (1.27 GB)
config.json                              # HF model config (BertForSequenceClassification)
training_args.bin
norman_id_class_dict.pkl                 # Perturbation ID → class index mapping
norman_labeled_train.dataset/            # Tokenized training split (123 MB, Arrow)
norman_labeled_test.dataset/             # Tokenized held-out test split (14 MB, Arrow)
```

## Loading

```python
from transformers import BertForSequenceClassification

model = BertForSequenceClassification.from_pretrained(
    "appliedscientific/vcbench-geneformer-perturbation",
    revision="8be3f3681718b706d351b12b19b0a0b4d76420ca",  # pin to a specific revision
    num_labels=247,
)
```

For the full Dim A evaluation pipeline (predicted probabilities → predicted
expression → PRR), see
[`AppliedScientific/VCBench/src/models/run_geneformer_perturbation.py`](https://github.com/AppliedScientific/VCBench).

## Citation

```bibtex
@misc{vcbench_geneformer_norman,
  author       = {{VCBench contributors}},
  title        = {Geneformer V2-316M fine-tuned on Norman 2019 (VCBench)},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/appliedscientific/vcbench-geneformer-perturbation}},
  note         = {Companion Dim A artefact to VCBench v1.0 (AppliedScientific/VCBench, release tag v1.0.0)}
}
```

## License

MIT (Geneformer V2 base checkpoint also MIT, attribution: Ying Lab).

## Access

Publicly available on HuggingFace.