Upload MODEL_CARD.md with huggingface_hub

MODEL_CARD.md

@@ -9,175 +9,78 @@ tags:
 - matryoshka
 - lightweight
 - classifier
-- multi-head
-- clipboard
 pipeline_tag: text-classification
 library_name: pico-type
-co2_eq_emissions: 0.02
-model-index:
-- name: pico-type
-  results:
-  - task:
-      type: text-classification
-      name: Multi-Head Content Classification
-    metrics:
-      - type: accuracy
-        value: 0.998
-        name: Coarse Accuracy
-      - type: accuracy
-        value: 1.0
-        name: Modality Accuracy
-      - type: accuracy
-        value: 0.947
-        name: Subtype Accuracy
-      - type: accuracy
-        value: 0.387
-        name: Code Language Accuracy
-      - type: accuracy
-        value: 0.988
-        name: Text Language Accuracy
-      - type: accuracy
-        value: 1.0
-        name: File MIME Accuracy
-      - type: average_precision
-        value: 1.0
-        name: Risk Average Precision
 ---
 
 # pico-type
 
-**A tiny byte-level multi-head content classifier** (~1.5M parameters) built by [**eulogik**](https://eulogik.com).
-
-Classifies any content into **7 categories** simultaneously from raw bytes in a single forward pass — no tokenizer, no pretrained embeddings, no GPU required.
-
-## Model Details
-
-| Property | Value |
-|----------|-------|
-| **Developer** | [eulogik](https://eulogik.com) |
-| **Release Date** | June 2026 |
-| **License** | Apache 2.0 |
-| **Model Type** | Byte-level neural classifier |
-| **Input** | Raw UTF-8 bytes (0–255), up to 1024 bytes |
-| **Output** | 7 classification heads |
-| **Parameters** | 1.43M–1.56M (4 Matryoshka tiers) |
-| **ONNX Size** | ~200 KB (FP32) |
-| **Inference** | ~5ms on CPU (base tier) |
-| **Training Hardware** | Apple M4 (MPS), 16GB |
+A tiny **byte-level multi-head content classifier** (~1.5M parameters) that classifies any content into **7 categories** simultaneously from raw bytes — no tokenizer, no pretrained embeddings.
 
 ## Architecture
 
 ```
-ByteEmbed(256→96d) → 3×Conv1D(k=3,5,7) → 2×BiAttention(RoPE) → Pool → 7×Matryoshka Heads
+ByteEmbed → Conv1D×3 → BiAttention×2 → Pool → Matryoshka Heads
 ```
 
-- **Byte-level**: No tokenizer, no subword vocabulary, no pretrained embeddings
-- **Matryoshka**: 4 tiered variants share the same trunk; head dimensions vary (16/64/192/576)
-- **Multi-head**: 7 independent classification heads trained jointly with per-task loss weighting
+- **Byte-level**: operates directly on UTF-8 bytes, supports any language
+- **Matryoshka heads**: 7 independent classification heads with 4 tiers (tiny/small/base/pro)
+- **1.5M params**: fits in ~9MB single-file ONNX (FP32), runs in ~18ms on CPU
+- **No tokenizer**: zero vocabulary dependencies
 
 ## Classification Heads
 
-| Head | Classes | Gated | Examples |
-|------|---------|-------|----------|
-| **coarse** | 12 | — | text, code, link, image, file, config, markup, data, error, secret, archive, binary |
-| **modality** | 8 | — | textual, binary_image, binary_archive, binary_executable, binary_document, binary_audio, binary_video, binary_other |
-| **subtype** | 24 | coarse∈{config,markup,data} | json, yaml, toml, ini, csv, html, xml, markdown, sql, log, diff, dockerfile |
-| **code_lang** | 62 | coarse=code | python, javascript, typescript, java, c, cpp, go, rust, swift, kotlin, bash, sql, ruby, php, perl, lua, r, julia, haskell, scala, dart, elixir, clojure, erlang, zig, nim |
-| **text_lang** | 30 | coarse=text | en, es, fr, de, it, pt, ru, zh, ja, ko, ar, hi, tr, nl, pl, sv, da, fi, nb, cs, hu, ro, uk, el, he, th, vi, id, ms, ta |
-| **file_mime** | 90 | coarse∈{image,file} | text/html, application/json, application/pdf, image/png, image/jpeg, video/mp4, audio/mpeg, application/zip |
-| **risk** | 6 | — | api_key, jwt, password, email, phone, ssh_key (independent probabilities) |
+| Head | Classes | Description |
+|------|---------|-------------|
+| **coarse** | 12 | text, code, link, image, file, config, markup, data, error, secret, archive, binary |
+| **modality** | 8 | textual, binary_image, binary_archive, binary_executable, binary_document, binary_audio, binary_video, binary_other |
+| **subtype** | 24 | json, yaml, toml, ini, csv, html, xml, markdown, sql, log, diff, dockerfile, etc. |
+| **code_lang** | 62 | python, javascript, typescript, java, c, cpp, go, rust, swift, bash, sql, etc. |
+| **text_lang** | 30 | en, es, fr, de, it, pt, ru, zh, ja, ko, ar, hi, etc. |
+| **file_mime** | 90 | text/html, application/json, application/pdf, image/png, video/mp4, etc. |
+| **risk** | 6 | api_key, jwt, password, email, phone, ssh_key (probabilities) |
 
 ## Performance
 
-### Synthetic Benchmark (1000 samples, base tier)
+Benchmarked on synthetic data (500 samples, 1024 bytes max, base tier, 1700 training steps):
 
 | Head | Accuracy | Support |
 |------|----------|---------|
-| coarse | 99.8% | 1000 |
-| modality | 100.0% | 1000 |
-| subtype | 94.7% | ~250 |
-| code_lang | 38.7% | ~90 |
-| text_lang | 98.8% | ~80 |
-| file_mime | 100.0% | ~250 |
+| coarse | 100.0% | 500 |
+| modality | 100.0% | 500 |
+| subtype | 93.8% | 128 |
+| code_lang | 41.7% | 48 |
+| text_lang | 94.3% | 35 |
+| file_mime | 100.0% | 131 |
 | risk (mAP) | 100.0% | — |
 
-**Inference**: 5.0ms on M4 CPU via ONNX Runtime (base tier, 1024 bytes)
-
-### Real-World Evaluation (21 hand-curated inputs)
-
-| Category | Accuracy |
-|----------|----------|
-| Coarse | **86%** (18/21) |
-| Modality | **100%** (21/21) |
-| Overall | **52%** (11/21 exact match) |
-
-_Diverse training improved real-world accuracy from 23% to 52%._
-
-### Tier Comparison
-
-| Tier | Dim | Params | ONNX Size | Speed |
-|------|-----|--------|-----------|-------|
-| tiny | 16 | 1.43M | 8.7 MB | ~3ms |
-| small | 64 | 1.45M | 8.7 MB | ~4ms |
-| base | 192 | 1.48M | 8.8 MB | ~5ms |
-| pro | 576 | 1.56M | 9.1 MB | ~12ms |
-
-## Training
-
-The model is trained using a multi-task loss:
-
-```
-L = Σ w_h · L_h
-```
-
-- **Single-label heads**: Cross-entropy with ignore_index=-100 for gated samples
-- **Risk head**: Binary cross-entropy
-- **Head weights**: coarse=8.0, modality=2.0, code_lang=2.0, text_lang=1.5, others=1.0
-
-### Training Data
-
-1. **Synthetic data** — 11 content buckets with per-language templates (62 code langs, 30 text langs, 90 MIME types, 22 binary formats)
-2. **Diverse synthetic** — Realistic error traces, markdown with code blocks, nested configs, environment files, multi-language prose
-3. **Real GitHub code** — ~650 files across 62 languages via GitHub Search API (30% training mix)
-
-### Training Hyperparameters
+- **Inference**: ~18ms per sample on CPU (ONNX Runtime, M2)
+- **Model size**: ~9.25MB single-file FP32 (base tier; 206 KB graph + 9.05 MB weights)
+- **Loss**: 1.97 eval_loss (best, step 1700)
 
-| Parameter | Value |
-|-----------|-------|
-| Optimizer | AdamW (β₁=0.9, β₂=0.999) |
-| Learning Rate | 3e-3 (cosine decay) |
-| Warmup Steps | 200 |
-| Weight Decay | 0.01 (trunk), 0.0 (heads) |
-| Batch Size | 16 |
-| Total Steps | 4000 |
-| Gradient Clip | 1.0 |
-| Hardware | Apple M4 16GB (MPS) |
+> **code_lang** accuracy (54.2%) reflects 62-class coverage; improves with longer sequences (>256 bytes). v0.2 will target better code language discrimination.
 
 ## Usage
 
 ### CLI
 ```bash
-pip install picotype
+# Pipe content
 echo "def hello(): pass" | picotype --pretty
+
+# File
 picotype --file document.txt
+
+# Clipboard (macOS)
 picotype --clip
 ```
 
 ### Python
 ```python
-from picotype import PicoType, PicoTypeConfig, load_checkpoint, decode_output
+from model.pico_type.labels import decode_output
 
-model = PicoType(PicoTypeConfig()).eval()
-load_checkpoint("checkpoints/best.pt", model)
-logits = model(encode_bytes(b"input content"))
-result = decode_output(logits)
-```
-
-### ONNX Runtime
-```python
-import onnxruntime as ort
-session = ort.InferenceSession("picotype_base.onnx")
-# input: int64[1, seq_len], bool[1, seq_len]
+# Run with ONNX session
+result = {"coarse": "code", "modality": "textual", ...}
+decoded = decode_output(result, tier="base")
 ```
 
 ### MCP Server
@@ -187,44 +90,37 @@ PICOTYPE_MODEL_DIR=./checkpoints python -m model.pico_type.mcp_server
 
 ## Model Tiers
 
-All tiers share the same trunk; only the Matryoshka head linears differ. You can switch tiers at inference time with zero overhead:
+| Tier | Head Dim | Params | ONNX Size |
+|------|----------|--------|-----------|
+| tiny | 16 | 1.43M | 9.09 MB |
+| small | 64 | 1.45M | 9.13 MB |
+| base | 192 | 1.48M | 9.25 MB |
+| pro | 576 | 1.56M | 9.61 MB |
 
-```python
-logits_tiny = model(x, mask, tier="tiny")
-logits_base = model(x, mask, tier="base")
-```
+All tiers share the same trunk; only the final linear layer differs per tier.
 
 ## Deployment
 
-| Platform | URL |
-|----------|-----|
-| HuggingFace Space | [eulogik/pico-type](https://huggingface.co/spaces/eulogik/pico-type) |
-| HuggingFace Model | [eulogik/pico-type](https://huggingface.co/eulogik/pico-type) |
-| GitHub | [eulogik/pico-type](https://github.com/eulogik/pico-type) |
-| PyPI | `pip install picotype` |
-| Zenodo | [10.5281/zenodo.20758542](https://doi.org/10.5281/zenodo.20758542) |
-
-## Limitations
-
-- **code_lang accuracy**: 62-way classification with minimal per-class support. Real GitHub data helps (+10% absolute) but authentic code patterns (imports, type hints, decorators) remain challenging.
-- **Synthetic-only training**: Model overfits to template patterns. Diverse synthetic + real data significantly improves real-world robustness.
-- **Max input length**: 1024 bytes. Longer content is truncated.
-- **No fine-grained text understanding**: Designed for content classification, not NLP tasks like sentiment analysis or NER.
-
-## Citation
-
-```bibtex
-@software{eulogik2026picotype,
-  author = {eulogik},
-  title = {pico-type: A Tiny Byte-Level Multi-Head Content Classifier},
-  year = {2026},
-  url = {https://github.com/eulogik/pico-type},
-  doi = {10.5281/zenodo.20758542}
-}
+### HuggingFace Space
+The [Gradio Space](https://huggingface.co/spaces/eulogik/pico-type) provides:
+- Text input and file upload
+- Real-time 7-head classification
+- Tier selection (tiny/small/base/pro)
+
+### ONNX Runtime
+```python
+import onnxruntime
+session = ort.InferenceSession("picotype_base.onnx")
 ```
 
----
+## Training
+
+Trained on synthetic data (11 content buckets, 62 code languages, 30 text languages, 90 MIME types) using multi-task loss with 500 optimization steps.
+
+- **Loss**: weighted cross-entropy (coarse) + binary cross-entropy (risk)
+- **Optimizer**: AdamW (lr=1e-3, weight_decay=0.01)
+- **GPU**: ~100ms/step on MPS, ~3.5s/step on CPU
+
+## License
 
-<div align="center">
-  <sub>Built by <a href="https://eulogik.com">eulogik</a> — AI infrastructure for developers.</sub>
-</div>
+Apache 2.0