Add model card

README.md

@@ -0,0 +1,176 @@
+---
+library_name: transformers
+license: apache-2.0
+tags:
+  - image-segmentation
+  - semantic-segmentation
+  - segformer
+  - road-scene
+  - cityscapes
+  - game-assets
+  - vision
+pipeline_tag: image-segmentation
+datasets:
+  - Chris1/cityscapes_segmentation
+metrics:
+  - mean_iou
+model-index:
+  - name: segformer-b0-road-scene-7class
+    results:
+      - task:
+          type: image-segmentation
+          name: Semantic Segmentation
+        dataset:
+          type: Chris1/cityscapes_segmentation
+          name: Cityscapes
+          split: validation
+        metrics:
+          - type: mean_iou
+            value: 84.0
+            name: Mean IoU
+---
+
+# SegFormer-B0 — Road Scene Segmentation (7 Game-Asset Classes)
+
+A **SegFormer-B0** model fine-tuned on [Cityscapes](https://huggingface.co/datasets/Chris1/cityscapes_segmentation) for **7-class semantic segmentation** of road scenes. Unlike standard road-scene models that target autonomous driving with 19+ fine-grained classes, this model uses a purpose-built taxonomy where every class maps directly to a game element — road texture, sky backdrop, tree sprites, building silhouettes, etc.
+
+| | |
+|---|---|
+| **Architecture** | SegFormer-B0 (Mix Transformer encoder + all-MLP decoder) |
+| **Parameters** | 3.7 M |
+| **Input** | RGB image, any resolution (resized to 512×512) |
+| **Output** | 7-class pixel mask (upsampled to input resolution at inference) |
+| **Best val mIoU** | **≥ 84.0 %** |
+| **Format** | SafeTensors (14.2 MB) |
+| **Training cost** | ~$0.60 (NVIDIA T4, 50 epochs) |
+
+## 7-Class Game Taxonomy
+
+| ID | Class | IoU | Game Function |
+|:--:|-------|:---:|---------------|
+| 0 | `road` | 97.5 % | Driving surface — grey asphalt texture sampling |
+| 1 | `sidewalk` | 80.3 % | Ground-level non-road surfaces (sidewalk, terrain) |
+| 2 | `building` | 88.1 % | Background vertical structures — building silhouettes |
+| 3 | `vegetation` | 89.1 % | Tall greenery — tree sprite extraction |
+| 4 | `sky` | 92.2 % | Sky band — direct crop for game background |
+| 5 | `vehicle` | 87.8 % | Road obstacles |
+| 6 | `roadside_object` | 52.7 % | Thin vertical roadside elements (poles, signs, people) |
+
+### Key design decisions
+
+- **Road ↔ sidewalk kept separate** so road color sampling produces pure grey asphalt without contamination from sidewalk / terrain tones.
+- **Terrain grouped with sidewalk**, not vegetation — ground-level grass strips serve the same game function as sidewalk.
+- **All building-like structures merged** (building, wall, fence) — the game treats them identically as background geometry.
+- **All vehicle types merged** — the game treats every vehicle as a road obstacle.
+- **Roadside objects** (poles, traffic lights, signs, persons, riders) are grouped into a single thin-element class with fallback stock sprites when extraction quality is low.
+
+## Usage
+
+```python
+from transformers import SegformerImageProcessor, SegformerForSemanticSegmentation
+from PIL import Image
+import torch
+import torch.nn.functional as F
+
+processor = SegformerImageProcessor.from_pretrained("Marco333/segformer-b0-road-scene-7class")
+model = SegformerForSemanticSegmentation.from_pretrained("Marco333/segformer-b0-road-scene-7class")
+
+image = Image.open("road_photo.jpg")
+inputs = processor(images=image, return_tensors="pt")
+
+with torch.no_grad():
+    outputs = model(**inputs)
+
+# Upsample logits to original image size
+mask = F.interpolate(
+    outputs.logits,
+    size=image.size[::-1],  # (H, W)
+    mode="bilinear",
+    align_corners=False,
+).argmax(dim=1)[0]
+
+# mask values: 0=road, 1=sidewalk, 2=building, 3=vegetation, 4=sky, 5=vehicle, 6=roadside_object
+```
+
+## Training Details
+
+### Dataset
+
+[Chris1/cityscapes_segmentation](https://huggingface.co/datasets/Chris1/cityscapes_segmentation) — urban street scenes from 50 European cities.
+
+| Split | Images |
+|-------|-------:|
+| Train | 2,975 |
+| Val | 500 |
+
+The original Cityscapes masks use **label IDs 0–33** stored as 3-channel RGB images. A custom preprocessing pipeline extracts channel 0 and applies a 256-element lookup table to remap all 34 Cityscapes classes into the 7-class game taxonomy in a single vectorized operation (unmapped classes → 255 = ignore).
+
+### Hyperparameters
+
+| Parameter | Value |
+|-----------|-------|
+| Base weights | `nvidia/segformer-b0-finetuned-ade-512-512` (encoder only) |
+| Optimizer | AdamW |
+| Learning rate | 6 × 10⁻⁵ |
+| LR schedule | Polynomial decay |
+| Warmup | 10 % of total steps |
+| Weight decay | 0.01 |
+| Effective batch size | 8 (4 × device · 2 grad accum) |
+| Training resolution | 512 × 512 |
+| Precision | FP16 mixed precision |
+| Epochs | 50 |
+| Augmentation | ColorJitter (brightness=0.25, contrast=0.25, saturation=0.25, hue=0.1) — train only |
+| Best-model selection | Highest mean IoU on validation set (`load_best_model_at_end=True`) |
+| Hardware | NVIDIA T4 (16 GB VRAM) |
+
+### Training Curve
+
+| Epoch | mIoU | Road | Sidewalk | Building | Vegetation | Sky | Vehicle | Roadside Obj |
+|------:|-----:|-----:|---------:|---------:|-----------:|----:|--------:|-------------:|
+| 1 | 59.5 % | 93.2 % | 54.6 % | 74.3 % | 73.7 % | 56.9 % | 63.9 % | 0.0 % |
+| 3 | 73.8 % | 95.4 % | 69.4 % | 82.7 % | 82.5 % | 84.3 % | 79.4 % | 23.1 % |
+| 5 | 80.4 % | 96.7 % | 76.5 % | 86.2 % | 87.0 % | 88.9 % | 84.3 % | 43.2 % |
+| 9 | 82.9 % | 97.4 % | 79.2 % | 87.5 % | 88.3 % | 91.0 % | 86.7 % | 50.4 % |
+| 16 | 84.0 % | 97.5 % | 80.3 % | 88.1 % | 89.1 % | 92.2 % | 87.8 % | 52.7 % |
+
+The model converges quickly thanks to transfer learning — the pretrained encoder already understands road scene features; only the 7-class decoder head is learned from scratch.
+
+### Implementation Notes
+
+Several non-obvious flags are required for correct training:
+
+- `ignore_mismatched_sizes=True` — the pretrained decoder head has a different number of output classes.
+- `remove_unused_columns=False` — prevents the Trainer from dropping image data columns.
+- `label_names=["labels"]` — tells the Trainer which key holds the segmentation targets.
+- `do_reduce_labels=False` — Cityscapes labels don't need ADE20K-style background subtraction.
+- **Logit upsampling in `compute_metrics`** — SegFormer outputs at ¼ resolution; logits must be upsampled before comparison with ground-truth masks.
+
+## Intended Use
+
+This model is designed for a **game asset extraction pipeline** where a user uploads a road photograph and the runtime transforms it into game elements:
+
+1. **Road shape** — fit road boundaries from the road mask; derive perspective and horizon.
+2. **Color palette** — sample dominant colors from each masked region of the original image.
+3. **Sky** — crop the sky band directly using the sky mask.
+4. **Tree sprites** — blob detection on the vegetation mask; crop with alpha transparency.
+5. **Building silhouettes** — extract from the building mask for background geometry.
+6. **Fallback system** — when extraction quality is poor for any element, use palette-matched stock assets.
+
+## Limitations
+
+- Trained exclusively on European urban street scenes (Cityscapes). Performance may degrade on rural roads, highways without sidewalks, non-European road styles, or indoor scenes.
+- `roadside_object` class (52.7 % IoU) is the weakest — thin elements like poles and signs are inherently difficult at 512×512 resolution. The intended runtime uses fallback sprites for this class.
+- Not suitable for safety-critical autonomous driving — the merged taxonomy intentionally discards distinctions (truck vs. car, wall vs. fence) that matter for driving but not for game art.
+
+## Citation
+
+If you use this model, please cite the SegFormer paper:
+
+```bibtex
+@inproceedings{xie2021segformer,
+  title={SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers},
+  author={Xie, Enze and Wang, Wenhai and Yu, Zhiding and Anber, Abualkasim and Lu, Tong and Alvarez, Jose M},
+  booktitle={NeurIPS},
+  year={2021}
+}
+```