# Analyzer B — speed/accuracy benchmarking copy

A working copy of the HVAC blueprint analyzer engine, instrumented for
**per-step timing** and wrapped in a **benchmark harness** that compares many
versions (parallelization and other speed ideas) on a fixed set of confirmed
pages — so we can see which version is *fastest* and which is *most accurate*.

## What's here

| File | Purpose |
|---|---|
| `analyze_blueprint_b.py` | Copy of `python_files/analyze_blueprint.py` + timing + page-level parallelism. |
| `hvac_pipeline/` | Copied dependency package (schedule extraction, reconcile, etc.). |
| `versions.py` | Registry of version configs (the speed ideas). Edit to add up to 10+. |
| `bench.py` | Runs versions over the 4 benchmark pages, scores accuracy + speed, writes the leaderboard. |
| `ground_truth/` | Human-confirmed unit boxes for the 4 pages (copied from the reviewed runs). |
| `results/benchmark_runs.csv` | Append-only log: one row per version+page. |
| `results/LEADERBOARD.md` | Auto-generated comparison, sorted fastest + most accurate. |
| `runs/<version>/<bp>/` | Per-run output (annotated images, review CSVs, `timing_b.json`). |

## Benchmark set (2 blueprints, 4 confirmed pages)

| Blueprint | Page | Floor | Confirmed units |
|---|---|---|---|
| 355 E 86th | 2 | Cellar | 13 |
| 355 E 86th | 13 | 15th | 9 |
| 953 Dean St | 4 | Canopy/Entry | 25 |
| 953 Dean St | 6 | P/Dn SSDS | 66 |

## Engine changes vs the original

1. **Per-step timing** — wall-clock for schedule / classify / detect written to
   `timing_b.json` in the out-dir.
2. **Page-level parallelism** — `HVAC_PAGE_WORKERS` (default 1) runs independent
   plan pages through detection concurrently. Total API concurrency =
   `HVAC_PAGE_WORKERS × HVAC_TILE_WORKERS`; keep the product modest to avoid 429s.

Everything else (detection, schedule logic, review CSV format) is unchanged, so
accuracy is comparable to the original.

## How to run

```powershell
# one version, cheap validation on the two 355 pages only
.venv\Scripts\python.exe analyzer_b\bench.py --versions v02_tiles4 --quick

# a few versions on all 4 pages
.venv\Scripts\python.exe analyzer_b\bench.py --versions v01_baseline,v02_tiles4,v04_pages2

# the whole sweep
.venv\Scripts\python.exe analyzer_b\bench.py --all

# just regenerate the leaderboard from existing results
.venv\Scripts\python.exe analyzer_b\bench.py --leaderboard-only
```

## Scoring

For each page, predicted boxes are matched to confirmed ground-truth boxes by
IoU ≥ 0.3 (boxes at non-300 DPI are rescaled first):

- **F1** — spatial detection F1 (headline accuracy).
- **Count acc** — `1 − |pred − gt| / gt` (how close the unit count is).
- **Tag acc** — fraction of matched boxes whose AC-tag is correct.

Speed is the engine's `detect`-step seconds (schedule + classify are pre-seeded
from cache so every version is compared on the detection work it actually changes).