# HVAC Blueprint Analyzer - Production & Demo Checklist

**Project**: HVAC Blueprint Analyzer  
**Status**: Pre-Production  
**Last Updated**: June 19, 2026

---

## 🎯 PHASE 1: CORE FEATURE COMPLETION

### Data Processing Pipeline
- [ ] **PDF Extraction & Parsing**
  - [ ] Extract all page content from mechanical plans
  - [ ] OCR fallback for low-quality PDFs
  - [ ] Handle multi-page PDFs with proper page segmentation
  - [ ] Validate extracted text quality metrics

- [ ] **Schedule Detection & Parsing**
  - [ ] Detect "Indoor Unit Schedule" pages automatically
  - [ ] Extract HVAC unit details (tag, type, quantity, capacity)
  - [ ] Parse equipment specifications from tables
  - [ ] Handle variations in schedule formatting
  - [ ] Export schedule data to structured format (CSV/JSON)

- [ ] **Floor Plan Analysis**
  - [ ] Detect floor plan pages (multiple floor apt check)
  - [ ] Identify apartment/unit boundaries
  - [ ] Map HVAC units to specific apartments
  - [ ] Validate unit placement logic
  - [ ] Support basement/mechanical room layouts

- [ ] **YOLO Model Integration**
  - [ ] Verify model accuracy on test blueprints
  - [ ] Handle edge cases (small units, overlapping units)
  - [ ] Optimize inference speed
  - [ ] Export detection results with confidence scores

---

## 🤖 PHASE 2: AI MODEL TESTING & COMPARISON

### Gemini vs OpenAI Testing
- [ ] **Setup Dual API Support**
  - [ ] Create abstraction layer for LLM providers
  - [ ] Implement Gemini API integration
  - [ ] Implement OpenAI API integration
  - [ ] Support configurable provider switching

- [ ] **A/B Testing Framework**
  - [ ] Create test harness for parallel runs
  - [ ] Log response time, accuracy, cost for each provider
  - [ ] Compare output quality metrics
  - [ ] Generate comparison reports

- [ ] **Speed Testing (Parallel Runs)**
  - [ ] Benchmark Gemini extraction vs OpenAI extraction
  - [ ] Test with 10+ sample PDFs
  - [ ] Measure: latency, throughput, token usage
  - [ ] Document performance findings

- [ ] **Accuracy Testing**
  - [ ] Validate extracted data against known-good answers
  - [ ] Test unit count accuracy
  - [ ] Test schedule parsing consistency
  - [ ] Track hallucination rate for each model

- [ ] **Cost Analysis**
  - [ ] Calculate cost per blueprint for Gemini
  - [ ] Calculate cost per blueprint for OpenAI
  - [ ] Include API call overhead
  - [ ] Recommend optimal provider based on volume

---

## 🔄 PHASE 3: SCHEDULE HANDLING MODES

### Schedule Display vs Auto-Discovery
- [ ] **Manual Schedule Input Mode**
  - [ ] UI for uploading/pasting schedule data
  - [ ] Schedule data validation
  - [ ] Conflict detection with floor plans
  - [ ] Store schedule as reference data

- [ ] **Auto-Discovery Mode**
  - [ ] Current implementation: find schedule in PDF
  - [ ] Extract schedule automatically
  - [ ] Fallback to manual input if not found
  - [ ] Log confidence score for auto-detected schedules

- [ ] **Mode Switching**
  - [ ] Allow users to toggle between modes
  - [ ] Re-run analysis with different schedule source
  - [ ] Compare results between modes

- [ ] **Validation & Reconciliation**
  - [ ] Reconcile auto-discovered schedule with floor plans
  - [ ] Flag mismatches (extra units, missing units)
  - [ ] Generate reconciliation report

---

## 📊 PHASE 4: TEST PLANS (5 Different Test Scenarios)

### Test Plan 1: Simple Single-Floor Apartment
- [ ] File: Small 2-bedroom apartment floor plan
- [ ] Expected: 2-4 HVAC units detected
- [ ] Schedule: Present in PDF
- [ ] Success Criteria: 100% unit detection, 0 false positives

### Test Plan 2: Multi-Floor Building
- [ ] File: 10-story residential building with mechanical plans
- [ ] Expected: 40-60 HVAC units across floors
- [ ] Schedule: Present in dedicated schedule page
- [ ] Success Criteria: >95% unit detection, unit-to-floor mapping correct

### Test Plan 3: Complex Commercial Building
- [ ] File: Office building with zone-based HVAC design
- [ ] Expected: Zoned units, VAV boxes, main units
- [ ] Schedule: Detailed equipment schedule with specs
- [ ] Success Criteria: Correct zone identification, accurate capacity readings

### Test Plan 4: Schedule Not in PDF
- [ ] File: Floor plan without unit schedule page
- [ ] Expected: Manual schedule upload required
- [ ] Schedule: Provided separately via CSV/Excel
- [ ] Success Criteria: Graceful fallback, accurate reconciliation

### Test Plan 5: Edge Cases & Stress Test
- [ ] Low-quality/blurry PDFs
- [ ] Mixed scales and orientations
- [ ] Handwritten annotations
- [ ] Multiple revisions/mark-ups
- [ ] Success Criteria: Graceful degradation, informative error messages

---

## ⚡ PHASE 5: FASTAPI SERVER DEPLOYMENT

### Server Setup
- [ ] **Create FastAPI Application**
  - [ ] Main app file: `server.py` or `main.py`
  - [ ] Configure CORS for frontend access
  - [ ] Set up logging and error handling
  - [ ] API documentation (Swagger UI enabled)

- [ ] **API Endpoints**
  - [ ] `POST /analyze` - Upload PDF and analyze
  - [ ] `POST /analyze-async` - Queue async analysis
  - [ ] `GET /results/{task_id}` - Retrieve results
  - [ ] `POST /schedule/upload` - Upload schedule data
  - [ ] `GET /status/{task_id}` - Check processing status
  - [ ] `POST /compare-models` - Run A/B test comparison
  - [ ] `GET /health` - Health check

- [ ] **Request/Response Schemas**
  - [ ] Pydantic models for validation
  - [ ] Request body validation
  - [ ] Response formatting consistency
  - [ ] Error response standardization

- [ ] **Configuration Management**
  - [ ] Environment variables for API keys
  - [ ] Model selection (Gemini vs OpenAI)
  - [ ] Timeout settings
  - [ ] Resource limits (file size, processing time)

### Local Development Testing
- [ ] [ ] Run server locally: `uvicorn main:app --reload`
- [ ] [ ] Test all endpoints with curl or Postman
- [ ] [ ] Verify error handling
- [ ] [ ] Check response times

### Containerization
- [ ] **Docker Setup**
  - [ ] Create Dockerfile
  - [ ] Define requirements.txt with all dependencies
  - [ ] Build image: `docker build -t hvac-analyzer .`
  - [ ] Test container locally
  - [ ] Push to Docker registry (Docker Hub, ECR, etc.)

- [ ] **Docker Compose (Optional)**
  - [ ] Multi-container setup if needed
  - [ ] Database/cache services
  - [ ] Environment configuration

### Cloud Deployment
- [ ] **Choose Cloud Provider** (Pick one)
  - [ ] ☐ AWS (EC2, ECS, Lambda)
  - [ ] ☐ Google Cloud (Cloud Run, App Engine)
  - [ ] ☐ Azure (App Service, Container Instances)
  - [ ] ☐ Heroku (Simple, good for demos)

- [ ] **Deploy to Production**
  - [ ] Set up production environment
  - [ ] Configure API keys securely (use secrets manager)
  - [ ] Set up logging to cloud (CloudWatch, Stackdriver, etc.)
  - [ ] Configure auto-scaling if applicable
  - [ ] Set up health checks and monitoring

- [ ] **Domain & SSL**
  - [ ] Acquire domain (if not using cloud domain)
  - [ ] Configure SSL/TLS certificate
  - [ ] Set up DNS records
  - [ ] Test HTTPS access

---

## 🧪 PHASE 6: TESTING STRATEGY

### Unit Tests
- [ ] **Data Extraction Module**
  - [ ] Test PDF parsing with sample files
  - [ ] Test text extraction accuracy
  - [ ] Test OCR fallback
  - [ ] Target: >90% code coverage

- [ ] **Tag/Unit Detection**
  - [ ] Test regex patterns for unit tags
  - [ ] Test normalization (AC-A, AC-B, etc.)
  - [ ] Test edge cases (partial tags, typos)

- [ ] **Schedule Parsing**
  - [ ] Test schedule extraction from tables
  - [ ] Test quantity parsing
  - [ ] Test unit type classification
  - [ ] Various schedule formats

### Integration Tests
- [ ] **End-to-End Pipeline**
  - [ ] Load PDF → Extract content → Parse schedule → Generate report
  - [ ] Verify data flows correctly through pipeline
  - [ ] Test error recovery

- [ ] **API Endpoint Tests**
  - [ ] Test file upload and processing
  - [ ] Test async task queueing
  - [ ] Test result retrieval
  - [ ] Test error responses

- [ ] **Multi-Floor Reconciliation**
  - [ ] Test floor-to-unit mapping
  - [ ] Test conflict detection
  - [ ] Verify reconciliation accuracy

### Performance Tests
- [ ] **Speed Benchmarks**
  - [ ] Single PDF: < 30 seconds (with Gemini)
  - [ ] Single PDF: < 15 seconds (with OpenAI if faster)
  - [ ] Batch processing: Linear scaling
  - [ ] API endpoint: < 500ms response time for result retrieval

- [ ] **Load Testing**
  - [ ] Concurrent requests: Test with 5, 10, 50 concurrent uploads
  - [ ] Monitor memory usage
  - [ ] Monitor API quota consumption
  - [ ] Identify bottlenecks

- [ ] **Stress Testing**
  - [ ] Large PDF files (50+ MB)
  - [ ] PDFs with 100+ pages
  - [ ] Rapid concurrent requests
  - [ ] Out-of-memory scenarios

### Data Quality Tests
- [ ] **Known Good Dataset**
  - [ ] Create 5-10 hand-verified blueprints
  - [ ] Document expected outputs
  - [ ] Run regression tests before each deployment
  - [ ] Track accuracy metrics over time

- [ ] **Model Comparison**
  - [ ] Run Gemini and OpenAI on same dataset
  - [ ] Compare accuracy, speed, cost
  - [ ] Generate comparison report

---

## 🚀 PHASE 7: DEMO PREPARATION

### Demo Scenarios
- [ ] **Quick Demo (5 minutes)**
  - [ ] Single apartment floor plan
  - [ ] Real-time schedule detection
  - [ ] Live API call and results display

- [ ] **Full Demo (15 minutes)**
  - [ ] Multi-floor building
  - [ ] Schedule extraction + reconciliation
  - [ ] A/B testing comparison (Gemini vs OpenAI)
  - [ ] API performance metrics

- [ ] **Interactive Features for Demo**
  - [ ] Upload custom blueprint
  - [ ] Toggle between Gemini/OpenAI
  - [ ] View extracted schedule data
  - [ ] Download analysis report

### Demo Infrastructure
- [ ] **Hosted Demo Server**
  - [ ] Deploy to live URL (not localhost)
  - [ ] Use pre-cached results for reliability
  - [ ] Have fallback demo data

- [ ] **Demo Data**
  - [ ] Prepare 5 sample blueprints
  - [ ] Pre-run analyses and cache results
  - [ ] Create "demo mode" that returns cached results instantly

- [ ] **UI/Frontend (if applicable)**
  - [ ] Create simple web interface or CLI demo
  - [ ] File upload form
  - [ ] Results display (table or PDF report)
  - [ ] Comparison view

---

## 📋 PHASE 8: DOCUMENTATION

### Technical Documentation
- [ ] **README.md**
  - [ ] Project overview
  - [ ] Installation instructions
  - [ ] Quick start guide
  - [ ] Architecture diagram

- [ ] **API Documentation**
  - [ ] Auto-generated from FastAPI (Swagger UI)
  - [ ] Example requests/responses
  - [ ] Error codes and meanings
  - [ ] Rate limits and quotas

- [ ] **Deployment Guide**
  - [ ] Step-by-step deployment instructions
  - [ ] Environment setup
  - [ ] Secret management
  - [ ] Monitoring and debugging

- [ ] **Architecture Document**
  - [ ] System design overview
  - [ ] Data flow diagrams
  - [ ] Component interactions
  - [ ] Scalability considerations

### User Documentation
- [ ] **User Guide**
  - [ ] How to upload blueprints
  - [ ] How to interpret results
  - [ ] Schedule format requirements
  - [ ] Troubleshooting common issues

- [ ] **FAQ**
  - [ ] Common questions
  - [ ] Known limitations
  - [ ] Cost information
  - [ ] Support contact

---

## 🔒 PHASE 9: SECURITY & COMPLIANCE

### Data Security
- [ ] **API Key Management**
  - [ ] Gemini API key secured (environment variable)
  - [ ] OpenAI API key secured (environment variable)
  - [ ] No keys in version control
  - [ ] Rotation policy documented

- [ ] **File Upload Security**
  - [ ] Validate file type (PDF only)
  - [ ] Validate file size (max 50MB)
  - [ ] Scan for malware/suspicious content
  - [ ] Store securely with access controls

- [ ] **Data Privacy**
  - [ ] Blueprints not shared between users
  - [ ] Results encrypted at rest
  - [ ] Audit logging of all access
  - [ ] GDPR/compliance checklist

### Infrastructure Security
- [ ] **Network Security**
  - [ ] HTTPS/TLS enforced
  - [ ] CORS properly configured
  - [ ] Rate limiting on API endpoints
  - [ ] DDoS protection if on public cloud

- [ ] **Authentication (if multi-user)**
  - [ ] API key authentication
  - [ ] JWT token support
  - [ ] User session management

---

## 📊 PHASE 10: MONITORING & OBSERVABILITY

### Logging
- [ ] **Application Logs**
  - [ ] Request/response logging
  - [ ] Error stack traces
  - [ ] Performance metrics (processing time)
  - [ ] Model usage tracking

- [ ] **Log Aggregation**
  - [ ] Centralized logging (CloudWatch, ELK, etc.)
  - [ ] Log retention policy (30 days minimum)
  - [ ] Searchable logs with timestamps

### Metrics & Monitoring
- [ ] **Key Metrics to Track**
  - [ ] API response time (p50, p95, p99)
  - [ ] Error rate (5xx, 4xx)
  - [ ] Gemini API calls/tokens used
  - [ ] OpenAI API calls/tokens used
  - [ ] Cost per blueprint analyzed
  - [ ] File upload success rate

- [ ] **Dashboards**
  - [ ] Create monitoring dashboard
  - [ ] Real-time request metrics
  - [ ] Error rate trends
  - [ ] Cost tracking

### Alerting
- [ ] **Set Up Alerts for**
  - [ ] High error rate (>5%)
  - [ ] High API latency (>60s)
  - [ ] API quota exhaustion (90%+ used)
  - [ ] Server down/unhealthy

---

## ✅ PHASE 11: PRE-PRODUCTION SIGN-OFF

### Final Validation Checklist
- [ ] **Functionality**
  - [ ] All features working as specified
  - [ ] All test plans passing (100% success)
  - [ ] No unresolved bugs
  - [ ] User workflows tested end-to-end

- [ ] **Performance**
  - [ ] Response times within SLA
  - [ ] Load testing passed
  - [ ] Cost per blueprint acceptable
  - [ ] Scalability plan documented

- [ ] **Quality**
  - [ ] Code reviewed and approved
  - [ ] Test coverage >80%
  - [ ] No security vulnerabilities
  - [ ] Documentation complete

- [ ] **Deployment**
  - [ ] Production environment ready
  - [ ] Monitoring configured
  - [ ] Rollback plan documented
  - [ ] Incident response plan ready

---

## 🎯 QUICK WINS (Do These First)

Priority order for fastest path to demo:

1. **FastAPI Server** (1-2 days)
   - Create basic API with file upload endpoint
   - Wire up existing analysis code
   - Deploy to cloud (Heroku or Google Cloud Run easiest)

2. **Schedule Display Mode** (1 day)
   - Add UI/endpoint for manual schedule upload
   - Create toggle between "auto-find" vs "use provided schedule"
   - Update analysis to use provided schedule

3. **A/B Testing Framework** (1 day)
   - Add parameter to choose Gemini vs OpenAI
   - Log timing/cost data
   - Generate simple comparison output

4. **Floor Apt Validation** (2 days)
   - Add logic to validate floor number from floor plans
   - Check unit mappings make sense
   - Add validation error reporting

5. **5 Test Plans** (2-3 days)
   - Create test dataset with known-good answers
   - Run through each scenario
   - Document results and success metrics

---

## 📈 SUCCESS METRICS

**For Demo:**
- ✅ Upload PDF → Get results in <60 seconds
- ✅ Detect all HVAC units in test blueprint
- ✅ Compare Gemini vs OpenAI (show speed + cost difference)
- ✅ Show floor-to-unit mapping validation

**For Production:**
- ✅ 95%+ accuracy on known test set
- ✅ <30s analysis time per blueprint
- ✅ <$0.10 cost per blueprint
- ✅ 99.9% uptime
- ✅ <5% error rate

---

## 🤝 Support & Next Steps

**Questions?** Contact: [your email]  
**Timeline**: 2-4 weeks to full production readiness  
**Resources needed**: API keys (Gemini, OpenAI), cloud account, monitoring tools