# Transcription Processing Rules

## Core Principles
1. **Download First**: Always download media to local storage before processing
2. **No Streaming**: Process complete files only for reliability
3. **Validate Early**: Check file integrity before expensive operations
4. **Track Progress**: Provide clear feedback during long operations

## Pipeline Rules

### V1: Basic Transcription
```python
# REQUIRED: Audio preprocessing
- Convert to 16kHz mono WAV
- Normalize audio levels
- Remove silence at start/end
- Chunk large files (>1 hour)

# REQUIRED: Whisper configuration
- Model: distil-large-v3 (M3 optimized)
- Compute type: int8_float32
- Language: auto-detect or specified
- Temperature: 0.0 for deterministic results

# REQUIRED: Output format
- Segments with timestamps
- Confidence scores per segment
- Word-level timestamps when available
- JSON structure with metadata
```

### V2: AI Enhancement
```python
# REQUIRED: Enhancement template
ENHANCEMENT_PROMPT = """
Fix the following transcript issues:
1. Correct punctuation and capitalization
2. Fix technical terms and proper nouns
3. Format into readable paragraphs
4. Preserve timestamps and speaker markers
5. Maintain original meaning

Transcript:
{transcript}

Return ONLY the enhanced transcript.
"""

# REQUIRED: Validation
- Compare lengths (should be similar)
- Check timestamp preservation
- Verify no content loss
- Score improvements
```

### V3: Multi-Pass Strategy
```python
# REQUIRED: Multiple passes
passes = [
    {"temperature": 0.0, "beam_size": 2, "best_of": 3},
    {"temperature": 0.2, "beam_size": 3, "best_of": 4},
    {"temperature": 0.4, "beam_size": 4, "best_of": 5}
]

# REQUIRED: Segment merging
- Use confidence-weighted voting
- Prefer consistent segments across passes
- Handle edge cases at boundaries
- Document disagreements
```

### V4: Speaker Diarization
```python
# REQUIRED: Diarization parameters
- Min segment duration: 1.0 seconds
- Max speakers: 10 (configurable)
- Clustering method: spectral
- Embedding model: speechbrain

# REQUIRED: Speaker assignment
- Map diarization to transcript segments
- Handle overlapping speech
- Create speaker profiles
- Track speaker statistics
```

## Error Handling Rules

### File Errors
```python
# Download failures
if download_error:
    retry_with_backoff(max_retries=3)
    if still_failing:
        return clear_error("Download failed: {reason}")

# Format errors
if unsupported_format:
    try_ffmpeg_conversion()
    if conversion_fails:
        return clear_error("Unsupported format: {format}")

# Corruption errors  
if corrupted_file:
    return clear_error("File corrupted at byte {position}")
```

### Processing Errors
```python
# Memory errors
if out_of_memory:
    chunk_file_smaller()
    process_in_parts()
    merge_results()

# Model errors
if whisper_fails:
    fallback_to_smaller_model()
    log_degraded_quality()
    continue_with_warning()

# Timeout errors
if processing_timeout:
    save_partial_progress()
    return recoverable_error()
```

## Quality Rules

### Accuracy Requirements
- V1: 95% minimum on clear audio
- V2: 99% after enhancement
- V3: 99.5% with multi-pass
- V4: 90% speaker identification

### Performance Requirements
- 5-minute audio: <30 seconds (V1)
- Enhancement: <5 seconds per minute
- Multi-pass: Parallel processing required
- Diarization: <2 seconds per speaker

### Validation Requirements
```python
# Every transcript MUST have:
assert transcript.get("segments")
assert transcript.get("text")
assert transcript.get("duration")
assert transcript.get("language")
assert transcript.get("confidence_score")

# Every segment MUST have:
assert segment.get("start")
assert segment.get("end")
assert segment.get("text")
assert segment.get("confidence", 0) > 0.5
```

## Testing Rules

### Test Data Requirements
```
tests/fixtures/audio/
├── sample_5s.wav      # Clear speech, single speaker
├── sample_30s.mp3     # Music + speech mix
├── sample_2m.mp4      # Video with dialogue
├── sample_noisy.wav   # Background noise test
├── sample_multi.wav   # Multiple speakers
└── sample_tech.mp3    # Technical terms
```

### Test Coverage Requirements
- Unit tests for each processing step
- Integration tests for full pipeline
- Performance benchmarks for each version
- Accuracy tests with ground truth
- Error recovery tests

## Monitoring Rules

### Metrics to Track
```python
metrics = {
    "processing_time": time_per_minute_of_audio,
    "accuracy_score": confidence_average,
    "enhancement_delta": before_after_diff,
    "memory_peak": max_memory_usage,
    "error_rate": errors_per_100_files,
    "cache_hit_rate": cached_vs_processed
}
```

### Alerts to Configure
- Processing time > 2x expected
- Accuracy < 90%
- Memory usage > 4GB
- Error rate > 5%
- Queue depth > 100 files

## Cost Management Rules

### API Usage
- Cache all AI enhancements (7 day TTL)
- Batch API calls when possible
- Use cheapest model that meets requirements
- Track costs per transcript

### Resource Usage
- Limit parallel processing to CPU cores
- Implement memory caps per process
- Clean up temp files immediately
- Compress stored transcripts

## Migration Rules

### Version Compatibility
- All versions must read V1 format
- Higher versions extend, not replace
- Provide downgrade paths
- Document breaking changes

### Data Migration
```python
# Upgrade path
def migrate_v1_to_v2(transcript_v1):
    transcript_v2 = transcript_v1.copy()
    transcript_v2["enhanced"] = enhance(transcript_v1)
    transcript_v2["version"] = "v2"
    return transcript_v2

# Downgrade path
def downgrade_v2_to_v1(transcript_v2):
    return transcript_v2.get("raw", transcript_v2)
```

---

*These rules ensure consistent, reliable transcription processing across all pipeline versions.*