--- description: Trax project structure patterns and conventions for consistent development globs: src/**/*.py, tests/**/*.py, docs/**/*.md, scripts/**/*.sh, *.toml, *.md alwaysApply: false --- # Trax Project Structure Rules **Trax** is a production-ready media transcription platform with protocol-based architecture, optimized for M3 MacBook performance using the ultra-fast `uv` package manager. > **Note**: For project overview, quick start, and navigation, see [agents.mdc](mdc:.cursor/rules/agents.mdc). This rule focuses on technical implementation patterns and directory structure standards. ## Architecture Principles - **Protocol-Based Design**: Use `typing.Protocol` for all service interfaces - **Database Registry Pattern**: Prevent SQLAlchemy "multiple classes" errors - **Download-First Architecture**: Always download media before processing - **Real Files Testing**: Use actual audio files, never mocks - **Backend-First Development**: Get data layer right before UI - **Single Responsibility**: Keep files under 300 LOC (350 max if justified) ## Directory Structure Standards ### Root Directory Organization ``` trax/ ├── AGENTS.md # Development rules for AI agents ├── CLAUDE.md # Project context for Claude Code ├── EXECUTIVE-SUMMARY.md # High-level project overview ├── PROJECT-DIRECTORY.md # Directory structure documentation ├── README.md # Project introduction and quick start ├── pyproject.toml # Project configuration and dependencies ├── requirements.txt # Locked dependencies (uv generated) ├── alembic.ini # Database migration configuration └── scratchpad.md # Temporary notes and ideas ``` ### Source Code Organization (`src/`) ``` src/ ├── config.py # Centralized configuration system ├── cli/ # Command-line interface │ ├── main.py # Click-based CLI implementation │ ├── enhanced_cli.py # Enhanced CLI with progress │ ├── research.py # Research agent CLI │ └── commands/ # Command modules ├── services/ # Business logic services │ ├── protocols.py # Service interfaces (REQUIRED) │ ├── transcription_service.py │ ├── media_service.py │ ├── enhancement/ # AI enhancement services │ ├── research/ # Research agent services │ └── mocks/ # Mock implementations for testing ├── repositories/ # Data access layer │ ├── media_repository.py │ ├── transcription_repository.py │ └── youtube_repository.py ├── database/ # Database layer │ ├── __init__.py # Registry pattern implementation │ ├── models.py # All models in single file │ ├── connection.py # Connection management │ └── utils.py # Database utilities ├── base/ # Base classes and shared functionality │ ├── services.py # Base service implementations │ ├── repositories.py # Repository base classes │ └── processors.py # Processing base classes ├── errors/ # Error handling system │ ├── base.py # Base error classes │ ├── codes.py # Error code definitions │ └── classification.py # Error classification ├── logging/ # Logging configuration │ ├── config.py # Logging setup │ ├── metrics.py # Performance metrics │ └── utils.py # Logging utilities ├── security/ # Security components │ ├── encrypted_storage.py # Secure storage │ ├── input_sanitization.py # Input validation │ └── user_permissions.py # Access control └── agents/ # AI agent components ├── rules/ # Agent rule files │ ├── TRANSCRIPTION_RULES.md │ ├── BATCH_PROCESSING_RULES.md │ ├── DATABASE_RULES.md │ ├── CACHING_RULES.md │ └── EXPORT_RULES.md └── tools/ # Agent tools ``` ### Testing Structure (`tests/`) ``` tests/ ├── conftest.py # Pytest configuration and fixtures ├── fixtures/ # Test fixtures and data │ ├── audio/ # REAL audio files (no mocks) │ │ ├── sample_5s.wav # 5-second test file │ │ ├── sample_30s.mp3 # 30-second test file │ │ ├── sample_2m.mp4 # 2-minute test file │ │ ├── sample_noisy.wav # Noisy audio test │ │ ├── sample_multi.wav # Multi-speaker test │ │ └── sample_tech.mp3 # Technical content test │ └── README.md # Test fixtures documentation ├── test_*.py # Individual test modules └── testing_suite.py # Comprehensive test suite ``` ### Documentation Structure (`docs/`) ``` docs/ ├── architecture/ # Architecture documentation │ ├── development-patterns.md # Historical learnings │ ├── audio-processing.md # Audio pipeline details │ ├── error-handling-and-logging.md # Error system │ └── iterative-pipeline.md # Version progression ├── reports/ # Analysis reports │ ├── 01-repository-inventory.md │ ├── 02-historical-context.md │ ├── 03-architecture-design.md │ ├── 04-team-structure.md │ ├── 05-technical-migration.md │ └── 06-product-vision.md ├── templates/ # Documentation templates │ ├── ai-friendly-prd-template.md │ ├── adaptive-prd-template.md │ └── ecosystem-prd-template.md ├── CLI.md # Command reference ├── API.md # API documentation ├── DATABASE.md # Database schema ├── RESEARCH_AGENT.md # Research agent docs └── TROUBLESHOOTING.md # Common issues ``` ### Data Organization (`data/`) ``` data/ ├── media/ # Media file storage │ ├── downloads/ # Downloaded media files │ └── processed/ # Processed audio files ├── exports/ # Export output files │ ├── json/ # JSON export files │ └── txt/ # Text export files └── cache/ # Cache storage (if used) ``` ### Scripts Directory (`scripts/`) ``` scripts/ ├── setup_dev.sh # Development environment setup ├── setup_postgresql.sh # Database initialization ├── tm_master.sh # Taskmaster master interface ├── tm_status.sh # Status checking ├── tm_search.sh # Task searching ├── tm_workflow.sh # Workflow management ├── tm_analyze.sh # Analysis tools ├── tm_quick.sh # Quick operations └── README_taskmaster_helpers.md # Helper scripts documentation ``` ## Coding Patterns and Conventions ### Service Layer Patterns ### Protocol-Based Interfaces ```python # ✅ DO: Use Protocol-Based Interfaces # src/services/protocols.py from typing import Protocol, runtime_checkable from pathlib import Path @runtime_checkable class TranscriptionServiceProtocol(Protocol): """Protocol for transcription services.""" async def transcribe_file( self, media_file: MediaFile, config: Optional[TranscriptionConfig] = None ) -> TranscriptionResult: """Transcribe a media file.""" ... # Implementation class WhisperService: """Implements TranscriptionServiceProtocol.""" async def transcribe_file(self, media_file, config=None): # Implementation here pass ``` ### Factory Functions ```python # ✅ DO: Use Factory Functions # src/services/factories.py def create_transcription_service(config: Dict[str, Any]) -> TranscriptionServiceProtocol: """Create transcription service instance.""" service_type = config.get("type", "whisper") if service_type == "whisper": return WhisperService(config) elif service_type == "mock": return MockTranscriptionService(config) else: raise ValueError(f"Unknown service type: {service_type}") ``` ### Database Layer Patterns ### Registry Pattern for Models ```python # ✅ DO: Use Registry Pattern for Models # src/database/__init__.py from typing import Dict, Type from sqlalchemy.ext.declarative import declarative_base Base = declarative_base() # Model registry to prevent SQLAlchemy conflicts _model_registry: Dict[str, Type[Base]] = {} def register_model(model_class: Type[Base]) -> Type[Base]: """Register a model in the central registry.""" name = model_class.__name__ if name in _model_registry: return _model_registry[name] # Return existing _model_registry[name] = model_class return model_class # Usage in models @register_model class MediaFile(Base): __tablename__ = "media_files" # Model definition here ``` ### JSONB for Flexible Data ```python # ✅ DO: Use JSONB for Flexible Data # src/database/models.py from sqlalchemy.dialects.postgresql import JSONB, UUID as PGUUID class TranscriptionResult(Base): __tablename__ = "transcription_results" id = Column(PGUUID(as_uuid=True), primary_key=True, default=uuid4) content = Column(JSONB, nullable=False) # Flexible transcript data segments = Column(JSONB) # Timestamped segments confidence_scores = Column(JSONB) # Confidence metrics processing_metadata = Column(JSONB) # Additional metadata ``` ### Repository Layer Patterns ### Repository Protocols ```python # ✅ DO: Define Repository Protocols # src/repositories/protocols.py @runtime_checkable class MediaRepositoryProtocol(Protocol): """Protocol for media file repository operations.""" async def create(self, media_data: Dict[str, Any]) -> MediaFile: """Create a new media file record.""" ... async def get_by_id(self, media_id: UUID) -> Optional[MediaFile]: """Get media file by ID.""" ... ``` ### Error Handling Patterns ### Hierarchical Error System ```python # ✅ DO: Use Hierarchical Error System # src/errors/base.py class TraxError(Exception): """Base exception for all Trax platform errors.""" def __init__( self, message: str, error_code: Optional[ErrorCode] = None, context: Optional[Dict[str, Any]] = None, original_error: Optional[Exception] = None ): super().__init__(message) self.message = message self.error_code = error_code self.context = context or {} self.original_error = original_error self.timestamp = datetime.now(timezone.utc) # Specific error types class TranscriptionError(TraxError): """Error raised when transcription processing fails.""" pass class MediaProcessingError(TraxError): """Error raised when media processing fails.""" pass ``` ### Configuration Patterns ### Centralized Configuration ```python # ✅ DO: Use Centralized Configuration # src/config.py class Config: """Centralized configuration for the trax project.""" # Project paths PROJECT_ROOT = Path(__file__).parent.parent DATA_DIR = PROJECT_ROOT / "data" # API Keys - AI Services (from root .env) ANTHROPIC_API_KEY: Optional[str] = os.getenv("ANTHROPIC_API_KEY") DEEPSEEK_API_KEY: Optional[str] = os.getenv("DEEPSEEK_API_KEY") OPENAI_API_KEY: Optional[str] = os.getenv("OPENAI_API_KEY") @classmethod def validate_required_keys(cls, required_keys: List[str]) -> bool: """Validate that required API keys are present.""" missing_keys = [] for key in required_keys: if not getattr(cls, key, None): missing_keys.append(key) if missing_keys: print(f"❌ Missing required API keys: {', '.join(missing_keys)}") return False return True # Create convenience instance config = Config() ``` ### CLI Patterns ### Click with Rich for User Interface ```python # ✅ DO: Use Click with Rich for User Interface # src/cli/main.py import click from rich.console import Console from rich.progress import Progress, SpinnerColumn, TextColumn console = Console() @click.group() @click.version_option(version="1.0.0") def cli(): """Trax: Personal Research Transcription Tool""" pass @cli.command() @click.argument("input_file", type=click.Path(exists=True)) @click.option("--output", "-o", help="Output directory") @click.option("--format", "-f", type=click.Choice(["json", "txt", "srt"])) def transcribe(input_file, output, format): """Transcribe a media file.""" with Progress( SpinnerColumn(), TextColumn("[progress.description]{task.description}"), console=console, ) as progress: task = progress.add_task("Transcribing...", total=None) # Processing logic here progress.update(task, description="Complete!") ``` ### Testing Patterns ### Real Audio Files for Testing ```python # ✅ DO: Use Real Audio Files for Testing # tests/conftest.py @pytest.fixture def sample_audio_files(): """Provide real audio files for testing.""" return { "short": Path("tests/fixtures/audio/sample_5s.wav"), "medium": Path("tests/fixtures/audio/sample_30s.mp3"), "long": Path("tests/fixtures/audio/sample_2m.mp4"), "noisy": Path("tests/fixtures/audio/sample_noisy.wav"), "multi_speaker": Path("tests/fixtures/audio/sample_multi.wav"), "technical": Path("tests/fixtures/audio/sample_tech.mp3"), } # Test implementation async def test_transcription_accuracy(sample_audio_files, transcription_service): """Test transcription with real audio files.""" result = await transcription_service.transcribe_file( sample_audio_files["short"] ) assert result.accuracy >= 0.95 # 95% accuracy requirement assert len(result.segments) > 0 assert result.processing_time < 30.0 # Performance requirement ``` ### Anti-Patterns ```python # ❌ DON'T: Use Mocks for Core Functionality # BAD: Mocking audio processing @patch("whisper.load_model") def test_transcription_mock(mock_whisper): # This won't catch real audio processing issues pass # GOOD: Use real files with small samples def test_transcription_real(sample_audio_files): # Tests actual audio processing pipeline pass ``` ## File Size and Organization Guidelines ### Keep Files Focused and Manageable - **Maximum 300 LOC** per file (350 if well-justified) - **Single responsibility** per module - **Clear naming** that describes purpose - **Logical grouping** of related functionality ### Protocol and Implementation Separation ```python # protocols.py - Interface definitions only @runtime_checkable class ServiceProtocol(Protocol): def method(self) -> Result: ... # service_impl.py - Implementation class ConcreteService: def method(self) -> Result: # Implementation here pass # __init__.py - Public API from .protocols import ServiceProtocol from .service_impl import ConcreteService __all__ = ["ServiceProtocol", "ConcreteService"] ``` ## Development Workflow Patterns ### Adding New Services 1. **Define Protocol** in `src/services/protocols.py` 2. **Create Implementation** in `src/services/service_name.py` 3. **Add Factory Function** in `src/services/factories.py` 4. **Write Tests** with real data in `tests/test_service_name.py` 5. **Update Documentation** in `docs/` ### Database Changes 1. **Update Models** in `src/database/models.py` 2. **Create Migration** with `alembic revision -m "description"` 3. **Test Migration** with up/down paths 4. **Update Documentation** in `docs/DATABASE.md` 5. **Update Changelog** in `CHANGELOG.md` ### CLI Enhancements 1. **Add Command** in appropriate `src/cli/commands/` module 2. **Register Command** in `src/cli/main.py` 3. **Add Progress Reporting** with Rich 4. **Write Integration Test** in `tests/test_cli.py` 5. **Update CLI Documentation** in `docs/CLI.md` ## Performance and Resource Management ### Memory Usage Guidelines - **Target <2GB** for v1 pipeline - **Monitor Memory** with progress callbacks - **Cleanup Resources** after processing - **Use Streaming** for large files when possible ### Concurrency Patterns ```python # ✅ DO: Use asyncio for I/O operations async def process_batch(files: List[Path]) -> List[Result]: """Process files concurrently.""" semaphore = asyncio.Semaphore(8) # M3 optimized async def process_with_limit(file_path): async with semaphore: return await process_file(file_path) tasks = [process_with_limit(f) for f in files] return await asyncio.gather(*tasks) ``` ## Documentation Standards ### Rule Files Structure ```markdown # Rule Title ## Core Principles 1. **Principle 1**: Description 2. **Principle 2**: Description ## Implementation Patterns ### Pattern Name ```code # Example implementation ``` ### Anti-Patterns ```code # What NOT to do ``` ```markdown ## Performance Guidelines - Guideline 1 - Guideline 2 ``` ### API Documentation - **Use Docstrings** for all public interfaces - **Include Examples** in documentation - **Document Protocols** with clear contracts - **Update README.md** for user-facing changes - **Update agents.mdc** for project context and navigation changes ## Security and Validation ### Input Sanitization ```python # ✅ DO: Sanitize and validate user input # src/security/input_sanitization.py def sanitize_file_path(path: str) -> Path: """Sanitize and validate file paths.""" # Remove dangerous characters clean_path = re.sub(r'[<>:"|?*]', '', path) # Prevent directory traversal if '..' in clean_path: raise ValidationError("Directory traversal not allowed") return Path(clean_path) ``` ### Environment Configuration - **API Keys** inherited from root project `.env` - **Local Overrides** via `.env.local` - **Validation** of required keys at startup - **Secure Storage** for sensitive data --- **This rule ensures consistent project structure and development patterns across the Trax media transcription platform.**