6.2 KiB

Raw Blame History

Testing Issues Documentation

Date: 2025-08-27
Status: Active Issues Documented
Reporter: Claude Code Agent

This document captures the testing problems encountered and their solutions for future reference.

Issue #1: SQLAlchemy Relationship Resolution Error

Problem: Test runner and direct pytest failing with SQLAlchemy errors:

NameError: Module 'models' has no mapped classes registered under the name 'batch_job'
sqlalchemy.exc.InvalidRequestError: When initializing mapper Mapper[User(users)], expression 'backend.models.batch_job.BatchJob' failed to locate a name

Root Cause: The BatchJob and BatchJobItem models existed in backend/models/batch_job.py but were not imported in the models __init__.py file. This caused SQLAlchemy to fail when resolving relationships defined in the User model.

Solution: Added missing imports to backend/models/__init__.py:

# Before (missing imports)
from .user import User, RefreshToken, APIKey, EmailVerificationToken, PasswordResetToken
from .summary import Summary, ExportHistory

# After (fixed)
from .user import User, RefreshToken, APIKey, EmailVerificationToken, PasswordResetToken
from .summary import Summary, ExportHistory
from .batch_job import BatchJob, BatchJobItem  # Added this line

__all__ = [
    # ... existing models ...
    "BatchJob",      # Added
    "BatchJobItem",  # Added
]

Verification: Auth service tests went from 20/21 failing to 21/21 passing.

Issue #2: Pydantic Configuration Validation Errors

Problem: Tests failing with Pydantic validation errors:

pydantic_core._pydantic_core.ValidationError: 19 validation errors for VideoDownloadConfig
anthropic_api_key: Extra inputs are not permitted [type=extra_forbidden]
openai_api_key: Extra inputs are not permitted [type=extra_forbidden]
...

Root Cause: The VideoDownloadConfig class extends BaseSettings and automatically loads environment variables. However, the environment contained many API keys and configuration variables that weren't defined in the model schema. Pydantic 2.x defaults to extra="forbid" which rejects unknown fields.

Solution: Modified the VideoDownloadConfig class configuration:

# File: backend/config/video_download_config.py
class VideoDownloadConfig(BaseSettings):
    # ... model fields ...
    
    class Config:
        env_file = ".env"
        env_prefix = "VIDEO_DOWNLOAD_"
        case_sensitive = False
        extra = "ignore"  # Added this line to allow extra environment variables

Verification: Enhanced video service tests went from collection errors to 13/23 passing.

Issue #3: Test Runner Result Parsing Bug

Problem: The custom test runner consistently reports "0/X passed" even when tests are actually passing.

Evidence:

Test runner shows: Completed unit tests: 0/229 passed
Direct pytest shows: 182 passed, 61 failed, 38 errors (75% pass rate)
Individual test files run successfully when tested directly

Root Cause: Bug in the test runner's pytest result parsing logic. The TestExecutor class in backend/test_runner/core/test_execution.py is not correctly parsing the pytest output or exit codes.

Current Status: UNRESOLVED - Test runner display bug exists but does not affect actual test functionality.

Workaround: Use direct pytest commands for accurate results:

# Instead of: ./run_tests.sh run-unit
# Use: PYTHONPATH=/path/to/project python3 -m pytest backend/tests/unit/

# For specific tests:
PYTHONPATH=/Users/enias/projects/my-ai-projects/apps/youtube-summarizer python3 -m pytest backend/tests/unit/test_auth_service.py -v

Issue #4: Test Environment Setup

Problem: Tests may fail if run without proper PYTHONPATH configuration.

Solution: Always set PYTHONPATH when running tests directly:

export PYTHONPATH="/Users/enias/projects/my-ai-projects/apps/youtube-summarizer"
# or
PYTHONPATH=/Users/enias/projects/my-ai-projects/apps/youtube-summarizer python3 -m pytest

Verification Script: Use ./scripts/validate_test_setup.py to check environment.

Current Test Status (as of 2025-08-27 01:45 UTC)

✅ Working Test Categories:

Authentication Tests: 21/21 passing (100%)
Core Service Tests: Most passing
Database Model Tests: Working after BatchJob fix
Basic Integration Tests: Many passing

❌ Known Failing Areas:

Enhanced Video Service: 10/23 failing (test implementation issues)
Video Downloader Tests: Multiple failures (mocking issues)
AI Service Tests: Some import/dependency errors
Complex Integration Tests: Various issues

Overall Stats:

Total Discovered: 241 tests
Passing: 182 tests (75%)
Failing: 61 tests
Errors: 38 tests

Recommended Next Steps

Fix Test Runner Parsing Bug:
- Investigate backend/test_runner/core/test_execution.py
- Fix pytest result parsing logic
- Ensure proper exit code handling
Address Remaining Test Failures:
- Fix mocking issues in video downloader tests
- Resolve import/dependency errors in AI service tests
- Update test implementations for enhanced video service
Improve Test Environment:
- Create more reliable test fixtures
- Improve test isolation
- Add better error reporting
Documentation:
- Update TESTING-INSTRUCTIONS.md with current status
- Document workarounds for known issues
- Create debugging guide for test failures

Commands for Testing Tomorrow

# Quick verification (works reliably)
PYTHONPATH=/Users/enias/projects/my-ai-projects/apps/youtube-summarizer python3 -m pytest backend/tests/unit/test_auth_service.py -v

# Full unit test run (accurate results)
PYTHONPATH=/Users/enias/projects/my-ai-projects/apps/youtube-summarizer python3 -m pytest backend/tests/unit/ --tb=no -q

# Debug specific failures
PYTHONPATH=/Users/enias/projects/my-ai-projects/apps/youtube-summarizer python3 -m pytest backend/tests/unit/test_enhanced_video_service.py -v --tb=short

# Test runner (has display bug but still useful for discovery)
./run_tests.sh list --category unit

Last Updated: 2025-08-27 01:45 UTC
Next Review: When test runner parsing bug is resolved

This document should be updated as issues are resolved and new problems are discovered.

6.2 KiB Raw Blame History