YouTube-Directus Automation Service

Automatically extract YouTube video thumbnails and upload them to Directus when YouTube videos are created or updated in your Directus CMS.

🎯 MVP Features

• Webhook Integration: Receives Directus webhooks on item create/update events
• YouTube Processing: Extracts video IDs from various YouTube URL formats
• Thumbnail Extraction: Downloads best quality thumbnail (maxres → high → medium → default)
• Directus Upload: Uploads thumbnails to Directus files collection
• Item Association: Updates YouTube video items with thumbnail file reference
• Error Handling: Robust error handling with logging and fallback mechanisms
• Security: Optional webhook signature verification

🏗️ Architecture

[Directus CMS] → [Webhook] → [FastAPI Service] → [YouTube API] → [Thumbnail Download]
                                    ↓
[Directus Item Update] ← [File Upload] ← [Image Processing] ← [Quality Selection]

🚀 Quick Start

1. Setup Environment

# Copy environment template
cp .env.example .env

# Edit .env with your configuration
nano .env

Required configuration:

DIRECTUS_URL="https://your-directus-instance.com/"
DIRECTUS_TOKEN="your_directus_api_token"

Optional but recommended:

YOUTUBE_API_KEY="your_youtube_api_key"  # For better reliability
DIRECTUS_WEBHOOK_SECRET="your_webhook_secret"  # For security

2. Start the Service

# Install dependencies and start service
./start.sh

The service will start on http://localhost:8000

3. Test the Service

# Run basic service tests
python3 test_service.py

4. Configure Directus Webhook

In your Directus admin panel:

1. Go to Settings → Webhooks
2. Create a new webhook:
   • Name: YouTube Thumbnail Automation
   • Method: POST
   • URL: http://localhost:8000/webhook/directus (or your deployed service URL)
   • Status: Active
   • Collections: Select your media_items collection
   • Actions: Create, Update

🔧 Configuration

Environment Variables

Variable	Required	Description
DIRECTUS_URL	✅	Your Directus instance URL
DIRECTUS_TOKEN	✅	Directus API token with file upload permissions
YOUTUBE_API_KEY	⚠️	YouTube Data API v3 key (recommended)
DIRECTUS_WEBHOOK_SECRET	⚠️	Secret for webhook signature verification
PORT	❌	Service port (default: 8000)

YouTube API Setup (Optional but Recommended)

1. Go to Google Cloud Console
2. Create a new project or select existing
3. Enable YouTube Data API v3
4. Create API credentials (API Key)
5. Add the API key to your .env file

Benefits of YouTube API:

• Better error handling for private/deleted videos
• Access to video metadata
• More reliable thumbnail URLs
• Rate limit information

Without YouTube API:

• Service uses fallback thumbnail URLs
• Works for most public videos
• Less error information available

📋 Collection Schema Requirements

Your Directus collection should have:

Required Fields:

• A field containing YouTube URLs (detected automatically from common field names)
• A file field for thumbnail association (assumes thumbnail field name)

Supported YouTube URL Field Names:

• youtube_url
• url
• link
• video_url
• youtube_link

Example Collection Schema:

-- media_items collection
id (Primary Key)
title (String)
youtube_url (String) -- YouTube URL field
thumbnail (File) -- Thumbnail association
description (Text)
created_at (DateTime)
updated_at (DateTime)

🧪 Testing

Manual Testing

1. Start the service: ./start.sh
2. Run tests: python3 test_service.py
3. Check service health: curl http://localhost:8000/health

Test with Real Directus

1. Create a new item in your media_items collection
2. Add a YouTube URL (e.g., https://www.youtube.com/watch?v=dQw4w9WgXcQ)
3. Check service logs for processing status
4. Verify thumbnail appears in Directus

Webhook Testing

# Test webhook endpoint directly
curl -X POST http://localhost:8000/webhook/directus \
  -H "Content-Type: application/json" \
  -d '{
    "event": "items.create",
    "payload": {
      "id": "test-123",
      "youtube_url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
    },
    "keys": ["test-123"]
  }'

📊 Monitoring

Service Endpoints

• GET / - Service information
• GET /health - Health check with configuration status
• POST /webhook/directus - Webhook endpoint for Directus

Health Check Response

{
  "status": "healthy",
  "service": "youtube-directus-automation",
  "timestamp": "2025-01-11T10:30:00Z",
  "config": {
    "directus_url": "https://your-instance.com/",
    "has_directus_token": true,
    "has_youtube_api_key": false,
    "has_webhook_secret": false
  }
}

Logging

Service logs include:

• Webhook event processing
• YouTube video ID extraction
• Thumbnail download status
• Directus upload results
• Error details and stack traces

🔒 Security

Webhook Signature Verification

Enable webhook security by setting DIRECTUS_WEBHOOK_SECRET:

1. Generate a secure random secret
2. Add to .env: DIRECTUS_WEBHOOK_SECRET="your-secret-here"
3. Configure the same secret in Directus webhook settings
4. Service will verify HMAC-SHA256 signatures automatically

API Token Permissions

Your Directus token needs:

• Read access to your YouTube videos collection
• Create/Upload access to files collection
• Update access to your YouTube videos collection

🚨 Troubleshooting

Common Issues

❌ "DIRECTUS_TOKEN not set" error

• Add your Directus API token to .env
• Verify token has correct permissions

❌ "Could not extract video ID" error

• Check YouTube URL format
• Ensure URL field contains valid YouTube URLs

❌ "Failed to download thumbnail" error

• Video might be private/deleted
• Check internet connectivity
• Try adding YouTube API key for better error handling

❌ "Directus upload failed" error

• Check Directus token permissions
• Verify Directus instance is accessible
• Check Directus storage configuration

❌ Webhook not triggering

• Verify webhook URL is correct
• Check Directus webhook configuration
• Ensure service is running and accessible
• Check webhook secret configuration

Debug Mode

For detailed debugging, check service logs when running ./start.sh.

🛠️ Development

Project Structure

youtube-automation/
├── src/
│   └── main.py              # Main FastAPI application
├── requirements.txt         # Python dependencies
├── .env.example            # Environment template
├── .env                    # Your configuration (not in git)
├── start.sh               # Service startup script
├── test_service.py        # Test suite
├── README.md              # This file
└── .taskmaster/           # Task Master project management

Extending the Service

The service is built with extensibility in mind:

• Add new webhook endpoints in main.py
• Extend YouTubeProcessor class for additional features
• Add new thumbnail processing logic
• Implement additional file format support

Task Master Integration

This project uses Task Master for project management:

# View project tasks
task-master list

# Get next task
task-master next

# Update task status
task-master set-status --id=1 --status=done

🤝 Contributing

1. Check existing tasks: task-master list
2. Pick a task: task-master next
3. Make changes and test: python3 test_service.py
4. Update task status: task-master set-status --id=X --status=done

📄 License

This project is part of the Personal AI Assistant ecosystem. See main project license.

---

🎉 Happy automating! If you run into issues, check the troubleshooting section or create a task in Task Master.