7.9 KiB

Raw Permalink Blame History

YouTube-Directus Automation Service

Automatically extract YouTube video thumbnails and upload them to Directus when YouTube videos are created or updated in your Directus CMS.

🎯 MVP Features

Webhook Integration: Receives Directus webhooks on item create/update events
YouTube Processing: Extracts video IDs from various YouTube URL formats
Thumbnail Extraction: Downloads best quality thumbnail (maxres → high → medium → default)
Directus Upload: Uploads thumbnails to Directus files collection
Item Association: Updates YouTube video items with thumbnail file reference
Error Handling: Robust error handling with logging and fallback mechanisms
Security: Optional webhook signature verification

🏗️ Architecture

[Directus CMS] → [Webhook] → [FastAPI Service] → [YouTube API] → [Thumbnail Download]
                                    ↓
[Directus Item Update] ← [File Upload] ← [Image Processing] ← [Quality Selection]

🚀 Quick Start

1. Setup Environment

# Copy environment template
cp .env.example .env

# Edit .env with your configuration
nano .env

Required configuration:

DIRECTUS_URL="https://your-directus-instance.com/"
DIRECTUS_TOKEN="your_directus_api_token"

Optional but recommended:

YOUTUBE_API_KEY="your_youtube_api_key"  # For better reliability
DIRECTUS_WEBHOOK_SECRET="your_webhook_secret"  # For security

2. Start the Service

# Install dependencies and start service
./start.sh

The service will start on http://localhost:8000

3. Test the Service

# Run basic service tests
python3 test_service.py

4. Configure Directus Webhook

In your Directus admin panel:

Go to Settings → Webhooks
Create a new webhook:
- Name: YouTube Thumbnail Automation
- Method: POST
- URL: http://localhost:8000/webhook/directus (or your deployed service URL)
- Status: Active
- Collections: Select your media_items collection
- Actions: Create, Update

🔧 Configuration

Environment Variables

Variable	Required	Description
`DIRECTUS_URL`	✅	Your Directus instance URL
`DIRECTUS_TOKEN`	✅	Directus API token with file upload permissions
`YOUTUBE_API_KEY`	⚠️	YouTube Data API v3 key (recommended)
`DIRECTUS_WEBHOOK_SECRET`	⚠️	Secret for webhook signature verification
`PORT`	❌	Service port (default: 8000)

YouTube API Setup (Optional but Recommended)

Go to Google Cloud Console
Create a new project or select existing
Enable YouTube Data API v3
Create API credentials (API Key)
Add the API key to your .env file

Benefits of YouTube API:

Better error handling for private/deleted videos
Access to video metadata
More reliable thumbnail URLs
Rate limit information

Without YouTube API:

Service uses fallback thumbnail URLs
Works for most public videos
Less error information available

📋 Collection Schema Requirements

Your Directus collection should have:

Required Fields:

A field containing YouTube URLs (detected automatically from common field names)
A file field for thumbnail association (assumes thumbnail field name)

Supported YouTube URL Field Names:

youtube_url
url
link
video_url
youtube_link

Example Collection Schema:

-- media_items collection
id (Primary Key)
title (String)
youtube_url (String) -- YouTube URL field
thumbnail (File) -- Thumbnail association
description (Text)
created_at (DateTime)
updated_at (DateTime)

🧪 Testing

Manual Testing

Start the service: ./start.sh
Run tests: python3 test_service.py
Check service health: curl http://localhost:8000/health

Test with Real Directus

Create a new item in your media_items collection
Add a YouTube URL (e.g., https://www.youtube.com/watch?v=dQw4w9WgXcQ)
Check service logs for processing status
Verify thumbnail appears in Directus

Webhook Testing

# Test webhook endpoint directly
curl -X POST http://localhost:8000/webhook/directus \
  -H "Content-Type: application/json" \
  -d '{
    "event": "items.create",
    "payload": {
      "id": "test-123",
      "youtube_url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
    },
    "keys": ["test-123"]
  }'

📊 Monitoring

Service Endpoints

GET / - Service information
GET /health - Health check with configuration status
POST /webhook/directus - Webhook endpoint for Directus

Health Check Response

{
  "status": "healthy",
  "service": "youtube-directus-automation",
  "timestamp": "2025-01-11T10:30:00Z",
  "config": {
    "directus_url": "https://your-instance.com/",
    "has_directus_token": true,
    "has_youtube_api_key": false,
    "has_webhook_secret": false
  }
}

Logging

Service logs include:

Webhook event processing
YouTube video ID extraction
Thumbnail download status
Directus upload results
Error details and stack traces

🔒 Security

Webhook Signature Verification

Enable webhook security by setting DIRECTUS_WEBHOOK_SECRET:

Generate a secure random secret
Add to .env: DIRECTUS_WEBHOOK_SECRET="your-secret-here"
Configure the same secret in Directus webhook settings
Service will verify HMAC-SHA256 signatures automatically

API Token Permissions

Your Directus token needs:

Read access to your YouTube videos collection
Create/Upload access to files collection
Update access to your YouTube videos collection

🚨 Troubleshooting

Common Issues

❌ "DIRECTUS_TOKEN not set" error

Add your Directus API token to .env
Verify token has correct permissions

❌ "Could not extract video ID" error

Check YouTube URL format
Ensure URL field contains valid YouTube URLs

❌ "Failed to download thumbnail" error

Video might be private/deleted
Check internet connectivity
Try adding YouTube API key for better error handling

❌ "Directus upload failed" error

Check Directus token permissions
Verify Directus instance is accessible
Check Directus storage configuration

❌ Webhook not triggering

Verify webhook URL is correct
Check Directus webhook configuration
Ensure service is running and accessible
Check webhook secret configuration

Debug Mode

For detailed debugging, check service logs when running ./start.sh.

🛠️ Development

Project Structure

youtube-automation/
├── src/
│   └── main.py              # Main FastAPI application
├── requirements.txt         # Python dependencies
├── .env.example            # Environment template
├── .env                    # Your configuration (not in git)
├── start.sh               # Service startup script
├── test_service.py        # Test suite
├── README.md              # This file
└── .taskmaster/           # Task Master project management

Extending the Service

The service is built with extensibility in mind:

Add new webhook endpoints in main.py
Extend YouTubeProcessor class for additional features
Add new thumbnail processing logic
Implement additional file format support

Task Master Integration

This project uses Task Master for project management:

# View project tasks
task-master list

# Get next task
task-master next

# Update task status
task-master set-status --id=1 --status=done

🤝 Contributing

Check existing tasks: task-master list
Pick a task: task-master next
Make changes and test: python3 test_service.py
Update task status: task-master set-status --id=X --status=done

📄 License

This project is part of the Personal AI Assistant ecosystem. See main project license.

🎉 Happy automating! If you run into issues, check the troubleshooting section or create a task in Task Master.

7.9 KiB Raw Permalink Blame History