media-downloader/docs/DEPENDENCY_UPDATES.md

# Automatic Dependency Updates

## Overview

The Dependency Updater automatically checks for and installs updates for critical components once per day when running in scheduler mode. This ensures FlareSolverr, Playwright browsers, and yt-dlp stay current without manual intervention.

## Why Auto-Updates?

**Critical dependencies that require frequent updates:**

1. **FlareSolverr** - Cloudflare bypass technology
   - Cloudflare frequently updates their bot detection
   - FlareSolverr updates to counter new blocks
   - Outdated version = downloads fail with Cloudflare errors

2. **yt-dlp** - Video download engine (TikTok, etc.)
   - TikTok/YouTube change their APIs constantly
   - yt-dlp releases updates almost daily
   - Outdated version = TikTok downloads fail

3. **Playwright Browsers** - Chromium/Firefox automation
   - Browser updates include security fixes
   - Anti-detection improvements
   - Outdated browsers are easier to detect

## How It Works

### Automatic Check Schedule

- **Runs**: Once every 24 hours (configurable)
- **Mode**: Scheduler only (not manual runs)
- **Time**: Checks every minute, but internal cooldown prevents spam
- **Location**: Integrated into scheduler loop

### Update Process

```
Scheduler Running
   ↓
Every 60 seconds:
   ↓
Check if 24 hours passed since last update check
   ↓ Yes
Update Components:
   1. FlareSolverr (docker pull + restart)
   2. Playwright (chromium + firefox)
   3. yt-dlp (pip upgrade)
   ↓
Log Results
   ↓
Send Notification (if updates installed)
   ↓
Save State with Timestamp
   ↓
Resume Scheduler
```

## Configuration

Located in `config/settings.json`:

```json
{
  "dependency_updates": {
    "enabled": true,
    "check_interval_hours": 24,
    "auto_install": true,
    "components": {
      "flaresolverr": {
        "enabled": true,
        "notify_on_update": true
      },
      "playwright": {
        "enabled": true,
        "notify_on_update": false
      },
      "yt_dlp": {
        "enabled": true,
        "notify_on_update": false
      }
    },
    "pushover": {
      "enabled": true,
      "priority": -1,
      "sound": "magic"
    }
  }
}
```

### Configuration Options

**Main Settings:**
- `enabled` (boolean) - Master switch for auto-updates (default: true)
- `check_interval_hours` (integer) - Hours between update checks (default: 24)
- `auto_install` (boolean) - Automatically install updates (default: true)

**Component Settings:**
- `enabled` (boolean) - Enable updates for this component
- `notify_on_update` (boolean) - Send Pushover notification when updated

**Pushover Settings:**
- `enabled` (boolean) - Enable update notifications
- `priority` (integer) - Notification priority (-2 to 2, -1 = low)
- `sound` (string) - Notification sound (default: "magic")

## Update Components

### 1. FlareSolverr (Docker Container)

**Why**: Cloudflare constantly updates bot detection; FlareSolverr must keep pace

**Update Process:**
```bash
1. docker pull ghcr.io/flaresolverr/flaresolverr:latest
2. If new image downloaded:
   a. docker stop flaresolverr
   b. docker rm flaresolverr
   c. docker run -d --name flaresolverr -p 8191:8191 ...
3. Container running with latest version
```

**Notification**: ✅ Enabled by default (important update)

**Downtime**: ~5 seconds during container restart

### 2. Playwright Browsers (Chromium + Firefox)

**Why**: Browser updates include anti-detection improvements and security fixes

**Update Process:**
```bash
1. python3 -m playwright install chromium
2. python3 -m playwright install firefox
3. Browsers updated in /opt/media-downloader/.playwright/
```

**Notification**: ❌ Disabled by default (routine update)

**Downtime**: None (browsers updated while not in use)

### 3. yt-dlp (Python Package)

**Why**: TikTok/YouTube change APIs constantly; yt-dlp updates almost daily

**Update Process:**
```bash
1. pip3 install --upgrade yt-dlp
2. Latest version installed system-wide
```

**Notification**: ❌ Disabled by default (very frequent)

**Downtime**: None

## Notification Examples

**FlareSolverr Update:**
```
🔄 Dependencies Updated

FlareSolverr has been updated to the latest version.

Updated at: Oct 29, 3:15 AM
```

**Multiple Updates:**
```
🔄 Dependencies Updated

The following components have been updated:

• FlareSolverr
• Playwright Browsers
• yt-dlp

Updated at: Oct 29, 3:15 AM
```

## State Tracking

State stored in `/opt/media-downloader/database/dependency_updates.json`:

```json
{
  "last_check": "2025-10-29T03:15:00",
  "components": {
    "flaresolverr": {
      "last_update": "2025-10-29T03:15:00",
      "last_check": "2025-10-29T03:15:00",
      "status": "updated"
    },
    "playwright": {
      "last_update": "2025-10-28T03:15:00",
      "last_check": "2025-10-29T03:15:00",
      "status": "current"
    },
    "yt_dlp": {
      "last_update": "2025-10-29T03:15:00",
      "last_check": "2025-10-29T03:15:00",
      "status": "updated"
    }
  }
}
```

## Testing

### Manual Update Check

```python
from modules.dependency_updater import DependencyUpdater
from modules.pushover_notifier import create_notifier_from_config
import json

# Load config
with open('/opt/media-downloader/config/settings.json') as f:
    config = json.load(f)

# Initialize updater
notifier = create_notifier_from_config(config)
updater = DependencyUpdater(
    config=config.get('dependency_updates', {}),
    pushover_notifier=notifier,
    scheduler_mode=True
)

# Force update check (ignores 24h cooldown)
results = updater.force_update_check()

print("Update Results:")
for component, updated in results.items():
    status = "✓ Updated" if updated else "Already current"
    print(f"  {component}: {status}")
```

### Check Last Update Time

```bash
cat /opt/media-downloader/database/dependency_updates.json | python3 -m json.tool
```

### Monitor Updates in Logs

```bash
tail -f /opt/media-downloader/logs/*.log | grep -i "dependency\|update"
```

## Troubleshooting

**Updates not running:**
- Check `dependency_updates.enabled` is `true`
- Verify running in scheduler mode (not manual)
- Check last_check timestamp in state file
- Ensure 24 hours have passed since last check

**FlareSolverr update fails:**
- Check Docker is running: `docker ps`
- Check internet connection
- Check Docker Hub access: `docker pull ghcr.io/flaresolverr/flaresolverr:latest`
- Review error in logs

**Playwright update fails:**
- Check disk space: `df -h`
- Check Python environment
- Manual update: `python3 -m playwright install chromium firefox`

**yt-dlp update fails:**
- Check pip permissions
- Manual update: `pip3 install --upgrade yt-dlp`
- Check internet connection

**Too many notifications:**
- Disable per-component: `notify_on_update: false`
- Disable all notifications: `pushover.enabled: false`
- Keep enabled only for critical (FlareSolverr)

**Want to disable auto-updates:**
```json
{
  "dependency_updates": {
    "enabled": false
  }
}
```

**Want to disable specific component:**
```json
{
  "dependency_updates": {
    "components": {
      "yt_dlp": {
        "enabled": false
      }
    }
  }
}
```

## Manual Updates

If you prefer manual updates, disable auto-updates and run:

```bash
# Update FlareSolverr
docker pull ghcr.io/flaresolverr/flaresolverr:latest
docker stop flaresolverr && docker rm flaresolverr
docker run -d --name flaresolverr -p 8191:8191 -e LOG_LEVEL=info --restart unless-stopped ghcr.io/flaresolverr/flaresolverr:latest

# Update Playwright
cd /opt/media-downloader
python3 -m playwright install chromium firefox

# Update yt-dlp
pip3 install --upgrade yt-dlp
```

## Logs

Update activity logged with `[DependencyUpdater]` tag:

```
2025-10-29 03:15:00 [DependencyUpdater] [INFO] Checking for dependency updates...
2025-10-29 03:15:05 [DependencyUpdater] [INFO] Checking FlareSolverr for updates...
2025-10-29 03:15:10 [DependencyUpdater] [INFO] ✓ FlareSolverr updated and restarted successfully
2025-10-29 03:15:15 [DependencyUpdater] [INFO] Checking Playwright browsers for updates...
2025-10-29 03:15:45 [DependencyUpdater] [INFO] Playwright browsers already up to date
2025-10-29 03:15:46 [DependencyUpdater] [INFO] Checking yt-dlp for updates...
2025-10-29 03:15:50 [DependencyUpdater] [INFO] ✓ yt-dlp updated successfully
2025-10-29 03:15:51 [DependencyUpdater] [INFO] Sent update notification for: FlareSolverr, yt-dlp
```

## Benefits

✅ **Zero Maintenance** - Updates install automatically
✅ **Always Current** - Critical dependencies stay up to date
✅ **Prevents Failures** - Outdated FlareSolverr/yt-dlp cause download failures
✅ **Non-Intrusive** - Low-priority notifications, doesn't interrupt workflow
✅ **Reliable** - Handles failures gracefully, won't crash scheduler
✅ **Configurable** - Enable/disable per component or globally

## Security Considerations

**Automatic updates are safe:**
- Only updates from official sources (Docker Hub, PyPI)
- Uses official image tags (`:latest`)
- No code execution from untrusted sources
- Same update process as manual updates

**Risk Mitigation:**
- Version backups taken before major changes
- Logs all update activity
- Can disable if stability is critical
- Can rollback FlareSolverr to specific version

**Recommended for most users:**
- ✅ Enable for production (scheduler mode)
- ✅ Keeps services working when APIs change
- ✅ Minimal risk, high benefit

## Future Enhancements

- Update rollback if service fails after update
- Pinning specific versions
- Update schedule (time of day)
- Pre-update testing
- Update changelog notifications
- Critical security update alerts