media-downloader/docs/COMPREHENSIVE_CODE_REVIEW.md

# Comprehensive Code Review

**Version:** 13.13.1
**Review Date:** 2025-12-04
**Last Updated:** 2026-03-29
**Status:** Active Review Document

---

## Table of Contents

1. [Executive Summary](#executive-summary)
2. [Codebase Statistics](#codebase-statistics)
3. [Architecture Overview](#architecture-overview)
4. [Critical Issues](#critical-issues)
5. [Code Quality Analysis](#code-quality-analysis)
6. [Performance Analysis](#performance-analysis)
7. [Security Analysis](#security-analysis)
8. [Technical Debt Tracker](#technical-debt-tracker)
9. [Inconsistencies Found](#inconsistencies-found)
10. [Recommended Next Big Features](#recommended-next-big-features)
11. [Version Update Checklist](#version-update-checklist)
12. [Changelog](#changelog)

---

## Executive Summary

Media Downloader is a **production-grade, enterprise-level media archival system** with approximately 79,000 lines of code across Python backend and React/TypeScript frontend. The codebase demonstrates mature software engineering practices with recent significant improvements:

1. **~~Monolithic API file~~ RESOLVED** - API modularized into 17 routers (828 lines down from 10,465)
2. **Broad exception handling** (622 instances of `except Exception`) - gradual improvement ongoing
3. **Synchronous HTTP in async context** - blocking calls in FastAPI (partial fix with run_in_executor)
4. **Large module files** exceeding 2,000+ lines each - still needs attention

### Health Score: 8.3/10 (up from 8.2)

| Category | Score | Notes |
|----------|-------|-------|
| Functionality | 9/10 | Feature-rich, comprehensive platform support |
| Code Quality | 8/10 | API modularization complete, well-structured |
| Performance | 8.5/10 | Correlated subqueries replaced with JOINs, PostgreSQL fully operational |
| Security | 8.5/10 | 12-point security audit + SQL injection defense-in-depth + proxy domain hardening |
| Maintainability | 7/10 | Significantly improved with modular routers |
| Test Coverage | 4/10 | Limited test suite |
| Documentation | 9/10 | Excellent inline and external docs |

---

## Codebase Statistics

### Current Version: 13.13.1

| Metric | Count | Change Since 12.0.0 |
|--------|-------|-------------------|
| **Total Lines (Python)** | ~70,500 | +21,800 (+ press.py, cloud_backup.py, cloud_backup_sync.py, backfill_press.py, besteyecandy_client.py, coppermine_client.py, reddit_client.py + all previous) |
| **Total Lines (TypeScript/TSX)** | ~39,500 | +14,000 (+ Gallery page, GalleryLightbox, Press page, PressArticleModal, cloud backup dashboard widget, shuffle support + all previous) |
| **Python Modules** | 56 | +26 (+ besteyecandy_client, coppermine_client, reddit_client + all previous) |
| **API Routers** | 28 | +4 (+ press, cloud_backup) |
| **API Endpoints** | 280+ | +110 (+ press CRUD/fetch + cloud backup config/sync/status + all previous) |
| **Frontend Pages** | 39 | +16 (+ Gallery page, Press page) |
| **Frontend Components** | 16+ | +5 (+ GalleryLightbox, PressArticleModal) |
| **Frontend Contexts** | 2 | BreadcrumbContext, PrivateGalleryContext |
| **Frontend Hooks** | 3 | useBreadcrumb, usePrivateGalleryAuth, useEnabledFeatures |
| **Database Tables** | 89 | +63 (PostgreSQL migration + private_media_reddit_communities + private_media_reddit_history) |
| **Database Backend** | PostgreSQL 16 | Migrated from SQLite in v13.0.0 via runtime adapter |
| **Database Indexes** | 51+ | +26 (paid content + private gallery + reddit community + message indexes) |

### File Size Analysis (Updated)

| File | Lines | Severity | Status |
|------|-------|----------|--------|
| `web/backend/api.py` | 828 | ✅ RESOLVED | Modularized into 17 routers |
| `web/frontend/src/pages/Configuration.tsx` | 5,945 | HIGH | Component extraction needed |
| `modules/forum_downloader.py` | 4,882 | HIGH | Modular redesign needed |
| `modules/unified_database.py` | 4,479 | MEDIUM | -200 lines (unused adapters removed) |
| `modules/imginn_module.py` | 3,485 | MEDIUM | Uses unified_db directly now |
| `media-downloader.py` | 2,866 | MEDIUM | Command pattern |
| `modules/fastdl_module.py` | 2,236 | MEDIUM | Uses centralized instagram_utils |
| `modules/instagram_client_module.py` | 1,651 | ✅ NEW | Direct Instagram API client (v13.4.0) |
| `modules/paid_content/instagram_client.py` | 1,081 | ✅ NEW | Instagram API for Paid Content (v13.4.0) |
| `modules/instagram_utils.py` | 429 | ✅ NEW | Centralized Instagram functions |

### API Router Breakdown (NEW)

| Router | Lines | Purpose |
|--------|-------|---------|
| `private_gallery.py` | 6,707 | Private gallery auth, media, persons, encryption, features, async job tracking, URL import, Reddit monitor endpoints |
| `paid_content.py` | 6,255 | Paid content CRUD, services, creators, feed, notifications, messages, OnlyFans/Fansly setup, auto health checks, bulk service sync |
| `media.py` | 1,334 | Media serving, thumbnails, gallery, date editing |
| `face.py` | 985 | Face recognition endpoints |
| `discovery.py` | 952 | Source discovery, platforms |
| `video.py` | 939 | Video downloads, info |
| `review.py` | 904 | Review queue management |
| `downloads.py` | 836 | Download history, analytics |
| `scheduler.py` | 630 | Scheduler control |
| `stats.py` | 521 | Dashboard stats, monitoring |
| `config.py` | 521 | Configuration management |
| `scrapers.py` | 485 | Scraper controls, errors |
| Others | ~2,000 | Auth, health, recycle, etc. |

### Async/Sync Function Distribution

- **Async functions in API:** 190
- **Sync functions in API:** 27
- **Concern:** Many sync functions call blocking `requests` library

---

## Architecture Overview

```
┌─────────────────────────────────────────────────────────────────────┐
│                         MEDIA DOWNLOADER                             │
├─────────────────────────────────────────────────────────────────────┤
│  FRONTEND (React 18 + TypeScript + Tailwind)                        │
│  ├── 35 Pages (Dashboard, Media, Downloads, Review, Config, etc.)   │
│  ├── 12 Reusable Components                                         │
│  ├── TanStack Query for data fetching                               │
│  └── WebSocket for real-time updates                                │
├─────────────────────────────────────────────────────────────────────┤
│  API LAYER (FastAPI + Uvicorn)                                      │
│  ├── 231+ REST Endpoints                                            │
│  ├── JWT + Session Authentication                                   │
│  ├── 2FA (TOTP, Duo, Passkeys)                                      │
│  ├── Rate Limiting (slowapi)                                        │
│  └── Redis Caching                                                  │
├─────────────────────────────────────────────────────────────────────┤
│  CORE MODULES (36 Python modules)                                   │
│  ├── Platform Downloaders (Instagram x4, TikTok, Snapchat, Forums)  │
│  ├── Face Recognition (InsightFace + DeepFace + dlib fallback)      │
│  ├── Semantic Search (CLIP embeddings)                              │
│  ├── Scheduler (randomized intervals)                               │
│  └── File Management (move, hash, dedupe)                           │
├─────────────────────────────────────────────────────────────────────┤
│  DATA LAYER                                                         │
│  ├── PostgreSQL 16 (via pg_adapter runtime translation, v13.0.0)   │
│  ├── Redis (caching, sessions)                                      │
│  └── File System (organized by platform/source)                     │
└─────────────────────────────────────────────────────────────────────┘
```

---

## Critical Issues

### 1. ✅ RESOLVED: Monolithic API File

**File:** `/opt/media-downloader/web/backend/api.py`
**Lines:** 828 (down from 10,465 - 92% reduction!)
**Status:** RESOLVED as of v6.52.44

**Solution Implemented:**
```
web/backend/
├── api.py (app initialization, WebSocket only - 828 lines)
├── core/
│   ├── dependencies.py (shared state, auth)
│   ├── exceptions.py (custom exception classes)
│   └── config.py (configuration)
├── routers/
│   ├── auth.py (authentication, 2FA)
│   ├── downloads.py (download history, analytics)
│   ├── media.py (media serving, gallery)
│   ├── scheduler.py (scheduler control)
│   ├── discovery.py (source discovery)
│   ├── face.py (face recognition)
│   ├── recycle.py (recycle bin)
│   ├── review.py (review queue)
│   ├── video.py (video downloads)
│   ├── stats.py (dashboard, monitoring)
│   ├── config.py (configuration)
│   ├── scrapers.py (scraper controls, errors)
│   ├── health.py (health checks)
│   ├── notifications.py (push notifications)
│   ├── manual_import.py (manual imports)
│   ├── semantic_search.py (CLIP search)
│   └── cache.py (cache management)
└── (17 routers total)
```

**Resolution Date:** 2025-12-05
**Effort Spent:** ~8 hours

---

### 2. HIGH: Broad Exception Handling

**Count:** 622 instances of `except Exception`
**Impact:** Debugging difficulty, silent failures, error masking

**Examples Found:**
```python
# Anti-pattern (throughout codebase)
try:
    result = do_something()
except Exception as e:
    logger.error(f"Error: {e}")
    return None
```

**Recommended Fix:**
```python
# Catch specific exceptions
try:
    result = do_something()
except requests.RequestException as e:
    logger.error(f"Network error: {e}")
    raise DownloadError(f"Failed to download: {e}") from e
except ValueError as e:
    logger.error(f"Invalid data: {e}")
    raise ValidationError(f"Invalid input: {e}") from e
```

**Priority:** MEDIUM (gradual refactoring)

---

### 3. HIGH: Synchronous HTTP in Async Context

**Issue:** FastAPI is async but many endpoints use synchronous `requests` library

**Impact:**
- Blocks event loop
- Reduces concurrent request handling
- Performance bottleneck

**Current Pattern:**
```python
@app.get("/api/something")
async def get_something():
    # BLOCKING CALL in async function!
    response = requests.get("https://external-api.com/data")
    return response.json()
```

**Solution Options:**

1. **Use httpx (recommended):**
```python
import httpx

@app.get("/api/something")
async def get_something():
    async with httpx.AsyncClient() as client:
        response = await client.get("https://external-api.com/data")
        return response.json()
```

2. **Use run_in_executor for sync code:**
```python
import asyncio
from concurrent.futures import ThreadPoolExecutor

executor = ThreadPoolExecutor(max_workers=10)

@app.get("/api/something")
async def get_something():
    loop = asyncio.get_event_loop()
    response = await loop.run_in_executor(
        executor,
        lambda: requests.get("https://external-api.com/data")
    )
    return response.json()
```

**Effort:** 24-40 hours
**Priority:** HIGH (for performance-critical endpoints)

---

### 4. MEDIUM: Instagram Module Duplication

**Files Affected:**
- `modules/fastdl_module.py` (2,236 lines)
- `modules/imginn_module.py` (2,870 lines)
- `modules/toolzu_module.py` (1,025 lines)

**Duplication:** 60-70% shared code including:
- Cookie management
- FlareSolverr integration
- HTML parsing
- Download logic
- Error handling

**Solution:** Create base class:
```python
# modules/instagram/base_instagram.py
class BaseInstagramDownloader(ABC):
    def __init__(self, config, unified_db):
        self.config = config
        self.unified_db = unified_db
        self.cookie_manager = CookieManager()
        self.flaresolverr = FlareSolverrClient()

    def _get_or_create_session(self): ...
    def _parse_stories(self, html): ...
    def _download_media(self, url, save_path): ...

    @abstractmethod
    def _get_content_urls(self, username) -> List[str]:
        pass
```

**Effort:** 12-16 hours
**Priority:** MEDIUM

---

## Code Quality Analysis

### Positive Patterns Found

1. **Unified Database Layer** - Single source of truth with adapters
2. **Universal Logger** - Centralized logging with web visibility
3. **Database-First File Tracking** - 50-100x faster than filesystem scanning
4. **Face Recognition Fallback Chain** - InsightFace → DeepFace → dlib
5. **Soft Delete with Recycle Bin** - File recovery capability
6. **Comprehensive Documentation** - 560KB+ in docs folder
7. **Real-Time Updates** - WebSocket for logs and errors

### Areas Needing Improvement

| Area | Current State | Target State |
|------|---------------|--------------|
| Exception handling | 622 broad catches | Specific exception types |
| Type hints | Partial coverage | Full type annotations |
| Test coverage | ~10% estimated | 70%+ target |
| API documentation | No OpenAPI docs | Full Swagger/ReDoc |
| Database migrations | Ad-hoc ALTER TABLE | Alembic migrations |
| Code formatting | Inconsistent | Black + isort |

### Code Smells Detected

1. **God Objects:** `ForumDownloader` class does too much
2. **Long Methods:** Several methods exceed 100 lines
3. **Deep Nesting:** 5+ levels in some functions
4. **Magic Numbers:** Hardcoded timeouts, limits, paths
5. **Missing Type Hints:** Many functions lack annotations

---

## Performance Analysis

### Current Optimizations (Good)

1. **Database Indexes:** 22+ optimized indexes
2. **Connection Pooling:** PostgreSQL ThreadedConnectionPool (2-30 connections, v13.0.0)
3. **Thumbnail Caching:** SHA256-keyed cache with on-demand generation
4. **Query Performance:** Sub-10ms for indexed queries
5. **Batch Face Lookup:** N+1 query pattern fixed in 6.52.32
6. **Lazy Loading:** On-demand metadata fetching in lightbox

### Performance Bottlenecks Identified

| Issue | Location | Impact | Fix |
|-------|----------|--------|-----|
| Sync HTTP in async | api.py | High | Use httpx/aiohttp |
| No query result caching | Multiple endpoints | Medium | Add Redis caching |
| Full table scans | Some search queries | Medium | Add missing indexes |
| Large response payloads | `/api/downloads` | Low | Pagination + field selection |

### Recommended Performance Improvements

1. **Add Redis caching for expensive queries:**
```python
@app.get("/api/stats")
async def get_stats():
    cached = await redis.get("stats:global")
    if cached:
        return json.loads(cached)

    stats = compute_expensive_stats()
    await redis.setex("stats:global", 300, json.dumps(stats))
    return stats
```

2. **Implement field selection for API responses:**
```python
@app.get("/api/downloads")
async def get_downloads(fields: str = None):
    if fields:
        # Return only requested fields
        selected = fields.split(",")
        return [{k: v for k, v in d.items() if k in selected}
                for d in downloads]
```

---

## Security Analysis

### Current Security Measures (Strong)

| Feature | Implementation | Status |
|---------|----------------|--------|
| Authentication | JWT + Session cookies | ✅ |
| Password Hashing | bcrypt | ✅ |
| 2FA | TOTP + Duo + Passkeys | ✅ |
| CSRF Protection | starlette-csrf | ✅ |
| Rate Limiting | slowapi | ✅ |
| SQL Injection | safe_query_builder module | ✅ |
| Path Traversal | Validation on file paths | ✅ |

### Security Concerns

1. **Hardcoded Paths:**
```python
# Found in multiple files
base_path = Path("/opt/immich/md")  # Should be configurable
```

2. **Potential Secret Exposure:**
```python
# .env file handling - ensure proper permissions
# Check: ls -la .env should show 600 permissions
```

3. **Missing Security Headers:**
```python
# Recommend adding in middleware
response.headers["X-Content-Type-Options"] = "nosniff"
response.headers["X-Frame-Options"] = "DENY"
response.headers["X-XSS-Protection"] = "1; mode=block"
```

---

## Technical Debt Tracker

### Outstanding Technical Debt

| ID | Description | Priority | Effort | Status |
|----|-------------|----------|--------|--------|
| TD-001 | Monolithic api.py (10,465 lines) | HIGH | 24h | **In Progress** - Router framework created |
| TD-002 | Broad exception handling (622 cases) | MEDIUM | 16h | **Resolved** - Custom exceptions created |
| TD-003 | Instagram module duplication | MEDIUM | 16h | **Resolved** - Base class created |
| TD-004 | Missing database migrations | MEDIUM | 8h | Open |
| TD-005 | Sync HTTP in async context | HIGH | 32h | Open |
| TD-006 | Missing API documentation | LOW | 6h | Open |
| TD-007 | Low test coverage (~10%) | MEDIUM | 32h | Open |
| TD-008 | Hardcoded configuration values | LOW | 4h | **Resolved** - Unified config manager |
| TD-009 | Missing type hints | LOW | 16h | Open |
| TD-010 | Large frontend components | MEDIUM | 12h | Open |

### Recently Resolved

| ID | Description | Resolved In | Notes |
|----|-------------|-------------|-------|
| TD-002 | Broad exception handling | 6.52.38 | Custom exception classes in `core/exceptions.py` |
| TD-003 | Instagram module duplication | 6.52.38 | Base class in `modules/instagram/base.py` |
| TD-008 | Hardcoded configuration values | 6.52.38 | Unified config in `core/config.py` |
| TD-015 | Inconsistent date formats | 6.52.38 | ISO 8601 utilities in `core/responses.py` |
| TD-016 | Inconsistent error responses | 6.52.38 | Standardized in `core/responses.py` |
| TD-011 | N+1 query in downloads | 6.52.32 | Batch face lookup added |
| TD-012 | Thread safety in forum downloads | 6.52.32 | Context priority fix |
| TD-013 | Asyncio.run() in threads | 6.52.32 | run_coroutine_threadsafe |
| TD-014 | Missing dimensions on download | 6.52.29 | PIL/ffprobe extraction |

---

## Inconsistencies Found

### 1. ✅ RESOLVED: Logging Inconsistency

**Issue:** Mix of logging approaches (duplicate log() methods across modules)

**Solution Implemented (v11.19.2):**
- Created `modules/base_module.py` with `LoggingMixin` class
- All download modules now inherit from `LoggingMixin`
- Provides consistent `log()` method with backwards-compatible `log_callback` support
- Removed 12+ duplicate `log()` method implementations
- Updated 13 modules: instaloader, tiktok, imginn, fastdl, toolzu, coppermine, forum_downloader, download_manager, move_module, scheduler, snapchat, instagram_repost_detector

**Resolution Date:** 2025-12-26

---

### 2. Error Response Format Inconsistency

**Issue:** Different error response formats across endpoints

```python
# Format 1
raise HTTPException(status_code=400, detail="Invalid request")

# Format 2
return {"error": True, "message": "Invalid request"}

# Format 3
return JSONResponse(status_code=400, content={"error": "Invalid request"})
```

**Solution:** Standardize on HTTPException with consistent detail format

---

### 3. Date Format Inconsistency

**Issue:** Multiple date formats in database and API

```python
# Format 1: ISO
"2025-12-04T10:30:00Z"

# Format 2: Unix timestamp
1733308200

# Format 3: Custom
"2025-12-04 10:30:00"
```

**Solution:** Standardize on ISO 8601 with timezone

---

### 4. Configuration Loading Inconsistency

**Issue:** Config loaded from multiple sources

```python
# Source 1: Environment variables
os.environ.get("MEDIA_PATH")

# Source 2: .env file
load_dotenv()

# Source 3: Database settings
settings_manager.get("media_path")

# Source 4: Hardcoded defaults
DEFAULT_PATH = "/opt/immich/md"
```

**Solution:** Single configuration manager with hierarchy:
1. Environment variables (highest priority)
2. .env file
3. Database settings
4. Hardcoded defaults (lowest priority)

---

## Recommended Next Big Features

Based on codebase analysis and user impact potential:

### Tier 1: High Impact, High Value

#### 1. **Webhook Integration System** ⭐ RECOMMENDED
**Effort:** 6-8 hours | **Value:** HIGH

Why this feature:
- Zero external dependencies
- Enables Discord, Slack, Home Assistant integration
- Automation workflows without code changes
- Natural extension of existing notification system

```python
# Implementation sketch
class WebhookManager:
    events = ['download_complete', 'download_error', 'face_matched', 'new_content']

    async def fire_webhook(self, event: str, data: dict):
        webhooks = await self.get_webhooks_for_event(event)
        for webhook in webhooks:
            await self.send_with_retry(webhook, event, data)
```

---

#### 2. **Duplicate Management Dashboard** ⭐ RECOMMENDED
**Effort:** 10-12 hours | **Value:** HIGH

Why this feature:
- File hash infrastructure already exists
- Users manually hunting duplicates today
- Could reclaim significant storage space
- Visual side-by-side comparison
- "Keep best quality" automation

```
┌─────────────────────────────────────────────────────────────┐
│ Duplicates Dashboard                            230 GB saved │
├─────────────────────────────────────────────────────────────┤
│ ┌─────────────────────┬─────────────────────┐               │
│ │ Original            │ Duplicate           │               │
│ │ [Image Preview]     │ [Image Preview]     │               │
│ │ 1920x1080 / 2.5 MB  │ 1280x720 / 1.8 MB   │               │
│ │ Instagram/user1     │ FastDL/user1        │               │
│ │ [✓ Keep]            │ [Delete]            │               │
│ └─────────────────────┴─────────────────────┘               │
└─────────────────────────────────────────────────────────────┘
```

---

#### 3. **Mobile-First PWA**
**Effort:** 4-6 hours | **Value:** HIGH

Why this feature:
- Installable app experience
- Offline mode capability
- Push notifications
- Already using React, easy to add
- No app store approval needed

---

### Tier 2: Medium Impact, Medium Value

| Feature | Effort | Value | Notes |
|---------|--------|-------|-------|
| Background Job Queue (Celery) | 12-16h | HIGH | Heavy ops in background |
| API Modularization | 16-24h | MEDIUM | Enables better testing |
| Prometheus Metrics | 6-8h | MEDIUM | Grafana dashboards |
| S3/Object Storage | 6-8h | MEDIUM | Cloud storage option |
| Plugin System | 16-24h | HIGH | User extensibility |

### Tier 3: Future Considerations

| Feature | Effort | Notes |
|---------|--------|-------|
| RBAC (Role-Based Access) | 12-16h | Multi-user environments |
| AI Auto-Tagging | 16-24h | ML-based categorization |
| Video Transcoding | 12-16h | Format conversion |
| Full-Text Search | 8-12h | Elasticsearch/Meilisearch |

---

## Version Update Checklist

Use this checklist when updating to a new version:

### Pre-Update
- [ ] Review CHANGELOG.md for breaking changes
- [ ] Backup database: `cp database/media_downloader.db database/backup_$(date +%Y%m%d).db`
- [ ] Note current version: `cat VERSION`
- [ ] Check disk space: `df -h /opt/media-downloader`

### During Update
- [ ] Stop services: `systemctl stop media-downloader`
- [ ] Pull new code or extract release
- [ ] Run database migrations if any
- [ ] Update dependencies: `pip install -r requirements.txt`
- [ ] Rebuild frontend: `cd web/frontend && npm run build`

### Post-Update
- [ ] Start services: `systemctl start media-downloader`
- [ ] Verify API health: `curl http://localhost:8000/api/health`
- [ ] Check logs for errors: `tail -f logs/$(date +%Y-%m-%d).log`
- [ ] Update this document's version number
- [ ] Test critical functionality

### Update This Review Document
- [ ] Update version number at top
- [ ] Update codebase statistics
- [ ] Add any new issues found
- [ ] Mark resolved issues
- [ ] Update recommendations if needed

---

## Changelog

### Review Document Updates

| Date | Version Reviewed | Changes |
|------|-----------------|---------|
| 2026-03-29 | 13.13.1 | **Pinned Post Highlighting, Mobile Header Layout, Feed Navigation Fixes** — Patch release. PostCard component in `Feed.tsx` adds conditional amber background for pinned posts: `bg-amber-50/60 dark:bg-amber-900/15` (unselected) and `bg-amber-100/80 dark:bg-amber-900/30` (selected), replacing the default blue `bg-primary/10` selection for pinned items. New `highlighted` boolean prop added to `PostDetailView.tsx` interface — swaps root div `bg-card` for `bg-amber-50/60 dark:bg-amber-900/15` and sticky header `bg-card` for `bg-amber-50/80 dark:bg-amber-900/25`. Passed as `highlighted={true}` for mobile social view pinned posts, `highlighted={selectedPost.is_pinned === 1}` for desktop social detail panel and mobile post detail modal. Mobile post header layout refactored: subtitle row split into three divs — (1) `@username` + platform badge + pinned indicator (visible always), (2) date-only row (`sm:hidden`), (3) desktop continuation with `· date · Pinned` (`hidden sm:flex`). Feed navigation fixes: new `useEffect` watches `pinnedCollapsed` — when true and `selectedPost.is_pinned === 1`, auto-selects `regularPosts[0]`. Keyboard navigation `handleKeyDown` now uses `pinnedCollapsed ? regularPosts : posts` for the navigation array, preventing arrow keys from selecting hidden pinned posts. Added `regularPosts` and `pinnedCollapsed` to useEffect dependency array. Files modified: Feed.tsx, PostDetailView.tsx. |
| 2026-03-29 | 13.13.0 | **Instagram Authenticated API Toggle, Cookie Management UI, Creator Card Badges** — Minor release. New per-creator `use_authenticated_api` toggle (INTEGER DEFAULT 0 column on `paid_content_creators`): when enabled, Instagram sync in `imginn_adapter.py` `_fetch_all_posts()` uses browser cookies as primary fetch method via `_fetch_via_browser_cookies()`, with automatic fallback to unauthenticated GraphQL if cookies fail. Return type changed from `List[Dict]` to `Optional[List[Dict]]` — `None` = API failed (fall back), `[]` = API worked but nothing new (no fallback). `_notify_cookie_expired()` method on 401: updates `scrapers.last_test_status='failed'` and broadcasts `cookie_health_alert` via `_broadcast_cookie_alert()` from health.py. `scraper.py` passes `use_authenticated_api=bool(creator.get('use_authenticated_api'))` to `get_posts()`. Backend: `use_authenticated_api` added to `UpdateCreatorRequest` model and boolean conversion loop in `paid_content.py`, added to `ALLOWED_CREATOR_COLUMNS` in `db_adapter.py`, added to migration list in `unified_database.py`. Frontend: `use_authenticated_api: number` added to `PaidContentCreator` interface and `updatePaidContentCreator()` params in `api.ts`. `CreatorSettingsModal` in `Creators.tsx` adds toggle (Instagram-only, `creator.platform === 'instagram'`). Creator card badges: green Shield + "Authenticated" badge when `use_authenticated_api` is truthy, purple Filter + "Filtered" badge (with hover tooltip showing `@usernames`) when `filter_tagged_users` JSON array is non-empty. Scrapers page: `instagram_browser` scraper renamed to "Instagram (Authenticated)" in DB. `Scrapers.tsx` adds `api` to `typeOrder` array, `typeLabels` ("Authenticated APIs"), and `TypeBadge` styles/labels (emerald). `mds` service manager script: added `backup` → `cloud-backup-sync` and `backupui` → `backup-central` to `SERVICE_MAP` and `ALL_ALIASES`. Files modified: imginn_adapter.py, scraper.py, paid_content.py, db_adapter.py, unified_database.py, api.ts, Creators.tsx, Scrapers.tsx, scripts/mds. |
| 2026-03-28 | 13.12.0 | **Sync Service Button on Creators Page** — Minor release. New `POST /api/paid-content/creators/sync-service/{service_id}` endpoint in `paid_content.py` fetches all enabled creators for a given service_id from database, queues sequential background sync via new `_sync_service_background()` helper (loops through creator IDs calling `scraper.sync_creator()` with per-creator error handling), returns `{status, count, creators}`. Rate limited to 5/minute. Frontend: new `syncPaidContentService(serviceId)` API method in `api.ts` with `syncService` namespace entry. `ServiceGroupedCreators` component in `Creators.tsx` gains `onSyncService` and `syncingServices` props. Multi-service collapsible headers now include a "Sync All" button (RefreshCw icon, text hidden on mobile) with `e.stopPropagation()` to prevent toggling the collapsible. Single-service view adds a header bar with service name, count, and sync button. Button disabled and shows spinning icon when `syncingServices` set contains the service ID or any creator in the service is in `syncingIds`. Parent component adds `syncServiceMutation` (useMutation) with success notification showing queued count and service name. Files modified: paid_content.py, api.ts, Creators.tsx. |
| 2026-03-28 | 13.11.2 | **Instagram User ID Fix, Unviewed Count Fix, Bundle Sidebar Default** — Patch release. Instagram `_fetch_user_id()` rewritten in `instagram_client_module.py`: new `_fetch_user_id_from_html()` method is now primary — scrapes `"user_id":"NNNNN"` from profile page HTML using unauthenticated GET request, bypassing 401 errors caused by 77 stale browser cookies with expired `sessionid` that made `_has_valid_session()` return True. `web_profile_info` API kept as fallback. User ID cache persistence fix: `self.unified_db` was being set to `None` when `use_database=False`, preventing `_save_user_id_cache()` from writing to the PostgreSQL `settings` table. Fixed by keeping `unified_db` reference regardless of `use_database` flag. `get_unviewed_posts_count()` in `db_adapter.py` now joins `paid_content_creators` and applies the same `filter_tagged_users` clause used by `get_posts()` and `get_posts_count()` — previously counted all unviewed posts with attachments regardless of per-creator tag filters, causing banner to show 12 when only 5 were visible (7 hidden posts from dimitrishair filter `["nicolescherzinger"]` and makkaroo filter `["kyliejenner"]` had no matching tagged users). Bundle sidebar in `BundleLightbox.tsx` and `PrivateGalleryBundleLightbox.tsx` now defaults to collapsed (`useState(false)`) on all devices instead of auto-opening on desktop. `PrivateGalleryBundleLightbox.tsx` auto-hide effect changed from `setSidebarOpen(!isCurrentVideo && !isMobile)` to only closing for videos, preventing sidebar from re-opening when navigating to images. Files modified: instagram_client_module.py, db_adapter.py, BundleLightbox.tsx, PrivateGalleryBundleLightbox.tsx. |
| 2026-03-27 | 13.11.1 | **Instagram GraphQL Post Fetching, Scan Progress, Feed Page Size** — Patch release. Instagram post sync switched from ImgInn API to direct Instagram GraphQL API (`OwnerToTimelineMediaLoggedOutQuery`) in `imginn_adapter.py` `_fetch_all_posts()`. ImgInn had a server-side 480-post limit (40 pages × 12 items) regardless of client (curl_cffi or FlareSolverr). New flow: `_fetch_all_posts()` creates an `InstagramClientDownloader` instance, calls `fetch_posts()` which paginates GraphQL at 50 posts/page with 3-5s delay. Raw v1 nodes converted to ImgInn-style items via `_ig_node_to_item()` with `_from_ig_api=True` flag. Phase 2 fast-path: items with `_from_ig_api=True` skip ImgInn detail page + tag fetching entirely (already have full-res URLs + usertags from GraphQL). `_best_media_url()` static method extracts highest quality URL from v1 media nodes. Real-time scan progress: `progress_callback(count)` called after each GraphQL page in `fetch_posts()` (line 617), passed from `_fetch_all_posts()` via `_scan_progress` wrapper. UI shows "Fetched N posts..." during scanning phase instead of static "Scanning profile". GraphQL delay increased from `random.uniform(2.0, 4.0)` to `random.uniform(3.0, 5.0)` to avoid HTTP 572 rate limiting on large accounts (tested: 5000+ posts with 3s delay succeeds, <3s hits 572 at ~1100 posts). `max_posts` increased from 5,000 to 15,000 in `instagram_adapter.py`. Update syncs remain efficient: `since_date` → small `days_back` → tight date cutoff, `consecutive_old >= 5` stops pagination after 1-2 pages for recent syncs. Feed.tsx `limit` increased from 30 to 50 for faster mobile initial load. Note: accounts blocking logged-out GraphQL ("Invalid owner id") fall back to authenticated API which requires valid session cookies. Files modified: imginn_adapter.py, instagram_adapter.py, instagram_client_module.py, Feed.tsx. |
| 2026-03-26 | 13.11.0 | **Media Gallery Page, GalleryLightbox, Server-Side Shuffle, Timeline Jump Fix** — Minor release. New `/gallery` page replacing `/media` with Immich-style justified thumbnail layout (`buildJustifiedRows()` targeting ~180px row height), daily date grouping with smart labels (Today, Yesterday, weekday names, abbreviated dates), timeline scrubber navigation, slideshow mode, and infinite scroll via `useInfiniteQuery`. New `GalleryLightbox.tsx` (~1000 lines) — simplified BundleLightbox without paid-content features, with slideshow auto-advance, embedded metadata display (ffprobe/exiftool via `/api/media/embedded-metadata` — title, description, artist, source URL), zoom/pan, swipe navigation, keyboard controls. Server-side shuffle for media gallery: new `shuffle` and `shuffle_seed` query params on `/api/media/gallery`, PostgreSQL `ORDER BY md5(fi.id::text \|\| seed::text)` for deterministic randomization, separate `useInfiniteQuery` for shuffled results, shuffle button (cyan when active) in lightbox slideshow toolbar. Timeline scrubber date jump fix for both galleries: now sets both `date_from` AND `date_to` (last day of target month) to bound the query, preventing DESC sort from returning newest items when jumping to an old month. `TimelineScrubber.tsx` fixed all `=== activeKey` comparisons to `activeKey.startsWith(key)` for daily section key compatibility (keys are `YYYY-MM-DD`, scrubber uses `YYYY-MM`). Bug fixes: added `autoPlayVideo={true}` to paid content gallery BundleLightbox props; feed unviewed count query adds `EXISTS (SELECT 1 FROM paid_content_attachments)` to exclude empty posts; media gallery date-range endpoint changed `fi.location = 'media'` to `'final'`; removed non-existent `fi.duration` from gallery queries. Route changes: `/media` redirects to `/gallery`, nav/breadcrumbs/feature toggles updated. Files created: Gallery.tsx, GalleryLightbox.tsx. Files modified: media.py, GalleryTimeline.tsx, TimelineScrubber.tsx, App.tsx, api.ts, breadcrumbConfig.ts, Configuration.tsx, Features.tsx, private_gallery.py, db_adapter.py. |
| 2026-03-26 | 13.10.0 | **Justified Gallery Layout, Timeline Navigation, Service Health Check Fixes** — Minor release. Paid content gallery now uses justified thumbnail rows (Immich-style) where images display at their natural aspect ratios instead of uniform squares. `buildJustifiedRows()` function packs items into rows targeting ~180px height, scaling widths to fill container exactly. New `JustifiedSection` React.memo component isolates per-section rendering. Timeline scrubber `onJumpToDate` callback: clicking a month not yet loaded via infinite scroll triggers `date_from` API query parameter, resetting infinite scroll to load from that month. Dismissible "Showing from {Month Year}" chip with X button clears filter. Timeline layout changed from centered `top-1/2 -translate-y-1/2 max-h-[70vh]` to full-height `top-16 bottom-4`. Three performance optimizations: (1) callback ref (`useCallback` ref) replaces `useRef` + `useLayoutEffect` for container width measurement — fires synchronously when conditionally-rendered div mounts, eliminating blank first frame; (2) `sectionRowsMap` useMemo pre-computes all justified rows keyed on `[sections, containerWidth]` instead of computing per-section inside render loop; (3) ResizeObserver deadband ignores sub-pixel width changes (threshold >1px). Backend: added dedicated health checks for coppermine (HTTP client availability), besteyecandy (site reachability with Cloudflare 403 acceptance), and reddit (gallery-dl binary check) in `_check_single_service_health()` — these were falling through to generic `PaidContentAPIClient.check_health()` which tried `{base_url}/api/v1/creators` (Coomer/Kemono pattern), producing false down/degraded status. Files modified: TimelineScrubber.tsx, GalleryTimeline.tsx, paid_content.py. |
| 2026-03-24 | 13.9.0 | **Press Monitoring, Cloud Backup, BestEyeCandy/Coppermine/Reddit Scrapers, Feed Shuffle** — Minor release. New press monitoring system: celebrity news tracking via Google News RSS + GDELT with full article extraction (readability + BeautifulSoup), local image caching with proxy serving, Pushover notifications, read/unread tracking, filtering by celebrity/domain/read status, full-text search. Dedicated Press page (Press.tsx, 422 lines) with stats dashboard, article cards, and PressArticleModal (292 lines) with HTML sanitization. New cloud backup system: rclone-based backup to B2/S3 with AES256 encryption, cloud_backup.py router (1526 lines), cloud_backup_sync.py daemon (1194 lines) with inotify file watching, per-directory cooldown sync, daily full backups at 3AM with Immich + Media Downloader PostgreSQL dumps, live progress tracking. Dashboard widget shows real-time sync progress with animated gradient border, transfer speed/ETA, phase detection. Three new paid content scrapers: BestEyeCandy client (469 lines) with cookie auth and URL pattern discovery for full-res images; Coppermine client for nested PHP gallery structure (categories > subcategories > albums > photos) with thumbnail-to-fullres URL derivation; Reddit client via gallery-dl integration with subreddit metadata API. Backend-driven shuffle for paid content slideshow: deterministic seed-based randomization in db_adapter.py (random.Random(seed) shuffles all matching IDs, then paginates), frontend mulberry32 seeded PRNG for cross-post attachment mixing. BundleLightbox.tsx gains onShuffleChange/isShuffled props for parent-managed shuffle. Feed page size increased 10→30 fixing mobile feed empty state (20 pinned posts filled first 2 pages at old limit with collapsed pinned section). Feed error handling improved with isPending tracking, retry count 3, error UI in all 4 view modes. Feed setSearchParams skipped on initial mount to prevent startTransition interference. Paid content feed endpoint limit max increased 100→500. Log viewer adds cloudbackup, cloudflare.fastdl/imginn/imginn-stories, taskcheckpoint components. Creators page adds hqcelebcorner/picturepub to SERVICE_URLS. New scripts: backfill_press.py (592 lines) for historical article bulk loading with sliding windows, cloud_backup_sync.py daemon. Configuration.tsx adds cloud_backup and press tabs. App.tsx adds Press route. Breadcrumb config adds /press. Cleaned __pycache__ directories. Files created: press.py, cloud_backup.py, besteyecandy_client.py, coppermine_client.py, reddit_client.py, backfill_press.py, cloud_backup_sync.py, Press.tsx, PressArticleModal.tsx. Files modified: db_adapter.py, scraper.py, utils.py, paid_content.py, __init__.py, Feed.tsx, BundleLightbox.tsx, api.ts, Dashboard.tsx, Configuration.tsx, App.tsx, Logs.tsx, breadcrumbConfig.ts, Creators.tsx. |
| 2026-03-19 | 13.8.1 | **Audio Player, Push Notification Accuracy, Reddit Reliability, Health Check Fixes** — Patch release. Inline audio player in PostDetailView (native HTML5 audio with filename, file size, playback controls) and mobile Feed PostCard (compact player with stopPropagation preventing post navigation on tap). Audio attachments excluded from thumbnail grid and BundleLightbox. Push notification audio fix: content_type from scraper metadata checked before file extension, fixing .mp4 audio files misidentified as video; added 🎵 audio category to titles and pushover PLURALS/CONTENT_ICONS; fixed 3 hardcoded `content_type: 'video'` in downloaded_file_info. Health check fixes: HQCelebCorner/PicturePub accept HTTP 200/403, Snapchat checks story.snapchat.com reachability. Reddit community monitor: switched from OAuth to REST API mode (`extractor.reddit.api=rest`) with `--range 1-200` limit; fixed cookie file directory creation. Code quality: fixed function name shadowing in celebrity.py and video_queue.py; fixed backup profile stale paths. Files modified: scraper.py, pushover_notifier.py, reddit_community_monitor.py, paid_content.py, celebrity.py, video_queue.py, PostDetailView.tsx, Feed.tsx, install.sh, add-backup-profile.sh. |
| 2026-03-18 | 13.8.0 | **XenForo Forum Support, Snapchat/TikTok HTTP Clients, Cookie Health Monitoring, Story Dedup** — Minor release. New XenForo forum client (xenforo_forum_client.py, hqcelebcorner_client.py) for celebrity image forums with external host resolution, FlareSolverr bypass, CSRF handling. Snapchat HTTP client (snapchat_client.py) with curl_cffi replacing Playwright, __NEXT_DATA__ JSON extraction, EXIF metadata. TikTok hybrid client (tiktok_client.py) with yt-dlp listing + gallery-dl downloading, lazy cookie loading. Instagram direct API module (fastdl_instagram_client.py) with GraphQL + REST via curl_cffi. Cookie health monitoring with per-platform toggles, 30-min cooldown, WebSocket + Pushover alerts. ImgInn adapter: two-phase discovery, FlareSolverr fallback, 1440p+ CDN URLs, NordVPN proxy rotation. Scheduler checkpoint system for crash recovery. Bellazon forum client, Soundgasm audio client, paid content URL parser expansion. Bug fixes: FastDL pk mapping, hash-based story dedup, recycle bin metadata, square video display, failed downloads creator_db_id. Files modified: 20+ files across modules/paid_content/, web/backend/routers/, web/frontend/src/. |
| 2026-03-12 | 13.7.0 | **Instagram Sync Settings, Appearance Role Combining, Tagged User Filter** — Minor release. Per-creator Instagram sync settings: new `sync_posts`, `sync_stories`, `sync_highlights` columns on `paid_content_creators` table with toggle UI in creator card menu (Instagram-only). Conditional sync in `scraper.py` wraps stories/highlights/posts fetching in `creator.get('sync_X', 1)` checks. Tagged user filter: new `filter_tagged_users` column (JSON array of usernames), new `GET /creators/{id}/tagged-users` endpoint returning distinct tagged usernames with counts, LIKE-based SQL filter in `get_posts()`, `get_posts_count()`, and `get_media_count()`. CreatorSettingsModal component in Creators.tsx with toggles and multi-select tagged user list. All paid content Instagram operations now route exclusively through `ImgInnAPIAdapter` — removed all `UnifiedInstagramAdapter` usage from scraper.py (was routing through real Instagram API via `instagram_client`). curl_cffi impersonation pinned from auto-resolving `chrome` (→chrome142, unsupported) to explicit `chrome136`. Appearances: upcoming endpoint now uses ShowStats CTE with `GROUP_CONCAT(DISTINCT credit_type)` and `ROW_NUMBER()` grouping matching aired endpoint. New `_enrich_with_roles()` helper fetches structured role data (credit_type, character_name, job_title) for multi-role appearances. Show episodes endpoint groups by season+episode+date, combining credit types. `COUNT(*)` → `COUNT(DISTINCT episode_key)` across all 4 endpoints. AppearanceDetailModal updated for multi-role display with consistent `creditTypeConfig` badge colors. Dashboard UpcomingAppearancesCard shows credit type badges. Notification INSERTs now set `is_sent=1, sent_at=CURRENT_TIMESTAMP`. FilterPopover max-height increased 200→320px. Removed unused `instagram_unified_adapter.py` (155 lines). Cleaned __pycache__ directories. Files modified: scraper.py, imginn_api_module.py, db_adapter.py, appearances.py, paid_content.py, unified_database.py, api.ts, Creators.tsx, AppearanceDetailModal.tsx, UpcomingAppearancesCard.tsx, FilterPopover.tsx. |
| 2026-03-07 | 13.6.0 | **ImgInn Paid Content Adapter, Reddit Timezone Fix, Lightbox Improvements, Cleanup** — Minor release. New `imginn_adapter.py` replaces Instagram API-based adapter with ImgInn-based scraping for Paid Content — no Instagram credentials needed, maps ImgInn API data to existing Post/Attachment models, handles posts/stories/reels with CDN URLs, built-in rate limiting and Cloudflare bypass. ImgInn API optimizations: FlareSolverr integration for Cloudflare bypass, OCR disabling for faster scraping, date pre-filtering, profile resolution fixes. Reddit private gallery timezone fix: gallery-dl UTC dates now properly converted to local time using `datetime.timezone`; created_at uses import time instead of post date. Bulk-fixed 556 existing reddit posts and 1366 media entries with incorrect UTC dates via decrypt→convert→re-encrypt cycle. Lightbox fixes: slideshow back arrow not showing (parent-managed shuffle mode), paid content bundle sidebar z-index fix (z-20 over z-10 overlay), bundle sidebar auto-collapse on slideshow, hidden for single-image posts, expand/collapse floating button, stationary slideshow controls. Dashboard platform labels improved. Cleanup: removed 36 stale scripts/utilities (one-off backfills, Fansly maintenance, test scripts, completed migration scripts), legacy SQLite database file, archive directory, __pycache__ directories. |
| 2026-03-05 | 13.5.1 | **Landscape Lightbox, Min Resolution Filter, TypeScript Fixes, Cleanup** — Patch release. Ported landscape phone detection and layout from BundleLightbox.tsx to PrivateGalleryBundleLightbox.tsx: touch-based device detection with `isLandscape` state (touch + height<500px), fixed positioning for landscape containers (100dvh), adjusted control positions/visibility, landscape-specific navigation arrows (compact rounded with bg-black/40), sidebar-aware image container (`left: 6rem`), landscape video sizing. Added per-person-group `min_resolution` setting: SQL filter `(file_type != 'image' OR (width >= ? AND height >= ?))` shows all videos but only images meeting threshold; gallery filter dropdown (300/500/720/1080px+ options) with active filter chip; import skip for low-resolution images with skipped count tracking. Fixed TypeScript errors: added `min_resolution?: number` to `getPersonGroups()` and `getPersonGroup()` return types in api.ts, removed unused `editMinRes` state from Config.tsx, fixed `task.next_run` null safety in Scheduler.tsx. Cleaned up stale files: removed migration_backup.json (682KB), media_downloader.db.old (308KB), all __pycache__ directories (~5MB). Files modified: PrivateGalleryBundleLightbox.tsx, private_gallery.py, Gallery.tsx, Config.tsx, api.ts, Scheduler.tsx. |
| 2026-03-04 | 13.5.0 | **ImgInn API Module, Instagram Auth Circuit Breaker, Private Gallery Bridge, FastDL Consolidation** — Minor release. New `imginn_api_module.py` (~600 lines) scraper using ImgInn's internal API for posts, stories, reels, and tagged content with subprocess wrapper. New `instagram_rate_limiter.py` (~160 lines) providing shared 180 calls/hour rolling window rate limiter with progressive slowdown, pause file persistence, and cross-module mutex. New `scraper_gallery_bridge.py` (~650 lines) for importing scraper downloads into private gallery with id-based incremental filtering via `last_imported_file_id` column on `private_media_scraper_accounts`. New `paid_content/fastdl_instagram_client.py` replacing unstable curl_cffi direct API with FastDL browser-based approach for Paid Content Instagram. New `fastdl_subprocess_wrapper.py` and `imginn_api_subprocess_wrapper.py` for subprocess isolation. Instagram auth failure circuit breaker: `instagram_client_module.py` sets `auth_failed` flag on 401/403, propagated through subprocess JSON results via `base_subprocess_wrapper.py`; `media-downloader.py` `download_instagram_client()` detects flag, skips stories/tagged for remaining users, continues posts/reels, updates scraper test status to 'failed' in database, sends Pushover notification. Cookie health banner updated: Instagram scrapers now shown in `/health/cookies` when test status is 'failed' (previously fully excluded from display); auto-clears when cookies re-saved. Dashboard "New In" cards fixed: all queries changed from `ORDER BY COALESCE(d.download_date, fi.created_date) DESC` to `ORDER BY fi.id DESC`; dismiss tracking changed from timestamp-based to ID-based (`number | null`). Private gallery bridge added to `download_instagram_client()` (was only wired for fastdl/imginn_api). `get_available_accounts()` now collects `phrase_search.usernames` from all Instagram scrapers. Instagram client TLS: changed from `chrome131` to `edge101` impersonation with matching Edge User-Agent. Subprocess timeout increased 120→300s. FastDL consolidated from 3 browser instances to 1 per user. FastDL scrolling tuned (200px steps, 3s API wait, 5 stale rounds). Cleanup: removed 5 debug scripts, debug screenshot directory, dead `_load_instaloader_session()`, unused imports (`re`, `Set`, `datetime`). Files modified: imginn_api_module.py (NEW), instagram_rate_limiter.py (NEW), scraper_gallery_bridge.py (NEW), paid_content/fastdl_instagram_client.py (NEW), fastdl_subprocess_wrapper.py (NEW), imginn_api_subprocess_wrapper.py (NEW), media-downloader.py, instagram_client_module.py, fastdl_module.py, move_module.py, unified_database.py, scheduler.py, base_subprocess_wrapper.py, dashboard.py, health.py, paid_content.py, platforms.py, private_gallery.py, scrapers.py, config.py (router), api.ts, Dashboard.tsx, Configuration.tsx, Platforms.tsx, Scheduler.tsx, Feed.tsx, Creators.tsx, Config.tsx (private-gallery). |
| 2026-02-26 | 13.4.1 | **Private Gallery Unread Fixes, Paid Content Unviewed/Messages Banners** — Patch release. Fixed three Private Gallery unread bugs: (1) `/media` endpoint count/shuffle/fast-path queries missing `LEFT JOIN private_media_posts` causing SQL errors when `unread_only=true`, (2) `markSeenMutation.onSuccess` used `invalidateQueries` instead of `resetQueries` leaving stale `useInfiniteQuery` pagination, (3) upload modal didn't pass `post_id` through `onSuccess` callback so new post wasn't auto-selected. Added Paid Content unviewed posts system: `get_unviewed_posts_count()` and `mark_all_posts_viewed()` db_adapter methods, `/unviewed-count` and `/mark-all-viewed` endpoints, Feed.tsx banner with "View unviewed"/"Mark all viewed" buttons using blue gradient+border style. Added Paid Content unread messages system: `get_total_unread_messages_count()` and `mark_all_messages_read()` db_adapter methods, `/messages/unread-count` and `/messages/mark-all-read` endpoints (placed before `/{creator_id}` path param route), violet gradient banners on Dashboard.tsx and Feed.tsx with "View messages" RouterLink and "Mark all read" button. Private Gallery unread banners restyled from solid to gradient+border matching dashboard pattern. CLAUDE.md updated with PostgreSQL syntax note for direct psql commands. Files modified: private_gallery.py, Gallery.tsx, PrivateGalleryUploadModal.tsx, paid_content.py, db_adapter.py, api.ts, Feed.tsx, Dashboard.tsx, CLAUDE.md. |
| 2026-02-23 | 13.4.0 | **Instagram Client API, Module Enable/Disable, Image Info Bar, Snapchat Client** - Minor release. New `instagram_client_module.py` (1,651 lines) provides direct Instagram GraphQL/REST API downloads using `curl_cffi` with browser TLS fingerprinting — 10-20x faster than ImgInn scraping. Posts via public GraphQL (`OwnerToTimelineMediaLoggedOutQuery_connection`), stories/reels/tagged via authenticated REST API with session cookies. File naming matches ImgInn format (`{profile}_{YYYYMMDD_HHMMSS}_{media_id}{ext}`) for dedup compatibility. Separate `paid_content/instagram_client.py` (1,081 lines) for Paid Content system. New `snapchat_client_module.py` with same direct API architecture. Both integrated into main downloader (`media-downloader.py`), scheduler, and platform router with subprocess wrappers. New module enable/disable system: platform-level `enabled` flags in config, scheduler filters by `config.get('<platform>', {}).get('enabled')`, health router `_is_scraper_module_enabled()` checks module status, frontend Scheduler.tsx filters `enabledTasks`. Image info bar added to `EnhancedLightbox.tsx` — images now show source, file size, date, resolution (emerald highlight), and platform below the image (matching video info bar, minus Duration). New `CookieHealthBanner.tsx` component for cookie expiration warnings via WebSocket. Major cleanup: removed 7.5 GB `backup_sqlite_migration/` directory, empty .db placeholders, 365 debug screenshots (19 MB), stale test logs, __pycache__ directories. Files modified: instagram_client_module.py (NEW), snapchat_client_module.py (NEW), paid_content/instagram_client.py (NEW), instagram_client_subprocess_wrapper.py (NEW), snapchat_client_subprocess_wrapper.py (NEW), CookieHealthBanner.tsx (NEW), media-downloader.py, scheduler.py, platforms.py, scrapers.py, health.py, EnhancedLightbox.tsx, Dashboard.tsx, Scheduler.tsx, Logs.tsx, Configuration.tsx, api.ts, Settings.tsx, Feed.tsx, Creators.tsx, Gallery.tsx, move_module.py. |
| 2026-02-21 | 13.3.0 | **Reddit Community Monitor, Private Gallery Fixes, Log & Scheduler Standardization** - Minor release. New `reddit_community_monitor.py` module (~1,050 lines) enables automated Reddit community monitoring for Private Gallery. Users map subreddits to persons via Config tab; system uses gallery-dl to download new posts (including imgur/redgifs attachments), encrypts files with AES-256-GCM, and imports into gallery with 'reddit' tag. Two-layer duplicate detection: post-level tracking via `private_media_reddit_history` table (survives post deletion) and file-level SHA-256 hash checking. Crypto key file approach (`/opt/immich/private/.reddit_monitor_key`) enables scheduler process to encrypt independently. Real-time progress tracking with phase updates (downloading/processing/importing/encrypting). Batch subreddit input (comma-separated). Auto-cleanup of empty reddit-tagged posts after dedup deletion. Fixed Content-Disposition headers for filenames with special characters (emojis, smart quotes, double quotes) using RFC 5987 `filename*=UTF-8''` encoding with `quote(filename, safe='')`. Fixed `total_media_found` showing accumulated count instead of live COUNT(*). Standardized scheduler task ID to `reddit_monitor`. Added `filesrouter`/`mediaidentifier`/`redditmonitor` to log viewer with display names and default components. Dashboard and Scheduler pages updated with Reddit Monitor platform mapping. DB: 2 new tables (`private_media_reddit_communities`, `private_media_reddit_history`), 3 new indexes. Files modified: reddit_community_monitor.py (NEW), private_gallery_crypto.py, private_gallery.py, unified_database.py, scheduler.py, api.ts, Config.tsx, Logs.tsx, Scheduler.tsx, Dashboard.tsx. |
| 2026-02-16 | 13.2.0 | **Profile Image Caching, FastDL Stories, TikTok Bio Fix, Log Viewer Fix** - Minor release. New `_cache_profile_image()` method in scraper.py downloads creator avatars/banners locally during sync across all 11 platforms (Instagram, Coomer/Kemono, YouTube, PornHub, XHamster, TikTok, Twitch, Fansly add/sync, OnlyFans add/sync), eliminating dependency on expiring CDN URLs. Uses `curl_cffi` with browser TLS impersonation for Instagram CDN (plain requests blocked). New `/api/paid-content/cache/profile-image/{filename}` endpoint serves cached images with 7-day browser cache. Instagram stories switched from ImgInn to FastDL for real post timestamps (`published_at` from `p.media-content__meta-time`), ImgInn kept as fallback. FastDL's duplicate tracking disabled (`use_database=False`) since paid content has own deduplication. `_files_to_posts()` regex updated for both FastDL/ImgInn filename formats. TikTok bio emoji corruption fixed: replaced `.encode().decode('unicode_escape')` with `json.loads()` and added `__UNIVERSAL_DATA_FOR_REHYDRATION__` JSON parsing as primary extraction. Log viewer `get_logs` endpoint fixed to handle `YYYYMMDD_component.log` format (was only matching `YYYYMMDD_HHMMSS_component.log`). Messages.tsx proxy wrapper fixed to skip local `/api/` URLs. Instagram profile always refreshes during sync (removed stale guard). Added `curl_cffi>=0.7.0` to requirements.txt and dependency updater. Added `data/cache/profile_images/` to installer. Files modified: scraper.py, instagram_client.py, tiktok_client.py, paid_content.py, config.py (router), Messages.tsx, requirements.txt, dependency_updater.py, install.sh, update-all-versions.sh, core/config.py. |
| 2026-02-16 | 13.1.1 | **PostgreSQL Boolean Fixes, Fansly Quality Recheck Scheduler, Dashboard Pinned Posts** - Patch release. Fixed celebrity appearances ON CONFLICT error by creating partial unique indexes for TV and Movie appearance types. Fixed forum subprocess boolean handling (`1` → `TRUE` for `searches.active`, `downloaded`, `active`). Added missing `download_queue` columns to PostgreSQL (`thread_id`, `post_id`, `forum_name`, `file_hash`, `downloaded_date`). Converted `paid_content_posts.downloaded` and `forum_posts.has_images` from integer to boolean; added both to pg_adapter `_BOOLEAN_COLUMNS`. Fixed FastDL URL redirect (`/en` → `/en2`). Fixed Fansly quality recheck scheduler: `_auto_quality_recheck_background()` was only called from API single-creator sync, not from `sync_paid_content_all()`. Added `skip_pinned` parameter to feed endpoints so Dashboard recent posts excludes pinned posts. Synced 11 missing fastdl/imginn accounts from SQLite to PostgreSQL. Files modified: appearances.py, forum_downloader.py, fastdl_module.py, pg_adapter.py, db_adapter.py, paid_content.py, Dashboard.tsx, api.ts, install.sh. |
| 2026-02-15 | 13.1.0 | **PostgreSQL Compatibility Fixes, Discovery Queue, Security Hardening** - Minor release. Comprehensive fixes for 15+ PostgreSQL edge cases: `db_bootstrap.py` loads `.env` before checking `DATABASE_BACKEND` (fixes all 5 entry points silently falling back to SQLite). pg_adapter datetime parameter interval translation fixed (`%s` double-escaped to `%%s` breaking forum thread monitoring). `download_monitor.success` converted from integer to boolean, `SUM(success)` rewritten as `SUM(CASE WHEN...)`. Analytics `DATE()` returns Python date objects (added `str()` conversion). Paid content quality recheck `text < timestamp` fixed with explicit CAST. Passkey/2FA `username` column→`user_id`, schema type integer→text. `error_tracking` migration replaced sqlite_master check with `information_schema` constraint check. Review page and media gallery correlated subqueries replaced with pre-aggregated LEFT JOINs. Recycle bin COALESCE type mismatch fixed. Forum threads GROUP BY rewritten with proper subquery JOIN. `strftime()` → `TO_CHAR()` and `datetime(col)` → `col::timestamp` translations added to pg_adapter. `fromisoformat()` now handles datetime objects from PostgreSQL. Celebrity appearances boolean migration fixed. Duplicate `/api/logs/context` route removed. 5 missing DiscoverySystem queue methods added (`get_queue_stats`, `get_pending_queue`, `add_to_queue`, `bulk_add_to_queue`, `clear_queue`). Image proxy switched to suffix-based domain matching adding Fansly/TikTok/xHamster CDN support. SQL injection defense: server-side `sort_order` and `table_alias` validation. Boolean column translation (`has_match`/`success`/`notified` = 1 → = TRUE). Systemd services include DATABASE_BACKEND env vars. Files modified: pg_adapter.py, db_bootstrap.py, downloader_monitor.py, discovery_system.py, unified_database.py, passkey_manager.py, scheduler.py, paid_content.py, Creators.tsx, downloads.py, private_gallery.py, core/utils.py, config.py, review.py, media.py, db_adapter.py, CLAUDE.md. |
| 2026-02-13 | 13.0.1 | **Dashboard Text-Only Post Placeholder Fix** - Patch. Paid Content Dashboard grey media placeholder no longer renders on text-only posts (has_attachments=0). Added `post.has_attachments === 1` condition gate. Files modified: Dashboard.tsx. |
| 2026-02-13 | 13.0.0 | **PostgreSQL Migration, Private Gallery Fix, Code Cleanup** - Major infrastructure migration. New `pg_adapter.py` (~840 lines) provides drop-in `sqlite3` replacement via `sys.modules` monkey-patching when `DATABASE_BACKEND=postgresql`. SQL translation engine with LRU cache (4096 entries) handles 27+ SQLite→PostgreSQL dialect conversions (PRAGMA→no-op, datetime()→NOW(), GROUP_CONCAT→STRING_AGG, INSERT OR REPLACE→ON CONFLICT, json_extract→jsonb, etc.). `_replace_question_marks()` converts `?` params to `%s` with proper string-literal awareness. `PgConnection`/`PgCursor` classes wrap psycopg2 with `Row` class providing full sqlite3.Row compatibility (index, name, keys, iteration). `ThreadedConnectionPool` (2-30 connections) with lazy init and double-check locking. `db_bootstrap.py` patches `sys.modules['sqlite3']` at import time in all 5 entry points. All 87 tables from 6 SQLite databases (media_downloader.db, auth.db, scheduler_state.db, thumbnails.db, media_metadata.db, easynews_monitor.db) migrated to single PostgreSQL instance. Bug fixes: `%` escaping in pg_adapter (was only escaping outside quotes, psycopg2 needs all `%` escaped), cursor leak in `commit()`/`rollback()` (cursors created but never closed), removed unused imports (datetime, psycopg2.extras), added explicit `psycopg2.errors` import. Fixed Private Gallery feature toggle 400 error (stale `/scraping-monitor` in saved config rejected by validation; now silently filters removed features). Added `_is_lock_error()` helper in unified_database.py consolidating 9 inline lock-error checks to support both SQLite and PostgreSQL errors (deadlock detected, could not obtain lock). Added psycopg2-binary to requirements.txt and dependency updater. Updated installer with PostgreSQL setup. Cleaned up 6 empty .db files, stale CSV, moved sync_pg_schema.py to archive. Files modified: pg_adapter.py (NEW), db_bootstrap.py (NEW), unified_database.py, private_gallery.py, activity_status.py, requirements.txt, dependency_updater.py, install.sh, media-downloader.py, api.py, thumbnail_cache_builder.py, generate-embeddings.py, enrich_celebrity_metadata.py, CRITICAL_RULES.md, media-cache-builder.service. |
| 2026-02-12 | 12.14.0 | **Streaming Decryption, Download Reliability, Auto-Migration** - Minor release. Generator-based streaming decryption for all encrypted videos: `decrypt_file_generator()` yields 8MB decrypted chunks for chunked files or entire content for single-shot files via `StreamingResponse`, eliminating memory spikes for multi-GB videos. `decrypt_file_range_generator()` calculates which encrypted chunks overlap a byte range, seeks to positions, decrypts only needed chunks. Updated `stream_video`, `get_file`, and `export_single` endpoints to use streaming. Auto-migration on gallery unlock: background thread converts single-shot encrypted files >50MB to chunked format via `re_encrypt_to_chunked()` with atomic rename and unique temp filenames (`secrets.token_hex(4)`). Once-per-process guard prevents duplicate migrations. `POST /migrate-chunked` endpoint for manual triggering. Download reliability: stall detection with 30s read timeout, 3 retries with HTTP Range resume from last byte. All direct downloads now multi-threaded with 5 parallel segments (removed 5MB threshold). Fixed subprocess stdin double-close: replaced `stdin.write()`/`stdin.close()` + `communicate()` with `communicate(input=...)`. Fixed duplicate filename check to verify `.enc` file exists on disk. Fixed `sync_tmdb_appearances` NoneType error in scheduler context: added `db` parameter, scheduler passes `self.unified_db`. Subprocess error→warning for FastDL/ImgInn/Toolzu/Snapchat timeouts/failures. Smart escalation for page load failures: warning normally, error after 5+ per session via `_page_load_failures` counter. Files modified: private_gallery_crypto.py, private_gallery.py, media-downloader.py, imginn_module.py, appearances.py, scheduler.py. |
| 2026-02-11 | 12.13.0 | **Security Hardening, Message Read Tracking, Easynews Fix** - Minor release. Comprehensive 12-point security audit across backend routers: fixed missing `require_admin` dependency in appearances.py, added CSRF enforcement to config/manual_import/files/video routers, corrected `get_connection(for_write=True)` on 8 write operations across celebrity.py/stats.py/dashboard.py/easynews.py/private_gallery.py/review.py, added path traversal validation to files.py serve endpoint, added rate limiting to paid_content.py sync/setup endpoints. New mark-as-read feature: IntersectionObserver on unread message bubbles (0.5 threshold) with 500ms debounced batch API calls via `POST /messages/{creator_id}/mark-read`, `mark_messages_read()` DB method, `markMessagesRead()` API client. Auto-scroll to first unread message on conversation open. Message sync now calls `_download_message_attachments()` for pending attachments after sync. Fixed Easynews search: removed strict content-type check in easynews_client.py that rejected valid JSON when Easynews API changed Content-Type from `application/json` to `text/html`. Fixed Dashboard dismiss button: added `parseAsUTC()` helper normalizing timezone-naive DB timestamps (e.g. `2026-02-11 12:25:21`) to UTC before comparing with ISO dismiss timestamps (`...Z`), fixing timezone mismatch in shouldShowCard/getVisibleItems. Health check fixes: auto-check persists timeout/error to DB, single service handles timeout, stores 'down' not 'error', first load runs checks synchronously. Files modified: appearances.py, celebrity.py, config.py, dashboard.py, easynews.py, files.py, manual_import.py, paid_content.py, private_gallery.py, review.py, stats.py, video.py, api.py, easynews_client.py, db_adapter.py, Dashboard.tsx, Messages.tsx, Creators.tsx, api.ts, App.tsx, Configuration.tsx, Login.tsx. |
| 2026-02-10 | 12.12.1 | **Health Check Reliability, Message Notifications, Sync Pagination Fix** - Patch release. Health checks no longer block async event loop: rate limiter delays disabled for health check clients via `_init_rate_limiter(min_delay=0)`, 30s per-service timeout with `asyncio.wait_for()`. Refactored 3 duplicate health check implementations into shared `_check_single_service_health()` function (~150 lines removed). Fixed OnlyFans scheduled sync paginating through ALL posts: timezone-aware vs timezone-naive datetime comparison in `get_posts()` silently failed in except block, now normalizes both to naive with `.replace(tzinfo=None)`. Added push notifications for new messages during paid content sync: `_send_creator_notification()` accepts `new_messages` param, shows "💬 N New Messages" title for message-only syncs at low priority (-1), includes "💬 Messages: N" line in combined download notifications. PiP menu item hidden on desktop lightboxes (both Paid Content and Private Gallery), only shown on mobile. Files modified: scraper.py, onlyfans_client.py, paid_content.py, BundleLightbox.tsx, PrivateGalleryBundleLightbox.tsx. |
| 2026-02-10 | 12.12.0 | **Messages & Chat, OnlyFans Direct Setup, Health Check Auto-Trigger** - Major feature release. New Messages/Chat page for viewing OnlyFans and Fansly creator direct messages with two-panel responsive chat UI (conversation list + chat thread), infinite scroll pagination, PPV/tip badges, and inline media lightbox. New `paid_content_messages` database table with `message_id` column on attachments. Message sync via OnlyFans `GET /chats/{user_id}/messages` and Fansly `GET /messaging/groups` + `GET /message?groupId={id}` APIs. New `Message` dataclass in models.py. DB adapter methods: `upsert_message()`, `get_conversations()`, `get_messages()`, `get_pending_message_attachments()`. OnlyFans Direct credential setup UI supporting manual cookie paste, HAR file upload, and Cookie-Editor JSON import with `POST /onlyfans-direct/verify-auth` and `POST /onlyfans-direct/sync` endpoints. Fansly Direct verify-auth and sync endpoints. Picture-in-Picture support in both BundleLightbox.tsx and PrivateGalleryBundleLightbox.tsx with Safari `webkitSetPresentationMode` fallback. Auto health checks: `GET /services` now triggers background checks when stale (>5min) via `asyncio.create_task()` with throttle. Mobile video unmute fix (start muted for autoplay policy, unmute after play()). Service worker caches Vite hashed JS bundles (cache-first). Multi-word search with COLLATE NOCASE in paid content and private gallery. Push notification platform names show "Fansly"/"OnlyFans" not "Fansly Direct"/"Onlyfans Direct" via `_get_platform_display_name()`. Feature enable/disable fix with `known_features` tracking. Search count functions (get_posts_count, get_media_count) fixed to include search/date filters. Settings page polls services every 60s. Cleaned up scheduler.py.migrated to archive. Files modified: unified_database.py, models.py, db_adapter.py, onlyfans_client.py, fansly_direct_client.py, scraper.py, paid_content.py, private_gallery.py, api.ts, App.tsx, Messages.tsx (NEW), Settings.tsx, BundleLightbox.tsx, PrivateGalleryBundleLightbox.tsx, sw.js, install.sh. |
| 2026-02-08 | 12.11.1 | **Lightbox Navigation Fixes & Gallery Filter Fix** - Lightbox arrow buttons, keyboard arrows, and swipe gestures now navigate in shuffled order when shuffle mode is enabled (was always navigating in date order). Added shuffle-aware `navigatePrev()`/`navigateNext()` functions replacing `handleManualNavigate()`. Navigation arrows now render above video player (z-index raised from z-10 to z-20). Desktop arrows auto-hide with `text-transparent hover:text-white` pattern matching info/close buttons. Widened arrow hover detection from small circular `p-2` targets to tall rectangular edge zones `px-6 py-16` anchored to screen edges. Videos autoplay during slideshow in Paid Content lightbox. Private Gallery file_type filter now filters individual media items within posts (was only filtering which posts to show, returning all media types within matched posts). Code review fixes: added EXIF date extraction to URL import background task, fixed asyncio.run() → asyncio.new_event_loop() in background tasks, progress modal title shows 'Processing Imports' for URL imports, session keepalive runs during URL imports. Files modified: PrivateGalleryBundleLightbox.tsx, BundleLightbox.tsx, PrivateGalleryUploadModal.tsx, private_gallery.py. |
| 2026-02-08 | 12.11.0 | **Config Page Redesign & Multi-URL Import** - Private Gallery Config page redesigned from sidebar + pill tabs to horizontal underline tabs matching Configuration.tsx and Paid Content Settings.tsx pattern. Removed sticky header with back arrow, replaced with standard h1 + subtitle. Content area spans full tab width. Page root changed from own container to `space-y-6` div matching parent layout. Upload modal now supports multi-URL import: textarea for pasting URLs (one per line), supports Bunkr/Pixeldrain/Gofile/Cyberdrop via FileHostDownloader + direct HTTP fallback. New `importUrlsMutation`, URL count display, Import/Upload/Create Post button switching. Backend: fixed ImportUrlRequest model (tag_ids now optional), added POST /import-urls endpoint, added `_import_urls_to_gallery_background()` background task with per-file download→hash→dedup→encrypt→thumbnail→DB insert pipeline. API client: added `importUrls` method, added 'import' to job operation types. Files modified: Config.tsx, PrivateGalleryUploadModal.tsx, api.ts, private_gallery.py. |
| 2026-02-08 | 12.10.0 | **Slideshow Mode & Gallery Improvements** - Added slideshow mode to both Private Gallery (`PrivateGalleryBundleLightbox`) and Paid Content (`BundleLightbox`) lightboxes with auto-advance, configurable interval (3s/5s/8s/10s), Fisher-Yates shuffle mode, and video-aware advancement. Slideshow toolbar button in both gallery pages is context-aware: gallery view slideshows all items across all posts, feed/social view slideshows the selected post only. Added `total_media` count to both Private Gallery and Paid Content backend APIs for accurate position indicators. Filter bar now shows "X items" in gallery view and "X posts" in feed/social view. Fixed gallery item ordering mismatch (galleryLightboxItems now sorts by media_date matching grid). Fixed Paid Content gallery not auto-loading pages on filter change. Added scroll-to-top on filter change. Slideshow controls hidden when only 1 item. Files modified: PrivateGalleryBundleLightbox.tsx, BundleLightbox.tsx, Gallery.tsx, Feed.tsx, api.ts, private_gallery.py, paid_content.py, db_adapter.py. |
| 2026-02-07 | 12.9.0 | **Thumbnail Progress Modal & Async Gallery Operations** - New shared `ThumbnailProgressModal` component replaces text-based progress displays with visual thumbnail-grid progress tracking across Copy to Gallery, Upload, and Manual Import. Private Gallery copy and upload operations now run asynchronously via BackgroundTasks with in-memory job tracking and 500ms polling. New `GET /private-gallery/job-status/{job_id}` endpoint with dual auth. Added 3 Instagram filename patterns for browser extension formats (video/photo/shortcode). Fixed video thumbnail generation for short videos (fallback seek time). Added temp/manual_import to ALLOWED_PATHS. Fixed ThrottledImage IntersectionObserver in nested scroll containers. private_gallery.py grew from ~1,800 to 3,696 lines. |
| 2026-02-06 | 12.7.5 | **Code Review: Bug Fixes & Code Quality** - Fixed critical bug in shared `get_or_create_thumbnail()` in core/utils.py where it queried non-existent `content_hash` column instead of correct `file_hash` primary key (caused cache miss on every request, regenerating thumbnails unnecessarily). Fixed `get_current_user_media()` token parameter missing `Query()` annotation (query param tokens from `<img>`/`<video>` tags silently ignored). Fixed 6 direct `sqlite3.connect()` calls without context managers in media.py (5) and video_queue.py (1) that could leak connections on exceptions. Fixed deprecated `datetime.utcnow()` (2 locations) and import ordering in core/responses.py. Added missing `@handle_exceptions` to `/api/health` endpoint. Added WAL checkpoint support for non-pool database connections. Consolidated 9 duplicate Pydantic models from api.py into models/api_models.py imports. Removed 3 duplicate thumbnail functions from media.py (now uses shared core/utils.py). Added debug logging to 4 bare `except: pass` blocks in downloads.py and review.py. Fixed 4 incorrect file paths in Worker Process Architecture Proposal document. |
| 2026-02-04 | 12.7.1 | **Security & Code Quality Fixes** - Fixed critical authentication bypass in dashboard.py where underscore-prefixed `_user` parameter was ignored by FastAPI (now uses `current_user`). Fixed missing authentication on public thumbnail endpoint in files.py. Fixed 7 SQLite connection leaks across auth.py, scheduler.py, health.py, and recycle.py (converted to context managers). Fixed SQL injection risk in paid_content/db_adapter.py by adding column whitelist validation. Fixed 5 bare `except:` clauses in unified_database.py to catch specific `sqlite3.OperationalError`. Fixed bare except in paid_content JSON parsing to specific exceptions. Consolidated duplicate ThumbnailLRUCache class into shared core/utils.py. Consolidated path validation into shared `validate_file_path()` function. Fixed forum_db_adapter `_execute_with_retry()` to properly raise instead of returning None. Deprecated logger usage updated in dashboard.py. |
| 2026-02-02 | 12.6.6 | **Unified Filter Bar, Expanded Search & Creator Bios** - Created unified FilterBar component with clean 'Filter Button + Popover' design across all pages (Downloads, Media, Review, Download Queue, Channel Monitors, Appearances, Recycle Bin, Notifications, Paid Content Feed/Creators/RecycleBin). Expanded search functionality now queries platform, source, content_type, URL, and description fields (not just filename). Fixed z-index stacking context issue with filter popover using React Portal to render at document body level. Fixed Dashboard 'New in Media/Review/Discovery' card lightboxes to have same action buttons as main pages (added Quick Add Face and Delete buttons). Added creator bio/description support for Fansly, YouTube, and Twitch - bios are fetched during sync and shown in popup modal on Creators page. Added proxy cache clearing documentation to CLAUDE.md and CRITICAL_RULES.md. |
| 2026-02-02 | 12.6.5 | **Universal Logger Migration** - Migrated 3 modules to universal logger (easynews_client.py, media_identifier.py, web/backend/routers/files.py) that were still using standard logging.getLogger(). Removed unused `import logging` from 7 modules (activity_status.py, date_utils.py, dependency_updater.py, forum_db_adapter.py, pushover_notifier.py, settings_manager.py, unified_database.py). All 78 core modules now use universal logger for web UI log visibility. |
| 2026-02-01 | 12.6.2 | **Comprehensive Code Review Bug Fixes** - Fixed RecycleBin.tsx API response mismatch (deleted_count accessed incorrectly). Added missing @handle_exceptions decorators to 3 dashboard.py endpoints (get_recent_items, get_dismissed_cards, set_dismissed_cards). Replaced 89 logger.log() calls with proper logger.debug/info/warning/error methods (video_queue.py: 39, celebrity.py: 50). Added logging to 3 silent exception handlers (config.py broadcast, health.py scheduler state and cache builder status). Removed unused `import logging` from 5 modules (move_module.py, download_manager.py, face_recognition_module.py, immich_face_integration.py, service_health_monitor.py). Added default cookie path to toolzu_subprocess_wrapper.py for consistency. Added config key validation to forum_subprocess_wrapper.py and base_subprocess_wrapper.py (returns error if required keys missing). |
| 2026-01-31 | 12.2.4 | **Fansly 4K Video & Image Quality Fixes** - Fixed critical 4K video download issue where CloudFront signed HLS streams failed because segment URLs didn't include signed params. Scraper now fetches variant playlist, normalizes CRLF line endings, and rewrites segment URLs with signed params. Fixed image quality issue where images were downloaded from 'variants' (scaled thumbnails maxing at 1080p) instead of 'main locations' (original full resolution up to 1875x2500). Fixed push notifications to correctly show post count and include image/video attachment preview. Added real-time progress tracking for HLS/m3u8 streaming downloads by parsing EXTINF duration tags and ffmpeg progress output. Created utility scripts: check_4k_upgrades.py, check_4k_batch.py, upgrade_fansly_to_4k.py (4K video upgrades), check_image_upgrades.py (image upgrades), refresh_fansly_post_urls.py (refresh expired signed URLs). Archived PostgreSQL migration scripts. |
| 2026-01-30 | 12.2.3 | **Podcast Notification Poster Fix & Taddy Improvements** - Podcast push notifications now include artwork images (was broken - URL was incorrectly prefixed with TMDb base URL). Taddy API fallback account now properly triggers on 500 errors (catches ServiceError exception). Taddy API response parsing updated for new 'search' query format (was using 'searchForTerm'). Taddy filtering changed from blacklist to whitelist - only accepts episodes matching interview patterns. Taddy API uses EXACT_PHRASE matching for more accurate celebrity name searches. Taddy API usage reduced (lookback 730→180 days, max results 250→100). Appearance detail modal redesigned for mobile/iOS with bottom sheet pattern and touch-friendly buttons. New Taddy fallback account support for quota management. |
| 2026-01-29 | 12.2.2 | **Mobile Touch Sensitivity & Lightbox Fixes** - Feed page 'pages.length' error fixed with defensive null checks for infinite query. Mobile lightbox close button now visible (zoom controls hidden on mobile via `hidden sm:flex`). Mobile Back button improved with larger touch target (44x44px) and `onTouchEnd` handler. Feed items no longer open accidentally while scrolling - added touch movement detection with 20px threshold and 50ms minimum duration. Thumbnails no longer open lightbox accidentally while scrolling (same threshold). Thumbnail taps properly isolated from post selection (event propagation stopped). Desktop click handlers separated from touch handlers. |
| 2026-01-29 | 12.2.1 | **Notifications Page Enhancements & Scheduler Fixes** - Scheduler now prioritizes most overdue task instead of alphabetical order. Scheduler preserves original scheduled time for missed tasks (was resetting to now). Push notification import fixed (was importing from non-existent modules.notifier, now modules.pushover_notifier). Notifications page newlines now render correctly (whitespace-pre-line CSS). Duplicate push notifications prevented (only sends when files actually downloaded, not when new_posts > 0). YouTube scheduled sync 10x faster with --dateafter and --playlist-items 1:20 filters. Twitch scheduled sync faster with --dateafter and --playlist-items 1:50 filters. Notification lightbox shows full metadata (resolution, size, post content, date, platform). Notification thumbnails show Feed-style overlay (filename, size, resolution, duration). Lightbox resolution falls back to loaded image dimensions when metadata unavailable. New /thumbnail and /preview endpoints for serving files by path. New get_attachment_by_path() method in db_adapter. |
| 2026-01-27 | 12.1.0 | **Fansly Direct API Integration** - Major new feature for downloading content directly from Fansly API (bypasses Coomer archives). FanslyDirectClient class with full API support (auth, posts, media, bundles). 4-hour scheduled sync checking last 3 days for new content. Push notifications with image attachment preview when new content downloaded. Fansly auth token configuration in Settings page. Service health check validates token and shows connected username. Added no_description column to exclude posts without descriptions from filter. YouTube and Twitch scheduled syncs now only check last 3 days (performance optimization). Fansly Direct client skips non-media content types (contentType 42001). Creators page shows "Fansly" instead of "fansly_direct". "View on Fansly" links to creator's profile. PPV filenames renamed from random Coomer-style to date-based format (YYYY-MM-DD). |
| 2026-01-26 | 12.0.1 | **Cloudflare Cookie Fingerprint Fix** - Critical fix for cf_clearance cookies not working. Fixed user_agent not being saved with cookies in imginn_module.py, toolzu_module.py, and snapchat_scraper.py. Added key_value_store database table for persistent settings. Added get_setting()/set_setting() to UnifiedDatabase. FlareSolverr fingerprint now persists to database (instant loading on restart). Added set_fingerprint_database() initialization in scheduler. All Playwright scrapers now use consistent fingerprinting from FlareSolverr. |
| 2026-01-23 | 12.0.0 | **Paid Content Feature - Creator Platform Archival** - Major new feature for tracking and downloading content from subscription-based creator platforms. Integrates with Coomer.su (OnlyFans, Fansly, CanDFans) and Kemono.su (Patreon, Fanbox, Gumroad, SubscribeStar, Discord). YouTube channel support via yt-dlp. 8 new pages: Dashboard, Feed, Creators, AddContent, Notifications, Settings, Queue, RecycleBin. Split-view feed browser with gallery/thumbnail toggle. Identity system for linking same creator across platforms. 10 new database tables. New paid_content module (api_client, db_adapter, scraper, file_host_downloader, embed_downloader, youtube_client). 25+ new API endpoints. Favorites system. Manual import from Bunkr, Pixeldrain, Gofile, Cyberdrop. iOS safe area fixes for mobile feed view. |
| 2026-01-15 | 11.29.1 | **Dashboard Cards Cross-Browser Sync & Fixes** - Dashboard cards now exclude items moved between Review/Media (added moved_from_review/moved_from_media flags). Removed catch-up notification feature. Cross-browser dismiss sync via server-side storage in user_preferences table. New GET/POST /api/dashboard/dismissed-cards endpoints. Optimistic updates for instant UI feedback. |
| 2026-01-15 | 11.29.0 | **Breadcrumb Navigation System** - Context-aware breadcrumb navigation below header on all 23 pages. Dynamic updates based on user actions (media type filter, celebrity selection). New BreadcrumbContext, Breadcrumb component, useBreadcrumb hook, and breadcrumbConfig. All levels clickable for navigation. Long labels truncate with ellipsis. |
| 2026-01-15 | 11.28.0 | **Dashboard New Items Cards** - Three dismissible cards showing recent additions with thumbnail previews (New in Media, New in Review, New in Internet Discovery). Click thumbnails to open lightbox or preview modal. Hover overlay with action buttons. Session-based tracking removes actioned items. Backend /api/dashboard/recent-items endpoint. New RecentItemsCard component. |
| 2026-01-14 | 11.27.0 | **Performance Optimizations & Frontend Modernization** - Review page SQL query fixed (158x faster). 3-tier thumbnail caching (memory → SQLite → generate). Thumbnails pre-generated during MoveManager processing. ThrottledImage optimized with shared IntersectionObserver. API retry logic with exponential backoff. New shared FilterBar/MediaGrid components. SuspenseBoundary with skeleton loading. useMediaFiltering hook with useTransition/useDeferredValue. |
| 2026-01-13 | 11.26.3 | **Cross-Process Sync Status & Notification Fixes** - Fixed dashboard cards not showing for scheduler-triggered TMDB/Easynews syncs (cross-process issue). Now uses ActivityStatusManager for database-based status tracking. TMDB sync now runs in background thread (non-blocking). Easynews notifications only sent for truly new content (not quality upgrades). Fixed Easynews thumbnail URL storage. Fixed YouTube channel monitor notifications not being sent from scheduler. |
| 2026-01-10 | 11.26.2 | **TMDb Appearance Prediction Fix** - Fixed false positive appearance predictions where guest/host credits predicted future episodes incorrectly. Added get_episode_credits() and is_person_in_episode() methods to TMDbClient. Now verifies celebrity is in specific episode credits before adding prediction. Removed 10 incorrect upcoming appearances. |
| 2026-01-10 | 11.26.1 | **Security Hardening & Code Cleanup** - Fixed SQL injection risks in maintenance.py (whitelist validation) and media.py (explicit SQL branches). Added require_admin to scheduler restart. Added missing for_write=True on video DELETE. Forum monitoring now uses configured auto_track_days. Subprocess wrappers refactored with factory pattern (67% code reduction). forum_db_adapter now uses _execute_with_retry consistently. Added utility functions to core/utils.py and core/responses.py. |
| 2026-01-09 | 11.26.0 | **Snapchat Direct Scraper** - Complete rewrite of Snapchat downloading. New Playwright-based scraper replaces StoryClon proxy module. Supports Stories and Spotlight content. EXIF metadata preservation. Removed video stitching feature (individual clips now). Archived old snapchat_module.py. Updated docs (README, SCRAPER_PROXY_SYSTEM, DOWNLOADER_MONITORING). |
| 2026-01-04 | 11.23.2 | **API Module Import Fix** - Fixed critical bug where all API endpoints returned 500 errors due to inconsistent module import paths. `api.py` was importing from `core.dependencies` (relative) while routers used `..core.dependencies`, resolving to different Python modules and creating separate AppState singletons. Changed to consistent `web.backend.core.dependencies` absolute imports. Removed 3 orphaned Python cache files (imdb_client, tvdb_client, picturepub_workaround). |
| 2026-01-04 | 11.23.1 | **ImgInn Download All Posts Option** - Added "Download All Posts (skip phrase filtering)" toggle to ImgInn phrase search configuration. When enabled, all posts from specified usernames are downloaded without phrase matching. UI dynamically hides phrase-related fields when Download All is enabled. |
| 2026-01-04 | 11.23.0 | **Appearances Navigation & Midnight Reminders** - Promoted Appearances to main navigation bar. Reminders now sent at midnight for same-day appearances only. Added scheduler `schedule_daily_at_time()` method. Added Plex 'Watch' button to notification cards. Fixed celebrity filter default from 'Eva Longoria' to 'all'. Fixed credit_type TypeScript type inconsistencies. Fixed bare except clauses in appearances.py. Fixed memory leak with Plex match polling cleanup. Added missing sync status fields to API types. Notification stats cards now match main Notifications page design. |
| 2026-01-02 | 11.22.4 | **Recycle Bin & Appearances Enhancements** - Fixed Review page batch delete failing when >500 files (now chunks into batches of 500). Fixed recycle bin lightbox showing deletion date instead of download date. Added download_date and post_date sort options to recycle bin (default: download_date). Added "On Plex" filter toggle to Appearances page. Fixed Plex filter to only show items with actual episode/movie match (not just show match). |
| 2026-01-02 | 11.22.3 | **Plex Integration Fixes & Automation** - Fixed Plex matching for TV show rating keys. Fixed false Plex links for episodes not in library. Removed Plex links from upcoming appearances. Added automatic Plex matching via systemd timer (6am/6pm). |
| 2026-01-01 | 11.22.2 | **Codebase Cleanup & Dead Code Removal** - Removed 3 unused Python modules (imdb_client.py, listen_notes_client.py, picturepub_workaround.py). Removed unused timeline_cache table. Removed orphaned /presets page. Cleaned up 34 unused API wrapper methods in api.ts (~327 lines, 12% reduction). No functional changes. |
| 2025-12-28 | 11.22.0 | **Manual Platform Run Monitoring** - New dashboard card showing real-time progress for manual downloads. Platform buttons persist running state across page navigation with stop functionality. Live Scraping Monitor loads active sessions on mount. Backend tracking with GET /api/platforms/running, POST /api/platforms/{platform}/stop, GET /api/scraper-sessions/active endpoints. Session cleanup fixed to prevent duplicates. Black glass button styling for consistency. |
| 2025-12-28 | 11.21.1 | **Coppermine Cloudflare retry fix** - Added automatic retry mechanism for Cloudflare challenges during downloads. New _request_with_retry() method detects and handles challenges in real-time with fresh cookies. |
| 2025-12-28 | 11.21.0 | **Server-side search & channel management** - Added "Add Channel" button to YouTube Channel Monitors page with modal. All pages now use server-side search (Internet Discovery, Search Presets). Fixed scheduler duplicate task. Search queries now search entire database instead of current page. |
| 2025-12-26 | 11.20.0 | **YouTube Channel Monitor Management** - Dedicated channel monitors page with auto-pause, always-active flag, bulk actions, status filters |
| 2025-12-26 | 11.19.4 | **Configuration & Monitoring fixes** - Fixed loading issues on Configuration tabs. Service Health Monitoring now uses consecutive runs instead of time windows. Removed scan interval from Error Monitoring. Face recognition hidden on Media page. |
| 2025-12-26 | 11.19.3 | **Face recognition fixes** - Lightbox now shows confidence for non-matched items. Fixed review API SQL JOIN to use filename matching. Fixed move_module UPDATE query for face scan paths. |
| 2025-12-26 | 11.19.2 | **Logging refactoring** - Created LoggingMixin, standardized logging across 13 modules. **Hash standardization** - All MD5 changed to SHA256 for consistency (11 files). Fixed instagram_repost_detector inverted logging priority. |
| 2025-12-26 | 11.19.1 | Server-side search for Media/Downloads/Review, fixed Dependencies section on Health page, removed 4 unused files, TypeScript fixes |
| 2025-12-22 | 11.17.0 | Dashboard stats fix - total downloads now includes all sources (media+review+recycle), file ownership/permissions fix |
| 2025-12-21 | 11.16.0 | Search boxes on Downloads/Internet Discovery, server-side Video Downloader search, jpg thumbnail standardization |
| 2025-12-21 | 11.15.0 | Portrait video resolution label fix, Download Queue thumbnail caching via backend |
| 2025-12-21 | 11.14.0 | Config page hover fix, Video Downloader settings now honored, removed unused Download Settings |
| 2025-12-21 | 11.16.0 | **Semantic versioning overhaul** - 246 versions renumbered for correct PATCH/MINOR/MAJOR classification |
| 2025-12-20 | 11.13.1 | Renamed Celebrity Discovery → Internet Discovery, removed unused config sections |
| 2025-12-20 | 11.13.0 | Face recognition path fix for Review page, parse_with_fallbacks() for Instagram filename patterns |
| 2025-12-20 | 11.12.0 | Downloads page batch operations, Instagram Stories filename fix, per-day summary stats, dead code cleanup |
| 2025-12-20 | 11.11.0 | Modern UI styling (custom scrollbars, animated counters, thumbnail zoom, glass cards), codebase cleanup (11 unused files removed) |
| 2025-12-17 | 11.10.2 | Code review security & stability fixes - cookie security, path traversal, thread safety, connection leaks |
| 2025-12-17 | 11.10.0 | Dynamic aspect ratio for embedded videos, max_width database column, yt-dlp width capture |
| 2025-12-17 | 11.9.0 | Forum filename date prefix, proportional video sizing, GIF fixes, forum notification fix |
| 2025-12-16 | 11.8.1 | Mobile/iOS Configuration UI, YouTube Monitor notification and app_state fixes |
| 2025-12-14 | 11.8.0 | Dailymotion HLS streaming, centralized timestamps, Dashboard video queue card |
| 2025-12-13 | 11.7.0 | YouTube scheduler interval fix, Search Presets Full Scan button, Instagram duplicate recording fix |
| 2025-12-13 | 11.6.1 | Scheduler UI fix - YouTube Monitor display name and description |
| 2025-12-13 | 11.6.0 | Video download timezone fix - UTC instead of local time |
| 2025-12-13 | 11.5.1 | YouTube monitor auto-start queue fix, direct queue processor access, service documentation |
| 2025-12-13 | 11.5.0 | YouTube Channel Monitor feature, thumbnail jpg fix, recycle bin metadata restore, error notification context fix |
| 2025-12-12 | 11.4.0 | Post date editing from lightbox, batch date changes, deferred database recording for reliability |
| 2025-12-11 | 11.3.1 | Notifications 503 fix with lazy loading, silent exception handling improvements, proxy rate limits |
| 2025-12-10 | 11.3.0 | Celebrity Discovery UX improvements, forum date fixes, queue settings persistence |
| 2025-12-10 | 11.2.0 | Anti-bot protection for yt-dlp, queue sort toggle, stuck download reset, cookie expiration handling |
| 2025-12-09 | 11.1.0 | Download Selected feature, configurable base directory, mobile menu safe area fix |
| 2025-12-09 | 11.0.0 | Major release - Celebrity Discovery, Mobile UI, Native Video Player, Unified Download Queue |
| 2025-12-07 | 9.1.0 | Video streaming settings UI, default player selection, queue resolution fixes |
| 2025-12-07 | 9.0.0 | Native video player with yt-dlp streaming, privacy-focused YouTube alternative |
| 2025-12-07 | 8.23.3 | Critical memory leak fixes - OOM kills resolved, error banner dropdown fix |
| 2025-12-07 | 8.23.2 | UI consistency update - standardized page headers, menu icons, layouts |
| 2025-12-07 | 8.20.0 | Recycle Bin platform/source filters, Celebrity Discovery system |
| 2025-12-06 | 8.18.1 | Error banner fixes, Instagram username extraction, memory optimizations |
| 2025-12-05 | 8.17.0 | API modularization complete, error monitoring UI, dashboard fix |
| 2025-12-05 | 8.14.0 | Phase 1 refactoring complete, debt tracker updated |
| 2025-12-04 | 8.13.0 | Initial comprehensive review |

### Key Findings by Version

#### 13.8.1
- **Inline Audio Player:**
  - PostDetailView: native HTML5 audio player with filename, file size, playback controls
  - Feed PostCard: compact mobile-only audio player with touch event isolation (stopPropagation)
  - Audio attachments excluded from thumbnail grid and BundleLightbox (image/video only)
- **Push Notification Audio Support:**
  - Notification code now checks `content_type` from scraper metadata before falling back to file extension
  - Fixes `.mp4` files with audio content_type being misidentified as video
  - New 🎵 audio category in notification titles, pushover PLURALS/CONTENT_ICONS updated
  - Fixed hardcoded `'content_type': 'video'` in 3 downloaded_file_info append sites to use `att.get('file_type', 'video')`
- **Service Health Check Fixes:**
  - HQCelebCorner/PicturePub: dedicated health checks accepting HTTP 200/403 (FlareSolverr used at runtime)
  - Snapchat: checks story.snapchat.com reachability (200/301/302)
  - Fixes discrepancy between paid dashboard ("degraded") and scrapers page ("healthy")
- **Reddit Community Monitor Reliability:**
  - Switched from OAuth API (shared rate-limited client-id `6N9uN0krSDE-ig`) to REST API mode (`extractor.reddit.api=rest`)
  - Added `--range 1-200` limit to prevent full history pagination timeouts
  - Fixed cookie file write error with `os.makedirs(temp_dir, exist_ok=True)`
- **Code Quality:**
  - Fixed function name shadowing: `get_preset_categories` in celebrity.py, `get_queue_settings` in video_queue.py
  - Fixed backup profile stale paths in add-backup-profile.sh
- Files modified: scraper.py, pushover_notifier.py, reddit_community_monitor.py, paid_content.py, celebrity.py, video_queue.py, PostDetailView.tsx, Feed.tsx, install.sh, add-backup-profile.sh

#### 13.8.0
- **XenForo Forum Support:**
  - Generic XenForo-based forum client (`xenforo_forum_client.py`) for celebrity image forums
  - HQCelebCorner/PicturePub compatibility layer (`hqcelebcorner_client.py`)
  - External image host resolution (ImageBam, Pixhost, Imgur, imgbox, postimg, etc.)
  - FlareSolverr Cloudflare bypass, CSRF token handling, junk URL filtering
- **Snapchat HTTP Client:**
  - Direct curl_cffi-based client replacing Playwright dependency (`snapchat_client.py`)
  - __NEXT_DATA__ JSON extraction from story.snapchat.com
  - EXIF metadata preservation including GPS, view counts, timestamps
- **TikTok Hybrid Client:**
  - Unified yt-dlp (listing) + gallery-dl (downloading) interface (`tiktok_client.py`)
  - Automatic cookie refresh and merge-back to database
  - Carousel/slideshow detection, creator profile scraping
- **Instagram Direct API Module:**
  - Direct GraphQL + REST API via curl_cffi (`instagram_client_module.py`)
  - No Playwright or Cloudflare dependency, 10-20x faster than ImgInn
  - Stories/reels/tagged via REST API, public posts via GraphQL
- **Cookie Health Monitoring:**
  - Comprehensive health endpoint with per-platform monitoring toggles
  - Background health checks with 30-min cooldown, WebSocket + Pushover alerts
  - Scraper module enable/disable detection to avoid false positives
- **ImgInn Adapter Enhancements:**
  - Two-phase post discovery (API + HTML profile scraping)
  - Detail page fetching for full-res 1440p+ CDN URLs
  - Direct Instagram tag fetching with NordVPN proxy rotation
- **Bug Fixes:**
  - FastDL browser fallback story duplicates (pk mapping + MD5 hash fallback)
  - Recycle bin restore metadata inference (platform from path, source from filename)
  - Square video display (1:1 aspect ratio treated as portrait)
  - Failed downloads missing creator_db_id
  - Function name shadowing in celebrity.py and video_queue.py
- **Cleanup:**
  - Removed 30 empty log files, empty unified.db, cleared __pycache__ directories
  - Fixed stale backup profile paths in add-backup-profile.sh
- **Files Created:** xenforo_forum_client.py, hqcelebcorner_client.py, bellazon_client.py, soundgasm_client.py, snapchat_client.py (paid_content), tiktok_client.py, youtube_client.py, backfill_tagged_users.py, task_checkpoint.py
- **Files Modified:** fastdl_module.py, scraper.py, db_adapter.py, unified_database.py, imginn_adapter.py, imginn_api_module.py, snapchat_client_module.py, instagram_client_module.py, health.py, scrapers.py, paid_content.py, PostDetailView.tsx, BundleLightbox.tsx, celebrity.py, video_queue.py, add-backup-profile.sh

#### 13.7.0
- **Per-Creator Instagram Sync Settings:**
  - Toggle sync for posts, stories, and highlights individually from creator card menu
  - Settings stored as database columns with defaults preserving existing behavior
- **Tagged User Filter:**
  - Per-creator feed filter that only shows posts where selected tagged users appear
  - Multi-select UI with username autocomplete from existing tagged data
- **Appearance Role Combining:**
  - Shows with multiple roles display as single entry with all roles
  - Episode counts use COUNT(DISTINCT) to avoid inflation
  - Consistent badge colors via shared creditTypeConfig

#### 13.6.0
- **ImgInn Paid Content Adapter:**
  - New `imginn_adapter.py` replaces Instagram API-based adapter with ImgInn-based scraping
  - No Instagram credentials needed — uses ImgInn's CDN URLs for posts, stories, reels
  - Maps ImgInn API data to existing Post/Attachment models
  - Built-in rate limiting and Cloudflare bypass
- **ImgInn API Optimizations:**
  - FlareSolverr integration for reliable Cloudflare bypass
  - OCR disabling for faster scraping performance
  - Date pre-filtering to skip old posts during sync
  - Profile resolution improvements
- **Reddit Timezone Fix:**
  - gallery-dl metadata dates were stored as naive UTC strings without timezone conversion
  - Added UTC→local conversion using `datetime.timezone`
  - Fixed `created_at` to use import time instead of post date
  - Bulk-fixed 556 existing reddit posts and 1366 media entries via decrypt→convert→re-encrypt cycle
- **Lightbox Improvements:**
  - Fixed slideshow back arrow not showing — parent-managed shuffle mode set `shuffleEnabled=true` but never built `shuffleOrder`
  - Fixed paid content bundle sidebar invisible on desktop — z-index stacking context (added relative z-20 to sidebar)
  - Bundle sidebar auto-collapses when slideshow starts, hidden for single-image posts
  - Expand/collapse with floating button, slideshow controls stay stationary regardless of sidebar state
- **Dashboard:** Improved platform labels for better display naming
- **Cleanup:**
  - Removed 20 stale scripts (one-off 4K checks, Fansly maintenance, enrichment, test scripts)
  - Removed 15 stale utilities (completed backfills, cleanups, test scripts, standalone db_manager)
  - Removed legacy SQLite database file (`database/media_downloader.db`)
  - Removed `archive/` directory (migrated scheduler + migration scripts)
  - Cleared all `__pycache__` directories
- **Files Modified:** imginn_adapter.py (NEW), reddit_community_monitor.py, private_gallery.py, BundleLightbox.tsx, PrivateGalleryBundleLightbox.tsx, Gallery.tsx, Feed.tsx, Dashboard.tsx

#### 13.5.1
- **Landscape Lightbox for Private Gallery:**
  - Ported all landscape phone detection from BundleLightbox.tsx to PrivateGalleryBundleLightbox.tsx
  - Touch-based device detection: `isMobile` (portrait phone), `isLandscape` (touch + height<500px), desktop (neither)
  - Fixed positioning for landscape containers using `100dvh` for mobile browser chrome compatibility
  - Controls use `text-white/70` on landscape (hover doesn't work on touch devices)
  - Navigation arrows: compact `p-3 bg-black/40 rounded-full` with `w-7 h-7` icons
  - Image container sidebar offset: `left: 6rem` accounts for `w-24` sidebar
  - Video bottom offset: `isMobile ? 67 : isLandscape ? 50 : 120`
- **Per-Person-Group Min Resolution Filter:**
  - SQL filter: `(file_type != 'image' OR (width >= ? AND height >= ?))` — shows all videos, filters only images
  - Gallery filter dropdown with preset options (300/500/720/1080px+) and active filter chip
  - Import skip: low-res images skipped during URL/upload import with status tracking
  - Setting stored per person group in config
- **TypeScript Fixes:**
  - Added `min_resolution?: number` to `getPersonGroups()` and `getPersonGroup()` return types in api.ts
  - Removed unused `editMinRes` state variable from Config.tsx
  - Fixed `task.next_run` null safety in Scheduler.tsx (`|| new Date().toISOString()` fallback)
- **Cleanup:**
  - Removed `data/migration_backup.json` (682KB stale migration data)
  - Removed `archive/media_downloader.db.old` (308KB obsolete database)
  - Removed all `__pycache__` directories (~5MB across 9 directories)
- **Files Modified:** PrivateGalleryBundleLightbox.tsx, private_gallery.py, Gallery.tsx, Config.tsx, api.ts, Scheduler.tsx

#### 12.15.0
- **UsernameListEditor Component:**
  - New reusable `UsernameListEditor.tsx` component replacing 6 comma-separated username fields
  - Add usernames via text input + Enter key or + button, all normalized to lowercase
  - Alphabetically sorted display with scrollable container (`max-h-48`) and `custom-scrollbar`
  - Duplicate detection with amber "DUP" badges and one-click "Remove Duplicates" button
  - "Paste List" modal for bulk import of usernames separated by commas or newlines
  - User count badge in header, empty state with dashed border placeholder
  - Remove individual usernames via X button on hover
- **Configuration Page Cleanup:**
  - Replaced FastDL regular + phrase search username fields (2 textareas)
  - Replaced ImgInn regular + phrase search username fields (1 input + 1 textarea)
  - Replaced Toolzu username field (1 textarea)
  - Replaced Snapchat username field (1 textarea)
  - FastDL and ImgInn phrase search usernames converted from uncontrolled (defaultValue/onBlur) to controlled components
  - ImgInn phrase search label dynamically switches based on `download_all` setting
- **Files Created:** `web/frontend/src/components/UsernameListEditor.tsx` (155 lines)
- **Files Modified:** `web/frontend/src/pages/Configuration.tsx` (6 field replacements + import)

#### 12.14.0
- **Streaming Decryption for All Encrypted Videos:**
  - `decrypt_file_generator()` yields 8MB decrypted chunks for chunked format files
  - `decrypt_file_range_generator()` calculates encrypted chunk indices from byte offsets for Range requests
  - `stream_video`, `get_file`, `export_single` endpoints all use `StreamingResponse`
  - Eliminates full-file memory loading for multi-GB encrypted videos
- **Auto-Migration to Chunked Encryption:**
  - On gallery unlock, background thread converts single-shot files >50MB to chunked format
  - `re_encrypt_to_chunked()` in crypto module with atomic rename
  - Unique temp filenames (`secrets.token_hex(4)`) prevent race conditions from double unlock
  - Once-per-process guard via `getattr(unlock_gallery, '_migration_done', False)`
  - `POST /migrate-chunked` endpoint for manual triggering
- **Download Reliability Improvements:**
  - Stall detection: 30-second read timeout on `requests.get()` via `timeout=(30, 30)` tuple
  - Retry logic: up to 3 retries with HTTP Range resume from last downloaded byte
  - All direct downloads multi-threaded with 5 parallel segments (removed 5MB size threshold)
- **Bug Fixes:**
  - Subprocess stdin double-close: `communicate(input=...)` replaces manual `stdin.write()`/`stdin.close()`
  - Duplicate filename check verifies `.enc` file exists on disk
  - `sync_tmdb_appearances` gets `db` parameter for scheduler context (was NoneType error)
  - Re-encryption race condition fixed with unique temp filenames
- **Log Level Tuning:**
  - FastDL, ImgInn, Toolzu, Snapchat subprocess errors → warnings
  - Page load timeout/failure: warning normally, error after 5+ per session (`_page_load_failures` counter)
  - "Failed to download single image" 403 → warning
- **Files Modified:**
  - `modules/private_gallery_crypto.py` - Streaming generators, re-encryption method
  - `web/backend/routers/private_gallery.py` - Streaming endpoints, duplicate check, download retry, migration
  - `media-downloader.py` - Subprocess stdin fix, error→warning changes
  - `modules/imginn_module.py` - Error→warning, page load failure counter
  - `web/backend/routers/appearances.py` - sync_tmdb_appearances db parameter
  - `modules/scheduler.py` - Pass db to sync_tmdb_appearances

#### 12.11.1
- **Lightbox Shuffle-Aware Navigation:**
  - Arrow buttons, keyboard arrows, and swipe gestures now respect shuffle order when shuffle mode is enabled
  - Added `navigatePrev()`/`navigateNext()` callbacks with `shuffleOrder`/`shufflePosition` tracking
  - Removed unused `handleManualNavigate()` function (caught by TypeScript)
  - Mobile position indicator shows shuffle position when shuffle active
- **Lightbox Arrow UX Improvements:**
  - z-index raised from `z-10` to `z-20` to render above video player elements
  - Desktop arrows auto-hide (`text-transparent hover:text-white`) matching info/close button pattern
  - Hover detection area widened from `p-2` circular to `px-6 py-16` edge-anchored rectangular zones
  - Subtle `hover:bg-white/5` feedback on desktop arrows
- **Video Autoplay in Slideshow:**
  - Paid Content lightbox: videos autoplay when `slideshowPlaying` is true
  - Added `slideshowPlaying` to `autoPlay` condition and `video.play()` call
- **Private Gallery File Type Filter Fix:**
  - Backend media query now includes `file_type` filter when fetching media items for posts
  - Previously only filtered which posts to show (via EXISTS subquery) but returned all media within those posts
  - Now selecting "Images" correctly hides videos within mixed-media posts
- **URL Import Code Review Fixes:**
  - Added EXIF date extraction (`_extract_date_from_exif()`) before filename-based extraction
  - Fixed `asyncio.run()` → `asyncio.new_event_loop()` to avoid RuntimeError in background tasks
  - Progress modal title shows "Processing Imports" for URL imports
  - Session keepalive timer runs during URL imports and job polling
- **Files Modified:**
  - `web/frontend/src/components/private-gallery/PrivateGalleryBundleLightbox.tsx` - Shuffle nav, arrow UX
  - `web/frontend/src/components/paid-content/BundleLightbox.tsx` - Shuffle nav, arrow UX, video autoplay
  - `web/frontend/src/components/private-gallery/PrivateGalleryUploadModal.tsx` - Import title, keepalive
  - `web/backend/routers/private_gallery.py` - File type filter, EXIF date, asyncio fix

#### 11.27.0
- **Review Page SQL Query Fix:**
  - Root cause: `LEFT JOIN face_recognition_scans frs ON frs.file_path LIKE '%' || fi.filename` caused full table scans
  - Changed to: `LEFT JOIN face_recognition_scans frs ON frs.file_path = fi.file_path` (exact matching)
  - Result: 3.475 seconds → 0.022 seconds (158x faster)
- **3-Tier Thumbnail Caching:**
  - In-memory LRU cache with 500 item limit and 100MB size cap
  - SQLite thumbnails.db as persistent storage (19,225 thumbnails, 401MB)
  - On-demand generation as fallback
  - Cache hierarchy: Memory → SQLite → Generate
- **Thumbnail Pre-generation:**
  - `_generate_thumbnail_cache()` method added to MoveManager
  - Thumbnails generated and cached when files are moved (after move completes)
  - Eliminates generation delay on page load
- **ThrottledImage Optimization:**
  - Shared IntersectionObserver across all image instances (was creating new observer per image)
  - Increased rootMargin from 50px to 400px for earlier preloading
  - Removed random delays that caused staggered loading
  - Memoized with React.memo()
- **Frontend Modernization:**
  - API retry logic with exponential backoff, jitter, configurable retry count
  - Retries on 408, 429, 500, 502, 503, 504 status codes
  - useTransition for non-blocking filter updates
  - useDeferredValue for smooth search input rendering
- **New Shared Components:**
  - `FilterBar.tsx` - Unified filter component for all pages
  - `MediaGrid.tsx` - Memoized thumbnail grid with selection support
  - `SuspenseBoundary.tsx` - Suspense wrapper with skeleton loading states
- **Browser Cache Headers:**
  - Extended from 1 hour to 24 hours with `immutable` flag
  - Reduces redundant thumbnail requests
- **Files Modified:**
  - `web/backend/routers/review.py` - SQL JOIN fix
  - `web/backend/routers/media.py` - LRU cache, extended cache headers
  - `web/frontend/src/lib/api.ts` - Retry logic
  - `web/frontend/src/components/ThrottledImage.tsx` - Shared observer, prefetch
  - `web/frontend/src/components/FilterBar.tsx` - New file
  - `web/frontend/src/components/MediaGrid.tsx` - New file
  - `web/frontend/src/components/SuspenseBoundary.tsx` - New file
  - `web/frontend/src/hooks/useMediaFiltering.ts` - useTransition, useDeferredValue
  - `modules/move_module.py` - Thumbnail pre-generation

#### 11.26.2
- **TMDb Appearance Prediction Fix:**
  - Fixed false positive appearance predictions where having ANY past credit for a show caused predictions for all future episodes
  - Added `get_episode_credits()` method to fetch credits for specific TV episodes
  - Added `is_person_in_episode()` method to verify a person is in episode cast/guest_stars
  - Updated `find_upcoming_tv_appearances()` to verify guest/host/cameo credits against actual episode credits
  - Credits with ≤5 episodes or credit types 'guest', 'host', 'cameo', 'self' now require episode-level verification
- **Data Cleanup:**
  - Removed 10 incorrect upcoming appearances (false predictions for talk shows)
  - Resynced TMDb data with corrected prediction logic
- **Files Modified:**
  - `modules/tmdb_client.py` - Added episode credits verification functions and logic

#### 11.26.1
- **Security Hardening:**
  - Fixed SQL injection risk in `maintenance.py` - added `ALLOWED_CLEANUP_TABLES` whitelist and `_CLEANUP_QUERIES` pre-built at module load time
  - Fixed SQL injection risk in `media.py` - replaced f-string column interpolation with explicit if/else branches for `post_date`/`download_date`
  - Added `require_admin` check to scheduler service restart endpoint (was only requiring authentication)
  - Added missing `for_write=True` flag on video.py DELETE operation
- **Bug Fixes:**
  - Forum monitoring now uses configured `auto_track_days` from Configuration page instead of hardcoded 30 days
  - Forum monitoring log messages now display actual configured days value
  - Scheduler `_check_monitored_threads` now uses config value for cleanup and logging
- **Code Cleanup & Refactoring:**
  - Subprocess wrappers refactored using factory pattern in `base_subprocess_wrapper.py`
  - Reduced wrapper code from ~450 lines to ~150 lines (67% reduction)
  - `forum_db_adapter.py` now uses `_execute_with_retry()` consistently instead of inline retry logic
  - Added `build_media_filter_query()` utility to `core/utils.py` for standardized filter building
  - Added `build_platform_list_filter()` utility to `core/utils.py`
  - Enhanced `offset_paginated()` response helper with `include_timestamp` and `**extra_fields` support
  - Added `batch_operation_response()` helper to `core/responses.py` for standardized batch operation responses
- **Files Modified:**
  - `web/backend/routers/maintenance.py` - SQL injection fix with whitelist validation
  - `web/backend/routers/media.py` - SQL injection fix with explicit branches
  - `web/backend/routers/scheduler.py` - Admin check + forum monitoring config fix
  - `web/backend/routers/video.py` - for_write=True fix
  - `web/backend/core/utils.py` - New filter builder utilities
  - `web/backend/core/responses.py` - Enhanced response helpers
  - `modules/forum_downloader.py` - auto_track_days parameter added to download_thread
  - `modules/forum_db_adapter.py` - Refactored to use _execute_with_retry
  - `modules/scheduler.py` - Forum config auto_track_days usage
  - `wrappers/base_subprocess_wrapper.py` - Factory pattern added
  - `wrappers/fastdl_subprocess_wrapper.py` - Refactored to use factory
  - `wrappers/imginn_subprocess_wrapper.py` - Refactored to use factory
  - `wrappers/toolzu_subprocess_wrapper.py` - Refactored to use factory
  - `wrappers/snapchat_subprocess_wrapper.py` - Refactored to use factory
  - `wrappers/forum_subprocess_wrapper.py` - auto_track_days parameter passed

#### 11.26.0
- **Snapchat Direct Scraper:**
  - Complete replacement of StoryClon proxy-based module with direct Playwright scraper
  - Scrapes directly from snapchat.com using `__NEXT_DATA__` JSON extraction
  - Supports both Stories and Spotlight content types
  - EXIF metadata (DateTimeOriginal, GPS if available) preserved on all downloads
  - Video stitching feature removed due to audio/video sync issues
  - Each highlight clip downloaded as individual file with proper metadata
  - Archived old `snapchat_module.py` to `docs/archive/snapchat_module_storyclon.py`
  - Updated unified_database.py scraper seed data
  - Updated subprocess wrapper to remove stitch_highlights parameter

#### 11.19.4
- **Configuration Page Loading Fixes:**
  - Manual Import, Monitoring, and Dependencies tabs now properly wait for all data before rendering
  - Fixed loading states to check all required queries (isLoading, isLoadingErrorMonitoring, config, editedConfig)
- **Service Health Monitoring Improvements:**
  - Changed from time-based "Failure Window (hours)" to consecutive run-based counting
  - Simplified logic to use `min_consecutive_failures` (default: 2 runs)
  - Alert delay now correctly reads from `error_monitoring.push_alert_delay_hours` setting
- **Error Monitoring Cleanup:**
  - Removed unnecessary `scan_interval_hours` field from UI and backend
  - Errors are logged immediately and checked hourly (hardcoded behavior)
- **Lightbox Enhancement:**
  - Added optional `hideFaceRecognition` prop to EnhancedLightbox component
  - Face recognition info now hidden on Media page, visible on Review page

#### 11.19.3
- **Face Recognition Display Fixes:**
  - EnhancedLightbox now shows confidence percentage for non-matched items (format: "No match (X%)" in red)
  - Fixed SQL JOIN in review.py to use filename-based matching instead of exact path matching
  - Face scans logged with /opt/immich/md/ paths but review files at /opt/immich/review/ - now matches correctly
  - Fixed move_module.py UPDATE query to use `original_intended_path` when updating face scan paths
  - Was incorrectly using `source` (temp file path) which never matched the logged md path

#### 11.19.2
- **Logging Refactoring:**
  - Created `modules/base_module.py` with `LoggingMixin` class for consistent logging
  - Updated 13 modules to use LoggingMixin (instaloader, tiktok, imginn, fastdl, toolzu, coppermine, forum_downloader, download_manager, move_module, scheduler, snapchat, instagram_repost_detector)
  - Removed 12+ duplicate `log()` method implementations
  - Fixed instagram_repost_detector.py inverted logging priority (was callback first, then creating new logger on each call)
- **Hash Standardization:**
  - Changed all MD5 usage to SHA256 for consistency (11 files updated)
  - Fixed potential duplicate detection failures between modules using different hash algorithms
  - Files: download_manager.py, forum_downloader.py, forum_db_adapter.py, universal_video_downloader.py, unified_database.py, universal_logger.py, manual_import.py, video.py, review.py, media.py, migrate_video_folders.py
  - No database migration needed - all existing hashes were already SHA256

#### 11.19.1
- **Server-Side Search:** Media, Downloads, Review pages now use server-side search filtering for better performance
- **Dependencies Fix:** System Health page Dependencies section now works correctly (fixed API path)
- **Codebase Cleanup:** Removed 4 unused files (auto_tagger.py, safe_query_builder.py, youtube_downloader.py, security.py)
- **TypeScript Fixes:** Resolved WebSocket callback type errors, unused variable warnings

#### 11.16.0
- **Semantic Versioning Overhaul:** 246 versions renumbered for correct PATCH/MINOR/MAJOR classification
- **Search Functionality:** Added search boxes to Downloads and Internet Discovery pages
- **Video Downloader Search:** Now uses server-side filtering (searches all pages, not just current)
- **Thumbnail Standardization:** All thumbnails now stored as JPEG, webp converted to jpg

#### 11.13.1
- **Internet Discovery Rename:** Celebrity Discovery renamed to Internet Discovery
- **Config Cleanup:** Removed two unused configuration sections:
  - Search Presets blocked channels (setting was never applied to filtering)
  - Video Streaming native player (toggle was never implemented)

#### 11.13.0
- **Face Recognition Path Fix:** Review page now shows face confidence for all scanned files
  - Root cause: `face_recognition_scans` paths weren't updated when files moved to review
  - `move_module.py` now updates paths alongside perceptual hashes
  - Fixed 140 existing database records
- **Instagram Fallback Patterns:** Added `parse_with_fallbacks()` function for cleaner pattern matching

#### 11.12.0
- **Downloads Page Batch Operations:** Full batch select/action functionality matching Media page
- **Instagram Stories Fix:** Filename pattern corrected from dashes to underscores
- **Dead Code Cleanup:** Removed ~40 lines of disabled face_recognition fallback code
- **Database Cleanup:** Removed 5 empty/orphaned database files

#### 11.11.0
- **Modern UI Styling:**
  - Applied custom scrollbars to scrollable areas across all pages (Dashboard, Discovery, VideoDownloader, Logs, etc.)
  - Added animated number counters on stat cards (Dashboard, Analytics, Monitoring, Scheduler, RecycleBin, etc.)
  - Added thumbnail zoom effects on media grids (Media, Review, RecycleBin, Downloads, VideoDownloader)
  - Applied glass cards with gradient borders on info boxes (Scheduler, Platforms, Scrapers)
  - Smooth theme transition effects when switching light/dark mode
- **Codebase Cleanup:**
  - Removed StyleDemo.tsx demo page and route from App.tsx
  - Removed 6 orphaned frontend components:
    - ForumConfig.tsx (never imported)
    - PlatformConfig.tsx (never imported)
    - TOTPSetupModal.tsx (never imported)
    - VideoLightbox.tsx (never imported)
    - VideoPlayerModal.tsx (never imported)
    - VirtualizedGrid.tsx (never imported)
  - Removed deprecated cleanup-old-logs.sh (replaced by Python version)
  - Removed test scripts: test_face_recognition_with_model.py, clear-phun-test-data.sh
  - Removed archive directory with 4 old maintenance scripts
  - Removed unused modules/instagram/ base class directory
- **Code Quality:**
  - All Python files pass syntax checks (py_compile)
  - All TypeScript files pass type checks (tsc --noEmit)
  - Identified but retained future-facing API methods (parseFilename, clearMonitoringHistory, verifyDuoCallback)

#### 7.0.18
- **Security Fixes:**
  - Fixed hardcoded `secure=False` in 2FA cookie settings (twofa_routes.py) - now uses `settings.SECURE_COOKIES`
  - Added path traversal protection in forum_subprocess_wrapper.py using `Path.name` and `.resolve().relative_to()`
- **Connection Leak Fixes:**
  - Fixed DatabaseAdapter methods (add_to_queue, get_pending_downloads, update_queue_status) with try/finally
  - Fixed health.py database connection leak with try/finally
- **Thread Safety Improvements:**
  - Added double-check locking to discovery_system.py singleton
  - Made SettingsManager thread-safe with RLock for writes
- **Exception Handling:**
  - Replaced bare `except:` clauses with specific exception types in:
    - semantic_search.py (OSError for temp file cleanup)
    - activity_status.py (json.JSONDecodeError, TypeError, ValueError)
    - youtube_channel_monitor.py (ValueError, TypeError for date parsing, OSError for cleanup)
    - video.py (OSError for cache file cleanup)
- **Bug Fixes:**
  - Fixed return type mismatch in coppermine_module.py:730 (return 0 → return ({}, 0))
  - Fixed Immich config KeyError risk using .get() with validation
  - Fixed React useEffect dependency arrays in Review.tsx
- **Signal Handling:**
  - Added SIGTERM/SIGINT handlers to all wrapper scripts (fastdl, imginn, toolzu, snapchat) for graceful cleanup
- **TypeScript Improvements:**
  - Added proper interfaces to api.ts: User, UserPreferences, AppConfig, PlatformConfig, SettingValue
  - Added WebAuthn credential types for passkey authentication
  - Added WebSocketMessage and WebSocketCallback types
  - Replaced `any` types with specific types throughout
- **Cleanup:**
  - Removed stale cookies_backup_20251201 folder
  - Removed redundant NEVER_RESTART_SERVICES.txt (info in CLAUDE.md)
  - Cleaned __pycache__ directories outside venv

#### 7.0.16
- **Video Width Metadata Capture:**
  - Added max_width column to celebrity_discovered_videos and video_download_queue tables
  - YouTube channel monitor captures width alongside resolution from yt-dlp metadata
  - API endpoints return max_width for future aspect ratio features
  - Embedded videos keep 16:9 containers (YouTube/DM handle internal aspect ratio)

#### 7.0.15
- **Forum Filename Date Prefix:**
  - Forum download filenames now include date/time prefix: `YYYYMMDD_HHMMSS_filename.ext`
  - Prevents filename collisions when same generic filename (like '3.jpg') appears in multiple downloads
  - Date extracted from thread title using DateHandler.extract_date_from_text()
  - Falls back to forum post date if no date in title
  - Updated `_get_image_filename()` in forum_downloader.py to accept post_date parameter
- **Post Date Recording:**
  - Forum db_adapter now passes post_date to unified database record_download()
  - Forum subprocess wrapper passes post_date when recording deferred downloads
  - Fixes Post Date showing wrong in lightbox (was falling back to file_inventory.created_date)
- **Proportional Video Sizing:**
  - Video players now resize based on actual video aspect ratio
  - Portrait videos display tall/narrow instead of forced 16:9
  - Local videos use natural sizing: `max-w-[90vw] max-h-[70vh] w-auto h-auto`
  - Embedded videos (YouTube/Dailymotion) keep 16:9 container (iframes can't report ratio)
  - Updated: EnhancedLightbox, VideoLightbox, CelebrityDiscovery, DownloadQueue
- **GIF Handling:**
  - GIF files now treated as images, not videos
  - Removed .gif from isVideo patterns, added to isImage patterns
  - Fixes thumbnail display and lightbox playback for GIFs
  - Updated: Dashboard, Downloads, Media, Review, RecycleBin, Discovery
- **Forum Notification Fix:**
  - Notifications now only sent for files with face matches
  - Files moved to review queue (no face match) don't trigger notifications
  - Uses move_manager.moved_files instead of tracking all downloads
- **Cleanup:**
  - Removed stale 0-byte database files from root: media-downloader.db, media_tracking.db

#### 7.0.13
- **Dailymotion Discovery & Search Presets:**
  - Added Dailymotion channel search as new source type in Search Presets
  - Searches Dailymotion API by celebrity name within specified channel
  - Fetches full metadata via yt-dlp after API discovery
  - Added Dailymotion platform filter on Celebrity Discovery page
  - `discover_and_enrich.py` has `search_dailymotion()` function using Dailymotion API
- **Dailymotion HLS Streaming:**
  - Added HLS.js streaming support for Dailymotion videos in lightbox native player mode
  - Fixed Native player button not working (playerMode started as null, useEffect didn't trigger)
  - Platform-specific button labels now show "Dailymotion" or "YouTube" correctly
  - Dailymotion buttons use cyan color, YouTube buttons use red for visual distinction
  - Applied fixes to both DownloadQueue.tsx and CelebrityDiscovery.tsx
- **Centralized Timestamp Handling:**
  - Fixed forum downloads having inconsistent EXIF timestamps (Immich showing different dates)
  - Root cause: exiftool only set DateTimeOriginal/CreateDate/ModifyDate, not MetadataDate
  - All 5 modules now use comprehensive flags: `-AllDates`, `-MetadataDate`, `-HistoryWhen=`, `-FileModifyDate`
  - MoveManager now calls DateHandler.update_file_timestamps() for centralized EXIF + filesystem updates
  - Forum downloads pass `timestamp=post_date` to MoveManager instead of manual os.utime
- **Dashboard Video Queue Card:**
  - Added Video Queue Processing card showing download progress
  - Shows spinner/paused icon, current title, completed/failed/remaining counts
  - Clickable link to Download Queue page
- **Code Cleanup:**
  - Removed unused `_update_file_timestamps` method from forum_downloader.py (~110 lines)
  - Removed unused `file_timestamp` variable
  - Added date pattern support for underscores in forum titles (e.g., 12_10_2025)
  - Removed empty `media_downloader.db` file from root directory

#### 7.0.12
- **YouTube Scheduler Fix:**
  - Removed two-level scheduling - scheduler now reads `check_interval_hours` directly from YouTube monitor settings
  - Previously scheduler ran every 1 hour, then monitor internally checked if due (redundant)
  - Now scheduler uses configured interval directly (e.g., 6 hours)
- **Search Presets Full Scan:**
  - Added "Full Scan" button to Search Presets page
  - Runs discovery and enrichment for all enabled presets
  - Dashboard shows activity card with real-time progress tracking
  - Added activity manager integration to `discover_and_enrich.py` script
- **Instagram Duplicate Fix:**
  - Fixed critical bug where files sent to recycle (as perceptual duplicates) weren't recorded to database
  - Caused re-downloading on subsequent runs (media_id not in downloads table)
  - Removed `moved > 0` condition - now records pending downloads regardless of moved count
  - Applied fix to imginn, fastdl, and toolzu modules

#### 7.0.11
- **Scheduler UI Improvements:**
  - YouTube Monitor task now displays proper name instead of raw task ID
  - Added platform mappings for `youtube_channel_monitor` and `youtube_monitor`
  - Shows "Channel phrase matching" description instead of "Unknown"

#### 7.0.10
- **Video Download Timezone Fix:**
  - `format_datetime_for_db()` was using `datetime.now()` (local time) instead of UTC
  - Frontend expects UTC timestamps, causing 5-hour offset on Downloads page
  - Changed to `datetime.utcnow()` for consistency with rest of system
  - Fixed existing records in database to match UTC

#### 7.0.9
- **YouTube Monitor Auto-Start Queue Fix:**
  - Manual checks (Check All and single channel) now properly respect `auto_start_queue` setting
  - Changed from unauthenticated HTTP call to direct queue processor access
  - HTTP call was silently failing with 401 Unauthorized
- **Error Notification Context:**
  - Scheduler now sets proper context (platform='system', content_type='error') for error notifications
  - Prevents error notifications from showing stale download metadata
- **Service Documentation:**
  - Added complete service reference to CRITICAL_RULES.md
  - Documents active, disabled, and legacy services

#### 7.0.8
- **YouTube Channel Monitor:**
  - New feature allowing background monitoring of YouTube channels for videos matching phrases
  - Matching videos automatically added to download queue
  - Management UI in Configuration → Platforms with search and alphabetical sorting
  - Channel names converted to @handle format for consistency
- **Thumbnail Fix (webp → jpg):**
  - YouTube's webp thumbnails not available for all videos (especially older ones)
  - Changed `youtube_channel_monitor.py` to prefer jpg thumbnails
  - Converted 279 existing webp URLs to jpg in database
- **Recycle Bin Metadata Restore:**
  - Files restored from recycle bin now correctly restore platform/source metadata
  - Fixed `restore_from_recycle_bin()` in unified_database.py to update file_inventory
- **Error Notification Context Fix:**
  - Error notifications were inheriting stale context from previous download notifications
  - Fixed by clearing `_current_notification_context` after recording in pushover_notifier.py
- **API Testing Helper Scripts:**
  - `/scripts/get-api-token.sh` - Gets token for claude_test account
  - `/scripts/api-call.sh` - Makes authenticated API calls using saved token

#### 7.0.5
- **Notifications 503 Fix:**
  - Implemented IntersectionObserver-based lazy loading for thumbnails
  - Thumbnails only load when scrolling into view (100px preload margin)
  - Failed thumbnails use inline base64 SVG placeholder instead of network request
  - Eliminates rate limit errors from loading dozens of thumbnails simultaneously
- **Silent Exception Handling:**
  - Replaced bare `except:` with specific exception types throughout codebase
  - Added debug logging to previously silent exception handlers
  - Files fixed: api.py, celebrity.py, scrapers.py, forum_downloader.py
  - Exception types now explicit: `ValueError`, `TypeError`, `json.JSONDecodeError`
- **Proxy Rate Limits:**
  - Created new `media_downloader` nginx rate zone (100r/s vs general 10r/s)
  - Increased burst limit to 500 for md.lic.ad
  - Increased connection limit to 200

#### 7.0.3
- **Celebrity Discovery UX:**
  - Default filters: Eva Longoria, Highest Resolution, Not Queued, Unwatched
  - Added Watched/Unwatched bulk action buttons for selected videos
  - Removed redundant Manage Presets dropdown from filter bar
- **Forum Date Fixes:**
  - Forum downloads now use correct post date from thread title
  - DateHandler extracts date from thread title before MoveManager processes files
  - File timestamps properly set via os.utime() matching imginn approach
- **Queue Settings Persistence:**
  - Fixed backend model missing `stop_on_cookie_error`, `send_cookie_notification`, `auto_start_on_restart`
  - Auto-start on restart now reads from correct settings key (`video_queue`)
  - All queue settings now properly save and load
- **Bug Fixes:**
  - Fixed file-not-found error for video titles with special characters (brackets) using glob.escape()
  - Notification attachments now open for files not in file_inventory (os.path.exists fallback)

#### 7.0.2
- **Anti-Bot Protection:**
  - Configurable User-Agent selection (Edge, Chrome, Firefox, Safari, Custom)
  - Rate limiting, throttle detection, sleep between requests
  - Settings UI in Configuration → Downloads
  - User-Agent matches cookie export browser to prevent session invalidation
- **Download Queue Improvements:**
  - Sort toggle: Download Order vs Recently Added
  - Queue displays items in actual download order (priority ASC, oldest first)
  - Pending filter now includes downloading items
  - Defaults to Pending filter on load
- **Reliability:**
  - Stuck downloads automatically reset to pending on API restart
  - Cookie expiration detection stops queue and sends push notification

#### 7.0.1
- **Download Selected Feature:**
  - New "Download Selected" button downloads only checked items
  - Backend endpoint `/processor/start-selected` with ID filtering
  - Supports pause/resume for selected batch
- **Configurable Download Paths:**
  - Base directory now configurable in Configuration → Downloads
  - Stored in settings table, read by universal_video_downloader
  - Platform subdirectories created automatically
- **Bug Fixes:**
  - Mobile menu now respects iOS safe-area-inset-top
  - Fixed route ordering conflict in video_queue.py (/settings vs /{queue_id})
  - Fixed NOT NULL constraint on settings.value_type column

#### 7.0.0 (Major Release)
- **Celebrity Discovery System:**
  - Automatic YouTube video discovery from celebrity channels
  - Bulk add to download queue with metadata
  - Filtering by celebrity, status, date range
- **iOS Mobile-Friendly UI:**
  - Touch-friendly 44px minimum button sizes
  - Bottom-sheet modals for mobile
  - Safe area support for notch devices
  - Responsive layouts across all pages
- **Native Video Player:**
  - Privacy-focused yt-dlp streaming
  - Quality selection (360p-1080p)
  - Toggle between YouTube embed and Native
  - 5-hour URL caching
- **Unified Download Queue:**
  - Bulk operations (select all, retry, pause, delete)
  - Configurable delay between downloads
  - Cookie support from Scrapers settings
  - Rate limiting protection
- **Memory Optimizations:**
  - Lazy loading for face_recognition module
  - Reduced idle memory from 8.5GB to ~100MB

#### 6.52.65
- **Installer Overhaul:**
  - Complete rewrite of install.sh to include ALL services (461 lines)
  - Now creates 7 systemd services:
    - `media-downloader` - Main scheduler service
    - `media-downloader-api` - Web API (port 8000)
    - `media-downloader-frontend` - Web UI (port 5173)
    - `xvfb-media-downloader` - Virtual display for browser automation
    - `media-cache-builder` - Thumbnail cache builder
    - `media-embedding-generator` - CLIP embedding generator
    - `media-celebrity-enrichment` - Celebrity metadata enrichment
  - Creates 3 systemd timers for daily background tasks (3-4 AM)
  - Added FlareSolverr Docker container installation prompt
  - Creates command-line wrapper at `/usr/local/bin/media-downloader`
  - Comprehensive backup creation during uninstall
- **Uninstaller Updated:**
  - Matches install.sh with all 7 services and 3 timers
  - Proper service stop/disable/remove sequence
- **Requirements.txt Updated:**
  - Added gallery-dl>=1.30.0 for forum/Erome downloads
  - Added opencv-python-headless>=4.8.0 for image processing
  - Added torch>=2.0.0 and torchvision>=0.15.0 for CLIP
  - Added PyJWT>=2.8.0 for token handling
  - Added cryptography>=41.0.0 for security
  - Added webauthn>=2.0.0 for passkey support
  - Fixed bcrypt version constraint (<5.0 for passlib compatibility)

#### 6.52.64
- **Video Streaming Settings UI:**
  - Added comprehensive settings section in Configuration → Downloads tab
  - Default Player setting (Native vs YouTube embed)
  - Default Quality setting (360p-1080p or Best)
  - Cache Duration setting (1-5 hours)
  - Proxy Mode toggle for privacy
- **Bug Fixes:**
  - Fixed race condition where quality/player settings weren't honored on load
  - Fixed max_resolution not being passed when adding videos to Download Queue
  - Backfilled resolution data for existing queue items
  - Corrected resolution values storing width instead of height

#### 6.52.63
- **Native Video Player:**
  - Privacy-focused alternative to YouTube embed using yt-dlp streaming
  - Backend /api/video/stream-url endpoint with quality selection
  - Custom video controls (play, pause, seek, mute, fullscreen)
  - Audio sync for separate video/audio streams
  - URL caching (5 hours before expiry)

#### 6.52.60
- **Critical Memory Leak Fixes:**
  - Fixed OOM kills occurring every 10-15 minutes during Instagram downloads
  - Scheduler was growing from ~800MB to 8GB+ due to unreleased image memory
  - Root cause: PIL Image.open() and cv2.imread() not being properly closed
- **Files Fixed:**
  - `fastdl_module.py` - EXIF update function now uses context managers
  - `instagram_repost_detector.py` - Hash calculation and OCR functions fixed
  - `semantic_search.py` - Image embedding generation uses context managers
  - `auto_tagger.py` - Image embedding generation uses context managers
  - `face_recognition_module.py` - Thumbnail generation and re-training loops fixed
  - `media-downloader.py` - Added gc.collect() after each user in FastDL loop
- **Error Banner Fix:**
  - Dropdown now shows only errors since last dashboard visit (matches displayed count)
  - Added `sinceVisit` parameter to `getRecentErrors()` API call

#### 6.52.59
- **UI Consistency Update:**
  - Standardized all page headers with consistent icon + `text-3xl font-bold` design pattern
  - Updated navigation menu icons in App.tsx to match corresponding page header icons
  - Manual Import page layout fixed (changed from `p-6` to `space-y-6` for consistent spacing)
  - Celebrity Discovery page simplified with single "Manage Presets" button linking to Search Presets
  - Removed unused `searchAllMutation` from CelebrityDiscovery.tsx
- **Navigation Icon Updates:**
  - Dashboard: LayoutDashboard
  - Downloads: FolderDown
  - Media: GalleryHorizontalEnd
  - Review: ClipboardList
  - Video Downloader: Clapperboard
  - Manual Import: FolderInput
  - Face Recognition: ScanFace
  - Scheduler: CalendarClock
  - Analytics: BarChart3
  - System Health: HeartPulse
  - Monitoring: Gauge
  - Notifications: BellRing
  - Platforms: Layers
  - Scrapers: Bot
  - Logs: ScrollText
  - Configuration: Settings

#### 6.52.58
- **Download Queue Processor:**
  - New queue processor with Start/Pause/Stop controls for batch downloads
  - Downloads pending videos sequentially with real-time progress updates
  - Status banner shows current download, completed/failed counts, remaining items
  - Fixed Celebrity Discovery "Add to Queue" inserting into wrong table
  - Fixed Download Queue page calling wrong API endpoints (/queue → /video-queue)
- **Celebrity Discovery Enhancements:**
  - Added "Not Queued" status filter (shows new/ignored/watched videos)
  - Resolution badge on thumbnails (e.g., "1080p", "720p")

#### 6.52.57
- **Recycle Bin Filters:**
  - Platform and Source filter dropdowns added to Recycle Bin page
  - New /api/recycle/filters endpoint for dynamic filter options
  - Fixed 'fastdl' platform entries corrected to 'instagram'

#### 6.52.55
- **Error Banner Fixes:**
  - Removed redundant `process_and_store_errors()` that was re-adding dismissed errors
  - Errors now only recorded in real-time by `universal_logger.py`
  - Fixed timestamp format consistency (all ISO format with T separator)
  - Dismissed errors no longer reappear unless they actually recur
- **Instagram Video Downloader Fixes:**
  - Fixed `nest_asyncio` / `uvloop` conflict preventing downloads
  - Username now extracted from post page (no more "unknown" filenames)
  - Activity status updates to show real username once extracted
- **API Memory Optimization (6.52.54):**
  - FaceRecognitionModule cached as singleton (saves ~400MB per request)
  - API memory reduced from 2.1GB to ~113MB

#### 6.52.47
- **Dashboard Error Display Fixed:**
  - Errors now show ALL unviewed items, not just since last visit
  - Error count reflects total unviewed regardless of visit time
- **Error Monitoring UI:**
  - Settings added to Configuration page Monitoring tab
  - Configurable push alert delay, dashboard banner, retention

#### 6.52.44-6.52.46
- **MAJOR: API Modularization Complete:**
  - api.py reduced from 10,465 to 828 lines (92% reduction)
  - 17 modular routers created in `web/backend/routers/`
  - Shared state via AppState singleton in `core/dependencies.py`
  - All 170+ endpoints migrated and tested
- Error monitoring settings API endpoints added
- Timezone bug fixed in error alert 24-hour check

#### 6.52.49
- React.lazy() code splitting for all 21 pages (40-50% faster initial load)
- Reduced Dashboard polling (stats 30s→60s, activity 2s→5s)
- React Query staleTime/gcTime caching (30s/5min)
- Memoized filteredMedia with useMemo() in Media.tsx
- Cache-Control headers on thumbnails/previews (1 hour)
- Initial bundle reduced from ~600KB to ~50KB app shell

#### 6.52.48
- Real-time error notifications via WebSocket (error_alert broadcast)
- ErrorBanner WebSocket listener for instant error updates
- Error polling reduced from 60s to 15s
- Scheduler OOM crashes fixed (face recognition disabled in subprocesses)
- Missing Tuple import in settings_manager.py fixed
- Added /api/downloads/filters route alias
- Memory optimization with gc.collect() calls

#### 6.52.53
- **Critical OOM Memory Fix:**
  - Fixed memory leak in face recognition causing 9GB+ memory usage and OOM kills
  - Added explicit `del` and `gc.collect()` in `detect_faces_insightface()`
  - Added memory cleanup in `detect_faces()` fallback method
  - Added `gc.collect()` after video and image processing in move_module.py
  - Memory now stable at ~743MB (was growing to 9GB+ before fix)
  - Peak memory reduced from 9GB+ to 1.2GB

#### 11.22.1
- **Podcast Appearances Fixes & Enhancements:**
  - Fixed episode list endpoint to use query parameters instead of path parameters (handles special characters like ó in Gomez-Rejón)
  - Fixed episode count calculation for podcast appearances (was hardcoded to 1, preventing full episode list display)
  - Modal episode list now excludes current episode to prevent duplication
  - Audio player now preloads metadata to show episode duration before playback starts
  - Mobile app now preserves page state when backgrounding/resuming (changed MemoryRouter to BrowserRouter)
  - RSS feed backfill automatically runs after Podchaser sync for Omny.fm-hosted podcasts (29/32 episodes now have audio URLs)

#### 11.28.0
- **Dashboard New Items Cards:**
  - Created `dashboard.py` router - new /api/dashboard/recent-items endpoint
  - Created `RecentItemsCard.tsx` component - reusable card with thumbnail grid
  - Three cards: "New in Media", "New in Review", "New in Internet Discovery"
  - Dismissible with localStorage persistence - reappears when new items arrive
  - Hover overlay with circular action buttons matching source page functionality
  - EnhancedLightbox integration for media/review, video preview modal for discovery
  - Session-based tracking removes actioned items from display
  - YouTube link conditional on platform (not shown for Easynews)

#### 6.52.52
- **Instagram Cross-Module Duplicate Detection:**
  - Created `instagram_utils.py` - centralized utility module
  - `extract_instagram_media_id()` - extracts 18-digit media IDs from filenames
  - `record_instagram_download()` - centralized database recording with normalized IDs
  - `is_instagram_downloaded()` - cross-module duplicate check
  - imginn module refactored to use unified_db directly (removed adapter dependency)
  - Removed unused adapters from unified_database.py (~200 lines)
  - Fixed regex pattern for media ID extraction (underscore boundaries)

#### 6.52.47
- Dashboard now shows ALL unviewed errors (not just since last visit)
- Error count reflects total unviewed regardless of visit time

#### 6.52.38-6.52.43
- **Phase 1 Refactoring Complete:**
  - Custom exception classes (20+ types)
  - Unified configuration manager
  - Standardized response formats
  - ISO 8601 date utilities
  - Router infrastructure created
  - Base Instagram downloader class
- Technical debt items resolved: TD-002, TD-003, TD-008, TD-015, TD-016
- All 16 routers enabled and registered

#### 6.52.37
- Lightbox dimensions fetching implemented
- Real-time error alerts via WebSocket
- Thread safety improvements in Playwright
- Database indexes added for performance

#### 6.52.62
- Fixed recycle bin restore error (cannot access local variable 'get_app_state')

#### 6.52.61
- Fixed additional memory leak in face_recognition_module.py add_reference_face
- Scheduler now stable at ~2GB for 3+ hours (previously crashed within 10-15 minutes)

#### 6.52.60
- Fixed critical memory leaks causing OOM kills during Instagram downloads
- Fixed Image.open() leaks in fastdl, repost detector, semantic search, auto_tagger
- Fixed cv2.imread() leaks in face_recognition_module thumbnail and re-training
- Added gc.collect() after each user in FastDL download loop
- Error banner dropdown now shows only errors since last dashboard visit

#### 12.11.0
- **Config Page Redesign:** Replaced sidebar + pill tab navigation with horizontal underline tabs matching Configuration.tsx and Paid Content Settings.tsx. Removed sticky header with ArrowLeft back button, replaced with standard `text-2xl sm:text-3xl font-bold` h1 heading with `text-slate-500` icon and `text-slate-600 dark:text-slate-400` subtitle. Page root element changed from `min-h-screen bg-background > container mx-auto px-4 py-6` to simple `space-y-6` div, inheriting padding from parent `<main>` layout (fixes breadcrumb spacing). Content wrapper changed from `max-w-3xl` to full-width. Removed `ArrowLeft` import from lucide-react.
- **Multi-URL Import (Frontend):** Added `Link` icon and `urls` textarea state to `PrivateGalleryUploadModal`. URL textarea shown below drop zone when no files selected, with placeholder listing supported hosts. New `importUrlsMutation` calls `api.privateGallery.importUrls()` and builds `ThumbnailProgressItem[]` from URL basenames for progress tracking. `handleSubmit` now routes: files → upload, URLs → import, existing media → create post. Footer shows "X URLs entered" when URLs present. Submit button shows "Import" with Link icon for URL mode.
- **Multi-URL Import (Backend):** Fixed `ImportUrlRequest` model: `tag_ids` changed from `Field(..., min_items=1)` to `List[int] = []`. Added `POST /import-urls` endpoint with post creation, job tracking, and background task launch. Added `_import_urls_to_gallery_background()`: iterates URLs, tries `FileHostDownloader.detect_host()` first for Bunkr/Pixeldrain/Gofile/Cyberdrop (via `asyncio.run()`), falls back to direct `requests.get()` with content-type validation. Each downloaded file: hash → duplicate check → `_get_file_info()` → date extraction → `_generate_thumbnail()` → encrypt file + thumb → DB insert with `source_type='import'` and encrypted source URL. Cleans up empty post if all files fail.
- **API Client:** Added `importUrls` method to `privateGallery` namespace in api.ts. Updated `getJobStatus` operation type union to include `'import'`.
- **Files Modified:** Config.tsx, PrivateGalleryUploadModal.tsx, api.ts, private_gallery.py

#### 12.10.0
- **Slideshow Mode:** Added play/pause button, configurable interval selector (3s/5s/8s/10s), and Space keyboard shortcut to both `PrivateGalleryBundleLightbox` and `BundleLightbox`. Images auto-advance on timer; videos play to completion via `ended` event before advancing. Slideshow collapses sidebar and hides metadata panel on start. Manual navigation resets the timer.
- **Shuffle Mode:** Fisher-Yates shuffle with cyan toggle indicator. Keeps current item at front of shuffled sequence. Shuffle order rebuilt when toggling on. Position indicator shows "(shuffled)" suffix.
- **Context-Aware Slideshow Button:** Gallery view slideshows all filtered media across all posts (flat attachment list with `attachmentPostMap` for post context). Feed/Social view slideshows only the currently selected post's attachments. Button hidden when 0 or 1 media items.
- **Server-Side total_media:** Added `total_media` count to Private Gallery API (`private_gallery.py` - counts media items across filtered posts with file_type support) and Paid Content API (`paid_content.py` + `db_adapter.py` `get_media_count()` method). Frontend uses this for accurate lightbox position indicators instead of loaded page count.
- **Filter Bar Labels:** Shows "X items" with `total_media` count in gallery view, "X posts" with `total` count in feed/social view, for both Private Gallery and Paid Content pages.
- **Gallery Item Ordering Fix:** Created `galleryLightboxItems` useMemo in Private Gallery that follows the same `media_date` sort order as the gallery grid, fixing position indicator mismatch.
- **Paid Content Gallery Refresh:** Added scroll-to-top on filter change (matching Private Gallery). Added data-change watcher effect that auto-loads more pages when gallery content doesn't fill the scroll container after filter change.
- **Infinite Scroll Integration:** Lightbox preloads next page when within 5 items of end. Slideshow calls `onLoadMore` instead of stopping when more pages available. `totalCount`/`hasMore`/`onLoadMore` props added to both lightbox components.
- **Slideshow Controls Visibility:** Play/pause, shuffle button, and interval selector all hidden when `(totalCount || items.length) <= 1`. Position indicator already hid for single items.
- **Files Modified:** PrivateGalleryBundleLightbox.tsx (+slideshow/shuffle/infinite scroll), BundleLightbox.tsx (+same), Gallery.tsx (+slideshow button, galleryLightboxItems, totalMediaItems), Feed.tsx (+slideshow button, flat list builder, scroll-to-top, auto-load), api.ts (+total_media type), private_gallery.py (+total_media query), paid_content.py (+total_media response), db_adapter.py (+get_media_count)

#### 12.9.1
- **Reliable 4K HLS Downloads:** Replaced ffmpeg HLS networking (stalled at ~1.2GB) with direct segment-by-segment download using aiohttp. Downloads 5 segments concurrently with 120s timeout and 3 retries per segment. ffmpeg only used for local concatenation/remux.
- **Duration Fix:** Added `-t` flag to ffmpeg remux to trim to exact playlist duration, eliminating ~7s padding accumulation from HLS segment boundaries.
- **Quality Recheck Fix:** `download_pending_for_creator()` was breaking because creator_id wasn't registered in `active_paid_content_syncs`. Fixed by registering before download call.
- **Quality Recheck Upgrade Flow:** Changed from in-place URL swap to delete old post/attachment + trigger full Fansly re-sync for clean 4K downloads.
- **Skipped Attachment Fix:** `upsert_attachment()` now handles 'skipped' status like 'failed' - resets to 'pending' with fresh URL on sync.
- **Master Playlist Resolution:** Added `_resolve_best_hls_variant()` to `fansly_direct_client.py` to select highest resolution variant from HLS master playlists upstream in `get_posts()`.
- **Remux Progress:** Download queue shows remux percentage during ffmpeg segment concatenation.
- **Dependency:** Added `aiohttp>=3.9.0` to requirements.txt (10 modules depend on it).
- **Files Modified:** scraper.py (segment download + remux), fansly_direct_client.py (master playlist resolution), db_adapter.py (skipped status), paid_content.py (recheck flow), requirements.txt (aiohttp)

#### 12.9.0
- **ThumbnailProgressModal:** New shared component replacing text-based progress displays with visual thumbnail-grid progress tracking. Shows per-file status overlays (pending/processing/success/error/duplicate) with progress bar, status chips, and collapsible error details. Used across Copy to Gallery, Upload, and Manual Import.
- **Async Job Tracking:** Private Gallery copy and upload operations now run asynchronously via FastAPI BackgroundTasks. In-memory job dict with threading Lock (mirrors manual_import.py pattern). New `GET /private-gallery/job-status/{job_id}` endpoint with dual auth.
- **Async Copy:** Extracted per-file copy loop into `_copy_to_gallery_background()`. Endpoint returns `{job_id, post_id, status, total_files}` immediately. Frontend polls at 500ms for per-file updates.
- **Async Upload:** Files saved to temp before responding (UploadFile invalidated after response). Extracted processing loop into `_upload_to_gallery_background()`. Two-phase frontend: XHR upload progress, then ThumbnailProgressModal for server-side processing.
- **Instagram Filename Patterns:** Added 3 new patterns to `INSTAGRAM_PATTERNS` for browser extension formats: `{username}-video-{id}-{YYYYMMDD}_{HHMMSS}_{description}`, `{username}-photo-{id}-...`, `{username}-{id}-...` (order matters - specific patterns before generic).
- **Video Thumbnail Fix:** `generate_video_thumbnail()` now tries seek at 1s then falls back to 0s for short videos. Moved `-ss` before `-i` for faster input seeking.
- **ALLOWED_PATHS Fix:** Added `/opt/media-downloader/temp/manual_import` to allow thumbnail generation for temp files during manual import.
- **ThrottledImage Priority:** Added `priority` prop to bypass IntersectionObserver in nested scroll containers (modal grids).
- **Files Modified:** private_gallery.py (+1,896 lines for job tracking and async), ThumbnailProgressModal.tsx (new, 245 lines), CopyToGalleryModal.tsx (polling integration), PrivateGalleryUploadModal.tsx (two-phase progress), ManualImport.tsx (swapped BatchProgressModal), api.ts (getJobStatus, updated return types), filename_parser.py (+3 patterns), core/utils.py (video thumbnail fix + ALLOWED_PATHS)

#### 12.8.0
- Tag sorting by usage count (most-used first) across paid content and private gallery
- Copy to Private Gallery from Recycle Bin with original filename preservation
- Redesigned CopyToGalleryModal: optional tags, TagSearchSelector, thumbnail grid, person default tags
- Multi-page selection persistence for copy operations
- EXIF orientation fix for private gallery thumbnails (ImageOps.exif_transpose)
- Double-submission prevention with ref-based guard in copy modal
- Nginx proxy timeout fix: dedicated /api/private-gallery/ location with 1h timeout
- Tag migration fix: syncs ALL paid content tags, not just used ones
- Feature-gated Copy to Private button (respects Features Management)
- Fixed Gallery.tsx React Query cache pollution (paidContent.getTags → privateGallery.getTags)

#### 6.52.32
- N+1 query pattern fixed
- Batch face lookup implemented
- Thread-safe WebSocket broadcasting

#### 6.52.29-6.52.31
- Dimensions captured on file move
- Recycle bin dimensions support
- Various lightbox fixes

---

## Appendix: Quick Reference

### File Locations

| Component | Path |
|-----------|------|
| Main CLI | `/opt/media-downloader/media-downloader.py` |
| API Main | `/opt/media-downloader/web/backend/api.py` |
| API Routers | `/opt/media-downloader/web/backend/routers/` |
| API Core | `/opt/media-downloader/web/backend/core/` |
| Frontend | `/opt/media-downloader/web/frontend/src/` |
| Modules | `/opt/media-downloader/modules/` |
| Database | PostgreSQL `media_downloader` (via pg_adapter) |
| Logs | `/opt/media-downloader/logs/` |
| Config | `/opt/media-downloader/.env` |

### Key Commands

```bash
# Check version
cat /opt/media-downloader/VERSION

# View recent logs
tail -100 /opt/media-downloader/logs/$(date +%Y-%m-%d).log

# Database stats
PGPASSWORD=... psql -h localhost -U media_downloader -d media_downloader -c "SELECT COUNT(*) FROM downloads;"

# Service status
systemctl status media-downloader

# Build frontend
cd /opt/media-downloader/web/frontend && npm run build

# Check API health
curl -s http://localhost:8000/api/health | jq
```

---

**Document Maintainer:** Code Review System
**Review Schedule:** Update after each major version release