LIComputerGuy/media-downloader
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
49e72207bf7526c6f9a5bd67a49d1b6146a9da5c
media-downloader/docs/CODE_MAP.md
Todd 0d7b2b1aab Initial commit 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 22:42:55 -04:00

14 KiB
Raw Blame History

Code Map - Feature Location Reference

This document provides a quick reference for locating features and components when making modifications to the Media Downloader application.

Last Updated: 2026-02-10 (v12.12.1)

Core Backend Files

Database Layer

  • File: /opt/media-downloader/modules/unified_database.py
  • Contains:
    • All database table schemas (downloads, media_gallery, review_queue, recycle_bin, etc.)
    • CRUD operations for all tables
    • Database connection pooling (DatabasePool class)
    • Settings management (get_setting, set_setting)
    • Recycle bin operations (move_to_recycle_bin, restore_from_recycle_bin)
    • Face recognition metadata storage

API Endpoints

  • Entry Point: /opt/media-downloader/web/backend/api.py (828 lines - router registration)
  • Routers: /opt/media-downloader/web/backend/routers/ (26 routers)
    • paid_content.py - Paid content CRUD, services, creators, feed, messages, OnlyFans/Fansly setup, health checks
    • private_gallery.py - Private gallery auth, media, persons, encryption, features, URL import
    • media.py - Media serving, thumbnails, gallery
    • face.py - Face recognition endpoints
    • downloads.py - Download history, analytics
    • review.py - Review queue management
    • config.py - Configuration management
    • scheduler.py - Scheduler control
    • And 18 more routers (auth, health, recycle, stats, discovery, video, etc.)

Feature-Specific Modules

Face Recognition

  • Main Module: /opt/media-downloader/modules/face_recognition_module.py
  • Detection Module: /opt/media-downloader/modules/face_detection_module.py
  • Database Manager: /opt/media-downloader/modules/face_recognition_db.py
  • Related Scripts:
    • /opt/media-downloader/scripts/add_reference_face.py - Add reference faces
    • /opt/media-downloader/scripts/batch_compare_faces.py - Batch comparison
    • /opt/media-downloader/scripts/list_reference_faces.py - List faces
    • /opt/media-downloader/scripts/delete_reference_face.py - Delete faces
  • UI Components:
    • Frontend API calls: /opt/media-downloader/web/frontend/src/lib/api.ts
    • Face recognition page: Check App.tsx for routing

File Movement & Organization

  • File: /opt/media-downloader/modules/move_module.py
  • Contains:
    • File movement logic (move_file)
    • Batch move context management
    • Review queue handling
    • Notification tracking for moved files
    • Separate tracking for review queue vs matched files
    • Integration with face recognition workflow

Push Notifications

  • File: /opt/media-downloader/modules/pushover_notifier.py
  • Contains:
    • Pushover API integration
    • Batch download notifications
    • Review queue notifications (separate from regular downloads)
    • Platform-specific icons and formatting
    • Image attachment support
    • Priority settings

Media Download Modules

  • Instagram: /opt/media-downloader/modules/instagram_module.py
  • Reddit: /opt/media-downloader/modules/reddit_module.py
  • TikTok: /opt/media-downloader/modules/tiktok_module.py
  • Bunkr: /opt/media-downloader/modules/bunkr_module.py
  • X/Twitter: /opt/media-downloader/modules/x_module.py

Utilities

  • Filename Cleaner: /opt/media-downloader/utilities/filename_cleaner.py
  • Metadata Manager: /opt/media-downloader/modules/metadata_manager.py
  • Cache Builder: /opt/media-downloader/utilities/cache_builder.py

Frontend Structure

Main Application Files

  • App Entry: /opt/media-downloader/web/frontend/src/App.tsx

    • Main routing configuration
    • Navigation menu (Downloads, Media, Review, System dropdowns)
    • WebSocket connection management
    • Global notification handling

  • API Client: /opt/media-downloader/web/frontend/src/lib/api.ts

    • All API call definitions
    • Authentication token management
    • Request/response handling

Page Components

Downloads Page

  • File: /opt/media-downloader/web/frontend/src/pages/Downloads.tsx
  • Features:
    • Comprehensive filter system (search, platform, media type, face recognition)
    • Advanced filters (date range, size range, sort options)
    • Grid/List view toggle
    • Batch operations
    • File preview modal

Media Gallery Page

  • File: /opt/media-downloader/web/frontend/src/pages/Media.tsx
  • Features:
    • Media browsing and organization
    • Batch delete operations
    • File viewing/download
    • Basic filtering (needs upgrade to match Downloads page)

Review Queue Page

  • File: /opt/media-downloader/web/frontend/src/pages/ReviewQueue.tsx
  • Features:
    • Files awaiting manual review (no face match)
    • Move to media gallery
    • Delete files
    • Face recognition results display

Recycle Bin Page

  • File: /opt/media-downloader/web/frontend/src/pages/RecycleBin.tsx
  • Features:
    • View deleted files from all sources (downloads, media, review)
    • Restore files to original location
    • Permanently delete files
    • Batch operations
    • Statistics dashboard
    • Filtering by source

Configuration Page

  • File: /opt/media-downloader/web/frontend/src/pages/Config.tsx
  • Features:
    • Application settings management
    • Platform credentials
    • Face recognition settings
    • Notification settings
    • Directory settings

Other Pages

  • /opt/media-downloader/web/frontend/src/pages/ChangeLog.tsx - Version history
  • /opt/media-downloader/web/frontend/src/pages/Logs.tsx - System logs viewer
  • /opt/media-downloader/web/frontend/src/pages/Health.tsx - Health monitoring

UI Libraries & Utilities

  • Notification Manager: /opt/media-downloader/web/frontend/src/lib/notificationManager.ts
    • Toast notifications
    • Success/error/info messages
  • React Query: Used throughout for data fetching and caching
  • Tailwind CSS: Styling framework (configured in tailwind.config.js)

Configuration Files

Application Settings

  • Database Settings: Stored in SQLite via SettingsManager (preferred method)

    • Access via: app_state.settings.get('key') or app_state.settings.set('key', value)
    • Settings categories: general, face_recognition, notifications, recycle_bin, etc.

  • Legacy JSON Config: /opt/media-downloader/config/settings.json

    • Being phased out - DO NOT ADD NEW SETTINGS HERE
    • Use database settings instead

Version Management

  • Version File: /opt/media-downloader/VERSION - Single source of truth
  • Package.json: /opt/media-downloader/web/frontend/package.json - Frontend version
  • README: /opt/media-downloader/README.md - Documentation version
  • API Version: Set in /opt/media-downloader/web/backend/api.py (FastAPI app)

Changelog

  • JSON Format: /opt/media-downloader/data/changelog.json - Structured changelog for API
  • Markdown Format: /opt/media-downloader/docs/CHANGELOG.md - Human-readable changelog

System Scripts

Maintenance Scripts

  • /opt/media-downloader/scripts/create-version-backup.sh - Creates timestamped backups
  • /opt/media-downloader/scripts/check-updates.sh - Checks for available updates

Database Scripts

  • /opt/media-downloader/scripts/repair-parent-chains.js - Fixes backup parent chains

Common Modification Scenarios

Adding a New API Endpoint

  1. Add endpoint function to /opt/media-downloader/web/backend/api.py
  2. Add corresponding database method to /opt/media-downloader/modules/unified_database.py (if needed)
  3. Add API client function to /opt/media-downloader/web/frontend/src/lib/api.ts
  4. Use in frontend component with React Query

Adding a New Page

  1. Create component in /opt/media-downloader/web/frontend/src/pages/YourPage.tsx
  2. Add route in /opt/media-downloader/web/frontend/src/App.tsx
  3. Add navigation menu item in App.tsx (if needed)
  4. Import required icons from 'lucide-react'

Modifying Download Behavior

  1. Platform-specific logic: /opt/media-downloader/modules/{platform}_module.py
  2. File movement logic: /opt/media-downloader/modules/move_module.py
  3. Face recognition integration: /opt/media-downloader/modules/face_recognition_module.py
  4. Metadata storage: /opt/media-downloader/modules/metadata_manager.py

Modifying Notifications

  1. Backend notification logic: /opt/media-downloader/modules/pushover_notifier.py
  2. WebSocket broadcasts: /opt/media-downloader/web/backend/api.py (ConnectionManager)
  3. Frontend toast handling: /opt/media-downloader/web/frontend/src/lib/notificationManager.ts
  4. Component notification listeners: Individual page components

Modifying Face Recognition

  1. Core recognition: /opt/media-downloader/modules/face_recognition_module.py
  2. Detection: /opt/media-downloader/modules/face_detection_module.py
  3. Database storage: /opt/media-downloader/modules/face_recognition_db.py
  4. API endpoints: /opt/media-downloader/web/backend/api.py (search for "face")
  5. Reference face scripts: /opt/media-downloader/scripts/ (face-related scripts)

Modifying Recycle Bin

  1. Database operations: /opt/media-downloader/modules/unified_database.py
    • move_to_recycle_bin(), restore_from_recycle_bin(), empty_recycle_bin()
  2. API endpoints: /opt/media-downloader/web/backend/api.py (search for "/api/recycle")
  3. UI component: /opt/media-downloader/web/frontend/src/pages/RecycleBin.tsx
  4. Delete operations: Update delete endpoints in api.py to call move_to_recycle_bin()

Adding New Settings

  1. Initialize in API startup: /opt/media-downloader/web/backend/api.py (lifespan function) 
    if not app_state.settings.get('your_setting'):
    app_state.settings.set('your_setting', default_value, category='category', description='desc')
  2. Add UI controls: /opt/media-downloader/web/frontend/src/pages/Config.tsx
  3. Add API endpoints: /opt/media-downloader/web/backend/api.py (if needed)

Updating Version

  1. Update /opt/media-downloader/VERSION (primary source)
  2. Update /opt/media-downloader/README.md (version badge)
  3. Update /opt/media-downloader/web/frontend/package.json (version field)
  4. Update API version in /opt/media-downloader/web/backend/api.py
  5. Update App.tsx version display
  6. Add entry to /opt/media-downloader/data/changelog.json
  7. Add entry to /opt/media-downloader/docs/CHANGELOG.md
  8. Run /opt/media-downloader/scripts/create-version-backup.sh

Database Schema Quick Reference

Core Tables

  • downloads - Downloaded files tracking
  • media_gallery - Organized media files
  • review_queue - Files awaiting manual review
  • recycle_bin - Soft-deleted files (UUID-based storage)
  • users - User accounts
  • settings - Application settings (key-value store)
  • face_recognition_db - Reference faces and metadata

Recycle Bin Schema

CREATE TABLE recycle_bin (
    id TEXT PRIMARY KEY,              -- UUID for storage
    original_path TEXT NOT NULL,      -- Full path for restore
    original_filename TEXT NOT NULL,  -- Display name
    recycle_path TEXT NOT NULL,       -- Current location
    file_extension TEXT,              -- .jpg, .mp4, etc.
    file_size INTEGER,                -- Bytes
    original_mtime REAL,              -- Preserved timestamp
    deleted_from TEXT NOT NULL,       -- 'downloads', 'media', 'review'
    deleted_at DATETIME,              -- When deleted
    deleted_by TEXT,                  -- Username
    metadata TEXT,                    -- JSON metadata
    restore_count INTEGER DEFAULT 0   -- Times restored
)

Directory Structure

/opt/media-downloader/
├── config/               - Configuration files (legacy JSON - avoid)
├── data/                 - Application data
│   ├── backup_cache.db   - Main SQLite database
│   └── changelog.json    - Structured changelog
├── docs/                 - Documentation (keep all docs here)
├── logs/                 - Application logs
├── modules/              - Python backend modules
├── scripts/              - Utility scripts
├── utilities/            - Helper utilities
└── web/
    ├── backend/          - FastAPI application
    │   └── api.py        - Main API file
    └── frontend/         - React application
        └── src/
            ├── lib/      - Utilities (api.ts, notificationManager.ts)
            └── pages/    - Page components

Quick Reference Cheat Sheet

Feature Backend File Frontend File API Endpoint
Downloads modules/*_module.py pages/Downloads.tsx /api/downloads/*
Media Gallery modules/unified_database.py pages/Media.tsx /api/media/*
Review Queue modules/move_module.py pages/ReviewQueue.tsx /api/review/*
Recycle Bin modules/unified_database.py pages/RecycleBin.tsx /api/recycle/*
Face Recognition modules/face_recognition_module.py N/A /api/face/*
Notifications modules/pushover_notifier.py lib/notificationManager.ts N/A
Settings modules/unified_database.py pages/Config.tsx /api/settings/*
Users web/backend/api.py N/A /api/auth/, /api/users/

Tips for Making Modifications

  1. Always use database settings - Don't add to JSON config files
  2. Update version numbers - Follow VERSION_UPDATE_CHECKLIST.md
  3. Test Python syntax - Run python3 -m py_compile <file> before committing
  4. Test TypeScript - Run npm run type-check in web/frontend/
  5. Rebuild frontend - Run npm run build after changes
  6. Restart services - sudo systemctl restart media-downloader-api and media-downloader-web
  7. Create backups - Run scripts/create-version-backup.sh before major changes
  8. Update changelog - Add entries to both changelog.json and CHANGELOG.md

Related Documentation

  • CHANGELOG.md - Version history and release notes
  • VERSION_UPDATE_CHECKLIST.md - Step-by-step version update process
  • FACE_RECOGNITION.md - Face recognition feature documentation
  • NOTIFICATIONS.md - Notification system documentation
  • REVIEW_QUEUE_STRUCTURE.md - Review queue architecture
  • FEATURE_ROADMAP_2025.md - Planned features and improvements
Reference in New Issue View Git Blame Copy Permalink