Initial commit

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 22:42:55 -04:00
commit 0d7b2b1aab
389 changed files with 280296 additions and 0 deletions
--- a/docs/CODE_MAP.md
+++ b/docs/CODE_MAP.md
@@ -0,0 +1,344 @@
+# Code Map - Feature Location Reference
+
+This document provides a quick reference for locating features and components when making modifications to the Media Downloader application.
+
+Last Updated: 2026-02-10 (v12.12.1)
+
+---
+
+## Core Backend Files
+
+### Database Layer
+- **File**: `/opt/media-downloader/modules/unified_database.py`
+- **Contains**:
+  - All database table schemas (downloads, media_gallery, review_queue, recycle_bin, etc.)
+  - CRUD operations for all tables
+  - Database connection pooling (DatabasePool class)
+  - Settings management (get_setting, set_setting)
+  - Recycle bin operations (move_to_recycle_bin, restore_from_recycle_bin)
+  - Face recognition metadata storage
+
+### API Endpoints
+- **Entry Point**: `/opt/media-downloader/web/backend/api.py` (828 lines - router registration)
+- **Routers**: `/opt/media-downloader/web/backend/routers/` (26 routers)
+  - `paid_content.py` - Paid content CRUD, services, creators, feed, messages, OnlyFans/Fansly setup, health checks
+  - `private_gallery.py` - Private gallery auth, media, persons, encryption, features, URL import
+  - `media.py` - Media serving, thumbnails, gallery
+  - `face.py` - Face recognition endpoints
+  - `downloads.py` - Download history, analytics
+  - `review.py` - Review queue management
+  - `config.py` - Configuration management
+  - `scheduler.py` - Scheduler control
+  - And 18 more routers (auth, health, recycle, stats, discovery, video, etc.)
+
+---
+
+## Feature-Specific Modules
+
+### Face Recognition
+- **Main Module**: `/opt/media-downloader/modules/face_recognition_module.py`
+- **Detection Module**: `/opt/media-downloader/modules/face_detection_module.py`
+- **Database Manager**: `/opt/media-downloader/modules/face_recognition_db.py`
+- **Related Scripts**:
+  - `/opt/media-downloader/scripts/add_reference_face.py` - Add reference faces
+  - `/opt/media-downloader/scripts/batch_compare_faces.py` - Batch comparison
+  - `/opt/media-downloader/scripts/list_reference_faces.py` - List faces
+  - `/opt/media-downloader/scripts/delete_reference_face.py` - Delete faces
+- **UI Components**:
+  - Frontend API calls: `/opt/media-downloader/web/frontend/src/lib/api.ts`
+  - Face recognition page: Check App.tsx for routing
+
+### File Movement & Organization
+- **File**: `/opt/media-downloader/modules/move_module.py`
+- **Contains**:
+  - File movement logic (move_file)
+  - Batch move context management
+  - Review queue handling
+  - Notification tracking for moved files
+  - Separate tracking for review queue vs matched files
+  - Integration with face recognition workflow
+
+### Push Notifications
+- **File**: `/opt/media-downloader/modules/pushover_notifier.py`
+- **Contains**:
+  - Pushover API integration
+  - Batch download notifications
+  - Review queue notifications (separate from regular downloads)
+  - Platform-specific icons and formatting
+  - Image attachment support
+  - Priority settings
+
+### Media Download Modules
+- **Instagram**: `/opt/media-downloader/modules/instagram_module.py`
+- **Reddit**: `/opt/media-downloader/modules/reddit_module.py`
+- **TikTok**: `/opt/media-downloader/modules/tiktok_module.py`
+- **Bunkr**: `/opt/media-downloader/modules/bunkr_module.py`
+- **X/Twitter**: `/opt/media-downloader/modules/x_module.py`
+
+### Utilities
+- **Filename Cleaner**: `/opt/media-downloader/utilities/filename_cleaner.py`
+- **Metadata Manager**: `/opt/media-downloader/modules/metadata_manager.py`
+- **Cache Builder**: `/opt/media-downloader/utilities/cache_builder.py`
+
+---
+
+## Frontend Structure
+
+### Main Application Files
+- **App Entry**: `/opt/media-downloader/web/frontend/src/App.tsx`
+  - Main routing configuration
+  - Navigation menu (Downloads, Media, Review, System dropdowns)
+  - WebSocket connection management
+  - Global notification handling
+
+- **API Client**: `/opt/media-downloader/web/frontend/src/lib/api.ts`
+  - All API call definitions
+  - Authentication token management
+  - Request/response handling
+
+### Page Components
+
+#### Downloads Page
+- **File**: `/opt/media-downloader/web/frontend/src/pages/Downloads.tsx`
+- **Features**:
+  - Comprehensive filter system (search, platform, media type, face recognition)
+  - Advanced filters (date range, size range, sort options)
+  - Grid/List view toggle
+  - Batch operations
+  - File preview modal
+
+#### Media Gallery Page
+- **File**: `/opt/media-downloader/web/frontend/src/pages/Media.tsx`
+- **Features**:
+  - Media browsing and organization
+  - Batch delete operations
+  - File viewing/download
+  - Basic filtering (needs upgrade to match Downloads page)
+
+#### Review Queue Page
+- **File**: `/opt/media-downloader/web/frontend/src/pages/ReviewQueue.tsx`
+- **Features**:
+  - Files awaiting manual review (no face match)
+  - Move to media gallery
+  - Delete files
+  - Face recognition results display
+
+#### Recycle Bin Page
+- **File**: `/opt/media-downloader/web/frontend/src/pages/RecycleBin.tsx`
+- **Features**:
+  - View deleted files from all sources (downloads, media, review)
+  - Restore files to original location
+  - Permanently delete files
+  - Batch operations
+  - Statistics dashboard
+  - Filtering by source
+
+#### Configuration Page
+- **File**: `/opt/media-downloader/web/frontend/src/pages/Config.tsx`
+- **Features**:
+  - Application settings management
+  - Platform credentials
+  - Face recognition settings
+  - Notification settings
+  - Directory settings
+
+#### Other Pages
+- `/opt/media-downloader/web/frontend/src/pages/ChangeLog.tsx` - Version history
+- `/opt/media-downloader/web/frontend/src/pages/Logs.tsx` - System logs viewer
+- `/opt/media-downloader/web/frontend/src/pages/Health.tsx` - Health monitoring
+
+### UI Libraries & Utilities
+- **Notification Manager**: `/opt/media-downloader/web/frontend/src/lib/notificationManager.ts`
+  - Toast notifications
+  - Success/error/info messages
+- **React Query**: Used throughout for data fetching and caching
+- **Tailwind CSS**: Styling framework (configured in tailwind.config.js)
+
+---
+
+## Configuration Files
+
+### Application Settings
+- **Database Settings**: Stored in SQLite via SettingsManager (preferred method)
+  - Access via: `app_state.settings.get('key')` or `app_state.settings.set('key', value)`
+  - Settings categories: general, face_recognition, notifications, recycle_bin, etc.
+
+- **Legacy JSON Config**: `/opt/media-downloader/config/settings.json`
+  - Being phased out - DO NOT ADD NEW SETTINGS HERE
+  - Use database settings instead
+
+### Version Management
+- **Version File**: `/opt/media-downloader/VERSION` - Single source of truth
+- **Package.json**: `/opt/media-downloader/web/frontend/package.json` - Frontend version
+- **README**: `/opt/media-downloader/README.md` - Documentation version
+- **API Version**: Set in `/opt/media-downloader/web/backend/api.py` (FastAPI app)
+
+### Changelog
+- **JSON Format**: `/opt/media-downloader/data/changelog.json` - Structured changelog for API
+- **Markdown Format**: `/opt/media-downloader/docs/CHANGELOG.md` - Human-readable changelog
+
+---
+
+## System Scripts
+
+### Maintenance Scripts
+- `/opt/media-downloader/scripts/create-version-backup.sh` - Creates timestamped backups
+- `/opt/media-downloader/scripts/check-updates.sh` - Checks for available updates
+
+### Database Scripts
+- `/opt/media-downloader/scripts/repair-parent-chains.js` - Fixes backup parent chains
+
+---
+
+## Common Modification Scenarios
+
+### Adding a New API Endpoint
+1. Add endpoint function to `/opt/media-downloader/web/backend/api.py`
+2. Add corresponding database method to `/opt/media-downloader/modules/unified_database.py` (if needed)
+3. Add API client function to `/opt/media-downloader/web/frontend/src/lib/api.ts`
+4. Use in frontend component with React Query
+
+### Adding a New Page
+1. Create component in `/opt/media-downloader/web/frontend/src/pages/YourPage.tsx`
+2. Add route in `/opt/media-downloader/web/frontend/src/App.tsx`
+3. Add navigation menu item in App.tsx (if needed)
+4. Import required icons from 'lucide-react'
+
+### Modifying Download Behavior
+1. Platform-specific logic: `/opt/media-downloader/modules/{platform}_module.py`
+2. File movement logic: `/opt/media-downloader/modules/move_module.py`
+3. Face recognition integration: `/opt/media-downloader/modules/face_recognition_module.py`
+4. Metadata storage: `/opt/media-downloader/modules/metadata_manager.py`
+
+### Modifying Notifications
+1. Backend notification logic: `/opt/media-downloader/modules/pushover_notifier.py`
+2. WebSocket broadcasts: `/opt/media-downloader/web/backend/api.py` (ConnectionManager)
+3. Frontend toast handling: `/opt/media-downloader/web/frontend/src/lib/notificationManager.ts`
+4. Component notification listeners: Individual page components
+
+### Modifying Face Recognition
+1. Core recognition: `/opt/media-downloader/modules/face_recognition_module.py`
+2. Detection: `/opt/media-downloader/modules/face_detection_module.py`
+3. Database storage: `/opt/media-downloader/modules/face_recognition_db.py`
+4. API endpoints: `/opt/media-downloader/web/backend/api.py` (search for "face")
+5. Reference face scripts: `/opt/media-downloader/scripts/` (face-related scripts)
+
+### Modifying Recycle Bin
+1. Database operations: `/opt/media-downloader/modules/unified_database.py`
+   - `move_to_recycle_bin()`, `restore_from_recycle_bin()`, `empty_recycle_bin()`
+2. API endpoints: `/opt/media-downloader/web/backend/api.py` (search for "/api/recycle")
+3. UI component: `/opt/media-downloader/web/frontend/src/pages/RecycleBin.tsx`
+4. Delete operations: Update delete endpoints in api.py to call `move_to_recycle_bin()`
+
+### Adding New Settings
+1. Initialize in API startup: `/opt/media-downloader/web/backend/api.py` (lifespan function)
+   ```python
+   if not app_state.settings.get('your_setting'):
+       app_state.settings.set('your_setting', default_value, category='category', description='desc')
+   ```
+2. Add UI controls: `/opt/media-downloader/web/frontend/src/pages/Config.tsx`
+3. Add API endpoints: `/opt/media-downloader/web/backend/api.py` (if needed)
+
+### Updating Version
+1. Update `/opt/media-downloader/VERSION` (primary source)
+2. Update `/opt/media-downloader/README.md` (version badge)
+3. Update `/opt/media-downloader/web/frontend/package.json` (version field)
+4. Update API version in `/opt/media-downloader/web/backend/api.py`
+5. Update App.tsx version display
+6. Add entry to `/opt/media-downloader/data/changelog.json`
+7. Add entry to `/opt/media-downloader/docs/CHANGELOG.md`
+8. Run `/opt/media-downloader/scripts/create-version-backup.sh`
+
+---
+
+## Database Schema Quick Reference
+
+### Core Tables
+- **downloads** - Downloaded files tracking
+- **media_gallery** - Organized media files
+- **review_queue** - Files awaiting manual review
+- **recycle_bin** - Soft-deleted files (UUID-based storage)
+- **users** - User accounts
+- **settings** - Application settings (key-value store)
+- **face_recognition_db** - Reference faces and metadata
+
+### Recycle Bin Schema
+```sql
+CREATE TABLE recycle_bin (
+    id TEXT PRIMARY KEY,              -- UUID for storage
+    original_path TEXT NOT NULL,      -- Full path for restore
+    original_filename TEXT NOT NULL,  -- Display name
+    recycle_path TEXT NOT NULL,       -- Current location
+    file_extension TEXT,              -- .jpg, .mp4, etc.
+    file_size INTEGER,                -- Bytes
+    original_mtime REAL,              -- Preserved timestamp
+    deleted_from TEXT NOT NULL,       -- 'downloads', 'media', 'review'
+    deleted_at DATETIME,              -- When deleted
+    deleted_by TEXT,                  -- Username
+    metadata TEXT,                    -- JSON metadata
+    restore_count INTEGER DEFAULT 0   -- Times restored
+)
+```
+
+---
+
+## Directory Structure
+
+```
+/opt/media-downloader/
+├── config/               - Configuration files (legacy JSON - avoid)
+├── data/                 - Application data
+│   ├── backup_cache.db   - Main SQLite database
+│   └── changelog.json    - Structured changelog
+├── docs/                 - Documentation (keep all docs here)
+├── logs/                 - Application logs
+├── modules/              - Python backend modules
+├── scripts/              - Utility scripts
+├── utilities/            - Helper utilities
+└── web/
+    ├── backend/          - FastAPI application
+    │   └── api.py        - Main API file
+    └── frontend/         - React application
+        └── src/
+            ├── lib/      - Utilities (api.ts, notificationManager.ts)
+            └── pages/    - Page components
+```
+
+---
+
+## Quick Reference Cheat Sheet
+
+| Feature | Backend File | Frontend File | API Endpoint |
+|---------|-------------|---------------|--------------|
+| Downloads | modules/*_module.py | pages/Downloads.tsx | /api/downloads/* |
+| Media Gallery | modules/unified_database.py | pages/Media.tsx | /api/media/* |
+| Review Queue | modules/move_module.py | pages/ReviewQueue.tsx | /api/review/* |
+| Recycle Bin | modules/unified_database.py | pages/RecycleBin.tsx | /api/recycle/* |
+| Face Recognition | modules/face_recognition_module.py | N/A | /api/face/* |
+| Notifications | modules/pushover_notifier.py | lib/notificationManager.ts | N/A |
+| Settings | modules/unified_database.py | pages/Config.tsx | /api/settings/* |
+| Users | web/backend/api.py | N/A | /api/auth/*, /api/users/* |
+
+---
+
+## Tips for Making Modifications
+
+1. **Always use database settings** - Don't add to JSON config files
+2. **Update version numbers** - Follow VERSION_UPDATE_CHECKLIST.md
+3. **Test Python syntax** - Run `python3 -m py_compile <file>` before committing
+4. **Test TypeScript** - Run `npm run type-check` in web/frontend/
+5. **Rebuild frontend** - Run `npm run build` after changes
+6. **Restart services** - `sudo systemctl restart media-downloader-api` and `media-downloader-web`
+7. **Create backups** - Run `scripts/create-version-backup.sh` before major changes
+8. **Update changelog** - Add entries to both changelog.json and CHANGELOG.md
+
+---
+
+## Related Documentation
+
+- **CHANGELOG.md** - Version history and release notes
+- **VERSION_UPDATE_CHECKLIST.md** - Step-by-step version update process
+- **FACE_RECOGNITION.md** - Face recognition feature documentation
+- **NOTIFICATIONS.md** - Notification system documentation
+- **REVIEW_QUEUE_STRUCTURE.md** - Review queue architecture
+- **FEATURE_ROADMAP_2025.md** - Planned features and improvements