LIComputerGuy/media-downloader
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
523f91788e158e9f7a2c84306499bc94ad1f3bc5
media-downloader/docs/CODE_MAP.md
Todd 0d7b2b1aab Initial commit 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 22:42:55 -04:00

345 lines
14 KiB
Markdown
Raw Blame History

# Code Map - Feature Location Reference
This document provides a quick reference for locating features and components when making modifications to the Media Downloader application.
Last Updated: 2026-02-10 (v12.12.1)
---
## Core Backend Files
### Database Layer
- **File**: `/opt/media-downloader/modules/unified_database.py`
- **Contains**:
- All database table schemas (downloads, media_gallery, review_queue, recycle_bin, etc.)
- CRUD operations for all tables
- Database connection pooling (DatabasePool class)
- Settings management (get_setting, set_setting)
- Recycle bin operations (move_to_recycle_bin, restore_from_recycle_bin)
- Face recognition metadata storage
### API Endpoints
- **Entry Point**: `/opt/media-downloader/web/backend/api.py` (828 lines - router registration)
- **Routers**: `/opt/media-downloader/web/backend/routers/` (26 routers)
- `paid_content.py` - Paid content CRUD, services, creators, feed, messages, OnlyFans/Fansly setup, health checks
- `private_gallery.py` - Private gallery auth, media, persons, encryption, features, URL import
- `media.py` - Media serving, thumbnails, gallery
- `face.py` - Face recognition endpoints
- `downloads.py` - Download history, analytics
- `review.py` - Review queue management
- `config.py` - Configuration management
- `scheduler.py` - Scheduler control
- And 18 more routers (auth, health, recycle, stats, discovery, video, etc.)
---
## Feature-Specific Modules
### Face Recognition
- **Main Module**: `/opt/media-downloader/modules/face_recognition_module.py`
- **Detection Module**: `/opt/media-downloader/modules/face_detection_module.py`
- **Database Manager**: `/opt/media-downloader/modules/face_recognition_db.py`
- **Related Scripts**:
- `/opt/media-downloader/scripts/add_reference_face.py` - Add reference faces
- `/opt/media-downloader/scripts/batch_compare_faces.py` - Batch comparison
- `/opt/media-downloader/scripts/list_reference_faces.py` - List faces
- `/opt/media-downloader/scripts/delete_reference_face.py` - Delete faces
- **UI Components**:
- Frontend API calls: `/opt/media-downloader/web/frontend/src/lib/api.ts`
- Face recognition page: Check App.tsx for routing
### File Movement & Organization
- **File**: `/opt/media-downloader/modules/move_module.py`
- **Contains**:
- File movement logic (move_file)
- Batch move context management
- Review queue handling
- Notification tracking for moved files
- Separate tracking for review queue vs matched files
- Integration with face recognition workflow
### Push Notifications
- **File**: `/opt/media-downloader/modules/pushover_notifier.py`
- **Contains**:
- Pushover API integration
- Batch download notifications
- Review queue notifications (separate from regular downloads)
- Platform-specific icons and formatting
- Image attachment support
- Priority settings
### Media Download Modules
- **Instagram**: `/opt/media-downloader/modules/instagram_module.py`
- **Reddit**: `/opt/media-downloader/modules/reddit_module.py`
- **TikTok**: `/opt/media-downloader/modules/tiktok_module.py`
- **Bunkr**: `/opt/media-downloader/modules/bunkr_module.py`
- **X/Twitter**: `/opt/media-downloader/modules/x_module.py`
### Utilities
- **Filename Cleaner**: `/opt/media-downloader/utilities/filename_cleaner.py`
- **Metadata Manager**: `/opt/media-downloader/modules/metadata_manager.py`
- **Cache Builder**: `/opt/media-downloader/utilities/cache_builder.py`
---
## Frontend Structure
### Main Application Files
- **App Entry**: `/opt/media-downloader/web/frontend/src/App.tsx`
- Main routing configuration
- Navigation menu (Downloads, Media, Review, System dropdowns)
- WebSocket connection management
- Global notification handling
- **API Client**: `/opt/media-downloader/web/frontend/src/lib/api.ts`
- All API call definitions
- Authentication token management
- Request/response handling
### Page Components
#### Downloads Page
- **File**: `/opt/media-downloader/web/frontend/src/pages/Downloads.tsx`
- **Features**:
- Comprehensive filter system (search, platform, media type, face recognition)
- Advanced filters (date range, size range, sort options)
- Grid/List view toggle
- Batch operations
- File preview modal
#### Media Gallery Page
- **File**: `/opt/media-downloader/web/frontend/src/pages/Media.tsx`
- **Features**:
- Media browsing and organization
- Batch delete operations
- File viewing/download
- Basic filtering (needs upgrade to match Downloads page)
#### Review Queue Page
- **File**: `/opt/media-downloader/web/frontend/src/pages/ReviewQueue.tsx`
- **Features**:
- Files awaiting manual review (no face match)
- Move to media gallery
- Delete files
- Face recognition results display
#### Recycle Bin Page
- **File**: `/opt/media-downloader/web/frontend/src/pages/RecycleBin.tsx`
- **Features**:
- View deleted files from all sources (downloads, media, review)
- Restore files to original location
- Permanently delete files
- Batch operations
- Statistics dashboard
- Filtering by source
#### Configuration Page
- **File**: `/opt/media-downloader/web/frontend/src/pages/Config.tsx`
- **Features**:
- Application settings management
- Platform credentials
- Face recognition settings
- Notification settings
- Directory settings
#### Other Pages
- `/opt/media-downloader/web/frontend/src/pages/ChangeLog.tsx` - Version history
- `/opt/media-downloader/web/frontend/src/pages/Logs.tsx` - System logs viewer
- `/opt/media-downloader/web/frontend/src/pages/Health.tsx` - Health monitoring
### UI Libraries & Utilities
- **Notification Manager**: `/opt/media-downloader/web/frontend/src/lib/notificationManager.ts`
- Toast notifications
- Success/error/info messages
- **React Query**: Used throughout for data fetching and caching
- **Tailwind CSS**: Styling framework (configured in tailwind.config.js)
---
## Configuration Files
### Application Settings
- **Database Settings**: Stored in SQLite via SettingsManager (preferred method)
- Access via: `app_state.settings.get('key')` or `app_state.settings.set('key', value)`
- Settings categories: general, face_recognition, notifications, recycle_bin, etc.
- **Legacy JSON Config**: `/opt/media-downloader/config/settings.json`
- Being phased out - DO NOT ADD NEW SETTINGS HERE
- Use database settings instead
### Version Management
- **Version File**: `/opt/media-downloader/VERSION` - Single source of truth
- **Package.json**: `/opt/media-downloader/web/frontend/package.json` - Frontend version
- **README**: `/opt/media-downloader/README.md` - Documentation version
- **API Version**: Set in `/opt/media-downloader/web/backend/api.py` (FastAPI app)
### Changelog
- **JSON Format**: `/opt/media-downloader/data/changelog.json` - Structured changelog for API
- **Markdown Format**: `/opt/media-downloader/docs/CHANGELOG.md` - Human-readable changelog
---
## System Scripts
### Maintenance Scripts
- `/opt/media-downloader/scripts/create-version-backup.sh` - Creates timestamped backups
- `/opt/media-downloader/scripts/check-updates.sh` - Checks for available updates
### Database Scripts
- `/opt/media-downloader/scripts/repair-parent-chains.js` - Fixes backup parent chains
---
## Common Modification Scenarios
### Adding a New API Endpoint
1. Add endpoint function to `/opt/media-downloader/web/backend/api.py`
2. Add corresponding database method to `/opt/media-downloader/modules/unified_database.py` (if needed)
3. Add API client function to `/opt/media-downloader/web/frontend/src/lib/api.ts`
4. Use in frontend component with React Query
### Adding a New Page
1. Create component in `/opt/media-downloader/web/frontend/src/pages/YourPage.tsx`
2. Add route in `/opt/media-downloader/web/frontend/src/App.tsx`
3. Add navigation menu item in App.tsx (if needed)
4. Import required icons from 'lucide-react'
### Modifying Download Behavior
1. Platform-specific logic: `/opt/media-downloader/modules/{platform}_module.py`
2. File movement logic: `/opt/media-downloader/modules/move_module.py`
3. Face recognition integration: `/opt/media-downloader/modules/face_recognition_module.py`
4. Metadata storage: `/opt/media-downloader/modules/metadata_manager.py`
### Modifying Notifications
1. Backend notification logic: `/opt/media-downloader/modules/pushover_notifier.py`
2. WebSocket broadcasts: `/opt/media-downloader/web/backend/api.py` (ConnectionManager)
3. Frontend toast handling: `/opt/media-downloader/web/frontend/src/lib/notificationManager.ts`
4. Component notification listeners: Individual page components
### Modifying Face Recognition
1. Core recognition: `/opt/media-downloader/modules/face_recognition_module.py`
2. Detection: `/opt/media-downloader/modules/face_detection_module.py`
3. Database storage: `/opt/media-downloader/modules/face_recognition_db.py`
4. API endpoints: `/opt/media-downloader/web/backend/api.py` (search for "face")
5. Reference face scripts: `/opt/media-downloader/scripts/` (face-related scripts)
### Modifying Recycle Bin
1. Database operations: `/opt/media-downloader/modules/unified_database.py`
- `move_to_recycle_bin()`, `restore_from_recycle_bin()`, `empty_recycle_bin()`
2. API endpoints: `/opt/media-downloader/web/backend/api.py` (search for "/api/recycle")
3. UI component: `/opt/media-downloader/web/frontend/src/pages/RecycleBin.tsx`
4. Delete operations: Update delete endpoints in api.py to call `move_to_recycle_bin()`
### Adding New Settings
1. Initialize in API startup: `/opt/media-downloader/web/backend/api.py` (lifespan function)
```python
if not app_state.settings.get('your_setting'):
app_state.settings.set('your_setting', default_value, category='category', description='desc')
```
2. Add UI controls: `/opt/media-downloader/web/frontend/src/pages/Config.tsx`
3. Add API endpoints: `/opt/media-downloader/web/backend/api.py` (if needed)
### Updating Version
1. Update `/opt/media-downloader/VERSION` (primary source)
2. Update `/opt/media-downloader/README.md` (version badge)
3. Update `/opt/media-downloader/web/frontend/package.json` (version field)
4. Update API version in `/opt/media-downloader/web/backend/api.py`
5. Update App.tsx version display
6. Add entry to `/opt/media-downloader/data/changelog.json`
7. Add entry to `/opt/media-downloader/docs/CHANGELOG.md`
8. Run `/opt/media-downloader/scripts/create-version-backup.sh`
---
## Database Schema Quick Reference
### Core Tables
- **downloads** - Downloaded files tracking
- **media_gallery** - Organized media files
- **review_queue** - Files awaiting manual review
- **recycle_bin** - Soft-deleted files (UUID-based storage)
- **users** - User accounts
- **settings** - Application settings (key-value store)
- **face_recognition_db** - Reference faces and metadata
### Recycle Bin Schema
```sql
CREATE TABLE recycle_bin (
id TEXT PRIMARY KEY, -- UUID for storage
original_path TEXT NOT NULL, -- Full path for restore
original_filename TEXT NOT NULL, -- Display name
recycle_path TEXT NOT NULL, -- Current location
file_extension TEXT, -- .jpg, .mp4, etc.
file_size INTEGER, -- Bytes
original_mtime REAL, -- Preserved timestamp
deleted_from TEXT NOT NULL, -- 'downloads', 'media', 'review'
deleted_at DATETIME, -- When deleted
deleted_by TEXT, -- Username
metadata TEXT, -- JSON metadata
restore_count INTEGER DEFAULT 0 -- Times restored
)
```
---
## Directory Structure
```
/opt/media-downloader/
├── config/ - Configuration files (legacy JSON - avoid)
├── data/ - Application data
│ ├── backup_cache.db - Main SQLite database
│ └── changelog.json - Structured changelog
├── docs/ - Documentation (keep all docs here)
├── logs/ - Application logs
├── modules/ - Python backend modules
├── scripts/ - Utility scripts
├── utilities/ - Helper utilities
└── web/
├── backend/ - FastAPI application
│ └── api.py - Main API file
└── frontend/ - React application
└── src/
├── lib/ - Utilities (api.ts, notificationManager.ts)
└── pages/ - Page components
```
---
## Quick Reference Cheat Sheet
| Feature | Backend File | Frontend File | API Endpoint |
|---------|-------------|---------------|--------------|
| Downloads | modules/*_module.py | pages/Downloads.tsx | /api/downloads/* |
| Media Gallery | modules/unified_database.py | pages/Media.tsx | /api/media/* |
| Review Queue | modules/move_module.py | pages/ReviewQueue.tsx | /api/review/* |
| Recycle Bin | modules/unified_database.py | pages/RecycleBin.tsx | /api/recycle/* |
| Face Recognition | modules/face_recognition_module.py | N/A | /api/face/* |
| Notifications | modules/pushover_notifier.py | lib/notificationManager.ts | N/A |
| Settings | modules/unified_database.py | pages/Config.tsx | /api/settings/* |
| Users | web/backend/api.py | N/A | /api/auth/*, /api/users/* |
---
## Tips for Making Modifications
1. **Always use database settings** - Don't add to JSON config files
2. **Update version numbers** - Follow VERSION_UPDATE_CHECKLIST.md
3. **Test Python syntax** - Run `python3 -m py_compile <file>` before committing
4. **Test TypeScript** - Run `npm run type-check` in web/frontend/
5. **Rebuild frontend** - Run `npm run build` after changes
6. **Restart services** - `sudo systemctl restart media-downloader-api` and `media-downloader-web`
7. **Create backups** - Run `scripts/create-version-backup.sh` before major changes
8. **Update changelog** - Add entries to both changelog.json and CHANGELOG.md
---
## Related Documentation
- **CHANGELOG.md** - Version history and release notes
- **VERSION_UPDATE_CHECKLIST.md** - Step-by-step version update process
- **FACE_RECOGNITION.md** - Face recognition feature documentation
- **NOTIFICATIONS.md** - Notification system documentation
- **REVIEW_QUEUE_STRUCTURE.md** - Review queue architecture
- **FEATURE_ROADMAP_2025.md** - Planned features and improvements
Reference in New Issue View Git Blame Copy Permalink