Initial commit

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 22:42:55 -04:00
commit 0d7b2b1aab
389 changed files with 280296 additions and 0 deletions
--- a/docs/FACE_RECOGNITION.md
+++ b/docs/FACE_RECOGNITION.md
@@ -0,0 +1,414 @@
+# Face Recognition System
+
+**Version:** 6.5.1
+**Status:** Production Ready
+**Last Updated:** 2025-11-01
+
+## Overview
+
+The Media Downloader now includes an automated face recognition system that analyzes downloaded media (images and videos) and routes them based on whether they match reference faces in the database.
+
+## Features
+
+- **Automatic Face Detection**: Scans all downloaded images and videos
+- **Video Support**: Extracts frames from videos for face analysis
+- **Smart Routing**: Matched media → final destination, unmatched → review queue
+- **Web UI Review Queue**: Manual review interface with batch operations
+- **Reference Training**: Build face database from known good images
+- **Configurable Tolerance**: Adjustable matching sensitivity
+
+## Architecture
+
+### Components
+
+1. **Face Recognition Module** (`modules/face_recognition_module.py`)
+   - Face detection using `face_recognition` library (dlib HOG model)
+   - Face encoding (128-dimensional vectors)
+   - Reference face database management
+   - Video frame extraction via ffmpeg
+
+2. **Move Module Integration** (`modules/move_module.py`)
+   - Integrated into file move workflow
+   - Checks after duplicate detection
+   - Routes to review queue on no-match
+
+3. **Review API** (`web/backend/api.py`)
+   - `/api/review/list` - List review queue
+   - `/api/review/keep` - Move to destination
+   - `/api/review/delete` - Delete from queue
+   - `/api/review/add-reference` - Add as reference face
+
+4. **Review UI** (`web/frontend/src/pages/Review.tsx`)
+   - Gallery view of unmatched media
+   - Single-file and batch operations
+   - Lightbox preview
+   - Action buttons: Keep, Add Reference, Delete
+
+### Database Schema
+
+```sql
+CREATE TABLE face_recognition_references (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    person_name TEXT NOT NULL,
+    encoding_data TEXT NOT NULL,  -- Base64 encoded pickle of numpy array
+    reference_image_path TEXT,
+    is_active INTEGER DEFAULT 1,
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+
+CREATE INDEX idx_face_ref_person ON face_recognition_references(person_name, is_active);
+```
+
+## Workflow
+
+```
+┌─────────────────────┐
+│  Download Media     │
+└──────────┬──────────┘
+           │
+           ▼
+┌─────────────────────┐
+│  Calculate Hash     │ ──── Duplicate? ──→ Skip
+└──────────┬──────────┘
+           │ New file
+           ▼
+┌─────────────────────┐
+│  Detect Faces       │
+│  (Image: direct)    │
+│  (Video: extract    │
+│   frame @ 1s)       │
+└──────────┬──────────┘
+           │
+           ▼
+┌─────────────────────┐
+│  Match Against      │
+│  Reference Faces    │
+│  (tolerance: 0.6)   │
+└──────────┬──────────┘
+           │
+           ├─── Match (>60%) ──→ Move to Final Destination
+           │
+           └─── No Match ─────→ Move to /opt/immich/review
+                                         │
+                                         ▼
+                                ┌─────────────────────┐
+                                │  Review Queue (UI)  │
+                                │  - Keep             │
+                                │  - Add Reference    │
+                                │  - Delete           │
+                                └─────────────────────┘
+```
+
+## Configuration
+
+### Settings (Web UI)
+
+Face recognition settings can be configured via the **Configuration → Downloads** page in the web UI:
+
+1. Navigate to http://your-server:5173/configuration
+2. Click the **Downloads** tab
+3. Scroll to the **Face Recognition** section
+4. Configure settings:
+   - **Enabled**: Toggle face recognition on/off
+   - **Person Name**: Name used for matching reference faces (e.g., "Eva Longoria")
+   - **Tolerance**: Match sensitivity 0.0-1.0 (default: 0.6, lower = stricter)
+   - **Review Queue Path**: Directory for unmatched media (default: /opt/immich/review)
+5. Click **Save Download Settings**
+
+### Settings (Database)
+
+Settings are stored in the database `settings` table:
+
+```json
+{
+  "enabled": true,
+  "tolerance": 0.6,
+  "person_name": "Eva Longoria",
+  "review_path": "/opt/immich/review"
+}
+```
+
+**Parameters:**
+- `enabled` (boolean): Enable/disable face recognition
+- `tolerance` (float 0.0-1.0): Lower = stricter matching (default: 0.6)
+- `person_name` (string): Default person name for references
+- `review_path` (string): Directory for unmatched media
+
+**Direct Database Access:**
+```bash
+sqlite3 /opt/media-downloader/database/media_downloader.db "SELECT key, value FROM settings WHERE key = 'face_recognition'"
+```
+
+### Supported Formats
+
+**Images:**
+- .jpg, .jpeg, .png, .gif, .bmp, .webp, .heic
+
+**Videos:**
+- .mp4, .mov, .avi, .mkv, .webm, .flv, .m4v
+
+## Usage
+
+### Training Reference Faces
+
+Add reference faces from known good images:
+
+```bash
+/opt/media-downloader/venv/bin/python3 /opt/media-downloader/scripts/add_reference_face.py "Person Name" "/path/to/image.jpg"
+```
+
+**Best Practices:**
+- Use 5-10 reference images per person
+- Include variety: different angles, lighting, expressions
+- Use high-quality, clear face images
+- Avoid group photos (will use first detected face)
+
+### Testing Face Recognition
+
+Test an image/video against reference database:
+
+```bash
+/opt/media-downloader/venv/bin/python3 /opt/media-downloader/scripts/test_face_recognition.py "/path/to/test.jpg" [tolerance]
+```
+
+### Managing Review Queue
+
+**Via Web UI:**
+1. Navigate to `/review` page
+2. View unmatched media in gallery
+3. For each item or batch selection:
+   - **Keep**: Move to destination without adding as reference
+   - **Add Reference**: Add face to database + move to destination
+   - **Delete**: Remove from review queue
+
+**Via CLI:**
+```bash
+# List review queue
+ls -lh /opt/immich/review/
+
+# Move to final destination manually
+mv /opt/immich/review/file.jpg /opt/immich/md/destination/
+
+# Add as reference then move
+venv/bin/python3 scripts/add_reference_face.py "Name" "/opt/immich/review/file.jpg"
+mv /opt/immich/review/file.jpg /opt/immich/md/destination/
+```
+
+### Batch Operations
+
+**Web UI:**
+1. Click "Select Multiple" button
+2. Click images to select (blue ring + checkbox)
+3. Use "Select All" for all items
+4. Choose batch action:
+   - **Keep Selected** - Bulk move to destination
+   - **Add as Reference** - Bulk add faces + move
+   - **Delete Selected** - Bulk delete
+
+## Performance
+
+### Speed
+- **Image Detection**: ~0.5-2s per image (HOG model)
+- **Video Detection**: ~2-5s per video (frame extraction + detection)
+- **Matching**: <0.1s per face against all references
+
+### Accuracy
+- **Same Person, Same Conditions**: 90-100% confidence
+- **Same Person, Different Conditions**: 50-80% confidence
+- **Different Person**: <40% confidence
+- **Threshold**: 60% (tolerance: 0.6)
+
+### Resource Usage
+- **CPU**: Moderate (HOG model is CPU-based)
+- **Memory**: ~200MB additional for face_recognition library
+- **Disk**: Minimal (encodings are ~1KB each)
+- **Temp Files**: Video frames auto-deleted after processing
+
+## Troubleshooting
+
+### No Faces Detected
+
+**Causes:**
+- Face too small in image/video
+- Face obscured or at extreme angle
+- Poor image quality
+
+**Solutions:**
+- Use higher quality source images
+- For videos, try different timestamp (currently fixed at 1s)
+- Check image isn't corrupted: `file /path/to/image.jpg`
+
+### Low Confidence Matches
+
+**Causes:**
+- Insufficient reference faces
+- References don't match current conditions (age, lighting, angle)
+- Tolerance too strict
+
+**Solutions:**
+- Add more reference faces (5-10 recommended)
+- Add references from similar conditions to target media
+- Increase tolerance in settings (0.6 → 0.65)
+
+### False Positives
+
+**Causes:**
+- Tolerance too loose
+- Similar-looking people
+- Insufficient reference diversity
+
+**Solutions:**
+- Decrease tolerance (0.6 → 0.55)
+- Add negative examples to recognize differences
+- Review reference faces for quality
+
+### Video Frame Extraction Fails
+
+**Causes:**
+- ffmpeg not installed
+- Video codec not supported
+- Video shorter than 1 second
+
+**Solutions:**
+```bash
+# Check ffmpeg
+which ffmpeg
+
+# Test frame extraction manually
+ffmpeg -ss 1 -i video.mp4 -frames:v 1 test_frame.jpg
+
+# Check video duration
+ffmpeg -i video.mp4 2>&1 | grep Duration
+```
+
+## API Reference
+
+### Face Recognition Module
+
+```python
+from modules.face_recognition_module import FaceRecognitionModule
+from modules.unified_database import UnifiedDatabase
+
+# Initialize
+db = UnifiedDatabase()
+face_module = FaceRecognitionModule(unified_db=db)
+
+# Add reference face
+face_module.add_reference_face("Person Name", "/path/to/image.jpg")
+
+# Check image
+result = face_module.check_image("/path/to/test.jpg", tolerance=0.6, is_video=False)
+# Returns: {'has_match': bool, 'person_name': str, 'confidence': float, 'face_count': int, 'faces': list}
+
+# Check video
+result = face_module.check_image("/path/to/video.mp4", tolerance=0.6, is_video=True)
+
+# Get reference faces
+refs = face_module.get_reference_faces()
+```
+
+### Review API Endpoints
+
+```javascript
+// List review queue
+GET /api/review/list?limit=50&offset=0
+
+// Keep image (move to destination)
+POST /api/review/keep
+Body: { file_path: "/opt/immich/review/file.jpg", destination: "social media/instagram/posts" }
+
+// Delete from review queue
+DELETE /api/review/delete
+Body: { file_path: "/opt/immich/review/file.jpg" }
+
+// Add as reference + move
+POST /api/review/add-reference
+Body: {
+  file_path: "/opt/immich/review/file.jpg",
+  person_name: "Person Name",
+  destination: "social media/instagram/posts"
+}
+```
+
+## Maintenance
+
+### Regular Tasks
+
+1. **Review Queue Cleanup** (Weekly)
+   - Process items in /opt/immich/review
+   - Keep: items that should have matched
+   - Delete: irrelevant items
+   - Add Reference: good quality faces to improve matching
+
+2. **Reference Database Audit** (Monthly)
+   - Remove poor quality references
+   - Add new references from recent media
+   - Check reference count per person
+
+3. **Performance Monitoring**
+   - Check review queue size: `ls /opt/immich/review | wc -l`
+   - Monitor match rate in logs
+   - Adjust tolerance if needed
+
+### Database Queries
+
+```sql
+-- Count active references by person
+SELECT person_name, COUNT(*) as count
+FROM face_recognition_references
+WHERE is_active = 1
+GROUP BY person_name;
+
+-- View recent references
+SELECT person_name, reference_image_path, created_at
+FROM face_recognition_references
+WHERE is_active = 1
+ORDER BY created_at DESC
+LIMIT 10;
+
+-- Disable a reference
+UPDATE face_recognition_references
+SET is_active = 0
+WHERE id = ?;
+```
+
+## Security & Privacy
+
+- **Face Encodings**: Stored as 128-dimensional vectors (not original images)
+- **Local Processing**: All face detection happens locally, no cloud services
+- **Access Control**: Review queue API requires authentication
+- **Data Retention**: Reference faces kept indefinitely until manually removed
+- **Audit Trail**: created_at/updated_at timestamps track reference changes
+
+## Dependencies
+
+- **face_recognition** (1.3.0): Face detection and recognition
+- **dlib** (20.0.0): Machine learning toolkit (face detection models)
+- **numpy** (2.3.4): Numerical computing (face encoding vectors)
+- **ffmpeg**: Video frame extraction (system package)
+
+## Future Enhancements
+
+- [ ] Multi-person recognition (tag all people in image)
+- [ ] Confidence threshold per person
+- [ ] Face clustering for unknown faces
+- [ ] GPU acceleration (dlib CNN model)
+- [ ] Multiple frame extraction for videos
+- [ ] Face detection quality scoring
+- [ ] Auto-training from high-confidence matches
+- [ ] REST API for external integrations
+
+## Version History
+
+**6.5.1** (2025-11-01)
+- Added face recognition settings to Configuration page (Web UI)
+- Settings now editable via Configuration → Downloads tab
+- Real-time settings updates without editing database directly
+
+**6.5.0** (2025-10-31)
+- Initial face recognition implementation
+- Image and video support
+- Review queue with batch operations
+- Reference face training
+- Web UI integration