Todd
958
docs/archive/AI_FACE_RECOGNITION_PLAN.md
Normal file
958
docs/archive/AI_FACE_RECOGNITION_PLAN.md
Normal file
@@ -0,0 +1,958 @@
|
||||
# AI-Powered Face Recognition & Auto-Sorting System
|
||||
|
||||
**Created**: 2025-10-31
|
||||
**Status**: Planning Phase
|
||||
**Target Version**: 6.5.0
|
||||
|
||||
---
|
||||
|
||||
## 📋 Overview
|
||||
|
||||
Automatic face recognition and sorting system that processes downloaded images, identifies people, and organizes them into person-specific directories. Unknown faces go to a review queue for manual identification.
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Goals
|
||||
|
||||
### Primary Goals
|
||||
1. **Automatic face detection** - Identify faces in downloaded images
|
||||
2. **Face recognition** - Match faces against known people database
|
||||
3. **Auto-sorting** - Move matched images to person-specific directories
|
||||
4. **Review queue** - Queue unknown faces for manual identification
|
||||
5. **Learning system** - Improve recognition from manual reviews
|
||||
|
||||
### Secondary Goals
|
||||
6. **Multi-face support** - Handle images with multiple people
|
||||
7. **Confidence scoring** - Only auto-sort high confidence matches
|
||||
8. **Performance** - Process images quickly without blocking downloads
|
||||
9. **Privacy** - All processing done locally (no cloud APIs)
|
||||
10. **Immich integration** - Sync sorted images to Immich
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
### High-Level Flow
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ Image Download │
|
||||
│ Complete │
|
||||
└────────┬────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Face Detection │ ◄── Uses face_recognition library
|
||||
│ (Find Faces) │ or DeepFace
|
||||
└────────┬────────┘
|
||||
│
|
||||
├─── No faces found ──► Skip (keep in original location)
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Face Recognition│ ◄── Compare against known faces DB
|
||||
│ (Identify Who) │
|
||||
└────────┬────────┘
|
||||
│
|
||||
├─── High confidence match ──► Auto-sort to person directory
|
||||
│
|
||||
├─── Low confidence/Multiple ──► Review Queue
|
||||
│
|
||||
└─── Unknown face ──────────► Review Queue
|
||||
```
|
||||
|
||||
### Database Schema
|
||||
|
||||
```sql
|
||||
-- New table: face_recognition_people
|
||||
CREATE TABLE face_recognition_people (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
name TEXT NOT NULL UNIQUE,
|
||||
directory TEXT NOT NULL, -- Target directory for this person
|
||||
face_encodings BLOB, -- Stored face encodings (multiple per person)
|
||||
created_at TEXT,
|
||||
updated_at TEXT,
|
||||
enabled INTEGER DEFAULT 1
|
||||
);
|
||||
|
||||
-- New table: face_recognition_queue
|
||||
CREATE TABLE face_recognition_queue (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
download_id INTEGER,
|
||||
file_path TEXT NOT NULL,
|
||||
thumbnail_path TEXT,
|
||||
face_encoding BLOB, -- Encoding of the face found
|
||||
face_location TEXT, -- JSON: bounding box coordinates
|
||||
confidence REAL, -- Match confidence if any
|
||||
suggested_person_id INTEGER, -- Best match suggestion
|
||||
status TEXT DEFAULT 'pending', -- pending, reviewed, skipped
|
||||
created_at TEXT,
|
||||
reviewed_at TEXT,
|
||||
reviewed_by TEXT,
|
||||
FOREIGN KEY (download_id) REFERENCES downloads(id),
|
||||
FOREIGN KEY (suggested_person_id) REFERENCES face_recognition_people(id)
|
||||
);
|
||||
|
||||
-- New table: face_recognition_history
|
||||
CREATE TABLE face_recognition_history (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
download_id INTEGER,
|
||||
file_path TEXT NOT NULL,
|
||||
person_id INTEGER,
|
||||
confidence REAL,
|
||||
action TEXT, -- auto_sorted, manually_sorted, skipped
|
||||
processed_at TEXT,
|
||||
FOREIGN KEY (download_id) REFERENCES downloads(id),
|
||||
FOREIGN KEY (person_id) REFERENCES face_recognition_people(id)
|
||||
);
|
||||
```
|
||||
|
||||
### Directory Structure
|
||||
|
||||
```
|
||||
/mnt/storage/Downloads/
|
||||
├── [existing platform directories]/
|
||||
│ └── [original downloads]
|
||||
│
|
||||
├── faces/
|
||||
│ ├── person1_name/
|
||||
│ │ ├── 20250131_120000_abc123.jpg
|
||||
│ │ └── 20250131_130000_def456.jpg
|
||||
│ │
|
||||
│ ├── person2_name/
|
||||
│ │ └── 20250131_140000_ghi789.jpg
|
||||
│ │
|
||||
│ └── review_queue/
|
||||
│ ├── unknown_face_20250131_120000_abc123.jpg
|
||||
│ ├── low_confidence_20250131_130000_def456.jpg
|
||||
│ └── multiple_faces_20250131_140000_ghi789.jpg
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Technical Implementation
|
||||
|
||||
### 1. Face Recognition Library Options
|
||||
|
||||
#### Option A: face_recognition (Recommended)
|
||||
**Pros**:
|
||||
- Built on dlib (very accurate)
|
||||
- Simple Python API
|
||||
- Fast face detection and recognition
|
||||
- Well-documented
|
||||
- Works offline
|
||||
|
||||
**Cons**:
|
||||
- Requires dlib compilation (can be slow to install)
|
||||
- Heavy dependencies
|
||||
|
||||
**Installation**:
|
||||
```bash
|
||||
pip3 install face_recognition
|
||||
pip3 install pillow
|
||||
```
|
||||
|
||||
**Usage Example**:
|
||||
```python
|
||||
import face_recognition
|
||||
import numpy as np
|
||||
|
||||
# Load and encode known face
|
||||
image = face_recognition.load_image_file("person1.jpg")
|
||||
encoding = face_recognition.face_encodings(image)[0]
|
||||
|
||||
# Compare with new image
|
||||
unknown_image = face_recognition.load_image_file("unknown.jpg")
|
||||
unknown_encodings = face_recognition.face_encodings(unknown_image)
|
||||
|
||||
matches = face_recognition.compare_faces([encoding], unknown_encodings[0])
|
||||
distance = face_recognition.face_distance([encoding], unknown_encodings[0])
|
||||
```
|
||||
|
||||
#### Option B: DeepFace
|
||||
**Pros**:
|
||||
- Multiple backend models (VGG-Face, Facenet, OpenFace, DeepID, ArcFace)
|
||||
- Very high accuracy
|
||||
- Age, gender, emotion detection
|
||||
|
||||
**Cons**:
|
||||
- Slower than face_recognition
|
||||
- More complex setup
|
||||
- Larger dependencies
|
||||
|
||||
#### Option C: OpenCV + dlib
|
||||
**Pros**:
|
||||
- Already installed (OpenCV used elsewhere)
|
||||
- Full control
|
||||
- Fast face detection
|
||||
|
||||
**Cons**:
|
||||
- More manual coding
|
||||
- Complex face encoding
|
||||
|
||||
**Recommendation**: Start with **face_recognition** (Option A) for best balance.
|
||||
|
||||
---
|
||||
|
||||
### 2. Core Module Structure
|
||||
|
||||
#### New File: `modules/face_recognition_manager.py`
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Face Recognition Manager
|
||||
Handles face detection, recognition, and auto-sorting
|
||||
"""
|
||||
|
||||
import os
|
||||
import json
|
||||
import logging
|
||||
import pickle
|
||||
import shutil
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
from typing import List, Dict, Optional, Tuple
|
||||
|
||||
import face_recognition
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class FaceRecognitionManager:
|
||||
"""Manages face recognition and auto-sorting"""
|
||||
|
||||
def __init__(self, db_path: str, config: dict):
|
||||
self.db_path = db_path
|
||||
self.config = config
|
||||
|
||||
# Configuration
|
||||
self.enabled = config.get('face_recognition', {}).get('enabled', False)
|
||||
self.confidence_threshold = config.get('face_recognition', {}).get('confidence_threshold', 0.6)
|
||||
self.auto_sort_threshold = config.get('face_recognition', {}).get('auto_sort_threshold', 0.5)
|
||||
self.base_directory = config.get('face_recognition', {}).get('base_directory', '/mnt/storage/Downloads/faces')
|
||||
self.review_queue_dir = os.path.join(self.base_directory, 'review_queue')
|
||||
|
||||
# Create directories
|
||||
os.makedirs(self.base_directory, exist_ok=True)
|
||||
os.makedirs(self.review_queue_dir, exist_ok=True)
|
||||
|
||||
# Initialize database tables
|
||||
self._init_database()
|
||||
|
||||
# Load known faces into memory
|
||||
self.known_faces = {} # person_id: [encodings]
|
||||
self._load_known_faces()
|
||||
|
||||
def _init_database(self):
|
||||
"""Create face recognition tables"""
|
||||
with sqlite3.connect(self.db_path) as conn:
|
||||
conn.execute("""
|
||||
CREATE TABLE IF NOT EXISTS face_recognition_people (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
name TEXT NOT NULL UNIQUE,
|
||||
directory TEXT NOT NULL,
|
||||
face_encodings BLOB,
|
||||
created_at TEXT,
|
||||
updated_at TEXT,
|
||||
enabled INTEGER DEFAULT 1
|
||||
)
|
||||
""")
|
||||
|
||||
conn.execute("""
|
||||
CREATE TABLE IF NOT EXISTS face_recognition_queue (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
download_id INTEGER,
|
||||
file_path TEXT NOT NULL,
|
||||
thumbnail_path TEXT,
|
||||
face_encoding BLOB,
|
||||
face_location TEXT,
|
||||
confidence REAL,
|
||||
suggested_person_id INTEGER,
|
||||
status TEXT DEFAULT 'pending',
|
||||
created_at TEXT,
|
||||
reviewed_at TEXT,
|
||||
reviewed_by TEXT,
|
||||
FOREIGN KEY (download_id) REFERENCES downloads(id),
|
||||
FOREIGN KEY (suggested_person_id) REFERENCES face_recognition_people(id)
|
||||
)
|
||||
""")
|
||||
|
||||
conn.execute("""
|
||||
CREATE TABLE IF NOT EXISTS face_recognition_history (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
download_id INTEGER,
|
||||
file_path TEXT NOT NULL,
|
||||
person_id INTEGER,
|
||||
confidence REAL,
|
||||
action TEXT,
|
||||
processed_at TEXT,
|
||||
FOREIGN KEY (download_id) REFERENCES downloads(id),
|
||||
FOREIGN KEY (person_id) REFERENCES face_recognition_people(id)
|
||||
)
|
||||
""")
|
||||
|
||||
conn.commit()
|
||||
|
||||
def _load_known_faces(self):
|
||||
"""Load known face encodings from database"""
|
||||
with sqlite3.connect(self.db_path) as conn:
|
||||
cursor = conn.execute("""
|
||||
SELECT id, name, face_encodings
|
||||
FROM face_recognition_people
|
||||
WHERE enabled = 1
|
||||
""")
|
||||
|
||||
for person_id, name, encodings_blob in cursor.fetchall():
|
||||
if encodings_blob:
|
||||
encodings = pickle.loads(encodings_blob)
|
||||
self.known_faces[person_id] = {
|
||||
'name': name,
|
||||
'encodings': encodings
|
||||
}
|
||||
|
||||
logger.info(f"Loaded {len(self.known_faces)} known people")
|
||||
|
||||
def process_image(self, file_path: str, download_id: Optional[int] = None) -> Dict:
|
||||
"""
|
||||
Process an image for face recognition
|
||||
|
||||
Returns:
|
||||
dict: {
|
||||
'status': 'success'|'error'|'no_faces'|'skipped',
|
||||
'action': 'auto_sorted'|'queued'|'skipped',
|
||||
'person_id': int or None,
|
||||
'person_name': str or None,
|
||||
'confidence': float or None,
|
||||
'faces_found': int,
|
||||
'message': str
|
||||
}
|
||||
"""
|
||||
if not self.enabled:
|
||||
return {'status': 'skipped', 'message': 'Face recognition disabled'}
|
||||
|
||||
if not os.path.exists(file_path):
|
||||
return {'status': 'error', 'message': 'File not found'}
|
||||
|
||||
# Only process image files
|
||||
ext = os.path.splitext(file_path)[1].lower()
|
||||
if ext not in ['.jpg', '.jpeg', '.png', '.heic', '.heif']:
|
||||
return {'status': 'skipped', 'message': 'Not an image file'}
|
||||
|
||||
try:
|
||||
# Load image
|
||||
image = face_recognition.load_image_file(file_path)
|
||||
|
||||
# Find faces
|
||||
face_locations = face_recognition.face_locations(image)
|
||||
|
||||
if not face_locations:
|
||||
logger.debug(f"No faces found in {file_path}")
|
||||
return {
|
||||
'status': 'no_faces',
|
||||
'action': 'skipped',
|
||||
'faces_found': 0,
|
||||
'message': 'No faces detected'
|
||||
}
|
||||
|
||||
# Get face encodings
|
||||
face_encodings = face_recognition.face_encodings(image, face_locations)
|
||||
|
||||
# Handle multiple faces
|
||||
if len(face_encodings) > 1:
|
||||
return self._handle_multiple_faces(
|
||||
file_path, download_id, face_encodings, face_locations
|
||||
)
|
||||
|
||||
# Single face - try to match
|
||||
encoding = face_encodings[0]
|
||||
location = face_locations[0]
|
||||
|
||||
match_result = self._find_best_match(encoding)
|
||||
|
||||
if match_result and match_result['confidence'] >= self.auto_sort_threshold:
|
||||
# High confidence - auto sort
|
||||
return self._auto_sort_image(
|
||||
file_path, download_id, match_result['person_id'],
|
||||
match_result['confidence'], encoding, location
|
||||
)
|
||||
else:
|
||||
# Low confidence or no match - queue for review
|
||||
return self._queue_for_review(
|
||||
file_path, download_id, encoding, location,
|
||||
match_result['person_id'] if match_result else None,
|
||||
match_result['confidence'] if match_result else None
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error processing {file_path}: {e}")
|
||||
return {'status': 'error', 'message': str(e)}
|
||||
|
||||
def _find_best_match(self, face_encoding: np.ndarray) -> Optional[Dict]:
|
||||
"""
|
||||
Find best matching person for a face encoding
|
||||
|
||||
Returns:
|
||||
dict: {'person_id': int, 'name': str, 'confidence': float} or None
|
||||
"""
|
||||
if not self.known_faces:
|
||||
return None
|
||||
|
||||
best_match = None
|
||||
best_distance = float('inf')
|
||||
|
||||
for person_id, person_data in self.known_faces.items():
|
||||
for known_encoding in person_data['encodings']:
|
||||
distance = face_recognition.face_distance([known_encoding], face_encoding)[0]
|
||||
|
||||
if distance < best_distance:
|
||||
best_distance = distance
|
||||
best_match = {
|
||||
'person_id': person_id,
|
||||
'name': person_data['name'],
|
||||
'confidence': 1.0 - distance # Convert distance to confidence
|
||||
}
|
||||
|
||||
if best_match and best_match['confidence'] >= self.confidence_threshold:
|
||||
return best_match
|
||||
|
||||
return None
|
||||
|
||||
def _auto_sort_image(self, file_path: str, download_id: Optional[int],
|
||||
person_id: int, confidence: float,
|
||||
encoding: np.ndarray, location: Tuple) -> Dict:
|
||||
"""Move image to person's directory"""
|
||||
|
||||
# Get person info
|
||||
with sqlite3.connect(self.db_path) as conn:
|
||||
cursor = conn.execute(
|
||||
"SELECT name, directory FROM face_recognition_people WHERE id = ?",
|
||||
(person_id,)
|
||||
)
|
||||
row = cursor.fetchone()
|
||||
if not row:
|
||||
return {'status': 'error', 'message': 'Person not found'}
|
||||
|
||||
person_name, person_dir = row
|
||||
|
||||
# Create person directory
|
||||
target_dir = os.path.join(self.base_directory, person_dir)
|
||||
os.makedirs(target_dir, exist_ok=True)
|
||||
|
||||
# Move file
|
||||
filename = os.path.basename(file_path)
|
||||
target_path = os.path.join(target_dir, filename)
|
||||
|
||||
try:
|
||||
shutil.move(file_path, target_path)
|
||||
logger.info(f"Auto-sorted {filename} to {person_name} (confidence: {confidence:.2f})")
|
||||
|
||||
# Record in history
|
||||
with sqlite3.connect(self.db_path) as conn:
|
||||
conn.execute("""
|
||||
INSERT INTO face_recognition_history
|
||||
(download_id, file_path, person_id, confidence, action, processed_at)
|
||||
VALUES (?, ?, ?, ?, 'auto_sorted', ?)
|
||||
""", (download_id, target_path, person_id, confidence, datetime.now().isoformat()))
|
||||
conn.commit()
|
||||
|
||||
return {
|
||||
'status': 'success',
|
||||
'action': 'auto_sorted',
|
||||
'person_id': person_id,
|
||||
'person_name': person_name,
|
||||
'confidence': confidence,
|
||||
'faces_found': 1,
|
||||
'new_path': target_path,
|
||||
'message': f'Auto-sorted to {person_name}'
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error moving file: {e}")
|
||||
return {'status': 'error', 'message': str(e)}
|
||||
|
||||
def _queue_for_review(self, file_path: str, download_id: Optional[int],
|
||||
encoding: np.ndarray, location: Tuple,
|
||||
suggested_person_id: Optional[int] = None,
|
||||
confidence: Optional[float] = None) -> Dict:
|
||||
"""Add image to review queue"""
|
||||
|
||||
# Copy file to review queue
|
||||
filename = os.path.basename(file_path)
|
||||
queue_filename = f"queue_{datetime.now().strftime('%Y%m%d_%H%M%S')}_{filename}"
|
||||
queue_path = os.path.join(self.review_queue_dir, queue_filename)
|
||||
|
||||
try:
|
||||
shutil.copy2(file_path, queue_path)
|
||||
|
||||
# Create thumbnail showing face location
|
||||
thumbnail_path = self._create_face_thumbnail(queue_path, location)
|
||||
|
||||
# Add to queue database
|
||||
with sqlite3.connect(self.db_path) as conn:
|
||||
conn.execute("""
|
||||
INSERT INTO face_recognition_queue
|
||||
(download_id, file_path, thumbnail_path, face_encoding,
|
||||
face_location, confidence, suggested_person_id, status, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, 'pending', ?)
|
||||
""", (
|
||||
download_id, queue_path, thumbnail_path,
|
||||
pickle.dumps([encoding]), json.dumps(location),
|
||||
confidence, suggested_person_id, datetime.now().isoformat()
|
||||
))
|
||||
conn.commit()
|
||||
|
||||
logger.info(f"Queued {filename} for review (confidence: {confidence:.2f if confidence else 0})")
|
||||
|
||||
return {
|
||||
'status': 'success',
|
||||
'action': 'queued',
|
||||
'suggested_person_id': suggested_person_id,
|
||||
'confidence': confidence,
|
||||
'faces_found': 1,
|
||||
'queue_path': queue_path,
|
||||
'message': 'Queued for manual review'
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error queueing file: {e}")
|
||||
return {'status': 'error', 'message': str(e)}
|
||||
|
||||
def _handle_multiple_faces(self, file_path: str, download_id: Optional[int],
|
||||
encodings: List, locations: List) -> Dict:
|
||||
"""Handle images with multiple faces"""
|
||||
|
||||
# For now, queue all multiple-face images for review
|
||||
filename = os.path.basename(file_path)
|
||||
queue_filename = f"multiple_{datetime.now().strftime('%Y%m%d_%H%M%S')}_{filename}"
|
||||
queue_path = os.path.join(self.review_queue_dir, queue_filename)
|
||||
|
||||
try:
|
||||
shutil.copy2(file_path, queue_path)
|
||||
|
||||
# Store all face encodings
|
||||
with sqlite3.connect(self.db_path) as conn:
|
||||
conn.execute("""
|
||||
INSERT INTO face_recognition_queue
|
||||
(download_id, file_path, face_encoding, face_location, status, created_at)
|
||||
VALUES (?, ?, ?, ?, 'pending_multiple', ?)
|
||||
""", (
|
||||
download_id, queue_path,
|
||||
pickle.dumps(encodings), json.dumps(locations),
|
||||
datetime.now().isoformat()
|
||||
))
|
||||
conn.commit()
|
||||
|
||||
logger.info(f"Queued {filename} (multiple faces: {len(encodings)})")
|
||||
|
||||
return {
|
||||
'status': 'success',
|
||||
'action': 'queued',
|
||||
'faces_found': len(encodings),
|
||||
'queue_path': queue_path,
|
||||
'message': f'Queued - {len(encodings)} faces detected'
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error queueing multiple face file: {e}")
|
||||
return {'status': 'error', 'message': str(e)}
|
||||
|
||||
def _create_face_thumbnail(self, image_path: str, location: Tuple) -> str:
|
||||
"""Create thumbnail with face highlighted"""
|
||||
try:
|
||||
from PIL import Image, ImageDraw
|
||||
|
||||
img = Image.open(image_path)
|
||||
draw = ImageDraw.Draw(img)
|
||||
|
||||
# Draw rectangle around face
|
||||
top, right, bottom, left = location
|
||||
draw.rectangle(((left, top), (right, bottom)), outline="red", width=3)
|
||||
|
||||
# Save thumbnail
|
||||
thumbnail_path = image_path.replace('.jpg', '_thumb.jpg')
|
||||
img.thumbnail((300, 300))
|
||||
img.save(thumbnail_path)
|
||||
|
||||
return thumbnail_path
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error creating thumbnail: {e}")
|
||||
return None
|
||||
|
||||
# Additional methods for managing people, review queue, etc...
|
||||
# (add_person, train_from_images, review_queue_item, etc.)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Integration Points
|
||||
|
||||
#### A. Post-Download Hook
|
||||
|
||||
Modify existing download completion to trigger face recognition:
|
||||
|
||||
```python
|
||||
# In modules/download_manager.py or relevant module
|
||||
|
||||
def on_download_complete(file_path: str, download_id: int):
|
||||
"""Called when download completes"""
|
||||
|
||||
# Existing post-download tasks
|
||||
update_database(download_id)
|
||||
send_notification(download_id)
|
||||
|
||||
# NEW: Face recognition processing
|
||||
if config.get('face_recognition', {}).get('enabled', False):
|
||||
from modules.face_recognition_manager import FaceRecognitionManager
|
||||
|
||||
face_mgr = FaceRecognitionManager(db_path, config)
|
||||
result = face_mgr.process_image(file_path, download_id)
|
||||
|
||||
logger.info(f"Face recognition result: {result}")
|
||||
```
|
||||
|
||||
#### B. Configuration
|
||||
|
||||
Add to `config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"face_recognition": {
|
||||
"enabled": false,
|
||||
"confidence_threshold": 0.6,
|
||||
"auto_sort_threshold": 0.5,
|
||||
"base_directory": "/mnt/storage/Downloads/faces",
|
||||
"process_existing": false,
|
||||
"async_processing": true,
|
||||
"batch_size": 10
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### C. Web UI Integration
|
||||
|
||||
New pages needed:
|
||||
1. **Face Recognition Dashboard** - Overview, stats, enable/disable
|
||||
2. **People Management** - Add/edit/remove people, train faces
|
||||
3. **Review Queue** - Manually identify unknown faces
|
||||
4. **History** - View auto-sort history, statistics
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Implementation Phases
|
||||
|
||||
### Phase 1: Core Foundation (Week 1)
|
||||
- [ ] Install face_recognition library
|
||||
- [ ] Create database schema
|
||||
- [ ] Build FaceRecognitionManager class
|
||||
- [ ] Basic face detection and encoding
|
||||
- [ ] Test with sample images
|
||||
|
||||
### Phase 2: People Management (Week 2)
|
||||
- [ ] Add person to database
|
||||
- [ ] Train from sample images
|
||||
- [ ] Store face encodings
|
||||
- [ ] Load known faces into memory
|
||||
- [ ] Test matching algorithm
|
||||
|
||||
### Phase 3: Auto-Sorting (Week 3)
|
||||
- [ ] Integrate with download completion hook
|
||||
- [ ] Implement auto-sort logic
|
||||
- [ ] Create person directories
|
||||
- [ ] Move files automatically
|
||||
- [ ] Log history
|
||||
|
||||
### Phase 4: Review Queue (Week 4)
|
||||
- [ ] Queue unknown faces
|
||||
- [ ] Create thumbnails
|
||||
- [ ] Build web UI for review
|
||||
- [ ] Manual identification workflow
|
||||
- [ ] Learn from manual reviews
|
||||
|
||||
### Phase 5: Web Interface (Week 5-6)
|
||||
- [ ] Dashboard page
|
||||
- [ ] People management page
|
||||
- [ ] Review queue page
|
||||
- [ ] Statistics and history
|
||||
- [ ] Settings configuration
|
||||
|
||||
### Phase 6: Optimization & Polish (Week 7-8)
|
||||
- [ ] Async/background processing
|
||||
- [ ] Batch processing for existing files
|
||||
- [ ] Performance optimization
|
||||
- [ ] Error handling and logging
|
||||
- [ ] Documentation and testing
|
||||
|
||||
---
|
||||
|
||||
## 📊 API Endpoints (New)
|
||||
|
||||
```python
|
||||
# Face Recognition Management
|
||||
GET /api/face-recognition/status
|
||||
POST /api/face-recognition/enable
|
||||
POST /api/face-recognition/disable
|
||||
|
||||
# People Management
|
||||
GET /api/face-recognition/people
|
||||
POST /api/face-recognition/people # Add new person
|
||||
PUT /api/face-recognition/people/{id} # Update person
|
||||
DELETE /api/face-recognition/people/{id} # Remove person
|
||||
POST /api/face-recognition/people/{id}/train # Train with new images
|
||||
|
||||
# Review Queue
|
||||
GET /api/face-recognition/queue # Get pending items
|
||||
GET /api/face-recognition/queue/{id} # Get specific item
|
||||
POST /api/face-recognition/queue/{id}/identify # Manual identification
|
||||
POST /api/face-recognition/queue/{id}/skip # Skip this image
|
||||
DELETE /api/face-recognition/queue/{id} # Remove from queue
|
||||
|
||||
# History & Stats
|
||||
GET /api/face-recognition/history
|
||||
GET /api/face-recognition/stats
|
||||
|
||||
# Batch Processing
|
||||
POST /api/face-recognition/process-existing # Process old downloads
|
||||
GET /api/face-recognition/process-status # Check batch progress
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎨 Web UI Mockup
|
||||
|
||||
### Dashboard Page
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Face Recognition Dashboard │
|
||||
├─────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Status: [✓ Enabled] [⚙️ Configure] │
|
||||
│ │
|
||||
│ ┌───────────────────────────────────────┐ │
|
||||
│ │ Statistics │ │
|
||||
│ │ │ │
|
||||
│ │ Known People: 12 │ │
|
||||
│ │ Auto-Sorted Today: 45 │ │
|
||||
│ │ Review Queue: 8 pending │ │
|
||||
│ │ Success Rate: 94.2% │ │
|
||||
│ └───────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌───────────────────────────────────────┐ │
|
||||
│ │ Recent Activity │ │
|
||||
│ │ │ │
|
||||
│ │ • 14:23 - Auto-sorted to "John" │ │
|
||||
│ │ • 14:20 - Queued unknown face │ │
|
||||
│ │ • 14:18 - Auto-sorted to "Sarah" │ │
|
||||
│ └───────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ [Manage People] [Review Queue] [Settings] │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### People Management Page
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ People Management │
|
||||
├─────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ [+ Add New Person] │
|
||||
│ │
|
||||
│ ┌───────────────────────────────────────┐ │
|
||||
│ │ 👤 John Doe │ │
|
||||
│ │ Directory: john_doe/ │ │
|
||||
│ │ Face Samples: 25 │ │
|
||||
│ │ Images Sorted: 142 │ │
|
||||
│ │ [Edit] [Train More] [Delete] │ │
|
||||
│ └───────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌───────────────────────────────────────┐ │
|
||||
│ │ 👤 Sarah Smith │ │
|
||||
│ │ Directory: sarah_smith/ │ │
|
||||
│ │ Face Samples: 18 │ │
|
||||
│ │ Images Sorted: 89 │ │
|
||||
│ │ [Edit] [Train More] [Delete] │ │
|
||||
│ └───────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Review Queue Page
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Review Queue (8 pending) │
|
||||
├─────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌───────────────────────────────────────┐ │
|
||||
│ │ [Image Thumbnail] │ │
|
||||
│ │ │ │
|
||||
│ │ Confidence: 45% (Low) │ │
|
||||
│ │ Suggested: John Doe │ │
|
||||
│ │ │ │
|
||||
│ │ This is: [Select Person ▼] │ │
|
||||
│ │ │ │
|
||||
│ │ [✓ Confirm] [Skip] [New Person] │ │
|
||||
│ └───────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ [◄ Previous] [Next ►] │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Privacy & Security
|
||||
|
||||
1. **Local Processing Only** - No cloud APIs, all processing local
|
||||
2. **Encrypted Storage** - Face encodings stored securely
|
||||
3. **User Control** - Easy enable/disable, delete data anytime
|
||||
4. **Access Control** - Face recognition UI requires authentication
|
||||
5. **Audit Trail** - All auto-sort actions logged with confidence scores
|
||||
|
||||
---
|
||||
|
||||
## ⚡ Performance Considerations
|
||||
|
||||
### Processing Speed
|
||||
- Face detection: ~0.5-1 sec per image
|
||||
- Face recognition: ~0.1 sec per comparison
|
||||
- Total per image: 1-3 seconds
|
||||
|
||||
### Optimization Strategies
|
||||
1. **Async Processing** - Process in background, don't block downloads
|
||||
2. **Batch Processing** - Process multiple images in parallel
|
||||
3. **Caching** - Keep known face encodings in memory
|
||||
4. **Smart Queueing** - Process high-priority images first
|
||||
5. **CPU vs GPU** - Optional GPU acceleration for faster processing
|
||||
|
||||
---
|
||||
|
||||
## 📝 Configuration Example
|
||||
|
||||
```json
|
||||
{
|
||||
"face_recognition": {
|
||||
"enabled": true,
|
||||
"confidence_threshold": 0.6,
|
||||
"auto_sort_threshold": 0.5,
|
||||
"base_directory": "/mnt/storage/Downloads/faces",
|
||||
"review_queue_dir": "/mnt/storage/Downloads/faces/review_queue",
|
||||
"process_existing": false,
|
||||
"async_processing": true,
|
||||
"batch_size": 10,
|
||||
"max_faces_per_image": 5,
|
||||
"create_thumbnails": true,
|
||||
"notify_on_queue": true,
|
||||
"gpu_acceleration": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing Plan
|
||||
|
||||
### Unit Tests
|
||||
- Face detection accuracy
|
||||
- Face matching accuracy
|
||||
- Database operations
|
||||
- File operations
|
||||
|
||||
### Integration Tests
|
||||
- End-to-end download → face recognition → sort
|
||||
- Review queue workflow
|
||||
- Training new people
|
||||
|
||||
### Performance Tests
|
||||
- Processing speed benchmarks
|
||||
- Memory usage monitoring
|
||||
- Concurrent processing
|
||||
|
||||
---
|
||||
|
||||
## 📈 Success Metrics
|
||||
|
||||
- **Accuracy**: >90% correct auto-sort rate
|
||||
- **Performance**: <3 seconds per image processing
|
||||
- **Usability**: <5 minutes to add and train new person
|
||||
- **Review Queue**: <10% of images requiring manual review
|
||||
- **Stability**: No crashes or errors during processing
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Getting Started (Once Implemented)
|
||||
|
||||
### 1. Enable Face Recognition
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip3 install face_recognition pillow
|
||||
|
||||
# Enable in config
|
||||
# Set "face_recognition.enabled": true
|
||||
```
|
||||
|
||||
### 2. Add Your First Person
|
||||
```python
|
||||
# Via Web UI or CLI
|
||||
# 1. Create person
|
||||
# 2. Upload 5-10 sample images
|
||||
# 3. Train face recognition
|
||||
```
|
||||
|
||||
### 3. Process Images
|
||||
```bash
|
||||
# Automatic: New downloads are processed automatically
|
||||
# Manual: Process existing downloads
|
||||
curl -X POST http://localhost:8000/api/face-recognition/process-existing
|
||||
```
|
||||
|
||||
### 4. Review Unknown Faces
|
||||
- Open Review Queue in web UI
|
||||
- Identify unknown faces
|
||||
- System learns from your identifications
|
||||
|
||||
---
|
||||
|
||||
## 🔮 Future Enhancements
|
||||
|
||||
### v2 Features
|
||||
- **Multiple face handling** - Split images with multiple people
|
||||
- **Age progression** - Recognize people across different ages
|
||||
- **Group detection** - Automatically create "group" folders
|
||||
- **Emotion detection** - Filter by happy/sad expressions
|
||||
- **Quality scoring** - Auto-select best photos of each person
|
||||
- **Duplicate detection** - Find similar poses/angles
|
||||
|
||||
### v3 Features
|
||||
- **Video support** - Extract faces from videos
|
||||
- **Live camera** - Real-time face recognition
|
||||
- **Object detection** - Sort by objects/scenes too
|
||||
- **Tag suggestions** - AI-powered photo tagging
|
||||
- **Smart albums** - Auto-generate albums by person/event
|
||||
|
||||
---
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
### Libraries
|
||||
- [face_recognition](https://github.com/ageitgey/face_recognition) - Main library
|
||||
- [dlib](http://dlib.net/) - Face detection engine
|
||||
- [OpenCV](https://opencv.org/) - Image processing
|
||||
|
||||
### Documentation
|
||||
- [Face Recognition Tutorial](https://www.pyimagesearch.com/2018/06/18/face-recognition-with-opencv-python-and-deep-learning/)
|
||||
- [DeepFace GitHub](https://github.com/serengil/deepface)
|
||||
|
||||
---
|
||||
|
||||
**Status**: Ready for implementation
|
||||
**Next Step**: Phase 1 - Install dependencies and build core foundation
|
||||
**Questions**: See [IMPLEMENTATION_GUIDE.md] for step-by-step instructions
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-10-31
|
||||
Reference in New Issue
Block a user