959 lines
33 KiB
Markdown
959 lines
33 KiB
Markdown
# AI-Powered Face Recognition & Auto-Sorting System
|
|
|
|
**Created**: 2025-10-31
|
|
**Status**: Planning Phase
|
|
**Target Version**: 6.5.0
|
|
|
|
---
|
|
|
|
## 📋 Overview
|
|
|
|
Automatic face recognition and sorting system that processes downloaded images, identifies people, and organizes them into person-specific directories. Unknown faces go to a review queue for manual identification.
|
|
|
|
---
|
|
|
|
## 🎯 Goals
|
|
|
|
### Primary Goals
|
|
1. **Automatic face detection** - Identify faces in downloaded images
|
|
2. **Face recognition** - Match faces against known people database
|
|
3. **Auto-sorting** - Move matched images to person-specific directories
|
|
4. **Review queue** - Queue unknown faces for manual identification
|
|
5. **Learning system** - Improve recognition from manual reviews
|
|
|
|
### Secondary Goals
|
|
6. **Multi-face support** - Handle images with multiple people
|
|
7. **Confidence scoring** - Only auto-sort high confidence matches
|
|
8. **Performance** - Process images quickly without blocking downloads
|
|
9. **Privacy** - All processing done locally (no cloud APIs)
|
|
10. **Immich integration** - Sync sorted images to Immich
|
|
|
|
---
|
|
|
|
## 🏗️ Architecture
|
|
|
|
### High-Level Flow
|
|
|
|
```
|
|
┌─────────────────┐
|
|
│ Image Download │
|
|
│ Complete │
|
|
└────────┬────────┘
|
|
│
|
|
▼
|
|
┌─────────────────┐
|
|
│ Face Detection │ ◄── Uses face_recognition library
|
|
│ (Find Faces) │ or DeepFace
|
|
└────────┬────────┘
|
|
│
|
|
├─── No faces found ──► Skip (keep in original location)
|
|
│
|
|
▼
|
|
┌─────────────────┐
|
|
│ Face Recognition│ ◄── Compare against known faces DB
|
|
│ (Identify Who) │
|
|
└────────┬────────┘
|
|
│
|
|
├─── High confidence match ──► Auto-sort to person directory
|
|
│
|
|
├─── Low confidence/Multiple ──► Review Queue
|
|
│
|
|
└─── Unknown face ──────────► Review Queue
|
|
```
|
|
|
|
### Database Schema
|
|
|
|
```sql
|
|
-- New table: face_recognition_people
|
|
CREATE TABLE face_recognition_people (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
name TEXT NOT NULL UNIQUE,
|
|
directory TEXT NOT NULL, -- Target directory for this person
|
|
face_encodings BLOB, -- Stored face encodings (multiple per person)
|
|
created_at TEXT,
|
|
updated_at TEXT,
|
|
enabled INTEGER DEFAULT 1
|
|
);
|
|
|
|
-- New table: face_recognition_queue
|
|
CREATE TABLE face_recognition_queue (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
download_id INTEGER,
|
|
file_path TEXT NOT NULL,
|
|
thumbnail_path TEXT,
|
|
face_encoding BLOB, -- Encoding of the face found
|
|
face_location TEXT, -- JSON: bounding box coordinates
|
|
confidence REAL, -- Match confidence if any
|
|
suggested_person_id INTEGER, -- Best match suggestion
|
|
status TEXT DEFAULT 'pending', -- pending, reviewed, skipped
|
|
created_at TEXT,
|
|
reviewed_at TEXT,
|
|
reviewed_by TEXT,
|
|
FOREIGN KEY (download_id) REFERENCES downloads(id),
|
|
FOREIGN KEY (suggested_person_id) REFERENCES face_recognition_people(id)
|
|
);
|
|
|
|
-- New table: face_recognition_history
|
|
CREATE TABLE face_recognition_history (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
download_id INTEGER,
|
|
file_path TEXT NOT NULL,
|
|
person_id INTEGER,
|
|
confidence REAL,
|
|
action TEXT, -- auto_sorted, manually_sorted, skipped
|
|
processed_at TEXT,
|
|
FOREIGN KEY (download_id) REFERENCES downloads(id),
|
|
FOREIGN KEY (person_id) REFERENCES face_recognition_people(id)
|
|
);
|
|
```
|
|
|
|
### Directory Structure
|
|
|
|
```
|
|
/mnt/storage/Downloads/
|
|
├── [existing platform directories]/
|
|
│ └── [original downloads]
|
|
│
|
|
├── faces/
|
|
│ ├── person1_name/
|
|
│ │ ├── 20250131_120000_abc123.jpg
|
|
│ │ └── 20250131_130000_def456.jpg
|
|
│ │
|
|
│ ├── person2_name/
|
|
│ │ └── 20250131_140000_ghi789.jpg
|
|
│ │
|
|
│ └── review_queue/
|
|
│ ├── unknown_face_20250131_120000_abc123.jpg
|
|
│ ├── low_confidence_20250131_130000_def456.jpg
|
|
│ └── multiple_faces_20250131_140000_ghi789.jpg
|
|
```
|
|
|
|
---
|
|
|
|
## 🔧 Technical Implementation
|
|
|
|
### 1. Face Recognition Library Options
|
|
|
|
#### Option A: face_recognition (Recommended)
|
|
**Pros**:
|
|
- Built on dlib (very accurate)
|
|
- Simple Python API
|
|
- Fast face detection and recognition
|
|
- Well-documented
|
|
- Works offline
|
|
|
|
**Cons**:
|
|
- Requires dlib compilation (can be slow to install)
|
|
- Heavy dependencies
|
|
|
|
**Installation**:
|
|
```bash
|
|
pip3 install face_recognition
|
|
pip3 install pillow
|
|
```
|
|
|
|
**Usage Example**:
|
|
```python
|
|
import face_recognition
|
|
import numpy as np
|
|
|
|
# Load and encode known face
|
|
image = face_recognition.load_image_file("person1.jpg")
|
|
encoding = face_recognition.face_encodings(image)[0]
|
|
|
|
# Compare with new image
|
|
unknown_image = face_recognition.load_image_file("unknown.jpg")
|
|
unknown_encodings = face_recognition.face_encodings(unknown_image)
|
|
|
|
matches = face_recognition.compare_faces([encoding], unknown_encodings[0])
|
|
distance = face_recognition.face_distance([encoding], unknown_encodings[0])
|
|
```
|
|
|
|
#### Option B: DeepFace
|
|
**Pros**:
|
|
- Multiple backend models (VGG-Face, Facenet, OpenFace, DeepID, ArcFace)
|
|
- Very high accuracy
|
|
- Age, gender, emotion detection
|
|
|
|
**Cons**:
|
|
- Slower than face_recognition
|
|
- More complex setup
|
|
- Larger dependencies
|
|
|
|
#### Option C: OpenCV + dlib
|
|
**Pros**:
|
|
- Already installed (OpenCV used elsewhere)
|
|
- Full control
|
|
- Fast face detection
|
|
|
|
**Cons**:
|
|
- More manual coding
|
|
- Complex face encoding
|
|
|
|
**Recommendation**: Start with **face_recognition** (Option A) for best balance.
|
|
|
|
---
|
|
|
|
### 2. Core Module Structure
|
|
|
|
#### New File: `modules/face_recognition_manager.py`
|
|
|
|
```python
|
|
#!/usr/bin/env python3
|
|
"""
|
|
Face Recognition Manager
|
|
Handles face detection, recognition, and auto-sorting
|
|
"""
|
|
|
|
import os
|
|
import json
|
|
import logging
|
|
import pickle
|
|
import shutil
|
|
import sqlite3
|
|
from pathlib import Path
|
|
from datetime import datetime
|
|
from typing import List, Dict, Optional, Tuple
|
|
|
|
import face_recognition
|
|
import numpy as np
|
|
from PIL import Image
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
class FaceRecognitionManager:
|
|
"""Manages face recognition and auto-sorting"""
|
|
|
|
def __init__(self, db_path: str, config: dict):
|
|
self.db_path = db_path
|
|
self.config = config
|
|
|
|
# Configuration
|
|
self.enabled = config.get('face_recognition', {}).get('enabled', False)
|
|
self.confidence_threshold = config.get('face_recognition', {}).get('confidence_threshold', 0.6)
|
|
self.auto_sort_threshold = config.get('face_recognition', {}).get('auto_sort_threshold', 0.5)
|
|
self.base_directory = config.get('face_recognition', {}).get('base_directory', '/mnt/storage/Downloads/faces')
|
|
self.review_queue_dir = os.path.join(self.base_directory, 'review_queue')
|
|
|
|
# Create directories
|
|
os.makedirs(self.base_directory, exist_ok=True)
|
|
os.makedirs(self.review_queue_dir, exist_ok=True)
|
|
|
|
# Initialize database tables
|
|
self._init_database()
|
|
|
|
# Load known faces into memory
|
|
self.known_faces = {} # person_id: [encodings]
|
|
self._load_known_faces()
|
|
|
|
def _init_database(self):
|
|
"""Create face recognition tables"""
|
|
with sqlite3.connect(self.db_path) as conn:
|
|
conn.execute("""
|
|
CREATE TABLE IF NOT EXISTS face_recognition_people (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
name TEXT NOT NULL UNIQUE,
|
|
directory TEXT NOT NULL,
|
|
face_encodings BLOB,
|
|
created_at TEXT,
|
|
updated_at TEXT,
|
|
enabled INTEGER DEFAULT 1
|
|
)
|
|
""")
|
|
|
|
conn.execute("""
|
|
CREATE TABLE IF NOT EXISTS face_recognition_queue (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
download_id INTEGER,
|
|
file_path TEXT NOT NULL,
|
|
thumbnail_path TEXT,
|
|
face_encoding BLOB,
|
|
face_location TEXT,
|
|
confidence REAL,
|
|
suggested_person_id INTEGER,
|
|
status TEXT DEFAULT 'pending',
|
|
created_at TEXT,
|
|
reviewed_at TEXT,
|
|
reviewed_by TEXT,
|
|
FOREIGN KEY (download_id) REFERENCES downloads(id),
|
|
FOREIGN KEY (suggested_person_id) REFERENCES face_recognition_people(id)
|
|
)
|
|
""")
|
|
|
|
conn.execute("""
|
|
CREATE TABLE IF NOT EXISTS face_recognition_history (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
download_id INTEGER,
|
|
file_path TEXT NOT NULL,
|
|
person_id INTEGER,
|
|
confidence REAL,
|
|
action TEXT,
|
|
processed_at TEXT,
|
|
FOREIGN KEY (download_id) REFERENCES downloads(id),
|
|
FOREIGN KEY (person_id) REFERENCES face_recognition_people(id)
|
|
)
|
|
""")
|
|
|
|
conn.commit()
|
|
|
|
def _load_known_faces(self):
|
|
"""Load known face encodings from database"""
|
|
with sqlite3.connect(self.db_path) as conn:
|
|
cursor = conn.execute("""
|
|
SELECT id, name, face_encodings
|
|
FROM face_recognition_people
|
|
WHERE enabled = 1
|
|
""")
|
|
|
|
for person_id, name, encodings_blob in cursor.fetchall():
|
|
if encodings_blob:
|
|
encodings = pickle.loads(encodings_blob)
|
|
self.known_faces[person_id] = {
|
|
'name': name,
|
|
'encodings': encodings
|
|
}
|
|
|
|
logger.info(f"Loaded {len(self.known_faces)} known people")
|
|
|
|
def process_image(self, file_path: str, download_id: Optional[int] = None) -> Dict:
|
|
"""
|
|
Process an image for face recognition
|
|
|
|
Returns:
|
|
dict: {
|
|
'status': 'success'|'error'|'no_faces'|'skipped',
|
|
'action': 'auto_sorted'|'queued'|'skipped',
|
|
'person_id': int or None,
|
|
'person_name': str or None,
|
|
'confidence': float or None,
|
|
'faces_found': int,
|
|
'message': str
|
|
}
|
|
"""
|
|
if not self.enabled:
|
|
return {'status': 'skipped', 'message': 'Face recognition disabled'}
|
|
|
|
if not os.path.exists(file_path):
|
|
return {'status': 'error', 'message': 'File not found'}
|
|
|
|
# Only process image files
|
|
ext = os.path.splitext(file_path)[1].lower()
|
|
if ext not in ['.jpg', '.jpeg', '.png', '.heic', '.heif']:
|
|
return {'status': 'skipped', 'message': 'Not an image file'}
|
|
|
|
try:
|
|
# Load image
|
|
image = face_recognition.load_image_file(file_path)
|
|
|
|
# Find faces
|
|
face_locations = face_recognition.face_locations(image)
|
|
|
|
if not face_locations:
|
|
logger.debug(f"No faces found in {file_path}")
|
|
return {
|
|
'status': 'no_faces',
|
|
'action': 'skipped',
|
|
'faces_found': 0,
|
|
'message': 'No faces detected'
|
|
}
|
|
|
|
# Get face encodings
|
|
face_encodings = face_recognition.face_encodings(image, face_locations)
|
|
|
|
# Handle multiple faces
|
|
if len(face_encodings) > 1:
|
|
return self._handle_multiple_faces(
|
|
file_path, download_id, face_encodings, face_locations
|
|
)
|
|
|
|
# Single face - try to match
|
|
encoding = face_encodings[0]
|
|
location = face_locations[0]
|
|
|
|
match_result = self._find_best_match(encoding)
|
|
|
|
if match_result and match_result['confidence'] >= self.auto_sort_threshold:
|
|
# High confidence - auto sort
|
|
return self._auto_sort_image(
|
|
file_path, download_id, match_result['person_id'],
|
|
match_result['confidence'], encoding, location
|
|
)
|
|
else:
|
|
# Low confidence or no match - queue for review
|
|
return self._queue_for_review(
|
|
file_path, download_id, encoding, location,
|
|
match_result['person_id'] if match_result else None,
|
|
match_result['confidence'] if match_result else None
|
|
)
|
|
|
|
except Exception as e:
|
|
logger.error(f"Error processing {file_path}: {e}")
|
|
return {'status': 'error', 'message': str(e)}
|
|
|
|
def _find_best_match(self, face_encoding: np.ndarray) -> Optional[Dict]:
|
|
"""
|
|
Find best matching person for a face encoding
|
|
|
|
Returns:
|
|
dict: {'person_id': int, 'name': str, 'confidence': float} or None
|
|
"""
|
|
if not self.known_faces:
|
|
return None
|
|
|
|
best_match = None
|
|
best_distance = float('inf')
|
|
|
|
for person_id, person_data in self.known_faces.items():
|
|
for known_encoding in person_data['encodings']:
|
|
distance = face_recognition.face_distance([known_encoding], face_encoding)[0]
|
|
|
|
if distance < best_distance:
|
|
best_distance = distance
|
|
best_match = {
|
|
'person_id': person_id,
|
|
'name': person_data['name'],
|
|
'confidence': 1.0 - distance # Convert distance to confidence
|
|
}
|
|
|
|
if best_match and best_match['confidence'] >= self.confidence_threshold:
|
|
return best_match
|
|
|
|
return None
|
|
|
|
def _auto_sort_image(self, file_path: str, download_id: Optional[int],
|
|
person_id: int, confidence: float,
|
|
encoding: np.ndarray, location: Tuple) -> Dict:
|
|
"""Move image to person's directory"""
|
|
|
|
# Get person info
|
|
with sqlite3.connect(self.db_path) as conn:
|
|
cursor = conn.execute(
|
|
"SELECT name, directory FROM face_recognition_people WHERE id = ?",
|
|
(person_id,)
|
|
)
|
|
row = cursor.fetchone()
|
|
if not row:
|
|
return {'status': 'error', 'message': 'Person not found'}
|
|
|
|
person_name, person_dir = row
|
|
|
|
# Create person directory
|
|
target_dir = os.path.join(self.base_directory, person_dir)
|
|
os.makedirs(target_dir, exist_ok=True)
|
|
|
|
# Move file
|
|
filename = os.path.basename(file_path)
|
|
target_path = os.path.join(target_dir, filename)
|
|
|
|
try:
|
|
shutil.move(file_path, target_path)
|
|
logger.info(f"Auto-sorted {filename} to {person_name} (confidence: {confidence:.2f})")
|
|
|
|
# Record in history
|
|
with sqlite3.connect(self.db_path) as conn:
|
|
conn.execute("""
|
|
INSERT INTO face_recognition_history
|
|
(download_id, file_path, person_id, confidence, action, processed_at)
|
|
VALUES (?, ?, ?, ?, 'auto_sorted', ?)
|
|
""", (download_id, target_path, person_id, confidence, datetime.now().isoformat()))
|
|
conn.commit()
|
|
|
|
return {
|
|
'status': 'success',
|
|
'action': 'auto_sorted',
|
|
'person_id': person_id,
|
|
'person_name': person_name,
|
|
'confidence': confidence,
|
|
'faces_found': 1,
|
|
'new_path': target_path,
|
|
'message': f'Auto-sorted to {person_name}'
|
|
}
|
|
|
|
except Exception as e:
|
|
logger.error(f"Error moving file: {e}")
|
|
return {'status': 'error', 'message': str(e)}
|
|
|
|
def _queue_for_review(self, file_path: str, download_id: Optional[int],
|
|
encoding: np.ndarray, location: Tuple,
|
|
suggested_person_id: Optional[int] = None,
|
|
confidence: Optional[float] = None) -> Dict:
|
|
"""Add image to review queue"""
|
|
|
|
# Copy file to review queue
|
|
filename = os.path.basename(file_path)
|
|
queue_filename = f"queue_{datetime.now().strftime('%Y%m%d_%H%M%S')}_{filename}"
|
|
queue_path = os.path.join(self.review_queue_dir, queue_filename)
|
|
|
|
try:
|
|
shutil.copy2(file_path, queue_path)
|
|
|
|
# Create thumbnail showing face location
|
|
thumbnail_path = self._create_face_thumbnail(queue_path, location)
|
|
|
|
# Add to queue database
|
|
with sqlite3.connect(self.db_path) as conn:
|
|
conn.execute("""
|
|
INSERT INTO face_recognition_queue
|
|
(download_id, file_path, thumbnail_path, face_encoding,
|
|
face_location, confidence, suggested_person_id, status, created_at)
|
|
VALUES (?, ?, ?, ?, ?, ?, ?, 'pending', ?)
|
|
""", (
|
|
download_id, queue_path, thumbnail_path,
|
|
pickle.dumps([encoding]), json.dumps(location),
|
|
confidence, suggested_person_id, datetime.now().isoformat()
|
|
))
|
|
conn.commit()
|
|
|
|
logger.info(f"Queued {filename} for review (confidence: {confidence:.2f if confidence else 0})")
|
|
|
|
return {
|
|
'status': 'success',
|
|
'action': 'queued',
|
|
'suggested_person_id': suggested_person_id,
|
|
'confidence': confidence,
|
|
'faces_found': 1,
|
|
'queue_path': queue_path,
|
|
'message': 'Queued for manual review'
|
|
}
|
|
|
|
except Exception as e:
|
|
logger.error(f"Error queueing file: {e}")
|
|
return {'status': 'error', 'message': str(e)}
|
|
|
|
def _handle_multiple_faces(self, file_path: str, download_id: Optional[int],
|
|
encodings: List, locations: List) -> Dict:
|
|
"""Handle images with multiple faces"""
|
|
|
|
# For now, queue all multiple-face images for review
|
|
filename = os.path.basename(file_path)
|
|
queue_filename = f"multiple_{datetime.now().strftime('%Y%m%d_%H%M%S')}_{filename}"
|
|
queue_path = os.path.join(self.review_queue_dir, queue_filename)
|
|
|
|
try:
|
|
shutil.copy2(file_path, queue_path)
|
|
|
|
# Store all face encodings
|
|
with sqlite3.connect(self.db_path) as conn:
|
|
conn.execute("""
|
|
INSERT INTO face_recognition_queue
|
|
(download_id, file_path, face_encoding, face_location, status, created_at)
|
|
VALUES (?, ?, ?, ?, 'pending_multiple', ?)
|
|
""", (
|
|
download_id, queue_path,
|
|
pickle.dumps(encodings), json.dumps(locations),
|
|
datetime.now().isoformat()
|
|
))
|
|
conn.commit()
|
|
|
|
logger.info(f"Queued {filename} (multiple faces: {len(encodings)})")
|
|
|
|
return {
|
|
'status': 'success',
|
|
'action': 'queued',
|
|
'faces_found': len(encodings),
|
|
'queue_path': queue_path,
|
|
'message': f'Queued - {len(encodings)} faces detected'
|
|
}
|
|
|
|
except Exception as e:
|
|
logger.error(f"Error queueing multiple face file: {e}")
|
|
return {'status': 'error', 'message': str(e)}
|
|
|
|
def _create_face_thumbnail(self, image_path: str, location: Tuple) -> str:
|
|
"""Create thumbnail with face highlighted"""
|
|
try:
|
|
from PIL import Image, ImageDraw
|
|
|
|
img = Image.open(image_path)
|
|
draw = ImageDraw.Draw(img)
|
|
|
|
# Draw rectangle around face
|
|
top, right, bottom, left = location
|
|
draw.rectangle(((left, top), (right, bottom)), outline="red", width=3)
|
|
|
|
# Save thumbnail
|
|
thumbnail_path = image_path.replace('.jpg', '_thumb.jpg')
|
|
img.thumbnail((300, 300))
|
|
img.save(thumbnail_path)
|
|
|
|
return thumbnail_path
|
|
|
|
except Exception as e:
|
|
logger.error(f"Error creating thumbnail: {e}")
|
|
return None
|
|
|
|
# Additional methods for managing people, review queue, etc...
|
|
# (add_person, train_from_images, review_queue_item, etc.)
|
|
```
|
|
|
|
---
|
|
|
|
### 3. Integration Points
|
|
|
|
#### A. Post-Download Hook
|
|
|
|
Modify existing download completion to trigger face recognition:
|
|
|
|
```python
|
|
# In modules/download_manager.py or relevant module
|
|
|
|
def on_download_complete(file_path: str, download_id: int):
|
|
"""Called when download completes"""
|
|
|
|
# Existing post-download tasks
|
|
update_database(download_id)
|
|
send_notification(download_id)
|
|
|
|
# NEW: Face recognition processing
|
|
if config.get('face_recognition', {}).get('enabled', False):
|
|
from modules.face_recognition_manager import FaceRecognitionManager
|
|
|
|
face_mgr = FaceRecognitionManager(db_path, config)
|
|
result = face_mgr.process_image(file_path, download_id)
|
|
|
|
logger.info(f"Face recognition result: {result}")
|
|
```
|
|
|
|
#### B. Configuration
|
|
|
|
Add to `config.json`:
|
|
|
|
```json
|
|
{
|
|
"face_recognition": {
|
|
"enabled": false,
|
|
"confidence_threshold": 0.6,
|
|
"auto_sort_threshold": 0.5,
|
|
"base_directory": "/mnt/storage/Downloads/faces",
|
|
"process_existing": false,
|
|
"async_processing": true,
|
|
"batch_size": 10
|
|
}
|
|
}
|
|
```
|
|
|
|
#### C. Web UI Integration
|
|
|
|
New pages needed:
|
|
1. **Face Recognition Dashboard** - Overview, stats, enable/disable
|
|
2. **People Management** - Add/edit/remove people, train faces
|
|
3. **Review Queue** - Manually identify unknown faces
|
|
4. **History** - View auto-sort history, statistics
|
|
|
|
---
|
|
|
|
## 🚀 Implementation Phases
|
|
|
|
### Phase 1: Core Foundation (Week 1)
|
|
- [ ] Install face_recognition library
|
|
- [ ] Create database schema
|
|
- [ ] Build FaceRecognitionManager class
|
|
- [ ] Basic face detection and encoding
|
|
- [ ] Test with sample images
|
|
|
|
### Phase 2: People Management (Week 2)
|
|
- [ ] Add person to database
|
|
- [ ] Train from sample images
|
|
- [ ] Store face encodings
|
|
- [ ] Load known faces into memory
|
|
- [ ] Test matching algorithm
|
|
|
|
### Phase 3: Auto-Sorting (Week 3)
|
|
- [ ] Integrate with download completion hook
|
|
- [ ] Implement auto-sort logic
|
|
- [ ] Create person directories
|
|
- [ ] Move files automatically
|
|
- [ ] Log history
|
|
|
|
### Phase 4: Review Queue (Week 4)
|
|
- [ ] Queue unknown faces
|
|
- [ ] Create thumbnails
|
|
- [ ] Build web UI for review
|
|
- [ ] Manual identification workflow
|
|
- [ ] Learn from manual reviews
|
|
|
|
### Phase 5: Web Interface (Week 5-6)
|
|
- [ ] Dashboard page
|
|
- [ ] People management page
|
|
- [ ] Review queue page
|
|
- [ ] Statistics and history
|
|
- [ ] Settings configuration
|
|
|
|
### Phase 6: Optimization & Polish (Week 7-8)
|
|
- [ ] Async/background processing
|
|
- [ ] Batch processing for existing files
|
|
- [ ] Performance optimization
|
|
- [ ] Error handling and logging
|
|
- [ ] Documentation and testing
|
|
|
|
---
|
|
|
|
## 📊 API Endpoints (New)
|
|
|
|
```python
|
|
# Face Recognition Management
|
|
GET /api/face-recognition/status
|
|
POST /api/face-recognition/enable
|
|
POST /api/face-recognition/disable
|
|
|
|
# People Management
|
|
GET /api/face-recognition/people
|
|
POST /api/face-recognition/people # Add new person
|
|
PUT /api/face-recognition/people/{id} # Update person
|
|
DELETE /api/face-recognition/people/{id} # Remove person
|
|
POST /api/face-recognition/people/{id}/train # Train with new images
|
|
|
|
# Review Queue
|
|
GET /api/face-recognition/queue # Get pending items
|
|
GET /api/face-recognition/queue/{id} # Get specific item
|
|
POST /api/face-recognition/queue/{id}/identify # Manual identification
|
|
POST /api/face-recognition/queue/{id}/skip # Skip this image
|
|
DELETE /api/face-recognition/queue/{id} # Remove from queue
|
|
|
|
# History & Stats
|
|
GET /api/face-recognition/history
|
|
GET /api/face-recognition/stats
|
|
|
|
# Batch Processing
|
|
POST /api/face-recognition/process-existing # Process old downloads
|
|
GET /api/face-recognition/process-status # Check batch progress
|
|
```
|
|
|
|
---
|
|
|
|
## 🎨 Web UI Mockup
|
|
|
|
### Dashboard Page
|
|
|
|
```
|
|
┌─────────────────────────────────────────────┐
|
|
│ Face Recognition Dashboard │
|
|
├─────────────────────────────────────────────┤
|
|
│ │
|
|
│ Status: [✓ Enabled] [⚙️ Configure] │
|
|
│ │
|
|
│ ┌───────────────────────────────────────┐ │
|
|
│ │ Statistics │ │
|
|
│ │ │ │
|
|
│ │ Known People: 12 │ │
|
|
│ │ Auto-Sorted Today: 45 │ │
|
|
│ │ Review Queue: 8 pending │ │
|
|
│ │ Success Rate: 94.2% │ │
|
|
│ └───────────────────────────────────────┘ │
|
|
│ │
|
|
│ ┌───────────────────────────────────────┐ │
|
|
│ │ Recent Activity │ │
|
|
│ │ │ │
|
|
│ │ • 14:23 - Auto-sorted to "John" │ │
|
|
│ │ • 14:20 - Queued unknown face │ │
|
|
│ │ • 14:18 - Auto-sorted to "Sarah" │ │
|
|
│ └───────────────────────────────────────┘ │
|
|
│ │
|
|
│ [Manage People] [Review Queue] [Settings] │
|
|
└─────────────────────────────────────────────┘
|
|
```
|
|
|
|
### People Management Page
|
|
|
|
```
|
|
┌─────────────────────────────────────────────┐
|
|
│ People Management │
|
|
├─────────────────────────────────────────────┤
|
|
│ │
|
|
│ [+ Add New Person] │
|
|
│ │
|
|
│ ┌───────────────────────────────────────┐ │
|
|
│ │ 👤 John Doe │ │
|
|
│ │ Directory: john_doe/ │ │
|
|
│ │ Face Samples: 25 │ │
|
|
│ │ Images Sorted: 142 │ │
|
|
│ │ [Edit] [Train More] [Delete] │ │
|
|
│ └───────────────────────────────────────┘ │
|
|
│ │
|
|
│ ┌───────────────────────────────────────┐ │
|
|
│ │ 👤 Sarah Smith │ │
|
|
│ │ Directory: sarah_smith/ │ │
|
|
│ │ Face Samples: 18 │ │
|
|
│ │ Images Sorted: 89 │ │
|
|
│ │ [Edit] [Train More] [Delete] │ │
|
|
│ └───────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────┘
|
|
```
|
|
|
|
### Review Queue Page
|
|
|
|
```
|
|
┌─────────────────────────────────────────────┐
|
|
│ Review Queue (8 pending) │
|
|
├─────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌───────────────────────────────────────┐ │
|
|
│ │ [Image Thumbnail] │ │
|
|
│ │ │ │
|
|
│ │ Confidence: 45% (Low) │ │
|
|
│ │ Suggested: John Doe │ │
|
|
│ │ │ │
|
|
│ │ This is: [Select Person ▼] │ │
|
|
│ │ │ │
|
|
│ │ [✓ Confirm] [Skip] [New Person] │ │
|
|
│ └───────────────────────────────────────┘ │
|
|
│ │
|
|
│ [◄ Previous] [Next ►] │
|
|
└─────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## 🔒 Privacy & Security
|
|
|
|
1. **Local Processing Only** - No cloud APIs, all processing local
|
|
2. **Encrypted Storage** - Face encodings stored securely
|
|
3. **User Control** - Easy enable/disable, delete data anytime
|
|
4. **Access Control** - Face recognition UI requires authentication
|
|
5. **Audit Trail** - All auto-sort actions logged with confidence scores
|
|
|
|
---
|
|
|
|
## ⚡ Performance Considerations
|
|
|
|
### Processing Speed
|
|
- Face detection: ~0.5-1 sec per image
|
|
- Face recognition: ~0.1 sec per comparison
|
|
- Total per image: 1-3 seconds
|
|
|
|
### Optimization Strategies
|
|
1. **Async Processing** - Process in background, don't block downloads
|
|
2. **Batch Processing** - Process multiple images in parallel
|
|
3. **Caching** - Keep known face encodings in memory
|
|
4. **Smart Queueing** - Process high-priority images first
|
|
5. **CPU vs GPU** - Optional GPU acceleration for faster processing
|
|
|
|
---
|
|
|
|
## 📝 Configuration Example
|
|
|
|
```json
|
|
{
|
|
"face_recognition": {
|
|
"enabled": true,
|
|
"confidence_threshold": 0.6,
|
|
"auto_sort_threshold": 0.5,
|
|
"base_directory": "/mnt/storage/Downloads/faces",
|
|
"review_queue_dir": "/mnt/storage/Downloads/faces/review_queue",
|
|
"process_existing": false,
|
|
"async_processing": true,
|
|
"batch_size": 10,
|
|
"max_faces_per_image": 5,
|
|
"create_thumbnails": true,
|
|
"notify_on_queue": true,
|
|
"gpu_acceleration": false
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 🧪 Testing Plan
|
|
|
|
### Unit Tests
|
|
- Face detection accuracy
|
|
- Face matching accuracy
|
|
- Database operations
|
|
- File operations
|
|
|
|
### Integration Tests
|
|
- End-to-end download → face recognition → sort
|
|
- Review queue workflow
|
|
- Training new people
|
|
|
|
### Performance Tests
|
|
- Processing speed benchmarks
|
|
- Memory usage monitoring
|
|
- Concurrent processing
|
|
|
|
---
|
|
|
|
## 📈 Success Metrics
|
|
|
|
- **Accuracy**: >90% correct auto-sort rate
|
|
- **Performance**: <3 seconds per image processing
|
|
- **Usability**: <5 minutes to add and train new person
|
|
- **Review Queue**: <10% of images requiring manual review
|
|
- **Stability**: No crashes or errors during processing
|
|
|
|
---
|
|
|
|
## 🚀 Getting Started (Once Implemented)
|
|
|
|
### 1. Enable Face Recognition
|
|
```bash
|
|
# Install dependencies
|
|
pip3 install face_recognition pillow
|
|
|
|
# Enable in config
|
|
# Set "face_recognition.enabled": true
|
|
```
|
|
|
|
### 2. Add Your First Person
|
|
```python
|
|
# Via Web UI or CLI
|
|
# 1. Create person
|
|
# 2. Upload 5-10 sample images
|
|
# 3. Train face recognition
|
|
```
|
|
|
|
### 3. Process Images
|
|
```bash
|
|
# Automatic: New downloads are processed automatically
|
|
# Manual: Process existing downloads
|
|
curl -X POST http://localhost:8000/api/face-recognition/process-existing
|
|
```
|
|
|
|
### 4. Review Unknown Faces
|
|
- Open Review Queue in web UI
|
|
- Identify unknown faces
|
|
- System learns from your identifications
|
|
|
|
---
|
|
|
|
## 🔮 Future Enhancements
|
|
|
|
### v2 Features
|
|
- **Multiple face handling** - Split images with multiple people
|
|
- **Age progression** - Recognize people across different ages
|
|
- **Group detection** - Automatically create "group" folders
|
|
- **Emotion detection** - Filter by happy/sad expressions
|
|
- **Quality scoring** - Auto-select best photos of each person
|
|
- **Duplicate detection** - Find similar poses/angles
|
|
|
|
### v3 Features
|
|
- **Video support** - Extract faces from videos
|
|
- **Live camera** - Real-time face recognition
|
|
- **Object detection** - Sort by objects/scenes too
|
|
- **Tag suggestions** - AI-powered photo tagging
|
|
- **Smart albums** - Auto-generate albums by person/event
|
|
|
|
---
|
|
|
|
## 📚 Resources
|
|
|
|
### Libraries
|
|
- [face_recognition](https://github.com/ageitgey/face_recognition) - Main library
|
|
- [dlib](http://dlib.net/) - Face detection engine
|
|
- [OpenCV](https://opencv.org/) - Image processing
|
|
|
|
### Documentation
|
|
- [Face Recognition Tutorial](https://www.pyimagesearch.com/2018/06/18/face-recognition-with-opencv-python-and-deep-learning/)
|
|
- [DeepFace GitHub](https://github.com/serengil/deepface)
|
|
|
|
---
|
|
|
|
**Status**: Ready for implementation
|
|
**Next Step**: Phase 1 - Install dependencies and build core foundation
|
|
**Questions**: See [IMPLEMENTATION_GUIDE.md] for step-by-step instructions
|
|
|
|
---
|
|
|
|
**Last Updated**: 2025-10-31
|