media-downloader/docs/archive/SNAPCHAT_IMPLEMENTATION_SUMMARY.md

Snapchat Downloader Implementation Summary

Overview

Successfully implemented a complete Snapchat downloader module for the media-downloader system, based on the ImgInn module architecture. The module downloads Snapchat stories via the StoryClon e proxy (https://s.storyclone.com/u//).

Files Created

1. Core Module

File: /opt/media-downloader/modules/snapchat_module.py

• Main SnapchatDownloader class
• Browser automation with Playwright
• FastDL-compatible file naming
• Cookie management
• Cloudflare challenge handling
• Database integration
• Timestamp updating (file system + EXIF)
• Story extraction and downloading

2. Subprocess Wrapper

File: /opt/media-downloader/snapchat_subprocess_wrapper.py

• Isolates Snapchat operations in separate process
• Avoids asyncio event loop conflicts
• JSON-based configuration input/output
• Stderr logging for clean stdout

3. Database Adapter

File: /opt/media-downloader/modules/unified_database.py (modified)

• Added SnapchatDatabaseAdapter class
• Tracks downloads by URL and metadata
• Platform: 'snapchat'
• Content type: 'story'

4. Main Integration

File: /opt/media-downloader/media-downloader.py (modified)

• Imported SnapchatDownloader module
• Added initialization in _init_modules()
• Added interval configuration (check_interval_hours)
• Created _download_snapchat_content() method
• Created download_snapchat() method
• Integrated into run() method (download all platforms)
• Added command-line argument support: --platform snapchat
• Added scheduler filtering support

5. Configuration Example

File: /opt/media-downloader/config/snapchat_example.json

• Sample configuration structure
• All available settings documented
• Ready to copy into main settings.json

6. Documentation

File: /opt/media-downloader/SNAPCHAT_README.md

• Complete usage guide
• Setup instructions
• Configuration options explained
• Troubleshooting section
• Architecture overview

Key Features Implemented

✅ Complete Feature Set

1. Browser Automation: Playwright-based Chromium automation
2. Proxy Support: Uses StoryClon e (s.storyclone.com) proxy
3. Story Downloads: Extracts and downloads all available stories
4. FastDL Naming: Compatible filename format (user_date_mediaid.ext)
5. Database Tracking: Full integration with unified database
6. Duplicate Prevention: Checks database before downloading
7. Timestamp Accuracy: Updates file system and EXIF timestamps
8. Cookie Persistence: Saves/loads cookies for faster runs
9. Cloudflare Bypass: Optional 2captcha integration
10. File Organization: Automatic moving to destination
11. Subprocess Isolation: Prevents event loop conflicts
12. Logging: Comprehensive logging with callback support
13. Error Handling: Robust error handling and recovery
14. Scheduler Integration: Supports scheduled downloads
15. Batch Processing: Supports multiple users

✅ Architecture Alignment

• Follows ImgInn module pattern exactly
• Uses same subprocess wrapper approach
• Integrates with move_module for file management
• Uses unified_database for tracking
• Compatible with scheduler system
• Supports Pushover notifications via move_module
• Works with Immich scanning

Configuration Structure

{
  "snapchat": {
    "enabled": true,
    "check_interval_hours": 6,
    "twocaptcha_api_key": "",
    "cookie_file": "/opt/media-downloader/cookies/snapchat_cookies.json",
    "usernames": ["user1", "user2"],
    "stories": {
      "enabled": true,
      "days_back": 7,
      "max_downloads": 50,
      "temp_dir": "temp/snapchat/stories",
      "destination_path": "/path/to/media/library/Snapchat"
    }
  }
}

Usage Examples

Download from all platforms (includes Snapchat):

cd /opt/media-downloader
./venv/bin/python media-downloader.py --platform all

Download only Snapchat:

./venv/bin/python media-downloader.py --platform snapchat

Run with scheduler:

./venv/bin/python media-downloader.py --scheduler

Test standalone module:

./venv/bin/python modules/snapchat_module.py username_to_test

Integration Points

Modified Files

1. media-downloader.py:

   • Line 47: Import SnapchatDownloader
   • Line 423-436: Module initialization
   • Line 511-513: Interval configuration
   • Line 1187-1325: Download methods
   • Line 1959-1962: Integration in run()
   • Line 1998: Command-line choices
   • Line 2179-2181, 2283-2285: Scheduler filtering
   • Line 2511-2512: Command-line handler

2. unified_database.py:

   • Line 1300-1325: SnapchatDatabaseAdapter class

File Naming Convention

Format: {username}_{YYYYMMDD_HHMMSS}_{media_id}.{ext}

Example: johndoe_20250123_143022_abc123def456789.jpg

Components:

• username: Snapchat username (lowercase)
• YYYYMMDD: Date the story was posted (or current date)
• HHMMSS: Time the story was posted (or current time)
• media_id: Unique identifier from the media URL
• ext: File extension (.jpg, .mp4, etc.)

Database Schema

Stories are recorded in the unified database:

• platform: 'snapchat'
• source: username
• content_type: 'story'
• url: Original media URL
• filename: Final filename
• post_date: Story date/time
• metadata: JSON with media_id and other info

Testing Checklist

Before First Run:

• Add configuration to settings.json
• Set enabled: true
• Add at least one username
• Set destination_path
• Configure download_settings.move_to_destination: true
• Ensure Xvfb is running (./run-with-xvfb.sh)

Test Execution:

• Test standalone module: ./venv/bin/python modules/snapchat_module.py username
• Test via main script: ./venv/bin/python media-downloader.py --platform snapchat
• Verify files downloaded to temp directory
• Verify files moved to destination
• Check database has records
• Verify no duplicate downloads on re-run
• Check logs for errors

Known Limitations

1. StoryClon e Dependency: Relies on s.storyclone.com being available
2. Stories Only: Only downloads stories, not direct posts/snaps
3. 24-Hour Expiry: Stories expire after 24 hours on Snapchat
4. Cloudflare: May require 2captcha API key for Cloudflare challenges
5. Date Accuracy: Story dates may not always be accurate (uses current date if unavailable)

Future Enhancements

Potential improvements:

1. Support additional Snapchat proxy services
2. Parallel processing of multiple users
3. Story caption/metadata extraction
4. Automatic retry on failures
5. Quality selection (if available)
6. Video thumbnail generation
7. Story highlights download

Comparison with ImgInn Module

Feature	ImgInn	Snapchat	Status
Posts	✅	❌	N/A for Snapchat
Stories	✅	✅	✅ Implemented
Browser Automation	✅	✅	✅ Implemented
Subprocess Isolation	✅	✅	✅ Implemented
Database Tracking	✅	✅	✅ Implemented
Cookie Persistence	✅	✅	✅ Implemented
2captcha Support	✅	✅	✅ Implemented
Phrase Search	✅	❌	N/A for stories
FastDL Naming	✅	✅	✅ Implemented
Timestamp Updates	✅	✅	✅ Implemented

Success Criteria

✅ All criteria met:

1. ✅ Module follows ImgInn architecture pattern
2. ✅ Uses StoryClon e proxy (s.storyclone.com/u//)
3. ✅ Downloads Snapchat stories
4. ✅ FastDL-compatible file naming
5. ✅ Integrated with unified database
6. ✅ Subprocess isolation implemented
7. ✅ Command-line support added
8. ✅ Scheduler integration complete
9. ✅ Configuration example created
10. ✅ Documentation written

Next Steps for User

1. Configure: Add Snapchat config to settings.json
2. Enable: Set snapchat.enabled: true
3. Add Users: Add Snapchat usernames to download from
4. Test: Run ./venv/bin/python media-downloader.py --platform snapchat
5. Schedule: Enable scheduler for automatic downloads
6. Monitor: Check logs and database for successful downloads

Support

For issues or questions:

1. Check SNAPCHAT_README.md for troubleshooting
2. Review logs in /opt/media-downloader/logs/
3. Test standalone module for detailed output
4. Check database entries: sqlite3 database/media_downloader.db "SELECT * FROM downloads WHERE platform='snapchat';"

---

Implementation Date: 2025-10-23 Based On: ImgInn module architecture Status: ✅ Complete and ready for testing