166 lines
4.8 KiB
Markdown
166 lines
4.8 KiB
Markdown
# Snapchat Downloader Module
|
|
|
|
This module downloads Snapchat stories using the StoryClon e proxy (https://s.storyclone.com).
|
|
|
|
## Features
|
|
|
|
- Downloads Snapchat stories via StoryClon e proxy (s.storyclone.com/u/<user>/)
|
|
- FastDL-compatible file naming: `{username}_{YYYYMMDD_HHMMSS}_{media_id}.{ext}`
|
|
- Integrated with unified database for tracking downloads
|
|
- Subprocess isolation to avoid event loop conflicts
|
|
- Browser automation with Playwright
|
|
- Cloudflare bypass support with 2captcha (optional)
|
|
- Cookie persistence for faster subsequent runs
|
|
- Automatic file organization and moving to destination
|
|
|
|
## Setup
|
|
|
|
### 1. Add Configuration
|
|
|
|
Add the following to your `config/settings.json`:
|
|
|
|
```json
|
|
{
|
|
"snapchat": {
|
|
"enabled": true,
|
|
"check_interval_hours": 6,
|
|
"twocaptcha_api_key": "",
|
|
"cookie_file": "/opt/media-downloader/cookies/snapchat_cookies.json",
|
|
"usernames": [
|
|
"username1",
|
|
"username2"
|
|
],
|
|
"stories": {
|
|
"enabled": true,
|
|
"days_back": 7,
|
|
"max_downloads": 50,
|
|
"temp_dir": "temp/snapchat/stories",
|
|
"destination_path": "/path/to/your/media/library/Snapchat"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### 2. Configure Settings
|
|
|
|
- **enabled**: Set to `true` to enable Snapchat downloads
|
|
- **check_interval_hours**: How often to check for new content (used by scheduler)
|
|
- **twocaptcha_api_key**: Optional - API key for 2captcha.com to solve Cloudflare challenges
|
|
- **cookie_file**: Path to store cookies for faster subsequent runs
|
|
- **usernames**: List of Snapchat usernames to download from
|
|
- **stories.enabled**: Enable/disable story downloads
|
|
- **stories.days_back**: How many days back to search for stories
|
|
- **stories.max_downloads**: Maximum number of stories to download per run
|
|
- **stories.temp_dir**: Temporary download directory
|
|
- **stories.destination_path**: Final destination for downloaded files
|
|
|
|
### 3. Set Download Settings
|
|
|
|
Make sure you have the download settings configured in `settings.json`:
|
|
|
|
```json
|
|
{
|
|
"download_settings": {
|
|
"move_to_destination": true
|
|
}
|
|
}
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Download from all platforms (including Snapchat):
|
|
```bash
|
|
cd /opt/media-downloader
|
|
./venv/bin/python media-downloader.py --platform all
|
|
```
|
|
|
|
### Download only from Snapchat:
|
|
```bash
|
|
cd /opt/media-downloader
|
|
./venv/bin/python media-downloader.py --platform snapchat
|
|
```
|
|
|
|
### Run with Xvfb (headless display):
|
|
```bash
|
|
./run-with-xvfb.sh
|
|
```
|
|
|
|
## File Naming
|
|
|
|
Files are saved using FastDL-compatible naming format:
|
|
- Format: `{username}_{YYYYMMDD_HHMMSS}_{media_id}.{ext}`
|
|
- Example: `johndoe_20250101_143022_abc123def456.jpg`
|
|
|
|
This ensures:
|
|
- Chronological sorting by file name
|
|
- Easy identification of source user
|
|
- Unique media IDs prevent duplicates
|
|
|
|
## Database Tracking
|
|
|
|
The module uses the unified database to track downloaded stories:
|
|
- Platform: `snapchat`
|
|
- Records URL, filename, post date, and metadata
|
|
- Prevents re-downloading the same content
|
|
- Supports database queries for download history
|
|
|
|
## How It Works
|
|
|
|
1. **Browser Automation**: Uses Playwright (Chromium) to navigate StoryClon e
|
|
2. **Story Detection**: Finds story media elements on the page
|
|
3. **Download**: Downloads images/videos via direct URL requests
|
|
4. **File Processing**: Saves with FastDL naming, updates timestamps
|
|
5. **Database Recording**: Marks downloads in unified database
|
|
6. **File Moving**: Moves files to destination if configured
|
|
7. **Cleanup**: Removes temporary files after successful processing
|
|
|
|
## Limitations
|
|
|
|
- Only downloads stories (no direct posts/snaps)
|
|
- Relies on StoryClon e proxy availability
|
|
- Stories may expire after 24 hours (download frequently)
|
|
- Cloudflare protection may require 2captcha API key
|
|
|
|
## Troubleshooting
|
|
|
|
### No stories found
|
|
- Check if the username is correct
|
|
- Verify the user has active stories on StoryClon e
|
|
- Try accessing https://s.storyclone.com/u/{username}/ manually
|
|
|
|
### Cloudflare blocking
|
|
- Add your 2captcha API key to config
|
|
- Ensure cookies are being saved and loaded
|
|
- Try running with headed mode to see the challenge
|
|
|
|
### Downloads not showing in database
|
|
- Check database path in config
|
|
- Verify unified_database module is working
|
|
- Check logs for database errors
|
|
|
|
## Testing
|
|
|
|
Test the module directly:
|
|
```bash
|
|
cd /opt/media-downloader
|
|
./venv/bin/python modules/snapchat_module.py username_to_test
|
|
```
|
|
|
|
This will download stories for the specified user and show detailed output.
|
|
|
|
## Architecture
|
|
|
|
- **snapchat_module.py**: Main downloader class with browser automation
|
|
- **snapchat_subprocess_wrapper.py**: Subprocess wrapper for isolation
|
|
- **SnapchatDatabaseAdapter**: Database adapter in unified_database.py
|
|
- **Integration**: Fully integrated into media-downloader.py
|
|
|
|
## Future Enhancements
|
|
|
|
Possible future improvements:
|
|
- Support for additional Snapchat proxy services
|
|
- Parallel download of multiple users
|
|
- Story metadata extraction (captions, timestamps)
|
|
- Automatic quality detection
|
|
- Retry logic for failed downloads
|