Todd
412
docs/VERSIONING.md
Normal file
412
docs/VERSIONING.md
Normal file
@@ -0,0 +1,412 @@
|
||||
# Media Downloader Versioning & Backup Guide
|
||||
|
||||
## Version Management
|
||||
|
||||
### Current Version
|
||||
The current version is stored in `/opt/media-downloader/VERSION`:
|
||||
```
|
||||
12.12.1
|
||||
```
|
||||
|
||||
### Versioning Scheme
|
||||
This project follows [Semantic Versioning](https://semver.org/) (SemVer):
|
||||
|
||||
**MAJOR.MINOR.PATCH** (e.g., 6.0.0)
|
||||
|
||||
- **MAJOR**: Incompatible API changes, major feature overhauls
|
||||
- **MINOR**: New features, backward-compatible changes
|
||||
- **PATCH**: Bug fixes, security patches, backward-compatible fixes
|
||||
|
||||
### Version History
|
||||
|
||||
See [CHANGELOG.md](../CHANGELOG.md) for complete version history.
|
||||
|
||||
**Recent Versions:**
|
||||
- **v6.0.0** (2025-10-26) - Database CLI, ImgInn fixes, installer updates, version control
|
||||
- **v5.0.0** (2025-10-25) - File hash deduplication, directory reorganization, documentation
|
||||
- **v4.x** - Multi-platform support, scheduler, Immich integration
|
||||
|
||||
---
|
||||
|
||||
## Backup System Integration
|
||||
|
||||
### Backup Central Integration
|
||||
|
||||
Media Downloader is integrated with Backup Central for automated backups.
|
||||
|
||||
**Profile ID:** `profile-media-downloader`
|
||||
**Schedule:** Daily at 00:00 (midnight)
|
||||
**Destination:** `/media/backups/Ubuntu/restic-repo` (shared restic repository)
|
||||
|
||||
### Re-adding the Backup Profile
|
||||
|
||||
If you need to recreate the backup profile, run:
|
||||
|
||||
```bash
|
||||
cd /opt/media-downloader
|
||||
sudo ./scripts/add-backup-profile.sh
|
||||
```
|
||||
|
||||
This script will:
|
||||
1. Remove existing profile (if present)
|
||||
2. Create new profile with correct settings
|
||||
3. Restart Backup Central service
|
||||
4. Verify profile was created
|
||||
|
||||
### What Gets Backed Up
|
||||
|
||||
**Included:**
|
||||
- `/opt/media-downloader/config/` - All configuration files
|
||||
- `/opt/media-downloader/database/` - SQLite databases (main + scheduler)
|
||||
- `/opt/media-downloader/cookies/` - Authentication cookies
|
||||
- `/opt/media-downloader/sessions/` - Instagram session files
|
||||
- `/opt/media-downloader/modules/` - All Python modules
|
||||
- `/opt/media-downloader/wrappers/` - Subprocess wrappers
|
||||
- `/opt/media-downloader/utilities/` - Utility scripts
|
||||
- `/opt/media-downloader/scripts/` - Backup and install scripts
|
||||
- `/opt/media-downloader/*.py` - Main application files
|
||||
- `/opt/media-downloader/VERSION` - Version file
|
||||
- `/opt/media-downloader/CHANGELOG.md` - Change log
|
||||
- `/opt/media-downloader/README.md` - Documentation
|
||||
- `/opt/media-downloader/INSTALL.md` - Installation guide
|
||||
- `/opt/media-downloader/requirements.txt` - Dependencies
|
||||
- `/opt/media-downloader/db` - Database CLI wrapper
|
||||
|
||||
**Excluded:**
|
||||
- `/opt/media-downloader/temp/` - Temporary downloads
|
||||
- `/opt/media-downloader/logs/` - Log files
|
||||
- `/opt/media-downloader/venv/` - Virtual environment (reproducible)
|
||||
- `/opt/media-downloader/.playwright/` - Playwright cache (reproducible)
|
||||
- `/opt/media-downloader/debug/` - Debug files
|
||||
- `*.log`, `*.log.*` - All log files
|
||||
- `*.pyc`, `__pycache__` - Python bytecode
|
||||
- `*.db-shm`, `*.db-wal` - SQLite temporary files
|
||||
- Swap files: `*.swp`, `*.swo`, `*~`
|
||||
|
||||
### Retention Policy
|
||||
|
||||
- **Daily:** 7 days
|
||||
- **Weekly:** 4 weeks
|
||||
- **Monthly:** 12 months
|
||||
- **Yearly:** 2 years
|
||||
|
||||
### Notifications
|
||||
|
||||
- **Success:** Disabled (runs daily, would spam)
|
||||
- **Warning:** Enabled (Pushover)
|
||||
- **Failure:** Enabled (Pushover)
|
||||
|
||||
---
|
||||
|
||||
## Creating Version Backups
|
||||
|
||||
### Manual Version Backup
|
||||
|
||||
To create a version-stamped locked backup:
|
||||
|
||||
```bash
|
||||
cd /opt/media-downloader
|
||||
./scripts/create-version-backup.sh
|
||||
```
|
||||
|
||||
This will:
|
||||
1. Read version from `VERSION` file
|
||||
2. Create timestamp
|
||||
3. Generate backup name: `{version}-{timestamp}`
|
||||
4. Run backup using Backup Central
|
||||
5. Lock the backup (prevent deletion)
|
||||
|
||||
**Example backup name:**
|
||||
```
|
||||
6.0.0-20251026-143000
|
||||
```
|
||||
|
||||
This matches backup-central's naming convention: `{version}-{YYYYMMDD-HHMMSS}`
|
||||
|
||||
### When to Create Version Backups
|
||||
|
||||
Create manual version backups:
|
||||
- **Before releasing a new version** - Capture stable state
|
||||
- **After major changes** - Database schema, config structure
|
||||
- **Before risky operations** - Large refactors, dependency updates
|
||||
- **Milestone achievements** - Feature completions, bug fixes
|
||||
|
||||
### Scheduled Backups
|
||||
|
||||
Daily backups run automatically:
|
||||
- **Time:** 00:00 (midnight)
|
||||
- **Managed by:** Backup Central scheduler
|
||||
- **Type:** Incremental (restic)
|
||||
- **Not locked** - Subject to retention policy
|
||||
|
||||
---
|
||||
|
||||
## Backup Management
|
||||
|
||||
### List All Backups
|
||||
|
||||
```bash
|
||||
backup-central list -P profile-media-downloader
|
||||
```
|
||||
|
||||
### View Profile Details
|
||||
|
||||
```bash
|
||||
backup-central profiles --info profile-media-downloader
|
||||
```
|
||||
|
||||
### Manual Backup
|
||||
|
||||
```bash
|
||||
backup-central backup -P profile-media-downloader
|
||||
```
|
||||
|
||||
### Create Custom Named Backup
|
||||
|
||||
```bash
|
||||
backup-central backup -P profile-media-downloader -n "before-upgrade" -l
|
||||
```
|
||||
|
||||
### Restore from Backup
|
||||
|
||||
```bash
|
||||
# List snapshots
|
||||
backup-central list -P profile-media-downloader
|
||||
|
||||
# Restore specific snapshot
|
||||
backup-central restore <snapshot-id> -P profile-media-downloader -t /opt/media-downloader-restore
|
||||
```
|
||||
|
||||
### Lock/Unlock Backups
|
||||
|
||||
```bash
|
||||
# Lock important backups (prevent deletion)
|
||||
backup-central lock <backup-id>
|
||||
|
||||
# Unlock backups
|
||||
backup-central unlock <backup-id>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Version Release Process
|
||||
|
||||
### 1. Update Code & Test
|
||||
- Make changes
|
||||
- Test thoroughly
|
||||
- Verify all platforms work
|
||||
- Check database operations
|
||||
|
||||
### 2. Update Version
|
||||
```bash
|
||||
# Edit VERSION file
|
||||
echo "6.1.0" > /opt/media-downloader/VERSION
|
||||
```
|
||||
|
||||
### 3. Update CHANGELOG
|
||||
- Document all changes in `CHANGELOG.md`
|
||||
- Follow existing format
|
||||
- Include:
|
||||
- New features
|
||||
- Bug fixes
|
||||
- Breaking changes
|
||||
- Upgrade notes
|
||||
|
||||
### 4. Create Version Backup
|
||||
```bash
|
||||
./scripts/create-version-backup.sh
|
||||
```
|
||||
|
||||
### 5. Tag & Commit (if using git)
|
||||
```bash
|
||||
git add VERSION CHANGELOG.md
|
||||
git commit -m "Release v6.1.0"
|
||||
git tag -a v6.1.0 -m "Version 6.1.0 release"
|
||||
git push && git push --tags
|
||||
```
|
||||
|
||||
### 6. Verify Backup
|
||||
```bash
|
||||
backup-central list -P profile-media-downloader --limit 5
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Disaster Recovery
|
||||
|
||||
### Full System Restore
|
||||
|
||||
1. **Install base system**
|
||||
```bash
|
||||
sudo mkdir -p /opt/media-downloader
|
||||
```
|
||||
|
||||
2. **Restore from backup**
|
||||
```bash
|
||||
backup-central restore <snapshot-id> -P profile-media-downloader -t /opt
|
||||
```
|
||||
|
||||
3. **Reinstall dependencies**
|
||||
```bash
|
||||
cd /opt/media-downloader
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
playwright install chromium
|
||||
```
|
||||
|
||||
4. **Set permissions**
|
||||
```bash
|
||||
sudo chown -R $USER:$USER /opt/media-downloader
|
||||
chmod +x /opt/media-downloader/media-downloader.py
|
||||
chmod +x /opt/media-downloader/db
|
||||
chmod +x /opt/media-downloader/scripts/*.sh
|
||||
```
|
||||
|
||||
5. **Verify**
|
||||
```bash
|
||||
/opt/media-downloader/media-downloader.py --version
|
||||
./db stats
|
||||
```
|
||||
|
||||
### Partial Restore (Config Only)
|
||||
|
||||
```bash
|
||||
# Restore just config directory
|
||||
backup-central restore <snapshot-id> \
|
||||
-P profile-media-downloader \
|
||||
-i "/opt/media-downloader/config" \
|
||||
-t /tmp/restore
|
||||
|
||||
# Copy to production
|
||||
sudo cp -r /tmp/restore/opt/media-downloader/config/* /opt/media-downloader/config/
|
||||
```
|
||||
|
||||
### Database Restore
|
||||
|
||||
```bash
|
||||
# Restore just database
|
||||
backup-central restore <snapshot-id> \
|
||||
-P profile-media-downloader \
|
||||
-i "/opt/media-downloader/database" \
|
||||
-t /tmp/restore
|
||||
|
||||
# Stop scheduler
|
||||
sudo systemctl stop media-downloader
|
||||
|
||||
# Replace database
|
||||
sudo cp /tmp/restore/opt/media-downloader/database/*.db /opt/media-downloader/database/
|
||||
|
||||
# Restart
|
||||
sudo systemctl start media-downloader
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Backup Verification
|
||||
|
||||
### Verify Backup Integrity
|
||||
|
||||
```bash
|
||||
# Check backup profile health
|
||||
backup-central health
|
||||
|
||||
# Verify specific profile
|
||||
backup-central profiles --stats profile-media-downloader
|
||||
```
|
||||
|
||||
### Test Restore
|
||||
|
||||
Periodically test restores to ensure backups are usable:
|
||||
|
||||
```bash
|
||||
# 1. Create test restore directory
|
||||
mkdir -p /tmp/media-downloader-test-restore
|
||||
|
||||
# 2. Restore to test location
|
||||
backup-central restore latest -P profile-media-downloader -t /tmp/media-downloader-test-restore
|
||||
|
||||
# 3. Verify critical files exist
|
||||
ls -la /tmp/media-downloader-test-restore/opt/media-downloader/config/
|
||||
ls -la /tmp/media-downloader-test-restore/opt/media-downloader/database/
|
||||
|
||||
# 4. Check database integrity
|
||||
sqlite3 /tmp/media-downloader-test-restore/opt/media-downloader/database/media_downloader.db "PRAGMA integrity_check;"
|
||||
|
||||
# 5. Clean up
|
||||
rm -rf /tmp/media-downloader-test-restore
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Backup Fails
|
||||
|
||||
**Check destination:**
|
||||
```bash
|
||||
ls -la /media/backups/Ubuntu/restic-repo
|
||||
```
|
||||
|
||||
**Check logs:**
|
||||
```bash
|
||||
backup-central list -P profile-media-downloader
|
||||
sudo journalctl -u backup-central -f
|
||||
```
|
||||
|
||||
**Manual test:**
|
||||
```bash
|
||||
backup-central backup -P profile-media-downloader --dry-run
|
||||
```
|
||||
|
||||
### Version Script Fails
|
||||
|
||||
**Check VERSION file:**
|
||||
```bash
|
||||
cat /opt/media-downloader/VERSION
|
||||
```
|
||||
|
||||
**Verify profile exists:**
|
||||
```bash
|
||||
backup-central profiles list | grep media-downloader
|
||||
```
|
||||
|
||||
**Test backup manually:**
|
||||
```bash
|
||||
backup-central backup -P profile-media-downloader -n "test-backup"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Version Management
|
||||
- ✅ Update VERSION file before creating version backup
|
||||
- ✅ Always update CHANGELOG.md with version
|
||||
- ✅ Use semantic versioning (MAJOR.MINOR.PATCH)
|
||||
- ✅ Lock important release backups
|
||||
- ✅ Tag releases in git (if using version control)
|
||||
|
||||
### Backup Strategy
|
||||
- ✅ Create version backup before major changes
|
||||
- ✅ Test restores quarterly
|
||||
- ✅ Verify backup notifications work
|
||||
- ✅ Monitor backup sizes (check for bloat)
|
||||
- ✅ Keep locked backups for major versions
|
||||
- ✅ Document any custom backup procedures
|
||||
|
||||
### Security
|
||||
- ✅ Backups include credentials (cookies, sessions, config)
|
||||
- ✅ Ensure backup destination is secure
|
||||
- ✅ Restrict access to backup restoration
|
||||
- ✅ Consider encryption for sensitive data
|
||||
- ✅ Don't commit credentials to git
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- [CHANGELOG.md](../CHANGELOG.md) - Full version history
|
||||
- [README.md](../README.md) - Main documentation
|
||||
- [INSTALL.md](../INSTALL.md) - Installation guide
|
||||
- [Backup Central Documentation](https://bu.lic.ad/docs)
|
||||
Reference in New Issue
Block a user