18 KiB
18 KiB
Web GUI Development - Quick Start Guide
What We're Building
Transform your CLI media downloader into a professional web application with:
✅ Real-time monitoring - Watch downloads happen live ✅ Visual queue management - Drag, drop, prioritize ✅ Live browser screenshots - See what scrapers are doing ✅ Automated scheduling - Set it and forget it ✅ Beautiful dashboard - Stats, charts, analytics ✅ Mobile responsive - Works on phone/tablet/desktop
Technology Stack Summary
┌─────────────────────────────────────────┐
│ Vue.js 3 + Vuetify (Frontend) │
│ Modern, beautiful Material Design UI │
└─────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ FastAPI (Backend API) │
│ Fast, async, auto-documented │
└─────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ Celery + Redis (Background Jobs) │
│ Existing modules run as workers │
└─────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ SQLite (Database - existing) │
│ Already have this, minimal changes │
└─────────────────────────────────────────┘
Key Point: Your existing downloader modules (fastdl_module.py, toolzu_module.py, etc.) are reused as-is. They become Celery workers instead of CLI commands.
What It Will Look Like
Dashboard View
┌──────────────────────────────────────────────────────────────┐
│ Media Downloader [Queue] [Scheduler] [Settings] [Logs] │
├──────────────────────────────────────────────────────────────┤
│ │
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌──────────┐ │
│ │Downloads │ │Queue Size │ │Success Rate│ │Storage │ │
│ │ 45 │ │ 2,731 │ │ 99.2% │ │ 42.5 GB │ │
│ │ Today │ │ Pending │ │ This Week │ │ Used │ │
│ └────────────┘ └────────────┘ └────────────┘ └──────────┘ │
│ │
│ Recent Downloads [LIVE] Platform Status │
│ ┌──────────────────────────┐ ┌──────────────────────┐ │
│ │ ⬇️ evalongoria_post.jpg │ │ 🟢 Instagram (35) │ │
│ │ ⬇️ evalongoria_story.jpg │ │ 🟢 TikTok (2) │ │
│ │ ✅ mariarbravo_post.jpg │ │ 🟢 Forums (8) │ │
│ │ ⬇️ picturepub_img_1.jpg │ └──────────────────────┘ │
│ └──────────────────────────┘ │
│ │
│ Download Activity (Last 7 Days) │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ ▂▄▅▇█▇▅ │ │
│ │ │ │
│ └──────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
Queue Manager with Live Screenshots
┌──────────────────────────────────────────────────────────────┐
│ Download Queue [+ Add Download]│
├───────────────────────────┬──────────────────────────────────┤
│ Queue Items (2,731) │ Live Scraper View - Instagram │
│ │ [LIVE] 🔴 │
│ 🔵 Instagram @evalongoria │ ┌─────────────────────────────┐ │
│ Status: Downloading │ │ │ │
│ Progress: ████░░ 65% │ │ [Browser Screenshot] │ │
│ 13/20 posts │ │ Showing Instagram page │ │
│ │ │ being scraped right now │ │
│ ⏸️ TikTok @evalongoria │ │ │ │
│ Status: Paused │ └─────────────────────────────┘ │
│ Priority: High │ Action: Scrolling to load... │
│ │ Updated 2s ago │
│ ⏳ Forum - PicturePub │ │
│ Status: Pending │ [Pause] [Save Screenshot] │
│ Priority: Normal │ │
│ │ │
│ [Bulk Actions ▾] │ │
│ □ Clear Completed │ │
│ □ Retry Failed │ │
└───────────────────────────┴──────────────────────────────────┘
Scheduler View
┌──────────────────────────────────────────────────────────────┐
│ Scheduled Downloads [+ New Schedule] │
├──────────────────────────────────────────────────────────────┤
│ │
│ ✅ Eva Longoria Instagram Posts │
│ Every 4 hours • Next: in 1h 23m • Last: 8 items │
│ [Edit] [Run Now] [Pause] │
│ │
│ ✅ TikTok Videos Check │
│ Daily at 2:00 AM • Next: in 6h 15m • Last: 3 items │
│ [Edit] [Run Now] [Pause] │
│ │
│ ⏸️ Maria Ramos Instagram Stories │
│ Every 6 hours • Paused • Last: 15 items │
│ [Edit] [Run Now] [Resume] │
│ │
│ Execution History │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ 2025-10-13 12:00 Eva Longoria Posts ✅ 8 items │ │
│ │ 2025-10-13 08:00 Eva Longoria Posts ✅ 12 items │ │
│ │ 2025-10-13 04:00 Eva Longoria Posts ❌ Failed │ │
│ └──────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
Development Approach
Option 1: Full Build (10 weeks)
Build everything from scratch following the full plan.
Pros:
- Complete control
- Exactly what you want
- Learning experience
Cons:
- Time investment (10 weeks full-time or 20 weeks part-time)
- Need web development skills
Option 2: Incremental (Start Small)
Build Phase 1 first, then decide.
Week 1-2: Proof of Concept
- Basic login
- Dashboard showing database stats
- Download list (read-only)
Result: See if you like it before committing
Option 3: Hybrid (Recommended)
Keep CLI for manual use, add web GUI for monitoring only.
Week 1: Simple Dashboard
- Flask (simpler than FastAPI)
- Read-only view of database
- Live log viewer
- No authentication needed
Result: 80% of value with 20% of effort
Quick Implementation - Option 3 (Monitoring Only)
Here's a 1-week implementation for a simple monitoring dashboard:
Step 1: Install Dependencies
cd /opt/media-downloader
pip3 install flask flask-socketio simple-websocket
Step 2: Create Simple Backend
# web_dashboard.py
from flask import Flask, render_template, jsonify
from flask_socketio import SocketIO
from modules.unified_database import UnifiedDatabase
import sqlite3
app = Flask(__name__)
socketio = SocketIO(app)
db = UnifiedDatabase('database/media_downloader.db')
@app.route('/')
def index():
return render_template('dashboard.html')
@app.route('/api/stats')
def get_stats():
return jsonify({
'downloads_today': get_downloads_today(),
'queue_size': get_queue_size(),
'recent_downloads': get_recent_downloads(20)
})
@app.route('/api/queue')
def get_queue():
items = db.get_queue_items(status='pending', limit=100)
return jsonify(items)
if __name__ == '__main__':
socketio.run(app, host='0.0.0.0', port=8080)
Step 3: Create Simple HTML
<!-- templates/dashboard.html -->
<!DOCTYPE html>
<html>
<head>
<title>Media Downloader Dashboard</title>
<script src="https://cdn.jsdelivr.net/npm/vue@3"></script>
<link href="https://cdn.jsdelivr.net/npm/vuetify@3/dist/vuetify.min.css" rel="stylesheet">
</head>
<body>
<div id="app">
<v-app>
<v-main>
<v-container>
<h1>Media Downloader</h1>
<!-- Stats -->
<v-row>
<v-col cols="3">
<v-card>
<v-card-text>
<div class="text-h4">{{ stats.downloads_today }}</div>
<div>Downloads Today</div>
</v-card-text>
</v-card>
</v-col>
<!-- More stats cards -->
</v-row>
<!-- Recent Downloads -->
<v-list>
<v-list-item v-for="download in recent" :key="download.id">
{{ download.filename }}
</v-list-item>
</v-list>
</v-container>
</v-main>
</v-app>
</div>
<script src="https://cdn.jsdelivr.net/npm/vuetify@3/dist/vuetify.min.js"></script>
<script>
const { createApp } = Vue;
const { createVuetify } = Vuetify;
const app = createApp({
data() {
return {
stats: {},
recent: []
}
},
mounted() {
this.loadStats();
setInterval(this.loadStats, 5000); // Refresh every 5s
},
methods: {
async loadStats() {
const response = await fetch('/api/stats');
const data = await response.json();
this.stats = data;
this.recent = data.recent_downloads;
}
}
});
const vuetify = createVuetify();
app.use(vuetify);
app.mount('#app');
</script>
</body>
</html>
Step 4: Run It
python3 web_dashboard.py
# Visit: http://localhost:8080
Result: Working dashboard in ~1 day!
Full Implementation Path
If you want the complete professional version:
Phase 1: Foundation (Week 1-2)
# Backend setup
cd /opt/media-downloader
mkdir -p backend/{api,models,services,workers,core}
pip3 install fastapi uvicorn celery redis pydantic
# Frontend setup
cd /opt/media-downloader
npm create vite@latest frontend -- --template vue
cd frontend
npm install vuetify axios pinia vue-router
Deliverable: Login + basic download list
Phase 2: Core (Week 3-4)
- Build queue manager
- Integrate Celery workers
- Add WebSocket for real-time
Deliverable: Functional queue management
Phase 3: Scheduler (Week 5-6)
- Build scheduler UI
- Settings pages
- Platform configs
Deliverable: Complete automation
Phase 4: Advanced (Week 7-8)
- History browser
- Log viewer
- Live screenshots
- Analytics
Deliverable: Full-featured app
Phase 5: Polish (Week 9-10)
- Testing
- Docker setup
- Documentation
- Deploy
Deliverable: Production ready
File Structure After Implementation
/opt/media-downloader/
├── backend/ # New FastAPI backend
│ ├── api/
│ ├── models/
│ ├── services/
│ └── workers/
├── frontend/ # New Vue.js frontend
│ ├── src/
│ │ ├── views/
│ │ ├── components/
│ │ └── stores/
│ └── package.json
├── modules/ # Existing (kept as-is)
│ ├── fastdl_module.py
│ ├── toolzu_module.py
│ ├── tiktok_module.py
│ └── unified_database.py
├── database/ # Existing (kept as-is)
│ └── media_downloader.db
├── downloads/ # Existing (kept as-is)
├── docker-compose.yml # New deployment
└── media-downloader.py # Can keep for CLI use
Deployment (Final Step)
Development
# Terminal 1: Backend
cd /opt/media-downloader/backend
uvicorn api.main:app --reload
# Terminal 2: Workers
celery -A workers.celery_app worker --loglevel=info
# Terminal 3: Frontend
cd /opt/media-downloader/frontend
npm run dev
Production
# One command to start everything
docker-compose up -d
# Access at:
# - Frontend: http://localhost:8080
# - Backend API: http://localhost:8000
# - API Docs: http://localhost:8000/docs
Cost Analysis
Time Investment
- Simple dashboard (monitoring only): 1 week
- Minimal viable product: 6 weeks
- Full professional version: 10 weeks
Skills Needed
- Basic: Python, HTML, JavaScript
- Intermediate: FastAPI, Vue.js, Docker
- Advanced: WebSockets, Celery, Redis
Infrastructure
- Hardware: Current server is fine
- Software: All free/open-source
- Hosting: Self-hosted (no cost)
Decision Matrix
| Feature | CLI | Simple Dashboard | Full Web GUI |
|---|---|---|---|
| Run downloads | ✅ | ❌ | ✅ |
| Monitor progress | ❌ | ✅ | ✅ |
| Queue management | ❌ | ❌ | ✅ |
| Scheduler config | ❌ | ❌ | ✅ |
| Live screenshots | ❌ | ❌ | ✅ |
| Mobile access | ❌ | ✅ | ✅ |
| Multi-user | ❌ | ❌ | ✅ |
| Development time | 0 | 1 week | 10 weeks |
| Maintenance | Low | Low | Medium |
Recommendation
Start with Simple Dashboard (1 week)
- See your downloads in a browser
- Check queue status visually
- Access from phone/tablet
- Decide if you want more
If you like it, upgrade to Full Web GUI
- Add interactive features
- Enable queue management
- Implement scheduling UI
- Add live screenshots
Keep CLI as fallback
- Web GUI is primary interface
- CLI for edge cases or debugging
- Both use same database
Next Steps
-
Review the plans in the markdown files I created:
WEB_GUI_DEVELOPMENT_PLAN.md- Complete architectureWEB_GUI_API_SPEC.md- API endpointsWEB_GUI_LIVE_SCREENSHOTS.md- Screenshot streamingWEB_GUI_QUICK_START.md- This file
-
Decide your approach:
- Quick monitoring dashboard (1 week)
- Full professional version (10 weeks)
- Hybrid (monitor now, expand later)
-
Let me know if you want me to:
- Build the simple dashboard (1 week)
- Start Phase 1 of full build (2 weeks)
- Create proof-of-concept (2-3 days)
The live screenshot feature alone makes this worth building - being able to watch your scrapers work in real-time is incredibly cool and useful for debugging!
What approach interests you most?