Documentation

DEMO JOBS GUIDE

Demo Jobs System - Complete Guide

Overview

The VideoVox demo jobs system demonstrates asynchronous job processing using RabbitMQ workers, REST APIs, Redis caching, and real-time frontend updates.

Architecture

shell

┌─────────────┐      ┌──────────────┐      ┌─────────────┐
│  Frontend   │─────▶│  Backend API │─────▶│  RabbitMQ   │
│  (React)    │      │   (Go/Gin)   │      │   Queue     │
└─────────────┘      └──────────────┘      └─────────────┘
       │                     │                      │
       │                     ▼                      ▼
       │              ┌──────────────┐      ┌─────────────┐
       │              │    Redis     │      │   Worker    │
       └──────────────│   (Cache)    │◀─────│  (Go)       │
           Polling    └──────────────┘      └─────────────┘
                             │                      │
                             ▼                      ▼
                      ┌──────────────────────────────────┐
                      │         MySQL Database           │
                      │      (demo_jobs table)           │
                      └──────────────────────────────────┘

Components

Backend Services

#### 1. Customer API (Port 8080)

Endpoints:

- POST /api/jobs - Create demo job - GET /api/jobs/:id - Get job status - GET /api/jobs - List user's jobs (paginated) - DELETE /api/jobs/:id - Cancel job

Authentication: JWT Bearer token
Database: MySQL (customer_api user - restricted permissions)
Queue: RabbitMQ customer_vhost

#### 2. Admin API (Port 8081)

Endpoints: Same as customer API
Authentication: JWT Bearer token (admin role)
Database: MySQL (admin_api user - full permissions)
Queue: RabbitMQ admin_vhost

#### 3. Demo Worker - Customer (Container)

Queue: customer_demo_jobs
Processing: 20-second jobs with 1-second increments
Progress: Updates every second (5% per iteration)
Caching: Updates Redis on start, progress, completion

#### 4. Demo Worker - Admin (Container)

Queue: admin_demo_jobs
Processing: Same as customer worker
Isolation: Separate queue and vhost

#### 5. Redis Cache (Port 6379)

TTL: 5 minutes
Keys: job:{uuid}
Purpose: Fast job status lookups without database queries

Frontend Applications

#### 1. Customer Frontend (Port 3000)

Dashboard: /dashboard
Features:

- JobDemo component (create and monitor jobs) - JobList component (paginated job history) - Real-time polling (1s for active jobs, 5s for job list) - Status filtering

#### 2. Admin Frontend (Port 3001)

Dashboard: /dashboard
Features: Same as customer frontend
Difference: Admin can see all users' jobs

Job Lifecycle

1. Job Creation

shell

User clicks "Start Demo Job"
    ↓
Frontend: POST /api/jobs {type: "demo_job"}
    ↓
Backend:
  - Generate UUID
  - Insert into demo_jobs table (status: pending)
  - Publish message to RabbitMQ
  - Return job ID to frontend
    ↓
Frontend: Start polling GET /api/jobs/{id}

2. Job Processing

shell

Worker receives message from queue
    ↓
Worker:
  - Fetch job from database
  - Check status (must be pending)
  - Mark as processing (started_at timestamp)
  - Cache job in Redis
    ↓
For i = 1 to 20:
  - Sleep 1 second
  - Check if cancelled in database
  - Update progress (i * 5%)
  - Update Redis cache
  - Log progress
    ↓
Worker:
  - Mark job as completed
  - Update completed_at timestamp
  - Cache final status
  - ACK message

3. Job Monitoring

shell

Frontend polls every 1 second:
    ↓
GET /api/jobs/{id}
    ↓
Backend:
  - Check Redis cache (O(1))
  - If cache miss, query database
  - Cache result for 5 minutes
  - Return job status
    ↓
Frontend:
  - Update progress bar
  - Update status badge
  - Update elapsed time
  - Stop polling if job complete

Database Schema

sql

CREATE TABLE demo_jobs (
    id VARCHAR(36) PRIMARY KEY,
    user_id INT NOT NULL,
    type VARCHAR(50) NOT NULL,
    status ENUM('pending', 'processing', 'completed', 'failed', 'cancelled'),
    progress INT DEFAULT 0,
    error_message TEXT,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    started_at TIMESTAMP NULL,
    completed_at TIMESTAMP NULL,
    cancelled_at TIMESTAMP NULL,
    timeout_at TIMESTAMP NULL,
    INDEX idx_user_id (user_id),
    INDEX idx_status (status),
    INDEX idx_created_at (created_at)
);

Testing Guide

Prerequisites

bash

# Start all services
docker compose up

# Verify services are running
docker ps | grep videovox

# Check logs
docker logs videovox-demo-worker-customer
docker logs videovox-demo-worker-admin

Test 1: Create and Monitor Job (Customer)

Navigate to customer dashboard

shell

   http://localhost:3000/dashboard

Login with test credentials

- Email: test@example.com - Password: (your test password)

Create a job

- Click "Start Demo Job" button - Observe job ID appears (e.g., a1b2c3d4...) - Status badge shows "Pending" (gray)

Monitor progress

- Status changes to "Processing" (blue) within 1-2 seconds - Progress bar animates from 0% to 100% - Updates every second (5%, 10%, 15%, etc.) - Elapsed time increments

Verify completion

- After 20 seconds, status changes to "Completed" (green) - Progress bar shows 100% - Completed timestamp appears - "Start New Job" button enabled

Test 2: Job Cancellation

Create a job

- Click "Start Demo Job" - Wait for status to change to "Processing"

Cancel the job

- Click "Cancel" button (appears during processing) - Status immediately changes to "Cancelled" (outline badge)

Verify worker behavior

- Check worker logs:

bash

     docker logs videovox-demo-worker-customer | grep cancelled

- Should see: "🚫 Job {id} was cancelled"

Test 3: Job List and Pagination

Create multiple jobs

- Create 15+ jobs (some pending, some completed, some cancelled)

Verify job list

- Scroll down to "Job History" section - See list of jobs with status badges - Total count shows correct number

Test status filtering

- Click "Completed" filter button - Verify only completed jobs show - Click "Processing" filter - Verify only processing jobs show - Click "All" to clear filter

Test pagination

- If 10+ jobs exist, pagination appears - Click "Next" to go to page 2 - Click "Previous" to return to page 1 - Verify page indicator (e.g., "Page 1 of 2")

Verify auto-refresh

- Create a new job - Wait 5 seconds - New job should appear in the list automatically

Test 4: Redis Caching

Monitor Redis cache

bash

   # Connect to Redis
   docker exec -it videovox-redis redis-cli

   # Watch for job keys
   MONITOR

Create a job

- Observe Redis SET commands when job starts - Observe Redis GET commands when frontend polls

Verify cache hit

- Check customer API logs for cache hits - Should see faster response times for cached jobs

Test 5: Admin Dashboard

Login as admin

- Navigate to: http://localhost:3001/dashboard - Email: jon@jon.com - Password: admin123

Verify admin privileges

- Job list should show ALL users' jobs (not just admin's) - Create admin jobs - Verify they process through admin worker

Test admin queue

bash

   # Check admin worker logs
   docker logs videovox-demo-worker-admin

Test 6: Error Handling

Stop a worker

bash

   docker stop videovox-demo-worker-customer

Create a job

- Job should stay in "Pending" status - No progress updates

Restart worker

bash

   docker start videovox-demo-worker-customer

- Worker should pick up pending job - Job should process normally

Test 7: RabbitMQ Management

Access RabbitMQ UI

- Customer: http://localhost:15672 - Admin: http://localhost:15673

Login credentials

- Customer: customer_queue / customerqueuepass - Admin: admin_queue / adminqueuepass

Verify queues

- Navigate to "Queues" tab - See customer_demo_jobs and admin_demo_jobs - Monitor message rates

Test message flow

- Create a job - Watch queue message count increase - Watch it decrease as worker consumes

Performance Metrics

Expected Performance

Metric	Value
Job creation	< 100ms
Cache hit response	< 5ms
Cache miss response	< 50ms
Job duration	20 seconds
Progress update interval	1 second
Frontend polling interval	1 second (active job)
Job list refresh interval	5 seconds
Redis TTL	5 minutes

Monitoring

bash

# Worker throughput
docker stats videovox-demo-worker-customer

# API response times
docker logs videovox-customer-api | grep "GET /api/jobs"

# Queue depth
# Access RabbitMQ UI to see queue metrics

# Redis memory usage
docker exec videovox-redis redis-cli INFO memory

Troubleshooting

Jobs stuck in "Pending"

Symptoms: Jobs never start processing

Causes:

Worker not running

RabbitMQ connection failed

Queue not declared

Solutions:

bash

# Check worker status
docker ps | grep demo-worker

# Check worker logs
docker logs videovox-demo-worker-customer

# Restart worker
docker restart videovox-demo-worker-customer

Progress not updating

Symptoms: Job stuck at same progress percentage

Causes:

Frontend polling stopped

Redis cache stale

Worker crashed mid-job

Solutions:

bash

# Check browser console for errors
# Clear Redis cache
docker exec videovox-redis redis-cli FLUSHALL

# Check worker logs
docker logs videovox-demo-worker-customer --tail 100

"Failed to create job" error

Symptoms: Frontend shows error when creating job

Causes:

Not authenticated (JWT expired)

API not reachable

Database connection failed

RabbitMQ unavailable

Solutions:

bash

# Check API logs
docker logs videovox-customer-api

# Verify JWT token in browser sessionStorage
# Key: videovox_token

# Check API health
curl http://localhost:8080/health

# Restart API
docker restart videovox-customer-api

Advanced Usage

Custom Job Types

To add new job types, modify:

Backend: Update DemoJobType enum in backend/internal/models/job.go

Worker: Add processing logic in processMessage()

Frontend: Update CreateJobRequest type

Adjusting Job Duration

Modify worker processing loop:

// Change from 20 iterations to 10 for 10-second jobs
for i := 1; i <= 10; i++ {
    time.Sleep(1 * time.Second)
    progress := i * 10 // 10% per iteration
    // ...
}

Custom Polling Intervals

tsx

<JobDemo autoRefresh={true} refreshInterval={2000} /> // 2 seconds
<JobList autoRefresh={true} refreshInterval={10000} /> // 10 seconds

API Examples

Create Job

bash

curl -X POST http://localhost:8080/api/jobs \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"type": "demo_job"}'

Get Job Status

bash

curl http://localhost:8080/api/jobs/{JOB_ID} \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

List Jobs

bash

curl "http://localhost:8080/api/jobs?page=1&per_page=10&status=completed" \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

Cancel Job

bash

curl -X DELETE http://localhost:8080/api/jobs/{JOB_ID} \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

Summary

This demo jobs system showcases:

✅ Asynchronous job processing with RabbitMQ
✅ Real-time progress updates via polling
✅ Redis caching for performance
✅ Clean separation of customer/admin concerns
✅ Docker containerization
✅ Professional error handling
✅ Graceful shutdown support
✅ Job cancellation
✅ Pagination and filtering
✅ Real-time UI updates

Perfect foundation for building production video processing features!