README

AI Audio Suite - MeloTTS + Whisper

A production-ready Docker deployment of AI audio services including:

MeloTTS: Multi-lingual text-to-speech synthesis
Whisper: OpenAI's speech-to-text transcription

Complete with Go API gateway, React frontend, GPU acceleration, and AWS deployment options. Process audio in both directions - text to speech and speech to text - with a unified API.

🚀 Quick Start (2 minutes)

📖 For detailed setup instructions, see DOCKER-QUICKSTART.md

bash

# Clone and run
git clone https://github.com/jonathanleahy/emlotts-docker.git
cd emlotts-docker

# Start all services with docker-compose
docker-compose up -d

# Or use smart start - automatically detects and uses GPU if available
./start-smart.sh

# Test it (wait ~30 seconds for startup)
curl -X POST http://localhost:8080/api/tts \
  -H "Content-Type: application/json" \
  -d '{"text": "Hello world"}' \
  --output test.wav

# Open web UI
open http://localhost

> 🎯 Smart GPU Detection: The system now detects if you have a GPU and recommends the optimal configuration!

📌 Quick Reference

Feature	Access Point	Description
Web UI	http://localhost	User interface for TTS & STT
TTS API	http://localhost:8080/api/tts	Text-to-speech endpoint
STT API	http://localhost:8080/api/stt/transcribe	Speech-to-text endpoint
Swagger Docs	http://localhost:8080/docs	Interactive API documentation
Health Check	http://localhost:8080/health	Service status with GPU info
Character Limit	5000 (frontend) / Unlimited (API)	Extended text support
GPU Status	Automatic detection	10-20x faster processing
Whisper Models	tiny, base, small, medium, large	Multiple accuracy/speed options

🚀 Features

Text-to-Speech (MeloTTS)

Multi-lingual TTS with English focus (US, British, Indian, Australian accents)
GPU acceleration - 10-20x faster synthesis with NVIDIA GPUs
Extended text support - Up to 5000 characters in frontend, unlimited via API
Production-ready - Pre-downloaded models for instant processing

Speech-to-Text (Whisper)

Multi-language transcription - Support for 100+ languages
Multiple models - From tiny (39M) to large (1550M) for accuracy/speed tradeoffs
Audio format support - WAV, MP3, MP4, M4A, OGG, FLAC
Real-time streaming - WebSocket support for live transcription
Translation mode - Transcribe and translate to English in one step

Infrastructure

Unified API - Single gateway for both TTS and STT services
GPU support - Automatic detection and configuration
Docker deployment - Easy setup with docker-compose
AWS integration - Spot instance support for cost-effective processing
Swagger documentation - Interactive API docs at /docs
Health monitoring - Service status with GPU detection
Flexible deployment - Run services together or separately

✅ Current Implementation

This repository contains a fully working Docker-based MeloTTS deployment that solves the common MeCab/Japanese dependency issues. The solution uses:

melotts_api_server.py - Clean FastAPI implementation without Japanese module patching
start_melotts.py - Startup script that automatically downloads required NLTK data
Pre-built Docker image - melotts-all:latest with all dependencies properly configured
Volume mounts - Python scripts mounted to avoid rebuilding images
Updated Go API - Properly configured to use the correct endpoints
Smart GPU Detection - Automatically detects GPUs and recommends optimal configuration
GPU Fixed - docker-compose.gpu-fixed.yml provides working GPU acceleration

🚀 AWS-Optimized Build

For production AWS deployments, use the AWS-optimized build that includes:

Pre-downloaded models - No runtime downloads, instant startup
Offline mode - Works without internet access after deployment
Smaller attack surface - No external downloads during runtime
Faster cold starts - Ideal for Lambda or spot instances
GPU Support - 10-20x faster with NVIDIA GPUs

Build options:

CPU: ./build-aws.sh
GPU: docker compose -f docker-compose.aws-gpu.yml build

🏗️ Architecture

shell

┌─────────────┐     ┌────────────┐     ┌─────────────────┐
│   React UI  │────▶│   Go API   │────▶│ MeloTTS Service │
│  (Port 80)  │     │ (Port 8080)│     │   (Port 8001)   │
└─────────────┘     └────────────┘     └─────────────────┘
                           │
                           │              ┌─────────────────┐
                           └─────────────▶│ Whisper Service │
                                         │   (Port 8002)   │
                                         └─────────────────┘
                           │
                    ┌──────▼──────┐
                    │   Lambda    │ (Optional)
                    │   Trigger   │
                    └──────┬──────┘
                           │
                    ┌──────▼──────┐
                    │ EC2 Spot    │
                    │  Instance   │
                    └─────────────┘

📋 Prerequisites

Docker and Docker Compose installed
(Optional) AWS account for spot instance processing
(Optional) NVIDIA GPU with drivers for acceleration

🚀 Quick Start

Local Development

#### Automatic GPU Detection (Recommended)

bash

# Clone the repository
git clone https://github.com/jonathanleahy/emlotts-docker.git
cd emlotts-docker

# Smart start - automatically uses GPU if available
./start-smart.sh

#### Manual Start

bash

# CPU version
docker compose up -d

# GPU version (if you have NVIDIA GPU)
./setup-gpu.sh
docker compose -f docker-compose.gpu.yml up -d

#### Test the System

bash

# Test the API
curl -X POST http://localhost:8080/api/tts \
  -H "Content-Type: application/json" \
  -d '{"text": "Hello world"}' \
  --output test.wav

# Check GPU status
curl http://localhost:8001/health | jq .

# Access the web UI
open http://localhost

Production Deployment (EC2)

#### Option 1: Build AWS-Optimized Images (Recommended)

bash

# Build images with pre-downloaded models
./build-aws.sh

# Deploy to AWS
./deploy-aws-optimized.sh

# Or run locally with pre-built images
docker compose -f docker-compose.aws.yml up -d

#### Option 2: Use Regular Images

bash

# Use standard deployment (models download on first run)
./deploy-ec2.sh

🔧 Configuration Options

Available Docker Compose Files

File	Purpose	Features
`docker-compose.yml`	Standard deployment	Basic CPU processing
`docker-compose.gpu-fixed.yml`	GPU acceleration	10-20x faster with NVIDIA GPU
`docker-compose.swagger.yml`	API documentation	Includes Swagger UI
`docker-compose.aws.yml`	AWS CPU deployment	Pre-downloaded models
`docker-compose.aws-gpu.yml`	AWS GPU deployment	Pre-downloaded models + GPU

CPU vs GPU Processing

#### Local Development

bash

# CPU Version (default)
docker compose up -d

# GPU Version (requires NVIDIA GPU)
./setup-gpu.sh
docker compose -f docker-compose.gpu-fixed.yml up -d

# With Swagger Documentation
docker compose -f docker-compose.swagger.yml up -d

#### AWS Deployment

bash

# CPU Version with pre-downloaded models
docker compose -f docker-compose.aws.yml up -d

# GPU Version with pre-downloaded models (10-20x faster)
docker compose -f docker-compose.aws-gpu.yml up -d

Performance Comparison:

CPU: ~30 seconds for a paragraph
GPU: ~2 seconds for a paragraph
Requires: NVIDIA GPU with 4GB+ VRAM

Enable AWS Spot Processing

bash

# Set environment variables
export S3_BUCKET=your-tts-bucket
export AWS_REGION=us-east-1

# Run with spot instance support
docker compose up -d

📡 API Endpoints

📚 Interactive API Documentation

Access the interactive Swagger UI documentation at:

shell

http://localhost:8080/docs

The Swagger UI provides:

Interactive API testing interface
Complete request/response documentation
Parameter descriptions and examples
Try-it-out functionality for all endpoints

To enable Swagger documentation:

bash

# Using docker-compose
docker compose -f docker-compose.swagger.yml up -d

# Or generate docs manually
cd go-api
./generate-swagger.sh

Text-to-Speech API

#### Direct Processing

bash

curl -X POST http://localhost:8080/api/tts \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Hello, world!",
    "language": "EN",
    "speaker": "EN-US",
    "speed": 1.0
  }' \
  --output speech.wav

Spot Instance Processing (Large Texts)

bash

# Submit job
curl -X POST http://localhost:8080/api/tts/spot \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Your hour-long lecture text here..."
  }'

# Check status
curl http://localhost:8080/api/tts/spot/status?job_id=<job-id>

Speech-to-Text API

#### Transcribe Audio

bash

# Transcribe audio file
curl -X POST http://localhost:8080/api/stt/transcribe \
  -F "file=@audio.mp3" \
  -F "model=base" \
  -F "language=en" \
  -F "output_format=json"

# Auto-detect language
curl -X POST http://localhost:8080/api/stt/transcribe \
  -F "file=@audio.wav" \
  -F "model=small"

# Translate to English
curl -X POST http://localhost:8080/api/stt/transcribe \
  -F "file=@foreign_audio.mp3" \
  -F "task=translate"

#### Available Models

tiny - Fastest, least accurate (39M parameters)
base - Good balance (74M parameters)
small - Better accuracy (244M parameters)
medium - High accuracy (769M parameters)
large - Best accuracy (1550M parameters)

Available TTS Speakers

EN-US - American English
EN-BR - British English
EN_INDIA - Indian English
EN-AU - Australian English
EN-Default - Default English

💰 Cost Analysis

Local Processing

Small texts: Instant, free (uses your hardware)
Large texts: ~30 minutes for 1-hour audio

AWS Spot Processing

1-hour lecture: ~$0.006-$0.008 (auto-terminates)
Processing time: ~3 minutes with GPU, ~30 minutes with CPU
Storage: First 1GB/month free

🐳 Docker Images

Build Locally

bash

# Production build with pre-downloaded models
docker build -f melotts-service/Dockerfile.production -t melotts-prod melotts-service/

# Minimal English-only build
docker build -f melotts-service/Dockerfile.english -t melotts-english melotts-service/

Push to Registry

bash

# Docker Hub
docker tag melotts-prod yourusername/melotts-service:latest
docker push yourusername/melotts-service:latest

# Amazon ECR
aws ecr get-login-password | docker login --username AWS --password-stdin <ecr-url>
docker tag melotts-prod <ecr-url>/melotts-service:latest
docker push <ecr-url>/melotts-service:latest

🛠️ Development

Project Structure

shell

tts-merotts/
├── melotts/             # MeloTTS text-to-speech service
│   ├── Dockerfile
│   └── requirements.txt
├── whisper/             # Whisper speech-to-text service
│   ├── Dockerfile
│   └── requirements.txt
├── shared/              # Shared components
│   ├── go-api/         # Go REST API gateway
│   └── react-frontend/ # React web UI
├── aws/                 # Lambda functions for spot processing
├── docker-compose.yml   # Main service orchestration
├── DOCKER-QUICKSTART.md # Docker setup guide
└── CLAUDE.md           # AI assistant instructions

Running Individual Services

bash

# MeloTTS only
cd melotts-service
python app.py

# Go API only
cd go-api
go run main.go

# React dev server
cd react-frontend
npm install && npm start

🚨 Troubleshooting

Common Issues

#### "Failed to synthesize speech" Error This usually means the MeloTTS service isn't ready yet:

bash

# Check service status
docker compose ps

# Check logs
docker compose logs melotts-service

# Wait for health check to pass
# The first startup takes time to download models

#### Docker Build Issues

bash

# Clear Docker cache
docker system prune -a

# Build with no cache
docker compose build --no-cache

#### Model Download Issues The models are downloaded on first startup. If download fails:

bash

# Restart the service
docker compose restart melotts-service

# Check logs
docker compose logs -f melotts-service

#### MeCab Dictionary Errors If you see MeCab-related errors:

bash

# The working Dockerfile includes fixes for this
# Make sure you're using Dockerfile.working

GPU Not Detected

bash

# Check NVIDIA drivers
nvidia-smi

# Verify Docker GPU support
docker run --rm --gpus all nvidia/cuda:11.8.0-base nvidia-smi

📊 Performance Benchmarks

Text Size	CPU Time	GPU Time	Cost (Spot)
1 paragraph	5 sec	0.5 sec	$0.00001
10 min speech	3 min	20 sec	$0.001
1 hour lecture	30 min	3 min	$0.006

🔒 Security Notes

Never expose ports 8001 or 8080 publicly without authentication
Use environment variables for sensitive configuration
Implement rate limiting for production deployments
Keep Docker images updated

📝 License

This project uses MeloTTS which is licensed under MIT License.

🤝 Contributing

Pull requests are welcome! Please ensure:

Code follows existing patterns
Docker builds complete successfully
Documentation is updated

📞 Support

Create an issue for bugs or feature requests
Check existing issues before creating new ones
Include relevant logs and system information