StateLoop - Complete System Specification

1. Overview

StateLoop is a stateless agent orchestration system for coordinating AI agents in multi-party negotiations. Agents negotiate to reach consensus on decisions - from policy debates to project approaches to resource allocation.

1.1 Core Philosophy

The system implements a "stateless agent" pattern where:

• Agents have no memory - All context is fetched fresh each turn
• State lives in the system - SQLite database holds all state
• Private agendas - Each agent only sees their own preferences
• Hot-swappable - Agents can be stopped/started without losing progress
• AI-driven setup - AI analyzes scenarios, determines appearances, identifies speakers

1.2 Key Benefits

2. System Architecture

┌─────────────────────────────────────────────────────────────────┐
│                         Web Browser                              │
│  ┌─────────────────────────────────────────────────────────┐    │
│  │                    Thronglet UI                          │    │
│  │  - 2D Canvas visualization                               │    │
│  │  - Conversation thread                                   │    │
│  │  - Case management                                       │    │
│  └─────────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────────┘
                               │
                               │ HTTP/REST
                               ▼
┌─────────────────────────────────────────────────────────────────┐
│                      Express.js Server                           │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐          │
│  │   Routes     │  │   Services   │  │   Storage    │          │
│  │  (REST API)  │──│  (Business)  │──│  (SQLite)    │          │
│  └──────────────┘  └──────────────┘  └──────────────┘          │
└─────────────────────────────────────────────────────────────────┘
                               │
                               │ HTTP (auto-play)
                               ▼
┌─────────────────────────────────────────────────────────────────┐
│                      Claude API                                  │
│  - Generates agent responses                                     │
│  - Receives private agenda per agent                            │
└─────────────────────────────────────────────────────────────────┘

2.1 Technology Stack

3. Data Models

3.1 Case

interface Case {
  id: string;                    // e.g., "case-a4573814"
  scenario: string;              // Full case description text
  status: 'active' | 'resolved' | 'abandoned';
  currentTurn: string | null;    // Participant ID whose turn it is
  outcome: 'agreed' | 'failed' | 'abandoned' | null;
  selectedOptionId: string | null;
  resolutionSummary: string | null;
  createdAt: string;             // ISO timestamp
  updatedAt: string;
  resolvedAt: string | null;
}

3.2 Participant

interface Participant {
  id: string;                    // e.g., "case-abc-person-0"
  caseId: string;
  name: string;                  // Display name, e.g., "Alice"
  preferences: string[];         // Public preferences (optional)
  constraints: string[];         // Public constraints (optional)
  isPayer: boolean;              // Whether this person is paying
}

3.3 Option

interface Option {
  id: string;                    // e.g., "case-abc-opt-1"
  caseId: string;
  name: string;
  category: string;
  priceRange: string;            // $, $$, $$$
  features: string[];
}

3.4 Message

interface Message {
  id: string;
  caseId: string;
  author: string;                // Participant ID
  type: 'proposal' | 'counter' | 'accept' | 'reject' | 'message';
  content: string;
  optionId: string | null;       // For proposals
  timestamp: string;
  agentContext: string | null;   // Private agenda shown to agent (debug)
}

3.5 BossMessage

interface BossMessage {
  id: string;
  caseId: string;
  content: string;
  targetAgent: string | null;    // null = all agents
  read: boolean;
  timestamp: string;
}

4. Case Description Format

Cases are created using a structured text format that defines agents, their private agendas, and negotiation options.

4.1 Format Specification

PUBLIC INFO:
[Optional public scenario description visible to all]

AGENT: <Name>
AGENDA: [Private instructions only this agent sees]
[Additional lines of private context...]

AGENT: <Name2>
AGENDA: [Different private instructions...]

OPTIONS:
- <Option Name>: <Category>, <Price>, <Features>
- <Option Name2>: ...

4.2 Example

PUBLIC INFO:
Scenario: Team decision on project approach.
Participants: Moderator, Alice, Bob

AGENT: Moderator
AGENDA: You introduce the case. Say "Welcome! You need to pick
an approach for the project. Alice, please start."

AGENT: Alice
AGENDA: You STRONGLY prefer the structured approach.
Budget is $40k max. You're willing to compromise but push for
Approach A first.

AGENT: Bob
AGENDA: You need flexibility. You prefer approaches that allow
for iteration. Try to get Approach B.

OPTIONS:
- Approach A: Structured, $$$, reliable, established process
- Approach B: Agile, $$, flexible, iterative development
- Approach C: Hybrid, $, balanced, combines both methods

4.3 Parsing Rules

4.4 Location-Based Environments

The system supports different visual backgrounds based on scenario location. Location can be specified explicitly or auto-detected from scenario keywords.

Explicit Location:

LOCATION: hospital

PUBLIC INFO:
A medical team discusses patient care options...

Supported Locations:

• "hospital", "patient", "nurse", "doctor" → hospital
• "library", "books", "reading" → library
• "office", "meeting", "conference" → office
• "school", "classroom", "student", "teacher" → school
• "cafe", "coffee", "barista" → cafe

• Location name (e.g., "Hospital Ward")
• Optional subtitle based on scenario context

4.5 AI Setup Phase

When a new case is created and auto-play is called for the first time (no messages yet), the system enters a Setup Phase where AI:

4.5.1 Determines Agent Appearances

AI analyzes each agent's personality and role to determine their visual appearance on the Thronglet map:

POST /api/cases/:id/set-appearance
{
  "agentName": "Alice",
  "appearance": {
    "accessory": "glasses",
    "bodyStyle": "normal",
    "color": "#3498db",
    "skinTone": "#ffcc80",
    "professionalRole": "nurse_scrubs"
  }
}

Appearance Options:

• Different genders (male/female)
• Physical disabilities (wheelchair users)
• Various skin tones
• Multiple accessory styles
• Professional uniforms

4.5.2 Identifies First Speaker

The system automatically determines who speaks first based on currentTurn in the database. The setup prompt tells AI exactly who should make the opening move.

4.5.3 Makes Opening Move

AI submits the first message as the identified speaker, kicking off the negotiation.

5. API Specification

5.1 Endpoint Summary

Company/Building Endpoints

Agent Profile Endpoints

5.2 Auto-Play Endpoint (Primary)

The main endpoint for advancing the negotiation.

GET /api/cases/:id/auto-play or GET /api/auto-play

What it does:

Response (text/plain):

YOU ARE: Alice

YOUR PRIVATE AGENDA (only you know this):
You STRONGLY prefer Japanese food...

OTHER PARTICIPANTS: Moderator, Bob
(You do NOT know their private preferences)

OPTIONS:
- case-abc-opt-1: Approach A (Structured, $$$) - reliable, established

CONVERSATION SO FAR:
[Moderator] (message): Welcome! Please start...

YOUR TASK:
Respond as Alice. Negotiate based on your agenda.

To respond, POST to: http://localhost:3000/api/cases/abc/submit
{...}

5.3 Submit Response

POST /api/cases/:id/submit

Request:

{
  "taskId": "task-xyz",
  "agentId": "case-abc-person-0",
  "agentContext": "Private agenda text (optional, for debug)",
  "response": {
    "type": "proposal",
    "thoughts": "This aligns with my goals",
    "content": "How about Approach A?",
    "optionId": "case-abc-opt-1",
    "documentUpdates": [
      { "document": "notes", "action": "append", "content": "Proposed Approach A" }
    ]
  }
}

Response Fields:

• type - Required: proposal, counter, accept, reject, or message
• content - Required: The spoken message
• thoughts - Optional: Internal reasoning (shown in UI, not to other agents)
• optionId - Required for proposal/counter types
• documentUpdates - Optional: Array of document edits

• proposal - Suggest an option (requires optionId)
• counter - Counter-proposal (requires optionId)
• accept - Accept the last proposal (auto-resolves case)
• reject - Reject without alternative
• message - General comment

When case is still active, the submit response includes the NEXT agent's prompt, enabling continuous processing without user intervention:

SUBMISSION ACCEPTED
===================
Your message was recorded. Case is still active.

NEXT TURN
=========
YOU ARE: Bob
...

This allows Claude to automatically continue processing turns until the case resolves, without waiting for user input between turns.

JSON Response (when Content-Type: application/json is set):

{
  "messageId": "msg-3",
  "caseStatus": "active",
  "nextTurn": "case-abc-person-1"
}

5.4 Auto-Resolution

When an agent submits type: "accept":

5.5 History Endpoint

GET /api/cases/:id/history

Returns full conversation history optimized for playback:

{
  "caseId": "case-abc",
  "scenario": "...",
  "status": "resolved",
  "outcome": "agreed",
  "participants": [
    { "id": "...", "name": "Alice", "appearance": {...} }
  ],
  "options": [...],
  "timeline": [
    {
      "index": 0,
      "messageId": "msg-xyz",
      "speaker": "Alice",
      "speakerId": "case-abc-person-1",
      "type": "proposal",
      "content": "How about Zen?",
      "timestamp": "2024-01-01T12:00:00Z"
    }
  ],
  "totalMessages": 5
}

5.6 Run-to-Completion Endpoint

POST /api/cases/:id/run

Pre-runs a case to completion using simulated agent responses. Useful for:

• Testing scenarios quickly
• Pre-generating content for playback
• Batch processing multiple negotiations

• maxRounds (optional): Maximum rounds before stopping (default: 20)

• Proposals → Appends to script document with the proposed option
• Accepts → Appends to decisions document with the agreed outcome
• Early messages → Appends to notes document (first 3 messages per agent)

Response:

{
  "status": "resolved",
  "outcome": "agreed",
  "rounds": 6,
  "messageCount": 6,
  "log": [
    "  [DOC] notes: append - Alice's opening position...",
    "[Alice] (message): I believe we should...",
    "  [DOC] script: append - [Alice's proposal: Option A]...",
    "[Alice] (proposal): I propose we go with Option A.",
    "  [DOC] decisions: append - AGREED: Option A...",
    "[Bob] (accept): Yes, let's do it."
  ],
  "historyUrl": "/api/cases/case-abc/history"
}

The log includes [DOC] entries showing document updates alongside agent messages.

5.7 Working Documents

Cases can have collaborative working documents that agents update during the negotiation:

Document Types:

• Input Documents - Read-only reference materials (character guides, style guides)
• Working Documents - Collaborative documents agents can edit (scripts, notes, decisions)

{
  "taskId": "task-abc",
  "agentId": "case-123-person-0",
  "response": {
    "type": "message",
    "content": "I've drafted the opening scene",
    "documentUpdates": [
      {
        "document": "script",
        "action": "append",
        "content": "INT. LOBBY - DAY\n\nBasil enters, practicing deep breaths..."
      }
    ]
  }
}

5.8 Swagger API Documentation

Full interactive API documentation is available at:

• Swagger UI: http://localhost:3000/api-docs
• OpenAPI Spec: http://localhost:3000/swagger-generated.json (auto-generated)

• All endpoints with request/response schemas
• Example payloads for each operation
• Schema definitions for all data types

6. UI Specification (Thronglet)

6.1 Layout

┌─────────────────────────────────────────────────────────────────┐
│  Header: Logo, Case Selector, New Case, Reset, Audio, 2D/3D,    │
│          Replay, Config, Theme Toggle                            │
├─────────────────────────────────┬───────────────────────────────┤
│                                 │  Conversation Thread          │
│    2D Canvas Map                │  - Chat-style messages        │
│    - Agents (varied looks)      ├───────────────────────────────┤
│    - Trees, Path                │  Agent Thoughts (collapsible) │
│    - Options (when case         │  - Internal reasoning         │
│      loaded)                    │  - Synced with speaker        │
│    - Speech bubbles             ├───────────────────────────────┤
│                                 │  Case Info Panel              │
│                                 │  - Participant cards          │
│                                 ├───────────────────────────────┤
│                                 │  Boss Messages Panel          │
├─────────────────────────────────┴───────────────────────────────┤
│  Agent Prompt Card - Curl command for auto-play                 │
├─────────────────────────────────────────────────────────────────┤
│  Request Log - Collapsible API request history                  │
└─────────────────────────────────────────────────────────────────┘

6.1.1 Agent Thoughts Panel

The Agent Thoughts panel displays the internal reasoning of each agent:

• Location: Sidebar, below Conversation Thread
• Collapsible: Click header to expand/collapse
• Auto-expand: Opens automatically when new thoughts arrive
• Synced: Shows thoughts from the currently speaking agent
• Styling: Purple theme to distinguish from public speech

6.2 Agent Visualization

Visual Variety: Each agent has distinct features:

• Body color (red, blue, purple, green, orange, teal, pink, cyan)
• Accessory (none, hat, glasses, bowtie, headphones, scarf, wheelchair)
• Gender representation (different voice types)
• Skin tone diversity
• Body style (normal, tall, short, wide)
• Skin tone variation
• Animation speed variation

• Idle: Various behaviors (standing, wandering, checking phone, etc.)
• Thinking: Orange indicator dot
• Speaking: Speech bubble visible

• Trees (50px radius)
• Options (60px radius)
• Canvas boundaries
• Speech bubbles (move away)

6.3 Idle Behaviors

Agents perform random actions when idle:

• Standing, looking, wandering
• Checking phone, looking at watch
• Stretching, waving, yawning
• Tapping foot, scratching head
• Crossing arms, humming
• Pacing, jumping, sitting
• Daydreaming, chitchat

6.4 Options

• Only drawn when a case is loaded
• Icons appropriate to option type
• Name labels below
• Color states:

6.5 Text-to-Speech and Audio System

The UI supports text-to-speech for agent messages with a comprehensive audio system:

Audio System Features:

• Audio is enabled by default on page load
• Toggle audio on/off via button in header
• Each agent has a unique voice (pitch, rate, voice type)
• Voice settings are determined during AI setup phase
• Speech bubble stays visible for 15 seconds after audio ends
• Polling pauses while speech is playing to avoid interruptions

6.6 Speaker Positioning

When an agent speaks, the system automatically repositions agents for optimal visibility:

Speaker Movement:

• Speaking agent moves to a clear center-left position (35% across canvas, y=380)
• Original position is stored so they can return after speaking
• Movement is animated smoothly via the speaking_move idle action
• Speaker's original position is restored when their message completes

• Non-speaking agents automatically move away from the speech bubble area
• Agents on the left of the speaker move further left
• Agents on the right of the speaker move further right
• All non-speakers move down slightly for clear visual separation
• Movement uses the avoiding idle action with faster speed (2.0x)

Speech Bubble Zone:
┌─────────────────────────────────────────────┐
│                                             │
│         ┌─────────────────┐                 │
│         │  Speech Bubble  │                 │
│         │     (320px)     │                 │
│         └────────┬────────┘                 │
│                  │                          │
│              ╔═══╧═══╗                      │
│              ║Speaker║  ← Moves to y=380    │
│              ╚═══════╝                      │
│   ○ ←──────              ──────→ ○          │
│ Agent A                        Agent B      │
│ (moves left)               (moves right)   │
└─────────────────────────────────────────────┘

Benefits:

• Speaker is always clearly visible without overlap
• Speech bubble doesn't obscure other agents
• Conversation flow is visually clear
• Agents return to natural positions between messages

6.7 Interactive Features

Agent Selection:

• Click on agents in the canvas to select them
• Click on participant cards in the side panel
• Selected agent shows expanded details (ID, gender, accessory, voice)
• Selection highlighted with colored border

• "Zoom In" button activates focused view mode
• When zoomed in, camera frames current speaker + previous 2 speakers
• Dynamic zoom level adjusts to fit all 3 agents with padding
• Smooth pan/zoom animation with camera lerping

• "Reset & Replay" button in the map header
• Soft reset: Clears messages, keeps agents and options
• Full reset: Regenerates entire case setup
• Automatically runs simulation after reset

• Agents perform idle animations when not their turn
• Chitchat disabled during active negotiations
• Agents avoid walking into speech bubbles

6.8 2D/3D View Toggle

The canvas supports switching between two rendering modes:

2D Mode:

• Classic pixelated top-down view
• Simpler rendering, lower resource usage
• All accessories and professional roles supported

• Isometric pseudo-3D rendering with depth
• Body shapes (box, cylinder, oval) affect rendering
• Enhanced visual appeal with shading and perspective
• Location furniture rendered with 3D depth
• Transparent backgrounds for furniture sprites

• Toggle button in the map header switches between modes
• Mode preference persisted in localStorage
• Smooth visual transition when switching
• Both modes support all agent customization options

6.9 Location Viewer (scenarios.html)

The scenarios page includes an interactive location viewer for previewing scenario environments.

Mouse-Based Rotation:

• Click and drag horizontally to rotate the room view
• Smooth, responsive rotation following mouse movement
• True 3D rotation in isometric space (not 2D canvas rotation)

• Room has 4 walls (front, back, left, right)
• Front wall is always culled to show room interior
• Other walls are culled when rotation makes them face the viewer
• Creates an intuitive "cutaway" view that always shows room contents

• Zoom In (+): Increase scale for detail view
• Zoom Out (-): Decrease scale for overview
• Reset: Return to default zoom level

7. Privacy Model

7.1 Information Boundaries

7.2 Agenda Extraction

The system extracts each agent's private agenda from the case description:

8. Resolution Logic

8.1 Automatic Resolution

When an accept message is submitted:

8.2 Manual Resolution

Operators can resolve via:

POST /api/cases/:id/resolve
{
  "outcome": "agreed" | "failed" | "abandoned",
  "selectedOption": "opt-id",
  "summary": "Resolution notes"
}

9. Testing

9.1 API Tests

Located in /tests/api.test.ts (15 tests):

9.2 Running Tests

npm test

All tests use an in-memory SQLite database for isolation.

10. Configuration

10.1 Environment Variables

10.2 Server Setup

# Install dependencies
npm install

# Run development server
npm run dev

# Run production
npm start

# Run tests
npm test

11. Example Workflow

11.1 Creating a Case

11.2 Running Negotiation

11.3 Using Auto-Play

Simply call:

curl "http://localhost:3000/api/auto-play"

This returns a prompt for the current agent, ready to send to Claude.

12. Files and Structure

stateLoop/
├── src/
│   ├── index.ts           # Entry point
│   ├── api/
│   │   └── routes.ts      # REST endpoints
│   ├── services/
│   │   ├── caseService.ts # Business logic
│   │   └── taskService.ts # Task management
│   ├── storage/
│   │   └── sqlite.ts      # Database operations
│   └── types/
│       └── index.ts       # TypeScript interfaces
├── public/
│   ├── index.html         # Main UI
│   ├── css/
│   │   └── styles.css     # Styling
│   └── js/
│       └── thronglet.js   # Canvas UI logic
├── scenarios/             # Pre-made scenario library
│   └── *.txt              # Scenario definition files
├── tests/
│   └── api.test.ts        # API tests
├── docs/
│   ├── SPEC.md            # This document
│   ├── requirements.md    # Requirements
│   ├── api-design.md      # API details
│   ├── data-models.md     # Data schemas
│   └── ui-design.md       # UI specification
├── package.json
└── tsconfig.json

13. Scenarios Library

13.1 Pre-made Scenarios

The system includes a library of pre-made scenarios in the /scenarios directory.

Accessing Scenarios:

• Web UI: /scenarios.html page with tabs for Scenarios, Furniture, Locations
• API: GET /api/scenarios lists available scenarios
• Loading: POST /api/scenarios/:name/load creates a case from a scenario

Example Scenarios:

13.2 Furniture Catalog

The furniture catalog (/public/data/furniture.json) provides location-appropriate furniture items:

Categories:

• Transparent backgrounds for PNG furniture sprites
• Proper depth sorting for layered rendering
• Isometric perspective matching agent rendering
• Shadow and lighting effects for visual consistency
• Dynamic scaling based on canvas size

13.3 Enhanced Simulation

The /run endpoint uses intelligent response generation based on agent roles:

AGREEABILITY Score (0-100):

• Low (< 45): Skeptical, raises concerns, needs more discussion rounds
• Medium (45-70): Open to discussion, will agree after reasonable debate
• High (> 70): Agreeable, quick to accept proposals

Automatic Document Writing: When working documents exist (script, notes, decisions), the simulation automatically:

14. Agent Customizer (scenarios.html)

The Scenarios page includes a comprehensive agent customizer for designing agent appearances.

14.1 Customization Options

14.1.1 Age Groups

14.1.2 Body Styles

14.1.3 Body Shapes (3D Mode)

14.1.4 Skin Tone Presets

The system provides 6 preset skin tones for diverse representation:

14.1.5 Hair Colors

14.1.6 Accessories

14.1.7 Professional Roles

14.1.8 Mobility Options

14.2 Live Preview

The customizer includes:

• Real-time 2D/3D preview canvas
• JSON code output for copying to scenarios
• Instant updates on any setting change

14.3 Age-Based Adjustments

15. Implemented Features

16. Future Enhancements

16.1 Planned: Workflows

Multi-stage workflows where output from one task feeds into the next:

• Workflow Templates - YAML-defined reusable workflow patterns
• Stage Chaining - Documents flow automatically between stages
• Progress Tracking - Visual progress through workflow stages
• Mermaid Diagrams - Auto-generated workflow visualizations

16.2 Planned: Goal-Driven Workflows

AI-planned workflows based on high-level goals:

• Goal Definition - Specify desired outcome and constraints
• Auto-Planning - AI generates execution plan
• Dynamic Stages - Stages created based on goal requirements

16.3 Planned: Variability & Randomness

Control agent behavior unpredictability:

• Temperature - Overall randomness (0=deterministic, 1=creative)
• Mood Swings - Moods evolve based on conversation
• Quirks & Triggers - Per-agent behavioral tendencies
• Chaos Levels - calm, normal, heated, chaotic