Documentation

API REFERENCE

API Reference

Audio Receiver HTTP API (KickPi :8081)

POST /audio

Receive raw PCM audio from the ESP32 microphone and process through the voice pipeline.

Request:

Content-Type: application/octet-stream
Body: Raw 16-bit PCM audio at 16kHz mono

Response: 200 OK with text ok (X.Xs) (duration)

Pipeline (async, returns immediately):

Wrap in WAV → Whisper ASR → transcription

Enrich with context → Ollama LLM (with tools) → response text

Strip directives → Piper TTS → PCM audio

POST to ESP32 /play for speaker output

Files Created:

msg_<timestamp>.raw - Original recording
msg_<timestamp>.txt - Transcription
msg_<timestamp>.reply.txt - LLM response
msg_<timestamp>.reply.raw - TTS audio

GET /latest

Returns the most recent conversation exchange. Polled by the ESP32 conversation screen.

Query Parameters:

greet=1 (optional): Trigger a spoken greeting from Robin

Response (JSON):

json

{
  "user": "What's the weather like?",
  "reply": "It's currently 12°C and cloudy in London.",
  "shopping": ["milk", "bread", "eggs"],
  "status": "done",
  "ts": 1708012345,
  "time": "14:32"
}

Field	Type	Description
`user`	string	Transcribed user message
`reply`	string	Robin's response (directives stripped)
`shopping`	string[]	Current shopping list
`status`	string	`"idle"`, `"processing"`, or `"done"`
`ts`	int64	Unix timestamp of the message
`time`	string	Current time in HH:MM (UK timezone)

GET /shopping

Returns the current shopping list.

Response (JSON):

json

["milk", "bread", "eggs"]

POST /notify

Receive a sensor event notification from the ESP32 and start the escalation sequence.

Query Parameters:

event (required): Event type string

Event Types:

vibration / vibration_night - Vibration sensor triggered
button / button_night - Button pressed
presence_front / presence_front_night - Front door presence
presence_front2 / presence_front2_night - Front door 2 presence
presence_basement / presence_basement_night - Basement presence

Response: 200 OK with text ok

GET /speak

Trigger Robin to speak a text message through the ESP32 speaker.

Query Parameters:

text (required): The text to speak

Response: 200 OK with text Speaking: <text>

Side Effects:

Creates a message file so it appears in conversation history
Generates TTS and sends to ESP32 asynchronously

GET /play

Stream a recorded audio file as WAV.

Query Parameters:

f (required): Filename (e.g., msg_12345.raw or msg_12345.reply.raw)

Response: WAV audio with proper headers

POST /delete

Delete a message and all its associated files.

Query Parameters:

f (required): Filename of the .raw file

Deletes: .raw, .txt, .reply.txt, .reply.raw files

POST /reset

Clear the in-memory conversation history.

Response: 200 OK with text conversation reset

Note: Does not delete files from disk - only clears the LLM's context window.

POST /dismiss-alert

Clear the family alert banner on the dashboard.

Response: 200 OK with text alert dismissed

GET /

Web dashboard showing all conversations, shopping list, and alert status. Auto-refreshes every 5 seconds.

ESP32 HTTP Server (:8082)

POST /play

Receive raw PCM audio and play through the speaker.

Request:

Content-Type: application/octet-stream
Body: Raw 16-bit PCM at 16kHz mono
Max size: 480KB (15 seconds)

Response: 200 OK with text ok

Behaviour:

Blocks until playback complete
Sets g_voice_playing = true during playback
Triggers speaking animation on conversation screen

GraphQL Backend (:8080)

Query: Get Events

graphql

{
  events(limit: 20) {
    events {
      deviceName
      payload
      timestamp
    }
    totalCount
  }
}

Response:

json

{
  "data": {
    "events": {
      "events": [
        {
          "deviceName": "Frontdoor",
          "payload": "{\"contact\":true,\"battery\":100}",
          "timestamp": "2026-02-15T14:30:00Z"
        }
      ],
      "totalCount": 1500
    }
  }
}

Mutation: Create Event

graphql

mutation {
  createEvent(deviceName: "Robin", payload: "{\"type\":\"boot\"}") {
    id
  }
}

Subscription: Real-time Events (WebSocket)

Protocol: graphql-transport-ws over WebSocket at ws://192.168.0.164:8080/query

graphql

subscription {
  eventAdded {
    deviceName
    payload
    timestamp
  }
}

WebSocket Handshake:

shell

1. Connect with Sec-WebSocket-Protocol: graphql-transport-ws
2. Send: {"type":"connection_init"}
3. Receive: {"type":"connection_ack"}
4. Send: {"id":"1","type":"subscribe","payload":{"query":"subscription { eventAdded { ... } }"}}
5. Receive: {"id":"1","type":"next","payload":{"data":{"eventAdded":{...}}}} (streaming)

Zigbee Device Payloads

Each sensor reports different JSON payloads:

Door Contact (Frontdoor, Basementdoor)

json

{"contact": true, "battery": 100, "voltage": 3100, "linkquality": 150}

contact: true = closed, false = open

Presence (Presence, Frontdoor2)

json

{
  "occupancy": true,
  "battery": 50,
  "temperature": 21.5,
  "illuminance": 150,
  "illuminance_lux": 80
}

Vibration (vibrartion)

json

{
  "vibration": true,
  "angle_x": -3,
  "angle_y": 5,
  "angle_z": 85,
  "battery": 95
}

Button (button)

json

{"action": "single", "battery": 100}

Actions: "single", "double", "long"

Plant Sensor (plant)

json

{
  "soil_moisture": 45,
  "temperature": 22.3,
  "illuminance": 2000,
  "fertility": 150,
  "battery": 80
}

Bridge (bridge)

json

{"type": "coordinator"}

No sensor data - just the Zigbee network coordinator.

External Services

Whisper ASR (GPU Server :9000)

URL: http://192.168.0.166:9000/asr?encode=true&task=transcribe&language=en&output=json Method: POST multipart form Field: audio_file (WAV) Response: {"text": "transcribed text"}

Ollama LLM (GPU Server :11435)

URL: http://192.168.0.166:11435/api/chat Method: POST JSON Model: qwen2.5:14b

Request:

json

{
  "model": "qwen2.5:14b",
  "messages": [
    {"role": "system", "content": "..."},
    {"role": "user", "content": "..."}
  ],
  "stream": false,
  "tools": [...]
}

Response:

json

{
  "message": {
    "content": "response text",
    "tool_calls": [
      {
        "function": {
          "name": "web_search",
          "arguments": "{\"query\": \"weather London\"}"
        }
      }
    ]
  }
}

Piper TTS (GPU Server :9001)

URL: http://192.168.0.166:9001 Method: POST JSON Request: {"text": "Hello world"} Response: Raw 16-bit PCM at 22050Hz (resampled to 16000Hz by audio-receiver)

ntfy Push Notifications

URL: https://ntfy.sh/jon-home-assistant-alert Method: POST Headers:

Title: Notification title
Priority: urgent or default
Tags: Emoji tags (e.g., warning,rotating_light)

Body: Plain text message

Weather (wttr.in)

URL: https://wttr.in/?format=%l:+%c+%t+%h+humidity+%w+wind+%p+rain Response: One-line weather summary