Documentation

The Building Blocks

Every HubUI integration is built from these components. The Agent is the autonomous core — everything else connects to it.

How Conversations Actually Work

There are two paths a conversation can take. Understanding this distinction is the most important concept in HubUI.

Two Ways to Connect Your Users

PAYG Billing

HubUI uses one prepaid PAYG credit balance. Voice minutes, AI response turns, and phone numbers draw down from the same balance with no monthly plan selection.

Full rate and balance details are available on the Billing page in your dashboard.

Step 1 — Create an Agent

The Agent is your autonomous AI persona. It handles all conversation in real-time — greetings, Q&A, follow-ups, multi-turn dialogue — using the knowledge you give it in System Instructions.

1. Navigate to the Agents page → Click "Create Agent".
2. Set a name, pick a voice, and select your language.
3. Write your System Instructions. This is where you define your Agent's personality, knowledge base, and rules. Put everything the Agent needs to know right here — company info, FAQs, product details, policies, how to greet callers, what to say when it doesn't know something.
4. Click Save. Your Agent is now live and ready to answer questions based on the knowledge you provided.

Step 2 — Test It

Test your Agent right from the dashboard — no frontend app required.

Option A — Embed in Your App

Use the Client SDK to let users talk or chat with your Agent directly from your website or web app.

1. Get a Client API Key — go to Settings → API Keys and generate one. It starts with pk_live_....
2. Copy your Agent ID from the Agents page.
3. Install and connect:

npm install @hubui/client

import { HubUI } from '@hubui/client';

const session = await HubUI.connect({
  agentId: 'your-agent-id',
  apiKey:  'pk_live_xxxxx',
  mode:    'voice',            // or 'text'
  userName: 'John Doe',
});

// Listen for real-time transcripts
session.on('transcript', (text, speaker, isFinal) => {
  console.log(`${speaker}: ${text}`);
});

// When done
await session.disconnect();

See the Client SDK section for text mode, error handling, mobile audio, and the full API reference.

Option B — Add a Brain for Deep Reasoning

A Brain is your custom Python backend that gives your Agent superpowers — LLM inference (OpenAI, Gemini, Claude), database lookups, REST API calls, LangChain/LangGraph pipelines, or any async logic. The Agent calls your Brain on-demand, only when your System Instructions tell it to.

1. Go to the Brain page → Click "Create Brain". Enter a name (e.g., order_lookup) and your Brain's WebSocket URL.
2. Copy the Brain API Key (br_...) — shown only once.
3. Build your Brain server:

pip install hubui-brain

# brain.py
from hubui_brain import Brain, QueryContext

brain = Brain(api_key="br_xxx", debug=True)

@brain.on_query()
async def handle(ctx: QueryContext):
    # Your custom logic here — LLM call, DB lookup, API request, etc.
    await ctx.reply(f"You asked: {ctx.query}")

brain.run(host="0.0.0.0", port=8080)

4. In your Agent's Brain tab, select your Brain from the dropdown.
5. Critical: Update your Agent's System Instructions to define exactly when the Agent should call the Brain — what types of questions, conditions, or triggers should route to it. For example: "When the user asks about order status, tracking, or returns, use the order_lookup brain to retrieve the information. For all other questions, answer directly." Just selecting the Brain in the dropdown is not enough.

See the Brain SDK section for heartbeats, auto_process(), deployment guides (Cloud Run & Docker), and the full API reference.

Option C — Add a Phone Number

Let callers dial in and talk to your Agent over regular phone lines — same personality, same knowledge, same Brain (if connected).

1. Go to the Phone Numbers page → Click "Buy Number".
2. Choose a number and assign it to your Agent.
3. Callers can now dial that number and talk to your Agent immediately.

See the Phone Numbers section for area code selection, SIP trunking, and number management.

Next Steps

Your Agent is up and running. Here's where to go deeper:

Settings Page

Manage your profile, credentials, and API keys.

API Key Types Explained

HubUI uses two distinct API keys with different purposes and security levels. Understanding the difference is critical.

Create a Brain

Navigate to the Brain page in the sidebar to establish a backend connection.

1. Click "Create Brain".
2. Enter a name — 1–16 characters, lowercase letters, numbers, and underscores only (^[a-z0-9_]+$). Example: orders_db.
3. Enter your Brain URL — the WebSocket endpoint where your Brain server listens (e.g., wss://my-brain.example.com).
4. Secondary URL (optional) — a fallback endpoint if the primary URL is unreachable.
5. Description (optional) — a note for your own reference.
6. Scope — choose "All Agents" or select specific agents that should have access to this Brain.
7. Click Create. Copy the Brain API Key (br_...) — it is shown once. Save it securely.

Tab 1 — Basic Settings

Voice Catalog — Premium HD Voices

Select one of 7 premium HD voices. Each voice has a distinct personality and gender. Voice selection is made on the Basic tab.

Tab 2 — Voice Configuration

Adjust speech speed with a slider. A preview card shows the selected voice's name, gender, language, and personality description.

Step size is 0.05x. For example, valid values include 0.50, 0.55, 0.60, … 1.45, 1.50.

Tab 3 — Call Behavior

Control how your Agent handles silence, goodbyes, and runaway calls.

Tab 4 — System Instructions

System Instructions are the single most important configuration for your Agent. They define its personality, knowledge, rules, and — critically — when it should call your Brain.

Tab 5 — Brain Integration

Connect your Agent to a Brain backend for on-demand custom logic.

Tab 6 — Phone Numbers

Available only in edit mode (not when creating a new Agent). Lists all phone numbers currently attached to this Agent with a Detach button on each. Detaching moves the number to the unassigned pool — it stays in your account and can be re-attached to any Agent later.

In create mode, this tab shows a "Save Agent First" placeholder. Attach numbers from the Phone Numbers page after creating the Agent.

Agent Status Management

Every Agent has one of three statuses. Change status from the Agent card's dropdown menu on the Agents page.

Status changes also update the phone number routing index — pausing or disabling an Agent stops its phone numbers from accepting calls immediately.

Deleting an Agent

Test Call

Test any active Agent directly from the dashboard. Click the Test button on any Agent card to open the Test Call modal.

Installation

Requires Python 3.10+, a Brain API key from the dashboard, and a Brain URL configured in your Agent.

# Install from PyPI (recommended)
pip install hubui-brain

# Or install from source
cd hubui-brain && pip install -e .

# Or install directly from GitHub
pip install "git+https://github.com/HubUI-AI/hubui-brain.git"

Quick Start — Manual Heartbeats

You control exactly when to send follow-up status updates during longer operations.

from hubui_brain import Brain, QueryContext

brain = Brain(api_key="br_xxx", debug=True)

@brain.on_query()
async def handle_query(ctx: QueryContext):
    answer = await my_reasoning_engine.process(ctx.query)
    await ctx.reply(answer)

brain.run(host="0.0.0.0", port=8080)

Note: You don't need to send an initial acknowledgment — the Voice Agent automatically speaks one (e.g. "Sure, let me check on that") before your Brain even receives the query.

Quick Start — Auto Heartbeats

Configure timing thresholds on Brain and let the SDK send follow-up updates automatically via auto_process().

from hubui_brain import Brain, QueryContext

brain = Brain(
    api_key="br_xxx",
    heartbeat_after=5.0,       # Follow-up status after 5 seconds
    hold_music_after=15.0,     # Hold music after 15 seconds
    debug=True
)

@brain.on_query()
async def handle_query(ctx: QueryContext):
    # SDK monitors elapsed time and sends heartbeats automatically
    answer = await ctx.auto_process(
        my_reasoning_engine.process(ctx.query)
    )
    await ctx.reply(answer)

brain.run(host="0.0.0.0", port=8080)

If heartbeat_after and hold_music_after are both None (the default), auto_process() runs your coroutine directly with zero monitoring overhead.

Instant Acknowledgment & Heartbeats

0s   → Caller asks a question
       Voice Agent instantly acknowledges
       Your Brain starts processing
5s   → Brain sends heartbeat: "Still working on that..."
15s  → Brain starts hold music: "Please hold."
20s  → Coroutine returns → ctx.reply(result)

result = await ctx.auto_process(
    process_query(ctx.query),
    heartbeat_after=7.0,       # Override Brain default
    hold_music_after=20.0,     # Override Brain default
    heartbeat_message="Checking a few more sources...",
    hold_music_message="Still working — thanks for your patience."
)

@brain.on_query()
async def handle(ctx: QueryContext):
    data = await fetch_from_database(ctx.query)
    if needs_extra_time(data):
        await ctx.say_processing("Still looking — checking a few more options.")
    if needs_even_more_time(data):
        await ctx.play_hold_music("This is taking a bit longer. Please hold.")
    result = await my_reasoning_layer(ctx.query, data)
    await ctx.reply(result)

Brain Constructor Reference

brain.run() vs brain.start()

brain.run(host="0.0.0.0", port=8080)

import asyncio

async def main():
    await brain.start(host="0.0.0.0", port=8080)

asyncio.run(main())

QueryContext — Properties

Passed to your handler on every query. Provides the request data and all response methods.

QueryContext — Response Methods

Methods are either non-final (can be called multiple times) or final (ends the turn — call exactly once).

auto_process() Method Reference

result = await ctx.auto_process(
    coroutine,
    heartbeat_after=None,      # Override Brain default (seconds)
    hold_music_after=None,     # Override Brain default (seconds)
    heartbeat_message=None,    # Override Brain default
    hold_music_message=None    # Override Brain default
)

Wraps your coroutine and automatically sends heartbeat/hold music based on elapsed time. Per-call arguments override Brain-level defaults. If no timing is configured anywhere, the coroutine runs directly with zero overhead.

Capability Checking

The Voice Agent advertises which tools it supports in each request. Always check before attempting to use a tool — _terminate_call is only available on SIP (phone) calls, not WebRTC.

# Check a specific tool
if ctx.has_tool("_terminate_call"):
    await ctx.terminate_call("Goodbye!")
else:
    await ctx.reply("Goodbye!")  # Graceful fallback

# Inspect all available tools
for tool in ctx.available_tools:
    print(f"{tool.name}: {tool.description}")

ConversationTurn

Each item in ctx.history is a ConversationTurn:

for turn in ctx.history:
    print(f"{turn.role}: {turn.content}")  # role: "user" or "assistant"
    print(f"  at: {turn.timestamp}")       # ISO 8601 string, or None

API Key Setup

Your Brain API key is generated automatically when you create a Brain in the dashboard. Provide it via constructor or environment variable:

# Option 1: Environment variable (recommended — keeps key out of source code)
# export HUBUI_BRAIN_API_KEY=br_xxx
brain = Brain()

# Option 2: Pass directly
brain = Brain(api_key="br_xxx")

Lookup order: api_key constructor param → HUBUI_BRAIN_API_KEY env var. The key is included in every WebSocket response for validation. Treat it like a password — never commit to source control.

Deployment

Your Brain is a standard async Python WebSocket server. It runs on any platform that supports long-lived connections.

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "main.py"]

docker build -t my-brain .
docker run -p 8080:8080 -e HUBUI_BRAIN_API_KEY=br_xxx my-brain

Examples

from hubui_brain import Brain, QueryContext

brain = Brain(debug=True)

@brain.on_query()
async def echo(ctx: QueryContext):
    await ctx.reply(f"You said: {ctx.query}")

brain.run(port=8080)

from hubui_brain import Brain, QueryContext
from openai import AsyncOpenAI

brain = Brain(debug=True)
client = AsyncOpenAI()

@brain.on_query()
async def handle(ctx: QueryContext):
    await ctx.say_processing("Still working on that — almost there.")

    messages = [{"role": t.role, "content": t.content} for t in ctx.history]
    messages.append({"role": "user", "content": ctx.query})

    response = await client.chat.completions.create(
        model="gpt-4o", messages=messages
    )
    await ctx.reply(response.choices[0].message.content)

brain.run(port=8080)

from hubui_brain import Brain, QueryContext
from openai import AsyncOpenAI

brain = Brain(heartbeat_after=5.0, hold_music_after=15.0, debug=True)
client = AsyncOpenAI()

@brain.on_query()
async def handle(ctx: QueryContext):
    messages = [{"role": t.role, "content": t.content} for t in ctx.history]
    messages.append({"role": "user", "content": ctx.query})

    async def call_openai():
        resp = await client.chat.completions.create(
            model="gpt-4o", messages=messages
        )
        return resp.choices[0].message.content

    result = await ctx.auto_process(call_openai())

    if should_end_call(result):
        if ctx.has_tool("_terminate_call"):
            await ctx.terminate_call(result)
            return

    await ctx.reply(result)

brain.run(port=8080)

from hubui_brain import Brain, QueryContext
from langgraph.graph import StateGraph

brain = Brain(debug=True)
graph = StateGraph(...)
agent = graph.compile()

@brain.on_query()
async def handle(ctx: QueryContext):
    await ctx.say_processing("Still pulling up the details...")

    result = await agent.ainvoke({
        "query": ctx.query,
        "history": [(t.role, t.content) for t in ctx.history],
        "caller": ctx.caller_number,
        "call_source": ctx.call_source,
    })
    await ctx.reply(result["response"])

brain.run(port=8080)

Installation

# From npm (recommended)
npm install @hubui/client
# or
yarn add @hubui/client

# From GitHub repository
npm install github:HubUI-AI/hubui-client

# Explicit Git URL
npm install git+https://github.com/HubUI-AI/hubui-client.git

Prerequisites

Quick Start — Voice Mode

import { HubUI } from '@hubui/client';

// Connect to your agent in voice mode
const session = await HubUI.connect({
  agentId: 'your-agent-id',      // From HubUI Dashboard
  apiKey: 'pk_live_xxxxx',        // From Dashboard > Settings > API Keys
  mode: 'voice',
  userName: 'John Doe',           // Optional
});

// Listen for real-time transcripts
session.on('transcript', (text, speaker, isFinal) => {
  console.log(`${speaker}: ${text}`);
});

// Handle connection state changes
session.on('connectionStateChanged', (state) => {
  console.log('Connection:', state);
});

// Mute/unmute controls
session.mute();
session.unmute();

// Disconnect when done
await session.disconnect();

Quick Start — Text Mode

import { HubUI } from '@hubui/client';

// Connect to your agent in text mode
const session = await HubUI.connect({
  agentId: 'your-agent-id',
  apiKey: 'pk_live_xxxxx',
  mode: 'text',
  userName: 'John Doe',
});

// Listen for agent responses
session.on('message', (text) => {
  console.log('Agent:', text);
});

// Send messages
await session.send('Hello! How can you help me?');

// Disconnect when done
await session.disconnect();

HubUI.connect(config) — Full Reference

Connects to a HubUI agent and returns a Promise<HubUISession>. Throws a HubUIError if connection fails.

* apiKey is required unless you provide token + serverUrl. The key must start with pk_live_.

HubUI.getAudioDevices()

Static method that requests microphone permission and lists all available audio devices. Use the returned deviceId values in audioInput / audioOutput config, or in setAudioInput() / setAudioOutput().

const { inputs, outputs } = await HubUI.getAudioDevices();

// inputs:  [{ deviceId, label, kind: 'audioinput' }, ...]
// outputs: [{ deviceId, label, kind: 'audiooutput' }, ...]

console.log('Microphones:', inputs);
console.log('Speakers:', outputs);

// Pass to connect config
const session = await HubUI.connect({
  agentId: 'your-agent-id',
  apiKey: 'pk_live_xxxxx',
  mode: 'voice',
  audioInput: inputs[0].deviceId,
  audioOutput: outputs[0].deviceId,
});

HubUISession — Properties

{
  id: string;                         // Unique entry ID
  speaker: 'user' | 'agent';         // Who spoke
  text: string;                       // The transcribed or sent text
  timestamp: Date;                    // When this entry was created
  isFinal: boolean;                   // true = final, false = interim
}

HubUISession — Methods

HubUISession — Events

Mobile Browser Support

iOS Safari and Android Chrome require a user gesture (tap/click) to play audio. The SDK emits audioPlaybackBlocked when the browser's autoplay policy blocks the agent's voice. Show a UI button and call enableAudio():

session.on('audioPlaybackBlocked', () => {
  // Show a button prompting the user to tap
  showButton('Tap to enable audio', async () => {
    await session.enableAudio();
    hideButton();
  });
});

The SDK also attempts to unlock audio automatically during connect() by playing a silent buffer, but this may not work on all browsers — always handle the event as a fallback.

Error Handling

All errors are instances of HubUIError with a code property from the ErrorCodes enum.

import { HubUI, HubUIError, ErrorCodes } from '@hubui/client';

try {
  const session = await HubUI.connect(config);
} catch (error) {
  if (error instanceof HubUIError) {
    switch (error.code) {
      case ErrorCodes.INVALID_API_KEY:
        console.error('Invalid API key');
        break;
      case ErrorCodes.INVALID_AGENT_ID:
        console.error('Agent not found');
        break;
      case ErrorCodes.MICROPHONE_ACCESS_DENIED:
        console.error('Microphone permission denied');
        break;
      default:
        console.error(`Error [${error.code}]: ${error.message}`);
    }
  }
}

All Error Codes

TypeScript Support

Full TypeScript definitions are bundled with the package — no separate @types/ install needed.

import type {
  HubUIConfig,
  HubUIConnectionState,
  HubUITranscriptEntry,
  HubUIEvents,
  AudioDevice,
} from '@hubui/client';

import { HubUI, HubUISession, HubUIError, ErrorCodes, SDK_VERSION } from '@hubui/client';
import type { ErrorCode } from '@hubui/client';

Browser Support

WebRTC support is required. Older browsers without WebRTC will fail at connection time with CONNECTION_FAILED.