Chat System

The chat system enables multi-turn AI conversations with tool-calling capabilities. It supports both REST (web UI) and WebSocket (device) interfaces.

Architecture

User Message
     ↓
┌─────────────────────┐
│  Build System Prompt │ ← SOUL.md + MEMORY.md + available actions
└──────────┬──────────┘
           ↓
┌─────────────────────┐
│  Load Message History│ ← Last 20 messages from conversation
└──────────┬──────────┘
           ↓
┌─────────────────────┐
│   Ollama Chat API    │ ← Messages + tools definition
└──────────┬──────────┘
           ↓
    ┌──────┴──────┐
    │ Tool calls? │
    ├── Yes ──────┤
    │  Execute    │ → Action Registry → result
    │  Append     │ → Add tool result to messages
    │  Re-query   │ → Back to Ollama (max 5 rounds)
    ├── No ───────┤
    │  Stream     │ → Response tokens to client
    └─────────────┘
           ↓
┌─────────────────────┐
│  Save to Database    │ ← User message + assistant response
└──────────┬──────────┘
           ↓
┌─────────────────────┐
│ Memory Extraction    │ ← Async, every 5 messages or on "remember"
└─────────────────────┘

Conversation Management

Conversations are per-device. Each device can have multiple conversations.

Key Functions (lib/chat/conversations.ts)

// Create a new conversation
createConversation(deviceId: string, title?: string): Promise<Conversation>

// Add a message to a conversation
addMessage(conversationId: string, role: string, content: string): Promise<Message>

// Get messages with optional limit (default: 20)
getConversationMessages(conversationId: string, limit?: number): Promise<Message[]>

// List all conversations for a device
getDeviceConversations(deviceId: string): Promise<Conversation[]>

// Update conversation title
updateConversationTitle(conversationId: string, title: string): Promise<void>

Auto-Title Generation

When a conversation has no title and reaches 2+ messages, the system asks Ollama to generate a concise title from the conversation content.

System Prompt

The system prompt (lib/chat/system-prompt.ts) is dynamically constructed at each conversation turn:

const prompt = await buildSystemPrompt();

It combines:

1. Soul — The personality definition from data/SOUL.md
2. Memory — Relevant knowledge from data/MEMORY.md
3. Available Actions — A formatted list of tools the AI can call, with descriptions and parameter schemas

This gives the AI awareness of its personality, stored knowledge, and available capabilities.

Tool Calling

When Ollama returns a response with tool_calls, the system executes each action and feeds the results back:

// Simplified flow
let response = await ollama.chatWithTools(messages, tools);

for (let round = 0; round < 5 && response.toolCalls; round++) {
  for (const call of response.toolCalls) {
    const result = await actionRegistry.execute(
      call.function.name,
      call.function.arguments
    );
    messages.push({ role: "tool", content: JSON.stringify(result) });
  }
  response = await ollama.chatWithTools(messages, tools);
}

The 5-round limit prevents infinite tool-calling loops.

Memory Extraction

After certain conversations, the system automatically extracts and stores facts. This runs asynchronously (doesn't block the response).

Triggers

• Every 5 messages in a conversation
• When the user says "remember" in their message

Process (lib/chat/memory-extraction.ts)

1. Send recent messages to Ollama with a structured extraction prompt
2. Parse the response for { category, key, value } entries
3. Upsert each entry into the memoryEntries table
4. Sync the MEMORY.md file

Categories

Extracted memories are categorized into:

• family — People and relationships
• preferences — Likes, dislikes, settings
• events — Important dates and occasions
• notes — General facts
• recent_context — Short-term context

REST Interface (server/routes/chat.ts)

The web UI uses a streaming POST endpoint:

POST /api/chat
Content-Type: application/json

{
  "message": "What's the weather?",
  "conversationId": "uuid",  // optional
  "deviceId": "web-device"   // optional
}

Response: A ReadableStream where the first line is metadata JSON and subsequent lines are content tokens:

{"conversationId":"uuid","messageId":"uuid"}
Based on the current data, the weather is...

WebSocket Interface (lib/ws/handlers/chat.ts)

Device apps send chat.message and receive streaming chat.delta responses. See WebSocket Protocol for message format details.

Key Files

File	Purpose
lib/chat/conversations.ts	Conversation and message CRUD
lib/chat/system-prompt.ts	Dynamic system prompt builder
lib/chat/memory-extraction.ts	AI-powered fact extraction
lib/ws/handlers/chat.ts	WebSocket chat handler
server/routes/chat.ts	REST chat endpoint (streaming)