AI Capabilities Overview

A default installation of zhin.js only sends and receives messages -- it does not "converse." To make it understand #check tomorrow's weather, you need to add the AI layer: install a few packages, write an ai: section in your config, and the CLI will automatically assemble the Agent Host at startup, routing messages that don't match any command to ZhinAgent for processing.

Installation

bash

pnpm add @zhin.js/agent zod ai
pnpm add @ai-sdk/openai   # Install the provider SDK for the vendor you use

The provider SDK is an optional peer dependency -- install whichever sdk package you use; if missing, the runtime will report Missing peer dependency with the install command. On the code side, import ZhinAgent, AIService, etc. from zhin.js/agent (or @zhin.js/agent).

Configuration skeleton

All configuration lives in the top-level ai: section of zhin.config.yml; ${VAR} expands from environment variables:

yaml

ai:
  providers:        # Vendor connections (required)
    openrouter:
      sdk: openai-compatible
      baseUrl: ${OPENROUTER_BASE_URL}
      apiKey: ${OPENROUTER_API_KEY}
  agents:           # Agent bindings (at least zhin is required)
    zhin:
      provider: openrouter
      model: openrouter/free
  mcpServers: []    # MCP client connections (optional)
  agent:
    inboundQueue:
      groupMode: supersede # default; use fifo to wait in arrival order
  trigger: {}       # Trigger rules (optional)
  access: {}        # Access gating (optional)

At startup, the ai: section undergoes soft-prune: providers whose credentials expand to empty are removed (logged at debug level only); agents bound to removed providers are skipped along with them. If the zhin binding is missing, the Agent Host is not assembled, but IM startup is unaffected.

ai.agent.inboundQueue.groupMode is also the default canonical Turn Intent policy: supersede preserves replacement compatibility, while fifo resolves ordinary ingress as new and waits in arrival order. Adapters, commands, or product policy still resolve steer, follow_up, and observe explicitly. Cross-participant steer and follow_up authorization may only come from the Host's trusted resolveTurnIntent policy; message metadata cannot self-assert it. Tools remain constrained by the active turn's authority, while the journal separately records the controlling principal and turn ID.

ai.providers

Each provider is named by alias; sdk determines which AI SDK adapter to use (closed set, ADR 0018):

sdk	Peer dependency	Key config
`openai`	`@ai-sdk/openai`	`apiKey`, optional `baseUrl`
`openai-compatible`	`@ai-sdk/openai-compatible`	`baseUrl` + `apiKey` (`baseUrl` auto-appends `/v1`); Cloudflare Workers AI can use `accountId` instead of `baseUrl`
`anthropic`	`@ai-sdk/anthropic`	`apiKey`, optional `baseUrl` (auto-appends `/v1`)
`deepseek`	`@ai-sdk/deepseek`	`apiKey`, optional `baseUrl`
`google`	`@ai-sdk/google`	`apiKey`, optional `baseUrl` (auto-appends `/v1beta`)
`ollama`	`@ai-sdk/openai-compatible` (uses OpenAI-compatible interface internally)	`host` (default `http://127.0.0.1:11434`, auto-appends `/v1`), no `apiKey` needed

Common fields: models (explicit model allowlist), contextWindow, imageGeneration (text-to-image defaults), and input (the explicit accepted modalities: text | image | audio | video | file). When input is omitted it fails closed to ['text']; model names and vendors are never used to guess vision/audio/video support.

When models is not specified, ModelRegistry calls /v1/models in the background after startup to auto-populate the available model list (restoring the previous cache first, then refreshing asynchronously). When models is specified, the YAML allowlist takes precedence. agents.<name>.model only needs to appear in the discovered list or allowlist -- no need to list each one manually.

ai.agents

Bind an agent name to a provider + model:

yaml

ai:
  agents:
    zhin:                       # Main Agent, must exist
      provider: openrouter
      model: openrouter/free
      mcpServers: [icqq]        # MCP servers mounted on this agent (referenced by name)
      nickname: Zhi             # LLM self-reference and UI display name
    planner:
      provider: openrouter
      model: openrouter/free
      match:                    # Optional routing rules (can be an array)
        scene: group
      permission:               # Sub-agent types visible to spawn_task
        task: { "researcher": allow, "*": deny }

Field	Description
`provider` / `model`	Reference to the `ai.providers` alias and model id (required)
`mcpServers`	List of `name` references from `ai.mcpServers`
`nickname`	Agent self-reference and UI display name
`match`	Routing rules: `adapter` / `endpoint` / `scene` / `sceneId` / `hasMedia` / `contentContains`
`permission.task`	glob -> `allow` / `deny`, constraining which sub-agents `spawn_task` can dispatch

ai.mcpServers

Register MCP (Model Context Protocol) servers. Candidate generation activation connects every configured server before publication; ready tools enter the owner-scoped capability index. Requires the peer dependency @modelcontextprotocol/sdk.

yaml

ai:
  mcpServers:
    - name: icqq
      transport: streamable-http      # stdio | streamable-http | sse
      url: ${ICQQ_MCP_URL}
      headers:
        Authorization: Bearer ${ICQQ_MCP_TOKEN}
    - name: filesystem
      transport: stdio
      command: npx
      args: ["-y", "@modelcontextprotocol/server-filesystem", "/tmp/zhin-mcp-test"]

A configured server must become ready while the candidate generation activates. Failure aborts that candidate and leaves the previous generation serving traffic (fail-closed). Agent bindings may reference only servers declared here; unknown names fail configuration validation.

Trigger rules (ai.trigger)

Messages that don't match any command are evaluated in a fixed order to determine whether they should be handed to AI:

Ignore prefixes: If matched by ignorePrefixes (default /, !, !), skip immediately to avoid conflicts with commands;
Prefix trigger: If matched by prefixes (default #, AI:, ai:), the text after the prefix is taken (works in both direct messages and groups);
@ trigger: In groups/channels, if respondToAt !== false and the adapter has flagged mentioned (e.g. CQ code, app_mention);
Direct message pass-through: When respondToPrivate !== false, the full text of direct messages goes straight to AI;
Keywords: Direct messages only, matches any substring in keywords.

yaml

ai:
  trigger:
    prefixes: ["ai:"]
    respondToAt: true        # Default true
    respondToPrivate: true   # Default true
    timeout: 60000           # Single turn timeout (ms)
    thinkingMessage: Thinking...  # Optional: placeholder reply before entering AI
    errorTemplate: "❌ AI processing failed: {error}"
    masters: ["10001"]       # Trigger-level masters (participate in Owner approval)
    trusted: []              # Trigger-level trusted (weaker than master)

Access gating (ai.access)

Controls which conversations the LLM reply path is open to (useful for platform AIGC compliance scenarios):

yaml

ai:
  access:
    mode: whitelist     # open (default) | closed | whitelist
    users: ["10001"]    # Allowlisted sender.id
    groups: ["123456"]  # Allowlisted group/channel id
    denyMessage: AI features are not enabled for this conversation.

Direct message denials reply with denyMessage; group/channel denials are silent.

Security policies (ai.agent)

Execution-type tools like bash are subject to defense-in-depth constraints under ai.agent:

yaml

ai:
  agent:
    execSecurity: allowlist       # deny | allowlist (default) | full
    execPreset: readonly          # readonly | network | development | custom
    execAllowlist: [make]         # Allowlist when execPreset=custom
    execApprovalMode: ask         # ask (default) | allow | deny
    gitStatus: true               # Default true: inject a one-line git status summary into the Runtime section (skipped outside git repos)
    contextPaths: []              # Extra context files injected into the system prompt (supports ~ and relative paths)
    systemPromptMaxChars: 100000  # Total system prompt char budget; truncatable sections are cut in sacrifice order when exceeded

In addition to contextPaths, the global context files ~/.config/zhin/AGENTS.md and ~/.config/zhin/ZHIN.md are loaded by default (skipped when missing) and injected as a # User Context section before the project bootstrap context; per-file cap is 8KB and the total cap is 16KB.

execPreset preset allowlists widen progressively: readonly (ls/cat/grep/find etc.) -> network (adds curl/wget/ping etc.) -> development (adds npm/node/git/python etc.). Regardless of the mode, dangerous commands like sudo, eval, dd, export are always rejected, and operations like rm -rf node_modules are hard-blocked.

The full check chain is: dangerous blocklist -> environment variable prefix stripping (FOO=bar cmd matches against cmd) -> wrapper stripping (timeout 10 cmd) -> compound command splitting (&&/| checked segment by segment) -> non-full mode rejects newlines / $(...) / backticks -> read-only commands auto-approved. When execApprovalMode: ask, commands exceeding permissions trigger Owner approval, where a master approves via /approve in IM; allow approves everything, deny rejects everything.

Next steps

Agent deep dive: ZhinAgent turns, deferred tools, sub-agents, orchestration, sessions & compaction
Voice capabilities: STT / TTS and voice_stt / voice_tts tools
Console: Observe agent sessions and orchestration runs in the web console

AI Capabilities Overview ​

Installation ​

Configuration skeleton ​

ai.providers ​

ai.agents ​

ai.mcpServers ​

Trigger rules (ai.trigger) ​

Access gating (ai.access) ​

Security policies (ai.agent) ​

Next steps ​

AI Capabilities Overview

Installation

Configuration skeleton

ai.providers

ai.agents

ai.mcpServers

Trigger rules (ai.trigger)

Access gating (ai.access)

Security policies (ai.agent)

Next steps