Skip to content

AI setup

Overview

  • Leverages Vercel AI SDK for an interactive portfolio analysis assistant in both frontend and backend.
  • Users chat with an AI that can query on-chain data, run SQL via tools, and fetch current market info via web search.

Providers & SDKs

  • Vercel AI SDK (ai, @ai-sdk/react, @ai-sdk/ui-utils): powers chat streaming and UI hooks in the frontend.
  • OpenAI (@ai-sdk/openai v1.3.22): used for chat completions and tool-enabled search via createOpenAI().responses(modelId) and tools.webSearchPreview().
  • Anthropic (@ai-sdk/anthropic v1.2.12): language models via createAnthropic({ apiKey }).languageModel(modelId).
  • Perplexity (@ai-sdk/perplexity v1.1.9): language models via createPerplexity({ apiKey }).languageModel(modelId).

Frontend Integration

  • Assistant Page: packages/frontend/src/pages/AssistantPage/AssistantPage.tsx sets up tabs and routes.
  • Chat UI: AssistantChat.tsx uses useChat from @ai-sdk/react to manage messages, stream responses, and handle stops.
  • State Management: nanostores ($assistantModel, $assistantMode, $activeAccount) control model selection and account context.
  • UI Components: model selector (AssistantModelSelect.tsx), settings (AssistantSettings.tsx), message toolbar and history components.

Backend / API Integration

  • Assistant HTTP API: packages/backend/src/api/account/assistant-http-api.ts exports handleAssistantChat.
    • Endpoint: POST /assistant-chat
    • Request: JSON { accountName, id (conversationId), modelId, messages: Message[] }.
    • Response: chunked streaming (Transfer-Encoding: chunked) via streamText from Vercel AI SDK.
  • Business Logic:
    • Fetch encrypted API key (getApiKey) from KV store.
    • Select model and provider options based on model.family and model.capabilities.
    • Assemble system prompt (getAssistantSystemPrompt in settings/assistant.ts).
    • Initialize tools (getAssistantTools) for on-chain data access.
  • Chat Persistence: assistant-api.ts provides upsert and query functions (upsertChatMessage, getChatHistoryByConversation).

Environment Variables & Configuration

  • Backend Service:
    • JWT_SECRET: decrypts per-account AI keys stored in KV.
    • PORT: HTTP server port (default 5555).
  • Provider Credentials (per account, stored encrypted in KV):
    • assistant_openai_key (prefixed sk-)
    • assistant_anthropic_key (prefixed sk-ant-)
    • assistant_perplexity_key (prefixed pplx-)

Prompt & Model Management

  • System Prompt: template in packages/backend/src/settings/assistant.ts via getAssistantSystemPrompt, includes timestamp and tool descriptions.
  • Models: definitions in packages/backend/src/settings/assistant-models.ts, listing IDs, families, capabilities, context windows, and cost per 1k tokens.
  • Metadata: each chat message stores modelId, usage, finishReason, and prompt context in JSON metadata.

Development & Testing

  • Local Setup: run yarn dev (frontend & backend) or yarn dev:electron for desktop.
  • AI Keys: load provider API keys into KV using the AuthSecrets mechanism and assistant_*_key entries.
  • Testing:
    • Frontend: packages/frontend uses Vitest and mocks useChat as needed.
    • Backend: run vitest in packages/backend, stub streamText or mock createOpenAI/createAnthropic calls.

Security & Cost Considerations

  • Store AI keys encrypted; never log raw keys.
  • Monitor token usage via metadata (usage returned by streamText) and use cost settings in assistant-models.ts.
  • Choose smaller, cost-efficient models (o4-mini, gpt-4.1-mini) when high throughput is needed.
  • Rate limits and authentication enforced via JWT and per-account secrets; no built-in rate throttling in AI API layer—implement externally if needed.