AI Observly

Documentation

Getting Started (For Founders - Use With an AI Coding Agent)

Non-technical founders integrating via an AI coding agent (Replit Agent, Cursor, Claude Code, Lovable, etc.)

Getting Started with AI Observly For Founders Using an AI Coding Agent

If you don't write code yourself but build or maintain your product using an AI coding agent, you don't need to understand the technical details below. Copy the block under "Paste this to your agent" and hand it directly to whichever agent you use; it will locate the right files in your project and adapt the code to fit, rather than copying it in blindly.

Paste this to your agent

Any
Integrate the AI Observly SDK into this codebase to track AI/LLM usage per customer and per feature. Adapt file paths, package manager, and coding style to match this project's existing conventions — don't copy the example code verbatim, use it only as a reference pattern.

1. Install the `ai-observly` package using whichever package manager this project already uses (npm, yarn, or pnpm).

2. Initialize the SDK once, in this project's main entry point or a shared config/lib file, following the reference pattern below. Require an `AI_OBSERVLY_KEY` environment variable for the API key. If a `.env.example` file exists in this project, add the key there too.

3. Find every place in this codebase where a call is made to an LLM/AI provider (OpenAI, Anthropic, Google, Azure, or similar). Immediately after each successful call, add a call to `observly.logUsage(...)`, passing the current customer's ID, a short label for which feature this call belongs to, and the token usage from the provider's response — following the reference pattern below.

4. Wrap each of those AI calls in error handling if they don't already have it, and log failed or rate-limited calls too using the `isError` fields shown in the reference pattern, so failures are visible in the dashboard as well as successes.

5. If you don't already have an AI Observly API key available in this project, ask me for it rather than inventing a placeholder, and store it as an environment variable — never hardcode it.

6. When finished, list exactly which files were changed and where each logging call was added, so I can review the changes.

Reference pattern — installation:
npm install ai-observly
# or: yarn add ai-observly
# or: pnpm add ai-observly

Reference pattern — initialization:
import AIObservly from "ai-observly";

const observly = new AIObservly({
  apiKey: process.env.AI_OBSERVLY_KEY,
  environment: process.env.NODE_ENV || "production",
  appVersion: "v2.1.0",
});

Reference pattern — logging after a successful AI call:
observly.logUsage({
  customerId: currentCustomer.id,
  customerName: currentCustomer.name,
  featureLabel: "doc_summarizer",
  model: completion.model,
  provider: "openai",
  inputTokens: completion.usage?.prompt_tokens || 0,
  outputTokens: completion.usage?.completion_tokens || 0,
  cachedInputTokens: completion.usage?.prompt_tokens_details?.cached_tokens || 0,
  latencyMs,
  finishReason: completion.choices[0]?.finish_reason || "stop",
  sessionId: req.body.sessionId,
  environment: process.env.NODE_ENV,
});

Reference pattern — logging a failed AI call:
observly.logUsage({
  customerId: currentCustomer.id,
  featureLabel: "doc_summarizer",
  model: "gpt-4o",
  isError: true,
  statusCode: error.status || 500,
  errorCode: error.code || "api_error",
  latencyMs: Date.now() - startTime,
});

Available options for initialization (AIObservlyOptions): apiKey (required), baseUrl (custom collector endpoint, optional), timeoutMs (default 3000), environment, appVersion, debug.

Available fields for logUsage (LogUsageParams): customerId (required), featureLabel (required), model (required), inputTokens, outputTokens, customerName, cachedInputTokens, latencyMs, provider, isError, statusCode, errorCode, finishReason, environment, version, sessionId.

After your agent finishes

Ask your agent to trigger one real AI call in your product (or trigger one yourself through normal use), then check your AI Observly dashboard; a usage entry should appear for that call within moments. If nothing shows up, ask your agent to double-check that the API key environment variable is actually set and that logUsage is being called after a successful provider response, not before it.

    Getting Started (For Founders - Use With an AI Coding Agent) | AI Observly Docs