Skip to main content

Strategy Codegen Progress Timeout and Token Billing

Summary​

Updated strategy code generation so long-running streaming generations are allowed to finish while they continue producing output. The previous 240 second wall-clock cap could abort healthy codegen streams even when code was actively being written. The new behavior fails only after 240 seconds without new output.

Also updated AI usage billing so Stripe usage is based on aggregated model/token cost instead of a per-request minimum. Usage ledger rows still store model, prompt tokens, completion tokens, total tokens, and cost. Stripe meter reporting now aggregates pending cost into whole $0.01 units and leaves later pending events unreported when they exceed the currently billable whole-cent amount.

Reasoning​

A codegen stream that is actively emitting chunks is not stalled, even if it runs longer than four minutes. The timeout should protect against hangs and dead streams, not cut off a healthy strategy file mid-write.

AI cost also needs to track the actual model and token usage. Charging a one-cent minimum for every event effectively turns billing into per-request billing, which is the wrong unit for long and short model calls. Aggregating pending usage before sending Stripe meter events keeps the ledger token-priced while respecting Stripe's integer meter quantity model.

Key Changes​

  • Replaced the strategy generation hard wall-clock timeout with a progress-aware idle timeout.
  • Reset the idle timer on every streamed code chunk.
  • Removed the OpenAI Responses adapter's hard total stream timer and kept the stream idle timeout.
  • Recorded partial usage receipts when a streaming generation fails after producing partial output.
  • Disabled codegen request-count quotas by treating maxStrategyCodegensPerDay: 0 as unlimited and preserving daily spend caps.
  • Removed the arbitrary $0.05 remaining-budget preflight check for strategy codegen.
  • Changed Stripe usage reporting to aggregate pending usage into $0.01 spend units.
  • Updated Billing UI and plan copy to describe token/model cost billing rather than request quotas.
  • Added bot-chat tool coverage cases for get_backtest_decisions and backfill_backtest_decision_stream.

Relevant Snippets​

const STRATEGY_CODEGEN_IDLE_TIMEOUT_MS = 240_000;

generatedCode = await withProgressIdleTimeout(
  (markProgress) => strategyAI.generateCodeWithStream(
    draftCode || description,
    generationContext,
    async (chunk) => {
      markProgress();
      accumulated += chunk;
    },
  ),
  STRATEGY_CODEGEN_IDLE_TIMEOUT_MS,
  `Strategy code generation stalled for ${Math.round(STRATEGY_CODEGEN_IDLE_TIMEOUT_MS / 1000)} seconds without new output.`,
);
const totalCostUsd = pending.reduce((sum, event) => sum + toNumber(event.cost_usd), 0);
const quantity = Math.floor((totalCostUsd + 1e-9) / AI_USAGE_UNIT_USD);
if (quantity < 1) return;

const maxBillableCostUsd = quantity * AI_USAGE_UNIT_USD;
const billable = selectBillablePendingEvents(pending, maxBillableCostUsd);
const reportQuantity = Math.floor((billable.includedCostUsd + 1e-9) / AI_USAGE_UNIT_USD);
if (billable.events.length === 0 || reportQuantity < 1) return;

Validation​

  • pnpm --dir /Users/adam/github/saas-stack/projects/quantago/services/backend test:tools passed: 71/71 tests.
  • pnpm --dir /Users/adam/github/saas-stack/projects/quantago/services/backend typecheck passed.
  • pnpm --dir /Users/adam/github/saas-stack/projects/quantago/services/backend lint passed with warnings only.
  • pnpm --dir /Users/adam/github/saas-stack/projects/quantago typecheck passed across API, frontend, admin, docs, and landing.
  • pnpm --dir /Users/adam/github/saas-stack/projects/quantago lint passed with warnings only.
  • Backend smoke check: https://api.quantago.co/api/health returned {"ok":true}.
  • Billing endpoint smoke check: unauthenticated /api/billing/summary returned 401.
  • Frontend smoke checks: deployed Pages URL and https://app.quantago.co returned 200.

Deployment​

  • Backend deployed successfully.
  • Final backend worker version: 09899c9a-6011-4b57-a307-5013622f2bb7.
  • Frontend deployed successfully.
  • Frontend deployment URL: https://7ee5fd50.quantago-app.pages.dev.