Respan is an AI development platform integrating model gateways, observability, and evaluation, suitable for engineering and product teams that need to connect multiple models, trace agent errors, and manage costs. Developers can point existing SDKs to a unified endpoint, or install the Respan SDK to record model calls, tool execution, and agent steps into a multi-layered inspectable trace tree.
Respan is an AI development platform integrating model gateways, observability, and evaluation, suitable for engineering and product teams that need to connect multiple models, trace agent errors, and manage costs. Developers can point existing SDKs to a unified endpoint, or install the Respan SDK to record model calls, tool execution, and agent steps into a multi-layered inspectable trace tree.
It places request routing, fallback, caching, and budget limits in one place, and can also create test datasets from production sampling, combining model-based evaluation, programmatic checks, or human review to compare versions. In addition to monitoring error rates, latency, and usage, teams can use natural language to define behaviors to watch for, return to raw logs to confirm issues, and make manual corrections for misjudgments.
Features
Unified model gateway: Access over a thousand models through a single endpoint, using existing SDKs while switching model providers.
Automatic fallback and retry: Configure alternative models to try when errors or rate limits are encountered, with automatic retry capabilities listed separately in the pricing table.
Request and response caching: Store requests and responses so identical calls directly use the cache, reducing repetitive processing costs and latency.
Budget and rate limits: Set budgets and request limits by API key, customer, or entire organization, with notifications and request blocking when thresholds are approached.
Agent workflow tracing: Chain model calls, tool operations, and agent steps into a trace tree to view inputs, outputs, execution time, usage, and cost for each step.
Monitoring dashboard and alerts: Analyze requests, errors, costs, and latency by model, key, or user, with notifications via Slack, email, or webhook when thresholds are exceeded.
Evaluation datasets: Extract real requests from logs based on conditions and sampling rates, or upload CSVs to create test cases close to actual usage scenarios.
Multi-method scoring and version comparison: Combine model-based evaluation, programmatic checks, and human review to side-by-side compare score distributions of different models or prompt versions.
Production sampling evaluation: Apply evaluators to production logs, filter and sample by status, customer, or conversation, and continuously observe quality changes.
Behavior detection and manual correction: Detect refusals, instruction violations, incomplete answers, and other behaviors, with custom definitions and overrides for misjudgments; requires Team plan or above.
Prompt management: Provides prompt version control, publishing management, model testing, and template functions, with different prompt quantity limits depending on the plan.
Use cases
When operating customer service agents connecting to multiple models, set model fallbacks and usage limits, then use trace trees to pinpoint failure steps and cost sources.
When preparing to change models or prompts, create datasets from production requests, compare version scores, and continue sampling quality checks after launch.
When checking whether agents violate output formats or miss requirements, set behavior definitions, filter triggered logs, and have personnel correct judgments.
Categories: ,AI Resources, Tool Resources
Pricing
Free plan; Team annual subscription at $199/month; Enterprise plan available upon inquiry (currency not specified on official website)