AI SaaS Starter: Building a Metered AI Product with Streaming Chat and Credit Billing

Introduction to the AI SaaS Starter

The AI SaaS Starter is purpose-built for developers building AI-powered products with usage-based pricing. Unlike traditional SaaS applications that charge per seat or per month, AI products need metered billing that charges based on actual usage — each AI request consumes tokens, compute time, and API costs that vary with response length and complexity. The AI SaaS Starter solves this problem with a production-ready architecture that includes streaming chat responses using the Anthropic Claude SDK, a credit-based metered billing system with reserve-then-true-up accounting, per-user rate limiting to prevent a single user from exhausting your API budget, an append-only credit ledger for complete audit trail, and Stripe integration for subscription management and metered billing. In this comprehensive guide, we will explore the architecture, implementation, and operational considerations for building an AI SaaS product on the AI SaaS Starter. We will cover the streaming chat implementation, the credit accounting system, the rate limiting strategy, the Stripe metered billing integration, and the deployment considerations for AI applications.

Why Metered Billing for AI Products

Metered billing is the right pricing model for AI products for several fundamental reasons. First, AI usage is inherently variable: a single user might send ten short queries one day and one long, complex analysis the next. A flat per-seat price either leaves money on the table from heavy users or prices out light users. With metered billing, every user pays in proportion to the value they receive. Second, AI API costs are variable and unpredictable at the individual request level. A credit-based system with reserve-then-true-up accounting ensures that you never let a request through that you cannot afford, because the credits are reserved from the user's balance before the API call is made. If the reservation fails due to insufficient credits, the request is rejected before incurring any cost. Third, metered billing aligns incentives correctly: users who need more AI processing power can simply add more credits, and users who only need occasional help pay a lower base price. The AI SaaS Starter implements this with a credit ledger that records every grant and consumption event, providing complete transparency for both the business and the customer. The Stripe integration supports both prepaid credit purchases and postpaid metered billing, depending on the business model.

Streaming Chat Implementation

The streaming chat implementation in the AI SaaS Starter delivers a superior user experience by displaying AI responses as they are generated, rather than making users wait for the complete response. The implementation uses the Anthropic SDK's streaming API with Server-Sent Events over a Next.js API route. When a user sends a message, the client creates an AbortController for cancellation support and sends a POST request to the chat API endpoint. The server validates the user's session and credit balance, reserves the estimated credits for the request, initializes a streaming connection to the Anthropic API, and begins streaming token chunks back to the client. Each chunk is sent as a server-sent event with the token text and metadata. The client processes these events incrementally, appending tokens to the response text as they arrive. When streaming completes, the server calculates the actual credits consumed based on the total token count, trues up the ledger entry, and sends a final event with the usage summary. The chat UI shows a smooth typing animation, displays token usage incrementally, and supports conversation history with the ability to continue previous conversations. The implementation handles errors gracefully, with automatic retry for transient failures and clear error messages for permanent failures like insufficient credits or authentication errors.

Credit Accounting System

The credit accounting system is the heart of the AI SaaS Starter's metered billing architecture. It uses a reserve-then-true-up pattern that is fundamentally different from the simpler check-then-deduct approach. In a check-then-deduct system, the server checks the user's balance at the moment of the request and then deducts after the request completes. This creates a window where simultaneous requests could overspend the balance, and failed requests still consume credits. The reserve-then-true-up system eliminates these problems. Before making an API call, the server reserves a portion of the user's credit balance based on an estimate of the request cost. If the reservation fails due to insufficient credits, the request is rejected immediately. The API call then proceeds, and when it completes, the actual cost is calculated based on actual token consumption. The ledger entry is updated from the reserved amount to the actual amount, and the difference is released back to the available balance. This ensures that one user cannot accidentally overspend even with concurrent requests, and that failed requests (where the API returns an error) do not consume credits. The credit ledger is an append-only table that records every grant and consumption event, providing a complete audit trail. The Stripe integration allows users to purchase credit packs or subscribe to monthly credit allotments, with the ledger automatically crediting the user upon successful payment.

Rate Limiting Strategy

Rate limiting is essential for AI products to prevent abuse and control costs. The AI SaaS Starter implements a multi-tier rate limiting strategy. At the first tier, global rate limiting protects the API from traffic spikes by limiting requests per second across all users. At the second tier, per-user rate limiting ensures that a single user cannot consume the entire API budget. The third tier implements credit-based rate limiting, where a user's rate limit is proportional to their remaining credit balance. The rate limiter is implemented as middleware that checks the current request against stored counters. For development, the limiter uses an in-memory store, with the exact swap point marked for Redis via Upstash when deploying to production. The rate limiting headers are returned to the client, allowing the UI to display remaining request capacity and reset times. The configuration is driven by environment variables that set the default limits, with per-user overrides stored in the database for premium subscribers. The rate limiter is integrated with the credit system so that users with higher-tier plans get higher rate limits.

Stripe Metered Billing Integration

The Stripe metered billing integration handles the unique requirements of usage-based pricing. The starter supports both prepaid credit packs and postpaid metered billing. For prepaid credits, users purchase credit packs through a Stripe Checkout Session, and upon successful payment, the Stripe webhook handler credits the user's ledger. For postpaid metered billing, Stripe's metered usage API reports consumption at the end of each billing period, and the customer is invoiced based on their actual usage. The integration includes a configurable pricing model with multiple tiers, automatic credit expiration for unused credits, usage alerts when the user's balance drops below configurable thresholds, and invoice reconciliation. The webhook handler processes checkout.session.completed for credit purchases, invoice.paid for subscription renewals, and customer.subscription.updated for plan changes. The Stripe product catalog is defined in a central configuration file, with prices, credits, and features all specified as constants. Adding a new pricing tier requires only adding an entry to this configuration array.

Getting Started with the AI SaaS Starter

The AI SaaS Starter Starter Kit provides everything you need to launch your AI product with metered billing. The setup process involves cloning the repository, configuring environment variables for Anthropic API key and Stripe credentials, running database migrations, and seeding test configurations. The kit includes documentation covering the credit accounting system, streaming chat implementation, rate limiting configuration, and Stripe integration. The development server provides a fully functional chat interface with credit tracking, and the Stripe test mode allows complete billing flow testing without real charges. You can have a working AI product with metered billing deployed within hours.

Conversation Memory and Context Management

Managing conversation context effectively is critical for AI products that provide coherent multi-turn interactions. The AI SaaS Starter implements a sophisticated memory system that balances context retention with token efficiency. Short-term memory uses the conversation message list within the current session, with a configurable window of recent messages that are included in each API call. Long-term memory uses a summarization approach where older conversation segments are periodically summarized and stored as condensed context. The memory system supports multiple strategies: sliding window that keeps the N most recent messages, token budget that allocates a maximum number of tokens for context and discards oldest messages first, semantic summarization that uses an LLM call to compress older conversation segments, and hybrid approaches that combine window and summarization. The memory configuration is stored per-user and per-conversation, allowing different strategies for different use cases. The system tracks token consumption per conversation for billing purposes and provides analytics on context usage. The memory implementation is designed to be extensible, with a plugin interface for custom memory strategies like vector database integration for semantic search across conversation history.

Analytics and Usage Dashboard

The AI SaaS Starter includes a comprehensive analytics dashboard that provides visibility into system usage, costs, and user behavior. The dashboard displays real-time metrics including active users, requests per minute, average response time, token consumption rate, and credit balance distribution. Historical analytics track usage trends over configurable time periods, with breakdowns by user tier, model, and conversation type. The cost analytics section shows API cost breakdown by model and user, projected costs based on current usage patterns, and budget alerts when spending exceeds configurable thresholds. The user analytics section provides per-user usage reports with conversation counts, token consumption, credits used, rate limit hits, and average session duration. The dashboard supports custom date ranges, data export to CSV for further analysis, and email report scheduling for weekly or monthly summaries. The analytics are powered by the append-only credit ledger, ensuring that all usage data is accurate and auditable. The dashboard is built with React Server Components for performance, with real-time updates through Server-Sent Events for the live metrics section. The analytics data is also available through a REST API for integration with external analytics platforms like Metabase or Grafana.

Error Handling and Reliability

Reliability is essential for AI products where each request has a real cost and users expect immediate responses. The AI SaaS Starter implements comprehensive error handling at every layer of the stack. LLM API errors are categorized as transient (rate limits, temporary outages) and permanent (invalid API key, unsupported model). Transient errors trigger automatic retry with exponential backoff and jitter, while permanent errors return clear error messages to the user. Streaming errors are handled gracefully, with the server closing the SSE connection and sending an error event that the client displays as a message in the chat interface. Credit accounting errors are logged with full context for manual reconciliation, and the reserve-then-true-up system ensures that credit balance errors cannot result in unbilled usage. Database errors are caught by Prisma's error handling, with automatic retry for connection timeout errors and proper error responses for constraint violations. The error tracking integration captures all unhandled errors with request context, user identity, and stack traces for debugging. Health check endpoints verify the LLM API connection, database connectivity, and credit system integrity, with Prometheus metrics for monitoring and alerting. The application includes a status page component that displays service health for transparency with users.

Model Provider Abstraction

The AI SaaS Starter is designed with a provider abstraction layer that allows switching between different LLM providers without changing the application code. The abstraction defines a common interface for chat completion, streaming, embedding generation, and token counting. Providers are configured through environment variables and can be swapped at the application level. The built-in providers include Anthropic Claude with streaming support, OpenAI GPT-4 and GPT-3.5 with streaming and function calling, and a mock provider for development and testing without API costs. The provider abstraction handles the differences between providers, including message format conversion, streaming protocol adaptation, token counting normalization, and error code translation. Custom providers can be added by implementing the provider interface. The abstraction also supports provider fallback where requests to the primary provider are retried on a secondary provider if the primary is unavailable, and provider routing where different models are used for different request types (for example, a fast model for simple queries and a powerful model for complex analysis). The provider configuration includes per-provider rate limits, cost tracking, and usage quotas. The token counting is normalized across providers to ensure consistent credit consumption regardless of which provider is serving the request.

Caching and Performance Optimization

Performance is critical for AI products where users expect immediate responses. The AI SaaS Starter implements multiple caching strategies to optimize response times and reduce API costs. Response caching stores complete AI responses for identical queries, with configurable time-to-live and cache invalidation based on context. The cache is particularly effective for common questions, error messages, and system prompts. Semantic caching uses vector similarity to return cached responses for semantically similar queries, reducing LLM API calls for common question patterns. The cache is stored in Redis for fast access and supports distributed caching across multiple instances. Cache hit rates are tracked and displayed in the analytics dashboard. Conversation history caching stores recent conversations in Redis for fast retrieval, with automatic migration to PostgreSQL for long-term storage. The rate limiter uses Redis for distributed rate limiting across multiple serverless function instances. Database query caching uses Prisma's result caching for frequently accessed but rarely changing data like user settings and plan configurations. The chat UI uses optimistic updates for a responsive feel, displaying the user message immediately while the AI response is being generated. The application uses Next.js's built-in image optimization and lazy loading for media-rich responses.

User Management and Admin Panel

The AI SaaS Starter includes a comprehensive user management system designed for the unique requirements of AI products. The user management features include user registration with email verification and optional Google OAuth, profile management with name, email, and notification preferences, credit balance display with grant and consumption history, API key management for programmatic access with usage tracking per key, rate limit status showing current limits and reset times, and subscription management with plan comparison and upgrade flow. The admin panel provides user search and filtering across all users, user detail view showing account information, credit balance, usage history, and current plan, credit adjustment for support credits or corrections with audit trail, plan override for manual subscription changes, usage analytics per user with charts and export, and account management for suspension, deletion, and data export. The admin panel is accessible only to users with the admin role and is built with the same component library as the main application. All admin actions are logged in the audit trail with actor identity, timestamp, and action details for compliance and troubleshooting. The admin panel includes a dashboard with key metrics including total users, active users, credits distributed and consumed, and revenue from credit purchases.

Customization and White-Labeling

The AI SaaS Starter is designed to be customized and white-labeled for different products and brands. The customization options include custom branding with logo, colors, and typography that are applied throughout the application, custom chat interface styling with configurable message bubble colors, font sizes, and layout options, custom welcome messages and system prompts that are shown to users on their first visit, custom credit pricing that can be set per product with different credit packs and monthly plans, custom rate limits per user tier with configurable thresholds, and custom email templates for password reset, welcome, and billing notifications. The white-labeling features remove all BreafIO branding from the application, including the login page, navigation, footer, and email templates. Custom domains are supported with SSL certificate provisioning through Vercel. The application can be deployed with a custom subdomain for each customer or a single domain with tenant-based routing. The theme configuration is stored in environment variables and database settings, allowing different configurations for different deployment instances. The customization API allows programmatic configuration of theme settings, making it possible to build a self-serve customization interface for end users. The chat interface supports plugin architecture for extending functionality with custom message types, custom tool integrations, and custom response formatting.

Compliance and Data Privacy

AI products must address compliance and data privacy concerns, and the AI SaaS Starter includes features for GDPR, CCPA, and SOC 2 compliance. Data retention policies are configurable, with automatic deletion of conversation history after a configurable period. User data export provides a complete archive of all user data including conversation history, credit ledger, and account settings in a portable format. Account deletion removes all user data from the system with configurable grace periods for recovery. Data processing agreements are available for customers who require them. The audit log records all access to user data for compliance monitoring. Conversation data can be encrypted at rest using database-level encryption, with options for application-level encryption of sensitive fields. The LLM API integration includes data processing addendums from providers like Anthropic and OpenAI that clarify data usage policies. The application supports data residency requirements through configurable database regions and LLM API endpoint selection. Privacy policy and terms of service templates are included with placeholders for custom legal text. Cookie consent banners and privacy preference centers are included for GDPR compliance. The compliance documentation provides guidance on achieving SOC 2 Type II certification with the boilerplate architecture.

Common Use Cases and Examples

The AI SaaS Starter is suitable for a wide range of AI-powered products. Common use cases include AI writing assistants that help users generate content with credit-based pricing per article or word count. AI customer support chatbots that handle inbound inquiries with usage billed per conversation or resolution. AI code review tools that analyze pull requests with credits consumed per review. AI document analysis platforms that process and summarize documents with pricing based on document length and complexity. AI language learning applications that provide conversation practice with metered pricing per session. Each of these products benefits from the streaming chat interface, credit-based billing, rate limiting, and usage analytics that the starter provides. The flexibility of the credit system allows for creative pricing models like daily free credits, credit packs for prepaid usage, and monthly subscriptions with included credit allotments. The analytics system provides visibility into usage patterns that inform pricing optimization and feature development. The provider abstraction allows choosing the most cost-effective AI model for each use case, optimizing the balance between response quality and operational costs.