Set Budget Templates and Budgets

Learn how to create and configure budget templates and budgets in Tetrate Agent Operations Director to establish effective governance and cost controls for LLM usage.

Overview

Budgets are the core control mechanism in Agent Operations Director, allowing you to monitor and manage LLM usage across your organization. Budget templates provide standardized patterns for consistent governance. This guide shows how to set up both components for effective LLM management.

Understanding Budget Components

Budget Templates

Budget templates are pre-configured budget structures that define standardized patterns for creating consistent budgets. They establish:

• Quota Allocations: Token limits for resources (measured in thousands or millions of tokens)
• Financial Quotas: Dollar amounts corresponding to token usage
• Time Periods: Only support per day

Budgets

A budget in Tetrate Agent Operations Director is a policy that governs the usage of one or more resources by consumers. Each budget is defined by:

• Resource managed: The specific LLM resource the budget governs
• Consumer ownership: The application or service using this budget
• Quota per time period: Token limits and remaining quota
• Financial budget: Calculated by quota and resource cost, with remaining balance
• Budget name: User-modifiable identifier
• Mode: None, Monitored, or Enforced

Creating Budget Templates

Step 1: Access the Budget Templates Section

1. Log in to the Tetrate Agent Operations Director console
2. Navigate to the "Budget Templates" section from the left navigation panel
3. Click the "+ Template" button to begin creating a new template

Step 2: Define Template Parameters

Configure the template with appropriate settings:

1. Template Name: Create a descriptive name following your naming convention

   • Provider-based: openai-t1, anthropic-starter, aws-max
   • Owner-based: EquityResearch-experimental, InvestmentBanking-max
   • Purpose-based: development, production, customer-facing

2. Quota Allocation: Set the token limits

   • Base tokens: Standard monthly allocation (e.g., 500k tokens)
   • Time period is per day

3. Financial Parameters: Define monetary equivalents

   • Budget amount: Dollar value for the allocation
   • Time period is per day

Example Template Structures

Consider creating a hierarchical template structure:

Provider-based Templates

• Tier 1 (Experimental):

  • openai-t1: Monthly experimental budget (500 tokens, $4,500)
  • anthropic-t1: Limited experimental usage (100 tokens, $1,500)
  • aws-t1: Basic AWS model access (200 tokens, $2,000)

• Tier 2 (Standard Production):

  • openai-t2: Standard production allocation (5,000 tokens, $45,000)
  • anthropic-t2: Regular production usage (3,000 tokens, $30,000)
  • aws-t2: Standard AWS production (4,000 tokens, $30,000)

• Tier 3 (High Volume):

  • openai-t3: High-volume production (50,000 tokens, $450,000)
  • anthropic-t3: Extensive production usage (30,000 tokens, $300,000)
  • aws-t3: Large-scale AWS usage (40,000 tokens, $300,000)

Business Unit Templates

• Experimental:

  • RetailBanking-experimental: Initial testing (200,000 tokens, $20,000)
  • EquityResearch-experimental: Research testing (100,000 tokens, $10,000)
  • CustomerService-experimental: Support testing (150,000 tokens, $15,000)

• Production:

  • RetailBanking-production: Full deployment (2,000,000 tokens, $200,000)
  • EquityResearch-production: Research operations (1,500,000 tokens, $150,000)
  • CustomerService-production: Support operations (3,000,000 tokens, $300,000)

Setting Up Individual Budgets

Step 1: Access the Budgets Section

1. Navigate to the "Budgets" section from the left navigation panel
2. Click the "+ Budget" button to begin creating a new budget

Step 2: Select a Budget Template

1. Review available templates in the dropdown menu
2. Select an appropriate template based on:
   • Provider type (e.g., OpenAI, AWS Bedrock, Anthropic)
   • Usage patterns (production vs. experimental)
   • Department or team requirements

Step 3: Configure Budget Parameters

Complete the required fields in the budget creation form:

1. Budget Name: Create a descriptive name (e.g., "ConversationAI-25H1")
2. Mode: Set the mode, start with Monitored
3. Owner: Select the owner of consumer using this budget
4. Consumer: Select the application or service using this budget
5. Provider: Choose the Provider of the resource being consumed
6. Resource: Choose the specific LLM resource being consumed

Step 4: Save and Activate

Complete the budget creation process:

1. Review all settings for accuracy
2. Click "Save" to create the budget
3. Verify the budget appears in the budgets list with the correct status

Budget Lifecycle Management

Understanding the Budget Lifecycle

Budgets typically progress through several modes:

1. Monitored: Monitoring usage without enforcement
2. Enforced: Active control of consumption with defined limits

Transitioning to Enforced

After a monitoring period:

1. Review collected usage data to validate quota settings
2. Communicate with application teams about enforcement plans
3. Update the budget:
   • Click "Edit" on the budget
   • Change mode from "Monitored" to "Enforced"
   • Save changes

Budget Renewal and Adjustment

At the end of budget cycles (typically quarterly or annually):

1. Review historical usage patterns
2. Assess effectiveness of current allocations
3. Create new budgets with adjusted parameters
4. Remove unused budgets