Budgets and Budget Templates

Budgets and Budget Templates in Tetrate Agent Operations Director provide standardized patterns for allocating and managing LLM resources across your organization. They enable consistent governance while maintaining flexibility for diverse use cases.

Budgets

A budget in Tetrate Agent Operations Director is a policy that governs the usage of one or more resources by consumers. Each budget is defined by:

• Resource managed: The specific LLM resource the budget governs
• Consumer ownership: The application or service using this budget
• Quota per time period: Token limits and remaining quota
• Financial budget: Calculated by quota and resource cost, with remaining balance
• Budget name: User-modifiable identifier
• Mode: None, Monitored, or Enforced

Budgets are the primary mechanism for implementing cost control and governance in your LLM infrastructure.

Budget Templates

Budget templates are pre-configured budget structures that define standardized patterns for creating consistent budgets. They establish:

• Quota Allocations: Token limits for resources (measured in thousands or millions of tokens)
• Financial Quotas: Dollar amounts corresponding to token usage

Example Templates

You can create templates based on provider or owner. For examples:

• Provider-based Templates:

  • openai-t1: Monthly experimental budget per team (500 tokens, $4,500)
  • aws-max: Monthly maximum budget (500k tokens, $180,000)
  • anthropic-starter: Basic allocation for initial usage (6k tokens, $15)

• Owner-based Templates:

  • EquityResearch-experimental: Experimental budget for equity research (100k tokens, $10,000)
  • InvestmentBanking-experimental: Experimental budget for investment banking (300k tokens, $30,000)
  • InvestmentBanking-max: Maximum budget for investment banking (15M tokens, $3,000,000)
  • RetailBanking-experimental: Experimental budget for retail banking applications (200k tokens, $20,000)

Template Administration

The Budget Templates interface allows administrators to:

• View Templates: See all available templates with their quota and financial allocations
• Create Templates: Define new templates with custom parameters
• Edit Templates: Modify existing templates as requirements change
• Delete Templates: Remove unused or outdated templates
• Search and Filter: Quickly find templates by name or type

Applying Templates

When creating a new Budget, administrators must select an appropriate template, which automatically populates the quota settings. Templates can be applied to:

• Specific Provider/Resource combinations
• Individual Owners
• Particular Consumers

The template system ensures consistent governance while providing the flexibility needed to accommodate different usage patterns across the organization.