Traffic Management

TSB's traffic routing capabilities let you easily control traffic flow between services under its control. TSB simplifies the approach to structuring how you interact with applications and services, and by extension, simplifies the processes behind tasks such as traffic routing, staged rollouts, and migrations.

This page will walk you through:
✓ Gateways in TSB
✓ Intelligent traffic routing
✓ Multi-cluster capabilities in TSB
✓ Traffic splitting, migrations, and canary deployments

TSB controls Istio and Envoy in each cluster. All traffic inside the cluster is proxied by Envoy, while your local control plane enforces the rules defined within the management plane. Any changes that you make or policies that you apply are stored and distributed by the global control plane to local control planes.

Gateways in TSB

As traffic enters through your TSB environment, it passes through several gateways to reach the final application it needs. First, it passes through an Application Edge Gateway ("edge gateway" for short), which is a shared multi-tenant gateway responsible for cross-cluster load balancing. The Edge Gateway forwards traffic into an Application Ingress Gateway ("app gateway" for short), or potentially even the workload itself (common for VM workloads).

Application Ingress Gateways are owned by a Workspace, and come in two flavors: shared and dedicated to a specific app. This means that workspace's owners remain in control of how traffic flows through the gateway -- whether that gateway is dedicated to their own application, or is shared by many. Tetrate Service Bridge uses this ownership to keep your mesh configuration isolated at runtime, but there are always classes of misconfiguration that TSB cannot prevent. For example, we cannot ensure that one regex based path match on a shared gateway does not shadow others.

So, we recommend deploying many Ingress Gateways: one for every application (roughly, namespace), if you can. Envoy is cheap and lightweight, and auto scaling keeps overhead minimal. Multiple Ingress Gateways allow for isolation and control of your traffic. It keeps your applications safe and reduces risk of outages because teams cannot write configuration that conflicts by design. As your team's usage of the mesh matures and your confidence grows, we can consolidate most application traffic onto a set of shared Ingress Gateways for efficiency, and leave a few dedicated gateways for critical applications. TSB lets you make this decision in a smooth gradient, running one gateway or thousands.

Intelligent Traffic Routing

For TSB to manage application traffic, it needs to know where all your services are and whether they're available. TSB relies on the local control planes in each cluster to provide information on the locality, health and availability of the services in that cluster. Using that information, Istio and TSB will keep your traffic as local as possible, including in the same availability zone, for multi-availability zone clusters.

Envoy's superpowers allow TSB to do this per-request, not per connection. This gives you fine grained control over how your application communicates: keeping all requests sticky to a specific backend instance, canarying a portion based on header or percentage to a new version, sending requests to any available backend no matter where it runs. TSB can do all of that, while keeping your traffic as local as possible, always.

TSB advertises information about which services are available in each cluster to local control plane instances in every other cluster. This information is used at runtime to enable seamless traffic failover across clusters. But it's also communicated back to the management plane so that application developers, platform owners, and security admins can see near-real-time information on service health, connectivity, and controls.

Multi-cluster Routing

TSB makes multi-cluster management easy for organizations -- whether you need multiple clusters running active-active handling user load or active-passive waiting for failover, and even if you have many clusters across teams in your organization. Of course, an Ingress Gateway can be configured to expose a hostname (e.g. echo.tetrate.io) for an application (e.g. echo) allowing it to be accessed by external clients. TSB enables the same kind behavior for services in the mesh, exposing them to other applications in the mesh privately even across clusters. Applications in the mesh (or external clients via an Edge Gateway) can access that application by hostname without knowing any details about where the service is deployed.

Combined with intelligent traffic routing, this gives us a super power: the mesh will route traffic to local instances of a service if they're available, or out into other clusters that expose the service if it's not local (or if it fails locally). And just because the application is exposed with a gateway doesn't mean we have to send our traffic through that gateway: when the application is local we'll communicate directly peer-to-peer, eliminating that hairpin traffic common in systems today.

What goes on underneath (thanks to the global control plane) makes intelligent traffic routing even smarter. The global control plane allows the local control planes to advertise available services, and makes service discovery available across all clusters. This means your system can serve more requests and maintain a higher availability.

API Gateway capabilities Everywhere

In the Service Mesh Concepts section we discussed API Gateways and the service mesh and how TSB brings API gateway functionality everywhere in the application traffic platform.

Using TSB, you can configure how traffic flows to your application by annotating your OpenAPI spec, and TSB will implement your intents across Gateways and - or - internally in the mesh: you have the flexibility to choose. But we don't force you to think about it either: the platform team can establish where rules are enforced and application developers can author configuration. You can easily implement the following across your entire infrastructure, or only on behalf of specific teams and services:

• Authentication and authorization for applications
• Rate limiting
• WAF policy
• Request transformation via WASM

Traffic Splitting and Migrations

Traffic management within TSB is designed for safety and simplicity across the entire infrastructure. As an application developer, you only need to push a configuration once to change or update how your services handle traffic. TSB makes the process of migrating to microservices, splitting traffic, and canarying deployments much easier to manage and safer to perform at scale.

The configuration to enable migrations, traffic splitting and canary deployments are all similar and you, as the application or platform owner, define how traffic flows to different version of your application or different parts of your infrastructure. This gives you incredible control over your application's runtime environment. With the observability capabilities TSB brings, you can also have confidence that everything is working as you expected. If not -- you can rollback or deploy an older version in a cinch.

One of TSB's biggest goals is keeping your applications available, even while you're migrating infrastructures or moving from monoliths to microservices.