Skip to main content
logoTetrate Service BridgeVersion: next

Ambient Multicluster Support

warning

Before getting started, ensure that you have followed the procedures described in the Installation section and have set up your Management Plane and Control Plane with the correct settings.

TSB's multicluster capabilities work seamlessly with ambient mode, providing the same service discovery, traffic routing, and failover functionality available to sidecar-based workloads. The key principle is that gateway configuration and hostname exposure remain identical between sidecar and ambient modes - what changes is how client traffic reaches those gateways.

Architecture Overview​

Gateway Configuration Consistency​

From an API perspective, multicluster routing setup is identical for both sidecar and ambient mode services:

  • Gateway configuration remains the same regardless of client or server mode
  • Hostname exposure on gateways works identically for both modes
  • Service discovery and cross-cluster endpoints are managed consistently by TSB's XCP (Cross Cluster Platform)

Traffic Flow Differences​

The difference lies in how client traffic reaches remote gateways.

Ambient Mode Clients​

  1. Traffic originates from client workload
  2. Goes through client's ztunnel (Layer 4 proxy)
  3. Routes to egress waypoint in xcp-multicluster namespace
  4. Waypoint forwards traffic to remote gateway
  5. Remote gateway routes to destination service (can be sidecar or ambient)

Sidecar Mode Clients​

  1. Traffic originates from client workload
  2. Goes through client's sidecar proxy
  3. Routes directly to remote gateway
  4. Remote gateway routes to destination service (can be sidecar or ambient)

Prerequisites for Multicluster Ambient​

Before setting up multicluster support with ambient mode, ensure:

  1. Multiple clusters onboarded to TSB with shared root of trust
  2. Ambient mode enabled in ControlPlane configuration
  3. Cross-cluster connectivity established between clusters
  4. East-West or Ingress gateways deployed for service exposure

Enabling Multicluster Features​

DNS Resolution for Cross-Cluster Communication​

For DNS-based multicluster hostname resolution, enable DNS resolution at XCP edge.

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  components:
    xcp:
      kubeSpec:
        overlays:
        - apiVersion: install.xcp.tetrate.io/v1alpha1
          kind: EdgeXcp
          name: edge-xcp
          patches:
          - path: spec.components.edgeServer.kubeSpec.deployment.env[-1]
            value:
              name: ENABLE_DNS_RESOLUTION_AT_EDGE
              value: "true"

Current Limitations​

Important: Enhanced East-West Routing for Kubernetes canonical hostname services (like foo.foons.svc.cluster.local) is not yet implemented for ambient mode. This means:

  • Direct service-to-service communication using canonical hostnames across clusters is not supported in ambient mode
  • Cross-cluster communication must use gateway-exposed hostnames
  • Enhanced East-West Routing configuration (ENABLE_ENHANCED_EAST_WEST_ROUTING) applies only to sidecar mode workloads

For the latest feature status, see the TSB release notes and or reach out to your Tetrate representative.

Service Exposure Patterns​

Application Ingress Gateway Exposure​

Application Ingress Gateways work for ambient services when accessed via gateway-exposed hostnames. The Ingress Gateway for Ambient Service can be set up as:

apiVersion: gateway.tsb.tetrate.io/v2
kind: Gateway
metadata:
  name: bookinfo-gateway
  organization: tetrate
  tenant: test
  workspace: bookinfo
  group: bookinfo-gw
spec:
  workloadSelector:
    namespace: bookinfo
    labels:
      app: bookinfo-gateway
  http:
  - name: bookinfo
    port: 80
    hostname: bookinfo.tetrate.com
    routing:
      rules:
      - route:
          serviceDestination:
            host: bookinfo/productpage.bookinfo.svc.cluster.local
            port: 9080

The above configuration works whether the productpage service is in sidecar or ambient mode.

Current Limitations for East-West Gateways​

Important: East-West Gateway exposure for ambient services is not yet implemented. This means:

  • Ambient services cannot be exposed through East-West gateways for cross-cluster communication
  • Direct service-to-service communication between ambient workloads across clusters via East-West gateways is not supported
  • Cross-cluster communication from ambient clients must use Application Ingress Gateway exposed hostnames

Ambient Waypoint for Cross-Cluster Traffic​

Automatic Egress Waypoint Deployment​

TSB automatically deploys egress waypoints in the xcp-multicluster namespace for ambient workloads that need to communicate across clusters. This waypoint:

  • Handles Layer 7 routing for cross-cluster traffic from ambient clients
  • Provides traffic management capabilities like retries, timeouts, and circuit breaking
  • Maintains traffic policies defined in TSB configurations
  • Automatically scales based on cross-cluster traffic demands

Waypoint Configuration for Cross-Cluster​

You can customize the egress waypoint behavior specifically for the xcp-multicluster namespace through ControlPlane configuration:

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  components:
    istio:
      ambient:
        enable: true
        waypoints:
          namespaceLevel:
          - name: xcp-multicluster
            config:
              enable: true
              overlays:
                istio-waypoint:
                  deployment:
                    metadata:
                      annotations:
                        cross-cluster: "enabled"
                  horizontalPodAutoscaler:
                    spec:
                      maxReplicas: 10
                      minReplicas: 2

Service Discovery and Failover​

Cross-Cluster Service Discovery​

TSB's service discovery works transparently with ambient workloads:

  1. Services exposed via Ingress gateways are automatically discovered
  2. ServiceEntries created in xcp-multicluster namespace for remote services
  3. WorkloadEntries populated with remote endpoint addresses
  4. DNS resolution allows ambient clients to resolve remote service FQDNs

Check cross-cluster service discovery:

# Check ServiceEntries for remote services 
kubectl get serviceentries -n xcp-multicluster
# Test DNS resolution from ambient client
kubectl exec deployment/ambient-http-client -n client -- nslookup bookinfo.tetrate.com

Automatic Failover​

Cross-cluster failover works seamlessly with ambient mode.

  • Local services are prioritized automatically (locality-aware routing)
  • Remote failover occurs when local services become unavailable
  • Health checking monitors service availability across clusters
  • Traffic shifting happens transparently to application code

The failover behavior is identical whether the client is in sidecar or ambient mode.

Mixed Mode Scenarios​

Ambient Client to Sidecar Service​

Ambient clients can seamlessly communicate with sidecar-based services across clusters:

  • Traffic flows: Ambient Client → ztunnel → Egress Waypoint → Remote Gateway → Sidecar Service
  • mTLS encryption maintained throughout the path
  • Traffic policies applied at waypoint and destination sidecar
  • Service identity preserved for authorization

Sidecar Client to Ambient Service​

Sidecar clients can communicate with ambient services across clusters:

  • Traffic flows: Sidecar Client → Remote Gateway → ztunnel → Ambient Service
  • Gateway configuration remains identical
  • Traffic routing works transparently
  • Security policies applied normally

Verification and Troubleshooting​

Verify Cross-Cluster Connectivity​

Test connectivity between ambient workloads across clusters:

note

The below command is only if the echo service is not present in the local cluster.

If echo service is present in both local and remote clusters, endpoint level aggregation is not supported.

# Test ambient client to remote ambient service 
kubectl exec deployment/ambient-http-client -n client -- curl -v http://echo.echo.svc.cluster.local:8080
# Check egress waypoint logs
kubectl logs -l app=global-waypoint -n xcp-multicluster
# Verify cross-cluster ServiceEntries
kubectl get serviceentries -n xcp-multicluster -o yaml

TSB Topology View​

The TSB UI topology view will display cross-cluster traffic flows for ambient workloads as shown below.

AmbientTopology

The topology visualization includes:

  • Ambient workload nodes with ztunnel indicators
  • Egress waypoint in xcp-multicluster namespace
  • Cross-cluster traffic flows with directional arrows
  • Gateway nodes on remote clusters
  • mTLS encryption status throughout the path
  • Traffic metrics and success rates

Common Troubleshooting Steps​

  1. Check waypoint deployment in xcp-multicluster namespace
  2. Verify ServiceEntry creation for remote services
  3. Confirm gateway accessibility from waypoint
  4. Test DNS resolution from ambient clients
  5. Review XCP edge logs for service discovery issues
# Check waypoint status 
kubectl get pods -n xcp-multicluster -l app=waypoint
# Verify gateway endpoints
kubectl get endpoints -n bookinfo eastwest-gateway
# Check XCP edge connectivity
kubectl logs deployment/edge -n istio-system

For detailed troubleshooting, follow the document on East-West Gateway and Multicluster Routing.