Connectivity and Failover between Clusters

Tetrate makes it easy for the App Owner to deploy an application across multiple clusters. East-West gateways can be deployed in selected Workspaces, and expose some or all services to other clusters. Tetrate's central Service Registry is used to then create virtual endpoints (ServiceEntries) for these exposed services in other clusters, so that clients can discover and consume these services exactly as if they were running locally.

This capability can be used to:

• Distribute components of an application across different clusters
• Access centralized, shared services such as a Database microservice from remote clusters
• Provide Failover between instances of services across different clusters
• Simplify testing by ensuring that the same application and addressing scheme works in one cluster, or distributed across multiple

The Platform Owner ("Platform") prepares the platform for failover use cases, and the Application Owner ("Apps") can take advantage of these capabilities with minimal modification to their workflow.

The Platform Owner will prepare the platform as follows:

1. Deploy an East-West Gateway

   Deploy an East-West Gateway in the namespace containing services to be shared.

2. Update the Workspace to expose the required services

   Update the Workspace to expose the required services so that they can be discovered and used on other clusters.

3. Deploy East-West Gateways for Internal Failover

   Deploy additional East-West Gateways in other clusters for service failover.

The Application Owner can then:

1. Access the Exposed Services

   Access the Exposed Services from other clusters.

2. Test Failover across Clusters

   Verify that services can failover from one cluster to a backup instance in another cluster.

Platform: Before you begin

Before you begin, you will need to know:

• The Workspace where the shared services are located
• Which services should be exposed; by default, all services within the Workspace are exposed

You can also review the TSE Getting Started use cases: High Availability and Cross-Cluster Communications.

Platform: Deploy an East-West Gateway

Deploy an East-West Gateway in the namespace containing services to be shared:

cat <<EOF > eastwest-gateway.yaml
apiVersion: install.tetrate.io/v1alpha1
kind: IngressGateway
metadata:
  name: eastwest-gateway
  namespace: bookinfo
spec:
  eastWestOnly: true
EOF

kubectl apply -f eastwest-gateway.yaml

List the East-West Gateways running in the local cluster:

kubectl get pods -A -l app=eastwest-gateway

Platform: Update the Workspace to expose the required services

Update the WorkspaceSettings associated with the Workspace:

cat <<EOF > bookinfo-settings.yaml
apiVersion: api.tsb.tetrate.io/v2
kind: WorkspaceSetting
metadata:
  name: bookinfo-ws-settings
  workspace: bookinfo-ws
  tenant: tse
  organization: tse
spec:
  defaultSecuritySetting:
    authenticationSettings:
      trafficMode: REQUIRED
    authorization:
      mode: WORKSPACE
  defaultEastWestGatewaySettings:
    - workloadSelector:
        namespace: bookinfo
        labels:
          app: eastwest-gateway
      exposedServices:
        - serviceLabels:
            app: details
        - serviceLabels:
            app: reviews
EOF

tctl apply -f bookinfo-settings.yaml

Note how the defaultEastWestGatewaySettings uses workloadSelector to identify the EastWest Gateway, and has an optional exposedServices stanza.

The Application Owner can discover and consume these shared services from the remote cluster. They are exposed as ServiceEntries in the xcp-multicluster namespace, and use the same FQDN as in the first cluster:

kubectl get serviceentry -n xcp-multicluster

Note that if the Application Owner deploys the same services in the remote cluster, then the Tetrate control plane will then remove the ServiceEntries so that clients in that cluster prefer service instances in the same cluster.

Apps: Access the Exposed Services

Once an East-West gateway is deployed in the origin cluster, and services are selected for exposure using WorkspaceSettings:defaultEastWestGatewaySettings, you can then deploy services in the origin cluster. Matching services are then exposed within remote clusters.

You can view the list of exposed services in the remote cluster as follows:

kubectl get serviceentry -n xcp-multicluster

The FQDNs of the services are addressable in the remote cluster, meaning that any client services in the remote cluster can access the origin service without any modification. The Tetrate platform maintains the ServiceEntries in sync with the presence and state of the origin services.

Platform: Deploy East-West Gateways for Internal Failover

In this use case, we'll prepare the platform for failover if internal services from one cluster to another. For example, an Application Owner could deploy a multi-component application (such as bookinfo) into two clusters. Each application will use the local version of the dependent services unless the local instance fails, in which case, the Tetrate platform will automatically fail traffic over to the other cluster. No application modifications are required on behalf of the Application Owner.

Prerequisites

1. The namespaces for the application must be created on each workload cluster.
2. The Tetrate workspace must include the namespaces, and must span each workload clusters.

For example, if the application is to be deployed in the namespace bookinfo, this namespace must be present on each cluster and the Workspace configuration should reference all instances, for example, using the following namespaceSelector:

spec:
  namespaceSelector:
    names:
      - "*/bookinfo"

Deploy an East-West gateway

Deploy additional East-West gateways in the other clusters, exactly as before:

cat <<EOF > eastwest-gateway.yaml
apiVersion: install.tetrate.io/v1alpha1
kind: IngressGateway
metadata:
  name: eastwest-gateway
  namespace: bookinfo
spec:
  eastWestOnly: true
EOF

kubectl apply -f eastwest-gateway.yaml

Apps: Test Failover across Clusters

With the above configuration, you can deploy your application into several clusters. If a service instance in one cluster were to fail for any reason, the Tetrate platform will detect this and transparently switch traffic over to a remote, working instance of the service.

The Tetrate platform creates shadow WorkloadEntries for each remote instance of a service in the cluster and namespace where the local instance exists. For example, if you deploy the bookinfo app into two clusters which are equipped with the HA configuration, you can then check each cluster for the presence of the shadow WorkloadEntries.

In the case below, we exposed the details and reviews services only, and the Cloud platform exposed the eastwest-gateway on three IP addresses:

kubectl get workloadentries -n bookinfo
# NAME                                           AGE   ADDRESS
# k-details-fc556d47e94d1cb435e513fa016c2243     17m   18.135.167.198
# k-details-fc556d47e94d1cb435e513fa016c2243-2   17m   18.168.99.230
# k-details-fc556d47e94d1cb435e513fa016c2243-3   17m   35.179.51.164
# k-reviews-3ab8d1334c8f22513cd591f84c978f88     17m   18.135.167.198
# k-reviews-3ab8d1334c8f22513cd591f84c978f88-2   17m   18.168.99.230
# k-reviews-3ab8d1334c8f22513cd591f84c978f88-3   17m   35.179.51.164

kubectl get svc -n bookinfo eastwest-gateway
# NAME               TYPE           CLUSTER-IP      EXTERNAL-IP                                                                     PORT(S)           AGE
# eastwest-gateway   LoadBalancer   10.100.17.100   k8s-bookinfo-eastwest-00a17af379-bdda7f4eb8c5da5c.elb.eu-west-2.amazonaws.com   15443:30082/TCP   82m

For more information, check out the TSE Getting Started exercise Failover Across Clusters.