Unified Gateway

This document introduces the concept of a Unified Gateway within the Tetrate Service Bridge (TSB) ecosystem, explaining its significance and providing detailed usage scenarios.

Introduction

Unified Gateway is a critical feature introduced in TSB 1.7.0 that merges the capabilities of Tier1Gateway and IngressGateway under a common resource called Gateway. This unification simplifies the gateway management process and offers a more cohesive experience.

From TSB 1.7.0 onwards, Tier1Gateway and IngressGateway resources will be deprecated, and we strongly recommend using the Gateway resource for all your gateway needs. The former Tier1 Gateway will now be collectively known as Edge Gateway.

The Unified Gateways tab is seamlessly integrated into the TSB UI, allowing easy configuration of any gateway, whether it functions as a Tier 1 or Tier 2 gateway.

Why the Unified Gateway?

Early in our journey, we recognized the distinctive needs of our customers for both cluster-specific (Tier 2) and cross-cloud vendor (Tier1) gateways. As a result, we developed separate gateway solutions to cater to these varying demands. However, as our Gateway API evolved and customer requirements grew more complex, the need to continually enhance the capabilities of our Tier1 gateway became evident.

This evolution brought challenges—ongoing engineering efforts, customer education on when to choose Tier1 or Tier2, and maintaining parallel codebases. We've embarked on a groundbreaking endeavor: Unified Gateway to streamline these complexities and provide a more cohesive experience.

The Unified Gateway Advantage

Unified Gateway isn't just a fusion of Tier 1 and Tier 2 gateways; it's a paradigm shift in gateway management. Here's what you need to know about this game-changing solution:

Comprehensive Capabilities

Unified Gateway combines the robust capabilities of Tier 1 and Tier 2 gateways from TSB version 1.6.X, ensuring you get the best of both worlds. Whether dealing with retries, failover, or any other advanced functionality, Unified Gateway has you covered, regardless of whether it's configured as a Tier 1 or Tier 2 gateway.

Seamless Transition

For our existing customers, we understand the importance of continuity. Fear not, as your Tier 1 and Tier 2 gateways will continue to function as usual with the capabilities provided by version 1.6.X. But we're not stopping there. We're introducing a seamless process to transition your existing gateways to the Unified Gateway model, enhancing Tier 1 functionality such as retries and more.

New API for Unified Gateway

Embracing innovation doesn't mean neglecting the past. While introducing the new Unified Gateway API for fresh opportunities, we're committed to supporting the previous APIs for the subsequent three TSB releases. This ensures you can switch at your own pace and without disruption.

Empowering Direct Mode

Unified Gateway isn't just about gateways—it's about empowerment. Both new and existing customers can harness the full capabilities of the Gateway API via the Direct mode, offering unparalleled control and customization over their mesh infrastructure.

Aligned with Open API Strategy

We believe in the power of open standards. Unified Gateway aligns seamlessly with our Open API strategy, enabling you to configure Unified Gateways using a standardized Open API specification. This approach promotes consistency and simplifies integration with your existing toolchain.

Use Cases

Let's dive into the unified gateway usage scenarios.

Preparing Clusters

The following image shows the deployment architecture we use in this document. We created 3clusters in GKE, deployed TSB in one of them, loaded the other three clusters into TSB, and deployed the bookinfo application in the cluster with the infrastructure shown below.

The following table describes the roles and applications of these clusters:

Cluster	gke-jimmy-us-central1-1	gke-jimmy-us-west1-1	gke-jimmy-us-west1-2	gke-jimmy-us-west2-3
Region	us-central1	us-west1	us-west1	us-west2
TSB Role	Management Plane	Control Plane	Control Plane	Control Plane
Application	-	bookinfo-frontend	bookinfo-backend	httpbin
Services	-	productpage	productpage, ratings, reviews, details	httpbin
Network	tier1	cp-cluster-1	cp-cluster-2	cp-cluster-3

This section describes the unified gateway usage scenarios.

Scenario 1: Cluster-Based Routing with HTTP Path and Header Match

In this scenario, we will use the Gateway resource to expose bookinfo.tetrate.io and httpbin.tetrate.io. We will leverage cluster-based routing capabilities based on the path prefix on Gateway to route bookinfo frontend services to cp-cluster-1 and other backend services to cp-cluster-2. Using Gateway, users can expose multiple such hosts with clusterDestination, provided the host:port combination is unique.

Deployment Topology and Traffic Routing

We have set up the following deployment topology:

• Tier 1 Cluster: This cluster acts as the entry point for external traffic and routes it to the respective backend clusters.

• Backend Clusters: There are three backend clusters, each hosting different services:

  1. cp-cluster-1 hosts the frontend service of the "Bookinfo" application.
  2. cp-cluster-2 hosts the backend services of the "Bookinfo" application.
  3. cp-cluster-3 hosts an HTTP service called httpbin.

Configuration

1. Tier 1 Cluster Gateway (edge-gateway):

In the tier1 cluster, we deploy a Gateway named edge-gateway. This Gateway receives incoming traffic and routes it to the appropriate backend clusters based on the host and path prefix.

Here's a snippet of the configuration for routing requests to the "Bookinfo" frontend and backend services:

apiVersion: gateway.tsb.tetrate.io/v2
 kind: Gateway
 metadata:
   name: edge-gateway
   namespace: tier1
   annotations:
     tsb.tetrate.io/organization: tetrate
     tsb.tetrate.io/tenant: tier1
     tsb.tetrate.io/workspace: tier1
     tsb.tetrate.io/gatewayGroup: edge-gateway-group
 spec:
   workloadSelector:
     namespace: tier1
     labels:
       app: edge-gateway
   http:
   - name: bookinfo
     hostname: bookinfo.tetrate.io
     port: 80
     routing:
       rules:
         - match:
             - uri:
                 prefix: "/productpage"
               headers:
                 X-CLUSTER-SELECTOR:
                   exact: gke-jimmy-us-west1-1
           route:
             clusterDestination:
               clusters:
                 - name: gke-jimmy-us-west1-1
                   weight: 100
         - match:
             - uri:
                 prefix: "/productpage"
               headers:
                 X-CLUSTER-SELECTOR:
                   exact: gke-jimmy-us-west1-2
           route:
             clusterDestination:
               clusters:
                 - name: gke-jimmy-us-west1-2
                   weight: 100
         - match:
             - uri:
                 prefix: "/productpage"
           route:
             clusterDestination:
               clusters:
                 - name: gke-jimmy-us-west1-1
                   weight: 100
         - match:
             - uri:
                 prefix: "/api/v1/products"
           route:
             clusterDestination:
               clusters:
                 - name: gke-jimmy-us-west1-2
                   weight: 100
   - name: httpbin
     hostname: httpbin.tetrate.io
     port: 80
     routing:
       rules:
         - route:
             clusterDestination:
               clusters:
                 - name: gke-jimmy-us-west2-3
                   weight: 100

These rules ensure that requests to bookinfo.tetrate.io with different path prefixes are routed to the appropriate backend clusters. Similarly, requests to httpbin.tetrate.io are directed to the cp-cluster-3.

2. Ingress Gateways in Backend Clusters

In each backend cluster (cp-cluster-1, cp-cluster-2, and cp-cluster-3), we deploy Ingress Gateways to receive traffic from the tier1 cluster and route it to the respective services.

Here's an example of the Ingress Gateway configuration in cp-cluster-1:

apiVersion: gateway.tsb.tetrate.io/v2
kind: Gateway
metadata:
  name: bookinfo-ingress-gateway
spec:
  # ... (metadata and selectors)
  http:
    - hostname: bookinfo.tetrate.io
      name: bookinfo-tetrate
      port: 80
      routing:
        rules:
          - route:
              serviceDestination:
                host: bookinfo-frontend/productpage.bookinfo-frontend.svc.cluster.local

This configuration ensures that traffic received by the Ingress Gateway in cp-cluster-1 for bookinfo.tetrate.io is routed to the frontend service.

Verification

We can use tools like curl to request the exposed services to verify the setup. For example, to test /productpage:

export GATEWAY_IP=$(kubectl -n tier1 get service edge-gateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

curl -Ss "http://bookinfo.tetrate.io/productpage" --resolve "bookinfo.tetrate.io:80:$GATEWAY_IP" -v

Similarly, you can test other routes and services per the defined routing rules.

Scenario 2: Host-Based Routing with Gateway Header Rewriting

This scenario demonstrates the unified gateway's authority rewriting or header rewriting function. We deploy an edge gateways in the tier1 cluster to route traffic between different clusters and use IngressGateways for each of the three control plane clusters to receive traffic.

Deployment Topology and Traffic Routing

We have set up the following deployment topology:

• Tier 1 Cluster: This cluster serves as the entry point for external traffic and routes it to the respective backend clusters.

• Backend Clusters: There are three backend clusters, each hosting different services:

  1. cp-cluster-1 hosts the frontend service of the "Bookinfo" application.
  2. cp-cluster-2 hosts the backend services of the "Bookinfo" application.

Configuration

1. Tier 1 Cluster Gateway (tier1-gateway)

In the Tier 1 cluster, we deploy a Gateway named tier1-gateway. This Gateway receives incoming traffic and routes it to the appropriate backend clusters based on the host and path prefix. Additionally, it performs host header rewriting for specific routes.

Here's a snippet of the configuration for routing requests to the "Bookinfo" frontend and backend services with header rewriting:

apiVersion: gateway.tsb.tetrate.io/v2
kind: Gateway
metadata:
  name: tier1-gateway
  namespace: tier1
spec:
  # ... (metadata and selectors)
  http:
  - name: bookinfo
    hostname: bookinfo.tetrate.io
    port: 80
    routing:
      rules:
        - match:
            - uri:
                prefix: "/productpage"
          modify:
            rewrite:
              authority: 'internal-bookinfo-frontend.tetrate.io'
          route:
            clusterDestination:
              clusters:
                - name: gke-jimmy-us-west1-1
                  weight: 100
        - match:
            - uri:
                prefix: "/api/v1/products"
          modify:
            rewrite:
              authority: 'internal-bookinfo-backend.tetrate.io'
          route:
            clusterDestination:
              clusters:
                - name: gke-jimmy-us-west1-2
                  weight: 100

These rules ensure that requests to bookinfo.tetrate.io with different path prefixes are routed to the appropriate backend clusters. Additionally, the host header is rewritten for these routes.

2. Ingress Gateways in Backend Clusters

In each backend cluster (cp-cluster-1 and cp-cluster-2), we deploy Ingress Gateways to receive traffic from the tier1 cluster and route it to the respective services. These Ingress Gateways listen to the rewritten host headers.

Here's an example of the Ingress Gateway configuration in cp-cluster-1:

apiVersion: gateway.tsb.tetrate.io/v2
kind: Gateway
metadata:
  name: bookinfo-ingress-gateway
  annotations:
    tsb.tetrate.io/organization: tetrate
    tsb.tetrate.io/tenant: payment
    tsb.tetrate.io/workspace: bookinfo-frontend-ws
spec:
  displayName: Bookinfo Ingress
  workloadSelector:
    namespace: bookinfo-frontend
    labels:
      app: bookinfo-gateway
  http:
    - hostname: internal-bookinfo-frontend.tetrate.io
      name: bookinfo-tetrate
      port: 80
      routing:
        rules:
          - route:
              serviceDestination:
                host: bookinfo-frontend/productpage.bookinfo-frontend.svc.cluster.local

This configuration ensures that traffic received by the Ingress Gateway in cp-cluster-1 with the rewritten host header is routed to the frontend service.

Verification

We can use tools like curl to request the exposed services to verify the setup. For example, to test /productpage:

export GATEWAY_IP=$(kubectl -n tier1 get service tier1-gateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

curl -Ss "http://bookinfo.tetrate.io/productpage" --resolve "bookinfo.tetrate.io:80:$GATEWAY_IP" -v

Similarly, you can test other routes and services per the defined routing rules.

Restrict Exposed Hostnames for a Limited Set of Apps/Namespaces

When services are exposed via an IngressGateway, by default, all workloads deployed within the same cluster across all namespaces gain access to the exposed services. This is because the ServiceEntry, created for the exposed services, automatically exports them to all namespaces within the cluster.

However this behaviour can be restricted using hostReachability in WorkspaceSettings, where you can configure only those hostnames that needs to be exported for the application namespaces.

Here internal-bookinfo-backend.tetrate.io hostname is restricted only for app-1 workspace namespace selectors

  apiVersion: tsb.tetrate.io/v2
  kind: WorkspaceSetting
  metadata:
    name: app-1-wss
    annotations:
      tsb.tetrate.io/organization: tetrate
      tsb.tetrate.io/tenant: app-1
      tsb.tetrate.io/workspace: app-1-ws
  spec:
    hostsReachability:
      hostnames:
      - exact: "internal-bookinfo-backend.tetrate.io"

Similarly internal-bookinfo-frontend.tetrate.io is restricted to app-2 workspace namespace selectors

  apiVersion: tsb.tetrate.io/v2
  kind: WorkspaceSetting
  metadata:
    name: app-2-wss
    annotations:
      tsb.tetrate.io/organization: tetrate
      tsb.tetrate.io/tenant: app-2
      tsb.tetrate.io/workspace: app-2-ws
  spec:
    hostsReachability:
      hostnames:
      - exact: "internal-bookinfo-frontend.tetrate.io"

Please verify the exportTo attribute in the translated ServiceEntry objects created under xcp-multicluster namespace once you configure the above settings.

kubectl get se -n xcp-multicluster

NAME                                                           HOSTS                                           LOCATION        RESOLUTION   AGE
default-gateway-internal-bookinfo-backend-tetrate-io           ["internal-bookinfo-backend.tetrate.io"]        MESH_INTERNAL   STATIC       3h33m
default-gateway-internal-bookinfo-frontend-tetrate-io          ["internal-bookinfo-frontend.tetrate.io"]       MESH_INTERNAL   STATIC       3h33m

Scenario 3: Create UnifiedGateway using TSB Application & API with OpenAPI Spec

This scenario demonstrates how to create Unified Gateways for Tier 1 and Tier 2 use cases using OpenAPI Specification with Tetrate Service Bridge (TSB). This approach allows you to define traffic routing using OpenAPI Specs for your application.

Deployment Topology and Traffic Routing

We aim to configure a Unified Gateway using OpenAPI Specifications for traffic routing. The following diagram illustrates the deployment topology and routing setup:

Configuration Steps

1. Tier 1 Cluster Configuration

   In the tier1 cluster, we configure the Application and API resources using OpenAPI Specs. These configurations use the x-tsb-clusters annotation for cluster-based routing to expose bookinfo.tetrate.io.

   x-tsb-clusters:
     clusters:
       - name: gke-jimmy-us-west1-2
         weight: 100

   This configuration routes traffic to multiple Tier 2 clusters specified in the x-tsb-clusters annotation.

2. Tier 2 Cluster Configuration

   In the Tier 2 cluster (cp-cluster-2 in this scenario), we configure the Application and API resources with service-based routing to expose bookinfo.tetrate.io. This configuration uses the x-tsb-service annotation to route to the productpage.bookinfo-backend service.

   x-tsb-service: productpage.bookinfo-backend

Verification

To verify the routing setup, you can use tools like curl to make requests to the exposed services. For example, to test the /api/v1/products/* route:

# Export the Load Balancer IP of the tier1-gateway
export GATEWAY_IP=$(kubectl -n tier1 get service tier1-gateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

# Send a request to the API
curl -Ss "http://bookinfo.tetrate.io/api/v1/products/1/reviews" --resolve "bookinfo.tetrate.io:80:$GATEWAY_IP" -v

Scenario 4: Implementing HTTP to HTTPS Redirection

This scenario demonstrates how to configure a Gateway resource to implement HTTP to HTTPS redirection. This is useful for exposing services securely via HTTPS while supporting legacy applications that use plain HTTP.

Deployment Topology

We aim to expose bookinfo.tetrate.io on port 80 with plain text (HTTP), and we'll configure HTTP to HTTPS redirection on port 443 (HTTPS) to ensure secure communication. The following diagram illustrates the deployment topology and the routing setup:

Configuration Steps

1. Tier 1 Cluster Configuration

   In the tier1 cluster, we create a Gateway resource named tier1-gateway. This gateway is responsible for HTTP to HTTPS redirection. We specify two HTTP listeners:

   • bookinfo-plaintext: This listener listens on port 80 and handles requests for bookinfo.tetrate.io. It performs an HTTP to HTTPS redirection to port 443 (HTTPS) using a 301 redirect code.

   • bookinfo: This listener listens on port 443 (HTTPS) for secure communication. It uses TLS with a secret named bookinfo-certs.

   apiVersion: gateway.tsb.tetrate.io/v2
   kind: Gateway
   metadata:
     name: tier1-gateway
     namespace: tier1
     annotations:
       tsb.tetrate.io/organization: tetrate
       tsb.tetrate.io/tenant: tier1
       tsb.tetrate.io/workspace: tier1
       tsb.tetrate.io/gatewayGroup: tier1-gateway-group
   spec:
     workloadSelector:
       namespace: tier1
       labels:
         app: tier1-gateway
     http:
     - name: bookinfo-plaintext
       port: 80
       hostname: bookinfo.tetrate.io
       routing:
         rules:
           - redirect:
               authority: bookinfo.tetrate.io
               port: 443
               redirectCode: 301
               scheme: https
     - name: bookinfo
       hostname: bookinfo.tetrate.io
       port: 443
       tls:
         mode: SIMPLE
         secretName: bookinfo-certs
       routing:
         rules:
           - match:
               - uri:
                   prefix: "/productpage"
             route:
               clusterDestination:
                 clusters:
                   - name: gke-jimmy-us-west1-2
                     weight: 100

2. Tier 2 Cluster Configuration

   In the Tier 2 cluster (cp-cluster-2 in this scenario), we configure an IngressGateway with eastWestOnly: true. This setting exposes the mTLS 15443 multi-cluster port only. We also define a Gateway resource named bookinfo-ingress-gateway to route requests.

   apiVersion: install.tetrate.io/v1alpha1
   kind: IngressGateway
   metadata:
     name: bookinfo-gateway
   spec:
     eastWestOnly: true
     kubeSpec:
       service:
         type: LoadBalancer
   apiVersion: gateway.tsb.tetrate.io/v2
   kind: Gateway
   metadata:
     name: bookinfo-ingress-gateway
     annotations:
       tsb.tetrate.io/organization: tetrate
       tsb.tetrate.io/tenant: payment
       tsb.tetrate.io/workspace: bookinfo-backend-ws
       tsb.tetrate.io/gatewayGroup: bookinfo-gg
   spec:
     displayName: Bookinfo Ingress
     workloadSelector:
       namespace: bookinfo-backend
       labels:
         app: bookinfo-gateway
     http:
       - hostname: bookinfo.tetrate.io
         name: bookinfo-tetrate
         routing:
           rules:
             - route:
                 serviceDestination:
                   host: bookinfo-backend/productpage.bookinfo-backend.svc.cluster.local

Verification

To verify the HTTP to HTTPS redirection, follow these steps:

1. To view the redirection in your browser, you need to update your /etc/hosts file to make bookinfo.tetrate.io resolve to your Edge Gateway IP:

   export GATEWAY_IP=$(kubectl -n tier1 get service tier1-gateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
   echo "$GATEWAY_IP bookinfo.tetrate.io" | sudo tee -a /etc/hosts

2. Access http://bookinfo.tetrate.io/productpage in your browser. You should be automatically redirected to https://bookinfo.tetrate.io/productpage, ensuring secure communication.

Scenario 5: Configuring External Service

In this scenario, we use a Unified Gateway to configure a dedicated egress gateway for forwarding external service traffic. We create an httpbin namespace, deploy IngressGateway in two clusters (cp-cluster-1 and cp-cluster-2), and set up ServiceEntry to define external services.

Deployment Topology

This deployment involves creating a new httpbin namespace and IngressGateway in both cp-cluster-1 and cp-cluster-2. To enable these clusters to access the external service httpbin.org, we add a ServiceEntry in both clusters and configure the gateway to override the request's authority.

Configuration Steps

1. Create ServiceEntry and IstioInternalGroup

   Define an external service using ServiceEntry and associate it with an IstioInternalGroup. This configuration enables the clusters to access httpbin.org. We create these resources in both clusters.

   apiVersion: v1
   kind: List
   items:
     - apiVersion: tsb.tetrate.io/v2
       kind: Workspace
       metadata:
         name: httpbin-ws
         annotations:
           tsb.tetrate.io/organization: tetrate
           tsb.tetrate.io/tenant: payment
       spec:
         namespaceSelector:
           names:
             - "gke-jimmy-us-west1-1/httpbin"
             - "gke-jimmy-us-west1-2/httpbin"
         displayName: httpbin-ws
     - apiVersion: istiointernal.tsb.tetrate.io/v2
       kind: Group
       metadata:
         name: httpbin-internal-gp
         annotations:
           tsb.tetrate.io/organization: tetrate
           tsb.tetrate.io/tenant: payment
           tsb.tetrate.io/workspace: httpbin-ws
       spec:
         namespaceSelector:
           names:
             - "gke-jimmy-us-west1-1/httpbin"
             - "gke-jimmy-us-west1-2/httpbin"
     - apiVersion: networking.istio.io/v1beta1
       kind: ServiceEntry
       metadata:
         name: httpbin-external-svc
         annotations:
           tsb.tetrate.io/organization: tetrate
           tsb.tetrate.io/tenant: payment
           tsb.tetrate.io/workspace: httpbin-ws
           tsb.tetrate.io/istioInternalGroup: httpbin-internal-gp
         labels:
           istio.io/rev: tsb
       spec:
         hosts:
           - httpbin.org
         exportTo:
           - "."
         location: MESH_EXTERNAL
         ports:
           - number: 443
             name: https
             protocol: HTTPS
         resolution: DNS

2. Apply Gateway Configuration

   Configure the Gateway resource to rewrite requests from httpbin.tetrate.io to httpbin.org. We set up this configuration in both clusters, cp-cluster-1 and cp-cluster-2.

   - apiVersion: gateway.tsb.tetrate.io/v2
     kind: Gateway
     metadata:
       name: httpbin-ingress-gateway
       annotations:
         tsb.tetrate.io/organization: tetrate
         tsb.tetrate.io/tenant: payment
         tsb.tetrate.io/workspace: httpbin-ws
         tsb.tetrate.io/gatewayGroup: httpbin-gg
     spec:
       displayName: Httpbin Ingress
       workloadSelector:
         namespace: httpbin
         labels:
           app: httpbin-gateway
       http:
         - hostname: httpbin.tetrate.io
           name: httpbin-tetrate
           port: 80
           routing:
             rules:
               - modify:
                   rewrite:
                     authority: httpbin.org
                 route:
                   serviceDestination:
                     host: httpbin/httpbin.org
                     tls:
                       mode: SIMPLE
                       files:
                         caCertificates: "/etc/ssl/certs/ca-certificates.crt"

Verification

To verify this configuration, you can follow these steps:

1. To obtain the IP address of the IngressGateway in both cp-cluster-1 and cp-cluster-2, run the following command in each cluster:

   export GATEWAY_IP=$(kubectl -n httpbin get service httpbin-gateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

2. Use the two gateways to proxy access to httpbin.org by executing the following command:

   curl -v 'http://httpbin.tetrate.io/get' --resolve "httpbin.tetrate.io:80:$GATEWAY_IP" -v

This will demonstrate that requests to httpbin.tetrate.io are correctly forwarded to httpbin.org through the egress gateway.