Skip to main content
logoTetrate Service BridgeVersion: 1.9.x

Unified Gateway

This document introduces the concept of a Unified Gateway within the Tetrate Service Bridge (TSB) ecosystem, explaining its significance and providing detailed usage scenarios.

Introduction

Unified Gateway is a critical feature introduced in TSB 1.7.0 that merges the capabilities of Tier1Gateway and IngressGateway under a common resource called Gateway. This unification simplifies the gateway management process and offers a more cohesive experience.

From TSB 1.7.0 onwards, Tier1Gateway and IngressGateway resources will be deprecated, and we strongly recommend using the Gateway resource for all your gateway needs. The former Tier1 Gateway will now be collectively known as Edge Gateway.

The Unified Gateways tab is seamlessly integrated into the TSB UI, allowing easy configuration of any gateway, whether it functions as a Tier 1 or Tier 2 gateway.

Unified Gateway

Why the Unified Gateway?

Early in our journey, we recognized the distinctive needs of our customers for both cluster-specific (Tier 2) and cross-cloud vendor (Tier1) gateways. As a result, we developed separate gateway solutions to cater to these varying demands. However, as our Gateway API evolved and customer requirements grew more complex, the need to continually enhance the capabilities of our Tier1 gateway became evident.

This evolution brought challenges—ongoing engineering efforts, customer education on when to choose Tier1 or Tier2, and maintaining parallel codebases. We've embarked on a groundbreaking endeavor: Unified Gateway to streamline these complexities and provide a more cohesive experience.

The Unified Gateway Advantage

Unified Gateway isn't just a fusion of Tier 1 and Tier 2 gateways; it's a paradigm shift in gateway management. Here's what you need to know about this game-changing solution:

Comprehensive Capabilities

Unified Gateway combines the robust capabilities of Tier 1 and Tier 2 gateways from TSB version 1.6.X, ensuring you get the best of both worlds. Whether dealing with retries, failover, or any other advanced functionality, Unified Gateway has you covered, regardless of whether it's configured as a Tier 1 or Tier 2 gateway.

Seamless Transition

For our existing customers, we understand the importance of continuity. Fear not, as your Tier 1 and Tier 2 gateways will continue to function as usual with the capabilities provided by version 1.6.X. But we're not stopping there. We're introducing a seamless process to transition your existing gateways to the Unified Gateway model, enhancing Tier 1 functionality such as retries and more.

New API for Unified Gateway

Embracing innovation doesn't mean neglecting the past. While introducing the new Unified Gateway API for fresh opportunities, we're committed to supporting the previous APIs for the subsequent three TSB releases. This ensures you can switch at your own pace and without disruption.

Empowering Direct Mode

Unified Gateway isn't just about gateways—it's about empowerment. Both new and existing customers can harness the full capabilities of the Gateway API via the Direct mode, offering unparalleled control and customization over their mesh infrastructure.

Aligned with Open API Strategy

We believe in the power of open standards. Unified Gateway aligns seamlessly with our Open API strategy, enabling you to configure Unified Gateways using a standardized Open API specification. This approach promotes consistency and simplifies integration with your existing toolchain.

New API for Gateway Install

Starting from TSB version 1.9.0, Unified Gateway can be installed using a single Gateway API install resource for the following different use-cases.

apiVersion: install.tetrate.io/v1alpha1
kind: Gateway
metadata:
name: gateway
spec:
type: UNIFIED/INGRESS/EGRESS/EASTWEST
kubeSpec:
service:
type: LoadBalancer
FunctionalityDeployment TypeService TypeCustomizable toDefault Ports ExposedMulticluster Service EntriesIstio LabelAdditional Notes
UnifiedUNIFIED (default)LoadBalancerNodePort80(HTTP), 443(HTTPS), 15443(ISTIO_MTLS)Yes (Transit, Ingress, Egress hosts)istio: ingressgateway-
IngressINGRESSLoadBalancerNodePort80(HTTP), 443(HTTPS), 15443(ISTIO_MTLS)Yes (Transit, Ingress hosts)istio: ingressgatewayValidations: No EastWest or Egress Gateway
EgressEGRESSClusterIPNodePort/LoadBalancer80(HTTP), 443(HTTPS), 15443(ISTIO_MTLS)Noistio: egressgatewayValidations: No EastWest or Ingress Gateway
EastWestEASTWESTLoadBalancerNodePort15443(ISTIO_MTLS)Yes (Canonical services)istio: ingressgatewayValidations: No Ingress or Egress Gateway

Use Cases

Let's dive into the unified gateway usage scenarios.

Preparing Clusters

The following image shows the deployment architecture we use in this document. We created 3clusters in GKE, deployed TSB in one of them, loaded the other three clusters into TSB, and deployed the bookinfo application in the cluster with the infrastructure shown below.

The following table describes the roles and applications of these clusters:

Clustergke-jimmy-us-central1-1gke-jimmy-us-west1-1gke-jimmy-us-west1-2gke-jimmy-us-west2-3
Regionus-central1us-west1us-west1us-west2
TSB RoleManagement PlaneControl PlaneControl PlaneControl Plane
Application-bookinfo-frontendbookinfo-backendhttpbin
Services-productpageproductpage, ratings, reviews, detailshttpbin
Networktier1cp-cluster-1cp-cluster-2cp-cluster-3

This section describes the unified gateway usage scenarios.

Scenario 1: Cluster-Based Routing with HTTP Path and Header Match

In this scenario, we will use the Gateway resource to expose bookinfo.tetrate.io and httpbin.tetrate.io. We will leverage cluster-based routing capabilities based on the path prefix on Gateway to route bookinfo frontend services to cp-cluster-1 and other backend services to cp-cluster-2. Using Gateway, users can expose multiple such hosts with clusterDestination, provided the host:port combination is unique.

Cluster-Based Routing

Deployment Topology and Traffic Routing

We have set up the following deployment topology:

  • Tier 1 Cluster: This cluster acts as the entry point for external traffic and routes it to the respective backend clusters.

  • Backend Clusters: There are three backend clusters, each hosting different services:

    1. cp-cluster-1 hosts the frontend service of the "Bookinfo" application.
    2. cp-cluster-2 hosts the backend services of the "Bookinfo" application.
    3. cp-cluster-3 hosts an HTTP service called httpbin.

Configuration

1. Tier 1 Cluster Gateway (edge-gateway):

In the tier1 cluster, we deploy a Gateway named edge-gateway. This Gateway receives incoming traffic and routes it to the appropriate backend clusters based on the host and path prefix.

Gateway Install resource

apiVersion: install.tetrate.io/v1alpha1
kind: Gateway
metadata:
name: gateway
spec:
type: UNIFIED
kubeSpec:
service:
type: LoadBalancer

Here's a snippet of the configuration for routing requests to the "Bookinfo" frontend and backend services:

apiVersion: gateway.tsb.tetrate.io/v2
kind: Gateway
metadata:
name: edge-gateway
namespace: tier1
annotations:
tsb.tetrate.io/organization: tetrate
tsb.tetrate.io/tenant: tier1
tsb.tetrate.io/workspace: tier1
tsb.tetrate.io/gatewayGroup: edge-gateway-group
spec:
workloadSelector:
namespace: tier1
labels:
app: edge-gateway
http:
- name: bookinfo
hostname: bookinfo.tetrate.io
port: 80
routing:
rules:
- match:
- uri:
prefix: "/productpage"
headers:
X-CLUSTER-SELECTOR:
exact: gke-jimmy-us-west1-1
route:
clusterDestination:
clusters:
- name: gke-jimmy-us-west1-1
weight: 100
- match:
- uri:
prefix: "/productpage"
headers:
X-CLUSTER-SELECTOR:
exact: gke-jimmy-us-west1-2
route:
clusterDestination:
clusters:
- name: gke-jimmy-us-west1-2
weight: 100
- match:
- uri:
prefix: "/productpage"
route:
clusterDestination:
clusters:
- name: gke-jimmy-us-west1-1
weight: 100
- match:
- uri:
prefix: "/api/v1/products"
route:
clusterDestination:
clusters:
- name: gke-jimmy-us-west1-2
weight: 100
- name: httpbin
hostname: httpbin.tetrate.io
port: 80
routing:
rules:
- route:
clusterDestination:
clusters:
- name: gke-jimmy-us-west2-3
weight: 100

These rules ensure that requests to bookinfo.tetrate.io with different path prefixes are routed to the appropriate backend clusters. Similarly, requests to httpbin.tetrate.io are directed to the cp-cluster-3.

2. Ingress Gateways in Backend Clusters

In each backend cluster (cp-cluster-1, cp-cluster-2, and cp-cluster-3), we deploy Ingress Gateways to receive traffic from the tier1 cluster and route it to the respective services.

Gateway Install resource

apiVersion: install.tetrate.io/v1alpha1
kind: Gateway
metadata:
name: gateway
spec:
type: UNIFIED
kubeSpec:
service:
type: LoadBalancer

Here's an example of the Ingress Gateway configuration in cp-cluster-1:

apiVersion: gateway.tsb.tetrate.io/v2
kind: Gateway
metadata:
name: bookinfo-ingress-gateway
spec:
# ... (metadata and selectors)
http:
- hostname: bookinfo.tetrate.io
name: bookinfo-tetrate
port: 80
routing:
rules:
- route:
serviceDestination:
host: bookinfo-frontend/productpage.bookinfo-frontend.svc.cluster.local

This configuration ensures that traffic received by the Ingress Gateway in cp-cluster-1 for bookinfo.tetrate.io is routed to the frontend service.

Verification

We can use tools like curl to request the exposed services to verify the setup. For example, to test /productpage:

export GATEWAY_IP=$(kubectl -n tier1 get service edge-gateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

curl -Ss -H "X-B3-Sampled: 1" "http://bookinfo.tetrate.io/productpage" --resolve "bookinfo.tetrate.io:80:$GATEWAY_IP" -v

Similarly, you can test other routes and services per the defined routing rules.

Scenario 2: Host-Based Routing with Gateway Header Rewriting

This scenario demonstrates the unified gateway's authority rewriting or header rewriting function. We deploy an edge gateways in the tier1 cluster to route traffic between different clusters and use IngressGateways for each of the three control plane clusters to receive traffic.

Host-Based Routing

Deployment Topology and Traffic Routing

We have set up the following deployment topology:

  • Tier 1 Cluster: This cluster serves as the entry point for external traffic and routes it to the respective backend clusters.

  • Backend Clusters: There are three backend clusters, each hosting different services:

    1. cp-cluster-1 hosts the frontend service of the "Bookinfo" application.
    2. cp-cluster-2 hosts the backend services of the "Bookinfo" application.

Configuration

1. Tier 1 Cluster Gateway (tier1-gateway)

In the Tier 1 cluster, we deploy a Gateway named tier1-gateway. This Gateway receives incoming traffic and routes it to the appropriate backend clusters based on the host and path prefix. Additionally, it performs host header rewriting for specific routes.

Gateway Install resource

apiVersion: install.tetrate.io/v1alpha1
kind: Gateway
metadata:
name: gateway
spec:
type: UNIFIED
kubeSpec:
service:
type: LoadBalancer

Here's a snippet of the configuration for routing requests to the "Bookinfo" frontend and backend services with header rewriting:

apiVersion: gateway.tsb.tetrate.io/v2
kind: Gateway
metadata:
name: tier1-gateway
namespace: tier1
spec:
# ... (metadata and selectors)
http:
- name: bookinfo
hostname: bookinfo.tetrate.io
port: 80
routing:
rules:
- match:
- uri:
prefix: "/productpage"
modify:
rewrite:
authority: 'internal-bookinfo-frontend.tetrate.io'
route:
clusterDestination:
clusters:
- name: gke-jimmy-us-west1-1
weight: 100
- match:
- uri:
prefix: "/api/v1/products"
modify:
rewrite:
authority: 'internal-bookinfo-backend.tetrate.io'
route:
clusterDestination:
clusters:
- name: gke-jimmy-us-west1-2
weight: 100

These rules ensure that requests to bookinfo.tetrate.io with different path prefixes are routed to the appropriate backend clusters. Additionally, the host header is rewritten for these routes.

2. Ingress Gateways in Backend Clusters

In each backend cluster (cp-cluster-1 and cp-cluster-2), we deploy Ingress Gateways to receive traffic from the tier1 cluster and route it to the respective services. These Ingress Gateways listen to the rewritten host headers.

Gateway Install resource

apiVersion: install.tetrate.io/v1alpha1
kind: Gateway
metadata:
name: gateway
spec:
type: UNIFIED
kubeSpec:
service:
type: LoadBalancer

Here's an example of the Ingress Gateway configuration in cp-cluster-1:

apiVersion: gateway.tsb.tetrate.io/v2
kind: Gateway
metadata:
name: bookinfo-ingress-gateway
annotations:
tsb.tetrate.io/organization: tetrate
tsb.tetrate.io/tenant: payment
tsb.tetrate.io/workspace: bookinfo-frontend-ws
spec:
displayName: Bookinfo Ingress
workloadSelector:
namespace: bookinfo-frontend
labels:
app: bookinfo-gateway
http:
- hostname: internal-bookinfo-frontend.tetrate.io
name: bookinfo-tetrate
port: 80
routing:
rules:
- route:
serviceDestination:
host: bookinfo-frontend/productpage.bookinfo-frontend.svc.cluster.local

This configuration ensures that traffic received by the Ingress Gateway in cp-cluster-1 with the rewritten host header is routed to the frontend service.

Verification

We can use tools like curl to request the exposed services to verify the setup. For example, to test /productpage:

export GATEWAY_IP=$(kubectl -n tier1 get service tier1-gateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

curl -Ss -H "X-B3-Sampled: 1" "http://bookinfo.tetrate.io/productpage" --resolve "bookinfo.tetrate.io:80:$GATEWAY_IP" -v

Similarly, you can test other routes and services per the defined routing rules.

Restrict Exposed Hostnames for a Limited Set of Apps/Namespaces

When services are exposed via an IngressGateway, by default, all workloads deployed within the same cluster across all namespaces gain access to the exposed services. This is because the ServiceEntry, created for the exposed services, automatically exports them to all namespaces within the cluster.

However this behaviour can be restricted using hostReachability in WorkspaceSettings, where you can configure only those hostnames that needs to be exported for the application namespaces.

Here internal-bookinfo-backend.tetrate.io hostname is restricted only for app-1 workspace namespace selectors

  apiVersion: tsb.tetrate.io/v2
kind: WorkspaceSetting
metadata:
name: app-1-wss
annotations:
tsb.tetrate.io/organization: tetrate
tsb.tetrate.io/tenant: app-1
tsb.tetrate.io/workspace: app-1-ws
spec:
hostsReachability:
hostnames:
- exact: "internal-bookinfo-backend.tetrate.io"

Similarly internal-bookinfo-frontend.tetrate.io is restricted to app-2 workspace namespace selectors

  apiVersion: tsb.tetrate.io/v2
kind: WorkspaceSetting
metadata:
name: app-2-wss
annotations:
tsb.tetrate.io/organization: tetrate
tsb.tetrate.io/tenant: app-2
tsb.tetrate.io/workspace: app-2-ws
spec:
hostsReachability:
hostnames:
- exact: "internal-bookinfo-frontend.tetrate.io"

Please verify the exportTo attribute in the translated ServiceEntry objects created under xcp-multicluster namespace once you configure the above settings.

kubectl get se -n xcp-multicluster

NAME HOSTS LOCATION RESOLUTION AGE
default-gateway-internal-bookinfo-backend-tetrate-io ["internal-bookinfo-backend.tetrate.io"] MESH_INTERNAL STATIC 3h33m
default-gateway-internal-bookinfo-frontend-tetrate-io ["internal-bookinfo-frontend.tetrate.io"] MESH_INTERNAL STATIC 3h33m

Scenario 3: Create UnifiedGateway using TSB Application & API with OpenAPI Spec

This scenario demonstrates how to create Unified Gateways for Tier 1 and Tier 2 use cases using OpenAPI Specification with Tetrate Service Bridge (TSB). This approach allows you to define traffic routing using OpenAPI Specs for your application.

Deployment Topology and Traffic Routing

We aim to configure a Unified Gateway using OpenAPI Specifications for traffic routing. The following diagram illustrates the deployment topology and routing setup:

Create UnifiedGateway using TSB Application & API with OpenAPI Spec

Configuration Steps

  1. Tier 1 Cluster Configuration

    In the tier1 cluster, we configure the Application and API resources using OpenAPI Specs. These configurations use the x-tsb-clusters annotation for cluster-based routing to expose bookinfo.tetrate.io.

    x-tsb-clusters:
    clusters:
    - name: gke-jimmy-us-west1-2
    weight: 100

    This configuration routes traffic to multiple Tier 2 clusters specified in the x-tsb-clusters annotation.

  2. Tier 2 Cluster Configuration

    In the Tier 2 cluster (cp-cluster-2 in this scenario), we configure the Application and API resources with service-based routing to expose bookinfo.tetrate.io. This configuration uses the x-tsb-service annotation to route to the productpage.bookinfo-backend service.

    x-tsb-service: productpage.bookinfo-backend

Verification

To verify the routing setup, you can use tools like curl to make requests to the exposed services. For example, to test the /api/v1/products/* route:

# Export the Load Balancer IP of the tier1-gateway
export GATEWAY_IP=$(kubectl -n tier1 get service tier1-gateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

# Send a request to the API
curl -Ss -H "X-B3-Sampled: 1" "http://bookinfo.tetrate.io/api/v1/products/1/reviews" --resolve "bookinfo.tetrate.io:80:$GATEWAY_IP" -v

Scenario 4: Implementing HTTP to HTTPS Redirection

This scenario demonstrates how to configure a Gateway resource to implement HTTP to HTTPS redirection. This is useful for exposing services securely via HTTPS while supporting legacy applications that use plain HTTP.

Deployment Topology

We aim to expose bookinfo.tetrate.io on port 80 with plain text (HTTP), and we'll configure HTTP to HTTPS redirection on port 443 (HTTPS) to ensure secure communication. The following diagram illustrates the deployment topology and the routing setup:

HTTP to HTTPS Redirection

Configuration Steps

  1. Tier 1 Cluster Configuration

    In the tier1 cluster, we create a Gateway resource named tier1-gateway. This gateway is responsible for HTTP to HTTPS redirection. We specify two HTTP listeners:

    • bookinfo-plaintext: This listener listens on port 80 and handles requests for bookinfo.tetrate.io. It performs an HTTP to HTTPS redirection to port 443 (HTTPS) using a 301 redirect code.

    • bookinfo: This listener listens on port 443 (HTTPS) for secure communication. It uses TLS with a secret named bookinfo-certs.

     apiVersion: install.tetrate.io/v1alpha1
    kind: Gateway
    metadata:
    name: gateway
    spec:
    type: UNIFIED
    kubeSpec:
    service:
    type: LoadBalancer
    apiVersion: gateway.tsb.tetrate.io/v2
    kind: Gateway
    metadata:
    name: tier1-gateway
    namespace: tier1
    annotations:
    tsb.tetrate.io/organization: tetrate
    tsb.tetrate.io/tenant: tier1
    tsb.tetrate.io/workspace: tier1
    tsb.tetrate.io/gatewayGroup: tier1-gateway-group
    spec:
    workloadSelector:
    namespace: tier1
    labels:
    app: tier1-gateway
    http:
    - name: bookinfo-plaintext
    port: 80
    hostname: bookinfo.tetrate.io
    routing:
    rules:
    - redirect:
    authority: bookinfo.tetrate.io
    port: 443
    redirectCode: 301
    scheme: https
    - name: bookinfo
    hostname: bookinfo.tetrate.io
    port: 443
    tls:
    mode: SIMPLE
    secretName: bookinfo-certs
    routing:
    rules:
    - match:
    - uri:
    prefix: "/productpage"
    route:
    clusterDestination:
    clusters:
    - name: gke-jimmy-us-west1-2
    weight: 100
  2. Tier 2 Cluster Configuration

    In the Tier 2 cluster (cp-cluster-2 in this scenario), we configure an IngressGateway with eastWestOnly: true. This setting exposes the mTLS 15443 multi-cluster port only. We also define a Gateway resource named bookinfo-ingress-gateway to route requests.

     apiVersion: install.tetrate.io/v1alpha1
    kind: Gateway
    metadata:
    name: gateway
    spec:
    type: UNIFIED
    kubeSpec:
    service:
    type: LoadBalancer
    apiVersion: install.tetrate.io/v1alpha1
    kind: IngressGateway
    metadata:
    name: bookinfo-gateway
    spec:
    eastWestOnly: true
    kubeSpec:
    service:
    type: LoadBalancer
    apiVersion: gateway.tsb.tetrate.io/v2
    kind: Gateway
    metadata:
    name: bookinfo-ingress-gateway
    annotations:
    tsb.tetrate.io/organization: tetrate
    tsb.tetrate.io/tenant: payment
    tsb.tetrate.io/workspace: bookinfo-backend-ws
    tsb.tetrate.io/gatewayGroup: bookinfo-gg
    spec:
    displayName: Bookinfo Ingress
    workloadSelector:
    namespace: bookinfo-backend
    labels:
    app: bookinfo-gateway
    http:
    - hostname: bookinfo.tetrate.io
    name: bookinfo-tetrate
    routing:
    rules:
    - route:
    serviceDestination:
    host: bookinfo-backend/productpage.bookinfo-backend.svc.cluster.local

Verification

To verify the HTTP to HTTPS redirection, follow these steps:

  1. To view the redirection in your browser, you need to update your /etc/hosts file to make bookinfo.tetrate.io resolve to your Edge Gateway IP:

    export GATEWAY_IP=$(kubectl -n tier1 get service tier1-gateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
    echo "$GATEWAY_IP bookinfo.tetrate.io" | sudo tee -a /etc/hosts
  2. Access http://bookinfo.tetrate.io/productpage in your browser. You should be automatically redirected to https://bookinfo.tetrate.io/productpage, ensuring secure communication.

Scenario 5: Configuring External Service

In this scenario, we use a Unified Gateway to configure a dedicated egress gateway for forwarding external service traffic. We create an httpbin namespace, deploy IngressGateway in two clusters (cp-cluster-1 and cp-cluster-2), and set up ServiceEntry to define external services.

Deployment Topology

This deployment involves creating a new httpbin namespace and IngressGateway in both cp-cluster-1 and cp-cluster-2. To enable these clusters to access the external service httpbin.org, we add a ServiceEntry in both clusters and configure the gateway to override the request's authority.

Configure External Service

Configuration Steps

  1. Create ServiceEntry and IstioInternalGroup

    Define an external service using ServiceEntry and associate it with an IstioInternalGroup. This configuration enables the clusters to access httpbin.org. We create these resources in both clusters.

    apiVersion: v1
    kind: List
    items:
    - apiVersion: tsb.tetrate.io/v2
    kind: Workspace
    metadata:
    name: httpbin-ws
    annotations:
    tsb.tetrate.io/organization: tetrate
    tsb.tetrate.io/tenant: payment
    spec:
    namespaceSelector:
    names:
    - "gke-jimmy-us-west1-1/httpbin"
    - "gke-jimmy-us-west1-2/httpbin"
    displayName: httpbin-ws
    - apiVersion: istiointernal.tsb.tetrate.io/v2
    kind: Group
    metadata:
    name: httpbin-internal-gp
    annotations:
    tsb.tetrate.io/organization: tetrate
    tsb.tetrate.io/tenant: payment
    tsb.tetrate.io/workspace: httpbin-ws
    spec:
    namespaceSelector:
    names:
    - "gke-jimmy-us-west1-1/httpbin"
    - "gke-jimmy-us-west1-2/httpbin"
    - apiVersion: networking.istio.io/v1beta1
    kind: ServiceEntry
    metadata:
    name: httpbin-external-svc
    annotations:
    tsb.tetrate.io/organization: tetrate
    tsb.tetrate.io/tenant: payment
    tsb.tetrate.io/workspace: httpbin-ws
    tsb.tetrate.io/istioInternalGroup: httpbin-internal-gp
    labels:
    istio.io/rev: tsb
    spec:
    hosts:
    - httpbin.org
    exportTo:
    - "."
    location: MESH_EXTERNAL
    ports:
    - number: 443
    name: https
    protocol: HTTPS
    resolution: DNS
  2. Apply Gateway Configuration

    Configure the Gateway resource to rewrite requests from httpbin.tetrate.io to httpbin.org. We set up this configuration in both clusters, cp-cluster-1 and cp-cluster-2.

     apiVersion: install.tetrate.io/v1alpha1
    kind: Gateway
    metadata:
    name: gateway
    spec:
    type: UNIFIED
    kubeSpec:
    service:
    type: LoadBalancer
    - apiVersion: gateway.tsb.tetrate.io/v2
    kind: Gateway
    metadata:
    name: httpbin-ingress-gateway
    annotations:
    tsb.tetrate.io/organization: tetrate
    tsb.tetrate.io/tenant: payment
    tsb.tetrate.io/workspace: httpbin-ws
    tsb.tetrate.io/gatewayGroup: httpbin-gg
    spec:
    displayName: Httpbin Ingress
    workloadSelector:
    namespace: httpbin
    labels:
    app: httpbin-gateway
    http:
    - hostname: httpbin.tetrate.io
    name: httpbin-tetrate
    port: 80
    routing:
    rules:
    - modify:
    rewrite:
    authority: httpbin.org
    route:
    serviceDestination:
    host: httpbin/httpbin.org
    tls:
    mode: SIMPLE
    files:
    caCertificates: "/etc/ssl/certs/ca-certificates.crt"

Verification

To verify this configuration, you can follow these steps:

  1. To obtain the IP address of the IngressGateway in both cp-cluster-1 and cp-cluster-2, run the following command in each cluster:
 export GATEWAY_IP=$(kubectl -n httpbin get service httpbin-gateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
  1. Use the two gateways to proxy access to httpbin.org by executing the following command:
 curl -v -H "X-B3-Sampled: 1" 'http://httpbin.tetrate.io/get' --resolve "httpbin.tetrate.io:80:$GATEWAY_IP" -v

This will demonstrate that requests to httpbin.tetrate.io are correctly forwarded to httpbin.org through the egress gateway.

Scenario 6: Configure EgressGateway with TLS Origination for External Services

In this scenario, we use a Unified Gateway to configure a dedicated egress gateway for forwarding external service traffic with TLS origination.

Prerequisites

  1. Self-signed certs for bookinfo.tetrate.com generated as per certificate-for-gateway.

  2. bookinfo.tetrate.com accessible as an external endpoint. For this setup we used k8s cluster with Istio installed, bookinfo app, IngressGW exposing bookinfo.tetrate.com over port 443, TLS mode: SIMPLE with secret name bookinfo-tls-server-secret. You can create the secret using this command:

  kubectl -n bookinfo create secret tls bookinfo-tls-server-secret \
--key bookinfo.key \
--cert bookinfo.crt
  1. TSB 1.9.0 environment is up and running, and GitOps has been enabled for the target cluster
    .

Deployment Topology

This deployment involves creating a new egressgw-ns namespace and EgressGateway in cp-cluster-1. To enable the sleep container to access the external service bookinfo.tetrate.com with TLS origination, we add a ServiceEntry in cp-cluster-1 and configure the gateway to override the request's authority.

Unified Gateway as Egress with TLS Origination

Apply TSB Configuration

bookinfo-conf.yaml
apiVersion: v1
kind: List
items:
- apiVersion: tsb.tetrate.io/v2
kind: Tenant
metadata:
name: payment
annotations:
tsb.tetrate.io/organization: tetrate
spec:
displayName: egressgw
- apiVersion: tsb.tetrate.io/v2
kind: Workspace
metadata:
name: egressgw-ws
annotations:
tsb.tetrate.io/organization: tetrate
tsb.tetrate.io/tenant: payment
spec:
displayName: egressgw-ws
namespaceSelector:
names:
- "*/egressgw-ns"
- apiVersion: security.tsb.tetrate.io/v2
kind: Group
metadata:
name: egressgw-gg
annotations:
tsb.tetrate.io/organization: tetrate
tsb.tetrate.io/tenant: payment
tsb.tetrate.io/workspace: egressgw-ws
spec:
displayName: egressgw-gg
namespaceSelector:
names:
- "*/egressgw-ns"
- apiVersion: istiointernal.tsb.tetrate.io/v2
kind: Group
metadata:
name: egressgw-ig
namespace: egressgw-ns
annotations:
tsb.tetrate.io/organization: tetrate
tsb.tetrate.io/tenant: payment
tsb.tetrate.io/workspace: egressgw-ws
spec:
namespaceSelector:
names:
- "*/egressgw-ns"

Apply EgressGateway Configuration

Create Egress Gateway Namespace

  kubectl create ns egressgw-ns

Apply EgressGateway deployment configuration

egress-deployment.yaml
  apiVersion: install.tetrate.io/v1alpha1
kind: EgressGateway
metadata:
name: egressgw
namespace: egressgw-ns
spec:
kubeSpec:
deployment:
labels:
app: egressgw
service:
type: NodePort #specifiies the type of resrouce that will be created in the cloud provider (AWS)
annotations:
traffic.istio.io/nodeSelector: '{"beta.kubernetes.io/os": "linux"}'

Create k8s secret for TLS client side that will be used by Egress GW.

  kubectl -n egressgw-ns create secret generic bookinfo-tls-client-secret --from-file=ca.crt=bookinfo.crt

Apply EgressGateway configuration

egress-gw-config.yaml
  apiVersion: gateway.tsb.tetrate.io/v2
kind: Gateway
metadata:
name: egressgw
namespace: egressgw-ns
annotations:
tsb.tetrate.io/organization: tetrate
tsb.tetrate.io/tenant: payment
tsb.tetrate.io/workspace: egressgw-ws
tsb.tetrate.io/gatewayGroup: egressgw-gg
spec:
workloadSelector:
namespace: egressgw-ns
labels:
app: egressgw
http:
- name: bookinfo
port: 443
hostname: "bookinfo.tetrate.com"
routing:
rules:
- modify:
rewrite:
authority: bookinfo.tetrate.com
route:
serviceDestination:
host: sleep/bookinfo.tetrate.com
port: 443
tls:
mode: SIMPLE
secretName: bookinfo-tls-client-secret

Deploy sleep service in the same cluster to generate traffic from its container to external bookinfo.tetrate.com via Egress Gateway with TLS origination.

  kubectl create ns sleep
kubectl label namespace sleep istio-injection=enabled
kubectl apply -n sleep -f https://docs.tetrate.io/assets/files/sleep-0e507364b65e01029474dddf45bc15df.yaml

Verification

Generate traffic from your TSB managed service mesh to external endpoint bookinfo.tetrate.com via Egress Gateway with TLS origination.

Run the below command from the sleep container.

  kubect exec -it -n sleep sleep-755c856df8-z5h5n sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.

~ $ while true; do curl http://bookinfo.tetrate.com -vvv -I ; sleep 1; done
* Host bookinfo.tetrate.com:80 was resolved.
* IPv6: (none)
* IPv4: 240.240.102.57
* Trying 240.240.102.57:80...
* Connected to bookinfo.tetrate.com (240.240.102.57) port 80
> HEAD / HTTP/1.1
> Host: bookinfo.tetrate.com
> User-Agent: curl/8.5.0
> Accept: */*
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< content-type: text/html; charset=utf-8
content-type: text/html; charset=utf-8
< content-length: 1683
content-length: 1683
< server: envoy
server: envoy
< date: Fri, 02 Feb 2024 20:33:49 GMT
date: Fri, 02 Feb 2024 20:33:49 GMT
< x-envoy-upstream-service-time: 18
x-envoy-upstream-service-time: 18

As you can see, curl returns 200 - means we can successfully access bookinfo.

verify traffic

You need to run curl to http (port 80) as the traffic within your service mesh is encrypted using ISTIO_MUTUAL mode. When traffic hits Egress Gateway it will be converted to TLS encryption and port 443 using credentials from k8s bookinfo-tls-client-secret secret generated earlier.

Also, the request is sent to an automatically allocated address, 240.240.0.1. These addresses will be picked from the 240.240.0.0/16 reserved IP address range to avoid conflicting with real services. See Istio doc for more information.

Verify Egress Gateway logs.

  kubectl get pods -n egressgw
NAME READY STATUS RESTARTS AGE
egressgw-7c56dc8b54-b4qhk 1/1 Running 0 2d4h

k logs -n egressgw-7c56dc8b54-b4qhk
[2024-02-02T20:33:49.190Z] "HEAD / HTTP/1.1" 200 - via_upstream - "-" 0 0 11 11 "172.21.25.49" "curl/8.5.0" "e48f4963-6d74-4db8-8c58-b5e7d2118d0d" "bookinfo.tetrate.com" "34.250.72.75:443" outbound|443||bookinfo.tetrate.com 172.21.4.217:34466 172.21.4.217:15443 172.21.25.49:34276 bookinfo.tetrate.com egressgw-bookinfo-external-0
[2024-02-02T20:33:50.208Z] "HEAD / HTTP/1.1" 200 - via_upstream - "-" 0 0 5 5 "172.21.25.49" "curl/8.5.0" "e4df8b47-01fa-408f-aa2e-8b69e5246839" "bookinfo.tetrate.com" "34.250.72.75:443" outbound|443||bookinfo.tetrate.com 172.21.4.217:34466 172.21.4.217:15443 172.21.25.49:34276 bookinfo.tetrate.com egressgw-bookinfo-external-0
[2024-02-02T20:33:51.222Z] "HEAD / HTTP/1.1" 200 - via_upstream - "-" 0 0 7 7 "172.21.25.49" "curl/8.5.0" "d0c29a43-db58-4b46-9c08-1d9ec9395ab6" "bookinfo.tetrate.com" "34.250.72.75:443" outbound|443||bookinfo.tetrate.com 172.21.4.217:46150 172.21.4.217:15443 172.21.25.49:52614 bookinfo.tetrate.com egressgw-bookinfo-external-0

As you can see from the logs, traffic generated from sleep container first hits the Egress Gateway and then goes to external bookinfo.tetrate.com endpoint resolving the IP address configured in bookinfo Service Entry which is our Ingress Gateway exposing bookinfo application on 34.250.72.75.

Troubleshooting

While using PKI generated certificates please check the ROOT_CA for both client and server workloads and ensure they are the same. Also, you may need to include the chain in your ca.crt before you create k8s secret. If the certificate is wrong you won’t be able to access your endpoint and might see this log in Egress GW pod

  [2024-02-01T19:00:35.364Z] "HEAD / HTTP/1.1" 503 UF,URX upstream_reset_before_response_started{remote_connection_failure,TLS_error:_268435581:SSL_routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED} - "TLS_error:_268435581:SSL_routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED" 0 0 113 - "10.76.33.96" "curl/8.6.0" "724ad8de-73d5-42c8-8c94-d5eb81075b2f" "bookinfo.tetrate.com" "10.53.84.5:443" outbound|443||bookinfo.tetrate.com - 10.76.34.113:15443 10.76.33.96:41084 bookinfo.tetrate.com egressgw-bookinfo-external-0
[2024-02-01T19:00:35.509Z] "HEAD / HTTP/1.1" 503 UH no_healthy_upstream - "-" 0 0 0 - "10.76.33.96" "curl/8.6.0" "724ad8de-73d5-42c8-8c94-d5eb81075b2f" "bookinfo.tetrate.com" "-" outbound|443||bookinfo.tetrate.com - 10.76.34.113:15443 10.76.33.96:41084 bookinfo.tetrate.com egressgw-bookinfo-external-0