Using the Gateway API with Tetrate Service Bridge

Beta Functionality

Gateway API support with TSB is currently at a Beta stage.

Tetrate will provide technical support for customers who wish to use this functionality. For more information on the Gateway API concept, refer to the Gateway API comparison.

Options for Configuring Gateways in TSB

TSB provides a 'briged-mode' Gateway Resource which is the supported method to configure Application (Tier2) and Edge (Tier1) gateways in a multi-cluster TSB deployment. Bridged-Mode APIs are provided by the TSB platform and support multi-tenant, multi-cluster deployments with high-level configuration and built-in safeguards.

As an alternative, Tetrate supports the use of the emerging Gateway APIs to configure local gateways, using TSB's 'direct-mode' APIs. Direct-Mode APIs provide direct access to underlying per-cluster Istio configuration, but without the safely and consistency of their bridged-mode alternatives.

This guide explains how to configure your cluster to use the Gateway API.

Supported versions

The following instructions are supported with TSB 1.12.1 and later.

Onboarding a Cluster

When you onboard a cluster that will use the Gateway API, there are some additional steps to take to enable this functionality;

1. Prepare the Cluster Configuration

   Prepare the cluster configuration ${CLUSTER_NAME}-controlplane_values.yaml as follows.

   Set the installation parameters

   Set these as appropriate for your installation. Use TSB version 1.12.1 or later:

   export CLUSTER_NAME="app-tetrate-east-0"
   export REGISTRY="us-east1-docker.pkg.dev/tetrateio/registryrepository"
   export TSB_VERSION="1.12.1"

   Register the cluster with the TSB Management Plane

   # Register the cluster with the TSB Management Plane
   tctl install cluster-service-account --cluster ${CLUSTER_NAME}
   
   # Obtain the values.yaml configuration file for Helm installation
   tctl x cluster-install-template --with-isolation-boundary ${CLUSTER_NAME} > ${CLUSTER_NAME}-controlplane_values.yaml

   Patch the $CLUSTER_NAME-controlplane_values.yaml

   Apply to an existing controlplane

   The following instructions explain how to install the controlplane with the require settings.

   You can also apply these settings to an existing controlplane, with kubectl edit -n istio controlplane controlplane or similar.

   # Patch the controlplane values to enable Gateway API support
   cat << EOF > controlplane-patch.yaml
   spec:
     components:
       istio:
         ambient:
           enabled: true
         mountInternalWasmExtensions: false
       xcp:
         isolationBoundaries:
         - name: global
           revisions:
           - name: default
             istio: 
               kubeSpec:
                 overlays:
                 - apiVersion: install.istio.io/v1alpha1
                   kind: IstioOperator
                   name: xcp-iop-default
                   patches:
                   - path: spec.components.pilot.k8s.env[-1]
                     value:
                       name: PILOT_ENABLE_ALPHA_GATEWAY_API
                       value: "true"
         kubeSpec:
           deployment:
             env:
             - name: DISABLE_K8S_GATEWAY_API_CRD_CREATION
               value: "true"
           overlays:
           - apiVersion: install.xcp.tetrate.io/v1alpha1
             kind: EdgeXcp
             name: edge-xcp
             patches:
             - path: spec.components.edgeServer.kubeSpec.overlays
               value:
               - apiVersion: rbac.authorization.k8s.io/v1
                 kind: ClusterRoleBinding
                 name: istio-system
                 patches:
                 - path: metadata.name
                   value: xcp-edge-istio-system
   EOF

   Note: you will need a recent build of yq (e.g. 4.44.x)

   yq eval-all 'select(fileIndex == 0) * select(fileIndex == 1)' ${CLUSTER_NAME}-controlplane_values.yaml controlplane-patch.yaml > ${CLUSTER_NAME}-controlplane_values.yaml.tmp && mv ${CLUSTER_NAME}-controlplane_values.yaml.tmp ${CLUSTER_NAME}-controlplane_values.yaml
   rm controlplane-patch.yaml

2. Onboard the Cluster

   Onboard the cluster, using helm, as follows:

   helm repo add tetrate-tsb-helm 'https://charts.dl.tetrate.io/public/helm/charts/'
   helm repo update
   
   helm install cp tetrate-tsb-helm/controlplane \
       --namespace istio-system --create-namespace \
       --timeout 5m \
       --version ${TSB_VERSION} \
       --values ${CLUSTER_NAME}-controlplane_values.yaml \
       --set image.registry=${REGISTRY}

   Wait until the installation completes:

   kubectl logs -f -n istio-system -l name=tsb-operator

   Deploy the Gateway API CRDs:

   kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/experimental-install.yaml

3. Validate the installation

   You can observe the progress of the installation as follows:

   Watch the installation progress

   kubectl logs -f -n istio-system -l name=tsb-operator

   Check the proper deployment of components within the istio-system namespace and the correct functionality of the installed components as follows:

   Validate Helm deployment status

   helm ls -n istio-system
   # Expected output:
   # NAME        	NAMESPACE   	REVISION	UPDATED                                	STATUS  	CHART              	APP VERSION
   # controlplane	istio-system	1       	2025-04-09 12:24:23.689007205 +0000 UTC	deployed	controlplane-1.12.1	1.12.1
   Validate component deployment status

   kubectl get pod -n istio-system
   # Expected output:
   # NAME                                          READY   STATUS    RESTARTS   AGE
   # edge-66dddd7c9-8lwmm                          1/1     Running   0          2m12s
   # istio-operator-66c8b7d688-frh4b               1/1     Running   0          2m12s
   # istiod-7466bd7759-s85rz                       1/1     Running   0          114s
   # oap-deployment-f84d87cbc-8f6zm                3/3     Running   0          105s
   # onboarding-operator-74d58bfbd9-7znvs          1/1     Running   0          2m31s
   # otel-collector-5b5b6599dd-k8ncw               2/2     Running   0          2m31s
   # tsb-operator-control-plane-78bb64bcd4-k6zb8   1/1     Running   0          3m28s
   # wasmfetcher-58c667bc48-g6g7n                  1/1     Running   0          2m31s
   # xcp-operator-edge-7ccb7b86cf-hj7w5            1/1     Running   0          2m31s
   Check the status of the cluster using Tetrate Command Line Tool

   tctl status cluster ${CLUSTER_NAME} -o yaml
   # Expected output:
   # apiVersion: api.tsb.tetrate.io/v2
   # kind: ResourceStatus
   # metadata:
   #   name: eks-owen14-eu-west-2-0
   #   organization: tetrate
   # spec:
   #   configEvents:
   #     events:
   #     - etag: '"mHSnTvqnxBI="'
   #       timestamp: "2025-04-09T12:24:22.859199619Z"
   #       type: XCP_ACCEPTED
   #     - etag: '"mHSnTvqnxBI="'
   #       timestamp: "2025-04-09T12:24:22.832884695Z"
   #       type: MPC_ACCEPTED
   #     - etag: '"mHSnTvqnxBI="'
   #       timestamp: "2025-04-09T12:24:21.497871959Z"
   #       type: TSB_ACCEPTED
   #   message: Cluster onboarded
   #   status: READY

   Check that the Gateway API CRDs were installed:

   Check Gateway API CRDs

   kubectl get crd 
   # Expected output:
   # backendlbpolicies.gateway.networking.k8s.io               2025-04-09T12:25:31Z
   # backendtlspolicies.gateway.networking.k8s.io              2025-04-09T12:25:31Z
   # gatewayclasses.gateway.networking.k8s.io                  2025-04-09T12:25:31Z
   # gateways.gateway.networking.k8s.io                        2025-04-09T12:25:31Z
   # grpcroutes.gateway.networking.k8s.io                      2025-04-09T12:25:31Z
   # httproutes.gateway.networking.k8s.io                      2025-04-09T12:25:33Z
   # referencegrants.gateway.networking.k8s.io                 2025-04-09T12:25:32Z
   # tcproutes.gateway.networking.k8s.io                       2025-04-09T12:25:32Z
   # tlsroutes.gateway.networking.k8s.io                       2025-04-09T12:25:32Z
   # udproutes.gateway.networking.k8s.io                       2025-04-09T12:25:32Z

Deploy the Tetrate Configuration

Once the cluster is successfully onboarded, we will

• Create a bookinfo namespace, deploy the BookInfo application and a Tetrate gateway
• Create the basic Tetrate configuration - Workspace, Gateway Group

1. Configure the cluster

   Define the parameters before you proceed:

   export NS=bookinfo

   Configure the cluster with a namespace, and deploy Bookinfo:

   kubectl create namespace ${NS}
   kubectl label namespace ${NS} istio-injection=enabled
   
   kubectl apply -n ${NS} -f https://raw.githubusercontent.com/istio/istio/master/samples/bookinfo/platform/kube/bookinfo.yaml

   Deploy a Tetrate gateway. Note that depending on your cloud platform, you may need to add additional annotations to expose the gateway externally, or set other operating parameters:

   Deploy gateway

   cat <<EOF > ${NS}-gw.yaml
   apiVersion: install.tetrate.io/v1alpha1
   kind: Gateway
   metadata:
     name: ${NS}-gw
     namespace: ${NS}
   spec:
     kubeSpec:
       service:
         type: LoadBalancer
         annotations:
           service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
   EOF
   
   kubectl apply -f ${NS}-gw.yaml

2. Create the corresponding Tetrate Configuration

   Export the environment parameters before you proceed:

   export NS=bookinfo
   export ORG=myorg # Tetrate organization
   export TEN=myten # Tetrate tenant

   We will create a Workspace and Gateway Group to encapsulate and contain the Tetrate configuration and services:

   cat <<EOF > ${NS}-ws.yaml
   apiversion: api.tsb.tetrate.io/v2
   kind: Workspace
   metadata:
     organization: ${ORG}
     tenant: ${TEN}
     name: ${NS}-ws
   spec:
     displayName: ${NS}
     description: Test ${NS} application
     namespaceSelector:
       names:
         - "*/${NS}"
   EOF
   
   tctl apply -f ${NS}-ws.yaml
   
   cat <<EOF > ${NS}-gwgroup.yaml
   apiVersion: gateway.tsb.tetrate.io/v2
   kind: Group
   metadata:
     organization: ${ORG}
     tenant: ${TEN}
     workspace: ${NS}-ws
     name: ${NS}-gwgroup
   spec:
     configMode: BRIDGED
     namespaceSelector:
       names:
         - "*/${NS}"
   EOF
   
   tctl apply -f ${NS}-gwgroup.yaml

3. Validate the configuration

   Check the status of the Gateway pod

   kubectl get pod -n $NS -l app=${NS}-gw
   # NAME                           READY   STATUS    RESTARTS   AGE
   # bookinfo-gw-586786d67d-gpvcn   1/1     Running   0          5m10s
   Check that the Gateway pod is exposed

   kubectl get svc -n $NS -l app=${NS}-gw
   # NAME          TYPE           CLUSTER-IP       EXTERNAL-IP                                                                     PORT(S)                                      AGE
   # bookinfo-gw   LoadBalancer   10.100.224.221   k8s-bookinfo-bookinfo-09ff5fac83-3eb47cd15d1903d3.elb.eu-west-2.amazonaws.com   15443:31266/TCP,80:32028/TCP,443:32529/TCP   5m51s

Expose the Service using a Gateway API resource

1. Configure the Cluster

   Export the environment parameters before you proceed:

   export NS=bookinfo
   export ORG=myorg # Tetrate organization
   export TEN=myten # Tetrate tenant

   Deploy the required AuthorizationPolicy and generate a certificate:

   Additional configuration

   cat <<EOF > ${NS}-config.yaml
   ---
   apiVersion: security.istio.io/v1
   kind: AuthorizationPolicy
   metadata:
     name: ${NS}-allow-https-443
     namespace: ${NS}
   spec:
     rules:
       - to:
         - operation:
             ports:
             - "8443"
             - "15443"
     selector:
       matchLabels:
         istio: ingressgateway
   ---
   apiVersion: cert-manager.io/v1
   kind: Certificate
   metadata:
     name: ${NS}-certificate
     namespace: ${NS}
   spec:
     secretName: ${NS}-certificate
     duration: 21600h # 900d
     privateKey:
       algorithm: ECDSA
       size: 256
     issuerRef:
       name: selfsigned-ca
       kind: ClusterIssuer
       group: cert-manager.io
     dnsNames:
       - "${NS}.tetrate.io"
   EOF
   
   kubectl apply -f ${NS}-config.yaml

2. Expose the Service using Gateway API

   Now, you can expose the service using the Gateway API. Note the additional annotations needed to link this Gateway API resource to the Tetrate hierarchy.

   Gateway API configuration

   cat <<EOF > ${NS}-gw-api.yaml
   apiVersion: gateway.networking.k8s.io/v1
   kind: Gateway
   metadata:
     name: ${NS}-gateway
     namespace: ${NS}
     annotations:
       tsb.tetrate.io/organization: ${ORG}
       tsb.tetrate.io/tenant: ${TEN}
       tsb.tetrate.io/workspace: ${NS}-ws
   spec:
     gatewayClassName: istio
     addresses:
     - value: ${NS}-gateway.${NS}.svc.cluster.local
       type: Hostname
     listeners:
     - name: http
       hostname: "${NS}.tetrate.io"
       port: 80
       protocol: HTTP
       allowedRoutes:
         namespaces:
           from: All
     - name: https
       hostname: "${NS}.tetrate.io"
       port: 443
       protocol: HTTPS
       allowedRoutes:
         namespaces:
           from: All
       tls:
         mode: Terminate
         certificateRefs:
         - name: ${NS}-certificate
           namespace: ${NS}
   ---
   apiVersion: gateway.networking.k8s.io/v1
   kind: HTTPRoute
   metadata:
     name: ${NS}-http-svc-a
     namespace: ${NS}
     annotations:
       tsb.tetrate.io/organization: ${ORG}
       tsb.tetrate.io/tenant: ${TEN}
       tsb.tetrate.io/workspace: ${NS}-ws
   spec:
     parentRefs:
     - name: ${NS}-gateway
       namespace: ${NS}
     hostnames: ["${NS}.tetrate.io"]
     rules:
     - matches:
       - path:
           type: PathPrefix
           value: /
       backendRefs:
       - name: svc-a
         port: 8000
   EOF
   
   kubectl apply -f ${NS}-gw-api.yaml

3. Validate Gateway Configuration

   The Tetrate Management Plane significantly enhances this setup by providing centralized control and visibility over Gateway configurations. It ensures efficient propagation of changes made through TSB APIs across all Gateways, according to parameters defined in Gateway Groups and workload selectors. This central management capability ensures consistency, reduces complexity, and enhances security across the entire network infrastructure.

   Tip: may need to get the ip, not hostname, depending on how the service is exposed

   export NS=bookinfo
   
   export GATEWAY_IP=$(kubectl -n ${NS} get service ${NS}-gw -o jsonpath="{.status.loadBalancer.ingress[0].hostname}")
   echo ${GATEWAY_IP}

   Execute a test curl command with X-B3-sampled: 1 header to ensure request tracing. For more details about tracing headers, see: https://github.com/openzipkin/b3-propagation

   curl -k -H "Host: ${NS}.tetrate.io" http://${GATEWAY_IP}/ -I -H "x-b3-sampled: 1"
   # Expected Output
   # HTTP/1.1 200 OK
   # server: istio-envoy
   # date: Wed, 11 Dec 2024 02:26:12 GMT
   # content-type: text/html; charset=utf-8
   # content-length: 15072
   # vary: Cookie
   # x-envoy-upstream-service-time: 110

If you submit requests using a benchmarking tool (or simply repeat the curl command for a couple of minutes), you will be able to observe the traffic in the Tetrate Dashboard and the related Grafana dashboards.

Summary

What have we achieved?

• We have updated a cluster configuration to enable the use of Gateway APIs
• We have exposed a service through a Tetrate gateway using a Gateway API
• We have used the Tetrate observability stack to gather metrics and traces for requests to that service