Tetrate Service ExpressVersion: Latest

Tier1 Gateway

The differences between TSB and TSE API

Tetrate Service Express (TSE) utilizes many of the same components as the Tetrate Service Bridge(TSB) product but has the several distinctions. Go to Comparing TSE and TSB for more details.

DEPRECATION: The functionality provided by the Tier1Gateway is now provided in Gateway object, and using it is the recommended approach. The Tier1Gateway resource will be removed in future releases.

Tier1Gateway configures a workload to act as a gateway that distributes traffic across one or more ingress gateways in other clusters.

NOTE: Tier1 gateways cannot be used to route traffic to the same cluster. A cluster with tier1 gateway cannot have any other gateways or workloads.

The following example declares a tier1 gateway running on pods with app: gateway labels in the ns1 namespace. The gateway exposes host movieinfo.com on ports 8080, 8443 and kafka.internal on port 9000. Traffic for these hosts at the ports 8443 and 9000 are TLS terminated and forwarded over Istio mutual TLS to the ingress gateways hosting movieinfo.com host on clusters c3 and c4 and the internal kafka.internal service in cluster c3 respectively. The server at port 8080 is configured to receive plaintext HTTP traffic and redirect to port 8443 with "Permanently Moved" (HTTP 301) status code.

apiVersion: gateway.tsb.tetrate.io/v2
kind: Tier1Gateway
metadata:
  name: tier1
  group: g1
  workspace: w1
  tenant: tse
  organization: tse
spec:
  workloadSelector:
    namespace: ns1
    labels:
      app: gateway
  externalServers:
  - name: movieinfo-plain
    hostname: movieinfo.com # Plaintext and HTTPS redirect
    port: 8080
    redirect:
      authority: movieinfo.com
      uri: "/"
      redirectCode: 301
      port: 8443
      scheme: https
  - name: movieinfo
    hostname: movieinfo.com # TLS termination and Istio mTLS to upstream
    port: 8443
    tls:
      mode: SIMPLE
      secretName: movieinfo-secrets
    clusters:
    - name: c3 # the target gateway IPs will be automatically determined
      weight: 90
    - name: c4
      weight: 10
    authentication:
      rules:
        jwt:
        - issuer: "auth.mycompany.com"
          jwksUri: https://auth.mycompany.com/oauth2/jwks
        - issuer: "auth.othercompany.com"
          jwksUri: https://auth.othercompany.com/oauth2/jwks
    authorization:
      external:
        uri: "https://auth.company.com"
        includeRequestHeaders:
        - authorization
  tcpExternalServers:
  - name: kafka
    hostname: kafka.internal
    port: 9000
    tls:
      mode: SIMPLE
      secretName: kafka-cred
    clusters:
    - name: c3
      weight: 100

In the following example, the clients are authenticated using an external OIDC provider using AUTHORIZATION_CODE grant type. Once the client request is authenticated, it gets forwarded to the c3 or c4. The access_token generated after client authentication is set as Bearer in request headers. The state of authentication is stored in cookies.

apiVersion: gateway.tsb.tetrate.io/v2
kind: Tier1Gateway
metadata:
  name: tier1
  group: g1
  workspace: w1
  tenant: tse
  organization: tse
spec:
  workloadSelector:
    namespace: ns1
    labels:
      app: gateway
  externalServers:
  - name: movieinfo-plain
    hostname: movieinfo.com # Plaintext and HTTPS redirect
    port: 8080
    redirect:
      authority: movieinfo.com
      uri: "/"
      redirectCode: 301
      port: 8443
      scheme: https
  - name: movieinfo
    hostname: movieinfo.com # TLS termination and Istio mTLS to upstream
    port: 8443
    tls:
      mode: SIMPLE
      secretName: movieinfo-secrets
    clusters:
    - name: c3 # the target gateway IPs will be automatically determined
      weight: 90
    - name: c4
      weight: 10
    authentication:
      oidc:
        grantType: AUTHORIZATION_CODE
        clientId: "my-client"
        clientTokenSecret: "my-secret"
        redirectUri: https://httpbin.example.com/bearer
        provider:
          issuer: https://accounts.google.com
          authorizationEndpoint: https://accounts.google.com/v1/authorize
          tokenEndpoint: https://accounts.google.com/v1/token
          jwksUri: https://www.googleapis.com/oauth2/v3/certs
    authorization:
      external:
        uri: "https://auth.company.com"
        includeRequestHeaders:
        - authorization

Tier1 gateways can also be used to forward mesh internal traffic for Gateway hosts from one cluster to another. This form of forwarding will work only if the two clusters cannot reach each other directly (e.g., they are on different VPCs that are not peered). The following example declares a tier1 gateway running on pods with app: gateway labels in the ns1 namespace. The gateway exposes hosts movieinfo.com, bookinfo.com, and a non-HTTP server called kafka.org-internal within the mesh. Traffic to movieinfo.com is load balanced across all clusters on vpc-02, while traffic to bookinfo.com and kafka.org-internal is load balanced across ingress gateways exposing bookinfo.com on any cluster. Traffic from the source (sidecars) is expected to arrive on the tier1 gateway over Istio mTLS.

apiVersion: gateway.tsb.tetrate.io/v2
kind: Tier1Gateway
metadata:
  name: tier1
  group: g1
  workspace: w1
  tenant: tse
  organization: tse
spec:
  workloadSelector:
    namespace: ns1
    labels:
      app: gateway
  internalServers: # forwarding gateway (HTTP traffic only)
  - name: movieinfo
    hostname: movieinfo.com
    clusters:
    - labels:
        network: vpc-02 # the target gateway IPs will be automatically determined
    authentication:
      rules:
        jwt:
        - issuer: "auth.mycompany.com"
          jwksUri: https://auth.company.com/oauth2/jwks
        - issuer: "auth.othercompany.com"
          jwksUri: https://auth.othercompany.com/oauth2/jwks
    authorization:
      external:
        uri: "https://auth.company.com"
        includeRequestHeaders:
        - authorization
  - name: bookinfo
    hostname: bookinfo.com # route to any ingress gateway exposing bookinfo.com
  tcpInternalServers: # forwarding non-HTTP traffic within the mesh
  - name: kafka
    hostname: kafka.org-internal

** NOTE:** If two clusters have direct connectivity, declaring a tier1 internal server will have no effect.

Tier1 gateways can also be configured to expose hostnames in the TLS passthrough mode. Tier1 gateway will forward the pasthrough server traffic to any tier2 pass through servers exposing the same hostname. In other words, To be able to leverage passthrough at tier1, it is a MUST that passthrough is configured at t2 IngressGateway as well.

** NOTE:** A hostname like abc.com can only be exposed either in passthrough mode OR in terminating tls mode(External/Internal servers), not in both the modes.

apiVersion: gateway.tsb.tetrate.io/v2
kind: Tier1Gateway
metadata:
  name: tier1-tls-gw
  group: g1
  workspace: w1
  tenant: tse
  organization: tse
spec:
  workloadSelector:
    namespace: ns1
    labels:
      app: gateway
  passthroughServers:
  - name: nginx
    port: 8443
    hostname: nginx.example.com

The Tier1Gateway above will require the corresponding, at least one or more, IngressGateway(s), e.g.:

apiVersion: gateway.tsb.tetrate.io/v2
kind: IngressGateway
metadata:
  name: tls-gw
  group: g1
  workspace: w1
  tenant: tse
  organization: tse
spec:
  workloadSelector:
    namespace: ns1
    labels:
      app: gateway
  tlsPassthrough:
    - name: nginx
      port: 443
      hostname: nginx.example.com
      route:
        host: "ns1/my-nginx.default.svc.cluster.local"
        port: 443

The following example customizes the Extensions field to enable the execution of the specified WasmExtensions list and details custom properties for the execution of each extension.

apiVersion: gateway.tsb.tetrate.io/v2
kind: Tier1Gateway
metadata:
  name: tier1-tls-gw
  group: g1
  workspace: w1
  tenant: tse
  organization: tse
spec:
  workloadSelector:
    namespace: ns1
    labels:
      app: gateway
  externalServers:
  - name: movieinfo-plain
    hostname: movieinfo.com # Plaintext and HTTPS redirect
    port: 8080
    redirect:
      authority: movieinfo.com
      uri: "/"
      redirectCode: 301
      port: 8443
      scheme: https
  extension:
  - fqn: hello-world # fqn of imported extensions in TSE config:
      foo: bar

Whenever traffic is to be sent from one cluster to another, one or more of the following would have to be true for it to succeed:

Both clusters belong to the same network.
Destination cluster network is not named.
Organization Setting is set up to send traffic from source cluster to destination cluster.

Tier1Gateway also allows you to apply ModSecurity/Coraza compatible Web Application Firewall rules to traffic passing through the gateway.

apiVersion: gateway.xcp.tetrate.io/v2
kind: Tier1Gateway
metadata:
  name: tier1-waf-gw
  group: g1
  workspace: w1
  tenant: tse
  organization: tse
spec:
  workloadSelector:
    namespace: ns1
    labels:
      app: gateway
  passthroughServers:
  - name: nginx
    port: 8443
    hostname: nginx.example.com
  waf:
    rules:
      - Include @owasp_crs/*.conf

Tier1ExternalServer

Tier1ExternalServer describes the properties of a server exposed outside the mesh. Traffic arriving at a Tier1 external server is usually TLS terminated and then forwarded over Istio mTLS to all the lower tier2 clusters.

Field	Description	Validation Rule
name	string REQUIRED A name assigned to the server. The name will be visible in the generated metrics. The name must be unique across all external servers in the gateway.	string = { min_len: `1` }
port	uint32 REQUIRED The port where the server is exposed. Note that port 15443 is reserved.	uint32 = { not_in: `0,15443` }
hostname	string REQUIRED Hostname with which the service can be expected to be accessed by clients.	string = { min_len: `1` }
tls	tetrateio.api.tsb.gateway.v2.ServerTLSSettings TLS certificate info. The gateway will terminate the TLS connection and forward it to the upstream ingress gateway using Istio mutual TLS on port 15443.	–
clusters	List of tetrateio.api.tsb.gateway.v2.ClusterDestination The destination clusters that contain ingress gateways exposing the hostname. If omitted, traffic will be automatically load balanced across all tier2 clusters whose ingress gateways expose the above hostname. If `redirect` is configured then this field cannot be configured. To do failover and locality based routing among clusters, either omit the clusters field or omit the weights from all the cluster destinations.	–
redirect	tetrateio.api.tsb.gateway.v2.Redirect Redirect allows configuring HTTP redirect. When this is configured, the `clusters` field cannot be configured.	–
authentication	tetrateio.api.tsb.auth.v2.Authentication Authentication is used to configure the authentication of end-user credentials like JWT. It is highly recommended to configure this with TLS	–
authorization	tetrateio.api.tsb.auth.v2.Authorization Authorization is used to configure authorization of end users. It is highly recommended to configure this with TLS	–
rateLimiting	tetrateio.api.tsb.gateway.v2.RateLimiting Configuration for rate limiting requests. This configuration is namespaced to a Tier1ExternalServer	–

Tier1Gateway

Tier1Gateway configures a workload to act as a tier1 gateway into the mesh.

Field	Description	Validation Rule
workloadSelector	tetrateio.api.tsb.types.v2.WorkloadSelector REQUIRED Specify the gateway workloads (pod labels and Kubernetes namespace) under the gateway group that should be configured with this gateway. There can be only one gateway for a workload selector in a namespace.	message = { required: `true` }
externalServers	List of tetrateio.api.tsb.gateway.v2.Tier1ExternalServer One or more servers exposed by the gateway externally.	–
internalServers	List of tetrateio.api.tsb.gateway.v2.Tier1InternalServer One or more servers exposed by the gateway internally for cross cluster forwarding.	–
passthroughServers	List of tetrateio.api.tsb.gateway.v2.Tier1PassthroughServer One or more tls passthrough servers exposed by the gateway externally.	–
tcpExternalServers	List of tetrateio.api.tsb.gateway.v2.Tier1TCPExternalServer One or more tcp servers exposed by the gateway externally.	–
tcpInternalServers	List of tetrateio.api.tsb.gateway.v2.Tier1TCPInternalServer One or more tcp servers exposed by the gateway for mesh internal traffic.	–
extension	List of tetrateio.api.tsb.types.v2.WasmExtensionAttachment Extensions specifies all the WasmExtensions assigned to this Tier1Gateway with the specific configuration for each extension. This custom configuration will override the one configured globally to the extension. Each extension has a global configuration including enablement and priority that will condition the execution of the assigned extensions.	–
waf	tetrateio.api.tsb.security.v2.WAFSettings WAF settings to be enabled for traffic passing through this Tier1 gateway.	–
configGenerationMetadata	tetrateio.api.tsb.types.v2.ConfigGenerationMetadata Metadata values that will be add into the Istio generated configurations. When using YAML APIs like`tctl` or `gitops`, put them into the `metadata.labels` or `metadata.annotations` instead. This field is only necessary when using gRPC APIs directly.	–

Tier1InternalServer

Tier1InternalServer describes the properties of a server exposed within the mesh, for the purposes of forwarding traffic between two clusters that cannot otherwise directly reach each other. Traffic arriving at a Tier1 internal server should be over Istio mTLS. After TLS termination and metrics extraction, it is forwarded to tier2 clusters based on the selection criteria.

Field	Description	Validation Rule
name	string REQUIRED A name assigned to the server. The name will be visible in the generated metrics. The name must be unique across all internal servers in the gateway.	string = { min_len: `1` }
hostname	string REQUIRED Hostname with which the service can be expected to be accessed by sidecars in the mesh.	string = { min_len: `1` }
clusters	List of tetrateio.api.tsb.gateway.v2.ClusterDestination The destination clusters that contain ingress gateways exposing the hostname. If omitted, traffic will be automatically load balanced across all tier2 clusters whose ingress gateways expose the above hostname.	–
authentication	tetrateio.api.tsb.auth.v2.Authentication Authentication is used to configure the authentication of end-user credentials like JWT.	–
authorization	tetrateio.api.tsb.auth.v2.Authorization Authorization is used to configure authorization of end user and traffic.	–

Tier1PassthroughServer

Tier1PassthroughServer describes the properties of a server exposed to the external world. Traffic arriving at a Tier1 passthrough server is not TLS terminated and rather forwarded over to all the lower tier2 clusters.

Field	Description	Validation Rule
name	string REQUIRED A name assigned to the server. The name will be visible in the generated metrics. The name must be unique across all external servers in the gateway.	string = { min_len: `1` }
port	uint32 REQUIRED The port where the server is exposed. Note that port 15443 is reserved.	uint32 = { not_in: `0,15443` }
hostname	string REQUIRED Hostname with which the service can be expected to be accessed by clients.	string = { min_len: `1` }
clusters	List of tetrateio.api.tsb.gateway.v2.ClusterDestination The destination clusters that contain ingress gateways exposing the hostname on passthrough servers. If omitted, traffic will be automatically load balanced across all tier2 clusters whose ingress gateways expose the above hostname.	–

Tier1TCPExternalServer

Tier1TCPExternalServer is used to describe the properties of a TCP server (used for opaque TCP or non-HTTP protocols) exposed to the external world. If the protocol is known to be HTTP, then please use externalServers as it allows using HTTP-specific features.

Caveat - Currently, we don't support multicluster routing when Tier2 gateway settings are specified in the direct mode for TCP services. So please use the bridged mode.

Field	Description	Validation Rule
name	string REQUIRED A name assigned to the server. This name is used in the generated metrics. The name must be unique across all TCP servers in the gateway	string = { min_len: `1` }
port	uint32 REQUIRED The port where the server is exposed. Note that the port 15443 is reserved. Also beware of the conflict among the services using different protocols on the same port. The conflict occurs in the following scenarios Using plaintext and TLS (passthrough/termination) Mixing multiple protocols without TLS (HTTP and non-HTTP protocols like Kafka, Zookeeper etc) Multiple non-HTTP protocols without TLS Valid scenarios (for same port, multiple services) Multiple protocols (HTTP, non-HTTP) with TLS passthrough/termination Multiple HTTP services Single non-HTTP service without TLS Note on service port - If a service is exposed on port 6789 in the tier1 gateway, then it must be exposed on the same port with the same hostname (without wildcard) in the tier2 gateway as well.	uint32 = { not_in: `0,15443` }
hostname	string REQUIRED Although hostname or authority does not make sense in the non-HTTP context, this is used to define the routing rules. Wildcard hostnames are not yet supported.	string = { min_len: `1` }
clusters	List of tetrateio.api.tsb.gateway.v2.ClusterDestination The destination clusters contain ingress gateways exposing the service.	–
tls	tetrateio.api.tsb.gateway.v2.ServerTLSSettings TLS certificate information to terminate TLS. If passthrough is required, then please use `passthroughServers` to specify them.	–

Tier1TCPInternalServer

Tier1TCPInternalServer is used to describe the properties of a TCP server which is used exclusively within the mesh.

Field	Description	Validation Rule
name	string REQUIRED A name assigned to the server. This name is used in the generated metrics. The name must be unique across all TCP servers in the gateway.	–
hostname	string REQUIRED The name of the service used. Although hostname or authority does not make sense in the non-HTTP context, this is used for the multicluster routing purposes. Consider the case where there are two non-HTTP services listening on the same port 6000, but are hosted on different workloads. Here, the service name is used to distinguish between the two for routing to the correct workload. We do not support wildcard hostnames yet. The ports are determined automatically by the cluster updates of the remote edge clusters. Suppose there is a service called `foo.com` and the remote cluster says that it exposes ports 8080 and 8443, then we can route east-west traffic for both the ports through this server. The changes to the port or protocol settings are picked up automatically.	string = { min_len: `1` }
clusters	List of tetrateio.api.tsb.gateway.v2.ClusterDestination The destination clusters contain ingress gateways exposing the service.	–

Tier1ExternalServer​

Tier1Gateway​

Tier1InternalServer​

Tier1PassthroughServer​

Tier1TCPExternalServer​

Tier1TCPInternalServer​