Configure and route HTTP, non-HTTP (multi-protocol) and multi-port service traffic in TSB
This how-to document will show you how to configure non-HTTP servers with TSB. After
reading this document you should be familiar with the usage of the TCP section in
IngressGateway
and Tier1Gateway
API.
Summary
Workflow is exactly the same as configuring HTTP servers in IngressGateway
and
Tier1Gateway
. However, multi-port services are supported for non-HTTP.
Before you get started, make sure you:
✓ Familiarize yourself with TSB concepts
✓ Familiarize yourself with onboarding clusters
✓ Create a Tenant
Setup
- Four clusters with TSB installed - Management plane, Tier-1 and Tier-2 edge clusters.
- In Tier-2 clusters, deploy Tier-2 gateways in
echo
namespace - In Tier-1 cluster, deploy Tier-1 gateway in
tier1
namespace - In both the gateways, the ports
8080
and9999
should be available (for simplicity, we consider the service and the target ports to be the same). For installing gateways, see here - Deploy apps in Tier-2 cluster which work with HTTP and non-HTTP traffic. In this demo, you will deploy
applications from the Istio's samples directory into the echo namespace.
helloworld
is used for HTTP traffic. Manifests heretcp-echo
is used for non-HTTP traffic. Manifests here
- Make sure you have the required privileges to deploy the gateway configurations.
TSB configuration
Configuring workspace and groups
First create the workspace. Here, we assume that the clusters were onboarded as cluster-0
, cluster-1
,
cluster-2
and cluster-3
. It is also assumed that cluster-3
is Tier-1, cluster-0
has TSB management
plane installed.
apiVersion: api.tsb.tetrate.io/v2
kind: Workspace
metadata:
name: tcp-http-demo
organization: tetrateio
tenant: tetrate
spec:
namespaceSelector:
names:
- "cluster-1/echo"
- "cluster-2/echo"
- "cluster-3/tier1"
Separate groups are configured for Tier-1 and Tier-2 gateways
apiVersion: gateway.tsb.tetrate.io/v2
kind: Group
metadata:
name: tcp-http-test-t2-group
organization: tetrateio
tenant: tetrate
workspace: tcp-http-demo
spec:
namespaceSelector:
names:
- "cluster-1/echo"
- "cluster-2/echo"
---
apiVersion: gateway.tsb.tetrate.io/v2
kind: Group
metadata:
name: tcp-http-test-t1-group
organization: tetrateio
tenant: tetrate
workspace: tcp-http-demo
spec:
configMode: BRIDGED
namespaceSelector:
names:
- "cluster-3/tier1"
Provisioning certificates and keys for the gateways
You have to create two secrets in the namespaces where gateway workloads are deployed
hello-tlscred
for the Helloworld applicationecho-tlscred
for the TCP-echo application
You can provision certificates with tools like openssl
and create secrets as follows.
# Create secrets for the helloworld application. Here the certificate and
# the keys are provisioned in helloworld.crt and helloworld.key
kubectl --context=<kube-cluster-context> -n <gateway-ns> create secret tls hello-tlscred \
--cert=helloworld.crt --key=helloworld.key
# Create secrets for the tcp-echo application
kubectl --context=<kube-cluster-context> -n <gateway-ns> create secret tls echo-tlscred \
--cert=echo.crt --key=echo.key
Configuring Ingress Gateway at Tier-2 clusters
A few notes
- For the configuration of port 8080, TLS is required and the hostnames must be different. Otherwise, it is an error.
- The credentials are stored as secret in the namespace where gateway workloads are running. In Tier-2, it is the
echo
namespace and in Tier-1 it is thetier1
namespace.
apiVersion: gateway.tsb.tetrate.io/v2
kind: IngressGateway
metadata:
name: tcp-http-t2-gateway
organization: tetrateio
tenant: tetrate
workspace: tcp-http-demo
group: tcp-http-test-t2-group
spec:
workloadSelector:
namespace: echo
labels:
app: tsb-gateway-echo
http:
- name: http-hello
port: 8080
hostname: hello.tetrate.io
tls:
mode: SIMPLE
secretName: hello-tlscred
routing:
rules:
- route:
host: echo/helloworld.echo.svc.cluster.local
port: 5000
tcp:
# echo.tetrate.io:8080 receives non-HTTP traffic. There
# is also hello.tetrate.io:8080 receiving HTTP traffic on
# this port. To distinguish between the two services, you
# need to have different hostnames with TLS so that the
# clients can use different SNI to distinguish between them.
# This is the "multi-protocol"/"multiple traffic type" part.
- name: tcp-echo
port: 8080 # Same port as the previous HTTP server,
# but different hostname.
hostname: echo.tetrate.io
tls:
mode: SIMPLE
secretName: echo-tlscred
route:
host: echo/tcp-echo.echo.svc.cluster.local
port: 9000
# There is already a service called echo.tetrate.io defined
# port 8080. There can be another TCP service with the same
# hostname on a different port. This is the "multi-port" part.
- name: tcp-echo-2
port: 9999
hostname: echo.tetrate.io
route:
host: echo/tcp-echo.echo.svc.cluster.local
port: 9001
Configuring Tier-1 gateway
The host:port defined here should match exactly with the host:port and the traffic type should
match exactly with the ones defined in IngressGateway
before
apiVersion: gateway.tsb.tetrate.io/v2
kind: Tier1Gateway
metadata:
name: tcp-http-t1-gateway
organization: tetrateio
tenant: tetrate
workspace: tcp-http-demo
group: tcp-http-test-t1-group
spec:
workloadSelector:
namespace: tier1
labels:
app: tsb-gateway-tier1
externalServers:
# This matches with the HTTP server hello.tetrate.io:8080
# defined in the Tier-2 gateway configuration
- name: http-hello
hostname: hello.tetrate.io
port: 8080
tls:
mode: SIMPLE
secretName: hello-tlscred
tcpExternalServers:
# This matches with echo.tetrate.io:8080. The names need not
# be the same between the configs, but hostname must match.
- name: tcp-echo
hostname: echo.tetrate.io
port: 8080
tls:
mode: SIMPLE
secretName: echo-tlscred
# This matches with echo.tetrate.io:9999 in Tier-2 configs.
- name: tcp-echo-2
hostname: echo.tetrate.io
port: 9999
Routing traffic between clusters
North-south (From Tier-1 to Tier-2 clusters)
First, find out the external IP address of the Tier-1 gateway and save it in TIER1_IP
variable
export TIER1_IP=<tier1-gateway-ip>
Routing HTTPS traffic
curl -H "X-B3-Sampled: 1" -svk --resolve hello.tetrate.io:8080:$TIER1_IP https://hello.tetrate.io:8080/hello
NOTE: Don't use -k
flag unless it is for testing purposes. It skips server certificate verification and is insecure
Routing non-HTTP traffic
- TLS traffic - There could be some warnings related to the server certificates. As this is a demo, they can be ignored.
openssl s_client -connect $TIER1_IP:8080 -servername echo.tetrate.io
- Plain TCP traffic
echo hello | nc -v $TIER1_IP 9999
East-west (Between Tier-2 clusters)
DNS names defined in the hostname
field is used for routing east-west traffic. Exec into an Istio sidecar-injected pod from where you would like to send the traffic and run nc
for non-HTTP (but TCP) traffic. No need to initiate TLS here as TLS origination is done by the sidecar.
kubectl -n echo exec -it <pod-name> -c app -- sh -c "echo hello | nc -v echo.tetrate.io 8080"
kubectl -n echo exec -it <pod-name> -c app -- sh -c "echo hello | nc -v echo.tetrate.io 9999"
kubectl -n echo exec -it <pod-name> -c app -- curl -sv hello.tetrate.io:8080 -H "X-B3-Sampled: 1"