Setting Up an External Rate Limiting Server

TSB supports using external rate limiting servers. This document will describe how to configure Envoy rate limit service and use it as external rate limiting server in TSB's Ingress Gateway through an example.

Before you get started, make sure you:
✓ Familiarize yourself with TSB concepts
✓ Install the TSB environment. You can use TSB demo for quick install
✓ Completed TSB usage quickstart. This document assumes you already created Tenant and are familiar with Workspace and Config Groups. Also you need to configure tctl to your TSB environment.

note

While this document will only describe how to apply rate limiting using an external server for Ingress Gateway, you can do the same for Tier-1 Gateways and service-to-service (through TSB Traffic Settings) using a similar configuration.

Create the Namespace

In this example we will install the external rate limit service in ext-ratelimit namespace. Create the namespace if not already present in the target cluster by running the following command:

kubectl create namespace ext-ratelimit

Configure Rate Limit Service

note

Please read the Envoy rate limit documentation to learn the details about the concept of domains and descriptors.

Create a file name ext-ratelimit-config.yaml with the following content. This configuration specifies that requests to every unique request path should be limited to 4 requests/minute.

apiVersion: v1
kind: ConfigMap
metadata:
  name: ratelimit-config
  namespace: ext-ratelimit
data:
  config.yaml: |
    domain: httpbin-ratelimit
    descriptors:
      - key: "request-path"
        rate_limit:
          unit: minute
          requests_per_unit: 4

Then create a ConfigMap using the file you created:

kubectl -n ext-ratelimit apply -f ext-ratelimit-config.yaml

Deploy Rate Limit Server and Redis

Deploy Redis and envoyproxy/ratelimit. Create a file called redis-ratelimit.yaml with the following contents:

# Copyright Istio Authors
#
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.

####################################################################################
# Redis service and deployment
# Ratelimit service and deployment
####################################################################################
apiVersion: v1
kind: Service
metadata:
  name: redis
  namespace: ext-ratelimit
  labels:
    app: redis
spec:
  ports:
    - name: redis
      port: 6379
  selector:
    app: redis
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: ext-ratelimit
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
        - image: redis:alpine
          imagePullPolicy: Always
          name: redis
          ports:
            - name: redis
              containerPort: 6379
      restartPolicy: Always
      serviceAccountName: ''
---
apiVersion: v1
kind: Service
metadata:
  name: ratelimit
  namespace: ext-ratelimit
  labels:
    app: ratelimit
spec:
  ports:
    - name: http-port
      port: 8080
      targetPort: 8080
      protocol: TCP
    - name: grpc-port
      port: 8081
      targetPort: 8081
      protocol: TCP
    - name: http-debug
      port: 6070
      targetPort: 6070
      protocol: TCP
  selector:
    app: ratelimit
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ratelimit
  namespace: ext-ratelimit
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ratelimit
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: ratelimit
    spec:
      containers:
        - image: envoyproxy/ratelimit:6f5de117 # 2021/01/08
          imagePullPolicy: Always
          name: ratelimit
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthcheck
              port: 8080
              scheme: HTTP
            periodSeconds: 3
            successThreshold: 1
            timeoutSeconds: 1
          command: ['/bin/ratelimit']
          env:
            - name: REDIS_HEALTH_CHECK_ACTIVE_CONNECTION
              value: 'true'
            - name: LOG_LEVEL
              value: debug
            - name: REDIS_SOCKET_TYPE
              value: tcp
            - name: REDIS_URL
              value: redis:6379
            - name: USE_STATSD
              value: 'false'
            - name: RUNTIME_ROOT
              value: /data
            - name: RUNTIME_SUBDIRECTORY
              value: ratelimit
          ports:
            - containerPort: 8080
            - containerPort: 8081
            - containerPort: 6070
          volumeMounts:
            - name: config-volume
              mountPath: /data/ratelimit/config/config.yaml
              subPath: config.yaml
      volumes:
        - name: config-volume
          configMap:
            name: ratelimit-config

kubectl -f redis-ratelimit.yaml

If everything is successful, you should have a working rate limit server. Make sure that Redis and the rate limit server are running by executing the following command:

kubectl get pods -n ext-ratelimit

You should see an output resembling the following:

NAME                        READY   STATUS    RESTARTS   AGE
ratelimit-d5c5b64ff-m87dt   1/1     Running   0          14s
redis-7d757c948f-42sxg      1/1     Running   0          14s

Configure Ingress Gateway

This example assumes that you are applying rate limit to the httpbin workload. If you have not already done so, deploy the httpbin service, create httpbin Workspace and Config Groups and expose the service through an Ingress Gateway.

The following sample sets rate limiting on requests in the httpbin-ratelimit domain. The request path is stored in descriptorKey named request-path, which is then used by the rate limit server.

apiVersion: gateway.tsb.tetrate.io/v2
kind: IngressGateway
metadata:
  name: httpbin-gateway # Need not be the same as spec.labels.app
  organization: tetrate
  tenant: tetrate
  group: httpbin-gateway
  workspace: httpbin
spec:
  workloadSelector:
    namespace: httpbin
    labels:
      app: httpbin-ingress-gateway # name of Ingress Gateway created for httpbin
  http:
    - name: httpbin
      hostname: 'httpbin.tetrate.com'
      port: 80
      routing:
        rules:
          - route:
              host: 'httpbin/httpbin.httpbin.svc.cluster.local'
              port: 8000
      rateLimiting:
        externalService:
          domain: 'httpbin-ratelimit'
          rateLimitServerUri: 'grpc://ratelimit.ext-ratelimit.svc.cluster.local:8081'
          rules:
            - dimensions:
                - requestHeaders:
                    headerName: ':path'
                    descriptorKey: request-path

Save this to a file named ext-ratelimit-ingress-gateway.yaml, and apply it using tctl:

tctl apply -f ext-ratelimit-ingress-gateway.yaml

Testing

You can test the rate limiting by sending HTTP requests from an external machine or your local environment to the httpbin Ingress Gateway, and observe the rate limiting take effect after a certain number of requests.

In the following example, since you do not control httpbin.tetrate.com, you will have to trick curl into thinking that httpbin.tetrate.com resolves to the IP address of the Ingress Gateway.

Obtain the IP address of the Ingress Gateway that you previously created using the following command.

kubectl -n httpbin get service httpbin-ingress-gateway \
  -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

Then execute the following command to send HTTP requests to the httpbin service through the Ingress Gateway. Replace the gateway-ip with the value you obtained in the previous step.

curl -k "http://httpbin.tetrate.com/get" \
    --resolve "httpbin.tetrate.com:80:<gateway-ip>" \
    -s \
    -o /dev/null \
    -w "%{http_code}\n" \
    -H "X-B3-Sampled: 1"

For the first 4 requests you should see "200" on your screen. After that, you should start seeing "429" instead.

You can change the request path to another unique value to get a successful response.

curl -k "http://httpbin.tetrate.com/headers" \
    --resolve "httpbin.tetrate.com:80:<gateway-ip>" \
    -s \
    -o /dev/null \
    -w "%{http_code}\n" \
    -H "X-B3-Sampled: 1"

After 4 requests, you should start seeing "429" again, until you change the request path.

Considerations for Using Rate Limiting Server over Multiple Clusters

In case you would like to share the same rate limiting rules against multiple cluster, there are two possible choices:

• Deploy a single rate limit service in one cluster, and make it reachable from all other clusters that share the rules, or
• Deploy rate limit services in each cluster, but make them all use the same Redis backend.

In the second scenario, you will have to make Redis accessible from all clusters. Each rate limit server should also use the same domain value.