Tetrate Service BridgeVersion: 1.9.x

Traffic Setting

Traffic Settings allow configuring the behavior of the proxy workloads in a set of namespaces owned by a traffic group. Specifically, it allows configuring the dependencies of proxy workloads on namespaces outside the traffic group as well as reliability settings for outbound calls made by the proxy workloads to other services.

The following example creates a traffic group for the proxy workloads in ns1, ns2 and ns3 namespaces owned by its parent workspace w1 under tenant mycompany. It then defines a traffic setting for the all workloads in these namespaces, adding a dependency on all the services in the shared db namespace, and forwarding all unknown traffic via the egress gateway in the istio-system namespace.

apiVersion: traffic.tsb.tetrate.io/v2
kind: Group
metadata:
  name: t1
  workspace: w1
  tenant: mycompany
  organization: myorg
spec:
  namespaceSelector:
    names:
    - "*/ns1"
    - "*/ns2"
    - "*/ns3"
  configMode: BRIDGED

And the associated traffic settings for the proxy workloads:

apiVersion: traffic.tsb.tetrate.io/v2
kind: TrafficSetting
metadata:
  name: defaults
  group: t1
  workspace: w1
  tenant: mycompany
  organization: myorg
spec:
  reachability:
    mode: CUSTOM
    hosts:
    - "ns1/*"
    - "ns2/*"
    - "ns3/*"
    - "db/*"
  resilience:
    circuitBreakerSensitivity: MEDIUM
  egress:
    host: istio-system/istio-egressgateway

Part of resilience setting as set in above example is deprecated in favor of upstreamTrafficSettings. To achieve the same behavior using the new upstreamTrafficSettings:

apiVersion: traffic.tsb.tetrate.io/v2
kind: TrafficSetting
metadata:
  name: defaults
  group: t1
  workspace: w1
  tenant: mycompany
  organization: myorg
spec:
  upstreamTrafficSettings:
  - hosts:
    - '*'
    settings:
      resilience:
        circuitBreakerSensitivity: MEDIUM

To setup load balancing algorithm as ROUND_ROBIN for all outbound requests to service foo.bar.svc.cluster.local from clients in t1 traffic group:

apiVersion: traffic.tsb.tetrate.io/v2
kind: TrafficSetting
metadata:
  name: defaults
  group: t1
  workspace: w1
  tenant: mycompany
  organization: myorg
spec:
  upstreamTrafficSettings:
  - hosts:
    - 'foo.bar.svc.cluster.local'
    settings:
      loadBalancer:
        simple: ROUND_ROBIN

upstreamTrafficSettings can be used to configure the outbound traffic with grouping a particular group of upstream hosts to have a certain setting. In the below example all outbound requests to hosts matching wildcard *.ns1.svc.cluster.local will use request timeout of 10s while hosts matching *.ns2.svc.cluster.local and *.ns3.svc.cluster.local will use request timeout of 5s.

apiVersion: traffic.tsb.tetrate.io/v2
kind: TrafficSetting
metadata:
  name: defaults
  group: t1
  workspace: w1
  tenant: mycompany
  organization: myorg
spec:
  upstreamTrafficSettings:
  - hosts:
    - '*.ns1.svc.cluster.local'
    settings:
      resilience:
        connectionPool:
          http:
            requestTimeout: 10s
  - hosts:
    - '*.ns2.svc.cluster.local'
    - '*.ns3.svc.cluster.local'
    settings:
      resilience:
        connectionPool:
          http:
            requestTimeout: 5s

The following traffic setting confines the reachability of proxy workloads in the traffic group t1 to other namespaces inside the group. The resilience and egress gateway settings will be inherited from the workspace wide traffic setting.

apiVersion: traffic.tsb.tetrate.io/v2
kind: TrafficSetting
metadata:
  name: defaults
  group: t1
  workspace: w1
  tenant: mycompany
  organization: myorg
spec:
  reachability:
    mode: GROUP

HTTPRetry

HTTPRetry defines the parameters for retrying API calls to a service.

Field Description Validation Rule

Field	Description	Validation Rule
attempts	int32 REQUIRED Number of retries for a given request. The interval between retries will be determined automatically (25ms+). Actual number of retries attempted depends on the httpReqTimeout.	int32 = { gte: `0` }
perTryTimeout	google.protobuf.Duration Timeout per retry attempt for a given request. format: 1h/1m/1s/1ms. MUST BE >=1ms.	–
retryOn	string Specifies the conditions under which retry takes place. One or more policies can be specified using a ‘,’ delimited list. See the retry policies and gRPC retry policies for more details.	string = { pattern: `^$\|^(5xx\|gateway-error\|reset\|connect-failure\|envoy-ratelimited\|retriable-4xx\|refused-stream\|retriable-status-codes\|retriable-headers\|cancelled\|deadline-exceeded\|internal\|resource-exhausted\|unavailable)(,(5xx\|gateway-error\|reset\|connect-failure\|envoy-ratelimited\|retriable-4xx\|refused-stream\|retriable-status-codes\|retriable-headers\|cancelled\|deadline-exceeded\|internal\|resource-exhausted\|unavailable))*$` }

attempts

int32
REQUIRED
Number of retries for a given request. The interval between retries will be determined automatically (25ms+).

Actual number of retries attempted depends on the httpReqTimeout.

int32 = {
gte: 0
}

perTryTimeout

google.protobuf.Duration
Timeout per retry attempt for a given request. format: 1h/1m/1s/1ms. MUST BE >=1ms.

–

retryOn

string
Specifies the conditions under which retry takes place. One or more policies can be specified using a ‘,’ delimited list. See the retry policies and gRPC retry policies for more details.

string = {
pattern: ^$|^(5xx|gateway-error|reset|connect-failure|envoy-ratelimited|retriable-4xx|refused-stream|retriable-status-codes|retriable-headers|cancelled|deadline-exceeded|internal|resource-exhausted|unavailable)(,(5xx|gateway-error|reset|connect-failure|envoy-ratelimited|retriable-4xx|refused-stream|retriable-status-codes|retriable-headers|cancelled|deadline-exceeded|internal|resource-exhausted|unavailable))*$
}

KeepAliveSettings

Keep Alive Settings.

Field	Description	Validation Rule
tcp	tetrateio.api.tsb.traffic.v2.TcpKeepAliveSettings TCP Keep Alive settings associated with the upstream and downstream TCP connections.	–

LoadBalancerSettings

Defines Load Balancing policies to be applied on the client requests.

Field	Description	Validation Rule
simple	tetrateio.api.tsb.traffic.v2.LoadBalancerSettings.SimpleLB ^{oneof _lb_policy} Use standard load balancing algorithms that require no tuning.	enum = { defined_only: `true` }
consistentHash	tetrateio.api.tsb.traffic.v2.LoadBalancerSettings.ConsistentHashLB ^{oneof _lb_policy} Use consistent hash load balancing which can provide soft session affinity.	–

ConsistentHashLB

Consistent Hash-based load balancing can be used to provide soft session affinity based on HTTP headers, cookies or other properties. The affinity to a particular destination host may be lost when one or more hosts are added/removed from the destination service.

Note: consistent hashing is less reliable at maintaining affinity than common "sticky sessions" implementations, which often encode a specific destination in a cookie, ensuring affinity is maintained as long as the backend remains. With consistent hash, the guarantees are weaker; any host addition or removal can break affinity for 1/backends requests.

Warning: consistent hashing depends on each proxy having a consistent view of endpoints. This is not the case when locality load balancing is enabled. Locality load balancing and consistent hash will only work together when all proxies are in the same locality, or a high level load balancer handles locality affinity.

Field	Description	Validation Rule
httpHeaderName	string ^{oneof _hash_key} Hash based on a specific HTTP header.	–
httpCookie	tetrateio.api.tsb.traffic.v2.LoadBalancerSettings.ConsistentHashLB.HTTPCookie ^{oneof _hash_key} Hash based on HTTP cookie.	–
useSourceIp	bool ^{oneof _hash_key} Hash based on the source IP address. This is applicable for both TCP and HTTP connections.	–
httpQueryParameterName	string ^{oneof _hash_key} Hash based on a specific HTTP query parameter.	–
ringHash	tetrateio.api.tsb.traffic.v2.LoadBalancerSettings.ConsistentHashLB.RingHash ^{oneof _hash_algorithm} The ring/modulo hash load balancer implements consistent hashing to backend hosts.	–
maglev	tetrateio.api.tsb.traffic.v2.LoadBalancerSettings.ConsistentHashLB.MagLev ^{oneof _hash_algorithm} The Maglev load balancer implements consistent hashing to backend hosts.	–

HTTPCookie

Describes a HTTP cookie that will be used as the hash key for the Consistent Hash load balancer. If the cookie is not present, it will be generated.

Field	Description	Validation Rule
name	string REQUIRED Name of the cookie.	string = { min_len: `1` }
path	string Path to set for the cookie.	–
ttl	google.protobuf.Duration REQUIRED Lifetime of the cookie.	duration = { required: `true` }

MagLev

Implements consistent hashing to upstream hosts. It can be used as a drop in replacement for RingHash. It has higher speed than RingHash with faster hash table lookups. Please refer https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/load_balancing/load_balancers#maglev

Field	Description	Validation Rule
tableSize	uint32 REQUIRED The table size for Maglev hashing. This helps in controlling the disruption when the backend hosts change. Increasing the table size reduces the amount of disruption.	uint32 = { gte: `1` }

RingHash

Implements consistent hashing to upstream hosts. Each upstream host is mapped onto a circle (ring) by hashing its address, each request is then routed using some hash property of the request. Please refer https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/load_balancing/load_balancers#ring-hash

Field	Description	Validation Rule
minimumRingSize	uint32 The minimum number of virtual nodes to use for the hash ring. Defaults to 1024. Larger ring sizes result in more granular load distributions. If the number of hosts in the load balancing pool is larger than the ring size, each host will be assigned a single virtual node.	–

ReachabilitySettings

ReachabilitySettings define the set of services and hosts accessed by a workload (and hence its sidecar) in the mesh. Defining the set of services accessed by a workload (i.e. its dependencies) in advance reduces the memory and CPU consumption both the Istio control plane and the individual Envoy proxy workloads in the data plane.

Field Description Validation Rule

Field	Description	Validation Rule
mode	tetrateio.api.tsb.traffic.v2.ReachabilitySettings.Mode A short cut for specifying the set of services accessed by the workload.	–
hosts	List of string When the mode is `CUSTOM`, `hosts` specify the set of services that the sidecar should be able to reach. Must be in the `\<namespace\>/\<fqdn\>` format. `./` indicates all services in the namespace where the sidecar resides. `ns1/` indicates all services in the `ns1` namespace. `ns1/svc1.com` indicates `svc1.com` service in `ns1` namespace. `*/svc1.com` indicates `svc1.com` service in any namespace.	–

mode

tetrateio.api.tsb.traffic.v2.ReachabilitySettings.Mode
A short cut for specifying the set of services accessed by the workload.

–

hosts

List of string
When the mode is CUSTOM, hosts specify the set of services that the sidecar should be able to reach. Must be in the \<namespace\>/\<fqdn\> format.

./* indicates all services in the namespace where the sidecar resides.
ns1/* indicates all services in the ns1 namespace.
ns1/svc1.com indicates svc1.com service in ns1 namespace.
*/svc1.com indicates svc1.com service in any namespace.

–

ResilienceSettings

ResilienceSettings control the reliability knobs in Envoy when making outbound connections from a gateway or proxy workload.

Field	Description	Validation Rule
httpRequestTimeout	google.protobuf.Duration This field is DEPRECATED in favor of `upstreamTrafficSettings.resilience.connectionPool.http.requestTimeout`. Timeout for HTTP requests. Disabled if not set.	–
httpRetries	tetrateio.api.tsb.traffic.v2.HTTPRetry This field is DEPRECATED in favor of `upstreamTrafficSettings.resilience.connectionPool.http.retries`. Retry policy for HTTP requests. Disabled if not set.	–
tcpKeepalive	google.protobuf.BoolValue Deprecated. This field will be removed in upcoming releases. Please use the `keep_alive` field instead. If enabled, sets SO_KEEPALIVE on the socket to enable TCP keepalive.	–
keepAlive	tetrateio.api.tsb.traffic.v2.KeepAliveSettings Keep Alive Settings.	–
circuitBreakerSensitivity	tetrateio.api.tsb.traffic.v2.ResilienceSettings.Sensitivity This field is DEPRECATED in favor of `upstreamTrafficSettings.resilience.circuitBreakerSensitivity`. Circuit breakers in Envoy are applied per endpoint in a load balancing pool. By default, circuit breakers are disabled. If set, the sensitivity level determines the maximum number of consecutive failures that Envoy will tolerate before ejecting an endpoint from the load balancing pool.	–

TcpKeepAlive

Field	Description	Validation Rule
probes	google.protobuf.UInt32Value The total number of unacknowledged probes to send before deciding the connection is dead. Default is to use the OS level configuration, Linux defaults to 9.	–
idleTime	google.protobuf.UInt32Value The number of seconds a connection needs to be idle before keep-alive probes start being sent. Default is to use the OS level configuration, Linux defaults to 7200s.	–
interval	google.protobuf.UInt32Value The number of seconds between keep-alive probes. Default is to use the OS level configuration, Linux defaults to 75s.	–

TcpKeepAliveSettings

TCP Keep Alive Settings.

Field	Description	Validation Rule
downstream	tetrateio.api.tsb.traffic.v2.TcpKeepAlive TCP Keep Alive Settings associated with the downstream (client) connection.	–
upstream	tetrateio.api.tsb.traffic.v2.TcpKeepAlive This field is DEPRECATED in favor of `upstreamTrafficSettings.resilience.connectionPool.tcp.keepAlive`. TCP Keep Alive Settings associated with the upstream (backend) connection.	–

TrafficSetting

A traffic setting applies configuration to a set of proxy workloads in a traffic group or a workspace. When applied to a traffic group, missing fields will inherit values from the workspace-wide setting if any.

Field	Description	Validation Rule
reachability	tetrateio.api.tsb.traffic.v2.ReachabilitySettings The set of services and hosts accessed by a workload (and hence its sidecar) in the mesh. Defining the set of services accessed by a workload (i.e. its dependencies) in advance reduces the memory and CPU consumption both the Istio control plane and the individual Envoy proxy workloads in the data plane.	–
resilience	tetrateio.api.tsb.traffic.v2.ResilienceSettings Resilience settings such as timeouts, retries, etc., affecting outbound traffic from proxy workloads.	–
egress	tetrateio.api.tsb.traffic.v2.TrafficSetting.EgressGateway Specifies the details of the egress proxy to which unknown traffic should be forwarded to from the proxy workload. If not specified, the proxy workload will send the unknown traffic directly to the IP requested by the application.	–
rateLimiting	tetrateio.api.tsb.gateway.v2.RateLimiting Configuration for rate limiting requests. These settings are only applied to sidecar proxies in the traffic group. Use the rateLimiting field in the Tier1Gateway and the Ingressgateway API to configure ratelimiting at the ingressgateway proxies.	–
upstreamTrafficSettings	List of tetrateio.api.tsb.traffic.v2.UpstreamTrafficSettings List of hosts and the associated traffic settings to be used by the clients that are downstreams to the defined upstream hosts.	–
configGenerationMetadata	tetrateio.api.tsb.types.v2.ConfigGenerationMetadata Metadata values that will be add into the Istio generated configurations. When using YAML APIs like`tctl` or `gitops`, put them into the `metadata.labels` or `metadata.annotations` instead. This field is only necessary when using gRPC APIs directly.	–

EgressGateway

EgressGateway specifies the gateway where traffic external to the mesh will be redirected.

Field	Description	Validation Rule
host	string REQUIRED Specifies the egress gateway hostname. Must be in `\<namespace\>/\<fqdn\>` format.	string = { pattern: `^[^/]+/[^/]+$` }
port	int32 Deprecated. This field is ignored and will be removed in upcoming releases. Specifies the port on the host to connect to.	–

UpstreamResilienceSettings

UpstreamResilienceSettings controls the reliability knobs for client connections to the upstream hosts.

Field	Description	Validation Rule
connectionPool	tetrateio.api.tsb.traffic.v2.UpstreamResilienceSettings.ConnectionPoolSettings Configures tolerance and other settings for TCP/HTTP connections to the service.	–
circuitBreakerSensitivity	tetrateio.api.tsb.traffic.v2.UpstreamResilienceSettings.Sensitivity Circuit breakers in Envoy are applied per endpoint in a load balancing pool. By default, circuit breakers are disabled. If set, the sensitivity level determines the maximum number of consecutive failures that Envoy will tolerate before ejecting an endpoint from the load balancing pool.	enum = { defined_only: `true` }

ConnectionPoolSettings

Connection pool settings for the upstream host.

Field	Description	Validation Rule
http	tetrateio.api.tsb.traffic.v2.UpstreamResilienceSettings.ConnectionPoolSettings.HTTP HTTP connection pool settings	–
tcp	tetrateio.api.tsb.traffic.v2.UpstreamResilienceSettings.ConnectionPoolSettings.TCP TCP connection pool settings	–

HTTP

HTTP Settings for outbound requests.

Field	Description	Validation Rule
requestTimeout	google.protobuf.Duration Timeout for HTTP requests. format: 1h/1m/1s/1ms. MUST BE >=1ms. Disabled if not set.	–
retries	tetrateio.api.tsb.traffic.v2.HTTPRetry Retry policy for HTTP requests. Disabled if not set.	–
maxRequests	uint32 Maximum number of active requests to the service. Applicable to both HTTP/1.1 and HTTP2. Default 0, meaning "unlimited", up to 2^32 - 1.	–
maxRequestsPerConnection	uint32 Maximum number of requests per connection to the service. If set to 1, it disables keep alive. Default 0, meaning "unlimited", up to 2^29.	–

TCP

TCP Settings for outbound requests.

Field	Description	Validation Rule
keepAlive	tetrateio.api.tsb.traffic.v2.TcpKeepAlive Keep Alive Settings.	–
maxConnections	uint32 Maximum number of HTTP1 /TCP connections to the service. Default 0, meaning "unlimited", up to 2^32 - 1.	–
connectTimeout	google.protobuf.Duration TCP connection timeout. format: 1h/1m/1s/1ms. MUST BE >=1ms. Default is 10s.	duration = { gte: `{nanos:1000000}` }

UpstreamTrafficSettings

Traffic settings for the clients that are downstreams to the defined upstream hosts.

Field	Description	Validation Rule
hosts	List of string List of hosts for which the settings will be created. Can contain wildcard hosts. The host should be a service from the service registry or a host declared by ServiceEntries.	repeated = { items: `{string:{min_len:1}}` }
settings	tetrateio.api.tsb.traffic.v2.UpstreamTrafficSettings.Settings A single setting to be applied to all the clients connecting to the upstream hosts.	–

Settings

Traffic settings to be applied to the clients of the upstream hosts.

Field	Description	Validation Rule
resilience	tetrateio.api.tsb.traffic.v2.UpstreamResilienceSettings Resilience settings for the clients.	–
loadBalancer	tetrateio.api.tsb.traffic.v2.LoadBalancerSettings Load balancing settings for the clients.	–

SimpleLB

Standard load balancing algorithms that require no tuning.

Field	Number	Description
UNSPECIFIED	0	No load balancing algorithm has been specified by the user. An appropriate default will be used.
RANDOM	2	The random load balancer selects a random healthy host. The random load balancer generally performs better than round robin if no health checking policy is configured.
PASSTHROUGH	3	This option will forward the connection to the original IP address requested by the caller without doing any form of load balancing. This option must be used with care. It is meant for advanced use cases. Refer to Original Destination load balancer in Envoy for further details.
ROUND_ROBIN	4	A basic round robin load balancing policy. This is generally unsafe for many scenarios (e.g. when enpoint weighting is used) as it can overburden endpoints. In general, prefer to use LEAST_REQUEST as a drop-in replacement for ROUND_ROBIN.
LEAST_REQUEST	5	The least request load balancer spreads load across endpoints, favoring endpoints with the least outstanding requests. This is generally safer and outperforms ROUND_ROBIN in nearly all cases. Prefer to use LEAST_REQUEST as a drop-in replacement for ROUND_ROBIN.

Mode

A short cut for defining the common reachability patterns

Field	Number	Description
UNSET	0	Inherit from parent if possible. Otherwise treated as `CLUSTER`.
NAMESPACE	1	The workload may talk to any service in its own namespace.
GROUP	2	The workload may talk to any service in the traffic group.
WORKSPACE	3	The workload may talk to any service in the workspace.
CLUSTER	4	The workload may talk to any service in the cluster.
CUSTOM	5	The workload may talk to services defined explicitly.

Sensitivity

Available sensitivity levels for the circuit breaker.

Field	Number	Description
UNSET	0	Default values will be used.
LOW	1	Tolerate up to 20 consecutive 5xx or connection failures from an endpoint before ejecting it temporarily from the load balancing pool.
MEDIUM	2	Tolerate up to 10 consecutive 5xx or connection failures from an endpoint before ejecting it temporarily from the load balancing pool.
HIGH	3	Tolerate up to 5 consecutive 5xx or connection failures from an endpoint before ejecting it temporarily from the load balancing pool.

Sensitivity

Available sensitivity levels for the circuit breaker.

Field	Number	Description
UNSET	0	Default values will be used.
LOW	1	Tolerate up to 20 consecutive 5xx or connection failures from an endpoint before ejecting it temporarily from the load balancing pool.
MEDIUM	2	Tolerate up to 10 consecutive 5xx or connection failures from an endpoint before ejecting it temporarily from the load balancing pool.
HIGH	3	Tolerate up to 5 consecutive 5xx or connection failures from an endpoint before ejecting it temporarily from the load balancing pool.

HTTPRetry​

KeepAliveSettings​

LoadBalancerSettings​

ConsistentHashLB​

HTTPCookie​

MagLev​

RingHash​

ReachabilitySettings​

ResilienceSettings​

TcpKeepAlive​

TcpKeepAliveSettings​

TrafficSetting​

EgressGateway​

UpstreamResilienceSettings​

ConnectionPoolSettings​

HTTP​

TCP​

UpstreamTrafficSettings​

Settings​

SimpleLB​

Mode​

Sensitivity​

Sensitivity​

HTTPRetry

KeepAliveSettings

LoadBalancerSettings

ConsistentHashLB

HTTPCookie

MagLev

RingHash

ReachabilitySettings

ResilienceSettings

TcpKeepAlive

TcpKeepAliveSettings

TrafficSetting

EgressGateway

UpstreamResilienceSettings

ConnectionPoolSettings

HTTP

TCP

UpstreamTrafficSettings

Settings

SimpleLB

Mode

Sensitivity

Sensitivity