Tetrate Service ExpressVersion: Latest

Agent Configuration

The differences between TSB and TSE API

Tetrate Service Express (TSE) utilizes many of the same components as the Tetrate Service Bridge(TSB) product but has the several distinctions. Go to Comparing TSE and TSB for more details.

Agent Configuration specifies configuration of the Workload Onboarding Agent.

In most cases, Workload Onboarding Agent can automatically recognize the host environment, e.g. AWS EC2, which makes explicit Agent Configuration optional.

By default, Workload Onboarding Agent comes with the minimal configuration:

apiVersion: config.agent.onboarding.tetrate.io/v1alpha1
kind: AgentConfiguration

which at runtime is interpreted as an equivalent of:

apiVersion: config.agent.onboarding.tetrate.io/v1alpha1
kind: AgentConfiguration
host:
  auto: \{\}
sidecar:
  istio: \{\}
  stdout:
    filename: /dev/stdout
  stderr:
    filename: /dev/stderr

The above configuration means that Workload Onboarding Agent should infer host environment automatically, should be in control of the Istio Sidecar pre-installed on that host, should redirect standard output of the Istio Sidecar into its own output.

Most users do not need to change the default configuration.

Users who make use of Istio revisions, need to specify the revision the pre-installed Istio Sidecar corresponds to, e.g.:

apiVersion: config.agent.onboarding.tetrate.io/v1alpha1
kind: AgentConfiguration
sidecar:
  istio:
    revision: canary

Users who want to redirect standard output of the Istio Sidecar into a separate file (instead of mixing together output of the Workload Onboarding Agent and output of the Istio Sidecar), should use the following configuration:

apiVersion: config.agent.onboarding.tetrate.io/v1alpha1
kind: AgentConfiguration
sidecar:
  stdout:
    filename: ./relative/path/to/file
  stderr:
    filename: /absolute/path/to/file

Relative path of a log file is interpreted as relative to the working directory of the Workload Onboarding Agent.

Advanced users who would like to utilize Workload Onboarding Agent in an environment that is not supported out-of-the-box, can develop custom Workload Onboarding Agent Plugins and use them by providing an explicit Agent Configuration, e.g.:

apiVersion: config.agent.onboarding.tetrate.io/v1alpha1
kind: AgentConfiguration
host:
  custom:
    credential:
    - plugin:
        name: custom-credential-provider
        path: /path/to/custom-credential-provider-binary
    hostinfo:
      plugin:
        name: custom-hostinfo-provider
        path: /path/to/custom-hostinfo-provider-binary
        env:
        - name: CONFIG
          value: /path/to/config
        args:
        - --name=value
settings:
  connection:
    timeout: 60s
    retryPolicy:
      exponentialBackoff:
        initialInterval: 10s
        maxInterval: 120s

Workload Onboarding Agent Plugin is an auxiliary executable (e.g. binary, shell script, Python script, etc) installed in addition to the Workload Onboarding Agent.

Workload Onboarding Agent executes a Workload Onboarding Agent Plugin to procure platform-specific information.

+--------------------------------------------------------+
| Host (e.g., VM or container)                           |
|                                                        |
|  +------------------+            +------------------+  |
|  |                  |            |                  |  |
|  |     Workload     | ---------\> |     Workload     |  |
|  | Onboarding Agent | (executes) | Onboarding Agent |  |
|  |                  |            |      Plugin      |  |
|  +------------------+            +------------------+  |
|                                                        |
+--------------------------------------------------------+

Workload Onboarding Agent Plugin is modeled as a gRPC service with unary call method(s). However, Workload Onboarding Agent Plugin does not run a network server. Instead, semantics of an unary RPC call is mapped onto execution of a process.

To make a call to the plugin, Workload Onboarding Agent:

runs executable of the Workload Onboarding Agent Plugin
passes parameters in via environment variables with the following names:
- PLUGIN_NAME - mandatory - e.g., aws-ec2-credential
- RPC_SERVICE_NAME - mandatory - e.g. tetrateio.api.onboarding.private.component.agent.plugin.credential.v1alpha1.CredentialPlugin
- RPC_METHOD_NAME - mandatory - e.g. GetCredential
writes request message serialized into JSON to the stdin of the plugin process
if plugin process exists with a 0 code, reads from stdout response message serialized into JSON
if plugin process exists with a non-0 code, reads from stdout RPC status message serialized into JSON
in a corner case where plugin process starts writing to stdout a response message, then encounters a failure and continues by writing to stdout an RPC status message, Workload Onboarding Agent should look at the exit code of the plugin process to decide how to interpret contents of stdout
plugin process must only print to stdout either a response message or an RPC status message
plugin process may print to stderr any data, e.g. diagnostic messages

In some cases instead of developing a custom plugin it is possible to reuse a built-in behavior.

E.g., instead of developing a custom HostInfo plugin you can reuse built-in behavior that simply lists available network interfaces instead of interacting with the platform-specific metadata API.

apiVersion: config.agent.onboarding.tetrate.io/v1alpha1
kind: AgentConfiguration
host:
  custom:
    credential:
    - plugin:
        name: custom-credential-provider
        path: /path/to/custom-credential-provider-binary
    hostinfo:
      basic:
        networkInterfaces:
          include:
          - ^eth[0-9]*$

AgentConfiguration

AgentConfiguration specifies configuration of the Workload Onboarding Agent.

Field	Description	Validation Rule
host	tetrateio.api.onboarding.config.agent.v1alpha1.HostEnvironment Configuration of the host environment. Defaults to automatically inferred configuration that will work out-of-the-box if `Workload Onboarding Agent` is deployed into `AWS EC2`.	–
sidecar	tetrateio.api.onboarding.config.agent.v1alpha1.Sidecar Configuration of the pre-installed sidecar. Defaults to `Istio Sidecar` installed at a well-known location (i.e., `/usr/local/bin/pilot-agent`, `/usr/local/bin/envoy`, etc).	–
settings	tetrateio.api.onboarding.config.agent.v1alpha1.Settings In-depth runtime configuration.	–

BasicHostInfo

BasicHostInfo specifies how to collect basic information about the host in a cross-platform way.

Field	Description	Validation Rule
networkInterfaces	tetrateio.api.onboarding.config.agent.v1alpha1.BasicHostInfo.NetworkInterfaces Filter on network interfaces that should be taken into account to determine IP addresses of the host. By default, all network interfaces will be taken into account, including Docker bridge(s) if any.	–

NetworkInterfaces

NetworkInterfaces specifies a filter on network interfaces that should be taken into account to determine IP addresses of the host.

For a network interface to be taken into account its name must be matched by one of the regular expressions on the include list and none of the regular expressions on the exclude list.

Field Description Validation Rule

Field	Description	Validation Rule
include	List of string Names of network interfaces that should be taken into account. The value is a regular expression (RE2 syntax). E.g., `^eth.*$`, `eth0`, etc. Empty list means take into account network interfaces with any name. See https://golang.org/s/re2syntax	repeated = { items: `{string:{min_len:1}}` }
exclude	List of string Names of network interfaces that should not be taken into account. The value is a regular expression (RE2 syntax). E.g., `^docker.*$`, `docker0`, etc. See https://golang.org/s/re2syntax	repeated = { items: `{string:{min_len:1}}` }

include

List of string
Names of network interfaces that should be taken into account.

The value is a regular expression (RE2 syntax).

E.g., ^eth.*$, eth0, etc.

Empty list means take into account network interfaces with any name.

See https://golang.org/s/re2syntax

repeated = {
items: {string:{min_len:1}}
}

exclude

List of string
Names of network interfaces that should not be taken into account.

The value is a regular expression (RE2 syntax).

E.g., ^docker.*$, docker0, etc.

See https://golang.org/s/re2syntax

repeated = {
items: {string:{min_len:1}}
}

ConnectionSettings

ConnectionSettings specifies settings that control execution of agent plugins, e.g. a timeout for a single plugin call, a retry policy for failed plugin calls, etc. The same settings apply to agent plugins of all kinds, e.g. credential plugins, host info plugins, etc.

Field	Description	Validation Rule
timeout	google.protobuf.Duration Timeout for a single plugin call. Must be greater than 1 millisecond. Defaults to `30s`.	duration = { gte: `{nanos:1000000}` }
retryPolicy	tetrateio.api.onboarding.config.agent.v1alpha1.RetryPolicy Retry policy for failed plugin calls. Defaults to the exponential backoff starting at `1s` and raising up to `15s` between retry attempts.	–

CredentialSource

CredentialSource specifies a source of a platform-specific credential.

Workload Onboarding Agent uses CredentialSource to procure a platform-specific credential.

Field	Description	Validation Rule
plugin	tetrateio.api.onboarding.config.agent.v1alpha1.Plugin ^{oneof _kind} Plugin (an executable binary) as a source of a platform-specific credential.	–

ExponentialBackoff

ExponentialBackoff specifies exponential backoff strategy.

Field	Description	Validation Rule
initialInterval	google.protobuf.Duration The initial interval between retry attempts. Must be greater than 1 millisecond.	duration = { gte: `{nanos:1000000}` }
maxInterval	google.protobuf.Duration The maximum interval between retry attempts. Must be greater than 1 millisecond.	duration = { gte: `{nanos:1000000}` }

HostEnvironment

HostEnvironment specifies information about the host environment.

Field Description Validation Rule

Field	Description	Validation Rule
auto	tetrateio.api.onboarding.config.agent.v1alpha1.HostEnvironment.Auto ^{oneof _kind} Automatically inferred environment. This is the default mode.	–
custom	tetrateio.api.onboarding.config.agent.v1alpha1.HostEnvironment.Custom ^{oneof _kind} Custom environment configured explicitly by the user.	–
aws	tetrateio.api.onboarding.config.agent.v1alpha1.HostEnvironment.Aws ^{oneof _kind} `AWS` environment.	–

auto

tetrateio.api.onboarding.config.agent.v1alpha1.HostEnvironment.Auto ^{oneof _kind}
Automatically inferred environment.

This is the default mode.

–

custom

tetrateio.api.onboarding.config.agent.v1alpha1.HostEnvironment.Custom ^{oneof _kind}
Custom environment configured explicitly by the user.

–

aws

tetrateio.api.onboarding.config.agent.v1alpha1.HostEnvironment.Aws ^{oneof _kind}
AWS environment.

–

Auto

Automatically inferred environment.

In this mode Workload Onboarding Agent checks whether the host environment is AWS EC2 which is supported out-of-the-box.

If Workload Onboarding Agent recognizes the environment, it will use builtin capabilities to interact with the respective platform-specific APIs to procure information about the host.

Aws

AWS environment.

Field	Description	Validation Rule
ec2	tetrateio.api.onboarding.config.agent.v1alpha1.HostEnvironment.Aws.Ec2 ^{oneof _kind} `AWS EC2` environment.	–

Ec2

AWS EC2 environment. In this mode Workload Onboarding Agent will use EC2 instance metadata service to procure information about the host, i.e. Private IP (aka VPC IP), Public IP (aka Internet IP), AWS IAM Role credentials, etc.

Custom

Custom environment configured explicitly by the user.

In this mode a user has to explicitly configure a list of Workload Onboarding Agent Plugins that procure information about the host using platform-specific APIs, e.g. plugin(s) to procure platform-specific credential of the host, a plugin to procure IP address(es) of the host, etc.

Field Description Validation Rule

Field	Description	Validation Rule
credential	List of tetrateio.api.onboarding.config.agent.v1alpha1.CredentialSource REQUIRED Source(s) of an environment-specific credential.	repeated = { min_items: `1` items: `{message:{required:true}}` }
hostinfo	tetrateio.api.onboarding.config.agent.v1alpha1.HostInfoSource Source of an environment-specific host information. Defaults to basic information about the host that can be collected in any environment.	–

credential

List of tetrateio.api.onboarding.config.agent.v1alpha1.CredentialSource
REQUIRED
Source(s) of an environment-specific credential.

repeated = {
min_items: 1
items: {message:{required:true}}
}

hostinfo

tetrateio.api.onboarding.config.agent.v1alpha1.HostInfoSource
Source of an environment-specific host information.

Defaults to basic information about the host that can be collected in any environment.

–

HostInfoSource

HostInfoSource specifies a source of platform-specific information about the host.

Workload Onboarding Agent uses HostInfoSource to procure platform-specific information about the host.

Field	Description	Validation Rule
plugin	tetrateio.api.onboarding.config.agent.v1alpha1.Plugin ^{oneof _kind} Plugin (an executable binary) as a source of platform-specific information about the host.	–
basic	tetrateio.api.onboarding.config.agent.v1alpha1.BasicHostInfo ^{oneof _kind} Collect basic information about the host in a cross-platform way.	–

IstioSidecar

Sidecar specifies configuration of the pre-installed Istio Sidecar.

Field Description Validation Rule

Field	Description	Validation Rule
revision	string Istio revision the pre-installed `Istio Sidecar` corresponds to. E.g., `canary`, `alpha`, etc. If omitted, it is assumed that the pre-installed `Istio Sidecar` corresponds to the `default` Istio revision. Notice that the value constraints here are stricter than the ones in Istio. Apparently, Istio validation rules allow values that lead to internal failures at runtime, e.g. values with capital letters or values longer than 56 characters. Stricter validation rules here are meant to prevent those hidden pitfalls.	string = { min_len: `1` max_len: `56` pattern: `^[a-z0-9](?:[-a-z0-9]*[a-z0-9])?$` ignore_empty: `true` }

revision

string
Istio revision the pre-installed Istio Sidecar corresponds to.

E.g., canary, alpha, etc.

If omitted, it is assumed that the pre-installed Istio Sidecar corresponds to the default Istio revision.

Notice that the value constraints here are stricter than the ones in Istio. Apparently, Istio validation rules allow values that lead to internal failures at runtime, e.g. values with capital letters or values longer than 56 characters. Stricter validation rules here are meant to prevent those hidden pitfalls.

string = {
  min_len: 1
  max_len: 56
  pattern: ^[a-z0-9](?:[-a-z0-9]*[a-z0-9])?$
  ignore_empty: true
}

Output

Destination for process output.

Field	Description	Validation Rule
filename	string ^{oneof _kind} Path to a file with standard output of the process.	–

Plugin

Plugin specifies a Workload Onboarding Agent Plugin as a source of platform-specific information.

Field	Description	Validation Rule
name	string REQUIRED Plugin reference name. E.g., `my-platform-credential`.	string = { min_len: `1` }
path	string Path to the plugin executable. Defaults to `onboarding-agent-\{\{ plugin name \}\}-plugin` that will be looked up on the `PATH`.	–
env	List of tetrateio.api.onboarding.config.agent.v1alpha1.Plugin.EnvVar Environment variables of the plugin.	–
args	List of string Arguments of the plugin.	repeated = { items: `{string:{min_len:1}}` }

EnvVar

EnvVar specifies a single environment variable.

Field	Description	Validation Rule
name	string REQUIRED Name of the environment variable.	string = { min_len: `1` }
value	string Value of the environment variable.	–

RetryPolicy

RetryPolicy specifies a retry policy for failed plugin calls.

Field	Description	Validation Rule
exponentialBackoff	tetrateio.api.onboarding.config.agent.v1alpha1.ExponentialBackoff ^{oneof _backoff} Exponential backoff strategy.	–

Settings

Settings specifies in-depth runtime configuration.

Field	Description	Validation Rule
connection	tetrateio.api.onboarding.config.agent.v1alpha1.ConnectionSettings Settings that control execution of agent plugins. Please notice that these settings apply only to execution of the Onboarding Agent plugins. These settings have no effect on requests from the Onboarding Agent to the Onboarding Plane. Also, notice that there is no physical "network connection" between the Onboarding Agent and its plugins. Onboarding Agent Plugin is a command-line tool that gets executed on demand; it uses standard input/output to receive/return data rather than network sockets.	–

Field

Description

Validation Rule

connection

tetrateio.api.onboarding.config.agent.v1alpha1.ConnectionSettings
Settings that control execution of agent plugins.

Please notice that these settings apply only to execution of the Onboarding Agent plugins. These settings have no effect on requests from the Onboarding Agent to the Onboarding Plane.

Also, notice that there is no physical "network connection" between the Onboarding Agent and its plugins. Onboarding Agent Plugin is a command-line tool that gets executed on demand; it uses standard input/output to receive/return data rather than network sockets.

–

Sidecar

Sidecar specifies configuration of the pre-installed sidecar.

Field	Description	Validation Rule
istio	tetrateio.api.onboarding.config.agent.v1alpha1.IstioSidecar ^{oneof _kind} Configuration of the pre-installed `Istio Sidecar`.	–
stdout	tetrateio.api.onboarding.config.agent.v1alpha1.Output Destination for the standard output of the sidecar. Relative path is interpreted as relative to the working directory of the `Workload Onboarding Agent`. Defaults to `/dev/stdout`.	–
stderr	tetrateio.api.onboarding.config.agent.v1alpha1.Output Destination for the standard error output of the sidecar. Relative path is interpreted as relative to the working directory of the `Workload Onboarding Agent`. Defaults to `/dev/stderr`.	–

AgentConfiguration​

BasicHostInfo​

NetworkInterfaces​

ConnectionSettings​

CredentialSource​

ExponentialBackoff​

HostEnvironment​

Auto​

Aws​

Ec2​

Custom​

HostInfoSource​

IstioSidecar​

Output​

Plugin​

EnvVar​

RetryPolicy​

Settings​

Sidecar​

AgentConfiguration

BasicHostInfo

NetworkInterfaces

ConnectionSettings

CredentialSource

ExponentialBackoff

HostEnvironment

Auto

Aws

Ec2

Custom

HostInfoSource

IstioSidecar

Output

Plugin

EnvVar

RetryPolicy

Settings

Sidecar