Skip to main content
logoTetrate Service ExpressVersion: Latest

Agent Configuration

The differences between TSB and TSE API

Tetrate Service Express (TSE) utilizes many of the same components as the Tetrate Service Bridge(TSB) product but has the several distinctions. Go to Comparing TSE and TSB for more details.

Agent Configuration specifies configuration of the Workload Onboarding Agent.

In most cases, Workload Onboarding Agent can automatically recognize the host environment, e.g. AWS EC2, which makes explicit Agent Configuration optional.

By default, Workload Onboarding Agent comes with the minimal configuration:

apiVersion: config.agent.onboarding.tetrate.io/v1alpha1
kind: AgentConfiguration

which at runtime is interpreted as an equivalent of:

apiVersion: config.agent.onboarding.tetrate.io/v1alpha1
kind: AgentConfiguration
host:
auto: \{\}
sidecar:
istio: \{\}
stdout:
filename: /dev/stdout
stderr:
filename: /dev/stderr

The above configuration means that Workload Onboarding Agent should infer host environment automatically, should be in control of the Istio Sidecar pre-installed on that host, should redirect standard output of the Istio Sidecar into its own output.

Most users do not need to change the default configuration.

Users who make use of Istio revisions, need to specify the revision the pre-installed Istio Sidecar corresponds to, e.g.:

apiVersion: config.agent.onboarding.tetrate.io/v1alpha1
kind: AgentConfiguration
sidecar:
istio:
revision: canary

Users who want to redirect standard output of the Istio Sidecar into a separate file (instead of mixing together output of the Workload Onboarding Agent and output of the Istio Sidecar), should use the following configuration:

apiVersion: config.agent.onboarding.tetrate.io/v1alpha1
kind: AgentConfiguration
sidecar:
stdout:
filename: ./relative/path/to/file
stderr:
filename: /absolute/path/to/file

Relative path of a log file is interpreted as relative to the working directory of the Workload Onboarding Agent.

Advanced users who would like to utilize Workload Onboarding Agent in an environment that is not supported out-of-the-box, can develop custom Workload Onboarding Agent Plugins and use them by providing an explicit Agent Configuration, e.g.:

apiVersion: config.agent.onboarding.tetrate.io/v1alpha1
kind: AgentConfiguration
host:
custom:
credential:
- plugin:
name: custom-credential-provider
path: /path/to/custom-credential-provider-binary
hostinfo:
plugin:
name: custom-hostinfo-provider
path: /path/to/custom-hostinfo-provider-binary
env:
- name: CONFIG
value: /path/to/config
args:
- --name=value
settings:
connection:
timeout: 60s
retryPolicy:
exponentialBackoff:
initialInterval: 10s
maxInterval: 120s

Workload Onboarding Agent Plugin is an auxiliary executable (e.g. binary, shell script, Python script, etc) installed in addition to the Workload Onboarding Agent.

Workload Onboarding Agent executes a Workload Onboarding Agent Plugin to procure platform-specific information.

+--------------------------------------------------------+
| Host (e.g., VM or container) |
| |
| +------------------+ +------------------+ |
| | | | | |
| | Workload | ---------\> | Workload | |
| | Onboarding Agent | (executes) | Onboarding Agent | |
| | | | Plugin | |
| +------------------+ +------------------+ |
| |
+--------------------------------------------------------+

Workload Onboarding Agent Plugin is modeled as a gRPC service with unary call method(s). However, Workload Onboarding Agent Plugin does not run a network server. Instead, semantics of an unary RPC call is mapped onto execution of a process.

To make a call to the plugin, Workload Onboarding Agent:

  • runs executable of the Workload Onboarding Agent Plugin
  • passes parameters in via environment variables with the following names:
    • PLUGIN_NAME - mandatory - e.g., aws-ec2-credential
    • RPC_SERVICE_NAME - mandatory - e.g. tetrateio.api.onboarding.private.component.agent.plugin.credential.v1alpha1.CredentialPlugin
    • RPC_METHOD_NAME - mandatory - e.g. GetCredential
  • writes request message serialized into JSON to the stdin of the plugin process
  • if plugin process exists with a 0 code, reads from stdout response message serialized into JSON
  • if plugin process exists with a non-0 code, reads from stdout RPC status message serialized into JSON
  • in a corner case where plugin process starts writing to stdout a response message, then encounters a failure and continues by writing to stdout an RPC status message, Workload Onboarding Agent should look at the exit code of the plugin process to decide how to interpret contents of stdout
  • plugin process must only print to stdout either a response message or an RPC status message
  • plugin process may print to stderr any data, e.g. diagnostic messages

In some cases instead of developing a custom plugin it is possible to reuse a built-in behavior.

E.g., instead of developing a custom HostInfo plugin you can reuse built-in behavior that simply lists available network interfaces instead of interacting with the platform-specific metadata API.

apiVersion: config.agent.onboarding.tetrate.io/v1alpha1
kind: AgentConfiguration
host:
custom:
credential:
- plugin:
name: custom-credential-provider
path: /path/to/custom-credential-provider-binary
hostinfo:
basic:
networkInterfaces:
include:
- ^eth[0-9]*$

AgentConfiguration

AgentConfiguration specifies configuration of the Workload Onboarding Agent.

FieldDescriptionValidation Rule

host

tetrateio.api.onboarding.config.agent.v1alpha1.HostEnvironment
Configuration of the host environment. Defaults to automatically inferred configuration that will work out-of-the-box if Workload Onboarding Agent is deployed into AWS EC2.

sidecar

tetrateio.api.onboarding.config.agent.v1alpha1.Sidecar
Configuration of the pre-installed sidecar. Defaults to Istio Sidecar installed at a well-known location (i.e., /usr/local/bin/pilot-agent, /usr/local/bin/envoy, etc).

settings

tetrateio.api.onboarding.config.agent.v1alpha1.Settings
In-depth runtime configuration.

BasicHostInfo

BasicHostInfo specifies how to collect basic information about the host in a cross-platform way.

FieldDescriptionValidation Rule

networkInterfaces

tetrateio.api.onboarding.config.agent.v1alpha1.BasicHostInfo.NetworkInterfaces
Filter on network interfaces that should be taken into account to determine IP addresses of the host. By default, all network interfaces will be taken into account, including Docker bridge(s) if any.

NetworkInterfaces

NetworkInterfaces specifies a filter on network interfaces that should be taken into account to determine IP addresses of the host.

For a network interface to be taken into account its name must be matched by one of the regular expressions on the include list and none of the regular expressions on the exclude list.

FieldDescriptionValidation Rule

include

List of string
Names of network interfaces that should be taken into account.

The value is a regular expression (RE2 syntax).

E.g., ^eth.*$, eth0, etc.

Empty list means take into account network interfaces with any name.

See https://golang.org/s/re2syntax

repeated = {
  items: {string:{min_len:1}}
}

exclude

List of string
Names of network interfaces that should not be taken into account.

The value is a regular expression (RE2 syntax).

E.g., ^docker.*$, docker0, etc.

See https://golang.org/s/re2syntax

repeated = {
  items: {string:{min_len:1}}
}

ConnectionSettings

ConnectionSettings specifies settings that control execution of agent plugins, e.g. a timeout for a single plugin call, a retry policy for failed plugin calls, etc. The same settings apply to agent plugins of all kinds, e.g. credential plugins, host info plugins, etc.

FieldDescriptionValidation Rule

timeout

google.protobuf.Duration
Timeout for a single plugin call. Must be greater than 1 millisecond. Defaults to 30s.

duration = {
  gte: {nanos:1000000}
}

retryPolicy

tetrateio.api.onboarding.config.agent.v1alpha1.RetryPolicy
Retry policy for failed plugin calls. Defaults to the exponential backoff starting at 1s and raising up to 15s between retry attempts.

CredentialSource

CredentialSource specifies a source of a platform-specific credential.

Workload Onboarding Agent uses CredentialSource to procure a platform-specific credential.

FieldDescriptionValidation Rule

plugin

tetrateio.api.onboarding.config.agent.v1alpha1.Plugin oneof _kind
Plugin (an executable binary) as a source of a platform-specific credential.

ExponentialBackoff

ExponentialBackoff specifies exponential backoff strategy.

FieldDescriptionValidation Rule

initialInterval

google.protobuf.Duration
The initial interval between retry attempts. Must be greater than 1 millisecond.

duration = {
  gte: {nanos:1000000}
}

maxInterval

google.protobuf.Duration
The maximum interval between retry attempts. Must be greater than 1 millisecond.

duration = {
  gte: {nanos:1000000}
}

HostEnvironment

HostEnvironment specifies information about the host environment.

FieldDescriptionValidation Rule

auto

tetrateio.api.onboarding.config.agent.v1alpha1.HostEnvironment.Auto oneof _kind
Automatically inferred environment.

This is the default mode.

custom

tetrateio.api.onboarding.config.agent.v1alpha1.HostEnvironment.Custom oneof _kind
Custom environment configured explicitly by the user.

aws

tetrateio.api.onboarding.config.agent.v1alpha1.HostEnvironment.Aws oneof _kind
AWS environment.

Auto

Automatically inferred environment.

In this mode Workload Onboarding Agent checks whether the host environment is AWS EC2 which is supported out-of-the-box.

If Workload Onboarding Agent recognizes the environment, it will use builtin capabilities to interact with the respective platform-specific APIs to procure information about the host.

Aws

AWS environment.

FieldDescriptionValidation Rule

ec2

tetrateio.api.onboarding.config.agent.v1alpha1.HostEnvironment.Aws.Ec2 oneof _kind
AWS EC2 environment.

Ec2

AWS EC2 environment. In this mode Workload Onboarding Agent will use EC2 instance metadata service to procure information about the host, i.e. Private IP (aka VPC IP), Public IP (aka Internet IP), AWS IAM Role credentials, etc.

Custom

Custom environment configured explicitly by the user.

In this mode a user has to explicitly configure a list of Workload Onboarding Agent Plugins that procure information about the host using platform-specific APIs, e.g. plugin(s) to procure platform-specific credential of the host, a plugin to procure IP address(es) of the host, etc.

FieldDescriptionValidation Rule

credential

List of tetrateio.api.onboarding.config.agent.v1alpha1.CredentialSource
REQUIRED
Source(s) of an environment-specific credential.

repeated = {
  min_items: 1
  items: {message:{required:true}}
}

hostinfo

tetrateio.api.onboarding.config.agent.v1alpha1.HostInfoSource
Source of an environment-specific host information.

Defaults to basic information about the host that can be collected in any environment.

HostInfoSource

HostInfoSource specifies a source of platform-specific information about the host.

Workload Onboarding Agent uses HostInfoSource to procure platform-specific information about the host.

FieldDescriptionValidation Rule

plugin

tetrateio.api.onboarding.config.agent.v1alpha1.Plugin oneof _kind
Plugin (an executable binary) as a source of platform-specific information about the host.

basic

tetrateio.api.onboarding.config.agent.v1alpha1.BasicHostInfo oneof _kind
Collect basic information about the host in a cross-platform way.

IstioSidecar

Sidecar specifies configuration of the pre-installed Istio Sidecar.

FieldDescriptionValidation Rule

revision

string
Istio revision the pre-installed Istio Sidecar corresponds to.

E.g., canary, alpha, etc.

If omitted, it is assumed that the pre-installed Istio Sidecar corresponds to the default Istio revision.

Notice that the value constraints here are stricter than the ones in Istio. Apparently, Istio validation rules allow values that lead to internal failures at runtime, e.g. values with capital letters or values longer than 56 characters. Stricter validation rules here are meant to prevent those hidden pitfalls.

string = {
  min_len: 1
  max_len: 56
  pattern: ^[a-z0-9](?:[-a-z0-9]*[a-z0-9])?$
  ignore_empty: true
}

Output

Destination for process output.

FieldDescriptionValidation Rule

filename

string oneof _kind
Path to a file with standard output of the process.

Plugin

Plugin specifies a Workload Onboarding Agent Plugin as a source of platform-specific information.

FieldDescriptionValidation Rule

name

string
REQUIRED
Plugin reference name. E.g., my-platform-credential.

string = {
  min_len: 1
}

path

string
Path to the plugin executable. Defaults to onboarding-agent-\{\{ plugin name \}\}-plugin that will be looked up on the PATH.

env

List of tetrateio.api.onboarding.config.agent.v1alpha1.Plugin.EnvVar
Environment variables of the plugin.

args

List of string
Arguments of the plugin.

repeated = {
  items: {string:{min_len:1}}
}

EnvVar

EnvVar specifies a single environment variable.

FieldDescriptionValidation Rule

name

string
REQUIRED
Name of the environment variable.

string = {
  min_len: 1
}

value

string
Value of the environment variable.

RetryPolicy

RetryPolicy specifies a retry policy for failed plugin calls.

FieldDescriptionValidation Rule

exponentialBackoff

tetrateio.api.onboarding.config.agent.v1alpha1.ExponentialBackoff oneof _backoff
Exponential backoff strategy.

Settings

Settings specifies in-depth runtime configuration.

FieldDescriptionValidation Rule

connection

tetrateio.api.onboarding.config.agent.v1alpha1.ConnectionSettings
Settings that control execution of agent plugins.

Please notice that these settings apply only to execution of the Onboarding Agent plugins. These settings have no effect on requests from the Onboarding Agent to the Onboarding Plane.

Also, notice that there is no physical "network connection" between the Onboarding Agent and its plugins. Onboarding Agent Plugin is a command-line tool that gets executed on demand; it uses standard input/output to receive/return data rather than network sockets.

Sidecar

Sidecar specifies configuration of the pre-installed sidecar.

FieldDescriptionValidation Rule

istio

tetrateio.api.onboarding.config.agent.v1alpha1.IstioSidecar oneof _kind
Configuration of the pre-installed Istio Sidecar.

stdout

tetrateio.api.onboarding.config.agent.v1alpha1.Output
Destination for the standard output of the sidecar. Relative path is interpreted as relative to the working directory of the Workload Onboarding Agent. Defaults to /dev/stdout.

stderr

tetrateio.api.onboarding.config.agent.v1alpha1.Output
Destination for the standard error output of the sidecar. Relative path is interpreted as relative to the working directory of the Workload Onboarding Agent. Defaults to /dev/stderr.