Status

The differences between TSB and TSE API

Tetrate Service Express (TSE) utilizes many of the same components as the Tetrate Service Bridge(TSB) product but has the several distinctions. Go to Comparing TSE and TSB for more details.

Each resource in TSE is able to provide a status to let the user know it's current integrity. Some resources, like configurations for ingress, traffic and security, are not immediately applied as soon as TSE accepts any modification from user. In these cases, the status will provide enough information to know when it is really applying to the affected workloads. This allows any user or CI/CD process to poll the status of any desired resource and proceed accordingly.

There are two types of resources, the ones that aggregate the status of children resources and the ones that do not. Check the documentation for the different details object types for further information.

As an example, lets say the user pushes an IngressGateway configuration. IngressGateway does not aggregate status of children resources, but the other way around: its parent resource GatewayGroup does aggregate its status.

When the requests succeeds in TSE server, that resource's status will reach the ACCEPTED status with a TSE _ACCEPTED event in its configEvents details:

apiVersion: api.tsb.tetrate.io/v2
kind: ResourceStatus
metadata:
  name: bookinfo-gateway
  organization: tse
  tenant: tse
  workspace: bookinfo-ws
  gatewaygroup: bookinfo-gw-group
spec:
  status: ACCEPTED
  configEvents:
    events:
    - etag: '"sMlEWPbvm6M="'
      timestamp: "2022-01-11T10:11:41.784168161Z"
      type: TSE _ACCEPTED

Then, when pushed to MPC it succeeds and stays in ACCEPTED status, and the event list reflects the new event data, which will become:

// omiting the rest of the fields for simplicity
spec:
  status: ACCEPTED
  configEvents:
    events:
    - etag: '"sMlEWPbvm6M="'
      timestamp: "2022-01-11T10:11:43.264330637Z"
      type: MPC_ACCEPTED
    - etag: '"sMlEWPbvm6M="'
      timestamp: "2022-01-11T10:11:41.784168161Z"
      type: TSE _ACCEPTED

Later on, if there is an error in the MPC underlying layers such as XCP Central, a new event will be propagated and appended to the resource status that will change to status FAILED with the corresponding message.

# omiting the rest of the fields for simplicity
spec:
  status: FAILED
  message: "IngressGateway.xcp.tetrate.io \"INVALID-96010ce1d9b7df5c\" is invalid: metadata.name:
    Invalid value: \"INVALID-96010ce1d9b7df5c\": a DNS-1123 subdomain must consist of lower case alphanumeric characters,
    '-' or '.', and must start and end with an alphanumeric character
    (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')"
  configEvents:
    events:
    - etag: '"sMlEWPbvm6M="'
      message: "IngressGateway.xcp.tetrate.io \"INVALID-96010ce1d9b7df5c\" is invalid: metadata.name:
        Invalid value: \"INVALID-96010ce1d9b7df5c\": a DNS-1123 subdomain must consist of lower case alphanumeric characters,
        '-' or '.', and must start and end with an alphanumeric character
        (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')"
      reason: "ValidationFailed"
      timestamp: "2022-01-11T10:11:43.444335769Z"
      type: XCP_REJECTED
    - etag: '"sMlEWPbvm6M="'
      timestamp: "2022-01-11T10:11:43.264330637Z"
      type: MPC_ACCEPTED
    - etag: '"sMlEWPbvm6M="'
      timestamp: "2022-01-11T10:11:41.784168161Z"
      type: TSE _ACCEPTED

Another example of a status of a resource that aggregates its children status could be the following:

apiVersion: api.tsb.tetrate.io/v2
kind: ResourceStatus
metadata:
  name: bookinfo
  organization: tse
  tenant: tse
  workspace: bookinfo
spec:
  aggregatedStatus:
    configEvents:
      events:
      - etag: '"XAdtTSjZGic="'
        timestamp: "2022-01-11T16:50:15.571985056Z"
        type: XCP_ACCEPTED
      - etag: '"XAdtTSjZGic="'
        timestamp: "2022-01-11T16:50:15.545956009Z"
        type: MPC_ACCEPTED
      - etag: '"XAdtTSjZGic="'
        timestamp: "2022-01-11T16:50:13.547777908Z"
        type: TSE _ACCEPTED
  status: ACCEPTED

In case of errors, the children_errors map would be filled.

Finally, an example of a status of a non-configurable resource like a Tenant would not have any details. This kind of resources don't aggregate status either. This kind of resource will reach the READY status once it's request has been processed by the TSE server.

apiVersion: api.tsb.tetrate.io/v2
kind: ResourceStatus
metadata:
  name: tetrate
  organization: tse
spec:
  status: READY

AggregatedStatus

AggregatedStatus is used by resources with children to aggregate both the sequence of events and the status of its children resources.

Field	Description	Validation Rule
configEvents	tetrateio.api.tsb.v2.ConfigEvents ConfigEvents is the list of resource events that occurred during the lifecycle of the resource.	–
children	map<string, tetrateio.api.tsb.v2.AggregatedStatus.ChildStatus> Map of children resource FQNs to their status.	–
childrenStatus	tetrateio.api.tsb.v2.AggregatedStatus.ChildStatus Children status is a status summary of all the children statuses. If all of them are READY, children_status will be READY as well. If any is not READY, the worst status will be used for children_status	–

ChildStatus

ChildStatus contains the status details for a particular child resource, and a human-friendly message further describing the status if it is an errored one.

Field	Description	Validation Rule
status	tetrateio.api.tsb.v2.ResourceStatus.Status Current status of the child resource.	–
message	string Contains the human-friendly message describing the status of the child resource.	–

ConfigEvents

ConfigEvents provides a way to notify the status of a configuration propagation as a sequence of events.

Field	Description	Validation Rule
events	List of tetrateio.api.tsb.v2.ConfigEvents.Event Sequence of events occurred under the configuration propagation flow. It's ordered by event timestamp, newest first.	–

EdgeConfigState

Field	Description	Validation Rule
status	tetrateio.api.tsb.v2.ConfigEvents.EdgeConfigStatus Edge level config status.	–
reason	string Accompanying reason when status is not APPLIED.	–

Event

Single Event event occurred in the configuration propagation flow.

Field	Description	Validation Rule
type	tetrateio.api.tsb.v2.ConfigEvents.EventType Type of the event.	–
reason	string Optional code that extends the type of the occurred event.	–
message	string Optional message describing the reason in a human readable way.	–
timestamp	google.protobuf.Timestamp Time of the event occurrence.	–
etag	string The etag of the resource which configuration triggered this event.	–
edgesState	map<string, tetrateio.api.tsb.v2.ConfigEvents.EdgeConfigState> Stores the edge cluster name to EdgeConfigState mapping. EdgeConfigState holds the [status + reason] for a resource config that is being applied at edges. Reason accompanying the Status is useful for pin-pointed debugging at edge level. For instance, a config whose config status is something other than APPLIED is accompanied by a reason telling why an error occurred while applying the config. This will help in debugging issues at an edge.	–

ResourceStatus

ResourceStatus provides the current status of any TSE resource.

Field	Description	Validation Rule
status	tetrateio.api.tsb.v2.ResourceStatus.Status Current status of the resource.	–
message	string User friendly message adding details of the status.	–
configEvents	tetrateio.api.tsb.v2.ConfigEvents oneof _details For resources without children resources, it provides the sequence of events that happened during the resource status lifecycle.	–
aggregatedStatus	tetrateio.api.tsb.v2.AggregatedStatus oneof _details For resources with children, it provides both the sequence of resource events and the children status.	–

EdgeConfigStatus

Field	Number	Description
UNKNOWN	0	UNKNOWN indicates an undefined status. Either the edge has not reported the status for the config or it is not available due to some delays or something else. This is a catch-all when we don't know what to do.
APPLIED	1	APPLIED indicates that the config has been successfully applied at the edge.
ERRORED	2	ERRORED indicates that some error occurred while applying config at an edge. This will be accompanied by a message which specifies the reason for the error.
IGNORED	3	IGNORED indicates that the config was ignored because of some misconfiguration in config yaml. For instance, applying DIRECT mode config within BRIDGED mode group.

EventType

Simple Status of the current resource. It's a projection of its details (events, etc.) that allows to easily know the status of the resource without requiring to check the details.

Field	Number	Description
INVALID	0	INVALID is the zero value and should never be reached.
TSB_ACCEPTED	1	TSB_ACCEPTED happens when the configuration has been validated and persisted by TSB. Note that there is no TSB_REJECTED because in case of an obvious syntax error, the client requests for the API will fail directly. The configuration will not be persisted and therefore no config status will be associated with it.
MPC_ACCEPTED	2	MPC_ACCEPTED happens when MPC receives the configuration from TSB. Note that there is no MPC_REJECTED because it's just a pass-through to XCP.
XCP_ACCEPTED	3	XCP_ACCEPTED happens when XCP validates the configuration and the XCP resource is properly created.
XCP_REJECTED	4	XCP_REJECTED happens when XCP reports that the configuration is not valid.
MPC_FAILED	5	MPC_FAILED happens when MPC fails to process some configuration received from TSB. These failures are prior to sending the translated configurations to XCP.
XCP_UNKNOWN	6	XCP_UNKNOWN happens when XCP reports that all edges are in UNKNOWN state.
XCP_PARTIALLY_APPLIED	7	XCP_PARTIAL happens when XCP reports that at least one edge is in APPLIED state, and the rest are UNKNOWN.
XCP_APPLIED	8	XCP_APPLIED happens when XCP reports that every edge is in APPLIED state.
XCP_ERRORED	9	XCP_ERRORED happens when XCP reports that any edge is in ERRORED state.
XCP_IGNORED	10	XCP_IGNORED happens when XCP reports that the config is IGNORED by all the edges. One of the cases where configs are ignored is when a BRIDGED mode config object like IngressGateway is part of a gateway group configured for the DIRECT mode and vice versa. More generally, this happens when there is a mismatch between the mode where a config is valid and the mode configured for the group.
MPC_DIRTY	11	MPC_DIRTY happens when a resource that is dependent on others have not reached the desired status (even when they are not FAILED). For instance, when a resource configuration affected by a STRICTER propagation strategy gets superseded (fully or partially) by a stricter resource configuration higher up in the hierarchy. Concretely, if a security group's security settings (which is in ACCEPTED configuration state) is affected by a STRICTER propagation strategy, and for instance an organization's default security settings (a resource higher up in the hierarchy) has been updated to restrict more the previously set authorization policy, then the previously ACCEPTED security group's security settings (a resource lower in the hierarchy) will become DIRTY if it is not stricter.

Status

Simple Status of the current resource. It's a projection of its details (events, etc.) that allows to easily know the status of the resource without requiring to check the details.

Field	Number	Description
INVALID	0	INVALID status should never be reached. It indicates some problem occurred with the resource status, and would need to contact the admin to troubleshoot it. It's the default value but it's always expected to have one of the other values.
ACCEPTED	1	ACCEPTED is reached when the provided configuration has been validated and persisted by the TSB server.
READY	2	READY is reached when the resource is ready to be used. Non-configurable resources, like Organizations, Tenants or Users, will be ready as soon they are created. The configurable ones are ready when its configuration has been propagated to all the clusters.
FAILED	3	FAILED is reached in different situations, such as when: a resource configuration triggered some internal error. an offending resource affects the correct behaviour of the configuration. The message and details fields of the ResourceStatus provides the root cause of the error.
DIRTY	4	DIRTY is reached when the resources that are dependent on others have not reached the desired status (even when they are not FAILED). For example, an API resource that caused the creation of an IngressGateway could reach this status if the IngressGateway has been modified or removed directly.
PARTIAL	5	PARTIAL is reached for those resources that are dependent on other resources statuses, and not all the resources share the same status.