Skip to main content
logoTetrate Service BridgeVersion: 1.8.x

Network Policy Recommendations

Network Policy Within a Mesh

In the context of a service mesh, Network Policies are set of rules and configurations that defines how various microservices can communicate with one another over the network. These policies play a pivotal role in controlling traffic flows, enforcing security measures and maintaining compliance within the service mesh architecture. Network Policies enable organizations to control communication between services by specifying permitted services, protocols, ports, and the allowable data types for exchange at the L3/L4 layer. This precise control stops unauthorized access, reduces areas vulnerable to attacks, and guarantees communication solely among trusted sources.

Network Policy & Zero Trust Architecture

Adding to the ideas of a service mesh, the Zero Trust Architecture (ZTA) approach to security considers that no user or device, whether from inside or outside the organization's network, should be automatically trusted. Instead of relying on old-fashioned perimeter defenses, ZTA emphasizes confirming identity and allowing access through rigorous authentication and authorization steps. You can learn more about it here.

What are the Challenges?

While TSB currently offers many security advantages via intelligent user controls and customized hierarchical access policies, the challenge arises when organizations aim to integrate Cilium or Calico network policies alongside existing identity-based access policies within a service mesh infrastructure. This situation necessitates the management of two distinct sets of policies, overseen by separate roles – Security Admins and Platform Owners, contributing to the complexities encountered by many organizations today.

How organizations can keep both identity based mesh access control policies and L3/L4 based network policies in sync. This is what TSB addresses by introducing Network Policy Recommendations.

What is Network Policy Recommendation?

Starting from TSB 1.7.0 version, TSB has the ability to suggest Kubernetes Network Policies. These recommendations are derived from the Hierarchical Access Control Policies set up by the Platform Owner or Application Owner, who handles these policies within TSB.

Any flows that would be allowed/denied by the recommended network policies are already allowed/denied by the TSB configuration. The policies are provided as a convenience, and are provided in a form that can be easily managed and understood by security teams and Kubernetes administrators. You can inspect the network policies to verify that TSB is applying appropriate access control policies to your mesh.

This feature can be enabled on each control plane cluster by setting ENABLE_NETWORK_POLICY_TRANSLATION to true in XCP component in ControlPlane CR. Once enabled, the recommended network policies are stored as namespace scoped configmap in the dedicated control plane clusters. Platform owner/Security owner who manages network specific access controls, can retrieve these policies from the configmap and verify & apply the policies in their desired namespaces.

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
name: controlplane
namespace: istio-system
spec:
components:
xcp:
...
kubeSpec:
overlays:
- apiVersion: install.xcp.tetrate.io/v1alpha1
kind: EdgeXcp
name: edge-xcp
patches:
- path: spec.components.edgeServer.kubeSpec.deployment.env
value:
- name: ENABLE_NETWORK_POLICY_TRANSLATION
value: "true"
...
...

What are the Use-cases?

TSB mostly focuses on the following use-cases -

Use-case 1: Recommendations to Secure North-South Traffic

When the user configures a Gateway object as Edge Gateway or an Ingress Gateway, TSB can make sure basic sanity by recommending the network policies that only allows the external traffic received on exposed ports like 80 or 443, for a particular host to only route to the TSB Gateway workloads ports i.e 8080, 8443 & 15443.

Pre-requisite

Enabling network policy enforcement on kubernetes clusters is not under the scope of TSB. Users need to make sure Container Network Interface (CNI) plugins are enabled in their respective kubernetes clusters before applying TSB recommended policies for it to have effect.

TSB Configurations

Configure an Edge Gateway to expose multiple hosts and route to other Tier2 clusters where an IngressGateway is deployed and configured to expose the same host names.

# gateway-config.yaml
apiVersion: v1
kind: List
items:
- apiVersion: install.tetrate.io/v1alpha1
kind: Tier1Gateway
metadata:
name: edge-gateway
namespace: edge-gw
spec:
kubeSpec:
service:
type: LoadBalancer
- apiVersion: gateway.tsb.tetrate.io/v2
kind: Gateway
metadata:
name: edge-gateway
namespace: edge-gw
annotations:
tsb.tetrate.io/organization: tetrate
tsb.tetrate.io/tenant: tetrate
tsb.tetrate.io/workspace: edge-gw-ws
tsb.tetrate.io/gatewayGroup: edge-gw-gp
spec:
workloadSelector:
namespace: edge-gw
labels:
app: edge-gateway
http:
- name: bookinfo
hostname: bookinfo.tetrate.io
port: 80
routing:
rules:
- match:
- uri:
prefix: "/productpage"
route:
clusterDestination:
clusters:
- name: gke-tetrate-us-west1-1
weight: 100
- match:
- uri:
prefix: "/api/v1/products"
route:
clusterDestination:
clusters:
- name: gke-tetrate-us-east1-2
weight: 100

When you apply the above configuration, based on the default Istio AuthZ policies that TSB creates and the Tier1Gateway install API configuration for creating the k8s service object for the exposed ports, TSB will start translating both in to a configmap with the recommended network policy configuration.

kubectl get configmap -l xcp.tetrate.io/recommended-network-policy=true -n edge-gw
Output
NAME                     DATA   AGE
np-authz-edge-gateway 1 31s
np-edge-gateway 1 31s

Retrieve the configmap and verify the recommended network policies before applying them on the clusters

kubectl get configmap -n edge-gw np-edge-gateway -o jsonpath='{.data.policy}' > edge-gw-policy.yaml

Recommended network policy restrict the ingress traffic only on the exposed ports configured in the service.

# edge-gw-policy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
creationTimestamp: null
labels:
xcp.tetrate.io/recommended-network-policy: "true"
xcp.tetrate.io/service: edge-gateway
name: np-edge-gateway
namespace: edge-gw
spec:
ingress:
- ports:
- port: 15443
- port: 8080
- port: 8443
podSelector:
matchLabels:
app: edge-gateway
istio: ingressgateway
policyTypes:
- Ingress
status: {}

Apply Network Policies on the cluster

kubectl apply -f edge-gw-policy.yaml
kubectl describe networkpolicy np-edge-gateway -n edge-gw
Output
Name:         np-edge-gateway
Namespace: edge-gw
Created on: 2023-08-16 21:40:49 +0530 IST
Labels: xcp.tetrate.io/recommended-network-policy=true
xcp.tetrate.io/service=edge-gateway
Annotations: <none>
Spec:
PodSelector: app=edge-gateway,istio=ingressgateway
Allowing ingress traffic:
To Port: 15443/TCP
To Port: 8080/TCP
To Port: 8443/TCP
From: <any> (traffic not restricted by source)
Not affecting egress traffic
Policy Types: Ingress

Use-case 2: Recommendations to Secure East-West Traffic

In a typical Tenant/Team/BU based access restrictions where when a user configures 2 Tenants i.e Tenant A and Tenant B and has configured deny_all as default in OrganizationSetting to deny all service to service communication. When the user configures an allow policy for Tenant A to talk to workspace-frontend under Tenant B but not workspace-backend under Tenant B. When TSB creates Istio AuthZ based policies to enforce this behaviour, TSB will also create recommended network policies as configmaps for the users to enforce the same behaviour at L3/L4 layer.

TSB Configurations

Creates the following configurations.

  • Create a new Tenant Marketing & Create 2 workspaces under Marketing Tenant
    • Workspace marketing-frontend
      • maps to marketing-frontend namespace in cluster-1
      • deploy productpage in marketing-frontend namespace
    • Workspace marketing-backend
      • maps to marketing-backend namespace in cluster-1
      • deploy details, reviews & ratings in marketing-backend namespace
  • Create a new Tenant Payment & Create a workspace under Payment Tenant
    • Workspace payment-chanel
      • maps to payment-channel namespace under cluster-1
      • deploy sleep service in payment-channel namespace
  • Allow payment-channel to talk to only marketing-frontend using TenantSettings
apiVersion: tsb.tetrate.io/v2
kind: TenantSetting
metadata:
name: default-setting
annotations:
tsb.tetrate.io/organization: tetrate
tsb.tetrate.io/tenant: marketing
spec:
defaultSecuritySetting:
authenticationSettings:
trafficMode: REQUIRED
authorization:
mode: RULES
rules:
allow:
- from:
fqn: organizations/tetrate/tenants/payment
to:
fqn: organizations/tetrate/tenants/marketing/workspaces/marketing-frontend
- from:
fqn: organizations/tetrate/tenants/marketing
to:
fqn: organizations/tetrate/tenants/marketing
displayName: default-setting
displayName: default-setting

When you apply the above configuration, TSB will create the following AuthZ policies.

kubectl get authorizationpolicy -A
Output
NAMESPACE            NAME                                     AGE
marketing-backend allow 8m26s
marketing-frontend allow 8m26s
payment-channel allow 8m26s

As we configured, since marketing-frontend and marketing-backend namespaces belong to same tenant Marketing, workloads belong to marketing-frontend namespace are allowed to talk to workloads belong to marketing-backend but not any other namespace i.e payment-channel

kubectl get authorizationpolicy allow -n marketing-backend -o yaml
Output
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
annotations:
tsb.tetrate.io/config-mode: bridged
tsb.tetrate.io/etag: '"VeslxrLeGvQ="'
tsb.tetrate.io/fqn: organizations/tetrate/tenants/marketing/workspaces/marketing-backend
tsb.tetrate.io/runtime-etag: '"enVCG/QQ2fc="'
xcp.tetrate.io/contentHash: a654b8f44747c21a28d0b44dcd193748
creationTimestamp: "2023-07-11T10:18:25Z"
generation: 1
name: allow
namespace: marketing-backend
resourceVersion: "4382832"
uid: 22c3265f-0be7-42da-9584-6913c2676f16
spec:
rules:
- from:
- source:
principals:
- gke-sreehari-us-west1-1.tsb.local/ns/marketing-backend/*
- gke-sreehari-us-west1-1.tsb.local/ns/marketing-frontend/*

But marketing-fontend allows requests from workloads belong to all workspaces under payment tenant and marketing tenant.

kubectl get authorizationpolicy allow -n marketing-frontend -o yaml
Output
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
annotations:
tsb.tetrate.io/config-mode: bridged
tsb.tetrate.io/etag: '"hjW1MOChsJU="'
tsb.tetrate.io/fqn: organizations/tetrate/tenants/marketing/workspaces/marketing-frontend
tsb.tetrate.io/runtime-etag: '"ZWzxphzAJEQ="'
xcp.tetrate.io/contentHash: f6098fa68d84259a71b1c42b3152402a
creationTimestamp: "2023-07-11T10:18:25Z"
generation: 1
name: allow
namespace: marketing-frontend
resourceVersion: "4382827"
uid: d4beae2c-1f66-4dd5-9662-7dd397519155
spec:
rules:
- from:
- source:
principals:
- gke-sreehari-us-east1-2.tsb.local/ns/payment-channel/*
- gke-sreehari-us-west1-1.tsb.local/ns/marketing-backend/*
- gke-sreehari-us-west1-1.tsb.local/ns/marketing-frontend/*
- gke-sreehari-us-west1-1.tsb.local/ns/payment-channel/*

With the above AuthZ based policies, sleep service in payment-channel namespace is allowed to call productpage in marketing-frontend but not allowed to call details, reviews, ratings in marketing-backend namespace.

kubectl exec "$(kubectl get pod -n payment-channel -l app=sleep -o jsonpath='{.items[0].metadata.name}')" -n payment-channel -c sleep -- curl -s http://productpage.marketing-frontend.svc:9080/api/v1/products -v
Output
< HTTP/1.1 200 OK
kubectl exec "$(kubectl get pod -n payment-channel -l app=sleep -o jsonpath='{.items[0].metadata.name}')" -n payment-channel -c sleep -- curl -s http://details.marketing-backend.svc:9080/details/1 -v
Output
RBAC: access denied
kubectl get configmap -l xcp.tetrate.io/recommended-network-policy=true,xcp.tetrate.io/authz-policy=allow -A
Output
NAMESPACE            NAME             DATA   AGE
marketing-backend np-authz-allow 1 1m2s
marketing-frontend np-authz-allow 1 1m2s
payment-channel np-authz-allow 1 1m2s

Apply authz specific network policy in marketing-backend namespace to allow ingress only from pods in marketing-frontend & marketing-backend namespace

kubectl get configmap -n marketing-backend np-authz-allow -o jsonpath='{.data.policy}' > backend-policy.yaml

Verify the recommended policy

# backend-policy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
annotations:
tsb.tetrate.io/config-mode: bridged
tsb.tetrate.io/etag: '"VeslxrLeGvQ="'
tsb.tetrate.io/fqn: organizations/tetrate/tenants/marketing/workspaces/marketing-backend
tsb.tetrate.io/runtime-etag: '"enVCG/QQ2fc="'
xcp.tetrate.io/contentHash: f53ddeb9e43de1ca56fd3ea20c21b919
creationTimestamp: null
labels:
xcp.tetrate.io/authz-policy: allow
xcp.tetrate.io/recommended-network-policy: "true"
name: np-authz-allow
namespace: marketing-backend
spec:
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: marketing-frontend
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: marketing-backend
podSelector: {}
policyTypes:
- Ingress
status: {}

Verify the same request after applying the above policy. you would notice that the request is denied while trying to call review,details, ratings from any other namespace other than marketing-frontend and marketing-backend.

kubectl exec "$(kubectl get pod -n payment-channel -l app=sleep -o jsonpath='{.items[0].metadata.name}')" -n payment-channel -c sleep -- curl -s http://details.marketing-backend.svc:9080/details/1 -v
Output
upstream connect error or disconnect/reset before headers

Troubleshooting

Once the network policy recommendation feature is enabled in a control plane cluster, XCP would consider all the existing namespaces which are mapped to TSB workspaces as the candidates for network policy translation. However users can choose to disable the network policy configmap generation on selected clusters by setting the ENABLE_NETWORK_POLICY_TRANSLATION to false on XCP component in ControlPlane CR.

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
name: controlplane
namespace: istio-system
spec:
components:
xcp:
...
kubeSpec:
overlays:
- apiVersion: install.xcp.tetrate.io/v1alpha1
kind: EdgeXcp
name: edge-xcp
patches:
- path: spec.components.edgeServer.kubeSpec.deployment.env
value:
- name: ENABLE_NETWORK_POLICY_TRANSLATION
value: "false"
...
...