Tetrate Global Load BalancerVersion: Latest

Create the AWS Resources

In this chapter, you'll create the necessary pre-requisite resources for the Tetrate GSLB solution on AWS. Before you begin, make sure to determine which AWS regions you intend to deploy the GSLB solution into.

Installing on AWS

These instructions explain the pre-requesites when installing on AWS. For installation on Azure, refer to the Azure prerequisites.

You need two or more Kubernetes clusters across two or more AWS regions. This example uses three: cluster-1 and cluster-2 in us-east-1, and cluster-3 in us-west-2.

Each cluster needs:

An IAM OIDC provider, so IRSA (IAM Roles for Service Account) works. Check with aws eks describe-cluster --name <cluster> --query cluster.identity.oidc.issuer, and create one with eksctl utils associate-iam-oidc-provider if missing.
VPC attributes enableDnsSupport and enableDnsHostnames set to true, otherwise the VPC resolver won't answer for private hosted zones.
Outbound access to the Route53 API.

You'll also need the aws CLI v2, kubectl, helm v3 and jq.

Using AWS Load Balancer Controller

AWS implementations differ from Azure: on AWS a load balancer is published as a DNS name rather than an IP address. Tetrate GSLB handles this for you: it resolves the name in the background and publishes the addresses behind it. You don't need to do anything special — a normal AWS Load Balancer Controller Service works as-is.

Step 1: Create the Route53 private hosted zones

You need one global zone and one regional zone per region:

Zone	Name	Associate with
Global	global.example.com	Every cluster VPC in every region, plus client VPCs
Regional (us-east-1)	local.example.com	us-east-1 cluster and client VPCs only
Regional (us-west-2)	local.example.com	us-west-2 cluster and client VPCs only

Both regional zones use the same name. That's intentional: each cluster resolves edge-internal.local.example.com from whichever regional zone is associated with its own VPC.

export GLOBAL_DNS_ZONE=global.example.com
export LOCAL_DNS_ZONE=local.example.com
export REGION_PRIMARY=us-east-1
export REGION_SECONDARY=us-west-2

export VPC_1=vpc-0aaa...   # cluster-1, us-east-1
export VPC_2=vpc-0bbb...   # cluster-2, us-east-1
export VPC_3=vpc-0ccc...   # cluster-3, us-west-2

Create the global zone and associate every cluster VPC with it:

GLOBAL_ZONE_ID=$(aws route53 create-hosted-zone \
  --name "${GLOBAL_DNS_ZONE}" \
  --vpc "VPCRegion=${REGION_PRIMARY},VPCId=${VPC_1}" \
  --hosted-zone-config PrivateZone=true \
  --caller-reference "global-$(date +%s)" \
  --query 'HostedZone.Id' --output text | sed 's|/hostedzone/||')

aws route53 associate-vpc-with-hosted-zone --hosted-zone-id "${GLOBAL_ZONE_ID}" \
  --vpc "VPCRegion=${REGION_PRIMARY},VPCId=${VPC_2}"
aws route53 associate-vpc-with-hosted-zone --hosted-zone-id "${GLOBAL_ZONE_ID}" \
  --vpc "VPCRegion=${REGION_SECONDARY},VPCId=${VPC_3}"

Create one regional zone per region, associating only that region's VPCs:

# us-east-1
LOCAL_ZONE_ID_PRIMARY=$(aws route53 create-hosted-zone \
  --name "${LOCAL_DNS_ZONE}" \
  --vpc "VPCRegion=${REGION_PRIMARY},VPCId=${VPC_1}" \
  --hosted-zone-config PrivateZone=true \
  --caller-reference "local-primary-$(date +%s)" \
  --query 'HostedZone.Id' --output text | sed 's|/hostedzone/||')

aws route53 associate-vpc-with-hosted-zone --hosted-zone-id "${LOCAL_ZONE_ID_PRIMARY}" \
  --vpc "VPCRegion=${REGION_PRIMARY},VPCId=${VPC_2}"

# us-west-2 -- a separate zone with the same name
LOCAL_ZONE_ID_SECONDARY=$(aws route53 create-hosted-zone \
  --name "${LOCAL_DNS_ZONE}" \
  --vpc "VPCRegion=${REGION_SECONDARY},VPCId=${VPC_3}" \
  --hosted-zone-config PrivateZone=true \
  --caller-reference "local-secondary-$(date +%s)" \
  --query 'HostedZone.Id' --output text | sed 's|/hostedzone/||')

Keep the three zone IDs — you'll need them in Step 3.

Finally, check that every cluster VPC is associated with the global zone and exactly one regional zone. A missing association is the most common cause of clusters not seeing each other:

aws route53 get-hosted-zone --id "${GLOBAL_ZONE_ID}" --query 'VPCs[].VPCId'
aws route53 get-hosted-zone --id "${LOCAL_ZONE_ID_PRIMARY}" --query 'VPCs[].VPCId'

Using multiple AWS Accounts

If your cluster VPCs are in different AWS accounts, run aws route53 create-vpc-association-authorization from the zone's account first.

Step 2: Give External DNS access to the zones

Each cluster runs two External DNS instances that share one Kubernetes service account, so one IAM role per cluster is enough.

Create a policy allowing that cluster to write to its own regional zone and the global zone. Repeat per cluster, substituting the right regional zone ID:

cat > external-dns-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["route53:ChangeResourceRecordSets"],
      "Resource": [
        "arn:aws:route53:::hostedzone/${LOCAL_ZONE_ID_PRIMARY}",
        "arn:aws:route53:::hostedzone/${GLOBAL_ZONE_ID}"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "route53:ListResourceRecordSets",
        "route53:GetHostedZone",
        "route53:ListTagsForResource"
      ],
      "Resource": [
        "arn:aws:route53:::hostedzone/${LOCAL_ZONE_ID_PRIMARY}",
        "arn:aws:route53:::hostedzone/${GLOBAL_ZONE_ID}"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "route53:ListHostedZones",
        "route53:ListHostedZonesByName",
        "route53:GetChange"
      ],
      "Resource": ["*"]
    }
  ]
}
EOF

Create the role and let the cluster's service account assume it:

CLUSTER_NAME=cluster-1
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
OIDC=$(aws eks describe-cluster --name "${CLUSTER_NAME}" \
  --query 'cluster.identity.oidc.issuer' --output text | sed 's|https://||')

cat > trust.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": { "Federated": "arn:aws:iam::${ACCOUNT_ID}:oidc-provider/${OIDC}" },
    "Action": "sts:AssumeRoleWithWebIdentity",
    "Condition": {
      "StringEquals": {
        "${OIDC}:sub": "system:serviceaccount:tetrate-gslb-system:tetrate-gslb-external-dns",
        "${OIDC}:aud": "sts.amazonaws.com"
      }
    }
  }]
}
EOF

aws iam create-role --role-name "${CLUSTER_NAME}-external-dns" \
  --assume-role-policy-document file://trust.json

aws iam put-role-policy --role-name "${CLUSTER_NAME}-external-dns" \
  --policy-name external-dns-route53 \
  --policy-document file://external-dns-policy.json

aws iam get-role --role-name "${CLUSTER_NAME}-external-dns" --query 'Role.Arn' --output text

Note the role ARN for each cluster.

Using EKS Pod Identities

If you prefer to use EKS Pod Identity, create a pod identity association for the same namespace and service account. Use this instead of the serviceAccount.annotations.eks.amazonaws.com/role-arn annotation when installing Tetrate GSLB.

Step 1: Create the Route53 private hosted zones​

Step 2: Give External DNS access to the zones​

Using EKS Pod Identities​

Step 1: Create the Route53 private hosted zones

Step 2: Give External DNS access to the zones

Using EKS Pod Identities