AWS Cloud Map

This guide will walk you through setting up istio-registry-sync integration with AWS Cloud Map.

Prerequisites

Before you begin, you will need the following:

• An EKS cluster with Istio installed. If you do not have an EKS cluster with TIS Istio installed, you can follow the Installing TIS Guide.
  • Ensure the EKS cluster is associated with OIDC since you will be using IAM roles for service accounts to access AWS Cloud Map. See IAM Roles for Service Accounts for more details.
  • You must enable the DNS proxy in Istio. If you use Helm, you can do this by using following Helm values.yaml:

    meshConfig:
      defaultConfig:
        proxyMetadata:
          ISTIO_META_DNS_CAPTURE: "true"
          ISTIO_META_DNS_AUTO_ALLOCATE: "true"
    Then install or update Istio with the following command:

    helm upgrade --install istiod tetratelabs/istiod -n istio-system -f values.yaml
• An EC2 instance running NGINX as external service.
• AWS CLI, eksctl and helm installed and configured with the correct credentials.

Integration Steps

Following are steps to set up istio-registry-sync integration with AWS Cloud Map:

tip

You can skip the first two steps if you already have services running in your AWS account that already registered with Cloud Map.

Before continue, let's set up some environment variables. Make sure that you have correctly set $VPC and $REGION environment variable and you have configured your AWS CLI with the correct credentials.

export VPC=<YOUR_VPC_ID>
export REGION=<YOUR_AWS_REGION>
export EKS_CLUSTER_NAME=<YOUR_EKS_CLUSTER_NAME>
export TIS_NS=tis
export SLEEP_NS=sleep

1. Create a Cloud Map namespace

To create a Cloud Map namespace, you can use the AWS Management Console or CLI. For more details on Cloud Map namespaces and options for creating them, see the AWS Cloud Map documentation.

aws servicediscovery create-private-dns-namespace \
    --name tis.example.com \
    --vpc $VPC \
    --region $REGION

You then need to get namespace ID for the namespace you just created. You can do this by running the following command. You might need to wait a few seconds for the namespace to be created before running this command:

export CLOUDMAP_NAMESPACE=$(aws servicediscovery list-namespaces \
    --query "Namespaces[?Name=='tis.example.com'].Id" \
    --output text \
    --region $REGION)

2. Register a Service to Cloud Map

To register the service and the service instance, run the following commands. You will need to replace the EC2_INSTANCE_ID with the ID of the EC2 instance you are running NGINX on. You can get this by running aws ec2 describe-instances command.

export EC2_INSTANCE_ID=<YOUR_EC2_INSTANCE_ID>

aws servicediscovery create-service \
    --name nginx \
    --namespace-id $CLOUDMAP_NAMESPACE \
    --dns-config 'DnsRecords=[{Type="A", TTL=60}]' \
    --region $REGION

export SERVICE_ID=$(aws servicediscovery list-services \
    --query "Services[?Name=='nginx'].Id" \
    --output text \
    --region $REGION)

export EC2_INSTANCE_IP=$(aws ec2 describe-instances \
    --instance-ids $EC2_INSTANCE_ID \
    --query "Reservations[].Instances[].PublicIpAddress" \
    --output text \
    --region $REGION)

aws servicediscovery register-instance \
    --service-id $SERVICE_ID \
    --instance-id $EC2_INSTANCE_ID \
    --attributes AWS_INSTANCE_IPV4=$EC2_INSTANCE_IP,AWS_INSTANCE_PORT=80 \
    --region $REGION

Now that you have a service registered with Cloud Map, the next step is to create an IAM policy and service account that enables Cloud Map access from the EKS cluster.

3. Create an IAM Policy and Service Account

Create the necessary policy with permissions to access Cloud Map.

tip

To limit the scope of the policy, you can replace the * in the Resource section with the ARN of the namespace you created earlier.

cat <<EOF > AllowCloudMapAccess.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "servicediscovery:ListServices",
        "servicediscovery:ListInstances",
        "servicediscovery:ListNamespaces",
        "servicediscovery:DiscoverInstances"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF

aws iam create-policy \
    --policy-name "AllowCloudMapAccess" \
    --policy-document file://AllowCloudMapAccess.json \
    --region $REGION

Take note of the Policy.Arn returned by the command above. You will need it in the next step. Or you can get it by running the following command:

export CLOUDMAP_POLICY_ARN=$(aws iam list-policies \
    --query "Policies[?PolicyName=='AllowCloudMapAccess'].Arn" \
    --output text \
    --region $REGION)

Then create a service account that uses the policy you just created.

eksctl create iamserviceaccount \
    --cluster $EKS_CLUSTER_NAME \
    --name "tis-cloud-map" \
    --namespace "tis" \
    --region $REGION \
    --attach-policy-arn $CLOUDMAP_POLICY_ARN \
    --approve

To make sure the service account was created and has the correct role ARN annotated, you can use kubectl:

kubectl get serviceaccount tis-cloud-map -n tis -o yaml

The service account should annotation eks.amazonaws.com/role-arn: \<YOUR_ROLE_ARN\>.

Once you've set this up, any pod that's created using this service account will have the permissions that the IAM role provides, that is to say, the permissions to access Cloud Map.

4. Install istio-registry-sync

Now that you have a service registered with Cloud Map and a service account with the correct permissions, you can install istio-registry-sync to your cluster.

Create the following values.yaml file with the correct values for your environment. Use the same region and service account that you created earlier.

cat <<EOF > values.yaml
cloudmap:
  region: $REGION

publishNamespace: $SLEEP_NS

imagePullSecrets: 
  - name: tetrate-addons-creds
EOF

Run the following command to deploy istio-registry-sync to your cluster using Helm.

helm upgrade --install istio-registry-sync tis-addons/istio-registry-sync \
    --namespace tis --create-namespace \
    -f values.yaml

Make sure the istio-registry-sync pod is running:

kubectl get pods -n tis

NAME                                   READY   STATUS    RESTARTS   AGE
istio-registry-sync-64f77bdb77-l9hrb   1/1     Running   0          7m4s

5. Verify Istio Registry Sync

Install sleep sample app and verify that it can reach the nginx service that is registered in Cloud Map.

kubectl create ns $SLEEP_NS
kubectl label namespace $SLEEP_NS istio-injection=enabled
kubectl apply -n $SLEEP_NS -f https://raw.githubusercontent.com/istio/istio/master/samples/sleep/sleep.yaml

If everything is set up correctly, you should see the following output:

kubectl get serviceentry -n $SLEEP_NS

NAME                             HOSTS                       LOCATION   RESOLUTION   AGE
cloudmap-nginx.tis.example.com   ["nginx.tis.example.com"]              STATIC       44s

This means that istio-registry-sync has successfully synced the service registered with Cloud Map to Istio ServiceEntry.

Request to nginx service should be successful:

kubectl exec -n $SLEEP_NS -it deploy/sleep -- curl nginx.tis.example.com

Cleanup

# Delete istio-registry-sync
helm delete -n tis istio-registry-sync

# Delete service account
eksctl delete iamserviceaccount --region $REGION --name tis-cloud-map --namespace tis --cluster $EKS_CLUSTER_NAME

# Delete Cloud Map service and instances
aws servicediscovery deregister-instance --service-id $SERVICE_ID --instance-id $EC2_INSTANCE_ID --region $REGION
aws servicediscovery delete-service --id $SERVICE_ID --region $REGION
aws servicediscovery delete-namespace --id $CLOUDMAP_NAMESPACE --region $REGION

# Detach and delete IAM policy
export EKS_ROLE_NAME=$(kubectl get serviceaccount tis-cloud-map -n tis -o=jsonpath='{.metadata.annotations.eks\.amazonaws\.com/role-arn}' | cut -d'/' -f2)

aws iam detach-role-policy --role-name $EKS_ROLE_NAME --policy-arn $CLOUDMAP_POLICY_ARN
aws iam delete-policy --policy-arn $CLOUDMAP_POLICY_ARN

# Remove sleep app
kubectl delete -n $SLEEP_NS -f https://raw.githubusercontent.com/istio/istio/master/samples/sleep/sleep.yaml
kubectl delete ns $SLEEP_NS