This guide walks you through setting up Flagger and AWS App Mesh on EKS.
The App Mesh integration with EKS is made out of the following components:
Kubernetes custom resources
mesh.appmesh.k8s.aws
defines a logical boundary for network traffic between the services
virtualnode.appmesh.k8s.aws
defines a logical pointer to a Kubernetes workload
virtualservice.appmesh.k8s.aws
defines the routing rules for a workload inside the mesh
CRD controller - keeps the custom resources in sync with the App Mesh control plane
Admission controller - injects the Envoy sidecar and assigns Kubernetes pods to App Mesh virtual nodes
Telemetry service - Prometheus instance that collects and stores Envoy's metrics
In order to create an EKS cluster you can use eksctl. Eksctl is an open source command-line utility made by Weaveworks in collaboration with Amazon.
On MacOS you can install eksctl with Homebrew:
brew tap weaveworks/tapbrew install weaveworks/tap/eksctl
Create an EKS cluster:
eksctl create cluster --name=appmesh \--region=us-west-2 \--nodes 3 \--node-volume-size=120 \--appmesh-access
The above command will create a two nodes cluster with App Mesh IAM policy attached to the EKS node instance role.
Verify the install with:
kubectl get nodes
Install the Helm command-line tool:
brew install kubernetes-helm
Create a service account and a cluster role binding for Tiller:
kubectl -n kube-system create sa tiller​kubectl create clusterrolebinding tiller-cluster-rule \--clusterrole=cluster-admin \--serviceaccount=kube-system:tiller
Deploy Tiller in the kube-system
namespace:
helm init --service-account tiller
You should consider using SSL between Helm and Tiller, for more information on securing your Helm installation see docs.helm.sh.
Install the Horizontal Pod Autoscaler (HPA) metrics provider:
helm upgrade -i metrics-server stable/metrics-server \--namespace kube-system \--set args[0]=--kubelet-preferred-address-types=InternalIP
After a minute, the metrics API should report CPU and memory usage for pods. You can very the metrics API with:
kubectl -n kube-system top pods
Create the appmesh-system
namespace:
kubectl create ns appmesh-system
Apply the App Mesh CRDs:
kubectl apply -k github.com/aws/eks-charts/stable/appmesh-controller//crds
Add the EKS repository to Helm:
helm repo add eks https://aws.github.io/eks-charts
Install the App Mesh CRD controller:
helm upgrade -i appmesh-controller eks/appmesh-controller \--wait --namespace appmesh-system
Install the App Mesh admission controller and create a mesh called global
:
helm upgrade -i appmesh-inject eks/appmesh-inject \--wait --namespace appmesh-system \--set mesh.create=true \--set mesh.name=global
Verify that the global mesh is active:
kubectl describe mesh​Status:Mesh Condition:Status: TrueType: MeshActive
In order to collect the App Mesh metrics that Flagger needs to run the canary analysis, you'll need to setup a Prometheus instance to scrape the Envoy sidecars.
Install the App Mesh Prometheus:
helm upgrade -i appmesh-prometheus eks/appmesh-prometheus \--wait --namespace appmesh-system
Add Flagger Helm repository:
helm repo add flagger https://flagger.app
Install Flagger's Canary CRD:
kubectl apply -f https://raw.githubusercontent.com/weaveworks/flagger/master/artifacts/flagger/crd.yaml
Deploy Flagger in the appmesh-system namespace:
helm upgrade -i flagger flagger/flagger \--namespace=appmesh-system \--set crd.create=false \--set meshProvider=appmesh \--set metricsServer=http://appmesh-prometheus:9090
You can enable Slack or MS Teams notifications with:
helm upgrade -i flagger flagger/flagger \--reuse-values \--namespace=appmesh-system \--set slack.url=https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK \--set slack.channel=general \--set slack.user=flagger
Flagger comes with a Grafana dashboard made for monitoring the canary analysis. Deploy Grafana in the appmesh-system namespace:
helm upgrade -i flagger-grafana flagger/grafana \--namespace=appmesh-system \--set url=http://appmesh-prometheus:9090
You can access Grafana using port forwarding:
kubectl -n appmesh-system port-forward svc/flagger-grafana 3000:80
Now that you have Flagger running you can try the App Mesh canary deployments tutorial.