198 lines
5.9 KiB
Markdown
198 lines
5.9 KiB
Markdown
|
# AWS - EKS
|
||
|
|
|
||
|
|
This is a POC of deploying an EKS stack on AWS, and some apps in it.
|
||
|
|
|
||
|
|
It uses Terraform for building the EKS cluster (1 node only, cuz $$), & another terraform configuration to deploy a couple of nginx nodes in the cluster.
|
||
|
|
|
||
|
|
## How?
|
||
|
|
|
||
|
|
### Before anything
|
||
|
|
|
||
|
|
Make sure to have a valid AWS account with the right permissions & policies.
|
||
|
|
|
||
|
|
Permissions required:
|
||
|
|
|
||
|
|
* AmazonEC2FullAccess
|
||
|
|
* IAMFullAccess
|
||
|
|
* AmazonEKSClusterPolicy
|
||
|
|
* AmazonVPCFullAccess
|
||
|
|
* AmazonEKSServicePolicy
|
||
|
|
|
||
|
|
Required policy:
|
||
|
|
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"Version": "2012-10-17",
|
||
|
|
"Statement": [
|
||
|
|
{
|
||
|
|
"Sid": "VisualEditor0",
|
||
|
|
"Effect": "Allow",
|
||
|
|
"Action": "eks:*",
|
||
|
|
"Resource": "*"
|
||
|
|
}
|
||
|
|
]
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
Once your IAM user is created, create the profile accordingly:
|
||
|
|
|
||
|
|
```sh
|
||
|
|
$ aws configure --profile infra-test
|
||
|
|
AWS Access Key ID [None]: AKxxxxxxxxxxxxxxxx
|
||
|
|
AWS Secret Access Key [None]: zWVxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||
|
|
Default region name [None]: eu-west-3
|
||
|
|
Default output format [None]: json
|
||
|
|
```
|
||
|
|
|
||
|
|
For all the next commands, make sure to use the `AWS_PROFILE` environment variable set to your profile id:
|
||
|
|
|
||
|
|
```sh
|
||
|
|
$ export AWS_PROFILE=infra-test
|
||
|
|
```
|
||
|
|
|
||
|
|
|
||
|
|
### First: EKS
|
||
|
|
|
||
|
|
Like any terraform deployments:
|
||
|
|
|
||
|
|
```sh
|
||
|
|
$ cd eks
|
||
|
|
$ terraform init
|
||
|
|
$ terraform plan -var "aws_profile=$AWS_PROFILE" -out tf.plan
|
||
|
|
$ terraform apply tf.plan
|
||
|
|
...
|
||
|
|
aws_eks_cluster.eks_cluster: Creation complete after 9m33s [id=eks-cluster-prod]
|
||
|
|
...
|
||
|
|
Apply complete! Resources: 4 added, 0 changed, 2 destroyed.
|
||
|
|
|
||
|
|
Outputs:
|
||
|
|
|
||
|
|
cluster_name = "eks-cluster-prod"
|
||
|
|
region = "eu-west-3"
|
||
|
|
|
||
|
|
$
|
||
|
|
```
|
||
|
|
|
||
|
|
Note that creating the initial EKS cluster will take up to 20 minutes in total (10 minutes for the eks cluster, 10 minutes to provision the nodes).
|
||
|
|
|
||
|
|
Once the cluster is built, make sure to configure your `.kube/config`:
|
||
|
|
|
||
|
|
```sh
|
||
|
|
$ terraform output
|
||
|
|
cluster_name = "eks-cluster-prod"
|
||
|
|
region = "eu-west-3"
|
||
|
|
|
||
|
|
$ aws eks --region $(terraform output -raw region) update-kubeconfig --name $(terraform output -raw cluster_name)
|
||
|
|
Added new context arn:aws:eks:eu-west-3:123456789012:cluster/eks-cluster-prod to /home/mycroft/.kube/config
|
||
|
|
|
||
|
|
$ kubectl get pods -A
|
||
|
|
NAMESPACE NAME READY STATUS RESTARTS AGE
|
||
|
|
kube-system aws-node-689lb 1/1 Running 0 111s
|
||
|
|
kube-system coredns-9b5d74bfb-b652h 1/1 Running 0 5m20s
|
||
|
|
kube-system coredns-9b5d74bfb-z6p6v 1/1 Running 0 5m20s
|
||
|
|
kube-system kube-proxy-xg5cp 1/1 Running 0 111s
|
||
|
|
```
|
||
|
|
|
||
|
|
### Second: Apps.
|
||
|
|
|
||
|
|
Once eks is deployed, and kubectl correctly configured, we can continue by deploying our app.
|
||
|
|
|
||
|
|
```sh
|
||
|
|
$ cd ../k8s
|
||
|
|
$ terraform init
|
||
|
|
$ terraform plan -var enable_nginx=1 -out tf.plan
|
||
|
|
$ terraform apply
|
||
|
|
...
|
||
|
|
Apply complete! Resources: 3 added, 0 changed, 1 destroyed.
|
||
|
|
```
|
||
|
|
|
||
|
|
As a result, let's verify there is our stuff deployed:
|
||
|
|
|
||
|
|
```sh
|
||
|
|
$ kubectl get pods --namespace testaroo
|
||
|
|
NAME READY STATUS RESTARTS AGE
|
||
|
|
alpine 1/1 Running 0 5m3s
|
||
|
|
nginx-98cf9b965-l785s 1/1 Running 0 5m3s
|
||
|
|
nginx-98cf9b965-smpkr 1/1 Running 0 5m3s
|
||
|
|
|
||
|
|
$ kubectl get deploy -n testaroo nginx -o wide
|
||
|
|
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
|
||
|
|
nginx 2/2 2 2 5m46s nginx-container nginx app=Nginx
|
||
|
|
|
||
|
|
$ kubectl get svc -n testaroo -o wide
|
||
|
|
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
|
||
|
|
nginx NodePort 172.20.10.182 <none> 80:31234/TCP 6m8s app=Nginx
|
||
|
|
```
|
||
|
|
|
||
|
|
### Reaching the app.
|
||
|
|
|
||
|
|
It is not possible with terraform output to retrieve the configured nodes. However, it is possible to retrieve IPs for our nodes using aws cli:
|
||
|
|
|
||
|
|
```sh
|
||
|
|
$ CLUSTER_IP=$(aws ec2 describe-instances \
|
||
|
|
--filters "Name=tag:k8s.io/cluster-autoscaler/eks-cluster-prod,Values=owned" \
|
||
|
|
--filters "Name=instance-state-name,Values=running" \
|
||
|
|
--query "Reservations[*].Instances[*].PublicIpAddress" \
|
||
|
|
--output text | head -1)
|
||
|
|
$ echo ${CLUSTER_IP}
|
||
|
|
52.47.91.179
|
||
|
|
$ curl http://$CLUSTER_IP:31234/
|
||
|
|
<!DOCTYPE html>
|
||
|
|
<html>
|
||
|
|
<head>
|
||
|
|
<title>Welcome to nginx!</title>
|
||
|
|
...
|
||
|
|
|
||
|
|
```
|
||
|
|
|
||
|
|
|
||
|
|
### Reaching the/a node ssh port:
|
||
|
|
|
||
|
|
Still using the AWS CLI to retrieve nodes, just:
|
||
|
|
|
||
|
|
```sh
|
||
|
|
$ ssh -i ~/.ssh/ec2-terraform.pem -l ec2-user $CLUSTER_IP
|
||
|
|
Last login: Fri Feb 11 13:21:00 2022 from xxxx.wanadoo.fr
|
||
|
|
|
||
|
|
__| __|_ )
|
||
|
|
_| ( / Amazon Linux 2 AMI
|
||
|
|
___|\___|___|
|
||
|
|
|
||
|
|
# docker ps|grep nginx
|
||
|
|
cc3aafd1a6ec nginx "/docker-entrypoint.…" 25 minutes ago Up 25 minutes k8s_nginx-container_nginx-98cf9b965-l785s_testaroo_e5ebf304-e156-4f6d-b00f-0f5dad0a9445_0
|
||
|
|
f4b998b0558e nginx "/docker-entrypoint.…" 25 minutes ago Up 25 minutes k8s_nginx-container_nginx-98cf9b965-smpkr_testaroo_eebe1868-fc5e-425e-948a-ce2cc2f2633e_0
|
||
|
|
14113cac359b 602401143452.dkr.ecr.eu-west-3.amazonaws.com/eks/pause:3.1-eksbuild.1 "/pause" 25 minutes ago Up 25 minutes k8s_POD_nginx-98cf9b965-l785s_testaroo_e5ebf304-e156-4f6d-b00f-0f5dad0a9445_0
|
||
|
|
c8c252673fbb 602401143452.dkr.ecr.eu-west-3.amazonaws.com/eks/pause:3.1-eksbuild.1 "/pause" 25 minutes ago Up 25 minutes k8s_POD_nginx-98cf9b965-smpkr_testaroo_eebe1868-fc5e-425e-948a-ce2cc2f2633e_0
|
||
|
|
```
|
||
|
|
|
||
|
|
|
||
|
|
### Going into a container
|
||
|
|
|
||
|
|
```sh
|
||
|
|
$ kubectl get pods -n testaroo alpine
|
||
|
|
NAME READY STATUS RESTARTS AGE
|
||
|
|
alpine 1/1 Running 0 29m
|
||
|
|
|
||
|
|
$ kubectl exec -ti -n testaroo alpine -- ps auxw
|
||
|
|
PID USER TIME COMMAND
|
||
|
|
1 root 0:00 sh -c while true; do sleep 3600; done
|
||
|
|
7 root 0:00 sleep 3600
|
||
|
|
8 root 0:00 ps auxw
|
||
|
|
|
||
|
|
$ kubectl exec -ti -n testaroo alpine -- sh
|
||
|
|
/ # echo "hello world"
|
||
|
|
hello world
|
||
|
|
/ #
|
||
|
|
```
|
||
|
|
|
||
|
|
## Todo:
|
||
|
|
|
||
|
|
* Move roles in the dedicated env;
|
||
|
|
|
||
|
|
|
||
|
|
## Notes
|
||
|
|
|
||
|
|
### Got an AWS error?
|
||
|
|
|
||
|
|
Decode it using `aws sts decode-authorization-message --encoded-message`.
|