Mount AWS Secret Manager Secrets with Amazon EKS

Amazon Elastic Kubernetes Service (Amazon EKS) has become one of the go-to choices for container orchestration in the cloud. Its managed Kubernetes service simplifies cluster management and scales effortlessly to meet the needs of your applications. However, securely managing sensitive information, such as API keys, database passwords, and other secrets, is a critical aspect of any application deployment. By default Kubernetes uses native secrets, but AWS offers a better single point management for the sensitive information. This is where AWS Secrets Manager comes into play. In this article, we'll explore how you can mount AWS Secret Manager secrets with Amazon EKS to enhance the security and manageability of your Kubernetes workloads.

Before diving into the integration with Amazon EKS, let's briefly discuss what AWS Secrets Manager is and why it's important.

AWS Secrets Manager is a fully managed service that helps you protect access to your applications, services, and IT resources without upfront investment and on-going maintenance costs. It enables you to rotate, manage, and retrieve database credentials, API keys, and other secrets centrally. Instead of hardcoding these secrets in your application code or configuration files, you can securely store and retrieve them from AWS Secrets Manager.

Below we will explore step by step procedure for integrating AWS EKS workloads with Amazon Secret Manager

Create Secret in AWS Secret Manager

We can create a secret in the AWS Secret Manager using below command

SECRET_ARN=$(aws --query ARN --output text secretsmanager create-secret --name 'prod/linux/test' --secret-string '{"username":"testuser", "password":"testpassword"}' --region <region> )

The obtained SECRET_ARN value is later used while creating the IAM policy

Install CSI Secret Store Driver

We need to install the CSI Secret Store. This driver integrates secrets stores with Kubernetes via a Container Storage Interface (CSI) volume.

~]$ helm repo add secrets-store-csi-driver https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
~]$ helm install -n kube-system csi-secrets-store secrets-store-csi-driver/secrets-store-csi-driver
NAME: csi-secrets-store
LAST DEPLOYED: Sat Sep 9 09:55:14 2023
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The Secrets Store CSI Driver is getting deployed to your cluster.

To verify that Secrets Store CSI Driver has started, run:

kubectl --namespace=kube-system get pods -l "app=secrets-store-csi-driver"

Now you can follow these steps https://secrets-store-csi-driver.sigs.k8s.io/getting-started/usage.html
to create a SecretProviderClass resource, and a deployment using the SecretProviderClass.

Install AWS Secrets and Config Provider

Below command can be used to install AWS secrets and config provider

~]$ helm repo add aws-secrets-manager https://aws.github.io/secrets-store-csi-driver-provider-aws
~]$ helm install -n kube-system secrets-provider-aws aws-secrets-manager/secrets-store-csi-driver-provider-aws
NAME: secrets-provider-aws
LAST DEPLOYED: Sat Sep 9 09:56:05 2023
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None

Create IAM Policy

We need to create an IAM policy which we can use to access Secret Manager and it should have GetSecretValue and DescribeSecret attributes.

Below command can be used to create the same

POLICY_ARN=$(aws --region "<region>"  --output text iam create-policy --query Policy.Arn --policy-name secretmanager-iam-policy --policy-document '{
"Version": "2012-10-17",
"Statement": [ {
"Effect": "Allow",
"Action": ["secretsmanager:GetSecretValue", "secretsmanager:DescribeSecret"],
"Resource": ["$SECRET_ARN"]
} ]
}')

The POLICY_ARN is later used while creating the service account

Create Service Account

We need to create a service account which will get the IAM role which we created above, this service Account is then later used in the deployment file

eksctl create iamserviceaccount --name abhi-deployment-sa --region="<region>" --cluster <clustername> --attach-policy-arn "$POLICY_ARN" --approve --override-existing-serviceaccounts

Create Secret Provider Class for AWS

Below yaml file can be used for creating the secret provider class. It should be noted that, we are specifying the secret which was created in the secret manager in the yaml. There are multiple ways of mentioning the secret, but those are beyond the scope of this article but is available over here

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: prod-linux-test
spec:
  provider: aws
  parameters:
    objects: |
        - objectName: "prod/linux/test"
          objectType: "secretsmanager"
          jmesPath:
            - path: username
              objectAlias: genericusername
            - path: password
              objectAlias: genericpassword #The file name to be mounted in the Amazon EKS pod

Mount the Secrets in a pod and verify

Below deployment file is used for mounting the secrets. In the volume section, we have mentioned the csi driver details, and we are providing the secret provider class which we have created before.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      serviceAccountName: abhi-deployment-sa
      volumes:
      - name: secrets-store-inline
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: "prod-linux-test"
      containers:
      - name: nginx-deployment
        image: nginx
        ports:
        - containerPort: 80
        volumeMounts:
        - name: secrets-store-inline
          mountPath: "/mnt/prod/linux"
          readOnly: true

Once the pods are up, we could see the secrets are mounted as required.

root@nginx-deployment-86dcdbf495-4vvzv:/# cd /mnt/prod/linux/
root@nginx-deployment-86dcdbf495-4vvzv:/mnt/prod/linux# ls
genericpassword genericusername prod_linux_test
root@nginx-deployment-86dcdbf495-4vvzv:/mnt/prod/linux# cat genericpassword
testpasswordroot@nginx-deployment-86dcdbf495-4vvzv:/mnt/prod/linux#
root@nginx-deployment-86dcdbf495-4vvzv:/mnt/prod/linux#
root@nginx-deployment-86dcdbf495-4vvzv:/mnt/prod/linux# cat genericusername
testuser

Conclusion

Integrating AWS Secrets Manager with Amazon EKS allows you to enhance the security and manageability of your Kubernetes workloads. By centralizing secret management, you can ensure that your applications always have access to the latest credentials while maintaining strong security practices. Whether you're building a new application or migrating an existing one to Amazon EKS, leveraging AWS Secrets Manager is a step toward a more secure and robust deployment.

Search on LinuxDataHub

Leave a Comment