May 6, 2021 by Thibault Debatty | 2134 views
https://cylab.be/blog/145/multi-tenant-kubernetes-cluster-namespace-quota-and-kubectl-user
When multiple users must use the same kubernetes cluster, a good practice consists in segregating them in isolated namespaces. In this tutorial we show how to create a namespace, apply quotas so the user cannot exhaust the resources (memory and cpu) of the cluster, and create a dedicated kubectl user.
kubectl create namespace example
Create a yaml file defining appropriate resource quotas (you can call it quota-4cpu.yaml):
apiVersion: v1
kind: ResourceQuota
metadata:
name: 4cpu
spec:
hard:
requests.cpu: "4"
requests.memory: 4Gi
limits.cpu: "8"
limits.memory: 8Gi
Apply these quota to your namespace:
kubectl apply -f quota-4cpu.yaml --namespace example
From now on, every container deployed in this namespace must have a memory request, memory limit, cpu request, and cpu limit. And (obviously), the total of each cannot exceed the values defined by the resource quota.
But what is the difference between request and limit quota?
Normally, Kubernetes is meant to use an external identity provider, that is used by Kubernetes to authenticate normal users. For this tutorial, we will use a service account to grant a user access to the namespace.
Create the service account example
inside the namespace example
:
kubectl --namespace example create serviceaccount example
Create a role with sufficient rights to manage the namespace (e.g. role-deployer.yaml):
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: deployer
rules:
- apiGroups: ["", "extensions", "apps"]
resources: ["services", "deployments", "replicasets", "pods", "configmaps", "persistentvolumeclaims", "pods/exec"]
verbs: ["*"]
The pods/exec
resource is not required, it allows a user with the deployer
role to execute a command inside a running pod. This is usually useful for debugging...
kubectl apply -f role-deployer.yaml --namespace example
Assign the role to the service account (role-binding-example.yaml):
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: example-deployer
subjects:
- kind: User
name: example
apiGroup: ""
roleRef:
kind: Role
name: deployer
apiGroup: ""
kubectl apply -f role-binding-example.yaml --namespace example
Now we can extract the token of the service account:
kubectl describe secrets --namespace example
"$(kubectl describe serviceaccount example
--namespace example | grep -i Tokens |
awk '{print $2}')" | grep token: | awk '{print $2}'
This will show a long token that we must copy and use later...
And we will also need the authority certificate of the server:
kubectl config view --minify --raw
--output 'jsonpath={..cluster.certificate-authority-data}' |
base64 -d | openssl x509 -text -out k8s.crt
This will create a file called k8s.crt containing the authority certificate of the server, that we will also need.
Now, on a client computer that has kubectl installed, we can import the token and server certificate (k8s.crt), so we can access the example
namespace of the cluster.
Create the cluster entry in the config:
kubectl config set-cluster mycluster --server https://my.cluster.com:6443
--certificate-authority=k8s.crt
Create the user entry in the config:
kubectl config set-credentials mycluster-example --token=eyJhbGciOiJSU...
Finally, we can create a context that will group the cluster, the appropriate user, and namespace:
kubectl config set-context mycluster-example --cluster=mycluster
--user=mycluster-example --namespace=example
We can test with:
kubectl config use-context mycluster-example
kubectl version
kubectl auth can-i create pods
kubectl auth can-i create sa
This blog post is licensed under CC BY-SA 4.0