Today, we deploy more and more applications and micro-services in Kubernetes. Managing all the entry points of these applications can be problematic. To facilitate this management, there are ingress controllers, Traefik is one of them.
Disclaimer : This post is a translated version of the blog post I made for my company, you can find the french version here, on WeScale blog.
Traefik 2 - One ingress controller to control them all
Pourquoi utiliser Traefik ?
- To use reverse proxy simply
- To centralize all certificate management
- To do TCP loadbalancing (only available from Traefik 2)
- For self-discovery from Kubernetes
- To load new services dynamically, without service interruption
- To do multibackend (Mesos, Kubernetes, Docker etc...)
Image from the official website
So I propose below a little "hands on" Traefik 2, where we will deploy Traefik in a daemonset and put a "whoami" service behind it. We will also set up a certificate via Let's Encrypt.
The goal, as you will have understood, is to manipulate and see the possibilities offered by Traefik.
We will first deploy CRD Kubernetes (Custom Resources Definition), the goal of these is to provide objects that Traefik will use to facilitate its deployment.
apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: ingressroutes.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: IngressRoute plural: ingressroutes singular: ingressroute scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: middlewares.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: Middleware plural: middlewares singular: middleware scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: ingressroutetcps.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: IngressRouteTCP plural: ingressroutetcps singular: ingressroutetcp scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: tlsoptions.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: TLSOption plural: tlsoptions singular: tlsoption scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: traefikservices.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: TraefikService plural: traefikservices singular: traefikservice scope: Namespaced
Deployment of Traefik
Now that we have deployed our prerequisites, we will be able to set up an RBAC for Traefik, which we will deploy as Daemonset.
The RBAC will allow Traefik to do self-discovery on Kubernetes, it will use read-only rights on several objects.
kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller rules: - apiGroups: - "" resources: - services - endpoints - secrets verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses/status verbs: - update - apiGroups: - traefik.containo.us resources: - middlewares - ingressroutes - traefikservices - ingressroutetcps - tlsoptions verbs: - get - list - watch --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: traefik-ingress-controller subjects: - kind: ServiceAccount name: traefik-ingress-controller namespace: default --- apiVersion: v1 kind: ServiceAccount metadata: name: traefik-ingress-controller namespace: default
Once the ServiceAccount is created and configured to be used, we can deploy our Traefik agent as a DaemonSet, in addition to a Traefik service to carry the dashboard.
About deployment as a DaemonSet
In this hands on, I chose to deploy Traefik as a DaemonSet, so it will be deployed on all the nodes of my Kubernetes cluster. This is the deployment mode that is often chosen for ingress controllers in order to have a strong redundancy of this critical component, to be as close as possible to the backends and thus to optimize flows and resources.
apiVersion: apps/v1 kind: DaemonSet metadata: name: traefik-ingress-controller labels: k8s-app: traefik-ingress-lb kubernetes.io/cluster-service: "true" spec: selector: matchLabels: k8s-app: traefik-ingress-lb template: metadata: labels: k8s-app: traefik-ingress-lb name: traefik-ingress-lb spec: hostNetwork: true dnsPolicy: ClusterFirstWithHostNet serviceAccountName: traefik-ingress-controller terminationGracePeriodSeconds: 60 tolerations: - key: node-role.kubernetes.io/master effect: NoSchedule containers: - image: traefik:v2.1.1 name: traefik-ingress-lb imagePullPolicy: Always volumeMounts: - mountPath: "/cert/" name: cert resources: requests: cpu: 100m memory: 20Mi args: - --providers.kubernetescrd - --entrypoints.web.address=:80 - --entrypoints.websecure.address=:443 - --api.insecure - --firstname.lastname@example.org - --certificatesresolvers.le.acme.storage=/cert/acme.json - --certificatesResolvers.le.acme.httpChallenge.entryPoint=web volumes: - name: cert hostPath: path: /home/kube/traefik/certs/ type: Directory --- apiVersion: v1 kind: Service metadata: name: traefik-web-ui spec: selector: k8s-app: traefik-ingress-lb app: traefik-ingress-lb ports: - port: 8080 targetPort: 8080 name: api - port: 443 targetPort: 443 name: websecure - port: 80 targetPort: 80 name: web
As can be seen in the block above, Traefik and its services are therefore deployed to expose the http and https ports, as well as the dashboard.
Warning, in this configuration, the dashboard is deployed without authentication!
Moreover, I create a local directory in which I will store my certificates, because Let's Encrypt limits the number of weekly requests for the same certificate. Note that Traefik, in its community version, does not manage its certificates in "cluster" mode, this option is specific to the Enterprise version. You will have to create this directory before applying the deployment.
|Traefik 1.7||TraefikEE 1||Traefik 2.1||TraefikEE 2|
|HTTP Load Balancing||Yes||Yes||Yes||Yes|
|TCP Load Balancing||No||No||Yes||Yes|
|Let’s Encrypt certificates||Yes, but limited number of queries||Yes, unlimited number of queries||Yes, but limited number of queries||Yes, unlimited number of queries|
|Distributed configuration||No||Yes, centralized on control plane nodes.||No||Yes, centralized on control plane nodes.|
In "prod ready" configuration, you can of course deport this storage to a shared storage rather than a local on the Kube node, which is to be avoided in most cases. Nevertheless, in the case of "ACME" certificate management, this can be a concern, as the node requesting the certificate is not necessarily the node receiving it.
At this point, you should be able to access Traefik's dashboard on port 8080.
Source : Official documentation
Deploying a backend: Whoami
To complete our deployment, I will add a backend, based on the containous/whoami image.
So we will create a deployment that will set up our pod.
apiVersion: apps/v1 kind: Deployment metadata: name: whoami labels: app: whoami spec: replicas: 3 selector: matchLabels: app: whoami template: metadata: labels: app: whoami spec: containers: - name: whoami image: whoami:v1.4.0 imagePullPolicy: Always ports: - containerPort: 80
Finally, we are going to add a service, which will allow us to expose our containers within our Kubernetes cluster, then an ingressroute, an object that we created via the CRDs earlier, which allows us to carry all the necessary configuration to Traefik.
kind: Service apiVersion: v1 metadata: name: whoami spec: ports: - port: 80 name: web protocol: TCP selector: app: whoami --- apiVersion: traefik.containo.us/v1alpha1 kind: IngressRoute metadata: name: whoami-web spec: entryPoints: - web routes: - kind: Rule match: Host(`mydomain.com`) && PathPrefix(`/`) services: - name: whoami port: 80 --- apiVersion: traefik.containo.us/v1alpha1 kind: IngressRoute metadata: name: whoami-web-tls spec: entryPoints: - websecure routes: - kind: Rule match: Host(`mydomain.com`) && PathPrefix(`/`) services: - name: whoami port: 80 tls: certResolver: le
Concerning the ingressroute, we can observe:
- That there are two definitions, one in http and one in https...
- That the https part is configured to automatically request its certificate from Let's Encrypt ("le" in certResolver) in case the application does not already have it.
- That the routing rules are carried by the ingressroute.
From this point, you should now be able to access your pods via port 80 or 443 for the secure part.
To go further...
We saw how to basically deploy Traefik with a backend in Kubernetes. However, we have only touched on the emerged part of Traefik.
To go further we could:
- Configure more finely the TLS part (ciphers / curves / TLS version)
- Set up middlewares to do, for example, ray tracing
- Implement TCP load balancing
- Doing multi backend
- Outsource logs
- Traefik Plugger to Prometheus to outsource its metrics
These points will probably be discussed later on this blog, so stay tuned!