Today, we deploy more and more applications and micro-services in Kubernetes. Managing all the entry points of these applications can be problematic. To facilitate this management, there are ingress controllers, Traefik is one of them.

Disclaimer : This post is a translated version of the blog post I made for my company, you can find the french version here, on WeScale blog.

Traefik 2 - One ingress controller to control them all

Pourquoi utiliser Traefik ?

  • To use reverse proxy simply
  • To centralize all certificate management
  • To do TCP loadbalancing (only available from Traefik 2)
  • For self-discovery from Kubernetes
  • To load new services dynamically, without service interruption
  • To do multibackend (Mesos, Kubernetes, Docker etc...)

Image from the official website

So I propose below a little "hands on" Traefik 2, where we will deploy Traefik in a daemonset and put a "whoami" service behind it. We will also set up a certificate via Let's Encrypt.

The goal, as you will have understood, is to manipulate and see the possibilities offered by Traefik.

Prerequisites

We will first deploy CRD Kubernetes (Custom Resources Definition), the goal of these is to provide objects that Traefik will use to facilitate its deployment.

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutes.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRoute
    plural: ingressroutes
    singular: ingressroute
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: middlewares.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Middleware
    plural: middlewares
    singular: middleware
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutetcps.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteTCP
    plural: ingressroutetcps
    singular: ingressroutetcp
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsoptions.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSOption
    plural: tlsoptions
    singular: tlsoption
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: traefikservices.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TraefikService
    plural: traefikservices
    singular: traefikservice
  scope: Namespaced

Deployment of Traefik

Now that we have deployed our prerequisites, we will be able to set up an RBAC for Traefik, which we will deploy as Daemonset.

The RBAC will allow Traefik to do self-discovery on Kubernetes, it will use read-only rights on several objects.

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller

rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - traefik.containo.us
    resources:
      - middlewares
      - ingressroutes
      - traefikservices
      - ingressroutetcps
      - tlsoptions
    verbs:
      - get
      - list
      - watch

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller

roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
  - kind: ServiceAccount
    name: traefik-ingress-controller
    namespace: default
    
---
apiVersion: v1
kind: ServiceAccount
metadata:
 name: traefik-ingress-controller
 namespace: default

Once the ServiceAccount is created and configured to be used, we can deploy our Traefik agent as a DaemonSet, in addition to a Traefik service to carry the dashboard.

About deployment as a DaemonSet

In this hands on, I chose to deploy Traefik as a DaemonSet, so it will be deployed on all the nodes of my Kubernetes cluster. This is the deployment mode that is often chosen for ingress controllers in order to have a strong redundancy of this critical component, to be as close as possible to the backends and thus to optimize flows and resources.

apiVersion: apps/v1
kind: DaemonSet
metadata:
 name: traefik-ingress-controller 
 labels:
   k8s-app: traefik-ingress-lb
   kubernetes.io/cluster-service: "true"
spec:
 selector:
   matchLabels:
      k8s-app: traefik-ingress-lb
 template:
   metadata:
     labels:
       k8s-app: traefik-ingress-lb
       name: traefik-ingress-lb
   spec:
     hostNetwork: true
     dnsPolicy: ClusterFirstWithHostNet
     serviceAccountName: traefik-ingress-controller
     terminationGracePeriodSeconds: 60
     tolerations:
     - key: node-role.kubernetes.io/master
       effect: NoSchedule
     containers:
     - image: traefik:v2.1.1
       name: traefik-ingress-lb
       imagePullPolicy: Always
       volumeMounts:
         - mountPath: "/cert/"
           name: cert
       resources:
         requests:
           cpu: 100m
           memory: 20Mi
       args:
       - --providers.kubernetescrd
       - --entrypoints.web.address=:80
       - --entrypoints.websecure.address=:443
       - --api.insecure
       - --certificatesresolvers.le.acme.email=myawesomemailadress@mymail.com
       - --certificatesresolvers.le.acme.storage=/cert/acme.json
       - --certificatesResolvers.le.acme.httpChallenge.entryPoint=web
     volumes:
     - name: cert
       hostPath:
         path: /home/kube/traefik/certs/
         type: Directory
---

apiVersion: v1
kind: Service
metadata:
 name: traefik-web-ui
spec:
 selector:
   k8s-app: traefik-ingress-lb
   app: traefik-ingress-lb
 ports:
 - port: 8080
   targetPort: 8080
   name: api
 - port: 443
   targetPort: 443
   name: websecure
 - port: 80
   targetPort: 80
   name: web

As can be seen in the block above, Traefik and its services are therefore deployed to expose the http and https ports, as well as the dashboard.

Warning, in this configuration, the dashboard is deployed without authentication!

Moreover, I create a local directory in which I will store my certificates, because Let's Encrypt limits the number of weekly requests for the same certificate. Note that Traefik, in its community version, does not manage its certificates in "cluster" mode, this option is specific to the Enterprise version. You will have to create this directory before applying the deployment.

Traefik 1.7 TraefikEE 1 Traefik 2.1 TraefikEE 2
HTTP Load Balancing Yes Yes Yes Yes
TCP Load Balancing No No Yes Yes
Let’s Encrypt certificates Yes, but limited number of queries Yes, unlimited number of queries Yes, but limited number of queries Yes, unlimited number of queries
Distributed configuration No Yes, centralized on control plane nodes. No Yes, centralized on control plane nodes.
LDAP Authentication No No No Yes

More informations about TraefikEE 1 here  and TraefikEE 2 here

In "prod ready" configuration, you can of course deport this storage to a shared storage rather than a local on the Kube node, which is to be avoided in most cases. Nevertheless, in the case of "ACME" certificate management, this can be a concern, as the node requesting the certificate is not necessarily the node receiving it.

At this point, you should be able to access Traefik's dashboard on port 8080.

Source : Official documentation

Deploying a backend: Whoami

To complete our deployment, I will add a backend, based on the containous/whoami image.

So we will create a deployment that will set up our pod.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
  labels:
    app: whoami
spec:
  replicas: 3
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami
        image: whoami:v1.4.0
        imagePullPolicy: Always
        ports:
        - containerPort: 80

Finally, we are going to add a service, which will allow us to expose our containers within our Kubernetes cluster, then an ingressroute, an object that we created via the CRDs earlier, which allows us to carry all the necessary configuration to Traefik.

kind: Service
apiVersion: v1
metadata:
  name: whoami
spec:
  ports:
  - port: 80
    name: web
    protocol: TCP
  selector:
    app: whoami
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: whoami-web
spec:
  entryPoints:
    - web
  routes:
  - kind: Rule
    match: Host(`mydomain.com`) && PathPrefix(`/`)
    services:
    - name: whoami
      port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: whoami-web-tls
spec:
  entryPoints:
    - websecure
  routes:
  - kind: Rule
    match: Host(`mydomain.com`) && PathPrefix(`/`)
    services:
    - name: whoami
      port: 80
  tls:
    certResolver: le

Concerning the ingressroute, we can observe:

  • That there are two definitions, one in http and one in https...
  • That the https part is configured to automatically request its certificate from Let's Encrypt ("le" in certResolver) in case the application does not already have it.
  • That the routing rules are carried by the ingressroute.

From this point, you should now be able to access your pods via port 80 or 443 for the secure part.

To go further...

We saw how to basically deploy Traefik with a backend in Kubernetes. However, we have only touched on the emerged part of Traefik.

To go further we could:

These points will probably be discussed later on this blog, so stay tuned!