Nuxtstop

For all things nuxt.js

Install Traefik Ingress Controller in Kubernetes

#traefik #kubernetes #k3s #ingress
10 0

Credits: Logo by traefik.io

Provide load balancing, SSL termination and name-based virtual hosting on a Kubernetes (k3s) cluster using Traefik ingress controller.

 

Note: The domain referenced in this post is MY_DOMAIN, please change accordingly. If you interested in a local-only work mode, you don’t have to pay for a new domain, just decide on a name and use it. For example, if your desired domain is homelab.com, replace MY_DOMAIN with homelab.

 

Prerequisites

 

traefik-overview.png

 

What is an Ingress Controller?
It is an API object that manages external access to a deployed service in a Kubernetes cluster, typically via HTTP/S. It provides load balancing, SSL termination and name-based virtual hosting.

Why would I want to use an Ingress Controller?
These are some of the immediate benefits using an ingress controller:

  • Simplify the way internal services interact with each other and re-route when required by changing only the Traefik routing rule.

     

    Note: Instead of relying on an IP address that might change, simply define a hosted name e.g. serviceA.domain.com to address serviceA and use that on other internal services and/or from outside the cluster when in need to call serviceA.
    Isn't serviceA.domain.com easier to remember than 192.169.200.xxx ?

  • Allows HTTPS traffic from outside the Kubernetes cluster while terminating encryption and allowing HTTP traffic between services within the cluster

  • Load balance traffic between services hosted outside the Kubernetes cluster

Note: Traefik is using custom cluster resource definitions. These CRDs gets installed via Traefik 2 Helm Chart. For CRDs schema click here.

 

K8s Controller

Traefik is a Kubernetes controller that manage the access to cluster services by supporting the Ingress specification. It receives requests on behalf of your system and finds out which components are responsible for handling them.

Prepare

We want Kubernetes to create the Traefik pod on the master node. In order to do that, we'll have to label that node and use nodeSelector attribute when installing Traefik Helm chart.

  1. Get all nodes names and labels

    kubectl get nodes --show-labels

     

  2. Label kmaster node with node-type=master

    kubectl label nodes kmaster node-type=master

     

    Note: To remove a label, use the same command with dash after the label name e.g. kubectl label nodes kmaster node-type-."

  3. Verify that label had been created successfully

    kubectl get nodes --show-labels | grep node-type

 

Install

  1. Create a traefik namespace

    kubectl create namespace traefik

     

  2. Add the containous Helm repository hosting the Traefik charts metadata

    helm repo add traefik https://containous.github.io/traefik-helm-chart

     

  3. Update local Helm chart repository cache

    helm repo update

     

  4. Search for latest traefik/traefik official Helm chart version

    helm search repo traefik

# NAME              CHART VERSION   APP VERSION
# traefik/traefik   9.1.1           2.2.8

     

  5. Install the Traefik Helm chart using the version from previous step

    helm upgrade --install traefik \
    --namespace traefik \
    --set dashboard.enabled=true \
    --set rbac.enabled=true \
    --set nodeSelector.node-type=master \
    --set="additionalArguments={--api.dashboard=true,--log.level=INFO,--providers.kubernetesingress.ingressclass=traefik-internal,--serversTransport.insecureSkipVerify=true}" \
    traefik/traefik \
    --version 9.1.1

     

  6. Verify installation

    # Make sure all traefik deployed pods are running
kubectl get pods --namespace traefik

# Make sure custom resources *.traefik.containo.us were created successfully 
kubectl get crd | grep traefik

     

Uninstall

  1. Remove traefik from the cluster

    helm uninstall traefik --namespace traefik

     

  2. Clear the traefik namespace

    kubectl delete namespaces traefik

     

Traefik Dashboard

Dashboard is installed but disabled by default for security reasons. We will want to avoid using the kubectl proxy-forward option and allow the dashboard via HTTPS with proper TLS/Cert.

Certificate

We will create a certificate using cert-manager to allow accessing the Traefik dashboard via the hosted name traefik.MY_DOMAIN.com within our home network. Create a self signed certificate as described in here under traefik namespace.

Verify that a TLS secret had been created for the certificate:

kubectl get secret MY_DOMAIN-com-cert-secret --namespace traefik
Enter fullscreen mode Exit fullscreen mode

 

Authentication

We will create a user / password basic authentication, please read here if you wish to use a different method.

  1. Generate a flat-file that stores a username and password for basic authentication

    bash <<'EOF'

# Change these credentials to your own
export TRAEFIK_UI_USER=admin
export TRAEFIK_UI_PASS=dashboard
export DESTINATION_FOLDER=${HOME}/temp/traefik-ui-creds

# Backup credentials to local files (in case you'll forget them later on)
mkdir -p ${DESTINATION_FOLDER}
echo $TRAEFIK_UI_USER >> ${DESTINATION_FOLDER}/traefik-ui-user.txt
echo $TRAEFIK_UI_PASS >> ${DESTINATION_FOLDER}/traefik-ui-pass.txt

htpasswd -Bbn ${TRAEFIK_UI_USER} ${TRAEFIK_UI_PASS} \
    > ${DESTINATION_FOLDER}/htpasswd

unset TRAEFIK_UI_USER TRAEFIK_UI_PASS DESTINATION_FOLDER

EOF

     

  2. Create a Kubernetes secret based on the basic authentication file

    kubectl create secret generic traefik-dashboard-auth-secret \
  --from-file=$HOME/temp/traefik-ui-creds/htpasswd \
  --namespace traefik

     

Ingress

  1. Create an IngressRoute for accessing the dashboard, make sure to replace MY_DOMAIN with your domain name

    cat <<EOF | kubectl apply -f -
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-dashboard
  namespace: traefik
spec:
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host(\`traefik.MY_DOMAIN.com\`) && (PathPrefix(\`/api\`) || PathPrefix(\`/dashboard\`))
      services:
        - name: api@internal
          kind: TraefikService
      middlewares:
        - name: traefik-dashboard-auth # Referencing the BasicAuth middleware
          namespace: traefik
  tls:
    secretName: MY_DOMAIN-com-cert-secret
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: traefik-dashboard-auth
  namespace: traefik
spec:
  basicAuth:
    secret: traefik-dashboard-auth-secret
EOF

     

  2. Check that resources were created successfully

    # IngressRoute
kubectl describe ingressroute traefik-dashboard --namespace traefik

# Middleware
kubectl describe middleware traefik-dashboard-auth --namespace traefik

     

  3. Check there are no error logs

    kubectl logs -f $(kubectl get pods --namespace traefik | grep "^traefik" | awk '{print $1}') --namespace traefik

     

  4. Set a hosted name traefik.MY_DOMAIN.com on a client machine (laptop/desktop)

    # Edit the file using `sudo /etc/hosts`
# Append manualy to the existing k3s hosted name 
111.222.333.444 kmaster, traefik.MY_DOMAIN.com

# Alternatively, add a new hosted name entry with a one-liner
echo -e "111.222.333.444\ttraefik.MY_DOMAIN.com" | sudo tee -a /etc/hosts

     

    Note: Replace 111.222.333.444 with the k3s master node IP address and MY_DOMAIN with your domain name.

  5. Open browser at https://traefik.MY_DOMAIN.com/dashboard/

    Note: If you encounter an untrusted certificate warning, please follow the cert-manager post Trust section for instructions on how to trust the traefik certificate.

    Dashboard:
    err-dashboard-img

     

Demo Application

We will deploy an example whoami application that returns basic information about the client issuing the request. It'll allow us to check that Traefik was deployed successfully by addressing our demo application using whoami.MY_DOMAIN.com.

Prepare

Before deploying our application to the Kubernetes cluster, we will have to define which node(s) are eligible of assigning pods for it.

We will use the node-type=master label created at the Prepare step of the Traefik installation. To assign the whoami pod(s) to the kmaster node, we will use the nodeSelector attribute on a Kubernetes resource (Deployment / Pod) with that label.

Install

  1. Create a playground namespace

    kubectl create namespace playground

     

  2. Create Kubernetes Deployment and Service with x2 running instances of the whoami application

    cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
  namespace: playground
  labels:
    app: containous
    name: whoami
spec:
  replicas: 2
  selector:
    matchLabels:
      app: containous
      task: whoami
  template:
    metadata:
      labels:
        app: containous
        task: whoami
    spec:
      nodeSelector:
        node-type: "master"
      containers:
        - name: containouswhoami
          image: containous/whoami
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: playground
spec:
  ports:
    - name: http
      port: 80
  selector:
    app: containous
    task: whoami
EOF

     

    Note: Please refer to the official docs to read more about Kubernetes Deployment or Service.

  3. Verify both Deployment and Service were created successfully

    # Deployment should indicate 2/2 running pods 
kubectl get deployments -n playground

# Service should indicate a cluster IP on port 80 without an external IP
kubectl get services -n playground

# Check that there are x2 running pods on kmaster node
kubectl describe pods whoami -n playground | grep "Status:"

     

Uninstall

  1. Follow the installation instructions for Deployment, Service, IngressRoute, Middleware and execute both scripts while replacing the kubectl action:

    # Instead of using apply
cat <<EOF | kubectl apply -f -  
...
EOF

# Replace with delete
cat <<EOF | kubectl delete -f - 
...
EOF

     

  2. Clear the playground namespace

    kubectl delete namespaces playground

     

Certificate

We will have to create a certificate using cert-manager to allow accessing the whoami application using the hosted name whoami.MY_DOMAIN.com within our home network. Create a self signed certificate as described in here under playground namespace.

Verify that a TLS secret had been created for the certificate:

kubectl get secret MY_DOMAIN-com-cert-secret --namespace playground
Enter fullscreen mode Exit fullscreen mode

 

Ingress

  1. Create ingress entry points to allow secure communication using HTTPS and allowing HTTP with HTTPS redirect middleware (make sure to replace MY_DOMAIN with your domain name)

    cat <<EOF | kubectl apply -f -
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: whoami-https
  namespace: playground
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(\`whoami.MY_DOMAIN.com\`)
      kind: Rule
      priority: 10
      services:
        - name: whoami
          port: 80
  tls:
    secretName: MY_DOMAIN-com-cert-secret
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: whoami
  namespace: playground
spec:
  entryPoints:
    - web
  routes:
    - match: Host(\`whoami.MY_DOMAIN.com\`)
      kind: Rule
      priority: 10
      services:
        # This IngressRoute will be never called due to the redirect middleware
        - name: whoami
          port: 80
      middlewares:
        - name: https-redirect
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: https-redirect
  namespace: playground
spec:
  redirectScheme:
    scheme: https
    permanent: true
EOF

     

  2. Verify success status of HTTP routers and middlewares using the traefik dashboard

    HTTP Routers:
    traefik-http-routers

    HTTP Middlewares:
    traefik-http-middlewares

     

  3. Set a hosted name whoami.MY_DOMAIN.com on a client machine (laptop/desktop)

    # Edit the file using `sudo /etc/hosts`
# Append manualy to the existing k3s hosted name 
111.222.333.444 kmaster, traefik.MY_DOMAIN.com, whoami.MY_DOMAIN.com

# Alternatively, add a new hosted name entry with a one-liner
echo -e "111.222.333.444\twhoami.MY_DOMAIN.com" | sudo tee -a /etc/hosts

     

    Note: Replace 111.222.333.444 with the k3s master node IP address and MY_DOMAIN with your domain name.

  4. Open a web-browser at https://whoami.MY_DOMAIN.com and check that you can get a response from the service(s)

    Note: Open http://whoami.MY_DOMAIN.com to check the HTTP -> HTTPS redirect.

 

Summary

Congratulations on completing a major part of deploying an ingress controller on your Kubernetes cluster ! 💪

What now? Now that you are familiar with how to use Traefik as an ingress controller, you can simplify the inter-services communication within the cluster and expose applications APIs / Web-UI and such via HTTPS in no time.

Please leave your comment, suggestion or any other input you think is relevant to this post in the discussion below.

Like this post?
You can find more by:

Checking out my blog: https://blog.zachinachshon.com
Following me on twitter: @zachinachshon

Thanks for reading! ❤️

Add comment