Customize Traefik in K3S

📅2023-07-13🧐144

By run

curl -sfL https://get.k3s.io | INSTALL_K3S_CHANNEL=latest sh -s - --write-kubeconfig-mode 644

We can easily install K3S and enable Traefik v2.x.

Caveat:

K3S uses IngressRoute which is a Custom Resource Definition (CRD) instead of ingress.

The IngressRoute takes over all the traffic. It exposes the service to external directly, kind of like bypass the ingress. Simplely, it's not a reverse proxy but more like port or traffic forwarding.

Know more:

https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/#kind-ingressroute

https://github.com/traefik/traefik/blob/60bc47d00e9ac7584a4bd437f53fdef347915dea/pkg/provider/kubernetes/crd/kubernetes.go

It uses the Kubernetes Gateway API under the hood:

https://gateway-api.sigs.k8s.io/

So you have two choices here:

  1. Use ingressroute
  2. Use ingress
Use ingressroute

Maybe you want to customrize your ingressroute. You just need to copy the traefik config file.

source: /var/lib/rancher/k3s/server/manifests/traefik.yaml

custom: /var/lib/rancher/k3s/server/manifests/traefik-config.yaml

For the configuration please check the helm-chart:

https://github.com/traefik/traefik-helm-chart/tree/master/traefik

If you use a customrized config, don't forget to update it when you upgrade K3S or Traefik. It could use different version of CRD like this:

---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: traefik-crd
  namespace: kube-system
spec:
  chart: https://%{KUBERNETES_API}%/static/charts/traefik-crd-10.14.100.tgz

Previously I created one for the traefik v2 since the helm-chart always disappoint me, but now they fixed tons of issues finally. So I back on the helm-chart.

Use Ingress

You just need to disable traefik from installtion:

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--no-deploy=traefik" sh -s - --write-kubeconfig-mode 644 --disable=traefik

But we still can deploy traefik as our ingress controller by install it manually:

https://doc.traefik.io/traefik/getting-started/install-traefik/

In the last

We can simplely replace Ingress by Ingressroute, but must be clearly understand what happend after that. I use Ingressroute over two years until now. But I also use Ingress in different project. It's depend on the specific needs.