Kubernetes services and Azure load balancers

Azure Kubernetes Service (AKS) makes use of many core Azure resources to provide the necessary functionality. In this post, we'll take a look at how Kubernetes LoadBalancer services and Ingress are mapped to an Azure Load Balancer.

Cluster creation

During the AKS cluster creation process, a public load balancer is created in the infrastructure resource group of the cluster.

You can see the load balancer and its public IP address in the resources list of the infrastructure resource group.

If we look at the IP address details, we can see it is being used to provide outbound connectivity from the cluster.

Kubernetes LoadBalancer services

Now let's create a simple Kubernetes LoadBalancer service.

apiVersion: v1
kind: Service
metadata:
  name: public-svc
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: public-app

The service gets a new public IP address.

And the new address is associated with the existing load balancer.

A look at the rule associated with this address tells us that it manages the inbound traffic to the service.

Ingress controllers

Now let's see how an Ingress controller is associated with a load balancer.

I created a separate public IP address (20.81.68.54) in the infrastructure resource group to be used for the Ingress controller.

Let's deploy a NginX Ingress controller using Helm.

helm install nginx-ingress ingress-nginx/ingress-nginx \
  --create-namespace \
  --namespace ingress-ns \
  --set controller.replicaCount=2 \
  --set controller.ingressClass=nginx \
  --set controller.nodeSelector."beta\.kubernetes\.io/os"=linux \
  --set defaultBackend.nodeSelector."beta\.kubernetes\.io/os"=linux \
  --set controller.admissionWebhooks.patch.nodeSelector."beta\.kubernetes\.io/os"=linux \
  --set controller.service.loadBalancerIP="20.81.68.54"

We can verify the public IP address has been associated with the Ingress controller.

Now if we check the load balancer again, we can see it manages the traffic for the Ingress IP address too.

The rules associated with this IP address show that it manages the inbound http and https traffic to the ingress controller.

Using internal load balancer

So far, we have been making services available publicly. To restrict access to services to the same virtual network as the AKS cluster, we can make use of an internal load balancer.

Let's create a new Ingress controller which will use an internal load balancer.

helm install nginx-ingress ingress-nginx/ingress-nginx \
  --create-namespace \
  --namespace ingress-ns-internal \
  --set controller.replicaCount=2 \
  --set controller.ingressClass=nginx-internal \
  --set controller.nodeSelector."beta\.kubernetes\.io/os"=linux \
  --set defaultBackend.nodeSelector."beta\.kubernetes\.io/os"=linux \
  --set controller.admissionWebhooks.patch.nodeSelector."beta\.kubernetes\.io/os"=linux \
  --set-string controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-internal"="true"

The annotation service.beta.kubernetes.io/azure-load-balancer-internal"="true" will result in creation of an internal load balancer in the infrastructure resource group of the cluster.

The internal load balancer gets a dynamic IP from the cluster subnet.

Conclusion

We get one public Azure load balancer for an AKS cluster and all the traffic on the public IP addresses associated with the Kubernetes LoadBalancer services and Ingress controllers is managed by it.

We can additionally create an internal load balancer to restrict traffic to the same virtual network as the AKS cluster.

Technobabble