How to set service-node-port-range and then be able to deploy services using the new port range?

Question

I have a requirement that a test server should use the port range 20000 - 22767

I edited the kubeadm-config with the command

kubectl edit cm kubeadm-config -n kube-system

When I look at the result I see that the change seems to have been stored:

The command $ kubeadm config view gives me

apiServer:
  extraArgs:
    authorization-mode: Node,RBAC
    service-node-port-range: 20000-22767
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta1
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: ""
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.13.3
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12
scheduler: {}

But when I later try to install something within hte new port range I get the error

helm upgrade --install --kubeconfig /external-storage/workspace/potapi-orchestration/clusters/at/admin.conf potapi-services charts/potapi-services -f charts/potapi-services/values.at.yaml
Error: UPGRADE FAILED: Service "potapi-services" is invalid: spec.ports[0].nodePort: Invalid value: 21011: provided port is not in the valid range. The range of valid ports is 30000-32767

I have fiddled with the suggestions here but with no luck: https://github.com/kubernetes/kubeadm/issues/122

Alicyclic

Did your apiserver restarted after the change? Can you see edited range in /etc/kubernetes/manifests/kube-apiserver.yaml – Aldredge

It did not restart. I am pretty sure that it wasn't restarted because I had left a backup file in /etc/kubernetes/manifests with the original values. That was a mistake. Removing the backup file resulted in a restart and the port range was updated. – Alicyclic

Answer 1

It is possible to update the service-node-port-rangefrom it's default values.

I updated the file /etc/kubernetes/manifests/kube-apiserver.yaml with --service-node-port-range=20000-22767.

The apiserver was restarted and the port range was updated.

I wrote a blog post about it.

Alicyclic

Your post is great but I use K0s distro and I have no kube-apiserver.yaml where to set the port range. Any suggestion on how to resolve the lack of this file – Lesley

Answer 2

I too couldn't find any docs on kubeadm configuration. The following works for me:

# your current k8s version
KUBE_VERSION=v1.14.1
KUBEADM_CONFIG=kubeadm-config.yaml

# save current kubeadm config
kubeadm config view > $KUBEADM_CONFIG

# add desired port range
vi kubeadm-config.yaml

  apiServer:
    extraArgs:
      authorization-mode: Node,RBAC
+       service-node-port-range: 20000-22767

# save kubeadm config into a configmap
kubeadm config upload from-file --config $KUBEADM_CONFIG

# check changes
sudo kubeadm upgrade diff --config $KUBEADM_CONFIG

# apply changes
sudo kubeadm upgrade apply $KUBE_VERSION --config $KUBEADM_CONFIG

Heraclitean