Kubernetes: how to enable API Server Bearer Token Auth?

Question

I've been trying to enabled token auth for HTTP REST API Server access from a remote client.

I installed my CoreOS/K8S cluster controller using this script: https://github.com/coreos/coreos-kubernetes/blob/master/multi-node/generic/controller-install.sh

My cluster works fine. This is a TLS installation so I need to configure any kubectl clients with the client certs to access the cluster.

I then tried to enable token auth via running:

 echo `dd if=/dev/urandom bs=128 count=1 2>/dev/null | base64 | tr -d "=+/" | dd bs=32 count=1 2>/dev/null`

this gives me a token. I then added the token to a token file on my controller containing a token and default user:

$> cat /etc/kubernetes/token

3XQ8W6IAourkXOLH2yfpbGFXftbH0vn,default,default

I then modified the /etc/kubernetes/manifests/kube-apiserver.yaml to add in:

 - --token-auth-file=/etc/kubernetes/token

to the startup param list

I then reboot (not sure the best way to restart API Server by itself??)

At this point, kubectl from a remote server quits working(won't connect). I then look at docker ps on the controller and see the api server. I run docker logs container_id and get no output. If I look at other docker containers I see output like:

    E0327 20:05:46.657679       1 reflector.go:188] 
    pkg/proxy/config/api.go:33: Failed to list *api.Endpoints: 
    Get http://127.0.0.1:8080/api/v1/endpoints?resourceVersion=0: 
dial tcp 127.0.0.1:8080: getsockopt: connection refused

So it appears that my api-server.yaml config it preventing the API Server from starting properly....

Any suggestions on the proper way to configure API Server for bearer token REST auth?

It is possible to have both TLS configuration and Bearer Token Auth configured, right?

Thanks!

Parquet

Answer 1

I think your kube-apiserver dies because it's can't find the /etc/kubernetes/token. That's because on your deployment the apiserver is a static pod therefore running in a container which in turn means it has a different root filesystem than that of the host.

Look into /etc/kubernetes/manifests/kube-apiserver.yaml and add a volume and a volumeMount like this (I have omitted the lines that do not need changing and don't help in locating the correct section):

kind: Pod
metadata:
  name: kube-apiserver
spec:
  containers:
  - name: kube-apiserver
    command:
    - ...
    - --token-auth-file=/etc/kubernetes/token
    volumeMounts:
    - mountPath: /etc/kubernetes/token
      name: token-kubernetes
      readOnly: true
  volumes:
  - hostPath:
      path: /etc/kubernetes/token
    name: token-kubernetes

One more note: the file you quoted as token should not end in . (dot) - maybe that was only a copy-paste mistake but check it anyway. The format is documented under static token file:

token,user,uid,"group1,group2,group3"

If your problem perists execute the command below and post the output:

journalctl -u kubelet | grep kube-apiserver

Malacostracan

Thanks so much! Makes complete sense that api server wouldn't have access to the file. and yes the '.' at the end was unintentional, removed from the question – Parquet