Need help on volume mount issue with kubernetes

Question

I have RBAC enabled kubernetes cluster created using kops version 1.8.0-beta.1, I am trying to run a nginx pod which should attach pre-created EBS volume and pod should start. But getting issue as not authorized even though i am a admin user. Any help would be highly appreciated.

kubectl version Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.3", GitCommit:"f0efb3cb883751c5ffdbe6d515f3cb4fbe7b7acd", GitTreeState:"clean", BuildDate:"2017-11-09T07:27:47Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.3", GitCommit:"f0efb3cb883751c5ffdbe6d515f3cb4fbe7b7acd", GitTreeState:"clean", BuildDate:"2017-11-08T18:27:48Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

namespace:default

cat test-ebs.yml

apiVersion: v1
kind: Pod
metadata:
  name: test-ebs
spec:
  containers:
  - image: nginx
    name: nginx
    volumeMounts:
    - mountPath: /test-ebs
      name: test-volume
  volumes:
  - name: test-volume
    awsElasticBlockStore:
      volumeID: <vol-IDhere>
      fsType: ext4

I am getting the below error:

Warning  FailedMount            8m               attachdetach                                        AttachVolume.Attach failed for volume "test-volume" : Error attaching EBS volume "<vol-ID>" to instance "<i-instanceID>": "UnauthorizedOperation: You are not authorized to perform this operation

Livvi

Could you please include relevant kubelet logs (journalctl -xeu kubelet executed on the node in question)? – Fergus

Please update the question with the complete authorization error and the exact kops version in which you faced the issue. – Illegal

Answer 1

In kops 1.8.0-beta.1, master node requires you to tag the AWS volume with:

KubernetesCluster: <clustername-here>

If you have created the k8s cluster using kops like so:

kops create cluster --name=k8s.yourdomain.com [other-args-here]

your tag on the EBS volume needs to be

KubernetesCluster: k8s.yourdomain.com

And the policy on master would contain a block which would contain:

{
  "Sid": "kopsK8sEC2MasterPermsTaggedResources",
  "Effect": "Allow",
  "Action": [
    "ec2:AttachVolume",
    "ec2:AuthorizeSecurityGroupIngress",
    "ec2:DeleteRoute",
    "ec2:DeleteSecurityGroup",
    "ec2:DeleteVolume",
    "ec2:DetachVolume",
    "ec2:RevokeSecurityGroupIngress"
  ],
  "Resource": [
    "*"
  ],
  "Condition": {
    "StringEquals": {
      "ec2:ResourceTag/KubernetesCluster": "k8s.yourdomain.com"
    }
  }
}

The condition indicates that master-policy has privilege to only attach volumes which contain the right tag.

Illegal

Answer 2

Issue is because of kops1.8 version. Rolled back to kops version v1.7.1. its working now.

Livvi