Docker push to AWS ECR private repo failing with malformed JSON

Question

I am trying out AWS ECR and pushing a new tag to our private repossitory.

it goes like this:

export DOCKER_REGISTRY=0123123123123.dkr.ecr.us-east-1.amazonaws.com
export TAG=0.1
docker build -t vendor/app-name .
`aws ecr get-login --region us-east-1`" # generates docker login
docker tag vendor/app-name $DOCKER_REGISTRY/vendor/app-name:$TAG
docker push $DOCKER_REGISTRY/vendor/app-name:$TAG

Login works, the tag is created and I see it with docker images, but the push fails cryptically.

The push refers to a repository [0123123123123.dkr.ecr.us-east-1.amazonaws.com/vendor/app-name] (len: 2)
b1a1d76b9e52: Pushing [==================================================>]     32 B/32 B
Error parsing HTTP response: unexpected end of JSON input: ""

It very well might be a misconfiguration, but I can't figure out how to get more output out of it. The command has no debug level options, there are no other logs and I can't intercept network traffic since it seems encrypted.

Nape

Are there any special characters in "vendor" or "app-name"? (Presuming these aren't the real values you're using) I'd try a quick test with no hyphens, underscores, etc... – Juno

I chose those generics matching ours, vendor is our company name (only small letters), app-name has a dash between two words. I just tried vendor/appname and it's the same scenario – Nape

Also this happens both on Travis and locally – Nape

What about versions? ECR uses registry v2, which (I think) was introduced into docker in 1.5. Any chance the client is 1.4 or older? – Juno

Good idea, unfortunately: $ docker --version Docker version 1.8.1, build d12ea79 – Nape

Answer 1

Ran into the same issue. For me, ensuring that the IAM user I was pushing as had the ecr:BatchCheckLayerAvailability permission cleared this up.

I had originally intended to have a "push-only" policy and didn't realize this permission was required to push successfully.

Tarpaulin

on spot, thanks! The error message sucks so hard though – Nape

Answer 2

Minimal policy you need:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Action": "ecr:GetAuthorizationToken",
      "Resource": "*"
    },
    {
      "Sid": "",
      "Effect": "Allow",
      "Action": [
        "ecr:UploadLayerPart",
        "ecr:PutImage",
        "ecr:InitiateLayerUpload",
        "ecr:CompleteLayerUpload",
        "ecr:BatchCheckLayerAvailability"
      ],
      "Resource": "arn:aws:ecr:<your region>:<your account id>:repository/<your repository name>"
    }
  ]
}

Klayman

Answer 3

In addition to @Ethan's answer: I tried to find minimal set of permissions which are needed to push a docker image to AWS registry. As of today, the minimal set is:

    {
        "Sid": "PushToEcr",
        "Effect": "Allow",
        "Action": [
            "ecr:BatchCheckLayerAvailability",
            "ecr:CompleteLayerUpload",
            "ecr:GetAuthorizationToken",
            "ecr:InitiateLayerUpload",
            "ecr:PutImage",
            "ecr:UploadLayerPart"
        ],
        "Resource": "*"
    }

As far as I understood Resource must be * because some of those actions do not work otherwise. Improvements are welcome!

Torsk

The only one of those which requires * is GetAuthorizationToken -- giving the rest * is gregarious. – Carrefour

Answer 4

If you have a virtual environment folder-mine was .venv, try removing it. Build and push your image again. That worked for me

Pizzicato