Docker hub cache with Harbor

Question

I need to cache docker images while pulling from the docker hub in my Harbor "Proxy Cache " project. Therefore I have configured a project with an option Proxy Cache. The registries section also added a new registry endpoint with a provider as "Docker Hub." I added the following configuration to the docker daemon.

cat /etc/docker/daemon.json 
{
 "registry-mirrors": ["https://registry.harbor.com"]
}

While I'm pulling images from the docker hub it is not caching on my harbor project. Need help to resolve this issue and how my request fulfills with the harbor.

example
my cache project name = proxy
i need to pull httpd:latest

This method not working also

Osteo

Answer 1

Updated TLDR;

At the time of answering this question originally, there wasn't a good solution. You can read my original answer. Or just scroll down to the update section where I tell you that Harbor v2.1's blog says they now support this.

Original Answer

I can answer part of your problem. But the answer to part 2 is that you can't. I can link you the issue to show you that they explicitly chose not to due to technical limitations. The good news is that they are aware that this is still somethign that the community wants.

Part 1

One thing you may not know, repos on hub.docker that do not have a project group (like docker pull nginx), still need a project in your harbor that match. It will match on the project name library. So make sure you have a project named library. Not having this library project probably wont effect pass through caching but it definately effects replication.

My setup contains: harbor url: harbor.mydomain.com

project:

• library
• cache_proxy-hub-docker

I got my pulls to work with the following example:

    docker pull harbor.mydomain.com/cache_proxy-hub-docker/goharbor/redis-photon:v2.1.0
    v2.1.0: Pulling from cache_proxy-hub-docker/goharbor/redis-photon
    b2823a5a3d08: Pull complete
    ...omitted...
    369af38cd511: Pull complete
    Digest: sha256:11bf4d11d81ef582401928b85aa2e325719b125821a578c656951f48d4c716be

Remember, for something like docker pull ngninx, you have to do it as if it were actually library/nginx

    docker pull harbor.mydomain.com/cache_proxy-hub-docker/library/nginx
    Using default tag: latest
    latest: Pulling from cache_proxy-hub-docker/library/nginx
    d121f8d1c412: Pull complete
    ...ommitted...
    Digest: sha256:fc66cdef5ca33809823182c9c5d72ea86fd2cef7713cf3363e1a0b12a5d77500

When I look in projects/cache_proxy-hub-docker I see:

• cache_proxy-hub-docker/library/nginx
• cache_proxy-hub-docker/goharbor/redis-photon

Please also remember, the pull command with the prefix, is also going to be what that image will be known as on your machine after the pull. You'll have to retag it to what you're expecting it to really be. That's why the docker daemon solution is so appealing...

Part 2

I ran around on this same issue. Finally, I suspected they didn't implement it this way. That is correct:

https://github.com/goharbor/harbor/issues/8082#issuecomment-698012277

question:

Is there anyway to configure harbor 2.1 as a transparent docker hub mirror? ...

answer:

not at this time ... we couldn't find a good enough solution in 2.1, but this requirement is known to us.

UPDATE

The Harbor blog for v2.1 indicates that they have now fully added this feature. My answer above is accurate for versions prior to 2.1. I haven't personally tested this but I will link the blog post talking about it.

Blog: https://goharbor.io/blog/harbor-2.1/

Seduce

For my use cases, this isn't really an effective pull through proxy. I want to have all pulls go through it. I don't want to have to change all my image keys out for ones that will work with this. This was to catch the stragglers. – Seduce

It is working now. Thank you for the detailed explanation. – Osteo

@DamithUdayanga, sure thing! – Seduce

it's weird though because here (github.com/goharbor/harbor/blob/master/contrib/…) it says its possible to use as registry-mirror? – Jezebel

@Jezebel it seems to be possible but it disables the use of harbor as a private push repository. – Toxic

Harbor 2.1 seems to have added this as a full feature now. goharbor.io/blog/harbor-2.1 – Seduce