DUCC ingest image from docker hub, allowing simple docker push to work

An interesting discussion we had today prompted this request.

Would be convenient if it were possible to push images into a docker registry and to have such image directly ingested into DUCC, without passing from the wish-list.

I can see two possible approach to this,

1. Webhook based
2. Custom registry

Let's explore both of them.

Webhook

The docker registry provide a webhook for when images are pushed or pulled. 
Docs on: https://docs.docker.com/registry/notifications/

It would be possible to just listen to such notification and when a new layer is pushed, we simply unpack it into unpacked.cern.ch

This raise some issue:

1. An HTTP endpoint from CERN need to listen to external calls.
2. We would need anyway some form of check and controls about what we allow in
3. It is going to grow very quickly, especially if we allow all the atlas/* or cms/* images, can we support it?
4. What about garbage collection?
5. How do we signal that we have finally ingested a layer or a whole image? (Some general broadcast mechanism?)
6. Gitlab seems to not support this solution quite simply yet.

A possible idea would be to run our own registry and subscribe to notification from that one. This will allow to relax the security checks but will also be more cumbersome for the users.

Custom Registry

Another approach is to deploy our own custom registry.

This is definitely not a simple task, but I believe it is in the realm of manageable.

Indeed docker registries have a pluggable storage backend to allow multiple implementation of the same functionality (store layers.)

We could hook up into this the storage backend to ingest layer into cvmfs during the publishing.

Issues with this approach:

1. Need to run and maintain our own docker registry
2. Need to develop and maintain our own custom storage backend
3. Push time will be quite long, potentially causing issues in the CI or even in the docker

The pluggable storage backend is defined by the interface in this file: https://github.com/docker/distribution/blob/master/registry/storage/driver/storagedriver.go

In order to simplify our development we could just fork the filesystem driver ( https://github.com/docker/distribution/blob/master/registry/storage/driver/filesystem/driver.go )