Feature proposal: multiple instances of one protocol handler

[kpfleming]

Debian publishes packages at deb.debian.org and security.debian.org. The proxy currently only offers one 'debian' route, with a potentially configurable upstream URL, but supporting the full Debian package set requires access to both sites.

It is possible that this could be handled by the multiple-upstream proposal I posted in #55 , but a better solution would probably be a 'debian-security' route in the proxy, with a different default upstream from the 'debian' route. Looking at the code this seems like it would be easy to add, I'd just need to extend the NewDebianHandler function to accept an upstream URL and then write a NewDebianSecurityHandler wrapper for it.

[andrew]

If we can solve it in the general sense that would ideal, hopefully the pypi example from #55 will provide a good pattern we can add to more ecosystems.

[ignusws]

It would be more flexible to make upstream repositories configurable and support multiple aliases, similar to how tools like apt-cacher-ng remap works instead of hardcoding two repos

upstream:
  apt:
    debian: http://deb.debian.org/debian/
    debian-security: http://security.debian.org/debian-security
    archive: http://archive.debian.org/debian/

source.list

deb http://localhost:8080/apt/debian trixie main non-free-firmware
deb http://localhost:8080/apt/debian-security trixie-security main non-free-firmware
deb http://localhost:8080/apt/archive bullseye main contrib

Each alias (e.g. /apt/debian, /apt/archive) maps to a configured upstream
Cache artifacts are stored separately per alias (e.g. in dedicated subdirectories)
The proxy becomes reusable for any APT-compatible repository, not just Debian

[kpfleming]

Yep, I'm planning to do something like that exactly. I have six Debian-style upstreams that I use, from various companies. I'm planning to use a single-level naming scheme though, so the existing debian default will work as it does today, but users could add debian-security, mozilla-debian, nodesource-debian, etc.

[damian-buho]

Oh, sorry, I didn't realize my issue was a duplicate.

[Derkades]

This would also be valuable for containers, to make available registries other than docker hub e.g. quay.io and ghcr.io

[mxey]

This would also be valuable for containers, to make available registries other than docker hub e.g. quay.io and ghcr.io

Doing this for OCI registries requires a little more logic. Either each registry would have to be a separate domain, or be mapped into a “virtual registry” of sorts. That's because the OCI registry API always talks to /v2/.

For example, listing the tags for image mycache.local/example.com/foo becomes https://mycache.local/v2/example.com/foo/tags/list.

@andrew Is this something you'd be interested in having? I am currently evaluating dependency caches and this would make it work as an OCI cache for us. AFAICT this is also what Sonatype Nexus, Artifactory and Gitlab Virtual Registry are doing.

[andrew]

Yeah I think it would be a good addition

[mxey]

For example, listing the tags for image mycache.local/example.com/foo becomes https://mycache.local/v2/example.com/foo/tags/list.

It might be possible to achieve this by mounting a handler for Docker Hub at /v2 and mounting other handlers at /v2/example.com