Pull images used in volumes with type=image - #13811
Conversation
glours
left a comment
There was a problem hiding this comment.
The fix proposed will introduce a regression for local images, I think we should reuse the same logic as isServiceImageToBuild
| } | ||
|
|
||
| eg.Go(func() error { | ||
| _, err := s.pullServiceImage(ctx, volService, opts.Quiet, project.Environment["DOCKER_DEFAULT_PLATFORM"]) |
There was a problem hiding this comment.
This unconditionally pulls types.VolumeTypeImage volume sources as external registry images, so docker compose pull --ignore-buildable still tries to pull a volume image that is actually produced by a buildable service in the same project, causing a regression for local/build-only images.
There was a problem hiding this comment.
Thanks for the review @glours. I've updated the PR to use isServiceImageToBuild — volume images produced by buildable services are now skipped when --ignore-buildable is set, and pull failures for buildable images are treated as warnings instead of errors.
80e530d to
39f6e06
Compare
glours
left a comment
There was a problem hiding this comment.
Thanks for working on this. The direction makes sense: docker compose pull should consider images referenced by type: image volumes, especially since up already handles missing volume images through pullRequiredImages.
I think the current implementation needs one more iteration though. It adds separate image-volume discovery directly in pull(), which makes this path diverge from the existing pullRequiredImages, mustPull, and isServiceImageToBuild logic.
That divergence is already visible with deduplication: volume images are queued before service images, so if a service image and a volume image use the same reference, the volume entry wins and the service-level pull logic can be bypassed (pull_policy, buildable image handling, platform, and the --policy flag applied to services).
For a first iteration, I think we should keep the existing service pull policies as the default source of truth when the same image is also used as a volume image. Those policies are explicit, either in the Compose file or through command flags, while image volumes currently don’t have an equivalent place to define pull behavior.
Could you refactor this around a shared helper that collects pull targets for both service images and image-volume sources, reusing the existing mustPull/isServiceImageToBuild semantics where applicable, instead of adding a separate loop in pull()?
Please also add tests for:
- pulling an image used only as a
type: imagevolume; - same image used by a service and a volume, with
pull_policy/--policy; - buildable/local image used as a volume image;
- deduplication between service images and volume images.
I have refactored the volume image discovery to address your feedback.
|
|
Recent compose-go changes (compose-spec/compose-go#894/#899) now resolve dependent images — both type: image volume sources and pre_start hook images — natively via WithImagesResolved. On top of that, #13937 is introducing an api.GetDependentImages() helper and touching the exact same pull() / pullRequiredImages() functions for the hook-image case. Merging this as-is would leave two divergent patterns for the same "dependent images" concept in pull.go (your collectPullTargets vs GetDependentImages) and conflict with #13937. So I'd like to converge on a single path rather than land both. Proposed plan: we merge #13937 first (it establishes GetDependentImages), then rebase this PR on top and have it extend GetDependentImages to also return type: image volume sources, so pull and up iterate service images + dependent images uniformly — dropping the separate collectPullTargets helper. You'd keep authorship for the type: image volume coverage / #13809. Happy to help wire it up once #13937 lands. Sorry for the extra round — the ground shifted under this one a bit. |
Hi! Thank you for the review and the explanation. The proposed plan sounds great to me. Let me know when #13937 is merged, and I will rebase my branch and update the code to use GetDependentImages(). I might need a little pointer on the exact changes when the time comes, but I'm ready to update it! |
This extends GetDependentImages to also return type=image volume sources, so that docker compose pull and up pull these images uniformly alongside pre_start hook images. Fixes docker#13809 Signed-off-by: Divyanshu Pandey <pandey.divyanshu03@gmail.com>
944e523 to
6718211
Compare
|
Hi! I've rebased this branch now that #13937 is merged. I dropped my custom helper and updated the new GetDependentImages() function to also return type: image volume sources as requested. Let me know if the implementation looks good! |
Description
When running
docker compose pull, images referenced in volume mounts withtype=imagewere not being pulled. Only the main service images (service.Image)were considered.
The
pullRequiredImages()function (used bydocker compose up) already handledthis case by iterating through service volumes and creating fake
ServiceConfigentries for volume images. However, the
pull()function (used bydocker compose pull)did not have this logic.
This adds the same volume image discovery to the
pull()function so thatdocker compose pullalso pulls images referenced intype=imagevolume mounts.Changes
pkg/compose/pull.go: Added a loop inpull()that iterates each service'svolumes, checks for
type=image, deduplicates againstimagesBeingPulled, andqueues them for pulling via
pullServiceImage().How to test
Fixes #13809