Skip to content

Restore registry-server startup probe FailureThreshold to 15#3876

Open
emmahone wants to merge 1 commit into
operator-framework:masterfrom
emmahone:OCPBUGS-78095-startup-probe-failure-threshold
Open

Restore registry-server startup probe FailureThreshold to 15#3876
emmahone wants to merge 1 commit into
operator-framework:masterfrom
emmahone:OCPBUGS-78095-startup-probe-failure-threshold

Conversation

@emmahone

Copy link
Copy Markdown

Description of the change:

Restores `StartupProbe.FailureThreshold` for the registry-server container from `10` back to `15`, returning the startup budget from 100s to 150s. This reverts the threshold portion of 401bfff, which changed FailureThreshold from 15 to 10 while adding TimeoutSeconds.

Test fixtures updated to reflect the recomputed `olm.pod-spec-hash` values.

Motivation for the change:

The 100s budget (`FailureThreshold: 10 × PeriodSeconds: 10`) can be too tight when there are performance constraints or large custom CatalogSources (OCPBUGS-78095). The commit message in 401bfff states the intent was to "maintain the ~150s startup time failure as originally designed," suggesting the reduction from 15 to 10 was unintentional. This change restores `FailureThreshold` to `15` (150s budget), which provides sufficient margin for these scenarios without being excessive.

Architectural changes:

None.

Restores StartupProbe.FailureThreshold from 10 back to 15, returning
the startup budget from 100s to 150s. This reverts the threshold
portion of 401bfff.

The 100s budget can be too tight when there are performance constraints
or large custom CatalogSources (OCPBUGS-78095). The commit message in
401bfff states the intent was to maintain the ~150s startup time as
originally designed, suggesting the reduction was unintentional.

Test fixtures updated to reflect recomputed olm.pod-spec-hash values.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 22, 2026 20:40
@openshift-ci
openshift-ci Bot requested a review from camilamacedo86 July 22, 2026 20:40
@openshift-ci

openshift-ci Bot commented Jul 22, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign oceanc80 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci
openshift-ci Bot requested a review from oceanc80 July 22, 2026 20:40
@openshift-ci openshift-ci Bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jul 22, 2026
@openshift-ci

openshift-ci Bot commented Jul 22, 2026

Copy link
Copy Markdown

Hi @emmahone. Thanks for your PR.

I'm waiting for a operator-framework member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adjusts the generated CatalogSource registry-server Pod spec to restore the original startup probe time budget, improving tolerance for slower starts (e.g., large catalogs or constrained environments) in the registry reconciler.

Changes:

  • Restore StartupProbe.FailureThreshold for the registry-server container from 10 to 15 (150s budget with PeriodSeconds: 10).
  • Update unit test fixtures to reflect the resulting olm.pod-spec-hash label changes and the restored probe threshold.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
pkg/controller/registry/reconciler/reconciler.go Restores StartupProbe.FailureThreshold to 15 for the registry-server Pod.
pkg/controller/registry/reconciler/reconciler_test.go Updates expected startup probe threshold and recomputed olm.pod-spec-hash values in fixtures.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

},
},
FailureThreshold: 10,
FailureThreshold: 15,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restores StartupProbe.FailureThreshold for the registry-server container from 10 back to 15, returning the startup budget from 100s to 150s...

But in #2919:

Also adjusted the failure threshold in order to maintain the ~150s startup time failure as originally designed (timeout of 5s + period of 10s = 15s per attempt * 10 failures = 150s).

So maybe you're going up to timeout of 5s + period of 10s = 15s per attempt * 15 failures = 225s)? I'm not clear on what "as originally designed" is talking about though, so not clear what the downsides would be of going too high here.

@emmahone emmahone Jul 22, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Today with the probe interval of 10s and failure threshold of 10, my understanding is the 10th probe is sent out at the 90s mark and times out at 5s. So technically the time to failure is 95s. After this change the time to failure would be 145s.

Versus what occurred prior to 401bfff where the last probe was sent out at 140s and times out based on the default timeout.

EDIT: Just to clarify, 'as originally designed' is just quoted from the commit message in 401bfff. The tuning of these probe interval and failure threshold is fairly arbitrary which is why I am pushing for increasing them slightly rather than simply telling customers to fix the underlying issues causing these issues and ignoring these values. This gives us some resiliency with a small cost in time to failure.

-mahoney

@perdasilva

Copy link
Copy Markdown
Collaborator

/ok-to-test

@openshift-ci openshift-ci Bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test Indicates a non-member PR verified by an org member that is safe to test.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants