Repository navigation
feat(controller): add --min-poll-interval floor - #441
Open
henrysachs wants to merge 6 commits into
Open
henrysachs wants to merge 6 commits into
henrysachs wants to merge 6 commits into
Conversation
…iler Lint forbids direct managed.NewReconciler calls. Signed-off-by: Henry Sachs <henry.sachs@deutschebahn.com>
…otation The crossplane.io/poll-interval annotation let any resource poll GitLab as often as it asked. --min-poll-interval (env MIN_POLL_INTERVAL, default 30s) raises shorter annotation values to the floor. --poll is unchanged. Closes #428 Signed-off-by: Henry Sachs <henry.sachs@deutschebahn.com>
Signed-off-by: Henry Sachs <henry.sachs@deutschebahn.com>
Signed-off-by: Henry Sachs <henry.sachs@deutschebahn.com>
1 of 2 tasks
main.go registers the MR metric recorder, but no reconciler used it, so the first_time_to_reconcile, first_time_to_readiness, drift and deletion histograms were never recorded. Signed-off-by: Henry Sachs <henry.sachs@deutschebahn.com>
common.AddMRStateRecorder skips the recorder when MetricOptions or MRStateMetrics is nil. forbidigo forbids direct NewMRStateRecorder. Signed-off-by: Henry Sachs <henry.sachs@deutschebahn.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of your changes
Fixes #428
Why
A managed resource can set
crossplane.io/poll-intervalto any value, for example2s. crossplane-runtime raises the annotation only to the value ofmanaged.WithMinPollInterval. No controller set that option, so the floor was 0. One resource could poll GitLab every 2 seconds and use up the rate limit of its ProviderConfig token.What changed
--min-poll-interval, envMIN_POLL_INTERVAL, default10s. Annotation values below it are raised to it.--pollis not clamped. This matches--min-poll-intervalin Crossplane core.common.ControllerOptionsembedscontroller.Optionsand addsMinPollInterval. EverySetupfunction takes it.common.NewReconcilerbuilds each managed reconciler with the shared poll intervals, logger, event recorder and management-policies flag. The 39 namespaced controllers use it. The cluster copies are regenerated.managed.NewReconcileroutsidepkg/common/controller.go. A new controller cannot skip the floor.common.AddMRStateRecorderadds the state metrics recorder and skips it whenMetricOptionsorMRStateMetricsis nil. All controllers use it, and forbidigo forbids directstatemetrics.NewMRStateRecordercalls.--poll,--min-poll-intervaland--sync.common.NewReconcileralso passesMetricOptions.MRMetricstomanaged.WithMetricRecorder.main.goregistered that recorder, but no controller used it, also onmaster. So thecrossplane_managed_resource_first_time_to_reconcile,first_time_to_readiness,driftanddeletionhistograms stayed empty. Theexists,readyandsyncedgauges already worked.Commit 1 is the refactor and does not change behavior. Commit 2 adds the floor with a
30sdefault. Commit 3 changes the default to10s.Scope
10snow polls every10s. No example in this repo sets the annotation.--min-poll-interval=0srestores the old behavior.30sdefault. The maintainer chose10s. Crossplane core defaults its floor to1s.Setup*functions now takecommon.ControllerOptionsinstead ofcontroller.Options.MinPollInterval.Merge order
Merge this PR last, after #301, #414, #426, #430, #432, #433, #434, #435, #436, #439, #442 and #443.
managed.NewReconcilerdirectly and takecontroller.Options.After those PRs land, I will rebase this PR and move their controllers to
common.NewReconcilerandcommon.ControllerOptions. On the rebased branch, the forbidigo rule fails the lint for any controller that still callsmanaged.NewReconciler, so none can be missed.I have:
make reviewable testto ensure this PR is ready for review. I ran its steps one by one, see below.How has this code been tested
Verification
TestNewReconcilersets a10sminimum and checksRequeueAfter. Annotation2sgives10s,45sgives45s, and no annotation givesPollInterval. WithWithMinPollIntervalremoved from the helper, the2scase fails with2s.TestNewReconcileralso enables the management-policies feature. A resource with empty policies is then paused and gets no requeue. WithWithManagementPoliciesremoved from the helper, that case fails with10m0s.managed.NewReconcilerfails forbidigo, and the message namescommon.NewReconciler.TestNewReconcilerchecks that a reconcile adds onefirst_time_to_reconcilesample to the recorder fromMetricOptions. WithWithMetricRecorderremoved from the helper, that case fails with0. Metric options without a recorder do not panic./metricscheck with one projectVariable. Themastercode path showed onlyexists/ready/synced. With the fix, it showedfirst_time_to_reconcile1,first_time_to_readiness1,drift1 after a GitLab-side edit, anddeletion2.go test -p 4 ./...andgolangci-lint run ./...pass.go generate -tags generateinapisandpkgandgo mod tidyleave no diff.Live test against a GitLab 19.4 test instance, with a kind cluster and the provider run out of cluster with
--debug --poll 15s. A scratch project held threeVariableresources: namespaced withpoll-interval: 2s, namespaced without the annotation, and cluster-scoped with2s. The interval is the time between twoExternal resource is up to datedebug lines for one resource.2s2smaster30s)MIN_POLL_INTERVAL=5sThe patched runs logged no reconcile errors. In the
5srun, a value changed in GitLab was reverted within 12s. That reconcile logs an update, not "up to date", so one gap reads 10.4s. Deleting the resources removed the variables from GitLab.Commit 3 changes only the flag default, the README and the test values. I did not repeat the live run for it.
Not tested: setting the flag through a
DeploymentRuntimeConfigin a real Crossplane install.