feat(sample transform): add optional discarded_event_tags callback#25477
Draft
p-parekh wants to merge 1 commit into
Draft
feat(sample transform): add optional discarded_event_tags callback#25477p-parekh wants to merge 1 commit into
p-parekh wants to merge 1 commit into
Conversation
Adds Sample::with_discarded_event_tags(f) — an optional callback whose returned (key, value) pairs are merged into the component_discarded_events_total counter labels at the drop site. Default None; behavior unchanged for every existing caller. Library consumers (e.g., downstream wrappers that pre-group events) can use this to attach per-event tags to the discard metric without forking the transform. When the callback is set, the drop site bypasses the standard ComponentEventsDropped<INTENTIONAL> path because that type registers its counter with a fixed label set at registration time — dynamic per-event labels cannot flow through it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
Thank you for your contribution! Before we can merge this PR, please sign our Contributor License Agreement. To sign, copy and post the phrase below as a new comment on this PR.
I have read the CLA Document and I hereby sign the CLA You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an optional
with_discarded_event_tagsbuilder on theSampletransform's runtime type. When set, the callback is invoked at the drop site and its returned(key, value)pairs are merged into thecomponent_discarded_events_totalcounter labels. DefaultNone— behavior unchanged for every existing caller.Motivation
Downstream wrappers (concretely, the Observability Pipelines Worker — a Datadog-internal consumer that uses Vector as a library) configure
Samplewith agroup_bytemplate but cannot currently attach per-event group-by tags to the discard metric. The discard counter today carries onlyintentional:true, so when a customer pipeline drops events the operator has to grep logs to know which group caused the drops.A wrapping fork or duplicating Sample wholesale was rejected; this small extension point lets library consumers attach the tags they own without Vector taking a position on what the tags mean.
Vector configuration
No new TOML/YAML config. The callback is a programmatic API used by library consumers, not the Vector binary. Existing pipelines using
samplesee no behavior change.How did you test this PR?
discarded_event_tags_callback_is_invoked_on_dropinsrc/transforms/sample/tests.rsthat:Samplewith an aggressiverate=100Sample::transformdiscarded_event_tagsfield isNonein every existing test path).SampleEventDiscardedemission path is taken when the callback isNone, so theComponentEventsDropped<INTENTIONAL>plumbing is unaffected for current users.Change Type
Is this a breaking change?
The new field defaults to
None.Sample::newandSample::new_with_dynamicsignatures are unchanged. The only addition is thewith_discarded_event_tagsbuilder method.Does this PR include user facing changes?
changelog.d/<id>_sample_discarded_event_tags.enhancement.md.Library consumers gain a new opt-in API; pipeline operators see no change unless a downstream consumer of Vector wires the callback through.
Notes
component_discarded_events_total; only the label set varies based on the (optional) callback's return value.metrics::counter!) and logs a debug message in the same shape asComponentEventsDropped<INTENTIONAL>::emitwould. This is necessary becauseComponentEventsDroppedregisters its counter with a fixed label set at registration time, so dynamic per-event labels cannot flow through it.Some. WithNone, the existingemit!(SampleEventDiscarded)path runs unchanged.&Event(not&LogEvent) so trace-event drops also benefit if a future consumer wants per-trace tags. Today the existing trace-event handling inSample::transformcallsemit!(SampleEventDiscarded)via the same branch, so it gains tag support automatically when the callback is set.