[branch-54] refactor: wrap HigherOrderUDFImpl in a concrete HigherOrderUDF struct (#22593)#22635
Open
alamb wants to merge 1 commit into
Open
[branch-54] refactor: wrap HigherOrderUDFImpl in a concrete HigherOrderUDF struct (#22593)#22635alamb wants to merge 1 commit into
alamb wants to merge 1 commit into
Conversation
…#22593) <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123. --> Part of #21172 `HigherOrderUDF` was the only UDF kind defined as a trait that callers used directly via `Arc<dyn HigherOrderUDF>`. The other UDFs: `ScalarUDF`, `AggregateUDF`, `WindowUDF` — are concrete structs that wrap their respective `*Impl trait`, which makes inherent methods like `with_aliases` ergonomic to call on the function object. With the trait-only setup, adding aliases to an existing higher-order function required an extension trait import or a free helper function. This PR brings higher order functions in line with the other UDFs so the same `with_aliases` pattern works. - Rename the `HigherOrderUDF` trait to `HigherOrderUDFImpl`, matching `ScalarUDFImpl`/`AggregateUDFImpl`. Add a concrete `HigherOrderUDF` struct wrapping `Arc<dyn HigherOrderUDFImpl>`, with the same shape as `ScalarUDF`: new_from_impl, new_from_shared_impl, inner, with_aliases, From<F: HigherOrderUDFImpl>, and delegate methods for every trait method. `with_aliases` is backed by a private `AliasedHigherOrderUDFImpl` decorator (same pattern as `AliasedScalarUDFImpl`). - Update `Expr::HigherOrderFunction`, `FunctionRegistry`, the `create_higher_order! `singleton macro, and all consumer files ( across several crates) to use `Arc<HigherOrderUDF>` instead of `Arc<dyn HigherOrderUDF>`. Existing impls (`ArrayFilter`, `ArrayTransform`, `ArrayAnyMatch`) now implement `HigherOrderUDFImpl`; their public constructors continue to return `Arc<HigherOrderUDF>` so external call sites need no changes. Callers can now write: `array_filter_higher_order_function().with_aliases(["filter"]) ` exactly like the existing scalar pattern: `make_array_udf().as_ref().clone().with_aliases(["array_construct"]) ` Covered by existing tests Yes, any code referring to `Arc<dyn HigherOrderUDF>` needs to become `Arc<HigherOrderUDF>`, and any code that wrote `impl HigherOrderUDF for MyHOF` needs to write i`mpl HigherOrderUDFImpl for MyType`. Constructing a HigherOrderUDF from an impl is HigherOrderUDF::new_from_impl(my_impl) (or my_impl.into()).
github-actions
Bot
added
sql
55 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
54.0.0(Apr 2026 / May 2026) #21080This PR:
branch-54lineNote on conflict resolution
The cherry-pick had conflicts in two test areas:
datafusion/functions-nested/src/array_any_match.rs— only the test-moduleuseline conflicted;branch-54's test does not reference the renamed symbols, so the existing import was kept (the productionHigherOrderUDF->HigherOrderUDFImplrename applied cleanly).datafusion/substrait/tests/cases/roundtrip_logical_plan.rs— the upstream PR modified theroundtrip_array_transform_higher_order_functiontest andArrayTransformhelper, but that test was added tomainafterbranch-54was cut and does not exist onbranch-54. It is out of scope for this refactor backport, so thebranch-54file was left unchanged.