You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Move datafusion_array_function specific rewrite rules like to datafusion_functions_array crate #9519
Is your feature request related to a problem or challenge?
As we port over functions out of the datafusion-physical-expr crate, we have discovered several parts of ExprSimplif that rely on semantics of the array functions themselves
As @jayzhan22 noted on #9496#9496 (comment), this rewrite is not correct if users supply their own implementation of array_has_all as that implementation may not be equal to DataFusion's array_has_all function. Thus the rule only makes sense when using the datafusion implementation
Describe the solution you'd like
Move the array_function specific rules such as rewrite_array_has_all_operator_to_func to the dataufusion_array_functions
Describe alternatives you've considered
Create a a User Defined Analyzer pass (defined in datafusion_functions_array) with the desired rewrite
Register the datafusion_function_array specific rewrite when the array functions in datafusion_array_function are registered (somewhere near here
Activity
jayzhan211 commentedon Mar 11, 2024
Should we change
add_analyzer_ruleto a map structure to avoid duplication similar toregister_udf?alamb commentedon Mar 11, 2024
I think the order of the rules is important. And some optimizer rules now are run multiple times
However, we may need to add some new way to add analyzer rules somewhere other than after all the other ("
push_front?) maybe 🤔