[fix](nereids) Bound outer join null-reject inference to nullable outputs#65250
[fix](nereids) Bound outer join null-reject inference to nullable outputs#65250foxtail463 wants to merge 2 commits into
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
TPC-H: Total hot run time: 29409 ms |
FE UT Coverage ReportIncrement line coverage |
TPC-DS: Total hot run time: 173640 ms |
ClickBench: Total hot run time: 25.41 s |
morrySnow
left a comment
There was a problem hiding this comment.
Thanks for this PR! The bounding of null-reject inference to nullable-side outputs is a clean optimization. I reviewed all angles (line-by-line, removed-behavior audit, cross-file tracer, reuse, simplification, efficiency, altitude, conventions) and found the implementation correct and well-tested.
One code organization observation (not a bug): The new getNullableSideOutput() in LogicalJoin re-encodes the join-type-to-nullable-side mapping (LEFT_OUTER→right nullable, RIGHT_OUTER→left nullable, FULL_OUTER→both) that already exists in JoinUtils.getJoinOutput() lines 417-433. If a new join type with different null-extension semantics is added, the mapping would need updating in multiple places. Consider whether a utility method on JoinType (e.g., isLeftSideNullable()/isRightSideNullable()) could consolidate this knowledge.
LGTM overall — the new test coverage is thorough (IN+OR-FALSE elimination, FULL OUTER degradation, early-return guard, negative cases).
FE Regression Coverage ReportIncrement line coverage |
yx-keith
left a comment
There was a problem hiding this comment.
Bounding the inference to getNullableSideOutput() is the right target set. Note this isn't purely skipping useless work — it changes elimination behavior: skipping mark-join slots as inference targets can drop an elimination, and the smaller target set interacts with the expression-complexity limit (a set that previously bailed on complexity may now fit and eliminate more).
P0 Regression / cloud_p0 are red. Since outer-join elimination affects results, please confirm these are expected plan changes (regenerated .out) rather than wrong results, and that skipping mark-join slots is always safe for eliminating the current join.
|
run buildall |
… and JoinUtils. Also remove the redundant mark-slot guard from target-based not-null inference
1722a9b to
1e9ee9c
Compare
|
run buildall |
Problem Summary:
Outer join elimination uses fold-based null-reject inference to decide whether
nullable-side rows from the current outer join can be filtered away. This check
only needs slots from that join's nullable-side outputs, but the previous flow
could also test unrelated predicate inputs. When a filter predicate references a
mark slot produced by another join, folding that slot is useless for eliminating
the current outer join and can be costly when the fixed-point rewrite batch
revisits the same predicate multiple times.
Solution:
Add an API to infer null-rejecting slots only for a given target slot set, and
use the current join's nullable-side outputs as that target in
EliminateOuterJoin. Skip mark-join slots as inference targets, preserve the
existing expression complexity limits, and avoid rewriting when the join type
does not change.