fix: don't duplicate volatile expressions when pushing projection into file scan

[fordN]

Which issue does this PR close?

• Closes Projection pushdown into file scan duplicates non-deterministic functions (regression in 52.0.0) #23220.

Rationale for this change

The reported bug showed that when volatile (non-deterministic) expressions are referenced multiple times in an outer function the query results can be incorrect.

To produce correct results, a volatile expression (e.g. random(), uuid()) aliased once in a subquery and referenced multiple times must be evaluated once and reused. Since 52.0.0 this evaluation pattern for volatile functions has been broken; the physical projection-pushdown rule merges the outer projection into the file DataSourceExec, inlining the aliased volatile expression at each reference site. Instead of being evaluated once, the volatile expression is evaluated N times, so the references diverge:

SELECT s.r AS x, s.r AS y FROM (SELECT random() AS r FROM t) AS s;  -- x != y on >= 52.0.0

This was correct in 51.0.0 and regressed in 52.0.0/53.0.0. It reproduces on file scans (Parquet/CSV) but not in-memory tables, and was surfaced downstream in Ibis. Worth noting, #10337 appears to report the same class of bug but a different cause (I'll try to look into that issue soon as well).

What changes are included in this PR?

FileScanConfig::try_swapping_with_projection now declines to merge a projection into the file source when the merge would inline a volatile expression that the incoming projection references. The check reuses the existing is_volatile() utility from datafusion_physical_expr_common — the same volatility gate the physical ProjectionPushdown and FilterPushdown rules already apply — so file-source projection merging is now consistent with them. Deterministic expressions still merge freely.

The guard blocks when a volatile inner expression is referenced at least once (not only more than once), because a single outer expression can itself duplicate the value (e.g.r + r).

Are these changes tested?

Yes:

• Unit tests in file_scan_config.rs:
  • volatile referenced ≥1× → blocked;
  • deterministic computed and column-only → allowed;
  • unreferenced volatile → allowed;
  • single-expression self-reference (r + r) → blocked;
  • volatile nested in arithmetic → blocked;
  • empty → allowed.
• A regression test in projection_pushdown.slt asserting x = y for the aliased-random() pattern over a Parquet scan.
• Full datafusion-sqllogictest suite passes.

Are there any user-facing changes?

No API changes! Query results for the affected pattern are corrected (a volatile expression aliased in a subquery is no longer duplicated by projection pushdown into a file scan). Physical plans for such queries now retain a ProjectionExec above the scan rather than inlining the volatile expression into the DataSourceExec projection.

[fordN]

First-time contributor here! So I'll need a committer to approve the CI workflow.
cc. @alamb since you were involved in related discussion on #12599

[xudong963]

That is safe but imprecise. A better implementation would count references with expr.apply(...), like try_collapse_projection_chain already does and block only when a volatile inner expression is referenced more than once.

[fordN]

@xudong963 good call. I've updated the reference counting algorithm to count references with multiplicity similar to how it's done in try_collapse_projection_chain(). Now with more precise counting we can block only when volatile expression is referenced more than once.

I've also added a new unit test for the single-reference case.

[github-actions]

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details

     Cloning apache/main
    Building datafusion-datasource v54.0.0 (current)
       Built [  38.702s] (current)
     Parsing datafusion-datasource v54.0.0 (current)
      Parsed [   0.030s] (current)
    Building datafusion-datasource v54.0.0 (baseline)
       Built [  36.774s] (baseline)
     Parsing datafusion-datasource v54.0.0 (baseline)
      Parsed [   0.032s] (baseline)
    Checking datafusion-datasource v54.0.0 -> v54.0.0 (no change; assume patch)
     Checked [   0.252s] 223 checks: 222 pass, 1 fail, 0 warn, 30 skip

--- failure function_missing: pub fn removed or renamed ---

Description:
A publicly-visible function cannot be imported by its prior path. A `pub use` may have been removed, or the function itself may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/function_missing.ron

Failed in:
  function datafusion_datasource::file_scan_config::output_partitioning_from_partition_fields, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/be6f5df98a10c764b1979e225f22ca386122c7c5/datafusion/datasource/src/file_scan_config/mod.rs:582

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [  77.023s] datafusion-datasource
    Building datafusion-sqllogictest v54.0.0 (current)
       Built [ 182.876s] (current)
     Parsing datafusion-sqllogictest v54.0.0 (current)
      Parsed [   0.021s] (current)
    Building datafusion-sqllogictest v54.0.0 (baseline)
       Built [ 177.463s] (baseline)
     Parsing datafusion-sqllogictest v54.0.0 (baseline)
      Parsed [   0.024s] (baseline)
    Checking datafusion-sqllogictest v54.0.0 -> v54.0.0 (no change; assume patch)
     Checked [   0.092s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [ 363.172s] datafusion-sqllogictest

[xudong963]

@fordN Thanks for the first contribution!