Skip to content

Fix JDK version diff in RelationalGroupedDataset#14192

Merged
GaryShen2008 merged 5 commits intoNVIDIA:mainfrom
GaryShen2008:fix-14188
Jan 26, 2026
Merged

Fix JDK version diff in RelationalGroupedDataset#14192
GaryShen2008 merged 5 commits intoNVIDIA:mainfrom
GaryShen2008:fix-14188

Conversation

@GaryShen2008
Copy link
Copy Markdown
Collaborator

@GaryShen2008 GaryShen2008 commented Jan 23, 2026

Another test case to use RelationalGroupedDataset.toString to expect the correct type string.
But in JDK11+ it returns empty string.
Created testRapids cases by checking JDK version to change the expected string.
Close #14188

Signed-off-by: Gary Shen <gashen@nvidia.com>
Copilot AI review requested due to automatic review settings January 23, 2026 07:40
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Jan 23, 2026

Greptile Summary

This PR fixes JDK 11+ test failures in DatasetSuite by creating JDK-aware replacement tests for RelationalGroupedDataset.toString behavior.

  • Added two testRapids tests in RapidsDatasetSuite that dynamically expect the correct type string based on JDK version (empty on JDK 11+, "GroupBy" on JDK 8)
  • Added corresponding exclusions in RapidsTestSettings to disable the original Spark tests that assume JDK 8 behavior
  • The fix follows the same pattern already used in RapidsDataFrameAggregateSuite for the similar SPARK-24788 test

Confidence Score: 5/5

  • This PR is safe to merge - it only modifies test code to handle JDK version differences and follows an established pattern.
  • The changes are limited to test infrastructure, follow an existing pattern already in use (RapidsDataFrameAggregateSuite), and correctly address the documented JDK 11+ behavior difference. The logic has been verified against the error messages from the original issue.
  • No files require special attention.

Important Files Changed

Filename Overview
tests/src/test/spark330/scala/org/apache/spark/sql/rapids/suites/RapidsDatasetSuite.scala Adds two new testRapids tests to handle JDK version differences in RelationalGroupedDataset.toString behavior. The logic correctly expects empty type string on JDK 11+ and "GroupBy" on JDK 8.
tests/src/test/spark330/scala/org/apache/spark/sql/rapids/utils/RapidsTestSettings.scala Adds exclusions for two original Spark tests that fail on JDK 11+ due to RelationalGroupedDataset.toString behavior differences. The excluded tests are replaced by JDK-aware testRapids versions.

Sequence Diagram

sequenceDiagram
    participant TestRunner
    participant RapidsTestSettings
    participant RapidsDatasetSuite
    participant getJavaMajorVersion
    participant RelationalGroupedDataset

    TestRunner->>RapidsTestSettings: Check test exclusions
    RapidsTestSettings-->>TestRunner: Exclude original DatasetSuite tests
    TestRunner->>RapidsDatasetSuite: Run testRapids version
    RapidsDatasetSuite->>getJavaMajorVersion: Query JDK version
    getJavaMajorVersion-->>RapidsDatasetSuite: Return major version (8 or 11+)
    RapidsDatasetSuite->>RapidsDatasetSuite: Set expectedTypeString based on JDK
    RapidsDatasetSuite->>RelationalGroupedDataset: Call toString()
    RelationalGroupedDataset-->>RapidsDatasetSuite: Return formatted string
    RapidsDatasetSuite->>RapidsDatasetSuite: Assert expected matches actual
Loading

Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 files reviewed, 2 comments

Edit Code Review Agent Settings | Greptile

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request addresses JDK 11+ test failures in DatasetSuite by creating JDK-version-aware test cases for RelationalGroupedDataset.toString behavior. In JDK 11+, the toString method returns an empty string for the type field instead of "GroupBy".

Changes:

  • Excluded two failing tests from the base DatasetSuite and replaced them with GPU-specific testRapids versions
  • Added JDK version detection logic to handle the different toString behavior across JDK versions
  • Created two new test cases in RapidsDatasetSuite that check for the correct output based on JDK version

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.

File Description
tests/src/test/spark330/scala/org/apache/spark/sql/rapids/utils/RapidsTestSettings.scala Excluded two original tests that fail on JDK 11+ and documented the reason for exclusion
tests/src/test/spark330/scala/org/apache/spark/sql/rapids/suites/RapidsDatasetSuite.scala Added two testRapids cases with JDK version-aware assertions for RelationalGroupedDataset.toString behavior

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@GaryShen2008
Copy link
Copy Markdown
Collaborator Author

build

GaryShen2008 and others added 2 commits January 23, 2026 15:49
…s/RapidsTestSettings.scala

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Gary Shen <gashen@nvidia.com>
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Signed-off-by: Gary Shen <gashen@nvidia.com>
Signed-off-by: Gary Shen <gashen@nvidia.com>
@GaryShen2008
Copy link
Copy Markdown
Collaborator Author

build

@GaryShen2008 GaryShen2008 merged commit 5013640 into NVIDIA:main Jan 26, 2026
46 of 47 checks passed
@sameerz sameerz added the bug Something isn't working label Feb 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] JDK11 Nightly Build: DatasetSuite test failures for RelationalGroupedDataset toString format

5 participants