Skip to content

Move AssertUtils to the api submodule#13654

Merged
abellina merged 2 commits intoNVIDIA:mainfrom
abellina:move_assert_utils_to_api
Oct 24, 2025
Merged

Move AssertUtils to the api submodule#13654
abellina merged 2 commits intoNVIDIA:mainfrom
abellina:move_assert_utils_to_api

Conversation

@abellina
Copy link
Copy Markdown
Collaborator

Fixes #13647

Description

Ok, so I believe the integration tests locally and in pre-merge are running against a specific non-parallel-world jar, so all classes are in the expected namespaces (not scribbled away over at sparkver/com/nvidia..), but the nightly doesn't seem to be doing that.

I had added a class that I need to be able to read from the integration_tests submodule. And that seems to be getting shimmed.

Here are my changes to help fix:

  • I moved it to the "api" submodule, since that's not shimmed.
  • I also added a dependency to integration_test to get the api submodule.

I am having trouble testing locally (network). I'd like @gerashegalov's opinion on this patch, as he knows the build system better than anyone.

Checklists

  • This PR has added documentation for new or modified features or behaviors.
  • This PR has added new tests or modified existing tests to cover new code paths.
    (Please explain in the PR description how the new code paths are tested, such as names of the new/existing tests that cover them.)
  • Performance testing has been performed and its results are added in the PR description. Or, an issue has been filed with a link in the PR description.

Signed-off-by: Alessandro Bellina <abellina@nvidia.com>
Copilot AI review requested due to automatic review settings October 23, 2025 21:44
@abellina abellina requested a review from a team as a code owner October 23, 2025 21:44
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR addresses an issue where AssertUtils was being shimmed when running in nightly builds, making it inaccessible from the integration_tests submodule. The fix moves AssertUtils to the api submodule (which is not shimmed) and adds the necessary dependency.

  • Moved AssertUtils class from a shimmed location to the api submodule
  • Added rapids-4-spark-sql-plugin-api dependency to integration_tests submodule

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Greptile Summary

This PR relocates AssertUtils.scala from the sql-plugin module to the sql-plugin-api module to resolve a nightly build failure. The issue arose because nightly builds use "parallel-world" shimming, which rewrites class package names to version-specific namespaces (e.g., sparkver/com/nvidia/...). When AssertUtils lived in sql-plugin, it was shimmed and relocated, breaking imports in integration_tests that expected it at com.nvidia.spark.rapids.AssertUtils. The sql-plugin-api module is explicitly excluded from shimming, so moving the class there keeps it in a stable namespace across all build types (local, pre-merge, and nightly). A corresponding dependency on rapids-4-spark-sql-plugin-api was added to integration_tests/pom.xml with provided scope, since the api module is already on the runtime classpath.

Confidence score: 5/5

  • This PR is safe to merge with minimal risk as it addresses a specific build infrastructure issue without altering business logic.
  • Score reflects a straightforward refactoring that moves a single utility class to a more appropriate module to avoid shimming side-effects; no code behavior changes.
  • No files require special attention, though verify that nightly builds succeed and integration_tests can resolve com.nvidia.spark.rapids.AssertUtils without errors.

2 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@abellina
Copy link
Copy Markdown
Collaborator Author

build

@pxLi pxLi requested a review from gerashegalov October 24, 2025 00:22
@pxLi
Copy link
Copy Markdown
Member

pxLi commented Oct 24, 2025

also requires an update to scala2.13/pom file

@abellina
Copy link
Copy Markdown
Collaborator Author

build

@abellina abellina requested a review from pxLi October 24, 2025 01:49
@pxLi
Copy link
Copy Markdown
Member

pxLi commented Oct 24, 2025

build

Copy link
Copy Markdown
Collaborator

@gerashegalov gerashegalov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sameerz sameerz added the bug Something isn't working label Oct 24, 2025
@abellina abellina merged commit e31356c into NVIDIA:main Oct 24, 2025
60 checks passed
@abellina abellina deleted the move_assert_utils_to_api branch October 24, 2025 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] AssertUtils is not a member of package com.nvidia.spark.rapids

6 participants