Skip to content

Introduce module-kind convention plugins#11620

Open
bric3 wants to merge 7 commits into
masterfrom
bdu/introduce-first-module-conventions
Open

Introduce module-kind convention plugins#11620
bric3 wants to merge 7 commits into
masterfrom
bdu/introduce-first-module-conventions

Conversation

@bric3

@bric3 bric3 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

Introduces module-kind convention plugins for Java-backed Gradle modules and applies them from plugins {} while keeping the existing script plugins as the behavior source.

The basic change is usually removing the apply from of the java script plugin, and use the modern recommended plugins block with the identified module kind :

- apply from: "$rootDir/gradle/java.gradle"
+ plugins {
+   id 'dd-trace-java.module.instrumentation'
+ }

In particular these are the identified module kind, in this PR, it's likely some kind are not represented in this PR, and that's not the goal to be exhaustive at this stage, it can be refined later (for example for the shared modules in the agent jar).

  • dd-trace-java.module.agent-product
  • dd-trace-java.module.annotation-processor
  • dd-trace-java.module.bootstrap-component
  • dd-trace-java.module.distributable.api (here the dot is intended)
  • dd-trace-java.module.instrumentation
  • dd-trace-java.module.internal-component
  • dd-trace-java.module.smoke-test
  • dd-trace-java.module.testing-support

I chose module prefix as a way to convey the "higher level" module kind. While there might be other lower level technical conventions.

Small point of attention to the :dd-java-agent:instrumentation module, this PR replaces this check

if (subProj.path != ':dd-java-agent:instrumentation:vertx:vertx-redis-client:vertx-redis-client-stubs') {
  // don't include the redis RequestImpl stubs
  parent_project.dependencies {
    addProvider("implementation", providers.provider { project(subProj.path) })
  }
}

By checking if the sub project has the dd-trace-java.module.instrumentation convention, if it does it applies the necessary conventions for the instrumentation, so it just wraps existing code within the following closure:

subProj.pluginManager.withPlugin("dd-trace-java.module.instrumentation") {
  // ...
  parent_project.dependencies {
     addProvider("implementation", providers.provider { project(subProj.path) })
  }
}

Motivation

Prepare a gradual migration from lower-level script plugin application to higher-level conventions that describe what each module is.

Gradle references used for this migration direction:

Additional Details

The current forbidden-apis wiring for instrumentation modules is preserved behaviorally, but the investigation found it is not the intended rule set. On master, instrumentation tasks effectively read only instrumentation.txt; the expected rule set should be main.txt plus instrumentation.txt. The old += ordering with forbidden-apis 3.10 convention mapping let APIs covered only by main.txt leak into instrumentation modules. Restoring the intended combined signature set should be a follow-up PR.

:dd-java-agent:instrumentation itself is intentionally not migrated further here. Its aggregator-specific wiring, especially around muzzle reports, needs preparatory work before moving more of that project into convention-plugin logic.

jardiff was used to compare the agent jar from master and this branch. The agent jar contains both .class and .classdata entries, so the stricter check includes both and coalesces .classdata as class-like bytecode:

$ mise exec java@corretto-25.0.3.9.1 -- jardiff --stat -c classdata -i '**/*.class,**/*.classdata' /private/tmp/dd-trace-java-jardiff-master-640b1156e1/dd-java-agent/build/libs/dd-java-agent-1.64.0-SNAPSHOT.jar ./build/libs/dd-java-agent-1.64.0-SNAPSHOT.jar | tail -n 1

0 files changed, 0 insertions(+), 0 deletions(-)

Contributor Checklist

  • Format the title according to the contribution guidelines
  • Assign the type: and (comp: or inst:) labels in addition to any other useful labels
  • Avoid using close, fix, or any linking keywords when referencing an issue
    Use solves instead, and assign the PR milestone to the issue
  • Update the CODEOWNERS file on source file addition, migration, or deletion
  • Update public documentation with any new configuration flags or behaviors
  • Add your completed PR to the merge queue by commenting /merge. You can also:
    • Customize the commit message associated with the merge with /merge --commit-message "..."
    • Remove your PR from the merge queue with /merge -c
    • Skip all merge queue checks with /merge -f --reason "reason"; please use this judiciously, as some checks do not run at the PR-level (note: the PR still needs to be mergeable, this will only skip the pre-merge build)
    • Get more information in this doc

Jira ticket: [N/A]

bric3 added 6 commits June 10, 2026 13:07
Introduce module-kind convention plugins that describe what a Gradle
project is, rather than only how it is wired.

These conventions keep the existing script plugins as the source of
truth for now, but let modules apply them through plugins {}.
This starts a gradual migration path away from direct apply from usage
without changing existing lower level plugin scripts at this time.

This only changes the application path for `gradle/java.gradle`.
It introduces module conventions for instrumentation, internal
components, smoke tests, sub-agents, annotation processors, and
distributable API modules.

The change also normalizes plugin declarations in touched build files so
dd-trace-java.* and core Gradle plugins are declared in plugins {} where
possible.

Gradle documents convention plugins from `buildSrc` or `build-logic` as
the recommended way to share build logic, while script plugin
application via `apply()` is listed as legacy/not recommended:
* https://docs.gradle.org/9.5.1/userguide/sharing_build_logic_between_subprojects.html#sec:sharing_logic_via_convention_plugins
* https://docs.gradle.org/9.5.1/userguide/plugins_intermediate.html#sec:script_plugins
Disable Spotless' Groovy Java exclusion after the local targets are declared.

This preserves the previous scoped formatting behavior now that scala is applied from the plugins block.
Keep the instrumentation aggregator non-Java while restoring archive conventions and the root muzzle report task used by CI.
Keep the module-kind hook, but leave the Java-based configurations and muzzle report behavior in place for now.
Keep instrumentation modules on the same forbidden API signature set that the
script-plugin ordering used before module conventions. Applying `java.gradle`
from `plugins {}` otherwise adds the global `main` filter to these tasks.

Checked with
* `:dd-java-agent:instrumentation:liberty:liberty-20.0:forbiddenApisMain --info`
* `:dd-java-agent:instrumentation:java:java-lang:java-lang-1.8:forbiddenApisMain --info`
* `:components:context:forbiddenApisMain --info`
* ...
on `master` and this branch. On `master` the tasks only reads
`gradle/forbiddenApiFilters/instrumentation.txt` on instrumantation
modules, while the modification in this branch, before this commit,
read `main.txt` + `instrumentation.txt`.

The ordering issue is subtle: the old append on `master` happened before
`forbiddenapis.gradle` installed `main.txt` on the extension.
The `forbidden-apis` 3.10 plugin uses convention mapping rather than a
live Provider-backed task property, so that early task actually read
captured the "then-empty" default when adding `instruentation.txt`
and the later extension assignment did not appended `main.txt` into the
task value.

The purpose of this branch is to keep the convention-plugin migration
behavior-neutral. A follow-up PR should restore the intended rule set,
i.e. `main.txt` + `instrumentation.txt`, because the current mechanism
is broken and allowed some forbidden APIs only covered by `main.txt`
to leak into instrumentation modules.
Apply the shadow plugin before the distributable API convention so `publish.gradle` sees the same plugin state it saw on `master`. This keeps `dd-trace-ot` on the shadow publication path and avoids publishing metadata that points at internal components.
@bric3 bric3 added comp: tooling Build & Tooling tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: refactoring labels Jun 10, 2026
@datadog-datadog-prod-us1

This comment has been minimized.

@bric3 bric3 marked this pull request as ready for review June 10, 2026 15:53
@bric3 bric3 requested review from a team as code owners June 10, 2026 15:53
@bric3 bric3 requested review from dd-oleksii and leoromanovsky and removed request for a team June 10, 2026 15:53
@bric3 bric3 requested review from amarziali, claponcet, dudikeleti, jandro996, mcculls and ygree and removed request for a team June 10, 2026 15:53
@bric3 bric3 marked this pull request as draft June 10, 2026 17:47
@bric3

bric3 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Going back to draft, as some sub-agent modules convention are just not right.

Rename the `sub-agent` module kind to `agent-product` and reserve it for
shipped agent products.
Add `bootstrap-component` and `testing-support` module kinds.
@bric3 bric3 force-pushed the bdu/introduce-first-module-conventions branch from 7c6df25 to b5b937b Compare June 11, 2026 13:41
@bric3 bric3 marked this pull request as ready for review June 11, 2026 13:47
@pr-commenter

pr-commenter Bot commented Jun 11, 2026

Copy link
Copy Markdown

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1781186098 1781186445
end_time 2026-06-11T13:56:24 2026-06-11T14:02:11
git_branch master bdu/introduce-first-module-conventions
git_commit_sha 93c8609 b5b937b
start_time 2026-06-11T13:54:59 2026-06-11T14:00:46
See matching parameters
Baseline Candidate
ci_job_id 1761839404 1761839404
ci_pipeline_id 118156918 118156918
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1781185243 1781185243

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 6 unstable metrics.

See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-50.262µs; +21.433µs] or [-16.480%; +7.028%]
unstable
[-67.482µs; +29.179µs] or [-19.270%; +8.332%]
unstable
[-86.859µs; +37.137µs] or [-23.456%; +10.029%]
unstable
[-237.936µs; +388.292µs] or [-17.998%; +29.372%]
same
scenario:basic same same same unstable
[+139.853µs; +389.890µs] or [+13.191%; +36.775%]
unstable
[-86.729op/s; +208.680op/s] or [-3.556%; +8.556%]
scenario:loop same unsure
[+2.908µs; +14.048µs] or [+0.032%; +0.156%]
same same same
Request duration reports for reports
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (350.188 µs) : 290, 410
.   : milestone, 350,
basic (300.036 µs) : 292, 308
.   : milestone, 300,
loop (8.985 ms) : 8980, 8991
.   : milestone, 8985,
section candidate
noprobe (331.036 µs) : 307, 355
.   : milestone, 331,
basic (296.931 µs) : 291, 303
.   : milestone, 297,
loop (8.994 ms) : 8989, 8998
.   : milestone, 8994,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 350.188 µs [289.876 µs, 410.499 µs]
basic 300.036 µs [292.112 µs, 307.961 µs]
loop 8.985 ms [8.98 ms, 8.991 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 331.036 µs [306.866 µs, 355.206 µs]
basic 296.931 µs [290.64 µs, 303.222 µs]
loop 8.994 ms [8.989 ms, 8.998 ms]

@dd-octo-sts

dd-octo-sts Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.00 s 13.98 s [-0.7%; +1.0%] (no difference)
startup:insecure-bank:tracing:Agent 12.91 s 12.95 s [-0.9%; +0.4%] (no difference)
startup:petclinic:appsec:Agent 16.18 s 15.99 s [-4.8%; +7.2%] (unstable)
startup:petclinic:iast:Agent 16.77 s 16.84 s [-1.4%; +0.6%] (no difference)
startup:petclinic:profiling:Agent 16.65 s 16.82 s [-1.9%; -0.1%] (maybe better)
startup:petclinic:sca:Agent 16.86 s 16.62 s [+0.5%; +2.3%] (maybe worse)
startup:petclinic:tracing:Agent 15.95 s 16.02 s [-1.4%; +0.5%] (no difference)

Commit: b5b937b3 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

Comment on lines -47 to +50
signaturesFiles += subProj.files("$rootDir/gradle/forbiddenApiFilters/instrumentation.txt")
}
// Preserve the historical instrumentation-only forbiddenApi rules.
subProj.tasks.withType(CheckForbiddenApis).configureEach {
signaturesFiles = subProj.files("$rootDir/gradle/forbiddenApiFilters/instrumentation.txt")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: This will be fixed by #11623 (at this time it needs to explicitly include main.text), I'll rebase this PR once it's merged.

@pr-commenter

pr-commenter Bot commented Jun 11, 2026

Copy link
Copy Markdown

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bdu/introduce-first-module-conventions
git_commit_date 1781088314 1781185243
git_commit_sha 640b115 b5b937b
See matching parameters
Baseline Candidate
ci_job_date 1781186601 1781186601
ci_job_id 1761839393 1761839393
ci_pipeline_id 118156918 118156918
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter

pr-commenter Bot commented Jun 11, 2026

Copy link
Copy Markdown

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bdu/introduce-first-module-conventions
git_commit_date 1781088314 1781185243
git_commit_sha 640b115 b5b937b
See matching parameters
Baseline Candidate
ci_job_date 1781186661 1781186661
ci_job_id 1761839399 1761839399
ci_pipeline_id 118156918 118156918
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: tooling Build & Tooling tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant