Skip to content

Update JUnit Tests to run with STRICT_TEST mode by default#11045

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 7 commits intomasterfrom
mhlidd/junit_strict_mode
Apr 6, 2026
Merged

Update JUnit Tests to run with STRICT_TEST mode by default#11045
gh-worker-dd-mergequeue-cf854d[bot] merged 7 commits intomasterfrom
mhlidd/junit_strict_mode

Conversation

@mhlidd
Copy link
Copy Markdown
Contributor

@mhlidd mhlidd commented Apr 2, 2026

What Does This Do

This PR updates JUnit tests to run with STRICT_TEST by default. This means that all Environment Variables that are queried but not documented will fail the test. We fail the test by throwing an error when querying such environment variable, but if the test has automatic error handling we also throw an error during test cleanup to log the missing configurations from supported-configurations.json.

If a test is using fake configurations, the test itself can override the strictness mode with TEST to bypass this mechanism.

In order to accomplish this, an extension for Config Inversion was created to implement Strict STRICT_TEST by default. This is added to the dd-trace-java.configure-tests Gradle task to automatically load the extension into all JUnit tests.

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

@pr-commenter
Copy link
Copy Markdown

pr-commenter bot commented Apr 3, 2026

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/junit_strict_mode
git_commit_date 1775497568 1775500878
git_commit_sha b664cc0 04a0332
release_version 1.61.0-SNAPSHOT~b664cc0f2a 1.61.0-SNAPSHOT~04a0332999
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1775502675 1775502675
ci_job_id 1570722664 1570722664
ci_pipeline_id 106247415 106247415
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-rl9eq22n 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-rl9eq22n 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 61 metrics, 10 unstable metrics.

Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~04a0332999, baseline=1.61.0-SNAPSHOT~b664cc0f2a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1058896
Total [baseline] (8.834 s) : 0, 8833571
Agent [candidate] (1.054 s) : 0, 1053509
Total [candidate] (8.846 s) : 0, 8846213
section iast
Agent [baseline] (1.222 s) : 0, 1221941
Total [baseline] (9.54 s) : 0, 9539872
Agent [candidate] (1.229 s) : 0, 1229274
Total [candidate] (9.545 s) : 0, 9544737
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.059 s -
Agent iast 1.222 s 163.045 ms (15.4%)
Total tracing 8.834 s -
Total iast 9.54 s 706.301 ms (8.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.054 s -
Agent iast 1.229 s 175.764 ms (16.7%)
Total tracing 8.846 s -
Total iast 9.545 s 698.524 ms (7.9%)
gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~04a0332999, baseline=1.61.0-SNAPSHOT~b664cc0f2a

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.242 ms) : 0, 1242
crashtracking [candidate] (1.221 ms) : 0, 1221
BytebuddyAgent [baseline] (634.851 ms) : 0, 634851
BytebuddyAgent [candidate] (631.294 ms) : 0, 631294
AgentMeter [baseline] (29.407 ms) : 0, 29407
AgentMeter [candidate] (29.337 ms) : 0, 29337
GlobalTracer [baseline] (249.42 ms) : 0, 249420
GlobalTracer [candidate] (248.694 ms) : 0, 248694
AppSec [baseline] (32.191 ms) : 0, 32191
AppSec [candidate] (31.915 ms) : 0, 31915
Debugger [baseline] (59.557 ms) : 0, 59557
Debugger [candidate] (58.961 ms) : 0, 58961
Remote Config [baseline] (606.046 µs) : 0, 606
Remote Config [candidate] (595.975 µs) : 0, 596
Telemetry [baseline] (8.119 ms) : 0, 8119
Telemetry [candidate] (7.986 ms) : 0, 7986
Flare Poller [baseline] (7.281 ms) : 0, 7281
Flare Poller [candidate] (7.366 ms) : 0, 7366
section iast
crashtracking [baseline] (1.221 ms) : 0, 1221
crashtracking [candidate] (1.233 ms) : 0, 1233
BytebuddyAgent [baseline] (799.457 ms) : 0, 799457
BytebuddyAgent [candidate] (805.523 ms) : 0, 805523
AgentMeter [baseline] (11.39 ms) : 0, 11390
AgentMeter [candidate] (11.645 ms) : 0, 11645
GlobalTracer [baseline] (238.981 ms) : 0, 238981
GlobalTracer [candidate] (240.117 ms) : 0, 240117
IAST [baseline] (25.875 ms) : 0, 25875
IAST [candidate] (25.99 ms) : 0, 25990
AppSec [baseline] (29.495 ms) : 0, 29495
AppSec [candidate] (34.283 ms) : 0, 34283
Debugger [baseline] (60.246 ms) : 0, 60246
Debugger [candidate] (55.058 ms) : 0, 55058
Remote Config [baseline] (537.26 µs) : 0, 537
Remote Config [candidate] (528.283 µs) : 0, 528
Telemetry [baseline] (13.833 ms) : 0, 13833
Telemetry [candidate] (13.892 ms) : 0, 13892
Flare Poller [baseline] (4.517 ms) : 0, 4517
Flare Poller [candidate] (4.716 ms) : 0, 4716
Loading
Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~04a0332999, baseline=1.61.0-SNAPSHOT~b664cc0f2a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1063034
Total [baseline] (11.022 s) : 0, 11022154
Agent [candidate] (1.067 s) : 0, 1066669
Total [candidate] (11.07 s) : 0, 11069709
section appsec
Agent [baseline] (1.248 s) : 0, 1248259
Total [baseline] (11.035 s) : 0, 11035372
Agent [candidate] (1.254 s) : 0, 1254479
Total [candidate] (11.165 s) : 0, 11165128
section iast
Agent [baseline] (1.231 s) : 0, 1231269
Total [baseline] (11.345 s) : 0, 11345158
Agent [candidate] (1.223 s) : 0, 1222934
Total [candidate] (11.349 s) : 0, 11349016
section profiling
Agent [baseline] (1.183 s) : 0, 1183191
Total [baseline] (11.098 s) : 0, 11098346
Agent [candidate] (1.189 s) : 0, 1188516
Total [candidate] (11.118 s) : 0, 11118158
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent appsec 1.248 s 185.224 ms (17.4%)
Agent iast 1.231 s 168.234 ms (15.8%)
Agent profiling 1.183 s 120.157 ms (11.3%)
Total tracing 11.022 s -
Total appsec 11.035 s 13.218 ms (0.1%)
Total iast 11.345 s 323.004 ms (2.9%)
Total profiling 11.098 s 76.192 ms (0.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.067 s -
Agent appsec 1.254 s 187.809 ms (17.6%)
Agent iast 1.223 s 156.265 ms (14.6%)
Agent profiling 1.189 s 121.847 ms (11.4%)
Total tracing 11.07 s -
Total appsec 11.165 s 95.419 ms (0.9%)
Total iast 11.349 s 279.307 ms (2.5%)
Total profiling 11.118 s 48.449 ms (0.4%)
gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~04a0332999, baseline=1.61.0-SNAPSHOT~b664cc0f2a

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.247 ms) : 0, 1247
crashtracking [candidate] (1.245 ms) : 0, 1245
BytebuddyAgent [baseline] (637.008 ms) : 0, 637008
BytebuddyAgent [candidate] (641.278 ms) : 0, 641278
AgentMeter [baseline] (29.633 ms) : 0, 29633
AgentMeter [candidate] (29.83 ms) : 0, 29830
GlobalTracer [baseline] (250.118 ms) : 0, 250118
GlobalTracer [candidate] (249.543 ms) : 0, 249543
AppSec [baseline] (32.253 ms) : 0, 32253
AppSec [candidate] (31.953 ms) : 0, 31953
Debugger [baseline] (60.282 ms) : 0, 60282
Debugger [candidate] (59.857 ms) : 0, 59857
Remote Config [baseline] (603.127 µs) : 0, 603
Remote Config [candidate] (596.384 µs) : 0, 596
Telemetry [baseline] (8.102 ms) : 0, 8102
Telemetry [candidate] (8.062 ms) : 0, 8062
Flare Poller [baseline] (7.455 ms) : 0, 7455
Flare Poller [candidate] (8.098 ms) : 0, 8098
section appsec
crashtracking [baseline] (1.216 ms) : 0, 1216
crashtracking [candidate] (1.221 ms) : 0, 1221
BytebuddyAgent [baseline] (661.92 ms) : 0, 661920
BytebuddyAgent [candidate] (665.658 ms) : 0, 665658
AgentMeter [baseline] (12.071 ms) : 0, 12071
AgentMeter [candidate] (12.115 ms) : 0, 12115
GlobalTracer [baseline] (249.183 ms) : 0, 249183
GlobalTracer [candidate] (250.426 ms) : 0, 250426
IAST [baseline] (24.564 ms) : 0, 24564
IAST [candidate] (24.788 ms) : 0, 24788
AppSec [baseline] (183.981 ms) : 0, 183981
AppSec [candidate] (184.533 ms) : 0, 184533
Debugger [baseline] (66.195 ms) : 0, 66195
Debugger [candidate] (66.511 ms) : 0, 66511
Remote Config [baseline] (623.23 µs) : 0, 623
Remote Config [candidate] (604.892 µs) : 0, 605
Telemetry [baseline] (8.741 ms) : 0, 8741
Telemetry [candidate] (8.693 ms) : 0, 8693
Flare Poller [baseline] (3.565 ms) : 0, 3565
Flare Poller [candidate] (3.558 ms) : 0, 3558
section iast
crashtracking [baseline] (1.222 ms) : 0, 1222
crashtracking [candidate] (1.222 ms) : 0, 1222
BytebuddyAgent [baseline] (806.264 ms) : 0, 806264
BytebuddyAgent [candidate] (800.314 ms) : 0, 800314
AgentMeter [baseline] (11.448 ms) : 0, 11448
AgentMeter [candidate] (11.383 ms) : 0, 11383
GlobalTracer [baseline] (240.374 ms) : 0, 240374
GlobalTracer [candidate] (238.512 ms) : 0, 238512
IAST [baseline] (26.103 ms) : 0, 26103
IAST [candidate] (25.842 ms) : 0, 25842
AppSec [baseline] (32.65 ms) : 0, 32650
AppSec [candidate] (31.875 ms) : 0, 31875
Debugger [baseline] (57.665 ms) : 0, 57665
Debugger [candidate] (58.366 ms) : 0, 58366
Remote Config [baseline] (556.3 µs) : 0, 556
Remote Config [candidate] (518.636 µs) : 0, 519
Telemetry [baseline] (14.489 ms) : 0, 14489
Telemetry [candidate] (14.075 ms) : 0, 14075
Flare Poller [baseline] (4.393 ms) : 0, 4393
Flare Poller [candidate] (4.551 ms) : 0, 4551
section profiling
crashtracking [baseline] (1.188 ms) : 0, 1188
crashtracking [candidate] (1.174 ms) : 0, 1174
BytebuddyAgent [baseline] (691.327 ms) : 0, 691327
BytebuddyAgent [candidate] (693.69 ms) : 0, 693690
AgentMeter [baseline] (9.08 ms) : 0, 9080
AgentMeter [candidate] (9.166 ms) : 0, 9166
GlobalTracer [baseline] (206.912 ms) : 0, 206912
GlobalTracer [candidate] (208.375 ms) : 0, 208375
AppSec [baseline] (32.607 ms) : 0, 32607
AppSec [candidate] (32.845 ms) : 0, 32845
Debugger [baseline] (65.68 ms) : 0, 65680
Debugger [candidate] (65.975 ms) : 0, 65975
Remote Config [baseline] (568.948 µs) : 0, 569
Remote Config [candidate] (565.531 µs) : 0, 566
Telemetry [baseline] (7.821 ms) : 0, 7821
Telemetry [candidate] (7.864 ms) : 0, 7864
Flare Poller [baseline] (3.568 ms) : 0, 3568
Flare Poller [candidate] (3.55 ms) : 0, 3550
ProfilingAgent [baseline] (93.33 ms) : 0, 93330
ProfilingAgent [candidate] (94.07 ms) : 0, 94070
Profiling [baseline] (93.9 ms) : 0, 93900
Profiling [candidate] (94.621 ms) : 0, 94621
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/junit_strict_mode
git_commit_date 1775497568 1775500878
git_commit_sha b664cc0 04a0332
release_version 1.61.0-SNAPSHOT~b664cc0f2a 1.61.0-SNAPSHOT~04a0332999
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1775503156 1775503156
ci_job_id 1570722666 1570722666
ci_pipeline_id 106247415 106247415
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-6kvfuoj1 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-6kvfuoj1 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 6 performance regressions! Performance is the same for 15 metrics, 15 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast:high_load worse
[+195.955µs; +346.742µs] or [+7.852%; +13.894%]
worse
[+371.130µs; +866.385µs] or [+5.067%; +11.829%]
unstable
[-262.732op/s; +11.295op/s] or [-18.408%; +0.791%]
2.767ms 7.943ms 1301.531op/s 2.496ms 7.324ms 1427.250op/s
scenario:load:insecure-bank:iast_FULL:high_load worse
[+222.570µs; +416.844µs] or [+4.330%; +8.109%]
worse
[+559.992µs; +1247.163µs] or [+4.572%; +10.181%]
unstable
[-120.086op/s; +28.586op/s] or [-15.045%; +3.581%]
5.460ms 13.153ms 752.438op/s 5.140ms 12.249ms 798.188op/s
scenario:load:petclinic:code_origins:high_load worse
[+0.832ms; +1.706ms] or [+4.748%; +9.732%]
worse
[+0.850ms; +2.808ms] or [+2.974%; +9.820%]
unstable
[-43.704op/s; +11.579op/s] or [-16.787%; +4.448%]
18.799ms 30.422ms 244.281op/s 17.530ms 28.593ms 260.344op/s
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~04a0332999, baseline=1.61.0-SNAPSHOT~b664cc0f2a
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.393 ms) : 17222, 17565
.   : milestone, 17393,
appsec (18.627 ms) : 18441, 18813
.   : milestone, 18627,
code_origins (17.923 ms) : 17744, 18102
.   : milestone, 17923,
iast (18.804 ms) : 18615, 18994
.   : milestone, 18804,
profiling (18.879 ms) : 18689, 19069
.   : milestone, 18879,
tracing (17.713 ms) : 17536, 17890
.   : milestone, 17713,
section candidate
no_agent (18.158 ms) : 17976, 18340
.   : milestone, 18158,
appsec (18.646 ms) : 18461, 18830
.   : milestone, 18646,
code_origins (19.107 ms) : 18913, 19302
.   : milestone, 19107,
iast (18.088 ms) : 17906, 18270
.   : milestone, 18088,
profiling (19.054 ms) : 18861, 19246
.   : milestone, 19054,
tracing (17.817 ms) : 17644, 17990
.   : milestone, 17817,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 17.393 ms [17.222 ms, 17.565 ms] -
appsec 18.627 ms [18.441 ms, 18.813 ms] 1.234 ms (7.1%)
code_origins 17.923 ms [17.744 ms, 18.102 ms] 529.529 µs (3.0%)
iast 18.804 ms [18.615 ms, 18.994 ms] 1.411 ms (8.1%)
profiling 18.879 ms [18.689 ms, 19.069 ms] 1.485 ms (8.5%)
tracing 17.713 ms [17.536 ms, 17.89 ms] 319.927 µs (1.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.158 ms [17.976 ms, 18.34 ms] -
appsec 18.646 ms [18.461 ms, 18.83 ms] 488.074 µs (2.7%)
code_origins 19.107 ms [18.913 ms, 19.302 ms] 949.69 µs (5.2%)
iast 18.088 ms [17.906 ms, 18.27 ms] -70.068 µs (-0.4%)
profiling 19.054 ms [18.861 ms, 19.246 ms] 895.984 µs (4.9%)
tracing 17.817 ms [17.644 ms, 17.99 ms] -340.929 µs (-1.9%)
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~04a0332999, baseline=1.61.0-SNAPSHOT~b664cc0f2a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.256 ms) : 1243, 1268
.   : milestone, 1256,
iast (3.205 ms) : 3162, 3248
.   : milestone, 3205,
iast_FULL (5.792 ms) : 5734, 5850
.   : milestone, 5792,
iast_GLOBAL (3.695 ms) : 3634, 3757
.   : milestone, 3695,
profiling (2.119 ms) : 2100, 2137
.   : milestone, 2119,
tracing (1.871 ms) : 1855, 1886
.   : milestone, 1871,
section candidate
no_agent (1.26 ms) : 1249, 1272
.   : milestone, 1260,
iast (3.521 ms) : 3468, 3574
.   : milestone, 3521,
iast_FULL (6.149 ms) : 6085, 6213
.   : milestone, 6149,
iast_GLOBAL (3.57 ms) : 3511, 3630
.   : milestone, 3570,
profiling (2.274 ms) : 2253, 2296
.   : milestone, 2274,
tracing (1.883 ms) : 1868, 1898
.   : milestone, 1883,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.256 ms [1.243 ms, 1.268 ms] -
iast 3.205 ms [3.162 ms, 3.248 ms] 1.95 ms (155.3%)
iast_FULL 5.792 ms [5.734 ms, 5.85 ms] 4.536 ms (361.3%)
iast_GLOBAL 3.695 ms [3.634 ms, 3.757 ms] 2.44 ms (194.3%)
profiling 2.119 ms [2.1 ms, 2.137 ms] 863.187 µs (68.7%)
tracing 1.871 ms [1.855 ms, 1.886 ms] 615.002 µs (49.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.26 ms [1.249 ms, 1.272 ms] -
iast 3.521 ms [3.468 ms, 3.574 ms] 2.261 ms (179.3%)
iast_FULL 6.149 ms [6.085 ms, 6.213 ms] 4.888 ms (387.8%)
iast_GLOBAL 3.57 ms [3.511 ms, 3.63 ms] 2.31 ms (183.3%)
profiling 2.274 ms [2.253 ms, 2.296 ms] 1.014 ms (80.4%)
tracing 1.883 ms [1.868 ms, 1.898 ms] 622.258 µs (49.4%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/junit_strict_mode
git_commit_date 1775497568 1775500878
git_commit_sha b664cc0 04a0332
release_version 1.61.0-SNAPSHOT~b664cc0f2a 1.61.0-SNAPSHOT~04a0332999
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1775502908 1775502908
ci_job_id 1570722670 1570722670
ci_pipeline_id 106247415 106247415
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-3-5nfx4ipj 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-3-5nfx4ipj 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava
gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~04a0332999, baseline=1.61.0-SNAPSHOT~b664cc0f2a
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.96 s) : 14960000, 14960000
.   : milestone, 14960000,
appsec (14.939 s) : 14939000, 14939000
.   : milestone, 14939000,
iast (18.298 s) : 18298000, 18298000
.   : milestone, 18298000,
iast_GLOBAL (18.07 s) : 18070000, 18070000
.   : milestone, 18070000,
profiling (15.496 s) : 15496000, 15496000
.   : milestone, 15496000,
tracing (15.04 s) : 15040000, 15040000
.   : milestone, 15040000,
section candidate
no_agent (15.303 s) : 15303000, 15303000
.   : milestone, 15303000,
appsec (14.89 s) : 14890000, 14890000
.   : milestone, 14890000,
iast (18.51 s) : 18510000, 18510000
.   : milestone, 18510000,
iast_GLOBAL (18.469 s) : 18469000, 18469000
.   : milestone, 18469000,
profiling (14.894 s) : 14894000, 14894000
.   : milestone, 14894000,
tracing (15.01 s) : 15010000, 15010000
.   : milestone, 15010000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.96 s [14.96 s, 14.96 s] -
appsec 14.939 s [14.939 s, 14.939 s] -21.0 ms (-0.1%)
iast 18.298 s [18.298 s, 18.298 s] 3.338 s (22.3%)
iast_GLOBAL 18.07 s [18.07 s, 18.07 s] 3.11 s (20.8%)
profiling 15.496 s [15.496 s, 15.496 s] 536.0 ms (3.6%)
tracing 15.04 s [15.04 s, 15.04 s] 80.0 ms (0.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.303 s [15.303 s, 15.303 s] -
appsec 14.89 s [14.89 s, 14.89 s] -413.0 ms (-2.7%)
iast 18.51 s [18.51 s, 18.51 s] 3.207 s (21.0%)
iast_GLOBAL 18.469 s [18.469 s, 18.469 s] 3.166 s (20.7%)
profiling 14.894 s [14.894 s, 14.894 s] -409.0 ms (-2.7%)
tracing 15.01 s [15.01 s, 15.01 s] -293.0 ms (-1.9%)
Execution time for tomcat
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~04a0332999, baseline=1.61.0-SNAPSHOT~b664cc0f2a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.482 ms) : 1471, 1494
.   : milestone, 1482,
appsec (3.802 ms) : 3580, 4024
.   : milestone, 3802,
iast (2.268 ms) : 2199, 2337
.   : milestone, 2268,
iast_GLOBAL (2.314 ms) : 2244, 2384
.   : milestone, 2314,
profiling (2.12 ms) : 2063, 2176
.   : milestone, 2120,
tracing (2.092 ms) : 2038, 2147
.   : milestone, 2092,
section candidate
no_agent (1.481 ms) : 1470, 1493
.   : milestone, 1481,
appsec (3.832 ms) : 3610, 4054
.   : milestone, 3832,
iast (2.268 ms) : 2199, 2337
.   : milestone, 2268,
iast_GLOBAL (2.305 ms) : 2236, 2375
.   : milestone, 2305,
profiling (2.117 ms) : 2060, 2175
.   : milestone, 2117,
tracing (2.089 ms) : 2035, 2144
.   : milestone, 2089,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.482 ms [1.471 ms, 1.494 ms] -
appsec 3.802 ms [3.58 ms, 4.024 ms] 2.319 ms (156.5%)
iast 2.268 ms [2.199 ms, 2.337 ms] 785.71 µs (53.0%)
iast_GLOBAL 2.314 ms [2.244 ms, 2.384 ms] 831.41 µs (56.1%)
profiling 2.12 ms [2.063 ms, 2.176 ms] 637.061 µs (43.0%)
tracing 2.092 ms [2.038 ms, 2.147 ms] 609.708 µs (41.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.481 ms [1.47 ms, 1.493 ms] -
appsec 3.832 ms [3.61 ms, 4.054 ms] 2.35 ms (158.7%)
iast 2.268 ms [2.199 ms, 2.337 ms] 786.988 µs (53.1%)
iast_GLOBAL 2.305 ms [2.236 ms, 2.375 ms] 824.26 µs (55.7%)
profiling 2.117 ms [2.06 ms, 2.175 ms] 636.419 µs (43.0%)
tracing 2.089 ms [2.035 ms, 2.144 ms] 608.212 µs (41.1%)

@mhlidd mhlidd added type: enhancement Enhancements and improvements comp: testing Testing comp: config Configuration labels Apr 6, 2026
@mhlidd mhlidd changed the title Update JUnit Tests to run with STRICT_TEST mode by default Update JUnit Tests to run with STRICT_TEST mode by default Apr 6, 2026
@mhlidd mhlidd force-pushed the mhlidd/junit_strict_mode branch 2 times, most recently from 8e46da5 to 929d1c2 Compare April 6, 2026 18:41
@mhlidd mhlidd marked this pull request as ready for review April 6, 2026 18:41
@mhlidd mhlidd requested a review from a team as a code owner April 6, 2026 18:41
@mhlidd mhlidd requested a review from dougqh April 6, 2026 18:41
@mhlidd mhlidd added this pull request to the merge queue Apr 6, 2026
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts bot commented Apr 6, 2026

/merge

@gh-worker-devflow-routing-ef8351
Copy link
Copy Markdown

gh-worker-devflow-routing-ef8351 bot commented Apr 6, 2026

View all feedbacks in Devflow UI.

2026-04-06 19:35:45 UTC ℹ️ Start processing command /merge


2026-04-06 19:35:49 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).


2026-04-06 20:27:22 UTC ℹ️ MergeQueue: Readding this merge request to the queue because another merge request processed with yours failed. No action is needed from your side.


2026-04-06 21:38:51 UTC ℹ️ MergeQueue: This merge request was merged

@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Apr 6, 2026
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot merged commit 0d8066b into master Apr 6, 2026
567 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot deleted the mhlidd/junit_strict_mode branch April 6, 2026 21:38
@github-actions github-actions bot added this to the 1.61.0 milestone Apr 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: config Configuration comp: testing Testing type: enhancement Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants