Skip to content

Fix log injection smoke test flakiness under CI load#10999

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit intomasterfrom
brian.marks/broaden-log-injection-flaky-jdk8
Mar 30, 2026
Merged

Fix log injection smoke test flakiness under CI load#10999
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit intomasterfrom
brian.marks/broaden-log-injection-flaky-jdk8

Conversation

@bm1549
Copy link
Copy Markdown
Contributor

@bm1549 bm1549 commented Mar 30, 2026 •
edited
Loading

What Does This Do

Increases BaseApplication.TIMEOUT_IN_NANOS from 10s to 30s to align with the test's PollingConditions timeout.

Motivation

This is the only smoke test with a bidirectional handshake — the app blocks waiting for remote config changes from the test, and the test blocks waiting for traces from the app. Every other smoke test is one-way. The app's waitForCondition crash-on-timeout exists nowhere else in the smoke test suite.

The two sides had misaligned timeouts: the test waits up to 30s for traces, but the app only waits 10s for config. Under CI load, the trace writer's first flush (including feature discovery) can exceed 10s. When the app's timeout fires first, it crashes the process, killing the daemon trace-writer thread before it delivers traces — so traceCount stays at 0.

Aligning both timeouts to 30s ensures the test's polling condition is always the binding constraint. The app never independently crashes, the daemon writer stays alive, and traces are always delivered.

Observed across all JDK versions (8, 11, 17, 21, 25) — 17 failures in the last 30 days. The @Flaky annotation only covered IBM 8 and Oracle JDK 8.

Reproduction

Simulated slow trace delivery locally by increasing dd.trace.flush.interval:

Old code (10s app timeout) New code (30s app timeout)
flush=9.5s FAIL PASS
flush=10.5s FAIL PASS

Additional Notes

  • The @Flaky annotation is kept as a safety net until CI confirms the fix
  • CI failure data: pup cicd tests search --query 'test_level:test @git.branch:master @test.name:"check raw file injection" @test.final_status:fail' --from 30d

Jira ticket: N/A — test infrastructure fix

🤖 Generated with Claude Code

@bm1549 bm1549 added type: bug Bug report and fix comp: testing Testing tag: no release notes Changes to exclude from release notes tag: ai generated Largely based on code generated by an AI or LLM labels Mar 30, 2026
@pr-commenter
Copy link
Copy Markdown

pr-commenter bot commented Mar 30, 2026 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master brian.marks/broaden-log-injection-flaky-jdk8
git_commit_date 1774872048 1774892069
git_commit_sha 92b8b82 1c9c252
release_version 1.61.0-SNAPSHOT~92b8b827eb 1.61.0-SNAPSHOT~1c9c252b82
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1774893855 1774893855
ci_job_id 1551428967 1551428967
ci_pipeline_id 105095731 105095731
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-008qj684 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-008qj684 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 63 metrics, 8 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~1c9c252b82, baseline=1.61.0-SNAPSHOT~92b8b827eb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1063750
Total [baseline] (11.008 s) : 0, 11008048
Agent [candidate] (1.06 s) : 0, 1059570
Total [candidate] (11.07 s) : 0, 11070356
section appsec
Agent [baseline] (1.257 s) : 0, 1256964
Total [baseline] (11.187 s) : 0, 11187479
Agent [candidate] (1.254 s) : 0, 1253918
Total [candidate] (11.14 s) : 0, 11140108
section iast
Agent [baseline] (1.232 s) : 0, 1232248
Total [baseline] (11.356 s) : 0, 11356300
Agent [candidate] (1.229 s) : 0, 1229031
Total [candidate] (11.31 s) : 0, 11309531
section profiling
Agent [baseline] (1.187 s) : 0, 1186912
Total [baseline] (11.016 s) : 0, 11016357
Agent [candidate] (1.187 s) : 0, 1186966
Total [candidate] (11.055 s) : 0, 11055076
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent appsec 1.257 s 193.214 ms (18.2%)
Agent iast 1.232 s 168.498 ms (15.8%)
Agent profiling 1.187 s 123.163 ms (11.6%)
Total tracing 11.008 s -
Total appsec 11.187 s 179.431 ms (1.6%)
Total iast 11.356 s 348.252 ms (3.2%)
Total profiling 11.016 s 8.309 ms (0.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.06 s -
Agent appsec 1.254 s 194.348 ms (18.3%)
Agent iast 1.229 s 169.461 ms (16.0%)
Agent profiling 1.187 s 127.395 ms (12.0%)
Total tracing 11.07 s -
Total appsec 11.14 s 69.752 ms (0.6%)
Total iast 11.31 s 239.175 ms (2.2%)
Total profiling 11.055 s -15.28 ms (-0.1%) 
gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~1c9c252b82, baseline=1.61.0-SNAPSHOT~92b8b827eb

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.216 ms) : 0, 1216
crashtracking [candidate] (1.218 ms) : 0, 1218
BytebuddyAgent [baseline] (631.58 ms) : 0, 631580
BytebuddyAgent [candidate] (630.323 ms) : 0, 630323
AgentMeter [baseline] (29.438 ms) : 0, 29438
AgentMeter [candidate] (29.326 ms) : 0, 29326
GlobalTracer [baseline] (257.857 ms) : 0, 257857
GlobalTracer [candidate] (257.363 ms) : 0, 257363
AppSec [baseline] (31.902 ms) : 0, 31902
AppSec [candidate] (31.776 ms) : 0, 31776
Debugger [baseline] (60.431 ms) : 0, 60431
Debugger [candidate] (60.525 ms) : 0, 60525
Remote Config [baseline] (600.344 µs) : 0, 600
Remote Config [candidate] (593.39 µs) : 0, 593
Telemetry [baseline] (8.102 ms) : 0, 8102
Telemetry [candidate] (8.053 ms) : 0, 8053
Flare Poller [baseline] (6.399 ms) : 0, 6399
Flare Poller [candidate] (4.297 ms) : 0, 4297
section appsec
crashtracking [baseline] (1.238 ms) : 0, 1238
crashtracking [candidate] (1.227 ms) : 0, 1227
BytebuddyAgent [baseline] (665.344 ms) : 0, 665344
BytebuddyAgent [candidate] (661.83 ms) : 0, 661830
AgentMeter [baseline] (12.22 ms) : 0, 12220
AgentMeter [candidate] (12.202 ms) : 0, 12202
GlobalTracer [baseline] (259.433 ms) : 0, 259433
GlobalTracer [candidate] (259.182 ms) : 0, 259182
AppSec [baseline] (178.393 ms) : 0, 178393
AppSec [candidate] (179.011 ms) : 0, 179011
Debugger [baseline] (67.003 ms) : 0, 67003
Debugger [candidate] (66.871 ms) : 0, 66871
Remote Config [baseline] (639.404 µs) : 0, 639
Remote Config [candidate] (641.638 µs) : 0, 642
Telemetry [baseline] (8.331 ms) : 0, 8331
Telemetry [candidate] (8.372 ms) : 0, 8372
Flare Poller [baseline] (3.601 ms) : 0, 3601
Flare Poller [candidate] (3.607 ms) : 0, 3607
IAST [baseline] (24.243 ms) : 0, 24243
IAST [candidate] (24.422 ms) : 0, 24422
section iast
crashtracking [baseline] (1.236 ms) : 0, 1236
crashtracking [candidate] (1.233 ms) : 0, 1233
BytebuddyAgent [baseline] (799.658 ms) : 0, 799658
BytebuddyAgent [candidate] (798.095 ms) : 0, 798095
AgentMeter [baseline] (11.418 ms) : 0, 11418
AgentMeter [candidate] (11.374 ms) : 0, 11374
GlobalTracer [baseline] (247.843 ms) : 0, 247843
GlobalTracer [candidate] (247.316 ms) : 0, 247316
AppSec [baseline] (26.534 ms) : 0, 26534
AppSec [candidate] (26.483 ms) : 0, 26483
Debugger [baseline] (69.39 ms) : 0, 69390
Debugger [candidate] (68.735 ms) : 0, 68735
Remote Config [baseline] (531.348 µs) : 0, 531
Remote Config [candidate] (524.338 µs) : 0, 524
Telemetry [baseline] (10.204 ms) : 0, 10204
Telemetry [candidate] (10.162 ms) : 0, 10162
Flare Poller [baseline] (3.758 ms) : 0, 3758
Flare Poller [candidate] (3.689 ms) : 0, 3689
IAST [baseline] (25.392 ms) : 0, 25392
IAST [candidate] (25.311 ms) : 0, 25311
section profiling
ProfilingAgent [baseline] (94.247 ms) : 0, 94247
ProfilingAgent [candidate] (94.163 ms) : 0, 94163
crashtracking [baseline] (1.186 ms) : 0, 1186
crashtracking [candidate] (1.172 ms) : 0, 1172
BytebuddyAgent [baseline] (685.426 ms) : 0, 685426
BytebuddyAgent [candidate] (685.322 ms) : 0, 685322
AgentMeter [baseline] (8.988 ms) : 0, 8988
AgentMeter [candidate] (8.967 ms) : 0, 8967
GlobalTracer [baseline] (215.373 ms) : 0, 215373
GlobalTracer [candidate] (215.642 ms) : 0, 215642
AppSec [baseline] (32.354 ms) : 0, 32354
AppSec [candidate] (32.39 ms) : 0, 32390
Debugger [baseline] (66.321 ms) : 0, 66321
Debugger [candidate] (66.367 ms) : 0, 66367
Remote Config [baseline] (573.537 µs) : 0, 574
Remote Config [candidate] (570.26 µs) : 0, 570
Telemetry [baseline] (7.78 ms) : 0, 7780
Telemetry [candidate] (7.733 ms) : 0, 7733
Flare Poller [baseline] (3.517 ms) : 0, 3517
Flare Poller [candidate] (3.542 ms) : 0, 3542
Profiling [baseline] (94.807 ms) : 0, 94807
Profiling [candidate] (94.711 ms) : 0, 94711
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~1c9c252b82, baseline=1.61.0-SNAPSHOT~92b8b827eb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.061 s) : 0, 1061146
Total [baseline] (8.883 s) : 0, 8882929
Agent [candidate] (1.058 s) : 0, 1057857
Total [candidate] (8.857 s) : 0, 8856961
section iast
Agent [baseline] (1.233 s) : 0, 1232991
Total [baseline] (9.57 s) : 0, 9569958
Agent [candidate] (1.248 s) : 0, 1248021
Total [candidate] (9.597 s) : 0, 9597400
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent iast 1.233 s 171.844 ms (16.2%)
Total tracing 8.883 s -
Total iast 9.57 s 687.029 ms (7.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.058 s -
Agent iast 1.248 s 190.163 ms (18.0%)
Total tracing 8.857 s -
Total iast 9.597 s 740.438 ms (8.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~1c9c252b82, baseline=1.61.0-SNAPSHOT~92b8b827eb

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.223 ms) : 0, 1223
crashtracking [candidate] (1.215 ms) : 0, 1215
BytebuddyAgent [baseline] (632.077 ms) : 0, 632077
BytebuddyAgent [candidate] (630.242 ms) : 0, 630242
AgentMeter [baseline] (29.348 ms) : 0, 29348
AgentMeter [candidate] (29.392 ms) : 0, 29392
GlobalTracer [baseline] (257.143 ms) : 0, 257143
GlobalTracer [candidate] (257.436 ms) : 0, 257436
AppSec [baseline] (31.814 ms) : 0, 31814
AppSec [candidate] (31.847 ms) : 0, 31847
Debugger [baseline] (59.736 ms) : 0, 59736
Debugger [candidate] (59.413 ms) : 0, 59413
Remote Config [baseline] (592.589 µs) : 0, 593
Remote Config [candidate] (578.766 µs) : 0, 579
Telemetry [baseline] (8.052 ms) : 0, 8052
Telemetry [candidate] (8.009 ms) : 0, 8009
Flare Poller [baseline] (4.964 ms) : 0, 4964
Flare Poller [candidate] (3.502 ms) : 0, 3502
section iast
crashtracking [baseline] (1.229 ms) : 0, 1229
crashtracking [candidate] (1.234 ms) : 0, 1234
BytebuddyAgent [baseline] (799.07 ms) : 0, 799070
BytebuddyAgent [candidate] (812.791 ms) : 0, 812791
AgentMeter [baseline] (11.422 ms) : 0, 11422
AgentMeter [candidate] (11.856 ms) : 0, 11856
GlobalTracer [baseline] (248.906 ms) : 0, 248906
GlobalTracer [candidate] (249.647 ms) : 0, 249647
AppSec [baseline] (26.834 ms) : 0, 26834
AppSec [candidate] (27.034 ms) : 0, 27034
Debugger [baseline] (68.524 ms) : 0, 68524
Debugger [candidate] (67.422 ms) : 0, 67422
Remote Config [baseline] (562.489 µs) : 0, 562
Remote Config [candidate] (531.444 µs) : 0, 531
Telemetry [baseline] (10.732 ms) : 0, 10732
Telemetry [candidate] (11.153 ms) : 0, 11153
Flare Poller [baseline] (3.877 ms) : 0, 3877
Flare Poller [candidate] (3.96 ms) : 0, 3960
IAST [baseline] (25.525 ms) : 0, 25525
IAST [candidate] (25.78 ms) : 0, 25780
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master brian.marks/broaden-log-injection-flaky-jdk8
git_commit_date 1774872048 1774892069
git_commit_sha 92b8b82 1c9c252
release_version 1.61.0-SNAPSHOT~92b8b827eb 1.61.0-SNAPSHOT~1c9c252b82
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1774894313 1774894313
ci_job_id 1551428968 1551428968
ci_pipeline_id 105095731 105095731
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-e4mev8pe 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-e4mev8pe 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 3 performance regressions! Performance is the same for 16 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load worse
[+67.672µs; +188.876µs] or [+2.399%; +6.695%]		 unsure
[+21.132µs; +468.273µs] or [+0.266%; +5.905%]		 unstable
[-179.364op/s; +73.239op/s] or [-13.901%; +5.676%]		 2.949ms 8.175ms 1237.219op/s 2.821ms 7.930ms 1290.281op/s
scenario:load:petclinic:iast:high_load worse
[+0.784ms; +1.649ms] or [+4.426%; +9.311%]		 worse
[+1.078ms; +2.394ms] or [+3.735%; +8.299%]		 unstable
[-41.143op/s; +11.143op/s] or [-15.941%; +4.318%]		 18.929ms 30.589ms 243.094op/s 17.713ms 28.853ms 258.094op/s
scenario:load:petclinic:no_agent:high_load better
[-3.341ms; -1.558ms] or [-17.272%; -8.053%]		 unstable
[-4.558ms; -0.941ms] or [-14.449%; -2.984%]		 unstable
[+1.997op/s; +58.191op/s] or [+0.837%; +24.399%]		 16.892ms 28.796ms 268.594op/s 19.341ms 31.546ms 238.500op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~1c9c252b82, baseline=1.61.0-SNAPSHOT~92b8b827eb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.245 ms) : 1233, 1258
.   : milestone, 1245,
iast (3.248 ms) : 3202, 3294
.   : milestone, 3248,
iast_FULL (6.102 ms) : 6039, 6165
.   : milestone, 6102,
iast_GLOBAL (3.553 ms) : 3502, 3604
.   : milestone, 3553,
profiling (2.306 ms) : 2280, 2331
.   : milestone, 2306,
tracing (1.971 ms) : 1954, 1988
.   : milestone, 1971,
section candidate
no_agent (1.251 ms) : 1239, 1263
.   : milestone, 1251,
iast (3.24 ms) : 3195, 3285
.   : milestone, 3240,
iast_FULL (6.084 ms) : 6021, 6147
.   : milestone, 6084,
iast_GLOBAL (3.709 ms) : 3647, 3771
.   : milestone, 3709,
profiling (2.25 ms) : 2228, 2272
.   : milestone, 2250,
tracing (1.85 ms) : 1835, 1864
.   : milestone, 1850,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.245 ms [1.233 ms, 1.258 ms] -
iast 3.248 ms [3.202 ms, 3.294 ms] 2.003 ms (160.8%)
iast_FULL 6.102 ms [6.039 ms, 6.165 ms] 4.857 ms (390.0%)
iast_GLOBAL 3.553 ms [3.502 ms, 3.604 ms] 2.308 ms (185.4%)
profiling 2.306 ms [2.28 ms, 2.331 ms] 1.061 ms (85.2%)
tracing 1.971 ms [1.954 ms, 1.988 ms] 725.541 µs (58.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.251 ms [1.239 ms, 1.263 ms] -
iast 3.24 ms [3.195 ms, 3.285 ms] 1.989 ms (159.0%)
iast_FULL 6.084 ms [6.021 ms, 6.147 ms] 4.833 ms (386.3%)
iast_GLOBAL 3.709 ms [3.647 ms, 3.771 ms] 2.458 ms (196.5%)
profiling 2.25 ms [2.228 ms, 2.272 ms] 999.149 µs (79.9%)
tracing 1.85 ms [1.835 ms, 1.864 ms] 598.828 µs (47.9%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~1c9c252b82, baseline=1.61.0-SNAPSHOT~92b8b827eb
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.573 ms) : 19368, 19779
.   : milestone, 19573,
appsec (18.695 ms) : 18506, 18884
.   : milestone, 18695,
code_origins (17.784 ms) : 17610, 17959
.   : milestone, 17784,
iast (18.076 ms) : 17897, 18255
.   : milestone, 18076,
profiling (20.384 ms) : 20175, 20593
.   : milestone, 20384,
tracing (17.726 ms) : 17549, 17902
.   : milestone, 17726,
section candidate
no_agent (17.368 ms) : 17191, 17545
.   : milestone, 17368,
appsec (18.618 ms) : 18430, 18805
.   : milestone, 18618,
code_origins (18.038 ms) : 17861, 18215
.   : milestone, 18038,
iast (19.201 ms) : 19003, 19400
.   : milestone, 19201,
profiling (19.933 ms) : 19730, 20136
.   : milestone, 19933,
tracing (17.851 ms) : 17673, 18029
.   : milestone, 17851,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.573 ms [19.368 ms, 19.779 ms] -
appsec 18.695 ms [18.506 ms, 18.884 ms] -878.458 µs (-4.5%)
code_origins 17.784 ms [17.61 ms, 17.959 ms] -1.789 ms (-9.1%)
iast 18.076 ms [17.897 ms, 18.255 ms] -1.497 ms (-7.6%)
profiling 20.384 ms [20.175 ms, 20.593 ms] 811.054 µs (4.1%)
tracing 17.726 ms [17.549 ms, 17.902 ms] -1.848 ms (-9.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 17.368 ms [17.191 ms, 17.545 ms] -
appsec 18.618 ms [18.43 ms, 18.805 ms] 1.25 ms (7.2%)
code_origins 18.038 ms [17.861 ms, 18.215 ms] 669.964 µs (3.9%)
iast 19.201 ms [19.003 ms, 19.4 ms] 1.834 ms (10.6%)
profiling 19.933 ms [19.73 ms, 20.136 ms] 2.566 ms (14.8%)
tracing 17.851 ms [17.673 ms, 18.029 ms] 483.316 µs (2.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master brian.marks/broaden-log-injection-flaky-jdk8
git_commit_date 1774872048 1774892069
git_commit_sha 92b8b82 1c9c252
release_version 1.61.0-SNAPSHOT~92b8b827eb 1.61.0-SNAPSHOT~1c9c252b82
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1774893982 1774893982
ci_job_id 1551428969 1551428969
ci_pipeline_id 105095731 105095731
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-v7dumo1c 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-v7dumo1c 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:dacapo:tomcat:appsec better
[-1.474ms; -1.125ms] or [-38.469%; -29.377%]		 2.531ms 3.831ms
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~1c9c252b82, baseline=1.61.0-SNAPSHOT~92b8b827eb
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.043 s) : 15043000, 15043000
.   : milestone, 15043000,
appsec (14.733 s) : 14733000, 14733000
.   : milestone, 14733000,
iast (17.965 s) : 17965000, 17965000
.   : milestone, 17965000,
iast_GLOBAL (17.758 s) : 17758000, 17758000
.   : milestone, 17758000,
profiling (14.971 s) : 14971000, 14971000
.   : milestone, 14971000,
tracing (14.788 s) : 14788000, 14788000
.   : milestone, 14788000,
section candidate
no_agent (14.964 s) : 14964000, 14964000
.   : milestone, 14964000,
appsec (14.832 s) : 14832000, 14832000
.   : milestone, 14832000,
iast (18.353 s) : 18353000, 18353000
.   : milestone, 18353000,
iast_GLOBAL (17.934 s) : 17934000, 17934000
.   : milestone, 17934000,
profiling (15.565 s) : 15565000, 15565000
.   : milestone, 15565000,
tracing (14.908 s) : 14908000, 14908000
.   : milestone, 14908000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.043 s [15.043 s, 15.043 s] -
appsec 14.733 s [14.733 s, 14.733 s] -310.0 ms (-2.1%)
iast 17.965 s [17.965 s, 17.965 s] 2.922 s (19.4%)
iast_GLOBAL 17.758 s [17.758 s, 17.758 s] 2.715 s (18.0%)
profiling 14.971 s [14.971 s, 14.971 s] -72.0 ms (-0.5%)
tracing 14.788 s [14.788 s, 14.788 s] -255.0 ms (-1.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.964 s [14.964 s, 14.964 s] -
appsec 14.832 s [14.832 s, 14.832 s] -132.0 ms (-0.9%)
iast 18.353 s [18.353 s, 18.353 s] 3.389 s (22.6%)
iast_GLOBAL 17.934 s [17.934 s, 17.934 s] 2.97 s (19.8%)
profiling 15.565 s [15.565 s, 15.565 s] 601.0 ms (4.0%)
tracing 14.908 s [14.908 s, 14.908 s] -56.0 ms (-0.4%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~1c9c252b82, baseline=1.61.0-SNAPSHOT~92b8b827eb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.48 ms) : 1468, 1491
.   : milestone, 1480,
appsec (3.831 ms) : 3609, 4053
.   : milestone, 3831,
iast (2.257 ms) : 2188, 2326
.   : milestone, 2257,
iast_GLOBAL (2.31 ms) : 2241, 2379
.   : milestone, 2310,
profiling (2.115 ms) : 2059, 2171
.   : milestone, 2115,
tracing (2.081 ms) : 2027, 2135
.   : milestone, 2081,
section candidate
no_agent (1.485 ms) : 1473, 1496
.   : milestone, 1485,
appsec (2.531 ms) : 2477, 2586
.   : milestone, 2531,
iast (2.266 ms) : 2197, 2335
.   : milestone, 2266,
iast_GLOBAL (2.308 ms) : 2238, 2377
.   : milestone, 2308,
profiling (2.096 ms) : 2042, 2151
.   : milestone, 2096,
tracing (2.096 ms) : 2041, 2150
.   : milestone, 2096,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.48 ms [1.468 ms, 1.491 ms] -
appsec 3.831 ms [3.609 ms, 4.053 ms] 2.351 ms (158.9%)
iast 2.257 ms [2.188 ms, 2.326 ms] 777.258 µs (52.5%)
iast_GLOBAL 2.31 ms [2.241 ms, 2.379 ms] 830.412 µs (56.1%)
profiling 2.115 ms [2.059 ms, 2.171 ms] 635.579 µs (43.0%)
tracing 2.081 ms [2.027 ms, 2.135 ms] 601.49 µs (40.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.485 ms [1.473 ms, 1.496 ms] -
appsec 2.531 ms [2.477 ms, 2.586 ms] 1.047 ms (70.5%)
iast 2.266 ms [2.197 ms, 2.335 ms] 781.164 µs (52.6%)
iast_GLOBAL 2.308 ms [2.238 ms, 2.377 ms] 822.928 µs (55.4%)
profiling 2.096 ms [2.042 ms, 2.151 ms] 611.571 µs (41.2%)
tracing 2.096 ms [2.041 ms, 2.15 ms] 611.051 µs (41.2%)

@bm1549 bm1549 force-pushed the brian.marks/broaden-log-injection-flaky-jdk8 branch from 454b292 to d07fb98 Compare March 30, 2026 17:07
@bm1549 bm1549 changed the title Fix log injection smoke test deadlock under CI load Fix log injection smoke test flakiness under CI load Mar 30, 2026
@bm1549 bm1549 force-pushed the brian.marks/broaden-log-injection-flaky-jdk8 branch 2 times, most recently from 485cafd to 5249f8d Compare March 30, 2026 17:24
@bm1549 @claude
Fix log injection smoke test flakiness under CI load
1c9c252 
Increase BaseApplication.TIMEOUT_IN_NANOS from 10s to 30s to align with the
test's PollingConditions timeout (also 30s).

This is the only smoke test with a bidirectional handshake: the app blocks
waiting for remote config changes, and the test blocks waiting for traces.
Under CI load, the trace writer's first flush (including feature discovery)
can exceed 10s. When the app's timeout fires first, it crashes the process,
killing the daemon trace-writer thread before it delivers traces — so
traceCount stays at 0. Aligning both timeouts to 30s ensures the test's
polling condition is always the binding constraint, preventing the app from
independently crashing.

Observed across all JDK versions (8, 11, 17, 21, 25) — 17 failures in the
last 30 days.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@bm1549 bm1549 force-pushed the brian.marks/broaden-log-injection-flaky-jdk8 branch from 5249f8d to 1c9c252 Compare March 30, 2026 17:34
@bm1549 bm1549 marked this pull request as ready for review March 30, 2026 17:39
@bm1549 bm1549 requested a review from a team as a code owner March 30, 2026 17:39
@bm1549 bm1549 requested review from mcculls and removed request for a team March 30, 2026 17:39
@bm1549 bm1549 enabled auto-merge March 30, 2026 18:17
@bm1549 bm1549 added this pull request to the merge queue Mar 30, 2026
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts bot commented Mar 30, 2026

/merge

@gh-worker-devflow-routing-ef8351
Copy link
Copy Markdown

gh-worker-devflow-routing-ef8351 bot commented Mar 30, 2026 •
edited
Loading

View all feedbacks in Devflow UI.

2026-03-30 18:22:27 UTC ℹ️ Start processing command /merge

2026-03-30 18:22:31 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

2026-03-30 19:33:16 UTC ℹ️ MergeQueue: This merge request was merged

@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 30, 2026
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot merged commit 847896e into master Mar 30, 2026
575 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot deleted the brian.marks/broaden-log-injection-flaky-jdk8 branch March 30, 2026 19:33
@github-actions github-actions bot added this to the 1.61.0 milestone Mar 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mhlidd mhlidd mhlidd approved these changes

@mcculls mcculls Awaiting requested review from mcculls mcculls is a code owner automatically assigned from DataDog/apm-sdk-capabilities-java

Assignees

No one assigned

Labels

comp: testing Testing tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: bug Bug report and fix

Projects

None yet

Milestone

1.61.0

Development

Successfully merging this pull request may close these issues.

2 participants

@bm1549 @mhlidd