Skip to content

Add Java 24 tests to CI#8739

Closed
sarahchen6 wants to merge 2 commits intomasterfrom
sarahchen6/add-jdk-24-test
Closed

Add Java 24 tests to CI#8739
sarahchen6 wants to merge 2 commits intomasterfrom
sarahchen6/add-jdk-24-test

Conversation

@sarahchen6
Copy link
Contributor

@sarahchen6 sarahchen6 commented Apr 25, 2025

What Does This Do

Add minimum Java 24 tests to CI

Motivation

Test Java 24

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@sarahchen6 sarahchen6 force-pushed the sarahchen6/add-jdk-24-test branch from a107c8b to 9d76aeb Compare April 25, 2025 21:07
@sarahchen6 sarahchen6 added type: enhancement Enhancements and improvements comp: testing Testing labels Apr 25, 2025
@pr-commenter
Copy link

pr-commenter bot commented Apr 25, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/add-jdk-24-test
git_commit_date 1745613844 1745615243
git_commit_sha 4d01d58 9d76aeb
release_version 1.49.0-SNAPSHOT~4d01d583bb 1.49.0-SNAPSHOT~9d76aeb34c
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1745618288 1745618288
ci_job_id 913756515 913756515
ci_pipeline_id 63401101 63401101
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-px9uuxed-project-304-concurrent-0-vy940zlk 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-px9uuxed-project-304-concurrent-0-vy940zlk 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 66 metrics, 5 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.007 s) : 0, 1007063
Total [baseline] (8.661 s) : 0, 8660791
Agent [candidate] (1.011 s) : 0, 1010595
Total [candidate] (8.647 s) : 0, 8646807
section iast
Agent [baseline] (1.135 s) : 0, 1134509
Total [baseline] (9.205 s) : 0, 9204626
Agent [candidate] (1.142 s) : 0, 1141888
Total [candidate] (9.274 s) : 0, 9274097
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.137 s) : 0, 1137326
Total [baseline] (9.179 s) : 0, 9179493
Agent [candidate] (1.142 s) : 0, 1142484
Total [candidate] (9.216 s) : 0, 9215848
section iast_TELEMETRY_OFF
Agent [baseline] (1.134 s) : 0, 1133965
Total [baseline] (9.18 s) : 0, 9179725
Agent [candidate] (1.135 s) : 0, 1135310
Total [candidate] (9.19 s) : 0, 9189537
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.007 s -
Agent iast 1.135 s 127.446 ms (12.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.137 s 130.263 ms (12.9%)
Agent iast_TELEMETRY_OFF 1.134 s 126.902 ms (12.6%)
Total tracing 8.661 s -
Total iast 9.205 s 543.835 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.179 s 518.702 ms (6.0%)
Total iast_TELEMETRY_OFF 9.18 s 518.934 ms (6.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.011 s -
Agent iast 1.142 s 131.293 ms (13.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.142 s 131.889 ms (13.1%)
Agent iast_TELEMETRY_OFF 1.135 s 124.715 ms (12.3%)
Total tracing 8.647 s -
Total iast 9.274 s 627.29 ms (7.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.216 s 569.041 ms (6.6%)
Total iast_TELEMETRY_OFF 9.19 s 542.729 ms (6.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (670.958 ms) : 0, 670958
BytebuddyAgent [candidate] (673.042 ms) : 0, 673042
GlobalTracer [baseline] (240.416 ms) : 0, 240416
GlobalTracer [candidate] (240.692 ms) : 0, 240692
AppSec [baseline] (54.593 ms) : 0, 54593
AppSec [candidate] (54.863 ms) : 0, 54863
Debugger [baseline] (6.872 ms) : 0, 6872
Debugger [candidate] (6.205 ms) : 0, 6205
Remote Config [baseline] (713.246 µs) : 0, 713
Remote Config [candidate] (725.354 µs) : 0, 725
Telemetry [baseline] (10.045 ms) : 0, 10045
Telemetry [candidate] (11.482 ms) : 0, 11482
section iast
BytebuddyAgent [baseline] (788.529 ms) : 0, 788529
BytebuddyAgent [candidate] (795.229 ms) : 0, 795229
GlobalTracer [baseline] (229.531 ms) : 0, 229531
GlobalTracer [candidate] (229.737 ms) : 0, 229737
IAST [baseline] (22.725 ms) : 0, 22725
IAST [candidate] (22.78 ms) : 0, 22780
AppSec [baseline] (56.091 ms) : 0, 56091
AppSec [candidate] (56.377 ms) : 0, 56377
Debugger [baseline] (5.852 ms) : 0, 5852
Debugger [candidate] (5.839 ms) : 0, 5839
Remote Config [baseline] (593.346 µs) : 0, 593
Remote Config [candidate] (586.118 µs) : 0, 586
Telemetry [baseline] (7.819 ms) : 0, 7819
Telemetry [candidate] (7.843 ms) : 0, 7843
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (789.516 ms) : 0, 789516
BytebuddyAgent [candidate] (792.633 ms) : 0, 792633
GlobalTracer [baseline] (230.305 ms) : 0, 230305
GlobalTracer [candidate] (231.764 ms) : 0, 231764
IAST [baseline] (22.856 ms) : 0, 22856
IAST [candidate] (23.045 ms) : 0, 23045
AppSec [baseline] (56.688 ms) : 0, 56688
AppSec [candidate] (57.069 ms) : 0, 57069
Debugger [baseline] (5.932 ms) : 0, 5932
Debugger [candidate] (5.94 ms) : 0, 5940
Remote Config [baseline] (601.238 µs) : 0, 601
Remote Config [candidate] (614.031 µs) : 0, 614
Telemetry [baseline] (7.967 ms) : 0, 7967
Telemetry [candidate] (8.004 ms) : 0, 8004
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (786.794 ms) : 0, 786794
BytebuddyAgent [candidate] (787.958 ms) : 0, 787958
GlobalTracer [baseline] (230.094 ms) : 0, 230094
GlobalTracer [candidate] (230.839 ms) : 0, 230839
IAST [baseline] (22.386 ms) : 0, 22386
IAST [candidate] (22.353 ms) : 0, 22353
AppSec [baseline] (56.884 ms) : 0, 56884
AppSec [candidate] (56.456 ms) : 0, 56456
Debugger [baseline] (5.974 ms) : 0, 5974
Debugger [candidate] (5.935 ms) : 0, 5935
Remote Config [baseline] (596.156 µs) : 0, 596
Remote Config [candidate] (614.423 µs) : 0, 614
Telemetry [baseline] (7.807 ms) : 0, 7807
Telemetry [candidate] (7.791 ms) : 0, 7791
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.008 s) : 0, 1008470
Total [baseline] (10.485 s) : 0, 10484825
Agent [candidate] (1.01 s) : 0, 1009968
Total [candidate] (10.518 s) : 0, 10517696
section appsec
Agent [baseline] (1.156 s) : 0, 1156323
Total [baseline] (10.672 s) : 0, 10672352
Agent [candidate] (1.158 s) : 0, 1158488
Total [candidate] (10.702 s) : 0, 10702382
section iast
Agent [baseline] (1.153 s) : 0, 1152544
Total [baseline] (10.862 s) : 0, 10862095
Agent [candidate] (1.153 s) : 0, 1153342
Total [candidate] (10.866 s) : 0, 10866446
section profiling
Agent [baseline] (1.262 s) : 0, 1261834
Total [baseline] (10.853 s) : 0, 10853009
Agent [candidate] (1.274 s) : 0, 1273757
Total [candidate] (10.936 s) : 0, 10935623
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.008 s -
Agent appsec 1.156 s 147.854 ms (14.7%)
Agent iast 1.153 s 144.074 ms (14.3%)
Agent profiling 1.262 s 253.365 ms (25.1%)
Total tracing 10.485 s -
Total appsec 10.672 s 187.527 ms (1.8%)
Total iast 10.862 s 377.27 ms (3.6%)
Total profiling 10.853 s 368.184 ms (3.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.01 s -
Agent appsec 1.158 s 148.52 ms (14.7%)
Agent iast 1.153 s 143.374 ms (14.2%)
Agent profiling 1.274 s 263.789 ms (26.1%)
Total tracing 10.518 s -
Total appsec 10.702 s 184.686 ms (1.8%)
Total iast 10.866 s 348.75 ms (3.3%)
Total profiling 10.936 s 417.927 ms (4.0%) 
gantt
    title petclinic - break down per module: candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (672.515 ms) : 0, 672515
BytebuddyAgent [candidate] (672.785 ms) : 0, 672785
GlobalTracer [baseline] (241.094 ms) : 0, 241094
GlobalTracer [candidate] (241.248 ms) : 0, 241248
AppSec [baseline] (54.555 ms) : 0, 54555
AppSec [candidate] (54.795 ms) : 0, 54795
Debugger [baseline] (6.147 ms) : 0, 6147
Debugger [candidate] (6.142 ms) : 0, 6142
Remote Config [baseline] (721.334 µs) : 0, 721
Remote Config [candidate] (714.486 µs) : 0, 714
Telemetry [baseline] (9.929 ms) : 0, 9929
Telemetry [candidate] (10.726 ms) : 0, 10726
section appsec
BytebuddyAgent [baseline] (693.031 ms) : 0, 693031
BytebuddyAgent [candidate] (694.26 ms) : 0, 694260
GlobalTracer [baseline] (237.99 ms) : 0, 237990
GlobalTracer [candidate] (238.357 ms) : 0, 238357
AppSec [baseline] (176.262 ms) : 0, 176262
AppSec [candidate] (176.697 ms) : 0, 176697
Debugger [baseline] (5.881 ms) : 0, 5881
Debugger [candidate] (5.972 ms) : 0, 5972
Remote Config [baseline] (632.786 µs) : 0, 633
Remote Config [candidate] (659.687 µs) : 0, 660
Telemetry [baseline] (7.752 ms) : 0, 7752
Telemetry [candidate] (7.864 ms) : 0, 7864
IAST [baseline] (22.331 ms) : 0, 22331
IAST [candidate] (22.214 ms) : 0, 22214
section iast
BytebuddyAgent [baseline] (801.029 ms) : 0, 801029
BytebuddyAgent [candidate] (802.481 ms) : 0, 802481
GlobalTracer [baseline] (232.571 ms) : 0, 232571
GlobalTracer [candidate] (232.624 ms) : 0, 232624
AppSec [baseline] (57.125 ms) : 0, 57125
AppSec [candidate] (56.656 ms) : 0, 56656
Debugger [baseline] (5.982 ms) : 0, 5982
Debugger [candidate] (5.961 ms) : 0, 5961
Remote Config [baseline] (611.757 µs) : 0, 612
Remote Config [candidate] (604.418 µs) : 0, 604
Telemetry [baseline] (8.059 ms) : 0, 8059
Telemetry [candidate] (8.014 ms) : 0, 8014
IAST [baseline] (23.426 ms) : 0, 23426
IAST [candidate] (23.155 ms) : 0, 23155
section profiling
BytebuddyAgent [baseline] (666.744 ms) : 0, 666744
BytebuddyAgent [candidate] (673.478 ms) : 0, 673478
GlobalTracer [baseline] (376.963 ms) : 0, 376963
GlobalTracer [candidate] (380.234 ms) : 0, 380234
AppSec [baseline] (54.615 ms) : 0, 54615
AppSec [candidate] (54.131 ms) : 0, 54131
Debugger [baseline] (6.143 ms) : 0, 6143
Debugger [candidate] (6.233 ms) : 0, 6233
Remote Config [baseline] (663.064 µs) : 0, 663
Remote Config [candidate] (671.362 µs) : 0, 671
Telemetry [baseline] (8.151 ms) : 0, 8151
Telemetry [candidate] (8.367 ms) : 0, 8367
ProfilingAgent [baseline] (97.976 ms) : 0, 97976
ProfilingAgent [candidate] (99.724 ms) : 0, 99724
Profiling [baseline] (98.0 ms) : 0, 98000
Profiling [candidate] (99.749 ms) : 0, 99749
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-04-25T21:28:30 2025-04-25T21:36:15
git_branch master sarahchen6/add-jdk-24-test
git_commit_date 1745613844 1745615243
git_commit_sha 4d01d58 9d76aeb
release_version 1.49.0-SNAPSHOT~4d01d583bb 1.49.0-SNAPSHOT~9d76aeb34c
start_time 2025-04-25T21:28:16 2025-04-25T21:36:00
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1745617372 1745617372
ci_job_id 913756516 913756516
ci_pipeline_id 63401101 63401101
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-syfpehly-project-304-concurrent-0-ayj9p1ia 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-syfpehly-project-304-concurrent-0-ayj9p1ia 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (386.362 µs) : 367, 406
.   : milestone, 386,
iast (527.965 µs) : 505, 551
.   : milestone, 528,
iast_FULL (736.601 µs) : 714, 760
.   : milestone, 737,
iast_GLOBAL (576.13 µs) : 552, 600
.   : milestone, 576,
iast_HARDCODED_SECRET_DISABLED (513.262 µs) : 491, 536
.   : milestone, 513,
iast_INACTIVE (466.749 µs) : 445, 489
.   : milestone, 467,
iast_TELEMETRY_OFF (520.31 µs) : 498, 543
.   : milestone, 520,
tracing (464.516 µs) : 442, 487
.   : milestone, 465,
section candidate
no_agent (385.965 µs) : 366, 406
.   : milestone, 386,
iast (522.769 µs) : 499, 546
.   : milestone, 523,
iast_FULL (732.808 µs) : 709, 756
.   : milestone, 733,
iast_GLOBAL (570.471 µs) : 546, 595
.   : milestone, 570,
iast_HARDCODED_SECRET_DISABLED (531.416 µs) : 509, 554
.   : milestone, 531,
iast_INACTIVE (471.595 µs) : 449, 494
.   : milestone, 472,
iast_TELEMETRY_OFF (505.718 µs) : 483, 529
.   : milestone, 506,
tracing (467.024 µs) : 445, 489
.   : milestone, 467,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 386.362 µs [366.501 µs, 406.222 µs] -
iast 527.965 µs [505.056 µs, 550.875 µs] 141.604 µs (36.7%)
iast_FULL 736.601 µs [713.629 µs, 759.572 µs] 350.239 µs (90.7%)
iast_GLOBAL 576.13 µs [552.399 µs, 599.862 µs] 189.769 µs (49.1%)
iast_HARDCODED_SECRET_DISABLED 513.262 µs [490.522 µs, 536.003 µs] 126.901 µs (32.8%)
iast_INACTIVE 466.749 µs [444.828 µs, 488.669 µs] 80.387 µs (20.8%)
iast_TELEMETRY_OFF 520.31 µs [497.787 µs, 542.833 µs] 133.948 µs (34.7%)
tracing 464.516 µs [442.325 µs, 486.707 µs] 78.155 µs (20.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 385.965 µs [366.271 µs, 405.66 µs] -
iast 522.769 µs [499.324 µs, 546.213 µs] 136.803 µs (35.4%)
iast_FULL 732.808 µs [709.493 µs, 756.123 µs] 346.843 µs (89.9%)
iast_GLOBAL 570.471 µs [546.384 µs, 594.558 µs] 184.506 µs (47.8%)
iast_HARDCODED_SECRET_DISABLED 531.416 µs [508.894 µs, 553.939 µs] 145.451 µs (37.7%)
iast_INACTIVE 471.595 µs [448.848 µs, 494.342 µs] 85.63 µs (22.2%)
iast_TELEMETRY_OFF 505.718 µs [482.917 µs, 528.519 µs] 119.753 µs (31.0%)
tracing 467.024 µs [444.694 µs, 489.354 µs] 81.059 µs (21.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.36 ms) : 1341, 1379
.   : milestone, 1360,
appsec (1.725 ms) : 1701, 1749
.   : milestone, 1725,
appsec_no_iast (1.735 ms) : 1712, 1758
.   : milestone, 1735,
code_origins (1.681 ms) : 1654, 1708
.   : milestone, 1681,
iast (1.518 ms) : 1494, 1541
.   : milestone, 1518,
profiling (1.531 ms) : 1508, 1554
.   : milestone, 1531,
tracing (1.495 ms) : 1470, 1520
.   : milestone, 1495,
section candidate
no_agent (1.376 ms) : 1357, 1396
.   : milestone, 1376,
appsec (1.724 ms) : 1699, 1748
.   : milestone, 1724,
appsec_no_iast (1.741 ms) : 1718, 1764
.   : milestone, 1741,
code_origins (1.676 ms) : 1649, 1704
.   : milestone, 1676,
iast (1.524 ms) : 1500, 1548
.   : milestone, 1524,
profiling (1.506 ms) : 1481, 1530
.   : milestone, 1506,
tracing (1.494 ms) : 1469, 1519
.   : milestone, 1494,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.36 ms [1.341 ms, 1.379 ms] -
appsec 1.725 ms [1.701 ms, 1.749 ms] 365.295 µs (26.9%)
appsec_no_iast 1.735 ms [1.712 ms, 1.758 ms] 375.164 µs (27.6%)
code_origins 1.681 ms [1.654 ms, 1.708 ms] 321.269 µs (23.6%)
iast 1.518 ms [1.494 ms, 1.541 ms] 157.791 µs (11.6%)
profiling 1.531 ms [1.508 ms, 1.554 ms] 171.481 µs (12.6%)
tracing 1.495 ms [1.47 ms, 1.52 ms] 134.929 µs (9.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.376 ms [1.357 ms, 1.396 ms] -
appsec 1.724 ms [1.699 ms, 1.748 ms] 347.282 µs (25.2%)
appsec_no_iast 1.741 ms [1.718 ms, 1.764 ms] 364.241 µs (26.5%)
code_origins 1.676 ms [1.649 ms, 1.704 ms] 300.067 µs (21.8%)
iast 1.524 ms [1.5 ms, 1.548 ms] 147.619 µs (10.7%)
profiling 1.506 ms [1.481 ms, 1.53 ms] 129.456 µs (9.4%)
tracing 1.494 ms [1.469 ms, 1.519 ms] 117.604 µs (8.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/add-jdk-24-test
git_commit_date 1745613844 1745615243
git_commit_sha 4d01d58 9d76aeb
release_version 1.49.0-SNAPSHOT~4d01d583bb 1.49.0-SNAPSHOT~9d76aeb34c
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1745617876 1745617876
ci_job_id 913756517 913756517
ci_pipeline_id 63401101 63401101
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-px9uuxed-project-304-concurrent-1-p44dvvql 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-px9uuxed-project-304-concurrent-1-p44dvvql 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.263 s) : 15263000, 15263000
.   : milestone, 15263000,
appsec (14.873 s) : 14873000, 14873000
.   : milestone, 14873000,
iast (18.907 s) : 18907000, 18907000
.   : milestone, 18907000,
iast_GLOBAL (18.258 s) : 18258000, 18258000
.   : milestone, 18258000,
profiling (15.029 s) : 15029000, 15029000
.   : milestone, 15029000,
tracing (14.816 s) : 14816000, 14816000
.   : milestone, 14816000,
section candidate
no_agent (15.489 s) : 15489000, 15489000
.   : milestone, 15489000,
appsec (14.947 s) : 14947000, 14947000
.   : milestone, 14947000,
iast (18.48 s) : 18480000, 18480000
.   : milestone, 18480000,
iast_GLOBAL (17.858 s) : 17858000, 17858000
.   : milestone, 17858000,
profiling (15.572 s) : 15572000, 15572000
.   : milestone, 15572000,
tracing (14.949 s) : 14949000, 14949000
.   : milestone, 14949000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.263 s [15.263 s, 15.263 s] -
appsec 14.873 s [14.873 s, 14.873 s] -390.0 ms (-2.6%)
iast 18.907 s [18.907 s, 18.907 s] 3.644 s (23.9%)
iast_GLOBAL 18.258 s [18.258 s, 18.258 s] 2.995 s (19.6%)
profiling 15.029 s [15.029 s, 15.029 s] -234.0 ms (-1.5%)
tracing 14.816 s [14.816 s, 14.816 s] -447.0 ms (-2.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.489 s [15.489 s, 15.489 s] -
appsec 14.947 s [14.947 s, 14.947 s] -542.0 ms (-3.5%)
iast 18.48 s [18.48 s, 18.48 s] 2.991 s (19.3%)
iast_GLOBAL 17.858 s [17.858 s, 17.858 s] 2.369 s (15.3%)
profiling 15.572 s [15.572 s, 15.572 s] 83.0 ms (0.5%)
tracing 14.949 s [14.949 s, 14.949 s] -540.0 ms (-3.5%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~9d76aeb34c, baseline=1.49.0-SNAPSHOT~4d01d583bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.47 ms) : 1459, 1482
.   : milestone, 1470,
appsec (2.377 ms) : 2330, 2424
.   : milestone, 2377,
iast (2.15 ms) : 2091, 2208
.   : milestone, 2150,
iast_GLOBAL (2.188 ms) : 2129, 2247
.   : milestone, 2188,
profiling (2.452 ms) : 2283, 2620
.   : milestone, 2452,
tracing (1.977 ms) : 1931, 2022
.   : milestone, 1977,
section candidate
no_agent (1.473 ms) : 1461, 1484
.   : milestone, 1473,
appsec (2.369 ms) : 2323, 2416
.   : milestone, 2369,
iast (2.15 ms) : 2091, 2209
.   : milestone, 2150,
iast_GLOBAL (2.187 ms) : 2129, 2246
.   : milestone, 2187,
profiling (2.459 ms) : 2279, 2638
.   : milestone, 2459,
tracing (1.985 ms) : 1939, 2030
.   : milestone, 1985,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.47 ms [1.459 ms, 1.482 ms] -
appsec 2.377 ms [2.33 ms, 2.424 ms] 906.88 µs (61.7%)
iast 2.15 ms [2.091 ms, 2.208 ms] 679.122 µs (46.2%)
iast_GLOBAL 2.188 ms [2.129 ms, 2.247 ms] 717.54 µs (48.8%)
profiling 2.452 ms [2.283 ms, 2.62 ms] 981.218 µs (66.7%)
tracing 1.977 ms [1.931 ms, 2.022 ms] 506.145 µs (34.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.461 ms, 1.484 ms] -
appsec 2.369 ms [2.323 ms, 2.416 ms] 896.558 µs (60.9%)
iast 2.15 ms [2.091 ms, 2.209 ms] 677.298 µs (46.0%)
iast_GLOBAL 2.187 ms [2.129 ms, 2.246 ms] 714.666 µs (48.5%)
profiling 2.459 ms [2.279 ms, 2.638 ms] 985.945 µs (66.9%)
tracing 1.985 ms [1.939 ms, 2.03 ms] 511.88 µs (34.8%)

PerfectSlayer
PerfectSlayer reviewed Apr 28, 2025
View reviewed changes
Copy link
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @sarahchen6

Where does this requirement come from?
I plan to address the question about JVM testing at next the guild meeting, and with the CI migration actors too, as it will need to be coordinated in the 3 CI platforms.
Can we sync about it internally first?

In addition, the value 24 won’t work as it is not supported yet in our builder image.

@PerfectSlayer PerfectSlayer added the tag: no release notes Changes to exclude from release notes label Apr 28, 2025
@sarahchen6
Copy link
Contributor Author

sarahchen6 commented Apr 28, 2025

Hi @PerfectSlayer yes, sounds great - let's sync internally first! I was trying to get started with adding Java 24 tests here: https://datadoghq.atlassian.net/browse/LANGPLAT-421, but missing a lot.. we should definitely sync first 😅 Thanks!

@sarahchen6 sarahchen6 closed this Apr 28, 2025
@sarahchen6 sarahchen6 deleted the sarahchen6/add-jdk-24-test branch October 16, 2025 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer left review comments

Assignees

No one assigned

Labels

comp: testing Testing tag: no release notes Changes to exclude from release notes type: enhancement Enhancements and improvements

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@sarahchen6 @PerfectSlayer