Skip to content

Move macrobenchmarks to apm-sdks-benchmarks repo #10066

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
sarahchen6 wants to merge 12 commits into
base: master
Choose a base branch
from
Draft

Move macrobenchmarks to apm-sdks-benchmarks repo #10066

sarahchen6 wants to merge 12 commits into from

Conversation

@sarahchen6
Copy link
Contributor

@sarahchen6 sarahchen6 commented Dec 2, 2025 •
edited
Loading

WIP!!

What Does This Do

Test the apm-sdks-benchmarks repo implementation of macrobenchmarks: https://github.com/DataDog/apm-sdks-benchmarks/pull/34

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@sarahchen6 sarahchen6 force-pushed the sarahchen6/move-macrobenchmarks branch from f391147 to 9602f73 Compare December 2, 2025 17:28
@sarahchen6 sarahchen6 force-pushed the sarahchen6/move-macrobenchmarks branch from ba1ae21 to b376785 Compare December 2, 2025 17:44
@pr-commenter
Copy link

pr-commenter bot commented Dec 2, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/move-macrobenchmarks
git_commit_date 1764964306 1764964429
git_commit_sha 5802e1e 39bf2ea
release_version 1.57.0-SNAPSHOT~5802e1e9f0 1.57.0-SNAPSHOT~39bf2eadcb
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1764966170 1764966170
ci_job_id 1273385533 1273385533
ci_pipeline_id 85067186 85067186
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-3-5f6iaecg 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-3-5f6iaecg 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 5 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~39bf2eadcb, baseline=1.57.0-SNAPSHOT~5802e1e9f0

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.078 s) : 0, 1078436
Total [baseline] (10.772 s) : 0, 10772465
Agent [candidate] (1.085 s) : 0, 1085197
Total [candidate] (10.874 s) : 0, 10873788
section appsec
Agent [baseline] (1.271 s) : 0, 1271378
Total [baseline] (11.19 s) : 0, 11189738
Agent [candidate] (1.268 s) : 0, 1267614
Total [candidate] (11.054 s) : 0, 11053811
section iast
Agent [baseline] (1.237 s) : 0, 1236937
Total [baseline] (11.143 s) : 0, 11142909
Agent [candidate] (1.225 s) : 0, 1224806
Total [candidate] (11.145 s) : 0, 11144523
section profiling
Agent [baseline] (1.202 s) : 0, 1202155
Total [baseline] (11.012 s) : 0, 11011724
Agent [candidate] (1.207 s) : 0, 1206789
Total [candidate] (11.141 s) : 0, 11141239
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.078 s -
Agent appsec 1.271 s 192.942 ms (17.9%)
Agent iast 1.237 s 158.501 ms (14.7%)
Agent profiling 1.202 s 123.718 ms (11.5%)
Total tracing 10.772 s -
Total appsec 11.19 s 417.273 ms (3.9%)
Total iast 11.143 s 370.445 ms (3.4%)
Total profiling 11.012 s 239.259 ms (2.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.085 s -
Agent appsec 1.268 s 182.417 ms (16.8%)
Agent iast 1.225 s 139.609 ms (12.9%)
Agent profiling 1.207 s 121.592 ms (11.2%)
Total tracing 10.874 s -
Total appsec 11.054 s 180.023 ms (1.7%)
Total iast 11.145 s 270.735 ms (2.5%)
Total profiling 11.141 s 267.451 ms (2.5%) 
gantt
    title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~39bf2eadcb, baseline=1.57.0-SNAPSHOT~5802e1e9f0

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.189 ms) : 0, 1189
crashtracking [candidate] (1.186 ms) : 0, 1186
BytebuddyAgent [baseline] (646.394 ms) : 0, 646394
BytebuddyAgent [candidate] (650.549 ms) : 0, 650549
GlobalTracer [baseline] (281.756 ms) : 0, 281756
GlobalTracer [candidate] (283.453 ms) : 0, 283453
AppSec [baseline] (32.153 ms) : 0, 32153
AppSec [candidate] (32.62 ms) : 0, 32620
Debugger [baseline] (68.004 ms) : 0, 68004
Debugger [candidate] (68.37 ms) : 0, 68370
Remote Config [baseline] (656.817 µs) : 0, 657
Remote Config [candidate] (656.721 µs) : 0, 657
Telemetry [baseline] (8.963 ms) : 0, 8963
Telemetry [candidate] (8.951 ms) : 0, 8951
Flare Poller [baseline] (3.733 ms) : 0, 3733
Flare Poller [candidate] (3.796 ms) : 0, 3796
section appsec
crashtracking [baseline] (1.2 ms) : 0, 1200
crashtracking [candidate] (1.203 ms) : 0, 1203
BytebuddyAgent [baseline] (694.578 ms) : 0, 694578
BytebuddyAgent [candidate] (691.045 ms) : 0, 691045
GlobalTracer [baseline] (259.474 ms) : 0, 259474
GlobalTracer [candidate] (259.784 ms) : 0, 259784
AppSec [baseline] (174.812 ms) : 0, 174812
AppSec [candidate] (175.194 ms) : 0, 175194
Debugger [baseline] (67.479 ms) : 0, 67479
Debugger [candidate] (66.63 ms) : 0, 66630
Remote Config [baseline] (670.155 µs) : 0, 670
Remote Config [candidate] (708.286 µs) : 0, 708
Telemetry [baseline] (8.95 ms) : 0, 8950
Telemetry [candidate] (8.948 ms) : 0, 8948
Flare Poller [baseline] (3.913 ms) : 0, 3913
Flare Poller [candidate] (3.826 ms) : 0, 3826
IAST [baseline] (24.541 ms) : 0, 24541
IAST [candidate] (24.64 ms) : 0, 24640
section iast
crashtracking [baseline] (1.202 ms) : 0, 1202
crashtracking [candidate] (1.186 ms) : 0, 1186
BytebuddyAgent [baseline] (799.751 ms) : 0, 799751
BytebuddyAgent [candidate] (791.427 ms) : 0, 791427
GlobalTracer [baseline] (258.293 ms) : 0, 258293
GlobalTracer [candidate] (255.326 ms) : 0, 255326
AppSec [baseline] (35.508 ms) : 0, 35508
AppSec [candidate] (35.672 ms) : 0, 35672
Debugger [baseline] (66.758 ms) : 0, 66758
Debugger [candidate] (66.075 ms) : 0, 66075
Remote Config [baseline] (532.757 µs) : 0, 533
Remote Config [candidate] (536.445 µs) : 0, 536
Telemetry [baseline] (8.428 ms) : 0, 8428
Telemetry [candidate] (8.474 ms) : 0, 8474
Flare Poller [baseline] (3.465 ms) : 0, 3465
Flare Poller [candidate] (3.458 ms) : 0, 3458
IAST [baseline] (27.281 ms) : 0, 27281
IAST [candidate] (27.108 ms) : 0, 27108
section profiling
ProfilingAgent [baseline] (97.354 ms) : 0, 97354
ProfilingAgent [candidate] (97.702 ms) : 0, 97702
crashtracking [baseline] (1.178 ms) : 0, 1178
crashtracking [candidate] (1.174 ms) : 0, 1174
BytebuddyAgent [baseline] (699.287 ms) : 0, 699287
BytebuddyAgent [candidate] (702.316 ms) : 0, 702316
GlobalTracer [baseline] (220.773 ms) : 0, 220773
GlobalTracer [candidate] (221.842 ms) : 0, 221842
AppSec [baseline] (31.991 ms) : 0, 31991
AppSec [candidate] (32.347 ms) : 0, 32347
Debugger [baseline] (68.584 ms) : 0, 68584
Debugger [candidate] (68.367 ms) : 0, 68367
Remote Config [baseline] (613.552 µs) : 0, 614
Remote Config [candidate] (630.299 µs) : 0, 630
Telemetry [baseline] (8.961 ms) : 0, 8961
Telemetry [candidate] (8.958 ms) : 0, 8958
Flare Poller [baseline] (3.765 ms) : 0, 3765
Flare Poller [candidate] (3.77 ms) : 0, 3770
Profiling [baseline] (97.948 ms) : 0, 97948
Profiling [candidate] (98.282 ms) : 0, 98282
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~39bf2eadcb, baseline=1.57.0-SNAPSHOT~5802e1e9f0

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.088 s) : 0, 1087659
Total [baseline] (8.808 s) : 0, 8808057
Agent [candidate] (1.082 s) : 0, 1081662
Total [candidate] (8.766 s) : 0, 8765756
section iast
Agent [baseline] (1.229 s) : 0, 1229226
Total [baseline] (9.492 s) : 0, 9492339
Agent [candidate] (1.223 s) : 0, 1223410
Total [candidate] (9.491 s) : 0, 9491004
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.088 s -
Agent iast 1.229 s 141.568 ms (13.0%)
Total tracing 8.808 s -
Total iast 9.492 s 684.282 ms (7.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.082 s -
Agent iast 1.223 s 141.748 ms (13.1%)
Total tracing 8.766 s -
Total iast 9.491 s 725.248 ms (8.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~39bf2eadcb, baseline=1.57.0-SNAPSHOT~5802e1e9f0

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.195 ms) : 0, 1195
crashtracking [candidate] (1.195 ms) : 0, 1195
BytebuddyAgent [baseline] (653.006 ms) : 0, 653006
BytebuddyAgent [candidate] (648.553 ms) : 0, 648553
GlobalTracer [baseline] (284.124 ms) : 0, 284124
GlobalTracer [candidate] (282.214 ms) : 0, 282214
AppSec [baseline] (32.529 ms) : 0, 32529
AppSec [candidate] (32.474 ms) : 0, 32474
Debugger [baseline] (67.442 ms) : 0, 67442
Debugger [candidate] (68.275 ms) : 0, 68275
Remote Config [baseline] (629.929 µs) : 0, 630
Remote Config [candidate] (648.19 µs) : 0, 648
Telemetry [baseline] (9.097 ms) : 0, 9097
Telemetry [candidate] (8.959 ms) : 0, 8959
Flare Poller [baseline] (3.755 ms) : 0, 3755
Flare Poller [candidate] (3.726 ms) : 0, 3726
section iast
crashtracking [baseline] (1.193 ms) : 0, 1193
crashtracking [candidate] (1.202 ms) : 0, 1202
BytebuddyAgent [baseline] (795.175 ms) : 0, 795175
BytebuddyAgent [candidate] (791.473 ms) : 0, 791473
GlobalTracer [baseline] (256.86 ms) : 0, 256860
GlobalTracer [candidate] (255.335 ms) : 0, 255335
AppSec [baseline] (35.419 ms) : 0, 35419
AppSec [candidate] (34.514 ms) : 0, 34514
Debugger [baseline] (65.02 ms) : 0, 65020
Debugger [candidate] (65.905 ms) : 0, 65905
Remote Config [baseline] (559.202 µs) : 0, 559
Remote Config [candidate] (542.256 µs) : 0, 542
Telemetry [baseline] (8.515 ms) : 0, 8515
Telemetry [candidate] (8.388 ms) : 0, 8388
Flare Poller [baseline] (3.438 ms) : 0, 3438
Flare Poller [candidate] (3.472 ms) : 0, 3472
IAST [baseline] (27.351 ms) : 0, 27351
IAST [candidate] (27.021 ms) : 0, 27021
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/move-macrobenchmarks
git_commit_date 1764964306 1764964429
git_commit_sha 5802e1e 39bf2ea
release_version 1.57.0-SNAPSHOT~5802e1e9f0 1.57.0-SNAPSHOT~39bf2eadcb
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1764966652 1764966652
ci_job_id 1273385540 1273385540
ci_pipeline_id 85067186 85067186
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-j9quqqzy 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-j9quqqzy 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 19 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast_FULL:high_load better
[-379.860µs; -182.368µs] or [-7.235%; -3.473%]		 unsure
[-905.821µs; -113.845µs] or [-7.221%; -0.908%]		 unstable
[-46.634op/s; +125.759op/s] or [-5.940%; +16.019%]		 4.969ms 12.034ms 824.625op/s 5.250ms 12.543ms 785.062op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~39bf2eadcb, baseline=1.57.0-SNAPSHOT~5802e1e9f0
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.211 ms) : 1199, 1223
.   : milestone, 1211,
iast (3.246 ms) : 3203, 3289
.   : milestone, 3246,
iast_FULL (5.891 ms) : 5830, 5952
.   : milestone, 5891,
iast_GLOBAL (3.606 ms) : 3546, 3666
.   : milestone, 3606,
profiling (1.99 ms) : 1973, 2007
.   : milestone, 1990,
tracing (1.799 ms) : 1785, 1814
.   : milestone, 1799,
section candidate
no_agent (1.204 ms) : 1192, 1216
.   : milestone, 1204,
iast (3.272 ms) : 3226, 3318
.   : milestone, 3272,
iast_FULL (5.605 ms) : 5550, 5659
.   : milestone, 5605,
iast_GLOBAL (3.544 ms) : 3490, 3598
.   : milestone, 3544,
profiling (1.886 ms) : 1871, 1901
.   : milestone, 1886,
tracing (1.785 ms) : 1770, 1799
.   : milestone, 1785,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.211 ms [1.199 ms, 1.223 ms] -
iast 3.246 ms [3.203 ms, 3.289 ms] 2.035 ms (168.0%)
iast_FULL 5.891 ms [5.83 ms, 5.952 ms] 4.68 ms (386.5%)
iast_GLOBAL 3.606 ms [3.546 ms, 3.666 ms] 2.395 ms (197.8%)
profiling 1.99 ms [1.973 ms, 2.007 ms] 778.709 µs (64.3%)
tracing 1.799 ms [1.785 ms, 1.814 ms] 588.411 µs (48.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.204 ms [1.192 ms, 1.216 ms] -
iast 3.272 ms [3.226 ms, 3.318 ms] 2.068 ms (171.8%)
iast_FULL 5.605 ms [5.55 ms, 5.659 ms] 4.401 ms (365.6%)
iast_GLOBAL 3.544 ms [3.49 ms, 3.598 ms] 2.34 ms (194.4%)
profiling 1.886 ms [1.871 ms, 1.901 ms] 682.116 µs (56.7%)
tracing 1.785 ms [1.77 ms, 1.799 ms] 580.84 µs (48.3%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~39bf2eadcb, baseline=1.57.0-SNAPSHOT~5802e1e9f0
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.12 ms) : 17938, 18302
.   : milestone, 18120,
appsec (19.115 ms) : 18924, 19306
.   : milestone, 19115,
code_origins (17.499 ms) : 17328, 17671
.   : milestone, 17499,
iast (17.581 ms) : 17409, 17753
.   : milestone, 17581,
profiling (18.643 ms) : 18457, 18830
.   : milestone, 18643,
tracing (17.498 ms) : 17324, 17673
.   : milestone, 17498,
section candidate
no_agent (18.968 ms) : 18781, 19155
.   : milestone, 18968,
appsec (18.841 ms) : 18652, 19030
.   : milestone, 18841,
code_origins (17.757 ms) : 17580, 17934
.   : milestone, 17757,
iast (17.659 ms) : 17480, 17837
.   : milestone, 17659,
profiling (18.585 ms) : 18399, 18772
.   : milestone, 18585,
tracing (17.632 ms) : 17459, 17806
.   : milestone, 17632,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.12 ms [17.938 ms, 18.302 ms] -
appsec 19.115 ms [18.924 ms, 19.306 ms] 994.918 µs (5.5%)
code_origins 17.499 ms [17.328 ms, 17.671 ms] -620.714 µs (-3.4%)
iast 17.581 ms [17.409 ms, 17.753 ms] -539.044 µs (-3.0%)
profiling 18.643 ms [18.457 ms, 18.83 ms] 523.207 µs (2.9%)
tracing 17.498 ms [17.324 ms, 17.673 ms] -621.725 µs (-3.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.968 ms [18.781 ms, 19.155 ms] -
appsec 18.841 ms [18.652 ms, 19.03 ms] -126.803 µs (-0.7%)
code_origins 17.757 ms [17.58 ms, 17.934 ms] -1.211 ms (-6.4%)
iast 17.659 ms [17.48 ms, 17.837 ms] -1.309 ms (-6.9%)
profiling 18.585 ms [18.399 ms, 18.772 ms] -382.66 µs (-2.0%)
tracing 17.632 ms [17.459 ms, 17.806 ms] -1.336 ms (-7.0%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/move-macrobenchmarks
git_commit_date 1764964306 1764964429
git_commit_sha 5802e1e 39bf2ea
release_version 1.57.0-SNAPSHOT~5802e1e9f0 1.57.0-SNAPSHOT~39bf2eadcb
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1764966470 1764966470
ci_job_id 1273385545 1273385545
ci_pipeline_id 85067186 85067186
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-3-uptkovbq 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-3-uptkovbq 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~39bf2eadcb, baseline=1.57.0-SNAPSHOT~5802e1e9f0
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.486 ms) : 1474, 1498
.   : milestone, 1486,
appsec (2.543 ms) : 2488, 2598
.   : milestone, 2543,
iast (2.217 ms) : 2153, 2281
.   : milestone, 2217,
iast_GLOBAL (2.271 ms) : 2206, 2336
.   : milestone, 2271,
profiling (2.094 ms) : 2040, 2147
.   : milestone, 2094,
tracing (2.059 ms) : 2009, 2110
.   : milestone, 2059,
section candidate
no_agent (1.483 ms) : 1471, 1494
.   : milestone, 1483,
appsec (3.668 ms) : 3452, 3884
.   : milestone, 3668,
iast (2.224 ms) : 2159, 2289
.   : milestone, 2224,
iast_GLOBAL (2.269 ms) : 2204, 2334
.   : milestone, 2269,
profiling (2.48 ms) : 2329, 2631
.   : milestone, 2480,
tracing (2.051 ms) : 2000, 2101
.   : milestone, 2051,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.486 ms [1.474 ms, 1.498 ms] -
appsec 2.543 ms [2.488 ms, 2.598 ms] 1.057 ms (71.1%)
iast 2.217 ms [2.153 ms, 2.281 ms] 731.12 µs (49.2%)
iast_GLOBAL 2.271 ms [2.206 ms, 2.336 ms] 785.266 µs (52.8%)
profiling 2.094 ms [2.04 ms, 2.147 ms] 607.545 µs (40.9%)
tracing 2.059 ms [2.009 ms, 2.11 ms] 573.419 µs (38.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.483 ms [1.471 ms, 1.494 ms] -
appsec 3.668 ms [3.452 ms, 3.884 ms] 2.185 ms (147.4%)
iast 2.224 ms [2.159 ms, 2.289 ms] 740.902 µs (50.0%)
iast_GLOBAL 2.269 ms [2.204 ms, 2.334 ms] 786.247 µs (53.0%)
profiling 2.48 ms [2.329 ms, 2.631 ms] 997.232 µs (67.2%)
tracing 2.051 ms [2.0 ms, 2.101 ms] 567.72 µs (38.3%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~39bf2eadcb, baseline=1.57.0-SNAPSHOT~5802e1e9f0
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.073 s) : 15073000, 15073000
.   : milestone, 15073000,
appsec (15.011 s) : 15011000, 15011000
.   : milestone, 15011000,
iast (18.097 s) : 18097000, 18097000
.   : milestone, 18097000,
iast_GLOBAL (17.755 s) : 17755000, 17755000
.   : milestone, 17755000,
profiling (14.392 s) : 14392000, 14392000
.   : milestone, 14392000,
tracing (14.726 s) : 14726000, 14726000
.   : milestone, 14726000,
section candidate
no_agent (15.774 s) : 15774000, 15774000
.   : milestone, 15774000,
appsec (14.595 s) : 14595000, 14595000
.   : milestone, 14595000,
iast (18.463 s) : 18463000, 18463000
.   : milestone, 18463000,
iast_GLOBAL (17.918 s) : 17918000, 17918000
.   : milestone, 17918000,
profiling (14.794 s) : 14794000, 14794000
.   : milestone, 14794000,
tracing (14.763 s) : 14763000, 14763000
.   : milestone, 14763000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.073 s [15.073 s, 15.073 s] -
appsec 15.011 s [15.011 s, 15.011 s] -62.0 ms (-0.4%)
iast 18.097 s [18.097 s, 18.097 s] 3.024 s (20.1%)
iast_GLOBAL 17.755 s [17.755 s, 17.755 s] 2.682 s (17.8%)
profiling 14.392 s [14.392 s, 14.392 s] -681.0 ms (-4.5%)
tracing 14.726 s [14.726 s, 14.726 s] -347.0 ms (-2.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.774 s [15.774 s, 15.774 s] -
appsec 14.595 s [14.595 s, 14.595 s] -1.179 s (-7.5%)
iast 18.463 s [18.463 s, 18.463 s] 2.689 s (17.0%)
iast_GLOBAL 17.918 s [17.918 s, 17.918 s] 2.144 s (13.6%)
profiling 14.794 s [14.794 s, 14.794 s] -980.0 ms (-6.2%)
tracing 14.763 s [14.763 s, 14.763 s] -1.011 s (-6.4%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@sarahchen6