Measure `Agent.start` for startup benchmarks #10090

sarahchen6 · 2025-12-05T17:58:46Z

What Does This Do

Measure Agent.start instead of GlobalTracer for startup benchmarks. Adjust the SLO values accordingly as well.

Motivation

Agent.start is a more accurate measurement of our start process because it includes all product behavior that delays the startup of the main application, whereas GlobalTracer is just one phase in the startup period.

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

bric3

Thanks @sarahchen6

bric3 · 2025-12-05T18:09:20Z

.gitlab/benchmarks/bp-runner.fail-on-breach.yml

+          # https://benchmarking.us1.prod.dog/trends?projectId=4&branch=master&trendsTab=per_scenario&scenario=startup%3Apetclinic%3Aprofiling%3AAgent.start&trendsType=scenario
+          # https://benchmarking.us1.prod.dog/trends?projectId=4&branch=master&trendsTab=per_scenario&scenario=startup%3Apetclinic%3Aappsec%3AAgent.start&trendsType=scenario
+          # https://benchmarking.us1.prod.dog/trends?projectId=4&branch=master&trendsTab=per_scenario&scenario=startup%3Apetclinic%3Aiast%3AAgent.start&trendsType=scenario
+          - name: "startup:petclinic:(profiling|appsec|iast):Agent.start"


note: FYI this string is interpreted as a regex: startup:petclinic:(profiling|appsec|iast):Agent.start

pr-commenter · 2025-12-05T18:49:05Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/change-startup-to-measure-agent-start
git_commit_date	1764948592	1764957407
git_commit_sha	`89639f7`	`498bf56`
release_version	1.57.0-SNAPSHOT~89639f7e8c	1.57.0-SNAPSHOT~498bf5659b

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1764959323	1764959323
ci_job_id	1273100398	1273100398
ci_pipeline_id	85049332	85049332
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-ctb9xv1l 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-ctb9xv1l 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 8 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~498bf5659b, baseline=1.57.0-SNAPSHOT~89639f7e8c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.081 s) : 0, 1081199
Total [baseline] (8.748 s) : 0, 8748201
Agent [candidate] (1.088 s) : 0, 1087517
Total [candidate] (8.796 s) : 0, 8795812
section iast
Agent [baseline] (1.223 s) : 0, 1222718
Total [baseline] (9.514 s) : 0, 9513864
Agent [candidate] (1.221 s) : 0, 1220672
Total [candidate] (9.443 s) : 0, 9443113

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.081 s	-
Agent	iast	1.223 s	141.519 ms (13.1%)
Total	tracing	8.748 s	-
Total	iast	9.514 s	765.663 ms (8.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.088 s	-
Agent	iast	1.221 s	133.155 ms (12.2%)
Total	tracing	8.796 s	-
Total	iast	9.443 s	647.301 ms (7.4%)

gantt
    title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~498bf5659b, baseline=1.57.0-SNAPSHOT~89639f7e8c

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.19 ms) : 0, 1190
crashtracking [candidate] (1.212 ms) : 0, 1212
BytebuddyAgent [baseline] (649.027 ms) : 0, 649027
BytebuddyAgent [candidate] (653.527 ms) : 0, 653527
GlobalTracer [baseline] (282.083 ms) : 0, 282083
GlobalTracer [candidate] (283.923 ms) : 0, 283923
AppSec [baseline] (32.487 ms) : 0, 32487
AppSec [candidate] (32.393 ms) : 0, 32393
Debugger [baseline] (67.459 ms) : 0, 67459
Debugger [candidate] (67.214 ms) : 0, 67214
Remote Config [baseline] (651.649 µs) : 0, 652
Remote Config [candidate] (646.873 µs) : 0, 647
Telemetry [baseline] (8.934 ms) : 0, 8934
Telemetry [candidate] (9.074 ms) : 0, 9074
Flare Poller [baseline] (3.76 ms) : 0, 3760
Flare Poller [candidate] (3.815 ms) : 0, 3815
section iast
crashtracking [baseline] (1.187 ms) : 0, 1187
crashtracking [candidate] (1.187 ms) : 0, 1187
BytebuddyAgent [baseline] (790.39 ms) : 0, 790390
BytebuddyAgent [candidate] (789.572 ms) : 0, 789572
GlobalTracer [baseline] (255.347 ms) : 0, 255347
GlobalTracer [candidate] (254.855 ms) : 0, 254855
AppSec [baseline] (35.35 ms) : 0, 35350
AppSec [candidate] (35.518 ms) : 0, 35518
Debugger [baseline] (65.327 ms) : 0, 65327
Debugger [candidate] (64.647 ms) : 0, 64647
Remote Config [baseline] (574.459 µs) : 0, 574
Remote Config [candidate] (558.351 µs) : 0, 558
Telemetry [baseline] (8.617 ms) : 0, 8617
Telemetry [candidate] (8.413 ms) : 0, 8413
Flare Poller [baseline] (3.471 ms) : 0, 3471
Flare Poller [candidate] (3.42 ms) : 0, 3420
IAST [baseline] (27.007 ms) : 0, 27007
IAST [candidate] (26.936 ms) : 0, 26936

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~498bf5659b, baseline=1.57.0-SNAPSHOT~89639f7e8c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.082 s) : 0, 1081698
Total [baseline] (10.833 s) : 0, 10833392
Agent [candidate] (1.079 s) : 0, 1078604
Total [candidate] (10.844 s) : 0, 10844023
section appsec
Agent [baseline] (1.263 s) : 0, 1263021
Total [baseline] (11.15 s) : 0, 11150182
Agent [candidate] (1.272 s) : 0, 1272237
Total [candidate] (11.18 s) : 0, 11179598
section iast
Agent [baseline] (1.226 s) : 0, 1226137
Total [baseline] (11.252 s) : 0, 11252426
Agent [candidate] (1.229 s) : 0, 1229222
Total [candidate] (11.31 s) : 0, 11310455
section profiling
Agent [baseline] (1.206 s) : 0, 1206438
Total [baseline] (11.198 s) : 0, 11198328
Agent [candidate] (1.206 s) : 0, 1206451
Total [candidate] (11.022 s) : 0, 11022029

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.082 s	-
Agent	appsec	1.263 s	181.323 ms (16.8%)
Agent	iast	1.226 s	144.439 ms (13.4%)
Agent	profiling	1.206 s	124.74 ms (11.5%)
Total	tracing	10.833 s	-
Total	appsec	11.15 s	316.789 ms (2.9%)
Total	iast	11.252 s	419.034 ms (3.9%)
Total	profiling	11.198 s	364.935 ms (3.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.079 s	-
Agent	appsec	1.272 s	193.633 ms (18.0%)
Agent	iast	1.229 s	150.618 ms (14.0%)
Agent	profiling	1.206 s	127.847 ms (11.9%)
Total	tracing	10.844 s	-
Total	appsec	11.18 s	335.575 ms (3.1%)
Total	iast	11.31 s	466.431 ms (4.3%)
Total	profiling	11.022 s	178.005 ms (1.6%)

gantt
    title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~498bf5659b, baseline=1.57.0-SNAPSHOT~89639f7e8c

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.194 ms) : 0, 1194
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (648.803 ms) : 0, 648803
BytebuddyAgent [candidate] (646.125 ms) : 0, 646125
GlobalTracer [baseline] (282.346 ms) : 0, 282346
GlobalTracer [candidate] (281.856 ms) : 0, 281856
AppSec [baseline] (32.462 ms) : 0, 32462
AppSec [candidate] (32.36 ms) : 0, 32360
Debugger [baseline] (67.999 ms) : 0, 67999
Debugger [candidate] (67.963 ms) : 0, 67963
Remote Config [baseline] (648.154 µs) : 0, 648
Remote Config [candidate] (641.274 µs) : 0, 641
Telemetry [baseline] (8.99 ms) : 0, 8990
Telemetry [candidate] (9.179 ms) : 0, 9179
Flare Poller [baseline] (3.659 ms) : 0, 3659
Flare Poller [candidate] (3.703 ms) : 0, 3703
section appsec
crashtracking [baseline] (1.199 ms) : 0, 1199
crashtracking [candidate] (1.198 ms) : 0, 1198
BytebuddyAgent [baseline] (688.145 ms) : 0, 688145
BytebuddyAgent [candidate] (691.786 ms) : 0, 691786
GlobalTracer [baseline] (258.736 ms) : 0, 258736
GlobalTracer [candidate] (262.124 ms) : 0, 262124
IAST [baseline] (24.498 ms) : 0, 24498
IAST [candidate] (24.809 ms) : 0, 24809
AppSec [baseline] (174.336 ms) : 0, 174336
AppSec [candidate] (175.006 ms) : 0, 175006
Debugger [baseline] (66.711 ms) : 0, 66711
Debugger [candidate] (67.869 ms) : 0, 67869
Remote Config [baseline] (703.297 µs) : 0, 703
Remote Config [candidate] (692.513 µs) : 0, 693
Telemetry [baseline] (9.073 ms) : 0, 9073
Telemetry [candidate] (9.154 ms) : 0, 9154
Flare Poller [baseline] (3.956 ms) : 0, 3956
Flare Poller [candidate] (3.893 ms) : 0, 3893
section iast
crashtracking [baseline] (1.188 ms) : 0, 1188
crashtracking [candidate] (1.194 ms) : 0, 1194
BytebuddyAgent [baseline] (792.168 ms) : 0, 792168
BytebuddyAgent [candidate] (795.166 ms) : 0, 795166
GlobalTracer [baseline] (255.635 ms) : 0, 255635
GlobalTracer [candidate] (255.478 ms) : 0, 255478
IAST [baseline] (27.857 ms) : 0, 27857
IAST [candidate] (27.047 ms) : 0, 27047
AppSec [baseline] (34.81 ms) : 0, 34810
AppSec [candidate] (35.816 ms) : 0, 35816
Debugger [baseline] (66.464 ms) : 0, 66464
Debugger [candidate] (66.531 ms) : 0, 66531
Remote Config [baseline] (556.997 µs) : 0, 557
Remote Config [candidate] (552.514 µs) : 0, 553
Telemetry [baseline] (8.516 ms) : 0, 8516
Telemetry [candidate] (8.443 ms) : 0, 8443
Flare Poller [baseline] (3.505 ms) : 0, 3505
Flare Poller [candidate] (3.463 ms) : 0, 3463
section profiling
crashtracking [baseline] (1.184 ms) : 0, 1184
crashtracking [candidate] (1.188 ms) : 0, 1188
BytebuddyAgent [baseline] (702.38 ms) : 0, 702380
BytebuddyAgent [candidate] (702.268 ms) : 0, 702268
GlobalTracer [baseline] (221.195 ms) : 0, 221195
GlobalTracer [candidate] (221.267 ms) : 0, 221267
AppSec [baseline] (32.184 ms) : 0, 32184
AppSec [candidate] (32.502 ms) : 0, 32502
Debugger [baseline] (68.884 ms) : 0, 68884
Debugger [candidate] (68.098 ms) : 0, 68098
Remote Config [baseline] (630.672 µs) : 0, 631
Remote Config [candidate] (648.562 µs) : 0, 649
Telemetry [baseline] (9.067 ms) : 0, 9067
Telemetry [candidate] (9.063 ms) : 0, 9063
Flare Poller [baseline] (3.794 ms) : 0, 3794
Flare Poller [candidate] (3.768 ms) : 0, 3768
ProfilingAgent [baseline] (97.451 ms) : 0, 97451
ProfilingAgent [candidate] (97.872 ms) : 0, 97872
Profiling [baseline] (98.03 ms) : 0, 98030
Profiling [candidate] (98.464 ms) : 0, 98464

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/change-startup-to-measure-agent-start
git_commit_date	1764948592	1764957407
git_commit_sha	`89639f7`	`498bf56`
release_version	1.57.0-SNAPSHOT~89639f7e8c	1.57.0-SNAPSHOT~498bf5659b

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1764959882	1764959882
ci_job_id	1273100399	1273100399
ci_pipeline_id	85049332	85049332
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-zff2iv5n 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-zff2iv5n 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 20 metrics, 15 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:petclinic:no_agent:high_load	better [-2.006ms; -0.436ms] or [-10.588%; -2.301%]	same [-2.940ms; +0.152ms] or [-9.380%; +0.486%]	unstable [-12.128op/s; +41.815op/s] or [-5.040%; +17.378%]	17.729ms	29.954ms	255.469op/s	18.950ms	31.348ms	240.625op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~498bf5659b, baseline=1.57.0-SNAPSHOT~89639f7e8c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.195 ms) : 1183, 1207
.   : milestone, 1195,
iast (3.287 ms) : 3245, 3329
.   : milestone, 3287,
iast_FULL (5.802 ms) : 5743, 5861
.   : milestone, 5802,
iast_GLOBAL (3.654 ms) : 3600, 3707
.   : milestone, 3654,
profiling (2.038 ms) : 2019, 2058
.   : milestone, 2038,
tracing (1.814 ms) : 1798, 1829
.   : milestone, 1814,
section candidate
no_agent (1.202 ms) : 1190, 1213
.   : milestone, 1202,
iast (3.173 ms) : 3136, 3209
.   : milestone, 3173,
iast_FULL (5.677 ms) : 5621, 5734
.   : milestone, 5677,
iast_GLOBAL (3.691 ms) : 3639, 3744
.   : milestone, 3691,
profiling (1.92 ms) : 1904, 1936
.   : milestone, 1920,
tracing (1.817 ms) : 1803, 1831
.   : milestone, 1817,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.195 ms [1.183 ms, 1.207 ms]	-
iast	3.287 ms [3.245 ms, 3.329 ms]	2.092 ms (175.0%)
iast_FULL	5.802 ms [5.743 ms, 5.861 ms]	4.607 ms (385.4%)
iast_GLOBAL	3.654 ms [3.6 ms, 3.707 ms]	2.458 ms (205.7%)
profiling	2.038 ms [2.019 ms, 2.058 ms]	843.112 µs (70.5%)
tracing	1.814 ms [1.798 ms, 1.829 ms]	618.387 µs (51.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.202 ms [1.19 ms, 1.213 ms]	-
iast	3.173 ms [3.136 ms, 3.209 ms]	1.971 ms (164.0%)
iast_FULL	5.677 ms [5.621 ms, 5.734 ms]	4.475 ms (372.4%)
iast_GLOBAL	3.691 ms [3.639 ms, 3.744 ms]	2.489 ms (207.1%)
profiling	1.92 ms [1.904 ms, 1.936 ms]	717.77 µs (59.7%)
tracing	1.817 ms [1.803 ms, 1.831 ms]	615.127 µs (51.2%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~498bf5659b, baseline=1.57.0-SNAPSHOT~89639f7e8c
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.401 ms) : 19204, 19598
.   : milestone, 19401,
appsec (19.277 ms) : 19080, 19475
.   : milestone, 19277,
code_origins (17.799 ms) : 17622, 17976
.   : milestone, 17799,
iast (17.725 ms) : 17546, 17905
.   : milestone, 17725,
profiling (18.546 ms) : 18358, 18734
.   : milestone, 18546,
tracing (17.421 ms) : 17252, 17590
.   : milestone, 17421,
section candidate
no_agent (18.271 ms) : 18082, 18461
.   : milestone, 18271,
appsec (18.673 ms) : 18485, 18860
.   : milestone, 18673,
code_origins (17.708 ms) : 17535, 17881
.   : milestone, 17708,
iast (17.625 ms) : 17452, 17798
.   : milestone, 17625,
profiling (18.673 ms) : 18488, 18858
.   : milestone, 18673,
tracing (17.626 ms) : 17451, 17802
.   : milestone, 17626,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.401 ms [19.204 ms, 19.598 ms]	-
appsec	19.277 ms [19.08 ms, 19.475 ms]	-123.851 µs (-0.6%)
code_origins	17.799 ms [17.622 ms, 17.976 ms]	-1.602 ms (-8.3%)
iast	17.725 ms [17.546 ms, 17.905 ms]	-1.676 ms (-8.6%)
profiling	18.546 ms [18.358 ms, 18.734 ms]	-855.003 µs (-4.4%)
tracing	17.421 ms [17.252 ms, 17.59 ms]	-1.98 ms (-10.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.271 ms [18.082 ms, 18.461 ms]	-
appsec	18.673 ms [18.485 ms, 18.86 ms]	401.29 µs (2.2%)
code_origins	17.708 ms [17.535 ms, 17.881 ms]	-563.484 µs (-3.1%)
iast	17.625 ms [17.452 ms, 17.798 ms]	-646.398 µs (-3.5%)
profiling	18.673 ms [18.488 ms, 18.858 ms]	401.638 µs (2.2%)
tracing	17.626 ms [17.451 ms, 17.802 ms]	-644.883 µs (-3.5%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/change-startup-to-measure-agent-start
git_commit_date	1764948592	1764957407
git_commit_sha	`89639f7`	`498bf56`
release_version	1.57.0-SNAPSHOT~89639f7e8c	1.57.0-SNAPSHOT~498bf5659b

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1764959620	1764959620
ci_job_id	1273100400	1273100400
ci_pipeline_id	85049332	85049332
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-z0xdc7y9 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-z0xdc7y9 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~498bf5659b, baseline=1.57.0-SNAPSHOT~89639f7e8c
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.952 s) : 14952000, 14952000
.   : milestone, 14952000,
appsec (14.635 s) : 14635000, 14635000
.   : milestone, 14635000,
iast (18.563 s) : 18563000, 18563000
.   : milestone, 18563000,
iast_GLOBAL (17.921 s) : 17921000, 17921000
.   : milestone, 17921000,
profiling (14.691 s) : 14691000, 14691000
.   : milestone, 14691000,
tracing (14.796 s) : 14796000, 14796000
.   : milestone, 14796000,
section candidate
no_agent (15.309 s) : 15309000, 15309000
.   : milestone, 15309000,
appsec (14.399 s) : 14399000, 14399000
.   : milestone, 14399000,
iast (18.392 s) : 18392000, 18392000
.   : milestone, 18392000,
iast_GLOBAL (18.165 s) : 18165000, 18165000
.   : milestone, 18165000,
profiling (14.583 s) : 14583000, 14583000
.   : milestone, 14583000,
tracing (15.0 s) : 15000000, 15000000
.   : milestone, 15000000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.952 s [14.952 s, 14.952 s]	-
appsec	14.635 s [14.635 s, 14.635 s]	-317.0 ms (-2.1%)
iast	18.563 s [18.563 s, 18.563 s]	3.611 s (24.2%)
iast_GLOBAL	17.921 s [17.921 s, 17.921 s]	2.969 s (19.9%)
profiling	14.691 s [14.691 s, 14.691 s]	-261.0 ms (-1.7%)
tracing	14.796 s [14.796 s, 14.796 s]	-156.0 ms (-1.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.309 s [15.309 s, 15.309 s]	-
appsec	14.399 s [14.399 s, 14.399 s]	-910.0 ms (-5.9%)
iast	18.392 s [18.392 s, 18.392 s]	3.083 s (20.1%)
iast_GLOBAL	18.165 s [18.165 s, 18.165 s]	2.856 s (18.7%)
profiling	14.583 s [14.583 s, 14.583 s]	-726.0 ms (-4.7%)
tracing	15.0 s [15.0 s, 15.0 s]	-309.0 ms (-2.0%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~498bf5659b, baseline=1.57.0-SNAPSHOT~89639f7e8c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.478 ms) : 1467, 1490
.   : milestone, 1478,
appsec (3.716 ms) : 3496, 3936
.   : milestone, 3716,
iast (2.207 ms) : 2142, 2272
.   : milestone, 2207,
iast_GLOBAL (2.252 ms) : 2187, 2317
.   : milestone, 2252,
profiling (2.083 ms) : 2029, 2137
.   : milestone, 2083,
tracing (2.043 ms) : 1992, 2094
.   : milestone, 2043,
section candidate
no_agent (1.474 ms) : 1463, 1486
.   : milestone, 1474,
appsec (3.712 ms) : 3493, 3932
.   : milestone, 3712,
iast (2.217 ms) : 2152, 2282
.   : milestone, 2217,
iast_GLOBAL (2.261 ms) : 2195, 2326
.   : milestone, 2261,
profiling (2.097 ms) : 2042, 2151
.   : milestone, 2097,
tracing (2.05 ms) : 1999, 2102
.   : milestone, 2050,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.478 ms [1.467 ms, 1.49 ms]	-
appsec	3.716 ms [3.496 ms, 3.936 ms]	2.237 ms (151.4%)
iast	2.207 ms [2.142 ms, 2.272 ms]	728.717 µs (49.3%)
iast_GLOBAL	2.252 ms [2.187 ms, 2.317 ms]	773.533 µs (52.3%)
profiling	2.083 ms [2.029 ms, 2.137 ms]	604.819 µs (40.9%)
tracing	2.043 ms [1.992 ms, 2.094 ms]	564.581 µs (38.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.474 ms [1.463 ms, 1.486 ms]	-
appsec	3.712 ms [3.493 ms, 3.932 ms]	2.238 ms (151.8%)
iast	2.217 ms [2.152 ms, 2.282 ms]	742.511 µs (50.4%)
iast_GLOBAL	2.261 ms [2.195 ms, 2.326 ms]	786.569 µs (53.4%)
profiling	2.097 ms [2.042 ms, 2.151 ms]	622.632 µs (42.2%)
tracing	2.05 ms [1.999 ms, 2.102 ms]	576.163 µs (39.1%)

Measure Agent.start for startup benchmarks

498bf56

sarahchen6 temporarily deployed to ci-build December 5, 2025 17:58 — with GitHub Actions Inactive

bric3 approved these changes Dec 5, 2025

View reviewed changes

sarahchen6 added type: enhancement Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Dec 5, 2025

sarahchen6 marked this pull request as ready for review December 5, 2025 19:51

sarahchen6 requested a review from a team as a code owner December 5, 2025 19:51

sarahchen6 requested review from TonyCTHsu and removed request for a team December 5, 2025 19:51

sarahchen6 merged commit 5802e1e into master Dec 5, 2025
551 checks passed

sarahchen6 deleted the sarahchen6/change-startup-to-measure-agent-start branch December 5, 2025 19:51

github-actions bot added this to the 1.57.0 milestone Dec 5, 2025

sarahchen6 mentioned this pull request Dec 5, 2025

Add missing space #10093

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Measure `Agent.start` for startup benchmarks #10090

Measure `Agent.start` for startup benchmarks #10090

Uh oh!

sarahchen6 commented Dec 5, 2025 •

edited

Loading

Uh oh!

bric3 left a comment

Uh oh!

bric3 Dec 5, 2025

Uh oh!

pr-commenter bot commented Dec 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Measure Agent.start for startup benchmarks #10090

Measure Agent.start for startup benchmarks #10090

Uh oh!

Conversation

sarahchen6 commented Dec 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

bric3 left a comment

Choose a reason for hiding this comment

Uh oh!

bric3 Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

pr-commenter bot commented Dec 5, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Measure `Agent.start` for startup benchmarks #10090

Measure `Agent.start` for startup benchmarks #10090

sarahchen6 commented Dec 5, 2025 •

edited

Loading