Mark AppSecActivationSmokeTest as flaky in oracle8 #10037
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
manuel-alvarez-alvarez
added
the
tag: no release notes
utils/test-utils/src/main/groovy/datadog/trace/test/util/Predicates.java
Outdated
Show resolved
Hide resolved
Contributor
AlexeyKuznetsov-DD
left a comment
AlexeyKuznetsov-DD
left a comment
There was a problem hiding this comment.
LGTM in general, left minor comment.
Maybe we do not need this new class and can use methods from JavaVirtualMachine ?
Member
Author
|
AlexeyKuznetsov-DD
approved these changes
Nov 26, 2025
BenchmarksStartupParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 56 metrics, 8 unstable metrics.
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~c34ff34e6c, baseline=1.57.0-SNAPSHOT~912bddeb5e
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1064193
Total [baseline] (8.744 s) : 0, 8744187
Agent [candidate] (1.058 s) : 0, 1058438
Total [candidate] (8.682 s) : 0, 8682218
section iast
Agent [baseline] (1.197 s) : 0, 1196955
Total [baseline] (9.333 s) : 0, 9332828
Agent [candidate] (1.192 s) : 0, 1191992
Total [candidate] (9.31 s) : 0, 9309781
gantt
title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~c34ff34e6c, baseline=1.57.0-SNAPSHOT~912bddeb5e
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.498 ms) : 0, 1498
crashtracking [candidate] (1.504 ms) : 0, 1504
BytebuddyAgent [baseline] (716.536 ms) : 0, 716536
BytebuddyAgent [candidate] (711.812 ms) : 0, 711812
GlobalTracer [baseline] (251.129 ms) : 0, 251129
GlobalTracer [candidate] (249.921 ms) : 0, 249921
AppSec [baseline] (32.255 ms) : 0, 32255
AppSec [candidate] (32.328 ms) : 0, 32328
Debugger [baseline] (6.463 ms) : 0, 6463
Debugger [candidate] (6.412 ms) : 0, 6412
Remote Config [baseline] (684.783 µs) : 0, 685
Remote Config [candidate] (680.673 µs) : 0, 681
Telemetry [baseline] (16.403 ms) : 0, 16403
Telemetry [candidate] (15.842 ms) : 0, 15842
Flare Poller [baseline] (4.09 ms) : 0, 4090
Flare Poller [candidate] (4.941 ms) : 0, 4941
section iast
crashtracking [baseline] (1.507 ms) : 0, 1507
crashtracking [candidate] (1.49 ms) : 0, 1490
BytebuddyAgent [baseline] (833.701 ms) : 0, 833701
BytebuddyAgent [candidate] (831.205 ms) : 0, 831205
GlobalTracer [baseline] (239.218 ms) : 0, 239218
GlobalTracer [candidate] (237.997 ms) : 0, 237997
AppSec [baseline] (30.855 ms) : 0, 30855
AppSec [candidate] (28.508 ms) : 0, 28508
Debugger [baseline] (6.108 ms) : 0, 6108
Debugger [candidate] (6.009 ms) : 0, 6009
Remote Config [baseline] (632.594 µs) : 0, 633
Remote Config [candidate] (599.227 µs) : 0, 599
Telemetry [baseline] (8.022 ms) : 0, 8022
Telemetry [candidate] (7.865 ms) : 0, 7865
Flare Poller [baseline] (10.881 ms) : 0, 10881
Flare Poller [candidate] (10.756 ms) : 0, 10756
IAST [baseline] (30.968 ms) : 0, 30968
IAST [candidate] (32.721 ms) : 0, 32721
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~c34ff34e6c, baseline=1.57.0-SNAPSHOT~912bddeb5e
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.056 s) : 0, 1055510
Total [baseline] (10.871 s) : 0, 10871441
Agent [candidate] (1.059 s) : 0, 1059316
Total [candidate] (10.718 s) : 0, 10718099
section appsec
Agent [baseline] (1.231 s) : 0, 1231019
Total [baseline] (10.924 s) : 0, 10924292
Agent [candidate] (1.234 s) : 0, 1233933
Total [candidate] (10.904 s) : 0, 10903923
section iast
Agent [baseline] (1.206 s) : 0, 1206109
Total [baseline] (11.114 s) : 0, 11113705
Agent [candidate] (1.196 s) : 0, 1195931
Total [candidate] (11.123 s) : 0, 11123488
section profiling
Agent [baseline] (1.199 s) : 0, 1198966
Total [baseline] (10.818 s) : 0, 10818203
Agent [candidate] (1.202 s) : 0, 1201826
Total [candidate] (10.847 s) : 0, 10846766
gantt
title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~c34ff34e6c, baseline=1.57.0-SNAPSHOT~912bddeb5e
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.499 ms) : 0, 1499
crashtracking [candidate] (1.493 ms) : 0, 1493
BytebuddyAgent [baseline] (709.571 ms) : 0, 709571
BytebuddyAgent [candidate] (713.303 ms) : 0, 713303
GlobalTracer [baseline] (249.938 ms) : 0, 249938
GlobalTracer [candidate] (250.375 ms) : 0, 250375
AppSec [baseline] (32.17 ms) : 0, 32170
AppSec [candidate] (31.85 ms) : 0, 31850
Debugger [baseline] (6.405 ms) : 0, 6405
Debugger [candidate] (6.401 ms) : 0, 6401
Remote Config [baseline] (678.008 µs) : 0, 678
Remote Config [candidate] (669.089 µs) : 0, 669
Telemetry [baseline] (15.499 ms) : 0, 15499
Telemetry [candidate] (15.357 ms) : 0, 15357
Flare Poller [baseline] (4.879 ms) : 0, 4879
Flare Poller [candidate] (4.877 ms) : 0, 4877
section appsec
crashtracking [baseline] (1.488 ms) : 0, 1488
crashtracking [candidate] (1.5 ms) : 0, 1500
BytebuddyAgent [baseline] (734.075 ms) : 0, 734075
BytebuddyAgent [candidate] (736.741 ms) : 0, 736741
GlobalTracer [baseline] (241.353 ms) : 0, 241353
GlobalTracer [candidate] (241.396 ms) : 0, 241396
IAST [baseline] (24.878 ms) : 0, 24878
IAST [candidate] (24.727 ms) : 0, 24727
AppSec [baseline] (175.109 ms) : 0, 175109
AppSec [candidate] (175.174 ms) : 0, 175174
Debugger [baseline] (6.279 ms) : 0, 6279
Debugger [candidate] (6.313 ms) : 0, 6313
Remote Config [baseline] (692.778 µs) : 0, 693
Remote Config [candidate] (697.615 µs) : 0, 698
Telemetry [baseline] (8.11 ms) : 0, 8110
Telemetry [candidate] (8.17 ms) : 0, 8170
Flare Poller [baseline] (3.998 ms) : 0, 3998
Flare Poller [candidate] (4.024 ms) : 0, 4024
section iast
crashtracking [baseline] (1.52 ms) : 0, 1520
crashtracking [candidate] (1.482 ms) : 0, 1482
BytebuddyAgent [baseline] (842.104 ms) : 0, 842104
BytebuddyAgent [candidate] (834.208 ms) : 0, 834208
GlobalTracer [baseline] (239.348 ms) : 0, 239348
GlobalTracer [candidate] (238.637 ms) : 0, 238637
IAST [baseline] (31.608 ms) : 0, 31608
IAST [candidate] (34.583 ms) : 0, 34583
AppSec [baseline] (30.712 ms) : 0, 30712
AppSec [candidate] (26.94 ms) : 0, 26940
Debugger [baseline] (6.147 ms) : 0, 6147
Debugger [candidate] (6.044 ms) : 0, 6044
Remote Config [baseline] (620.601 µs) : 0, 621
Remote Config [candidate] (608.332 µs) : 0, 608
Telemetry [baseline] (8.139 ms) : 0, 8139
Telemetry [candidate] (7.875 ms) : 0, 7875
Flare Poller [baseline] (10.77 ms) : 0, 10770
Flare Poller [candidate] (10.702 ms) : 0, 10702
section profiling
crashtracking [baseline] (1.444 ms) : 0, 1444
crashtracking [candidate] (1.442 ms) : 0, 1442
BytebuddyAgent [baseline] (733.584 ms) : 0, 733584
BytebuddyAgent [candidate] (735.666 ms) : 0, 735666
GlobalTracer [baseline] (222.894 ms) : 0, 222894
GlobalTracer [candidate] (223.096 ms) : 0, 223096
AppSec [baseline] (32.003 ms) : 0, 32003
AppSec [candidate] (32.023 ms) : 0, 32023
Debugger [baseline] (6.821 ms) : 0, 6821
Debugger [candidate] (6.828 ms) : 0, 6828
Remote Config [baseline] (688.682 µs) : 0, 689
Remote Config [candidate] (688.976 µs) : 0, 689
Telemetry [baseline] (16.457 ms) : 0, 16457
Telemetry [candidate] (16.694 ms) : 0, 16694
Flare Poller [baseline] (4.144 ms) : 0, 4144
Flare Poller [candidate] (4.187 ms) : 0, 4187
ProfilingAgent [baseline] (111.644 ms) : 0, 111644
ProfilingAgent [candidate] (111.808 ms) : 0, 111808
Profiling [baseline] (112.304 ms) : 0, 112304
Profiling [candidate] (112.443 ms) : 0, 112443
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 2 performance regressions! Performance is the same for 17 metrics, 17 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~c34ff34e6c, baseline=1.57.0-SNAPSHOT~912bddeb5e
dateFormat X
axisFormat %s
section baseline
no_agent (1.221 ms) : 1209, 1233
. : milestone, 1221,
iast (3.215 ms) : 3172, 3257
. : milestone, 3215,
iast_FULL (5.76 ms) : 5703, 5817
. : milestone, 5760,
iast_GLOBAL (3.713 ms) : 3665, 3761
. : milestone, 3713,
profiling (2.084 ms) : 2066, 2102
. : milestone, 2084,
tracing (1.881 ms) : 1865, 1897
. : milestone, 1881,
section candidate
no_agent (1.207 ms) : 1195, 1220
. : milestone, 1207,
iast (3.452 ms) : 3401, 3504
. : milestone, 3452,
iast_FULL (5.959 ms) : 5898, 6020
. : milestone, 5959,
iast_GLOBAL (3.705 ms) : 3652, 3759
. : milestone, 3705,
profiling (2.008 ms) : 1990, 2026
. : milestone, 2008,
tracing (1.83 ms) : 1815, 1846
. : milestone, 1830,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~c34ff34e6c, baseline=1.57.0-SNAPSHOT~912bddeb5e
dateFormat X
axisFormat %s
section baseline
no_agent (18.456 ms) : 18261, 18651
. : milestone, 18456,
appsec (18.426 ms) : 18239, 18613
. : milestone, 18426,
code_origins (17.61 ms) : 17435, 17785
. : milestone, 17610,
iast (17.708 ms) : 17531, 17884
. : milestone, 17708,
profiling (18.497 ms) : 18311, 18682
. : milestone, 18497,
tracing (17.515 ms) : 17340, 17689
. : milestone, 17515,
section candidate
no_agent (19.229 ms) : 19031, 19428
. : milestone, 19229,
appsec (18.431 ms) : 18246, 18615
. : milestone, 18431,
code_origins (18.045 ms) : 17864, 18225
. : milestone, 18045,
iast (17.676 ms) : 17497, 17855
. : milestone, 17676,
profiling (18.609 ms) : 18420, 18797
. : milestone, 18609,
tracing (17.59 ms) : 17414, 17766
. : milestone, 17590,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~c34ff34e6c, baseline=1.57.0-SNAPSHOT~912bddeb5e
dateFormat X
axisFormat %s
section baseline
no_agent (1.476 ms) : 1465, 1488
. : milestone, 1476,
appsec (3.719 ms) : 3503, 3936
. : milestone, 3719,
iast (2.211 ms) : 2146, 2276
. : milestone, 2211,
iast_GLOBAL (2.258 ms) : 2193, 2324
. : milestone, 2258,
profiling (2.068 ms) : 2015, 2120
. : milestone, 2068,
tracing (2.059 ms) : 2008, 2111
. : milestone, 2059,
section candidate
no_agent (1.475 ms) : 1464, 1487
. : milestone, 1475,
appsec (3.647 ms) : 3431, 3862
. : milestone, 3647,
iast (2.208 ms) : 2143, 2272
. : milestone, 2208,
iast_GLOBAL (2.263 ms) : 2197, 2328
. : milestone, 2263,
profiling (2.505 ms) : 2342, 2669
. : milestone, 2505,
tracing (2.045 ms) : 1994, 2096
. : milestone, 2045,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~c34ff34e6c, baseline=1.57.0-SNAPSHOT~912bddeb5e
dateFormat X
axisFormat %s
section baseline
no_agent (15.086 s) : 15086000, 15086000
. : milestone, 15086000,
appsec (14.554 s) : 14554000, 14554000
. : milestone, 14554000,
iast (18.797 s) : 18797000, 18797000
. : milestone, 18797000,
iast_GLOBAL (17.881 s) : 17881000, 17881000
. : milestone, 17881000,
profiling (14.85 s) : 14850000, 14850000
. : milestone, 14850000,
tracing (14.989 s) : 14989000, 14989000
. : milestone, 14989000,
section candidate
no_agent (15.556 s) : 15556000, 15556000
. : milestone, 15556000,
appsec (14.745 s) : 14745000, 14745000
. : milestone, 14745000,
iast (18.859 s) : 18859000, 18859000
. : milestone, 18859000,
iast_GLOBAL (17.863 s) : 17863000, 17863000
. : milestone, 17863000,
profiling (14.863 s) : 14863000, 14863000
. : milestone, 14863000,
tracing (14.886 s) : 14886000, 14886000
. : milestone, 14886000,
|
Contributor
Not sure that if my understanding correct, but I can see in our code base code in groovy like this: Does it make sense? Or it is because |
Contributor
Just checked, this should work: |
utils/test-utils/src/main/groovy/datadog/trace/test/util/Predicates.java
Outdated
Show resolved
Hide resolved
PerfectSlayer
added
tag: flaky test
manuel-alvarez-alvarez
force-pushed
the
malvarez/appsec-activation-flaky-oracle8
branch
from
929aaf3 to
c34ff34
Compare
manuel-alvarez-alvarez
requested review from
claponcet,
jandro996 and
sarahchen6
and removed request for
a team
Member
Author
That's what I mentioned before, @Flaky(
condition = __spock_feature_2_0_closure2.class,
value = "Telemetry product change event flakes in oracle8"
)Which is no longer a PS: IgnoreIf works as it's groovy and receives a closure @Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE, ElementType.METHOD})
@ExtensionAnnotation(IgnoreIfExtension.class)
@Repeatable(Container.class)
public @interface IgnoreIf {
Class<? extends Closure> value();
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE, ElementType.METHOD})
public @interface Container {
IgnoreIf[] value();
}
} |
sarahchen6
approved these changes
Dec 1, 2025
AlexeyKuznetsov-DD
approved these changes
Dec 1, 2025
Contributor
PerfectSlayer
left a comment
PerfectSlayer
left a comment
There was a problem hiding this comment.
Left a minor request before merging.
Looking good otherwise 👍
Comment on lines
+111
to
+114
| public static boolean isIbm() { | ||
| return runtime.vendor.contains("IBM"); | ||
| } | ||
|
|
Contributor
There was a problem hiding this comment.
🎯 suggestion: Can you update the JavaVirtualMachineTest test suite too?
|
Looks like my github tag as accidentally been added into the PR description, could it be corrected please? 🙂 |
Contributor
|
Sorry, this was a misspelled Java annotation (they start with an |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
Marks the
AppSecActivationSmokeTestas flaky in oracle8Added support for using closures in
@Flakyannotations, alongside Predicate implementations, to simplify flaky-test filtering.Motivation
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any useful labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]