-
Notifications
You must be signed in to change notification settings - Fork 469
feat(profiling): profile asyncio.BoundedSemaphore primitives with Python Lock profiler #15532
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
|
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 245 ± 2 ms. The average import time from base is: 248 ± 2 ms. The import time difference between this PR and base is: -2.78 ± 0.09 ms. Import time breakdownThe following import paths have shrunk:
|
Performance SLOsComparing candidate vlad/lockprof-asyncio-bounded-semaphore (7e79ad4) with baseline main (ca3c521) 📈 Performance Regressions (3 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 0.402µs (SLO: <10.000µs 📉 -96.0%) vs baseline: ~same Memory: ✅ 40.187MB (SLO: <41.500MB -3.2%) vs baseline: +4.8% ✅ add_inplace_aspectTime: ✅ 0.407µs (SLO: <10.000µs 📉 -95.9%) vs baseline: +0.6% Memory: ✅ 40.187MB (SLO: <41.500MB -3.2%) vs baseline: +4.8% ✅ add_inplace_noaspectTime: ✅ 0.317µs (SLO: <10.000µs 📉 -96.8%) vs baseline: -0.7% Memory: ✅ 40.206MB (SLO: <41.500MB -3.1%) vs baseline: +4.8% ✅ add_noaspectTime: ✅ 0.280µs (SLO: <10.000µs 📉 -97.2%) vs baseline: -2.0% Memory: ✅ 40.285MB (SLO: <41.500MB -2.9%) vs baseline: +4.7% ✅ bytearray_aspectTime: ✅ 1.350µs (SLO: <10.000µs 📉 -86.5%) vs baseline: -0.5% Memory: ✅ 40.246MB (SLO: <41.500MB -3.0%) vs baseline: +4.9% ✅ bytearray_extend_aspectTime: ✅ 1.504µs (SLO: <10.000µs 📉 -85.0%) vs baseline: -0.4% Memory: ✅ 40.206MB (SLO: <41.500MB -3.1%) vs baseline: +4.7% ✅ bytearray_extend_noaspectTime: ✅ 0.614µs (SLO: <10.000µs 📉 -93.9%) vs baseline: ~same Memory: ✅ 40.206MB (SLO: <41.500MB -3.1%) vs baseline: +4.0% ✅ bytearray_noaspectTime: ✅ 0.479µs (SLO: <10.000µs 📉 -95.2%) vs baseline: +0.5% Memory: ✅ 40.265MB (SLO: <41.500MB -3.0%) vs baseline: +4.7% ✅ bytes_aspectTime: ✅ 1.304µs (SLO: <10.000µs 📉 -87.0%) vs baseline: +2.0% Memory: ✅ 40.265MB (SLO: <41.500MB -3.0%) vs baseline: +5.2% ✅ bytes_noaspectTime: ✅ 0.493µs (SLO: <10.000µs 📉 -95.1%) vs baseline: +0.6% Memory: ✅ 40.167MB (SLO: <41.500MB -3.2%) vs baseline: +3.7% ✅ bytesio_aspectTime: ✅ 1.334µs (SLO: <10.000µs 📉 -86.7%) vs baseline: +1.0% Memory: ✅ 40.246MB (SLO: <41.500MB -3.0%) vs baseline: +5.1% ✅ bytesio_noaspectTime: ✅ 0.498µs (SLO: <10.000µs 📉 -95.0%) vs baseline: +0.5% Memory: ✅ 40.324MB (SLO: <41.500MB -2.8%) vs baseline: +4.3% ✅ capitalize_aspectTime: ✅ 0.736µs (SLO: <10.000µs 📉 -92.6%) vs baseline: -0.2% Memory: ✅ 40.226MB (SLO: <41.500MB -3.1%) vs baseline: +3.9% ✅ capitalize_noaspectTime: ✅ 0.438µs (SLO: <10.000µs 📉 -95.6%) vs baseline: +0.4% Memory: ✅ 40.246MB (SLO: <41.500MB -3.0%) vs baseline: +4.8% ✅ casefold_aspectTime: ✅ 0.737µs (SLO: <10.000µs 📉 -92.6%) vs baseline: +0.1% Memory: ✅ 40.187MB (SLO: <41.500MB -3.2%) vs baseline: +4.9% ✅ casefold_noaspectTime: ✅ 0.371µs (SLO: <10.000µs 📉 -96.3%) vs baseline: +0.8% Memory: ✅ 40.285MB (SLO: <41.500MB -2.9%) vs baseline: +4.1% ✅ decode_aspectTime: ✅ 0.724µs (SLO: <10.000µs 📉 -92.8%) vs baseline: +0.3% Memory: ✅ 40.285MB (SLO: <41.500MB -2.9%) vs baseline: +5.0% ✅ decode_noaspectTime: ✅ 0.416µs (SLO: <10.000µs 📉 -95.8%) vs baseline: -1.2% Memory: ✅ 40.128MB (SLO: <41.500MB -3.3%) vs baseline: +3.7% ✅ encode_aspectTime: ✅ 0.715µs (SLO: <10.000µs 📉 -92.8%) vs baseline: +1.1% Memory: ✅ 40.246MB (SLO: <41.500MB -3.0%) vs baseline: +4.9% ✅ encode_noaspectTime: ✅ 0.403µs (SLO: <10.000µs 📉 -96.0%) vs baseline: ~same Memory: ✅ 40.246MB (SLO: <41.500MB -3.0%) vs baseline: +4.1% ✅ format_aspectTime: ✅ 3.437µs (SLO: <10.000µs 📉 -65.6%) vs baseline: -0.7% Memory: ✅ 40.305MB (SLO: <41.500MB -2.9%) vs baseline: +4.1% ✅ format_map_aspectTime: ✅ 3.543µs (SLO: <10.000µs 📉 -64.6%) vs baseline: -1.6% Memory: ✅ 40.167MB (SLO: <41.500MB -3.2%) vs baseline: +4.3% ✅ format_map_noaspectTime: ✅ 0.770µs (SLO: <10.000µs 📉 -92.3%) vs baseline: -0.5% Memory: ✅ 40.246MB (SLO: <41.500MB -3.0%) vs baseline: +4.7% ✅ format_noaspectTime: ✅ 0.596µs (SLO: <10.000µs 📉 -94.0%) vs baseline: +0.3% Memory: ✅ 40.108MB (SLO: <41.500MB -3.4%) vs baseline: +3.7% ✅ index_aspectTime: ✅ 0.358µs (SLO: <10.000µs 📉 -96.4%) vs baseline: +1.0% Memory: ✅ 40.069MB (SLO: <41.500MB -3.4%) vs baseline: +4.4% ✅ index_noaspectTime: ✅ 0.277µs (SLO: <10.000µs 📉 -97.2%) vs baseline: -0.9% Memory: ✅ 40.285MB (SLO: <41.500MB -2.9%) vs baseline: +4.3% ✅ join_aspectTime: ✅ 1.332µs (SLO: <10.000µs 📉 -86.7%) vs baseline: +0.6% Memory: ✅ 40.285MB (SLO: <41.500MB -2.9%) vs baseline: +5.1% ✅ join_noaspectTime: ✅ 0.490µs (SLO: <10.000µs 📉 -95.1%) vs baseline: +0.3% Memory: ✅ 40.285MB (SLO: <41.500MB -2.9%) vs baseline: +4.8% ✅ ljust_aspectTime: ✅ 2.611µs (SLO: <20.000µs 📉 -86.9%) vs baseline: +1.4% Memory: ✅ 40.305MB (SLO: <41.500MB -2.9%) vs baseline: +4.8% ✅ ljust_noaspectTime: ✅ 0.403µs (SLO: <10.000µs 📉 -96.0%) vs baseline: +0.3% Memory: ✅ 40.246MB (SLO: <41.500MB -3.0%) vs baseline: +4.3% ✅ lower_aspectTime: ✅ 2.300µs (SLO: <10.000µs 📉 -77.0%) vs baseline: +3.2% Memory: ✅ 40.088MB (SLO: <41.500MB -3.4%) vs baseline: +4.4% ✅ lower_noaspectTime: ✅ 0.369µs (SLO: <10.000µs 📉 -96.3%) vs baseline: -1.2% Memory: ✅ 40.147MB (SLO: <41.500MB -3.3%) vs baseline: +3.8% ✅ lstrip_aspectTime: ✅ 2.261µs (SLO: <20.000µs 📉 -88.7%) vs baseline: +0.7% Memory: ✅ 40.147MB (SLO: <41.500MB -3.3%) vs baseline: +3.9% ✅ lstrip_noaspectTime: ✅ 0.379µs (SLO: <10.000µs 📉 -96.2%) vs baseline: -0.7% Memory: ✅ 40.108MB (SLO: <41.500MB -3.4%) vs baseline: +3.5% ✅ modulo_aspectTime: ✅ 1.044µs (SLO: <10.000µs 📉 -89.6%) vs baseline: +4.1% Memory: ✅ 40.226MB (SLO: <41.500MB -3.1%) vs baseline: +5.0% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 1.534µs (SLO: <10.000µs 📉 -84.7%) vs baseline: -1.1% Memory: ✅ 40.265MB (SLO: <41.500MB -3.0%) vs baseline: +5.0% ✅ modulo_aspect_for_bytesTime: ✅ 0.976µs (SLO: <10.000µs 📉 -90.2%) vs baseline: -0.5% Memory: ✅ 40.187MB (SLO: <41.500MB -3.2%) vs baseline: +4.7% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 1.243µs (SLO: <10.000µs 📉 -87.6%) vs baseline: +1.5% Memory: ✅ 40.285MB (SLO: <41.500MB -2.9%) vs baseline: +4.9% ✅ modulo_noaspectTime: ✅ 0.624µs (SLO: <10.000µs 📉 -93.8%) vs baseline: -0.6% Memory: ✅ 40.324MB (SLO: <41.500MB -2.8%) vs baseline: +4.2% ✅ replace_aspectTime: ✅ 4.886µs (SLO: <10.000µs 📉 -51.1%) vs baseline: -0.8% Memory: ✅ 40.167MB (SLO: <41.500MB -3.2%) vs baseline: +4.4% ✅ replace_noaspectTime: ✅ 0.459µs (SLO: <10.000µs 📉 -95.4%) vs baseline: -0.5% Memory: ✅ 40.128MB (SLO: <41.500MB -3.3%) vs baseline: +3.7% ✅ repr_aspectTime: ✅ 0.904µs (SLO: <10.000µs 📉 -91.0%) vs baseline: -1.0% Memory: ✅ 40.029MB (SLO: <41.500MB -3.5%) vs baseline: +4.5% ✅ repr_noaspectTime: ✅ 0.419µs (SLO: <10.000µs 📉 -95.8%) vs baseline: +1.0% Memory: ✅ 40.147MB (SLO: <41.500MB -3.3%) vs baseline: +4.7% ✅ rstrip_aspectTime: ✅ 1.953µs (SLO: <20.000µs 📉 -90.2%) vs baseline: +1.5% Memory: ✅ 40.147MB (SLO: <41.500MB -3.3%) vs baseline: +3.6% ✅ rstrip_noaspectTime: ✅ 0.380µs (SLO: <10.000µs 📉 -96.2%) vs baseline: -0.8% Memory: ✅ 40.206MB (SLO: <41.500MB -3.1%) vs baseline: +3.9% ✅ slice_aspectTime: ✅ 0.490µs (SLO: <10.000µs 📉 -95.1%) vs baseline: -0.7% Memory: ✅ 40.088MB (SLO: <41.500MB -3.4%) vs baseline: +4.2% ✅ slice_noaspectTime: ✅ 0.448µs (SLO: <10.000µs 📉 -95.5%) vs baseline: +0.4% Memory: ✅ 40.265MB (SLO: <41.500MB -3.0%) vs baseline: +3.9% ✅ stringio_aspectTime: ✅ 1.779µs (SLO: <10.000µs 📉 -82.2%) vs baseline: 📈 +15.7% Memory: ✅ 40.226MB (SLO: <41.500MB -3.1%) vs baseline: +4.9% ✅ stringio_noaspectTime: ✅ 0.712µs (SLO: <10.000µs 📉 -92.9%) vs baseline: -0.1% Memory: ✅ 40.305MB (SLO: <41.500MB -2.9%) vs baseline: +4.1% ✅ strip_aspectTime: ✅ 2.241µs (SLO: <20.000µs 📉 -88.8%) vs baseline: +0.9% Memory: ✅ 40.206MB (SLO: <41.500MB -3.1%) vs baseline: +4.8% ✅ strip_noaspectTime: ✅ 0.381µs (SLO: <10.000µs 📉 -96.2%) vs baseline: -2.3% Memory: ✅ 40.226MB (SLO: <41.500MB -3.1%) vs baseline: +4.1% ✅ swapcase_aspectTime: ✅ 2.493µs (SLO: <10.000µs 📉 -75.1%) vs baseline: +2.1% Memory: ✅ 40.167MB (SLO: <41.500MB -3.2%) vs baseline: +4.7% ✅ swapcase_noaspectTime: ✅ 0.536µs (SLO: <10.000µs 📉 -94.6%) vs baseline: +0.2% Memory: ✅ 40.167MB (SLO: <41.500MB -3.2%) vs baseline: +3.8% ✅ title_aspectTime: ✅ 2.446µs (SLO: <10.000µs 📉 -75.5%) vs baseline: +2.8% Memory: ✅ 40.187MB (SLO: <41.500MB -3.2%) vs baseline: +5.1% ✅ title_noaspectTime: ✅ 0.500µs (SLO: <10.000µs 📉 -95.0%) vs baseline: +0.7% Memory: ✅ 40.206MB (SLO: <41.500MB -3.1%) vs baseline: +3.7% ✅ translate_aspectTime: ✅ 3.313µs (SLO: <10.000µs 📉 -66.9%) vs baseline: +0.2% Memory: ✅ 40.324MB (SLO: <41.500MB -2.8%) vs baseline: +4.9% ✅ translate_noaspectTime: ✅ 1.042µs (SLO: <10.000µs 📉 -89.6%) vs baseline: +0.1% Memory: ✅ 40.324MB (SLO: <41.500MB -2.8%) vs baseline: +4.8% ✅ upper_aspectTime: ✅ 2.316µs (SLO: <10.000µs 📉 -76.8%) vs baseline: +3.1% Memory: ✅ 40.206MB (SLO: <41.500MB -3.1%) vs baseline: +4.9% ✅ upper_noaspectTime: ✅ 0.364µs (SLO: <10.000µs 📉 -96.4%) vs baseline: -1.4% Memory: ✅ 40.324MB (SLO: <41.500MB -2.8%) vs baseline: +4.3% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 5.194µs (SLO: <10.000µs 📉 -48.1%) vs baseline: 📈 +27.6% Memory: ✅ 40.147MB (SLO: <41.000MB -2.1%) vs baseline: +4.6% ✅ ospathbasename_noaspectTime: ✅ 1.083µs (SLO: <10.000µs 📉 -89.2%) vs baseline: -0.5% Memory: ✅ 40.246MB (SLO: <41.000MB 🟡 -1.8%) vs baseline: +5.0% ✅ ospathjoin_aspectTime: ✅ 6.087µs (SLO: <10.000µs 📉 -39.1%) vs baseline: -1.6% Memory: ✅ 40.324MB (SLO: <41.000MB 🟡 -1.6%) vs baseline: +5.1% ✅ ospathjoin_noaspectTime: ✅ 2.286µs (SLO: <10.000µs 📉 -77.1%) vs baseline: -0.1% Memory: ✅ 40.206MB (SLO: <41.000MB 🟡 -1.9%) vs baseline: +4.3% ✅ ospathnormcase_aspectTime: ✅ 3.405µs (SLO: <10.000µs 📉 -65.9%) vs baseline: -0.2% Memory: ✅ 40.246MB (SLO: <41.000MB 🟡 -1.8%) vs baseline: +4.8% ✅ ospathnormcase_noaspectTime: ✅ 0.571µs (SLO: <10.000µs 📉 -94.3%) vs baseline: -0.2% Memory: ✅ 40.403MB (SLO: <41.000MB 🟡 -1.5%) vs baseline: +5.1% ✅ ospathsplit_aspectTime: ✅ 4.726µs (SLO: <10.000µs 📉 -52.7%) vs baseline: -0.4% Memory: ✅ 40.128MB (SLO: <41.000MB -2.1%) vs baseline: +4.7% ✅ ospathsplit_noaspectTime: ✅ 1.588µs (SLO: <10.000µs 📉 -84.1%) vs baseline: -0.4% Memory: ✅ 40.147MB (SLO: <41.000MB -2.1%) vs baseline: +4.4% ✅ ospathsplitdrive_aspectTime: ✅ 3.578µs (SLO: <10.000µs 📉 -64.2%) vs baseline: -1.8% Memory: ✅ 40.344MB (SLO: <41.000MB 🟡 -1.6%) vs baseline: +5.5% ✅ ospathsplitdrive_noaspectTime: ✅ 0.692µs (SLO: <10.000µs 📉 -93.1%) vs baseline: -0.4% Memory: ✅ 40.246MB (SLO: <41.000MB 🟡 -1.8%) vs baseline: +4.7% ✅ ospathsplitext_aspectTime: ✅ 4.507µs (SLO: <10.000µs 📉 -54.9%) vs baseline: -0.4% Memory: ✅ 40.069MB (SLO: <41.000MB -2.3%) vs baseline: +4.5% ✅ ospathsplitext_noaspectTime: ✅ 1.376µs (SLO: <10.000µs 📉 -86.2%) vs baseline: -1.1% Memory: ✅ 40.285MB (SLO: <41.000MB 🟡 -1.7%) vs baseline: +4.9% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.362µs (SLO: <20.000µs 📉 -83.2%) vs baseline: 📈 +15.2% Memory: ✅ 34.721MB (SLO: <35.500MB -2.2%) vs baseline: +4.6% ✅ 1-count-metrics-100-timesTime: ✅ 199.878µs (SLO: <220.000µs -9.1%) vs baseline: -0.5% Memory: ✅ 34.760MB (SLO: <35.500MB -2.1%) vs baseline: +5.0% ✅ 1-distribution-metric-1-timesTime: ✅ 3.270µs (SLO: <20.000µs 📉 -83.6%) vs baseline: -0.1% Memory: ✅ 34.800MB (SLO: <35.500MB 🟡 -2.0%) vs baseline: +5.1% ✅ 1-distribution-metrics-100-timesTime: ✅ 217.633µs (SLO: <230.000µs -5.4%) vs baseline: +0.1% Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +5.0% ✅ 1-gauge-metric-1-timesTime: ✅ 2.168µs (SLO: <20.000µs 📉 -89.2%) vs baseline: ~same Memory: ✅ 34.780MB (SLO: <35.500MB -2.0%) vs baseline: +5.0% ✅ 1-gauge-metrics-100-timesTime: ✅ 135.644µs (SLO: <150.000µs -9.6%) vs baseline: -0.6% Memory: ✅ 34.741MB (SLO: <35.500MB -2.1%) vs baseline: +4.7% ✅ 1-rate-metric-1-timesTime: ✅ 3.081µs (SLO: <20.000µs 📉 -84.6%) vs baseline: +0.4% Memory: ✅ 34.780MB (SLO: <35.500MB -2.0%) vs baseline: +4.6% ✅ 1-rate-metrics-100-timesTime: ✅ 214.949µs (SLO: <250.000µs 📉 -14.0%) vs baseline: +0.2% Memory: ✅ 34.721MB (SLO: <35.500MB -2.2%) vs baseline: +4.9% ✅ 100-count-metrics-100-timesTime: ✅ 20.473ms (SLO: <22.000ms -6.9%) vs baseline: +0.3% Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +5.0% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.297ms (SLO: <2.550ms -9.9%) vs baseline: -0.3% Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +5.0% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.410ms (SLO: <1.550ms -9.0%) vs baseline: ~same Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +5.1% ✅ 100-rate-metrics-100-timesTime: ✅ 2.226ms (SLO: <2.550ms 📉 -12.7%) vs baseline: +0.3% Memory: ✅ 34.800MB (SLO: <35.500MB 🟡 -2.0%) vs baseline: +4.9% ✅ flush-1-metricTime: ✅ 4.590µs (SLO: <20.000µs 📉 -77.0%) vs baseline: -0.7% Memory: ✅ 35.075MB (SLO: <35.500MB 🟡 -1.2%) vs baseline: +5.0% ✅ flush-100-metricsTime: ✅ 173.579µs (SLO: <250.000µs 📉 -30.6%) vs baseline: -0.1% Memory: ✅ 35.075MB (SLO: <35.500MB 🟡 -1.2%) vs baseline: +4.6% ✅ flush-1000-metricsTime: ✅ 2.185ms (SLO: <2.500ms 📉 -12.6%) vs baseline: +0.1% Memory: ✅ 35.940MB (SLO: <36.500MB 🟡 -1.5%) vs baseline: +4.8% 🟡 Near SLO Breach (17 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
98663f9 to
8b01f85
Compare
8b01f85 to
578d2ec
Compare
…hon Lock profiler
578d2ec to
7e79ad4
Compare
https://datadoghq.atlassian.net/browse/PROF-13186
Description
Add support for profiling
asyncio.BoundedSemaphoreobjects in the Python Lock profiler.Changes
note: Follows the same pattern as
threading.BoundedSemaphoreprofiling (PR #15368)AsyncioBoundedSemaphoreCollectorforasyncio.BoundedSemaphoreprofilingTestAsyncioBoundedSemaphoreCollectorfollowing the established test hierarchy patternValueErroris raised when releasing beyond initial valueWhy no internal lock detection?
Unlike
threading.BoundedSemaphorewhich internally uses athreading.LockviaCondition,asyncio.BoundedSemaphoredoes not compose other asyncio primitives internally. This simplifies the implementation - nois_internaldetection needed.Testing
TestAsyncioBoundedSemaphoreCollectorinheriting fromBaseAsyncioLockCollectorTesttest_bounded_behavior_preservedto verifyValueErroron excess releasetest_collector_reprto includeAsyncioBoundedSemaphoreCollector