-
Notifications
You must be signed in to change notification settings - Fork 10
ref: add retention policy, remove old CAs, stop cache rollups in new TA #444
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
CodSpeed Performance ReportMerging #444 will not alter performanceComparing Summary
Footnotes |
a0540ec to
906532b
Compare
libs/shared/shared/django_apps/ta_timeseries/migrations/0034_remove_old_summary_caggs.py
Outdated
Show resolved
Hide resolved
906532b to
d5072f0
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #444 +/- ##
==========================================
+ Coverage 94.14% 94.24% +0.09%
==========================================
Files 1276 1277 +1
Lines 47221 47089 -132
Branches 1509 1509
==========================================
- Hits 44458 44380 -78
+ Misses 2456 2402 -54
Partials 307 307
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
ElioDiNino
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lovely cleanup 🧹
d5072f0 to
20ff74a
Compare
1861d44 to
8e116aa
Compare
7af87b1 to
25bf7a3
Compare
25bf7a3 to
440a2bc
Compare
we remove the old CAs that are no longer being used in the GQL api since we’re using the new ones exclusively the hourly CAs are just used as a “bridge” to the daily CAs so we do some intermediary deduplication within the hour, just so we amortize the cost of aggregation a bit more throughout the day, so we don’t have to retain them for long
we don’t need to cache rollups in the new pipeline anymore since the API is consuming from timescale directly So we’re removing all code related to caching rollups in the timescale implementation of the TA pipeline
440a2bc to
6cd56a2
Compare
we remove the old CAs that are no longer being used in the GQL api since
we’re using the new ones exclusively
the hourly CAs are just used as a “bridge” to the daily CAs so we do
some intermediary deduplication within the hour, just so we amortize the
cost of aggregation a bit more throughout the day, so we don’t have to
retain them for long
we don’t need to cache rollups in the new pipeline anymore since the API
is consuming from timescale directly
So we’re removing all code related to caching rollups in the timescale
implementation of the TA pipeline