Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sampling-in-storage): preflight mode #6979

Merged
merged 18 commits into from
Mar 21, 2025
Merged

feat(sampling-in-storage): preflight mode #6979

merged 18 commits into from
Mar 21, 2025

Conversation

xurui-c
Copy link
Member

@xurui-c xurui-c commented Mar 19, 2025

Copy link

codecov bot commented Mar 19, 2025

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
219 1 218 0
View the top 1 failed test(s) by shortest run time
tests.test_generic_metrics_api.TestGenericMetricsApiDistributions::test_retrieval_basic_sampled
Stack Traces | 0.413s run time
Traceback (most recent call last):
  File ".../snuba/tests/test_generic_metrics_api.py", line 354, in test_retrieval_basic_sampled
    assert data["data"][0]["dist_sum"] == 400.0
AssertionError: assert 200.0 == 400.0

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Copy link
Member

@onkar onkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is a draft PR, but left a few comments.

@xurui-c xurui-c force-pushed the rachel/preflight branch 2 times, most recently from a705d07 to 3c08f19 Compare March 19, 2025 20:20
@xurui-c xurui-c marked this pull request as ready for review March 19, 2025 21:56
@xurui-c xurui-c requested review from a team as code owners March 19, 2025 21:56
@xurui-c xurui-c requested review from volokluev and onkar March 20, 2025 16:28
Copy link
Member

@volokluev volokluev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, let's just remove the seeded RNGs and just test that:

  1. the preflight mode returns fewer queries
  2. the response metadata is updated correctly

Copy link
Member

@onkar onkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM mostly, added a few nits. Didn't look into the test failures, but are those happening because of the use of random? Maybe better to simplify to not use random if that is the cause of the failure.

@@ -104,6 +109,17 @@
}


def add_tier_to_query_settings(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: naming could be changed to something more specific like set_sampling_tier_512 to be explicit. Also, I see that this function is called conditionally, so that condition can be handled inside this function instead of outside:

def set_sampling_tier_512(
    request: TimeSeriesRequest | TraceItemTableRequest,
    query_settings: HTTPQuerySettings,
) -> None:
    if not request.meta.HasField("downsampled_storage_config"):
        return  # No downsampled storage config, so do nothing

    if (
        request.meta.downsampled_storage_config.mode
        == DownsampledStorageConfig.MODE_PREFLIGHT
    ):
        query_settings.set_sampling_tier(Tier.TIER_512)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the mode isn't preflight we shouldn't add tier 512 to query settings. the behavior of the function depends on the mode so I didn't want to specify which tier we're setting in the name

Comment on lines 366 to 368
if request.meta.HasField("downsampled_storage_config"):
add_tier_to_query_settings(request, query_settings)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and other occurrences become:

set_sampling_tier_512(request, query_settings)

@xurui-c xurui-c requested review from volokluev and onkar March 20, 2025 20:06
Copy link
Member

@volokluev volokluev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly good., just a few comments

@xurui-c xurui-c requested a review from volokluev March 20, 2025 22:47
Copy link
Contributor

@davidtsuk davidtsuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@xurui-c xurui-c merged commit a1f72bb into master Mar 21, 2025
32 checks passed
@xurui-c xurui-c deleted the rachel/preflight branch March 21, 2025 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants