1
1
---
2
2
title : Sampling
3
3
description :
4
- Learn about sampling, and the different sampling options available in
4
+ Learn about sampling and the different sampling options available in
5
5
OpenTelemetry.
6
6
weight : 80
7
7
---
8
8
9
- With distributed tracing, you observe requests as they move from one service to
10
- another in a distributed system. It’ s superbly practical for a number of
9
+ With distributed tracing, you can observe requests as they move from one service
10
+ to another in a distributed system. It' s superbly practical for a number of
11
11
reasons, such as understanding your service connections and diagnosing latency
12
12
issues, among many other benefits.
13
13
14
- However, if the majority of all your requests are successful 200s and finish
15
- without unacceptable latency or errors, do you really need all that data? Here’s
16
- the thing—you don’t always need a ton of data to find the right insights. _ You
17
- just need the right sampling of data._
14
+ However, if the majority of your requests are successful 200s and finish without
15
+ unacceptable latency or errors, do you really need all that data? Here’s the
16
+ thing—you don’t always need a ton of data to find the right insights. _ You just
17
+ need the right sampling of data._
18
18
19
19
![ Illustration shows that not all data needs to be traced, and that a sample of data is sufficient.] ( traces-venn-diagram.svg )
20
20
21
21
The idea behind sampling is to control the spans you send to your observability
22
22
backend, resulting in lower ingest costs. Different organizations will have
23
- their own reasons for not just _ why_ they want to sample, but also _ what_ they
23
+ their own reasons for not just _ why_ they want to sample but also _ what_ they
24
24
want to sample. You might want to customize your sampling strategy to:
25
25
26
26
- ** Manage costs** : If you have a high volume of telemetry, you risk incurring
27
27
heavy charges from a telemetry backend vendor or cloud provider to export and
28
28
store every span.
29
- - ** Focus on interesting traces** : For example, your frontend team may only want
30
- to see traces with specific user attributes.
31
- - ** Filter out noise** : For example, you may want to filter out health checks.
29
+ - ** Focus on interesting traces** : For example, your frontend team might only
30
+ want to see traces with specific user attributes.
31
+ - ** Filter out noise** : For example, you might want to filter out health checks.
32
32
33
33
## Terminology
34
34
@@ -41,8 +41,8 @@ or span is considered "sampled" or "not sampled":
41
41
- ** Not sampled** : A trace or span is not processed or exported. Because it is
42
42
not chosen by the sampler, it is considered "not sampled".
43
43
44
- Sometimes, the definitions of these terms get mixed up. You may find someone
45
- state that they are "sampling out data" or that data not processed or exported
44
+ Sometimes, the definitions of these terms get mixed up. You might find someone
45
+ states that they are "sampling out data" or that data not processed or exported
46
46
is considered "sampled". These are incorrect statements.
47
47
48
48
## Head Sampling
@@ -53,8 +53,8 @@ inspecting the trace as a whole.
53
53
54
54
For example, the most common form of head sampling is
55
55
[ Consistent Probability Sampling] ( /docs/specs/otel/trace/tracestate-probability-sampling/#consistent-probability-sampling ) .
56
- It may also be referred to as Deterministic Sampling. In this case, a sampling
57
- decision is made based on the trace ID and a desired percentage of traces to
56
+ This is also be referred to as Deterministic Sampling. In this case, a sampling
57
+ decision is made based on the trace ID and the desired percentage of traces to
58
58
sample. This ensures that whole traces are sampled - no missing spans - at a
59
59
consistent rate, such as 5% of all traces.
60
60
@@ -65,12 +65,12 @@ The upsides to head sampling are:
65
65
- Efficient
66
66
- Can be done at any point in the trace collection pipeline
67
67
68
- The primary downside to head sampling is that it is not possible make a sampling
69
- decision based on data in the entire trace. This means that head sampling is
70
- effective as a blunt instrument, but is wholly insufficient for sampling
71
- strategies that must take whole-system information into account. For example, it
72
- is not possible to use head sampling to ensure that all traces with an error
73
- within them are sampled. For this, you need Tail Sampling.
68
+ The primary downside to head sampling is that it is not possible to make a
69
+ sampling decision based on data in the entire trace. This means that head
70
+ sampling is effective as a blunt instrument, but is wholly insufficient for
71
+ sampling strategies that must take whole-system information into account. For
72
+ example, it is not possible to use head sampling to ensure that all traces with
73
+ an error within them are sampled. For this, you need Tail Sampling.
74
74
75
75
## Tail Sampling
76
76
@@ -92,7 +92,7 @@ Some examples of how you can use Tail Sampling include:
92
92
93
93
As you can see, tail sampling allows for a much higher degree of sophistication.
94
94
For larger systems that must sample telemetry, it is almost always necessary to
95
- use Tail Sampling to balance data volume with usefulness of that data.
95
+ use Tail Sampling to balance data volume with the usefulness of that data.
96
96
97
97
There are three primary downsides to tail sampling today:
98
98
@@ -105,19 +105,19 @@ There are three primary downsides to tail sampling today:
105
105
tail sampling must be stateful systems that can accept and store a large
106
106
amount of data. Depending on traffic patterns, this can require dozens or even
107
107
hundreds of nodes that all utilize resources differently. Furthermore, a tail
108
- sampler may need to "fall back" to less computationally- intensive sampling
108
+ sampler might need to "fall back" to less computationally intensive sampling
109
109
techniques if it is unable to keep up with the volume of data it is receiving.
110
- Because of these factors, it is critical to monitor tail sampling components
110
+ Because of these factors, it is critical to monitor tail- sampling components
111
111
to ensure that they have the resources they need to make the correct sampling
112
112
decisions.
113
113
- Tail samplers often end up being in the domain of vendor-specific technology
114
114
today. If you're using a paid vendor for Observability, the most effective
115
- tail sampling options available to you may be limited to what the vendor
115
+ tail sampling options available to you might be limited to what the vendor
116
116
offers.
117
117
118
- Finally, for some systems, tail sampling may be used in conjunction with Head
118
+ Finally, for some systems, tail sampling might be used in conjunction with Head
119
119
Sampling. For example, a set of services that produce an extremely high volume
120
- of trace data may first use head sampling to only sample a small percentage of
120
+ of trace data might first use head sampling to sample only a small percentage of
121
121
traces, and then later in the telemetry pipeline use tail sampling to make more
122
122
sophisticated sampling decisions before exporting to a backend. This is often
123
123
done in the interest of protecting the telemetry pipeline from being overloaded.
@@ -133,7 +133,7 @@ The OpenTelemetry Collector includes the following sampling processors:
133
133
134
134
### Language SDKs
135
135
136
- For the individual language specific implementations of the OpenTelemetry API &
137
- SDK you will find support for sampling at the respective documentation pages:
136
+ For the individual language- specific implementations of the OpenTelemetry API &
137
+ SDK, you will find support for sampling in the respective documentation pages:
138
138
139
139
{{% sampling-support-list " " %}}
0 commit comments