Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

profiling: update ffi build script to cross compile 32 bit on 64 bit linux #866

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

taegyunkim
Copy link
Contributor

What does this PR do?

A brief description of the change being made with this pull request.

Motivation

What inspired you to submit this pull request?

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Describe here in detail how the change can be validated.

@taegyunkim taegyunkim requested review from a team as code owners February 10, 2025 18:43
@taegyunkim taegyunkim changed the title profiling: update build script to cross compiling 32 bit on 64 bit linux profiling: update ffi build script to cross compiling 32 bit on 64 bit linux Feb 10, 2025
@taegyunkim taegyunkim changed the title profiling: update ffi build script to cross compiling 32 bit on 64 bit linux profiling: update ffi build script to cross compile 32 bit on 64 bit linux Feb 10, 2025
@codecov-commenter
Copy link

codecov-commenter commented Feb 10, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.90%. Comparing base (1a7e673) to head (6a1be43).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #866      +/-   ##
==========================================
- Coverage   71.91%   71.90%   -0.02%     
==========================================
  Files         324      324              
  Lines       48044    48048       +4     
==========================================
- Hits        34549    34547       -2     
- Misses      13495    13501       +6     
Components Coverage Δ
crashtracker 43.25% <ø> (ø)
crashtracker-ffi 6.30% <ø> (ø)
datadog-alloc 98.73% <ø> (+0.14%) ⬆️
data-pipeline 92.36% <ø> (ø)
data-pipeline-ffi 91.45% <ø> (ø)
ddcommon 80.11% <100.00%> (+0.03%) ⬆️
ddcommon-ffi 61.05% <ø> (ø)
ddtelemetry 61.74% <ø> (ø)
ddtelemetry-ffi 22.46% <ø> (ø)
dogstatsd 90.01% <ø> (ø)
dogstatsd-client 79.77% <ø> (ø)
ipc 82.68% <ø> (ø)
profiling 80.04% <ø> (ø)
profiling-ffi 70.68% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 41.50% <ø> (ø)
sidecar-ffi 9.26% <ø> (ø)
spawn-worker 54.37% <ø> (ø)
tinybytes 92.79% <ø> (ø)
trace-mini-agent 72.48% <ø> (ø)
trace-normalization 98.23% <ø> (ø)
trace-obfuscation 95.96% <ø> (ø)
trace-protobuf 77.67% <ø> (ø)
trace-utils 93.56% <ø> (ø)

@pr-commenter
Copy link

pr-commenter bot commented Feb 10, 2025

Benchmarks

Comparison

Benchmark execution time: 2025-03-11 19:30:14

Comparing candidate commit dab9ab7 in PR branch taegyunkim/prof-11194-32bit with baseline commit 0ecd32f in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 52 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz dab9ab7 1741720719 taegyunkim/prof-11194-32bit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 244.703ns 253.912ns ± 10.823ns 249.314ns ± 2.499ns 253.104ns 280.589ns 288.911ns 291.379ns 16.87% 1.993 2.893 4.25% 0.765ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [252.412ns; 255.412ns] or [-0.591%; +0.591%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz dab9ab7 1741720719 taegyunkim/prof-11194-32bit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.659µs 26.033µs ± 10.778µs 18.252µs ± 0.178µs 34.262µs 44.526µs 46.444µs 86.645µs 374.72% 1.779 5.767 41.30% 0.762µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [24.539µs; 27.526µs] or [-5.738%; +5.738%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz dab9ab7 1741720719 taegyunkim/prof-11194-32bit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.416µs 2.453µs ± 0.017µs 2.454µs ± 0.014µs 2.466µs 2.480µs 2.486µs 2.491µs 1.52% 0.061 -0.732 0.67% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.451µs; 2.455µs] or [-0.093%; +0.093%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz dab9ab7 1741720719 taegyunkim/prof-11194-32bit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 504.236µs 505.755µs ± 0.575µs 505.735µs ± 0.268µs 506.003µs 506.406µs 506.541µs 511.600µs 1.16% 5.129 51.885 0.11% 0.041µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1954651.011op/s 1977243.241op/s ± 2233.278op/s 1977321.876op/s ± 1046.306op/s 1978342.698op/s 1979823.890op/s 1981035.870op/s 1983197.688op/s 0.30% -5.036 50.700 0.11% 157.917op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 452.565µs 453.723µs ± 1.121µs 453.583µs ± 0.225µs 453.830µs 454.303µs 454.811µs 465.303µs 2.58% 8.615 80.692 0.25% 0.079µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2149137.691op/s 2204001.276op/s ± 5332.789op/s 2204669.501op/s ± 1092.103op/s 2205656.820op/s 2206640.460op/s 2208022.588op/s 2209629.299op/s 0.22% -8.553 79.824 0.24% 377.085op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 175.329µs 176.518µs ± 0.331µs 176.591µs ± 0.197µs 176.744µs 176.955µs 177.071µs 177.215µs 0.35% -0.836 0.580 0.19% 0.023µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5642858.640op/s 5665167.318op/s ± 10634.698op/s 5662796.640op/s ± 6311.735op/s 5670621.734op/s 5686917.872op/s 5693524.212op/s 5703556.820op/s 0.72% 0.847 0.604 0.19% 751.987op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 37.670µs 37.775µs ± 0.050µs 37.775µs ± 0.033µs 37.810µs 37.857µs 37.894µs 37.918µs 0.38% 0.155 -0.381 0.13% 0.004µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 26372480.623op/s 26472395.391op/s ± 35037.788op/s 26472838.292op/s ± 22821.854op/s 26493927.288op/s 26530726.520op/s 26537629.863op/s 26546057.023op/s 0.28% -0.149 -0.385 0.13% 2477.546op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 48.095µs 48.355µs ± 0.385µs 48.198µs ± 0.101µs 48.555µs 48.654µs 50.044µs 51.337µs 6.51% 4.518 29.956 0.79% 0.027µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 19479036.383op/s 20681844.964op/s ± 158985.928op/s 20747689.968op/s ± 43589.576op/s 20787316.174op/s 20790565.123op/s 20791935.675op/s 20792015.866op/s 0.21% -4.252 27.319 0.77% 11242.003op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [505.676µs; 505.835µs] or [-0.016%; +0.016%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1976933.730op/s; 1977552.752op/s] or [-0.016%; +0.016%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [453.568µs; 453.878µs] or [-0.034%; +0.034%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2203262.203op/s; 2204740.350op/s] or [-0.034%; +0.034%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [176.472µs; 176.564µs] or [-0.026%; +0.026%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5663693.451op/s; 5666641.184op/s] or [-0.026%; +0.026%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [37.768µs; 37.782µs] or [-0.018%; +0.018%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [26467539.491op/s; 26477251.292op/s] or [-0.018%; +0.018%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [48.301µs; 48.408µs] or [-0.110%; +0.110%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20659811.044op/s; 20703878.885op/s] or [-0.107%; +0.107%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz dab9ab7 1741720719 taegyunkim/prof-11194-32bit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 68.288µs 68.577µs ± 0.262µs 68.567µs ± 0.069µs 68.621µs 68.733µs 68.895µs 71.984µs 4.98% 11.066 141.387 0.38% 0.019µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [68.541µs; 68.613µs] or [-0.053%; +0.053%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz dab9ab7 1741720719 taegyunkim/prof-11194-32bit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.893µs 3.914µs ± 0.003µs 3.914µs ± 0.001µs 3.915µs 3.918µs 3.920µs 3.921µs 0.19% -1.990 15.811 0.07% 0.000µs 1 200
credit_card/is_card_number/ throughput 255020484.982op/s 255495943.932op/s ± 177250.572op/s 255501877.949op/s ± 88275.525op/s 255580341.681op/s 255743424.280op/s 255831244.467op/s 256852482.431op/s 0.53% 2.019 16.061 0.07% 12533.508op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 81.904µs 82.459µs ± 0.222µs 82.399µs ± 0.109µs 82.576µs 82.777µs 83.023µs 84.205µs 2.19% 2.692 18.040 0.27% 0.016µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 11875777.036op/s 12127394.049op/s ± 32468.562op/s 12136035.209op/s ± 16077.151op/s 12146433.019op/s 12164401.186op/s 12178765.663op/s 12209456.224op/s 0.60% -2.591 17.065 0.27% 2295.874op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 76.271µs 77.215µs ± 0.294µs 77.185µs ± 0.163µs 77.368µs 77.680µs 77.895µs 78.704µs 1.97% 0.542 3.238 0.38% 0.021µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 12705819.617op/s 12951092.995op/s ± 49283.126op/s 12955889.085op/s ± 27339.238op/s 12978862.306op/s 13018728.889op/s 13077813.753op/s 13111084.979op/s 1.20% -0.486 3.089 0.38% 3484.843op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.897µs 3.914µs ± 0.003µs 3.914µs ± 0.001µs 3.916µs 3.918µs 3.919µs 3.920µs 0.15% -1.709 9.393 0.07% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 255081844.333op/s 255483652.999op/s ± 169786.573op/s 255470467.614op/s ± 95698.466op/s 255566722.084op/s 255719169.229op/s 256003252.218op/s 256615456.064op/s 0.45% 1.726 9.515 0.07% 12005.724op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 72.918µs 73.970µs ± 0.489µs 73.940µs ± 0.338µs 74.280µs 74.862µs 75.156µs 75.408µs 1.98% 0.453 -0.095 0.66% 0.035µs 1 200
credit_card/is_card_number/378282246310005 throughput 13261238.998op/s 13519661.309op/s ± 89198.720op/s 13524450.400op/s ± 61705.928op/s 13584527.620op/s 13638609.652op/s 13685739.279op/s 13714039.092op/s 1.40% -0.420 -0.137 0.66% 6307.302op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 51.961µs 52.190µs ± 0.107µs 52.189µs ± 0.076µs 52.265µs 52.364µs 52.438µs 52.531µs 0.66% 0.179 -0.221 0.20% 0.008µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 19036378.519op/s 19160693.762op/s ± 39302.475op/s 19161265.523op/s ± 27751.337op/s 19188788.954op/s 19222342.906op/s 19242587.965op/s 19245191.559op/s 0.44% -0.168 -0.233 0.20% 2779.105op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.468µs 6.618µs ± 0.065µs 6.614µs ± 0.046µs 6.661µs 6.721µs 6.771µs 6.788µs 2.63% 0.181 -0.229 0.98% 0.005µs 1 200
credit_card/is_card_number/x371413321323331 throughput 147308977.860op/s 151128105.043op/s ± 1486640.733op/s 151184092.592op/s ± 1051986.708op/s 152143111.452op/s 153521032.847op/s 154091626.279op/s 154610025.277op/s 2.27% -0.130 -0.261 0.98% 105121.374op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.894µs 3.915µs ± 0.003µs 3.915µs ± 0.001µs 3.916µs 3.919µs 3.921µs 3.925µs 0.25% -1.478 10.024 0.08% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 254795432.463op/s 255439158.095op/s ± 196305.241op/s 255438447.954op/s ± 97745.860op/s 255535004.672op/s 255766154.694op/s 255915463.410op/s 256772758.219op/s 0.52% 1.500 10.178 0.08% 13880.877op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 63.406µs 63.860µs ± 0.154µs 63.853µs ± 0.076µs 63.923µs 64.103µs 64.311µs 64.480µs 0.98% 0.770 2.647 0.24% 0.011µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 15508785.974op/s 15659467.374op/s ± 37762.563op/s 15660875.234op/s ± 18702.542op/s 15680585.868op/s 15716289.831op/s 15746603.822op/s 15771333.743op/s 0.71% -0.740 2.585 0.24% 2670.216op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 57.399µs 57.542µs ± 0.103µs 57.524µs ± 0.025µs 57.549µs 57.698µs 57.932µs 58.446µs 1.60% 4.810 32.523 0.18% 0.007µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 17109760.292op/s 17378545.913op/s ± 30741.831op/s 17384107.449op/s ± 7602.792op/s 17391676.778op/s 17401730.863op/s 17409850.905op/s 17422002.247op/s 0.22% -4.750 31.749 0.18% 2173.776op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.892µs 3.914µs ± 0.003µs 3.914µs ± 0.002µs 3.915µs 3.918µs 3.919µs 3.920µs 0.15% -1.831 10.338 0.08% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 255117929.161op/s 255515155.329op/s ± 206737.811op/s 255492437.684op/s ± 109432.087op/s 255605565.817op/s 255878025.663op/s 255956935.204op/s 256950712.812op/s 0.57% 1.853 10.517 0.08% 14618.571op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 54.569µs 54.802µs ± 0.255µs 54.685µs ± 0.046µs 54.854µs 55.284µs 55.605µs 55.857µs 2.14% 1.814 2.900 0.47% 0.018µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 17902715.470op/s 18247895.294op/s ± 84365.703op/s 18286463.449op/s ± 15299.000op/s 18299160.526op/s 18310365.773op/s 18319313.441op/s 18325527.460op/s 0.21% -1.792 2.778 0.46% 5965.556op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 51.937µs 52.193µs ± 0.091µs 52.188µs ± 0.063µs 52.260µs 52.337µs 52.412µs 52.420µs 0.45% 0.018 -0.219 0.17% 0.006µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 19076555.609op/s 19159762.090op/s ± 33270.985op/s 19161513.144op/s ± 23125.724op/s 19181646.536op/s 19215845.963op/s 19226725.387op/s 19254034.522op/s 0.48% -0.008 -0.219 0.17% 2352.614op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.433µs 6.586µs ± 0.074µs 6.583µs ± 0.053µs 6.636µs 6.720µs 6.796µs 6.804µs 3.36% 0.557 -0.007 1.12% 0.005µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 146979003.845op/s 151844780.521op/s ± 1696508.792op/s 151915609.411op/s ± 1206722.649op/s 153101076.192op/s 154382948.879op/s 155169767.252op/s 155444861.839op/s 2.32% -0.501 -0.089 1.11% 119961.287op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.914µs; 3.914µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ throughput [255471378.708op/s; 255520509.157op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [82.428µs; 82.489µs] or [-0.037%; +0.037%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12122894.219op/s; 12131893.880op/s] or [-0.037%; +0.037%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [77.174µs; 77.255µs] or [-0.053%; +0.053%] None None None
credit_card/is_card_number/ 378282246310005 throughput [12944262.827op/s; 12957923.162op/s] or [-0.053%; +0.053%] None None None
credit_card/is_card_number/37828224631 execution_time [3.914µs; 3.915µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/37828224631 throughput [255460122.213op/s; 255507183.785op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/378282246310005 execution_time [73.902µs; 74.037µs] or [-0.092%; +0.092%] None None None
credit_card/is_card_number/378282246310005 throughput [13507299.224op/s; 13532023.393op/s] or [-0.091%; +0.091%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [52.176µs; 52.205µs] or [-0.028%; +0.028%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [19155246.817op/s; 19166140.707op/s] or [-0.028%; +0.028%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.609µs; 6.627µs] or [-0.137%; +0.137%] None None None
credit_card/is_card_number/x371413321323331 throughput [150922070.935op/s; 151334139.151op/s] or [-0.136%; +0.136%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.914µs; 3.915µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/ throughput [255411952.077op/s; 255466364.114op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [63.838µs; 63.881µs] or [-0.033%; +0.033%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15654233.846op/s; 15664700.902op/s] or [-0.033%; +0.033%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [57.528µs; 57.557µs] or [-0.025%; +0.025%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [17374285.391op/s; 17382806.435op/s] or [-0.025%; +0.025%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.913µs; 3.914µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255486503.457op/s; 255543807.201op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [54.767µs; 54.837µs] or [-0.065%; +0.065%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [18236203.019op/s; 18259587.569op/s] or [-0.064%; +0.064%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [52.180µs; 52.205µs] or [-0.024%; +0.024%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [19155151.051op/s; 19164373.129op/s] or [-0.024%; +0.024%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.576µs; 6.597µs] or [-0.156%; +0.156%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [151609660.718op/s; 152079900.323op/s] or [-0.155%; +0.155%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz dab9ab7 1741720719 taegyunkim/prof-11194-32bit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.179µs 3.177µs ± 1.423µs 2.984µs ± 0.025µs 3.007µs 3.611µs 13.894µs 14.802µs 395.97% 7.387 55.516 44.68% 0.101µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.979µs; 3.374µs] or [-6.208%; +6.208%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz dab9ab7 1741720719 taegyunkim/prof-11194-32bit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 54.828ms 55.146ms ± 0.198ms 55.112ms ± 0.068ms 55.185ms 55.591ms 55.840ms 56.388ms 2.32% 2.598 9.895 0.36% 0.014ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [55.119ms; 55.174ms] or [-0.050%; +0.050%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz dab9ab7 1741720719 taegyunkim/prof-11194-32bit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 5.015µs 5.068µs ± 0.036µs 5.054µs ± 0.022µs 5.105µs 5.129µs 5.131µs 5.135µs 1.61% 0.460 -1.331 0.71% 0.003µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.063µs; 5.073µs] or [-0.098%; +0.098%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz dab9ab7 1741720719 taegyunkim/prof-11194-32bit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 208.611µs 208.946µs ± 0.184µs 208.943µs ± 0.116µs 209.040µs 209.270µs 209.536µs 209.586µs 0.31% 0.723 0.901 0.09% 0.013µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 4771314.223op/s 4785918.927op/s ± 4201.101op/s 4785998.404op/s ± 2654.206op/s 4788962.877op/s 4792258.362op/s 4793092.617op/s 4793603.416op/s 0.16% -0.717 0.886 0.09% 297.063op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 18.265µs 18.330µs ± 0.035µs 18.327µs ± 0.017µs 18.346µs 18.370µs 18.389µs 18.672µs 1.88% 4.611 43.614 0.19% 0.002µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 53557042.751op/s 54556020.442op/s ± 103331.669op/s 54563565.984op/s ± 50057.550op/s 54604977.726op/s 54692878.821op/s 54709750.136op/s 54750452.956op/s 0.34% -4.474 41.902 0.19% 7306.652op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.669µs 10.730µs ± 0.031µs 10.722µs ± 0.020µs 10.752µs 10.789µs 10.816µs 10.838µs 1.08% 0.632 0.233 0.29% 0.002µs 1 200
normalization/normalize_name/normalize_name/good throughput 92263886.070op/s 93193726.185op/s ± 269442.780op/s 93262535.119op/s ± 172100.551op/s 93371519.935op/s 93601347.112op/s 93652049.455op/s 93727672.086op/s 0.50% -0.616 0.201 0.29% 19052.482op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [208.921µs; 208.972µs] or [-0.012%; +0.012%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [4785336.695op/s; 4786501.159op/s] or [-0.012%; +0.012%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [18.325µs; 18.335µs] or [-0.026%; +0.026%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [54541699.666op/s; 54570341.217op/s] or [-0.026%; +0.026%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.726µs; 10.735µs] or [-0.040%; +0.040%] None None None
normalization/normalize_name/normalize_name/good throughput [93156384.007op/s; 93231068.363op/s] or [-0.040%; +0.040%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz dab9ab7 1741720719 taegyunkim/prof-11194-32bit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 5.952ms 5.963ms ± 0.005ms 5.963ms ± 0.003ms 5.966ms 5.971ms 5.982ms 5.997ms 0.58% 1.910 9.383 0.09% 0.000ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [5.963ms; 5.964ms] or [-0.012%; +0.012%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz dab9ab7 1741720719 taegyunkim/prof-11194-32bit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 146.951µs 147.550µs ± 0.375µs 147.502µs ± 0.180µs 147.711µs 148.000µs 148.498µs 151.158µs 2.48% 4.819 41.918 0.25% 0.027µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [147.498µs; 147.602µs] or [-0.035%; +0.035%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz dab9ab7 1741720719 taegyunkim/prof-11194-32bit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 32.992µs 33.643µs ± 1.199µs 33.080µs ± 0.036µs 33.156µs 36.222µs 36.275µs 36.985µs 11.80% 1.701 0.944 3.56% 0.085µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [33.477µs; 33.809µs] or [-0.494%; +0.494%] None None None

Baseline

Omitted due to size.

Copy link
Member

@ivoanjo ivoanjo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Arguably this could go in as-is, and we could fix the sharp edges separately, up to you :D

Comment on lines -31 to 35
while getopts f:hT flag
while getopts f:ht:T flag
do
case "${flag}" in
f)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two notes here:

  • I think we're misusing the shift -- it actually breaks if we try to pass more than one setting now; e.g. -f foo -t triplet or -T -t triplet. My bash-foo isn't great but this suggests how we may fix it. I think it has "worked" until now since nobody tried to use -f with -T, but it seems reasonable to want to combine -f and -t for instance?

  • As a script UX thing, consider using something other than -t for the triplet, to avoid the collision with the test. Maybe -a (arch?) or -c (cpu) or something like that? Alternatively, rename the -T to something else, I think that works too :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants