Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Singleton Metrics Provider for Listener #285

Merged
merged 12 commits into from
Feb 12, 2025
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# Changelog

## v1.5.1 (Unreleased)
## v1.5.1

### Updates

* Use Singelton metrics provider instead of IScaleMonitor and ITargetScaler ([#285]https://github.com/microsoft/durabletask-mssql/pull/285)
* Updated repo to use central package management
* Resolved multiple CVEs in dependencies

Expand Down
Binary file modified Directory.Packages.props
Binary file not shown.
21 changes: 8 additions & 13 deletions src/DurableTask.SqlServer.AzureFunctions/SqlDurabilityProvider.cs
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,7 @@ class SqlDurabilityProvider : DurabilityProvider
readonly SqlDurabilityOptions durabilityOptions;
readonly SqlOrchestrationService service;

SqlScaleMonitor? scaleMonitor;
#if FUNCTIONS_V4
SqlTargetScaler? targetScaler;
#endif
SqlMetricsProvider? singletonSqlMetricsProvider;

public SqlDurabilityProvider(
SqlOrchestrationService service,
Expand Down Expand Up @@ -200,13 +197,12 @@ public override bool TryGetScaleMonitor(
string storageConnectionString,
out IScaleMonitor scaleMonitor)
{
if (this.scaleMonitor == null)
if (this.singletonSqlMetricsProvider == null)
{
var sqlMetricsProvider = new SqlMetricsProvider(this.service);
this.scaleMonitor = new SqlScaleMonitor(hubName, sqlMetricsProvider);
this.singletonSqlMetricsProvider = new SqlMetricsProvider(this.service);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last thing. I recall the final fix that Alexey made to the Azure Storage metrics provider was to throttle it so that we only fetch metrics once every few seconds or so. This is how we ensured that we don't have too many storage transactions, even if the app has a lot of triggers. I think it will be important to do that for the SqlMetricsProvider too so that we don't overwhelm the customer's database with expensive SQL queries.

}

scaleMonitor = this.scaleMonitor;
scaleMonitor = new SqlScaleMonitor(functionId, hubName, this.singletonSqlMetricsProvider);

return true;
}

Expand All @@ -218,13 +214,12 @@ public override bool TryGetTargetScaler(
string connectionName,
out ITargetScaler targetScaler)
{
if (this.targetScaler == null)
if (this.singletonSqlMetricsProvider == null)
{
var sqlMetricsProvider = new SqlMetricsProvider(this.service);
this.targetScaler = new SqlTargetScaler(hubName, sqlMetricsProvider);
this.singletonSqlMetricsProvider = new SqlMetricsProvider(this.service);
}

targetScaler = this.targetScaler;
targetScaler = new SqlTargetScaler(functionId, this.singletonSqlMetricsProvider);
return true;
}
#endif
Expand Down
9 changes: 4 additions & 5 deletions src/DurableTask.SqlServer.AzureFunctions/SqlScaleMonitor.cs
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,13 @@ class SqlScaleMonitor : IScaleMonitor<SqlScaleMetric>

int? previousWorkerCount = -1;

public SqlScaleMonitor(string taskHubName, SqlMetricsProvider sqlMetricsProvider)
public SqlScaleMonitor(string functionId, string taskHubName, SqlMetricsProvider sqlMetricsProvider)
{
// Scalers in Durable Functions are shared for all functions in the same task hub.
// So instead of using a function ID, we use the task hub name as the basis for the descriptor ID.
string id = $"DurableTask-SqlServer:{taskHubName ?? "default"}";
// Scalers in Durable Functions is per function ids. And scalers share the same sqlMetricsProvider in the same taskhub.
string id = $"{functionId}-DurableTask-SqlServer:{taskHubName ?? "default"}";

#if FUNCTIONS_V4
this.Descriptor = new ScaleMonitorDescriptor(id: id, functionId: id);
this.Descriptor = new ScaleMonitorDescriptor(id: id, functionId: functionId);
#else
this.Descriptor = new ScaleMonitorDescriptor(id);
#endif
Expand Down
8 changes: 3 additions & 5 deletions src/DurableTask.SqlServer.AzureFunctions/SqlTargetScaler.cs
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,12 @@ public class SqlTargetScaler : ITargetScaler
{
readonly SqlMetricsProvider sqlMetricsProvider;

public SqlTargetScaler(string taskHubName, SqlMetricsProvider sqlMetricsProvider)
public SqlTargetScaler(string functionId, SqlMetricsProvider sqlMetricsProvider)
{
this.sqlMetricsProvider = sqlMetricsProvider;

// Scalers in Durable Functions are shared for all functions in the same task hub.
// So instead of using a function ID, we use the task hub name as the basis for the descriptor ID.
string id = $"DurableTask-SqlServer:{taskHubName ?? "default"}";
Copy link
Contributor Author

@nytian nytian Feb 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like we don't need the hub name and id , but only the function id to create the target scaler descriptor. Reference about how we defined this constructor at AzureStorage (here ) and how we passed parameters (here)

this.TargetScalerDescriptor = new TargetScalerDescriptor(id);
// Scalers in Durable Functions is per function ids. And scalers share the same sqlMetricsProvider in the same taskhub.
this.TargetScalerDescriptor = new TargetScalerDescriptor(functionId);
}

public TargetScalerDescriptor TargetScalerDescriptor { get; }
Expand Down
Loading