feat: update remote write sidecar to write to eastus2 workspace (#1090)

bragi92 · web-flow · commit a709e4c8b528 · 2025-03-13T16:44:54.000-07:00
[comment]: # (Note that your PR title should follow the conventional commit format: https://conventionalcommits.org/en/v1.0.0/#summary) # PR Description [comment]: # (The below checklist is for PRs adding new features. If a box is not checked, add a reason why it's not needed.) # New Feature Checklist - [ ] List telemetry added about the feature. - [ ] Link to the one-pager about the feature. - [ ] List any tasks necessary for release (3P docs, AKS RP chart changes, etc.) after merging the PR. - [ ] Attach results of scale and perf testing. [comment]: # (The below checklist is for code changes. Not all boxes necessarily need to be checked. Build, doc, and template changes do not need to fill out the checklist.) # Tests Checklist - [ ] Have end-to-end Ginkgo tests been run on your cluster and passed? To bootstrap your cluster to run the tests, follow [these instructions](/otelcollector/test/README.md#bootstrap-a-dev-cluster-to-run-ginkgo-tests). - Labels used when running the tests on your cluster: - [ ] `operator` - [ ] `windows` - [ ] `arm64` - [ ] `arc-extension` - [ ] `fips` - [ ] Have new tests been added? For features, have tests been added for this feature? For fixes, is there a test that could have caught this issue and could validate that the fix works? - [ ] Is a new scrape job needed? - [ ] The scrape job was added to the folder [test-cluster-yamls](/otelcollector/test/test-cluster-yamls/) in the correct configmap or as a CR. - [ ] Was a new test label added? - [ ] A string constant for the label was added to [constants.go](/otelcollector/test/utils/constants.go). - [ ] The label and description was added to the [test README](/otelcollector/test/README.md). - [ ] The label was added to this [PR checklist](/.github/pull_request_template). - [ ] The label was added as needed to [testkube-test-crs.yaml](/otelcollector/test/testkube/testkube-test-crs.yaml). - [ ] Are additional API server permissions needed for the new tests? - [ ] These permissions have been added to [api-server-permissions.yaml](/otelcollector/test/testkube/api-server-permissions.yaml). - [ ] Was a new test suite (a new folder under `/tests`) added? - [ ] The new test suite is included in [testkube-test-crs.yaml](/otelcollector/test/testkube/testkube-test-crs.yaml).
diff --git a/.pipelines/azure-pipeline-build.yml b/.pipelines/azure-pipeline-build.yml
@@ -24,7 +24,7 @@ variables:
   IS_MAIN_BRANCH: $[eq(variables['Build.SourceBranchName'], 'main')]
   BUILD_WINDOWS: true
   Codeql.Enabled: true
-  GOLANG_VERSION: '1.22.11'
+  GOLANG_VERSION: '1.23.7'
   TESTKUBE_GOLANG_VERSION: '1.23.2'
   FLUENT_BIT_VERSION: '3.2.2'
 
diff --git a/.trivyignore b/.trivyignore
@@ -19,4 +19,9 @@ CVE-2024-45341 # stdlib
 # targetallocator
 # config-reader
 CVE-2025-22866 # stdlib
+CVE-2025-22870 # golang.org/x/net
+#otelcollector
+CVE-2025-22870 # golang.org/x/net
+#promconfigvalidator
+CVE-2025-22870 # golang.org/x/net
 
diff --git a/RELEASENOTES.md b/RELEASENOTES.md
@@ -1,4 +1,15 @@
 # Azure Monitor Metrics for AKS clusters
+## Release <TBD>
+* Linux image - `mcr.microsoft.com/azuremonitor/containerinsights/ciprod/prometheus-collector/images:<TBD>`
+* Windows image - `mcr.microsoft.com/azuremonitor/containerinsights/ciprod/prometheus-collector/images:<TBD>-win`
+* TA image - `mcr.microsoft.com/azuremonitor/containerinsights/ciprod/prometheus-collector/images:<TBD>-targetallocator`
+* cfg sidecar image - `mcr.microsoft.com/azuremonitor/containerinsights/ciprod/prometheus-collector/images:<TBD>-cfg`
+* AKS and Arc Container Images:
+  * 
+
+* Pipeline/Docs/Templates Updates:
+   * ci/cd: update remote write sidecar to write to eastus2 workspace (https://github.com/Azure/prometheus-collector/pull/1090) 
+   
 ## Release 02-21-2025
 * Linux image - `mcr.microsoft.com/azuremonitor/containerinsights/ciprod/prometheus-collector/images:6.15.0-main-02-21-2025-4acb2b4c`
 * Windows image - `mcr.microsoft.com/azuremonitor/containerinsights/ciprod/prometheus-collector/images:6.15.0-main-02-21-2025-4acb2b4c-win`
diff --git a/internal/docs/BUILDANDRELEASE.md b/internal/docs/BUILDANDRELEASE.md
@@ -97,4 +97,19 @@ https://msazure.visualstudio.com/CloudNativeCompute/_git/aks-rp?path=/toolkit/ve
       - Verify first minimalIngestionProfile scenario by applying [test config enabling all ccp targets, with MIP false, and empty keep list](https://github.com/Azure/prometheus-collector/blob/main/otelcollector/test/test-cluster-yamls/configmaps/controlplane/ama-metrics-settings-configmap-mipfalse-emptykeep.yaml). All the metrics from all targets should have been scraped.
       - Verify second minimalIngestionProfile scenario by applying [test config enabling all ccp targets, with MIP false, and keep list populated](https://github.com/Azure/prometheus-collector/blob/main/otelcollector/test/test-cluster-yamls/configmaps/controlplane/ama-metrics-settings-configmap-mipfalse-emptykeep.yaml). Only the metrics specified in the test config .yaml for each targets' keeplist should be scraped.
   - Post about the candidate image release for the AKS team to update in the [Managed Prometheus (Azure Monitor Metrics addon)](https://teams.microsoft.com/l/channel/19%3Ae9dd234c60824ac5b494dbe3ec7dcb6b%40thread.skype/Managed%20Prometheus%20(Azure%20Monitor%20Metrics%20addon)?groupId=e121dbfd-0ec1-40ea-8af5-26075f6a731b&tenantId=72f988bf-86f1-41af-91ab-2d7cd011db47) teams channel
+- **🚀 Remote Write Deployment Steps**
+  - **Skip if Not Releasing**  
+    - If you’re not releasing a new remote write image, leave the default value as is and skip the following steps.  
+
+  - **Get the Latest Image Tag**  
+   - Retrieve the latest image tag from the most recent build in the [Remote Write Release Pipeline](https://msazure.visualstudio.com/InfrastructureInsights/_release?_a=releases&view=mine&definitionId=77).  
+
+  - **Update Prometheus Collector Release**  
+   - While creating the [Prometheus Collector Prod Release](https://github-private.visualstudio.com/azure/_release?_a=releases&view=mine&definitionId=79), update the release variables with the new candidate image tag.  
+
+  - **Deploy Remote Write Image**  
+   - Click **Deploy** in the `Deploy to Remote Write Clusters` step to release the new remote write image to the monitoring cluster.  
+
+  - **Update Pipeline Variables**  
+   - After a successful deployment, update the `RemoteWriteTag` variable in the release pipeline with the new remote write release tag.  
 - **Arc**: Start Arc release to Canary regions. The new version will be automatically deployed to each region batch every 24 hours.
diff --git a/internal/remotewrite/sidecar.yaml b/internal/remotewrite/sidecar.yaml
@@ -44,4 +44,9 @@ prometheus:
     externalLabels:
       cluster: monitoring-metrics-prod-aks-eus2euap
     remoteWrite:
-    - url: http://localhost:8081/api/v1/write
+    - url: http://localhost:8081/api/v1/write
+    - url: https://remotewritemonitor-eus2-lzt0.eastus2-1.metrics.ingest.monitor.azure.com/dataCollectionRules/dcr-c61eca33833b44fa8eddd257a0baa02a/streams/Microsoft-PrometheusMetrics/api/v1/write?api-version=2023-04-24
+      azureAd:
+        cloud: AzurePublic
+        managedIdentity:
+          clientId: $USER_ASSIGNED_MANAGED_IDENTITY_CLIENT_ID
diff --git a/otelcollector/go.mod b/otelcollector/go.mod
@@ -1,6 +1,8 @@
 module prometheus-collector
 
-go 1.21
+go 1.23.0
+
+toolchain go1.23.7
 
 replace github.com/prometheus-collector/shared => ./shared
 
@@ -19,6 +21,8 @@ require (
 	github.com/pelletier/go-toml v1.9.5 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect
 	github.com/stretchr/testify v1.9.0 // indirect
+	golang.org/x/net v0.36.0 // indirect
+	golang.org/x/sys v0.30.0 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )
diff --git a/otelcollector/go.sum b/otelcollector/go.sum
@@ -18,12 +18,12 @@ github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZb
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
 github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
-golang.org/x/net v0.25.0 h1:d/OCCoBEUq33pjydKrGQhw7IlUPI2Oylr+8qLx49kac=
-golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
-golang.org/x/sys v0.20.0 h1:Od9JTbYCk261bKm4M/mw7AklTlFYIa0bIp9BgSm1S8Y=
-golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
-golang.org/x/text v0.15.0 h1:h1V/4gjBv8v9cjcR6+AR5+/cIYK5N/WAgiv4xlsEtAk=
-golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
+golang.org/x/net v0.36.0 h1:vWF2fRbw4qslQsQzgFqZff+BItCvGFQqKzKIzx1rmoA=
+golang.org/x/net v0.36.0/go.mod h1:bFmbeoIPfrw4sMHNhb4J9f6+tPziuGjq7Jk/38fxi1I=
+golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc=
+golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/text v0.22.0 h1:bofq7m3/HAFvbF51jz3Q9wLg3jkvSPuiZu/pD1XwgtM=
+golang.org/x/text v0.22.0/go.mod h1:YRoo4H8PVmsu+E3Ou7cqLVH8oXWIHVoX0jqUWALQhfY=
 golang.org/x/tools v0.21.0 h1:qc0xYgIbsSDt9EyWz05J5wfa7LOVW0YTLOXrqdLAWIw=
 golang.org/x/tools v0.21.0/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
