Add vpc support

[vishesh92]

To be merged after: #415

Issue #, if available:
Fixes #314
Description of changes:

Testing performed:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[k8s-ci-robot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vishesh92

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [vishesh92]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[netlify]

✅ Deploy Preview for kubernetes-sigs-cluster-api-cloudstack ready!

Name	Link
🔨 Latest commit	3a28780
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-cluster-api-cloudstack/deploys/67da93e67279770008bafde5
😎 Deploy Preview	https://deploy-preview-416--kubernetes-sigs-cluster-api-cloudstack.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 26.31579% with 196 lines in your changes missing coverage. Please review.

Project coverage is 25.61%. Comparing base (d597e80) to head (6c03dde).
Report is 68 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/cloud/isolated_network.go	21.42%	34 Missing and 10 partials ⚠️
api/v1beta2/zz_generated.conversion.go	0.00%	36 Missing ⚠️
pkg/cloud/vpc.go	63.52%	31 Missing ⚠️
api/v1beta3/zz_generated.deepcopy.go	0.00%	26 Missing ⚠️
api/v1beta2/conversion.go	0.00%	13 Missing ⚠️
pkg/cloud/network.go	25.00%	10 Missing and 2 partials ⚠️
api/v1beta1/zz_generated.conversion.go	0.00%	11 Missing ⚠️
api/v1beta3/cloudstackisolatednetwork_types.go	0.00%	7 Missing ⚠️
controllers/utils/isolated_network.go	0.00%	7 Missing ⚠️
api/v1beta1/conversion.go	0.00%	6 Missing ⚠️
... and 1 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #416      +/-   ##
==========================================
- Coverage   25.66%   25.61%   -0.06%     
==========================================
  Files          59       69      +10     
  Lines        5563     6699    +1136     
==========================================
+ Hits         1428     1716     +288     
- Misses       3996     4824     +828     
- Partials      139      159      +20     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[vishesh92]

/test all

[blueorangutan]

Test Results : (tid-531)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.27.2
Kubernetes Version upgrade from: v1.26.5
Kubernetes Version upgrade to: v1.27.2
CloudStack Version: 4.20
Template: ubuntu-2004-kube
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr416-sl-531.zip

Summarizing 9 Failures:
 [FAIL] When testing project [AfterEach] Should create a cluster in a project
 /jenkins/workspace/capc-e2e-new/test/e2e/project.go:103
 [FAIL] with two clusters [It] should successfully add and remove a second cluster without breaking the first cluster
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/cluster_helpers.go:144
 [FAIL] When testing horizontal scale out/in [TC17][TC18][TC20][TC21] [It] Should successfully scale machine replicas up and down horizontally
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/cluster_helpers.go:144
 [FAIL] When testing subdomain [It] Should create a cluster in a subdomain
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/cluster_helpers.go:144
 [FAIL] When testing creation of CKS cluster in ACS [It] Should create a workload cluster
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/cluster_helpers.go:144
 [FAIL] When testing clusters in a VPC network [It] Should successfully create a cluster in a VPC network
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/cluster_helpers.go:144
 [FAIL] When testing with disk offering [It] Should successfully create a cluster with disk offering
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/cluster_helpers.go:144
 [FAIL] When testing multiple CPs in a shared network with kubevip [It] Should successfully create a cluster with multiple CPs in a shared network
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/cluster_helpers.go:144
 [FAIL] When testing Kubernetes version upgrades [It] Should successfully upgrade kubernetes versions when there is a change in relevant fields
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/machinedeployment_helpers.go:127

Ran 29 of 32 Specs in 9666.342 seconds
FAIL! -- 20 Passed | 9 Failed | 0 Pending | 3 Skipped
--- FAIL: TestE2E (9666.34s)
FAIL

[blueorangutan]

Test Results : (tid-532)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.27.2
Kubernetes Version upgrade from: v1.26.5
Kubernetes Version upgrade to: v1.27.2
CloudStack Version: 4.20
Template: ubuntu-2004-kube
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr416-sl-532.zip

Summarizing 1 Failure:
 [FAIL] When testing project [AfterEach] Should create a cluster in a project
 /jenkins/workspace/capc-e2e-new/test/e2e/project.go:103

Ran 29 of 32 Specs in 10638.733 seconds
FAIL! -- 28 Passed | 1 Failed | 0 Pending | 3 Skipped
--- FAIL: TestE2E (10638.73s)
FAIL

[weizhouapache]

Summarizing 1 Failure:
[FAIL] When testing project [AfterEach] Should create a cluster in a project
/jenkins/workspace/capc-e2e-new/test/e2e/project.go:103

Ran 29 of 32 Specs in 10638.733 seconds
FAIL! -- 28 Passed | 1 Failed | 0 Pending | 3 Skipped
--- FAIL: TestE2E (10638.73s)
FAIL

@vishesh92
This happened again.
Worth checking

[vishesh92]

Summarizing 1 Failure:
[FAIL] When testing project [AfterEach] Should create a cluster in a project
/jenkins/workspace/capc-e2e-new/test/e2e/project.go:103
Ran 29 of 32 Specs in 10638.733 seconds
FAIL! -- 28 Passed | 1 Failed | 0 Pending | 3 Skipped
--- FAIL: TestE2E (10638.73s)
FAIL

@vishesh92 This happened again. Worth checking

I ran the test suite only for the failing test and it was successful. Probably some other test is causing issues with this one.

[blueorangutan]

Test Results : (tid-572)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.27.2
Kubernetes Version upgrade from: v1.26.5
Kubernetes Version upgrade to: v1.27.2
CloudStack Version: 4.20
Template: ubuntu-2004-kube
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr416-sl-572.zip

Summarizing 4 Failures:
 [FAIL] When testing clusters in a VPC network [It] Should successfully create a cluster in a VPC network
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/cluster_helpers.go:144
 [FAIL] When testing multiple CPs in a shared network with kubevip [It] Should successfully create a cluster with multiple CPs in a shared network
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/cluster_helpers.go:144
 [FAIL] When testing app deployment to the workload cluster with slow network [ToxiProxy] [It] Should be able to download an HTML from the app deployed to the workload cluster
 /jenkins/workspace/capc-e2e-new/test/e2e/deploy_app_toxi.go:135
 [TIMEDOUT] When testing resource cleanup [It] Should create a new network when the specified network does not exist
 /jenkins/workspace/capc-e2e-new/test/e2e/resource_cleanup.go:65

Ran 28 of 32 Specs in 10815.100 seconds
FAIL! - Suite Timeout Elapsed -- 24 Passed | 4 Failed | 0 Pending | 4 Skipped
--- FAIL: TestE2E (10815.10s)
FAIL

[blueorangutan]

Test Results : (tid-574)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.27.2
Kubernetes Version upgrade from: v1.26.5
Kubernetes Version upgrade to: v1.27.2
CloudStack Version: 4.20
Template: ubuntu-2004-kube
Jobs filtered by: Should successfully create a cluster in a VPC network
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr416-sl-574.zip

Ran 1 of 32 Specs in 458.829 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 31 Skipped
PASS

[blueorangutan]

Test Results : (tid-575)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.27.2
Kubernetes Version upgrade from: v1.26.5
Kubernetes Version upgrade to: v1.27.2
CloudStack Version: 4.20
Template: ubuntu-2004-kube
Jobs filtered by: Should successfully create a cluster in a VPC network
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr416-sl-575.zip

Ran 1 of 32 Specs in 463.343 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 31 Skipped
PASS

[blueorangutan]

Test Results : (tid-576)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.27.2
Kubernetes Version upgrade from: v1.26.5
Kubernetes Version upgrade to: v1.27.2
CloudStack Version: 4.20
Template: ubuntu-2004-kube
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr416-sl-576.zip

Summarizing 5 Failures:
 [FAIL] When testing app deployment to the workload cluster with slow network [ToxiProxy] [It] Should be able to download an HTML from the app deployed to the workload cluster
 /jenkins/workspace/capc-e2e-new/test/e2e/deploy_app_toxi.go:129
 [FAIL] When testing affinity group [It] Should have host affinity group when affinity is pro
 /jenkins/workspace/capc-e2e-new/test/e2e/common.go:359
 [FAIL] When testing affinity group [It] Should have host affinity group when affinity is anti
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/controlplane_helpers.go:116
 [FAIL] When testing app deployment to the workload cluster [TC1][PR-Blocking] [It] Should be able to download an HTML from the app deployed to the workload cluster
 /jenkins/workspace/capc-e2e-new/test/e2e/deploy_app.go:109
 [TIMEDOUT] When testing multiple CPs in a shared network with kubevip [It] Should successfully create a cluster with multiple CPs in a shared network
 /jenkins/workspace/capc-e2e-new/test/e2e/shared_network_kubevip.go:59

Ran 27 of 32 Specs in 10816.449 seconds
FAIL! - Suite Timeout Elapsed -- 22 Passed | 5 Failed | 0 Pending | 5 Skipped
--- FAIL: TestE2E (10816.45s)
FAIL

[blueorangutan]

Test Results : (tid-604)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.31.6
Kubernetes Version upgrade from: v1.30.10
Kubernetes Version upgrade to: v1.31.6
CloudStack Version: 4.20
Template: ubuntu-2404-kube
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr416-sl-604.zip

Summarizing 3 Failures:
 [PANICKED!] When testing machine remediation [It] Should replace a machine when it is destroyed
 /root/sdk/go1.22.12/src/runtime/panic.go:261
 [FAIL] When the specified resource does not exist When starting with a healthy cluster [It] Should fail to upgrade control plane machine due to insufficient compute resources
 /jenkins/workspace/capc-e2e-new/test/e2e/invalid_resource.go:253
 [FAIL] When testing Kubernetes version upgrades [It] Should successfully upgrade kubernetes versions when there is a change in relevant fields
 /root/go/pkg/mod/sigs.k8s.io/cluster-api/test@v1.4.8/framework/controlplane_helpers.go:193

Ran 29 of 32 Specs in 8840.826 seconds
FAIL! -- 26 Passed | 3 Failed | 0 Pending | 3 Skipped
--- FAIL: TestE2E (8840.83s)
FAIL

[rohityadavcloud]

LGTM - code & new tests. Needs one more review @weizhouapache and/or @g-gaston ? We still must review/test again after upgrading CAPI.

[weizhouapache]

code and test results look good

/lgtm