Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat:gwapi] Add context-aware TargetGroupConfiguration and LoadBalancerConfiguration CRDs for Gateway API #4096

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

shraddhabang
Copy link
Collaborator

@shraddhabang shraddhabang commented Mar 18, 2025

Description

This commit implements a new context-aware TargetGroupConfiguration mechanism for the Gateway API. We are introducing a new Namespace scoped TargetGroupConfiguration CRD which will give more granular control over Target group properties for the tgs provisioned for Gateway API resources. It adds a RouteConfiguration map to the TargetGroupConfiguration CRD, enabling users to define Target Group properties specific to individual routes. Users can use the same map to define the default configuration for the routes using "::" as route name to provide fallback configuration. A longest-prefix matching algorithm will be implemented to resolve the correct TargetGroupDefinition based on the route name (KIND:NAME:NAMESPACE).

This change addresses limitations in the previous approach taken by the controller in Ingress/Service resources, enhancing service reusability and providing granular control over Target Group properties.

Example CRD looks like this

apiVersion: gateway.k8s.aws/v1beta1
kind: TargetGroupConfiguration
metadata:
  name: test-tg-config
  namespace: example-ns
spec:
  targetReference:
    name: "game-2048"
  routeConfigurations:
    - name: "::example-ns"
      targetGroupProps:
        vpcID: "vpc-23467812w"
        ipAddressType: "ipv4"
        tags:
          - key: "hello"
            value: "world"
        targetGroupAttributes:
          - key: "deregistration_delay.timeout_seconds"
            value: "30"
          - key: "slow_start.duration_seconds"
            value: "30"
    - name: "TCPRoute:tcproute1:example-ns"
      targetGroupProps:
        targetType: "ip"
        healthCheckConfig:
          healthyThresholdCount: 5
          healthCheckInterval: 34
          healthCheckPath: "/healthz-tcp"
          healthCheckPort: 443
          healthCheckProtocol: "tcp"
          healthCheckTimeout: 110
          unhealthyThresholdCount: 2
          matcher:
            httpCode: "200"
    - name: "HTTPRoute:httproute1:example-ns"
      targetGroupProps:
        targetType: "instance"
        protocolVersion: "grpc"
        healthCheckConfig:
          healthyThresholdCount: 6
          healthCheckInterval: 56
          healthCheckPath: "/healthz-http"
          healthCheckPort: 80
          healthCheckProtocol: "http"
          healthCheckTimeout: 90
          unhealthyThresholdCount: 3
          matcher:
            grpcCode: "80"
        nodeSelector:
          matchExpressions:
            - key: "k2"
              operator: "o2"

Notes: All the target groups provisioned for the routes in example-ns namespace for the service specified in targetReference will by default have vpcID, IpAddressType, Tags and TargetGroupAttributes values specified in route section named "::example-ns". The specific tg properties will be applied to specific tgs provisioned for Routes defined in CRD.

As this feature is currently under development, this PR also adds this new CRD to separate directory to keep our existing IngressClassParams and TargetGroupBinding CRDs segregated.

The LoadBalancerConfiguration CRD is added for configuration load balancer config at a granular level for both GatewayClass and Gateways.

Example:

apiVersion: gateway.k8s.aws/v1beta1
kind: LoadBalancerConfiguration
metadata:
  name: lb-config-test
  namespace: my-ns
spec:
  loadBalancerName: my-loadbalancer
  ipAddressType: ipv4
  scheme: internal
  securityGroups: [sg-xxxx, nameOfSg1, nameOfSg2]
  loadBalancerSubnets:
    - identifier: subnet-1234
      eipAllocation: eipalloc-xyz
      privateIPv4Allocation: 128.98.76.8
    - identifier: subnet-4567
      eipAllocation: eipalloc-abc
  listenerConfigurations:
    - protocolPort: 'HTTPS:443'
      defaultCertificate: arn:aws:acm:us-east-1:123456789:certificate/test-arn-1
      certificates: ['arn:aws:acm:us-east-1:123456789:certificate/test-arn-2','arn:aws:acm:us-east-1:123456789:certificate/test-arn-3']
      listenerAttributes:
        - key: routing.http.response.server.enabled
          value: "true"
    - protocolPort: 'HTTPS:8080'
      defaultCertificate: arn:aws:acm:us-east-1:123456789:certificate/test-arn-2
      certificates: [ 'arn:aws:acm:us-east-1:123456789:certificate/test-arn-5','arn:aws:acm:us-east-1:123456789:certificate/test-arn-4' ]
  loadBalancerAttributes:
    - key: deletion_protection.enabled
      value: 'true'
    - key: idle_timeout.timeout_seconds
      value: '120'

Checklist

  • Added tests that cover your change (if possible)
  • Added/modified documentation as required (such as the README.md, or the docs directory)
  • Manually tested
  • Made sure the title of the PR is a good description that can go into the release notes

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

  • Backfilled missing tests for code in same general area 🎉
  • Refactored something and made the world a better place 🌟

Sorry, something went wrong.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: shraddhabang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Mar 18, 2025
@shraddhabang shraddhabang changed the title feat:gwapi Add context-aware TargetGroupConfiguration CRDs [feat:gwapi] Add context-aware TargetGroupConfiguration CRDs Mar 18, 2025
@k8s-ci-robot k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Mar 18, 2025
//
// Defaults to "Service" when not specified.
//
// ExternalName services can refer to CNAME DNS records that may live
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably should drop the ExternalName blurb as it's not applicable to our use-case. We just want the CRD to attach to a service.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@shraddhabang shraddhabang changed the title [feat:gwapi] Add context-aware TargetGroupConfiguration CRDs [feat:gwapi] Add context-aware TargetGroupConfiguration and LoadBalancerConfiguration CRDs for Gateway API Mar 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants