v2.12.0 Cross Account TG Binding enforcing Availability Zone = All

[listellm]

Bug Description
As documented in the AWS EKS Best Practises Guide under the "Sharing VPC across multiple accounts" section, we are using AWS RAM to share our cluster private and cluster public subnets with other accounts in our organisation. It allows us to reduce hops between our cluster account and our customer account.

When using the new cross-account TG binding feature in v2.12.0, the AWS LB controller forces the AZ to be set to all as shown in your code here.

	if tgb.Spec.IamRoleArnToAssume != "" {
		// If we're interacting with another account, then we should always be setting "all" AZ to allow this
		// target to get registered by the ELB API.
		overrideAzFn = func(_ netip.Addr) bool {
			return true
		}
	}

This is in-turn then denied by the AWS API when using shared subnets as the pod IP is on the same VPC CIDR as the LB subnets.

operation error Elastic Load Balancing v2: RegisterTargets, https response error StatusCode: 400, RequestID: fxxxx-xxxx-xxxx-xxxx--xxxxx, api error ValidationError: The IP address '10.80.45.136' is within the VPC, and cannot have its Availability Zone overridden to 'all' from 'eu-central-1b'

This prevents using cross-account TG binding with shared subnets.

Steps to Reproduce
In the following examples, we will refer to Cluster Owner (CO) and Target Group Owner (TGO) accounts.

1/Ensure that your CO VPC has shared subnets
2/ Create your LB in the TGO account on a shared subnet from CO account
3/ Create a TG Binding utilising the cross-account LB feature per documented instructions
4/ Observe the LB controller logs which will state the above AWS API error

Expected Behavior
The LB controller detects the AZ that the target pod IP is in and sets this on the TG binding request for the AZ instead of forcing all.

Actual Behavior
AWS API denies the request as the source POD IP is within the same VPC CIDR that the LB is in but the LB controller tries to set all which should only be set if the private IP address is outside of the VPC scope

Regression
Was the functionality working correctly in a previous version? N/A. its a newly launched feature
If yes, specify the last version where it worked as expected

Current Workarounds
I have had to keep my LB in my CO account on the CO EKS not utilising the new cross-account feature

Environment

• AWS Load Balancer controller version: 2.12.0
• Kubernetes version: 1.32.0
• Using EKS (yes/no), if so version?: yes - 1.32.0
• Using Service or Ingress: Service
• AWS region: eu-central01

Possible Solution
The LB controller can automatically deduce what AZ the pod IP is within and pass this across in the TG binding request.

[zac-nixon]

Hi! Thanks for reaching out, and I'm sorry for the trouble with the new feature. Can you please DM me the request ID? It would make things easier to debug on my end.

[listellm]

Hi! Thanks for reaching out, and I'm sorry for the trouble with the new feature. Can you please DM me the request ID? It would make things easier to debug on my end.

Hi,

I have sent you an email

[zac-nixon]

Thanks. I got the email. I'll post back later.

[zac-nixon]

Hey, I have found the issue. It doesn't look like a trivial fix so I'm inclined to say that RAM shared VPC don't work with the feature and we can try to fix it in the next release. I'm sorry for the trouble.

[listellm]

Hey, I have found the issue. It doesn't look like a trivial fix so I'm inclined to say that RAM shared VPC don't work with the feature and we can try to fix it in the next release. I'm sorry for the trouble.

Thanks for investigating. I've waited months for this feature, im sure I can wait a little longer 👍🏽

[zac-nixon]

Actually, I might have over complicated the problem. Can you please tell me if this your set up?

btw, the link you posted took me nowhere, I assume this is the article you're referring to. https://aws.amazon.com/blogs/containers/use-shared-vpcs-in-amazon-eks/

Originally, I had thought that you were peering with another VPC and had overlapping subnets. After reading the shared vpc article, I don't think that is the case. Can you confirm:

You're using the same VPC id, vpc-foo in both the CO and TGO accounts?

I had honestly forgot about sharing VPCs when working / reviewing this feature.

Thanks for your patience!

[listellm]

Yes that's the setup. Absolutely no overlapping subnets.

Same VPC ID (vpc-foo), created in the CO account. Then, share subnets from VPC-FOO to other accounts. Then your pods in the CO account and your LB in the TGO account are on VPC FOO. No need for a TGW, Endpoint service etc. Less latency, less hops, reduced cost.

Link here under "Sharing VPC across multiple accounts"