feat: Become IDNA aware in Plan and DomainFilter

[kimsondrup]

Description

Ensure support for Internationalized Domain Names for Applications (aka. domains using Unicode) using golang.org/x/net/idna

Disclaimer, this is my first Go code so please look at it with extra skepticism

This PR replaces #4689 as I no longer have access to the neticdk organization

Checklist

• Unit tests updated
• End user documentation updated

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: kimsondrup / name: Kim Sondrup (f5c8ceb)

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign raffo for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Hi @kimsondrup. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ivankatliarchuk]

You may need to do

if err != nil {
    log.Warnf(`Got error while parsing domain %s: %v`, domain, err)
     return domain
}

[kimsondrup]

Agreed, we might as well

[kimsondrup]

It has been so long since I started this PR that I myself forgot why I did it the way I did.

The raw domain fallback allows unintended bypasses. A single failing subdomain prevents the rest of the domain from being encoded.

Normalizing each part of the domain is possible, but I'm unsure it's the most elegant solution if we are to also use dna.Lookup.ToUnicode in other places.

[kimsondrup]

I like to just trust that the IDNA lib developers know more about how to handle this problem then whatever we can come up with. But one solution could be something like this (not tested yet, just an example)

// normalizeDomain converts a domain to a canonical form, so that we can filter on it.
// it: trim "." suffix, get Unicode version of domain compliant with Section 5 of RFC 5891
func normalizeDomain(domain string) string {
        domain = strings.TrimSuffix(domain, ".")
        labels := strings.Split(domain, ".")
        normalizedLabels := make([]string, len(labels))

        for i, label := range labels {
                s, err := idna.Lookup.ToUnicode(label)
                if err != nil {
                        log.Warnf(`Got error while parsing domain label %s of domain %s: %v`, label, domain, err)
                        normalizedLabels[i] = label // Use original label on error
                } else {
                        normalizedLabels[i] = s // Use normalized label on success
                }
        }

        return strings.Join(normalizedLabels, ".")
}

[kimsondrup]

Example of solutions when using input xn--nordic--w1a.xn--xn--kItty-pd34d-hn01b3542b.com

Logic	Output
Return domain on err != nil	xn--nordic--w1a.xn--xn--kItty-pd34d-hn01b3542b.com
Encode each label seperatly and skip encoding on err != nil	nordic-ø.xn--xn--kItty-pd34d-hn01b3542b.com
Always return result of idna.Lookup.ToUnicode()	nordic-ø.xn--kitty-點看pd34d.com

Given the goal here is to normalize the domain and in the spirit of keeping it simple I believe that we should return the result of idna.Lookup.ToUnicode() even if some of the encoding experiences errors.

[ivankatliarchuk]

/label tide/merge-method-squash

[mloiseleur]

/ok-to-test

[ivankatliarchuk]

fixes #5090

[mloiseleur]

@kimsondrup Do you think you can address review comments and rebase this PR ?

[lexisother]

Hi all,
My issue (#5090) was linked to this PR by @ivankatliarchuk, and I was hoping it'd fix it, but after having built my own Docker image off @kimsondrup's branch and using it in a deployment to my cluster, it still displays the exact same message I was hoping not to see:

DEBU[0003] no zoneIDFilter configured, looking at all zones
DEBU[0004] Skipping record a.xn--ccka2b6azt.xn--q9jyb4c because no hosted zone matching record DNS Name was detected
DEBU[0004] Skipping record a.a.xn--ccka2b6azt.xn--q9jyb4c because no hosted zone matching record DNS Name was detected

My ingress config can be found in my original issue, but I believe this PR to not fix the issue I described. Also, I've got a txtPrefix of %{record_type}. so that's why it shows a.a. in the second message.

[ivankatliarchuk]

Worth to submit a soluiton, if this PR is not going to resolve it

[lexisother]

I'm just providing whatever context I have, if I had a solution I'd have submitted a PR instead of an issue.
I've got a tiny bit more info, namely that Cloudflare (in my case) returns the zone name as "アリクシア.みんな", whereas external-dns likely tries finding the zone by xn--ccka2b6azt.xn--q9jyb4c, which obviously then doesn't match anything.

I hope this is enough to help someone make the necessary adjustments. I'll keep poking at it myself as well.

[lexisother]

I managed to resolve the problem!

time="2025-03-04T00:57:46Z" level=info msg="Changing record." action=CREATE record=a.xn--ccka2b6azt.xn--q9jyb4c ttl=1 type=A zone=49619bc8e3ea12578435348ffa4707f1
time="2025-03-04T00:57:46Z" level=info msg="Changing record." action=CREATE record=a.a.xn--ccka2b6azt.xn--q9jyb4c ttl=1 type=TXT zone=49619bc8e3ea12578435348ffa4707f1

Horrible hack, but this works for me. Up to you folks if this is a "good" fix or not, but I'd wager it breaks things unrelated to my specific usecase.
Here's the changes to provider/zonefinder.go:

func (z ZoneIDName) FindZone(hostname string) (suitableZoneID, suitableZoneName string) {
+ 	name, err := idna.Lookup.ToUnicode(hostname)
+ 	if err != nil {
+ 		name = hostname
+ 	}
	for zoneID, zoneName := range z {
- 		if hostname == zoneName || strings.HasSuffix(hostname, "."+zoneName) {
+		if name == zoneName || strings.HasSuffix(name, "."+zoneName) {
			if suitableZoneName == "" || len(zoneName) > len(suitableZoneName) {
				suitableZoneID = zoneID
				suitableZoneName = zoneName
			}
		}
	}
	return
}

[ivankatliarchuk]

Hi @lexisother. Would you mind creating a pull request?

[mloiseleur]

@kimsondrup You need to fix the cla before we can proceed

[kimsondrup]

@kimsondrup You need to fix the cla before we can proceed

Working on the CLA.

But I also noticed that a suggested changes has some unintended side effects, so I am also awaiting feedback on that.

[ivankatliarchuk]

Hi @kimsondrup have you tested on a cluster real or local?

Could you also provide a set of manual test steps using manifests and kubectl commands and arguments for external-dns?

Example
#5085 (comment)

[kimsondrup]

Hi @ivankatliarchuk,

Yep I tested it, but it was some months ago.
I will have some test steps ready in two weeks time so that you can reproduce it.

If anyone else want to do it before me then some sample domains can be found in the units tests of this PR.