[kind/api-change] Allow wildcard suffix in hostnames

[simonfelding]

What type of PR is this?
/kind gep

What this PR does / why we need it:
It adds support for wildcard suffixes in addition to wildcard prefixes in the Hostname type.
It is fully backwards-compatible.

Which issue(s) this PR fixes:

Fixes #3643

Does this PR introduce a user-facing change?:

Added support for wildcard suffixes in hostnames (like example.*)

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: simonfelding (37a6a93, 0f4be7c)

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: simonfelding
Once this PR has been reviewed and has the lgtm label, please assign aojea for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Welcome @simonfelding!

It looks like this is your first PR to kubernetes-sigs/gateway-api 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/gateway-api has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @simonfelding. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[howardjohn]

I think this needs a lot more discussion around interactions with RFCs, implementation, and between objects.

For example, what is the semantics for matching between a listener hostname suffix match with a HTTPRoute hostname prefix match?

Which implementations can support this today? I am fairly sure envoy does not support suffix SNI matching.

This also has tricky impacts on certs which only support a prefix wildcard.

Further, most suffix match use cases I would imagine are a string suffix match. All wildcards in the API today are a single label wildcard match. So grafana.* would not match grafana.example.com which probably makes the use case not met.

[simonfelding]

@howardjohn Totally agree it needs careful review and discussion of interactions! I haven't been able to think of any that aren't already supported in the major implementations, nginx for example, but there might be something I can't come up with.

Envoy also supports it and it sounds like the developers have already thought these issues through: (Wildcard hosts are supported in the suffix or prefix form).

The top level element in the routing configuration is a virtual host. Each virtual host has a logical name as well as a set of domains that get routed to it based on the incoming request’s host header. This allows a single listener to service multiple top level domain path trees. Once a virtual host is selected based on the domain, the routes are processed in order to see which upstream cluster to route to or whether to perform a redirect.

domains
(repeated string, REQUIRED) A list of domains (host/authority header) that will be matched to this virtual host. Wildcard hosts are supported in the suffix or prefix form.

Domain search order:

1. Exact domain names: www.foo.com.
2. Suffix domain wildcards: *.foo.com or *-bar.foo.com.
3. Prefix domain wildcards: foo.* or foo-*.
4. Special wildcard * matching any domain.

Note
The wildcard will not match the empty string. e.g. *-bar.foo.com will match baz-bar.foo.com but not -bar.foo.com. The longest wildcards match first. Only a single virtual host in the entire route configuration can match on *. A domain must be unique across all virtual hosts or the config will fail to load.

I don't think it has any interaction with certs? It's true that certs only match on a single wildcard subdomain; The gateway API already violates this by matching *.example.com with cringle.bingle.example.com if there is no better match. SNI is all about finding the right certificate for a given hostname before the cert is served and the TLS handshake begins.

Let's say a server has two virtual hosts, grafana.example.com and grafana.example.org. When a HTTPS request comes in, the client_hello stage is still unencrypted and contains the requested hostname.
In the case of Host: grafana.example.com and a HTTPRoute matching the exact hostname, it's obvious what cert to serve, in the case of Host: grafana.another.com, it should be rejected as there are no matching certs. In the case of Host: grafana.example.org, the principle of the best match should be followed, where a HTTPRoute with grafana.example.org and .*.example.org is a better match than grafana.*. This is how Envoy does it too. Either way, this decision is made before the cert is served.

[howardjohn]

Envoy supports it for HTTP Host headers, but the API you are changing here impacts SNI where its not supported: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/listener/v3/listener_components.proto#config-listener-v3-filterchainmatch

[simonfelding]

It's not in the given examples in that document, but I don't see why it wouldn't work? SNI is just a simple string extracted from the client_hello.

I'm not deep enough in the envoy code to be sure, but from a superficial look at the code I can't see any validation that treats the SNI as a special type of hostname string that cannot be wildcard-matched like other hostnames can.

Nginx does it too:

When searching for a virtual server by name, if name matches more than one of the specified variants, e.g. both wildcard name and regular expression match, the first matching variant will be chosen, in the following order of precedence:

1. exact name
2. longest wildcard name starting with an asterisk, e.g. “*.example.org”
3. longest wildcard name ending with an asterisk, e.g. “mail.*”
4. first matching regular expression (in order of appearance in a configuration file)

The name is determined in the following order:

Virtual server selection
First, a connection is created in a default server context. Then, the server name can be determined in the following request processing stages, each involved in server configuration selection:

1. during SSL handshake, in advance, according to SNI

2. after processing the request line

3. after processing the Host header field

4. if the server name was not determined after processing the request line or from the Host header field, nginx will use the empty name as the server name.

At each of these stages, different server configurations can be applied.

I also know for sure that the Citrix Netscaler has no problem treating the client hello SNI string as any other string and making complex routing decisions based on that - prior to serving a cert.

tl;dr: SNI just means sending the hostname before the cert is served, which makes routing a TLS connected work like unencrypted HTTP. After hostname is matched with a server (by SNI or by Host header), a cert is served. I'm pretty confident this GEP does not impact SNI - it's only about whether or not we can allow a Gateway implementation to match a hostname to a string ending with a wildcard :)

[howardjohn]

I'm not deep enough in the envoy code to be sure, but from a superficial look at the code I can't see any validation that treats the SNI as a special type of hostname string that cannot be wildcard-matched like other hostnames can.

https://github.com/envoyproxy/envoy/blob/8784289b97003c3b2a36824b4a810112d62b7bfa/source/common/listener_manager/filter_chain_manager_impl.cc#L121-L123

I'm pretty confident this GEP does not impact SNI

Its changing the valid set of matchers against SNI so it is.

[simonfelding]

Dang, nice find. Youre right, Envoy does not support it right now because of that line. Deserves a PR to that as well :)

But anyway, why should the Gateway API decide whether or not the implementation can get the wildcard-suffixed hostname? I think it's pretty clear that nginx supports it for example, and it looks like to me that it's just a matter of a submitting a minor patch to make envoy support it too.

[youngnick]

Thanks for this PR @simonfelding, I appreciate you being so proactive in working on solving a problem you've identified.

However, there are a few problems with accepting this as is, as @howardjohn said earlier. There's the "Can data planes even do this?" question, which it seems that some may be able to, and the "hostname also matches SNI, and SNI can't do that" problem.

However, the larger problem for me is the one about the interaction between the hostname field on the Gateway Listener, and the hostname field on a HTTPRoute.

These fields have a reasonably complex interaction already:

• The hostname field on Gateway Listener specifies which hostnames should be allowed by that Listener, covering both SNI and Host header matches.
• The hostname field on HTTPRoute specifies which hostnames should be allowed by that specific HTTPRoute, and is also used for picking which HTTPRoutes can attach to which Gateway Listeners.
• When both Gateway Listener hostname and HTTPRoute hostname are specified, then they need to intersect, where intersection is defined in the HTTPRoute hostname field

  gateway-api/apis/v1/httproute_types.go

  Lines 59 to 114 in 8c7f335

  	// Hostnames defines a set of hostnames that should match against the HTTP Host
  	// header to select a HTTPRoute used to process the request. Implementations
  	// MUST ignore any port value specified in the HTTP Host header while
  	// performing a match and (absent of any applicable header modification
  	// configuration) MUST forward this header unmodified to the backend.
  	//
  	// Valid values for Hostnames are determined by RFC 1123 definition of a
  	// hostname with 2 notable exceptions:
  	//
  	// 1. IPs are not allowed.
  	// 2. A hostname may be prefixed with a wildcard label (`*.`). The wildcard
  	// label must appear by itself as the first label.
  	//
  	// If a hostname is specified by both the Listener and HTTPRoute, there
  	// must be at least one intersecting hostname for the HTTPRoute to be
  	// attached to the Listener. For example:
  	//
  	// * A Listener with `test.example.com` as the hostname matches HTTPRoutes
  	// that have either not specified any hostnames, or have specified at
  	// least one of `test.example.com` or `*.example.com`.
  	// * A Listener with `*.example.com` as the hostname matches HTTPRoutes
  	// that have either not specified any hostnames or have specified at least
  	// one hostname that matches the Listener hostname. For example,
  	// `*.example.com`, `test.example.com`, and `foo.test.example.com` would
  	// all match. On the other hand, `example.com` and `test.example.net` would
  	// not match.
  	//
  	// Hostnames that are prefixed with a wildcard label (`*.`) are interpreted
  	// as a suffix match. That means that a match for `*.example.com` would match
  	// both `test.example.com`, and `foo.test.example.com`, but not `example.com`.
  	//
  	// If both the Listener and HTTPRoute have specified hostnames, any
  	// HTTPRoute hostnames that do not match the Listener hostname MUST be
  	// ignored. For example, if a Listener specified `*.example.com`, and the
  	// HTTPRoute specified `test.example.com` and `test.example.net`,
  	// `test.example.net` must not be considered for a match.
  	//
  	// If both the Listener and HTTPRoute have specified hostnames, and none
  	// match with the criteria above, then the HTTPRoute is not accepted. The
  	// implementation must raise an 'Accepted' Condition with a status of
  	// `False` in the corresponding RouteParentStatus.
  	//
  	// In the event that multiple HTTPRoutes specify intersecting hostnames (e.g.
  	// overlapping wildcard matching and exact matching hostnames), precedence must
  	// be given to rules from the HTTPRoute with the largest number of:
  	//
  	// * Characters in a matching non-wildcard hostname.
  	// * Characters in a matching hostname.
  	//
  	// If ties exist across multiple Routes, the matching precedence rules for
  	// HTTPRouteMatches takes over.
  	//
  	// Support: Core
  	//
  	// +optional
  	// +kubebuilder:validation:MaxItems=16

  And there is some Listener specific definitions at

  gateway-api/apis/v1/gateway_types.go

  Lines 337 to 385 in 8c7f335

  	// Hostname specifies the virtual hostname to match for protocol types that
  	// define this concept. When unspecified, all hostnames are matched. This
  	// field is ignored for protocols that don't require hostname based
  	// matching.
  	//
  	// Implementations MUST apply Hostname matching appropriately for each of
  	// the following protocols:
  	//
  	// * TLS: The Listener Hostname MUST match the SNI.
  	// * HTTP: The Listener Hostname MUST match the Host header of the request.
  	// * HTTPS: The Listener Hostname SHOULD match both the SNI and Host header.
  	// Note that this does not require the SNI and Host header to be the same.
  	// The semantics of this are described in more detail below.
  	//
  	// To ensure security, Section 11.1 of RFC-6066 emphasizes that server
  	// implementations that rely on SNI hostname matching MUST also verify
  	// hostnames within the application protocol.
  	//
  	// Section 9.1.2 of RFC-7540 provides a mechanism for servers to reject the
  	// reuse of a connection by responding with the HTTP 421 Misdirected Request
  	// status code. This indicates that the origin server has rejected the
  	// request because it appears to have been misdirected.
  	//
  	// To detect misdirected requests, Gateways SHOULD match the authority of
  	// the requests with all the SNI hostname(s) configured across all the
  	// Gateway Listeners on the same port and protocol:
  	//
  	// * If another Listener has an exact match or more specific wildcard entry,
  	// the Gateway SHOULD return a 421.
  	// * If the current Listener (selected by SNI matching during ClientHello)
  	// does not match the Host:
  	// * If another Listener does match the Host the Gateway SHOULD return a
  	// 421.
  	// * If no other Listener matches the Host, the Gateway MUST return a
  	// 404.
  	//
  	// For HTTPRoute and TLSRoute resources, there is an interaction with the
  	// `spec.hostnames` array. When both listener and route specify hostnames,
  	// there MUST be an intersection between the values for a Route to be
  	// accepted. For more information, refer to the Route specific Hostnames
  	// documentation.
  	//
  	// Hostnames that are prefixed with a wildcard label (`*.`) are interpreted
  	// as a suffix match. That means that a match for `*.example.com` would match
  	// both `test.example.com`, and `foo.test.example.com`, but not `example.com`.
  	//
  	// Support: Core
  	//
  	// +optional

  .
• The reason that we picked label-based, leftward-only wildcard match is that it makes this interaction easier to define - in particular, using whole labels rather than a string match makes defining intersection much easier (although it makes it harder for many implementations to actually implement).

Adding a rightwards, string-wise match also requires understanding and updating all of that documentation, and the required conformance tests (particularly ones like https://github.com/kubernetes-sigs/gateway-api/blob/main/conformance/tests/gateway-http-listener-isolation.go that test how the hostname intersection works).

I'll be honest here, I think that the decrease in amount of required config may not be worth the increase in complexity this would entail, as users of the API often take a long while to understand this intersection as it is now.

That said, I'd be happy to be proven wrong here, so I encourage you to take a look at those places, and look at adding language to handle the interactions.