Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistencies in the docs around InferencePool #510

Open
christian-posta opened this issue Mar 15, 2025 · 0 comments
Open

Inconsistencies in the docs around InferencePool #510

christian-posta opened this issue Mar 15, 2025 · 0 comments

Comments

@christian-posta
Copy link

I think I'm seeing some inconsistency in the docs wrt names and spec definitions.

From this page: https://gateway-api-inference-extension.sigs.k8s.io/concepts/api-overview/#inferencepool

InferencePool represents a set of Inference-focused Pods and an extension that will be used to route to them. Within the broader Gateway API resource model, this resource is considered a "backend". In practice, that means that you'd replace a Kubernetes Service with an InferencePool. This resource has some similarities to Service (a way to select Pods and specify a port), but has some unique capabilities. With InferenceModel, you can configure a routing extension as well as inference-specific routing optimizations. For more information on this resource, refer to our InferencePool documentation or go directly to the InferencePool spec.

Specifically "With InferenceModel, you can configure a routing extension...."

This section is about InferencePool, why is it referring to InferenceModel? Seems this is a mistake and should refer to InferencePool.

Additionally, that particular part of the doc is talking about configuring a "routing extension". Checking the spec on the docs, it doesn't show how to configure that (ie, there should be an extensionRef field, but there isn't):
https://gateway-api-inference-extension.sigs.k8s.io/reference/spec/#inferencepoolspec

Image

Let me know if these are oversights that need to be corrected, or if I am missing something.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant