From 3cf0d5129351ed120a217c4f589f8b04f99a14d8 Mon Sep 17 00:00:00 2001 From: zachary painter <60552605+ZPain8464@users.noreply.github.com> Date: Tue, 30 Jan 2024 10:30:52 -0500 Subject: [PATCH 1/8] adds robots.txt notice --- .../docs/reference/routes/public-access.mdx | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/content/docs/reference/routes/public-access.mdx b/content/docs/reference/routes/public-access.mdx index c7ccccab1..ab8fd34eb 100644 --- a/content/docs/reference/routes/public-access.mdx +++ b/content/docs/reference/routes/public-access.mdx @@ -16,9 +16,24 @@ import TabItem from '@theme/TabItem'; ## Summary -**Use with caution:** **Public Access** allows all requests for a given route, bypassing authentication and authorization. Suitable for publicly exposed web services. +:::caution -If this setting is enabled, no whitelists (for example, Allowed Users) should be provided in this route. +Because the **Public Access** setting bypasses authentication and authorization checks, you should only enable it for publicly exposed web services. + +::: + +**Public Access** allows all requests to a given route and bypasses authentication and authorization. If you enable this setting, no whitelists should be provided for the route (for example, **Allowed Users**). + +### Robots.txt behavior + +By default, Pomerium serves a custom **robots.txt** file that instructs user-agents *not* to crawl a given route: + +```txt +User-agent: * +Disallow: / +``` + +For routes with `allow_public_unauthenticated_access` enabled, Pomerium does not overwrite the contents of the **robots.txt** file. ## How to configure From e6238db249a77cc2e2c7d90d4612f79100b792eb Mon Sep 17 00:00:00 2001 From: zachary painter <60552605+ZPain8464@users.noreply.github.com> Date: Tue, 30 Jan 2024 10:33:14 -0500 Subject: [PATCH 2/8] runs prettier --- content/docs/reference/routes/public-access.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/reference/routes/public-access.mdx b/content/docs/reference/routes/public-access.mdx index ab8fd34eb..68c5efb00 100644 --- a/content/docs/reference/routes/public-access.mdx +++ b/content/docs/reference/routes/public-access.mdx @@ -26,7 +26,7 @@ Because the **Public Access** setting bypasses authentication and authorization ### Robots.txt behavior -By default, Pomerium serves a custom **robots.txt** file that instructs user-agents *not* to crawl a given route: +By default, Pomerium serves a custom **robots.txt** file that instructs user-agents _not_ to crawl a given route: ```txt User-agent: * From 22e811428a4b4ac8bdf3a436d7dd78e3d9492481 Mon Sep 17 00:00:00 2001 From: zachary painter <60552605+ZPain8464@users.noreply.github.com> Date: Tue, 30 Jan 2024 16:12:28 -0500 Subject: [PATCH 3/8] Update content/docs/reference/routes/public-access.mdx Co-authored-by: Kenneth Jenkins <51246568+kenjenkins@users.noreply.github.com> --- content/docs/reference/routes/public-access.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/reference/routes/public-access.mdx b/content/docs/reference/routes/public-access.mdx index 68c5efb00..363ca115b 100644 --- a/content/docs/reference/routes/public-access.mdx +++ b/content/docs/reference/routes/public-access.mdx @@ -26,7 +26,7 @@ Because the **Public Access** setting bypasses authentication and authorization ### Robots.txt behavior -By default, Pomerium serves a custom **robots.txt** file that instructs user-agents _not_ to crawl a given route: +By default, Pomerium serves a **robots.txt** response directly, instructing search engines _not_ to crawl the route domain: ```txt User-agent: * From 686bb46f08d6502de02e1276acd7aebed6836b83 Mon Sep 17 00:00:00 2001 From: zachary painter <60552605+ZPain8464@users.noreply.github.com> Date: Tue, 30 Jan 2024 16:12:52 -0500 Subject: [PATCH 4/8] Update content/docs/reference/routes/public-access.mdx Co-authored-by: Kenneth Jenkins <51246568+kenjenkins@users.noreply.github.com> --- content/docs/reference/routes/public-access.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/reference/routes/public-access.mdx b/content/docs/reference/routes/public-access.mdx index 363ca115b..ef7a04317 100644 --- a/content/docs/reference/routes/public-access.mdx +++ b/content/docs/reference/routes/public-access.mdx @@ -33,7 +33,7 @@ User-agent: * Disallow: / ``` -For routes with `allow_public_unauthenticated_access` enabled, Pomerium does not overwrite the contents of the **robots.txt** file. +For routes with `allow_public_unauthenticated_access` enabled, Pomerium will not serve **robots.txt** directly. Instead, requests for `/robots.txt` will be proxied to the upstream service. ## How to configure From 218fba24761574b392d66697e5e6e456959706cb Mon Sep 17 00:00:00 2001 From: zachary painter <60552605+ZPain8464@users.noreply.github.com> Date: Tue, 30 Jan 2024 16:14:19 -0500 Subject: [PATCH 5/8] Update content/docs/reference/routes/public-access.mdx Co-authored-by: Kenneth Jenkins <51246568+kenjenkins@users.noreply.github.com> --- content/docs/reference/routes/public-access.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/reference/routes/public-access.mdx b/content/docs/reference/routes/public-access.mdx index ef7a04317..bace7d6fb 100644 --- a/content/docs/reference/routes/public-access.mdx +++ b/content/docs/reference/routes/public-access.mdx @@ -22,7 +22,7 @@ Because the **Public Access** setting bypasses authentication and authorization ::: -**Public Access** allows all requests to a given route and bypasses authentication and authorization. If you enable this setting, no whitelists should be provided for the route (for example, **Allowed Users**). +**Public Access** allows all requests to a given route and bypasses authentication and authorization. If you enable this setting, no other policy should be provided for the route. ### Robots.txt behavior From 0c4f9e38c30f53c3d3ccaa757cd8bde57e169a6a Mon Sep 17 00:00:00 2001 From: zachary painter <60552605+ZPain8464@users.noreply.github.com> Date: Tue, 30 Jan 2024 16:14:39 -0500 Subject: [PATCH 6/8] Update content/docs/reference/routes/public-access.mdx Co-authored-by: Kenneth Jenkins <51246568+kenjenkins@users.noreply.github.com> --- content/docs/reference/routes/public-access.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/reference/routes/public-access.mdx b/content/docs/reference/routes/public-access.mdx index bace7d6fb..61345449b 100644 --- a/content/docs/reference/routes/public-access.mdx +++ b/content/docs/reference/routes/public-access.mdx @@ -18,7 +18,7 @@ import TabItem from '@theme/TabItem'; :::caution -Because the **Public Access** setting bypasses authentication and authorization checks, you should only enable it for publicly exposed web services. +Because the **Public Access** setting bypasses authentication and authorization checks, you should enable it only for publicly exposed web services. ::: From 19a7995620e00792a25fd012c0fb81a5e50cca85 Mon Sep 17 00:00:00 2001 From: zachary painter <60552605+ZPain8464@users.noreply.github.com> Date: Tue, 30 Jan 2024 16:25:08 -0500 Subject: [PATCH 7/8] rewrites text --- content/docs/reference/routes/public-access.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/reference/routes/public-access.mdx b/content/docs/reference/routes/public-access.mdx index 61345449b..4190c2776 100644 --- a/content/docs/reference/routes/public-access.mdx +++ b/content/docs/reference/routes/public-access.mdx @@ -22,7 +22,7 @@ Because the **Public Access** setting bypasses authentication and authorization ::: -**Public Access** allows all requests to a given route and bypasses authentication and authorization. If you enable this setting, no other policy should be provided for the route. +The **Public Access** setting instructs Pomerium to grant unautherized and unauthenticated access to all requests to the upstream service. If you enable this setting, no other policy should be provided for the route. ### Robots.txt behavior From 23a6e87508fe5bb54a090d7ee6df04af231ac7bc Mon Sep 17 00:00:00 2001 From: zachary painter <60552605+ZPain8464@users.noreply.github.com> Date: Tue, 30 Jan 2024 16:27:44 -0500 Subject: [PATCH 8/8] fixes cspell typo --- content/docs/reference/routes/public-access.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/reference/routes/public-access.mdx b/content/docs/reference/routes/public-access.mdx index 4190c2776..e7c68a90f 100644 --- a/content/docs/reference/routes/public-access.mdx +++ b/content/docs/reference/routes/public-access.mdx @@ -22,7 +22,7 @@ Because the **Public Access** setting bypasses authentication and authorization ::: -The **Public Access** setting instructs Pomerium to grant unautherized and unauthenticated access to all requests to the upstream service. If you enable this setting, no other policy should be provided for the route. +The **Public Access** setting instructs Pomerium to grant unauthorized and unauthenticated access to all requests to the upstream service. If you enable this setting, no other policy should be provided for the route. ### Robots.txt behavior