Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: adds robots.txt notice #1208

Merged
merged 8 commits into from
Jan 30, 2024
Merged

Docs: adds robots.txt notice #1208

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
3cf0d51
adds robots.txt notice
ZPain8464 Jan 30, 2024
e6238db
runs prettier
ZPain8464 Jan 30, 2024
22e8114
Update content/docs/reference/routes/public-access.mdx
ZPain8464 Jan 30, 2024
686bb46
Update content/docs/reference/routes/public-access.mdx
ZPain8464 Jan 30, 2024
218fba2
Update content/docs/reference/routes/public-access.mdx
ZPain8464 Jan 30, 2024
0c4f9e3
Update content/docs/reference/routes/public-access.mdx
ZPain8464 Jan 30, 2024
19a7995
rewrites text
ZPain8464 Jan 30, 2024
23a6e87
fixes cspell typo
ZPain8464 Jan 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions content/docs/reference/routes/public-access.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,24 @@ import TabItem from '@theme/TabItem';

## Summary

**Use with caution:** **Public Access** allows all requests for a given route, bypassing authentication and authorization. Suitable for publicly exposed web services.
:::caution

If this setting is enabled, no whitelists (for example, Allowed Users) should be provided in this route.
Because the **Public Access** setting bypasses authentication and authorization checks, you should enable it only for publicly exposed web services.

:::

The **Public Access** setting instructs Pomerium to grant unauthorized and unauthenticated access to all requests to the upstream service. If you enable this setting, no other policy should be provided for the route.

### Robots.txt behavior

By default, Pomerium serves a **robots.txt** response directly, instructing search engines _not_ to crawl the route domain:

```txt
User-agent: *
Disallow: /
```

For routes with `allow_public_unauthenticated_access` enabled, Pomerium will not serve **robots.txt** directly. Instead, requests for `/robots.txt` will be proxied to the upstream service.

## How to configure

Expand Down
Loading