DOCS: Production Deployment rewrite (#3129) (#3196)

* sort and consolidate service url keys

* Production doc first pass

* move Production Deployment to top level

* Apply suggestions from code review

Co-authored-by: Travis Groth <travisgroth@users.noreply.github.com>

* Update docs/docs/production-deployment.md

Co-authored-by: Travis Groth <travisgroth@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: cmo-pomerium <91488121+cmo-pomerium@users.noreply.github.com>

Co-authored-by: Travis Groth <travisgroth@users.noreply.github.com>
Co-authored-by: cmo-pomerium <91488121+cmo-pomerium@users.noreply.github.com>

Co-authored-by: Alex Fornuto <afornuto@pomerium.com>
Co-authored-by: Travis Groth <travisgroth@users.noreply.github.com>
Co-authored-by: cmo-pomerium <91488121+cmo-pomerium@users.noreply.github.com>

docs/.vuepress/config.js

@@ -187,6 +187,10 @@ module.exports = {
             "k8s/ingress",
           ]
         },
+        {
+          title: "Production Deployment",
+          path: "/docs/production-deployment",
+        },
         {
           title: "Topics",
           collapsable: false,
@@ -201,7 +205,6 @@ module.exports = {
             "topics/original-request-context",
             "topics/mutual-auth",
             "topics/ppl",
-            "topics/production-deployment",
             "topics/programmatic-access",
             "topics/single-sign-out",
             "topics/load-balancing",

docs/docs/production-deployment.md

@@ -0,0 +1,161 @@
+---
+title: Production Deployment
+description: >-
+ This article covers production deployment requirements and concerns
+meta:
+  - name: keywords
+    content: pomerium, identity access proxy, production, deployment, scale, scaling, horizontal,
+---
+
+
+# Production Deployment
+
+This page covers the topic of running Pomerium in a production configuration.
+
+Before deploying Pomerium to Production, you should have already tested Pomerium in one or more demo environments and confirmed:
+
+ - A working connection to your [IdP](/docs/glossary.md#identity-provider).
+ - Working test routes to your upstream services, including [JWT verification] where applicable.
+ - For Pomerium Enterprise, a working demo of the Pomerium Enterprise Console, with confirmed access for your [Admins and Managers][rbac].
+
+
+## Service Mode
+
+Pomerium is designed to be run in two modes: All-In-One or Split Service. These modes are not mutually exclusive, meaning you can run one or multiple instances of Pomerium in all-in-one mode, and spin up additional instances for specific components as needed.
+
+Each instance of Pomerium runs in all-in-one mode unless specified to run as a specific component by the `services` key. See [All-In-One vs Split Service mode](/reference/readme.md#all-in-one-vs-split-service-mode) for more details.
+
+It's important to note that any production deployment with more than one instance of Pomerium (in any combination of modes) should be configured to use Redis as the [`databroker_storage_type`](/reference/readme.md#data-broker-storage-type). See [Data Storage - Backends](/docs/topics/data-storage.md#backends) for more details.
+
+### All-in-One
+
+It may be desirable to run in "all-in-one" mode in smaller deployments or while testing. This reduces the resource footprint and simplifies DNS configuration. All URLs point at the same Pomerium service instance.
+
+### Discrete Services
+
+In larger footprints, it is recommended to run Pomerium as a collection of discrete service clusters. This limits blast radius in the event of vulnerabilities and allows for per-service [scaling](#scaling) and monitoring.
+
+Please also see [Architecture](/docs/architecture.md) for information on component interactions.
+
+## Scaling
+
+In split service mode, you have the opportunity to scale the components of Pomerium independently.
+
+All of Pomerium's components are designed to be stateless, and may all be scaled horizontally or vertically. In general, horizontal scaling is recommended. Vertical scaling will lead to diminished returns after ~8 vCPUs.
+
+The Databroker service, which is responsible for session and identity related data, must be [configured for external persistence](/docs/topics/data-storage.md) to be fully stateless.
+
+Pomerium's individual components can be divided into two categories; the data plane and control plane. Regardless of which mode you run Pomerium in, we strongly recommend multiple instances of each service for fault tolerance.
+
+::: tip
+Our [Helm deployment](/docs/k8s/helm.md) supports [Horizontal Pod Autoscaling](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/).
+:::
+
+### Data Plane
+
+#### Proxy
+
+The Proxy service, as the name implies, is responsible for proxying all user traffic, in addition to performing checks to the Authorization service. The proxy is directly in path for user traffic.
+
+Proxy will need resources scaled in conjunction with request count and may need average request size accounted for. The heavier your user traffic, the more resources the Proxy service should have provisioned.
+
+#### Authorize
+
+The Authorize service is responsible for policy checks during requests. It is in the hot path for user requests but does not directly handle user traffic.
+
+Authorize will need resources scaled in conjunction with request count. Request size and type should be of a constant complexity. In most environments, Authorize and Proxy will scale linearly with request volume (user traffic).
+
+Note that the compute cost of each request is about two times (2x) greater for the Authorize service compared to Proxy; if Proxy utilizes 5% of CPU resources, Authorize would likely use 10%.
+
+### Control Plane
+
+#### Authenticate
+
+The Authenticate service handles session cookie setup, session storage, and authentication with your Identity Provider.
+
+Authenticate requires significantly fewer resources than other components due to the only-occasional requirement to establish new sessions. This happens when users first sign in, and when their authentication expires (determined by your IdP).
+
+Add resources to the Authenticate service if you have a high session/user churn rate. The requests should be constant time and complexity, but may vary by Identity Provider implementation. Resources for the Authenticate service should scale *roughly* with your total user count.
+
+Regardless of the low resource utilization, we recommend running no less than 2 instances for resiliency and fault tolerance.
+
+### Databroker
+
+The Databroker service is responsible for background identity data retrieval and storage. It is in the hot path for user authentication. However, it does not directly handle user traffic and is not in-path for authorization decisions.
+
+The Databroker service does not require significant resources, as it provides streaming updates of state changes to the other services. There will be utilization spikes when Authorize services are restarted and perform an initial synchronization.
+
+Databroker resource requirements scale with the number of replicated services in the [data plane](#data-plane). That is to say, additional instances of the Proxy and Authorize services will increase demand on Databroker. Additionally, the size of the user directory contributes to the resource requirements for the data storage, Redis.
+
+The eBook Redis in Action has a chapter on [Scaling Redis](https://redis.com/ebook/part-3-next-steps/chapter-10-scaling-redis/). Pomerium is compatible with both Redis HA and Redis cluster scaling.
+
+In many deployments, 2 replicas of Databroker is enough to provide resilient service.
+
+::: warning
+In a production configuration, Databroker CPU/IO utilization also translates to IO load on the [underlying storage system](/docs/topics/data-storage.md). Ensure it is scaled accordingly!
+:::
+
+## Load Balancing
+
+In any production deployment, running multiple replicas of each Pomerium service is strongly recommended. Each service has slightly different concerns about utilizing the replicas for high availability and scaling, enumerated below.
+
+### Proxy
+
+You should deploy Layer 4 load balancing between end users and Pomerium Proxy services to provide high availability and horizontal scaling. Do not use L7 load balancers, since the Proxy service handles redirects, sticky sessions, etc.
+
+Note that deployments on Kubernetes can utilize The [Pomerium Ingress Controller](/docs/k8s/ingress.md) to simplify configuration.
+
+### Authenticate
+
+The suggested practice is to use the Pomerium Proxy service to load-balance Authenticate. Alternately, you could use an independent Layer 4 or Layer 7 load balancer, but this increases complexity.
+
+
+### Authorize and Databroker
+
+You do **not** need to provide a load balancer in front of Authorize and Databroker services. Both utilize GRPC and have special requirements if you should choose to use an external load balancer. GRPC can perform client based load balancing and is the best architecture for most configurations.
+
+By default, Pomerium gRPC clients will automatically connect to all IPs returned by a DNS query for the name of an upstream service. They will then regularly re-query DNS for changes to the Authorize or Databroker service cluster. Health checks and failover are automatic.
+
+You can also define multiple domain names for Databroker or Authorize services with the [`databroker_service_url`](/reference/readme.md#data-broker-service-url) and [`authorize_service_urls`](/reference/readme.md#authorize-service-url) shared config keys.
+
+## High Availability
+
+As mentioned in [scaling](#scaling), Pomerium components themselves are stateless and support horizontal scale out for both availability and performance reasons.
+
+A given service type does not require communication with its peer instances to provide high availability. E.g., a Proxy service instance does not communicate with Proxy instances.
+
+Regardless of the service mode, it is recommended you run at least 2 instances of Pomerium with as much physical and logical separation as possible. For example, in Cloud environments, you should deploy instances of each service to at least 2 different zones. On-prem environments should deploy >=2 instances to independent hardware.
+
+Ensure that you have enough spare capacity to handle the scope of your failure domains.
+
+::: warning
+Multiple replicas of Databroker or all-in-one service are only supported with [external storage](/docs/topics/data-storage.md) configured
+:::
+
+## SSL/TLS Certificates
+
+Pomerium utilizes TLS end to end, so the placement, certificate authorities and covered subjects are critical to align correctly.
+
+In a typical deployment, a minimum of two certs are required:
+
+- A wildcard certificate which covers the external `from` names, the Proxy service's external name and the Authenticate service's external name
+  - Must be trusted by user browsers or clients
+  - Must cover subject names from the user perspective
+- A certificate which covers the Authorize service's name
+  - Must be trusted by the Proxy
+  - Must cover the subject name from the Proxy's perspective
+
+If you have L7 load balancing in front of the Proxy/Authenticate:
+
+- Your wildcard certificate should live on the load balancer
+- Your Authenticate and Proxy services will need a certificate accepted by the load balancer
+- Your load balancer can be configured to verify the identity of the Authenticate and Proxy certificates
+
+If you have TLS enabled applications behind the proxy:
+
+- you may provide a client certificate for the Proxy service to connect to downstream services with and verify
+- the Proxy may be configured to verify the name and certificate authority of downstream services with either the standard Root CA bundle or a custom CA
+
+
+[JWT verification]: /docs/topics/mutual-auth.md#jwt-verification-application-based-mutual-authentication
+[rbac]: /enterprise/concepts.md#rbac-for-enterprise-console-users

docs/docs/topics/production-deployment.md

@@ -1,127 +0,0 @@
----
-title: Production Deployment
-description: >-
-  This article covers production deployment requirements and concerns
----
-
-# Production Deployment
-
-This page covers the topic of running Pomerium in a production configuration. See the [quick start section](/docs/install/readme.md) for canned example configurations.
-
-Please also see [architecture](/docs/architecture.md) for information on component interactions.
-
-## Service Mode
-
-For configuration of the service mode, see [Service Mode](/reference/readme.md#service-mode).
-
-### All in One
-
-In smaller deployments or while testing, it may be desirable to run in "all-in-one" mode. This reduces resource footprint and simplifies DNS configuration. All URLs point at the same pomerium service instance.
-
-### Discrete Services
-
-In larger footprints, it is recommended to run Pomerium as a collection of discrete service clusters. This limits blast radius in the event of vulnerabilities and allows for per-service [scaling](#scaling) and monitoring.
-
-## Scaling
-
-In dedicated service mode, you have the opportunity to scale the components of Pomerium independently.
-
-All of Pomerium's components are designed to be stateless, and may all be scaled horizontally or vertically. In general, horizontal scaling is recommended.  Vertical scaling will lead to diminished returns after ~8 vCPUs.
-
-The Data Broker service, which is responsible for session and identity related data, must be [configured for external persistence](/docs/topics/data-storage.md) to be fully stateless.
-
-### Proxy
-
-Proxy service, as the name implies, is responsible for proxying all user traffic, in addition to performing Authentication and Authorization checks. The proxy is directly in path for user traffic.
-
-Proxy will need resources scaled in conjunction with request count and may need average request size accounted for. The heavier your user traffic, the more resources the Proxy service should have provisioned.
-
-### Authorize
-
-The Authorize service is responsible for policy checks during requests. It is in the hot path for user requests but does not directly handle user traffic.
-
-Authorize will need resources scaled in conjunction with request count. Request size and type should be of a constant complexity.  In most environments, Authorize and Proxy will scale linearly with request volume.
-
-### Authenticate
-
-The Authenticate service handles session cookie setup, session storage, and authentication with your Identity Provider.
-
-Authenticate requires significantly fewer resources than other components due to the only-occasional requirement to establish new sessions.  Add resources to the Authenticate service if you have a high session/user churn rate. The requests should be constant time and complexity, but may vary by Identity Provider implementation.
-
-### Data Broker
-
-The Data Broker service is responsible for background identity data retrieval and storage.  It is in the hot path for user authentication.  However, it does not directly handle user traffic and is not in-path for authorization decisions.
-
-The Data Broker service does not require significant resources, as it provides streaming updates of state changes to the Authorize service.  There will be utilization spikes when Authorize services are restarted and perform an initial synchronization.  Add resources if running many Authorize services and performing restarts in large batches.  In many deployments, 2 replicas of Data Broker is enough to provide resilient service.
-
-::: warning
-In a production configuration, Data Broker CPU/IO utilization also translates to IO load on the [underlying storage system](/docs/topics/data-storage.md).  Ensure it is scaled accordingly!
-:::
-
-## Load Balancing
-
-In any production deployment, running multiple replicas of each Pomerium service is strongly recommended. Each service has slightly different concerns about utilizing the replicas for HA and scaling, enumerated below.
-
-### Proxy
-
-You should provide a TCP or HTTP(s) load balancer between end users and the Proxy services.
-
-Proxy can handle its own SSL termination but is not a full web server. If you need any special capabilities such as redirects, sticky sessions, etc, it is typical to put Pomerium behind an L7 load balancer or ingress controller.
-
-### Authenticate
-
-You should provide a TCP or HTTP(s) load balancer between end users and the Authorize services.
-
-Authenticate is compatible with any L4 or L7/HTTP load balancer. Session stickiness should not be required and it is typical to have Authenticate be a named vhost on the same L7 load balancer as the Proxy service.
-
-### Authorize and Data Broker
-
-You do **not** need to provide a load balancer in front of Authorize and Data Broker services. Both utilize GRPC, and thus has special requirements if you should choose to use an external load balancer. GRPC can perform client based load balancing, and in most configurations is the best architecture.
-
-By default, Pomerium gRPC clients will automatically connect to all IPs returned by a DNS query for the name of an upstream service. They will then regularly re-query DNS for changes to the Authorize or Data Broker service cluster. Health checks and failover are automatic.
-
-**Many load balancers do not support HTTP2 yet. Please verify with your hardware, software or cloud provider**
-
-If you choose to use an external proxying load balancer instead of the default client implementation:
-
-- L7 mode requires HTTP2 support or Authorize requests will fail
-- L4 (TCP) mode, GRPC/HTTP2 traffic from a Proxy instance will be pinned to a single Authorize instance due to the way HTTP2 multiplexes requests over a single established connection
-- Due to the above limitations it is highly desirable to only use a load balancer which supports HTTP2 at Layer 7
-
-## High Availability
-
-As mentioned in [scaling](#scaling), Pomerium components themselves are stateless and support horizontal scale out for both availability and performance reasons.
-
-A given service type does not require communication with its peer instances to provide high availability. Eg, a proxy service does not communicate with other proxies.
-
-Regardless of the service mode, it is recommended you run 2+ instances of Pomerium with as much physical and logical separation as possible. For example, in Cloud environments, you should deploy instances of each service to at least 2 different zones. On-prem environments should deploy 2+ instances to independent hardware.
-
-Ensure that you have enough spare capacity to handle the scope of your failure domains.
-
-::: warning
-Multiple replicas of Data Broker or all-in-one service are only supported with [external storage](/docs/topics/data-storage.md) configured
-:::
-
-## SSL/TLS Certificates
-
-Pomerium utilizes TLS end to end, so the placement, certificate authorities and covered subjects are critical to align correctly.
-
-In a typical deployment, a minimum of two certs are required:
-
-- A wildcard certificate which covers the external `from` names, the Proxy service's external name and the Authenticate service's external name
-  - Must be trusted by user browsers or clients
-  - Must cover subject names from the user perspective
-- A certificate which covers the Authorize service's name
-  - Must be trusted by the Proxy
-  - Must cover the subject name from the Proxy's perspective
-
-If you have L7 load balancing in front of the Proxy/Authenticate:
-
-- Your wildcard certificate should live on the load balancer
-- Your Authenticate and Proxy services will need a certificate accepted by the load balancer
-- Your load balancer can be configured to verify the identity of the Authenticate and Proxy certificates
-
-If you have TLS enabled applications behind the proxy:
-
-- you may provide a client certificate for the Proxy service to connect to downstream services with and verify
-- the Proxy may be configured to verify the name and certificate authority of downstream services with either the standard Root CA bundle or a custom CA

docs/reference/readme.md

@@ -48,6 +48,16 @@ These configuration variables are shared by all services, in all service modes.
 Address specifies the host and port to serve HTTP requests from. If empty, `:443` is used. Note, in all-in-one deployments, gRPC traffic will be served on loopback on port `:5443`.
 
 
+### Authenticate Service URL
+- Environmental Variable: `AUTHENTICATE_SERVICE_URL`
+- Config File Key: `authenticate_service_url`
+- Type: `URL`
+- Required
+- Example: `https://authenticate.corp.example.com`
+
+Authenticate Service URL is the externally accessible URL for the authenticate service. In split service mode, this key is required by all services other than Databroker.
+
+
 ### Autocert
 - Environmental Variable: `AUTOCERT`
 - Config File Key: `autocert`
@@ -297,6 +307,16 @@ Setting this to false enables hostile javascript to steal session cookies and im
 Sets the lifetime of session cookies. After this interval, users must reauthenticate.
 
 
+### Data Broker Service URL
+- Environmental Variable: `DATABROKER_SERVICE_URL` or `DATABROKER_SERVICE_URLS`
+- Config File Key: `databroker_service_url` or `databroker_service_urls`
+- Type: `URL`
+- Example: `https://databroker.corp.example.com`
+- Default: in all-in-one mode, `http://localhost:5443`
+
+The data broker service URL points to a data broker which is responsible for storing associated authorization context (e.g. sessions, users and user groups). Multiple URLs can be specified with `databroker_service_urls`.
+
+
 ### Debug
 - Environmental Variable: `POMERIUM_DEBUG`
 - Config File Key: `pomerium_debug`
@@ -745,16 +765,6 @@ See also:
 - [Google - Setting Redirect URI](https://developers.google.com/identity/protocols/OpenIDConnect#setredirecturi)
 
 
-### Authenticate Service URL
-- Environmental Variable: `AUTHENTICATE_SERVICE_URL`
-- Config File Key: `authenticate_service_url`
-- Type: `URL`
-- Required
-- Example: `https://authenticate.corp.example.com`
-
-Authenticate Service URL is the externally accessible URL for the authenticate service.
-
-
 ### Authenticate Internal Service URL
 - Environmental Variable: `AUTHENTICATE_INTERNAL_SERVICE_URL`
 - Config File Key: `authenticate_internal_service_url`
@@ -877,25 +887,17 @@ Use it at your own risk, if you set a too low value, you may reach IDP API rate
 
 ## Proxy Service
 
-### Authenticate Service URL
-- Environmental Variable: `AUTHENTICATE_SERVICE_URL`
-- Config File Key: `authenticate_service_url`
-- Type: `URL`
-- Required
-- Example: `https://authenticate.corp.example.com`
-
-Authenticate Service URL is the externally accessible URL for the authenticate service.
-
-
 ### Authorize Service URL
-- Environmental Variable: `AUTHORIZE_SERVICE_URL`
+- Environmental Variable: `AUTHORIZE_SERVICE_URL or `AUTHORIZE_SERVICE_URLS`
-- Config File Key: `authorize_service_url`
+- Config File Key: `authorize_service_url` or `authorize_service_urls`
 - Type: `URL`
 - Required; inferred in all-in-one mode to be localhost.
-- Example: `https://pomerium-authorize-service.default.svc.cluster.local` or `https://localhost:5443`
+- Example: `https://pomerium-authorize-service.default.svc.cluster.local`, `https://localhost:5443`, `https://authorize.corp.example.com`
 
 Authorize Service URL is the location of the internally accessible Authorize service. NOTE: Unlike authenticate, authorize has no publicly accessible http handlers so this setting is purely for gRPC communication.
 
+Multiple URLs can be specified with `authorize_service_urls`.
+
 If your load balancer does not support gRPC pass-through you'll need to set this value to an internally routable location (`https://pomerium-authorize-service.default.svc.cluster.local`) instead of an externally routable one (`https://authorize.corp.example.com`).
 
 
@@ -1082,16 +1084,6 @@ More details on this problem are available in [Github Issue #2150](https://githu
 ## Data Broker Service
 The databroker service is used for storing user session data.
 
-
-### Data Broker Service URL
-- Environmental Variable: `DATABROKER_SERVICE_URL` or `DATABROKER_SERVICE_URLS`
-- Config File Key: `databroker_service_url` or `databroker_service_urls`
-- Type: `URL`
-- Example: `https://databroker.corp.example.com`
-- Default: in all-in-one mode, `http://localhost:5443`
-
-The data broker service URL points to a data broker which is responsible for storing associated authorization context (e.g. sessions, users and user groups). Multiple URLs can be specified with `databroker_service_urls`.
-
 By default, the `databroker` service uses an in-memory databroker.
 
 To create your own data broker, implement the following gRPC interface:
@@ -1805,16 +1797,6 @@ If set, enables proxying of websocket connections.
 
 ## Authorize Service
 
-### Authorize Service URL
-- Environmental Variable: `AUTHORIZE_SERVICE_URL` or `AUTHORIZE_SERVICE_URLS`
-- Config File Key: `authorize_service_url` or `authorize_service_urls`
-- Type: `URL`
-- Required
-- Example: `https://authorize.corp.example.com`
-
-Authorize Service URL is the location of the internally accessible Authorize service. Multiple URLs can be specified with `authorize_service_urls`.
-
-
 ### Google Cloud Serverless Authentication Service Account
 - Environmental Variable: `GOOGLE_CLOUD_SERVERLESS_AUTHENTICATION_SERVICE_ACCOUNT`
 - Config File Key: `google_cloud_serverless_authentication_service_account`

docs/reference/settings.yaml

@@ -65,6 +65,18 @@ settings:
           Address specifies the host and port to serve HTTP requests from. If empty, `:443` is used. Note, in all-in-one deployments, gRPC traffic will be served on loopback on port `:5443`.
         shortdoc: |
           Address specifies the host and port to serve HTTP requests from.
+      - name: "Authenticate Service URL"
+        keys: ["authenticate_service_url"]
+        attributes: |
+          - Environmental Variable: `AUTHENTICATE_SERVICE_URL`
+          - Config File Key: `authenticate_service_url`
+          - Type: `URL`
+          - Required
+          - Example: `https://authenticate.corp.example.com`
+        doc: |
+          Authenticate Service URL is the externally accessible URL for the authenticate service. In split service mode, this key is required by all services other than Databroker.
+        shortdoc: |
+          Authenticate Service URL is the externally accessible URL for the authenticate service.
       - name: "Autocert"
         keys: ["autocert"]
         attributes: |
@@ -347,6 +359,16 @@ settings:
               Sets the lifetime of session cookies. After this interval, users must reauthenticate.
             shortdoc: |
               Sets the lifetime of session cookies. After this interval, users must reauthenticate.
+      - name: "Data Broker Service URL"
+        keys: ["databroker_service_url"]
+        attributes: |
+          - Environmental Variable: `DATABROKER_SERVICE_URL` or `DATABROKER_SERVICE_URLS`
+          - Config File Key: `databroker_service_url` or `databroker_service_urls`
+          - Type: `URL`
+          - Example: `https://databroker.corp.example.com`
+          - Default: in all-in-one mode, `http://localhost:5443`
+        doc: |
+          The data broker service URL points to a data broker which is responsible for storing associated authorization context (e.g. sessions, users and user groups). Multiple URLs can be specified with `databroker_service_urls`.
       - name: "Debug"
         keys: ["pomerium_debug"]
         attributes: |
@@ -844,18 +866,6 @@ settings:
           - [Google - Setting Redirect URI](https://developers.google.com/identity/protocols/OpenIDConnect#setredirecturi)
         shortdoc: |
           The authenticate callback path is the path/url from the authenticate service that will receive the response from your identity provider.
-      - name: "Authenticate Service URL"
-        keys: ["authenticate_service_url"]
-        attributes: |
-          - Environmental Variable: `AUTHENTICATE_SERVICE_URL`
-          - Config File Key: `authenticate_service_url`
-          - Type: `URL`
-          - Required
-          - Example: `https://authenticate.corp.example.com`
-        doc: |
-          Authenticate Service URL is the externally accessible URL for the authenticate service.
-        shortdoc: |
-          Authenticate Service URL is the externally accessible URL for the authenticate service.
       - name: "Authenticate Internal Service URL"
         keys: ["authenticate_internal_service_url"]
         attributes: |
@@ -996,28 +1006,22 @@ settings:
           :::
   - name: "Proxy Service"
     settings:
-      - name: "Authenticate Service URL"
-        keys: ["authenticate_service_url"]
-        attributes: |
-          - Environmental Variable: `AUTHENTICATE_SERVICE_URL`
-          - Config File Key: `authenticate_service_url`
-          - Type: `URL`
-          - Required
-          - Example: `https://authenticate.corp.example.com`
-        doc: |
-          Authenticate Service URL is the externally accessible URL for the authenticate service.
       - name: "Authorize Service URL"
         keys: ["authorize_service_url"]
         attributes: |
-          - Environmental Variable: `AUTHORIZE_SERVICE_URL`
+          - Environmental Variable: `AUTHORIZE_SERVICE_URL or `AUTHORIZE_SERVICE_URLS`
-          - Config File Key: `authorize_service_url`
+          - Config File Key: `authorize_service_url` or `authorize_service_urls`
           - Type: `URL`
           - Required; inferred in all-in-one mode to be localhost.
-          - Example: `https://pomerium-authorize-service.default.svc.cluster.local` or `https://localhost:5443`
+          - Example: `https://pomerium-authorize-service.default.svc.cluster.local`, `https://localhost:5443`, `https://authorize.corp.example.com`
         doc: |
           Authorize Service URL is the location of the internally accessible Authorize service. NOTE: Unlike authenticate, authorize has no publicly accessible http handlers so this setting is purely for gRPC communication.
 
+          Multiple URLs can be specified with `authorize_service_urls`.
+
           If your load balancer does not support gRPC pass-through you'll need to set this value to an internally routable location (`https://pomerium-authorize-service.default.svc.cluster.local`) instead of an externally routable one (`https://authorize.corp.example.com`).
+        shortdoc: |
+          Authorize Service URL is the location of the internally accessible Authorize service.
       - name: "Authorize Internal Service URL"
         keys: ["authorize_internal_service_url"]
         attributes: |
@@ -1213,27 +1217,17 @@ settings:
   - name: "Data Broker Service"
     doc: |
       The databroker service is used for storing user session data.
+
+      By default, the `databroker` service uses an in-memory databroker.
+
+      To create your own data broker, implement the following gRPC interface:
+
+      - [pkg/grpc/databroker/databroker.proto](https://github.com/pomerium/pomerium/blob/main/pkg/grpc/databroker/databroker.proto)
+
+      For an example implementation, the in-memory database used by the databroker service can be found here:
+
+      - [pkg/databroker/memory](https://github.com/pomerium/pomerium/tree/main/pkg/databroker/memory)
     settings:
-      - name: "Data Broker Service URL"
-        keys: ["databroker_service_url"]
-        attributes: |
-          - Environmental Variable: `DATABROKER_SERVICE_URL` or `DATABROKER_SERVICE_URLS`
-          - Config File Key: `databroker_service_url` or `databroker_service_urls`
-          - Type: `URL`
-          - Example: `https://databroker.corp.example.com`
-          - Default: in all-in-one mode, `http://localhost:5443`
-        doc: |
-          The data broker service URL points to a data broker which is responsible for storing associated authorization context (e.g. sessions, users and user groups). Multiple URLs can be specified with `databroker_service_urls`.
-
-          By default, the `databroker` service uses an in-memory databroker.
-
-          To create your own data broker, implement the following gRPC interface:
-
-          - [pkg/grpc/databroker/databroker.proto](https://github.com/pomerium/pomerium/blob/main/pkg/grpc/databroker/databroker.proto)
-
-          For an example implementation, the in-memory database used by the databroker service can be found here:
-
-          - [pkg/databroker/memory](https://github.com/pomerium/pomerium/tree/main/pkg/databroker/memory)
       - name: "Data Broker Internal Service URL"
         keys: ["databroker_internal_service_url"]
         attributes: |
@@ -1958,18 +1952,6 @@ settings:
           :::
   - name: "Authorize Service"
     settings:
-      - name: "Authorize Service URL"
-        keys: ["authorize_service_url"]
-        attributes: |
-          - Environmental Variable: `AUTHORIZE_SERVICE_URL` or `AUTHORIZE_SERVICE_URLS`
-          - Config File Key: `authorize_service_url` or `authorize_service_urls`
-          - Type: `URL`
-          - Required
-          - Example: `https://authorize.corp.example.com`
-        doc: |
-          Authorize Service URL is the location of the internally accessible Authorize service. Multiple URLs can be specified with `authorize_service_urls`.
-        shortdoc: |
-          Authorize Service URL is the location of the internally accessible Authorize service.
       - name: "Google Cloud Serverless Authentication Service Account"
         keys: ["google_cloud_serverless_authentication_service_account"]
         attributes: |