mirror of
https://github.com/pomerium/pomerium.git
synced 2025-07-25 20:49:30 +02:00
init Ingress Controller doc
Co-authored-by: travisgroth <tgroth@pomerium.com> Co-authored-by: wasaga <dmishin@pomerium.com>
This commit is contained in:
alexfornuto
parent
f59bf0e2dc
commit
636ac2cfcb
1 changed files with 264 additions and 3 deletions
|
|
@ -8,12 +8,273 @@ meta:
|
|||
|
||||
# Kubernetes Ingress Controller
|
||||
|
||||
A Sentence or two on what the ingress controller is, and why it's a good idea to use it compared to other solutions.
|
||||
Use Pomerium as a first class secure-by-default Ingress Controller. Dynamically provision routes from Ingress resources and set policy based on annotations.
|
||||
|
||||
TODO: Funfact: you can dynamically create and remove routes with OSS Pomerium using the Ingress Controller, which you can't do otherwise.
|
||||
|
||||
## Installation
|
||||
|
||||
This may only need to reference the existing helm instructions, updated to use the controller already.
|
||||
## Prerequisites
|
||||
|
||||
TODO: Cert manager. This is covered by the [helm] instructions, but to do make one independently...
|
||||
|
||||
TODO: REDIS Backend with persistence is highly recommended.
|
||||
|
||||
TODO: CloudRun endpoints can be easily supported using "internal traffic policy", if they are deployed to the same cloud project as Pomerium.
|
||||
|
||||
### Helm
|
||||
Our instructions for [Installing Pomering Using Helm](/docs/k8s/helm.md) includes the Ingress Controller as part of the documented configuration. The values to adjust based on your configuration usually include:
|
||||
|
||||
```yaml
|
||||
ingressController:
|
||||
enabled: true
|
||||
```
|
||||
### Docker Image
|
||||
|
||||
You may deploy your own manifests by using the `pomerium/ingress-controller` docker image.
|
||||
|
||||
### System Requirements
|
||||
|
||||
- Kubernetes v0.19.0+
|
||||
- Pomerium [Helm Chart](https://github.com/pomerium/pomerium-helm/tree/master/charts/pomerium) v25.0.0+
|
||||
|
||||
### Limitations
|
||||
|
||||
::: warning
|
||||
Only one Ingress Controller instance/replica is supported per Pomerium cluster.
|
||||
:::
|
||||
|
||||
## Configuration
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Flag | Description |
|
||||
| ------------------------------ | -------------------------------------------------------------------- |
|
||||
| `--databroker-service-url` | the databroker service url
|
||||
| `--databroker-tls-ca` | base64 encoded tls CA
|
||||
| `--databroker-tls-ca-file` | tls CA file path for the databroker connection connection
|
||||
| `--health-probe-bind-address` | The address the probe endpoint binds to. (default ":8081")
|
||||
| `--metrics-bind-address` | The address the metric endpoint binds to. (default ":8080")
|
||||
| `--name` | IngressClass controller name (default "pomerium.io/ingress-controller")
|
||||
| `--namespaces` | namespaces to watch, or none to watch all namespaces
|
||||
| `--prefix` | Ingress annotation prefix (default "ingress.pomerium.io")
|
||||
| `--shared-secret` | base64-encoded shared secret for communicating with databroker
|
||||
| `--update-status-from-service` | update ingress status from given service status (pomerium-proxy)|
|
||||
|
||||
The helm chart exposes a subset of these flags for appropriate customization.
|
||||
|
||||
## Usage
|
||||
|
||||
### Defining Routes
|
||||
|
||||
If you've tested Pomerium using the all-in-one service, you're probably familiar with configuring routes in Pomerium's `config.yaml`. In this environmenzt, each route is defined as a.... what @travis?
|
||||
|
||||
The Ingress Controller will monitor Ingress resources in the cluster, creating a Pomerium route definition for each one. Policy and other configuration options for the route are set by using annotations starting with `ingress.pomerium.io/`.
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
annotations:
|
||||
ingress.pomerium.io/policy: '[{"allow":{"and":[{"email":{"is":"user@yourdomain.com"}}]}}]' # This can also be a yaml block quote
|
||||
spec:
|
||||
rules:
|
||||
- host: hello.localhost.pomerium.io
|
||||
http:
|
||||
paths:
|
||||
- backend:
|
||||
service:
|
||||
name: nginx-hello
|
||||
port:
|
||||
name: http
|
||||
path: /
|
||||
pathType: Prefix
|
||||
```
|
||||
Becomes:
|
||||
|
||||
```yaml
|
||||
routes:
|
||||
- from: https://hello.localhost.pomerium.io
|
||||
to: http://nginx-hello.default.svc.cluster.local
|
||||
policy:
|
||||
- allow:
|
||||
and:
|
||||
- email:
|
||||
is: user@yourdomain.com
|
||||
```
|
||||
|
||||
|
||||
### Supported Annotations
|
||||
|
||||
| Annotation | Description |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------- |
|
||||
| [`cors_allow_preflight`] | [See Reference for details.][`cors_allow_preflight`]
|
||||
| [`allow_public_unauthenticated_access`] | [See Reference for details.][`allow_public_unauthenticated_access`]
|
||||
| [`allow_any_authenticated_user`] | [See Reference for details.][`allow_any_authenticated_user`]
|
||||
| [`timeout`] | [See Reference for details.][`timeout`]
|
||||
| [`idle_timeout`] | [See Reference for details.][`idle_timeout`]
|
||||
| [`allow_websockets`] | [See Reference for details.][`allow_websockets`]
|
||||
| [`set_request_headers`] | [See Reference for details.][`set_request_headers`]
|
||||
| [`remove_request_headers`] | [See Reference for details.][`remove_request_headers`]
|
||||
| [`set_response_headers`] | [See Reference for details.][`set_response_headers`]
|
||||
| [`rewrite_response_headers`] | [See Reference for details.][`rewrite_response_headers`]
|
||||
| [`preserve_host_header`] | [See Reference for details.][`preserve_host_header`]
|
||||
| [`pass_identity_headers`] | [See Reference for details.][`pass_identity_headers`]
|
||||
| [`tls_skip_verify`] | [See Reference for details.][`tls_skip_verify`]
|
||||
| [`tls_server_name`] | [See Reference for details.][`tls_server_name`]
|
||||
| [`allowed_users`] | [See Reference for details.][`allowed_users`]
|
||||
| [`allowed_groups`] | [See Reference for details.][`allowed_groups`]
|
||||
| [`allowed_domains`] | [See Reference for details.][`allowed_domains`]
|
||||
| [`allowed_idp_claims`] | [See Reference for details.][`allowed_idp_claims`]
|
||||
| [`policy`] | [See Reference for details.][`policy`]
|
||||
| [`health_checks`] | [See Reference for details.][`health_checks`]
|
||||
| [`outlier_detection`] | [See Reference for details.][`outlier_detection`]
|
||||
| [`lb_config`] | [See Reference for details.][`lb_config`]
|
||||
| [`tls_custom_ca_secret`] | Name of Kubernetes `tls` Secret containing a custom [CA certificate](https://www.pomerium.com/reference/#tls-custom-certificate-authority) for the upstream
|
||||
| [`tls_client_secret`] | Name of Kubernetes `tls` Secret containing a [client certificate](https://www.pomerium.com/reference/#tls-client-certificate) for connecting to the upstream
|
||||
| [`tls_downstream_client_ca_secret`] | Name of Kubernetes `tls` Secret containing a [Client CA](https://www.pomerium.com/reference/#tls-downstream-client-certificate-authority) for validating downstream clients
|
||||
| [`secure_upstream`] | When set to true, use `https` when connecting to the upstream endpoint.
|
||||
|
||||
::: tip
|
||||
Every value for the annotations above must be in `string` format.
|
||||
:::
|
||||
|
||||
### Cert Manager Integration
|
||||
|
||||
TODO: @travisgroth
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
|
||||
### View Event History
|
||||
|
||||
Pomerium Ingress Controller will add **events** to the ingress objects as it processes them.
|
||||
|
||||
```
|
||||
kubectl describe ingress/my-ingress
|
||||
```
|
||||
|
||||
```log
|
||||
Events:
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Normal Updated 18s pomerium-ingress updated pomerium configuration
|
||||
```
|
||||
|
||||
If an error occurs, it may be reflected in the events:
|
||||
|
||||
```log
|
||||
Events:
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Normal Updated 5m53s pomerium-ingress updated pomerium configuration
|
||||
Warning UpdateError 3s pomerium-ingress upsert routes: parsing ingress: annotations: applying policy annotations: parsing policy: invalid rules in policy: unsupported conditional "maybe", only and, or, not, nor and action are allowed
|
||||
```
|
||||
|
||||
### HSTS
|
||||
|
||||
If your domain has HSTS enabled, and you visit i.e. _authenticate_ endpoint while Pomerium is using self-signed bootstrap certificate,
|
||||
or i.e. LetsEncrypt staging certificate, before cert-manager provisioned a production certificate, it may get pinned in your browser and need be reset.
|
||||
|
||||
https://www.ssl2buy.com/wiki/how-to-clear-hsts-settings-on-chrome-firefox-and-ie-browsers
|
||||
|
||||
## HTTPS endpoints
|
||||
|
||||
`Ingress` spec defines that all communications to the service should happen in cleartext. Pomerium supports HTTPS endpoints, including mTLS.
|
||||
|
||||
Annotate your `Ingress` with
|
||||
|
||||
```yaml
|
||||
ingress.pomerium.io/secure_upstream: true
|
||||
```
|
||||
|
||||
Additional TLS may be supplied by creating a Kubernetes secret(s) in the same namespaces as `Ingress` resource. Note we do not support file paths or embedded secret references.
|
||||
|
||||
- [`tls_client_secret`](https://pomerium.io/reference/#tls-client-certificate)
|
||||
- [`tls_custom_ca_secret`](https://pomerium.io/reference/#tls-custom-certificate-authority)
|
||||
- [`tls_downstream_client_ca_secret`](https://pomerium.io/reference/#tls-downstream-client-certificate-authority)
|
||||
|
||||
Note the referenced `tls_client_secret` must be a [TLS Kubernetes secret](https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets). `tls_custom_ca_secret` and `tls_downstream_client_ca_secret` must contain `ca.crt` containing a .PEM encoded (Base64-encoded DER format) public certificate.
|
||||
|
||||
|
||||
### External services
|
||||
|
||||
You may refer to external services by defining a `Service` with `externalName`.
|
||||
|
||||
I.e. if you have `https://my-existing-service.corp.com`:
|
||||
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: external
|
||||
spec:
|
||||
type: ExternalName
|
||||
externalName: "my-existing-service.corp.com"
|
||||
ports:
|
||||
- protocol: TCP
|
||||
name: https
|
||||
port: 443
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: external
|
||||
annotations:
|
||||
cert-manager.io/cluster-issuer: letsencrypt-prod-http
|
||||
ingress.pomerium.io/secure_upstream: "true"
|
||||
ingress.pomerium.io/policy: |
|
||||
- allow:
|
||||
and:
|
||||
- domain:
|
||||
is: pomerium.com
|
||||
spec:
|
||||
ingressClassName: pomerium
|
||||
tls:
|
||||
- hosts:
|
||||
- "external.localhost.pomerium.io"
|
||||
secretName: external-localhost-pomerium.io
|
||||
rules:
|
||||
- host: "external.localhost.pomerium.io"
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: external
|
||||
port:
|
||||
name: https
|
||||
```
|
||||
|
||||
|
||||
[`cors_allow_preflight`]: /reference/#cors-allow-preflight
|
||||
[`allow_public_unauthenticated_access`]: /reference/#allow-public-unauthenticated-access
|
||||
[`allow_any_authenticated_user`]: /reference/#allow_any_authenticated_user
|
||||
[`timeout`]: /reference/#timeout
|
||||
[`idle_timeout`]: /reference/#idle-timeout
|
||||
[`allow_websockets`]: /reference/#allow-websockets
|
||||
[`set_request_headers`]: /reference/#set-request-headers
|
||||
[`remove_request_headers`]: /reference/#remove-request-headers
|
||||
[`set_response_headers`]: /reference/#set-response-headers
|
||||
[`rewrite_response_headers`]: /reference/#rewrite-response-headers
|
||||
[`preserve_host_header`]: /reference/#preserve-host-header
|
||||
[`pass_identity_headers`]: /reference/#pass-identity-headers
|
||||
[`tls_skip_verify`]: /reference/#tls-skip-verify
|
||||
[`tls_server_name`]: /reference/#tls-server-name
|
||||
[`allowed_users`]: /reference/#allowed-users
|
||||
[`allowed_groups`]: /reference/#allowed-groups
|
||||
[`allowed_domains`]: /reference/#allowed-domains
|
||||
[`allowed_idp_claims`]: /reference/#allowed-idp-claims
|
||||
[`policy`]: /reference/#policy
|
||||
[`health_checks`]: /reference/#health-checks
|
||||
[`outlier_detection`]: /reference/#outlier-detection
|
||||
[`lb_config`]: /reference/#lb-config
|
||||
[`tls_custom_ca_secret`]: /reference/#tls-custom-ca-secret
|
||||
[`tls_client_secret`]: /reference/#tls-client-secret
|
||||
[`tls_downstream_client_ca_secret`]: /reference/#tls-downstream-client-ca-secret
|
||||
[`secure_upstream`]: /reference/#secure-upstream
|
||||
[`tls_custom_ca_secret`]: /reference/#tls-custom-certificate-authority
|
||||
[`tls_client_secret`]: /reference/#tls-client-certificate
|
||||
Loading…
Add table
Add a link
Reference in a new issue