diff --git a/docs/.vuepress/config.js b/docs/.vuepress/config.js index 5442945f2..b56de69d4 100644 --- a/docs/.vuepress/config.js +++ b/docs/.vuepress/config.js @@ -122,6 +122,7 @@ module.exports = { "topics/kubernetes-integration", "topics/production-deployment", "topics/programmatic-access", + "topics/tcp-support", "topics/impersonation", ], }, diff --git a/docs/docs/topics/tcp-support.md b/docs/docs/topics/tcp-support.md new file mode 100644 index 000000000..86d95a902 --- /dev/null +++ b/docs/docs/topics/tcp-support.md @@ -0,0 +1,106 @@ +--- +title: TCP Support +description: >- + This article describes how to leverage pomerium for TCP proxying +meta: + - name: keywords + content: pomerium pomerium-cli proxy identity-access-proxy ssh tcp postgres database redis mysql application non-http +--- + +# TCP Support + +## About + +Operations and engineering teams frequently require access to lower level administrative and data protocols such as SSH, RDP, Postgres, MySQL, Redis, etc. + +In addition to managing HTTP based applications, Pomerium can be used to protect non-HTTP systems with the same consistent authorization policy. This is achieved by tunneling TCP over HTTP with the help of a client side command built into [`pomerium-cli`](/docs/installation.md#pomerium-cli). + + +Internally, Pomerium uses the [`CONNECT` method](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT) to establish the TCP tunnel. + +::: warning +To minimize issues with TCP support, Pomerium should not be placed behind another HTTP proxy. Instead, configure your load balancer in L4 or TCP mode. + +Otherwise, the HTTP proxy in front of Pomerium must know how to properly handle the `CONNECT` command and proxy it upstream. This capability will be specific to each proxy implementation. +::: + +## Configuring + +TCP configuration is simple. Just specify the correct scheme and ports in your policy [`to`](/reference/#to) and [`from`](/reference/#from) fields. + +Example: +```yaml +policy: + - from: tcp+https://redis.corp.example.com:6379 + to: tcp://redis.internal.example.com:6379 + allowed_users: + - contractor@not-example.com + allowed_groups: + - datascience@example.com +``` + +Notes: + +* When configuring a TCP route, any HTTP specific settings such as `regex_rewrite_pattern`, or `set_request_headers` have no effect. +* While data is encrypted from a user system to Pomerium's proxy, the underlying application protocol must also support encryption for data to be fully encrypted end-to-end. +* The ports in `from` and `to` are independent. Users only need to know the `from` URL to connect. The `to` can be changed without end user participation. + +## Using + +While HTTP routes can be consumed with just a normal browser, `pomerium-cli` must serve as a proxy for TCP routes. It is [available](/docs/installation.md#pomerium-cli) for a variety of platforms in various formats. + +To connect, you normally need just the external hostname and port of your TCP route: + +```bash +% pomerium-cli tcp redis.corp.example.com:6379 +5:57PM INF tcptunnel: listening on 127.0.0.1:52046 +``` + +By default, `pomerium-cli` will start a listener on loopback on a random port. + +On first connection, you will be sent through a standard Pomerium HTTP authentication flow. After completing this, your TCP connection should be established! + +```bash +% redis-cli -h localhost -p 52046 +localhost:52046> keys * +(empty array) +localhost:52046> +``` + +## Advanced Usage + +### Listen Configuration + +You may specify an optional address and port for the `tcp` command to listen on. + +`-` specifies that STDIN and STDOUT should be directly attached to the remote TCP connection. This is useful for [SSH](#ssh-helper-configuration) or for sending data through a shell pipe. + +### SSH Helper Configuration + +A sample SSH ProxyCommand configuration in `ssh_config`: + +``` +Host *.corp.example.com + ProxyCommand ~/bin/pomerium-cli tcp --listen - %h:%p +``` + +A sample SSH command using `pomerium-cli` as a ProxyCommand: + +```bash +ssh -o ProxyCommand='pomerium-cli tcp --listen - %h:%p' ssh.localhost.pomerium.io +``` + +More information on SSH ProxyCommand: + +- [https://man.openbsd.org/ssh_config.5#ProxyCommand](https://man.openbsd.org/ssh_config.5#ProxyCommand) +- [https://www.redhat.com/sysadmin/ssh-proxy-bastion-proxyjump](https://www.redhat.com/sysadmin/ssh-proxy-bastion-proxyjump) + +### Custom URL + +If the Pomerium proxy is not reachable through port `443` or the route is not in external DNS, a custom URL may be specified: + +```bash +% pomerium-cli tcp --pomerium-url https://pomerium.corp.example.com:8443 redis.corp.example.com:6379 +``` + +The above command connects to `https://pomerium.corp.example.com:8443` and then requests the TCP route named `redis.corp.example.com:6379`. diff --git a/docs/reference/readme.md b/docs/reference/readme.md index 40b8ade00..4dd9dfd33 100644 --- a/docs/reference/readme.md +++ b/docs/reference/readme.md @@ -503,10 +503,9 @@ Expose a prometheus endpoint on the specified port. | pomerium_config_last_reload_success | Gauge | Whether the last configuration reload succeeded by service | | pomerium_config_last_reload_success_timestamp | Gauge | The timestamp of the last successful configuration reload by service | | redis_conns | Gauge | Number of total connections in the pool | -| redis_idle_conns | Gauge | Number of idle connections in the pool | -| redis_stale_conns | Gauge | Number of stale connections in the pool | -| redis_miss_count_total | Counter | Total number of times a connection was NOT found in the pool | -| redis_hit_count_total | Counter | Total number of times a connection was found in the pool | +| redis_idle_conns | Gauge | Total number of times free connection was found in the pool | +| redis_wait_count_total | Counter | Total number of connections waited for | +| redis_wait_duration_ms_total | Counter | Total time spent waiting for connections | | storage_operation_duration_ms | Histogram | Storage operation duration by operation, result, backend and service | #### Envoy Proxy Metrics @@ -556,10 +555,10 @@ Each unit work is called a Span in a trace. Spans include metadata about the wor #### Shared Tracing Settings -Config Key | Description | Required -:------------------ | :----------------------------------------------------------------------------------- | -------- -tracing_provider | The name of the tracing provider. (e.g. jaeger, zipkin) | ✅ -tracing_sample_rate | Percentage of requests to sample in decimal notation. Default is `0.0001`, or `.01%` | ❌ +| Config Key | Description | Required | +| :------------------ | :----------------------------------------------------------------------------------- | -------- | +| tracing_provider | The name of the tracing provider. (e.g. jaeger, zipkin) | ✅ | +| tracing_sample_rate | Percentage of requests to sample in decimal notation. Default is `0.0001`, or `.01%` | ❌ | #### Jaeger (partial) @@ -573,10 +572,10 @@ tracing_sample_rate | Percentage of requests to sample in decimal notation. Defa - Service dependency analysis - Performance / latency optimization -Config Key | Description | Required -:-------------------------------- | :------------------------------------------ | -------- -tracing_jaeger_collector_endpoint | Url to the Jaeger HTTP Thrift collector. | ✅ -tracing_jaeger_agent_endpoint | Send spans to jaeger-agent at this address. | ✅ +| Config Key | Description | Required | +| :-------------------------------- | :------------------------------------------ | -------- | +| tracing_jaeger_collector_endpoint | Url to the Jaeger HTTP Thrift collector. | ✅ | +| tracing_jaeger_agent_endpoint | Send spans to jaeger-agent at this address. | ✅ | #### Zipkin @@ -584,9 +583,9 @@ Zipkin is an open source distributed tracing system and protocol. Many tracing backends support zipkin either directly or through intermediary agents, including Jaeger. For full tracing support, we recommend using the Zipkin tracing protocol. -Config Key | Description | Required -:---------------------- | :------------------------------- | -------- -tracing_zipkin_endpoint | Url to the Zipkin HTTP endpoint. | ✅ +| Config Key | Description | Required | +| :---------------------- | :------------------------------- | -------- | +| tracing_zipkin_endpoint | Url to the Zipkin HTTP endpoint. | ✅ | #### Example @@ -1043,10 +1042,13 @@ Requires setting [Google Cloud Serverless Authentication Service Account](./#goo ### From - `yaml`/`json` setting: `from` - Type: `URL` (must contain a scheme and hostname, must not contain a path) +- Schemes: `https`, `tcp+https` - Required -- Example: `https://verify.corp.example.com` +- Example: `https://verify.corp.example.com`, `tcp+https://ssh.corp.example.com:22` -`From` is the externally accessible source of the proxied request. +`From` is the externally accessible URL for the proxied request. + +Specifying `tcp+https` for the scheme enables [TCP proxying](../docs/topics/tcp-support.md) support for the route. You may map more than one port through the same hostname by specifying a different `:port` in the URL. ### Kubernetes Service Account Token @@ -1233,11 +1235,14 @@ Remove Request Headers allows you to remove given request headers. This can be u ### To - `yaml`/`json` setting: `to` - Type: `URL` (must contain a scheme and hostname) +- Schemes: `http`, `https`, `tcp` - Required - Example: `http://verify` , `https://192.1.20.12:8080`, `http://neverssl.com`, `https://verify.org/anything/` `To` is the destination of a proxied request. It can be an internal resource, or an external resource. +Must be `tcp` if `from` is `tcp+https`. + :::warning Be careful with trailing slash. diff --git a/docs/reference/settings.yaml b/docs/reference/settings.yaml index 41b9c74ad..a36df6c8f 100644 --- a/docs/reference/settings.yaml +++ b/docs/reference/settings.yaml @@ -1159,10 +1159,13 @@ settings: attributes: | - `yaml`/`json` setting: `from` - Type: `URL` (must contain a scheme and hostname, must not contain a path) + - Schemes: `https`, `tcp+https` - Required - - Example: `https://verify.corp.example.com` + - Example: `https://verify.corp.example.com`, `tcp+https://ssh.corp.example.com:22` doc: | - `From` is the externally accessible source of the proxied request. + `From` is the externally accessible URL for the proxied request. + + Specifying `tcp+https` for the scheme enables [TCP proxying](../docs/topics/tcp-support.md) support for the route. You may map more than one port through the same hostname by specifying a different `:port` in the URL. - name: "Kubernetes Service Account Token" keys: [ @@ -1359,11 +1362,14 @@ settings: attributes: | - `yaml`/`json` setting: `to` - Type: `URL` (must contain a scheme and hostname) + - Schemes: `http`, `https`, `tcp` - Required - Example: `http://verify` , `https://192.1.20.12:8080`, `http://neverssl.com`, `https://verify.pomerium.com/anything/` doc: | `To` is the destination of a proxied request. It can be an internal resource, or an external resource. + Must be `tcp` if `from` is `tcp+https`. + :::warning Be careful with trailing slash. diff --git a/scripts/generate-settings-docs.py b/scripts/generate-settings-docs.py index 21bfa1d09..8d99beabe 100755 --- a/scripts/generate-settings-docs.py +++ b/scripts/generate-settings-docs.py @@ -1,4 +1,4 @@ -#!/bin/env python3 +#!/usr/bin/env python3 import os.path from typing import Any, IO import yaml