## Summary
Adds `mcp_tool` PPL criterion, that matches MCP tool names like
```yaml
- from: https://db.localhost.pomerium.io
to: http://localhost:3000/mcp
policy:
allow:
and:
- email:
in: ["user@pomerium.com"]
- mcp_tool:
in: ["list_tables", "read_table", "search_records"]
mcp: {}
```
## Related issues
Fix
https://linear.app/pomerium/issue/ENG-2393/mcp-authorize-each-incoming-request-to-an-mcp-route
## User Explanation
<!-- How would you explain this change to the user? If this
change doesn't create any user-facing changes, you can leave
this blank. If filled out, add the `docs` label -->
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
In order to inspect the MCP requests and use the request payload in the
authorization decisions,
configure `ext_authz` to send the request payload as well.
the body then would be available for inspection as it would contain the
json-rpc message like
```json
{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"_meta":{"progressToken":1},"name":"list_tables","arguments":{}}}
```
## Related issues
Ref:
https://linear.app/pomerium/issue/ENG-2393/mcp-authorize-each-incoming-request-to-an-mcp-route
## User Explanation
<!-- How would you explain this change to the user? If this
change doesn't create any user-facing changes, you can leave
this blank. If filled out, add the `docs` label -->
## Checklist
- [ ] reference any related issues
- [ ] updated unit tests
- [ ] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [ ] ready for review
## Summary
Backport `telemetry.Component` that provides centralized tracing,
logging and metrics for operations.
## Related issues
<!-- For example...
- #159
-->
## User Explanation
<!-- How would you explain this change to the user? If this
change doesn't create any user-facing changes, you can leave
this blank. If filled out, add the `docs` label -->
## Checklist
- [ ] reference any related issues
- [ ] updated unit tests
- [ ] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [ ] ready for review
---------
Co-authored-by: Kenneth Jenkins <51246568+kenjenkins@users.noreply.github.com>
## Summary
Add a new `circuit_breaker_thresholds` option:
```yaml
circuit_breaker_thresholds:
max_connections: 1
max_pending_requests: 2
max_requests: 3
max_retries: 4
max_connection_pools: 5
```
This option can be set at the global level or at the route level. Each
threshold is optional and when not set a default will be used. For
internal clusters we will disable the circuit breaker. For normal routes
we will use the envoy defaults.
## Related issues
-
[ENG-2310](https://linear.app/pomerium/issue/ENG-2310/add-circuit-breaker-settings-per-route)
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
Adds `in` string matcher to PPL, that matches a string with an array of
strings.
## Related issues
Related: https://linear.app/pomerium/issue/ENG-2393
## User Explanation
<!-- How would you explain this change to the user? If this
change doesn't create any user-facing changes, you can leave
this blank. If filled out, add the `docs` label -->
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
Enables metrics for the `pgxpool` that is used by the PostgreSQL
databroker backend.
Metrics are updated at 1s interval.
Will add the following metric output in the regular Prometheus
`/metrics` endpoint:
```
# HELP pomerium_pgxpool_acquire_duration_nanoseconds_total Total duration of all successful acquires from the pool in nanoseconds.
# TYPE pomerium_pgxpool_acquire_duration_nanoseconds_total counter
pomerium_pgxpool_acquire_duration_nanoseconds_total{db_client_connection_pool_name="localhost:5432/pomerium",db_system="postgresql",otel_scope_name="github.com/exaring/otelpgx",otel_scope_version="v0.9.1",hostname="MacBookPro"} 5.1058702e+07
# HELP pomerium_pgxpool_acquired_connections Number of currently acquired connections in the pool.
# TYPE pomerium_pgxpool_acquired_connections gauge
pomerium_pgxpool_acquired_connections{db_client_connection_pool_name="localhost:5432/pomerium",db_system="postgresql",otel_scope_name="github.com/exaring/otelpgx",otel_scope_version="v0.9.1",hostname="MacBookPro"} 0
# HELP pomerium_pgxpool_acquires_total Cumulative count of successful acquires from the pool.
# TYPE pomerium_pgxpool_acquires_total counter
pomerium_pgxpool_acquires_total{db_client_connection_pool_name="localhost:5432/pomerium",db_system="postgresql",otel_scope_name="github.com/exaring/otelpgx",otel_scope_version="v0.9.1",hostname="MacBookPro"} 91
# HELP pomerium_pgxpool_canceled_acquires_total Cumulative count of acquires from the pool that were canceled by a context.
# TYPE pomerium_pgxpool_canceled_acquires_total counter
pomerium_pgxpool_canceled_acquires_total{db_client_connection_pool_name="localhost:5432/pomerium",db_system="postgresql",otel_scope_name="github.com/exaring/otelpgx",otel_scope_version="v0.9.1",hostname="MacBookPro"} 0
# HELP pomerium_pgxpool_constructing_connections_milliseconds Number of connections with construction in progress in the pool.
# TYPE pomerium_pgxpool_constructing_connections_milliseconds gauge
pomerium_pgxpool_constructing_connections_milliseconds{db_client_connection_pool_name="localhost:5432/pomerium",db_system="postgresql",otel_scope_name="github.com/exaring/otelpgx",otel_scope_version="v0.9.1",hostname="MacBookPro"} 0
# HELP pomerium_pgxpool_empty_acquire_total Cumulative count of successful acquires from the pool that waited for a resource to be released or constructed because the pool was empty.
# TYPE pomerium_pgxpool_empty_acquire_total counter
pomerium_pgxpool_empty_acquire_total{db_client_connection_pool_name="localhost:5432/pomerium",db_system="postgresql",otel_scope_name="github.com/exaring/otelpgx",otel_scope_version="v0.9.1",hostname="MacBookPro"} 6
# HELP pomerium_pgxpool_idle_connections Number of currently idle connections in the pool.
# TYPE pomerium_pgxpool_idle_connections gauge
pomerium_pgxpool_idle_connections{db_client_connection_pool_name="localhost:5432/pomerium",db_system="postgresql",otel_scope_name="github.com/exaring/otelpgx",otel_scope_version="v0.9.1",hostname="MacBookPro"} 5
# HELP pomerium_pgxpool_max_connections Maximum size of the pool.
# TYPE pomerium_pgxpool_max_connections gauge
pomerium_pgxpool_max_connections{db_client_connection_pool_name="localhost:5432/pomerium",db_system="postgresql",otel_scope_name="github.com/exaring/otelpgx",otel_scope_version="v0.9.1",hostname="MacBookPro"} 10
# HELP pomerium_pgxpool_max_idle_destroys_total Cumulative count of connections destroyed because they exceeded MaxConnectionsIdleTime.
# TYPE pomerium_pgxpool_max_idle_destroys_total counter
pomerium_pgxpool_max_idle_destroys_total{db_client_connection_pool_name="localhost:5432/pomerium",db_system="postgresql",otel_scope_name="github.com/exaring/otelpgx",otel_scope_version="v0.9.1",hostname="MacBookPro"} 0
# HELP pomerium_pgxpool_max_lifetime_destroys_total Cumulative count of connections destroyed because they exceeded MaxConnectionsLifetime.
# TYPE pomerium_pgxpool_max_lifetime_destroys_total counter
pomerium_pgxpool_max_lifetime_destroys_total{db_client_connection_pool_name="localhost:5432/pomerium",db_system="postgresql",otel_scope_name="github.com/exaring/otelpgx",otel_scope_version="v0.9.1",hostname="MacBookPro"} 0
# HELP pomerium_pgxpool_new_connections_total Cumulative count of new connections opened.
# TYPE pomerium_pgxpool_new_connections_total counter
pomerium_pgxpool_new_connections_total{db_client_connection_pool_name="localhost:5432/pomerium",db_system="postgresql",otel_scope_name="github.com/exaring/otelpgx",otel_scope_version="v0.9.1",hostname="MacBookPro"} 6
# HELP pomerium_pgxpool_total_connections Total number of resources currently in the pool. The value is the sum of ConstructingConnections, AcquiredConnections, and IdleConnections.
# TYPE pomerium_pgxpool_total_connections gauge
pomerium_pgxpool_total_connections{db_client_connection_pool_name="localhost:5432/pomerium",db_system="postgresql",otel_scope_name="github.com/exaring/otelpgx",otel_scope_version="v0.9.1",hostname="MacBookPro"} 5
```
## Related issues
<!-- For example...
- #159
-->
## User Explanation
<!-- How would you explain this change to the user? If this
change doesn't create any user-facing changes, you can leave
this blank. If filled out, add the `docs` label -->
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
Fixes to MCP code registration and token requests.
1. ease some requirements on fields that are RECOMMENDED
2. fill in defaults
3. store both request and response in the client registration
4. check client secret in the /token request
## Related issues
- Fixes
https://linear.app/pomerium/issue/ENG-2462/mcp-ignore-unknown-grant-types-in-the-client-registration
- Fixes
https://linear.app/pomerium/issue/ENG-2461/mcp-support-client-secret-in-dynamic-client-registration
## User Explanation
<!-- How would you explain this change to the user? If this
change doesn't create any user-facing changes, you can leave
this blank. If filled out, add the `docs` label -->
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [ ] ready for review
## Summary
In our efforts to sign and notarize binaries we want to consolidate
builds to a private repo for darwin. This moves the build to a workflow
in mac-builds, and moves goreleaser mac config to a separate file.
## Related issues
<!-- For example...
- #159
-->
## User Explanation
<!-- How would you explain this change to the user? If this
change doesn't create any user-facing changes, you can leave
this blank. If filled out, add the `docs` label -->
## Checklist
- [ ] reference any related issues
- [ ] updated unit tests
- [ ] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [ ] ready for review
---------
Co-authored-by: Kenneth Jenkins <51246568+kenjenkins@users.noreply.github.com>
## Summary
Refactor the local reply config to its own file, add support to the lua
function to detect gRPC requests, and add dedicated gRPC mappers for
response flags. I attempted to map the envoy error codes into things
that made sense for gRPC.
With grpcurl you'd see something like this:
```bash
$ grpcurl -v -vv -insecure example.com:443 list
Failed to list services: rpc error: code = Unavailable desc = {
"requestId": "f9ce923a-4444-4a2a-9b60-12c86f82fe10",
"status": "503",
"statusText": "upstream_reset_before_response_started{remote_connection_failure|delayed_connect_error:_Connection_refused}"
}
```
Whereas previously it would render an HTML error.
## Related issues
-
[ENG-2426](https://linear.app/pomerium/issue/ENG-2426/core-error-formatting-for-grpc)
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
If a certificate was its own authority it would result in `NormalizePEM`
going into an infinite loop. This PR updates the code to avoid cycles
using a set.
## Related issues
-
[ENG-2423](https://linear.app/pomerium/issue/ENG-2423/enterprise-console-updatekeypair-check-is-too-restrictive)
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [ ] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
Go requires that the first certificate in a bundle be the one associated
with a private key:
> LoadX509KeyPair reads and parses a public/private key pair from a pair
of files. The files must contain PEM encoded data. The certificate file
may contain intermediate certificates following the leaf certificate to
form a certificate chain. On successful return, Certificate.Leaf will be
populated.
I don't think Go is unusual in this regard, but to make the code more
tolerant, add a new `NormalizePEM` function which will take raw PEM data
and rewrite it so that leaf certificates appear first. This will be used
in zero and the enterprise console.
## Related issues
-
[ENG-2433](https://linear.app/pomerium/issue/ENG-2423/enterprise-console-updatekeypair-check-is-too-restrictive)
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
Some clients may send RFC7591 Client Registration Request with extra
fields that are not part of the spec, and we used too restrictive
decoder for that. This PR ignores the unknown fields.
## Related issues
<!-- For example...
- #159
-->
## User Explanation
<!-- How would you explain this change to the user? If this
change doesn't create any user-facing changes, you can leave
this blank. If filled out, add the `docs` label -->
## Checklist
- [ ] reference any related issues
- [ ] updated unit tests
- [ ] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [ ] ready for review
## Summary
adds implementation of `/.pomerium/mcp/connect` method, that takes a
`redirect_url` parameter and would ensure the user goes thru required
redirects so that its session is hydrated with the upstream Oauth token
for the MCP server.
the `redirect_url` parameter host must match one of the _client_ mcp
routes (currently identified by the presence of `mcp:
pass_upstream_access_token: true` in the route.
## Related issues
Fix
https://linear.app/pomerium/issue/ENG-2321/mcp-support-handling-external-oauth-servers
## User Explanation
<!-- How would you explain this change to the user? If this
change doesn't create any user-facing changes, you can leave
this blank. If filled out, add the `docs` label -->
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [ ] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
Add a new `SyncCache`:
```go
type SyncCache interface {
// Clear deletes all the data for the given record type in the sync cache.
Clear(recordType string) error
// Records yields the databroker records stored in the cache.
Records(recordType string) iter.Seq2[*Record, error]
// Sync syncs the cache with the databroker.
Sync(ctx context.Context, client DataBrokerServiceClient, recordType string) error
}
```
The cache maintains databroker records in a local pebble database (which
could be on-disk or in-memory). The way it's used is you first call
`.Sync(ctx, client, recordType)` and then `.Records(recordType)`, which
returns an iterator over all the records.
Internally we store the databroker records in a pebble key-value
database. Pebble was chosen because its fast and well-tested, but any
ordered key-value store would work. The first time we call `SyncLatest`
to retrieve all the records. Each subsequent time we call `Sync` with
the current server and record versions to retrieve only the changes.
This is significantly more efficient than calling `SyncLatest` every
time.
The primary use for this is in the enterprise-console as part of
directory sync to improve performance with large datasets.
## Related issues
-
[ENG-2401](https://linear.app/pomerium/issue/ENG-2401/enterprise-console-improve-performance-of-directory-sync-using-cached)
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
---------
Co-authored-by: Denis Mishin <dmishin@pomerium.com>
## Summary
Add a new `wait` field to the sync request for the databroker. The
current behavior is to always wait for changes in a never-ending stream
of records, but there are cases where it would be useful to stream the
changes and stop when there are no changes remaining. The storage
backends already support this.
The `wait` field is optional and the default will be to wait, preserving
the existing behavior.
## Related issues
-
[ENG-2401](https://linear.app/pomerium/issue/ENG-2401/enterprise-console-improve-performance-of-directory-sync-using-cached)
## Checklist
- [x] reference any related issues
- [ ] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
the `mcp: pass_upstream_access_token` option should take effect even if
no upstream oauth config is set.
## Related issues
<!-- For example...
- #159
-->
## User Explanation
<!-- How would you explain this change to the user? If this
change doesn't create any user-facing changes, you can leave
this blank. If filled out, add the `docs` label -->
## Checklist
- [ ] reference any related issues
- [ ] updated unit tests
- [ ] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [ ] ready for review
## Summary
Add metrics for the global cache. Configure `otel` to export metrics to
prometheus.
## Related issues
-
[ENG-2407](https://linear.app/pomerium/issue/ENG-2407/add-additional-metrics-and-tracing-spans-to-pomerium)
## Checklist
- [x] reference any related issues
- [ ] updated unit tests
- [ ] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
Currently in core we support weighted URLs like this in the config file:
```yaml
to:
- https://a.example.com,1
- https://b.example.com,2
```
However in the protobuf we use a separate `load_balancing_weights`
field:
```proto
message Route {
repeated string to = 3;
repeated uint32 load_balancing_weights = 37;
}
```
This PR updates the code to convert from protobuf so that it also
supports weights directly in the `to` addresses. The existing
`load_balancing_weights` behavior is preserved and will take precedence
when provided.
## Related issues
-
[ENG-2398](https://linear.app/pomerium/issue/ENG-2398/enterprise-api-upstream-weight-is-not-parsed)
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
Update the `RouteID` to use the `policy.ID` if it is set. This makes it
so that updated routes use a stable identifier between updates so if the
envoy control plane is updated before the authorize service's internal
definitions (or vice-versa) the authorize service will still be able to
match the route.
The current behavior results in a 404 if envoy passes the old route id.
The new behavior will result in inconsistency, but it should be quickly
remedied. To help with debugging 4 new fields were added to the
authorize check log. The `route-id` and `route-checksum` as the
authorize sees it and the `envoy-route-id` and `envoy-route-checksum` as
envoy sees it.
I also updated the way we send updates to envoy to try and model their
recommended approach:
> In general, to avoid traffic drop, sequencing of updates should follow
a make before break model, wherein:
>
> - CDS updates (if any) must always be pushed first.
> - EDS updates (if any) must arrive after CDS updates for the
respective clusters.
> - LDS updates must arrive after corresponding CDS/EDS updates.
> - RDS updates related to the newly added listeners must arrive after
CDS/EDS/LDS updates.
> - VHDS updates (if any) related to the newly added RouteConfigurations
must arrive after RDS updates.
> - Stale CDS clusters and related EDS endpoints (ones no longer being
referenced) can then be removed.
This should help avoid 404s when configuration is being updated.
## Related issues
-
[ENG-2386](https://linear.app/pomerium/issue/ENG-2386/large-number-of-routes-leads-to-404s-and-slowness)
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
- fix a typo in `pkg/hpke/hpke.go` comment
## Testing
- `go vet` *(fails: module download timed out)*
- `go test ./...` *(fails: module download timed out)*
[n.b. this was an experiment using codex]
IdP-token-based session creation makes requests to the authenticate
service to verify tokens. We have a singleflight group to avoid having
duplicate requests in flight, but it looks like this is not working as
intended. Move the IncomingIDPTokenSessionCreator initialization into
the main authorize state object, and out of the request path.
Add an integration test to assert that making a large number of requests
with the same IdP token will result in only one token verification
request to the authenticate service.
## Summary
Implement direct access token support for GitHub. GitHub doesn't have
identity tokens, so that isn't supported. The "IdP Access Token Allowed
Audiences" option is also not supported because GitHub doesn't populate
an `aud` claim.
## Related issues
-
[ENG-2137](https://linear.app/pomerium/issue/ENG-2137/core-implement-token-validation-for-github)
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
Invalidate the sync querier when records are updated so that we fallback
to databroker querying until the sync is complete.
## Related issues
For
[ENG-2377](https://linear.app/pomerium/issue/ENG-2377/core-initial-access-with-idp-accessidentity-tokens-sometimes-fails)
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
For the generic `oidc` provider, used by `auth0`, `cognito`, `gitlab`,
`google`, `oidc`, `okta`, `onelogin` and `ping`, add support for direct
access and identity token verification. Because Keycloak uses `oidc`
this also adds support for Keycloak.
Access tokens are verified by using the user info endpoint. If a call to
this endpoint succeeds using the access token, that access token is
considered valid and the user info claims will be returned.
Identity tokens are verified by using the jwks endpoint to retrieve the
signing key, and verifying that the identity token was signed with that
key. If the identity token is valid the claims in the JWT will be
returned.
## Related issues
-
[ENG-2312](https://linear.app/pomerium/issue/ENG-2312/core-implement-token-validation-for-keycloak)
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
Add support for IdP identity token authorization for Apple. Apple does
not appear to support access token validation.
This allows a user to pass an identity token directly as a bearer token:
```
curl -H 'Authorization: Bearer Apple-Identity-Token' ...
```
## Related issues
-
[ENG-2000](https://linear.app/pomerium/issue/ENG-2000/core-implement-token-validation-for-apple)
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
## Summary
Adds global runtime flag to enable/disable MCP support. (off by
default).
```yaml
runtime_flags:
mcp: true
```
## Related issues
Fix:
https://linear.app/pomerium/issue/ENG-2367/place-mcp-support-behind-a-runtime-flag
## User Explanation
<!-- How would you explain this change to the user? If this
change doesn't create any user-facing changes, you can leave
this blank. If filled out, add the `docs` label -->
## Checklist
- [x] reference any related issues
- [ ] updated unit tests
- [ ] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [ ] ready for review
## Summary
Upgrade `google.golang.org/grpc/health/grpc_health_v1` which has a new
`List` method, so use the `UnimplementedHealthServer`.
## Related issues
- https://github.com/pomerium/ingress-controller/pull/1152
## User Explanation
<!-- How would you explain this change to the user? If this
change doesn't create any user-facing changes, you can leave
this blank. If filled out, add the `docs` label -->
## Checklist
- [ ] reference any related issues
- [ ] updated unit tests
- [ ] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
