mirror of
https://github.com/pomerium/pomerium.git
synced 2025-08-03 00:40:25 +02:00
ff607fa018
## Summary
Add a new `SyncCache`:
```go
type SyncCache interface {
// Clear deletes all the data for the given record type in the sync cache.
Clear(recordType string) error
// Records yields the databroker records stored in the cache.
Records(recordType string) iter.Seq2[*Record, error]
// Sync syncs the cache with the databroker.
Sync(ctx context.Context, client DataBrokerServiceClient, recordType string) error
}
```
The cache maintains databroker records in a local pebble database (which
could be on-disk or in-memory). The way it's used is you first call
`.Sync(ctx, client, recordType)` and then `.Records(recordType)`, which
returns an iterator over all the records.
Internally we store the databroker records in a pebble key-value
database. Pebble was chosen because its fast and well-tested, but any
ordered key-value store would work. The first time we call `SyncLatest`
to retrieve all the records. Each subsequent time we call `Sync` with
the current server and record versions to retrieve only the changes.
This is significantly more efficient than calling `SyncLatest` every
time.
The primary use for this is in the enterprise-console as part of
directory sync to improve performance with large datasets.
## Related issues
-
[ENG-2401](https://linear.app/pomerium/issue/ENG-2401/enterprise-console-improve-performance-of-directory-sync-using-cached)
## Checklist
- [x] reference any related issues
- [x] updated unit tests
- [x] add appropriate label (`enhancement`, `bug`, `breaking`,
`dependencies`, `ci`)
- [x] ready for review
---------
Co-authored-by: Denis Mishin <dmishin@pomerium.com>
|
||
|---|---|---|
| .. | ||
| authenticateapi | ||
| cmd/pomerium | ||
| contextutil | ||
| counter | ||
| cryptutil | ||
| derivecert | ||
| encoding/base58 | ||
| envoy | ||
| fanout | ||
| grpc | ||
| grpcutil | ||
| health | ||
| hpke | ||
| httputil | ||
| identity | ||
| logutil | ||
| metrics | ||
| netutil | ||
| pebbleutil | ||
| policy | ||
| protoutil | ||
| slices | ||
| storage | ||
| telemetry | ||
| webauthnutil | ||
| zero | ||