Commit graph

84 commits

Author SHA1 Message Date
Denis Mishin
bf1cd0aa18
authorize: build evaluators cache in parallel (#4722)
* authorize: build evaluators cache in parallel

* session: add unit tests for gRPC wrapper methods (#4713)

* core/config: add support for maps in environments (#4717)

* reconciler: allow custom comparison function (#4726)

* add loopvar alias

---------

Co-authored-by: Kenneth Jenkins <51246568+kenjenkins@users.noreply.github.com>
Co-authored-by: Caleb Doxsey <cdoxsey@pomerium.com>
2023-11-09 11:49:34 -05:00
Denis Mishin
bfcc970839
databroker: build config concurrently, option to bypass validation (#4655)
* validation: option to bypass

* concurrently build config

* add regex_priority_order and route sorting

* rm mutex
2023-11-06 13:21:29 -05:00
Kenneth Jenkins
d5da872157
databroker: add patch method (#4704)
Add a Patch() method to the databroker gRPC service.

Update the storage.Backend interface to include the Patch() method now
that all the storage.Backend implementations include it.

Add a test to exercise the patch method under concurrent usage.
2023-11-02 15:07:37 -07:00
Kenneth Jenkins
4f648e9ac1
databroker: remove redis storage backend (#4699)
Remove the Redis databroker backend. According to
https://www.pomerium.com/docs/internals/data-storage#redis it has been
discouraged since Pomerium v0.18.

Update the config options validation to return an error if "redis" is 
set as the databroker storage backend type.
2023-11-02 11:53:25 -07:00
Caleb Doxsey
53573dc046
core/config: remove version (#4653)
* core/config: remove version

* lint

* fix
2023-11-01 10:19:55 -06:00
Caleb Doxsey
ae420f01c6
core/config: add config version, additional telemetry (#4645)
* core/config: add config version, additional telemetry

* typo
2023-10-27 15:16:40 -06:00
Caleb Doxsey
baf964f44a
config: update logic for checking overlapping certificates (#4216)
* config: update logic for checking overlapping certificates

* add test

* go mod tidy
2023-06-01 09:30:46 -06:00
Caleb Doxsey
10662d7034
databroker: fix fast forward (#4192)
* databroker: sort configs

* databroker: fix fast-forward

* newest not oldest
2023-05-23 15:30:27 -06:00
Caleb Doxsey
fe8e788076
databroker: sort configs (#4190) 2023-05-23 10:08:29 -06:00
Caleb Doxsey
bbed421cd8
config: remove source, remove deadcode, fix linting issues (#4118)
* remove source, remove deadcode, fix linting issues

* use github action for lint

* fix missing envoy
2023-04-21 17:25:11 -06:00
Caleb Doxsey
7895bf431f
databroker: add list types method (#3937)
* databroker: add list types method

* fix test

* Update pkg/storage/redis/redis.go

Co-authored-by: Denis Mishin <dmishin@pomerium.com>

---------

Co-authored-by: Denis Mishin <dmishin@pomerium.com>
2023-02-03 13:16:28 -07:00
Denis Mishin
e019885218
mTLS: allow gRPC TLS for all in one (#3854)
* make grpc_insecure an optional bool

* use internal addresses for all in one databroker and tls
2023-01-03 12:45:04 -05:00
Denis Mishin
a3cfe8fa42
keep trace span context (#3724) 2022-11-04 17:52:13 -04:00
Caleb Doxsey
c178819875
move directory providers (#3633)
* remove directory providers and support for groups

* idp: remove directory providers

* better error messages

* fix errors

* restore postgres

* fix test
2022-11-03 11:33:56 -06:00
dependabot[bot]
60b9f3d92d
chore(deps): bump github.com/golangci/golangci-lint from 1.47.3 to 1.48.0 (#3541)
* chore(deps): bump github.com/golangci/golangci-lint

Bumps [github.com/golangci/golangci-lint](https://github.com/golangci/golangci-lint) from 1.47.3 to 1.48.0.
- [Release notes](https://github.com/golangci/golangci-lint/releases)
- [Changelog](https://github.com/golangci/golangci-lint/blob/master/CHANGELOG.md)
- [Commits](https://github.com/golangci/golangci-lint/compare/v1.47.3...v1.48.0)

---
updated-dependencies:
- dependency-name: github.com/golangci/golangci-lint
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix linting issues

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Caleb Doxsey <cdoxsey@pomerium.com>
2022-08-09 08:25:57 -06:00
Caleb Doxsey
24a9d627cd
postgres: registry support (#3454) 2022-07-13 09:14:47 -06:00
Caleb Doxsey
45a29ea879
databroker: add support for syncing by type (#3412)
* databroker: add support for syncing by type

* add type url, fix query
2022-06-13 09:52:13 -06:00
Caleb Doxsey
1c2aad2de6
postgres: databroker storage backend (#3370)
* wip

* storage: add filtering to SyncLatest

* don't increment the record version, so intermediate changes are requested

* databroker: add support for query filtering

* fill server and record version

* postgres: databroker storage backend

* wip

* serialize puts

* add test

* skip tests for macos

* add test

* return error from protojson

* set data

* exclude postgres from cover tests
2022-05-25 10:23:58 -06:00
Caleb Doxsey
994faba0c8
databroker: add support for query filtering (#3369)
* wip

* storage: add filtering to SyncLatest

* don't increment the record version, so intermediate changes are requested

* databroker: add support for query filtering

* fill server and record version

* add test checks

* add explanation to query filter error
2022-05-19 09:07:32 -06:00
Caleb Doxsey
1669b601ea
storage: add filtering to SyncLatest (#3368)
* wip

* storage: add filtering to SyncLatest

* don't increment the record version, so intermediate changes are requested

* fix stream filter
2022-05-17 16:00:23 -06:00
Caleb Doxsey
f73c5c615f
databroker: add support for putting multiple records (#3291)
* databroker: add support for putting multiple records

* add OptimumPutRequestsFromRecords function

* replace GetAll with SyncLatest

* fix stream when there are no records
2022-04-26 16:41:38 -06:00
Caleb Doxsey
761c17b8ac
grpc: wait for connect to be ready before making calls (#3253)
* grpc: wait for connect to be ready before making calls

* make sure to stop the ticker
2022-04-08 12:18:52 -06:00
Caleb Doxsey
a243056cfa
Revert "databroker: add support for field masks on Put (#3210)" (#3217)
This reverts commit 2dc778035d.
2022-03-31 11:17:57 -04:00
Caleb Doxsey
2dc778035d
databroker: add support for field masks on Put (#3210)
* databroker: add support for field masks on Put

* return errors

* clean up go.mod
2022-03-29 16:36:40 -06:00
Caleb Doxsey
1238f0506d
databroker: add additional log for config source (#2718) 2021-10-27 13:02:37 -06:00
Caleb Doxsey
b2c76c3816
grpc: remove peer field from logs (#2712) 2021-10-26 14:43:59 -06:00
Denis Mishin
30664cd307
skip configuration updates to the most recent one (#2690) 2021-10-21 11:03:26 -04:00
Caleb Doxsey
f5a558d4a0
grpc: disable gRPC connection re-use across services (#2515) 2021-08-24 11:47:16 -06:00
Caleb Doxsey
bbec2cae9f
grpc: send client traffic through envoy (#2469)
* wip

* wip

* handle wildcards in override name

* remove wait for ready, add comment about sync, force initial sync complete in test

* address comments
2021-08-16 16:12:22 -06:00
Caleb Doxsey
6af0655206
protoutil: add NewAny method for deterministic serialization (#2462) 2021-08-09 17:51:57 -06:00
Caleb Doxsey
ac8ae3ef5b
directory: add logging http client to help with debugging outbound http requests (#2385) 2021-07-22 11:58:52 -06:00
Caleb Doxsey
d45a7e9996
databroker: tests (#2367)
* databroker: tests

* fix lint
2021-07-16 10:26:29 -06:00
wasaga
41a2622736
certs: reject certs from databroker if they conflict with local (#2309) 2021-06-24 18:40:59 -04:00
Caleb Doxsey
94aa0b1a48
databroker: implement leases (#2172)
* databroker: implement leases

* return error

* handle gRPC errors
2021-05-10 13:30:25 -06:00
Caleb Doxsey
a54d43b937
registry: implement redis backend (#2179) 2021-05-10 10:33:37 -06:00
Caleb Doxsey
0adbf4f24c
controlplane: save configuration events to databroker (#2153)
* envoy: save events to databroker

* controlplane: add tests for envoy configuration events

* format imports
2021-04-29 15:51:46 -06:00
Caleb Doxsey
91c7dc742f
databroker: store server version in backend (#2142) 2021-04-28 09:12:52 -06:00
Caleb Doxsey
636b3d6846
databroker: add options for maximum capacity (#2095)
* databroker: add options

* implement redis

* add trace for enforce options
2021-04-26 17:14:54 -06:00
Caleb Doxsey
b1d62bb541
config: remove validate side effects (#2109)
* config: default shared key

* handle additional errors

* update grpc addr and grpc insecure

* update google cloud service authentication service account

* fix set response headers

* fix qps

* fix test
2021-04-22 15:10:50 -06:00
wasaga
e0c09a0998
log context (#2107) 2021-04-22 10:58:13 -04:00
wasaga
a935c1ba30
config related metrics (#2065) 2021-04-07 12:29:36 -07:00
Caleb Doxsey
294addd857
databroker: remove unused installation id, close streams when backend is closed (#2062) 2021-04-06 13:41:19 -06:00
Caleb Doxsey
d7ab817de7
authorize: add databroker server and record version to result, force sync via polling (#2024)
* authorize: add databroker server and record version to result, force sync via polling

* wrap inmem store to take read lock when grabbing databroker versions

* address code review comments

* reset max to 0
2021-03-31 10:09:06 -06:00
wasaga
80c55dd50c
databroker: return server version in Get (#2039) 2021-03-29 13:18:38 -04:00
Caleb Doxsey
e2ebef44ef
telemetry: add installation id (#2017)
* telemetry: add installation id

* set installation id globally

* remove unneeded changes
2021-03-24 07:22:54 -06:00
Caleb Doxsey
f396c2a0f7
config: log config source changes (#1959)
* config: log config source changes

* use internal log import
2021-03-03 09:54:08 -07:00
Caleb Doxsey
664358dfad
config: multiple endpoints for authorize and databroker (#1957)
* wip

* update docs

* remove dead code
2021-03-03 09:53:19 -07:00
Caleb Doxsey
5d60cff21e
databroker: refactor databroker to sync all changes (#1879)
* refactor backend, implement encrypted store

* refactor in-memory store

* wip

* wip

* wip

* add syncer test

* fix redis expiry

* fix linting issues

* fix test by skipping non-config records

* fix backoff import

* fix init issues

* fix query

* wait for initial sync before starting directory sync

* add type to SyncLatest

* add more log messages, fix deadlock in in-memory store, always return server version from SyncLatest

* update sync types and tests

* add redis tests

* skip macos in github actions

* add comments to proto

* split getBackend into separate methods

* handle errors in initVersion

* return different error for not found vs other errors in get

* use exponential backoff for redis transaction retry

* rename raw to result

* use context instead of close channel

* store type urls as constants in databroker

* use timestampb instead of ptypes

* fix group merging not waiting

* change locked names

* update GetAll to return latest record version

* add method to grpcutil to get the type url for a protobuf type
2021-02-18 15:24:33 -07:00
wasaga
67f6030e1e
upstream endpoints load balancer weights (#1830) 2021-01-28 09:11:14 -05:00
Caleb Doxsey
84e8f6cc05
config: fix databroker policies (#1821) 2021-01-25 17:18:50 -07:00