Commit graph

72 commits

Author SHA1 Message Date
Denis Mishin
a3cfe8fa42
keep trace span context (#3724) 2022-11-04 17:52:13 -04:00
Caleb Doxsey
c178819875
move directory providers (#3633)
* remove directory providers and support for groups

* idp: remove directory providers

* better error messages

* fix errors

* restore postgres

* fix test
2022-11-03 11:33:56 -06:00
dependabot[bot]
60b9f3d92d
chore(deps): bump github.com/golangci/golangci-lint from 1.47.3 to 1.48.0 (#3541)
* chore(deps): bump github.com/golangci/golangci-lint

Bumps [github.com/golangci/golangci-lint](https://github.com/golangci/golangci-lint) from 1.47.3 to 1.48.0.
- [Release notes](https://github.com/golangci/golangci-lint/releases)
- [Changelog](https://github.com/golangci/golangci-lint/blob/master/CHANGELOG.md)
- [Commits](https://github.com/golangci/golangci-lint/compare/v1.47.3...v1.48.0)

---
updated-dependencies:
- dependency-name: github.com/golangci/golangci-lint
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix linting issues

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Caleb Doxsey <cdoxsey@pomerium.com>
2022-08-09 08:25:57 -06:00
Caleb Doxsey
24a9d627cd
postgres: registry support (#3454) 2022-07-13 09:14:47 -06:00
Caleb Doxsey
45a29ea879
databroker: add support for syncing by type (#3412)
* databroker: add support for syncing by type

* add type url, fix query
2022-06-13 09:52:13 -06:00
Caleb Doxsey
1c2aad2de6
postgres: databroker storage backend (#3370)
* wip

* storage: add filtering to SyncLatest

* don't increment the record version, so intermediate changes are requested

* databroker: add support for query filtering

* fill server and record version

* postgres: databroker storage backend

* wip

* serialize puts

* add test

* skip tests for macos

* add test

* return error from protojson

* set data

* exclude postgres from cover tests
2022-05-25 10:23:58 -06:00
Caleb Doxsey
994faba0c8
databroker: add support for query filtering (#3369)
* wip

* storage: add filtering to SyncLatest

* don't increment the record version, so intermediate changes are requested

* databroker: add support for query filtering

* fill server and record version

* add test checks

* add explanation to query filter error
2022-05-19 09:07:32 -06:00
Caleb Doxsey
1669b601ea
storage: add filtering to SyncLatest (#3368)
* wip

* storage: add filtering to SyncLatest

* don't increment the record version, so intermediate changes are requested

* fix stream filter
2022-05-17 16:00:23 -06:00
Caleb Doxsey
f73c5c615f
databroker: add support for putting multiple records (#3291)
* databroker: add support for putting multiple records

* add OptimumPutRequestsFromRecords function

* replace GetAll with SyncLatest

* fix stream when there are no records
2022-04-26 16:41:38 -06:00
Caleb Doxsey
761c17b8ac
grpc: wait for connect to be ready before making calls (#3253)
* grpc: wait for connect to be ready before making calls

* make sure to stop the ticker
2022-04-08 12:18:52 -06:00
Caleb Doxsey
a243056cfa
Revert "databroker: add support for field masks on Put (#3210)" (#3217)
This reverts commit 2dc778035d.
2022-03-31 11:17:57 -04:00
Caleb Doxsey
2dc778035d
databroker: add support for field masks on Put (#3210)
* databroker: add support for field masks on Put

* return errors

* clean up go.mod
2022-03-29 16:36:40 -06:00
Caleb Doxsey
1238f0506d
databroker: add additional log for config source (#2718) 2021-10-27 13:02:37 -06:00
Caleb Doxsey
b2c76c3816
grpc: remove peer field from logs (#2712) 2021-10-26 14:43:59 -06:00
Denis Mishin
30664cd307
skip configuration updates to the most recent one (#2690) 2021-10-21 11:03:26 -04:00
Caleb Doxsey
f5a558d4a0
grpc: disable gRPC connection re-use across services (#2515) 2021-08-24 11:47:16 -06:00
Caleb Doxsey
bbec2cae9f
grpc: send client traffic through envoy (#2469)
* wip

* wip

* handle wildcards in override name

* remove wait for ready, add comment about sync, force initial sync complete in test

* address comments
2021-08-16 16:12:22 -06:00
Caleb Doxsey
6af0655206
protoutil: add NewAny method for deterministic serialization (#2462) 2021-08-09 17:51:57 -06:00
Caleb Doxsey
ac8ae3ef5b
directory: add logging http client to help with debugging outbound http requests (#2385) 2021-07-22 11:58:52 -06:00
Caleb Doxsey
d45a7e9996
databroker: tests (#2367)
* databroker: tests

* fix lint
2021-07-16 10:26:29 -06:00
wasaga
41a2622736
certs: reject certs from databroker if they conflict with local (#2309) 2021-06-24 18:40:59 -04:00
Caleb Doxsey
94aa0b1a48
databroker: implement leases (#2172)
* databroker: implement leases

* return error

* handle gRPC errors
2021-05-10 13:30:25 -06:00
Caleb Doxsey
a54d43b937
registry: implement redis backend (#2179) 2021-05-10 10:33:37 -06:00
Caleb Doxsey
0adbf4f24c
controlplane: save configuration events to databroker (#2153)
* envoy: save events to databroker

* controlplane: add tests for envoy configuration events

* format imports
2021-04-29 15:51:46 -06:00
Caleb Doxsey
91c7dc742f
databroker: store server version in backend (#2142) 2021-04-28 09:12:52 -06:00
Caleb Doxsey
636b3d6846
databroker: add options for maximum capacity (#2095)
* databroker: add options

* implement redis

* add trace for enforce options
2021-04-26 17:14:54 -06:00
Caleb Doxsey
b1d62bb541
config: remove validate side effects (#2109)
* config: default shared key

* handle additional errors

* update grpc addr and grpc insecure

* update google cloud service authentication service account

* fix set response headers

* fix qps

* fix test
2021-04-22 15:10:50 -06:00
wasaga
e0c09a0998
log context (#2107) 2021-04-22 10:58:13 -04:00
wasaga
a935c1ba30
config related metrics (#2065) 2021-04-07 12:29:36 -07:00
Caleb Doxsey
294addd857
databroker: remove unused installation id, close streams when backend is closed (#2062) 2021-04-06 13:41:19 -06:00
Caleb Doxsey
d7ab817de7
authorize: add databroker server and record version to result, force sync via polling (#2024)
* authorize: add databroker server and record version to result, force sync via polling

* wrap inmem store to take read lock when grabbing databroker versions

* address code review comments

* reset max to 0
2021-03-31 10:09:06 -06:00
wasaga
80c55dd50c
databroker: return server version in Get (#2039) 2021-03-29 13:18:38 -04:00
Caleb Doxsey
e2ebef44ef
telemetry: add installation id (#2017)
* telemetry: add installation id

* set installation id globally

* remove unneeded changes
2021-03-24 07:22:54 -06:00
Caleb Doxsey
f396c2a0f7
config: log config source changes (#1959)
* config: log config source changes

* use internal log import
2021-03-03 09:54:08 -07:00
Caleb Doxsey
664358dfad
config: multiple endpoints for authorize and databroker (#1957)
* wip

* update docs

* remove dead code
2021-03-03 09:53:19 -07:00
Caleb Doxsey
5d60cff21e
databroker: refactor databroker to sync all changes (#1879)
* refactor backend, implement encrypted store

* refactor in-memory store

* wip

* wip

* wip

* add syncer test

* fix redis expiry

* fix linting issues

* fix test by skipping non-config records

* fix backoff import

* fix init issues

* fix query

* wait for initial sync before starting directory sync

* add type to SyncLatest

* add more log messages, fix deadlock in in-memory store, always return server version from SyncLatest

* update sync types and tests

* add redis tests

* skip macos in github actions

* add comments to proto

* split getBackend into separate methods

* handle errors in initVersion

* return different error for not found vs other errors in get

* use exponential backoff for redis transaction retry

* rename raw to result

* use context instead of close channel

* store type urls as constants in databroker

* use timestampb instead of ptypes

* fix group merging not waiting

* change locked names

* update GetAll to return latest record version

* add method to grpcutil to get the type url for a protobuf type
2021-02-18 15:24:33 -07:00
wasaga
67f6030e1e
upstream endpoints load balancer weights (#1830) 2021-01-28 09:11:14 -05:00
Caleb Doxsey
84e8f6cc05
config: fix databroker policies (#1821) 2021-01-25 17:18:50 -07:00
Caleb Doxsey
a4c7381eba
config: support multiple destination addresses (#1789)
* config: support multiple destination addresses

* use constructor for string slice

* add docs

* add test for multiple destinations

* fix name
2021-01-20 15:18:24 -07:00
bobby
f837c92741
dev: update linter (#1728)
- gofumpt everything
- fix TLS MinVersion to be at least 1.2
- add octal syntax
- remove newlines
- fix potential decompression bomb in ecjson
- remove implicit memory aliasing in for loops.

Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-12-30 09:02:57 -08:00
Caleb Doxsey
3b634de550
implement new redis storage backend with go-redis package (#1649) 2020-12-10 12:21:31 -07:00
Caleb Doxsey
d18e8c661d
improve ca cert error message, use GetCertPool for databroker storage (#1666) 2020-12-09 11:16:39 -07:00
bobby
652e8bb3d3
deps: update hashstructure v2 (#1632)
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-11-30 16:53:21 -08:00
Caleb Doxsey
aad8ac2e61
replace GetAllPages with InitialSync, improve merge performance (#1624)
* replace GetAllPages with InitialSync, improve merge performance

* fmt proto

* add test for base64 function

* add sync test

* go mod tidy

Co-authored-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-11-30 12:21:44 -07:00
Caleb Doxsey
a41c37f9e0
add paging support to GetAll (#1601)
* add paging support to GetAll

* fix import
2020-11-18 17:02:57 -07:00
Caleb Doxsey
eb79cc0957
databroker: require JWT for access (#1503) 2020-10-09 11:08:40 -06:00
Caleb Doxsey
f4c61a0cdc
redis: use pubsub instead of keyspace events (#1450) 2020-09-23 14:40:05 -06:00
Caleb Doxsey
2364da14c8
databroker: add support for querying the databroker (#1443)
* databroker: add support for querying the databroker

* remove query method, use getall so encryption works

* add test

* return early
2020-09-22 16:01:37 -06:00
Caleb Doxsey
0a6796ff71
authorize: add support for service accounts (#1374) 2020-09-04 10:37:00 -06:00
Caleb Doxsey
49d1a71ff2
databroker: add tracing for rego evaluation and databroker sync, fix bug in databroker config source (#1367) 2020-09-03 08:11:34 -06:00