Caleb Doxsey
a243056cfa
Revert "databroker: add support for field masks on Put ( #3210 )" ( #3217 )
...
This reverts commit 2dc778035d
2022-03-31 11:17:57 -04:00
Caleb Doxsey
2dc778035d
databroker: add support for field masks on Put ( #3210 )
...
* databroker: add support for field masks on Put
* return errors
* clean up go.mod
2022-03-29 16:36:40 -06:00
Caleb Doxsey
1238f0506d
databroker: add additional log for config source ( #2718 )
2021-10-27 13:02:37 -06:00
Caleb Doxsey
b2c76c3816
grpc: remove peer field from logs ( #2712 )
2021-10-26 14:43:59 -06:00
Denis Mishin
30664cd307
skip configuration updates to the most recent one ( #2690 )
2021-10-21 11:03:26 -04:00
Caleb Doxsey
f5a558d4a0
grpc: disable gRPC connection re-use across services ( #2515 )
2021-08-24 11:47:16 -06:00
Caleb Doxsey
bbec2cae9f
grpc: send client traffic through envoy ( #2469 )
...
* wip
* wip
* handle wildcards in override name
* remove wait for ready, add comment about sync, force initial sync complete in test
* address comments
2021-08-16 16:12:22 -06:00
Caleb Doxsey
6af0655206
protoutil: add NewAny method for deterministic serialization ( #2462 )
2021-08-09 17:51:57 -06:00
Caleb Doxsey
ac8ae3ef5b
directory: add logging http client to help with debugging outbound http requests ( #2385 )
2021-07-22 11:58:52 -06:00
Caleb Doxsey
d45a7e9996
databroker: tests ( #2367 )
...
* databroker: tests
* fix lint
2021-07-16 10:26:29 -06:00
wasaga
41a2622736
certs: reject certs from databroker if they conflict with local ( #2309 )
2021-06-24 18:40:59 -04:00
Caleb Doxsey
94aa0b1a48
databroker: implement leases ( #2172 )
...
* databroker: implement leases
* return error
* handle gRPC errors
2021-05-10 13:30:25 -06:00
Caleb Doxsey
a54d43b937
registry: implement redis backend ( #2179 )
2021-05-10 10:33:37 -06:00
Caleb Doxsey
0adbf4f24c
controlplane: save configuration events to databroker ( #2153 )
...
* envoy: save events to databroker
* controlplane: add tests for envoy configuration events
* format imports
2021-04-29 15:51:46 -06:00
Caleb Doxsey
91c7dc742f
databroker: store server version in backend ( #2142 )
2021-04-28 09:12:52 -06:00
Caleb Doxsey
636b3d6846
databroker: add options for maximum capacity ( #2095 )
...
* databroker: add options
* implement redis
* add trace for enforce options
2021-04-26 17:14:54 -06:00
Caleb Doxsey
b1d62bb541
config: remove validate side effects ( #2109 )
...
* config: default shared key
* handle additional errors
* update grpc addr and grpc insecure
* update google cloud service authentication service account
* fix set response headers
* fix qps
* fix test
2021-04-22 15:10:50 -06:00
wasaga
e0c09a0998
log context ( #2107 )
2021-04-22 10:58:13 -04:00
wasaga
a935c1ba30
config related metrics ( #2065 )
2021-04-07 12:29:36 -07:00
Caleb Doxsey
294addd857
databroker: remove unused installation id, close streams when backend is closed ( #2062 )
2021-04-06 13:41:19 -06:00
Caleb Doxsey
d7ab817de7
authorize: add databroker server and record version to result, force sync via polling ( #2024 )
...
* authorize: add databroker server and record version to result, force sync via polling
* wrap inmem store to take read lock when grabbing databroker versions
* address code review comments
* reset max to 0
2021-03-31 10:09:06 -06:00
wasaga
80c55dd50c
databroker: return server version in Get ( #2039 )
2021-03-29 13:18:38 -04:00
Caleb Doxsey
e2ebef44ef
telemetry: add installation id ( #2017 )
...
* telemetry: add installation id
* set installation id globally
* remove unneeded changes
2021-03-24 07:22:54 -06:00
Caleb Doxsey
f396c2a0f7
config: log config source changes ( #1959 )
...
* config: log config source changes
* use internal log import
2021-03-03 09:54:08 -07:00
Caleb Doxsey
664358dfad
config: multiple endpoints for authorize and databroker ( #1957 )
...
* wip
* update docs
* remove dead code
2021-03-03 09:53:19 -07:00
Caleb Doxsey
5d60cff21e
databroker: refactor databroker to sync all changes ( #1879 )
...
* refactor backend, implement encrypted store
* refactor in-memory store
* wip
* wip
* wip
* add syncer test
* fix redis expiry
* fix linting issues
* fix test by skipping non-config records
* fix backoff import
* fix init issues
* fix query
* wait for initial sync before starting directory sync
* add type to SyncLatest
* add more log messages, fix deadlock in in-memory store, always return server version from SyncLatest
* update sync types and tests
* add redis tests
* skip macos in github actions
* add comments to proto
* split getBackend into separate methods
* handle errors in initVersion
* return different error for not found vs other errors in get
* use exponential backoff for redis transaction retry
* rename raw to result
* use context instead of close channel
* store type urls as constants in databroker
* use timestampb instead of ptypes
* fix group merging not waiting
* change locked names
* update GetAll to return latest record version
* add method to grpcutil to get the type url for a protobuf type
2021-02-18 15:24:33 -07:00
wasaga
67f6030e1e
upstream endpoints load balancer weights ( #1830 )
2021-01-28 09:11:14 -05:00
Caleb Doxsey
84e8f6cc05
config: fix databroker policies ( #1821 )
2021-01-25 17:18:50 -07:00
Caleb Doxsey
a4c7381eba
config: support multiple destination addresses ( #1789 )
...
* config: support multiple destination addresses
* use constructor for string slice
* add docs
* add test for multiple destinations
* fix name
2021-01-20 15:18:24 -07:00
bobby
f837c92741
dev: update linter ( #1728 )
...
- gofumpt everything
- fix TLS MinVersion to be at least 1.2
- add octal syntax
- remove newlines
- fix potential decompression bomb in ecjson
- remove implicit memory aliasing in for loops.
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-12-30 09:02:57 -08:00
Caleb Doxsey
3b634de550
implement new redis storage backend with go-redis package ( #1649 )
2020-12-10 12:21:31 -07:00
Caleb Doxsey
d18e8c661d
improve ca cert error message, use GetCertPool for databroker storage ( #1666 )
2020-12-09 11:16:39 -07:00
bobby
652e8bb3d3
deps: update hashstructure v2 ( #1632 )
...
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-11-30 16:53:21 -08:00
Caleb Doxsey
aad8ac2e61
replace GetAllPages with InitialSync, improve merge performance ( #1624 )
...
* replace GetAllPages with InitialSync, improve merge performance
* fmt proto
* add test for base64 function
* add sync test
* go mod tidy
Co-authored-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-11-30 12:21:44 -07:00
Caleb Doxsey
a41c37f9e0
add paging support to GetAll ( #1601 )
...
* add paging support to GetAll
* fix import
2020-11-18 17:02:57 -07:00
Caleb Doxsey
eb79cc0957
databroker: require JWT for access ( #1503 )
2020-10-09 11:08:40 -06:00
Caleb Doxsey
f4c61a0cdc
redis: use pubsub instead of keyspace events ( #1450 )
2020-09-23 14:40:05 -06:00
Caleb Doxsey
2364da14c8
databroker: add support for querying the databroker ( #1443 )
...
* databroker: add support for querying the databroker
* remove query method, use getall so encryption works
* add test
* return early
2020-09-22 16:01:37 -06:00
Caleb Doxsey
0a6796ff71
authorize: add support for service accounts ( #1374 )
2020-09-04 10:37:00 -06:00
Caleb Doxsey
49d1a71ff2
databroker: add tracing for rego evaluation and databroker sync, fix bug in databroker config source ( #1367 )
2020-09-03 08:11:34 -06:00
Caleb Doxsey
a1378c81f8
cache: support databroker option changes ( #1294 )
2020-08-18 07:27:20 -06:00
Caleb Doxsey
bd5c784670
config: validate databroker settings ( #1260 )
...
* config: validate databroker settings
* fix test
2020-08-12 11:32:34 -06:00
Caleb Doxsey
1285a9d91d
databroker: add support for config settings ( #1253 )
2020-08-11 07:50:19 -06:00
Cuong Manh Le
02edbb7748
internal/databroker: make Sync send data in smaller batches ( #1226 )
...
* internal/databroker: make Sync send data in smaller batches
GRPC streaming is better at sending multiple smaller message instead of
a big one.
Benchmark result for sending 10k messages at once vs multiple batches,
each with 100 messages:
name old time/op new time/op delta
Sync-12 14.5ms ± 3% 12.4ms ± 2% -14.40% (p=0.000 n=10+9)
* cache: add test for databroker sync
2020-08-07 23:12:41 +07:00
Cuong Manh Le
bc61206b78
pkg/storage/redis: add redis TLS support ( #1163 )
...
Fixes #1156
2020-07-31 19:37:23 +07:00
Travis Groth
3c4513a91e
telmetry: add databroker storage metrics and tracing ( #1161 )
...
* telmetry: add databroker storage metrics and tracing
2020-07-30 18:19:23 -04:00
Caleb Doxsey
29fb96a955
databroker: add encryption for records ( #1168 )
2020-07-30 14:04:31 -06:00
Caleb Doxsey
97f85481f8
fix redirect loop, remove user/session services, remove duplicate deleted_at fields ( #1162 )
...
* fix redirect loop, remove user/session services, remove duplicate deleted_at fields
* change loop
* reuse err variable
* wrap errors, use cookie timeout
* wrap error, duplicate if
2020-07-30 09:41:57 -06:00
Cuong Manh Le
3039407597
pkg/storage/redis: add authentication support ( #1159 )
...
Fixes #1157
2020-07-29 23:08:38 +07:00
Caleb Doxsey
557aef2a33
fix databroker restart versioning, handle missing sessions ( #1145 )
...
* fix databroker restart versioning, handle missing sessions
* send empty server version to detect change
* only rebuild if there are updated records
2020-07-29 08:45:41 -06:00
