Commit graph

74 commits

Author SHA1 Message Date
Caleb Doxsey
7c69e612b6
postgres: return unknown records instead of skipping them (#3876) (#3877) 2023-01-09 19:32:35 -07:00
backport-actions-token[bot]
ebee7c7920
storage: ignore removed fields when deserializing the data (#3772) 2022-11-29 17:26:51 -05:00
Caleb Doxsey
1b596115e9
postgres: increase record batch size (#3708) 2022-10-26 07:40:13 -06:00
dependabot[bot]
ec495bb682
chore(deps): bump github.com/golangci/golangci-lint from 1.48.0 to 1.50.0 (#3667)
* chore(deps): bump github.com/golangci/golangci-lint

Bumps [github.com/golangci/golangci-lint](https://github.com/golangci/golangci-lint) from 1.48.0 to 1.50.0.
- [Release notes](https://github.com/golangci/golangci-lint/releases)
- [Changelog](https://github.com/golangci/golangci-lint/blob/master/CHANGELOG.md)
- [Commits](https://github.com/golangci/golangci-lint/compare/v1.48.0...v1.50.0)

---
updated-dependencies:
- dependency-name: github.com/golangci/golangci-lint
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* lint

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Caleb Doxsey <cdoxsey@pomerium.com>
2022-10-19 09:36:59 -06:00
Caleb Doxsey
3b2cc6720a
postgres: return an empty list of addresses on dns errors (#3637) 2022-09-30 12:00:40 -06:00
Caleb Doxsey
3fec00f2a8
postgres: handle unknown types (#3632) 2022-09-28 09:41:29 -06:00
Caleb Doxsey
23c42da8ec
postgres: remove not null constraint on data column of record changes table (#3594) 2022-08-31 10:16:19 -06:00
Caleb Doxsey
89a105c8e6
authorize: add request id to context (#3497)
* authorize: add request id to context

* fix context keys
2022-07-26 14:34:48 -06:00
Caleb Doxsey
0b48da1e2f
databroker: support rotating shared secret (#3502)
* databroker: support rotating shared secret

* fix test

* run tests on linux

* fix tests

* fix typo

* increase timeout
2022-07-26 10:59:54 -06:00
Caleb Doxsey
fe61a74e1b
authorize: fix device synchronization (#3482) 2022-07-15 17:27:06 -06:00
Caleb Doxsey
24a9d627cd
postgres: registry support (#3454) 2022-07-13 09:14:47 -06:00
Caleb Doxsey
1727d178ef
postgres: fix record deletion (#3446) 2022-06-24 09:32:44 -06:00
Caleb Doxsey
45a29ea879
databroker: add support for syncing by type (#3412)
* databroker: add support for syncing by type

* add type url, fix query
2022-06-13 09:52:13 -06:00
Caleb Doxsey
a2d5d8062b
postgres: use CTE and GENERATED version number instead of serialized transaction (#3408)
* postgres: use CTE and GENERATED version number instead of serialized transaction

* update server version

* fix indexing CIDRs
2022-06-09 12:18:20 -06:00
Caleb Doxsey
493148b13f
authorize: fix not found check (#3410) 2022-06-08 09:15:57 -06:00
Caleb Doxsey
dafead3122
postgres: fix CIDR query (#3389) 2022-06-03 12:32:01 -06:00
Caleb Doxsey
f61e7efe73
authorize: use query instead of sync for databroker data (#3377) 2022-06-01 15:40:07 -06:00
Caleb Doxsey
1c2aad2de6
postgres: databroker storage backend (#3370)
* wip

* storage: add filtering to SyncLatest

* don't increment the record version, so intermediate changes are requested

* databroker: add support for query filtering

* fill server and record version

* postgres: databroker storage backend

* wip

* serialize puts

* add test

* skip tests for macos

* add test

* return error from protojson

* set data

* exclude postgres from cover tests
2022-05-25 10:23:58 -06:00
Caleb Doxsey
1669b601ea
storage: add filtering to SyncLatest (#3368)
* wip

* storage: add filtering to SyncLatest

* don't increment the record version, so intermediate changes are requested

* fix stream filter
2022-05-17 16:00:23 -06:00
Caleb Doxsey
70f5d8b173
storage: add filter expressions, upgrade go to 1.18.1 (#3365)
* storage: add filter expressions

* upgrade go
2022-05-16 20:09:50 -06:00
Caleb Doxsey
2e1366c417
databroker: fix in-memory backend deadlock (#3300) 2022-04-27 15:33:29 -04:00
Caleb Doxsey
f73c5c615f
databroker: add support for putting multiple records (#3291)
* databroker: add support for putting multiple records

* add OptimumPutRequestsFromRecords function

* replace GetAll with SyncLatest

* fix stream when there are no records
2022-04-26 16:41:38 -06:00
Caleb Doxsey
a243056cfa
Revert "databroker: add support for field masks on Put (#3210)" (#3217)
This reverts commit 2dc778035d.
2022-03-31 11:17:57 -04:00
Caleb Doxsey
2dc778035d
databroker: add support for field masks on Put (#3210)
* databroker: add support for field masks on Put

* return errors

* clean up go.mod
2022-03-29 16:36:40 -06:00
Caleb Doxsey
aaff52fc61
databroker: use contextual logging for errors, use original record type for encryption (#3096) 2022-03-04 14:40:15 -05:00
Caleb Doxsey
6af0655206
protoutil: add NewAny method for deterministic serialization (#2462) 2021-08-09 17:51:57 -06:00
Caleb Doxsey
3f8617cd93
fix go get, improve redis test (#2450)
* add pomerium.go to fix go get

* try to make redis test less flaky
2021-08-06 12:07:20 -06:00
Caleb Doxsey
5e2ca68e94
redis: increase timeout on test (#2425) 2021-08-02 17:11:33 -06:00
Caleb Doxsey
031fa1a91b
storage/inmemory: add tests for close behavior (#2336) 2021-07-06 09:36:04 -06:00
Caleb Doxsey
0bca5c9556
redis: refactor change signal test to be more deterministic (#2335) 2021-07-06 09:14:15 -06:00
Caleb Doxsey
d705b219ea
redis: enforce capacity via ZREVRANGE to avoid race (#2267) 2021-06-04 07:03:55 -06:00
Caleb Doxsey
94aa0b1a48
databroker: implement leases (#2172)
* databroker: implement leases

* return error

* handle gRPC errors
2021-05-10 13:30:25 -06:00
Caleb Doxsey
a54d43b937
registry: implement redis backend (#2179) 2021-05-10 10:33:37 -06:00
Caleb Doxsey
91c7dc742f
databroker: store server version in backend (#2142) 2021-04-28 09:12:52 -06:00
Caleb Doxsey
636b3d6846
databroker: add options for maximum capacity (#2095)
* databroker: add options

* implement redis

* add trace for enforce options
2021-04-26 17:14:54 -06:00
wasaga
e0c09a0998
log context (#2107) 2021-04-22 10:58:13 -04:00
wasaga
6aa716bc95
propagate changes back from encrypted backend (#2079) 2021-04-12 09:42:45 -04:00
Caleb Doxsey
aeb8aaf9cd
directory: remove provider from user id (#2068) 2021-04-07 15:06:08 -06:00
Caleb Doxsey
294addd857
databroker: remove unused installation id, close streams when backend is closed (#2062) 2021-04-06 13:41:19 -06:00
wasaga
8f97b0d6ee
skip redis cluster on non-linux systems (#2045) 2021-03-31 10:42:49 -04:00
Caleb Doxsey
77fe37c8c0
redis: add redis cluster support (#1992)
* redis: add redis cluster support

* redis: update docs
2021-03-17 13:48:41 -06:00
Caleb Doxsey
975b56d2d2
redis: add support for redis-sentinel (#1991)
* redis: add support for redis-sentinel

* try setting hostname

* try using container ips

* try the default network

* use container ip address
2021-03-16 16:45:08 -06:00
Caleb Doxsey
5d60cff21e
databroker: refactor databroker to sync all changes (#1879)
* refactor backend, implement encrypted store

* refactor in-memory store

* wip

* wip

* wip

* add syncer test

* fix redis expiry

* fix linting issues

* fix test by skipping non-config records

* fix backoff import

* fix init issues

* fix query

* wait for initial sync before starting directory sync

* add type to SyncLatest

* add more log messages, fix deadlock in in-memory store, always return server version from SyncLatest

* update sync types and tests

* add redis tests

* skip macos in github actions

* add comments to proto

* split getBackend into separate methods

* handle errors in initVersion

* return different error for not found vs other errors in get

* use exponential backoff for redis transaction retry

* rename raw to result

* use context instead of close channel

* store type urls as constants in databroker

* use timestampb instead of ptypes

* fix group merging not waiting

* change locked names

* update GetAll to return latest record version

* add method to grpcutil to get the type url for a protobuf type
2021-02-18 15:24:33 -07:00
Caleb Doxsey
b5d52ceb3d
redis: fix deletion versioning (#1871) 2021-02-08 16:12:13 -07:00
Travis Groth
64816720c8
internal/telemetry/metrics: update redis metrics for go-redis (#1694) 2020-12-16 14:53:39 -05:00
Caleb Doxsey
35f871ad42
fix concurrency race (#1675) 2020-12-11 14:43:26 -07:00
Caleb Doxsey
3b634de550
implement new redis storage backend with go-redis package (#1649) 2020-12-10 12:21:31 -07:00
Caleb Doxsey
1ad3646326
fix config race (#1660) 2020-12-07 10:12:40 -07:00
Caleb Doxsey
c4f675d7a7
fix panic when deleting a record twice from the inmemory data store (#1639) 2020-12-01 14:23:36 -07:00
Travis Groth
85c109114c
pkg/storage/redis: Prevent connection churn (#1603) 2020-11-19 14:15:59 -05:00