* pkg/storage/redis: use SANs cert
Since go1.15, X.509 CommonName is deprecated, switch to a SANs
certificate for test redis TLS.
While at it, add instruction to genearte cert and build test image.
See: https://golang.org/doc/go1.15#commonname
* config: do not test for exact route id
Different go version can genearte different route id, due to the fact
that we are relying on xxhash.
* internal/controlplane: mocking policy name in test
We don't have to test for exact policy name, as it does not make sense
and force us to change test every new go release.
Co-authored-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
There are two data race in current code:
- Call to close pub sub conn and renew pub sub conn
- Call to close notify channel and send data to it
Fixing them by:
- Moving pub sub conn creation/renew in the doNotifyLoop
- Add a lock to guard before close/send data to channel, and also add
another check for context was done at the beginning of notify loop.
Verifying by running:
for _ in {1..100}; do
go test -race -count=1 ./pkg/storage/redis/...
done
with no failure.
Instead, we run the loop in goroutine, and when context was done,
closing the underlying connection of PubSubConn, so the Receive will
return.
Fixes#1154
Currently, we're doing "sync" in databroker server. If we're going to
support multiple databroker servers instance, this mechanism won't work.
This commit moves the "sync" to storage backend, by adding new Watch
method. The Watch method will return a channel for the caller. Everytime
something happens inside the storage, we notify the caller by sending a
message to this channel.
* config,docs: add databroker storage backend configuration
* cache: allow configuring which backend storage to use
Currently supported types are "memory", "redis".
* pkg/storage: add redis storage backend
* pkg/storage/redis: set record create time correctly
* pkg/storage/redis: add docs
* pkg/storage/redis: run test with redis tag only
* pkg/storage/redis: use localhost
* pkg/storage/redis: use 127.0.0.1
* pkg/storage/redis: honor REDIS_URL env
* .github/workflows: add missing config for redis service
* .github/workflows: map redis ports to host
* pkg/storage/redis: use proto marshaler instead of json one
* pkg/storage/redis: use better implementation
By using redis supported datastructure:
- Hash for storing record
- Sorted set for storing by version
- Set for storing deleted ids
List operation will be now performed in O(log(N)+M) instead of O(N) like
previous implementation.
* pkg/storage/redis: add tx to wrap redis transaction
* pkg/storage/redis: set record type in New
* pkg/storage/redis: make sure tx commands appear in right order
* pkg/storage/redis: make deletePermanentAfter as argument
* pkg/storage/redis: make sure version is incremented when deleting
* pkg/storage/redis: fix linter
* pkg/storage/redis: fix cmd construction
* pkg: add storage package
Which contains storage.Backend interface to initial support for multiple
backend storage.
* pkg/storage: add inmemory storage
* internal/databroker: use storage.Backend interface
Instead of implementing multiple databroker server implementation for
each kind of storage backend, we use only one databroker server
implementation, which is supported multiple storage backends, which
satisfy storage.Backend interface.
