* replace GetAllPages with InitialSync, improve merge performance
* fmt proto
* add test for base64 function
* add sync test
* go mod tidy
Co-authored-by: Bobby DeSimone <bobbydesimone@gmail.com>
* internal/databroker: make Sync send data in smaller batches
GRPC streaming is better at sending multiple smaller message instead of
a big one.
Benchmark result for sending 10k messages at once vs multiple batches,
each with 100 messages:
name old time/op new time/op delta
Sync-12 14.5ms ± 3% 12.4ms ± 2% -14.40% (p=0.000 n=10+9)
* cache: add test for databroker sync
Currently, we're doing "sync" in databroker server. If we're going to
support multiple databroker servers instance, this mechanism won't work.
This commit moves the "sync" to storage backend, by adding new Watch
method. The Watch method will return a channel for the caller. Everytime
something happens inside the storage, we notify the caller by sending a
message to this channel.
* config,docs: add databroker storage backend configuration
* cache: allow configuring which backend storage to use
Currently supported types are "memory", "redis".
Storing server version when creating new server. After then, we can
retrieve the version from backend when server restart.
With storage backend which supports persistent, the server version
won't change after restarting.
* pkg: add storage package
Which contains storage.Backend interface to initial support for multiple
backend storage.
* pkg/storage: add inmemory storage
* internal/databroker: use storage.Backend interface
Instead of implementing multiple databroker server implementation for
each kind of storage backend, we use only one databroker server
implementation, which is supported multiple storage backends, which
satisfy storage.Backend interface.
2020-07-15 09:42:01 +07:00
Renamed from internal/databroker/memory/server.go (Browse further)
