Commit graph

745 commits

Author SHA1 Message Date
bobby
fbd8c8f294
deployment: add goimports with path awareness (#1316)
Plus fix some spelling

Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-08-24 13:04:55 -07:00
Cuong Manh Le
ffaceadfdd
internal/urlutil: remove un-used constants (#1326) 2020-08-25 02:07:56 +07:00
bobby
c1b3b45d12
proxy: remove unused handlers (#1317)
proxy: remove unused handlers

authenticate: remove unused references to refresh_token

Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-08-22 10:02:12 -07:00
Caleb Doxsey
79741d5345
autocert: fix locking issue (#1310) 2020-08-20 14:08:52 -06:00
Caleb Doxsey
c4c8ef8e53
azure: support deriving credentials from client id, client secret and provider url (#1300) 2020-08-18 10:17:28 -06:00
Caleb Doxsey
a1378c81f8
cache: support databroker option changes (#1294) 2020-08-18 07:27:20 -06:00
Cuong Manh Le
a4408ab6cf internal/directory/okta: fix wrong API query filter
Okta uses space " " instead of plus sign "+" in query filter.
See https://developer.okta.com/docs/reference/api-overview/#filtering
2020-08-18 20:24:15 +07:00
bobby
8a384985f0
autocert: fix bootstrapped cache store path (#1283)
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-08-17 13:27:11 -07:00
Caleb Doxsey
6dee647a16
authorize: use atomic state for properties (#1290) 2020-08-17 14:24:06 -06:00
Caleb Doxsey
d9a224a5e8
proxy: move properties to atomically updated state (#1280)
* authenticate: remove cookie options

* authenticate: remove shared key field

* authenticate: remove shared cipher property

* authenticate: move properties to separate state struct

* proxy: allow local state to be updated on configuration changes

* fix test

* return new connection

* use warn, collapse to single line

* address concerns, fix tests
2020-08-14 11:44:58 -06:00
Cuong Manh Le
23eea09ed0 internal/directory/okta: use okta filter to get updated groups
Okta API supports filter to get updated groups only, we can adopt that
to reduce number of requests to okta API, hence reduce chance that we
reach the rate limit.

Updates #1256
2020-08-14 22:01:31 +07:00
Cuong Manh Le
d1c0ae730f internal/directory/okta: honor rate limit reset header
So we can wait until the rate limit release time to continue query okta
API.

Updates #1256
2020-08-14 22:01:31 +07:00
Cuong Manh Le
598102f587 internal/directory/okta: add limiter to query okta API
Okta only allows 100 requests per minute, so apply the default rate
limit 1 QPS for it.

Fixes #1256
2020-08-14 09:50:49 +07:00
Caleb Doxsey
045c10edc6
authenticate: support reloading IDP settings (#1273)
* identity: add name method to provider

* authenticate: support dynamically loading the provider
2020-08-13 12:14:30 -06:00
Caleb Doxsey
fbf5b403b9
config: allow dynamic configuration of cookie settings (#1267) 2020-08-13 08:11:34 -06:00
Caleb Doxsey
2afd7b6864
envoy: add support for hot-reloading bootstrap configuration (#1259)
* envoy: add support for hot-reloading bootstrap configuration

* use passed in log level

* fix unnecessary firstNonEmpty

* move process release to after new command start
2020-08-12 16:13:19 -06:00
Cuong Manh Le
82b1daae50
internal/directory/okta: increase default batch size to 200 (#1264)
See: https://developer.okta.com/docs/reference/api/groups/#list-groups-with-membership-updated-after-timestamp

Updates #1256
2020-08-13 02:27:01 +07:00
Caleb Doxsey
bd5c784670
config: validate databroker settings (#1260)
* config: validate databroker settings

* fix test
2020-08-12 11:32:34 -06:00
Cuong Manh Le
4b3e07c5f5 internal/controlplane: mocking policy name in test
We don't have to test for exact policy name, as it does not make sense
and force us to change test every new go release.
2020-08-12 22:20:50 +07:00
Caleb Doxsey
f822c9a5d2
config: allow reloading of telemetry settings (#1255)
* metrics: support dynamic configuration settings

* add test

* trace: update configuration when settings change

* config: allow logging options to be configured when settings change

* envoy: allow changing log settings

* fix unexpected doc change

* fix tests

* pick a port at random

* update based on review
2020-08-12 08:14:15 -06:00
Caleb Doxsey
1285a9d91d
databroker: add support for config settings (#1253) 2020-08-11 07:50:19 -06:00
Cuong Manh Le
277e6b56e9 internal/autocert: refactoring updateAutocert
By factor out obtain and renew certification process, return specific
error for each process if failed to contact with letsencrypt server.
2020-08-10 23:26:35 +07:00
Cuong Manh Le
3c23164347 internal/autocert: re-use cert if renewing failed but cert not expired
Fixes #1232
2020-08-10 23:26:35 +07:00
Cuong Manh Le
02edbb7748
internal/databroker: make Sync send data in smaller batches (#1226)
* internal/databroker: make Sync send data in smaller batches

GRPC streaming is better at sending multiple smaller message instead of
a big one.

Benchmark result for sending 10k messages at once vs multiple batches,
each with 100 messages:

name     old time/op  new time/op  delta
Sync-12  14.5ms ± 3%  12.4ms ± 2%  -14.40%  (p=0.000 n=10+9)

* cache: add test for databroker sync
2020-08-07 23:12:41 +07:00
Travis Groth
7a53e6bb42
proxy: add support for spdy upgrades (#1203) 2020-08-04 13:26:14 -04:00
Cuong Manh Le
4f0d6bee68
internal/urlutil: add tests for GetDomainsForURL (#1183)
Updates #959
2020-08-01 09:59:40 -07:00
Cuong Manh Le
f7ebf54305
authorize: strip port from host header if necessary (#1175)
After #1153, envoy can handle routes for `example.com` and `example.com:443`.
Authorize service should be updated to handle this case, too.

Fixes #959
2020-07-31 21:41:58 +07:00
Cuong Manh Le
bc61206b78
pkg/storage/redis: add redis TLS support (#1163)
Fixes #1156
2020-07-31 19:37:23 +07:00
Travis Groth
3c4513a91e
telmetry: add databroker storage metrics and tracing (#1161)
* telmetry: add databroker storage metrics and tracing
2020-07-30 18:19:23 -04:00
Caleb Doxsey
29fb96a955
databroker: add encryption for records (#1168) 2020-07-30 14:04:31 -06:00
Caleb Doxsey
97f85481f8
fix redirect loop, remove user/session services, remove duplicate deleted_at fields (#1162)
* fix redirect loop, remove user/session services, remove duplicate deleted_at fields

* change loop

* reuse err variable

* wrap errors, use cookie timeout

* wrap error, duplicate if
2020-07-30 09:41:57 -06:00
Cuong Manh Le
3039407597
pkg/storage/redis: add authentication support (#1159)
Fixes #1157
2020-07-29 23:08:38 +07:00
Caleb Doxsey
557aef2a33
fix databroker restart versioning, handle missing sessions (#1145)
* fix databroker restart versioning, handle missing sessions

* send empty server version to detect change

* only rebuild if there are updated records
2020-07-29 08:45:41 -06:00
Caleb Doxsey
a5e8abd6af
handle example.com and example.com:443 (#1153)
* handle example.com and example.com:443

* fix domain comparisons
2020-07-28 15:30:41 -06:00
Cuong Manh Le
a7bd2caae9
pkg/storage: introduce storage.Backend Watch method (#1135)
Currently, we're doing "sync" in databroker server. If we're going to
support multiple databroker servers instance, this mechanism won't work.

This commit moves the "sync" to storage backend, by adding new Watch
method. The Watch method will return a channel for the caller. Everytime
something happens inside the storage, we notify the caller by sending a
message to this channel.
2020-07-27 21:10:47 +07:00
Cuong Manh Le
aedfbc4c71
pkg/storage: change backend interface to return error (#1131)
Since when storage backend like redis can be fault in many cases, the
interface should return error for the caller to handle.
2020-07-24 09:02:37 +07:00
Travis Groth
90d95b8c10
Set ExtAuthz Cluster name to URL Host (#1132) 2020-07-23 14:10:16 -04:00
Cuong Manh Le
1867feb5b9
internal/databroker: handle new db error (#1129)
Since when we now support other storage, not only memory storage, we
need to handle the error when we can't connect to storage.
2020-07-23 22:48:34 +07:00
Cuong Manh Le
1640151bc1
databroker server backend config (#1127)
* config,docs: add databroker storage backend configuration

* cache: allow configuring which backend storage to use

Currently supported types are "memory", "redis".
2020-07-23 10:42:43 +07:00
Caleb Doxsey
1ad243dfd1
directory.Group entry for groups (#1118)
* store directory groups separate from directory users

* fix group lookup, azure display name

* remove fields restriction

* fix test

* also support email

* use Email as name for google'

* remove changed file

* show groups on dashboard

* fix test

* re-add accidentally removed code
2020-07-22 11:28:53 -06:00
Cuong Manh Le
489cdd8b63
internal/controlplane: using envoy strip host port matching (#1126)
* internal/controlplane: using envoy strip host port matching

With envoy 1.15.0 release, strip host port matching setting allows
incoming request with Host "example:443" will match again route with
domains match set to "example".

Not that this is not standard HTTP behavior, but it's more convenient
for users.

Fixes #959

* docs/docs: add note about enable envoy strip host port matching
2020-07-22 23:51:57 +07:00
Caleb Doxsey
504197d83b
custom rego in databroker (#1124)
* add support for sub policies

* add support for sub policies

* update authz rego policy to support sub policies
2020-07-22 10:44:05 -06:00
Cuong Manh Le
1d82be2c0e
internal/databroker: fix wrong server version init (#1125) 2020-07-22 04:32:44 +07:00
Cuong Manh Le
99785cbb5b
internal/databroker: store server version (#1121)
Storing server version when creating new server. After then, we can
retrieve the version from backend when server restart.

With storage backend which supports persistent, the server version
won't change after restarting.
2020-07-22 03:50:22 +07:00
Cuong Manh Le
9dae633fe5
internal/frontend/assets/html: make timestamp human readable (#1107)
Since we switch to use databroker, time in template is now protobuf
timestamp instead of time.Time, that causes it appears in raw form
instead of human-readable format.

Fix this by converting protobuf timestamp to time.Time in template.
There's still a breaking change, though. The time will now appears in
UTC instead of local time.

Fixes #1100
2020-07-20 13:35:57 +07:00
Caleb Doxsey
b79e73b8b8
config: add support for policies stored in the databroker (#1099)
* wip

* always use databroker config source

* add test

* valid policy, remove debug lines
2020-07-17 10:35:29 -06:00
Cuong Manh Le
821f2e9000
config: allow setting directory sync interval and timeout (#1098)
Updates #567
2020-07-17 23:11:27 +07:00
Cuong Manh Le
408f201d16
authorize/evaluator/opa/policy: fix allow rules with impersonate (#1094)
Currently, with impersonated request, the real user email/group still
has effects.

Example:

	data.route_policies as [{
	    "source": "example.com",
	    "allowed_users": ["x@example.com"]
	}] with
	input.databroker_data as {
	    "session": {
	        "user_id": "user1"
	    },
	    "user": {
	        "email": "x@example.com"
	    }
	} with
	input.http as { "url": "http://example.com" } with
	input.session as { "id": "session1", "impersonate_email": "y@example.com" }

Here user "x@example.com" is allowed, but was impersonated as
"y@example.com". As the rules indicated, the request must be denied,
because it only allows "x@example.com", not "y@example.com". The current
bug causes the request is still allowed.

To fix it, when evaluates rules for allowed email/group/domain, we must checking
that the impersonate email/groups is not set/empty.

Fixes #1091
2020-07-17 22:07:11 +07:00
Cuong Manh Le
59c17fb497
internal/directory: improve google user groups list (#1092)
Skip group without members, so it saves us time to handle group members,
and reduce the size of groups.

While at it, also querying API with the fields we need.

Fixes #567
2020-07-17 16:51:26 +07:00
Caleb Doxsey
d3a7ee38be
options refactor (#1088)
* refactor config loading

* wip

* move autocert to its own config source

* refactor options updaters

* fix stuttering

* fix autocert validate check
2020-07-16 14:30:15 -06:00