Commit graph

618 commits

Author SHA1 Message Date
Cuong Manh Le
bc61206b78
pkg/storage/redis: add redis TLS support (#1163)
Fixes #1156
2020-07-31 19:37:23 +07:00
Travis Groth
3c4513a91e
telmetry: add databroker storage metrics and tracing (#1161)
* telmetry: add databroker storage metrics and tracing
2020-07-30 18:19:23 -04:00
Caleb Doxsey
29fb96a955
databroker: add encryption for records (#1168) 2020-07-30 14:04:31 -06:00
Caleb Doxsey
97f85481f8
fix redirect loop, remove user/session services, remove duplicate deleted_at fields (#1162)
* fix redirect loop, remove user/session services, remove duplicate deleted_at fields

* change loop

* reuse err variable

* wrap errors, use cookie timeout

* wrap error, duplicate if
2020-07-30 09:41:57 -06:00
Cuong Manh Le
3039407597
pkg/storage/redis: add authentication support (#1159)
Fixes #1157
2020-07-29 23:08:38 +07:00
Caleb Doxsey
557aef2a33
fix databroker restart versioning, handle missing sessions (#1145)
* fix databroker restart versioning, handle missing sessions

* send empty server version to detect change

* only rebuild if there are updated records
2020-07-29 08:45:41 -06:00
Caleb Doxsey
a5e8abd6af
handle example.com and example.com:443 (#1153)
* handle example.com and example.com:443

* fix domain comparisons
2020-07-28 15:30:41 -06:00
Cuong Manh Le
a7bd2caae9
pkg/storage: introduce storage.Backend Watch method (#1135)
Currently, we're doing "sync" in databroker server. If we're going to
support multiple databroker servers instance, this mechanism won't work.

This commit moves the "sync" to storage backend, by adding new Watch
method. The Watch method will return a channel for the caller. Everytime
something happens inside the storage, we notify the caller by sending a
message to this channel.
2020-07-27 21:10:47 +07:00
Cuong Manh Le
aedfbc4c71
pkg/storage: change backend interface to return error (#1131)
Since when storage backend like redis can be fault in many cases, the
interface should return error for the caller to handle.
2020-07-24 09:02:37 +07:00
Travis Groth
90d95b8c10
Set ExtAuthz Cluster name to URL Host (#1132) 2020-07-23 14:10:16 -04:00
Cuong Manh Le
1867feb5b9
internal/databroker: handle new db error (#1129)
Since when we now support other storage, not only memory storage, we
need to handle the error when we can't connect to storage.
2020-07-23 22:48:34 +07:00
Cuong Manh Le
1640151bc1
databroker server backend config (#1127)
* config,docs: add databroker storage backend configuration

* cache: allow configuring which backend storage to use

Currently supported types are "memory", "redis".
2020-07-23 10:42:43 +07:00
Caleb Doxsey
1ad243dfd1
directory.Group entry for groups (#1118)
* store directory groups separate from directory users

* fix group lookup, azure display name

* remove fields restriction

* fix test

* also support email

* use Email as name for google'

* remove changed file

* show groups on dashboard

* fix test

* re-add accidentally removed code
2020-07-22 11:28:53 -06:00
Cuong Manh Le
489cdd8b63
internal/controlplane: using envoy strip host port matching (#1126)
* internal/controlplane: using envoy strip host port matching

With envoy 1.15.0 release, strip host port matching setting allows
incoming request with Host "example:443" will match again route with
domains match set to "example".

Not that this is not standard HTTP behavior, but it's more convenient
for users.

Fixes #959

* docs/docs: add note about enable envoy strip host port matching
2020-07-22 23:51:57 +07:00
Caleb Doxsey
504197d83b
custom rego in databroker (#1124)
* add support for sub policies

* add support for sub policies

* update authz rego policy to support sub policies
2020-07-22 10:44:05 -06:00
Cuong Manh Le
1d82be2c0e
internal/databroker: fix wrong server version init (#1125) 2020-07-22 04:32:44 +07:00
Cuong Manh Le
99785cbb5b
internal/databroker: store server version (#1121)
Storing server version when creating new server. After then, we can
retrieve the version from backend when server restart.

With storage backend which supports persistent, the server version
won't change after restarting.
2020-07-22 03:50:22 +07:00
Cuong Manh Le
9dae633fe5
internal/frontend/assets/html: make timestamp human readable (#1107)
Since we switch to use databroker, time in template is now protobuf
timestamp instead of time.Time, that causes it appears in raw form
instead of human-readable format.

Fix this by converting protobuf timestamp to time.Time in template.
There's still a breaking change, though. The time will now appears in
UTC instead of local time.

Fixes #1100
2020-07-20 13:35:57 +07:00
Caleb Doxsey
b79e73b8b8
config: add support for policies stored in the databroker (#1099)
* wip

* always use databroker config source

* add test

* valid policy, remove debug lines
2020-07-17 10:35:29 -06:00
Cuong Manh Le
821f2e9000
config: allow setting directory sync interval and timeout (#1098)
Updates #567
2020-07-17 23:11:27 +07:00
Cuong Manh Le
408f201d16
authorize/evaluator/opa/policy: fix allow rules with impersonate (#1094)
Currently, with impersonated request, the real user email/group still
has effects.

Example:

	data.route_policies as [{
	    "source": "example.com",
	    "allowed_users": ["x@example.com"]
	}] with
	input.databroker_data as {
	    "session": {
	        "user_id": "user1"
	    },
	    "user": {
	        "email": "x@example.com"
	    }
	} with
	input.http as { "url": "http://example.com" } with
	input.session as { "id": "session1", "impersonate_email": "y@example.com" }

Here user "x@example.com" is allowed, but was impersonated as
"y@example.com". As the rules indicated, the request must be denied,
because it only allows "x@example.com", not "y@example.com". The current
bug causes the request is still allowed.

To fix it, when evaluates rules for allowed email/group/domain, we must checking
that the impersonate email/groups is not set/empty.

Fixes #1091
2020-07-17 22:07:11 +07:00
Cuong Manh Le
59c17fb497
internal/directory: improve google user groups list (#1092)
Skip group without members, so it saves us time to handle group members,
and reduce the size of groups.

While at it, also querying API with the fields we need.

Fixes #567
2020-07-17 16:51:26 +07:00
Caleb Doxsey
d3a7ee38be
options refactor (#1088)
* refactor config loading

* wip

* move autocert to its own config source

* refactor options updaters

* fix stuttering

* fix autocert validate check
2020-07-16 14:30:15 -06:00
Cuong Manh Le
ee1f9093ee
internal/directory: use both id and name for group (#1086)
Fixes #1085
2020-07-17 00:15:11 +07:00
Caleb Doxsey
96424dac0f
implement google cloud serverless authentication (#1080)
* add google cloud serverless support

* force ipv4 for google cloud serverless

* disable long line linting

* fix destination hostname

* add test

* add support for service accounts

* fix utc time in test
2020-07-16 08:25:14 -06:00
Travis Groth
58810cdb52
internal/directory/google: return both group e-mail and id (#1083)
* internal/directory/google: return both group e-mail and id
2020-07-16 07:19:30 -04:00
Cuong Manh Le
2f84dd2aff
Add storage backend interface (#1072)
* pkg: add storage package

Which contains storage.Backend interface to initial support for multiple
backend storage.

* pkg/storage: add inmemory storage

* internal/databroker: use storage.Backend interface

Instead of implementing multiple databroker server implementation for
each kind of storage backend, we use only one databroker server
implementation, which is supported multiple storage backends, which
satisfy storage.Backend interface.
2020-07-15 09:42:01 +07:00
Caleb Doxsey
a70254ab76
kubernetes apiserver integration (#1063)
* sessions: support bearer tokens in authorization

* wip

* remove dead code

* refactor signed jwt code

* use function

* update per comments

* fix test
2020-07-14 08:33:24 -06:00
Cuong Manh Le
d764981618
internal/controlplane: set envoy prefix rewrite if present (#1034)
While at it, also refactoring buildPolicyRoutes.

Fixes #1033
Fixes #880
2020-07-03 09:35:36 +07:00
Caleb Doxsey
fae02791f5
cryptutil: move to pkg dir, add token generator (#1029)
* cryptutil: move to pkg dir, add token generator

* add gitignored files

* add tests
2020-06-30 15:55:33 -06:00
Cuong Manh Le
f938554968
internal/controlplane: enable envoy use remote address (#1023)
Fixes #1013
2020-06-29 23:06:34 +07:00
Caleb Doxsey
091b71f12e
grpc: rename internal/grpc to pkg/grpc (#1010)
* grpc: rename internal/grpc to pkg/grpc

* don't ignore pkg dir

* remove debug line
2020-06-26 09:17:02 -06:00
Travis Groth
917d8ec61b
envoy: disable idle timeouts to controlplane (#1000)
* envoy: disable idle timeouts to controlplane to support streaming requests

* envoy: add request timeout for attack mitigation
2020-06-25 13:14:24 -04:00
bobby
dbd1eac97f
identity: support custom code flow request params (#998)
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-06-25 08:28:46 -07:00
Cuong Manh Le
963e1c015a
authorize/evaluator/opa: use route policy object instead of array index (#1001)
Make the code more readable, and slightly reduce memory alloc:

	opa test -v --bench --count 5 --format gobench

Output:

name                                       old alloc/op                     new alloc/op                     delta
DataPomeriumAuthzTestEmailAllowed                               109kB ± 0%                       108kB ± 0%  -0.89%  (p=0.008 n=5+5)
DataPomeriumAuthzTestExample                                   95.4kB ± 0%                      93.4kB ± 0%  -2.06%  (p=0.008 n=5+5)
DataPomeriumAuthzTestEmailDenied                               63.6kB ± 0%                      61.6kB ± 0%  -3.09%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPublicAllowed                              103kB ± 0%                       101kB ± 0%  -1.86%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPublicDenied                               100kB ± 0%                        98kB ± 0%  -1.64%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPomeriumAllowed                           62.6kB ± 0%                      60.7kB ± 0%  -3.14%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPomeriumDenied                            64.5kB ± 0%                      62.5kB ± 0%  -3.11%  (p=0.008 n=5+5)
DataPomeriumAuthzTestCorsPreflightAllowed                      66.7kB ± 0%                      64.5kB ± 0%  -3.33%  (p=0.008 n=5+5)
DataPomeriumAuthzTestCorsPreflightDenied                       65.8kB ± 0%                      63.3kB ± 0%  -3.92%  (p=0.008 n=5+5)
DataPomeriumAuthzTestParseUrl                                  13.8kB ± 0%                      13.8kB ± 0%    ~     (p=0.167 n=5+5)
DataPomeriumAuthzTestAllowedRouteSource                         243kB ± 0%                       243kB ± 0%    ~     (p=1.000 n=5+5)
DataPomeriumAuthzTestAllowedRoutePrefix                        80.9kB ± 0%                      80.9kB ± 0%    ~     (p=0.690 n=5+5)
DataPomeriumAuthzTestAllowedRoutePath                           108kB ± 0%                       108kB ± 0%    ~     (p=0.452 n=5+5)
DataPomeriumAuthzTestAllowedRouteRegex                         90.0kB ± 0%                      89.9kB ± 0%    ~     (p=0.095 n=5+5)

name                                       old allocs/op                    new allocs/op                    delta
DataPomeriumAuthzTestEmailAllowed                               1.76k ± 0%                       1.74k ± 0%  -1.24%  (p=0.008 n=5+5)
DataPomeriumAuthzTestExample                                    1.54k ± 0%                       1.51k ± 0%  -2.18%  (p=0.008 n=5+5)
DataPomeriumAuthzTestEmailDenied                                1.05k ± 1%                       1.01k ± 1%  -3.21%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPublicAllowed                              1.65k ± 0%                       1.63k ± 0%  -1.20%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPublicDenied                               1.61k ± 0%                       1.58k ± 0%  -1.42%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPomeriumAllowed                            1.04k ± 1%                       1.00k ± 1%  -3.27%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPomeriumDenied                             1.06k ± 1%                       1.03k ± 1%  -3.19%  (p=0.008 n=5+5)
DataPomeriumAuthzTestCorsPreflightAllowed                       1.14k ± 1%                       1.09k ± 0%  -3.96%  (p=0.008 n=5+5)
DataPomeriumAuthzTestCorsPreflightDenied                        1.09k ± 1%                       1.05k ± 0%  -4.04%  (p=0.008 n=5+5)
DataPomeriumAuthzTestParseUrl                                     222 ± 0%                         222 ± 0%    ~     (all equal)
DataPomeriumAuthzTestAllowedRouteSource                         3.66k ± 0%                       3.66k ± 0%    ~     (all equal)
DataPomeriumAuthzTestAllowedRoutePrefix                         1.23k ± 0%                       1.23k ± 0%    ~     (all equal)
DataPomeriumAuthzTestAllowedRoutePath                           1.62k ± 0%                       1.62k ± 0%    ~     (all equal)
DataPomeriumAuthzTestAllowedRouteRegex                          1.36k ± 0%                       1.36k ± 0%    ~     (all equal)
2020-06-25 21:28:54 +07:00
Cuong Manh Le
505ff5cc5c
internal/sessions: handle claims "ver" field generally (#990)
"ver" field is not specified by RFC 7519, so in practice, most providers
return it as string, but okta returns it as number, which cause okta
authenticate broken.

To fix it, we handle "ver" field more generally, to allow both string and
number in json payload.
2020-06-24 22:06:17 +07:00
Cuong Manh Le
1e3c381e1e
internal/directory/okta: store directory information by user id (#991)
Same as #988
2020-06-24 21:56:51 +07:00
Cuong Manh Le
a042bb7b82
internal/directory/onelogin: store directory information by user id (#992)
Same as #988
2020-06-24 21:56:33 +07:00
Caleb Doxsey
2501463dc9
google: store directory information by user id (#988) 2020-06-23 14:41:16 -06:00
Caleb Doxsey
0d277cf662
azure: use OID for user id in session (#985) 2020-06-23 12:02:17 -06:00
Cuong Manh Le
17ba595ced
authenticate: support hot reloaded config (#984)
By implementinng OptionsUpdater interface.

Fixes #982
2020-06-24 00:18:20 +07:00
Travis Groth
eaa0c980d2
telemetry: add tracing spans to cache and databroker (#987) 2020-06-23 13:08:21 -04:00
Cuong Manh Le
fb4dfaea44
authenticate: hide impersonation form from non-admin users (#979)
Fixes #881
2020-06-23 22:09:33 +07:00
Travis Groth
88a77c42bb
cache: add client telemetry (#975) 2020-06-22 18:18:44 -04:00
Caleb Doxsey
09b8d2864f
directory: add service account struct and parsing method (#971) 2020-06-22 15:04:20 -06:00
bobby
f94f45d9a2
controlplane: add robots route (#966)
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-06-22 11:48:59 -07:00
Caleb Doxsey
8362f18355
authenticate: move impersonate from proxy to authenticate (#965) 2020-06-22 11:58:27 -06:00
Cuong Manh Le
99142b7293
authenticate: revoke current session oauth token before sign out (#964)
authenticate: revoke current session oauth token before sign out

After #926, we don't revoke access token before sign out anymore. It
causes sign out can not work, because right after user click on sign out
button, we redirect user to idp provider authenticate page with a valid
access token, so user is logged in immediately again.

To fix it, just revoke the access token before sign out.
2020-06-23 00:55:55 +07:00
Caleb Doxsey
dbf020a532
github: implement github directory provider (#963)
* github: implement github directory provider

* fix test
2020-06-22 11:33:37 -06:00
Caleb Doxsey
b3ccdfe00f
idp: delete sessions on refresh error, handle zero times in oauth/id tokens for refresh (#961) 2020-06-22 08:49:28 -06:00