Commit graph

745 commits

Author SHA1 Message Date
Cuong Manh Le
ee1f9093ee
internal/directory: use both id and name for group (#1086)
Fixes #1085
2020-07-17 00:15:11 +07:00
Caleb Doxsey
96424dac0f
implement google cloud serverless authentication (#1080)
* add google cloud serverless support

* force ipv4 for google cloud serverless

* disable long line linting

* fix destination hostname

* add test

* add support for service accounts

* fix utc time in test
2020-07-16 08:25:14 -06:00
Travis Groth
58810cdb52
internal/directory/google: return both group e-mail and id (#1083)
* internal/directory/google: return both group e-mail and id
2020-07-16 07:19:30 -04:00
Cuong Manh Le
2f84dd2aff
Add storage backend interface (#1072)
* pkg: add storage package

Which contains storage.Backend interface to initial support for multiple
backend storage.

* pkg/storage: add inmemory storage

* internal/databroker: use storage.Backend interface

Instead of implementing multiple databroker server implementation for
each kind of storage backend, we use only one databroker server
implementation, which is supported multiple storage backends, which
satisfy storage.Backend interface.
2020-07-15 09:42:01 +07:00
Caleb Doxsey
a70254ab76
kubernetes apiserver integration (#1063)
* sessions: support bearer tokens in authorization

* wip

* remove dead code

* refactor signed jwt code

* use function

* update per comments

* fix test
2020-07-14 08:33:24 -06:00
Cuong Manh Le
d764981618
internal/controlplane: set envoy prefix rewrite if present (#1034)
While at it, also refactoring buildPolicyRoutes.

Fixes #1033
Fixes #880
2020-07-03 09:35:36 +07:00
Caleb Doxsey
fae02791f5
cryptutil: move to pkg dir, add token generator (#1029)
* cryptutil: move to pkg dir, add token generator

* add gitignored files

* add tests
2020-06-30 15:55:33 -06:00
Cuong Manh Le
f938554968
internal/controlplane: enable envoy use remote address (#1023)
Fixes #1013
2020-06-29 23:06:34 +07:00
Caleb Doxsey
091b71f12e
grpc: rename internal/grpc to pkg/grpc (#1010)
* grpc: rename internal/grpc to pkg/grpc

* don't ignore pkg dir

* remove debug line
2020-06-26 09:17:02 -06:00
Travis Groth
917d8ec61b
envoy: disable idle timeouts to controlplane (#1000)
* envoy: disable idle timeouts to controlplane to support streaming requests

* envoy: add request timeout for attack mitigation
2020-06-25 13:14:24 -04:00
bobby
dbd1eac97f
identity: support custom code flow request params (#998)
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-06-25 08:28:46 -07:00
Cuong Manh Le
963e1c015a
authorize/evaluator/opa: use route policy object instead of array index (#1001)
Make the code more readable, and slightly reduce memory alloc:

	opa test -v --bench --count 5 --format gobench

Output:

name                                       old alloc/op                     new alloc/op                     delta
DataPomeriumAuthzTestEmailAllowed                               109kB ± 0%                       108kB ± 0%  -0.89%  (p=0.008 n=5+5)
DataPomeriumAuthzTestExample                                   95.4kB ± 0%                      93.4kB ± 0%  -2.06%  (p=0.008 n=5+5)
DataPomeriumAuthzTestEmailDenied                               63.6kB ± 0%                      61.6kB ± 0%  -3.09%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPublicAllowed                              103kB ± 0%                       101kB ± 0%  -1.86%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPublicDenied                               100kB ± 0%                        98kB ± 0%  -1.64%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPomeriumAllowed                           62.6kB ± 0%                      60.7kB ± 0%  -3.14%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPomeriumDenied                            64.5kB ± 0%                      62.5kB ± 0%  -3.11%  (p=0.008 n=5+5)
DataPomeriumAuthzTestCorsPreflightAllowed                      66.7kB ± 0%                      64.5kB ± 0%  -3.33%  (p=0.008 n=5+5)
DataPomeriumAuthzTestCorsPreflightDenied                       65.8kB ± 0%                      63.3kB ± 0%  -3.92%  (p=0.008 n=5+5)
DataPomeriumAuthzTestParseUrl                                  13.8kB ± 0%                      13.8kB ± 0%    ~     (p=0.167 n=5+5)
DataPomeriumAuthzTestAllowedRouteSource                         243kB ± 0%                       243kB ± 0%    ~     (p=1.000 n=5+5)
DataPomeriumAuthzTestAllowedRoutePrefix                        80.9kB ± 0%                      80.9kB ± 0%    ~     (p=0.690 n=5+5)
DataPomeriumAuthzTestAllowedRoutePath                           108kB ± 0%                       108kB ± 0%    ~     (p=0.452 n=5+5)
DataPomeriumAuthzTestAllowedRouteRegex                         90.0kB ± 0%                      89.9kB ± 0%    ~     (p=0.095 n=5+5)

name                                       old allocs/op                    new allocs/op                    delta
DataPomeriumAuthzTestEmailAllowed                               1.76k ± 0%                       1.74k ± 0%  -1.24%  (p=0.008 n=5+5)
DataPomeriumAuthzTestExample                                    1.54k ± 0%                       1.51k ± 0%  -2.18%  (p=0.008 n=5+5)
DataPomeriumAuthzTestEmailDenied                                1.05k ± 1%                       1.01k ± 1%  -3.21%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPublicAllowed                              1.65k ± 0%                       1.63k ± 0%  -1.20%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPublicDenied                               1.61k ± 0%                       1.58k ± 0%  -1.42%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPomeriumAllowed                            1.04k ± 1%                       1.00k ± 1%  -3.27%  (p=0.008 n=5+5)
DataPomeriumAuthzTestPomeriumDenied                             1.06k ± 1%                       1.03k ± 1%  -3.19%  (p=0.008 n=5+5)
DataPomeriumAuthzTestCorsPreflightAllowed                       1.14k ± 1%                       1.09k ± 0%  -3.96%  (p=0.008 n=5+5)
DataPomeriumAuthzTestCorsPreflightDenied                        1.09k ± 1%                       1.05k ± 0%  -4.04%  (p=0.008 n=5+5)
DataPomeriumAuthzTestParseUrl                                     222 ± 0%                         222 ± 0%    ~     (all equal)
DataPomeriumAuthzTestAllowedRouteSource                         3.66k ± 0%                       3.66k ± 0%    ~     (all equal)
DataPomeriumAuthzTestAllowedRoutePrefix                         1.23k ± 0%                       1.23k ± 0%    ~     (all equal)
DataPomeriumAuthzTestAllowedRoutePath                           1.62k ± 0%                       1.62k ± 0%    ~     (all equal)
DataPomeriumAuthzTestAllowedRouteRegex                          1.36k ± 0%                       1.36k ± 0%    ~     (all equal)
2020-06-25 21:28:54 +07:00
Cuong Manh Le
505ff5cc5c
internal/sessions: handle claims "ver" field generally (#990)
"ver" field is not specified by RFC 7519, so in practice, most providers
return it as string, but okta returns it as number, which cause okta
authenticate broken.

To fix it, we handle "ver" field more generally, to allow both string and
number in json payload.
2020-06-24 22:06:17 +07:00
Cuong Manh Le
1e3c381e1e
internal/directory/okta: store directory information by user id (#991)
Same as #988
2020-06-24 21:56:51 +07:00
Cuong Manh Le
a042bb7b82
internal/directory/onelogin: store directory information by user id (#992)
Same as #988
2020-06-24 21:56:33 +07:00
Caleb Doxsey
2501463dc9
google: store directory information by user id (#988) 2020-06-23 14:41:16 -06:00
Caleb Doxsey
0d277cf662
azure: use OID for user id in session (#985) 2020-06-23 12:02:17 -06:00
Cuong Manh Le
17ba595ced
authenticate: support hot reloaded config (#984)
By implementinng OptionsUpdater interface.

Fixes #982
2020-06-24 00:18:20 +07:00
Travis Groth
eaa0c980d2
telemetry: add tracing spans to cache and databroker (#987) 2020-06-23 13:08:21 -04:00
Cuong Manh Le
fb4dfaea44
authenticate: hide impersonation form from non-admin users (#979)
Fixes #881
2020-06-23 22:09:33 +07:00
Travis Groth
88a77c42bb
cache: add client telemetry (#975) 2020-06-22 18:18:44 -04:00
Caleb Doxsey
09b8d2864f
directory: add service account struct and parsing method (#971) 2020-06-22 15:04:20 -06:00
bobby
f94f45d9a2
controlplane: add robots route (#966)
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-06-22 11:48:59 -07:00
Caleb Doxsey
8362f18355
authenticate: move impersonate from proxy to authenticate (#965) 2020-06-22 11:58:27 -06:00
Cuong Manh Le
99142b7293
authenticate: revoke current session oauth token before sign out (#964)
authenticate: revoke current session oauth token before sign out

After #926, we don't revoke access token before sign out anymore. It
causes sign out can not work, because right after user click on sign out
button, we redirect user to idp provider authenticate page with a valid
access token, so user is logged in immediately again.

To fix it, just revoke the access token before sign out.
2020-06-23 00:55:55 +07:00
Caleb Doxsey
dbf020a532
github: implement github directory provider (#963)
* github: implement github directory provider

* fix test
2020-06-22 11:33:37 -06:00
Caleb Doxsey
b3ccdfe00f
idp: delete sessions on refresh error, handle zero times in oauth/id tokens for refresh (#961) 2020-06-22 08:49:28 -06:00
bobby
452c9be06d
cache: remove unused metrics and options (#957)
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-06-22 06:59:04 -07:00
Caleb Doxsey
f7760c413e
directory: generate user/directory.User ID in a consistent way (#944) 2020-06-22 07:42:57 -06:00
Cuong Manh Le
8d0deb0732
config: add PassIdentityHeaders option (#903)
Currently, user's identity headers are always inserted to downstream
request. For privacy reason, it would be better to not insert these
headers by default, and let user chose whether to include these headers
per=policy basis.

Fixes #702
2020-06-22 10:29:44 +07:00
Caleb Doxsey
fbce3dd359
idp: set github timestamps (#943) 2020-06-21 15:50:56 -06:00
Caleb Doxsey
dbd7f55b20
feature/databroker: user data and session refactor project (#926)
* databroker: add databroker, identity manager, update cache (#864)

* databroker: add databroker, identity manager, update cache

* fix cache tests

* directory service (#885)

* directory: add google and okta

* add onelogin

* add directory provider

* initialize before sync, upate google provider, remove dead code

* add azure provider

* fix azure provider

* fix gitlab

* add gitlab test, fix azure test

* hook up okta

* remove dead code

* fix tests

* fix flaky test

* authorize: use databroker data for rego policy (#904)

* wip

* add directory provider

* initialize before sync, upate google provider, remove dead code

* fix flaky test

* update authorize to use databroker data

* implement signed jwt

* wait for session and user to appear

* fix test

* directory service (#885)

* directory: add google and okta

* add onelogin

* add directory provider

* initialize before sync, upate google provider, remove dead code

* add azure provider

* fix azure provider

* fix gitlab

* add gitlab test, fix azure test

* hook up okta

* remove dead code

* fix tests

* fix flaky test

* remove log line

* only redirect when no session id exists

* prepare rego query as part of create

* return on ctx done

* retry on disconnect for sync

* move jwt signing

* use !=

* use parent ctx for wait

* remove session state, remove logs

* rename function

* add log message

* pre-allocate slice

* use errgroup

* return nil on eof for sync

* move check

* disable timeout on gRPC requests in envoy

* fix gitlab test

* use v4 backoff

* authenticate: databroker changes (#914)

* wip

* add directory provider

* initialize before sync, upate google provider, remove dead code

* fix flaky test

* update authorize to use databroker data

* implement signed jwt

* wait for session and user to appear

* fix test

* directory service (#885)

* directory: add google and okta

* add onelogin

* add directory provider

* initialize before sync, upate google provider, remove dead code

* add azure provider

* fix azure provider

* fix gitlab

* add gitlab test, fix azure test

* hook up okta

* remove dead code

* fix tests

* fix flaky test

* remove log line

* only redirect when no session id exists

* prepare rego query as part of create

* return on ctx done

* retry on disconnect for sync

* move jwt signing

* use !=

* use parent ctx for wait

* remove session state, remove logs

* rename function

* add log message

* pre-allocate slice

* use errgroup

* return nil on eof for sync

* move check

* disable timeout on gRPC requests in envoy

* fix dashboard

* delete session on logout

* permanently delete sessions once they are marked as deleted

* remove permanent delete

* fix tests

* remove groups and refresh test

* databroker: remove dead code, rename cache url, move dashboard (#925)

* wip

* add directory provider

* initialize before sync, upate google provider, remove dead code

* fix flaky test

* update authorize to use databroker data

* implement signed jwt

* wait for session and user to appear

* fix test

* directory service (#885)

* directory: add google and okta

* add onelogin

* add directory provider

* initialize before sync, upate google provider, remove dead code

* add azure provider

* fix azure provider

* fix gitlab

* add gitlab test, fix azure test

* hook up okta

* remove dead code

* fix tests

* fix flaky test

* remove log line

* only redirect when no session id exists

* prepare rego query as part of create

* return on ctx done

* retry on disconnect for sync

* move jwt signing

* use !=

* use parent ctx for wait

* remove session state, remove logs

* rename function

* add log message

* pre-allocate slice

* use errgroup

* return nil on eof for sync

* move check

* disable timeout on gRPC requests in envoy

* fix dashboard

* delete session on logout

* permanently delete sessions once they are marked as deleted

* remove permanent delete

* fix tests

* remove cache service

* remove kv

* remove refresh docs

* remove obsolete cache docs

* add databroker url option

* cache: use memberlist to detect multiple instances

* add databroker service url

* remove cache service

* remove kv

* remove refresh docs

* remove obsolete cache docs

* add databroker url option

* cache: use memberlist to detect multiple instances

* add databroker service url

* wip

* remove groups and refresh test

* fix redirect, signout

* remove databroker client from proxy

* remove unused method

* remove user dashboard test

* handle missing session ids

* session: reject sessions with no id

* sessions: invalidate old sessions via databroker server version (#930)

* session: add a version field tied to the databroker server version that can be used to invalidate sessions

* fix tests

* add log

* authenticate: create user record immediately, call "get" directly in authorize (#931)
2020-06-19 07:52:44 -06:00
Cuong Manh Le
39cdb31170
internal/envoy: improve handleLogs (#929)
The log line has a well defined structure that we can process by simple
string manipulation, instead of relying on regex.

name            old time/op    new time/op    delta
_handleLogs-12    17.3µs ±23%     1.1µs ±11%  -93.81%  (p=0.000 n=10+10)

name            old alloc/op   new alloc/op   delta
_handleLogs-12    20.1kB ± 0%     4.1kB ± 0%  -79.59%  (p=0.002 n=8+10)

name            old allocs/op  new allocs/op  delta
_handleLogs-12       141 ± 0%         1 ± 0%  -99.29%  (p=0.000 n=10+10)
2020-06-19 09:14:10 +07:00
Cuong Manh Le
9df4dc4aca
internal/envoy: fix handleLogs causes envoy hang forever (#927)
handleLogs uses bufio scanner to process log output from envoy. When in
debug mode, envoy produces very long log line, causing the scanner
fails, handleLogs stop processing log. But envoy continue writing to its
stdout, which is now not consumed by any process, envoy hangs there
forever.

Fixing this by switching to use bufio.Reader instead. This is also the
real fix for failed integration test, which is interpreted wrongly by
me in #910.
2020-06-19 00:03:42 +07:00
Cuong Manh Le
f62bb686d8
internal/controlplane: make sure options.Headers are set for response (#907)
When switching to envoy, we forgot to adopt the middleware to set
response headers with options.Headers, which causes HSTS header is
missing in v0.9.0 release.

Fixes #901
2020-06-17 00:56:01 +07:00
Travis Groth
ee2170f5f5
config: add a consistent route ID (#905) 2020-06-16 09:20:18 -04:00
Cuong Manh Le
34d06e521d
internal/telemetry/metrics: document concurrently using (#891)
Document that metricRegistry is not safe for concurrently use. While at
it, remove t.Parallel() in tests which use metricRegistry, which causes
data race, caught by:

	go test -race ./internal/telemetry/metrics
2020-06-15 23:08:03 +07:00
Cuong Manh Le
896467c4bf
internal/cmd/pomerium: fix data race in handling context (#890)
Caught by:

	go test -race ./internal/cmd/pomerium

The ctx in Run is both read (in handle signal goroutine) and write
(when passing to errgroup context in Run), causes data race.

Fixing it, by passing the ctx to goroutine via argument instead of
accessing it directly.
2020-06-15 22:38:45 +07:00
Bobby DeSimone
200bc7e836
controlplane: use previous preferred cipher suite (#889)
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-06-14 17:53:18 -07:00
Bobby DeSimone
79d793d122
controlplane: fix missing full cert chain (#888)
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-06-14 17:53:02 -07:00
Bobby DeSimone
3fbcb8ff13
frontend: fix logo fill on chrome (#893)
- on error, if reason is empty use the status text of the http status code

Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-06-13 13:55:01 -07:00
Bobby DeSimone
b00acad517
internal/controlplane: set minimum tls version (#854)
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-06-10 09:08:05 -07:00
Caleb Doxsey
fe2369400c
proxy: only set validation context if trusted_ca is used (#863)
* proxy: only set validation context if trusted_ca is used

* fix test
2020-06-09 13:45:03 -06:00
Cuong Manh Le
9e711b4612
internal/httputil: add HTTPStatsRoundTripper to DefaultClient (#828) 2020-06-08 14:34:32 -04:00
Cuong Manh Le
4d5edb0d64
Feature/remove request headers (#822)
* config: add RemoveRequestHeaders

Currently, we have "set_request_headers" config, which reflects envoy
route.Route.RequestHeadersToAdd. This commit add new config
"remove_request_headers", which reflects envoy RequestHeadersToRemove.

This is also a preparation for future PRs to implement disable user
identity in request headers feature.

* integration: add test for remove_request_headers
* docs: add documentation/changelog for remove_request_headers
2020-06-03 07:46:51 -07:00
Caleb Doxsey
b80a419699
xds: use ipv4 address when ipv6 is disabled (#823) 2020-06-02 13:05:44 -06:00
Caleb Doxsey
fca17d365a
xds: force ipv4 for localhost to workaround ipv6 issue in docker compose (#819) 2020-06-01 08:58:28 -06:00
Travis Groth
f97341dcb8
Fix autocache telemetry labels (#805) 2020-05-29 17:47:45 -04:00
Travis Groth
06e3f5def5
Fix missing/incorrect grpc labels (#804) 2020-05-29 15:57:58 -04:00
Travis Groth
6761cc7a14
telemetry: service label updates (#802) 2020-05-29 15:16:22 -04:00