Commit graph

876 commits

Author SHA1 Message Date
Travis Groth
3c4513a91e
telmetry: add databroker storage metrics and tracing (#1161)
* telmetry: add databroker storage metrics and tracing
2020-07-30 18:19:23 -04:00
Caleb Doxsey
29fb96a955
databroker: add encryption for records (#1168) 2020-07-30 14:04:31 -06:00
bobby
8cae3f27bb
docs: refactor sections, consolidate examples (#1164) 2020-07-30 11:02:14 -07:00
Diep Pham
f41eeaf138
docs: add recipe for TiddlyWiki on Node.js (#1143) 2020-07-30 08:59:04 -07:00
Caleb Doxsey
97f85481f8
fix redirect loop, remove user/session services, remove duplicate deleted_at fields (#1162)
* fix redirect loop, remove user/session services, remove duplicate deleted_at fields

* change loop

* reuse err variable

* wrap errors, use cookie timeout

* wrap error, duplicate if
2020-07-30 09:41:57 -06:00
Caleb Doxsey
714363fb07
grpcutil: add functions for JWTs in gRPC metadata (#1165) 2020-07-30 09:40:55 -06:00
Cuong Manh Le
3039407597
pkg/storage/redis: add authentication support (#1159)
Fixes #1157
2020-07-29 23:08:38 +07:00
Cuong Manh Le
05545b3e1d
pkg/storage/redis: do not use timeout to signal redis conn to stop (#1155)
Instead, we run the loop in goroutine, and when context was done,
closing the underlying connection of PubSubConn, so the Receive will
return.

Fixes #1154
2020-07-29 21:48:50 +07:00
Caleb Doxsey
557aef2a33
fix databroker restart versioning, handle missing sessions (#1145)
* fix databroker restart versioning, handle missing sessions

* send empty server version to detect change

* only rebuild if there are updated records
2020-07-29 08:45:41 -06:00
Caleb Doxsey
a5e8abd6af
handle example.com and example.com:443 (#1153)
* handle example.com and example.com:443

* fix domain comparisons
2020-07-28 15:30:41 -06:00
Travis Groth
996f0251b2
Add kubectl config commands (#1152) 2020-07-28 17:14:22 -04:00
Cuong Manh Le
fd544b7072
authenticate: fix wrong condition checking in VerifySession (#1146)
The code checks session client not nil, but using databroker client in
the body instead.
2020-07-28 21:18:39 +07:00
Cuong Manh Le
bec908b9af
docs/.vuepress: fix missing local-oidc recipes section (#1147) 2020-07-28 19:51:17 +07:00
Miguel
72b6347886
docs: Add required in cookie_secret (#1142) 2020-07-27 22:59:54 +07:00
Cuong Manh Le
a7bd2caae9
pkg/storage: introduce storage.Backend Watch method (#1135)
Currently, we're doing "sync" in databroker server. If we're going to
support multiple databroker servers instance, this mechanism won't work.

This commit moves the "sync" to storage backend, by adding new Watch
method. The Watch method will return a channel for the caller. Everytime
something happens inside the storage, we notify the caller by sending a
message to this channel.
2020-07-27 21:10:47 +07:00
Renovate Bot
d9711c8055 Update module google/go-cmp to v0.5.1 2020-07-27 05:28:15 +00:00
Renovate Bot
94c4db3eb2 Update google.golang.org/genproto commit hash to da3ae01 2020-07-27 04:56:06 +00:00
Cuong Manh Le
a8fd23a2be
pkg/storage/redis: move last version to redis (#1134)
So we can support multiple databroker servers, we can't do it if we
store last version inside Server struct.
2020-07-24 11:27:11 +07:00
Cuong Manh Le
aedfbc4c71
pkg/storage: change backend interface to return error (#1131)
Since when storage backend like redis can be fault in many cases, the
interface should return error for the caller to handle.
2020-07-24 09:02:37 +07:00
Travis Groth
90d95b8c10
Set ExtAuthz Cluster name to URL Host (#1132) 2020-07-23 14:10:16 -04:00
Cuong Manh Le
1867feb5b9
internal/databroker: handle new db error (#1129)
Since when we now support other storage, not only memory storage, we
need to handle the error when we can't connect to storage.
2020-07-23 22:48:34 +07:00
Cuong Manh Le
1640151bc1
databroker server backend config (#1127)
* config,docs: add databroker storage backend configuration

* cache: allow configuring which backend storage to use

Currently supported types are "memory", "redis".
2020-07-23 10:42:43 +07:00
Travis Groth
c9182f757e
docs: Fix incorrect example middleware (#1128)
* Fix incorrect example middleware
2020-07-22 14:33:45 -04:00
Caleb Doxsey
1ad243dfd1
directory.Group entry for groups (#1118)
* store directory groups separate from directory users

* fix group lookup, azure display name

* remove fields restriction

* fix test

* also support email

* use Email as name for google'

* remove changed file

* show groups on dashboard

* fix test

* re-add accidentally removed code
2020-07-22 11:28:53 -06:00
Cuong Manh Le
489cdd8b63
internal/controlplane: using envoy strip host port matching (#1126)
* internal/controlplane: using envoy strip host port matching

With envoy 1.15.0 release, strip host port matching setting allows
incoming request with Host "example:443" will match again route with
domains match set to "example".

Not that this is not standard HTTP behavior, but it's more convenient
for users.

Fixes #959

* docs/docs: add note about enable envoy strip host port matching
2020-07-22 23:51:57 +07:00
Caleb Doxsey
504197d83b
custom rego in databroker (#1124)
* add support for sub policies

* add support for sub policies

* update authz rego policy to support sub policies
2020-07-22 10:44:05 -06:00
Cuong Manh Le
1d82be2c0e
internal/databroker: fix wrong server version init (#1125) 2020-07-22 04:32:44 +07:00
Cuong Manh Le
99785cbb5b
internal/databroker: store server version (#1121)
Storing server version when creating new server. After then, we can
retrieve the version from backend when server restart.

With storage backend which supports persistent, the server version
won't change after restarting.
2020-07-22 03:50:22 +07:00
Cuong Manh Le
26f099b49d
redis storage backend (#1082)
* pkg/storage: add redis storage backend

* pkg/storage/redis: set record create time correctly

* pkg/storage/redis: add docs

* pkg/storage/redis: run test with redis tag only

* pkg/storage/redis: use localhost

* pkg/storage/redis: use 127.0.0.1

* pkg/storage/redis: honor REDIS_URL env

* .github/workflows: add missing config for redis service

* .github/workflows: map redis ports to host

* pkg/storage/redis: use proto marshaler instead of json one

* pkg/storage/redis: use better implementation

By using redis supported datastructure:

 - Hash for storing record
 - Sorted set for storing by version
 - Set for storing deleted ids

List operation will be now performed in O(log(N)+M) instead of O(N) like
previous implementation.

* pkg/storage/redis: add tx to wrap redis transaction

* pkg/storage/redis: set record type in New

* pkg/storage/redis: make sure tx commands appear in right order

* pkg/storage/redis: make deletePermanentAfter as argument

* pkg/storage/redis: make sure version is incremented when deleting

* pkg/storage/redis: fix linter

* pkg/storage/redis: fix cmd construction
2020-07-22 03:07:20 +07:00
Caleb Doxsey
858077b3b6
authorize: custom rego policies (#1123)
* add support for custom rego policies

* add support for passing custom policies
2020-07-21 12:09:26 -06:00
bobby
d5433f8431
depedency: bump envoy to 1.15.0 (#1119)
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-07-21 08:25:37 -07:00
bobby
1912e32927
deps: update envoy to 1.14.4 (#1076)
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-07-20 21:12:44 -07:00
Travis Groth
75f2ed93ea
Set loopback address by ipv4 IP (#1116) 2020-07-20 22:31:48 -04:00
Travis Groth
bfc726ec3f
prevent dirty git state during release (#1117) 2020-07-20 18:00:03 -04:00
Travis Groth
a1b6bfec56
docs: Cloud Run / GCP Serverless (#1101)
* Add GCP Serverless and Cloud Run docs
2020-07-20 14:00:52 -04:00
Cuong Manh Le
e8d3ce1a2e
authorize,proxy: allow traefik forward auth without uri query (#1103)
In #1030, the fix was done without aware of the context that traefik
forward auth mode did allow request without the "?uri=". Previosuly,
this is done in proxy, and by converting the forward auth request to
actual request. The fix is #1030 prevent this conversion, to makre
authorize service aware of which is forward auth request.

But that causes traefik forward auth without "?uri" stop working. Fixing
it by making the authorize service also honor the forwarded uri header,
too.

Fixes #1096
2020-07-21 00:58:14 +07:00
bobby
e85226b609
grpc: use relative paths in codegen (#1106)
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-07-20 06:40:11 -07:00
Caleb Doxsey
fff782e04c
authorize: add evaluator store (#1105)
* add evaluator store

* handle arrays
2020-07-20 07:39:34 -06:00
Renovate Bot
d2656ecd67 chore(deps): update google.golang.org/genproto commit hash to 11fb19a 2020-07-20 08:00:29 +00:00
Cuong Manh Le
9dae633fe5
internal/frontend/assets/html: make timestamp human readable (#1107)
Since we switch to use databroker, time in template is now protobuf
timestamp instead of time.Time, that causes it appears in raw form
instead of human-readable format.

Fix this by converting protobuf timestamp to time.Time in template.
There's still a breaking change, though. The time will now appears in
UTC instead of local time.

Fixes #1100
2020-07-20 13:35:57 +07:00
Renovate Bot
0f17fb0d95 chore(deps): update module spf13/cobra to v1 2020-07-20 03:22:08 +00:00
Renovate Bot
2a99088cdf chore(deps): update module open-policy-agent/opa to v0.22.0 2020-07-20 02:27:34 +00:00
Renovate Bot
c2e487a996 chore(deps): update github.com/skratchdot/open-golang commit hash to eef8423 2020-07-20 01:06:57 +00:00
bobby
ffb6b395e2
deployment: fix pomerium-cli release (#1104)
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
2020-07-17 12:45:16 -07:00
Travis Groth
ca6715d3c5
Move examples repo into main repo (#1102) 2020-07-17 14:23:06 -04:00
Caleb Doxsey
b79e73b8b8
config: add support for policies stored in the databroker (#1099)
* wip

* always use databroker config source

* add test

* valid policy, remove debug lines
2020-07-17 10:35:29 -06:00
Cuong Manh Le
821f2e9000
config: allow setting directory sync interval and timeout (#1098)
Updates #567
2020-07-17 23:11:27 +07:00
Travis Groth
25867501d4
ci: Add cloudrun build (#1097)
* add cloudrun image build
2020-07-17 12:01:30 -04:00
Cuong Manh Le
408f201d16
authorize/evaluator/opa/policy: fix allow rules with impersonate (#1094)
Currently, with impersonated request, the real user email/group still
has effects.

Example:

	data.route_policies as [{
	    "source": "example.com",
	    "allowed_users": ["x@example.com"]
	}] with
	input.databroker_data as {
	    "session": {
	        "user_id": "user1"
	    },
	    "user": {
	        "email": "x@example.com"
	    }
	} with
	input.http as { "url": "http://example.com" } with
	input.session as { "id": "session1", "impersonate_email": "y@example.com" }

Here user "x@example.com" is allowed, but was impersonated as
"y@example.com". As the rules indicated, the request must be denied,
because it only allows "x@example.com", not "y@example.com". The current
bug causes the request is still allowed.

To fix it, when evaluates rules for allowed email/group/domain, we must checking
that the impersonate email/groups is not set/empty.

Fixes #1091
2020-07-17 22:07:11 +07:00
Cuong Manh Le
59c17fb497
internal/directory: improve google user groups list (#1092)
Skip group without members, so it saves us time to handle group members,
and reduce the size of groups.

While at it, also querying API with the fields we need.

Fixes #567
2020-07-17 16:51:26 +07:00