Commit graph

81 commits

Author SHA1 Message Date
Andrey Antukh
eaf546ba5e Add improvements to telemetry task 2024-03-25 15:07:55 +01:00
Andrey Antukh
fd24831c71 Move audit tasks to separated namespace files 2024-03-25 10:46:15 +01:00
Andrey Antukh
91118bec70 Improve internal naming of setup/props
This reverts commit a6f70c77cb.
2024-03-14 10:48:23 +01:00
alonso.torres
a6f70c77cb Revert " Improve internal naming of setup/props"
This reverts commit f525c6df5e.
2024-03-13 16:21:12 +01:00
Andrey Antukh
f525c6df5e Improve internal naming of setup/props 2024-03-13 11:39:53 +01:00
Andrey Antukh
8231890ee4 🔥 Remove unnecesary line on audit ns 2024-02-05 20:10:44 +01:00
Andrey Antukh
dabb9d0a82 Improve internal API of retry mechanism 2024-02-01 17:24:42 +01:00
Andrey Antukh
0b29aaecc4 💄 Format backend code 2024-01-08 09:32:50 +01:00
Andrey Antukh
addb392ecc Add safety mechanism for direct object deletion
The main objective is prevent deletion of objects that can leave
unreachable orphan objects which we are unable to correctly track.

Additionally, this commit includes:

1. Properly implement safe cascade deletion of all participating
   tables on soft deletion in the objects-gc task;

2. Make the file thumbnail related tables also participate in the
   touch/refcount mechanism applyign to the same safety checks;

3. Add helper for db query lazy iteration using PostgreSQL support
   for server side cursors;

4. Fix efficiency issues on gc related task using server side
   cursors instead of custom chunked iteration for processing data.

   The problem resided when a large chunk of rows that has identical
   value on the deleted_at column and the chunk size is small (the
   default); when the custom chunked iteration only reads a first N
   items and skip the rest of the set to the next run.

   This has caused many objects to remain pending to be eliminated,
   taking up space for longer than expected. The server side cursor
   based iteration does not has this problem and iterates correctly
   over all objects.

5. Fix refcount issues on font variant deletion RPC methods
2024-01-03 10:56:57 +01:00
Andrey Antukh
bb5a4c0fa5 Update yetti and adapt for ring-2.0 2023-11-27 14:25:12 +01:00
Andrey Antukh
5ca3d01ea1 🎉 Add malli based validation and coersion subsystem 2023-05-17 16:05:29 +02:00
Andrey Antukh
4e7f32aa88 Improve retry mechanism and macros 2023-05-17 15:47:21 +02:00
Alejandro Alonso
890583a13a Add mvp access-token support 2023-05-04 22:14:55 +02:00
Andrey Antukh
b37a92aaf7 Merge remote-tracking branch 'origin/staging' into develop 2023-04-25 13:34:28 +02:00
Andrey Antukh
aafbf6bc15 ♻️ Refactor cocurrency model on backend
Mainly the followin changes:

- Pass majority of code to the old and plain synchronous style
  and start using virtual threads for the RPC (and partially some
  HTTP server middlewares).
- Make some improvements on how CLIMIT is handled, simplifying code
- Improve considerably performance reducing the reflection and
  unnecesary funcion calls on the whole stack-trace of an RPC call.
- Improve efficiency reducing considerably the total threads number.
2023-03-14 12:30:27 +01:00
Andrey Antukh
1d21ee7089 Merge remote-tracking branch 'origin/staging' into develop 2023-02-24 18:30:05 +01:00
Andrey Antukh
f0c0e5e43a Add proper audit log for invitations 2023-02-24 10:28:07 +01:00
Andrey Antukh
78aea0f24e 🐛 Fix incorrect props cleaning on auditlog 2023-02-08 10:35:57 +01:00
Andrey Antukh
8161d3ae09 🐛 Fix incorrect props cleaning on auditlog 2023-02-07 18:38:54 +01:00
Andrey Antukh
50ee0ad3fd Merge remote-tracking branch 'origin/staging' into develop 2023-02-01 23:25:25 +01:00
Andrey Antukh
86712f977d 🐛 Skip unnecesary mutation events from audit log 2023-02-01 18:06:12 +01:00
Alejandro Alonso
11db7590eb Merge remote-tracking branch 'origin/staging' into develop 2023-01-30 15:39:17 +01:00
Alejandro Alonso
2c38f31aa9 🐛 Fix clean archived auditlog 2023-01-30 13:11:50 +01:00
Andrey Antukh
db689d151e ♻️ Refactor profile and session handling
- makes the profile access more efficient (replace in-app joins to a
  simple select query on profile table
- add partial support for access-tokens (still missing some RPC methods)
- move router definitions to specific modules and simplify the main http
  module definitions to simple includes
- simplifiy authentication code related to access-tokens and sessions
- normalize db parameters with proper namespaced props
- more work on convert all modules initialization to use proper specs
  with fully-qualified keyword config props
2023-01-18 10:51:58 +01:00
Andrey Antukh
b235d3f0f2 Improve update-file webhook batching
make it per user
2023-01-09 11:43:02 +01:00
Andrey Antukh
bdd00be5e4 🐛 Fix incorrect deduplication of tasks on webhooks 2022-12-15 11:27:43 +01:00
Andrey Antukh
6a7a25121e Improve default update-file webhook batch timeout 2022-12-14 16:22:13 +01:00
Andrey Antukh
c820c49fc5 Add generic (blocking) retry macro
And use it on audit handling
2022-12-14 16:22:13 +01:00
Andrey Antukh
be5053ce22 Use the same value for created_at and tracked_at on audit 2022-12-14 16:22:13 +01:00
Andrey Antukh
d56082307b 🎉 Add update-file (batched) to webhooks 2022-12-13 23:14:55 +01:00
Andrey Antukh
5b9f0ed0b1 🎉 Add webhook processing worker 2022-12-13 16:17:31 +01:00
Andrey Antukh
7f589b09ca ♻️ Move audit http handler to RPC 2022-12-13 16:17:31 +01:00
Andrey Antukh
861328af3e ⬆️ Update promesa library to 10.0.570 2022-12-05 09:06:24 +01:00
Andrey Antukh
8bad9d8340 ♻️ Refactor loggers/audit, auth/oidc, and http/clent modules 2022-12-05 08:53:00 +01:00
Alejandro Alonso
6894d90137 Merge remote-tracking branch 'origin/staging' into develop 2022-10-19 09:06:11 +02:00
Andrey Antukh
3f5ac58c73 🐛 Fix unexpected exception audit archive task 2022-10-19 08:48:23 +02:00
luz paz
e30bea0b6f 🔧 Fix typos in source code
Found via `codespell -q 3 -S *.po,./frontend/yarn.lock -L childs,clen,fpr,inflight,ody,ot,ro,te,trys,ue`
2022-10-04 10:40:34 +02:00
Andrey Antukh
41134f22e9 📎 Update license header 2022-09-20 23:23:22 +02:00
Andrey Antukh
435c627afd Make the audit log gc more agressive 2022-09-20 11:32:45 +02:00
Andrey Antukh
d6d9d25fce ♻️ Refactor token generation API 2022-08-31 11:01:07 +02:00
Andrey Antukh
73f1418c95 🐛 Normalize return value from parse-client-ip function 2022-06-29 10:59:50 +02:00
Alejandro Alonso
07881eed65 Merge remote-tracking branch 'origin/staging' into develop 2022-06-27 09:28:54 +02:00
Andrey Antukh
c211e84498 🐛 Fix incorrect register-profile audit log handling 2022-06-23 14:17:26 +02:00
Andrey Antukh
adf2d82a52 🎉 Add proper logging reports on audit-log-archive task 2022-06-15 12:21:23 +02:00
Andrey Antukh
8acc9af1f5 📎 Add more events instrumentation 2022-03-22 14:48:10 +01:00
Andrey Antukh
f33d6610e7 📎 Properly log error on audit archive task fail 2022-03-11 16:21:11 +01:00
Andrey Antukh
d9bd63d34f 📎 Reduce audit log archive task chunk size 2022-03-11 15:14:40 +01:00
Andrey Antukh
1b444a42f2 ♻️ Refactor http server layer
Make it fully asynchronous.
2022-03-09 17:18:06 +01:00
Andrey Antukh
9e4a50fb15 ♻️ Refactor backend to be more async friendly 2022-03-03 16:05:52 +01:00
Andrey Antukh
340ee859f9 📎 Fix linter issues 2022-02-23 12:17:18 +01:00