Andrey Antukh
87615ce221
💄 Fix format issues on backend module
2023-11-29 12:55:58 +01:00
Andrey Antukh
c64e14859c
✨ Simplify internal executor module
2023-11-27 14:25:12 +01:00
Andrey Antukh
a23d1908e9
✨ Improve worker logging
2023-09-04 11:35:31 +02:00
Andrey Antukh
a3495800b5
✨ Add minor logging improvements to worker namespace
2023-08-31 21:09:18 +02:00
Andrey Antukh
d2d9aeff25
📎 Reduce log level of worker submit operation
...
Start logging to as TRACE instead of DEBUG
2023-08-31 20:59:58 +02:00
Andrey Antukh
5275c35002
🐛 Prevent rollback for idle-in-transaction errors on cron tasks
2023-08-24 11:18:56 +02:00
Andrey Antukh
259b05db51
✨ Add more improvements to error reporting
2023-08-04 13:10:36 +02:00
Andrey Antukh
aafbf6bc15
♻️ Refactor cocurrency model on backend
...
Mainly the followin changes:
- Pass majority of code to the old and plain synchronous style
and start using virtual threads for the RPC (and partially some
HTTP server middlewares).
- Make some improvements on how CLIMIT is handled, simplifying code
- Improve considerably performance reducing the reflection and
unnecesary funcion calls on the whole stack-trace of an RPC call.
- Improve efficiency reducing considerably the total threads number.
2023-03-14 12:30:27 +01:00
Andrey Antukh
04b321caae
✨ Add several improvements to internal worker impl
...
Mainly for make the cron jobs do not block the scheduled executor
and offload all work to a separate threads
2023-03-14 12:30:27 +01:00
Andrey Antukh
cad1851e95
🔥 Replace own scheduled executor with the one defined in promesa lib
2023-03-14 12:30:27 +01:00
Andrey Antukh
ab3b9cba45
♻️ Refactor storage and assets related modules
...
- improve internal error handling
- add more specs and more asserts
2023-02-07 18:16:55 +01:00
Andrey Antukh
071ecca875
🐛 Fix internal executor naming issue
2023-02-02 13:38:04 +01:00
Andrey Antukh
db689d151e
♻️ Refactor profile and session handling
...
- makes the profile access more efficient (replace in-app joins to a
simple select query on profile table
- add partial support for access-tokens (still missing some RPC methods)
- move router definitions to specific modules and simplify the main http
module definitions to simple includes
- simplifiy authentication code related to access-tokens and sessions
- normalize db parameters with proper namespaced props
- more work on convert all modules initialization to use proper specs
with fully-qualified keyword config props
2023-01-18 10:51:58 +01:00
Andrey Antukh
226afe98e0
💄 Change worker logging
2022-12-15 11:33:10 +01:00
Andrey Antukh
d7459db292
🎉 Add task deduplication by label
2022-12-13 23:13:11 +01:00
Andrey Antukh
c0a4b7dc76
✨ Improve worker queue management
...
and add specific worker instance for webhooks
2022-12-13 16:17:31 +01:00
Andrey Antukh
8bad9d8340
♻️ Refactor loggers/audit, auth/oidc, and http/clent modules
2022-12-05 08:53:00 +01:00
Andrey Antukh
7f7efc5760
✨ Improve exception formating on backend
2022-12-05 08:51:54 +01:00
Andrey Antukh
c451c7bb9d
🐛 Fix regression on worker task handling
2022-11-30 13:13:11 +01:00
Andrey Antukh
0600b2abe4
♻️ Make the worker abstraction more scalable
...
Start using redis for dispatcher to worker communication
and add the ability to start multiple threads to worker
for increase the concurrency.
2022-11-28 12:21:44 +01:00
Andrey Antukh
13a092b192
♻️ Normalize internal naming on the worker module
2022-11-28 12:20:58 +01:00
luz paz
e30bea0b6f
🔧 Fix typos in source code
...
Found via `codespell -q 3 -S *.po,./frontend/yarn.lock -L childs,clen,fpr,inflight,ody,ot,ro,te,trys,ue`
2022-10-04 10:40:34 +02:00
Andrey Antukh
41134f22e9
📎 Update license header
2022-09-20 23:23:22 +02:00
Andrey Antukh
9bfdcc6277
✨ Make the task retry algorithm use better backoff values
2022-09-20 23:04:37 +02:00
Andrey Antukh
6f42f4ec45
♻️ Refactor semaphore and executors
2022-09-20 11:32:45 +02:00
Andrey Antukh
7fa609d5f4
✨ Allow disable worker
2022-08-12 08:52:36 +02:00
Andrey Antukh
f2140a1421
🐛 Fix cron scheduler locking mechanism
...
And add improved logging to the worker/cron code
2022-07-04 11:32:36 +02:00
Andrey Antukh
9e4a50fb15
♻️ Refactor backend to be more async friendly
2022-03-03 16:05:52 +01:00
Andrey Antukh
d1c834e647
🐛 Fix minor issue on executors monitor
2022-03-01 14:34:13 +01:00
Andrey Antukh
e60b8a7aef
🐛 Minor fix on worker executors monitor
2022-02-28 17:21:36 +01:00
Andrey Antukh
5d2715dd32
✨ Improve monitors monitor
2022-02-28 15:29:30 +01:00
Andrey Antukh
7cf27ac86d
♻️ Refactor general resource and concurrency model on backend
2022-02-22 13:05:41 +01:00
Andrey Antukh
7eed8c5ee5
Merge remote-tracking branch 'origin/main' into develop
2022-02-16 11:23:26 +01:00
Andrey Antukh
9b78b2a432
✨ Improve error reporting on background tasks
2022-02-15 12:26:36 +01:00
Andrey Antukh
321b2c7c23
🐛 Fix error handling on s3 delete-in-bulk operation
2022-02-15 12:26:36 +01:00
Andrey Antukh
741d2b3f3c
Merge remote-tracking branch 'origin/main' into develop
2022-02-12 17:33:28 +01:00
Andrey Antukh
827c2140b7
♻️ Refactor error reporting and logging context formatting
...
The prev approach uses clojure.pprint without any limit extensivelly
for format error context data and all this is done on the calling
thread. The clojure.pprint seems very inneficient in cpu and memory
usage on pprinting large data structures.
This is improved in the following way:
- All formatting and pretty printing is moved to logging thread,
reducing unnecesary blocking and load on jetty http threads.
- Replace the clojure.pprint with fipp.edn that looks considerably
faster than the first one.
- Add some safe limits on pretty printer for strip printing some
data when the data structure is very large, very deep or both.
2022-02-10 15:12:35 +01:00
Andrey Antukh
bc2a0432b9
✨ Allow connect to read-only databases.
2022-01-27 16:11:32 +01:00
Andrey Antukh
2d118ecc65
Merge remote-tracking branch 'origin/staging' into develop
2021-12-27 09:50:15 +01:00
Andrey Antukh
1839397ebc
✨ Minor enhacements on log processing.
2021-12-23 18:36:58 +01:00
Josh Soref
cd2d3d5fa3
🐛 Fix typos in backend
2021-11-15 09:53:10 -05:00
Andrey Antukh
b64d5ef357
🎉 Add unified logging api.
2021-09-30 11:48:14 +02:00
Andrey Antukh
d7eec3b92b
Merge remote-tracking branch 'origin/main' into develop
2021-06-03 12:56:37 +02:00
Andrey Antukh
bae709df5b
🐛 Fix custom font deletion task.
2021-06-03 12:55:31 +02:00
Andrey Antukh
548664f6ce
♻️ Internal directory refactor.
...
Make common as first-class module.
2021-05-31 11:04:32 +02:00
Andrey Antukh
f545e41d10
📎 Fix license header.
2021-04-12 16:49:43 +02:00
Andrey Antukh
8daf6e822e
🎉 Add profile activity registry logger.
2021-04-09 15:28:18 +02:00
Andrey Antukh
e12a6e65a6
♻️ Refactor logging.
2021-04-09 15:28:18 +02:00
Andrey Antukh
0926fbcbc6
♻️ Minor code reorganization.
...
Improves modularity and reusability and allows usage
of backend code as a library.
2021-03-31 09:28:15 +02:00
Andrey Antukh
fbe2e2a285
✨ Improve tasks metrics.
2021-02-22 14:50:01 +01:00