Andrey Antukh
fe83c5faea
✨ Add better reporting for s3 storage backend errors
2024-10-29 11:17:01 +01:00
Andrey Antukh
cb9839223e
✨ Add safer mechanism for tempfile naming
...
Using a uuidv8 that has strong guarranties about councurrent
ids generation that a simple random long
2024-10-29 11:17:01 +01:00
Andrey Antukh
15c42fba5e
🐛 Add retry mechanism for internal tmp file handling on s3 backend
2024-10-29 11:17:01 +01:00
Andrey Antukh
cc6e071f48
♻️ Remove all usage of graalvm js runtime
...
And replace it with a commandline call to nodejs
for execute a custom svgo based command line script.
2024-10-22 23:30:56 +02:00
Andrey Antukh
f949649ba3
⬆️ Update backend dependencies
2024-10-22 20:23:38 +02:00
Andrey Antukh
ceaafdbb1c
✨ Add offload mechanism for file snapshots
2024-08-26 13:52:42 +02:00
Andrey Antukh
0e92bcc0de
🎉 Add file-data offload mechanism
2024-08-09 14:28:18 +02:00
Andrey Antukh
f6bfe3931c
⚡ Add performance enhacements to storage/gc-touched task
2024-08-09 14:13:33 +02:00
Andrey Antukh
ec4260830c
♻️ Add API consistency fixes for task calling
...
Also adds a helper for calling tasks synchronously
2024-06-25 13:24:17 +02:00
Andrey Antukh
a31be7e2ff
✨ Use a prefixed dir for storing temp files
...
And mark them for deletion on JVM exit.
2024-02-14 09:53:54 +01:00
Andrey Antukh
9b59b92464
🐛 Improve not-found error report on s3 storage backend
2024-01-25 15:59:45 +01:00
Andrey Antukh
4fc391763e
✨ Prevent unexpected exception raising on closing s3 file
2024-01-25 15:59:45 +01:00
Andrey Antukh
0b29aaecc4
💄 Format backend code
2024-01-08 09:32:50 +01:00
Andrey Antukh
471fd78174
✨ Spawn vthread on s3 internal io completion
...
Instead of using platform threads
2024-01-04 12:41:16 +01:00
Andrey Antukh
746d898245
✨ Improve the db api efficiency
...
Mainly setup proper defaults and reduce unnecesary allocations
on every db api call.
2024-01-04 12:41:16 +01:00
Andrey Antukh
addb392ecc
✨ Add safety mechanism for direct object deletion
...
The main objective is prevent deletion of objects that can leave
unreachable orphan objects which we are unable to correctly track.
Additionally, this commit includes:
1. Properly implement safe cascade deletion of all participating
tables on soft deletion in the objects-gc task;
2. Make the file thumbnail related tables also participate in the
touch/refcount mechanism applyign to the same safety checks;
3. Add helper for db query lazy iteration using PostgreSQL support
for server side cursors;
4. Fix efficiency issues on gc related task using server side
cursors instead of custom chunked iteration for processing data.
The problem resided when a large chunk of rows that has identical
value on the deleted_at column and the chunk size is small (the
default); when the custom chunked iteration only reads a first N
items and skip the rest of the set to the next run.
This has caused many objects to remain pending to be eliminated,
taking up space for longer than expected. The server side cursor
based iteration does not has this problem and iterates correctly
over all objects.
5. Fix refcount issues on font variant deletion RPC methods
2024-01-03 10:56:57 +01:00
Andrey Antukh
87615ce221
💄 Fix format issues on backend module
2023-11-29 12:55:58 +01:00
Andrey Antukh
c64e14859c
✨ Simplify internal executor module
2023-11-27 14:25:12 +01:00
Andrey Antukh
4c190e385e
✨ Make the s3 client io-threads configurable and adaptable
2023-11-15 13:49:26 +01:00
Andrey Antukh
3b463f334c
✨ Add minor improvement to s3 storage upload thread
...
Start using virtual threads
2023-11-08 17:11:42 +01:00
Andrey Antukh
f370f28ca6
✨ Add improvements to the tmp file deletion scheduling
2023-11-08 14:13:41 +01:00
Andrey Antukh
6f93b41920
🎉 Add features assignation for teams
2023-11-07 12:48:31 +01:00
Andrey Antukh
dfdc9c9fa5
♻️ Refactor storage internal concurrency model
2023-03-14 12:30:27 +01:00
Andrey Antukh
cad1851e95
🔥 Replace own scheduled executor with the one defined in promesa lib
2023-03-14 12:30:27 +01:00
Andrey Antukh
ab3b9cba45
♻️ Refactor storage and assets related modules
...
- improve internal error handling
- add more specs and more asserts
2023-02-07 18:16:55 +01:00
Andrey Antukh
e372e8ba3e
🐛 Fix s3 client issues with s3 compatible services
2022-12-31 16:37:42 +01:00
Andrey Antukh
7f7efc5760
✨ Improve exception formating on backend
2022-12-05 08:51:54 +01:00
Andrey Antukh
69011007ac
✨ Change execution model of storage.tmp cleaner
2022-11-28 12:21:46 +01:00
Andrey Antukh
13a092b192
♻️ Normalize internal naming on the worker module
2022-11-28 12:20:58 +01:00
luz paz
e30bea0b6f
🔧 Fix typos in source code
...
Found via `codespell -q 3 -S *.po,./frontend/yarn.lock -L childs,clen,fpr,inflight,ody,ot,ro,te,trys,ue`
2022-10-04 10:40:34 +02:00
Andrey Antukh
41134f22e9
📎 Update license header
2022-09-20 23:23:22 +02:00
Andrey Antukh
e657c1bbfa
✨ Start use datoteka.io ns
2022-09-09 12:19:49 +02:00
Andrey Antukh
ec3651d85b
🎉 Add optional rate limit support for RPC calls
2022-08-31 12:55:48 +02:00
Andrey Antukh
ebcb385593
♻️ Minor refactor on storages
...
Fix many issues on FS & S3 backend; removes the unused and broken
DB backend. Normalize operations on bytes and byte streams on a
separated namespace: app.util.bytes
2022-06-22 11:37:45 +02:00
Andrey Antukh
20f357d75d
✨ Make the region param optional on s3 storage backend
...
Defaulting to the eu-central-1
2022-06-14 12:13:47 +02:00
Andrey Antukh
65e99cabbf
📎 Fix linter issues
...
Related to the linter update on devenv
2022-06-13 11:18:35 +02:00
Andrey Antukh
7f3c67724e
🐛 Fix svg media asset upload internal server error
2022-03-20 13:04:12 +01:00
Andrey Antukh
9e4a50fb15
♻️ Refactor backend to be more async friendly
2022-03-03 16:05:52 +01:00
Andrey Antukh
350663b7ce
🎉 Add support for alternative S3 compatible services
...
And also add support for all AWS regions (prevoiosly onlu eu-central-1)
was supported.
2022-02-16 13:58:19 +01:00
Andrey Antukh
321b2c7c23
🐛 Fix error handling on s3 delete-in-bulk operation
2022-02-15 12:26:36 +01:00
Andrey Antukh
c6054f7ab2
💄 Improve json namespace API (and fix linter).
2021-12-23 00:04:37 +01:00
Andrey Antukh
2728fa2b8d
✨ Add proper fdata objects deletion.
2021-06-15 09:25:37 +02:00
Andrey Antukh
0c97a44a2a
🎉 Add file offloading to external storage mechanism.
2021-06-14 15:41:27 +02:00
Andrey Antukh
e15a212b14
🎉 Add dashboard custom fonts management.
2021-05-04 14:21:31 +02:00
Andrey Antukh
9aee88f9f1
⬆️ Update backend dependencies.
2021-04-20 16:42:21 +02:00
Andrey Antukh
f545e41d10
📎 Fix license header.
2021-04-12 16:49:43 +02:00
Andrey Antukh
784a4f8ecd
✨ Add some type hints and remove legacy code.
2021-02-18 09:35:37 +01:00
Andrey Antukh
b80295a21c
✨ Fix all linter issues on backend code.
2021-02-01 09:48:28 +01:00
Andrey Antukh
a69a35a0b6
✨ Improve storage recheck task and add more specs.
2021-01-26 11:13:21 +01:00
Andrey Antukh
d68286821b
✨ Add the notion of temporal files on the storage.
2021-01-26 11:13:21 +01:00
