Commit graph

31 commits

Author SHA1 Message Date
Andrey Antukh
fe83c5faea Add better reporting for s3 storage backend errors 2024-10-29 11:17:01 +01:00
Andrey Antukh
15c42fba5e 🐛 Add retry mechanism for internal tmp file handling on s3 backend 2024-10-29 11:17:01 +01:00
Andrey Antukh
9b59b92464 🐛 Improve not-found error report on s3 storage backend 2024-01-25 15:59:45 +01:00
Andrey Antukh
4fc391763e Prevent unexpected exception raising on closing s3 file 2024-01-25 15:59:45 +01:00
Andrey Antukh
471fd78174 Spawn vthread on s3 internal io completion
Instead of using platform threads
2024-01-04 12:41:16 +01:00
Andrey Antukh
87615ce221 💄 Fix format issues on backend module 2023-11-29 12:55:58 +01:00
Andrey Antukh
4c190e385e Make the s3 client io-threads configurable and adaptable 2023-11-15 13:49:26 +01:00
Andrey Antukh
3b463f334c Add minor improvement to s3 storage upload thread
Start using virtual threads
2023-11-08 17:11:42 +01:00
Andrey Antukh
f370f28ca6 Add improvements to the tmp file deletion scheduling 2023-11-08 14:13:41 +01:00
Andrey Antukh
dfdc9c9fa5 ♻️ Refactor storage internal concurrency model 2023-03-14 12:30:27 +01:00
Andrey Antukh
ab3b9cba45 ♻️ Refactor storage and assets related modules
- improve internal error handling
- add more specs and more asserts
2023-02-07 18:16:55 +01:00
Andrey Antukh
e372e8ba3e 🐛 Fix s3 client issues with s3 compatible services 2022-12-31 16:37:42 +01:00
Andrey Antukh
41134f22e9 📎 Update license header 2022-09-20 23:23:22 +02:00
Andrey Antukh
ec3651d85b 🎉 Add optional rate limit support for RPC calls 2022-08-31 12:55:48 +02:00
Andrey Antukh
ebcb385593 ♻️ Minor refactor on storages
Fix many issues on FS & S3 backend; removes the unused and broken
DB backend. Normalize operations on bytes and byte streams on a
separated namespace: app.util.bytes
2022-06-22 11:37:45 +02:00
Andrey Antukh
20f357d75d Make the region param optional on s3 storage backend
Defaulting to the eu-central-1
2022-06-14 12:13:47 +02:00
Andrey Antukh
65e99cabbf 📎 Fix linter issues
Related to the linter update on devenv
2022-06-13 11:18:35 +02:00
Andrey Antukh
9e4a50fb15 ♻️ Refactor backend to be more async friendly 2022-03-03 16:05:52 +01:00
Andrey Antukh
350663b7ce 🎉 Add support for alternative S3 compatible services
And also add support for all AWS regions (prevoiosly onlu eu-central-1)
was supported.
2022-02-16 13:58:19 +01:00
Andrey Antukh
321b2c7c23 🐛 Fix error handling on s3 delete-in-bulk operation 2022-02-15 12:26:36 +01:00
Andrey Antukh
2728fa2b8d Add proper fdata objects deletion. 2021-06-15 09:25:37 +02:00
Andrey Antukh
0c97a44a2a 🎉 Add file offloading to external storage mechanism. 2021-06-14 15:41:27 +02:00
Andrey Antukh
9aee88f9f1 ⬆️ Update backend dependencies. 2021-04-20 16:42:21 +02:00
Andrey Antukh
f545e41d10 📎 Fix license header. 2021-04-12 16:49:43 +02:00
Andrey Antukh
b80295a21c Fix all linter issues on backend code. 2021-02-01 09:48:28 +01:00
Andrey Antukh
d68286821b Add the notion of temporal files on the storage. 2021-01-26 11:13:21 +01:00
Andrey Antukh
ef25f8a721 Avoid reflection on s3 storage backend. 2021-01-19 15:37:26 +01:00
Andrey Antukh
d494e44df3 🎉 Add builtin copy fast path operation for storage. 2021-01-19 15:01:33 +01:00
Andrey Antukh
25177898e1 🐛 Fix prefix bug on s3 storage backend. 2021-01-07 11:29:57 +01:00
Andrey Antukh
ab944fb9ae ♻️ Integrate new storage subsystem. 2021-01-07 11:29:57 +01:00
Andrey Antukh
760eb926bf 🎉 Add plugable storages abstraction layer (with support for fs, s3 and db). 2021-01-07 11:29:57 +01:00