Commit graph

27 commits

Author SHA1 Message Date
Andrey Antukh
16a051d7e0 Improve efficiency of thumbnails creation RPC methods
Moving the retry mechanism out of the transaction
2024-02-01 17:24:42 +01:00
Andrey Antukh
746d898245 Improve the db api efficiency
Mainly setup proper defaults and reduce unnecesary allocations
on every db api call.
2024-01-04 12:41:16 +01:00
Andrey Antukh
addb392ecc Add safety mechanism for direct object deletion
The main objective is prevent deletion of objects that can leave
unreachable orphan objects which we are unable to correctly track.

Additionally, this commit includes:

1. Properly implement safe cascade deletion of all participating
   tables on soft deletion in the objects-gc task;

2. Make the file thumbnail related tables also participate in the
   touch/refcount mechanism applyign to the same safety checks;

3. Add helper for db query lazy iteration using PostgreSQL support
   for server side cursors;

4. Fix efficiency issues on gc related task using server side
   cursors instead of custom chunked iteration for processing data.

   The problem resided when a large chunk of rows that has identical
   value on the deleted_at column and the chunk size is small (the
   default); when the custom chunked iteration only reads a first N
   items and skip the rest of the set to the next run.

   This has caused many objects to remain pending to be eliminated,
   taking up space for longer than expected. The server side cursor
   based iteration does not has this problem and iterates correctly
   over all objects.

5. Fix refcount issues on font variant deletion RPC methods
2024-01-03 10:56:57 +01:00
Andrey Antukh
0ad2e8a0f2 Make retrieving fdata for thumbnail to no modify the file
This prevents that file to be considered opened just for creating
the thumbnail for it.
2023-12-15 15:18:04 +01:00
Andrey Antukh
417366d998 Reorganize fdata/pointer-map feature helpers
Mainly move all pointer-map related helpers from app.rpc.commands.files
to the the app.features.fdata namespace and normalizes codestile around
feature handling on all affected code.

This commit also comes with several features related bugifxes on the
components-v2 migration code:

- properly migrate legacy feature names on apply components-v2 migration
- start using new fdata feature related functions
- prevent generation of a ephimeral pointer on each graphic migration
  operation; on large files this caused a very noticiable overhead of
  creating a big number of completly unused pointer maps
- do persistence after validation and not before
2023-12-13 14:20:07 +01:00
Andrey Antukh
76a6f077a6 🐛 Fix incorrect feature handling on absorb-library! fn
Used in shared flag assignation and library deletion
2023-12-13 11:56:20 +01:00
Andrey Antukh
ad0378270f Add missing sm/define on some file and file-thumbnails rpc methods 2023-12-11 17:14:20 +01:00
Andrey Antukh
87615ce221 💄 Fix format issues on backend module 2023-11-29 12:55:58 +01:00
Andrey Antukh
d82ebdc034 Add deduplication for file object thumbnails 2023-11-24 11:06:47 +01:00
Andrey Antukh
925f2dc30f Remove duplicated rpc method for creating file object thumbnails 2023-11-24 10:41:27 +01:00
Andrey Antukh
52fbc678f3 ♻️ Move app.common.pages to app.common.files 2023-11-16 11:07:36 +01:00
Aitor
e24309b883 ♻️ Refactor thumbnails persistence 2023-11-10 14:15:54 +01:00
Andrey Antukh
da68eae7c6 Add proper climit configuration for file-thumbnails 2023-11-08 17:11:42 +01:00
Aitor
7951350762 🐛 Fix db table tagged thumbnails 2023-11-07 16:01:00 +01:00
Andrey Antukh
6f93b41920 🎉 Add features assignation for teams 2023-11-07 12:48:31 +01:00
Aitor
bb4d3583e1 🐛 Fix missing schema tag option 2023-11-02 11:06:30 +01:00
Aitor
c28c55bf0b 🎉 Add tag property to thumbnails 2023-11-02 11:06:30 +01:00
Andrey Antukh
36506ec360 🐛 Replace :use-for-thumbnail? with :use-for-thumbnail 2023-09-05 12:01:40 +02:00
Andrey Antukh
31323703a8 Merge remote-tracking branch 'origin/staging' into develop 2023-08-09 13:36:42 +02:00
Andrey Antukh
36b167956c Add improvements to api doc 2023-08-08 13:44:47 +02:00
Andrey Antukh
dfc2ab56a9 💄 Fix code style consistency on schema declarations on file ns 2023-07-11 17:27:36 +02:00
Andrey Antukh
405aa66357 🎉 Add new shape & rect data structures
Also optimizes some functions for faster shape and rect props
access (there is still a lot of work ahead optimizing the rest of
the functions)

Also normalizes shape creation and validation for ensuring
correct setup of all the mandatory properties.
2023-07-11 17:27:35 +02:00
Andrey Antukh
a97929992e Convert to schema some specs on file-thumbnails rpc methods 2023-06-22 09:34:13 +02:00
Andrey Antukh
d11b007795 Add thumbnail renderer
And integrate the dashboard thumbnails to use that service
2023-06-21 20:10:49 +02:00
Aitor
48834f96d3 ♻️ Refactor thumbnail rendering on workspace 2023-05-22 10:56:46 +02:00
Andrey Antukh
5ca3d01ea1 🎉 Add malli based validation and coersion subsystem 2023-05-17 16:05:29 +02:00
Andrey Antukh
e8ffcbae69 🎉 Add support for multipart upload of thumbnails
and improve the thumbnails storage to offloading it
to the storage subsystem
2023-05-05 17:00:35 +02:00