Commit graph

51 commits

Author SHA1 Message Date
alonso.torres
ecb7f0a2f6 File history versions management 2024-10-29 14:23:35 +01:00
Andrey Antukh
9da891e9b0 📎 Enable auto-file-snapshot feature scripts/repl 2024-09-04 12:18:31 +02:00
Andrey Antukh
a6de12323e ♻️ Refactor file-update for make it more reusable 2024-09-04 12:18:31 +02:00
Andrey Antukh
50df2279a7 🐛 Make the media cleaning on file-gc task aware of snapshots
It now takes in account the snapshots, and prevents
deletion of media files used in snapshots.
2024-09-03 14:50:17 +02:00
Andrey Antukh
0117a4767d 📎 Rename file-snapshot to auto-file-snapshot 2024-09-02 11:52:19 +02:00
Andrey Antukh
ceaafdbb1c Add offload mechanism for file snapshots 2024-08-26 13:52:42 +02:00
Andrey Antukh
cacee40d11 🎉 Add proper schema encoding/decoding mechanism
this allows almost all api operations to success usin application/json
encoding with the exception of the update-file, which we need to
approach a bit differently;

the reason update-file is different, is because the operations vector
is right now defined without the context of shape type, so we are just
unable to properly parse the value to correct type using the schema
decoding mechanism
2024-08-21 11:27:36 +02:00
Andrey Antukh
0e92bcc0de 🎉 Add file-data offload mechanism 2024-08-09 14:28:18 +02:00
Andrey Antukh
5cf54c6384 Improve file snapshoting mechanism 2024-07-29 10:19:34 +02:00
Andrey Antukh
28c2197ba7 ♻️ Add string length validation to backend RPC methods fields 2024-07-04 12:30:28 +02:00
Andrey Antukh
293b460cab Fix a performance regression with file validation with some features
The feature fdata/pointer-map tracking mechanism interacts pretty bad
with possible local mutations on the validation subsystem. The fix
consist on disabling the tracking mechanism on the validation.
2024-06-17 15:56:06 +02:00
Andrey Antukh
6436ef334b ♻️ Refactor persistence layer 2024-06-04 10:15:32 +02:00
Andrey Antukh
b718a282e0 ♻️ Add minor refactor to file migrations
Relevant changes:

- Add the ability to create migration in both directions, defaulting
  to identity if not provided
- Move the version attribute to file table column for to make it more
  accessible (previously it was on data blob)
- Reduce db update operations on file-update rpc method
2024-02-19 09:20:47 +01:00
Andrey Antukh
29d48f0a98 Add minor code cleaning on file-update ns 2024-02-14 17:38:53 +01:00
Andrey Antukh
722cb6351d 💄 Add minor cosmetic changes to file-update ns 2024-02-12 14:55:42 +01:00
Andrey Antukh
a5c6d78ee5 ♻️ Fix some fundamental bugs on climit module
The climit previously of this commit is heavily used inside a
transactions, so in heavy contention operation such that file thumbnail
creation can cause a db pool exhaust.

This commit fixes this issue setting up a better resource limiting
mechanism that works outside the transactions so, contention will
no longer hold an open connection/transaction.

It also adds general improvement to the traceability to the climit
mechanism: it now properly logs the profile-id that is currently
cause some contention on specific resources.

It also add a general/root climit that is applied to all requests
so if someone start making abussive requests, we can clearly detect
it.
2024-02-01 17:37:49 +01:00
Andrey Antukh
6ad6e6f856 🐛 Fix objects-map and pointer-map issues on file crud 2024-01-25 16:03:29 +01:00
Andrey Antukh
4899b3af6e 🐛 Fix incorrect pointer deref on validating file on update operation 2024-01-04 16:31:57 +01:00
Andrey Antukh
746d898245 Improve the db api efficiency
Mainly setup proper defaults and reduce unnecesary allocations
on every db api call.
2024-01-04 12:41:16 +01:00
Andrey Antukh
611594a392 Add general features handling improvements 2023-12-14 10:35:24 +01:00
Andrey Antukh
ba01f314dd 🐛 Fix incorrect feature context setup on file update 2023-12-14 10:35:24 +01:00
Andrey Antukh
2dbe7bca07 Add general improvements to file validation and repair api 2023-12-14 10:35:24 +01:00
Alejandro
47bf817462
Merge pull request #3906 from penpot/palba-bugfixing9
🐛 Bugfixing
2023-12-14 06:41:52 +01:00
Andrey Antukh
417366d998 Reorganize fdata/pointer-map feature helpers
Mainly move all pointer-map related helpers from app.rpc.commands.files
to the the app.features.fdata namespace and normalizes codestile around
feature handling on all affected code.

This commit also comes with several features related bugifxes on the
components-v2 migration code:

- properly migrate legacy feature names on apply components-v2 migration
- start using new fdata feature related functions
- prevent generation of a ephimeral pointer on each graphic migration
  operation; on large files this caused a very noticiable overhead of
  creating a big number of completly unused pointer maps
- do persistence after validation and not before
2023-12-13 14:20:07 +01:00
Andrey Antukh
76a6f077a6 🐛 Fix incorrect feature handling on absorb-library! fn
Used in shared flag assignation and library deletion
2023-12-13 11:56:20 +01:00
Pablo Alba
b41f63c16e 🐛 Fix component restoration is not notified to files using the library 2023-12-12 23:39:16 +01:00
Andrey Antukh
afa735a9c1 Add protection for version inconsistency on opening or editing file 2023-12-11 17:14:20 +01:00
Andrey Antukh
87615ce221 💄 Fix format issues on backend module 2023-11-29 12:55:58 +01:00
Andrey Antukh
54341d5b22 Make the RPC climit subsystem more robust 2023-11-27 14:25:12 +01:00
Andrey Antukh
83c6354a0a Change order of hard and soft validation on file update 2023-11-23 16:58:38 +01:00
Andrey Antukh
bb8c8f5a0c Add minor improvements to error reporting 2023-11-21 23:35:35 +01:00
Andrey Antukh
f66228c19d 🐛 Clean fdata from nils 2023-11-21 12:11:26 +01:00
Andrey Antukh
d90e184b4d Add file soft validation support 2023-11-20 19:54:53 +01:00
Andrey Antukh
ac3d7f00d5 Make file schema validation configurable using flags 2023-11-20 19:42:46 +01:00
Andrey Antukh
ec8b68721b Improve schema validation handling
And properly honor the file-validation flag
2023-11-20 16:57:05 +01:00
Andrey Antukh
52fbc678f3 ♻️ Move app.common.pages to app.common.files 2023-11-16 11:07:36 +01:00
Andrey Antukh
9e4ed0ea92 Improve file validation process on update-file rpc method 2023-11-14 10:55:55 +01:00
Andrey Antukh
6f93b41920 🎉 Add features assignation for teams 2023-11-07 12:48:31 +01:00
Andrés Moya
2fe820304e Add some migrations and fixes to component-v2 migration 2023-10-20 16:13:01 +02:00
Andrés Moya
24efa867e7 🎉 Do file validation on each file change 2023-09-27 13:26:28 +02:00
Andrés Moya
836b4538dd Add validate & repair functions 2023-09-20 15:40:43 +02:00
Andrey Antukh
dfc2ab56a9 💄 Fix code style consistency on schema declarations on file ns 2023-07-11 17:27:36 +02:00
Andrey Antukh
405aa66357 🎉 Add new shape & rect data structures
Also optimizes some functions for faster shape and rect props
access (there is still a lot of work ahead optimizing the rest of
the functions)

Also normalizes shape creation and validation for ensuring
correct setup of all the mandatory properties.
2023-07-11 17:27:35 +02:00
Andrey Antukh
5ca3d01ea1 🎉 Add malli based validation and coersion subsystem 2023-05-17 16:05:29 +02:00
Andrés Moya
8a42a53522 Notify library updates when really needed 2023-05-17 14:12:49 +02:00
Andrey Antukh
5e89aa2726 Improve file-gc task
make it more aware of fragments referenced on changes snapshots
2023-04-26 13:28:32 +02:00
Andrés Moya
ce3e30ea02 🐛 Fix linter issues 2023-03-21 17:12:54 +01:00
Andrey Antukh
aafbf6bc15 ♻️ Refactor cocurrency model on backend
Mainly the followin changes:

- Pass majority of code to the old and plain synchronous style
  and start using virtual threads for the RPC (and partially some
  HTTP server middlewares).
- Make some improvements on how CLIMIT is handled, simplifying code
- Improve considerably performance reducing the reflection and
  unnecesary funcion calls on the whole stack-trace of an RPC call.
- Improve efficiency reducing considerably the total threads number.
2023-03-14 12:30:27 +01:00
Andrey Antukh
2e717882f1 ♻️ Refactor websockets impl to use virtual threads
Removing the use of core.async code and implement code using
plain old and familiar synchronous code
2023-03-14 12:30:27 +01:00
Andrey Antukh
4258a840ac ♻️ Use proper namespace qualified access to pool 2023-01-18 10:51:58 +01:00