Reduce locking on objects-gc task

The main issue was the long running gc operation that
affects storage objects with deduplication. The long running
transacion ends locking some storage object rows which collaterally
made operations like import-binfile become blocked indefinitelly
because of the same rows (because of deduplication).

The solution used in this commit is split operations on small
chunks so we no longer use long running transactions that holds
too many locks. With this approach we will make a window to work
concurrently all operarate the distinct operations that requires
locks on the same rows.
This commit is contained in:
Andrey Antukh 2024-05-20 16:53:36 +02:00
parent 4c7a30a029
commit 584a0fdba1
11 changed files with 364 additions and 193 deletions

View file

@ -1189,6 +1189,7 @@
(t/is (nil? error))
(t/is (map? result)))
;; insert another thumbnail with different revn
(let [data {::th/type :create-file-thumbnail
::rpc/profile-id (:id prof)
:file-id (:id file)
@ -1207,8 +1208,6 @@
(t/is (= 2 (count rows)))))
(t/testing "gc task"
;; make the file eligible for GC waiting 300ms (configured
;; timeout for testing)
(let [res (th/run-task! :file-gc {:min-age 0})]
(t/is (= 1 (:processed res))))