mirror of
https://github.com/penpot/penpot.git
synced 2025-05-10 03:56:37 +02:00
📚 Merge penpot/penpot-docs repository
This commit is contained in:
David Barragán Merino
Andrey Antukh
parent
3932054ea6
commit
88296480ec
665 changed files with 17621 additions and 0 deletions
136
docs/technical-guide/developer/architecture/backend.md
Normal file
136
docs/technical-guide/developer/architecture/backend.md
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
---
|
||||
title: Backend app
|
||||
---
|
||||
|
||||
# Backend app
|
||||
|
||||
This app is in charge of CRUD of data, integrity validation and persistence
|
||||
into a database and also into a file system for media attachments.
|
||||
|
||||
To handle deletions it uses a garbage collector mechanism: no object in the
|
||||
database is deleted instantly. Instead, a field <code class="language-bash">deleted_at</code> is set with the
|
||||
date and time of the deletion, and every query ignores db rows that have this
|
||||
field set. Then, an async task that runs periodically, locates rows whose
|
||||
deletion date is older than a given threshold and permanently deletes them.
|
||||
|
||||
For this, and other possibly slow tasks, there is an internal async tasks
|
||||
worker, that may be used to queue tasks to be scheduled and executed when the
|
||||
backend is idle. Other tasks are email sending, collecting data for telemetry
|
||||
and detecting unused media attachment, for removing them from the file storage.
|
||||
|
||||
## Backend structure
|
||||
|
||||
Penpot backend app code resides under <code class="language-text">backend/src/app</code> path in the main repository.
|
||||
|
||||
@startuml BackendGeneral
|
||||
!include https://raw.githubusercontent.com/plantuml-stdlib/C4-PlantUML/master/C4_Container.puml
|
||||
!define DEVICONS https://raw.githubusercontent.com/tupadr3/plantuml-icon-font-sprites/master/devicons
|
||||
!include DEVICONS/react.puml
|
||||
!include DEVICONS/java.puml
|
||||
!include DEVICONS/clojure.puml
|
||||
!include DEVICONS/postgresql.puml
|
||||
!include DEVICONS/redis.puml
|
||||
!include DEVICONS/chrome.puml
|
||||
|
||||
HIDE_STEREOTYPE()
|
||||
|
||||
Container(frontend_app, "Frontend app", "React / ClojureScript", "", "react")
|
||||
|
||||
System_Boundary(backend, "Backend") {
|
||||
Container(backend_app, "Backend app", "Clojure / JVM", "", "clojure")
|
||||
ContainerDb(db, "Database", "PostgreSQL", "", "postgresql")
|
||||
ContainerDb(redis, "Broker", "Redis", "", "redis")
|
||||
}
|
||||
|
||||
BiRel(frontend_app, backend_app, "Open", "websocket")
|
||||
Rel(frontend_app, backend_app, "Uses", "RPC API")
|
||||
Rel(backend_app, db, "Uses", "SQL")
|
||||
Rel(redis, backend_app, "Subscribes", "pub/sub")
|
||||
Rel(backend_app, redis, "Notifies", "pub/sub")
|
||||
|
||||
@enduml
|
||||
|
||||
```
|
||||
▾ backend/src/app/
|
||||
▸ cli/
|
||||
▸ http/
|
||||
▸ migrations/
|
||||
▸ rpc/
|
||||
▸ setup/
|
||||
▸ srepl/
|
||||
▸ util/
|
||||
▸ tasks/
|
||||
main.clj
|
||||
config.clj
|
||||
http.clj
|
||||
metrics.clj
|
||||
migrations.clj
|
||||
notifications.clj
|
||||
rpc.clj
|
||||
setup.clj
|
||||
srepl.clj
|
||||
worker.clj
|
||||
...
|
||||
```
|
||||
|
||||
* <code class="language-text">main.clj</code> defines the app global settings and the main entry point of the
|
||||
application, served by a JVM.
|
||||
* <code class="language-text">config.clj</code> defines of the configuration options read from linux
|
||||
environment.
|
||||
* <code class="language-text">http</code> contains the HTTP server and the backend routes list.
|
||||
* <code class="language-text">migrations</code> contains the SQL scripts that define the database schema, in
|
||||
the form of a sequence of migrations.
|
||||
* <code class="language-text">rpc</code> is the main module to handle the RPC API calls.
|
||||
* <code class="language-text">notifications.clj</code> is the main module that manages the websocket. It allows
|
||||
clients to subscribe to open files, intercepts update RPC calls and notify
|
||||
them to all subscribers of the file.
|
||||
* <code class="language-text">setup</code> initializes the environment (loads config variables, sets up the
|
||||
database, executes migrations, loads initial data, etc).
|
||||
* <code class="language-text">srepl</code> sets up an interactive REPL shell, with some useful commands to be
|
||||
used to debug a running instance.
|
||||
* <code class="language-text">cli</code> sets a command-line interface, with some more maintenance commands.
|
||||
* <code class="language-text">metrics.clj</code> has some interceptors that watches RPC calls, calculate
|
||||
statistics and other metrics, and send them to external systems to store and
|
||||
analyze.
|
||||
* <code class="language-text">worker.clj</code> and <code class="language-text">tasks</code> define some async tasks that are executed in
|
||||
parallel to the main http server (using java threads), and scheduled in a
|
||||
cron-like table. They are useful to do some garbage collection, data packing
|
||||
and similar periodic maintenance tasks.
|
||||
* <code class="language-text">db.clj</code>, <code class="language-text">emails.clj</code>, <code class="language-text">media.clj</code>, <code class="language-text">msgbus.clj</code>, <code class="language-text">storage.clj</code>,
|
||||
<code class="language-text">rlimits.clj</code> are general libraries to use I/O resources (SQL database,
|
||||
send emails, handle multimedia objects, use REDIS messages, external file
|
||||
storage and semaphores).
|
||||
* <code class="language-text">util/</code> has a collection of generic utility functions.
|
||||
|
||||
### RPC calls
|
||||
|
||||
The RPC (Remote Procedure Call) subsystem consists of a mechanism that allows
|
||||
to expose clojure functions as an HTTP endpoint. We take advantage of being
|
||||
using Clojure at both front and back ends, to avoid needing complex data
|
||||
conversions.
|
||||
|
||||
1. Frontend initiates a "query" or "mutation" call to <code class="language-text">:xxx</code> method, and
|
||||
passes a Clojure object as params.
|
||||
2. Params are string-encoded using
|
||||
[transit](https://github.com/cognitect/transit-clj), a format similar to
|
||||
JSON but more powerful.
|
||||
3. The call is mapped to <code class="language-text"><backend-host>/api/rpc/query/xxx</code> or
|
||||
<code class="language-text"><backend-host>/api/rpc/mutation/xxx</code>.
|
||||
4. The <code class="language-text">rpc</code> module receives the call, decode the parameters and executes the
|
||||
corresponding method inside <code class="language-text">src/app/rpc/queries/</code> or <code class="language-text">src/app/rpc/mutations/</code>.
|
||||
We have created a <code class="language-text">defmethod</code> macro to declare an RPC method and its
|
||||
parameter specs.
|
||||
5. The result value is also transit-encoded and returned to the frontend.
|
||||
|
||||
This way, frontend can execute backend calls like it was calling an async function,
|
||||
with all the power of Clojure data structures.
|
||||
|
||||
### PubSub
|
||||
|
||||
To manage subscriptions to a file, to be notified of changes, we use a redis
|
||||
server as a pub/sub broker. Whenever a user visits a file and opens a
|
||||
websocket, the backend creates a subscription in redis, with a topic that has
|
||||
the id of the file. If the user sends any change to the file, backend sends a
|
||||
notification to this topic, that is received by all subscribers. Then the
|
||||
notification is retrieved and sent to the user via the websocket.
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue