3pp-mirror/neko
1
0
Fork 0
mirror of https://github.com/m1k1o/neko.git synced 2025-07-30 06:50:55 +02:00
Code Issues Projects Releases Packages Wiki Activity

update docs.

Browse source
This commit is contained in:
Miroslav Šedivý 2025-02-16 00:24:37 +01:00
parent 8a015ffd5f
commit bb6fe698ad
5 changed files with 926 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

363
webpage/docs/getting-started/configuration/authentication.md Normal file
View file

@ -0,0 +1,363 @@
---
sidebar_position: 1
---
# Authentication
Authentication is split into two modules:
- **[Member Provider](#member-providers)** - handles authentication and authorization of users, can be used to authenticate users against a database, LDAP, or any other system.
- **[Session Provider](#session-provider)** - handles session management, after the module authenticates the user, it creates a session and handles the session lifecycle.
## Member Profile
A member profile is a structure that describes the user and what the user is allowed to do in the system.
| Field | Description | Type |
|----------------------------|-------------|------|
| `name` | User's name as shown in the UI, must not be unique within the system (not used as an identifier). | string |
| `is_admin` | Whether the user can perform administrative tasks that include managing users, sessions, and settings. | boolean |
| `can_login` | Whether the user can log in to the system and use the HTTP API. | boolean |
| `can_connect` | Whether the user can connect to the room using the WebSocket API (needs `can_login` to be enabled). | boolean |
| `can_watch` | Whether the user can connect to the WebRTC stream and watch the room's audio and video (needs `can_connect` to be enabled). | boolean |
| `can_host` | Whether the user can grab control of the room and control the mouse and keyboard. | boolean |
| `can_share_media` | Whether the user can share their webcam and microphone with the room. | boolean |
| `can_access_clipboard` | Whether the user can read and write to the room's clipboard. | boolean |
| `sends_inactive_cursor` | Whether the user sends the cursor position even when the user is not hosting the room, this is used to show the cursor of the user to other users. | boolean |
| `can_see_inactive_cursors` | Whether the user can see the cursor of other users even when they are not hosting the room. | boolean |
| `plugins` | A map of plugin names and their configuration, plugins can use this to store user-specific settings. | object |
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
<Tabs>
<TabItem value="yaml" label="YAML" default>
```yaml title="Example member profile in YAML"
name: User Name
is_admin: false
can_login: true
can_connect: true
can_watch: true
can_host: true
can_share_media: true
can_access_clipboard: true
sends_inactive_cursor: true
can_see_inactive_cursors: true
plugins:
<key>: <value>
```
</TabItem>
<TabItem value="json" label="JSON">
```json title="Example member profile in JSON"
{
"name": "User Name",
"is_admin": false,
"can_login": true,
"can_connect": true,
"can_watch": true,
"can_host": true,
"can_share_media": true,
"can_access_clipboard": true,
"sends_inactive_cursor": true,
"can_see_inactive_cursors": true,
"plugins": {
"<key>": "<value>"
}
}
```
</TabItem>
</Tabs>
## Member Providers
Member providers are responsible for deciding whether given credentials are valid or not. This validation can either be done against a local database or an external system.
### Multi-User Provider
This is the **default provider** that works exactly like the authentication used to work in v2 of neko.
This provider allows you to define two types of users: **regular** users and **admins**. Which user is an admin is determined by the password they provide when logging in. If the password is correct, the user is an admin; otherwise, they are a regular user. Based on those profiles, the users are generated on demand when they log in and they are removed when they log out. Their username is prefixed with 5 random characters to avoid conflicts when multiple users share the same username.
Profiles for regular users and admins are optional, if not provided, the default profiles are used (see below in the example configuration).
```yaml title="config.yaml"
member:
provider: multiuser
multiuser:
# Password for admins, in plain text.
admin_password: "adminPassword"
# Profile fields as described above
admin_profile:
...
# Password for regular users, in plain text.
user_password: "userPassword"
# Profile fields as described above
user_profile:
...
```
<details>
<summary>See example configuration</summary>
The default profiles for regular users and admins are as follows, highlighting the differences between them.
```yaml title="config.yaml"
member:
provider: multiuser
multiuser:
admin_password: "admin"
admin_profile:
name: "" # if empty, the login username is used
# highlight-start
is_admin: true
# highlight-end
can_login: true
can_connect: true
can_watch: true
can_host: true
can_share_media: true
can_access_clipboard: true
sends_inactive_cursor: true
# highlight-start
can_see_inactive_cursors: true
# highlight-end
user_password: "neko"
user_profile:
name: "" # if empty, the login username is used
# highlight-start
is_admin: false
# highlight-end
can_login: true
can_connect: true
can_watch: true
can_host: true
can_share_media: true
can_access_clipboard: true
sends_inactive_cursor: true
# highlight-start
can_see_inactive_cursors: false
# highlight-end
```
</details>
:::tip
For easier configuration, you can specify only passwords using environment variables:
```yaml title="docker-compose.yml"
environment:
NEKO_MEMBER_MULTIUSER_USER_PASSWORD: "neko"
NEKO_MEMBER_MULTIUSER_ADMIN_PASSWORD: "admin"
```
:::
### File Provider
This provider reads the user's credentials from a file. It is useful for small deployments where you don't want to set up a database or LDAP server and still want to have persistent users.
```yaml title="config.yaml"
member:
provider: file
file:
# Absolute path to the file containing the users and their passwords.
path: /opt/neko/members.json
# Whether the passwords are hashed using sha256 or not.
hash: true
```
It allows you to store the user's credentials in a JSON file. The JSON structure maps user logins to their passwords and profiles.
```json title="members.json"
{
"<user_login>": {
"password": "<user_password>",
"profile": /* Member Profile, as described above */
}
}
```
You can leave the file empty and add users later using the HTTP API.
<details>
<summary>See example `members.json` file</summary>
We have two users, `admin` and `user` with their passwords and profiles. `admin` is a regular user, while `user` is an admin.
Please note that the passwords are stored in plain text. To store them securely, set the `hash` field to `true` in the configuration. After that, the passwords are expected to be hashed using the bcrypt algorithm.
```json title="members.json"
{
"admin": {
"password": "admin",
"profile": {
"name": "Administrator",
"is_admin": true,
"can_login": true,
"can_connect": true,
"can_watch": true,
"can_host": true,
"can_share_media": true,
"can_access_clipboard": true,
"sends_inactive_cursor": true,
"can_see_inactive_cursors": true,
"plugins": {}
}
},
"user": {
"password": "neko",
"profile": {
"name": "User",
"is_admin": false,
"can_login": true,
"can_connect": true,
"can_watch": true,
"can_host": true,
"can_share_media": true,
"can_access_clipboard": true,
"sends_inactive_cursor": true,
"can_see_inactive_cursors": false,
"plugins": {}
}
}
}
```
If you want to hash the passwords, you can use the following command to generate a sha256 hash:
```bash
echo -n "password" | sha256sum
```
</details>
### Object Provider
This provider is the same as the file provider, but it saves the users only in memory. That means that the users are lost when the server is restarted. However, the default users can be set in the configuration file. The difference from the multi-user provider is that the users are not generated on demand and we define exactly which users with their passwords and profiles are allowed to log in. They cannot be logged in twice with the same username.
```yaml title="config.yaml"
member:
provider: object
object:
# List of users with their passwords and profiles
- username: "admin"
# Password in plain text
password: "admin"
# Profile fields as described above
profile:
...
```
<details>
<summary>See example configuration</summary>
We have two users, `admin` and `user` with their passwords and profiles. `admin` is an admin, while `user` is a regular user.
```yaml title="config.yaml"
member:
provider: object
object:
users:
- username: "admin"
password: "admin"
profile:
name: "Administrator"
is_admin: true
can_login: true
can_connect: true
can_watch: true
can_host: true
can_share_media: true
can_access_clipboard: true
sends_inactive_cursor: true
can_see_inactive_cursors: true
- username: "user"
password: "neko"
profile:
name: "User"
is_admin: false
can_login: true
can_connect: true
can_watch: true
can_host: true
can_share_media: true
can_access_clipboard: true
sends_inactive_cursor: true
can_see_inactive_cursors: false
```
</details>
### No-Auth Provider
This provider allows any user to log in without any authentication. It is useful for testing and development purposes.
```yaml title="config.yaml"
member:
provider: noauth
```
:::danger
Do not use this provider in production environments unless you know exactly what you are doing. It allows anyone to log in and control neko as an admin.
:::
## Session Provider
Currently, there are only two providers available for sessions: **memory** and **file**.
Simply by specifying the `session.file` to a file path, the session provider will store the sessions in a file. Otherwise, the sessions are stored in memory and are lost when the server is restarted.
```yaml title="config.yaml"
session:
file: /opt/neko/sessions.json
```
:::info
In the future, we plan to add more session providers, such as Redis, PostgreSQL, etc. So the Configuration Options may change.
:::
## API User
The API User is a special user that is used to authenticate the HTTP API requests. It cannot connect to the room, but it can perform administrative tasks. The API User does not have a password but only a token that is used to authenticate the requests. If the token is not set, the API User is disabled.
```yaml title="config.yaml"
session:
api_token: "apiToken"
```
:::tip
This user is useful in some situations when the rooms are generated by the server and the token is guaranteed to be random every time a short-lived room is run. It is not a good idea to define this token for long-lived rooms, as it can be stolen and used to perform administrative tasks.
You can generate a random token using the following command:
```bash
openssl rand -hex 32
```
:::
## Cookies
The authentication between the client and the server can be done using cookies or the `Authorization` header. The cookies are used by default, but you can disable them by setting the `session.cookie.enabled` to `false`.
:::warning
If you disable the cookies, the token will be sent to the client in the login response and saved in local storage. This is less secure than using cookies, as the token **can be stolen using XSS attacks**. Therefore, it is recommended to use cookies.
:::
```yaml title="config.yaml"
session:
cookie:
# Whether the cookies are enabled or not.
enabled: true
# Name of the cookie used to store the session.
name: "NEKO_SESSION"
# Expiration time of the cookie in seconds.
expiration: 86400
# Whether the cookie is secure (HTTPS only) or not.
secure: true
```
:::info
The `session.cookie.secure` is set to `true` by default, which means that the cookie is only sent over HTTPS. If you are using HTTP, you should really consider using HTTPS. Only for testing and development purposes should you consider setting it to `false`.
:::

480
webpage/docs/getting-started/configuration/capture.md Normal file
View file

@ -0,0 +1,480 @@
---
sidebar_position: 2
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Audio & Video Capture
This guide will show you how to configure the audio and video capture settings in neko.
Neko uses [Gstreamer](https://gstreamer.freedesktop.org/) to capture and encode audio and video in the following scenarios:
- WebRTC clients use the [Video](#webrtc-video) and [Audio](#webrtc-audio) pipelines to receive the audio and video streams from the server.
- The [Broadcast](#broadcast) feature allows you to broadcast the audio and video to a third-party service using RTMP.
- The WebRTC Fallback mechanism allows you to capture the display in the form of JPEG images and serve them over HTTP using [Screencast](#screencast).
- Clients can share their [Webcam](#webcam) and [Microphone](#microphone) with the server using WebRTC.
## WebRTC Video
Neko allows you to capture the display and encode it in real-time using Gstreamer. The encoded video is then sent to the client using WebRTC. This allows you to share the display with the client in real-time.
There can exist multiple video pipelines in neko that are referenced by their unique pipeline id. Each video pipeline can have its own configuration settings and clients can either choose which pipeline they want to use or let neko choose the best pipeline for them.
:::info Limitation
All video pipelines must use the same video codec (defined in the `capture.video.codec` setting).
:::
The Gstreamer pipeline is started when the first client requests the video stream and is stopped after the last client disconnects.
```yaml title="config.yaml"
capture:
video:
display: "<display_name>"
codec: "vp8" # default video codec
ids: [ <pipeline_id1>, <pipeline_id2>, ... ]
pipelines:
<pipeline_id1>: <pipeline_config>
<pipeline_id2>: <pipeline_config>
...
```
- `display` is the name of the [X display](https://www.x.org/wiki/) that you want to capture. If not specified, the environment variable `DISPLAY` will be used.
- `codec` available codecs are `vp8`, `vp9`, `av1`, `h264`. [Supported video codecs](https://developer.mozilla.org/en-US/docs/Web/Media/Guides/Formats/WebRTC_codecs#supported_video_codecs) are dependent on the WebRTC implementation used by the client, `vp8` and `h264` are supported by all WebRTC implementations.
- `ids` is a list of pipeline ids that are defined in the `pipelines` section. The first pipeline in the list will be the default pipeline.
- `pipelines` is a dictionary of pipeline configurations. Each pipeline configuration is defined by a unique pipeline id. They can be defined in two ways: either by building the pipeline dynamically using [Expression-Driven Configuration](#expression-driven-configuration) or by defining the pipeline using a [Gstreamer Pipeline Description](#gstreamer-pipeline-description).
### Expression-Driven Configuration
Expression allows you to build the pipeline dynamically based on the current resolution and framerate of the display. Expressions are evaluated using the [gval](https://github.com/PaesslerAG/gval) library. Available variables are `width`, `height`, and `fps` of the display at the time of capture.
```yaml title="config.yaml"
capture:
video:
...
pipelines:
<pipeline_id>:
width: "<expression>"
height: "<expression>"
fps: "<expression>"
gst_prefix: "<gst_pipeline>"
gst_encoder: "<gst_encoder_name>"
gst_params:
<param_name>: "<expression>"
gst_suffix: "<gst_pipeline>"
show_pointer: true
```
- `width`, `height`, and `fps` are the expressions that are evaluated to get the stream resolution and framerate. They can be different from the display resolution and framerate if downscaling or upscaling is desired.
- `gst_prefix` and `gst_suffix` allow you to add custom Gstreamer elements before and after the encoder. Both parameters need to start with `!` and then be followed by the Gstreamer elements.
- `gst_encoder` is the name of the Gstreamer encoder element, such as `vp8enc` or `x264enc`.
- `gst_params` are the parameters that are passed to the encoder element specified in `gst_encoder`.
- `show_pointer` is a boolean value that determines whether the mouse pointer should be captured or not.
<details>
<summary>Example pipeline configuration</summary>
<Tabs>
<TabItem value="vp8" label="VP8 configuration">
```yaml title="config.yaml"
capture:
video:
codec: vp8
# HQ is the default pipeline
ids: [ hq, lq ]
pipelines:
hq:
fps: 25
gst_encoder: vp8enc
gst_params:
target-bitrate: round(3072 * 650)
cpu-used: 4
end-usage: cbr
threads: 4
deadline: 1
undershoot: 95
buffer-size: (3072 * 4)
buffer-initial-size: (3072 * 2)
buffer-optimal-size: (3072 * 3)
keyframe-max-dist: 25
min-quantizer: 4
max-quantizer: 20
lq:
fps: 25
gst_encoder: vp8enc
gst_params:
target-bitrate: round(1024 * 650)
cpu-used: 4
end-usage: cbr
threads: 4
deadline: 1
undershoot: 95
buffer-size: (1024 * 4)
buffer-initial-size: (1024 * 2)
buffer-optimal-size: (1024 * 3)
keyframe-max-dist: 25
min-quantizer: 4
max-quantizer: 20
```
</TabItem>
<TabItem value="h264" label="H264 configuration">
```yaml title="config.yaml"
capture:
video:
codec: h264
ids: [ main ]
pipelines:
main:
width: (width / 3) * 2
height: (height / 3) * 2
fps: 20
gst_prefix: "! video/x-raw,format=I420"
gst_encoder: "x264enc"
gst_params:
threads: 4
bitrate: 4096
key-int-max: 15
byte-stream: true
tune: zerolatency
speed-preset: veryfast
gst_suffix: "! video/x-h264,stream-format=byte-stream"
```
</TabItem>
</Tabs>
</details>
### Gstreamer Pipeline Description
If you want to define the pipeline using a [Gstreamer pipeline description](https://gstreamer.freedesktop.org/documentation/tools/gst-launch.html?gi-language=c#pipeline-description), you can do so by setting the `gst_pipeline` parameter.
```yaml title="config.yaml"
capture:
video:
...
pipelines:
<pipeline_id>:
gst_pipeline: "<gstreamer_pipeline>"
```
Since now you have to define the whole pipeline, you need to specify the src element to get the video frames and the sink element to send the encoded video frames to neko. In your pipeline, you can use `{display}` as a placeholder for the display name that will be replaced by the actual display name at runtime. You need to set the `name` property of the sink element to `appsink` so that neko can capture the video frames.
Your typical pipeline string would look like this:
```
ximagesrc display-name={display} show-pointer=true use-damage=false ! <your_elements> ! appsink name=appsink"
```
See documentation for [ximagesrc](https://gstreamer.freedesktop.org/documentation/ximagesrc/index.html) and [appsink](https://gstreamer.freedesktop.org/documentation/app/appsink.html) for more information.
<details>
<summary>Example pipeline configuration</summary>
<Tabs>
<TabItem value="vp8" label="VP8 configuration">
```yaml title="config.yaml"
capture:
video:
codec: vp8
ids: [ hq, lq ]
pipelines:
hq:
gst_pipeline: |
ximagesrc display-name={display} show-pointer=true use-damage=false
! videoconvert
! vp8enc
target-bitrate=3072000
cpu-used=4
end-usage=cbr
threads=4
deadline=1
undershoot=95
buffer-size=12288
buffer-initial-size=6144
buffer-optimal-size=9216
keyframe-max-dist=25
min-quantizer=4
max-quantizer=20
! appsink name=appsink
lq:
gst_pipeline: |
ximagesrc display-name={display} show-pointer=true use-damage=false
! videoconvert
! vp8enc
target-bitrate=1024000
cpu-used=4
end-usage=cbr
threads=4
deadline=1
undershoot=95
buffer-size=4096
buffer-initial-size=2048
buffer-optimal-size=3072
keyframe-max-dist=25
min-quantizer=4
max-quantizer=20
! appsink name=appsink
```
</TabItem>
<TabItem value="h264" label="H264 configuration">
```yaml title="config.yaml"
capture:
video:
codec: h264
ids: [ main ]
pipelines:
main:
gst_pipeline: |
ximagesrc display-name={display} show-pointer=true use-damage=false
! videoconvert
! x264enc
threads=4
bitrate=4096
key-int-max=15
byte-stream=true
tune=zerolatency
speed-preset=veryfast
! video/x-h264,stream-format=byte-stream
! appsink name=appsink
```
</TabItem>
</Tabs>
</details>
## WebRTC Audio
Only one audio pipeline can be defined in neko. The audio pipeline is used to capture and encode audio, similar to the video pipeline. The encoded audio is then sent to the client using WebRTC.
The Gstreamer pipeline is started when the first client requests the video stream and is stopped after the last client disconnects.
```yaml title="config.yaml"
capture:
audio:
device: "audio_output.monitor" # default audio device
codec: "opus" # default audio codec
pipeline: "<gstreamer_pipeline>"
```
- `device` is the name of the [pulseaudio device](https://wiki.archlinux.org/title/PulseAudio/Examples) that you want to capture. If not specified, the default audio device will be used.
- `codec` available codecs are `opus`, `g722`, `pcmu`, `pcma`. [Supported audio codecs](https://developer.mozilla.org/en-US/docs/Web/Media/Guides/Formats/WebRTC_codecs#supported_audio_codecs) are dependent on the WebRTC implementation used by the client, `opus` is supported by all WebRTC implementations.
- `pipeline` is the Gstreamer pipeline description that is used to capture and encode audio. You can use `{device}` as a placeholder for the audio device name that will be replaced by the actual device name at runtime.
<details>
<summary>Example pipeline configuration</summary>
```yaml title="config.yaml"
capture:
audio:
codec: opus
pipeline: |
pulsesrc device={device}
! audioconvert
! opusenc
bitrate=320000
! appsink name=appsink
```
</details>
## Broadcast
Neko allows you to broadcast out-of-the-box the display and audio capture to a third-party service. This can be used to broadcast the display and audio to a streaming service like [Twitch](https://www.twitch.tv/) or [YouTube](https://www.youtube.com/), or to a custom RTMP server like [OBS](https://obsproject.com/), [Nginx RTMP module](https://github.com/arut/nginx-rtmp-module), or [MediaMTX](https://github.com/bluenviron/mediamtx).
The Gstreamer pipeline is started when the broadcast is started and is stopped when the broadcast is stopped regardless of the clients connected.
```yaml title="config.yaml"
capture:
broadcast:
audio_bitrate: 128 # in KB/s
video_bitrate: 4096 # in KB/s
preset: "veryfast"
pipeline: "<gstreamer_pipeline>"
url: "rtmp://<server>/<application>/<stream_key>"
autostart: true
```
The default encoder uses `h264` for video and `aac` for audio, muxed in the `flv` container and sent over the `rtmp` protocol. You can change the encoder settings by setting a custom Gstreamer pipeline description in the `pipeline` parameter.
- `audio_bitrate` and `video_bitrate` are the bitrate settings for the default audio and video encoders expressed in kilobits per second.
- `preset` is the encoding speed preset for the default video encoder. See available presets [here](https://gstreamer.freedesktop.org/documentation/x264/index.html?gi-language=c#GstX264EncPreset).
- `pipeline` when set, encoder settings above are ignored and the custom Gstreamer pipeline description is used. In the pipeline, you can use `{display}`, `{device}` and `{url}` as placeholders for the X display name, pulseaudio audio device name, and broadcast URL respectively.
- `url` is the URL of the RTMP server where the broadcast will be sent. This can be set later using the API if the URL is not known at the time of configuration or is expected to change.
- `autostart` is a boolean value that determines whether the broadcast should start automatically when neko starts, works only if the URL is set.
<details>
<summary>Example pipeline configuration</summary>
<Tabs>
<TabItem value="x264" label="X264 configuration">
```yaml title="config.yaml"
capture:
broadcast:
audio_bitrate: 128
video_bitrate: 4096
preset: veryfast
pipeline: |
flvmux name=mux
! rtmpsink location={url}
pulsesrc device={device}
! audio/x-raw,channels=2
! audioconvert
! voaacenc
! mux.
ximagesrc display-name={display} show-pointer=false use-damage=false
! video/x-raw,framerate=28/1
! videoconvert
! queue
! x264enc bframes=0 key-int-max=0 byte-stream=true tune=zerolatency speed-preset=veryfast
! mux.
```
</TabItem>
<TabItem value="nvh264enc" label="NVENC H264 configuration">
This configuration requires [Nvidia GPU](https://developer.nvidia.com/cuda-gpus) with [NVENC](https://developer.nvidia.com/nvidia-video-codec-sdk) support.
```yaml title="config.yaml"
capture:
broadcast:
audio_bitrate: 128
video_bitrate: 4096
preset: veryfast
pipeline: |
flvmux name=mux
! rtmpsink location={url}
pulsesrc device={device}
! audio/x-raw,channels=2
! audioconvert
! voaacenc
! mux.
ximagesrc display-name={display} show-pointer=false use-damage=false
! video/x-raw,framerate=30/1
! videoconvert
! queue
! video/x-raw,format=NV12
! nvh264enc name=encoder preset=low-latency-hq gop-size=25 spatial-aq=true temporal-aq=true bitrate=2800 vbv-buffer-size=2800 rc-mode=6
! h264parse config-interval=-1
! video/x-h264,stream-format=byte-stream,profile=high
! h264parse
! mux.
```
</TabItem>
</Tabs>
</details>
## Screencast
As a fallback mechanism, neko can capture the display in the form of JPEG images and the client can request these images over HTTP. This is useful when the client does not support WebRTC or when the client is not able to establish a WebRTC connection, or there is a temporary issue with the WebRTC connection and the client should not miss the content being shared.
:::note
This is a fallback mechanism and should not be used as a primary video stream because of the high latency, low quality, and high bandwidth requirements.
:::
The Gstreamer pipeline is started in the background when the first client requests the screencast and is stopped after a period of inactivity.
```yaml title="config.yaml"
capture:
screencast:
enabled: true
rate: "10/1"
quality: 60
pipeline: "<gstreamer_pipeline>"
```
- `enabled` is a boolean value that determines whether the screencast is enabled or not.
- `rate` is the framerate of the screencast. It is expressed as a fraction of frames per second, for example, `10/1` means 10 frames per second.
- `quality` is the quality of the JPEG images. It is expressed as a percentage, for example, `60` means 60% quality.
- `pipeline` when set, the default pipeline settings above are ignored and the custom Gstreamer pipeline description is used. In the pipeline, you can use `{display}` as a placeholder for the X display name.
<details>
<summary>Example pipeline configuration</summary>
```yaml title="config.yaml"
capture:
screencast:
enabled: true
rate: "10/1"
quality: 60
pipeline: |
ximagesrc display-name={display} show-pointer=true use-damage=false
! video/x-raw,framerate=10/1
! videoconvert
! queue
! jpegenc quality=60
! appsink name=appsink
```
</details>
## Webcam
:::danger
This feature is experimental and may not work on all platforms.
:::
Neko allows you to capture the webcam on the client machine and send it to the server using WebRTC. This can be used to share the webcam feed with the server.
The Gstreamer pipeline is started when the client shares their webcam and is stopped when the client stops sharing the webcam. Maximum one webcam pipeline can be active at a time.
```yaml title="config.yaml"
capture:
webcam:
enabled: true
device: "/dev/video0" # default webcam device
width: 640
height: 480
```
- `enabled` is a boolean value that determines whether the webcam capture is enabled or not.
- `device` is the name of the [video4linux device](https://www.kernel.org/doc/html/v4.12/media/v4l-drivers/index.html) that will be used as a virtual webcam.
- `width` and `height` are the resolution of the virtual webcam feed.
In order to use the webcam feature, the server must have the [v4l2loopback](https://github.com/v4l2loopback/v4l2loopback) kernel module installed and loaded. The module can be loaded using the following command:
```bash
# Install the required packages (Debian/Ubuntu)
sudo apt install v4l2loopback-dkms v4l2loopback-utils linux-headers-`uname -r` linux-modules-extra-`uname -r`
# Load the module with exclusive_caps=1 to allow multiple applications to access the virtual webcam
sudo modprobe v4l2loopback exclusive_caps=1
```
This is needed even if neko is running inside a Docker container. In that case, the v4l2loopback module must be loaded on the host machine and the device must be mounted inside the container.
```yaml title="docker-compose.yml"
services:
neko:
...
# highlight-start
devices:
- /dev/video0:/dev/video0
# highlight-end
...
```
## Microphone
Neko allows you to capture the microphone on the client machine and send it to the server using WebRTC. This can be used to share the microphone feed with the server.
The Gstreamer pipeline is started when the client shares their microphone and is stopped when the client stops sharing the microphone. Maximum one microphone pipeline can be active at a time.
```yaml title="config.yaml"
capture:
microphone:
enabled: true
device: "audio_input"
```
- `enabled` is a boolean value that determines whether the microphone capture is enabled or not.
- `device` is the name of the [pulseaudio device](https://wiki.archlinux.org/title/PulseAudio/Examples) that will be used as a virtual microphone.

79
webpage/docs/getting-started/configuration/desktop.md Normal file
View file

@ -0,0 +1,79 @@
---
sidebar_position: 3
---
# Desktop environment
This section describes how to configure the desktop environment inside neko.
Neko uses the [X Server](https://www.x.org/archive/X11R7.6/doc/man/man1/Xserver.1.xhtml) as the display server with [Openbox](http://openbox.org/wiki/Main_Page) as the default window manager. For audio, [PulseAudio](https://www.freedesktop.org/wiki/Software/PulseAudio/) is used.
```yaml title="config.yaml"
desktop:
display: "<display>"
screen: "1280x720@30" # default
```
- `display` refers to the X server that is running on the system. If it is not specified, the environment variable `DISPLAY` is used. The same display is referred to in the [Capture](capture#webrtc-video) configuration to capture the screen. In most cases, we want to use the same display for both.
- `screen` refers to the screen resolution and refresh rate. The format is `<width>x<height>@<refresh rate>`. If not specified, the default is `1280x720@30`.
:::tip
You can specify the screen resolution using the environment variable `NEKO_DESKTOP_SCREEN`.
:::
## Input devices
Neko uses the [XTEST Extension Library](https://www.x.org/releases/X11R7.7/doc/libXtst/xtestlib.html) to simulate keyboard and mouse events. However, for more advanced input devices like touchscreens, we need to use a custom driver that can be loaded as a plugin to the X server and then neko can connect to it.
:::note
Currently, only touchscreens are supported through the custom driver.
:::
```yaml title="config.yaml"
desktop:
input:
enabled: true # default
socket: "/tmp/xf86-input-neko.sock" # default
```
- `enabled` enables the input device support. If not specified, the default is `false`.
- `socket` refers to the socket file that the custom driver creates. If not specified, the default is `/tmp/xf86-input-neko.sock`.
:::info
When using Docker, the custom driver is already included in the image and the socket file is created at `/tmp/xf86-input-neko.sock`. Therefore, no additional configuration is needed.
:::
## Unminimize
Most of the time, only a single application is used in the minimal desktop environment without any taskbar or desktop icons. It could happen that the user accidentally minimizes the application and then it is not possible to restore it. To prevent this, we can use the `unminimize` feature that simply listens for the minimize event and restores the window back to the original state.
```yaml title="config.yaml"
desktop:
unminimize: true # default
```
## Upload Drop
The upload drop is a feature that allows the user to upload files to the application by dragging and dropping them into the application window. The files are then uploaded to the application and the application can process them.
The current approach is to catch the drag and drop events on the client side, upload them to the server along with the coordinates of the drop event, and then open an invisible overlay window on the server that has set the file path to the uploaded file and allows it to be dragged and dropped into the application. Then the mouse events are simulated to drag the file from the overlay window to the application window.
```yaml title="config.yaml"
desktop:
upload_drop: true # default
```
## File Chooser Dialog
:::danger
This feature is experimental and may not work as expected.
:::
The file chooser dialog is a feature that allows handling the file chooser dialog in the application (for example, when uploading a file) externally. This means that the file chooser dialog is not displayed inside the neko desktop environment, but the neko client is requested to upload the file from the local filesystem.
The current approach is to put the file chooser dialog in the background as soon as it is displayed, prompt the user to upload the file, and then select this file in the file chooser dialog by simulating the keyboard events to navigate to the file and press the open button. **This is very error-prone and may not work as expected.**
```yaml title="config.yaml"
desktop:
file_chooser_dialog: false # default
```

3
webpage/docs/getting-started/configuration/index.mdx → webpage/docs/getting-started/configuration/index.md
View file

@ -5,6 +5,9 @@ sidebar_position: 5
# Configuration
Configuration options can be set using Environment Variables, as an argument to the CLI, or in a configuration file.
We use the [Viper](https://github.com/spf13/viper) library to manage configuration. Viper supports JSON, TOML, YAML, HCL, and Java properties files.
The configuration file is optional and is not required for Neko to run.
If a configuration file is present, it will be read in and merged with the default configuration values.
Highest priority is given to the Environment Variables, followed by CLI arguments, and then the configuration file.