docusaurus/website/docs/i18n/i18n-crowdin.mdx
Sébastien Lorber db79d462ab
feat(v2): auto-generated sidebars, frontmatter-less sites (#4582)
* POC of autogenerated sidebars

* use combine-promises utility lib

* autogenerated sidebar poc working

* Revert "autogenerated sidebar poc working"

This reverts commit c81da980

* POC of auto-generated sidebars for community docs

* update tests

* add initial test suite for autogenerated sidebars + fix some edge cases

* Improve autogen sidebars: strip more number prefixes in folder breadcrumb + slugs

* fix typo!

* Add tests for partially generated sidebars + fix edge cases + extract sidebar generation code

* Ability to read category metadatas file from a file in the category

* fix tests

* change position of API

* ability to extract number prefix

* stable system to enable position frontmatter

* fix tests for autogen sidebar position

* renamings

* restore community sidebars

* rename frontmatter position -> sidebar_position

* make sidebarItemsGenerator fn configurable

* minor changes

* rename dirPath => dirName

* Make the init template use autogenerated sidebars

* fix options

* fix docusaurus site: remove test docs

* add _category_ file to docs pathsToWatch

* add _category_ file to docs pathsToWatch

* tutorial: use sidebar_position instead of file number prefixes

* Adapt Docusaurus tutorial for autogenerated sidebars

* remove slug: /

* polish the homepage template

* rename _category_ sidebar_position to just "position"

* test for custom sidebarItemsGenerator fn

* fix category metadata + add link to report tutorial issues

* fix absolute path breaking tests

* fix absolute path breaking tests

* Add test for floating number sidebar_position

* add sidebarItemsGenerator unit tests

* add processSidebars unit tests

* Fix init template broken links

* windows test

* increase code translations test timeout

* cleanup mockCategoryMetadataFiles after windows test fixed

* update init template positions

* fix windows tests

* fix comment

* Add autogenerated sidebar items documentation + rewrite the full sidebars page doc

* add useful comment

* fix code block title
2021-04-15 16:20:11 +02:00

528 lines
16 KiB
Text

---
id: crowdin
title: i18n - Using Crowdin
slug: /i18n/crowdin
---
The i18n system of Docusaurus is **decoupled from any translation software**.
You can integrate Docusaurus with the **tools and SaaS of your choice**, as long as you put the **translation files at the correct location**.
We document the usage of [Crowdin](https://crowdin.com/), as **one** possible **integration example**.
:::caution
This is **not an endorsement of Crowdin** as the unique choice to translate a Docusaurus site, but it is successfully used by Facebook to translate documentation projects such as [Jest](https://jestjs.io/), [Docusaurus](https://docusaurus.io/) and [ReasonML](https://reasonml.github.io/).
Refer to the **[Crowdin documentation](https://support.crowdin.com/)** and **[Crowdin support](mailto:support@crowdin.com)** for help.
:::
:::tip
Use this **[community-driven GitHub issue](https://github.com/facebook/docusaurus/discussions/4052)** to discuss anything related to Docusaurus + Crowdin.
:::
## Crowdin overview {#crowdin-overview}
Crowdin is a translation SaaS, offering a [free plan for open-source projects](https://crowdin.com/page/open-source-project-setup-request).
We recommend the following translation workflow:
- **Upload sources** to Crowdin (untranslated files)
- Use Crowdin to **translate the content**
- **Download translations** from Crowdin (localized translation files)
Crowdin provides a [CLI](https://support.crowdin.com/cli-tool/) to **upload sources** and **download translations**, allowing you to automate the translation process.
The [`crowdin.yml` configuration file](https://support.crowdin.com/configuration-file/) is convenient for Docusaurus, and permits to **download the localized translation files at the expected location** (in `i18n/<locale>/..`).
Read the **[official documentation](https://support.crowdin.com/)** to know more about advanced features and different translation workflows.
## Crowdin tutorial {#crowdin-tutorial}
This is a walk-through of using Crowdin to translate a newly initialized English Docusaurus website into French, and assume you already followed the [i18n tutorial](./i18n-tutorial.md).
The end result can be seen at [docusaurus-crowdin-example.netlify.app](https://docusaurus-crowdin-example.netlify.app/) ([repository](https://github.com/slorber/docusaurus-crowdin-example)).
### Prepare the Docusaurus site {#prepare-the-docusaurus-site}
Initialize a new Docusaurus site:
```bash
npx @docusaurus/init@latest init website classic
```
Add the site configuration for the French language:
```js title="docusaurus.config.js"
module.exports = {
i18n: {
defaultLocale: 'en',
locales: ['en', 'fr'],
},
themeConfig: {
navbar: {
items: [
// ...
{
type: 'localeDropdown',
position: 'left',
},
// ...
],
},
},
// ...
};
```
Translate the homepage:
```jsx title="src/pages/index.js"
import React from 'react';
import Translate from '@docusaurus/Translate';
import Layout from '@theme/Layout';
export default function Home() {
return (
<Layout>
<h1 style={{margin: 20}}>
<Translate description="The homepage main heading">
Welcome to my Docusaurus translated site!
</Translate>
</h1>
</Layout>
);
}
```
### Create a Crowdin project {#create-a-crowdin-project}
Sign up on [Crowdin](https://crowdin.com/), and create a project.
Use English as source language, and French as target language.
![Create a Crowdin project with english as source language, and french as target language](/img/crowdin/crowdin-create-project.png)
Your project is created, but it is empty for now. We will upload the files to translate in the next steps.
### Create the Crowdin configuration {#create-the-crowdin-configuration}
This configuration ([doc](https://support.crowdin.com/configuration-file/)) provides a mapping for the Crowdin CLI to understand:
- Where to find the source files to upload (JSON and Markdown)
- Where to download the files after translation (in `i18n/<locale>`)
Create `crowdin.yml` in `website`:
```yml title="crowdin.yml"
project_id: '123456'
api_token_env: 'CROWDIN_PERSONAL_TOKEN'
preserve_hierarchy: true
files: [
# JSON translation files
{
source: '/i18n/en/**/*',
translation: '/i18n/%two_letters_code%/**/%original_file_name%',
},
# Docs Markdown files
{
source: '/docs/**/*',
translation: '/i18n/%two_letters_code%/docusaurus-plugin-content-docs/current/**/%original_file_name%',
},
# Blog Markdown files
{
source: '/blog/**/*',
translation: '/i18n/%two_letters_code%/docusaurus-plugin-content-blog/**/%original_file_name%',
},
]
```
Crowdin has its own syntax for declaring source/translation paths:
- `**/*`: everything in a subfolder
- `%two_letters_code%`: the 2-letters variant of Crowdin target languages (`fr` in our case)
- `**/%original_file_name%`: the translations will preserve the original folder/file hierarchy
:::info
The Crowdin CLI warnings are not always easy to understand.
We advise to:
- change one thing at a time
- re-upload sources after any configuration change
- use paths starting with `/` (`./` does not work)
- avoid fancy globbing patterns like `/docs/**/*.(md|mdx)` (does not work)
:::
#### Access token {#access-token}
The `api_token_env` attribute defines the **env variable name** read by the Crowdin CLI.
You can obtain a `Personal Access Token` on [your personal profile page](https://crowdin.com/settings#api-key).
:::tip
You can keep the default value `CROWDIN_PERSONAL_TOKEN`, and set this environment variable and on your computer and on the CI server to the generated access token.
:::
:::caution
A Personal Access Tokens grant **read-write access to all your Crowdin projects**.
You should **not commit** it, and it may be a good idea to create a dedicated **Crowdin profile for your company** instead of using a personal account.
:::
#### Other configuration fields {#other-configuration-fields}
- `project_id`: can be hardcoded, and is found on `https://crowdin.com/project/<MY_PROJECT_NAME>/settings#api`
- `preserve_hierarchy`: preserve the folder's hierarchy of your docs on Crowdin UI instead of flattening everything
### Install the Crowdin CLI {#install-the-crowdin-cli}
This tutorial use the CLI in version `3.5.2`, but we expect `3.x` releases to keep working.
Install the Crowdin CLI as a NPM package to your Docusaurus site:
```bash npm2yarn
npm install @crowdin/cli@3
```
Add a `crowdin` script:
```json title="package.json"
{
"scripts": {
"crowdin": "crowdin"
}
}
```
Test that you can run the Crowdin CLI:
```bash npm2yarn
npm run crowdin -- --version
```
Set the `CROWDIN_PERSONAL_TOKEN` env variable on your computer, to allow the CLI to authenticate with the Crowdin API.
:::tip
Temporarily, you can hardcode your personal token in `crowdin.yml` with `api_token: 'MY-TOKEN'`.
:::
### Upload the sources {#upload-the-sources}
Generate the JSON translation files for the default language in `website/i18n/en`:
```bash npm2yarn
npm run write-translations
```
Upload all the JSON and Markdown translation files:
```bash npm2yarn
npm run crowdin upload
```
![Crowdin CLI uploading Docusaurus source files](/img/crowdin/crowdin-upload-sources-cli.png)
Your source files are now visible on the Crowdin interface: `https://crowdin.com/project/<MY_PROJECT_NAME>/settings#files`
![Crowdin UI showing Docusaurus source files](/img/crowdin/crowdin-source-files.png)
### Translate the sources {#translate-the-sources}
On `https://crowdin.com/project/<MY_PROJECT_NAME>`, click on the French target language.
![Crowdin UI showing French translation files](/img/crowdin/crowdin-french-translations.png)
Translate some Markdown files.
![Crowdin UI to translate a Markdown file](/img/crowdin/crowdin-translate-markdown.png)
:::tip
Use `Hide String` to make sure translators **don't translate things that should not be**:
- Frontmatter: `id`, `slug`, `tags` ...
- Admonitions: `:::`, `:::note`, `:::tip` ...
![Crowdin UI hide string](/img/crowdin/crowdin-hide-string.png)
:::
Translate some JSON files.
![Crowdin UI to translate a JSON file](/img/crowdin/crowdin-translate-json.png)
:::info
The `description` attribute of JSON translation files is visible on Crowdin to help translate the strings.
:::
:::tip
**[Pre-translate](https://support.crowdin.com/pre-translation-via-machine/)** your site, and **fix pre-translation mistakes manually** (enable the Global Translation Memory in settings first).
Use the `Hide String` feature first, as Crowdin is pre-translating things too optimistically.
:::
### Download the translations {#download-the-translations}
Use the Crowdin CLI to download the translated JSON and Markdown files.
```bash npm2yarn
npm run crowdin download
```
The translated content should be downloaded in `i18n/fr`.
Start your site on the French locale:
```bash npm2yarn
npm run start -- --locale fr
```
Make sure that your website is now translated in French at `http://localhost:3000/fr/`.
### Automate with CI {#automate-with-ci}
We will configure the CI to **download the Crowdin translations at build time**, and keep them outside of Git.
Add `website/i18n` to `.gitignore`.
Set the `CROWDIN_PERSONAL_TOKEN` env variable on your CI.
Create a npm script to `sync` Crowdin (extract sources, upload sources, download translations):
```json title="package.json"
{
"scripts": {
"crowdin:sync": "docusaurus write-translations && crowdin upload && crowdin download"
}
}
```
Call the `npm run crowdin:sync` script in your CI, just before building the Docusaurus site.
:::tip
Keep your deploy-previews fast: don't download translations, and use `npm run build -- --locale en` for feature branches.
:::
:::caution
Crowdin does not support well multiple concurrent uploads/downloads: it is preferable to only include translations to your production deployment, and keep deploy previews untranslated.
:::
## Advanced Crowdin topics {#advanced-crowdin-topics}
### MDX {#mdx}
:::caution
Pay special attention to the JSX fragments in MDX documents!
:::
Crowdin **does not support officially MDX**, but they added **support for the `.mdx` extension**, and interpret such files as Markdown (instead of plain text).
#### MDX problems {#mdx-problems}
Crowdin thinks the JSX syntax is embedded HTML, and can mess-up with the JSX markup when you download the translations, leading to a site that fails to build due to invalid JSX.
Simple JSX fragments using simple string props like `<Username name="Sebastien"/>` will work fine.
More complex JSX fragments using object/array props like `<User person={{name: "Sebastien"}}/>` are more likely to fail due to a syntax that does not look like HTML.
#### MDX solutions {#mdx-solutions}
We recommend moving the complex embedded JSX code as separate standalone components.
We also added a `mdx-code-block` escape hatch syntax:
`````text
# How to deploy Docusaurus
To deploy Docusaurus, run the following command:
````mdx-code-block
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
<Tabs
defaultValue="bash"
values={[
{ label: 'Bash', value: 'bash' },
{ label: 'Windows', value: 'windows' }
]}>
<TabItem value="bash">
```bash
GIT_USER=<GITHUB_USERNAME> yarn deploy
```
</TabItem>
<TabItem value="windows">
```batch
cmd /C "set "GIT_USER=<GITHUB_USERNAME>" && yarn deploy"
```
</TabItem>
</Tabs>
````
`````
This will:
- be interpreted by Crowdin as code blocks (and not mess-up with the markup on download)
- be interpreted by Docusaurus as regular JSX (as if it was not wrapped by any code block)
- unfortunately opt-out of MDX tooling (IDE syntax highlighting, Prettier...)
### Docs versioning {#docs-versioning}
Configure translation files for the `website/versioned_docs` folder.
When creating a new version, the source strings will generally be quite similar to the current version (`website/docs`), and you don't want to translate the new version docs again and again.
Crowdin provides a `Duplicate Strings` setting.
![Crowdin Duplicate Strings option setting](/img/crowdin/crowdin-settings-duplicate-strings.png)
We recommend using `Hide`, but the ideal setting depends on how much your versions are different.
:::caution
Not using `Hide` leads to a much larger amount of `source strings` in quotas, and will affect the pricing.
:::
### Multi-instance plugins {#multi-instance-plugins}
You need to configure translation files for each plugin instance.
If you have a docs plugin instance with `id=ios`, you will need to configure those source files as well
- `website/ios`
- `website/ios_versioned_docs` (if versioned)
### Maintaining your site {#maintaining-your-site}
Sometimes, you will **remove or rename a source file** on Git, and Crowdin will display CLI warnings:
![Crowdin CLI: download translation warning](/img/crowdin/crowdin-download-translations-warning.png)
When your sources are refactored, you should use the Crowdin UI to **update your Crowdin files manually**:
![Crowdin UI: renaming a file](/img/crowdin/crowdin-files-rename.png)
### VCS (Git) integrations {#vcs-git-integrations}
Crowdin has multiple VCS integrations for [GitHub](https://support.crowdin.com/github-integration/), GitLab, Bitbucket.
:::warning
We recommend avoiding them.
:::
It could have been helpful to be able to edit the translations in both Git and Crowdin, and have a **bi-directional sync** between the 2 systems.
In practice, **it didn't work very reliably** for a few reasons:
- The Crowdin -> Git sync works fine (with a pull request)
- The Git -> Crowdin sync is manual (you have to press a button)
- The heuristics used by Crowdin to match existing Markdown translations to existing Markdown sources are not 100% reliable, and you have to verify the result on Crowdin UI after any sync from Git
- 2 users concurrently editing on Git and Crowdin can lead to a translation loss
- It requires the `crowdin.yml` file to be at the root of the repository
### In-Context localization {#in-context-localization}
Crowdin has an [In-Context localization](https://support.crowdin.com/in-context-localization/) feature.
:::caution
Unfortunately, it does not work yet for technical reasons, but we have good hope it can be solved.
Crowdin replaces markdown strings with technical ids such as `crowdin:id12345`, but it does so too aggressively, including hidden strings, and mess-up with the frontmatter, admonitions, jsx...
:::
### Localize edit urls {#localize-edit-urls}
When the user is browsing a page at `/fr/doc1`, the edit button will link by default to the unlocalized doc at `website/docs/doc1.md`.
You may prefer the edit button to link to the Crowdin interface instead, and can use the `editUrl` function to customize the edit urls on a per-locale basis.
```js title="docusaurus.config.js"
const DefaultLocale = 'en';
module.exports = {
presets: [
[
'@docusaurus/preset-classic',
{
docs: {
// highlight-start
editUrl: ({locale, versionDocsDirPath, docPath}) => {
// Link to Crowdin for French docs
if (locale !== DefaultLocale) {
return `https://crowdin.com/project/docusaurus-v2/${locale}`;
}
// Link to Github for English docs
return `https://github.com/facebook/docusaurus/edit/master/website/${versionDocsDirPath}/${docPath}`;
},
// highlight-end
},
blog: {
// highlight-start
editUrl: ({locale, blogDirPath, blogPath}) => {
if (locale !== DefaultLocale) {
return `https://crowdin.com/project/docusaurus-v2/${locale}`;
}
return `https://github.com/facebook/docusaurus/edit/master/website/${blogDirPath}/${blogPath}`;
},
// highlight-start
},
},
],
],
};
```
:::note
It is currently **not possible to link to a specific file** in Crowdin.
:::
### Example configuration {#example-configuration}
The **Docusaurus v2 configuration file** is a good example of using versioning and multi-instance:
```mdx-code-block
import CrowdinConfigV2 from '!!raw-loader!@site/../crowdin-v2.yaml';
import CodeBlock from '@theme/CodeBlock';
<CodeBlock className="language-yaml" title="crowdin.yml">
{CrowdinConfigV2.split('\n')
// remove comments
.map((line) => !line.startsWith('#') && line)
.filter(Boolean)
.join('\n')}
</CodeBlock>
```