docusaurus/website/docs/i18n/i18n-crowdin.mdx
2023-10-19 19:38:32 +02:00

517 lines
16 KiB
Text
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
id: crowdin
slug: /i18n/crowdin
toc_max_heading_level: 4
---
# i18n - Using Crowdin
The i18n system of Docusaurus is **decoupled from any translation software**.
You can integrate Docusaurus with the **tools and SaaS of your choice**, as long as you put the **translation files at the correct location**.
We document the usage of [Crowdin](https://crowdin.com/), as **one** possible **integration example**.
:::warning
This is **not an endorsement of Crowdin** as the unique choice to translate a Docusaurus site, but it is successfully used by Facebook to translate documentation projects such as [Jest](https://jestjs.io/), [Docusaurus](https://docusaurus.io/), and [ReasonML](https://reasonml.github.io/).
Refer to the **[Crowdin documentation](https://support.crowdin.com/)** and **[Crowdin support](mailto:support@crowdin.com)** for help.
:::
:::tip
Use this **[community-driven GitHub discussion](https://github.com/facebook/docusaurus/discussions/4052)** to discuss anything related to Docusaurus + Crowdin.
:::
## Crowdin overview {#crowdin-overview}
Crowdin is a translation SaaS, offering a [free plan for open-source projects](https://crowdin.com/page/open-source-project-setup-request).
We recommend the following translation workflow:
- **Upload sources** to Crowdin (untranslated files)
- Use Crowdin to **translate the content**
- **Download translations** from Crowdin (localized translation files)
Crowdin provides a [CLI](https://support.crowdin.com/cli-tool/) to **upload sources** and **download translations**, allowing you to automate the translation process.
The [`crowdin.yml` configuration file](https://support.crowdin.com/configuration-file/) is convenient for Docusaurus, and permits to **download the localized translation files at the expected location** (in `i18n/[locale]/..`).
Read the **[official documentation](https://support.crowdin.com/)** to know more about advanced features and different translation workflows.
## Crowdin tutorial {#crowdin-tutorial}
This is a walk-through of using Crowdin to translate a newly initialized English Docusaurus website into French, and assume you already followed the [i18n tutorial](./i18n-tutorial.mdx).
The end result can be seen at [docusaurus-crowdin-example.netlify.app](https://docusaurus-crowdin-example.netlify.app/) ([repository](https://github.com/slorber/docusaurus-crowdin-example)).
### Prepare the Docusaurus site {#prepare-the-docusaurus-site}
Initialize a new Docusaurus site:
```bash
npx create-docusaurus@latest website classic
```
Add the site configuration for the French language:
```js title="docusaurus.config.js"
export default {
i18n: {
defaultLocale: 'en',
locales: ['en', 'fr'],
},
themeConfig: {
navbar: {
items: [
// ...
{
type: 'localeDropdown',
position: 'left',
},
// ...
],
},
},
// ...
};
```
Translate the homepage:
```jsx title="src/pages/index.js"
import React from 'react';
import Translate from '@docusaurus/Translate';
import Layout from '@theme/Layout';
export default function Home() {
return (
<Layout>
<h1 style={{margin: 20}}>
<Translate description="The homepage main heading">
Welcome to my Docusaurus translated site!
</Translate>
</h1>
</Layout>
);
}
```
### Create a Crowdin project {#create-a-crowdin-project}
Sign up on [Crowdin](https://crowdin.com/), and create a project.
Use English as the source language, and French as the target language.
![Create a Crowdin project with english as source language, and french as target language](/img/crowdin/crowdin-create-project.png)
Your project is created, but it is empty for now. We will upload the files to translate in the next steps.
### Create the Crowdin configuration {#create-the-crowdin-configuration}
This configuration ([doc](https://support.crowdin.com/configuration-file/)) provides a mapping for the Crowdin CLI to understand:
- Where to find the source files to upload (JSON and Markdown)
- Where to download the files after translation (in `i18n/[locale]`)
Create `crowdin.yml` in `website`:
```yml title="crowdin.yml"
project_id: '123456'
api_token_env: CROWDIN_PERSONAL_TOKEN
preserve_hierarchy: true
files:
# JSON translation files
- source: /i18n/en/**/*
translation: /i18n/%two_letters_code%/**/%original_file_name%
# Docs Markdown files
- source: /docs/**/*
translation: /i18n/%two_letters_code%/docusaurus-plugin-content-docs/current/**/%original_file_name%
# Blog Markdown files
- source: /blog/**/*
translation: /i18n/%two_letters_code%/docusaurus-plugin-content-blog/**/%original_file_name%
```
Crowdin has its own syntax for declaring source/translation paths:
- `**/*`: everything in a subfolder
- `%two_letters_code%`: the 2-letters variant of Crowdin target languages (`fr` in our case)
- `**/%original_file_name%`: the translations will preserve the original folder/file hierarchy
:::info
The Crowdin CLI warnings are not always easy to understand.
We advise to:
- change one thing at a time
- re-upload sources after any configuration change
- use paths starting with `/` (`./` does not work)
- avoid fancy globbing patterns like `/docs/**/*.(md|mdx)` (does not work)
:::
#### Access token {#access-token}
The `api_token_env` attribute defines the **env variable name** read by the Crowdin CLI.
You can obtain a `Personal Access Token` on [your personal profile page](https://crowdin.com/settings#api-key).
:::tip
You can keep the default value `CROWDIN_PERSONAL_TOKEN`, and set this environment variable and on your computer and on the CI server to the generated access token.
:::
:::warning
A Personal Access Tokens grant **read-write access to all your Crowdin projects**.
You should **not commit** it, and it may be a good idea to create a dedicated **Crowdin profile for your company** instead of using a personal account.
:::
#### Other configuration fields {#other-configuration-fields}
- `project_id`: can be hardcoded, and is found on `https://crowdin.com/project/<MY_PROJECT_NAME>/settings#api`
- `preserve_hierarchy`: preserve the folder's hierarchy of your docs on Crowdin UI instead of flattening everything
### Install the Crowdin CLI {#install-the-crowdin-cli}
This tutorial uses the CLI version `3.5.2`, but we expect `3.x` releases to keep working.
Install the Crowdin CLI as an npm package to your Docusaurus site:
```bash npm2yarn
npm install @crowdin/cli@3
```
Add a `crowdin` script:
```json title="package.json"
{
"scripts": {
// ...
"write-translations": "docusaurus write-translations",
"crowdin": "crowdin"
}
}
```
Test that you can run the Crowdin CLI:
```bash npm2yarn
npm run crowdin -- --version
```
Set the `CROWDIN_PERSONAL_TOKEN` env variable on your computer, to allow the CLI to authenticate with the Crowdin API.
:::tip
Temporarily, you can hardcode your personal token in `crowdin.yml` with `api_token: 'MY-TOKEN'`.
:::
### Upload the sources {#upload-the-sources}
Generate the JSON translation files for the default language in `website/i18n/en`:
```bash npm2yarn
npm run write-translations
```
Upload all the JSON and Markdown translation files:
```bash npm2yarn
npm run crowdin upload
```
![Crowdin CLI uploading Docusaurus source files](/img/crowdin/crowdin-upload-sources-cli.png)
Your source files are now visible on the Crowdin interface: `https://crowdin.com/project/<MY_PROJECT_NAME>/settings#files`
![Crowdin UI showing Docusaurus source files](/img/crowdin/crowdin-source-files.png)
### Translate the sources {#translate-the-sources}
On `https://crowdin.com/project/<MY_PROJECT_NAME>`, click on the French target language.
![Crowdin UI showing French translation files](/img/crowdin/crowdin-french-translations.png)
Translate some Markdown files.
![Crowdin UI to translate a Markdown file](/img/crowdin/crowdin-translate-markdown.png)
:::tip
Use `Hide String` to make sure translators **don't translate things that should not be**:
- Front matter: `id`, `slug`, `tags` ...
- Admonitions: `:::`, `:::note`, `:::tip` ...
![Crowdin UI hide string](/img/crowdin/crowdin-hide-string.png)
:::
Translate some JSON files.
![Crowdin UI to translate a JSON file](/img/crowdin/crowdin-translate-json.png)
:::info
The `description` attribute of JSON translation files is visible on Crowdin to help translate the strings.
:::
:::tip
**[Pre-translate](https://support.crowdin.com/pre-translation-via-machine/)** your site, and **fix pre-translation mistakes manually** (enable the Global Translation Memory in settings first).
Use the `Hide String` feature first, as Crowdin is pre-translating things too optimistically.
:::
### Download the translations {#download-the-translations}
Use the Crowdin CLI to download the translated JSON and Markdown files.
```bash npm2yarn
npm run crowdin download
```
The translated content should be downloaded in `i18n/fr`.
Start your site on the French locale:
```bash npm2yarn
npm run start -- --locale fr
```
Make sure that your website is now translated in French at [`http://localhost:3000/fr/`](http://localhost:3000/fr/).
### Automate with CI {#automate-with-ci}
We will configure the CI to **download the Crowdin translations at build time** and keep them outside of Git.
Add `website/i18n` to `.gitignore`.
Set the `CROWDIN_PERSONAL_TOKEN` env variable on your CI.
Create an npm script to `sync` Crowdin (extract sources, upload sources, download translations):
```json title="package.json"
{
"scripts": {
"crowdin:sync": "docusaurus write-translations && crowdin upload && crowdin download"
}
}
```
Call the `npm run crowdin:sync` script in your CI, just before building the Docusaurus site.
:::tip
Keep your deploy-previews fast: don't download translations, and use `npm run build -- --locale en` for feature branches.
:::
:::warning
Crowdin does not support well multiple concurrent uploads/downloads: it is preferable to only include translations to your production deployment, and keep deploy previews untranslated.
:::
## Advanced Crowdin topics {#advanced-crowdin-topics}
### MDX {#mdx}
:::warning
Pay special attention to the JSX fragments in MDX documents!
:::
Crowdin **does not support officially MDX**, but they added **support for the `.mdx` extension**, and interpret such files as Markdown (instead of plain text).
#### MDX problems {#mdx-problems}
Crowdin thinks that the JSX syntax is embedded HTML and can mess up with the JSX markup when you download the translations, leading to a site that fails to build due to invalid JSX.
Simple JSX fragments using simple string props like `<Username name="Sebastien"/>` will work fine; more complex JSX fragments using object/array props like `<User person={{name: "Sebastien"}}/>` are more likely to fail due to a syntax that does not look like HTML.
#### MDX solutions {#mdx-solutions}
We recommend extracting the complex embedded JSX code as separate standalone components. We also added an `mdx-code-block` escape hatch syntax:
`````text
# How to deploy Docusaurus
To deploy Docusaurus, run the following command:
````mdx-code-block 
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
<Tabs>
<TabItem value="bash" label="Bash">
```bash
GIT_USER=<GITHUB_USERNAME> yarn deploy
```
</TabItem>
<TabItem value="windows" label="Windows">
```batch
cmd /C "set "GIT_USER=<GITHUB_USERNAME>" && yarn deploy"
```
</TabItem>
</Tabs>
````
`````
This will:
- be interpreted by Crowdin as code blocks (and not mess-up with the markup on download)
- be interpreted by Docusaurus as regular JSX (as if it was not wrapped by any code block)
- unfortunately opt-out of MDX tooling (IDE syntax highlighting, Prettier...)
### Docs versioning {#docs-versioning}
Configure translation files for the `website/versioned_docs` folder.
When creating a new version, the source strings will generally be quite similar to the current version (`website/docs`), and you don't want to translate the new version docs again and again.
Crowdin provides a `Duplicate Strings` setting.
![Crowdin Duplicate Strings option setting](/img/crowdin/crowdin-settings-duplicate-strings.png)
We recommend using `Hide`, but the ideal setting depends on how much your versions are different.
:::warning
Not using `Hide` leads to a much larger amount of `source strings` in quotas, and will affect the pricing.
:::
### Multi-instance plugins {#multi-instance-plugins}
You need to configure translation files for each plugin instance.
If you have a docs plugin instance with `id=ios`, you will need to configure those source files as well
- `website/ios`
- `website/ios_versioned_docs` (if versioned)
### Maintaining your site {#maintaining-your-site}
Sometimes, you will **remove or rename a source file** on Git, and Crowdin will display CLI warnings:
![Crowdin CLI: download translation warning](/img/crowdin/crowdin-download-translations-warning.png)
When your sources are refactored, you should use the Crowdin UI to **update your Crowdin files manually**:
![Crowdin UI: renaming a file](/img/crowdin/crowdin-files-rename.png)
### VCS (Git) integrations {#vcs-git-integrations}
Crowdin has multiple VCS integrations for [GitHub](https://support.crowdin.com/github-integration/), GitLab, Bitbucket.
:::warning TL;DR
We recommend avoiding them.
:::
It could have been helpful to be able to edit the translations in both Git and Crowdin, and have a **bi-directional sync** between the 2 systems.
In practice, **it didn't work very reliably** for a few reasons:
- The Crowdin -> Git sync works fine (with a pull request)
- The Git -> Crowdin sync is manual (you have to press a button)
- The heuristics used by Crowdin to match existing Markdown translations to existing Markdown sources are not 100% reliable, and you have to verify the result on Crowdin UI after any sync from Git
- 2 users concurrently editing on Git and Crowdin can lead to a translation loss
- It requires the `crowdin.yml` file to be at the root of the repository
### In-Context localization {#in-context-localization}
Crowdin has an [In-Context localization](https://support.crowdin.com/in-context-localization/) feature.
:::warning
Unfortunately, it does not work yet for technical reasons, but we have good hope it can be solved.
Crowdin replaces Markdown strings with technical IDs such as `crowdin:id12345`, but it does so too aggressively, including hidden strings, and messes up with front matter, admonitions, JSX...
:::
### Localize edit URLs {#localize-edit-urls}
When the user is browsing a page at `/fr/doc1`, the edit button will link by default to the unlocalized doc at `website/docs/doc1.md`.
You may prefer the edit button to link to the Crowdin interface instead by using the `editUrl` function to customize the edit URLs on a per-locale basis.
```js title="docusaurus.config.js"
const DefaultLocale = 'en';
export default {
presets: [
[
'@docusaurus/preset-classic',
{
docs: {
// highlight-start
editUrl: ({locale, versionDocsDirPath, docPath}) => {
// Link to Crowdin for French docs
if (locale !== DefaultLocale) {
return `https://crowdin.com/project/docusaurus-v2/${locale}`;
}
// Link to GitHub for English docs
return `https://github.com/facebook/docusaurus/edit/main/website/${versionDocsDirPath}/${docPath}`;
},
// highlight-end
},
blog: {
// highlight-start
editUrl: ({locale, blogDirPath, blogPath}) => {
if (locale !== DefaultLocale) {
return `https://crowdin.com/project/docusaurus-v2/${locale}`;
}
return `https://github.com/facebook/docusaurus/edit/main/website/${blogDirPath}/${blogPath}`;
},
// highlight-start
},
},
],
],
};
```
:::note
It is currently **not possible to link to a specific file** in Crowdin.
:::
### Example configuration {#example-configuration}
The **Docusaurus configuration file** is a good example of using versioning and multi-instance:
```mdx-code-block
import CrowdinConfigV2 from '!!raw-loader!@site/../crowdin-v2.yaml';
import CodeBlock from '@theme/CodeBlock';
<CodeBlock className="language-yaml" title="crowdin.yml">
{CrowdinConfigV2.split('\n')
// remove comments
.map((line) => !line.startsWith('#') && line)
.filter(Boolean)
.join('\n')}
</CodeBlock>
```