Weblate server

This page describes how the Weblate service running at translate.rx.studio was configured, and how it is maintained.

Infrastructure

The service runs on a single t3a.large node with 20GB storage (was extended from 10GB due to Docker pulls) in the us-east-1 region of AWS, with termination protection enabled. AWS Backup is configured to take weekly snapshots of the node with a retention window of 1 year.

Docker images and configuration

Weblate is running using Docker and Docker compose:

version: '3'
services:
  weblate:
    image: weblate/weblate
    volumes:
    - weblate-data:/app/data
    env_file:
    - ./environment
    restart: always
    depends_on:
    - database
    - cache
    environment:
      WEBLATE_ENABLE_HTTPS: 1
      WEBLATE_IP_PROXY_HEADER: HTTP_X_FORWARDED_FOR
  database:
    image: postgres:14-alpine
    env_file:
    - ./environment
    volumes:
    - postgres-data:/var/lib/postgresql/data
    restart: always
  cache:
    image: redis:6-alpine
    restart: always
    command: [redis-server, --save, '60', '1']
    volumes:
    - redis-data:/data
  https-portal:
    image: steveltn/https-portal:1
    ports:
    - 80:80
    - 443:443
    restart: always
    environment:
      STAGE: production
      PROXY_READ_TIMEOUT: 3600
      CLIENT_MAX_BODY_SIZE: 100M
    volumes:
    - ssl-certs:/var/lib/https-portal
volumes:
  weblate-data: {}
  postgres-data: {}
  redis-data: {}
  ssl-certs: {}

The environment file has no modifications, as the env var overrides are defined in the docker-compose-https.override.yml file:

version: '3'
services:
  weblate:
    environment:
      WEBLATE_EMAIL_HOST: email-smtp.us-east-1.amazonaws.com
      WEBLATE_EMAIL_PORT: 587
      WEBLATE_EMAIL_USE_SSL: 0
      WEBLATE_EMAIL_USE_TLS: 1
      WEBLATE_EMAIL_HOST_USER: *******
      WEBLATE_EMAIL_HOST_PASSWORD: *******
      WEBLATE_SERVER_EMAIL: noreply@translate.rx.studio
      WEBLATE_DEFAULT_FROM_EMAIL: noreply@translate.rx.studio
      WEBLATE_SITE_DOMAIN: translate.rx.studio
      WEBLATE_ADMIN_EMAIL: *******
      WEBLATE_ADMIN_PASSWORD: *******
      # TODO migrate to Google Analytics v4
      WEBLATE_GOOGLE_ANALYTICS_ID: UA-*******-*
      # enable daily pulls from R core subversion
      WEBLATE_AUTO_UPDATE: true
  https-portal:
    environment:
      DOMAINS: 'translate.rx.studio -> http://weblate:8080'

The current image ids:

$ sudo docker images
REPOSITORY              TAG         IMAGE ID       CREATED         SIZE
weblate/weblate         latest      2ea7d086a8c4   4 days ago      1.1GB
redis                   6-alpine    3616f0c0705d   7 days ago      27.1MB
postgres                14-alpine   9d94e6318ef2   7 days ago      242MB
steveltn/https-portal   1           1f41166c2e81   2 weeks ago     351MB
weblate/weblate         <none>      02c6327da166   9 months ago    804MB
postgres                <none>      07c710d28b91   9 months ago    216MB
redis                   <none>      57c580553a4d   10 months ago   25.5MB
postgres                <none>      ea498678e2bd   10 months ago   189MB
weblate/weblate         <none>      475045de4358   10 months ago   847MB
steveltn/https-portal   <none>      26e5bea459df   12 months ago   274MB

Current weblate etc versions listed at https://translate.rx.studio/about/

Commands to start the service:

docker-compose -f docker-compose-https.yml -f docker-compose-https.override.yml build
docker-compose -f docker-compose-https.yml -f docker-compose-https.override.yml up

To update, follow the official docs, but in short:

docker-compose -f docker-compose-https.yml -f docker-compose-https.override.yml pull
docker-compose -f docker-compose-https.yml -f docker-compose-https.override.yml down
docker-compose -f docker-compose-https.yml -f docker-compose-https.override.yml up -d

Quarterly patch for R Core

NB: Weblate links below assume you’re logged in there, otherwise you’ll be redirected to the home page

To generate a report on the translation updates in a time period:

1. Visit https://translate.rx.studio/projects/r-project/#reports
2. Select time period and generate report in rST format
3. Convert ~markdown to HTML and share in the R Contributors slack group’s #core-translation channel

To submit a patch file on the translations found in Weblate but not in the trunk of the main R subversion repo:

The basic idea is to compare the Weblate repo (which copies the R subversion repo, but also adds translations provided via Weblate) to the “official” R sources; any difference in .po files should be submitted as a patch.

1. Make sure the Weblate repo is fully up-to-date. Check status at https://translate.rx.studio/projects/r-project/#repository – be sure there are no Update or Commit actions needed.

2. Get the two repos cloned on any machine. If this is your first time generating a patch on this machine:

   mkdir weblate-diff && cd weblate-diff
   git clone -o weblate https://translate.rx.studio/git/r-project/base-r-gui/ . # remote #1: Weblate source
   git remote add svn git@github.com:r-devel/r-svn.git                          # remote #2: SVN source
   git fetch svn main                                                           # retrieve the latest from SVN

   If you’ve got the repo set up locally already, update:

   git fetch svn main
   git fetch weblate master
   git reset --hard weblate/master # force-reset to match state on Weblate (usual caution with --hard applies)

3. Make needed manual changes to reduce noise and ensure quality, e.g. something like:

   library(data.table)
   library(crayon)
   library(logger)
   
   po_files <- \() list.files(pattern = '\\.po$', recursive = TRUE)
   # pocount is available from e.g. 'apt install translate-toolkit'
   po_counts <- function(f) {
     x = fread(cmd = paste('pocount --csv', paste(f, collapse = " ")), sep = ',', fill = TRUE)
     setnames(x, c("Filename", "Translated Messages"), c("filename", "n_translated"))
     x
   }
   set_branch <- \(branch) system2('git', c('checkout', branch))
   
   set_branch('weblate/master')
   po_report <- tools::checkPoFiles("")
   if (length(po_report)) {
     print(po_report)
     stop("Fix above issues in .po files identified by tools::checkPoFiles() to proceed.")
   }
   
   weblate_summary <- po_counts(po_files())
   set_branch('svn/main')
   svn_summary <- po_counts(po_files())
   set_branch('master')
   
   # SUMMARIZE the changed translations, and delete empty .po files
   po_summary <- merge(weblate_summary, svn_summary, by = "filename", all = TRUE, suffixes = c("_weblate", "_svn"))
   po_summary[, package := basename(dirname(dirname(filename)))]
   # Drop empty & record files
   po_summary[n_translated_weblate == 0 & is.na(n_translated_svn), { # NB: keep empty files if they're already in SVN, hence is.na() check
      log_info('Dropping {.N} empty files:')
      .SD[, by = package, {
         log_level(INFO, 'From package {blue(.BY$package)}:')
         log_level(INFO, '  {toString(green(basename(filename)))}', .topenv = .SD) # NB: Need .SD since data.table doesn't pick up the variable from the string
      }]
      unlink(filename)
      NULL
   }]
   po_summary <- po_summary[n_translated_weblate > 0]
   
   # (optional) Summarize the update for the rest of the files
   setnafill(po_summary, fill = 0L, cols = which(sapply(po_summary, is.numeric)))
   po_summary[, n_newly_translated := n_translated_weblate - n_translated_svn]
   po_summary[n_newly_translated > 0, {
     log_info('New translations in {.N} files')
     .SD[, by = package, {
       log_level(INFO, 'From package {blue(.BY$package)}:')
       log_level(INFO, '  {sprintf("%s [+%s]", green(format(basename(filename))), red(format(n_newly_translated)))}', .topenv = .SD)
     }]
     NULL
   }]
   
   # REPLACE 'Report-Msgid-Bugs-To' metadata field pointing to #core-translation-po-bugs
   system2("sed", c("-i", "'s/sourcestringbugs.o40l9j@zapiermail.com/bugs.r-project.org/'", po_summary$filename))
   
   # CHECK msgfmt works -- 'make update*' commands run by R-core check this as well
   tmp <- tempfile()
   ## NB: lapply() over for() to show all failures at once
   res <- suppressWarnings(lapply(
     po_summary$filename,
     \(po_file) system2("msgfmt", c("-c", "--statistics", "-o", tmp, po_file), stdout=TRUE, stderr=TRUE)
   ))
   failed <- which(!sapply(res, \(r) is.null(attr(r, "status"))))
   if (length(failed)) {
     cat("'msgfmt' failed for some .po files:\n")
     for (idx in failed) {
       cat(sprintf("  %s:\n", red(po_summary$filename[idx])))
       writeLines(paste0("    ", res[[idx]]))
     }
   }

4. Generate a patch file from the diff, going back to the most recent commit with translations merged, e.g.

   # NB: _not_ 'git diff weblate/master svn/main' since we've just deleted the empty .po files locally
   git diff svn/main --no-prefix -- "*.po"

5. Share the patch file on the R Contributors Slack group’s #core-translation channel and kindly ping @MichaelLawrence for his assistance on getting the patch file applied on the trunk of R dev to get it merged. We should do this ~once per quarter.

Maintenance

Components might become locked due to “not being able to push back to the upstream repo” (expected) or when upstream is not updated for a long time. In such case, components can be unlocked on the UI, but this needs to happen at the component level, so better to use the CLI instead:

weblate unlock_translation r-project

When there’s a conflict between SVN and the Weblate git repo, and Weblate cannot pull from the upstream repo, you will need to resolve the conflicts manually after seeing an error like the below:

Rebasing (1/210)
Rebasing (2/210)
...
Rebasing (50/210)
error: could not apply 93fd9017... Added translation using Weblate (Spanish)
Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".
Could not apply 93fd9017... Added translation using Weblate (Spanish)
CONFLICT (add/add): Merge conflict in src/library/stats/po/es.po
Auto-merging src/library/stats/po/es.po
rebase refs/remotes/origin/trunk: command returned error: 1

Weblate will provide a suggestion on how to fix, but in short:

1. SSH to the Weblate server

2. Make a backup on the Git repo before messing with the rebase process:

   tar -zcvf /root/base-r-gui-repo-$(date '+%Y-%m-%d').tar.gz \
     /var/lib/docker/volumes/weblate-docker_weblate-data/_data/vcs/r-project/base-r-gui

3. Attach the Docker container (this image is offered by weblate):

   sudo docker exec -ti weblate-docker_weblate_1 bash

4. Enter the git folder:

   cd app/data/vcs/r-project/base-r-gui

   As noted above when we made a copy of the repo, this folder can also be found in Weblate outside Docker:

   /var/lib/docker/volumes/weblate-docker_weblate-data/_data/vcs/r-project/base-r-gui

5. Pull from SVN and start the rebase process to see the actual errors:

   git svn fetch
   git svn rebase

6. (repeat until rebase completes) Resolve any conflicts that come up, usually using the script deconflict_add_continue.py found at the repo root:

   ./deconflict_add_continue.py weblate

   This finds the one file causing conflicts, selects the changes from Weblate (in order to preserve string edit history in the UI), removes the changes from SVN and the git conflict markers, then continues the rebase process. It also prints some helpful output, e.g. the number of conflicts resolved and the git status which tells about progress in the ongoing rebase.

   For now, if there are more than one files affected by a given rebase step (e.g. more than one file marked both modified: in the git status output), these needed to be edited manually (with the text editor of your choice, e.g. nano or mcedit), then added (git add) before continuing (git rebase --continue). This should be rare as Weblate typically associates one file edit per commit. The deconflict_add_continue.py step could be extended to handle several files, but we’ve not yet done so. Note also the script show_conflicts.py which takes a file name as an argument and prints a side-by-side view of any git conflicts in the file.

7. Restore changes that may have been lost from SVN, e.g. newly updated source strings or source line numbers, by running msgmerge on all pot files using the script update_pkg_po.py.

Administrators

Currently @daroczig and @MichaelChirico have admin access to Weblate, both in the web UI and via SSH.

In case of any questions, reach out to them on the R-devel Slack’s #core-translation channel, pinging both their usernames.

To add a new admin in the Weblate app:

1. Go to the Users page
2. Select the user
3. Go to “Groups”
4. Add to the “The R Project for Statistical Computing / Administration” group

To grant SSH access:

1. Share your public key with one of the admins, who will add that to ~/.ssh/authorized_keys
2. Ask for the SSH config from one of the admins

Team leaders

There are language teams defined in each project with the below extra permissions:

• Manage glossary
• Review strings
• Manage translation memory

See the general Weblate FAQ on how to manage members.