GettextForWesnothDevelopers

This page is used to help Wesnoth developers to work with the internationalization (i18n) system, based on GNU gettext.

Warning: this page still contains a couple of outdated items to be removed.

Current status

Gettext support is available beginning with version 0.8.3.

The support is there; some calls to the old API remain, and should be handled on a case by case basis before the 1.0 release; most translated strings will be automatically imported.

How to manage the translation work once the transition has been done

We should definitely talk with other projects already accustomed to gettext. The main issue I see is interaction with SVN.

I think we should really never touch po/*.po directly for translating, or you get annoying conflicts when someone commits the file you're editing behind your back. A possible solution is to copy them before editting, and use msgmerge to merge in any cvs-updated version, and have only one person (or well-synchronized group of person) committing them.

How to move strings from one textdomain to another

run make -C po update-po and commit, to be sure to only commit your own changes
move the file into the corect po/*/POTFILES.in
add or change #define GETTEXT_DOMAIN "wesnoth-lib" at top of the file, before the includes
update the target POT file to include the new strings in its template (eg. make -C po/wesnoth-editor

wesnoth-editor.pot-update)

copy the translations using utils/po2po (eg. ./utils/po2po wesnoth wesnoth-editor)
update the source POT file to get rid of the old strings (eg. make -C po/wesnoth update-po), then preferably

remove the translation from obsolete strings in all languages, to make sure, in case the strings have to move back, that any translation update gets used instead of the current one)

check cvs diff and commit

TODO / Known bugs

terrain types should have ID's like we did for units, that will get them out of english.cfg
wmlxgettext should be smart about multiple abilities
(unreproducible ?) sometimes, some strings are apparently not extracted from cpp files, eg. "Alignment" in

help.cpp

problem on macosx - see below
A number of currently untranslatable strings, see end of page.

Open items

understand how symbols["noun"] is currently set in playrurn.cpp::delete_recall_unit::button_pressed() and

find a way to handle that.

understand mapgen.cpp::generate_name() and find a way to handle that.
find how we will handle hotkeys and their localization

(https://savannah.nongnu.org/bugs/?func=detailitem&item_id=9982)

prefix and suffix stuff (eg. "game name prefix|") should be allowed to be empty in the translation when they are

not in English. Find a clean way to do that, other than using a space (ideas: unicode no-width space ? use a format string instead, like "${name}'s game") (**solved**, basically:

       string_map i18n_symbols;
       i18n_symbols["login"] = preferences::login();
       name_entry_.assign(new gui::textbox(disp_,width-20,
       vgettext("$login's game", i18n_symbols)));

)

In the pre-gettext era, the list of languages presented to the user is detected at runtime. We could do this as

well by looking at available wesnoth message catalogs for gettext. But the values we would get are things like "fr", which are problematic because:

- the locales to use are things like fr_FR or fr_CA, and the one to use appears to depend on the system what locales

are available on the local system (please prove me I'm wrong), whereas the available translation usually have no area code (eg. "fr"). This makes choosing a locale to get an autodetected translation quite tricky, so we'll hardcode the list for now.

- we need a simple way of presenting the user with the name of the language, in that language. Since those strings

should be the same regardless of the current locale being used, gettext itself is of no use here. Those language names are available on GNU/Linux in /usr/share/locale/all_languages (thanks Cedric), but we need a portable API to access this data. Otherwise we can just hardcode a list of known languages - indeed, we start with an hardcoded list (like what gcompris does).

Tasks

This is the current state of plans regarding gettextisation. It is subject to change. Get in touch with yann to get involved.

Phase 3

Goal: finetuning and improvements

Replace the hardcoded list of known languages with autodetection of available translations

Things that are not i18n'd even with the pre-gettext system:

character traits - see http://savannah.nongnu.org/bugs/index.php?func=detailitem&item_id=9716

Finished things:

Gettextize the editor
Allow running in build-tree and still finding translations
Add a new declaration to WML files to declare a non-default textdomain to be used, allowing campaigns to ship their

own po file.

Add support in po/ directory for multiple textdomains.

General design of gettext use

Gettextized programs usually contain the English strings within program code, with calls like printf (_("Hello world. "));, so that the binary can work (in English) when the system does not support i18n. However, in Wesnoth all strings were moved into translations/english.cfg, and fetched using a label, like in translate_string("hello_world");.

So we will need to put such strings (mostly GUI material) back into the C++ files. That part will be quite easy, except we'll have to deal with importing existing translations. We will use the wesnoth text domain for this (that is, a single wesnoth.po file for each language).

The general idea for strings in WML files is to use distinct text domains for each campaign, so that campaign writers can easily ship translations together with their campaigns. It will require WML files to declare which text domain they belong to.

If some strings look the same in english but should not necessarily look identical in translations (eg. all those prefix/suffix strings, many of which are empty in english). To hande this, those strings can be prefixed with any descriptive string and a ^ character, thanks to the sgettext implementation partly stolen from the gettext manual (eg. foo_prefix = _ "foo prefix^")

Related efforts

Artur Czechowski also has started to work on a transition to gettext, but with a different approach. His work can be found at http://blabluga.hell.pl/wesnoth/po-migration/

Basically, as I (yann) understand it, his current work does not aim at using gettext completely, but only to use the po format, by providing tools to convert back and forth between po and wml.

I first thought we could use one of his script for the migration, but it seems to rely on an uptodate version of sample_translation, which we have not.

Untranslatable Strings

Due to some problems already mentioned earlier some strings remain untranslatable. Others are untranslatable for an as yet unknown reason. Below is a list of known issues.

In wesnoth.po it is possible to translate the Orcish Crossbow, but it appears twice in the units list of the help

browser (once untranslated and once translated) -- WILL BE FIXED SOON

The skills, traits and terrain types are untranslated.
In the help browser the names of units into which a unit can promote are untranslated.
Weapons are untranslatable and so are their type (blade,fire,...) or speciality (magical)

There is a bug report on savannah as well.

Non-working translations

There are a number of places in the Help browser where translations exist, but are not shown in-game. Clearing the cache does not help//. It seems to be independent on language (tested Swedish, French, and Czech), and might therefore be a general error/bug. The following places are affected:

Main menu items: Units, Abilities, Weapon Specials, About.
Fundamentals of Gameplay: sections Recruiting and Recalling, Healing, Income and Upkeep
Abilities menu: submenu items and titles
Traits section text (although the menu item is shown translated)
Weapon Specials menu: submenu items and titles
Terrain menu: submenu items and titles
About menu: section title