Difference between revisions of "GettextForTranslators"

From The Battle for Wesnoth Wiki
(Categorize)
m (Update list of PO file editors)
(20 intermediate revisions by 11 users not shown)
Line 1: Line 1:
 
==  Gettext for translators  ==
 
==  Gettext for translators  ==
  
Wesnoth translation is now divided into several text-domains:
+
The target audience of this page is anyone who wants to help with a language that's already in the list on [[WesnothTranslations]]. The files for these languages have already been set up, someone is already the maintainer, and the pages linked from that page say where to go and how to introduce yourself to the team. The effort is split with separate teams for each language, and each team can have its own working style.
 +
 
 +
Because each team can work in different ways, please do not submit translations as pull requests on Github. While that seems like a good idea, it causes potential complications, so we ask that everyone talks to their language's maintainer and submits changes in the way that the maintainer decides to use.
 +
 
 +
If you're starting a completely new translation, or taking over as the maintainer of a translation, then the next place to read would be the [[WesnothTranslationsHowTo]] page.
 +
 
 +
== Textdomains and getting the files to translate ==
 +
 
 +
The progress for each language is shown on https://www.wesnoth.org/gettext/ . Click on your language, and you'll see a breakdown into sections ''(textdomains)'', such as
 
* wesnoth
 
* wesnoth
 
* wesnoth-editor
 
* wesnoth-editor
 +
* wesnoth-help
 +
* wesnoth-units
 
* wesnoth-lib (contains strings shared by game and editor)
 
* wesnoth-lib (contains strings shared by game and editor)
* plus one text-domain for each campaign. The names of these text-domains are constructed from the word wesnoth and the common abbreviation of the campaign (wesnoth-ei, wesnoth-httt, wesnoth-trow, wesnoth-tb, wesnoth-tsg and wesnoth-utbs).
+
* wesnoth-httt (the Heir to the Throne campaign)
 +
* wesnoth-utbs (the Under the Burning Suns campaign)
 +
* etc
  
We will use the Swedish (sv) language and the ''wesnoth'' domain as an example in this page.
+
Each has a separate file with a .po extension. For example, the Swedish translation has abbreviation ''sv'', and its translation of the editor's strings is in ''wesnoth-editor/sv.po''. The page on https://www.wesnoth.org/gettext/?view=langs&version=branch&lang=sv links to the current version in the main Git repository.
There are three important files for each text-domain:
 
* ''wesnoth/wesnoth.pot'' file, which contains all the strings used in the main game (neither editor, nor campaign strings). You can see how a ''.pot'' file looks like in SVN : [http://svn.gna.org/viewcvs/wesnoth/trunk/po/wesnoth/wesnoth.pot?view=markup Current SVN version].
 
* ''wesnoth/sv.po'', the file containing the translation, you can see how a ''.po'' file looks like in SVN: [http://svn.gna.org/viewcvs/wesnoth/trunk/po/wesnoth/sv.po?view=markup Current SVN version].
 
* ''{translations_dir}/sv/LC_MESSAGES/wesnoth.gmo'', which is a binary version of <lang>.po that the game will use for efficiency reasons. This is not found in cvs, but is created during compilation of the game.
 
  
The first of these files, we update every once in a while, when there is new or modified text,
+
Although the .po files contain text, please send the complete files as email attachments (or whichever method your team uses), rather than cutting and pasting lines from the file into an email. The translated strings are
for instance when a new campaign scenario hits the SVN.
+
very sensitive to formatting and whitespace changes.
  
Then, ''msgmerge'' is used to merge this new "state of the game strings" with your current translations, and generate a new, up-to-date, ''sv.po''.
+
=== Files when running the game ===
  
Then, a tool called ''msgfmt'' is called, which generates a new ''wesnoth.gmo'' file from your ''wesnoth.po'' file, which will be used by the game once installed.
+
When the game runs it will look for an .mo file, for example ''translations/sv/LC_MESSAGES/wesnoth-editor.mo''. If you want to test your text in-game and you're happy to modify your installation:
  
"make -C po update-po" does these three steps in one pass.
+
* Some .po editors can automatically generate .mo files.
 +
* Deleting the .mo file will make the game look for ''translations/wesnoth-editor/sv.po'' instead.
  
That was for the "Big Picture".
+
However you can also send your untested .po file to the maintainer and they should check that it looks correct in-game.
But you really just need to be concerned with the po file:
 
translating new strings, checking "fuzzy" entries.
 
We can handle the administrative tasks of updating etc.
 
You really don't need to bother with any of these commands, unless you want to compile gmo files to check in-game.
 
  
Ivanovic is now responsible for integrating translations into Wesnoth, with the help of Torangan and ott.
+
== How to translate ==
You should contact Ivanovic (crazy-ivanovic AT gmx DOT net), and he'll put them in the game.
 
  
When sending po files to a maintainer to be included in SVN, please create a zip or tar.gz/bz2 archive that you attach, rather than cutting and pasting the translations into an email. The translated strings are
+
Now that you have the .po file to edit, it can be worked on using any of the programs listed in the [[#Tools|Tools section]]. The general preference in Wesnoth seems to be towards Poedit, but they all work on the same .po file format.
very sensitive to formatting and the archive will also help distinguish which file is which when sending more than one po file.
+
 
 +
The general concept is that the GUI will show the strings in English, along with a text-box to add the translation.
 +
 
 +
=== Warnings ===
 +
 
 +
Some editors can automatically detect inconsistencies between the English and the translated text, for example if the original ends with a full-stop and the translation ends with an exclamation mark. Poedit defaults to showing these above even the untranslated strings, but these can be false-positives - generally someone's already looked at these and decided that the translated text is better as-is.
 +
 
 +
=== Fuzzy strings ===
 +
 
 +
One of the downsides of Gettext is that spelling and grammar corrections in the English text break the link between the original and the translated text. The tools that generate .po files try to recover from this by using the old translation for the new English text, and marking the string as ''fuzzy''; in Poedit these are sorted below the completely untranslated strings, and shown with the "Needs Work" button lit along with a note about what the previous English text was.
 +
 
 +
Be wary, this mechanism can also generate incorrect suggestions. For example it may decide that "Landar left $number troops to guard the council" is a spelling correction of "Kalenz left $number troops to guard the council".
  
 
==  FAQ  ==
 
==  FAQ  ==
* What's a fuzzy string?
+
* '''What are "Plural-forms"?'''
** It's a string calling for translator revision. For example, if an English string gets a small change and you run the ''msgmerge'' program, it will mark the translation of that string fuzzy. When we switched to gettext we marked all translations as fuzzy as lots of them were outdated.
 
* What are "Plural-forms"?
 
 
** Some languages have different word forms for different numbers of things (for example in English we have "1 thing" but "2 thing'''s'''"). The rules are different for different languages. You can find them [http://translate.sourceforge.net/wiki/l10n/pluralforms here].
 
** Some languages have different word forms for different numbers of things (for example in English we have "1 thing" but "2 thing'''s'''"). The rules are different for different languages. You can find them [http://translate.sourceforge.net/wiki/l10n/pluralforms here].
* My language needs a different font to get displayed correctly.
+
* '''How do I use ' within a single-quote delimited string when translating text= in help screen texts?'''
** Wesnoth uses several different fonts, and the number is likely to grow with the support for new alphabets/writing systems. Right now we are using DejaVuSans (Roman and Cyrillic), FreeSans (Greek), sazanami-gothic (Japanese) and gkai00mp (Chinese). In the wesnoth/sv.po you will find a string containing the currently used fonts. By changing the order of fonts listed, you choose which font is preferred for your translations. A language using Greek letters would for instance translate that string with:
+
** Add a backslash before it, however the preferred method now is to use [[Typography_Style_Guide#Character_Usage_Summary|typographic punctuation]] instead.
  msgid "DejaVuSans.ttf,FreeSans.ttf,sazanami-gothic.ttf,gkai00mp.ttf"
+
* '''What should I do with strings like "Prefs section^General"?'''
  msgstr "FreeSans.ttf,DejaVuSans.ttf,sazanami-gothic.ttf,gkai00mp.ttf"
 
Note that if a certain letter/glyph is not found in the first font listed, the second font will be scanned for it and so on.
 
* What should I do to insert a newline in a string?
 
** If you want to insert a newline in a string you need to include the pair of characters "". You can use normal newlines as you wish as they don't affect the translation.
 
* How do I use ' within a single-quote delimited string when translating text= in help screen texts?
 
** Add a backslash before it.
 
* What should I do with strings like "Prefs section^General"?
 
 
** There are ambiguous strings which should be translated in a different way depending on where they appear. For example, we have "General" in the preferences as "General preferences" and we can also have "a General". These strings can have different translations for a given language, so we use "context" to solve this. The prefix only tries to give a hint about the string, and should be not translated, for example:
 
** There are ambiguous strings which should be translated in a different way depending on where they appear. For example, we have "General" in the preferences as "General preferences" and we can also have "a General". These strings can have different translations for a given language, so we use "context" to solve this. The prefix only tries to give a hint about the string, and should be not translated, for example:
 
   msgid "Prefs section^General"
 
   msgid "Prefs section^General"
 
   msgstr "General"
 
   msgstr "General"
* Who can I ask for further information?
+
* '''My PO file already has that translated to "Prefs section^General", should it work?'''
** You can ask Ivanovic, Torangan, ott, Yann or Isaac in IRC (irc.freenode.net, #wesnoth or #wesnoth-dev).
+
** It does, but probably shouldn't. For strings that have no translation, the part before the "^" will be removed. This is also done when the PO's msgstr is exactly the same as the msgid, which will later break if someone makes any change to the text. For example:
If you don't like IRC, send a mail to crazy-ivanovic AT gmx DOT net, or pm him (ivanovic) at the forum.
+
  # wrong but seems to work (shows as "Root")
* How do I quickly test PO file changes?
+
  msgid "filesystem_path_system^Root"
** When you modify a PO file, run "make update-gmo" in the main po directory to create a new MO file, then run "make install" there to install the new MO file.
+
  msgstr "filesystem_path_system^Root"
** Then force cache rebuild. This is tricky -- if you only update MO files, the game may use a cached (i.e., old) version letting you wonder why it still displays the old translation. Either change game.cfg modification time, e.g. with "touch /usr/local/share/wesnoth/game.cfg", or delete the corresponding cache[s] in ~/.wesnoth/cache.
+
  # wrong and shows as "filesystem_path_system^Wurzel"
* The .po file in SVN has changed, how do I move over the translations I've made?
+
  msgid "filesystem_path_system^Root"
** If your file with additional translations is called old.po and the new file from SVN is called svn.po, create a new file called new.po by doing
+
  msgstr "filesystem_path_system^Wurzel"
 
+
* '''Who can I ask for further information?'''
  msgmerge -o new.po old.po svn.po
+
** You can ask in [[Support|Discord or IRC]]. Ping Ivanovic in Discord's #development or IRC's #wesnoth-dev channel. If you don't like IRC, send a mail to crazy-ivanovic AT gmx DOT net, or pm him (ivanovic) at the forum.
 
+
* '''Why is the diff from the previous version so huge? I have only made a small change to the .po file with poedit.'''
The new file new.po will keep your translations from old.po as well as the new strings from svn.po, but check the result -- some translations will not be possible to move across if the original strings changed, these are kept as comments at the end of the file.
+
** When saving .po file poedit unwraps all strings. Usually, all .po files are wrapped at 80 characters so if you want smaller diffs and less merge conflicts you can execute the following commands each time after editing with poedit:
 
 
* How to generate man files from wesnoth-man(current 1.3 trunk) .po files?
 
To create "real" manpages out of the po files you have to install a prog
 
named "po4a" and run make update-po4a inside the maindir before running
 
make and make install. They are not automatically created out of po
 
files. With that command the files will be created and afterwards installed.
 
  
Note: At least 80% of the .po file must be translated for the creation of a man file
+
  msgattrib file.po > file.po1
 +
  mv file.po1 file.po
  
 
==  Tools  ==
 
==  Tools  ==
  
 
There are several tools to work with .po files:
 
There are several tools to work with .po files:
* [http://i18n.kde.org/tools/kbabel/ KBabel] (Unix KDE)
+
* [https://pofile.net/free-po-editor Free PO editor] (Web based)
* [http://gtranslator.sourceforge.net/ GTranslator] (Unix Gnome)
+
* [http://userbase.kde.org/Lokalize Lokalize] (Unix KDE)
 +
* [http://wiki.gnome.org/Apps/Gtranslator GTranslator] (Unix Gnome)
 +
* [http://virtaal.translatehouse.org Virtaal] (Multiplatform)
 
* [http://www.poedit.net/ poEdit] (Multiplatform)
 
* [http://www.poedit.net/ poEdit] (Multiplatform)
 
* [http://www.gnu.org/software/emacs/emacs.html Emacs with po-mode] (Multiplatform)
 
* [http://www.gnu.org/software/emacs/emacs.html Emacs with po-mode] (Multiplatform)
Line 89: Line 92:
 
* [[WesnothTranslationsHowTo]]
 
* [[WesnothTranslationsHowTo]]
 
* [[GettextForWesnothDevelopers]]
 
* [[GettextForWesnothDevelopers]]
* [http://gettext.wesnoth.org/ Translation statistics]
+
* [[TranslatorShellscript]]
 +
* [http://www.wesnoth.org/gettext/ Translation statistics]
  
 
[[Category:Translations]]
 
[[Category:Translations]]

Revision as of 09:49, 25 October 2021

Gettext for translators

The target audience of this page is anyone who wants to help with a language that's already in the list on WesnothTranslations. The files for these languages have already been set up, someone is already the maintainer, and the pages linked from that page say where to go and how to introduce yourself to the team. The effort is split with separate teams for each language, and each team can have its own working style.

Because each team can work in different ways, please do not submit translations as pull requests on Github. While that seems like a good idea, it causes potential complications, so we ask that everyone talks to their language's maintainer and submits changes in the way that the maintainer decides to use.

If you're starting a completely new translation, or taking over as the maintainer of a translation, then the next place to read would be the WesnothTranslationsHowTo page.

Textdomains and getting the files to translate

The progress for each language is shown on https://www.wesnoth.org/gettext/ . Click on your language, and you'll see a breakdown into sections (textdomains), such as

  • wesnoth
  • wesnoth-editor
  • wesnoth-help
  • wesnoth-units
  • wesnoth-lib (contains strings shared by game and editor)
  • wesnoth-httt (the Heir to the Throne campaign)
  • wesnoth-utbs (the Under the Burning Suns campaign)
  • etc

Each has a separate file with a .po extension. For example, the Swedish translation has abbreviation sv, and its translation of the editor's strings is in wesnoth-editor/sv.po. The page on https://www.wesnoth.org/gettext/?view=langs&version=branch&lang=sv links to the current version in the main Git repository.

Although the .po files contain text, please send the complete files as email attachments (or whichever method your team uses), rather than cutting and pasting lines from the file into an email. The translated strings are very sensitive to formatting and whitespace changes.

Files when running the game

When the game runs it will look for an .mo file, for example translations/sv/LC_MESSAGES/wesnoth-editor.mo. If you want to test your text in-game and you're happy to modify your installation:

  • Some .po editors can automatically generate .mo files.
  • Deleting the .mo file will make the game look for translations/wesnoth-editor/sv.po instead.

However you can also send your untested .po file to the maintainer and they should check that it looks correct in-game.

How to translate

Now that you have the .po file to edit, it can be worked on using any of the programs listed in the Tools section. The general preference in Wesnoth seems to be towards Poedit, but they all work on the same .po file format.

The general concept is that the GUI will show the strings in English, along with a text-box to add the translation.

Warnings

Some editors can automatically detect inconsistencies between the English and the translated text, for example if the original ends with a full-stop and the translation ends with an exclamation mark. Poedit defaults to showing these above even the untranslated strings, but these can be false-positives - generally someone's already looked at these and decided that the translated text is better as-is.

Fuzzy strings

One of the downsides of Gettext is that spelling and grammar corrections in the English text break the link between the original and the translated text. The tools that generate .po files try to recover from this by using the old translation for the new English text, and marking the string as fuzzy; in Poedit these are sorted below the completely untranslated strings, and shown with the "Needs Work" button lit along with a note about what the previous English text was.

Be wary, this mechanism can also generate incorrect suggestions. For example it may decide that "Landar left $number troops to guard the council" is a spelling correction of "Kalenz left $number troops to guard the council".

FAQ

  • What are "Plural-forms"?
    • Some languages have different word forms for different numbers of things (for example in English we have "1 thing" but "2 things"). The rules are different for different languages. You can find them here.
  • How do I use ' within a single-quote delimited string when translating text= in help screen texts?
  • What should I do with strings like "Prefs section^General"?
    • There are ambiguous strings which should be translated in a different way depending on where they appear. For example, we have "General" in the preferences as "General preferences" and we can also have "a General". These strings can have different translations for a given language, so we use "context" to solve this. The prefix only tries to give a hint about the string, and should be not translated, for example:
  msgid "Prefs section^General"
  msgstr "General"
  • My PO file already has that translated to "Prefs section^General", should it work?
    • It does, but probably shouldn't. For strings that have no translation, the part before the "^" will be removed. This is also done when the PO's msgstr is exactly the same as the msgid, which will later break if someone makes any change to the text. For example:
  # wrong but seems to work (shows as "Root")
  msgid "filesystem_path_system^Root"
  msgstr "filesystem_path_system^Root"
  # wrong and shows as "filesystem_path_system^Wurzel"
  msgid "filesystem_path_system^Root"
  msgstr "filesystem_path_system^Wurzel"
  • Who can I ask for further information?
    • You can ask in Discord or IRC. Ping Ivanovic in Discord's #development or IRC's #wesnoth-dev channel. If you don't like IRC, send a mail to crazy-ivanovic AT gmx DOT net, or pm him (ivanovic) at the forum.
  • Why is the diff from the previous version so huge? I have only made a small change to the .po file with poedit.
    • When saving .po file poedit unwraps all strings. Usually, all .po files are wrapped at 80 characters so if you want smaller diffs and less merge conflicts you can execute the following commands each time after editing with poedit:
  msgattrib file.po > file.po1
  mv file.po1 file.po

Tools

There are several tools to work with .po files:

Of course, you can edit po files with any UTF-8 capable text editor, but the tools listed above have great advantages over any text editor regarding .po translation, like going to next fuzzy/untranslated string, searching only in specific fields (msgid, msgstr, comment), ...

See Also