#59 Easy access to localisation files
Closed: Fixed by jibecfed. Opened by fwolff.

I'm interested in obtaining the localisation files produced in the Fedora project. My immediate project is to build a big translation memory for the whole FOSS community, see https://amagama-live.translatehouse.org/
This service has been running for years, and is integrated into some localisation tools.

Years ago we cloned everything on fedorahosted, but that is obviously not possible any more. I can't find some of the things that used to be there before, and some of the translations seem to be nowhere obvious. Furthermore, important packages like Anaconda do not have the translation files in the VCS, which makes it hard to find the canonical versions easily. I realise that the files can be downloaded from Zanata, but automated access (e.g. git pull) would be preferable.

Even translations of retired projects (some of the guides look like they are not used any more) could be useful to the localisation community.

My request is therefore for an easy and obvious way to find up to date localisation files, also for retired projects if possible.

Although I localise some Fedora packages, I might have some factual errors. I discussed this briefly with @jibecfed who agreed that this is worth looking into.


hello, interesting indeed

  1. do you want any languages or do you have a threshold?
  2. how often do you update the source content?
  3. can you elaborate the https://amagama-live.translatehouse.org/ so people know which source content are taken and where to access it/download it?
  4. do you produce TMX based on all po files?
  5. Would it be ok to provide you some TMX files?

1) Any and all languages, unless there are known quality problems. In the case of bad quality, we might prefer to filter afterwards anyway.

2) Updates were ad hoc until now, and might stay that way. Ideally we would want to update it a few times a year. If all steps can be automated, obviously it becomes easier to update more regularly.

3) The list is in flux, partly because of varying availability (like in this case), or questions on what is appropriate, and what can be handled. The import scripts should give a reasonable idea of what we try to do: https://github.com/translate/amagama-updater/ (mostly in the conf/ directory).

4) No. The translation memory is available as a web service meant for integration in tools. I guess such functionality is possible in future, but no provision has been made for that as of yet.

5) I think the original files (mostly Gettext PO) would be ideal, but obviously a TMX dump could be usable too. TMX doesn't support all features of PO files (obsolete units, headers, plurals, context, etc.), although admittedly amagama doesn't use those at the moment. Our tools also only support bilingual TMX files at the moment.

thank you for all these answers.

do you produce TMX based on all po files?

4) No. The translation memory is available as a web service meant for integration in tools. I guess such functionality is possible in future, but no provision has been made for that as of yet.

Which tools are using it?
Would it cost you to provide one TMX per language?

The tools I know of is Virtaal (desktop translation tool) and Pootle (web based system, for example used to translate LibreOffice). The web front-end I linked to is probably not used as much. It is specifically meant for integration in tools. It doesn't restrict access to other tools, but I don't specifically know of any, although I have heard of developers implementing support.

For the TMX files: we're talking about pretty large files. Some of the over 200 languages are smaller, of course, but the biggest 50 ones have more than 100000 strings. Our donated hosting might need to be renegotiated, but maybe this is possible in future. It was definitely not a goal of the project, though. You can of course use the amagama-updater scripts and po2tmx from the toolkit, although a bit of work might be required to get exactly what you want. There is a long list of things that I think will get attention before doing this, but obviously it is not only for me to decide.

A simple alternative that is available right away is the Debian compendia:
https://i18n.debian.org/compendia/
po2tmx might already give usable results, apart from the #-#-#-#- markers.
As I understand, it includes all PO files in the source packages of Debian. That is a lot of translation work!

By way of example, I see this commit in the deprecated release-notes project:
https://pagure.io/release-notes/c/1ecc8ecfda81ef7bf89a951ed5a8a625acf7f6f6?branch=master
It is unclear to me why the translations are removed. In this case I can use this specific hash, but that is pretty hard to find in general. The localisation contributions are valuable, and this makes it harder to find them to obtain value from them again.

In this case, the new replacement project doesn't seem to have ever had translations in git.

all zanata tm are available at this link: https://jibecfed.fedorapeople.org/partage/2019-07-06-all-fedora-zanata-org-tm.7z it takes 40 minutes to extract from Zanata, which is less than expected

Code is there: https://github.com/Jibec/fedora-translation-statistics/commit/1a8fa855ab52b45646940f1c0aefbf91f763e6a1

I don't know yet how to split it per language and would appreciate to have you help on this.

please look into this: https://jibecfed.fedorapeople.org/partage/fedora-localization-statistics/f32/language/
an official website will exist in the future.

Metadata Update from @jibecfed:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

Metadata