We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
IndexProgramming Questions & HelpSyntax Questions › Localization [ i18n ]
Page Index Toggle Pages: 1
Localization [ i18n ] (Read 4277 times)
Localization [ i18n ]
Aug 25th, 2009, 3:10am
 
Hi!

I'm developing this app for live visuals and I'd be really happy to have testers in other countries. but I made the whole app in Spanish and so i thought it would be a good Idea to localize for several countries.

I tried a Java tutorial but it didn't work ( http://java.sun.com/docs/books/tutorial/i18n/intro/quick.html ), which took me to the question: Does Processing Support Localization? I mean is it capable of loading .properties files? Do I have to take another approach to locale? I'd hate too hardcode all my current version code, it would take quite a while i could spend working in the program!

Any ideas?

Thank you! Smiley
Re: Localization [ i18n ]
Reply #1 - Aug 26th, 2009, 5:17am
 
Good question! Since lot of Processing code is purely graphical, I suppose not many people tried to make translatable sketches.
I just tried... and found out it was surprisingly hard! Smiley
I love this kind of question because when trying to solve it, I have to do quite some researches, experiment with new code, and learn a lot in the process!

To summarize: i18n (internationalization) in Java is usually done by using ResourceBundle class.
This class basically just loads a property file (whole lines are made only of key = value pairs) chosen according to a locale, and provides a translation string corresponding to a key (ID of string to translate).
Somehow, we can do that manually with a simple loadString()...
But ResourceBundle is smarter than that: it can load a default file providing the untranslated, base strings (usually in English...), and override these strings with those defined in a file for a given language (say fr) and even override those with the ones defined in a country dialect (fr_FR, fr_CA, etc.).
This hierarchical loading is powerful, providing fallback strings for untranslated terms, and allowing to take in account dialects that usually change only a few strings.

The main problem in Processing that you probably encountered, is that ResourceBundle always get its resources from the classpath, ie. Processing libs and where the class files are, typically in a randomly named build folder in system's temp dir.
Fortunately, we can specify a specific class loader, which will look inside the data folder of the sketch instead.
Why there Because files in this folder will be included when exporting the application. I haven't tried that, yet...

Another problem, generic to Java, is that property files must use the ISO-8859-1 encoding, which is quite an annoying limitation.
If you want to translate in Greek, Turkish, Russian, Arabic, Hebrew, or most Asian languages, to name but a few, you have to put Unicode escapes in these files (like \u2202 or similar!).
Java 1.6 allows a mechanism to load UTF-8 resource bundles, but I avoided using it because 32bit Mac users doesn't have it (at this time, it is coming with new version of MacOS X).
Fortunately, I found a hack allowing to load and convert UTF-8 properties.

These two tricks combined allow to use i18n in Processing. How cool is that Smiley

I made a demo sketch (reusing some other unrelated sketch), trying to highlight the hierarchical loading of resources.

The code can be seen and downloaded at the Launchpad repository

Here is the (simplified) test code:
Code:
Locale esLocale = new Locale("es");

ResourceBundle res;
String bundleName = "Localization";
PFont fDisplay, fTitle;
int i18bPos = 500;
int menuPos = i18bPos + 50;
int infoPos = menuPos + 50;

void setup()
{
 size(600, 800);
 smooth();
 background(#AAFFEE);

 // Get the strings corresponding to YOUR locale
 GetStrings(Locale.getDefault());
 frameRate(5);

//  fDisplay = loadFont("Silkscreen-8.vlw");
 fDisplay = loadFont("Arial-Black-12.vlw");
 fTitle = loadFont("Impact-48.vlw");
}

void draw()
{
 background(#CAFEBA);

 textFont(fTitle);
 fill(#5599AA);
 textAlign(LEFT);
 text(appName + " (" + appAuth + ")", 5, i18bPos);
 textFont(fDisplay, 24);
 text(slogan, 50, i18bPos + 24);

 textFont(fDisplay);
 fill(#557799);
 textAlign(CENTER);
 text(en, 100, menuPos);
 text(es, 200, menuPos);
 text(fr, 300, menuPos);

 fill(#335577);
 textAlign(LEFT);
 text(title + ": " + titleValue, 50, infoPos);
 text(artist + ": " + artistValue, 50, infoPos + 20);
 text(album + ": " + albumValue, 50, infoPos + 40);
 text(genre + ": " + genreValue, 50, infoPos + 60);
}

void mouseReleased()
{
 boolean bAlt = keyPressed && key == CODED && keyCode == CONTROL;
 if (mouseY > menuPos - 20 && mouseY < menuPos + 20)
 {
   if (mouseX > 60 && mouseX < 140)
   {
if (bAlt)
{
 GetStrings(Locale.US); // Locale("en", "US")
}
else
{
 GetStrings(Locale.ENGLISH); // Locale("en")
}
   }
   else if (mouseX > 160 && mouseX < 240)
   {
GetStrings(esLocale);
   }
   else if (mouseX > 260 && mouseX < 340)
   {
if (bAlt)
{
 GetStrings(Locale.CANADA_FRENCH); // Locale("fr", "CA")
}
else
{
 GetStrings(Locale.FRENCH); // Locale("fr")
}
   }
   else
   {
GetStrings(Locale.getDefault());
   }
 }
//  println(mouseX + " " + mouseY);
}

String appName, appAuth, slogan;
String title, artist, album, genre;
String en, es, fr;
String titleValue = "Here Comes the Sun", artistValue = "The Beatles",
   albumValue = "Abbey Road", genreValue = "Pop";
void GetStrings(Locale locale)
{
 res = UTF8ResourceBundle.getBundle(bundleName, locale,
new ProcessingClassLoader(this));

 appName = GetString("APP_NAME");
 appAuth = GetString("APP_AUTH");
 slogan = GetString("slogan");
 title = GetString("Title");
 artist = GetString("Artist");
 album = GetString("Album");
 genre = GetString("Genre");
 en = GetString("en");
 es = GetString("es");
 fr = GetString("fr");
}

String GetString(String key)
{
 String value = null;
 try
 {
   value = res.getString(key);
 }
 catch (MissingResourceException e)
 {
   value = key; // Poor substitute, but hey, might give an information anyway
 }
 return value;
}
Re: Localization [ i18n ]
Reply #2 - Aug 26th, 2009, 5:19am
 
UTF8ResourceBundle.java Code:
import java.util.ResourceBundle;
import java.util.PropertyResourceBundle;
import java.util.Locale;
import java.util.Enumeration;

// ResourceBundles are loaded in ISO-8859-1 codepage by Java.
// I found a trick in a blog article to load them in UTF-8 instead.
// Quick and Dirty Hack for UTF-8 Support in ResourceBundle
// <http://www.thoughtsabout.net/blog/archives/000044.html>
// Doesn't extend ResourceBundle because getBundle are final...
public abstract class UTF8ResourceBundle
{
// I keep the public API compatible with ResourceBundle
public static final ResourceBundle getBundle(String baseName)
{
ResourceBundle bundle = ResourceBundle.getBundle(baseName);
return CreateUTF8ResourceBundle(bundle);
}

public static final ResourceBundle getBundle(String baseName, Locale locale)
{
ResourceBundle bundle = ResourceBundle.getBundle(baseName, locale);
return CreateUTF8ResourceBundle(bundle);
}

public static final ResourceBundle getBundle(String baseName, Locale locale, ClassLoader loader)
{
ResourceBundle bundle = ResourceBundle.getBundle(baseName, locale, loader);
return CreateUTF8ResourceBundle(bundle);
}

private static ResourceBundle CreateUTF8ResourceBundle(ResourceBundle bundle)
{
// Trick is used only for property resource bundles, not for class resource bundles!
if (!(bundle instanceof PropertyResourceBundle))
return bundle;

return new UTF8PropertyResourceBundle((PropertyResourceBundle) bundle);
}

private static class UTF8PropertyResourceBundle extends ResourceBundle
{
PropertyResourceBundle m_bundle;

private UTF8PropertyResourceBundle(PropertyResourceBundle bundle)
{
m_bundle = bundle;
}

@Override
public Object handleGetObject(String key)
{
// Use getString (instead of handleGetObject) because:
// 1) It is only a PropertyResourceBundle
// 2) It allows fallback on parent bundle
String value = (String) m_bundle.getString(key);
if (value == null)
return null;

try
{
// The default resource bundle returns ISO-8859-1 strings. We get the bytes using this encoding,
// then transform them to string using UTF-8 encoding, which is the real encoding of the files
// (UTF-8 chars are valid ISO-8859-1 chars!)
return new String(value.getBytes("ISO-8859-1"), "UTF-8");
}
catch (java.io.UnsupportedEncodingException e)
{
return null; // Shouldn't go there if encoding strings above are OK...
}
}

@Override
public Enumeration<String> getKeys()
{
return m_bundle.getKeys();
}
}
}


ProcessingClassLoader.java Code:
import java.net.URL;

import processing.core.PApplet;

/**
* Class loader used only to load resources in typical Processing setup.
* Default class loaders look in class path, ie. Processing libs and where the class files are,
* typically in a randomly named build folder in system's temp dir.
* This class loader looks in the sketch's data folder instead, because that's where
* they are likely to be put and will be kept in an export.
*/
public class ProcessingClassLoader extends ClassLoader
{
private PApplet m_pa;

public ProcessingClassLoader(PApplet pa)
{
super();
m_pa = pa;
}

@Override
public URL getResource(String name)
{
String textURL = "file:///" + m_pa.dataPath(name);
// System.out.println("getResource " + textURL);
URL url = null;
try
{
url = new URL(textURL);
}
catch (java.net.MalformedURLException e)
{
System.out.println("ProcessingClassLoader - Incorrect URL: " + textURL);
}
return url;
}

// Not necessary, mostly there to see if it is used...
/*
@Override
public Class loadClass(String name, boolean resolve)
throws ClassNotFoundException
{
System.out.println("loadClass: " + name);
return findSystemClass(name);
}
*/
}
Re: Localization [ i18n ]
Reply #3 - Aug 26th, 2009, 5:30am
 
Localization.properties
APP_NAME = PolygotProcessing
APP_AUTH = PhiLho
slogan = That's a fine software!

fr = French
es = Spanish
en = English

# No need to define them, as the key is the value...
# Only because I fallback on the key as value in the string loading
#~ Title = Title
#~ Artist = Artist
#~ Album = Album
#~ Genre = Genre

Localization_en.properties
# Here only to catch generic English requests, and allow fallback on default strings

Localization_en_US.properties
en = American
slogan = That's a good software, boy!

Localization_es.properties
# app name and auth as in default translation
slogan = ¡Magnífico software!

fr = Francés
es = Español
en = Inglés

Title = Titulo
Artist = Artista
Album = Album
Genre = Género

Localization_fr.properties
APP_NAME = Processing Polygotte
slogan = C'est un bon logiciel !

fr = Français
es = Espagnol
en = Anglais

Title = Titre
Artist = Artiste
Album = Album
Genre = Genre

Localization_fr_CA.properties
slogan = Tabernacle !
Re: Localization [ i18n ]
Reply #4 - Aug 26th, 2009, 6:13am
 
Quote:
slogan = Tabernacle !

Grin
Re: Localization [ i18n ]
Reply #5 - Aug 28th, 2009, 4:33am
 
Follow-up: I wanted to play a bit with the message format (sentences with parameters) and choice format (handling plural form, etc.), so I extended my example.
I added two lines to draw():
Code:

fill(#337755);
text(releaseInfo, 50, infoPos + 80, width - 100, 50);
}

and I extended "a bit" the GetStrings function:
Code:

String appName, appAuth, slogan;
String title, artist, album, genre;
String en, es, fr, enC, esC, frC;
String titleValue = "Here Comes the Sun", artistValue = "The Beatles",
albumValue = "Abbey Road", genreValue = "Pop";
String releaseInfo;
void GetStrings(Locale locale)
{
println(locale.getLanguage() + " / " + locale.getCountry());
res = UTF8ResourceBundle.getBundle(bundleName, locale,
new ProcessingClassLoader(this));

// We distinguish between 0 items, 1 item and 2 or more items.
// According to the PluralForm.jsm file I found in Firefox 3 folders,
// this array should be dependent of the language: Latvian is different, so is Russian, etc.
double[] pluralLimits = { 0, 1, 2 };

// Simple translations, no parameters
appName = GetString("APP_NAME");
appAuth = GetString("APP_AUTH");
slogan = GetString("slogan");
title = GetString("Title");
artist = GetString("Artist");
album = GetString("Album");
genre = GetString("Genre");
en = GetString("en");
es = GetString("es");
fr = GetString("fr");

// Translations including parameters: the value order might depend on language
// So we use MessageFormat to handle this order, and formatting information (date, decimal/thousand separators...)
// depending on locale.

// Generic, will give patterns later
MessageFormat formatter = new MessageFormat("", locale);

// Disk number, I have to use a choice format
// to select the correct the plural form depending on the quantity.
String diskNbMsgPat = GetString("disk number");
String [] diskNbPats =
{
GetString("DN.zero"),
GetString("DN.one"),
GetString("DN.more")
};
ChoiceFormat choice = new ChoiceFormat(pluralLimits, diskNbPats);

int diskNb = (int) random(0, 4);
int diskNbMore = (int) (diskNb * 1E6 + random(1000, 1E6));

formatter.applyPattern(diskNbMsgPat);
// Apply the choice on pattern {0}
formatter.setFormatByArgumentIndex(0, choice);
Object[] diskStats =
{
diskNb, diskNbMore
};
String diskNbMsg = formatter.format(diskStats);

// Release information
String releaseInfoPat = GetString("release");
formatter.applyPattern(releaseInfoPat);
// I just want to display data according to the chosen locale...
String country = GetString(locale.getLanguage().toUpperCase()); // EN, ES, FR or other
Object[] information =
{
country,
// Some random date in late XXth century...
new Date((long) random(1E10, 1E12)),
diskNbMsg,
diskNbMore / 1.42E5,
};
releaseInfo = formatter.format(information);
}

I added comments hoping to highlight the usage of the classes.
Re: Localization [ i18n ]
Reply #6 - Aug 28th, 2009, 4:33am
 
I had also to change the localization files:
Quote:
Localization.properties
APP_NAME = PolygotProcessing
APP_AUTH = PhiLho
slogan = That's a fine software!

fr = French
es = Spanish
en = English

FR = France
ES = Spain
EN = England

# No need to define them, as the key is the value...
# Only because I fallback on the key as value in the string loading
#~ Title = Title
#~ Artist = Artist
#~ Album = Album
#~ Genre = Genre

# Dealing with compound messages and plurals
# 0 = country name, 1 = date & time, 2 = disk number (choice below), 3 = disk number per day
release = Released on {1,date,long} at {1,time,short} precisely, \
   it sold {2} in {0}, ie. {3,number,0.##} per day.

# 0 = million of disks (choice), 1 = exact number
disk\ number = {0} of disks ({1,number,integer} exactly)
DN.zero = below a million
DN.one = one million
DN.more = {0,number,integer} million

Localization_en.properties
# Here only to catch generic English requests, and allow fallback on default strings

Localization_en_US.properties
en = American
EN = United States of America
slogan = That's a good software, boy!

Localization_es.properties
# app name and auth as in default translation
slogan = ¡Magnífico software!

fr = Francés
es = Español
en = Inglés

FR = Francia
ES = España
EN = Inglaterra

Title = Titulo
Artist = Artista
Album = Album
Genre = Género

# Google translation...
# Liberado el 6 de enero 1961 a las 12:55, precisamente, que vendió 6 millones de discos (666 exactamente) en Francia, que es de 55 por día.
release = Liberado el {1,date,long} a las {1,time,short} precisamente, \
   que vendió {2} en {0}, que es {3,number,0.##} por día.
disk\ number = {0} de discos ({1,number,integer} exactamente)
DN.zero = debajo de un millón
DN.one = un millón
DN.more = {0,number,integer} millones

Localization_fr.properties
APP_NAME = Processing Polygotte
slogan = C'est un bon logiciel !

fr = Français
es = Espagnol
en = Anglais

FR = France
ES = Espagne
EN = Angleterre

Title = Titre
Artist = Artiste
Album = Album
Genre = Genre

# Warning: using MessageFormat which uses single quote as escape,
# you have to double them...
release = L''artiste a vendu {2} en {0} lors de la sortie de son album \
   le {1,date,long} (à {1,time,short} précisément). \
     Soit {3,number,0.##} par jour.
disk\ number = {0} de disques ({1,number,integer} exactement)
DN.zero = moins d'un million
DN.one = un million
DN.more = {0,number,integer} millions

Localization_fr_CA.properties
slogan = Tabernacle !
# I voluntarily leave here an inconsistency in French, to illustrate some translation issues
# that people not knowing the language might overlook:
# in English, we write 'in France', 'in Quebec', etc.
# in French, we write 'en France', 'au Québec' (au Yémen, aux États-Unis, etc.)
# So sometime code must be improved to handle linguistic peculiarities
# I suppose here that FR, EN, etc. can have a generic usage, so including the word in the country name is not an option...
# Same issue with country gender... Yes, we have genders and number, plus elisions for countries names in French,
# how strange! For example: la France, l'Espagne, le Japon, les États-Unis...
# I18n isn't a simple problem! Smiley
FR = Québec (Canada)
Re: Localization [ i18n ]
Reply #7 - Aug 28th, 2009, 4:35am
 
The same files, in UTF-8 encoding (instead of ISO-8859-1):
Quote:
Localization.properties
APP_NAME = PolygotProcessing
APP_AUTH = PhiLho
slogan = That's a fine software!

fr = French
es = Spanish
en = English

FR = France
ES = Spain
EN = England

# No need to define them, as the key is the value...
# Only because I fallback on the key as value in the string loading
#~ Title = Title
#~ Artist = Artist
#~ Album = Album
#~ Genre = Genre

# Dealing with compound messages and plurals
# 0 = country name, 1 = date & time, 2 = disk number (choice below), 3 = disk number per day
release = Released on {1,date,long} at {1,time,short} precisely, \
   it sold {2} in {0}, ie. {3,number,0.##} per day.

# 0 = million of disks (choice), 1 = exact number
disk\ number = {0} of disks ({1,number,integer} exactly)
DN.zero = below a million
DN.one = one million
DN.more = {0,number,integer} million

Localization_en.properties
# Here only to catch generic English requests, and allow fallback on default strings

Localization_en_US.properties
en = American
EN = United States of America
slogan = That's a good software, boy!

Localization_es.properties
# app name and auth as in default translation
slogan = ¡Magnífico software!

fr = Francés
es = Español
en = Inglés

FR = Francia
ES = España
EN = Inglaterra

Title = Titulo
Artist = Artista
Album = Album
Genre = Género

# Google translation...
# Liberado el 6 de enero 1961 a las 12:55, precisamente, que vendió 6 millones de discos (666 exactamente) en Francia, que es de 55 por día.
release = Liberado el {1,date,long} a las {1,time,short} precisamente, \
   que vendió {2} en {0}, que es {3,number,0.##} por día.
disk\ number = {0} de discos ({1,number,integer} exactamente)
DN.zero = debajo de un millón
DN.one = un millón
DN.more = {0,number,integer} millones

Localization_fr.properties
APP_NAME = Processing Polygotte
slogan = C'est un bon logiciel !

fr = Français
es = Espagnol
en = Anglais

FR = France
ES = Espagne
EN = Angleterre

Title = Titre
Artist = Artiste
Album = Album
Genre = Genre

# Warning: using MessageFormat which uses single quote as escape,
# you have to double them...
release = L''artiste a vendu {2} en {0} lors de la sortie de son album \
   le {1,date,long} (à {1,time,short} précisément). \
     Soit {3,number,0.##} par jour.
disk\ number = {0} de disques ({1,number,integer} exactement)
DN.zero = moins d'un million
DN.one = un million
DN.more = {0,number,integer} millions

Localization_fr_CA.properties
slogan = Tabernacle !
# I voluntarily leave here an inconsistency in French, to illustrate some translation issues
# that people not knowing the language might overlook:
# in English, we write 'in France', 'in Quebec', etc.
# in French, we write 'en France', 'au Québec' (au Yémen, aux États-Unis, etc.)
# So sometime code must be improved to handle linguistic peculiarities
# I suppose here that FR, EN, etc. can have a generic usage, so including the word in the country name is not an option...
# Same issue with country gender... Yes, we have genders and number, plus elisions for countries names in French,
# how strange! For example: la France, l'Espagne, le Japon, les États-Unis...
# I18n isn't a simple problem! Smiley
FR = Québec (Canada)
Re: Localization [ i18n ]
Reply #8 - Aug 29th, 2009, 11:18pm
 
Man! that's a huge lot of code!!!

I have no words to thank you! Cheesy

I'll check it in deep right away!
Re: Localization [ i18n ]
Reply #9 - Aug 30th, 2009, 1:14am
 
No problem, it was fun and interesting to do.
I wrote a French blog article on the topic: Internationalisation des sketchs Processing (et des programmes Java).
Re: Localization [ i18n ]
Reply #10 - Aug 30th, 2009, 5:14am
 
It worked beatifully!

You are now in the credits of the app, If you'd like to make the french translaton, You'd be very welcome to do so.

I'll pass you the link when I have a new update up in my site.

Again, Thank you very much!  Cool
I18n for Processing now in a Library
Reply #11 - Sep 15th, 2009, 1:32am
 
I encapsulated most of the code in these examples you published here into a Processing Library.

I also managed to make it super easy to use, you just make the object, initialize it with any of two constructors and then just request the strings.

Check it out over here! http://0p0media.com/i18n/

Page in english don't worry! Cheesy
Re: Localization [ i18n ]
Reply #12 - Sep 15th, 2009, 8:03am
 
Related library thread: Internationalization: I18n for Processing. with a fix of the above code for working in exported applications and applets.
Page Index Toggle Pages: 1