One Application, Many Locales
One of the goals of Qt is to make it easy, or at least easier to port from one operating system to another. Windows moving to Linux desktop? Sure. Macintosh OS X to Symbian? Ok.
What about going from Norway to Germany? Australia to the US of A? Is there any reason that this shouldn’t be nearly automatic? While there are a number of details to be aware of, The QLocale class tries to encapsulate many of these.
The reason I’m bringing this up is that QLocale has some shortcomings. While this isn’t new, a few use cases have come up recently that highlight the issue. These cases have been accompanied by pointers to other locale data implementations. One example is the MeeGo Touch MLocale class. Another is the Symbian localization APIs. It would be nice to provide some of these features on all Qt platforms.
Quantifying the Problem
We’ve come up with a short list of things that are missing from QLocale, in no particular order.
- Currency
- Collation
- Writing script
- Numeral script
- Calendar
- Work week days
- First day of the week
- 24 hour time format
- Time zone
- Phone number format
- Language endonym
Additionally, we’ve been asked about mixed locales. To illustrate: I’m sitting here in Norway writing using US English rules, but when I look up the location of the nearest grocery store, I want the distance units to be in kilometers. The locale database built into Qt doesn’t have an en_US for some things, *_NO for others locale. MeeGo Touch divides the locale into a number of categories. This division brings up a question: Given the ability to go from a locale (en_US, for example) to the handling of a particular piece of data (the date, to continue), does Qt need to include the ability to reverse the operation? If I know that it’s January 1, 2011, is knowing that this is an en_US formatted date important?
Another issue raised is the value of the built-in locales in Qt. Are these useful, or should Qt rely on the operating system’s knowledge of locales? Should the codeset and modifiers documented be supported in a meaningful manner?
Planning a Solution
The thought I want to leave you with is: What do you need from Qt’s locale support? Validation or rejection of the features discussed here are encouraged. And if I’ve missed something, please speak up.
Possibly related posts:
30 comments
I’d suggest looking at KDE’s KLocale for inspiration.
As a former .Net developer, this is one of the areas I missed the most when moving to Qt. The .Net CultureInfo and associated classes were woven into the system in every relevant location, i.e. String.Compare would have an overload that accepted a culture. This was extremely useful. Looking at some of their functionality might be a good place for inspiration.
In Qt, I have had to fall back to the ICU library. I have found it quite powerful, but the API and documentation have A LOT to be desired. Wrapping much of this into a Qt-friendly API could go a long way.
The features I have missed the most:
– Collation
– Tight integration with collection classes, i.e. the ability to create a QMap, say, using case-insensitive Thai rules for string comparison.
Interestingly, many of those problems are solved by KDE’s KLocale class [1], including different calendar systems.
[1] http://api.kde.org/4.x-api/kdelibs-apidocs/kdecore/html/classKLocale.html
Have a look at Boost.Locale [1]. I’ve found it an interesting read.
[1] http://cppcms.sourceforge.net/boost_locale/html/tutorial.html
The most useful additions for me would be: collation, calendar, first day of the week, 24h time format, and time zone. I’d also appreciate mixed-locales support.
Language.
That, in my opinion, is the most glaring shortcoming of QLocale. It’s designed for dealing with numbers and dates and such, and has proven to be inappropriate to determine what language the application should display its localizable text in.
@Adam Higerd: Can you clarify how QLocale::language() is falling short?
Having written a financial package with Qt I missed QLocale support for currency. It is US-centric right now but I’ve rolled my own (untested) currency support for future International versions. Would be nice if it was just “handled” for me.
On Mac OS X, there are severe performance issues as well: http://wimleers.com/blog/qcachinglocale-speeding-up-qsystemlocalequery-calls. This forced me to write my own caching layer: QCachingLocale, available in the public domain, code on GitHub: https://github.com/wimleers/QCachingLocale.
Please consider including similar functionality in Qt 4.8 by default.
Time zone is the most important one for me. I really don’t want to go fiddling about with ICU just for that.
I’m glad you bring up this subject, as it’s been a point of frustration for me time and again. Your example (writing en in an otherwise no setting) gives me hope that you’ll understand the problem and its complicated cases well enough to get it right.
I’m swiss, so I prefer my unit systems and currency formats to be in what we’re accustomed to here. While german is my native language, I prefer english as interface languages for my electronic devices, since technical terms seem to me more standardized there; something always gets lost in translation when localizing UIs. I communicate in several languages however, swiss german, the local dialect, standard german, english, and sometimes french. This alone is a difficult enough situation so that I rarely if ever use assists like spell checkers or typing prediction on mobile devices. And I prefer my dates to be in ISO8601 format (“2010-12-31″).
I hope you can implement a solution that allows developers to deal with the complexity of such a situation in a sane way.
@Wim: That’s interesting. I believe that Windows and OS X are unique in that Qt queries the system for locale configuration. Being primarily a Linux user, I haven’t pushed either very far. Please file a bug report with your test case[1]. I applaud the fix-it-yourself ethic, but this sounds like it deserves a look in Qt proper. Unfortunately, unless it is submitted as either a contribution or reported as a bug, it’s pretty much invisible to us. Unfortunately blog post replies are easily forgotten.
[1] Excellent test case! Short, sweet, and to the point.
@Benjamin: I believe that I understand your situation, and I sympathize.
One of the things that I was thinking about is the language preference list that KDE, Firefox, and probably a few others offer. That said, there are some things that I don’t trust translations of. Government documents frequently fall into that category.
Sometimes the priority list works as desired, and sometimes I need to explicitly select a second or third choice language because my first choice is available but unusable. I can imagine Qt offering a list of languages the user is interested in, but I feel that applications will still need to provide a manual override.
Platform specific issues are always going to be a problem, or how will they be handled in Qt? Will having a totally different implementation of locale create apps that are even less platform native?
In any case I have these sort of issues that drive me crazy. For example, on Windows I use en_US in Finland, so I want metric and €. But if I set my English Windows to Finnish locale I get all my dates in Finnish even though the UI setting is English!
In Symbian if I set my input language to English it also sets the keyboard to English which makes inputting Finnish street names extremely difficult (so language and keyboard layout should be separate). In Maemo at least it is possible to have at least two input languages, but not in Symbian.
After checking out Boost.Locale (thanks Cristian) I would suggest: If you do anything, make sure it’s compatible with boost
There are a lot of migrant coders. The tools themselves will be used by Chinese, Indians and English-speakers.
There are a lot of migrant users; mobile phones and PDAs are, um, mobile. They travel. Users need local info in their native language.
QEmergency: Local way to dial emergency help, (911 in US). Some countries don’t have a unified number. Israel has 3: One for police, one for fire, one for ambulance. Especially important for mobile phone users. The cell/mobile companies will probably have to help. One universal number in every phone, locally translated by the cell phone company? Or make 911 work for Americans all over the world by translating 911 to the local system.
Changes: Russia recently changed from 11 time zones to 9. Some countries pass a new daylight-savings time law every year. Eventually the US will go metric. Option to query a localization website for the latest news? Eventually the ICANN and/or the UN will administer this. International politics, here we come!
A complete QLocale requires that they complete the projects known as Anthropology and Linguistics. Humanities, here we come!
Some Locale issues become apps all to themselves: Translation, shopping & local taxes, local emergency calling, weather.
Check out the Bible for the story of the Tower of Babel. Religion, here we come!
Dates. I never get one. And when it comes to Qt, I never get the right date/time. See, back in the 80s up to the mid 90s, computers rarely talked to another computer that wasn’t in the same time zone. Now, you talk to more computers out of your time zone than ones in it. When I write 2:10, I mean 2:10 ET. But if I am working collaboratively with people in other timezones, they may write 1:30, which in CT, is 10 minutes after my time stamp, but will be sorted incorrectly. Additionally, when thinks like DST kick in and out, problems arise. Workflow is interrupted. In the case of time, it is easy to say all storage is in UTC, then translated to local time. For all other metrics, we should choose the most arcane storage so there is no default. Imagine if NASA used rods and hogsheads in their specifications. No one would ever to convert to/from English/Metric.
Of course, QLocale should translarently provide the conversion.
This then begs the question if QLocale needs things like QDistance, QPressure, QWeight, etc..
No one would ever to *forget to* convert to/from English/Metric.
Calendar, first day of the week, Work week days and collation are the most useful thing for me…
I have also been thinking about the limitations of QLocale, QString and QDateTime.
I think using ICU for collations, writing and number scripts integrated into QString would be positive since ICU is cross-platform, popular and quite comprehensive.
I also understand ICU has functionality for date manipulation and might be added to QDateTime. For time-zones, we had to write our own code.
I think you ought to ignore phone numbers since the the formatting of phone and fax numbers throughout the world is too variable. Better to constrain the user to inputting +country_code (area_code) local_number where local_number can accept string containing dots, spaces, hypens etc.
Hope this helps
If you come up with somethin that covers every case, it will almost certainly be too complex to use. I suggest you concentrate on the 20% of functionality that covers 80% of real world cases.
@Paul: The mentioned MLocale of MeeGoTouch is actually internally using ICU that way. Undersigned being the original author of those classes.
ICU does provide quite a nice set of internationalization interfaces, but also does have some flaws I became quite annoyed from, e.g.:
- API difficult to use and not too well documented
- Library size is big (data file library being 16MB for 4.2.1 here, though can be stripped on compilation time)
- Doesn’t provide ABI stability between versions
- Has somewhat schizophrenic attitude being C/C++ and Java library as two independent implementations, but trying to keep the same changes on both.
But being implemented in whatever way, I gladly welcome these enhancements to QLocale.
we made a qt widget using qt4 and deployed it in linux. but how to make it compatible with orca screen reader?? our project is orca support.can u please help us…
What – apart from the already mentioned localization issues – may be interesting to have, is information regarding the formatting of adresses. Postal codes are an important part of that, but not the only one. For instance, in some countries you put a number first, and then a street name, in others it is other way around.
Name formatting is also an issue. The order in which parts of names appear differs between countries, as does the status of prefixes for things like sorting.
How about adding mutability to QLocale? eg. setDecimalPoint()
Can’t see a reason to why the locale should be updated automatically as from my point of view the Locale is tied to the user “home” location (or user’s preferred formatting) not its current location, why would the date format change just because you travelled from norway to US.
As other have mentioned you should have the possibility to customize the settings freely, like a different localization than what is currently default in the locale.
timezones – you probably need a default and a current one as there shouldn’t be a need to change the locale just because you are travelling.
Can we also have text versions of the languages/locations which can be put in a menu? There are some methods which return mangled names from the IDs but you can’t find out how many there are (unless checking for string unknown is somehow valid).
For my app the users often want numbers in english format even if they are based worldwide because it is standard in science.
I miss everything which are not in MLocale
(also the word/sentence break iterator which is not on your list: http://apidocs.meego.com/git-tip/mtf/class_m_break_iterator.html)
Also Qt is too tightly coupled with POSIX locale, which controls the QLocale outputs. CLDR is the future.
@Mohammad: For what it’s worth, there’s QTextBoundaryFinder that can help finding grapheme/word/line/sentence boundaries:
http://doc.qt.nokia.com/latest/qtextboundaryfinder.html
Also, if you are looking at timezones, you may want to look at KTimezone and its related classes. The original rationale for them was (a) to provide OS-independent access to the timezone database, and (b) allow arbitrary timezones to be created on-the-fly. The latter was needed because calendar formats such as vCAL/iCAL carry their own “timezone” entry, and if you want to interact with these in a timezone-sensitive application, some kind of unifying framework is needed.
Disclaimer: although I was the originator of KTimezone, the vast bulk of the effort of maturing it has been done by others, notably djarvie.
Thanks for all the feedback. We’ll be updating a bug tracker task as the project progresses.
http://bugreports.qt.nokia.com/browse/QTBUG-16178
Comments on this entry are closed.