Best practice multi language website

Question

I've been struggling with this question for quite some months now, but I haven't been in a situation that I needed to explore all possible options before. Right now, I feel like it's time to get to know the possibilities and create my own personal preference to use in my upcoming projects.

Let me first sketch the situation I'm looking for

I'm about to upgrade/redevelop a content management system which I've been using for quite a while now. However, I'm feeling multi language is a great improvement to this system. Before I did not use any frameworks but I'm going to use Laraval4 for the upcoming project. Laravel seems the best choice of a cleaner way to code PHP. Sidenote: Laraval4 should be no factor in your answer. I'm looking for general ways of translation that are platform/framework independent.

What should be translated

As the system I am looking for needs to be as user friendly as possible the method of managing the translation should be inside the CMS. There should be no need to start up an FTP connection to modify translation files or any html/php parsed templates.

Furthermore, I'm looking for the easiest way to translate multiple database tables perhaps without the need of making additional tables.

What did I come up with myself

As I've been searching, reading and trying things myself already. There are a couple of options I have. But I still don't feel like I've reached a best practice method for what I am really seeking. Right now, this is what I've come up with, but this method also has it side effects.

1. PHP Parsed Templates: the template system should be parsed by PHP. This way I'm able to insert the translated parameters into the HTML without having to open the templates and modify them. Besides that, PHP parsed templates gives me the ability to have 1 template for the complete website instead of having a subfolder for each language (which I've had before). The method to reach this target can be either Smarty, TemplatePower, Laravel's Blade or any other template parser. As I said this should be independent to the written solution.
2. Database Driven: perhaps I don't need to mention this again. But the solution should be database driven. The CMS is aimed to be object oriented and MVC, so I would need to think of a logical data structure for the strings. As my templates would be structured: templates/Controller/View.php perhaps this structure would make the most sense: Controller.View.parameter. The database table would have these fields a long with a value field. Inside the templates we could use some sort method like echo __('Controller.View.welcome', array('name', 'Joshua')) and the parameter contains Welcome, :name. Thus the result being Welcome, Joshua. This seems a good way to do this, because the parameters such as :name are easy to understand by the editor.
3. Low Database Load: Of course the above system would cause loads of database load if these strings are being loaded on the go. Therefore I would need a caching system that re-renders the language files as soon as they are edited/saved in the administration environment. Because files are generated, also a good file system layout is needed. I guess we can go with languages/en_EN/Controller/View.php or .ini, whatever suits you best. Perhaps an .ini is even parsed quicker in the end. This fould should contain the data in the format parameter=value; . I guess this is the best way of doing this, since each View that is rendered can include it's own language file if it exists. Language parameters then should be loaded to a specific view and not in a global scope to prevent parameters from overwriting each other.
4. Database Table translation: this in fact is the thing I'm most worried about. I'm looking for a way to create translations of News/Pages/etc. as quickly as possible. Having two tables for each module (for example News and News_translations) is an option but it feels like to much work to get a good system. One of the things I came up with is based on a data versioning system I wrote: there is one database table name Translations, this table has a unique combination of language, tablename and primarykey. For instance: en_En / News / 1 (Referring to the English version of the News item with ID=1). But there are 2 huge disadvantages to this method: first of all this table tends to get pretty long with a lot of data in the database and secondly it would be a hell of a job to use this setup to search the table. E.g. searching for the SEO slug of the item would be a full text search, which is pretty dumb. But on the other hand: it's a quick way to create translatable content in every table very fast, but I don't believe this pro overweights the con's.
5. Front-end Work: Also the front-end would need some thinking. Of course we would store the available languages in a database and (de)active the ones we need. This way the script can generate a dropdown to select a language and the back-end can decide automatically what translations can be made using the CMS. The chosen language (e.g. en_EN) would then be used when getting the language file for a view or to get the right translation for a content item on the website.

So, there they are. My ideas so far. They don't even include localization options for dates etc yet, but as my server supports PHP5.3.2+ the best option is to use the intl extension as explained here: http://devzone.zend.com/1500/internationalization-in-php-53/ - but this would be of use in any later stadium of development. For now the main issue is how to have the best practics of translation of the content in a website.

Besides everything I explained here, I still have another thing which I haven't decided yet, it looks like a simple question, but in fact it's been giving me headaches:

URL Translation? Should we do this or not? and in what way?

So.. if I have this url: http://www.domain.com/about-us and English is my default language. Should this URL be translated into http://www.domain.com/over-ons when I choose Dutch as my language? Or should we go the easy road and simply change the content of the page visible at /about. The last thing doesn't seem a valid option because that would generate multiple versions of the same URL, this indexing the content will fail the right way.

Another option is using http://www.domain.com/nl/about-us instead. This generates at least a unique URL for each content. Also this would be easier to go to another language, for example http://www.domain.com/en/about-us and the URL provided is easier to understand for both Google and Human visitors. Using this option, what do we do with the default languages? Should the default language remove the language selected by default? So redirecting http://www.domain.com/en/about-us to http://www.domain.com/about-us ... In my eyes this is the best solution, because when the CMS is setup for only one language there is no need to have this language identification in the URL.

And a third option is a combination from both options: using the "language-identification-less"-URL (http://www.domain.com/about-us) for the main language. And use an URL with a translated SEO slug for sublanguages: http://www.domain.com/nl/over-ons & http://www.domain.com/de/uber-uns

I hope my question gets your heads cracking, they cracked mine for sure! It did help me already to work things out as a question here. Gave me a possibility to review the methods I've used before and the idea's I'm having for my upcoming CMS.

I would like to thank you already for taking the time to read this bunch of text!

// Edit #1:

I forgot to mention: the __() function is an alias to translate a given string. Within this method there obviously should be some sort of fallback method where the default text is loaded when there are not translations available yet. If the translation is missing it should either be inserted or the translation file should be regenerated.

Fairspoken

what about gettext? php.net/manual/en/book.gettext.php – Corot

What about i18next.com + locize.com ? – Calculable

Answer 1

Topic's premise

There are three distinct aspects in a multilingual site:

• interface translation
• content
• url routing

While they all interconnected in different ways, from CMS point of view they are managed using different UI elements and stored differently. You seem to be confident in your implementation and understanding of the first two. The question was about the latter aspect - "URL Translation? Should we do this or not? and in what way?"

What the URL can be made of?

A very important thing is, don't get fancy with IDN. Instead favor transliteration (also: transcription and romanization). While at first glance IDN seems viable option for international URLs, it actually does not work as advertised for two reasons:

• some browsers will turn the non-ASCII chars like 'ч' or 'ž' into '%D1%87' and '%C5%BE'
• if user has custom themes, the theme's font is very likely to not have symbols for those letters

I actually tried to IDN approach few years ago in a Yii based project (horrible framework, IMHO). I encountered both of the above mentioned problems before scraping that solution. Also, I suspect that it might be an attack vector.

Available options ... as I see them.

Basically you have two choices, that could be abstracted as:

• http://site.tld/[:query]: where [:query] determines both language and content choice

• http://site.tld/[:language]/[:query]: where [:language] part of URL defines the choice of language and [:query] is used only to identify the content

Query is Α and Ω ..

Lets say you pick http://site.tld/[:query].

In that case you have one primary source of language: the content of [:query] segment; and two additional sources:

• value $_COOKIE['lang'] for that particular browser
• list of languages in HTTP Accept-Language ^{(1), (2)} header

First, you need to match the query to one of defined routing patterns (if your pick is Laravel, then read here). On successful match of pattern you then need to find the language.

You would have to go through all the segments of the pattern. Find the potential translations for all of those segments and determine which language was used. The two additional sources (cookie and header) would be used to resolve routing conflicts, when (not "if") they arise.

Take for example: http://site.tld/blog/novinka.

That's transliteration of "блог, новинка", that in English means approximately "blog", "latest".

As you can already notice, in Russian "блог" will be transliterated as "blog". Which means that for the first part of [:query] you (in the best case scenario) will end up with ['en', 'ru'] list of possible languages. Then you take next segment - "novinka". That might have only one language on the list of possibilities: ['ru'].

When the list has one item, you have successfully found the language.

But if you end up with 2 (example: Russian and Ukrainian) or more possibilities .. or 0 possibilities, as a case might be. You will have to use cookie and/or header to find the correct option.

And if all else fails, you pick the site's default language.

Language as parameter

The alternative is to use URL, that can be defined as http://site.tld/[:language]/[:query]. In this case, when translating query, you do not need to guess the language, because at that point you already know which to use.

There is also a secondary source of language: the cookie value. But here there is no point in messing with Accept-Language header, because you are not dealing with unknown amount of possible languages in case of "cold start" (when user first time opens site with custom query).

Instead you have 3 simple, prioritized options:

1. if [:language] segment is set, use it
2. if $_COOKIE['lang'] is set, use it
3. use default language

When you have the language, you simply attempt to translate the query, and if translation fails, use the "default value" for that particular segment (based on routing results).

Isn't here a third option?

Yes, technically you can combine both approaches, but that would complicate the process and only accommodate people who want to manually change URL of http://site.tld/en/news to http://site.tld/de/news and expect the news page to change to German.

But even this case could probable be mitigated using cookie value (which would contain information about previous choice of language), to implement with less magic and hope.

Which approach to use?

As you might already guessed, I would recommend http://site.tld/[:language]/[:query] as the more sensible option.

Also in real word situation you would have 3rd major part in URL: "title". As in name of the product in online shop or headline of article in news site.

Example: http://site.tld/en/news/article/121415/EU-as-global-reserve-currency

In this case '/news/article/121415' would be the query, and the 'EU-as-global-reserve-currency' is title. Purely for SEO purposes.

Can it be done in Laravel?

Kinda, but not by default.

I am not too familiar with it, but from what I have seen, Laravel uses simple pattern-based routing mechanism. To implement multilingual URLs you will probably have to extend core class(es), because multilingual routing need access to different forms of storage (database, cache and/or configuration files).

It's routed. What now?

As a result of all you would end up with two valuable pieces of information: current language and translated segments of query. These values then can be used to dispatch to the class(es) which will produce the result.

Basically, the following URL: http://site.tld/ru/blog/novinka (or the version without '/ru') gets turned into something like

$parameters = [
   'language' => 'ru',
   'classname' => 'blog',
   'method' => 'latest',
];

Which you just use for dispatching:

$instance = new {$parameter['classname']};
$instance->{'get'.$parameters['method']}( $parameters );

.. or some variation of it, depending on the particular implementation.

Maier

Thanks for yet another insight! Very thoughtfull! I was thinking of having the language parameter in the URL aswell. This simply seems the best way to identify a particular language, not only for the user but for SEO purposes aswel. In case a user changes /en/news to /de/news, my idea was to do a 301 (permanent) redirect to /de/nachrichten for example. Just to make sure each language has just one unique URL per page (again for SEO purposes) – Fairspoken

It's getting harder and harder to select the best answer, currently there are about 3/4 answers that deserve at least a part of the bounty each. Combined they become a solid answer to everything I wanted to clear up together :) – Fairspoken

I acccepted your answer to give you at least some extra rep for the detailed answer you gave on URL translation. Highly appreciated! However, the bounty is awarder to the person below you since he answered every aspect of my question in a platform independent way. – Fairspoken

Answer 2

Implementing i18n Without The Performance Hit Using a Pre-Processor as suggested by Thomas Bley

At work, we recently went through implementation of i18n on a couple of our properties, and one of the things we kept struggling with was the performance hit of dealing with on-the-fly translation, then I discovered this great blog post by Thomas Bley which inspired the way we're using i18n to handle large traffic loads with minimal performance issues.

Instead of calling functions for every translation operation, which as we know in PHP is expensive, we define our base files with placeholders, then use a pre-processor to cache those files (we store the file modification time to make sure we're serving the latest content at all times).

The Translation Tags

Thomas uses {tr} and {/tr} tags to define where translations start and end. Due to the fact that we're using TWIG, we don't want to use { to avoid confusion so we use [%tr%] and [%/tr%] instead. Basically, this looks like this:

`return [%tr%]formatted_value[%/tr%];`

Note that Thomas suggests using the base English in the file. We don't do this because we don't want to have to modify all of the translation files if we change the value in English.

The INI Files

Then, we create an INI file for each language, in the format placeholder = translated:

// lang/fr.ini
formatted_value = number_format($value * Model_Exchange::getEurRate(), 2, ',', ' ') . '€'

// lang/en_gb.ini
formatted_value = '£' . number_format($value * Model_Exchange::getStgRate())

// lang/en_us.ini
formatted_value = '$' . number_format($value)

It would be trivial to allow a user to modify these inside the CMS, just get the keypairs by a preg_split on \n or = and making the CMS able to write to the INI files.

The Pre-Processor Component

Essentially, Thomas suggests using a just-in-time 'compiler' (though, in truth, it's a preprocessor) function like this to take your translation files and create static PHP files on disk. This way, we essentially cache our translated files instead of calling a translation function for every string in the file:

// This function was written by Thomas Bley, not by me
function translate($file) {
  $cache_file = 'cache/'.LANG.'_'.basename($file).'_'.filemtime($file).'.php';
  // (re)build translation?
  if (!file_exists($cache_file)) {
    $lang_file = 'lang/'.LANG.'.ini';
    $lang_file_php = 'cache/'.LANG.'_'.filemtime($lang_file).'.php';

    // convert .ini file into .php file
    if (!file_exists($lang_file_php)) {
      file_put_contents($lang_file_php, '<?php $strings='.
        var_export(parse_ini_file($lang_file), true).';', LOCK_EX);
    }
    // translate .php into localized .php file
    $tr = function($match) use (&$lang_file_php) {
      static $strings = null;
      if ($strings===null) require($lang_file_php);
      return isset($strings[ $match[1] ]) ? $strings[ $match[1] ] : $match[1];
    };
    // replace all {t}abc{/t} by tr()
    file_put_contents($cache_file, preg_replace_callback(
      '/\[%tr%\](.*?)\[%\/tr%\]/', $tr, file_get_contents($file)), LOCK_EX);
  }
  return $cache_file;
}

Note: I didn't verify that the regex works, I didn't copy it from our company server, but you can see how the operation works.

How to Call It

Again, this example is from Thomas Bley, not from me:

// instead of
require("core/example.php");
echo (new example())->now();

// we write
define('LANG', 'en_us');
require(translate('core/example.php'));
echo (new example())->now();

We store the language in a cookie (or session variable if we can't get a cookie) and then retrieve it on every request. You could combine this with an optional $_GET parameter to override the language, but I don't suggest subdomain-per-language or page-per-language because it'll make it harder to see which pages are popular and will reduce the value of inbound links as you'll have them more scarcely spread.

Why use this method?

We like this method of preprocessing for three reasons:

1. The huge performance gain from not calling a whole bunch of functions for content which rarely changes (with this system, 100k visitors in French will still only end up running translation replacement once).
2. It doesn't add any load to our database, as it uses simple flat-files and is a pure-PHP solution.
3. The ability to use PHP expressions within our translations.

Getting Translated Database Content

We just add a column for content in our database called language, then we use an accessor method for the LANG constant which we defined earlier on, so our SQL calls (using ZF1, sadly) look like this:

$query = select()->from($this->_name)
                 ->where('language = ?', User::getLang())
                 ->where('id       = ?', $articleId)
                 ->limit(1);

Our articles have a compound primary key over id and language so article 54 can exist in all languages. Our LANG defaults to en_US if not specified.

URL Slug Translation

I'd combine two things here, one is a function in your bootstrap which accepts a $_GET parameter for language and overrides the cookie variable, and another is routing which accepts multiple slugs. Then you can do something like this in your routing:

"/wilkommen" => "/welcome/lang/de"
... etc ...

These could be stored in a flat file which could be easily written to from your admin panel. JSON or XML may provide a good structure for supporting them.

Notes Regarding A Few Other Options

PHP-based On-The-Fly Translation

I can't see that these offer any advantage over pre-processed translations.

Front-end Based Translations

I've long found these interesting, but there are a few caveats. For example, you have to make available to the user the entire list of phrases on your website that you plan to translate, this could be problematic if there are areas of the site you're keeping hidden or haven't allowed them access to.

You'd also have to assume that all of your users are willing and able to use Javascript on your site, but from my statistics, around 2.5% of our users are running without it (or using Noscript to block our sites from using it).

Database-Driven Translations

PHP's database connectivity speeds are nothing to write home about, and this adds to the already high overhead of calling a function on every phrase to translate. The performance & scalability issues seem overwhelming with this approach.

Flirt

I see that I've confused you with "Front-end Translations", what I meant was a way to parse the translated strings on screen. I'm definitly not looking for a way to translate it on the client side! What I meant was the easiest way to switch languages on the front-end, but that's obviously using a cookie or a user setting :) – Fairspoken

Oh, and by Database-Driven I was more aiming on the method of managing all translations, so my ideal solution would be a back-end which writes translations to a database followed by a function that generates the pre-processing component which generates the PHP file. Why?: simple.. I want don't want to be bother by little changes in text, the users should be able to do so themselfs without using a code editor and/or ftp programm :) – Fairspoken

@PENDO I know you didn't mean front-end translations, that was a thinly veiled comment towards the user who suggested front-end translation frameworks using JS. ;) – Flirt

@PENDO I agree, I'd use the backend as you suggested but instead of a database I'd use a flat file for performance reasons. Of course, the core suggestion here is pre-rendering the templates upon change so you could replace the .INI files with a 3-column database table with placeholder, replacement, language. Compound key on placeholder and language. Then have another 2-col with tempfile (path to template) and modified (DATETIME). – Flirt

I think that's indeed the way I'm going for. Since it isn't a daily task to change the translations and I'm going for a "categorised" setup of language strings as much as possible I don't think the load will be that much for the endusers on the front-end of the website. Thanks again! – Fairspoken

Congratulations, for me this is the best answer since every question is taking into consideration in your answer. Combined with the answer about (about URL translation) it makes one perfect answer, therefor I wen with accepting his: to make it obvious that both answers should belong together. – Fairspoken

@PENDO Thanks. I've put 250 back up and I plan to award it to teresko in 24 hours when the site lets me, as you selected both answers as correct, and I think a split would best represent your intentions. – Flirt

Hmm, I wonder if a proper frontend caching won't do this better and more fine-grained... . – Shayn

@AlulaErrorpone, thanks for the cool post! Do you might have a complete example showing how the method works? i.e. some complete code to download? Some parts where not so clear to me, since I'm not so experienced with webprogramming as you, .e.g how to define the .ini files, etc. – Woodenhead

define('LANG', 'en_us'); what is that? hardcode... I want browser to detect language automatically. – Sthilaire

I got inspired by this answer and create some kind of 'proof-of-concept' package at github.com/allofmex/translating-autoloader It also includes auto-loading and translate-but-exclude-section feature. – Isidore

Answer 3

I suggest you not to invent a wheel and use gettext and ISO languages abbrevs list. Have you seen how i18n/l10n implemented in popular CMSes or frameworks?

Using gettext you will have a powerful tool where many of cases is already implemented like plural forms of numbers. In english you have only 2 options: singular and plural. But in Russian for example there are 3 forms and its not as simple as in english.

Also many translators already have experience to work with gettext.

Take a look to CakePHP or Drupal . Both multilingual enabled. CakePHP as example of interface localization and Drupal as example of content translation.

For l10n using database isn't the case at all. It will be tons on queries. Standard approach is to get all l10n data in memory in early stage (or during first call to i10n function if you prefer lazy loading). It can be reading from .po file or from DB all data at once. And than just read requested strings from array.

If you need to implement online tool to translate interface you can have all that data in DB but than still save all data to file to work with it. To reduce amount of data in memory you can split all your translated messages/strings into groups and than load only that groups you need if it will be possible.

So you totally right in your #3. With one exception: usually it is one big file not a per-controller file or so. Because it is best for performance to open one file. You probably know that some highloaded web apps compiles all PHP code in one file to avoid file operations when include/require called.

About URLs. Google indirectly suggest to use translation:

to clearly indicate French content: http://example.ca/fr/vélo-de-montagne.html

Also i think you need to redirect user to default language prefix e.g. http://examlpe.com/about-us will redirects to http://examlpe.com/en/about-us But if your site use only one language so you don't need prefixes at all.

Check out: http://www.audiomicro.com/trailer-hit-impact-psychodrama-sound-effects-836925 http://nl.audiomicro.com/aanhangwagen-hit-effect-psychodrama-geluidseffecten-836925 http://de.audiomicro.com/anhanger-hit-auswirkungen-psychodrama-sound-effekte-836925

Translating content is more difficult task. I think it will be some differences with different types of content e.g. articles, menu items etc. But in #4 you're in the right way. Take a look in Drupal to have more ideas. It have clear enough DB schema and good enough interface for translating. Like you creating article and select language for it. And than you can later translate it to other languages.

I think it isn't problem with URL slugs. You can just create separate table for slugs and it will be right decision. Also using right indexes it isn't problem to query table even with huge amount of data. And it wasn't full text search but string match if will use varchar data type for slug and you can have an index on that field too.

PS Sorry, my English is far from perfect though.

Territoriality

Thanks for the time taken to answer to my question. Your English is well enough for me to understand! I'm gonna +1 you already for your efforts! – Fairspoken

Yaroslav, once again, thanks for your answer. However I went with 2 other answers that where a bit more complete and explain the methods used behind the code instead of pointing out it is already there. – Fairspoken

No problem. Indeed that answers more complete and interesting to read for me too. But i hope you got something useful from my answer too. – Territoriality

Answer 4

It depends on how much content your website has. At first I used a database like all other people here, but it can be time-consuming to script all the workings of a database. I don't say that this is an ideal method and especially if you have a lot of text, but if you want to do it fast without using a database, this method could work, though, you can't allow users to input data which will be used as translation-files. But if you add the translations yourself, it will work:

Let's say you have this text:

Welcome!

You can input this in a database with translations, but you can also do this:

$welcome = array(
"English"=>"Welcome!",
"German"=>"Willkommen!",
"French"=>"Bienvenue!",
"Turkish"=>"Hoşgeldiniz!",
"Russian"=>"Добро пожаловать!",
"Dutch"=>"Welkom!",
"Swedish"=>"Välkommen!",
"Basque"=>"Ongietorri!",
"Spanish"=>"Bienvenito!"
"Welsh"=>"Croeso!");

Now, if your website uses a cookie, you have this for example:

$_COOKIE['language'];

To make it easy let's transform it in a code which can easily be used:

$language=$_COOKIE['language'];

If your cookie language is Welsh and you have this piece of code:

echo $welcome[$language];

The result of this will be:

Croeso!

If you need to add a lot of translations for your website and a database is too consuming, using an array can be an ideal solution.

Parturifacient

This is no-where near the answer I was asking for. Besides, rather than having all languages available at each page, you'd better create files like lang.en.php which be included and use $lang['welcome'] which is declared in each file. – Fairspoken

Answer 5

I will suggest you not to really depend of database for translation it could be really a messy task and could be a extreme problem in case of data encoding.

I had face similar issue while ago and written following class to solve my problem

Object: Locale\Locale

<?php

  namespace Locale;

  class Locale{

// Following array stolen from Zend Framework
public $country_to_locale = array(
    'AD' => 'ca_AD',
    'AE' => 'ar_AE',
    'AF' => 'fa_AF',
    'AG' => 'en_AG',
    'AI' => 'en_AI',
    'AL' => 'sq_AL',
    'AM' => 'hy_AM',
    'AN' => 'pap_AN',
    'AO' => 'pt_AO',
    'AQ' => 'und_AQ',
    'AR' => 'es_AR',
    'AS' => 'sm_AS',
    'AT' => 'de_AT',
    'AU' => 'en_AU',
    'AW' => 'nl_AW',
    'AX' => 'sv_AX',
    'AZ' => 'az_Latn_AZ',
    'BA' => 'bs_BA',
    'BB' => 'en_BB',
    'BD' => 'bn_BD',
    'BE' => 'nl_BE',
    'BF' => 'mos_BF',
    'BG' => 'bg_BG',
    'BH' => 'ar_BH',
    'BI' => 'rn_BI',
    'BJ' => 'fr_BJ',
    'BL' => 'fr_BL',
    'BM' => 'en_BM',
    'BN' => 'ms_BN',
    'BO' => 'es_BO',
    'BR' => 'pt_BR',
    'BS' => 'en_BS',
    'BT' => 'dz_BT',
    'BV' => 'und_BV',
    'BW' => 'en_BW',
    'BY' => 'be_BY',
    'BZ' => 'en_BZ',
    'CA' => 'en_CA',
    'CC' => 'ms_CC',
    'CD' => 'sw_CD',
    'CF' => 'fr_CF',
    'CG' => 'fr_CG',
    'CH' => 'de_CH',
    'CI' => 'fr_CI',
    'CK' => 'en_CK',
    'CL' => 'es_CL',
    'CM' => 'fr_CM',
    'CN' => 'zh_Hans_CN',
    'CO' => 'es_CO',
    'CR' => 'es_CR',
    'CU' => 'es_CU',
    'CV' => 'kea_CV',
    'CX' => 'en_CX',
    'CY' => 'el_CY',
    'CZ' => 'cs_CZ',
    'DE' => 'de_DE',
    'DJ' => 'aa_DJ',
    'DK' => 'da_DK',
    'DM' => 'en_DM',
    'DO' => 'es_DO',
    'DZ' => 'ar_DZ',
    'EC' => 'es_EC',
    'EE' => 'et_EE',
    'EG' => 'ar_EG',
    'EH' => 'ar_EH',
    'ER' => 'ti_ER',
    'ES' => 'es_ES',
    'ET' => 'en_ET',
    'FI' => 'fi_FI',
    'FJ' => 'hi_FJ',
    'FK' => 'en_FK',
    'FM' => 'chk_FM',
    'FO' => 'fo_FO',
    'FR' => 'fr_FR',
    'GA' => 'fr_GA',
    'GB' => 'en_GB',
    'GD' => 'en_GD',
    'GE' => 'ka_GE',
    'GF' => 'fr_GF',
    'GG' => 'en_GG',
    'GH' => 'ak_GH',
    'GI' => 'en_GI',
    'GL' => 'iu_GL',
    'GM' => 'en_GM',
    'GN' => 'fr_GN',
    'GP' => 'fr_GP',
    'GQ' => 'fan_GQ',
    'GR' => 'el_GR',
    'GS' => 'und_GS',
    'GT' => 'es_GT',
    'GU' => 'en_GU',
    'GW' => 'pt_GW',
    'GY' => 'en_GY',
    'HK' => 'zh_Hant_HK',
    'HM' => 'und_HM',
    'HN' => 'es_HN',
    'HR' => 'hr_HR',
    'HT' => 'ht_HT',
    'HU' => 'hu_HU',
    'ID' => 'id_ID',
    'IE' => 'en_IE',
    'IL' => 'he_IL',
    'IM' => 'en_IM',
    'IN' => 'hi_IN',
    'IO' => 'und_IO',
    'IQ' => 'ar_IQ',
    'IR' => 'fa_IR',
    'IS' => 'is_IS',
    'IT' => 'it_IT',
    'JE' => 'en_JE',
    'JM' => 'en_JM',
    'JO' => 'ar_JO',
    'JP' => 'ja_JP',
    'KE' => 'en_KE',
    'KG' => 'ky_Cyrl_KG',
    'KH' => 'km_KH',
    'KI' => 'en_KI',
    'KM' => 'ar_KM',
    'KN' => 'en_KN',
    'KP' => 'ko_KP',
    'KR' => 'ko_KR',
    'KW' => 'ar_KW',
    'KY' => 'en_KY',
    'KZ' => 'ru_KZ',
    'LA' => 'lo_LA',
    'LB' => 'ar_LB',
    'LC' => 'en_LC',
    'LI' => 'de_LI',
    'LK' => 'si_LK',
    'LR' => 'en_LR',
    'LS' => 'st_LS',
    'LT' => 'lt_LT',
    'LU' => 'fr_LU',
    'LV' => 'lv_LV',
    'LY' => 'ar_LY',
    'MA' => 'ar_MA',
    'MC' => 'fr_MC',
    'MD' => 'ro_MD',
    'ME' => 'sr_Latn_ME',
    'MF' => 'fr_MF',
    'MG' => 'mg_MG',
    'MH' => 'mh_MH',
    'MK' => 'mk_MK',
    'ML' => 'bm_ML',
    'MM' => 'my_MM',
    'MN' => 'mn_Cyrl_MN',
    'MO' => 'zh_Hant_MO',
    'MP' => 'en_MP',
    'MQ' => 'fr_MQ',
    'MR' => 'ar_MR',
    'MS' => 'en_MS',
    'MT' => 'mt_MT',
    'MU' => 'mfe_MU',
    'MV' => 'dv_MV',
    'MW' => 'ny_MW',
    'MX' => 'es_MX',
    'MY' => 'ms_MY',
    'MZ' => 'pt_MZ',
    'NA' => 'kj_NA',
    'NC' => 'fr_NC',
    'NE' => 'ha_Latn_NE',
    'NF' => 'en_NF',
    'NG' => 'en_NG',
    'NI' => 'es_NI',
    'NL' => 'nl_NL',
    'NO' => 'nb_NO',
    'NP' => 'ne_NP',
    'NR' => 'en_NR',
    'NU' => 'niu_NU',
    'NZ' => 'en_NZ',
    'OM' => 'ar_OM',
    'PA' => 'es_PA',
    'PE' => 'es_PE',
    'PF' => 'fr_PF',
    'PG' => 'tpi_PG',
    'PH' => 'fil_PH',
    'PK' => 'ur_PK',
    'PL' => 'pl_PL',
    'PM' => 'fr_PM',
    'PN' => 'en_PN',
    'PR' => 'es_PR',
    'PS' => 'ar_PS',
    'PT' => 'pt_PT',
    'PW' => 'pau_PW',
    'PY' => 'gn_PY',
    'QA' => 'ar_QA',
    'RE' => 'fr_RE',
    'RO' => 'ro_RO',
    'RS' => 'sr_Cyrl_RS',
    'RU' => 'ru_RU',
    'RW' => 'rw_RW',
    'SA' => 'ar_SA',
    'SB' => 'en_SB',
    'SC' => 'crs_SC',
    'SD' => 'ar_SD',
    'SE' => 'sv_SE',
    'SG' => 'en_SG',
    'SH' => 'en_SH',
    'SI' => 'sl_SI',
    'SJ' => 'nb_SJ',
    'SK' => 'sk_SK',
    'SL' => 'kri_SL',
    'SM' => 'it_SM',
    'SN' => 'fr_SN',
    'SO' => 'sw_SO',
    'SR' => 'srn_SR',
    'ST' => 'pt_ST',
    'SV' => 'es_SV',
    'SY' => 'ar_SY',
    'SZ' => 'en_SZ',
    'TC' => 'en_TC',
    'TD' => 'fr_TD',
    'TF' => 'und_TF',
    'TG' => 'fr_TG',
    'TH' => 'th_TH',
    'TJ' => 'tg_Cyrl_TJ',
    'TK' => 'tkl_TK',
    'TL' => 'pt_TL',
    'TM' => 'tk_TM',
    'TN' => 'ar_TN',
    'TO' => 'to_TO',
    'TR' => 'tr_TR',
    'TT' => 'en_TT',
    'TV' => 'tvl_TV',
    'TW' => 'zh_Hant_TW',
    'TZ' => 'sw_TZ',
    'UA' => 'uk_UA',
    'UG' => 'sw_UG',
    'UM' => 'en_UM',
    'US' => 'en_US',
    'UY' => 'es_UY',
    'UZ' => 'uz_Cyrl_UZ',
    'VA' => 'it_VA',
    'VC' => 'en_VC',
    'VE' => 'es_VE',
    'VG' => 'en_VG',
    'VI' => 'en_VI',
    'VN' => 'vn_VN',
    'VU' => 'bi_VU',
    'WF' => 'wls_WF',
    'WS' => 'sm_WS',
    'YE' => 'ar_YE',
    'YT' => 'swb_YT',
    'ZA' => 'en_ZA',
    'ZM' => 'en_ZM',
    'ZW' => 'sn_ZW'
);

/**
 * Store the transaltion for specific languages
 *
 * @var array
 */
protected $translation = array();

/**
 * Current locale
 *
 * @var string
 */
protected $locale;

/**
 * Default locale
 *
 * @var string
 */
protected $default_locale;

/**
 *
 * @var string
 */
protected $locale_dir;

/**
 * Construct.
 *
 *
 * @param string $locale_dir            
 */
public function __construct($locale_dir)
{
    $this->locale_dir = $locale_dir;
}

/**
 * Set the user define localte
 *
 * @param string $locale            
 */
public function setLocale($locale = null)
{
    $this->locale = $locale;

    return $this;
}

/**
 * Get the user define locale
 *
 * @return string
 */
public function getLocale()
{
    return $this->locale;
}

/**
 * Get the Default locale
 *
 * @return string
 */
public function getDefaultLocale()
{
    return $this->default_locale;
}

/**
 * Set the default locale
 *
 * @param string $locale            
 */
public function setDefaultLocale($locale)
{
    $this->default_locale = $locale;

    return $this;
}

/**
 * Determine if transltion exist or translation key exist
 *
 * @param string $locale            
 * @param string $key            
 * @return boolean
 */
public function hasTranslation($locale, $key = null)
{
    if (null == $key && isset($this->translation[$locale])) {
        return true;
    } elseif (isset($this->translation[$locale][$key])) {
        return true;
    }

    return false;
}

/**
 * Get the transltion for required locale or transtion for key
 *
 * @param string $locale            
 * @param string $key            
 * @return array
 */
public function getTranslation($locale, $key = null)
{
    if (null == $key && $this->hasTranslation($locale)) {
        return $this->translation[$locale];
    } elseif ($this->hasTranslation($locale, $key)) {
        return $this->translation[$locale][$key];
    }

    return array();
}

/**
 * Set the transtion for required locale
 *
 * @param string $locale
 *            Language code
 * @param string $trans
 *            translations array
 */
public function setTranslation($locale, $trans = array())
{
    $this->translation[$locale] = $trans;
}

/**
 * Remove transltions for required locale
 *
 * @param string $locale            
 */
public function removeTranslation($locale = null)
{
    if (null === $locale) {
        unset($this->translation);
    } else {
        unset($this->translation[$locale]);
    }
}

/**
 * Initialize locale
 *
 * @param string $locale            
 */
public function init($locale = null, $default_locale = null)
{
    // check if previously set locale exist or not
    $this->init_locale();
    if ($this->locale != null) {
        return;
    }

    if ($locale == null || (! preg_match('#^[a-z]+_[a-zA-Z_]+$#', $locale) && ! preg_match('#^[a-z]+_[a-zA-Z]+_[a-zA-Z_]+$#', $locale))) {
        $this->detectLocale();
    } else {
        $this->locale = $locale;
    }

    $this->init_locale();
}

/**
 * Attempt to autodetect locale
 *
 * @return void
 */
private function detectLocale()
{
    $locale = false;

    // GeoIP
    if (function_exists('geoip_country_code_by_name') && isset($_SERVER['REMOTE_ADDR'])) {

        $country = geoip_country_code_by_name($_SERVER['REMOTE_ADDR']);

        if ($country) {

            $locale = isset($this->country_to_locale[$country]) ? $this->country_to_locale[$country] : false;
        }
    }

    // Try detecting locale from browser headers
    if (! $locale) {

        if (isset($_SERVER['HTTP_ACCEPT_LANGUAGE'])) {

            $languages = explode(',', $_SERVER['HTTP_ACCEPT_LANGUAGE']);

            foreach ($languages as $lang) {

                $lang = str_replace('-', '_', trim($lang));

                if (strpos($lang, '_') === false) {

                    if (isset($this->country_to_locale[strtoupper($lang)])) {

                        $locale = $this->country_to_locale[strtoupper($lang)];
                    }
                } else {

                    $lang = explode('_', $lang);

                    if (count($lang) == 3) {
                        // language_Encoding_COUNTRY
                        $this->locale = strtolower($lang[0]) . ucfirst($lang[1]) . strtoupper($lang[2]);
                    } else {
                        // language_COUNTRY
                        $this->locale = strtolower($lang[0]) . strtoupper($lang[1]);
                    }

                    return;
                }
            }
        }
    }

    // Resort to default locale specified in config file
    if (! $locale) {
        $this->locale = $this->default_locale;
    }
}

/**
 * Check if config for selected locale exists
 *
 * @return void
 */
private function init_locale()
{
    if (! file_exists(sprintf('%s/%s.php', $this->locale_dir, $this->locale))) {
        $this->locale = $this->default_locale;
    }
}

/**
 * Load a Transtion into array
 *
 * @return void
 */
private function loadTranslation($locale = null, $force = false)
{
    if ($locale == null)
        $locale = $this->locale;

    if (! $this->hasTranslation($locale)) {
        $this->setTranslation($locale, include (sprintf('%s/%s.php', $this->locale_dir, $locale)));
    }
}

/**
 * Translate a key
 *
 * @param
 *            string Key to be translated
 * @param
 *            string optional arguments
 * @return string
 */
public function translate($key)
{
    $this->init();
    $this->loadTranslation($this->locale);

    if (! $this->hasTranslation($this->locale, $key)) {

        if ($this->locale !== $this->default_locale) {

            $this->loadTranslation($this->default_locale);

            if ($this->hasTranslation($this->default_locale, $key)) {

                $translation = $this->getTranslation($this->default_locale, $key);
            } else {
                // return key as it is or log error here
                return $key;
            }
        } else {
            return $key;
        }
    } else {
        $translation = $this->getTranslation($this->locale, $key);
    }
    // Replace arguments
    if (false !== strpos($translation, '{a:')) {
        $replace = array();
        $args = func_get_args();
        for ($i = 1, $max = count($args); $i < $max; $i ++) {
            $replace['{a:' . $i . '}'] = $args[$i];
        }
        // interpolate replacement values into the messsage then return
        return strtr($translation, $replace);
    }

    return $translation;
  }
}

Usage

 <?php
    ## /locale/en.php

    return array(
       'name' => 'Hello {a:1}'
       'name_full' => 'Hello {a:1} {a:2}'
   );

$locale = new Locale(__DIR__ . '/locale');
$locale->setLocale('en');// load en.php from locale dir
//want to work with auto detection comment $locale->setLocale('en');

echo $locale->translate('name', 'Foo');
echo $locale->translate('name', 'Foo', 'Bar');

How it works

{a:1} is replaced by 1st argument passed to method Locale::translate('key_name','arg1') {a:2} is replaced by 2nd argument passed to method Locale::translate('key_name','arg1','arg2')

How detection works

• By default if geoip is installed then it will return country code by geoip_country_code_by_name and if geoip is not installed the fallback to HTTP_ACCEPT_LANGUAGE header

Conchita

In what way would a database be messy? Because of the possible characters in different languages? So far I'm mainly having English, French, Dutch, German websites so that is no problem for now. Thanks for the answer, but since it's just a part of the answer it won't win the bounty. – Fairspoken

well i guess your question is only helpful to you only there would be some guys who would consider using languages like Hindi, Thai, Chinese and Arabic (these language will take more the 1 byte to represent a characters) against your required languages. if you are using db then utf8_general_ci collation is appropriate way of doing it. – Conchita

I agree, I got a little bit of track myself there. Thanks for pointing out, also multi bit characters are important enough to be mentioned in this question :) – Fairspoken

Answer 6

Just a sub answer: Absolutely use translated urls with a language identifier in front of them: http://www.domain.com/nl/over-ons
Hybride solutions tend to get complicated, so I would just stick with it. Why? Cause the url is essential for SEO.

About the db translation: Is the number of languages more or less fixed? Or rather unpredictable and dynamic? If it is fixed, I would just add new columns, otherwise go with multiple tables.

But generally, why not use Drupal? I know everybody wants to build their own CMS cause it's faster, leaner, etc. etc. But that is just really a bad idea!

Feme

Thanks for your answer. The reason I don't want to use Drupal/Joomla is simple: I want to make sure that I know all ins and outs of my system, pausible flaws, how the code is build (and important: not build by 300 programmers together). I've got more than enough reasons to not choose open source. Besides that, I want my company to be an important factor for my clients, it's a bad thing that they can go to any other developer and leave me behind with nothing. – Fairspoken

I think all these reasons are disputed in tons of articles. Your customers will hopefully not choose you exactly cause you have a proprietary CMS that nobody else can maintain. But anyway, that is a totally different discussion. – Feme

I understand your point, still I prefer a system that I know all ins and outs for and I'm not feeling anything for relying on anyone elses work when I use a plugin. – Fairspoken

Besides, I tend to document my work well enough, since I'm a "one man army" people that work for me should have no hard time getting to know the system. – Fairspoken

Bad idea is to pick Drupal and even google says that they don't care if url is translated or not. It must contain a locale identifier though. – Mutilate

Answer 7

I am not going to attempt to refine the answers already given. Instead I will tell you about the way my own OOP PHP framework handles translations.

Internally, my framework use codes like en, fr, es, cn and so on. An array holds the languages supported by the website: array('en','fr','es','cn') The language code is passed via $_GET (lang=fr) and if not passed or not valid, it is set to the first language in the array. So at any time during program execution and from the very beginning, the current language is known.

It is useful to understand the kind of content that needs to be translated in a typical application:

1) error messages from classes (or procedural code) 2) non-error messages from classes (or procedural code) 3) page content (usually store in a database) 4) site-wide strings (like website name) 5) script-specific strings

The first type is simple to understand. Basically, we are talking about messages like "could not connect to the database ...". These messages only need to be loaded when an error occurs. My manager class receives a call from the other classes and using the information passed as parameters simply goes to relevant the class folder and retrieves the error file.

The second type of error message is more like the messages you get when the validation of a form went wrong. ("You cannot leave ... blank" or "please choose a password with more than 5 characters"). The strings need to be loaded before the class runs.I know what is

For the actual page content, I use one table per language, each table prefixed by the code for the language. So en_content is the table with English language content, es_content is for spain, cn_content for China and fr_content is the French stuff.

The fourth kind of string is relevant throughout your website. This is loaded via a configuration file named using the code for the language, that is en_lang.php, es_lang.php and so on. In the global language file you will need to load the translated languages such as array('English','Chinese', 'Spanish','French') in the English global file and array('Anglais','Chinois', 'Espagnol', 'Francais') in the French file. So when you populate a dropdown for language selection, it is in the correct language ;)

Finally you have the script-specific strings. So if you write a cooking application, it might be "Your oven was not hot enough".

In my application cycle, the global language file is loaded first. In there you will find not just global strings (like "Jack's Website") but also settings for some of the classes. Basically anything that is language or culture-dependent. Some of the strings in there include masks for dates (MMDDYYYY or DDMMYYYY), or ISO Language Codes. In the main language file, I include strings for individual classes becaue there are so few of them.

The second and last language file that is read from disk is the script language file. lang_en_home_welcome.php is the language file for the home/welcome script. A script is defined by a mode (home) and an action (welcome). Each script has its own folder with config and lang files.

The script pulls the content from the database naming the content table as explained above.

If something goes wrong, the manager knows where to get the language-dependent error file. That file is only loaded in case of an error.

So the conclusion is obvious. Think about the translation issues before you start developing an application or framework. You also need a development workflow that incorporates translations. With my framework, I develop the whole site in English and then translate all the relevant files.

Just a quick final word on the way the translation strings are implemented. My framework has a single global, the $manager, which runs services available to any other service. So for example the form service gets hold of the html service and uses it to write the html. One of the services on my system is the translator service. $translator->set($service,$code,$string) sets a string for the current language. The language file is a list of such statements. $translator->get($service,$code) retrieves a translation string. The $code can be numeric like 1 or a string like 'no_connection'. There can be no clash between services because each has its own namespace in the translator's data area.

I post this here in the hope it will save somebody the task of reinventing the wheel like I had to do a few long years ago.

Fallfish

Answer 8

I had the same probem a while ago, before starting using Symfony framework.

1. Just use a function __() which has arameters pageId (or objectId, objectTable described in #2), target language and an optional parameter of fallback (default) language. The default language could be set in some global config in order to have an easier way to change it later.

2. For storing the content in database i used following structure: (pageId, language, content, variables).

   • pageId would be a FK to your page you want to translate. if you have other objects, like news, galleries or whatever, just split it into 2 fields objectId, objectTable.

   • language - obviously it would store the ISO language string EN_en, LT_lt, EN_us etc.

   • content - the text you want to translate together with the wildcards for variable replacing. Example "Hello mr. %%name%%. Your account balance is %%balance%%."

   • variables - the json encoded variables. PHP provides functions to quickly parse these. Example "name: Laurynas, balance: 15.23".

   • you mentioned also slug field. you could freely add it to this table just to have a quick way to search for it.

3. Your database calls must be reduced to minimum with caching the translations. It must be stored in PHP array, because it is the fastest structure in PHP language. How you will make this caching is up to you. From my experience you should have a folder for each language supported and an array for each pageId. The cache should be rebuilt after you update the translation. ONLY the changed array should be regenerated.

4. i think i answered that in #2

5. your idea is perfectly logical. this one is pretty simple and i think will not make you any problems.

URLs should be translated using the stored slugs in the translation table.

Final words

it is always good to research the best practices, but do not reinvent the wheel. just take and use the components from well known frameworks and use them.

take a look at Symfony translation component. It could be a good code base for you.

Laflam

Thanks for the comment, aswel a +1 for your time taken. Laravel (in my case) is using some Symfony parts if I'm not mistaken, so you're absolutely right about not reinventing the wheel. I started this question (and bounty) to get some insights in the way others do the translations, I'm starting to believe there are alot best practices out there :-) – Fairspoken

Answer 9

I've been asking myself related questions over and over again, then got lost in formal languages... but just to help you out a little I'd like to share some findings:

I recommend to give a look at advanced CMS

Typo3 for PHP (I know there is a lot of stuff but thats the one I think is most mature)

Plone in Python

If you find out that the web in 2013 should work different then, start from scratch. That would mean to put together a team of highly skilled/experienced people to build a new CMS. May be you'd like to give a look at polymer for that purpose.

If it comes to coding and multilingual websites / native language support, I think every programmer should have a clue about unicode. If you don't know unicode you'll most certainly mess up your data. Do not go with the thousands of ISO codes. They'll only save you some memory. But you can do literally everything with UTF-8 even store chinese chars. But for that you'd need to store either 2 or 4 byte chars that makes it basically a utf-16 or utf-32.

If it's about URL encoding, again there you shouldn't mix encodings and be aware that at least for the domainname there are rules defined by different lobbies that provide applications like a browser. e.g. a Domain could be very similar like:

ьankofamerica.com or bankofamerica.com samesamebutdifferent ;)

Of course you need the filesystem to work with all encodings. Another plus for unicode using utf-8 filesystem.

If its about translations, think about the structure of documents. e.g. a book or an article. You have the docbook specifications to understand about those structures. But in HTML its just about content blocks. So you'd like to have a translation on that level, also on webpage level or domain level. So if a block doesn't exist its just not there, if a webpage doesn't exist you'll get redirected to the upper navigation level. If a domain should be completely different in navigation structure, then.. its a complete different structure to manage. This can already be done with Typo3.

If its about frameworks, the most mature ones I know, to do the general stuff like MVC(buzzword I really hate it! Like "performance" If you want to sell something, use the word performance and featurerich and you sell... what the hell) is Zend. It has proven to be a good thing to bring standards to php chaos coders. But, typo3 also has a Framework besides the CMS. Recently it has been redeveloped and is called flow3 now. The frameworks of course cover database abstraction, templating and concepts for caching, but have individual strengths.

If its about caching... that can be awefully complicated / multilayered. In PHP you'll think about accellerator, opcode, but also html, httpd, mysql, xml, css, js ... any kinds of caches. Of course some parts should be cached and dynamic parts like blog answers shouldn't. Some should be requested over AJAX with generated urls. JSON, hashbangs etc.

Then, you'd like to have any little component on your website to be accessed or managed only by certain users, so conceptually that plays a big role.

Also you'd like to make statistics, maybe have distributed system / a facebook of facebooks etc. any software to be built on top of your over the top cms ... so you need different type of databases inmemory, bigdata, xml, whatsoever.

well, I think thats enough for now. If you haven't heard of either typo3 / plone or mentioned frameworks, you have enough to study. On that path you'll find a lot of solutions for questions you haven't asked yet.

If then you think, lets make a new CMS because its 2013 and php is about to die anyway, then you r welcome to join any other group of developers hopefully not getting lost.

Good luck!

And btw. how about people will not having any websites anymore in the future? and we'll all be on google+? I hope developers become a little more creative and do something usefull(to not be assimilated by the borgle)

//// Edit /// Just a little thought for your existing application:

If you have a php mysql CMS and you wanted to embed multilang support. you could either use your table with an aditional column for any language or insert the translation with an object id and a language id in the same table or create an identical table for any language and insert objects there, then make a select union if you want to have them all displayed. For the database use utf8 general ci and of course in the front/backend use utf8 text/encoding. I have used url path segments for urls in the way you already explaned like

domain.org/en/about you can map the lang ID to your content table. anyway you need to have a map of parameters for your urls so you'd like to define a parameter to be mapped from a pathsegment in your URL that would be e.g.

domain.org/en/about/employees/IT/administrators/

lookup configuration

pageid| url

1 | /about/employees/../..

1 | /../about/employees../../

map parameters to url pathsegment ""

$parameterlist[lang] = array(0=>"nl",1=>"en"); // default nl if 0
$parameterlist[branch] = array(1=>"IT",2=>"DESIGN"); // default nl if 0
$parameterlist[employertype] = array(1=>"admin",1=>"engineer"); //could be a sql result 

$websiteconfig[]=$userwhatever;
$websiteconfig[]=$parameterlist;
$someparameterlist[] = array("branch"=>$someid);
$someparameterlist[] = array("employertype"=>$someid);
function getURL($someparameterlist){ 
// todo foreach someparameter lookup pathsegment 
return path;
}

per say, thats been covered already in upper post.

And to not forget, you'd need to "rewrite" the url to your generating php file that would in most cases be index.php

Chiaki

Thanks for the comment, there most certainly are things I should think about. I've been using utf8 encoding for a couple of years now already, I used to struggle with character once ;-) On the other side the type of CMS/Framework was meant to be no factor in your answer as I was looking for a platform independent method as if we were coding from scratch. – Fairspoken

if you really want to code from scratch I recommend to give a look at Dartlang and polymer. Since dartlang is working in the browser and has 32 and 64 bit support and can be used for most purposes on the serverside and has a dart2js compiler its really worth to be studied. If people talk about platform independence they think about java... we know what that means. Buildprocess... I think I'd use JSON for interchange. generated website clientside with hashbangs and serverside.. well do whatever you want to ensure collaboration. – Chiaki

The databaselayout and generation logic is the main task. Nobodys going to do that here for you ... but the Idea itself is what counts. Since I don't care about lobbies but to get things done, I hope you can create models and share some stuff. I'm working on similar tasks right now. But I'm still in planing. I am considering Typo3 as a backend and create a new client structure. The Multilingual pattern is solved in the backend and will share information in a dedicated way for search engines / webservices. Anyway its all contextsensitive and a continuous building task – Chiaki

Answer 10

The real challenge when making a multilingual website is the content. How are you going to store different versions of the same article? Are you using a relational database or a non-relational one?

Using a relational DB such as MySQL, you can take advantage of the JSON data type to store all different versions of the same field altogether.

When using a non-relational DB you can simply store different versions in the same object identifiable by their keys.

If you are using Laravel you may find Laravel Translatable package to be useful when working with traditional relational databases.

Guidon

Answer 11

If you're hosting static content, then Google's Firebase Hosting supports i18n hosting rules that return either Country, Language or Language + Country specific content including index.html, 404.html or manifest.json files (for progressive web apps).

Their hosting will select the Country based on users IP address and select the language based on their browser's Accept-Language header and then apply prioritised rules to return each requested file content.

1. Language code + Country code (for example, content from fr_ca/)

2. Country code only (for example, content from ALL_ca/)

3. Language code only (for example, content from fr/ or es_ALL/)

4. "Default" content that's outside the "i18n content" directory, like at the root of the public directory.

Rules 1 & 3 are applied in order of quality values for each language in the request's Accept-Language header.

Example

public/
    index.html  // Default homepage
    manifest.json  // Default manifest.json
    404.html  // Default custom 404 page

    localized-files/
        ALL_ca/
            index.html
        es_ALL/
            index.html
            404.html
            manifest.json   << Spanish
        fr/
            index.html
            404.html
            manifest.json   << French
        fr_ca/
            index.html
            manifest.json

// firebase.json

"hosting": {

  "public": "public",

  "ignore": [
    "firebase.json",
    "**/.*",
    "**/node_modules/**"
  ],

  "i18n": {
    "root": "/localized-files"  // <<< "i18n content" folder
  }

  ...
}

Full details ...

• Configure internationalization (i18n) rewrites

Apothecium

Answer 12

Database work:

Create Language Table ‘languages’:

Fields:

language_id(primary and auto increamented)

language_name

created_at

created_by

updated_at

updated_by

Create a table in database ‘content’:

Fields:

content_id(primary and auto incremented)

main_content

header_content

footer_content

leftsidebar_content

rightsidebar_content

language_id(foreign key: referenced to languages table)

created_at

created_by

updated_at

updated_by

Front End Work:

When user selects any language from dropdown or any area then save selected language id in session like,

$_SESSION['language']=1;

Now fetch data from database table ‘content’ based on language id stored in session.

Detail may found here http://skillrow.com/multilingual-website-in-php-2/

Mena

This is a way to simple language integration then needed, did you even try to read the full posts and given answers? – Fairspoken

Answer 13

As a person who live in Quebec where almost all site is french and english... i have try many if not most multilanguage plugin for WP... the one an only usefull solution that work nive with all my site is mQtranslate... i live and die with it !

https://wordpress.org/plugins/mqtranslate/

Einsteinium

yeah well, WP wasn't any factor of the question. This could have been a comment aswel – Fairspoken

Answer 14

What about WORDPRESS + MULTI-LANGUAGE SITE BASIS(plugin) ? the site will have structure:

• example.com/eng/category1/....
• example.com/eng/my-page....
• example.com/rus/category1/....
• example.com/rus/my-page....

The plugin provides Interface for Translation all phrases, with simple logic:

(ENG) my_title - "Hello user"
(SPA) my_title - "Holla usuario"

then it can be outputed:
echo translate('my_title', LNG); // LNG is auto-detected

p.s. however, check, if the plugin is still active.

Benkley

and is not "Holla userio" in spanish is "Hola Usuario" – Sheepshearing

Lol Holla userio, that was funny! – Earley

for the reason i didnt know spanish (just used example), hurry hurry to downvote !! :) – Benkley

Answer 15

A really simple option that works with any website where you can upload Javascript is www.multilingualizer.com

It lets you put all text for all languages onto one page and then hides the languages the user doesn't need to see. Works well.

Buttery

Beware, SEO would be very bad! Plus you load all the content while you just need a part of it which is really bad practice. – Volin

weird stuff that the site is only in english... why they dont use their solution?? – Chichihaerh