Dev Journal #15: So viele Sprachen

[p]The original Elemental: War of Magic was not just “English-only”. It was hard coded into English in such a way to make it nearly impossible to localize it. Under the covers, the game and the underlying engine, Kumquat, uses char * everywhere. This means that all text is byte-oriented, single-code unit that only works nicely with ASCII. Even German is tough because of ä ö ü ß and so on.[/p][p]And our code is full of toupper(myText) which can only handle A-Z. The code was full of sprintf().[/p][p][/p][p][/p][p]In short, localizing this game has been a monumental effort. Let me walk you through it.[/p][p][/p][h2]Step 1: You have to get the strings out of the C++ code as well as the data code.[/h2][p]This involves going through hundreds of thousands of lines of code and finding where we inserted things like “It smells kind of bad” right inside a C++ file or a .XML data file and then putting it in a string table like this:[/p][p][c] Banner of the Brave [/c][/p][p][/p][h2]Step 2: You have to deal with variables[/h2][p]So we used sprintf all over the place in C++. That means the code assumed that variables would be in a certain order and in a certain format, but different languages have different grammars, so you can’t assume “Must have a Intelligent stat value of 3.1” will be in that order. Lots of languages would have that flipped.[/p][p]So you have to develop a system in which you have variable replacement and, again, go through the hundreds of thousands of lines of code and find every place we did that.[/p][p][c] Must have a {STAT_NAME} stat value of {FLOAT_0} [/c][/p][p]And if you’ve played the game, you have probably seen places where we at least got the string into a table but we didn’t put in the variable replacement code.[/p][p][/p][h2]Step 3: You have to switch from char* to Unicode-safe text[/h2][p]Oof. This has been hard. This means every UI element, tooltip, error message, status line, etc. has to be found and redone in a new set of engine APIs don’t rely on char*.[/p][p][/p][h2]Step 4: You have to choose fonts that can actually render these characters[/h2][p]Our original fonts supported just a few hundred types of characters. But it doesn’t have all of the aforementioned German characters or French and don’t get me started on Polish, Russian, Chinese, Japanese, Korean, etc.[/p][p]And there was a reason why we couldn’t have even think of doing this back in 2010. These character atlases consume hundreds of megabytes of memory.[/p][p][/p][h2]Step 5: You have to update your controls to handle very long words[/h2][p]I don’t just mean German. Chinese and other languages don’t have spaces which is a giant challenge for an engine that relies on word-wrapping via a space to solve all its problems.[/p][p]你好欢迎来到这里[/p][p]This has no spaces but is a full sentence “Hello, welcome here.”[/p][p]Not to mention four-character idioms, different metaphors for strength, bravery, luck, misfortune. In a fantasy game, there’s a lot of nuance and thus you can’t just randomly add spacing to fix your word wrapping issues.[/p][p][/p][h2]Step 6: Grammar Rules[/h2][p]Gendered Nouns, Plurals, where articles need to be placed. We probably won’t make it to this step but instead leave that to modders (language packs will be in the Steam Workshop for those who want to improve on what we do).[/p][p]So if you see a character say Hallo, ich bin sehr glücklich für das Essen machen in der groß Haus heute! We’ll do our best to fix that since German is one of the officially supported languages.[/p][p]But on the other hand, if you’re Greek, you can happily write «Γεια, εγώ είμαι πολύ χαρούμενος για το φαγητό να κάνω στο μεγάλο σπίτι σήμερα!» knowing that you are truly showing the world your expertise in Greek…[/p][p][/p][h2]Where we are at now[/h2][p]So we’re pretty close to having a Preview ready to go up. The current plan is to go up next week with Preview 1 of Beta 2 which will have the initial Steam Workshop modding support and our first crude stab at English, French, German, Spanish.[/p][p]We plan to put up an Italian language pack on Steam Workshop as a demonstration on how to make your own language packs to share.[/p][p]As soon as we are confident that we have Unicode working decently then we will start adding Chinese, Russian, Polish, Japanese, Korean.[/p][p]The Dynasty System has been ported but it needs more work. It might be ready by next week, but it’s going to be tight. We’re adding arranged marriages to make it a little more fun and we’re working on having genetically passable traits to characters besides just the usual. We figure if we’re going to have Dynasties we should really open it up.[/p][p]We also have a lot of documentation to do. For example, let’s say you want to make a quest. How do you test it? We need to have ways to make it easy for people to try out their stuff (like having a fun map where you put your whole campaign on to try it out).[/p][p][/p][p][/p][h2]Final Thoughts[/h2][p]We are so grateful to you guys. It has been really fun working on this with you.[/p][p]It still blows my mind that after 15 years, we have the opportunity to revisit all this. It’s pretty crazy to think that the same individual who coded the original Dynasty system in 2008/2009 is now able to bring it to fruition. She’s also the one who is looking at getting tooltips within tooltips in there and will be one of the people making the text customization so us old guys can read the text easier.[/p][p]Similarly, the UI in Reforged is the same artist who made the original UI in War of Magic, Fallen Enchantress and GalCiv III/GalCiv IV. We have purposely kept the UI pretty toned down so that it’s easy to make changes to it. The UI will be moddable too btw. It’s just PNGs and a text file for layout.[/p][p]More to come. Stay tuned![/p]