FreeOrion

Forums for the FreeOrion project
It is currently Wed Dec 13, 2017 3:09 pm

All times are UTC




Post new topic Reply to topic  [ 252 posts ]  Go to page Previous  1 ... 13, 14, 15, 16, 17  Next
Author Message
 Post subject: Re: fr_stringtable
PostPosted: Fri Apr 17, 2015 9:59 pm 
Offline
AI Lead, Programmer
User avatar

Joined: Sat Sep 22, 2012 6:25 pm
Posts: 4390
Geoff the Medio wrote:
Dilvish wrote:
...please test these additional tweaks out to see if they solve it for you.
They do not.
Could you try changing the comment definition to
Code:
    const sregex COMMENT = keep('#' >> keep(*(~_n)) >> _n);
and see if that does it (based on a suggestion from here)

_________________
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0


Top
 Profile  
 
 Post subject: Re: fr_stringtable
PostPosted: Fri Apr 17, 2015 10:09 pm 
Offline
Programming, Design, Admin
User avatar

Joined: Wed Oct 08, 2003 1:33 am
Posts: 12041
Location: Munich
Dilvish wrote:
Could you try changing the comment definition...
With that, starting up with fr.txt selected causes the game window to pop up, but it is all white and not responding, and using a lot of cpu. The log file is empty. After a minute or so I killed it.

My own attempt:
Code:
*(space | +COMMENT) >>

Has a similar result.

In case it was missed at the end of the page...
Geoff the Medio wrote:
After some additional debug logging and selective string uncommenting, it seems to be failing on the long string of comment lines in the Name Lists section.


Edit: Adding a non-commented entry near the end seems to fix it.
Code:
MIDWAY_IGNORED
Ignored
inserted on line 12662 lets it parse. Just adding spaces doesn't help (eg. breaking up the big list of star names, if all those lists remain commented out).


Top
 Profile  
 
 Post subject: Re: fr_stringtable
PostPosted: Sat Apr 18, 2015 7:37 am 
Offline
AI Lead, Programmer
User avatar

Joined: Sat Sep 22, 2012 6:25 pm
Posts: 4390
Geoff the Medio wrote:
Edit: Adding a non-commented entry near the end seems to fix it.
Code:
MIDWAY_IGNORED
Well, that's certainly better than no fix at all. Ouaz, you could just manually add that, deleting a corresponding number of commented-out names from the starname list, which won't really matter.

That will probably muck with Cj trying to run his scripts on the file again without a fair bit of monkeying on his part, but he doesn't really need to so I guess that's ok.

I do have one more idea for a possible coding fix, though Geoff, if you could try this when you have a few spare minutes. Please try making the second line of the ENTRY definition be
Code:
keep(*(space | COMMENT)) >>
and if that causes the weird hang again then try deleting the keep() from the COMMENT definition (but leaving this new keep in ENTRY).

_________________
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0


Top
 Profile  
 
 Post subject: Re: fr_stringtable
PostPosted: Sat Apr 18, 2015 8:03 am 
Offline
Programming, Design, Admin
User avatar

Joined: Wed Oct 08, 2003 1:33 am
Posts: 12041
Location: Munich
Dilvish wrote:
Please try making the second line of the ENTRY definition be
Code:
keep(*(space | COMMENT)) >>
and if that causes the weird hang again...
It doesn't hang, but it still produces the stack space exhausted error.
Quote:
...then try deleting the keep() from the COMMENT definition (but leaving this new keep in ENTRY).
Same result.

Edit:
Code:
    const sregex COMMENT = '#' >> *(~_n) >> _n;
[...]
    const sregex ENTRY =
        keep(*(space | +COMMENT)) >>
        KEY >> *blank >> (_n | COMMENT) >>
        (("'''" >> MULTI_LINE_VALUE >> "'''" >> *space >> _n) | SINGLE_LINE_VALUE >> _n);

Seems to work without errors.


Top
 Profile  
 
 Post subject: Re: fr_stringtable
PostPosted: Sat Apr 18, 2015 3:58 pm 
Offline
Dyson Forest
User avatar

Joined: Wed Aug 13, 2014 7:21 pm
Posts: 213
Location: France
Dilvish wrote:
Geoff the Medio wrote:
Edit: Adding a non-commented entry near the end seems to fix it.
Code:
MIDWAY_IGNORED
Well, that's certainly better than no fix at all. Ouaz, you could just manually add that, deleting a corresponding number of commented-out names from the starname list, which won't really matter.


I did this (line 12652, deleting two lines before). It fixes the problem, but new errors appears in the log file:

Code:
2015-04-18 16:55:21.049588 [debug] Client : Logger initialized
2015-04-18 16:55:21.049588 [debug] Client : v0.4.4+  [build 2015-04-13.d489d44] MSVC 2010
2015-04-18 16:55:21.080788 [error] Client : Unresolved key expansion: AI_CAPITOL_NAMES_BEGINNER in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.080788 [error] Client : Unresolved key expansion: AI_CAPITOL_NAMES_TURTLE in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.080788 [error] Client : Unresolved key expansion: AI_CAPITOL_NAMES_CAUTIOUS in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.080788 [error] Client : Unresolved key expansion: AI_CAPITOL_NAMES_TYPICAL in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.080788 [error] Client : Unresolved key expansion: AI_CAPITOL_NAMES_AGGRESSIVE in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.080788 [error] Client : Unresolved key expansion: AI_CAPITOL_NAMES_MANIACAL in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.080788 [error] Client : Unresolved reference: INFRASTRUCTURE_TITLE in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.080788 [error] Client : Unresolved reference: INFRASTRUCTURE_TITLE in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.096388 [error] Client : Unresolved reference: INFRASTRUCTURE_TITLE in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.096388 [error] Client : Unresolved reference: INFRASTRUCTURE_TITLE in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.423988 [debug] Client : OpenAL initialized. Version 1.1Renderer SoftwareVendor Creative Labs Inc.
Extensions: EAX EAX2.0 EAX3.0 EAX4.0 EAX5.0 EAX3.0EMULATED EAX4.0EMULATED AL_EXT_OFFSET AL_EXT_LINEAR_DISTANCE AL_EXT_EXPONENT_DISTANCE


If i remove the comment before [INFRASTRUCTURE_TITLE] (line 3684), and before each [AI_CAPITOL_NAMES] (line 10675, 681, 685, 689, 694, 698), the log is fine again.

But like you said, the Name lists section is not needed, i.e. I could delete it from the FR file. The number of lines before it would be preserved and still be matched with the english file.

So I deleted all the Name Lists section and launched FO:

- The links in the Pedia are broken again (key instead of value)
- The greek letters (system and planet names) are not displayed when I launch a game.
- Only one new error in the log (INFRASTRUCTURE_TITLE and AI_CAPITOL_NAMES are gone):

Code:
2015-04-18 17:40:56.749167 [debug] Client : Logger initialized
2015-04-18 17:40:56.749167 [debug] Client : v0.4.4+  [build 2015-04-13.d489d44] MSVC 2010
2015-04-18 17:40:56.764767 [error] Client : StringTable file "C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt" is malformed around line 10802
2015-04-18 17:40:57.045568 [debug] Client : OpenAL initialized. Version 1.1Renderer SoftwareVendor Creative Labs Inc.
Extensions: EAX EAX2.0 EAX3.0 EAX4.0 EAX5.0 EAX3.0EMULATED EAX4.0EMULATED AL_EXT_OFFSET AL_EXT_LINEAR_DISTANCE AL_EXT_EXPONENT_DISTANCE


Line 10802 being the last line of the fr file.

Attachment:
broken_display.PNG
broken_display.PNG [ 253.5 KiB | Viewed 603 times ]


With my working FR file, it is Thorne α , Thorne β , Nitzer β, and the Pedia links is correctly displayed.

By the way, if I use the "cleaned" fr file that is in Git main repository (https://github.com/freeorion/freeorion/ ... les/fr.txt), the same problem happens (except the Pedia links which are fine, as there are no commented lines).

On the both files, if I add only the STAR_GROUP_NAMES and STAR_GROUP_CHARS at the end of the file, the greek letters are displayed again, but I still have the same log error than above ("malformed around line *), and still the broken pedia links (key instead of value).

What a mess, I'm sorry to bother you with this. :?

EDIT:

Geoff the Medio wrote:
Edit:
Code:
    const sregex COMMENT = '#' >> *(~_n) >> _n;
[...]
    const sregex ENTRY =
        keep(*(space | +COMMENT)) >>
        KEY >> *blank >> (_n | COMMENT) >>
        (("'''" >> MULTI_LINE_VALUE >> "'''" >> *space >> _n) | SINGLE_LINE_VALUE >> _n);

Seems to work without errors.


I will wait the next build then.

_________________
I release every updated file under the CC-BY-SA 3.0 license.


Top
 Profile  
 
 Post subject: Re: fr_stringtable
PostPosted: Sat Apr 18, 2015 5:25 pm 
Offline
AI Lead, Programmer
User avatar

Joined: Sat Sep 22, 2012 6:25 pm
Posts: 4390
Ouaz wrote:
I did this (line 12652, deleting two lines before). It fixes the problem, but new errors appears in the log file:
Code:
2015-04-18 16:55:21.049588 [debug] Client : Logger initialized
2015-04-18 16:55:21.049588 [debug] Client : v0.4.4+  [build 2015-04-13.d489d44] MSVC 2010
2015-04-18 16:55:21.080788 [error] Client : Unresolved key expansion: AI_CAPITOL_NAMES_BEGINNER in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt


If i remove the comment before [INFRASTRUCTURE_TITLE] (line 3684), and before each [AI_CAPITOL_NAMES] (line 10675, 681, 685, 689, 694, 698), the log is fine again.
Hmm, that sounds familiar, that lookups had to be in the same stringtable. I think we had discussed this issue before; I think there is a potential solution via loading the default stringtable first as a backup for these lookups, and then providing that backup list into the routine that reads the stringtable. Now that we have started trying to using more macro subsitutions wherever possible it seems more important than ever. I'll try to follow up on that this weekend. Thanks much for your patience with this-- this is a good issue to get sorted out, with significance beyond the FR stringtable.

Quote:
The greek letters (system and planet names) are not displayed when I launch a game.
Ah, yes, we had seen that quite some time back with the DE stringtable, and then forgot about that issue recently. Until we think of a solution the greek letter list will need to be an exception to the comment-out-if-not-translated rule.
Quote:
...What a mess, I'm sorry to bother you with this. :?
No, no, we need to get these problems sorted out, even if you didn't care about how the FR stringtable was structured.

_________________
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0


Top
 Profile  
 
 Post subject: Re: fr_stringtable
PostPosted: Sat Apr 18, 2015 6:10 pm 
Offline
AI Contributor
User avatar

Joined: Tue Jun 24, 2014 9:55 pm
Posts: 444
Dilvish wrote:
Ouaz wrote:
Hmm, that sounds familiar, that lookups had to be in the same stringtable. I think we had discussed this issue before; I think there is a potential solution via loading the default stringtable first as a backup for these lookups, and then providing that backup list into the routine that reads the stringtable.
My bad. This is unexpected behavior for me. I like you potential solution. Should I wait for it or make pull request with add of this keys?

_________________
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0


Top
 Profile  
 
 Post subject: Re: fr_stringtable
PostPosted: Sat Apr 18, 2015 6:41 pm 
Offline
AI Lead, Programmer
User avatar

Joined: Sat Sep 22, 2012 6:25 pm
Posts: 4390
Cjkjvfnby wrote:
Should I wait for it or make pull request with add of this keys?
I said I would try to get to it this weekend (i.e., today or tomorrow, my time), but no promises on that. I expect very very few people are updating their stringtables over the weekend, but if you want to submit the pull request I won't argue with it, though I would hope to revert it within a day.

_________________
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0


Top
 Profile  
 
 Post subject: Re: fr_stringtable
PostPosted: Sat Apr 18, 2015 11:16 pm 
Offline
AI Lead, Programmer
User avatar

Joined: Sat Sep 22, 2012 6:25 pm
Posts: 4390
ok, I have put in a github PR for a solution. I explain a basic test in the comments there. Any suggestions on ways to improve it before I commit to the main repo?

_________________
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0


Top
 Profile  
 
 Post subject: Re: fr_stringtable
PostPosted: Tue Apr 21, 2015 3:45 pm 
Offline
Dyson Forest
User avatar

Joined: Wed Aug 13, 2014 7:21 pm
Posts: 213
Location: France
Hi,

I tested the latest build:

- With the current FR stringtables in the main repo, everything works fine (values instead of keys in the Pedia) but the greek letters don't display (known issue). The AI_CAPITOL_NAMES and INFRASTRUCTURE_TITLE are checked with your "fallback" patch:

Code:
2015-04-21 16:34:20.450366 [debug] Client : Key expansion: AI_CAPITOL_NAMES_MANIACAL not found in primary stringtable: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt; checking in fallback fileC:\Program Files (x86)\FreeOrion\default\stringtables\en.txt
2015-04-21 16:34:20.465966 [debug] Client : Key reference: INFRASTRUCTURE_TITLE not found in primary stringtable: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt; checking in fallback fileC:\Program Files (x86)\FreeOrion\default\stringtables\en.txt


- With my work-in-progress "commented" FR stringtables, the greek letters are displayed, but the Pedia links are broken:

Code:
2015-04-21 16:29:28.992109 [debug] Client : Logger initialized
2015-04-21 16:29:28.992109 [debug] Client : v0.4.4+  [build 2015-04-20.e356dec] MSVC 2010
2015-04-21 16:29:29.070109 [error] Client : Exception caught regex parsing Stringtable: Regex stack space exhausted
2015-04-21 16:29:29.070109 [error] Client : Last and prior keys matched: , HOTKEY_FOCUS_NEXT_WND
2015-04-21 16:29:29.382109 [debug] Client : OpenAL initialized. Version 1.1Renderer SoftwareVendor Creative Labs Inc.
Extensions: EAX EAX2.0 EAX3.0 EAX4.0 EAX5.0 EAX3.0EMULATED EAX4.0EMULATED AL_EXT_OFFSET AL_EXT_LINEAR_DISTANCE AL_EXT_EXPONENT_DISTANCE


I fixed it by deleting all STAR_NAMES and all the stuff after. The FR file ended now with:

Code:
10816 #ω'''
10817
10818 #STAR_GROUP_NAMES
10819 #Centauri
10820


I kept "Centauri" as last line because it's an unique string, so it will be easy to check if the line number are matched with the english one (currently line 10819 on both files).

The log still mentions the fallback though:

Code:
2015-04-21 16:45:03.492804 [debug] Client : Logger initialized
2015-04-21 16:45:03.492804 [debug] Client : v0.4.4+  [build 2015-04-20.e356dec] MSVC 2010
[...]
2015-04-21 16:45:03.570804 [debug] Client : Key expansion: AI_CAPITOL_NAMES_MANIACAL not found in primary stringtable: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt; checking in fallback fileC:\Program Files (x86)\FreeOrion\default\stringtables\en.txt
2015-04-21 16:45:03.586405 [debug] Client : Key reference: INFRASTRUCTURE_TITLE not found in primary stringtable: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt; checking in fallback fileC:\Program Files (x86)\FreeOrion\default\stringtables\en.txt
[...]
2015-04-21 16:45:03.836005 [debug] Client : OpenAL initialized. Version 1.1Renderer SoftwareVendor Creative Labs Inc.
Extensions: EAX EAX2.0 EAX3.0 EAX4.0 EAX5.0 EAX3.0EMULATED EAX4.0EMULATED AL_EXT_OFFSET AL_EXT_LINEAR_DISTANCE AL_EXT_EXPONENT_DISTANCE
.

Otherwise, evererything is OK.

I pull a request to update the FR stringtable in the main repo: https://github.com/freeorion/freeorion/pull/37

Thanks.

_________________
I release every updated file under the CC-BY-SA 3.0 license.


Top
 Profile  
 
 Post subject: Re: fr_stringtable
PostPosted: Tue Apr 21, 2015 4:19 pm 
Offline
AI Contributor
User avatar

Joined: Tue Jun 24, 2014 9:55 pm
Posts: 444
Ouaz wrote:
Hi,
I pull a request to update the FR stringtable in the main repo: https://github.com/freeorion/freeorion/pull/37
Thanks.

In that case it will be better to have two commits:
1) bulk update with commented lines (update by script)
2) translation work.
It will be more easy to track changes.

_________________
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0


Top
 Profile  
 
 Post subject: Re: fr_stringtable
PostPosted: Tue Apr 21, 2015 4:38 pm 
Offline
AI Lead, Programmer
User avatar

Joined: Sat Sep 22, 2012 6:25 pm
Posts: 4390
Cjkjvfnby wrote:
In that case it will be better to have two commits:
1) bulk update with commented lines (update by script)
2) translation work.
It will be more easy to track changes.
Ah, yes, you are probably right about that, but I have already accepted the PR. We could revert it & split it up, but I don't think it's really worth doing that, is it?


Ouaz wrote:
The log still mentions the fallback though:
It still has to get those keys from the fallback en.txt, doesn't it?

_________________
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0


Top
 Profile  
 
 Post subject: Re: fr_stringtable
PostPosted: Tue Apr 21, 2015 5:01 pm 
Offline
Dyson Forest
User avatar

Joined: Wed Aug 13, 2014 7:21 pm
Posts: 213
Location: France
EDIT: Oops, didn't see there was new posts.

Cjkjvfnby wrote:
Ouaz wrote:
Hi,
I pull a request to update the FR stringtable in the main repo: https://github.com/freeorion/freeorion/pull/37
Thanks.

In that case it will be better to have two commits:
1) bulk update with commented lines (update by script)
2) translation work.
It will be more easy to track changes.


You mean separate PRs each time? One for the "structure" modification (added, removed or modified lines in en.txt, merged in fr.txt) and one for updated french translation?

It will complicate things again...

Dilvish wrote:
Ouaz wrote:
The log still mentions the fallback though:
It still has to get those keys from the fallback en.txt, doesn't it?


Yes. However, for the INFRASTRUCTURE_TITLE, it could be avoided as it is actually translated. See below.


----------------------------------
Thanks for the merge.

Now... ^^

Dilvish wrote:
Quote:
2) As I said, some french words are the same than in english (e.g. production = production). So, there is a bunch of "translated strings" that are commented: These three french words are the same than in english. So, can I "uncomment" them? (to mark that is actually translated)
Since the translations are the same as in en.txt, there is no real harm in them being commented out, it seems. But to avoid confusion it probably would be better to flag such things (whether or not Cj's script were adjusted to recognize the flag). It would be nice to be able to simply add a "// translated" on the same line as the key for these entries that are properly unchanged when translated. Unfortunately, right now it appears that adding a comment to the end of a stringtable line for a key breaks the stringtable formatting. Geoff, being able to have a comment on the same line after a stringtable key would probably be reasonable to implement, would you have any objections? Since adding a comment line before the key would throw off the line matching you are trying to maintain, I think we should just leave such entries commented out.


What about this?

It seems Geoff and you have worked on this, but I don't understand very well (as always...)

Can you give me an example on how to flag these lines, to "mark" them as translated?

For example, for "Infrastructure" being "Infrastructure" also in french:

Code:
#INFRASTRUCTURE_TITLE
#Infrastructure
INFRASTRUCTURE_TEXT
L'Infrastructure englobe toutes les structures et organisations qui assurent la pérennité et la production des populations de l'empire : l'énergie, les transports, les télécommunications, les services sociaux, etc.


Thanks again.

_________________
I release every updated file under the CC-BY-SA 3.0 license.


Top
 Profile  
 
 Post subject: Re: fr_stringtable
PostPosted: Tue Apr 21, 2015 5:27 pm 
Offline
AI Lead, Programmer
User avatar

Joined: Sat Sep 22, 2012 6:25 pm
Posts: 4390
Ouaz wrote:
Can you give me an example on how to flag these lines, to "mark" them as translated?
The following should now be fine for the parser:

For example, for "Infrastructure" being "Infrastructure" also in french:

Code:
INFRASTRUCTURE_TITLE  # translated
Infrastructure
(you may want to test and confirm that to be sure)

Then if Cj were to plan on running a script to help with processing the file, he could have the script look for the extra "# translated" markers and know to leave those entries even if they are not otherwise noticeably translated. I think there is no rush for him to do this processing again, but it would be helpful if you mark as "# translated" any of the entries you uncomment. Please note that the "# translated" only goes after the entry's key, on the same line.

_________________
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0


Top
 Profile  
 
 Post subject: Re: fr_stringtable
PostPosted: Tue Apr 21, 2015 5:39 pm 
Offline
Dyson Forest
User avatar

Joined: Wed Aug 13, 2014 7:21 pm
Posts: 213
Location: France
Dilvish wrote:
Ouaz wrote:
Can you give me an example on how to flag these lines, to "mark" them as translated?
The following should now be fine for the parser:

For example, for "Infrastructure" being "Infrastructure" also in french:

Code:
INFRASTRUCTURE_TITLE  # translated
Infrastructure
(you may want to test and confirm that to be sure)


It works. Great. 8)

Just to be sure:

[key][space][space][#][space][translated] : that's right? (two spaces needed before #)

Quote:
Then if Cj were to plan on running a script to help with processing the file, he could have the script look for the extra "# translated" markers and know to leave those entries even if they are not otherwise noticeably translated. I think there is no rush for him to do this processing again, but it would be helpful if you mark as "# translated" any of the entries you uncomment. Please note that the "# translated" only goes after the entry's key, on the same line.


No problem. I'm working on it.

Thanks!

_________________
I release every updated file under the CC-BY-SA 3.0 license.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 252 posts ]  Go to page Previous  1 ... 13, 14, 15, 16, 17  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group