Time Nick Message 14:20 MTDiscord vector.from_string(minetest.pos_to_string(vector.new(1.1, 1.1, 1.1))) if a de_DE.utf8 locale is set. Consider setting a locale to make MT more platform-independent. 14:21 MTDiscord ^ it will fail because the german locale formats numbers using commas (,), which are used as component separators by pos_to_string 14:21 MTDiscord cripes, are we really doing that 14:22 MTDiscord It'll probably fail in some French locales as well 14:22 MTDiscord There is hardly any way around doing it 14:23 MTDiscord MT could do os.setlocale("all", "C"), but it sounds like it might affect other Lua processes 14:23 MTDiscord The problem is that even formatting like ("%g"):format(num) will respect the locale 14:23 MTDiscord We'd have to write our own number formatting 14:23 MTDiscord that's unfortunet 14:24 celeron55 wait what, don't we set LC_NUMERIC? 14:24 MTDiscord I'd love it if the POSIX / ISO C locale was forced for any Lua running inside MT 14:24 MTDiscord celeron55: only LC_NUMERIC? 14:24 celeron55 also, yes, even if that's done we probably should have our own number formatting 14:25 MTDiscord Yeah, that'd be saner at least. I'm used to APIs where you can determine the locale per-operation, but I suppose lua's lighter than that 14:25 celeron55 but in any case there's no way pos_to_string can be allowed to give different results for different locales 14:25 MTDiscord Indeed: setlocale(LC_NUMERIC, "C"); 14:25 MTDiscord Why not setlocale(LC_ALL, "C");? 14:25 celeron55 well 14:26 celeron55 go and test what happens with translations 14:26 celeron55 i don't know, but it might be bad 14:26 MTDiscord https://github.com/minetest/minetest/blob/master/doc/lua_api.txt#L4714 could be fixed this way tho 14:27 MTDiscord actually, even setlocale(LC_COLLATE, "C"); would suffice to fix string.char and the like 14:28 MTDiscord I'd additionally like setlocale(LC_CTYPE, "C"); for consistent character classes 14:28 celeron55 well, it's pretty easy to test what happens at least on linux 15:01 sfan5 I doubt pos_to_string is actually broken if locale is set to German 15:01 sfan5 minetest sets LC_NUMERIC and if this was broken I'm sure someone would have noticed 15:04 sfan5 speaking of Lua any serialization of floating point numbers would be broken 15:04 sfan5 if it affects more than just Lua you'd see shaders fail to compile due to broken number format 15:06 MTDiscord Assuming shader compilation is handled by OpenGL, I'm pretty sure that process is locale-independent 15:06 sfan5 it sure is 15:07 sfan5 Minetest dynamically puts a few constants into shader sources, if it said 1,2 instead of 1.2 that would fail too 15:08 MTDiscord Ah, yeah that would break if the locale was wrong then. @Luatic can you confirm that what you're reporting is actually current behavior? 15:11 sfan5 wait why is os.setlocale in our whitelist 15:11 rubenwardy is it called by builtin? 15:15 sfan5 if it does then I hope it undefines the method after that 15:44 Krock https://github.com/minetest/minetest/pull/11846#issuecomment-1011643923 15:45 Krock @Hugues Ross this might be helpful to you: https://gist.github.com/SmallJoker/4d411b958295d9e694cd9aa5984563db 15:46 Krock grep -r setlocale yields no results in builtin. unused. 16:08 rubenwardy oh nice 16:08 rubenwardy I use git hub, I can do `git checkout https://github.com/minetest/minetest/pull/11846` 16:10 MTDiscord On my end I have a script that mostly makes this sort of thing automatic, but one of the recent irrlichtmt changes makes it incompatible with prs from before a certain date... 16:10 MTDiscord and in this case, updating the code the pr is touching seems to break its functionality 16:11 MTDiscord It would be good to see if anyone else has that issue though, I can't rule out user error alone 16:12 Krock oh yes indeed. merge conflict in game.cpp and c_content.cpp 16:14 MTDiscord I resolved them locally, but got incorrect results in the following test. Either I messed up the resolution or there's a less visible conflict in the ~40 commits difference git's reporting 16:15 erlehmann Krock why would you download a patch file and apply it if you can add the PR source as a remote and use git internal tooling? 16:16 Krock in my option it's more convenient this way. I can also ensure that the PR will work in current master 16:17 erlehmann i almost always do “git remote add $FOO $WHATEVER; git checkout $FOO/$BRANCHNAME”, then merge 16:17 erlehmann which leads me to the question of why the script does not do it 16:17 erlehmann am i missing something here? 16:18 erlehmann i mean is “ensure that the PR will work in current master” more than rebasing it on current master? 16:18 Krock you are not missing anything. it's another approach to the same end goal 16:18 erlehmann ah ok! 16:18 erlehmann thx 16:18 erlehmann i mean in the end commits are just patches anyway 17:09 MTDiscord sfan5: it isn't broken, I just didn't know if MT was setting LC_NUMERIC 17:10 MTDiscord sfan5: LEAVE IT IN THE WHITELIST 18:30 x2048 Planning to push https://gist.github.com/x2048/ba8f99f94103cbfe56876e7a2b7df3be 18:30 x2048 It's annoying that client/mod 18:30 x2048 *storage sticks in the diffs 18:31 Krock why's this file not in clientmods/ ? 18:31 Krock odd. but somehow also correct.. client/ contains the cache files *shrug* 18:41 x2048 So, ok to push it? 18:42 Krock yes sure 19:46 sfan5 @luatic mods do not have a valid reason to call setlocale 19:57 MTDiscord sfan5: They do: The engine currently only does it for LC_NUMERIC 19:57 MTDiscord If they want fixed character classes, or even a sane collation, they must do it 19:57 sfan5 and where do those matter inside Lua? 19:58 MTDiscord Otherwise it's a hit-or-miss that's just "platform-independent" as long as the platform isn't configured weirdly 19:58 MTDiscord Character classes matter a lot in patterns 19:58 MTDiscord And collations matter even more 19:58 MTDiscord The engine uses [A-za-z] patterns for instance to work around the character class issue (because a letter class might be overly permissive) 19:59 MTDiscord A-Z* 19:59 MTDiscord unfortunately this fails for weird collations where the A-Z range is *not ABCDEFGHIJKLMNOPQRSTUVWXYZ 19:59 sfan5 that's news to me 20:02 sfan5 https://github.com/minetest/minetest/blob/97248c695730c29ca91fb7e56456d061f77744f5/lib/lua/src/lstrlib.c#L259-L260 this does not look locale specific to me 20:02 sfan5 nvm you meant something else 20:02 MTDiscord Yeah 20:03 MTDiscord The problem are the character codes 20:03 MTDiscord I say we expect POSIX, so we should enforce it 20:03 MTDiscord Granted, if os.setlocale is called, it bleeds over into MT code 20:04 MTDiscord And most likely can be used to break it if setting something like a german locale that breaks the LC_NUMERIC thing 20:04 MTDiscord I suggest forcing at least the collation locale to be POSIX 20:05 sfan5 you mean "C" 20:06 MTDiscord Yeah 20:07 MTDiscord "monetary" doesn't matter to us I believe 20:07 MTDiscord ctype should probably be C as well 20:07 MTDiscord (character classes) 20:08 MTDiscord I'm torn on time; we might want to keep locale-specific date formatting, but many modders are presumably relying on POSIX behavior of os.time (UNIX timestamp) 20:09 sfan5 how are those related 20:09 sfan5 surely os time can be told to return a number? 20:10 MTDiscord sfan5: it always returns a number, but the meaning of the number may differ 20:10 MTDiscord according to the reference manual: "The returned value is a number, whose meaning depends on your system. In POSIX, Windows, and some other systems, this number counts the number of seconds since some given start time (the "epoch"). In other systems, the meaning is not specified, and the number returned by time can be used only as an argument to date and difftime. " 20:11 sfan5 on those "other" systems calling setlocale probably activates the built-in self destruction mode 20:11 sfan5 this is neither a real problem nor one we can solve 20:13 sfan5 btw as for patterns I read what you said again: A-Z will always works, if you have a system without ASCII you have a bigger problem 20:13 sfan5 %w is indeed locale-dependent 20:15 sfan5 HOWEVER since Lua operates on C chars and not unicode codepoints isalpha() will never return a valid result for characters outside of the ASCII range 20:15 sfan5 and since Minetest only really supports UTF-8 internally this is a problem which in practice cannot happen to anyone 20:16 MTDiscord UTF-8 is an entire different point 20:16 MTDiscord sfan5: What about Latin-1? 20:16 sfan5 not supported 20:16 MTDiscord BTW regarding character sets and a-z: http://teaching.idallen.com/cst8177/13w/notes/000_character_sets.html 20:17 MTDiscord it indeed seems [A-Za-z] is safe 20:17 MTDiscord but [A-Z] and [a-z] are not 20:18 sfan5 I don't know why you think bash behaviour appalies to Lua 20:19 MTDiscord sfan5: because the collate issue should be the same? 20:20 sfan5 why? Lua patters are made up from scratch and have nothing to do with glob as specified by posix 20:22 MTDiscord huh? I'd assume a character range works the same in both 20:27 sfan5 you can assume that but it's not true 20:30 sfan5 I also tried some of the examples in the article and can't reproduce any of it 20:32 sfan5 anyway back to the original point: mods still don't have a reason to call setlocale, it'd be on the engine to improve the defaults 20:33 MTDiscord arguably true 20:33 MTDiscord but please only remove setlocale after you are sure that the engine settings are decent 20:35 sfan5 one reason that we have it might be that it can also be used to *get* the locale 21:14 rubenwardy Lua is always given utf-8 text 21:17 sfan5 I wonder why os.tmpname is on the whitelist too, not like Lua can write there, can it? 21:31 sfan5 https://github.com/minetest/minetest/pull/11950/commits/971d8ff5df85ac88217003f2fbc7d5762c8ad3cd here's what I decided to change @luatic 21:33 sfan5 I think it might been time for feature freeze soon(tm), features in the milestone exempt of course 21:34 sfan5 s/been/be/ 21:37 MTDiscord >remove core table from insecure env wont this break any mod that uses core.api call if mod security is turned off? 21:38 sfan5 nah, initializeSecurity() is never called then 21:40 MTDiscord ah ok 22:43 MTDiscord NOOOO 22:43 MTDiscord not getinfo 22:43 MTDiscord you can't do this 23:43 MTDiscord probably next time