Time Nick Message 00:04 rubenwardy updated #5154 00:04 ShadowBot https://github.com/minetest/minetest/issues/5154 -- Add ItemStack key-value meta storage by rubenwardy 00:13 bkeys I compiled minetest from source and am trying out the portable option (and I built the server too). But I get this error 00:13 bkeys http://paste.fedoraproject.org/545156/48616718/ 00:13 rubenwardy you need to download minetest game 00:14 thePalindrome The default game is minetest_game, not minetest 00:14 thePalindrome That too :P 00:14 rubenwardy http://github.com/minetest/minetest_game 00:14 rubenwardy thePalindrome, for some reason gameid = minetest 00:14 rubenwardy in world.mt 00:14 thePalindrome huh 00:14 rubenwardy not sure how it does that 00:16 red-001 I'm pretty sure changing it is out of the question to change that now 00:16 red-001 I'm pretty sure changing it is out of the question now* 00:17 rubenwardy ah 00:17 rubenwardy find_paths.push_back(GameFindPath(path + DIR_DELIM + id, false)); 00:17 rubenwardy find_paths.push_back(GameFindPath(path + DIR_DELIM + id + "_game", false)); 00:17 rubenwardy it looks for "minetest" then for "minetest_game" 00:18 rubenwardy that's subgame.cpp:84 00:19 rubenwardy then in getAvailableGameIds() it removes _game 00:25 octacian rubenwardy, red-001: why out of the question? Minetest is subject to change, I can't imagine it breaking too many things. 00:26 red-001 well it can be changed but that will mean more compatibly code 00:26 red-001 and mt-game was suppose to be separate from mt 01:09 baboon2 anyone knows where is the cpp code that handles the carts in 0.4.15? 01:09 Fixer carts are in lua 01:10 Fixer games\minetest_game\mods\carts\ 01:10 baboon2 ah ok i thought they were integrated in the core engine as of 0.4.15 01:20 Zeno` not in the engine. added to the game 01:20 Zeno` which is in Lua 01:23 baboon2 ok thanks 04:00 paramat #5157 needs review. simple, tested and fixes a crash 04:00 ShadowBot https://github.com/minetest/minetest/issues/5157 -- Objectpos over limit: Avoid error caused by sector over limit by paramat 04:14 paramat will merge #5167 later, fairly trivial mapgen stuff 04:14 ShadowBot https://github.com/minetest/minetest/issues/5167 -- MapgenBasic node resolver: Various fixes by paramat 04:35 paramat will also merge #5168 later, again fairly trivial mapgen stuff 04:35 ShadowBot https://github.com/minetest/minetest/issues/5168 -- Mgv6: Add stairs to desert stone dungeons by paramat 04:43 paramat bbl 06:13 paramat added 2nd commit to #5168 also trivial 06:13 ShadowBot https://github.com/minetest/minetest/issues/5168 -- Mgv6: Add stairs to desert dungeons. Add fallback node for gravel by paramat 06:15 paramat will merge that with #5167 in 30mins 06:15 ShadowBot https://github.com/minetest/minetest/issues/5167 -- MapgenBasic node resolver: Various fixes by paramat 07:17 paramat will merge 5167 5168 in a moment 07:23 paramat now merging to engine 07:29 paramat done 07:40 Zeno` k 11:47 VanessaE so, here's my weekly gripe about the entities-deletes-player, and flat-skin-or-green guy bug 11:47 VanessaE http://pastebin.ubuntu.com/23918690/ the former 11:47 VanessaE (no idea about how the latter looks) 11:50 VanessaE I'm at about 5 tries now to log into VE-C and not have one of the above happen 11:50 sfan5 i don't see a check for player objects in the code that does the deleting 11:50 sfan5 wasn't this already fixed supposedly? 11:50 VanessaE supposedly. 11:50 VanessaE but there are apparently at least two sources for the bug. 11:50 VanessaE because I've long since updated to a point after that fix. 11:55 sfan5 >if(!force_delete && !obj->isStaticAllowed()) should keep out the players though 12:02 sfan5 i guess adding a call to the function that prints the debug stacks to PlayerSAO::removingFromEnvironment() might help 12:03 nrzkt sfan5, also we have a type check just after i added twoweeks ago 12:04 sfan5 where? 12:05 nrzkt just after the piece of code you mention 12:06 sfan5 i see no check for a playersao..? 12:07 nrzkt oh sfan5 where is the check... 12:08 nrzkt https://github.com/minetest/minetest/commit/2ea60156437962d7d29d20606bf5d9189059f76b 12:08 Zeno` I'm sorry but this is a mess 12:08 sfan5 oh there 12:08 nrzkt but 12:08 nrzkt where is it in master 12:08 sfan5 nrzkt: https://github.com/minetest/minetest/blob/master/src/serverenvironment.cpp#L2091 12:09 sfan5 however it has zero effects because it's inside the if(obj->isStaticAllowed()) 12:09 nrzkt seems the fix has disappears 12:09 sfan5 and isStaticAllowed() is false for players anyway 12:09 nrzkt disappeared 12:09 nrzkt look at the diff 12:09 sfan5 yeah 12:09 nrzkt it's before verbosestream 12:09 sfan5 but a similar check is above 12:10 nrzkt oh maybe somebody changes this then 12:10 sfan5 nrzkt: https://github.com/minetest/minetest/commit/0eede97af2927dcda3545192403b0a44f30bcd1f 12:10 sfan5 you did 12:10 nrzkt oh right, i didn't remember 12:10 sfan5 and with your commit you made the check useless 12:10 sfan5 for reasons i explained above 12:11 nrzkt you are right 12:11 nrzkt we should restore the previous PR then, right ? 12:12 nrzkt i missed the static thing 12:12 sfan5 hm 12:12 sfan5 are players supposed to be deleted in the force_delete==true case= 12:12 sfan5 ? 12:12 sfan5 since it only happens at server shutdown 12:12 nrzkt when server shutdown the deletion is properly done 12:13 nrzkt the force_delete is not used 12:13 sfan5 it is https://github.com/minetest/minetest/blob/master/src/serverenvironment.cpp#L378 12:13 nrzkt hmmm exact 12:14 sfan5 it seems that it's actually intended that deactivateFarObjects deletes players in that case 12:14 nrzkt but in fact, the warning pointed at L2091 is shown when we are a player, it's very strange 12:15 sfan5 :/ 12:16 nrzkt it's why i did the second pr, very very strange 12:16 sfan5 maybe the thing returned by getType() is wrong? 12:17 sfan5 sounds unlikely though 12:18 Zeno` nrzkt, did you get my email? 12:19 nrzkt Zeno`, i didn't read them, looking 12:19 Zeno` no rush. Just making sure you got it 12:20 nrzkt Zeno`, you like detached inventories right ? :p 12:20 Zeno` I'm not sure 12:20 nrzkt armor and bags use them heavily, and yes it's broadcast to all client and yes it's shit 12:20 Zeno` I am not even sure they're the problem anymore 12:20 Zeno` anyway, look at the look when you get the time 12:20 nrzkt i don't know, for the processAOmessage it's due to distance no ? 12:20 Zeno` look at the log* 12:21 Zeno` no idea now. I'm too tired after looking at it all day 12:21 nrzkt no problem 12:21 Zeno` and yes the mods are updated 12:22 Zeno` notice the timestamps between my annotations as well ;) 12:22 Zeno` bbiab 12:28 sfan5 nrzkt: having the "forcing delete" message for players is actually impossible 12:28 sfan5 it calls obj->getStaticData before 12:28 sfan5 but PlayerSAO::getStaticData has an unconditional FATAL_ERROR 12:30 sfan5 nrzkt: anyway, what about this? http://sprunge.us/fCNV?diff 12:35 nrzkt sfan5, lgtm 12:36 sfan5 pushed 12:42 sfan5 VanessaE: you might wanna recompile, the bug should not happen again ... in theory 13:11 VanessaE sfan5: ok 13:16 VanessaE done. 13:22 VanessaE seems to work, at least at the first connect anyway 13:28 sfan5 does it print a warning? 13:29 VanessaE yep 13:30 VanessaE http://pastebin.ubuntu.com/23925208/ 13:30 VanessaE but I was still "in" so it worked. 13:32 sfan5 hm then that's just look for now i guess? 13:32 sfan5 s/look/luck/ wtf 13:32 VanessaE eh? 13:32 VanessaE oh maybe 13:33 VanessaE I won't know for a while yet 13:34 VanessaE the other question of course is those entities 13:34 VanessaE the block limit is 500, and after some extensive testing by tenplus1 and I, we ruled out things like signs_lib or digilines LCD as the cause 13:39 VanessaE notice how both counts increase over time. 13:40 VanessaE these are what I call "ghost" entities, because whatever they are, they're invisible and have no definite in-game source. 14:19 sfan5 VanessaE: mind pasting me the raw data of the problematic mapblock as returned by this tool https://gist.github.com/sfan5/52917a0e1338bcda2176edbc416dae60#file-y_leveldb_read-cpp ? 14:19 VanessaE moment 14:22 VanessaE just need to figure out how to compile it :) 14:23 sfan5 g++ leveldb_read.cpp -o leveldb_read -lleveldb -O2 14:23 VanessaE had to add -std=c++11 14:23 sfan5 oh that's default on arch 14:24 VanessaE ok, now where did I put that paste... 14:24 VanessaE there it is. 14:25 VanessaE http://pastebin.ubuntu.com/23925553/ 14:26 VanessaE that's about useless :) 14:26 sfan5 >need to sign in to download the plain text 14:26 sfan5 wtf ubuntu 14:27 VanessaE it's just a shitton of hex in one line 14:27 sfan5 yes that's correct 14:27 VanessaE moment 14:27 VanessaE I'll put the raw file up for download 14:27 sfan5 nah already got it 14:27 VanessaE oh ok 14:28 VanessaE inb4 "holy shit how does that block even load?" :)_ 14:29 sfan5 didn't you say it wasn't digilines? 14:30 sfan5 VanessaE: https://0x0.st/WKh.txt 14:30 VanessaE that's what tests indicated with worledit. there are some LCD panels and some signs in that block, but tenplus1 and I checked pretty thoroughly that each has precisely one entity 14:30 VanessaE ok then there are entities there that find_objects_in_radius() can't see 14:30 sfan5 i don't think this is what "precisely one" looks like :D 14:31 VanessaE maybe this is a bug in find_objects_in_radius() then? 14:31 sfan5 possibly 14:31 VanessaE because when I select a node with a single sign and run it on that node, I get exactly one entity for the sign and a few for the worldedit selection box stuff 14:33 sfan5 i guess it might be possible that the engine deduplicates the entities on loading them 14:33 sfan5 and thus never gets to removing all the duplicate ones(?) 14:33 VanessaE that sounds reasonable 14:40 VanessaE sfan5: so, that's something to chew on.. need some salt and ketchup for that? :) 14:43 sfan5 does also only one object appear when you use radius=2 in lua? 14:43 VanessaE good question. 14:43 VanessaE let me check 14:51 VanessaE damn it why can I never remember these commands... 14:52 VanessaE there we go 14:53 VanessaE http://pastebin.ubuntu.com/23925699/ 14:53 VanessaE (with a single sign node selected) 14:53 VanessaE correction, that's with a single LCD node selected. 14:53 sfan5 :/ 14:54 VanessaE (I chose one that had adequate clearance around it so that no other signs/LCDs would get included in the search) 14:54 VanessaE lemme try one at the coords from your paste 14:54 sfan5 what's your max_objects_per_block? 14:54 VanessaE 500. 14:55 VanessaE ok, the LCD indicated by your paste produces this: 14:55 VanessaE http://pastebin.ubuntu.com/23925715/ 14:55 sfan5 looks correct 14:55 VanessaE (there is a sign within the "2" search radius) 14:56 sfan5 if your max_objects_per_block was 500 then all the objects in the block would have been deleted on load 14:56 sfan5 the question is why didn't that happen 14:57 VanessaE the bigger question is, why doesn't /clearobjects (quick or not) permanently delete them? 14:57 sfan5 wait it doesn't 14:57 sfan5 wut 14:57 VanessaE unless get_objects_with_radius() is broken 14:57 sfan5 clearobjects does not use the radius thingy 14:57 VanessaE yep I kn ow 14:57 VanessaE know* 14:57 sfan5 it pretty much unconditionally clears all active & static objects 14:58 VanessaE but if it's deleting the objects like it should, and signs/LCD are re-creating them because that ^^^ API call it broken, the effect is the same, yes? 14:58 sfan5 i guess 14:59 sfan5 well then /clearobjects might not be broken 15:00 VanessaE right. 15:00 VanessaE just kinda brainstorming 15:01 sfan5 the code prevents storing more than max_objects_per_block in a single MapBlock 15:01 sfan5 so the check on loading one is kinda redundant 15:03 VanessaE and yet, the number of stored objects is somehow exceeding the configured limit 15:03 VanessaE http://pastebin.ubuntu.com/23925763/ here, you can see where it managed to store one, or at least it incremented the count 15:04 VanessaE (I'm not sure why it's talking about another mapblock's coords, unless this other user who is online and actively building, is in that area) 15:04 VanessaE (actually, he isn't) 15:06 VanessaE http://pastebin.ubuntu.com/23925785/ here's a similar example from the mapblock we were talking about. 15:22 sfan5 i have no idea how this could happen 15:24 sfan5 VanessaE: s/verbosestream/warningstream/ here https://github.com/minetest/minetest/blob/master/src/serverenvironment.cpp#L1857 might help me debug this 15:25 VanessaE sure. 15:25 sfan5 wait actually lemme just prepare a patch with more debugging stuff 15:25 VanessaE ok. 15:25 * VanessaE closes the editor she didn't quite have time to open ;) 15:49 sfan5 VanessaE: http://sprunge.us/bBiV 15:49 sfan5 i think git apply can use that as input 15:49 VanessaE ok 15:50 VanessaE yep, it took it 15:50 * VanessaE compiles... 15:52 VanessaE ok, that's loaded up. 15:53 VanessaE http://pastebin.ubuntu.com/23926117/ 15:54 sfan5 what 15:54 sfan5 how 15:54 VanessaE ? 15:54 sfan5 hm i guess it loaded the blocks but not activated them? 15:54 VanessaE oh the numbering :) 15:55 sfan5 can you place a player at the mapblock (-20,12,44) 15:55 VanessaE well, no one's signed onto the server yet so I can only assume something force-loaded the block 15:55 VanessaE sure, moment 15:55 sfan5 however force-loaded blocks should go through the "activation stage"... probably 15:56 VanessaE ok, I'm standing at those coords. 15:56 VanessaE http://pastebin.ubuntu.com/23926141/ 15:57 VanessaE I joined near the target, then teleported there 15:57 sfan5 well it did what it is supposed to do and deleted all the objects 15:57 sfan5 however why didn't it do that before when you tested the get_objects_inside_radius thing? 15:58 VanessaE beats me 15:58 sfan5 try teleporting into the (-12,2,-146) block? 15:58 VanessaE doing that now 15:59 VanessaE ok, I'm there. 15:59 VanessaE a small building in the area with a few signs inside it. 15:59 VanessaE nothing else of note 15:59 sfan5 anything in the logs? 15:59 VanessaE just more of the same "forcing delete" messages 16:00 VanessaE lemme wait a minute to see if something else happens there 16:00 sfan5 are the digiline signs active inside forceloaded blocks perhaps? 16:00 VanessaE I'm not sure 16:00 VanessaE possibly, I think mesecons uses forceloading 16:00 VanessaE not sure if digilines does 16:01 sfan5 if the forceload code missed the "activation" step that would cause activeobjects not to be added to the global list and the digilines signs to think no entity is there thereby creating a new one 16:01 VanessaE ah 16:01 VanessaE that sounds reasonable, too 16:03 VanessaE this showed up shortly after I teleported away, http://pastebin.ubuntu.com/23926193/ 16:03 VanessaE (a few more such messages in the log after that, with different ID numbers) 16:12 sfan5 VanessaE: you can de-apply the patch, uncomment https://github.com/minetest/minetest/blob/de664b1c6d4b2bca47f918a6a865a920434bf664/src/serverenvironment.cpp#L1242 and s/infostream/actionstream/, then restart the server but DONT join with a player 16:14 VanessaE sure. 16:18 VanessaE ok, it's up 16:19 VanessaE http://pastebin.ubuntu.com/23926307/ 16:19 sfan5 several questions arise 16:19 sfan5 also my theory from above is incorrect 16:20 sfan5 well not necessarily 16:21 sfan5 why is it trying to store stuff in block (-20,12,44) when it's not even active and miles away from the others? 16:21 VanessaE :) 16:24 sfan5 the only possibility i see is objects with incorrect coords being stored in some blocks 16:34 VanessaE if you have the storage and bandwidth to spare, you can download the raw map for local experiments 16:35 sfan5 hm 16:35 VanessaE http://daconcepts.com/vanessa/hobbies/minetest/worlds/Creative_World.tar.gz (3GB download, expands to around 3x that size) 16:35 VanessaE meanwhile, gotta run. back later. 16:38 rubenwardy updated #5154 16:38 ShadowBot https://github.com/minetest/minetest/issues/5154 -- Add ItemStack key-value meta storage by rubenwardy 16:57 sfan5 VanessaE: your backup doesn't work 16:58 sfan5 2017-02-04 17:57:23: ERROR[Main]: src/main.cpp:ef: int main(int, char **): A fatal error occured: LevelDB error: IO error: Creative_World/map.db/MANIFEST-3489868: No such file or directory 17:58 VanessaE sfan5: shit. 17:58 VanessaE could I just send you the missing manifest file? 17:58 sfan5 dunno 17:59 sfan5 you could test my code on your db though 17:59 VanessaE hm, nope. the filename is already different 17:59 VanessaE sure 18:00 sfan5 https://kitsunemimi.pw/i/d918c5bb66f54d8f.txt 18:00 sfan5 oh my ssl cert 18:00 sfan5 well just use a http link for now 18:01 VanessaE ok and what am I applying this on top of? 18:01 sfan5 master 18:02 VanessaE ok 18:04 VanessaE hm. that's odd, in my compress-worlds script, the whole map.db is grabbed in one go. that file shouldn't be missing. 18:05 VanessaE ok that patch is in place. 18:07 VanessaE so far, I just see a bunch of "trying to store", starting at id=1 with 664 objects in the block 18:07 sfan5 that would mean this isn't the problem 18:07 sfan5 then i'm out of ideas 18:08 sfan5 tho keep the patch applied, maybe it comes up with something sometime 18:08 sapier #5068 any comments? this allows updating animation framerate without resetting animation to start 18:08 ShadowBot https://github.com/minetest/minetest/issues/5068 -- Fix animation frame_speed and blend loosing precision due to incorrec… by sapier 18:09 VanessaE :( 18:09 sapier and fixes some loss of precision due to incorrect datatypes 18:10 VanessaE sfan5: what precipitates the name of that manifest file? I looked at the backup that was generated on Monday (which is where the downloadable file comes from), and it has a manifest, with a different ID number 18:10 sfan5 dunno 18:11 VanessaE and the server instance is always offline when the backup is made 18:11 sapier and #5073 please too it's a major improvement for 3d modes ... yes I know those aren't used by a lot of ppl atm 18:11 ShadowBot https://github.com/minetest/minetest/issues/5073 -- Stereo: replace toe-in by off-axis method (rebased #3620) by sapier 18:24 * VanessaE rebuilds the downloadable backups with --hard-dereference added. 18:28 VanessaE sfan5: re-download the backup and try again. it compresses and uncompresses cleanly now. 18:28 VanessaE (idk if the backup "works" but there were no errors) 18:30 VanessaE meanwhile, that patch has not produced any result yet. 18:35 sapier hmmm vanessae seems to be more carefull then the gitlab guys :-) good 18:36 VanessaE heh 18:36 VanessaE wisenheimer :) 18:36 sapier those guys did have multiple levels of backups but when they needed it last weekend none did work flawless 18:39 sapier talking about "hmmm" ... well not exactly, but using the name ;-) did anyone see him lately? 18:39 VanessaE I heard he left. 18:39 sapier sad to hear 18:40 sapier but better leave then ... 18:49 VanessaE I'm not entirely sure why he left 18:50 rubenwardy probably burnout, tbh 18:51 sapier that's not good hope he get's better soon 18:51 VanessaE :-/ 19:16 sapier hmm does anyone know where github stores it's data? maybe we should have look for european solutions ... unless we wanna pay taxes :-/ 19:17 juhdanad What is the problem with Github? 19:17 Krock lolwhat? is trumpy going to stand on the internet cables? 19:18 sapier welll problem ain't github but americans going crazy ;-) 19:18 sapier ok in fact only some americans, sadly the most powerfull one too 19:23 sapier crazy world the bad-country of 20th century seems to become the good-country of begining 21st ... 19:25 sapier but back to minetest what about #5068? ;-) 19:25 ShadowBot https://github.com/minetest/minetest/issues/5068 -- Fix animation frame_speed and blend loosing precision due to incorrec… by sapier 19:28 Calinou sapier: I don't think GitLab is hosted in the EU either 19:28 Calinou and switching away from GitHub would result in an obvious loss of popularity 19:28 sapier that's the bad thing about it almost everything is from us 19:29 sapier well Calinou I was more talking about preparing a plan "b" just in case 19:38 nerzhul sapier, github is geolocalized but it's an american company like gitlab 19:39 sapier well the gitlab guys have some backup issues ;-) yet that'd be irrelevant anyway if they're a american company too 19:39 nerzhul sapier, is there any news on crazy trump is missed, triggering some fear from you ? 19:40 nerzhul sapier, yeah, it affect only the database, and i think now they know they should do better? I'm very surprised they only have 2 db 19:40 sapier not exactly trump more the court decision claiming any american company having to provide any data stored anywhere in world 19:40 nerzhul but it was already the case before trump 19:40 sapier which is exactly different to court decision of last week where microsoft didn't have to doo that 19:41 sapier well I didn't limit my concerns to the primary bad guy ;-) 19:42 sapier but yes there's no reason for immediate actions by now as I said ... just thinking about options in case we need it 19:43 rubenwardy may be worth saving all the PRs and issues 19:43 est31 gitlab is fucking slow 19:43 est31 but i dont really care 19:44 sapier est31 if they have only one db we now now why too ;-) 19:44 sapier now know 19:44 nerzhul rubenwardy, gitlab can import all github pr and issues :p 19:44 rubenwardy they also like deleting them randomly :D 19:44 nerzhul gitlab CI is very very nice, we can publish easily snapshot builds :p and use docker for builing our apps 19:44 est31 either way, I think github will move countries before trump will hurt them 19:44 est31 also, they are a company,trump never hurts companies 19:45 nerzhul est31, i don't think it's possible, they are using AWS, and what about Amazon itself, Azure and Google Cloud ? They are american, they own the server who contains everybody on public cloud datas 19:45 sapier est31 depends on court decisions if the new style of deciding from fbi vs google will become official way of interpretation that wouldn't help 19:45 est31 what is the actual threat you want to protect from? 19:46 sapier I don't believe github will switch their company heat out of america 19:46 sapier company head ... grr 19:46 est31 trump finding out that you develop a block game` 19:46 est31 ? 19:46 est31 nothing scary about that 19:46 nerzhul it's not simple to change your company head location, having many fake societies is easy 19:47 sapier est31 well to be honest I don't believe there's any way to really protect yourself from american agencys you just can make it a little bit difficult 19:48 est31 *shrug* 19:49 est31 you will probably lose many contributors 19:49 sapier est31 as mentioned for the time beeing just a plan "b" 19:49 est31 yeah 19:49 est31 you know whats american too? 19:49 est31 biggest nuke arsenal in the world 19:49 est31 trump has direct access to it 19:50 juhdanad I don't understand... Why is Github bad suddenly? 19:50 sapier true ... and most powerfull military forces 19:50 est31 and he has the temper of a little child. single judge makes him go crazy 19:50 nerzhul i don't know juhdanad , but github is and was always bad, because one enterprise has all major repositories, like sourceforge at a moment 19:50 sapier juhdanad: it's not about github 19:51 est31 well idk, I think trump is powerful, but he doesnt seem to want to interfere with github 19:53 sapier est31 for sure he will interfere directly but transforming most powerfull democracy in world to some sort of tyranny most likely will have sideeffects to any american company 19:54 nerzhul sapier, yeah, i think he is a purely nationalist, like Marine Le Pen in france, but there are some rules if he doesn't want to trigger WW3 19:54 est31 idk, if usa becomes a tyranny I feel we have bigger problems than the repo for some open source game 19:55 sofar ya thunk? 19:55 est31 also, what nerzhul says 19:55 sapier nerzhul: I don't think he does care about triggering ww3 or not that's why I believe him to be dangerous 19:55 sofar hmm #politics 19:55 est31 maybe move to #minetest-project ? 19:55 sapier yes sofar is right :-) not correct channel 19:56 sofar =) 19:56 Fixer remember google code? :trollface: 19:56 nerzhul in France there are elections this year, and... big political parties are just dangerous, because they are against people 19:56 Fixer now imagine this to github 19:56 est31 elections this year in germany too 19:56 Fixer or imgur 19:56 sapier :-) what about #5068 (every good thing is three) 19:56 ShadowBot https://github.com/minetest/minetest/issues/5068 -- Fix animation frame_speed and blend loosing precision due to incorrec… by sapier 19:56 Fixer i need to stop using imgur for sure 19:56 nerzhul Fixer, google code was down because of GH. GH doesn't have a challenger atm 19:56 sofar let's discuss the flowerpot and frame mods I made - should I submit them to minetest_game? 19:56 sofar anyone? 19:56 sofar :) 19:56 red-001 why would you have to pay tax?? 19:57 sapier est31 true but for the moment risk germany falls to any radical party is minor (yet bigger then anytime before since ww2) 19:57 sapier flowerpot and frame? 19:58 sofar https://forum.minetest.net/viewtopic.php?f=9&t=16574 https://forum.minetest.net/viewtopic.php?f=9&t=16585 20:01 Fixer juhdanad: github is not bad in itself, it is just huge size of it, lots of project rely on it, now imagine closing of it... that will be a lot of headache for everybody 20:02 sapier sofar: not sure about the frame mod but flowerpot seems nice to me 20:03 rubenwardy agreed - it's pretty 20:07 sofar why not the frame? awww! 20:08 red-001 what is the general option on security for client-sided scripting? 20:09 nerzhul which option ? 20:09 red-001 it seems that lua hasn't had any major bugs that would allow a sandbox escape as long as there is no bytecode loading, but it's still possible for it to take up all the system memory 20:09 red-001 ehh I meant opinion 20:10 red-001 not sure how I managed to misspell it so badly 20:12 rubenwardy It should be very high priority, imo 20:19 red-001 are most people ok with a malicious server being able to crash or lag the client as long as it can't destroy the system/install malware 20:19 rubenwardy that's fine 20:19 rubenwardy well 20:19 rubenwardy using too many system resources isn't 20:19 red-001 or is a separate process with limitations on memory usage/cpu ussage needed 20:20 sfan5 that's way too much work 20:20 sfan5 as you need to somehow pass messages between then 20:20 sfan5 them* 20:20 red-001 I agree 20:21 red-001 well it might be possible to make attacks like that harder 20:21 red-001 by limiting the input to certain apis 20:21 rubenwardy having a separate process would also allow you to benefit from not having client scripting block rendering 20:21 rubenwardy but it is a lot of work 20:22 sfan5 that sounds like it would go into the "stupid limitations" territory way too fast 20:22 rubenwardy true 20:24 red-001 true 20:25 nerzhul make it working with good security, sufficient security for first released, and improve it after if needed 20:25 rubenwardy ideally we'd also start a bounty scheme for this. But - money 20:27 nerzhul before talking about making money, just finish the dev on the branch and merge it. 20:47 est31 limiting CPU or memory is very very hard 20:48 est31 better not attempt it 20:48 est31 everything else should be limited 20:48 sapier let os do cpu and memory things that's what it's purpose is 20:49 est31 yeah 20:50 sfan5 est31: it's not hard, use the apis provided by the os 20:51 est31 you can do cpu limiting using cgroups I think 20:51 est31 but these things would best benefit from a separate process ... 20:51 sfan5 you don't need cgroups 20:51 est31 also, would be hard to port it to all OSs minetest supports 20:52 est31 sfan5: ? 20:52 sfan5 https://linux.die.net/man/2/setrlimit 20:52 sfan5 RLIMIT_CPU / RLIMIT_RSS 20:52 sfan5 s/_RSS/_DATA/ 20:53 est31 interesting 20:53 est31 didn't know 20:53 est31 good catch 20:53 est31 still, it would benefit from a separate process 20:53 est31 bc then only the script process crashes 20:53 sfan5 it only really works in a seperate process 20:53 sapier est31 nope using os specifics will just cause trouble 20:54 sfan5 sapier: you're forced to use os specifics for things like this 20:54 est31 ^ i guess there is no way around this 20:54 sapier I wouln't do things like that for a game 20:54 red-001 ^ 20:54 red-001 I think the game crashing if the scripting crashes is a good thing anyway 20:54 sapier any reasonable os will provide ways to kill a programm running havoc 20:55 est31 it still gets played 20:55 est31 and most times on the same endpoints the people use for online banking 20:55 red-001 lets the mod authors know the messed up badly 20:55 est31 I want to be safe to play minetest on my computer 20:55 est31 and not fear that someone gets hold of my github account and can push shit to minetest repo 20:55 red-001 lua is quite safe as it has an internal vm 20:55 sapier est31 then don't hide the problems but make program abort 20:56 sfan5 red-001: no 20:56 sfan5 lua's vm contributes nothing to security 20:56 sapier usually adding workarounds just takes pressure from ppl to do things right 20:56 sfan5 it's just there because they use bytecode 20:56 sapier and cpu/memory limits are a workaround 20:56 sfan5 it takes pressure from people? wut 20:57 sfan5 if your client side mod gets killed because of cpu usage 20:57 est31 sapier: what would be the way to do it right 20:57 red-001 sfan5, so you would prefer to execute untrusted assembly? 20:57 sfan5 you can't just say "i dont care" 20:57 sfan5 red-001: that's not what i'm saying, i'm saying that luas vm is not designed as a security measure 20:57 sapier e.g. if game still runs but runns slow someone will have a fast cpu and tell "works for me" . 20:57 est31 I dont think luajit has ever been audited security wise 20:57 est31 I bet its full of holes 20:57 sfan5 that problem is impossible to eliminate 20:58 sapier sfan5: it's gonna get killed anyway 20:58 sapier if you limit cpu usage to 50% you'll just make it more difficult to decide what's reason 20:58 sapier 100% cpu usage is obvious 20:58 sfan5 i agree that cpu / memory limiting are not perfect solutions 20:59 sfan5 and it would be best if we are not forced to add them 20:59 est31 I too think they are not important 20:59 est31 what is important that nobody can execute code as the user who plays the game 20:59 sapier I'm completely against adding it because it's stuff a game is NOT supposed to do that's cpu stuff 20:59 sapier os stuff 20:59 est31 either way, gotta go 20:59 est31 bye 21:00 red-001 I don't think a major bug in lua jit is more likely the a similar one in minetest 21:00 sfan5 a buffer overflow in luajit would be way easier to exploit than on in minetest 21:00 sapier I agree to est31 stop malicious code is most important 21:01 red-001 so maybe use normal Lua? 21:01 sapier and malicious code usually tries to hide and not cause 100% cpu or memory usage 21:02 sfan5 cpu or mem limits are not supposed to stop malicious code 21:02 red-001 lua should have a lot more testing done on it 21:02 sfan5 they are supposed to stop code that uses way too many resources 21:02 sfan5 which player won't want 21:02 sfan5 (for something as unimportant as clientside mods) 21:02 sapier sfan5: yes but player already has wayst to prevent this ... better ways then everything we can do in minetest 21:03 sfan5 and those are? 21:03 red-001 limit the amount of memory minetest can use? 21:03 sapier so while I agree this is a interesting task and I understand why you're curious to try it ... I believe it's nothing for minetest 21:04 sfan5 i don't want to try it, it's hard and adds lots of additional complexity 21:04 sapier sfan5: 1) taskmanager/ shell (kill) 2) remove the broken mod 21:04 sapier simple and easy 21:04 sfan5 client-side scripting uses mods sent by the server 21:04 sapier then even more easy 21:05 sapier kill and don't connect to server once again 21:05 sapier server admin will realize noone connecting 21:05 sfan5 you expect the kids to understand that? 21:06 red-001 yeah 21:06 sapier sfan5: I expect ppl to understand "if I connect to that server minetest will hang" yes 21:06 red-001 if they are running a server then yes 21:06 sfan5 we're talking client-side here 21:06 sapier if they don't understand this they probably will have issues understand why moving that device next to the keyboard will cause cursor to move too 21:07 sfan5 I wouldn't be so sure of that 21:07 red-001 really servers can already make the game unplayable 21:07 sapier sfan5: maybe but we can't solve all worlds issues 21:07 sapier and reliably limiting cpu usage for all os in all versions is a task we just can't handle 21:07 red-001 e.g by sending so many textures it takes about 10 hours to connect to spawning particles or entities all over the place 21:08 sfan5 sapier: that's correct, but in theory it can be done for linux -> android which is a major platform 21:08 sapier especially as most os require root/admin privileges to set cpu limits which reduce security instead of increase it 21:09 sfan5 linux doesn't 21:10 sapier sfan5 maybe but linux does require root privileges to change to roundrobin scheduler thus you'd still be in same thread after cpu limit hit 21:10 sapier so you'll never have a chance to detect the cpu limit hit from within this proces ---> still hanging 21:10 sfan5 why do you need to change the scheduler to use setrlimit() ? 21:11 sapier you don't but the limit wont help if you intend to preserve minetests responsiveness 21:11 sfan5 well if you don't put the lua stuff into a seperate process you won't be able to limit it appropriately 21:12 sapier yes then you need ipc ... adding all that stuff just to prevent lua mods from doing same as textures? 21:12 sapier I don't see the actual benefit 21:12 sapier especially as you're talking about linux users which usually will understand reason <-> cause relationship 21:13 sfan5 you are correct it's way to much work to have a benefit 21:13 sfan5 however android is also linux 21:14 sapier true but there android already does way more limiting stuff 21:20 red-001 ^ 21:21 red-001 it takes quite a bit of work to downright crash android itself 21:21 red-001 and not just make the app close 21:22 red-001 plus I doubt android give enough access to apps to do something like that 22:09 rubenwardy merging #5154 in 20 minutes 22:09 ShadowBot https://github.com/minetest/minetest/issues/5154 -- Add ItemStack key-value meta storage and per-stack descriptions by rubenwardy 22:10 juhdanad You can see a bug report on #minetest. 22:10 juhdanad By Spidergirl321 22:10 rubenwardy debug_log_level? Yeah, that should crash more gracefully 22:10 rubenwardy well, it shouldn't crash at all 22:30 rubenwardy merging #5154... 22:30 ShadowBot https://github.com/minetest/minetest/issues/5154 -- Add ItemStack key-value meta storage and per-stack descriptions by rubenwardy 22:39 ShadowNinja rubenwardy: I really don't like that serialization method something standard like JSON would be much better. 22:40 rubenwardy agreed 22:40 rubenwardy preparing PR... 22:44 ShadowNinja rubenwardy: Great :-) It would be great if you got proper typing to work too so you're not just storing everything as strings. 22:45 rubenwardy yeah 22:45 rubenwardy separate issue though - node meta has always been untyped 22:45 rubenwardy *atyped 22:46 Fixer /clearobjects uses only one core? 23:06 rubenwardy what should I use as a value to indicate the meta is of the new format? 23:07 rubenwardy the problem with always attempting to JSON parse and doing backwards compatibility on fail is that strings in the old format may already be in JSON 23:19 rubenwardy ShadowNinja: #5176 23:19 ShadowBot https://github.com/minetest/minetest/issues/5176 -- Use JSON serialize to store ItemStack Metadata by rubenwardy 23:21 sofar Fixer: yes, that's entirely single threaded anyway 23:23 ShadowNinja rubenwardy: I'd remove the start character completely and just unconditionally set the "" field to the original string. 23:25 ShadowNinja Just try the serialization and skip it if it fails (and maybe print something to warningstream or infostream). 23:25 rubenwardy if there's no "" in the table though 23:25 ShadowNinja Yeah, you can check that first. 23:25 sfan5 using an empty key sounds like a bad idea 23:25 sofar rubenwardy: "How to properly detect old serialization (rather than #)" 23:26 sofar rubenwardy: did you mean "before this PR" or using get_metadata() ? 23:26 sfan5 (for a modder not for this compat scheme) 23:26 rubenwardy metadata() 23:26 sofar come to think of it, I'd ignore it either way 23:26 sofar mods can safely convert 23:27 sofar if you convert the data you will actually break mods, too 23:28 sofar if you leave it as is, nothing breaks (just uses the old method) 23:30 ShadowNinja rubenwardy: also, there's a space missing after the error message you added. 23:30 ShadowNinja (before the JSON root) 23:31 rubenwardy thanks 23:31 sfan5 "Malformed ItemMeta!" is a quite bad error msg 23:31 rubenwardy it's temporary 23:31 sfan5 and i agree with the just unconditionally set the "" field to the original string. thing 23:31 sfan5 the error msg should be "ItemStackMetadata::deSerialize(): Data looked like a JSON string but failed to decode, ignoring!" 23:33 rubenwardy what if a mod isn't updated, and still uses get_metadata()? Eg: set_metadata("foobar") get_meta():set_string("foo", "bar") gives you {"":"foobar","foo":"bar"} which will be the output of get_metadata after load 23:35 ShadowNinja rubenwardy: Don't write that field when serializing. 23:35 rubenwardy so set_metadata() will be lost then? 23:36 ShadowNinja Or rather, if that field exists, write it out, and if it doesn't then serialization an object. 23:37 ShadowNinja Using both APIs can be left as undefined behavior or something, as long as there's a defined way to update to a JSON system. 23:48 rubenwardy updated: #5176 23:48 ShadowBot https://github.com/minetest/minetest/issues/5176 -- Use JSON serialize to store ItemStack Metadata by rubenwardy