Time Nick Message 01:13 sofar wheps, didn't see I left here 01:13 sofar got a problem with minetest.compress/decompress 01:13 sofar files written on windows are not processable on linux 01:14 sofar https://github.com/sofar/luscious/issues/2#issuecomment-355088870 01:14 sofar known limitation? 01:15 sfan5 nope 01:15 sfan5 should definitely not be happening 01:16 sofar i couldn't `zcat` them either 01:17 sofar and I've used the method before on compressed files made by minetest 01:17 sofar so I know that works 01:18 sfan5 doesn't zcat only take gzip input? 01:18 sfan5 because what minetest produces is zlib (no gzip header) 01:20 sofar we use minetest.compress in itb 01:20 sofar 118.box.gz: zlib compressed data 01:20 sofar that's the blob I get out of sqlite where we store it 01:20 sofar # zcat 118.box.gz |wc -c 01:20 sofar 198369 01:20 sofar no issues 01:20 sofar I wonder if the method matters 01:21 sofar we explicitly use "deflate" in itb 01:21 sfan5 does gzip -d <118.box.gz work? 01:21 rubenwardy doesn't zcat only take gzip input? 01:21 sfan5 and no the method is ignored 01:21 rubenwardy you want zless for both types of input, as zcat doesn't work with plain text 01:21 sofar gzip -d < .... works 01:21 sfan5 well idk anymore 01:22 sofar hehe, well, me neither :) 01:23 sofar commuting, maybe I'll see later what the heck is up with the windows-created files 01:23 sofar my bet, MT core messes up network order 01:23 sofar or something like that 01:24 sofar afk 01:54 paramat tested #6838 needs reviews, it and #6765 have a new approach to these bugs now, see https://github.com/minetest/minetest/pull/6765#issuecomment-355017397 01:54 ShadowBot https://github.com/minetest/minetest/issues/6838 -- Fix ambient occlusion by numberZero 01:54 ShadowBot https://github.com/minetest/minetest/issues/6765 -- Add more neighbors on mesh update by numberZero 01:54 paramat 6838 now fixes the dark line bug and the occlusion bug 04:26 paramat merging #6875 trivial docs 04:26 ShadowBot https://github.com/minetest/minetest/issues/6875 -- Mgv7: Add docs for the new floatland exponent parameter by paramat 04:32 paramat merged 07:21 nerzhul merging #6087 now 07:21 ShadowBot https://github.com/minetest/minetest/issues/6087 -- [CSM] Don't load the IO library. by red-001 13:25 nerzhul hi, merging trivial #6855 now 13:25 ShadowBot https://github.com/minetest/minetest/issues/6855 -- Fix Wstringop-overflow warning from util/srp.cpp by HybridDog 14:37 Wayward_One #6876 14:37 ShadowBot https://github.com/minetest/minetest/issues/6876 -- Prevent Android from automatically locking display by Wayward1 19:15 Hijiri reviews for #6688? 19:15 ShadowBot https://github.com/minetest/minetest/issues/6688 -- Custom particle generators for particle spawners by raymoo 20:04 paramat please can #6838 be reviewed? high priority bugfix, needs another +1 20:04 ShadowBot https://github.com/minetest/minetest/issues/6838 -- Fix ambient occlusion by numberZero 20:07 paramat it now also fixes the long dark lines on the landscape 20:18 nerzhul merging #6876 20:18 ShadowBot https://github.com/minetest/minetest/issues/6876 -- Prevent Android from automatically locking display by Wayward1 20:27 paramat i suggest considering numberzero as a new dev 21:18 paramat thanks Krock #6853 updated. would you like me to explain in the thread how the PR works to make review easier? am happy to 21:18 ShadowBot https://github.com/minetest/minetest/issues/6853 -- Biomes: Add vertical biome blend by paramat 21:19 Krock sorry? "how the PR works"? That's your PR 21:25 red-001 lol 21:32 paramat erm i mean, explain the implementation more clearly if that helps review progress :} 21:36 paramat added some info to help reviewers to #6455 which is also ready 21:36 ShadowBot https://github.com/minetest/minetest/issues/6455 -- Get biome id at pos by paramat 21:42 paramat is there any preference in MT code for using, for example, a 'u8'/'u16'/'s16' etc. instead of an 'int' for a variable? 21:42 red-001 seems so 21:43 paramat indeed 21:43 red-001 int doesn't have the same size on all platforms so you need to use it for network 21:44 red-001 not sure why it gets used so much in other code 21:45 rubenwardy "optimisation" probably 21:45 Krock paramat, 6455 should probably wait for 6853 to possibly make use of BiomeGenOriginal::calcBiomeFromNoise to not have duplicated code 21:46 paramat i've noticed old MT code has ints, newer code tends to avoid them 21:46 paramat Krock yes 21:46 red-001 nerzhul, good spot totally missed that 21:46 red-001 easy to forget insert doesn't replace existing data 21:47 Krock s32 should be used instead of int, as it ensures being 32 bits wide. Especially important for networking code ofc 21:47 rubenwardy paramat, probably because nerzhul insists on them in reviews 21:49 paramat heh 21:50 nerzhul paramat, use the alias is very important and also ensure we have consistency in binary types, yes for network but also databases 21:50 paramat i guess less memory use? 21:50 paramat as a tiny advantage 21:51 red-001 nerzhul, any idea why block transfer is so slow? 21:51 nerzhul also yes 21:51 nerzhul red-001, on which part ? 21:51 nerzhul network ? map loading ? 21:51 red-001 not sure that why I ask 21:51 nerzhul gzip comprssion is the huge bottleneck 21:51 nerzhul 90% of network transfer time is comprss/decomprss gz 21:51 nerzhul :p 21:52 nerzhul same for mapblock load, it's around 75% 21:52 red-001 it's like the biggest user experince issue on desktop imo 21:54 nerzhul yes, especially in singleplayer mode 21:54 sofar cpu for compression may not be a problem 21:55 nerzhul sofar, just run callgrind and laugh :D 21:55 sofar the important factor is "how much latency does it give" 21:55 nerzhul changing it to LZ4 make mapblock loading blazing fast 21:55 paramat \O/ 21:55 sofar just don't -9? 21:55 sofar you can compress faster with zlib too 21:56 nerzhul i did it two years ago on my fork it's nice, but it's a little bit slower on bad network connection because it comprss a little bit less 21:56 nerzhul sofar, we don't use -9 in MT 21:56 red-001 I wonder what effect the local loopback has in singleplayer 21:56 sofar the default is max compression in zlib, I think 21:56 nerzhul the client loopback you mean red-001? 21:56 sofar does the client compress? 21:57 sofar I don't think it should even have to 21:57 nerzhul we pass -1 21:57 red-001 yea like how much faster it would be if it passed the data directly between the threads 21:57 red-001 like even just the networkpacket object 21:57 nerzhul red-001, shared buffer :p 21:57 nerzhul but it break the client/server model 21:58 red-001 not really 21:58 nerzhul #define Z_DEFAULT_COMPRESSION (-1) 21:58 nerzhul it should be 5 then 21:59 numzero nerzhul, types like s32 can't be used in network/database code anyway due to endianness 22:00 nerzhul numzero, it's not our problem it's libpq and libsqlite problem 22:00 nerzhul but yes for network and we have the writeU32 writeS32 etc wrappers for this 22:01 numzero and in other code, binary compatibility is irrelevant 22:01 numzero there are no binary modules AFAIK 22:01 nerzhul binary compat is on the distro level 22:02 nerzhul you will never run a BSD binary on Linux (invert is possible there is a linuxlator) 22:02 numzero yes, but that's a different problem 22:03 numzero MT provides no ABI, does it? 22:03 nerzhul no 22:04 nerzhul it's only a binary there is no library 22:04 sofar it is not a DSO 22:04 sofar nor does it dlopen() 22:05 numzero so why would you ever care of changes in types not used in serialization? 22:05 sofar nerzhul: did you see about minetest.compress output not being portable? 22:06 sofar write a file on windows, read it on linux == corrupt zlib data 22:07 sofar https://github.com/sofar/luscious/issues/2#issuecomment-355172600 22:07 nerzhul sofar, yeah i see it 22:07 nerzhul i did this test 1 year ago 22:07 sofar I'm shocked 22:07 nerzhul i published a MTS on my BSD server and i was unable to import it on Linux :p 22:07 sofar that's even worse 22:07 nerzhul ofc it's very bad 22:11 numzero btw, there are also types like std::uint_fast16_t 22:12 nerzhul numzero, yeah i just the std:: types but MT doesn't use it and we are bound to irrlicht types 22:12 numzero you just what? 22:13 nerzhul i like* 22:13 nerzhul :p 22:13 red-001 can someone give me an example of a mod that shows a new formspec without closing the old one? 22:14 numzero anyway, while I can understand using u* types in structs like MapNode, where that can save a lot of RAM, I don't understand using them everywhere, up to loop counters 22:16 nerzhul use less memory everywhere 22:16 nerzhul we have many memory usage in a correct gaming usage (> 1GB !) each memory segment should be optimized 22:17 numzero local variables don't consume memory per se 22:17 nerzhul i know lua is one of the main server problem because modders doesn't care about memory, but if we can optimize MT to run on less memory it's good (also android is one of our target and doesn't have 8GB memory) 22:17 numzero stack is allocated already 22:17 nerzhul i never trust the compiler to properly handle that 22:18 numzero it can't optimize structs 22:18 numzero because that would violate some rules 22:18 numzero but it can optimize local vars, and even if it doesn't, that's tiny amount in comparison to that 1 GiB 22:19 nerzhul i tend to prefer controlling maximum things instead of letting compiler doing everything for me, heh :p 22:19 nerzhul numzero, you are right, but there is no tiny profit :p 22:19 numzero btw, I have an idea of how to optimize meshgen memory usage 22:19 numzero and speed 22:19 nerzhul personnaly it's simple for me, if i know range i use the correct container 22:19 nerzhul nice 22:20 numzero nerzhul, the downside is that you actually prevent the compiler from optimizing things 22:20 numzero (maybe) 22:20 numzero not for memory, more like for performance 22:21 numzero but that's important as well 22:21 numzero *for speed 22:22 nerzhul for integer i'm pretty sure he does it if he can 22:25 red-001 there is a really good chance most of the local ints are just stored in cpu registers 22:25 numzero red-001, true 22:26 red-001 like 99.8% for loops 22:27 numzero only disassembling can help 22:27 numzero I will probably do that, but certainly not now 22:28 numzero btw nerzhul, did you noticed add_node(int ...)? 22:28 red-001 numzero, help what? 22:28 numzero help to understand what does actually happen 22:29 nerzhul i don't use that function 22:29 nerzhul (currently i'm working on the Game class refactor 22:29 rubenwardy ) 22:32 numzero oohhh... that's a big work 22:32 nerzhul it's not simple but i first create a GameUI class and pushed all texts to it 22:33 nerzhul and added some first unittests, but i cannot test all because i didn't get time to find if we can use irrlicht nulldevice to instanciate components without UI 22:34 rubenwardy you certainly can, not sure what happens if you try to render though 22:34 nerzhul the problem i get is i cannot instanciate any component because there is no driver currently, and then test the component modifications 22:34 nerzhul but if i solve this problem we should be able to unittest many UI parts 22:34 nerzhul i think i will look at this on another PR 22:35 nerzhul #6872 is sufficiently huge :p 22:35 ShadowBot https://github.com/minetest/minetest/issues/6872 -- [WIP] Refactor Game class to extract GameUI part by nerzhul 22:35 nerzhul it's not perfect but at least it enhances things for a first step 22:36 nerzhul and game.cpp lost 320 lines of code 22:36 nerzhul 4500 remaining lol 22:36 red-001 rm game.cpp 22:36 rubenwardy rm * 22:37 red-001 nerzhul, what did you change, just extract the formspec stuff? 22:37 nerzhul no 22:37 red-001 crash_and_catch_fire 22:37 rubenwardy GameUI is HUD 22:37 nerzhul formspec stuff is already outside it (except the main reference) 22:37 nerzhul GameUI is hud yes 22:37 nerzhul i should add minimap to it, maybe 22:37 rubenwardy really it should be GameHUD, but that's confusing when you have HUD 22:37 rubenwardy yeah 22:38 rubenwardy HUD could become LuaHUD, except for the fact that the item hotbar is used by core. So LuaHUD isn't correct 22:38 red-001 huh will this conflict with CSM hud stuff? 22:38 rubenwardy it's meh anyway 22:38 rubenwardy no 22:38 rubenwardy nerzhul, make sure to define the role of any classes 22:38 nerzhul red-001, no i'm not working on it 22:39 nerzhul but the extract can permit to easily add CSM bindings to customize the position of the core elements :p 22:40 red-001 ?? who changed the CSM doc to be a txt file? 22:40 nerzhul paramat, ? 22:40 red-001 oh 22:46 paramat yes =) see the PR for why 22:49 red-001 alright I see, you have a good point, better to keep it as a .txt till github improves markdown support 22:50 paramat nerzhul #6838 is updated, if anyone +1s it i can merge it later with #6853 22:50 ShadowBot https://github.com/minetest/minetest/issues/6838 -- Fix ambient occlusion by numberZero 22:50 ShadowBot https://github.com/minetest/minetest/issues/6853 -- Biomes: Add vertical biome blend by paramat 22:52 nerzhul sorry paramat it's late i cannot look at this 23:02 nerzhul #6872 is now ready for a merge 23:02 ShadowBot https://github.com/minetest/minetest/issues/6872 -- Refactor Game class to extract GameUI by nerzhul 23:10 nerzhul merging #6853 23:10 ShadowBot https://github.com/minetest/minetest/issues/6853 -- Biomes: Add vertical biome blend by paramat 23:11 nerzhul PR is good 23:12 nerzhul just a question, what is the 0.4.17 release state ? will this be released soon ? 23:12 nerzhul i missed some things the last 2 months :p 23:12 nerzhul i think we are a little bit late and some things can be nice for our customers (android included) 23:13 red-001 if there isn't an issue with the formspec mimigation stuff that really should be backported 23:13 nerzhul red-001, i think yes 23:14 red-001 also what happened to that better linter I heard about? 23:20 paramat no problem 23:22 paramat sfan5 prefers to release 0.4.17 at 0.5.0 feature freeze, to include as many backports as possible 23:23 sfan5 oh about the formspec mitigation PR 23:23 sfan5 i'd prefer if this throw warnings for now 23:23 sfan5 we can then ask owners of popular servers to run this patch and report the results 23:23 sfan5 then if we're sure that this does not accidentally break common cases (it shouldn't, but you never know), it can be set to reject suspicious things 23:24 red-001 can't really do this that way, I change the name thats sent to the client 23:24 sfan5 oh 23:24 sfan5 but 23:24 sfan5 why not 23:24 red-001 if I just passs it through it will sent stuff like "default:book_3344653" to lua 23:26 red-001 well I guess I could just store the name and not bother to give each formspec a uid 23:27 red-001 the uid doesn't really do much expect make it a bit harder to submit the formspec, you need to get the server to tell the client to open it and copy the uid it sends to you