Time Nick Message 00:14 paramat i'll merge https://github.com/minetest/minetest_game/pull/720 in an hour since it's biome tweaks 00:19 hmmmm i sorta agree with the comment on that pr... there is probably too much desert/arctic compared to regular biomes 00:20 paramat yeah it's an illusion, i'm adding a comment 00:22 paramat i was wondering what noise thresholds create an equal balance of frozen/temperate/hot biomes, currently it's -0.4, 0.4 as in mgv6 00:24 paramat i wrote a mod to test this and seemed to find the current thresholds actually create temperate areas with over a third of surface area 00:25 paramat it's a problem how close frozen gets to hot though 00:25 hmmmm i think the main problem with perlin noise based biomes is that you can't control the rarity independent of the size for each biome 00:27 paramat see https://github.com/minetest/minetest_game/pull/718 for comparison of persistence 0.5 and 0.4 00:27 paramat i was trying to increase the distance between frozen and hot blobs, it didn't help much 00:29 paramat and 0.5 has more interesting biome shapes so i'm staying with the classic mgv6 noise of 3 oct 0.5 persist 00:29 hmmmm ok then 00:31 paramat it's a dilemma. creating a nice separation between hot and frozen blobs then makes the extreme biomes too rare 00:31 paramat i had a few reports of players travelling 5kn to find desert 00:32 paramat ..or saying all they can find is temperate biomes 00:32 paramat in mgv7 00:47 hmmmm how do you feel about generating a permanent biome map on map creation 00:56 paramat hm, still using noise though? 00:57 hmmmm it's all noise 00:58 hmmmm but this would free biomes from having to use point-polled noise 00:58 paramat some mgflat dev https://github.com/minetest/minetest/pull/3313#issuecomment-154650568 00:58 hmmmm hills?? 00:58 hmmmm this was supposed to be super simple and flat 00:58 hmmmm a one liner basically 00:59 hmmmm well it makes sense I'd say 01:00 paramat i think it'll be popular 01:01 paramat as options disabled by default 01:02 paramat without the lakes there would be no papyrus or clay, so they are needed for resources 01:03 paramat ok so biome areas would be defined without calculation on every point. perhaps this could be part of your alternative biome manager? 01:04 hmmmm i've tossed some ideas around but never came up with definite plans 01:04 hmmmm it's not like we're trying to "save" on calculations here, it's just that deciding that node X will be thing based on value Y at point Z is very limiting since it rules out any procedural generation 01:05 hmmmm hmm I might get back into the mapgen business soon since I don't need to spend any time on client side modding :) 01:06 paramat i like the current system so i would support another system as an alternative, not a replacement 01:06 hmmmm I'm so glad I didn't have to do that after all 01:06 paramat it would be good to see you do some mapgen, and something you enjoy. nether and aether realms? 01:06 hmmmm that's on the list of priorities for mapgen-related things 01:07 paramat oh and v8 of course 01:08 paramat btw is it normal for the limit of 5000 blocks in client memory to use 2GB? seems excessive for such a small volume of world 01:09 hmmmm no idea 01:09 paramat just wondering if there is a bug 01:09 paramat ok 01:09 hmmmm the mapgen uses a huuuuge amount of memory, thanks to the buffering system 01:10 hmmmm oh I have a good idea on how to cut down on memory consumption 01:10 hmmmm I need to implement this for sure 01:11 paramat memory used seems to be the limit on how large a structure can be viewed, hopefully farblocks would help 02:50 paramat i will make the extreme biomes a little smaller, it's been requested a few times. also i've seen that with biome blend disabled mgv6 desert threshold is 0.45, so i'll go with +-0.45 02:52 hmmmm well I have an idea 02:52 hmmmm instead of blindly adjusting parameters why not manipulate the parameters to have the properties you're looking for? 02:53 hmmmm do a custom transform on the biome noise so that "moderate" values (i.e. values in between a certain threshhold) are dilated 02:53 hmmmm right now the way it works is that a bunch of random points on the noise grid are chosen and the greater the difference between them, the steeper the slope 02:54 hmmmm whenever you have really hot areas close to really cold areas, you have a very steep slope 02:54 hmmmm if you just move the threshholds that will make the extreme biomes much smaller since they don't have as much range 02:54 hmmmm of possible values 02:55 paramat i see 02:55 hmmmm wait, is this mathematically possible? 02:55 * hmmmm thinks 02:57 hmmmm or would this be equivalent to increasing the scale 02:58 paramat i'll consider it. maybe i'll try it to see if it helps. for now i'll go ahead with these tweaks 02:59 gregorycu hmmmm: Was it you or est that was working on the settings fixes? 02:59 hmmmm est 02:59 hmmmm paramat: disregard what i said, it would be equivalent to increasing the scale 03:00 gregorycu I just started a server before, and joined it, it plays like a different game 03:00 paramat ok 03:00 gregorycu With regards to the frustum culling, I think I can do it properly, but c99 said to leave it cause he is working on something similar 03:00 hmmmm ahh ok 03:03 gregorycu By the way, I have a funny feeling that Irrlicht is doing its own frustum culling as well as us 04:06 paramat much better i'm going with this https://github.com/minetest/minetest_game/pull/720#issuecomment-154774411 04:07 paramat will merge soon 04:08 VanessaE paramat: I dunno.. if it were up to me, I'd keep the sizes of the coldest and warmest, but increase the overall scale of the thing 04:08 VanessaE put even more of an interval between the extremes 04:09 VanessaE because honestly, a map like that ----> doesn't leave a lot of room for building large cities in a nice, grassy area 04:09 VanessaE (if that makes sense) 04:28 gregorycu I would have made the biomes bigger 04:28 gregorycu But I am not expert on this stuff 04:39 paramat the map has biomes 7.5 times smaller than default, for testing 04:39 gregorycu Ahh ok 04:39 gregorycu :) 04:39 paramat heh so no tiny biomes 05:03 paramat now merging game 720 05:07 paramat done 05:14 paramat next i'm going to consider a small part of desert becoming a new stone-only biome, like parts of mgv6 05:48 hmmmm uh og 05:48 hmmmm i just realized there's always been a race condition in myrand() 05:50 gregorycu lol 05:50 hmmmm well I guess it's not too bad unless somebody were to mistake it for being a secure random and not just an "i really don't care, anything will do" value 05:52 gregorycu Was it based on time? 05:53 hmmmm it's shared between multiple threads 05:53 hmmmm it's this way on purpose for simplicity and hard to fix 06:02 gregorycu I don't know why, but I find optimising so much fun 06:04 gregorycu probably not what minetest really needs however 06:04 gregorycu hmmmm: Do you have something in mind for me to look at? 06:05 hmmmm :) one of the optimizations I always wanted to do but never got around to doing was making SSE versions of Noise 06:05 hmmmm and NEON 06:05 hmmmm the latter is more valuable honestly 06:05 hmmmm but then again overall, perlin noise generation takes a miniscule amount of the cpu time spent 06:06 hmmmm the majority of it is allocating, copying, and deallocating memory thanks to extensive use of the STL 06:06 hmmmm death by a thousand cuts 06:06 gregorycu Yeah, I'm about to replace a std::map with a std::vector 06:06 hmmmm this is the kind of crap you don't optimize later on, but rather write it correctly the first time 06:07 hmmmm using a std::map here is 50% slower. "but that's okay", they reckon 06:07 gregorycu Yeah, I'm not bad at the... higher level optimisations, or at least, not the instruction level stuff 06:07 hmmmm "so what? it's just 50% of a very small amount of time" 06:07 hmmmm and that same design choice is repeated literally everywhere 06:07 hmmmm the result is an application that's 50% slower than it should be and it's nearly impossible to fix 06:08 gregorycu yep 06:08 gregorycu It's hard to fix the death by a tiny cuts cause many changes are very hard to quantify in terms of performance improvement 06:09 gregorycu You know it's there, it's hard to prove 06:09 hmmmm honestly the kind of "optimization" minetest needs involve changing the way entire things work 06:09 gregorycu lol 06:09 hmmmm stuff that will take way more involvement than changing a couple of data structures around 06:09 hmmmm and more than a weekend for sure 06:09 gregorycu I don't have a level of understandig with the engine that means I could do that 06:10 hmmmm exactly 06:10 gregorycu I'm an alright programmer, but yeah, need more knowledge before I can tackle that 06:10 hmmmm but this is the same problem as with every other FOSS project 06:10 hmmmm hard to penetrate and make meaningful changes to because it requires so much familiarity 06:14 paramat http://i.imgur.com/vMCH6eg.png found the ultimate protruding dungeon 06:14 hmmmm lol 06:15 gregorycu Can you call it a prolapsed dungeon? 06:15 hmmmm so we call these things "dungeons" but they honestly look like a random set of rooms connected by random hallways and stairs 06:15 hmmmm what makes them dungeons 06:15 gregorycu What is a dungeon? 06:15 hmmmm there needs to be a mod added to default that fills dungeons with interesting things like libraries (a la minecraft strongholds) or chests filled with goodies 06:15 hmmmm :( 06:15 paramat http://i.imgur.com/HKs8y7R.png and a good place for a base in a floating island, just build steps from the spike on the ground 06:16 paramat yeah okay i'll try to get loot added 06:16 paramat there are mods out there already that do that 06:17 hmmmm I bet they don't use GenNotifier 06:17 paramat hehe 06:18 hmmmm what 06:19 hmmmm ermm 06:19 hmmmm is print() in lua supposed to automatically append a newline? 06:19 paramat sometimes i wish it didn't 06:19 hmmmm this is a regression 06:20 hmmmm oh god 06:21 hmmmm thanks SN :( 06:22 paramat it's always appended a newline 06:23 hmmmm actually nvm this is an est bug :- 06:26 paramat blockmen has recently started working on a dungeon loot mod :) it uses gen notify 06:26 hmmmm good i'm glad people are using it for their mods 06:26 paramat i'll try to get it considered for game 06:30 hmmmm so what is holding back mobs from being put in the default game exactly? 06:30 hmmmm i keep hearing they're "laggy" but nobody ever explains further and several people say the opposite 06:34 hmmmm PTAL: https://github.com/kwolekr/minetest/commit/70a3113141ceec8b29f868722c2f4dfba3c32283 06:35 paramat perhaps also because many players don't want hostile mobs by default 06:35 hmmmm this is est's pull request, with some improvements 06:35 hmmmm well lots of improvements actually 06:39 paramat time to omnom 07:04 gregorycu https://github.com/minetest/minetest/pull/3348 07:05 hmmmm ah, but what is the absolute improvement 07:05 gregorycu In terms of... fps? 07:05 hmmmm FPS, CPU time spent, etc. 07:07 gregorycu How often updateDrawListPerf is called depends on how much you look around 07:07 gregorycu If you look around lots, it's called every frame 07:08 gregorycu I circumvented the logic to make it be called every frame (to make sure my tests were consistant) 07:08 gregorycu CPU usage for that function from 3.73% down to 2.27% 07:09 hmmmm how much actual time is spent per call? 07:09 hmmmm microseconds? milliseconds? 07:09 gregorycu I'll find out for you 07:09 hmmmm TimeTaker("asdf", NULL, PRECISION_MICRO); 07:09 hmmmm err 07:10 hmmmm TimeTaker t("asdf", NULL, PRECISION_MICRO); ... printf("updateDrawListPerf: %uus\n", t.stop()); 07:11 gregorycu There is a scopeprofiler there already 07:11 hmmmm but does it have microsecond precision 07:13 gregorycu Just milli 07:13 gregorycu Ok, I'll use the code you said 07:31 gregorycu Where does printf go? 07:32 gregorycu (When you're running as a client, not singleplayer) 07:35 hmmmm what, minetest doesn't open a console window for you? 07:35 hmmmm you run windows right? 07:36 gregorycu As it turns out, I'm retarded 07:36 gregorycu Yeah, it's there *sigh* 07:42 gregorycu results are in 07:43 gregorycu Before: 780us, After: 517us 07:43 hmmmm that's really significant for something that runs in a render loop 07:43 hmmmm we only have ~16ms to do our stuff 07:44 gregorycu It runs at least 5 times a second 07:44 hmmmm oh so not every loop, right 07:44 gregorycu (It doesn't run every frame, unless you're looking around a lot) 07:44 hmmmm er frame 07:45 gregorycu I'll add this detail to the PR 07:46 hmmmm it's a rather simple change so i don't see any reason why this shouldn't be merged 07:46 hmmmm even though it might not be a massive win 07:47 hmmmm gregorycu: see how much more time you can save by reserving vector space 07:47 gregorycu Yeah, thought about that 07:47 gregorycu But the vector is going to go organically, and stay inflated 07:47 gregorycu .clear doesn't delete memory 07:48 hmmmm oh nevermind 07:48 gregorycu So yeah, first couple of frames may be a little slower than they need to be 07:48 hmmmm the vector in question is a member of ClientMap 07:48 gregorycu Yep 07:48 gregorycu Used to be a map, where nobody ever uses the key to lookup 07:48 gregorycu It's possible the other function that uses this member will be a little faster 07:49 hmmmm looks good to me 07:51 gregorycu Thanks 07:53 paramat reducing fps jitter is much needed 07:59 paramat fps stutter seems bad recently, someone has been analysing it https://forum.minetest.net/viewtopic.php?f=6&t=13360 08:01 hmmmm obviously the background threads need to be made lower priority 08:01 hmmmm but setting thread priority in non-windows is hard 08:03 gregorycu How many threads does minetest use? 08:04 gregorycu In singleplayer, looks like 4? 08:04 hmmmm nah. 08:05 hmmmm game thread, mesh making thread, minimap thread (if that's enabled), 4 AsyncRunQueue threads, server thread, connection send thread, connection receive thread (one of each for client and server), emerge thread, thread thread, thread thread thread 08:05 gregorycu So a few 08:05 hmmmm OpenAL thread I think 08:06 gregorycu I honestly think est's settings changes will improve this a lot 08:06 gregorycu We often read settings in a loop a lot 08:07 hmmmm yeah but is it really that slow? 08:07 hmmmm even under contention 08:07 gregorycu My singleplayer experience is magnitudes different than my client+server experience 08:07 gregorycu In terms of jitter 08:07 hmmmm well of course 08:07 gregorycu Of course? 08:08 hmmmm you're running half as many threads 08:08 gregorycu Same box 08:08 hmmmm oh 08:08 gregorycu I'd put money on it. Anyway, we can hoist some settings out of loops 08:09 gregorycu We should probably consider this - there is correctness issues too 08:09 gregorycu And yeah, segmenting the settings would be good 08:09 paramat i've actually noticed stutter gets worse when i look around a lot (singleplayer) 08:10 gregorycu Well, my latest change will make that a little better 08:10 gregorycu (feel free to merge and see) 08:11 paramat yeah good 08:19 gregorycu I had a run around, went high 08:20 gregorycu Make sure a lot of blocks were loaded. updateDrawListPerf used to take 3ms. Did the same thing after my change. 1.7ms. Which is much better. 08:20 gregorycu However, 1.7ms is a lot to add 08:20 gregorycu When turning 09:15 gregorycu Only one more +1 needed: https://github.com/minetest/minetest/pull/3348 09:16 Krock I'd give you one. 09:27 paramat i can't judge the implementation but +1 09:30 gregorycu Hooray! 09:35 RealBadAngel https://github.com/minetest/minetest/pull/3348/files#diff-37ea431d4de34ca3cc1fb0c79a36d1fbL414 09:36 RealBadAngel why is that removed? 09:39 gregorycu Nothing is using it, and populating the supporting member took CPU time 09:39 RealBadAngel thats irrlicht stuff i think 09:40 RealBadAngel maybe better just to comment it out instead of deleting? 09:41 gregorycu Irrlicht sources have no mention of sectorWasDrawn 09:41 gregorycu And no mention of sector, apart from "SecToRead" 09:41 celeron55 my current changes are going to remove it anyway 09:41 gregorycu :) 09:41 celeron55 merge conflict incoming 09:41 gregorycu Why comment out? This isn't the 90s. We have source control 09:42 gregorycu grrrr 09:42 celeron55 as i said, don't touch that area of the code; it will be just annoying for no good reason 09:42 gregorycu I didn't touch that area of code 09:42 RealBadAngel ok, nvm 09:42 gregorycu I was going to change the frustum culling 09:43 gregorycu In any case, are they incompatable changes? 09:44 celeron55 please, no 09:45 celeron55 this is hard enough without random conflicting changes being knowingly added upstream 09:46 celeron55 i am reworking map rendering; please don't rework map rendering 09:47 celeron55 i am also reworkring map transfer; please don't work map transfer 09:47 celeron55 -r 09:47 celeron55 rework* 09:48 celeron55 there are millions of other things in minetest that can be worked on meanwhile 09:51 gregorycu I was going to fix settings, but apparently est is fixing them. Failing that, i was going to do frustum culling, but you said not to touch then. Ok, so I looked at container inefficiencies with regards to the drawList calculation 09:51 gregorycu So, what do I do here? Withdraw my PR? 09:53 celeron55 well you can leave it hang around until i get this stuff into an upstream-worthy state 09:53 gregorycu How long is that? 09:53 celeron55 then it serves as a reminder to check whether the equivalent changes are still applicable or not 09:54 gregorycu If you're totally redoing map rendering, then it's pointless 09:54 gregorycu Assuming the code you replace it with doesn't also have the same wrong container type being used? 09:54 celeron55 not totally, but for example i already removed std::set m_last_drawn_sectors and replaced it with another system 09:55 celeron55 (it was originally used for the original FarMesh that i am making a successor to) 09:56 gregorycu You could merge my changes and rebase now 09:57 celeron55 i guess i'll add that to my to-do list 09:57 celeron55 this is now the eleventh thing on it 09:59 gregorycu I have other changes I'm working on around perf, I'll undo them 10:00 celeron55 if they aren't in map rendering or transfer, they aren't in conflict with what i'm doing 10:00 gregorycu I can wait until you're done, and I'll take another pass 10:01 celeron55 i'm done after maybe one week 10:01 celeron55 assuming i can keep up with this pace which i'm quite concerned about actually though 10:02 gregorycu Well, I can come back in 7 months like I did last time 10:02 celeron55 i might just burn out and be away for half a year if i fail 8-) 10:02 gregorycu 7 months > 6 months 10:02 gregorycu Should be good 10:02 celeron55 well in that case you can make anything you want meanwhile 10:02 RealBadAngel just find another random file and look for something to work on ;) 10:03 gregorycu My optimisations are guided by profiling data 10:03 gregorycu But maybe I'll look at a bug 10:05 gregorycu maybe I can look at windows bugs 10:36 celeron55 hmm 10:36 celeron55 looks like minetest will still be primarily using server-side block selection 10:37 celeron55 it's ending up making way more sense 10:37 celeron55 i'm still including a way to fetch blocks from any positions though 11:09 celeron55 i have so far found at least one bug in this block sending 11:09 celeron55 so maybe i am able to get rid of the mysterious block transfer holes 11:10 celeron55 (no promises) 11:15 gregorycu celeron55: How do you do sampling profiling on linux? 11:17 celeron55 pretty much the only way i ever do profiling is callgrind 11:17 celeron55 veeeeery slow but works well 11:17 gregorycu isn't that rather disruptive (slows down the app) 11:17 gregorycu ok 11:17 celeron55 yeah; good luck getting 0.1fps 11:19 gregorycu Urgh 11:19 gregorycu Is there a non-disruptive sampling profiler for linux? 12:40 celeron55 bleh 12:41 celeron55 i think this whole emerge system has been broken for a huge amount of time making so that a server will waste a lot of I/O resources if max_block_send_distance is larger than max_block_generate_distance 12:41 celeron55 this is because how this system was supposed to work is that if a block is not found on disk and won't be generated, it will be added to the Map as a dummy block 12:41 celeron55 but for some reason something has been changed so that it won't be added at all 12:42 celeron55 and this results in infinite polling of the same blocks from the database or disk 12:43 celeron55 i'm not sure yet but something doesn't seem right in how this is behaving 12:44 gregorycu :/ 13:01 celeron55 i wonder at which point this really is broken 13:17 celeron55 hmm... looks like they actually end up full of CONTENT_IGNORE if i am not grossly mistaken 13:19 celeron55 okay i think that is actually fine; the issue is how this algorithm treats non-generated blocks 13:20 celeron55 maybe that has changed some years (lol) ago and not surprisingly nobody figured that this wasn't originally prepared to handle it 8) 13:22 celeron55 i think the issue is that block->isGenerated() has changed meaning 13:24 Calinou I make my server use same generate and send distance, problem solved 13:26 celeron55 this assumes that if a block isn't generated and isn't dummy, throwing it at the emergethread will cause it to get generated - which simply doesn't happen now as in emergethread's opinion a block like that is the block's final form until it is allowed to generate it; which is obvious but previously all of those might have been dummies in practice 13:28 celeron55 ...previously as in "maybe a few years ago" 8) 13:38 celeron55 wait, what... i think there might still be a problem with missing dummy blocks too 13:40 celeron55 maybe not; i am not getting consistent results with repeated tests 14:31 celeron55 i wonder if we should drop the view distance auto-maybe-figure-it-out thing and just tell the user to use shortcuts or settings to tweak it 14:32 Calinou yeah, IMO the auto-tuner is a flawed idea 14:32 Calinou set a sane value by default, and tell user to tweak it on first run 14:32 celeron55 it would be much easier with this farthing to just add ctrl-[+-] or something to tune the far distance instead of trying to make something that would juggle those automatically 14:32 celeron55 i mean, i still think it's fine for one value, but with these two it becomes practically impossible 15:37 celeron55 hmm; i measured an 1.2% worse rendering performance when using std::map instead of std::map> 15:38 celeron55 i guess this is just a measurement error 15:38 celeron55 but this would speak against dropping the sector stuff 15:42 celeron55 i even measured std::map first so std::map> should have been subjet to more CPU throttling due to heating too 16:01 PilzAdam celeron55, what about std::map>>? 16:02 celeron55 i have tens of more important things to do than to test these, franky, so that will stay as a mysteryr 16:02 celeron55 -r 16:02 celeron55 but you can try if you want; just fill in about 600 heap-allocated pointers into it and iterate through it a lot 17:17 est31 pushing in 10 mins: https://github.com/est31/minetest/commit/b55477798ae999d58e4ee84d538b4ee1757912c9 17:17 est31 fixes #3349 17:18 est31 https://github.com/minetest/minetest/issues/3349 17:33 est31 ah perhaps the commit message should have been a bit more nicer 17:33 est31 e.g. to point out that its only about srp pcgrandom 17:33 est31 not about numeric.h pcgrandom (or wherever our pcgrandom now resides) 17:39 est31 and no answer for http://lists.gnu.org/archive/html/bug-gettext/2015-10/msg00025.html 17:39 est31 very sad, PilzAdam seems I'll run updatepo.sh then without a fix for the issue 17:40 PilzAdam :-/ 17:41 est31 but before, I'll merge all the translation commits 17:41 est31 59 commits 17:41 est31 oh man 17:43 hmmmm est, that's not the only problem though. if that random is supposed to be "secure", why does it produce the same result every clock() tick? 17:44 est31 what do you mean? 17:45 est31 you mean if time(NULL) and clock() have the same output than the seeded value, the random can be reproduced? 17:45 est31 well, thats the problem if we have no /dev/urandom 17:45 est31 I dont want to rewrite the kernel's pseudorandom pool 17:46 hmmmm well if that system doesn't have a urandom you're lying to the user that their account is cryptographically secure 17:46 est31 you want me to call abort() or something? 17:46 hmmmm because then you have a random that's about as secure as debian's RNG if called fast enough 17:47 est31 or better, pass the error to the calling function 17:48 hmmmm did you look at this https://github.com/kwolekr/minetest/commit/70a3113141ceec8b29f868722c2f4dfba3c32283 17:49 est31 not yet 17:51 est31 what is changed between this impl and my pr? 17:52 hmmmm a lot 17:52 hmmmm but the biggest thing is that the PcgRandom fallback stays secure in between clock ticks, it alerts the user, the fill loop logic is simplified and a rather serious mistake was fixed 17:53 est31 serious mistake? 17:54 hmmmm the first m_rand_idx bytes of outputbuf are uninitialized 17:54 hmmmm unset rather 17:54 est31 oh 17:55 est31 I'll have a closer look at it later, right now I want to review the translations, ok? 17:55 hmmmm also this version tries to fall back to /dev/random if /dev/urandom can't be found 17:55 hmmmm and there's more error checking in general 17:55 est31 thats not good 17:55 hmmmm why not 17:55 est31 /dev/random is blocking 17:55 hmmmm so? 17:56 hmmmm also that sounds to me like platform-specific behavior 17:56 est31 Rather fall back to something entirely insecure than something blocking 17:56 hmmmm that's a horrible justification for being insecure 17:56 est31 This is a game, no banking app 17:56 hmmmm security should come above all else 17:56 rubenwardy What's the point of allocating 4 bytes when you only use 2? Not important, just wondering if I'm missing anything: https://github.com/minetest/minetest/blob/master/src/server.cpp#L731 17:57 est31 people want to play it, not wait for /dev/random 17:57 sfan5 in wonder what kind of broken or outdates you unix must be to have /dev/random but not urandom 17:57 sfan5 s/in/i/ 17:57 hmmmm if they decided that security wasn't a very important part of their random, they probably would not be using a class called "SecureRandom" 17:58 hmmmm sfan5, Solaris 8 17:58 sfan5 does irrlicht run on solaris? 17:58 hmmmm probably 17:58 sfan5 .. 17:58 sfan5 do we intend to support running MT on solaris? 17:58 hmmmm completely irrelevant question. 17:59 hmmmm if you code to a set of platforms you explicitly "support", you're setting yourself up for failure 17:59 est31 hmmmm, in the case /dev/urandom doesn't exist we should expose it to lua, and make the constructor return nil. 17:59 est31 my opinion 17:59 hmmmm and the PcgRandom backup? 17:59 hmmmm just remove that altogether? 18:00 sfan5 i agree that doing stuff like only supporting specific versions of distros is stupid 18:00 sfan5 but deciding which kernel to support is different 18:00 sfan5 it can be done if it doesn't hurt 18:00 est31 "Starting from Solaris 9, /dev/random and /dev/urandom became device nodes since a kernel-based implementation was done" 18:01 est31 https://blogs.oracle.com/yenduri/entry/dev_random_in_solaris 18:01 sfan5 (which is debateable in the /dev/random case) 18:01 sfan5 but we don't need to write code to run MT on PonyOS 18:01 hmmmm well maybe you don't think you should 18:01 hmmmm I won't +1 code that *breaks* on PonyOS because you decided on being stubborn 18:02 celeron55 here's an interesting picture: http://i.imgur.com/9JJYg7T.jpg 18:02 sfan5 it's not stubbornness 18:02 sfan5 it's supporting a sane set of platforms 18:02 celeron55 i happened to walk around on the left side of the mountain before jumping down where i took that screenshot 18:02 est31 hmmmm, your patch won't run on PonyOS either, it will block, not run. 18:02 celeron55 and just enabled rendering of everything 18:02 est31 your version of my patch* 18:02 hmmmm est31, what does it wait for? 18:03 est31 for input from the hw 18:03 hmmmm and again the blocking thing is totally implementation specific 18:03 hmmmm that is a Linux-ism 18:03 est31 do a hexdump /dev/random 18:03 est31 try it, 18:03 est31 it will spew some bits, then be silent 18:03 hmmmm in general /dev/random has nothing to do with blocking on randomness 18:03 est31 it does 18:03 hmmmm est31: I get an immediate number of bytes, like I want 18:03 hmmmm est31: on my system /dev/random is symlinked to /dev/urandom 18:04 hmmmm and I'll be willing to bet it's different on every single OS 18:05 est31 yeah seems you're right 18:05 est31 "Unlike the Linux /dev/random, the FreeBSD /dev/random device never blocks" 18:05 est31 https://en.wikipedia.org/wiki//dev/random 18:05 hmmmm we're talking about "Linux" and "FreeBSD" 18:05 hmmmm how about "All OSes that conform to the POSIX standard" 18:06 est31 well, /dev/random blocks sometimes, sometimes it doesn't 18:06 hmmmm the point is 18:06 hmmmm /dev/random is not required by the standard at all 18:06 hmmmm neither is urandom 18:07 hmmmm in fact the only files you can guarantee for /dev to have are /dev/null, /dev/tty, and /dev/console 18:07 est31 both my patch and yours work for Linux, freebsd, and solaris 18:08 est31 the question is behaviour outside of those operating systems 18:08 hmmmm the answer is: it's a crapshoot 18:08 hmmmm some might block 18:08 hmmmm some might not 18:09 hmmmm but in general I think it is safe to assume that if an OS does have /dev/random, it has /dev/urandom as well 18:09 est31 OpenBSD has urandom too 18:09 celeron55 can someone explain why the blocking of a random device is such a problem 18:10 hmmmm that assumption could definitely be incorrect, but things should be coded in such a manner to fail in a safe way 18:10 est31 celeron55, blocking can happen for long periods of time, e.g. seconds. 18:10 celeron55 they only block if you pull huge amounts of bytes from them and you shouldn't be doing thatt 18:10 hmmmm using an unsafe random in "SecureRandom"... that's unacceptable IMHO 18:10 celeron55 -t 18:11 est31 due to minetest's single threaded nature, you would block the entire server 18:11 hmmmm I advocate for returning nil on secure rand failure 18:11 est31 okay for me 18:11 sfan5 that' the only sane choice 18:11 hmmmm okay then I'll change it 18:11 est31 without /dev/random or pcgrandom 18:12 est31 btw see what openbsd says on /dev/random 18:12 est31 http://nixdoc.net/man-pages/openbsd/man4/urandom.4.html 18:12 est31 "reserved for future support of hw random" 18:13 est31 so, /dev/random on openbsd isnt available in most cases 18:13 est31 it seems what /dev/random is on linux, is /dev/srandom on openbsd 18:13 hmmmm and /dev/srandom blocks on enthropy 18:13 hmmmm yup 18:14 hmmmm i suppose /dev/random exists for newer intel processors with RDRAND 18:14 est31 ah that thingy 18:14 hmmmm or any systems with TPMs or cryptographic modules (like, an actual PCI card that does nothing but crypto) 18:14 hmmmm (those exist believe it or not) 18:15 est31 I think I've heard that the devs of some crypto library, perhaps it was actually openssl, have a strong dislike against intel^TM random generator 18:15 est31 they believe it is backdoore 18:15 est31 d 18:15 hmmmm everybody with half a brain wouldn't trust rdrand 18:15 sfan5 >some crypto library 18:15 sfan5 you mean linux 18:15 est31 ah that can be it 18:15 hmmmm anybody* 18:16 hmmmm it probably uses the NSA Powered(tm) Dual Eliptic Curve rand 18:16 sfan5 https://en.wikipedia.org/wiki/RdRand#Reception 18:24 est31 LOL Kartenerzeugerbiomluftfeuchtigkeitsübergangsrauschparameter 18:24 est31 wuzzy likes komposita 18:24 Krock Oh gott.. 18:25 * Krock adds that to his "long words" list 18:27 celeron55 holy shit this FarBlock geometry consumes a lot of memory 18:28 celeron55 basically two gigabytes 18:28 Calinou new splash screen proposition: "More RAM than Minecraft ever consumes!" 18:28 celeron55 well i think this is good use for RAM 18:28 celeron55 you can always disable it 8) 18:30 celeron55 i need to make more features that make good use of RAM because 2 gigabytes is only 13% of what i have 18:32 hmmmm yeah I said the same thing about my perlin noise algorithm 18:32 hmmmm it's causing problems now... people constantly complain about high memory usage 18:34 est31 well I have 4 GB 18:34 est31 and I usually have other programs open than minetest as well 18:34 est31 like a browser with github 18:34 est31 or hexchat 18:34 est31 or a second instance of minetest 18:34 est31 one server, other one client 18:34 est31 to test sth 18:35 * est31 doesnt want to know what Krock has 18:35 * Krock tells est31 that he has 2 GiB ram - more than a RPI 18:36 hmmmm celeron used to use a really underpowered laptop iirc 18:36 celeron55 of course just lowering the range gets the memory consumption down while still being useful 18:36 Krock RAM can be extended using the harddisk. The problem is mostly the CPU 18:37 est31 is it server RAM or client RAM? 18:37 hmmmm est31: https://github.com/kwolekr/minetest/commit/21287bc2cd0bdead0d62c05c79de407abd5fc0ed 18:37 celeron55 client 18:37 est31 hrmm I guess it creates a copy of all the textures? 18:37 est31 what if you try one of VE's high res texture packs 18:38 est31 https://forum.minetest.net/viewtopic.php?t=1583 18:38 hmmmm celeron55: please, please, please work on reducing memory usage before calling it "done" 18:38 celeron55 hmmmm: you can't; it's just how much memory that much terrain geometry consumes 18:39 celeron55 have you ever looked at how much memory modern PC games use 18:39 hmmmm i don't want another situation of "oh it just takes up 100% cpu usage because AMD cpus don't work well with threads" 18:39 celeron55 large meshes are large; it's just how it is 18:40 hmmmm you can try constructing the meshes more efficiently 18:40 celeron55 but there are a few options i can add that trade RAM to more CPU time 18:40 hmmmm or optimize them with weldMesh() 18:40 hmmmm for mostly static meshes this ^ might be a good choice 18:40 hmmmm especially if you're constructing it in a separate thread 18:40 celeron55 what weldMesh 18:41 celeron55 are you insane; i might just as well stop right now if i was to do evrything you think is needed 18:41 hmmmm in my irrlicht terrain viewer i implemented a basic minetest-like block renderer and welding the meshes reduces vertices by half 18:41 celeron55 what i will do is make a stable feature 18:41 celeron55 just don't use the feature while you think it's not optimized enough 18:42 celeron55 i'm already proud of this and you should be too 18:42 est31 from the screenshots it looks really great 18:43 hmmmm yea I guess 18:43 hmmmm the nodes look really... line-y for some reason but I can live with that 18:43 celeron55 it's a good way of cheating the eye of more detail than there actually is 18:44 celeron55 it's what makes it look even remotely reasonable 18:44 hmmmm yeah agreed 18:44 hmmmm this is soo much better than the old farmesh 18:46 est31 side note, it mostly has worked with the protocol mismatch stuff 18:47 est31 most of it ... makes sense. 18:59 est31 woow only 10 commits to review 19:13 Krock Not much ^^ 19:16 RealBadAngel that farmesh mountain could look quite nice if it was blurred somehow 19:16 hmmmm shaders 19:18 RealBadAngel https://imgrush.com/X6Zh5Z_DBJ_X.jpg 19:18 RealBadAngel with a simple blur it does look nice 19:20 est31 what the hell 19:21 est31 somebody has changed the translation of "If disabled " from "Отключить мультиплеер" to "Если выключено_" 19:21 est31 now the old version is BS 19:21 est31 automatic BS 19:21 est31 it means "disable multiplayer" 19:21 est31 but neither microsoft translator nor google translator can cope with the new version 19:22 est31 and I really wonder about that last char 19:22 est31 its displayed as _ (underscore) for me in gitk and everywhere else 19:22 RealBadAngel Отключить мультиплеер == enable multiplayer 19:22 est31 but kate displays it at space 19:22 RealBadAngel 2nd string means "if disabled" 19:23 est31 hrmm, the problem seems to be that last char 19:23 RealBadAngel underscore here whatever it is is wrong 19:23 est31 will see whether its that way in the final version as well, and remove it if its there 19:44 est31 well, then lets push it all 20:19 est31 PTAL https://github.com/est31/minetest/commit/9a91f58e66e84bfba13c967753d537996e4c5d6d 20:20 PilzAdam +1 20:20 PilzAdam (yes, I just reviewed that in 14 seconds) 20:20 est31 xD 20:26 est31 hrmm I should probably have squashed those PilzAdam commits that set whether fallback font is enabled 20:26 est31 well shrug, commit spam stays commit spam 20:26 est31 to finish it, I push two additional commits 20:26 PilzAdam my commit count doesn't complain either ;-) 20:27 est31 Congrats to commit #200 PilzAdam 20:53 hmmmm ridiculous 20:53 hmmmm that spams the commit history 20:53 hmmmm i think it should've been squashed 20:55 est31 all translations? 20:55 hmmmm yes 20:56 est31 this way we have working git blame 20:56 hmmmm if attribution is the problem with that, we can individually list each contributor 20:56 hmmmm btw what was your final word on the LuaSecureRandom 20:56 est31 +1 20:56 hmmmm alright pushing then 21:22 hmmmm PTAL https://github.com/kwolekr/minetest/commit/a0333be25c71a9f454ac0fef3a74f5711624d1e8 21:25 est31 can you add a comment why NULL isnt used here : https://github.com/kwolekr/minetest/commit/a0333be25c71a9f454ac0fef3a74f5711624d1e8#diff-3465219267c4baa9779cd47acef1bf20R267 21:25 est31 otherwise sb will change it back one day thinking 0 is bad 21:25 hmmmm bleh 21:26 est31 and about the is.gcount() check, what if gcount is larger than u32? 21:27 est31 the cast will cut off the higher bits, no? 21:27 hmmmm it will, and then it won't be equal to s_size 21:27 hmmmm throwing the exception 21:28 hmmmm besides, the max long string length is 4gb 21:28 est31 well but what if the gcount is 2^33 + 42 and s_size is 42 21:28 hmmmm then i would have to argue that your platform's std::istream implementation is horrendously broken 21:29 est31 ? 21:29 est31 what if the client actually sends such a large packet 21:29 est31 just to be evil or so 21:30 hmmmm gcount is the number of characters read from the *last* operation 21:31 hmmmm if you request 42 bytes to be read, but istream actually reads 0x10000000 + 42 characters... well then 21:31 est31 ah 21:31 est31 ok nvm then 21:31 est31 +1 21:31 hmmmm you'd have to have a packet that's actually that many characters 21:32 * est31 should RTFM 21:32 est31 before talking 21:32 hmmmm and you'd also have to have a broken std::istream implementation that has a literal bug 21:34 est31 yea, your commit looks ok 21:46 est31 PTAL https://github.com/est31/minetest/commit/92b8b38883cb9cf88bf813b47a07629760e0bb83 21:47 est31 new version https://github.com/est31/minetest/commit/8afb8b3a75d58278628ad4fb5826bd4235d3ffb0 21:48 hmmmm yup, looks good 21:50 est31 ah 21:50 est31 stdout is text mode right? 21:50 hmmmm yes? 21:50 est31 so fputs(stdout, "\n"); will write \r\n on windows? 21:51 hmmmm maybe it does 21:51 hmmmm it doesn't matter because lua does the same thing 21:51 hmmmm the goal is to make this equivalent to what lua did 21:51 est31 well yeah 21:52 est31 I use std::cout here 21:52 est31 so the question is, what is the analogon for fputs(stdout, "\n") 21:52 est31 is it std::cout << std::endl or std::cout << '\n'. 21:53 est31 in the worst case, I just can use fputs as well 21:53 est31 then its even more the same 21:53 hmmmm the hell is wrong with exceptions now 21:54 hmmmm no exceptions display the error message 21:55 est31 I'll usestd::endl and hope the best 22:02 est31 ? 22:07 nrzkt std::endl is perfect for cross platform ending lines 22:11 celeron55 the code of VoxelManipulator looks ridiculous 22:11 celeron55 it has comments that are downright wrong, and commented-out code that has nothing to do with how minetest works these days 22:13 celeron55 straight from 2010 22:14 hmmmm it works so there's no reason to fix it 22:14 hmmmm unless you're SN 22:14 celeron55 i might remove the misleading comments some day 22:14 celeron55 but otherwise who cares 22:16 celeron55 it's just funny how this was a blazing new feature a year or two ago while nothing has changed at any point in its 5-year lifespan 22:17 celeron55 except for the lua interface 8) 22:18 hmmmm how plausible is it to decouple objects from mapblocks? 22:18 celeron55 decouple into where 22:19 hmmmm another table in the map db, or maybe a new db 22:19 hmmmm sqlite can easily perform efficient range queries; i'm not so sure about leveldb and others however 22:19 celeron55 it's possible but you still need some kind of a spatial database for storing them that mapblocks already essentially are 22:20 celeron55 you kind of need a reason for it 22:20 hmmmm exactly 22:20 hmmmm if objects are decoupled from the map, that's one step closer to locklessness 22:20 hmmmm it's also greatly simplify the whole static object storage 22:21 hmmmm thinking that it might be a good idea to separate a map lock from an object lock 22:21 celeron55 what about copying blocks with objects included 22:21 celeron55 does anyone do that in any form 22:21 celeron55 currently it's kind of trivial at least in theory 22:22 hmmmm it's not because of like i said, leveldb 22:23 JohnnyComeL8ly Well, hmmmm, I thought the default was sqlite. 22:24 celeron55 sqlite has never failed me and i will use it until the end of time for the exact reason 22:25 paramat nore PilzAdam sfan5 i would be interested in your comments on https://github.com/minetest/minetest_game/pull/719 Default, fire: Disable catch-up in some ABMs 22:26 hmmmm well 22:26 hmmmm maybe it's not very advantageous to decouple objects from mapblocks since you're going to need to query the map for collision detection 22:27 hmmmm i'm trying to think what objects spend most of their time doing 22:27 est31 objecting what else 22:28 est31 "I don't want to work properly" 22:28 est31 I want 1 s more lag! 22:28 hmmmm we should stop objectifying them 22:29 hmmmm SAOs have feelings too you know 22:29 est31 the moment they object to being objectified they prove they are just objects 22:29 hmmmm std::vector m_feels; 22:29 celeron55 you have to stab them dead while the server is running a lua map generator; then they don't feel pain 22:30 hmmmm >tfw i get deleted because my mapblock has too many static objects 22:30 hmmmm >tfw no player wants to pick me up 22:30 hmmmm feels bad man 22:30 celeron55 meme overload 22:31 celeron55 or maybe virtual meme, because we don't use java 22:31 hmmmm dank memes 22:31 celeron55 maybe i need to sleep 22:31 hmmmm same 22:31 hmmmm later. 22:32 celeron55 wait, that was actually a really bad joke because what i was thinking was override, not overload 22:40 JohnnyComeL8ly celeron55, needs sleep... sweet dreams to him! 22:41 paramat +1 for removing auto view range 23:00 est31 btw I can make a native alternative to /clearobjects which works without anything loaded but the database. It can run in 20 minutes on a VanessaE server , which is much faster than the normal /clearobjects 23:00 est31 I could make a patch, interested?? 23:08 paramat i'm sure that will be very useful 23:14 hmmmm back 23:15 hmmmm PTAL https://github.com/kwolekr/minetest/commit/fe8b4d78e404f393f5c215fd7db121177340b250 23:15 hmmmm whoops hold on 23:16 hmmmm okay PTAL https://github.com/kwolekr/minetest/commit/88a3977954f537b391c714be2a52da6b19711154 23:21 est31 is there no thread safe way of doing it 23:22 est31 ah 23:22 est31 nvm 23:22 est31 "" 23:22 est31 The WSAGetLastError function returns the last error that occurred for the calling thread. 23:25 est31 you aren't including errno.h 23:25 hmmmm https://github.com/kwolekr/minetest/commit/88a3977954f537b391c714be2a52da6b19711154#diff-d999b22f05a7d6687dee3151d4552f0aL26 23:28 est31 other thing: ah 23:28 est31 ermm 23:28 est31 ah 23:28 est31 other thing: if user comes on the forums "creating socket failed : error number 123" 23:28 est31 how will we debug it 23:28 est31 is it perhaps an idea to append whether its unix or windows? 23:29 hmmmm nah that's something they can tell us 23:29 hmmmm also version is reported in the logs 23:29 est31 well, it requires one more back and forth 23:29 hmmmm and besides, windows network error codes are that of unix, + 10000 23:29 hmmmm so you can tell right away what platform this is 23:29 est31 ah okay then 23:30 est31 is there a thread safe way to access the errno? 23:30 hmmmm errno is local to the running thread... 23:30 est31 is it? isn't it a global variable? 23:30 hmmmm nope 23:31 est31 how would that work 23:31 est31 virtual memory? 23:31 hmmmm thread local storage 23:31 est31 ah I see 23:32 est31 lol errno is even used as example on the TLS wiki page 23:32 est31 well +1 then 23:37 paramat i broke snowy pines last night, will fix very soon =s