Time Nick Message 00:19 paramat i decided to go ahead with #2720 will push very soon. note i am changing humidity octaves to 3 and persistence to 0.5 for fewer tiny jungle biome bubbles 00:19 ShadowBot https://github.com/minetest/minetest/issues/2720 -- Mgv6: Enable snowbiomes by default. Double biome noise spread. by paramat 01:06 paramat now pushing 01:11 paramat complete 01:19 hmmmm see? biome bubbles are why I prefer minecraft's biome shape 01:21 paramat i don't mind a few, an occasional one is cute. lower octaves and persistence helps to reduce their number 01:22 hmmmm yeah that also makes biomes... square 01:22 hmmmm rather, that predictable off-center blob shape 01:22 hmmmm it's a natural side effect of easeCurve + bilinear interpolation 01:23 paramat that's very noticeable with < 3 octaves 01:23 hmmmm see 01:23 hmmmm this is why I wanted to check out cubic interpolation 01:24 sofar paramat: which mapgen has your changes? v6 ? 01:26 paramat yes 01:27 paramat it has kick-ass big jungles now 01:29 sofar it's creating tunnels underwater? 01:34 paramat that's probably a previous bug 01:34 paramat um.. 'characteristic' 01:35 sofar lol 01:35 paramat i've noticed v6 jungles get very thick and dark sometimes, now they're larger they're a real adventure, instead of pathetic 60 node-wide patches 01:35 * sofar cant find snow 01:36 paramat might need to travel 1kn 01:36 paramat or more 01:48 paramat you need to start a new world and make sure you don't have 'mgv6_spflags = nosnowbiomes' in .conf. the underwater tunnels are i think air-dungeons forming 01:49 sofar oh, that's on by default 01:51 sofar huh 01:51 sofar something put it back, too 01:54 sofar ugh 01:54 sofar something keeps putting the "nosnowbiomes" param back in there 01:54 sofar paramat: ? 01:59 sofar paramat: can't find snow... had a hard time removing nosnowbiomes, and now I get this: 01:59 sofar http://i.imgur.com/BVlwbOX.png 02:00 sofar maybe my minetest_game is old 02:00 * sofar pulls 02:01 sofar ahh that did the trick 02:01 sofar paramat: looks great! 02:05 sofar http://i.imgur.com/LAb2xuT.jpg 02:07 paramat oh yes, need latest MTGame *=/ 05:51 cd2 heyo :D 05:52 est31 hi 15:09 RealBadAngel https://github.com/RealBadAngel/minetest/tree/minimap3 15:09 RealBadAngel if somebody wants to try minimap progress 15:10 RealBadAngel only config option is minimap_shape_round = true/false 15:30 hmmmm hrmm let's see this minimap 15:31 hmmmm a thread 15:31 hmmmm there's a new thread!??! 15:32 hmmmm hrmm I need to take a better look at this but my hope was that this could be done without introducing more client-side contention 15:34 hmmmm there's no locking done at all? 15:35 hmmmm the only way this isn't crashing is through pure luck 15:35 hmmmm also wtf is the "minimap mode"? 15:37 celeron55 umm 15:37 celeron55 i started reading this too and noticed the same thing 15:37 celeron55 there is no locking at all when reading the client's map from a thread 15:39 hmmmm okay, the particulars of the code are a little sloppy but nothing that can't be fixed 15:39 hmmmm is having a separate minimap thread the optimal approach, though? 15:39 hmmmm what if we throw this into the client step 15:39 hmmmm also i personally feel like 10ms is a bit too short of an update time 15:40 celeron55 do you mean 100ms 15:40 hmmmm why? 15:40 celeron55 what's how long it sleeps 15:40 hmmmm it says sleep_ms(10); in minimap.cpp line 73 15:40 celeron55 what's* 15:41 celeron55 https://github.com/RealBadAngel/minetest/blob/minimap3/src/minimap.cpp#L74 15:41 celeron55 i wonder what version you are reading 15:41 hmmmm an older version 15:41 hmmmm he changed it in the commit at HEAD 15:42 hmmmm woops 15:42 hmmmm i dunno, celeron, what do you think about having the minimap in a separate thread? 15:43 celeron55 i think if this was run in the render thread the x*z loop would have to be paused between frames and done in multiple parts that way; i feel it's not going to be (and shouldn't need to be) fast enough to be run within a single frame 15:43 celeron55 if it would be fast enough to run within a single frame, then it should be split so that we can have larger map areas 15:43 hmmmm why, because the nodes could have updates throughout the render thread? 15:44 celeron55 no but because it's a rather heavy operation to find the ground level for tens of thousands or more positions 15:45 hmmmm oh, that is true 15:45 celeron55 frametime jitter is a terrible and embarrassing thing to have, especially if it was due to a thing like this 15:45 hmmmm didn't think of that 15:47 celeron55 i 15:48 celeron55 i'm also not sure at all that these irrlicht texture operations that are being done in the thread are actually thread-safe 15:48 celeron55 the image ones are re-entrant as far as i know (i don't know if they are specified to be, but they likely are), but when you create an image from a texture, that might go on scarier territory 15:48 celeron55 dunno 15:49 hmmmm good to note 15:50 celeron55 it might also be platform dependent or whatever; i'm pretty sure irrlicht was never designed to run at all in multiple threads so it's up to us to figure out what can be done and what cannot 15:50 hmmmm another thing 15:51 hmmmm all of the ground levels are recalculated every getMap() call 15:51 hmmmm since users are most likely walking from one place to another, shouldn't there be a heightmap cache 15:54 celeron55 probably, altough i don't think the performance should be dependent on it 15:54 RealBadAngel hmmm, thread and mapper share their data and trigger events between them, so no crashing possible, its no luck 15:55 celeron55 oh god 15:55 RealBadAngel also whole scan have to be completed each time, theres no fixed ground level for surface scanner 15:56 celeron55 i'm 100% sure that kind of synchronization has subtle bugs in it 15:56 RealBadAngel had, ive eliminated them ;) 15:56 RealBadAngel simply, if scan is not complete draw cannot pick another frame 15:57 celeron55 i wonder if minetest runs cleanly enough in helgrind or drd for checking things like this 15:57 celeron55 it might not 15:57 RealBadAngel anyway i may change it, never touched threads before 16:00 rubenwardy How to use minimap? 16:00 RealBadAngel press f9 16:00 rubenwardy awesome 16:00 RealBadAngel it looks better with shaders ;) 16:01 rubenwardy is that not in minimap3? 16:01 RealBadAngel it is there 16:02 RealBadAngel also in config: minimap_shape_round = true/false 16:04 rubenwardy shape_round also affects the behaviour when rotating, it seems 16:05 RealBadAngel yes, shadows are dynamic then 16:05 RealBadAngel and theres no player marker 16:06 rubenwardy quite weird behaviour underground 16:06 rubenwardy no player marker, and the thing spins, which is cool 16:07 rubenwardy I guess radar is recommended for underground 16:07 RealBadAngel exactly 16:07 RealBadAngel it scans for air nodes and count them, so more air, px on the map is more green 16:08 RealBadAngel i found that works pretty good when digging tunnels 16:08 rubenwardy I'd like to see a player marker on the circle map - just a dot - it's hard to judge exactly where you are 16:08 rubenwardy That was probably badly worked 16:08 RealBadAngel always in the middle 16:08 rubenwardy * worded 16:09 rubenwardy Yeah, but it's human eyes are weird with judges halfs 16:09 rubenwardy It's not required 16:09 RealBadAngel also do notice that marker occupies the middle, it could be hard to see nodes you are placing 16:09 VanessaE player marker on the circular map can be done - as soon as RealBadAngel is able to again use the texture I created ;) 16:10 rubenwardy A pixel 16:10 rubenwardy Yeah, digging is quite cool 16:10 VanessaE a 5x5 crosshair with an empty center pixel. maybe. 16:10 RealBadAngel you can do that with overlay texture 16:11 RealBadAngel so its up for texture pack 16:11 hmmmm RealBadAngel, what do you mean by this [11:56 AM] simply, if scan is not complete draw cannot pick another frame 16:12 RealBadAngel it locks itself, data cannot be grabbed if scan is not done 16:13 rubenwardy Sounds logical. Like a buffer locking, if it's not finished the reader just keeps reading from the old. 16:13 rubenwardy * double buffer 16:13 RealBadAngel scanner work on images 16:13 hmmmm i don't see where this happens 16:13 RealBadAngel not threaded part uses textures 16:14 RealBadAngel first of all, getMap gets copies of parameters so changes during scan doesnt affect scanner 16:14 hmmmm okay, that's fine 16:15 RealBadAngel when its done it changes image_grabbed flag 16:15 RealBadAngel so draw code can pick another frame 16:15 RealBadAngel if theres no new frame ready, draw uses old one 16:15 rubenwardy Would it be possible to hide jolting (ie, on mine when walking the map only moves every 1/10 or a second) by moving where the texture is drawn, if the minimap takes too long? 16:16 hmmmm hrmm 16:16 rubenwardy I probably misunderstand how it's done. 16:16 hmmmm I am not understanding how this works 16:16 RealBadAngel on my box i do get minimap fps like half of overall fps 16:17 RealBadAngel in 256x256 mode 16:17 RealBadAngel in others scanner is faster than 60fps 16:17 hmmmm what thread is Mappger::getMinimapTexture() called in? 16:18 RealBadAngel main 16:18 rubenwardy Is the minimap constantly updating, and the jolting is when it finishes a tick? Or does it sleep? 16:18 rubenwardy Maybe I should just read the code, lol. :P 16:18 hmmmm somehow I wonder if uploading a texture every 100 ms is a good thing 16:19 RealBadAngel huh its done everywhere 16:19 RealBadAngel each frame 16:19 RealBadAngel rendering to textures for example 16:19 hmmmm so getMinimapTexture only swaps the texture if image_grabbed is false 16:20 hmmmm those are cached 16:20 hmmmm image_grabbed is sort of misleading 16:20 hmmmm more like... "minimap_texture_invalidated" 16:20 RealBadAngel could be named better ;) 16:21 RealBadAngel also about thread, only getMap will remain there 16:21 RealBadAngel i already moved getColorFromId from runtime 16:22 RealBadAngel so thread will work only on open images and will create just one new image each run 16:22 hmmmm so I don't understand how the whole image_grabbed thing prevents a race condition 16:25 RealBadAngel if false, it cannot start new scan 16:25 RealBadAngel ooops 16:25 RealBadAngel hold on a sec, i will make a diagram 16:26 hmmmm it doesn't matter, it doesn't need a diagram 16:26 hmmmm the fact remains that any other thread can access map and not respect this image_grabbed variable at all 16:26 hmmmm unless i'm totally mistaken, you're depending upon the order in which things happen inside of the render thread 16:27 celeron55 and likely the duration of things too 16:28 hmmmm this is remarkably fragile code... 16:28 hmmmm RealBadAngel, even if I'm totally wrong, what's the cost of adding uncontended locks around map accesses? 16:28 hmmmm 70 nanoseconds? 16:29 hmmmm the texture locks are a bit more subtle, however... 16:31 RealBadAngel scanner cant make new scan if old one is not used, renderer cannot pick incomplete scan for texture, wheres depending on duration there? 16:31 RealBadAngel theres no such thing in whole code 16:33 RealBadAngel hmmmm, and i dont depend on order of anything 16:33 RealBadAngel if renderer doesnt have new frame it will try next time 16:33 RealBadAngel otherwise old texture is used 16:34 hmmmm I'm talking about map access here 16:34 RealBadAngel here im not sure 16:34 RealBadAngel i do copy blocks to vmanip 16:35 RealBadAngel should i do that under locking? 16:35 hmmmm YES..... 16:35 RealBadAngel so why mapblock mesh updates doesnt do that? 16:35 hmmmm i don't know 16:35 hmmmm but it should 16:35 celeron55 they copy their content to vmanips in the client thread 16:35 celeron55 and then give the vmanips to the mesh thread 16:36 celeron55 you can do the same thing here if you want and if it's fast enough 16:36 hmmmm in any case, image_grab needs to be volatile 16:38 RealBadAngel celeron55, i can move copying to main, no problem 16:41 RealBadAngel hmmmm, that variable is the lock in fact, i can mention bout it in name 16:42 RealBadAngel and i will do cleanin now, theres lotsa dead code in that branch 16:43 RealBadAngel btw, have you guys checked the performance with minimap? 16:43 RealBadAngel how it actually works for you? 16:52 hmmmm also Mapper::setPos() has a potential race condition 16:52 RealBadAngel doesnt have 16:52 RealBadAngel getMap works on copies of parameters 16:53 RealBadAngel on the next run it will get updated pos 16:53 RealBadAngel same applies to mode 16:54 RealBadAngel you can check that when flying fast and 256x256 map 16:54 RealBadAngel scanner stays a little behind the player 16:55 RealBadAngel not much but thats visible (at least on my box) 16:56 hmmmm they might not be updated before they're copied 16:56 hmmmm this is why they need to be volatile 16:57 VanessaE actually it works just fine in practice. 16:57 hmmmm yes, but we are not concerned about "in practice" 16:57 hmmmm we need to worry about what can happen in theory, because what can happen will happen 16:57 VanessaE fair enough 16:59 RealBadAngel if its locked it cant do anything 16:59 RealBadAngel scanner cannot scan, renderer cannot pick incomplete data 16:59 hmmmm RealBadAngel, yes it can 16:59 hmmmm do you not realize that compilers sometimes reorder operations? 17:00 RealBadAngel how they can reorder variable value? 17:00 RealBadAngel its the condition for all the actions here 17:00 hmmmm read about this http://preshing.com/20120515/memory-reordering-caught-in-the-act/ 17:01 RealBadAngel ok 17:01 hmmmm what you think is thread safe really is not 17:01 hmmmm you need to worry about not only atomicity, but also memory access reordering 17:02 hmmmm so you have in one thread 17:02 hmmmm texture = mapper.getMinimapTexture(); .... mapper.setPos(newpos); 17:03 hmmmm image_grabbed = true, so the thread starts to call getMap() 17:03 hmmmm and it copies over the parameters 17:03 hmmmm well it just so happens that newpos didn't completely finish writing to data->pos, it's only half written 17:04 hmmmm so your old position was (200, 0, 400) or something and the new position is (4000, 30, 12) 17:04 hmmmm it only had time to write 2 of the 4 16-bit numbers within that access 17:04 hmmmm so it ends up being, say... (200, 30, 12) 17:05 hmmmm and the same thing with mode too 17:05 hmmmm assuming that accesses to variables of type bool are atomic on our supported platforms is pretty fair 17:05 RealBadAngel hold on a sec 17:05 hmmmm but a v3s16 is NOT ATOMIC on any of our platforms 17:05 hmmmm it simply isn't 17:06 RealBadAngel only thing that can trigger something here is that flag 17:06 RealBadAngel before it gets written nothin can happen 17:06 hmmmm I just explained in great detail how this race condition happens 17:07 hmmmm and then you handwave it away with "but the flag is there!" 17:07 RealBadAngel im just trying to understand hows that possible 17:07 hmmmm read what i wrote again, carefully 17:13 RealBadAngel ok, so it may happen. whats your suggestion to prevent that? 17:13 hmmmm locking 17:28 RealBadAngel hmmmm, any examples? 17:47 RealBadAngel hmmmm, from what i read it seems like converting v3s16 to three atomic ints would do the trick and have lock-free code 18:07 Krock hmmmm, Is it thought to be changed like this? https://github.com/SmallJoker/yappy/commit/1f0b 18:08 Krock (Talking about the memory efficiency improvements)