Time Nick Message 00:51 hmmmm paramat, the positional format can only support lacunarity 00:56 paramat okay thanks, guess i'll add that into the positional examples 03:19 paramat hmmmm https://github.com/minetest/minetest/pull/2001 is ready for review 03:30 gregorycu Yay! I contributed! 03:49 paramat can anyone confirm i get the value of a node's leaves group with 'ndef->get(c).groups[leaves]'? 05:17 hmmmm doesn't look like it 05:17 hmmmm i would highly recommend against hard coding crap like that in your mapgen 05:18 hmmmm what do you mean by "value" of a node's leaves group 05:18 hmmmm without looking at the source, iirc groups is a std::map from group names to a std::vector of content_ts 05:19 hmmmm so that's your value, a std::vector 05:19 hmmmm you'd get it by std::vector c_leaves; ndef->getIds("leaves", c_leaves); 05:37 hmmmm actually, getIds works with a std::set, not a vector 05:46 hmmmm you know something, VoxelManipulator flags are worthless 05:46 hmmmm they have a theoretical use of marking specific nodes as being set during some process or whatever except that's invalidated between the lifecycle of VoxelManipulator objects 06:48 gregorycu So, I've been getting this annoying behaviour 06:48 gregorycu Decided to take a look 06:48 gregorycu Basically, it involves terrain fading in, and then disappearing, then fading in again, then disappearing... etc. 06:49 gregorycu Is this a known issue? 06:59 gregorycu Would be dandy if Zeno came on 07:25 gregorycu VanessaE: So, I've figured out how to reduce that jitter, when terrain fades in and disappears 07:25 gregorycu I believe you said that you experience this issue from time to time 07:28 gregorycu The person who caused this is some joker called Perttu Ahola 07:28 gregorycu This is a regression 07:29 Calinou feel free to set your view range to a fixed one, gregorycu 07:29 Calinou viewing_range_nodes_min = N 07:29 Calinou viewing_range_nodes_max = N 07:29 Calinou where N is a number, probably between 60 and 150 07:29 gregorycu Yep, I saw I could do that 07:30 Calinou I wish view range was changeable in GUI 07:30 gregorycu I'm just wondering why other people don 07:30 gregorycu 't have this problem 07:32 gregorycu I think this is related to the mesh processing threads dumping a shit load of processed meshes onto the GUI thread 07:32 gregorycu The GUI thread suffers a framerate hit, which in turn, drops the range 07:32 gregorycu But the framerate takes a hit for a single frame only, so it go back to normal, and with it the view range 07:33 gregorycu Until the next dump of processed meshes 08:58 gregorycu It's real fun when you optimise something and it's slower 09:10 Krock gregorycu, I agree. The mapblocks show/disappear too sensitive, it would be better to "smooth" it 09:53 gregorycu I have by tweaking a value 10:29 celeron55 gregorycu: it depends on the details of your system's performance; some systems seem to have bad loading times for new meshes 10:30 celeron55 i assume you are talking about the rendering distance tuner; it's intentionally as fast as possible because if it's not, then it causes other issues, namely long FPS drops when you turn to look at something rendering-heavy 10:31 celeron55 when you combine unusually slow processing of something in the main thread due to system details with that, you can get what you describe 10:31 celeron55 not sure if this is your case but it can be 10:32 celeron55 also, one notable thing is that in irrlicht, the mesh processing delay does not occur at the moment when the mesh is created; it occurs when it is first rendered, so that's quite obscure too (at least on my old desktop with a sandy bridge GPU) 10:34 celeron55 whatever you do, please don't just tweak the tuner to fit your system and then say it's perfect because then it's likely worse on other systems 8) 10:36 gregorycu Sorry, was handling a scam call on my phone, was awesome, 45 minutes 10:36 gregorycu Um... yeah, I can see you tweaked the value, which is cool and dandy 10:37 gregorycu I think I'm running into issues because I optimised the Mesh thread, or whatever it is 10:38 gregorycu The MeshUpdateThread 10:38 gregorycu So basically, these meshes are arriving really fast 10:38 gregorycu Anyway, I'm trying to see if there is a nice solution which doesn't involve tweaking that value 10:39 celeron55 you're going to have to rate-limit it per frame on systems that have bad mesh creation performance 10:39 celeron55 the fact that the tuner is freaking out tells me that you are getting bad frametime delays too 10:39 celeron55 which won't look good 10:39 celeron55 nor feel good 10:40 celeron55 i.e. bad fps spikes 10:41 celeron55 open the debug view's graphs (F5) and see how it looks, i'd like to see a screenshot too 10:47 gregorycu Will do 10:48 celeron55 (possibly a way to fix the tuner for this case would be to make its response speed be dependent on the camera's rotational speed) 10:48 celeron55 so that if you're turning towards something rendering-heavy, it will work like before and prevent a huge fps drop, but if you're standing still and meshes are being loaded on your system, it won't try to compensate) 10:48 celeron55 +( 10:49 celeron55 (or something like that) 10:50 celeron55 but you're still going to get the fps spikes; you probably won't like them at all even if the tuner isn't freaking out from them 10:50 celeron55 so i would suggest starting from those instead 10:53 gregorycu So, my concern is, what happens when the MeshUpdateThread has shit loads of meshes, the main loop will get saturated with them 10:54 gregorycu Do you catch what I mean? 10:55 celeron55 of course i do; i've hurt main brain more than a fair share with this same issue 10:55 celeron55 my* 10:55 celeron55 lol 10:55 celeron55 due to this exact reason, the client acknowledges blocks to the server only after creating the mesh for them 10:56 gregorycu Yep 10:56 celeron55 it's silly, but it has to be done 10:56 gregorycu Nah, that makes sense 10:56 gregorycu I added a limit to the number of meshes that could be processed by the client at once 10:56 gregorycu I think it was 8 10:57 gregorycu And that mitigated most of the problem, but it's a very arbitrary limit, not based on anything apart from some number I made up 10:57 celeron55 one some systems even one mesh per frame is terrible; on those systems minecraft (yes, minecraft) is so choppy you can't practically play it at all 10:57 celeron55 even while they could get good fps if meshes weren't being created 10:58 celeron55 i guess the GPU designers thought that games load their meshes when they load up, not when you're playing 11:00 celeron55 gregorycu: additionally, if a server has a high max_simultaneous_block_sends_per_client (much higher than the rate limit per frame), there will be a noticeable delay in mesh generation 11:01 celeron55 while it does increase network perforamnce over laggy connections 11:01 gregorycu Yeah, I can obviously optimise whatever I can in app code land with regards to mesh loading, but if the bottleneck is on the GPU or inside irrlicht, well, I'm out of my league. I don't do graphics. 11:01 gregorycu Interesting 11:01 celeron55 i strongly think it's the GPU's fault; irrlicht such a thin wrapper over opengl 11:01 celeron55 +is 11:02 celeron55 some systems can take almost infinite meshes per frame 11:03 celeron55 of course the "real" way to solve this would be asynchronous loading of stuff to the GPU, but it can't be done with older opengl versions (i don't even know if it's possible with new ones? maybe it works only with newest direct3d) 11:04 celeron55 and probably forever impossible with irrlicht 11:04 gregorycu For the record, I have a Radeon HD 5770 11:04 gregorycu I'll get you a screenie, one sec 11:05 celeron55 there's a chance that this would be an irrlicht problem, but given that minecraft, which uses pretty much raw opengl, has the same issues on same systems, it doesn't seem to be 11:07 gregorycu My issue with minecraft was related, but different, I couldn't receive enough chunks fast enough before the server booted me cause I was falling too far behind 11:07 gregorycu But that was cause of TCP retransmits 11:07 gregorycu (Worked fine on my local network) 11:14 gregorycu http://s4.postimg.org/qo6cmitml/perf.jpg 11:15 Krock gregorycu, interesting. I've got a drawtime of 3 - 6 but I'm stuck at maximal 30 fps 11:15 gregorycu celeron55: There you go 11:16 gregorycu I've sent my frame limit quite high 11:16 gregorycu Is yours set to 30 fps? 11:16 Krock Yes but removing it does not change anything 11:17 Krock well, sometimes I might have fps spikes of 35 fps but more not 11:17 gregorycu Do you know how to get the graphs up? 11:17 gregorycu Can you give me a similar screenshot? 11:17 Krock yeah sure 11:18 gregorycu I'll be back in 3 min 11:20 Krock Lol. I had #fps_max 2 times in my minetest.conf *headdesks* 11:21 gregorycu Someone needs a cuddle now :) 11:21 Krock Disabling the graphs results speeds up draw time by 2 11:22 celeron55 gregorycu: lol that's terrible 11:22 gregorycu I'm not sure what appalls you 11:22 Krock http://i.imgur.com/29qYGbR.png 11:22 celeron55 it's terrible what the graphs show of your system's mesh loading performance 11:23 celeron55 throw it in the trash, i wouldn't want to try playing minetest on that :P 11:23 celeron55 Krock: you should take the screenshsot when meshes are being loaded 11:23 Krock celeron55, = when mapblock load? 11:23 celeron55 yes 11:23 Krock oke 11:24 gregorycu It's a FX 4100 quad core, HD 5770, 8 GB RAM 11:24 celeron55 http://i.imgur.com/XVWHxk8.jpg 11:24 celeron55 here's mine 11:24 celeron55 16 meshes per frame with no effect on frametime whatsoever 11:25 celeron55 intel HD4000 or something like that 11:25 gregorycu I'm doing twice that... 11:25 gregorycu 32 11:25 celeron55 well it wouldn't have a difference either 11:26 Krock http://i.imgur.com/d3zAEj4.png 11:26 Krock sleeps stop when meshes load 11:26 celeron55 gregorycu: well, also, i'm doing almost twice the fps so the loading speed is about the same 11:27 celeron55 looks like krock's system performs somewhere in the middle of these other two 11:27 Krock ..and I'm on a single core CPU 11:28 celeron55 oh, that might mean that your bottleneck is the CPU and not the GPU like for us 11:28 celeron55 i.e. the mesh generation is using up the spare CPU time 11:28 Krock yes. Minetest uses up to 98% CPU 11:28 Krock when loading 11:29 gregorycu hmmm. 11:29 Krock But well. A ATI Radeon X300 doesn't have much power anyway 11:29 celeron55 gregorycu: have you updated your video drivers and tried both, opengl and direct3d? 11:30 celeron55 because those could make a massive difference 11:30 gregorycu I've not tried direct3d to be honest, I'll give that a crack 11:30 gregorycu I should be pretty up to date 11:31 gregorycu It's about the same 11:32 gregorycu I'll cap the number of mesh loads to 16 11:32 gregorycu See what it looks like 11:33 gregorycu There is less jitter 11:36 chchjesus Minetest uses Direct3D now as well? 11:36 * Krock loves time_speed = 12000 11:36 celeron55 it has always supported d3d 11:36 Krock *irrlicht 11:36 Krock minetest uses irrlicht and irrlichts supports it since loong time 11:38 chchjesus Oh that's right. 11:38 chchjesus I forgot. 12:03 gregorycu celeron55: So, you believe my problems stem from a graphics cards that struggles to load meshes on the fly? 12:05 celeron55 http://fpaste.org/162008/14192499/ 12:08 celeron55 this explains somewhat why MT can perform extremely badly on some hardware and why there's no easy solution to it 12:09 celeron55 and why a modern integrated GPU works best 12:10 Krock so it depends on the memory allocator speed? 12:10 celeron55 no, that issue is not relevant today 12:12 celeron55 the issue is that GPUs simply aren't designed for handling so many small buffer objects, and many of them pretty much only accidentally are fast at it 12:13 celeron55 (frankly which is what has really already been obvious) 12:13 Krock :/ 12:13 gregorycu Can you string together lots of small ones and somehow draw part of a buffer object? 12:14 celeron55 (the only solution to that is using larger ones, and irrlicht is quite inflexible in doing that) 12:14 chrisf gregorycu: absolutely, but your middleware has to know how to 12:14 celeron55 that's the best you can attempt with irrlicht 12:14 gregorycu Wow, it's the guy in question 12:14 chrisf yeah i lurk here :) 12:15 gregorycu I have only fleeting experience in graphics, my day-to-day job is systems programming 12:15 gregorycu Well, I suppose it's closer to app programming 12:15 celeron55 then we have the pain in the ass problem of supporting old systems that are way more limited by vertices per frame than objects per frame 12:16 celeron55 so we need quite a flexible system 12:17 chrisf celeron55: incidentally, if you wanted to get async uploads happening, MapBufferRange is a good place to start 12:17 celeron55 yeah, and before that can be used, irrlicht must be replaced 12:18 celeron55 good luck people, lol 12:18 chrisf is the engine you use for your other project more helpful for this? 12:19 celeron55 i don't even know; but buildat has better texture atlasing and it uses much larger buffers to begin with so it's way less affected 12:20 celeron55 it doesn't aim to work on such low-end hardware that MT does 12:21 celeron55 but my butt feeling is that all of these irrlicht/ogre3d/urho3d/panda3d/whatever style libraries are bad at dynamic loading and dynamic combining of buffers 12:21 celeron55 because you can't do it at the abstraction level that they work with 12:22 celeron55 you either need a higher level with much cleverness in the library, or lower level where you do it by hand 12:24 celeron55 the first one being something like unity3d which is quite limiting and the second one being basically raw opengl 12:24 celeron55 which is tedious 12:24 sfan5 hm https://cdn.mediacru.sh/J/JOejP0IHfgWy.png 12:25 celeron55 sfan5: looks similar to my nvidia GPU (if i enable it) 12:25 gregorycu Damn it sfan5 12:25 gregorycu I want your box 12:28 sfan5 celeron55: it looks about the same with my nvidia gpu: https://cdn.mediacru.sh/D/DHFu1A0TzBAq.png 12:29 celeron55 that's pretty much what happens on the average mid-range box that has no major bottlenecks; it's too slow but tolerable 12:29 sfan5 gregorycu: it's a 3-year-old middle-end laptop (i5, GT 520M) 12:33 gregorycu Hmm... 12:33 gregorycu It's like there needs to be a system-wide "struggle-factor" 12:34 gregorycu The more your system is struggling, the lower your view distance, the slower the meshes are loaded, etc. 12:34 gregorycu View distance is only one thing that can be lowered 13:10 celeron55 gregorycu: generally nothing else than view distance has a considerable performance impact though 13:11 gregorycu number of meshes loaded a second 13:11 celeron55 i have a very old laptop where the mesh loading rate has no effect whatsoever 13:11 gregorycu Not sure if you have others 13:11 gregorycu I have a somewhat new desktop where it does :P 13:12 celeron55 i guess they should be adjusted separately 13:39 luizrpgluiz hi 13:40 luizrpgluiz celeron55: because you left the minetest the project? 13:41 celeron55 ? 13:42 luizrpgluiz celeron55: I thought you had left the minetest the project, he has a great future :) 13:43 celeron55 he? i'm not sure i understand but no, i have never left minetest, i'm just rather inactive 13:49 luizrpgluiz celeron55: you are a professional programmer? 13:51 celeron55 i guess i could say i am 13:52 luizrpgluiz celeron55: where did the inspiration to create the game? 13:55 luizrpgluiz celeron55: it came from the minecraft is not it? 14:01 celeron55 i started making minetest after getting a bit bored with the minecraft alpha version at the time, mostly to see if i could do it at all 14:02 celeron55 turned out i could 14:09 luizrpgluiz celeron55: I currently do not know anything about programming, year in my small town here in Brazil, will have a course to make computer games and perhaps to make games for android, I hope to learn to program codes and not work with engines. 14:14 luizrpgluiz celeron55: I hope that the course focus on major programming languages that is used in games 14:15 celeron55 you have a long way ahead 8) 14:25 Megaf [12:13] (the only solution to that is using larger ones, and irrlicht is quite inflexible in doing that) 14:25 Megaf [12:13] that's the best you can attempt with irrlicht 14:26 Megaf there you go, yet another example where Irrlicht limits Minetest 14:26 kahrl lamenting about that fact is not useful :P 14:28 celeron55 has hmmmm said that he will take care of declaring the release? 14:28 celeron55 or is it up to someone else 14:31 kahrl I don't remember any statement to such effect, but I've not read all the logs 14:39 celeron55 i bet nobody is going to do it 14:39 celeron55 and everyone who would build windows binaries are on holiday 14:40 kahrl sfan5 said he'd do that if BlockMen doesn't miraculously appear 14:41 Krock *cough* 15:00 luizrpgluiz celeron55: use the Ogre engine would be better than using other graphics engine? 15:32 Megaf luizrpgluiz: No, it wouldn't be better. Ittlicht is actually very good indeed. But it has some bugs and limitations. 17:34 luizrpgluiz which command I use to open a server with sub game in windows? 17:35 Calinou --game gameid 17:35 Calinou where gameid is the game name 17:39 Megaf dont use windows :) 17:40 kilbith luizrpgluiz: that kind of question in #minetest - thanks. 17:41 Calinou Megaf, … 17:42 Calinou we support it 17:47 luizrpgluiz :) 20:41 luizrpgluiz someone had many ideas to put in the 0.5 minetest? 20:44 hmmmm luzrpgluiz, perhaps your questions are better suited to #minetest 22:34 hmmmm sorta dead today... 22:35 shadowzone Yep 22:35 shadowzone Tomorrow it won't be 22:35 shadowzone I am pretty sure of that 22:35 hmmmm lol http://i.imgur.com/RQrsscP.jpg cool pattern generated by reading out-of-bounds memory 22:49 hmmmm hrm 23:06 Zeno` Hi. I'd like to disable dropping of items from liquid reflow queue until issues such as #2003 can be looked at in more detail. This will reintroduce the RAM leak but... 23:06 ShadowBot https://github.com/minetest/minetest/issues/2003 -- Liquid doesnt update on generation. 23:07 Zeno` see #2005 23:07 ShadowBot https://github.com/minetest/minetest/issues/2005 -- Make limiting of the reflow liquids queue size optional by Zeno- 23:09 Zeno` Also, related to #2000 that we were discussing the other day, struct MapNode *is* POD-type which is why everything (including the existing memcpy etc) works 23:09 ShadowBot https://github.com/minetest/minetest/issues/2000 -- Large increase in performance by Zeno- 23:10 kahrl yeah, mapgen adds *every* generated liquid node to the queue 23:10 kahrl which, as you might imagine, is a lot... 23:11 Zeno` kahrl, it's a @#&@ load :) 23:11 kahrl they are handled quickly (faster than a source that has just been placed with a bucket, say) but confuse the limiter 23:12 Zeno` 10000 seems to be the problem. That value cannot keep up with the increasing queue size since optimisations introduced. But maybe leaking RAM is the lesser of two evils at this point (with release so close) 23:12 Zeno` There is probably a better way to handle this anyway 23:13 kahrl I haven't really had time to optimize transformLiquids as I promised :/ 23:13 Zeno` I'm not sure when 10000 was introduced (max iterations per loop) or how that number was determined, but maybe increasing that will mitigate most RAM leaks server operators seems to be experiencing 23:13 kahrl but I grabbed the low-hanging fruit, which is all the nodedef->get calls 23:14 kahrl will push that in a sec 23:14 kahrl (to my repo) 23:14 Zeno` kahrl, it's too close to Christmas... I would have looked at this RAM issue in more detail but there's too much going on IRL heh 23:14 kahrl true true :) 23:14 celeron55 Zeno`: why doesn't your dynamically scaling smoothing code handle it? 23:15 celeron55 it was supposed to do it, you know? 8) 23:16 Zeno` celeron55, well it does but I just timed it (without the smoothing) and when a whole bunch of areas are generated at once it takes up to 4 minutes for my computer to catch up. With the dumping set to less than that then thing were being left out 23:16 Zeno` celeron55, if you'd like me to say you were right, you were right :P 23:16 Zeno` but I dunno if removing the 10000 is a good idea either 23:17 celeron55 well 23:17 Zeno` I have no idea what the 10000 means... I'm setting it higher on my server for now 23:17 celeron55 i suggest increasing the value to 100000 and hoping that it handles the issue 23:17 celeron55 right in this release 23:18 celeron55 we don't want liquids to be broken; no way 23:18 Zeno` set it to 100000 now? 23:18 celeron55 it's just about high enough to allow handling them in common cases afaik 23:19 Zeno` I'll see where max iterations per loop was added 23:19 Zeno` (i.e. was it in 0.4.10?) 23:20 celeron55 i don't think you'll find anything useful 23:20 celeron55 just do it 23:20 celeron55 it's a value that happened to work back then for whatever reason 23:22 Zeno` so just change it to 100000? Ok. I'll add that into the PR 23:25 kahrl https://github.com/kahrl/minetest/commit/17d6a4355afcc5c452e4ddd695715bcdc97cabaf 23:25 kahrl (this has some debug code in it that should be removed when pushed upstream) 23:26 Zeno` Ok, pushed. For those server operator having RAM problems liquid_queue_purge_time can still be set anyway so everyone should be happy 23:31 Zeno` a new TimeTaker :) 23:32 kahrl it would probably be better to add a PRECISION_NANO_CPU instead... not sure 23:32 kahrl less code duplication though 23:33 Zeno` maybe 23:33 Zeno` so the other stuff aims at making the loop faster I'm taking it 23:33 kahrl yep 23:33 celeron55 how much faster it is, roughly? 23:33 Zeno` That's always good 23:33 kahrl I haven't had time to profile it 23:35 kahrl Some stuff might not be worth it, e.g. the extra check for mapgen-generated source nodes 23:35 kahrl so take this with a grain of salt 23:37 Zeno` I use rock salt 23:41 Zeno` 2000 is 40% faster for VoxelManipator::addArea() 23:42 Zeno` (~5% faster compared to whole program) 23:45 paramat #2001 needs editing so don't push it yet 23:45 ShadowBot https://github.com/minetest/minetest/issues/2001 -- Update mapgen stuff in minetest.conf by paramat 23:45 Zeno` paramat, should it be labelled WIP? 23:46 paramat yes i will change the comment 23:46 celeron55 easiest way to just put WIP in the title 23:46 paramat ok 23:52 paramat done 23:55 hmmmm [06:09 PM] Also, related to #2000 that we were discussing the other day, struct MapNode *is* POD-type which is why everything (including the existing memcpy etc) works 23:55 ShadowBot https://github.com/minetest/minetest/issues/2000 -- Large increase in performance by Zeno- 23:55 hmmmm are you sure? 23:55 hmmmm MapNode has a user-defined constructor, therefore it's non-POD I thought 23:55 hmmmm it would be POD under C++11... but this isn't C++11 23:56 celeron55 i guess it's POD in practice 23:57 celeron55 and C++11 is just making it officially so 23:57 celeron55 C++03 doesn't define anything that would force it to be handled in a non-POD way, right? 23:57 hmmmm you mean memory layout? no 23:58 hmmmm Zeno`: That's an awesome optimization for such a stupidly simple thing you'd think would be optimized by the compiler 23:59 hmmmm the reality is that the compiler isn't as smart as you would hope for it to be... you write some code and you always give the compiler the benefit of the doubt - "oh it doesn't matter, that'll get optimized anyway"