Time Nick Message 00:18 davexunit hello all. 00:19 davexunit sfan5: ping 01:25 RealBadAngel hi 01:26 RealBadAngel kahrl, here? 01:32 RealBadAngel anyway, im rebasing now meshnodes, PR should be up in a moment 02:06 OldCoder kahrl, I may have found a patch for the RE-SEND problem though the issue is still unexplained 02:15 RealBadAngel https://github.com/RealBadAngel/minetest/commit/8a625de529af79be8ac63f2320576faf8eb326e1 02:16 RealBadAngel that add meshnodes and conversion of nodeboxes on startup 02:17 RealBadAngel atm there are no docs 02:28 kahrl nice! 02:28 RealBadAngel kahrl, no worries about converting nodeboxes and time it takes 02:29 RealBadAngel unnoticeable for minetest_game 02:29 kahrl yeah I saw that in the log 02:29 RealBadAngel whole updateTextures takes 15s in case of dreambuilder 02:30 RealBadAngel which means i added there just a few seconds 02:30 kahrl I'm working on a way to have only one extruded mesh for everything 02:30 kahrl that would cut down that time a lot 02:31 RealBadAngel please make sure that extrude will work on textures with semi transparent pixels 02:32 kahrl well, it doesn't really work currently, does it? 02:32 RealBadAngel alpha trigger in original code had to be changed for that to work 02:33 RealBadAngel i found out that changing it from 1 to 0.5 caused hires textures with some smooth applied to work 02:34 kahrl well the new code has no alpha threshold (it doesn't look at the texture at all, that's how it can work with all of them at once) so it might just work 02:34 RealBadAngel btw, dreambuilder needs a bit more than 2GB, it uses swap a lot ;) 02:34 RealBadAngel kahrl, sounds good 02:35 RealBadAngel VE have some textures that used to make extrude code work for a few minutes 02:36 kahrl I know, that's one problem I wanted to solve 02:45 RealBadAngel kahrl, any comments on my code? 02:47 kahrl haven't looked at it in detail yet 02:47 kahrl for updateTextures you can get rid of the first two parameters since you can get the texture and shader source from a gamedef 02:48 RealBadAngel indeed 02:51 kahrl wouldn't it be better to calculate MapBlock_LightColor(255, getInteriorLight(n, 1, nodedef), f.light_source) once and not for every meshbuffer? 02:52 kahrl and the cast above to scene::SMeshBuffer* seems unneeded 02:55 RealBadAngel a) right, its can be used more than once 02:56 kahrl rotateMeshBy6dFacedir could probably be optimized a little by doing stuff like tmp = x; x = y; y = -tmp; instead of rotateXYBy(90) and so on (which involves converting to radians and calculating sin and cos and some multiplications)... not sure if that's worth the bother 02:57 RealBadAngel about casting im not sure 02:57 RealBadAngel i had to change types all the time from IMesh to SMesh 02:57 RealBadAngel rotate code is old, taken from runtime applying rotations 02:58 kahrl actually the compiler might be optimizing the rotateXYBy stuff fairly well since it can be inlined 02:58 kahrl still it'd probably do extra multiplications due to rounding errors 02:59 kahrl I'm pretty sure the cast in content_mapblock.cpp 02:59 kahrl ...:1722 can be removed 02:59 kahrl any removed cast is a good cast 03:01 kahrl cloneMesh should be a little bit faster if you copy the bounding box from the old mesh instead of recalculating it 03:02 RealBadAngel cast can be removed, done 03:02 kahrl i.e. temp_buf->setBoundingBox(buf->getBoundingBox()) and dst_mesh->setBoundingBox(src_mesh->getBoundingBox()) 03:03 kahrl (I know that could be done in scaleMesh etc. as well with a little bit of extra math) 03:04 kahrl actually s/could/should/ 03:04 kahrl but that's for another time 03:04 RealBadAngel i wanted to apply the scale just once 03:05 RealBadAngel its not needed to apply that again for clones 03:05 RealBadAngel but bboxes will be different when you will rotate the mesh 03:05 RealBadAngel so recalculating them is needed 03:06 kahrl oh 03:06 kahrl then it has to be done in rotateMeshBy6dFacedir 03:09 RealBadAngel or maybe separate function to call when needed? 03:09 RealBadAngel it could be useful in more cases 03:10 kahrl perhaps 03:10 kahrl or maybe instead of translateMesh + scaleMesh + rotate... we could have one function transformMesh that takes a matrix 03:11 RealBadAngel ouch ;) 03:11 kahrl that would also fix what I dislike about rotateMeshBy6dFacedir, which is that it re-checks the axisdir and facedir for every vertex 03:12 RealBadAngel note that is done now only on startup and is fast 03:12 kahrl hmm, ok 03:12 kahrl in that case just add the recalculateBoundingBox calls to rotateMeshBy6dFacedir 03:12 RealBadAngel on my box the speedup is amazing 03:13 kahrl I'd still change cloneMesh so it copies the bounding box instead of recalculating it 03:14 RealBadAngel imho its safer to recalculate it 03:14 kahrl if that's safer then the original mesh had wrong bounding boxes which would be a problem in itself 03:15 RealBadAngel it applies also to nodeboxes and they can come with wrong ones 03:16 kahrl how? 03:16 kahrl IMeshBuffer::append() updates the bounding box 03:17 RealBadAngel ah ok 03:20 RealBadAngel btw, before you will notice that: passing ContentFeatures *f instead of pointer to nodeboxes is needed 03:21 RealBadAngel i will have to take care of nodes that use just one tile for all faces 03:21 RealBadAngel so then one meshbuffer will be made 03:21 kahrl I wondered about that 03:22 kahrl seems fine, but make it const ContentFeatures *f 03:22 RealBadAngel ok 03:22 RealBadAngel it works anyway because append checks for tiles used 03:22 kahrl also the indentation in the whole function looks wrong on github 03:22 RealBadAngel but no point to make extra meshbuffers 03:23 RealBadAngel idk whats wrong with github, in geany its perfectly fine 03:23 kahrl spaces vs. tabs? 03:23 RealBadAngel im not using spaces rather 03:24 kahrl strange 03:24 RealBadAngel ah, one more thing 03:24 RealBadAngel when i started to use timer i noticed one thing 03:24 kahrl maybe it's some secret message from github in WHITESPACE ;) 03:25 RealBadAngel if you have a variable that will be used in a call, its a bit faster if you put that in a call itself 03:26 RealBadAngel initializing it before a call and later use is reasonable only if you will use it more than once 03:26 kahrl I don't think general rules like that apply anymore, compilers are complex beasts 03:26 RealBadAngel maybe but i noticed general speedup in gone thx to such approach 03:27 RealBadAngel *in code 03:28 RealBadAngel thats why content_mapblock code is so compact 03:28 kahrl hmm 03:28 kahrl recalculating the light every iteration doesn't seem beneficial to me 03:29 RealBadAngel i missed the fact its needed for each meshbuffer 03:29 kahrl oh ok 03:30 kahrl still I have doubts about that statistic 03:30 kahrl it shouldn't matter at all to an optimizing compiler if you do f(g()) or int x = g(); f(x); 03:31 RealBadAngel i saw timings like 5-10% faster code with that 03:31 kahrl how did you measure? 03:31 kahrl and how many samples? 03:31 RealBadAngel hold on 03:32 kahrl also TimeTaker is a bit problematic for profiling because it measures real time and not CPU time 03:32 RealBadAngel http://pastebin.com/fDBpEZYZ 03:33 RealBadAngel i just noticed a general pattern, kinda average 03:33 kahrl ah 03:33 kahrl hrm 03:34 RealBadAngel for testing purposed i built a tower to go up like 500 nodes, unloded the world and started again 03:34 kahrl maybe it's below statistical significance, who knows 03:34 RealBadAngel so i had kinda empty world with a few nodes being rendered 03:35 RealBadAngel i believe the times got then were kinda accurate 03:37 kahrl I might try doing some statistics on the weekend 03:38 kahrl let's continue with the code review 03:38 RealBadAngel ok 03:39 kahrl any reason for & over && in nodedef.cpp:831 and :844? 03:41 RealBadAngel i cant think of any 03:41 kahrl also the code inside the ifs is repeated, maybe move it to a single if (f->mesh_ptr[0] && f->param_type_2 == CPT2_FACEDIR) below 03:42 OldCoder For those who are interested, my patch for the RE-SEND RELIABLE problem still seems to be working a full day in. I will discuss particulars with Sapier if I can find him. 03:42 kahrl except the part that creates and scales f->mesh_ptr[0] of course 03:43 RealBadAngel kahrl, indeed && will be better since it stops on first value fail 03:46 kahrl typo in nodedef.h:155 03:47 kahrl and we're through :) 03:47 kahrl that was quicker than expected 03:49 RealBadAngel :) i found out that changes needed to add meshes were way easier than i thought 03:49 RealBadAngel but still i had to read a lot bout them 04:00 RealBadAngel kahrl, one more thing, we need to disable loading of .mtl files by irrlicht 04:00 kahrl why? 04:01 RealBadAngel we do not need them, we are defining materials based on tiles 04:02 kahrl does it hurt in any way if irrlicht tries to load them and doesn't find them? 04:02 RealBadAngel its not a problem, theres a flag for that in the engine i think 04:02 kahrl other than some extra log messages 04:03 RealBadAngel no 04:03 RealBadAngel just the logs 04:03 kahrl well we already trained people to ignore the PNG warnings 04:03 kahrl shouldn't be hard to do the same for MTL warnings 04:03 RealBadAngel we already have too many strange warnings ;) 04:05 kahrl don't some entities from various mods use .obj files with .mtl files? 04:05 kahrl if so make sure not to break them 04:06 RealBadAngel they dont use them 04:07 RealBadAngel i mean mtl files 04:07 RealBadAngel we are basing on tiles (in case of meshnodes) and textures (in case of entities) 04:11 kahrl well if using .mtl never worked then sure, disable it 04:35 OldCoder Zeno`, wb 04:35 OldCoder Zeno`, I seem to have a working patch. But there is clearly something wrong in connection.cpp. 04:35 OldCoder There aren't supposed to be millions of RE-SEND RELIABLE lines 04:36 Zeno` yes that doesn't sound like something wanted or required :) 04:41 OldCoder Zeno`, I'd like you to review the new patch. But it's simply yours with minor tweaks... 04:42 OldCoder I increased minimum and maximum timeout resend periods, added a slight delay between resends, and accounted for possible corruption in the resend counter; possibly 1 or 2 other things 04:42 OldCoder My world has now run for a day with neither the server nor my client locking up 04:48 Zeno` Where is it? 04:49 RealBadAngel Zeno`, https://github.com/RealBadAngel/minetest/commit/8a625de529af79be8ac63f2320576faf8eb326e1 and https://drive.google.com/file/d/0B6CLV6iwDkmPUXhYaE5tZUZrcE0/view?usp=sharing 04:49 RealBadAngel kahrl, you may also want to try some meshnodes: https://drive.google.com/file/d/0B6CLV6iwDkmPUXhYaE5tZUZrcE0/view?usp=sharing 04:50 Zeno` cool, I'll give that a try *very* soon RBA :D 04:50 Zeno` Been waiting for this 05:16 OldCoder Zeno`, I will prepare a copy of the patch now 05:20 OldCoder http://minetest.org/141014.txt 05:20 OldCoder Zeno`, ^ 05:26 Zeno` does it work with that usleep outside of the loop?... not sure of the implications related to that 05:35 OldCoder Where would you put the usleep? 05:35 OldCoder Zeno`, ^ 05:36 * OldCoder is not clear 05:36 OldCoder I was getting millions of messages 05:36 OldCoder Thought something might be overlapping something else 05:36 OldCoder Figured a 50ms delay wouldn't hurt in this context 06:07 Zeno` maybe 25ms 06:07 Zeno` I'm not sure 06:08 Zeno` I'm not sure if it's throttling resends for /all/ net traffic I mean 06:09 Zeno` Also, I don't think usleep() is portable 06:10 Zeno` seems to be POSIX only... perhaps there is something in porting.h? 06:11 Zeno` hmm, I'm not sure about that one... other files seems to use it (indirectly through the macro sleep_ms in porting.h 06:12 OldCoder Zeno`, usleep is probably portable enough but can be replaced by nanosleep or sleep_ms 06:12 OldCoder 25ms should be enough, yes 06:13 OldCoder The real question is, why are there millions of RE-SEND RELIABLE lines without this patch? 06:14 OldCoder Zeno`, your own patch seemed to help but was not sufficient 06:14 OldCoder So one of the other odd tweaks that I made is related to the issue 06:14 Zeno` well, with this there is only 5 attempts made 06:15 Zeno` let me look again 06:15 OldCoder Isn't that your limit as well? 06:15 OldCoder I was STILL getting the millions of lines 06:15 Zeno` I see your point 06:15 OldCoder with a 5-attempt limit; I don't know how. This is why I added the strange code that fiddles with the resend counter 06:15 OldCoder I wondered if something was stomping on that 06:16 Zeno` well it may be overflowing but the if (k->resend_count > 5) should catch that before it happens 06:16 OldCoder Yes. Accordingly, there is no clear explanation. 06:16 Zeno` unless it's not set to something > 0 in the first place :/ 06:16 OldCoder Ouch 06:16 OldCoder That should cause lots more problems though 06:17 Zeno` but that would have shown up if valgrind way before now 06:17 OldCoder Yes 06:17 OldCoder Unless it has been 06:17 OldCoder casted somehow out of existence 06:18 Zeno` it's init to 0 06:18 Zeno` and it's unsigned so I can't see how it can be < 0 06:19 Zeno` this is weird 06:20 Zeno` what's the diff between this and my patch? 06:20 Zeno` I can really only see the sleep 06:21 OldCoder Hang on 06:21 OldCoder The differences are: 06:21 OldCoder #1. This might be key: changes in constants.h to RESEND parameters 06:21 OldCoder those might be it 06:22 OldCoder #2. Sleep 50ms before retry 06:22 OldCoder #3. Fix any corrupted resend counts 06:22 OldCoder I think that is all 06:22 OldCoder 06:22 Zeno` does it work with my path and just #1 ? 06:22 Zeno` patch* 06:22 OldCoder I will need to try this 06:22 OldCoder May do so soon 06:23 OldCoder Pretty tired this late and need to be awake for a few hours after changes 06:23 OldCoder So that I can fix the lockups 06:23 Zeno` I really don't think #3 is needed because it's kind of checked by if (k->resend_count > 5) anyway. I'm uneasy about the sleep (if the sleep is "fixing" things then I would consider it more hiding the error than fixing it) 06:24 Zeno` Yeah I'm pretty distracted at the moment as well... things to do, things to do xD 06:24 OldCoder Zeno`, one step at a time. And #3 is needed if there is corruption elsewhere. 06:24 OldCoder We will see what we will see 06:24 OldCoder There is still 06:24 Zeno` Yeah 06:24 OldCoder the possibility that 06:25 OldCoder somebody here or Sapier can comment on what could possibly cause 06:25 OldCoder *Millions* of those messages 06:25 OldCoder Seems like a lot 06:25 OldCoder 06:25 Zeno` I agree that there should not be millions... I'm not getting even one though. So I suspect you're right about *something* happening too fast 06:25 OldCoder Note that my server is very fast 06:26 OldCoder Could this be an issue? 06:26 OldCoder s/server/hardware/ 06:26 Zeno` not sure... I'm using 4 cores of a Xeon something 06:26 OldCoder I have 8 but yours is faster than usual too 06:26 Zeno` But I don't get this on my home server (i7) either 06:26 OldCoder The world is busy 06:27 OldCoder Or was when this was happening 06:27 OldCoder Plenty of possible things to look at 06:27 Zeno` actually let me just see what the server is 06:27 OldCoder Your server or mine? 06:27 Zeno` mine :) 06:28 Zeno` oh it's 8 cores 06:28 Zeno` 8 cores of Intel(R) Xeon(R) CPU E5-2630L 0 @ 2.00GHz 06:29 Zeno` oops 6 06:29 Zeno` all hyperthreaded, so 12 threads to use 06:29 Zeno` I dunno why it would affect you... maybe more traffic 06:31 OldCoder Or some totally random factor 06:31 OldCoder This problem seems to have occurred to other people 06:31 OldCoder Who may have had ordinary hardware 06:31 OldCoder The only clue is that... not much should be able to cause that many passes through the loop. 06:31 OldCoder Is this correct? 06:32 Zeno` at a glance I would have thought my original patch would have stopped that happening 06:33 Zeno` but obviously not 06:33 OldCoder Yep 06:33 OldCoder And yet 06:33 OldCoder It reduced the problem 06:33 OldCoder Leaving only occasional cases where the problem was triggered 06:33 OldCoder By, presumably, something happening when something else was not completed 06:34 OldCoder Putting things possibly into an infinite loop 06:34 * OldCoder bets that either longer timeout periods or the usleep are needed 06:34 OldCoder Zeno`, you have seen 06:35 OldCoder http://minetest.org/lockup.txt 06:35 OldCoder Right? Anything in the frequency or sequence numbers indicate how this is even possible? 06:36 Zeno` yike 06:36 Zeno` s 06:36 OldCoder Yep again 06:36 OldCoder Yikes and not trikes 06:36 OldCoder This is not as safe 11:07 Megaf_ what happened to minetest? 6 days without a single commit or merge 11:07 Megaf_ and this bloody bug still happening 11:07 Megaf_ 00:02:38: ACTION[ServerThread]: CHAT: OI 11:07 Megaf_ *** glibc detected *** ./minetestserver: corrupted double-linked list: 0xac51497a *** 11:07 Megaf_ Aborted 11:29 RealBadAngel kahrl? 12:27 Dragonop I have voice? 14:07 RealBadAngel kahrl i have coded all the changes 14:07 RealBadAngel makin PR now 14:08 Amaz For meshnodes? 14:09 RealBadAngel Amaz, yes 14:09 Amaz Great! 14:09 PenguinDad Yay meshnodes! 14:10 Zeno` so normal nodes are converted to meshnodes? 14:12 RealBadAngel https://github.com/minetest/minetest/pull/1738 14:13 RealBadAngel nodeboxes are converted to meshes 14:13 Amaz Do you have some example code for a meshnode? 14:13 RealBadAngel thx to that theyre like 10-15times faster 14:14 RealBadAngel yes i do, hold on 14:14 RealBadAngel https://drive.google.com/file/d/0B6CLV6iwDkmPUXhYaE5tZUZrcE0/view?usp=sharing 14:14 RealBadAngel apply the patch and use this mod 14:15 RealBadAngel it adds some meshnodes examples 14:19 ninnghazad uuh, meshnodes... sweet! 14:20 RealBadAngel just static, please no orgasms in the public :P 14:21 RealBadAngel but yes, they work and theyre COOL 14:21 Zeno` if this works as intended then it will take MT out of the dark ages 14:21 RealBadAngel just fucking try it 14:21 PenguinDad Nodeboxes lost their cracks :/ 14:21 Zeno` i have fucking tried it :) 14:22 RealBadAngel and? 14:22 Zeno` I haven't profiled yet 14:22 Zeno` but I didn't notice anything wrong 14:22 RealBadAngel but still nodeboxes can stay here 14:23 RealBadAngel as kinda easier way to define a mesh 14:23 PenguinDad Wow rotating stairs is faster than ever o_o 14:23 RealBadAngel PenguinDad, because all the rotations are cached 14:24 Zeno` so can I get more than 20fps now? 14:24 RealBadAngel if node is facedired resulting meshes are precalculated and stored 14:25 Zeno` 'cause I'm really sick of this 20fps crap heh 14:25 RealBadAngel my is 1,8ghz box and amd HD4600 14:25 RealBadAngel depending on env i get 40-60 fps 14:26 RealBadAngel but ofc propertiary drivers 14:26 Zeno` I'm using nvidia gtx780 and i7 and get 20fps (before this patch... I really need to try it more) 14:27 RealBadAngel btw, i wont be resolving opensource drivers issues anymore 14:27 Zeno` drivers are not within the domain of this project, surely 14:27 RealBadAngel with opensource drivers i got 10-20fps 14:28 RealBadAngel with propertiary it hits 60 14:28 Zeno` *shrug* that's not a minetest issue I wouldn't think 14:28 Zeno` drivers are at a way lower level 14:28 RealBadAngel i am aware of another bottlenecks 14:28 RealBadAngel but anyway theyre not at drivers level 14:29 Zeno` RBA, aim for > 100fps :p 14:29 RealBadAngel when we move cracks and HL out of mapblock mesh updates? 14:29 RealBadAngel quite possible 14:30 Amaz :D 14:33 RealBadAngel its not said i made the fastest version possible 14:34 RealBadAngel even if mine is faster 10 times as before it can happen another solution will be so much faster comparing to mine 14:34 RealBadAngel and i know such solution ;) 14:35 RealBadAngel its called VBO and i propably know how to use it 14:36 RealBadAngel like in shrek, i do have a donkey and im not afraid to use it ;) 14:36 RealBadAngel storing complete meshes on GPU side will be way faster 14:37 proller its not main thing ;) 14:37 RealBadAngel step by ste[ 14:37 RealBadAngel *p 14:37 RealBadAngel that PR is a fuckin revolution 14:38 RealBadAngel in fact when its proven to be working ok, it will make like 2000 lines of code obsolete 14:39 RealBadAngel also we will be able to re add texture atlases 14:44 Amaz https://mediacru.sh/be0ec57afa14 14:45 ninnghazad is it just the meshes that are not VBO-ed or are nodes actually pushed to gpu each frame without VBOs? 14:55 RealBadAngel first step was to operate on meshes 14:55 RealBadAngel which is damn faster 14:55 RealBadAngel 2nd will be to store them on gpu side 14:56 PilzAdam RealBadAngel, last time I tried enabling VBOs it caused memory leaks in RAM 15:01 ninnghazad so the VBOs irrlicht uses on MeshBuffers are intentionally disabled because of this? or is there another reason, like not working on android or something? 15:03 PilzAdam ninnghazad, I can't quite remember the last investigation; but there was a problem with Minetest's basic architecture that doesn't work with Irrlicht's way of handling VBOs or something along that lines 15:04 exio4 IIRC the problem with the leak of VBOs wasn't that it "leaked", but that meshes get unloaded after 10 minutes... 15:04 VanessaE PilzAdam: as I recall it was because no one had come up with a good way to discard meshes once they become obsolete 15:04 jin_xi so many reasons 15:04 VanessaE for reference, the last rebase of that code, btw, by Shadowninja: https://github.com/ShadowNinja/minetest/tree/vbo 15:07 ninnghazad ah ok, thx for info. 15:08 PilzAdam I guess RealBadAngel is going to just enable VBO's without caring about the memory leaks? 15:09 RealBadAngel PilzAdam, i do know how to use them 15:10 RealBadAngel but its not the subject of the current pull 20:38 iqualfragile https://github.com/minetest/minetest/blob/master/doc/mapformat.txt#L325 states that there is a timestamp containing the last time a mapblock was saved, question is: are mapblocks saved after they were loaded regardless of change or are they only saved when they were changed? 20:38 iqualfragile in the 2nd case it would be a lot more usefull to caching mechanisms