Time Nick Message 00:00 hmmmm say it out loud in a sentence: "oh this variable is the mtsetting value" 00:00 hmmmm vs. "this is the setting instead" 00:01 hmmmm the latter of the two sounds like you were having a conversation with somebody, and you don't have a sentence or two of context 00:01 hmmmm the former can stand on its own 00:06 paramat hmmmm i'm beginning to think #2665 may just be a natural result of using a lua mapgen with a heavy use of noise. my own tests show watershed uses up to 1.5GB of memory, riverdev uses up to 2GB (10 3D noises, 6 2D noises) 00:06 ShadowBot https://github.com/minetest/minetest/issues/2665 -- get_perlin_map consumes all memory and brings down servers 00:06 hmmmm no there's a memory leak somewhere 00:07 hmmmm i was trying something out before 00:07 paramat aha 00:07 hmmmm also see the two other issues related to memory 00:08 paramat ok 00:12 paramat it's also present in 0.4.12 stable, same results 00:21 paramat testing mgv7, it has a steady rise in memory use when flying and generating, after 5 mins i'm up to 2GB usage. if i stop exploring it stays level 00:21 paramat doesn't fall 00:22 hmmmm sure... but that has nothing to do with noise 00:23 hmmmm set the unused block unload timeout to be very short, after flying around you should expect the memory usage to be around what you started at 00:23 paramat whereas a lua mapgen has a rise and fall of memory use during flying and generating 00:23 hmmmm like after flying around, stare at the ground and do nothing for a few minutes 00:24 paramat seems a core mapgen problem 00:24 hmmmm eh, i'm not so sure it's a mapgen problem 00:24 hmmmm the mapgen allocates no memory when it's generating map 00:24 hmmmm unless there was some kind of change i wasn't aware of 00:40 paramat ok with block unload timeout of 1 min the memory use rise is slower, 1GB after 5min. now staring at the ground but no reduction yet 01:13 paramat same behaviour in mgv5. flying back and forth over generated terrain doesn't increase memory use, seems to be due to generating new terrain 01:47 hmmmm est31, https://github.com/minetest/minetest/pull/1885 01:51 Zeno` noooooO! 01:51 Zeno` That is not good 01:51 hmmmm good or not good 01:51 Zeno` not 01:51 hmmmm [05:49 PM] hmmmm, what do you think is the preprocessor hack ugly or nice? https://github.com/est31/minetest/commit/4129674ad8446ada991b842041d0bcae775cd738 01:51 hmmmm est31 wants to bring CoreSettings back 01:52 Zeno` oh hang on./. 01:52 hmmmm I think it's great, personally, instead of adding some kind of callback mechanism why not just make settings fast 01:52 hmmmm and atomic 01:53 Zeno` that's my second version, so it's half ok 01:54 Zeno` The reason I left this and didn't complete what I started is I realised it wasn't worth it 01:55 Zeno` The whole (current) design is kind of wrong 01:55 Zeno` hang on... 01:55 hmmmm est is very eager to code stuff =] 01:57 Zeno` How is this new version faster? 01:57 Zeno` oh i see 01:58 hmmmm a single memory location read vs. a lock, a std::map lookup, and then a string-to-whatever conversion?? 01:58 Zeno` Yes, well this is more like how it should have been done 01:58 Zeno` in the first place I mean 01:58 hmmmm right 01:59 hmmmm this doesn't replace callbacks 01:59 Zeno` *nod* 01:59 hmmmm callbacks and this can coexist peacefully 01:59 Zeno` I don't give a @*(#*@ about callbacks tbh :) 01:59 hmmmm they need to be there so events can be triggered on changing a setting other than setting the value to something new 01:59 Zeno` If I had gotten back to this, this (est31 current) is more like how I would have approached it 02:00 hmmmm well actually no that doesn't need to be there 02:00 Zeno` the callback cannot hurt... it's actually nice to know if a setting has changed 02:00 hmmmm well if a setting has been Set 02:00 Zeno` yes 02:00 hmmmm if a setting had set called on it and it happened to be the same value as it was previously, there's no telling 02:01 Zeno` yes, but there is a fine line there :) 02:02 hmmmm btw zeno, how do you feel about overloading functions and operator overloading in general 02:02 Zeno` kahrl and I had this discussion when I noticed that something was using some setting like it was a variable 02:02 Zeno` which was obviously very, very, very slow 02:02 hmmmm est wants to get rid of the setU32() setBool() etc. distinctions and have overloaded set() 02:03 Zeno` :( 02:03 Zeno` I don't like it 02:03 hmmmm all in the name of 'expressiveness' 02:03 hmmmm grr 02:03 hmmmm hold a vote 02:03 Zeno` why, grrr? :) 02:03 hmmmm because I want things to be immediately obvious and not dependent upon context 02:04 Zeno` just let me remember why I don't like it... I have to look something up 02:04 Zeno` oh this is just for settings you mean? 02:04 hmmmm yeah 02:04 hmmmm btw what is our policy on operator overloading? 02:05 Zeno` that's different. I thought you meant in general (everywhere) 02:05 Zeno` for settings it makes sense 02:06 Zeno` My "policy" is that if the calling "program" should never, ever, care what the internals should be then it's ok 02:07 Zeno` it can get confusing 02:08 Zeno` I think maybe getNode() is a good example 02:08 Zeno` nope, that's not a good example of this, but it is confusing 02:09 hmmmm so in general we can agree that being explicit is better than implicit 02:09 Zeno` in general, yes 02:09 hmmmm overloaded things require context to make sense of them, and context means that you have a higher cognitive burden 02:10 hmmmm minetest is already complicated enough as it is 02:10 hmmmm ugh 02:10 Zeno` correct, so the more abstracted something is the more implicit it can be 02:11 Zeno` *sigh* 02:11 Zeno` minetest is not like that though 02:11 Zeno` just look at how intermingled and entrenched irrlicht is 02:11 Zeno` there is no reason for that... there is no "separation" 02:12 hmmmm switching to a new engine would be a great time for fixing this 02:12 Zeno` so I think adding implicitness all over the place is, in general, not a good idea at this time 02:12 hmmmm i think it can be done without restarting the entire project from scratch 02:12 Zeno` I believe it could be done as well 02:13 Zeno` I even started 02:13 Zeno` :) 02:13 Zeno` but then got sidetracked haha 02:13 Zeno` (I don't start things that I don't believe can be done) 02:18 Zeno` going back, for settings I think explicit like setU32() is silly because a) The programmer *must know the type anyway* of the setting (i.e. selecting the right function depends upon the actual setting so you have to have a general idea at least of the setting type to store a value of that type in a local object, which you are likely to base your new set() call off)) 02:18 Zeno` hmmmm, do you know Qt? 02:18 hmmmm nope 02:19 Zeno` http://doc.qt.io/qt-4.8/qvariant.html 02:19 hmmmm so I guess set() works, but definitely not get() 02:19 Zeno` That is a class I actually like 02:19 hmmmm that's downright dangerous 02:20 Zeno` so you have a settings class that returns a QVariant (I'm just rambling for an example)... 02:20 hmmmm QVariant.asString() or something? 02:20 Zeno` then you can do assert(var.isBool); then bool var = var.toBool() 02:20 Zeno` asBool() 02:20 Zeno` yeah 02:21 hmmmm it does seem architecturally nice but isn't that going to be more efficient in the long run 02:21 hmmmm less efficient i mean 02:21 hmmmm what's wrong with what we got 02:22 Zeno` correct. I'm just using it as an example where I think the implicitness is done Right(TM) 02:22 Zeno` QVariant is mostly used for settings and callbacks 02:22 hmmmm the beer we got drank pretty gud, don't it? 02:22 Zeno` heh 02:22 hmmmm https://www.youtube.com/watch?v=MZoTJzh13n8 02:23 Zeno` By example, it was more an answer to your question in general... not an answer for minetest :P 02:23 hmmmm you're going to write the same amount of code to do it either way 02:23 hmmmm probably more code with the QVariant-type thing 02:24 Zeno` lots more code 02:24 hmmmm each getThing() maps to a get() + an asThing() 02:24 Zeno` I'm not suggesting it as an answer :) 02:24 Zeno` Sometimes I just like discussing things heh 02:24 Zeno` I should preclude these discussions with 02:25 Zeno` Next we'll be using templates for Settings.... 02:26 hmmmm pew 02:27 Zeno` yay 02:27 Zeno` oops, spoke to soon. 02:27 Zeno` android is annoying :( 02:53 Zeno` maybe it's a java error :( 02:55 Zeno` E/minetest(17340): Shutting down minetest. 02:55 Zeno` V/threaded_app(17340): android_app_destroy! 02:55 Zeno` and yet... it's still there in the task manager 02:57 Zeno` maybe I can check the android version somehow and just put the exit(0) hack back 03:29 Zeno` hmmmm 03:29 Zeno` https://github.com/minetest/minetest/commit/f1ccfd3c3d7d626087f70b8c5798110cd28b248a#diff-60aa608aac2fefb1a86d4c2861a08eb7R361 03:29 hmmmm yes i know 03:29 hmmmm didn't somebody fix that already? 03:29 Zeno` doesn't look like it 03:29 hmmmm :| 03:33 Zeno` I wonder if I added the incorrect assert 03:33 Zeno` (i.e. should have been assert (i <= 6);) 03:34 hmmmm no, because i can never be equal to 6 inside of that loop 03:34 hmmmm it's supposed to range from 0 to 5 03:34 hmmmm 0 to 5 is 6 materials 03:34 Zeno` lol 03:35 hmmmm but the assert was completely incorrect. this is a class of bugs 03:35 Zeno` yes 03:35 hmmmm developers too smart for their own good think they're adding correctness by using an assert() in a place where input is provided by a remote peer 03:35 hmmmm i think this is the 4th? time this particular class of bug cropped up 03:37 Zeno` when I changed the asserts to remain in release builds the only ones I left as assert were ones I were not sure of, or ones I put back because sapier said "no, they're real asserts" 03:37 Zeno` I were, I were! 03:37 Zeno` I must have missed that one 03:38 Zeno` wait... filename starts with w 03:38 Zeno` I would have been sick to death of checking everything by the time I got to w lol 03:38 Zeno` =D 03:43 Zeno` Did a recent commit change font sizes? 03:45 hmmmm doesn't look like it 03:47 Zeno` weird 04:20 Zeno` #2665 for anyone awake 04:21 ShadowBot https://github.com/minetest/minetest/issues/2665 -- get_perlin_map consumes all memory and brings down servers 04:21 Zeno` I dunno if the noise has a leak 04:21 hmmmm it doesn't 04:21 Zeno` I made a beautiful graph 04:21 hmmmm did you run valgrind? 04:21 Zeno` massif, yeah 04:22 hmmmm the last time you ran massif, noise didn't even show up 04:22 Zeno` with minimal game and just the test he provided 04:22 hmmmm and what 04:22 Zeno` took me a bit to realise I had to type /regen to start the test 04:22 hmmmm lua is garbage collected; i can see the kind of problem he's talking about IF he has a low amount of ram, a disabled swapfile, and lua doesn't garbage collect often enough 04:22 Zeno` (which is why there is about 10 seconds at the start where no mem at all is really used) 04:23 Zeno` yeah 04:23 Zeno` as I said in my comment it seems to be normal Lua GC behaviour 04:23 hmmmm that guy won't give any details about his test machine 04:23 hmmmm in any case, the lua gc should kick in long before there are any crashes due to lack of memory 04:26 Zeno` I would think so 04:27 Zeno` Although there is a leak *somewhere* 04:27 Zeno` just not there 04:28 Zeno` probably still the liquids hehe 04:28 Zeno` his comment: If you type "/regen" fast enough, memory will keep going up as it does not free up the old memory fast enough. 04:29 Zeno` that cannot be right... my test is using the infinite loop so I'm sure the loop is faster than his typing 04:29 Zeno` and memory did not just keep going up 04:38 hmmmm regardless, the memory consumption is too high for such a mundane task 04:38 hmmmm I'm going to split up generating noise and then fetching the table of values 04:38 Zeno` yes 04:38 hmmmm that also might even make it faster 04:38 Zeno` I'm just trying with a larger threshold but splitting would be better 04:39 hmmmm i can do splitting in the core Noise object too, but the benefit is nowhere nearly as much 04:41 Zeno` I wonder if it's feasible to run massif on my server for "real" data 04:42 Zeno` probably way too slow :( 04:43 Zeno` valgrind hooks can be in the C++ though... maybe I could utilise that to make a fast version heh 04:46 Zeno` 866GB is kind of high 04:47 Zeno` err MB 04:48 hmmmm :/ 04:48 hmmmm gonna require some alcohol for this 04:57 Zeno` what kind? Pure ethanol? 04:58 hmmmm an ethanol/water/grain mash blend with hints of oak flavor 05:00 Zeno` bourbon, whiskey or brandy? lol 05:00 hmmmm wild turkey 05:01 hmmmm by the way, nerzhul pretty much admitted he's not coming back to minetest. i think it's time to fix the HP damage flash 05:01 Zeno` ok 05:02 Zeno` he did? 05:02 hmmmm ye in a PR on github 05:02 Zeno` ok, I'll fix it after I finish my tests 05:02 Zeno` I didn't bother making a branch because it was 2 lines 05:04 hmmmm good riddance, right? i think he could've been an awesome dev 05:14 Zeno` http://dpaste.com/0EY7MRN :/ 05:25 hmmmm dammit lua, y u take up so much space 05:39 hmmmm https://github.com/kwolekr/minetest/commit/6e2c214dd172ee1bf21f2172fb8c59b0628bb26f 05:41 hmmmm actually no that's stupid 05:42 hmmmm I'm going to add a force_place = true boolean per node definition for lua schematics 05:42 hmmmm why should I expose lua users to scary bitfields 05:42 Zeno` *nod* 05:43 hmmmm i'm glad i thought about this more before pushing it 06:14 hmmmm alrighty, here is my final version https://github.com/kwolekr/minetest/commit/faf2a7ea91e5c35e922427a3ebd412fce5332b40 06:50 hmmmm will push if nobody has anything to say... 17:28 est31 est wants to get rid of the setU32() setBool() etc. distinctions and have overloaded set() 17:28 est31 in fact no 17:28 hmmmm I thought that's what you said 17:28 est31 original plan was just to overload once 17:28 est31 so that you have the same method for the string name and the enum setting type 17:29 hmmmm ahh 17:29 est31 so setBool(mt_setting_draw_clouds, true) and setBool("draw_clouds", true) 17:29 est31 (imagine the mt_setting stuff in uppercase) 17:29 hmmmm yes 17:30 hmmmm by the way you know how I always harp on readability 17:30 hmmmm over brevity 17:30 hmmmm when you have something as commonly used as MT_SETTING_DRAW_CLOUDS or whatever it probably makes sense to have a shorter constant name 17:31 hmmmm I would have recommended CS_ (for CoreSetting) but that's being used by ClientState 17:31 est31 like? 17:31 est31 I'm ok with shorter prefixes 17:31 hmmmm what if MT_SETTING_ were switched with CFG_ 17:31 hmmmm CFG_DRAW_CLOUDS ? 17:31 est31 sounds good 17:31 hmmmm yea 17:31 hmmmm because people have to type this all the time 17:32 est31 yes 17:32 hmmmm and precisely because people use it all the time, this is why it's not as necessary to have things more verbose and clear 17:32 hmmmm at least i would argue 17:32 hmmmm because it's so commonplace and better understood than the SRP implementat would be 17:33 est31 what does this have to do with srp? 17:35 hmmmm because I realize what I'm saying sounds contradictory to what I was saying about the naming of constructs in SRP 17:35 est31 ah 17:35 est31 yea 17:36 est31 gtg brb 18:48 paramat hmmmm yeah a per-node force-place bool is much better 18:49 paramat you just need to update mg_schematic.h lines 54-56 18:49 paramat and add notes about the bool 18:53 paramat so now soon i will add the full set of biomes to mgv5/v7 19:05 hmmmm what do you mean update 54-56? 19:05 hmmmm that's the file format, not the lua table format 19:06 hmmmm when you define a schematic through a lua table you write the nodes like 19:06 hmmmm {name="default:tree", prob=255, param2=0, force_place=true} 19:08 paramat aha so bit 7 is still used as the force-place flag in the schem format.. 19:09 hmmmm why would you care about user friendliness of a binary file format 19:09 paramat exactly 19:11 hmmmm so also because yslice probabilities follow the same bitfield format there's a free bit for each vertical level of the schematic 19:11 paramat so we actually have 128 levels of prob, but written as 0-255 in the lua table 19:12 hmmmm yes 19:12 hmmmm so 254 and 255 are technically the same probability value, as is 0 and 1 19:12 paramat clever. is serialize_schematic updated yet, i guess not 19:12 hmmmm oh shoot 19:13 hmmmm well there's a bug 19:52 TenPlus2 Hi folks, any devs around today ? 19:53 est31 yess 19:53 est31 :p 19:53 TenPlus2 lol, joy!! 19:54 est31 what does the error say 19:54 TenPlus2 any idea what would cause an std::bad_alloc error ? 19:54 est31 how does it happen? 19:54 TenPlus2 2015-04-30 12:12:35: ACTION[ServerThread]: player G1127 crafts default:stick 4 19:54 TenPlus2 terminate called after throwing an instance of 'std::bad_alloc' 19:54 TenPlus2 what(): std::bad_alloc 19:54 TenPlus2 that's one of them 19:54 est31 so you know how to reproduce? 19:54 TenPlus2 it's random 19:55 est31 what version does the server run? 19:55 TenPlus2 not always stick being crafted, can be farming, opening chest etc 19:55 TenPlus2 server is using 0.4.12 stable for linux... 19:55 TenPlus2 I've also tried latest dev to see if it was inadvertanly fixed, bu nope 19:55 est31 ok 19:56 TenPlus2 *inadvertently 19:56 est31 then we obviously have to do something about it :) 19:56 est31 (as devs) 19:56 TenPlus2 I sit on a test platform far above spawn where nothing happens/spawns/no abms... and wait for the fire sound to disappear, that's when I know server has crashed 19:57 est31 so it also happens when no players are online? 19:57 TenPlus2 yeah, and the odd time I run standalone server it fails and comes up either std:bad_alloc or segfault 19:57 TenPlus2 just ooutta the blue 19:58 est31 what do you mean with that? 19:58 TenPlus2 I have a copy of the server standalone also so I can test mods before they go live... it happens standalone (singleplayer) also 19:58 TenPlus2 but mostly on server 19:59 est31 so it seems to be a mod problem? 19:59 TenPlus2 no error for ANY of the mods, I was told it was memory problem (running out) so stripped down server to see if that helped... nothing... disabled each mod in turn... nope... 19:59 est31 I mean an error a mod caused, but the engine has a bug too and doesnt show the error as such 20:00 TenPlus2 most of the errors is simply: "terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc" 20:01 est31 There are two possible reasons 20:01 est31 1. not enough space 20:02 TenPlus2 HDD Space ? 20:02 est31 no ram space 20:02 TenPlus2 ah, server has 4gb... 20:02 TenPlus2 it never goes above 1gb... never! 20:02 est31 should be enough in theory 20:02 TenPlus2 and the Os (lubuntu 14.04) only uses 102mb 20:03 TenPlus2 what's the other reason ? 20:04 est31 not enough space in one piece 20:04 est31 but I guess thats just wrong 20:04 est31 I mean with virtual memory 20:04 est31 (read it on the internet) 20:04 est31 so in theory it may happen 20:04 TenPlus2 we dont use virtual memory... that's disabled and all in tmp space 20:04 est31 sorry meant virtual address space 20:05 TenPlus2 ahh 20:05 TenPlus2 https://github.com/minetest/minetest/issues/2661 20:06 est31 next idea 20:06 est31 open it in a debugger 20:06 est31 and let it print a stacktrace 20:06 TenPlus2 standalone or server ? server isnt physically with me, it's in Wisconsin 20:07 est31 whereever you want to debug :) 20:07 est31 usually debuggers dont cause overhead 20:07 est31 if you only let them print stacktraces 20:07 TenPlus2 so add --debug to command line ? 20:07 est31 no 20:07 est31 thats setting it to debug logleve 20:07 est31 l 20:07 est31 might be worth the try too 20:07 TenPlus2 in minetest.conf... 20:08 TenPlus2 gotcha 20:08 est31 just expect to have much much more output 20:08 est31 really much mor 20:08 est31 e 20:08 TenPlus2 in that case I'll prolly run it on standalone server tomorrow, the server is on 120gb ssd... 20:09 est31 ok then 20:09 est31 do you daily truncate the log? 20:10 TenPlus2 no, server script copies previous log to another file and starts a new one on restart or crash... so I have a copy... 20:10 TenPlus2 I always get to see the last error though and a log of how many times it's restarted 20:11 est31 nice 20:11 TenPlus2 #!/bin/bash 20:11 TenPlus2 while true 20:11 TenPlus2 do 20:11 TenPlus2 echo "- = R E S T A R T I N G S E R V E R = -" 20:11 TenPlus2 sleep 1 20:11 TenPlus2 date >> ~/lastrun.txt 20:11 TenPlus2 minetest --server |& tee ~/log.txt 20:11 TenPlus2 cp ~/log.txt ~/crashlog.txt -f 20:11 TenPlus2 done 20:11 est31 next time use a pastebin :) 20:11 TenPlus2 oops, my bad 20:11 TenPlus2 am too use to using yahoo chat :) 20:13 TenPlus2 normally I'd go back to using 0.4.11 but too many mods have been changed over to easily do that, and the players enjoy the new features and new mapgen 20:15 est31 try whether verbose logging will help 20:15 est31 verbose loglevel 20:15 est31 what generally will help is a stacktrace 20:15 est31 at least isolates the region 20:16 TenPlus2 will give that a go... thanks for the help... and I'll definitely use pastebin for that output :) 20:16 TenPlus2 you guys really do work hard on Minetest and it really is an amazing game... thanks for all of that... 20:16 est31 loglevels cant give you a stacktrace though 20:17 est31 I dont know a method to open gdb for example so that it does nothing more than print stacktraces 20:17 est31 and exit 20:17 est31 (because I guess that would be what you need) 20:18 TenPlus2 will see what can be done tommorrow :) thanks again est... nite all 22:55 hmmmm Hmm 22:56 hmmmm I think i have a good interm solution for high lua mapgen memory usage 22:57 est31 ? 23:29 hmmmm garbagecollect() 23:31 hmmmm oh wait the guy reporting the bug already knows about this 23:32 hmmmm jeez he sure talks a lot