Time  Nick        Message
00:19 TheRealNate Hey @VanessaE
00:41 JDCodeIt    VanessaE: Any issues to update plantlife_modpak from a fairly ancient one (2013-2014ish)?
01:32 VanessaE    JDCodeIt: should be fine, but note it's on gitlab now.
01:32 VanessaE    hi TheRealNate
02:06 p_gimeno    woot, I've got advtrains to face the slopes correctly
02:07 VanessaE    I had poor luck with that mod..
18:27 JDCodeIt    In lua, would it be surprising that table element access is much slower than variable access time?
18:35 p_gimeno    "much" slower, yes it would be surprising. Just slower is expected.
18:37 p_gimeno    That's true for LuaJIT. I don't know about plain Lua, I don't use it.
18:37 p_gimeno    It should also be slower, but since it's overall much slower, it may not be noticieable.
18:43 JDCodeIt    I was looking at ways to restructure the on_generated in caverealms because it is incredibly slow - there must be some other major factor there.
18:44 JDCodeIt    Lots of calls to area:index() and math.random - is the index function inefficient?
18:47 JDCodeIt    After moving all the function calls for get_content_id to a single file-global table, then lookup in the table - gen time doubled!
18:48 sfan5       there's a topic on the forums about optimization
18:48 sfan5       calling :index() a lot is another thing you're not supposed to do
18:49 JDCodeIt    Just looking at the structure that would be the most non-sensible part - too many calls to that in the inner loops
19:31 JDCodeIt    So, did some profiling and index() is not really the culprit. 200 ms to do this one step: minetest.get_perlin_map(np_cave, chulens):get3dMap_flat(minposxyz)
19:33 JDCodeIt    And another 100ms to do this: minetest.get_perlin_map(np_wave, chulens):get3dMap_flat(minposxyz)
19:45 sfan5       well you should only get the perlin map once
19:47 JDCodeIt    So the time is really in generating the perlin maps (370 ms), setting the data back to the vm (90ms), and setting and calculating lighting (170ms). The actual traversing and setting data for the chuck takes 50 ms.
19:47 JDCodeIt    That's a good idea - cache and use...
19:49 p_gimeno    making use of 'buffer' would probably help
19:50 p_gimeno    as in, avoid constant reallocation
19:51 p_gimeno    have nvals_cave, nvals_wave, nvals_biome out of the function, and pass them to get3dMap_flat (which BTW should be renamed)
19:52 JDCodeIt    this could be done really efficiently if we know the calls to on_generated are for the same chunk size - is that true?
19:53 p_gimeno    doesn't matter, instead of one reallocation per call you still have reduced the number of reallocations substantially
19:54 p_gimeno    then there's #6863 which didn't go anywhere sadly
19:54 ShadowBot   https://github.com/minetest/minetest/issues/6863 -- Add LuaJIT FFI-friendly memory-intensive functions
19:55 JDCodeIt    How does it not matter if the volume of the chunk being generated changes? - this is a param passed to get_perlin_map.
19:56 JDCodeIt    I could see checking if the sizes changed versus what is already calculated, then calc again if needed.
19:57 p_gimeno    hm, we are not in the same page it seems...
19:57 p_gimeno    get3dMap_flat accepts a buffer as the second parameter
19:57 p_gimeno    that way of calling it is designed so the buffer can be reused
19:58 JDCodeIt    Aha, I missed that point, yes
19:59 p_gimeno    most reallocations will happen on the first call, and maybe a few more if the size grows
19:59 JDCodeIt    But to the question of the perlin result - could that also not be brought outside the function and only calculated on the first call?
20:00 JDCodeIt    But the next call has to check if the size if minp to maxp has changed, so the perlin calc is invalid
20:00 p_gimeno    probably, IDK how much of a factor that is, I expected get3dMap_flat to be the expensive one
20:00 JDCodeIt    Ok, I'll try it
20:03 JDCodeIt    Hmmm couldn't find documentation on this second parameter - buffer. lua_api.txt or dev wiki
20:03 p_gimeno    it's right there in get_3d_map_flat
20:04 p_gimeno    https://github.com/minetest/minetest/blob/master/doc/lua_api.txt#L5444
20:06 JDCodeIt    got it - dev wiki is out of date, though - put that on the TODO page
20:07 p_gimeno    thanks for your work on the wiki :)
20:09 JDCodeIt    duh - I see what you mean about the naming - it's get_3d_map_flat not get3dMap_flat, which is why I didn't find in the first place
20:09 p_gimeno    yeah it got renamed, and get3dMap_flat obsoleted
20:10 JDCodeIt    https://github.com/minetest/minetest/blob/master/doc/lua_api.txt#L3118 has a ref to the old name
20:11 p_gimeno    yeah, and the description refers to get2dMap_flat instead of the new name
20:13 p_gimeno    to be clear, this is what I meant: https://notabug.org/pgimeno/Gists/commit/f190a612135a0e939a4eaa35550222dc78d17b4a
20:17 p_gimeno    didn't the profiler say something about the get_perlin_map call, by the way?
20:17 p_gimeno    I mean, as opposed to the get3dMap_flat call
20:28 JDCodeIt    Guessing this is not thread-safe - the third thread to access the buffer went out of bounds
20:29 JDCodeIt    I can try with emerge threads = 1
20:31 p_gimeno    emerge_threads > 1 is very broken, there's no locking at all
20:31 p_gimeno    at least of the noise objects and some other objects
20:32 JDCodeIt    nope not thread-related - third call to the function fails when accessing part of nval_biome
20:33 p_gimeno    did you misspell that here or in the file? it's nvals_biome
20:33 JDCodeIt    typo nvals_biome
20:36 p_gimeno    scratching head... there's just one user of nvals_biome and it's straightforward
20:40 p_gimeno    can't reproduce
20:44 JDCodeIt    my thought was to only call get_perlin_map once and only again if the inputs there changed - I think you need to call get_3d_map_flat anew every time for each chunk, but I was going to use the cached perlin object.
21:06 makayabou   VanessaE: nice "Las Vegas Parano" effect with homedecors and shaders !! https://pictshare.net/vip53a.gif
21:08 JDCodeIt    p_gimeno: simple copy/paste error. Unfortunately, even after caching the perlin object - no significant performance improvement
21:11 p_gimeno    even with the buffer version?
21:12 JDCodeIt    Yes, got that running, along with caching the perlin object - no significant change - that function is expensive
21:13 p_gimeno    k, then there's little else that can be done, sorry
21:21 JDCodeIt    in the engine it's a bunch of lua_pushnumber and lua_rawseti - these are not particularly slow, are they?
21:23 p_gimeno    they aren't too slow, but for the amount of data here, they become significant. That's where #6863 was an improvement.
21:23 ShadowBot   https://github.com/minetest/minetest/issues/6863 -- Add LuaJIT FFI-friendly memory-intensive functions
21:23 p_gimeno    Also, lua_rawseti is where reallocation may happen.
21:31 VanessaE    makayabou: weird :P
21:40 JDCodeIt    It is a lot of elements - 80^3 - is there a better way to visit the vm and set the data?
21:42 JDCodeIt    maybe paramat has an idea?
21:43 JDCodeIt    paramat: I was trying to improve caverealms on_generate performance.
21:43 paramat     i was about to write about caverealms, reading channel logs
21:44 paramat     i suspect it's slow due to far too many light sources being placed. 'caverealms lite' is much more tastefull
21:44 paramat     *tasteful
21:44 JDCodeIt    My profiling also found getting the 3d map was expensive
21:45 paramat     yes 3D noise is intensive
21:45 VanessaE    makayabou: just so you know, none of homedecor's stuff is meant to wave like that, and has no flags for the purpose. :P
21:45 JDCodeIt    But is the approach bad in general? Get 512000 elements, traverse them, then write it back?
21:47 VanessaE    JDCodeIt: fair warning, paramat seems to be intent on making sure MT can still run on an IBM 5160 ;)
21:47 JDCodeIt    What! The Z80 is no longer supported?
21:47 VanessaE    lol
21:48 cheapie     So I can't run it on an 8008 and now it won't run on a Z80 either?
21:48 cheapie     What's next, it'll require a screen too?
21:48 makayabou   VanessaE: I'm using minetest 5.0.0 and homedecor commit ad75dba87b
21:49 VanessaE    makayabou: maybe upgrade to 5.0.1..  if those items are waving, it ain't anything in my code :P
21:49 paramat     a 3D perlinmap has to be calculated for every mapchunk, i'll look at the code of the original mod. however, 0.5s per chunk gen time is fairly normal for a lua mapgen
21:49 makayabou   ok no prob
21:50 VanessaE    makayabou: if it still does it, file an issue against https://github.com/minetest/minetest/
21:51 paramat     ah, waving nodeboxes
21:51 VanessaE    yup
21:51 paramat     not intentional?
21:51 VanessaE    nope.
21:51 makayabou   yes sure, I just wanted to show you the gif, I didn't try to reproduce in a clean env
21:51 VanessaE    paramat: none of my nodes have that feature enabled, afaik.
21:51 paramat     makayabou did you add 'waving = 3' to those nodeboxes?
21:52 VanessaE    makayabou: still, it would be worth reporting if you can reproduce in a clean env
21:52 makayabou   no i didn't. neither to other nodes
21:52 makayabou   VanessaE: yes, was going to
21:52 * cheapie   eats part of a nodedef and goes all wavy
21:52 paramat     ugh a bug
21:53 paramat     someone else reported this on april 1st and then thought it was a core dev april fool's joke
21:53 paramat     weird
21:54 p_gimeno    makayabou: are you sure it's 5.0.0?
21:54 VanessaE    fwiw, only one item in all of the homedecor modpack has "waving" (=1), fake fire.
21:54 paramat     are all the waving nodes meshes?
21:54 makayabou   yes sure
21:55 paramat     because MTG nodeboxes aren't affected by unintentional waving
21:55 paramat     probably a bug in my recent commit
21:55 makayabou   sorry i don't understand "are all the waving nodes meshes?"
21:56 paramat     VanessaE ^ see the gif
21:56 paramat     only meshes affected?
21:57 paramat     see http://irc.minetest.net/minetest/2019-04-01#i_5522533 heh
21:58 makayabou   yes only meshes
21:59 paramat     ok
21:59 paramat     commit https://github.com/minetest/minetest/commit/42e1a127140965cac1be6e51e48192e341c2a29e
22:00 p_gimeno    paramat: that's post-5.0.0 isn't it?
22:01 paramat     yes 5.1.0-dev
22:01 paramat     makayabou are you using latest MT code? it's actually called 5.1.0-dev not 5.0.0-dev
22:02 p_gimeno    makayabou insists it's happening on 5.0.0
22:03 VanessaE    what'd I miss?
22:03 paramat     well, some people think 5.1.0-dev is 5.0.0-dev
22:05 paramat     hmm i can't see any obvious bug in code
22:05 VanessaE    no one caught the texture rotation bug until after 5.0.0 either :P
22:05 paramat     some people call 5.1.0-dev 5.0.0
22:06 p_gimeno    I can't reproduce on 5.0.0
22:06 cheapie     I gave up on following the versions a while back and just call it "latest dev"
22:06 cheapie     (or "some random old dev version" as needed)
22:07 VanessaE    inb4 downloaded it from someplace other than MT official sites
22:08 makayabou   i reproduced in the same flathub for debian build 5.0.0 and the same versions of mods without other mods than those from minetest-game activated
22:08 VanessaE    flathub?
22:09 paramat     gitpack?
22:09 p_gimeno    makayabou: what waving options do you have active exactly? water only? leaves too? plants too?
22:09 makayabou   *flatpak
22:10 p_gimeno    wait, leaves and plants don't wave up and down
22:10 VanessaE    yeah, that's the waving liquids shader moving stuff it shouldn't
22:11 makayabou   flatpak install flathub net.minetest.Minetest
22:11 paramat     JDCodeIt caverealms is partly redundant because most core mapgens now have the massive caverns. 'caverealms' lite just adds decorations to the core mapgen caverns
22:12 VanessaE    paramat: define "massive"
22:12 paramat     100s high, up to 1000 long
22:12 makayabou   p_gimeno: don't use any waving options
22:12 makayabou   ah sorry. in settings.. wait
22:12 p_gimeno    yes
22:13 makayabou   the 3 (liquids, plants and leaves)
22:15 kurtzmusch  a cople quesions: is it possible to use custom shaders with csmodding? is it possible to change cloud shape/color?
22:17 paramat     kurtzmusch 1. no 2. there is a clouds API. cloud coverage, thickness and colour is adjustable
22:17 kurtzmusch  what coverage means?
22:17 kurtzmusch  like how much of the sky it covers?
22:18 paramat     yes
22:18 kurtzmusch  huum, ok thanks
22:18 paramat     the XZ shape of the clouds is still determined by 2D noise
22:18 kurtzmusch  would make a mesh entity ro replace clouds be 'sane'?
22:19 kurtzmusch  to replace*
22:19 paramat     entities disappear at 3 or so mapblocks away
22:20 kurtzmusch  oh, thats too close for clouds =(
22:20 kurtzmusch  thats 16*3 right?
22:20 paramat     JDCodeIt i looked at caverealms code, it seems correct apart from missing memory use optimisations. it's slowness is expected from a lua mapgen
22:20 paramat     mapblock = 16^3 nodes
22:22 JDCodeIt    OK, I thought of some other performance tweaks - not calling index() unnecessarily, checkinf nothing changed in the chunk - don't bother writing it back
22:22 paramat     sure
22:23 paramat     there may be some further optimisations, but the general approach is correct
22:28 kurtzmusch  thanks paramat
22:29 paramat     i assume moreblocks slopes are meshes? they don't wave for me
22:29 VanessaE    they are.
22:30 paramat     thanks, hm
22:32 p_gimeno    I've tried to find where to download the flatpak, to examine it, but I couldn't
22:32 p_gimeno    (I don't want to install it, just take a look)
22:33 makayabou   wait it was an error it was a build.. sorry. i'm trying with flatpak. where do i puts mods in a flatpak install?
22:35 p_gimeno    makayabou: ok, that changes things but there still may be a bug worth being looked into
22:36 paramat     !tell nepugia your waving solar panels were not an april fools joke, some people are experiencing a bug where meshnodes wave like water. please give us more details, what version of MT? what build?
22:36 MinetestBot paramat: I'll pass that on when nepugia is around
22:36 makayabou   I know but sorry again I removed my build so I'm not sure I will be able to reproduce..
22:43 p_gimeno    can't reproduce with current master, either with MTG master or 5.0.0
22:49 paramat     good, thanks
22:49 makayabou   if I use git clone minetest, to test with different commits I'll have to rebuild each time?
22:50 paramat     recompile yes
22:50 Lone-Star   yes
22:51 makayabou   ok thanks