Time  Nick      Message
01:56 lhofhansl Looks like part of the trick with MAP_BLOCKSIZE=32 is to reduce max_simultaneous_block_sends_per_client to 5 or 10. Along with chunksize = 3 it all works reasonable. (Again, not saying we do that. Blocksize = 16 is good. Just if folks want to see the possible FPS at higher view ranges)
05:14 lhofhansl I found it very helpful to spread map saving out over multiple server steps. Steps of 100ms seem to be quite nice. The server stays responsive, and can get some packets for the client in while the map is being saved. It's very simple and quite effective. I'll have PR soon (waiting for 10715 so I can avoid rebasing).
13:06 pgimeno   will #10714 (or further memory management changes if it gets applied) eliminate the possibility to look back at what you have recently visited by enabling Full Range View? I consider that a feature
13:06 ShadowBot https://github.com/minetest/minetest/issues/10714 -- Keep mapblocks in memory if they're in range by hecktest
13:17 hecks     pgimeno: no, it will simply keep blocks in the player's LOS alive unconditionally
13:18 hecks     blocks outside of the vision sphere will behave as they used to
13:18 hecks     but wait
13:18 hecks     I'll check it just in case
13:19 hecks     if (!m_control.range_all &&
13:19 hecks     the range check doesn't even happen if you have "unlimited range" enabled
13:20 hecks     of course as soon as you turn that off, the clock will start ticking for out-of-range objects again
13:20 hecks     all I did was make the clock reset itself before the frustum cull, not after
13:20 pgimeno   will it start or resume?
13:20 hecks     restart
13:21 pgimeno   oh so enabling unlimited range resets the timer?
13:21 pgimeno   that's good to know, thanks
13:21 hecks     yup
13:21 hecks     basically the usage timer is reset every time a block is "used" by the renderer
13:21 hecks     before, it happened after a frustum cull and allowed unloading a block that was right behind your back
13:22 pgimeno   I heard something about the current code preventing future cache optimizations and got worried that those future cache optimizations could affect that mechanic
13:22 hecks     caching blocks should ideally be reworked around this
13:23 hecks     to only consider things outside of the range for caching, though it implicitly works that way right now anyway
13:23 hecks     it's just a matter of semantics and whether you want the ram limits in config to count the vision sphere or not
13:23 hecks     my strong opinion on this is that attempting to do anything clever with the frustum in this context was idiotic
13:24 hecks     and the client should always be prepared to handle all the blocks within range if it wants to use said range
13:25 hecks     in practice, ram usage shouldn't soar - active clients load the whole thing anyway, only to forget it for a dumb reason
13:25 hecks     unless your mouse is welded to your desk, you will probably load your entire LOS anyway
13:28 pgimeno   I agree that using the frustum for caching does not make sense, because you can turn at any moment
13:30 pgimeno   also, I may be wrong, but I think that the number of extra blocks to keep in memory for this reason is relatively small in comparison with the 10 minutes rule
13:30 hecks     You mean the default setting for the client block cache?
13:31 hecks     because that's not even enough to contain the visible stuff, see https://github.com/minetest/minetest/issues/10567#issuecomment-718144411
13:32 pgimeno   yes, the default 10 minutes for the cache to expire blocks
13:32 hecks     with this, one can unload way more frequently than 10 minutes
13:33 hecks     which is actually a decent strategy to reduce unload related stalls, don't let the work build up
13:36 pgimeno   I don't understand, 5000 blocks? that's a misery and provably wrong (just look back at everything you have visited)
13:36 pgimeno   can it be 5000 mapblocks?
13:36 hecks     blocks in this case mean mapblocks
13:36 hecks     but even 5k mapblocks aren't enough to cover your vision range
13:36 hecks     the math is in that thread
13:37 pgimeno   with what range?
13:37 pgimeno   ah 200
13:37 pgimeno   I get 2000 mapblocks
13:37 pgimeno   er wrong
13:37 hecks     here's how you test this properly
13:38 pgimeno   yeah 8x that, 15625
13:38 hecks     make the cache huge so that it doesn't constrain anything, stand in one place with your chosen vision range, rotate the camera around
13:38 hecks     or just compute the volume of a sphere
13:38 hecks     16k is a comfortable cache size and doesn't even eat that much ram
13:39 hecks     most of the blocks are air or solid and have no meshes to speak of
13:42 pgimeno   ok, so IIUC, what you suggest seems to imply that we would basically cache a "wormhole" of cubes that the player goes through, with no regards to whether the player has looked at them at any time
13:44 hecks     I just say that the client should keep a sphere of mapblocks around them loaded because the frustum shouldn't be at all relevant to this
13:44 hecks     just pretend that the client has 360 degree vision
13:45 hecks     and honestly, the server should also attempt to send the blocks that are behind a player, even if at a lower priority
13:45 hecks     instead of waiting until the player inevitably turns around and sees a gaping void in the world
13:46 pgimeno   yes, load priority wise, the frustum should be given priority
13:47 hecks     yup, but the blocks outside of it should be sent if there's opportunity for it
13:47 pgimeno   right
15:09 hecks     What do I have to do to hit this code path?
15:09 hecks     https://github.com/minetest/minetest/blob/4d41ed09750c7a2fbfeeeccb7a2c3452e3dd26dc/src/serverenvironment.cpp#L2080
15:12 hecks     oh it looks like another player must be observing an entity death
15:12 hecks     no wait
15:13 hecks     I literally cannot get that branch to execute
15:14 sfan5     I assume your client is told about the entity removal fast enough that waiting for it is never a problem
15:40 hecks     oh yeah, it's localhost
15:40 hecks     oh well, I found a way to deal with this regardless
15:55 pgimeno   hecks: the ratio from cube to sphere in volume is approximately 2:1 (from 4/3*pi*1³ ~= 4.1888, (2*1)³=8, 8/4.1888 ~= 1.91)
15:55 hecks     yes, and?
15:55 pgimeno   so, you can about halve the number of blocks that need to be cached
15:56 hecks     that's... not how you compute this at all
15:56 hecks     just slash your radius by 16 and compute a sphere's volume, this is how many blocks you need
15:56 hecks     I mean, divide
15:57 pgimeno   that's 8181 for a radius of 200
15:57 hecks     refer to that thread again
15:58 pgimeno   how do you calculate the volume of a sphere?
15:58 hecks     4/3πr³
15:59 hecks     where r=range/MAP_BLOCKSIZE
15:59 hecks     possibly +1 for healthy padding
15:59 pgimeno   >>> 4/3*pi*(200/16)**3
15:59 pgimeno   8181.230868723419
15:59 hecks     now add one block of padding
16:00 pgimeno   (I use Python as a calculator)
16:00 hecks     with one more block you get 10k
16:00 hecks     with two, 12k
16:01 hecks     in any case 5k is not enough
16:01 pgimeno   you mean (200/16 + 1)**3?
16:01 hecks     yeah, in my original calculations I also used ceil(200/16)
16:02 pgimeno   right
16:02 hecks     ceil and +1 is 11.3k
16:02 hecks     pretty consistent with what I actually measured, which was 12k
16:02 hecks     like that's how many blocks the server actually sent me
16:03 pgimeno   ok
16:03 hecks     you can test it yourself, set a huge block cache, turn off server side culling for a moment, log in and just look around
16:03 hecks     if you turn the culling back on it'll be less by about one third *but* you shouldn't ever rely on culling
16:04 hecks     just make a tall enough tower and culling won't help you
16:05 hecks     most of those blocks will be air, but they will count towards the cache
16:06 pgimeno   with sqrt(3) padding (which I believe is the correct padding) I get 12080
16:07 hecks     and that's pretty much how many blocks the server will send you =]
16:08 hecks     in any case, I play with a 32k block cache and very aggressive unloads
16:08 hecks     it works well
16:33 hecks     #10723 please have a look
16:33 ShadowBot https://github.com/minetest/minetest/issues/10723 -- Add on_deactivate callback for luaentities by hecktest
17:11 lhofhansl sfan5: You cool with the updated #10715?
17:11 ShadowBot https://github.com/minetest/minetest/issues/10715 -- Allow configuring block disk and net compression. Change default disk level. by lhofhansl
17:56 Krock     will merge #10697 and #10679 in 10 minutes
17:56 ShadowBot https://github.com/minetest/minetest/issues/10697 -- add mod_orgin to node def in lua_api.txt by wsor4035
17:56 ShadowBot https://github.com/minetest/minetest/issues/10679 -- Formspec: Allow to specify frame loop for model[] by Thomas--S
18:01 sfan5     lhofhansl: yea
18:02 Krock     hecks: why do culled blocks need animations? https://github.com/minetest/minetest/pull/10714/files#diff-338620f4b79e1a00e394bb90133eb962a50bfa461512d619452e879a38ae8bacL313
18:03 Krock     oh wait. that seems to be the same m_drawlist vector
18:03 hecks     yes and the other use of the frustum culler was literally just to get the distance from it
18:04 Krock     :/
18:05 Krock     merging (2)
18:06 lhofhansl sfan5: Thanks! Merging #10715 in a few.
18:06 ShadowBot https://github.com/minetest/minetest/issues/10715 -- Allow configuring block disk and net compression. Change default disk level. by lhofhansl
18:09 lhofhansl The frustum culler isn't expensive, I doubt we'll see any measurable change from it - still good to change it, though.
18:09 hecks     it just hurt to look at
18:09 lhofhansl hehe
18:09 Krock     it's mainly about the code quality, not about speed IMO
18:11 lhofhansl speaking on which, I find the hardcoded distance logic for animated meshes right below it highly dubious.
18:11 Krock     also. opinions on #10458 ? whereas it does not fix all inputs, it certainly helps to input some special chars
18:11 lhofhansl of
18:11 ShadowBot https://github.com/minetest/minetest/issues/10458 -- Chat Input: Allow more input keys by SmallJoker
18:12 Krock     "Pretty random but this should work somewhat nicely" seems good enough to me
18:13 Krock     unless you'd want to use 17 FOV on a 8k monitor...
18:13 lhofhansl Seems to depend on your viewing_range, etc.
18:23 hecks     I noticed no difference from 10458
18:24 hecks     out of the following chars in my keyboard layout: ¹²³⁴⁵⁶⁷⁸⁹⁰₁₂₃₄₅₆₇₈₉₀⁻⁺₋₊≈·× only the dot and cross work in MT chat
18:24 hecks     while 0 superscript is somehow turned to a degree symbol
18:25 Krock     are those created using dead keys/compose characters? i.e. ^ or `
18:27 hecks     no, just custom altgr combos
18:27 Krock     I observed that it only fixes "Could not find EKEY_CODE, using orig. X11 keycode instead" inputs, where Irrlicht cannot map them somehow
18:28 hecks     actual grave accents work even in master for me, though I still don't use dead keys to type them, I removed that from my layout
18:29 Krock     hmm. cannot input those here, although key 35 now works (!, ], ")
18:29 kilbith   how hard it is to move away from zlib to zstd? is this just about changing `(de)compressZlib` to `ZSTD_(de)compress` or there's a compat layer to build up for the old compressed data?
18:30 Krock     check against the mapblock ser ver
18:30 Krock     but that might also be packed into the zlib blob
18:32 Krock     actually no.  ServerMap::saveBlock does not do any further compressing. it happens in  MapNode::serializeBulk and MapBlock::serialize
18:33 Krock     so yes, you could bump the mapblock ser ver to switch the compression algorithm
18:35 Krock     kilbith: either this would be done incrementally, or similar to #10293 for an entire migration
18:35 ShadowBot https://github.com/minetest/minetest/issues/10293 -- Add /upgrade_mapblocks command by SmallJoker
18:36 Krock     also yes, you need to load a real Server (without network tho) to migrate a map
18:36 Krock     the backend switch in minetestserver only copies the data blob over, which is not sufficient here
19:12 lhofhansl https://github.com/minetest/minetest/pull/10715#issuecomment-743781540
19:13 lhofhansl Zlib is quite slow. There's at least Snappy, Brotli, and zstd to try out.
19:14 sfan5     i suggest looking at the "switch to brotli/zstd" issue
19:14 sfan5     or was it a pull?
19:14 lhofhansl I would write an extra byte for each block that declares the compression used. That way no migration is needed.
19:15 kilbith   zstd beats out the hell of everything according to various benchmarks
19:15 lhofhansl (well, need to detect when that byte is absent)
19:15 lhofhansl yep. We're using it at work to compress our databases. :)
19:23 kilbith   #4495
19:23 ShadowBot https://github.com/minetest/minetest/issues/4495 -- Using Zstandard for mapblock compression?
21:01 lhofhansl #10724
21:01 ShadowBot https://github.com/minetest/minetest/issues/10724 -- Enforce a 100ms time budget per server step for map save. by lhofhansl