| Time |
Nick |
Message |
| 01:23 |
|
lhofhansl joined #minetest-dev |
| 01:56 |
lhofhansl |
Looks like part of the trick with MAP_BLOCKSIZE=32 is to reduce max_simultaneous_block_sends_per_client to 5 or 10. Along with chunksize = 3 it all works reasonable. (Again, not saying we do that. Blocksize = 16 is good. Just if folks want to see the possible FPS at higher view ranges) |
| 03:36 |
|
systwi_ joined #minetest-dev |
| 03:57 |
|
lhofhansl joined #minetest-dev |
| 03:59 |
|
olliy joined #minetest-dev |
| 05:00 |
|
MTDiscord joined #minetest-dev |
| 05:14 |
lhofhansl |
I found it very helpful to spread map saving out over multiple server steps. Steps of 100ms seem to be quite nice. The server stays responsive, and can get some packets for the client in while the map is being saved. It's very simple and quite effective. I'll have PR soon (waiting for 10715 so I can avoid rebasing). |
| 07:08 |
|
YuGiOhJCJ joined #minetest-dev |
| 08:00 |
|
ShadowNinja joined #minetest-dev |
| 08:36 |
|
hecks joined #minetest-dev |
| 08:47 |
|
calcul0n joined #minetest-dev |
| 09:46 |
|
proller joined #minetest-dev |
| 12:21 |
|
hecks joined #minetest-dev |
| 12:28 |
|
Fixer joined #minetest-dev |
| 13:02 |
|
Wuzzy joined #minetest-dev |
| 13:06 |
pgimeno |
will #10714 (or further memory management changes if it gets applied) eliminate the possibility to look back at what you have recently visited by enabling Full Range View? I consider that a feature |
| 13:06 |
ShadowBot |
https://github.com/minetest/minetest/issues/10714 -- Keep mapblocks in memory if they're in range by hecktest |
| 13:08 |
|
turtleman joined #minetest-dev |
| 13:17 |
hecks |
pgimeno: no, it will simply keep blocks in the player's LOS alive unconditionally |
| 13:18 |
hecks |
blocks outside of the vision sphere will behave as they used to |
| 13:18 |
hecks |
but wait |
| 13:18 |
hecks |
I'll check it just in case |
| 13:19 |
hecks |
if (!m_control.range_all && |
| 13:19 |
hecks |
the range check doesn't even happen if you have "unlimited range" enabled |
| 13:20 |
hecks |
of course as soon as you turn that off, the clock will start ticking for out-of-range objects again |
| 13:20 |
hecks |
all I did was make the clock reset itself before the frustum cull, not after |
| 13:20 |
pgimeno |
will it start or resume? |
| 13:20 |
hecks |
restart |
| 13:21 |
pgimeno |
oh so enabling unlimited range resets the timer? |
| 13:21 |
pgimeno |
that's good to know, thanks |
| 13:21 |
hecks |
yup |
| 13:21 |
hecks |
basically the usage timer is reset every time a block is "used" by the renderer |
| 13:21 |
hecks |
before, it happened after a frustum cull and allowed unloading a block that was right behind your back |
| 13:22 |
pgimeno |
I heard something about the current code preventing future cache optimizations and got worried that those future cache optimizations could affect that mechanic |
| 13:22 |
hecks |
caching blocks should ideally be reworked around this |
| 13:23 |
hecks |
to only consider things outside of the range for caching, though it implicitly works that way right now anyway |
| 13:23 |
hecks |
it's just a matter of semantics and whether you want the ram limits in config to count the vision sphere or not |
| 13:23 |
hecks |
my strong opinion on this is that attempting to do anything clever with the frustum in this context was idiotic |
| 13:24 |
hecks |
and the client should always be prepared to handle all the blocks within range if it wants to use said range |
| 13:25 |
hecks |
in practice, ram usage shouldn't soar - active clients load the whole thing anyway, only to forget it for a dumb reason |
| 13:25 |
hecks |
unless your mouse is welded to your desk, you will probably load your entire LOS anyway |
| 13:28 |
pgimeno |
I agree that using the frustum for caching does not make sense, because you can turn at any moment |
| 13:30 |
pgimeno |
also, I may be wrong, but I think that the number of extra blocks to keep in memory for this reason is relatively small in comparison with the 10 minutes rule |
| 13:30 |
hecks |
You mean the default setting for the client block cache? |
| 13:31 |
hecks |
because that's not even enough to contain the visible stuff, see https://github.com/minetest/minetest/issues/10567#issuecomment-718144411 |
| 13:32 |
pgimeno |
yes, the default 10 minutes for the cache to expire blocks |
| 13:32 |
hecks |
with this, one can unload way more frequently than 10 minutes |
| 13:33 |
hecks |
which is actually a decent strategy to reduce unload related stalls, don't let the work build up |
| 13:36 |
pgimeno |
I don't understand, 5000 blocks? that's a misery and provably wrong (just look back at everything you have visited) |
| 13:36 |
pgimeno |
can it be 5000 mapblocks? |
| 13:36 |
hecks |
blocks in this case mean mapblocks |
| 13:36 |
hecks |
but even 5k mapblocks aren't enough to cover your vision range |
| 13:36 |
hecks |
the math is in that thread |
| 13:37 |
pgimeno |
with what range? |
| 13:37 |
pgimeno |
ah 200 |
| 13:37 |
pgimeno |
I get 2000 mapblocks |
| 13:37 |
pgimeno |
er wrong |
| 13:37 |
hecks |
here's how you test this properly |
| 13:38 |
pgimeno |
yeah 8x that, 15625 |
| 13:38 |
hecks |
make the cache huge so that it doesn't constrain anything, stand in one place with your chosen vision range, rotate the camera around |
| 13:38 |
hecks |
or just compute the volume of a sphere |
| 13:38 |
hecks |
16k is a comfortable cache size and doesn't even eat that much ram |
| 13:39 |
hecks |
most of the blocks are air or solid and have no meshes to speak of |
| 13:42 |
pgimeno |
ok, so IIUC, what you suggest seems to imply that we would basically cache a "wormhole" of cubes that the player goes through, with no regards to whether the player has looked at them at any time |
| 13:44 |
hecks |
I just say that the client should keep a sphere of mapblocks around them loaded because the frustum shouldn't be at all relevant to this |
| 13:44 |
hecks |
just pretend that the client has 360 degree vision |
| 13:45 |
hecks |
and honestly, the server should also attempt to send the blocks that are behind a player, even if at a lower priority |
| 13:45 |
hecks |
instead of waiting until the player inevitably turns around and sees a gaping void in the world |
| 13:46 |
pgimeno |
yes, load priority wise, the frustum should be given priority |
| 13:47 |
hecks |
yup, but the blocks outside of it should be sent if there's opportunity for it |
| 13:47 |
pgimeno |
right |
| 14:14 |
|
olliy joined #minetest-dev |
| 14:26 |
|
lisac joined #minetest-dev |
| 14:39 |
|
hecks joined #minetest-dev |
| 14:54 |
|
Fixer_ joined #minetest-dev |
| 15:09 |
hecks |
What do I have to do to hit this code path? |
| 15:09 |
hecks |
https://github.com/minetest/minetest/blob/4d41ed09750c7a2fbfeeeccb7a2c3452e3dd26dc/src/serverenvironment.cpp#L2080 |
| 15:12 |
hecks |
oh it looks like another player must be observing an entity death |
| 15:12 |
hecks |
no wait |
| 15:13 |
hecks |
I literally cannot get that branch to execute |
| 15:14 |
sfan5 |
I assume your client is told about the entity removal fast enough that waiting for it is never a problem |
| 15:40 |
hecks |
oh yeah, it's localhost |
| 15:40 |
hecks |
oh well, I found a way to deal with this regardless |
| 15:42 |
|
calcul0n_ joined #minetest-dev |
| 15:55 |
pgimeno |
hecks: the ratio from cube to sphere in volume is approximately 2:1 (from 4/3*pi*1³ ~= 4.1888, (2*1)³=8, 8/4.1888 ~= 1.91) |
| 15:55 |
hecks |
yes, and? |
| 15:55 |
pgimeno |
so, you can about halve the number of blocks that need to be cached |
| 15:56 |
hecks |
that's... not how you compute this at all |
| 15:56 |
hecks |
just slash your radius by 16 and compute a sphere's volume, this is how many blocks you need |
| 15:56 |
hecks |
I mean, divide |
| 15:57 |
pgimeno |
that's 8181 for a radius of 200 |
| 15:57 |
hecks |
refer to that thread again |
| 15:58 |
pgimeno |
how do you calculate the volume of a sphere? |
| 15:58 |
hecks |
4/3πr³ |
| 15:59 |
hecks |
where r=range/MAP_BLOCKSIZE |
| 15:59 |
hecks |
possibly +1 for healthy padding |
| 15:59 |
pgimeno |
>>> 4/3*pi*(200/16)**3 |
| 15:59 |
pgimeno |
8181.230868723419 |
| 15:59 |
hecks |
now add one block of padding |
| 16:00 |
pgimeno |
(I use Python as a calculator) |
| 16:00 |
hecks |
with one more block you get 10k |
| 16:00 |
hecks |
with two, 12k |
| 16:01 |
hecks |
in any case 5k is not enough |
| 16:01 |
pgimeno |
you mean (200/16 + 1)**3? |
| 16:01 |
hecks |
yeah, in my original calculations I also used ceil(200/16) |
| 16:02 |
pgimeno |
right |
| 16:02 |
hecks |
ceil and +1 is 11.3k |
| 16:02 |
hecks |
pretty consistent with what I actually measured, which was 12k |
| 16:02 |
hecks |
like that's how many blocks the server actually sent me |
| 16:03 |
pgimeno |
ok |
| 16:03 |
hecks |
you can test it yourself, set a huge block cache, turn off server side culling for a moment, log in and just look around |
| 16:03 |
hecks |
if you turn the culling back on it'll be less by about one third *but* you shouldn't ever rely on culling |
| 16:04 |
hecks |
just make a tall enough tower and culling won't help you |
| 16:05 |
hecks |
most of those blocks will be air, but they will count towards the cache |
| 16:06 |
pgimeno |
with sqrt(3) padding (which I believe is the correct padding) I get 12080 |
| 16:07 |
hecks |
and that's pretty much how many blocks the server will send you =] |
| 16:08 |
hecks |
in any case, I play with a 32k block cache and very aggressive unloads |
| 16:08 |
hecks |
it works well |
| 16:33 |
hecks |
#10723 please have a look |
| 16:33 |
ShadowBot |
https://github.com/minetest/minetest/issues/10723 -- Add on_deactivate callback for luaentities by hecktest |
| 17:01 |
|
lhofhansl joined #minetest-dev |
| 17:09 |
|
calcul0n__ joined #minetest-dev |
| 17:11 |
lhofhansl |
sfan5: You cool with the updated #10715? |
| 17:11 |
ShadowBot |
https://github.com/minetest/minetest/issues/10715 -- Allow configuring block disk and net compression. Change default disk level. by lhofhansl |
| 17:31 |
|
hecks joined #minetest-dev |
| 17:50 |
|
lhofhansl joined #minetest-dev |
| 17:56 |
Krock |
will merge #10697 and #10679 in 10 minutes |
| 17:56 |
ShadowBot |
https://github.com/minetest/minetest/issues/10697 -- add mod_orgin to node def in lua_api.txt by wsor4035 |
| 17:56 |
ShadowBot |
https://github.com/minetest/minetest/issues/10679 -- Formspec: Allow to specify frame loop for model[] by Thomas--S |
| 18:01 |
sfan5 |
lhofhansl: yea |
| 18:02 |
Krock |
hecks: why do culled blocks need animations? https://github.com/minetest/minetest/pull/10714/files#diff-338620f4b79e1a00e394bb90133eb962a50bfa461512d619452e879a38ae8bacL313 |
| 18:03 |
Krock |
oh wait. that seems to be the same m_drawlist vector |
| 18:03 |
hecks |
yes and the other use of the frustum culler was literally just to get the distance from it |
| 18:04 |
Krock |
:/ |
| 18:05 |
Krock |
merging (2) |
| 18:06 |
lhofhansl |
sfan5: Thanks! Merging #10715 in a few. |
| 18:06 |
ShadowBot |
https://github.com/minetest/minetest/issues/10715 -- Allow configuring block disk and net compression. Change default disk level. by lhofhansl |
| 18:09 |
lhofhansl |
The frustum culler isn't expensive, I doubt we'll see any measurable change from it - still good to change it, though. |
| 18:09 |
hecks |
it just hurt to look at |
| 18:09 |
lhofhansl |
hehe |
| 18:09 |
Krock |
it's mainly about the code quality, not about speed IMO |
| 18:11 |
lhofhansl |
speaking on which, I find the hardcoded distance logic for animated meshes right below it highly dubious. |
| 18:11 |
Krock |
also. opinions on #10458 ? whereas it does not fix all inputs, it certainly helps to input some special chars |
| 18:11 |
lhofhansl |
of |
| 18:11 |
ShadowBot |
https://github.com/minetest/minetest/issues/10458 -- Chat Input: Allow more input keys by SmallJoker |
| 18:12 |
Krock |
"Pretty random but this should work somewhat nicely" seems good enough to me |
| 18:13 |
Krock |
unless you'd want to use 17 FOV on a 8k monitor... |
| 18:13 |
lhofhansl |
Seems to depend on your viewing_range, etc. |
| 18:23 |
hecks |
I noticed no difference from 10458 |
| 18:24 |
|
kilbith joined #minetest-dev |
| 18:24 |
hecks |
out of the following chars in my keyboard layout: ¹²³⁴⁵⁶⁷⁸⁹⁰₁₂₃₄₅₆₇₈₉₀⁻⁺₋₊≈·× only the dot and cross work in MT chat |
| 18:24 |
hecks |
while 0 superscript is somehow turned to a degree symbol |
| 18:25 |
Krock |
are those created using dead keys/compose characters? i.e. ^ or ` |
| 18:27 |
hecks |
no, just custom altgr combos |
| 18:27 |
Krock |
I observed that it only fixes "Could not find EKEY_CODE, using orig. X11 keycode instead" inputs, where Irrlicht cannot map them somehow |
| 18:28 |
hecks |
actual grave accents work even in master for me, though I still don't use dead keys to type them, I removed that from my layout |
| 18:29 |
Krock |
hmm. cannot input those here, although key 35 now works (!, ], ") |
| 18:29 |
kilbith |
how hard it is to move away from zlib to zstd? is this just about changing `(de)compressZlib` to `ZSTD_(de)compress` or there's a compat layer to build up for the old compressed data? |
| 18:30 |
Krock |
check against the mapblock ser ver |
| 18:30 |
Krock |
but that might also be packed into the zlib blob |
| 18:32 |
Krock |
actually no. ServerMap::saveBlock does not do any further compressing. it happens in MapNode::serializeBulk and MapBlock::serialize |
| 18:33 |
Krock |
so yes, you could bump the mapblock ser ver to switch the compression algorithm |
| 18:33 |
|
homthack joined #minetest-dev |
| 18:35 |
Krock |
kilbith: either this would be done incrementally, or similar to #10293 for an entire migration |
| 18:35 |
ShadowBot |
https://github.com/minetest/minetest/issues/10293 -- Add /upgrade_mapblocks command by SmallJoker |
| 18:36 |
Krock |
also yes, you need to load a real Server (without network tho) to migrate a map |
| 18:36 |
Krock |
the backend switch in minetestserver only copies the data blob over, which is not sufficient here |
| 18:42 |
|
calcul0n joined #minetest-dev |
| 18:47 |
|
olliy joined #minetest-dev |
| 18:51 |
|
proller joined #minetest-dev |
| 18:54 |
|
fluxflux joined #minetest-dev |
| 19:12 |
lhofhansl |
https://github.com/minetest/minetest/pull/10715#issuecomment-743781540 |
| 19:13 |
lhofhansl |
Zlib is quite slow. There's at least Snappy, Brotli, and zstd to try out. |
| 19:14 |
sfan5 |
i suggest looking at the "switch to brotli/zstd" issue |
| 19:14 |
sfan5 |
or was it a pull? |
| 19:14 |
lhofhansl |
I would write an extra byte for each block that declares the compression used. That way no migration is needed. |
| 19:15 |
kilbith |
zstd beats out the hell of everything according to various benchmarks |
| 19:15 |
lhofhansl |
(well, need to detect when that byte is absent) |
| 19:15 |
lhofhansl |
yep. We're using it at work to compress our databases. :) |
| 19:23 |
kilbith |
#4495 |
| 19:23 |
ShadowBot |
https://github.com/minetest/minetest/issues/4495 -- Using Zstandard for mapblock compression? |
| 19:27 |
|
kilbith joined #minetest-dev |
| 19:48 |
|
kilbith joined #minetest-dev |
| 20:06 |
|
Fixer joined #minetest-dev |
| 20:10 |
|
homthack joined #minetest-dev |
| 20:11 |
|
amk joined #minetest-dev |
| 20:14 |
|
homthack82 joined #minetest-dev |
| 20:15 |
|
olliy joined #minetest-dev |
| 20:32 |
|
Fixer_ joined #minetest-dev |
| 20:39 |
|
proller joined #minetest-dev |
| 20:40 |
|
Fixer joined #minetest-dev |
| 20:41 |
|
Wuzzy joined #minetest-dev |
| 20:51 |
|
kilbith joined #minetest-dev |
| 21:01 |
lhofhansl |
#10724 |
| 21:01 |
ShadowBot |
https://github.com/minetest/minetest/issues/10724 -- Enforce a 100ms time budget per server step for map save. by lhofhansl |
| 21:04 |
|
kilbith_ joined #minetest-dev |
| 21:06 |
|
kilbith__ joined #minetest-dev |
| 22:13 |
|
nathanfranke[m] joined #minetest-dev |
| 23:07 |
|
lhofhansl joined #minetest-dev |
| 23:08 |
|
Taoki joined #minetest-dev |
| 23:26 |
|
Taoki joined #minetest-dev |
