| Time |
Nick |
Message |
| 00:00 |
lhofhansl |
but why do we need to store them. When the block is loaded time is ticking. When we load a block we set the time to now. |
| 00:00 |
lhofhansl |
I guess we do not want to miss timers firing. |
| 00:00 |
lhofhansl |
when a block got unloaded and then reloaded |
| 00:02 |
sfan5 |
https://github.com/minetest/minetest/blob/3176daee79a8e91bd16fc275704f49fe1648db32/src/serverenvironment.cpp#L947-L961 |
| 00:02 |
sfan5 |
anyway, bedtime for me |
| 00:04 |
hecks |
waaaait |
| 00:04 |
hecks |
one question |
| 00:04 |
hecks |
if I make a packet unreliable on the server side only, will the clients explode? |
| 00:05 |
lhofhansl |
sfan5: heh, good night. |
| 00:05 |
sfan5 |
the client doesn't care which channel a packet arrives on or if it's set reliable |
| 00:05 |
hecks |
so I can apply this dirty fix server side? |
| 00:06 |
sfan5 |
yes |
| 00:06 |
hecks |
thanks =] |
| 00:11 |
|
twoelk left #minetest-dev |
| 00:23 |
lhofhansl |
hecks: what's the "dirty fix"? |
| 00:24 |
hecks |
make animations unreliable, for my queue problems |
| 00:25 |
hecks |
it's not the end of the world if an animation fails to play, it is if lost anim packets clog up the reliable queue |
| 00:25 |
hecks |
lhofhansl, which continent are you on? |
| 00:28 |
MTDiscord |
<oneplustwo> probably europe |
| 00:28 |
hecks |
hm, it seems that the dirty fix worked |
| 00:29 |
hecks |
I'm starting to doubt if animation packets ever needed reliable delivery |
| 00:30 |
hecks |
oh wait |
| 00:32 |
hecks |
okay, now it's confirmed that it works |
| 00:38 |
pgimeno |
would database compression changes have an effect on server bandwidth? |
| 00:39 |
hecks |
you could always compress harder in transit if that were ever an issue |
| 00:44 |
MTDiscord |
<exe_virus> So, if I were to make a PR that forces .x, .obj, and .b3d to go through zlib compression and decompression to help with bandwidth load for new clients, would that create a breaking change requiring version 6.x? |
| 00:45 |
MTDiscord |
<exe_virus> Thinking is that compression for these using standard deflate is about 30% of original size or smaller. |
| 00:51 |
hecks |
ah, crap, making animations unreliable did not entirely fix this |
| 00:52 |
hecks |
something is still creating a black hole in the packet queue |
| 01:08 |
hecks |
okay, so in many cases, only the last packet really matters |
| 01:09 |
hecks |
maybe if the queue grows too large, the server could walk the whole thing and squash it to only what is actually important? |
| 01:12 |
lhofhansl |
hecks: I'm US west coast. |
| 01:27 |
|
turtleman joined #minetest-dev |
| 01:53 |
hecks |
this is odd |
| 01:53 |
hecks |
we tested the same game on the same connection months before, it worked and didn't chug this badly |
| 02:00 |
hecks |
has anybody touched SAO messages, the packet queue or default settings related to bandwidth? |
| 02:03 |
lhofhansl |
I increased the default packet queue sizes recently. |
| 02:08 |
|
T4im joined #minetest-dev |
| 02:13 |
hecks |
what were they before? |
| 02:18 |
lhofhansl |
https://github.com/minetest/minetest/pull/10627 |
| 02:18 |
lhofhansl |
If that causes a lag we have a serious problem. |
| 02:59 |
hecks |
it did get better when I nerfed that |
| 03:00 |
hecks |
I might still have to fix the queue thing... it's not going to go away no matter how you budget the up bandwidth |
| 03:01 |
hecks |
it would be very helpful if you were to connect again and tell me if there's any difference |
| 04:55 |
|
lhofhansl joined #minetest-dev |
| 05:00 |
|
MTDiscord joined #minetest-dev |
| 07:22 |
|
m42uko joined #minetest-dev |
| 08:00 |
|
ShadowNinja joined #minetest-dev |
| 08:11 |
|
NetherEran joined #minetest-dev |
| 09:45 |
|
calcul0n joined #minetest-dev |
| 10:32 |
|
calcul0n_ joined #minetest-dev |
| 10:38 |
|
proller joined #minetest-dev |
| 12:25 |
|
Wuzzy joined #minetest-dev |
| 12:40 |
|
Fixer joined #minetest-dev |
| 12:41 |
zughy[m] |
uhm, a ton of PRs result being updated 3 hours ago, we're talking about 5 pages of PRs, when they actually didn't. What happened exactly? |
| 12:42 |
zughy[m] |
the only thing happened at that time is this: https://github.com/minetest/minetest/commit/ecd4f45318e7e510ebe4ebe8420ea739122d2edf |
| 12:43 |
zughy[m] |
exactly one minute before |
| 12:43 |
sfan5 |
github bug, it does that when you force-push master |
| 13:08 |
|
Fixer_ joined #minetest-dev |
| 13:54 |
|
proller joined #minetest-dev |
| 13:56 |
|
turtleman joined #minetest-dev |
| 14:18 |
|
lisac joined #minetest-dev |
| 15:09 |
|
Taoki joined #minetest-dev |
| 17:49 |
|
absurb joined #minetest-dev |
| 18:24 |
|
homthack joined #minetest-dev |
| 18:28 |
|
homthack13 joined #minetest-dev |
| 18:39 |
|
homthack joined #minetest-dev |
| 18:42 |
|
mizux joined #minetest-dev |
| 18:51 |
|
homthack joined #minetest-dev |
| 19:00 |
Krock |
will merge #9247, #10633 and #10663 in 15 minutes |
| 19:01 |
ShadowBot |
https://github.com/minetest/minetest/issues/9247 -- Fix MSAA stripes by HybridDog |
| 19:01 |
ShadowBot |
https://github.com/minetest/minetest/issues/10633 -- Cross-reference the node level manipulation functions by Oblomov |
| 19:01 |
ShadowBot |
https://github.com/minetest/minetest/issues/10663 -- Cleanup shader generation code by numberZero |
| 19:15 |
Krock |
merging |
| 19:16 |
|
fluxflux joined #minetest-dev |
| 19:17 |
Krock |
last PR failed. new merge conflict \o/ |
| 19:23 |
|
lhofhansl joined #minetest-dev |
| 19:24 |
lhofhansl |
After 10663 I |
| 19:25 |
lhofhansl |
let's have a look at #10688 |
| 19:25 |
ShadowBot |
https://github.com/minetest/minetest/issues/10688 -- Remove hardcoded matrices from OpenGL shaders. by lhofhansl |
| 19:34 |
Krock |
!tell lhofhansl None, though I'm not familiar with shaders to judge whether it's a good concept |
| 19:34 |
ShadowBot |
Krock: O.K. |
| 20:04 |
|
proller joined #minetest-dev |
| 20:15 |
|
hecks joined #minetest-dev |
| 20:24 |
hecks |
Tell me about packet channels, how hard would it be to allocate a new one and move some traffic to it? |
| 20:32 |
Krock |
easy |
| 20:32 |
Krock |
see networkprotocol.h or so |
| 20:34 |
Krock |
hecks: clientopcodes.cpp, and server respectively |
| 20:40 |
hecks |
so you just change the channel number and that makes a new queue? |
| 20:40 |
Krock |
yes |
| 20:40 |
hecks |
thanks |
| 20:40 |
|
wsor joined #minetest-dev |
| 20:40 |
hecks |
I'm gonna experiment with splitting the big queue because most of those messages do not need to wait for one another |
| 20:41 |
Krock |
please keep in mind that some actions must be in order so that mods work properly, such as object properties and attachments |
| 20:42 |
hecks |
I know, but those things do not need to wait, for example, on chat or forms |
| 20:42 |
hecks |
or skybox operations, camera parameters |
| 20:44 |
MTDiscord |
<Jonathon> hello: was wondering if i can get some input from the core devs on https://github.com/minetest/minetest/pull/10697 if you dont mind? |
| 20:44 |
hecks |
the server sends out an order of magnitude more data per client than it receives, and this out stream must be managed wisely because it's too easy to clog it beyond repair |
| 20:57 |
celeron55 |
i'm a bit freaked out about https://github.com/minetest/minetest/pull/10627 actually |
| 20:58 |
Krock |
more traffic |
| 20:58 |
celeron55 |
there's a reason for the original limit, and it's exactly to allow the server to work in most network environments |
| 20:59 |
celeron55 |
really shouldn't just increase it based on some tests in a local or good network |
| 20:59 |
celeron55 |
and based on tests that don't include trying to animate stuff at the same time or such |
| 21:00 |
celeron55 |
and yes, the network stack is crap, something like enet would handle more stuff better, but not infinitely better |
| 21:01 |
celeron55 |
it's 100% certain that if you optimize for the rate of sending mapblocks without looking any other statistics or trying to do anything else at the same time, everything else _will_ work badly with the configuration |
| 21:03 |
celeron55 |
it's like pressing the gas until you can't steer |
| 21:04 |
celeron55 |
it could be dynamic though |
| 21:10 |
Krock |
the mapblock sending is dynamic to the connected players |
| 21:11 |
Krock |
-> https://github.com/minetest/minetest/pull/6393 |
| 21:14 |
hecks |
https://pastebin.com/SbWtmqV8 What do you think of these categories for s->c traffic? |
| 21:14 |
Krock |
looks like an ENUM is needed in the code |
| 21:14 |
hecks |
for the channels? |
| 21:15 |
hecks |
I can do that |
| 21:15 |
Krock |
yeah |
| 21:15 |
hecks |
but how's this for a draft of message categories, any logical conflicts here? |
| 21:15 |
Krock |
I'm sure there will be some obscure edge-cases but overall it seems okay |
| 21:18 |
Krock |
what happens if you spawn an entity and set its position right after? |
| 21:19 |
Krock |
afaik it's contained in TOCLIENT_ACTIVE_OBJECT_MESSAGES, so that would be okay |
| 21:22 |
Krock |
or sending a formspec after creating the node. item lists should re-generate themselves on refresh, though that might only happen after resizing the window or pressing a button |
| 21:24 |
Krock |
to be fair, most of those packets are only used once, especially the auth stuff will never be called twice per session |
| 21:30 |
|
lhofhansl joined #minetest-dev |
| 21:35 |
lhofhansl |
Looks like I should just undo #10627...? Too much discussion about it. |
| 21:35 |
ShadowBot |
https://github.com/minetest/minetest/issues/10627 -- Increase limit for simultaneous blocks sent per client and the meshgen cache. by lhofhansl |
| 21:36 |
lhofhansl |
The intent was to allow the server CPU to be busy, and not just idly wait for queues to empty out. |
| 21:37 |
lhofhansl |
That is especially an issue with larger client viewing_range (and matching server max_block_send_distance). The current limits seem to low for that. |
| 21:40 |
Krock |
depends on the network structure. celeron55, what would you suggest there? |
| 21:41 |
lhofhansl |
Very related to what hecks is looking at. A single queue for everything is bad. |
| 21:41 |
hecks |
I'm already breaking it up |
| 21:41 |
lhofhansl |
But there's a PR on client stutter already, hecks seen packet build-up, celeron's concern, etc. Perhaps better to revert and then regroup. |
| 21:42 |
lhofhansl |
What I'm looking at is getting MT ready for larger viewing_distances. |
| 21:42 |
hecks |
well, what I'm doing is only tangentially related to your default limit bump, it's a much more fundamental issue |
| 21:43 |
hecks |
the limit bump makes it a little more visible by taking up upload bandwidth from the server, I've had it happen before that too |
| 21:44 |
hecks |
lhofhansl: we could cooperate on the LOD and map renderer rewrite when everything else is ready for it |
| 21:44 |
lhofhansl |
hecks: Yep! |
| 21:45 |
hecks |
the last one was a false start because the block management just wasn't good enough to handle it |
| 21:46 |
lhofhansl |
Perhaps have a separate CLIENT_BLOCKSIZE and then build client blocks from the server blocks (with MAP_BLOCKSIZE). |
| 21:47 |
lhofhansl |
Can be done off the main thread. But then other things are tricky. |
| 21:47 |
hecks |
let's just do the octree thing |
| 21:48 |
hecks |
blocksize 16 is fine for generation and transfer, it's enough to merge LODs into superblocks |
| 21:48 |
hecks |
and then do some atlasing tricks for the closest blocks |
| 21:48 |
lhofhansl |
cool |
| 21:48 |
hecks |
I know that atlasing has been tried and flopped before but I got an idea to make it viable |
| 21:48 |
lhofhansl |
main goal would to reduce the number of draw calls on the client. |
| 21:50 |
lhofhansl |
I can easily draw a scene at range 700 at 60 FPS if set the MAP_BLOCKSIZE to 64. Same number of vertices, etc, just fewer draw calls (it does keep my graphics card more busy) |
| 21:50 |
lhofhansl |
celeron55: so revert #10627 for now? |
| 21:50 |
ShadowBot |
https://github.com/minetest/minetest/issues/10627 -- Increase limit for simultaneous blocks sent per client and the meshgen cache. by lhofhansl |
| 21:51 |
hecks |
we've basically done this whole drawcall and blocksize investigation like half a year ago, so everyone knows what's up |
| 21:51 |
lhofhansl |
It's just a default anyway. Folks can increase it for fast networks. |
| 21:52 |
lhofhansl |
hecks: that one was news to me, but I was gone for a bit too. |
| 21:52 |
lhofhansl |
I realized that MT cannot keep graphics card busy due to all the drawcalls. |
| 21:53 |
hecks |
yup, we're wasting gigaflops of perfectly good silicon |
| 21:53 |
hecks |
as for that default limit thing, I'm not sure if the logic was right on that one |
| 21:53 |
lhofhansl |
how so? |
| 21:53 |
hecks |
I mean, there's nothing wrong if the server can generate blocks faster than they're being sent |
| 21:54 |
lhofhansl |
well, it doesn't. It then waits for the queue to empty, so when I increased that value the map would be sent to the client faster. |
| 21:54 |
hecks |
but as you can see, the upload bandwidth budget is serious business |
| 21:55 |
lhofhansl |
yeah, but that's another issue. That increase brought load time for a scene from 35s to 18s (that on a local network). |
| 21:56 |
lhofhansl |
Anyway, as I said, happy to revert that change. |
| 21:56 |
hecks |
maybe it doesn't need reverting, hang on |
| 21:56 |
lhofhansl |
* hangs on |
| 21:57 |
hecks |
you'd be better off asking people who run actual servers and not 1 mbps poverty boxes |
| 21:57 |
lhofhansl |
or dialup :) |
| 21:58 |
hecks |
there's actually another issue here |
| 21:59 |
hecks |
do network messages have any concept of priority at all? |
| 21:59 |
hecks |
like gameplay before blocks, close blocks before far blocks |
| 22:00 |
Krock |
block updates are already handled this way |
| 22:00 |
hecks |
but do they wait for gameplay? |
| 22:01 |
Krock |
I don't understand. why would block updates wait? |
| 22:01 |
Krock |
they're already a separate channel |
| 22:02 |
hecks |
they shouldn't wait in that sense, but if the server is short on bandwidth, it could prioritize certain channels |
| 22:02 |
hecks |
throttle the block updates a bit to prevent other things from feeling laggy |
| 22:02 |
Krock |
uhm.. I guess it just sends whatever's waiting |
| 22:02 |
hecks |
yup, and that might need to change |
| 22:03 |
hecks |
okay, let's build this monstrosity |
| 22:04 |
hecks |
segfault =] |
| 22:10 |
lhofhansl |
Also there's #10616, which increases (doubles) the default emerge queue sizes, and only allows 1/2 of the total queue size for emege requests on behalf of active objects. |
| 22:10 |
ShadowBot |
https://github.com/minetest/minetest/issues/10616 -- Increase default emerge queue limits and limit enqueue requests for active blocks. by lhofhansl |
| 22:10 |
hecks |
ah, so there *is* a CHANNEL_COUNT |
| 22:16 |
hecks |
okay, localhost seems happy |
| 22:56 |
hecks |
well I'll be damned, this might be working |
| 22:57 |
hecks |
lhofhansl: you got time to try the server again? |
| 23:01 |
lhofhansl |
not now, sorry. What did you change? |
| 23:02 |
hecks |
https://github.com/hecktest/minetest/tree/queue-death |
| 23:03 |
hecks |
there are now 16 channels instead of 3 |
| 23:04 |
hecks |
chat, audio, physics, entities, modchannels, camera, ambience, fx do not wait on one another |
| 23:04 |
hecks |
it seems to prevent a hopeless choke, at least on my setup |
| 23:05 |
sfan5 |
<hecks> https://pastebin.com/SbWtmqV8 What do you think of these categories for s->c traffic? |
| 23:05 |
sfan5 |
seems sane from a quick look but just throwing more channels at the problem isn't done lightly |
| 23:06 |
hecks |
well guess what, I just did it and it worked |
| 23:06 |
sfan5 |
I didn't say it doesn't work |
| 23:06 |
hecks |
=] |
| 23:06 |
sfan5 |
Krock: do you think TOCLIENT_NODEMETA_CHANGED should be on channel 2? I can make a commit if you agree. |
| 23:07 |
sfan5 |
current ones for reference: https://github.com/minetest/minetest/blob/master/src/network/serveropcodes.cpp#L215 |
| 23:16 |
hecks |
sfan5: but what's the problem exactly? Are channels expensive? Are you worried about things being too out of sync? |
| 23:17 |
sfan5 |
someone needs to bother to write the compatibility code |
| 23:17 |
hecks |
hm, wait, do clients need a rebuild? |
| 23:18 |
sfan5 |
there's an assert for {channel count from the packet} < CHANNEL_COUNT in the networking code |
| 23:18 |
sfan5 |
also I'd rather have the packet loss issue fixed (also https://github.com/minetest/minetest/issues/9107) than adding 16+ channels for lots of things that could ordinarily be just fine in one |
| 23:19 |
hecks |
that's not what I'm trying to solve |
| 23:19 |
hecks |
#10694 |
| 23:19 |
ShadowBot |
https://github.com/minetest/minetest/issues/10694 -- Reliable packet queue can't recover after getting jammed |
| 23:19 |
sfan5 |
it's somewhat related |
| 23:20 |
sfan5 |
or maybe not actually |
| 23:20 |
hecks |
not really, even if you deal with packet loss, it won't help the "dropped for one second" case |
| 23:20 |
sfan5 |
ignore that |
| 23:20 |
hecks |
this is like when you get a "connection interrupted" in quake |
| 23:20 |
sfan5 |
yeah perhaps the server knowing about which packets it can throw away would be better for that |
| 23:20 |
hecks |
minetest currently just hammers the client with more packets during that time and that can't ever be caught up with |
| 23:20 |
hecks |
that would be a step 2 to fixing this, compacting the queue after a soft disconnect |
| 23:21 |
hecks |
but splitting the queues is also a logically sound idea, why the hell is chat waiting on entity attachment |
| 23:21 |
hecks |
constraining the channel count with an assert may have been a very stupid idea |
| 23:22 |
hecks |
but let's just eat the compat problem, and make it so that further channel count bumps do not cost as much |
| 23:22 |
sfan5 |
it's not just an assert, the client code has exactly as many queues as CHANNEL_COUNT |
| 23:22 |
sfan5 |
so channels[3] points into the wild west (and the compiler might yell at you) |
| 23:22 |
hecks |
well then, it might need some reengineering to make it dynamic |
| 23:23 |
hecks |
I don't expect more fundamental channel count changes, but |
| 23:23 |
hecks |
certain "channels" like entities and modchannels are a whole new beast; ideally they should be dynamically allocated |
| 23:23 |
sfan5 |
I can understand modchannels but how would you split up entities? |
| 23:24 |
hecks |
well, logically, two entities that do not interact with one another should not wait on one another |
| 23:25 |
sfan5 |
right, so one per entity |
| 23:25 |
hecks |
but I also have a bad feeling about attachments |
| 23:25 |
hecks |
that area might need extra sanity checks if entities are desynced from one another |
| 23:25 |
hecks |
other than that, entities do not seem to interact with one another much |
| 23:26 |
hecks |
one channel per entity is okay, I guess entity messages mostly make sense to arrive in order |
| 23:26 |
hecks |
and instead, I'd prune the queue aggressively to reduce the bandwidth there |
| 23:27 |
hecks |
so let's see... change all the channel arrays to vectors |
| 23:28 |
hecks |
negotiate MAX_CHANNELS between the client and the server |
| 23:28 |
hecks |
if the client uses an old protocol, emulate the old 3-channel setup |
| 23:28 |
hecks |
it's trivial to make a legacy channel map |
| 23:29 |
hecks |
does that satisfy compat? sfan5 |
| 23:29 |
sfan5 |
it does |
| 23:30 |
sfan5 |
(there's nothing to negotiate though, the server says it wants 255 channels and the client can decide if it likes that or not) |
| 23:31 |
hecks |
oh, honestly, it could just be a map<> and not a vector |
| 23:32 |
hecks |
but, but |
| 23:32 |
hecks |
the channel count is a u8 right? |
| 23:32 |
sfan5 |
yes |
| 23:32 |
hecks |
and this is a part of the packet format? |
| 23:32 |
sfan5 |
yes |
| 23:32 |
sfan5 |
are you suggesting more than 255 might make sense |
| 23:33 |
hecks |
yes, for the entity and modchannel dynamic allocation |
| 23:33 |
hecks |
let's utf8 this then: channel 255 is an escape for a bigger channel id |
| 23:34 |
hecks |
ideally if the channel array is now a map<u32> or something like that, entities and modchannels can each get a whole range to allocate from |
| 23:34 |
hecks |
or, actually, there's another way |
| 23:35 |
hecks |
wait let me check if the id isn't a part of the packet in both cases |
| 23:36 |
hecks |
okay so the active object messages have a u16 id right at the beginning |
| 23:37 |
hecks |
and modchannels are keyed by string |
| 23:37 |
hecks |
making modchannels dynamic can wait |
| 23:38 |
hecks |
so, I guess it would be acceptable to just make another queue map keyed by object id and leave it at that |
| 23:39 |
hecks |
I'm guessing that the u16 is an absolute limit for all active objects observed by players on the whole server, + the players themselves? |
| 23:40 |
sfan5 |
yes but minus the "observed" |
| 23:40 |
sfan5 |
or rather s|observed|loaded/considered active| |
| 23:41 |
hecks |
that's a sad limitation but oh well, it can be dealt with later |
| 23:41 |
hecks |
no server exists yet that could hit it but it's realistically reachable |
| 23:43 |
hecks |
the biggest currently active server has 92 players; this translates to a budget of 712 entities per player, still a reasonable count |
| 23:45 |
hecks |
but who knows what the future will bring =] |
