Time  Nick        Message
02:14 MTDiscord   <kimapr> KILLbith
15:31 Sokomine    hi. i do have a question regarding network handling. there was a massive fight of 10-15 players vs perhaps 100-200 mobs on a server. eventually i saw the following: all entities were frozen, no movement updates arrived. hitting an entity with a sword resulted in no reaction whatsoever. i could still move around, but server and my client were totally out of sync
15:32 nrz         Sokomine, clearly due the fact we are using very low level movement code to sync client i think, we batch very much atomic operations
15:32 nrz         at a point devs we should really create new orders for movement code in order to let client do the whole movement process itself
15:32 Sokomine    in the end the mobs killed me while apparently (in my client!) i was running around and even far from them. there was server lag but not enough to explain this. after i died, i was able to move around for a few seconds before the death screen showed up. after respawn, my character was turned around randomly for maybe half a minute. had to look away because i got a headache from that. chat was also
15:33 Sokomine    delayed (irc was working fine)
15:34 Sokomine    i'm aware that entities still(?) can't tell they want to move to xyz and then stop (or move for t seconds and then stop). unless that's been changed?
15:34 nrz         Sokomine, yeah the move to xyz is clearly a thing we must implement in the future, that'll help all modders and reduce network/sync issues
15:34 Sokomine    but this wasn't rubberbanding. it was a complete almost-nothing-gets-through. sounds eventually arrived, but very much delayed (10-20 seconds)
15:35 Sokomine    that'd definitely help a lot and make things easier
15:36 Sokomine    admittedly that's a very hardcore situation for client and server. usually there is way less going on. given how it all started, it's very impressive that the game can (almost) handle such situations
15:38 nrz         the problem with such code, is if we add it, it's clearly a network protocol bump, because old client cannot understand this, and being retrocompatible is just a no go, it's impossible to maintain
15:39 nrz         but clearly i was multiple times in favor of this, part of a very huge pr around entites and never merged.
15:39 nrz         at a point i should just implement this part, i think it's clearly a path for better entities
15:39 Sokomine    yes. perhaps something for a 6.0 version? such code would help a lot of mobs as well
15:40 nrz         sfan5, celeron55, after this release, cannot we think about better entities ? It should require a proto bump at least. Minetest is very good in multiple places, but entites is clearly the worst, and for me, it's that who make a world alive
15:40 Sokomine    oh yes. this has definitely been hoped for by many modders
15:40 nrz         no need to do a big breakage version i think, except if some coredev have things they want to break
15:43 Sokomine    there is also another potential issue, though that is server-side: minetest.get_objects_inside_radius/area. players *love* itemframes and signs, so there are plenty of entities. most of those are very static and of zero intrest to a mob that is looking for other mobs it could attack or ought to avoid collision with
15:43 Sokomine    fluxionary_ can tell you more about that and has started doing some research in that regard
15:44 nrz         it's werid i don't see everyone messages on my irc client, i suspect bridge is not using query but note
15:45 nrz         an no you quoted it 🙂
15:45 Sokomine    if you're intrested in what happens if huge amounts of entities (and, seen relatively) players meet each other: the your land server is very intresting in that regard
15:46 nrz         its the server name ?
15:46 Sokomine    er? i see only you, nrz. nobody else saying anything here
15:46 nrz         i don't know if i have much time for dev-ing or digging- but regarding what you said and what i know engine side that sounds at least a first good step, but it's not a trivial one
15:47 Sokomine    yes, your land. it's intresting in many areas, but one other aspect is that i have some hope that things like lag and problems with players getting more are adressed and researched. it's already handling more players than in the past
15:48 Sokomine    though the other problem - spawn getting unusable - persists even there, and there is no clue as to why. perhaps the many signs, smartshops (even though already heavily optimized by fluxionary_) and itemframes are the problem
15:48 fluxionary_ Soko: the terrible lag during events w/ 100s of hostile mobs is mostly due to them using `find_path` w/ Dijkstra's algorithm instead of A*
15:48 fluxionary_ particularly if there's no path to the target (e.g. a player on a nerdpole)
15:48 fluxionary_ Soko: then the anticheat kicks in and teleports you back to the place you tried to run away from
15:49 MTDiscord   <Bla> flux: YL mobs_redo uses A* on the main and mobkit has its own lua pathfinding afaik
15:50 Sokomine    fluxionary_: that's server-side, yes. but no updates of movements arriving is another issue. you were present at the same time, suffered lag as well. were all entities frozen for you for a considerable time (went away only about 30 seconds after death)?
15:50 fluxionary_ Bla: that change isn't live yet though, i think, that's part of 1.1.118
15:51 fluxionary_ Soko: yeah, that's what happens when a single server step takes like 10-15 seconds
15:51 MTDiscord   <Bla> flux: thought so too, its not updated but @Bastrabun changed it already for main
15:51 fluxionary_ what's main? do you mean yl_stable?
15:52 MTDiscord   <Bla> mainserver, not testserver
15:53 fluxionary_ hm, the pathfinding is still the problem, reportedly, i thought it'd get a lot better
15:53 Sokomine    other players did complain about lag as well, but not to the degree i had it. that was a long lack of movement updates. my connection has very low bandwidth but otherwise (speed, relative upload, esp. stability) ought to be fine
15:53 nrz         fluxionary_: if that is calculated client side: pb solved 🙂
15:54 nrz         on MMORPGs like WoW or FF XIV server send : TO_POS(x,y,z) and client calculate. and then every 2 sec you have a new POS(x,y,z) to resync mob current pos, in case of
15:54 nrz         TO_POS(x,y,z,speed) sorry
15:54 Sokomine    fluxionary_: yes, A* gets to the result "no path found :-(" quicker than djiekstra. players are very intrested in getting the result "no path found" due to wishing to stay alive. but that is still something mod-side...
15:55 MTDiscord   <Bla> tried to activate pathfinding c++ debug today, looks like that is broken and only supports time measuring on linux
15:55 nrz         tbh we should not have 10 algorithms, only 1 which is the same client & server side to ensure entity position and we just sync the pos regularly during movement, but it's a huge rework to perform on our client<->server movement code maybe
15:56 Sokomine    nrz: yes. that'd help a lot and also make movement in mods a lot easier
15:58 Sokomine    nrz: diffrent mobs need diffrent algorithms. the one in the engine is not good enough for 2 node high entities most of the time. but that's another problem
15:59 Sokomine    just when the server tells the client that the entity will move to xyz and then stop there will remove rubberbanding and make movement easier. especially if the mod can have a callback for entity-arrived-at-xyz
16:00 fluxionary_ Bla: just checked, and `mob_pathfinding_algorithm` isn't set on the main server, so it's Dijkstra.
16:00 MTDiscord   <Bla> you shouldnt mix pathfinding and entity movement
16:00 nrz         Sokomine: clearly the thing you describe around movement with callback, sounds nice
16:01 nrz         it make me wanting to do some hacking on engine to implement it haha (not yet i have children to take back home)
16:01 Desour      higher-order object movement should be implemented with sscsm imo
16:01 MTDiscord   <Bla> flux: ask Alias thats what he told me the last time I claimed its still Dijkstra lol
16:02 fluxionary_ unless the code on the server is dirty, i suppose
16:03 sfan5       Sokomine: what you're describing sounds like a network stuckage/lockup
16:03 Sokomine    that's right. pathfinding was why the server was slowed down, but lack of updates of entity movements is a diffrent issue (both caused by many entities)
16:03 sfan5       improving entity movement code will probably help too but I don't know what the current status is
16:04 nrz         if we do client side pathfinding, remember server must do it too. The only think which will reduce is the amount of net code to send atomically movements
16:05 nrz         another thing which should be improved at a point, but i don't know how to do this properly, is the animations. If i remember we send key frames of animations from server to client. It's not very nice. Instead we should register animations on entities and just ask client to run some, and some should be automatic (like movement ones)
16:05 Desour      btw. if you can reproduce some laggy situation, you could also try to do some profiling with a tracing profiler. I recently tried out the tracy profiler (here: <https://github.com/Desour/minetest/tree/try_tracy>), it makes it easier to find out what was the cause for a too slow frame
16:07 Sokomine    hmm. doesn't really have to be client-side pathfinding. just..*stopping* and not going on forever would be a huge help
16:08 Desour      nrz: non-baked-in animations is also something that sscsm should do
16:08 Sokomine    sscsm?
16:08 Desour      server-sent client-side modding
16:08 Sokomine    aah. intresting to some degree as well
16:09 Sokomine    but this is more basic i fear
16:10 Sokomine    can someone point me to where/how updates of positions/movements are sent? a quick glance at the TOCLIENT package definiteions in src/networking didn't reveal it
16:10 Sokomine    ah. hmpf. wrote that - then saw it :/
16:11 sfan5       as part of TOCLIENT_ACTIVE_OBJECT_MESSAGES, the origin of them is somewhere in the SAO code
16:11 sfan5       or TOSERVER_PLAYERPOS for player input
16:13 Guest54     i doubt that client-side pathfinding or a no-backwards-compatibility protocol break are necessary to solve the described issue. most situations i have seen that are like that are either a) result of long server steps caused by something else that does accidentally quadratic things (e.g. each mob iterates over each other mob, then have a ton of
16:13 Guest54     mobs) b) result of laggy-but-playable network conditions. the first is obviously something you have to address in the mod responsible (e.g. mapgen), the second is a thing that could be addressed by putting a movement prediction path in entity metadata (something i have proposed in the past).
16:15 Guest54     in fact, a very common case that would be easy to address regarding movement prediction would be minecarts going around curvy tracks
16:15 sfan5       "easy to address" as in "just hardcode the rail following logic into the client"?
16:15 Guest54     the minecart entity handling server-side could easily attach a spline or even just line segments to the cart and then send fewer updates if the client is new enough to understand those things
16:16 Guest54     nope, i don't think hardcoding is the solution
16:16 Guest54     but i have worked on mod code regarding rails in the past and it's pretty easy to know where you are going
16:18 Guest54     it could be easy from the lua-side to set a list of tuples (timedelta, position) on the entity, old clients would not understand it but do not have to
16:18 Sokomine    hm. the advtrains mod is very impressive in many regards. the normal carts not so. after some initial fun they too soon get useless. and very...irritating...because the player's look dir is not updated when the cart turns
16:19 Guest54     well that can be updated. i'd advise against doing it without extensive testing though, it can be very jarring on curvy tracks
16:19 Sokomine    and carts going on for a while even if there is no rail is an annoyance. code to make them go just to the end of the current straight line and then stop until there's a package update would help them as well
16:20 Sokomine    aren't carts like that already? perhaps the client needs to rotate the player's head slowly in such a case
16:20 Desour      an interesting hack that one can try is to use animations for client-side movement (was once proposed for pipeworks tubed items)
16:20 Guest54     Sokomine would attaching a list of (timedelta, position) things to an entity and making the client understand it as movement prediction solve it or not?
16:21 Guest54     Desour i doubt animations can be reliably synced, or can they? i have experienced stuff with falling anvils where doing a little trolling resulted in them playing their fall animation over and over again
16:22 Guest54     besides, movement prediction exists already. it's just that it goes in a straight line forever.
16:22 Desour      (one could actually make a longer attachment chain to stack the animations. i.e. obj1->obj2->cart, obj1's animation moves forward from t0-t1, obj2's animation moves right from t1-t2 => cart moves forward then right)
16:22 Desour      Guest54: what do you mean with synced?
16:23 Guest54     Desour well at some point you have to change the position of the moving entity or can you animate it away from any position?
16:23 Sokomine    guest54: the problem is usually not the future movement - except in this particular case of lag i was describing, perhaps. more often the problem is that the entities *know* they want to go to position xyz and then reconsider (stand there for a while or change direction). but the client doesn't get that information. the client thinks they'll move on forever until the client gets updates
16:23 Desour      ah I see. yes, that's an issue
16:24 Guest54     Sokomine i do not understand. what you are describing is exactly future movement?
16:24 Sokomine    though in this case they weren't rubberbanding. but definitely not where they ought to be. guess all entities that were moving moved client-side until they got stuck somewhere
16:25 Sokomine    while nicer carts would certainly please first-time users, experienced players usually...tend to care less about carts. they're too slow and cumbersome. they make new players and children happy i guess
16:25 Guest54     like “going to position xyz and then waiting there” is, if i am not mistaken, a movement path
16:26 Sokomine    hmm. can't the package be extended with a "stop at xyz" information? and if the client can't process that information then he just lets the entity move on like now until the client gets its update? that way new clients could stop things but old ones would still be consistent
16:27 Guest54     Sokomine again, you probably want to attach that to the entity, not do a network thingy.
16:27 Guest54     also i have been on several servers with extensive minecart systems, so i beg to differ.
16:27 Sokomine    guest54: it is. but also an extremly simple one
16:28 Guest54     you probably want to seamlessly have it work with object:move_to(), right?
16:28 Sokomine    that'd be great, yes :-)
16:30 Guest54     btw, does anyone know about the null pointer thing in the logs? like why it is that way?
16:30 Guest54     src/log.h:304:21: runtime error: member access within null pointer of type 'struct LogStream'
16:30 Guest54     src/log.h:304:42: runtime error: member access within null pointer of type 'struct LogStream'
16:31 Guest54     like i saw there is a literal null pointer defined for not having logging, but i fear i do not understand its purpose
16:33 Guest54     or why you'd do the member access null pointer thing anyway, i think it's highly illegal in C++ to do it
16:34 sfan5       you're asking about a complex issue with barely any context
16:34 Desour      you're calling operator<< on a nullptr. just don't do that
16:34 Guest54     well i compiled minetest using ubsan, you can do that yourself with compile flags -fsanitize=undefined -fno-sanitize=vptr
16:35 Guest54     and then i found that the logging got refactored to do a little null pointer action
16:35 Guest54     (technically the -fno-sanitize=vptr is not necessary, but you get a ton of log spam if you do not do it)
16:36 Guest54     Desour am i understanding it correctly that m_dummy_proxy(nullptr) makes the code eventually go nullptr << "logmessage"?
16:39 rubenwardy  I find it hard to follow when you flip between topics and don't really compose your thoughts that might
16:39 Guest54     this is probably when that was introduced https://github.com/minetest/minetest/pull/12247
16:39 rubenwardy  *much
16:39 Guest54     i am sorry
16:39 Guest54     i should probably write forum posts instead
16:42 sfan5       did you check the place where the operator<< call happens?
16:42 MTDiscord   <Bastrabun> flux and Bla: For YL main I added a way to switch PF algo from ingame. Read yl_events.pf to see which one is live. This only affects mobs_redo and their PF in the mob step.
16:46 Guest54     i do not know how to check for that. the offending line 304 of src/log.h seems to be “StreamProxy& sp = m_target.hasOutput() ? m_proxy : m_dummy_proxy;“ which has no << as far as i can see it. the next line has something, but i am pretty sure i do not understand that code, i basically never do template metaprogramming.
17:21 nrz         sfan5, i don't know what is rail logic exactly but having a follow path client side can be nice. Not specially rail
17:24 Guest54     nrz what do you think of my proposal here? https://forum.minetest.net/viewtopic.php?p=406922#p406922
17:28 nrz         It's intermediate between my Prposal and what we have. Not so bad, but pathfinding require recalcul ation, for collisions
17:29 Guest54     i think if your lag is so bad that you require pathfinding, you are better off playing a turn-based game
17:29 Guest54     i mean that you require pathfinding *on the client*
17:39 Desour      Guest54: fyi, I've just tried UB sanitizer. the warning in log.h only appears in release builds, not in debug builds
17:39 Guest54     Desour well that explains why no one caught it in the review i guess
17:40 Desour      what review?
17:40 Guest54     when the logging was refactored
17:40 Guest54     #12247
17:40 ShadowBot   https://github.com/minetest/minetest/issues/12247 -- Make logging cost free when there is no output target by paradust7
17:41 Guest54     Desour does the debug build always have some other logging target maybe?
17:42 Sokomine    nrz: still need to check newer mods (was a bit busy with the rpg-like chat system recently), but it seems collision avoidance isn't that easily available in mods. we have moveresult in on_step, which is very exciting and now passed on as a parameter in mobs_redo, but...mobs actually avoiding collusion seems to be a long way to go
17:43 Sokomine    guest54: that might be true i'm afraid. still, not going on forever but automaticly stopping at the destination would help a lot. right now a mob constantly has to check if it arrived at its destination (might be a single step on the path)
17:44 Guest54     Sokomine uh, what might be true? context?
17:44 Desour      Guest54: not that i knew of
17:44 Sokomine    if lag is so bad that pathfinding is required clientside
17:44 Guest54     ah yes
17:44 Guest54     Sokomine i believe mcl2 has some cursed spring model for mobs to avoid dealing with collisions too much. cursed, as in: i let several cows fall into a hole and when i made an opening on the side … cow cannon.
17:46 Sokomine    isn't mcl2 mobs_redo based? would be surprised if not. afaik they don't have overly advanced stuff. in particular not when not trying to kill something
17:47 Guest54     it got heavily refucktored at least two times, so maybe it is based on mobs_redo, but it probably isn't. you can ask cora about it ig?
17:47 Guest54     (she was maintainer of mcl2 before the current one and dealt with at least one awful mob refactoring)
17:48 Guest54     btw, in general i wouldn't put too much faith in any spring model over real collisions, unless you like a lot of things being in the same place and then going at full speed in some random direction
17:49 MTDiscord   <josiah_wi> Might the sanitizer only notice the UB when the compiler has optimizations enabled?
17:49 MTDiscord   <josiah_wi> I think I've read in documentation somewhere that the UB checker needs optimization flags enabled, but maybe I'm remembering wrong.
17:49 Guest54     josiah_wi it could indeed be that the offending condition is only triggered with some optimization.
17:51 Guest54     i don't think you actually *need* optimization flags, but code relying on undefined behaviour (in a very liberal sense of “rely”) is usually optimization-unstable
17:51 Guest54     also nowadays compilers might even warn you at compile time if a case is simple, even without ubsan
17:53 Sokomine    ah! thank you for the info, Guest54
17:54 Guest54     btw, regarding rails: minecarts lagging off the rails in curves is one of the most common things where players *notice* that the prediction is awful
17:54 Guest54     players don't notice it immediately when a zombie goes in a straight line a bit too often
17:55 Guest54     the other time where prediction is awful (often caused by long server steps) is when mobs bounce on water and then fly up into the sky
17:55 MTDiscord   <josiah_wi> The documentation I was recalling was from developers.redhat.com and applied only to a specific feature in GCC at the time of writing. It is as follows: Some features are currently under development. The first one is object size checking. This makes use of the __builtin_object_size function, which returns the size of an object. Typically, compiler optimizations must be enabled for __builtin_object_size to work properly. If the
17:55 MTDiscord   compiler can prove that the program is accessing bytes outside an object, it churns out a run-time error.
17:55 Guest54     obviously, both of that can be improved by simply not having a laggy server
17:56 Guest54     josiah_wi well i don't even know if the size checking is something that ubsan does
17:56 Guest54     there are a bunch of these runtime checkers
17:56 Guest54     where the compiler comes to you at runtime, to tell you that your code miscompiled ;D
17:58 Sokomine    ah yes. luckily havn't seen the shooting up in the sky of fishes etc. for a while now
17:58 MTDiscord   <josiah_wi> The Clang UBSan documentation: -fsanitize=object-size: An attempt to potentially use bytes which the optimizer can determine are not part of the object being accessed. This will also detect some types of undefined behavior that may not directly access memory, but are provably incorrect given the size of the objects involved, such as invalid downcasts and calling methods on invalid pointers. These checks are made in terms of
17:58 MTDiscord   __builtin_object_size, and consequently may be able to detect more problems at higher optimization levels.
17:58 nrz         Guest54 you musn't trust network, and 10 ms per atomic movement is huged compared to micro second on your cpu
17:59 MTDiscord   <josiah_wi> This is the only one documented to depend on the optimization flags.
18:00 Guest54     nrz you could pathfind once and then set the path on the entity and that's it. i really don't see the value of client-side pathfinding.
18:01 Guest54     josiah_wi as i understand it, “may be able to detect more problems at higher optimization levels” means “the optimizer takes advantage of undefined behaviour”, which is trivially true … eg. the compiler can always assume that A + 1 > A if overflow is illegal for A.
18:09 Desour      just fyi, the errors that the sanitizer is finding is (1) std::memcpy is undefined if a ptr is null (even if size=0) and (2) content features bools can be uninitialized (i.e. for unknown node)
18:13 Guest54     uh, why would anyone ever think that memcpy with null pointers would be safe?
18:57 MTDiscord   <Bla> Guest54: replace pathfinding with player movement calculation in Sokomines & nrz discussion, they are not talking about pathfinding stuff like A* etc
20:29 MTDiscord   <jordan4ibanez> You could wrapper it
20:31 MTDiscord   <jordan4ibanez> pseudo code: c++ some* memcopy(some* myOriginal) {   if (pointer == nullptr) {       // throw some kind of error   }   memcopy(ohgodwhatevengoeshere, other)   return other; }
20:31 MTDiscord   <jordan4ibanez> Then you could use that to track down if things are trying to copy garbage
20:32 MTDiscord   <jordan4ibanez> Obviously don't call it memcopy haha, but it's just an idea :P
20:32 MTDiscord   <Warr1024> Just having the option of doing a simple linear interpolation of player movement, even if they clip through obstacles sometimes, would be way better than the current back-and-forth bouncy motion.
20:33 MTDiscord   <jordan4ibanez> I edited on discord, I have no idea how that looks in the irc, my apologies if it's a cryptic mess
20:33 MTDiscord   <Warr1024> IRC bridge doesn't carry anything like edits, replies, reacts, or any other discordisms
20:33 MTDiscord   <Warr1024> They see your name and the original text you posted.
20:33 MTDiscord   <jordan4ibanez> I know, that's why I said it
20:35 MTDiscord   <Warr1024> Was it in a code block when you originally posted it?  Because if it was, apparently the IRC bridge doesn't even attempt to replicate the effect there either.
20:35 MTDiscord   <Warr1024> https://cdn.discordapp.com/attachments/747163566800633906/1093635044544884816/Screenshot_2023-04-06_at_16.34.22.png
20:35 MTDiscord   <jordan4ibanez> OOF. That's brutal. But anyways. Perhaps in this feature freeze, safety wrapper funcs could be a cool priority maybe? I dunno, basic things like that make your life a billion times easier down the line
20:36 MTDiscord   <Warr1024> It does transmit images, but they have a "scary" discordcdn URL and it's up to IRC clients to decide whether to try to load a preview/embed, which, IIRC most don't.  And people who didn't choose to run discord are probably less likely to open any link with a discord URL too.
20:41 MTDiscord   <jordan4ibanez> Hmm, I suppose another one could be wrappering the constructor of vec3f in irrlicht to clamp to map limitations? so the original one stays as private but then the new wrapper would do a clamp(min,max) so it's not possible to spawn or move things out of bounds? Then just chain this new one to the original
20:41 Desour      a very simple memcpy implementation would just loop and copy. it's weird that you have to always pass a non
20:41 Desour      non-null ptr
20:42 MTDiscord   <jordan4ibanez> Well, if you're copying a nullptr you might as well be copying math.random into an array of bytes and casting it into the pointer type because that's what you're going to get
20:44 Desour      or it could copy your password. your sentence is trivial
20:45 MTDiscord   <jordan4ibanez> So is copying a nullptr, it makes a much sense as copying the alphabet to do algebra, should be a safety check
21:07 Guest54     > wrappering the constructor of vec3f in irrlicht to clamp to map limitations
21:07 Guest54     this will make for funny bugs when the next person think it should be clamped to MAX_MAPGEN_LIMIT or how it is called (people tend to use that constant for stuff, even though it is in almost all cases ill-advised)
21:11 Guest54     (IMO the correct way to do it is to clamp everything to the range of the underlying datatype)
21:13 Guest54     Desour IMO if you think about C or C++ constraints in terms of “what would a naive macro assembler do” you probably come to conclusions like “a simple implementation would not touch the pointer if zero bytes are copied”, but the less constraints you have, the less the compiler is free to optimize stuff away.
21:17 Guest54     also glibc literally announces that pointers have to be non-null, so i guess it's appropriate to assume that … ?
21:21 Guest54     jordan4ibanez regarding null pointer checks, i am pretty sure compilers can optimize those away in the case that they are deemed redundant (e.g. if it can be inferred that the pointer can not be legally null) – in 2009 just that happened with the linux kernel.
21:23 Guest54     some enterprising joker had added a line containing “tun->sk” before a check for “!tun” and that meant the check for !tun was optimized away, since tun can never be null in that scenario (CVE-2009-1897)
21:34 Guest54     jordan4ibanez btw you absolutely *have* to be able to spawn things at least slightly out of (visible, not s32) bounds for a bunch of reasons which mostly come down to “mods can write code without worrying if anything is near the map border, since it works mostly similar to unloaded/non-generated terrain”.