Minetest logo

IRC log for #minetest-dev, 2022-01-18

| Channels | #minetest-dev index | Today | | Google Search | Plaintext

All times shown according to UTC.

Time Nick Message
00:00 MTDiscord <josiah_wi> I have no thread safety knowledge or experience.
00:39 _Zaizen_[m] joined #minetest-dev
00:41 erlehmann josiah_wi you can use thread sanitizer maybe
00:49 MTDiscord <josiah_wi> I'm not multithreading it at all, so thread sanitizer might not help?
00:49 MTDiscord <josiah_wi> I'm merely thinking ahead.
01:02 Pexin dont you just lock access to that resource behind a semaphore or some such? or am i misunderstanding the question?
01:05 proller joined #minetest-dev
01:06 Pexin is the raw pointer only used for reads? and are those reads atomic?
01:07 Pexin i am a bit rusty and i dont know your problem domain and i just woke up. ima shu up..
01:10 proller joined #minetest-dev
01:12 MTDiscord <josiah_wi> I think you're right. I'm just scared someone might be dumb and share the raw pointer between threads.
01:12 MTDiscord <josiah_wi> See, SpatialIndex isn't thread safe for reads either.
01:13 MTDiscord <josiah_wi> Any C++ gurus want to explain how a really simple move copy constructor can segfault?
01:26 MTDiscord <Warr1024> erlehmann 10 minutes is by far the longest interval I've heard about lag issues, nothing I know matches it.
01:26 MTDiscord <Warr1024> Longest I've seen is a 1 minute interval or so caused by Sqlite WAL flushing to a slow disk.
01:56 proller joined #minetest-dev
03:00 tekakutli joined #minetest-dev
03:28 queria joined #minetest-dev
03:29 v-rob joined #minetest-dev
03:34 queria joined #minetest-dev
03:48 erlehmann joined #minetest-dev
04:35 Pexin if anybody has the time to care about bouncy: I want to add the ability for seperate types of bouncy blocks, so modders can specify controllable or not. I'm thinking negative can be interpreted as uncontrollable. thoughts?
04:36 Jordan_ joined #minetest-dev
04:37 jordan4ibanez joined #minetest-dev
04:43 erlehmann Pexin grep contentdb if it is being used already i guess
04:43 erlehmann Pexin and figure out what it does?
05:00 MTDiscord joined #minetest-dev
05:46 MTDiscord <Bla> erlehmann @Bastrabun had problems with a lagspike every 5mins caused by postgresql
05:54 erlehmann Bla can you be more specific?
06:04 Pexin Bla you talking about #10113 or something more recent?
06:04 ShadowBot https://github.com/minetest/minetest/issues/10113 -- PostgreSQL: Severe performance degradation in 5.3 compared to 5.2
06:08 MTDiscord <Bla> sorry I dont remember the details how he fixed it but he will probably explain it himself after he got the ping.
07:36 appguru joined #minetest-dev
07:36 v-rob joined #minetest-dev
07:45 appguru1 joined #minetest-dev
08:05 olliy joined #minetest-dev
08:35 Pexin #11939 that was extremely easy
08:35 ShadowBot https://github.com/minetest/minetest/issues/11939 -- Restore bouncy behavior by pecksin
08:41 MTDiscord joined #minetest-dev
08:42 tekakutli joined #minetest-dev
08:49 Pexin (old clients can't interpret negative bouncy but oh well)
09:00 Fixer joined #minetest-dev
09:02 Fixer_ joined #minetest-dev
09:31 calcul0n_ joined #minetest-dev
10:17 proller joined #minetest-dev
13:42 MTDiscord <Warr1024> That postgresql thing was a regression that got fixed either right before release, or as a patch release right after.  I don't remember which but it was short-lived.
13:52 Alias joined #minetest-dev
13:57 MTDiscord <josiah_wi> Minetest's long line length is an actual issue now for me; VSCode only shows about 85 characters on my monitor, with the side bar turned off.
14:01 MTDiscord <Bastrabun> Regarding this "performance degradation": I do not have any comparison between versions. The server is on 5.4.1 with a bazillion mods, the database is postgres 13.5. I found when around 20 to 30 players do stuff, I had a lagspike of about 2 to 3 seconds every 5.3 seconds. I f I increase the server_map_save_interval from default 5.3 to 30, I have a 3 second lagspike every 30 seconds.
14:01 MTDiscord <Bastrabun> I'm not sure this is the same event you are searching for, but that's what @Bla is referring to.
14:03 sfan5 sounds like the "save in background thread" idea could help here
14:03 sfan5 or actually, does Minetest wait for the data to be committed? does it need to?
14:06 queria joined #minetest-dev
14:57 MTDiscord <Bastrabun> I tried to reproduce on the testserver (same machine, same version, same database, same everything) with 30 testaccounts. As long as they are only "standing there" and doing nothing, the lag does not show.
15:23 MTDiscord <Warr1024> It seems like MT must wait for the database to signal all-OK, which can take different amounts of time depending on the DB engine.  Sqlite has to reconcile its journal, which happens on each write with the default modes, and about once a minute with WAL mode.
15:24 MTDiscord <Warr1024> I guess in WAL mode it writes sequentially, which is fast, most of the time, but then it has to flush out the journal when it fills up once a minute, and that's a synchronous operation that happens after the write is committed before returning.
15:24 MTDiscord <Warr1024> With pgsql, I assume it writes out to a WAL internally, but pgsql doesn't lagspike because the flushing happens in background on pgsql's side.
15:25 sfan5 Bastrabun was just talking about map saving lag with postgres
15:25 MTDiscord <Warr1024> I had considered the possibility of actually making a db server specifically for MT that would speak redis protocol on the front-end, but write to a sqlite db async on the backend, actually, though for now I'm pretty content with pgsql
15:26 MTDiscord <Warr1024> map saving lag doesn't happen for me with postgres because postgres is fast for me, but I suspect it's "only barely"
15:26 sfan5 specifically for postgres MT doesn't have to (and should not) wait for the data to hit the disk
15:26 MTDiscord <Warr1024> My system is not very busy but CAN have spikes of disk activity, and it's on a 7200RPM platter drive, so disk write can bottleneck, and pgsql is able to hide that from MT.
15:27 MTDiscord <Warr1024> MT as I understand it probably waits for a transaction commit?
15:27 sfan5 I guess
15:27 sfan5 but what does that entail? I have no idea
15:27 MTDiscord <Warr1024> That means the data hits the disk in some form at least
15:27 MTDiscord <Warr1024> On "smarter" db systems this usually means a sequentally-written WAL, which is fast enough
15:28 MTDiscord <Warr1024> with sqlite, at least, transactions are mandatory and thus I wouldn't be surprised if that assumption carried over to other backends
15:28 MTDiscord <Warr1024> If you do a bunch of statements without a transaction in sqlite then the db just implicitly creates one for each statement, which actually can be significantly slower.
15:29 sfan5 I guess if postgres had something like sqlite_synchronous = 0|1 that'd fix the problem
15:29 MTDiscord <Warr1024> Let me double-check my setup and make sure I don't have any kind of postgresql patches or something anymore
15:29 sfan5 https://www.postgresql.org/docs/9.5/wal-async-commit.html or whatever this is
15:29 erlehmann uh so
15:30 MTDiscord <Warr1024> I'm just using postgres in docker using their "official" images with no custom config.  I dunno if there are config options that look like they'd help but are actually traps maybe?
15:31 MTDiscord <Warr1024> My MT build is based on https://github.com/pickardjoe/docker-minetest-postgres/commit/a9d6c188b1879aa86369aa262c6cf083ea4c3f4a#diff-3254677a7917c6c01f55212f86c57fbf
15:32 MTDiscord <Warr1024> I'm running this on a VPS w/ SSD, and another one on an i5 with the spinning disk.  In both cases I don't experience any lagspikes from the database.
15:32 MTDiscord <Warr1024> No custom config for postgresql, no patches for MT other than the change to the build process to enable postgresql and the added dependencies.
15:33 MTDiscord <Warr1024> Having an async background map write thread would help make postgresql not mandatory for a lag-free map on modest hardware, though.
15:34 sfan5 sqlite_synchronous does pretty much that except it's no extra work for us
15:34 sfan5 in that case the background write thread so to speak is provided by the OS
15:34 MTDiscord <Bastrabun> We're on default config except that I set WAL to minimal. The server is a VPS with NVMe storage
15:35 MTDiscord <Bastrabun> I will look into asynchronous commits
15:36 sfan5 what does "I set WAL to minimal" mean?
15:38 sfan5 ah you must mean wal_level=minimal
15:38 MTDiscord <Bastrabun> wal_level = minimal            # minimal, replica, or logical
15:38 sfan5 > The default value is minimal, [...]
15:38 sfan5 ?
15:39 sfan5 nvm I shouldn't be reading outdated docs
15:39 sfan5 and I have bad news
15:39 sfan5 nevermind I don't
15:40 sfan5 (async commits are disabled by default)
15:40 nrz i don't understand people with pg issue, i ran on a low end RPI 4 with PG and the pg is clearly not the issue
15:41 nrz and i'm using wal_level=replica. Pg is pretty cool on performance. You know that we had PG on HDD before SSD and more NVMe ? xD
15:43 erlehmann i have bad news though
15:43 erlehmann the end dimension, the one with the static sky
15:44 erlehmann i enabled shadows and oof
15:45 erlehmann the end dimension is deep underground and it has no sun or moon. how is that not enough to not have dynamic shadows?
15:47 sfan5 unloaded blocks can't cast shadows has also been mentioned by hecks in his issue
15:48 erlehmann well if they did the entire dimension would be darker than before, not?
15:48 erlehmann i mean it is basically a 21km high cave or so
15:50 sfan5 minetest already has light level calculations so shouldn't it be exactly as dark as before?
15:50 erlehmann dynamic shadows make stuff actually darker than before
15:50 erlehmann unless it is perfectly flat
15:50 sfan5 I know
15:50 sfan5 I'm saying they shouldn't
15:50 erlehmann ah
15:50 erlehmann so while a set_shadows or whatever call is not available, what about not casting shadows when there is not a sun or moon?
15:51 erlehmann it would be a bit weird but perfectly backwards compatible
15:51 sfan5 there is no API for saying "we don't have a sun, moon or even a time of day here", that's the whole problem
15:53 erlehmann to quote the source code for the end sky
15:53 erlehmann player:set_sun({visible = false , sunrise_visible = false})
15:53 erlehmann player:set_moon({visible = false})
15:53 erlehmann player:set_stars({visible = false})
15:54 erlehmann i am not sure what the sunrise thing does, but this looks to me like “the sun is invisible” works in gameplay terms as “no sun where the player is”
16:03 appguru joined #minetest-dev
16:12 Pexin i can file an issue later today, but i have discovered hitting -120 bouncy freezes the game. -100 doesnt but fills the terminal with msgs about avoiding an infinite loop. #11939 corrects it, but you probably want a standalone fix for only that in 5.5.0 (which is just adding abs() to 2 separate lines)
16:12 ShadowBot https://github.com/minetest/minetest/issues/11939 -- Restore bouncy behavior by pecksin
16:13 MTDiscord <Warr1024> The problem with sqlite_synchronous is that from what I understood, it doesn't only remove the durability guarantee (what we want) but it also removes the consistency guarantee.
16:14 erlehmann Warr1024 what do you think about “do not render shadows when sun/moon are invisible” btw
16:15 MTDiscord <Warr1024> erlehmann, it's not a bad approach but a separate flag of some kind would still be nice.  In ColourHop, during boss fights, the "moon" is still visible since it acts like a countdown timer, but as displayed it's not really something that would cast strong directional light.  In that case I'd still rather just turn it off.
16:18 Pexin is sun/moon position independent of time of day?
16:24 proller joined #minetest-dev
16:25 MTDiscord <Warr1024> As far as I can tell, the built-in sun/moon, shadow direction, and time of day are all linked, and all global per-world.
16:25 MTDiscord <Warr1024> If you want a custom sun/moon position per-player you basically have to paint them onto a custom skybox yourself.
16:26 MTDiscord <Warr1024> You can't have, say, different dimensions in a single world with different times of day; the best you can do is like MC[2/5/ia] and have one dimension with normal time flow and the rest "frozen" in time by using time-non-varying skyboxes.
16:27 MTDiscord <Warr1024> honestly, making all of these things independent of one another and per-player would be nice, but probably a lot of work.
16:31 Pexin so simply doing say, shadow_direction_mode = [timeofday, fixed] could cause later incompatibility
16:31 erlehmann well the beauty of “no dynamic shadows if sun/moon are not rendered” is a) backwards compat b) it just makes sense, like where is the light coming from seriously
16:32 erlehmann i mean maybe you can have dynamic shadows but do not treat invisible sun as a light source
16:33 erlehmann that sky has the color of tv static for a reason
16:34 Pexin timeofday | fixed rather
16:47 MTDiscord <Warr1024> something like player:set_shadows({intensity = float, direction = vector or "timeofday"}) (with intensity=0 also disabling the shader and improving framewrate) might be really nice, but it'll have to be sorted for 5.6 I guess.
16:48 MTDiscord <Warr1024> Worst case, making shadows follow time of day could be done by a mod, so if it's either ONLY time-of-day-based or ONLY fixed, I'd go with the latter.
17:01 Pexin if you really want a headache think about overcast sky or other weather, and if you REALLY want a headache think about an explosion particle or something casting ephemeral positional shadows. :]  i'm sure i'm late to that conversation though haha.
17:07 erlehmann cancel shadows lol
17:16 v-rob joined #minetest-dev
17:16 proller joined #minetest-dev
17:24 Yad joined #minetest-dev
17:43 MTDiscord <Benrob0329> I'd prefer to see support for other light sources in the future tbh, rather than designing an API around it only being based on 1 light source
17:44 MTDiscord <Benrob0329> But will that happen? Who knows
17:44 MTDiscord <Warr1024> That'd be pretty cool, but for a lot of people just the single directional source is already quite expensive...
17:44 MTDiscord <Benrob0329> Thats because the implementation sucks :^)
17:45 erlehmann no it is just a lot of computation
17:45 erlehmann to make shadows look good
17:45 MTDiscord <Benrob0329> Can it not suck with our current pipeline? I don't know, but shadows themselves aren't crazy expensive usually
17:45 MTDiscord <luatic> For multiple light sources, you usually have to render from the perspective of each light source IIRC
17:46 erlehmann all scenarios that come to mind where shadows are not expensive in 3d engines … employ trickery
17:46 MTDiscord <Benrob0329> Thats realtime rendering in a nutshell
17:47 erlehmann yeah and that is expensive
17:47 MTDiscord <Benrob0329> Hell, the very basis of motion picture is trickery, nothing is actually moving but its good enough to fool our brain
17:47 MTDiscord <Warr1024> I'd just hate to see rendering become inflexible in a way that excludes players too quickly at the low end of hardware.  I'm fairly close to that myself, and I don't plan on buying a new gaming rig just specifically for minetest.
17:48 erlehmann with trickery, i mean stuff like games having detailed avatars, somewhat diffuse lighting and then the “shadows” are just blurred ovals projected on the ground
17:48 erlehmann or pre-rendering specific silhouettes and switching between them
17:48 MTDiscord <Warr1024> Trickery isn't just graphics, it's basically ALL of computing ?  https://xkcd.com/722/
17:49 MTDiscord <Benrob0329> Realtime shadows aren't new, they can be done reasonably, just like shader pipelines in general.
17:49 MTDiscord <Warr1024> the shape of the things casting shadows isn't the only issue, though, tbh the fact that the shape of the "ground" itself varies so much is probably a bigger problem.
17:50 MTDiscord <Warr1024> The problem with tricks is that you can quickly get deep enough into the weeds with them that the work necessary to figure out which time-saving trick to use for each thing ends up being more work than just doing it the brute way would have been.
18:09 Pexin #11969 trivial fix for bug that can freeze the game if modmakers are especially sloppy. pls add to milestone
18:09 ShadowBot https://github.com/minetest/minetest/issues/11969 -- Use absolute value for bouncy in collision by pecksin
18:22 sfan5 https://github.com/minetest/minetest/blob/42839fa1db85ab6ce61c5a185f91919a5a9c067c/src/client/content_cao.cpp#L1805-L1814
18:22 sfan5 wat
18:27 MTDiscord <savilli> it actually makes sense
18:27 MTDiscord <savilli> a little
18:28 Krock but it's totally out of place. should be in updateAnimation()
18:46 MTDiscord <luatic> sfan5: this should be fixed in the engine I believe, server-sent animations should always override local anims (perhaps add a boolean param "override_local" to set_animation?)
18:46 sfan5 I was actually expecting that they never override the local animation
18:47 sfan5 changing this in some way is tricky because compat
18:50 MTDiscord <Jonathon> dont people just set local animations to a empty table to always show server side animations because the local ones are to limiting and only show those?
18:52 MTDiscord <Jonathon> https://github.com/sirrobzeroone/3d_armor_flyswim/blob/main/init.lua#L107 yeah
18:54 sfan5 that's not very portable
18:54 sfan5 but this makes me realize a bug in my PR
18:55 MTDiscord <luatic> sfan5: that'd be the cleaner way to go for your PR IMO, temporarily disabling local animations
18:56 MTDiscord <Jonathon> local animations are inherently flawed with hardcoding only a certain allowed set, and hardcoding the same speed
18:56 sfan5 well you can work around the speed thing, it also roughly scales with local speed
18:58 sfan5 also
18:58 sfan5 @luatic isn't the set_eye_offset in 'beds' not superfluous when player_api correctly sets eye_height? thoughts?
18:58 sfan5 s/not/now/
18:58 MTDiscord <luatic> it is
18:58 MTDiscord <luatic> I have fixed that somewhere
18:59 MTDiscord <luatic> but probably not in the redo PR
18:59 sfan5 check 'boats' too while you're at it
19:00 MTDiscord <luatic> only beds and carts do it
19:03 MTDiscord <luatic> huh, that's odd
19:03 MTDiscord <luatic> carts reduce the eye offset by -0.4
19:03 MTDiscord <luatic> but beds apparently just reset it?
19:05 MTDiscord <luatic> sfan5: https://github.com/minetest/minetest_game/pull/2918
19:08 sfan5 man this content_cao thing I linked earlier is just too buggy
19:09 sfan5 it appears set_animation needs to happen after set_local_animation and no animation must be ongoing
19:09 sfan5 I bet this isn't documented so we could actually just trash it
19:10 MTDiscord <luatic> obligatory erlehmann ping - does MCL rely on any of this?
19:11 sfan5 well MTG relies on it
19:11 sfan5 the basic idea is that if an animation is set that is already covered by local animation then ignore it
19:11 MTDiscord <luatic> that's a good idea, but how is it implemented?
19:12 sfan5 I linked the code earlier
19:12 sfan5 it's ????
19:12 MTDiscord <luatic> it looks incorrect to me
19:12 sfan5 it surely is
19:12 MTDiscord <luatic> for example, is it comparing only the starting point?
19:12 sfan5 no, Y is the ending point
19:12 MTDiscord <luatic> ah right
19:13 MTDiscord <luatic> why are the first two local anims considered special?
19:13 MTDiscord <luatic> player->local_animations[1].Y + player->local_animations[2].Y < 1
19:14 sfan5 that's not the first two
19:14 MTDiscord <Warr1024> I haven't used set_local_animation for a long time, since it basically broke swimming animations completely at the time.  It'd be nice to fix the animation latency issues though.
19:14 sfan5 I don't know regardless
19:14 MTDiscord <luatic> apparently it checks whether two local animations are set
19:43 calcul0n joined #minetest-dev
19:51 MTDiscord <luatic> sfan5: do we - and should we - have certain annotations for "HACK"/TODO/FIXMEs?
19:51 MTDiscord <luatic> the engine code uses "// HACK: ..." or "// TODO ..." sometimes. This makes these easily greppable.
19:55 appguru sfan5: shouldn't game#2917 be documented in game_api.txt as well?
19:55 v-rob joined #minetest-dev
19:55 ShadowBot https://github.com/minetest/minetest_game/issues/2917 -- Override local_animation settings for certain animations by sfan5
19:56 MTDiscord <Warr1024> Personally I use HACK/TODO/FIXME, plus XXX for everything, so like "// XXX: FIXME: bug" or something.  Having a catch-all makes it easier to do audits and not forget one of the keywords.
19:56 Krock personally I append the lorem ipsum after each FIXME
20:01 MTDiscord <Warr1024> // TODO: Document what there is to do here.
20:03 MTDiscord <josiah_wi> Why doesn't the Doxygen build find a libspatialindex or libspatialindex-dev package? Both are in the Alpine repos.
20:05 MTDiscord <josiah_wi> And where are dependencies for the Android CI build specified?
20:13 proller joined #minetest-dev
20:30 MTDiscord <Bastrabun> If you add a //TODO, then maybe also open an issue for this TODO and reference it.
21:21 MTDiscord <josiah_wi> Why do irrlicht includes use the "" syntax? It's a third party lib, and clang tidy even reports it as a compiler error because it assumes it's local.
21:34 sfan5 @luatic I guess; but did you document your additions there?
21:36 sfan5 @josiah_wi most irrlicht includes use that, irrlichttypes*.h is a local header if you're wondering about that
21:41 MTDiscord <josiah_wi> Should I include irrlichttypes instead of irr_aabb3d?
21:42 sfan5 absolutely
21:50 MTDiscord <josiah_wi> Thanks sfan!
21:51 MTDiscord <josiah_wi> Will implement as sees as I fix a really dumb memory leak. SpatialIndex apparently does not delete the storage manager, but it does use it during its destructor apparently.
21:52 MTDiscord <josiah_wi> This memory leak exists in current master and isn't caught by CI because it doesn't enable SpatialIndex.
21:55 MTDiscord <josiah_wi> I'm finding out there's a ton of small changes necessary in various places to make a PR come together. I have to whitelist the files for clang format, and I haven't even figured out how to generate the compile instructions for clang tidy.
21:57 MTDiscord <josiah_wi> I think you or another dev has to provide spatial index builds for MinGW before that will pass.
21:59 v-rob joined #minetest-dev
22:00 MTDiscord <luatic> sfan5: I did
22:15 appguru joined #minetest-dev
22:48 proller joined #minetest-dev
23:04 welfarejazz joined #minetest-dev
23:06 welfarejazz Hi, I'm trying to study the minetest code and I'm wondering how collision detection works. I've look at collision.cpp and I'm only able to get a rough idea of how everything works, and am specifically curious about why everytime you jump up a block it counts as a "step up" and why that's also essential for other entities
23:14 welfarejazz joined #minetest-dev
23:30 MTDiscord <josiah_wi> And I also have a question about the collision detection. When the server calls getObjectsInArea() it checks only the objects' base positions. What about checking against the object's collisionbox?
23:34 welfarejazz cool, thought I was just stupid lol
23:45 proller joined #minetest-dev
23:49 MTDiscord <josiah_wi> Decided not to break any compat.
23:49 welfarejazz wym?
23:49 MTDiscord <josiah_wi> Checking against the collisionbox would break compatibility.
23:50 MTDiscord <josiah_wi> I and another contributor are making some improvements to the collision system. It's my first major engine modification ever. xD
23:50 welfarejazz I'm just studying the code in my own little fork rn
23:51 welfarejazz trying to figure out why mountain flying is a thing, and how to fix it
23:51 MTDiscord <Warr1024> Checking against collisionbox would also probably make an existing performance issue worse.
23:51 MTDiscord <luatic> Minetest teleports entities up by step height if they hit a a X- or Z-face
23:52 MTDiscord <luatic> If you don't want that, set the step_height property to 0
23:52 welfarejazz yeah but then you can't step up stairs and slabs
23:53 welfarejazz also I think it breaks entities as well, not sure
23:54 MTDiscord <josiah_wi> Warr, we're fixing the performance issue.

| Channels | #minetest-dev index | Today | | Google Search | Plaintext