| Time |
Nick |
Message |
| 00:30 |
|
MTDiscord1 joined #minetest-dev |
| 04:26 |
|
calcul0n_ joined #minetest-dev |
| 04:44 |
|
fluxionary_ joined #minetest-dev |
| 05:00 |
|
MTDiscord joined #minetest-dev |
| 05:28 |
|
ivanbu joined #minetest-dev |
| 10:40 |
|
beanzilla joined #minetest-dev |
| 10:44 |
|
bigfoot547 joined #minetest-dev |
| 12:18 |
|
proller joined #minetest-dev |
| 12:32 |
|
vampirefrog joined #minetest-dev |
| 13:00 |
|
proller joined #minetest-dev |
| 14:17 |
|
json joined #minetest-dev |
| 14:29 |
|
Niklp43 joined #minetest-dev |
| 14:30 |
|
Niklp43 joined #minetest-dev |
| 14:34 |
Niklp |
sfan5: I ported the caverealms mapgen (https://github.com/Archtec-io/caverealms_lite/tree/mglua) to your new async api, really impressive! But the max lag goes up to ~1.5s (even more than w/ "normal" on_generated), is this a problem in my implementation or a mglua issue? |
| 16:50 |
sfan5 |
implementation looks ok, would have to profile to see why it still causes lag |
| 16:50 |
sfan5 |
if it works correctly any lua mapgen should not be causing more lag than the engine mapgen |
| 16:53 |
|
appguru joined #minetest-dev |
| 17:26 |
|
Niklp joined #minetest-dev |
| 17:30 |
|
Niklp joined #minetest-dev |
| 17:31 |
MTDiscord |
<.niklp> I'll profile it with jitprofiler later |
| 17:41 |
rubenwardy |
you'll probably want a C++ profiler. The performance of the Lua shouldn't matter as it's on a different thread |
| 17:48 |
|
TheCoffeMaker joined #minetest-dev |
| 17:50 |
Niklp |
I ran /emergeblocks, every 20-30 seconds theres a ~1-2 seconds long server freeze. Which C++ profiler should I use? |
| 18:00 |
Niklp |
the freeze comes from map saving, I'll try the default save interval |
| 18:34 |
|
appguru joined #minetest-dev |
| 18:40 |
sfan5 |
perf record -z --call-graph dwarf -- ./bin/minetestserver ... |
| 18:40 |
sfan5 |
(debug symbols needed) |
| 18:41 |
sfan5 |
after that I put the results into https://github.com/KDAB/hotspot |
| 18:53 |
Niklp |
Are debug symbols included in RelWithDebInfo? |
| 18:57 |
MTDiscord |
<warr1024> Yes |
| 18:58 |
MTDiscord |
<warr1024> I can confirm that doing a Release build does not include debug symbols but doing a RelWithDebInfo does. |
| 19:16 |
|
appguru joined #minetest-dev |
| 19:20 |
Niklp |
oke how must I "read" the results in hotspot? |
| 19:22 |
sfan5 |
find the main thread at the bottom, figure out when it froze and filter for that |
| 19:22 |
sfan5 |
then the top graph gives you a good idea what it was busy with |
| 19:23 |
MTDiscord |
<warr1024> "every 20-30 seconds theres a ~1-2 seconds long server freeze" ... You wouldn't happen to have a mechanical HDD, would you? |
| 19:24 |
MTDiscord |
<warr1024> That is suspiciously close to the issues I had with database activity on HDD. Enabling pragma journal_mode=wal on sqlite mitigated it a bit (spread out the interval). Moving to postgresql got rid of write spikes, but I still had a lot of delay loading mapblocks from disk. The only thing that fully resolved it was upgrading to an SSD. |
| 19:25 |
MTDiscord |
<warr1024> HDD wasn't always bad but it got bad pretty consistently any time the disk was busy with anything other than MT. |
| 19:25 |
Niklp |
No, I have a fast(TM) nvme SSD. I did some tests, with the default interval (5.3 seconds), there's just a ~400ms freeze |
| 19:26 |
MTDiscord |
<warr1024> Hmm, you wouldn't happen to have a machine that's slow in other ways, like CPU, would you? Could be compression or something maybe... Or maybe you just have some really big mapblocks (like lots of meta or ents) |
| 19:27 |
Niklp |
I've never experienced a similar issue in the last two years with my pc... |
| 19:30 |
Niklp |
The mapblocks are new generated since I test mapgen. There shouldn't be any nodemeta or ents |
| 19:31 |
celeron55 |
Niklp: you are using sqlite, right? |
| 19:31 |
celeron55 |
(wasn't mentioned yet) |
| 19:32 |
Niklp |
yes, it's just a debug server, otherwise I would use postgres |
| 19:33 |
Niklp |
(context: I'm using sfan's async lua mapgen PR) |
| 19:33 |
sfan5 |
consider using the dummy backend to rule out any disk activity |
| 19:33 |
Niklp |
+1 I'll test this |
| 19:33 |
celeron55 |
what filesystem? |
| 19:34 |
celeron55 |
you could try setting sqlite_synchronous to 1 (possibly permanently) or 0 (for testing) to see if the filesystem is somehow very slow to sync |
| 19:35 |
celeron55 |
(it sets PRAGMA synchronous, see https://www.sqlite.org/pragma.html) |
| 19:35 |
celeron55 |
on android it defaults to 1 and 1 should be safe enough |
| 19:35 |
celeron55 |
on desktop it defaults to 2 |
| 19:36 |
Niklp |
I'm using btrfs, w/o any weird config |
| 19:36 |
celeron55 |
(warr1024's suggestion about wal is also relevant, but i haven't personally played with that) |
| 19:37 |
celeron55 |
btrfs is a bit weird but i don't know anything about its sync characteristics |
| 19:39 |
Niklp |
max_lag w/ dummy map is 0.53 seconds definitely better than ~1.5 seconds |
| 19:40 |
MTDiscord |
<warr1024> Re: WAL, it's basically the "modern" way to run a sqlite database and I would recommend it for ALL non-pathological setups (i.e. unless you NEED a single file only, or are running stuff over nfs or something). I think the only reason it's not the default is because sqlite is really super strict about compatibility or something. |
| 19:41 |
sfan5 |
wasn't WAL enabled by default? |
| 19:41 |
celeron55 |
i wonder if changing map_compression_level_disk would have an effect |
| 19:41 |
MTDiscord |
<warr1024> WAL is not enabled by default that I know of in sqlite in general, and it's definitely not in Minetst. |
| 19:41 |
sfan5 |
hmm no |
| 19:41 |
sfan5 |
celeron55: better yet move compression out of the main thread |
| 19:42 |
celeron55 |
well that's not a simple config change that can be tested right now |
| 19:42 |
MTDiscord |
<warr1024> I would actually really like if there were an option in MT to turn it on immediately upon creating ANY sqlite db, so I wouldn't have to go in with scripts and do it post hoc. |
| 19:42 |
celeron55 |
unless you're about to write a patch 8) |
| 19:42 |
MTDiscord |
<warr1024> Re: compression, I'd try both lowering the compression (in case it's CPU pressure) and possibly raising it (in case it's I/O somehow) |
| 19:43 |
celeron55 |
maybe map_compression_level_disk = 2 or something like that for test |
| 19:43 |
celeron55 |
but also sqlite_synchronous |
| 19:45 |
MTDiscord |
<warr1024> heh, test each combination of config in nested for loops and graph the data ... in principle if you can measure the spikes somehow, like looking at globalstep dtime for 1 minute after the first minute (to ensure the game warms up first) and then calling request_shutdown, you can actually automate testing stuff like this. Really helpful when you have to bisect. |
| 19:50 |
Niklp |
map_compression_level_disk = -1 and sqlite_synchronous = 1 keep the maxlag at ~0.7. I'll test this w/o sfan's PR tomorrow |
| 19:50 |
[MTMatrix] |
<localhost> btw, can recreate database with synchronous=0 and other danger options, from raw sql dump (just adding some SQL lines to head of file), and it will be fast as possible, but... can be easy breakable |
| 20:03 |
celeron55 |
Niklp: -1 is the default value, it won't change anything |
| 20:04 |
celeron55 |
i don't know what the default level actually is but it's 0 to 9, 0 is least compression, so try 0 or 1 |
| 20:07 |
Niklp |
oops, you're right |
| 20:16 |
sfan5 |
zstd actually has levels beyond 9 but it seems we don't map those |
| 21:14 |
|
proller joined #minetest-dev |
| 22:32 |
|
Niklp joined #minetest-dev |
| 23:04 |
|
celeron55 joined #minetest-dev |
| 23:32 |
|
panwolfram joined #minetest-dev |
| 23:32 |
|
appguru joined #minetest-dev |
