Time Nick Message 07:00 [MatrxMT] since forum reports are still down.. this is a massive, pointless necro https://forum.minetest.net/viewtopic.php?p=439182#p439182 11:52 Road_Killer Another sneaky spammer: https://forum.minetest.net/viewtopic.php?p=439187&sid=bd479ec5e65fce0dd1c850f742eadda9#p439187 12:01 [MatrxMT] When trying to run a Minetest server in a Docker container, I keep receiving the following error: 12:01 [MatrxMT] ``` 12:01 [MatrxMT] 2024-09-15 13:59:34: ACTION[Main]: Server: Shutting down 12:01 [MatrxMT] terminate called after throwing an instance of 'DatabaseException' 12:01 [MatrxMT] what(): Failed to prepare query 'BEGIN;': unable to open database file 12:01 [MatrxMT] ``` 12:01 [MatrxMT] Does anyone know what this means? The Minetest server doesn't restart, but just gets stuck on this error. 12:01 [MatrxMT] Rejoin what? I restart the Docker container, throw away the image, and restart, and the issue keeps popping up. 12:02 [MatrxMT] It's not a log that appears when a client joins the server. The server won't start as a result of this error. 12:04 Krock Bram: make sure the file is still intact. run an integrity check from command line 12:05 [MatrxMT] Which file? The error doesn't mention a file. 12:05 [MatrxMT] I was trying to start a new world, so there was no file. I was assuming that perhaps the code just needed a `map.sqlite` file and I created one using my client. After having added the file, the same error is thrown. 12:06 sfan5 don't manually create it 12:07 sfan5 a possible issue is that the folder can't be written from inside the container 12:09 [MatrxMT] Oh yes, that was it. That also explains why it only gave this error for the new world, and not for an existing container. The server now halts because of a different error. _(Which I think I know how to fix.)_ Thanks! 14:39 [MatrxMT] My server now works, but it keeps crashing whenever I try to load a new chunk. The first few chunks worked (because the mod was still inactive back then), but now the server crashes when people join the server too close to the border to trigger an `on_generated` event in the mapgen async environment. 14:40 [MatrxMT] I'm not getting logs of that, though! Is there a way to get logs from the async environment, or to verify whether the mapgen environment causes an error that triggers to server to restart? I'd like to see if something in that environment causes everything to break, or that I'm overlooking something else. 15:04 [MatrxMT] Hmmm - I've rewritten the mapgen to be in the main thread, and it seems to work just fine. The server still restarts without given an explanation: 15:04 [MatrxMT] ``` 15:04 [MatrxMT] mt-survival | 2024-09-15 17:02:48: [Emerge-0]: Starting cavegen! 15:04 [MatrxMT] mt-survival | 2024-09-15 17:02:49: [Emerge-0]: Finished cavegen 15:04 [MatrxMT] mt-survival | 2024-09-15 17:02:49: [Main]: Using game specified by --gameid on the command line 15:04 [MatrxMT] mt-survival | - Found space 15:04 [MatrxMT] mt-survival | - Found space 15:04 [MatrxMT] ``` 15:04 [MatrxMT] After these logs, it's just logs of mods initializing. 16:23 [MatrxMT] <🇬regon> I think this was written by ChatGPT: https://forum.minetest.net/viewtopic.php?t=30980 19:10 v-rob More AI spam, it appears: https://forum.minetest.net/viewtopic.php?t=30981 19:17 Krock this user appears to be kind of legit but I really question what this person is doing there 19:23 MTDiscord Wait, that person is in the discord I think lol 19:23 MTDiscord Nvm I'm wrong 19:23 [MatrxMT] People can use bad behaviour with good intentions. This might be a confused person who's using ChatGPT to create slightly related Minetest posts. 19:29 Krock or using a chat AI to translate stuff, or correct bad writing. That's not a proper way to learn it. 19:58 [MatrxMT] I think there's a bug in the mapgen environment that causes my mod to break. Whenever I use `register_on_generated` in the main thread, the mod works fine, but the server breaks without notice when run in the mapgen environment. 19:59 ireallyhateirc perhaps related to this Out of Memory bug: https://github.com/minetest/minetest/issues/15169 19:59 [MatrxMT] As a result, _(and for backward compatibility purposes)_ I'm going to rewrite the mod so that it can run on either thread, depending on what the server admin prefers. Is there a recommended way for a script to determine whether it's in the main thread vs. the mapgen environment, or would you recommend me to program my own solution in my mod? 20:00 ireallyhateirc any errors? 20:00 [MatrxMT] Yes, I'm highly certain that that's it. My local Minetest installation doesn't crash, but it does slurp up a LOT of memory. 20:00 ireallyhateirc I do this: 20:00 ireallyhateirc local mapgen_env = minetest.save_gen_notify 20:01 [MatrxMT] Unfortunately not, that's why I've spent the entire day on it today. It took me a while to figure out the cause, considering that it only appeared after I uploaded a new world with ~10 new & updated mods. 20:01 ireallyhateirc then: 20:01 ireallyhateirc if mapgen_env then do_something() end 20:01 [MatrxMT] save_gen_notify is not available on the main thread? 20:01 ireallyhateirc it isn't 20:01 [MatrxMT] Oh, you're right. That's an elegant solution, thanks. 20:02 ireallyhateirc rewriting the code to work in the mapgen env is painful but it's worth it 20:02 ireallyhateirc will get easier with the IPC 20:03 [MatrxMT] I hope it will be worth it if it no longer crashes my server. 😅 20:04 [MatrxMT] On my local computer it did work amazingly, though. 20:05 [MatrxMT] I recently pushed out a cave gen mod that built caves using the mapgen env, but now I feel like I need to push an update so people don't brick their machines. 😬 20:05 [MatrxMT] https://content.minetest.net/packages/Noordstar/ns_cavegen/ 20:05 ireallyhateirc Could be the OOM bug honestly 20:05 ireallyhateirc did you fix performance issues with your thing? I plan taking a look at it one day 20:07 [MatrxMT] Honestly, the JITprofiler mod helped most. After reducing it from 15s per chunk to 0.8s per chunk, the JITprofiler helped it reduce to ~0.1s. 20:07 [MatrxMT] The mapgen env just makes the experience slightly more smooth, in those cases where you happen to place a torch and the torch appearing there only 10 frames later. 20:16 ireallyhateirc using more emerge threads would make it faster 20:16 ireallyhateirc for example in Exile we have our volcano mod 20:17 ireallyhateirc it generates the whole volcano in 48 seconds 20:17 ireallyhateirc with the mapgen env and 4 threads it got down to 18 seconds 20:17 ireallyhateirc There's a problem with using more than 1 emerge threads: https://github.com/minetest/minetest/issues/9357 20:18 ireallyhateirc so we're not using more than 1 until this gets fixed 20:18 ireallyhateirc The volcano is like 16x16 mapchunks, 300 nodes tall 20:18 ireallyhateirc I think, it's big anyway 20:30 [MatrxMT] That's pretty quick. 👌 20:35 Mantar Not the whole volcano, just a small pre-selected chunk thereof for performance testing. 20:37 Mantar the whole thing can be up to 900 nodes high, and goes several hundred below 0, and spreads out for somewhere around 1.5-2k nodes from the center, ain't nobody got time for that 20:37 ireallyhateirc t-thanks for clarifying 21:12 [MatrxMT] Hmmm - the issue seems to persist after downgrading back to 5.8.0 - which I assumed was a bassline for having it work. 21:12 [MatrxMT] Perhaps the function takes too much memory of itself, not even _needing_ the memory leak to overload the program. 21:36 [MatrxMT] It's extremely frustrating. The mod doesn't like the cave generator and I can't figure out why. 21:36 [MatrxMT] It keeps crashing and giving no logs. It would be _really_ nice if it could just give some out-of-memory error. 21:37 [MatrxMT] Running `docker stats` suggests that the container doesn't even take that much memory though. It remains quite stable around 8.00%, even when run on the main thread. 21:39 ireallyhateirc can happen if you have an infinite loop somewhere 21:39 [MatrxMT] The server quits without notice on an infinite loop? 21:40 ireallyhateirc freezes and becomes unresponsive 21:40 [MatrxMT] I mean that not a single line is put through to stdout or `debug.txt`. The server just stops. 21:40 [MatrxMT] It stops and _immediately_ restarts. 21:40 ireallyhateirc is it programmed to do so? 21:41 [MatrxMT] Any connected client times out. 21:41 [MatrxMT] Most definitely not. 21:41 [MatrxMT] The strange thing is that it never does it when I run it on my machine. When run in a Docker container, however, it always does it. 21:43 [MatrxMT] A freezing server would make sense to me. But the fact that it offers zero explanation and just decide to reload all mods and restart, is very confusing to me. 21:43 sfan5 are you sure it's not docker hiding the error from you? 21:44 sfan5 processes don't just randomly exit 21:47 [MatrxMT] file minetest_server_freezing.mp4 too big to download (11763458 > allowed size: 1000000) 21:47 [MatrxMT] minetest_server_freezing.mp4 21:47 [MatrxMT] I'm running with Docker compose. If the program exits, it tends to display in the terminal. 21:48 [MatrxMT] I will test by configuring that the container doesn't restart actually, just to make sure. But I'm fairly certain that the server doesn't exit. 21:50 [MatrxMT] Yeah no, it doesn't exit. Partially because I believe the container is configured to not exit. 21:51 [MatrxMT] Can you view the video I sent in the chat? Perhaps it's related to the 100% spike that happens when it restarts. I can't really tell which one causes the other. 21:54 cheapie [16:47:10] <[MatrxMT]> file minetest_server_freezing.mp4 too big to download (11763458 > allowed size: 1000000) 21:54 cheapie no, we can't 21:54 [MatrxMT] lol - I believe it's 11MB. Let's see if I can compress it. 21:56 [MatrxMT] Oh, it needs to be less than 1MB? Let's see if I can get it down to that. Or if I can publish it somewhere. 22:01 [MatrxMT] I put it up on PeerTube: https://spectra.video/w/u6AMFF9CVFxmxkWsdTEX9U 22:17 [MatrxMT] What is odd to me, is that it seems related to my `minetest.on_generated` function, but it always seems to finish the execution of that function (including the `write_to_map()` call before it crashes and restarts. Does `write_to_map` secretly happen afterwards and does that then cause a crash? 22:19 ireallyhateirc you don't do `write_to_map` in the mapgen env 22:20 ireallyhateirc you just set VM data, write_to_map is implicit after all mapgen scripts finish doing their stuff 22:20 [MatrxMT] That I'm aware of - and I don't do that. That would cause an error, which the server doesn't here. 22:21 ireallyhateirc just a hypothesis, but maybe you're adding something weird into the VM array? 22:21 ireallyhateirc check its length and such 22:21 [MatrxMT] I'll check that. 22:22 ireallyhateirc I managed to crash minetest silently once, don't remember what that was but I think in node definition I set name to a nested array like this {{}} 22:22 ireallyhateirc something like that 22:35 [MatrxMT] No, I'm afraid that's not the case. 22:36 [MatrxMT] It also wouldn't make sense. I set the array first, and then use the function to place schematics on the VM. So my expectation is that the function wouldn't finish if the table was malformed. 22:37 [MatrxMT] Just to be sure, here's the code snippet with which I checked: 22:37 [MatrxMT] ```lua 22:37 [MatrxMT] for i = 1, va:getVolume(), 1 do 22:37 [MatrxMT] if type(data[i]) ~= "number" then 22:37 [MatrxMT] minetest.debug("Data index " .. i .. " is not a number! It is type " .. type(type(data[i])) .. "!") 22:37 [MatrxMT] end 22:37 [MatrxMT] end 22:37 [MatrxMT] vm:set_data(data) 22:37 [MatrxMT] What does rejoin mean? Does that mean that the messages don't come through? 22:39 ireallyhateirc rejoin where? you mean IRC/Matrix? 22:39 [MatrxMT] I sometimes see the message: " rejoin" from the chatbridge. 22:39 ireallyhateirc could be unstable net, idk 22:39 [MatrxMT] So I'm not sure if all the messages bridge. Did you see my code snippet? 22:40 [MatrxMT] I'll repeat just in case: it also wouldn't make sense. I set the array first, and then use the function to place schematics on the VM. So my expectation is that the function wouldn't finish if the table was malformed. 22:40 [MatrxMT] Just to be sure, here's the code snippet with which I checked: 22:40 [MatrxMT] ```lua 22:40 [MatrxMT] for i = 1, va:getVolume(), 1 do 22:40 [MatrxMT] if type(data[i]) ~= "number" then 22:40 [MatrxMT] minetest.debug("Data index " .. i .. " is not a number! It is type " .. type(type(data[i])) .. "!") 22:40 [MatrxMT] end 22:40 [MatrxMT] end