Time Nick Message 01:01 pgimeno found https://stackoverflow.com/questions/42587607/why-is-imul-used-for-multiplying-unsigned-numbers about exactly how signed and unsigned multiply differ - tldr: they only differ in the high word of the result, which isn't used in the case in point 08:46 sfan5 interesting 10:53 erlehmann pgimeno sfan5 would a trace like this help with future bugfixing? https://mister-muffin.de/p/5f_y.txt 10:54 erlehmann if there exists CI that runs the game with --random-input and i can *seed* the random input, i bet i can make it spit that ut 10:54 erlehmann out 10:54 erlehmann and yes, that was created on an x86-64 machine i have access to (it does not belong to me) 10:55 pgimeno that helps somewhat, though the backtrace lacks args info, which is sometimes useful 10:56 erlehmann i have no idea how to coax that out of the thing right now 10:57 erlehmann but i hope it is better proof 10:57 erlehmann and helps fix the callers of the functions 10:57 erlehmann bc if the function is never even called with some arguments, maybe that can be a solution too! 10:57 erlehmann though i think the burden is on the function 10:57 erlehmann every function should work for all input it accepts or crash 10:58 erlehmann (that way you fix them fast) 10:58 erlehmann i am pretty certain that none of the devs has systematically looked for stuff like this so far, because there's a bunch of low hanging fruit 10:59 erlehmann even though people are generally aware of it 10:59 erlehmann that is why i proposed to add it to the CI, if there is CI that can run the game 11:01 pgimeno I guess a proper backtrace could be generated with a) a debug build (not sure if you're already using it) and b) gdb 11:01 erlehmann probably, i did that in the past at work 11:02 erlehmann ohhh https://stackoverflow.com/questions/30809022/how-can-i-break-on-ubsan-reports-in-gdb-and-continue 11:02 erlehmann > You can stop on an ubsan report via a breakpoint on __ubsan::ScopedReport::~ScopedReport or __ubsan::Diag::~Diag. These are private implementation details which might change in the future though. 11:03 erlehmann so you would compile with ubsan and break on its error reporting 11:03 erlehmann nice 12:46 erlehmann why exactly was the shadow mapping thing merged? i have followed the discussion and most of the comments seem to indicate it is low quality and breaks things. 12:46 erlehmann maybe i am misunderstanding something about it. 12:47 MTDiscord I'd assume it was because it didn't break things too much to be in the way, and it will be fixed up before 5.5.0's actual release 12:47 erlehmann i guess it will just be disabled 12:47 MTDiscord aw. why would it be? 12:48 erlehmann that was the suggestion i read 12:48 MTDiscord like, it doesn't work on android, but on desktop it seems to work fairly well at a reasonable performance cost 12:48 erlehmann i think it breaks 3d rendering and stuff like the drawers mod 12:48 MTDiscord the fact games can't control it can be said for other graphical settings as well 12:48 erlehmann https://github.com/minetest/minetest/issues/11600 12:49 erlehmann i don't get the whining about games not being able to control it tbh 12:49 erlehmann that only matters because it is buggy, right? 12:50 MTDiscord I think originally it was believed that shadows had minor issues and could be fixed up in time for 5.5, but now we're really itching to get SOMETHING out to users for 5.5 instead of holding back every other improvement already merged behind shadowmaps. 12:50 MTDiscord So we'd rather just disable shadowmaps, release 5.5, then re-enable them for the 5.6 stream and try to get them fixed up for 5.6 12:50 MTDiscord is 5.4.2 for android still a thing? 12:51 MTDiscord Misestimations of how much work something will actually take to get it working right are par for the course in the software world, so this sort of thing shouldn't be too big a deal. 12:51 MTDiscord We are still hoping to get a fix out for the android thing, but again, the problems the change faced turned out to be worse than anticipated, so 5.4.2 and 5.5.1 to fix it would be out of band. 12:52 erlehmann this looks like problems i have already seen 12:52 erlehmann i.e. 1. merge stuff prematurely 2. “fixing it would take too much time” 3. release anyway 12:53 erlehmann well, i am not here to whine, i just don't understand the dev process very well 12:53 MTDiscord Disabling instead of backing out the merge does have one advantage in that it reaffirms our commitment to actually fixing it instead of just scrapping it though. 12:53 erlehmann some people like Wuzzy are *very* careful about their code and what it affects 12:53 erlehmann others, not so much 12:54 erlehmann so i guess the matter is not some objective quality, but if ppl are convinced then 12:55 erlehmann and even groups can be in error about the quality of something 12:55 MTDiscord Error avoidance and error tolerance are 2 different strategies for making a system robust, and each one eventually starts to yield diminishing marginal returns. 12:55 erlehmann or maintainers can be pressured into “just merge it, it works” 12:56 erlehmann (i have been pressured like that and regretted it every time, every “let's skip testing” was a lie) 12:56 MTDiscord I tend to use low error avoidance strategies in almost everything I build, and it works pretty well for the most part. If I want more error avoidance, it usually requires bringing in somebody who thinks differently than me. 12:56 erlehmann i mean “let's skip testing *because it works*” 12:56 erlehmann what is “low error avoidance strategies”?? 12:58 MTDiscord I've never done "let's skip testing" per se, but the actual amount of testing is weighed against the cost of testing failure, and so you end up wanting less testing when you minimize the cost of failure. 12:58 erlehmann „If I want more error avoidance, it usually requires bringing in somebody who thinks differently than me.” – i once had a programming project that i had to do entirely myself as a junior dev, because the senior dev quit and the freelancer replacement was not capable. for code review i got a coworker who had nothing to do with the project. only when i could convince her that my code was sound, i merged it. 12:59 MTDiscord I think it would be hard to make MTE's process that level of fault-tolerant though. 12:59 erlehmann well at the rate of bugs i have found since starting looking for them a few days before there def seems room for improvement 13:00 erlehmann i mean i guess others find bugs at the same rate 13:00 erlehmann after all they have much more experience with the project 13:02 erlehmann hmm rollback code seems faulty too? 13:02 erlehmann brb 13:04 MTDiscord I just hate seeing people throwing more and more effort into preventing bugs, well into the realm of diminishing returns, and completely ignore factors that reduce the cost of inevitable bugs, like architectural or deployment changes. It's hard to get into specifics, though. MTE does SOME things right, like having a lot of users running the dev branch so we are likely to catch errors before they hit the official release, at least. 13:05 erlehmann i think most of the effort can be automated through having more tests 13:05 erlehmann to prevent regressions 13:05 MTDiscord I just don't sweat most bugs because I put a lot of thought up front into assuming there will be bugs and minimizing the cost of fixing them. 13:05 erlehmann this is not meant as a personal attack, but in my entire life, everyone who feared one might “test too much” was proven wrong when both approaches were actually tried 13:06 erlehmann oh yeah the architecture can ofc mitigate some stuff 13:06 MTDiscord Tests are great, it's just that it's pretty rare to find someone actually good at making them :-D 13:07 MTDiscord Personally I'm pretty terrible at designing, writing, or maintaining tests, especially compared to my ability to make forwards-code. 13:07 MTDiscord There are some areas where it's just not worth trying to cobble together a proper test and I just try to sweep it in my manual integration testing process, but there are a few others where I'll muddle through because the cases are just so byzantine that there's no way manual testing wouldn't be way too error-prone. 13:08 erlehmann in my experience, if you do not require tests for new features or regression tests, you can not get ppl to write them tbh. not only will they forever be optional (thus more work), if people write the tests *after* the code wa written, it is a long painful process, because the code often is not testable if not written with testing in mind. 13:09 erlehmann Warr1024 care to link me to some tests? if it is about nodecore, we should discuss in #minetest i guess 13:09 erlehmann i want to learn 13:09 MTDiscord Haha, specific to NodeCore, I don't think I've got any automated tests. 13:09 MTDiscord what's minetest engine's unit test coverage like? 13:09 MTDiscord I can see there's some tests in src/unittest/ 13:09 MTDiscord I've only really done serious testing for like work stuff 13:09 MTDiscord For NodeCore I pretty much test manually 13:10 MTDiscord There aren't a lot of units in there that wouldn't be a real pain to wire up for testing, though tbh there are probably a few pieces that could benefit from them by now. 13:10 erlehmann at work i did test driven development, that is how i learned that writing test first is easy, writing tests last is abysmal 13:11 erlehmann but like, you can refactor without fear when you have tests 13:11 erlehmann (that are 100% complete) 13:11 MTDiscord it'd be cool to have some sort of in-game gameplay unit testing similar to what minecraft does 13:11 erlehmann this is commonly known as a “cheat client” and not wanted as far as i know 13:12 erlehmann i have suggested it many times 13:12 MTDiscord aw man 13:12 MTDiscord actually it would probably more be for minetest games specifically 13:12 MTDiscord unless it'd be like... devtest 13:13 MTDiscord tbh I'm skeptical that writing tests first vs last is a really significant factor, at least compared to whether you designed the unit to be testable in the first place. As long as it's POSSIBLE to write tests I think it shouldn't be too bad. 13:13 erlehmann i mean, the obvious way to figure out if say, a user can still dig a block (yes, mineclonia and mineclone5 managed to independently run into the same bug & tool stopped working) is to script it. but you can only do that with a cheat client right now. 13:13 sfan5 what would "in game unit testing" imply? 13:13 MTDiscord I've written code first, tested it by hand, found it working well enough, used it, then wrote regression tests, tightened them up, discovered a bunch of bugs, fixed them, and then maintained the unit and tests together after that, and it felt fine :-D 13:14 MTDiscord But the way I see it you write code to be tested, or you write code to be thrown away and replaced ... but like 90% of code I write is probably the throw away kind. 13:14 erlehmann Warr1024 i know such things, you might be good at designing interfaces – the idea of “tests first” is trying to work out where you want to go. it forces you to have a mental model of the resulting output and the interface. 13:15 MTDiscord Over the years I've found I'm gravitating toward the modern auto manufacturer strategy: design things in tight assemblies that are loosely linked to other things, and if a problem crops up in some component, just rip it out and replace the whole thing. 13:15 erlehmann when you do not write them first, you can either make something testable or something that relies on global state or has too many branches to be easily testable. 13:15 MTDiscord Yeah, maybe that's a good excuse, I write the tests "mentally" but they just usually don't show up in actual code. XD 13:15 erlehmann yeah 13:16 erlehmann it is rare though that ppl are good in that, they usually need to be *able* to write the test at that point 13:16 erlehmann oh also one more thing, if you have tests, you can fuzz! 13:17 MTDiscord I've tried formalizing tests in actual code and met with at least partial success. I find that the actual effort of writing and maintaining tests are much easier to justify when a unit is very "load bearing", so tests tend to be more robust near the foundational units of an architecture. New stuff gets added at the top, then pushed down as more utility for it is found, and it ends up getting assimilated into the test plan as that 13:17 MTDiscord happens... 13:17 erlehmann i think a 5.5 release with many bugs fixed would be nice btw, but it should not have regressions. 13:18 MTDiscord sfan5: doing automated tests in-game to see that everything works like it should. one example could be checking liquid spread, that it spreads to the specified amount of blocks, or placing a falling node and seeing so it falls to the ground properly. 13:18 erlehmann sfan5 so about the zstd thing, i may have missed an answer if you gave it, can i force minetest to save maps not in zstd? 13:18 MTDiscord I hope this makes sense 13:18 erlehmann sfan5, the reason is testing backwards compat with the same world 13:19 sfan5 yeah that makes sense 13:19 sfan5 the tests you're describing would not even involve a client 13:19 sfan5 it's doable today, someone just has to write the test framework 13:19 MTDiscord A big problem for MTE is that users don't want fewer bugs, users want more features. If we want MT to be more popular then we need to add risky stuff, and that takes limited time and energy away from robustness. We aren't trying to maximize our audience as fervently as a commercial project would, but we don't want to languish in obscurity either; people need to be interested enough in the project in general for us to have a chance 13:19 MTDiscord at finding new contributors. 13:19 erlehmann is it possible to feed input to a headless minetest? like /setblock somewhere? 13:20 sfan5 a headless minetest what? client? no, server? sure write a mod 13:20 sfan5 >08:18 sfan5 erlehmann: you can edit src/serialization.h to only write format 28, this will keep your map compatible 13:20 sfan5 (yesterday) 13:20 erlehmann sfan5, thx 13:21 MTDiscord There is one command line switch for MT that creates random inputs, which I suppose is intended to be a sort of fuzzer. I don't know how efficient it is necessarily though; haven't actually tried using it yet... 13:21 MTDiscord Also doesn't MT have a "null" renderer that's intended to be used for automated testing e.g. in the CI containers or something? 13:22 erlehmann “users don't want fewer bugs, users want more features” – in my experience as a player, developers of games think this all the time, but it is generally false. there usually exists a small very vocal subset of players for any game that cares about *new* features, the silent majority cares about stability and having no regressions. 13:22 MTDiscord Eh, well, in the case of commercial software, your goal is to make people BUY the game, but retention is less of a concern :-) 13:22 erlehmann in fact, i know 3 servers that did stuff like running forks of minetest or not upgrading games because of lag 13:22 MTDiscord irrlichtmt's automated ci testing uses xvfb which, uh, I think makes it not use the null renderer 13:23 MTDiscord oh no wait, it does both xvfb and null 13:24 MTDiscord The thing to remember though is that developers think of bugs as some kind of fundamentally different thing to features, but to users lack of bugs is a feature pretty much like any other and it's weighed in the balance. 13:24 sfan5 @Warr1024 we have an integration test on the CI using the null renderer 13:24 MTDiscord I've heard some people say "always fix bugs before adding new features" but that's only necessarily true if you use a very loose definition for "always" 13:24 erlehmann Warr1024 to users a bug is “can't play this thing” or “can't use this thing in the game, even though i could before” 13:25 erlehmann like, assume the TGA thing would *not* be fixed by the next release, server owners would definitely share what the last “good” release was etc. (i.e. such things already happened) 13:26 erlehmann bc players are very quick to leave 13:26 erlehmann if your server ruins something 13:26 MTDiscord Yeah, a bug basically just cancels out a feature (and some portion of a user's baseline goodwill toward you) but it's not literally impossible for you to make an improvement elsewhere whose benefit outweighs the bug's cost, thus having it actually make sense to defer the bug. 13:26 erlehmann “a user's baseline goodwill toward you”, yeah that's it 13:27 MTDiscord Also resources spent on one thing are not strictly rival against resources spent on another thing. If someone is only really good with visuals, then you're not really taking away resources that might have otherwise been spent fixing a database bug by merging their improvement. 13:27 erlehmann oh, i only mean case where stuff legitimately has an either/or choice 13:27 erlehmann like a new feature that breaks old stuff 13:29 erlehmann i mean: everyone who told me that users totally prefer new features even if it breaks some old stuff was a developer (for minetest this applies both to engine and games/mods) – whereas almost everyone who complained about devs adding new features that break old stuff was just a normal user, the type of person who has a grievance but won't even make a bug report. 13:29 MTDiscord The MT project sees a fair amount of turnover (including even just cyclical activity levels) among contributors, and so I don't think we have a definite risk-affinity/risk-avoidance level. That means that if you look at the project as a whole over time and expect some consistency as to how risky decisions are made, you'd see a lot of stuff that looks paradoxical, like rejecting some low-risk stuff as too risky, but merging big-risk 13:29 MTDiscord stuff later :-) 13:30 erlehmann well i'm happy if it moves into the right direction, like “do not make stuff slower on old computers to make it faster on new computers” 13:31 erlehmann which so far has worked out (see the icon scaling code) 13:31 MTDiscord Developers are not as good as users are at understanding what users really value, and users are not as good as developers at understanding what things actually cost to do. But then, developers themselves are terrible at estimating, and users are pretty terrible at predicting what they'll actually want, so I guess that equalizes things somewhat. 13:31 erlehmann i'd say it is more nuanced 13:31 MTDiscord Sure, it always is :-) 13:32 erlehmann i.e. users are absolutely untrustworthy in what they want if it is about features that do not yet exist. but if they had a taste of a feature and you take it away, they will only accept it if that feature itself caused trouble. 13:32 erlehmann this can be as simple as “we removed X because it caused lag” 13:33 MTDiscord Haha, my favorite is "we removed X because you lied to us when you said you wanted it and would use it." 13:34 erlehmann oh that one is always mean, because the followup is either nothing or that someone who did use it is extremely pissed 13:35 erlehmann i.e. the people did lie, but it does not matter 13:35 erlehmann so, regarding engine stuff, i am trying to figure out how to trap on UB 13:35 erlehmann to deliver you stack trace 13:36 MTDiscord Well, I was once on a project where the whole project was canceled because the customers said they loved it and would pay thousands for it, but when we actually delivered 6 months later, not a single one of them was interested, even at a lower price than they themselves bid. That's maybe the real reason why software shops tend to sell things and take the money first, and then actually build them later under threat of breach of 13:36 MTDiscord contract. 13:37 MTDiscord One person being extremely pissed is also a thing you have to weigh in the balance, and to be frank every project can have its own more or less arbitrary function for determining what exactly they want to optimize. 13:40 erlehmann “customers said they loved it and would pay thousands for it” – get it in writing 13:41 MTDiscord If by "in writing" you mean wait for the check to actually clear, yeah. :-D 13:41 erlehmann hehe 13:42 erlehmann Warr1024 in my experience, the “one person being extremely pissed” case usually means there are tens or hundreds sometimes that are pissed, but a) not so much or b) have no idea how to do anything about it, i.e. not even how te reach the devs 13:42 erlehmann te → to 13:43 MTDiscord Sure, but that's just another correction factor to apply when weighing it. Also the potential that person has to create damage by being disproportionately vocal. 13:44 MTDiscord But on the other hand, once a project reaches a certain scope, the risk that somebody is going to be seriously pissed off regardless of the nature of the change goes up too, i.e. there stops being obvious unequivocally good things that you can do ... so you have to get good at handling that kind of thing regardless. 13:54 erlehmann Warr1024 not really. there exist a bunch of projects out there that can go 10, 15, 20 years adding features but they do it in a way that stuff does not break. 13:54 erlehmann and then there is stuff like inkscape that changes the meaning of command line parameters randomly lol 13:54 erlehmann btw, i think minetest building is cursed 13:55 erlehmann it seems to me that a) too much stuff is rebuilt in the common case (i.e. going between commits/branches) b) the dependencies are underdetermined, i.e. swhen recompiling it does not create a binary corresponding to a clean build 13:57 erlehmann i assume that this has been known but disregarded as not important, but you should know that the result of the latter actually means you could have tested a binary that does not correspond to the branch you tried to verify 14:00 sfan5 so you're saying too much stuff is rebuild but also too few 14:00 sfan5 rebuilt* 14:00 erlehmann yes, but different cases 14:01 erlehmann doing a rebuild on the same branch/commit leads to the “to few dependencies are considered, build might be faulty unless you rebuild from scratch” thing 14:02 erlehmann whereas checking out some branch/commit appears to needlessly rebuild stuff 14:06 erlehmann i could explain it in depth, but it does not really matter, as it is easy to verify & generally both bugs are long-standing issues with cmake & make that can not be fixed 14:06 erlehmann i just want ppl to be aware of it 14:07 erlehmann bc *usually* devs trust the build system 14:08 erlehmann and by “can not be fixed” i mean “the architecture of both cmake and make means it is imposible to fully work around or patch the behaviour” 14:09 erlehmann the “not rebuilding enough” thing basically means you have to rebuild everything if you change anything to be sure you get the proper output, but that is basically always the case for C++ projects that use cmake / make, even for a hello world. 14:10 erlehmann ppl usually do not notice these things anyway, bc “not rebuilding enough” often means “my binary is at some previous commit” which can be close enough 14:11 erlehmann but i have a guy who after i told them that were like “oof the last commit before the release was an important fix, we definitely have the wrong binary here” 14:11 erlehmann just remember it if i.e. some feature does not show up even though you had a “proper” partial rebiuld 14:11 erlehmann rebuild 14:12 erlehmann btw this can be caught through testing the binaries, but i have no idea who does that 14:13 erlehmann Warr1024 if you divide the mouse movement on the --random-input option by 10 or so it works well 14:13 erlehmann Warr1024 otherwise it moves too fast to be usable for testing 14:14 erlehmann Warr1024 i also suggest to increae the digging time in the RandomInputHandler or what it is called 14:14 erlehmann Warr1024 and add a CSM that a) respawns you on death b) switches to the first hotbar slot that has contents if your current slot is empty 14:15 MTDiscord If an issue for the build stuff is started or if we discuss it here I will try to improve it. I've frequently had to rebuild from scratch every time the CMakeLists has a diff. 14:16 MTDiscord I too easily "forget" the pain of ordinary routine stuff like compiling MT because that pain is encapsulated in my scripts :-) 14:17 erlehmann well, if you can coax cmake into outputting files for a build system that checks timestamp & hashs instead not just timestamp, the issue will not be completely fixed, but it is a step into the right direction 14:18 MTDiscord Will Ninja do that? It's as easy as passing -G ninja then. 14:18 erlehmann no 14:18 erlehmann ninja is one of the systems that can be fixed, though. ninja has a PR for hash bassed checking, but i think it was rejected. 14:20 MTDiscord What build system would you recommend for starting to work on this? 14:20 erlehmann in general, the only build systems that can be improved to handle both of these cases (rebuilding too much and not rebuilding enough) are those who do both hash-based comparisons if the timestamp differs and can handle non-existence dependencies (i.e. if a previouly non-existing file at a path exists, a target needs to be rebuilt). 14:21 erlehmann the number of systems that can do both is pretty small 14:21 erlehmann for some reason 14:24 erlehmann josiah_wi i may be a bit biased, but i have written an implementation of the “redo” design by DJB in about 400 lines of shell script that is amusingly even smaller than the years-old patch that tried to fix cmake (which turned out to be unworkable) http://news.dieweltistgarnichtso.net/bin/redo-sh.html 14:25 erlehmann josiah_wi if i wanted to fix this as fast as possible, i would use this. not because i wrote it, but because it is among all build systems i know the one with the smallest set of dependencies and minimal special syntax (build scripts can be any language) 14:26 erlehmann josiah_wi to see this in action, you can build this C++ game with it: https://github.com/linleyh/liberation-circuit 14:27 erlehmann josiah_wi note that there is also a small ~100 line script called “do” there that always rebuilds everything 14:28 erlehmann josiah_wi i have a small test case for this thing being faster than ninja on incremental rebuilds (but tbh ninja would be A LOT faster if they would merge the hash checking code): https://mister-muffin.de/p/W8EA.gz 14:31 erlehmann josiah_wi btw a good question to figure out if build systems are fixable in terms of not rebuilding when they should is “is the dependency graph attempted to be generated before the build or is it generated as a side effect of the build itself?” because there are some common cases in which that matters, but just read the papers linked on my page, others have expressed it better than i can. 14:32 erlehmann josiah_wi also note that i do not advocate ripping out cmake/make. i would like faster incremental builds myself too. 14:32 sfan5 I would advocate not wasting any time on this 14:32 erlehmann yeah that is usually the response. 14:33 erlehmann i mean “just rebuild everything completely if you test it” is workable for most things minetest 14:35 erlehmann sfan5, i noticed the incremental rebuilds problem while trying to coax minetest to spit out a trace including variables btw 14:35 MTDiscord Integration with CMake is kind of an important limitation. 14:35 erlehmann that's how i got to it 14:36 erlehmann josiah_wi not necesarily, non-devs do not need fast incremental rebuilds. 14:37 erlehmann liberation circuit has both dofiles and cmake / make integration. granted, the make invocation builds the project only correctly on every second invocation and the cmake thing will *probably* only on full rebuilds, but it seems to work out. 14:38 erlehmann josiah_wi Warr1024 if you want to share solutions for faster incremental rebuilds, i have a full build time of >30 min (which shows how bad this problem can be), so i am highly interested. 14:38 erlehmann especially i am interested in already existing solutions 14:39 erlehmann (compared to something i just sketched that requires putting time into it) 14:41 MTDiscord Ninja > Makefiles I've heard, so might be worth trying. Besides that, if you aren't using an out of source build you're digging your own pit. 14:42 erlehmann “if you aren't using an out of source build you're digging your own pit” wdym? 14:42 MTDiscord Switching branches should be seamless afaik unless the build script changes. 14:42 erlehmann yeah 14:43 erlehmann josiah_wi ninja will probably not solve these 2 specific problem unless you can make it rely on hashes (then it solves the incremental rebuild problem, but not the “not rebuilding enough” problem). i suggest to just try it. 14:43 erlehmann i mean, it has been a while since i looked at the state of ninja 14:44 erlehmann josiah_wi btw this is the ninja bug on the issue https://github.com/ninja-build/ninja/issues/1459 14:44 erlehmann > Ideally, I would be able to switch from my current work branch to some other branch, not touch any files and then switch back to the original branch and not have to rebuild anything at all. 14:44 celeron55 i'll just throw my opinion here if it helps: i think our current build system works just fine, no need to get stuck on this 14:46 erlehmann it works fine for you, but even on a fast machine switching branches would mean waiting 5 minute for a rebuild, or not?? 14:47 sfan5 depends on how far you switch, my builds are usually done within 2 minutes 14:47 MTDiscord I’m tired of people contributing under the guise of fixing problems that don’t exist 14:47 sfan5 if you switch across the zstd introduction you're bound to have a lot of changes and the build scripts were touched too 14:47 erlehmann sfan5, then the fastest machine you have is about twice as fast as the work machine i have access too i guess 14:47 MTDiscord Less Don Quixote more look at the thousand long issues 14:48 erlehmann josiah_wi i suggest to try ninja with the changes from this PR https://github.com/ninja-build/ninja/pull/929 14:48 erlehmann Jordach btw do you use md2 animations anywhere or did you plan to? 14:48 MTDiscord Negative 14:48 MTDiscord I have a personal interest of improving the build system so I'm volunteering to see what I can do. I am fairly confident you should be able to switch branches with no recompiling at all. 14:48 erlehmann josiah_wi please tell me if it works if you do 14:49 erlehmann using that ninja branch is *probably* the least effort (but you need to build ninja for that first) 14:49 MTDiscord With Makefiles. 14:49 MTDiscord Yuck 14:49 erlehmann uh 14:50 erlehmann josiah_wi make uses timestamp based rebuilds, i am pretty certain that can't work 14:50 MTDiscord Does git not preserve timestamps? 14:50 MTDiscord No 14:51 MTDiscord And if you’re that adamant on switching between branches why not use multiple dev folders 14:52 MTDiscord Well, git not preserving timestamps kind of mandates the hash checking lol. Now I understand where erlehmann is coming from! 14:52 erlehmann from all build systems i have seen, make is about the worst by a very far margin. you will only waste your time trying workarounds, seriously. i have spent a long time on build systems and depending how good you are at this, you can waste any time between 2 hours and 2 weeks on failing to come up with a solution before you give up. 14:52 erlehmann therefore, try the ninja thing 14:52 MTDiscord The obvious solution is to keep it deadpan simple 14:53 MTDiscord But this is Minetest and unfortunately that is folly 14:55 erlehmann Jordach i have years of experience that “this 100 line shell script builds faster than this overengineered 3000 line python monstrosity with dozens of dependencies” usually gets you laughed out of the room, but let's not go there too much. there is a possible solution (the ninja PR 929) and it has not been tried. 14:55 erlehmann i wlll not comment on rebuilds further unless i find a new concrete problem. 14:55 erlehmann (or someone else asks things) 14:56 MTDiscord Unfortunately the younguns don’t seem to understand what minimalism and simplicity are 14:58 erlehmann apart from that, anyone eperienced with gdb? 14:58 erlehmann experienced 14:58 erlehmann i want to break on uban trapping to examine if/how the thing could be exploited 15:47 erlehmann please take this trace for a bug (i guess) in the rollback code as a token of my goodwill: https://mister-muffin.de/p/fSgo.txt 15:53 erlehmann i can only guess, but could it be that this is the case of an uninitialized bool just taking whatever was there before? 15:59 Desour erlehmann: the inventory_add bool member of RollbackAction is not initialized by the default constructor, and the ub in your trace happens in the copy constructor, so my guess is it's just that. you could try again with a default value for that member 16:06 erlehmann Desour ok cool thx 16:33 erlehmann https://gist.github.com/appgurueu/63ffccbdd72c18506dc22d5206394de1 mentions 1233 files with the ending dae. those maybe COLLADA static meshes that irrlicht could load, but wasn't supprt for that removed recently? 16:33 sfan5 Minetest never supported loading Collada meshes 16:34 erlehmann bc it was gated at some point? 16:34 erlehmann i have no idea where to get the smaller set of what minetest could load 16:34 sfan5 not at some point, always 16:35 erlehmann from the larger set of “what is in contentdb and what irrlicht could load” 16:35 sfan5 https://github.com/minetest/minetest/blob/5.0.1/src/client/client.cpp#L681-L684 16:37 erlehmann LOL https://github.com/minetest/minetest/pull/9753/files 16:37 erlehmann what could have been 16:40 erlehmann i guess those 1000 files get somehow exported then 16:40 erlehmann well, i am off 17:09 pgimeno hm, I've seen clang do funny things with bools, I've seen it using them as a multiplier for a factor added to an index, assuming they can only be 0 or 1 (and causing a crash for indexing out of bounds when not) 17:13 pgimeno so I'd say a bool taking an illegal value is a serious issue 18:13 sfan5 rubenwardy: do you know how reproducible #9107 was? 18:13 ShadowBot https://github.com/minetest/minetest/issues/9107 -- arm64-v8a. Packets time out in singleplayer. RE-SENDING timed-out 18:14 rubenwardy The symptom is very reproduceable, it kills loading performance 18:15 rubenwardy 32bit loading is measured in tens of seconds, 64bit in many minutes 18:17 sfan5 well with #11607 I can't reproduce it anymore (64bit, mineclone5, cache deleted everytime) 18:17 ShadowBot https://github.com/minetest/minetest/issues/11607 -- Shave off buffer copies in networking code by sfan5 18:18 sfan5 the messages shown in the bug report still appear and loading takes about 2 minutes but that mostly appears to be MT actually doing something 18:22 sfan5 less actually 2021-09-11 20:20:23 -> 2021-09-11 20:21:31 18:23 sfan5 up until 2021-09-11 20:21:03 is just server init 18:26 sfan5 (Mineclone would really benefit from sending jukebox tracks on demand using dynamic media) 18:39 Krock ambience mods too 18:46 sfan5 rubenwardy: if you get a moment to test please do 18:47 MTDiscord assume mods that play background music would as well? 18:48 sfan5 yes 18:48 sfan5 some of the slowness comes from the engine (#10603) but not transferring the files at all (until needed) is the best solution 18:48 ShadowBot https://github.com/minetest/minetest/issues/10603 -- Music / large sound files significantly slow down game startup 18:50 Desour not only large sound files slow down startup. recently I've measured that in mtg 1/4 of the client startup time (about 0.5 s) is used for sound loading. (I'm actually currently trying to improve the sound loading) 18:52 sfan5 does anyone have any comments on irr#64 18:52 ShadowBot https://github.com/minetest/irrlicht/issues/64 -- Readd TGA format support by rollerozxa 19:01 sfan5 merging #11604 in 5m 19:01 ShadowBot https://github.com/minetest/minetest/issues/11604 -- Make sure relevant std::stringstreams are set to binary by sfan5 19:01 Krock tbh I don't see the necessity for TGA 19:01 rubenwardy I currently don't have Android Studio installed, I'm planning on distro hopping again soon 19:02 rubenwardy Krock: compatability 19:02 sfan5 CI builds an apk 19:02 Krock so re-add support, add a deprecation warning, and remove in 6.x 19:02 Krock PNG and JPEG are good enough 19:02 sfan5 but you need apksigner from the SDK to sign it because android doesn't accept it otherwise... 19:03 sfan5 .../android-sdk-linux/build-tools/30.0.3/apksigner sign --ks ./.android/debug.keystore app-arm64-v8a-release-unsigned.apk the default password is android 19:03 MTDiscord Krock: TGA is used in mineclone for generating map textures on the fly because it is a very easy format to write 19:04 sfan5 thats not the full story, if they did it on the fly they could switch to minetest.encode_png 19:04 sfan5 the fully story is that TGA textures are generated once and stay around 19:29 appguru Considering that escaping doesn't apply to the [inventorycube texture modifier, I assume nesting that modifier is not possible? 19:35 sfan5 I guess 19:36 Krock appguru: try to escape inner args with str:gsub("%^%[", "\\%^\\%["):gsub(":", "\\:"). does that work? 19:37 Krock at least that works for [combine 19:37 appguru The docs say escaping doesn't apply to inventorycube - it applies to combine though. I'm currently trying to write a compliant parser in Lua. 19:37 Krock nvm then I guess 19:38 appguru We really need a better syntax 19:38 appguru What would you recommend in terms of backwards compatibility? 19:39 Krock to not change anything 19:39 appguru A clean format would greatly simplify the parser. 20:52 erlehmann hey everyone, i think i made rebuilds much faster, just use this commit hook which sets the file time to the last commit that the file had: https://mister-muffin.de/p/SeoO 20:53 erlehmann josiah_wi sfan5 take note :) 21:54 fluxionary anyone have any insight on the following stacktrace? https://pastebin.com/34AT4z3B 21:54 fluxionary segfault on trying to step an SAO 21:56 sfan5 I've spoken to Alias about it 21:56 sfan5 I believe the crash isn't the true cause but rather just the server crashing while trying to shut down and before it gets to print the error message 21:57 Alias We can use this channel for common exchange. If you want me to add some more logging into the code and rebuild, I can do that. 21:57 fluxionary ah, that makes some sense 21:58 fluxionary because it's still trying to process things while in the middle of an emergency shutdown 22:00 Alias And that may also be the cause why I only capture "different errors" in the logs. 22:01 erlehmann regarding threads, i have built minetest with -fsenitize=thread before and it is full of data races. a regular shutdown triggers none of them though. 22:02 hendursaga Anyone aware of a mod that uses cqueues or otherwise has non-blocking IO? 22:02 erlehmann but i assume the threads are not properly managed from that 22:03 erlehmann i'll try to crash it in a known way (the nether portal stack overflow trick) and see what it says then 22:04 erlehmann explanation: basically any game action that causes enough recursion can crash minetest, for example, destroying a huge portal in a mod that handles destruction in a recursive manner 22:04 Alias If you have guesses, we can try them out on a testserver, on the same machine with the same set of mods. 22:05 sfan5 Alias: in your gdb command line swap 'bt full' for 'thread apply all bt full' and apply this patch http://sprunge.us/v4waNf 22:05 erlehmann nice 22:05 sfan5 will hopefully fix *or* delay the crash so that the real error is printed first 22:06 Alias Wilco. (Now I need to quicklearn how to apply patch files. I weaseled myself around this part of git until now) 22:10 erlehmann Alias, “man git apply” 22:11 sfan5 patch -p1 also works 22:12 Alias I added the patch to both test and live server 22:13 Pexin note: if using "patch" you may need to remove .orig files for build to not choke. at least the language stuff 22:14 Pexin (if the patch touches any language / translation files) 22:26 Alias The testserver is up now, anthing specific I could do to reproduce the error? 22:28 sfan5 dunno, maybe spawn some yl_nether_mobs:wither and it'll break 22:46 erlehmann i am sorry, i was mistaken: there seems to be indeed some kind of thread weirdness if you manage to cause a c stack overflow from the lua side, but it that point you are already overflowing the stack, so basically anything can happen. 22:46 erlehmann i think it has nothing to do with the error Alias experiences 22:47 erlehmann it seems to be possible to crash minetest into a state where if you start it again it immediately loads a world instead of the menu, anyone have any info on that? 22:48 sfan5 that's just you holding return for too long 22:48 erlehmann ohhh 22:49 erlehmann i feel p dumb, thx