| Time |
Nick |
Message |
| 01:01 |
pgimeno |
found https://stackoverflow.com/questions/42587607/why-is-imul-used-for-multiplying-unsigned-numbers about exactly how signed and unsigned multiply differ - tldr: they only differ in the high word of the result, which isn't used in the case in point |
| 01:23 |
|
fluxionary joined #minetest-dev |
| 02:29 |
|
queria^clone joined #minetest-dev |
| 02:33 |
|
queria^clone joined #minetest-dev |
| 04:00 |
|
MTDiscord joined #minetest-dev |
| 07:01 |
|
twoelk left #minetest-dev |
| 07:18 |
|
specing joined #minetest-dev |
| 07:20 |
|
Alias joined #minetest-dev |
| 07:36 |
|
ssieb joined #minetest-dev |
| 08:46 |
sfan5 |
interesting |
| 09:33 |
|
Fixer joined #minetest-dev |
| 10:21 |
|
Bond00758 joined #minetest-dev |
| 10:21 |
|
calcul0n_ joined #minetest-dev |
| 10:53 |
erlehmann |
pgimeno sfan5 would a trace like this help with future bugfixing? https://mister-muffin.de/p/5f_y.txt |
| 10:54 |
erlehmann |
if there exists CI that runs the game with --random-input and i can *seed* the random input, i bet i can make it spit that ut |
| 10:54 |
erlehmann |
out |
| 10:54 |
erlehmann |
and yes, that was created on an x86-64 machine i have access to (it does not belong to me) |
| 10:55 |
pgimeno |
that helps somewhat, though the backtrace lacks args info, which is sometimes useful |
| 10:56 |
erlehmann |
i have no idea how to coax that out of the thing right now |
| 10:57 |
erlehmann |
but i hope it is better proof |
| 10:57 |
erlehmann |
and helps fix the callers of the functions |
| 10:57 |
erlehmann |
bc if the function is never even called with some arguments, maybe that can be a solution too! |
| 10:57 |
erlehmann |
though i think the burden is on the function |
| 10:57 |
erlehmann |
every function should work for all input it accepts or crash |
| 10:58 |
erlehmann |
(that way you fix them fast) |
| 10:58 |
erlehmann |
i am pretty certain that none of the devs has systematically looked for stuff like this so far, because there's a bunch of low hanging fruit |
| 10:59 |
erlehmann |
even though people are generally aware of it |
| 10:59 |
erlehmann |
that is why i proposed to add it to the CI, if there is CI that can run the game |
| 11:01 |
pgimeno |
I guess a proper backtrace could be generated with a) a debug build (not sure if you're already using it) and b) gdb |
| 11:01 |
erlehmann |
probably, i did that in the past at work |
| 11:02 |
erlehmann |
ohhh https://stackoverflow.com/questions/30809022/how-can-i-break-on-ubsan-reports-in-gdb-and-continue |
| 11:02 |
erlehmann |
> You can stop on an ubsan report via a breakpoint on __ubsan::ScopedReport::~ScopedReport or __ubsan::Diag::~Diag. These are private implementation details which might change in the future though. |
| 11:03 |
erlehmann |
so you would compile with ubsan and break on its error reporting |
| 11:03 |
erlehmann |
nice |
| 11:18 |
|
proller joined #minetest-dev |
| 12:46 |
erlehmann |
why exactly was the shadow mapping thing merged? i have followed the discussion and most of the comments seem to indicate it is low quality and breaks things. |
| 12:46 |
erlehmann |
maybe i am misunderstanding something about it. |
| 12:47 |
MTDiscord |
<Sublayer plank> I'd assume it was because it didn't break things too much to be in the way, and it will be fixed up before 5.5.0's actual release |
| 12:47 |
erlehmann |
i guess it will just be disabled |
| 12:47 |
MTDiscord |
<Sublayer plank> aw. why would it be? |
| 12:48 |
erlehmann |
that was the suggestion i read |
| 12:48 |
MTDiscord |
<Sublayer plank> like, it doesn't work on android, but on desktop it seems to work fairly well at a reasonable performance cost |
| 12:48 |
erlehmann |
i think it breaks 3d rendering and stuff like the drawers mod |
| 12:48 |
MTDiscord |
<Sublayer plank> the fact games can't control it can be said for other graphical settings as well |
| 12:48 |
erlehmann |
https://github.com/minetest/minetest/issues/11600 |
| 12:49 |
erlehmann |
i don't get the whining about games not being able to control it tbh |
| 12:49 |
erlehmann |
that only matters because it is buggy, right? |
| 12:50 |
MTDiscord |
<Warr1024> I think originally it was believed that shadows had minor issues and could be fixed up in time for 5.5, but now we're really itching to get SOMETHING out to users for 5.5 instead of holding back every other improvement already merged behind shadowmaps. |
| 12:50 |
MTDiscord |
<Warr1024> So we'd rather just disable shadowmaps, release 5.5, then re-enable them for the 5.6 stream and try to get them fixed up for 5.6 |
| 12:50 |
MTDiscord |
<Sublayer plank> is 5.4.2 for android still a thing? |
| 12:51 |
MTDiscord |
<Warr1024> Misestimations of how much work something will actually take to get it working right are par for the course in the software world, so this sort of thing shouldn't be too big a deal. |
| 12:51 |
MTDiscord |
<Warr1024> We are still hoping to get a fix out for the android thing, but again, the problems the change faced turned out to be worse than anticipated, so 5.4.2 and 5.5.1 to fix it would be out of band. |
| 12:52 |
erlehmann |
this looks like problems i have already seen |
| 12:52 |
erlehmann |
i.e. 1. merge stuff prematurely 2. “fixing it would take too much time” 3. release anyway |
| 12:53 |
erlehmann |
well, i am not here to whine, i just don't understand the dev process very well |
| 12:53 |
MTDiscord |
<Warr1024> Disabling instead of backing out the merge does have one advantage in that it reaffirms our commitment to actually fixing it instead of just scrapping it though. |
| 12:53 |
erlehmann |
some people like Wuzzy are *very* careful about their code and what it affects |
| 12:53 |
erlehmann |
others, not so much |
| 12:54 |
erlehmann |
so i guess the matter is not some objective quality, but if ppl are convinced then |
| 12:55 |
erlehmann |
and even groups can be in error about the quality of something |
| 12:55 |
MTDiscord |
<Warr1024> Error avoidance and error tolerance are 2 different strategies for making a system robust, and each one eventually starts to yield diminishing marginal returns. |
| 12:55 |
erlehmann |
or maintainers can be pressured into “just merge it, it works” |
| 12:56 |
erlehmann |
(i have been pressured like that and regretted it every time, every “let's skip testing” was a lie) |
| 12:56 |
MTDiscord |
<Warr1024> I tend to use low error avoidance strategies in almost everything I build, and it works pretty well for the most part. If I want more error avoidance, it usually requires bringing in somebody who thinks differently than me. |
| 12:56 |
erlehmann |
i mean “let's skip testing *because it works*” |
| 12:56 |
erlehmann |
what is “low error avoidance strategies”?? |
| 12:58 |
MTDiscord |
<Warr1024> I've never done "let's skip testing" per se, but the actual amount of testing is weighed against the cost of testing failure, and so you end up wanting less testing when you minimize the cost of failure. |
| 12:58 |
erlehmann |
„If I want more error avoidance, it usually requires bringing in somebody who thinks differently than me.” – i once had a programming project that i had to do entirely myself as a junior dev, because the senior dev quit and the freelancer replacement was not capable. for code review i got a coworker who had nothing to do with the project. only when i could convince her that my code was sound, i merged it. |
| 12:59 |
MTDiscord |
<Warr1024> I think it would be hard to make MTE's process that level of fault-tolerant though. |
| 12:59 |
erlehmann |
well at the rate of bugs i have found since starting looking for them a few days before there def seems room for improvement |
| 13:00 |
erlehmann |
i mean i guess others find bugs at the same rate |
| 13:00 |
erlehmann |
after all they have much more experience with the project |
| 13:02 |
erlehmann |
hmm rollback code seems faulty too? |
| 13:02 |
erlehmann |
brb |
| 13:04 |
MTDiscord |
<Warr1024> I just hate seeing people throwing more and more effort into preventing bugs, well into the realm of diminishing returns, and completely ignore factors that reduce the cost of inevitable bugs, like architectural or deployment changes. It's hard to get into specifics, though. MTE does SOME things right, like having a lot of users running the dev branch so we are likely to catch errors before they hit the official release, at least. |
| 13:05 |
erlehmann |
i think most of the effort can be automated through having more tests |
| 13:05 |
erlehmann |
to prevent regressions |
| 13:05 |
MTDiscord |
<Warr1024> I just don't sweat most bugs because I put a lot of thought up front into assuming there will be bugs and minimizing the cost of fixing them. |
| 13:05 |
erlehmann |
this is not meant as a personal attack, but in my entire life, everyone who feared one might “test too much” was proven wrong when both approaches were actually tried |
| 13:06 |
erlehmann |
oh yeah the architecture can ofc mitigate some stuff |
| 13:06 |
MTDiscord |
<Warr1024> Tests are great, it's just that it's pretty rare to find someone actually good at making them :-D |
| 13:07 |
MTDiscord |
<Warr1024> Personally I'm pretty terrible at designing, writing, or maintaining tests, especially compared to my ability to make forwards-code. |
| 13:07 |
MTDiscord |
<Warr1024> There are some areas where it's just not worth trying to cobble together a proper test and I just try to sweep it in my manual integration testing process, but there are a few others where I'll muddle through because the cases are just so byzantine that there's no way manual testing wouldn't be way too error-prone. |
| 13:08 |
erlehmann |
in my experience, if you do not require tests for new features or regression tests, you can not get ppl to write them tbh. not only will they forever be optional (thus more work), if people write the tests *after* the code wa written, it is a long painful process, because the code often is not testable if not written with testing in mind. |
| 13:09 |
erlehmann |
Warr1024 care to link me to some tests? if it is about nodecore, we should discuss in #minetest i guess |
| 13:09 |
erlehmann |
i want to learn |
| 13:09 |
MTDiscord |
<Warr1024> Haha, specific to NodeCore, I don't think I've got any automated tests. |
| 13:09 |
MTDiscord |
<Sublayer plank> what's minetest engine's unit test coverage like? |
| 13:09 |
MTDiscord |
<Sublayer plank> I can see there's some tests in src/unittest/ |
| 13:09 |
MTDiscord |
<Warr1024> I've only really done serious testing for like work stuff |
| 13:09 |
MTDiscord |
<Warr1024> For NodeCore I pretty much test manually |
| 13:10 |
MTDiscord |
<Warr1024> There aren't a lot of units in there that wouldn't be a real pain to wire up for testing, though tbh there are probably a few pieces that could benefit from them by now. |
| 13:10 |
erlehmann |
at work i did test driven development, that is how i learned that writing test first is easy, writing tests last is abysmal |
| 13:11 |
erlehmann |
but like, you can refactor without fear when you have tests |
| 13:11 |
erlehmann |
(that are 100% complete) |
| 13:11 |
MTDiscord |
<Sublayer plank> it'd be cool to have some sort of in-game gameplay unit testing similar to what minecraft does |
| 13:11 |
erlehmann |
this is commonly known as a “cheat client” and not wanted as far as i know |
| 13:12 |
erlehmann |
i have suggested it many times |
| 13:12 |
MTDiscord |
<Sublayer plank> aw man |
| 13:12 |
MTDiscord |
<Sublayer plank> actually it would probably more be for minetest games specifically |
| 13:12 |
MTDiscord |
<Sublayer plank> unless it'd be like... devtest |
| 13:13 |
MTDiscord |
<Warr1024> tbh I'm skeptical that writing tests first vs last is a really significant factor, at least compared to whether you designed the unit to be testable in the first place. As long as it's POSSIBLE to write tests I think it shouldn't be too bad. |
| 13:13 |
erlehmann |
i mean, the obvious way to figure out if say, a user can still dig a block (yes, mineclonia and mineclone5 managed to independently run into the same bug & tool stopped working) is to script it. but you can only do that with a cheat client right now. |
| 13:13 |
sfan5 |
what would "in game unit testing" imply? |
| 13:13 |
MTDiscord |
<Warr1024> I've written code first, tested it by hand, found it working well enough, used it, then wrote regression tests, tightened them up, discovered a bunch of bugs, fixed them, and then maintained the unit and tests together after that, and it felt fine :-D |
| 13:14 |
MTDiscord |
<Warr1024> But the way I see it you write code to be tested, or you write code to be thrown away and replaced ... but like 90% of code I write is probably the throw away kind. |
| 13:14 |
erlehmann |
Warr1024 i know such things, you might be good at designing interfaces – the idea of “tests first” is trying to work out where you want to go. it forces you to have a mental model of the resulting output and the interface. |
| 13:15 |
MTDiscord |
<Warr1024> Over the years I've found I'm gravitating toward the modern auto manufacturer strategy: design things in tight assemblies that are loosely linked to other things, and if a problem crops up in some component, just rip it out and replace the whole thing. |
| 13:15 |
erlehmann |
when you do not write them first, you can either make something testable or something that relies on global state or has too many branches to be easily testable. |
| 13:15 |
MTDiscord |
<Warr1024> Yeah, maybe that's a good excuse, I write the tests "mentally" but they just usually don't show up in actual code. XD |
| 13:15 |
erlehmann |
yeah |
| 13:16 |
erlehmann |
it is rare though that ppl are good in that, they usually need to be *able* to write the test at that point |
| 13:16 |
erlehmann |
oh also one more thing, if you have tests, you can fuzz! |
| 13:17 |
MTDiscord |
<Warr1024> I've tried formalizing tests in actual code and met with at least partial success. I find that the actual effort of writing and maintaining tests are much easier to justify when a unit is very "load bearing", so tests tend to be more robust near the foundational units of an architecture. New stuff gets added at the top, then pushed down as more utility for it is found, and it ends up getting assimilated into the test plan as that |
| 13:17 |
MTDiscord |
happens... |
| 13:17 |
erlehmann |
i think a 5.5 release with many bugs fixed would be nice btw, but it should not have regressions. |
| 13:18 |
MTDiscord |
<Sublayer plank> sfan5: doing automated tests in-game to see that everything works like it should. one example could be checking liquid spread, that it spreads to the specified amount of blocks, or placing a falling node and seeing so it falls to the ground properly. |
| 13:18 |
erlehmann |
sfan5 so about the zstd thing, i may have missed an answer if you gave it, can i force minetest to save maps not in zstd? |
| 13:18 |
MTDiscord |
<Sublayer plank> I hope this makes sense |
| 13:18 |
erlehmann |
sfan5, the reason is testing backwards compat with the same world |
| 13:19 |
sfan5 |
yeah that makes sense |
| 13:19 |
sfan5 |
the tests you're describing would not even involve a client |
| 13:19 |
sfan5 |
it's doable today, someone just has to write the test framework |
| 13:19 |
MTDiscord |
<Warr1024> A big problem for MTE is that users don't want fewer bugs, users want more features. If we want MT to be more popular then we need to add risky stuff, and that takes limited time and energy away from robustness. We aren't trying to maximize our audience as fervently as a commercial project would, but we don't want to languish in obscurity either; people need to be interested enough in the project in general for us to have a chance |
| 13:19 |
MTDiscord |
at finding new contributors. |
| 13:19 |
erlehmann |
is it possible to feed input to a headless minetest? like /setblock somewhere? |
| 13:20 |
sfan5 |
a headless minetest what? client? no, server? sure write a mod |
| 13:20 |
sfan5 |
>08:18 sfan5 erlehmann: you can edit src/serialization.h to only write format 28, this will keep your map compatible |
| 13:20 |
sfan5 |
(yesterday) |
| 13:20 |
erlehmann |
sfan5, thx |
| 13:21 |
MTDiscord |
<Warr1024> There is one command line switch for MT that creates random inputs, which I suppose is intended to be a sort of fuzzer. I don't know how efficient it is necessarily though; haven't actually tried using it yet... |
| 13:21 |
MTDiscord |
<Warr1024> Also doesn't MT have a "null" renderer that's intended to be used for automated testing e.g. in the CI containers or something? |
| 13:22 |
erlehmann |
“users don't want fewer bugs, users want more features” – in my experience as a player, developers of games think this all the time, but it is generally false. there usually exists a small very vocal subset of players for any game that cares about *new* features, the silent majority cares about stability and having no regressions. |
| 13:22 |
MTDiscord |
<Warr1024> Eh, well, in the case of commercial software, your goal is to make people BUY the game, but retention is less of a concern :-) |
| 13:22 |
erlehmann |
in fact, i know 3 servers that did stuff like running forks of minetest or not upgrading games because of lag |
| 13:22 |
MTDiscord |
<Sublayer plank> irrlichtmt's automated ci testing uses xvfb which, uh, I think makes it not use the null renderer |
| 13:23 |
MTDiscord |
<Sublayer plank> oh no wait, it does both xvfb and null |
| 13:24 |
MTDiscord |
<Warr1024> The thing to remember though is that developers think of bugs as some kind of fundamentally different thing to features, but to users lack of bugs is a feature pretty much like any other and it's weighed in the balance. |
| 13:24 |
sfan5 |
@Warr1024 we have an integration test on the CI using the null renderer |
| 13:24 |
MTDiscord |
<Warr1024> I've heard some people say "always fix bugs before adding new features" but that's only necessarily true if you use a very loose definition for "always" |
| 13:24 |
erlehmann |
Warr1024 to users a bug is “can't play this thing” or “can't use this thing in the game, even though i could before” |
| 13:25 |
erlehmann |
like, assume the TGA thing would *not* be fixed by the next release, server owners would definitely share what the last “good” release was etc. (i.e. such things already happened) |
| 13:26 |
erlehmann |
bc players are very quick to leave |
| 13:26 |
erlehmann |
if your server ruins something |
| 13:26 |
MTDiscord |
<Warr1024> Yeah, a bug basically just cancels out a feature (and some portion of a user's baseline goodwill toward you) but it's not literally impossible for you to make an improvement elsewhere whose benefit outweighs the bug's cost, thus having it actually make sense to defer the bug. |
| 13:26 |
erlehmann |
“a user's baseline goodwill toward you”, yeah that's it |
| 13:27 |
MTDiscord |
<Warr1024> Also resources spent on one thing are not strictly rival against resources spent on another thing. If someone is only really good with visuals, then you're not really taking away resources that might have otherwise been spent fixing a database bug by merging their improvement. |
| 13:27 |
erlehmann |
oh, i only mean case where stuff legitimately has an either/or choice |
| 13:27 |
erlehmann |
like a new feature that breaks old stuff |
| 13:29 |
erlehmann |
i mean: everyone who told me that users totally prefer new features even if it breaks some old stuff was a developer (for minetest this applies both to engine and games/mods) – whereas almost everyone who complained about devs adding new features that break old stuff was just a normal user, the type of person who has a grievance but won't even make a bug report. |
| 13:29 |
MTDiscord |
<Warr1024> The MT project sees a fair amount of turnover (including even just cyclical activity levels) among contributors, and so I don't think we have a definite risk-affinity/risk-avoidance level. That means that if you look at the project as a whole over time and expect some consistency as to how risky decisions are made, you'd see a lot of stuff that looks paradoxical, like rejecting some low-risk stuff as too risky, but merging big-risk |
| 13:29 |
MTDiscord |
stuff later :-) |
| 13:30 |
erlehmann |
well i'm happy if it moves into the right direction, like “do not make stuff slower on old computers to make it faster on new computers” |
| 13:31 |
erlehmann |
which so far has worked out (see the icon scaling code) |
| 13:31 |
MTDiscord |
<Warr1024> Developers are not as good as users are at understanding what users really value, and users are not as good as developers at understanding what things actually cost to do. But then, developers themselves are terrible at estimating, and users are pretty terrible at predicting what they'll actually want, so I guess that equalizes things somewhat. |
| 13:31 |
erlehmann |
i'd say it is more nuanced |
| 13:31 |
MTDiscord |
<Warr1024> Sure, it always is :-) |
| 13:32 |
erlehmann |
i.e. users are absolutely untrustworthy in what they want if it is about features that do not yet exist. but if they had a taste of a feature and you take it away, they will only accept it if that feature itself caused trouble. |
| 13:32 |
erlehmann |
this can be as simple as “we removed X because it caused lag” |
| 13:33 |
MTDiscord |
<Warr1024> Haha, my favorite is "we removed X because you lied to us when you said you wanted it and would use it." |
| 13:34 |
erlehmann |
oh that one is always mean, because the followup is either nothing or that someone who did use it is extremely pissed |
| 13:35 |
erlehmann |
i.e. the people did lie, but it does not matter |
| 13:35 |
erlehmann |
so, regarding engine stuff, i am trying to figure out how to trap on UB |
| 13:35 |
erlehmann |
to deliver you stack trace |
| 13:36 |
MTDiscord |
<Warr1024> Well, I was once on a project where the whole project was canceled because the customers said they loved it and would pay thousands for it, but when we actually delivered 6 months later, not a single one of them was interested, even at a lower price than they themselves bid. That's maybe the real reason why software shops tend to sell things and take the money first, and then actually build them later under threat of breach of |
| 13:36 |
MTDiscord |
contract. |
| 13:37 |
MTDiscord |
<Warr1024> One person being extremely pissed is also a thing you have to weigh in the balance, and to be frank every project can have its own more or less arbitrary function for determining what exactly they want to optimize. |
| 13:40 |
erlehmann |
“customers said they loved it and would pay thousands for it” – get it in writing |
| 13:41 |
MTDiscord |
<Warr1024> If by "in writing" you mean wait for the check to actually clear, yeah. :-D |
| 13:41 |
erlehmann |
hehe |
| 13:42 |
erlehmann |
Warr1024 in my experience, the “one person being extremely pissed” case usually means there are tens or hundreds sometimes that are pissed, but a) not so much or b) have no idea how to do anything about it, i.e. not even how te reach the devs |
| 13:42 |
erlehmann |
te → to |
| 13:43 |
MTDiscord |
<Warr1024> Sure, but that's just another correction factor to apply when weighing it. Also the potential that person has to create damage by being disproportionately vocal. |
| 13:44 |
MTDiscord |
<Warr1024> But on the other hand, once a project reaches a certain scope, the risk that somebody is going to be seriously pissed off regardless of the nature of the change goes up too, i.e. there stops being obvious unequivocally good things that you can do ... so you have to get good at handling that kind of thing regardless. |
| 13:48 |
|
calcul0n__ joined #minetest-dev |
| 13:54 |
erlehmann |
Warr1024 not really. there exist a bunch of projects out there that can go 10, 15, 20 years adding features but they do it in a way that stuff does not break. |
| 13:54 |
erlehmann |
and then there is stuff like inkscape that changes the meaning of command line parameters randomly lol |
| 13:54 |
erlehmann |
btw, i think minetest building is cursed |
| 13:55 |
erlehmann |
it seems to me that a) too much stuff is rebuilt in the common case (i.e. going between commits/branches) b) the dependencies are underdetermined, i.e. swhen recompiling it does not create a binary corresponding to a clean build |
| 13:57 |
erlehmann |
i assume that this has been known but disregarded as not important, but you should know that the result of the latter actually means you could have tested a binary that does not correspond to the branch you tried to verify |
| 14:00 |
sfan5 |
so you're saying too much stuff is rebuild but also too few |
| 14:00 |
sfan5 |
rebuilt* |
| 14:00 |
erlehmann |
yes, but different cases |
| 14:01 |
erlehmann |
doing a rebuild on the same branch/commit leads to the “to few dependencies are considered, build might be faulty unless you rebuild from scratch” thing |
| 14:02 |
erlehmann |
whereas checking out some branch/commit appears to needlessly rebuild stuff |
| 14:06 |
erlehmann |
i could explain it in depth, but it does not really matter, as it is easy to verify & generally both bugs are long-standing issues with cmake & make that can not be fixed |
| 14:06 |
erlehmann |
i just want ppl to be aware of it |
| 14:07 |
erlehmann |
bc *usually* devs trust the build system |
| 14:08 |
erlehmann |
and by “can not be fixed” i mean “the architecture of both cmake and make means it is imposible to fully work around or patch the behaviour” |
| 14:08 |
|
appguru joined #minetest-dev |
| 14:09 |
erlehmann |
the “not rebuilding enough” thing basically means you have to rebuild everything if you change anything to be sure you get the proper output, but that is basically always the case for C++ projects that use cmake / make, even for a hello world. |
| 14:10 |
erlehmann |
ppl usually do not notice these things anyway, bc “not rebuilding enough” often means “my binary is at some previous commit” which can be close enough |
| 14:11 |
erlehmann |
but i have a guy who after i told them that were like “oof the last commit before the release was an important fix, we definitely have the wrong binary here” |
| 14:11 |
erlehmann |
just remember it if i.e. some feature does not show up even though you had a “proper” partial rebiuld |
| 14:11 |
erlehmann |
rebuild |
| 14:12 |
erlehmann |
btw this can be caught through testing the binaries, but i have no idea who does that |
| 14:13 |
erlehmann |
Warr1024 if you divide the mouse movement on the --random-input option by 10 or so it works well |
| 14:13 |
erlehmann |
Warr1024 otherwise it moves too fast to be usable for testing |
| 14:14 |
erlehmann |
Warr1024 i also suggest to increae the digging time in the RandomInputHandler or what it is called |
| 14:14 |
erlehmann |
Warr1024 and add a CSM that a) respawns you on death b) switches to the first hotbar slot that has contents if your current slot is empty |
| 14:15 |
MTDiscord |
<josiah_wi> If an issue for the build stuff is started or if we discuss it here I will try to improve it. I've frequently had to rebuild from scratch every time the CMakeLists has a diff. |
| 14:16 |
MTDiscord |
<Warr1024> I too easily "forget" the pain of ordinary routine stuff like compiling MT because that pain is encapsulated in my scripts :-) |
| 14:17 |
erlehmann |
well, if you can coax cmake into outputting files for a build system that checks timestamp & hashs instead not just timestamp, the issue will not be completely fixed, but it is a step into the right direction |
| 14:18 |
MTDiscord |
<josiah_wi> Will Ninja do that? It's as easy as passing -G ninja then. |
| 14:18 |
erlehmann |
no |
| 14:18 |
erlehmann |
ninja is one of the systems that can be fixed, though. ninja has a PR for hash bassed checking, but i think it was rejected. |
| 14:20 |
MTDiscord |
<josiah_wi> What build system would you recommend for starting to work on this? |
| 14:20 |
erlehmann |
in general, the only build systems that can be improved to handle both of these cases (rebuilding too much and not rebuilding enough) are those who do both hash-based comparisons if the timestamp differs and can handle non-existence dependencies (i.e. if a previouly non-existing file at a path exists, a target needs to be rebuilt). |
| 14:21 |
erlehmann |
the number of systems that can do both is pretty small |
| 14:21 |
erlehmann |
for some reason |
| 14:24 |
erlehmann |
josiah_wi i may be a bit biased, but i have written an implementation of the “redo” design by DJB in about 400 lines of shell script that is amusingly even smaller than the years-old patch that tried to fix cmake (which turned out to be unworkable) http://news.dieweltistgarnichtso.net/bin/redo-sh.html |
| 14:25 |
erlehmann |
josiah_wi if i wanted to fix this as fast as possible, i would use this. not because i wrote it, but because it is among all build systems i know the one with the smallest set of dependencies and minimal special syntax (build scripts can be any language) |
| 14:26 |
erlehmann |
josiah_wi to see this in action, you can build this C++ game with it: https://github.com/linleyh/liberation-circuit |
| 14:26 |
|
Desour joined #minetest-dev |
| 14:27 |
erlehmann |
josiah_wi note that there is also a small ~100 line script called “do” there that always rebuilds everything |
| 14:28 |
erlehmann |
josiah_wi i have a small test case for this thing being faster than ninja on incremental rebuilds (but tbh ninja would be A LOT faster if they would merge the hash checking code): https://mister-muffin.de/p/W8EA.gz |
| 14:31 |
erlehmann |
josiah_wi btw a good question to figure out if build systems are fixable in terms of not rebuilding when they should is “is the dependency graph attempted to be generated before the build or is it generated as a side effect of the build itself?” because there are some common cases in which that matters, but just read the papers linked on my page, others have expressed it better than i can. |
| 14:32 |
erlehmann |
josiah_wi also note that i do not advocate ripping out cmake/make. i would like faster incremental builds myself too. |
| 14:32 |
sfan5 |
I would advocate not wasting any time on this |
| 14:32 |
erlehmann |
yeah that is usually the response. |
| 14:33 |
erlehmann |
i mean “just rebuild everything completely if you test it” is workable for most things minetest |
| 14:35 |
erlehmann |
sfan5, i noticed the incremental rebuilds problem while trying to coax minetest to spit out a trace including variables btw |
| 14:35 |
MTDiscord |
<josiah_wi> Integration with CMake is kind of an important limitation. |
| 14:35 |
erlehmann |
that's how i got to it |
| 14:36 |
erlehmann |
josiah_wi not necesarily, non-devs do not need fast incremental rebuilds. |
| 14:37 |
erlehmann |
liberation circuit has both dofiles and cmake / make integration. granted, the make invocation builds the project only correctly on every second invocation and the cmake thing will *probably* only on full rebuilds, but it seems to work out. |
| 14:38 |
erlehmann |
josiah_wi Warr1024 if you want to share solutions for faster incremental rebuilds, i have a full build time of >30 min (which shows how bad this problem can be), so i am highly interested. |
| 14:38 |
erlehmann |
especially i am interested in already existing solutions |
| 14:39 |
erlehmann |
(compared to something i just sketched that requires putting time into it) |
| 14:41 |
MTDiscord |
<josiah_wi> Ninja > Makefiles I've heard, so might be worth trying. Besides that, if you aren't using an out of source build you're digging your own pit. |
| 14:42 |
erlehmann |
“if you aren't using an out of source build you're digging your own pit” wdym? |
| 14:42 |
MTDiscord |
<josiah_wi> Switching branches should be seamless afaik unless the build script changes. |
| 14:42 |
erlehmann |
yeah |
| 14:43 |
erlehmann |
josiah_wi ninja will probably not solve these 2 specific problem unless you can make it rely on hashes (then it solves the incremental rebuild problem, but not the “not rebuilding enough” problem). i suggest to just try it. |
| 14:43 |
erlehmann |
i mean, it has been a while since i looked at the state of ninja |
| 14:44 |
erlehmann |
josiah_wi btw this is the ninja bug on the issue https://github.com/ninja-build/ninja/issues/1459 |
| 14:44 |
erlehmann |
> Ideally, I would be able to switch from my current work branch to some other branch, not touch any files and then switch back to the original branch and not have to rebuild anything at all. |
| 14:44 |
celeron55 |
i'll just throw my opinion here if it helps: i think our current build system works just fine, no need to get stuck on this |
| 14:46 |
erlehmann |
it works fine for you, but even on a fast machine switching branches would mean waiting 5 minute for a rebuild, or not?? |
| 14:47 |
sfan5 |
depends on how far you switch, my builds are usually done within 2 minutes |
| 14:47 |
MTDiscord |
<Jordach> I’m tired of people contributing under the guise of fixing problems that don’t exist |
| 14:47 |
sfan5 |
if you switch across the zstd introduction you're bound to have a lot of changes and the build scripts were touched too |
| 14:47 |
erlehmann |
sfan5, then the fastest machine you have is about twice as fast as the work machine i have access too i guess |
| 14:47 |
MTDiscord |
<Jordach> Less Don Quixote more look at the thousand long issues |
| 14:48 |
erlehmann |
josiah_wi i suggest to try ninja with the changes from this PR https://github.com/ninja-build/ninja/pull/929 |
| 14:48 |
erlehmann |
Jordach btw do you use md2 animations anywhere or did you plan to? |
| 14:48 |
MTDiscord |
<Jordach> Negative |
| 14:48 |
MTDiscord |
<josiah_wi> I have a personal interest of improving the build system so I'm volunteering to see what I can do. I am fairly confident you should be able to switch branches with no recompiling at all. |
| 14:48 |
erlehmann |
josiah_wi please tell me if it works if you do |
| 14:49 |
erlehmann |
using that ninja branch is *probably* the least effort (but you need to build ninja for that first) |
| 14:49 |
MTDiscord |
<josiah_wi> With Makefiles. |
| 14:49 |
MTDiscord |
<Jordach> Yuck |
| 14:49 |
erlehmann |
uh |
| 14:50 |
erlehmann |
josiah_wi make uses timestamp based rebuilds, i am pretty certain that can't work |
| 14:50 |
MTDiscord |
<josiah_wi> Does git not preserve timestamps? |
| 14:50 |
MTDiscord |
<Jordach> No |
| 14:51 |
MTDiscord |
<Jordach> And if you’re that adamant on switching between branches why not use multiple dev folders |
| 14:52 |
MTDiscord |
<josiah_wi> Well, git not preserving timestamps kind of mandates the hash checking lol. Now I understand where erlehmann is coming from! |
| 14:52 |
erlehmann |
from all build systems i have seen, make is about the worst by a very far margin. you will only waste your time trying workarounds, seriously. i have spent a long time on build systems and depending how good you are at this, you can waste any time between 2 hours and 2 weeks on failing to come up with a solution before you give up. |
| 14:52 |
erlehmann |
therefore, try the ninja thing |
| 14:52 |
MTDiscord |
<Jordach> The obvious solution is to keep it deadpan simple |
| 14:53 |
MTDiscord |
<Jordach> But this is Minetest and unfortunately that is folly |
| 14:55 |
erlehmann |
Jordach i have years of experience that “this 100 line shell script builds faster than this overengineered 3000 line python monstrosity with dozens of dependencies” usually gets you laughed out of the room, but let's not go there too much. there is a possible solution (the ninja PR 929) and it has not been tried. |
| 14:55 |
erlehmann |
i wlll not comment on rebuilds further unless i find a new concrete problem. |
| 14:55 |
erlehmann |
(or someone else asks things) |
| 14:56 |
MTDiscord |
<Jordach> Unfortunately the younguns don’t seem to understand what minimalism and simplicity are |
| 14:58 |
erlehmann |
apart from that, anyone eperienced with gdb? |
| 14:58 |
erlehmann |
experienced |
| 14:58 |
erlehmann |
i want to break on uban trapping to examine if/how the thing could be exploited |
| 15:47 |
|
Desour joined #minetest-dev |
| 15:47 |
erlehmann |
please take this trace for a bug (i guess) in the rollback code as a token of my goodwill: https://mister-muffin.de/p/fSgo.txt |
| 15:53 |
erlehmann |
i can only guess, but could it be that this is the case of an uninitialized bool just taking whatever was there before? |
| 15:54 |
|
Extex joined #minetest-dev |
| 15:59 |
Desour |
erlehmann: the inventory_add bool member of RollbackAction is not initialized by the default constructor, and the ub in your trace happens in the copy constructor, so my guess is it's just that. you could try again with a default value for that member |
| 16:06 |
erlehmann |
Desour ok cool thx |
| 16:07 |
|
fluxionary joined #minetest-dev |
| 16:09 |
|
hendursaga joined #minetest-dev |
| 16:33 |
erlehmann |
https://gist.github.com/appgurueu/63ffccbdd72c18506dc22d5206394de1 mentions 1233 files with the ending dae. those maybe COLLADA static meshes that irrlicht could load, but wasn't supprt for that removed recently? |
| 16:33 |
sfan5 |
Minetest never supported loading Collada meshes |
| 16:34 |
erlehmann |
bc it was gated at some point? |
| 16:34 |
erlehmann |
i have no idea where to get the smaller set of what minetest could load |
| 16:34 |
sfan5 |
not at some point, always |
| 16:35 |
erlehmann |
from the larger set of “what is in contentdb and what irrlicht could load” |
| 16:35 |
sfan5 |
https://github.com/minetest/minetest/blob/5.0.1/src/client/client.cpp#L681-L684 |
| 16:37 |
erlehmann |
LOL https://github.com/minetest/minetest/pull/9753/files |
| 16:37 |
erlehmann |
what could have been |
| 16:40 |
erlehmann |
i guess those 1000 files get somehow exported then |
| 16:40 |
erlehmann |
well, i am off |
| 16:58 |
|
Fixer_ joined #minetest-dev |
| 17:09 |
pgimeno |
hm, I've seen clang do funny things with bools, I've seen it using them as a multiplier for a factor added to an index, assuming they can only be 0 or 1 (and causing a crash for indexing out of bounds when not) |
| 17:11 |
|
Extex joined #minetest-dev |
| 17:13 |
pgimeno |
so I'd say a bool taking an illegal value is a serious issue |
| 18:13 |
sfan5 |
rubenwardy: do you know how reproducible #9107 was? |
| 18:13 |
ShadowBot |
https://github.com/minetest/minetest/issues/9107 -- arm64-v8a. Packets time out in singleplayer. RE-SENDING timed-out |
| 18:14 |
rubenwardy |
The symptom is very reproduceable, it kills loading performance |
| 18:15 |
rubenwardy |
32bit loading is measured in tens of seconds, 64bit in many minutes |
| 18:17 |
sfan5 |
well with #11607 I can't reproduce it anymore (64bit, mineclone5, cache deleted everytime) |
| 18:17 |
ShadowBot |
https://github.com/minetest/minetest/issues/11607 -- Shave off buffer copies in networking code by sfan5 |
| 18:18 |
sfan5 |
the messages shown in the bug report still appear and loading takes about 2 minutes but that mostly appears to be MT actually doing something |
| 18:22 |
sfan5 |
less actually 2021-09-11 20:20:23 -> 2021-09-11 20:21:31 |
| 18:23 |
sfan5 |
up until 2021-09-11 20:21:03 is just server init |
| 18:26 |
sfan5 |
(Mineclone would really benefit from sending jukebox tracks on demand using dynamic media) |
| 18:39 |
Krock |
ambience mods too |
| 18:43 |
|
longerstaff13 joined #minetest-dev |
| 18:46 |
sfan5 |
rubenwardy: if you get a moment to test please do |
| 18:47 |
MTDiscord |
<Jonathon> assume mods that play background music would as well? |
| 18:48 |
sfan5 |
yes |
| 18:48 |
sfan5 |
some of the slowness comes from the engine (#10603) but not transferring the files at all (until needed) is the best solution |
| 18:48 |
ShadowBot |
https://github.com/minetest/minetest/issues/10603 -- Music / large sound files significantly slow down game startup |
| 18:50 |
Desour |
not only large sound files slow down startup. recently I've measured that in mtg 1/4 of the client startup time (about 0.5 s) is used for sound loading. (I'm actually currently trying to improve the sound loading) |
| 18:52 |
sfan5 |
does anyone have any comments on irr#64 |
| 18:52 |
ShadowBot |
https://github.com/minetest/irrlicht/issues/64 -- Readd TGA format support by rollerozxa |
| 19:01 |
sfan5 |
merging #11604 in 5m |
| 19:01 |
ShadowBot |
https://github.com/minetest/minetest/issues/11604 -- Make sure relevant std::stringstreams are set to binary by sfan5 |
| 19:01 |
Krock |
tbh I don't see the necessity for TGA |
| 19:01 |
rubenwardy |
I currently don't have Android Studio installed, I'm planning on distro hopping again soon |
| 19:02 |
rubenwardy |
Krock: compatability |
| 19:02 |
sfan5 |
CI builds an apk |
| 19:02 |
Krock |
so re-add support, add a deprecation warning, and remove in 6.x |
| 19:02 |
Krock |
PNG and JPEG are good enough |
| 19:02 |
sfan5 |
but you need apksigner from the SDK to sign it because android doesn't accept it otherwise... |
| 19:03 |
sfan5 |
.../android-sdk-linux/build-tools/30.0.3/apksigner sign --ks ./.android/debug.keystore app-arm64-v8a-release-unsigned.apk the default password is android |
| 19:03 |
MTDiscord |
<Sublayer plank> Krock: TGA is used in mineclone for generating map textures on the fly because it is a very easy format to write |
| 19:04 |
sfan5 |
thats not the full story, if they did it on the fly they could switch to minetest.encode_png |
| 19:04 |
sfan5 |
the fully story is that TGA textures are generated once and stay around |
| 19:14 |
|
specing_ joined #minetest-dev |
| 19:29 |
appguru |
Considering that escaping doesn't apply to the [inventorycube texture modifier, I assume nesting that modifier is not possible? |
| 19:35 |
sfan5 |
I guess |
| 19:36 |
Krock |
appguru: try to escape inner args with str:gsub("%^%[", "\\%^\\%["):gsub(":", "\\:"). does that work? |
| 19:37 |
Krock |
at least that works for [combine |
| 19:37 |
appguru |
The docs say escaping doesn't apply to inventorycube - it applies to combine though. I'm currently trying to write a compliant parser in Lua. |
| 19:37 |
Krock |
nvm then I guess |
| 19:38 |
appguru |
We really need a better syntax |
| 19:38 |
appguru |
What would you recommend in terms of backwards compatibility? |
| 19:39 |
Krock |
to not change anything |
| 19:39 |
appguru |
A clean format would greatly simplify the parser. |
| 20:46 |
|
erlehmann joined #minetest-dev |
| 20:47 |
|
Taoki joined #minetest-dev |
| 20:52 |
erlehmann |
hey everyone, i think i made rebuilds much faster, just use this commit hook which sets the file time to the last commit that the file had: https://mister-muffin.de/p/SeoO |
| 20:53 |
erlehmann |
josiah_wi sfan5 take note :) |
| 21:11 |
|
Extex joined #minetest-dev |
| 21:54 |
fluxionary |
anyone have any insight on the following stacktrace? https://pastebin.com/34AT4z3B |
| 21:54 |
fluxionary |
segfault on trying to step an SAO |
| 21:56 |
sfan5 |
I've spoken to Alias about it |
| 21:56 |
sfan5 |
I believe the crash isn't the true cause but rather just the server crashing while trying to shut down and before it gets to print the error message |
| 21:57 |
Alias |
We can use this channel for common exchange. If you want me to add some more logging into the code and rebuild, I can do that. |
| 21:57 |
fluxionary |
ah, that makes some sense |
| 21:58 |
fluxionary |
because it's still trying to process things while in the middle of an emergency shutdown |
| 22:00 |
Alias |
And that may also be the cause why I only capture "different errors" in the logs. |
| 22:01 |
erlehmann |
regarding threads, i have built minetest with -fsenitize=thread before and it is full of data races. a regular shutdown triggers none of them though. |
| 22:02 |
hendursaga |
Anyone aware of a mod that uses cqueues or otherwise has non-blocking IO? |
| 22:02 |
erlehmann |
but i assume the threads are not properly managed from that |
| 22:03 |
erlehmann |
i'll try to crash it in a known way (the nether portal stack overflow trick) and see what it says then |
| 22:04 |
erlehmann |
explanation: basically any game action that causes enough recursion can crash minetest, for example, destroying a huge portal in a mod that handles destruction in a recursive manner |
| 22:04 |
Alias |
If you have guesses, we can try them out on a testserver, on the same machine with the same set of mods. |
| 22:05 |
sfan5 |
Alias: in your gdb command line swap 'bt full' for 'thread apply all bt full' and apply this patch http://sprunge.us/v4waNf |
| 22:05 |
erlehmann |
nice |
| 22:05 |
sfan5 |
will hopefully fix *or* delay the crash so that the real error is printed first |
| 22:06 |
Alias |
Wilco. (Now I need to quicklearn how to apply patch files. I weaseled myself around this part of git until now) |
| 22:10 |
erlehmann |
Alias, “man git apply” |
| 22:11 |
sfan5 |
patch -p1 also works |
| 22:12 |
Alias |
I added the patch to both test and live server |
| 22:13 |
Pexin |
note: if using "patch" you may need to remove .orig files for build to not choke. at least the language stuff |
| 22:14 |
Pexin |
(if the patch touches any language / translation files) |
| 22:26 |
Alias |
The testserver is up now, anthing specific I could do to reproduce the error? |
| 22:28 |
sfan5 |
dunno, maybe spawn some yl_nether_mobs:wither and it'll break |
| 22:46 |
erlehmann |
i am sorry, i was mistaken: there seems to be indeed some kind of thread weirdness if you manage to cause a c stack overflow from the lua side, but it that point you are already overflowing the stack, so basically anything can happen. |
| 22:46 |
erlehmann |
i think it has nothing to do with the error Alias experiences |
| 22:47 |
erlehmann |
it seems to be possible to crash minetest into a state where if you start it again it immediately loads a world instead of the menu, anyone have any info on that? |
| 22:48 |
sfan5 |
that's just you holding return for too long |
| 22:48 |
erlehmann |
ohhh |
| 22:49 |
erlehmann |
i feel p dumb, thx |
| 23:55 |
|
AliasAlreadyTake joined #minetest-dev |
