Time Nick Message 00:04 erlehmann ok so it seems that test 52 (polygonOffset) breaks with the XML stuff removed. this makes no sense to me. maybe i should just stop. 00:05 erlehmann oh no, it is worse 00:08 erlehmann sfan5, so it turns out that SURPRISE the XML removal broke tests that have … SURPRISE the test data specified as XML 00:08 erlehmann like which textures to use 00:09 erlehmann i may appear very rude, but this is just frustrating. first break the tests, then remove them bc no one can get them to run? 00:10 erlehmann what should be done in such a case? i mean the game obv does not use xml anymore (since it was removed) 00:10 erlehmann but i bet some fun hater will now say just to rewrite the tests to not use xml or something 00:11 MTDiscord Yes, because that's what should probably be done 00:11 erlehmann well i think that ppl who break stuff should fix it 00:11 MTDiscord If you're not testing the XML, and you don't need it for anything else... 00:12 erlehmann well it broke the pixel fonts too without intending too – but when i complained about that, they were declared unsupported, bc minetest had been in an on-off-relationship with working pixel font for years (which i understand) 00:13 erlehmann what i am hinting at is that gutting the entire irrlicht thing *in that reckless way* is not making anything easier except for producing bugs probably 00:13 MTDiscord Anyhow, the ideal would probably be to get the person who broke things (hecks I suppose?) to fix 'em. But this isn't a corporate environment, we can't really mandate that specific people do specific things 00:14 erlehmann Hugues Ross it is not the first thing hecks (or anyone) broke though, letting this kind of thing slide is just policy it seems 00:15 MTDiscord Sure, I've broken stuff myself so I'm well aware 00:15 erlehmann i am just pointing out it is a bad one. i can't change it. 00:15 erlehmann yeah so how did you find out? 00:15 MTDiscord Folks discovering issues after the fact, generally 00:15 erlehmann i also think that “i don't know how this thing is used, so it is probably useless” as a maxim has introduced too many bugs 00:16 erlehmann see for example the ticket about not teleporting when you are attached 00:17 erlehmann where apparently no one even thought that this could have use (and there is no real alternative, as games/mods rely on it) 00:17 MTDiscord I haven't kept up with the github for a while so I can't common on how common it is, but yeah I'd agree that it's bad practice to pull features if you genuinely don't know how they're used 00:17 erlehmann i think the problem is that ppl *think* they know 00:17 erlehmann and then don't do the research 00:17 erlehmann or don't benchmark 00:18 erlehmann we had an example a few hours ago, when a bunch of ppl including me were surprised that webp saves so many bytes with small images 00:18 erlehmann this is ofc not a problem 00:19 erlehmann but claiming “no one uses TGA/XML/whatever” has made stuff difficult 00:19 MTDiscord Seems possible. I get the impression sometimes that there's not enough crossover between PRs and the community, so we wind up with minimal modder input in places where it's needed until the update lands and becomes public knowledge that way 00:19 erlehmann it's a mindset thing 00:19 erlehmann look it is not about lack of input 00:20 MTDiscord On my end, it has been almost exclusively a lack of input. I can't speak to your circumstances, but that's what I've seen 00:20 erlehmann for example, to figure out if anyone besides mineclone2 and mineclone5 uses TGA images, someone scanned the entire contentdb with a script for filetypes 00:20 erlehmann but the person who removed TGA did not do it 00:20 erlehmann even though it is a trivial task to verify 00:21 MTDiscord Did they do it before or after the TGA removal? 00:21 erlehmann after ppl complained 00:21 MTDiscord If it was after, it speaks to my point 00:22 MTDiscord People notice something's wrong after it's too late, because devs aren't communicating enough to the people relying on the systems they touch. So they find out when the system changes in a way that causes them problems, but it's now difficult to change. 00:22 erlehmann when ppl complained before, they were not taken seriously enough (also mcl2 devs have a habit of claiming mt engine devs are idiots, but they don't do it on github obv – could be that they did not have the motivation to actually spell it out after not being taken seriously) 00:22 erlehmann well this is why i advocate tests and stuff 00:23 erlehmann and i think josiah_wi is doing great work 00:23 erlehmann but if the next person who breaks a test can just delete it, nothing is gained 00:24 erlehmann can i still compile minetest with standard irrlicht? 00:24 erlehmann i think it makes more sense to backport the improvements there 00:24 erlehmann and ignore the removals 00:25 MTDiscord I don't think it's possible anymore, unless that was changed 00:25 erlehmann why exactly? 00:25 MTDiscord Probably because they wanted the freedom to break API compat 00:26 erlehmann are you aware of any breakage that is not straight up removal of features? 00:26 MTDiscord I'm not involved in the process, so I wouldn't know either way 00:26 erlehmann like i see improvements in irrlichtmt ofc 00:26 MTDiscord I'm just a random guy on the internet 00:26 erlehmann but for a lack of tests i can not say what breaks lol 00:26 erlehmann (currently testing more) 00:27 MTDiscord given that the goal is to nuke irrlicht, keeping the ability to compile with mainstream irrlicht seems like a joke 00:28 erlehmann Jonathon the problem i have is that regardless of if the process of irrlichtmt development is guided by “i don't know about breakage” or “i don't care about breakage” it's a huge mess 00:29 MTDiscord erlehmann_rants = nil would be nice 00:29 erlehmann again, please do *mandatory* code reviews on irrlichtmt 00:30 erlehmann Jonathon i am only ranting because i am trying to improve the thing and finding it sucks, but i get it 00:30 erlehmann i won't convince anyone, i can go back to filing issues ig 00:31 MTDiscord >ranting >improving seems oxymoronic simple quote for you. "talk is cheap, show me the code" 00:31 erlehmann i'll suggest to my coworkers to try to find bugs in minetest at our next team event, maybe that brings some fuzzing action to the table 00:31 erlehmann Jonathon, code's compiling 00:31 erlehmann i use the time to rant actually lol 00:31 MTDiscord Hey, this issue I have just got dirtier. 00:31 erlehmann ok compile done 00:31 erlehmann brb 00:32 MTDiscord The undefined symbol is in libIrrlichtMt.a and it should be linked with the -ldl flag. I added this to the tests, but it still doesn't link. 00:32 MTDiscord I'm going to try building libIrrlichtMt.so as a dynamic library, to see whether it changes anything. 00:33 erlehmann sfan5 i have confirmed now that the existing tests were broken by your removal of XML. that is probably why hecks could not get them to run, they were already broken at that point. 00:34 erlehmann i still think straight up deleting them was a bad idea, but since the tests did not compile anyway, hecks probably assumed they were broken already 00:34 erlehmann a bit of git bisect could have changed that impression, but as i said, the problem is not verifying assumptions 00:34 erlehmann josiah_wi so what are you going to do with tests that break bc their config is in xml, rewrite them? 00:36 MTDiscord I'm going to start by disabling all the tests, and choosing which tests are worth fixing and pushing. By not re-adding all the tests, the commit size will be reduced too. 00:37 MTDiscord I've confirmed that building IrrlichtMt as a dynamic library fixes the missing dclose reference. I haven't the slightest clue why haha. 00:37 MTDiscord dlclose* 00:37 MTDiscord It's going to need to work as a static library, so I have to fix this. 00:37 erlehmann well dlopen and dlclose are for dynamic libs? 00:38 MTDiscord I'm not familiar with that. 00:39 MTDiscord All I know is I'm getting ../lib/Linux/libIrrlichtMt.a(CGLXManager.cpp.o): undefined reference to symbol 'dlclose@@GLIBC_2.2.5' 00:40 erlehmann josiah_wi do “man 3 dlopen” 00:40 erlehmann and read the man page 00:40 erlehmann it will probably help a bit 00:41 erlehmann no idea why it complains about close, but not open lol 00:41 erlehmann josiah_wi the man page is long, if you get the gist, just scroll down to the example code 00:41 erlehmann it make it pretty clear i think 00:42 MTDiscord I'm not sure that will help me understand how to fix this. 00:42 MTDiscord I haven't written any code. libIrrlichtMt.a is very clearly not a dynamic library. And linking to -ldl doesn't help either (I got that from the man page) 00:43 erlehmann can you point at the line that is bugged? 00:43 erlehmann maybe i can help, i have written code that used dlopen(3) sometimes (PKCS#11 library interfacing) 00:43 erlehmann but i am not good with computer and C/C++ so i prob can't figure it out either 00:44 erlehmann just if you want a second person to try 00:45 MTDiscord It's probably a misconfiguration in the CMake build. At least that's my best guess. 00:46 MTDiscord I'm going to compile Minetest to make sure Minetest still works hahaha. 00:46 erlehmann can't help you with that, but did you try building the makefile and observing what happens? 00:46 erlehmann i mean the tests dir had a makefile 00:47 MTDiscord I did not try building the Makefile, but I read it to the best of my ability. 00:47 erlehmann if you build using it, it shows the commands 00:47 MTDiscord I have CMake printing out the commands, too. 00:47 erlehmann yeah so you can compare then 00:48 erlehmann bc the makefile works, right up until the commit that removes XML support 00:48 erlehmann then it can't compile obv 00:48 erlehmann so then you diff the cmake output and the makefile? 00:48 erlehmann i mean the make output 00:48 erlehmann sorry sleepy 00:48 MTDiscord It won't work for me because it builds every .cpp file, and I need to disable pretty much everything. 00:49 MTDiscord I can probably list just the one .cpp file I want, though. 00:49 MTDiscord one or two* 00:49 MTDiscord That reminds me. I disabled a file I need by accident, oops. 00:49 erlehmann welcome to the unintentionally disabling vital code club ;) 00:50 erlehmann i'd lie if i say it hasn't happened to me in the last 24 hours 00:51 MTDiscord Yep, I commented out the main.cpp in the tests folder. Talk about vital. 00:52 MTDiscord Anyway, that's unrelated to the issue at hand. I'll finish building Minetest, and if Minetest works then I probably did something weird. 00:52 MTDiscord If Minetest doesn't work we have a massive issue we gotta solve ASAP lol. 00:58 erlehmann josiah_wi i conclude, if you did not do something weird, we have a massive issue 00:59 MTDiscord We shall see. IrrlichtMt has worked with Minetest up until now, so I don't see why it would break. But I also don't understand why it would work with Minetest and not with my unit test executable. 01:02 MTDiscord > Are you really sure you want to statically link a program which is calling the dynamic linker at runtime thru dlopen?? Statically linking such programs is generally a bad idea. 01:02 erlehmann ok i am donee for today. i have spent too much time testing this thing. 01:02 MTDiscord A stack overflow post says this. This would seem to apply to us. 01:02 erlehmann josiah_wi i have a gift 01:03 MTDiscord A gift? 01:03 MTDiscord Ok phew, Minetest links. 01:05 erlehmann josiah_wi https://mister-muffin.de/p/MCA0.txt 01:07 erlehmann josiah_wi this is the test log i made from commit accf9278 … 01:07 erlehmann josiah_wi note that a lot of stuff is already broken, but those was the last time tests actually worked 01:07 MTDiscord Nice. 01:07 erlehmann like just before the xml removal 01:08 erlehmann having a single binary for all the tests is bad, as you said 01:08 erlehmann like any undefined behaviour in one test will taint all of them 01:08 erlehmann it is not localized actually 01:08 erlehmann ppl may think that 01:09 erlehmann but if a program is undefined in one location, it is likely undefined in every location 01:09 erlehmann also it is stupid to not be able to run all tests that way 01:10 erlehmann so yeah, originally i wanted to figure out what the xml removal broke by diffing the test output 01:10 erlehmann but as i said, it broke the tests itself 01:10 erlehmann should i file an issue? 01:11 MTDiscord I don't see the point in doing so. 01:11 erlehmann yeah 01:12 MTDiscord I suggest we revive the XML test and then if the test fails we investigate. 01:12 erlehmann you can't compile it actually 01:12 erlehmann as it stores the test configuration (which textures to load i think) in xml 01:12 MTDiscord Ok, so this keeps getting weirder. Apparently I need to link -ldl to IrrlichtMt itself, but then why does Minetest work. 01:12 erlehmann and reads it at runtime 01:12 MTDiscord This all makes 0 sense. 01:13 erlehmann software was a mistake. we should have stayed in the ocean. 01:13 erlehmann josiah_wi, ok let me give you the error brb 01:14 MTDiscord Whelp, linking -ldl to libIrrlichtMt.a fixed the issue. I'm completely stumped. 01:18 erlehmann i have no idea what is happening 01:19 MTDiscord Got those XML errors. 01:20 erlehmann good, then you see why i am tired of this crap 01:21 erlehmann like just typing "make" in the test directory would have caught it 01:21 erlehmann (i guess) 01:22 MTDiscord Sweet, test is running. 01:22 erlehmann nice show 01:23 MTDiscord Turns out the .png test textures were axed. 01:23 erlehmann well 01:23 erlehmann josiah_wi what do you think of the workaround of just taking the last known-good state of irrlichtmt and then just cherry-picking all the improvements, but leaving out the “let's delete everything that is not used (oh it was actually used, well that's not my problem)” commits? 01:24 erlehmann i think it would save work actually 01:24 erlehmann prob can even be done automatically 01:24 MTDiscord It might be a better solution if everyone agreed that was desirable, but I don't think that's something that will reasonably be merged. 01:25 MTDiscord I have the test passing. 01:25 erlehmann i would not want it to be merged 01:25 erlehmann the history of the repo is poisoned by the huge “remove everything” commit 01:25 MTDiscord If the goal is to contribute something useful back to the Minetest community, then we definitely should take their wishes into consideration. 01:25 MTDiscord Otherwise why are we here. 01:26 erlehmann if you want to look what “the community” thinks of feature removals, you won't find it in this channel 01:26 erlehmann maybe on the forums 01:27 erlehmann but it does not matter, i think the whole dev process has become so hard through the “throw away stuff” thing 01:27 erlehmann i have been testing stuff for hours just to figure out what worked when 01:27 MTDiscord Look, if nobody uses our changes, why is it worth making them? 01:28 erlehmann josiah_wi honestly i just want a working minetest version on my hardware. 01:28 erlehmann cora has made her own cheat client, mostly for herself 01:28 erlehmann i'd like to go back to mineclonia development and not care about ppl ripping out format or hardware support 01:29 erlehmann bc what is my time worth if in 2 months time someone declares that tough shit, none of my laptops is good enough to run this thing? 01:30 erlehmann and honestly, i am not interested in fighting hecks in a court of public opinion or something. the damage was done, it is easier to fork it and make a better irrlichtmt than fighting against a feature removal policy. 01:30 erlehmann if minetest still links against it, no one has to support it even 01:31 erlehmann and again, i am opposed to *how* this was/is done. removing big chunks, then everything that adds it back in has to be justified. 01:31 erlehmann and constantly breaking stuff that used to work, then declaring it unsupported. 01:31 erlehmann i mean, i am overdoing it a bit here, but on purpose, this stuff is just too hard to even get to run. 01:32 erlehmann i'll probably just go back to creating crash tests 01:32 erlehmann josiah_wi btw one of the first things i would advocate is adding my crash examples to the unit tests as regression tests 01:32 MTDiscord Well, I intend to compromise and try to actually get unit tests merged back into the official repo. 01:33 erlehmann yeah that is good 01:33 erlehmann you probably have more energy to spend 01:33 MTDiscord I have ctest working. Just 1 test, but I'm glad haha. 01:33 erlehmann it is nice yes 01:33 erlehmann i wish i had approval rights, then i would review it 01:33 erlehmann but alas 01:34 erlehmann but please show 01:34 erlehmann i want to see 01:34 MTDiscord Ok, I have to decide what to commit now, hehe. 01:35 erlehmann git add -p 01:35 erlehmann you can also make several commits 01:35 erlehmann to make clear how you did it 01:35 erlehmann please do it now, i can test it 01:35 erlehmann so you got all the things 01:36 erlehmann but i can't possibly help you get it merged 01:37 MTDiscord Yeah, I'm taking my time to make good commits here. Or trying to, anyway. 01:37 erlehmann so which is your one test anyway? 01:37 erlehmann test 1, 'testIrrArray'? 01:40 MTDiscord disambiguateTextures.cpp There's a comment saying it's supposed to be run first. 01:42 erlehmann btw, for future reference, here is the compile error log for the commit that broke the tests https://mister-muffin.de/p/sy7-.txt 01:44 erlehmann now why would drawPixel.cpp or 2dmaterial.cpp need XML? i don't care, but it should not be broken 01:52 MTDiscord erlehmann, before I commit, I'm considering trying to fix the missing log file. 01:53 erlehmann josiah_wi wdym missing log file? 01:57 MTDiscord "sh: 1: test.log: not found" 01:57 MTDiscord tests.log* 01:58 erlehmann lol ok no idea 01:58 erlehmann sorry 01:59 erlehmann josiah_wi i am happy that you committed to this task, it will make development less error prone 02:04 MTDiscord Alright, it seems like it's a working directory issue or something probably. 02:04 MTDiscord I'm just going to push what I have. 02:04 erlehmann if you include documentation, i can look at it 02:04 erlehmann like what commands to issue 02:05 MTDiscord https://github.com/JosiahWI/irrlicht on the feat/unit_tests branch 02:06 MTDiscord I didn't include documentation because I'm still working on it, and it's relatively straightforward. Just build with CMake like normal and run the tests binary in bin/ 02:06 MTDiscord tests are enabled by default but there's an option to toggle them. 02:06 MTDiscord You should try removing the texture in media, so that the test fails, because it happens the test fails with a segfault which I find curious. 02:10 erlehmann josiah_wi where can i get the PGP pubkey for your signaturee 02:10 erlehmann it is not on keys.openpgp.org 02:10 MTDiscord Yeah, It's only on GitHub. I made it just for fun. 02:11 MTDiscord If you really want my pubkey I can send it to you, I guess? ... 02:11 erlehmann look i need the public key to verify it 02:12 erlehmann wait, before you publish it 02:12 erlehmann add an expiry date 02:12 erlehmann if you don't already have 02:12 MTDiscord I'm not going to publish my GPG key at this time. 02:12 erlehmann therefore, if you lose the private key, it will resolve itself 02:12 erlehmann strange, you sign commits, but no one should verify them? 02:13 MTDiscord It's so that GitHub will do the verified checkmark. 02:13 MTDiscord I just wanted to have the checkmark because it was cool? ok? XD 02:14 MTDiscord Maybe I shouldn't have made a key so hastily haha. 02:14 erlehmann uh 02:14 erlehmann ok do you understand that this will literally not benefit you? 02:15 MTDiscord It will only benefit me as far as that GitHub will verify that I am the person who made the commits it says I made. 02:15 erlehmann without the key, my shell for example will show a big red warning message (bc i configured it so) 02:15 erlehmann yeah but github can lie 02:15 erlehmann it is about git 02:15 MTDiscord Oh boy. 02:15 MTDiscord I better publish my key. I don't have time to try to figure that out tonight though. 02:15 erlehmann yeah i get it 02:16 erlehmann i can help you with it lateer 02:16 erlehmann oh btw, did you know that your GPG agent can also pretend to be an SSH agent? that way you need no separate ssh keys 02:16 erlehmann just an authentication subkey 02:17 MTDiscord Interesting, I did not know that. 02:17 erlehmann aaaand … with git-remote-gcrypt you can make git repos – even on github, that only people with specified public keys (by default: only your own) can read 02:17 erlehmann you just add gcrypt:: in front of the remote and it does its magic 02:18 erlehmann github will then ofc not be able to read your repo, but i use it for config files for example 02:18 erlehmann that need to be secret 02:18 MTDiscord Now that sounds like a really handy trick. 02:18 MTDiscord Any luck building the tests? 02:19 erlehmann also the pass(1) password manager uses gpg encrypted text files and commits them to git https://www.passwordstore.org/ 02:19 erlehmann give me a min 02:24 erlehmann josiah_wi it complains that tests directory does not contain a CMakeLists.txt file 02:24 MTDiscord Ack I forgot to commit it. 02:24 erlehmann you probably haven't tested your test build on another computer :D 02:24 erlehmann imo you can fold it in using git commit --amend 02:25 erlehmann i can do git reset --hard 02:25 erlehmann but do what you want 02:25 erlehmann tell me when you are ready 02:25 MTDiscord Yeah, I was going to ask first before doing an amend to make sure it wouldn't mess up your repo. 02:25 erlehmann oh, git will reject it 02:25 erlehmann but many ppl don't know how to get out of it 02:25 erlehmann but since i am *probably* the only one checking this out, i think it is ok 02:26 MTDiscord Pushed. I'm afraid you'll probably have to suffer the GPG warning again. :/ 02:26 erlehmann i once had to help my bf to get out of a force push situation like this … 02:26 erlehmann no problem, it just hangs my shell each time for a few seconds since it assumes it can find the key online maybe, then gives up 02:26 erlehmann honestly i did not know it could automatically search for keys online 02:27 erlehmann i thought i had to configure that 02:27 erlehmann but my own fault 02:27 erlehmann after all, i do use git signatures mostly for my own repos 02:27 erlehmann i have been very careful about this after a server with repos of mine got compromised 02:28 erlehmann and also i live in a country (germany) where it is legal for the state to hack ppl 02:28 erlehmann the easiest way is to actually compromise your test suite ;) 02:28 erlehmann (i won't run it on the computer i am typing on ofc) 02:29 MTDiscord Ouch. Security is an important thing to understand, and a lot of computer users including myself aren't too familiar with it I'm afraid. 02:29 erlehmann well i have made enough mistakes so far 02:29 erlehmann but the setup with git signatures is pretty cool 02:30 erlehmann basically, my shell does: git show '--color=always' --no-patch '--pretty=%h %s' HEAD --show-signature 02:30 MTDiscord Anyway, this is the dev channel so we should probably keep it related to Minetest development concerns. ? 02:30 erlehmann ah right 02:30 erlehmann ok lets get to the tests 02:34 erlehmann josiah_wi can not find source file disambiguateTextures.cpp 02:35 MTDiscord I should've written a list of every file I needed instead of trying to remember them off the top of my head. 02:35 erlehmann josiah_wi i suggest you git clone it into a temp folder (by using git clone $PATH) and then try to build there 02:35 erlehmann you will find outwhat you missed 02:35 erlehmann ping me when you think you have them all? 02:36 erlehmann try "git status" to figure out what files it could have been 02:36 erlehmann it will show all unversioned files 02:40 MTDiscord OK, that should be everything now. I got all the source files, all the CMakeLists.txt files, and the media file. 02:41 MTDiscord Yep, can confirm it compiles now after stashing all unrelated stuff. 02:51 erlehmann ok 02:51 erlehmann cool 02:51 MTDiscord Works? 02:52 erlehmann why the working directory weirdness 02:53 MTDiscord The tests are required to be run from the working directory. This is the fault of the original test author's, not mine. xD 02:53 MTDiscord ctest is set up to run them from the working directory automatically, but it doesn't print the logging information and I haven't figured out how to fix that. 02:53 erlehmann also you have windows line endings 02:54 erlehmann like … other irrlicht code 02:54 erlehmann ewwwww 02:54 erlehmann ._. 02:54 MTDiscord Yeah, exactly. CMake files should be LF though. 02:54 erlehmann yeah they are 02:54 erlehmann i bet irrlicht authors mix tabs and spaces 02:54 erlehmann or meat and cheese 02:55 MTDiscord There was mixed tab and spaces indentation in the main.cpp file in tests as I said earlier. 02:55 erlehmann different line endings in same project is absolutely haram 02:55 MTDiscord I fixed it though haha. 02:55 erlehmann yeah lol 02:58 erlehmann josiah_wi how to run tests 02:58 erlehmann you really should add documentation 02:59 erlehmann the original tests had to be run in the tests dir 02:59 erlehmann regardless of where the binary was 02:59 erlehmann this does not work 03:00 MTDiscord Right, it's still that way. 03:00 MTDiscord Unless you run ctest, but it leaves out logging. 03:00 MTDiscord To run ctest just ctest from the build directory. 03:01 MTDiscord Ah, ctest --verbose gets the logging back in. 03:01 MTDiscord Try that. 03:01 erlehmann try running the test binary 03:01 erlehmann it does not find some stuff 03:01 erlehmann i think you messed up 03:02 MTDiscord This must be related to the missing log file. 03:02 MTDiscord Not sure. 03:02 erlehmann i am on commit d0a… 03:02 erlehmann is that ok? 03:03 MTDiscord yes 03:03 erlehmann so here is the output 03:04 MTDiscord Ok, I used the wrong media directory. My bad. 03:05 erlehmann https://mister-muffin.de/p/saCC.txt 03:07 MTDiscord I got this error locally too so I'll get it fixed. 03:08 erlehmann ok cool 03:08 erlehmann once you have this working, i suggest to annoy ppl to put it in the CI 03:09 erlehmann and then advocate that failing tests should prevent merges 03:09 erlehmann and then bully everyone who has a shirt with “move fast and break things” until they stop doing that xD 03:10 erlehmann josiah_wi can ctest spit out coverage info? 03:12 MTDiscord I don't know, I haven't used ctest before. 03:15 erlehmann ok it seems it can 03:15 erlehmann i suggest to add that 03:15 erlehmann it helps to see if your tests are good 03:20 MTDiscord I've pushed a fix to the test. 03:20 erlehmann ok 03:20 MTDiscord I'll look at how to add coverage information. 03:22 erlehmann look i think this is not working 03:22 erlehmann like still not 03:22 erlehmann if it work locally, something is different than here 03:22 erlehmann i'll be off 03:23 erlehmann but i hope i could help a bit 03:23 erlehmann need for sleep 03:38 MTDiscord Oof, that's a lot of requests, move fast and break things is good in the dev stage, and tests will never catch all our bugs, and if we write too many tests, we'll end up having to maintain or update more tests when PRs making knowing changes that break tests. I.e. a test that is useful will likely break every 5-10th PR because our codebase is still quite.... Young? Naive? Weak? You get the point 03:39 MTDiscord I'm all for some testing, I'm not for reject everything that fails the tests 03:39 MTDiscord That is arguably why test cases are useful. 03:39 MTDiscord Because they require updating for certain PR's? 03:39 MTDiscord If test cases are good, a failing test should indicate that something needs to be fixed, or the test needs to be updated. 03:40 MTDiscord Gotcha, we'll see but I've seen some code bases dragged down by too many tests 03:40 MTDiscord I.e. one change = 12 test updates 03:40 erlehmann exe_virus how can you have *too many* tests? 03:40 MTDiscord That sort of thing 03:41 erlehmann well then you prob broke 12 systems that way 03:41 erlehmann and it should be very clear that that is a bad thing 03:41 MTDiscord Yes, but not in a breaking way, just a breaking way for the tests 03:41 erlehmann good code rarely needs test updates 03:41 MTDiscord I.e. tests are often fragile 03:41 erlehmann wdym fragile? 03:42 MTDiscord Change a directory and suddenly a bunch of test cases fail 03:42 erlehmann well if the requirement is that it only works in a specific directory, then it is right. otherwise, the test might not test what you want to test. 03:42 MTDiscord Right, it's hard to write good code, and it's ten times as hard to write good tests 03:43 erlehmann “move fast and break things is good in the dev stage” i have literally never seen this work except if someone wanted to map out the solution space. deleting a bunch of things without regard for what is broken by it is not included in “mapping out the solution space”. 03:43 MTDiscord Having 20 mediocre tests can hurt a code base more than it helps 03:43 erlehmann i literally will not believe you that your code is good without a test case 03:44 MTDiscord Very different than what you have been griping about, that was carelessness in a way 03:44 erlehmann > a test that is useful will likely break every 5-10th PR because our codebase is still quite.... Young? Naive? Weak? 03:44 erlehmann no if the test breaks either the dev is reckless or the test is too broad 03:44 MTDiscord I am more than willing to be careful about the quality of the tests. 03:44 erlehmann i have seen „dev is reckless“ a lot 03:44 erlehmann me too 03:44 MTDiscord How many tests have you written for C++ code bases? 03:45 erlehmann i can definitely tell you that minetest does not have too many tests 03:45 MTDiscord Me? None, that's why I'm not intending to write any. 03:45 erlehmann come on, some bugs are only noticed because months later some stuff stops working 03:45 erlehmann on some persons machine 03:45 MTDiscord Haha fair Josiah, I'm more curious about where erlehmanns zealousness comes from 03:46 erlehmann i am the guy at work who is asked when you have to write fire-and-forget software (i.e. it needs to be as good as it can get), but i do not write it in C++ 03:46 MTDiscord Is it just that "tests would defend against backwards compat breaks" is the goal here, or does it come from experience with great code bases with lots of good tests? 03:46 MTDiscord What language? I think you're IT so I assume scripts and setup files? 03:46 erlehmann oh yeah i actually have written a backend service that had basically no bugs 03:47 MTDiscord Doubtful, no bugs yet* should always be the response haha 03:47 erlehmann i usually use python, but i have also written elixir and typescript 03:47 MTDiscord >no bugs, except for the spider nest that setup shop to eat the other bugs 03:48 erlehmann no i literally exercize all code paths in the testing. the app has only several thousand paths through it, so it ought to catch them all. 03:48 erlehmann it also takes half an hour 03:48 MTDiscord Okay cool, do you have some examples of good tests you have written or seen in the wild that you recommend that we utilize to defend minetest against bad PRs? Testing all paths works if you don't have to support arbitrary user input 03:48 erlehmann and also at runtime i check every input and every output of each function that is called externally against a context-free or regular grammar 03:49 MTDiscord Gotcha, so you used test everything and restrict to sanitary input. Makes sense 03:49 erlehmann i.e. you hand me base64? it will first go to a function that determines if it is actually base64 encoded data, only then i try to decode. 03:49 erlehmann also a function bugs out if its output does not match what i intended 03:50 MTDiscord Give us some examples for minetest that could use that sort of testing so we can add them 03:50 erlehmann therefore during testing i called every function with every possible value, which took long 03:50 MTDiscord I think you are plenty zealous about it to have good suggestions for them 03:50 MTDiscord (don't make our CI run for 24 hours, that's expensive) 03:50 erlehmann ok, so you know where you put mapblock coords in to get mapblock seed? 03:50 erlehmann there is a limited number of u16 and floats, so you can test them all 03:51 MTDiscord erlehmann, it's fixed, finally. I realized at long last that a tests/empty/empty directory has to exist for it to work. 03:51 erlehmann wait i look for evidence so you do not think i am joking 03:51 MTDiscord Sure, what would you expect to see from that? Also exhaustive testing is expensive 03:51 erlehmann exe_virus https://randomascii.wordpress.com/2014/01/27/theres-only-four-billion-floatsso-test-them-all/ 03:51 MTDiscord Right, per function, in our codebase, per PR, that's... A lot 03:52 erlehmann basically i actually know that minetest is bad at boundary checking, so you only need to figure out interesting valuess 03:52 MTDiscord Off to bed. Goodnight. 03:52 MTDiscord Kk, that's much*** more reasonable 03:52 erlehmann exe_virus good, cheap, fast, choose maximum 2 03:52 erlehmann but in my experience, it is still faster to add exhaustive tests than to fix the bugs 03:52 erlehmann like if you have a schedule, you can deliver on schedule 03:53 MTDiscord I'm also going to bed, but for your specific example could you find those magic numbers? No need to write C++ but pseudo code would be helpful. I'm not great at writing tests so I'm trying to see how you would personally like to see it done with our codebase 03:53 erlehmann exe_virus, take the biome seed bug 03:54 erlehmann exe_virus wait, i give you a much better example. take any of the files i submitted that crashed minetest until sfan5 limited texture size. 03:54 MTDiscord Unfamiliar with 99.9% of our bugs, I'm new to the dev side of the engine sorry 03:54 erlehmann exe_virus making them test cases is guaranteed to help. they will never have to be modified. 03:54 MTDiscord Gotcha, so test extreme values 03:54 erlehmann and 100% prevent re-introducing the crash 03:54 erlehmann no, test *known* bugs 03:55 MTDiscord Ah 03:55 erlehmann you start with requiring a test for every regression 03:55 MTDiscord Okay so ad hoc, that's generally cheap 03:55 erlehmann that way you quickly get a bunch of interesting values 03:55 erlehmann and can extrapolate from that 03:55 erlehmann yeah 03:55 MTDiscord Though it does lead to the 20 cases needing updated for one small directory change thing sometimes 03:55 erlehmann and it is not hard, bc the person fixing the bug usually knows how to triggr it 03:55 MTDiscord We'll just have to be careful to keep the tests from being fragile 03:55 erlehmann well, the regression should *never* occur again 03:56 MTDiscord Sometimes it has to 03:56 MTDiscord That's the nature of coding 03:56 erlehmann so if it is fragile, the regression test is bullshit 03:56 erlehmann have an example? 03:56 MTDiscord Formspecs 03:56 MTDiscord Version updates 03:56 MTDiscord New mechanics there 03:56 MTDiscord Also mapgen, being multithreaded 03:56 erlehmann if you do version updates, not changing the original test case ensures that you do them in a backwards compatible way 03:56 MTDiscord That changes tests 03:57 erlehmann uh mapgen is already not deterministic 03:57 MTDiscord Formspecs have an expected replacement that will invalidate a lot of the test cases 03:57 MTDiscord It could be deterministic, but that's a different consideration 03:57 erlehmann well, the thing is, your test cases are a stand-in for all the mods you break 03:57 erlehmann so if you break a bunch of formspec tests, you break a bunch of mods 03:57 erlehmann lets not do that 03:57 MTDiscord Right, but sometimes you need to break mods 03:57 MTDiscord And the test needs to be invalid 03:57 erlehmann sometimes, ideally you find a way to not do it 03:58 erlehmann for example, instead of changing a function with buggy behaviour that ppl rely on, introduce a new one 03:58 MTDiscord Sometimes possible, sometimes not. Good example is deprecation 03:58 MTDiscord We deprecate a lot of things 03:58 MTDiscord Keeping around buggy code make maintainability a nightmare 03:58 erlehmann yeah and honestly, you could have a test case there that helps 03:59 erlehmann for example, scan the contentdb for any occurence of the deprecated function 03:59 erlehmann that is a simple to write test 03:59 erlehmann and basically guarantees that you'll not break too much accidentally 03:59 MTDiscord That's expensive, and fragile due to dependency and not knowing how each mod works 04:00 MTDiscord We can't test those mods, only startup a new world and if nothing crashes then... Success? Bad test 04:00 MTDiscord Mediocre test** 04:00 erlehmann no, you can make an AST from the lua and figure out if the function is called 04:00 erlehmann look, if you want to deprecate stuff, you have to be *really* sure it is actually obsolete once you remove it 04:01 erlehmann everything else is a “fuck you” to users 04:01 MTDiscord Maybe it really is obsolete but someone likes the format because they hate change? 04:01 erlehmann as if 04:01 erlehmann come on 04:01 MTDiscord For example, .b3d is bad 04:01 MTDiscord It's just a plain all around mediocre formst 04:01 erlehmann it is used all over the place, tough shit 04:01 MTDiscord If we supported a modern one like .gtlf or collada, we should deprecate it 04:02 erlehmann you can introduce better stuff, but as long as ppl use it, if you rip it out you sabotage their stuff 04:02 erlehmann collada support was actually ripped out lol, no xml 04:02 erlehmann i think 04:02 MTDiscord True, go talk to Google or other open source projects that sometime require that 04:02 MTDiscord Android updates occur, apps move on, your phone is obsolete 04:02 erlehmann no i mean in minetest. the xml support was removed. 04:03 MTDiscord Oh sure, it'll be back when we have a good use case 04:03 erlehmann exe_virus look at the web for that. chromium recently tried to remove alert() from javascript lol, it broke a bunch of big websites. 04:03 MTDiscord True, they've removed all sorts of things... Flash? 04:03 erlehmann basically, you have to support widely used stuff forever unless you can automate the transition or justify its exclusion 04:03 erlehmann flash was never part of the web 04:03 MTDiscord It was though 04:03 erlehmann and alert() is literally the first thing js coders leern lol 04:04 MTDiscord Everyone used it 04:04 erlehmann well it did not run on iphone 04:04 erlehmann or elinks 04:04 MTDiscord Which wasn't the web haha 04:04 erlehmann but yes removing flash was bad for users prob 04:04 MTDiscord It was 04:04 erlehmann now the flash games do not work 04:04 MTDiscord But it was good that it was removed 04:04 erlehmann yeah because it was insecure and not portable 04:04 MTDiscord (you can still run all of them) 04:04 erlehmann but b3d is not it 04:04 MTDiscord B3d is a source of crashes 04:05 MTDiscord Got lots of files that break it 04:05 erlehmann good test cases then 04:05 MTDiscord It's a fragile format 04:05 erlehmann not necessarily 04:05 erlehmann can it be validated against a context-free or regular or calc-regular grammar 04:05 erlehmann anything xml-based is context-sensitive btw 04:05 erlehmann because ids need to be unique 04:06 MTDiscord Why does context break or fix things? 04:06 erlehmann can't parse that context-free 04:06 erlehmann LANGSEC basically 04:06 MTDiscord So basically just throw out anything that can't be processed context free? 04:06 erlehmann basically everything that is too complex to not be valitated by a deterministic pushdown automaton is a security risk by design or something 04:07 erlehmann calc-regular stuff works, but yes. you basically can't validate a bunch of file formats, so you can *never* parse them provably correctly. 04:07 MTDiscord Sigh, we're an open source game engine, we make tradeoffs 04:07 erlehmann i actually work in IT security 04:07 erlehmann the only way to be sure is to specify your data formats 04:07 MTDiscord Right, you don't have that luxury 04:07 erlehmann no i mean, i do make tradeoffs 04:08 erlehmann work means that 04:08 erlehmann otherwise you end up with a stone without power 04:08 erlehmann the brick is 100% is unhackable 04:08 erlehmann but not functional 04:08 MTDiscord Lol 04:08 MTDiscord Great analogy 04:09 MTDiscord Alrighty I'll go to bed on that one, do keep up your enthusiasm, it'll probably help our codebase, just expect more pushback:) 04:09 MTDiscord (tests are good, just don't go crazy) 04:09 erlehmann exe_virus the thing with the context-free or regular or calc-regular stuff is that even IT students or math ppl do not “get” it on an intuitive level that it works 04:09 erlehmann ppl think it limits you much more than it actually does 04:10 erlehmann but for that you have to try 04:10 erlehmann it is similar to TDD (writing tests before the implementation), ppl usually need to try it to figure out that it saves time AND makes better APIs (since the test uses the API) 04:10 erlehmann they almost never believe it without trying it 04:11 erlehmann exe_virus honestly i would already be happier if each bugfix absolutely needd a regression test 04:11 erlehmann but i bet even that is too much for ppl 04:12 erlehmann > do keep up your enthusiasm, it'll probably help our codebase, just expect more pushback:) 04:12 erlehmann well, i am already complaining a lot, i think i won't do it for a while 04:12 erlehmann but i'll be back :P 04:12 MTDiscord :thumbs up: 04:12 erlehmann did you read the float article btw? 04:12 erlehmann it is really funny 04:13 MTDiscord Yeah we call that good code design (no article has to wait till lunch tomorrow) 04:13 erlehmann like they tested all floats with ceil, floor, and round 04:13 erlehmann and found that a bunch of libraries literally can't do math 04:14 erlehmann > _mm_ceil_ps2 claimed to handle all numbers in the range of 32-bit integers, which is already ignoring about 38% of floating-point numbers. Even in that limited range it had 872,415,233 errors – that’s a 33% failure rate over the 2,650,800,128 floats 04:15 erlehmann so if even rounding functions can benefit from a bit of testing … 04:16 erlehmann btw, minetest def has what looks like floating point bugs 04:16 erlehmann but lets discuss that some time later 04:53 erlehmann basically everything that is too complex to not be valitated by a deterministic pushdown automaton is a security risk by design or something 04:54 erlehmann this should have beenn 04:54 erlehmann basically everything that is too complex to be valitated by a deterministic pushdown automaton is a security risk by design or something 04:54 erlehmann sorry sleepy 08:24 sfan5 i am not saying it is malicious, but it is creating a lot of work 08:24 sfan5 that's a funny thing to say, you are creating that work yourself by wanting to get the tests we inherited from Irrlicht running 08:26 sfan5 and if you "just want a working minetest version on my hardware" you must have missed the possibility of compiling the latest commit of both Minetest and IrrlichtMt 08:51 sfan5 merging #11716 in 10m 08:51 ShadowBot https://github.com/minetest/minetest/issues/11716 -- Fix no_texture.png activation w/ simple leaves by Wuzzy2 08:56 sfan5 also sprunge.us/Yhhdif?diff at the same time 11:16 rubenwardy ./util/bump_release.sh fails when going from 5.4.1 -> 5.4.2 11:16 rubenwardy it tries to release 5.4.1 again 11:37 sfan5 do you really need to run that 12:09 rubenwardy yeah, probably easier to do it manually 14:00 MTDiscord pak and npk archives do not work in Irrlicht, did we remove support for them? 14:01 sfan5 yes 14:08 MTDiscord Did we also remove lzma compression? 14:09 sfan5 yes 14:17 MTDiscord What are your thoughts on readding some of the Irrlicht unit tests, sfan? 14:19 sfan5 if they work and are properly integrated they can be readded 14:32 MTDiscord There was a test commented out which fails. It looks like a memory error, because the filename it finds is gibberish bytes. 14:42 MTDiscord Maybe I logged it incorrectly. I'll have to investigate that later. 16:38 erlehmann sfan5, your argument can prove that i could be a hypocrite, but not that i am wrong 16:39 MTDiscord #11656 needs another approval 16:39 ShadowBot https://github.com/minetest/minetest/issues/11656 -- Add variable to use existing IrrlichtMt build by JosiahWI 16:41 MTDiscord erlehmann, do you want to debug (or rewrite) a failing test for archive creation/reading? 16:44 erlehmann josiah_wi what archive? do we need the feature? 16:44 erlehmann josiah_wi is it pk3 support? 16:48 MTDiscord The test is not commented and doesn't explain its purpose. It's called testMountFile, and it was commented out because it was running before another test that fails if archives are already mounted. It broke after I changed the ordering of the tests. 16:48 MTDiscord I've published my GPG key to keyring.debian.org. Thank you for letting me know I should do that. 16:48 erlehmann if the ordering of tests makes some of them fail, they are fragile 16:49 erlehmann can you somehow make the tests into separate binaries? 16:50 MTDiscord Right, that's a bad sign, especially since other developers are concerned about how fragile the tests are. Making separate binaries would be a bit of work because there's one main file that runs all of the tests. 16:52 erlehmann “other developers are concerned about how fragile the tests are” not more fragile than the rest of the code. compile minetest with asan or msan or ubsan or thread sanitizer and you see that it is full of holes. 16:52 erlehmann and irrlicht proper is even worse ig 16:52 MTDiscord I do not know what is considered a good test framework, so maybe someone who does could provide input about how to structure the tests well. 16:53 erlehmann well, i want them separate binaries so you can run them separately and compile with ubsan etc. 16:53 erlehmann as it stands now, one test failing too hard masks all later ones 16:54 erlehmann like, segfaulting 16:54 MTDiscord erlehmann, I have been given the green light to PR unit tests. I want to do a good job, and I want to compromise to work with developers (who also happen to be way more experienced than I am) instead of getting in an argument. I know you're interested in the test structure and maybe getting your regression tests in, so we should discuss that, but let's try to be polite to the other developers even where we disagree. 16:54 erlehmann to answer your question, i am not willing to make a test work that no one knows what its for 16:54 erlehmann but i am willing to help you with debugging why it does not work ;) 16:55 erlehmann compromising with developers is a given ofc if you want your PR merged 16:56 MTDiscord We should get the test structure right before making too many tests, I think. The developers who wrote the tests designed it so that you could turn off all the tests with an ifdef and run single tests for debugging instead. (See the comment in main.cpp) 16:56 erlehmann for me, the regression tests are definitely a case of “if the regression happens again, the other tests should still run” 16:56 erlehmann yeah that is kinda bad for CI though 16:56 MTDiscord Tests should be independent where possible 16:56 erlehmann this 16:56 erlehmann test order should not matter, or how many you run 16:56 MTDiscord Although I imagine the Minetest way of fixing a SEGFAULTing test case would be to outcomment it ;) 16:57 MTDiscord Luatic, should we put a main() function in every test file? I can do that. 16:57 MTDiscord > test order should not matter 16:57 erlehmann luatic no, to declare the code path unsupported :P 16:57 MTDiscord There is literally a comment explaining why one test should be run first to verify the working directory hahaha. 16:57 erlehmann josiah_wi that one is meant well, but mostly serves to explain that all other tests would fail anyway ig? 16:58 erlehmann it is not that the first test sets up something important for other tests, right? 16:58 MTDiscord Yeah, and it shouldn't be hard to run it first anyway. 16:58 erlehmann > a main() function in every test file? I can do that. 16:58 erlehmann ye 16:58 erlehmann yes 17:01 erlehmann > We should get the test structure right before making too many tests, I think. 17:01 erlehmann i agree 17:07 MTDiscord I've written a PNG decoder now and am testing it using PngSuite 17:12 erlehmann nice 17:13 erlehmann all the png test images should probably be in a test 17:13 erlehmann oh also think of parametrized subtests 17:13 erlehmann is it acceptable that no more png images are tested if one is failing? 17:14 erlehmann luatic did you test the lua png thingy against the builtin implementation? 17:34 MTDiscord no, not yet; but I compared implementations and am confident that mine is correct 17:38 erlehmann luatic to become truly confident in your code you must become the code 17:38 erlehmann t. bruce lee 17:39 erlehmann how did you compare without a test case, transliterate the code? 17:39 erlehmann also what is correct? surely you both make correct PNGs, but is it equivalent? 17:49 MTDiscord wipes sweat off brow 17:50 MTDiscord First time ever writing a function for C++ codebase of Irrlicht or Minetest haha. 17:50 MTDiscord I should probably get this reviewed before writing the test cases for it. 17:51 erlehmann which function? 17:51 erlehmann honestly, you should write the test case *first* 17:52 erlehmann then write the minimal amount of code to make it pass 17:52 erlehmann this may seem a bit weird, but if your function is then not perfect, you need more tests 17:52 MTDiscord The function to execute a test case. 17:52 erlehmann cycle ad nauseum 17:52 erlehmann ah ok lol 17:53 erlehmann well you need a single test that is an expected success and a single expected failure i guess 17:53 MTDiscord I've slaughtered the log file, I'm afraid. Not sure how important that is/was. 17:54 erlehmann most important is having stuff in separate files honestly 17:54 MTDiscord I also removed the parallelism and counting failed tests, because ctest does both of those itself. 17:54 MTDiscord Separate files is different than separate binaries. All the tests were in separate files to start with. 17:55 MTDiscord What I'm doing now is compiling a separate binary for each test case, which I'm a little queasy about, but it works so I won't complain. 17:57 erlehmann ah separate binaries yes 17:57 erlehmann that is what i mean 17:57 erlehmann what is your prob with separate binary for each test case? 17:58 MTDiscord Hmm, it looks like I'm expected to correct gamma ? 18:01 erlehmann luatic you are 18:01 MTDiscord erlehmann, I pushed the new test design. You can look at it if you're curious. 18:02 MTDiscord I'll add the other tests I had back in after I eat. 18:22 MTDiscord Oh my. I know you were joking erlehmann, but... your joke has come true. 18:23 MTDiscord The color test has spaces for indentation. 18:23 MTDiscord The other tests so far use tabs. 18:25 erlehmann absolutely haram 18:25 erlehmann add a linter to the test cases i guess 18:25 erlehmann if you feel strongly about it 18:26 erlehmann it reminds me of that tv scene where the guy breaks up with the girl bc one of them uses tabs and the other one uses spaces 18:26 erlehmann he falls down the stairs trying to leave 18:26 erlehmann she asks what happens 18:26 erlehmann he said he tried to take 8 stairs at the same time 18:33 MTDiscord Why are there empty lines at the end of some of the source files? If I accidentally remove those will I wreck some tool or system? 18:35 MTDiscord What series is that? It sounds fun. 18:38 MTDiscord there should be a single empty line at the end of every source file, and yes its so old tools dont break 18:39 MTDiscord it is also nice for qol when seeking to eof or concating multiple files 18:43 * Pexin tries to count the number of the word "test" in the past few hours of chatlog. Pexin fails. 18:44 MTDiscord just wget the files from irc.minetest.net, then grep + count 18:45 erlehmann Pexin testing is very important and, most importantly, FUN. 18:45 MTDiscord 18 tests passing. x) 18:45 erlehmann there are whole computer games about testing 18:45 MTDiscord 18 out of? 18:46 erlehmann like portal 18:46 Pexin you know how when you hear a word too much and it starts to sound weird 18:46 erlehmann GreenXenith originally irrlicht had like >70 tests or so, of which i was able to figure >50 out before it was botched 18:46 MTDiscord Just 18, green. I started by disabling all the tests, and I'm enabling them one by one. 18:46 MTDiscord Pexin: semantic satiation 18:47 cheapie Pexin: Starting at about the point the test spew started: 18:47 cheapie $ grep -i test test.log | wc -l 18:47 cheapie 145 18:47 erlehmann don't believe me, research yourself. josiah_wi will not add all of those anyway. 18:47 erlehmann “banana” is the best word for semantic satiation IMO. i start to writing banene. 18:47 MTDiscord Test 21 is the first test that does not pass, and it probably isn't worth trying to figure out why because the part that's broken was disabled originally anyway. 18:47 erlehmann bananana sha dubi dubi 18:48 erlehmann josiah_wi anything is worth figuring out 18:48 erlehmann show test 21 pls 18:48 erlehmann i'll look into it 18:48 MTDiscord One sec. I want to push the simple tests and then I'll readd the filesystem tests. 18:50 MTDiscord Also I just want to mention, watching it spit out 18 executables one by one is a little bit weird. I feel like with even 20-30 tests this could get frustrating. 18:53 Pexin when I was a C++ lab grader, our auto-reportgen script compiled 3 course sections of about 30 students each and ran test cases on each exe. sometimes a student left in an infinite loop and we'd come in the morning and be AAAAAAAAAAAA, kill process, do tests on remaining students! aaaaaaaaa! 18:53 Pexin on the department server 18:54 Pexin after the first or second time, we adjusted the script to timeout if test took too long :| 18:55 specing right, you didn't consider this as a problem when initially developing it ?! 18:55 Pexin never underestimate the negligence of first year students.. 18:56 MTDiscord Passing tests are peak satisfaction. 19:00 erlehmann > sometimes a student left in an infinite loop and we'd come in the morning and be AAAAAAAAAAAA, kill process, do tests on remaining students! aaaaaaaaa! 19:00 erlehmann LOL 19:00 erlehmann Pexin timeout(1) is a godsend! 19:00 erlehmann by which i mean, god's end ;) 19:00 erlehmann Also I just want to mention, watching it spit out 18 executables one by one is a little bit weird. I feel like with even 20-30 tests this could get frustrating. 19:00 erlehmann compile faster? 19:01 erlehmann i mean what else should it do, build in parallel? 19:05 MTDiscord erlehmann, I've pushed the broken test. I left it where it originally was, so the test technically works, but the test after it fails. 19:06 MTDiscord It's the function testMountFile in tests/archiveReader.cpp. The test is called testArchiveReader. 19:06 erlehmann why though 19:06 erlehmann why does the test break the one after it? 19:06 MTDiscord Because the test after it checks whether an archive is mounted and fails if one is lol. 19:07 MTDiscord And the mountFile test doesn't clean up after itself, presumably because the test after it is the one that tests cleanig up LOL. 19:07 erlehmann that is fragile 19:07 erlehmann extremly so 19:07 MTDiscord I thought "ok just switch the order of the tests" but if you do that then the mountFile test goes boom for some reason. 19:07 erlehmann classic example of why test cases should be in separate binaries 19:07 MTDiscord So if you want to try to figure out what's going on, I'll be happy to try to write up a solution. 19:08 erlehmann we are witnessing an accidental dependency 19:08 erlehmann i can try 19:08 erlehmann give me a bit of timee 19:08 MTDiscord Uh, these are like sections of a test case. 19:08 MTDiscord If every single test performed were it's own binary we'd easily have over 300 I estimate. 19:09 MTDiscord And that would be unreasonale. 19:10 erlehmann it would not be unreasonable, but these seem to comprise a single test case though 19:10 erlehmann setup, test, cleanup 19:12 erlehmann josiah_wi if you think testing a lot is unreasonable, consider the following: you are writing a function that adds two signed numbers in C++. to ensure that it *never* behaves in an unexpected way, will the tests have more or less lines of code than the function? 19:13 erlehmann i am convinced the tests will have at least about the same amount of code, most likely more 19:14 erlehmann unless you have a framework like python's hypothesis that solves this for you 19:14 MTDiscord No, I'm saying building a seperate binary for every single assertion is unreasonable. 19:14 MTDiscord separate* 19:15 erlehmann no you should build a separate binary for every test case 19:15 erlehmann a test can have multiple assertions 19:15 erlehmann but it should be clear what is tested 19:15 MTDiscord Ok, that's fine. Yeah, I'm assuming here that every file the authors made is one test case. 19:15 erlehmann like all of them should lead back to a common cause 19:15 erlehmann in case of the addition thought experiment, you'd assert a lot of stuff 19:16 erlehmann oh btw 19:16 MTDiscord Right, that makes sense. We are on the same page then. 19:16 erlehmann josiah_wi i strongly suggest to implement a mechanism in your cmake thingy to build the tests with different compiler options in an automated way 19:16 erlehmann you may remember, i already found out that the biome code is optimization-unstable 19:17 erlehmann this is a way to find bugs easily 19:17 MTDiscord Because I made a macro to register the tests it is difficult to build one test with options different than the others. 19:17 erlehmann the test cases must pass at ALL optimization level 19:17 MTDiscord However I was just thinking about this, and I can add parameters to the macro. 19:17 erlehmann otherwise it is illegal c++ most likely 19:17 erlehmann different compilers also btw 19:18 erlehmann but that is another issue 19:18 erlehmann i bet minetest has more cases where clang++ compiles it different than g++ 19:18 sfan5 it looks like you're building something overengieered that will never be finished, have fun 19:18 erlehmann (than the biome code) 19:19 erlehmann sfan5 pretty sure josiah_wi is on the right track here and you are not seeing the benefits until it benefits you personally 19:19 sfan5 I don't remember questioning the benefits 19:20 erlehmann well once josiah_wi has that macro, i can make the tests run with different sanitizer options 19:20 erlehmann which will probably reveal a host of bugs in irrlicht 19:20 erlehmann or a nest of bugs? how do you say in english 19:20 erlehmann also you can add that to your CI 19:21 sfan5 the CI would never succeed again 19:21 erlehmann for the ppl worried about testing “all floats”, the blog post i linked last night asserted it took 90 seconds for them 19:21 erlehmann sfan5 you have low confidence in the tests 19:21 erlehmann or in the codebase 19:22 erlehmann sfan5 btw i think adding the regressions i found for the stuff you fixed regarding image loaders is good, do you see any bad thing with it? 19:22 erlehmann i mean adding the regression tests 19:22 erlehmann could anything bad happen if you require that for each bugfix? 19:23 erlehmann after all, you wouldn't fix a bug if you can not trigger it 19:23 erlehmann so you must have the test case 19:23 erlehmann little additional work 19:23 erlehmann for great benefit 19:23 sfan5 that's great, I'll ask you to write the tests then 19:23 erlehmann i will 19:24 erlehmann for stuff i find ofc 19:24 erlehmann or found 19:24 erlehmann it is a much better way than requiring ppl to download mods from github issues 19:24 erlehmann and load them 19:24 MTDiscord erlehmann, please keep in mind I want to listen to sfan5s advice and keep under his instruction here. 19:24 sfan5 write one for this commit https://github.com/minetest/irrlicht/commit/57ff34b1edb3e477b811ddf1165aa360a2e3eda6 19:24 erlehmann well sfan5 just agreed with me, so i think we are good :D 19:25 MTDiscord If sfan5 has a way he prefers this to be done that's going to be important to me because he's mostly in charge of what gets merged. 19:25 erlehmann sfan5 that is not a bug i found, so i can not agree it 19:25 sfan5 okay 19:25 sfan5 then no test 19:25 sfan5 if you are not willing to do the "little" additional work we won't have tests 19:25 erlehmann sfan5, i have literally no idea how to trigger it, do you? 19:25 sfan5 end of story 19:26 erlehmann look usually either the person finding the bug or the person doing the bugfix is writing a test case (that is how it is at my work) 19:26 erlehmann so i am only saying i write tests for stuff i find 19:26 sfan5 will you pay me then? 19:26 MTDiscord I guess if I want tests I'll have to write a test for that PR. 19:26 erlehmann why should i pay you for writing tests myself? 19:26 erlehmann i am confused 19:27 sfan5 you keep filling this channel with bullshit and complains about things not being they way you want 19:27 sfan5 and then YOU do not want to put in the work to get it that way 19:27 sfan5 complaints* 19:27 erlehmann i offered to add regression test and asked you about what would happen if they were required 19:27 sfan5 yes 19:27 sfan5 there is nothing wrong with your question but I'm having no more of your bullshit 19:27 erlehmann i want to put in the work for stuff i know, but it is unreasonable to ask me to write a test for code i do not know for a bug i have not investigated 19:28 MTDiscord I'd actually like to talk to sfan5 about testing a bit without erlehmann's interruption. 19:28 erlehmann ok! 19:28 erlehmann i be silent 19:28 sfan5 no, I will go and do something else now 19:28 MTDiscord Ok, you got it. I'll get back to work. 19:29 erlehmann well then … josiah_wi do you think the test thing is good enough i can add regression tests for the stuff i found regarding image loading? the oversized PNG & BMP? 19:29 erlehmann or is it too early? 19:30 erlehmann in case of no answer i'll wait until the testing PR is merged 19:30 erlehmann i do not wish to endanger its chances 19:30 erlehmann by meddling more 19:33 MTDiscord erlehmann, I have added a variable you can set -DUNITTEST_FLAGS to add compilation flags to all the unit tests. Modifying flags on a single unit test is not something developers will normally want to do, so I do not think it deserves a special configuration. You can use add_compile_options to add flags to a single unit test by its target name. 19:35 erlehmann josiah_wi i don't want to change flags on a single unit test, i want to run all of them normally, then with ubsan, then with asan usw. 19:35 erlehmann the sanitizers are incompatible with each other 19:36 MTDiscord Ah great, -DUNITTEST_FLAGS will work then. 19:36 erlehmann also i want to run all of them with -O0, -O1, -O2, -O3, to make sure something like the biome code returning different seeds at all optimization levels is excluded 100% 19:36 MTDiscord Once I push it, that is. 19:36 erlehmann nice thx 19:36 erlehmann tell me once it is there so i can try that 19:36 MTDiscord Be aware that standard CMake flags and CMAKE_CXX_FLAGS will also affect unit test compilation. 19:37 erlehmann requiring all sanitizers to not warn will be good payoff for little benefit i think 19:37 erlehmann ok 19:37 erlehmann but shouldn't it be named UNITTEST_CXX_FLAGS then? 19:37 erlehmann after all, you could have other stuff set for unittests 19:38 erlehmann or even UNITTEST_CMAKE_CXX_FLAGS if you want to handle it by prefix, dunno 19:38 erlehmann i am not knowledgeful enough to know what would be good 19:38 erlehmann you know about cmake so i trust you it will be right 19:41 MTDiscord Ok, everything is ready, I'm taking a break. 19:41 erlehmann yay 19:41 erlehmann you do good work! 19:41 erlehmann cora told me also that you help people with dragonfire :) 19:43 MTDiscord I was on the support team briefly, but I am not currently associated with them. 19:43 MTDiscord I wouldn't be so quick to say my work is good. I think that's for the core devs to judge. 19:46 erlehmann josiah_wi you are determined to get to a goal and listen to everyone, that is a good quality 19:46 erlehmann i'd like to know some opinions about testing from someone except sfan5 actually 19:47 erlehmann i mean “CI would never pass again” is probably not the future mt is headed 19:48 erlehmann towards 20:07 sfan5 I said that because sanitizers (judging by the results I've seen so far) tend to reveal a lot of bugs that are of low severity and possibly don't even affect day-to-day usage of Minetest 20:10 erlehmann well, ubsan has no false positives 20:10 MTDiscord you want opinions from people other than sfan5? tests are useful abut at this point your talking here is turning into bullshit rants about wanting stuff but not wanting to put that much work into it yourself or excuses why 20:11 erlehmann ok 20:11 erlehmann hey i told you i make tests for bugs i know one the test PR is accepted, measure me by that pls 20:12 sfan5 erlehmann: that does not change my statement in any way nor did I imply otherwise 20:13 erlehmann well i get it, the bugs are interesting from my “the program must do the same under all circumstances POV”, but not from your “do users notice it?” POV 20:13 erlehmann that is why we agree on tests regarding crashes, but on barely anything else 20:13 erlehmann right? 20:13 MTDiscord "the program must do the same under all circumstances POV" Is that even possible once GPUs get involved? 20:14 MTDiscord Like, not even multi-billion dollar companies manage that 20:14 erlehmann Hugues Ross i think most companies just don't see the value in it. aerospace and banking are *much* more careful than FAANG companies 20:15 erlehmann and even then you get boeing 747 max (which was a management/human error more than a software one) 20:15 MTDiscord you mean the 737 max? 20:16 erlehmann oh yes, sorry. i am not careful 20:16 erlehmann the one whomst crashed 20:18 erlehmann regarding the small things that are not noticeable, i already know how to pull values out of RAM from the game proper and write them, but i was not able to do that in a way to not crash the game (and i have looked at related issues, so ig it is so far out no one would believe it is possible). once i have an exploit, you get it. 20:18 MTDiscord on a random tangent, its rather funny you imply sfan5 doesnt care about tests, but https://github.com/Uberi/Minetest-WorldEdit/commit/770601dd5d6ed899c191aff4be2a94a8722b3b1b exists 20:19 sfan5 not the best example given the amount of tests and that WorldEdit was without any for years 20:19 erlehmann Jonathon sfan5 cares about a different level of scrutiny regarding testing than i do 20:19 erlehmann i believe it will all work out 20:19 erlehmann neither will everything be blocked, nor will there be “too many fragile tests” 20:19 erlehmann (bc someone would have to write them) 20:19 MTDiscord sfan5: fair enough, was the most recent testing thing i recall you doing 20:21 erlehmann sfan5, i like the WE tests 20:22 erlehmann can any mod do minetest.forceload_block? 20:25 sfan5 sure 20:25 erlehmann big oof 20:25 sfan5 but the amount is globally limited to 10 blocks 20:25 sfan5 I can see why that was done but it's extremely in the way 20:26 erlehmann from having cloned the repo, what is the fastest way to get to run the test cases? https://github.com/Uberi/Minetest-WorldEdit/blob/master/.util/run_tests.sh needs ../../bin/minetestserver and is prob not the entry point 20:27 MTDiscord i assume the ideal way would be to calculate max forceload blocks would be a percentage of available ram? 20:27 sfan5 the script works if you indeed have ../../bin/minetestserver, for what github does check .github/workflows/ 20:27 erlehmann ok 20:35 erlehmann sfan5 here is a tiny patch to the script, just pipe it into “git am” https://mister-muffin.de/p/oGcQ.diff 20:35 erlehmann “git am” will automatically make the commit 20:36 MTDiscord >but the amount is globally limited to 10 blocks 20:36 MTDiscord that explains why my mod to turn the world on its side choked hard 20:37 MTDiscord is that even documented 20:37 MTDiscord i dont think so 20:37 erlehmann GreenXenith lol which mod 20:37 MTDiscord obviously not released since it doesnt work 20:38 MTDiscord https://github.com/minetest/minetest/blob/master/doc/lua_api.txt#L6037-L6047 well, if its documented, its not documented in a sane place 20:40 erlehmann sfan5, do you appreciate it? in that case i deliver more small fixe 20:40 erlehmann s 20:44 MTDiscord Cool, Irrlicht has its own atof implementation which is faster only in Release configuration. And if it isn't fast enough it fails a unit test. 20:44 erlehmann wait i had that thing too 20:45 erlehmann it was randomly failing 20:45 erlehmann why does it have that anyway 20:45 x2048 Hey everyone, I'd like to know more about situation with shadow mapping. Is it fine to keep working on fixes such as GLES 2.0 shaders or are we waiting for the new rendering pipeline? 20:50 sfan5 x2048: yes that's fine 20:51 sfan5 erlehmann: thanks but I don't think that'll ever end up being a problem 20:51 sfan5 (just like I didn't quote $tempdir) 20:52 erlehmann sfan5, tempdir is a fixed value though 20:52 erlehmann sfan5 i am tired of discussion, but will you apply it or not? 20:53 sfan5 I don't plan to 20:53 MTDiscord Apply what? Why are you bringing this up again if you're tired of it? 20:54 erlehmann josiah_wi i provided a very small fix to https://github.com/Uberi/Minetest-WorldEdit/blob/master/.util/run_tests.sh because there is a variable that is not set in the script, so it may come from outside, so it may introduce errors if it ever contains a character in $IFS (like a space, tab, newline) 20:54 erlehmann just quoting the variable 20:55 MTDiscord Why telling me? Not even my script. 20:55 sfan5 you asked? 20:55 erlehmann because you asked? 20:55 MTDiscord Oh, that, I see. 20:56 erlehmann sfan5 are you even the right person to send this patch to? like is there someone else who should know? 20:56 erlehmann (i thought: you did the last commit on this) 20:57 MTDiscord seems #minetest is the correct place for this anyways 20:57 erlehmann yes correct 20:57 erlehmann sorry 20:57 sfan5 I doubt asking the second person with push access who hasn't committed to WorldEdit in years would help 20:57 erlehmann ok i give up then thx 20:58 erlehmann sfan5 if you ever make a shell script for the CI, shellcheck is a good linter. do not blindly follow its advice though. it will always tell you to quote all variables, even if you *want* to split them. 20:58 erlehmann it will also catch some other stuff btw 20:58 erlehmann (i hope that helps at some point in the future) 21:04 erlehmann sfan5 i understand your response as “don't send sfan5 bug fixes for really minor stuff” ig? 21:04 erlehmann correct? 21:04 erlehmann i don't want to put in work just to annoy you 21:05 sfan5 it's "don't send sfan5 correctness fixed he doesn't think are necessary" but not like you can find out without sending them to me first 21:05 sfan5 not a topic for here in either case, if you have anything else -> #minetest 21:06 erlehmann ok 21:18 MTDiscord Can someone confirm that the software driver, and Bunning's video driver were removed from Irrlicht? 21:18 sfan5 yes 21:18 MTDiscord Burning's* 21:19 MTDiscord Thank you. 21:20 erlehmann josiah_wi both did not work very well last time i checked and you can always get *VERY* slow software emulation with LIBGL_ALWAYS_SOFTWARE=1 if you need it 21:20 erlehmann like on CI 21:21 erlehmann josiah_wi if you want to see for yourself, in the examples folder should be some stuff that renders a scene with them, it looks just wrong 21:25 MTDiscord I pushed a test testCursorSetInvisible that segfaults. It's commented out. 22:09 josiah_wi #join minetest 22:09 josiah_wi bah, please forgive my interruption 22:33 x2048 #11331 and #11341 are a possible close (fixed by #11385), but I cannot tag them. Can someone help? 22:33 ShadowBot https://github.com/minetest/minetest/issues/11331 -- Mapped shadows grow faint with distance from the player. 22:33 ShadowBot https://github.com/minetest/minetest/issues/11341 -- Shadows are not filtered at distance (apparent with zoom) 22:33 ShadowBot https://github.com/minetest/minetest/issues/11385 -- Improve shadow rendering with non-default camera FOV by x2048 22:38 rubenwardy sfan5: should this release be 5.4.1-someExtra or 5.4.2-android or 5.4.2? 22:38 sfan5 5.4.2-android is my suggestion 22:39 rubenwardy OK, will versionExtra act as I expect, setting the part after -? 22:39 rubenwardy or would that just be present on the tag 22:41 sfan5 i'd only tag it like that 22:43 rubenwardy ok https://github.com/rubenwardy/minetest/releases/tag/5.4.2-android 22:43 rubenwardy if you could look over the PR again 22:43 rubenwardy if it looks good, then merge into stable-5, tag, update playstore? 22:52 sfan5 if it's android only maybe don't touch CMakeLists.txt and appdata.xml 22:52 sfan5 dunno 23:02 rubenwardy it would be good to know if a desktop has is using 5.4.2, even if it's only really deployed to android 23:02 rubenwardy if it's merged tagged and/or merged into stable-5, that's a possibility 23:06 sfan5 good point 23:31 erlehmann https://danluu.com/testing/