Time Nick Message 09:36 rubenwardy merging #13390 in 10 09:36 ShadowBot https://github.com/minetest/minetest/issues/13390 -- Add `vector.in_area()` utility function by AFCMS 13:14 Jon[m] I'm far enough down the rabbit hole to ask in this text channel now: how do I run unit tests as performed by minetest/games/devtest/mods/unit tests/*.lua? 13:14 Jon[m] I have done: 13:14 Jon[m] minetest --run-unittests as well as modified settingtypes to have devtest_unittests_autostart bool true 13:26 BuckarooBanzai Jon[m]: have you tried running them with `busted`? 13:29 rubenwardy busted builtin 13:29 rubenwardy oh wait, devtest 13:30 rubenwardy those are actually integration tests 13:30 rubenwardy you need to run dev test and then type /unittests 13:30 Guest54 Jon[m] if you want to find bugs in minetest, you can compile minetest using a sanitizer, e.g. with CXXFLAGS `-fsanitize=undefined -fno-sanitize=vptr` for ubsan 13:31 rubenwardy not what they asked 13:31 Guest54 yeah but it shows when unittests break 13:31 Guest54 or when they trigger bugs in code 13:32 rubenwardy you can also use devtest_unittests_autostart = true to do /unittests automatically when opening devtest 13:32 Guest54 neat 13:33 rubenwardy Minetest's CI runs the devtest "unit" tests using this script https://github.com/minetest/minetest/blob/master/util/test_multiplayer.sh 13:35 Guest54 is there a way to run each minetest binary test separately? 13:38 Guest54 that's kinda needed if you want tests to fail on scenarios that the test itself can not dig itself out of 14:13 Guest54 the scenario i have is that i want to trap on these runtime errors in the “passing” unit tests btw https://mister-muffin.de/p/rDda.txt 14:14 MTDiscord exceptions were a mistake 14:15 Guest54 what do exceptions have to do with this? if the test case is undefined at any point, it is entirely undefined. 14:16 Fleckenstein the problem is creating scenarios that "tests can't dig themselves out of" 14:16 Guest54 uh, in any case, i do not see how you can recover from a test case that unconditionally runs into undefined behaviour 14:16 Fleckenstein undefined behavior is an ever bigger mistake than exceptions 14:17 Guest54 look, i can cut this short: computers were a mistake, daddy turing should not have done that 14:17 Fleckenstein i agree 14:17 Fleckenstein lambda calculus > 14:17 Fleckenstein turing 14:17 Fleckenstein whatever 14:18 Guest54 anyways, if you have test scenarios 1, 2, 3 and the program under test is also the test runner and you get UB in test 2, you can't really continue. 14:18 Fleckenstein just don't UB 14:18 Fleckenstein only rely on behavior that is defined 14:18 Guest54 “if i were you, i just would not make any mistakes” hehehehe 14:18 Fleckenstein whar 14:19 Fleckenstein im just shilling rust 14:19 Guest54 as far as i can see it it's excessively hard to not run into UB in both C and C++ unless you are a real smartie (and i know very few of them) 14:19 Fleckenstein see above message 14:19 Guest54 well then just say it: the mistake is to use c++ 14:19 Fleckenstein yes 14:19 Fleckenstein but i was a bit more specific than that 14:19 Guest54 i agree, but minetest is c++, so the discussion is worthless until you have your rust-based minetest replacement ready 14:20 Guest54 or rather, fruitless 14:20 Guest54 anyways, is there a way to run only a single test? because i don't see one right now. 14:20 Fleckenstein i wanted to use your issue to point out the benefits of not having certain things in a language 14:21 Fleckenstein just change the unit testing system 14:21 Guest54 but if i had one, i'd try to modify the CI to check for address/memory/ub issues 14:21 Fleckenstein i agree that separate binaries are better in C(++), that's how I do it in my programs 14:21 Guest54 care to show an example? 14:22 Fleckenstein I doubt that it's relevant/helpful, but 14:22 Fleckenstein https://github.com/dragonblocks/dragonstd/tree/main/dragonstd/test 14:25 Guest54 well, it looks nice except for the way you build it (ask me in query about it if you want). btw, did you know that compilers can optimize code when you assert stuff? 14:26 Guest54 like after the assert the condition is obviously true 14:26 Guest54 anyways, thanks 14:26 Fleckenstein > after the assert the condition is obviously true 14:27 Fleckenstein smark 14:27 Fleckenstein t 14:27 Fleckenstein it's new to me 14:27 Guest54 i suggest you play around with code on godbolt.org – you might learn stuff regarding optimizations 14:29 Guest54 circling back to dev topics btw, how ready *is* your rust-based minetest replacement? 14:29 Fleckenstein i wouldn't call it anywhere near ready 14:29 Fleckenstein currently working on blending n stuff 14:29 Fleckenstein water 14:29 Guest54 obviously, but is it a good platform for bots already? 14:30 Fleckenstein well I'd have to extract the auth from the client implementation to a library 14:30 Fleckenstein currently some parts are very monolithic and i dont like that 14:31 Fleckenstein and i think there are issues where the server resends some reliables 14:31 Fleckenstein i dont understand why 14:31 Fleckenstein it seems to work after having resent them 14:31 Guest54 if you are doing texture blending i suggest to not do it like minetest, read up about gamma and make your blending so that in principle it could be gamma-aware 14:31 Guest54 you'll avoid a bunch of conceptual mistakes that way 14:32 Fleckenstein whar 14:34 Guest54 Fleckenstein install the gamma nodes that i could not get into devtest and see for yourself https://github.com/minetest/minetest/pull/12089 14:34 Guest54 the bottom left node has a larger black bar where there should be a gradient. the reason for that is that the blending occurs in the wrong colorspace, which just so works when your entire rendering is not gamma-aware 14:34 Guest54 the problem is that PNGs can contain gamma information 14:36 Fleckenstein is a gamma aware pipeline significatly architecturally different from a non gamma aware pipeline 14:36 Guest54 not really 14:37 Guest54 the gradient issue you see in minetest is basically a precision issue 14:39 Guest54 Fleckenstein the non-gamma-aware-pipeline basically just leaves out some steps that look superfluous at first glance, but lead to weird bugs like “my gradient looks weird if the gamma is not exactly what i expect” 14:41 Guest54 if you want to experience yourself why handling blending correctly is important, try implementing a function that displays a gradient that blends from red to transparent in RGBA. if you define transparent as (0, 0, 0, 0) and full red as (255, 0, 0, 255). if you do it correctly, you get a more transparent red. if you do it incorrectly, the red gets 14:41 Guest54 darker while it gets more transparent, because the transparent color is transparent black. 14:42 Guest54 does that example make sense to you? 14:42 Guest54 like, the naive, obvious, but wrong solution is to just linearly interpolate all values R G B A to get the gradient, but it looks weird 14:43 Guest54 (halfway between those is, for example (127, 0, 0, 127), which is certainly not as red) 14:51 Fleckenstein just use GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA 14:51 Guest54 why? does that do premultiplied blending? 14:52 Fleckenstein i dont even get it 14:53 Fleckenstein you start with the object that is furthest away 14:53 Guest54 uh what 14:53 Guest54 i am talking about texture blending here 14:53 Fleckenstein im talking about alpha blending while rendering 14:53 Guest54 that's like, the same? 14:54 Guest54 anyways, if you do the red-to-transparent thing with premultiplied instead of linear blending it will look “correct” 14:54 Guest54 it's basically the same amount of effort to handle gamma correctly if i am not mistaken 14:54 Guest54 a few multiplications here and there 14:55 Fleckenstein the destination cannot be transparent 14:55 Fleckenstein you always begin with a fully opaque background color 14:55 Fleckenstein e.g. from the skybox 14:56 Guest54 i have the feeling you are putting the cart before the horse here. think about texmods. 14:56 Guest54 like, construct your approach to blending from atoms and go up 14:58 Zughy[m] MT staff: I've posted the next meeting notice, taking place on Sunday 14:58 Fleckenstein the alpha value for the source color *is* the ratio 14:58 Fleckenstein the destination does not have an alpha channel 14:59 Guest54 anyways, i am sure you will figure it out. i just wanted to note that if you do not cargo-cult the approach of always using linear you will probably have less weird corner-cases in blending, such as the above-mentioned gradient that looks weird when gamma is not matching expectations. 15:00 Fleckenstein either your remark had nothing to do with what im implementing or you don't get it 15:00 Guest54 or the other above-mentioned gradient where halfway between red and transparent is half-transparent half-bright red 15:00 Guest54 i think it makes sense to query me 15:01 Fleckenstein you dont blend (0, 0, 0, 0) and (255, 0, 0, 255) 15:01 Zughy[m] aren't you, like, flooding the chat? 15:01 Guest54 the naive approach does that 15:01 Fleckenstein you could for example however have (255, 0, 0) and lay (0, 0, 0, 0) on top of that 15:01 Fleckenstein which results in (255, 0, 0) 15:01 Guest54 well, i think the only thing of value that can come out of this discussion is that handling gamma is not that hard 15:02 Fleckenstein if you have (0, 0, 0) and lay (255, 0, 0, 255) on top of it, you get (255, 0, 0) 15:02 Guest54 so the value to others is limited, which is why i suggested the query 15:02 Guest54 Fleckenstein conceptually i think you can not blend an RGB and RGBA value together, it's a type mismatch 15:02 MisterE[m] Would it be possible to make a noise have a variable spread, such as spread=function that returns a vector? That could be ... useful. My guess is no..., since I don't see a way to pass the position being calculated to it. 15:03 Guest54 MisterE[m] what is your actual use case for which you want it? 15:03 Fleckenstein Guest54 no, I am talking about alpha blending while rendering 15:03 MisterE[m] And if no, is that due to the nature of perlin noise or minetest's implementation? 15:03 Guest54 Fleckenstein and i am talking about https://en.wikipedia.org/wiki/Alpha_compositing#Straight_versus_premultiplied 15:04 Guest54 i'll be quiet until you tell me you read that :P 15:04 MisterE[m] Guest54: I want mountains that are elongated in a certain direction, and that direction changes over the map 15:04 MisterE[m] Caves too 15:05 Guest54 MisterE[m] so like a mountain range that curves? 15:05 MisterE[m] So, I want to stretch a noise with a noise 15:05 MisterE[m] In a vector direction 15:05 MisterE[m] Ideally, without unnecessary calculation 15:06 MisterE[m] Guest54, exactly 15:06 Guest54 well, does it have to be able to curve into itself? 15:06 MisterE[m] Or a 3d cave system that tends to have tunnels in a certain direction, that that direction changes over space 15:07 MisterE[m] I don't understand your question 15:08 Guest54 i should probably ask a more well-defined question such as “why do you not want to multiply a noise heightmap with something that generates smooth mountains?” 15:08 Guest54 or add them, whatever 15:10 MisterE[m] Because that does not make long valleys and long mountain ranges, that makes mountains that are roughly equal in every direction 15:10 MisterE[m] I could have two noises that are elongated in each direction, that are mixed, but that seems needlessly calculation-intensive 15:11 MisterE[m] I would prefer to modulate the spread vector with another large-scale noise 15:13 Guest54 here is an article from ken perlin about how to improve perlin noise, maybe the rendering of the blue sphere at the bottom fits your use case? https://web.archive.org/web/20080409215811/http://developer.download.nvidia.com/books/HTML/gpugems/gpugems_ch05.html 15:13 Guest54 > Figure 5-11 Bump Mapping a Sphere 15:13 Guest54 the middle one 15:14 MisterE[m] ... that is not relevant to my question, which is about minetest's implementation. 15:15 Guest54 it does look like it might be useful for mountain ranges and valles 15:15 Guest54 valleys 15:15 Guest54 that's why i brought it up, sorry 15:16 Fleckenstein > c_0 = C_a + C_b * (1 - a_a) 15:16 Guest54 my face when looking at this stuff: a_a 15:16 Fleckenstein if C_a is premultiplied with a_a, that's literally what GL_ONE_MINUS_SRC_ALPHA does 15:17 Fleckenstein i mean GL_SRC_ALPHA,GL_ONE_MINUS_SRC_ALPHA 15:17 Guest54 well, good for you that you already found the correct thing ig 15:17 Fleckenstein no 15:17 MisterE123 Fleckenstein, is that formula for me? 15:17 Fleckenstein i think this has nothing to do with what I'm doing 15:18 Guest54 why not? in case you did not have it before, now you have the conceptual understanding *why* it works 15:18 Fleckenstein what 15:18 Fleckenstein no 15:18 Fleckenstein again; this is not related 15:18 Fleckenstein there is no alpha target 15:19 Fleckenstein the target is opaque 15:19 Fleckenstein you just blend transparent values over that 15:19 Guest54 look, the alpha is a red herring. the common thing is ”doing blending calculations in the correct colorspace with the correct precision is only a little more work than doing it with the naive linear interpolation approach, but fixes a bunch of blending weirdnesses” 15:20 Guest54 that was about what i wanted to communicate 15:20 Guest54 because minetest does not do it correctly at several places 15:20 Fleckenstein its possible that your remark is replated to what I'm doing 15:21 Guest54 ig you'll be able to tell me in the future if it was or not 15:21 Fleckenstein but if it is, you're not doing a good job in actually explaining how it is related 15:21 Fleckenstein the example you named was having 2 RGBA values and interpolating between them 15:22 Guest54 because there it is obvious what's wrong with linear interpolation when you get to the dark red in the middle of the gradient 15:22 Fleckenstein i am interpolating between two RGB values, using an alpha value as ratio 15:22 Guest54 with the gamma example, most people just shrug and say “too much work to get it right, too bad” even though it's essentially the same thing 15:22 Guest54 as i said, you might already work in premultiplied space 15:23 Fleckenstein i just dont see how it's related 15:23 Fleckenstein the article you linked has output alpha 15:24 Fleckenstein and alpha for the bottom texture 15:24 Guest54 maybe it's easier to ask “what has to happen so two textures from the set of textures in the not-in-devtest gamma test nodes are correctly blended?” 15:25 Fleckenstein ok, lets say i have an opaque blue background (0, 0, 200) and want to overlay a half transparent red over that (255, 0, 0, 127) 15:25 Fleckenstein the result would be (127, 0, 100) 15:25 Fleckenstein is there an issue with that? 15:28 Fleckenstein if not, I'll keep in mind to research it / ask you about it when I work on texture modifiers but it's not related to blending during rendering 15:28 Guest54 Fleckenstein look at figure 9, the left image is the gamma-correct version, the right image is what i believe to be your approach: https://blog.johnnovak.net/2016/09/21/what-every-coder-should-know-about-gamma/#alpha-blending--compositing 15:29 Fleckenstein > the top one is drawn with 100% opacity and bottom one with 50% 15:29 Guest54 you see the shifts in brightness? 15:29 Fleckenstein i fail to see how this is related 15:29 Fleckenstein the bottom one has *always* 100% opacity in my case 15:29 Guest54 top and bottom here refers to the position in the image 15:29 Guest54 not to the layers 15:30 Guest54 or so i believe 15:30 Guest54 there are solid bars and below them are half-transparent bars 15:30 Guest54 both are blended on top of solid background 15:30 Guest54 so it's like your example, or not? 15:31 Fleckenstein makes sense, yes 15:31 Fleckenstein it is related 15:31 Guest54 yeah and you see the difference between doing it correctly and incorrectly, even though it is subtle 15:31 Guest54 that page has a lot of different examples regarding what happens if you use the naive approach 15:31 Fleckenstein they are different, but I can't judge which one is more "correct" 15:32 Guest54 it would be more easy if you read the entire page and look at all examples 15:32 Guest54 what is ”correct” depends on if you want to have a ”natural light” look 15:32 Guest54 i think the fundamental problem is that light emission does not equal perceptual brightness 15:33 Guest54 look at these grey bars here: https://blog.johnnovak.net/2016/09/21/what-every-coder-should-know-about-gamma/#light-emission-vs-perceptual-brightness 15:33 Guest54 btw you could also fix that stuff in the minetest engine once you understand it ;) 15:34 Guest54 anyway, john novak's page explains it probably better then i can why this is important and how your rendering fails if you think it is not 15:34 Guest54 if you can live with these rendering artifacts, then don't do it, but to me it looks ugly 15:36 Fleckenstein I'd assume that some people hack their textures in a way that tries to mitigates some of these issues, subsequently gamma correction should maybe be a setting? 15:36 Guest54 for added insight you can figure out which programs fail to render these images that i used for the devtest stuff correctly (spoiler: a lot) http://www.schaik.com/pngsuite/pngsuite_gam_png.html 15:36 Guest54 gamma correction *is* metadata in PNG 15:36 Guest54 and no you can't really “hack your textures” to work around these issues 15:37 Guest54 i mean you can so they look correctly on your computer with your gamma and whatever 15:37 Guest54 but i fail to see how you can workaround the blending/rendering artifacts that occur in linear space 15:38 Fleckenstein yea that's why I used the term hack 15:39 Guest54 well this is also an issue of precision. if your engine blends in a space where your colors get mushed together, no amount of work will make up for it. 15:39 Guest54 anyway, the effects are subtle enough that fixing it in minetest was rejected so hard that not even tests for ”is this fixed” were accepted. 15:40 Guest54 and also many applications kinda get away with not doing it. i say “kinda” because once you know what to look for (like novak's examples) you can find those. 15:40 Fleckenstein as far as I understand they didn't want to have tests for a feature that doesn't exist? 15:42 Guest54 well, it leads to a general question “do you want to have failing test cases for stuff you may implement in the future, to highlight that this does not work yet, or at all”. TDD methodology as i understand it says yes, a lot of people think it is stupid though. 15:42 Guest54 in any case, not wanting the tests with the justification ”we are not doing this, ever” is a clear signal that either the work to be done is over-estimated or it is not considered worth it anyway 15:43 Guest54 but if you are bulding something anew, IMO it makes sense to do it correctly the first time, because the correct approach can easily emulate the incorrect one, but not vice versa 15:43 Guest54 i.e. you just skip some steps or move them around to emulate the naive linear thing 15:44 MTDiscord TDD methodology says you should have 1 failing unit test for something you will implement presently. 15:45 MTDiscord You don't want failing tests for features you won't implement yet. Failing tests should block PRs, TDD or not. 15:45 rubenwardy TDD means you should write the tests before the implementation, you typically make the tests work before submitting the PR though 15:45 Fleckenstein if it's not a lot of work, why don't you just implement it instead of complaining they won't merge the tests 15:46 Guest54 have i misunderstood the part where i think that gamma-aware rendering is not wanted? 15:47 Guest54 there was a coredev meeting on it, it's linked in the issue i think 15:47 Guest54 josiah_wi regarding TDD, there are different approaches and you can easily permit failing test cases if they never worked before 15:48 Guest54 josiah_wi also “failing tests should block PRs” is an approach that is subtly different from “failing tests that worked in the past should block PRs” 15:48 Fleckenstein I think you are failing to see this from the core dev's perspective 15:49 Guest54 i think i understand it perfectly well, the blending artifacts are subtle enough to not care at all 15:49 Fleckenstein If it's not a feature they plan on making, it doesn't make sense to put tests in the game 15:49 Fleckenstein this does not mean they will necessarily reject the feature 15:49 Fleckenstein it's just not on the roadmap 15:50 Guest54 Fleckenstein i have seen what happens to topics not on the roadmap, e.g. prollers maps 15:50 Fleckenstein lots of minor PRs get merged that are not part of the roadmap 15:51 rubenwardy roadmap also doesn't apply to bug fixes 15:53 Guest54 yes, but in this case a bunch of gamma-handling code was also dropped from irrlichtmt some time later. maybe that was unrelated, i interpreted it as “the horse is dead”. 15:54 Guest54 maybe i misread it https://github.com/minetest/irrlicht/commit/538c9e5cde425ecee1e802fd53344111683b60be 15:55 Fleckenstein I think so too 15:56 Fleckenstein won't be wanting to set the gamma *for the entire display* 15:56 MTDiscord that seems to just be for configuring gamma for the entire screen 15:56 Fleckenstein ^ 15:56 Guest54 well i saw it as ”gamma is not important”, i may have ben too eager here 15:56 Fleckenstein nothing in the core dev meeting or the issue suggests that gamma is an unwanted feature 15:57 Guest54 let me read it again so i can repent if necessary 16:12 Guest54 reading it again, i think that you are both correct in that failing test cases are never accepted, but i probably focused on “From the estimated amount of work and also being a feature approximately nobody cares about this would be rejected” 16:12 Guest54 which leads me back to “the artifacts are subtle enough that you can get away with not caring, but it looks ugly” 16:13 Guest54 anyways, i hope the topic of how to blend correctly is done for today ^^ 16:13 Guest54 i feel sorry for bringing it up here 16:18 sfan5 great, 200 messages I won't be reading 16:18 Guest54 it's almost all about premultiplied alpha and gamma correction, there is no need to 16:19 Krock sfan5: ChatGPT /s 16:19 Krock (because it can do summaries) 16:41 Desour Jon[m]: FYI, you can't change settings by modifying settingtypes.txt. that file is just for mainmenu to display the settings menu. use minetest.conf instead 16:42 Jon[m] Desour: Doh! That explains that. Thank you 16:43 Desour !next 16:43 ShadowBot Another satisfied customer. Next! 16:43 Jon[m] Thank you for the quick responses folks. I'll be following up after work today 17:08 MTDiscord I still have the add favorite button. Based on the video, is it something I should make a PR for, just keep working on it, or keep it to myself? https://www.youtube.com/watch?v=f13dPP5etII 19:04 Fleckenstein Guest54: https://mister-muffin.de/p/JDlZ.diff 19:06 Guest54 for context, Fleckenstein and me are in a call and our goal is to have fast smooth shadows using builtin irrlicht facilities 19:16 Guest54 i believe the patch above already does work and looks quite nice, but all the light comes from the camera ^^ 20:07 sfan5 you mean stencil shadows?