Time Nick Message 03:06 MTDiscord Minemine 03:06 MTDiscord Or Testtest 08:14 MTDiscord Tbh a serious good name could be "Libre-/Open-" + "-voxel/-box" 08:21 MTDiscord Openbox is already taken 10:11 [MTMatrix] There is an internal poll going on, discussing potential new names is pointless by now 11:20 MTDiscord which domain names will be checked for availability/secured? (is that a public information?) will it be first come first served for the country top-level domains? 11:53 rubenwardy .com .org .net 18:43 sfan5 merging #14448, #14474 in 10m 18:43 ShadowBot https://github.com/minetest/minetest/issues/14448 -- Faster blit_with_alpha() by HybridDog 18:43 ShadowBot https://github.com/minetest/minetest/issues/14474 -- update Lua BitOp's stdint.h check for MSVC by goodusername123 19:04 Krock > Server: Maximum lag peaked at 37.891 (steplen=0.05) 19:04 Krock hmm. valgrind isn't too happy about MineClone 2 19:05 MTDiscord Krock: does asan not suffice to find the issue? 19:06 Krock It might. I just didn't want to recompile Minetest. Turns out that asan would've been faster by a long shot 19:07 sfan5 valgrind is kind of out of fashion 19:42 [MTMatrix] Krock: you might want to have a look at monitoring.aes.land: we have huge peaks every time a new user joins. We profiled everything we could and it doesn't looks like a mistake on our end. We've also started using a cache media server but it's not helping much. 🤷‍♀️ 19:46 sfan5 have you profiled with perf? 20:43 [MTMatrix] no idea such command existed. We used the LuaJIT profiler mod written by TurkeyMcMac (https://content.minetest.net/packages/jwmhjwmh/jitprofiler/) 20:46 Desour profiling with the luajit profiler will only pofile what you're doing in lua, not anything that the engine does when it's not directly called from lua 20:50 sfan5 https://github.com/minetest/minetest/blob/master/doc/developing/misc.md <- linux tool, not a command 21:10 Desour why does imageCleanTransparent have a threshold for alpha that we set to 127 in case of applyfiltersformesh? isn't the ALPHA_CHANNEL_REF argument totally irrelevant? 21:16 MTDiscord The filter is supposed to compute the RGB values for transparent pixels (where we suspect correct RGB values have been discarded by image optimizers) using the RGB values only of opaque pixels (where we assume optimizers haven't mangled the RGB values) 21:17 Desour yes, I know. but why would we consider pixels with 127 >= alpha > 0 as transparent? 21:17 MTDiscord I know one reason we might: image blend mode = "clip" would. 21:17 MTDiscord as far as the original use, i.e. image optimizers, though ... I would expect a threshold of 0 or 1 (depending on whether it's > or >=) to make more sense. 21:17 Desour that's also what the code comment says, but it doesn't make sense in this context 21:18 MTDiscord I think tbh the threshold should be independent of image blend mode, i.e. it should be some value that always makes sense regardless of the context in which the image will eventually be used. It's an image loading filter, not a texture use one. 21:19 sfan5 theory: if you have blend mode = clip you could in theory discard all colors in pixels with a < 128 as an optimization and it would work fine, so in theory imageCleanTransparent has to fix those cases up too 21:20 MTDiscord Hmm, we were already weighting the average by alpha, so it's not necessary to do complicated threshold shit. 21:20 Desour ahh, that would logically make sense 21:21 Desour but we have half-transparent nodes nowadays 21:21 sfan5 (i put way too many "theory" in that sentence) 21:21 MTDiscord u32 a = d.getAlpha() <= threshold ? 255 : d.getAlpha(); <-- lol, wtf is this? 21:22 Desour ^ that's because the newly set pixels have a 1 in the bitmap, but alpha < threshold 21:22 MTDiscord pretty sure that's supposed to be a 0, not a 255. Which isn't really necessary anyway, since getAlpha() will be low enough for them anyway. 21:22 Desour (makes not much sense to weight them full 255 though) 21:23 MTDiscord Yeah, this code all looks wrong, and the only reason why it probably doesn't cause issues in practice was because the whole alpha-weighted averaging thing would only happen in rare cases anyway, and would probably be subtle in any event. 21:24 MTDiscord The idea was dealing with "high contrast corner" cases, like where a bright red flower meets a green stem, to try to avoid having a weird gray spot there. 21:26 MTDiscord That 255 looks like somebody tried to make a change to the code without actually understanding the idea behind it. 21:27 Desour ^ high contrast cases would actually best be dealt by conserving the histogram (= the set of occurring colors + commonness), to not make green+red=yellow spot in the middle 21:27 MTDiscord The inclusion of a "threshold" parameter was originally my mistake. 21:28 MTDiscord There's no "conserving the histogram" here, you've got 8 neighbor pixels and you need to pick a color to set the central one. Just picking the "most central" color is probably the closest you could get, but doesn't necessarily entirely work, and it's hard to pick a "central" color when you have histograms on 3 separate dimensions. 21:29 MTDiscord (up to 8 neighbors; we don't wrap around on image edges) 21:31 MTDiscord Okay, actually, I was wrong about that 255 thing: it turns out it's a noop. The code is basically: if (alpha > threshold) { /* ... / weight = alpha <= threshold ? 255 : alpha; /... */ } 21:31 MTDiscord So the 255 case is basically a noop and the entire introduction of the ternary operator is dead code anyway. 21:31 Desour only in the first round 21:32 Desour afterwards, you change the color, but keep the alpha, and set the bitmap bit 21:32 MTDiscord Oh no, I might be wrong, it looks like the "skip transparent pixels" thing was changed. And I can't tell what the new thing means. 21:37 MTDiscord I can explain what the old thing was trying to do: For any pixel that had zero alpha (we assume the RGB information may be garbage, potentially arbitrarily chosen by image optimizers to optimize compression) we replace the RGB values by an estimate of a set of RGB values that will likely not clash with neighboring pixels if filtered; a linear alpha-weighted average of all neighboring valid pixels was chosen for this. This was added 21:37 MTDiscord to fix the "black borders when filtering is enabled" issue, and along with texture_min_size, made it possible to get decent filtered output even with a bit of upscaling involved. 21:38 MTDiscord (We've attempted to remove filtered upscaling, but it doesn't seem to be fully independent of filtered downscaling on all hardware, and loss of filtered downscaling can be much worse) 21:38 MTDiscord What I don't understand now is whether the current implementation is supposed to accomplish the same thing as the old one but just has a bunch of optimizations sprinkled around it, or whether the entire approach to the problem has changed in some way. 21:40 sfan5 see https://github.com/minetest/minetest/pull/11145 21:41 MTDiscord So "improve its' algorithm" is the only documentation about what the change was supposed to do or why 21:42 MTDiscord Well, there's this: "makes the filter fill all transparent pixels not just neighboring ones" but I don't see any justification for that...? 21:42 sfan5 do you see the comparison images in the table? 21:42 Desour btw, I'm currently trying to implement a different approach: first scale the image down to x1/2, x1/4, ..., 1x1, and then where the alpha is <= threshold, bilinear sample the smaller texture 21:43 MTDiscord I see comparison images but I don't know what I'm supposed to be looking for there. 21:44 MTDiscord I'm also not sure what "before" means. Before the algorithm change, or before clean_transparent was enabled? 21:44 sfan5 former 21:44 sfan5 mipmapping can end up sampling more than just the neighboring pixels which in the case of the grass texture caused that false white color when viewed from distance 21:45 MTDiscord oh, that's ... surprisingly nasty. 21:45 sfan5 Desour: is there a performance problem with it or do you just feel like doing it? 21:45 Desour sfan5: for performance 21:45 MTDiscord I had figured that mipmapping would apply transparency thresholding BEFORE sampling or something. 21:45 Desour #14322 21:45 ShadowBot https://github.com/minetest/minetest/issues/14322 -- Texture modifiers and imageCleanTransparent are too slow 21:46 MTDiscord The pixel values you're ending up with in the "after" picture look ... weird. But, given that the original texture is all "greenish", while I can point out features that look odd, none that I would expect to cause a major visual problem. 21:46 MTDiscord I'd be much more curious to see what it looks like when you throw a high contrast case at it, like red flower on green stem. 21:46 MTDiscord (those won't necessarily mip all that gracefully either, granted, regardless of what you do here, of course) 21:47 Desour btw. besides mipmapping (where we could instead generate the mipmaps ourselves), anisotropic filter can also sample transparent pixels more than one pixel away 21:48 MTDiscord I can confirm that imageCleanTransparent can be "slow". I originally designed that stuff to work with generally like 16px textures and such, and it works without issue in those cases for me ... but even though I don't use HD texture packs (since I know my machine's limitations) it can also affect things like the main menu header image, which can reasonably be quite large even for folks who wouldn't run HD textures. 21:50 MTDiscord I think I've said this already: We can probably optimize the time complexity of imageCleanTransparent via a BFS approach. I don't know whether this would directly translate to an improvement in practical time. Maybe. 21:50 MTDiscord For a long time I actually haven't trusted people to have the transparent cleaning filter enabled, so I actually applied it manually to all my textures in nodecore so transparent RGB is baked in. I worked around optimizers basically by setting alpha=1 for those border pixels to trick the optimizer into not discarding them. It sacrifices a bit of space efficiency but I've got plenty to spare, and it's nice having filtering behave 21:50 MTDiscord consistently. 21:51 sfan5 Warr1024: https://0x0.st/Xrek.png 21:51 MTDiscord I feel that we may be missing a lot of potential for parallelization, though. Esp. at startup we should be able to parallelize calculating texmods? 21:51 sfan5 mtg rose looks quite funny after the algorithm as you can see 21:52 MTDiscord I would probably have run clean transparent recursively myself, which sounds similar to what LMD is suggesting. I probably would do it the naive way, rescanning the image each time, since I suspect the complexity of managing a queue might cost too much in practical time. Though there might actually be a heuristic (e.g. how much of the image is still RGBless) that could make a hybrid approach better. 21:52 MTDiscord The MTG rose looks ... weird ... but also methinks quite reasonable. 21:53 MTDiscord I do wonder where the darkness around the stem comes from, whether that's in the original pixels, or just some algorithmic quirk. 21:53 MTDiscord You practically don't need a queue. You can use a new vector for every level, or you can have two vectors and swap & clear them if you're worried about the allocations. 21:53 MTDiscord So I guess I don't understand the approach now that the problem space has expanded, but the result looks fine to me so far. 21:53 sfan5 yes https://0x0.st/Xred.png 21:54 MTDiscord Also, whether we use a queue or vector, they should be pretty fine in terms of cache locality and memory usage. 21:54 MTDiscord Oh, interesting, the dark pixels on the outside edges seem to contribute a lot more "weight", probably because they have more transparent neighbors to propagate into. 21:55 MTDiscord I think I'll just implement the BFS for fun and we'll see. 21:55 MTDiscord Lol, I wish you found it as "fun" to solve problems with bigger impact than this 😄 21:56 MTDiscord I like optimization problems, esp. if they are constrained (like this one) 21:56 MTDiscord It is a bit like competitive programming but it's actually useful :D 21:56 sfan5 little known secret: figuring out how to render things efficiently (esp. with shaders) is also an optimization problem 21:57 Desour the impact would be very relevant. it's awful how when you walk around in cities or teleport to other players' bases, the client freezes for a split second 21:57 MTDiscord sfan5: yes, but with our architecture it isn't as constrained unfortunately :/ 21:57 MTDiscord Yeah, I get the feeling that MT engine dev attracts people who like to solve constrained optimization problems, and that's one of the reasons why we have a lot of things that work really well buried in a tangle of mess elsewhere. 21:57 MTDiscord I do have long term hopes / plans to fix bigger picture optimization issues but it's not always easy 21:58 MTDiscord For example Josiah and I tried slapping an R-tree index on objects and it didn't go nearly as well as I hoped 21:58 sfan5 Desour: do we mipmap entity textures? 21:58 MTDiscord as in, it didn't yield the performance improvements I had hoped for 21:58 Desour sfan5: idk 21:58 MTDiscord Desour: how confident are you that imageCleanTransparent is what's causing the freezes? In my experience the biggest client performance issues are usually related to the 3D scene, like slow particle and entity rendering, rather than to 2D image handling. 21:58 sfan5 ah so you just meant texmods in general 21:58 MTDiscord Especially since 2D image handling stuff tends to be once-and-then-done 21:59 Desour warr: 100%. I've done the profiling :P 21:59 MTDiscord Oh, is that in 14322? 21:59 MTDiscord I'll take a look 21:59 MTDiscord Ooooh, I see, it's signs and stuf 21:59 MTDiscord that makes more sense, yeah. 21:59 Desour sfan5: the sign entities each use their own texture. and we filter it regardless of whether a node or entity uses it 21:59 MTDiscord anyways imma have a very late dinner now 22:00 Desour Warr1024: also join times 22:00 Desour bb luatic 22:01 Mantar hey, are modchannels busted? I can't succesfully join a channel from the client mod, and the server joins, says the channel is writeable, but when I send a message, on_modchannel_message is never called 22:02 MTDiscord There are 2 cases that imageCleanTransparent is trying to solve: (1) linear filtering, and (2) mipmapping. For (1), the original single-pass algorithm would have worked fine. For the mipmapping case, actually, Desours idea of doing our own downsampling might work pretty well. We don't even need to do like bilinear stuff on the upscale, I suspect, since most cases where we're looking at >1 pixel away, we'd get "good enough" results 22:02 MTDiscord anyway. 22:02 sfan5 modchannels are unmaintained and essentially useless so might be that they are broken for years and nobody has noticed 22:02 Mantar yeah that's what I figured -- none of the client mods I found in the forums used them 22:02 Mantar tested back to 5.4.0 before I gave up 22:03 Mantar All I wanted was to have the client send a signal to alert the server when the Aux1 key was pressed, because doing it on the server side misses the keypress a lot 22:03 Desour there might also be a restriction flag for cpcsm channels 22:03 Mantar I cleared all restrictions, one of the first things I did :) 22:04 sfan5 Desour: then that'd be a very low hanging fruit, CAOs don't use any texture filtering 22:04 sfan5 (we should make the configurable tho) 22:04 sfan5 (same for nodes I guess) 22:05 sfan5 simply sed 's/getTextureForMesh/getTexture/' -i src/client/content_cao.cpp and free perf improvement 22:05 Mantar if modchannels had worked I could have told players complaining of lag to install the csm and things would be fine, but guess I just have to shrug and point upstream at you guys :D 22:06 Desour sfan5: haha didn't know. well great then 22:06 MTDiscord Both sides joined the proper channel? I was pretty confident modchannels worked 22:06 Desour but still, for faster join time, it's still a bottleneck 22:06 MTDiscord I think we should remove textures from the engine so we can decrease load times and improve performance 22:07 Mantar host joined successfully, client mod failed. I made it try to rejoin on failure every second, the result is just endless failure messages 22:07 Desour jordan4ibanez: users might not want it if we disable the textures feature though 22:07 Mantar but even on the host, when I send a message on the channel, on_modchannel_message just doesn't get called 22:07 Mantar I have it set to spit out any messages on any channel, and it's silent 22:08 sfan5 Mantar: open a bug 22:08 Mantar yeah, will do 22:08 Mantar I wanted to ask in case there was something undocumented I needed to do, in which case it'd be a different bug 22:18 Mantar Nevermind, LandarVargan got me to double check and discover that "enable_mod_channels = true" had been set in the client's minetest.conf, not the server's. Oops on my part, seems it works now! 22:42 MTDiscord Warr1024: I think there may be a third one: It's neat to have somewhat sensible colors filled in e.g. for opaque leaves, even if the PNG optimizer screwed up. Not sure if this always was an "explicit" goal though. 22:42 MTDiscord tbh I've "kind of" done this actually. 22:43 MTDiscord The problem turned out to be though that you don't actually want to completely smooth over the transparency information. 22:43 MTDiscord Case in point: NodeCore's leaves. For a while I allowed the cleanTransparent filter to interpolate the values. The problem was that NC trees looked WEIRD with opaque leaves. The whole hexagonal motif just disappeared. 22:44 MTDiscord So I found a compromise: I interpolated the RGB values between the leaves, and then halved them, so there's now a visible shadow with opaque leaves and the hexagon motif is conserved. 22:45 MTDiscord It could be possible to generate some sane infill colors, but it requires some artistic direction still. 22:46 MTDiscord (though tbh, 50% interpolated, 50% black is not a bad default to try for starters) 22:46 Desour AFAIK, leaves don't do imagecleantransparent before turning them opaque 22:47 Desour would actually not be a bad idea to change this 22:49 MTDiscord hmm i might've been confusing something here 22:49 MTDiscord In theory every texture should be getting cleaned at load time (leaves along with every other texture) and then when they're ^[noalpha'd, that should make those RGB values show up. If they don't, I don't know why that would have changed. 22:50 MTDiscord i'm thinking actually we probably aren't even really allowed to do this, since we could compromise modder's artistic choices for opaque leaves if they do their job properly 22:50 MTDiscord i mean we could leave (0, 0, 0, 0) as a special case but that might be a bit of a surprise then 22:50 sfan5 texture cleaning is the last step 22:50 MTDiscord It does seem like we every now and then run into a thing where the original intent was "do it consistently everywhere" and then somebody throws a "but I don't like the way it looks in this one case" at it, and then we end up with "it's only done in a few random specific places and we missed it everywhere else" 22:51 MTDiscord "texture cleaning is the last step" ... ah, okay, then that's the bug 😄 22:51 sfan5 IIRC modders have the ability to choose their own texture for the opaque leaves option without relying on any transparency magic 22:51 MTDiscord Can they really? What is there like a special_tiles thing that lets you substitute in a thing? ... well, it might be moot anyway, if few people actually know about it in practice. 22:52 MTDiscord Warr1024: I think the special tiles are only for "simple leaves" here :P 22:52 MTDiscord See "allfaces_optional" drawtype in the docs 22:52 MTDiscord All I know is that when I inherited Piranesi, it was using MTG leaf textures, which are atrocious when noalpha'd ... thankfully, because the geometry in that game is very limited, it was safe for me to turn allfaces_optional into just allfaces 22:54 sfan5 if this isn't possible we should make it so 22:56 MTDiscord It would be good to offer devs more artistic control, sure ... but we still want to have sane defaults and not force them to have to learn about an obscure feature. Often people can't be arsed to test how well their shit works with any configuration other than their own preferences, so we have the opportunity to not force the end-user to go complaining to modders right away. 22:57 MTDiscord Considering that we're already paying the cost for mipmapping, anyway, at least. 22:58 MTDiscord tbh when I fixed the old "black borders" thing ... the pure interpolated colors did feel a little too light sometimes. 22:58 MTDiscord Like maybe a slightly darkened but not completely black thing could have worked just fine for them too. 22:59 sfan5 texture cleaning is only enabled if you have a graphics option enabled that makes it useful, fyi 23:03 MTDiscord you can't turn it on manually anymore? 23:03 sfan5 nope 23:04 MTDiscord LMD does point out that "opaque leaves" might actually be one of those options now. 23:04 sfan5 https://github.com/minetest/minetest/blob/master/src/client/texturesource.cpp#L577-L580 it's not 23:06 MTDiscord I didn't mean "options that cause it to be enabled", I just meant "graphics options that could cause it to be useful" 23:06 sfan5 ah 23:06 sfan5 dunno if anyone has recently tested that interaction 23:07 MTDiscord ...which, is weird, because does this meant that opaque leaves + mipmapping might look different than opaque leaves without? 23:07 MTDiscord Hmm, I guess if we noalpha first and then we clean, then it wouldn't 🤔 23:07 Desour my new approach: https://github.com/Desour/minetest/tree/imagecleantransparent_different_alg (half finished. still needs bilinear filtering as opposed to nearest. and needs testing. but it's considerably faster) 23:09 MTDiscord What is remaining on #13825? 23:09 ShadowBot https://github.com/minetest/minetest/issues/13825 -- Add button_url[] and hypertext element to allow mods to open web pages by rubenwardy 23:10 MTDiscord Desour, it'd also be pretty cool to see some before and after images, like the grass, and especially that rose. 23:10 Desour yeah 23:10 rubenwardy Review 23:37 Desour I guess I'll have to implement the bilinear filter... https://github.com/minetest/minetest/assets/7613443/92c9dee4-155e-4142-a398-78a15314c161 23:45 Desour if you also want to test: https://codeberg.org/Desour/test_tmp/src/branch/imagecleantransparent_test