Minetest logo

IRC log for #minetest-dev, 2024-03-19

| Channels | #minetest-dev index | Today | | Google Search | Plaintext

All times shown according to UTC.

Time Nick Message
03:06 MTDiscord <jordan4ibanez> Minemine
03:06 MTDiscord <jordan4ibanez> Or Testtest
03:24 Lupercus joined #minetest-dev
04:00 MTDiscord joined #minetest-dev
04:12 fluxionary_ joined #minetest-dev
08:14 MTDiscord <herowl> Tbh a serious good name could be "Libre-/Open-" + "-voxel/-box"
08:21 MTDiscord <zmv7> Openbox is already taken
08:46 Warr1024 joined #minetest-dev
10:04 shangul joined #minetest-dev
10:11 [MTMatrix] <Zughy> There is an internal poll going on, discussing potential new names is pointless by now
11:20 MTDiscord <lemente> which domain names will be checked for availability/secured? (is that a public information?) will it be first come first served for the country top-level domains?
11:44 appguru joined #minetest-dev
11:53 rubenwardy .com .org .net
12:01 Lupercus joined #minetest-dev
13:08 proller joined #minetest-dev
13:24 shangul joined #minetest-dev
13:59 proller joined #minetest-dev
15:27 fluxionary joined #minetest-dev
15:44 fluxionary joined #minetest-dev
16:45 appguru joined #minetest-dev
18:43 sfan5 merging #14448, #14474 in 10m
18:43 ShadowBot https://github.com/minetest/minetest/issues/14448 -- Faster blit_with_alpha() by HybridDog
18:43 ShadowBot https://github.com/minetest/minetest/issues/14474 -- update Lua BitOp's stdint.h check for MSVC by goodusername123
18:57 Desour joined #minetest-dev
19:04 Krock > Server: Maximum lag peaked at 37.891 (steplen=0.05)
19:04 Krock hmm. valgrind isn't too happy about MineClone 2
19:05 MTDiscord <luatic> Krock: does asan not suffice to find the issue?
19:06 Krock It might. I just didn't want to recompile Minetest. Turns out that asan would've been faster by a long shot
19:07 sfan5 valgrind is kind of out of fashion
19:42 [MTMatrix] <Zughy> Krock: you might want to have a look at monitoring.aes.land: we have huge peaks every time a new user joins. We profiled everything we could and it doesn't looks like a mistake on our end. We've also started using a cache media server but it's not helping much. 🤷‍♀️
19:46 sfan5 have you profiled with perf?
20:00 imi joined #minetest-dev
20:43 [MTMatrix] <Zughy> no idea such command existed. We used the LuaJIT profiler mod written by TurkeyMcMac (https://content.minetest.net/packages/jwmhjwmh/jitprofiler/)
20:46 Desour profiling with the luajit profiler will only pofile what you're doing in lua, not anything that the engine does when it's not directly called from lua
20:50 sfan5 https://github.com/minetest/minetest/blob/master/doc/developing/misc.md <- linux tool, not a command
21:10 Desour why does imageCleanTransparent have a threshold for alpha that we set to 127 in case of applyfiltersformesh? isn't the ALPHA_CHANNEL_REF argument totally irrelevant?
21:16 MTDiscord <warr1024> The filter is supposed to compute the RGB values for transparent pixels (where we suspect correct RGB values have been discarded by image optimizers) using the RGB values only of opaque pixels (where we assume optimizers haven't mangled the RGB values)
21:17 Desour yes, I know. but why would we consider pixels with 127 >= alpha > 0 as transparent?
21:17 MTDiscord <warr1024> I know one reason we might: image blend mode = "clip" would.
21:17 MTDiscord <warr1024> as far as the original use, i.e. image optimizers, though ... I would expect a threshold of 0 or 1 (depending on whether it's > or >=) to make more sense.
21:17 Desour that's also what the code comment says, but it doesn't make sense in this context
21:18 MTDiscord <warr1024> I think tbh the threshold should be independent of image blend mode, i.e. it should be some value that always makes sense regardless of the context in which the image will eventually be used.  It's an image loading filter, not a texture use one.
21:19 sfan5 theory: if you have blend mode = clip you could in theory discard all colors in pixels with a < 128 as an optimization and it would work fine, so in theory imageCleanTransparent has to fix those cases up too
21:20 MTDiscord <warr1024> Hmm, we were already weighting the average by alpha, so it's not necessary to do complicated threshold shit.
21:20 Desour ahh, that would logically make sense
21:21 Desour but we have half-transparent nodes nowadays
21:21 sfan5 (i put way too many "theory" in that sentence)
21:21 MTDiscord <warr1024> u32 a = d.getAlpha() <= threshold ? 255 : d.getAlpha(); <-- lol, wtf is this?
21:22 Desour ^ that's because the newly set pixels have a 1 in the bitmap, but alpha < threshold
21:22 MTDiscord <warr1024> pretty sure that's supposed to be a 0, not a 255.  Which isn't really necessary anyway, since getAlpha() will be low enough for them anyway.
21:22 Desour (makes not much sense to weight them full 255 though)
21:23 MTDiscord <warr1024> Yeah, this code all looks wrong, and the only reason why it probably doesn't cause issues in practice was because the whole alpha-weighted averaging thing would only happen in rare cases anyway, and would probably be subtle in any event.
21:24 MTDiscord <warr1024> The idea was dealing with "high contrast corner" cases, like where a bright red flower meets a green stem, to try to avoid having a weird gray spot there.
21:26 MTDiscord <warr1024> That 255 looks like somebody tried to make a change to the code without actually understanding the idea behind it.
21:27 Desour ^ high contrast cases would actually best be dealt by conserving the histogram (= the set of occurring colors + commonness), to not make green+red=yellow spot in the middle
21:27 MTDiscord <warr1024> The inclusion of a "threshold" parameter was originally my mistake.
21:28 MTDiscord <warr1024> There's no "conserving the histogram" here, you've got 8 neighbor pixels and you need to pick a color to set the central one.  Just picking the "most central" color is probably the closest you could get, but doesn't necessarily entirely work, and it's hard to pick a "central" color when you have histograms on 3 separate dimensions.
21:29 MTDiscord <warr1024> (up to 8 neighbors; we don't wrap around on image edges)
21:31 MTDiscord <warr1024> Okay, actually, I was wrong about that 255 thing: it turns out it's a noop. The code is basically: if (alpha > threshold) { /* ... / weight = alpha <= threshold ? 255 : alpha; /... */ }
21:31 MTDiscord <warr1024> So the 255 case is basically a noop and the entire introduction of the ternary operator is dead code anyway.
21:31 Desour only in the first round
21:32 Desour afterwards, you change the color, but keep the alpha, and set the bitmap bit
21:32 MTDiscord <warr1024> Oh no, I might be wrong, it looks like the "skip transparent pixels" thing was changed.  And I can't tell what the new thing means.
21:37 MTDiscord <warr1024> I can explain what the old thing was trying to do: For any pixel that had zero alpha (we assume the RGB information may be garbage, potentially arbitrarily chosen by image optimizers to optimize compression) we replace the RGB values by an estimate of a set of RGB values that will likely not clash with neighboring pixels if filtered; a linear alpha-weighted average of all neighboring valid pixels was chosen for this.  This was added
21:37 MTDiscord to fix the "black borders when filtering is enabled" issue, and along with texture_min_size, made it possible to get decent filtered output even with a bit of upscaling involved.
21:38 MTDiscord <warr1024> (We've attempted to remove filtered upscaling, but it doesn't seem to be fully independent of filtered downscaling on all hardware, and loss of filtered downscaling can be much worse)
21:38 MTDiscord <warr1024> What I don't understand now is whether the current implementation is supposed to accomplish the same thing as the old one but just has a bunch of optimizations sprinkled around it, or whether the entire approach to the problem has changed in some way.
21:40 sfan5 see https://github.com/minetest/minetest/pull/11145
21:41 MTDiscord <warr1024> So "improve its' algorithm" is the only documentation about what the change was supposed to do or why
21:42 MTDiscord <warr1024> Well, there's this: "makes the filter fill all transparent pixels not just neighboring ones" but I don't see any justification for that...?
21:42 sfan5 do you see the comparison images in the table?
21:42 Desour btw, I'm currently trying to implement a different approach: first scale the image down to x1/2, x1/4, ..., 1x1, and then where the alpha is <= threshold, bilinear sample the smaller texture
21:43 MTDiscord <warr1024> I see comparison images but I don't know what I'm supposed to be looking for there.
21:44 MTDiscord <warr1024> I'm also not sure what "before" means.  Before the algorithm change, or before clean_transparent was enabled?
21:44 sfan5 former
21:44 sfan5 mipmapping can end up sampling more than just the neighboring pixels which in the case of the grass texture caused that false white color when viewed from distance
21:45 MTDiscord <warr1024> oh, that's ... surprisingly nasty.
21:45 sfan5 Desour: is there a performance problem with it or do you just feel like doing it?
21:45 Desour sfan5: for performance
21:45 MTDiscord <warr1024> I had figured that mipmapping would apply transparency thresholding BEFORE sampling or something.
21:45 Desour #14322
21:45 ShadowBot https://github.com/minetest/minetest/issues/14322 -- Texture modifiers and imageCleanTransparent are too slow
21:46 MTDiscord <warr1024> The pixel values you're ending up with in the "after" picture look ... weird.  But, given that the original texture is all "greenish", while I can point out features that look odd, none that I would expect to cause a major visual problem.
21:46 MTDiscord <warr1024> I'd be much more curious to see what it looks like when you throw a high contrast case at it, like red flower on green stem.
21:46 MTDiscord <warr1024> (those won't necessarily mip all that gracefully either, granted, regardless of what you do here, of course)
21:47 Desour btw. besides mipmapping (where we could instead generate the mipmaps ourselves), anisotropic filter can also sample transparent pixels more than one pixel away
21:48 MTDiscord <warr1024> I can confirm that imageCleanTransparent can be "slow".  I originally designed that stuff to work with generally like 16px textures and such, and it works without issue in those cases for me ... but even though I don't use HD texture packs (since I know my machine's limitations) it can also affect things like the main menu header image, which can reasonably be quite large even for folks who wouldn't run HD textures.
21:50 MTDiscord <luatic> I think I've said this already: We can probably optimize the time complexity of imageCleanTransparent via a BFS approach. I don't know whether this would directly translate to an improvement in practical time. Maybe.
21:50 MTDiscord <warr1024> For a long time I actually haven't trusted people to have the transparent cleaning filter enabled, so I actually applied it manually to all my textures in nodecore so transparent RGB is baked in.  I worked around optimizers basically by setting alpha=1 for those border pixels to trick the optimizer into not discarding them.  It sacrifices a bit of space efficiency but I've got plenty to spare, and it's nice having filtering behave
21:50 MTDiscord consistently.
21:51 sfan5 Warr1024: https://0x0.st/Xrek.png
21:51 MTDiscord <luatic> I feel that we may be missing a lot of potential for parallelization, though. Esp. at startup we should be able to parallelize calculating texmods?
21:51 sfan5 mtg rose looks quite funny after the algorithm as you can see
21:52 MTDiscord <warr1024> I would probably have run clean transparent recursively myself, which sounds similar to what LMD is suggesting.  I probably would do it the naive way, rescanning the image each time, since I suspect the complexity of managing a queue might cost too much in practical time.  Though there might actually be a heuristic (e.g. how much of the image is still RGBless) that could make a hybrid approach better.
21:52 MTDiscord <warr1024> The MTG rose looks ... weird ... but also methinks quite reasonable.
21:53 MTDiscord <warr1024> I do wonder where the darkness around the stem comes from, whether that's in the original pixels, or just some algorithmic quirk.
21:53 MTDiscord <luatic> You practically don't need a queue. You can use a new vector for every level, or you can have two vectors and swap & clear them if you're worried about the allocations.
21:53 MTDiscord <warr1024> So I guess I don't understand the approach now that the problem space has expanded, but the result looks fine to me so far.
21:53 sfan5 yes https://0x0.st/Xred.png
21:54 MTDiscord <luatic> Also, whether we use a queue or vector, they should be pretty fine in terms of cache locality and memory usage.
21:54 MTDiscord <warr1024> Oh, interesting, the dark pixels on the outside edges seem to contribute a lot more "weight", probably because they have more transparent neighbors to propagate into.
21:55 MTDiscord <luatic> I think I'll just implement the BFS for fun and we'll see.
21:55 MTDiscord <warr1024> Lol, I wish you found it as "fun" to solve problems with bigger impact than this 😄
21:56 MTDiscord <luatic> I like optimization problems, esp. if they are constrained (like this one)
21:56 MTDiscord <luatic> It is a bit like competitive programming but it's actually useful :D
21:56 sfan5 little known secret: figuring out how to render things efficiently (esp. with shaders) is also an optimization problem
21:57 Desour the impact would be very relevant. it's awful how when you walk around in cities or teleport to other players' bases, the client freezes for a split second
21:57 MTDiscord <luatic> sfan5: yes, but with our architecture it isn't as constrained unfortunately :/
21:57 MTDiscord <warr1024> Yeah, I get the feeling that MT engine dev attracts people who like to solve constrained optimization problems, and that's one of the reasons why we have a lot of things that work really well buried in a tangle of mess elsewhere.
21:57 MTDiscord <luatic> I do have long term hopes / plans to fix bigger picture optimization issues but it's not always easy
21:58 MTDiscord <luatic> For example Josiah and I tried slapping an R-tree index on objects and it didn't go nearly as well as I hoped
21:58 sfan5 Desour: do we mipmap entity textures?
21:58 MTDiscord <luatic> as in, it didn't yield the performance improvements I had hoped for
21:58 Desour sfan5: idk
21:58 MTDiscord <warr1024> Desour: how confident are you that imageCleanTransparent is what's causing the freezes?  In my experience the biggest client performance issues are usually related to the 3D scene, like slow particle and entity rendering, rather than to 2D image handling.
21:58 sfan5 ah so you just meant texmods in general
21:58 MTDiscord <warr1024> Especially since 2D image handling stuff tends to be once-and-then-done
21:59 Desour warr: 100%. I've done the profiling :P
21:59 MTDiscord <warr1024> Oh, is that in 14322?
21:59 MTDiscord <warr1024> I'll take a look
21:59 MTDiscord <warr1024> Ooooh, I see, it's signs and stuf
21:59 MTDiscord <warr1024> that makes more sense, yeah.
21:59 Desour sfan5: the sign entities each use their own texture. and we filter it regardless of whether a node or entity uses it
21:59 MTDiscord <luatic> anyways imma have a very late dinner now
22:00 Desour Warr1024: also join times
22:00 Desour bb luatic
22:01 Mantar hey, are modchannels busted? I can't succesfully join a channel from the client mod, and the server joins, says the channel is writeable, but when I send a message, on_modchannel_message is never called
22:02 MTDiscord <warr1024> There are 2 cases that imageCleanTransparent is trying to solve: (1) linear filtering, and (2) mipmapping.  For (1), the original single-pass algorithm would have worked fine.  For the mipmapping case, actually, Desours idea of doing our own downsampling might work pretty well.  We don't even need to do like bilinear stuff on the upscale, I suspect, since most cases where we're looking at >1 pixel away, we'd get "good enough" results
22:02 MTDiscord anyway.
22:02 sfan5 modchannels are unmaintained and essentially useless so might be that they are broken for years and nobody has noticed
22:02 Mantar yeah that's what I figured -- none of the client mods I found in the forums used them
22:02 Mantar tested back to 5.4.0 before I gave up
22:03 Mantar All I wanted was to have the client send a signal to alert the server when the Aux1 key was pressed, because doing it on the server side misses the keypress a lot
22:03 Desour there might also be a restriction flag for cpcsm channels
22:03 Mantar I cleared all restrictions, one of the first things I did :)
22:04 sfan5 Desour: then that'd be a very low hanging fruit, CAOs don't use any texture filtering
22:04 sfan5 (we should make the configurable tho)
22:04 sfan5 (same for nodes I guess)
22:05 sfan5 simply sed 's/getTextureForMesh/getTexture/' -i src/client/content_cao.cpp and free perf improvement
22:05 Mantar if modchannels had worked I could have told players complaining of lag to install the csm and things would be fine, but guess I just have to shrug and point upstream at you guys :D
22:06 Desour sfan5: haha didn't know. well great then
22:06 MTDiscord <landarvargan> Both sides joined the proper channel? I was pretty confident modchannels worked
22:06 Desour but still, for faster join time, it's still a bottleneck
22:06 MTDiscord <jordan4ibanez> I think we should remove textures from the engine so we can decrease load times and improve performance
22:07 Mantar host joined successfully, client mod failed. I made it try to rejoin on failure every second, the result is just endless failure messages
22:07 Desour jordan4ibanez: users might not want it if we disable the textures feature though
22:07 Mantar but even on the host, when I send a message on the channel, on_modchannel_message just doesn't get called
22:07 Mantar I have it set to spit out any messages on any channel, and it's silent
22:08 sfan5 Mantar: open a bug
22:08 Mantar yeah, will do
22:08 Mantar I wanted to ask in case there was something undocumented I needed to do, in which case it'd be a different bug
22:10 LandarVargan joined #minetest-dev
22:18 Mantar Nevermind, LandarVargan got me to double check and discover that "enable_mod_channels = true" had been set in the client's minetest.conf, not the server's. Oops on my part, seems it works now!
22:42 MTDiscord <luatic> Warr1024: I think there may be a third one: It's neat to have somewhat sensible colors filled in e.g. for opaque leaves, even if the PNG optimizer screwed up. Not sure if this always was an "explicit" goal though.
22:42 MTDiscord <warr1024> tbh I've "kind of" done this actually.
22:43 MTDiscord <warr1024> The problem turned out to be though that you don't actually want to completely smooth over the transparency information.
22:43 MTDiscord <warr1024> Case in point: NodeCore's leaves.  For a while I allowed the cleanTransparent filter to interpolate the values.  The problem was that NC trees looked WEIRD with opaque leaves. The whole hexagonal motif just disappeared.
22:44 MTDiscord <warr1024> So I found a compromise: I interpolated the RGB values between the leaves, and then halved them, so there's now a visible shadow with opaque leaves and the hexagon motif is conserved.
22:45 MTDiscord <warr1024> It could be possible to generate some sane infill colors, but it requires some artistic direction still.
22:46 MTDiscord <warr1024> (though tbh, 50% interpolated, 50% black is not a bad default to try for starters)
22:46 Desour AFAIK, leaves don't do imagecleantransparent before turning them opaque
22:47 Desour would actually not be a bad idea to change this
22:49 MTDiscord <luatic> hmm i might've been confusing something here
22:49 MTDiscord <warr1024> In theory every texture should be getting cleaned at load time (leaves along with every other texture) and then when they're ^[noalpha'd, that should make those RGB values show up.  If they don't, I don't know why that would have changed.
22:50 MTDiscord <luatic> i'm thinking actually we probably aren't even really allowed to do this, since we could compromise modder's artistic choices for opaque leaves if they do their job properly
22:50 MTDiscord <luatic> i mean we could leave (0, 0, 0, 0) as a special case but that might be a bit of a surprise then
22:50 sfan5 texture cleaning is the last step
22:50 MTDiscord <warr1024> It does seem like we every now and then run into a thing where the original intent was "do it consistently everywhere" and then somebody throws a "but I don't like the way it looks in this one case" at it, and then we end up with "it's only done in a few random specific places and we missed it everywhere else"
22:51 MTDiscord <warr1024> "texture cleaning is the last step" ... ah, okay, then that's the bug 😄
22:51 sfan5 IIRC modders have the ability to choose their own texture for the opaque leaves option without relying on any transparency magic
22:51 MTDiscord <warr1024> Can they really?  What is there like a special_tiles thing that lets you substitute in a thing? ... well, it might be moot anyway, if few people actually know about it in practice.
22:52 MTDiscord <luatic> Warr1024: I think the special tiles are only for "simple leaves" here :P
22:52 MTDiscord <luatic> See "allfaces_optional" drawtype in the docs
22:52 MTDiscord <warr1024> All I know is that when I inherited Piranesi, it was using MTG leaf textures, which are atrocious when noalpha'd ... thankfully, because the geometry in that game is very limited, it was safe for me to turn allfaces_optional into just allfaces
22:54 sfan5 if this isn't possible we should make it so
22:56 MTDiscord <warr1024> It would be good to offer devs more artistic control, sure ... but we still want to have sane defaults and not force them to have to learn about an obscure feature.  Often people can't be arsed to test how well their shit works with any configuration other than their own preferences, so we have the opportunity to not force the end-user to go complaining to modders right away.
22:57 proller joined #minetest-dev
22:57 MTDiscord <warr1024> Considering that we're already paying the cost for mipmapping, anyway, at least.
22:58 MTDiscord <warr1024> tbh when I fixed the old "black borders" thing ... the pure interpolated colors did feel a little too light sometimes.
22:58 MTDiscord <warr1024> Like maybe a slightly darkened but not completely black thing could have worked just fine for them too.
22:59 sfan5 texture cleaning is only enabled if you have a graphics option enabled that makes it useful, fyi
23:03 MTDiscord <warr1024> you can't turn it on manually anymore?
23:03 sfan5 nope
23:04 MTDiscord <warr1024> LMD does point out that "opaque leaves" might actually be one of those options now.
23:04 sfan5 https://github.com/minetest/minetest/blob/master/src/client/texturesource.cpp#L577-L580 it's not
23:06 MTDiscord <warr1024> I didn't mean "options that cause it to be enabled", I just meant "graphics options that could cause it to be useful"
23:06 sfan5 ah
23:06 sfan5 dunno if anyone has recently tested that interaction
23:07 MTDiscord <warr1024> ...which, is weird, because does this meant that opaque leaves + mipmapping might look different than opaque leaves without?
23:07 MTDiscord <warr1024> Hmm, I guess if we noalpha first and then we clean, then it wouldn't 🤔
23:07 Desour my new approach: https://github.com/Desour/minetest/tree/imagecleantransparent_different_alg (half finished. still needs bilinear filtering as opposed to nearest. and needs testing. but it's considerably faster)
23:09 MTDiscord <greenxenith> What is remaining on #13825?
23:09 ShadowBot https://github.com/minetest/minetest/issues/13825 -- Add button_url[] and hypertext element to allow mods to open web pages by rubenwardy
23:10 MTDiscord <warr1024> Desour, it'd also be pretty cool to see some before and after images, like the grass, and especially that rose.
23:10 Desour yeah
23:10 rubenwardy Review
23:33 panwolfram joined #minetest-dev
23:37 Desour I guess I'll have to implement the bilinear filter... https://github.com/minetest/minetest/assets/7613443/92c9dee4-155e-4142-a398-78a15314c161
23:39 appguru joined #minetest-dev
23:45 Desour if you also want to test: https://codeberg.org/Desour/test_tmp/src/branch/imagecleantransparent_test

| Channels | #minetest-dev index | Today | | Google Search | Plaintext