Time  Nick        Message
11:18 MTDiscord   <luatic> erle: Why do you think that all textures are loaded into graphics RAM? Texmods are generated on the CPU.
11:19 erle        luatic mostly because i am a dumbo and guess shit
11:19 erle        i think i know where my modifications lie now
11:20 erle        the mod name i chose was unicode_text, i may have uploaded an older folder
11:20 MTDiscord   <luatic> erle: [png has overhead, yes, but [combine typically has much more overhead except for the smallest of textures; a single pixel using combine is typically like 8-10 bytes (much more if you also have to use a [colorize / [multiply) at least and it isn't compressed at all
11:20 erle        i may have to rename the repo
11:21 MTDiscord   <luatic> [png has the advantage of being a texmod (consider proposing [tga and perhaps [jpg ;)), meaning it's tied to the entity whereas dynamic media will effectively have to be global
11:22 erle        luatic if you ignore the horrible consequences for validating texmods with a grammar and the hilarious ”write into a tempfile, then read it again”, my most major beef with [png is that it's probably one of the worst ones for base64 encoded inline textures.
11:22 erle        like imagine how you would design a texmod from first principles
11:22 erle        you'd probably arrive at something else than base64 encoded binaries that contain chunks with zlibbed pixels and checksums and all that
11:24 MTDiscord   <luatic> erle: which grammar?
11:24 MTDiscord   <luatic> tile.cpp is a mess of a shotgun parser
11:25 MTDiscord   <luatic> the only semi-clean texmod parser is the one in modlib, and that is slightly more strict than tile.cpp in some edge cases
11:26 erle        i think if i would design a texmod for inline textures it would probably look either a bit more like portable anymaps (if ease of use was highest priority) or, base64-encoded, optionally zlibbed, tga (because that has been smaller than PNG in every single one of my benchmarks up to 512×512 textures)
11:26 erle        luatic well you wrote modlib right?
11:26 MTDiscord   <luatic> erle: the temp file is in memory, so it's fine i think (though ideally the PNG reader API should not force us to provide a file handle)
11:26 MTDiscord   <luatic> erle: maybe
11:26 erle        luatic so how is the texmod validated?
11:27 erle        i know the shotgun parser thing (i may be the first person to have pointed it out)
11:27 erle        but having a base64 encoded binary means the following for your validator
11:27 erle        1. you need to validate and decode base64
11:27 MTDiscord   <luatic> yes
11:27 erle        2. you need to validate and decode the binary format inside
11:27 erle        so just like that [png (and everything similar to it) gives you a horrible grammar nightmare for the simple question of “is this input a valid texmod”
11:28 erle        i mean, maybe not everything else. portable anymaps have a header, then pixels. tga has a short header than pixels.
11:28 MTDiscord   <luatic> i think i properly did the base64 validation, but didn't do the PNG reading :P
11:28 erle        it's pretty much the chunk structure of png and the checksums
11:28 MTDiscord   <luatic> though modlib does have a strict PNG reader
11:28 MTDiscord   <luatic> but it makes more sense for my DSL to allow you to feed it raw bytes rather than requiring a modlib PNG
11:29 erle        so basically with [png the texmod grammar can no longer be parsed by a deterministic pushdown automaton (assuming that may have been possible before, who knows)
11:29 MTDiscord   <luatic> esp. since, just like minetest.encode_png, modlib's PNG writer doesn't write the most efficient PNGs :P
11:29 MTDiscord   <luatic> erle: pretty sure it could never be parsed by a PDA
11:29 erle        could be
11:29 MTDiscord   <luatic> exponential escaping says hi
11:30 erle        luatic did you try my tga.z benchmark cases? even the most efficient PNG encoder is unable to beat “write a short header, dump the pixels, then zip the thing up” on speed or filesize at the small (<512×512) texture sizes we have in minetest.
11:31 erle        if you wonder why i keep saying that, it is so people try it themselves and are amazed
11:32 erle        luatic the escaping thing is problematic from a parser POV how?
11:32 MinetestBot 02[git] 04Loppansson -> 03minetest/minetest_game: New water textures. 13d19d5d1 https://github.com/minetest/minetest_game/commit/d19d5d1baa7c2672adee8d8541958aa83b0c7f05 (152023-09-04T11:30:37Z)
11:32 MinetestBot 02[git] 04Loppansson -> 03minetest/minetest_game: Reduced image size. 132f53f2d https://github.com/minetest/minetest_game/commit/2f53f2d47f97be514a84d98483461d0afdd57aa7 (152023-09-04T11:30:37Z)
11:32 MinetestBot 02[git] 04Loppansson -> 03minetest/minetest_game: Transparent(er) River water 13b2a69e5 https://github.com/minetest/minetest_game/commit/b2a69e50033dd9f482106ecd004959e3566d6241 (152023-09-04T11:30:37Z)
11:32 MinetestBot 02[git] 04Loppansson -> 03minetest/minetest_game: Added my name to license.txt 137beac98 https://github.com/minetest/minetest_game/commit/7beac983c22035d2f9296d8e201b4da0a29871b6 (152023-09-04T11:30:37Z)
11:32 MinetestBot 02[git] (2 newer commits not shown)
11:32 erle        > Reduced image size
11:32 erle        lol ;)
11:32 erle        let's see
11:33 erle        -7.03 KB
11:33 erle        that's a lot of bytes
11:33 erle        was this the entire script of the bee movie in PNG metadata or what
11:36 MinetestBot 02[git] 04Loppansson -> 03minetest/minetest_game: Replace nonfree water textures 13bbc70c9 https://github.com/minetest/minetest_game/commit/bbc70c95ef0d211962060b842b8c614a7633633e (152023-09-04T11:34:39Z)
11:38 appguru     should have squashed, sorry for the commit spam (fixed via force push now)
11:40 erle        clean history fetishists, the bane of “git bisect” :P
11:47 erle        appguru given you pushed this texture optimization, are you aware that default_water_flowing_animated.png is 8086 bytes and optipng will reduce it to 8081 bytes, but you can use mtpaint to create a 4638 byte default_water_flowing_animated.tga?
11:48 appguru     erle: Interesting, but I'm not changing the file extension from PNG to TGA since that would break texture pack compat.
11:48 erle        appguru, yes, i was not suggesting that
11:48 erle        i am just pointing out how ridiculous the space savings are
11:49 appguru     Fun fact: You could use default_water_flowing_animated as texture string to be extension-agnostic, but I think this feature hasn't been documented so no one uses it :P
11:49 erle        appguru, go document it before some assclown removes it :P
11:50 appguru     wait no
11:50 appguru     it's half documented
11:50 appguru     "Textures are referred to by their complete name, or alternatively by stripping out the file extension"
11:50 erle        that's enough
11:50 appguru     but it doesn't say that it will try all file extension if you strip the file extension :P
11:51 erle        maybe i also can generate a 3364 byte png and a 3337 byte tga.z for default_water_flowing_animated, WHO KNOWS
11:51 appguru     lmao there's the following sentence in the docs:
11:51 appguru     Supported texture formats are PNG (.png), JPEG (.jpg), Bitmap (.bmp) and Targa (.tga). Since better alternatives exist, the latter two may be removed in the future.
11:51 erle        > better alternatives
11:51 erle        as if
11:51 erle        also if you remove tga you break every map created with mcl_maps or xmaps ever
11:51 erle        and at least one texture pack
11:52 erle        anyway, since approx. no one listens to me regarding texture optimizations, i'll leave it as an exercize to the reader why the current default_water_flowing_animated.png my not even be the optimal PNG. good luck!
11:53 erle        i'm way to deep in the topic of texture formats
11:54 erle        appguru, any idea how could i get people to optimize their textures much further? it seems everyone just uses optipng
11:54 erle        like, i'm sure it's a social thing
11:54 erle        like they don't *really* care about having the smallest textures or transfer size
11:56 erle        because otherwise they'd at least be using tga texture atlases for mostly-singleplayer mods (which, incidentally, a lot of other games seemed to do)
11:56 MTDiscord   <luatic> is it worth the effort?
11:56 erle        kinda, yeah
11:56 MTDiscord   <luatic> texture atlases are a mess
11:56 erle        yeah but just choosing a different file format or encoding parameter is easy
11:56 MTDiscord   <luatic> I think if anything optimizing this should be engine business
11:57 erle        it can't be
11:57 MTDiscord   <luatic> from a modder perspective, we just load a bunch of media, and the engine should decide how to transfer it
11:57 erle        also if it was, it would use zlib-compressed tga's like quake or so
11:57 erle        but i give you one example, every 16×16 image has exactly 256 pixels right?
11:57 MTDiscord   <luatic> yes
11:57 erle        so that's exactly enough for a single-byte-index colormap
11:57 MTDiscord   <luatic> yes
11:58 erle        so if your texture is 16×16 or less you should ALWAYS check if it is smaller as a colormapped image
11:58 erle        it may not be
11:58 MTDiscord   <luatic> typically it makes sense to use palettes for these
11:58 erle        palette/colormap same thing different name
11:58 MTDiscord   <luatic> yes
11:58 erle        yeah so how about no one really does this
11:58 MTDiscord   <luatic> erle: pretty sure MTG uses palettes in some textures
11:58 erle        MTG is a special case, take random modders
11:59 MTDiscord   <luatic> random modders typically don't care about optimizing at all i think
11:59 MTDiscord   <luatic> like, running optipng isn't even on the back of their mind
11:59 erle        oh, they care enough to let run optipng on 3000 textures and wait for an hour or so
11:59 erle        i mean some of them do
11:59 erle        i have seen it in mineclone2 lol
11:59 MTDiscord   <luatic> well, mineclone2 has some semi competent people
11:59 MTDiscord   <luatic> will optipng not try palettes?
12:00 erle        no idea, but optipng is the wrong thing entirely
12:00 erle        i have used optipng and zopflipng and pngcrush and so on
12:01 erle        in any case, i have made forum posts on this, like which circumstances should make you think
12:02 erle        but so far almost all replies were *mostly* nothing or “you suck and your results can not be true” or so
12:02 erle        as if it is impossible to improve upon some tooling
12:03 erle        i like the semi-competent, hehe
12:04 erle        some coworkers recently suggested i should submit patches to mcl2 that add new funny bugs, like dupes or coord exploits
12:04 erle        but not only do i think that is immoral
12:04 erle        i am also perfectly confident that mcl2 will be able to add these by themselves lol
12:05 erle        i mean, cora told me about how she cherry-picked some stuff recently and found a get-an-arbitrary-item exploit. it was so bad that she felt she had to report it lol
12:06 erle        this is *usually* stuff that projects catch in code reviews
12:06 erle        but not if the review is just “looks good to me, merge”
12:06 erle        so yeah, semi-competent
12:06 erle        :P
12:07 erle        luatic do you have mods that have a lot of textures btw?
12:07 appguru     erle: uh idk, probably not
12:07 appguru     i'm not really good at textures
12:08 erle        well, sometimes it's letters for signs or so
12:08 appguru     Spiraling Down (which my team and I worked on during the last game jam) has many textures since tem (great pixel artist) made them; I think I ran optipng on them but not much more?
12:08 erle        link to repo?
12:09 appguru     https://github.com/spiraling-down/game
12:09 erle        lol recommended specs what
12:09 erle        are you spamming particles or why
12:09 appguru     oh also my nametag mod has some textures but I don't think I optimized them at all - it's been years since I wrote that thing :P
12:09 appguru     erle: mapgen, we don't have multithreaded mapgen yet
12:09 erle        oh i see
12:14 erle        appguru, heay this might be the first mod/game i have seen where the textures are not easily improved filesize-wise by just saving as tga or converting to colormapped (or both)
12:16 erle        still, halo.png is 157 bytes and halo.tga.z is 116, so you see probably why i want tga.z to be a thing
12:17 erle        like it's not 30% savings filesize wise, but close
12:17 erle        just by zipping up the damn pixels instead of doing the optipng dance
12:18 erle        anyway appguru if you have an idea how to actually get the message out better about how to improve png or tga files, please tell. i doubt this *can* be an engine thing however, for the same reason that optipng is shit: you have a combinatorial explosion of encoding options and the only way to be sure is to try all of them – unless you have smarts.
12:19 erle        i mean it's not *shit*, but it's pretty stupid
12:19 appguru     erle: have you considered writing a tool?
12:19 appguru     like "optitest" which would automatically convert to tga if it's worth it
12:20 erle        i actually have done that already for mtg and mcl2 and you can too. but a) existing texture names are sacred b) you are the first person even asking for this c) it's like what, 10 lines of code? anyone who wanted that would make it in 20 minutes.
12:21 erle        i think the basic lesson for me is that “optimizing textures” is not some magic fairy dust that you can sprinkle on the project at the end, unless you have a lot of domain knowledge regarding texture file formats, because the tools either use heuristics that may not apply to your case or brute-force everything and you have 3000 textures
12:21 erle        (and wait hours)
12:22 erle        and also ppl are too trusting in their tools to recognize the failure case
12:23 erle        similarly how most users of make or cmake or ccache claim that they “never” encountered a miscompile (they never notice, you have a binary, it's probably okay, right? hehehe)
12:23 erle        zopflipng for example has this weird case in which it converts images to grayscale
12:23 erle        i'd not expect anyone who used it on hundreds of textures to actually verify that they are pixel-wise identical
12:23 erle        because people don't do tests
12:25 erle        anyway, it is not enough to be correct if people don't listen
12:26 erle        appguru given the (sometimes) massive savings, is there any way i *can* convert textures in a project from png to tga (or the other way around) without breaking much? i doubt it, right?
12:26 appguru     erle: this is very hacky and I'm not sure it works, but I think you can call a TGA .png and the engine will guzzle it
12:26 erle        basically, each texture is stuck with the file format it chose at the start
12:27 erle        lol
12:27 erle        appguru, tell me again when you have proved it works. if so, i'll have a bunch of space savings for textures to sell to every project under the moon lol
12:27 erle        :P
12:27 erle        (it's obv pay what you want, i.e. 0 monies)
12:32 erle        i renamed the renderer mod, seems gitea (?) redirects properly https://git.minetest.land/erlehmann/unicode_text
12:33 erle        now for the dofile thing
12:33 erle        appguru care to link me, again, a dofile definition that works inside and outside of minetest?
12:34 appguru     erle: just tested, seems to work, will look for the code which determines the file type (probably by magic bytes)
12:35 appguru     erle: something like function my_dofile(path) if minetest then path = minetest.get_modpath(minetest.get_current_modname()) .. "/" .. path end; return dofile(path) end
12:36 appguru     the ; after the end is technically not needed
12:36 appguru     (you would probably write this out in two or more lines anyways tho)
12:42 appguru     okay erle, here are my findings: Irrlicht does the "find the proper loader" thing. See `IImage *CNullDriver::createImageFromFile(io::IReadFile* file)` in `CNullDriver.cpp`: It first tries loading the file by its extension, then tries all loaders if that fails. Loaders provide "isALoadableFileFormat" for this.
12:42 erle        appguru guess it's file size optimization time then
12:43 appguru     For TGA, see `bool CImageLoaderTGA::isALoadableFileFormat(io::IReadFile* file) const` in `CImageLoaderTGA.cpp`: It just checks whether the footer is "TRUEVISION-XFILE."
12:43 erle        yeah, that's stupid
12:43 erle        it makes the TGA bigger than it should be
12:43 erle        by exactly that thing
12:43 erle        but you can never remove it
12:44 erle        e.g. imagemagick does not generate TGA files that have it, so you have to append it (but imagemagick has been written by 1000 monkeys on 1000 typewriters, they have a bunch of long-standing bugs that no other project has)
12:44 appguru     lol
12:44 appguru     the PNG loader just checks the signature btw
12:44 appguru     (pretty reasonable)
12:45 erle        i mean you have read about the random-flip in imagemagick lol
12:46 erle        appguru try this one on minetest_game and see how it goes https://mister-muffin.de/p/XF1_
12:46 erle        also if convert(1) flips the images upside-down, add -auto-orient to trigger the flipping bug twice
12:47 erle        you need to uncomment two lines actually
12:48 erle        and it's not the most optimized thing
12:48 erle        just seeing if RLE TGA is better than optimized png, which it is *not* for most textures
12:49 appguru     erle: https://mister-muffin.de/p/JFI0.txt
12:50 erle        yes, as i said
12:50 erle        this is a dumbo thing
12:51 erle        ideally you would want to check a) all color formats b) colormapped and non-colormapped c) RLE and non-RLE with the cavet that minetest will not load colormapped RLE TGA files (and don't think about adding it)
12:55 erle        bones_bottom e.g. seems to be better as an uncompressed TGA than as a compressed TGA or optimized PNG? o.0
12:56 erle        as i said, highly unintuitive
12:57 erle        appguru, now check this funny thing here: https://mister-muffin.de/p/mH-D.txt
12:58 erle        and this is not the optimized tga.z
12:58 erle        anyway
13:02 erle        appguru, btw, one more argument against “engine should do this automatically” is “apparently none of the people who advocate doing this automatically knows what exactly to do”, so i think it's kind of magical thinking
13:02 erle        like the people who say ”oh, i'll just let chatGPT write my job applications in the future”
13:02 erle        just [insert magic]
13:10 appguru     erle: probably not the engine, but this definitely belongs in a collection of "build tools" like translation update scripts etc.
13:10 appguru     erle: the entire point of "doing it automatically" is that i can let someone like you do the work for me, and then i just have to run the script :)
13:16 erle        appguru you made a PNG writer, right? what optimizations do you use there?
13:16 appguru     erle: none
13:16 erle        i mean the bar is on the floor basically, you can't be forse than minetests png encoding
13:16 erle        oh you can be exactly as bad lol
13:16 erle        why even write one then?
13:16 appguru     yes :)
13:16 erle        backwards compat?
13:16 appguru     yes
13:17 erle        i see
13:18 erle        appguru, who is mtg maintainer? i mean that person is probably the first litmus test for “do people care about the filesize thing enough”
13:19 appguru     I don't understand your question. Are you asking me to list active MTG maintainers?
13:20 ROllerozxa  appguru is the mtg maintainer now :P
13:20 erle        great lol
13:20 appguru     I'm a MTG maintainer yes (not "the" though lol, sfan5 is rather active and Krock occasionally provides input too).
13:20 erle        appguru would you be willing to a) convert the dozen files to tga if they are smaller b) work out the minimal tga.z engine change for the bigger savings?
13:21 appguru     erle: I don't think the savings are worth it so far without zlib compression?
13:22 erle        i have the impression that apart from security issues, the only times sfan5 listened to me was when i was being really fucking annoying, so i'd try to avoid doing that.
13:22 appguru     as for b): I'm not an engine dev, you'd have to discuss with the engine devs whether such a change would be accepted into the engine
13:22 erle        appguru, well what makes it worth it then if “lower filesize” is not? i mean you have seen the claim about the water thing
13:23 erle        and you may have noticed the script i provided did not make the water texture smaller
13:23 erle        could that be because it's not doing a lot of smart stuff? maybe :P
13:23 appguru     erle: the filesizes would have to be significantly lower for it to be worth consideration
13:24 erle        what is SIGNIFICANTLY
13:24 erle        to quote myslf
13:24 erle        > are you aware that default_water_flowing_animated.png is 8086 bytes and optipng will reduce it to 8081 bytes, but you can use mtpaint to create a 4638 byte default_water_flowing_animated.tga?
13:25 erle        if this is not enough for you, i'm done here :P
13:25 erle        and if you are not willing to give a hard boundary of byte savings, i'm done too. because that boundary could be put into a script.
13:25 erle        like “convert file if you save more than X bytes or Y% or so”
13:26 erle        but being non-commital sounds to me like “how about you fuck off”
13:26 erle        i am big on enthusiastic consent, everything that is not a “yes” is a “nopity nope”
13:28 erle        so yeah, i'm awaiting a clarification (if i don't get one, that's a “it was just a prank bro”)
13:31 appguru     erle: The problem is that we can't change the texture names, so we'd be forced to use the ugly "call a TGA .png" hack, and I don't see why we should be doing that unless there are tangible end user benefits.
13:32 erle        appguru look how about you say “no”
13:32 erle        there are only two benefits
13:32 erle        lower transfer size for single textures
13:32 erle        and lower transfer size for the entire mod zip
13:32 erle        (because png does not compress well further, whereas tga does)
13:32 appguru     ok no I don't think it pays off
13:32 appguru     are you happy now?
13:32 erle        yeah, but why didn't you say it in the beginning?
13:33 erle        so what's the downside actually, cache invalidation?
13:33 erle        or users that have an OS that cares about file extensions?
13:33 appguru     programs caring about file extensions are one downside, yes
13:34 erle        okay, but then “the filesizes would have to be significantly lower for it to be worth consideration” was not exactly the truth, right?
13:34 erle        look, i am not trying to prove you are a liar, i am trying to understand your thought process here
13:35 erle        and i perfectly well understand that there are other concerns trumping filesize. i mean, i only discovered this because i looked for “what's the *fastest* way to create a texture for dynamic use”
13:35 erle        i just got lucky :P
13:35 appguru     how was it not the truth? doing this change doesn't come for free, it has to be worth it, and even with some size reductions I just don't see it being worth it yet
13:35 appguru     so I was trying to see whether you could make a compelling case for the benefits of the size reductions
13:36 erle        well it can not be the thruth if there is *no* size reduction that actually goes over your threshold
13:36 appguru     erle: when did I say that?
13:36 erle        you did not give a threshold when i asked, which is about the same
13:36 erle        you can still give one lol
13:37 appguru     I want actual numbers. Something like "poor cell connections will be 100kbps and I can reduce total filesizes by 1 Mb which works out to 10s reduction in download time".
13:37 erle        yeah but you have to give the target firs
13:37 erle        t
13:37 appguru     (the numbers here are obviously made up, but you get the gist)
13:37 erle        i know how this goes when you give the numbers first, people move the goal posts
13:37 erle        if they don't actually care about the numbers
13:37 appguru     erle: well, there's one tiny problem, I can't seem to find a good distribution of internet speeds
13:37 erle        okay, i think i need to talk to someone else
13:38 erle        tangential, how fast is your internet?
13:38 appguru     fast, not the bottleneck when connecting to Minetest servers
13:38 appguru     fast enough*
13:38 erle        okay, so what does “significantly” mean to you then
13:39 erle        like you used that word, not me
13:42 erle        look, i don't get it. what i do get is that other concerns trump filesize, but i don't get why you'd want to talk about filesize at all. as long as you can't give a clear goal, i will interpret this as “no”, not because i think you are stringing me along, but because it has been useful for me to treat every unclear communication as a “nope, not interested“.
13:42 appguru     hard to give numbers, but let me give you a rough upper bound: if > 10% of Minetest users will have their media transfer time reduced by > 5s
13:42 erle        that's a funny goal, given i have no analytics and am morally opposed to even getting them
13:45 erle        appguru also you know there are other modifications that can reduce the media transfer time much more
13:45 erle        e.g. convert all textures to uncompressed TGA, then put them in a zip file and transmit that
13:45 erle        i'm not doing that
13:45 appguru     TBH it is possible that optimizing MTG textures isn't worth it at all anymore because they're already small enough as-is. Summing up PNG file sizes I get 262 kB, so unless you can shave off.. at least a hundred kB? it's probably not worth it
13:46 erle        you can't shave off 100kb of 262kb like that lolo
13:46 appguru     yeah
13:46 erle        but you can probably shave off some
13:46 appguru     how many?
13:47 erle        well, the test script i showed you which generated RLE TGA files saved between 1% and 40% depending on file, so that's a wide range. let me see if the .tga.z thing can do something, wait a moment.
13:50 erle        appguru, have a quick way to visualize a bunch of percentages per files?
13:50 erle        like i have a bunch of percentages, i want a plot
13:50 appguru     you could dump it in a CSV and then use libre office
13:51 appguru     or if you want a more shell-ey workflow you could use gnuplot
13:51 appguru     or matplotlib if you prefer python
13:51 appguru     anyways the savings from the script you sent me weren't particularly large
13:51 erle        yes but
13:52 erle        you only say that because you seem to not believe in proof-of-concepts
13:52 appguru     ?
13:52 erle        well, this script was the dumbest thing i had that proves my assertion
13:52 appguru     it proves a very weak assertion
13:52 erle        no, the assertion is “optipng is not the end of the road here”
13:53 appguru     if we're talking about pure .tga vs .tga.z here, it clearly isn't worth it
13:53 erle        i am not spending hours writing a full-fledged script to prove an argument like this
13:53 erle        that is surprising
13:53 appguru     damn i phrased that wrong
13:53 appguru     I just wanted to clarify that we're talking about converting PNGs to .tga for space savings, not converting PNGs to .tga.z, right?
13:54 erle        both actually
13:54 appguru     and for that you sent me a script which demonstrated that for a few files it could shave off a few more bytes
13:54 erle        yeah so
13:54 erle        i proved that even the dumbest way possible of doing it can shave off some bytes
13:55 erle        i mean originally i just used mtpaint to resave the file lol
13:55 erle        proving that doing a very dumb thing can beat a smarter thing (optipng) is a strong assertion, not a weak one
13:55 erle        because you can still get smarter from the dumb thing
13:55 appguru     given png overhead.. no, not really
13:55 erle        you prob can't make the smart thing much smarter
13:56 erle        like basically i am comparing a lower bound of a new solution to an upper bound of an existing one
13:56 erle        those are not the same
13:56 erle        an analogy: in IT security, if you manage to leak one byte from encrypted data, that's usually where people get worried
13:56 erle        and rightly so
13:56 erle        but non-IT-security people usually are like “what's the deal”
13:57 erle        because they focus on it being a single byte
13:57 erle        anyway, i'm trying something that may or may not please you, please stand by for results
13:59 erle        btw, space savings are between “basically cut filesize in half” and 1%, it's a wide range
14:00 erle        obv you won't get much cut-in-half
14:00 appguru     Looking at the min-max range is often misleading. The distribution between the two matters very much.
14:00 erle        which is why i want a plot lol
14:00 erle        the 1% savings are also misleading because that is where my cutoff is
14:00 erle        obv i am not analyzing stuff that gets larger, because why should i
14:01 erle        appguru, when you use optipng, how much space does it usually save?
14:01 erle        appguru, i think that's a good thing to know because users do actually create giant shitballs of PRs just to crush the textures using optipng
14:01 appguru     The thing is, your "lower bound" still tells me nothing about the projected upper bound (why should it be much higher?), and so far it mostly looks like shaving off PNG overhead (or can you demonstrate something like constant factor vs. constant shavings?), and that is just looking at the textures it improves which are so far only a small subset of all textures.
14:02 erle        why should i advocate changing textures where i do not have an improvement?
14:02 appguru     erle: I haven't used optipng in a while, I don't know.
14:02 erle        obviously i am cutting out the case where we don't gain anything
14:02 erle        it only has downsides
14:02 appguru     erle: you shouldn't, it's about how much it matters in the bigger picture
14:02 appguru     and so far all I see is microoptimization
14:03 appguru     (converting PNGs to .tga, that is)
14:03 erle        i am not debating that it is a microoptimization, every filesize thing is
14:03 appguru     heck, MTG with its ~250k of PNGs is probably the wrong project to target
14:03 celeron55   one should remember it's also polite to use png, as that's what most people's image editors can use these days. tga is essentially less open source because it is less accessible
14:03 appguru     you should rather be looking at projects which (1) typically don't optimize textures at all and (2) have many textures
14:04 erle        celeron55 i mainly want tga.z because of my unicode rendering code, have you looked at it?
14:04 appguru     (I'm thinking something like Dreambuilder for (2))
14:05 erle        celeron55 i have not found any image editors that can't open tga, but i am on linux. anyways, i think focusing on filesize for existing projects is probably a lost cause, hence the suggestion of appguru to make a script is one i will not follow, unless i need it myself.
14:06 erle        because i am pretty sure there is *no* filesize that trumps “some (windows?) user will be confused by the file having the wrong extension”
14:06 erle        whereas in the case of generated textures i make, well, no one is touching them with an image editor at all
14:06 erle        so i can focus on speed of rendering and transfer size
14:08 erle        appguru ok i have worked it out and considering *all* files (even the ones that get bigger) i can not promise more than 3% savings. it is irrelevant of course, if i would produce 97% savings (impossible), this would still not trump other arguments. but it is a good lower bound for “how much better can i get than optipng if i'm trying really fucking stupid things”.
14:09 erle        appguru, so given that other considerations always win in the end, did you just not have made your mind up or was it my focus on filesize that led you to vague statements regarding it?
14:10 erle        celeron55, in case you have too much time, i'd highly appreciate your opinion on what is missing for, e.g. finnish to render correct (if something is missing) and how you like the API https://git.minetest.land/erlehmann/unicode_text
14:21 erle        appguru, btw, for the upper bound of possible space savings, just for completeness: consider a 16×16 image with 50% transparent purple. the most optimized png is 96 bytes, tga_encoder can write a TGA that is 54 bytes, compressing it using zlib-flate yields 45 bytes. so i can say with confidence that regardless of what you do, you'll never get much more than 50% for a uniform colorized RGBA image.
14:21 erle        appguru out of curiosity, how big is the PNG that your modlib thing produces?
14:22 erle        celeron55 btw regarding design decisions: given that irrlicht can do quake-style WAD files for filesystem access, why are mods unpacked at all? is this a performance consideration, bc seeking is slow?
14:23 celeron55   if you're thinking there's any design involved, you're thinking too far
14:23 celeron55   it's nice and accessible, that's all
14:23 erle        hehe
14:24 erle        “historically grown” instead of design hehe
14:24 erle        like evolution, poke here and there and optimize greedily :D
14:25 erle        celeron55 btw were you at ccc camp or are you planning to come to ccc congress?
14:26 celeron55   no. i was in thereabouts almost for the entirety of june, that'll last me for years
14:26 erle        what does “in thereabouts” mean?
14:27 celeron55   central europe
14:27 erle        ah
14:27 erle        too warm? too bright? :D
14:27 erle        an acquiantance met some ppl from finland and asked them for licorice and got something that he said tasted like tar
14:27 erle        maybe you know what it could be?
14:28 celeron55   too much people, too much traffic, too expensive accomodations
14:29 celeron55   and too far away
14:30 erle        i see
14:31 erle        it reminds me of a joke, everyone being happy when the mandatory 2 meters of social distancing ended – so they could go back to the normal 5 meters
14:31 celeron55   if it didn't cost anything, it would be worth it, but the cost kind of ruins it for me
14:31 erle        i don't like crowds
14:31 celeron55   and, i don't like flying. that's one thing also
14:31 erle        i like trains
14:32 erle        recently someone on irc from a country without trains asked me what trains feel like
14:32 erle        i answered a train is like a very long slightly vibrating room
14:34 celeron55   my answer would be, like a huge bus on rails
14:34 erle        busses vibrate more
14:35 celeron55   well, that's where the "huge" and "on rails" come in. but yeah, your description also makes sense
14:36 erle        i was in an ICE train
14:36 erle        so it was not as bus-like as it could be
14:36 celeron55   "like a ship, but smaller and instead of the motion being all wavey and whatnot, it's defined by the rails"
14:36 erle        oh, that's good too
14:36 erle        ships have noticeable acceleration after all
14:36 erle        that is more than the short thing in buses or trams
14:37 erle        like over longer time, like trains!
14:38 celeron55   it would be interesting to look at the acceleration/vibration characteristics of different transportation methods on some kind of graphs
14:40 erle        i'm sure rollercoaster tycoon has you covered hehehehe
14:40 erle        failing that, some openttd nerds maybe?
14:40 celeron55   one way i'd describe it is this way: in a train, your acceleration is very strictly defined on two axis, one comes from the turns in the rails, and the other comes from the train speeding up or slowing down
14:42 celeron55   in a bus, the complex road surface bounces you around in a haphazard way, plus the engine is close to you so you feel and hear it also
14:43 erle        indeed
14:43 celeron55   a ship feels like a train but on top of it the waves move you haphazardly but slowly
14:43 erle        e buses are better though
14:47 appguru     "so given that other considerations always win in the end, did you just not have made your mind up" - erle: I hadn't made up my mind. That's why I was discussing it with you rather than flat out saying no.
14:47 celeron55   in a bus, the driving style of the driver and the road conditions can also vary a lot and you will feel it. in a train or a bus, the human in charge doesn't have a lot of say in the moment-to-moment behavior of the thing
14:49 celeron55   a bus is a good reference point because almost everyone has been on one, even the 'muricans and even people in less developed countries
15:10 erle        appguru, ah, that explains it
15:10 erle        appguru good that we discussed it then, because it means i will not make the tool
15:34 celeron55   erle: i believe if your unicode library can render german properly using generic methods, then it can render finnish properly. there's no fundamental difference
15:34 celeron55   it's all just latin with some accents
15:35 erle        HEIZÖLRÜCKSTOẞABDÄMPFUNG!
15:35 celeron55   ah well ẞ actually makes german more involved as it's a character whose unaccented form cannot be found in ascii
15:36 erle        celeron55, see for yourself, i did find a finnish word: https://mister-muffin.de/p/XfT_.png
15:36 rubywarden  <erle> clean history fetishists, the bane of “git bisect” :P
15:36 rubywarden  actually, the saviour of git bisect. It's incredibly painful to use git bisect if the git history is messy, with broken commits and such. So it's a good idea to make sure any commits that go into the main branch are done properly
15:36 erle        ẞ is not an accented character
15:36 erle        rubywarden the correct way to do it though is to “always make clean commits”. this can involve squashing, but the vast majority of projects i have seen that squash everything lose clean intermediate states.
15:37 celeron55   i mean the finnish language only uses characters whose unaccented form can be found in ascii
15:37 erle        basically i don't want to end up bisecting and finding a 3000 lines commit
15:37 erle        celeron55, so what does the finnish word in the example mean lol
15:38 celeron55   that's a curious word
15:38 erle        i looked for words with many characters
15:38 erle        or phrases
15:39 erle        probably most are nonsense
15:39 erle        as you can see, i can render accented characters in NFC and NFD, it only looks slighty different
15:40 erle        i wonder if i should move the rendering of accents a bit so you can have proper zalgo text heheheheh
15:42 rubywarden  erle: I also oppose hard "squash everything" rules - the key thing is to be intentional with your commits so they make sense as steps
15:43 celeron55   it doesn't necessarily mean anything because the relation of words to each other in a compound word is not intrinsically defined. if you just take the words one by one and translate them "kind of in context", you get something like törkylempijävongahdus = filth lover whimper. but that doesn't really mean anything, the relation of the words to each other is unclear without external context
15:43 erle        rubenwardy indeed!
15:43 erle        celeron55, well “heizölrückstoßabdämpfung” also means nothing (i think), it's literally “heating oil recoil dampener”
15:44 celeron55   (external context can of course be replaced by common usage. but that's pretty far from common usage)
15:44 erle        it just has äoüß
15:44 erle        which is why i chose it
15:45 erle        rubenwardy do you maybe have any comments on the API of my text rendering library thingie? i want it to have a stable API when i put it on cdb, so i am collecting feedback.
15:47 celeron55   (it seems to be a good word as a finnish pronunciation challenge though. altough it's missing double wovels and double consonants which are difficult to non native speakers also)
15:47 erle        also, is there a downside to this thing i do where i use an argument table for optional arguments for easy extensibility? i have not seen many people do it, they rather arbitrarily add some arguments usually.
15:47 erle        in lua i mean
15:47 erle        like if i have foo({ bar = baz, ham = spam}), i think it's much better than foo(bar=baz, ham=spam)
15:47 erle        for optional arguments
15:48 erle        because in the second example you'd have to set bar explicitly to nil or so if you don't want to use it
15:49 celeron55   it is appropriate, in lua, to use a table as the argument if you either have now or are preparing to have in the future a large number of optional arguments
15:50 erle        yeah, but why are people usually not doing it for that purpose?
15:50 erle        like, are they just bad at designing extensible APIs or is there a downside
15:50 celeron55   because usually you don't think far enough to be able to decide
15:51 erle        i take this as a veiled compliment for my far-sight hahahaha
15:51 celeron55   a good rule of thumb can also be to use something else than a table or object as the first argument, so that you could in the future replace all the arguments with a single table and detect that just besed on the first parameter being a table instead of what it was in the non-table version of the function
15:52 celeron55   well, it's also a prime place for overdesigning something
15:52 erle        well i do it differently
15:53 erle        for the text rendering function, i have one mandatory argument that will never change, then a table for all others
15:53 erle        the one mandatory argument is the text being rendered ofc
15:53 erle        i'd think switching on parameter type creates mental load and more unnecessary code
15:53 celeron55   one could also consider it bad api design if your function arguments are so complex that you need to use a table to opt out from some of them without it becoming a mess
15:54 erle        in general i don't like spaghetti data types
15:54 erle        you tell me? some people want to set the tab width, some not. some people want to set a colormap, some not.
15:54 celeron55   of course in your case it may be appropriate
15:54 celeron55   it's essentially a stylesheet or whatever
15:55 erle        pretty sure stuff like that *should* be optional, or the “why do i have to care about scanline order?” (because otherwise your rendering might be upside down) people are going to whine about having to provide mandatory parameters
15:55 erle        kinda yeah
15:55 erle        i also have really stupid kerning
15:55 erle        that just shaves off rows of empty pixels of letters hehe
15:55 erle        i mean columns
15:56 celeron55   also meaning, the insides of the table are not necessarily function arguments. the table is essentially an object with a named purpose, a type of its own, altough it is not enforced by lua itself (like almost nothing is)
15:56 erle        any common font rendering things like tabulator width or kerning you can think of?
15:56 erle        indeed
15:56 celeron55   you will find this way of designing things in the minetest api
15:56 MTDiscord   <luatic> complex table arguments are a hint that maybe this should be something like an "object"
15:56 erle        the table is, conceptually, of function parameter type
15:56 erle        lua has no objects you know that as well as i do
15:57 erle        it's tables all the way down
15:57 MTDiscord   <luatic> it has objects, you just have to implement them yourself :-)
15:57 erle        and metatables
15:57 erle        well i did lol
15:57 celeron55   lua is the hammer with the replaceable head. so it's essentially just the stick
15:57 erle        i put a table in my table so i could table while i table
15:57 erle        celeron55 that's a great saying
15:57 celeron55   or, handle, to put it better
15:58 erle        instructions unclear: i now think that lua is the stick from nodecore hehe
15:59 erle        the one that can be used to make fire
16:01 MinetestBot 02[git] 04Panquesito7 -> 03minetest/minetest_game: Update creative Spanish translation 13a132872 https://github.com/minetest/minetest_game/commit/a132872ad86ae065d1975af9deb93cf1a29bd002 (152023-09-04T16:01:14Z)
19:42 erle        i submitted my content to the blog! https://github.com/minetest/blog/issues/129