Time Nick Message 00:11 erle maybe there should be a “i found this cool new texture file format on hacker news” FAQ 00:11 erle with the answer being no 00:19 erle pgimeno decoding speed is a *major* reason why some games have giant assets. but those games usually deal with graphics bigger than 16×16 and have sprite sheets. meanwhile, minetest has a lot of very small files. i have benchmarked this quite a lot and even on a potato computer you are not going to notice any speedup just by using a different image loader. minetest does a ridiculous amount of useless busywork for texture 00:19 erle decoding, but in the big picture, about *none* of it matters. 00:27 erle pgimeno, what matters is: a) if a format is in use b) tooling support c) format features: RGB / RGBA / palette / alpha channel (alpha bit is not enough) / compression d) how easy you can emit / read / rewrite it 00:45 erle pgimeno i have both read the QOI spec and i think you are better off making paletted TGA work. additionally, QOI cheats on filesizes: i have tried 3 of their PNG images in the “comparison” archive and they could easily be shrinked by 25% to 30% 00:45 erle honestly, if some poser comes up with a new file format and you can outperform it in filesize & implementation size & decoding speed with paletted TGA, it seems absolutely useless 00:46 erle (for minetest purposes) 00:48 pgimeno I believe they are comparing against a PNG size as compressed by a specific popular single-file library, not against an optimized PNG 00:49 pgimeno and yes, for small images it's probably useless for Minetest 01:00 erle pgimeno you did get it from hacker news or reddit though, did yuo? 01:00 pgimeno no, from love2d forums as I said 01:00 erle ah ok 01:01 pgimeno I don't read either of those unless they appear in search 01:01 erle sorry then 01:02 erle pgimeno, for the future, if you are fascinated by file formats that are easy to implement and work with, read the TGA spec. it doesn't get simpler than that while still being useful. and also then you will be vaccinated against “the newest cool thing”. QOI is clever, but needlessly clever. 01:03 pgimeno ... 01:03 erle btw, the reason why i mention palettes so often is because 16×16 is 256 01:03 pgimeno I advocated for TGA support right here because I know how simple and straightforward it is 01:03 erle oh okay sorry 01:04 erle i seem to be needlessly mean today. i hope you can forgive me. 01:04 erle i promise to be better 01:04 erle pgimeno, i think the only format i want is some vector format. do you know any simple to implement vector formats? SVG is not simple. 01:04 erle i mean something that can be hooked up to the engine to get non-ugly menu icons 01:05 erle pgimeno, i apologize 01:06 pgimeno can't think of any simple vector format besides dxf, which is probably not suitable 01:27 MTDiscord A somewhat less dumb option might be to offer the ability to receive PNG over the network, and write TGA format as like *.png.tga to the local cache so they can be located based on the original filename. If we have readers for these files, then it's not a stretch to consider that the image libraries may have writers that make conversion an option. 01:31 erle Warr1024 this makes very little sense, why would you do that? 01:31 erle *little sense to me 01:33 erle if it's to optimize loading times by having uncompressed assets, i already mentioned that you are not getting any noticeable speedup from that 01:34 erle only thing i have not profiled so far is having texture atlases 01:35 erle (and they make caching a sad story ig) 02:35 fluxionary does anyone have insight into https://github.com/minetest/minetest/issues/11877 ? 03:14 Pexin !title 03:14 ShadowBot New player causes: duplicate key value violates unique constraint "auth_name_key" · Issue #11877 · minetest/minetest · GitHub 07:11 MTDiscord https://github.com/minetest/minetest/issues/12134 this issue should probably be closed 07:12 MTDiscord https://github.com/minetest/minetest/issues/12056 this one as well, the issue is one that has already been fixed but they are on an older version, see "snap version works" 16:56 erle pgimeno the big issue with vector files as texture formats is how they are setup right now. basically, everything that is decoded … is decoded into an R8G8B8A8 pixel buffer. so you need to know image size at decoding time. 16:58 erle it might be easier to get crisp menu files by making sure they are only scaled at defined intervals, like openttd does. 16:58 erle i.e. normal icon, 2x icon, 4x icon. 16:58 erle or 1/2 icon etc. 16:59 erle that would neatly avoid the distortion effects you get right now 17:01 MTDiscord speaking about the main menu game textures, I still can't figure out why it is so picky about header sizes and just doesn't show it if it's not up to liking 17:15 rubenwardy Updated #12185 , ready for review 17:15 ShadowBot https://github.com/minetest/minetest/issues/12185 -- Add login/register dialog to separate register and login by rubenwardy 17:54 rubenwardy Who was it looking into glTF? Did that go anywhere 17:57 MTDiscord @ExeVirus 18:26 rubenwardy trivial PR #11876 18:26 ShadowBot https://github.com/minetest/minetest/issues/11876 -- Update mods_here.txt to mention installing mods via CDB by qwerty123a2 18:35 rubenwardy any everyone please go through this list before the next meeting, so we don't have to spend so long: https://github.com/minetest/minetest/pulls?q=is%3Apr+is%3Aopen+sort%3Aupdated-desc+label%3A%22Roadmap%3A+Needs+approval%22 18:39 rubenwardy (I've also not finished going through) 18:43 MTDiscord @rubenwardy , yeah I was looking at it, though I have not gotten around to making it. Totally possible, just requires a JSON dependency, and some conversion coding 18:43 rubenwardy we have JSON? 18:43 rubenwardy in irrlicht I guess? 18:43 MTDiscord Well, this file reader would be in the minetest codebase, not irrlichtmt 18:44 MTDiscord As should all extension readers (webp, avif, etc) 18:46 erle exe_virus why though? 18:46 erle also again, webp is useless 18:57 MTDiscord Webp is the smallest lossless format on the market that is mature 18:57 MTDiscord The file reader interface was designed to be outside of irrlicht. Just trying to be consistent with it's design philosophy 18:58 MTDiscord Webp2 is far better, but not stable yet 18:58 MTDiscord Avif lossless is worse than webp. We care about lossless for the most part here in MT 18:58 ROllerozxa don't really understand the hype behind webp, either lossless or lossy webp 18:58 MTDiscord Just better compression than png 18:59 MTDiscord And single file format, for purests 18:59 MTDiscord Webp2 though, is gunning for best overall, they're not to avif lossy levels yet, but they're getting there 18:59 MTDiscord unfortunately webp suffers from being browser tech 18:59 ROllerozxa so any lossless webp image is going to be smaller than its fully optimised png format? 18:59 MTDiscord good luck finding an editor for it that uses the old format in a year 19:00 MTDiscord Webp? It's stable. Already in gimp. 19:00 MTDiscord also why care about saving something like 100 bytes 19:00 MTDiscord Almost every lossless webp is smaller than optimized png, we tested this against nodecore and dream builder. About 20% smaller 19:01 MTDiscord So for nodecore, it was about 20-50Kb irrc, for dream builder.... A lot more 19:01 MTDiscord maximum PNG optimisation is still better considering it's tooling is practically everywhere 19:01 MTDiscord Why not both? 19:01 rubenwardy remember that our images are mostly 16x16. png is super well supported 19:01 rubenwardy maintenance 19:02 MTDiscord Toss a library in, done? 19:02 MTDiscord yes because like web browsers are going to agree on that standard in a year 19:02 MTDiscord They have for 3-5 years now? 19:02 MTDiscord it just means it's maintenance is slaved towhatever chromium developers are up to at the behest of google 19:03 MTDiscord Well, webp is in maintenance, and wont be changing. 19:03 MTDiscord Also, 16x16 is exactly the biggest reason to use webp2 later, because the headers are non existent, and we're talking extremely small textures, like 50% of png sizes 19:04 MTDiscord saving less than 1k bytes for a massive dependancy IS NOT WORTH IT 19:04 MTDiscord Single header? 19:04 MTDiscord One dep 19:04 MTDiscord the dependancy is probably larger than the image it's trying to deal with 19:04 MTDiscord Not over the wire 19:04 MTDiscord and anyways, we don't need more random shit in MT 19:04 MTDiscord It's an engine, we should support more modern formats over time. 19:04 erle look, if you want a file size optimization, make paletted TGA work with RLE compression. that gets you under PNG size for a lot of small files. but if i thought it was even worth it i would have done it by now. there are no big savings here. 19:05 erle when you reach 100 or 200 bytes per file, nothing matters much 19:05 MTDiscord That's what webp and avif, and other modern formats do, with better support than TGA 19:05 MTDiscord modern does not mean better 19:05 erle file size is not the problem. the overhead is elsewhere. 19:05 MTDiscord Arguing over individual image file formats is just silly. It's the 21st century, and image format support should be the responsibility of your image loading library. 19:06 MTDiscord So include said libraries? 19:06 erle given that the existing file formats can not be rendered right, more complex file formats mean more problems. 19:06 MTDiscord They all get expanded to RBG anyways 19:07 erle look, what do you want? small files? 19:07 MTDiscord Yep. Though gltf is a different story 19:07 MTDiscord I like small files and I cannot lie. 19:07 erle i mean look at the QOI mess 19:08 MTDiscord Quality of life? 19:08 MTDiscord and bmp lol 19:08 erle quite okay image format. it's “modern” in the sense that it uses a bunch of tricks. and it can “outperform PNG” in a very very narrow use case. 19:08 erle yeah BMP is the stupid end of the file format scale lol 19:09 erle so many complex interacting and needlessly convoluted ways to save a file 19:09 erle and read it 19:09 MTDiscord Image library does that work for you though? 19:11 MTDiscord QOL = Quality of Life. QOI = Quality of IIFE. Obviously important for JS devs. 19:15 erle exe_virus bold of you to think that BMP is supported well anywhere lol 19:26 pgimeno about #11877 isn't that a TOCTTOU race? 19:26 ShadowBot https://github.com/minetest/minetest/issues/11877 -- New player causes: duplicate key value violates unique constraint "auth_name_key" 19:26 pgimeno this might be a solution: https://www.postgresqltutorial.com/postgresql-tutorial/postgresql-upsert/ 19:32 MTDiscord I don't think bmp is well supported, except in paint 19:32 MTDiscord In my experience, BMP is a much more widely supported format than, say, PCX. 19:34 MTDiscord Well, yeah, and far more supported than RAW or TIFF 19:43 rubenwardy Pushing trivial bug fix in 10 19:43 rubenwardy !title https://github.com/rubenwardy/minetest/commit/9f6d84d7244b21803fd6d1b2dcbb139c2bbb3eec 19:43 ShadowBot rubenwardy: ContentDB: Fix ungraceful crash on aliases when no connection · rubenwardy/minetest@9f6d84d · GitHub 19:43 rubenwardy Krock ^ 19:44 rubenwardy with amended title: ContentDB: Fix ungraceful crash on aliases when list download fails 19:45 rubenwardy ( basically, store.aliases = {} was only set after a successful load, but was also being accessed after a failed load) 19:45 erle exe_viros Warr1024 i think what you mean is that some subformats of BMP are supported, see https://upload.wikimedia.org/wikipedia/commons/d/da/AllBMPformats.png 19:46 Krock rubenwardy: oh, I thought that error was already fixed? 19:46 rubenwardy I don't think there's many things more boring than discussing image formats 19:46 rubenwardy Krock: well, I can reproduce it on master (: 19:47 erle exe_virus btw regarding webp for small image sizes, i think this should settle it: https://www.ctrl.blog/entry/webp-vs-guetzli-zopfli.html 19:47 rubenwardy I think that the issue reporter tried in 5.6.0-dev, but didn't see the error because the request succeeded 19:48 Krock alright. thanks for the quick fix 20:26 erle exe_virus also https://siipo.la/blog/is-webp-really-better-than-jpeg 20:26 erle > Google’s result of 25-34% smaller files is mostly caused by the fact that they compared their WebP encoder to the JPEG reference implementation, Independent JPEG Group’s cjpeg, not Mozilla’s improved MozJPEG encoder. 20:27 erle exe_virus also the followup https://siipo.la/blog/whats-the-best-lossless-image-format-comparing-png-webp-avif-and-jpeg-xl 20:29 erle i might write a forum post to settle this 20:29 erle it's really polluting this channel 20:30 MTDiscord Oh MozJPEG is an alternate encoder that produces compatible files but does them better? Well ... now I gotta look into this. I've used guetzli for loss-optimization, and jpegtran to optimize the huffman tables for lossless optimization post-loss, but mozjpeg is new to me... 20:30 MTDiscord Heh, the forums DO seem like a better place for a long-form argument :-D 20:48 erle yeah 20:49 erle for me, the takeaway seems to be “people who do filesize and encoding optimizations on some random new format they invented seem to almost always compare against the unoptimized reference encoder” 20:49 erle which only makes sense if you have an optimized encoder like optipng or mozjpeg 20:50 erle then comparing to the baseline is honest 20:50 erle otherwise, it's just straight up useless 20:51 MTDiscord I mean ideally we'd just have independent 3rd party benchmarks to look at, and that would also include varying corpuses of inputs, since almost every encoding strategy works better for some inputs and worse for others. 20:55 MTDiscord Well, jpeg also doesn't support transparency and lossy, which I forgot about. That's actually a good use case that we can't do rn 21:10 erle “since almost every encoding strategy works better for some inputs and worse for others”. true for arbitrary images, but there are very clear boundaries within the texture sizes that are usually used in minetest. 21:11 MTDiscord Just because something is rare based on one person's assumptions about how minetest would/should be used doesn't necessarily mean we shouldn't support it. "Minetest only ever has smallish pixel-art images" assumptions are easily violated by skybox mods. 21:12 erle exe_virus paletted R8G8B8A8 or even A1R5G5B5 is about as lossy as you can get right now with transparency. and very useful for my purposes, i might add. 21:12 erle (to be clear, the A1R5G5B5 is lossy) 21:13 MTDiscord Yes but it's wasteful lossy, basically 21:13 MTDiscord Inefficient lossy 21:13 MTDiscord Right, given the existing formats, that's the best we can do 21:13 MTDiscord A DCT-based format with a proper alpha channel could be pretty nice though, like if you want to have a skybox with a separate cloud layer that you can fade in or out. 21:14 MTDiscord Sure, I'm just saying another image format or two is helpfuo 21:14 MTDiscord But either way, gltf would be higher and is higher on my long priority list... Curse having a family haha 21:22 erle look, if you want some kind of performance, there are bigger fish to fry 21:23 erle but always, measure measure measure 21:23 erle case in point, the SHA1 checks when loading files. a complete waste and optimizing them away makes such a little change unless you have many big files that it's not even interesting. 21:27 MTDiscord Wait, aren't those used to check cache? I.e. for joining servers? 21:27 MTDiscord Are you saying we still use and check the cache in single player mode? 21:28 erle worse 21:28 MTDiscord Worse? 21:28 erle i am saying that something that *can not* have meaningfully changed (e.g. because its inode has never been touched) is still hashed. 21:28 erle in single and multiplayer 21:29 MTDiscord Inode? Image node? You mean metadata? 21:29 erle oh 21:29 erle i can explain 21:32 erle an inode is a filesystem structure that contains metadata of a file. it not having been changed *usually* means the file has not been modified (as in: when this does not hold, some ricer did something weird). https://en.wikipedia.org/wiki/Inode 21:33 erle in any case, a config option to “not recheck cached files” might be the simplest change here 21:34 erle i have already made sure the only files that are hashed are files sent by the server to make sure no cache poisoning happens 21:34 erle in my own builds 21:35 erle and yes, handling this correctly makes the code more complex. but please do not focus on it. i brought it as an example of some inefficiency that is dwarfed by other stuff,. 21:35 sfan5 I haven't kept count of how often the hashing thing has been brought up but I notice that to date nobody has sent a PR for this seemingly simple fix 21:35 erle as in: image file formats are similar. for the most part the gains are not in the file formats. 21:36 MTDiscord Sorry sfan, first time I've heard of it 21:37 erle sfan5, i am bringing it up as something that literally does not matter. like file formats don't matter unless they provide a tangible benefit (like encoding in O(n) or in-place editing or RGBA or animation). 21:38 erle if i knew how to fix it on windows, you'd already have a patch btw. but i do not. 21:39 erle in any case, it's time for me to make a forum post about textures some time soon so we can avoid this 21:39 erle it pops up every few weeks and is annoying 21:54 MTDiscord Doesn't stdlib give you access to file metadata like that? 22:02 erle exe_virus if you can figure that out, feel free to make a patch