| Time |
Nick |
Message |
| 00:11 |
erle |
maybe there should be a “i found this cool new texture file format on hacker news” FAQ |
| 00:11 |
erle |
with the answer being no |
| 00:19 |
erle |
pgimeno decoding speed is a *major* reason why some games have giant assets. but those games usually deal with graphics bigger than 16×16 and have sprite sheets. meanwhile, minetest has a lot of very small files. i have benchmarked this quite a lot and even on a potato computer you are not going to notice any speedup just by using a different image loader. minetest does a ridiculous amount of useless busywork for texture |
| 00:19 |
erle |
decoding, but in the big picture, about *none* of it matters. |
| 00:27 |
erle |
pgimeno, what matters is: a) if a format is in use b) tooling support c) format features: RGB / RGBA / palette / alpha channel (alpha bit is not enough) / compression d) how easy you can emit / read / rewrite it |
| 00:45 |
erle |
pgimeno i have both read the QOI spec and i think you are better off making paletted TGA work. additionally, QOI cheats on filesizes: i have tried 3 of their PNG images in the “comparison” archive and they could easily be shrinked by 25% to 30% |
| 00:45 |
erle |
honestly, if some poser comes up with a new file format and you can outperform it in filesize & implementation size & decoding speed with paletted TGA, it seems absolutely useless |
| 00:46 |
erle |
(for minetest purposes) |
| 00:48 |
pgimeno |
I believe they are comparing against a PNG size as compressed by a specific popular single-file library, not against an optimized PNG |
| 00:49 |
pgimeno |
and yes, for small images it's probably useless for Minetest |
| 01:00 |
erle |
pgimeno you did get it from hacker news or reddit though, did yuo? |
| 01:00 |
pgimeno |
no, from love2d forums as I said |
| 01:00 |
erle |
ah ok |
| 01:01 |
pgimeno |
I don't read either of those unless they appear in search |
| 01:01 |
erle |
sorry then |
| 01:02 |
erle |
pgimeno, for the future, if you are fascinated by file formats that are easy to implement and work with, read the TGA spec. it doesn't get simpler than that while still being useful. and also then you will be vaccinated against “the newest cool thing”. QOI is clever, but needlessly clever. |
| 01:03 |
pgimeno |
... |
| 01:03 |
erle |
btw, the reason why i mention palettes so often is because 16×16 is 256 |
| 01:03 |
pgimeno |
I advocated for TGA support right here because I know how simple and straightforward it is |
| 01:03 |
erle |
oh okay sorry |
| 01:04 |
erle |
i seem to be needlessly mean today. i hope you can forgive me. |
| 01:04 |
erle |
i promise to be better |
| 01:04 |
erle |
pgimeno, i think the only format i want is some vector format. do you know any simple to implement vector formats? SVG is not simple. |
| 01:04 |
erle |
i mean something that can be hooked up to the engine to get non-ugly menu icons |
| 01:05 |
erle |
pgimeno, i apologize |
| 01:06 |
pgimeno |
can't think of any simple vector format besides dxf, which is probably not suitable |
| 01:27 |
MTDiscord |
<Warr1024> A somewhat less dumb option might be to offer the ability to receive PNG over the network, and write TGA format as like *.png.tga to the local cache so they can be located based on the original filename. If we have readers for these files, then it's not a stretch to consider that the image libraries may have writers that make conversion an option. |
| 01:31 |
erle |
Warr1024 this makes very little sense, why would you do that? |
| 01:31 |
erle |
*little sense to me |
| 01:33 |
erle |
if it's to optimize loading times by having uncompressed assets, i already mentioned that you are not getting any noticeable speedup from that |
| 01:34 |
erle |
only thing i have not profiled so far is having texture atlases |
| 01:35 |
erle |
(and they make caching a sad story ig) |
| 02:02 |
|
fluxionary joined #minetest-dev |
| 02:14 |
|
sometalgoo joined #minetest-dev |
| 02:35 |
fluxionary |
does anyone have insight into https://github.com/minetest/minetest/issues/11877 ? |
| 03:14 |
Pexin |
!title |
| 03:14 |
ShadowBot |
New player causes: duplicate key value violates unique constraint "auth_name_key" · Issue #11877 · minetest/minetest · GitHub |
| 04:00 |
|
MTDiscord joined #minetest-dev |
| 04:23 |
|
sometalgoo left #minetest-dev |
| 06:04 |
|
calcul0n joined #minetest-dev |
| 06:58 |
|
appguru joined #minetest-dev |
| 07:11 |
MTDiscord |
<ROllerozxa> https://github.com/minetest/minetest/issues/12134 this issue should probably be closed |
| 07:12 |
MTDiscord |
<ROllerozxa> https://github.com/minetest/minetest/issues/12056 this one as well, the issue is one that has already been fixed but they are on an older version, see "snap version works" |
| 07:13 |
|
YuGiOhJCJ joined #minetest-dev |
| 09:16 |
|
appguru joined #minetest-dev |
| 10:17 |
|
HuguesRoss joined #minetest-dev |
| 10:58 |
|
proller joined #minetest-dev |
| 11:12 |
|
Fixer joined #minetest-dev |
| 11:39 |
|
proller joined #minetest-dev |
| 12:24 |
|
proller joined #minetest-dev |
| 15:08 |
|
proller joined #minetest-dev |
| 15:57 |
|
fluxionary joined #minetest-dev |
| 16:56 |
erle |
pgimeno the big issue with vector files as texture formats is how they are setup right now. basically, everything that is decoded … is decoded into an R8G8B8A8 pixel buffer. so you need to know image size at decoding time. |
| 16:58 |
erle |
it might be easier to get crisp menu files by making sure they are only scaled at defined intervals, like openttd does. |
| 16:58 |
erle |
i.e. normal icon, 2x icon, 4x icon. |
| 16:58 |
erle |
or 1/2 icon etc. |
| 16:59 |
erle |
that would neatly avoid the distortion effects you get right now |
| 17:01 |
MTDiscord |
<ROllerozxa> speaking about the main menu game textures, I still can't figure out why it is so picky about header sizes and just doesn't show it if it's not up to liking |
| 17:15 |
rubenwardy |
Updated #12185 , ready for review |
| 17:15 |
ShadowBot |
https://github.com/minetest/minetest/issues/12185 -- Add login/register dialog to separate register and login by rubenwardy |
| 17:54 |
rubenwardy |
Who was it looking into glTF? Did that go anywhere |
| 17:57 |
MTDiscord |
<Benrob0329> @ExeVirus |
| 18:26 |
rubenwardy |
trivial PR #11876 |
| 18:26 |
ShadowBot |
https://github.com/minetest/minetest/issues/11876 -- Update mods_here.txt to mention installing mods via CDB by qwerty123a2 |
| 18:35 |
rubenwardy |
any everyone please go through this list before the next meeting, so we don't have to spend so long: https://github.com/minetest/minetest/pulls?q=is%3Apr+is%3Aopen+sort%3Aupdated-desc+label%3A%22Roadmap%3A+Needs+approval%22 |
| 18:39 |
rubenwardy |
(I've also not finished going through) |
| 18:43 |
MTDiscord |
<exe_virus> @rubenwardy , yeah I was looking at it, though I have not gotten around to making it. Totally possible, just requires a JSON dependency, and some conversion coding |
| 18:43 |
rubenwardy |
we have JSON? |
| 18:43 |
rubenwardy |
in irrlicht I guess? |
| 18:43 |
MTDiscord |
<exe_virus> Well, this file reader would be in the minetest codebase, not irrlichtmt |
| 18:44 |
MTDiscord |
<exe_virus> As should all extension readers (webp, avif, etc) |
| 18:46 |
erle |
exe_virus why though? |
| 18:46 |
erle |
also again, webp is useless |
| 18:57 |
MTDiscord |
<exe_virus> Webp is the smallest lossless format on the market that is mature |
| 18:57 |
MTDiscord |
<exe_virus> The file reader interface was designed to be outside of irrlicht. Just trying to be consistent with it's design philosophy |
| 18:58 |
MTDiscord |
<exe_virus> Webp2 is far better, but not stable yet |
| 18:58 |
MTDiscord |
<exe_virus> Avif lossless is worse than webp. We care about lossless for the most part here in MT |
| 18:58 |
ROllerozxa |
don't really understand the hype behind webp, either lossless or lossy webp |
| 18:58 |
MTDiscord |
<exe_virus> Just better compression than png |
| 18:59 |
MTDiscord |
<exe_virus> And single file format, for purests |
| 18:59 |
MTDiscord |
<exe_virus> Webp2 though, is gunning for best overall, they're not to avif lossy levels yet, but they're getting there |
| 18:59 |
MTDiscord |
<Jordach> unfortunately webp suffers from being browser tech |
| 18:59 |
ROllerozxa |
so any lossless webp image is going to be smaller than its fully optimised png format? |
| 18:59 |
MTDiscord |
<Jordach> good luck finding an editor for it that uses the old format in a year |
| 19:00 |
MTDiscord |
<exe_virus> Webp? It's stable. Already in gimp. |
| 19:00 |
MTDiscord |
<Jordach> also why care about saving something like 100 bytes |
| 19:00 |
MTDiscord |
<exe_virus> Almost every lossless webp is smaller than optimized png, we tested this against nodecore and dream builder. About 20% smaller |
| 19:01 |
MTDiscord |
<exe_virus> So for nodecore, it was about 20-50Kb irrc, for dream builder.... A lot more |
| 19:01 |
MTDiscord |
<Jordach> maximum PNG optimisation is still better considering it's tooling is practically everywhere |
| 19:01 |
MTDiscord |
<exe_virus> Why not both? |
| 19:01 |
rubenwardy |
remember that our images are mostly 16x16. png is super well supported |
| 19:01 |
rubenwardy |
maintenance |
| 19:02 |
MTDiscord |
<exe_virus> Toss a library in, done? |
| 19:02 |
MTDiscord |
<Jordach> yes because like web browsers are going to agree on that standard in a year |
| 19:02 |
MTDiscord |
<exe_virus> They have for 3-5 years now? |
| 19:02 |
MTDiscord |
<Jordach> it just means it's maintenance is slaved towhatever chromium developers are up to at the behest of google |
| 19:03 |
MTDiscord |
<exe_virus> Well, webp is in maintenance, and wont be changing. |
| 19:03 |
MTDiscord |
<exe_virus> Also, 16x16 is exactly the biggest reason to use webp2 later, because the headers are non existent, and we're talking extremely small textures, like 50% of png sizes |
| 19:04 |
MTDiscord |
<Jordach> saving less than 1k bytes for a massive dependancy IS NOT WORTH IT |
| 19:04 |
MTDiscord |
<exe_virus> Single header? |
| 19:04 |
MTDiscord |
<exe_virus> One dep |
| 19:04 |
MTDiscord |
<Jordach> the dependancy is probably larger than the image it's trying to deal with |
| 19:04 |
MTDiscord |
<exe_virus> Not over the wire |
| 19:04 |
MTDiscord |
<Jordach> and anyways, we don't need more random shit in MT |
| 19:04 |
MTDiscord |
<exe_virus> It's an engine, we should support more modern formats over time. |
| 19:04 |
erle |
look, if you want a file size optimization, make paletted TGA work with RLE compression. that gets you under PNG size for a lot of small files. but if i thought it was even worth it i would have done it by now. there are no big savings here. |
| 19:05 |
erle |
when you reach 100 or 200 bytes per file, nothing matters much |
| 19:05 |
MTDiscord |
<exe_virus> That's what webp and avif, and other modern formats do, with better support than TGA |
| 19:05 |
MTDiscord |
<Jordach> modern does not mean better |
| 19:05 |
erle |
file size is not the problem. the overhead is elsewhere. |
| 19:05 |
MTDiscord |
<Warr1024> Arguing over individual image file formats is just silly. It's the 21st century, and image format support should be the responsibility of your image loading library. |
| 19:06 |
MTDiscord |
<exe_virus> So include said libraries? |
| 19:06 |
erle |
given that the existing file formats can not be rendered right, more complex file formats mean more problems. |
| 19:06 |
MTDiscord |
<exe_virus> They all get expanded to RBG anyways |
| 19:07 |
erle |
look, what do you want? small files? |
| 19:07 |
MTDiscord |
<exe_virus> Yep. Though gltf is a different story |
| 19:07 |
MTDiscord |
<Warr1024> I like small files and I cannot lie. |
| 19:07 |
erle |
i mean look at the QOI mess |
| 19:08 |
MTDiscord |
<exe_virus> Quality of life? |
| 19:08 |
MTDiscord |
<ROllerozxa> and bmp lol |
| 19:08 |
erle |
quite okay image format. it's “modern” in the sense that it uses a bunch of tricks. and it can “outperform PNG” in a very very narrow use case. |
| 19:08 |
erle |
yeah BMP is the stupid end of the file format scale lol |
| 19:09 |
erle |
so many complex interacting and needlessly convoluted ways to save a file |
| 19:09 |
erle |
and read it |
| 19:09 |
MTDiscord |
<exe_virus> Image library does that work for you though? |
| 19:11 |
MTDiscord |
<Warr1024> QOL = Quality of Life. QOI = Quality of IIFE. Obviously important for JS devs. |
| 19:15 |
erle |
exe_virus bold of you to think that BMP is supported well anywhere lol |
| 19:26 |
pgimeno |
about #11877 isn't that a TOCTTOU race? |
| 19:26 |
ShadowBot |
https://github.com/minetest/minetest/issues/11877 -- New player causes: duplicate key value violates unique constraint "auth_name_key" |
| 19:26 |
pgimeno |
this might be a solution: https://www.postgresqltutorial.com/postgresql-tutorial/postgresql-upsert/ |
| 19:32 |
MTDiscord |
<exe_virus> I don't think bmp is well supported, except in paint |
| 19:32 |
MTDiscord |
<Warr1024> In my experience, BMP is a much more widely supported format than, say, PCX. |
| 19:34 |
MTDiscord |
<exe_virus> Well, yeah, and far more supported than RAW or TIFF |
| 19:43 |
rubenwardy |
Pushing trivial bug fix in 10 |
| 19:43 |
rubenwardy |
!title https://github.com/rubenwardy/minetest/commit/9f6d84d7244b21803fd6d1b2dcbb139c2bbb3eec |
| 19:43 |
ShadowBot |
rubenwardy: ContentDB: Fix ungraceful crash on aliases when no connection · rubenwardy/minetest 9f6d84d · GitHub |
| 19:43 |
rubenwardy |
Krock ^ |
| 19:44 |
rubenwardy |
with amended title: ContentDB: Fix ungraceful crash on aliases when list download fails |
| 19:45 |
rubenwardy |
( basically, store.aliases = {} was only set after a successful load, but was also being accessed after a failed load) |
| 19:45 |
erle |
exe_viros Warr1024 i think what you mean is that some subformats of BMP are supported, see https://upload.wikimedia.org/wikipedia/commons/d/da/AllBMPformats.png |
| 19:46 |
Krock |
rubenwardy: oh, I thought that error was already fixed? |
| 19:46 |
rubenwardy |
I don't think there's many things more boring than discussing image formats |
| 19:46 |
rubenwardy |
Krock: well, I can reproduce it on master (: |
| 19:47 |
erle |
exe_virus btw regarding webp for small image sizes, i think this should settle it: https://www.ctrl.blog/entry/webp-vs-guetzli-zopfli.html |
| 19:47 |
rubenwardy |
I think that the issue reporter tried in 5.6.0-dev, but didn't see the error because the request succeeded |
| 19:48 |
Krock |
alright. thanks for the quick fix |
| 20:06 |
|
pgimeno left #minetest-dev |
| 20:26 |
erle |
exe_virus also https://siipo.la/blog/is-webp-really-better-than-jpeg |
| 20:26 |
erle |
> Google’s result of 25-34% smaller files is mostly caused by the fact that they compared their WebP encoder to the JPEG reference implementation, Independent JPEG Group’s cjpeg, not Mozilla’s improved MozJPEG encoder. |
| 20:27 |
erle |
exe_virus also the followup https://siipo.la/blog/whats-the-best-lossless-image-format-comparing-png-webp-avif-and-jpeg-xl |
| 20:29 |
erle |
i might write a forum post to settle this |
| 20:29 |
erle |
it's really polluting this channel |
| 20:30 |
MTDiscord |
<Warr1024> Oh MozJPEG is an alternate encoder that produces compatible files but does them better? Well ... now I gotta look into this. I've used guetzli for loss-optimization, and jpegtran to optimize the huffman tables for lossless optimization post-loss, but mozjpeg is new to me... |
| 20:30 |
MTDiscord |
<Warr1024> Heh, the forums DO seem like a better place for a long-form argument :-D |
| 20:48 |
erle |
yeah |
| 20:49 |
erle |
for me, the takeaway seems to be “people who do filesize and encoding optimizations on some random new format they invented seem to almost always compare against the unoptimized reference encoder” |
| 20:49 |
erle |
which only makes sense if you have an optimized encoder like optipng or mozjpeg |
| 20:50 |
erle |
then comparing to the baseline is honest |
| 20:50 |
erle |
otherwise, it's just straight up useless |
| 20:51 |
MTDiscord |
<Warr1024> I mean ideally we'd just have independent 3rd party benchmarks to look at, and that would also include varying corpuses of inputs, since almost every encoding strategy works better for some inputs and worse for others. |
| 20:53 |
|
proller joined #minetest-dev |
| 20:55 |
MTDiscord |
<exe_virus> Well, jpeg also doesn't support transparency and lossy, which I forgot about. That's actually a good use case that we can't do rn |
| 21:10 |
erle |
“since almost every encoding strategy works better for some inputs and worse for others”. true for arbitrary images, but there are very clear boundaries within the texture sizes that are usually used in minetest. |
| 21:11 |
MTDiscord |
<Warr1024> Just because something is rare based on one person's assumptions about how minetest would/should be used doesn't necessarily mean we shouldn't support it. "Minetest only ever has smallish pixel-art images" assumptions are easily violated by skybox mods. |
| 21:12 |
erle |
exe_virus paletted R8G8B8A8 or even A1R5G5B5 is about as lossy as you can get right now with transparency. and very useful for my purposes, i might add. |
| 21:12 |
erle |
(to be clear, the A1R5G5B5 is lossy) |
| 21:13 |
MTDiscord |
<exe_virus> Yes but it's wasteful lossy, basically |
| 21:13 |
MTDiscord |
<exe_virus> Inefficient lossy |
| 21:13 |
MTDiscord |
<Warr1024> Right, given the existing formats, that's the best we can do |
| 21:13 |
MTDiscord |
<Warr1024> A DCT-based format with a proper alpha channel could be pretty nice though, like if you want to have a skybox with a separate cloud layer that you can fade in or out. |
| 21:14 |
MTDiscord |
<exe_virus> Sure, I'm just saying another image format or two is helpfuo |
| 21:14 |
MTDiscord |
<exe_virus> But either way, gltf would be higher and is higher on my long priority list... Curse having a family haha |
| 21:22 |
erle |
look, if you want some kind of performance, there are bigger fish to fry |
| 21:23 |
erle |
but always, measure measure measure |
| 21:23 |
erle |
case in point, the SHA1 checks when loading files. a complete waste and optimizing them away makes such a little change unless you have many big files that it's not even interesting. |
| 21:27 |
MTDiscord |
<exe_virus> Wait, aren't those used to check cache? I.e. for joining servers? |
| 21:27 |
MTDiscord |
<exe_virus> Are you saying we still use and check the cache in single player mode? |
| 21:28 |
erle |
worse |
| 21:28 |
MTDiscord |
<exe_virus> Worse? |
| 21:28 |
erle |
i am saying that something that *can not* have meaningfully changed (e.g. because its inode has never been touched) is still hashed. |
| 21:28 |
erle |
in single and multiplayer |
| 21:29 |
MTDiscord |
<exe_virus> Inode? Image node? You mean metadata? |
| 21:29 |
erle |
oh |
| 21:29 |
erle |
i can explain |
| 21:32 |
erle |
an inode is a filesystem structure that contains metadata of a file. it not having been changed *usually* means the file has not been modified (as in: when this does not hold, some ricer did something weird). https://en.wikipedia.org/wiki/Inode |
| 21:33 |
erle |
in any case, a config option to “not recheck cached files” might be the simplest change here |
| 21:34 |
erle |
i have already made sure the only files that are hashed are files sent by the server to make sure no cache poisoning happens |
| 21:34 |
erle |
in my own builds |
| 21:35 |
erle |
and yes, handling this correctly makes the code more complex. but please do not focus on it. i brought it as an example of some inefficiency that is dwarfed by other stuff,. |
| 21:35 |
sfan5 |
I haven't kept count of how often the hashing thing has been brought up but I notice that to date nobody has sent a PR for this seemingly simple fix |
| 21:35 |
erle |
as in: image file formats are similar. for the most part the gains are not in the file formats. |
| 21:36 |
MTDiscord |
<exe_virus> Sorry sfan, first time I've heard of it |
| 21:37 |
erle |
sfan5, i am bringing it up as something that literally does not matter. like file formats don't matter unless they provide a tangible benefit (like encoding in O(n) or in-place editing or RGBA or animation). |
| 21:38 |
erle |
if i knew how to fix it on windows, you'd already have a patch btw. but i do not. |
| 21:39 |
erle |
in any case, it's time for me to make a forum post about textures some time soon so we can avoid this |
| 21:39 |
erle |
it pops up every few weeks and is annoying |
| 21:54 |
MTDiscord |
<exe_virus> Doesn't stdlib give you access to file metadata like that? |
| 22:02 |
erle |
exe_virus if you can figure that out, feel free to make a patch |
| 22:32 |
|
panwolfram joined #minetest-dev |
| 23:22 |
|
AliasAlreadyTake joined #minetest-dev |
