Minetest logo

IRC log for #minetest-dev, 2023-04-05

| Channels | #minetest-dev index | Today | | Google Search | Plaintext

All times shown according to UTC.

Time Nick Message
00:01 proller joined #minetest-dev
00:26 Fleckenstein joined #minetest-dev
01:49 PrairieWind joined #minetest-dev
01:49 PrairieWind joined #minetest-dev
02:44 tekakutli joined #minetest-dev
02:46 tekakutli joined #minetest-dev
02:54 tekakutli joined #minetest-dev
02:57 YuGiOhJCJ joined #minetest-dev
03:00 Noisytoot joined #minetest-dev
03:16 olliy joined #minetest-dev
03:43 tekakutli joined #minetest-dev
03:45 tekakutli joined #minetest-dev
03:46 tekakutli joined #minetest-dev
03:46 tekakutli joined #minetest-dev
04:00 MTDiscord joined #minetest-dev
05:16 tekakutli joined #minetest-dev
05:17 calcul0n joined #minetest-dev
05:36 tekakutli joined #minetest-dev
05:39 Noisytoot joined #minetest-dev
05:47 tekakutli joined #minetest-dev
05:50 tekakutli joined #minetest-dev
05:55 diceLibrarian joined #minetest-dev
05:59 tekakutli joined #minetest-dev
08:38 sfan5 PRs welcome
10:01 Noisytoot joined #minetest-dev
13:10 tekakutli joined #minetest-dev
13:34 Noisytoot joined #minetest-dev
13:43 turtleman joined #minetest-dev
13:59 tekakutli joined #minetest-dev
14:00 tekakutli joined #minetest-dev
14:01 tekakutli joined #minetest-dev
14:12 fluxionary_ joined #minetest-dev
14:12 Noisytoot joined #minetest-dev
14:26 proller joined #minetest-dev
14:38 Noisytoot joined #minetest-dev
15:01 tekakutli joined #minetest-dev
15:40 Desour joined #minetest-dev
15:48 diceLibrarian joined #minetest-dev
15:54 Guest54 joined #minetest-dev
15:57 Guest54 given that the filesystem interface supports zip files and the irrlicht media loaders use that filesystem interface AFAIK, would there be any benefit to loading mod assets from zip files? except for saving filesystem space, i mean.
15:58 rubenwardy would prevent needing to extract assets on Android
15:58 Guest54 well, actual mod directories could, of course, still be needed for dynamic asset generation.
15:58 rubenwardy mod directories aren't necessarily always writable
15:58 rubenwardy ie:   /usr/share/minetest/mods/
15:59 Guest54 well, this is a case where doing semgrep on everything in cdb would help to see if anyone does it
16:20 MTDiscord <Warr1024> Not having to extract could be pretty nice, actually.  Android is one example but probably not the only filesystem/context in which handling lots of little files isn't necessarily that efficient.
16:21 Guest54 many games do it by default. and i have also found that this is the explanation for why they can get away with having ridiculous asset size uncompressed, everything is zipped.
16:21 MTDiscord <Warr1024> Contiguity of related data could also be a nice side effect, e.g. you might get better cache hit ratios that affect stuff like HDDs.  Also, some assets might actually compress pretty well, e.g. the lua code files, some models...
16:29 MTDiscord <Jonathon> windows also sucks at handling small files
16:38 Noisytoot joined #minetest-dev
16:39 Desour joined #minetest-dev
16:50 Guest54 Warr1024 counter-intuitively (to me at least), zipping uncompressed bitmaps can work much better than using optipng (and is faster too) – e.g. more than half the textures in mineclone2 (over 1200 out of about 2300) get smaller as a tga.gz than the smallest thing optipng has spit out for me (by average of about 26 bytes per texture, if i am not
16:50 Guest54 mistaken, probably a bit more because of the gzip framing). if you cheat by using A1R5G5B5 colors for the uncompressed bitmap, over 2000 out of 2300 pics are smaller.
16:51 Guest54 in practice, putting them in a zip will prevent each one from consuming a 4k filesystem block
16:52 Guest54 (together, not individually)
16:55 MTDiscord <Warr1024> Doesn't seem THAT counterintuitive to me.  Most image formats that support built-in compression neglect to compress headers, and on small enough images, and formats that have complex header structure, those can become dominant.
16:56 MTDiscord <Warr1024> If MT supported sending and caching assets with external compression, e.g. tga.gz or something, then I'd have an easier time supporting their use in the engine.  But as it is, compressing assets on disk at the expense of crappier network sends is probably not a trade-off I'm interested in.
16:56 Guest54 the counterintuitive result for me was mainly that optipng can't beat it, despite wasting a lot of CPU cycles
16:57 MTDiscord <Warr1024> Diminishing returns.  It's not terribly hard to find simple and small enough images where running zopfli brute-forcing for hours can't beat a pretty naive gzip.  Recompressors just try more and more desperate strategies to squeeze out the last few bits, but sometimes there just weren't any hidden opportunities to begin with.
16:58 Guest54 indeed
16:58 Guest54 i was pretty surprised how well it worked for the devtest checkerboard though
16:59 MTDiscord <Warr1024> Also, basically everything uses the same deflate compression algorithms these days, so if you can get a certain result with a PNG compressor, then you should be able to get very comparable, or slightly better, with like .bmp.gz as long as the gz engine you're using is as thorough as the PNG recompressor's.  You can get the same for zip files too (though it's still per-file of course).
17:00 Guest54 not really, it's a speed-space tradeoff. gimp with default settings for example does produce PNGs that are about as good/bad as throwing raw pixels into deflate.
17:01 Guest54 the reason is probably because that makes saving instant … meanwhile optipng can cut the size in half sometimes, but you don't want to wait 5 seconds on even the fastest machine until it has figured out which knobs to turn.
17:02 Guest54 Warr1024 i think i may have mentioned it already, since i want to improve the handling of generated textures i do care a lot about network transfer size.
17:02 Guest54 and speed, of course
17:04 MTDiscord <Warr1024> I played around with adding per-packet lzo to MT, and was a bit disappointed to see like 3% or less improvement, so I didn't push the project further.  Granted, that may have been because most of the data dominating the bandwidth usage was mapblock sends, and those were already zstd'd.
17:04 MTDiscord <Warr1024> Also, testing against NodeCore was probably not a great idea because I probably should have used content that was more wasteful to begin with 😄
17:04 Guest54 hehe
17:05 Guest54 i wouldn't want to automate it actually
17:07 MTDiscord <Warr1024> I would think that MT should be able to make at least reasonable guesses as to what kind of data would be good for compression and what would not.  Also, for media, MT could cache the compressed versions in advance, and choose to fall back to the uncompressed version if the compressed version isn't smaller enough to warrant it.
17:08 MTDiscord <Warr1024> Being able to turn off automatic compression, and just provide like .obj.gz files or something as assets, which the engine knows to send with compression intact and decompress at load time, might be nice for some of us (more serious game devs and server owners and such) but having sane default behavior for the "average (naive) case" would be nice too
17:09 Guest54 well, you can actually not make a guess and just try it out server-side if the client supports it.
17:14 Guest54 by the way i strongly advise against using BMP. the format is a big mess (and compresses worse than the alternatives)
17:14 MTDiscord <Warr1024> You could, though I'd follow the pattern that a lot of web servers use for on-the-fly compression, which includes them often having a list of filetypes that they can identify as being very likely not worth even trying.  If we know that, say, PNG, OGG, and JPEG (perhaps with certain size constraints) aren't worth trying, we can keep things pretty simple and still have fast startup times.  Also, not every server operator will want to
17:14 MTDiscord keep trying the same files over and over again on every restart, even if we can do it inteligently in the background without delaying startup, because they may e.g. pay for CPU burst or something.
17:15 Guest54 my problem is that you may not want to auto-compress stuff based on the contents, not on file type in particular.
17:16 Guest54 e.g. some random noise texture is not very compressible
17:17 Guest54 given the amount of useless sha1 hashing going on during asset load i have a suspicion few people would notice on-the-fly compression ;)
17:22 MTDiscord <Warr1024> If you would want to inform the engine what it should try and not try to compress, that'd be fine too.  But in the absence of info provided by the dev, the engine will need to be able to make the decision based on some sane default heuristic.  "Try to compress it all" is not a terrible idea but I could see some issues with it at least, while guessing by file type is a pretty decent mitigation.
17:23 Guest54 well, i do agree with exclusions by filetype – you won't get png, vorbis or jpeg content much smaller anyways – it's already compressed (even if it is compressed badly)
17:25 Guest54 btw, xmaps does already store deflated bitmaps in item meta (so the maps work in world downloads)
17:25 Guest54 i suggest to do the same for anything in nodecore (but ig it's a #minetest topic, sorry)
17:53 proller joined #minetest-dev
17:57 Guest54 given that zstd has negative levels up to -7 or so, why is the default compression level in builtin/settingtypes.txt limited to -1?
18:19 olliy1or joined #minetest-dev
19:52 Guest54 joined #minetest-dev
20:21 tekakutli joined #minetest-dev
20:47 tekakutli joined #minetest-dev
21:23 Guest5 joined #minetest-dev
21:24 Guest5 we know where your workplace is, andrew
21:24 Guest5 don't forget the condom when you get fucked hard in the ass
21:30 MTDiscord <Fleckenstein> imagine sinking this low
21:48 rubenwardy Pathetic little man
21:48 Guest5 was kicked by rubenwardy: kick
22:19 MTDiscord <PrairieWind> uhh
22:19 MTDiscord <PrairieWind> yeesh
22:24 MTDiscord <Fatalist error> What just happened
22:24 MTDiscord <Fatalist error> Was context deleted?
22:31 MTDiscord <Fleckenstein> not yet, see Guest5's message above
22:31 MTDiscord <Fleckenstein> or don't, it's probably better to not engage
22:33 tekakutli joined #minetest-dev
22:34 Guest54 indeed, engaging is a mistake: i asked guest5 for an explanation and got only a “you know the problem“ and “who are you?”. i should have known better. :(
22:35 MTDiscord <Fleckenstein> kilibith
22:35 MTDiscord <Fleckenstein> likely
22:46 proller joined #minetest-dev
22:52 tekakutli joined #minetest-dev

| Channels | #minetest-dev index | Today | | Google Search | Plaintext