Time Nick Message 00:25 erlehmann minetest_game, the game where water can be on fire https://mister-muffin.de/p/IHtW.png 00:30 independent56 Can't worldedit act on unloaded blocks? Can you issue a command, and when someone laods that mapblock, worldedit does the command? 00:33 MTDiscord mineclonia: the game performs terribly 00:46 erlehmann Jonathon could yo be more specific? 00:46 erlehmann currently the only performance hogs i am aware of are rain/snow and fire 00:46 MTDiscord i was being sarcastic 00:47 erlehmann if you doubt that we improved performance by, like, a lot, compare the packet count to mineclone2 or mineclone5 00:47 erlehmann rain is cursed though 00:48 MTDiscord weather in general is cursed because players can eventually out run the weather 00:48 MTDiscord and using hud overlays would spam to many packets 00:48 MTDiscord unless you can use animated images in huds? 00:49 MTDiscord that would be a ideal way to doweather 00:57 MTDiscord however it appears that does not exist 00:57 mazes_81 jonadab the autocrafter group: recipe patch is for the pipeworks autocrafter ? 01:00 mazes_81 I took a look, and will have a try on it 01:03 mazes_81 still about weather, does the regional_weather/climate_api author chat here ? 01:05 MTDiscord if they use the testificate handle here, i dont think so 03:04 MTDiscord I am bored 09:38 independent56 is 111000 "us" equivlent to 111 ms? 09:38 independent56 (probably microseconds) 09:38 sfan5 let's see 09:38 sfan5 !c 11000 / 1000 09:38 MinetestBot 11.0 09:39 sfan5 wait 09:39 sfan5 forgot an 1 but you can see that yes, it's correct 09:41 independent56 Cool. So microseconds also follows metric. 09:42 independent56 None of that silly "a minute is 1/60th of an hour unless your clock is wrong" 09:46 independent56 My mother is getting a new computer because the "o" key keeps on spamming, making it unusable for her. Is it worth the pain of installing ubuntu server on a different machine, getting it prepared, moving the Drive, and then using that as my server just to escape the outdated 32 bit decade old webook?? 09:51 Tusha Well, I would say it depends how good is the "failing" computer compared to what are you using, and I would check if user configs could be ported saving you a great hassle on the setup 09:52 sfan5 or simplier: swap out the hard disks, zero changes needed in the ideal case 09:53 Tusha Well, if the destination is using uefi and the old one uses legacy bios, trouble will arise 09:54 Tusha It also depends if it is a laptop or desktop, as some laptops tends to be picky with linux, sadly 09:56 independent56 it's a nice laptop, with being 3 years old at most. 09:56 independent56 Of course, i will be using ubuntu server instead of bodhi because "i no need gui" 09:57 Tusha In any case, independent56, if you want better feedback, if you can put the specs from the origin and destination computers it could make tech savvy people here (I can't include myself, to be honest) give you the best advices 09:58 Tusha Also the use-case (what you intend to do with it) could be really helpful 09:59 independent56 1 gb ram, 300 gb hd at most with the 32 bit. >4 gb ram, ~500 gb ssd for the medion laptop. 10:00 independent56 And i will use it for the same reasons i do now: web and minetest server hosting. 10:00 independent56 But i may use it split - the 32 bit hosts web, and the superior lapop can do more complex tasks 10:04 Tusha With my limited knowledge on Minetest I understand that the ssd should improve the server performance 10:04 Tusha I would also measure the different energy consumption on heavy load, so the powerbill don't give you a surprise at the month's end 10:05 independent56 I propobably will have a moral objecitionto using it: it stores my mother's files. 10:05 independent56 And i guess it only eats as much energy as it being constantly plugged in 10:06 Tusha I though you were going to do a zero install on it? Or your mother will stil be using it? 10:06 Tusha still* 10:11 independent_ No, mum will be getting a new laptop. But i will use it with a hard disk from a computer which installed ubuntu server for that purpouse. 10:13 Tusha So your mother will keep the ssd the computer is actually using then? If not a repartitioning could easily solve the file's problem 10:14 independent_ I will object - putting sensitive files on a hard drive accessible to the open web is ahuge risk. partitioned or not. 10:14 independent_ I will use a spare hdd 10:16 Tusha I was talking about deleting existing partitions and creating new ones, giving slow format (although not sure if that helps with data erasing on ssds) 10:16 grouinos fuckoff asshole 10:16 grouinos and suck my big dick 10:18 Tusha I was going to recommend using dban, but seems it doesn't destroy data securely on ssds 10:19 independent_ I do not wish to destroy any data. I have a horrible reputation for data loss, losing sentimental data biannually 10:20 independent_ I will make two copies of the data,but the mainamount is to be preserved on my mother'scomputer. 10:20 independent_ *the ssd drava for the computer 10:21 Tusha Ok so if I understand well the ssd the computer is using now will be extracted and used on your mother's new PC? 10:21 grouinos you can put yours sentimental data in your ass, more secure 10:22 grouinos and put your motherboard too 10:23 independent_ Tusha, i was thinkingit would just sit around, and we couldcopy files. The computer isn'teven fromthe same company. There is bound tobe cross-compatibility issues 10:24 independent_ \whois independent_ 10:24 independent_ Wait wrong slash direction 10:26 grouinos fuckoff asshole 10:26 sfan5 independent_: the manufacturer doesn't matter as much as you think 10:26 Tusha Well, in that case that would be the sensible option, although it could work out-the-box if you just plug in it in the new laptop, if it is easy to do so (some laptops are a pita to do anthing to them) 10:27 sfan5 moving from a 32-bit to a 64-bit system should obviously mean a reinstall but other than that Linux installations are very portable 10:27 Tusha yeah as sfan5 said, if both laptops have uefi it could work, although I'm not sure if the are using secure boot, which is also a pita to work with 10:27 independent_ Cool, but i would like a fresh install of windows, so we can take the minimumm amount, and then let mum grow the amount by their own. 10:27 sfan5 oh, windows 10:27 sfan5 I suggest reinstalling that 10:28 Tusha I was afraid of that 10:28 independent_ Mum uses windows and it's what she was grown up with. I do not wish to switch her to another OS. it would be too much work 10:28 Tusha Yeah, if you have an usb hdd adaptor, just copy files to the new computer and then use the old ssd on the "failing" pc 10:28 independent_ But she is using a spare linux mintcomputer, and she oes just fine. 10:29 Tusha Or use an usb stick to copy all the "sensible" data to the new one 10:29 independent_ Tusha, The thing is, whenever i try to copyfiles, some inevitably get missed. I want to ensure mum dosent get that onosecond. 10:30 Tusha Uhm, how big the space used in the ssd? 10:30 independent_ No idea. It's at the repair shop, who said they couldn't repair the ssd 10:30 grouinos you can put your ssd in your ass 10:31 Tusha But the computer was failing besides the o key? I mean, it could boot? 10:32 independent_ Yeah, it could boot. It could do anything. But it can't do anything. Logging in is impossible. daily functions are. because the keyboard is screaming "oooooooooooo" 10:33 independent_ She can't write an email. She can't open files (as typing will open upthe search prompt). 10:33 Tusha I see, but that doesn't have anything to do about repairing the ssd, as the store said, there is nothing to repair there to start with 10:33 sfan5 I'm pretty sure you can stop it from doing that if you clean the keyboard somehow 10:34 independent_ I meant that the store said "they can't repair the computer". The ssd is fine. Sorry for the mistake. 10:34 Tusha yeah, probably a good vacuum cleaner could help with it. If you have the laptop model chances are there are a few videos on youtube about how to open it 10:34 Tusha Oh ok, that makes sense independent_ don't worry :) 10:35 independent_ But she is getting a new computer. hopefully the placebo effectcan make her more productive or something. 10:36 independent_ And there will at least be a spare computer. 10:36 Tusha It depends for what she uses the computer, if she is using MS Office, sadly getting outside of windows will impact her, if she just uses it for web-browsing and web applications, a transition to a linux distro with win-like desktop distribution could be easy 10:37 independent_ She does a lot. She has to downlaod bank things and view/edit them. She is fine with the linux mint computer i gave her, but i can't see things staying like that. Windows is the only thing she is even close to being fluent in. 10:38 independent_ The spare computer is my sister's. It's HDD was failing, so i gave it a strong ssd and installed linux mint. I find it impossible to create a windows boot disk without a windows OS. I will get her windows hen i can. 10:39 Tusha Uhm, so maybe Excel is beign used, I guess LibreOffice Calc is one of the nearest feture-wise replacements for ms office, but sadly it isn't enough 10:42 Tusha If you have an spare usb drive I think you can create a win10 installation from linux 10:43 independent_ I did try. Using balenaetcher, it wouldn 10:43 independent_ 't work 10:44 Tusha I would suggest rufus, it hasn't failed me yet, at least on relative new computers 10:45 Tusha Also there is a itsfoss guide about win10 boot creation using ventoy 10:47 Tusha https://itsfoss.com/bootable-windows-usb-linux/ 10:48 Tusha I would try rufus first, and if it doesn't work then give a try to that tutorial, since ventoy seems overcomplicated for what it does 10:48 independent_ intresting, willhave a look when the computer isn't being used for family-critical stuff and mum has her computer with all files. 10:54 Tusha If you want to backup data with a 90% chance of not missing anything, "x:/Users" contains all the user data, and I would copy it using a dedicated program or on linux to avoid permissions issues 10:55 Tusha If you want to keep the entire filesystem, then a ddrescue backup (which allows to omit non-used sectors) could be the safest bet, although re-mounting the image isn't completly straightfoward though 10:58 independent_ hmm 10:58 independent_ Maybe i just give her the SSD and say "copy on demand" 11:00 Tusha As long as you have a ssd usb adapter, yeah that could be the safest bet to not be blamed for anything 11:01 Tusha Although you will need to buy a new ssd for the new laptop, if you plan on using the old one too 11:14 Tusha Anyone knows if using btrfs can extend an hdd life compared to default config ext4? 11:28 erlehmann ported the meshnode mod to mcl2-derived mods and got the elusive half-bed as a reward https://mister-muffin.de/p/lGoG.png 11:58 Tusha Is possible to change the first-person tool animation? I haven't seen anything about it so far, and even weapon-oriented games/mods still use the same animation when the weapon is used (so far checked capture the flag and the weapons mod) 12:01 sfan5 nope, there was a PR for it once I think 12:01 Tusha So sfan5, it is implemented in core then? 12:02 sfan5 the animation is hardcoded client-side yes 12:03 Tusha Dang, well better know about it that trying to find how to change it and not fiding anything, thanks :) 12:04 Tusha Also I guess support for changing textures inside a tool isn't supported too? At least I saw that the compass just changes the holded object instead of updating just the texture 12:06 sfan5 indeed 12:21 erlehmann https://git.minetest.land/Mineclonia/mcl_meshnode/pulls/10 12:33 MTDiscord "Is possible to change the first-person tool animation?" There is a roundabout way to accomplish this. You can disable the first person using "player:hud_set_flags({wielditem = false})" and then attack an item entity to the model's hand, set to be visible in first person and to match the player's wielded item in an on_step. A bit unwieldy and I'm not fully sure if a publicly released mod can do this yet? But the end result is that 12:33 MTDiscord changing the first person animation is as easy as changing the third person animation, now that they're one and the same. An empty hand will be invisible in first person by default, but I'll let you decide if that's a big deal or not. 13:05 Tusha Noodlemire I could explore that, fixing the invisible empty hand could be accomplished by creating an item that resembles the hand itself, seems like a complete roundabound fix to a problem that shouldn't exists, I guess :P 13:18 MTDiscord Roundabout solutions to problems that shouldn't exist are the best solutions. :) 13:19 independent_ What is the worldedit command to replace anything that isn't "air", "advtrains:dtrack_st", "default:torch", and "default:stone" with "default:stone"? 13:32 erlehmann regarding hands 13:33 erlehmann cheat clients can help you to lend someone a hand 13:33 erlehmann literally 13:33 erlehmann i have a chest will illegal items somewhere and one of it is a player hand 13:33 erlehmann will → with 14:04 specing lol 14:32 Tusha This is a bit offtopic but I will ask, I have a forgotten ubuntu server machine which I intend to use as a relay server of sorts, and was thinking on setting some kind of bouncer for irc. I reduced the list to TheLounge and soju, which would be the recommended one? 14:34 sfan5 only ever heard of the former, supposedly it's nice 14:40 specing ZNC? 14:40 specing Tusha: https://matrix.to/#/#minetest:libera.chat?via=libera.chat 14:41 specing substitute channel name as you wish, they all work 14:41 Tusha Yeah But I wanted something direct, without vector getting in the way, I don't completly trust them 14:42 Tusha I though theloungue is a full featured hosted client which can also act as a bouncer 14:42 Tusha I though on ZNC, but reading about bouncers seems soju is more lightweight 14:44 Tusha https://git.sr.ht/~emersion/soju https://github.com/emersion/soju 14:44 sfan5 looking at which is more lightweight does not make sense at this scale 14:45 sfan5 znc consumes 10MB of RAM and barely any CPU on my server 14:46 Tusha I think it was more how it is configured-wise than resource-wise, but since I don't know any in depth I'm just repeating someone else opinion 14:47 Tusha Just for reference, here's where I read it: https://old.reddit.com/r/irc/comments/lh12hu/which_is_the_best_irc_bouncer/ 14:56 Tusha How much resources thelouge uses? I'm more concerned about the CPU, as the PC has 8GB RAM but it is an intel AoC (or however is is said, can't recall the acronym) and as far as I know it will be the bottleneck 15:11 jonadab mazes_81: The autocrafter is a pipeworks component, yes. 15:12 jonadab Though theoretically it could be extracted and made a separate mod; the _core_ autocrafter functionality would be useful even if pipeworks were not installed. 15:12 jonadab Being able to pipe things in and out makes it much MORE useful though. 15:14 * jonadab has a love-hate relationship with pipeworks. It's just so darned useful, you put up with the lag it causes, and the need to constantly check for broken tubes. 17:27 erlehmann so i have a voxelmanip that takes a bunch of node from HERE and puts them THERE. problem: they are chests, they spawn chest entities via LBM. 17:27 erlehmann is there some way to let the voxelmanip run the “run at every load” LBM upon placing the node maybe? 17:29 MTDiscord you can call the func yourself directly after you've committed the vmanip data to the map 17:40 independent_ https://imgur.com/a/3yuFeAq What do you think of this monstorus viaduct (i built this as a preview for other builders to get an idea) 17:41 erlehmann i must admit i do not understad the SCCSM push 17:41 erlehmann understand i mean 17:42 erlehmann i have consistently seen in the wild that people who want to execute arbitrary code on some random computers that are not theirs produce code that is horrible in both performance and security time after time 17:42 erlehmann i mean, if huge corporations can't get their javascript right with millions of dollars … 17:43 MTDiscord Consider 17:43 MTDiscord Every fucking client server video game 17:45 MTDiscord By the way, erlehmann, I've seen you doing a lot of stuff with meshnode. It's super outdated and the author is gone, though. I have a mod that does the node-as-entity thing that is much more up to date and complete https://github.com/GreenXenith/lvae/ (yes, this is a shameless plug). It's just an API mod, so theres no ships or anything-- that will come in a later mod. I figured you might be interested in taking a look, maybe you could 17:45 MTDiscord write your own ship mod, or port meshnode's ship-making system. 17:47 erlehmann GreenXenith nice. is there any way i can use your stuff today? 17:48 erlehmann Jordach in fact, some of the best games do not ever send executable code, but highly structured data 17:48 erlehmann Jordach regardless, popularity is a useless metric to evaluate maintenance & performance & security 17:49 MTDiscord What do you mean by "use it today"? Of course you can download and use it? It is public after all... If you mean using it for ships, again, you'd need to make your own ship constructor mod using it or port meshnode's construction system. 17:49 erlehmann GreenXenith is there any mod that uses that API? 17:49 MTDiscord There is not, as far as I know 17:50 MTDiscord I havent advertised it much 17:50 MTDiscord (at all, until now) 17:50 erlehmann please do 17:50 erlehmann sadly, i have little energy. i will continue improving meshnode for now. 17:51 MTDiscord That's fair; I'll let you know when I have a complete replacement for meshnode. 17:52 erlehmann GreenXenith are stairs and slabs supported? 17:52 erlehmann also panes 17:52 erlehmann and fences 17:52 erlehmann -- NOTE: Most of this file is a mess and is likely broken. Could use a rewrite. 17:52 erlehmann lol 17:53 erlehmann i like the ASCII art here https://github.com/GreenXenith/lvae/blob/master/nodebox.lua 17:54 MTDiscord Every node except liquid and plantlike_rooted is supported 17:56 MTDiscord Nodeboxes are iffy 17:56 MTDiscord So fences, panes, and stairs should in theory work, but in practice they may break 17:57 MTDiscord (Oh, also glasslike_framed is not implemented) 17:59 erlehmann what is it with the liquid filled glasslikes 18:00 MTDiscord Ah yes, I forgot glasslike_liquidlevel was a thing 18:01 MTDiscord Lacks support for glasslike_framed/liquidlevel and plantlike_rooted. Fortunately the glasslikes are not commonly used. 18:12 Tusha Uhm, could the twitch bot let us see the others' teams creations, so far we can see only an small fraction of the winner 18:14 erlehmann i have seen https://github.com/minetest/minetest/pull/11567 and while i have not tested it, i want to say that i have seen “lets use sqlite instead of the file system” a few times now and strongly advise against storing blobs in a database. have implemented a build system (redo) and want to note that usually approaches relying on atime/ctime/mtime are much simpler and faster. benchmarking is really important h 18:14 erlehmann ere – because of course you can have a lot of small files, in which a database wins. 18:14 Desour lol, twitch bridge to irc 18:15 Desour #11567 18:15 ShadowBot https://github.com/minetest/minetest/issues/11567 -- Use an sqlite database for the media cache by Desour 18:15 Desour media is a lot of small files, actually 18:15 erlehmann for example, apenwarr redo uses sqlite “for speed” (i.e. to cheat on benchmarks), as it is possible to beat it with a shell script using the filesystem as a k/v store. 18:16 erlehmann Desour, with *a lot* i mean “you may run out of inodes” 18:16 erlehmann but regardless, robust benchmarking with real-world data is important 18:17 Desour well, I have done some benchmarking. and the loading from media cache, and updating new files, does not seem to be a bottleneck 18:17 erlehmann Desour, if i wish to benchmark your approach, what to run? 18:17 Tusha Sorry that was in the wrong channel, my apologies :P 18:18 Desour erlehmann: start the game and watch the terminal 18:18 Desour and look into the code to see what it benchmarks 18:18 Desour and fill your db. there's a script in the title comment 18:19 erlehmann so how do i figure out if it is faster or slower than the file approach *on my machine*? 18:20 Desour I've also added some timers to (nearly) master. see link at https://github.com/minetest/minetest/pull/11567#issuecomment-904933673 18:20 Desour but note that there's some things that I still have to do to improve performance. ie. the indexes 18:20 erlehmann nice 18:21 erlehmann Desour as the db fill script exists, is it executed automatically to populate the db? 18:22 Desour no 18:22 erlehmann why not 18:22 Desour 1) running the lua script from c++ would require installing the dependencies (luafilesystem, lsqlite3) 18:23 Desour so, it'd have to be done in C++ 18:23 erlehmann also, will the caching backend be configurable? i.e. file or db? 18:23 Desour 2) there's no real benefit. it fills automatically anyways 18:23 erlehmann ah ok 18:23 Desour no 18:26 erlehmann Desour wait if you remove the old caching are you invalidating everyones cache once it is changed? o.0 18:27 Desour no, it just stays there 18:27 erlehmann so it still gets loaded? 18:27 erlehmann but not saved? 18:27 Desour I thought ppl might have some nostalgic feelings about their cache 18:27 Desour it's just there and not used 18:28 erlehmann so if i joined a server that has a TON of mods like meseclams where it can take several minutes to connect 18:28 erlehmann then the change would mean ALL of that is downloaded again? 18:28 Desour !mod meseclams 18:28 MinetestBot Desour: There are no results for this query :( 18:28 Desour yes 18:29 erlehmann weelllllll … 18:29 Desour new players would have to download it too 18:30 erlehmann on the test machine i am typing the cache is 431M (as reported by du), on another machine on which i play more it is 943M (as reported by du) 18:32 erlehmann lets say an average cache is 500M. there are currently around 500 players reported by “minetest-servers clients_list |wc -l” 18:32 erlehmann have i mentioned i am on a metered connection? 18:32 Desour they prolly don't have to redownload the whole cache 18:33 erlehmann if you want to improve the caching system i think it is imperative to import the previous cache 18:33 Desour ppl like you can still import the previous cache with a script 18:33 erlehmann “ppl like you” 18:33 erlehmann what the fuck 18:33 MTDiscord Is 500M really a reasonable cache size? I clear mine on every reboot, but I don't think it gets above like 5M... 18:34 Desour mine is also around 450M I think 18:34 erlehmann look most users are not technical and i know ppl who have worse connections and worse computers than my T60 and still run minetest 18:35 MTDiscord Do you play a bunch of different games/servers, or do they just have huge media or something? 18:35 erlehmann not importing the existing cache to me seems as misguided as the “yes our website loads 10MB of javascript at startup, but after tht it is blindingly fast” 18:35 MTDiscord The cache folder of people who don't ever clear it can become super big...my server alone adds 70-80 MB of media 18:36 erlehmann Warr1024 i play on like 3 servers regularly, my cache folder of >900M represents prob 8 or 9 servers 18:36 Desour if coredevs want me to add code to populate the new cache with the old stuff, I will do that of course. but I hardly doubt that's they consider it worth adding more code complexity, just to make ppl wait once shorter 18:37 erlehmann well the code is already really complex and i don't really get for what gain if not performance 18:37 MTDiscord But it's not just once...it's once per server they play on 18:37 MTDiscord Haha, okay, last I checked my server, it was 2MB, and I don't bother worrying about keeping the cache around ... but I've also had some users on shitty satellite internet who have all sorts of trouble joining, including media... 18:37 erlehmann usually on a shitty connection, once you get the media, it is workable 18:37 MTDiscord Cache is shared across servers if there's a hash collision, isn't it? 18:38 Desour IhrFussel: yes, but it accumulates to a constant (the cache size) ;) 18:38 erlehmann i play on 3 different servers since they have different mods ofc (mineclonia and meseclams mostly) 18:38 erlehmann i mean 18:38 erlehmann mesecraft 18:38 erlehmann and a bit of mineclone5 18:38 Desour warr: yes 18:38 Tusha My cache, which I haven't cleaned after playing on 3 servers and tested several mods and games, adds up to 910MB 18:39 erlehmann “if coredevs want me to add code to populate the new cache with the old stuff” – considering that the purpose of a cache is to AVOID new downloads, i hope someone requires that! 18:39 Tusha Also I would add that current in-game ContentDB doesn't like crappy connections 18:39 Desour request it in the PR and see what the coredevs say 18:40 erlehmann or running out of inodes lol 18:40 Desour >the purpose of a cache is to AVOID new downloads 18:41 Desour it's purpose is to avoid frequent re-downloads 18:41 erlehmann if you don't agree that it is at least counterproductive to invalidate everyones cache at once i don't see what you are up to 18:42 MTDiscord Many players live in countries that limit the traffic per month or you have to charge up to get another GB of traffic...and losing the entire cache in those cases sounds horrible 18:42 MTDiscord The most sensible way to manage a cache would be to keep track of how much value is being gotten out of each object in it (e.g. how much data has been avoided downloading due to a cache hit) over a recent time period, and evict items offering the lowest value if the cache is ever too large. 18:42 erlehmann i am actually on a connection that gets throttled after a few GB 18:42 Desour the benefits of the PR are stated in the PR: cache cleaning (and the possibility to store media zlib-compressed, for the future) 18:43 erlehmann for what reason was cache cleaning based on atime/ctime/mtime not considered? i mean, i can do that in a shell script in the background. 18:43 erlehmann in fact, i happen to be an expert on the topic of doing exactly that, which is why i chime in. 18:44 MTDiscord The benefits of zlibbing media, where most of the bulkiest stuff like sound and images is already compressed, sounds a bit dubious to me? 18:44 erlehmann lol yes 18:44 erlehmann what Warr1024 says 18:45 MTDiscord There are some things like obj meshes or locale files that could benefit a lot from zlib, but in that case it would make more sense to just support .obj.gz or .tr.gz files directly as media on the server-side, or have the server zlib them upon loading, so they're transferred compressed as well, and only need to be unpacked into memory for actual use... 18:45 MTDiscord But either way if we added zlib to any media, it would make more sense to be selective about the types it applies to. 18:45 MTDiscord AND not limit the application just to the cache... 18:45 Desour Warr102: yes, but approximations of that should be fine too. also note that this approach would mean that big files are favored (because they save more data), but they also take up more space 18:46 erlehmann so what is the problem with existing cache eviction techniques based on the file system? 18:46 Desour >for what reason was cache cleaning based on atime/ctime/mtime not considered? 18:47 Desour multiple reasons. ie. databases are useful and easy. and I don't have to fear to run into synchronization issues 18:47 MTDiscord Yeah, probably one of the reasons why we don't have a lot of compression features is because much of the media that would benefit the most from compression (e.g. obj, tr) are already pretty small and don't take up as much of the media space as, say, ogg files. 18:47 erlehmann > databases are useful and easy 18:47 erlehmann lol no 18:47 erlehmann useful yes, easy not at all 18:47 Krock btw, there's a file limit for FAT32-formatted SD-cards (for example) 18:47 Krock hence in favour for database-based or aggressive deletion policy 18:47 Desour >The benefits of zlibbing media, where most of the bulkiest stuff like sound and images is already compressed, sounds a bit dubious to me? 18:48 Krock compression is futile on most assets. there's no need for that 18:48 MTDiscord Plain files have some transparency advantages, esp. for storing binary blobs ... but then, MT also moved its map format away from file-per-mapblock and into databases due to filesystem non-idealities.... 18:48 erlehmann Krock considering that minetest cdb stops working once inodes run out, i am in favor of detecting this condition 18:48 Desour many of our png images are very small. and there are some media files that might still be compressed very well, ie. translation files, .obj, sscsm .lua 18:49 erlehmann Warr1024 the map *always* consists of many small things though 18:49 erlehmann and those are much smaller than media typically (with the exception of inventories and books) 18:50 Desour >useful yes, easy not at all 18:51 Desour if you do more than just writing unstructured data, and also want some of the db features, databases are actually relatively easy. you just need to know how to use them (ie. know sql) 18:51 Desour >transparency advantages 18:51 Desour what do you mean with transparency? 18:52 erlehmann i know enough sql to shoot myself in the foot not too often (i.e. in production) 18:52 erlehmann Desour the database blobs are opaque to almost everything. filesystems are not. 18:53 Desour that's just negating the word 18:53 MTDiscord The filesystem is essentially a universal database designed for storing blobs and which every program on the system that handles storable data can interact with. Specialized databases require you to use only tools that are specifically compatible with it. 18:53 Desour if something is transparent, you can't see it 18:54 Desour Warr1024: ahh, I see. 18:54 MTDiscord Transparency isn't an all or nothing thing. Things can be more transparent if there are more tools that allow access to it in different ways, and more opaque if you're more constrained in how you access them. 18:54 erlehmann and also transparency means the backing storage does not matter 18:54 erlehmann like which filesystem you have does not matter to unix tools 18:54 MTDiscord SQLite, for example, is a database within a file, so it must be by nature at least as opaque as the filesystem it's in. 18:55 Desour do we have many programs or so, that operate on the meida cache? 18:55 MTDiscord The downside to the filesystem is that the places where you can store metadata are limited, so e.g. you can keep track of recency of use via atime, but not frequency of use. That might still be adequate to make a useful cache eviction tool anyway, though. 18:55 erlehmann i ca do 18:55 erlehmann find cache/media -type f |xargs file --mime |cut -d' ' -f2 |sort |uniq -c 18:56 erlehmann Warr1024 redo-sh proves that it is possible to be faster than sqlite if you just rely on the filesystem 18:56 MTDiscord We don't have programs for operating on the filesystem cache because they can be so trivial to make that people who do may not even think about distributing them :-) 18:56 erlehmann 331 application/octet-stream; 18:56 erlehmann 1255 audio/ogg; 18:56 erlehmann 61 image/jpeg; 18:56 erlehmann 36833 image/png; 18:56 erlehmann 1123 text/plain; 18:56 erlehmann that is my 431MB cache contents 18:56 erlehmann now, Desour what is the equivalent of “find cache/media -type f |xargs file --mime |cut -d' ' -f2 |sort |uniq -c” for your database? 18:57 Desour wait a moment 18:57 Tusha Well, I guess that a SQL query could do that, not sure exactly which is needed for SQlite though 18:58 MTDiscord I could frankly see a use-case for storing the cache content as files, but having a small, optional metadata-only database that could be used to better inform cache eviction tools. I'm not sure if overall a bunch of "could" and "optional" is enough to justify the extra work to add one though. 18:58 erlehmann Warr1024 is exactly right, cache eviction with unix shell is a oneliner 18:59 erlehmann Warr1024 that is the approach that apenwarr redo takes … so far it performs worse than every tool i have seen which stores the metadata in the filesystem as key/value store 19:00 MTDiscord Yep, it's a heavy trade-ff 19:00 erlehmann the thing is, you don't easily get enough metadata to justify it for any cache eviction strategy i know of 19:00 MTDiscord you'd need to have a very strong need to do very accurate cache eviction to warrant doing that. 19:01 MTDiscord Cache eviction is ultimately guesswork at some level though no matter how sophisticated your statistics are 19:01 erlehmann i actually wrote my own build system because i wanted a very accurate up-to-date calculation (no unnecessary rebuilds, no skipping rebuilding if it is necessary) 19:02 MTDiscord Caching with a rich/complex database is an interesting academic exercise but it might be overengineered for what we've got in MT. It might be useful as a research tool to help us find out if what we assume we know about how caches behave in MT matches reality, but we probably don't have enough hard evidence to suggest it's a good idea for production use. 19:02 erlehmann regarding the 36833 PNG images in my cache, they are already zlib compressed most likely 19:03 MTDiscord PNG files with no compression are I think technically possible according to the spec but I've never seen one in the wild :-) 19:03 erlehmann just in case anyone wonders how to delete files with unix tools 19:03 Desour is there something like `file` that takes the file data and not a path? 19:03 erlehmann somethig like this: find /path/to/files -type f -mtime +30 -exec rm {} \; 19:04 erlehmann Desour pipe it to stdin 19:04 MTDiscord file can read stdin 19:04 Desour ok 19:04 erlehmann ; printf '' |file --brief --mime - 19:04 erlehmann text/html; charset=us-ascii 19:04 erlehmann # exited 0 0 19:09 erlehmann Desour is writing the SQL taking time or running it? 19:09 erlehmann i mean i have no idea how much more complex it is in SQL 19:09 Desour writing the bash is taking time 19:16 erlehmann Desour but i already posted it: “find cache/media -type f |xargs file --mime |cut -d' ' -f2 |sort |uniq -c” 19:16 Desour yeah, I'm not good at bash 19:16 erlehmann well, bash is very complex, try to keep to bourne shell (sh) 19:16 erlehmann i can help 19:17 erlehmann want to show what you have? 19:17 erlehmann Desour may it be that databases are easier for you precisely bc you can't do the shell stuff and can do the db stuff? 19:18 Desour tbh, I don't do db stuff often. but shell stuff neither 19:18 erlehmann oh 19:18 erlehmann well then i just wait until you show something and i'll tell you if it can be improved 19:18 erlehmann (if i see something) 19:22 Desour I want to do something like ` | xargs echo " bla"` 19:22 Desour how do I do that? 19:24 Desour ahh, -I 19:24 erlehmann eh 19:24 erlehmann are you *sure* you are doing the right thing? 19:24 erlehmann Desour use printf it is portable, echo is not 19:26 Desour I'm currently trying this: sqlite3 media_db.sqlite "select sha1 from files limit 10;" | xargs -d '\n' -I '{}' `sqlite3 media_db.sqlite "select data from files where sha1='{}';" | file --mime /dev/stdin` 19:26 Desour but files reading from stdin is complicated 19:27 Desour all the shell stuff is made for paths, not for raw data, that's my main problem I think 19:32 Tusha Doesn't sqlite has ways to extract the blob info you want instead of the raw data itself? 19:33 Desour what do u mean with blob info? 19:33 Tusha Like, size, mime tipe, and that stuff, although I could just have missunderstood what you are trying to accomplish 19:34 MTDiscord The SQLite command line tool is not really useful for extracting individual items out into files or separate streams 19:34 MTDiscord Piping them to file to discover type based on file magic is actually sort of a worst-case for SQLite in particular. 19:35 MTDiscord The idea is that you have a database that lets you define more or less arbitrary metadata and handle it reasonably efficiently, so it only makes sense to use really if you're actually adding metadata. 19:37 Desour Tusha: I can get the length of a blob, but not the mime, afaik. see also https://www.sqlite.org/lang_corefunc.html 19:37 Tusha Just fiddling with DB Browser for SQlite and seeing if there is an internal way of checking that 19:38 Tusha I see, somehow the db reader detects I saved a .png, but it may be an extension inside the program itself 19:38 MTDiscord More useful would be to get information that only SQLite could actually store, like how frequently the file has been accessed over a certain time window. To do that, though, you'd have to also actually store that data. 19:38 Desour yeah, reading stuff that was never stored is quite impossible 19:39 MTDiscord Desour: Well, most file formats have a 4-byte magic number / header at the beginning usually. 19:39 Desour Tusha: the PNG stuff that you see in the data is the filedata. the file format has some magic bytes in the beginning, to allow things such as `file` to get the file type 19:39 MTDiscord Shell script's advantage here is that it's very easy to extend with utilities and scripts, so it can just use file to infer the type at the time the script is run. SQLite doesn't let you add extensions like that nearly as easily. 19:40 erlehmann Desour i have an opinion on the “it is easier to do stuff using a database” regarding this and many similar tasks (like cache eviction) 19:41 MTDiscord For an MT specific cache you could probably just add the magic bytes to a table and join on that ... but there are ambiguous cases (locale and obj files both look like text) and recreating the work of the file utility in SQL is just too much work to prove a point :-) 19:41 MTDiscord Well well 19:41 MTDiscord Databases are noice and abstract away all the FS bewlshit 19:41 erlehmann look once you are starting stuff like “join on magic bytes” you are just putting lipstick on a pig 19:41 erlehmann luatic databases ARE nice but you need to have a problem that fits the solution 19:42 MTDiscord Databases are always the solution 19:42 erlehmann and i have my doubts that the caching problem is even in the same ballpark 19:42 MTDiscord Say, checking for old files 19:42 erlehmann yeah, if you have a really big hammer you can feel like you can nail the whole world 19:42 erlehmann if you get what i mean 19:42 erlehmann (i mean exactly what i wrote) 19:42 MTDiscord SQLite lets you have an index, so this is actually logarithmic 19:42 MTDiscord (this is what nrz tells in a comment) 19:42 MTDiscord A filesystem would require linear cache 19:42 MTDiscord Which is just ugh 19:42 erlehmann the point is you nneed to have A LOT of records to make this outperform the filesystem 19:42 MTDiscord search* 19:43 MTDiscord No, databases outperform poor filesystems all the time 19:43 erlehmann years ago i did live maintenance on a database with hundreds of thousands of records. you usually don't want *that* in files. 19:43 erlehmann and yes it was an emergency 19:44 erlehmann i was assured i would not be fired 19:44 erlehmann if it went haywire 19:44 erlehmann (it worked out) 19:44 MTDiscord Well, nowadays, I'm not much of a fan of "throw SQL at it" anymore. But I'm still a fan of databases. 19:44 erlehmann luatic you always need benchmarks. there are so many tools that make sense if you have problems the size of facebook or google or amazon. 19:45 MTDiscord SQL is where most bloat comes from anyways, which is why simple k/v DBs are so popular nowadays 19:45 erlehmann and then there is the COST 19:45 erlehmann the filesystem is a simple k/v store 19:45 erlehmann with a cache 19:45 MTDiscord yes 19:45 MTDiscord but without proper indexing 19:46 MTDiscord it only indexes on filenames, usually, and even for that, it doesn't even sort them 19:46 MTDiscord it's also different from platform to platform 19:47 erlehmann > it only indexes on filenames, usually 19:47 erlehmann oh lol 19:47 erlehmann have you read the COST paper? 19:51 MTDiscord Databases in general are a good solution to a lot of data storage problems. There are a lot of specialty databases that are good for specific sub-problems. For instance, for storing a lot of blob data in essentially a tree-structured key/value store with a bit of metadata, filesystems are a pretty nice kind of database for that application. 19:53 Desour I give up. xargs is too difficult for me 19:54 MTDiscord I'm actually curious if there's any quantitative empirical evidence for the SQLite approach? Like, are there measurable savings or improvements it makes? 19:55 MTDiscord In theory these kind of things can just be better in some cases and worse in others, and it would be nice to know e.g. if there's an actual crossover point. 19:55 Desour (but yes, I see the transparency issue that you were talking about) 19:56 Desour btw., other backends could be added later, of course 19:58 erlehmann the “in theory” thing has the problem IMO that it leads to the kind of stuff that apenwarr redo is 19:59 erlehmann as i said, using sqlite “for performance” has to be justified with benchmarks 19:59 erlehmann and tbh, if your stuff performs about the same i would gladly take the battle-tested less complex solution that allows me to write my own cache eviction strategy in one line of shell script 19:59 erlehmann i volunteer even to compete if its just about clearing up the cache 20:00 erlehmann like what approach can handle the problem faster 20:00 erlehmann (for the same algorithm) 20:01 Desour the reason why I chose sqlite was not performance 20:01 erlehmann well it obv can't be ease of use if the “count the mime types” task is any hint of future usability 20:01 Desour (at least not the top reason, I hoped for not making anything worse, of course) 20:01 erlehmann what is the top reason though? 20:02 erlehmann the zlib everything thing is a bad idea, we already know that 20:02 celeron55_ i'd like to see a cache eviction implementation for the files without using a database, but before that i'd like to hear how you'd do it as the filesystem, given all the platforms we support, doesn't give you that much metadata 20:02 Desour the problem with the count the mime types is that I'm too unexperienced with the shell. I couldn't have written the one liner you presented before 20:03 erlehmann then do it in pure sql? 20:04 sfan5 celeron55_: atime and mtime are POSIX but I don't think it's wise to rely on them (esp. atime, which may be disabled) 20:04 erlehmann i agree that atime is not a very good measurement 20:04 celeron55_ we all know the fs based cache works well and has worked well for 9 years already, and the only thing missing is eviction 20:04 Desour that's like saying "do it in pure VHDL". sql isn't made to operate with things such as filetypes. (and shell couldn't do it either without the tools you've installed (ie. files)) 20:05 celeron55_ the question is, what's the best solution from this starting point 20:05 erlehmann i think the best solution is trying to figure out 1. ideas for when to evict 2. what strategies to use 20:06 erlehmann i already pointed out one point of when to evict the cache, when you run out of inodes. granted, that's an extreme case. 20:06 erlehmann lets say you do it based on cache directory size, then the naive strategy is to get rid of files that have not been touched longest first. 20:07 erlehmann no additional state would need to be maintained for that naive solution 20:08 sfan5 bad strategy, there are likely files that are very old (because every server has them for a long time) yet are often used 20:09 erlehmann i meant “not used” with “have not been touched” 20:09 sfan5 well how do you determine that from file metadata 20:10 erlehmann if atime is not a useful concept, one can just update ctime by slightly tickling the file 20:12 sfan5 workable idea 20:12 erlehmann celeron55_ anything except windows that is not POSIX stat(2) compatible? 20:12 sfan5 that merely reading the files will cause disk writes is too bad but ¯\_(ツ)_/¯ 20:12 erlehmann technically, having atime does that already 20:13 erlehmann and noatime disables it lol 20:13 erlehmann how about 20:13 erlehmann using atime and whoever does explicitly use noatime can enjoy their breakage (lots of stuff breaks then AFAIK) 20:14 celeron55_ one option is to use the files, and add sqlite on the side just for the metadata 20:14 celeron55_ it's kind of silly, but so is storing files in sqlite 20:14 erlehmann yeah but the thing is, if you do that, you can also just save the metadata in the filesystem 20:15 erlehmann in redo, i just have filename.md5sum and filename.ctime for freshness checks 20:15 erlehmann its ugly but it works out 20:15 celeron55_ but you really need more metadata than just the timestamp 20:15 erlehmann well what do you need? 20:15 erlehmann doing that kind of stuff works really well actually, considering i use the read builtin from bourne shell and some implementations make that read 1 byte at a time 20:16 erlehmann like i was surprised it as so fast 20:16 celeron55_ https://github.com/minetest/minetest/pull/11567#issuecomment-904888258 20:16 erlehmann but compared to even loading sqlite into memory, ofc it is 20:17 erlehmann celeron55_ my suggested naive expiration strategy is different than the one you suggested there. it is just “to stay under a threshold, delete the files where the access data is the oldest”. this does not have the expiration problem you pose at all! 20:17 erlehmann access data → access date 20:18 erlehmann celeron55_ what problem does the two-stage expiration solve that my naive approach does not? 20:18 erlehmann i.e. in which situations is it actually better? 20:19 celeron55_ "stay under a threshold" probably works fine too, as long as the threshold is wisely set 20:19 celeron55_ so that would indeed work with the filesystem 20:20 erlehmann yeah but what made you come up with the two-stage expiration? there *must* be a motivation for that 20:20 celeron55_ we just have to calculate what the maximum reasonable cache size is, given how many servers and games a user normally plays and what the average data size per server or game is 20:20 erlehmann ppl don't just pull this out of thin air 20:21 celeron55_ what's the comment missing, it already has my reasoning 20:22 erlehmann ok if it does not get any more sophisticated than that, i guess that is it then. 20:22 erlehmann i mean it is a simple strategy, maybe i was expecting a complicated explanation ^^ 20:22 erlehmann but you delivered a simple one 20:22 erlehmann so ig i thought there was more to it and there is not 20:23 erlehmann thanks 20:23 celeron55_ lol 20:23 celeron55_ anyway, if someone can come up with a reason why erlehmann's proposal wouldn't work, please do 20:26 celeron55_ of course if there's a weird corner case or obscure platform where it doesn't work, they can just disable the cache; in practice cache eviction can be done manually or never as everyone indeed has done so far 20:26 erlehmann yeah that is the beauty of it, it literally can't get worse lol 20:27 erlehmann i must say i have rarely cared for the cache, is it a recent thing that ppl care? are they playing minetest on 4GB sdcards on raspis or something? 20:27 Desour I don't care much how big my media cache folder is (as long as it's not tooo big). and I don't want to keep metadata of servers where I didn't play for ages. if the max cache size is too small, it will get useless, though. therefore I prefer a time-based strategy. 20:28 celeron55_ i'd guesstimate the cache threshold size could be 500 MB or something like that. on the machines MT is playable, that size barely counts but it already covers tens of servers or games 20:28 erlehmann well, that is not a problem, you can have both 20:29 erlehmann celeron55_ 500MB is most likely too low. let me check something. 20:30 celeron55_ also, there could be multiple thresholds, like 1 year @ 500 MB and 1 month @ 2 GB, or whatever 20:34 erlehmann yeah 20:34 erlehmann it's fairly simple 20:34 erlehmann you can also have multiple strategies at the same time 20:35 erlehmann let me check something 20:35 erlehmann i renamed my cache dir to cache2 20:35 erlehmann and made a new empty one 20:35 erlehmann i will now connect to a few popular servers and a few others i play on and report back 20:40 erlehmann server 1: your-land.de:30000 cache size 86M 20:41 erlehmann server 2: rubenwardy ctf. not accepting new players. 20:44 erlehmann server 3: edgy1.net (catlandia, not a dating server) cache size 126M 20:44 erlehmann and yes i am testing on a machine that has not a metered connection lol 20:44 erlehmann (i.e. not the one i am writing from) 20:47 erlehmann server 4: kay27.ignorelist.com (the mineclone5 test server) cache size 165M 20:49 Desour are errorstream and co. thread-safe? 20:51 erlehmann server 5: kay27.ignorelist.com (mineclone2) cache size still 165M 20:51 sfan5 I don't think so but does it really matter? 20:53 Desour I don't know if it matters 20:56 erlehmann server 6: kay27.ignorelist.com (mineclone5 creative) cache size 167M 21:01 Desour erlehmann: are you always deleting your whole cache or are you checking how many files there are with a new enough timestamp? 21:01 Desour (if you have atime) 21:01 erlehmann i am not deleting any files right now, i am looking what happens if i visit a buch of servers 21:01 erlehmann a bunch 21:02 erlehmann server 7: clam.minetest.land:48140 (mineclonia with minecraft-like textures) cache size 201M (note: loading this stuff took a *long* time) 21:08 erlehmann server 8: mesecraft.net:30000 (mesecraft) cache size 241M 21:09 erlehmann server 9: minetest.foo-projects.org:30394 (inside the box) cache size 272M 21:10 erlehmann server 10: nodecore.mine.nu:30000 (nodecore) cache size 274M 21:11 erlehmann server 11: skyblock.telesight.nl (telesight skyblock) cache size 276M 21:12 MTDiscord it would be interesting to see those as sort of like a venn diagram or something to understand the probability distribution of media sizes for a "first" server vs subsequent ones, and how much of an impact cross-server reuse has 21:13 erlehmann server 12: server.minetest.one:30001 (braincraft) cache size 307M 21:15 erlehmann server 13: nico.multicraft.world:40779 (survivalgo) cache size 309M 21:15 erlehmann server 14: i start exile locally cache size 315M 21:16 erlehmann server 15: i start repixture locally cache size 320M 21:16 Tusha The only method to compare actual cache with sqlite based cache in terms of speed, although it is a bit crude and black boxy, would be to use 2 clients on the same machine with clear data, and measure first logging load and second login load on the same server, on several servers. Maybe a cache size check could be a plus, but it depends what we want to know in the first place 21:17 erlehmann so i stop now 21:17 Tusha load times* 21:17 erlehmann what i think i gathred 21:17 erlehmann 1. singleplayer games affect the cache. i had not thought of that. 21:18 erlehmann 2. the cache makes a HUGE difference. erroneous cache eviction must not be allowed! 21:18 erlehmann 3. mineclone2/mineclone5/mineclonia … all the same. unless someone uses a different texture pack. 21:19 Desour media cache loading is no bottleneck, you might be able to achive 100ms less waiting times on servers where there are masses of media, but you still have to wait >0.5s. 21:19 erlehmann 4. by visiting a few servers and having a few singleplayer games you can easily get to 500M 21:19 erlehmann loading from the cache is not a bottleneck probably, but erroneously evicting it makes everything so much slower 21:20 Desour it just makes first joins slow again 21:20 erlehmann i gather you have never waited minutes for your friend to come and play 21:20 Tusha I tend to have the same load speeds on servers before/after caching, at least it feels like the same time, as I haven't measured it 21:20 erlehmann bc they have to dl the media 21:21 Tusha I have a good speed, yet it takes about 20 minutes to get into the server 21:21 Desour dl? 21:21 erlehmann if anyone would ask me, i would set the cache eviction threshold at about 1G disk space. 21:21 Tusha Internet speed that is 21:21 erlehmann download 21:21 Desour I don't want 1G of media if half of it is unused 21:22 erlehmann nah because it will be used 21:22 erlehmann i just figured that out 21:22 Desour it will not 21:23 erlehmann look any worlds folder is *easily* easily bigger than that if you play for a few months 21:23 erlehmann way bigger 21:23 Desour I join at servers, play some days, and never come back for months or years => cache for that server not used 21:23 erlehmann oh i think you misunderstand how the caching works 21:24 Desour pls explain what I'm musinderstanding 21:24 erlehmann as you can see in my notes above, if the same asset is used locally or on another server, you profit from it 21:24 Desour not if it's a server with some texture pach, ie. fussels's server 21:24 erlehmann like, all the mineclone2/mineclone5/mineclonia stuff was only one single blob except for clamity using a different texture pack and music 21:24 Desour pack* 21:25 erlehmann as i said, you can combine strategies, like: “prune oldest files when 1G is hit and also prune all files older than a year” 21:26 erlehmann and by older, i mean “not used in” 21:26 erlehmann and by oldest i mean “not used for the longest time” 21:26 erlehmann Desour, would that satisfy you? 21:28 Desour I do like the twice-expiration though 21:28 Desour but, a combination allowing both would of course satisfy me 21:29 erlehmann i think the twice-expiration is a weird idea that probably performs in non-obvious ways 21:29 Desour (if the threshold is configurable, it's basically possible to just turn it off) 21:29 erlehmann like i sometimes do not play minetest for a while, like months 21:29 erlehmann or do not connect to a server for a while 21:30 Desour it's not that weird imo. I'm not sure where, but I think I've seen a similar thing elsewhere already 21:30 erlehmann well the thing is, i like the “solution fits in head” algorithms approach 21:31 Desour yes, and it does fit in the head 21:31 erlehmann mechanically yes. but i can't really reason about it. 21:31 erlehmann i mean, i can, if i put effort in it 21:32 erlehmann but compared to “take out the trash if the cache is 1G” it is harder 21:34 erlehmann in general, for size based eviction the threshold must be comfortably high with safety margin 21:35 erlehmann bc otherwise you can connect to 5 servers and play 3 different singleplayer games and if say, the cache was 250M then it is basically evicted all the time 21:35 erlehmann which would undeniably suck 21:37 Desour well, you can easily get some guarantees from this size-too-big approach: the upper size limit. but from the twice-expire time-based approach you can also get some (more complicated to formulate) guarantees: unused files are removed after some time if you were active in this time 21:38 Desour (I prefer the guaranteed deletion of old stuff) 21:42 erlehmann as i told you, just run all of the strategies lol 21:42 Desour btw. about the compression thing: using the file type for this has the downside that the client can't decide to compress the file independently from the server 21:42 erlehmann look if you recompress stuff you don't gain much space 21:42 erlehmann and you lose the caching 21:43 Desour the sha1 digests in the db are meant to be over the uncompressed data 21:44 Desour recompressing obviously doesn't make much sense. but old servers send ie. translation files uncompressed 21:45 erlehmann i am not trying to be mean here … but i do have the suspicion that you are trying to solve a problem that is not really big enough to warrant attention with the recompression approach 21:46 erlehmann meanwhile, i should probably run “optipng -o7 -strip all” on mineclonia 21:46 Desour yeah, I don't have a strong feeling about the compression thing. it doesn't hurt though 21:47 erlehmann it does 21:47 erlehmann code is a liability 21:47 erlehmann or to say it more brazen: deleted code is debugged code 21:47 erlehmann the best code is code not written 21:48 erlehmann brazenly 21:48 erlehmann it's an adverb 21:48 Desour yes, but it costs nearly no code 21:48 erlehmann it costs code, CPU time and RAM 21:49 erlehmann the space/time tradeoff is not good 21:49 erlehmann you are better off convincing mod authors to compress their stuff well 21:50 erlehmann disk-space and traffic savings-wise 21:50 Tusha I don't think experimenting is a bad thing, prototypes are always neccesary to advance 21:50 Tusha Also there is the experience gain in the process 21:51 erlehmann we have a workable model of caching resources already, it is in web browsers 21:51 erlehmann 25 years of experience 21:51 erlehmann i posit it makes sense to read the existing literature on that *first* and then figure out if anything is missing 21:52 erlehmann except, of course, if it is a thing of having enlightenment 21:52 Tusha Well I was talking about the experience the dev gets from trying it, even if it isn't the best approach 21:52 erlehmann i wrote a quine without cheating 21:52 Tusha lol 21:52 erlehmann here 21:52 erlehmann #!/usr/bin/rc 21:52 erlehmann p='''';q='#!/usr/bin/rc 21:52 erlehmann p=%s;q=%s;printf $q $p$p$p$p $p$q$p 21:52 erlehmann ';printf $q $p$p$p$p $p$q$p 21:52 erlehmann the rc shell is a funny thing 21:54 Tusha Just trying to gasp zsh over sh or bash, and why it could be an alternative, never heard about rc shell 21:54 erlehmann the rc shell is the standard shell in plan9 and solves a bunch of things that always bugged me in sh/bash/zsh 21:54 erlehmann it has a very small grammar too 21:55 erlehmann i suggest to try it if you are looking for something that has very little historical warts 21:55 erlehmann (plan9 design was mostly sound … and incompatible with other stuff) 21:57 Tusha Anything is better than powershell, at least for me xD...