Time Nick Message 04:05 sr20det hi. 'minetest --verbose' outputs 'Aborted' and exits 09:41 fkbm Well, managed to bring Minetest to 9 fps... I guess using meshes is indeed more expensive; now that I think about it instead of 6 faces I'd have 22 09:42 fkbm per node 09:42 erlehmann how fkbm 09:42 erlehmann fkbm, is it legos 09:42 erlehmann are you making lego minetest 09:44 fkbm I tried; well square-gos (top part would be a square, not a cylinder); I'd manage to bring it to 1 fps then :D 09:45 fkbm I do get where the speed penalty comes from, though... even if I'd make it efficient, there's no way I can make eliminating faces viable... You know, like not drawing faces that are between two solid blocks. 11:39 sr20det hi. http://0x0.st/-xVZ.txt can you help me? 11:45 sfan5 doesn't it print the exception it receives? 11:54 sr20det the only output is 'Aborted' 11:57 sfan5 well something is wrong with exceptions either way, check your compiler flags 13:12 wsor4035 sfan5: would https://github.com/minetest/minetest/issues/11544#issuecomment-917792590 be enough to remove unconfirmed bug? 14:06 erlehmann Desour i think your caching thing is among the worst things i have encountered in terms of complexity thrown at a simple problem, but please do not take it as an insult. I do not think you are incompetent, just that all you have is a hammer and then all problems look like nails. 14:07 Desour what are you referring to, erlehmann ? 14:07 erlehmann https://github.com/minetest/minetest/pull/11567 14:07 Desour !title 14:07 MinetestBot No title found. 14:07 Desour ~title 14:07 ShadowBot Use an sqlite database for the media cache by Desour · Pull Request #11567 · minetest/minetest · GitHub 14:07 Desour ah 14:07 erlehmann Desour, I just realized that I should clarify that I do not mean it as an insult. 14:08 erlehmann i just realized that it is worse than i thought looking at it 14:08 MTDiscord wouldn't the sqlite cache database make it take up less space? 14:08 MTDiscord Ideally it might, but unfortunately I don't think sqlite works that way 14:08 erlehmann slightly, yes, but read my comment here: https://github.com/minetest/minetest/pull/11567#issuecomment-918226499 14:08 erlehmann it could be 14:08 erlehmann but i doubt it 14:09 Desour the complexity of the PR somewhat grew over time. it used to be more simple 14:09 erlehmann i mean, originally i thought it was a bad idea with a capital B 14:09 MTDiscord Filesystems tend to "waste" slack space as a speed trade-off, but sqlite does this as well, but filesystems tend to be better at responding to space pressure and rebalancing the trade-off, while I don't think sqlite does. 14:10 erlehmann yeah sqlite stalls for 10s in Desour tests 14:10 erlehmann right? 14:10 erlehmann the thing is: leaving the cache folder just there without importing stuff means: 1. afterwards more stuff will be downloaded 2. afterwards more disk space will be used. 3. afterwards there is more code to maintain. 4. i can no longer use the neat shell tricks i have used in debugging the cache or other stuff easily. 14:11 MTDiscord The PR may have been a useful learning exercise but probably turned into one of those rabbit holes that you end up at the bottom of and find that after all that meandering it didn't get you nearly where you had hoped... :-) 14:11 erlehmann yeah lol 14:11 Desour yes, it did stall 10s in a test, because I made it stall 10s via a sleep for testing 14:11 erlehmann having an “improve the cache” PR that makes the client use more disk space and causes more traffic is not really close to the end goal 14:11 erlehmann oh ok? 14:11 erlehmann but the “breaking local multiplayer” thing is what made me write here 14:12 erlehmann that's the worst 14:12 Desour it doesn't break local multiplayer at all 14:12 erlehmann i use local multiplayer all the time, e.g. to test if protection applies to a node 14:12 erlehmann Q: What happens if two or more minetest instances load media at the same time? 14:12 erlehmann A: In the worst case, the one that comes later silently goes back to main menu (I hope at least, that there's no worse case). There are some warnings and error messages in the terminal and debug.txt about the database being locked. 14:12 Desour the db is just locked at media loading 14:13 erlehmann that's … not cool? 14:13 MTDiscord Sometimes the value you get out of something is not what you were aiming for, e.g. when you hope to get an optimization but end up instead producing evidence that the idea is impractical. 14:14 erlehmann Warr1024 i actually had this with my redo build system. the overhead for parallelization coordination is so big that building in parallel is slower if you have many targets that build in less than 1s. 14:14 MTDiscord It does seem like there might be corner cases where a database cache actually makes more sense, e.g. on Android though, where performance of storage systems might be very different from first principles due to weirdness of stuff like JNI. 14:14 MTDiscord Oh, yeah, parallelization is a great place to run into unexpected trade-off costs :-D 14:17 erlehmann Desour, are you on windows or os x maybe? 14:17 Desour `-´ no 14:18 Desour I'm using linux 14:18 erlehmann then i am really confused 14:18 erlehmann are you aware of atime? 14:18 erlehmann linux fs aren't really slow i mean 14:19 Desour of course I am aware of atime 14:19 Desour many (including me) turn it off afaik 14:19 erlehmann what did you use? 14:19 Desour "use"? 14:20 erlehmann which mount option 14:20 erlehmann are you aware of relatime? 14:20 Desour rw,noatime 14:20 erlehmann it is the mount option that people think commonly of “turning off atime updates” 14:21 erlehmann ah 14:21 erlehmann that is unfortunate 14:21 erlehmann how much performance could you get with it? 14:22 Desour idk, I never did any benchmarks or anything. 14:22 erlehmann you just turned it off? 14:22 erlehmann that seems unwise 14:22 Desour and it's more for increasing my ssd longlivety 14:22 erlehmann ah! 14:23 MTDiscord I use relatime because I suspect that noatime might break stuff unexpectedly... 14:23 MTDiscord My SSDs will just have to suck it up I guess. 14:23 erlehmann noatime does indeed break common applications, such as mutt 14:24 Desour let's make minetest not one of those broken applications 14:24 erlehmann relatime vs atime is the big speedup though 14:25 erlehmann LOL 14:25 erlehmann minetest would not break with that 14:25 Desour the cache clearing would not work properly though if you used atime 14:26 erlehmann what i don't get is why you had to put the files in the db 14:26 erlehmann why not just update the timestamps? 14:26 Desour it made some things simpler 14:27 erlehmann and completely breaks my workflow how i interact with the cache 14:27 erlehmann and probably many other peoples, because it is a one-liner to cleanup old files from the cache 14:27 erlehmann right now 14:27 MTDiscord A nice way to handle cache timestamping would be just update mtime (which nobody disables) if the mtime is more than, say, 1 minute old (to reduce SSD thrash). 14:27 erlehmann Warr1024 relatime is already good enough for it 14:28 erlehmann atime is required by posix, everyone who turns it off basically says “i want some applications, including some backup apps, to no longer work correctly anymore, but have a faster system” 14:29 erlehmann or are there systems that turn it off by default? 14:30 erlehmann Desour, i have another performance boost for you: fsync=off for postgresql 14:30 erlehmann it will make postgresql faster and sometimes eat your data 14:30 erlehmann while we are at it: #define struct union 14:30 erlehmann huge RAM saver! 14:32 erlehmann Desour, i strongly suggest to benchmark both noatime and relatime. you will probably not measure any relevant difference. 14:32 Desour nah, that'd be definitely too much work for nothing 14:33 erlehmann then switch to relatime, you'll probably not notice any slowdown and if anything is broken by noatime, you'll probably not waste time debugging it. 14:34 Desour are you suggesting to not support any system that doesn't strictly support posix? 14:34 erlehmann no 14:34 Desour I haven't had any problems due to atime yet 14:34 erlehmann well, you only notice them when they are too late 14:34 Desour and this is not about me. there are many people that use noatime :P 14:35 erlehmann yes but usually they know what they are doing, like setting it on a NAS or tmp partition 14:35 erlehmann cargo-culting noatime is not good 14:36 erlehmann if you have slow flash memory you probably won't even get atime 14:41 MTDiscord I wouldn't say we "won't support" systems that have some features broken, but we will only provide partial support, i.e. only stuff that depends on the non-broken parts. As far as I understand, MT cache will work fine with noatime, but your "cleanup" strategy in that case is limited to just invalidating and nuking the whole thing and forcing re-downloads. 14:43 erlehmann Desour Warr1024 i think the original suggestion for noatime comes from ingo molnar, who later said relatime is cool 14:45 erlehmann you can see the start of this here http://web.archive.org/web/20110427023154/http://kerneltrap.org/node/14148 14:45 erlehmann relatime was made default in 2009 14:46 erlehmann I will post my oneliner in the issue lol 14:46 Desour I don't really care about the history about atime mount settings right now. :P And I don't plan to change my configuration just because someone from the internet doesn't like it. 14:47 erlehmann it is not that i don't *like* it, it is that i think you have no idea what you are doing because you did not benchmark it 14:47 erlehmann Desour, do you know tmpreaper? 14:49 Desour of course I have no idea what I'm doing. I didn't even consider using freebsd or some other kernel than the default linux yet, and I definitely didn't do benchmarks or read the source code. 14:50 Desour idk tmpreaper 14:50 erlehmann it is a program that deletes files that have not been accessed in a number of days 14:51 sfan5 these days we call that systemd-tmpfiles-clean.service 14:54 erlehmann “tmpreaper 30d cache” or >1000 lines of C++ code, who would win 14:54 MTDiscord I think it's fine if you use a weird configuration, but if you have to jump through a bunch of hoops to work around the limitations imposed by weird configuration, don't be surprised when people don't want to merge those hoop-jumps upstream. 14:55 erlehmann yeah lol 15:02 MTDiscord The ability to accomplish useful things in a very "agree to disagree" kind of environment is sort of one of the superpowers of open-source, so on some level weirdness should be celebrated :-D 15:15 sfan5 wsor4035: sure but I'd need to test it 15:23 wsor4035 thanks 19:02 Desour what Jordach is currently doing: https://thumbs.dreamstime.com/z/cleaning-sky-11320462.jpg 19:03 Krock seems to be an impossible task 19:03 jonadab relatime is very cool 19:04 jonadab Whereas tmpreaper sounds like something I wouldn't inflict on my worst enemy's nightmares. 19:05 sfan5 I put files I want to more or less keep for the duration of my system uptime in /tmp so yeah I ended up disabling the systemd service that something equivalent 19:05 jonadab Tools that automatically delete data need to be *very* specialized and *very* careful about what they delete. e.g., logrotate is ok. 19:30 erlehmann jonadab what is the problem with tmpreaper? you run it manually 19:31 erlehmann jonadab basically tmpreaper is the answer to “why has no one made a tool for cleaning up the cache” – bc it is trivial to clean up your cache with a oneliner, probably even on windows 19:33 sfan5 the whole point is that minetest should do it automatically for users who don't know or care 19:33 sfan5 "just run this command line tool" is not going to impress any windows user and android users will stare at you confused 19:34 erlehmann sfan5 i am not saying that minetest should not do it. it is just one of those tasks where if you really think about it the solution is so simple, that you may as well scratch your own itch and forget the rest. 19:37 erlehmann it is a god explanation, not a good justification 19:37 erlehmann good 19:38 erlehmann but it is funny, on windows it is also a oneliner, but people come up with very long batch files :d 19:38 erlehmann :D 19:44 appguru Today's texture modifier oddness: "[combine:42x69:" creates a blank 42x69 image. 19:45 sfan5 makes sense if you think about it 19:45 erlehmann what is the oddness 19:46 appguru Well, it's odd that the colon is required for one, and it's kind of odd that you can just omit the images like that; the docs don't mention that. 19:46 sfan5 what does "blank" mean though? transparent or white? 19:49 MTDiscord transparenty 19:49 MTDiscord I use [combine:1x1: a lot myself 19:49 MTDiscord it's blank.png with one less media transfer 19:49 MTDiscord [combine:1x1:^[noalpha is a not COMPLETELY terrible way to make a simple black pixel. 19:50 Desour but there's blank.png in the base pack 19:54 MTDiscord Haha, seriously? Ick. 20:33 appguru Warr1024: I'd kinda prefer blank.png^[colorize:black:255 20:33 MTDiscord ew ... but to be fair, I don't think there's a completely NON-gross way to do this... 20:51 jonadab Minetest cleaning up its _own_ data, especially a cache, makes sense. 20:52 jonadab A general purpose tool that wanders around your filesystem looking for "not recently accessed files", sounds like a way to lose important data. 21:00 MTDiscord "Wandering around your filesystem" it really shouldn't be doing. The fact that stuff in /tmp is not guaranteed to survive a reboot (and in fact generally not supposed to) makes that a bit different. 21:16 erlehmann jonadab minetest should totally clean up its own data. but it does not need >1000 additional lines of code for that. 21:17 jonadab erlehmann: Oh, true. 21:17 jonadab Though it's not going to be a one-liner either. 21:17 jonadab It could be in some languages, but Minetest isn't written in Perl. 21:18 Desour btw. many of the >1000 lines are not for the media cache, ie. unittests 21:19 jonadab Shouldn't unit tests only install in the first place, if the user is building from source? 21:19 erlehmann ok let me quickly hack something together 21:19 Desour install? 21:19 jonadab Desour: Be installed. 21:19 jonadab Whatever. 21:20 jonadab The pre-compiled binary packages for end users, really don't need the unit tests in them, IMO. 21:20 Desour I think there's a compiler flag for this 21:55 erlehmann Desour this is probably horrible, but … wouldn't it be only a little bit more code to clear the cache from here? https://mister-muffin.de/p/6ZPz.c 21:58 MTDiscord I smell the beginnings of a draft PR... 21:59 Desour lol, "or" 22:00 Desour well, it's missing most of the things, like synchronisation or integrating it in minetest 22:02 erlehmann Desour synchronization with what? 22:03 Desour between multiple minetest instances 22:04 Desour it basically lacks everything apart from iterating through the dir and getting atimes :P 23:59 erlehmann why would a cache cleaner need syncing between multiple minetest instances? o. 23:59 erlehmann o.0