Minetest logo

IRC log for #minetest-dev, 2021-10-20

| Channels | #minetest-dev index | Today | | Google Search | Plaintext

All times shown according to UTC.

Time Nick Message
00:33 erlehmann pgimeno do you think this helps development? https://github.com/minetest/minetest/issues/11631#issuecomment-947170934
00:34 erlehmann i ask because all i did was figure out that biome generation is more broken than i thought
00:34 erlehmann but i have no solution :(
00:34 MTDiscord <josiah_wi> erlehmann, I think people are looking for solutions, not more issues, haha
00:35 erlehmann so i guess biome generation depends on optimization level and compiler version and possibly CPU architecture
00:35 erlehmann i mean, the mapgen is not reproducible anyway, but was it ever intended to be? is it a goal?
00:35 erlehmann if it is not a goal to get the same map with the same seed it is only a minor issue
00:36 erlehmann if it should be reproducible though, i'd suggest to make it a major one
00:36 MTDiscord <josiah_wi> Apparently mapgen stuff that breaks reproducibility isn't wanted... so I spy a conflict of interest here.
00:36 erlehmann this could explain some strange artifacts we have encountered
00:37 erlehmann josiah_wi where is the conflict of interest, it is already not reproducible
00:37 MTDiscord <josiah_wi> That's what I meant. If we can start PRing fixes for stuff that's another matter. I still need to write unit tests for my PR to fix the noise UB.
00:38 MTDiscord <josiah_wi> And #11656 is higher priority.
00:38 ShadowBot https://github.com/minetest/minetest/issues/11656 -- Add variable to use existing IrrlichtMt build by JosiahWI
00:38 erlehmann yeah so for the noise UB i want to talk to you
00:38 erlehmann the thing is, there is probably one major configuration
00:38 MTDiscord <josiah_wi> Oops that's the issues; meant to link the PR.
00:39 erlehmann i would assume that many ppl have servers on x86_64 using gcc of a specific version etc. pp.
00:39 erlehmann which *might* have predictable results
00:40 MTDiscord <josiah_wi> It isn't guaranteed be predictable if the configuration changes. But it seems to generally be predictable, hence why it's low priority.
00:40 erlehmann josiah_wi how are you approaching the unit tests – there are a limited amount of u16, so we can test them all?
00:40 erlehmann “it seems to generally be predictable”, uh no, it is not
00:41 erlehmann i have had encountered maps that generated slightly different on the same machine, same binary
00:41 erlehmann i never managed to reproduce it
00:41 erlehmann but now it makes sense
00:41 MTDiscord <josiah_wi> Well the core devs aren't freaking out so it's probably ok to be patient.
00:42 erlehmann if they do not freak out then ig the map should only be largely similar, not exactly
00:42 erlehmann but i would prefer exact
00:42 erlehmann josiah_wi can i help you in any way with the noise UB fix?
00:42 MTDiscord <josiah_wi> I was going to try unit tests that would test edge cases and check against a precomputed result.
00:43 MTDiscord <josiah_wi> If you want to write unit tests I recommend reading the PR (don't have link; I can find it later) which discusses the tests.
00:43 erlehmann i was not joking when i meant there are a limited amount of u16 or float or whatever
00:43 erlehmann ok!
00:43 erlehmann thank you for your effort
00:43 erlehmann oh btw
00:48 erlehmann i think if you use ubsan or a debugger it might change the values you observe ;)
00:49 pgimeno well, spamming 5 tests in 5 different posts is not helpful. People who subscribe to issues will receive as many emails as messages you post. They won't be happy about that.
00:52 erlehmann oh sorry
00:53 erlehmann it was not intended to annoy
00:53 MTDiscord <josiah_wi> As much as I want to get into doing some C++ stuff, I've found a niche for myself in the build system and I trust myself there more because I have a little experience now and got critique.
00:54 MTDiscord <josiah_wi> Fixing the CMake deps is something I believe I can do better than fixing UB.
00:55 MTDiscord <josiah_wi> And if anyone wants to help with that, I'll at least have some process in mind for how to move forward with that. xD
00:58 erlehmann i believe that you can only fix the cmake deps though if you eliminate all non-existence dependencies, as make can not handle them.
00:59 erlehmann embedded devs often do that, for the most part by ensuring there are no fallback paths for any includes
01:00 erlehmann josiah_wi do i need to elaborate?
01:02 erlehmann https://stackoverflow.com/questions/16875664/preventing-cmake-from-finding-installed-libraries-instead-of-local-libraries
01:02 erlehmann > By default, find_path (or find_library, etc.) first checks for files in the system standard locations, before searching in the values provided in PATH. That's why "installed" libraries are always found first on Linux (but not on Windows, that doesn't have standard locations for installed libraries).
01:02 Extex joined #minetest-dev
01:03 erlehmann i believe that option helps, i have not tested it
01:04 MTDiscord <josiah_wi> We have variables for setting explicit library paths. That won't fall back to system paths, though.
01:05 erlehmann well if you have no fallback paths ever in the build, you are good
01:05 MTDiscord <josiah_wi> Now that you mention it I still disable system paths for IRRLICHTMT_BUILD_DIR. Thank you for bringing that up.
01:06 MTDiscord <josiah_wi> I should*
01:06 erlehmann but any fallback path will be an issue
01:06 erlehmann because the not-found-headers or not-found-libraries are non-existence dependencies
01:06 MTDiscord <josiah_wi> CMake is designed to have fallback paths. Working against the design is usually messy, so there better be a good reason.
01:07 erlehmann ok, i explain: cmake looks for a header or library at location A, location B, location C, in that order
01:07 MTDiscord <josiah_wi> erlehmann, the fix for that is to make them required deps or not link them if they aren't found.
01:07 erlehmann it finds the thing at location C
01:08 erlehmann now you invoke make, build, everything works. C is the path of the dependency.
01:08 erlehmann but A and B are paths of non-existence dependencies. if in the future a header or library shows up there, the binary needs to be rebuilt.
01:08 erlehmann otherwise you get a different result in an incremental build than a clean build
01:08 erlehmann which would be at least a bit weird
01:09 MTDiscord <josiah_wi> Correct. And CMake will not change the dependency path after it's been found.
01:09 erlehmann long story short, last i checked cmake could not handle this case and it would be a waste of time to make it handle it.
01:09 erlehmann because make can not handle it AND architecture-wise can not be fixed to handle it.
01:10 MTDiscord <josiah_wi> CMake deals in absolutes. Paths passed to make should be absolute paths. If they aren't that's a bug.
01:10 erlehmann yeah but that means that the compiler also needs to know that
01:11 MTDiscord <josiah_wi> The compiler is given the paths explicitly when it's invoked. You can observe this by building in verbose mode.
01:11 erlehmann non-trivial C++ projects usually have hundreds of thousands of non-existence dependencies, which usually means that the build system looks at absolute paths, but the compiler, preprocessor, etc. have fallback paths
01:11 erlehmann good!
01:12 MTDiscord <josiah_wi> As I said, CMake uses absolute paths. Library paths are passed to the compiler with options. Now I'll bet the compiler has fallbacks.
01:12 erlehmann josiah_wi i gather this means that headers are only ever included from the location given to cmake?
01:12 erlehmann otherwise there is a problem
01:13 MTDiscord <josiah_wi> I am not sure what the search order is for the compilers.
01:13 erlehmann i have made the build scripts for the game liberation circuit years ago, i actually straced the compiler there
01:13 erlehmann so now comes the funny part
01:14 MTDiscord <josiah_wi> What exactly is the issue that arises due to these fallbacks?
01:14 erlehmann if you have *any* kind of fallback path mechanism in your build process *anywhere*, the only build systems that can build it reliably are the ones which produce a list of deps and ne deps as side effect of the build. all other ones will sometimes genrate a binary not corresponding to the source, usually without ppl noticing.
01:14 erlehmann it is very funny to show ppl this issue in their own projects
01:15 MTDiscord <josiah_wi> I don't understand the issue yet.
01:15 erlehmann they first go “oh, that is never an issue”, then “maybe sometimes”
01:15 erlehmann wait i have a writeup
01:17 MTDiscord <josiah_wi> It seems like you shouldn't care which library binary is used as long as it works and doesn't introduce bugs.
01:17 erlehmann josiah_wi https://web.archive.org/web/20170326021857/http://news.dieweltistgarnichtso.net/posts/redo-gcc-automatic-dependencies.html
01:17 MTDiscord <josiah_wi> By that logic fallbacks are an excellent idea, and if you need a specific version of the library you specify that.
01:17 erlehmann all the dotted lines in the graphic represent stuff that make can not handle, in a simple hello world
01:19 MTDiscord <josiah_wi> Ah, I see now.
01:19 erlehmann you will probably go from “oh this is not an issue” to “oof this is everywhere” now, within several minutes or hours
01:20 erlehmann but ppl who work on build systems usually try to solve this
01:21 erlehmann just don't try to fix cmake or make, please. it will waste several hours of your life to figure out that you can not fix them and keep existing behaviour.
01:22 MTDiscord <josiah_wi> CMake does work this way, but I wouldn't say it's an issue. Accidentally including the wrong library is a programmer mistake, not a build system bug.
01:22 erlehmann it is though
01:22 erlehmann since the build system can not represent this relation
01:22 MTDiscord <josiah_wi> In fact I'd argue the current behavior of CMake is desirable.
01:22 erlehmann only if you have no fallback paths!
01:23 erlehmann in the compiler
01:23 MTDiscord <josiah_wi> I don't understand.
01:23 MTDiscord <josiah_wi> IrrlichtMt.so should always be included from the path you passed to CMake.
01:23 MTDiscord <josiah_wi> If it's included from anywhere else, that's a bug,
01:23 erlehmann yes i think for libraries this might work
01:23 erlehmann but for headers it will certainly not
01:24 erlehmann and with libraries it will be a programmer bug i guess?
01:24 MTDiscord <josiah_wi> Headers should be included from the directory CMake determines, and nowhere else.
01:24 erlehmann yes!
01:24 erlehmann i agree
01:24 erlehmann but you need to verify that somehow
01:25 MTDiscord <josiah_wi> CMake might not error if headers get deleted; I'm not sure how that would go.
01:25 erlehmann so the dot thingy, i do not know how to make it with cmake. if you find it, please tell me.
01:25 erlehmann if a header is deleted that is not a problem
01:25 erlehmann after all that is a dependency, the build will fail
01:25 MTDiscord <josiah_wi> If CMake does not error out then fallbacks will be a problem.
01:26 MTDiscord <josiah_wi> Otherwise I believe everything should be fine assuming proper precedence of compiler options.
01:26 erlehmann the error class i want to highlight here is not rebuilding all the parts you should rebuild.
01:26 erlehmann i hope it is so
01:26 erlehmann now that you are aware of this and explained to me all is well i feel better :)
01:26 MTDiscord <josiah_wi> CMake won't rebuild if you create a dependency it already found on a different path.
01:27 erlehmann yeah that is a big problem if the compiler, preprocessor etc. uses that thing then
01:27 MTDiscord <josiah_wi> There might be an option to change that but if there is I do not think it should be changed.
01:27 erlehmann as i said, if you use cmake or make, the only way to be sure is to rebuild everything from scratch every time
01:28 erlehmann in fact, debian does that and has tools to check reproducibility
01:28 MTDiscord <josiah_wi> If you change the library path you have to mirror that in CMake. And that can be done without regenerating cache.
01:29 MTDiscord <josiah_wi> Although in my PR I explicitly said to rm the cache and rebuild if you change the IrrlichtMt path.
01:29 erlehmann yes, rather rebuild too much than too little!
01:29 erlehmann building too much only costs time
01:29 erlehmann building to little gets you the wrong binary
01:30 MTDiscord <josiah_wi> CMake shouldn't rebuild too little.
01:30 MTDiscord <josiah_wi> From the compile times this should be obvious hehe.
01:30 erlehmann have you tried my git timestamp thingy?
01:31 MTDiscord <josiah_wi> I have not.
01:31 MTDiscord <josiah_wi> If CMake is rebuilding too little that's probably a bug in our CMakeListse
01:32 MTDiscord <josiah_wi> And that's what I'm asking about.
01:33 erlehmann josiah_wi i will tell you once i encounter it again
01:35 MTDiscord <josiah_wi> There's probably way more important changes that can be made to the CMake, such as organizing dependencies with targets to improve build system quality.
02:03 tekakutli joined #minetest-dev
02:13 v-rob joined #minetest-dev
02:28 queria joined #minetest-dev
02:34 queria joined #minetest-dev
03:04 v-rob joined #minetest-dev
03:45 specing_ joined #minetest-dev
04:00 MTDiscord joined #minetest-dev
05:08 MTDiscord1 joined #minetest-dev
06:58 m42uko_ joined #minetest-dev
07:14 Menchers joined #minetest-dev
08:01 macaw joined #minetest-dev
08:09 hendursa1 joined #minetest-dev
08:18 sfan5 joined #minetest-dev
08:53 sfan5 rubenwardy: might be that google wants you to use app bundles these days (they're deprecating support for that too in Nov. IIRC)
08:53 sfan5 you can manually do it using apksigner still
09:06 MTDiscord2 joined #minetest-dev
10:07 BuckarooBanzai joined #minetest-dev
10:19 calcul0n_ joined #minetest-dev
10:35 sfan5 joined #minetest-dev
11:05 appguru joined #minetest-dev
11:17 sfan5 joined #minetest-dev
12:02 sfan5 joined #minetest-dev
12:13 sfan5 joined #minetest-dev
12:26 proller joined #minetest-dev
12:29 appguru joined #minetest-dev
13:16 sfan5 joined #minetest-dev
13:17 sfan5 joined #minetest-dev
13:20 rubenwardy This Android update is likely to be a bit risky
13:20 rubenwardy I wonder if there should be an option to copy rather than move, and if there should be a confirmation dialog
13:20 rubenwardy If you uninstall the app now, it'll delete any saved worlds
13:22 sfan5 https://developer.android.com/guide/topics/manifest/application-element#fragileuserdata btw
13:23 rubenwardy oooh
13:25 sfan5 also can't the user use their file manager to interact with /sdcard/Android/data/net.minetest.minetest?
13:25 sfan5 the ZIP thing would ofc be useful but is not strictly necessary
13:26 rubenwardy hm, I can't see MT in the Apps section of my file manager
13:26 rubenwardy (pixel 4a)
13:26 rubenwardy oh it's not alphabetically sorted
13:27 rubenwardy that opens up app info, rather than the files
13:28 rubenwardy I can't see a way using Pixel's Files app
13:28 rubenwardy it might be possible using adb, which is not very user friendly but neither was using the file manager
13:29 rubenwardy InternalStorage > Android/data is empty for me
13:30 sfan5 let me reinstall minetest and check
13:30 MTDiscord <Sublayer plank> I'm able to access /sdcard/Android/data/net.minetest.minetest/ with total commander, I just need to do a minor slight of hand with permissions which it shows how to do
13:32 sfan5 huh yeah it shows as empty, must be some permission thing
13:37 rubenwardy Should we start using app bundles? I'm not familiar with them
13:37 rubenwardy hmm, we still need apks for github releases and testing
13:38 sfan5 you can build both
13:38 rubenwardy yeah, makes sense
13:39 sfan5 I remember reading that Google wanted to force bundles for existing apps too but can't find a timeline
13:39 rubenwardy https://developer.android.com/distribute/play-policies
13:39 rubenwardy only appears to be for new apps
13:39 rubenwardy but still worth doing
13:40 rubenwardy maybe we won't need to +2 for each release with this? If it's clever enough to create 32/64 as needed
13:50 rubenwardy OHHH
13:50 rubenwardy the signed APKs are going to a different directory
13:52 rubenwardy https://github.com/rubenwardy/minetest/releases/tag/5.4.2-rc1
13:52 rubenwardy (eh, "rc1" maybe isn't right, as it's not an official release candidate)
13:53 sfan5 I suggest '5.4.2-android'
13:53 rubenwardy for the actual tag? sure
13:53 sfan5 yea
13:54 rubenwardy probably won't break anything
13:55 rubenwardy hmm, I'll probably need to bump the release properly rather than updating the code manually
14:03 rubenwardy Loading is noticably faster on 5.4.2
14:03 rubenwardy he says, as the bar stops moving
14:04 rubenwardy errr..
14:04 rubenwardy my camera is upside down
14:04 sfan5 what
14:05 rubenwardy https://i.rubenwardy.com/qZyy5.jpeg
14:06 sfan5 ?
14:06 rubenwardy singleplayer is the same
14:06 rubenwardy it's definitely worked before
14:06 sfan5 maybe you backported too much
14:07 MTDiscord <Jonathon> its a feature ™️
14:10 rubenwardy grr
14:11 rubenwardy my phone is unable to open .conf files
14:12 MTDiscord <josiah_wi> Total Commander has a built-in editor that I assume should be able to do that.
14:12 rubenwardy Android Studio has a file manager I can use
14:12 rubenwardy is annoying though
14:12 rubenwardy > Important note: This app does NOT contain any ads. However, it contains a link "Add plugins (download)" in the home folder. This is treated as an ad by the Play Store because it links to our other apps (plugins).
14:12 rubenwardy que
14:13 rubenwardy disabling undersampling fixes it
14:15 sfan5 not a new bug then
14:24 Fixer joined #minetest-dev
14:27 rubenwardy ok, it looks fine to me
14:28 rubenwardy what do you think about adding a warning dialog? Is it necessary?
14:28 rubenwardy Any changes now increase the risk of issues, and we won't be able to release updates for a while after 1 nov
14:29 rubenwardy warning dialog probably isn't necessary if we can find ways to extract the worlds currently (ie: using Total Commander / adb)
14:29 rubenwardy and with the uninstall hasFragileUserData
14:29 sfan5 yeah
14:30 rubenwardy adb pull /sdcard/Android/data/net.minetest.minetest/files/Minetest/worlds
14:30 rubenwardy simple as that
14:30 rubenwardy well, plus setting up adb and developer mode
14:50 hendursaga joined #minetest-dev
14:57 fluxionary joined #minetest-dev
15:15 v-rob joined #minetest-dev
15:33 erlehmann <rubenwardy> disabling undersampling fixes it
15:33 erlehmann <sfan5> not a new bug then
15:33 erlehmann is this only on android?
15:35 sfan5 so far nobody has reproduced it on desktop
15:45 specing_ joined #minetest-dev
16:00 Extex joined #minetest-dev
16:10 v-rob joined #minetest-dev
16:17 hendursaga joined #minetest-dev
16:20 ShadowNinja Have you tried with an OGLES2 build on desktop?
16:22 sfan5 I have
16:28 olliy joined #minetest-dev
16:31 tekakutli joined #minetest-dev
16:50 MTDiscord <Jordach> Are you sure it’s not corresponding to the upright position of the phone
16:51 MTDiscord <Jordach> For which it would be in landscape
17:15 v-rob joined #minetest-dev
18:42 MTDiscord <josiah_wi> Do I need both ogles 1 and 2 to compile on Android?
18:43 sfan5 yes, but why are you asking?
18:43 MTDiscord <josiah_wi> Because I'm doing something stupid.
18:44 MTDiscord <josiah_wi> I'm trying to compile Minetest with Termux out of curiosity.
18:48 MTDiscord <josiah_wi> Minetest from the app store works so I must have GLES 2? CMake can only find OGLES1, ouch.
18:48 MTDiscord <josiah_wi> Actually it didn't find either.
18:49 sfan5 do you want to build it for the system inside termux (which doesn't even run graphical apps) or for Android as an APK?
18:49 sfan5 the former makes no sense and the latter is not possible
18:51 MTDiscord <josiah_wi> I was trying the former, and I can't find a gles library so I guess it won't even work.
18:52 MTDiscord <josiah_wi> I could compile the headless server, I suppose. xD
18:53 MTDiscord <Sublayer plank> I actually tried to compile a headless minetest server in termux for funsies a while back, but it kept thinking I wanted to compile it as a native library for an APK and would throw errors
18:54 erlehmann sfan5 do the ntfs extraction problems you encountered on windows go away if you disable 8dot3 names? https://www.neowin.net/forum/topic/1366224-tweak-win-10-ntfs-by-disabling-8dot3-and-stripping-8dot3-huge-per-gain/
18:54 erlehmann sfan5 i think windows makes a short DOS name for every file by default, so i hope this helps you
18:54 sfan5 I don't know but I doubt it, number of files per directory isn't the issue
18:55 erlehmann the 8dot3thing is reported to make windows file handling slower at about 1000 files or so
18:55 erlehmann no idea if that is still the case
18:56 MTDiscord <josiah_wi> @ROllerozxa Alright, I'm doing this. I think I can tell CMake I'm cross compiling for Linux or something.
18:57 MTDiscord <Sublayer plank> yeah, that would probably be the best way of going about it
18:58 erlehmann sfan5, btw i appreciate the link to https://www.sqlite.org/fasterthanfs.html – but what i meant with benchmarking was pitting the actual file handling code vs db blob handling code. the article even mentions that itself, you have to measure the actual code you have, not their test cases.
18:59 erlehmann i bet a lot of the cases where someone was like “nice, i'll just use a relational db” were relying on those benchmarks but had different usage of the storage backend or different things than only tiny blobs
19:00 erlehmann sfan5 celeron55 regarding the UB in the noise, is “same seed should generate same map regardless of compiler version or optimization level or CPU architecture” a hard or soft goal of minetest?
19:01 erlehmann hard as in, important
19:01 erlehmann soft as in, well, nice to have
19:01 sfan5 didn't someone already figure out a way to get an identical calculation without UB?
19:03 erlehmann i mainly asked bc of the negative-float-into-unsigned-int test case i unintentionally spammed everyone with
19:03 erlehmann because there is no “identical calculation” unless you say this compiler, this optimization level, this platform, this architecture
19:04 erlehmann it would be good to know what the canonical minetest binaries are compiled with to figure out what the most common behaviour is
19:04 MTDiscord <josiah_wi> Compilation started. ETA: a very long time
19:04 MTDiscord <josiah_wi> nvmnd it still thisks it's Android.
19:04 erlehmann if there is such a thing as a canonical minetest binary
19:05 erlehmann the mapgen UB might explain basically all artifacts i have seen btw – stone map blocks hanging in the sky, jarring discontinuities in biomes, maybe even the 1 block high “stone clouds”
19:06 erlehmann appguru do you have an idea if the view bobbing affects head bone with playerplus? bc if so, that is packet spam
19:06 erlehmann cora already found that the player bones seem to never really come to rest
19:06 erlehmann there is some small movement
19:07 erlehmann even without idle anim or so
19:07 MTDiscord <luatic> Bobbing is entirely clientside. I'm pretty sure playerplus (assuming it's a mod) doesn't implement it at all.
19:09 erlehmann luatic so it does not send the view direction ig?
19:15 Extex joined #minetest-dev
19:23 MTDiscord <luatic> The client does send the view direction
19:31 v-rob joined #minetest-dev
19:39 sfan5 merging #11429, #11695 in 10m
19:39 ShadowBot https://github.com/minetest/minetest/issues/11429 -- Add no_texture.png as fallback for unspecified textures by Wuzzy2
19:39 ShadowBot https://github.com/minetest/minetest/issues/11695 -- Fix view bobbing not resetting when resting by appgurueu
20:37 proller joined #minetest-dev
21:00 v-rob joined #minetest-dev
21:18 Extex joined #minetest-dev
21:36 adfeno joined #minetest-dev
22:53 proller joined #minetest-dev
23:02 erlehmann joined #minetest-dev
23:14 AliasAlreadyTake joined #minetest-dev
23:23 wsor4035 joined #minetest-dev

| Channels | #minetest-dev index | Today | | Google Search | Plaintext