Minetest logo

IRC log for #minetest-dev, 2015-11-13

| Channels | #minetest-dev index | Today | | Google Search | Plaintext

All times shown according to UTC.

Time Nick Message
00:35 est31 joined #minetest-dev
00:38 est31 PTAL https://github.com/est31/minetest/commit/dc2e97deb8c73d1d1da9f702dc4e774bab8f2898
00:39 cib0 joined #minetest-dev
00:43 est31 hmmmm, ?
00:44 est31 this is the code I've removed: https://github.com/est31/minetest/commit/5e507c9829942c434a6f1ae7a4f3a488c7e50bef#diff-ac9ac0a3a5e432320dd8784aaeeef672L1092
00:44 est31 in the regressing commit
00:45 est31 the commit now adds it back
00:46 est31 I'll push it in 30 minutes.
00:55 turtleman_ joined #minetest-dev
02:10 cib0 joined #minetest-dev
02:15 crazyR__ joined #minetest-dev
02:42 Zeitgeist_ joined #minetest-dev
02:44 turtleman_ joined #minetest-dev
02:44 Robby joined #minetest-dev
02:52 damiel joined #minetest-dev
03:01 fling joined #minetest-dev
04:03 paramat joined #minetest-dev
04:11 paramat docs added will merge in 20mins https://github.com/minetest/minetest/pull/3359
04:22 paramat now merging
04:32 paramat done
05:19 paramat hmmmm https://github.com/minetest/minetest/pull/3364 Mgfractal: Create a choice of 4 mandelbrot formulas
05:36 paramat left #minetest-dev
06:26 paramat joined #minetest-dev
06:30 VanessaE so, can someone tell me what THIS means?
06:30 VanessaE terminate called after throwing an instance of 'FileNotGoodException'
06:30 VanessaE what():  LevelDB error: IO error: lock /home/minetest/.minetest/worlds/Creative_World/map.db/LOCK: Resource temporarily unavailable
06:31 VanessaE one of my servers went down for a number of hours (while I was afk), then just randomly came back up on its own after I don't know how many restart attempts
06:31 paramat a plan to fix the problem with low decoration densities: if 'area * nval' is between 0 and 1, instead of rounding 'deco count' down to 0, calculate a chance for 'deco count' to be 1. i'll work on this
06:32 sfan5 VanessaE: lock file is held by another process
06:33 VanessaE could a server crash result in a stale lock file?
06:33 VanessaE (because otherwise I can't think of any way that what you suggest can happen at all)
06:35 paramat sfan5 any comments? https://github.com/minetest/minetest_game/pull/719
06:35 sfan5 I'm not exactly sure how lock files work
06:35 sfan5 so maybe
06:36 VanessaE ok.
06:36 sfan5 paramat: +1
06:36 sfan5 paramat: since when do we have a "catch_up" param?
06:36 VanessaE this is what I *think* the chain of crashes started with:  http://pastebin.ubuntu.com/13245765/
06:36 paramat since i created it =)
06:37 sfan5 looking at the "write failed" by xban
06:38 sfan5 maybe a disc hiccup?
06:38 VanessaE well that "write failed" has been happening for a few days now, with no explanation
06:39 VanessaE permissions and ownership are file and the xban db is readable
06:39 cheapie Apparently no disk errors were logged, either by the kernel or the drives.
06:39 VanessaE ^^^ and none of the other servers on the machine were affected.
06:39 cheapie So the xban DB is readable... is it writeable?
06:40 VanessaE yep.
06:40 VanessaE -rw-r--r-- 1 minetest minetest 79060 Nov 13 01:39 /home/minetest/.minetest/worlds/Creative_World/xban.db
06:41 cheapie Are the other servers doing the xban "write failed"?
06:41 * VanessaE checks
06:41 VanessaE yes
06:41 cheapie OK, so I don't think that's related.
06:43 VanessaE is something preventing files being written to the world dir?
06:44 VanessaE (mod security is not enabled)
06:46 Hunterz joined #minetest-dev
07:36 paramat hmmmm another PR https://github.com/minetest/minetest/pull/3365 Decoration API: Allow very low density decorations. bbl
07:36 paramat left #minetest-dev
08:03 Darcidride joined #minetest-dev
08:30 rubenwardy joined #minetest-dev
08:33 est31 joined #minetest-dev
08:50 celeron55 wait what
08:50 celeron55 why is the emerge queue not a queue and instead just an std::map indexed by block positions
08:51 celeron55 that messes up the load priorities
08:52 celeron55 the block selecting and sending code is very careful to first load more important things and later less important things, and then the emergethread just shits all over it
08:55 celeron55 this makes it so that if player A is standing at a position which is sorted first in an std::map compared to a position where player B is, player A's block loads are very much prioritized over player B's
08:55 celeron55 uncontrollably much
08:56 celeron55 basically player B gets whatever is left over from player A, if any
08:59 celeron55 wait no, actually it's working fine
08:59 celeron55 the queue is just per-thread while the data is global for... no reason
09:02 hmmmm there
09:02 hmmmm is a reason
09:03 est31 src/emerge.{cpp,h} is full of weirdnesses, until commit 0850d3bb930a where hmmmm cleaned it up, EmergeManager had a vector<EmergeThread *> named "emergethread".
09:04 est31 singular
09:04 celeron55 i don't think that's weird but whatever
09:05 hmmmm that's because at that point in my life i was still debating if an array should be a singular or plural
09:06 hmmmm it kind of made sense when i wrote the multithreaded emergemanager because emergethread[4] was like saying "Emerge Thread #4"
09:06 celeron55 hmmmm: not that i really care, but what is the reason? maybe i need to tell it to somebody some day
09:07 hmmmm the map is global for uniqueness
09:07 hmmmm i thought that was kind of obvious with the "update_existing" bit of logic in pushBlockEmergeData()
09:08 celeron55 but it needs to find the thread to which it was queued previously anyway, right?
09:08 hmmmm ?? why
09:09 celeron55 i don't find it doing that though; maybe it doesn't
09:09 hmmmm in any case you might notice that I'm on a minetest break
09:09 hmmmm when I come back I want to fix the EmergeThread disk/generate queue distinction
09:10 celeron55 what's happening then if enqueueBlockEmerge is spammed by the same block position all the time, like the block selection and sending code does?
09:10 hmmmm this is going to be kind of big because I'm going to have to begin storing which blocks exist in a very efficient manner
09:10 hmmmm bloom filter probably
09:10 hmmmm what's happening?  it's getting updated
09:11 celeron55 where does it see that it's already queued and doesn't add it to the thread's queue each time
09:11 hmmmm if it's getting enqueued with an option that was not present before, it'll set that flag
09:12 hmmmm oh this is not good
09:12 hmmmm this is a regression
09:12 hmmmm there should be a return false after this line https://github.com/minetest/minetest/blob/master/src/emerge.cpp#L409
09:13 hmmmm i wonder why nobody said anything about the emerge thread being super slow lately... did this bug have any effect in practice, I wonder?
09:14 celeron55 okay; in that case i also need to make an enum for the return value so that enqueueBlockEmerge can distinct between queue being full and block being already in queue (the block selecting and sending system obviously needs that information)
09:15 celeron55 distinguish* (how does this word even work)
09:15 est31 hmmmm, it can be fancier, can't it?
09:15 est31 e.g. old_flags = bedata.flags;
09:15 est31 bedata.flags |= flags;
09:16 est31 return bedata.flags == old_flags
09:16 hmmmm that's kinda redundant seeing as you already have a boolean parameter telling you if it existed or not...
09:17 hmmmm so
09:18 hmmmm I am going to just make the 5 second fix for this
09:18 hmmmm i realize it'll merge conflict with celeron's thing but this is pretty important imo
09:22 celeron55 what boolean parameter
09:23 celeron55 oh you answered to est31
09:24 celeron55 hmmmm: do realize that if you just make that return false, then you break the return value of enqueueBlockEmerge because then it will return false if the block existed in the queue in addition to returning false when the queue was full
09:24 hmmmm ?
09:24 hmmmm why is that broken
09:25 hmmmm maybe it's broken for your purposes, but i thought we established that you're going to change the return value to an enumeration
09:25 celeron55 enqueueBlockEmerge should return false only if the queue is full, because that is what the block selecting and sending code expects; it's the only code that calls the function
09:25 celeron55 if it gets false, it stops adding stuff to the queue
09:25 hmmmm the *current* code returns false?
09:25 celeron55 i am talking about what is in upstream right now
09:25 hmmmm ohh okay
09:26 hmmmm i see
09:26 hmmmm well minetest hasn't exploded yet so I'll let it go till tomorrow I really need to go to sleep
09:27 celeron55 the thread-specific queues handle the spam fairly well because they check the in-memory MapBlock as the very first thing they do
09:28 celeron55 and because they have no length limits
09:28 est31 with the "boolean param" you mean update_existing ?
09:30 est31 hrmm I guess my "fancy solution" isnt fancy at all
09:30 est31 but outright wrong
09:30 est31 simple return false is 100% ok
09:45 paramat joined #minetest-dev
09:47 paramat can anyone review https://github.com/minetest/minetest/pull/3364 ? since hmmmmm is taking a break
09:48 nrzkt joined #minetest-dev
09:57 kilbith joined #minetest-dev
09:58 kilbith paramat, have you an opinion on these headers
09:58 kilbith https://github.com/minetest/minetest/pull/3275#issuecomment-154723731
09:58 kilbith https://github.com/minetest/minetest/pull/3275#issuecomment-155092512
09:59 est31 If we change the header image, then I'd favor your second one.
10:00 paramat ah nice i didn't see these
10:02 paramat i like the first one because it's more universal so more suitable for an engine
10:02 kilbith thanks for the input both
10:08 Amaz joined #minetest-dev
10:12 rubenwardy joined #minetest-dev
10:44 paramat left #minetest-dev
10:59 kilbith left #minetest-dev
11:06 deltib joined #minetest-dev
11:08 proller joined #minetest-dev
11:23 celeron55 i updated this again: https://github.com/celeron55/minetest/commits/far_map_wip
11:23 celeron55 it now should be more responsive at choosing what it's generating even on slow systems
11:48 Calinou joined #minetest-dev
11:58 who_wants_some joined #minetest-dev
12:01 Player2 joined #minetest-dev
12:30 gregorycu joined #minetest-dev
13:28 zat joined #minetest-dev
13:34 proller joined #minetest-dev
14:10 est31 joined #minetest-dev
14:30 celeron55 http://i.imgur.com/WCftPPS.jpg
14:30 celeron55 this bug is interesting for sure
14:31 celeron55 in that screenshot, it has decided that various kinds of water should generally look like various kinds of bricks
14:31 celeron55 but at that one point, it should look like water, and instead trees should be yellow and dirt should be whatever the hell that is
14:36 damiel joined #minetest-dev
14:40 Hunterz joined #minetest-dev
14:47 who_wants_some joined #minetest-dev
14:50 celeron55 valgrind doesn't report anything being weird
14:51 est31 then its super w3ird
14:54 celeron55 i tried making the texture atlases to contain 64 nodes each instead of 256, and got a different result
14:54 celeron55 this means node ids are not truncated to 8 bits and the atlases are getting mixed up
14:55 celeron55 it's very weird
14:55 celeron55 it's not like i'm not trying to solve this
14:55 celeron55 no bug just seems to be found anywhere
14:57 BlockMen joined #minetest-dev
15:05 celeron55 (and the problem is fixed once i make the atlas image large enough to fit all nodes)
15:07 who_wants_some joined #minetest-dev
15:10 who_wants_some joined #minetest-dev
15:23 celeron55 now i even tried generating meshes in the main thread
15:23 celeron55 needless to say, this has no effect whatsoever
15:24 celeron55 interestingly my farmap-optimized mesh generator is so fast that it doesn't even create any FPS spikes
15:26 slow_mapgen joined #minetest-dev
15:26 celeron55 well okay, it does
15:27 celeron55 (something like 20ms each i guess)
15:34 est31 what precisely is the atlas?
15:37 est31 joined #minetest-dev
15:37 est31 hmm as it seems generating it causes to pile up so much RAM so that my OOM killer kicks in
15:37 est31 and kills KDE
15:38 est31 well, thats nice, better than my system freezing and me having to sysrq
15:38 celeron55 you need to tune the range down if you don't have many gigabytes of free RAM
15:39 est31 I didnt even spawn
15:39 est31 I was inside theloading screen
15:39 est31 at "generating atlas"
15:39 celeron55 and each node in a game takes about 50kB of space in the texture atlas
15:39 est31 but yeah I admit I've tried dreambuilder
15:40 celeron55 it has like 20k
15:40 est31 trivial optimisation would be to use texture names instead of node ids
15:40 est31 there are less or equal numer of textures as nodes
15:41 celeron55 it's done already
15:41 celeron55 that's why it actually isn't 50kB as long as nodes share the same textures
15:42 celeron55 that can be helped with a special shader that allows the atlas to not contain repeated textures
15:43 celeron55 it's bound to be a problem without shaders though
15:45 est31 why is a shader needed for that?
15:46 celeron55 each farnode face needs to have 4x4 nodes repeated on it (and then there's the 16x16 LOD level too)
15:46 est31 ah I see
15:47 est31 so its taking 4x times of RAM?
15:47 celeron55 as it is now, the atlas consists of 8x8 repetitions of node faces with pre-baked lighting effects on them
15:47 celeron55 because with 4x4 the neighboring nodes would bleed to them when rendered
15:47 celeron55 so it takes the middle 4x4 and it looks fine
15:47 celeron55 so each node uses 128x128px of atlas space
15:48 celeron55 i mean
15:48 celeron55 not each node but each texture that is used by some node
15:49 est31 so it pre-bakes lighting?
15:49 who_wants_some joined #minetest-dev
15:49 est31 how much lighting levels do we have, 2^4?
15:49 celeron55 not that kind of lighting
15:49 celeron55 just some cheats so that it looks more like it's 4x4 nodes
15:52 celeron55 updating an large image into a texture is so slow in irrlicht that it can't be done at runtime so the atlas has to have all nodes on it that will be used
15:53 celeron55 slowness wouldn't be an issue if it could be threaded but of course it can't
15:53 celeron55 and once you make them small, you lose all the benefit from not having to switch textures when rendering
15:54 celeron55 or have to work around it with more complex systems
15:56 est31 I really like the idea of sending divs_per_mb over the network
15:56 celeron55 TL;DR: rendering a lot of stuff at once takes a lot of RAM or half of a lot of RAM plus very complex code
15:56 celeron55 or a quarter of a lot of RAM and even more complex code
15:57 est31 I like the shader idea
15:57 damiel joined #minetest-dev
15:59 celeron55 of course i need to somehow make this such that it doesn't just crash and die if you open dreambuilder on an old computer with farmap enabled
15:59 celeron55 that might be a matter of just giving up on it but anyway
15:59 est31 it can do some calculations
15:59 est31 it knows all the nodedef and which textures it uses
15:59 est31 so take the rough size, and add it all up
16:00 est31 if larger than 50% of the free RAM on the system, dont use farmap
16:00 est31 perhaps overrideable with a setting, farmap_no_shader_ram_limit or so
16:00 celeron55 i mean, it should be able to just make an atlas with like 2x2 pixels per node
16:01 est31 thats even smarter
16:01 celeron55 so that it can render stuff with at least some colors
16:05 est31 but from trying it in minimal game it really looks great
16:13 celeron55 i think i'll actually default the far map texture atlas to something less than 16x16 per simulated node face; it's kind of excessive for creating the effect to begin with anyway
16:13 celeron55 8x8 seems to work fine
16:13 celeron55 one can always configure it higher if he wants quality over quantity
16:14 celeron55 or has the RAM to support both
16:30 est31 joined #minetest-dev
16:39 celeron55 the nice thing about this setting is that by setting it low enough, the bug that weirdly swaps the atlas texture doesn't happen as you only have a single atlas texture 8)
16:40 Hunterz joined #minetest-dev
16:43 rubenwardy joined #minetest-dev
16:52 who_wants_some joined #minetest-dev
16:56 who_wants_some joined #minetest-dev
17:09 celeron55 pushed the new setting
17:15 hmmmm joined #minetest-dev
17:27 nrzkt joined #minetest-dev
17:31 Krock joined #minetest-dev
17:45 VanessaE celeron55: it's looking pretty good now.  texture swapping glitch is pretty much gone.  did you notice though that on newly-generated terrain, the farmap blocks are dark along the edges of not-yet-generated areas?
17:46 Gael-de-Sailly joined #minetest-dev
17:55 VanessaE fwiw, under Dreambuilder with HDX256, my machine only uses 2.4 GB or so before it creates the farmap atlases.  after I get into a world (in this case, local copy of VE-Survival), call it 4 GB
17:55 VanessaE so I don't think you actually need to worry TOO much about RAM usage
17:57 VanessaE and your map loading speed is greatly improved now - even this local copy of VE-S loads at a tolerable speed once it gets past the initial 2 mins worth of crap that technic mod does on startup (and that's an achievement ;) )
18:05 VanessaE bbl
18:06 cib0 joined #minetest-dev
18:15 Krock joined #minetest-dev
18:55 zupoman joined #minetest-dev
18:55 zupoman joined #minetest-dev
19:08 rubenwardy game#726
19:10 rubenwardy ShadowBot!
19:11 rubenwardy Issue - Split default into smaller mods - https://github.com/minetest/minetest_game/issues/726
19:14 Amaz joined #minetest-dev
19:40 kaeza joined #minetest-dev
20:01 DFeniks joined #minetest-dev
20:13 cib0 joined #minetest-dev
20:18 Icedream joined #minetest-dev
20:20 crazyR joined #minetest-dev
20:46 turtleman_ joined #minetest-dev
20:55 Gael-de-Sailly joined #minetest-dev
21:10 Amaz joined #minetest-dev
21:40 kaeza joined #minetest-dev
21:49 twoelk joined #minetest-dev
22:09 who_wants_some https://imgur.com/a/F7SuR lol
22:09 who_wants_some sorry, wrong channel
22:09 hmmmm ehh, this is the right channel i'd say
22:10 hmmmm is this reproducible?
22:10 who_wants_some it is mod problem, not related to minetest core developement, sorry
22:10 hmmmm ok

| Channels | #minetest-dev index | Today | | Google Search | Plaintext