Minetest logo

IRC log for #minetest-dev, 2015-11-08

| Channels | #minetest-dev index | Today | | Google Search | Plaintext

All times shown according to UTC.

Time Nick Message
00:00 Fritigern joined #minetest-dev
00:14 paramat i'll merge https://github.com/minetest/minetest_game/pull/720 in an hour since it's biome tweaks
00:19 hmmmm i sorta agree with the comment on that pr... there is probably too much desert/arctic compared to regular biomes
00:20 paramat yeah it's an illusion, i'm adding a comment
00:22 paramat i was wondering what noise thresholds create an equal balance of frozen/temperate/hot biomes, currently it's -0.4, 0.4 as in mgv6
00:24 paramat i wrote a mod to test this and seemed to find the current thresholds actually create temperate areas with over a third of surface area
00:25 paramat it's a problem how close frozen gets to hot though
00:25 hmmmm i think the main problem with perlin noise based biomes is that you can't control the rarity independent of the size for each biome
00:27 paramat see https://github.com/minetest/minetest_game/pull/718 for comparison of persistence 0.5 and 0.4
00:27 paramat i was trying to increase the distance between frozen and hot blobs, it didn't help much
00:28 Fritigern joined #minetest-dev
00:29 paramat and 0.5 has more interesting biome shapes so i'm staying with the classic mgv6 noise of 3 oct 0.5 persist
00:29 hmmmm ok then
00:31 paramat it's a dilemma. creating a nice separation between hot and frozen blobs then makes the extreme biomes too rare
00:31 paramat i had a few reports of players travelling 5kn to find desert
00:32 paramat ..or saying all they can find is temperate biomes
00:32 paramat in mgv7
00:42 Fritigern joined #minetest-dev
00:45 Fritigern joined #minetest-dev
00:47 hmmmm how do you feel about generating a permanent biome map on map creation
00:54 Fritigern joined #minetest-dev
00:56 rubenwardy joined #minetest-dev
00:56 paramat hm, still using noise though?
00:57 hmmmm it's all noise
00:58 hmmmm but this would free biomes from having to use point-polled noise
00:58 paramat some mgflat dev https://github.com/minetest/minetest/pull/3313#issuecomment-154650568
00:58 hmmmm hills??
00:58 hmmmm this was supposed to be super simple and flat
00:58 hmmmm a one liner basically
00:59 Fritigern joined #minetest-dev
00:59 hmmmm well it makes sense I'd say
01:00 paramat i think it'll be popular
01:01 paramat as options disabled by default
01:02 paramat without the lakes there would be no papyrus or clay, so they are needed for resources
01:03 paramat ok so biome areas would be defined without calculation on every point. perhaps this could be part of your alternative biome manager?
01:04 hmmmm i've tossed some ideas around but never came up with definite plans
01:04 hmmmm it's not like we're trying to "save" on calculations here, it's just that deciding that node X will be thing based on value Y at point Z is very limiting since it rules out any procedural generation
01:05 hmmmm hmm I might get back into the mapgen business soon since I don't need to spend any time on client side modding :)
01:06 paramat i like the current system so i would support another system as an alternative, not a replacement
01:06 hmmmm I'm so glad I didn't have to do that after all
01:06 paramat it would be good to see you do some mapgen, and something you enjoy. nether and aether realms?
01:06 hmmmm that's on the list of priorities for mapgen-related things
01:07 paramat oh and v8 of course
01:08 paramat btw is it normal for the limit of 5000 blocks in client memory to use 2GB? seems excessive for such a small volume of world
01:09 hmmmm no idea
01:09 paramat just wondering if there is a bug
01:09 Fritigern joined #minetest-dev
01:09 paramat ok
01:09 hmmmm the mapgen uses a huuuuge amount of memory, thanks to the buffering system
01:10 hmmmm oh I have a good idea on how to cut down on memory consumption
01:10 hmmmm I need to implement this for sure
01:11 paramat memory used seems to be the limit on how large a structure can be viewed, hopefully farblocks would help
01:31 paramat left #minetest-dev
01:35 Fritigern joined #minetest-dev
02:06 Fritigern joined #minetest-dev
02:20 Fritigern joined #minetest-dev
02:31 Fritigern joined #minetest-dev
02:48 Fritigern joined #minetest-dev
02:48 paramat joined #minetest-dev
02:50 paramat i will make the extreme biomes a little smaller, it's been requested a few times. also i've seen that with biome blend disabled mgv6 desert threshold is 0.45, so i'll go with +-0.45
02:52 hmmmm well I have an idea
02:52 hmmmm instead of blindly adjusting parameters why not manipulate the parameters to have the properties you're looking for?
02:53 hmmmm do a custom transform on the biome noise so that "moderate" values (i.e. values in between a certain threshhold) are dilated
02:53 hmmmm right now the way it works is that a bunch of random points on the noise grid are chosen and the greater the difference between them, the steeper the slope
02:54 hmmmm whenever you have really hot areas close to really cold areas, you have a very steep slope
02:54 hmmmm if you just move the threshholds that will make the extreme biomes much smaller since they don't have as much range
02:54 hmmmm of possible values
02:55 paramat i see
02:55 hmmmm wait, is this mathematically possible?
02:55 * hmmmm thinks
02:57 hmmmm or would this be equivalent to increasing the scale
02:57 gregorycu joined #minetest-dev
02:58 paramat i'll consider it. maybe i'll try it to see if it helps. for now i'll go ahead with these tweaks
02:59 gregorycu hmmmm: Was it you or est that was working on the settings fixes?
02:59 hmmmm est
02:59 hmmmm paramat:  disregard what i said, it would be equivalent to increasing the scale
03:00 gregorycu I just started a server before, and joined it, it plays like a different game
03:00 paramat ok
03:00 gregorycu With regards to the frustum culling, I think I can do it properly, but c99 said to leave it cause he is working on something similar
03:00 hmmmm ahh ok
03:03 gregorycu By the way, I have a funny feeling that Irrlicht is doing its own frustum culling as well as us
03:04 gtown joined #minetest-dev
04:06 paramat much better i'm going with this https://github.com/minetest/minetest_game/pull/720#issuecomment-154774411
04:07 paramat will merge soon
04:08 VanessaE paramat: I dunno..  if it were up to me, I'd keep the sizes of the coldest and warmest, but increase the overall scale of the thing
04:08 VanessaE put even more of an interval between the extremes
04:09 VanessaE because honestly, a map like that ----> doesn't leave a lot of room for building large cities in a nice, grassy area
04:09 VanessaE (if that makes sense)
04:12 Miner_48er joined #minetest-dev
04:28 gregorycu I would have made the biomes bigger
04:28 gregorycu But I am not expert on this stuff
04:39 paramat the map has biomes 7.5 times smaller than default, for testing
04:39 gregorycu Ahh ok
04:39 gregorycu :)
04:39 paramat heh so no tiny biomes
04:53 FR^3 joined #minetest-dev
04:56 FR^3 joined #minetest-dev
05:03 paramat now merging game 720
05:06 FR^3 joined #minetest-dev
05:07 paramat done
05:14 paramat next i'm going to consider a small part of desert becoming a new stone-only biome, like parts of mgv6
05:16 FR^3 joined #minetest-dev
05:48 hmmmm uh og
05:48 hmmmm i just realized there's always been a race condition in myrand()
05:50 gregorycu lol
05:50 hmmmm well I guess it's not too bad unless somebody were to mistake it for being a secure random and not just an "i really don't care, anything will do" value
05:52 gregorycu Was it based on time?
05:53 hmmmm it's shared between multiple threads
05:53 hmmmm it's this way on purpose for simplicity and hard to fix
06:02 gregorycu I don't know why, but I find optimising so much fun
06:04 gregorycu probably not what minetest really needs however
06:04 gregorycu hmmmm: Do you have something in mind for me to look at?
06:05 hmmmm :)  one of the optimizations I always wanted to do but never got around to doing was making SSE versions of Noise
06:05 hmmmm and NEON
06:05 hmmmm the latter is more valuable honestly
06:05 hmmmm but then again overall, perlin noise generation takes a miniscule amount of the cpu time spent
06:06 hmmmm the majority of it is allocating, copying, and deallocating memory thanks to extensive use of the STL
06:06 hmmmm death by a thousand cuts
06:06 gregorycu Yeah, I'm about to replace a std::map with a std::vector
06:06 hmmmm this is the kind of crap you don't optimize later on, but rather write it correctly the first time
06:07 hmmmm using a std::map here is 50% slower.  "but that's okay", they reckon
06:07 gregorycu Yeah, I'm not bad at the... higher level optimisations, or at least, not the instruction level stuff
06:07 hmmmm "so what?  it's just 50% of a very small amount of time"
06:07 hmmmm and that same design choice is repeated literally everywhere
06:07 hmmmm the result is an application that's 50% slower than it should be and it's nearly impossible to fix
06:08 gregorycu yep
06:08 gregorycu It's hard to fix the death by a tiny cuts cause many changes are very hard to quantify in terms of performance improvement
06:09 gregorycu You know it's there, it's hard to prove
06:09 hmmmm honestly the kind of "optimization" minetest needs involve changing the way entire things work
06:09 gregorycu lol
06:09 hmmmm stuff that will take way more involvement than changing a couple of data structures around
06:09 hmmmm and more than a weekend for sure
06:09 gregorycu I don't have a level of understandig with the engine that means I could do that
06:10 hmmmm exactly
06:10 gregorycu I'm an alright programmer, but yeah, need more knowledge before I can tackle that
06:10 hmmmm but this is the same problem as with every other FOSS project
06:10 hmmmm hard to penetrate and make meaningful changes to because it requires so much familiarity
06:14 paramat http://i.imgur.com/vMCH6eg.png found the ultimate protruding dungeon
06:14 hmmmm lol
06:15 gregorycu Can you call it a prolapsed dungeon?
06:15 hmmmm so we call these things "dungeons" but they honestly look like a random set of rooms connected by random hallways and stairs
06:15 hmmmm what makes them dungeons
06:15 gregorycu What is a dungeon?
06:15 hmmmm there needs to be a mod added to default that fills dungeons with interesting things like libraries (a la minecraft strongholds) or chests filled with goodies
06:15 hmmmm :(
06:15 paramat http://i.imgur.com/HKs8y7R.png and a good place for a base in a floating island, just build steps from the spike on the ground
06:16 paramat yeah okay i'll try to get loot added
06:16 paramat there are mods out there already that do that
06:17 hmmmm I bet they don't use GenNotifier
06:17 paramat hehe
06:18 hmmmm what
06:19 hmmmm ermm
06:19 hmmmm is print() in lua supposed to automatically append a newline?
06:19 paramat sometimes i wish it didn't
06:19 hmmmm this is a regression
06:20 hmmmm oh god
06:21 hmmmm thanks SN :(
06:22 paramat it's always appended a newline
06:23 hmmmm actually nvm this is an est bug :-
06:26 paramat blockmen has recently started working on a dungeon loot mod :) it uses gen notify
06:26 hmmmm good i'm glad people are using it for their mods
06:26 paramat i'll try to get it considered for game
06:30 hmmmm so what is holding back mobs from being put in the default game exactly?
06:30 hmmmm i keep hearing they're "laggy" but nobody ever explains further and several people say the opposite
06:34 hmmmm PTAL:  https://github.com/kwolekr/minetest/commit/70a3113141ceec8b29f868722c2f4dfba3c32283
06:35 paramat perhaps also because many players don't want hostile mobs by default
06:35 hmmmm this is est's pull request, with some improvements
06:35 hmmmm well lots of improvements actually
06:39 paramat time to omnom
06:39 paramat left #minetest-dev
07:04 gregorycu https://github.com/minetest/minetest/pull/3348
07:05 hmmmm ah, but what is the absolute improvement
07:05 gregorycu In terms of... fps?
07:05 hmmmm FPS, CPU time spent, etc.
07:07 gregorycu How often updateDrawListPerf is called depends on how much you look around
07:07 gregorycu If you look around lots, it's called every frame
07:08 gregorycu I circumvented the logic to make it be called every frame (to make sure my tests were consistant)
07:08 gregorycu CPU usage for that function from 3.73% down to 2.27%
07:09 hmmmm how much actual time is spent per call?
07:09 hmmmm microseconds?  milliseconds?
07:09 gregorycu I'll find out for you
07:09 hmmmm TimeTaker("asdf", NULL, PRECISION_MICRO);
07:09 hmmmm err
07:10 hmmmm TimeTaker t("asdf", NULL, PRECISION_MICRO); ... printf("updateDrawListPerf: %uus\n", t.stop());
07:11 gregorycu There is a scopeprofiler there already
07:11 hmmmm but does it have microsecond precision
07:13 gregorycu Just milli
07:13 gregorycu Ok, I'll use the code you said
07:28 Krock joined #minetest-dev
07:31 gregorycu Where does printf go?
07:32 gregorycu (When you're running as a client, not singleplayer)
07:35 hmmmm what, minetest doesn't open a console window for you?
07:35 hmmmm you run windows right?
07:36 gregorycu As it turns out, I'm retarded
07:36 gregorycu Yeah, it's there *sigh*
07:42 gregorycu results are in
07:43 gregorycu Before: 780us, After: 517us
07:43 hmmmm that's really significant for something that runs in a render loop
07:43 hmmmm we only have ~16ms to do our stuff
07:44 gregorycu It runs at least 5 times a second
07:44 hmmmm oh so not every loop, right
07:44 gregorycu (It doesn't run every frame, unless you're looking around a lot)
07:44 hmmmm er frame
07:45 gregorycu I'll add this detail to the PR
07:46 hmmmm it's a rather simple change so i don't see any reason why this shouldn't be merged
07:46 hmmmm even though it might not be a massive win
07:47 hmmmm gregorycu:  see how much more time you can save by reserving vector space
07:47 gregorycu Yeah, thought about that
07:47 paramat joined #minetest-dev
07:47 gregorycu But the vector is going to go organically, and stay inflated
07:47 gregorycu .clear doesn't delete memory
07:48 hmmmm oh nevermind
07:48 gregorycu So yeah, first couple of frames may be a little slower than they need to be
07:48 hmmmm the vector in question is a member of ClientMap
07:48 gregorycu Yep
07:48 gregorycu Used to be a map, where nobody ever uses the key to lookup
07:48 gregorycu It's possible the other function that uses this member will be a little faster
07:49 hmmmm looks good to me
07:50 Hunterz joined #minetest-dev
07:51 gregorycu Thanks
07:53 paramat reducing fps jitter is much needed
07:59 paramat fps stutter seems bad recently, someone has been analysing it https://forum.minetest.net/viewtopic.php?f=6&t=13360
08:01 hmmmm obviously the background threads need to be made lower priority
08:01 hmmmm but setting thread priority in non-windows is hard
08:03 gregorycu How many threads does minetest use?
08:04 gregorycu In singleplayer, looks like 4?
08:04 hmmmm nah.
08:05 hmmmm game thread, mesh making thread, minimap thread (if that's enabled), 4 AsyncRunQueue threads, server thread, connection send thread, connection receive thread (one of each for client and server), emerge thread, thread thread, thread thread thread
08:05 gregorycu So a few
08:05 hmmmm OpenAL thread I think
08:06 gregorycu I honestly think est's settings changes will improve this a lot
08:06 gregorycu We often read settings in a loop a lot
08:07 hmmmm yeah but is it really that slow?
08:07 hmmmm even under contention
08:07 gregorycu My singleplayer experience is magnitudes different than my client+server experience
08:07 gregorycu In terms of jitter
08:07 hmmmm well of course
08:07 gregorycu Of course?
08:08 hmmmm you're running half as many threads
08:08 gregorycu Same box
08:08 hmmmm oh
08:08 gregorycu I'd put money on it. Anyway, we can hoist some settings out of loops
08:09 gregorycu We should probably consider this - there is correctness issues too
08:09 gregorycu And yeah, segmenting the settings would be good
08:09 paramat i've actually noticed stutter gets worse when i look around a lot (singleplayer)
08:10 gregorycu Well, my latest change will make that a little better
08:10 gregorycu (feel free to merge and see)
08:11 paramat yeah good
08:19 gregorycu I had a run around, went high
08:20 gregorycu Make sure a lot of blocks were loaded. updateDrawListPerf used to take 3ms. Did the same thing after my change. 1.7ms. Which is much better.
08:20 gregorycu However, 1.7ms is a lot to add
08:20 gregorycu When turning
08:36 jin_xi joined #minetest-dev
08:52 nrzkt joined #minetest-dev
08:53 Player2 joined #minetest-dev
09:03 Calinou joined #minetest-dev
09:15 gregorycu Only one more +1 needed: https://github.com/minetest/minetest/pull/3348
09:16 Krock I'd give you one.
09:27 paramat i can't judge the implementation but +1
09:30 gregorycu Hooray!
09:35 RealBadAngel https://github.com/minetest/minetest/pull/3348/files#diff-37ea431d4de34ca3cc1fb0c79a36d1fbL414
09:36 RealBadAngel why is that removed?
09:39 gregorycu Nothing is using it, and populating the supporting member took CPU time
09:39 RealBadAngel thats irrlicht stuff i think
09:40 RealBadAngel maybe better just to comment it out instead of deleting?
09:41 gregorycu Irrlicht sources have no mention of sectorWasDrawn
09:41 gregorycu And no mention of sector, apart from "SecToRead"
09:41 celeron55 my current changes are going to remove it anyway
09:41 gregorycu :)
09:41 celeron55 merge conflict incoming
09:41 gregorycu Why comment out? This isn't the 90s. We have source control
09:42 gregorycu grrrr
09:42 celeron55 as i said, don't touch that area of the code; it will be just annoying for no good reason
09:42 gregorycu I didn't touch that area of code
09:42 RealBadAngel ok, nvm
09:42 gregorycu I was going to change the frustum culling
09:43 gregorycu In any case, are they incompatable changes?
09:44 celeron55 please, no
09:45 celeron55 this is hard enough without random conflicting changes being knowingly added upstream
09:46 celeron55 i am reworking map rendering; please don't rework map rendering
09:47 celeron55 i am also reworkring map transfer; please don't work map transfer
09:47 celeron55 -r
09:47 celeron55 rework*
09:48 celeron55 there are millions of other things in minetest that can be worked on meanwhile
09:51 gregorycu I was going to fix settings, but apparently est is fixing them. Failing that, i was going to do frustum culling, but you said not to touch then. Ok, so I looked at container inefficiencies with regards to the drawList calculation
09:51 gregorycu So, what do I do here? Withdraw my PR?
09:53 celeron55 well you can leave it hang around until i get this stuff into an upstream-worthy state
09:53 gregorycu How long is that?
09:53 celeron55 then it serves as a reminder to check whether the equivalent changes are still applicable or not
09:54 gregorycu If you're totally redoing map rendering, then it's pointless
09:54 gregorycu Assuming the code you replace it with doesn't also have the same wrong container type being used?
09:54 celeron55 not totally, but for example i already removed std::set<v2s16> m_last_drawn_sectors and replaced it with another system
09:55 celeron55 (it was originally used for the original FarMesh that i am making a successor to)
09:56 gregorycu You could merge my changes and rebase now
09:57 celeron55 i guess i'll add that to my to-do list
09:57 celeron55 this is now the eleventh thing on it
09:59 gregorycu I have other changes I'm working on around perf, I'll undo them
10:00 celeron55 if they aren't in map rendering or transfer, they aren't in conflict with what i'm doing
10:00 gregorycu I can wait until you're done, and I'll take another pass
10:01 celeron55 i'm done after maybe one week
10:01 celeron55 assuming i can keep up with this pace which i'm quite concerned about actually though
10:02 gregorycu Well, I can come back in 7 months like I did last time
10:02 celeron55 i might just burn out and be away for half a year if i fail 8-)
10:02 gregorycu 7 months > 6 months
10:02 gregorycu Should be good
10:02 celeron55 well in that case you can make anything you want meanwhile
10:02 RealBadAngel just find another random file and look for something to work on ;)
10:03 gregorycu My optimisations are guided by profiling data
10:03 gregorycu But maybe I'll look at a bug
10:05 gregorycu maybe I can look at windows bugs
10:36 celeron55 hmm
10:36 celeron55 looks like minetest will still be primarily using server-side block selection
10:37 celeron55 it's ending up making way more sense
10:37 celeron55 i'm still including a way to fetch blocks from any positions though
10:49 paramat left #minetest-dev
11:09 celeron55 i have so far found at least one bug in this block sending
11:09 celeron55 so maybe i am able to get rid of the mysterious block transfer holes
11:10 celeron55 (no promises)
11:15 gregorycu celeron55: How do you do sampling profiling on linux?
11:17 celeron55 pretty much the only way i ever do profiling is callgrind
11:17 celeron55 veeeeery slow but works well
11:17 gregorycu isn't that rather disruptive (slows down the app)
11:17 gregorycu ok
11:17 celeron55 yeah; good luck getting 0.1fps
11:19 gregorycu Urgh
11:19 gregorycu Is there a non-disruptive sampling profiler for linux?
11:21 PilzAdam joined #minetest-dev
11:29 nrzkt joined #minetest-dev
11:46 gregorycu_ joined #minetest-dev
11:49 gregorycu joined #minetest-dev
12:07 Fixer joined #minetest-dev
12:25 rubenwardy joined #minetest-dev
12:40 celeron55 bleh
12:41 celeron55 i think this whole emerge system has been broken for a huge amount of time making so that a server will waste a lot of I/O resources if max_block_send_distance is larger than max_block_generate_distance
12:41 zupoman joined #minetest-dev
12:41 zupoman joined #minetest-dev
12:41 celeron55 this is because how this system was supposed to work is that if a block is not found on disk and won't be generated, it will be added to the Map as a dummy block
12:41 celeron55 but for some reason something has been changed so that it won't be added at all
12:42 celeron55 and this results in infinite polling of the same blocks from the database or disk
12:43 celeron55 i'm not sure yet but something doesn't seem right in how this is behaving
12:44 gregorycu :/
13:01 celeron55 i wonder at which point this really is broken
13:17 celeron55 hmm... looks like they actually end up full of CONTENT_IGNORE if i am not grossly mistaken
13:19 celeron55 okay i think that is actually fine; the issue is how this algorithm treats non-generated blocks
13:20 celeron55 maybe that has changed some years (lol) ago and not surprisingly nobody figured that this wasn't originally prepared to handle it 8)
13:22 celeron55 i think the issue is that block->isGenerated() has changed meaning
13:22 Amaz joined #minetest-dev
13:24 Calinou I make my server use same generate and send distance, problem solved
13:26 celeron55 this assumes that if a block isn't generated and isn't dummy, throwing it at the emergethread will cause it to get generated - which simply doesn't happen now as in emergethread's opinion a block like that is the block's final form until it is allowed to generate it; which is obvious but previously all of those might have been dummies in practice
13:28 celeron55 ...previously as in "maybe a few years ago" 8)
13:37 BlockMen joined #minetest-dev
13:38 celeron55 wait, what... i think there might still be a problem with missing dummy blocks too
13:40 celeron55 maybe not; i am not getting consistent results with repeated tests
13:43 zat joined #minetest-dev
14:31 celeron55 i wonder if we should drop the view distance auto-maybe-figure-it-out thing and just tell the user to use shortcuts or settings to tweak it
14:32 Calinou yeah, IMO the auto-tuner is a flawed idea
14:32 Calinou set a sane value by default, and tell user to tweak it on first run
14:32 celeron55 it would be much easier with this farthing to just add ctrl-[+-] or something to tune the far distance instead of trying to make something that would juggle those automatically
14:32 celeron55 i mean, i still think it's fine for one value, but with these two it becomes practically impossible
15:01 zupoman joined #minetest-dev
15:37 celeron55 hmm; i measured an 1.2% worse rendering performance when using std::map<v3s16, FarBlock*> instead of std::map<v2s16, std::map<s16, FarBlock*>>
15:38 celeron55 i guess this is just a measurement error
15:38 celeron55 but this would speak against dropping the sector stuff
15:39 cib0 joined #minetest-dev
15:42 celeron55 i even measured std::map<v3s16, FarBlock*> first so std::map<v2s16, std::map<s16, FarBlock*>> should have been subjet to more CPU throttling due to heating too
15:45 liq_ovr_ignore joined #minetest-dev
16:01 PilzAdam celeron55, what about std::map<s16, std::map<s16, std::map<s16, FarBlock*>>>?
16:02 celeron55 i have tens of more important things to do than to test these, franky, so that will stay as a mysteryr
16:02 celeron55 -r
16:02 celeron55 but you can try if you want; just fill in about 600 heap-allocated pointers into it and iterate through it a lot
16:05 rubenwardy joined #minetest-dev
17:08 est31 joined #minetest-dev
17:17 est31 pushing in 10 mins: https://github.com/est31/minetest/commit/b55477798ae999d58e4ee84d538b4ee1757912c9
17:17 est31 fixes #3349
17:18 est31 https://github.com/minetest/minetest/issues/3349
17:33 est31 ah perhaps the commit message should have been a bit more nicer
17:33 est31 e.g. to point out that its only about srp pcgrandom
17:33 est31 not about numeric.h pcgrandom (or wherever our pcgrandom now resides)
17:39 DFeniks joined #minetest-dev
17:39 est31 and no answer for http://lists.gnu.org/archive/html/bug-gettext/2015-10/msg00025.html
17:39 est31 very sad, PilzAdam seems I'll run updatepo.sh then without a fix for the issue
17:40 PilzAdam :-/
17:41 est31 but before, I'll merge all the translation commits
17:41 hmmmm joined #minetest-dev
17:41 est31 59 commits
17:41 est31 oh man
17:43 hmmmm est, that's not the only problem though.  if that random is supposed to be "secure", why does it produce the same result every clock() tick?
17:44 est31 what do you mean?
17:45 est31 you mean if time(NULL) and clock() have the same output than the seeded value, the random can be reproduced?
17:45 est31 well, thats the problem if we have no /dev/urandom
17:45 est31 I dont want to rewrite the kernel's pseudorandom pool
17:46 hmmmm well if that system doesn't have a urandom you're lying to the user that their account is cryptographically secure
17:46 cib0 joined #minetest-dev
17:46 est31 you want me to call abort() or something?
17:46 hmmmm because then you have a random that's about as secure as debian's RNG if called fast enough
17:47 est31 or better, pass the error to the calling function
17:48 hmmmm did you look at this https://github.com/kwolekr/minetest/commit/70a3113141ceec8b29f868722c2f4dfba3c32283
17:49 est31 not yet
17:51 est31 what is changed between this impl and my pr?
17:52 hmmmm a lot
17:52 hmmmm but the biggest thing is that the PcgRandom fallback stays secure in between clock ticks, it alerts the user, the fill loop logic is simplified and a rather serious mistake was fixed
17:53 est31 serious mistake?
17:54 hmmmm the first m_rand_idx bytes of outputbuf are uninitialized
17:54 hmmmm unset rather
17:54 est31 oh
17:55 est31 I'll have a closer look at it later, right now I want to review the translations, ok?
17:55 hmmmm also this version tries to fall back to /dev/random if /dev/urandom can't be found
17:55 hmmmm and there's more error checking in general
17:55 est31 thats not good
17:55 hmmmm why not
17:55 est31 /dev/random is blocking
17:55 hmmmm so?
17:56 hmmmm also that sounds to me like platform-specific behavior
17:56 est31 Rather fall back to something entirely insecure than something blocking
17:56 hmmmm that's a horrible justification for being insecure
17:56 est31 This is a game, no banking app
17:56 hmmmm security should come above all else
17:56 rubenwardy What's the point of allocating 4 bytes when you only use 2? Not important, just wondering if I'm missing anything: https://github.com/minetest/minetest/blob/master/src/server.cpp#L731
17:57 est31 people want to play it, not wait for /dev/random
17:57 sfan5 in wonder what kind of broken or outdates you unix must be to have /dev/random but not urandom
17:57 sfan5 s/in/i/
17:57 hmmmm if they decided that security wasn't a very important part of their random, they probably would not be using a class called "SecureRandom"
17:58 hmmmm sfan5, Solaris 8
17:58 sfan5 does irrlicht run on solaris?
17:58 hmmmm probably
17:58 sfan5 ..
17:58 sfan5 do we intend to support running MT on solaris?
17:58 hmmmm completely irrelevant question.
17:59 hmmmm if you code to a set of platforms you explicitly "support", you're setting yourself up for failure
17:59 est31 hmmmm, in the case /dev/urandom doesn't exist we should expose it to lua, and make the constructor return nil.
17:59 est31 my opinion
17:59 hmmmm and the PcgRandom backup?
17:59 hmmmm just remove that altogether?
18:00 sfan5 i agree that doing stuff like only supporting specific versions of distros is stupid
18:00 sfan5 but deciding which kernel to support is different
18:00 sfan5 it can be done if it doesn't hurt
18:00 est31 "Starting from Solaris 9, /dev/random and /dev/urandom became device nodes since a kernel-based implementation was done"
18:01 est31 https://blogs.oracle.com/yenduri/entry/dev_random_in_solaris
18:01 sfan5 (which is debateable in the /dev/random case)
18:01 sfan5 but we don't need to write code to run MT on PonyOS
18:01 hmmmm well maybe you don't think you should
18:01 hmmmm I won't +1 code that *breaks* on PonyOS because you decided on being stubborn
18:02 celeron55 here's an interesting picture: http://i.imgur.com/9JJYg7T.jpg
18:02 sfan5 it's not stubbornness
18:02 sfan5 it's supporting a sane set of platforms
18:02 celeron55 i happened to walk around on the left side of the mountain before jumping down where i took that screenshot
18:02 est31 hmmmm, your patch won't run on PonyOS either, it will block, not run.
18:02 celeron55 and just enabled rendering of everything
18:02 est31 your version of my patch*
18:02 hmmmm est31, what does it wait for?
18:03 est31 for input from the hw
18:03 hmmmm and again the blocking thing is totally implementation specific
18:03 hmmmm that is a Linux-ism
18:03 est31 do a hexdump /dev/random
18:03 est31 try it,
18:03 est31 it will spew some bits, then be silent
18:03 hmmmm in general /dev/random has nothing to do with blocking on randomness
18:03 est31 it does
18:03 hmmmm est31:  I get an immediate number of bytes, like I want
18:03 hmmmm est31:  on my system /dev/random is symlinked to /dev/urandom
18:04 hmmmm and I'll be willing to bet it's different on every single OS
18:05 est31 yeah seems you're right
18:05 est31 "Unlike the Linux /dev/random, the FreeBSD /dev/random device never blocks"
18:05 est31 https://en.wikipedia.org/wiki//dev/random
18:05 hmmmm we're talking about "Linux" and "FreeBSD"
18:05 hmmmm how about "All OSes that conform to the POSIX standard"
18:06 est31 well, /dev/random blocks sometimes, sometimes it doesn't
18:06 hmmmm the point is
18:06 hmmmm /dev/random is not required by the standard at all
18:06 hmmmm neither is urandom
18:07 hmmmm in fact the only files you can guarantee for /dev to have are /dev/null, /dev/tty, and /dev/console
18:07 est31 both my patch and yours work for Linux, freebsd, and solaris
18:08 est31 the question is behaviour outside of those operating systems
18:08 hmmmm the answer is:  it's a crapshoot
18:08 hmmmm some might block
18:08 hmmmm some might not
18:09 hmmmm but in general I think it is safe to assume that if an OS does have /dev/random, it has /dev/urandom as well
18:09 est31 OpenBSD has urandom too
18:09 celeron55 can someone explain why the blocking of a random device is such a problem
18:10 hmmmm that assumption could definitely be incorrect, but things should be coded in such a manner to fail in a safe way
18:10 est31 celeron55, blocking can happen for long periods of time, e.g. seconds.
18:10 celeron55 they only block if you pull huge amounts of bytes from them and you shouldn't be doing thatt
18:10 hmmmm using an unsafe random in "SecureRandom"... that's unacceptable IMHO
18:10 celeron55 -t
18:11 est31 due to minetest's single threaded nature, you would block the entire server
18:11 hmmmm I advocate for returning nil on secure rand failure
18:11 est31 okay for me
18:11 sfan5 that' the only sane choice
18:11 hmmmm okay then I'll change it
18:11 est31 without /dev/random or pcgrandom
18:12 est31 btw see what openbsd says on /dev/random
18:12 est31 http://nixdoc.net/man-pages/openbsd/man4/urandom.4.html
18:12 est31 "reserved for future support of hw random"
18:13 est31 so, /dev/random on openbsd isnt available in most cases
18:13 est31 it seems what /dev/random is on linux, is /dev/srandom on openbsd
18:13 hmmmm and /dev/srandom blocks on enthropy
18:13 hmmmm yup
18:14 hmmmm i suppose /dev/random exists for newer intel processors with RDRAND
18:14 est31 ah that thingy
18:14 hmmmm or any systems with TPMs or cryptographic modules (like, an actual PCI card that does nothing but crypto)
18:14 hmmmm (those exist believe it or not)
18:15 est31 I think I've heard that the devs of some crypto library, perhaps it was actually openssl, have a strong dislike against intel^TM random generator
18:15 est31 they believe it is backdoore
18:15 est31 d
18:15 hmmmm everybody with half a brain wouldn't trust rdrand
18:15 sfan5 >some crypto library
18:15 sfan5 you mean linux
18:15 est31 ah that can be it
18:15 hmmmm anybody*
18:16 hmmmm it probably uses the NSA Powered(tm) Dual Eliptic Curve rand
18:16 sfan5 https://en.wikipedia.org/wiki/RdRand#Reception
18:16 DFeniks joined #minetest-dev
18:24 est31 LOL Kartenerzeugerbiomluftfeuchtigkeitsübergangsrauschparameter
18:24 est31 wuzzy likes komposita
18:24 Krock Oh gott..
18:25 * Krock adds that to his "long words" list
18:27 celeron55 holy shit this FarBlock geometry consumes a lot of memory
18:28 celeron55 basically two gigabytes
18:28 Calinou new splash screen proposition: "More RAM than Minecraft ever consumes!"
18:28 celeron55 well i think this is good use for RAM
18:28 celeron55 you can always disable it 8)
18:30 celeron55 i need to make more features that make good use of RAM because 2 gigabytes is only 13% of what i have
18:32 hmmmm yeah I said the same thing about my perlin noise algorithm
18:32 hmmmm it's causing problems now... people constantly complain about high memory usage
18:34 est31 well I have 4 GB
18:34 est31 and I usually have other programs open than minetest as well
18:34 est31 like a browser with github
18:34 est31 or hexchat
18:34 est31 or a second instance of minetest
18:34 est31 one server, other one client
18:34 est31 to test sth
18:35 * est31 doesnt want to know what Krock has
18:35 * Krock tells est31 that he has 2 GiB ram - more than a RPI
18:36 hmmmm celeron used to use a really underpowered laptop iirc
18:36 celeron55 of course just lowering the range gets the memory consumption down while still being useful
18:36 Krock RAM can be extended using the harddisk. The problem is mostly the CPU
18:37 est31 is it server RAM or client RAM?
18:37 hmmmm est31:  https://github.com/kwolekr/minetest/commit/21287bc2cd0bdead0d62c05c79de407abd5fc0ed
18:37 celeron55 client
18:37 est31 hrmm I guess it creates a copy of all the textures?
18:37 est31 what if you try one of VE's high res texture packs
18:38 est31 https://forum.minetest.net/viewtopic.php?t=1583
18:38 hmmmm celeron55:  please, please, please work on reducing memory usage before calling it "done"
18:38 celeron55 hmmmm: you can't; it's just how much memory that much terrain geometry consumes
18:39 celeron55 have you ever looked at how much memory modern PC games use
18:39 hmmmm i don't want another situation of "oh it just takes up 100% cpu usage because AMD cpus don't work well with threads"
18:39 celeron55 large meshes are large; it's just how it is
18:40 hmmmm you can try constructing the meshes more efficiently
18:40 celeron55 but there are a few options i can add that trade RAM to more CPU time
18:40 hmmmm or optimize them with weldMesh()
18:40 hmmmm for mostly static meshes this ^ might be a good choice
18:40 hmmmm especially if you're constructing it in a separate thread
18:40 celeron55 what weldMesh
18:41 celeron55 are you insane; i might just as well stop right now if i was to do evrything you think is needed
18:41 hmmmm in my irrlicht terrain viewer i implemented a basic minetest-like block renderer and welding the meshes reduces vertices by half
18:41 celeron55 what i will do is make a stable feature
18:41 celeron55 just don't use the feature while you think it's not optimized enough
18:42 celeron55 i'm already proud of this and you should be too
18:42 est31 from the screenshots it looks really great
18:43 hmmmm yea I guess
18:43 hmmmm the nodes look really... line-y for some reason but I can live with that
18:43 celeron55 it's a good way of cheating the eye of more detail than there actually is
18:44 celeron55 it's what makes it look even remotely reasonable
18:44 hmmmm yeah agreed
18:44 hmmmm this is soo much better than the old farmesh
18:45 Krock2 joined #minetest-dev
18:46 est31 side note, it mostly has worked with the protocol mismatch stuff
18:47 est31 most of it ... makes sense.
18:59 est31 woow only 10 commits to review
19:13 Krock Not much ^^
19:16 RealBadAngel that farmesh mountain could look quite nice if it was blurred somehow
19:16 hmmmm shaders
19:18 RealBadAngel https://imgrush.com/X6Zh5Z_DBJ_X.jpg
19:18 RealBadAngel with a simple blur it does look nice
19:20 est31 what the hell
19:21 est31 somebody has changed the translation of "If disabled " from "Отключить мультиплеер" to "Если выключено_"
19:21 est31 now the old version is BS
19:21 est31 automatic BS
19:21 est31 it means "disable multiplayer"
19:21 est31 but neither microsoft translator nor google translator can cope with the new version
19:22 est31 and I really wonder about that last char
19:22 est31 its displayed as _ (underscore) for me in gitk and everywhere else
19:22 RealBadAngel Отключить мультиплеер == enable multiplayer
19:22 est31 but kate displays it at space
19:22 RealBadAngel 2nd string means "if disabled"
19:23 est31 hrmm, the problem seems to be that last char
19:23 RealBadAngel underscore here whatever it is is wrong
19:23 est31 will see whether its that way in the final version as well, and remove it if its there
19:44 est31 well, then lets push it all
19:53 JohnnyComeL8ly joined #minetest-dev
19:56 Warr1024 joined #minetest-dev
20:03 Warr1024 joined #minetest-dev
20:19 est31 PTAL https://github.com/est31/minetest/commit/9a91f58e66e84bfba13c967753d537996e4c5d6d
20:20 PilzAdam +1
20:20 PilzAdam (yes, I just reviewed that in 14 seconds)
20:20 est31 xD
20:26 est31 hrmm I should probably have squashed those PilzAdam commits that set whether fallback font is enabled
20:26 est31 well shrug, commit spam stays commit spam
20:26 est31 to finish it, I push two additional commits
20:26 PilzAdam my commit count doesn't complain either ;-)
20:27 est31 Congrats to commit #200 PilzAdam
20:38 Darcidride_ joined #minetest-dev
20:53 hmmmm ridiculous
20:53 hmmmm that spams the commit history
20:53 hmmmm i think it should've been squashed
20:55 est31 all translations?
20:55 hmmmm yes
20:56 est31 this way we have working git blame
20:56 hmmmm if attribution is the problem with that, we can individually list each contributor
20:56 hmmmm btw what was your final word on the LuaSecureRandom
20:56 est31 +1
20:56 hmmmm alright pushing then
21:12 Miner_48er joined #minetest-dev
21:18 liq_ovr_ignore_b joined #minetest-dev
21:22 hmmmm PTAL  https://github.com/kwolekr/minetest/commit/a0333be25c71a9f454ac0fef3a74f5711624d1e8
21:25 est31 can you add a comment why NULL isnt used here : https://github.com/kwolekr/minetest/commit/a0333be25c71a9f454ac0fef3a74f5711624d1e8#diff-3465219267c4baa9779cd47acef1bf20R267
21:25 est31 otherwise sb will change it back one day thinking 0 is bad
21:25 hmmmm bleh
21:26 est31 and about the is.gcount() check, what if gcount is larger than u32?
21:27 est31 the cast will cut off the higher bits, no?
21:27 hmmmm it will, and then it won't be equal to s_size
21:27 hmmmm throwing the exception
21:28 hmmmm besides, the max long string length is 4gb
21:28 est31 well but what if the gcount is 2^33 + 42 and s_size is 42
21:28 hmmmm then i would have to argue that your platform's std::istream implementation is horrendously broken
21:29 est31 ?
21:29 est31 what if the client actually sends such a large packet
21:29 est31 just to be evil or so
21:30 hmmmm gcount is the number of characters read from the *last* operation
21:31 hmmmm if you request 42 bytes to be read, but istream actually reads 0x10000000 + 42 characters... well then
21:31 est31 ah
21:31 est31 ok nvm then
21:31 est31 +1
21:31 hmmmm you'd have to have a packet that's actually that many characters
21:32 * est31 should RTFM
21:32 est31 before talking
21:32 hmmmm and you'd also have to have a broken std::istream implementation that has a literal bug
21:34 est31 yea, your commit looks ok
21:46 est31 PTAL https://github.com/est31/minetest/commit/92b8b38883cb9cf88bf813b47a07629760e0bb83
21:47 est31 new version https://github.com/est31/minetest/commit/8afb8b3a75d58278628ad4fb5826bd4235d3ffb0
21:48 hmmmm yup, looks good
21:50 est31 ah
21:50 est31 stdout is text mode right?
21:50 hmmmm yes?
21:50 est31 so fputs(stdout, "\n"); will write \r\n on windows?
21:51 hmmmm maybe it does
21:51 hmmmm it doesn't matter because lua does the same thing
21:51 hmmmm the goal is to make this equivalent to what lua did
21:51 est31 well yeah
21:52 est31 I use std::cout here
21:52 est31 so the question is, what is the analogon for fputs(stdout, "\n")
21:52 est31 is it  std::cout << std::endl or  std::cout << '\n'.
21:53 est31 in the worst case, I just can use fputs as well
21:53 est31 then its even more the same
21:53 hmmmm the hell is wrong with exceptions now
21:54 hmmmm no exceptions display the error message
21:55 est31 I'll usestd::endl and hope the best
22:02 est31 ?
22:07 nrzkt std::endl is perfect for cross platform ending lines
22:11 celeron55 the code of VoxelManipulator looks ridiculous
22:11 celeron55 it has comments that are downright wrong, and commented-out code that has nothing to do with how minetest works these days
22:13 celeron55 straight from 2010
22:14 hmmmm it works so there's no reason to fix it
22:14 hmmmm unless you're SN
22:14 celeron55 i might remove the misleading comments some day
22:14 celeron55 but otherwise who cares
22:16 celeron55 it's just funny how this was a blazing new feature a year or two ago while nothing has changed at any point in its 5-year lifespan
22:17 celeron55 except for the lua interface 8)
22:18 hmmmm how plausible is it to decouple objects from mapblocks?
22:18 celeron55 decouple into where
22:19 hmmmm another table in the map db, or maybe a new db
22:19 hmmmm sqlite can easily perform efficient range queries; i'm not so sure about leveldb and others however
22:19 celeron55 it's possible but you still need some kind of a spatial database for storing them that mapblocks already essentially are
22:20 celeron55 you kind of need a reason for it
22:20 hmmmm exactly
22:20 paramat joined #minetest-dev
22:20 hmmmm if objects are decoupled from the map, that's one step closer to locklessness
22:20 hmmmm it's also greatly simplify the whole static object storage
22:21 hmmmm thinking that it might be a good idea to separate a map lock from an object lock
22:21 celeron55 what about copying blocks with objects included
22:21 celeron55 does anyone do that in any form
22:21 celeron55 currently it's kind of trivial at least in theory
22:22 hmmmm it's not because of like i said, leveldb
22:23 JohnnyComeL8ly Well, hmmmm, I thought the default was sqlite.
22:24 celeron55 sqlite has never failed me and i will use it until the end of time for the exact reason
22:25 paramat nore PilzAdam sfan5 i would be interested in your comments on https://github.com/minetest/minetest_game/pull/719 Default, fire: Disable catch-up in some ABMs
22:26 hmmmm well
22:26 hmmmm maybe it's not very advantageous to decouple objects from mapblocks since you're going to need to query the map for collision detection
22:27 hmmmm i'm trying to think what objects spend most of their time doing
22:27 est31 objecting what else
22:28 est31 "I don't want to work properly"
22:28 est31 I want 1 s more lag!
22:28 hmmmm we should stop objectifying them
22:29 hmmmm SAOs have feelings too you know
22:29 est31 the moment they object to being objectified they prove they are just objects
22:29 hmmmm std::vector<Feeling> m_feels;
22:29 celeron55 you have to stab them dead while the server is running a lua map generator; then they don't feel pain
22:30 hmmmm >tfw i get deleted because my mapblock has too many static objects
22:30 hmmmm >tfw no player wants to pick me up
22:30 hmmmm feels bad man
22:30 celeron55 meme overload
22:31 celeron55 or maybe virtual meme, because we don't use java
22:31 hmmmm dank memes
22:31 celeron55 maybe i need to sleep
22:31 hmmmm same
22:31 hmmmm later.
22:32 celeron55 wait, that was actually a really bad joke because what i was thinking was override, not overload
22:40 JohnnyComeL8ly celeron55, needs sleep... sweet dreams to him!
22:41 paramat +1 for removing auto view range
23:00 est31 btw I can make a native alternative to /clearobjects which works without anything loaded but the database. It can run in 20 minutes on a VanessaE server , which is much faster than the normal /clearobjects
23:00 est31 I could make a patch, interested??
23:07 Darcidride_ joined #minetest-dev
23:08 paramat i'm sure that will be very useful
23:14 hmmmm back
23:15 hmmmm PTAL https://github.com/kwolekr/minetest/commit/fe8b4d78e404f393f5c215fd7db121177340b250
23:15 hmmmm whoops hold on
23:16 hmmmm okay PTAL  https://github.com/kwolekr/minetest/commit/88a3977954f537b391c714be2a52da6b19711154
23:18 Puka joined #minetest-dev
23:21 est31 is there no thread safe way of doing it
23:22 est31 ah
23:22 est31 nvm
23:22 est31 ""
23:22 est31 The WSAGetLastError function returns the last error that occurred for the calling thread.
23:25 est31 you aren't including errno.h
23:25 hmmmm https://github.com/kwolekr/minetest/commit/88a3977954f537b391c714be2a52da6b19711154#diff-d999b22f05a7d6687dee3151d4552f0aL26
23:28 est31 other thing: ah
23:28 est31 ermm
23:28 est31 ah
23:28 est31 other thing: if user comes on the forums "creating socket failed : error number 123"
23:28 est31 how will we debug it
23:28 est31 is it perhaps an idea to append whether its unix or windows?
23:29 hmmmm nah that's something they can tell us
23:29 hmmmm also version is reported in the logs
23:29 est31 well, it requires one more back and forth
23:29 hmmmm and besides, windows network error codes are that of unix, + 10000
23:29 hmmmm so you can tell right away what platform this is
23:29 est31 ah okay then
23:30 est31 is there a thread safe way to access the errno?
23:30 hmmmm errno is local to the running thread...
23:30 est31 is it? isn't it a global variable?
23:30 hmmmm nope
23:31 est31 how would that work
23:31 est31 virtual memory?
23:31 hmmmm thread local storage
23:31 est31 ah I see
23:32 est31 lol errno is even used as example on the TLS wiki page
23:32 est31 well +1 then
23:37 paramat i broke snowy pines last night, will fix very soon =s
23:45 paramat left #minetest-dev

| Channels | #minetest-dev index | Today | | Google Search | Plaintext