| Time |
Nick |
Message |
| 05:24 |
|
celeron55_ joined #minetest-dev |
| 05:33 |
|
VanessaE joined #minetest-dev |
| 06:11 |
|
VanessaE joined #minetest-dev |
| 06:46 |
|
VanessaE joined #minetest-dev |
| 07:39 |
|
sema4 joined #minetest-dev |
| 07:44 |
|
Taoki joined #minetest-dev |
| 09:40 |
|
rsiska joined #minetest-dev |
| 10:17 |
|
BackupCoder joined #minetest-dev |
| 11:39 |
|
Taoki joined #minetest-dev |
| 11:55 |
|
iqualfragile joined #minetest-dev |
| 11:56 |
|
iqualfragile joined #minetest-dev |
| 11:59 |
|
jojoa1997 joined #minetest-dev |
| 11:59 |
|
jojoa1997 left #minetest-dev |
| 12:06 |
|
Gambit joined #minetest-dev |
| 14:22 |
|
darkrose joined #minetest-dev |
| 14:22 |
|
darkrose joined #minetest-dev |
| 15:21 |
|
PilzAdam joined #minetest-dev |
| 15:36 |
|
Calinou joined #minetest-dev |
| 16:45 |
|
hmmmm joined #minetest-dev |
| 17:30 |
iqualfragile |
please merge that in: https://github.com/celeron55/minetest/pull/426 |
| 17:34 |
celeron55_ |
ehm |
| 17:34 |
celeron55_ |
that will just make huge liquid transform queues when a server runs a long time? |
| 17:35 |
celeron55_ |
and they will all be tried every second, or whatever the way that works was |
| 17:35 |
celeron55_ |
shouldn't be merged |
| 17:36 |
celeron55_ |
(without testing that the queue does not actually grow in a real situation) |
| 17:45 |
hmmmm |
jeepers.... that's insane |
| 17:45 |
hmmmm |
don't do it |
| 17:47 |
celeron55_ |
hmmmm: what is the status of your... thing? people are telling all kinds of rumors about it, most of which are quite far from reality 8) |
| 17:47 |
hmmmm |
oh shoot |
| 17:47 |
hmmmm |
it's done |
| 17:47 |
celeron55_ |
i might want to write a blog post about the status of many things |
| 17:48 |
hmmmm |
i haven't been doing anything minetest related for about a week, i apologize. |
| 17:48 |
celeron55_ |
doesn't matter; can you summarize what it does and what it needs, so that it can be replied to anyone who asks |
| 17:49 |
hmmmm |
sure |
| 17:49 |
celeron55_ |
+? |
| 17:49 |
hmmmm |
where would this be posted? |
| 17:49 |
|
doserj joined #minetest-dev |
| 17:49 |
celeron55_ |
dunno, maybe c55.me/blog, and if not, then at least everybody in this channel knows it and can inform others properly |
| 17:50 |
hmmmm |
alright |
| 17:50 |
hmmmm |
i should probably mention that not all the changes in my branch are mapgen related; i did take some time out to fix some minor nagging issues along the way |
| 17:54 |
iqualfragile |
oh, yeah, i have to admid i did not look at it, i was just annoyed by that problem |
| 17:54 |
iqualfragile |
maybee load the needed blocks? |
| 17:54 |
celeron55_ |
it will create infinite loadings of blocks |
| 17:55 |
celeron55_ |
we have an infinite world; everything non-linear just stops at some point, there's no silver bullet to that |
| 17:55 |
celeron55_ |
(dunno if non-linear is the right term though) |
| 17:57 |
doserj |
the proper way would be to store the queue with the block |
| 17:57 |
doserj |
and add it to the real queue when the block is loaded |
| 17:57 |
doserj |
(I guess) |
| 17:58 |
celeron55_ |
i think it would work if water would stop, without spreading, when it touches unloaded mapblocks, and when a mapblock is loaded, it would "activate" all water touching it to make it move if it is about to move |
| 17:58 |
celeron55_ |
s/water/liquid/ |
| 17:59 |
celeron55_ |
if somebody does that, it will be merged |
| 17:59 |
doserj |
you need more than just touching |
| 17:59 |
doserj |
you need to activate all water in the block itself |
| 17:59 |
doserj |
(the block may be unloaded while water is flowing inside it) |
| 17:59 |
celeron55_ |
well that is obvious (dunno if it is done already) |
| 18:01 |
doserj |
I'm pretty sure it doesn't |
| 18:04 |
doserj |
And you have to be careful with your solution that it doesn't blow up when you load an ocean block full of water |
| 18:09 |
celeron55_ |
indeed that could be bad |
| 18:10 |
celeron55_ |
maybe the queue should be stored then, for the in-block cases - it's actually a different case compared to the one that started this discussion though |
| 18:45 |
|
sfan5[iPod] joined #minetest-dev |
| 18:56 |
|
sfan5[iPod] joined #minetest-dev |
| 19:15 |
|
sapier1 joined #minetest-dev |
| 19:17 |
|
sapier joined #minetest-dev |
| 19:18 |
|
sfan5[iPod] joined #minetest-dev |
| 21:49 |
* celeron55_ |
gives a bit of more visibility to this: http://forum.minetest.net/viewtopic.php?id=4243 |
| 22:07 |
hmmmm |
guys |
| 22:07 |
* VanessaE |
raises an eyebrow |
| 22:08 |
hmmmm |
i think we should all start working on minecraft more |
| 22:08 |
hmmmm |
minetest |
| 22:08 |
VanessaE |
I thought we were?> |
| 22:08 |
hmmmm |
we were but not enough |
| 22:08 |
hmmmm |
there's like nothing going on :( |
| 22:08 |
hmmmm |
i have a goal in mind; by friday at the latest we should have master with all the mapgen stuff and leveldb stuff added |
| 22:09 |
hmmmm |
and more than that.. before the last bit of snow melts for me outside, I want to have snow on the blocky ground in a winter biome in minetest |
| 22:10 |
VanessaE |
how much leveldb "stuff"? |
| 22:10 |
hmmmm |
all the leveldb 'stuff' |
| 22:10 |
VanessaE |
just the framework, or the whole backend like oldcoder wants? |
| 22:10 |
hmmmm |
the whole backend |
| 22:10 |
hmmmm |
plus a converter for sqlite3 maps |
| 22:10 |
hmmmm |
oh wow that might be a bit taller of an order |
| 22:10 |
hmmmm |
okay nevermind friday |
| 22:10 |
VanessaE |
sweet |
| 22:10 |
VanessaE |
get with oldcoder - he's already done this |
| 22:10 |
hmmmm |
well |
| 22:11 |
VanessaE |
he runs leveldb on his server 'nexus' |
| 22:11 |
hmmmm |
thexyz is the person i'd really want to talk to |
| 22:11 |
VanessaE |
at least for linux... |
| 22:11 |
init |
leveldb? |
| 22:11 |
hmmmm |
it's an alternative to sqlite3 that runs a whole lot better |
| 22:11 |
init |
what is it? new format for maps faster? |
| 22:11 |
hmmmm |
yes, it makes maps faster :). |
| 22:11 |
init |
;P |
| 22:11 |
hmmmm |
so here we are with leveldb |
| 22:12 |
hmmmm |
the framework and the backend itself have been tried and proven to work well for a couple months now |
| 22:12 |
hmmmm |
plopping that in should be no problem |
| 22:12 |
hmmmm |
i don't know what it does about conversion of old maps, which is where i'd ask xyz |
| 22:13 |
hmmmm |
but ideally what would be done is all old maps are _converted_ to the new leveldb format |
| 22:13 |
hmmmm |
and then eventually we remove the code supporting loading from and everything from 0.3 sector directory and and 0.4 sqlite3 |
| 22:13 |
sapier |
I don't want to offend someone but issues are queuing up really fast and noone doesn't even seem to categorize them |
| 22:14 |
hmmmm |
issues on what |
| 22:14 |
init |
github i think |
| 22:14 |
sapier |
yes github... isn't bugfixing more important than adding new featues? |
| 22:15 |
hmmmm |
a lot of those are enhancements and very non-critical bugs |
| 22:15 |
sapier |
of course but without categorizing it you can't decide wich are critical bugs and which are feature requests |
| 22:15 |
VanessaE |
hmmmm: oldcoder has also gotten quite proficient at migrating sqlite maps -> leveldb btw |
| 22:16 |
hmmmm |
sapier, i know, but just froma glance it really does look like the majority of them are enhacements |
| 22:16 |
hmmmm |
from a* |
| 22:16 |
hmmmm |
vanessa, that's nice, but it's not particularily difficult to do so |
| 22:16 |
hmmmm |
the plan is to have it completely automated |
| 22:17 |
VanessaE |
hmmmm: just trying to save some time. |
| 22:17 |
init |
i wonder |
| 22:17 |
hmmmm |
user upgrades minetest, copies over map, starts it up, says "hey your map format is out of date, to use this version you need to update it!" |
| 22:17 |
VanessaE |
init: I already know, and I'm not sure I like what I know ;-) |
| 22:17 |
init |
why not a "big upgrade" between adding the new mapgen / new format |
| 22:17 |
VanessaE |
(as opposed to just wondering ;) ) |
| 22:17 |
hmmmm |
and then it asks for a confirmation if you'd like to overwrite the file or move it to another or something |
| 22:17 |
VanessaE |
oh yes |
| 22:17 |
VanessaE |
+1 |
| 22:18 |
sapier |
there are at least some important bugs ... entity attachement is buggy, collision handling too, entity client/server out of sync and of course the famouse emerge thread bug is still there |
| 22:18 |
hmmmm |
hm |
| 22:18 |
VanessaE |
leveldb should be enough to bump the version to 0.4.5. |
| 22:18 |
sapier |
I know none of that bugs is easy to solve and won't give lots of merits too ... but they are really anoying |
| 22:18 |
hmmmm |
the emerge bug thing.... i'll take a better look at that once my current work is done |
| 22:18 |
hmmmm |
collision handling is totally not my area |
| 22:18 |
hmmmm |
i have no idea what's going on there or even what source file it resides in |
| 22:20 |
sapier |
for this special one I've already written a changeset. But it needs discussion if this way of fixing is accepted or not |
| 22:21 |
VanessaE |
hmmmm: regarding emergethread, one thing I've noticed is that the more map content there is (even if it's nothing but a bunch of idle blocks), the sooner the emergethread bug manifests. |
| 22:21 |
VanessaE |
is that of any use? |
| 22:21 |
VanessaE |
at least it *seems* to be that way |
| 22:21 |
hmmmm |
we already know what the problem is exactly, just not a good way to solve it |
| 22:21 |
hmmmm |
i have an idea that i'd like to try out, but after i get my other stuff in |
| 22:21 |
sapier |
I had a thought about this ... did I understand correct that the real problem is some client not receiving data fast enough? |
| 22:22 |
hmmmm |
no |
| 22:22 |
hmmmm |
the real problem is that the emerge queue fills up too much and it keeps getting enumerated every tick in the serverthread by getnextblocks |
| 22:23 |
hmmmm |
sendblocks holds an envlock which the emerge thread waits on |
| 22:23 |
sapier |
why? |
| 22:23 |
hmmmm |
so when the emergethread finally gets a chance to run it only clears a couple hundred blocks from the queue and more keep getting added on |
| 22:24 |
sapier |
yes but why is sendblocks holding that lock? |
| 22:25 |
hmmmm |
because it reads from the environment and it needs to be in a consistent state |
| 22:25 |
sapier |
so the real problem is environment being a non shareable resource |
| 22:26 |
hmmmm |
i would have to take a better look but i feel like SendBlocks has too broad of an envlock |
| 22:26 |
hmmmm |
really envlock is too big |
| 22:27 |
hmmmm |
it would be really great if we sit down someday and actually take a look at what resources are being used at what times and make some smaller granularity locks for things within the environment |
| 22:27 |
sapier |
I agree |
| 22:28 |
sapier |
I haven't had a look if this idea might work but what about block range locking of env? |
| 22:28 |
VanessaE |
[01-16 17:28] <Menche> could they fix the bug where you have to jiggle the mouse to open a formspec after just closing one? |
| 22:28 |
VanessaE |
for the logs ^^^ |
| 22:28 |
sapier |
jiggle? |
| 22:28 |
hmmmm |
sapier, that too |
| 22:28 |
sapier |
what exactly happens there? |
| 22:29 |
VanessaE |
sapier: if you open a chest and then close it, you can't hit any other key to do anything, even move. you have to jiggle the mouse a little to reset keyboard focus back onto the world. |
| 22:29 |
VanessaE |
or something like that. |
| 22:30 |
sapier |
might be combinable with on_close callback |
| 22:30 |
VanessaE |
(I did not want to steer the conversation this direction, that was supposed to just be for the logs) |
| 22:31 |
sapier |
I think as there's a easy workaround for this bug it's not quite high priority |
| 22:31 |
hmmmm |
in the meantime i think it might be a good idea for my code to actually ship, the emergethread getting done with its stuff quicker would be a way to alleviate the problem (but not actually fix the consumer starvation) |
| 22:31 |
sapier |
related to emerge thread is a problem in on_generated callback |
| 22:31 |
hmmmm |
oh god.... |
| 22:31 |
hmmmm |
let's hear it |
| 22:32 |
sapier |
some ppl use on_generated to heavyly modify world |
| 22:32 |
VanessaE |
oh boy. |
| 22:32 |
sapier |
but behaviour of setnode is not intended to do that at all |
| 22:32 |
VanessaE |
jeez, speaking of emergethread, my server just went down hard :-/ |
| 22:33 |
hmmmm |
hrmm |
| 22:33 |
hmmmm |
there are also other remedies for fixing that |
| 22:33 |
sapier |
any node modified by setnode will touch at least 6 other nodes queuing them to transform queue in worst case and doing light calculations |
| 22:33 |
hmmmm |
like make the queue count for each peer an O(1) lookup so it spends the same amount of time in sendblocks |
| 22:34 |
hmmmm |
and also my work that i'm planning to do with the emergethread would fix that as well, that is, do a trylock when blitting the new blocks back to the environment |
| 22:34 |
hmmmm |
instead of waiting |
| 22:34 |
sapier |
that's not a solution hmmmm it will improve situation but won't fix it |
| 22:34 |
hmmmm |
what, the trylock? |
| 22:35 |
hmmmm |
all of these are things that should have been done in the first place |
| 22:35 |
sapier |
not the separate queuing for each client |
| 22:35 |
sapier |
still it should be done for other reasons |
| 22:40 |
sapier |
puhhh ... I've just had a look at sendblocks function ... where I work we do call functions like that "infinite loop" |
| 22:42 |
|
Gambit joined #minetest-dev |
| 22:43 |
sapier |
those loops don't have a worst case limit |
| 22:43 |
VanessaE |
sapier: do you think you can fix it? |
| 22:44 |
sapier |
you're kidding ;-) to fix bugs like that you need to reconsider what you're doing and what you really need to be done |
| 22:45 |
VanessaE |
ohheh |
| 22:45 |
sapier |
I think hmmmm has already some interesting ideas |
| 22:45 |
VanessaE |
agreed, for as much as I can understand it anyway. |
| 22:46 |
sapier |
hmmmm I still don't understand why envlock is taken for whole sendblocks function |
| 22:47 |
sapier |
is consistency between different clients really required? |
| 22:49 |
|
sema4 joined #minetest-dev |
| 23:09 |
sapier |
VanessaE could you have a look if this change reduces emerge thread problems? https://github.com/sapier/minetest/commit/81098c8e112c24bd503473a4b5b5eb26b388d8f2 |
| 23:09 |
VanessaE |
sure, lemme pull that in real quick |
| 23:10 |
sapier |
I don't think it'll fix it completely but as far as I understand that code it most likely won't harm |
| 23:11 |
VanessaE |
building it now... |
| 23:11 |
sapier |
still real problem is the way block updates are done |
| 23:12 |
VanessaE |
ok, it's installed. |
| 23:12 |
sapier |
and we need some testcase to decide if a change improves or makes things worse ... as I don't think there's a single fix to make it instantly run |
| 23:12 |
init |
VanessaE: how can you "reproduce" the bug? |
| 23:13 |
VanessaE |
init: just have a really detailed, heavily-loaded map |
| 23:13 |
VanessaE |
e.g. tons of buildings, or lots of conifers, jungle trees, etc. |
| 23:13 |
VanessaE |
fly around for a while |
| 23:13 |
sapier |
:-) not qute a testcase |
| 23:13 |
init |
my netbook can't handle that, so i can't try :P |
| 23:13 |
sapier |
you can't even decide "fixed" with it |
| 23:14 |
VanessaE |
sapier: I don't see any improvement... :( |
| 23:15 |
sapier |
I thought so ... env lock is huge spliting it wasn't quite promising ... 2big locks are still to big |
| 23:16 |
VanessaE |
it was worth a try |
| 23:17 |
sapier |
so this bug obviously needs mor investigation maybe whole mechanism needs to be redesigned ... but maybe some small adjustments are enough ... don't quite know by now |
| 23:26 |
sapier |
VanessaE do you run linux? |
| 23:26 |
VanessaE |
sapier: yep |
| 23:28 |
sapier |
hmm ... I have to check something maybe there is a quick solution for this problem |
| 23:29 |
VanessaE |
oh? |
| 23:30 |
sapier |
linux uses fifo as default scheduler ... this results in not changing from a busy thread to a waiting thread without being forced |
| 23:30 |
sapier |
this isn't as critical in multicore environments ... how many cores do you have? |
| 23:33 |
VanessaE |
6 cores. |
| 23:33 |
VanessaE |
and I think it uses a different kind of scheduler these days, "Completely Fair" they call it. |
| 23:33 |
sapier |
hmmm ... ok that doesn't comply with my guess |
| 23:33 |
sapier |
process scheduling but not pthread thread scheduling |
| 23:34 |
VanessaE |
oh right |
| 23:48 |
sapier |
https://github.com/sapier/minetest/commit/3861bf3d86689d85cf3b08324dfea98b160326fc |
| 23:48 |
sapier |
VanessaE probably adding this improves behaviour ... but switching from fifo to rr may show bugs not beeing visible before |
| 23:50 |
sapier |
I remember you had a link to that world triggering the problem yesterday? can you give it to me so I can test myself? |
| 23:50 |
VanessaE |
sure, world: http://digitalaudioconcepts.com/vanessa/hobbies/minetest/My_World_server.tar.bz2 game: http://digitalaudioconcepts.com/vanessa/hobbies/minetest/vanessa_game.tar.bz2 |
| 23:51 |
VanessaE |
(you'll need the game or the world will be like 50% unknown blocks ;-) |
| 23:52 |
sapier |
127MB? .... I think that'll take some time |
| 23:53 |
sapier |
that commit changes two things, SCHED_RR for all threads and lift emerge thread above all other threads |
| 23:54 |
VanessaE |
127MB is nothin' compared to redcrab's server. His world file is in the 5GB range now :-) |
| 23:55 |
sapier |
:-) |
| 23:55 |
VanessaE |
also, the map: http://digitalaudioconcepts.com/vanessa/hobbies/minetest/images/my_world_map.png |
| 23:55 |
VanessaE |
so you can see where the dense areas are. |
| 23:56 |
sapier |
do you have some coordinates? :-) |
| 23:56 |
VanessaE |
not really, just fly around and eventually it'll barf :-) |
| 23:58 |
VanessaE |
the spawn is somewhere around 130,25,100 or so |
| 23:58 |
sapier |
is this bug occuring singleplayer too? |
| 23:58 |
VanessaE |
-138,24,100 to be exact. |
| 23:59 |
VanessaE |
the only thing I've seen singleplayer is the map sometimes loads slowly, but I never play in that mode anymore |
| 23:59 |
VanessaE |
I just use it for the occasional test of a mod |
