Time Nick Message 02:09 Mallot1 hey, where should i learn Lua? 03:07 dzho <.< 03:10 Mallot1 what/ 03:10 Mallot1 ? 03:20 hmmmm muwahaha 03:20 hmmmm mapgen v7 is now playable 03:20 hmmmm though i wouldn't recommend using it. 03:28 hmmmm http://oi41.tinypic.com/2nthcvk.jpg 03:30 ShadowNinja What is the name of a 80x80 mapgen chunk? Chunk? 03:30 ShadowNinja hmmmm: ^ 03:31 hmmmm 80x80x80 you mean... it's just called "default-sized chunk" 03:32 ShadowNinja Er, yes. 04:08 kahrl hmmmm: awesome! 04:08 hmmmm yeah 04:09 hmmmm just writing up the lua api docs right now 04:09 hmmmm i'm going to commit soon 04:30 hmmmm https://github.com/kwolekr/minetest/commit/6d1c9d61d74336a479750c9a9746d1a33148e1ab 04:44 kahrl https://github.com/kwolekr/minetest/commit/6d1c9d61d74336a479750c9a9746d1a33148e1ab#L1R675 04:44 kahrl ^ shouldn't there be another "For each node in schematic:" below this line? 04:44 hmmmm nope 04:44 hmmmm i pair the params together. they're both almost always 0 and compress well that way 04:45 kahrl I thought serializeBulk always separated them 04:45 hmmmm oh shoot 04:46 hmmmm you're right, i forgot to change that when i decided to use serializeBulk instead of my own 04:59 kahrl couldn't SWAP be replaced with std::swap? 04:59 hmmmm ah std::swap 04:59 hmmmm i'm glad i didn't use that, i recall std::swap giving problems with MS VC++ 04:59 kahrl haha 05:00 kahrl why does anyone call that a C++ compiler :P 05:00 hmmmm you know what i'm talking about, right? can't remember exactly what it was, but it broke compilation if they didn't add something extra 05:00 hmmmm something about namespaces 05:01 hmmmm ah here we are http://connect.microsoft.com/VisualStudio/feedback/details/499152/valid-c-program-using-std-swap-fails-to-compile-in-vc10 05:01 hmmmm err it was http://connect.microsoft.com/VisualStudio/feedback/details/387055/using-declaration-using-std-swap-should-not-disable-argument-dependent-lookup 05:02 hmmmm actually it seems like there are a whole bunch of different problems with std::swap with MSVC++ 05:03 kahrl the reason I'm asking is because the way SWAP is written it might lead to weird things 05:03 hmmmm hmm? like what 05:03 kahrl for example, you can't add an else in sortBoxVerticies 05:03 kahrl at least I'm pretty sure you can't 05:04 hmmmm don't get what you mean... i can change it to do {} while (0) if you'd like 05:04 kahrl yeah that would be a solution 05:04 hmmmm although i technically shouldn't need to do that 05:04 hmmmm i don't think other macros do either 05:05 hmmmm ahh 05:05 hmmmm i see what you mean haha 05:06 kahrl oh yeah CHECKKEY and QUICKTUNE have the same problem 05:07 kahrl and there are some others 05:07 hmmmm anyway i fixed that here 05:11 hmmmm fixed two things in lua docs. looks good now 05:13 hmmmm now i need to add rotations :/ 05:14 kahrl when placing, Would it be a good idea to skip the CONTENT_AIR/CONTENT_IGNORE check when param1 is != 0? 05:14 hmmmm the L-system decoration type is still sort of up in the air. i'm not entirely sure what to do, because the system currently in place is really only for trees. iirc jin_xi has a more complete, generalized version, but it requires lua callbacks to carry out the rules. 05:15 kahrl so that if someone really wants to place air in a schematic, just set param1 to 255 05:15 hmmmm maybe 05:15 hmmmm keep in mind they can forcibly place air in a schematic if they manually place it 05:16 hmmmm i'm not sure what sort of use case setting air can have as a decoration though.. 05:16 hmmmm we'll see how much they want to be placing air 05:16 kahrl oh wait param1 = 255 would still be less than 100% 05:17 kahrl hmmmm, placing rooms underground 05:17 hmmmm how would that work? these are placed on ground level 05:18 kahrl not with decorations I guess... but place_schematic 05:18 hmmmm again, with place_schematic, it doesn't check for air/content ignore 05:18 kahrl I'd like to close the "set multiple nodes at once" feature request when place_schematic is merged 05:18 kahrl oh, I thought they were the same 05:19 hmmmm nope... see placeSchematic vs. generate 05:19 kahrl kind of confusing but it works 05:28 hmmmm can i make a pull request on github without forking the repository? 05:28 hmmmm i think you do something with the patches 05:28 kahrl pretty sure a pull request requires a branch 05:30 kahrl the branch could be in the upstream repository so technically a fork isn't needed 06:37 hmmmm https://images.4chan.org/g/src/1371882632988.jpg 06:37 hmmmm crazy - wonder if that was generated, probably not though 06:38 hmmmm in noiseparams, the spread factor is always the same for the three components but it doesn't need to be like this 06:39 hmmmm increase the y component of np_ridge in mgv7 and you should be able to get nice vertical gloops in terrain, sort of like that 06:39 hmmmm certainly not anything that cool 11:25 PilzAdam hmmm, you have reverted the changes of two commits in lua-api.txt 11:26 PilzAdam +m 12:35 Nekofire Hello anyone active in here at the moment 12:41 * sfan5 14:50 proller try#6 everyone ready for ipv6? 16:59 Nekofire Hello 17:02 sapier https://github.com/sapier/minetest/tree/async_lua_functions hmmmm what do you think about this proof of concept for asynchronous lua function calls? 17:03 * hmmmm blinks 17:03 hmmmm wait you're making a new thread for this? 17:04 sapier it's only 202 line change so quite small for me 17:04 hmmmm don't half of the api that don't need locks return a result? 17:04 hmmmm doesn't half* 17:05 hmmmm also you and your friend claasses 17:05 hmmmm nobody here uses that :/ 17:05 sapier but it works and encapsulation is done quite well 17:06 sapier I don't need to publish full api up to main.cpp 17:06 sapier but in this special case adding a job dispatcher may even result in removing it 17:06 hmmmm alright i only read through half of the source 17:06 hmmmm but it seems to be sort of like this would only work for new api 17:07 hmmmm get job results? 17:07 sapier what do you mean with new api? 17:07 sapier if you issue a job request you get back an id 17:07 hmmmm how is the lua caller supposed to get the result 17:07 sapier once the job is finished a callback is called passing id and result 17:07 hmmmm through that api? 17:08 hmmmm see? exactly, like i said, this would only work for new api 17:08 hmmmm unless you want to make a new set of asynchronous versions 17:08 sapier yes it's a new feature asynchronous tasks 17:08 hmmmm which you'd have to basically force people to adopt 17:08 sapier it was never meant to replace current api but just provide a interface for long run functions 17:09 hmmmm because why on earth would somebody use local jobid = minetest.foo_async(blahblahblah) ... minetest.get_job_results(jobid) -- can this block? 17:09 sapier no it can't 17:09 hmmmm what happens if it's not ready... 17:09 sapier if what isn't ready? if there's another job running the new one is queued 17:10 hmmmm no i mean what if the lua caller tries to get the result of a job that hasn't completed yet 17:10 sapier there's no way to activly fetch a result by now 17:11 hmmmm maybe i'm misreading something 17:11 sapier waiting for a asynchronous result is somehow useless you wouldn't have to do it asynchrounous if you're waiting for it 17:11 hmmmm what is the point of get_job_result then 17:11 hmmmm who calls it 17:12 sapier it's internal job/result queuing 17:12 sapier in environmental on_step there's checked if results are ready 17:12 hmmmm ah 17:12 sapier if this is case the result callbacks are called 17:12 hmmmm are you sure that's a good idea 17:13 hmmmm because now every asynchronous call's results have a maximum latency of an entire server step 17:13 hmmmm that could be 100ms for all you know 17:13 hmmmm which it is by default 17:13 hmmmm see what i would've done here is have the caller register a callback function 17:13 sapier it's meant for long run tasks so imho 100ms isn't a problem 17:14 hmmmm i sort of wonder what kind of long run tasks we're talking about... 17:14 sapier someone needs to call the callback 17:14 sapier I'm not sure if it'd be a good idea to call it from another thread ... but yes that'd be a possibility 17:14 hmmmm i would do it from the async task thraed 17:15 hmmmm er wait 17:15 hmmmm no that's bad, lua can only have one thing running at a time 17:15 hmmmm wow, lua is really dumb 17:15 hmmmm who came up with this crap 17:15 hmmmm alright your way is good then 17:15 sapier yes ... i realized this when trying to get luathreads working ;-) 17:17 hmmmm what's with async_yield 17:17 PilzAdam hmmmm, have you read the logs yet? 17:17 hmmmm 1 ms?? isn't that kind of arbitrary 17:17 hmmmm what's wrong with pthread_yield()? 17:17 sapier some quick solution to stop the worker thread from consuming full cpu power 17:18 Mallot1 we should add a computer into the game 17:18 Mallot1 a programmable computer 17:18 sapier hmmm nothing but phtread_yield isn't available on windows ;-) 17:18 Mallot1 like computercraft :D that you can program with Lua and you can program the mesecons and stuff :D 17:18 hmmmm sapier, you can do Sleep(0) on windows 17:19 PilzAdam Mallot1, mesecons already has a Luacontroller 17:19 Mallot1 sapier i edited your mod pack 17:19 hmmmm hmmm, you have reverted the changes of two commits in lua-api.txt 17:19 sapier good idea I'll fix it 17:19 Mallot1 they do? 17:19 hmmmm saw that 17:19 sapier thx Mallot1 I already pushed your fix 17:19 hmmmm i was going to fix it back along with some other things in a separate fix random crap commit 17:20 sapier the big question about the async feature is "does it really fix problems?" ;-) 17:20 hmmmm i suppose what happened was that i rebased on current master, and i had lua_api.txt open in a plain text editor, which didn't reload the file when it was modified by an external source (like git) 17:20 hmmmm sapier, it doesn't, it might make something more comfortable in the future however 17:21 hmmmm if we do have something that could use that 17:21 PilzAdam hmmmm, then get a proper text editor :-p 17:21 hmmmm asynchronous calls would probably too hard for most lua coders to understand, i fear 17:21 sapier I guess there won't ever be access to a lot of things from a async call 17:21 hmmmm for some reason, async baffles novices 17:21 sapier basicaly almost every data needs to be passed by argument or available per minetest api 17:22 D-Man :/ 17:24 sapier but it could be used for jejias curl feature as well as worldedit(once propper map locking is established) 17:24 sapier maybe even for custom on_generated 17:24 hmmmm i wonder if worldedit would ever need this 17:25 hmmmm the only things that'd take a lot of time *in the core* would be blitting to and from the map 17:25 hmmmm not only is that a lot faster than you're giving it credit for, but it also requires that envlock 17:25 hmmmm which makes it useless 17:27 hmmmm anyway now i have to take a break from mapgen v7 and do the mapvoxelmanipulator really fast, or else people will get the wrong idea and start abusing schematics 17:27 PilzAdam people already do that... 17:28 Mallot1 humminah humminah 17:28 sapier pathfinder lua version had a mode wher it first fetched data then calculated path ... using a similar design would gain benefit ... but I'm not sure if there are lots of task that can be done this way 17:28 Mallot1 print("Hello" .. "World"); 17:29 hmmmm Mallot1, probably a good idea to keep it in #minetest 17:29 Mallot1 lol 17:29 Mallot1 ooops wrong chat i have both open 17:29 Mallot1 *mallot1 is noobin 17:29 Mallot1 bye 17:31 hmmmm hm, pilzadam, http://forum.minetest.net/viewtopic.php?id=6225 sounds like many of his data files are being corrupted by some sort of malware or virus 17:31 hmmmm like one of those things where it'd infect .exes by appending a hook into the entry point 17:34 hmmmm or what if he has bad ram 17:36 Exio if he had bad ram windows would throw a BSOD, i guess? 17:37 hmmmm would it really? the boot process is quite predictable because it's the first thing that happens 17:37 hmmmm it's totally possible for kernel mode things to be unaffected simply because they're never in the physical address range where the corruption occurs 17:37 sapier exio no badram may result in random application crashes only 17:38 Exio the last time i had a bad ram + windows, it randomly crashed and threw a BSOD 17:38 hmmmm he might want to try out my BadRAM for Windows(tm) 17:38 hmmmm :D 17:38 hmmmm just kidding, i wouldn't ever recommend people to use that 17:38 Exio haha 17:38 sapier I'm on a pc with some bad ram took me half a year to realize source of random crashes was a memory error ... I appreciate linux bad ram avoidance feature :-) 17:39 Exio linux crashes less with that 18:05 Nekofire is any body active in themod chats 18:11 kaeza Nekofire, if it's not engine related, it's best to post it in #minetest 18:53 kahrl one of my machines developed bad ram recently, it first manifested in grsec detecting a stack smashing attack in python 18:54 kahrl then random application crashes and system freezes 18:54 sapier the more ram you have the less the problems you experience 18:55 kahrl it appears it wasn't really the ram that was faulty but something on the bus or chipset 18:55 kahrl because it happened with either of the two ram sticks 18:55 sapier wow a very rare error 18:58 sapier oops :-) I just realized I've forgotten to add the most important files to the async branch :-) 19:31 kaeza sapier, would that bring any performance benefits when using it instead of e.g. Lua coroutines? 19:32 sapier yes coroutines do not run parallel while async calls run i own thread 19:32 kaeza ah ok 19:32 sapier ok "yes" is only true for multicore processors 19:32 sapier but it's bought by quite limited data exchange 19:33 kaeza does that create a new lua_State? IIRC Lua is not friends with multithreading 19:34 sapier so if you have really long run tasks , e.g. downloads, complex calculations, map modifications this might help 19:34 sapier exactly the new lua_State is cause for limited data exchange 19:34 kaeza a gotcha 19:34 kaeza ah* 19:35 kahrl map modifications? how are locks handled? (might have missed it) 19:35 PilzAdam sapier, see #minetest, people dont like falling nodes colliding with objects 19:36 sapier I have said what I wanted to say to this It's your turn to decide what you do PilzAdam 19:37 sapier kahrl that's a todo for this changes it's only prototype atm ... but map modifications is one of main use cases 19:37 celeron55 i say don't add asynchronous modification of maps 19:38 celeron55 i don't know the details of what you have though, is there a description somewhere? 19:38 sapier no it's very early prototyping 19:38 hmmmm just read the channel logs from this morning, it'll explain everything you need to know 19:39 celeron55 ---- description follows (sapier writes it now) ---- 19:39 sapier basicaly (one or multiple) separate lua thread running in parallel 19:39 sapier celeron55 I've never seen description beeing implemented ;-) 19:39 celeron55 what can the threads access and by what means? 19:40 sapier any data not beeing part of main luaengine basicaly minetest api only 19:40 sapier or data explicitly passed to it 19:40 celeron55 any of minetest api? 19:41 celeron55 so how does it handle two threads reading and modifying the same nodes at the same time? 19:41 sapier not exactly there are minetest api calls using lua stack too those wont work too 19:42 celeron55 that isn't valid english 19:42 sapier some minetest api calls access lua stack from within c code 19:42 sapier e.g. auth functions 19:43 sapier accessing those would result in readding impicit synchronisation to main thread 19:43 hmmmm [03:41 PM] so how does it handle two threads reading and modifying the same nodes at the same time? <-- the idea of these async calls is to carry out complex computations that don't involve env/conlock 19:44 celeron55 so they have a limited api? 19:44 sapier yes extremely limited 19:44 celeron55 i mean, VERY limited 19:44 hmmmm the async call framework is only for new api 19:44 celeron55 where's it listed and who is going to make sure nobody starts messing with it? 19:45 sapier it's not even decided if this is a suitable way celeron55 it's just for discussion atm 19:46 hmmmm sapier, did you profile the API with the three locking schemes, like it is now/unlock when not needed/lock only when needed ? 19:46 celeron55 what problem does it solve? was this answered already? 19:46 sapier atm it only solves curl api 19:47 sapier without further changes no other problems are fixed 19:48 sapier hmmmm it's difficult to profile as I don't have a suiteable testcase 19:48 hmmmm typical mods 19:49 hmmmm grab the top 10 most popular mods and try them all with each 19:49 kahrl locking only on each api call that needs it is not enough 19:49 celeron55 do you guys know you could just use lua coroutines and add api functions that will automatically yield() when they are waiting for i/o? 19:49 sapier no it doesn celeron 19:49 hmmmm ??? kahrl, why? 19:49 kahrl you need to be able to lock groups of calls 19:49 celeron55 i think that is possible, but i'm not exactly sure 19:49 celeron55 sapier: why? 19:49 sapier lua coroutines don't have any concurrency 19:50 celeron55 you don't need actual concurrency 19:50 kahrl sapier: minetest.get_node(pos) --> decision in lua depending on result of that ---> minetest.set_node(pos, newnode) 19:50 celeron55 it just needs to look like that from lua side 19:50 sapier it's manual preemption you can't even yield on io 19:50 kahrl if it unlocks between get_node and set_node the decision might rely on data that is now invalid 19:50 hmmmm oh, that's true, good point.. 19:50 celeron55 sapier: the coroutines would be put in a table where the engine would resume them periodically 19:51 sapier kahrl that's not a problem 19:51 kahrl sapier: why not? 19:51 hmmmm celeron said he tried that already 19:51 hmmmm (that is, lock only when needed) 19:51 hmmmm didn't think of the case of a get and set node right after 19:51 sapier you can't avoid it either 19:52 sapier celeron55 but what problem would this fix? 19:52 celeron55 sapier: the same thing: you could do asynchronous curl operations in lua 19:52 celeron55 (with code that isn't all spread around) 19:52 sapier celeron55 only if the curl operations are non blocking 19:53 celeron55 the curl C interface has a good non-blocking API if that's needed 19:53 celeron55 it's not a reasoning for anything 19:54 sapier ok so curl is no usecase 19:54 celeron55 hmmmm: what i have said is that one of the early versions of MT had a lock that locked on each node access 19:54 sapier so only longrun lua calculations are left 19:54 hmmmm celeron, yeah, and kahrl said that wouldn't work anyway 19:54 hmmmm you mentioned that you didn't stick with that because of performance reasons 19:55 celeron55 really there's no good reason to even do any coroutine stuff for curl, becasue there could be just some start_curl(callback) thing that would use the asynchronous api in the background 19:55 celeron55 but whatever 19:55 hmmmm how is curl used in lua right now? 19:55 celeron55 hmmmm: mutexes are too slow for that 19:55 sapier hmmmm it's an open pull request 19:55 hmmmm oh. i thought it invoked an external utility. 19:56 sapier I don't know I expected it to use curl lib as it's already linked 19:56 hmmmm lots of people do dumb things which is why i didn't leave out the possibility 19:56 hmmmm and since the asynchronous api isn't being used, that's the only rational explanation i would have why 19:57 hmmmm the only remaining possibility is that the person who added the curl api in the lua api using the lib directly didn't know how or didn't care to use the async functions 19:57 sapier hmmmm sorry didn't get it explanation for what? 19:58 celeron55 hmmmm: minetest has some internal utilities for starting threads that operate curl in a blocking way 19:58 celeron55 it's probably based on that (haven't looked) 19:58 hmmmm that's pretty stupid 19:58 hmmmm ugh 19:58 hmmmm i wish people knew how to properly use the tools they're working with 19:58 celeron55 well it really doesn't matter for what's in currently 19:58 celeron55 but for the lua thing it does 19:59 celeron55 ...or actually not even for that 19:59 Mallot1 was minetest made in voxel3d? 19:59 hmmmm sapier, why the curl api in the pull request isn't asynchronous at present 19:59 celeron55 where is the pull request in the first place? 8) 20:00 kaeza Mallot1, minetest was made with MegaZeux 20:00 sapier It's from jeija and about 6 months old 20:02 sapier ok so async api isn't way to go unless someone wants to add lots of calculations in lua 20:02 hmmmm anyway 20:03 kahrl hmmmm: small bug in the schematic doc in lua_api.txt: in line 1953 the node tables should using name, param1, param2 keys instead of 1,2,3 20:03 hmmmm the main reason why we brought up the entire thing about locking in lua is because of people needing to do heavy computations in lua 20:04 sapier that can be done with async ... but seems to be the only thing it's suitable 20:04 hmmmm oh whoops 20:04 hmmmm i'll fix it up with the random fix patch 20:06 hmmmm anyway i'll be sure to take more care with lua_api.txt when rebasing my branch 20:06 kahrl huh, I placed a 50x1x50 structure with minetest.place_structure and one corner mapblock didn't get updated 20:06 hmmmm didn't get updated, or didn't get written to? 20:07 hmmmm did you try logging out and logging back in 20:07 kahrl didn't get written to (it still isn't there after restarting) 20:07 hmmmm 20:07 hmmmm i guess you were up against a boundary, and i did something wrong with what i had to emerge? 20:08 hmmmm + v3s16 bp2 = getNodeBlockPos(p + size - v3s16(1,1,1)); 20:09 hmmmm if size was 1,1,1, i'd need to emerge an area from p1 = p2 inclusively 20:09 hmmmm so that subtraction is correct 20:10 kahrl can't see anything useful in debug.txt even with debug_log_level = 4 20:10 hmmmm vm->just one block is missing? 20:12 kahrl the block from (48,16,2064) to (63,31,2079) wasn't written to 20:12 hmmmm initialEmerge uses inclusive coordinates too so that's not the problem 20:13 hmmmm and it's the z coordinate 20:13 hmmmm you placed this using placeDecoration, just to make sure, right? 20:13 hmmmm placeStructure rather 20:13 kahrl the structure was placed at 2028,21,48 20:14 kahrl well minetest.place_schematic 20:14 hmmmm that yeah 20:15 kahrl http://paste.dy.fi/anM <-- that's the code 20:16 kahrl 22:04:34: ACTION[ServerThread]: singleplayer uses sandjoy:sandjoy, pointing at [node under=2053,21,73 above=2053,22,73] 20:16 hmmmm stumped 20:17 hmmmm er.. do you get that problem if you remove the - v3s16(1,1,1) from mapgen.cpp:583? 20:17 sapier1 why don't we have a jsemaphore? 20:18 kahrl hmmmm: let me first see if I can reproduce the problem 20:22 sapier1 hmmmm if you want to do numbercrunching the current (and atm final version) supports multiple parallel lua async calls 20:23 kahrl doesn't seem like it wants to happen again (didn't change mapgen.cpp yet) 20:24 hmmmm hmmm odd... if it happens again, i bet what i pointed out was the problem 20:24 hmmmm if anything is, that's what the problem would be 20:24 kahrl I tried the exact same spot though 20:25 hmmmm hahaha, what if the rand() had a hiccup and it just so happened that all those nodes in that spot didn't show up 20:25 hmmmm that would be funny 20:25 kahrl hehe 20:26 kahrl anything but the outer edge is param1=0 though :P 20:26 hmmmm i'm pretty sure that the correct coordinate of the blockpos to emerge is p + size - v3s16(1,1,1) .... it only makes sense. it made sense to me while i was writing it and it makes sense now 20:26 hmmmm but it doesn't hurt to over-emerge either 20:26 hmmmm so i might remove that bit 20:28 kahrl maybe something (mapgen?) was still working on that block and replaced it after the place_schematic was done? 20:28 hmmmm ahh 20:28 kahrl (I think the formula in mapgen.cpp:583 is correct) 20:28 hmmmm it's possible and it would make sense 20:31 kahrl I was able to reproduce it by running into a new area 20:31 kahrl if it's the mapgen it's hard to reproduce because mapgen is so fast ;) 20:31 hmmmm ehehe 20:31 hmmmm that's being called on_generate? 20:32 kahrl this time it didn't happen at the edge: http://imgur.com/6pKCqFM 20:32 kahrl *corner 20:32 kahrl it happened at the edge but not the corner 20:32 hmmmm i'm sure i'll figure it out eventually 20:33 kahrl called in on_use, my updated code: http://paste.dy.fi/aO0 20:39 kahrl oh hold on 20:39 kahrl this time is was actually written but not sent to the client 20:39 kahrl it* 20:44 hmmmm the block update code should be solid 20:57 kahrl is there a way to set some nodes placed with place_schematic to probability zero? 20:59 kahrl since it doesn't skip air and ignore like the version called during mapgen does 21:00 kahrl wait a sec... if (c != CONTENT_AIR && c != CONTENT_IGNORE) continue; <-- does that mean the mapgen version only placed air and ignore? 21:00 kahrl places* 21:01 kahrl or am I just dumb now 21:01 kahrl oh wait, that's not the schematic but the map data 21:02 hmmmm right 21:02 hmmmm don't want to replace things that were already placed 21:02 kahrl makes sense 21:02 hmmmm is there a way to set some nodes placed with place_schematic to probability zero? <---- well, nope, but that's probably a good idea 21:03 hmmmm i'll add that in a later commit 21:03 kahrl perhaps: if (schematic[i] == CONTENT_IGNORE) continue; 21:03 hmmmm mmm =/ how is content ignore going to get into the schematic 21:03 hmmmm it's a better idea to just have a zero probability 21:04 kahrl yeah probably 21:05 hmmmm i guess, what i could do is set aside an additional probabilility value like -1, and that would mean "node is never written to" 21:05 kahrl just change myrand_range(1, 256) to myrand_range(2, 256) 21:05 hmmmm in applyProbabilities i'd set the content to CONTENT_IGNORE 21:06 hmmmm that would avoid reducing the range of possible probabilities, and avoid changing the meaning of "0 probability" 21:06 kahrl how would you set param1 to -1? 21:06 hmmmm i don't ever do that 21:07 hmmmm i check for -1 in applyProbabilities, never actually set param1 to that 21:07 hmmmm and if it's set raw, then i can just have them add in content ignore as the actual content 21:07 kahrl oh you mean when you generate a schematic with minetest.create_schematic 21:07 hmmmm yeah 21:07 kahrl that makes sense 21:07 hmmmm that's probably the best way to do this 21:09 sapier1 is there any chance to get chunk/block granular map locking? 21:09 hmmmm yep, by making the map container sectors are held in atomic 21:09 hmmmm err 21:10 hmmmm misread 21:10 hmmmm let's have a set that contains blocks that are in use 21:10 hmmmm reading/writing to this set needs a lock of course 21:10 sapier1 maybe my sentence wasn't cler enough is map datastructure designe suited for locks at that level? 21:11 hmmmm that would be a really lot of locks if you do it that way 21:11 hmmmm it's just not plausible 21:12 kahrl it could certainly be done but the performance would likely be abysmal 21:12 sapier1 I thought more in direction of metainformation "in_use" with access to that information is locked 21:12 hmmmm so we have our own mapblockref function that checks to see if the block you want to grab is in use; it locks the mapblock set lock, checks to see, unlocks that, if it is in use, then it should wait until there's a mapblock free event somewhere and then check again (??) 21:12 kahrl if it's just a flag then a lockfree algorithm could be used I guess 21:13 hmmmm lol 21:13 hmmmm i checked out how big a bitmap of all possible blocks would be 21:13 hmmmm that's not plausible, we'd need to use a data structure 21:13 sapier1 not node granular of course 21:13 hmmmm it's probably best to roll our own then 21:14 kahrl do unloaded blocks need to be locked? otherwise you could just add a flag or lock in class MapBlock 21:15 hmmmm the flag would need to be atomic 21:15 hmmmm and what happens if it's already in use? it'd have to wait 21:15 hmmmm which is basically..... the definition of a lock 21:15 kahrl yeah 21:16 sapier1 the flag would be just to avoid thousands of locks 21:16 hmmmm and having a lock for every single mapblock like i said, that's probably not going to work. i mean, it seems like an obvious solution, but i have my doubts 21:16 sapier1 another completely different aproach what about transactions? 21:16 kahrl then with that many locks without any strategy to sort them there's a real danger of deadlocks 21:16 hmmmm transactions? 21:17 hmmmm you mean like, just atomic writes to mapblocks for everything? 21:17 sapier1 changes not beeing executed at once but only on commit if someone else changed anything that is required for that transaction it's not going to be executed and has to be done again 21:17 sapier1 in common case no conflicts will occur 21:18 sapier1 of course in conflict case overhead is big 21:18 kahrl guess that's something that needs to be done if RCU for mapblocks is added 21:18 hmmmm yeah, but wouldn't that make it not updated as soon as it's modified? 21:18 hmmmm right, RCU is basically the next step 21:18 hmmmm i considered all these things a while back and that's basically what i decided 21:19 sapier1 yes but rcu wouldn't fix changes depending on some map state 21:19 sapier1 e.g. if mapnode == ? then set some other node 21:19 kahrl how would you implement transactions without rcu? 21:20 sapier1 with transactions you could call start transaction mark all nodes you checked as relevant and if anythin is wrong restart 21:20 sapier1 capture set_node calls storing in transaction data 21:20 sapier1 add a counter to map that is updated at each real modification to speedup changes, if counter hasn't changed since start_transaction no check is required at all 21:22 sapier1 but I don't know map structure well ... so my thoughts might be completely wrong 21:22 kahrl the counter will become useless once you have installed enough mods that do async map updates in transactions 21:23 kahrl if lets say something in on_generate is done that way 21:23 sapier1 why? mods don't run in paralell regularly 21:23 kahrl they don't run in parallel at all now :P 21:23 sapier1 they never will 21:24 sapier1 lua is just not capable of running parallel tasks 21:24 kahrl hmm? I thought that was the entire purpose of this along with the async calls 21:24 sapier1 it won't runn the whole mod in parallel 21:25 sapier1 and yes with async it can happen without it the counter isn't even usefull at all 21:25 sapier1 I thought async api is burried due to lack of real usecases? 21:25 sapier1 at least for the moment 21:28 hmmmm https://github.com/minetest/minetest/commit/4d77781ce7cba571701e731b1f442af691933720 21:31 hmmmm not sure how RCU wouldn't cover that problem 21:31 hmmmm it's used almost exactly like a regular lock 21:32 hmmmm once you 'lock' the block, you'll be reading from the version of the mapblock you locked it the whole while until you unlock it 21:32 sapier what would be benefits of rcu in that case? 21:32 hmmmm it's after you 'unlock' it, that it'll delete the version of the mapblock you were using 21:33 hmmmm multiple readers and writers doing things at the same time without screwing stuff up 21:33 sapier ok so on lock a copy is created? 21:33 hmmmm well 21:33 sapier more precise on second lock 21:34 hmmmm right 21:34 sapier third ... 21:34 sapier but that'd require transactions per definition 21:34 sapier e.g. if second unlocks changes need to be commited 21:35 hmmmm yeah, the whole thing each reader and writer gets their own version to screw around with, and a transaction is done where all the changes are comitted once that lock is given up 21:35 hmmmm yeah 21:35 sapier so to sumarize 21:36 sapier start_transaction does only create an administration structure 21:36 hmmmm iirc, to preserve the order, it'd save the second copy for changes until the first lock is done 21:36 hmmmm commits the first set of changes, then commits the second changes 21:36 sapier on first access from within a transaction (read or write) the block is locked 21:36 sapier on transaction_commit the changes are saved 21:37 sapier if someone commits while other transactions are running changes need to be merged ore commit fails 21:37 PilzAdam anyone objects: https://github.com/PilzAdam/minetest/commit/b23d20e8ba799c60a132a7a40908fecc596f74b9 ? 21:37 hmmmm pilzadam, seems good 21:37 sapier pilzadam me too 21:37 hmmmm sapier, yeah, that commit is saved until older locks are finished 21:38 sapier within a transaction any read or write of a block marks this block as relevant for merge 21:38 proller why player ip not in logs on connect? 21:38 sapier no hmmmm that's wrong if you do it that way the changes of this commit may be based on invalid mapdata 21:38 PilzAdam proller, the IP is in the logs AFAIK 21:39 PilzAdam infostream IIRC 21:39 hmmmm how so? everybody would get their own version of how the mapblock looked *at that time* 21:40 proller PilzAdam, 00:32:36: ACTION[ServerThread]: Guest4456 joins game. List of players: Guest4456 21:40 hmmmm that's infostream 21:40 sapier hmmmm yes and that's exactly the problem 21:40 PilzAdam proller, look at the infostream output arround this 21:40 hmmmm that's not infostream* 21:40 sapier if e.g. second one checked node(1,1,1) for water 21:40 PilzAdam proller, you need debug_log_level >= 3 for this 21:40 hmmmm oh that's true 21:40 proller 8( 21:40 sapier to do something to block 1,1,2 21:40 hmmmm right right, i get it 21:41 hmmmm rcu must do something different from what i thought 21:41 proller PilzAdam, i think it must be in like: Guest4456 [ip.here] joins game. 21:41 PilzAdam proller, yea, could be useful 21:42 proller enabling verbose log could be late 21:43 sapier hmmmm the transactions would fix it but of course anyone using it has to expect a transaction to fail and handle that case 21:43 PilzAdam proller, I agree on changing it 21:43 hmmmm right 21:44 hmmmm but there aren't too many people using this 21:44 hmmmm only the primitive mapblock access functions in map.cpp 21:44 sapier true 21:45 PilzAdam it was announced that common mod support will be dropped after 0.4.7 -> https://github.com/PilzAdam/minetest/commit/c4d22d6e666c89658434b9636519560e956473e8 21:46 ShadowNinja proller: With my function you can write mod to do that. 21:47 proller ShadowNinja, its must be in core by default 21:47 PilzAdam proller, minetest.get_player_ip(name) is in 0.4.7 21:49 ShadowNinja IMO player names should be case-insensitive. But case should be preserved, like IRC. 21:51 proller +1 21:51 proller and maybe more agressive preserve like 0 -> O 1->l , .... 22:04 PilzAdam any comments on the common drop? 23:20 Mallot1 what common drop? 23:20 Mallot1 i was sleeping 23:21 kahrl PilzAdam: I thought GetDirListing already excluded files starting with ".", but apparently that is only the case on POSIX 23:22 PilzAdam kahrl, a windows user requested this 23:22 PilzAdam there is a similiar workaround for something else (mods?) 23:22 kahrl imho GetDirListing should behave the same everywhere 23:22 kahrl I might have made a bad assumption somewhere in the path code I wrote that sapier included in next_gen_main_menu 23:23 kahrl because of that 23:23 PilzAdam mods.cpp:103 23:23 VanessaE I asked if anyone plans to fix the 4096-node limig. Pilzadam suggested: VanessaE, src/mapnoge.h:38 increase MAX_CONTENT and see what happens 23:24 VanessaE I changed it to fffd and deployed to my server. 23:24 VanessaE it works fine. 23:24 PilzAdam *mapnode.h 23:24 VanessaE er yeah. 23:24 VanessaE anyway, if someone else could test that or comment, and push that one-character change to master, I'd appreciate it, as old clients can't connect to a server that's been so-modified 23:25 VanessaE (push, unless it presents a problem) 23:25 PilzAdam *only if it has more items registered than the former limit 23:25 VanessaE right. 23:32 Mallot1 i think the game needs asoundtrack 23:42 hmmmm woah woah whoa 23:42 hmmmm you're changing MAX_CONTENT? 23:42 hmmmm hold on a minute there 23:42 hmmmm first of all in order to do that we'd need to change it from a statically sized array to a vector 23:42 hmmmm and then you also need to fix node id mapping serialization 23:43 hmmmm it's not a one liner fix