Time Nick Message 00:10 Zeno` port plant_lib? 00:10 VanessaE I was kidding :P 00:10 * Zeno` wipes the sweat off his brow 03:05 Zeno` Zeno`> I'm wondering if some functions could be added to l_vanimp.cpp. In particular, something like l_get_u16data() or l_get_rawdata() which make a copy of vm->m_data and pass it to Lua as a userdata type. Also the equivalent set function. It'd be good to have these for the noise functions as well. The main reason for this is that I'm writing critical code in C and most of the time is spent traversing the Lua tabl 03:05 Zeno` <Zeno`> e for both noise and the vmdata. The additional functions would, when using C, increase performance dramatically because memcpy() could be used rather than a loop 03:05 Zeno` <Zeno`> each of the userdata types would be **u16 03:07 Zeno` A further speed increase would be gained by Lua mods that only use these "raw" data types rather than the Lua tables 03:07 Zeno` mainly because set_data() and get_data() won't need to iterate 03:08 Zeno` This, by the way, is my "compromise" where I don't modify the minetest source code but instead write C modules for Lua, keeping everyone happy 03:10 hmmmm Zeno`, you don't need to iterate with the table returned by get_data()... 03:10 hmmmm I have no clue where you got that idea from 03:10 hmmmm it's a flattened 2d array. 03:11 Zeno` It's a Lua table isn't it? 03:11 hmmmm yes 03:12 Zeno` so when I pass control to my C module I have to manipulate the data in the Lua table rather than simply dereferencing the data. Unless I'm mistaken 03:13 Zeno` I can't do *(data + blah) = c_id; // or can I? 03:14 hmmmm we're talking about Lua 03:14 Zeno` I'm talking about writing the bulk of the mod in Lua, but having a C module do some heavy lifting 03:14 Zeno` i.e. I am calling *my* C function from Lua 03:15 hmmmm that's really not a good idea, Lua especially with LuaJIT is pretty fast 03:15 Zeno` It's very fast, yes. But this way is ~10x faster 03:15 Zeno` why isn't it a good idea? 03:16 hmmmm because it defeats the entire purpose of the scripting and it introduces cross-platform incompatibility 03:16 Zeno` only in the mod 03:16 hmmmm not to mention a security risk and it also requires more to be done by the user 03:16 hmmmm I don't know what sort of mechanism you have for marshalling data between the lua script and your C program 03:16 Zeno` if a mod creator decides they don't mind about cross-platform incompatibility either they write portable C or live with it. Surely it's their choice 03:17 hmmmm but, yeah, it seems like you're going to have to do the lua table -> plain array of u16 conversion back again 03:17 hmmmm yes, but now you're limiting the audience of minetest 03:17 hmmmm let's say your mod is really good and people love it 03:17 Zeno` nope. Only limiting the audience of my mod 03:17 hmmmm but you only write your module for windows 03:17 hmmmm peoples' games are going to depend on your mod and therefore depend on windows 03:18 Zeno` I understand that, but if I did that it'd be a crappy mod 03:18 hmmmm then you're going to have a huge division of players 03:18 hmmmm there's going to be players who run windows and play a game with your mod, and then the rest of them 03:18 hmmmm i'm saying that it's a bad idea for those reasons. it's certainly your own choice though. 03:19 Zeno` I am marshelling data between Lua and C simply by passing the data as arguments (referring to your earlier query) 03:19 hmmmm so you have a Lua VM running in your program as well? 03:20 Zeno` no, you don't need that... the Lua stack is passed to my "program" (it's a shared lib like any Lua extension) 03:20 hmmmm and I guess you have a bunch of C API exposed that correspond to the Lua callbacks? 03:20 Zeno` no 03:20 hmmmm I don't quite understand how you're doing this 03:20 hmmmm how do you mean "passing data as arguments" 03:20 Zeno` hold on... I'll paste some 03:21 Zeno` http://codepad.org/LyB0kE5P 03:22 Zeno` so the only thing "exposed" is luaopen_cavemapgenc() and l_gencaverealm() via registration of the function with Lua 03:22 hmmmm yes, you ARE running a lua vm there 03:22 hmmmm you clearly are doing exactly what I asked if you were doing 03:23 Zeno` I don't see the creation of the VM 03:23 Zeno` I'm using the VM created by minetest 03:23 hmmmm likewise I don't see the entry point 03:23 hmmmm wait 03:23 hmmmm I'm confused now 03:23 Zeno` the entry point is int luaopen_cavemapgenc(lua_State *L) 03:23 hmmmm how do you have access to the same VM used by minetest 03:23 hmmmm unless you're running inside the same process 03:23 Zeno` that's just how Lua works 03:24 Zeno` mystuff = I have require("cavemapgenc") in the Lua mod 03:24 hmmmm oh 03:24 Zeno` and it loads the shared library, passing the Lua stack to my code when required 03:24 Zeno` never the VM itself though 03:24 hmmmm that's interesting 03:24 hmmmm i never knew about that 03:25 hmmmm so let me get this straight 03:25 hmmmm mystuff is what, in that context? a table object containing those methods you defined? 03:26 hmmmm and in the mod itself you'd just do mystuff.gencaverealm()? 03:26 hmmmm in onmapgeninit I assume 03:26 hmmmm okay I'm understanding how this works 03:26 Zeno` correct 03:27 Zeno` and I'd like to, apart from the argument passing via the Lua stack, avoid using the Lua stack 03:27 hmmmm I don't see how that's possible 03:27 hmmmm are you talking about passing a userdata object or something?? 03:28 Zeno` yep 03:28 hmmmm why couldn't you just say that instead 03:28 Zeno` part of my Lua looks like this: http://codepad.org/PaqizkP8 (CMG is mystuff) 03:29 hmmmm oh this is a horrible slippery slope 03:29 Zeno` if noise params and 'data' could be gained as a "raw" userdata (**u16) then it would simplify a whole lot of things 03:29 hmmmm people are going to want a C api by exposing tables passed to lua as structs in userdata instead 03:29 hmmmm voxelmanip, I can understand... 03:30 Zeno` this avoids having a "complete" C api though :) 03:30 hmmmm in any case, when I first benchmarked LuaVoxelManip vs. a native mapgen, it only came out 2.5x slower 03:30 Zeno` it's a limited exposure via Lua API for just voxelmanip data and the noise 03:30 hmmmm definitely not 10x slower 03:30 hmmmm yeah, limited exposure my ass 03:30 hmmmm you're going to want another thing and then another thing and another thing 03:31 Zeno` My tests show on average 10x increase 03:31 hmmmm you might have been using it the wrong way 03:31 Zeno` I don't know why anyone would want anything other than noise and the voxelmanip data "raw" 03:31 Zeno` For everything else luajit is fast enough 03:32 hmmmm there's this one guy, I forget who exactly, but he keeps asking me to add something new to the schematics feature 03:32 Zeno` I do see your point though 03:32 hmmmm and then he'll say "oh this isn't working, this is a bug!" when it was never added as a feature 03:32 hmmmm e.g. total rotation in all 24 directions 03:33 hmmmm "oh it gets all crazy if facedir is above 4!" 03:33 hmmmm no shit 03:33 hmmmm I specifically made it only rotate along the Y axis 03:33 Zeno` The other advantage is that mapgens using these "advanced" functions would be faster just by the fact that you would not have to populate a Lua table (and depopulate it back when finished); you'd just use memcpy() 03:33 hmmmm ugh, Zeno`, if you write it I'll definitely commit it 03:33 ShadowNinja RealBadAngel: Mesecons uses seperate nodeboxes for each node, so just switching between tiles and special_tiles won't work. (And such a thing would be very ugly) (If I understand what you're suggesting) 03:33 hmmmm this is like a 10 liner patch 03:34 Zeno` yep, it's not a lot at all 03:34 Zeno` I'll write it :) 03:34 hmmmm because you can test it, I can't 03:34 Zeno` thanks for at least listening! I'll write it when I get home and put it up on github 03:34 Zeno` for review 03:35 hmmmm Zeno`: use light userdata instead 03:35 Zeno` ok 03:43 gentoobro the internet seem to say that luajit and gcc are roughly level on performance in simple math and data manipulation 03:43 gentoobro i'm a natural skeptic of such claims, but the luajit disassembly posted on some of the forums was impressive 03:45 Zeno` luajit is indeed impressive 03:46 Zeno` The speed increase I am talking about, though, isn't just about the luajit (it's about copying data which can be done via a loop or via something like memcpy() which, at least on x86, uses fast cpu microcode rather than a simple loop) 03:46 Zeno` so the increases are 3 fold 03:46 Zeno` 3 fold in the sense that there will be improvement in 3 areas 03:48 Zeno` I've done quite a few tests with simple O(n^3) loops that just add or subtract in the inner loop and depending on the expression because evaluated I'm seeing ~10x speed increases 03:48 Zeno` but even 2.5x speed increase for 3 times is still significant 03:49 Zeno` luajit is, I concede, pretty darn fast and fast enough for most things 03:49 Zeno` but, this is a complex loop and C really is better 03:49 Zeno` this doesn't affect anyone either 03:51 gentoobro is that a 10x for just the loops, or the entire block generation time? 03:51 Zeno` not really. I should make a document stating that "best practice" is to use just Lua, but here are the alternative (and limited) other things you can do. Most mod creators would at least write a mapgen prototype in Lua first anyway, unless they're inane 03:51 Zeno` just for the loops 03:51 gentoobro ah 03:51 Zeno` ignoring minetest and doing tests without minetest at all 03:51 Zeno` s/inane/insane 03:52 gentoobro it seems in my uneducated opinion that the speed boost from memcpy would be drowned out by the mapgen's other overhead 03:52 Zeno` You'd be surprised 03:52 gentoobro but i am intested it fou gt serious gains. my mapgen might benefit... 03:52 gentoobro *if you get 03:52 Zeno` the copy/modify/copy cycle is the slowest operation happening 03:53 Zeno` I already know I get serious gains, because I tested it with hacky changes to MT source. I'll make the hacks not hacky though. 03:54 gentoobro are you generating extra 3d noise beyond what the mapgen provides? 03:54 Zeno` no, I'm getting 3d noise from the map and generating my own 2d noise 03:55 Zeno` I'm using two instances of 3d noise though 03:55 Zeno` and kind of morphing them 03:58 gentoobro do you generate canyons? 03:59 Zeno` not at the moment... caves 04:30 Zeno` \o for now 05:17 swaawsios ShadowNinja: commit #1500 is Descripted 05:17 ShadowNinja swaawsios: Huh? 05:17 swaawsios my mistake 05:19 swaawsios ShadowNinja: ? 05:19 ShadowNinja swaawsios: Oh, the commit message is still undescriptive. One sec, lemme see if that could actually be the source of that bug. 05:23 ShadowNinja swaawsios: Nope, adding a semicolon won't affect that bug. 05:23 ShadowNinja swaawsios: That bug is rare and unreproducable it seems. 05:24 swaawsios if you blaspheme run the server longer time passes the error in (with longer I mean 48h) 05:28 swaawsios with random users, no mods, default config, release Build 05:29 swaawsios ShadowNinja: Creative, and above-average use TNT 05:32 swaawsios ShadowNinja: This error occurs only when errors occur in the game. 05:37 RealBadAngel https://github.com/minetest/minetest/pull/1519 05:40 RealBadAngel ShadowNinja, any comments on above? 05:40 ShadowNinja swaawsios: Yes, it happens when an error occurs and Lua can't handle the error. I checked Lua's code and it only happens when the error handler isn't a function or there's a stack overflow. 05:42 ShadowNinja RealBadAngel: Seems O.K., but wait, there are compatability concerns. 05:43 RealBadAngel thats why bump is here 05:48 swaawsios ShadowNinja: I have the commit recorded in my fork with and it runs without this error 05:50 ShadowNinja swaawsios: A fluke. The semicolon isn't changing the error, you're just not getting it because it doesn't happen consistently. 05:51 ShadowNinja I had an idea about what it might be though... 05:54 swaawsios END_DEBUG_EXCEPTION_HANDLER(errorstream); 05:56 swaawsios is as a function of the error is all the transfers to act to this 05:57 ShadowNinja I think the error is occuring in a C function called from Lua that calls Lua. Such calls have a specially manipulated stack so that index 1 is the start of function arguments, instead of the real start of the scack. 06:00 swaawsios so that the main loop can make their service and all the errors to kill gently so that it continues 06:02 swaawsios ShadowNinja: Garbage Collection for errors 06:03 swaawsios ^^ 06:07 ShadowNinja swaawsios: Huh? 06:07 RealBadAngel btw, what do you think about making param2 16bits instead of 8? 06:08 ShadowNinja RealBadAngel: Highly incompatible, and then why not 32 or 64 bits? It takes up more stace, but probably not enough to matter. 06:09 RealBadAngel we are about to bump version anyway 06:09 ShadowNinja param1 and param2 are a bit limiting. A variable-length list of params would be great. 06:09 RealBadAngel and i do need extra bits badly 06:09 hmmmm a variable length list of params is called 'node metadata' 06:10 hmmmm if you were to make the base node data variable length, you can kiss random access any any semblance of speed goodbye 06:10 hmmmm i explained this to vanessae a while ago 06:10 RealBadAngel i dont wanna meta in this case, i need fast accesible data 06:10 ShadowNinja hmmmm: That's only string-indexable though, right? This would be disigned to be compact and fast. 06:10 hmmmm ? 06:10 hmmmm indexable as in you can do random access to a mapblock to retrieve a node 06:11 RealBadAngel and i do need just a few bits, makin param2 16 bits is far more than enough for me 06:11 hmmmm lol 06:11 hmmmm no 06:11 hmmmm stop it 06:11 hmmmm you're going to double the size of everything 06:11 ShadowNinja RealBadAngel: What do you need it for? 06:11 hmmmm the same arguments against colored lighting applies here 06:11 hmmmm s/applies/apply/ 06:12 ShadowNinja hmmmm: zlib should slim it down quite a bit. 06:12 hmmmm twice as big in memory though 06:12 RealBadAngel one bit could be used for switching tiles/special tiles usage for example 06:13 ShadowNinja Yes, but do mapblocks really take up that much memory space? 06:13 hmmmm and it's not just storage space 06:13 ShadowNinja RealBadAngel: Nah, that technique is ugly. 06:13 hmmmm it's speed as well 06:14 hmmmm you take up more cache which means less nodes cached 06:14 hmmmm twice as many cache misses 06:14 RealBadAngel ShadowNinja, that is fast, way more faster than manipulating the meta 06:14 hmmmm look, if you're going to double the size of a mapnode (which I highly recommend against), for a single extra byte, then you need to add colored lighting as well. 06:15 RealBadAngel single if to switch to another set of tiles in rendering loop 06:15 RealBadAngel hmmm, irrlicht can do coloured lights for us, we dont have to reinvent the wheel 06:15 ShadowNinja hmmmm: That needs two more bytes. At least for all colors. 06:16 hmmmm what I'm saying is that 06:16 hmmmm if we really could double a mapnode's size, you could do lots of things that you could not before 06:16 hmmmm but we'd also be on par with minecraft 06:16 hmmmm yuck 06:16 ShadowNinja I don't agree to the extra bytes though. 06:17 RealBadAngel how many bytes one node takes already? 06:17 hmmmm ..... 06:17 hmmmm how don't you know 06:17 hmmmm 4 06:17 hmmmm if you add another byte, that's 5. that makes a mapnode 8. 06:17 RealBadAngel i c 06:17 ShadowNinja That would take us from 16 to 24 kb per block. 06:17 hmmmm you mean 32 06:18 ShadowNinja Well, if you do it for both params. 06:18 ShadowNinja hmmmm: No, param0 os left as-is. 06:18 hmmmm no, you can't have 6 byte mapnodes 06:18 ShadowNinja is* 06:18 hmmmm they'd be padded to 8 06:18 hmmmm and then you'd be left with 2 completely useless bytes 06:19 RealBadAngel or a place for something new ;) 06:19 hmmmm i don't like it at all 06:19 hmmmm this is why i did not want colored lighting then and not now 06:20 RealBadAngel ok i will try to stick to what we have 06:26 RealBadAngel hmmmm, are you ok with #1519 ? 06:26 hmmmm what's #1519 06:26 RealBadAngel change special tiles count to 6 and protocol bump 06:27 RealBadAngel https://github.com/minetest/minetest/pull/1519 06:28 hmmmm OH YAY 06:28 hmmmm we're making ContentFeatures even larger! 06:29 hmmmm I don't think that's the correct protocol version to bump 06:30 RealBadAngel game can handle already different count so imho bump wouldnt be needed, but c55 and sapier insisted on bumping 06:31 hmmmm when people last changed ContentFeatures, what did they do? I am looking at ContentFeatures::deSerialize and it expects a constant version number of 6 06:31 hmmmm why that isn't a constant is beyond me. 06:32 hmmmm ah yes, protocol_version there is the global protocol version 06:32 hmmmm I thought there was a ContentFeatures version as well 06:35 RealBadAngel there is 06:35 RealBadAngel 6 atm 06:36 ShadowNinja swaaws/swaawsweb: If you get that error again please tell me exactly what mod error caused it and provide the code please. Minetest's code doesn't seem to have the issue I thought of. 07:16 swaawsios ShadowNinja: ok 08:35 sfan5 I'm going to merge https://github.com/minetest/minetest/pull/1516 in 15 minutes 10:04 Zeno` what happens if git is not installed? 10:10 sfan5 Zeno`: talking to me? 10:15 Zeno` sorry, yes 10:16 sfan5 there will be no git version detection then 10:17 Zeno` so it'll just say version-githash? 10:18 sfan5 no 10:18 sfan5 it won't say anything 10:18 Zeno` no version string at all? 10:18 sfan5 yes 10:18 sfan5 0.4.10-dev 10:18 Zeno` ahh, ok 10:18 Zeno` cool 10:19 Zeno` and if I modify somerandomsrcfile.cpp does it have to recompile every single src file now? 10:19 sfan5 no 10:20 Zeno` great :) 10:20 sfan5 only version.cpp client.cpp lua_api/somefile.cpp server.cpp serverlist.cpp and main.cpp 10:20 Zeno` sounds a lot better than the previous set up 12:02 celeron55 wat 12:04 celeron55 https://github.com/minetest/minetest/pull/1519#issuecomment-50140205 12:04 celeron55 i can't believe RBA is being this ignorant again 12:16 sfan5 celeron55: https://github.com/minetest/minetest/pull/1518 <- does that look like it might break something I'm not aware of? 12:16 sfan5 and can we merge https://github.com/minetest/minetest/pull/1510 12:16 sfan5 #1510 -> Add DISABLE_GIT_DETECTION CMake option 12:19 celeron55 dunno 12:23 sfan5 soo.. what about #1510 then? 12:33 celeron55 it was an answer to both 12:34 RealBadAngel celeron55, i wasnt sure whats have to be done when bumping version 12:35 RealBadAngel can you explain what do you mean with by "compatibility in (de)serialization" ? 12:36 RealBadAngel code checks already how many tiles being sent 13:04 sfan5 now that more people are here: how about merging #1510 ? 17:37 ^v does anyone use ffi? 17:39 sfan5 no 17:39 ^v why not 17:39 sfan5 becuase not available everywhere 17:40 sfan5 and not very portable 17:40 ^v LuaJIT's ffi works everywhere ._. 17:40 sfan5 but not everyone uses LuaJIT 17:41 ^v why not ._. 17:41 sfan5 because not everyone needs it 17:41 sfan5 and luajit does not work everywhere 17:41 ^v its more portable than normal lua 17:42 sfan5 uh 17:42 sfan5 no 17:43 ^v give me an OS that has opengl and doesnt have luajit 17:45 sfan5 anything that runs mesa on a processor arch not supported by luajit 17:45 ^v x86 x64 ARM PPC e500 MIPS 17:45 sfan5 (yes, mesa has a software driver) 17:46 sfan5 AArch64? 17:46 sfan5 iPhone 5S in that case 17:46 sfan5 has OpenGL ES and not supported by LuaJIT 17:48 VanessaE sfan5: wouldn't it make more sense to bundle LuaJIT (to hell with what the author thereof wants) and just finally update Lua to 5.2 so that this is no longer an issue for those platforms that can't use it? 17:48 VanessaE I mean, wasn't the latter already partly in progress? 17:48 sfan5 <VanessaE> sfan5: wouldn't it make more sense to bundle LuaJIT 17:48 sfan5 no 17:48 sfan5 bundling things is bad 17:48 sfan5 and just finally update Lua to 5.2 so that this is no longer an issue for those platforms that can't use it? 17:48 sfan5 I'd really like 5.2 17:49 ^v LuaJIT has goto support by default 17:49 VanessaE goto is bad. 17:49 VanessaE and you should be shot for suggesting it :) 17:49 ^v 5.3 is the only thing that would actually be better than luajit 17:50 ^v VanessaE, whats wrong with goto 17:50 VanessaE if you have to rely on goto to write your code, you're doing it wrong., 17:50 sfan5 VanessaE: goto is good 17:50 sfan5 but often abused 17:51 VanessaE and I come from the world of BASIC. 17:51 ^v goto is good if you want to break out of multiple loops 17:51 ^v exept i still dont use goto 17:51 sfan5 ^ 17:52 ^v the reason i am stuck with luajit is because ffi 17:52 ^v you can use liblua functions with it 17:53 ^v so you can control and thread your own lua_state 17:53 VanessaE so, again... what happened with that 5.2 update that was supposedly in progress? 17:58 ^v sfan5, "You can cross-compile for iOS 3.0+ (iPhone/iPad) using the iOS SDK." 17:58 sfan5 VanessaE: IIRC sapier didn't like it 17:58 VanessaE *grumble* 17:58 sfan5 ^v: if luajit does not support AArch64 that is pointless 18:11 Calinou what does 5.2 really offer in comparison to 5.1? is 5.3 finished yet? 18:11 Calinou do distros reliably ship 5.2 and 5.3? 18:11 ^v 5.3 is just 5.2 with bitwise operators >_> 18:12 ^v the env system in 5.2 is really weird 18:12 ^v you have to use debug in order to set the env of a function already loaded 18:12 ^v loadstring is now just load 18:13 ^v a lot of libraries dont support 5.2 yet 18:15 Sokomine as for lua and mt, most mods do not seem to use libraries anyway. it reduces chances of someone installing the mod if there are other external mods it depends on 18:16 ^v do some mods even include c fileS? 18:16 Sokomine not to my knowledge 18:16 ^v :/ 18:17 Sokomine quite a lot of mods achieve what they're good for with very simple lua. that's ok - it's a blocky world, and nice nodes are always welcome 18:17 ^v oh and what really makes me mad in 5.3 is that the int64s are signed 18:17 Sokomine library usage of any kind is quite limited due to the fear that dependencies would decrease will and capabilites of users to try the mod 18:20 Exio the irc mod 18:20 sfan5 Calinou: http://www.lua.org/work/doc/#changes 18:21 ^v i totally forgot to mention bitwise operators 18:22 ^v ** is cool too 18:24 Calinou “support for integers (64-bit by default)” uh? 18:24 ^v er 18:24 ^v / 18:24 ^v //* 18:24 ^v integer division 18:25 ^v <v^> .l53 1<<63 18:25 ^v <^v> v^, -9223372036854775808