Time Nick Message 00:00 est31 hrm, /me wonders how hard it would be to add scrypt support 00:01 est31 perhaps we should handle it transparently 00:01 est31 and give it the scrypt hash 00:01 est31 as "password" 00:02 est31 that sounds quite good 00:02 est31 Only thing that needs modification then is the salt generation 00:03 est31 (as in: use our salt, dont invent your own :p) 00:03 est31 then we can use one salt for the scrypt and the srp 00:06 est31 another problem csrp has is seeding portability 00:06 est31 we have to manually seed it on platforms that aren't windows or linux 00:07 est31 adding that as //TODO 00:07 * est31 is no bsd expert, cant even test behaviour there 00:40 paramat hmmmm i'd like to push #2611 in 2 hours, please could you review before then? 00:40 ShadowBot https://github.com/minetest/minetest/issues/2611 -- Cavegen: Remove now unnecessary checks for water, lava, ice. Various mgv5/mgv7 fixes by paramat 00:49 hmmmm looks good to me 00:51 est csrp implementation looks much better now 00:52 paramat :) 00:52 hmmmm est, this might be useful (probably not) https://github.com/kwolekr/chiisai-bnetd/blob/master/src/srp.c 00:53 est oh you have implemented srp yourself already? 00:53 hmmmm I reverse engineered the SRP implementation used by WC3 00:53 hmmmm it's not 100% standard but 00:54 hmmmm here is the client-side computation: https://github.com/kwolekr/phyros/blob/master/src/crypto/srp.c 00:55 est what are your dependencies? 00:55 hmmmm libgmp 00:55 hmmmm that being said, it's probably more secure to use a current implementation of SRP using modern hash algorithms 00:56 hmmmm this only works on 32 byte integers 00:56 hmmmm and 20 byte sha1s 00:56 est 32 bytes seem ok 00:56 hmmmm yeah but 00:56 hmmmm the standard is now sha-512 iirc 00:56 hmmmm what are the dependencies of csrp? 00:56 est openssl 00:56 hmmmm oh come on 00:56 hmmmm what does it use openssl for? hashing? 00:57 est hashing and bignum 00:57 hmmmm hrmm 00:57 hmmmm are there any alternatives? 00:58 est I haven't done a really thorough study yet 00:58 est only taken the one linked by ShadowNinja 00:58 hmmmm i'd like to not bring in openssl 00:58 hmmmm if possible 00:58 est isnt it there already? 00:58 hmmmm i didn't think so... is it? 00:59 hmmmm cURL maybe, but that's an optional dependency on a dependency 00:59 est yes 00:59 hmmmm two optionals 00:59 hmmmm bringing in dependencies are serious business 00:59 est yes 00:59 hmmmm because generally they're a big pain in the butt 01:00 hmmmm libgmp in particular works fine on windows so I'm okay with that, but anything GNU i would be wary of because they try to make it as difficult as possible 01:01 est can we ship libgmp ourselves? 01:01 hmmmm yes 01:01 est now we only need a method for secure random 01:01 hmmmm we have 01:02 est how? 01:02 hmmmm PcgRandom() 01:02 hmmmm although it doesn't hurt to use OS random facilities too 01:03 hmmmm rand_s() for Windows, /dev/random for anything else 01:03 est PcgRandom is a pseudorandom 01:03 est with a quite narrow state 01:03 hmmmm PCG32 looks simple, doesn't it? 01:04 hmmmm it's crypto-quality though 01:04 hmmmm read up on it 01:04 est I've just seen the u64 01:05 est I mean, what does sha-512 use when you only have 64 bits of state in your PRNG 01:07 est when you say it I believe you 01:07 hmmmm PcgRandom::bytes(64) 01:08 hmmmm :p 01:08 hmmmm its state is unpredictable 01:08 hmmmm er wrong word 01:08 hmmmm assuming you randomize both the sequence and initial seed 01:08 hmmmm http://www.pcg-random.org/ 01:08 hmmmm read 01:09 hmmmm my point is that because you can't predict the state given the current output of PcgRandom::next(), you can create as much of a range as you'd like 01:10 est yes 01:10 est but still the entropy is less than or equal than 64 bits 01:11 est but that can be solved 01:12 est http://www.pcg-random.org/useful-features.html#multiple-streams-sequences 01:12 est ^ cool trick I found 01:14 est when we have 16 bytes from the server and 16 bytes from the client, thats enough 01:14 hmmmm cool stuff 01:14 est then we combine, and have those 512 bits 01:14 hmmmm i think you should still add in the OS enthropy as well 01:14 est yes, thats the only source 01:14 hmmmm erm I mean enthropy from the OS random generator 01:15 est so when a server starts, its 4 PRNGs get seeded by the OS generator, and then we live off that 01:15 est on the client side similar 01:15 hmmmm sure 01:21 est what do you think, can we use your srp implementation? 01:22 hmmmm no 01:22 hmmmm i don't trust it 01:22 hmmmm it's not my implementation either, it's a nonstandard implementation with some questionable changes 01:23 est ok 01:23 hmmmm looking for: 01:23 est option 2: use the openssl one 01:23 est option 3: use libgmp and roll our own 01:23 est option 4: search for better implementations 01:23 hmmmm something with lightweight dependencies 01:24 hmmmm uses the current official SRP standard 01:24 hmmmm which I think is SRPv7 01:25 est 7? I only know of 6a 01:25 hmmmm oh it is 01:25 hmmmm i assumed there was a newer one out 01:26 hmmmm doesn't openssl contain an SRP implementation? 01:26 est dunno 01:27 est the so called "reference implementation" is an openssl patch 01:27 est so perhaps it has been merged 01:27 hmmmm which is why i say that some "csrp" with openssl as a dependency is a bad idea 01:27 est why? 01:30 est also, if this is the srp API of openssl, then we might not want it: https://github.com/libressl/libressl/blob/master/src/crypto/srp/srp.h 01:30 hmmmm oh BS 01:30 hmmmm I am looking at how to use SRP in OpenSSL right now 01:30 hmmmm you need to actually execute some things from the openssl command line if i'm reading this correctly 01:35 hmmmm maybe csrp isn't that bad after all 01:35 est if you have read the same wordpress article I have, then I fully agree 01:36 hmmmm I wonder how tough it would be to take csrp and modify it to use libgmp for big numbers and then use our own hash algorithm 01:36 est not that hard 01:36 est also csrp is hash alg agnostic 01:36 hmmmm i'd go with that then 01:36 est so we can give it sha-512 01:39 est including it may have issues though 01:39 est (libgmp) 01:39 est "Since version 6, GMP is distributed under the dual licenses, GNU LGPL v3 and GNU GPL v2" 01:39 est we are LGPL v2.1+ 01:45 hmmmm hrmmmm 01:46 hmmmm well 01:47 hmmmm freebsd distributes GPLed software in their base and they're able to keep their BSD license 01:47 hmmmm I think it only matters if you are modifying libgmp 01:47 est ok 01:49 VanessaE "error in error handling". real fucking useful, Lua. :P 02:09 est hmmmm, what's this LPNLS stuff? 02:09 est I cant find it in libgmp 02:19 est guess I dont need it 02:20 est cool, the API of csrp doesnt even require openssl 02:26 hmmmm NLS is just a structure that contains all the SRP variables for that user session 02:44 est ah I see thought it was gmp stuff 03:40 paramat now pushing #2611 03:40 ShadowBot https://github.com/minetest/minetest/issues/2611 -- Cavegen: Remove now unnecessary checks for water, lava, ice. Various mgv5/mgv7 fixes by paramat 03:49 paramat push complete 03:53 est gmp is more powerful 03:53 est ^ first impression 04:00 paramat hmmmm i have a random idea: multithreaded lua is only really needed in one specific case, when a lua mapgen is reading, processing and writing a newly generated chunk. as it's unlikely any other lua operation will be accessing that new mapchunk why not use a (erm) 'local envlock' on the entire voxelmanip volume of that new mapchunk? 04:02 paramat so essentially focussing on one specific need instead of more general multithreading 04:02 hmmmm multithreaded lua is for running multiple mods at the same itme 04:02 hmmmm s/itme/time/ 04:03 est fulfilling that specific need is easier than a "solution to solve it all" 04:13 paramat mmm i don't mean 'instead of', general multithreading would be excellent. it just seems easier in that circumstance 04:14 ShadowNinja Multithreading has to be "opt-in". A tweaked versionof the current async API would be good. 04:15 ShadowNinja You need to be able to do things like specify "this task will run for a long time, or until the server shuts down, so you should spawn a new thread just for it instead of having it back up the queue". 04:15 ShadowNinja Or better yet, automatic thread pool management. 05:11 est only one function left to convert 05:12 est (and then random stuff ofc too) 05:12 ShadowNinja Apparently Jsemaphores use a global mutex lock on OSX... 05:14 ShadowNinja Can someone explain this ^? 05:30 ShadowNinja It seems like OSX semaphores don't store their value, but the global mutex seems like a bug. 05:30 est hmmmm, I suggest we use SHA-512 internally for all housekeeping hashing (inside the srp auth), and scrypt or another such fancy new hash for the verifier generation. That will be the only part where we will be susceptible to brute-force. 05:31 est I can make a PR that replaces our current SHA-1 implementation with the implementation of OpenSSL, which also has other SHA functions like SHA-512 05:36 Zeno` ShadowNinja, can I add to the protocol without breaking it? 05:36 Zeno` i.e. I imagine that older clients would just ignore the unknown bits (yeah I can look, but you know already so I thought I'd ask) 05:36 ShadowNinja Zeno`: Yes, you might have to bump the protocol version so that you don't send the messages to old clients though. 05:37 Zeno` ok thanks 05:37 ShadowNinja Oh, to a current message? Yes. 05:37 VanessaE_ there's an optional/extensions section of the protocol isn't there? 05:41 est btw protocol 25 isn't used right now Zeno`. 05:41 est its half finished 05:42 est I'm making a change that makes it even less finished :p 05:42 Zeno` yeah it's a current message 05:43 Zeno` hehe est 06:24 hmmmm ah scrypt 06:25 hmmmm what's wrong with hashing SHA-512 like 500 times? 06:25 hmmmm I think a good target should be for the operation to take ~200ms on a current gen i7 06:25 hmmmm you need to respect older computers too 06:28 hmmmm anyway I looked into it a bit more, preemptive threading in lua is pretty much impossible. i thought you could do it with debug hooks, but you can't set the execution to somewhere else 06:29 hmmmm so yeah we're gonna need to make like 5000 threads :( 06:30 est SHA-512 500 times can easily be parallelized 06:31 hmmmm how on earth can you parallelize that 06:31 est you can make a pipeline 06:31 hmmmm ? 06:31 est of 500 sha-512 steps 06:32 est at least thats what I think why its bad 06:32 hmmmm what do you mean by a pipeline 06:33 hmmmm unless i'm seriously misunderstanding something, it's no more parallizable than running scrypt on 500 different passwords on different processors all at once 06:34 est I think the trick about scrypt was that it also requires alot of ram 06:34 est and at the time it came out, it wasn't abundant in these programmable logic chips 06:35 est but with those cryptocurrencies we now have the situation that there are ASICS for scrypt 06:36 hmmmm :( 06:36 hmmmm hmm i can't find exactly how "large" these pseudorandom bit strings are 06:37 hmmmm sure, scrypt looks good enough 06:37 hmmmm I think we should have this as an option though 06:38 hmmmm there are certainly a lot of cases where this could get annoying (like debugging) or too expensive for servers on low-end hardware 06:38 est servers shouldn't touch scrypt 06:38 est clients only 06:38 hmmmm oh okay 06:38 hmmmm so that case is out 06:39 hmmmm so like a strength parameter 06:39 est servers can use SHA-512 instead 06:39 hmmmm one could be sha-512, and the number of iterations, and then the other option scrypt, and the number of iterations (i guess multiple scrypt rounds if you're feeling crazy?) 06:39 est scrypt has a parameter thats controlling that 06:39 hmmmm oh great 06:40 est it even has two one for memory one for cpu 06:43 est so, which SHA-512 implementation to use? 06:50 denis_ gor what purpose you use sha-512? 06:50 denis_ *for 06:50 est srp internal hashing 06:55 est sha-256 sounds good enough though too 07:04 denis_ mods can be written only using Lua? Is it possible, to use C++? 07:04 hmmmm technically yes 07:04 hmmmm you'd still have to use lua as an intermediary though 07:19 VanessaE_ it is a tantalizing thought, C++ mods with access to the same API calls as Lua. just-in-time compiling with caching of the compiled results would make it pretty painless to use I think. 07:19 est stupid nacl 07:19 est they claim to be the best of the greatest 07:19 est but then they dont even have separate init and update functions 07:20 est only a large crypto_hash function 07:20 est as if I knew my hash content from the beginning 07:38 est gonna take openssl and extract it somehow 07:38 est shouldnt be too hard 07:44 hmmmm this for sha-512 or scrypt? 07:44 hmmmm that's pretty pathetic that you can't find a decent implementation anywhere 07:45 sfan5 libnettle has sha-512 07:45 sfan5 in case that helps 07:51 est I can find impls but I want one that doesnt leak side-channels 07:52 sfan5 This is a game not a banking app. 07:52 est nettle looks good too, I'll have to find out which is easier to steal from, openssl or nettle 07:52 hmmmm heh agreed 07:52 hmmmm but that's the same argument made when the auth was first implemented 07:52 hmmmm if we're doing it right now it better actually be done right 07:53 est yes, but wasnt it you who wanted it to be perfect sfan5? 07:53 sfan5 no 07:53 sfan5 also: please don't use openssl 07:53 est (or perhaps it was sapier, no idea) 07:53 sfan5 it's build system sucks 07:53 est I'll only copy code 07:53 hmmmm ty. 07:54 sfan5 est: copying code out of openssl will probably end up with copying half of the other openssl stuff 07:56 hmmmm est, you're looking for a side-channel resistant sha-512? how would timings differ in sha assuming the data being hashed fits inside the size of chunks it operates on? 07:56 hmmmm you know how sha-1 operates on like 256 bytes at a time, and anything below that is padded with zeros 07:56 est dunno 07:57 est sfan5, I got quite far with removing includes in openssl 07:57 est one file only was included to make openssl run on cray operating system 07:57 sfan5 who says the resulting code is gonna work 07:58 est thats an assumption I havent confirmed yet 07:59 est but ofc I'll have to test it then 08:01 sfan5 I'd rather link to a smaller crypto lib than rip out code from the bigger openssl and hope it works 08:05 est adding deps is pain in the butt hmmmm is right 08:05 est sha functions aren't black magic 08:08 hmmmm ugh 08:08 hmmmm so I gave Kdevelop a fair try 08:08 hmmmm it looks solid at first, but it actually functions rather poorly 08:18 jin_xi i like qtcreator 08:43 est ok the sha impl from openssl seems to work 08:43 est and only two files! 08:44 est (sha256.c and sha2.h) 08:49 est no, sorry. 3 files 08:49 est md32_common.h is needed too 08:58 denis_ OpenSSL rulezzz 09:01 sfan5 OpenSSL sucksss 09:02 est where to place gmp library? 09:03 est directly in src/ like lua? 09:04 sfan5 we're not including gmp into the minetest souce 09:04 sfan5 +r 09:04 sfan5 or rather I think thats a bad idea 09:05 est the other alternative would be to depend on it 09:05 est but dependencies are ... bad. 09:05 sfan5 the sqlite source initially was in the minetest src too 09:06 sfan5 but it was added as a dependency because of reasons 09:06 sfan5 such as that it was never updated 09:06 sfan5 also if we're going to depend on libgmp why not depend on just libnettle 09:06 est we dont need libnettle 09:07 sfan5 why not 09:07 est why yes 09:07 sfan5 it's 1 dependency and does what we want 09:07 sfan5 if we use openssl we also have one 1 dependency 09:07 est I dont want to use openssl 09:07 sfan5 (and some code we may or may not need to update [sha256.c, sha2.h, md32_common.h]) 09:08 sfan5 if we use the files from openssl* 09:08 est we rather may not need to update 09:09 sfan5 actually 09:09 sfan5 why not just implement sha256 ourselves? 09:10 est adding dependencies just because isnt good 09:10 est there are still things that need to be maintained if you have dependencies 09:10 est like cmake finder files 09:11 sfan5 we need a dependency anyway 09:11 sfan5 including more code into the minetest src because "dependencies are bad" is not good either 09:11 est so its one dependency against two dependencies 09:11 sfan5 two dependencies? 09:12 est we'll still need libgmp 09:12 sfan5 yeah 09:13 denis_ Are you trying to make some logon/password verification? 09:13 est yes 09:14 est I have ported csrp from openssl to libgmp, but I also want to use a better hashing algorithm 09:14 est therefore the discussion about where to get the hash alg 09:14 denis_ I think it's easy to find source code for any hashing algorithm 09:15 est minetest has high requirements 09:15 est for example, the license has to be right 09:15 sfan5 did i mention that it's a game and not a banking app 09:15 est still if its GPL we cant include it 09:15 sfan5 and: why not just implement sha256 ourselves? 09:16 est why not depend on systemd just because? 09:16 sfan5 what does this have to do with systemd 09:17 sfan5 using openssl's code is 1 dep. (libgmp), nettle is 1 dep. (libnettle) and writing the code ourselves is 0 deps 09:18 denis_ https://code.google.com/p/blockchain/source/browse/trunk/SHA256.cpp 09:18 denis_ the second google link. MIT license 09:19 est no, using openssl's code isnt 1 dep 09:19 denis_ I can't understand, why you need only hash algo, if you are writing some security stuff, you need much more than that 09:19 est that code only depends on std c lib 09:20 est yes we do need more than that 09:20 est for srp we need libgmp 09:20 est (not for the openssl sha2) 09:21 sfan5 ohh 09:21 sfan5 you should've said that 09:21 sfan5 including libgmp into the minetest source is still a bad idea imo 09:21 est sorry bout that 09:22 * sfan5 away 09:24 denis_ Java easier in that way, it already has a lot of stuff. 09:25 est yea 09:55 kilbith seeing and confirming that : https://github.com/minetest/minetest/issues/2541#issuecomment-90158381 the best fix would to revert bfc4652 and aa340fd 09:56 kilbith i don't think nerzhul is able to provide a sane fix 09:56 kilbith ^ Zeno`, what are tour thoughts ? 09:56 kilbith *your 10:03 est I'm not sure, but bfc4652 only fixed an error 10:03 est the packet is supposed to be unreliable 10:05 kilbith it cause an huge freeze with 10:07 kilbith so i dunno whether the unrealiable packet is inherently not the good way here, or the nrz's adaptation ain't good 10:08 est That packet was unreliable in the past 10:08 est then it got reliable due to 10:08 kilbith ok, so it's nerzhul shit 10:08 Krock +'s 10:08 est seems caused by his changes 10:15 Krock Somehow I can't reproduce the errors of 2541 14:46 hmmmm libgmp is something that doesn't need to be updated often 14:46 hmmmm I don't think bundling it by default is a horrible idea - don't get me wrong, users should still be able to use their system version 19:54 kilbith ShadowNinja, as requested... game#486 19:54 ShadowBot https://github.com/minetest/minetest_game/issues/486 -- Convert beds in mesh by kilbith 19:57 VanessaE_ aw, not using homedecor's model :( 20:19 est31 I think I can live with all options -- bundled gmp or not. just hmmmm is right, dependencies are a PITA. 20:19 hmmmm especially on windows 20:19 hmmmm god dammit 20:20 hmmmm every time is included, it compiles all that code 20:20 hmmmm so including STL containers inside of a header files is a double-whammy 20:20 hmmmm really wish it was easier to forward define templated classes 20:21 hmmmm forward declare rather 22:20 est31 hm, I have now some method chunks for the new protocol 22:20 est31 I can do it nrz style and commit them to master, so that everybody can see 22:21 est31 (nothing functional will change, as the protocol is still v24) 22:21 est31 or I can make a PR 22:22 est31 I dunno whether its a smart idea to share my design for comments 22:22 est31 but I also dunno whether ppl actually care, after the overwhelming storm of discussion my mail has started. 22:22 est31 I mean it is already shares 22:23 est31 shared* 22:23 est31 sharing method chunks isn't better either I think 22:23 est31 (btw now I think crypto stuff is for later, and can be done in its own packets, outside of init, too) 22:46 jin_xi hm, so i've been checking out pathfinding a bit 22:46 jin_xi hacked in a way for paths to avoid a node and removed strange line of sight optimization - ta da! 22:47 jin_xi works much better and probably more as expected for mob makers 22:47 jin_xi avoid a node - i mean to avoid all nodes of a given type 22:49 jin_xi the line of sight thing is especially bad, it will happily produce paths over thin air