Time Nick Message 00:11 VanessaE kahrl: your anti-non-cURL patch has been deployed to all five servers now for testing. 01:24 celeron55 "We have received your attached DMCA notice, however we need additional information from you in order to continue investigating. For each application you've provided, please identify the exact code that you claim infringes upon your copyright." 01:24 celeron55 what 01:24 celeron55 what do they expect 01:25 celeron55 like, links to minetest's source files on github? commits? or source code of the infringing thing? 01:26 VanessaE my guess would be the first one 01:26 us|0gb celeron55: Who is infringing on Minetest's copyrights? 01:27 VanessaE but every bit of evidence you can subit that shows how they got it, including the forum thread where they showed their intent to fork would be a good idea 01:35 hmmmmm yoyoyoyo 01:35 hmmmmm what's this enet thing all about 01:35 hmmmmm looks like I missed quite the discussion 01:35 VanessaE hmmmmm: the short version: fixing media_FUBAR by removing it entirely. 01:35 VanessaE basically. 01:36 hmmmmm why can't anybody just fix TCP 01:36 VanessaE because random? :) 01:36 hmmmmm I don't understand the need for yet another dependency for one 01:36 VanessaE it's not a dep 01:36 VanessaE it's gonna be bundled 01:36 VanessaE but your point stands 01:36 hmmmmm two, it's still not going to have the same performance for bulk transfers as TCP 01:37 hmmmmm games that suck using TCP are using it the wrong way 01:37 hmmmmm media should be TCP, block transfers should be TCP, everything else doesn't really matter that much 01:37 hmmmmm and probably should stick with the current protocol 01:37 E4xoi enet is what is used by other games for getting "tcp-like" transfer speeds with udp 01:38 hmmmmm if anything, my opinion is that TCP should be implemented the right way, and some investigation should be done on what enet does better than the current reliable udp packets 01:38 E4xoi http://enet.bespin.org/ 01:40 hmmmmm the reverse compatibility thing is a real killer though 01:40 hmmmmm if we use enet there's no turning back 01:40 hmmmmm and we don't NEED to do this, somebody just wants to because it's the quick-n-easy solution 01:41 E4xoi tcp would need to wait for 0.6.x 01:41 E4xoi ;P 01:42 hmmmmm I'd rather wait a little while instead of just break everything that we had 01:42 E4xoi nobody seems to be working in 'fixing TCP' 01:43 hmmmmm I totally would if I had an extended amount of time to blow 01:43 hmmmmm all I can say is this: 01:43 hmmmmm if you do all decide to go ahead with this enet bullshit, it might be the right time to check out application-layer libraries like protobuf, or bvkl, or valuepack 01:44 hmmmmm at least make the two big reverse compatibility murderers in the same release 01:44 celeron55 us|0gb: the source code -less forks on google play 01:45 * us|0gb goes to find them 01:49 VanessaE us|0gb: Buildcraft, zombie-something, starvation-game, one or two others. 01:53 E4xoi hmmmmm: that was the idea.. 01:53 celeron55 hmmmmm: it ultimately is a matter of what somebody is willing to make well 01:55 celeron55 including future maintainability (which kind of falls on all of us) 01:55 hmmmmm all I'm saying is that we can have it all - one less dependency, complete reverse compatibility, flexible protocol, the speed of TCP, etc. - if we're willing to wait for good things 01:56 celeron55 running tcp and udp side-by-side isn't exactly flexible compared to a well-behaving fully udp-based thing 01:56 celeron55 almost everything is a compromise 01:56 celeron55 or, well, everything 01:57 hmmmmm what's inflexible about it 01:57 hmmmmm you can use udp for the hole punching if that's why you're worried 01:57 us|0gb The screenshot images don't seem to be loading for me, but I managed to find the pages. 02:00 celeron55 hmmmmm: how do you do congestion control? the only way really is to hope that tcp doesn't fill the connection so much that bursts of udp packets won't be lost, and send all udp packets right away hoping that there is still space for tcp traffic 02:01 celeron55 something like enet (either enet itself or a fixed version of what minetest uses now) gives more control to that kind of behavior 02:03 hmmmmm hwo does enet do that? 02:03 celeron55 and then, even if tcp and udp are used simultaneously, minetest will still need reliable udp transfers because some things need to be sent reliably with minimal latency (which won't happen if there's a bunch of mapblocks waiting for transfer in a tcp buffer) 02:03 hmmmmm that's fine, we already have reliable udp 02:04 celeron55 what do you think about NIH? thexyz always thinks it's inherently bad 02:04 hmmmmm using NIH things means we lose backwards compatibility 02:04 hmmmmm I think it's REALLY cool that we can still connect to 0.4.4 servers 02:04 celeron55 we don't need infinite network backwards compatibility; it's not an issue here 02:04 hmmmmm and a lot of other people do too probably 02:04 celeron55 the goal is to make a decision that will be usable for years to come 02:07 celeron55 if it's compatible with what there is now, fine; but it's weight should be minimal and it should only matter in an otherwise tie situation 02:07 celeron55 its* 02:09 hmmmmm maybe you're right, i'm putting too much weight into this 02:09 hmmmmm it's just a damn video game 02:09 celeron55 i think that the real issue with enet is that it lacks stuff like advanced handling of threads and ipv6; so we'd need to implement stuff ourselves in any case 02:12 celeron55 if we were infinitely good at doing things, the right thing to do would be to modify the current network stack to be as good as enet but with containing all those details we want from it, and then even make it a library so that everyone can take advantage of it (because obviously something like that isn't available; otherwise we'd use it rather than make it ourselves) 02:12 celeron55 sapier seems to be willing to do that (excluding the library) and he already started, but i have no idea how good it will/would be 02:13 celeron55 and kahrl thinks tcp+udp would work best 02:14 hmmmmm i agree with kahrl here 02:14 hmmmmm if we know what we need to do to make it better, we can just do that with our own code 02:17 celeron55 majority vote goes to tcp+udp then 02:17 celeron55 or, well, some kind of vote 02:18 iqualfragile udp+tcp seems to be the best idea to me, too 02:22 iqualfragile it does increase the work for server administators a bit (forwarding/allowing two ports) but thats allmost nothing 02:23 iqualfragile but out of order reliable delivery would be nice (think enet contains that) for nodes 02:23 celeron55 oh there's also yet another point here: if we continue to use an everything-over-clever-udp approach, then the internal interfaces of minetest stay pretty much the same; if adding tcp to the mix, then stuff there needs to be modified to send things with the correct method (tcp/udp/reliable_udp) and to do stuff suitably for it (i think for block sending there currently is a direct feedback from the low level packet sending queue size ... 02:23 celeron55 ... to the block sending logic, limiting the rate and keeping the queue small so that if a player moves, unnecessary blocks aren't sent from some huge accumulated queue) 02:25 iqualfragile blocks would be tcp/ reliable out of order udp anyways 02:25 iqualfragile (contrary to player movements which should be delivered via udp) 02:26 celeron55 iqualfragile: you're wrong in half of the things you say 02:26 iqualfragile please do correct me 02:26 celeron55 "blocks would be tcp/ reliable out of order udp anyways" <- that doesn't make any sense whatsoever, or at least it doesn't add any information to anything 02:27 celeron55 and tcp and udp can have the same port, leading to no effort from anyone's part 02:27 iqualfragile udp and tcp ports are seperate entities 02:28 hmmmmm block sending is something that I want to rip out and redo anyway 02:28 celeron55 yes but every UI allows you to filter/unfilter both at the same time 02:28 hmmmmm ugh 02:28 iqualfragile not rue 02:28 iqualfragile not true 02:28 hmmmmm who thought that block sending algorithm was good 02:28 iqualfragile like 50% i have seen do not 02:29 celeron55 out of order delivery of large packets would be useful; with TCP we don't get it and it doesn't make sense to implement it for UDP if we use TCP for large data 02:29 hmmmmm basically what I think we should do is see what enet does right and see what we do wrong, fix reliable UDP packets a bit, but don't rely on them and instead use TCP for the real work 02:30 hmmmmm if out of order delivery is so good, why is it that virtually all file transfer protocols are TCP based 02:30 celeron55 because latency doesn't matter in bulk file transfer 02:30 hmmmmm I only see it increasing latency a tiny bit, which again, doesn't really matter for bulk transfers 02:30 celeron55 in a game hiccups like that do matter 02:33 celeron55 it's also ridiculous that internet audio streams have buffers of many seconds just because tcp just can't drop any bit from the data before giving the rest to the application 02:33 celeron55 but anyway, we can live without it 02:34 hmmmmm and udp would need the same in case of dropped packets 02:34 hmmmmm unless you like skipping 05:22 thexyz okay so we're going with tcp+udp now? 06:34 thexyz hm... it seems sapier got a bit upset by the fact i didn't notice his messages about 10 hours ago 06:52 thexyz I think we should at least compare enet performance to such of UDP+TCP+whatever else you guys come up with 06:54 E4xoi i would use udp or tcp, not both 06:54 thexyz code this then! 06:55 thexyz I should already begin the testing 06:56 thexyz for now I only compared to proller's magical improvements using media transfer; with enet it takes about 14.5 seconds to receive the ~10 megabytes cache; with proller's magic fixes it took me more than 2 minutes and then I turned it off because there was no real progress 06:56 thexyz if sapier can fix it with his something something fixes and make it better than enet then I'll eat my^W^W be very surprised 06:57 E4xoi haha 06:57 VanessaE fwow it takes me 30 seconds to pull 35 MB from my media server with httpfetch. 06:57 VanessaE fwiw* 06:58 thexyz yes, I'm not talking just about fetching media, the whole connect from pressing the button to displaying the blocks on screen 06:58 thexyz I should just dd some random big file but that's not the point 06:58 thexyz so, we've decided to do TCP+UDP in the end; have we decided who' 06:58 thexyz who's going to code it? 07:08 thexyz anyway, I'll do a more proper test (with Minetest master and 100 megabytes) later 07:11 VanessaE thexyz: I retested. 25 seconds from "[Connect]" until the world appeared. 26MB for the cache plus whatever amount it took to get the initial bit of world to display. 07:12 VanessaE (I removed MOBF and Simple MOBs since the previous figure, hence the difference in size) 07:13 VanessaE 100% of the media was received via httpfetch. 07:17 VanessaE about 19 seconds on the second login, with the cache already populated 07:18 E4xoi we can't rely 100% on an external httpd, unless you make curl a 'forced' dependency 07:19 VanessaE E4x-dyslex-io: I already tried that argument months ago. 07:19 E4xoi haha 07:20 VanessaE excuse me..E4-dysle-xio ;) 07:20 VanessaE that works better ;) 07:23 thexyz VanessaE: this actually is meaningless; you should test enet and curl-less minetest now to be able to compare those 07:23 VanessaE not really interested, honestly. I just provided the benchmark for the sake of comparison, not competition. 07:24 VanessaE I already know how fast (not) curl-less minetest is... 07:24 VanessaE 6-24 kBytes/sec at most depending on how much grunt the server's CPU can muster. 07:24 E4xoi but enet? 07:32 thexyz fine, you're the one who have always been asking us to fix lags and do something about slow media transfers 07:32 thexyz but when something like this happens you're no longer interested in it 07:33 thexyz those results you gave are pointless because they're true only for your setup 07:33 VanessaE er... 07:33 VanessaE I'm not the only one having problems, I'm just the only one who bothers to complain about it :P 07:37 VanessaE I'm just not much in a "let's just test this on yet *another* server when I've already got enough to do" mood. 07:38 VanessaE I've countered most of the problem with httpfetch, the rest I figured I'd just let happen however you guys plan to, which is how it always happens *anyway* 09:16 xyz|imaginary can someone explain what is m_peer_id used for on the client? 09:26 xyz|imaginary lol 09:26 xyz|imaginary it eats 100% of cpu apparently because it's too fast 09:26 xyz|imaginary without enet receive() in server thread takes about 30 ms every step on localhost 09:27 xyz|imaginary with enet it's 0 10:13 xyz|imaginary celeron55: did you reply to google btw? 10:18 celeron55 xyz|imaginary: not yet 10:18 celeron55 xyz|imaginary: i'm not exactly motivated to do that but i guess i will some day 10:18 xyz|imaginary fine, I think just giving them link to github and proving that they use minetest is fine 10:31 celeron55 for that a file list and something like objdump output of the library file could be useful 10:32 celeron55 closed source is a bitch because it's closed 10:34 proller NO sense to make tcp rakes now! 10:35 proller you can change 5 lines in connection.cpp ang get 10mbps transfer (and backward incompatibility) 10:36 proller if you cant find this 5 lines - DO NOT make tcp shit, it will bad as current 11:15 thexyz celeron55: by the way, builtin lua stuff is not closed source for obvious reasons 11:17 celeron55 thexyz: have you diffed it? 11:17 thexyz no 11:17 thexyz also how do i find the exact revision? 11:18 celeron55 well shit, it's nearly impossible 11:19 celeron55 i'd hope it's exactly 0.4.8 but it's just as likely to not be 16:04 troller mt can send 5-6 mbps with small tune of connection.sh 16:15 thexyz ssshhhhh 16:16 thexyz submit a patch 16:21 troller not compatible and not very stable speed 16:21 troller https://github.com/proller/freeminer/tree/fast 16:21 E4xoi that with the nick troller; sounds like trolling 16:21 E4xoi ;) 16:22 thexyz then there's not very much point in doing it 16:22 thexyz because it neither compatible nor good 16:23 troller if leave 512b packet - it will be ~compatible and ~3mbps 16:24 thexyz if we switch to enet we won't have to tune the current connection code anymore 16:47 thexyz > can someone explain what is m_peer_id used for on the client? 19:44 PilzAdam sapier, why does tab close the menu? 19:44 sapier no idea 19:44 sapier didn't program this on purpose (if it was me to program it) 19:50 ShadowNinja Theylast mentioned that they based their app on 0.4.8. And actually you can compile Lua to bytecode if you want to keep it closed source. 23:40 VanessaE All right that does it 23:40 VanessaE We need to add something to Minetest server that can officially ban the Buildcraft client and its relatives. Somehow. 23:41 VanessaE until such time as it becomes properly compliant, I do NOT want users connecting to my servers with it. 23:41 VanessaE and since we can't fix the compliance short of complaining, hence my suggestion to block it. 23:41 VanessaE if users can't connect to minetest servers, they'll go "Buildcraft is crap!" without knowing it's minetest. 23:42 VanessaE downloads drop off, their project dies. 23:42 kaeza that is hard to do, if not just plain impossible 23:42 VanessaE there must surely be some way 23:43 kaeza see the User-Agent string in browsers; the Buildcraft people will just change it, while it will also bring problems to people using the latest release/previous git commit(s) 23:43 VanessaE true. 23:44 hmmmm it's possible 23:44 hmmmm but I think we'd be violating the GPL 23:44 VanessaE do we have to be interoperable to a project that is itself violating our license? 23:44 hmmmm oh, and it would break ALL development versions 23:45 VanessaE yeah.... 23:45 hmmmm what a lot of game companies do is add integrity validation 23:48 hmmmm hmm! it may be doable 23:49 hmmmm but implementing it would take quite a bit 23:49 hmmmm for every commit to master, we create a new version to challenge response set 23:49 hmmmm and this is centralized of course 23:49 VanessaE would it be worth the effort? 23:49 hmmmm no 23:50 hmmmm so people could enable/disable client validation on their servers of course 23:50 hmmmm if it's enabled, the server asks the central auth server to set up a challenge for this client 23:50 hmmmm it then directs the client to connect to the auth server, request a new validation module, download and execute it 23:50 VanessaE meh 23:50 VanessaE overcomplicated 23:50 hmmmm then it spits back the result 23:51 hmmmm if the check is okay with the claimed version, then the auth server recognizes that client and then notifies the server it was connecting to 23:51 hmmmm which then allows the client to connect 23:51 hmmmm of course this would be trivial if everything were proprietary 23:51 hmmmm with everything completely open source, this gets more complicated 23:51 hmmmm at least I mean if you make the authentication scheme open source 23:52 hmmmm it would involve writing a set of base code, and the algorithm that is actually executed on the client machine is polymorphed at runtime 23:52 kaeza ...or you could just tell every Buildcrap user to fuck off ;) 23:52 VanessaE kaeza: I already do that, though more politely. 23:52 hmmmm and of course there's the entire security issue with that 23:52 kaeza s/Buildcrap user/user mentioning he's using Buildcrap/ 23:53 hmmmm there could also be a central ban service 23:53 hmmmm like MCBans 23:53 VanessaE hmmmm: I was thinking something far simpler. idk what exactly, because anything I could suggest rides dangerously close to (or gets directly into) DRM territory 23:53 VanessaE which is precisely where I refuse to go 23:53 hmmmm lol 23:54 VanessaE code fingerprinting, et al. 23:54 hmmmm there's nothing simpler 23:54 hmmmm no, this would be code fingerprinting in effect 23:54 hmmmm if you build it into the client itself then somebody can just fudge it to return the correct challenge responses every time 23:54 VanessaE right 23:54 VanessaE fundamental problem with any DRM scheme 23:55 VanessaE (and face it, this is DRM) 23:55 hmmmm we'd probably have more success with a global ban list 23:55 VanessaE perhaos 23:55 VanessaE perhaps 23:55 kaeza I think a centralized ban list would be simple enough 23:55 VanessaE and there have been calls for such a thing already 23:55 VanessaE for just regular minetest usage 23:55 hmmmm it doesn't exist for minetest yet..? 23:55 VanessaE no, it does not 23:56 VanessaE should be trivial to make though 23:56 VanessaE we have cURL, so 90% of the work is done already 23:56 VanessaE (but we all know about that last 10%......) 23:56 VanessaE but again, worth the effort? 23:57 kaeza this could be done in a mod... if only Lua could connect to the internet without ugly hacks (/me points to IRC mod) 23:58 kaeza didn't Jeija have a pull request for HTTP access from Lua? 23:58 hmmmm I'm not too crazy about the fact that there is internet connectivity from a mod 23:59 VanessaE kaeza: I believe he does have such a pull, yeah 23:59 VanessaE hmmmm: no worse than we have now, any mod could already access the internet anyway 23:59 VanessaE either via the IRC mod, or os.execute() plus some clever scripting.