Time Nick Message 00:00 rambomedic well if a texture gets reduced from 513 to 512 that's one less packet you have to send... cutting time in half, no? 00:00 rambomedic (cutting time in half for sending that texture) 00:00 sapier roughly yes 00:00 sapier but we think about increasing that packet size too 00:00 kahrl sapier: well, the reason for 512 bytes is RFC 1122 00:01 rambomedic http://tools.ietf.org/html/rfc1122 00:01 rambomedic what specifically in rfc 1122? 00:02 sapier there's not even mentioned the number 512 in this document? 00:02 kahrl 3.3.3 (a) 00:03 kahrl we could increase the size to 556 bytes - sizeof(udp header), but that won't make much difference 00:04 sapier 556 bytes? guess I should read that one a little bit more carefully I thought udp packet size isn't that limited 00:05 iqualfragile strangely important feature request: while downloading media the minetest client should display the currently load things filename, that way people would wait more patiently 00:05 sapier and of course usually routers do fragmentation way better then we can do it 00:05 proller you cant increase size without broken backward compatibility 00:05 sapier I don't think so 00:06 sapier but I have to check this first of course 00:06 kahrl backbone routers usually don't implement fragmentation for performance reasons 00:06 rambomedic If those limitations are part of the standard, why aren't all applications crippled by it? Why is it minetest particularly struggling? 00:07 sapier backbone routers usually don't even use ip ;-) 00:07 sapier and for local subnet (till adsl modem) os should take care about fragmentation 00:09 kahrl also there's the issue of firewalls and NATs 00:09 sapier there's no way to fix this for a application 00:09 kahrl since those are stateful devices and usually cheap some of them don't implement fragmentation properly 00:10 sapier let me have a closer look what rfc1122 is about ... udp defines maximum packet lenght of 64k ... this wouldn't be of any use if there wasn't propper fragmentation handling in os 00:11 kahrl well, if you exactly know your network you can use jumbo frames 00:12 sapier "The Internet model requires that every host support reassembly." actually we could even rely on fragmentation and it wasn't our fault if it didn't work ;-) 00:12 rambomedic If those limitations are part of the standard, why aren't all applications crippled by it? Why is it minetest particularly struggling? 00:13 sapier it's not a hard limit 00:13 sapier it's only a minimum that is required to be supported 00:13 sapier it's been defined prior ethernet was defacto standard 00:15 kahrl there's another particular reason ip fragmentation is worse than minetest's split packets 00:15 kahrl in ip fragmentation, if any of the fragments is lost the whole packet is lost 00:15 sapier present time you usually have only two network times, ethernet ad client/server edges and atm networks for transfer ... but atm is declining 00:15 sapier -times + types 00:16 kahrl in minetest's split packets, any of the fragments can be resent independently (if the reliable flag is turned on) 00:16 sapier kahrl minetests internal connection handling is crap ... of course the packet size is less important point for this 00:18 kahrl sapier: you forgot at least the whole mobile sector 00:18 sapier yes I forgot interplanetary connections too ;-) 00:19 sapier 512 bytes seem to a little bit to low but if it's even possible to change i I'll make it configurable setting 512 as default value 00:19 kahrl I would support using tcp instead of udp for reliable messages, yes 00:19 kahrl that would also immediately take care of the packet size issue 00:20 sapier I'd support that too kahrl but we need to stop servers running havoc on connection of old client 00:20 sapier or decide to break compatibility once and forever 00:20 kahrl how would increasing the packet size help with that? 00:21 sapier not that much ... would only be one point of multiple things that need to be done 00:22 sapier increasing packet size will reduce time spent for sending textures ... at least on non special network conditions 00:23 kahrl I'd say fix whatever causes the server to prioritize sending media over map updates etc., then the packet size will be a non-issue 00:23 sapier there ain't any priorization ;-) 00:24 sapier that's what makes packet size worse 100000 texture packets 1 other in queue ... guess what's handled first ;-) 00:24 sapier don't take the numbers for real ;-) just an example 00:24 kahrl then add proper priorization 00:25 kahrl otherwise the problem will just come back when server owners add more and more mods 00:25 sapier that's what I intended to do but on already rewriting that code I'll have a look where to tweek too ... and 512 byte packet size is ancient ;-) 00:25 kahrl ancient but the only thing you can take for granted 00:26 sapier german telekom finished transition to all ethernet backbone in 2012 00:26 sapier come on kahrl we require 3d graphics cards too guess you won't find anyone using a 14.400 dial up modem for playing minetest 00:26 kahrl (even 556 bytes - sizeof(udp header) might not be enough if someone uses stuff like IPSec) 00:27 sapier and for sure noone running ipsec over dialup modem ;-) 00:27 kahrl so what do you tell someone whose provider happens to use a 576 byte MTU link somewhere (which is acceptable according to the RFCs)? tough luck? 00:29 sapier that one most likely wont be able to download minetest as his os will sent 1390 byte tcp http get requests ;-= 00:30 kahrl well no, because TCP has PMTUD 00:30 kahrl I just don't see the point of violating RFC 1122 when adding priorization would be enough 00:31 sapier I don't see a reason to force server admins to stick to a almost quater centenial old technical specification that isn't of any use 00:32 sapier btw for mobile communications 512 is way to much minimum guaranteed packet size of gsm is exactly 1 byte ;-) 00:32 sapier stop 00:32 sapier 1kbyte 00:33 sapier so for mobile at least 900 would be fine too ;-) 00:34 sapier but I wont change it if you feel so strong for rfc1122 ;-) 00:38 sapier btw kahrl did you read the comment on top of page 60? guess if it was updated today there would be something like 1536 ;-) 00:40 rambomedic are you sure that rfc isn't deprecated by a newer version of the same standard? 00:40 sapier I don't think so it's not a limitation 00:41 sapier why update something telling "you have at least" if you get ten times more in any case? 00:41 kahrl rambomedic: if it was deprecated, it would say at the top of the link you posted 00:42 kahrl it has been updated but if you click on each of those RFC linked, they are about something else 00:42 sapier I don't think that point is worth any more discussion I wont change default size for sure ... if ever I'll decrease it to be adequate for ipv6 too 00:43 sapier wait maybe it's defined different for ipv6 00:43 kahrl 1280 according to rfc 2460 00:45 sapier kahrl if links need to support 1280 for ipv6 and ipv6 is supposed to be supported everywhere (soon(tm)) wouldn't it be safe to assume it for ipv4 too? 00:45 kahrl the (tm) is the problem ;) 00:46 sapier I don't think backbone networks will have issues 00:47 kahrl well I guess that's at least 99% true 00:47 sapier yet I won't change default .... I don't even know if noone didin't use that size as hardcoded length in parsing code ;-) 00:48 kahrl well you can try setting packet size to maybe 1024 if ipv6_server is true 00:48 iqualfragile hmm… looks like minetest would not support ftp for remote_media 00:48 sapier if it's changeable I'm gonna add a configuration option but I'll do the priorization things first 00:49 proller 0.4.8 have hardcoded 512 read buffer 00:49 sapier who did this? 00:49 proller and problem not in packet size 00:50 sapier proller there isn't a hardcoded buffer 00:50 sapier ohhh wait 00:51 proller then increase it!!1 00:51 sapier it's linked to datasize 00:51 sapier argh 00:51 sapier no way 00:51 iqualfragile ftp for remotemedia would be nice 00:51 proller why? 00:51 proller ftp slow old and must die 00:52 proller and use 2 connections 00:52 sapier because we will send data to clients exceeding their buffer size 00:52 proller wow! 00:53 sapier so actually it's not routers to be broken but minetest itself 00:55 sapier "// Double it just to be safe" lol 00:55 sapier so we have at most 1024 00:56 sapier +7 so 1031 00:58 proller no. 00:58 proller recv buffer still 5122 00:58 proller 512 01:01 iqualfragile proller: minetest media consist mainly of small files, the http header might have quite an impact 01:02 Warr1024 zip download could be neat. 01:02 iqualfragile Warr1024: wtf? 01:02 Warr1024 wtf? 01:02 Warr1024 what wtf? 01:02 iqualfragile what would zip download be neat for? 01:02 Warr1024 media 01:03 iqualfragile no 01:03 Warr1024 you were just mentioning the http overhead. 01:03 iqualfragile if you would do anything like that you would use tar 01:03 Warr1024 it makes little real difference. 01:03 iqualfragile but i doubt that ists really nescesary 01:03 Warr1024 both of those formats have a lot of unnecessary stuff in them. 01:03 Warr1024 compression is probably irrelevant in zip, and stuff like file modes and such... 01:04 Warr1024 how much of a problem is media dl really? 01:04 Warr1024 if people would just stick to 16x16 base textures (bigger media in texture packs) it shouldn't be too bad 01:04 iqualfragile for my server its quite a big problem, about 80% of players do not finish downloading 01:04 iqualfragile (and leave) 01:05 Warr1024 how much media do you have? 01:05 iqualfragile 32mb 01:06 Warr1024 whoa daaaaaaaaaaaamn 01:06 Warr1024 yeah, I'd probably bail too :-) 01:06 Warr1024 and I don't think there's much protocol magic you could do to make that not a painful download for many users. 01:07 iqualfragile why? even on slow connections 32 mb should just take about 42 secs 01:08 Warr1024 lol @ your definition of "slow connection" 01:10 iqualfragile 6000kbit/s is a slow connection 01:11 iqualfragile why does it take the minetest client that long to start downloading media from remotemedia? 01:11 iqualfragile it took 3 minutes for it to start looking for media… 01:12 Warr1024 I think when people are talking about slow connections, they're probably not talking about that "first world" kind of slow... 01:12 Warr1024 we're talking like cellular or crowded wifi. 01:12 iqualfragile yes, my server is not for those people 01:13 Warr1024 clearly 01:13 Warr1024 sounds like what you need is more of a "those people" filter. 01:13 Warr1024 that'd get your 80% down 01:14 iqualfragile ok, so when i just comment the remote_media setting out it starts loading using the usual (slow) minetest-internal data connection 01:28 iqualfragile ooh, great there was some change to the way the serving http servers are supposed to act… where is the documentation on how to configure the server? 01:31 iqualfragile kahrl: how to generate index.mth? 01:48 kahrl iqualfragile: it should be compatible with the old way if the request for index.mth returns a 404 01:49 kahrl to generate index.mth: https://gist.github.com/sfan5/6351560 01:50 VanessaE I just fill the dir full of media files, and make sure they're written roughly in the order that the server needs to send them 01:50 VanessaE (e.g. default game stuff first, then mods, then world mods, etc, making sure each general category of files properly overwrites the previous as needed) 01:52 Warr1024 how do you get an entitie's properties in lua? 01:52 kahrl I haven't gotten around to writing proper documentation for the new remote_media fetcher 01:52 kahrl but there is some info in this commit's description: https://github.com/minetest/minetest/commit/0404bbf67196e83d04620180e704916671371ca1 01:53 iqualfragile kahrl: was the option renamed? 01:53 kahrl no 01:53 iqualfragile from remote_media to remote_server? 01:53 VanessaE kahrl: this is how I do it: http://pastebin.ubuntu.com/6593416/ 01:53 iqualfragile your commit mentions remote_server 01:53 VanessaE (though that has been tweaked slightly since, to account for an oversight in Nostalgia game) 01:54 iqualfragile my webserver just returned that post would not be supported 01:54 iqualfragile minetest seemed to be unhappy about that information 01:54 Warr1024 ok, wtf, where's the api for getting an entity's collisionbox? 01:55 kahrl iqualfragile: oops, that's a mistake, should be remote_media 01:56 kahrl iqualfragile: it has to return 404 for index.mth specifically 01:57 VanessaE kahrl: that would explain why it wasn't working for Megaf last night - his server was returning error code 417, of all things, for that file. 01:59 kahrl should it be changed to revert to compatibility mode on response code >= 400? 01:59 VanessaE maybe so, idk how http works well enough to say if that's really safe or not. 01:59 kahrl me either 02:01 iqualfragile hmm… at least 501 needs to be handled 02:01 iqualfragile there are some webservers not implementing post 02:01 iqualfragile not allowed should allso be handled 02:02 VanessaE of course, 02:03 iqualfragile 03:04:13: INFO[CurlFetchThread]: http://mineguild.de:8000/media/index.mth not found (HTTP response code said error) (response code 501) 02:03 VanessaE if the whole idea is "can i download this index file or not?", then sure 02:03 iqualfragile still does not fall back to using the old method 02:03 iqualfragile seems to be the wrong behaviour 02:03 VanessaE code > 400 == revert to non-hashed method. 02:03 VanessaE so yeah 02:03 VanessaE kahrl: in practice, what's the worst that can happen? 02:04 VanessaE you get the file, you get a corrupt file, you get a partial file, or you get nothing at all, right? 02:05 kahrl well if index.mth exists but for some reason the download fails, it will try all filenames which will likely return a 404 for each 02:05 kahrl but if this happens often on cdn.minetest.net, it might overload that server 02:05 kahrl thexyz: any opinion on this? 02:06 iqualfragile there is not cdn.minetest.net, you might referr to cnd.freeminer.com? 02:06 iqualfragile *org 02:06 iqualfragile kahrl: why would that happen 02:06 iqualfragile the official cdn obviously has an index.mth 02:06 kahrl temporary proxy error, or whatever 02:07 iqualfragile hmm… that would not kill cdn.minetest.net 02:07 iqualfragile i will read the list of status codes and write problematic ones here 02:08 kahrl btw, what servers do not implement post? 02:08 kahrl (so I can avoid those) 02:09 ShadowNinja Could you just check code != 200? 02:11 kahrl maybe 02:11 iqualfragile 401* 403* 404 405 407* 410 418* 500* 501 502* 504* 505* 520* 02:11 iqualfragile for * ones you can assume that not even the fallback will work due to greater problems 02:14 kahrl why can't a teapot serve media files? 02:14 iqualfragile because that protocol is for serving tea, not media files 02:15 Warr1024 though it works if you encode your media in tea leaves 02:16 Warr1024 dammit, how do you get an entity's collision box in lua? there's no way there can not be an API for this. 02:19 iqualfragile well, at least for me it works with s/== 404/!= 200/ 02:19 iqualfragile not surprizing 02:19 iqualfragile still, its not the "right" fallback 02:19 Warr1024 holy shit, there is no get_properties 02:19 Warr1024 every setter has a corresponding getter 02:19 Warr1024 but entity properties are write-only. 02:20 Warr1024 just RTFS'ed it. 02:20 iqualfragile as the right behaviour would be using the index.mth but not pushing it 02:20 kahrl pushing? 02:21 Warr1024 you mean requring? 02:21 Warr1024 (only speld beter) 02:22 iqualfragile yes, using push, to push index.mth 02:23 iqualfragile just because that does not work does not mean that not index.mth exists 02:23 kahrl you mean post? 02:23 iqualfragile damn, yes, post 02:23 iqualfragile i need to sleep 02:23 Warr1024 no reason you couldn't follow a failed post with a get. 02:23 kahrl the client reports the list of files it wants, so that the server doesn't have to send its whole index.mth in case that file is huge. because of size restrictions this list is POSTed, not encoded in the URL 02:24 kahrl Warr1024: I didn't expect there to be HTTP servers that don't implement POST 02:24 kahrl it's pretty basic 02:24 Warr1024 kahrl: a lot of servers implement post, but posting to a static file doesn't necessarily work the way you expect. 02:25 iqualfragile yeah, but python2 -m SimpleHTTPServer does not support it :D 02:25 iqualfragile and there would need to be some kind of logic to actually handle the posted file 02:25 iqualfragile it might be smarter to have a cdn setting for which the post part actually happens 02:26 iqualfragile like 02:26 Warr1024 does this thing handle 3xx redirects? 02:26 iqualfragile remote_media = http://mineguild.de:8000/media/ 02:26 iqualfragile cdn = http://cdn.minetest.net/ 02:26 iqualfragile Warr1024: most likely not, curl does not do that on its own 02:27 Warr1024 hm, tricky 02:27 kahrl damn the whole thing is already too complicated 02:28 iqualfragile after rethinking: the only reason for that whole renaming-to-sha1 and index.mth and stuff is allowing cdns to exist 02:28 iqualfragile why not use the "old" mehod for remote_media 02:28 iqualfragile and use the proposed cdn setting for cdns and actually do cnd stuff only for those servers 02:29 kahrl iqualfragile: well, look at VanessaE's server 02:29 iqualfragile simple: she just sets up a vanessacds 02:29 VanessaE naw, better you don't, you'll burn your eys :) 02:29 VanessaE eyes* 02:29 iqualfragile :D 02:30 iqualfragile that index. mth is to advoid shitloads of 404 to slow down the downloading isnt it? 02:30 iqualfragile it does not feel right 02:31 iqualfragile cdns will most likely have like 95% of files that are requested from them 02:31 iqualfragile there is no real reason for an index.mth 02:31 kahrl not if a server uses some obscure texture pack 02:32 kahrl how about we discuss this when thexyz is here? 02:32 iqualfragile jup 02:32 kahrl I don't have any statistics so all I can do is guess 02:35 VanessaE fwiw, my servers, as of yesterday evening anyway, had 100% of the requested files :) 02:36 VanessaE (well my remote media server had 100% of what my minetest servers used anyways) 02:36 iqualfragile obviously but minetestcdn should be for be for every server 02:38 VanessaE I'm not so sure this will be possible, unfortunately, if only because mods will be updated more frequently than the server can be, but it can come close. 02:39 kahrl hopefully mod releases will be posted to mmdb frequently, then the cdn can automagically use mmdb as a source 02:43 VanessaE so,, 02:43 VanessaE can we please disable building of clients without cURL support? :) 02:43 VanessaE so I can wave bye-bye to media_FUBAR? :) 02:45 Warr1024 where's this mmdb anyway? 02:45 VanessaE https://forum.minetest.net/mmdb/ 02:45 Warr1024 ah, thanks. 02:46 Warr1024 is mmdb also an mgdb? 02:50 VanessaE g? 02:50 Warr1024 game 02:50 Warr1024 minetest game database? 02:50 VanessaE oh 02:51 Warr1024 is the whole minetest modstore business just going to be for stuff atop minetest_game? 02:52 VanessaE afaik it should work for any game. 02:52 VanessaE depending on the mod of course. 02:56 iqualfragile Warr1024: as long as the mods themselves have correctly defined their dependencies you can use them in any gamemode (as long as you have their dependencies) 02:56 iqualfragile but no, mmdb is not a game database, not yet 03:43 rambomedic (mmdb should also have texture database) 06:03 celeron55 about the UDP packet size: it was originally larger and i decreased it until it worked for everyone 06:03 celeron55 that's why it's what it is now 06:03 celeron55 it could be a few bytes shorter than it needs to, but it definitely didn't work eg. at 2x the current size 06:04 celeron55 it worked for some people of course, but it's not fair to leave random people out 06:06 celeron55 the main issue causing low transfer speed for large chunks of data now is the non-adaptive RTT-based congestion control 06:06 Warr1024 I guess you could try to do some kind of per-client packet size negotiation, but that time would probably be better spent evaluating a 3rd party network lib... 06:06 Warr1024 it's kind of odd that you'd be having consistent issues with large UDP packets. 06:06 Warr1024 they should fragment pretty gracefully... 06:06 celeron55 Warr1024: no; it would be better spent making a better congestion control algorithm or implementing TCP 06:07 VanessaE celeron55: there is a secondary problem though: the server itself deadlocks at 100% CPU while it is sending media via the old method as well. surely that is limiting the bandwidth a bit :P 06:07 rambomedic ^ Couldn't you determine what making the large UDP packets not work on certain systems, and then doing some sort of check for that system, at which point you'd fall back to the smaller size? 06:07 VanessaE s/the old method/media_FUBAR/ ;) 06:07 Warr1024 I don't see why an NIH solution is necessary for networking... 06:07 celeron55 VanessaE: i hear it was fixed; just send a pull request with proller's fix (assuming the fix is sane) 06:07 VanessaE celeron55: word has it that, in the context of Minetest, proller's fix is not sane... 06:08 VanessaE (but in the context of fm, I guess it is?) 06:10 Warr1024 what's the issue with a TCP implementation? 06:11 VanessaE Warr1024: that a working one doesn't exist? :) 06:11 Warr1024 OpenVPN has a real brain-dead simple TCP mechanism (it's natively UDP) 06:11 VanessaE (for MT) 06:11 Warr1024 they just prefix a 2-byte datagram length, followed by the datagram payload, and mux it all over one TCP channel. 06:11 celeron55 Warr1024: well i'm fine with any non-hacky solution 06:13 Warr1024 I could literally write a proxy that way that lets me tunnel MT UDP traffic over TCP, and get all the benefits of in-order reliable transport (and all the drawbacks of waiting for retries). 06:13 Warr1024 it'd be a start. 06:19 VanessaE celeron55: btw, how goes your idea of reinstating the texture atlas? 06:29 Warr1024 I tried that "limit the number of meshes updated in the foreground" thread thing but I must not have done it right. 06:29 Warr1024 it doesn't seem either faster or smoother 06:29 Warr1024 at least, more than can be accounted for by the fact that the compiler is no longer hogging the CPU. 06:30 thexyz Warr1024: someone has suggested the enet library 06:30 thexyz I'm poking it at the moment by the way 06:37 Warr1024 thexyz: hope it works out well. how's the licensing for it? 06:46 thexyz it's not that hard to find out http://enet.bespin.org/License.html 07:39 celeron55 VanessaE: nowhere i guess 07:39 VanessaE oh, too bad 07:39 VanessaE that plus VBO could rally improve the rendering engine I think 07:39 * VanessaE shrugs 07:46 VanessaE really* 07:58 celeron55 well i hope someone can do it at some point 08:14 celeron55 http://nmap.org/favicon/ 08:15 celeron55 ha - minetest.net is there, and not even at the smallest size 08:17 VanessaE you have WAY too much time on your hands :) 08:18 celeron55 by the way, did anyone now send a DMCA notice to google other than me? 08:18 VanessaE idk, but as I read it, I don't think anyone expects it to do any good 08:18 celeron55 if not, and otherwise too, see this http://irc.minetest.ru/minetest-dev/2013-12-18#i_3501585 08:20 celeron55 i don't really know what could result of it; there are pretty much exactly two things we can do: 1) send some dmca notices, 2) compete with them 08:20 celeron55 i think we should do both; i personally can't really do the latter 08:24 celeron55 except for some very minor things which i will work on at some point 08:34 VanessaE it does raise a question.... 08:35 VanessaE HDX is or was included in that build as some sort of high-end option, but as all I did was post-process someone else's imagery, should I even bother? 10:18 Megaf What does this means? 10:18 Megaf 08:17:54: ERROR[main]: WARNING: updateListener: invalid value 10:19 Megaf I got a lot of this on client side 10:19 Megaf and on server side I got this 10:17:51: ACTION[ServerThread]: Player Megaf moved too fast; resetting position 10:21 sapier last one occurs on huge lag 10:24 Megaf When I place a node, server see sees that almost immediately, no lag at all. But it takes some time to the nodes quantity decrease, so it seems like my download link on client side is bad 10:25 sapier what exactly is "some time"? 10:26 Megaf sapier, about 5 second for each node 10:27 sapier your download link can't be that slow ;-) guess this is effect of map download queuing node update in server 10:28 Megaf That would explain a lot 10:28 Megaf sapier, how can I improve that? 10:28 sapier you can't it's a engine flaw I'm working at 10:28 sapier but as it's a design error there's no quick bugfix 10:29 Megaf good to know you are working on it 10:31 sapier yes but it's hard to test 10:31 Megaf sapier, well, I have tones of free cpu time and some free ram and a server really far from me 10:32 Megaf I'd like to help 10:32 sapier network and especialy network topologies are complex it's not that difficult to do false asumptions there 10:37 sapier and sometimes you have to start from scratch 10:37 Megaf That's the most fun part 10:38 sapier can someone please give the guy who wrote connection.cpp some characters to place comments in her/his code? ;-) 10:38 proller Wooow! now i got 1.5 mbps download from FM 10:39 proller without http 10:39 sapier proller it's useless if you don't provide mergable patches 10:39 Megaf proller, SHARE YOUR CODE 10:39 Megaf in a good way so they can use it 10:39 Megaf well commented code 10:40 proller and 100 nodes per second speed without stuck on ARM cpu 10:40 Megaf sapier, He can just share his code then someone else can use it and make a patch 10:40 proller FM mergeable to MT 10:40 proller it use full mt history 10:40 Megaf proller, I too get good performance with ARM CPU 10:40 proller no. you cant 10:40 proller i have 2 mobs mod enabled 10:40 sapier Megaf it's as good as writing the code on your own as proller doesn't comment what's his intention too ;-) 10:42 proller https://github.com/freeminer/freeminer 10:43 sapier proller I don't have time to reverse engeneer both your code and minetest code 11:17 thexyz sapier: and proller doesn't have time to spoonfeed everyone incapable of reading the commit log 11:20 kahrl it would help if the commit log didn't consist of "berrter verbose" "clean" "small" "working" ... 11:20 kahrl git rebase exists for a reason 11:21 proller rebase after push ? 11:21 kahrl no, in feature branches 11:21 proller i push my branches to test on 2 servers 11:22 proller and i want make game, not cute log 11:22 proller also you can make diff to minetest/master 11:22 thexyz that's true 11:23 kahrl that huge diff is going to help I'm sure 11:23 kahrl well you can make your game but don't tell anyone to read your commit logs 11:23 proller current transfer from ARM : http://paste.org.ru/?hujaap 11:25 proller where i tell to read logs ? 11:25 sapier thexyz I stopped reading prollers commit logs at "oooh" "better" "fixed" ... ;-) 11:27 thexyz sapier: fine 11:27 thexyz then please tell your users to not rage at us for "not providing source" 11:28 sapier I'd like to merge your patches but actually prollers patches feel like he's adding to his private project ... you can do this for freeminer of course 11:29 thexyz well it gets better 11:29 sapier change something here something there and a little bit of there ... and don't tell anyone what this all is good about ... someone couldn't like the change of engine behaviour 11:30 sapier what is "it"? 11:30 thexyz those messages you pointed at were done in a hurry iirc 11:30 thexyz when masterserver report got broken 11:31 sapier in a haurry? he didn't change it for months 11:31 thexyz after httpfetch merge 11:31 sapier always telling "I don't rebase prior merge" 11:31 thexyz you're free to not merge our stuff but please don't hate us ;_; 11:32 sapier if you keep on telling "you're so much better" you have to live about beeing told you're not playing well with minetest community 11:32 sapier "to life with" 11:33 thexyz ah fine i'll ask him to stop telling that then 11:33 thexyz people can make a choice on their own after all 11:33 sapier of course 11:38 sapier and btw thexyz noone hates you, I'm just a little bit annoyed about waste of developer time we need to fix same things multiple times due to split of community 12:24 thexyz a daily reminder 12:24 thexyz "Thank you for using our online DMCA complaint form. You should receive a confirmation email shortly." 12:24 thexyz https://support.google.com/legal/contact/lr_dmca?product=googleplay 12:25 sapier did we get that mail? ;-) 12:25 thexyz "We have received your legal request. We receive many such complaints each day; your message is in our queue, and we'll get to it as quickly as our workload permits. " 12:25 thexyz I've just submitted it 12:27 thexyz I think every Minetest developer should do it 12:37 proller [2-0917000002359] done 13:07 thexyz great 15:06 ShadowNinja How does #675 with proller's patch look? 19:20 VanessaE so, about disallowing compiling without cURL support. Are we gonna just do it or not? 19:20 VanessaE seems to me that's a simple matter of a tweak to some makefile somewhere (or whatever cmake calls it) 19:20 sapier NO 19:20 VanessaE why no? 19:21 sapier it's hack not a fix 19:21 VanessaE I mean client-side, not server-side. 19:21 sapier I'm working at a propper solution but it's gonna be to late for christmas release 19:21 VanessaE define "a proper solution" 19:22 VanessaE (and I do believe, a year ago, I decribed cURL as hacky also and that media_FUBAR should have been fixed then, but as usual I was ignored and told to just follow the leaders) 19:22 VanessaE (but since we have it, I'm using it) 19:23 sapier proper solution means splitting send from receive and add priorization for different packet types 19:24 sapier guess it's gonna be testable in about 1-2 days 19:25 proller btw fm can serve 1.5mbps from arm cpu now ;) 19:26 sapier apache can serve gigabyts of data from an arm cpu ;-P 19:28 VanessaE sapier: sounds good 19:28 VanessaE proller: big deal, my C64 could do that from a 20 MHz 65816. 19:28 sapier hope it's gonna work as good as it sounds ;-) 19:36 thexyz sapier: i've integrated enet into freeminer 19:36 thexyz there's a testing branch available 19:36 thexyz https://github.com/xyzz/freeminer branch `enet` 19:36 thexyz it's still in testing (no ipv6 for example) and i guess i did some things wrong, but it works and you can connect to servers and play 19:37 thexyz there's one server, xyz.is:30000 19:37 sapier is enet compatible to old clients? 19:37 thexyz no of course 19:38 thexyz it's a completely new network stack 19:38 sapier then I guess freeminer will drift away even more from minetest 19:38 Warr1024 unless minetest pulls this in 19:38 Warr1024 which may be so, if FM can prove its value. 19:38 thexyz yes, that's why I'm posting it here 19:39 thexyz it still doesn't work very well so any help would be amazing 19:39 sapier Minetest can only pull this if it's going to 0.5 AND break compatibility completely 19:39 thexyz then we can think on other things to integrate while breaking protcol 19:39 thexyz i.e. key-value messages instead of current format 19:39 thexyz sapier: indeed 19:40 thexyz proller has suggested messagepack which seems a nice idea 19:40 proller or json ;) 19:41 sapier maybe you should start persuading more core devs 19:41 Warr1024 we really don't need more than one serialization format. 19:41 thexyz we won't have more than one 19:41 Warr1024 if JSON serialization can match/beat our existing format, then we might as well switch. 19:41 sapier we have 3 as of shadows merge of json writer 19:42 Warr1024 I guess it would mean that serialization could move to C for performance as well. 19:42 sapier serialize which is 100% feature identical to json writer as well as marshalling 19:42 thexyz messagepack is basically json 19:42 sapier later one is capable to serialize functions too 19:42 thexyz but takes less size 19:42 sapier can you please discuss this in freeminder-dev ? 19:42 thexyz marshal is used for some internal async stuff, isn't it? 19:42 sapier -d 19:43 thexyz I thought this may interest Minetest core devs because the current tcp-over-udp implementation is broken 19:43 PilzAdam what about that BKVL thing? 19:43 thexyz that's another case of NIH 19:43 sapier You don't seem to have any interest in explaining WHY your changes are worth breaking everything 19:43 thexyz I think we should do less NIH 19:43 thexyz sapier: what changes exactly? 19:43 sapier json formspecs json protocol enet 19:44 PilzAdam NIH? 19:44 thexyz I haven't said anything about json formspecs 19:44 Warr1024 PilzAdam: National Institute of Health :-) 19:44 thexyz enet is better just because after that little test of mine (without fully checking the enet and only with one channel, etc) it is faster to load on localhost 19:44 thexyz enet is better because that's a library which has been around for 9 years it seems 19:45 thexyz enet is better because many projects already use it and we won't have to support it 19:45 sapier wasn't there a problem with enet and ipv6? 19:45 PilzAdam http://devopsreactions.tumblr.com/post/36872312242/when-someone-mentions-ipv6 19:46 thexyz sapier: it doesn't support ipv6 but there's a pull request which adds this and proller has volunteered to implement and test it 19:46 thexyz PilzAdam: can you please not post funny reaction images when a discussion is in process? 19:46 sapier so it's not better then current implementation in this case 19:46 thexyz sapier: no, it is better 19:46 thexyz please reread my messages if you still feel like it's not 19:46 thexyz I've given three primary reasons, is that not enough? 19:47 sapier no it isn't if I want to use ipv6 now 19:47 Warr1024 I think the best way to convince people will be with empirical evidence. 19:47 thexyz ipv6 is not that hard to implement and there's a pull request which does that 19:47 Warr1024 try doing some controlled side-by-side comparisons. 19:47 sapier but it's even more untested then current protocol 19:47 thexyz I'm not sure what comparisons to do; and I still feel I fucked something while implementing it so I need someone to review my code and fix bugs 19:47 thexyz sapier: it's been around for 9 years 19:47 thexyz many projects use it 19:48 Warr1024 luckily, it sounds like FM has volunteered to guinea pig this new feature that may potentially benefit MT too :-) 19:48 thexyz I don't like to repeat things but if you insist 19:48 thexyz sapier: what untested is my implementation, not protocol 19:48 VanessaE Warr1024: you mean that may benetif FM (because it won't ever be merged into MT) 19:48 VanessaE benefit* 19:48 thexyz but it really is simple and I feel like any core dev more experienced than me can check it 19:49 Warr1024 VanessaE: maybe, maybe not. some hard evidence may go a long way towards swaying the naysayers. 19:49 thexyz shall we continue with enet or may I switch to the second argument? the key-value protocol change 19:49 sapier while I usually are absolutely positive about using tested implementations ... this doesn't superseed my hate of breaking protocol ... at least if there isn't something much better 19:49 thexyz sapier: I've already mentioned three reasons to use it 19:50 thexyz can you please read my messages once again? 19:50 E4xoi we just need more things for the 0.5.x release 19:51 thexyz it is 1) faster 2) older and used by many projects 3) we will only benefit from removing that pile of code 19:51 VanessaE Warr1024: just going by what I've seen over the year and a half I've been following the project. the bigger and grander the idea, the less likely it is to proceed. 19:51 sapier " enet is better because many projects already use it" --> not a proof 19:51 sapier won't have to support it --> guess as we not have to support irrlicht 19:51 sapier I didn't find your third reason 19:52 Warr1024 VanessaE: well, I guess that's why people occassionally jump ship onto forks and the like. They say "evolve or die" like it's not a choice, but technically it *is* a choice... 19:52 Warr1024 I'd considered making the jump to FM too when I saw how fast they added cool new shit, but there is also value in MT's conservative approach to backward compatibility. 19:53 VanessaE Warr1024: which is why fm was born. 19:53 sapier So keep working on separating freeminer even more from minetest while I try to fix minetest in a compatible way 19:53 VanessaE because some shit that should be in MT just won't ever make it in. 19:53 Warr1024 frankly, I'd like to see BOTH approaches tried (enet in FM and MT sticking with its existing proto for a while longer yet) and just see who wins on the merits. 19:53 thexyz sapier: I don't understand your counter-points 19:53 Warr1024 we could sit here arguing about it forever, or we could work on the code :-) 19:53 thexyz 3) we will only benefit from removing that pile of code 19:54 sapier thexyz COMPATIBILITY 19:54 E4xoi you won't merge it like any other pull request 19:54 thexyz compatibility is good 19:54 VanessaE (while some stuff that should not be, would have gone in had all of fm's changes stayed in mt - e.g. finite water, I still disagree with how that works) 19:54 thexyz but sometimes it has to be broken 19:54 thexyz for the good 19:54 sapier you did not convince mee this is one of those times 19:54 E4xoi you will merge when you have other shit that is awesome so you break the compat and remove all the useless code for supporting shits down to 0.4.4 19:54 thexyz sapier: it is faster 19:54 thexyz we don't have to support it 19:55 thexyz why is that not enough? 19:55 sapier I don't believe later point 19:55 thexyz the later being what exactly? 19:55 sapier "we don't have to support it" 19:55 thexyz about "support"? why do you think we'll have to support it? 19:55 sapier because we have to support all of the things we use 19:55 thexyz many other projects already make use of it so if shit breaks they're more likely to fix it 19:55 thexyz perhaps we have a mixed definition of the "support" word? 19:56 thexyz I mean we won't have to fix bugs in it 19:56 thexyz and "optimize" it 19:56 sapier exactly this is what I don't believe 19:56 thexyz why? 19:56 sapier because we WILL have to fix bugs in it 19:57 thexyz why? 19:57 sapier because there ain't any software in universe not having bugs 19:57 thexyz I will repeat once again 19:57 Warr1024 what will happen if the bugs don't get fixed? 19:57 thexyz there are projects bigger than Minetest that use enet 19:57 thexyz I think Calinou can give some examples of projects using enet? 19:58 sapier JThread was used by a lots of projects too 19:58 thexyz were there any bugs? 19:58 thexyz or were there things you weren't happy with? 19:58 E4xoi jthread didn't have bug but was too damn simple for what 'we' wanted, wasn't that the reason of changing it? 19:58 celeron55 i'm all in for changing minetest to use enet 19:58 thexyz I can't find any other project which uses jthread 19:58 Calinou thexyz, sauerbraten, red eclipse, assaultcube, tesseract 19:59 Calinou (three cube 2 games, 1 game based on cube 1) 19:59 celeron55 really the only reason i didn't make it use enet originally was because i wanted to learn stuff and was just messing around without a goal of making the whole world use it anyway 19:59 Calinou also, I've seen some commercial games use it 19:59 thexyz celeron55: then perhaps you can look at the code? 19:59 sapier I leave this discussion now it's waste of time I'll use for fixing minetest 19:59 Calinou oh, Stunt Rally uses that too 19:59 Calinou but its networking model is P2P-based 19:59 thexyz and tell me where have I fucked up 20:00 sapier celeron55 when did you change mind about compatibility? ;-) 20:00 E4xoi can you connect to 0.3.x servers with 0.4.8? 20:00 kahrl I don't understand, what's the point of adding another library dependency? 20:01 kahrl there already is a widely tested library for reliable communications 20:01 thexyz I've listed three points already, please read the log 20:01 kahrl it is called tcp 20:01 thexyz go on then? 20:01 VanessaE E4xoi: give darkrose enough time and I bet she'll fix that, too :P 20:01 thexyz we might want a mix of reliable and unreliable packets 20:01 E4xoi hah 20:02 kahrl so use one udp socket and one tcp socket? 20:02 thexyz on its FAQ page http://enet.bespin.org/FAQ.html enet says it's better than TCP while giving no reason 20:02 thexyz while giving an obscure one, rather 20:02 celeron55 a plain udp+tcp implementation is very fine to me too (especially if someone as capable as kahrl does it) 20:04 kahrl I might even have some time during the next two weeks or so 20:04 thexyz you can ask for more details on #enet I think 20:04 celeron55 kahrl: you might want to see this before getting into it: http://www.iis.sinica.edu.tw/~swc/pub/tcp_in_games.html 20:05 sapier what about starting a poll if minetest shall break compatibility and switch to enet? 20:05 celeron55 basically the issue with tcp in games is that tcp's design assumes that data transfer is limited by network speed, not by program's limited transfer requirements 20:05 celeron55 which leads to certain kinds of behavior which may or may not be bad for minetest with a combination of tcp and udp 20:05 kahrl true 20:05 celeron55 what it mainly means is that tcp sacrifices latency 20:06 thexyz well, in the end I too am fine with either; the problem is that often things are talked about a lot but never actually done 20:08 sapier fix for major latency issues with current implementation will be ready by christmas ... to late for christmas version but in state for review ... and if it's still necessary to add tcp support this variant will be easy to adopt 20:09 thexyz so we've decided to not use enet, right? 20:09 sapier I don't know we don't have a way to do decisions like that 20:09 kahrl if it's easier for you to implement in freeminer, use enet 20:10 celeron55 atm i would say let's use enet; it can be easily bundled within minetest for cross-platform usage and it pretty surely does what we need well 20:10 VanessaE for G*D sakes 20:11 VanessaE make sure as a server owner I can REJECT connections from protocols I don't want! 20:11 celeron55 wut 20:11 VanessaE if you're gonna add something new that works better, make sure I can reject the old crap, assuming it can still connect 20:11 celeron55 we won't be including support for many protocols 20:11 celeron55 if a new one comes, the old one is removed 20:11 VanessaE ok 20:11 VanessaE good 20:11 sapier VanessaE your servers won't be able to accept connections from stable clients for quite a long time 20:12 VanessaE I'm thinking of media_FUBAR vs. httpfetch :) 20:12 VanessaE sapier: ok, so nothing unusual then? ;) 20:12 celeron55 thexyz: poke me about your enet stuff after a few days if i forget it 20:13 sapier If we decide to use enet then at least majority of core devs need to agree so get that poll started 20:13 VanessaE I am all in for enet if it works well 20:13 sapier celeron55 that's crap either decide or don't 20:13 celeron55 sapier: ? 20:14 celeron55 it's about the particular implementation, not about the idea 20:14 thexyz kahrl: I don't want to break protocol; on the other hand I don't want to have the current UDP implementation either 20:14 thexyz it' 20:14 sapier thexyz is working on enet I'm working on fixing minetest in a compatible way you already loose a lot of work if you delay decision you'll loos much more work 20:14 celeron55 sapier: stop what you're doing then 20:15 thexyz it's still not clear for me what are we going to do: TCP or enet 20:15 sapier so you decide to use enet? 20:15 specing 21:29 < specing> What hash does minetest use for passwords? 20:15 specing 21:29 < specing> It looks kinda ... short 20:15 specing 21:29 <+ShadowNinja> specing: SHA1 salted with the player name 20:15 specing http://arstechnica.com/security/2013/12/bitcoin-only-poker-site-resets-user-credentials-after-42000-passwords-leak/ 20:15 sapier current state is inacceptable it's insane to have a connecting client kill server 20:15 specing 1000 SHA1 cracked in <9 mins 20:16 specing So... bcrypt please? 20:16 celeron55 well ffs, i decide then: we use enet, unless it turns out to be crap while trying to make a clean implementation of it (assuming thexyz's wasn't) 20:16 VanessaE specing: offtopic 20:16 specing Ontopic 20:16 celeron55 in which case we look for alternatives 20:16 thexyz celeron55: I'm not sure; I think you should check the code 20:16 specing minetest uses a weak hashing function for passwords 20:16 celeron55 thexyz: i'm not sure either, but people demand answers 20:16 E4xoi specing: >bitcoin, if you're mining bitcoins you probably could crack 100k SHA1 hashes in less time ;) 20:17 sapier if I stop working on this now I'm not gonna start all over again for sure 20:18 sapier I have time now and understand how minetest meant to do it so doing this in 8 weeks is not an option 20:19 sapier "i decide then: we use enet" ok thexyz do you have a prototype? 20:19 thexyz yes, I've mentioned this multiple times 20:19 sapier where to pull? 20:20 thexyz https://github.com/xyzz/freeminer/compare/enet 20:20 sapier NO 20:20 sapier come one guys can you pleas write a little bit more acceptable comments? 20:20 thexyz that is a prototype 20:21 thexyz I needed the source to be in sync between my home PC and a laptop 20:21 thexyz in the end everything should be squashed into one commit 20:21 thexyz and a proper description written 20:22 sapier when do you think this will be ready? 20:22 thexyz not sure, need someone else to look into it 20:22 VanessaE "something enet something something" 20:22 VanessaE lol 20:23 VanessaE http://xkcd.com/1296/ 20:23 sapier I wont look at it if you don't explain it 20:23 sapier I don't care about the way you explain it 20:23 thexyz you didn't even ask ;_; 20:24 sapier that was meant as asking ;-) 20:24 sapier usually I try to read commit messages for explanation but that's impossible here 20:24 thexyz then what did you ask? 20:24 thexyz what explanation do you expect there? 20:25 sapier what is done and what needs to be done? 20:25 thexyz indeed, I can explain that 20:25 VanessaE "Add experimental support for enet in lieu of the default networking protocol" maybe? 20:25 kahrl hmm, it seems that if a player gets banned while receiving itemdef/nodedef/media, the access denied message is not shown (until all media is received) 20:25 thexyz there's no ban support 20:25 thexyz in my branch 20:25 thexyz but I see, you're not talking about it 20:25 kahrl because that loop in game.cpp doesn't check client.accessDenied() 20:26 kahrl thexyz: yeah, I mean in minetest 20:26 thexyz anyway, what is done: minetest protocol implementation is replaced by enet completely 20:26 sapier thexyz can we create a wiki page to add information? 20:26 thexyz what information do you need? 20:27 sapier ALL ;-) but I guess I'll read what you did by now and see myself what is missing 20:28 kahrl anyone mind me pushing this quick fix? https://gist.github.com/kahrl/8045691 20:28 thexyz I'm not sure what's that ALL you're talking about; anyway, I will do that tomorrow 20:28 kahrl I copied this code straight from the connect loop 20:28 thexyz for now it indeed is easier to read the diff 20:29 sapier L144 is about asigning new peer ids isn't it? 20:30 thexyz right 20:30 thexyz there's also "// TODO: why do we even need to know our peer id?" 20:30 thexyz which has to be fixed too 20:31 thexyz then the other problem is that this (or server?) thread seems to eat 100% of cpu constantly now 20:31 sapier ok first of all do you intend to change paket format too? 20:32 thexyz adding a proper timeout in enet_host_service doesn't help 20:32 thexyz packet format? yes, we were thinking about messagepack 20:32 sapier of xourse you didn't fix the main issue of minetest connection yet 20:32 thexyz what is it? 20:32 sapier receive and send linked within same thread 20:32 thexyz why is that a problem? 20:33 sapier because you are delayed for as much time as you need to send all packets 20:33 thexyz hm, no this is not true 20:33 sapier explain why 20:34 thexyz enet does this in its enet_host_service 20:34 sapier is this a own thread? 20:35 thexyz I don't really know how ENet works; this part of code is right from the official tutorial http://enet.bespin.org/Tutorial.html so I've assumed this is how it should be done 20:36 sapier I don't like the idea using a lib noone know how it's working ... at least basics should be known 20:38 thexyz what are those basics you're talking about? apparently, it both sends and receives when you call enet_host_service 20:38 thexyz but I can't be too sure about this 20:38 Calinou you could ask eihrul or quin, they are the people around Cube 2 stuff 20:38 Calinou they're in #redeclipse 20:39 Calinou (maybe) 20:39 thexyz indeed 20:39 thexyz if you really want to know how it works then just do that 20:42 sapier1 you're great thexyz you keep telling it's better but a single question for details results in a "I don't know" answer 20:42 thexyz yes, I do 20:43 thexyz because I've just tested it with this weird half-working implementation and found it better 20:43 thexyz err, I don't, rather 20:43 thexyz for some reason I feel like you don't want to have a constructive discussion here but want to mock me and make yourself feel smart 20:44 celeron55 i think thexyz is acting just fine; people just assume everything thexyz does is wrong because he's directly supporting proller's freeminer stuff 20:44 sapier1 is it better because it's better or because you haven't run in any of the problems current implementation ties solve? 20:45 sapier1 no I don't think he's doing wrong ... enet MAY be a good way to go 20:45 sapier1 but I'd like to know where are the open issues and how to solve them 20:45 celeron55 kahrl: it seems fine; push it if it works 20:45 sapier1 e.g. 100% cpu usage ... that's exactly SAME issue minetest suffers about SOMETIMES 20:45 kahrl yeah, I tested it (locally) 20:46 kahrl pushing... 20:50 thexyz then it could be fixed in the same way; and since there's now less code in the connection.cpp then it's even easier to fix 20:50 thexyz sorry, have to go to sleep now; highlight me if you want to say something rude again 20:51 sapier yes thexyz everyone is bad to you ... I'll try to continue your work as decision is made I want to get this done as quick as possible 20:53 thexyz no, not everyone; anyway it's good to know you will be working on it, please keep us informed 20:55 celeron55 thexyz: this enet stuff is surprisingly straightforward 20:55 celeron55 i'm surprised it maps so directly to the Connection stuff 20:56 sapier basic features seem to be almost equivalent to current minetest implementation 20:59 celeron55 as it seems sapier is now looking at it, i think i'll too wait for sapier's findings 20:59 sapier first impression is it's way more simple then minetest implementation 21:00 proller current implemerntation designed for singleplayer 21:00 celeron55 it seems there was some mutex things commented out without proper care at least so those need to be thought out... and then the 100% CPU usage 21:01 sapier I'm looking at enet right now not the changes 21:02 celeron55 (and also a possibility would be removing the thread inside Connection, as it was just a way to try to handle some things in the old implementation somehow sanely) 21:03 sapier enet doesn't have it's own thread 21:03 proller now 50+% players come from android with slow networks and loooot of RE-SENDING timed-out RELIABLE 21:04 VanessaE oh and if you put this new protocol in, you're gonna HAVE to prevent servers from showing in the public list that wo't connect to those old clients 21:04 VanessaE so that the rest of us don't have to field 1000 complaints a day about no-connect failures 21:04 sapier not my issue ;-P users will recognize if they can't connect 21:04 VanessaE no they won't 21:04 VanessaE they'll just complain 21:05 VanessaE or "this server sucks" 21:05 sapier they'll complain anyway 21:05 VanessaE if the server isn't there at all, they won't try to connect 21:05 sapier users always complain on total compatibility loss 21:05 proller 90% servers will switch to new proto in ~week 21:05 VanessaE there are 10x as many n00bs coming in as regulars 21:05 sapier 90% of users wont switch for about 2 months 21:05 VanessaE regulars know what to do, n00bs don't 21:05 sapier maybe more 21:06 VanessaE so if the users can't fix it, the servers owners have to 21:06 VanessaE (which relies on features in core of course) 21:06 sapier but it's been decided by celeron compatibility isn't an issue so please forward you complains there ;-P 21:06 E4xoi at the time tou switch to enet, i guess you are removing all the old code, no? 21:06 sapier VanessaE there ain't a fix for this issue 21:06 VanessaE sapier: that's my point - if this breaks compatibility, then it needs to completely break it - as in break it at the master server level also 21:06 E4xoi s/tou/you/ 21:07 sapier of course E4xoi 21:07 E4xoi heck yeah ;) 21:07 VanessaE don't show my server in the master server list if a client that can read from that list cannot connect to it. 21:07 sapier minetest will be 0.5 after this switch 21:10 VanessaE can't the web server hosting the master list filter it according to the features of the client that is reading from it? like if the client fails to tell the server "hey, I can connect to minetest servers that use enet" ? 21:10 VanessaE in other words, if you can't filter based on what the client says it has, filter on what it *doesn't* say 21:11 E4xoi you won't be able to connect to 0.5.x with 0.4.x or viceversa 21:11 sapier everything is possible 21:11 proller now list.JSON ;) is static 21:11 VanessaE E4xoi: it isn't about connecting. it's about them showing up in the list to begin with 21:11 proller we can rename to list0.5 but..but... 21:11 sapier isn't there a ignore command in irc 21:11 VanessaE stop the attempt before it's even initiated 21:12 proller sapier, /ignore 21:12 sapier already tried /ignore proller ... doesn't work 21:12 VanessaE sapier: don't ignore, you never know what your opposition might have to say that may be of benefit. 21:12 sapier if I don't ignore him I'm gonna stop working at minetest soon 21:13 VanessaE sapier: soft-ignore then, like everyone here does with me :P 21:13 sapier I'm a all or nothing guy 21:13 VanessaE I know :) 21:13 sapier I accept a lot of insane comments but once threshold is reached it's over 21:13 VanessaE as anyone who has seen your rants about minetest security can attest to ;) 21:14 proller {"sapier":false} 21:14 sapier can someone tell me name of the guy writing enet? 21:15 VanessaE Lee Salzman 21:15 VanessaE so says the license anyway 21:15 sapier I'd like to know for what reason he duplicated his own code within a loop 21:17 sapier and that guy isn't a friend of commenting his functions too, despite of header comment 21:18 VanessaE you've seen my code, I'm not exactly fond of tons of comments either 21:19 VanessaE too many comments makes the code hard to read, imho 21:19 VanessaE (not enough is just as bad though) 21:19 sapier if they don't match the code you're right vanessae ;-) 21:19 VanessaE haha 21:23 sapier ok we actually need to remove the connection thread to use enet 21:24 sapier enet_host_service has to be called in main loop with all consequences 21:30 sapier celeron55 can you have a look at unix.c from enet? enet_socket_wait really waits for timeout to pass without any way to interrupt? 21:43 sapier Ok for what I see enet has following benefits: 21:43 sapier 1) quite clean code compared to minetest connection 21:43 sapier 2) a more or less proofen peer handling 21:43 sapier 3) fragmentation handling 21:44 sapier But there are issues too: 21:44 sapier 1) lack of priority queue handling 21:45 sapier 2) lack of multithreading support ( telling it doesn't have issues because it all has to be done within a single thread isn't mt support) 21:45 sapier 3) coupled sending/receiving 21:45 sapier 4) lack of Ipv6 support 21:46 sapier point 2 may be fixable by adding an additional layer to create a sender thread per peer 21:46 sapier 1 and 3 are not fixable at all 21:47 sapier at least without rewriting enet ... which is exactly what we don't want to do 21:48 sapier Can someone please tell me I'm wrong (with explanation why) ... because I wont waste my time to do this if it's just postphoning current issues by a couple of additional peers 21:51 celeron55 what does 1 mean 21:53 sapier e.g. we can't add packets to be sent with lower priority then others. Map data textures models for example 21:55 celeron55 enet's channels do that; it prioritizes channels based on their number so that lowest channels' buffers get sent first 21:55 celeron55 at least according to some random text i found 21:56 sapier I'd be glad if this was true I have a closer look at it 21:57 celeron55 i don't know if 3 is an actual problem, and 2 and 4 might not be right now, but they will be in the future and then there needs to be solutions 21:59 sapier 3 is a problem if enet doesn't do propoper send time division for peers 22:00 sapier e.g. if it just sends packets as queued for all peers without any limit 22:00 sapier any per peer limit of course 22:01 sapier actually enet uses send windows so this shouldn't be a big issue 22:01 celeron55 they say it has congestion control 22:02 celeron55 i think it should mostly take care of that 22:02 proller can it support 1000+ connections? 22:02 sapier no 22:02 sapier found some forum postings about issues as of 16 peer 22:02 sapier s 22:03 celeron55 if we want MMO-tier numbers of clients, we need something more sophisticated 22:04 celeron55 there are so many problems to pile up at 1000 connections though that it's not a sane goal 22:04 celeron55 MMOs usually handle only hundreds of players per instance 22:04 celeron55 or less 22:05 sapier true but despite of "we don't have to maintain it" I don't see a killer feature in enet by now 22:05 celeron55 but minetest can't do multiple instances for a single world 22:05 sapier and this assumption has to be proofen 22:06 celeron55 it's clearly a compromise - but so is tcp + dumb utp a compromise too (a perfect solution would be like enet, but without it's limitations) 22:07 sapier I hate people writing c++ in c 22:07 sapier that guy wrote a 785 chars printf 22:11 Calinou I've seen sauer servers run well with 20-25 players, and 2 players + ~50 spectators 22:11 sapier we need to test, noone can tell in advance how enet will behave for typical minetest network communication profile 22:13 celeron55 there are some other libraries that could be looked at: http://opensource.linuxgamepublishing.com/grapple/ http://sourceforge.net/projects/openplay/ 22:13 celeron55 but it's somewhat hard to find about ipv6 or really anything in stuff like this 22:13 sapier ? 22:15 sapier It wasn't my suggestion to replace current marture but buggy network communication mechanism ... evaluating a single protocol takes about 1-2 days and noone writes his findings ... do you really believe we will ever make a decision based uppon facts but feelings? 22:17 celeron55 why are you talking like this 22:17 sapier because you just decided to go with enet and now put 2 additional protocols to game? 22:18 celeron55 did you read what i decided 22:18 sapier no offence but I don't understand 22:18 sapier yes I did everyone could read like she/he wanted ;-) 22:18 celeron55 i decided that enet must be investigated right now, and now that it was looked at and there were many potential issues, i am looking whether other options have similar issues or not 22:19 sapier I for example read as "we use enet if there isn't any unsolvable problem" 22:19 sapier but I thought about reading it "I say use enet to stop the discussion and decide later" too ;-) 22:20 sapier ok channels aren't exactly priority but they can be used this way 22:22 sapier each channel has it's own window. we can't prevent a channel from using it's windows if high prio is maxed out but at least the fixed bandwith for high prio packets is guaranteed 22:23 celeron55 we now know what benefits and problems enet likely has; now we have to decide whether the problems are acceptable; if i understood correctly, you think it should be tested in real usage and a final decision should be based on that 22:24 sapier as of reading code I'm unsure about enet really solving all our issues ... it could do but I assume we need to use it carefully to achieve this 22:24 celeron55 do you think investigating bare tcp+udp is useful? 22:25 sapier I assume droping in enet in a quick way (not beeing carefull) is about 2-3 days of work ... carefully doing it is about a week (full time) 22:25 celeron55 "bare" udp probably means that some of the existing code in minetest has to be used too 22:26 sapier tcp+udp ... difficult to tell but rough estimation about 2 weeks full time too 22:26 sapier and of course for tcp there ain't a way to do a quick check 22:27 sapier and tcp+udp increases firewall complexity (slightly) ;-) 22:27 celeron55 so what things do you think need extra care in enet? 22:27 celeron55 is there a chance that some other similar library would be easier for those parts? 22:27 sapier main reason for assuming minetest core protocol to be broken is huge lag on player downloading textures 22:28 sapier so moving those packets to a low prio sending has to be done some way 22:29 sapier I don't know of any but I'll have a quick look at your other suggestions (not as deep as enet of course) 22:29 celeron55 really all i know about the minetest protocol's performance is that for low usage, it performs fine, and under high traffic, it just completely chokes up 22:29 celeron55 (and doesn't use the available bandwidth) 22:30 proller and it for singleplayer 22:31 proller okay, less 10 players 22:31 celeron55 altough in theory bandwidth usage is an implementation problem - but many people have already solved the same problem for code that they've put into libraries 22:32 sapier there are some really bad decisions within connection ... a little amount can be fixed quick but most things need major rework 22:32 sapier I'd assume fixing minetest internal protocol is about half a week to a week too 22:34 celeron55 enet simply has the highest chance of being very good; i think everyone appreciates it if you (or someone) test whether it really works 22:34 sapier grapple seems a little bit too much for what we need 22:35 sapier but grapple would provide facilities for better passwords and data encryption ;-) 22:35 sapier but I don't see any priorization support in there too 22:36 sapier enet is politicaly best as everyone assumes minetest network code is bad ... it's not as bad as it's reputation 22:37 sapier when did sourceforge stopp to support konqueror? 22:39 sapier openplay is dead 22:39 sapier no need to look at code not beeing changed for a decade 22:40 celeron55 hmm 22:40 sapier grapple is more then 3 years old to so forget about the security things I told they're most likely nonsense by now 22:41 celeron55 how good, compared to enet, do you think you could make minetest's protocol with a reasonable amount of work? 22:41 celeron55 this would be an easy decision if enet didn't have any issues, but it does 22:42 celeron55 i don't want to base this on politics; it's never good 22:43 celeron55 what does it require? 22:43 sapier hard to tell you can make it even better then enet of course ... but you need to put at least half a week to a week in it to have a real chance to compete 22:44 sapier our protocol supports channels too. But I haven't seen anyone use a different channel then 0 22:44 celeron55 i want to have a decision tonight because talking about this is only one step in actually getting something done 22:46 celeron55 there's actually a benefit in using enet other than that it could possibly almost "just work" 22:47 celeron55 it's at least that it would make clients other than minetest's official one easier to make, eg. for testing or bots or whatever; increased interoperability anyway 22:47 sapier as far as I know we don't have send windows in our implementation 22:47 sapier but I didn't get to that point by now 22:48 celeron55 i'd imagine one thing you would do is replace some of the current crap with a congestion window 22:48 sapier I don't think so celeron main effort is still minetest packet format which wouldn't change due to enet 22:48 sapier only peer handling is changed 22:49 celeron55 sapier: it would remove one link from the chain, and a very dirty link it is 22:49 sapier yes I have about 60% completed split of send and receive by now, cleaned up peer handling and started looking at channel usage 22:50 sapier that's true the outer envelope would be enet instead of minetest 22:51 celeron55 it's especially relevant because the inner formats are likely to be changed some day too 22:51 celeron55 to something less obscure 22:52 sapier I hope you don't think about something pseudo less obscure like json 22:52 sapier having only 512 bytes json will fragment almost any packet 22:53 celeron55 probably messagepack; nothing else really makes sense 22:53 celeron55 (with generally using integer keys to lessen the bloat) 22:55 celeron55 the current formats do make some sense too though, as they're almost maximally size-efficient 22:55 sapier you know enet is a one man show too? 22:56 celeron55 if you want something that isn't, it's probably going to be commercial 22:57 sapier multi person projects tend to be more stable ... maybe minetest isn't best example ;-) 22:57 celeron55 what if we make a library out of our own; then there exists a non-commercial enet that isn't a one-man show 22:57 celeron55 *enet-like 22:58 celeron55 i'd like to hear thexyz's and kahrl's views on this 22:58 sapier me too 22:58 celeron55 but it's going to make this discussion take longer 22:58 sapier enet isn't bad for sure 22:58 celeron55 i guess we just need a break; let's continue tomorrow 22:59 sapier it's prooven code but we need to understand how to use it correctly if we don't we most likely end up in same problems as now 23:00 sapier yes hope thexyz and kahrl can add additional perspectives we missed by now 23:30 * VanessaE is back (yay.) 23:30 kaeza can someone take a look at #1060? 23:30 ShadowBot https://github.com/minetest/minetest/issues/1060