Minetest logo

IRC log for #minetest-dev, 2013-12-19

| Channels | #minetest-dev index | Today | | Google Search | Plaintext

All times shown according to UTC.

Time Nick Message
00:00 rambomedic well if a texture gets reduced from 513 to 512 that's one less packet you have to send... cutting time in half, no?
00:00 rambomedic (cutting time in half for sending that texture)
00:00 sapier roughly yes
00:00 sapier but we think about increasing that packet size too
00:00 kahrl sapier: well, the reason for 512 bytes is RFC 1122
00:01 rambomedic http://tools.ietf.org/html/rfc1122
00:01 rambomedic what specifically in rfc 1122?
00:02 sapier there's not even mentioned the number 512 in this document?
00:02 kahrl 3.3.3 (a)
00:03 kahrl we could increase the size to 556 bytes - sizeof(udp header), but that won't make much difference
00:04 sapier 556 bytes? guess I should read that one a little bit more carefully I thought udp packet size isn't that limited
00:05 iqualfragile strangely important feature request: while downloading media the minetest client should display the currently load things filename, that way people would wait more patiently
00:05 sapier and of course usually routers do fragmentation way better then we can do it
00:05 proller you cant increase size without broken backward compatibility
00:05 sapier I don't think so
00:06 sapier but I have to check this first of course
00:06 kahrl backbone routers usually don't implement fragmentation for performance reasons
00:06 rambomedic If those limitations are part of the standard, why aren't all applications crippled by it? Why is it minetest particularly struggling?
00:07 sapier backbone routers usually don't even use ip ;-)
00:07 sapier and for local subnet (till adsl modem) os should take care about fragmentation
00:09 kahrl also there's the issue of firewalls and NATs
00:09 sapier there's no way to fix this for a application
00:09 kahrl since those are stateful devices and usually cheap some of them don't implement fragmentation properly
00:10 sapier let me have a closer look what rfc1122 is about ... udp defines maximum packet lenght of 64k ... this wouldn't be of any use if there wasn't propper fragmentation handling in os
00:11 kahrl well, if you exactly know your network you can use jumbo frames
00:12 sapier "The Internet model requires that every host support reassembly." actually we could even rely on fragmentation and it wasn't our fault if it didn't work ;-)
00:12 rambomedic If those limitations are part of the standard, why aren't all applications crippled by it? Why is it minetest particularly struggling?
00:13 sapier it's not a hard limit
00:13 sapier it's only a minimum that is required to be supported
00:13 sapier it's been defined prior ethernet was defacto standard
00:15 kahrl there's another particular reason ip fragmentation is worse than minetest's split packets
00:15 kahrl in ip fragmentation, if any of the fragments is lost the whole packet is lost
00:15 sapier present time you usually have only two network times, ethernet ad client/server edges and atm networks for transfer ... but atm is declining
00:15 sapier -times + types
00:16 kahrl in minetest's split packets, any of the fragments can be resent independently (if the reliable flag is turned on)
00:16 sapier kahrl minetests internal connection handling is crap ... of course the packet size is less important point for this
00:18 kahrl sapier: you forgot at least the whole mobile sector
00:18 sapier yes I forgot interplanetary connections too ;-)
00:19 sapier 512 bytes seem to a little bit to low but if it's even possible to change i I'll make it configurable setting 512 as default value
00:19 kahrl I would support using tcp instead of udp for reliable messages, yes
00:19 kahrl that would also immediately take care of the packet size issue
00:20 sapier I'd support that too kahrl but we need to stop servers running havoc on connection of old client
00:20 sapier or decide to break compatibility once and forever
00:20 kahrl how would increasing the packet size help with that?
00:21 sapier not that much ... would only be one point of multiple things that need to be done
00:22 sapier increasing packet size will reduce time spent for sending textures ... at least on non special network conditions
00:23 kahrl I'd say fix whatever causes the server to prioritize sending media over map updates etc., then the packet size will be a non-issue
00:23 us{0gb joined #minetest-dev
00:23 sapier there ain't any priorization ;-)
00:24 sapier that's what makes packet size worse 100000 texture packets 1 other in queue ... guess what's handled first ;-)
00:24 sapier don't take the numbers for real ;-) just an example
00:24 kahrl then add proper priorization
00:25 kahrl otherwise the problem will just come back when server owners add more and more mods
00:25 sapier that's what I intended to do but on already rewriting that code I'll have a look where to tweek too ... and 512 byte packet size is ancient ;-)
00:25 kahrl ancient but the only thing you can take for granted
00:26 sapier german telekom finished transition to all ethernet backbone in 2012
00:26 sapier come on kahrl we require 3d graphics cards too guess you won't find anyone using a 14.400 dial up modem for playing minetest
00:26 kahrl (even 556 bytes - sizeof(udp header) might not be enough if someone uses stuff like IPSec)
00:27 sapier and for sure noone running ipsec over dialup modem ;-)
00:27 kahrl so what do you tell someone whose provider happens to use a 576 byte MTU link somewhere (which is acceptable according to the RFCs)? tough luck?
00:29 sapier that one most likely wont be able to download minetest as his os will sent 1390 byte tcp http get requests ;-=
00:30 kahrl well no, because TCP has PMTUD
00:30 kahrl I just don't see the point of violating RFC 1122 when adding priorization would be enough
00:31 sapier I don't see a reason to force server admins to stick to a almost quater centenial old technical specification that isn't of any use
00:32 sapier btw for mobile communications 512 is way to much minimum guaranteed packet size of gsm is exactly 1 byte ;-)
00:32 sapier stop
00:32 sapier 1kbyte
00:32 OldCoder joined #minetest-dev
00:33 sapier so for mobile at least 900 would be fine too ;-)
00:34 sapier but I wont change it if you feel so strong for rfc1122 ;-)
00:38 sapier btw kahrl did you read the comment on top of page 60? guess if it was updated today there would be something like 1536 ;-)
00:40 werwerwer_ joined #minetest-dev
00:40 rambomedic are you sure that rfc isn't deprecated by a newer version of the same standard?
00:40 sapier I don't think so it's not a limitation
00:41 sapier why update something telling "you have at least" if you get ten times more in any case?
00:41 kahrl rambomedic: if it was deprecated, it would say at the top of the link you posted
00:42 kahrl it has been updated but if you click on each of those RFC linked, they are about something else
00:42 sapier I don't think that point is worth any more discussion I wont change default size for sure ... if ever I'll decrease it to be adequate for ipv6 too
00:43 sapier wait maybe it's defined different for ipv6
00:43 kahrl 1280 according to rfc 2460
00:45 sapier kahrl if links need to support 1280 for ipv6 and ipv6 is supposed to be supported everywhere (soon(tm)) wouldn't it be safe to assume it for ipv4 too?
00:45 kahrl the (tm) is the problem ;)
00:46 sapier I don't think backbone networks will have issues
00:47 kahrl well I guess that's at least 99% true
00:47 sapier yet I won't change default .... I don't even know if noone didin't use that size as hardcoded length in parsing code ;-)
00:48 kahrl well you can try setting packet size to maybe 1024 if ipv6_server is true
00:48 iqualfragile hmm… looks like minetest would not support ftp for remote_media
00:48 sapier if it's changeable I'm gonna add a configuration option but I'll do the priorization things first
00:49 proller 0.4.8 have hardcoded 512 read buffer
00:49 sapier who did this?
00:49 proller and problem not in packet size
00:50 sapier proller there isn't a hardcoded buffer
00:50 sapier ohhh wait
00:51 proller then increase it!!1
00:51 sapier it's linked to datasize
00:51 sapier argh
00:51 sapier no way
00:51 iqualfragile ftp for remotemedia would be nice
00:51 proller why?
00:51 proller ftp slow old and must die
00:52 proller and use 2 connections
00:52 sapier because we will send data to clients exceeding their buffer size
00:52 proller wow!
00:53 sapier so actually it's not routers to be broken but minetest itself
00:55 sapier "// Double it just to be safe" lol
00:55 sapier so we have at most 1024
00:56 sapier +7 so 1031
00:56 Warr1024 joined #minetest-dev
00:57 sapier left #minetest-dev
00:58 proller no.
00:58 proller recv buffer still  5122
00:58 proller 512
01:01 iqualfragile proller: minetest media consist mainly of small files, the http header might have quite an impact
01:02 Warr1024 zip download could be neat.
01:02 iqualfragile Warr1024: wtf?
01:02 Warr1024 wtf?
01:02 Warr1024 what wtf?
01:02 iqualfragile what would zip download be neat for?
01:02 Warr1024 media
01:03 iqualfragile no
01:03 Warr1024 you were just mentioning the http overhead.
01:03 iqualfragile if you would do anything like that you would use tar
01:03 Warr1024 it makes little real difference.
01:03 iqualfragile but i doubt that ists really nescesary
01:03 Warr1024 both of those formats have a lot of unnecessary stuff in them.
01:03 Warr1024 compression is probably irrelevant in zip, and stuff like file modes and such...
01:04 Warr1024 how much of a problem is media dl really?
01:04 Warr1024 if people would just stick to 16x16 base textures (bigger media in texture packs) it shouldn't be too bad
01:04 iqualfragile for my server its quite a big problem, about 80% of players do not finish downloading
01:04 iqualfragile (and leave)
01:05 Warr1024 how much media do you have?
01:05 iqualfragile 32mb
01:06 Warr1024 whoa daaaaaaaaaaaamn
01:06 Warr1024 yeah, I'd probably bail too :-)
01:06 Warr1024 and I don't think there's much protocol magic you could do to make that not a painful download for many users.
01:07 iqualfragile why? even on slow connections 32 mb should just take about 42 secs
01:08 Warr1024 lol @ your definition of "slow connection"
01:10 iqualfragile 6000kbit/s is a slow connection
01:11 iqualfragile why does it take the minetest client that long to start downloading media from remotemedia?
01:11 iqualfragile it took 3 minutes for it to start looking for media…
01:12 Warr1024 I think when people are talking about slow connections, they're probably not talking about that "first world" kind of slow...
01:12 Warr1024 we're talking like cellular or crowded wifi.
01:12 iqualfragile yes, my server is not for those people
01:13 Warr1024 clearly
01:13 Warr1024 sounds like what you need is more of a "those people" filter.
01:13 Warr1024 that'd get your 80% down
01:14 iqualfragile ok, so when i just comment the remote_media setting out it starts loading using the usual (slow) minetest-internal data connection
01:28 iqualfragile ooh, great there was some change to the way the serving http servers are supposed to act… where is the documentation on how to configure the server?
01:31 iqualfragile kahrl: how to generate index.mth?
01:48 kahrl iqualfragile: it should be compatible with the old way if the request for index.mth returns a 404
01:49 kahrl to generate index.mth: https://gist.github.com/sfan5/6351560
01:50 VanessaE I just fill the dir full of media files, and make sure they're written roughly in the order that the server needs to send them
01:50 VanessaE (e.g. default game stuff first, then mods, then world mods, etc, making sure each general category of files properly overwrites the previous as needed)
01:52 Warr1024 how do you get an entitie's properties in lua?
01:52 kahrl I haven't gotten around to writing proper documentation for the new remote_media fetcher
01:52 kahrl but there is some info in this commit's description: https://github.com/minetest/minetest/commi​t/0404bbf67196e83d04620180e704916671371ca1
01:53 iqualfragile kahrl: was the option renamed?
01:53 kahrl no
01:53 iqualfragile from remote_media to remote_server?
01:53 VanessaE kahrl: this is how I do it:  http://pastebin.ubuntu.com/6593416/
01:53 iqualfragile your commit mentions remote_server
01:53 VanessaE (though that has been tweaked slightly since, to account for an oversight in Nostalgia game)
01:54 iqualfragile my webserver just returned that post would not be supported
01:54 iqualfragile minetest seemed to be unhappy about that information
01:54 Warr1024 ok, wtf, where's the api for getting an entity's collisionbox?
01:55 kahrl iqualfragile: oops, that's a mistake, should be remote_media
01:56 kahrl iqualfragile: it has to return 404 for index.mth specifically
01:57 VanessaE kahrl: that would explain why it wasn't working for Megaf last night - his server was returning error code 417, of all things, for that file.
01:59 kahrl should it be changed to revert to compatibility mode on response code >= 400?
01:59 VanessaE maybe so, idk how http works well enough to say if that's really safe or not.
01:59 kahrl me either
02:01 iqualfragile hmm… at least 501 needs to be handled
02:01 iqualfragile there are some webservers not implementing post
02:01 iqualfragile not allowed should allso be handled
02:02 VanessaE of course,
02:03 iqualfragile 03:04:13: INFO[CurlFetchThread]: http://mineguild.de:8000/media/index.mth not found (HTTP response code said error) (response code 501)
02:03 VanessaE if the whole idea is "can i download this index file or not?", then sure
02:03 iqualfragile still does not fall back to using the old method
02:03 iqualfragile seems to be the wrong behaviour
02:03 VanessaE code > 400 == revert to non-hashed method.
02:03 VanessaE so yeah
02:03 VanessaE kahrl: in practice, what's the worst that can happen?
02:04 VanessaE you get the file, you get a corrupt file, you get a partial file, or you get nothing at all, right?
02:05 kahrl well if index.mth exists but for some reason the download fails, it will try all filenames which will likely return a 404 for each
02:05 kahrl but if this happens often on cdn.minetest.net, it might overload that server
02:05 kahrl thexyz: any opinion on this?
02:06 iqualfragile there is not cdn.minetest.net, you might referr to cnd.freeminer.com?
02:06 iqualfragile *org
02:06 iqualfragile kahrl: why would that happen
02:06 iqualfragile the official cdn obviously has an index.mth
02:06 kahrl temporary proxy error, or whatever
02:07 iqualfragile hmm… that would not kill cdn.minetest.net
02:07 iqualfragile i will read the list of status codes and write problematic ones here
02:08 kahrl btw, what servers do not implement post?
02:08 kahrl (so I can avoid those)
02:09 ShadowNinja Could you just check code != 200?
02:11 kahrl maybe
02:11 iqualfragile 401* 403* 404 405 407* 410 418* 500* 501 502* 504* 505* 520*
02:11 iqualfragile for * ones you can assume that not even the fallback will work due to greater problems
02:14 kahrl why can't a teapot serve media files?
02:14 iqualfragile because that protocol is for serving tea, not media files
02:15 Warr1024 though it works if you encode your media in tea leaves
02:16 Warr1024 dammit, how do you get an entity's collision box in lua?  there's no way there can not be an API for this.
02:19 iqualfragile well, at least for me it works with s/== 404/!= 200/
02:19 iqualfragile not surprizing
02:19 iqualfragile still, its not the "right" fallback
02:19 Warr1024 holy shit, there is no get_properties
02:19 Warr1024 every setter has a corresponding getter
02:19 Warr1024 but entity properties are write-only.
02:20 Warr1024 just RTFS'ed it.
02:20 iqualfragile as the right behaviour would be using the index.mth but not pushing it
02:20 kahrl pushing?
02:21 Warr1024 you mean requring?
02:21 Warr1024 (only speld beter)
02:22 iqualfragile yes, using push, to push index.mth
02:23 iqualfragile just because that does not work does not mean that not index.mth exists
02:23 kahrl you mean post?
02:23 iqualfragile damn, yes, post
02:23 iqualfragile i need to sleep
02:23 Warr1024 no reason you couldn't follow a failed post with a get.
02:23 kahrl the client reports the list of files it wants, so that the server doesn't have to send its whole index.mth in case that file is huge. because of size restrictions this list is POSTed, not encoded in the URL
02:24 kahrl Warr1024: I didn't expect there to be HTTP servers that don't implement POST
02:24 kahrl it's pretty basic
02:24 Warr1024 kahrl: a lot of servers implement post, but posting to a static file doesn't necessarily work the way you expect.
02:25 iqualfragile yeah, but python2 -m SimpleHTTPServer does not support it :D
02:25 iqualfragile and there would need to be some kind of logic to actually handle the posted file
02:25 iqualfragile it might be smarter to have a cdn setting for which the post part actually happens
02:26 iqualfragile like
02:26 Warr1024 does this thing handle 3xx redirects?
02:26 iqualfragile remote_media = http://mineguild.de:8000/media/
02:26 iqualfragile cdn = http://cdn.minetest.net/
02:26 iqualfragile Warr1024: most likely not, curl does not do that on its own
02:27 Warr1024 hm, tricky
02:27 kahrl damn the whole thing is already too complicated
02:28 iqualfragile after rethinking: the only reason for that whole renaming-to-sha1 and index.mth and stuff is allowing cdns to exist
02:28 iqualfragile why not use the "old" mehod for remote_media
02:28 iqualfragile and use the proposed cdn setting for cdns and actually do cnd stuff only for those servers
02:29 kahrl iqualfragile: well, look at VanessaE's server
02:29 iqualfragile simple: she just sets up a vanessacds
02:29 VanessaE naw, better you don't, you'll burn your eys :)
02:29 VanessaE eyes*
02:29 iqualfragile :D
02:30 iqualfragile that index. mth is to advoid shitloads of 404 to slow down the downloading isnt it?
02:30 iqualfragile it does not feel right
02:31 iqualfragile cdns will most likely have like 95% of files that are requested from them
02:31 iqualfragile there is no real reason for an index.mth
02:31 kahrl not if a server uses some obscure texture pack
02:32 kahrl how about we discuss this when thexyz is here?
02:32 iqualfragile jup
02:32 kahrl I don't have any statistics so all I can do is guess
02:32 djdduty joined #minetest-dev
02:32 djdduty joined #minetest-dev
02:35 VanessaE fwiw, my servers, as of yesterday evening anyway, had 100% of the requested files :)
02:36 VanessaE (well my remote media server had 100% of what my minetest servers used anyways)
02:36 iqualfragile obviously but minetestcdn should be for be for every server
02:38 VanessaE I'm not so sure this will be possible, unfortunately, if only because mods will be updated more frequently than the server can be, but it can come close.
02:39 kahrl hopefully mod releases will be posted to mmdb frequently, then the cdn can automagically use mmdb as a source
02:43 VanessaE so,,
02:43 VanessaE can we please disable building of clients without cURL support? :)
02:43 VanessaE so I can wave bye-bye to media_FUBAR? :)
02:45 Warr1024 where's this mmdb anyway?
02:45 VanessaE https://forum.minetest.net/mmdb/
02:45 Warr1024 ah, thanks.
02:46 Warr1024 is mmdb also an mgdb?
02:50 VanessaE g?
02:50 Warr1024 game
02:50 Warr1024 minetest game database?
02:50 VanessaE oh
02:51 Warr1024 is the whole minetest modstore business just going to be for stuff atop minetest_game?
02:52 VanessaE afaik it should work for any game.
02:52 VanessaE depending on the mod of course.
02:56 iqualfragile Warr1024: as long as the mods themselves have correctly defined their dependencies you can use them in any gamemode (as long as you have their dependencies)
02:56 iqualfragile but no, mmdb is not a game database, not yet
03:43 rambomedic (mmdb should also have texture database)
04:46 Weedy joined #minetest-dev
05:36 kaeza joined #minetest-dev
05:47 NakedFury joined #minetest-dev
05:58 Miner_48er joined #minetest-dev
06:03 celeron55 about the UDP packet size: it was originally larger and i decreased it until it worked for everyone
06:03 celeron55 that's why it's what it is now
06:03 celeron55 it could be a few bytes shorter than it needs to, but it definitely didn't work eg. at 2x the current size
06:04 celeron55 it worked for some people of course, but it's not fair to leave random people out
06:06 celeron55 the main issue causing low transfer speed for large chunks of data now is the non-adaptive RTT-based congestion control
06:06 Warr1024 I guess you could try to do some kind of per-client packet size negotiation, but that time would probably be better spent evaluating a 3rd party network lib...
06:06 Warr1024 it's kind of odd that you'd be having consistent issues with large UDP packets.
06:06 Warr1024 they should fragment pretty gracefully...
06:06 celeron55 Warr1024: no; it would be better spent making a better congestion control algorithm or implementing TCP
06:07 VanessaE celeron55: there is a secondary problem though:  the server itself deadlocks at 100% CPU while it is sending media via the old method as well.  surely that is limiting the bandwidth a bit :P
06:07 rambomedic ^ Couldn't you determine what making the large UDP packets not work on certain systems, and then doing some sort of check for that system, at which point you'd fall back to the smaller size?
06:07 VanessaE s/the old method/media_FUBAR/  ;)
06:07 Warr1024 I don't see why an NIH solution is necessary for networking...
06:07 celeron55 VanessaE: i hear it was fixed; just send a pull request with proller's fix (assuming the fix is sane)
06:07 VanessaE celeron55: word has it that, in the context of Minetest, proller's fix is not sane...
06:08 VanessaE (but in the context of fm, I guess it is?)
06:10 Warr1024 what's the issue with a TCP implementation?
06:11 VanessaE Warr1024:  that a working one doesn't exist? :)
06:11 Warr1024 OpenVPN has a real brain-dead simple TCP mechanism (it's natively UDP)
06:11 VanessaE (for MT)
06:11 Warr1024 they just prefix a 2-byte datagram length, followed by the datagram payload, and mux it all over one TCP channel.
06:11 celeron55 Warr1024: well i'm fine with any non-hacky solution
06:13 Warr1024 I could literally write a proxy that way that lets me tunnel MT UDP traffic over TCP, and get all the benefits of in-order reliable transport (and all the drawbacks of waiting for retries).
06:13 Warr1024 it'd be a start.
06:19 VanessaE celeron55: btw, how goes your idea of reinstating the texture atlas?
06:29 Warr1024 I tried that "limit the number of meshes updated in the foreground" thread thing but I must not have done it right.
06:29 Warr1024 it doesn't seem either faster or smoother
06:29 Warr1024 at least, more than can be accounted for by the fact that the compiler is no longer hogging the CPU.
06:30 thexyz Warr1024: someone has suggested the enet library
06:30 thexyz I'm poking it at the moment by the way
06:37 Warr1024 thexyz: hope it works out well.  how's the licensing for it?
06:46 thexyz it's not that hard to find out http://enet.bespin.org/License.html
07:37 OldCoder joined #minetest-dev
07:39 celeron55 VanessaE: nowhere i guess
07:39 VanessaE oh, too bad
07:39 VanessaE that plus VBO could rally improve the rendering engine I think
07:39 * VanessaE shrugs
07:46 VanessaE really*
07:58 celeron55 well i hope someone can do it at some point
08:14 celeron55 http://nmap.org/favicon/
08:15 celeron55 ha - minetest.net is there, and not even at the smallest size
08:17 VanessaE you have WAY too much time on your hands :)
08:18 celeron55 by the way, did anyone now send a DMCA notice to google other than me?
08:18 VanessaE idk, but as I read it, I don't think anyone expects it to do any good
08:18 celeron55 if not, and otherwise too, see this http://irc.minetest.ru/minet​est-dev/2013-12-18#i_3501585
08:20 celeron55 i don't really know what could result of it; there are pretty much exactly two things we can do: 1) send some dmca notices, 2) compete with them
08:20 celeron55 i think we should do both; i personally can't really do the latter
08:24 celeron55 except for some very minor things which i will work on at some point
08:34 VanessaE it does raise a question....
08:35 VanessaE HDX is or was included in that build as some sort of high-end option, but as all I did was post-process someone else's imagery, should I even bother?
09:00 Megaf joined #minetest-dev
09:09 sapier joined #minetest-dev
09:25 us}0gb joined #minetest-dev
09:42 proller joined #minetest-dev
10:18 Megaf What does this means?
10:18 Megaf 08:17:54: ERROR[main]: WARNING: updateListener: invalid value
10:19 Megaf I got a lot of this on client side
10:19 Megaf and on server side I got this 10:17:51: ACTION[ServerThread]: Player Megaf moved too fast; resetting position
10:20 jin_xi joined #minetest-dev
10:21 sapier last one occurs on huge lag
10:24 Megaf When I place a node, server see sees that almost immediately, no lag at all. But it takes some time to the nodes quantity decrease, so it seems like my download link on client side is bad
10:25 sapier what exactly is "some time"?
10:26 Megaf sapier, about 5 second for each node
10:27 sapier your download link can't be that slow ;-) guess this is effect of map download queuing node update in server
10:28 Megaf That would explain a lot
10:28 Megaf sapier, how can I improve that?
10:28 sapier you can't it's a engine flaw I'm working at
10:28 sapier but as it's a design error there's no quick bugfix
10:29 Megaf good to know you are working on it
10:31 sapier yes but it's hard to test
10:31 Megaf sapier, well, I have tones of free cpu time and some free ram and a server really far from me
10:32 Megaf I'd like to help
10:32 sapier network and especialy network topologies are complex it's not that difficult to do false asumptions there
10:37 sapier and sometimes you have to start from scratch
10:37 Megaf That's the most fun part
10:38 sapier can someone please give the guy who wrote connection.cpp some characters to place comments in her/his code? ;-)
10:38 proller Wooow! now i got 1.5 mbps download from FM
10:39 proller without http
10:39 sapier proller it's useless if you don't provide mergable patches
10:39 Megaf proller, SHARE YOUR CODE
10:39 Megaf in a good way so they can use it
10:39 Megaf well commented code
10:40 proller and 100 nodes per second speed without stuck on ARM cpu
10:40 Megaf sapier, He can just share his code then someone else can use it and make a patch
10:40 proller FM mergeable to MT
10:40 proller it use full mt history
10:40 Megaf proller, I too get good performance with ARM CPU
10:40 proller no. you cant
10:40 proller i have 2 mobs mod enabled
10:40 sapier Megaf it's as good as writing the code on your own as proller doesn't comment what's his intention too ;-)
10:42 proller https://github.com/freeminer/freeminer
10:43 sapier proller I don't have time to reverse engeneer both your code and minetest code
10:45 werwerwer joined #minetest-dev
11:17 thexyz sapier: and proller doesn't have time to spoonfeed everyone incapable of reading the commit log
11:20 kahrl it would help if the commit log didn't consist of "berrter verbose" "clean" "small" "working" ...
11:20 kahrl git rebase exists for a reason
11:21 proller rebase after push ?
11:21 kahrl no, in feature branches
11:21 proller i push my branches to test on 2 servers
11:22 proller and i want make game, not cute log
11:22 proller also you can make diff to minetest/master
11:22 thexyz that's true
11:23 kahrl that huge diff is going to help I'm sure
11:23 kahrl well you can make your game but don't tell anyone to read your commit logs
11:23 proller current transfer from ARM : http://paste.org.ru/?hujaap
11:25 proller where i tell to read logs ?
11:25 sapier thexyz I stopped reading prollers commit logs at "oooh" "better" "fixed" ... ;-)
11:26 blaaaaargh joined #minetest-dev
11:27 thexyz sapier: fine
11:27 thexyz then please tell your users to not rage at us for "not providing source"
11:28 sapier I'd like to merge your patches but actually prollers patches feel like he's adding to his private project ... you can do this for freeminer of course
11:29 thexyz well it gets better
11:29 sapier change something here something there and a little bit of there ... and don't tell anyone what this all is good about ... someone couldn't like the change of engine behaviour
11:30 sapier what is "it"?
11:30 thexyz those messages you pointed at were done in a hurry iirc
11:30 thexyz when masterserver report got broken
11:31 sapier in a haurry? he didn't change it for months
11:31 thexyz after httpfetch merge
11:31 sapier always telling "I don't rebase prior merge"
11:31 thexyz you're free to not merge our stuff but please don't hate us ;_;
11:32 sapier if you keep on telling "you're so much better" you have to live about beeing told you're not playing well with minetest community
11:32 sapier "to life with"
11:33 thexyz ah fine i'll ask him to stop telling that then
11:33 thexyz people can make a choice on their own after all
11:33 sapier of course
11:38 sapier and btw thexyz noone hates you, I'm just a little bit annoyed about waste of developer time we need to fix same things multiple times due to split of community
11:54 kshawn757 joined #minetest-dev
12:14 PilzAdam joined #minetest-dev
12:24 thexyz a daily reminder
12:24 thexyz "Thank you for using our online DMCA complaint form. You should receive a confirmation email shortly."
12:24 thexyz https://support.google.com/legal/c​ontact/lr_dmca?product=googleplay
12:25 sapier did we get that mail? ;-)
12:25 thexyz "We have received your legal request. We receive many such complaints each day; your message is in our queue, and we'll get to it as quickly as our workload permits. "
12:25 thexyz I've just submitted it
12:27 thexyz I think every Minetest developer should do it
12:37 proller [2-0917000002359] done
12:55 EvergreenTree joined #minetest-dev
13:07 thexyz great
13:12 rambomedic joined #minetest-dev
13:33 hmmmm joined #minetest-dev
13:46 iqualfragile joined #minetest-dev
14:10 zat joined #minetest-dev
14:14 ImQ009 joined #minetest-dev
14:28 Zeitgeist_ joined #minetest-dev
14:28 Zeitgeist_ joined #minetest-dev
14:30 mrtux joined #minetest-dev
14:31 proller joined #minetest-dev
14:36 troller joined #minetest-dev
14:47 OldCoder joined #minetest-dev
14:59 Megaf_ joined #minetest-dev
15:04 blaaaaargh joined #minetest-dev
15:06 ShadowNinja How does #675 with proller's patch look?
15:09 diemartin joined #minetest-dev
15:11 ShadowBot joined #minetest-dev
15:13 zsoltisawesome joined #minetest-dev
15:15 troller joined #minetest-dev
15:32 Akien joined #minetest-dev
15:47 EvergreenTree joined #minetest-dev
15:58 Jordach joined #minetest-dev
15:59 Jordach_ joined #minetest-dev
16:10 VanessaE joined #minetest-dev
16:24 kaeza joined #minetest-dev
16:25 Megaf joined #minetest-dev
16:25 Calinou joined #minetest-dev
16:31 jin_xi joined #minetest-dev
16:42 werwerwer_ joined #minetest-dev
16:44 Megaf_ joined #minetest-dev
16:46 hmmmmm joined #minetest-dev
17:27 rubenwardy joined #minetest-dev
17:39 Miner_48er joined #minetest-dev
17:39 smoke_fumus joined #minetest-dev
17:57 Warr1024 joined #minetest-dev
18:01 rubenwardy_ joined #minetest-dev
18:09 proller joined #minetest-dev
18:10 diemartin joined #minetest-dev
18:12 Warr1024 joined #minetest-dev
18:17 EvergreenTree joined #minetest-dev
18:24 kaeza joined #minetest-dev
18:34 Warr1024 joined #minetest-dev
18:35 rubenwardy joined #minetest-dev
18:43 kaeza joined #minetest-dev
19:03 rubenwardy_ joined #minetest-dev
19:17 proller joined #minetest-dev
19:20 VanessaE so, about disallowing compiling without cURL support.  Are we gonna just do it or not?
19:20 VanessaE seems to me that's a simple matter of a tweak to some makefile somewhere (or whatever cmake calls it)
19:20 sapier NO
19:20 VanessaE why no?
19:21 sapier it's hack not a fix
19:21 VanessaE I mean client-side, not server-side.
19:21 sapier I'm working at a propper solution but it's gonna be to late for christmas release
19:21 VanessaE define "a proper solution"
19:22 VanessaE (and I do believe, a year ago, I decribed cURL as hacky also and that media_FUBAR should have been fixed then, but as usual I was ignored and told to just follow the leaders)
19:22 VanessaE (but since we have it, I'm using it)
19:23 sapier proper solution means splitting send from receive and add priorization for different packet types
19:24 sapier guess it's gonna be testable in about 1-2 days
19:25 proller btw fm can serve 1.5mbps from arm cpu now ;)
19:26 sapier apache can serve gigabyts of data from an arm cpu ;-P
19:28 VanessaE sapier: sounds good
19:28 VanessaE proller: big deal, my C64 could do that from a 20 MHz 65816.
19:28 sapier hope it's gonna work as good as it sounds ;-)
19:36 thexyz sapier: i've integrated enet into freeminer
19:36 thexyz there's a testing branch available
19:36 thexyz https://github.com/xyzz/freeminer branch `enet`
19:36 thexyz it's still in testing (no ipv6 for example) and i guess i did some things wrong, but it works and you can connect to servers and play
19:37 thexyz there's one server, xyz.is:30000
19:37 sapier is enet compatible to old clients?
19:37 thexyz no of course
19:38 thexyz it's a completely new network stack
19:38 sapier then I guess freeminer will drift away even more from minetest
19:38 Warr1024 unless minetest pulls this in
19:38 Warr1024 which may be so, if FM can prove its value.
19:38 thexyz yes, that's why I'm posting it here
19:39 thexyz it still doesn't work very well so any help would be amazing
19:39 sapier Minetest can only pull this if it's going to 0.5 AND break compatibility completely
19:39 thexyz then we can think on other things to integrate while breaking protcol
19:39 thexyz i.e. key-value messages instead of current format
19:39 thexyz sapier: indeed
19:40 thexyz proller has suggested messagepack which seems a nice idea
19:40 proller or json ;)
19:41 sapier maybe you should start persuading more core devs
19:41 Warr1024 we really don't need more than one serialization format.
19:41 thexyz we won't have more than one
19:41 Warr1024 if JSON serialization can match/beat our existing format, then we might as well switch.
19:41 sapier we have 3 as of shadows merge of json writer
19:42 Warr1024 I guess it would mean that serialization could move to C for performance as well.
19:42 sapier serialize which is 100% feature identical to json writer as well as marshalling
19:42 thexyz messagepack is basically json
19:42 sapier later one is capable to serialize functions too
19:42 thexyz but takes less size
19:42 sapier can you please discuss this in freeminder-dev ?
19:42 thexyz marshal is used for some internal async stuff, isn't it?
19:42 sapier -d
19:43 thexyz I thought this may interest Minetest core devs because the current tcp-over-udp implementation is broken
19:43 PilzAdam what about that BKVL thing?
19:43 thexyz that's another case of NIH
19:43 sapier You don't seem to have any interest in explaining WHY your changes are worth breaking everything
19:43 thexyz I think we should do less NIH
19:43 thexyz sapier: what changes exactly?
19:43 sapier json formspecs json protocol enet
19:44 PilzAdam NIH?
19:44 thexyz I haven't said anything about json formspecs
19:44 Warr1024 PilzAdam: National Institute of Health :-)
19:44 thexyz enet is better just because after that little test of mine (without fully checking the enet and only with one channel, etc) it is faster to load on localhost
19:44 thexyz enet is better because that's a library which has been around for 9 years it seems
19:45 Calinou joined #minetest-dev
19:45 thexyz enet is better because many projects already use it and we won't have to support it
19:45 sapier wasn't there a problem with enet and ipv6?
19:45 PilzAdam http://devopsreactions.tumblr.com/post/​36872312242/when-someone-mentions-ipv6
19:46 thexyz sapier: it doesn't support ipv6 but there's a pull request which adds this and proller has volunteered to implement and test it
19:46 thexyz PilzAdam: can you please not post funny reaction images when a discussion is in process?
19:46 sapier so it's not better then current implementation in this case
19:46 thexyz sapier: no, it is better
19:46 thexyz please reread my messages if you still feel like it's not
19:46 thexyz I've given three primary reasons, is that not enough?
19:47 sapier no it isn't if I want to use ipv6 now
19:47 Warr1024 I think the best way to convince people will be with empirical evidence.
19:47 thexyz ipv6 is not that hard to implement and there's a pull request which does that
19:47 Warr1024 try doing some controlled side-by-side comparisons.
19:47 sapier but it's even more untested then current protocol
19:47 thexyz I'm not sure what comparisons to do; and I still feel I fucked something while implementing it so I need someone to review my code and fix bugs
19:47 thexyz sapier: it's been around for 9 years
19:47 thexyz many projects use it
19:48 Warr1024 luckily, it sounds like FM has volunteered to guinea pig this new feature that may potentially benefit MT too :-)
19:48 thexyz I don't like to repeat things but if you insist
19:48 thexyz sapier: what untested is my implementation, not protocol
19:48 VanessaE Warr1024: you mean that may benetif FM (because it won't ever be merged into MT)
19:48 VanessaE benefit*
19:48 thexyz but it really is simple and I feel like any core dev more experienced than me can check it
19:49 Warr1024 VanessaE: maybe, maybe not.  some hard evidence may go a long way towards swaying the naysayers.
19:49 thexyz shall we continue with enet or may I switch to the second argument? the key-value protocol change
19:49 sapier while I usually are absolutely positive about using tested implementations ... this doesn't superseed my hate of breaking protocol ... at least if there isn't something much better
19:49 thexyz sapier: I've already mentioned three reasons to use it
19:50 thexyz can you please read my messages once again?
19:50 E4xoi we just need more things for the 0.5.x release
19:51 thexyz it is 1) faster 2) older and used by many projects 3) we will only benefit from removing that pile of code
19:51 VanessaE Warr1024: just going by what I've seen over the year and a half I've been following the project.  the bigger and grander the idea, the less likely it is to proceed.
19:51 sapier " enet is better because many projects already use it" --> not a proof
19:51 sapier won't have to support it --> guess as we not have to support irrlicht
19:51 sapier I didn't find your third reason
19:52 Warr1024 VanessaE: well, I guess that's why people occassionally jump ship onto forks and the like.  They say "evolve or die" like it's not a choice, but technically it *is* a choice...
19:52 Warr1024 I'd considered making the jump to FM too when I saw how fast they added cool new shit, but there is also value in MT's conservative approach to backward compatibility.
19:53 VanessaE Warr1024: which is why fm was born.
19:53 sapier So keep working on separating freeminer even more from minetest while I try to fix minetest in a compatible way
19:53 VanessaE because some shit that should be in MT just won't ever make it in.
19:53 Warr1024 frankly, I'd like to see BOTH approaches tried (enet in FM and MT sticking with its existing proto for a while longer yet) and just see who wins on the merits.
19:53 thexyz sapier: I don't understand your counter-points
19:53 Warr1024 we could sit here arguing about it forever, or we could work on the code :-)
19:53 thexyz 3) we will only benefit from removing that pile of code
19:54 sapier thexyz COMPATIBILITY
19:54 E4xoi you won't merge it like any other pull request
19:54 thexyz compatibility is good
19:54 VanessaE (while some stuff that should not be, would have gone in had all of fm's changes stayed in mt - e.g. finite water, I still disagree with how that works)
19:54 thexyz but sometimes it has to be broken
19:54 thexyz for the good
19:54 sapier you did not convince mee this is one of those times
19:54 E4xoi you will merge when you have other shit that is awesome so you break the compat and remove all the useless code for supporting shits down to 0.4.4
19:54 thexyz sapier: it is faster
19:54 thexyz we don't have to support it
19:55 thexyz why is that not enough?
19:55 sapier I don't believe later point
19:55 thexyz the later being what exactly?
19:55 sapier "we don't have to support it"
19:55 thexyz about "support"? why do you think we'll have to support it?
19:55 sapier because we have to support all of the things we use
19:55 thexyz many other projects already make use of it so if shit breaks they're more likely to fix it
19:55 thexyz perhaps we have a mixed definition of the "support" word?
19:56 thexyz I mean we won't have to fix bugs in it
19:56 thexyz and "optimize" it
19:56 sapier exactly this is what I don't believe
19:56 thexyz why?
19:56 sapier because we WILL have to fix bugs in it
19:57 thexyz why?
19:57 sapier because there ain't any software in universe not having bugs
19:57 thexyz I will repeat once again
19:57 Warr1024 what will happen if the bugs don't get fixed?
19:57 thexyz there are projects bigger than Minetest that use enet
19:57 thexyz I think Calinou can give some examples of projects using enet?
19:58 sapier JThread was used by a lots of projects too
19:58 thexyz were there any bugs?
19:58 thexyz or were there things you weren't happy with?
19:58 E4xoi jthread didn't have bug but was too damn simple for what 'we' wanted, wasn't that the reason of changing it?
19:58 celeron55 i'm all in for changing minetest to use enet
19:58 thexyz I can't find any other project which uses jthread
19:58 Calinou thexyz, sauerbraten, red eclipse, assaultcube, tesseract
19:59 Calinou (three cube 2 games, 1 game based on cube 1)
19:59 celeron55 really the only reason i didn't make it use enet originally was because i wanted to learn stuff and was just messing around without a goal of making the whole world use it anyway
19:59 Calinou also, I've seen some commercial games use it
19:59 thexyz celeron55: then perhaps you can look at the code?
19:59 sapier I leave this discussion now it's waste of time I'll use for fixing minetest
19:59 Calinou oh, Stunt Rally uses that too
19:59 Calinou but its networking model is P2P-based
19:59 thexyz and tell me where have I fucked up
20:00 sapier celeron55 when did you change mind about compatibility? ;-)
20:00 E4xoi can you connect to 0.3.x servers with 0.4.8?
20:00 kahrl I don't understand, what's the point of adding another library dependency?
20:01 kahrl there already is a widely tested library for reliable communications
20:01 thexyz I've listed three points already, please read the log
20:01 kahrl it is called tcp
20:01 thexyz go on then?
20:01 VanessaE E4xoi: give darkrose enough time and I bet she'll fix that, too :P
20:01 thexyz we might want a mix of reliable and unreliable packets
20:01 E4xoi hah
20:02 kahrl so use one udp socket and one tcp socket?
20:02 thexyz on its FAQ page http://enet.bespin.org/FAQ.html enet says it's better than TCP while giving no reason
20:02 thexyz while giving an obscure one, rather
20:02 celeron55 a plain udp+tcp implementation is very fine to me too (especially if someone as capable as kahrl does it)
20:04 kahrl I might even have some time during the next two weeks or so
20:04 thexyz you can ask for more details on #enet I think
20:04 celeron55 kahrl: you might want to see this before getting into it: http://www.iis.sinica.edu.tw​/~swc/pub/tcp_in_games.html
20:05 sapier what about starting a poll if minetest shall break compatibility and switch to enet?
20:05 celeron55 basically the issue with tcp in games is that tcp's design assumes that data transfer is limited by network speed, not by program's limited transfer requirements
20:05 celeron55 which leads to certain kinds of behavior which may or may not be bad for minetest with a combination of tcp and udp
20:05 kahrl true
20:05 celeron55 what it mainly means is that tcp sacrifices latency
20:06 thexyz well, in the end I too am fine with either; the problem is that often things are talked about a lot but never actually done
20:08 sapier fix for major latency issues with current implementation will be ready by christmas ... to late for christmas version but in state for review ... and if it's still necessary to add tcp support this variant will be easy to adopt
20:09 thexyz so we've decided to not use enet, right?
20:09 sapier I don't know we don't have a way to do decisions like that
20:09 kahrl if it's easier for you to implement in freeminer, use enet
20:10 celeron55 atm i would say let's use enet; it can be easily bundled within minetest for cross-platform usage and it pretty surely does what we need well
20:10 VanessaE for G*D sakes
20:11 VanessaE make sure as a server owner I can REJECT connections from protocols I don't want!
20:11 celeron55 wut
20:11 VanessaE if you're gonna add something new that works better, make sure I can reject the old crap, assuming it can still connect
20:11 celeron55 we won't be including support for many protocols
20:11 celeron55 if a new one comes, the old one is removed
20:11 VanessaE ok
20:11 VanessaE good
20:11 sapier VanessaE your servers won't be able to accept connections from stable clients for quite a long time
20:12 VanessaE I'm thinking of media_FUBAR vs. httpfetch :)
20:12 VanessaE sapier: ok, so nothing unusual then? ;)
20:12 celeron55 thexyz: poke me about your enet stuff after a few days if i forget it
20:13 sapier If we decide to use enet then at least majority of core devs need to agree so get that poll started
20:13 VanessaE I am all in for enet if it works well
20:13 sapier celeron55 that's crap either decide or don't
20:13 celeron55 sapier: ?
20:14 celeron55 it's about the particular implementation, not about the idea
20:14 thexyz kahrl: I don't want to break protocol; on the other hand I don't want to have the current UDP implementation either
20:14 thexyz it'
20:14 sapier thexyz is working on enet I'm working on fixing minetest in a compatible way you already loose a lot of work if you delay decision you'll loos much more work
20:14 celeron55 sapier: stop what you're doing then
20:15 thexyz it's still not clear for me what are we going to do: TCP or enet
20:15 sapier so you decide to use enet?
20:15 specing 21:29 < specing> What hash does minetest use for passwords?
20:15 specing 21:29 < specing> It looks kinda ... short
20:15 specing 21:29 <+ShadowNinja> specing: SHA1 salted with the player name
20:15 specing http://arstechnica.com/security/2013/12​/bitcoin-only-poker-site-resets-user-cr​edentials-after-42000-passwords-leak/
20:15 sapier current state is inacceptable it's insane to have a connecting client kill server
20:15 specing 1000 SHA1 cracked in <9 mins
20:16 specing So... bcrypt please?
20:16 celeron55 well ffs, i decide then: we use enet, unless it turns out to be crap while trying to make a clean implementation of it (assuming thexyz's wasn't)
20:16 VanessaE specing: offtopic
20:16 specing Ontopic
20:16 celeron55 in which case we look for alternatives
20:16 thexyz celeron55: I'm not sure; I think you should check the code
20:16 specing minetest uses a weak hashing function for passwords
20:16 celeron55 thexyz: i'm not sure either, but people demand answers
20:16 E4xoi specing: >bitcoin, if you're mining bitcoins you probably could crack 100k SHA1 hashes in less time ;)
20:17 sapier if I stop working on this now I'm not gonna start all over again for sure
20:18 sapier I have time now and understand how minetest meant to do it so doing this in 8 weeks is not an option
20:19 sapier "i decide then: we use enet" ok thexyz do you have a prototype?
20:19 thexyz yes, I've mentioned this multiple times
20:19 sapier where to pull?
20:20 thexyz https://github.com/xyzz/freeminer/compare/enet
20:20 sapier NO
20:20 sapier come one guys can you pleas write a little bit more acceptable comments?
20:20 thexyz that is a prototype
20:21 thexyz I needed the source to be in sync between my home PC and a laptop
20:21 thexyz in the end everything should be squashed into one commit
20:21 thexyz and a proper description written
20:22 sapier when do you think this will be ready?
20:22 thexyz not sure, need someone else to look into it
20:22 VanessaE "something enet something something"
20:22 VanessaE lol
20:23 VanessaE http://xkcd.com/1296/
20:23 sapier I wont look at it if you don't explain it
20:23 sapier I don't care about the way you explain it
20:23 thexyz you didn't even ask ;_;
20:24 sapier that was meant as asking ;-)
20:24 sapier usually I try to read commit messages for explanation but that's impossible here
20:24 thexyz then what did you ask?
20:24 thexyz what explanation do you expect there?
20:25 sapier what is done and what needs to be done?
20:25 thexyz indeed, I can explain that
20:25 VanessaE "Add experimental support for enet in lieu of the default networking protocol" maybe?
20:25 kahrl hmm, it seems that if a player gets banned while receiving itemdef/nodedef/media, the access denied message is not shown (until all media is received)
20:25 thexyz there's no ban support
20:25 thexyz in my branch
20:25 thexyz but I see, you're not talking about it
20:25 kahrl because that loop in game.cpp doesn't check client.accessDenied()
20:26 kahrl thexyz: yeah, I mean in minetest
20:26 thexyz anyway, what is done: minetest protocol implementation is replaced by enet completely
20:26 sapier thexyz can we create a wiki page to add information?
20:26 thexyz what information do you need?
20:27 sapier ALL ;-) but I guess I'll read what you did by now and see myself what is missing
20:28 kahrl anyone mind me pushing this quick fix? https://gist.github.com/kahrl/8045691
20:28 thexyz I'm not sure what's that ALL you're talking about; anyway, I will do that tomorrow
20:28 kahrl I copied this code straight from the connect loop
20:28 thexyz for now it indeed is easier to read the diff
20:29 sapier L144 is about asigning new peer ids isn't it?
20:30 thexyz right
20:30 thexyz there's also "// TODO: why do we even need to know our peer id?"
20:30 thexyz which has to be fixed too
20:31 thexyz then the other problem is that this (or server?) thread seems to eat 100% of cpu constantly now
20:31 sapier ok first of all do you intend to change paket format too?
20:32 thexyz adding a proper timeout in enet_host_service doesn't help
20:32 thexyz packet format? yes, we were thinking about messagepack
20:32 sapier of xourse you didn't fix the main issue of minetest connection yet
20:32 thexyz what is it?
20:32 sapier receive and send linked within same thread
20:32 thexyz why is that a problem?
20:33 sapier because you are delayed for as much time as you need to send all packets
20:33 thexyz hm, no this is not true
20:33 sapier explain why
20:34 thexyz enet does this in its enet_host_service
20:34 sapier is this a own thread?
20:35 thexyz I don't really know how ENet works; this part of code is right from the official tutorial http://enet.bespin.org/Tutorial.html so I've assumed this is how it should be done
20:36 sapier I don't like the idea using a lib noone know how it's working ... at least basics should be known
20:38 thexyz what are those basics you're talking about? apparently, it both sends and receives when you call enet_host_service
20:38 thexyz but I can't be too sure about this
20:38 Calinou you could ask eihrul or quin, they are the people around Cube 2 stuff
20:38 Calinou they're in #redeclipse
20:39 Calinou (maybe)
20:39 thexyz indeed
20:39 thexyz if you really want to know how it works then just do that
20:40 sapier1 joined #minetest-dev
20:41 zat joined #minetest-dev
20:42 sapier1 you're great thexyz you keep telling it's better but a single question for details results in a "I don't know" answer
20:42 thexyz yes, I do
20:43 thexyz because I've just tested it with this weird half-working implementation and found it better
20:43 thexyz err, I don't, rather
20:43 thexyz for some reason I feel like you don't want to have a constructive discussion here but want to mock me and make yourself feel smart
20:44 celeron55 i think thexyz is acting just fine; people just assume everything thexyz does is wrong because he's directly supporting proller's freeminer stuff
20:44 sapier1 is it better because it's better or because you haven't run in any of the problems current implementation ties solve?
20:45 sapier1 no I don't think he's doing wrong ... enet MAY be a good way to go
20:45 sapier1 but I'd like to know where are the open issues and how to solve them
20:45 celeron55 kahrl: it seems fine; push it if it works
20:45 sapier1 e.g. 100% cpu usage ... that's exactly SAME issue minetest suffers about SOMETIMES
20:45 kahrl yeah, I tested it (locally)
20:46 kahrl pushing...
20:50 thexyz then it could be fixed in the same way; and since there's now less code in the connection.cpp then it's even easier to fix
20:50 thexyz sorry, have to go to sleep now; highlight me if you want to say something rude again
20:51 sapier yes thexyz everyone is bad to you ... I'll try to continue your work as decision is made I want to get this done as quick as possible
20:53 thexyz no, not everyone; anyway it's good to know you will be working on it, please keep us informed
20:55 celeron55 thexyz: this enet stuff is surprisingly straightforward
20:55 celeron55 i'm surprised it maps so directly to the Connection stuff
20:56 sapier basic features seem to be almost equivalent to current minetest implementation
20:59 celeron55 as it seems sapier is now looking at it, i think i'll too wait for sapier's findings
20:59 sapier first impression is it's way more simple then minetest implementation
21:00 proller current implemerntation designed for singleplayer
21:00 celeron55 it seems there was some mutex things commented out without proper care at least so those need to be thought out... and then the 100% CPU usage
21:01 sapier I'm looking at enet right now not the changes
21:02 celeron55 (and also a possibility would be removing the thread inside Connection, as it was just a way to try to handle some things in the old implementation somehow sanely)
21:03 sapier enet doesn't have it's own thread
21:03 proller now 50+% players come from android with slow networks and loooot of RE-SENDING timed-out RELIABLE
21:04 VanessaE oh and if you put this new protocol in, you're gonna HAVE to prevent servers from showing in the public list that wo't connect to those old clients
21:04 VanessaE so that the rest of us don't have to field 1000 complaints a day about no-connect failures
21:04 sapier not my issue ;-P users will recognize if they can't connect
21:04 VanessaE no they won't
21:04 VanessaE they'll just complain
21:05 VanessaE or "this server sucks"
21:05 sapier they'll complain anyway
21:05 VanessaE if the server isn't there at all, they won't try to connect
21:05 sapier users always complain on total compatibility loss
21:05 proller 90% servers will switch to new proto in ~week
21:05 VanessaE there are 10x as many n00bs coming in as regulars
21:05 sapier 90% of users wont switch for about 2 months
21:05 VanessaE regulars know what to do, n00bs don't
21:05 sapier maybe more
21:06 VanessaE so if the users can't fix it, the servers owners have to
21:06 VanessaE (which relies on features in core of course)
21:06 sapier but it's been decided by celeron compatibility isn't an issue so please forward you complains there ;-P
21:06 E4xoi at the time tou switch to enet, i guess you are removing all the old code, no?
21:06 sapier VanessaE there ain't a fix for this issue
21:06 VanessaE sapier: that's my point - if this breaks compatibility, then it needs to completely break it - as in break it at the master server level also
21:06 E4xoi s/tou/you/
21:07 sapier of course E4xoi
21:07 E4xoi heck yeah ;)
21:07 VanessaE don't show my server in the master server list if a client that can read from that list cannot connect to it.
21:07 sapier minetest will be 0.5 after this switch
21:10 VanessaE can't the web server hosting the master list filter it according to the features of the client that is reading from it?  like if the client fails to tell the server "hey, I can connect to minetest servers that use enet" ?
21:10 VanessaE in other words, if you can't filter based on what the client says it has, filter on what it *doesn't* say
21:11 E4xoi you won't be able to connect to 0.5.x with 0.4.x or viceversa
21:11 sapier everything is possible
21:11 proller now list.JSON ;) is static
21:11 VanessaE E4xoi: it isn't about connecting.  it's about them showing up in the list to begin with
21:11 proller we can rename to list0.5 but..but...
21:11 sapier isn't there a ignore command in irc
21:11 VanessaE stop the attempt before it's even initiated
21:12 proller sapier, /ignore
21:12 sapier already tried /ignore proller ... doesn't work
21:12 VanessaE sapier: don't ignore, you never know what your opposition might have to say that may be of benefit.
21:12 sapier if I don't ignore him I'm gonna stop working at minetest soon
21:13 VanessaE sapier: soft-ignore then, like everyone here does with me :P
21:13 sapier I'm a all or nothing guy
21:13 VanessaE I know :)
21:13 sapier I accept a lot of insane comments but once threshold is reached it's over
21:13 VanessaE as anyone who has seen your rants about minetest security can attest to ;)
21:14 proller {"sapier":false}
21:14 sapier can someone tell me name of the guy writing enet?
21:15 VanessaE Lee Salzman
21:15 VanessaE so says the license anyway
21:15 sapier I'd like to know for what reason he duplicated his own code within  a loop
21:17 sapier and that guy isn't a friend of commenting his functions too, despite of header comment
21:18 VanessaE you've seen my code, I'm not exactly fond of tons of comments either
21:19 VanessaE too many comments makes the code hard to read, imho
21:19 VanessaE (not enough is just as bad though)
21:19 sapier if they don't match the code you're right vanessae ;-)
21:19 VanessaE haha
21:23 sapier ok we actually need to remove the connection thread to use enet
21:24 sapier enet_host_service has to be called in main loop with all consequences
21:26 rambomedic joined #minetest-dev
21:30 sapier celeron55 can you have a look at unix.c from enet? enet_socket_wait really waits for timeout to pass without any way to interrupt?
21:32 Megaf joined #minetest-dev
21:43 sapier Ok for what I see enet has following benefits:
21:43 sapier 1) quite clean code compared to minetest connection
21:43 sapier 2) a more or less proofen peer handling
21:43 sapier 3) fragmentation handling
21:44 sapier But there are issues too:
21:44 sapier 1) lack of priority queue handling
21:45 sapier 2) lack of multithreading support ( telling it doesn't have issues because it all has to be done within a single thread isn't mt support)
21:45 sapier 3) coupled sending/receiving
21:45 sapier 4) lack of Ipv6 support
21:46 sapier point 2 may be fixable by adding an additional layer to create a sender thread per peer
21:46 sapier 1 and 3 are not fixable at all
21:47 sapier at least without rewriting enet ... which is exactly what we don't want to do
21:48 sapier Can someone please tell me I'm wrong (with explanation why) ... because I wont waste my time to do this if it's just postphoning current issues by a couple of additional peers
21:51 celeron55 what does 1 mean
21:53 sapier e.g. we can't add packets to be sent with lower priority then others. Map data textures models for example
21:55 celeron55 enet's channels do that; it prioritizes channels based on their number so that lowest channels' buffers get sent first
21:55 celeron55 at least according to some random text i found
21:56 sapier I'd be glad if this was true I have a closer look at it
21:57 celeron55 i don't know if 3 is an actual problem, and 2 and 4 might not be right now, but they will be in the future and then there needs to be solutions
21:57 us_0gb joined #minetest-dev
21:58 rambomedic left #minetest-dev
21:59 sapier 3 is a problem if enet doesn't do propoper send time division for peers
22:00 sapier e.g. if it just sends packets as queued for all peers without any limit
22:00 sapier any per peer limit of course
22:01 sapier actually enet uses send windows so this shouldn't be a big issue
22:01 celeron55 they say it has congestion control
22:02 celeron55 i think it should mostly take care of that
22:02 proller can it support 1000+ connections?
22:02 sapier no
22:02 sapier found some forum postings about issues as of 16 peer
22:02 sapier s
22:03 celeron55 if we want MMO-tier numbers of clients, we need something more sophisticated
22:04 celeron55 there are so many problems to pile up at 1000 connections though that it's not a sane goal
22:04 celeron55 MMOs usually handle only hundreds of players per instance
22:04 celeron55 or less
22:05 sapier true but despite of "we don't have to maintain it" I don't see a killer feature in enet by now
22:05 celeron55 but minetest can't do multiple instances for a single world
22:05 sapier and this assumption has to be proofen
22:06 celeron55 it's clearly a compromise - but so is tcp + dumb utp a compromise too (a perfect solution would be like enet, but without it's limitations)
22:07 sapier I hate people writing c++ in c
22:07 sapier that guy wrote a 785 chars printf
22:11 Calinou I've seen sauer servers run well with 20-25 players, and 2 players + ~50 spectators
22:11 sapier we need to test, noone can tell in advance how enet will behave for typical minetest network communication profile
22:13 celeron55 there are some other libraries that could be looked at: http://opensource.linuxgamepublishing.com/grapple/ http://sourceforge.net/projects/openplay/
22:13 celeron55 but it's somewhat hard to find about ipv6 or really anything in stuff like this
22:13 sapier ?
22:15 sapier It wasn't my suggestion to replace current marture but buggy network communication mechanism ... evaluating a single protocol takes about 1-2 days and noone writes his findings ... do you really believe we will ever make a decision based uppon facts but feelings?
22:17 celeron55 why are you talking like this
22:17 sapier because you just decided to go with enet and now put 2 additional protocols to game?
22:18 celeron55 did you read what i decided
22:18 sapier no offence but I don't understand
22:18 sapier yes I did everyone could read like she/he wanted ;-)
22:18 celeron55 i decided that enet must be investigated right now, and now that it was looked at and there were many potential issues, i am looking whether other options have similar issues or not
22:19 sapier I for example read as "we use enet if there isn't any unsolvable problem"
22:19 sapier but I thought about reading it "I say use enet to stop the discussion and decide later" too ;-)
22:20 sapier ok channels aren't exactly priority but they can be used this way
22:22 sapier each channel has it's own window. we can't prevent a channel from using it's windows if high prio is maxed out but at least the fixed bandwith for high prio packets is guaranteed
22:23 celeron55 we now know what benefits and problems enet likely has; now we have to decide whether the problems are acceptable; if i understood correctly, you think it should be tested in real usage and a final decision should be based on that
22:24 sapier as of reading code I'm unsure about enet really solving all our issues ... it could do but I assume we need to use it carefully to achieve this
22:24 celeron55 do you think investigating bare tcp+udp is useful?
22:25 sapier I assume droping in enet in a quick way (not beeing carefull) is about 2-3 days of work ... carefully doing it is about a week (full time)
22:25 celeron55 "bare" udp probably means that some of the existing code in minetest has to be used too
22:26 sapier tcp+udp ... difficult to tell but rough estimation about 2 weeks full time too
22:26 sapier and of course for tcp there ain't a way to do a quick check
22:27 sapier and tcp+udp increases firewall complexity (slightly) ;-)
22:27 celeron55 so what things do you think need extra care in enet?
22:27 celeron55 is there a chance that some other similar library would be easier for those parts?
22:27 sapier main reason for assuming minetest core protocol to be broken is huge lag on player downloading textures
22:28 sapier so moving those packets to a low prio sending has to be done some way
22:29 sapier I don't know of any but I'll have a quick look at your other suggestions (not as deep as enet of course)
22:29 celeron55 really all i know about the minetest protocol's performance is that for low usage, it performs fine, and under high traffic, it just completely chokes up
22:29 celeron55 (and doesn't use the available bandwidth)
22:30 proller and it for singleplayer
22:31 proller okay, less 10 players
22:31 celeron55 altough in theory bandwidth usage is an implementation problem - but many people have already solved the same problem for code that they've put into libraries
22:32 sapier there are some really bad decisions within connection ... a little amount can be fixed quick but most things need major rework
22:32 sapier I'd assume fixing minetest internal protocol is about half a week to a week too
22:34 celeron55 enet simply has the highest chance of being very good; i think everyone appreciates it if you (or someone) test whether it really works
22:34 sapier grapple seems a little bit too much for what we need
22:34 us|0gb joined #minetest-dev
22:35 sapier but grapple would provide facilities for better passwords and data encryption ;-)
22:35 sapier but I don't see any priorization support in there too
22:36 sapier enet is politicaly best as everyone assumes minetest network code is bad ... it's not as bad as it's reputation
22:37 sapier when did sourceforge stopp to support konqueror?
22:39 sapier openplay is dead
22:39 sapier no need to look at code not beeing changed for a decade
22:40 celeron55 hmm
22:40 sapier grapple is more then 3 years old to so forget about the security things I told they're most likely nonsense by now
22:41 celeron55 how good, compared to enet, do you think you could make minetest's protocol with a reasonable amount of work?
22:41 celeron55 this would be an easy decision if enet didn't have any issues, but it does
22:42 celeron55 i don't want to base this on politics; it's never good
22:43 celeron55 what does it require?
22:43 sapier hard to tell you can make it even better then enet of course ... but you need to put at least half a week to a week in it to have a real chance to compete
22:43 Megaf joined #minetest-dev
22:44 sapier our protocol supports channels too. But I haven't seen anyone use a different channel then 0
22:44 celeron55 i want to have a decision tonight because talking about this is only one step in actually getting something done
22:46 celeron55 there's actually a benefit in using enet other than that it could possibly almost "just work"
22:47 celeron55 it's at least that it would make clients other than minetest's official one easier to make, eg. for testing or bots or whatever; increased interoperability anyway
22:47 sapier as far as I know we don't have send windows in our implementation
22:47 sapier but I didn't get to that point by now
22:48 celeron55 i'd imagine one thing you would do is replace some of the current crap with a congestion window
22:48 sapier I don't think so celeron main effort is still minetest packet format which wouldn't change due to enet
22:48 sapier only peer handling is changed
22:49 celeron55 sapier: it would remove one link from the chain, and a very dirty link it is
22:49 sapier yes I have about 60% completed split of send and receive by now, cleaned up peer handling and started looking at channel usage
22:50 sapier that's true the outer envelope would be enet instead of minetest
22:51 celeron55 it's especially relevant because the inner formats are likely to be changed some day too
22:51 celeron55 to something less obscure
22:52 sapier I hope you don't think about something pseudo less obscure like json
22:52 sapier having only 512 bytes json will fragment almost any packet
22:53 celeron55 probably messagepack; nothing else really makes sense
22:53 celeron55 (with generally using integer keys to lessen the bloat)
22:55 celeron55 the current formats do make some sense too though, as they're almost maximally size-efficient
22:55 sapier you know enet is a one man show too?
22:56 celeron55 if you want something that isn't, it's probably going to be commercial
22:57 sapier multi person projects tend to be more stable ... maybe minetest isn't best example ;-)
22:57 celeron55 what if we make a library out of our own; then there exists a non-commercial enet that isn't a one-man show
22:57 celeron55 *enet-like
22:58 celeron55 i'd like to hear thexyz's and kahrl's views on this
22:58 sapier me too
22:58 celeron55 but it's going to make this discussion take longer
22:58 sapier enet isn't bad for sure
22:58 celeron55 i guess we just need a break; let's continue tomorrow
22:59 sapier it's prooven code but we need to understand how to use it correctly if we don't we most likely end up in same problems as now
23:00 sapier yes hope thexyz and kahrl can add additional perspectives we missed by now
23:04 kaeza joined #minetest-dev
23:30 * VanessaE is back (yay.)
23:30 kaeza can someone take a look at #1060?
23:30 ShadowBot https://github.com/minetest/minetest/issues/1060
23:43 WarrTab joined #minetest-dev

| Channels | #minetest-dev index | Today | | Google Search | Plaintext