Minetest logo

IRC log for #minetest-dev, 2014-01-10

| Channels | #minetest-dev index | Today | | Google Search | Plaintext

All times shown according to UTC.

Time Nick Message
00:00 us_0gb joined #minetest-dev
00:31 salamanderrake joined #minetest-dev
01:26 bas080 joined #minetest-dev
01:43 kaeza joined #minetest-dev
03:51 VanessaE sapier:  So far, no issues with your pull.  I think it's good to merge.
04:02 Miner_48er joined #minetest-dev
04:41 ImQ009 joined #minetest-dev
05:40 OldCoder joined #minetest-dev
05:41 prestotron55 joined #minetest-dev
05:46 proller left #minetest-dev
05:49 proller joined #minetest-dev
06:08 us_0gb joined #minetest-dev
07:33 mrtux joined #minetest-dev
08:19 toabi joined #minetest-dev
08:21 toabi left #minetest-dev
08:38 ImQ009 joined #minetest-dev
08:44 toabi joined #minetest-dev
08:47 toabi left #minetest-dev
08:52 jin_xi joined #minetest-dev
09:00 ImQ009 joined #minetest-dev
09:04 sapier joined #minetest-dev
09:07 Taoki[mobile] joined #minetest-dev
09:08 sapier Ok preparing network fixes for merge, I'm gonna merge the mutex queue in same step but as separate commit as it's required too
09:09 sapier mutex fixes have been part of 1090 so they're tested too
09:14 kahrl those names (pop_frontNoEx) are a really weird mix of snake_case and camelCase
09:14 VanessaE is THAT what you call this_sort_of_naming ?
09:14 VanessaE (I'm rather partial to that rather than camelCase)
09:15 kahrl even wikipedia calls it that! https://en.wikipedia.org/wiki/Snake_case
09:16 VanessaE I never knew that.
09:20 sapier that naming was already used somewhere else within minetest for no exception functions so I reused it
09:20 Taoki[mobile] joined #minetest-dev
09:21 sapier I didn't want to introduce a new naming
09:21 kahrl maybe pop_front_no_ex
09:22 kahrl or get rid of the exception-throwing variant completely
09:22 kahrl (and call the new version pop_front)
09:23 sapier was suggested before but as you didn't mention this to be a problem for more then 3 weeks now (mutex changes are there for so long) noone tested sideeffects of a change like that
09:23 sapier I principle I think you're right
09:23 kahrl iirc someone complained about the names before
09:24 kahrl don't remember for sure though
09:24 sapier I remember only the suggestion about removing the non exceptioning functions, but I'm not 100% sure too
09:25 darkrose joined #minetest-dev
09:26 sapier but as I said, mutex fixes already had some side effects I'm almost sure replacing all exceptioning variants by non exceptionings will result in problems
09:26 kahrl right
09:27 kahrl anyway I won't block those changes just because of the names
09:28 sapier thanks ... there's a lot of cleanup to do in server removing usage of exceptions as return values is one of the things
09:29 sapier basically it's abused as some weird variant of cross stack goto :-)
09:29 Megaf joined #minetest-dev
09:30 sapier ok lets make more people test those changes mergeing now
10:02 proller joined #minetest-dev
10:19 Gethiox joined #minetest-dev
10:49 rsiska joined #minetest-dev
11:02 proller workaround_window_size max_packets_per_iteration - WHYYY?
11:02 proller nobody never will change it
11:03 proller network settings must have good hardcoded defaults
11:03 sapier if you don't know what this is for you don't have to change it ;-)
11:03 proller nobody know except you
11:04 sapier there's no way to hardcode settings for the variaty of our usecases
11:04 VanessaE what's good for someone in rural backwoods Russia (hi xyz ;) ) won't work well for someone in inner-city New York
11:04 sapier we've got adsl bandwith up to 100mbit ... no sane defaults will ever match this
11:04 proller okay, who can change it, for what and to what ?
11:05 sapier window size is supposed to be removed once old clients die out ... so about 2016 ;-)
11:06 sapier I would've set it to 1 in order to fix the old clients reliability issue ... but this doesn't work for high latency connections so 5 is a tradeoff between client bug occurance and speed
11:06 proller 1989         u32 datasize = m_max_packet_size * 2;  // Double it just to be safe
11:06 proller ---
11:06 sapier but you're right, I shoud've added a sentence about it, I'm gonna fix it
11:06 sapier that's one of the rare parts I haven't changed proller ;-P
11:06 proller increase recv buffer now!!!
11:07 sapier for what reason?
11:07 proller to 1500 or 65535
11:07 proller to increase packet size in future
11:07 sapier we can't increase packet size in a compatible way so why do it?
11:08 sapier and don't forget this is not the final solution
11:08 proller you can increase this bufeer now, and +2 versions increase sending packet
11:09 proller increasing very needed, because average block larger than current 500 bytes
11:09 proller but less than 1000
11:09 sapier we had a discussion about this some days ago as I suggested increasing it to 1200 too
11:10 proller recv buffer can be any, better to 10000-65000
11:11 proller it not broke anything, but helps in future
11:11 sapier basicaly the result was for ipv4 networks biggest reliable packet size is 576 bytes, if we assume we have ipv6 infrastructure everywhere we might be able to increase to 1360
11:12 proller WHAT
11:12 sapier proller if you suggested this yesterday it'd be already in
11:13 proller you can send 60k packet via udp
11:13 sapier ipv4 only guarantees to send datagrams with 576 bytes any other packet is legal to be dropped without notification
11:13 proller udp and notification ??
11:14 sapier of course most network components will support 14xx today
11:14 sapier yes udp does notifications e.g. port unreachable
11:15 proller inside game packets stream??
11:15 sapier I guess you won't get a port unreachable in middle of communication ;-) maybe at end ... but due to inet packet structure you might receive delayed packets later
11:15 proller can you increase it now to 64 without 2 days talking about nothing?
11:16 sapier not now
11:16 proller okay, in 5 mins ;)
11:16 proller and
11:16 proller -                       MeshUpdateResult r = m_mesh_update_thread.m_queue_out.pop_front();
11:16 proller +                       MeshUpdateResult r = m_mesh_update_thread.m_queue_out.pop_frontNoEx();
11:16 proller WAT
11:16 sapier no on next protocol fixes
11:17 proller +1 for removing NoEx  and making all pop_front no-exeption
11:17 sapier +1 too but will cause a lot of new issues
11:17 proller or add Ex version if very needed
11:20 proller when you will make next protocol fixes?
11:21 sapier don't know depends on what's gonna happen next days
11:53 proller -std::list<BufferedPacket> ReliablePacketBuffer::getTimedOuts(float timeout)
11:53 proller +std::list<BufferedPacket> ReliablePacketBuffer::getTimedOuts(float timeout,
11:53 proller +                                                                                                       unsigned int max_packets)
11:53 proller wat
11:53 proller you have 4sp tab?
11:55 VanessaE blame c55
11:56 darkrose blame the gnomes!
11:57 VanessaE hey darkrose
11:57 darkrose hi
11:58 VanessaE darkrose: well seeing as how gnome sucks anyways, I guess it's safe to blame it.
11:58 VanessaE wait, what? :)
11:58 xyz codestyle!
11:59 xyz http://i.imgur.com/3B8cOTT.png
11:59 xyz to illustrate the problem
12:01 sapier iirc we use 4 space tabs as default
12:01 proller sapier, and your diff (as diff) is WTF
12:01 proller everyone use 8
12:02 Exio4 8 sucks
12:02 sapier no you use 8
12:02 Exio4 but what is the point of having tabs if it needs to be fixed
12:02 xyz you aren't supposed to do this with tabs
12:02 xyz yup, Exio4 is right
12:02 VanessaE no, minetest code style uses 4.
12:02 VanessaE it's weird, but that's how it is.
12:04 VanessaE interesting, it isn't mentioned in the code style guidelines anymore.
12:04 sapier I'm gonna change it to two tabs only next time I touch it ... and I always wonder why it's not possible to mention those things PRIOR merge was done, it'd be way less work to fix it then
12:05 Exio4 just use spaces
12:05 sapier no I use the two tabs that seem to be used everywhere else in this file
12:06 sapier code style guidelines feel to be changed once a month the last half year so they're starting to be useless
12:06 VanessaE must have been in the old dev wiki/site/thingy that it was mentioned
12:07 VanessaE http://c55.me/minetest2/wiki/doku.php?id=contrib"Tabs are tabs, and they are visually 4 spaces in width"
12:07 VanessaE oops.
12:08 VanessaE well there it is/was.
12:08 proller щлфн
12:08 proller okay
12:08 VanessaE when was this requirement dropped?
12:09 sapier guess someone who demanded tabs to be 8 spaces dropped it ;-)
12:12 Exio4 tabs should be tabs, not X spaces
12:12 Exio4 something is wrong when talking about them in that way
12:13 sapier code is skrewd up on using a different tab width no matter what you do, it's only about HOW MUCH it's skrewed up
12:14 sapier if you don't believe, set tab width to 32 and you'll see it ;-)
12:14 Exio4 that is an extreme
12:14 Exio4 if you changed to six-spaces tabs, and the code looks fucked up, can anyone tell me what is the whole point of using tabs?
12:14 sapier it's meant as an extreme example ;-)
12:15 Exio4 whatever
12:15 sapier I could say size reduction ... but I guess those 3/7/5 bytes wont really make a difference
12:18 Exio4 my .git directory is about 1.4G
12:18 Exio4 would it be 1.5 if we were using spaces?
12:19 sapier as git uses compression and spaces can be compressed quite good I don't think so
12:19 ImQ009 joined #minetest-dev
12:23 VanessaE maybe because it takes less time to tap the tab key to indent, than to hold the space bar?
12:23 VanessaE one tab and *click*, there you are
12:23 VanessaE it's about convenience while coding, not about storage
12:25 xyz so we're using tabs instead of spaces because it takes less, erm, space?
12:25 xyz that's pretty autistic, i like it
12:26 Exio4 haha
12:26 Exio4 VanessaE: proper editors let you "place spaces" when you press the tab key
12:26 Exio4 and also remove them by a single backspace if wanted
12:28 sapier xyz "I could say" but I didn't say ;-)
12:28 bas080 joined #minetest-dev
12:28 xyz right
12:28 xyz is this something to argue about?
12:45 ImQ009 joined #minetest-dev
13:03 proller фпфшт сщкувгьзы щт ыргевщцт
13:03 proller again coredumps on shutdown
13:09 grrk-bzzt joined #minetest-dev
13:12 proller sapier, your changes still x3 slower than in freeminer..
13:12 proller but in FM only ~5 lines tuned..
13:17 sapier I didn't do a performance fix
13:17 sapier I fixed protocol reliability
13:18 sapier current implementation had major flaws resulting in "reliable" communication not beeing reliable at all
13:18 proller okay
13:18 sapier if there are performance improvements they're result of cleanup
13:19 sapier what test did you do to get that 3 times slower?
13:19 sapier what freeminer variant enet or udp?
13:20 hmmmm joined #minetest-dev
13:20 proller udp
13:20 sapier and what did you test?
13:20 proller transfer
13:20 proller enet must be 10x faster
13:20 sapier media transfer?
13:20 proller yes
13:21 sapier curl or non curl?
13:21 proller non of course
13:22 sapier hmm you should do a sha sum check on transfered media with freemine
13:22 sapier r
13:23 sapier old implementation did sometimes corrupt files on download due to assembling wrong packets in a split packet
13:35 proller LOG(a)
13:35 xyz yea but you know
13:35 proller NEED MORE LOCKS!!!
13:35 xyz client checks hashes of all files
13:36 sapier after reception too?
13:36 VanessaE proller:   http://onepeggenius.com/wp-content/uploads/2009/07/IMG_5775.jpg
13:36 VanessaE here ya go.
13:36 VanessaE wait, wrong kind of lox.. ;)
13:36 xyz sapier: yeah, like, all the time
13:36 xyz wait
13:37 xyz what do you mean? when else would it check hashes?
13:37 sapier if I recognize correctly files in cache arend hashed any longer as their filename should be the correct hash
13:38 sapier so once a file is there it's just assumed to be correct
13:39 sapier I've seen quite some broken files in cache, they never have been replaced by correct ones so I assume they aren't checked
13:39 xyz maybe, I only checked receiving code, to get into cache it should match sha1
13:40 xyz so your argument is invalid this time, try something else
13:40 ImQ009 joined #minetest-dev
13:43 proller if (current_bytes_transfered > (window_size*512/2))
13:43 proller why 512 hardcoded
13:43 sapier proller read back history discussion with hmmmm about 1 or two weeks ago or ask him
13:44 sapier xyz you're right, the files are checked prior storage ... so they most likely have been result of debugging while downloading
13:45 sapier but doesn't change the fact that communication in old implementation wasn't reliable or did you ever manage to transfer 150mb of texture data ? ;-)
13:46 sapier does anyone understand how curl support is supposed to work?
13:48 sapier yet it's interesting if how freeminer achieves to be three times faster ;-) even enet is only 30% faster for me
13:53 nore joined #minetest-dev
13:58 proller JMutexAutoLock internal(m_internal_mutex);
13:58 proller current_packet_too_late++;
13:58 proller sapier, can you never use JMutex* ,
13:58 proller ?
13:59 proller never more
14:03 PilzAdam joined #minetest-dev
14:06 sapier of course I can but why should I?
14:06 sapier there's one code location where I had to use it for good reason but in general the autolock avoids forgetting to unlock something
14:07 proller can you stop locking every line?
14:08 sapier no I can't stop locking variables that are accessed by multiple threads as their result may be invalid
14:08 xyz sapier: for some reason your enet felt slower to me than freeminer's enet, i.e. in FM I've managed to get instantly connected to localhost, while in your branch it took a second or two or three
14:09 proller okay, then  add more locks to be more stable
14:09 sapier xyz that's been a bug in that implementation it's fixed
14:09 xyz ah, i should check once again
14:09 xyz what branch?
14:09 xyz you have quite a lot
14:09 sapier once I merged it up to enet yes
14:11 sapier proller I don't say everything has to be locked, if you find a lock where you're absolutely sure it's not required we can remove it. I usually add a lock if I have reason to believe a variable is accessed by multiple threads. If something in implementation changes and this reason is gone, it's absolutely possible a now useless lock remains in code.
14:13 sapier actually the less locks the better ... but removing a lock and risiking data corruption isn't an option too
14:16 zat joined #minetest-dev
14:16 xyz sapier: so what branch should I use?
14:16 xyz for your enet with all fixes
14:16 sapier it's noit there for testing yet
14:18 sapier I'm looking for reason why curl seems to slow down with new mutexqueues
14:18 xyz ah, alright
14:27 proller ConnectionReceiveThread  ConnectionSendThread  for WAT???
14:35 proller sapier, maybe you fix one rare bug, but you make connection much more shitty than was
14:35 proller now you must add tcp to it to increase shitness
14:36 VanessaE proller: you know, you are of course free to help improve the situation.
14:36 VanessaE (without trying to shoehorn weather et al. into the system)
14:36 proller can you show exact lines which solves reliable bug ?
14:37 proller VanessaE, yes, enet.
14:37 VanessaE enet may or may not be the solution.
14:42 proller adding 25 locks is perfect solution!
14:43 proller sapier, when you first time start use threads ?
14:49 Exio4 what do you mean with that
14:49 Exio4 that english is worse than mine
14:52 werwerwer_ joined #minetest-dev
14:57 sapier proller I haven't read a single constructive suggestion from you since you started freeminer the only thing you seem to do since then is throwing in unprooven accusations and unprooven success storys about freeminer. If you have a real, at least somehow specific describet problem with vanilla minetest head I'm gonna investigat it but I'll ignore anything else starting NOW
14:58 xyz hey no need to get so upset over simple things like this
15:00 sapier It's not this simple thing I'm anoyed to waste time for comments that don't seem to be valid at least lots of times ... e.g. 3 times faster ... what does this tell 3seconds instead of one where you have accuracy of 2s?
15:00 proller TOP SECRET: http://paste.org.ru/?7oc144
15:00 NakedFury joined #minetest-dev
15:01 proller old minetest have ~ 250k stable transfer
15:01 xyz yes, I can agree to that, comparing two broken protocols isn't fair; that's why I'm waiting for your enet implementation
15:02 sapier 250k? you're doing something wrong I get 1.2mb stable on localhost
15:02 proller can you show exact lines which solves reliable bug ?
15:02 sapier of course not because it's been a design flaw
15:03 proller but now we have x3 more design flaw's [unprooven]
15:03 sapier old implementation did asume 65k packet Id's are enough to never receive a packet from last iteration
15:04 sapier ol tell me about them
15:04 proller threads, locks
15:05 sapier lol ... you're great proller :-)
15:10 proller you add 1200 lines to connection.cpp to solve imagined problem
15:11 sapier actually it'd been only 800 if I didn't include design corrections for tcp/enet in this version ... but this wouldn't matter you'd call anything bad
15:13 proller +400 .h
15:13 Yepoleb joined #minetest-dev
15:13 john_minetest joined #minetest-dev
15:19 sapier yes I'm evil I added 200 lines of rtt and jitter statistic definition to header ;-P
15:23 sapier did anyone else have issues with curl download through proxy server?
15:37 ImQ009_ joined #minetest-dev
15:53 proller joined #minetest-dev
15:54 sapier kahrl do you have any idea where to look for a 200ms delay in get requests beeing sent by httpfetch thread?
15:57 sapier requests are sent up to max parallelity then I see a 200ms delay
16:00 Jordach joined #minetest-dev
16:01 sapier ttps://github.com/minetest/minetest/commit/3aa28bc7a2014dcefd9f24517ff32223bec32a35 as of printfing I see a sequence of one 1ms sleep followd by two (sometimes three or more) 100ms sleeps
16:01 sapier https://github.com/minetest/minetest/commit/3aa28bc7a2014dcefd9f24517ff32223bec32a35
16:06 Neological joined #minetest-dev
16:13 ShadowNinja kahrl: https://github.com/minetest/minetest/commit/b03135548bbd9bcb73d14b067b96ec913404dd5f breaks announcement for me.  Any ideas why?  Config: http://ix.io/9Ne
16:14 ShadowNinja kahrl: It doesn't send any TCP packets on port 80 at all.
16:21 sapier https://gist.github.com/sapier/8357314 can someone check if I'm really right about this?
16:24 EvergreenTree joined #minetest-dev
16:24 EvergreenTree joined #minetest-dev
16:29 darkrose joined #minetest-dev
16:29 darkrose joined #minetest-dev
16:34 proller -       T pop_front(u32 wait_time_max_ms=0)
16:34 proller +       T pop_front(u32 wait_time_max_ms)
16:34 proller why?
16:35 sapier because the default value wasn't necessary as I had to touch all files where it was used anyway
16:37 sapier and if remember correctly timeout 0 would've meant something completely different for semaphore based mutex queue
16:37 sapier that's been the reason why I had to touch those locations
16:37 sapier and additionally it was wrong before too as timeout 0 actually was timeout 10ms
16:39 sapier hope I don't mix up this with another change as you didn't post file and line of change
16:53 rubenwardy joined #minetest-dev
17:10 BlockMen joined #minetest-dev
17:31 Ritchie joined #minetest-dev
17:32 john_minetest left #minetest-dev
17:44 Ritchie joined #minetest-dev
17:46 EvergreenTree joined #minetest-dev
17:46 EvergreenTree joined #minetest-dev
17:50 Gethiox2 joined #minetest-dev
17:52 Zeitgeist_ joined #minetest-dev
17:52 Ritchie joined #minetest-dev
17:57 iqualfragile joined #minetest-dev
17:57 sapier https://gist.github.com/sapier/8357314 can someone check if I'm really right about this? ANY comments?
18:12 kahrl sapier: uhh yeah... how did this work before?
18:13 VanessaE kahrl: when I applied that change, it increased my rate from <1.2 Mbps to 1.5-3 Mbps.  Add curl_parallel_limit = 32 to my minetest.conf and I got >28 Mbps.
18:13 VanessaE (tested several times)
18:13 sapier it waited till all transmissions have been completed
18:13 sapier then you get -1 ... but I should've failed on windows
18:14 Ritchie joined #minetest-dev
18:14 sapier merging it now
18:15 sapier done
18:17 kahrl ShadowNinja: can you try it again with the fix sapier just merged?
18:20 kahrl VanessaE: that change = changing "==" to "!="? or the original commit that added the "if"?
18:21 VanessaE kahrl: changing == to !=
18:21 kahrl ah
18:21 kahrl yeah, it should increase performance definitely
18:21 VanessaE and,
18:21 VanessaE since we seem to advise using nginx
18:22 VanessaE and clearly nginx can handle 32 simultaneous transfers at once with its default settings
18:22 VanessaE perhaps you should up it from the default of 8
18:22 VanessaE that is, the minetest client setting
18:23 kahrl not sure if setting aggressive defaults is a good idea
18:23 VanessaE well, idk if 32 is what I'd call aggressive, really
18:23 VanessaE based on my measurements,
18:23 sapier but there's still might be a race condition, if transmission is finished prior reading the fd's, at least if curl uses it's own thread to handle communication
18:23 kahrl if you open 32 simultaneous tcp transfers to one server, you effectively disable congestion control
18:23 VanessaE that's only about 64 kB per transfer request
18:24 sapier in this special case we would wait 100ms instead of processing the incoming data
18:25 kahrl sapier: I'm pretty sure curl doesn't use its own thread
18:25 kahrl if it did, it would probably export some kind of simple API to do async requests
18:26 sapier ok so this case propably can't happen
18:26 kahrl but all we get is curl_easy_perform and curl_multi_perform
18:26 troller joined #minetest-dev
18:26 proller joined #minetest-dev
18:26 sapier so the only thing to think about might be the 100ms timeout beeing a little bit too big
18:31 kahrl sapier: there is also the main thread that checks for completion of requests only every so often
18:32 sapier so probably that doesn't make a difference
18:33 sapier well case we choose to use tcp as successor of our current protocol curl download might be less important ... despite of beeing able to download from a dedicated server
18:34 VanessaE fwiw, I can max out at around 9 Mbps over pure UDP alone now with sapier's latest code that's in master
18:35 VanessaE that's perhaps two orders of magnitude faster than it used to be?
18:36 Ritchie joined #minetest-dev
18:42 Calinou joined #minetest-dev
18:50 proller i got 8-9mbps transfer for single player in fm..
19:02 EvergreenTree joined #minetest-dev
19:02 EvergreenTree joined #minetest-dev
19:06 jin_xi joined #minetest-dev
19:27 Ritchie joined #minetest-dev
20:19 mrtux joined #minetest-dev
20:22 ShadowNinja kahrl: Still nothing.
20:25 Ritchie joined #minetest-dev
20:31 Calinou joined #minetest-dev
20:43 sapier1 joined #minetest-dev
20:50 kaeza joined #minetest-dev
21:13 Miner_48er joined #minetest-dev
21:26 proller joined #minetest-dev
21:30 BlockMen left #minetest-dev
21:47 Evolykane joined #minetest-dev
21:48 tomasbrod joined #minetest-dev
22:11 VanessaE joined #minetest-dev
23:10 tomasbrod left #minetest-dev
23:17 EvergreenTree joined #minetest-dev
23:28 EvergreenTree joined #minetest-dev
23:32 iqualfragile ok, minetest realy needs a primitive for "on" and "off"
23:38 sapier https://github.com/sapier/minetest/tree/network_addon_tcp_2 tcp version (does anyone know how to reduce stall time of a tcp socket on linux?)
23:40 sapier https://github.com/sapier/minetest/tree/network_addon_enet_4 enet variant enet doesn't seem to provide reliable rtt and jitter information help is welcome
23:57 sapier left #minetest-dev

| Channels | #minetest-dev index | Today | | Google Search | Plaintext