Time Nick Message 00:47 VanessaE can we PLEASE do something about the constant re-re-re-re-re-[...]-rewriting of player files? 00:48 us_0gb That would be nice. 00:48 VanessaE I've got almost 24,000 of the fucking things between five servers now on my machine. 00:48 us_0gb Ooh, that can't be easy on the drives. 00:48 VanessaE do they really *ALL* need to be constantly rewritten all the time? 00:49 VanessaE us_0gb: I've got a constant 3MB/sec write going on my disk now, which is not doing my server's performance any favors. 00:49 kahrl uh, that shouldn't happen if you are running 6f6f289db4 or later 00:50 VanessaE I am past that commit, 00:50 VanessaE but something's eating disk bandwidth for breakfast 00:51 VanessaE even when the servers are at a fairly low point (read: less than 5 users between them all) 00:51 kahrl I used inotifywait to verify that the commit fixed it 00:51 VanessaE ok so maybe player files aren't being rewritten then. something surely is. 00:51 kahrl then again, I'm not running many mods 00:52 us_0gb Could it be the map file? 00:52 VanessaE us_0gb: I would blame the map file if not for this: http://digitalaudioconcepts.com/vanessa/hobbies/minetest/stats.html 00:53 VanessaE look at say 0200 to 0500 or so. notice how few users there were during that time 00:53 VanessaE but notice the disk usage? 00:53 VanessaE it's constant. 00:53 VanessaE and atop blames minetet. 00:53 VanessaE minetest* 00:54 VanessaE can a mod be doing this somehow? 00:54 VanessaE (and if so, why isn't it causing the server in question to lag?) 00:55 kahrl I don't think a mod can do it but I'm not sure 00:55 us_0gb It might be worth shutting down the worlds, backing them up, and running them all as minetest_game to rule out module issues. 00:55 kahrl but if it did, you wouldn't notice because, well, it used to write all player files anyway up to 6f6f289db4 00:55 VanessaE us_0gb: two of the servers DO run minetest_game. 00:56 VanessaE (one heavily modded, one barely so) 00:56 us_0gb The heavily-modded one doesn't count for the purposes of the experiment. THe other might. 00:57 VanessaE the lightly-modded one is my so-called "Almost Vanilla" one, and it uses nothing that could conceivably do this 00:57 us_0gb It might only take one module in one of the worlds. Then again, I have no idea what is going on here. 00:57 VanessaE and in fact it's the most popular of the 6. 00:59 us_0gb I find that very surprising. The one I hear the most talk about is the fully-loaded creative server. 01:01 kahrl VanessaE: try this to gather some initial data: https://gist.github.com/kahrl/8679721 01:02 VanessaE actually I might know what it is. 01:04 Sokomine i prefer the fully-loaded creative and survival servers as well. guess the reason why there are so many players on vanessas vanilla server is that it has way less media data and players do not time out 01:04 VanessaE lemme see if what I just did fixed it first. 01:05 * VanessaE waits for her stats scripts to collect some data... 01:07 VanessaE yep, I think that did it. 01:07 ShadowNinja The server keeps all files loaded and iterates over every file in the players directory to save each player file, so even if only changes are saved you are still using a lot of disk, mostly reading though. 01:07 VanessaE It was PilzAdam's stats mod. 01:08 VanessaE which he kept hocking me a chinik to install. fucking thing was sitting there rewriting its stats files constantly. For all six servers. generating 3.5 MB/sec of disk traffic. 01:08 VanessaE for a 170K log file I think it was 01:09 ShadowNinja He uses timer and timer2, and confuses them. So instead of rewriting every 30 steps (3 seconds) it writes every step after the first 30. 01:10 ShadowNinja Removeing all timer2 lines except the last one and changing that to timer should fix it. 01:10 ShadowNinja -e 01:13 ShadowNinja https://github.com/PilzAdam/stats/pull/1 01:18 VanessaE well one thing's for sure; 01:18 VanessaE my server must be DAMNED fast if it was able to keep up with THAT much disk traffic :D 02:15 xyz til 3mb/s is considered much disk traffic 09:35 proller VanessaE, you know, where player files problem already fixed ;) 12:27 VanessaE proller: if you'd read the log, you'd notice the problem was actually one of PilzAdam's mods doing wrong things, and that the "player files problem" HAS been fixed. 12:27 proller no 8) 12:28 VanessaE no, what? 12:31 VanessaE xyz: I wouldn't have figured 3 (well, a bit over 3.5 actually) MB/sec was a lot of traffic, but I have to guess the issue stems from seek times in this case - 6 files being rewritten 5x/sec (or 10x/sec, I forget what the server step is) and competing with map loading/saving, player file loads/saves, media loads, and the other stuff my server has to do. 12:31 xyz yea that was stupid 12:31 xyz the bug 12:31 VanessaE spinning rust can get pretty damn slow when you're slamming the heads back and forth a lot :) 12:32 proller all player files still saved in every save step 12:32 proller * in mt 12:32 VanessaE proller: not according to the time stamps in my files. 12:32 xyz BUILDCRAFT MAKES ITS GLORIOUS RETURN 12:32 VanessaE xyz: oh? 12:32 xyz http://www.playstarz.com/ 12:32 VanessaE ha! 12:33 VanessaE they're gonna have a hard time being found in a search, I think 12:34 VanessaE I went looking just yesterday, hoping to find at least an outdated link or something (I don't have a copy of the client, and needed to try something)... kept finding nothing minecraft-related stuff for the mods (?) of those names. 12:35 VanessaE or something like that, I'm not entirely sure what they were but they weren't the minetest forks anyway 12:37 xyz could contact their domain or hosting provider 12:38 xyz or call the guy since there's some phone in whois output 19:35 sapier https://github.com/minetest/minetest/pull/1116 any comments? 19:51 ShadowNinja sapier: How does it work with old servers/clients? 19:51 sapier no major issues 19:52 sapier worst thing to happen is a old client may have to connect twice on media change at server 19:53 sapier in exchange client disconnect, intitialization, reconnect and join is quite better 20:01 ShadowNinja The server will only accept packets smaller than or equal to 1500 bytes? 20:02 sapier right now it will only accept packets < 1000 bytes 20:02 sapier I increased this to 1500 as this is the minimal required packet size to be compliant to ipv6 standard 20:03 ShadowNinja It should accept <= -1. That way you can add maximum packet size detection in the future and send with that size. 20:03 sapier no 20:03 sapier the size is limited by the receive buffer which has to have a fixed size 20:04 sapier and I don't think there will be a ipv8 anytime soon ;-) 20:04 sapier and still we want to get rid of this protocol in mid to long term 20:05 ShadowNinja L"Sorry, too many concurrent connects try again in a few minutes." --> L"Sorry, too many people are connecting at the same time. Try again in a few minutes." 20:06 sapier yes that's very important 20:06 ShadowNinja You also mix "x< 4gb memory at once 20:13 ShadowNinja That's not so bad then. 20:13 ShadowNinja http://digitalaudioconcepts.com/vanessa/hobbies/minetest/stats.html Looks like there's a mem leak somewhere... Unless mods could be doing that? 20:14 sapier could be loaded blocks too 20:15 sapier I'm not sure about it but I think mapblocks aren't unloaded if not necessary ... but I'm not that used to mapping code maybe someone else can answer this question 20:19 sapier but it's at least suspicious enough to do further investigations 20:33 sapier proller I've got a strange issue with nohead clients, I remember them moving around, but they don't do this any more. do you have any idea how the "moving" did work? 20:34 proller random input? 20:34 proller where is servers with 200mb textures? 20:35 sapier where to get random input? 20:36 xyz --random-input 20:36 proller sapier, do not fix imagined issues 20:36 sapier really? *g* funny ... 20:37 sapier vanessaE's hdx 256 mb texture pack is about 180 mb on her full game, it's about 5 min downloading so this is far from unrealistic to have at a server 20:37 proller it one server vwith average 0 players 20:38 proller you want to make problems to clients of all other servers 20:38 sapier yea ok with 0 players download may take about 7-10 minutes true 20:39 sapier and noone needs more then 640kb memory as bill gates said years ago ;-P 20:40 VanessaE bill gates never said that :P 20:40 proller every servers have 8-64G memory 20:41 xyz max_concurrent_joins that reta^W stupid IMO 20:41 xyz s/that/that's/ 20:41 xyz esp. the default limit set to 3 20:42 proller need set it to 1 ! 20:42 xyz zero 20:43 sapier ok wait 20:44 sapier https://github.com/minetest/minetest/pull/1116 any >>>USEFULL<<< comments? 20:44 sapier forgit I need to be very very very precise in here 20:46 xyz yea what's non-constructive in my comment? 20:46 xyz it's stupid that you set max_concurrent_joins to 3 20:46 VanessaE and for the record, NONE of my servers use big texture packs like that one. They're all at the defaults that come with their respective games/mods. 20:47 xyz it's stupid because it's only reasonable for servers that have 200 mb texture packs, 20 simultaneous client connecting to it and less than 4 gb of ram 20:47 xyz but you limit it for everybody 20:47 sapier1 it's a configurable setting 20:47 sapier1 not a limit 20:47 xyz its default is 3 20:47 xyz that's unreasonable 20:48 xyz you wanted comments, here they are 20:48 sapier1 and it's just a limit for simultaneous connect! 20:48 xyz tbh I think it should be turned off by default 20:48 xyz well what else could it be? 20:48 sapier1 in practice you wont realize that limit very often 20:48 xyz you just said "minetest will never get popular" 20:48 sapier1 but I dont care about the number anyway 20:49 xyz if you don't care about the number don't write the number 20:49 xyz why did you write 3? 20:49 xyz what was the reason? 20:49 sapier1 so if you think 2000000 is ok I set it to that value too 20:49 VanessaE sapier1: I think the point here is that the default limit should either be so insanely high as to be essentially non-existent, or it should at least be relatively high anyway 20:49 VanessaE and that it can be *lowered* if it turns out to be a problem for a given piece of hardware. 20:50 sapier1 because I had to set some value and if you don't need to download textures clients don't stay in "connecting" state for very long 20:50 VanessaE I would guess, based on my own stats, that a limit of around 20 for this particular variable is more reasonable 20:50 xyz yes as I said I think it should be disabled by default 20:50 VanessaE (if not effectively disabled) 20:51 sapier1 VanessaE especially you are very lucky noone did attack your servers by now it'd be quite easy to kill em 20:51 VanessaE indeed so 20:51 VanessaE however OVH has some kind of DDoS protection in place, or so they say 20:52 xyz haha 20:52 sapier1 your default game server could allocate more then 1.5GB ram at once ... and it's only the most simple one 20:52 VanessaE nevertheless, your idea is a good one in general 20:52 xyz hey 20:52 xyz i like it how you care about memory 20:52 xyz yet you completely ignored my bug report 20:52 sapier1 that ddos protection wont have any effect as client doesn't need to send a lot of packets 20:52 xyz the one where you make server allocate a lot of ram 20:53 sapier1 I don't make server allocate ram that code is in there for ages 20:53 xyz yeah well 20:53 sapier1 you could do this with ANY minetest server out there 20:54 xyz wow! 20:54 sapier1 I wont add a "disable" as I don't see a reason to add that complexity, are you fine with 1000 as default? 20:54 xyz probably 20:55 xyz the other question is what's the point of this setting if no one is going to use it 20:55 xyz anyway, didn't reviewed the code, you'll have to find someone else to do that 20:55 Calinou VanessaE, their DDoS protection is "shut up and shut down your server" 20:55 Calinou OVH has a track record of doing this :P 20:55 VanessaE heh 20:55 VanessaE well I don't plan to rely on it anyway :P 20:56 sapier1 VannessaE how mad would you be at me if I crash your server ? 20:56 VanessaE erm 20:56 xyz that's so childish 20:56 VanessaE well ask the 27 users who are on right now :P 20:57 xyz what are you trying to prove? 20:57 xyz that it's easy to DoS mt server? 20:57 sapier1 I can remove it too but I don't wanna hear complains about "that bad guy crashed my server" later 20:57 xyz your patch won't fix it 20:57 xyz yeah sure 20:57 xyz well I honestly don't care much 20:57 sapier1 and right now someone wouldn't even have to do an attack on purpose 20:57 xyz proller: make sure to revert this when merging btw 21:24 ShadowNinja max_concurrent_joins is the people downloading media at the same time, not the number of people on the server. I think about 8 is reasonable. 21:32 ShadowNinja Is there a way for mods to get the server's version? 21:33 sapier1 I think this wasn't added in favor of feature support flags 21:40 xyz ShadowNinja: i know right? we already have max_players 21:40 xyz this doesn't make the limit less stupid 21:41 xyz anyway now it feels like i'm only arguing without doing anything so let's let someone else decide if we need it 21:43 sapier1 right now I don't see anyone who likes it. Imho it would harden servers to a quite simple sort of attack but guess hardening is as popular as security in minetest community ;-P 21:44 proller xyz one big 1000 lines conflict expected ;( 21:45 sapier1 I don't care about freeminer conflicts proller you did fork it's your issue ;-P 21:46 proller no problem if conflicts good useful code 21:46 proller but if it remix and fighting with imagined problems.. 21:47 sapier1 It's your problem if you wanna wait till a bug really causes a problem instead of fixing it before 21:48 proller 01:48:27: ERROR[main]: Fatal error n=6600 : not enough memory 21:49 proller hellp from mobf 21:49 proller hello 21:49 sapier1 did you allow a client to fly along whole map within 20 seconds again? 21:49 proller no 21:50 sapier1 why do you think this is a mobf issue? 21:50 proller we have 6000+ road with fast allowed 21:50 proller 01:50:39: VERBOSE[main]: MOBF: spawning animal_vombie:vombie_spawner_at_night entity by at_night_spawner at position (7376.000000,3311.000000,91.000000) 21:50 proller 1000++++ 21:50 proller in log 21:51 sapier1 then add as much memory as required to support it 21:51 sapier1 or disable vombies if your server can't handle them in combination with fast clients 21:51 proller i have 32g 21:51 proller 1818M 1030M uwait 6 168:13 100.39% freeminerserver 21:51 sapier1 I thought memory will never bee an issue? 21:51 VanessaE O_O 21:51 proller its lua error 21:52 sapier1 no it isn't if you trigger 10000 blocks to be loaded you'll trigger 10000*mob density spawners 21:52 proller 3 minutes of shutdown again.. 21:53 VanessaE ... -> #minetest ? 21:53 proller but lua cant use more than 1-1.5G memory 21:54 sapier1 obviously it can 21:54 sapier1 wait lua itself can't? 21:55 proller maybe limit 21:55 sapier1 -->minetest 22:17 paramat hi celeron55, sapier1, please merge the per player custom skybox https://github.com/minetest/minetest/pull/960 this is essential for my space realms and moonrealm mod :) 22:22 sapier1 I don't see a final decision of that discussion in there. does this collide with recent time based coloring? 22:22 ShadowNinja sapier1: Right, I want it to have the server reply to CTCP VERSION with it's version. 22:23 ShadowNinja sfan5: You've got a good use for that pull now. :-) 22:24 sapier1 I wasn't involved in that dscussion but if I remember correct version wasn't made available to mods to avoid silly modders write code like if version == X.X.X then enable feature y 22:24 ShadowNinja I've added a return parameter to chatcommand functions. Normally it is simply sent with chat_send_player, but it can be captused and sent to, eg, the server's terminal or a box in a formspec(for a commandblock). I'd also like to add a scuuess return value. 22:25 sapier1 I suggest looking up that discussion prior adding that version command 22:25 ShadowNinja Yes, it was added because version comparison is more complicated than "if minetest.feature_i_need then ... end 22:25 ShadowNinja " 22:26 sapier1 and less error prone 22:26 sapier1 but as I said I wasn't involved by that time and neither do I really care now 22:30 sapier1 I'm gonna merge #1116 after removing the max concurrent fix this weekend so everyone is welcome to review and look for REAL problems. 22:30 ShadowBot https://github.com/minetest/minetest/issues/1116 -- Add proper client initialization by sapier 22:41 * proller facepalm 22:44 sapier1 nonsense comments to minetest please 22:45 proller nonsense pulls to trash please 22:46 sapier1 I'm really sorry irc doesn't have a ignore feature 22:46 sapier1 did you have any VALID comment about minetest development the last two months? 22:47 sapier1 maybe I missed it in between all of your "I'm pissed" comments 22:48 ShadowNinja sapier1: Actually it does, /mode +q server-side, and /ignore cllient-side. 22:48 sapier1 sadly my client doesn't support /ignore .. already tried 22:49 sapier1 and as far as I know +q is muting I don't thing it's fair to be that rude :-) 22:50 VanessaE proller G*d damn it stop trolling. ENOUGH already. 22:51 sapier1 but back to 1116 I tested it with loop join/leave looped texture downloading both with no other clients and with 5 10 30 clients logged in ... just standing there. 22:52 celeron55 i'm very busy with non-minetest stuff as people have probably very easily noticed, but i'll comment about two things: 22:53 paramat sapier1, pull 960: there are good suggestions for future improvements, but nothing that need delay the original pull which is extremely useful to modders as it is, AFAIK it does not conflict with anything. 22:53 sapier1 #960 22:53 ShadowBot https://github.com/minetest/minetest/issues/960 -- set_sky()/override_day_night_ratio() by celeron55 22:54 celeron55 1) the only "issue" in #960 is the slightly non-perfect network protocol, but it's probably good enough 22:54 ShadowBot https://github.com/minetest/minetest/issues/960 -- set_sky()/override_day_night_ratio() by celeron55 22:54 celeron55 and nobody is going to improve on it so that's a reason to merge it as is 22:55 sapier1 good enough for me, ShadowNinja it's scriptapi to any veto from your side? 22:55 celeron55 2) NEVER add a function in the API to read the version of the server; it is missing completely intentionally in order to allow forks of minetest (and this was decided way before freeminer became to exists; do not let it affect your standing on the feature in any way) 22:56 celeron55 exist* 22:57 celeron55 (it could be that whoever here has to fork minetest some day for any purpose, and a version-based feature check immediately becomes a pain in the ass at such point) 22:58 celeron55 (to both ways) 23:09 ShadowNinja sapier1: Nope. 23:09 ShadowNinja celeron55: I don't want to check the version, only use it as a response in a CTCP VERSION reply in the IRC mod. 23:10 sapier1 does anyone volonter to rebase it to current master? 23:10 celeron55 but if you make it possible, then almost any modder will use it for the wrong purpose 23:11 celeron55 we know from experience that modders are evil and will abuse anything that they possibly can 8) 23:12 kahrl see the second latest post of "The Old New Thing" for the sadness caused by getVersion type APIs 23:12 kahrl and there aren't even (officially supported) forks of Windows 23:12 ShadowNinja We can add "DO NOT DEPEND ON THE VERSION FOR CHECKING FEATURES, USE IT ONLY FOR DISPLAY. USE MINETEST.FEATURES OR 'if minetest.thing_i_need' INSTEAD." TO THE DOC. 23:12 ShadowNinja -Caps at the end. 23:12 celeron55 ShadowNinja: doesn't matter; they'll say "works for me" 23:12 sapier1 which will be ignoread 23:13 sapier1 -a 23:13 kahrl http://blogs.msdn.com/b/oldnewthing/archive/2014/01/28/10493174.aspx 23:13 ShadowNinja We can't do anything about moders that refuse to read. 23:13 celeron55 yes we can 23:13 celeron55 and it's simple 23:13 ShadowNinja And they'll break things in worse ways even without this feature. 23:13 celeron55 i already said it 8) 23:15 ShadowNinja So, the official way to get the version is "os.execute('git describe --tags --always > hash.txt ...')"? 23:15 celeron55 from the standpoint of mod code, there are no versions 23:15 ShadowNinja I'll add that after the patch I'm working on now. 23:16 ShadowNinja Or is there something else I can read? 23:16 sapier1 git describe won't work for a installed server 23:16 celeron55 what kind of benefit do you expect to get from giving out a minetest version number for CTCP VERSION? 23:16 sapier1 imho it should contain the version of the irc client not the server version 23:17 celeron55 i guess it should say "minetest irc mod version " 23:21 ShadowNinja celeron55: That's what it does now, but I CTCPd a server it check it's Minetest version. 23:21 ShadowNinja * Received CTCP-VERSION answer from Merlin: Minetest IRC mod 0.2.0 23:22 kahrl why not simply check the serverlist? 23:22 ShadowNinja I want to add " running on Minetest 0.4.x-13579ace-dirty" or some such to that. 23:23 ShadowNinja The server list isn't that precise, and although in this case it will work, announcement is not madatory. 23:24 ShadowNinja Actually, it isn't on the list, so that won't work. 23:24 kahrl why do you need the version anyway? 23:24 ShadowNinja Comments on http://ix.io/abZ ? 23:24 ShadowNinja (I plan to open a PR...) 23:24 kahrl I guess you could connect to the server to see its version 23:25 ShadowNinja To figure out if it's running 0.4.8 or 0.4.9. 23:26 ShadowNinja But why do we need these workarounds? if minetest.thing_i_need is far easier to implement than a version parser, so lazy moders won't write one. 23:27 kahrl it's easier if you know about it 23:27 kahrl but lazy modders don't read, so they don't know about it 23:28 kahrl but they know they need a feature from version >=X, so they look for a function that checks for version X 23:28 ShadowNinja They'll have to read quite a bit to get a working version parser. 23:28 Exio4 modders will abuse the function anyway 23:29 Exio4 even if needs to be run in an integrated brainfuck interpreter 23:29 ShadowNinja Modders have abused just about every function in the MT API. 23:29 Exio4 most modders aren't "pro coders" 23:29 ShadowNinja But if they're looking for a version checking function they'll see the notice and use the recommended and simpler method. 23:30 kahrl no, the notice is in a separate line so it won't be read 23:31 Exio4 or because they'll be using more than a single feature from 0.4.9 23:32 Exio4 ShadowNinja: modders are like long time users, http://xkcd.com/1172/