Time  Nick       Message
00:25 Megaf      28 hours
00:26 Megaf      I think the new requirement for new minetest devs will be having a 1+ year old server with a database of at least 1000 players and testing Minetest in production :D
00:26 Megaf      I'm sure code would improve a lot
02:03 paramat    LOL Megaf. i don't want to, have time or money to, run a server, so you would then have lost all my very good contributions (highest number of contributions of active devs). also you would have lost lhofhansl who is an excellent dev
02:04 paramat    and others
02:05 Megaf      I guess we have to fine a way to balance things out
02:05 Megaf      well, I am partly to blame tho
02:05 paramat    it would make no difference to code quality and mostly kill MT dev :)
02:05 Megaf      I used to download patches from PRs and test them. I stopped doing that. So I'm not reviewing stuff myself... To easy to simply complain in hindsight
02:06 * Megaf    likes how some people reacts to his attacks. (when I get a legit and logical answer to a stupid attack for example)
02:06 paramat    no balancing is necessary. and we are now much better at listening to server owners, since we gained Shara and -hub channel. for example CSM
02:07 paramat    anyway heh no problem :)P
02:11 paramat    for progress bars, open an issue or code it :)
02:11 Megaf      is it even doable?
02:14 paramat    sfan5 we don't close 'bugs' due to lack of core dev interest, bugs are left open. there's good reason to bump then eventually close (with careful labelling) very old feature requests that have no core dev support, because otherwise they grow endlessly and then become unmanageable and therefore ignored. if you actually were active in managing issues like i am you would realise this ;)
02:14 paramat    erm not sure
02:16 paramat    no-one else has time to properly manage and process issues, i work on them a lot. most devs neglect feature requests due to lack of time, which is fine :)
02:17 paramat    i'm actively bumping feature requests for core dev attention after 1 year, i'm helping those who make feature requests
02:23 Megaf      I bumped a couple of things today
02:31 paramat    sfan5, to clarify, if any core dev supports the concept of a 'feature request' or 'request/suggestion' and comments so it will be kept open forever. so please go through them and consider them. i've labelled such issues with 'core dev opinions needed'
02:37 * paramat  is not paralax :)
02:39 paramat    progress bar for migrate players, good idea and i'm sure is possible
02:39 paramat    certainly needed
02:43 paramat    issue please :)
02:49 Megaf      Thanks, will do
02:53 Megaf      #7697
02:53 ShadowBot  https://github.com/minetest/minetest/issues/7697 -- [Request] Progress bar/status for --migrate-players
02:54 Megaf      feel free to adjust the title
13:52 Megaf      42 hours, still going
13:52 rubenwardy It shouldn't take that long
13:53 rubenwardy Is the file increasing at least?
14:17 Megaf      rubenwardy: which file?
14:22 rubenwardy The file you're migrating into
14:33 Megaf      minetest@Minetest:~/MinetestBin/worlds/world$ du -bs players*
14:33 Megaf      22067214        players
14:33 Megaf      4096    players.bak
14:34 Jordach    are you sure it's not disk I/O bottlenecking all the things
14:34 Jordach    especially on ARM based things like rPIs
14:40 p_gimeno   maybe this is sort of a progress indicator: watch ls -l players
14:40 Megaf      sent 416,844 bytes  received 6,265,552 bytes  183,079.34 bytes/sec
14:40 Megaf      total size is 21,514,254  speedup is 3.22
14:41 Megaf      just downloaded my players database
14:42 p_gimeno   does that mean you need to wait three times that?
15:30 Megaf      #7701
15:30 ShadowBot  https://github.com/minetest/minetest/issues/7701 -- [Bug] --migrate-players Does not work.
15:31 Megaf      Gentlemen ^ ^
16:44 sfan5      rubenwardy: i'm guessing it's "stuck" on disk io because beginSave() isn't called and sqlite3 thus periodically saves
16:52 sfan5      though, it should be logging each player it migrates to stdout
16:53 sfan5      right, so it's stuck on srcdb->listPlayers(player_list);
17:27 sfan5      ~telle paramat I feel it should be other way around, feature requests should only be closed if a coredev says "no this makes no sense / is not feasible / whatever" rather than closing them *unless* one says "yes this is worthwhile"
17:27 sfan5      ~tell paramat I feel it should be other way around, feature requests should only be closed if a coredev says "no this makes no sense / is not feasible / whatever" rather than closing them *unless* one says "yes this is worthwhile"
17:27 ShadowBot  sfan5: O.K.
17:34 Megaf      sfan5: https://github.com/minetest/minetest/issues/7701#issuecomment-418454052
17:34 sfan5      right that looks identical
17:34 sfan5      it's just buy deserializing every single of your 21k players
17:36 sfan5      busy*
17:37 Krock      so it's not just slow?
17:37 Megaf      and why would it deserialize everything first in the first place?
17:37 Megaf      why not one by one?
17:37 Megaf      why no feedback?
17:37 Megaf      why no IO
17:38 Megaf      PS. bug is valid, should be reopened
17:38 sfan5      by "everything" i mean "one by one"
17:38 Megaf      maybe title should be changed
17:38 sfan5      there is no feedback because this is just the listPlayers method, it's not designed to take a long time
17:38 sfan5      thus there is no progress indication
17:38 Megaf      18:03 <Megaf> great design ^ ^
17:40 Krock      well, vector<> only guarantees to hold 65k entries (size_t) but it's unlikely the issue
17:42 Krock      https://github.com/minetest/minetest/blob/master/src/database/database-files.cpp#L168 missing os.close() here?
17:42 Krock      *is
17:42 * Megaf    hugs Krock
17:42 sfan5      leaking fd would be the least problem here
17:42 sfan5      also std::ifstream should(?) close it anyway if it goes out of scope
17:43 Krock      not sure about it, was just positing it here in case someone knows more out of their head
17:43 Krock      *posting
17:43 Sires      Uuh
17:43 Sires      can someone help me in something, I accidently clicked in a button "resolve conversation" in a PR
17:43 Sires      is there a way to undo it?
17:44 sfan5      Krock: the problem here is that it has to deserialize the whole player just to get the name
17:44 Sires      Idk if I affected something, I'm not accostumated to github
17:44 sfan5      secondary issue is that those results are not even cached, so it goes on a search of the player again if you call getAuth()
17:44 Krock      stackoverflow says "No need for is.close()" https://stackoverflow.com/questions/748014/do-i-need-to-manually-close-an-ifstream
17:45 Sires      https://github.com/minetest/minetest/pull/7569/files <<< This is the pr
17:46 Krock      marked as unsolved
17:46 Sires      Thx :P
17:47 sfan5      Krock: for the usual working of the server, the "read every player" thing is sorta okay; but for migration we don't actually care about this, it would suffice to iterate over the files, *then* determining the player name and migrating them
17:47 sfan5      in other words: migration is twice (if not more) as slow as it could be
17:48 Krock      thing is, the migration is universal/generic
17:48 Krock      how long does it even take to read those files?
17:49 Krock      "too long" from what I've seen in the logs
17:49 sfan5      maybe it actually gets stuck in a loop along the way, but this is the first problem i can determine here
17:50 Krock      listPlayers could pass a RemotePlayer pointer since it's loaded anyway
17:50 Krock      but that again results i a huge memory consumption
17:51 sfan5      if creating iterators in C++ wasn't needlessly complicated, this is what you'd use
17:52 Krock      listPlayers also doesn't take the alternative file names in account on case-insensitive file systems
17:52 sfan5      it doesn't need to since it touches every single file, no?
17:53 Krock      ah right, it grabs the name from the "name" field
17:54 Krock      so  PlayerDatabase::getNextPlayer()  for all backends to kinda iterate through the stuff or maybe there's a more elegant container format
17:55 sfan5      you'd need to keep state, unless you want to skip ahead in the dir list every time
17:55 sfan5      which doesn't work if the listing is unsorted (is it?)
17:55 Krock      ah.. yes :/
17:56 Krock      it's a vector, so sorting isn't a thing by default
17:56 sfan5      right
17:57 Krock      but if that one could loaded by an iterator initializing function and unloaded afterwards, it could work around this sorting issue
17:58 Krock      so the file list would be the only cached thing in this process
17:58 Krock      and only limited to the files backend; other backends might need their own iterators
17:59 Krock      but if loading the player files twice should be avoided then that's one of the possible solutions. Although the best implementation of it is a different question
18:40 nerzhul    Oh i should develop the postgres player backend too... i missed it
18:48 Krock      processing some of the existing PRs might have higher priority for 5.0.0 than that I think