| Time |
Nick |
Message |
| 00:09 |
|
ANAND_ joined #minetest-dev |
| 00:15 |
|
opal joined #minetest-dev |
| 00:25 |
Megaf |
28 hours |
| 00:26 |
Megaf |
I think the new requirement for new minetest devs will be having a 1+ year old server with a database of at least 1000 players and testing Minetest in production :D |
| 00:26 |
Megaf |
I'm sure code would improve a lot |
| 01:59 |
|
paramat joined #minetest-dev |
| 02:03 |
paramat |
LOL Megaf. i don't want to, have time or money to, run a server, so you would then have lost all my very good contributions (highest number of contributions of active devs). also you would have lost lhofhansl who is an excellent dev |
| 02:04 |
paramat |
and others |
| 02:05 |
Megaf |
I guess we have to fine a way to balance things out |
| 02:05 |
Megaf |
well, I am partly to blame tho |
| 02:05 |
paramat |
it would make no difference to code quality and mostly kill MT dev :) |
| 02:05 |
Megaf |
I used to download patches from PRs and test them. I stopped doing that. So I'm not reviewing stuff myself... To easy to simply complain in hindsight |
| 02:06 |
* Megaf |
likes how some people reacts to his attacks. (when I get a legit and logical answer to a stupid attack for example) |
| 02:06 |
paramat |
no balancing is necessary. and we are now much better at listening to server owners, since we gained Shara and -hub channel. for example CSM |
| 02:07 |
paramat |
anyway heh no problem :)P |
| 02:11 |
paramat |
for progress bars, open an issue or code it :) |
| 02:11 |
Megaf |
is it even doable? |
| 02:14 |
paramat |
sfan5 we don't close 'bugs' due to lack of core dev interest, bugs are left open. there's good reason to bump then eventually close (with careful labelling) very old feature requests that have no core dev support, because otherwise they grow endlessly and then become unmanageable and therefore ignored. if you actually were active in managing issues like i am you would realise this ;) |
| 02:14 |
paramat |
erm not sure |
| 02:16 |
paramat |
no-one else has time to properly manage and process issues, i work on them a lot. most devs neglect feature requests due to lack of time, which is fine :) |
| 02:17 |
paramat |
i'm actively bumping feature requests for core dev attention after 1 year, i'm helping those who make feature requests |
| 02:23 |
Megaf |
I bumped a couple of things today |
| 02:31 |
paramat |
sfan5, to clarify, if any core dev supports the concept of a 'feature request' or 'request/suggestion' and comments so it will be kept open forever. so please go through them and consider them. i've labelled such issues with 'core dev opinions needed' |
| 02:37 |
* paramat |
is not paralax :) |
| 02:39 |
paramat |
progress bar for migrate players, good idea and i'm sure is possible |
| 02:39 |
paramat |
certainly needed |
| 02:43 |
paramat |
issue please :) |
| 02:49 |
Megaf |
Thanks, will do |
| 02:53 |
Megaf |
#7697 |
| 02:53 |
ShadowBot |
https://github.com/minetest/minetest/issues/7697 -- [Request] Progress bar/status for --migrate-players |
| 02:54 |
Megaf |
feel free to adjust the title |
| 04:23 |
|
reductum joined #minetest-dev |
| 05:25 |
|
Darcidride__ joined #minetest-dev |
| 05:42 |
|
ssieb joined #minetest-dev |
| 06:50 |
|
YuGiOhJCJ joined #minetest-dev |
| 07:30 |
|
Gael-de-Sailly joined #minetest-dev |
| 09:57 |
|
Jordach joined #minetest-dev |
| 10:00 |
|
jordach_ joined #minetest-dev |
| 10:03 |
|
p_gimeno joined #minetest-dev |
| 10:14 |
|
Fixer joined #minetest-dev |
| 11:37 |
|
CBugDCoder joined #minetest-dev |
| 13:36 |
|
calcul0n joined #minetest-dev |
| 13:52 |
Megaf |
42 hours, still going |
| 13:52 |
rubenwardy |
It shouldn't take that long |
| 13:53 |
rubenwardy |
Is the file increasing at least? |
| 14:08 |
|
Helenah joined #minetest-dev |
| 14:17 |
Megaf |
rubenwardy: which file? |
| 14:17 |
|
jordach_ joined #minetest-dev |
| 14:22 |
rubenwardy |
The file you're migrating into |
| 14:33 |
Megaf |
minetest Minetest:~/MinetestBin/worlds/world$ du -bs players* |
| 14:33 |
Megaf |
22067214 players |
| 14:33 |
Megaf |
4096 players.bak |
| 14:34 |
Jordach |
are you sure it's not disk I/O bottlenecking all the things |
| 14:34 |
Jordach |
especially on ARM based things like rPIs |
| 14:40 |
p_gimeno |
maybe this is sort of a progress indicator: watch ls -l players |
| 14:40 |
Megaf |
sent 416,844 bytes received 6,265,552 bytes 183,079.34 bytes/sec |
| 14:40 |
Megaf |
total size is 21,514,254 speedup is 3.22 |
| 14:41 |
Megaf |
just downloaded my players database |
| 14:42 |
p_gimeno |
does that mean you need to wait three times that? |
| 14:52 |
|
p_gimeno joined #minetest-dev |
| 14:55 |
|
CBugDCoder joined #minetest-dev |
| 15:18 |
|
Icedream joined #minetest-dev |
| 15:24 |
|
TC01 joined #minetest-dev |
| 15:30 |
Megaf |
#7701 |
| 15:30 |
ShadowBot |
https://github.com/minetest/minetest/issues/7701 -- [Bug] --migrate-players Does not work. |
| 15:31 |
Megaf |
Gentlemen ^ ^ |
| 15:34 |
|
crazy_baboon joined #minetest-dev |
| 15:36 |
|
SLBelden joined #minetest-dev |
| 15:53 |
|
SLBelden joined #minetest-dev |
| 15:55 |
|
SLBelden left #minetest-dev |
| 16:28 |
|
sys4 joined #minetest-dev |
| 16:44 |
sfan5 |
rubenwardy: i'm guessing it's "stuck" on disk io because beginSave() isn't called and sqlite3 thus periodically saves |
| 16:46 |
|
Krock joined #minetest-dev |
| 16:48 |
|
ssieb joined #minetest-dev |
| 16:52 |
sfan5 |
though, it should be logging each player it migrates to stdout |
| 16:53 |
sfan5 |
right, so it's stuck on srcdb->listPlayers(player_list); |
| 17:10 |
|
Cornelia joined #minetest-dev |
| 17:22 |
|
basxto joined #minetest-dev |
| 17:27 |
sfan5 |
~telle paramat I feel it should be other way around, feature requests should only be closed if a coredev says "no this makes no sense / is not feasible / whatever" rather than closing them *unless* one says "yes this is worthwhile" |
| 17:27 |
sfan5 |
~tell paramat I feel it should be other way around, feature requests should only be closed if a coredev says "no this makes no sense / is not feasible / whatever" rather than closing them *unless* one says "yes this is worthwhile" |
| 17:27 |
ShadowBot |
sfan5: O.K. |
| 17:34 |
Megaf |
sfan5: https://github.com/minetest/minetest/issues/7701#issuecomment-418454052 |
| 17:34 |
sfan5 |
right that looks identical |
| 17:34 |
sfan5 |
it's just buy deserializing every single of your 21k players |
| 17:36 |
sfan5 |
busy* |
| 17:37 |
Krock |
so it's not just slow? |
| 17:37 |
Megaf |
and why would it deserialize everything first in the first place? |
| 17:37 |
Megaf |
why not one by one? |
| 17:37 |
Megaf |
why no feedback? |
| 17:37 |
Megaf |
why no IO |
| 17:38 |
Megaf |
PS. bug is valid, should be reopened |
| 17:38 |
sfan5 |
by "everything" i mean "one by one" |
| 17:38 |
Megaf |
maybe title should be changed |
| 17:38 |
sfan5 |
there is no feedback because this is just the listPlayers method, it's not designed to take a long time |
| 17:38 |
sfan5 |
thus there is no progress indication |
| 17:38 |
Megaf |
18:03 <Megaf> great design ^ ^ |
| 17:40 |
Krock |
well, vector<> only guarantees to hold 65k entries (size_t) but it's unlikely the issue |
| 17:42 |
Krock |
https://github.com/minetest/minetest/blob/master/src/database/database-files.cpp#L168 missing os.close() here? |
| 17:42 |
Krock |
*is |
| 17:42 |
* Megaf |
hugs Krock |
| 17:42 |
sfan5 |
leaking fd would be the least problem here |
| 17:42 |
sfan5 |
also std::ifstream should(?) close it anyway if it goes out of scope |
| 17:43 |
|
Sires joined #minetest-dev |
| 17:43 |
Krock |
not sure about it, was just positing it here in case someone knows more out of their head |
| 17:43 |
Krock |
*posting |
| 17:43 |
Sires |
Uuh |
| 17:43 |
Sires |
can someone help me in something, I accidently clicked in a button "resolve conversation" in a PR |
| 17:43 |
Sires |
is there a way to undo it? |
| 17:44 |
sfan5 |
Krock: the problem here is that it has to deserialize the whole player just to get the name |
| 17:44 |
Sires |
Idk if I affected something, I'm not accostumated to github |
| 17:44 |
sfan5 |
secondary issue is that those results are not even cached, so it goes on a search of the player again if you call getAuth() |
| 17:44 |
Krock |
stackoverflow says "No need for is.close()" https://stackoverflow.com/questions/748014/do-i-need-to-manually-close-an-ifstream |
| 17:45 |
Sires |
https://github.com/minetest/minetest/pull/7569/files <<< This is the pr |
| 17:46 |
Krock |
marked as unsolved |
| 17:46 |
Sires |
Thx :P |
| 17:47 |
sfan5 |
Krock: for the usual working of the server, the "read every player" thing is sorta okay; but for migration we don't actually care about this, it would suffice to iterate over the files, *then* determining the player name and migrating them |
| 17:47 |
sfan5 |
in other words: migration is twice (if not more) as slow as it could be |
| 17:48 |
Krock |
thing is, the migration is universal/generic |
| 17:48 |
Krock |
how long does it even take to read those files? |
| 17:49 |
Krock |
"too long" from what I've seen in the logs |
| 17:49 |
sfan5 |
maybe it actually gets stuck in a loop along the way, but this is the first problem i can determine here |
| 17:50 |
Krock |
listPlayers could pass a RemotePlayer pointer since it's loaded anyway |
| 17:50 |
Krock |
but that again results i a huge memory consumption |
| 17:51 |
sfan5 |
if creating iterators in C++ wasn't needlessly complicated, this is what you'd use |
| 17:52 |
Krock |
listPlayers also doesn't take the alternative file names in account on case-insensitive file systems |
| 17:52 |
sfan5 |
it doesn't need to since it touches every single file, no? |
| 17:53 |
Krock |
ah right, it grabs the name from the "name" field |
| 17:54 |
Krock |
so PlayerDatabase::getNextPlayer() for all backends to kinda iterate through the stuff or maybe there's a more elegant container format |
| 17:55 |
sfan5 |
you'd need to keep state, unless you want to skip ahead in the dir list every time |
| 17:55 |
sfan5 |
which doesn't work if the listing is unsorted (is it?) |
| 17:55 |
Krock |
ah.. yes :/ |
| 17:56 |
Krock |
it's a vector, so sorting isn't a thing by default |
| 17:56 |
sfan5 |
right |
| 17:57 |
|
Jordach joined #minetest-dev |
| 17:57 |
Krock |
but if that one could loaded by an iterator initializing function and unloaded afterwards, it could work around this sorting issue |
| 17:58 |
Krock |
so the file list would be the only cached thing in this process |
| 17:58 |
Krock |
and only limited to the files backend; other backends might need their own iterators |
| 17:59 |
Krock |
but if loading the player files twice should be avoided then that's one of the possible solutions. Although the best implementation of it is a different question |
| 18:40 |
nerzhul |
Oh i should develop the postgres player backend too... i missed it |
| 18:48 |
Krock |
processing some of the existing PRs might have higher priority for 5.0.0 than that I think |
| 18:56 |
|
CBugDCoder joined #minetest-dev |
| 19:25 |
|
Jordach joined #minetest-dev |
| 19:40 |
|
Gael-de-Sailly joined #minetest-dev |
| 21:08 |
|
longerstaff13-m joined #minetest-dev |
| 21:09 |
|
Jordach joined #minetest-dev |
| 21:48 |
|
fwhcat joined #minetest-dev |
| 22:07 |
|
wowaname joined #minetest-dev |
| 22:07 |
|
YuGiOhJCJ joined #minetest-dev |
| 22:20 |
|
Jordach joined #minetest-dev |
| 22:56 |
|
ANAND joined #minetest-dev |
