Minetest logo

IRC log for #minetest, 2016-09-07

| Channels | #minetest index | Today | | Google Search | Plaintext

All times shown according to UTC.

Time Nick Message
00:01 Yst I know that there's normally a German speaker in the room, but I don't see them around right now.
00:02 GunshipPenguin Lots of German speakers active in the Minetest community, that's why I asked here :)
00:02 swift1102 joined #minetest
00:03 Yst I haven't been active for a while, so I only know the one. Sorry.
00:03 KaadmY joined #minetest
00:03 Yst Minetest is an awesome game, but life has gotten too busy.
00:06 swift110 joined #minetest
00:06 swift110 joined #minetest
00:07 * Chinchou stares in disbelief at the tutorial on cmake's website
00:08 Tmanyo joined #minetest
00:08 Chinchou Managing my Makefiles by hand is easier than this.
00:09 KaadmY Chinchou: same
00:10 Chinchou Cmake is nice for the user, but...  Dear gods, what were they smoking when they wrote that tutorial?
00:10 Chinchou Is it really this bad, or are they just crap at explaining?
00:11 KaadmY nah
00:11 KaadmY cmake is really that bad
00:11 Chinchou D:
00:12 Chinchou That said, even Makefiles are annoying...
00:12 KaadmY makefiles are pretty easy imo
00:13 Chinchou Hmm...
00:13 Chinchou I should probably try to re-learn it.  Do you have any good resources for that?
00:13 KaadmY not really
00:14 * Chinchou has a headache from looking at cmake stuff
00:14 KaadmY i learned mostly from looking at others' makefiles
00:14 Chinchou Gack
00:14 Chinchou I've never been good with that.
00:14 KaadmY same
00:17 Chinchou Been trying to go through some SDL2 tutorials...  Got spoiled by Allegro 4 and am having one hell of a time here
00:17 Chinchou A4's documentation was some of the best I've seen, and with so much other material, it was easy to pick up.
00:18 Chinchou Seriously considering staying with A4, but it is so old...
00:55 xunto joined #minetest
01:21 Hirato joined #minetest
01:24 HonoredGlory https://forum.minetest.net/viewtopic.php?f=3&t=15476
01:26 Void7 joined #minetest
01:40 siva_machina joined #minetest
01:44 Yst joined #minetest
02:05 swift110-phone joined #minetest
02:07 Freejack joined #minetest
02:37 Yves1 joined #minetest
02:43 Megaf joined #minetest
03:11 agaran Chinchou: think what if they haven't smoke to write that tutorial...
03:17 swift110-phone joined #minetest
03:26 AntumDeluge joined #minetest
03:40 lisac joined #minetest
03:47 MinetestBot [git] paramat -> minetest/minetest_game: Change WTFPL to MIT (part 3) 6171fb4 https://git.io/vindO (2016-09-07T04:44:53+01:00)
04:37 Hawk777 joined #minetest
05:09 AnotherBrick joined #minetest
05:20 siva_machina joined #minetest
05:22 namach joined #minetest
05:34 DMackey joined #minetest
05:53 siva_machina joined #minetest
05:58 Thomas-S joined #minetest
06:24 AnotherBrick joined #minetest
06:41 ssieb joined #minetest
06:42 torgdor joined #minetest
06:50 Markow joined #minetest
07:05 CWz joined #minetest
07:48 Trustable joined #minetest
07:51 Jordach joined #minetest
08:00 IhrFussel joined #minetest
08:02 IhrFussel agaran, Hi there! I'm doing a backup right now, after that we can check the backup map file for the largest map blocks
08:06 agaran joined #minetest
08:06 agaran joined #minetest
08:12 aheinecke joined #minetest
08:25 Szkodnix joined #minetest
08:43 IhrFussel Okay I extracted now the pos of the map blocks with the largest sizes...how do I convert the position data into ingame positions? Are they saved as position*10 ?
08:51 IhrFussel agaran, http://pastebin.com/yj2MByy9 these are the current DB stats now
09:07 SylvieLorxu joined #minetest
09:14 tpe joined #minetest
09:19 AcidNinjaFWHR joined #minetest
09:23 PseudoNoob joined #minetest
09:23 jojoa1997 joined #minetest
09:52 IhrFussel agaran, with the help of celeron55 I found a python script that converts the DB pos into x,y,z and extracted the 4 largest http://pastebin.com/mNuXrvU8
10:02 Markow joined #minetest
10:14 Darcidride joined #minetest
10:31 Not_a_Robot joined #minetest
10:32 Not_a_Robot joined #minetest
10:32 Megaf joined #minetest
10:34 Fixer joined #minetest
10:37 Megaf Hi
10:59 nyuszika7h joined #minetest
11:13 Jordach joined #minetest
11:23 AcidNinjaFWHR left #minetest
11:46 IhrFussel joined #minetest
12:03 yang2003 joined #minetest
12:15 lisac joined #minetest
12:16 yang2003 joined #minetest
12:16 yang2003 joined #minetest
12:17 est31 joined #minetest
12:18 Megaf joined #minetest
12:19 LionLAD joined #minetest
12:19 yang2003_tab joined #minetest
12:22 Megaf !server Megaf
12:22 MinetestBot Megaf: No results
12:22 Megaf Megaf Server is in coma. =/  https://forum.minetest.net/viewtopic.php?f=10&t=8484
12:25 Megaf 4.3G    MegafServer/worlds/world/map.sqlite
12:25 Megaf I wish there was a way to shrink that
12:31 proller joined #minetest
12:34 YangTabBot joined #minetest
12:36 IhrFussel Is it possible that mapblocks that are larger than 10 KB in size cause lags? 10 KB doesn't sound like a lot but I'm not sure how much the engine takes to load/store that amount
12:42 xunto joined #minetest
12:43 LionLAD joined #minetest
12:50 aix Megaf: rip
12:51 aix Megaf: serves you right for using sqlitr
12:51 aix do --migrate leveldb
12:51 aix run that on your world after compiling in support for it
12:54 Fixer hi Megaf
13:11 HonoredGlory joined #minetest
13:13 est31 joined #minetest
13:19 T4im joined #minetest
13:41 sonicpp joined #minetest
13:46 swift110 joined #minetest
13:46 swift110_ joined #minetest
13:49 swift110 joined #minetest
14:00 LionLAD joined #minetest
14:05 LNJ2GO joined #minetest
14:15 Fixer_ joined #minetest
14:22 agaran IhrFussel: moment, just got home
14:24 agaran IhrFussel: biggest one is 170k so quite.. something for mapblock if other take few kb
14:24 xSmurf joined #minetest
14:25 est31 joined #minetest
14:27 IhrFussel agaran, hey =) yes I know most mapblocks are < 10000
14:31 IhrFussel agaran, will you check the coordinates in a few?
14:32 agaran sure
14:32 agaran though I can't tp to coords so I'll have to walk so it will take a bit
14:32 IhrFussel They seem to be close to each other
14:33 IhrFussel I will teleport you there just let me know when you are on
14:35 Fixer_ Megaf: there is a way, google minetest unexplore script
14:36 IhrFussel agaran, ready?
14:38 agaran IhrFussel: sure I am on
14:38 agaran pitch black around I can't see anything, can I get noclip?
14:38 IhrFussel agaran, I already teleported you..or did it not work?
14:39 agaran IhrFussel: so you got new moderator or mod-helper?
14:39 swift110 joined #minetest
14:39 IhrFussel agaran, yes PsychoDragon is a new moderator now with limited functions
14:40 agaran IhrFussel: well.. nothing there, I mean little dug up, but I don't see any protected areas (though it did not show my place as protected either yesterday)
14:40 swift110 joined #minetest
14:41 IhrFussel agaran, is there nothing? Just the generated nodes?
14:41 agaran within 20blocks or so, yes, few corridors dug, etc but nothing that would explain huge amount of metadata..
14:42 IhrFussel Cause that is the largest block in DB with 170k...
14:42 M-geir joined #minetest
14:42 GCHQ joined #minetest
14:42 Jousway joined #minetest
14:42 sofar joined #minetest
14:42 Sokomine joined #minetest
14:42 mransom joined #minetest
14:42 agaran I can't check if it belongs to any owned area.. if not maybe erase block?
14:42 LionLAD joined #minetest
14:42 GCHQ` joined #minetest
14:43 agaran if you are sure that coords are in map coordinates not block coords
14:43 IhrFussel agaran, did you check underground? The exact block position is at -2
14:43 agaran so multiply by 16 each..
14:43 agaran thats why I asked for noclip.. to not have to dig around
14:44 IhrFussel Wait...the block positions are NOT the real map positions?
14:44 agaran not sure but they may be block coords.. so x+1 == next block on x.. so 16 nodes?
14:45 IhrFussel agaran, I'll teleport you to the other position with large blocks
14:45 agaran sure
14:46 agaran I'd check 373*16,-2*16,11*16 too
14:46 IhrFussel At -1 is the mapblock
14:47 KrimZon_2 joined #minetest
14:49 theTroy joined #minetest
14:49 agaran IhrFussel: meh tried to put torch to light cave got interact removed;) for 2 seconds, well some build but to much empty space to explain data amount
14:49 IhrFussel agaran, tell me when done investigating I'll teleport you to pos*16
14:49 LionLAD joined #minetest
14:49 agaran ready
14:49 agaran now it loads MUCH slower..
14:50 KaadmY joined #minetest
14:50 agaran hmm no, nothing here..
14:50 agaran empty generated nodes
14:51 Void7 joined #minetest
14:51 IhrFussel agaran, but it really loaded very slow?
14:52 agaran not sure, lag 4.6 so not really much slow, might be network as well
14:53 agaran next position?
14:53 IhrFussel Let's try the other large map block coords as * 16
14:53 agaran I wonder how to check if that position has really lot of metadata..
14:54 agaran IhrFussel: normal random map there,
14:55 agaran lag risen to 5.2 after some flying underground but thats still not a huge change nor explaining problems..
14:55 IhrFussel I should be able to check that I added a dumpmeta command for terminal output
14:56 agaran hmm lag got to 7, and 2 Player[digits] tried to log in..
14:56 agaran can it be accesslist related ?
14:58 IhrFussel agaran, let's check the dumpmeta command first...look for any node with metadata attached to it and tell me its position please
14:59 agaran i don't have permission to execute it..
14:59 IhrFussel agaran, no I do via the terminal after you told me the position
14:59 agaran here is just normal rock walls etc.. nothing to have meta at all
15:00 IhrFussel But the game can't simply generate a 170k mapblock randomly
15:01 agaran well I would do it other way... add command that I can execute and send result to debug log which walks all nodes within current block dumping their metadata size after serialization
15:01 agaran if > 1k then we have suspect..
15:02 IhrFussel No we don't need to check every mapblock the 4 largest are most likely the problem so something is wrong with them
15:02 agaran yes, every node- in one of those four mapblocks
15:03 agaran so tp me to spot/ ddump whole mapblock nodes metadata checking just for size after serialization.. or sth like that
15:03 agaran or just dumping out, whatever works
15:04 agaran IhrFussel: can I give some wool (I do have creative.. just asking IF)
15:05 IhrFussel agaran, yes any non-rare item can be given out when someone needs it...Cycling through every node is too complicated I think
15:06 agaran hmm.. I can imagine code but...
15:07 IhrFussel I'll logon with test account and see for myself
15:07 agaran sure
15:07 T4im "Wait...the block positions are NOT the real map positions?" no, they are positions of the mapblocks, not node-positions
15:07 T4im i.e. indeed /16 or *16 depending on conversation direction
15:08 IhrFussel T4im, wait when to use /16 ?
15:09 T4im when getting the block position from node positions
15:09 T4im doing the obvious rounding/flooring
15:10 IhrFussel T4im, but I got my mapblock positions from getIntegerAsBlock() python function
15:10 T4im python functions? did i miss something? :o
15:11 IhrFussel T4im, https://github.com/minetest/minetest/blob/master/doc/world_format.txt#L210
15:11 IhrFussel I used these 2 functions and gave them the INT position from the DB
15:12 IhrFussel getBlocksAsInteger()*
15:12 IhrFussel No wait..I confused it...below getIntegerAsBlock() is the correct one
15:13 T4im ah
15:13 T4im block position are also in (x, y, z) notation btw
15:14 T4im so the integer there is probably rather something like the hash position
15:14 T4im but yes, that should be blockpositions returned from it
15:14 IhrFussel T4im, yes the function converts the DB pos INT in X,Y,Z and I already got those
15:15 IhrFussel So those should be the correct node positions right?
15:15 agaran but problem is that those blocks look soo normal.. unless somebody had like book with LOT of data in it
15:16 agaran book is only generic object that can store arbitrary amount of data afaik
15:17 T4im nah, if you search you'll probably find more :P
15:17 T4im but unlikely
15:17 agaran 170kb of metadata on mapblock where nothing seem be odd?
15:17 T4im serialized entities maybe?
15:18 agaran none visible..
15:18 T4im IhrFussel: nah, getIntegerAsBlock(i) would give you block positions, i.e. you'd have to *16 every coordinate
15:19 T4im agaran: hmm, underground?
15:19 agaran T4im: y, not even much lava either
15:21 IhrFussel So when getIntegerAsBlock(i) says 1,1,1 the nodes it affects are 16,16,16 ?
15:22 T4im that's just the first node of the block
15:22 T4im it affects from there to 32, 32, 32
15:23 T4im eh 31, 31 ,31?
15:23 * T4im might be off by one here
15:24 agaran IhrFussel: check how much my place occupies.. I build some there especially mesecons lag-machine
15:25 IhrFussel T4im, so that's what you mean..the mapblock DOES start at the position getIntegerAsBlock(i) gives
15:26 T4im *16
15:26 T4im block * 16 gives you the first node position
15:27 T4im and it extends to ((block + 1) * 16) - 1 or so
15:27 IhrFussel T4im, but you said when the mapblock positions are 1,1,1 it will start there and go until 31,31,31
15:27 T4im "the nodes it affects are 16,16,16 ?" <-- that was the yes for :)
15:27 T4im 16,16,16 to 31,31,31
15:27 T4im is the 1,1,1 block
15:28 IhrFussel T4im, yes but not mapblock * 16 you mean + 16
15:28 agaran 2,1,1 is 32,16,16 to 47,31,31..
15:28 T4im ^^
15:29 agaran so 0,0,0 block start on 0,0,0 and ends on 15,15,15 ?
15:29 T4im yes, unless I'm off by one :P
15:29 T4im but i think yes
15:29 agaran I think it would be most efficient way in binary so probably it is like that
15:29 agaran it is shift by 4 bit..
15:30 hmmmm joined #minetest
15:30 IhrFussel So mapblock positions 373 -2 11 mean the nodes start somewhere at 5968,-32,176
15:31 T4im seems right
15:31 IhrFussel And the block goes to current node position + 16 and - 16 in all directions
15:31 IhrFussel Or just in certain directions?
15:31 T4im in 3 of them :)
15:32 agaran 5969,-32,176 to 5969+16,-32+16,176+16
15:32 agaran block is 16x16x16 not 32x32x32..
15:32 agaran IhrFussel: letters are 'special' blocks or not?
15:33 IhrFussel agaran, as long as the player don't want "f*** you" as letters it's fine to give them out but ask what exactly they need them for
15:34 IhrFussel T4im, well I'm at that exact position right now and all I see is stone nodes surrounded by some lava
15:35 agaran yup, I told you..
15:35 T4im is stuff inside the lava by any chance?
15:35 T4im like, uhm, fallen gravel?
15:35 IhrFussel T4im, the lava is more than 16 nodes away from the 170k mapblock
15:37 IhrFussel You know what I will do /deleteblocks here at that position
15:38 twoelk joined #minetest
15:38 agaran IhrFussel: I think it is valid solution..
15:39 twoelk IhrFussel: how about making a copy of the map that only has these large blocks and investigate that?
15:41 T4im i wonder how hard it would be to deserialize the mapdata manually and just take a look at the plain strings inside :D
15:41 twoelk just unzip the blob
15:41 T4im it's just zipped? well then
15:41 agaran it is just zipped blob?
15:41 twoelk gzipped
15:41 agaran and then it is json/whatever serialization?
15:42 twoelk nope minetest serialization
15:42 twoelk but readable
15:42 T4im can't get the strings out of the metadata :D
15:42 T4im not all anyway
15:43 twoelk can you see which?
15:44 T4im well keys could probably be collected in some sort of central lookup table and be references by integer
15:44 * T4im guesses blindly here, not having seen the unzipped blob yet, maybe it's already doen
15:44 FirePowi joined #minetest
15:45 agaran 170k gzip.. hmm.. even with deflate it can be easilly >1M of text data..
15:49 IhrFussel agaran, I deleted the 2 largest mapblocks...let's see if they are still the same size
15:49 T4im aw, and now we were all excited to see the ungezipped metadata :o
15:49 agaran so you need to visit them then check database?
15:49 Megaf sqlite vacuum needs an insane amount of space to work
15:50 Megaf 25 GB free is not enough to work on a 4.1 GB database
15:50 agaran Hmm, then better version of checking for size needs be done..
15:50 Megaf common
15:50 T4im why do you only have 25 free? O_o
15:50 agaran IhrFussel: server crashed?
15:51 agaran T4im: common vps limit is 20-40G space..
15:51 KaadmY i have 300mb free on my SSD
15:51 T4im ah
15:51 T4im :O
15:53 T4im KaadmY: that actually is quite bad for wear leveling; although better ssd's probably reserve space for that anyway
15:53 Megaf T4im: 32 GB usb flash drive
15:53 Megaf will try now on a 40 GB partition on my SSD
15:53 twoelk http://sqlitebrowser.org/ <- you can use this and double click on a blob to see the content
15:54 Megaf 47G total, - 4.1 GB for the database
15:54 Megaf hm, you know what, will create a LVM
15:55 KaadmY T4im: i have a 64 or 32gb SSD iirc
15:56 Megaf twoelk: I will take a look at that
15:56 Megaf my server is off line basically because I ran out of disk space. If I reduce the map size I might get it back online
16:00 est31 joined #minetest
16:00 Tux[Qyou] joined #minetest
16:02 IhrFussel WAIT A SEC...quote(data) will SHOW the node names I think since the DB browser shows them too after a whole bunch of random characters
16:03 agaran gzip header may have file name?
16:03 agaran IhrFussel: server is down intentionally?
16:04 IhrFussel agaran, the server is down cause I'm searching the DB...and yes every mapblock of my local test world (2.2 MB) in the DB browser shows the node names at the very end when doubleclicking BLOB
16:04 agaran ah,
16:04 T4im huh? why are the node names persisted in there?
16:05 T4im shouldn't they be only referenced by content_id?
16:06 Megaf done, 76 GB in a LVM volume now, should be enough to deal with a 4.1 GB DB!
16:06 IhrFussel T4im, I chose option "show binary data as TEXT"
16:07 IhrFussel And I'm thinking quote(data) will do the same inside sqlite3 program
16:07 agaran twoelk: any proof that it uses gzip.. I can't decode it as gzip()
16:08 T4im "zlib-compressed node data:" according to world_format
16:09 agaran hmm any zlib stream is recognized by file (5), but this is not..
16:09 T4im and "zlib-compressed node metadata"
16:09 agaran probably I need to find how it is encoded before packing into db..
16:10 Megaf map.sqlite
16:10 Megaf 745,774,787  16%   10.64MB/s    0:05:50
16:10 Megaf not bad
16:10 Megaf (downloading the file from my server to my local computer)
16:10 agaran ah.. I was going to ask what speed it is..
16:10 Megaf (compressed transfer using rsync)
16:11 Megaf rsync -az --progress
16:11 IhrFussel All users on a v server SHARE CPU power right? So even if I see in htop 0,0% average load it doesn't mean the cores aren't busy elsewhere?
16:13 agaran yes for your virtualization instance.. but if other virtual machines eat cpu then you won't see it
16:13 IhrFussel agaran, so I cannot really know if my 2 cores that are supposed to be mine are working for another user right now?
16:14 basxto joined #minetest
16:14 agaran yup..
16:14 * twoelk tries to find his long forgoten tools for exploring mt databases and remembers it wasn't as easy as he had hoped
16:14 agaran :) I found explanation and perl comes to rescue
16:15 agaran fear not we have DBI/DBD::Sqlite.
16:17 twoelk IhrFussel: show binary opens in a hexeditor - might give a little more insight
16:18 LionLAD joined #minetest
16:22 agaran map_Format.txt is indeed usefull.. but decoding is not just gzip over all data
16:26 twoelk maybe the mapper code might help
16:27 twoelk https://github.com/minetest/minetestmapper
16:29 twoelk https://github.com/minetest/minetest/blob/master/util/minetestmapper.py can't the old python code be hijacked and canibalized as an mt-database tool?
16:30 agaran twoelk: I already got to zlib layer
16:30 agaran so on good way..
16:31 agaran but I am doing it in perl.. python is foreign language for me
16:32 twoelk there is a great need for a third party tool to manipulate nodes in bulk in mt-maps from the outside
16:33 agaran got to seeing node names.. so..
16:33 twoelk OldCoder only used SQL commands in his tools if I remember correctly
16:34 agaran well sqlite isn't particularly bright when comes to SQL capabilities..
16:37 Megaf $ sqlite3 map.sqlite "VACUUM;"
16:37 Megaf Error: database or disk is full
16:37 Megaf whatta!?
16:37 Megaf where is sqlite storing the temp files?
16:42 rubenwardy joined #minetest
16:44 IhrFussel agaran, just to make sure: This line will tell the block size plus position of block right? "select length(data), pos from blocks where length(data) > 30000;"
16:45 agaran y
16:47 IhrFussel Weird..I ran this exact line on the backup map.sqlite and it was done after a few seconds..now it takes a loong time
16:47 agaran still my biggest block on first db is mere 4k..
16:47 agaran which is oddly low
16:48 IhrFussel Does sqlite cache the most recent query or something?
16:56 twoelk hm, when I export the blob data I find the hex  value set of  (78 9C) which should translate to zlib Default Compression but somehow the byte positions seems offset by 4. Does sqlite add something to the blob?
16:56 agaran dunno
16:56 agaran twoelk: no, 4 bytes are described in map
16:56 agaran u8 version u8 flags u8 content_width u8 params_width
16:56 Megaf ok, managed to vacuum, 100 MB smaller
16:56 Megaf =/
16:57 Megaf twoelk: I installed sqlite browser here, how can I use it now? Or what can I do to reduce db size
16:57 twoelk so the blob has 4 bytes of mapdata and then the compressed part?
16:58 agaran y
16:58 agaran node timers are there to..
16:58 twoelk tailed by a plaintext list of nodes it seems
16:59 agaran not only but yes, after unzip
16:59 IhrFussel twoelk, the node names are separated by 4 spaces in TEXT mode or 4 dots in binary mode
17:00 IhrFussel Or wait
17:00 IhrFussel Maybe the 4 dots are there for the timers etc
17:01 sfan5 i suggest reading the specification instead of doing wild guesses
17:02 Krock joined #minetest
17:02 Krock joined #minetest
17:02 AcidNinjaFWHR joined #minetest
17:03 agaran sfan5: what is uncertain to me if whole map got serialized then passed to gzip.. or there are few gzipped blocks concatenated..
17:03 agaran seems that it is latter tho
17:03 DMackey- joined #minetest
17:04 LionLAD joined #minetest
17:05 twoelk hehe those are not wild guesses, those are educated guesses based on interpretations of the manual and what we see according to our limited understanding of the matter :D
17:05 sfan5 if your educated guess is 4 dots seperate the node names after reading the spec
17:05 sfan5 then you haven't understood shit
17:06 IhrFussel sfan5, in HEX mode of DB browser the node names ARE separated by 4 "dots" I think those mean NULL bytes or something
17:06 IhrFussel Not HEX I mean binary*
17:06 twoelk 00 00 00 0d
17:07 sfan5 ok here's a tip
17:07 sfan5 those bytes are not just delimiters
17:07 sfan5 they have a meaning
17:07 IhrFussel Yeah most likely metadata and timers
17:08 sfan5 no
17:08 twoelk so lets call dots just non ascii representable information carriers ;-P
17:09 LNJ2GO joined #minetest
17:11 rom1504 joined #minetest
17:11 * twoelk returns to trying to parse the world_format.txt as sfan5 will not give freely information to the blundering noobs
17:11 sfan5 well try reading the code
17:13 IhrFussel So the metadata cannot be read in plain text I guess
17:13 twoelk not without hacking the blob to pieces it seems
17:13 twoelk so much for using a standard database tool
17:14 IhrFussel But I wonder why the devs decided to make node name readable in TEXT mode
17:14 sfan5 that wasn't a decision
17:14 sfan5 not a explicit one at least
17:14 sfan5 it just happens to be that way
17:15 IhrFussel sfan5, is metadata stored before the list of nodes?
17:15 sfan5 most likely not
17:16 proller joined #minetest
17:16 twoelk eh?
17:16 IhrFussel So mapblocks that contain metadata will list it AFTER all the nodes?
17:16 * twoelk searches for a better blob example
17:17 sfan5 yes
17:17 IhrFussel Same with timers?
17:18 sfan5 yes
17:18 IhrFussel Are timers separated from metadata or part of it?
17:19 sfan5 i dont know
17:19 est31 IhrFussel, there is docs in the doc folder of minetest source code
17:19 agaran ok, decoded some metadata too
17:19 agaran they are glued
17:20 est31 https://github.com/minetest/minetest/blob/master/doc/world_format.txt
17:20 agaran there is block of compressed data of map, then nodes, then metadata, each in own inflate package
17:20 IhrFussel agaran, it seems to have worked..only mapblock > 30k now is 64751|-7182422423
17:23 Krock joined #minetest
17:24 agaran so good..
17:24 agaran my mapblock that in db takes 2700 bytes roughly contains much over 16k data.. so compression does good
17:25 twoelk I seem to have found a blob with metadata and no, the plain text list is the last thing, followed by 0a 00 00. all metadat is before the list
17:25 IhrFussel agaran, but when i cast(data as TEXT) at that pos in DB I only get like 20 characters
17:26 agaran data is not text..
17:26 IhrFussel No but cast converts data into TEXT AFAIK
17:27 IhrFussel So it means the block probably doesn't contain any nodes
17:28 sfan5 try hex(data)
17:28 T4im doesn't it always contain nodes? even if air or ignore?
17:29 sfan5 yes it does
17:29 twoelk http://pastebin.com/LWtGMwME <-example blob
17:29 agaran yay... got closer..
17:30 agaran IhrFussel: could I convince you to dump my place block to file and make it somehow downloadable?
17:30 GunshipPenguin joined #minetest
17:31 agaran and fun part, biggest blob in my db is.. just one chest in world..
17:31 IhrFussel agaran, what do you mean "place block"?
17:31 agaran oh, one of block in region where I have my place.. but I think I'll fabricate more data locally will be easier
17:35 IhrFussel Why can't I get the same result via sqlite CLI as the DB Browser displays...doesn't the DB Browser use simple sqlite commands to convert the BLOBs?
17:36 agaran no idea..
17:36 twoelk hrmpf, that was only a cow in my example, wasn't it........why are all the chests empty on my map?
17:37 HonoredGlory joined #minetest
17:45 IhrFussel agaran, I converted to HEX and pasted the HEX into http://string-functions.com/hex-string.aspx ... it contains A LOT of oerkki lines
17:46 LionLAD joined #minetest
17:48 agaran twoelk: hmm you damaged your db?
17:49 twoelk nope, the chests are just empty. I grabed the first map I had locally and in it chests are just used for deco
17:50 Markow joined #minetest
17:50 agaran IhrFussel: even circular saw when has anything to work on consumes metadata space..
17:51 twoelk i guess I should create a new map with lots of metadata and explore that - although there should be hundreds of torches, just can't find them
17:52 agaran heh added chest with few items, blob size got like 12 bytes.. after decompression I got +1.2k..
17:52 T4im even empty chests do
17:52 T4im for the formspecs at least
17:52 twoelk hm
17:52 agaran T4im: yup.. but my decoder even partly works, at least shows amount of data and dumps them to file
17:53 MegaSand joined #minetest
17:53 agaran IhrFussel: bad news is, size of data field is not proportional to actual data size.. and other thing, increase in size there by 200 bytes may mean tens of kb after decompression...
17:55 MegaSand I have heard that you can duplicate items by placing them into a chest before a server restarts, so there is a duplicate in your inventory. is this true?
17:56 HonoredGlory joined #minetest
17:58 IhrFussel agaran, oh...but the byte sizes give an idea how much data the mapblock stores I guess... the mapblock with all the oerkkis is at the end of the map (almost -7000 in all directions)
17:58 twoelk http://pastebin.com/2HMMn6En <- better example with chests, torches, trapdoors and other nodes that should have metadata
17:59 agaran IhrFussel: it gives indication that there is more metadata but does not quantify how much, I made crude perl script that can dump if you know pos value for sql db..
18:00 resident joined #minetest
18:02 resident who wants to start making some mods with me
18:02 IhrFussel agaran, if it still lags now it must be caused by mapblocks < 30k
18:02 twoelk agaran: could it generate a list with blocks sorted by blobsize?
18:04 agaran 730 bytes of deflated data produce 13k of data.. 20+ empty chests
18:04 agaran twoelk: not yet
18:04 agaran IhrFussel: shall I connect?
18:04 IhrFussel agaran, server is online again yup
18:08 agaran lag 0.6
18:08 agaran can you tp me to problematic nodes?
18:09 agaran or just cities will do?
18:10 IhrFussel agaran, wait if you move it will reset you to -6500 I think cause I prevent players from building at the edge of the map since it causes crashes
18:11 Megaf Fixer_: have you ever used https://github.com/AndrejIT/map_unexplore ?
18:11 agaran IhrFussel: ok
18:11 agaran now i noticed that I am at very odd coordinate
18:12 IhrFussel Yes I will teleport you once again to the oerkki mapblock but don't move just look around
18:12 agaran ok
18:12 IhrFussel Do you see lots of them?
18:13 agaran nothing really I am inside stone
18:13 IhrFussel But you have noclip
18:13 agaran and some tunnels above me
18:13 agaran still I see no monsters.. but they may be in dark tunnels above
18:13 agaran i see -6500,-6928,-6500 coords
18:14 sfan5 IhrFussel: just for you https://gist.github.com/sfan5/52917a0e1338bcda2176edbc416dae60
18:14 sfan5 because you wouldnt shut up
18:14 sfan5 instead of just reading the docs
18:14 namach joined #minetest
18:14 sfan5 use the sql command given to acquire the data
18:14 sfan5 then give the data to the script when asked to do so
18:14 sfan5 then press ctrl+d
18:15 sfan5 it will list the stuff inside
18:15 sfan5 no idea whether it works on windows
18:16 agaran sfan5: nice :)
18:17 agaran IhrFussel: for moment I saw mese monster flying next to me..
18:17 agaran inside stone it seems
18:18 IhrFussel sfan5, even though the way you talk to me is a bit rude, thanks... agaran like I said the game prevents any player from reaching the map edge (map limit is 7000 on my server) so if you reach X or Z 6900 or -6900 it teleports you back to 6500/-6500
18:19 IhrFussel Y position is unaffected
18:19 agaran IhrFussel: thats fine.. I even can't rotate around too much as it gets jumpy..
18:20 agaran but does your code handle case when y > 6900 or y < -6900 ?
18:20 agaran it seems to tp me to x/z coords of limit but it is y which is overlimit..
18:21 agaran so I feel jumpy.. still staying on same cords.. rotating and tping in place when rotating..
18:21 IhrFussel I'm not sure it was just a quick implementation cause some players LOVED to build their stuff at the generation void and after some minutes the server crashed
18:21 sfan5 can't you use the mapgen limit to create world borders?
18:21 Megaf Humanity is in deep trouble, https://www.youtube.com/watch?v=b6CPsGanO_U
18:21 sfan5 !title
18:21 MinetestBot sfan5: What If All The Ice Melted On Earth? ft. Bill Nye - YouTube
18:22 Megaf !title
18:22 MinetestBot Megaf: What If All The Ice Melted On Earth? ft. Bill Nye - YouTube
18:22 IhrFussel sfan5, when players build directly at the last node of generation the server crashes after some time
18:22 sfan5 did you file a bug?
18:22 agaran there is no tnt on server.. just make wall im impenetrable nodes..
18:23 sfan5 IhrFussel: anyway if you still have huge mapblocks i suggest putting them into that script to find out why they are so big
18:23 agaran IhrFussel: and please check if code works right when y is over limit not trying to teleport to same y, limiting x/z instead..
18:23 agaran sfan5: biggest we saw was 170k compressed then few more about 60k+.. which is rather.. huge
18:23 sfan5 yeah
18:24 IhrFussel No teleporting the players is fine..maybe I could reduce the distance the players gets thrown back though like to 6800 instead of 6500 ... sfan5 the problem is very old I'm not sure if I ever filed one
18:24 agaran node data seems to fit easilly in 2kb with small metadata overhead even..
18:24 twoelk IhrFussel: what mod do you use to limit the exploring players?
18:24 sfan5 even uncompressed the node data would be just 16k
18:24 agaran IhrFussel: ok. Anyway rotating there is odd.. jumpy a lot.
18:24 agaran sfan5: I know.. I already got to that part working :)
18:25 IhrFussel twoelk, I check the X and Z position of each player every few global steps
18:26 agaran IhrFussel: does your code generate log entry for every teleporting due to rangelimit?
18:26 IhrFussel So no mod just a few lines Lua
18:27 twoelk maybe try worldedge or the other one from hybrid dog
18:27 Yst joined #minetest
18:28 IhrFussel No I'm fine...I don't even want that players build at the edge, there is enough free space on the map
18:28 IhrFussel Plus it should tell the player what happened...or did I accidentally disable it?
18:28 twoelk https://github.com/HybridDog/stay_inside or  https://forum.minetest.net/viewtopic.php?t=10753 (worldedge)
18:28 agaran IhrFussel: it does not say anything..
18:29 IhrFussel Well then I will enable that again so the player doesn't try to reach the end again
18:34 agaran IhrFussel: I thinksomebody was building there..
18:34 agaran I moved -abit- up so I am not at less than -6900, so it stopped to be jumpy (seems that code has flaw in regard to that case..) and I see some items whcih aren't generated
18:35 agaran and some poor oerki or 2 are there trapped as well
18:36 Fixer_ Megaf: maikerumine used
18:36 Fixer_ Megaf: it shrinks the map and leaves only modified parts
18:37 twoelk LOL would be fun if the script tried to teleport oerkkis
18:38 IhrFussel agaran, I fixed the disabled info for the player and also improved it..now when pos x or pos z is > 6900 it teleports the player back to pos.x - 100 and pos.z - 100 and when pos is < -6900 it teleports to pos.x + 100 and pos.z + 100
18:38 agaran does it care about 'y' axis at all?
18:39 agaran in position check ?
18:39 agaran because it does react if I am 6500,6901,6500..
18:39 IhrFussel agaran, no cause as far as I can tell the crashes only happen on X and Z
18:39 IhrFussel Yes it did react before my changes so it will stop reacting after restart
18:39 agaran if I am above -6900 it is ok if I try get below I get tped to -6900..
18:39 agaran ok
18:39 lumidify joined #minetest
18:39 agaran so I was right about jumpy-tp
18:41 IhrFussel agaran, math works without any function calls in Lua right? Like local newpos.x = oldpos.x - 100
18:42 agaran yep
18:43 IhrFussel agaran, alright then there shouldn't be any syntax errors...could you check the lag?
18:43 IhrFussel If you are still online
18:43 agaran local newpos = { x=oldpos.x-100, ... }  I would do it like that
18:43 agaran or newpos = table.copy(oldpos); if (newpos.x > ...) newpos.x = newpos.x - 100  etc..
18:43 agaran 2.3866
18:44 twoelk remembering my cow in example blob, maybe all them oerkkis are generating a lot of data
18:44 agaran and decreases
18:44 IhrFussel agaran, then it's a smaller block and the ones we deleted weren't the problem
18:45 agaran IhrFussel: could be now some monster attacks me causing still that error about missing fire nodes..
18:45 agaran mind if I kill it?;)
18:45 IhrFussel agaran, but I added the ABM
18:45 agaran yes.. but monster still -emit- wrong node
18:45 agaran then abm eats it..
18:46 agaran abm was to fix pre-existing nodes of wrong kind..
18:46 agaran I thought you patched monsters too
18:46 twoelk are you sure there are not some hidden secret nuclear tool chains or quarrys or computer built with mesecons in the area?
18:46 IhrFussel I changed the set_node() of lava_flans to air
18:47 IhrFussel Are Dungeon Masters causing the errors?
18:47 agaran white one
18:47 IhrFussel The big ones shooting fire balls?
18:47 agaran no clue how it was called didn't introduce himself started to throw things at me
18:47 agaran yes but there are few colors of them
18:47 IhrFussel Yeah all Dungeon Masters
18:47 IhrFussel I can fix that one sec
18:48 agaran well I don't mind.. I just try to do qa-work if I got called to that;)
18:48 LNJ2GO left #minetest
18:50 IhrFussel agaran, Commented "node bursts into flames" out
18:51 IhrFussel I mean the code block not the line haha
18:57 agaran so restart is comming?
18:57 agaran or not yet?
18:57 IhrFussel Yep
19:04 agaran hmm tried modcity, skycity, and seems to not cause lag now
19:05 IhrFussel agaran, it only starts to lag when more players connect and go to different locations
19:05 agaran hmm so like loading/unloading mapblock..
19:05 agaran you have fast cpu there, right?
19:06 agaran so some deflate/inflate often shouldn't hurt?
19:06 twoelk lots of players, lots of position checking?
19:07 IhrFussel twoelk, no it's not Lua code I already tested the world without any mods
19:08 IhrFussel Must be mapblocks data...quite a few are > 20k
19:10 twoelk found out yet whats in them mapblocks that is so much data?
19:10 IhrFussel For that I need to extract every block > 20k and compare the contents to each other
19:12 STHGOM joined #minetest
19:13 twoelk hm, maybe log in as new player with savetolocal on and teleport to some affected area. Then logout and investigate the created smaller database
19:13 Void7 joined #minetest
19:13 twoelk or let agaran do that ;-P
19:15 IhrFussel twoelk, haha nah I don't mind the extra work...I'll just take each block position write it down then extract the blocks as HEX and convert that into a string
19:17 twoelk well if you know the positions agarans or sfans script should help
19:21 agaran lag=8.5
19:22 ptv joined #minetest
19:23 swift110_ joined #minetest
19:23 IhrFussel agaran, I'm doing the > 20k query right now we will see in a few what content inside the large blocks is the problem
19:24 agaran sure, just reporting
19:27 IhrFussel I'm still confused why queries are fast sometimes and the same queries are slower at other times...I'm guessing sqlite caches some data to process the table quicker
19:27 agaran it may be not sqlite even but VFS
19:27 agaran kernel may have some part of file in cache.. and other not especially if file is few GB..
19:28 LionLAD joined #minetest
19:32 shamoanjac joined #minetest
19:32 Telesight joined #minetest
19:32 agaran IhrFussel: what universal admin toold does?
19:33 IhrFussel agaran, removes any node with 1 click
19:33 agaran well then not so interesting :)
19:34 IhrFussel To griefers it is^^
19:34 agaran yea..
19:34 agaran I would more like item that protect from lava damage..;)
19:36 edaq joined #minetest
19:37 Calinou agaran: possible, but it won't have any effect if player has <= 8 HP
19:37 Calinou and it won't remove red flash/shake
19:40 agaran Calinou: well.. I spend lot of time yesterday building from bottom of lava lake..
19:40 agaran tricky...
19:40 STHGOM hey, quick question: what is the lua for teleporting players?
19:40 T4im get the player object and set the position, see lua_api.txt in your minetest doc directory for details
19:40 STHGOM ok
19:41 twoelk http://irc.minetest.ru/minetest/2016-09-07#i_4700797 two examples
19:41 IhrFussel STHGOM, player:set_pos({x=x,y=y,z=z})
19:41 IhrFussel But before you need to do minetest.get_player_by_name(playername) if it's via command
19:41 T4im one way to get the player object would be `get_player_by_name`, some functions also pass it directly
19:42 STHGOM k, thanks :)
19:42 T4im don't use that ^^ if it's already passed, or you do unnecessary extra work :p
19:42 T4im callbacks etc
19:42 IhrFussel T4im, in commands you have to use that
19:43 T4im you know that, I know that, but I've seen people do get_player_by_name(user:get_player_name())  before
19:43 IhrFussel Oh sorry I thought you meant me with "^^"
19:49 agaran IhrFussel: you did a bit.. problem
19:49 agaran I got teleported to -212117,y,5000
19:49 agaran and now system tries teleport me 100 units closer to limit.. every few cycles
19:49 agaran but.. it fixes x coord, and adds to z.. so it will not end..
19:50 twoelk LOL- sounds like fun
19:50 twoelk lots of new map generated :D
19:50 IhrFussel agaran, seems like I need to check each position individually
19:50 agaran yea.. especially that -212k is -faaar- away map limiters
19:50 agaran IhrFussel: I am afraid so
19:50 IhrFussel twoelk, I don't think it generates anything there
19:51 agaran but it looks fun.. stars.. sun.. void
19:51 agaran and lag increases every TP..
19:51 IhrFussel Will do after sqlite finished...found 1 mapblock so far
19:51 agaran and it is that client bug which teleports you to edge of world..
19:51 * twoelk looks at number again - oops - misread
19:51 agaran IhrFussel: i could /home so no problem
19:52 IhrFussel Even at 21000 the game doesn't generate on my server..only till 7000/-7000
19:52 agaran that was funniest part in fact
19:52 agaran FAR FAR away limits
19:53 IhrFussel agaran, so it started at 7000 and got more every few cycles?
19:54 agaran I got x=-212xxx, y same as I was, z=5000 or so when I noticed, then every repeat x was getting bit more positive and z bit more positive by same amount too
19:54 agaran so when X will get eventually to limit Z will be FAR above
19:55 LionLAD joined #minetest
19:56 IhrFussel agaran, well do the messages work?
19:56 agaran yep that did work.. got it every repeat.
20:00 agaran anyone knows if you can get displayed edges of mapblocks ?
20:00 LionLAD joined #minetest
20:06 Grandolf joined #minetest
20:06 twoelk maybe modify the old landrush mod?
20:10 jojoa1997 joined #minetest
20:37 STHGOM is it possible to "attach" sounds to players or entities, so the 3d position of the sound will follow them?
20:38 Calinou STHGOM: yes, the sound has to be attached to an object
20:38 Calinou see: most mob mods
20:38 STHGOM ok
20:39 Fixer_ 94% of "earth" in minetest is stone
20:39 Fixer_ underground
20:39 Fixer_ by volume
20:40 Fixer_ actual ores = 4,42% of volume
20:41 Fixer_ so if you dig 10x10x10 mine, you will get 44 blocks of ore (roughly)
20:41 Fixer_ out of 1000 blocks
20:41 Fixer_ .______.
20:44 Fixer_ https://i.imgur.com/RYBNa8r.png more info
20:45 LionLAD joined #minetest
20:45 Fixer_ mese blocks are ultra rare
20:45 Szkodnix joined #minetest
20:48 agaran Fixer_: heh, interesting.. would be nice to see how gloop/moreores/technic ores change that
20:49 Fixer_ agaran: i can do that ._.
20:49 theTroy joined #minetest
20:50 agaran Fixer_: I don't doubt, I wonder if you are bored enough
20:50 Fixer_ yes
20:50 Fixer_ enough
20:50 Fixer_ gloopblocks?
20:50 Fixer_ or gloopore?
20:51 Fixer_ glooptest
20:51 Fixer_ found it
20:51 agaran dunno, I mostly have moreores enabled, gloopore ares fun though I found little use for precious gems from them
20:51 Fixer_ added, lets see
20:53 Fixer_ searching for really big place
21:00 Fixer_ agaran: see lots of ore
21:02 LionLAD joined #minetest
21:11 Fixer_ agaran: done
21:20 Fixer_ agaran: https://i.imgur.com/KtD6wKo.png data
21:21 nore joined #minetest
21:25 swift110_ joined #minetest
21:28 agaran so technic tosses too much ores..
21:28 agaran though it is odd.. I always had hard time to find uranium where zinc was easy..
21:29 agaran Fixer_: :) thx, interesting results
21:29 T4im zinc, chromium, uranium, sulfur are genereated differently than other ores, they can be frequent in some areas, non-existant in others
21:32 SylvieLorxu joined #minetest
21:32 agaran I guess I generate sleep block for now.. good night :)
21:32 agaran I just realized that I need to wake up in 6h at latest..
21:32 Fixer_ test area size: 71*72*79 nodes
21:45 LionLAD joined #minetest
22:13 halt_ joined #minetest
22:31 siva_machina joined #minetest
22:31 ptv joined #minetest
22:39 Yst joined #minetest
22:42 LionLAD joined #minetest
22:48 Szkodnix joined #minetest
22:56 Fixer joined #minetest
23:04 theTroy joined #minetest
23:14 GCHQ` joined #minetest
23:17 kaen joined #minetest
23:17 kaen joined #minetest
23:18 Tmanyo joined #minetest
23:23 paramat joined #minetest
23:23 Dolus joined #minetest
23:24 paramat Fixer it's because you raised water level. you need to redefine biomes and decorations with new y limits
23:24 LionLAD joined #minetest
23:25 paramat in a mod 'clear registered biomes/decorations' then redefine them all with changes
23:26 paramat essentially it's best to leave water level at 1 and move the terrain up or down
23:31 GrimKriegor joined #minetest
23:32 Fixer paramat: interesting
23:33 Fixer paramat: looks like pain in the ass, I just wanted to rise the damn water level
23:33 Fixer not user friendly
23:33 LionLAD joined #minetest
23:41 LionLAD joined #minetest
23:42 Fixer paramat: compared to minecraft custom world creator, this just plain sucks
23:44 swift110 joined #minetest
23:47 STHGOM joined #minetest
23:48 swift110 joined #minetest
23:49 paramat it may work in mgv6. i suggested linking biome limits to water level but hmmmm strongly opposes, indeed doing that may cause it's own problems

| Channels | #minetest index | Today | | Google Search | Plaintext