Time  Nick      Message
07:04 nerzhul   merging #5569
07:04 ShadowBot https://github.com/minetest/minetest/issues/5569 -- [CSM] Add function to set minimap shape by bigfoot547
07:16 red-004   #5554
07:16 ShadowBot https://github.com/minetest/minetest/issues/5554 -- [CSM] Allow enabling and disabling mods. by red-001
07:18 nerzhul   i need a reviewer to merge #5493 it's finished and improved
07:18 ShadowBot https://github.com/minetest/minetest/issues/5493 -- Formspec: Add options to set background color and opacity (fullscreen mode + default mode) by nerzhul
07:24 nerzhul   i push an android build fix
09:41 rdococ    hm
09:41 Krock     me, trying to reduce the amount of indents using a single goto instruction. "Warning: Variable foobar skipped by goto end_bar". Possible solution "{ code here }". Wow thanks for preventing me from using goto.
09:42 Krock     +1 for the compiler /s
09:42 rdococ    does anyone know where the code in map.cpp (I assume) for instantly removing flowing water blocks (whose same_level neighbours are unfloodable blocks) is?
09:42 rdococ    I've been trying to search for it but couldn't find it.
09:42 rdococ    I'm working on a modification to make the water more realistic, and believe it or not it has been successful so far.
09:43 Krock     instantly removing? the level will decrease until it's not flooded anymore
09:43 rdococ    really?
09:44 rdococ    in my experience, when a flowing water node is isolated by solid nodes around it, it instantly disappears.
09:44 rdococ    in fact, it doesn't have to be isolated by solid nodes
09:44 rdococ    it can just be on its own
09:44 Krock     this means there was no flowing node around
09:45 rdococ    ?
09:46 rdococ    what do you mean?
09:46 rdococ    Krock, I don't understand what you mean.
09:46 Krock     if there's no flowing node on either side of the flowing node, it surely will disappear
09:47 rdococ    I am aware of that.
09:47 rdococ    (It doesn't happen if there are any source blocks nearby, though.)
09:47 rdococ    I'm asking for where in the code it checks for this - can't see to find it.
09:48 nerzhul   goto is a bad usecase, especially in C++ without any justification
09:48 nerzhul   goto good usage it to skip tasks and go t oa common return case, not for skipping indent it's one of the worst usecases
09:49 rdococ    But spaghetti code is tasty :P
09:49 nerzhul   rdococ, cf Openssl :p
09:49 nerzhul   Krock, are you okay with #5493 ? sofar seems okay
09:49 ShadowBot https://github.com/minetest/minetest/issues/5493 -- Formspec: Add options to set background color and opacity (fullscreen mode + default mode) by nerzhul
09:51 rdococ    hm.
09:52 rdococ    hm
09:52 rdococ    if the node_level variable is set to 0, is that the smallest possible amount of water, or no water?
09:53 rdococ    ofc there is no node_level variable, but I'm talking about x_node_level variables in general
10:20 Krock     I think nothing is shown but the node still exists
10:28 rdococ    k
11:55 Remy_     Hi! I'm trying to use SQLcipher to encrypt SQLite database for Minetest. Is somebody have already done something similar ?
12:53 Zeno`     why though?
12:56 nerzhul   Zeno`: if i show you a trivial patch can i push it to master ?
12:58 nerzhul   https://pastebin.com/jzLRcpYw
12:58 nerzhul   it's a little cleanup on sendToAll method, which take useless arguments, instead of honoring packet configuration present in table :)
12:59 nerzhul   or sfan5 or nore ? or Kray
12:59 nerzhul   Krock:
13:01 Remy_     Zeno` Because i want have a client that can work offline. So i have to download the map.
13:02 Remy_     If i can avoid avoid a plaintext db ... It's better
13:03 Krock     nerzhul, did you even test that? see line 48
13:04 nerzhul   yeah, it seems i removed it by error when paste
13:05 nerzhul   https://pastebin.com/PTUVgbq9 :)
13:05 Krock     also, the pointer is still on the wrong side. but I can't say more about it since networking isn't something that I looked at yet
13:06 nerzhul   purely codecode i just removed the custom channel & reliable versions and make sentToAll more intelligent by using the packet configuration table
13:06 nerzhul   look at Server::Send method
13:06 nerzhul   it's the unicast version
13:08 nerzhul   yeah i fixed the code style
13:11 nerzhul   seems except sapier and me nobody knows network :p but here we are not really in network it's just a loop with a variable removed using configuration table :)
13:13 nerzhul   Krock: #5590
13:13 ShadowBot https://github.com/minetest/minetest/issues/5590 -- ClientIface::sendToAll: honor packet configuration by nerzhul
13:13 sfan5     nerzhul: if you test it and it works, then you can push it
13:14 nerzhul   clientCommandFactory is a table containing packet configuration (channels, name, reliable flag) used when sending packet, it permits to make sendToAll using it , like all send packet
13:14 Krock     okay, thanks for the explanation
13:14 nerzhul   sent* :)
13:15 nerzhul   i'm working on the PR for chat procotol enhancemnts (especially for having very good CSM handlers on it) and i saw that when changing sendChat packet :)
13:16 nerzhul   i'm waiting for CI and i iwll merge it, it works as intended without doubt, there is no mismatch between configuration table & sendToAll calls
13:33 nerzhul   CI ok, tests okay, merging
13:54 nerzhul   i also noticed we have a mismatch on channel type into structure, it's a u8 not a u16, pushing a fix (shown by clion IDE)
14:08 nerzhul   pushed, see https://github.com/minetest/minetest/commit/093e621643edf3bc7fa46e58cb1b5d940365ce22
15:56 Zeno`     nerzhul, hi
15:56 Zeno`     sorry I didn't look at the PR before, I was in between doing things :(
15:57 Zeno`     anyway, can you save me looking at the code please because you will know the answer already.  Does our network code include some kind of checksum?
15:57 sfan5     no it doesn't
15:57 Zeno`     Because I'm seeing semi-regular reports of serialization errors and I'm starting to wonder if they are corrupt packets throwing the exceptions
15:58 Zeno`     sfan5, thanks. In that case should a corrupt packet throw an exception?
15:58 sfan5     what do you mean
15:58 sfan5     also UDP has a checksum, IP has a checksum (at least v4)
15:58 sfan5     the chance of encountering corrupted packets is near zero
15:59 Zeno`     2017-04-11 17:52:53: ERROR[Main]: A serialization error occurred:
15:59 Zeno`     2017-04-11 17:52:53: ERROR[Main]: decompressZlib: inflate failed
15:59 Zeno`     2017-04-13 22:35:58: ERROR[Main]: A serialization error occurred:
15:59 Zeno`     2017-04-13 22:35:58: ERROR[Main]: too many items
15:59 Zeno`     stuff like that
15:59 Zeno`     (client-side)
15:59 Zeno`     (obviously)
15:59 sfan5     would be good to know which pkts that happens in
15:59 Zeno`     yeah, I'm trying to get a backtrace
16:00 nerzhul   Zeno`: if i remember there is a crc included
16:00 sfan5     no there is no crc
16:00 Zeno`     I'll keep trying. With more data available it's obviously going to be easier to track down
16:00 nerzhul   but i agree with sfan5 underlying IP/UDP layer has CRC
16:01 Zeno`     It happens less often when I am running with gdb though which is frustrating
16:01 Zeno`     Can those be warnings instead of exceptions? Or is that a really bad thing to do?
16:02 sfan5     warnings make no sense here
16:02 nerzhul   Zeno`: generally zlib decompressions problems mean packet was created wrong server side, it's not due to network
16:02 Zeno`     the packets cannot be simply ignored?
16:02 Zeno`     hmm ok
16:03 Zeno`     I'll keep trying to get proper backtrace then. Thanks sfan5 and nerzhul
16:03 sfan5     they are ignoed if a serialization error is encountered aren't they?
16:03 sfan5     also a packet trace would also be helpful
16:03 Zeno`     They not. What I pasted above are exceptions
16:03 Zeno`     But I'll get as much data as I can
16:03 Zeno`     hopefully
16:04 sfan5     does the client crash/exit after that?
16:04 Zeno`     it exits back to main menu
16:04 Zeno`     (after displaying a dialog saying the error)
16:04 sfan5     huh and when does this happen?
16:04 Zeno`     just playing on servers
16:04 Zeno`     I have increased my logs to verbose etc
16:05 nerzhul   very strange issue
16:05 nerzhul   on all servers ?
16:05 Zeno`     on at least 3
16:05 sfan5     when we had the problem with AO messages these errors just appeared in the chat window and nothing eles
16:05 sfan5     else*
16:05 nerzhul   Zeno`: on which commit areyou ?
16:06 Zeno`     here is the only one I've managed to get since I increased log level: http://dpaste.com/356ESHB
16:06 Zeno`     one sec nerzhul I'll look
16:07 sfan5     --verbose is not enough, we need a backtrace
16:07 Zeno`     commit 8ad3dad137df740ce63101bf224e977e66b9df2c
16:07 Zeno`     sfan5, I know. I'm trying :)
16:08 Zeno`     I'll probably join a server and let client run all night I think
16:10 nerzhul   Zeno`: lol
16:11 Zeno`     :D
16:11 Zeno`     it's better than staring at it for hours hahaha
16:13 Zeno`     It's not just me experiencing these btw. It was first reported about 2 weeks ago and I kind of dismissed it thinking it was android versions or something. But then other using linux started emailing or messaging me with the same problem and then I started seeing it as well (it's relatively rare though which makes it harder)
16:14 Zeno`     My mission is to get a bt though
16:14 Zeno`     :)
16:21 nerzhul   oh i found a copy string in NetworkPacket i did some years ago i should fix
16:24 sofar     TileSpec needs to be shrunk, it's full of holes and takes up 2 cachelines
16:24 sofar     #5591
16:24 ShadowBot https://github.com/minetest/minetest/issues/5591 -- Reorder TileSpec. by sofar
16:26 nerzhul   I'm pushing a trivial fix to remove a copy string error when inserting string/wstring into packets
16:26 nerzhul   (copy string coding error, not functional)
16:27 nerzhul   pushed see https://github.com/minetest/minetest/commit/b1e6c2a9b8f10254c027fe227811fc300bae2048
16:27 nerzhul   this will remove copy string especially on debug builds
16:28 nerzhul   on release i hope compiler will optimize, else it's fixed and will permit to create mapblock packets faster :)
17:35 celeron55 i'm going to write a weird mapblock cache thing for the purpose of minimizing the time spent in the main thread copying map content to VMs to be passed to the mesh generator
17:35 celeron55 i hope at least someone is ok what that idea
17:36 nore      that was an idea I at least wanted to experiment with :)
17:36 celeron55 before someone asks: yes, i have confirmed too much time is spent doing that and it causes stutter
17:37 celeron55 (also others have confirmed it, eg. in 5239)
17:42 nerzhul   celeron55: you want to case results on mesh generator ?
17:56 celeron55 i want it to work so that a mapblock is only copied to the mesh generator if it has been updated since last time or it hasn't been used in mesh generation in a while
17:56 celeron55 from the standpoint of the main thread
17:57 sofar     another thought I had
17:57 sofar     supposedly mesh updates are done in a different thread?
17:57 celeron55 they are
17:57 sofar     well good
17:58 sofar     but does the main thread wait for that to finish?
17:58 celeron55 of course not? lol
17:58 sofar     well did you check?
17:58 celeron55 don't even try to imagine what playing MT would be like if that happened
17:58 sofar     ok, I'm just wondering if there's something like that going on
17:59 sofar     some unexpected lock or synchronization
17:59 celeron55 the mesh generator has a queue in both directions and has absolutely zero synchronization otherwise
18:00 celeron55 the issue is just that cobbling up the large amount of input data is really expensive even while the basic operations in it are very optimized
18:01 celeron55 there's some duplication in the data going in that should be removed but it's not trivial and needs some kind of a more clever system
18:01 celeron55 which is what i'm talking about
18:03 celeron55 a stupid but fast system would be to duplicate the entire Map but it would waste memory; it has to be something that retains some mapblocks but not all of them
18:06 sofar     ack
18:07 celeron55 the mesh generator requires the node data from 3*3*3 MapBlocks around the MapBlock that is being generated (really it needs 1 and all the border nodes from all the neighbors, but it's cheaper to copy entire MapBlocks in practice); currently the main thread always copies all of them, for each ingoing task involving a single MapBlock
18:07 celeron55 s/that is being generated/for which a mesh is being generated/
18:07 sofar     ok so you're thinking a cache for all the border nodes?
18:07 sofar     or cache for the meshes of those?
18:08 celeron55 no, a cache for mapblocks
18:08 sofar     ok
18:11 nore      hmmm, I'm wondering what happens when a new mapblock is received or when a set node event is
18:11 nore      does it regenerate all 26 neighbours?
18:12 celeron55 no
18:12 sofar     26? 9 * 6 = 54?
18:12 celeron55 it checks whether the node is at the edge and generates the ones it touches
18:12 nore      okay
18:12 nore      what about whole mapblock updates?
18:12 nore      sofar: 9*6?
18:13 sofar     3*3 times 6 faces?
18:13 sofar     maybe I'm just misreading
18:13 nore      oh no, I was thinking about mapblock neighbours
18:13 sofar     ok
18:15 celeron55 on a mapblock update the mapblock itself and... well i guess it should update all 3*3*3 mapblocks but it only generates 6 more
18:16 celeron55 i guess i or someone else decided that the very rare glitches are worth the speed
18:16 sofar     can you back-queue them?
18:17 sofar     iow put them on a "do these later if you have time" list?
18:17 celeron55 there generatelly is no such thing as back-queue as the server aims to send exactly enough mapblock updates to saturate the mesh generator
18:17 celeron55 it's what usually limits the mapblock transfer speed
18:17 celeron55 generally* (wtf)
18:18 celeron55 well, i guess there is time after stuff has loaded actually
18:19 celeron55 i'm imagining the backqueue getting infinitely large in heavy use though 8)
18:20 celeron55 and, currently as mesh updates involve too much work in the main thread it's mostly unimaginable to do a backqueue like that
18:21 nore      <celeron55> on a mapblock update the mapblock itself and... well i guess it should update all 3*3*3 mapblocks but it only generates 6 more <-- would it be very expensive to check all border nodes to avoid as many mesh updates as possible?
18:21 nore      (I mean, comparing them to the previous node that was there)
18:22 nore      (and only update if that one or the previous one has a drawtype that needs adjacent nodes and node was changed)
18:23 celeron55 that would involve storing the old neighbors somewhere
18:23 celeron55 maybe hashing them somehow might work
18:25 nore      hmm, I guess that would be checked when receiving the new mapblock, just before overwriting the previous one
18:25 celeron55 i don't think referring to node definitions is a good idea, but just comparing or hashing the raw node data sounds feasible
18:25 nore      hmmm, true
18:25 celeron55 the goal would be to avoid updates in the most common cases
18:26 celeron55 i.e. when literally nothing changes
18:26 nore      and anyway connect_to with groups would require updates anyway
18:26 nore      (I mean, even if both the old & new nodes are normal drawtype)