Time Nick Message 00:30 MTDiscord Got the answer for zlib, it's about 368 bytes on avg for MTG. Over here on discord there's been a lot of discussion about using disctionaries for mapblocks. The current thoughts are that they should be able to be applied differently from the storage side and the network side. The reason is the network side already recompress on the fly, and the client would have to load that dictionary with media at first connect 05:43 nrz_ dictionaries are far more huge than the current level, unfortunately 05:43 nrz_ i tested various way to store the mapblocks and the current serialization model is the far more efficient, not regarding the compression method 06:31 MTDiscord Anything keeping #11307 from getting a quick once-over? It's a trivial fix but kinda needed for my game to look right. 06:31 ShadowBot https://github.com/minetest/minetest/issues/11307 -- falling.lua - Fix Meshnodes Being Too Big by benrob0329 13:27 MTDiscord Far more huge? Dictionaries are typically 64-256 KB 13:27 MTDiscord And relatively easy to generate with zstd's professional grade dictionary builder 13:28 MTDiscord And this is actually about sending the mapblocks over the network not storage, mostly 20:23 ShadowNinja I'm working on rework of the server list to make it store servers perisitently and fix the multiprocessing issue (so that you can have multiple processes instead of just multiple threads that have to share the GIL). 20:25 ShadowNinja One issue is how to identify servers. Currently we use a combination of announce ip / port, but the IP could change. A better option would be announce address / port, but the address can be spoofed if we don't enable verification that the announce IP is included in the records for the address. 20:27 ShadowNinja Address / port could still change though. A server might change domains or ports, or a address/port might be reused. 20:27 ShadowNinja I'd like to have some sort of persistent identifier in the announcement that can be used to isedtify the world. 20:27 MTDiscord The only way to really preclude spoofing would be for servers to provide an additional identifier, really ideally something like an ed25519 pubkey that's compact enough to fit in a config file but verifiable publicly. 20:28 MTDiscord Without making that change it does feel like address/port is de facto what players are actually using right now to identify servers. 20:28 MTDiscord have you considered that the server list should send a UUID to that server 20:28 ShadowNinja I think some other servers use a private key, but if we want to keep it simple we could probably get away with just a large random number as long as it's kept secret. 20:28 MTDiscord UUID is the poor man's pubkey 20:29 MTDiscord UUID is also easily randomisable 20:29 MTDiscord Pretty sure it can't be kept secret though? 20:29 rubenwardy Store a random private key and store it in a world file, fallback to IP/port 20:30 MTDiscord openssl imports already exist in MT for use for SRP, so you should be able to do EC pubkey stuff using the existing dependencies at least... 20:30 ShadowNinja If we use a private key we'd have to have some sort of protocol to verify the key though. 20:30 MTDiscord we already use SRP 20:30 MTDiscord why not reuse SRP for logging server in 20:30 MTDiscord who is doing the verifying? Clients verifying their server, or just the server list verifying for the listing? 20:30 MTDiscord via minetest.conf or world.conf 20:31 rubenwardy private key can be a random string 20:31 MTDiscord I mean, I guess we could use SRP to identify servers to the server list... 20:31 MTDiscord it's not like you need an "account creation" process per se 20:31 ShadowNinja SRP is challenge-response IIRC, I'd like to verify the server with just the single request. 20:31 MTDiscord though I think SRP would have to generate new random parameters and store those 20:32 MTDiscord SRP is interactive challenge-response so you couldn't really use it. A pubkey though could be used to sign timestamp sort of OCSP-stapling-style, so you could do that as a single one-way communicatin. 20:33 MTDiscord So like a server announce could just include (1) the pubkey, and (2) a signature of a timestamp that's recent enough that verifies with that pubkey, and if they match, the server list can publish that server's pubkey (or even just a hash of it) as a UID. 20:34 rubenwardy JWT could also be used 20:34 MTDiscord sure, that's one specific implementation of the general idea 20:35 MTDiscord whether you publish the full pubkeys or just the hashes in the serverlist, clients can use that to verify that the server they're about to log into is (at least at the time) the same one they were told about by the serverlist 20:35 MTDiscord though of course, the MT wire protocol doesn't have any session security so an attacker could always just selectively alter traffic that way 20:35 MTDiscord so there's relatively little value in that I suppose 20:36 MTDiscord but at least with the pubkey thing spoofing a server to the server list server would require an MITM of some kind to intercept the announce call, it's not like you could just steal the secret and then reuse it indefinitely from wherever. 20:37 pgimeno ShadowNinja: how do you plan to support current Lua mapgens? 20:39 ShadowNinja pgimeno: This doesn't have anything to do with mapgen. The engine would just generate a keypair / UUID for the world when you create it, just like it generates world.mt, map_meta.txt, etc. 20:39 MTDiscord Putting it in the world feels sorta wrong somehow, but it really is the best place after all. 20:40 ShadowNinja Yeah, I think we really want to track worlds, that way you can move between hosts, domains, and ports and keep all of the announce history for that world. 20:43 pgimeno [...] fix the multiprocessing issue (so that you can have multiple processes instead of just multiple threads that have to share the GIL) 20:43 ShadowNinja It sounds like there are two good options: 1) generate a random UUID and use that essentially as a world password (nice and simple) 2) generate a pubkey/privkey pair, send the pubkey in the announce, and sign the announce message with the private key (less simple, but could be used for other things like authenticating the server on connect) 20:43 pgimeno Lua mapgens are the only use case for the GIL AFAIK 20:44 rubenwardy They're talking about the serverlist and the Python GIL 20:44 pgimeno ahh 20:45 pgimeno never mind then 20:45 MTDiscord It depends on whether using pubkey signing/verifying code is more "complex" to you, or having to have a challenge-response interactive exchange is more "complex" 20:45 ShadowNinja See here, you have to tell mod_wsgi not to use multiprocessing so that you don't have several processes with different copies of the server list: https://github.com/minetest/serverlist/blob/master/README.md#setting-up-the-server-apache-version 20:46 ShadowNinja @Warr1024 no need for challenge response, the server could just send the UUID. 20:47 MTDiscord I guess that's not unreasonable given that security is already out the window in a lot of other contexts 20:47 MTDiscord Presumably at least you can't make it worse than it is now 20:49 ShadowNinja This would of course have some of the same issues that using plaintext passwords has. Your UUID could get leaked by e.g. MITM, whereas with a key pair only the pubkey can be MITMed. 20:49 MTDiscord On a side note allow me to point out that, while it doesn't particularly matter here, UUID is itself one of the stupidest data types available. Instead of a 128-bit random number, it's a 122-bit random number that introduces utterly bizarre endianness issues. It's almost as bad as every datetime format out there. 20:50 MTDiscord The UUID plaintext scenario is only worse than things are currently if it causes players to expect to be able to rely on the security of the server list more than they already do. 20:50 MTDiscord As it stands right now users are just always assuming that the server they're connecting to isn't some clone or DNS hijack or spoof or whatever of a real server. Using UUIDs in plaintext arguably makes things no better, no worse. 20:51 MTDiscord It's only a risk if it carries some perception or assumption of better security... 20:52 MTDiscord If we could wrap the whole session in mutually-authenticated DTLS or something then that would be fantacular but I just don't see that happening :-) 20:53 MTDiscord tbh for the MT game session, I don't think DTLS really ever got traction anyway, and the wireguard protocol might be better if we really wanted to consider it. 20:56 ShadowNinja QUIC is a pretty interesting option that offers encryption with multiple queues. Wireguard doesn't really have much of a protocol AFAIK, it's basically just signs packets and sends them over UDP. 20:59 MTDiscord My idea behind wireguard is just to wrap raw datagrams of the existing protocol, and we'd be able to keep the original UDP implementation available for easier debugging, LAN play without mucking about with key exchanges, etc. Something richer with multiple queues sounds more like something to replace parts of what we already have, which may or may not be an additional can of worms we would want to open. 21:00 MTDiscord The idea behind wireguard is to just shove it in there as a layer in the protocol, without having to have its tentacles everywhere quite yet... 21:01 MTDiscord Other things like QUIC might be able to fit this role too, I dunno. There's probably a lot of options to shop among, when you keep your needs pretty simple. 21:10 ShadowNinja Hmmm, I think it would be better to integrate it with the protocol.