| Time |
Nick |
Message |
| 00:09 |
|
v-rob joined #minetest |
| 00:28 |
|
AliasAlreadyTake joined #minetest |
| 01:51 |
|
v-rob joined #minetest |
| 02:01 |
|
Jason232_ joined #minetest |
| 02:10 |
|
delta23 joined #minetest |
| 02:13 |
|
olliy joined #minetest |
| 02:15 |
|
Verticen joined #minetest |
| 02:54 |
|
Hawk777 joined #minetest |
| 02:55 |
erlehmann |
assuming i have binary data, like the output of minetest.compress() and want to save that in node meta. is it safe? |
| 02:55 |
erlehmann |
is it smart? probably not |
| 02:55 |
erlehmann |
oh, i want to save it in item meta too hehe |
| 02:58 |
erlehmann |
just minetest.serialize? i wonder |
| 03:16 |
|
frostsnow joined #minetest |
| 03:28 |
|
queria^clone joined #minetest |
| 03:31 |
|
jingkaimori joined #minetest |
| 03:32 |
jingkaimori |
Can this issue be closed? https://github.com/minetest/minetest/issues/3374 |
| 03:34 |
|
queria^clone joined #minetest |
| 03:36 |
MTDiscord |
<Jonathon> how about explaining why? |
| 03:36 |
MTDiscord |
<Jonathon> and env_meta.txt still randomly gets corrupted |
| 03:38 |
jingkaimori |
now mod storage is migrate to database |
| 03:40 |
MTDiscord |
<Jonathon> sure, but mods dont have db access, which is ultimately what is wanted by a lot of devs |
| 03:43 |
MTDiscord |
<Jonathon> which sorta puts https://github.com/minetest/minetest/issues/10371 and it as duplicates |
| 03:46 |
|
Hawk777 joined #minetest |
| 03:46 |
MTDiscord |
<Jonathon> ban mods are the ones already starting to role there own solutions, see sban, verbanna. other mods such as some afcm? and others are starting to use them. key point of contention being that they require insequire env and external setup |
| 03:47 |
|
v-rob joined #minetest |
| 03:47 |
MTDiscord |
<Jonathon> thanks @rubenwardy |
| 04:07 |
|
oliverwood joined #minetest |
| 04:09 |
|
Jason232 joined #minetest |
| 04:11 |
|
erlehmann joined #minetest |
| 04:17 |
|
erlehmann joined #minetest |
| 04:28 |
|
RenMinWanSui joined #minetest |
| 05:00 |
|
MTDiscord joined #minetest |
| 05:06 |
|
v-rob joined #minetest |
| 05:32 |
|
Hawk777 joined #minetest |
| 06:01 |
|
riff-IRC joined #minetest |
| 06:12 |
|
mmmax joined #minetest |
| 06:20 |
|
Jason232 joined #minetest |
| 07:03 |
|
SwissalpS left #minetest |
| 07:07 |
|
v-rob joined #minetest |
| 07:09 |
|
grouinos joined #minetest |
| 07:18 |
|
specing_ joined #minetest |
| 07:26 |
|
Gustavo6046 joined #minetest |
| 07:28 |
|
search_social joined #minetest |
| 07:28 |
|
mazes_83 joined #minetest |
| 07:28 |
|
Lunatrius joined #minetest |
| 07:28 |
|
luk3yx joined #minetest |
| 07:28 |
|
BuckarooBanzai joined #minetest |
| 07:28 |
|
bwarden joined #minetest |
| 07:28 |
|
erstazi joined #minetest |
| 07:28 |
|
asdflkj_sh joined #minetest |
| 07:28 |
|
Krock joined #minetest |
| 07:28 |
|
dabbill joined #minetest |
| 07:28 |
|
syl joined #minetest |
| 07:28 |
|
kevinsan_ joined #minetest |
| 07:28 |
|
Peppy joined #minetest |
| 07:28 |
|
Norkle joined #minetest |
| 07:28 |
|
lumidify joined #minetest |
| 07:46 |
|
calcul0n_ joined #minetest |
| 07:48 |
|
appguru joined #minetest |
| 08:15 |
MinetestBot |
[git] bell07 -> minetest/minetest_game: player_api - fix compatibility code 584a6a2 https://github.com/minetest/minetest_game/commit/584a6a21029dcac694efdf12b2a94ece06186f85 (2022-02-03T08:10:53Z) |
| 08:15 |
MinetestBot |
[git] appgurueu -> minetest/minetest_game: player_api - fix compatibility code a840ac0 https://github.com/minetest/minetest_game/commit/a840ac0a4d3db935b20ce6c6259795fbe263fefe (2022-02-03T08:15:14Z) |
| 08:17 |
sfan5 |
twice the same commit? |
| 08:18 |
sfan5 |
ah, the second is a merge commit |
| 08:18 |
sfan5 |
how did that happen |
| 08:31 |
|
Boingo joined #minetest |
| 09:02 |
|
Peppy joined #minetest |
| 09:08 |
|
Jason232_ joined #minetest |
| 09:22 |
|
mazes_80 joined #minetest |
| 10:04 |
|
grouinos joined #minetest |
| 10:14 |
|
Fixer joined #minetest |
| 10:36 |
|
HuguesRoss joined #minetest |
| 10:45 |
MinetestBot |
[git] Desour -> minetest/minetest: Add more documentation for the list[] fs element (#11979) d387e9b https://github.com/minetest/minetest/commit/d387e9b6d39d9689b79da21fa263c93d2b26e840 (2022-02-03T10:43:28Z) |
| 10:45 |
MinetestBot |
[git] TurkeyMcMac -> minetest/minetest: Clean up ClientInterface locking 1c73902 https://github.com/minetest/minetest/commit/1c73902005bb5c7a40be5571bff9c232d8c69536 (2022-02-03T10:43:51Z) |
| 10:46 |
mazes_80 |
about log: why complaining about depends.txt or description.txt when mod.conf is ab |
| 10:46 |
mazes_80 |
about log: why complaining about depends.txt or description.txt when mod.conf is there ? |
| 10:48 |
mazes_80 |
i think the case warning should be emitted is when no mod.conf, aren't previous files ignored when mod.conf exists |
| 10:54 |
|
GNUHacker joined #minetest |
| 11:06 |
|
Jason232 joined #minetest |
| 11:14 |
mazes_80 |
leveldb --recompress: for one of my local games I use leveldb. Size was 1.6G, I tried to recompress -> 2.6G |
| 11:14 |
mazes_80 |
after converting to sqlite3 and back to leveldb -> 1.4G |
| 11:31 |
|
Markow joined #minetest |
| 11:47 |
|
olliy joined #minetest |
| 11:53 |
|
SOMBRIO joined #minetest |
| 12:14 |
|
proller joined #minetest |
| 12:21 |
|
definitelya joined #minetest |
| 12:22 |
|
tech_exorcist joined #minetest |
| 13:15 |
|
olliy1or joined #minetest |
| 13:17 |
|
olliy1or joined #minetest |
| 13:20 |
|
appguru joined #minetest |
| 14:14 |
|
olliy joined #minetest |
| 14:45 |
|
kamdard_ joined #minetest |
| 15:49 |
|
oliverwood joined #minetest |
| 15:56 |
Gustavo6046 |
the stone axe in MineClone 5 is a bit overpowered, either that or the monsters are a bit too low health |
| 15:56 |
Gustavo6046 |
I don't mean this as in "it should be more accurate to Minecraft!!111!1one", I mean more like, balance |
| 15:57 |
Gustavo6046 |
Not to imply that minecraft is anyhow any sort of benchmark of balance to begin with lol |
| 15:58 |
Gustavo6046 |
but two good critical hits with a stone axe seems a bit too easy, doesn't even give time for a skeleton to shoot back at you, which would punish you for meleeing him so early game. |
| 15:59 |
|
grouinos joined #minetest |
| 16:08 |
|
ShadowNinja joined #minetest |
| 16:12 |
|
ShadowBot joined #minetest |
| 16:23 |
|
JimBer110 joined #minetest |
| 16:35 |
|
Fixer joined #minetest |
| 16:37 |
|
jluc joined #minetest |
| 16:50 |
erlehmann |
Gustavo6046 axe has high damage, but is slow. try fighting an ice bear with it. |
| 16:59 |
Gustavo6046 |
yeah, that's the case in minecraft too, but I think it feels a bit too high damage on mineclone |
| 17:13 |
erlehmann |
Gustavo6046 i doubt mineclone2 development is going anywhere tbh, but you can file a bug on mineclonia if you have evidence (e.g. youtube videos, wiki entries) that the damage should be lower for balance reasons |
| 17:13 |
|
v-rob joined #minetest |
| 17:13 |
Gustavo6046 |
ah |
| 17:13 |
erlehmann |
and mineclone5 will take literally any patch that does not crash the game (and sometimes even ones that do), so you can try there as well |
| 17:13 |
Gustavo6046 |
dang |
| 17:14 |
Gustavo6046 |
that's two extremes |
| 17:14 |
erlehmann |
there is a reason we are not all developing the same project ^^ |
| 17:14 |
|
tech_exorcist joined #minetest |
| 17:15 |
|
GNUHacker left #minetest |
| 17:31 |
definitelya |
r.i.p Mineclone 2 - supplanted by Memeclone |
| 17:31 |
MTDiscord |
<Sublayer plank> hahaha, memeclone is funny |
| 17:31 |
definitelya |
xD |
| 17:31 |
definitelya |
It's bad... |
| 17:37 |
|
Taoki joined #minetest |
| 17:39 |
erlehmann |
memeclone has gender fluid |
| 17:39 |
erlehmann |
it is left by genderman |
| 17:39 |
erlehmann |
hehehehe |
| 17:39 |
erlehmann |
i hope fleckenstein develops it further |
| 17:39 |
erlehmann |
it is just too silly |
| 17:41 |
definitelya |
Scariest man after Slenderpants. |
| 17:42 |
definitelya |
But yeah that makes two I guess. All for it. |
| 17:55 |
erlehmann |
appguru do you have an example of compressed data being serialized/deserialized for item meta? |
| 17:55 |
erlehmann |
output of minetest.compress() i mean |
| 17:55 |
appguru |
I don't |
| 17:55 |
erlehmann |
then i use base64 instead i guess |
| 17:55 |
MTDiscord |
<Jonathon> never use minetest.deserialize with big data |
| 17:56 |
erlehmann |
this is like 10k maximum data |
| 17:56 |
appguru |
is it for ender chests? |
| 17:57 |
erlehmann |
no, it is for pixel arrays in maps. i figured that if i save the pixel data in the item, it works in world downloads |
| 17:57 |
erlehmann |
and also you can re-encode it later |
| 17:57 |
appguru |
I'm still convinced that the best way to store large item-related data would be files or a proper database |
| 17:57 |
appguru |
unfortunately there are no proper callbacks for when an item goes missing |
| 17:57 |
erlehmann |
oh, they are stored in files server-side |
| 17:57 |
appguru |
and when are they dropped? |
| 17:57 |
erlehmann |
never lol |
| 17:58 |
erlehmann |
so i want the files to be disposable |
| 17:58 |
erlehmann |
i.e. when the file does not exist, code encodes the pixel to a texture and sends dynamic media |
| 17:58 |
erlehmann |
seriously, i know this makes the item heavier, but i see no other way to make world dls work |
| 17:58 |
erlehmann |
also this would be a good test case for the anon5 proxy thing |
| 17:59 |
erlehmann |
cora told me that anon5 proxy is just cleverer about minetest protocol than minetest itself hehe |
| 17:59 |
erlehmann |
i feared it might break map dls |
| 17:59 |
|
Talkless joined #minetest |
| 18:01 |
MTDiscord |
<Warr1024> If you use zlib compression and then base64 on plaintext, it can still often come out more compact than the original text, even if it's just an ascii->ascii transformation in the end |
| 18:01 |
MTDiscord |
<Warr1024> What is the "anon5 proxy thing"? |
| 18:02 |
erlehmann |
https://github.com/OysterityAnarchy/mt-netopt-proxy |
| 18:03 |
erlehmann |
i do not understand the code, but i was told it makes for much less inventory traffic |
| 18:06 |
erlehmann |
i have a question: when exactly is mod security blocking my file writes and when not? |
| 18:06 |
erlehmann |
i have one mod calling a function from another mod writing files and that is not blocked |
| 18:06 |
erlehmann |
but when i do it in the first mod itself, mod security blocks it |
| 18:06 |
erlehmann |
in 5.4 |
| 18:06 |
appguru |
mods can write to their own folders |
| 18:06 |
appguru |
to do so, they must obtain the handles at load time AFAIK |
| 18:07 |
appguru |
mods can only read from the folders of other (world)mods |
| 18:07 |
erlehmann |
so i guess i somehow did not give the first mod that handle. but how does the handle look like? |
| 18:07 |
appguru |
the handle is the return value of io.open |
| 18:08 |
erlehmann |
local f = assert(io.open(filename, "wb")) |
| 18:08 |
erlehmann |
f:write(self.data) |
| 18:08 |
erlehmann |
f:close() |
| 18:08 |
appguru |
f is the handle |
| 18:08 |
erlehmann |
yeah, but it is not a handle to the folder |
| 18:08 |
erlehmann |
the thing is, i do not understand when i can write new textures at runtime |
| 18:08 |
appguru |
no, the handles are always to files which must be within the mod folder |
| 18:08 |
appguru |
erlehmann: it might be impossible |
| 18:09 |
erlehmann |
oh but this is in world folder |
| 18:09 |
erlehmann |
appguru it is evidently possible |
| 18:09 |
erlehmann |
tga_encoder does it, for example |
| 18:09 |
appguru |
you should be able to write almost anywhere in the world folder, except for the worldmods |
| 18:09 |
erlehmann |
then maybe i get the path wrong |
| 18:09 |
appguru |
I write textures in epidermis to, and I write them to the world folder as well |
| 18:09 |
rubenwardy |
erlehmann: you can only write to your mods folder whilst your mod is loading |
| 18:09 |
appguru |
I was saying it might be impossible to write to the mod folder at runtime |
| 18:10 |
rubenwardy |
once the game has loaded, all mod folders are read only |
| 18:10 |
erlehmann |
i want to write to the world folder though |
| 18:10 |
rubenwardy |
well, you can have handles open at load time and then write later |
| 18:10 |
rubenwardy |
you can write to the world folder at any time |
| 18:10 |
appguru |
not if the file names are only known at runtime though |
| 18:10 |
rubenwardy |
providing it's not in worldmods or minetest.conf |
| 18:10 |
appguru |
or if you don't want to open 1000s of handles in preparation xD |
| 18:10 |
rubenwardy |
both are protected for security reasons |
| 18:11 |
erlehmann |
i guess i got the filepath wrong |
| 18:11 |
rubenwardy |
yeah |
| 18:11 |
rubenwardy |
use minetest.get_worldpath() .. "/filepath.txt" |
| 18:11 |
rubenwardy |
(MT converts / to \ on windows correctly) |
| 18:11 |
MTDiscord |
<luatic> is that documented anywhere? |
| 18:13 |
rubenwardy |
don't know, but should be |
| 18:13 |
MTDiscord |
<Jonathon> no |
| 18:13 |
MTDiscord |
<Warr1024> Haha, crap, I got all excited but I guess it looks like that proxy wouldn't help me much. |
| 18:14 |
erlehmann |
Warr1024 what is the matter with it? |
| 18:15 |
erlehmann |
can i maybe not write to existing files o.0? |
| 18:28 |
MTDiscord |
<Warr1024> I mean that proxy sounds like it mainly solves a problem I don't have. |
| 18:28 |
MTDiscord |
<Warr1024> Mainly what I've been seeing are object properties and block sends. |
| 18:28 |
MTDiscord |
<Warr1024> I assume block sends I can't do much about, but I know object properties are pretty hardcore-inefficient. |
| 18:37 |
|
jfred joined #minetest |
| 18:43 |
erlehmann |
rubenwardy appguru i had indeed used the wrong path |
| 18:43 |
erlehmann |
2022-02-03 19:43:30: [Server]: uncompressing texture |
| 18:43 |
erlehmann |
segmentation violation |
| 18:43 |
erlehmann |
nice lol |
| 19:06 |
|
Verticen joined #minetest |
| 19:07 |
|
v-rob joined #minetest |
| 19:10 |
|
lagash joined #minetest |
| 19:12 |
ghoti |
Er, newbie lua question here... I know how I can read data from a file with io.input(), but how do I read the array stored in xban.db so that I can do things with it? It there some soft of "eval()" that I haven't found yet? |
| 19:15 |
MTDiscord |
<Warr1024> if it's like return { stuff } then you'd use minetest.deserialize |
| 19:16 |
MTDiscord |
<Warr1024> if it's like {"stuff":"more stuff"} then that's JSON and you'd look for the JSON-reading/parsing functions, I don't remember what they're called. |
| 19:19 |
|
specing_ joined #minetest |
| 19:19 |
erlehmann |
local function load_json_file(name) |
| 19:19 |
erlehmann |
local file = assert(io.open(modpath .. "/" .. name .. ".json", "r")) |
| 19:20 |
erlehmann |
local data = minetest.parse_json(file:read()) |
| 19:20 |
erlehmann |
file:close() |
| 19:20 |
erlehmann |
return data |
| 19:20 |
erlehmann |
end |
| 19:20 |
erlehmann |
ghoti, does that help you? |
| 19:20 |
MTDiscord |
<Jonathon> dont use xban without patching its data lose |
| 19:20 |
MTDiscord |
<Jonathon> *loss |
| 19:21 |
MTDiscord |
<Jonathon> xban should use json to store its data so it doesnt lose it, but krock closed that issue as fixed when it really isnt |
| 19:21 |
MTDiscord |
<Jonathon> areas, and (mt-mods) travelnets used to have this issue but have been fixed |
| 19:23 |
ghoti |
erlehmann: it is surely helpful information, excalt that the file appears to be in lua array format rather than JSON. `return { .. [234] = { ["names"] = { "foo" = true, ...` |
| 19:23 |
ghoti |
This is xban.db |
| 19:23 |
erlehmann |
well then listen to jonathon instead of me ig |
| 19:23 |
erlehmann |
i know nothing about xban db format |
| 19:23 |
erlehmann |
on the data louse |
| 19:24 |
MTDiscord |
<Jonathon> minetest.deserialize(xbandb) |
| 19:24 |
MTDiscord |
<Jonathon> but you should really patch it to use json |
| 19:24 |
MTDiscord |
<Jonathon> unless you like all your bans going away every so often |
| 19:24 |
ghoti |
Jonathon, thanks, I am only looking to analyze my daily snapshots of xban.db using a standalone tool, the results do not need to be complete. Just dealing with the basic of lua still. |
| 19:24 |
ghoti |
JSON would be much better, yes. |
| 19:25 |
ghoti |
Jonathon, where would I look for the patch? Should I search closed issues on github? |
| 19:26 |
MTDiscord |
<Warr1024> If you're capturing daily snaps, make sure to keep some old copies around in case the aforementioned data loose happens so you can recover from it. |
| 19:26 |
ghoti |
I started keeping daily snapshots when I realized I was losing old data. :/ |
| 19:27 |
MTDiscord |
<Jonathon> anyways, you know xban has a api right? |
| 19:27 |
MTDiscord |
<Jonathon> xban.db gets you it |
| 19:27 |
MTDiscord |
<Warr1024> Maybe they should consider switching to mod_storage now that it's actually using a database backend... |
| 19:27 |
ghoti |
MTDiscord: is that API available to a external scripts? (Sorry, I am very new to lua.) |
| 19:28 |
MTDiscord |
<Jonathon> external as in external mods? or something outside of minetest |
| 19:28 |
ghoti |
oops, s/MTDiscord/Jonathon/ |
| 19:28 |
ghoti |
Jonathon, external as in, a munin or nagios plugin? |
| 19:28 |
ghoti |
Outside of minetest entirely. |
| 19:28 |
MTDiscord |
<Jonathon> ah, "a standalone tool" |
| 19:29 |
ghoti |
yep. |
| 19:29 |
MTDiscord |
<Jonathon> then you would need to reimplement minetest.deserialize outside minetest |
| 19:29 |
ghoti |
JSON seems like a way easier option. :) |
| 19:29 |
MTDiscord |
<Jonathon> even more reason you should patch it to use json, just import any json library you want |
| 19:29 |
ghoti |
Or, require()? |
| 19:29 |
ghoti |
Yes, looking for the patch now.. |
| 19:30 |
MTDiscord |
<Jonathon> its not public |
| 19:30 |
MTDiscord |
<Warr1024> Lua has loadstring/dofile that I think can be used to deserialize minetest.serialize()d data; it's basically just lua code. Be absolutely sure you trust any data that you're attempting to decode though, and/or run your scripts as sandboxed as you reasonably can. |
| 19:30 |
MTDiscord |
<Jonathon> https://github.com/minetest-mods/xban2/issues/19 here is the issue that krock claims fixes it, which is literally just a clean up command to make it smaller, which eventually it will lose its mind anyways |
| 19:31 |
MTDiscord |
<Jonathon> https://github.com/minetest-mods/areas/commit/a9c05f0e387ddcbc196d263e6338d27eb286b04e areas fix, https://github.com/mt-mods/travelnet/commit/d8c0c9d42318126b1759f589a38d093c9e3cc389 travelnet fix |
| 19:31 |
MTDiscord |
<Warr1024> Any format that involves serializing and deserializing the entire database to/from a string is gonna run into SOME problem eventually though, so switching over to JSON just changes it from an outright crash to a slower degradation over time. |
| 19:32 |
MTDiscord |
<Jonathon> . s/crash/data loss and you would be correct |
| 19:32 |
MTDiscord |
<Jonathon> but minetest is yet to provide proper db access, so sban, verbanna, etc have to turn of mod sec and use external dbs |
| 19:33 |
MTDiscord |
<Warr1024> when mod_storage was just a flat text file, there was no solution for this other than either doing your own custom multi-file database, or suffering external dependencies and complex installation. I'm hopeful that db-driven mod_storage fixes it, but tbh haven't tested it yet. |
| 19:33 |
ghoti |
And minetest doesn't use posgresql transactions, does it? I get the impression that postgres support is basically "swap in postgres commands instead od sqlite ones" mostly? |
| 19:33 |
MTDiscord |
<Warr1024> mod_storage won't give you a full-blown DB, but it should hopefully at least give you a competent key/value store, upon which you can implement DB-ish things. |
| 19:33 |
MTDiscord |
<Jonathon> the problem with mod storage is searching it |
| 19:34 |
MTDiscord |
<Warr1024> yeah, you'd have to maintain an index manually |
| 19:34 |
MTDiscord |
<Warr1024> not impossible, but also not pretty |
| 19:34 |
MTDiscord |
<Jonathon> but yeah, now that its sql backed, its probably the most secure storage available for mt mods within the sand box |
| 19:34 |
MTDiscord |
<Warr1024> Also, extra irony points if you implement a table store on top of a key value store that's now already on top of a table store :-D |
| 19:35 |
MTDiscord |
<Jonathon> well, loading from mod storage into a table gives you faster lookup |
| 19:35 |
MTDiscord |
<Warr1024> most SQL databases offer some ACID guarantees at least within individual queries, so postgresql and sqlite should have comparable safety guarantees |
| 19:35 |
MTDiscord |
<Jonathon> the problem with giving mods access to pure sql is how do you handle db migration/differences in queries |
| 19:35 |
MTDiscord |
<Warr1024> considering that MT saves all its data fully async and offers mod authors no durability guarantees, this is all about the same anyway. |
| 19:36 |
MTDiscord |
<Jonathon> course the easy option is give mods only sqlite3 access since you always have to have it |
| 19:36 |
rubenwardy |
Minetest deserialize is just Lua loadstring |
| 19:36 |
MTDiscord |
<Warr1024> MT doesn't guarantee that your back-end is SQL either. You can use redis, and mongodb wouldn't be a stretch. |
| 19:36 |
rubenwardy |
But with sandboxing |
| 19:38 |
MTDiscord |
<Jonathon> using Minetest.deserialize is like randomly waving a shotgun around and firing, then hoping you didnt get shot |
| 19:38 |
MTDiscord |
<Warr1024> I briefly looked into making an external pluggable backend system by just emulating the subset of the Redis protocol that MT uses and pretending to be a Redis server for it. That would allow me to e.g. record changes into a sqlite db but also separate out the WAL checkpointing, so I would retain having the most universal portable format, but not have to pay the cost in lagspikes. |
| 19:38 |
MTDiscord |
<Warr1024> minetest.deserialize isn't exactly terrible, we just have better tools than it in every way but are stuck with it for legacy reasons. |
| 19:39 |
MTDiscord |
<Jonathon> basically |
| 19:39 |
MTDiscord |
<Jonathon> if you really want legacy, look at minetest.compress |
| 19:40 |
MTDiscord |
<Warr1024> minetest.compress ... looks fine to me? Or at least about what I'd expect for a simplified zlib API with room to expand that nobody ever actually availed themselves of. |
| 19:40 |
erlehmann |
it is used |
| 19:41 |
erlehmann |
zlib is in png and … mts ig? |
| 19:41 |
MTDiscord |
<Jonathon> minetest.compress(table.concat(table, "")) |
| 19:41 |
MTDiscord |
<Jonathon> ^dont use that please |
| 19:41 |
ghoti |
Warr1024, I see regular crashes when the server tries to create a user that already exits, |
| 19:41 |
MTDiscord |
<Warr1024> Wat? That doesn't even make sense. It's not the compress that doesn't make sense there, it's the concat |
| 19:42 |
MTDiscord |
<Warr1024> like there's no local table = table.unconcat(string, "") or something XD |
| 19:42 |
ghoti |
'ERROR: duplicate key value violates unique constraint "auth_name_key"' |
| 19:42 |
MTDiscord |
<Jonathon> yeah, ive seen it and ran away screaming |
| 19:43 |
MTDiscord |
<Warr1024> concat/split is a can of worms because whatever you use as the separator string, you have to find some way to guard against that appearing INSIDE the strings that you're concatting. |
| 19:43 |
MTDiscord |
<Jonathon> warr, i know, i already said it was terrible |
| 19:43 |
MTDiscord |
<Warr1024> Then of course you can only actually join a single-layer flat array with integer keys this way ... basically you'll have just reinvented the mess that is texturemod escaping. |
| 19:44 |
MTDiscord |
<Warr1024> Yeah, terrible just like texturemod escaping XD |
| 19:44 |
MTDiscord |
<Warr1024> ghoti, I know of no reason why MT would be attempting to insert duplicate users into the database instead of just looking up and finding the existing one. |
| 19:44 |
MTDiscord |
<Warr1024> MT's auth system is not really my area of expertise though |
| 19:45 |
MTDiscord |
<Warr1024> IIRC if you modify the auth data directly on the back-end, or I suppose maybe share auth data between worlds (which might itself not be a sane thing to attempt) then you might have to do some kind of /auth_reload kind of thing to refresh the MT-side cache. |
| 19:47 |
ghoti |
Warr, I wondered if perhaps it could be as simple as someone double-clicking a Register button in the client gui, and there being insufficient error handling on the server. It sure feels like an error I should not see. |
| 19:47 |
rubenwardy |
ghoti: what database backend? |
| 19:47 |
rubenwardy |
It might be due to case sensitive bugs |
| 19:47 |
ghoti |
postgresql. It has happened with both minetest and multicraft servers. |
| 19:48 |
ghoti |
I have seen 56 of these since the start of the year. :) |
| 19:48 |
rubenwardy |
Yeah, there's an issue for that |
| 19:49 |
MTDiscord |
<Warr1024> Haha, good thing I heard about this before I decided to convert my auth backend to postgresql ? |
| 19:50 |
MTDiscord |
<Jonathon> anyways, tldr minetest data storage, always make backups because its guaranteed to eventually lose it |
| 19:51 |
ghoti |
Warr, I still prefer having everything in postgres, so it is easier to include in dumps. |
| 19:51 |
MTDiscord |
<Warr1024> Oh, also, since DB experts are around atm: if I, say, update and blank out the password hash column in auth.sqlite, then use a vacuum command, since that basically dumps and rewrites the database from scratch, that should mean that there's no residual password hash data left in the file that it writes out, right? |
| 19:52 |
MTDiscord |
<Jonathon> >experts leaves the room |
| 19:52 |
MTDiscord |
<Warr1024> ghoti, I really like using postgres for running the server, and I also back up the postgres data files themselves, but I like using sqlite as a "transfer format" for MT worlds because of its universality and portability. |
| 19:52 |
MTDiscord |
<Warr1024> So my public backups are actually a sync from postgresql to sqlite. |
| 19:53 |
ghoti |
Warr, I hear ya, that is a valid use. But is there a way to convert things back from postgres to sqlite for transport? |
| 19:53 |
MTDiscord |
<Warr1024> Also means that verifying my backups are just a matter of downloading the image and trying to run it in SP, I don't have to setup a local postgresql server. |
| 19:53 |
MTDiscord |
<Jonathon> minetest --migrate |
| 19:53 |
ghoti |
I do a daily pg_dump, but I have been told that doing this while the server is running guarantees a corrupt dump. I just dont know if it is corrupt enough for me to worry... |
| 19:53 |
MTDiscord |
<Warr1024> --migrate works okay, but it recreates the sqlite db each time, which means it can be organized differently, meaning you get less deduping in backups |
| 19:54 |
ghoti |
Jonathan, oh, --migrate goes both ways? That is good news indeed. |
| 19:54 |
MTDiscord |
<Warr1024> pg_dump really looks like it should at least give you consistency and row-level atomicity, though, which is about as good as MT itself guarantees anyway. |
| 19:54 |
MTDiscord |
<Warr1024> Migrate goes every direction, assuming, of course, that the source db isn't already corrupted. |
| 19:54 |
ghoti |
The fellow who warned me about this is very familiar with minetest internals but not so familiar with postgres. :) |
| 19:55 |
MTDiscord |
<Jonathon> pg_dump should give you valid db |
| 19:55 |
MTDiscord |
<Jonathon> pretty sure its what buckaroo uses |
| 19:55 |
MTDiscord |
<Warr1024> What I know WILL give you a risk of corruption is trying to zip up a sqlite db while it's open by MT. |
| 19:55 |
ghoti |
Jonathan, "valid" as a db, sure. But valid as a world? |
| 19:55 |
MTDiscord |
<Jonathon> now dont be the idiot who cp -r a db while minetest is running |
| 19:56 |
MTDiscord |
<Jonathon> i believe\ so |
| 19:56 |
ghoti |
confident++ |
| 19:56 |
MTDiscord |
<Jonathon> but also test first when it comes to data |
| 19:56 |
ghoti |
thanks |
| 19:56 |
MTDiscord |
<Jonathon> better to check your backup before needing it and finding out its broken |
| 19:56 |
MTDiscord |
<Jonathon> totally not talking from experience there |
| 19:56 |
ghoti |
I would expect there to be some loss when it comes to minetest's async updates, which is to be expected. |
| 19:57 |
MTDiscord |
<Warr1024> For my snapshots, I pull all the world files via rsync, and then do some basic validation of them. The players and mapblock data are pulled separately out of postgresql, and I actually use pg_dump, pipe it through a short perl script to convert it into a sqlite script, and then pipe that into sqlite to apply it to the backup. |
| 19:57 |
ghoti |
Warr, you pull in sqlite3 files via rsync _while the server is running_? |
| 19:57 |
MTDiscord |
<Warr1024> I pull the old backup snapshot into a staging dir, update it there, validate it, and if it all succeeds, I do an rsync --delay-updates to deploy it back into my backup share. |
| 19:58 |
MTDiscord |
<Warr1024> The only sqlite3 file in my world is auth.sqlite, which I know isn't corrupted because if it were then the command to scrub out password hashes in the public copy would fail. |
| 19:58 |
MTDiscord |
<Warr1024> My published auth.sqlite has passwords removed so that people can't offline-brute-force attempt against other peoples' accounts, and only auth.sqlite.pgp has the passwords in it. |
| 19:59 |
MTDiscord |
<Warr1024> https://nodecore.mine.nu/world/ |
| 19:59 |
MTDiscord |
<Warr1024> oh, I guess it's a .asc file |
| 19:59 |
ghoti |
I store everything I can in postgres, and pg_dump daily. The dump gets stored for a week, but each one is also imported into an alternate server in case I want to get up and running quickly after a failure of the primary server. |
| 20:00 |
ghoti |
What I would really like to get going is database replication. |
| 20:00 |
MTDiscord |
<Jonathon> doesnt postgresql support backing up to another instance on a different server |
| 20:00 |
appguru |
ghoti: Serialization is incorrectly demonized here. JSON is inferior to minetest.serialize in general. There is one special case if you happen to (mis)use serialization for large amounts of data where a crash will mean a server running LuaJIT won't be able to deserialize it anymore. |
| 20:01 |
MTDiscord |
<Warr1024> I basically dump from pgsql to sqlite every 2 hours or so. I tend to pull backup images about every 8 hours, which includes both the raw and exported data, and I commit backup snaps to borgbackup every ~24 hours, and then thin out the images over time. |
| 20:01 |
MTDiscord |
<Warr1024> Also some players pull backups too, though I don't know their schedules. |
| 20:02 |
MTDiscord |
<Warr1024> appguru: JSON is probably at least as good as Lua serialization for things that can be represented in JSON. Lua serialization supports more structures, but still is missing some (like ItemStacks) that you'd think should be supported. |
| 20:02 |
ghoti |
appguru: Are you suggesting that I would in general be better off not converting various pieces to store data in JSON rather than serialized lua? |
| 20:02 |
appguru |
The real issue here is in fact that xban chooses to store large amounts of k/v data as a serialized file instead of a proper DB structure. |
| 20:03 |
appguru |
I'm saying for the "small" data sizes serialization to Lua is intended for, you can and should use it. |
| 20:03 |
MTDiscord |
<Warr1024> To be fair for most of xban's life, the options provided by MT itself have been pretty bad. |
| 20:03 |
MTDiscord |
<Jonathon> xban is in minetest-mods which is basically dead anyways |
| 20:03 |
MTDiscord |
<Warr1024> mod_storage in an actual proper DB is also new enough in 5.5, and 5.5 itself new enough, that it might be unwise to rely on people having access to "the good stuff" for now. |
| 20:03 |
appguru |
Warr1024: Which is why (one of) modlib's luon serializers has support for ItemStacks :D |
| 20:04 |
ghoti |
appguru: I would be perfectly happy with a small tool to convert lua serialized data to json for analysys by my external tools rather than having to write stuff in lua.. I am much more comfortable in other platforms. |
| 20:04 |
MTDiscord |
<Jonathon> bloatlib shills now heh |
| 20:05 |
appguru |
such a tool is trivial to write in Lua hehe |
| 20:05 |
ghoti |
:) |
| 20:05 |
appguru |
anyways, MT's JSON implementation is IMO trash |
| 20:05 |
appguru |
good enough for simple JSON though |
| 20:06 |
ghoti |
appguru: does the data stored in xban.db count as "simple"? |
| 20:06 |
appguru |
for more details, have a look at https://github.com/minetest/minetest_docs/pull/13 |
| 20:06 |
appguru |
ghoti: yes, you could migrate it to JSON |
| 20:06 |
appguru |
I'm pretty sure writing will be roughly half as fast though |
| 20:09 |
MTDiscord |
<Warr1024> Really would be nice if the Lua language would just have included a "native" serialization format. One of the reasons why JSON is basically unquestioningly the format of note in JS is because of its explicit support in the language itself. |
| 20:09 |
|
TomTom joined #minetest |
| 20:09 |
MTDiscord |
<Warr1024> It started its life actually as about the equivalent of MT's lua serialization, i.e. something designed to be eval'd, and ended up growing strictness and a non-eval parser along the way. |
| 20:10 |
MTDiscord |
<Jonathon> good luck with that given lua 5.1 is eol |
| 20:10 |
MTDiscord |
<luatic> https://www.npmjs.com/package/luon ? |
| 20:10 |
|
proller joined #minetest |
| 20:10 |
MTDiscord |
<Warr1024> Lua is a bit lispy though in that they provide you with very flexible tools to solve a problem, and then more or less refuse to be prescriptive about how, and thus that tends to encourage solution fragmentation. |
| 20:11 |
MTDiscord |
<Warr1024> Is luon part of the lua language, or just something built on top of it |
| 20:11 |
MTDiscord |
<luatic> Luon is like JSON for JS, except it's not built on Lua but on JS |
| 20:11 |
MTDiscord |
<Warr1024> wait whytf is that on npm? |
| 20:11 |
MTDiscord |
<luatic> hey, I wanted to practice some JS |
| 20:12 |
MTDiscord |
<Warr1024> oh, I guess if lua can attempt to write JSON, then JS attempting to read Luon is not all that different. ¯_(ツ)_/¯ |
| 20:13 |
MTDiscord |
<luatic> that's came out a lot shorter though: https://github.com/appgurueu/modlib/blob/master/json.lua |
| 20:15 |
sfan5 |
is that a json parser? |
| 20:17 |
ghoti |
I found this.. https://github.com/agladysh/lua2json .. but I don't seem to be able to run it. |
| 20:18 |
MTDiscord |
<Warr1024> Heh, in cdbrelease, I just remembered that I wanted to have JS call a lua function that returns a table ... so I write a tiny JSON serializer for the output so it can be parsed on the JS side. |
| 20:36 |
|
v-rob joined #minetest |
| 20:38 |
|
oliverwood joined #minetest |
| 20:39 |
|
jordan4ibanez joined #minetest |
| 21:14 |
|
v-rob joined #minetest |
| 21:17 |
|
Jason232 joined #minetest |
| 21:18 |
|
v_rob joined #minetest |
| 21:20 |
|
DarkSide joined #minetest |
| 21:25 |
lagash |
Why not use MsgPack? https://github.com/kengonakajima/lua-msgpack-native |
| 21:26 |
|
vrob joined #minetest |
| 21:29 |
|
mazes_80 joined #minetest |
| 21:41 |
|
vrob joined #minetest |
| 21:59 |
|
v-rob joined #minetest |
| 22:16 |
|
v-rob joined #minetest |
| 22:29 |
|
proller joined #minetest |
| 22:58 |
|
Jason232 joined #minetest |
| 23:01 |
|
Verticen joined #minetest |
| 23:20 |
|
Markow joined #minetest |
| 23:32 |
|
jordan4ibanez joined #minetest |
| 23:37 |
erlehmann |
ghoti, you need to do file.read("a") to read all of the file sorry. |
| 23:38 |
erlehmann |
otherwise you only get one line |
| 23:39 |
jordan4ibanez |
Fun fact: His name is pronounced "fish" |
| 23:39 |
|
Peppy joined #minetest |
| 23:47 |
erlehmann |
i do know that |
| 23:47 |
MTDiscord |
<Warr1024> ffici is another fun one |
| 23:48 |
MTDiscord |
<Warr1024> that one only comes from one word. |
| 23:48 |
|
v-rob joined #minetest |
| 23:56 |
|
Markow joined #minetest |
