Minetest logo

IRC log for #minetest-dev, 2021-01-29

| Channels | #minetest-dev index | Today | | Google Search | Plaintext

All times shown according to UTC.

Time Nick Message
00:09 calcul0n__ joined #minetest-dev
00:22 Zughy[m] joined #minetest-dev
00:22 Icedream joined #minetest-dev
00:25 LoneWolfHT joined #minetest-dev
00:26 Newbyte joined #minetest-dev
00:44 Taoki joined #minetest-dev
01:12 numzero joined #minetest-dev
01:12 jonadab joined #minetest-dev
01:12 jomat joined #minetest-dev
01:31 fluxflux_ joined #minetest-dev
01:36 flux__ joined #minetest-dev
01:36 Newbyte joined #minetest-dev
01:40 freshreplicant[m joined #minetest-dev
01:42 kb1000 joined #minetest-dev
01:43 Zughy[m] joined #minetest-dev
01:43 LoneWolfHT joined #minetest-dev
01:44 anon5[m] joined #minetest-dev
01:44 _Zaizen_[m] joined #minetest-dev
01:45 giov4[m] joined #minetest-dev
01:45 zughy[m]1 joined #minetest-dev
01:59 ShadowBot joined #minetest-dev
02:33 Zughy[m] joined #minetest-dev
02:33 giov4[m] joined #minetest-dev
02:34 Newbyte joined #minetest-dev
02:35 _Zaizen_[m] joined #minetest-dev
02:35 freshreplicant[m joined #minetest-dev
02:35 anon5[m] joined #minetest-dev
02:35 zughy[m]1 joined #minetest-dev
02:35 LoneWolfHT joined #minetest-dev
03:06 kb1000 joined #minetest-dev
03:27 absurb joined #minetest-dev
03:29 absurb_ joined #minetest-dev
03:31 Krock_ joined #minetest-dev
03:31 Sokomine_ joined #minetest-dev
03:39 absurb2 joined #minetest-dev
03:43 Icedream joined #minetest-dev
03:43 MTDiscord joined #minetest-dev
03:44 _Zaizen_[m] joined #minetest-dev
03:44 anon5[m] joined #minetest-dev
03:44 olliy joined #minetest-dev
03:47 Newbyte joined #minetest-dev
05:00 MTDiscord joined #minetest-dev
05:00 jonadab joined #minetest-dev
05:43 ShadowNinja joined #minetest-dev
05:44 systwi joined #minetest-dev
06:57 search_social joined #minetest-dev
07:11 MTDiscord joined #minetest-dev
07:15 nerzhul joined #minetest-dev
07:17 JTE joined #minetest-dev
08:00 ShadowNinja joined #minetest-dev
08:33 appguru joined #minetest-dev
08:51 ShadowBot joined #minetest-dev
09:17 Icedream joined #minetest-dev
09:23 Icedream joined #minetest-dev
09:51 proller joined #minetest-dev
10:47 sfan5 man all that character conversion shit is a mess
10:48 sfan5 where even is the difference between narrow_to_wide and utf8_to_wide?
10:48 calcul0n__ joined #minetest-dev
10:50 rubenwardy oof
10:51 rubenwardy imo it should be   "utf8_to_utf16" to be precise
10:51 rubenwardy assuming wide is utf16
10:51 sfan5 wchar_t is 4 bytes on linux
10:52 rubenwardy what why
10:53 rubenwardy you need to be able to predict the encodings of strings, having them be different on different platforms is a PITA
10:53 rubenwardy *having the same type be different
10:53 sfan5 now that you mention that I see the issue
10:53 sfan5 our network code assumes wchar_t is 2 bytes
10:57 sfan5 the only reason I can think of for narrow_to_wide/wide_to_narrow is interfacing with the filesystem on windows, but I don't see it used for that
11:00 sfan5 one way I see out of this mess is to turn utf-32 into utf-16 + surrogates when writing to network
11:00 celeron55 windows uses utf16 and linux uses utf32, right?
11:01 celeron55 and irrlicht uses wchar for some of its UI stuff, right?
11:01 sfan5 but I don't know if an old server wouldn't mangle those anyway
11:01 sfan5 yes, yes and no idea
11:02 sfan5 >//! Returns caption of this element.
11:02 sfan5 >virtual const wchar_t* getText() const
11:02 sfan5 looks like a yes
11:02 celeron55 i recall the only reason i had to deal with anything other than 8-bit character sets was that irrlicht required themm
11:02 celeron55 -m
11:03 celeron55 of course when dealing with filesystems it would be important to retain the exact original bytes instead of trying to translate them to utf-8 or any other charset, because filenames aren't necessary valid in any charset
11:03 celeron55 necessarily*
11:04 celeron55 but many serious programs fail at that, trying to convert them to utf-8
11:10 rubenwardy Yeah, we should use utf-8 everywhere and then convert to another format for specific APIs
11:10 rubenwardy or for manip
11:11 MTDiscord <F​lamore> ASCII
11:13 rubenwardy no
11:16 MTDiscord <F​lamore> :(
11:22 celeron55 yeah, always store as utf-8
11:23 celeron55 also, network serialization should always be utf-
11:23 celeron55 utf-8*
11:23 celeron55 the only exception is file names and paths, which should be stored raw and only converted for displaying
11:25 rubenwardy does the network use wide / utf-16 currently?
11:26 sfan5 yes, for chat messages and a few other things
11:27 celeron55 those are only because i didn't know better at the time
11:28 rubenwardy it shouldn't be too hard to add compat based on protocol versions for that
11:28 sfan5 ...and because nobody looked into this pre-5.0 when we had all chance to change the protocol
11:28 rubenwardy yeah :'(
11:28 rubenwardy > the only exception is file names and paths, which should be stored raw and only converted for displaying
11:28 rubenwardy What about paths stored in config files?
11:29 sfan5 really someone should have looked at every serialized packet at the time and looked which changes should be done
11:29 rubenwardy I think the same rule should apply - store in utf8, convert for APIs.   Where store means in common classes, rather than platform-specific local variables
11:29 sfan5 (another thing that's been annoying me is serializing ten or more u8's when a flag field would be enough)
11:29 rubenwardy > really someone should have looked at every serialized packet at the time and looked which changes should be done
11:29 rubenwardy Yeah, this was a really massive oversight. 5.0 was poorly planned
11:31 rubenwardy I don't think we should ever break network compat again, but instead remove backwards compat after a few releases and raise minimum version. Obviously, this doesn't help for things that are very hard to fix and keep compat
11:35 rubenwardy Something that's been annoying me is the fact that player attachments positions don't take BS into account - they're 10x larger than expected
11:50 Fixer joined #minetest-dev
12:15 sfan5 technically wide_to_narrow can be useful to support system locales that aren't utf-8, but I don't think that worked before either
12:16 sfan5 so nobody will miss it if that disappears
12:29 sfan5 oh and wide strings do make sense when you know you're working on unicode string, stuff like isspace() won't work on utf-8
12:33 calcul0n_ joined #minetest-dev
12:34 MTDiscord <e​xe_virus> Wait, never break network compat again? That's not going to fly
12:34 MTDiscord <e​xe_virus> Oh, you mean just have a sliding window of support
12:34 MTDiscord <e​xe_virus> That makes more sense
12:36 MTDiscord <e​xe_virus> So if we change how a block is sent, we might have three functions in play max, and select the right one based on reported client version
12:37 MTDiscord <e​xe_virus> For new stuff, like a "play midi" packet, we would just not send it to clients that don't know what to do with it
12:37 sfan5 a sliding window of support is what we had before
12:37 sfan5 with the exception that 5.0 was a planned protocol break
12:38 MTDiscord <e​xe_virus> Gotcha, I've only been around since 4.17 and only learned about the engine after 5.0
12:39 MTDiscord <e​xe_virus> We still have a lot of work ahead, haha
12:46 Beton joined #minetest-dev
13:01 Ritchie joined #minetest-dev
13:01 Thomas-S_ joined #minetest-dev
13:01 freshreplicant[m joined #minetest-dev
13:03 T4im joined #minetest-dev
13:03 BakerPrime_ joined #minetest-dev
13:32 systwi_ joined #minetest-dev
13:32 LoneWolfHT joined #minetest-dev
13:34 Zughy[m] joined #minetest-dev
13:37 _Zaizen_[m] joined #minetest-dev
13:42 freshreplicant[m joined #minetest-dev
14:16 Zughy[m]1 joined #minetest-dev
14:27 olliy joined #minetest-dev
14:43 proller joined #minetest-dev
14:59 anon5[m] joined #minetest-dev
14:59 zughy[m] joined #minetest-dev
14:59 LoneWolfHT joined #minetest-dev
14:59 kb1000 joined #minetest-dev
14:59 freshreplicant[m joined #minetest-dev
14:59 Newbyte joined #minetest-dev
14:59 _Zaizen_[m] joined #minetest-dev
14:59 giov4[m] joined #minetest-dev
15:03 sfan5 just wasted half an hour because Server::handleCommand_ChatMessage does its own serialization instead of using the function provided by NetworkPacket
15:03 sfan5 wtf
15:11 sfan5 #10879
15:11 ShadowBot https://github.com/minetest/minetest/issues/10879 -- [NO SQUASH] Fix narrow/wide and UTF-16 mess by sfan5
15:40 olliy joined #minetest-dev
15:53 BakerPrime_ joined #minetest-dev
16:22 sfan5 merging #10859, #10819, #10799, #10661 in 10m
16:22 ShadowBot https://github.com/minetest/minetest/issues/10859 -- Handle changes caused by CMake minimum version bump by sfan5
16:22 ShadowBot https://github.com/minetest/minetest/issues/10819 -- [DONT SQUASH] Rework use_texture_alpha by sfan5
16:22 ShadowBot https://github.com/minetest/minetest/issues/10799 -- Clarify key_value_swap's edge case by Grissess
16:22 ShadowBot https://github.com/minetest/minetest/issues/10661 -- [MANUAL SQUASH] Settings: Proper priority hierarchy by SmallJoker
16:24 Krock_ oh boi
16:29 rubenwardy very nice
16:45 numzero joined #minetest-dev
16:46 sfan5 zsh: segmentation fault (core dumped)  ./bin/minetest
16:46 sfan5 now this isn't a good sign
16:47 sfan5 0x00005555556e5547 in PlayerDatabaseFiles::serialize(RemotePlayer*, std::ostream&) ()
16:47 sfan5 ^ Krock
16:47 Krock checking
16:52 Krock reproduced when exiting the world
16:54 sfan5 happening after ~10s in-game here
16:55 Krock on quit, PlayerSAO is NULL
16:55 Lunatrius joined #minetest-dev
16:55 Krock same after a few seconds in-game. interesting.
17:00 Krock okay, it's a typo
17:00 Krock -serialize(ss, player);
17:00 Krock + serialize(&testplayer, ss);
17:03 Krock tested, works. will push https://krock-works.uk.to/u/patches/0001-PlayerDatabaseFiles-Fix-segfault-while-saving-a-play.patch in 10 minutes
17:13 Krock pushing
17:13 Krock 9a177f009
17:15 flux__ joined #minetest-dev
17:21 flux__ joined #minetest-dev
17:56 sfan5 thanks
17:56 sofar joined #minetest-dev
19:31 sfan5 #10881
19:31 ShadowBot https://github.com/minetest/minetest/issues/10881 -- Set UTF-8 codepage in Windows manifest by sfan5
19:31 sfan5 thoughts?
19:37 MTDiscord <J​onathon> is it possible that https://github.com/minetest/minetest/pull/10731 could be merged before the feature freeze starting tomorrow?
19:39 Krock can't test that but it looks good
19:40 sfan5 I tested it
19:41 sfan5 @Jonathon I'll add it to the milestone, if we have consensus we can make an exception for it
19:41 MTDiscord <J​onathon> thank you
19:43 MTDiscord <J​onathon> also, one more (mtg this time) https://github.com/minetest/minetest_game/pull/2808 since addressed the old concerns, and probably will useful for servers
19:47 rubenwardy nice find   re: #10881
19:48 ShadowBot https://github.com/minetest/minetest/issues/10881 -- Set UTF-8 codepage in Windows manifest by sfan5
19:59 systwi joined #minetest-dev
20:20 olliy_ joined #minetest-dev
20:42 systwi_ joined #minetest-dev
21:24 flux__ joined #minetest-dev
21:29 flux__ joined #minetest-dev
21:36 flux__ joined #minetest-dev
21:36 sfan5 merging #10881 in 5m
21:38 ShadowBot https://github.com/minetest/minetest/issues/10881 -- Set UTF-8 codepage in Windows manifest by sfan5
21:57 flux__ joined #minetest-dev
23:45 Taoki joined #minetest-dev

| Channels | #minetest-dev index | Today | | Google Search | Plaintext