| Time |
Nick |
Message |
| 00:01 |
hecks |
it's not significant now as we're throwing out thousands of draws |
| 00:01 |
hecks |
but it may become significant in the future, it depends on your expected player count too |
| 00:20 |
|
specing_ joined #minetest-dev |
| 00:27 |
|
v-rob joined #minetest-dev |
| 00:42 |
hecks |
oh i think i found the mouse in the .x loader |
| 00:43 |
hecks |
there are two places where it may be looking for a material |
| 00:43 |
MTDiscord |
<Jordach> something was definitely infinite looping |
| 00:43 |
hecks |
i'm not sure about the infinite looping bit but |
| 01:07 |
hecks |
Jordach try again |
| 01:07 |
MTDiscord |
<Jonathon> @JorDACh ^ |
| 01:09 |
MTDiscord |
<Jordach> can do |
| 01:10 |
hecks |
i don't have high hopes because i don't actually know why it breaks yet |
| 01:11 |
hecks |
but at least it handles materials as unknown blocks now |
| 01:11 |
MTDiscord |
<Jordach> hecks: |
| 01:11 |
MTDiscord |
<Jordach> source/Irrlicht/CXMeshFileLoader.cpp:92:2: error: use of undeclared identifier 'MaterialSlotCount' |
| 01:11 |
hecks |
aaaaaaa |
| 01:11 |
MTDiscord |
<Jordach> clang will go clang |
| 01:12 |
hecks |
cleaned up |
| 01:12 |
hecks |
this is why keeping more than one class in a file is bad |
| 01:13 |
MTDiscord |
<Jordach> building |
| 01:13 |
MTDiscord |
<Jordach> should take about 1 minute for both CPU archs |
| 01:13 |
hecks |
both, huh |
| 01:13 |
MTDiscord |
<Jordach> arm64 and amd64 |
| 01:14 |
MTDiscord |
<Jordach> my M1 is ludicrously handy for compiling |
| 01:14 |
hecks |
does your machine emulate one of those? |
| 01:14 |
MTDiscord |
<Jordach> it comes with a coprocessor to turn x64 into arm64 |
| 01:14 |
hecks |
i'd assume amd64 is the emulated one |
| 01:14 |
hecks |
yup |
| 01:14 |
hecks |
some power processors do this too |
| 01:15 |
MTDiscord |
<Jordach> the emulation is pretty much 85% of a comparable Intel CPU |
| 01:15 |
MTDiscord |
<Jordach> it's that efficient |
| 01:15 |
hecks |
good news because i can't stay on this arch forever |
| 01:15 |
MTDiscord |
<Jordach> why do you think i took up Mac maintainer as a hobby |
| 01:16 |
hecks |
it's just that pc hardware has become kinda crap, i don't feel like upgrading |
| 01:16 |
MTDiscord |
<Jordach> mem leaks |
| 01:17 |
hecks |
again? |
| 01:17 |
MTDiscord |
<Jordach> game freezes to death and i hear mp3 stuttering |
| 01:17 |
hecks |
what would even cause it to do that |
| 01:18 |
hecks |
the first int is indeed a slot count |
| 01:18 |
MTDiscord |
<Jordach> well now it's not doing it |
| 01:18 |
MTDiscord |
<Jordach> wat |
| 01:19 |
hecks |
then it's an array of slots, equal to the face count |
| 01:19 |
MTDiscord |
<Jordach> 0 net.minetest.minetest 0x0000000100d21600 checkMeshNormals(irr::scene::IMesh*) + 96 |
| 01:19 |
MTDiscord |
<Jordach> segfaults there again |
| 01:20 |
hecks |
hang on |
| 01:20 |
hecks |
i need to solve this mystery about the file first |
| 01:20 |
MTDiscord |
<Jordach> well you can freely download it |
| 01:20 |
hecks |
i know, i'm going through it |
| 01:20 |
hecks |
shotgun has 271 material assignments |
| 01:21 |
MTDiscord |
<Jordach> wat |
| 01:21 |
hecks |
not materials |
| 01:21 |
MTDiscord |
<Jordach> original blender had what 6 materials? |
| 01:21 |
hecks |
but faces(?) that have materials |
| 01:21 |
hecks |
you have 5 mats |
| 01:21 |
hecks |
anyway spnkr has 245 of these things whatever they are |
| 01:21 |
hecks |
this is conveniently the one part of the .x spec i never bothered with... |
| 01:22 |
hecks |
i don't even use material assignments, i just use separate meshes myself |
| 01:22 |
MTDiscord |
<Jordach> that works too but that's one too many CAO for my liking |
| 01:22 |
hecks |
but look at that |
| 01:22 |
hecks |
spnkrx has a face count of 245 |
| 01:23 |
hecks |
ok shotgun does have 271 faces |
| 01:23 |
hecks |
so this array length is fine |
| 01:24 |
hecks |
so yeah it's 271 face mats and then |
| 01:24 |
hecks |
just material blocks |
| 01:24 |
hecks |
the material blocks are absolutely not needed for anything, we already know the slots |
| 01:24 |
MTDiscord |
<Jordach> basically assigning that face to a material |
| 01:24 |
hecks |
yeah they're assignments |
| 01:24 |
hecks |
we already know the slot count (it's the first number) so we know how many submeshes to generate |
| 01:25 |
hecks |
but oh |
| 01:26 |
hecks |
ok so irrlicht does split it into separate mesh buffers |
| 01:26 |
hecks |
mesh->Buffers.reallocate(mesh->MaterialSlotCount); does make sense, this is what we're supposed to do |
| 01:27 |
MTDiscord |
<Jordach> but for some reason recalculating normals seems to shit the bed |
| 01:27 |
hecks |
now what the hell is BETTER_MESHBUFFER_SPLITTING_FOR_X and do we have it on or off |
| 01:27 |
hecks |
there are two implementations of this in this thing |
| 01:27 |
hecks |
i might just wipe the slate clean and write my own if i can't figure it out |
| 01:27 |
MTDiscord |
<Jordach> well the stk-editor has it disabled |
| 01:27 |
MTDiscord |
<Jordach> https://github.com/supertuxkart/stk-editor/blob/master/lib/irrlicht/source/Irrlicht/CXMeshFileLoader.cpp#L22 |
| 01:28 |
hecks |
in what way is it "better" i wonder |
| 01:28 |
MTDiscord |
<Jordach> fuck knows |
| 01:28 |
hecks |
we do not define it either |
| 01:28 |
MTDiscord |
<Jordach> only thing we can try is test |
| 01:28 |
hecks |
/the same vertex can be used in many different meshbuffers, but it's slow to work out |
| 01:28 |
hecks |
oh |
| 01:28 |
hecks |
hmmmmm |
| 01:29 |
hecks |
now if i were writing this code from scratch, with a reasonable mesh system |
| 01:29 |
hecks |
the way this works would be that we have a shared VBO and several separate element buffers that index into it |
| 01:29 |
hecks |
and that might be what this code path is about |
| 01:38 |
hecks |
try again |
| 01:38 |
hecks |
Jordach |
| 01:39 |
hecks |
i swear if it's this line... |
| 01:39 |
MTDiscord |
<Jordach> it didn't memory leak this time i believe |
| 01:40 |
hecks |
does it still die? |
| 01:40 |
MTDiscord |
<Jordach> recompiling |
| 01:41 |
hecks |
oh right, you have to static link everything |
| 01:41 |
MTDiscord |
<Jordach> making just the arm64 version for speed |
| 01:41 |
hecks |
what i changed was, i initialized MeshBuffer.Material with a blank SMaterial() |
| 01:42 |
hecks |
maybe irrlicht is so "smart" that on top of storing the material with the mesh buffer (????) it also refuses to use calloc |
| 01:44 |
MTDiscord |
<Jordach> thar she blows |
| 01:44 |
MTDiscord |
<Jordach> https://cdn.discordapp.com/attachments/747163566800633906/870119452300161084/Screenshot_2021-07-29_at_02.43.46.png |
| 01:44 |
MTDiscord |
<Jordach> the rocket entity and particles are obj |
| 01:45 |
MTDiscord |
<Jordach> yet it just died right there |
| 01:45 |
hecks |
okay this is getting out of hand, i'll actually set up a test bed myself |
| 01:45 |
MTDiscord |
<Jordach> nope, another meshnormals crash |
| 01:46 |
MTDiscord |
<Jordach> also this was the screenshot as it OOM'd |
| 01:46 |
MTDiscord |
<Jordach> https://cdn.discordapp.com/attachments/747163566800633906/870120142535790702/Screenshot_2021-07-29_at_02.43.27.png |
| 01:48 |
hecks |
does that mean anything? |
| 01:48 |
MTDiscord |
<Jordach> things are rendering fine |
| 01:48 |
hecks |
i see the viewmodel being rendered |
| 01:48 |
MTDiscord |
<Jordach> yes |
| 01:48 |
hecks |
so it renders and just leaks memory now? |
| 01:48 |
MTDiscord |
<Jordach> animates fine too |
| 01:48 |
MTDiscord |
<Jordach> by leaks memory i'd gather ~30GB per second |
| 01:48 |
MTDiscord |
<Jordach> it took three seconds for me to move that far away from the launch position before it dies |
| 01:49 |
hecks |
does spnkrx cause this too? |
| 01:49 |
MTDiscord |
<Jordach> not that i'm sure of |
| 01:49 |
hecks |
but shotgun would? |
| 01:49 |
MTDiscord |
<Jordach> it seems normals might just be wank |
| 01:49 |
hecks |
what does this have to do with normals though |
| 01:49 |
MTDiscord |
<Jordach> seems to cause it less frequently |
| 01:49 |
hecks |
that's the mystery |
| 01:49 |
MTDiscord |
<Jordach> entities have face dependant shading |
| 01:49 |
hecks |
i just deleted the part where it was assigning default materials that we never used |
| 01:50 |
MTDiscord |
<Jordach> it shouldn't need to calculate |
| 01:50 |
MTDiscord |
<Jordach> but if the X reader is missing pieces |
| 01:50 |
hecks |
look at CXMeshFileLoader::parseDataObjectMaterial in the original and tell me if that looks hazardous to delete at all... |
| 01:51 |
MTDiscord |
<Jordach> can do, the minetest terminal reported nothing as it died |
| 01:51 |
MTDiscord |
<Jordach> as seen here, note that i've just exceeded virtual memory usage |
| 01:51 |
MTDiscord |
<Jordach> https://cdn.discordapp.com/attachments/747163566800633906/870121275333435422/Screenshot_2021-07-29_at_02.43.27_2.png |
| 01:52 |
hecks |
i sure love being datamined by discord even when using irc |
| 01:52 |
MTDiscord |
<Jordach> i'd doubt that |
| 01:52 |
MTDiscord |
<Jordach> who has the cpu horsepower to do that 24/7 at hundreds of thousands or higher messages per second |
| 01:53 |
hecks |
fine |
| 01:53 |
hecks |
it just scans *your* process list =] |
| 01:54 |
hecks |
now why would it be running out of memory or even allocating any while rendering, this is demented |
| 01:54 |
MTDiscord |
<Jordach> it doesn't happen on the build before you took the axe to irrlicht |
| 01:54 |
hecks |
i'm aware |
| 01:54 |
MTDiscord |
<Jordach> because that's dated 24th of june |
| 01:55 |
MTDiscord |
<Jordach> lemme go get the hot sauce to the file in question |
| 01:56 |
pgimeno |
valgrind? |
| 01:57 |
MTDiscord |
<Jordach> did the X loader ever interface with setMeshTextureLoader |
| 01:58 |
hecks |
it did |
| 01:58 |
hecks |
that's the part i axed |
| 01:58 |
hecks |
because i wanted to kill that loader |
| 02:00 |
MTDiscord |
<Jordach> is it possible that one of the loaders doesn't terminate their loop |
| 02:00 |
MTDiscord |
<Jordach> and get stuck |
| 02:01 |
hecks |
i just reverted the file and i'm basically doing it again but more carefully |
| 02:01 |
MTDiscord |
<Jordach> revert that file |
| 02:01 |
MTDiscord |
<Jordach> and push it |
| 02:01 |
MTDiscord |
<Jordach> i want to see if that changes anything |
| 02:01 |
hecks |
it's not possible to just use that version as it is because the texture loader doesn't exist |
| 02:01 |
hecks |
also i want to solve this because SMaterial will be gone sooner or later |
| 02:04 |
pgimeno |
if hecks can't reproduce it I'd say that a Mac GL bug should be given consideration |
| 02:04 |
hecks |
it isn't |
| 02:04 |
MTDiscord |
<Jordach> i believe it's getting hung up on parsing |
| 02:04 |
hecks |
and i have not tested it myself yet |
| 02:05 |
MTDiscord |
<Jordach> because a few seconds pass and then everything locks up |
| 02:05 |
MTDiscord |
<Jordach> followed by the out of memory window |
| 02:05 |
hecks |
it's just a stupid edge case probably thanks to the pants on head way irrlicht's mesh buffers are designed |
| 02:05 |
hecks |
who puts a material definition in the mesh asset class |
| 02:08 |
hecks |
once more |
| 02:09 |
hecks |
Jordach |
| 02:09 |
hecks |
it's making dummy materials again now, just not filling them |
| 02:09 |
hecks |
i'll have to figure this out sooner or later but maybe it will work for now |
| 02:10 |
MTDiscord |
<Jordach> building |
| 02:12 |
hecks |
i can set your viewmodels as the player model and it doesn't break |
| 02:12 |
MTDiscord |
<Jordach> they're attachments |
| 02:12 |
hecks |
that shouldn't matter |
| 02:12 |
MTDiscord |
<Jordach> chaching |
| 02:12 |
MTDiscord |
<Jordach> UNSUPPORTED (log once): POSSIBLE ISSUE: unit 0 GLD_TEXTURE_INDEX_2D is unloadable and bound to sampler type (Float) - using zero texture because texture unloadable |
| 02:13 |
MTDiscord |
<Jordach> this error message has shown before |
| 02:13 |
MTDiscord |
<Jordach> mem usage seems normal |
| 02:13 |
MTDiscord |
<Jordach> post recompile |
| 02:13 |
hecks |
do you mean you got this error normally? |
| 02:14 |
MTDiscord |
<Jordach> yes |
| 02:14 |
hecks |
before this |
| 02:14 |
MTDiscord |
<Jordach> only X models would do it |
| 02:14 |
hecks |
consider that irrelevant once i rewrite shit |
| 02:14 |
MTDiscord |
<Jordach> nope, whatever was in the X loaders is gone |
| 02:14 |
MTDiscord |
<Jordach> causing the OOM |
| 02:14 |
hecks |
it's fixed? |
| 02:15 |
MTDiscord |
<Jordach> not sure if the normal calc is fixed |
| 02:15 |
hecks |
oh actually |
| 02:15 |
MTDiscord |
<Jordach> everything works fine |
| 02:15 |
MTDiscord |
<Jordach> even joining, leaving and rejoining |
| 02:16 |
hecks |
i may have an idea now |
| 02:16 |
MTDiscord |
<Jordach> which causes the normalcalc segfault |
| 02:16 |
hecks |
this had nothing to do with normals |
| 02:16 |
hecks |
this was a material slot count thing |
| 02:16 |
hecks |
i think it isn't enough to trust the value in the face material list |
| 02:16 |
hecks |
do you have time to test this? |
| 02:16 |
MTDiscord |
<Jordach> ye |
| 02:16 |
hecks |
sure, let me try |
| 02:17 |
MTDiscord |
<Jordach> yeah, memory usage didn't budge even when changing model out on the fly |
| 02:17 |
MTDiscord |
<Jordach> nor did my CPU hit 100% on all cores |
| 02:19 |
MTDiscord |
<Jordach> lemme reload my system so i can clear some cruft from the kernel |
| 02:20 |
hecks |
try this |
| 02:21 |
MTDiscord |
<Jordach> ye |
| 02:21 |
hecks |
works? |
| 02:22 |
MTDiscord |
<Jordach> need to rebuild |
| 02:22 |
hecks |
ah |
| 02:22 |
MTDiscord |
<Jordach> i rebuild it all to prevent any cache |
| 02:22 |
hecks |
have you fetched it already at least? |
| 02:22 |
MTDiscord |
<Jordach> was using 6f20f54 |
| 02:22 |
hecks |
i want to push the next experiment |
| 02:22 |
MTDiscord |
<Jordach> now at de97c7a |
| 02:22 |
hecks |
okay |
| 02:24 |
hecks |
i pushed the next experiment but don't test it until you're done with this one |
| 02:26 |
MTDiscord |
<Jordach> sigsegv, 0 net.minetest.minetest 0x0000000100629474 checkMeshNormals(irr::scene::IMesh*) + 96 |
| 02:27 |
MTDiscord |
<Jordach> everytime, no matter what viewmodel |
| 02:27 |
hecks |
man, this is dumb |
| 02:28 |
MTDiscord |
<Jordach> push your test |
| 02:28 |
hecks |
second one? it's already pushed |
| 02:28 |
MTDiscord |
<Jordach> Updating de97c7a..88aba8e |
| 02:29 |
hecks |
although if the first one didn't work, this one won't either |
| 02:29 |
MTDiscord |
<Jordach> let's find out |
| 02:32 |
MTDiscord |
<Jordach> that was the OOM event |
| 02:32 |
MTDiscord |
<Jordach> because it just froze rather than segfault |
| 02:32 |
hecks |
ok can you open cxmeshfileloader.cpp |
| 02:32 |
MTDiscord |
<Jordach> ye |
| 02:33 |
hecks |
around line 130, there's mesh->Buffers.push_back( AnimatedMesh->addMeshBuffer() ); |
| 02:33 |
hecks |
wherever that is, insert this line right below |
| 02:33 |
hecks |
mesh->Buffers.getLast()->Material = video::SMaterial(); |
| 02:34 |
hecks |
I'm pretty sure we tried this before but do it anyway |
| 02:34 |
hecks |
and build |
| 02:34 |
MTDiscord |
<Jordach> rebuilding |
| 02:36 |
hecks |
and if that is still broken, try my last commit |
| 02:37 |
hecks |
i'm basically just checking this matrix of conditions now |
| 02:37 |
MTDiscord |
<Jordach> adding that line sigsegv with 0 net.minetest.minetest 0x00000001043c14a0 checkMeshNormals(irr::scene::IMesh*) + 96 ** |
| 02:37 |
hecks |
read the material count vs count it manually, generate materials now vs do it later |
| 02:37 |
hecks |
ok, fetch and build again |
| 02:38 |
hecks |
i'll be shocked if it doesn't work now |
| 02:38 |
hecks |
because all this changes is that instead of creating a list of blank materials and counting it, it just increments a counter and makes them up on the spot |
| 02:39 |
MTDiscord |
<Jordach> making irr takes about 30 seconds |
| 02:39 |
MTDiscord |
<Jordach> reconstructing |
| 02:40 |
MTDiscord |
<Jordach> had to uh merge conflict oof |
| 02:40 |
MTDiscord |
<Jordach> might redownload the branch |
| 02:40 |
MTDiscord |
<Jordach> just in case |
| 02:40 |
hecks |
i reverted some |
| 02:40 |
hecks |
and force pushed |
| 02:40 |
MTDiscord |
<Jordach> fair |
| 02:40 |
hecks |
just overwrite whatever you've got |
| 02:40 |
MTDiscord |
<Jordach> just did that |
| 02:41 |
MTDiscord |
<Jordach> git restore ... |
| 02:41 |
MTDiscord |
<Jordach> OOM'd |
| 02:41 |
MTDiscord |
<Jordach> just watched it munch 2GB |
| 02:42 |
hecks |
explain to me the logic here |
| 02:42 |
hecks |
have an std::vector (sort of) of structs |
| 02:42 |
hecks |
push back blank struct N times |
| 02:43 |
hecks |
use .size() as iteration limit and retrieve the blank structs |
| 02:43 |
hecks |
w |
| 02:43 |
hecks |
works |
| 02:43 |
hecks |
have a counter |
| 02:43 |
hecks |
increment the counter when you'd be pushing back |
| 02:43 |
hecks |
use the counter as the iteration limit |
| 02:43 |
hecks |
spawn the blank struct inline |
| 02:43 |
hecks |
doesn't work |
| 02:44 |
MTDiscord |
<Jordach> the only thing coming to mind is new CMTL |
| 02:45 |
hecks |
just uhh these aren't real STL types irrlicht is using |
| 02:45 |
hecks |
it's rolling its own :DDDD |
| 02:46 |
hecks |
okay now fetch and test one last time to verify i reverted to the right commit |
| 02:46 |
hecks |
and we'll leave it at that |
| 02:50 |
hecks |
i checked the b3d loader real quick too and luckily i did not touch the materials too much there |
| 02:50 |
MTDiscord |
<Jordach> there it is again |
| 02:50 |
MTDiscord |
<Jordach> UNSUPPORTED (log once): POSSIBLE ISSUE: unit 0 GLD_TEXTURE_INDEX_2D is unloadable and bound to sampler type (Float) - using zero texture because texture unloadable |
| 02:50 |
hecks |
yeah yeah |
| 02:50 |
hecks |
does it actually work? |
| 02:51 |
MTDiscord |
<Jordach> works fine |
| 02:51 |
MTDiscord |
<Jordach> even the offending shotgun |
| 02:51 |
MTDiscord |
<Jordach> even joining and leaving |
| 02:52 |
MTDiscord |
<Jordach> this might have something to do with it |
| 02:52 |
MTDiscord |
<Jordach> 2021-07-29 03:51:34: WARNING[Main]: Irrlicht: Could not create ITexture, texture needs to have a non-empty name. |
| 02:52 |
hecks |
it's the blank material |
| 02:52 |
hecks |
it doesn't matter because this is the material you're not actually using |
| 02:52 |
MTDiscord |
<Jordach> why does it segfault? |
| 02:53 |
MTDiscord |
<Jordach> spaghetti |
| 02:53 |
hecks |
it just did? |
| 02:53 |
MTDiscord |
<Jordach> i'm referring to early |
| 02:53 |
MTDiscord |
<Jordach> earlier |
| 02:53 |
MTDiscord |
<Jordach> why does a seemingly innocuous material cause OOM and segfaults |
| 02:53 |
hecks |
i actually have no idea, malformed MeshBuffer i guess |
| 02:53 |
MTDiscord |
<Jordach> actually |
| 02:53 |
hecks |
i'll say it for the third time, who puts materials in a mesh buffer type |
| 02:53 |
MTDiscord |
<Jordach> it might be using the material |
| 02:54 |
hecks |
in either case |
| 02:54 |
MTDiscord |
<Jordach> doing some weird memory manipulation |
| 02:54 |
hecks |
i was giving it a blank one |
| 02:54 |
MTDiscord |
<Jordach> then the fuck is it doing |
| 02:54 |
hecks |
i don't know dude, this is deep gremlins |
| 02:55 |
hecks |
anyway as soon as the client environment actually creates the stupid scene node |
| 02:55 |
hecks |
it sets new materials |
| 02:55 |
hecks |
so the ones generated on import are actually just trash |
| 02:56 |
hecks |
so as long as this doesn't crash, it's fine |
| 02:56 |
hecks |
it was either generating the mesh buffers wrong somehow due to something i missed |
| 02:56 |
hecks |
or using uninitialized memory somewhere that i missed |
| 02:56 |
hecks |
irrlicht has its own container types and allocators because it's "clever" like that |
| 02:56 |
hecks |
so you never know |
| 02:57 |
hecks |
the latest fetch works right? |
| 02:58 |
hecks |
merging irr#50 in five if it does |
| 02:58 |
ShadowBot |
https://github.com/minetest/irrlicht/issues/50 -- Fix .x material slot parsing by hecktest |
| 03:01 |
MTDiscord |
<Jordach> one moment |
| 03:01 |
MTDiscord |
<Jordach> oh that was the commit |
| 03:01 |
hecks |
6f20f54 |
| 03:01 |
MTDiscord |
<Jordach> yeah i'm on that one |
| 03:01 |
hecks |
good to merge? |
| 03:02 |
MTDiscord |
<Jordach> one sec |
| 03:02 |
MTDiscord |
<Jordach> rapidly constructing package |
| 03:03 |
hecks |
sorry it's just 5am and i'm retarded and i want to merge this already |
| 03:03 |
MTDiscord |
<Jordach> it's alright, it's late for me too |
| 03:03 |
MTDiscord |
<Jordach> even if it's 4am |
| 03:04 |
MTDiscord |
<Jordach> verified |
| 03:04 |
hecks |
cool |
| 03:04 |
MTDiscord |
<Jordach> it works fine including multiple rejoins |
| 03:04 |
hecks |
also merging #11485 since sfan said it's fine |
| 03:04 |
ShadowBot |
https://github.com/minetest/minetest/issues/11485 -- Add a simple PNG image encoder with Lua API by hecktest |
| 03:05 |
MTDiscord |
<Jordach> i'll get around to updating mac builds shortly |
| 03:07 |
MTDiscord |
<Jordach> it's a great feeling to have a machine that can rebuild both MT and Irr within 90 seconds |
| 03:09 |
hecks |
it would be a great feeling if c++ did not need that much in the first place |
| 03:09 |
MTDiscord |
<Jordach> that too |
| 03:09 |
MTDiscord |
<Jordach> but yes i'm able to test wildly unstable stuff |
| 03:09 |
MTDiscord |
<Jordach> just ping me when you want it |
| 03:11 |
hecks |
i link dynamically and don't make clean so i can rebuild things quickly enough |
| 03:11 |
hecks |
i just needed you for this in particular because i don't have your game installed |
| 03:11 |
MTDiscord |
<Jordach> ah, there is a repo for it |
| 03:11 |
hecks |
and because going out of memory might be more disastrous on my machine in particular |
| 03:12 |
MTDiscord |
<Jordach> you run windows, yeah that'd probably commit the bsod |
| 03:12 |
hecks |
i'm one of those lunatics who don't use a paging file |
| 03:12 |
MTDiscord |
<Jordach> i probably just burnt about 300GB's worth of writes on my NVMe |
| 03:12 |
hecks |
it's a coin flip whether or not going out of ram actually requires a reset or not |
| 03:12 |
MTDiscord |
<Jordach> because of the OOM writing |
| 03:13 |
hecks |
going swapless is pretty snappy but when you're out, you're out |
| 03:13 |
hecks |
but i actually prefer this to everything just freezing |
| 03:13 |
MTDiscord |
<Jordach> yeah until a certain point my mac won't even turn on swap |
| 03:14 |
MTDiscord |
<Jordach> should you want to debug my game further https://github.com/Dumpster-Studios/BattleForArkhos/ |
| 03:15 |
hecks |
i'll keep it bookmarked |
| 03:15 |
hecks |
it's probably a decent stress test |
| 03:15 |
MTDiscord |
<Jordach> to the point of exploding the network handler |
| 03:16 |
hecks |
not sure if i asked this before but |
| 03:17 |
hecks |
does #10699 help at al? |
| 03:17 |
ShadowBot |
https://github.com/minetest/minetest/issues/10699 -- Break up the message queue by category by hecktest |
| 03:17 |
hecks |
at all |
| 03:17 |
MTDiscord |
<Jordach> we can test that later |
| 03:17 |
MTDiscord |
<Jordach> i'd have to build it on the arch machine downstairs first |
| 03:17 |
MTDiscord |
<Jordach> and then watch netstat |
| 03:18 |
hecks |
it won't send less packets |
| 03:18 |
hecks |
it'll stall less |
| 03:18 |
MTDiscord |
<Jordach> well, you know what i mean |
| 03:18 |
hecks |
reason i made it was that |
| 03:18 |
MTDiscord |
<Jordach> when it got blocked up packets weren't being sent freely, rather than in batches |
| 03:19 |
hecks |
i was trying to test mibi on that crappy connection we played on back there and it was possible for the packet queue to just clog up |
| 03:19 |
hecks |
the server did not have enough upload to catch up with the packets produced |
| 03:19 |
hecks |
you'd be seeing your character do actions you did 30 seconds ago |
| 03:20 |
hecks |
MT currently doesn't have any mechanism to flush a clogged queue |
| 03:20 |
MTDiscord |
<Jordach> yep had that before with hud updates |
| 03:20 |
hecks |
no way to recover from this |
| 03:20 |
MTDiscord |
<Jordach> hud is another known clogger |
| 03:20 |
hecks |
so on one end i did reduce the bandwidth i use but on another i made that pr |
| 03:20 |
hecks |
to at least reduce the chances because we only have 3 queues, one of them hogging most of the traffic |
| 03:21 |
MTDiscord |
<Jordach> bandwidth isn't a concern with VPS hosts with symmetrical gigabit |
| 03:21 |
hecks |
i will be hosting that thing in a proper datacenter eventually |
| 03:21 |
hecks |
but even a couple megabits should serve at least a few people |
| 03:21 |
MTDiscord |
<Jordach> yeah |
| 03:22 |
hecks |
online games are usually not too heavy |
| 03:22 |
MTDiscord |
<Jordach> my game tries extremely hard from Lua side from sending packets |
| 03:22 |
MTDiscord |
<Jordach> even making decimal places tolerant |
| 03:22 |
hecks |
we could probably reduce a lot by being smarter about object properties |
| 03:22 |
hecks |
just sending a diff of those |
| 03:22 |
MTDiscord |
<Jordach> oh good god |
| 03:22 |
hecks |
that's your bottleneck right? |
| 03:22 |
MTDiscord |
<Jordach> that would make my game less horrifying |
| 03:22 |
hecks |
it doesn't sound hard |
| 03:23 |
MTDiscord |
<Jordach> no it doesn't |
| 03:23 |
MTDiscord |
<Jordach> might be a worthwhile experiment |
| 03:27 |
hecks |
rendering comes first |
| 03:31 |
hecks |
speaking of which, another day wasted |
| 03:32 |
hecks |
i sure hope for no more changes to the build system... i'd like to actually do things |
| 03:40 |
MTDiscord |
<Jordach> i'd rather have pages documenting everything right down to caveats than have a few words that don't explain what it does |
| 05:02 |
MTDiscord |
<exe_virus> What's the use case for the png encoder? Just procedural png generation? Why bake it into minetest? |
| 05:03 |
MTDiscord |
<exe_virus> I guess you could procedurally generate a daily map texture or something for the server. Can't think of much else you need to modify daily or on restart |
| 05:21 |
MTDiscord |
<GreenXenith> Dynamic textures |
| 05:21 |
MTDiscord |
<GreenXenith> player-made textures, custom/dynamic skins |
| 05:21 |
MTDiscord |
<GreenXenith> There are uses |
| 05:36 |
MTDiscord |
<Jordach> Using minetests built in noise sources you can generate the entire map on all 62km sides |
| 05:37 |
MTDiscord |
<Jordach> And have it as a portable map |
| 05:37 |
MTDiscord |
<Jordach> And generate various zoom levels |
| 05:37 |
MTDiscord |
<GreenXenith> Yeah, but good luck reading it later for use |
| 06:38 |
|
Extex joined #minetest-dev |
| 07:01 |
|
olliy joined #minetest-dev |
| 08:40 |
|
entuland joined #minetest-dev |
| 10:15 |
|
calcul0n joined #minetest-dev |
| 10:39 |
|
freshreplicant[m joined #minetest-dev |
| 10:48 |
|
LW joined #minetest-dev |
| 10:48 |
|
wsor4035 joined #minetest-dev |
| 11:31 |
MTDiscord |
<exe_virus> That's for noise. Not for actual map. Basically at startup it takes a solid 20-60 seconds and calculates the top node in each x-z location (like minimap) and places that into a texture |
| 11:32 |
MTDiscord |
<exe_virus> Could obviously limit the area rendered. |
| 12:04 |
|
olliy1or joined #minetest-dev |
| 12:21 |
|
specing_ joined #minetest-dev |
| 12:44 |
|
calcul0n_ joined #minetest-dev |
| 14:35 |
|
Fixer joined #minetest-dev |
| 15:39 |
|
Extex joined #minetest-dev |
| 16:19 |
|
hecks joined #minetest-dev |
| 16:25 |
hecks |
exe_virus: the use case for the png encoder is that some lost soul was writing TGA files in his mod and we just got rid of that format (it was never actually supported, worked by accident) |
| 16:26 |
MTDiscord |
<Warr1024> We support dynamic media so it seems reasonable to support at least one tool to create it properly. Makes sense to me. |
| 16:26 |
hecks |
it is useful |
| 16:26 |
hecks |
within limits though |
| 16:26 |
hecks |
this only helps if you can pre-generate data |
| 16:26 |
hecks |
and it's always better to use a texture modifier if it can handle what you want |
| 16:28 |
MTDiscord |
<Warr1024> If only we had a dynamic_remove_media that let us reclaim the texture memory :-D |
| 16:28 |
hecks |
i should finish that ^[raw: thing, it would complete the feature set by allowing embedded images in texstrings |
| 16:28 |
hecks |
dynamic media is problematic in general and i don't like it |
| 16:28 |
hecks |
the fact that the media is propagated to all clients makes it barely useful |
| 16:29 |
MTDiscord |
<Warr1024> I originally liked the idea of a ^[png: that used base64-encoded png files, with the idea of using external encoders to optimize the data. Texturemod strings have a pretty tight size limit, and the zlib in PNG would help compensate for the encoding expansion. |
| 16:29 |
hecks |
my ^[raw prototype is almost png but |
| 16:29 |
hecks |
a little leaner |
| 16:29 |
MTDiscord |
<Warr1024> Not being able to free up memory from textures when they're no longer used is a serious limitation for texturemods though |
| 16:29 |
hecks |
there's no need to conform to the png spec but it is similar in spirit |
| 16:30 |
hecks |
you know, in general we do not gar |
| 16:30 |
hecks |
b |
| 16:30 |
MTDiscord |
<Warr1024> If you did zlibbed raw RGBA32 then that'd be PNG enough for me. |
| 16:30 |
hecks |
stupid keyboard, we do not garbage collect texmods and that's bad |
| 16:30 |
hecks |
i actually thought ^[raw should only use palettes because of size concerns |
| 16:31 |
hecks |
so it was zlibbed base64 palette and pixels blob |
| 16:31 |
MTDiscord |
<Warr1024> I would imagine that zlib would compensate a lot for that. When I automatically optimize real-world png images, it's a crap shoot whether it ends up indexed vs RGBA |
| 16:31 |
hecks |
palettes always help |
| 16:31 |
MTDiscord |
<Warr1024> Yeah, except when they don't. |
| 16:31 |
MTDiscord |
<Warr1024> Having to generate an optimized palette is probably going to add a lot of complexity |
| 16:32 |
MTDiscord |
<Warr1024> I guess if we provide tools for it in the engine then it doesn't matter too much exactly what we use |
| 16:32 |
hecks |
you know the real reason your textures are fat? not using ^[sheet |
| 16:32 |
hecks |
that's the thing that helps zlib the most |
| 16:32 |
hecks |
only for actual files of course |
| 16:32 |
MTDiscord |
<Warr1024> That only affects the media transfer which is a very tiny part of the problem for me. I'm much more concerned with in-memory growth |
| 16:32 |
hecks |
i was thinking that the user should explicitly specify the palette |
| 16:33 |
hecks |
and build their images with palettes in mind, basically enforcing that ^[raw should not be used for heavy duty work |
| 16:33 |
MTDiscord |
<Warr1024> Sounds like I would need ^[rawer: then. I'm looking more for cases like user-provided media, like player skins |
| 16:33 |
hecks |
we need to start actually collecting unused textures on the client |
| 16:33 |
hecks |
are you telling me you can't make a good skin in 256 colors |
| 16:33 |
MTDiscord |
<Warr1024> If we start GCing unused textures then that opens up a massive world of possibility, I think. |
| 16:33 |
hecks |
you can also combine raws |
| 16:34 |
MTDiscord |
<Warr1024> I never said I couldn't make a good skin in 256 colors |
| 16:34 |
MTDiscord |
<Warr1024> I said I wasn't going to be making the skins |
| 16:34 |
hecks |
=] well |
| 16:34 |
hecks |
user uploaded skins are cancer |
| 16:34 |
MTDiscord |
<Warr1024> Yeah, too bad we're game engine devs and not oncologists |
| 16:35 |
MTDiscord |
<Warr1024> I mean shit, I've got an official TODON'T list on my game |
| 16:35 |
hecks |
allow users to upload skins -> end up with a server full of crappy copyrighted characters |
| 16:38 |
MTDiscord |
<Warr1024> I'm not here to deal with should and shouldn't decisions, those are the responsibility of downstream users. I'm here to deal with can and can't. |
| 16:38 |
MTDiscord |
<Warr1024> Limiting an image to 256-color indexed is too arbitrary for me to justify. |
| 16:39 |
hecks |
sure whatever, 16 24 and 32 bits wouldn't be hard |
| 16:39 |
hecks |
i just don't think it's the right use case for the feature |
| 16:40 |
hecks |
maybe embedding a png really is simpler |
| 16:40 |
MTDiscord |
<Warr1024> It's frankly a little iffy whether it makes more sense to add another texturemod vs. improve the way existing texturemods work. You can already transmit an arbitrary RGBA image with enough tmod fuckery, but there are serious limits. |
| 16:40 |
hecks |
but we would need to actually link libpng on the server |
| 16:41 |
MTDiscord |
<Warr1024> For one thing, I don't think texture names are compressed, so the expansion would be really painful, plus there's a pretty tight total size limit |
| 16:41 |
hecks |
well okay, as a stop gap we could just support what encode_png does |
| 16:41 |
MTDiscord |
<Warr1024> yeah, ^[png: would actually be easiest given how much we can reuse for it, and it gets us all the universality AND optimization we could want. |
| 16:41 |
hecks |
sure we should start compressing texstrings |
| 16:41 |
hecks |
but |
| 16:42 |
hecks |
no compression algorithm will beat base64 pixel data, don't go that far |
| 16:42 |
MTDiscord |
<Warr1024> compressing text strings could be nice. |
| 16:42 |
hecks |
we do need ^[png |
| 16:42 |
hecks |
actually the amount of ^ operators in a texmod really matters |
| 16:42 |
hecks |
each one is a new image allocation |
| 16:42 |
MTDiscord |
<Warr1024> I think the unused texture GC might have the biggest impact, since it unfucks existing capabilities we already have in theory but can't use due to memory leakage. Adding ^[png: would be probably a second, and then optimizing string sends maybe 3rd. |
| 16:43 |
hecks |
and i guess with encode_png and the existing base64 helper it should be easy enough |
| 16:43 |
hecks |
not GCing textures is a separate concern |
| 16:43 |
MTDiscord |
<Jordach> The problem with modifiers is that there’s no real way to do it otherwise |
| 16:43 |
MTDiscord |
<Warr1024> I looked into compressing every network packet using LZO/LZ4 at one point, and got ... okay results. Maybe like 2% to 5% reduction. Most of the bulk data seems to be mapblock sends which are already compressed. However, if we start opening up more texturemod magic, it could make a difference in the long run. |
| 16:44 |
hecks |
basically we need ^[png to help all those people who compose text or straight up individual pixels to make up for the lack of it |
| 16:44 |
MTDiscord |
<Jordach> You know the real question is why haven’t we just simply started to save texture modifiers to disk |
| 16:44 |
hecks |
^[string could maybe help too |
| 16:44 |
hecks |
^[string:font.png;"example text" |
| 16:45 |
MTDiscord |
<Warr1024> Oh, rendering freetype into a texture would be awesome. Current hacks require use of bitmap fonts which really screw over i18n. Since texturemods are computed client-side too, we could do the translations in a pass before the text rendering :-D |
| 16:45 |
hecks |
where the font path is a mono ascii font or perhaps |
| 16:45 |
hecks |
^[string:(path);(string);(kerningtable) |
| 16:46 |
MTDiscord |
<Jordach> Wouldn’t it be easier to implement vector graphics |
| 16:46 |
hecks |
though embedding a kerning table in the texstring is still hard |
| 16:46 |
MTDiscord |
<Jordach> Bring your own path |
| 16:46 |
hecks |
no support for server sent fonts yet... |
| 16:47 |
hecks |
making a texmod result depend on client settings = epic fail |
| 16:47 |
hecks |
a bitmap font is basically easier to understand and implement |
| 16:47 |
hecks |
and certainly faster to evaluate |
| 16:47 |
hecks |
this is also an important concern with texmods because complicated texmods cauase pauses and stutter |
| 16:48 |
hecks |
it would really not be hard to patch ^[string with ttf support if we can ever send those |
| 16:49 |
MTDiscord |
<Warr1024> I wouldn't bother implementing [string if it's just going to be a shortcut for things that people can already do with a library function. |
| 16:49 |
hecks |
no, look |
| 16:50 |
hecks |
the amount of ^ operators in a texstring matters |
| 16:50 |
hecks |
signs are causing gigantic lag |
| 16:50 |
MTDiscord |
<Warr1024> Freetype would be justification for it. If we just support a bitmap font, I'd tell people to just use [combine and move onto something else that they can't work around for now. |
| 16:50 |
hecks |
^[string should exist because ^[combine is slow |
| 16:50 |
hecks |
^[combine makes the texmod like 30x as long and take possibly seconds to render |
| 16:51 |
MTDiscord |
<Warr1024> We should fix the things that are most broken first. [png is more broken than [combine. |
| 16:51 |
hecks |
sure, you can always render text to ^[png on your end and that's pretty robust |
| 16:51 |
MTDiscord |
<Warr1024> and frankly lack of GC is more broken than any of these because any [png or [combine or [string that we implement is going to make lack-of-GC problems we already have worse |
| 16:52 |
hecks |
also, something flew over my head |
| 16:52 |
hecks |
client side ^[string translations? dude, that'll explode |
| 16:52 |
MTDiscord |
<Warr1024> Actually [png serves the string needs for large servers with relatively competent admins, it's only really needed for SP games and such... |
| 16:52 |
MTDiscord |
<Warr1024> texturemods are generally rendered client-side, so translations are basically a freebie |
| 16:52 |
hecks |
you don't understand |
| 16:52 |
hecks |
many languages have homonym words |
| 16:53 |
MTDiscord |
<Warr1024> that would probably be because you're not explaining it to me |
| 16:53 |
MTDiscord |
<Warr1024> yes, languages are complicated |
| 16:53 |
hecks |
not to mention that universal translation like that is generally a sisyphean task if not straight up lunacy |
| 16:54 |
hecks |
there's like... grammar and stuff |
| 16:54 |
MTDiscord |
<Warr1024> You DO know that translation is a thing in minetest already, don't you? |
| 16:54 |
hecks |
yes i know |
| 16:54 |
MTDiscord |
<Warr1024> It exists only client-side and only works for things available as text when they're processed on the client side |
| 16:54 |
hecks |
and i think it's a quixotic effort |
| 16:54 |
MTDiscord |
<Warr1024> if we sent texture modifiers as [string: then that would qualify them |
| 16:54 |
hecks |
and then you basically encourage people who don't speak a server's language to play said server |
| 16:54 |
hecks |
sure it would just be one of the many ways the translation system can blow things up |
| 16:55 |
hecks |
minetest can be too ambitious sometimes... |
| 16:55 |
MTDiscord |
<Warr1024> Sure, you have negative opinions about a lot of things. That doesn't mean that we shouldn't support them. |
| 16:55 |
hecks |
=] |
| 16:56 |
hecks |
no_translate flag in ^[string ok? |
| 16:56 |
MTDiscord |
<Warr1024> No, "no translate" is implied |
| 16:56 |
hecks |
last arg, optional |
| 16:56 |
MTDiscord |
<Warr1024> you have to actually explicitly add escaping things to strings to tell the translator they're translatable at all |
| 16:56 |
hecks |
oh that's even dumber actually |
| 16:56 |
MTDiscord |
<Warr1024> all I'd do is pass the string through the existing translation system for mod content that already exists, and it'd pick those up |
| 16:56 |
hecks |
but at least it doesn't invade everything and explode so that's good |
| 16:56 |
MTDiscord |
<Warr1024> yes, arguably the way translations are done in MT mods is pretty shitty |
| 16:57 |
hecks |
it's like |
| 16:57 |
MTDiscord |
<Warr1024> but it could have been worse :-) |
| 16:57 |
hecks |
basically the only right way to make languages work in your game is to do it yourself server side and know said languages yourself so that you can place your %s correctly in every case in your stringer code |
| 16:58 |
MTDiscord |
<Warr1024> Ironically the engine supports translations that have an ID/context, an original (English) string, and then translated strings. The mod content only maps original->translated, so they're making the assumption that a string of characters in English only has one way to translate into another langauge, which is not true when the string may be used in multiple different contexts. |
| 16:59 |
MTDiscord |
<Warr1024> But it's also possible to make games that have very little text, making the translation tools we have good enough to pass. |
| 16:59 |
hecks |
translating "Red Wool" and "Mese Ore" to french is not the same as translating a template like "%s %s %s%s!" which can produce anything depending on the situation |
| 16:59 |
hecks |
so it only works for really simple content =] |
| 16:59 |
MTDiscord |
<Warr1024> It seems like MOST players can at least muddle through in English, but it's nice to allow those who can to do some translation work to help support those who can't. The idea of supporting users who don't speak the language of my server doesn't bother me since they generally play on their own servers anyway. |
| 17:00 |
hecks |
sort of but that's only good in singleplayer |
| 17:00 |
hecks |
and actually a liability in multi |
| 17:01 |
MTDiscord |
<Warr1024> The way I support things like Red Wool is by doing the painful thing, i.e. baking all the concatenation in on the source language strings, then passing the whole damn thing into translation. That means you don't just have to translate Wool once, but once for each color, which sucks, but at least allows translation into languages that don't e.g. retain the same word order to work. |
| 17:02 |
MTDiscord |
<Warr1024> The rules on my server require that you be able to handle English well enough to read and understand the rules to even play. Beyond that, I don't really care what your UI language is. People running their own servers can have their own language rules. |
| 17:02 |
hecks |
yup, oh well |
| 17:02 |
hecks |
as long as translations are opt-in then i don't mind |
| 17:03 |
MTDiscord |
<Warr1024> Yeah, anything that adds complexity like that should be opt-in. |
| 17:03 |
hecks |
anyway [png |
| 17:03 |
hecks |
we can do it today honestly |
| 17:03 |
hecks |
right now, this is trivial |
| 17:04 |
hecks |
we have a server side encoder api, and a base64 helper |
| 17:04 |
hecks |
decoding it client side is just reading the string, decoding, feeding it to the loader |
| 17:04 |
MTDiscord |
<Warr1024> As I understand it you're working on a "game as a service" where you run the servers and such. I'm working on a redistributable game where people can play SP, run their own servers, whatever. I think we may have different priorities, the union of us representing the community as a whole better than either individually, and the engine should ideally try to support both. |
| 17:04 |
MTDiscord |
<Warr1024> I think [png would be useful for a wide variety of needs |
| 17:05 |
MTDiscord |
<Warr1024> and texture GC would be awesome for everyone too :-) |
| 17:05 |
MTDiscord |
<Warr1024> I imagine texture GC must be a bit more complex though because otherwise surely someone would have done it by now. |
| 17:06 |
MTDiscord |
<Warr1024> Probably because MT (1) doesn't distinguish between derived textures that it could recreate vs. raw ones it would need to reload from disk, and (2) doesn't keep track of how many things are using the texture. We might have to effectively split computed textures out into like an MRU kind of space and then be prepared to dynamically recreate them if needed again... |
| 17:07 |
hecks |
lack of GC is basically a mildly annoying bug where the clients have to relog every now and then |
| 17:07 |
MTDiscord |
<Warr1024> Well, the "every now and then" varies widely. |
| 17:07 |
hecks |
whereas the lack of something like [png is more serious in that it causes heavy lag and wastes bandwidth |
| 17:08 |
hecks |
i don't think i ever had to relog due to a lack of vram even on absolutely ridiculous mall servers |
| 17:08 |
MTDiscord |
<Warr1024> I was running out of memory every couple of minutes with my skyboxes until I implemented some optimizations, but at least some of those I would like to not need. |
| 17:08 |
hecks |
but let me run the numbers |
| 17:08 |
hecks |
a unique clothinglayer set in mibi is 512K |
| 17:09 |
MTDiscord |
<Warr1024> Your "sky layer API" though might be a better approach for me than texture mods in that specific case, but I do feel that GC in general might save us from a larger class of things we've become blind to the pain from. |
| 17:09 |
hecks |
okay, you could actually run out relatively quick |
| 17:09 |
hecks |
all it takes is for someone to change their armor a lot |
| 17:09 |
hecks |
s |
| 17:09 |
hecks |
sky layers were also actually conceived as a way to avoid texstring monkeying on skyboxes |
| 17:09 |
MTDiscord |
<Warr1024> I have a star in my sky that always points in the direction of spawn, and indicates distance ... that means I'm rebuilding a 256x256 texture each time the player's X/Z pos changes enough to change the pixel location of the star on the sky. |
| 17:10 |
hecks |
because not only the vram waste is gigantic, the time it takes to render 6 cube faces at a reasonable speed is noticeable |
| 17:10 |
MTDiscord |
<Warr1024> Hilariously I was bleeding out memory within 30 seconds before I discovered that [combine: positions are rounded to integers in rendering but having the decimals in the texture strings still causes a cache miss :-D |
| 17:10 |
hecks |
i re-render the sky depending on time of day, tinting and fading stars in and out |
| 17:11 |
hecks |
the resolution is basically barely good enough |
| 17:11 |
hecks |
and the vram waste is awful, this doesn't scale at all |
| 17:11 |
MTDiscord |
<Warr1024> Yeah, I'd also really like to make my skybox more dynamic, i.e. have more moving stars and multiple layers and stuff. No idea whether texmods w/ GC or a new sky layer API is the best way to go. |
| 17:11 |
MTDiscord |
<Warr1024> The sky layer thing would help great with the skybox but it'd be very specific to this one problem |
| 17:12 |
hecks |
texmods are absolutely not a good way to do skyboxes even with GC and embedded images |
| 17:12 |
hecks |
the sky layer api will happen (after the renderer cleanup) |
| 17:12 |
MTDiscord |
<Warr1024> texturemods might still be a bit clunky but you could use them with all sorts of things, like character skins showing dynamic damage, unique mob variations, etc. |
| 17:12 |
hecks |
texmods exist for material variation |
| 17:12 |
MTDiscord |
<Warr1024> In theory there's no reason we can't do both ... but we still have to pick which to do first then. |
| 17:13 |
hecks |
if you could look into texmod GC that would be helpful |
| 17:13 |
MTDiscord |
<Warr1024> Haha, at this point it's kind of silly to say what "purpose" texture mods are supposed to have because they're now used for a ton of stuff and there's no putting the genie back in the bottle on that one... |
| 17:13 |
MTDiscord |
<Warr1024> I'm not too familiar with the rendering side of the engine but I can put it on my todo list |
| 17:14 |
MTDiscord |
<Warr1024> I'll let you know if I reach the point where it looks like I might be able to make it work before you do, if you let me know the same :-) |
| 17:24 |
hecks |
hm? |
| 17:25 |
hecks |
well you can't stop idiots from abusing texmods, what you can do is make the abuse unnecessary by providing healthy alternatives |
| 17:32 |
MTDiscord |
<Kimapr> hhaha, i just thinked of a pretty brutal punishment for nasty players |
| 17:33 |
MTDiscord |
<Kimapr> 1. teleport them to a distant place |
| 17:33 |
MTDiscord |
<Kimapr> 2. keep making huge textures with texture mods around them |
| 17:33 |
MTDiscord |
<Warr1024> Might be better for #general or #modding on the Discord side, not the engine dev IRC channel... |
| 17:34 |
MTDiscord |
<Kimapr> well, it's sorta related to the discussion here |
| 17:34 |
MTDiscord |
<Kimapr> but yeah |
| 17:35 |
MTDiscord |
<Warr1024> If you're talking about crashing their client by abusing texture mods, yeah, I think there might even be troll mods out there that already do that. You can present particles to individual players so you don't even need to teleport. You can just immediately start filling up their RAM. |
| 17:35 |
MTDiscord |
<Kimapr> i think their whole OS might lag to death |
| 17:35 |
MTDiscord |
<Warr1024> I don't think we intend to specifically mitigate that though. If a server operator wants to play nasty like that then they should arguably be shunned for not dealing with their players in an honest manner... or at least players likely to receive that treatment should beware. |
| 17:36 |
MTDiscord |
<Warr1024> Yeah, it's possible for the whole OS to lag out. That's sort of not really MT's responsibility though, or at least not as clearly MT's responsibility as some of the other issues we're pushing... |
| 17:56 |
MTDiscord |
<Jordach> My machines contain more than 4GBs of video memory and that’s take a while to fill |
| 17:56 |
MTDiscord |
<Jordach> It’s not really doable for certain people |
| 18:04 |
|
v-rob joined #minetest-dev |
| 18:26 |
sfan5 |
<hecks> the fact that the media is propagated to all clients makes it barely useful <- on my TODO list |
| 18:27 |
hecks |
you won't oppose [png though right? |
| 18:27 |
hecks |
cause i'm writing it right now |
| 18:29 |
sfan5 |
well... |
| 18:29 |
celeron55 |
i think [png sounds fun |
| 18:29 |
sfan5 |
I think its' usecases are better solved by being able to get media to specific player(s) at runtime |
| 18:29 |
celeron55 |
whether fun is what we're here for is debatable |
| 18:29 |
celeron55 |
but [png is definitely fun |
| 18:30 |
hecks |
^[png is for truly disposable things and crazy experiments like skybox farmap |
| 18:30 |
MTDiscord |
<Warr1024> Ultimately [png is a nice rejection of our habitual problem of saying that a solution is not good enough but then not making the "correct" solution either. |
| 18:30 |
sfan5 |
but without texture GC nothing is disposable |
| 18:30 |
hecks |
i know, that's next to solve |
| 18:31 |
hecks |
it's basically a loophole for rendering things specific to one client at one time |
| 18:31 |
MTDiscord |
<Warr1024> [png is a nice low-hanging fruit. |
| 18:31 |
celeron55 |
just go ahead and document it as "not recommended for anything but proof of concept implementations of future features" |
| 18:31 |
hecks |
there are uses for it even if we were to do "correct" dynamic assets |
| 18:31 |
sfan5 |
dynamic assets could be GC'd when unused too, couldn't they? |
| 18:32 |
hecks |
it's legit though, there's nothing to "not recommend" |
| 18:32 |
hecks |
it's fine to have it alongside |
| 18:32 |
hecks |
another case; remember that painting mod? |
| 18:32 |
sfan5 |
except encoding images as strings... |
| 18:32 |
celeron55 |
well, websites do it all the time these days |
| 18:32 |
celeron55 |
it can't be that bad |
| 18:32 |
hecks |
if this string would be the only string in existence referring to said image |
| 18:32 |
sfan5 |
heh |
| 18:32 |
hecks |
then what's the difference? |
| 18:32 |
hecks |
it's less lines to send it in-band |
| 18:33 |
hecks |
again, this is for disposable or unique images |
| 18:33 |
celeron55 |
just do it |
| 18:33 |
hecks |
sure =] |
| 18:33 |
sfan5 |
the differece to me is that you don't have a blob of data shoved around in the tile image code where it doesn't belong |
| 18:33 |
hecks |
it already works i just need to make some cool devtest nodes |
| 18:34 |
hecks |
oh come on |
| 18:34 |
celeron55 |
how large is a 16x16 png generally? |
| 18:34 |
v-rob |
Presumably, less than 16*16*4 bytes |
| 18:35 |
celeron55 |
probably like a line of text |
| 18:35 |
celeron55 |
assuming hand drawn |
| 18:35 |
celeron55 |
or generated in a hand drawn style |
| 18:35 |
hecks |
64x64 mandelbrot was like a couple lines in my terminal |
| 18:35 |
sfan5 |
it's not that we can't have [png but I think it should be discouraged |
| 18:35 |
hecks |
...... |
| 18:35 |
celeron55 |
let's put that this way: we need to have a way to benchmark if it becomes a performance problem |
| 18:35 |
sfan5 |
also I guarantee modders will use this to set 1MB images in entity props and then wonder when the network stalls every time those are updated |
| 18:36 |
celeron55 |
do we have a way of doing that in the long term |
| 18:36 |
sfan5 |
s/when/why/ |
| 18:36 |
Krock |
and why clients run out of RAM |
| 18:36 |
hecks |
they already do this |
| 18:36 |
hecks |
by abusing ^[combine |
| 18:36 |
v-rob |
They do it with [combine, which is even worse |
| 18:36 |
hecks |
except the result is 10x bigger |
| 18:36 |
Krock |
current modifiers already can cause similar side-effects |
| 18:36 |
sfan5 |
(the same problem applies to dynamic media right now, but it wouldn't to the future implementation) |
| 18:36 |
v-rob |
And [combine caches _every single_ intermediate result |
| 18:36 |
hecks |
join any mall server and your client will be frozen for SECONDS blitting those colored text signs |
| 18:37 |
v-rob |
See, I have an idea for text on signs |
| 18:37 |
hecks |
^[string? |
| 18:37 |
sfan5 |
speaking of text https://github.com/minetest/minetest/pull/9999 |
| 18:37 |
v-rob |
Hang on, let me type it out. It doesn't involve texture modifiers |
| 18:37 |
v-rob |
Or that |
| 18:37 |
hecks |
anyway i see nothing... dirty about ^[png, it has uses |
| 18:38 |
hecks |
not involving texture modifiers worries me because it's probably very complicated and depends on client installed fonts =] |
| 18:38 |
v-rob |
Oh hush, you always worry about that |
| 18:38 |
hecks |
well |
| 18:38 |
sfan5 |
well "just display this text I don't care how it looks" would have been good enough so far |
| 18:38 |
v-rob |
:P |
| 18:38 |
hecks |
did you know that a TTF is allowed to give you any glyph size... |
| 18:38 |
sfan5 |
if you want pixel perfect text bake it into the textures |
| 18:39 |
sfan5 |
(yes yes it's an annoying "solution") |
| 18:39 |
hecks |
scroll up in the logs, i suggested a ^[string too which could later be extended with ttf support |
| 18:39 |
v-rob |
Remember how I talked earlier about separate HUD and GUI environments? I think that these environments should be registerable from the server, but they don't necessarily render to the screen. You could render them to a render texture, and then put that render texture on a node or entity. So, you can have text, _and_ it supports the entire rich text functionality that the GUI will have. |
| 18:40 |
v-rob |
I.e. GUIs on nodes |
| 18:40 |
hecks |
that sounds very heavy |
| 18:40 |
v-rob |
Well, you would obviously cache the render textures, not doing it every frame |
| 18:40 |
hecks |
most people just want to render a signpost... |
| 18:41 |
hecks |
it's heavy API wise too, lots of lines of code to set up a whole gui just to render a thing |
| 18:41 |
hecks |
simplicity is important too |
| 18:41 |
hecks |
texmods are the king of simplicity i think |
| 18:41 |
v-rob |
Well, maybe so, but I think that my idea does have quite a bit of merit |
| 18:41 |
v-rob |
Maybe not as the only solution |
| 18:42 |
hecks |
can't get any easier than ^[string:font_name.ttf:16:hello\nworld! |
| 18:42 |
hecks |
proper escaping applied of course |
| 18:42 |
v-rob |
...Until people want changing colors and different fonts and then images and... |
| 18:42 |
hecks |
^[multiply |
| 18:43 |
hecks |
and you know, other stuff |
| 18:43 |
v-rob |
Color changing in the middle of the text |
| 18:43 |
sfan5 |
we have colored text already |
| 18:43 |
sfan5 |
no reason that can't be applied to a modifier |
| 18:43 |
hecks |
oh right, just use color escapes lol |
| 18:43 |
v-rob |
I hate EnrichedString |
| 18:43 |
hecks |
:DDDD |
| 18:43 |
hecks |
you really hate simple solutions |
| 18:43 |
hecks |
so anyway i see no reason why you couldn't have both |
| 18:44 |
v-rob |
Yeah, I have no objection to the texmod |
| 18:44 |
hecks |
rather than heckling about texmod solutions being haram and deprecated at birth |
| 18:44 |
hecks |
they're fine, they have their place |
| 18:44 |
hecks |
i like that system a lot =] |
| 18:44 |
hecks |
it's absurdly powerful, we have our own substance clone |
| 18:44 |
hecks |
i just wish it was zlibbed on transfer, that's about it |
| 18:45 |
hecks |
or even better, brotli'd |
| 18:45 |
hecks |
with a custom dictionary |
| 18:45 |
v-rob |
I don't like EnrichedString because it conflicts with better solutions. I guess it's OK for a lot of things, but it doesn't belong in the GUI, where solutions like hypertext are much better. |
| 18:46 |
sfan5 |
you want hypertext to render colored stuff in chat? |
| 18:46 |
* celeron55 |
the proud designer of texmods (for questionable purposes) |
| 18:46 |
hecks |
questionable or not it was a jackpot |
| 18:46 |
hecks |
anyway hypertext has a problem like sfan pointed out, you just have a lot more shit to validate and sanitize and strip |
| 18:47 |
v-rob |
No, a lightweight rich text element for chat. I just said hypertext because we have no lightweight rich text element again |
| 18:47 |
hecks |
more cpu cost overall for all text if everything was hypertext |
| 18:47 |
v-rob |
And it won't use HTML |
| 18:47 |
hecks |
well a subset then i guess |
| 18:47 |
hecks |
it could actually use html tags |
| 18:47 |
celeron55 |
"here's the texture name" "or actually it's a miniaturized text based photoshop" |
| 18:48 |
hecks |
c55: it's a procedural image generation expression tree, like Substance |
| 18:48 |
hecks |
it's the reason my texture folder is like |
| 18:48 |
hecks |
512K |
| 18:48 |
sfan5 |
v-rob: but isn't EnrichedString a lightweight rich text element? |
| 18:49 |
v-rob |
yeah, but it shouldn't be implemented where it is. It belongs in the GUI, not as an intermediate between the GUI and the font renderer. That's the reason vertlabels can't show color, since they ignore the existence of color escape strings. |
| 18:50 |
hecks |
looks like you guys are having fun |
| 18:51 |
sfan5 |
<hecks> not to mention that universal translation like that is generally a sisyphean task if not straight up lunacy |
| 18:51 |
sfan5 |
^ we don't do this btw |
| 18:51 |
sfan5 |
even just attempting it would be stupid |
| 18:52 |
hecks |
yeah i guess i didn't exactly get what the documentation was saying |
| 18:52 |
hecks |
but it sounded just ambitious enough for minetest =] |
| 18:52 |
|
olliy joined #minetest-dev |
| 18:52 |
sfan5 |
gettext except instead of running it on the server you add an escape code to tell the client to run gettext on a fragment of the string |
| 18:53 |
sfan5 |
...well I don't know if that is an understandable explanation |
| 18:54 |
sfan5 |
it's actually quite neat so I dunno what @Warr1024's issue with it is |
| 18:54 |
MTDiscord |
<Warr1024> what my issue with what is? |
| 18:55 |
hecks |
uhhhh forget this whole thing |
| 18:55 |
sfan5 |
client-side translations |
| 18:55 |
MTDiscord |
<Warr1024> You mean the trimmed down source/target translation thing instead of the context/source/target thing that gettext does? |
| 18:55 |
MTDiscord |
<Warr1024> lack of context, basically |
| 18:56 |
sfan5 |
you can define contexts freely |
| 18:56 |
MTDiscord |
<Warr1024> Text strings in client-side translations are identified by source |
| 18:56 |
MTDiscord |
<Warr1024> there's no way to distinguish two equal source strings with different contexts |
| 18:56 |
MTDiscord |
<Warr1024> For the record I'd be surprised if we could actually fix this though |
| 18:57 |
sfan5 |
??? |
| 18:57 |
sfan5 |
you know you can freely set the textdomain in Lua right? |
| 18:57 |
MTDiscord |
<Warr1024> It may be the kind of thing we just have to live with. |
| 18:57 |
MTDiscord |
<Warr1024> Are you suggesting that I bake context into textdomain? |
| 18:57 |
MTDiscord |
<Warr1024> that sounds messy. It seemed implied that textdomain should be like a mod or a package |
| 18:57 |
sfan5 |
yea |
| 18:57 |
MTDiscord |
<Warr1024> not like this menu or that menu or whatever... |
| 18:58 |
MTDiscord |
<Warr1024> I mean I guess in a pinch I could do that |
| 18:58 |
MTDiscord |
<Warr1024> I'm sort of torn about how baked vs. dynamic to make lots of things. Right now my translations are highly baked. If they get too long for translators to manage then I could try switching to more dynamic, and then this could become an issue, but for now, it's probably good enough. |
| 18:58 |
sfan5 |
I've never seen that context thing you mentioned with gettext so maybe that's why I'm suggesting this |
| 18:59 |
hecks |
i'm just trying to imagine the complexity of using this to translate all sorts of templated roguelike messages and my head melts |
| 19:00 |
hecks |
best if i stick to english... |
| 19:01 |
celeron55 |
those would require a template-aware translation system that can reorder the arguments |
| 19:01 |
MTDiscord |
<Warr1024> I dunno, it's definitely there in Weblate. Maybe that's only for other formats but not gettext? |
| 19:01 |
sfan5 |
(we have that) |
| 19:01 |
MTDiscord |
<Warr1024> My own use-case is pretty simple; I have very little templating. It's mostly the built-in guide, names of items, and some achievements/hints. |
| 19:02 |
sfan5 |
as far as I know Weblate showing you where the translation appears is merely to be helpful, translation strings are not differentiated by context |
| 19:02 |
hecks |
https://a.uguu.se/EUHQBkZH4wrh_mandels.png |
| 19:02 |
hecks |
"Generated In-Band Dest Blit Mandelbrot PNG Test Node" |
| 19:03 |
v-rob |
GIBDBMPTN? |
| 19:03 |
hecks |
good thing we do not use / or + anywhere in texmod syntax |
| 19:03 |
hecks |
base64 can be embedded without any tricks |
| 19:04 |
sfan5 |
there is working escaping too, if you wanted to use ^ for any reason |
| 19:04 |
sfan5 |
or ( or ) |
| 19:05 |
hecks |
sure, it's just convenient that you don't have to escape anything |
| 19:06 |
sfan5 |
yea was just saying |
| 19:06 |
hecks |
it's literally "[png"..minetest.encode_base64(minetest.encode_png(...)) |
| 19:08 |
MTDiscord |
<josiah_wi> sfan5, I fixed the issue the CI had found on the Irrlicht PR to allow importing the target without installing. |
| 19:22 |
pgimeno |
[OT] v-rob: strcpy_s is not MS proprietary, unless you mean they force-fed it into the C11 standards committee. That said, it sounds stupid to call strcpy 'deprecated'. |
| 19:23 |
v-rob |
Huh, interesting. All the same, the warnings are really annoying |
| 19:26 |
sfan5 |
if it's not ms-properietary why does glibc or musl not implement it? |
| 19:26 |
hecks |
#11498 pomf |
| 19:26 |
ShadowBot |
https://github.com/minetest/minetest/issues/11498 -- Add embedded PNG texture modifier by hecktest |
| 19:31 |
celeron55 |
you should make a separate commit of the extension whitelist edit |
| 19:33 |
sfan5 |
could also push that edit directly to master |
| 19:34 |
pgimeno |
"An implementation that defines __STDC_LIB_EXT1__ shall conform to the specifications in this annex." (the annex about bounds-checking interfaces such as strcpy_s et al) |
| 19:34 |
pgimeno |
you can be standard and not implement those |
| 19:35 |
pgimeno |
http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1548.pdf page 597 |
| 19:36 |
sfan5 |
I see |
| 19:46 |
celeron55 |
it's the standard way of being annoying |
| 19:48 |
hecks |
pushing the extensions thing to master directly if nobody minds |
| 19:49 |
celeron55 |
hmm, actually might there be some concern regards to it |
| 19:49 |
celeron55 |
probably not |
| 19:50 |
celeron55 |
there's the cross-version network compatibility aspect where an older client could support something using the old formats that a newer client would support in some new way but i don't think there's any |
| 19:53 |
hecks |
we no longer have loaders for these extensions |
| 19:53 |
hecks |
their presence in tile.cpp is a bug |
| 19:53 |
hecks |
we never documented those in the first place |
| 19:54 |
hecks |
this code is entirely client side and just decides what counts as an image in a texstring |
| 19:54 |
celeron55 |
ah it's client side |
| 19:54 |
celeron55 |
then whatever |
| 19:54 |
hecks |
and their presence in the code could confuse people |
| 19:54 |
sfan5 |
there's a server side list of them too |
| 19:54 |
hecks |
do tell me the server side list and i'll take a look at it separately |
| 19:55 |
hecks |
pushing |
| 19:55 |
sfan5 |
minetest/src/server.cpp:".png", ".jpg", ".bmp", ".tga", |
| 19:57 |
hecks |
removed the change from PR, please review |
| 19:58 |
sfan5 |
ah btw the thing I said last time, you'd do that by configuring your local git with "hecks <42101236+hecktest users.noreply.github.com>" as author |
| 20:00 |
hecks |
done |
| 20:05 |
|
Kimapr joined #minetest-dev |
| 20:17 |
|
Kimapr joined #minetest-dev |
| 20:23 |
|
Kimapr7 joined #minetest-dev |
| 20:39 |
hecks |
Pushing server.cpp unsupported media cleanup |
| 21:27 |
|
LW joined #minetest-dev |
| 21:27 |
|
v-rob joined #minetest-dev |
| 21:55 |
pgimeno |
@hecks for the sake of clarity, since you seem to have understood me wrongly, I said I routinely saved indexed TGAs but I didn't mean with Minetest. It was long ago with VGA in DOS. I just wanted to emphasize that indexed TGAs do exist, not that I was using them with Minetest. I would have liked TGA to have stayed because it's simple and powerful, that's all. |
| 21:55 |
hecks |
not you, someone else had issues with it |
| 21:55 |
pgimeno |
ok |
| 21:55 |
hecks |
so i wrote him the png thing |
| 21:56 |
hecks |
you can now use an array of colorspec to make PNGs, can't get simpler than that |
| 22:27 |
|
Extex joined #minetest-dev |
| 23:06 |
|
Extex joined #minetest-dev |
| 23:25 |
hecks |
https://a.uguu.se/2hc7CRHvJokP_sheets.png I can't imagine how, but did I break these? |
| 23:40 |
hecks |
hm, no |
| 23:47 |
|
AliasAlreadyTake joined #minetest-dev |
| 23:47 |
hecks |
oh okay, it's 11206 |
