Time  Nick         Message
13:28 erle         here is my unicode rendering library. keep in mind i created this half a year ago, so if you have questions, i may not be too good at answering them. help with bidirectional text rendering is appreciated. https://git.minetest.land/erlehmann/tga_unicode_renderer
13:30 erle         my goal is to have unicode on signs
13:32 erle         here is an example rendering, i believe no existing signs library can do even a fraction of it https://mister-muffin.de/p/lgrI.png
13:34 erle         i would appreciate input from someone writing arabic or hebrew how they even get these characters to appear inside minetest and how they should render
13:39 erle         appguru please extrapolate your onions on my unicode rendering library
13:59 MTDiscord    <luatic> i got a notification of "appguru please extrapolate your onions on my unico" which my head completed to unicorn
13:59 erle         well anyway
13:59 erle         luatic what are your opinions
13:59 erle         also can you fix the bidi text rendering
13:59 appguru      let me take a peek
14:09 appguru      erle: the `i = nil; j = nil; k = nil` at the end of `pixelops.lua` is dead code
14:10 erle         appguru, spoken like someone who thinks that thing will not get extended
14:10 appguru      erle: then use do-end blocks to scope your variables?
14:10 appguru      your variables and your test cases in general
14:10 erle         better idea yes
14:10 erle         patches welcome
14:11 erle         look i wrote this half a year ago, i just now published it because i was like “come on, it's stupid to talk about it if i am not publishing it right”
14:11 erle         like why wait until i got the rendering perfect
14:11 erle         if no one is even able to put hebrew on signs now
14:11 erle         or arabic (yes the rendering looks horrible, i have not implemented glyph shaping)
14:11 erle         but for the simple purpose of having random-ass emoji or chinese or so on signs, it's alright already
14:12 erle         appguru i am sorry i was not good with patches in the past
14:13 erle         like accepting patches
14:13 erle         i'll get better!
14:14 appguru      what's wrong with the bidi rendering btw?
14:15 erle         i had to add a single line to test case W2 to make it not crash in an assertion
14:15 erle         but generally, it is incomplete
14:15 erle         the handling of mixed RTL and LTR text is borked
14:15 erle         so if you include numbers (which are often LTR) in RTL scripts, already problems occur ig
14:16 erle         see the hebrew example bidi text
14:22 erle         appguru i do not use bidi rendering, that is why i need help
14:22 erle         appguru also what do you think about my benchmarks?
14:22 erle         i mean it's quite a surprising result that optipng and even zopflipng is a dead end
14:24 MTDiscord    <rollerozxa> text renderer!
14:24 erle         yes very text
14:25 erle         please take it and make some better signs libraries
14:25 erle         i will put it on contentdb as soon as some people say it's not shit API-wise
14:25 erle         i want emoji and weird unicode garbage on signs please
14:26 erle         and japanese and capital ẞ and random latin variants from the math glyphs codeblocks
14:41 erle         appguru ROllerozxa is the API for the unicode renderer fine with you two?
14:44 appguru      erle: looks ok, though it feels slightly weird that load_glyphs takes a line iterator, whereas save just takes a file path; would you ever want to not give it a file path?
14:45 erle         appguru this is a funny question and one that i leave you to ponder for a minute or two before i answer
14:45 erle         there is a pretty clear reason that becomes obvious once you think about how you might want to use the function
14:45 appguru      also, why are you exposing details like scanline order?
14:46 erle         because the output format is actually a pixel array and pixel arrays do have a scanline order
14:46 appguru      I'd just define it to be top to bottom
14:46 erle         yeah that's stupid
14:46 erle         i mean that was a bit crass
14:47 erle         but you can't really do it for reasons that become obvious once you try to implement it
14:47 erle         it is part of the reason why imagemagick convert(1) randomly flips images when you convert them and they have a scanline order that is not the default scanline order for the file format hardcoded in imagemagick
14:48 appguru      you could just normalize everything to top to bottom by reversing if it's bottom to top...
14:49 erle         look, having it bottom-to-top serves multiple purposes: 1. it is the default in tga (though that can change) 2. it is the default in tga_encoder 3. if you “don't care about scanline order” your image will be flipped upside down, which is a hint that you should care
14:49 erle         like, the answer to “why is my image flipped upside down” is “you thought you could ignore a thing that you should not and the default is setup so that you *will* run into this if you ignore it”
14:50 appguru      you can't ignore it in your code, but why can't users of your code ignore it?
14:50 erle         making an “edge case” into a default case if no info is given about scanlines makes programmers handle it correctly
14:50 erle         because they have to know
14:50 erle         if you have a pixel array, it does not encode if 0,0 is top left, top right, bottom left, bottom right
14:51 erle         and if you remember school, 0,0 is bottom left
14:51 erle         in a coordinate system
14:51 erle         minetest internally does this too at some places i think
14:52 erle         any hardcoded assumption will randomly flip your images
14:52 erle         which is why imagemagick is shit (the devs are not willing to fix it lol)
14:53 erle         all i did was chose a default that is guaranteed to a) work with tga_encoder b) trip up people who think they don't have to care about scanline order
14:53 erle         basically i write code that makes it hard to pretend something is an edge case
14:54 erle         appguru, also as you may have noticed, you don't need to give scanline order, as long as you adhere to the traditional (you learned it in school) idea that 0,0 is the bottom left pixel
14:54 ROllerozxa   why *should* I need to care about scanline order in the first place?
14:54 appguru      I think in programming the convention is typically the opposite.
14:55 appguru      If you specify an image as a Lua table, it will be upside down in your code if you use a scanline order from bottom to top.
14:55 erle         ROllerozxa, because whenever you do not, you assume some default that breaks some use cases (e.g. the “images are randomly flipped” thing)
14:56 erle         the only thing that i did that trips you up was choose a default that is entirely appropriate for tga_encoder (because it assumes that default, i can not change it)
14:57 erle         i mean i can change it
14:57 erle         but then every image will be flipped upside down lol
14:57 erle         also i am pretty sure that the axes are labeled x and z and z goes up
14:58 erle         in any case “why should i care about scanline order” is like “why should i care about gamma”
14:59 erle         i.e. you ask it only if you have not explored the problem space enough of “well what happens if i don't care”
15:01 erle         ROllerozxa appguru apart from “hurr durr scanline order” anything else that trips you up about the API?
15:02 erle         i mean it can't be much higher level without being shit
15:02 erle         for example, the whole color format thing is also something that people don't want to care about, but have to
15:02 erle         not caring about it gets you the fliesizes that minetest.encode_png() produces ;)
15:03 erle         i mean, i fully expect most signs to contain exactly 2 colormap entries
15:03 erle         which means the pixels in the pixel array are an index into the colormap then
15:03 erle         which unfortunately looks exactly the same as a grayscale pixel array
15:03 erle         but that's life
15:03 erle         in largely untyped programming languages
15:07 erle         ROllerozxa appguru i'll leave it simmer for a few days, then put it on contentdb based on further feedback, so pls gib more of it
15:12 erle         tbh i have the feeling almost everyone who fancies themselves an EXPERT PROGRAMMER in some domain asks the same questions as the hacker news commenters, i.e. “why have / haven't you done this thing that is obviously (not) a good idea to a person with domain language” coming from someone seeing a thing for the first time
15:13 erle         like people who can't program at all don't do this. people who realize that their expertise in one domain does not transfer easily don't do it. but “i have drawn a bunch of textures with gimp so i know about file formats” is the sweet spot ;)
15:13 erle         (not directed to ROllerozxa and appguru, just a general observation)
15:15 erle         another good one is “i do not believe in benchmarks because the result is obviously false” (instead of pointing out what the problem with the benchmark code is; this is how you spot a huge ego)
15:17 erle         at work i once got issues with a project manager who was literally “i have done some stuff with raspberry pi, so i think you can do this in a few days”. i believe that person may no longer be allowed to question anyones estimates lol.
15:18 erle         i meant “a person with domain knowledge” not “a person with domain language”
16:09 erle         for future reference: the reason why you want a line iterator and not a file when loading a font is because overriding a single glyph is common enough that it does not make sense and because having to handle file loading error cases sucks
16:10 erle         given that the .hex font format has one glyph per line, a line is the most obvious unit to iterate over
16:10 erle         other font file format loaders might use other separators than newlines or even other data structures, but even then it would make sense to separate file loading and parsing the input
16:11 erle         as it is, definitely, separate concerns
16:11 erle         also, notably, writing files is not done by filenames at all. you get a pixel array.
16:12 erle         the only reason that is stuffed into tga_encoder is because that is the easiest and fastest solution both in terms of CPU and in terms of mental load
16:47 erle         appguru see log for iterator explanation
16:47 erle         TL;DR: you want to load single glyphs often enough, the file format has one glyph per line
16:47 erle         appguru also the saving is done using tga_encoder and not using tga_unicode_renderer, that one merely gives you a pixel array
16:57 appguru      erle: seems reasonable then
16:59 erle         appguru, any other comments? the only useful one i got so far was from someone who was like “how to do CJK variant selection”
17:00 erle         and i must admit i had not thought about this
17:00 erle         appguru in other words: if you would use this library to generate textures for signs in minetest, what would be missing or wrong API-wise?
17:07 appguru      erle: I'm not familiar with Unicode rendering (I wouldn't have thought of CJK variants, for example). I'm not really missing anything at the moment.
17:20 erle         appguru anything you think that is absolutely stupid and should not be in there?
17:20 erle         again, in the context of a signs library
17:20 erle         or really any texture you can generate
17:22 appguru      I see exclusively `dofile`s without `get_modpath` in the code. Have you tested this in Minetest yet, from different working directories?
17:25 appguru      erle: ^
17:34 erle         appguru i have, but ig i did not commit it!
17:35 erle         appguru first person to notice that the code in the repo does not work in minetest, conclusively proving no one ever tried to make a mod yet ;)
17:35 erle         appguru what's the minetest-safe idiom to do this both in and out of minetest?
17:35 erle         surely you have one
17:35 erle         as a LUA EXPERT
17:36 appguru      I don't think there is a terribly simple "idiom", I usually end up writing myself some "load file" helper function which prepends the mod file if necessary (if minetest is present)
17:39 erle         appguru, ok show me yours (after all, i showed you mine)
17:39 erle         your helper function i mean
17:42 appguru      erle: wait a sec
17:42 appguru      do you use luacheck?
17:42 appguru      there seems to be some global pollution there
17:43 Fleckenstein my phone was logged in as Fleckenstein
17:43 Fleckenstein and i was wondering why i cant do it on laptop
17:44 appguru      hmm the global pollution probably is limited to examples.lua and test.lua where it may be considered fine
17:45 appguru      still usually you put all your global variables under an API table which has the same name as your mod, or you pass them around using require or similar mechanisms
17:46 appguru      for example you set an utf8 global variable to the utf8 functions you need in the format you need them - this would be incompatible with any other mod attempting to do the same
17:47 appguru      oh also why do you explicitly list 0-9 and A-F in your hex charsets?
17:47 appguru      why not use %x or if you only want to allow uppercase, [%dA-F]
17:48 appguru      also what's with the "coding: utf8" comment, that seems like a python thing you don't need in lua
17:51 erle         appguru, i did not use luacheck and the examples are stuff i just shat out with no regards for code quality
17:54 erle         appguru, the coding comment is an emacs thing
17:55 erle         appguru, the explicit list of hex chars is a funny thing. the code is easier to read that way.
17:55 erle         at least that is what i think
17:55 erle         very explicit
17:55 erle         in my experience people usually don't know lua patterns
17:56 erle         evidence: funny bugs where inserting stuff into formspecs hangs servers
20:58 MinetestBot  02[git] 04chmodsayshello -> 03minetest/minetest: Send ever lasting particle spawners to all players (#13774) 13294ad98 https://github.com/minetest/minetest/commit/294ad987763014c5d20d4260b76fee831581c87a (152023-09-02T20:58:11Z)
21:02 MinetestBot  02[git] 04grorp -> 03minetest/minetest: Rewrite the gamebar (#13768) 1383b85ba https://github.com/minetest/minetest/commit/83b85ba16aaab7bddc479beada4ca3af9fa54a1b (152023-09-02T21:02:02Z)