Time Nick Message 01:32 ANAND Oh nice! 17:40 Krock wow rubenwardy, that was quick 17:41 Krock although this will require running Minetest with the default settings and DPI to compare it with the "official" screenshots 17:43 rubenwardy Settings are baked in 17:43 rubenwardy I didn't consider dpi 17:43 rubenwardy Will need to override it 17:45 Krock you can also add image tolerance by downscaling it 17:46 Krock linear scaling will hide details 17:55 Calinou you might not want to perform "binding" automated tests using screenshots in general 17:55 Calinou posting screenshots in pull requests as comments might be a better idea 17:55 Calinou https://bartwronski.com/2019/08/14/how-not-to-test-graphics-algorithms/ 18:00 rubenwardy The better approach isn't possible with our code without big rewrites, making the thing useless 18:01 rubenwardy There will need to be manual intervention, but that's fine. It's a lot less time than bugs being introduced 18:01 rubenwardy Perhaps it should comment any failing screenshots in the thread only, and not mark the build as failed 18:04 Krock Calinou: good intentions are there, but you cannot do that with a rendering engine 18:04 Krock multiple components have to interact with eachother and be rendered to check whether it looks okay 18:06 Calinou indeed, I'm arguing we shouldn't perform automated checks for screenshots :) 18:06 Calinou the ROI just isn't there due to the false positives 18:06 Calinou keeping screenshots for archiving can be very useful though, so you can easily bisect regressions 18:07 Krock right. comparison would best be done by eye 18:07 Krock otherwise it will be impossible to "pass" the tests on GUI bugfixes 18:08 rubenwardy The intention is for some comparison to be done by eye, but for most of it to be done in bulk 18:15 rubenwardy Perhaps it doesn't need to be automated by CI, it could be ran by a developer when testing