Handle semi-invalid vaccination certificate QR-codes (EXPOSUREAPP-7755)

This introduces "modes" to qrcode data parsing.

UI based parsing will be STRICT and show error codes, because we can show pop a dialog and abort.
Storage based parsing will be LENIENT. If the user managed to scan something invalid, then we can't just delete it, and we also can't (shouldn't) crash the app. Future updates could introduce "error UI states" for this.

In this case, users scanned a vaccination qr code that contained more than 1 data set. Unclear whether this was just hacking around, or some (non DE?) sources generate such codes.
This is a test vaccination qr code that contain two data entries.
DO NOT SCAN THIS
With your prod version :expressionless:, or your app will crash until the next update is out  :shrug:.
Before:

Error dialog about more than one list item.
Crash after dismissing dialog
Crash on app open

java.lang.IllegalArgumentException:
at kotlin.collections.ArraysKt___ArraysKt.single (_Arrays.kt:10)
at de.rki.coronawarnapp.vaccination.core.repository.storage.VaccinationContainer.getVaccination (VaccinationContainer.kt:1)
at de.rki.coronawarnapp.vaccination.core.repository.storage.VaccinationContainer$toVaccinationCertificate$1.getDoseNumber (VaccinationContainer.kt:1)
at de.rki.coronawarnapp.vaccination.core.VaccinatedPerson.getTimeUntilImmunity (VaccinatedPerson.kt:4)
at de.rki.coronawarnapp.vaccination.core.VaccinatedPerson.getVaccinationStatus$default (VaccinatedPerson.kt:4)
at de.rki.coronawarnapp.ui.main.home.HomeFragmentViewModel$special$$inlined$combine$1$3.invokeSuspend (Zip.kt:20)
at de.rki.coronawarnapp.ui.main.home.HomeFragmentViewModel$special$$inlined$combine$1$3.invoke (Zip.kt:2)
at kotlinx.coroutines.flow.internal.CombineKt$combineInternal$2.invokeSuspend (Combine.kt:26)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith (ContinuationImpl.kt:3)
at kotlinx.coroutines.DispatchedTask.run (DispatchedTask.kt:15)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely (CoroutineScheduler.kt:1)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run (CoroutineScheduler.kt:12)

Now:

Error dialog about qr code not being supported yet
No crash on dismiss
If you have already scanned the bad qrcode and update, your app should no longer crash. The app will show the newest out of multiple available data entries.

DO NOT SCAN THIS

😱
But seriously - while you're having your hands on it, will you consider to remove the  error dialog's "details" button here also?
See #3400 , #3413 , #3394

I don't understand the problem here. Is the QR-Code you shared invalid under Android, and if yes why? It works smoothly on my iOS device.

@vaubaehn

DO NOT SCAN THIS

But seriously - while you're having your hands on it, will you consider to remove the error dialog's "details" button here also?
See #3400 , #3413 , #3394

The details button is part of the general error dialog code that is used in many places.
Changing that is outside of the hotfix scope.
Whether all dialogs with a more specific error message should have the details button removed is something to discuss as part of a enhancement ticket.
Does every specific error type make the stacktrace obsolete, or are there cases where the stacktrace helps us fix it? The with error logs available, we could get the stacktrace that way still.
@Ein-Tim

I don't understand the problem with the QR-Code here. Why is the QR-Code you shared invalid under Android? It works smoothly on my iOS device.

The specifications say that a vaccination certificate looks like this
{
  // Version (required)
  "ver": "1.0.0",
  // Name (required)
  "nam": {
    // Family name
    "fn": "d'Arsøns - van Halen",
    // Given name
    "gn": "François-Joan",
    // Standardized family name (required)
    "fnt": "DARSONS<VAN<HALEN",
    // Standardized given name
    "gnt": "FRANCOIS<JOAN"
  },
  // Date of birth (required)
  "dob": "2009-02-28",
  // Vaccination certificate (assumption: exactly one entry)
  "v": [
    {
      // Disease or agent targeted (required)
      "tg": "840539006",
      // Vaccine or prophylaxis (required)
      "vp": "1119349007",
      // Vaccine medicinal product (required)
      "mp": "EU/1/20/1528",
      // Marketing Authorization Holder (required)
      "ma": "ORG-100030215",
      // Dose Number (required)
      "dn": 2,
      // Total Series of Doses (required)
      "sd": 2,
      // Date of Vaccination (required)
      "dt": "2021-04-21",
      // Country of Vaccination (required)
      "co": "NL",
      // Certificate Issuer (required)
      "is": "Ministry of Public Health, Welfare and Sport",
      // Unique Certificate Identifier (required)
      "ci": "urn:uvci:01:NL:PlA8UWS60Z4RZXVALl6GAZ"
    }
  ]
}

The invalid certificate contains two elements in the v array.
Android internally used List.single() which caused the crash.
For everyone already affected, this fix changes it to show the newest of both elements.
According to @nickguendling
iOS currently just takes the first entry if there are multiple, and will get additional validiation in one of the next updates.

@d4rken

The details button is part of the general error dialog code that is used in many places.
Changing that is outside of the hotfix scope.

My full understanding. I naively thought, it could be easy to make use of @chiljamgossow 's code in #3400 , setting VC_MULTIPLE_VACCINATION_ENTRIES in the error list and using her error dialog builder, but it's probably much more complex :)

Whether all dialogs with a more specific error message should have the details button removed is something to discuss as part of a enhancement ticket.
Does every specific error type make the stacktrace obsolete, or are there cases where the stacktrace helps us fix it? With error logs available, we could get the stacktrace that way still. 🤔

I think good candidates for a more user-friendly/simpler error handling do not depend only on their specifity, but also how well-known they are, and that no "surprises" can be expected, like API 39508. I think everyone would agree that no stacktrace is needed for them, and I guess most vc errors could also be set on this list. But right, this is something that needs to be discussed in your whole team and could be part of a bigger refactoring of error handling. Afair, there are already open tickets for this, so there is no need to create a new issue for this, is there?

My full understanding. I naively thought, it could be easy to make use of @chiljamgossow 's code in #3400 , setting VC_MULTIPLE_VACCINATION_ENTRIES in the error list and using her error dialog builder, but it's probably much more complex :)

That PR overwrites the default "neutral action" (the details -> stacktrace) of the error dialog builder on that specific screen. Yes we could do the same in this case, but we don't want to do this manually for every screen. We want an automatic solution :grin:.
The actual code may be as easy as "if(isKnownError) { setNeutralButton(null) }", but we need to think about the edge cases, and that we move all error handling into "nice and consistent" territory :coffee:.

I think good candidates for a more user-friendly/simpler error handling do not depend only on their specifity, but also how well-known they are, and that no "surprises" can be expected, like API 39508. I think everyone would agree that no stacktrace is needed for them, and I guess most vc errors could also be set on this list. But right, this is something that needs to be discussed in your whole team and could be part of a bigger refactoring of error handling.

Afair, there are already open tickets for this, so there is no need to create a new issue for this, is there?

Discussion is always welcome. I'm not sure atm if there is a suitable ticket yet. I don't think we need to discuss each dialog individually, we'd need to discuss this on a general level.
We have a few links already, maybe "links to faq" should be part of the error dialog system and if such a link is present, automatically show that instead of the stacktrace button.
There is also the issue that only some parts of the app use the HasHumanReadableError based exception system, for it all to work nicely together we need to use it everywhere, and get rid of the old CWA 1.0 error code stuff (e.g. CODE 900X).

That PR overwrites the default "neutral action" (the details -> stacktrace) of the error dialog builder on that specific screen. Yes we could do the same in this case, but we don't want to do this manually for every screen. We want an automatic solution 😁.

Automatic is good 🤖 Also, if I interpreted the code right after looking again, you would not get a specific "not yet supported" error here, but the general "invaild certificate" together with a link to the faq where nothing is explained, if you joined VC_MULTIPLE_VACCINATION_ENTRIES to codesVcInvalid , so...

The actual code may be as easy as "if(isKnownError) { setNeutralButton(null) }", but we need to think about the edge cases, and that we move all error handling into "nice and consistent" territory ☕.

👍

We have a few links already, maybe "links to faq" should be part of the error dialog system and if such a link is present, automatically show that instead of the stacktrace button.

🚀

There is also the issue that only some parts of the app use the HasHumanReadableError based exception system, for it all to work nicely together we need to use it everywhere, and get rid of the old CWA 1.0 error code stuff (e.g. CODE 900X). I want to delete this 😄

😄 Yes... But as "I grew up here with them", I'd kind a miss them, although I never saw one of them for months... 😉

@chiljamgossow will improve some of that already for 2.5,

👍

but overall there are a few more 🚧 sites, and while refactoring that and nice architectures make me happy, other parties may see priorities else where 😉 .

There are other priorities? 😁
Have a nice night, see you soon! 👋


Start with adding ribasim_cli tests

Fixes #339

This is ready for review from my side.
I couldn't get the tests to run on TeamCity for reasons that are completely non-obvious to me.
The tests fail with some Julia error, my theory was that it somehow installs an old version of Ribasim Python, but I wouldn't know how. Locally, the tests work fine. I've now disabled the TeamCity tests to be revisited later.


Cloning existing database from a different namespace

Is it possible to clone database from another namespace?
I didn't find any reference in the documentation.

yes it is.


Add PR patches to version 0.16

Tauri version 1.x will stick to tao v0.16.
There are some patches we would like to merge into it as well. Here are the patches we need:

#689

I opened a v0.16 branch. Please open a PR against it.

Closing as this serves nothing, we will add bug fixes directly to v0.16 branch if needed


Feature implib

Little improvement on top of my previous PR.
I had forgotten to test the implib feature when you specify the --output-dir, now it should work fine no matter whether you use --output-dir or not. Sorry I didn't test it properly before :/

Also kind of looks like you are not using "os.path.join()" but string "format", which is rather bad way of doing it. Never should one hard code "\" not even in OS specific code, just because it makes it always harder to see what is going on, not to mention how it is better at handling corner cases.
As an aside, I also expect that Python 2.6 will not work, because "format" there only accepts keyword arguments, which is why I only use it like that in Nuitka. As you have probably noticed, I like named arguments very much. so I don't see that as an issue. Of course, using f strings of 3.6 would be more cool anyway, but that's not happening for a while.
But yeah, even those are worse than os.path.join. Honestly I don't even quite know what splitpath does on most days. :-)
Yours,
Kay

BTW: As for conflicts, do not bother, I will remove the merge and do another squash merge later.

Also kind of looks like you are not using "os.path.join()" but string "format", which is rather bad way of doing it. Never should one hard code "" not even in OS specific code, just because it makes it always harder to see what is going on, not to mention how it is better at handling corner cases.
As an aside, I also expect that Python 2.6 will not work, because "format" there only accepts keyword arguments, which is why I only use it like that in Nuitka. As you have probably noticed, I like named arguments very much. so I don't see that as an issue. Of course, using f strings of 3.6 would be more cool anyway, but that's not happening for a while.
But yeah, even those are worse than os.path.join. Honestly I don't even quite know what splitpath does on most days. :-)

You're absolutely right here! Hardcoded paths like the ones I've provided in the commit should be avoided all the times, period. Thing why I've used in this case is mainly because in the past when I was writing a metacompiler that would produce visual studio commands (using cl & link) I would find myself than sometimes visual studio options (don't recall which ones specifically) weren't playing nicely when using os.path.join and I had to hardcode paths to produce the right outcome. In this case, I think IMPLIB should be fine with os.path.join so I'll definitely change it though :)
You know? These days I've almost forgotten about using os.path module at all... each time I see in some of my old codebase os.path stuff I try to replace it with the lovely pathlib.Path module... seriously, pathlib.Path is a wonderful module... I've falled in loved with that module the first time I've discovered it... stem, with_suffix, with_name, suffix, name, path concatenation, etc... The final code become much much less verbose and readable than when using the old os.path utilities :)
One fast question, which python version should I assume when improving nuitka? So far I've tried to avoid some stuff from python3.6, which it's the version I tend to use all the time... I almost don't use 2.6/2.7 anymore nowadays... and 3.3? I only use it when coding SublimeText plugins :/

BTW: As for conflicts, do not bother, I will remove the merge and do another squash merge later.

That's great, but please, don't make it so easy to me hehe :) , I'd really like to eventually master git... so if there is any way I can squash the whole set of commits here and provide a PR with one single commit, let me practice and polish my poor git skills... Right now I was reading this interesting article talking about it https://eli.thegreenplace.net/2014/02/19/squashing-github-pull-requests-into-a-single-commit

@kayhayen Hehe, ok, I've managed to make this PR into a single commit, how cool is that, eh? :'D . Please let me know if I've done something wrong here though, as I told you... I'm still learning how to become proficient with git :)

I am still confused, why you join the absolute and normalized source_dir in the first place, why not just source_dir. I have really bad experience with Scons and other tools on Windows with absolute paths, because many tools will happily build below strange paths, for as long as they do not appear on the command line. Then all sorts of errors pop up. Also it's not needed.
And then the basename of source_dir is not useful either, result_basepath would be better for that, but why not just something hard coded. I assume the name doesn't really matter, so why not use os.path.join(source_dir, "import.lib") and be done with it?

Sure thing, if you consider the current commit could produce issues and if you also believe having hardcoded names for the import lib is just fine, please be my guest. Actually I can't think of any real-case where we'd like to preserve the original names of the import libraries... I mean, maybe if you wanted to use the dlls on some c++ projects the names could be relevant... but that's not very real case so I guess harcoded names that makes a simpler code is alright.
Btw, in cases like this one where you think some changes on top of it will produce a much cleaner/performant/elegant code, don't even ask me, just tweak it, merge it and I'll eventually test it on windows :) .  Plus, mostly of the times I'll want to tweak anything I'll probably won't be able to choose the optimal way because I won't be aware about all the gritty details and implications.
I can give good advices at very high-level architectural decissions... but a very low-level I'm not sure I'll be very helpful (at least for now) ;)
PS. If something can be simplified, simplify it. If there is dead-code, get rid of it, and so and so forth... Less code means less maintenance. I'd used the normalize_path there as it's one functions I tend to use almost in all my projects, so..
\o

As for your suggestion that I could just do it myself. Indeed. It's a mistake a made in the past and it drives away contributors. As soon as somebody touched an area of Nuitka, I was also there and did improve it myself.The majority of people then consider themselves unnecessary or bad. So I have changed my stance, and actually will try and not follow impulses to just do it myself, and let other people know, why I think it should or must be different.
As for your "normalize", I dislike it, and I suggest you consider input sanitizing. Every value you get from the user side, must be checked, converted, then, and exactly then, when you got it. Not any later. Randomly normalizing paths in the middle of somewhere doesn't do any good, except bug hiding potentially. I would try and not do it. That is not to say that I have not done it wrong. But that is never the point. The code review is for 2 people to notice things and be rather sure of the outcome.
So yeah, I am not going to do this commit, even if it's fast. The whole point of this being a community is to get more stuff done as a whole, and to learn making better changes. If I get to do cleanups on your changes each time, that is not good.
And for your architectural skills, I am a skeptical thinker. The right time to believe something is when there is evidence for it. Somebody telling me he has skills, doesn't count as evidence. Somebody actually applying them successfully does.
The idea of your other PR is very interesting. It feels like it should be a best practice actually. I am pretty sure I have not seen it in Python projects I fancy. I have applied "property sheets" (that is the name I know for this) for GUIs, where these values are also dynamic. I can totally see the benefit for a CLI now having seen your example, that is great.
But as you can imagine, I find this innovative and cool, and even very useful. I am not so convinced this is really a break through that I must be working on, now that I have the idea. Unlike what you said elsewhere, I am not spending a lot of time upfront on infrastructure. In fact, when I set out to prototype the initial design of Nuitka, it was a feasibility study only, with the purpose of demonstrating that it can be done at all and achieve real work.
Compiling huge Python programs like Mercurial correctly was a goal. Initially I set out and used C++11 and Python code templates as well as C++ template functions and classes on the backend side, and was working from bytecode on the frontend side. I had to give up on bytecode, because there was so much that it could never yield information about. And I had to give up on C++ mainly for the exception handling not performing at all, something I didn't anticipate.
I also had an sad optimization phases that was only indicating the possiblity that I will introduce proper SSA later, but really only could do peephole stuff, which since then I have replaced with that, first variable traces, more recently also value tracing.
I am normally relatvely good at leaving open spots for things, from where I can generalize, e.g. having a "CType" class for the "PyObject *" which got many friends immediately, and more interesting ones recently.
I do not mind, that only when doing it for "bool", I realized which way the API should evolve, that that actually the accessed value is its own type, and needs richer interface, than mere storage classes, i.e. "PyCellObject *" grants access to a "PyObject *" value, where more operations will happen, than ever on cells. So far, these had be all joined, but on factory it's more obvious.
The API of that, is now very ugly. I did cleanups of it, but more will be needed. In fact, there should be two different kinds of classes, and I think there is still base classes with code that is only used for one child, it's messy. But insanely powerful results. Yeah, but cleanups to do.
The issue with Nuitka and design mainly for me is that the requirements were and are largely unknown to me, until I actually do it. My goto generators approach worked out as originally designed, and that felt cool for once, but the Ctype thing is a total unknown to me. But rather I know it will evolve further if I go from "bool" (complete for 0.6.0) via "void"  (should be complete already, but enabling will happen 0.6.1 likely) to "int", not sure how long that will take.
I really think Nuitka, unlike other software that I have designed, is a prototype project that gradually turns more and more into the real thing. I have literally spent years to inject proper design in steps into optimization phase, what I call SSA, value tracing, and it is very much there now. I am probably going to spend similar amounts of time, to execute on applying type inference results to the code generation.
So I turned that into something working with strings to something working with variable declaration objects knowing their type for the goto generators, aiming at C types. All the while carrying the full weight of passing every test here is.
Then e.g. cleaning up module variables to no longer have their special branch, but a pseudo C type, that makes them like everything else. Great. But when I first introduced the new thing, I postponed that, because I could sooner apply its benefits to some things and get experience from it.
Making a whole Nuitka design upfront and then executing it, would instead give a very high probability of failing in the real world. I am therefore applying the more agile approach, where I make things work. And then continue to work while I clean it up. For every feature I added, I actively go out, and change the thing, that made it hard. Always. I think Nuitka is largely developed by cleanups and refactoring. Goto generators were a fine  example of that, solving many of the issues by injecting variable declarations objects into code generation, made it easy to indicate storage (heap or object or stack) right there.
That is not to say that Nuitka didn*t have the typical compiler design. Like parsing inputs, optimizing a tree internally, producing outputs. But that grand top level design only tells you the obvious things really and is stolen anyway from knowing similar projects like gcc.
I guess, I am an extreme programmer, or agile, or however they call it these days. At least for Nuitka. In my professional life, I have designed software for ATC on the drawing board, then in paper, and then in code, the design just worked, and got operational right after completion, which is rare I can tell you.
But the complexity of Nuitka is so dramatically higher. it is doing a complicated, i.e. detail rich work, and then it also is doing hard jobs where many things play together. And the wish to have something working before it is completed, if it ever is, makes things very different from projects I typically did. So the first version of Nuitka already had a use, and the desire was to evolve gradually.
I think I have desribed this elsewhere, but for large parts of the well or bad designed solutions of Nuitka, there is reliable ways of demonstrating it works. Far better than I have ever encountered. i believe it's the main reason I managed to get this off the ground is that. Having a test "oracle" is what makes Nuitka special, i.e. comparing to existing implementations. Like a calculator can be tested comparing it to one of the many already perfect ones out there.
So for me, Nuitka is on the design level, something that goes through iterations, discovery, prototyping, and is actually really exciting in that. Compilers typically are boring. But for Nuitka that is totally not the case, because Python is not made for it. Well, that*s technically untrue, lets say not for optimizing compilers, not for type inference, etc.
Coming back to you, and some of that content above is for TWN 6 that I am supposed to write today, you need to decide if you want to finish what you start or not. I think you are on a nice path of getting into it. But I have other PRs, where I tell people my remarks, and if they agree, they don't make a lot of arguments, they just push the fixup commit, and I merge, esp. when it's on this level of complexity.
Also I did mention that "{}".format("something") will not work on Python 2.6, which I am using it very little in Nuitka, and if I do, always like this: "{name}",format(name = "something") as that is the only way it works there too. And no, I am not interested in explaining why 2.6 is relevant still. I prefer the notation "%(name)s" % { "name" : "something" } for its flexibility, or if it's short and doesn't benefit from names, like it is the case here, or wait, it is not going to be needed at all anymore, right.
I guess I should add those things to the coding rules I guess.

As for your suggestion that I could just do it myself. Indeed. It's a mistake a made in the past and it drives away contributors. As soon as somebody touched an area of Nuitka, I was also there and did improve it myself.The majority of people then consider themselves unnecessary or bad. So I have changed my stance, and actually will try and not follow impulses to just do it myself, and let other people know, why I think it should or must be different.

That's really interesting and a great lesson to learn from. I understand what you mean here and that feeling, actually I think you're absolutely right with that position.
The fact you've thought about it speaks very nice from you, (ie: you've identified a problem that happened to you before and you have taken measurements to try to fix it and prevent it to happen again). Pointing out this cos many people don't have the slight bit of self-criticism and when they've made something "wrong" they won't try to fix it nor thinking about it not even a bit, IMHO that's really bad attitude.
For a project to become succesfull eventually you just don't need talented coders all around, you need those coders to become good mentors for the newcomers to grow up, talented coders that get people motivated.... that's the ideal.
On the other hand, while it's true in OSS there isn't anything like "managers/seniors/..." I do believe repo harcoders/authors should take this sort of responsability a little bit more serious sometimes... at least with large projects. To handle large projects you need team splitting and delegation, period.

That is not to say that I have not done it wrong. But that is never the point. The code review is for 2 people to notice things and be rather sure of the outcome.
So yeah, I am not going to do this commit, even if it's fast. The whole point of this being a community is to get more stuff done as a whole, and to learn making better changes. If I get to do cleanups on your changes each time, that is not good.

I understand and makes sense. As I told you, my git skills are really poor right now and my contribution to Open Source projects is mostly of the times about trivial stuff (don't involve really complex git workflows). In this case, nuitka is a project that has catched my attention very much and really interests me, as I've told you zillion of times I think the project itself will become eventually a really fast/stable tool people can rely on when it comes to create standalone python deployments. So improving my skills as a contributor here definitely interests me. So yeah, thanks to point this out! I do think your stance here is the most benefitial one in the long term.

And for your architectural skills, I am a skeptical thinker. The right time to believe something is when there is evidence for it. Somebody telling me he has skills, doesn't count as evidence.

Just for the record, don't get me wrong, I'm not claiming I've got architectural skills but it's more like at this moment I don't have the low-level knowledge in my head about the code to make the best decissions mostly of the time. In order to make the best decissions you really need to be aware about the existing code, so you'll avoid code duplication or hacks. Ask yourself, why newcomers add project hacks all the time when get involved on new projects? That's mainly because they don't know the project but they still need to get the job done... yet another proof than tight deadlines go in detriment of the project quality. And of course, to reach that level of awareness you need to become really familiar with the code/framework itself, which in this particular case, it's not trivial as the project is already medium/large.
Btw, I think I've already explained this in other threads but let me try again, just to be sure we're on the same page.  I'm not the type of person who likes to prove anything to anybody... I just really love coding and I love teaching from experience... but the most important, I do love to be teached and learning new stuff about almost everything.
So yeah, when I talk about stuff or giving some tips about how to improve something is just based on succesfully personal experience, that's all... you'll never hear from me the solutions I propose are the only&right way to go and actually, I do like to be proved wrong when necessary, that's the best way to learn. You don't teach a child by not correcting his bad behaviour, right? Same with us, professional developers, I think it'd be really wrong to assume we know almost everything and our ways are the only ways... out there you'll see coders with a lot of ego but I think that's just wrong. Does it make sense?
Actually, I'm quite a self-critical person and while I can't deny I've got quite the confidence with my skills mainly because I've coded since I was a little kid for more than 20 years on almost on any type of language and created almost any type of project my ego is low :)

Somebody actually applying them successfully does.

Well, you know? In an ideal world I would be able to spend all my time/efforts on hobby projects as interesting as nuitka but that's not the case and spare time is really limiting, so not sure how many evidences you'll be able to see here haha :)
If you check my github profile you'll see I haven't uploaded any of my personal projects and mostly of my contributions to OSS have been really lightweight/trivial, I have never got involved in a serious way on any github existing project tbh... and that doesn't mean I don't believe in open source philosophy but you won't see me uploading any of my private code anytime soon, why? Because OSS takes quite a lot of time and effort :/ . Dealing/interacting with people, delivering features to make people happy, answering casual users, answering casual devs, answering casual question, all of it really takes time. And don't get me wrong, I think all of it is fun & rewarding but in my case I'm dealing with dozens of active projects in a daily basis so.
For instance, take a look below how I store source code, in the others folders I keep projects from companies/forks/github/others...  in the personal one I keep all the stuff I've coded... And no, that size isn't because databases/assets/3rdparty libraries/static_files ... :/ . When I say I'm dealing with dozens of projects in a daily basis I really mean it ;D

The idea of your other PR is very interesting. It feels like it should be a best practice actually. I am pretty sure I have not seen it in Python projects I fancy. I have applied "property sheets" (that is the name I know for this) for GUIs, where these values are also dynamic. I can totally see the benefit for a CLI now having seen your example, that is great.

Kind of unrelated but you know what? I do really like Eclipse plugin framework... I've adopted some of the concepts to some of my plugin frameworks, the only bad about eclipse is the iteration is slow because the nature of the language but if you apply the same philosophy with interpreted languages like python you end up doing it the right way.
Usually I like the approach where you can run your software 0 plugins or with hundred of them. In fact, the "everything is a plugin" is great, you know why? It allows team splitting! not sure how much you're into listening podcasts but if sometime you feel like it, here's a really nice podcast to listen to. I've listened few times and it exposes a lot of interesting ideas/concepts you can learn from.

I guess I should add those things to the coding rules I guess.

Actually, that's something I've asked and I think you missed on previous threads, which python version coding style should we consider to nuitka?   2.6, 2.7, ...?
Anyway, sorry for not commenting upon your other stuff but I need to leave now, plus I guess that's content you'll add into your blog anyway so it wasn't directed specifically to me. Thanks for such a detailed explanation from yours anyway, I love those really big chunks of text :) . Nowadays people tend to write in a very twitter-way for almost everything... and while sometimes I understand the point of it (writing is hard and takes effort) I really dislike the twitter-way to communicate with people hahah and you'll see very twits from me :D . Hopefully my writing is not very wrong and boring to read, but yeah, improving my English writing skills is already on my TODO list.
Yours,
Bruno.
Ps. Oh, yeah, about your review, today I'll try to find the time to update this PR. While I was writing this I could have updated already, couldn't have? lol :D

One last question, even if it doesn't matter too much as I've already clarified on my previous post. Why do you think in this particular is better using a harcoded name for the import lib instead a more meaningful one?
I mean, for this particular case I don't see you're losing any type of information... if you decided to grep all the importlibs of your build directories you'd get the name of the build folder themselves but, answer me something. If a solution is simpler at expenses of not storing "not-useful-but-maybe-relevant-information-in-the-future-or-not" information, would be that a good thing?
Asking you cos sometimes I struggle about that... ie: Should i leave space to store metadata that can be used in the future (or not), should i add extra fields on this data structure for future usage (or not)?
Anyway, this PR should be ready to be merged, in case you find anything else to tweak please let me know y I'll update.
B.

Going to merge it the other day. I am very close to making a new release, and don't want to mess with that. But this make it for 0.6.0 and next pre-release surely.
Thanks for your contribution.
As for your question, I think "DRY" applies. Do not repeat yourself unless necessary. With dirname/dirname.ext and that file being considered for use outside of that directory ever, it would make most sense.
Other example is reponame/.git where that is not going to happen, and then you want to label it as, yeah, this is the git config for that specific folder.
In my view, the build directory is much like an inseparable something. And that .LIB and .EXT file are garbage to being with anyway. I don't see much value in keeping it at all after the linker has succeeded.

Cool, all clear.

As for your "normalize", I dislike it, and I suggest you consider input sanitizing. Every value you get from the user side, must be checked, converted, then, and exactly then, when you got it. Not any later. Randomly normalizing paths in the middle of somewhere doesn't do any good, except bug hiding potentially. I would try and not do it. That is not to say that I have not done it wrong. But that is never the point.

I think you're right here, input parsing/sanitizing/checking/conversion should be done by the first modules/plugins/subsystems so they'll provide sanitized data to the subsequent subsystems. In fact, sometimes my transformation functions tend to accept very random input data (they have weak preconditions) so they end up just doing too much, that's not good :/ . This refers to what I've explained on some previous post about newcomers providing hacks to projects they start contributing to... they want to add a particular feature but they don't want to dig further low-level to figure out what's the proper way to do it... At the end of the day, everybody like high-level abstractions without know whether there is other type of code already achieving the task, in this particular case I didn't want to read through the code to know what was exactly the meaning of those path variables used on the scons file.
Btw, let me tell you a little secret I always like to apply when coding my stuff :) ... When I code any kind of module the first thing I like to consider is this, "alright, If I had to pay some coders to improve/extend/use my stuff and they were going to charge me per hour, what'd be the cheaper way to do so....?" You know what's the answer? actually it is an answer every coder learn almost when starting to code, proper level of abstraction of code.
If there is a system composed by {A,B,C,D,E,F} and C->D and I wanted somebody else to improve C I'll just want him to know about C public interfaces... and if I want him to improve D I'll just want him to know about C&D. But definitely I've got no interest they know not a single bit about A,B,E,F (not even the public interfaces). I'll keep them like black-boxes
Alright, that's all I wanted to say, not posting anything else in this thread unless the feature itself needs more changes.

Thanks for your contribution.

No, thanks to you to listen to my non-required long and probably boring "speeches" ;D . I'll eventually would like to write a book/blog but haven't found yet the time to do so :)

@brupelo Writing a book is on my list too, so we have that in common. :)
As far as blackbox goes, the scons part is pretty well interfaced part of Nuitka with a really clear interface from our side.
And my replacement for that cost evaluation is laziness and necessity. I think a coder must be lazy, but do what it necessary, then he will automatically become good.

@brupelo Now finish the "wip" and I would love to add you for that to the contributors list. :)

I did a small amend for this one, so Python 2.6 works (I take that on my all the time), and removed the now unused normalize_path function. I hope it doesn't break from me doing it, but I am glad it will then have your name attached. ;-)

@brupelo Writing a book is on my list too, so we have that in common. :)

Cool, when you publish let me know and if the subject is something I'm interested in maybe I'll buy you 1 ;)

@brupelo Now finish the "wip" and I would love to add you for that to the contributors list. :)

Contributor list? Oh... I thought I was already added on the "most chatty contributors list" hehe ;'-)
One little aside note about me, so you know me a little bit, if some project catches my interest I've got this "habit" about becoming quite passionate about it, I guess that's pretty obvious after all text chunks I've been posting here at github the last days. Usually I don't have "middle term", either I'm quite passionate about a project or I just lose completely interest about it.
Usually I'm the type of person who likes to reinvent the wheel all the time and I like my custom solutions before considering existing tool. But if there is some piece of tech I don't fully understand that intrigues me quite a lot... and nuitka, a one-man project such as nuitka inspires me tbh. I respect people who thinks out of the box and I respect that, as I usually do that as well, so yeah, hats off.
On the other hand, I wouldn't call myself a very good team player, that's something I've realized few years ago, the good thing about that realization is thanks I've been aware about it I've tried to put quite a lot of effort to improve that "skill"... I think while being an "I can do it all by myself" type of person is quite rewarding in the long term it proves to pretty limiting and slow.
About the wip branch... hehe, yeah, when I find out the spare time budget I'll definitely come back to it, no worries ;) . One of the main reasons about that branch was I wanted to use my own clis on top of nuitka so I wouldn't have to bother you with very specific issues that maybe are interesting for some of my use-cases.
Right now one of my local branchs contains some custom options that modifies MainControl behaviour to do some custom stuff and it's getting me the job done... But of course, that's not feasible in the long term and I'd like eventually having the wip branch finished.

@brupelo One thing I meant to get back on, is how you said:

The fact you've thought about it speaks very nice from you, (ie: you've identified a problem that
happened to you before and you have taken measurements to try to fix it and prevent it to happen
again).

That is how I tend to think of me myself. I found it very nice to see that you got that. I cannot reply to all your context, but I will most often read it.
I am abusing this PR for meta communication now. I have no reason to know, but when you say, you might not stay in a project, I feel that your switch is likely more often caused by you getting on the nerves and negative feedback.
Therefore I want to be clear. I like energy, it makes for creative stuff. I even more like action derived from ideas. If you are going to do that, all fine.
I noticed one thing, and that is when 3rd parties are involved, e.g. a change suggestion that you disagree with, I am having a hard time, to want and let you speak with authority there. I read that you said, people should never take you serious. I know that attitude. It's easy and convenient, but also irresponsible. Others do not know that, and nor should they be required to.
As you noticed, flat out criticism, on spot or wrong, both don't irate me at all. However I know for a fact that people really and rightfully do not appreciate to get questions and pre-emptive answers along.
I think the right approach after asking a question in that issue, where I asked for deletion, was to wait for answers. And stating facts is OK, but making personal conclusions I don't think so.
OK: Why do you want to have this feature, potentially the solution may not satisfy you, so we are interested in known why you want this.
OK: When solved this will produce very large binaries and then it's having problems when delivering multiple binaries still. Do you do this?
I am stressed, because to a reader, it might simply discourage if you merely express disdain, because if you are going to the length of making a PR, you need to 100% know, or want to know, that it is not going to rejected, because e.g. I share your sentiments. Therefore I am forced to reject them and make it clear, and in best case, I am not only loosing time on that, but more important what would actually be a useful contribution and potentially new contributor for Nuitka to gain even more traction.
 This is never about not caring what you think. In fact I do think. For that issue however, I found your comments to be irrelevant. Occasionally that's bound to happen, isn't it?
Factually you are also incorrect for the worry about larger binaries. People deliver whole docker images these days, for trivial things. These might be well larger. Surely the ".dist" folder derived can only be smaller. And even then, people will be free to not care.
In fact, occasionally I hear complaints about ".dist" size, but rarely so. Most people seem OK, and not everybody is using PyQt, some people use this for really small stuff. The "clcache" binary for e.g. would be nice candidate.
And one thing is absolutely clear. Forever, people are going to think of compilers as capable of producing running binaries that are single files. They just are. Forever these are going to find directories cumbersome, that is bound to happen. If a subset of these people solves it eventually, and standalone was much similar, I didn't do it myself, I will gladly carry it.
But going back, I would really want to delete our comments there. They are noise. If that offends you, I am not doing it, that is why I asked you if it's OK. But I am surely unhappy and ask you to be more sure to be constructive in sharing opinions about enhancements that you don't care about.
Also I would very much like it, it actually did and went out and tried to represent the project, like with new issues as you already do. That makes me happy.
In my mind, there is only little coaching to be done. If you think this is impossible for you, I would disagree. But maybe you don't like to constrain yourself in public interactions. I would understand, you are not obliged to anything.
I occasionally try to change. I recommend trying it. :-)


Add test for empty dictionary and some comments

The empty dictionary test fails, because currently empty dictionaries are not handled.
I've not fixed anything, just added/renamed tests. Also added some comments.

closing this PR and re-opening a modified version (with fixes) - denied permission to update this branch


README: fix link

Thanks for thanking us :smile:

Oh wow.... that's quite embarrassing... I'm still quite new to markdown :)


Pass encryption key into CheckDatabaseEncryptionKey

CheckDatabaseEncryptionKey takes encryption key as a parameter. Database should be encrypted and the key somehow should be stored on computer (OS-specific key storage???).

Speaking of local storage, personally I would advice either of those: Realm, LiteDB.

Yeah we are gonna need some DB. I am up to anything really. I didn't try Realm, but it looks cool.


Functional tests may crash if created tasks use floating points

Describe the bug
Some functional tests create tasks without using the OS_FP_ENABLED flag. Depending on the platform they are running, the spawned tasks might require floating points to be enabled for the task, otherwise we get an exception.
Code snips
These two tests don't use OS_FP_ENABLED to create a task:
https://github.com/nasa/cFE/blob/c161cf04fe7c5fd2f2a533ac1d7522b31191eab6/modules/cfe_testcase/src/es_task_test.c#L159-L162
https://github.com/nasa/cFE/blob/c161cf04fe7c5fd2f2a533ac1d7522b31191eab6/modules/cfe_testcase/src/tbl_registration_test.c#L245-L248
Possible fix is to add OS_FP_ENABLED to the function call.
es_task_test.c:
    /* Now do the same but instead of a CFE child task, make an OSAL task that is not associated with a CFE app */
    UtAssert_INT32_EQ(OS_TaskCreate(&OtherTaskId, "NonCfe", TaskFunctionCheckNonAppContext, OSAL_TASK_STACK_ALLOCATE,
                                    4096, OSAL_PRIORITY_C(200), OS_FP_ENABLED),
                      OS_SUCCESS);

tbl_registration_test.c:
    /* Create a separate task to run the tests, to confirm TBL context checks */
    UtAssert_INT32_EQ(OS_TaskCreate(&OtherTaskId, "NonCfe", TestTblNonAppContext, OSAL_TASK_STACK_ALLOCATE, 16384,
                                    OSAL_PRIORITY_C(200), OS_FP_ENABLED),
                      OS_SUCCESS);

System observed on:

Hardware: DLEON processor
OS: RTEMS
Versions: cFS Bundle Caelum RC4

Reporter Info
Jose F. Martinez Pedraza / NASA GSFC

Sounds to me like you've encountered a platform that requires floating point to be enabled for all tasks, regardless of if the task requires floating point support.  In this case for portability it should be added from the PSP, not from cFE.

Closing this since there's no plan to add the floating point flag for all tasks to fix this bug at the system level.  Systems that rely on it can do it from the PSP.


WIP: Make HttpNetworkSender mockable and add unit test

Part of #3181
todo

[x] split files
[x] maybe remove some mocks and replace them with NSubsitute
[x] cleanup
[x] check coverage

looks too good to be true:


feat: UniversalProposal for produceBlockV3

This introduces UniversalProposal to support produceBlockV3 endpoint.
When I created the PR, I realized there is another one for the same purpose: https://github.com/attestantio/go-eth2-client/pull/107 done by @y0sher just a couple days ago.
In my implementation I did not deprecate the v2 endpoints, I believe they may co-exist safely in go-eth2-client project.
If you want to proceed with the other implementation, that's fine, just leave a note in the comment so I can close the PR.
cc @mcdee

Hi thanks for the PR.  Internally we have been working on unifying the underlying blinded and non-blinded proposal flows, and have an alternate pr #116
Please could you take a look at this and let me know if it meets your requirements?  Thank you.

Please could you take a look at this and let me know if it meets your requirements? Thank you.

Thank you @mcdee !
I quickly checked the changes, it will certainly cause bunch of changes on our side, because of the breaking changes.
If you are certain about these changes will go master soon or late, then I can start migrating our code on this week.

Closed in favor of @mcdee 's implementation.


Update hardening guide links with broken redirects

While testing a build, I noticed that the self-assessment guide redirects weren't resolving from pages-for-subheaders/rancher-hardening-guides.md. I updated the links on that page to point to the current names for those pages. When I checked the links in docusaurus.config, it seemed like the redirects were formatted correctly and pointed to the right paths. I know we had a similar issue some months ago with other redirects. Updating the links on pages-for-subheaders/rancher-hardening-guides.md is a quick fix -- we still need to look into why the redirects weren't resolving.

Well...according to https://github.com/rancher/rancher-docs/issues/463#issuecomment-1494716941, the hardening guide pages have a known issue where periods within file names are incompatible with the redirects format. So, updating the links on the pages instead of  redirecting is the way to go.


Add Hungarian localisation

I have 8 years of localisation experience, but this is the first time I've tackled a social network, so feedback is very welcome.
I also have some questions/notes:

Is there a way to get notified of any changes before build-time? I would like to avoid the localisation always being one version behind, with all the new text left untranslated, if possible.
I have removed the special plural formatting, as Hungarian doesn't use plurals when counting. I hope this is acceptable.
I have translated all ‘# minutes,’ ‘# hours,’ etc. strings as ‘# minutes ago,’ ‘# hours ago,‘ etc., since these alternate forms are achieved with suffixes in Hungarian and it would be impossible to separate them. Are they used in any other way? If so, would it be possible to make unique strings for other cases?
Will there be a possibility to translate Bsky-associated content, like the name/description and (most importantly) categories in the Bluesky Moderation Service, as well as the names/descriptions of first-party feeds?

Thank you in advance.

It works well for me.

It seems that after removing the plural, like time and others work properly.

You can safely translate terms like “hours” to “hours ago” since, as far as I know, there aren’t other usages. I did the same in Chinese translations as well.

Update the translation strings by running yarn intl:extract --locale hu each time manually, then you can get translation updates instantly. By the way, usually, there’s no need to translate strings starting with #~ as they have already been deprecated.

Thank you very much for your feedback and your screenshot – I've made some small tweaks thanks to the latter.
Where and when exactly should I run the yarn intl:extract --locale hu command, if I may ask?

Hmm, I've just looked closer at your screenshot and noticed that the ‘Liked by # users’ string did actually break. I wonder how to fix it…

Thank you very much for your feedback and your screenshot – I've made some small tweaks thanks to the latter.
Where and when exactly should I run the yarn intl:extract --locale hu command, if I may ask?

First need to set up a local build environment, including tools like Node.js (following the build instructions).
Typically, if you only need to test translations on the web, just need Node.js and Python, and make sure to add them to your system’s PATH.
After that, run yarn install and wait for a bit—long enough to enjoy a cup of coffee. Once the local setup is ready, execute yarn intl:extract --locale hu, and you’ll see the output like this:

It will update the messages.po file, and then you can use an appropriate tool to fill in any missing translation strings.
After completing the translation of the messages.po file, you can run yarn intl:compile to generate the correct target translation files. Then, execute yarn web to see the results.
That's it. Good luck and wishing you all the best.

Thank you very much.
In the meantime, I believe to have deciphered the proper formatting of the plural strings and have applied a possible fix.

Thank you very much!

Honestly, adding a space was just a stylistic choice at the time, but now I've checked Twitter and Facebook and it turns out they both do use spaces in Hungarian as well. The addition of non-breaking spaces is a good idea, though. Thank you for that.

This looks like a great effort, nice work! 👌
I see that @auroursa has already been very helpful in answering questions and giving feedback :)
I just wanted to note a few other things:

Will there be a possibility to translate Bsky-associated content, like the name/description and (most importantly) categories in the Bluesky Moderation Service...

It looks like you've translated the four "global" content labels like Adult Content, Sexually Suggestive, etc, so if you're asking if you can translate all the other labels on the profile page for the Bluesky Moderation Service: unfortunately, that isn't possible yet.

...as well as the names/descriptions of first-party feeds?

Disappointing answer here too, I'm afraid, as Discover and Following are currently hardcoded in English.
And finally, one important thing to mention is that the devs usually ask to have at least one native speaker review a localization before merging it. Would it be possible to ask another Hungarian speaker to do that review?
Once they've done their review, my completely unofficial opinion as a fellow translator is that this PR would look in a good position to be merged 🚀

Thank you so much. I've asked my friend, @SonoSooS to help with proofreading – he should report back soon™.

Thanks @surfdude29 for the additional information. I feel frustrated about the inability to translate "Following," "Discover," and the Bluesky Moderation Service list, as this has been an issue affecting the Chinese translation as well.
But hey yes, I would say that technically this PR looks ready to merge🤗
Look forward to other Hungarian native speakers helping to refine the translation. You're doing a great job!

We went through it with @smileyhead, and correct some things that - while correct - felt really unnatural, like short "ago" units for example. Those may work in other languages, but not here. Otherwise yeah, he did a fantastic job, as usual.
However, there are some "dialect" issues that we couldn't agree on, so I'll list them here, just in case stylistic "overrides" are preferred over language correctness.

pagination order, forwards "2/3" ("2nd page out of 3" (basically the same word order as in Hungarian)) vs. backwards "3/2" ("2nd out of 3 pages" (the spoken order is with the 3 first, then 2, so this one rolls off the tongue)). This one is very much so a mixed bag. Some people prefer "3/2", and others (like myself) utterly despise it, and prefer the "2/3" one. He even asked linguistics, and they said, that there is no rule for this. Media uses the backwards format, but most (but not all) people that I know around me use the forwards format. A lot of people will be displeased, no matter which one gets hardcoded.
couldn't agree on whether to use "alsókötőjel" or "alulvonás" for the underscore (_). I have never heard of anyone using "alsókötőjel", and I've been from barely inhabited remote villages with "dialects" almost impossible to understand, to big cities, and I have never in my life heard the word "alsókötőjel", everyone I have ever met calls it an "alulvonás". However, according to Wikipedia, it is officially called "alsókötőjel", but in everyday language we say "alulvonás" for sure.
this is not a problem with his translation, but there is no thousands separator for big numbers (for example 5-digit follower count). The official thousands separator here is the dot (.), but basically every website uses space ( ) as a thousands separator instead. Personally I prefer the comma (,) for readability though, but then there is a possibility of people confusing it for a decimal separator, so it sadly can not be used.

We also found a bunch of usability issues, but those have nothing to do with this PR, I'll report those separately.

The official thousands separator here is the dot (.), but basically every website uses space ( ) as a thousands separator instead.

Correction: The main thousands separator is a space, and the second option is a dot (and we use a decimal comma).
So, yeah, a long number with digits would look like this: 12 345. But for some reason, the site displays them as 12345.


Reorganize files, remove patch generation

Removes "support" files: The -support files were born when we were trying to use export default for everything. Now that we've done away with that, they don't make any sense.
Removes "static" files: Again, because we're not default exporting anything any more, we can have the static implementations live along side their member counterparts.
Moves "-static" patch files: Takes all the of the patch files like add/operator/zip-static and moves them to a more appropriate add/observable/zip.
Removes patch generator: We don't need to generate patch files anymore. It's something we can maintain manually, as it isn't a lot of work, and we do have special cases from time to time.
Renames observable class files. Since we no longer have a patch generator that is using file names to create other files, we can name the Observable files something sane again. ArrayObservable is once again under ArrayObservable.ts and not fromArray.ts.
Renames InfiniteObservable to NeverObservable: This is a non-breaking change as we don't export this type, per say. It was done because the name is more descriptive and, like other Observable subclasses, it now aligns with the method associated with it.

LGTM.


Make useHandleRequest return response and make success callback optional

This is to make useHandleRequest more flexible. With this change, the success function will return the response of the request. Also, the success function is optional now.
You can now pass the request parameters to the onRequest function, which will also return the response on success, or undefined on error.
Here's an example (same as in Storybook):
import { useHandleRequest } from '@aboutbits/react-ui'
import axios, { AxiosResponse } from 'axios'

const validate = (formData: FormData) => {
  return axios.post<void, AxiosResponse<SuccessResponse>>(
    'http://localhost:8080/validate-form',
    formData
  )
}

const { apiErrorMessage, isRequesting, onRequest } = useHandleRequest(validate)

// ...

const result = await onRequest(formData)

if (result === undefined) {
  console.log('Error')
} else {
  console.log('Success', result.data) // result is of type AxiosResponse<SuccessResponse>
}

SWR returns the response on success and undefined on error. But it also has the option to throw the error if it fails. I can use this approach too.
Note:
This is something I wanted to use for one of our projects but turns out it does not work as expected in that case :/ I now adapted the useHandleFormSubmission hook which seems to be better for my use case (PR incoming). But this change is somewhere related, and I think it would be nice to have it anyway. Wdyt?

This is the related PR for useHandleFormSubmission: #189

@mmalfertheiner I'v thought about it and I think SWR's approach is cleaner, since it makes sense to throw an error than to return it. But I'll wait for your feedback before changing it.

See https://github.com/aboutbits/react-ui/pull/189 for the reference implementation

@lukasvice May we close this PR or at least mark it as a draft? The best implementation is currently at RIS React UI and we look at that when updating React UI.

@devgioele Yes, let's close it. We should reconsider this when migrating the changes from RIS.


Add how to upgrade CLI with homebrew

https://github.com/iasql/docs/issues/17 to keep track of other OSes


Downloading model every test.

I noticed that in every test that I run the model is downloaded again. This doesn't happen when running the OCR, only when testing, that's a factor that impacts directly in execution time when testing.

@Lucs1590 I saw this is a test, called "test_get_model", I think it's better to let it without any test because the model has almost 97MB to download every execution.

Right, the outcome of this Issue was to divide the tests into folders:

integration;
unit;
fixtures.

And in order not to download an EAST model in all executions, it is enough to standardize the execution of the tests in the unit folder.
To execute via command line, use:
python3 -m unittest discover -s tests/unit

The integration tests will be performed on the GitHub pipelines, but it is worth saying that good practice is always to have tests.
The separation of the tests was done following these criteria:

Calling an HTTP REST API;
Calling a Python API;
Calling a web service;
Use model, ANN or similar.

For more details, go to: https://realpython.com/python-testing/#writing-integration-tests


Missing namespace in v1beta1 upgrade guide

Description
In https://deploy-preview-4749--karpenter-docs-prod.netlify.app/preview/upgrade/upgrade-guide/#prerequisites
The sample command is missing the namespace name:
kubectl describe pod -n karpenter-xxxxxxxxxx-xxxxx | grep Image: | grep v0.....
Should be:
kubectl describe pod -n karpenter karpenter-xxxxxxxxxx-xxxxx | grep Image: | grep v0.....

Fixed in PR #4749.


spring-boot-dependencies 2.6.0 doesn't show any content

What are you trying to do?
I searched for spring-boot-dependencies and clicked on the 2.6.0 release, ending on this empty page:
https://search.maven.org/artifact/org.springframework.boot/spring-boot-dependencies/2.6.0/pom
A click on the browse button in the upper right corner works, the download button is broken though.
Another thing that troubles here is that when you switch to a different release, say 2.5.7 and switch back to 2.6.0 again, there is the pom from 2.5.7 shown for the 2.6.0 release.
You can find a short discussion in this topic here already:
https://github.com/spring-projects/spring-boot/issues/28698

@hennr thank you for creating this. It appears that the underlying BE services is unable to retrieve the correct data for version 2.6.0, i'll checkin with our developers on that.
Additionally I think we can/should handle that a bit more gracefully on the UI when we can retrieve it. I'll take a look at that as well (likely not as part of this issue).

Thanks, this is good news.
I'll stay tuned!
Am 25. November 2021 16:56:43 MEZ schrieb Moncef Ben-Soula @.>:
@. thank you for creating this. It appears that the underlying BE services is unable to retrieve the correct data for version 2.6.0, i'll checkin with our developers on that.

Additionally I think we can/should handle that a bit more gracefully on the UI when we can retrieve it. I'll take a look at that as well (likely not as part of this issue).
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/sonatype-nexus-community/search-maven-org/issues/159#issuecomment-979322189
--
Von unterwegs, deshalb etwas kürzer.

@hennr hope you had a good weekend.
We found the cause, we were applying some aggressive cache for the 404 returned on retrieving the data. We busted the cache, and all should be good again, which my testing confirmed.
Hope you have a great day!

Hi @SirMaster, same issue for spring-boot 2.6.2: https://search.maven.org/artifact/org.springframework.boot/spring-boot-dependencies/2.6.2/pom
Could you check your cache again?

@hennr Thanks for the report.  We've purged our caches for that particular artifact.  The details should now show up correctly in the search results.  We will need to do some digging on our side to see why this seems to keep happening.

Hello @jorlina
the problem seems to happen again for spring-boot 2.6.6
The search does not show the new version as well (yet?)
https://search.maven.org/artifact/org.springframework.boot/spring-boot-dependencies

It took a while but 2.6.6 should be there now.


Question/Feature request: Any way to do required_ifs "flags exists".

 Issue by 8176135
Tuesday Dec 12, 2017 at 04:54 GMT
Originally opened as https://github.com/clap-rs/clap/issues/1126

Clap version: 2.29.0
I'll start with the problem I am trying to solve:
I have the flags -A, -B, -C, which are all optional by themselves. However, if both -A, and -B are used, -C must also be used. (-C takes some values that -A and -B need)

Clap has required_unless_all(name), but doesn't have the opposite of that e.g. required_if_all(name)?

I know clap has required_ifs(), but that compares the input from an argument with a value, but the value doesn't matter in this case, only that the flags are present or not.

Having requires() on -A and -B also won't work in this case, as only having one of them will make -C required.

requires_all() does the opposite of what I want, requiring both -A and -B to be used -C can be used

None of the above is unexpected, (the documentation is really good at explaining each option 👍 ), but there doesn't seem to be a required_if_all([names]). Any ideas?

 Comment by kbknapp
Tuesday Jan 09, 2018 at 16:51 GMT

I'm not opposed to adding something like that, however I'll explain why it doesn't exist (yet).
When dealing with flags and requirements, since they are a simple on/off switch, if you know -A and -B requires -C (as the programmer), you could just act as if -C was used. I'd recommend telling the user via a help message or something, ("using -A and -B together implies -C", etc.).
From a UX standpoint, the user isn't supplying any additional information by being forced to pass -C since it's a flag. The only thing they're saying is essentially, "I know the -C switch is required."
The only time requiring an additional flag is a good idea is when the task is destructive (like deleting, etc.) and the user needs to confirm that they're sure they know what they're doing (by including the flag) such as --force usually, or -D in git branch.
However, in these circumstances it's far better to have a custom error message telling the user what is about to happen and why the extra flag is required. A generic clap error of, "-C is required but wasn't used" could actually be harmful here, since the user will almost always just blindly copy the -C argument and not know why.

 Comment by kbknapp
Sunday Jul 22, 2018 at 02:49 GMT

Closing for now to get 3.x out the door. We can re-address after release if needed.

 Comment by CreepySkeleton
Sunday Feb 02, 2020 at 02:18 GMT

I'm leaving it closed for the time being


Dependabot reports for potential security vulnerabilities

https://github.com/nspcc-dev/neo-go/security/dependabot

I know about it, but I'm not sure we have an actual problem here, because originally it's for a different version of the package we use. But it needs to be checked.

I believe the main actual problem is in the banner itself, as github is the front page for our code.

It's not visible to outsiders.

I've just looked at the code, and it seems like it affects us. As well as go-yaml/yaml#665, so we need to move on with #2085.


Inconsistent naming get_json() in EventGridEvent and EventGridOutputEvent

Both mentioned classes have methods called get_json which return the classes __data field. This seems odd for two reasons:

They don't actually return JSON but a dict (if the typing hints in __init__ are correct). In any case this is a major source of confusion.
All other getters are implemented as properties. (So why not just do the same here?)

https://github.com/Azure/azure-functions-python-library/blob/44712a77ce6ba68065d1c16ed6015bb998ddfe48/azure/functions/_eventgrid.py#L33-L34
https://github.com/Azure/azure-functions-python-library/blob/44712a77ce6ba68065d1c16ed6015bb998ddfe48/azure/functions/_eventgrid.py#L86-L87

Hi, I corrected this problem, thanks


user_name: Prevent users from setting forbidden format.

https://github.com/zulip/zulip/issues/13923.
Preventing users from setting their usernames in the format "Alice|999".
EDIT: The regex finally decided upon and used is r"\|\d+$".

Perhaps the regex should be relaxed from ^\w+\|+\d to \|+\d$? With the current regex you won't be able to set a name with any spaces or special characters.

@pemontto Thanks! Fixed it.

As you're not matching the whole string now you'll probably want to use re.search instead.

@timabbott A review please

LGTM.  Tim, you may want to tweak the error message here to be something like "Invalid format!  Please do not end your name with a pipe and digits.".  I don't think the message needs to be super clear; as long as it hints toward the problem, we should be ok.

@timabbott A review please. It's a small one.

Merged as 43e39718c429900f92de54193894aa289c5b9543, after fixing several things:

The commit message didn't close the issue
The commit message was a bit unclear about the issue
Added a comment documenting this more unusual element.
Fixed the import re placement.

@Abhishek-Balaji I'd encourage you to spend more time on these details.  If you don't, even a "small PR" like this one is actually quite time-consuming to review and merge.


Reduce runtime dependencies

Runtime dependency count is decently low already, but a few more can be cut out with minimal changes.

Hi! I completely missed that and actually did exactly this a few minutes ago myself!
Maybe I saw your PR and did it unconsciously...
Anyway, thanks a lot for your contribution and sorry to not merge this!

Hi! I completely missed that and actually did exactly this a few minutes ago myself! Maybe I saw your PR and did it unconsciously...
Anyway, thanks a lot for your contribution and sorry to not merge this!

I see you removed can-use-dom from simplebar-core. Glad to see that. There are a few things this PR was doing that didn't get implemented though, so I opened another PR to handle the remaining stuff.


Watch a key should wait forever!

Watch a key should wait forever Not raise timeout.
n [7]: c.read('/users/created/callback', wait=True)
---------------------------------------------------------------------------
timeout                                   Traceback (most recent call last)
/home/wiliam/.virtualenvs/echo/lib/python3.4/site-packages/urllib3/response.py in read(self, amt, decode_content, cache_content)
    238                     # cStringIO doesn't like amt=None
--> 239                     data = self._fp.read()
    240                     flush_decoder = True

/home/wiliam/.pyenv/versions/3.4.3/lib/python3.4/http/client.py in read(self, amt)
    505             if self.chunked:
--> 506                 return self._readall_chunked()
    507

/home/wiliam/.pyenv/versions/3.4.3/lib/python3.4/http/client.py in _readall_chunked(self)
    586                 try:
--> 587                     chunk_left = self._read_next_chunk_size()
    588                     if chunk_left == 0:

/home/wiliam/.pyenv/versions/3.4.3/lib/python3.4/http/client.py in _read_next_chunk_size(self)
    551         # Read the next chunk size from the file
--> 552         line = self.fp.readline(_MAXLINE + 1)
    553         if len(line) > _MAXLINE:

/home/wiliam/.pyenv/versions/3.4.3/lib/python3.4/socket.py in readinto(self, b)
    373             try:
--> 374                 return self._sock.recv_into(b)
    375             except timeout:

timeout: timed out

During handling of the above exception, another exception occurred:

ReadTimeoutError                          Traceback (most recent call last)
/home/wiliam/.virtualenvs/echo/lib/python3.4/site-packages/etcd/client.py in _result_from_response(self, response)
    523         try:
--> 524             res = json.loads(response.data.decode('utf-8'))
    525             r = etcd.EtcdResult(**res)

/home/wiliam/.virtualenvs/echo/lib/python3.4/site-packages/urllib3/response.py in data(self)
    165         if self._fp:
--> 166             return self.read(cache_content=True)
    167

/home/wiliam/.virtualenvs/echo/lib/python3.4/site-packages/urllib3/response.py in read(self, amt, decode_content, cache_content)
    257                 # there is yet no clean way to get at it from this context.
--> 258                 raise ReadTimeoutError(self._pool, None, 'Read timed out.')
    259

ReadTimeoutError: HTTPConnectionPool(host='<IP_ADDRESS>', port=4001): Read timed out.

During handling of the above exception, another exception occurred:

EtcdException                             Traceback (most recent call last)
<ipython-input-7-47b4a8c29981> in <module>()
----> 1 c.read('/users/created/callback', wait=True)

/home/wiliam/.virtualenvs/echo/lib/python3.4/site-packages/etcd/client.py in read(self, key, **kwdargs)
    346         response = self.api_execute(
    347             self.key_endpoint + key, self._MGET, params=params, timeout=timeout)
--> 348         return self._result_from_response(response)
    349
    350     def delete(self, key, recursive=None, dir=None, **kwdargs):

/home/wiliam/.virtualenvs/echo/lib/python3.4/site-packages/etcd/client.py in _result_from_response(self, response)
    530         except Exception as e:
    531             raise etcd.EtcdException(
--> 532                 'Unable to decode server response: %s' % e)
    533
    534     def _next_server(self):

EtcdException: Unable to decode server response: HTTPConnectionPool(host='<IP_ADDRESS>', port=4001): Read timed out.

In [8]:

Can I see some code to reproduce the issue?
Are you using the latest version of python-etcd from github? I solved a similar issue in the past.

Completing infos:
python-etcd==0.3.3
Example code:
$ ipython
Python 3.4.3 (default, Jun 28 2015, 22:23:43)
...
In [1]: import etcd
In [2]: client = etcd.Client()
In [3]: client.read('/nodes/n2', wait=True)
-------------------------------------------------------------------
timeout                                   Traceback (most recent call last)
/home/wiliam/.virtualenvs/echo/lib/python3.4/site-packages/urllib3/response.py in read(self, amt, decode_content, cache_content)
    238                     # cStringIO doesn't like amt=None
--> 239                     data = self._fp.read()
    240                     flush_decoder = True

/home/wiliam/.pyenv/versions/3.4.3/lib/python3.4/http/client.py in read(self, amt)
    505             if self.chunked:
--> 506                 return self._readall_chunked()
    507

/home/wiliam/.pyenv/versions/3.4.3/lib/python3.4/http/client.py in _readall_chunked(self)
    586                 try:
--> 587                     chunk_left = self._read_next_chunk_size()
    588                     if chunk_left == 0:

/home/wiliam/.pyenv/versions/3.4.3/lib/python3.4/http/client.py in _read_next_chunk_size(self)
    551         # Read the next chunk size from the file
--> 552         line = self.fp.readline(_MAXLINE + 1)
    553         if len(line) > _MAXLINE:

/home/wiliam/.pyenv/versions/3.4.3/lib/python3.4/socket.py in readinto(self, b)
    373             try:
--> 374                 return self._sock.recv_into(b)
    375             except timeout:

timeout: timed out

During handling of the above exception, another exception occurred:

ReadTimeoutError                          Traceback (most recent call last)
/home/wiliam/.virtualenvs/echo/lib/python3.4/site-packages/etcd/client.py in _result_from_response(self, response)
    523         try:
--> 524             res = json.loads(response.data.decode('utf-8'))
    525             r = etcd.EtcdResult(**res)

/home/wiliam/.virtualenvs/echo/lib/python3.4/site-packages/urllib3/response.py in data(self)
    165         if self._fp:
--> 166             return self.read(cache_content=True)
    167

/home/wiliam/.virtualenvs/echo/lib/python3.4/site-packages/urllib3/response.py in read(self, amt, decode_content, cache_content)
    257                 # there is yet no clean way to get at it from this context.
--> 258                 raise ReadTimeoutError(self._pool, None, 'Read timed out.')
    259

ReadTimeoutError: HTTPConnectionPool(host='<IP_ADDRESS>', port=4001): Read timed out.

During handling of the above exception, another exception occurred:

EtcdException                             Traceback (most recent call last)
<ipython-input-3-8f72634683fa> in <module>()
----> 1 client.read('/nodes/n2', wait=True)

/home/wiliam/.virtualenvs/echo/lib/python3.4/site-packages/etcd/client.py in read(self, key, **kwdargs)
    346         response = self.api_execute(
    347             self.key_endpoint + key, self._MGET, params=params, timeout=timeout)
--> 348         return self._result_from_response(response)
    349
    350     def delete(self, key, recursive=None, dir=None, **kwdargs):

/home/wiliam/.virtualenvs/echo/lib/python3.4/site-packages/etcd/client.py in _result_from_response(self, response)
    530         except Exception as e:
    531             raise etcd.EtcdException(
--> 532                 'Unable to decode server response: %s' % e)
    533
    534     def _next_server(self):

EtcdException: Unable to decode server response: HTTPConnectionPool(host='<IP_ADDRESS>', port=4001): Read timed out.

In [4]:

@wiliamsouza have you tried passing a timeout of 0 to the read instruction?
c = etcd.Client()
c.read('/some-key', wait=True, timeout=0)

This works with the latest git version.

@lavagetto Thanks!
This works with 0.3.3 too!
I would like to suggest two possible improvements:

Document the timeout=0 as a way to disable cascading timeout.
Change the code to wait=True implies timeout=0.


Show database error information

If database reports error while executing query, this information is not shown in UI. So they have to switch browser to developer mode to view error description

can't reproduce neither on site nor embedded version (main)

better diagnostics requires #8294

the problem was because of absence of compute nodes in serverless database. it was fixed by using shared database nodes.


Ignore EINTR while waiting for the emulator

pebble install --emulator basalt crashes with unhandled EINTR when waiting for the emulator.
This ignores the interruption and resumes waiting, the app then installs fine.

Thanks! Sorry for the delay getting this merged.


Long time eco system strategy

Open for discussion:

Continue to increase library count to create a position of strength
Create a documentation page that shows library conformance status with the vcpkg proposed packaging rules
Cooperate with cmake team to push features for better integration into an package management system
Community clean up builds and try to push changes upstream. Use position of strength to 'blaim' non-conformant behavior
Find a proper way to early embed module support, to prevent competitor managers that could split the community
Continue to evolve CI, port tests, and custom checkers to find remaining issues
At this stage it might become feasible to get the issue count low (Since January it seems to be already stabilized after continuously growing over the last year. Great work!) But still a frighteningly large amount of work is wasted on fixing recurrent issues.
With all libraries in one place, cross platform support and a working tool-chain infrastructure, Conan could start to cache vcpkg build results, leading to an unified infrastructure for source build, and binary distributions. (Assuming vs tooling supports all major three compilers at that time)
With all libraries in one place, community and SG15 can focus on analyzing the required set of features, to either evolve cmake or to create a sane replacement
Cooperate with cmake to enable auto translation/migration to the new system.
Declare the new build system and vcpkg/conan duo as the de facto standard for c++ build and package management

In my opinion, the Standard C++ Foundation should get more involved into these problems. I would not mind if they provided a build-system that works on all relevant platforms and a package manager implementation. The C++ community would hugely benefit from having one official way for building stuff and getting packages.

I fear this will not happen. As an observer I would say C++ ISO is mainly influenced by the large players that have their specialized build solutions that are all incompatible. Also getting things into ISO is complicated because it needs to be fine for all platforms and beyond. Not just the relevant ones. So a 98% solution is not acceptable.
My remaining hope is that the community can come up with a system (for the relevant platforms) that is de-facto standard due to its widespread adoption. Vcpgk is by far the most realistic approach towards this status.
Also the new SG15 tooling group looks promising. I hope someone from the vcpkg core team is attending at the meetings.

@Fleischner I believe we're already doing pretty much everything you've enumerated here; I don't really see an "issue" though. Is there concrete action you're suggesting?

@Fleischner, let us know if you have more suggestions, thanks for your contribution!


Circular json reference wasnt completaly resolved

Hi,
Based on the changes here https://github.com/SAP/ui5-inspector/pull/94 still the issue persists when you use SmartLink control. The complete Properties panel dissapear in case of sap.ui.comp.navpopover.SmartLink.
I resolved the issue with circular dependencies by adjusting _prepareMessage method, setting the current.target[key] = null in case of circular reference is found. I dont understand why we should use an existing dependency in the first place --> This may be the root of the new circular reference, because it may re-introduce  a circular dependency. And anyway you dont really care for the information so deep into the JSON.
Once I did the above, the circular dependency disappeared, but I was playing around with a Fiori Application Smart Template List Report and the performance became a disaster. The _prepareMessage is very slow and makes the whole experience very bad. I would suggest adding an extra parameter called "level" (like the node.js util.inspect method) where you can determine in what kind of depth should the reading of properties stop. I think is a waste of time reading the complete depth, especially in aggregation, association object, where the depth can become very deep to handle.
Let me know if I can help you with something!

@gtreskas As the author of the referenced fix I'm genuinly baffled about basically every sentence I just read here 😕

Based on the changes here #94 still the issue persists when you use SmartLink control. The complete Properties panel dissapear in case of sap.ui.comp.navpopover.SmartLink.

No, this is a different issue. It happens while executing port.postMessage(detectEvent.detail). Circular references are handled by its structured clone algorithm. Therefore having those is not an issue here - except there is actually a bug in this algorithm (sounds quite unlikely, but it is still human written code after all).

I resolved the issue with circular ~dependencies~ references by adjusting _prepareMessage method, setting the current.target[key] = null in case of circular reference is found

This portion of code does not find/handle circular references explicitely, it just re-uses already parsed references - circular or not. And with this, it also resolves circular references properly. So you did not "resolve the issue", you introduced a bug by altering the result.

I dont understand why we should use an existing ~dependency~ references in the first place --> This may be the root of the new circular reference, because it may re-introduce a circular ~dependency~ reference.

Cause the aim is on purpose to keep the data as it is, not getting rid of any references. Resolving circular dependencies does not mean removing them but handle them properly while removing methods from the structure (which is btw the one and only reason why the message is parsed in the first place).

Once I did the above, the circular dependency disappeared, but I was playing around with a Fiori Application Smart Template List Report and the performance became a disaster. The _prepareMessage is very slow and makes the whole experience very bad.

Some simple measurements confirm my suspicion: The _prepareMessage implementation is actually faster than the previous used JSON.parse(JSON.stringify(object)) coding. Do you have other results?
The only "disaster" in performance I did notice, is when clicking on a sap.ui.comp.navpopover.SmartLink node in the control-tree and its message is parsed async in background. I don't know what is wrong with the sap.ui.comp.navpopover.SmartLink control, but the retrieved data for it is HUGE (which is still not the root cause here but responsible for the bad perceived performance when further navigating within the control tree).
Of course, this is performance wise worse than a directly failing JSON.stringify. I really hope nobody is surprised here: Properly parsing an object is slower than not parsing an object at all. (Konfuzius) 🙈

I would suggest adding an extra parameter called "level" (like the node.js util.inspect method) where you can determine in what kind of depth should the reading of properties stop. I think is a waste of time reading the complete depth, especially in aggregation, association object, where the depth can become very deep to handle.

The data that is parsed is not only the control/model information, but also e.g. the control tree. For each hierarchy level, the datastructure requires 2 "levels" within the provided message (plus some additional fixed structural overhead). So a control tree of e.g. depth 20 requires a minimum(!) of "level" 40 to be parsed in _prepareMessage. The issue with sap.ui.comp.navpopover.SmartLink happens already with a max-level of 6 (in words: "six").
Even though an "infinite" depth parsing of the control- or model-data is rarely (to never) required, it would not resolve this particular issue, and would furthermore only be useful in the same amount of rare (to none) cases. And it could cause missing information e.g. in the control tree.

Let me know if I can help you with something!

Lets find the actual root cause, then we can make informed decisions based on it.
And for that, I would start at where the issue actually occurs: The postMessage method of the chrome.runtime Port - what are its limitations, what are potential faulty inputs, what are known issues, checking bug reports etc.
And from there, check the actual input of the call in case serializing sap.ui.comp.navpopover.SmartLink control information. As mentioned, the input can be stripped down to depth level 6 in the _prepareMessage method.
THE END

@SeeSharpSoft : Thanks for the detail explanation, well it may be I was working too late yesterday and I miss to state the facts in the right way, sorry don't get offended or something from the issue here, I m sure you and your team are doing great job here.
So lets just go to the concrete case and try to reproduce the issue and I can show you on my screen how you can reproduce the issue or setup a call and have a look together.
The actual root cause is a circular reference and there was a comment on your code,, which was stating that you are handling it through copying an existing structure, but you are right, maybe I missunderstood your code. Currently I build a small fallback method in my fork which  doesnt resolve the problem directly, but at least it works for now.
Let me know concretaly how to proceed?
Best Regards

@gtreskas Just to clarify: I am a returning, but still individual contributor to this project. Not part of the team, not having any more saying in this than you.
My lengthy response is about making informed decisions based on a proper root cause analysis, instead of trying to patch its effects. Those patches often causes additional issues that are then patched again etc. I am an advocate of the saying: "If you have one hour to solve a problem, then spent 50min on the problem and 10min on the solution, not vice versa!"
Don't get me wrong: I do acknowledge the issue and can reproduce it fairly easy on the sap.ui.comp.navpopover.SmartLink control - but I myself didn't figure its actual root cause yet. I do not know the problem well enough to start talking about a solution.

The actual root cause is a circular reference

If you found the root cause - awesome!
So I assume you are refering to a specific circular reference then (as in general, circular references shouldn't be an issue). So here are my thoughts:

Why is this particular circular reference an issue while others aren't?
And with respect to the used postMessage method, its provided API and documented restrictions: Shouldn't it be able to handle this circular reference?

If so, it is a bug/issue in Chrome/V8 and should be reported.
If not, then how to detect/prevent such a specific circular reference? (here finally comes the actual fix into play)

@gtreskas Seems that I finally reached the state you were already in for a bit (I apologize), realizing circular references in general are indeed a problem for port.postMessage method, which is a different one than mentioned here.
I also noticed this on additional controls, e.g. sap.ushell.components.container.ApplicationContainer. There are references to the sap.ui.Core instance within its control data 😮
Possible solutions

Ignore generally already processed references (what @gtreskas mentioned in the first post): This does not only prevent circular references but also non-circular references to already processed parts.
Detecting and preventing only circular dependencies: Probably decreases the performance (notably?). Is it sufficient?
Ignoring controls/messages with a circular dependeny (basically catching JSON.stringify): User might miss information.

In all three cases, data gets potentially lost, which seems inevitable. Or do I miss something? Is there another solution? What is most favorable?

@SeeSharpSoft :Yes thank you very much for taking your time and having a look, and stating the facts. Yes you are right is inevitable to keep everything, we have to loose something here, but in this case we should see if what we loose is ok or not. I think the best would be to understand what is the data that we are loosing and if its important for the inspector...
Here we have a tradeoff which we have also to consider , which you rightfully state: Data Amount vs. Performance, as more data we include in the inspector as slower and dangerous (circular refs) it becomes..
The thing is here for me, circular references is no go, this has to be handled and fixed. About performance maybe we need to adjust the existing architecture of the inspector and require "smaller" portions of data packed into "smaller messages". I think this could correct the performance significantly, since we will only require "specific" portions of data and not everything at once...
But maybe the other colleagues may have a better ideas here, what do you say colleagues?

@gtreskas Just pushed a rework of the message parsing algorithm, removing circular references (2) but also ignoring complex object types.
I am with you: The performance in those cases is super bad even when ignoring all references, but when applying the check for circular references it is literally unusable.
Manual sanity checks with the references change are looking good. I also checked what happens when ignoring all references (1): Then e.g. the context binding information in the bindings tab gets already lost, so this is actually not really an option.

@SeeSharpSoft  Awesome! Thank you very much for your work, i didnt had time lately, but I will try it out and let you know! I will pull the changes from your PR.

Hello,
This should have been resolved in latest versions.
Best regards,
Jordan


Fixed Build Errors from Outdated Dependency

ttypescript v1.5.13 caused a build error to occur when node version was v18.17.1 or more.
Updating ttypescript to v1.5.15 seems to resolve this issue.

I put some commits yesterday on this PR that also changed the version and did "audit fix". I was trying to be helpful and was looking at how Meshiest did updates, but I realized that updating the version in a PR itself is probably not the best approach.
So, I reverted those commits.
Best to leave it to the maintainers, lesson learned lol.

This looks good, but I have a suggestion if you're interested.
It looks like ttypescript is now deprecated and will not work for versions of TS >=5. Its README says it has been replaced by ts-patch, which seems to function more or less the same.
If you'd like, you could replace our usage of ttypescript with ts-patch. From what I can tell, you would only have to remove the dependency on ttypescript and add ts-patch, then replace instances of ttsc with tpsc in our build scripts (like in package.json).
This gives us some future breathing room for when we decide to bump TypeScript and lets us rely on one fewer deprecated package :)

This PR should be irrelevant now, so I'm going to close it. Have a look at this more fitting PR:
Replace ttypescript with tsc-alias


[NCL-4724] Do not fail on a missing repository attribute

In case it's missing, it will be defaulted to empty string.

I guess we're waiting if it's a repour or PME bug right?

@thescouser89 I think this one covers even more cases of possible failures so I would probably suggest to review it. Not sure if this will be backported to any of our current releases, probably not.

@goldmann  LGTM! Could you resolve the merge conflict? After that I'll merge it

Closing this one since it was implemented in the other PR's you merged today.


Provide explanations for flags

I'm submitting a feature request
Brief Description
OSMCha provides flags on some changesets. These include "Invalid Tag Modification", "Suspect Word", "Mass Modification", and "User has multiple blocks". These flags indicate that something is wrong with the changeset. Unfortunately, little information is given about what these mean.
I propose the following:
Provide details when appropriate:

Suspect Word: Provide the word (or words)
Invalid Tag Modification: Provide the tags and what changes were made
Mass Modification: Provide how many items

Also, the flags themselves should be clickable with links to descriptions of what they mean. What does it mean that a "user has multiple blocks"?
What is the motivation / use case for this feature?
This will provide reviewers with more information to help them make the map better.
Please tell us about your environment:
I'm not sure if this is relevant but here is my setup:

Operating System: Debian GNU/Linux 12 (Bookworm)
Desktop Environment: Plasma 5.27.2 (from Debian repositories)
Web Browser: Firefox 102.12ESR (from Debian repositories)

I totally agree with the need to have details on the meaning of the messages.
I get often "Invalid Tag Modification" on my own changesets, and I am quite sure that there is no invalid tag.
"user has multiple blocks" means probably that a user has been blocked in the past by the data working group of OSM.

Just bumping this. @willemarcel this would be really great to prioritize - I think it would help immensely for people to understand the current flags (and also help clarify that these are not always "errors" but just the output of specific functions and checks) - I think we have a description field for Flags in the backend that we can work on populating, and maybe get some help from the community? And just find some good way to surface that on the frontend?
If we want to do work to improve the flags and also get more community involvement in writing custom flags, it would be really great to make sure the current flags have a clear explanation that's communicated, and hopefully this is not too heavy a lift.