Collapse all redundant version(OS) blocks

This was done in #1069 but proved controversial, so I've spun it off into this PR so as not to block the rest of the changes in that PR.
If reviewers can provide a reason why any particular version block might be needed in the future, say because some future OS we'd like to support varies, I'll add a comment indicating that instead and leave that block in.  Otherwise, I'd rather clean these up and not have this unnecessary boilerplate.

I guess if you can show that all of this is specified in POSIX standard it should be enough to avoid versioning because it's all in sys.posix.

@joakim-noah thanks, for going through this trouble. Now I think just left out these that are not 100% pinned by the standard (such as TCP_NODELAY value) and this should be good to go.

Which did you have in mind?  Why keep those if they do not vary on any of the currently supported platforms?

Which did you have in mind? Why keep those if they do not vary on any of the currently supported platforms?

They might once somebody comes to implement a new one. If it's not defined the person porting D to a new OS would easily recognize potential place to add his definition. Highly unlikely that it would be a different value but just staying on the safe side here.

The separate version blocks is deliberately done this way. This is because:

Having a mix of version'd and common blocks makes for tangled code that is harder to figure out what is what.
When it comes time to port to a new system, the else static assert(0); at the end of the version blocks signals to the porter that he has to find out what the correct declaration for the new OS is. With common blocks, the porter may simply accept the default - and this can cause VERY OBSCURE and weird bugs. Yes, this has happened to us.
There is no reason to conserve file space for these. Declarations that happen by chance to be the same are NOT boilerplate and are NOT dirty.
By having a version block for each OS, it gives some comfort that the porter actually did check the declarations rather than copy/pasta them, or worse, simply assume that the default is common across all systems.

I strongly oppose this change.

By having a version block for each OS, it gives some comfort that the porter actually did check the declarations rather than copy/pasta them, or worse, simply assume that the default is common across all systems.

Except that if the porter sees 5 identical version blocks, they're just going to copy paste them.
Wouldn't it be nice if you could just do this!
version( linux || OSX || FreeBSD || Solaris || Android )
{
    enum INET6_ADDRSTRLEN = 46;
}
else
{
    static assert(0);
}

@ WalterBright  Except that 90% of this is FIXED for all POSIX by the STANDARD that is not going to change, so this is just copy-paste junk we accumulated.

@DmitryOlshansky that's all fine except when a particular posix .h file is slightly different, and nobody checked. This happened to druntime for pthreads.h.

Wouldn't it be nice if you could just do this!

Yeah, and then someone added a declaration to it for Linux that turned out to be different on OSX, but of course he didn't have an OSX machine and didn't check.
I'm fed up with bugs like that in druntime. All those convenient merged version blocks turned out to cost a lot of extra time in the long run.

I'd like to remind everyone that there is currently no tool to check .d header files against their .h counterparts. None. Not at all. The only way to do it is to manually check. This checking must be done for each platform, line by line. It's much harder to do when some declarations are merged and some are not. I speak from experience.
No declarations should go into druntime without being checked. There's been a lot of trouble in the past with merged and unchecked declarations.

Having a mix of version'd and common blocks makes for tangled code that is harder to figure out what is what.

This is the status quo, I see no way around it.

When it comes time to port to a new system, the else static assert(0); at the end of the version blocks signals to the porter that he has to find out what the correct declaration for the new OS is. With common blocks, the porter may simply accept the default - and this can cause VERY OBSCURE and weird bugs. Yes, this has happened to us.

6 of the 14 blocks collapsed here have no static assert, so very likely they'd have simply been missed.  I know that common blocks are not going to be checked, as I didn't check them for my Android port, only after something went amiss.  I don't think that's as bad as you claim.

There is no reason to conserve file space for these. Declarations that happen by chance to be the same are NOT boilerplate and are NOT dirty.

It's not a matter of file space, it's a matter of a waste of time for porters, at least for the blocks with a static assert, the rest will probably just be ignored.  These declarations are not the same "by chance," they are defined this way by the POSIX standard.  The "chance" would be if some OS was different, in which case they can put their definitions in their own OS directory, as linux and OS X do for some definitions.

By having a version block for each OS, it gives some comfort that the porter actually did check the declarations rather than copy/pasta them, or worse, simply assume that the default is common across all systems.

Sure, but where do you draw the line?  Why are these definitions randomly chosen to separate, just in case they may vary on some future unspecified platform, and not the hundreds of remaining common definitions in druntime?
For example, I noted earlier that signal is defined differently on Android, so I'll need to modify the common definition in core.stdc.signal to get it to work on Android.  This wasn't some hair-pulling experience, I simply tried to use signal in some D code and the linker told me the symbol wasn't available.  I realize it may not always be so easy, but that is no reason to arbitrarily separate some declarations and not others.

I'd like to remind everyone that there is currently no tool to check .d header files against their .h counterparts. None. Not at all. The only way to do it is to manually check. This checking must be done for each platform, line by line. It's much harder to do when some declarations are merged and some are not. I speak from experience.

This will not be true someday soon, I'm surprised it isn't already.  I used Dstep to check a lot of these, it can certainly be automated for function declarations.  The missing piece is that libclang, which Dstep is built on, doesn't provide a C API for their preprocessor, which will hopefully be remedied soon.

No declarations should go into druntime without being checked. There's been a lot of trouble in the past with merged and unchecked declarations.

That is a lofty goal, but druntime has so many declarations that are currently not separated by OS.  How do you pick and choose which go in common blocks and which should be speculatively separated, just in case they vary on some future unspecified platform?  That is the key question here, as none of these blocks vary on any of the currently supported platforms.

as I didn't check them for my Android port

This is exactly the problem. If proper version declarations are used, the worst is the Android port is borked. With them all merged, then sloppy ports put the other platforms at risk. Nobody wants to go and recheck all the OSX declarations, for example.

I realize it may not always be so easy, but that is no reason to arbitrarily separate some declarations and not others.

These can be VERY HARD to track down bugs to. C declarations, for example, do not have types encoded in the name. So having a type that is off can lead to erratic behavior. Having a constant with a different value can behave weirdly. Having a struct with a different layout is awful. We have NO WAY to mechanically check the accuracy of this stuff. It is boring, tedious, manual labor. The version scheme helps compartmentalize things so if you're working on OSX declarations you're not inadvertently mucking up other platforms.

but druntime has so many declarations that are currently not separated by OS.

That's because it gets worked on by dozens of people and is in a constant state of flux; I don't see everything before it goes in. There are often attempts to undo the versioning, I only happened to catch this one by chance. This debate happens regularly, because the version scheme used is unusual, and peoples' first reaction is it is something that should be "cleaned up".

The version scheme helps compartmentalize things so if you're working on OSX declarations you're not inadvertently mucking up other platforms.

Nobody is suggesting using common declarations where platforms like OS X differ.  The question is why are we arbitrarily separating some declarations that do not differ on any of the supported platforms and are defined by the POSIX standard to be a certain way, which is why they are currently the same across OSs and not random "chance?"

There are often attempts to undo the versioning, I only happened to catch this one by chance.

I like the version scheme, that is not the problem.  The problem is when to use it.  You have not answered the key question above:
"druntime has so many declarations that are currently not separated by OS. How do you pick and choose which go in common blocks and which should be speculatively separated, just in case they vary on some future unspecified platform?"
Insisting that these few stay separated while hundreds of others aren't seems arbitrary, given that none of these vary on any of the currently supported POSIX platforms.

I'll repeat - operating system APIs should all be separated into version blocks. The fact that this is not consistent across druntime is because lots of people have contributed to it and not followed this pattern.

I'll repeat - operating system APIs

Then start with deleting sys.posix.
Seriously POSIX is a standard for a reason and this file is dedicated to it. No point in denying hard facts about that standard pretending that they can change.

I agree with @WalterBright on most of these things. The POSIX standard does not specify layouts or implementations. And C macros can make this much more difficult than you think. pthread.h is a good example of a Posix standard API that varies wildly on implementation.
However, there are definitely some that should be common.
2 examples:
INET6_ADDRSTRLEN should be common, it's a constant. No chance for odd definitions.
struct group should NOT be common.
Even though struct group is defined by the posix standard, it says:
The <grp.h> header shall declare the group structure, which shall include the following members:

shall include, not shall only include. The presence here of gr_passwd as identified by @joakim-noah is evidence that we cannot be certain of exact layouts here.
I'd say anywhere you can point to the standard defining a specific layout or specific value some constant should be, we can simply define that struct/constant as expected. Anywhere it's left underspecified, we should have versions, even if most of them are identical.
I'll note that I strongly also agree with @yebblies about using version boolean or-ing. Copy-pasted code is copy-pasted code, whether it's done by the user or compiler. Making the user use his CTRL-C and CTRL-V keys isn't going to stop the same issue from happening. If I'm trying to get a port up and running, I pick the most similar version, copy paste it, see if it breaks, and if not, I'm done. I realize this isn't the right way to do it, but there's no possible way to force the right way on developers. I speak from experience when I did the array append update, but didn't have a Mac to test with.

I'll repeat - operating system APIs should all be separated into version blocks. The fact that this is not consistent across druntime is because lots of people have contributed to it and not followed this pattern.

@WalterBright, how committed are you to this?  If I instead submit a PR that separates all common blocks in core.sys.posix by OS and adds static asserts everywhere, would you merge it?  For example, if I separated out these 8 function declarations and this block of 38 function declarations from core.sys.posix.pthread by OS, so that they were copy-pasted many more times, would that be a change worth merging?
Of course, I wouldn't actually check all those declarations on every OS, which is your real goal from all this separation by OS, but I'd add a comment saying that certain OS's weren't checked, while I could check a couple, like linux and FreeBSD.  This could certainly be an improvement for those two OS's, while it wouldn't be any worse than the status quo for the OS's that aren't checked, plus the comment might help get them checked by someone eventually.

I'll note that I strongly also agree with @yebblies about using version boolean or-ing. Copy-pasted code is copy-pasted code, whether it's done by the user or compiler. Making the user use his CTRL-C and CTRL-V keys isn't going to stop the same issue from happening. If I'm trying to get a port up and running, I pick the most similar version, copy paste it, see if it breaks, and if not, I'm done. I realize this isn't the right way to do it, but there's no possible way to force the right way on developers. I speak from experience when I did the array append update, but didn't have a Mac to test with.

I'll repeat.

By doing copy-pasta without checking, the porter is specifically taking action he knows is wrong. This is quite different from having a default version that is overlooked.

The copy-pasta damage will be limited to the ported to platform. It will not inadvertently and unwittingly break the other platforms.

Another one: Frankly, people tend to get confused by blocks of Boolean versions, especially when they get nested. Someone working on ABC operating system is totally focused on ABC, and having DEF stuff mixed in is a distraction and just asking for typos and inadvertent modifications creeping in.

I'll pull the experience card here. I have been doing ports, and working with ports, for 35 years. I have a LOT of experience with what goes wrong. Do these things I allege will happen actually happen?  It's common and an ongoing nuisance. It has happened repeatedly in druntime, too, by well meaning and competent professionals who do not believe me. Declarations separated out by OS version blocks have fared a LOT better in the stability department.
I could let this go and allow people to learn the hard way again and again. But I don't think we have the resources to waste on that.

The copy-pasta damage will be limited to the ported to platform. It will not inadvertently and unwittingly break the other platforms.

I can buy that. And when you're speaking from experience it's I can trust.
In any case I wouldn't go against your position. I hate it, but you have good reasons to hold it.
I guess with that we are closing it, @joakim-noah my appologies for the wasted work.

thank you, @DmitryOlshansky

@WalterBright, I think you missed my question above.

Maybe a compromise could be helpful? While every platform could potentially have its own definitions, most don't (at least for the function signatures; structs may be different). So, could there be a (package-private) module with common POSIX definitions, the relevant parts of which would however have to be publicly imported by the OS specific blocks? This should reduce C&P, while still requiring an explicit import statement for each platform, reducing the risk of mistakes...


Rewrite doc base paths

This introduces support for {owner}/{repo}/~/documenation style doc urls that are base-path rewritten from our existing doc sets. It does not attempt to make any further changes to our URLs with respect to the remaining steps laid out in #2797. It's a pre-requisite to make these changes.
This is a rebased version of the (now named) no-direct-old branch, which gates all the changes to dev only.
I think I know how to refactor and clean this up but it's going to be a large change and I don't think we want to roll this into this same PR.
We should thoroughly review this change here as is, while it closely resembles the existing routes and test it on staging. Once we're happy we should roll it out completely (i.e. more the dev-gating).
Only once that's done we can work in the remaining changes for stable hosting and then we can look into refactoring with a DocRoute type (as described in the FIXME comment).

I've run some additional manual tests against the ad deployed branch. Process recorded in the tracking issue #2797 so it doesn't get closed with the PR: https://github.com/SwiftPackageIndex/SwiftPackageIndex-Server/issues/2797#issuecomment-2022375073


Disallow using '.' in 'ob thunk update'

Probably want to check the canonicalized path to catch other variations of ., like

Probably want to check the canonicalized path to catch other variations of ., like

Thanks for the heads up. The deed is done.

Why would we want to disallow this completely? If the thunk is packed then updating on . should be harmless. Updating unpacked thunks probably isn't safe in general.

This is an improvement over the current broken behavior, but supporting ob thunk update . would be nice to have

@luigy Agreed. Just seems like if we need to special case this anyway it might be no harder to just support it instead of prohibiting it.


Clean up rate limit config and add an example topology

Discussed offline. Keeping the functions with @Deprecated annotation to be safe.

👍


Use named error return for service#StartShim

For service#StartShim, the deferred func was referencing error variable err which is declared prior to line 204. And the error return is not named.
Meaning, the deferred func wouldn't check err variables on and after line 211.
This PR changes the return values to be named (and makes the deferred func operational).

/cc @thaJeztah @dmcgowan

The one test failure in:
https://travis-ci.org/containerd/containerd/builds/655571904?utm_source=github_status&utm_medium=notification
doesn't seem to be related to the PR.

The one test failure from https://travis-ci.org/containerd/containerd/builds/655822066?utm_source=github_status&utm_medium=notification is not related to the PR.

AppVeyor CI passed but GitHub did not update status: https://ci.appveyor.com/project/mlaventure/containerd-3g73f/builds/31099817

Are you sure? According to https://play.golang.org/p/Ls6JMz1P5l1 the defer statement would take the latest value of err.

Only if the original err is being written to, which it generally isn't because of if err :=
Defining retErr as a return argument means whatever error is returned will be assigned to that variable regardless of scoping.

@oldium see https://play.golang.org/p/DDODq9NBcgJ

@cpuguy83 @thaJeztah thanks, I see it now, scoping together with := was problem

No worries! It's definitely confusing (error handling and defer), and commenting if you suspect there may be an issue doesn't hurt (reviewers could've missed a mistake) 👍


Consider migrating to LocoKit

I just stumbled over LocoKit, a LGPL licensed iOS framework to track positional data.
Given its maturity and features, consider using it as base for the Traccar iOS client. It might help to increase tracking precision, reduce battery draining and maybe work around the background execution issues reported occasionally.

Unless there is an analog for Android, I'm afraid we won't be able to use it. We want to keep Android and iOS functionality as close as possible.


Any chance this can be modified to work on macOS?

I have a Hackintosh, built with MSI Z270 M6, running the latest version of macOS. it would be nice to be able to change the colors of my computer while running macOS. Thanks in advance for all and any responses!

Unknown.


access older version of code?

hi,
I foolishly deleted code after I had commented it out and decided I did not like the change. I grabbed too much when I was deleting.
Is it possible to access earlier versions of the same file in Brackets?

Not natively, as far as I know. There are several VCS extensions available but that doesn't help after the fact.


Spark operator will not delete the pods after an app update

Spark operator will not delete the pods after an app update  and that leads updating to fail since the same pods exists. See attached log scale-spark.log. For some reason code here  never worked. That implies that app.Status.DriverInfo.PodName is empty. That name should have been filled due to the successful submission before the scaling. Not sure what is happening in the meantime.
This was fixed before here: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/commit/0fa38e4fc9f008c4fd1fccb4152d9db3acdeab32 but now the logic (code has been refactored since that PR) sets the driverInfo at the time of a submission which should be preserved until the update time (when invalidating state is being processed).

We recently changed the way the driver pod is tracked. It used to be that the operator listed pods belonging to an application and went through the list of pods to find out the driver pod. Since the operator always sets the driver pod name if the user does not specify one, it makes more sense to keep track of the driver pod by name. The updated logic is that the driver pod name is stored in the status section while the job gets successfully submitted.

Same problem here.
Running on master@62ac9668 and the sparkapplication is not getting its status updated thus deleteSparkResources obtain empty PodName from status and driver is not removed with sparkapplication.
It seems the problem starts with 55a1eebc.
Also if there is a problem with status the driver stay alive after sparkapplication has been deleted.
It would be nice to add something like this: (sorry for non-idiomatic code I've no practice with golang)
	possibleNames := []string{app.Status.DriverInfo.PodName,string(*app.Spec.Driver.PodName),app.Name}
	var driverPodName = ""
	for i := range possibleNames {
		if len(possibleNames[i]) > 0 {
			driverPodName = possibleNames[i]
			break
		}
	}

We set the driver pod name in the status section when submitting the application. The driver pod name is set to either the one in app.Spec.Driver.PodName or an auto-generated one. So the name is set in a way similar to what you suggested above.

See #624.


Automated Kubernetes version update 1.26.0 1.25.5 1.24.9 1.23.15 1.22.17

Automated changes by the cron-kubernetes-update GitHub action
Adds [Kubernetes](https://kurl.sh/docs/add-ons/kubernetes) version(s) 1.26.0 1.25.5 1.24.9 1.23.15 1.22.17.

The automation says that it added multiple versions but the diff is only for 1.26.

Closing as this was generated out of a branch. This will be re-opened from main.


Crash on 1.19.2 (fixed in later version)

Hello, it seems that I've been encountering a client crash on my game that is related to this mod, "apparently" it happens only when I change windows either minimizing the game or just changing it directly when I'm inside of my world, I don't remember if it happens while playing normally, but I stick with the 1st option just because the 2nd one isn't that common/doesn't happen, this are the game, loader and mod versions:
Game version: 1.19.2
Fabric version: 0.14.25
Mod version: [Beta] 1.6+1.19
World: Singleplayer
I've seen a similar issue here but on the 1.20.1-server with fabric 0.14.22, but since it was closed, I wasn't sure to mention the crash there, another thing is that the crash may be random, because there's sometimes it happens and sometimes not, but yesterday it happened twice with the 1st option I said, I'll leave the crash log here if there's someting useful I can't see despite the lines that indicate the mod crashing the game.
crash-2024-03-16_21.40.40-client.txt

Hello, this is indeed the same issue as #48.
This was fixed in version 1.7 of my mod. This version was not backported to older Minecraft version, so they still have the bug.
If you really need the fix, I will see if I can backport it to 1.19.2, but I don't promise as it's quite old and may take time that I don't want to use that way.

Hi again, to port back to 1.19.2, I would need to undo too many changes. So, I'm not going to backport the fix to 1.19.2 sorry

Hi, well that's a bit sad, but understandable, knowing the big work that is present on backporting to previous versions, at least I confirmed the crash isn't present anymore on later versions because if I'm not wrong there's a previous version of the mod for the 1.20 [1.6.2-1.6.4] version of the game in wich I saw the crash was present on, thing that I wasn't sure if it was fixed in those version of the mod you said.
Thanks for your attention and explaining, good luck with the mod.


October 31 error

As the example shows the 31/october doesn't appears on callendar.

Hi, this is a duplicate of
#142 (I know is a tooo  broad name...)
#109
and #54
you can see there thing related, so continue the topic either on #109 or #142
you can close this for now, thanks.


add #include <stdio.h>

I think the #include <stdio.h> should be added to these two examples :)
these maybe unnacessary for the latest compiler, but I'm trying to use it on
Linux version 2.6.32-220.el6.x86_64<EMAIL_ADDRESS>(gcc version 4.4.5 20110214 (Red Hat 4.4.5-6) (GCC) )

Thank you for the pull request!


Cleanup ideas

I have some ideas on the redesign, and want to put them down:
First, I think we should not hard-code flags, and instead allow them to be passed through, like TARGET_LANGUAGE and LANGUAGE_LEVEL unless there's a benefit for us (possibly with TARGET_LANGUAGE? Edit: Yes, we set the extension from this). Users should generally set these options in source files anyway, and if not, they can just as easily pass them through CYTHON_ARGS without us adding lots of custom options. We should also not try to set defaults for the user, and leave that up to Cython.
The command supports adding multiple input files and a single output file - not sure what that does, but if it's useful, we should mimic that here. I don't think there's a lot of benefit in supporting multiple outputs (users can just write a loop) - I'd like to mimic cythonize as closely as possible, just trying to bring it into CMake.
I think instead of an output variable, maybe we should support setting the OUTPUT file directly? That way users can choose where to put the output file location easily. (This also would work best if we don't support multiple input files to multiple output files, above)
Once these are addressed, I think the back-compat "add_cython_target" could be designed using "cython_compile_pyx" Ahh, nice, already done. It could even only exist in scikit-build (classic), as it has a really poor name (it doesn't have anything to do with targets). The "official" version upstreamed to Cython could be called "cythonize".

Seee https://github.com/cython/cython/issues/5486#issuecomment-2187180678 - maybe we should require the LANGUAGE keyword? Currently it changes based on the LANGUAGES set (C unless only CXX is available), which might be slightly supposing? Unless we can detect the language, might be best to just require this for now. Also, I think LANGUAGE is better than TARGET_LANGUAGE, as it's unambiguous already.


feat: add skip_macro_invocations option

Tracking issue: https://github.com/rust-lang/rustfmt/issues/5346
Original feature request and design: https://github.com/rust-lang/rustfmt/issues/5324#issuecomment-1111797851
Globally skippable macro names are defined in the config, and added to the existing SkipContext. Thanks to @ytmimi for the pointers on where to implement this, it made the changes super painless!

Thanks so much @tommilligan!
I wanted to echo @ytmimi's question about definitions vs. bodies. The latter was what I understood the original scope to be, and which I believe to be the more important of two. I feel like definitions can be more easily handled via the outer skip attribute (since definition occurs only once), whereas calls can be scattered throughout the codebase.
Additionally, while pondering the above and the implications for the config name, I was also wondering if we should provide a mechanism to disable all calls in a simple manner, e.g. ["*"]?

Additionally, while pondering the above and the implications for the config name, I was also wondering if we should provide a mechanism to disable all calls in a simple manner, e.g. ["*"]?

@calebcartwright this is now implemented! Nice idea.

definitions vs. bodies

ref: this thread, I think this may be a question of renaming the option to clarify, if you have any opinion: https://github.com/rust-lang/rustfmt/pull/5347#discussion_r894704462

@ytmimi thanks for your comments, they were super helpful. I've implemented or commented on all your points, so ready for another review.

@tommilligan Thanks for taking the time to address all my comments. I've been away on a trip for the last two weeks so I haven't had much time for the project, but I'm back now and I plan on going through this again early next week (likely tomorrow!)

Will just add that I agree with skip_macro_invocations as the name (and have updated the PR title to that effect), agree with the proposed approach in https://github.com/rust-lang/rustfmt/pull/5347#discussion_r895000579 but doing so in a follow up, and also agree with the additional test cases outlined in https://github.com/rust-lang/rustfmt/pull/5347#pullrequestreview-1005832979
Everything else looks to be in good order to me :+1:

Updated to:

Expand the documentation
Rename the option to skip_macro_invocations
Add test cases as suggested

@ytmimi thanks for your test case list - there were some that I disagreed with:

calling a macro with a fully qualified path (e.g. self::items), where items is in the ignore list -> Expect that self::items would be ignored
This isn't the current behaviour - and I don't think it should be. As written, the feature is based purely on the literal invocation name, not on resolving the macro's identity to some more rigorous level - and I think that's fine. I think it's out of scope to handle path resolution.

We could do a really simple pass of 'remove all but the last path segment of the invocation name, but then you couldn't do a more qualified match for e.g. pretty_assertions::assert_eq`. You also wouldn't be able to selectively ignore just some macro invocations with the same name, but qualified by different paths.

aliasing a macro so that it matches up with the name in the ignore list (e.g. self::items as renamed_items), where renamed_items is in the ignore list. -> Expect the aliased macro to be ignored

This is implicitly tested in the different cases I added, because the use statement doesn't impact this rule, it's just the invocation name.

calling a macro with a fully qualified path (e.g. self::items), where items is in the ignore list -> Expect that self::items would be ignored
This isn't the current behaviour - and I don't think it should be. As written, the feature is based purely on the literal invocation name, not on resolving the macro's identity to some more rigorous level - and I think that's fine. I think it's out of scope to handle path resolution.

We could do a really simple pass of 'remove all but the last path segment of the invocation name, but then you couldn't do a more qualified match for e.g. pretty_assertions::assert_eq`. You also wouldn't be able to selectively ignore just some macro invocations with the same name, but qualified by different paths.

That's a good point. I'm happy to go along with this behavior, and not ignore fully qualified macros if they aren't explicitly listed. I don't want to expand the scope of the current PR, and maybe this next idea needs to be discussed in more detail, but I wonder if we also need to consider the wild card case for fully qualified paths? Would "self::*" in the ignore list prevent self::item and self::item as renamed from being formatted? Again, I don't want to expand the scope of what we have right now, but I could see users asking for something like this in the future.

aliasing a macro so that it matches up with the name in the ignore list (e.g. self::items as renamed_items), where renamed_items is in the ignore list. -> Expect the aliased macro to be ignored

This is implicitly tested in the different cases I added, because the use statement doesn't impact this rule, it's just the invocation name.

Yup, I get that it's implicitly tested, but I think having at least one test where we're explicitly testing for it will make it clearer in the future that we've also considered this case.

..., I wonder if we also need to consider the wild card case for fully qualified paths? Would "self::*" in the ignore list prevent self::item and self::item as renamed from being formatted? Again, I don't want to expand the scope of what we have right now, but I could see users asking for something like this in the future.

I think that's a reasonable feature request, and one that's compatible with the current design. As those patterns are invalid macro names, I think we're fine to give them special functionality in future without breaking existing behaviour.
I think that functionality would also be reasonable simple to add, but let's keep the scope down for now and wait for someone to ask!

Yup, I get that it's implicitly tested, but I think having at least one test where we're explicitly testing for it will make it clearer in the future that we've also considered this case.

Fair enough - added a new test case file to demonstrate the current behaviour, and ensure use statements don't impact the behaviour (I guess basically a regression test actually)

Thanks for adding the additional test! I'm looking to give this another review later this week.

Rebased on master to resolve merge conflict.


c8d/pull: Don't unpack manually

We pass WithPullUnpack anyway
Signed-off-by: Paweł Gronowski<EMAIL_ADDRESS>
- What I did
- How I did it
- How to verify it
- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

possibly combine with https://github.com/rumpl/moby/pull/12


[SPARK-43025][SQL] Eliminate Union if filters have the same child plan

What changes were proposed in this pull request?
There are a lot of SQL with union multiple subquery with filter in user scenarios.  We just need merging the predicates into one single predicate by connecting these Filters with Or.
Combines multiple Filter operators into one if all the child of these Filters are same. We just need merging the predicates into one single predicate by connecting these Filters with Or too.

Why are the changes needed?
Simply the SQL plan and improve the performance.
Does this PR introduce any user-facing change?
'No'.
New feature and just update the inner implementation.
How was this patch tested?
New test cases.
The micro benchmark for q1 and q2.
Before this PR
Java HotSpot(TM) 64-Bit Server VM 1.8.0_311-b11 on Mac OS X 10.16
Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
TPCDS Snappy:                             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q1                                               51569          52030         653          0.6        1627.9       1.0X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_311-b11 on Mac OS X 10.16
Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
TPCDS Snappy:                             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q2                                                4255           4287          45          6.8         147.7       1.0X

After this PR
Java HotSpot(TM) 64-Bit Server VM 1.8.0_311-b11 on Mac OS X 10.16
Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
TPCDS Snappy:                             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q1.                                              46806          47462         929          0.7        1477.5       1.0X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_311-b11 on Mac OS X 10.16
Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
TPCDS Snappy:                             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q2.                                                2655           2674          28         10.8          92.2       1.0X

ping @cloud-fan

I think @peter-toth did something similar before, can you share some ideas @peter-toth ?

I guess @peter-toth did the similar thing for scalar subquery, but this one try to fix non-scalar subquery.

I think @peter-toth did something similar before, can you share some ideas @peter-toth ?

I guess @peter-toth did the similar thing for scalar subquery, but this one try to fix non-scalar subquery.

Sorry, I haven't got time to fully review the PR (maybe next week) but at first sight it seems to copy some fuctions (e.g. checkIdenticalPlans(), mergeNamedExpressions()) from MergeScalarSubqueries so there seems to be some room for improvement and we could share the common functions. Also, some names (e.g. subqueryIndex) might need some changes here.
This PR combines UNION ALL legs if they return disjoint set of rows from the same source node. I think this makes sense in those cases when there are overlaping scans in the legs (despite the disjoint filters), and by "overlapping" I mean that the scans use some common set of files.
So seems like the only case when this change doesn't bring improvement is when the filter is a partitioning/bucketing column and the scans in union legs doesn't overlap. But even in that case I'm not sure if this PR has any disadvantage, just doesn't improve anything...
BTW, MergeScalarSubqueries (https://github.com/apache/spark/pull/32298) does very similar merging, but we run that only once because merging can be costly when there are many candidates. Do we need EliminateUnions in operatorOptimizationBatch?
Sidenote: MergeScalarSubqueries doesn't work with different filters currently. This is because merging filters in subqueries is more comlicated as we need to propogate the filters up to an aggregate, and because it can cause performance degradation when we have non-overlapping scans. (See this WIP PR: https://github.com/apache/spark/pull/37630).

@peter-toth Thank you for your first look.
In fact, this PR references some functions as you mentioned above (e.g. checkIdenticalPlans). If we can share these functions, that be good.
The partitioning/bucketing column filter seems doesn't improve anything. I will optimize it in further.


how to make Allow Once work in IOS 13

current my info.plist file looks like this
<key>NSLocationWhenInUseUsageDescription</key>
<string>This app needs access to location when open.</string>
<key>NSLocationAlwaysUsageDescription</key>
<string>This app needs access to location when in the background.</string>
<key>NSLocationAlwaysAndWhenInUseUsageDescription</key>
<string>This app needs access to location when open and in the background.</string>

When I run in the ios13.3, Allow while using app works, while allow once does not work. Do I need to add something else?

ok I remove NSLocationAlwaysUsageDescription from info.plist file and change the target version of ios from 8.0 to 11.0. Now it works.


Players online counter.

Banned accounts are incuded in the online counter.
It makes it easy to keep cooldown high.

Possible fix would be just not to add banned users to UndertowServer.authedUsers.


Implement an IT test for a Flow Auto Layout

In #2934, we encountered a significant issue with the Flow Auto Layout feature. To prevent regressions and ensure the stability of this feature, we should implement a dedicated IT test for it.

Note that there are platform IT tests related with layout and routing features. We should share them between Flow and Hilla.


Remove dependency on k8s.io/kubernetes

In https://github.com/microsoft/hcsshim/pull/789, hcsshim removed its
dependency on k8s.io/kubernetes. 5 imports upstream, we need to
replace it in order to not pull in k8s.io/kubernetes. Rather than
that, would be ideal for oras to just update.

/ok-to-test

/test


[BUG] Writing JSON to STDOUT

Describe the bug
A clear and concise description of what the bug is.
I have been attempting to use rich to output a very large JSON into STDOUT.
To Reproduce
A minimal code example that reproduces the problem would be a big help if you can provide it. If the issue is visual in nature, consider posting a screenshot.
test_rich.py
from rich.console import Console

console = Console()
console.print_json(data=[{"hello": {"big wide": "world"}} for _ in range(50000)])
# Please note that my data is a decent bit larger

$ python test_rich.py | jq
parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 14, column 9
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "my_module/cli/__init__.py", line 21, in main
    args.func(args)
  File "my_module/cli/commands/search.py", line 143, in cli_search
    write_file(results, **write_args)
  File "my_module/cli/utils.py", line 115, in write_file
    _write_screen(results_list)
  File "my_module/cli/utils.py", line 82, in _write_screen
    console.print_json(data=search_results)
  File "./.venv/lib/python3.9/site-packages/rich/console.py", line 1643, in print_json
    self.print(json_renderable)
  File "./.venv/lib/python3.9/site-packages/rich/console.py", line 1615, in print
    self._buffer.extend(new_segments)
  File "./.venv/lib/python3.9/site-packages/rich/console.py", line 825, in __exit__
    self._exit_buffer()
  File "./.venv/lib/python3.9/site-packages/rich/console.py", line 784, in _exit_buffer
    self._check_buffer()
  File ./.venv/lib/python3.9/site-packages/rich/console.py", line 1872, in _check_buffer
    self.file.write(text)
BrokenPipeError: [Errno 32] Broken pipe

Platform
What platform (Win/Linux/Mac) are you running on? What terminal software are you using?
macOS, I have tried the normal Terminal,  iTerm 2, and VSCode Integrated Terminal.
Diagnose
$ python -m rich.diagnose
╭─────────────────────── <class 'rich.console.Console'> ───────────────────────╮
│ A high level console interface.                                              │
│                                                                              │
│ ╭──────────────────────────────────────────────────────────────────────────╮ │
│ │ <console width=80 ColorSystem.TRUECOLOR>                                 │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│     color_system = 'truecolor'                                               │
│         encoding = 'utf-8'                                                   │
│             file = <_io.TextIOWrapper name='<stdout>' mode='w'               │
│                    encoding='utf-8'>                                         │
│           height = 40                                                        │
│    is_alt_screen = False                                                     │
│ is_dumb_terminal = False                                                     │
│   is_interactive = True                                                      │
│       is_jupyter = False                                                     │
│      is_terminal = True                                                      │
│   legacy_windows = False                                                     │
│         no_color = False                                                     │
│          options = ConsoleOptions(                                           │
│                        size=ConsoleDimensions(width=80, height=40),          │
│                        legacy_windows=False,                                 │
│                        min_width=1,                                          │
│                        max_width=80,                                         │
│                        is_terminal=True,                                     │
│                        encoding='utf-8',                                     │
│                        max_height=40,                                        │
│                        justify=None,                                         │
│                        overflow=None,                                        │
│                        no_wrap=False,                                        │
│                        highlight=None,                                       │
│                        markup=None,                                          │
│                        height=None                                           │
│                    )                                                         │
│            quiet = False                                                     │
│           record = False                                                     │
│         safe_box = True                                                      │
│             size = ConsoleDimensions(width=80, height=40)                    │
│        soft_wrap = False                                                     │
│           stderr = False                                                     │
│            style = None                                                      │
│         tab_size = 8                                                         │
│            width = 80                                                        │
╰──────────────────────────────────────────────────────────────────────────────╯
$ python -m rich._windows
platform="Darwin"
WindowsConsoleFeatures(vt=False, truecolor=False)
$ pip freeze | grep rich
rich @ file:///Users/aidan/Library/Caches/pypoetry/artifacts/a2/2b/30/30eb6b7558c858219ff0094233e8dd6414d7733fe81ef92169292d774b/rich-10.12.0-py3-none-any.whl

Since I can't reproduce it, can you help me narrow down what the trigger is? What happens if you output a just 10 objects? Does it break if you don't pipe it in to jq?
Does your three line example break for you? I can see that the traceback has some additional code.

@thehappydinoa Any more information. Will close in a week if I can't reproduce this.

Hi @willmcgugan I have been unable to reproduce. My apologies but the best thing for my use case was to be able to revert to standard print statements. Thank you for all the help.

Hmm yes just saw this regress after I switched to rich, but it can be remedied in the interim for integration in library code for example (who would otherwise see the same error if using regular print statements) with the following context manager.

error_handlers.py

from contextlib import AbstractContextManager
from types import TracebackType

__all__ = ("SuppressBrokenPipeError",)

class SuppressBrokenPipeError(AbstractContextManager):
    def __exit__(
        self,
        __exc_type: type[BaseException] | None,
        __exc_value: BaseException | None,
        __traceback: TracebackType | None,
    ) -> bool | None:
        if __exc_type is BrokenPipeError:
            return pipe_cleanup()
        else:
            return super().__exit__(__exc_type, __exc_value, __traceback)

I incorporated this into my library like so:

[io.py](https://github.com/lmmx/fugit/blob/master/src/fugit/core/io.py

from contextlib import contextmanager

from rich.console import Console

from .error_handlers import SuppressBrokenPipeError

__all__ = ("FugitConsole", "fugit_console")

class FugitConsole:
    console: Console
    use_pager: bool
    page_with_styles: bool

    def __init__(self, use_pager: bool = False, page_with_styles: bool = True):
        self.console: Console = Console()
        self.use_pager: bool = use_pager
        self.page_with_styles: bool = page_with_styles

    @contextmanager
    def pager_available(self):
        """Uses console pagination if `DisplayConfig` switched this setting on."""
        if self.use_pager:
            with self.console.pager(styles=self.page_with_styles):
                yield self
        else:
            yield self

    def print(self, output: str, end="", style=None) -> None:
        """
        Report output through the rich console, but don't style at all if rich was set to
        no_color (so no bold, italics, etc. either), and avoid broken pipe errors when
        piping to `head` etc.
        """
        with_style = style if fugit_console.no_color else None
        with SuppressBrokenPipeError():
            fugit_console.console.print(output, end=end, style=with_style)

"""
Global `rich.console.Console` instance modified by a model validator upon initialisation
of `fugit.interfaces.display.DisplayConfig` or its subclass, the main `DiffConfig` model.
"""
fugit_console = FugitConsole()

I can then test that piping output of my package's CLI entrypoint to | head doesn't truncate with a BrokenPipeError afterwards.

Opened a PR that fixes it:

#3233


[ja] Merge dev-ja into main (for the third Japanese localization live version)

Describe your changes
Merge dev-ja into main to release the Japanese localization v3!!
Terms to be added

#2727
#2820
#2898
#2935
#2948
#2916
#2966
#2970
#2975
#2986

The following modifications/updates are included

#2964
#2965
#2993

Related issue number or link (ex: resolves #issue-number)
No issue related to this PR
Checklist before opening this PR (put x in the checkboxes)

[x] This PR does not contain plagiarism

don’t copy other people’s work unless you are quoting and contributing it to them.

[x] I have signed off on all commits

signing off (ex: git commit -s) is to affirm that commits comply DCO. If you are working locally, you could add an alias to your gitconfig by running git config --global alias.ci "commit -s".

Hi @kaitoii11 @naonishijima @seokho-son @jihoon-seo !!
I think Japanese Localization v3 is now ready for merge. WDYT? Before merging this pr, please take a look #2969 to sync up. thank you :)
acknowledgments
Thank you for all contributions @shizhenhu @bashi8128 @kujiraitakahiro !! Your contributions will be released soon!!


How can i train a mono3D model like smoke by waymo dataset

i have waymo dateset
then i converter the waymo dataset to kitti_format by create_data.py
and then i want to train a mono3D detection model like smoke
can i run create_data.py again by kitti_format data and then i can use kitti-mono3d data pipeline?

@lindahua

Just running create_data.py once can generate mono3D annotation file.

@JingweiZhang12  i run the create_data.py by waymo
it create the pickle like "waymo_infos_train.pkl" but no json

@JingweiZhang12
thanks for your ans!!
i have tried to run create_data.py once by next:
"python create_data.py waymo --rootp-path ... --out-dir ... --extra-tag waymo "
I got some pickle bag like "waymo_info_train_pkl",but it without json file like "......_infos_train_mono3d_coco.json"

my env:
mm3d:1.01.0rc2
mmdet:2.25.2
mmcv-full:1.4.8
mmseg:0.28.0
i used to try nuscenes dataset ,by create_data.py once and it worked.(i got pkl and json)

master branch does not support waymo monocular detection task currently. It has been supported in det-1.x branch.

@Xiangxu-0103
dev-1.x branch?

Camera-only 3D detection features on Waymo are just supported in the dev-1.x/1.1 branch. It may be unstable for now, but you can have a try first. We will update it in the next release and please stay tuned.

Camera-only 3D detection features on Waymo are just supported in the dev-1.x/1.1 branch. It may be unstable for now, but you can have a try first. We will update it in the next release and please stay tuned.

Do we support training smoke on waymo currently on 1.1 branch?


Cannot render more than N documents on the same Asciidoctor instance (fixed with jRuby <IP_ADDRESS>)

AsciidoctorJ: 1.5.7
jRuby: <IP_ADDRESS> (default)
OS: Windows 10
JDK: 1.8.0_181
This is one of those weird issues...
Since version 1.5.7, if I try to render a set of documents reusing the same Asciidoctor instance, the process fails stating that it cannot read a document (see below). The error is always the same and happens on the same document.
I have been able to reproduce the issue rendering the same document in a loop and I have found the following:

The iteration in which it fails depends on the document. With the documents I have used for testing it sometimes fails on the loop 53, others on 62 🤷♂ .
Using JRuby <IP_ADDRESS> seems to fix it (500 loops without issues), jRuby <IP_ADDRESS> does not work at all.

sep 25, 2018 3:09:53 PM org.asciidoctor.internal.JRubyAsciidoctor renderFile
GRAVE: (NoMethodError) asciidoctor: FAILED: C:/Users/asalgadr/github/asciidoctorj-gradle/src/asciidoc/user-password-expiration.adoc: Failed to load AsciiDoc document - undefined method `start_with?' for nil:NilClass
Exception in thread "main" org.asciidoctor.internal.AsciidoctorCoreException: org.jruby.exceptions.RaiseException: (NoMethodError) asciidoctor: FAILED: C:/Users/asalgadr/github/asciidoctorj-gradle/src/asciidoc/user-password-expiration.adoc: Failed to load AsciiDoc document - undefined method `start_with?' for nil:NilClass
	at org.asciidoctor.internal.JRubyAsciidoctor.renderFile(JRubyAsciidoctor.java:371)
	at org.asciidoctor.internal.JRubyAsciidoctor.renderFile(JRubyAsciidoctor.java:484)
	at org.asciidoctor.internal.JRubyAsciidoctor.convertFile(JRubyAsciidoctor.java:585)
	at org.asciidoctor.demos.AsciidoctorRunner.main(AsciidoctorRunner.java:30)
Caused by: org.jruby.exceptions.RaiseException: (NoMethodError) asciidoctor: FAILED: C:/Users/asalgadr/github/asciidoctorj-gradle/src/asciidoc/user-password-expiration.adoc: Failed to load AsciiDoc document - undefined method `start_with?' for nil:NilClass
	at RUBY.parse_section_title(uri:classloader:/gems/asciidoctor-<IP_ADDRESS>/lib/asciidoctor/parser.rb:1734)
	at RUBY.initialize_section(uri:classloader:/gems/asciidoctor-<IP_ADDRESS>/lib/asciidoctor/parser.rb:1564)
	at RUBY.next_section(uri:classloader:/gems/asciidoctor-<IP_ADDRESS>/lib/asciidoctor/parser.rb:290)
	at RUBY.parse(uri:classloader:/gems/asciidoctor-<IP_ADDRESS>/lib/asciidoctor/parser.rb:96)
	at RUBY.parse(uri:classloader:/gems/asciidoctor-<IP_ADDRESS>/lib/asciidoctor/document.rb:565)
	at RUBY.load(uri:classloader:/gems/asciidoctor-<IP_ADDRESS>/lib/asciidoctor.rb:1361)
	at RUBY.convert(uri:classloader:/gems/asciidoctor-<IP_ADDRESS>/lib/asciidoctor.rb:1473)
	at RUBY.block in convert_file(uri:classloader:/gems/asciidoctor-<IP_ADDRESS>/lib/asciidoctor.rb:1589)
	at org.jruby.RubyIO.open(org/jruby/RubyIO.java:1171)
	at RUBY.convert_file(uri:classloader:/gems/asciidoctor-<IP_ADDRESS>/lib/asciidoctor.rb:1589)
	at RUBY.convertFile(<script>:78)

I just ran that class inside IntelliJ and it worked fine for me rendering the document 500 times.
I ran it in IntelliJ with Java 1.8.0_131 and these dependencies:

Anything else that I have to take care of?
I am definitely on the branch issue_asciidoctorj_680.

Windows I guess.
I colleague just reproduced it with Windows 7 and JDK 1.8.0_111.
Exactly behavior as in my machine -> <IP_ADDRESS> fail, <IP_ADDRESS> works, I'll try on my Mac and Linux when I get home later.

I just reproduced on MacOS.

I ran it in IntelliJ with Java 1.8.0_131 and these dependencies:

Note the example sets jRuby to <IP_ADDRESS>. Comment it to use the default provided by AsciidoctorJ (v16).

I reproduced the issue in Travis and AppVeyour with the test should_render_the_same_document_a_100_times.

Apologies to pushing this @robertpanzer , but I just to be sure AsciidoctorJ will upgrade to <IP_ADDRESS> as recommended version in include this in a hotfix for the maven plugin.
No need for a Asciidoctorj <IP_ADDRESS> release, but if we agree at least I can upgrade it in the maven side and if someone asks mention this is aligned with Asciidoctorj :%

I just reproduced the problem with Ruby <IP_ADDRESS>.
Imo it makes sense to bump it to <IP_ADDRESS> if it doesn't produce other problems.
I can create a PR to bump the version and I'll merge your PR asap.

Thanks! I will proceed with the maven changes.

As an addition. We ran into the same problem with the latest jbake release.
https://github.com/jbake-org/jbake.org/issues/106

I think we can close this issue. Even when not release, anyone with this issue just needs to use JRuby <IP_ADDRESS> instead of the version provided by AsciidoctorJ.


Tensor-Core support (cuBLAS)

The aim of this PR is to enable use of Tensor-Core via cuBLAS. The following modifications are mainly included in this PR.

Add interfaces to use cublasSetMathMode(), cublasGetMathMode() and cublasGemmEx() in cuBLAS.
Use cublasGemmEx() with Tensor-Core configs instead of cublasSgemmEx() when Tensor-Core is available.

Now, this branch depends on #494. Please merge #494 first.

jenkins, test please.

jenkins, test this please.

Please fix import order.
./cupy/core/core.pyx:12:1: H306  imports not in alphabetical order (cupy.cuda.stream, cupy.cuda.device)

Thank you for pointing it out. It was fixed.

Thanks.
Jenkins, test this please.

LGTM!


Converting Kaggle lit review into ground truth judgments

This is my initial attempt at turning this: https://www.kaggle.com/covid-19-contributions#Length of viral shedding after illness onset
Into machine-reading ground-truth judgments we can use for testing highlighting.

@daemon @rodrigonogueira4 What do you think?

Looks like a good start.


Update ActivityEx.cs mark GetStateClient() as depreciated

This method causes a lot of confusion for people using custom state clients.

@JasonSowers,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


What's Stream.Close real do ?

I try to test it and i can't understand what Stream.Close real do.
How to close stream on client and on server ?
I try to call on client side:
Stream.Close AND Stream.CancelRead
but it's don't closing stream on server.
Only when i call Stream.CancelWrite AND Stream.CancelRead then stream on server closed and deleted !

Please refer to the GoDoc.

Close closes the write-direction of the stream.

So it’s closed only write direction in local stream ?

@marten-seemann
I can't understand how to close stream correct ? (with deleting from streamsMap) !
I did some tests and found out:
stream.Close - closing only write direction (as i understood this menthod flush write buffer and send fin bit), but it's not removed stream from system(streamsMap).
stream.CancelRead() - ask peer to stop write data. But it's not removed stream from system too.
And stream.Close + stream.CancelRead() to my surprise, don't removed stream from system too. It's normal behavior ?
I continued find proper way to close stream with deleting from streamsMap. So correct way is stream.CancelRead() + stream.CancelWrite() but this combination some times produce error StreamDataAfterTermination because some write data buffered and try to send later.
And finally proper way to close stream is
stream.CancelRead()
stream.Close()
stream.CancelWrite() <- yep in the doc says that "do not call this method after stream.Close"

Only after this combination stream correctly removed from streamsMap on server and client side.
What am I doing wrong ? Maybe it's bug ?

A bidirectional stream has to be closed from both sides.

@marten-seemann
Of course i try to close stream on both sides ! But streams don't removed from streamsMap
Ok, lets see example:
test_client.go
package main

import (
	"github.com/lucas-clemente/quic-go"
	"crypto/tls"
	"time"
)

func main() {
	sess, err := quic.DialAddr("localhost:4242", &tls.Config{InsecureSkipVerify: true}, &quic.Config{
		KeepAlive: true,
	})

	if err != nil {
		panic(err)
	}

	go openStreams(sess)

	for {
		_, err := sess.AcceptStream()
		if err != nil {
			panic(err)
		}

	}
}

func openStreams(sess quic.Session) {
	for i := 0; i < 10; i++ {
		stream, err := sess.OpenStream()
		if err != nil {
			panic(err)
		}

		stream.CancelRead(0)
		stream.Close()
	}
}

Client logs:
client.txt
First in the head of client logs you can see another strange behaviour: "Starting new connection" and immediately "Closing session with error: closing session in response to a stateless retry".  Maybe it's normal, i don't no, but just in case i'll notice you, because it repeat every time
Second i set timer and every 15 second i log count of streams in streamMap, and in log you can see
this message: "COUNT OUTGOING STREAMS 5 [4 0 12 16 8]", this means that half of streams (5 out of 10) not deleted, but why ?
test_server.go
package main

import (
	"github.com/lucas-clemente/quic-go"
	"crypto/rsa"
	"crypto/rand"
	"crypto/x509"
	"math/big"
	"encoding/pem"
	"crypto/tls"
	"fmt"
)

func main() {
	l, err := quic.ListenAddr("localhost:4242", generateTLSConfig(), &quic.Config{
		KeepAlive: true,
	})
	if err != nil {
		panic(err)
	}

	for {
		sess, err := l.Accept()
		if err != nil {
			panic(err)
		}

		go handleSess(sess)
	}
}

func handleSess(sess quic.Session) {
	for {
		stream, err := sess.AcceptStream()
		if err != nil {
			fmt.Println(err)
			return
		}

		stream.CancelRead(0)
		stream.Close()
	}
}

func generateTLSConfig() *tls.Config {
	key, err := rsa.GenerateKey(rand.Reader, 1024)
	if err != nil {
		panic(err)
	}
	template := x509.Certificate{SerialNumber: big.NewInt(1)}
	certDER, err := x509.CreateCertificate(rand.Reader, &template, &template, &key.PublicKey, key)
	if err != nil {
		panic(err)
	}
	keyPEM := pem.EncodeToMemory(&pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(key)})
	certPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: certDER})

	tlsCert, err := tls.X509KeyPair(certPEM, keyPEM)
	if err != nil {
		panic(err)
	}
	return &tls.Config{Certificates: []tls.Certificate{tlsCert}}
}

Server logs:
server.txt
In server logs you can see message: "COUNT INCOMING STREAMS 10 [12 16 28 32 0 4 24 36 8 20]",
and that means that all streams on server side don't removed from streamsMap !
I waited 1 minute but streams don't cleared. I think it's wrong behaviour !!!

@marten-seemann what do you think about my logs ? Maybe it’s a bug ?

This works exactly as it should. I advise you again to have a look at the stream state diagrams in the specification.
A bidirectional stream has to be closed in both directions. You're only closing the stream in one direction only, which is why the stream stays in the map.
If you want to use CancelRead as you do, you have to first accept the stream at the peer, and then read the EOF.

@marten-seemann I apologize for my perseverance !
Yes i read it, but i can't understand one thing. Help me to understand it please :).
For example, client open UniStream, my server knows that client send first "handshake" stream as 10 bytes message and thats all, after receive 10 bytes i want to close stream. How to proper way to close this stream after read this 10 bytes ? You are suggest to read until accept io.EOF only !? Yes ?
client.go
stream, err := session.OpenUniStream()
	if err != nil {
		panic(err)
	}
stream.Write([]byte("hello"))
stream.Close() // i complete to write and want to close stream

server.go
stream, err := sess.AcceptUniStream()
	if err != nil {
		panic(err)
	}
chunk := make([]byte, 1024)
n, err := stream.Read(chunk) // i read handshake message and want to close stream !
if err != nil {
     panic(err)
}
fmt.Println("incoming handshake message", chunk[:n])
// at this time stream not closed yet, if i call stream.CancelRead(0), it's don't closed too

Why in my example only one way to proper close stream is read io.EOF ? It doesn't look pretty. So we always need to remember, that stream.CancelRead() don't closed stream, and we need to read io.EOF, and only AFTER that stream is proper close (or another way is call stream.CancelWrite on client side, but it's not right)

Once again, I can only point you to the specification. It seems like you didn't understand the state machine of a stream. I know it's not an easy topic, but it makes sense if you take the time to think it through.
For a unidirectional stream, on the sender side it is deleted from the map as soon as you call Close.  For the receiver side, it is deleted from the map as soon as you read the EOF.
For a bidirectional stream, both sides have to call Close and read the EOF. Not that in the normal case, you don't need the Cancel... functions at all.

Thank you so much for the explanation and your patience. Now I understand everything


BLE protocol missing getSn interface

Is your feature request related to a problem? Please describe.
BLE protocol includes the function getSn, but the framework not implemented.
Describe the solution you'd like
Implement this function in the app Network.
Describe alternatives you've considered

closed by #593


Bootstrap-select popup position

Hello,
I would like to know how can I specify a specific position for the popup to appear.
As I have a lot of elements, popup has space to open in the top. However, when searching through the items and the list gets smaller, it does go to bottom, and it looks very weird.
Any suggestions?

Set dropupAuto: false.
From the docs:

dropupAuto is set to true by default, which automatically determines whether or not the select should be a dropup. If dropupAuto is set to false, manually make the select a dropup menu by adding the .dropup class to the select.


How I can set a root key?

Hello,
I have models:
class Country
  include Surrealist
  surrealize_with CountrySerializer

  belongs_to :region
end

class Region
  include Surrealist
  surrealize_with RegionSerializer

  has_many :countries
end

And searializers:
class RegionSerializer < Surrealist::Serializer
    json_schema do
      {
        name: String,
        slug: String,
        countries: Array
      }
    end

    def countries
      object.countries.to_a
    end
end

class CountrySerializer < Surrealist::Serializer
  include Location

  json_schema do
    {
      name: String,
      slug: String
    }
  end
end

And in controller I do this:
Regions::RegionSerializer.new(Region.includes(:countries)).surrealize(root: :regions)

And I want that result will be:
{
  regions: [
    {
      name: '...',
      slug: '...',
      countries: [
         {
           name: '...',
           slug: '...',
         },
         ...
      ]
    },
    ...
  ]
}

But I had root key for each result of regions...
How I can set the real root key which can wrap all results? Please don't tell me that I need to create a new serializer :)

Does something like this work for you:
{ regions: Regions::RegionSerializer.new(Region.includes(:countries)).build_schema }

Thanks. The current behavior of root key is very strange :)


How to disable North Compensation on 3DM-CX5-AHRS

Can I disable the north compensation by this library? thanks for replying.

Hi @peichunhuang-1,
North compensation is labeled as heading_enable in the Complementary Filter command interface. Setting that value to false and writing that command to the device will disable it.
You can see the C implementations here:

heading_enable
mip_3dm_write_complementary_filter

and C++ implementations here:

heading_enable
writeComplementaryFilter

Hope that helps.

Thanks for your help, I have tried these command and check by readComplementaryFilter, however the behavior of sensor was not the same as I sent by SensorConnect.
        std::unique_ptr<DeviceInterface>& device = utils->device;
        if(commands_base::ping(*device) != CmdResult::ACK_OK)
           exit_gracefully("ERROR: Could not ping the device!");
        if(commands_base::setIdle(*device) != CmdResult::ACK_OK)
            exit_gracefully("ERROR: Could not set the device to idle!");
        if(commands_3dm::defaultDeviceSettings(*device) != CmdResult::ACK_OK)
            exit_gracefully("ERROR: Could not load default device settings!");

        uint16_t sensor_base_rate;
        if(commands_3dm::imuGetBaseRate(*device, &sensor_base_rate) != CmdResult::ACK_OK)
            exit_gracefully("ERROR: Could not get sensor base rate format!");

        const uint16_t sensor_decimation = sensor_base_rate / sensor_sample_rate;
        std::array<DescriptorRate, 2> sensor_descriptors = {{
            { data_sensor::DATA_ACCEL_SCALED,   sensor_decimation },
            { data_sensor::DATA_GYRO_SCALED,    sensor_decimation }
        }};
        if(commands_3dm::writeImuMessageFormat(*device, sensor_descriptors.size(), sensor_descriptors.data()) != CmdResult::ACK_OK)
            exit_gracefully("ERROR: Could not set sensor message format!");
        float sensor_to_vehicle_transformation_euler[3] = {0, 0, 0}; // M_PI, 0, 0
        if(commands_filter::writeSensorToVehicleRotationEuler(*device, sensor_to_vehicle_transformation_euler[0], sensor_to_vehicle_transformation_euler[1], sensor_to_vehicle_transformation_euler[2]) != CmdResult::ACK_OK)
            exit_gracefully("ERROR: Could not set sensor-to-vehicle transformation!");

        uint16_t filter_base_rate;

        if(commands_3dm::filterGetBaseRate(*device, &filter_base_rate) != CmdResult::ACK_OK)
            exit_gracefully("ERROR: Could not get filter base rate format!");
        const uint16_t filter_decimation = filter_base_rate / filter_sample_rate;

        std::array<DescriptorRate, 4> filter_descriptors = {{
            { data_filter::DATA_FILTER_TIMESTAMP, filter_decimation },
            { data_filter::DATA_FILTER_STATUS,    filter_decimation },
            { data_filter::DATA_ATT_QUATERNION , filter_decimation },
            { data_filter::DATA_ATT_UNCERTAINTY_QUATERNION, filter_decimation},
        }};

        if(commands_3dm::writeFilterMessageFormat(*device, filter_descriptors.size(), filter_descriptors.data()) != CmdResult::ACK_OK)
            exit_gracefully("ERROR: Could not set filter message format!");
        if(commands_3dm::writeComplementaryFilter(*device, true, false, 10., 1.) != CmdResult::ACK_OK)
            exit_gracefully("ERROR: Could not set north complement off!");
        if(commands_filter::writeAutoInitControl(*device, 1) != CmdResult::ACK_OK)
            exit_gracefully("ERROR: Could not set filter autoinit control!");
        if(commands_filter::reset(*device) != CmdResult::ACK_OK)
            exit_gracefully("ERROR: Could not reset the filter!");

        bool heading_align = true, gravity_align = true;
        float heading_time, gravity_time;
        if(commands_3dm::loadComplementaryFilter(*device) != CmdResult::ACK_OK)
            exit_gracefully("ERROR: Could not load filter complement!");
        if(commands_3dm::readComplementaryFilter(*device, &gravity_align, &heading_align, &gravity_time, &heading_time) == CmdResult::ACK_OK)
            std::cout << "heading algn: " << heading_align << "\n"; // the heading align is printed as 0

Here is the data collected by SensorConnect with/without North compensation:

and here is the data collected by settings of my code above:

The heading alignment disable is for the complementary filter in the sensor dataset.  Looking at your code, I think you are plotting the kalman filter data coming out of the filter descriptor set.  The complementary filter is provided as a backup to the kalman filter.  To disable the heading update for the kalman filter, you would use the Heading Update Control command and disable all heading inputs (value of 0).  Please reference the DCP here for more details: https://www.microstrain.com/sites/default/files/3dm-gx5-25_dcp_manual_8500-0065_reference_document.pdf

thank you very much, I solve it.


update index.html

adding shruti

@shruti-santosh  contribute to https://github.com/Ishaan28malik/Get-Dev-Resources instead


Fix selections in sorted database search dialogs

Close #90

Hi Lars,
Thanks for your pull request! Nice work!

Tina


Normalize mimetype

Description
Normalize mimetypes to make sure they can be understood by other libraries.
Fixes #CQ-4339915, #ASSETS-347
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] I have signed the Adobe Open Source CLA.
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

Replaced by https://github.com/adobe/asset-compute-pipeline/pull/44


Python 3 compatibility for C extension

Probably because of the try/except, I never noticed that the extension does not really work on Python 3. But the import error is triggered in the documentation build, so it needs some fixing.

:+1: works for me :smile:


Add 'exposure' parameter to light material

Here's a simple PR for https://github.com/appleseedhq/appleseed-max/issues/32

Hi @jjeziorski ,
This parameter is the synonym of m_light_power right? It's fine I think.
Have you tried to open scene saved with previous version of the plugin? We are trying to maintain backward compatibility.

Hi @usakhelo,
Thank you for review.
I'm not sure if I'd call it synonym of m_light_power, but indeed it modifies light power just using exponential scale.
Yes, I've checked opening old scene and it works for me. Anyway as far as I know, if you keep paramblock definition consistent (ids / types) than scenes are always backward compatible.

@jjeziorski this PR should be rebased. Would be great if @dictoon review this. If not let's merge this too, it looks fine to me.

@usakhelo I don't see any conflicts, so rebase should be trivial upon merge (github option).
Having said that, please let me know if there are any conflicts/issues.

Thanks for the contribution @jjeziorski!

Thanks @dictoon, I feel much better about this PR now :)

@jjeziorski it should rebase with no issues, you are right.


fix: allow localhost as notary server

The configration pattern no longer blocks localhost as a valid notary host path and also allows port specifications.
fixes #445

[x] PR is rebased to/aimed at branch develop
[x] PR follows Contributing Guide
[ ] Added tests (if necessary)
[x] Extended README/Documentation (if necessary)
[x] Adjusted versions of image and Helm chart in values.yaml and Chart.yaml (if necessary)

Codecov Report

Merging #446 (77cac16) into develop (165f518) will increase coverage by 0.95%.
The diff coverage is 94.73%.

@@             Coverage Diff             @@
##           develop     #446      +/-   ##
===========================================
+ Coverage    93.52%   94.47%   +0.95%
===========================================
  Files           15       22       +7
  Lines          633     1105     +472
===========================================
+ Hits           592     1044     +452
- Misses          41       61      +20

Impacted Files
Coverage Δ

connaisseur/__main__.py
0.00% <0.00%> (ø)

connaisseur/logging_wrapper.py
0.00% <0.00%> (ø)

connaisseur/kube_api.py
87.50% <66.66%> (ø)

...naisseur/validators/notaryv2/notaryv2_validator.py
80.00% <80.00%> (ø)

connaisseur/flask_application.py
91.35% <91.35%> (ø)

connaisseur/validators/static/static_validator.py
93.75% <93.75%> (ø)

connaisseur/util.py
95.45% <95.45%> (-4.55%)
:arrow_down:

connaisseur/config.py
97.29% <97.29%> (ø)

connaisseur/alert.py
97.97% <97.97%> (ø)

...naisseur/validators/notaryv1/notaryv1_validator.py
98.30% <98.30%> (ø)

... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a409447...77cac16. Read the comment docs.


Joern server data loss

I use the version of commit 7755b3adc7b9b9662cada774ef16d5bb6aacf49f.
When I run joern --server and use cpgqls_client to access the http api, I found that the result is not complete.
The result should include key-value pairs, in forms of key = value, but the key is lost like this(lost key should in the red boxes in the picture)

The source code of the script is
from cpgqls_client import CPGQLSClient, import_code_query, workspace_query

def get_function_ast(client,function_name,filename=""):
    if filename != "":
        query="cpg.method.filename(\"%s\").name(\"%s\").ast.l" % (filename,function_name)
    else:
        query="cpg.method.name(\"%s\").ast.l" % function_name
    return client.execute(query)

server_endpoint = "localhost:8080"
basic_auth_credentials = ("username", "password")
client = CPGQLSClient(server_endpoint, auth_credentials=basic_auth_credentials)

load_cpg_query="loadCpg(\"./workspace/libwebsockets/cpg.bin\")"
result=client.execute(load_cpg_query)

res=get_function_ast(client,"lws_ssl_capable_read_no_ssl")
print(res)

Thanks for the report @IzaiahSun, I've started looking into it, will post updates in the next couple of days


Allow changing search_mode during interactive search

Press Ctrl+s to change the search mode.
Closes #494

I tried this, and while I don't know if I'm sold on the UI here I do like the ability to switch the search functions. I would consider doing something like replacing the filter mode name with the new search mode name whenever the search mode is switched, and then reverting back on the next user input. The user is rarely going to switch search modes, and the filter mode is more important to provide context.
Alternately, consider switching to this new display for the rest of the search after the user tries to switch search modes for the first time.
Please don't go changing things on my account though. I'd suggest waiting for Ellie or Conrad to give their opinion.
Unrelated, but I like how you implemented the switching for search_mode, we should do that for filter_mode as well though probably not in this PR.

I don't know if I'm sold on the UI here

Agreed, even I am a bit hesitant to truncate the filter modes like this. Maybe show the search mode on the right side? Open to more suggestions on the display.

replacing the filter mode name with the new search mode name whenever the search mode is switched, and then reverting back on the next user input

That does sound useful, but I'm not sure how easy it'll be to implement. It might complicate the state logic a lot.

I like how you implemented the switching for search_mode

Thanks!

Hey! Sorry for being slow here 🙃
I like the idea for sure - but like @patricksjackson, the UX I am less of a fan of. The user will be switching search mode far less often (I suspect, anyway), so making it such a prominent part of the UI does not feel right to me. If we could put it elsewhere then it would work?
Or even having a character to indicate the search mode 🤔 ^ for prefix, * or % or similar for fuzzy, etc

replacing the filter mode name with the new search mode name whenever the search mode is switched, and then reverting back on the next user input

this sounded like the best approach to me. I have updated the code to do exactly this. Here's an asciicast for the new UI-

(I'm slightly irked by the lack of padding to the left of SRCH: when it is shown. Happy to discuss alternative words for it.

That looks great! Thanks for finishing this, I'm looking forward to using it :)
We should probably add some documentation for this. I'd do it myself but I'm headed out on vacation soon.


Fixing UI5-induced vulnerabilities

This PR fixes 13 of 20 vulnerabilities by bumping dependencies.
6 of the remaining 7 are caused by karma-ui5
1 is caused by karma-mocha-reporter
0 are caused by CAP
@drktfl could you take this to UI5 colleagues to get this addressed, please?
Keeping this state is not an option, when trust is a concern.
Btw: Why do we need all these dependencies at all?
Why does a frontend need such many backend things? :)

Btw: Why do we need all these dependencies at all?

I realized we do not need all of them. #74 reduces the number of dependencies.

Why does a frontend need such many backend things? :)

For building the app, running integration tests, ... 😉


Revert appLogic.allowInSubform requirement

Description
The subform concept was initially configured in a way where the app developer had to declare applicationmedadata->dataTypes->applogic.allowInSubform in order for the data model to be usable as a subform.
This functionality is no longer necessary: Slack thread.
Solution

[ ] Storage: Mark ApplicationLogic.AllowInSubform as obsolete
[ ] Frontend: Remove allowInSubform check from SubformValidator and all type+mock references: https://github.com/search?q=repo%3AAltinn%2Fapp-frontend-react allowinsubform&type=code
[ ] Docs: Remove reference from app metadata page

Everything in this issue has been implemented, but we need to wait until frontend releases the contents of #2682 before storage and docs can be merged.


feature: external table for private S3 bucket and ObjectStoreProvider

Background
as of 0.2.10, Seafowl is able to read external table with http(s) protocol, e.g.

files in public S3 bucket
pre-signed file in private S3 bucket

However, it fails to read private S3 files without pre-signed URL. E.g. register a S3 folder, which contains many files. File path could follow hive partitioned conventions.
Assume the seaflow config already use S3 object store in bucketA and the IAM user also have read access to another private bucket bucketB
[object_store]
type = "s3"
access_key_id = "******"
secret_access_key = "*"
region = "us-east-1"
bucket = "bucketA"

Expect
curl -i -H "Content-Type: application/json" localhost:8080/q -H "Authorization: Bearer ******" -d@- <<EOF
{"query": "CREATE EXTERNAL TABLE tripdata \
STORED AS PARQUET \
LOCATION 's3://bucketB/yellow_tripdata_2022-10.parquet'
"}
EOF

HTTP/1.1 200
Actual
HTTP/1.1 400 Bad Request
Internal error: No suitable object store found for s3://bucketB/yellow_tripdata_2022-10.parquet

Root Cause
the root cause looks like Seaflow register object store for seaflow:// and http(s):// schema
https://github.com/splitgraph/seafowl/blob/272c92d058a564f68299b5ef144e04c3f36b783d/src/config/context.rs#L148-L155, but does not recognize s3://.
Looks like using http(s):// will not respect the IAM credentials.
Proposal
Create a ObjectStoreProvider like DatafusionCLI (link). So it dynamically "register" future S3 bucket with same object store.
https://github.com/splitgraph/seafowl/blob/272c92d058a564f68299b5ef144e04c3f36b783d/src/config/context.rs#L147
There seems to be extra benefits that:

consolidate seaflow:// and s3:// handlers
simplify ANYHOST hacking for http(s)://

https://github.com/splitgraph/seafowl/blob/272c92d058a564f68299b5ef144e04c3f36b783d/src/object_store/http.rs#L372-L391

@jychen7 this is a great idea (apologies for not picking this up sooner)! Seafowl 0.4.3 is going to include this feature so you can test it out.
By default if you use s3:///gs:// to refer to an external CSV/Parquet file and also have setup an object store on S3/GCS, we will use the already provided credentials to query that file.
Otherwise, you can supply an explicit OPTIONS clause in the CREATE EXTERNAL TABLE statement that with appropriate keys for creating the needed client dynamically (access_key_id, secret_access_key and region for S3 and google_application_credentials for GCS).


What is in the webhook request?

I wasn't able to find any documentation on Contentful, but exactly does the request contain after an entry has been published? I'm curious to know if it contains the actual record IDs and content-types for each publish event.

@rossthedevigner — The documentation can be found here.
I will close this issue but feel free to re-open it if you think the documentation does not answer your question.
Thanks and have a great weekend!


Add T1/T2 description to docs

Context: Landsat data is segregated into Tier 1 and Tier 2 "collections" (not to be confused with the LS Collection 1 / Collection 2 nomenclature). Some DE Africa Landsat data services use T2 only, while some include both T1 and T2.
Applies to: all Landsat data and services as it is tiered at the surface reflectance, which is an input source for surface temperature
Actions to resolve this issue:

Add a separate descriptor page explaining the difference between T1 and T2 (can keep it short, link to the official Landsat website description of this) and why it is an important piece of metadata
Update all the Landsat data/service specs pages to include a row in the table to indicate whether T1 data is included or not (or if it can optionally be loaded, as is with lsX_sr using the collection_category argument)

https://www.usgs.gov/media/videos/landsat-collections-what-are-tiers
https://github.com/digitalearthafrica/deafrica-sandbox-notebooks/blob/main/Tools/deafrica_tools/datahandling.py#L391
Looks like the default is "load Tier 1 (T1) data only". It is confusing that the label is called collection_category but that does refer to Tiers, not to Collection1 / Collection 2.
I'm thinking the article could be linked from the data specs landing page as it applies to all our Landsat services, but there are some restructurings going on there so we don't need to lock it in yet.

PR merged https://github.com/digitalearthafrica/deafrica-docs/pull/49.


Validation seems to fail for boolean 'false'

Package version:
v10
Issue, Question or Enhancement:
Seems that validation fails on boolean types when value is passed as 'false', but works for 'true'.  Appreciate any insight - thanks!
$ curl -s -X POST -H 'Content-Type: application/json' http://localhost:8080/doFoo -d '{"fieldx":true}'
{
    "fieldx": true
}
$ curl -s -X POST -H 'Content-Type: application/json' http://localhost:8080/doFoo -d '{"fieldx":false}'
{
    "message": "Key: 'something.Fieldx' Error:Field validation for 'Fieldx' failed on the 'required' tag"
}

Postman submission gives the same result.
Code sample, to showcase or reproduce:
package main

import (
	"net/http"
	"encoding/json"

	"github.com/go-playground/validator/v10"
	"github.com/labstack/echo/v4"
	"github.com/labstack/echo/v4/middleware"
)

type (
	CustomValidator struct {
		validator *validator.Validate
	}

	something struct {
                Fieldx  bool `json:"fieldx" validate:"required"`
	}
)

func (cv *CustomValidator) Validate(i interface{}) error {
	return cv.validator.Struct(i)
}

func doFoo (c echo.Context) (err error) {
	req := new(something)
	if err := c.Bind(req); err != nil {
		return echo.NewHTTPError(http.StatusBadRequest, err.Error())
	}
	if err := c.Validate(req); err != nil {
		return echo.NewHTTPError(http.StatusBadRequest, err.Error())
	}
	dataString, err := json.Marshal(req)
        if err != nil {
                return c.String(http.StatusBadRequest, "failed to marshal JSON")
        }
	return c.String(http.StatusOK, string(dataString))
}

func main() {
	e := echo.New()
	e.Validator = &CustomValidator{validator: validator.New()}

	// Middleware
	e.Use(middleware.Logger())
	e.Use(middleware.Recover())

	// Routes
	e.POST("/doFoo", doFoo)

	// Start server
	e.Logger.Fatal(e.Start(":8080"))
}

the same issue

if true and false are valid values then you don't really need a validate tag because the unset/zero value will be false which itself is a valid value...
unless you want to require the value to be true always ?

Same issue here. validate:"boolean" fails whichever value (true/false) is passed in a json request

Same issue here. validate:"boolean" fails whichever value (true/false) is passed in a json request

I am having the same issue.

Same issue here. validate:"boolean" fails whichever value (true/false) is passed in a json request

I am having the same issue.

Me too. Anyone can help me?

Forgive the short reply, answering from phone.
Please read the documentation for required and it should make sense why false fails the required check AND this was explained above by @joebartels
Please let me know if I’m missing something and something else is wrong.

If you want to validate that the value is present you will have to use a pointer there.
So for the example given, the change should be something like this.
	something struct {
-                Fieldx  bool `json:"fieldx" validate:"required"`
+               Fieldx   *bool `json:"fieldx" validate:"required"`
	}

I will close this as answered.

You can use this way:
Name  *bool `json:"name" validate:"required"`

Use a pointer
thanks @zemzale

by default required will fail when the value is zero value. so with that in mind, it makes sense to fail on false
however, if true and false are valid values then you don't really need a validate:"required" tag at all
see https://pkg.go.dev/gopkg.in/go-playground/validator.v10#hdr-Required

This is extremely dangerous because if a caller forgets/does not to specify a field when it is required by contract then it would lead to unintended consequences.
This is very bad design.

@SharkFourSix not sure how long you’ve been using Go but there is no concept of an uninitialized variable and so ever variable will get a default value including when unmarshalling data into a struct.
and so to that end there are two options:

Design so the default types work in your favour eg. If not present the default makes sense.(which is good design in and language)
Provide the ability to detect the difference between presence and default values. This can be done in several ways including using a pointer and using a Nullable type like sql.NullBool or equivalent to keep this on the stack when possible in concert with Registering the custom type with validator.

I think you may be confusing bad design with the facts and semantics of the Go language. I would be glad to be wrong but as far as I know there is no other way to detect presence vs default.

@deankarn
Well, at the same time, it doesn't make sense to treat false as bad input because it's just as important as any other value.
That right there is where the bad design is. Without even looking at semantics or language behavior.
Assignment of default values to uninitialized variables is not exclusive to go. So this is not a problem with the underlying language.
You want to leave that behavior to the language and let it do what it does (throw a null pointer exception or panic or whatever or assign whatever initialization value the compiler may decide) because then and only then will the programmer realize that they did not provide a value to that value, instead of the library hijacking that behavior because it decided to interpret certain valid values as if the memory/variable containing that said value doesn't exist, which again, is bad design.
Also, it's not like the library is allocating the memory for me to worry about initialization. That's my business, not the library's.

@deankarn @SharkFourSix
I absolutely agree with SharkFourSix. While for strings this might make sense, it doesn't even make sense for integers, much less for booleans. I'm not arguing the design of Golang here, I'm arguing that to my brain the tag required implies just that the field needs to be in the json I'm verifying, not that it has to have a certain value. And requiring one to use a pointer when one does not need a pointer just in order to make sense of a tag is just puting the cart before the horse.


Ability to change OtpTextField cursor color

Added the ability to change the cursor color of the OtpTextField.

Also checking when this may be merged, code looks good.


Zulrah helper plugin JAD ICON change

hello can you put the zulrah helper icon back to the normal JAD image with the two colours its changed and become very confusing, thanks.

No, the image is generated. The prayers indicate the phase already.


Deleting the Cluster in cleanup

What this PR does / why we need it:
Deleting the clusters in Cleanup

Aetos link: http://aetos.pwx.purestorage.com/resultSet/testSetID/92120

There are testrail IDs missing for many of the tests. Please update them.

@nikolaypopov The functions that you see in backup_test.go are just wrappers around the actual backup APIs. The interface in backup.go has the method signature only for the backup APIs and nothing outside it.
The helper functions are mainly helping us with 2 things:

Build the request body and make the desired Backup API call
Handling the logic based on the response we get from the Backup API call

As for the issue of backup functions not returning anything, we are aware that we should not have any function that is not returning anything. At minimum we should return some error. We have this as part of out TODO list where we will make sure every helper function returns the error and the errors are handled appropriately.
Also, we will be splitting up the backup_test.go file into appropriate categories instead of cramming everything in one huge file. This is also in our TODO list.

Aetos DashBoard Link: http://aetos.pwx.purestorage.com/resultSet/testSetID/92699

Make sure that BackupLocationMap is changed to backupLocationMap everywhere


feat: workspace updates

This contains several workspace updates @ammarahm-ed - Using 8.8 prep on Android, which uses gradle 8 tasks looks like some adjustments may need to be made so leaving as draft until all clear:

npm start > demo.android

  System.err: Calling js method onClick failed
  System.err: Error: Building UI from XML. @plugin-demos/core.xml:11:7
  System.err:  > Module './view' not found for element './view:ModuleTestView'.
  System.err:    > Cannot read properties of undefined (reading 'moduleMetadata')

@ammarahm-ed I've pushed up some nice cleanups just now to get this in good shape. Fully works on iOS.
Android has this remaining case if get a chance to check:
packages/core/react-android/bridge/src/main/java/com/bridge/Packages.java:38: error: TurboReactPackage is abstract; cannot be instantiated
      new TurboReactPackage(),
      ^

When trying this branch, best to:
// ensure all deps are reset
yarn setup

// ensure demo hooks are regenerated
yarn demo:hooks

// now run android demo
yarn nx run demo:android

@NathanWalker I will try to fix it today.

@ammarahm-ed sorry for inconvenience realized tailwind.config was being gitignored, latest push fixes that.

Hey @NathanWalker just fixed the issue on android

Awesome @ammarahm-ed i think this can become final 2.0 (no more alpha) 🎉


Main

Description
There was a mismatch here where Organizations Id was marked as
non-optional, when in it is actually an optional field in the response.
See related
docs.
Documentation
Does this require changes to the Works Docs? E.g. the API
Reference or code snippets need
updates.
[ ] Yes

If yes, link a related docs PR and add a docs maintainer as a reviewer.
Their approval is required.
Description
Documentation
Does this require changes to the Works Docs? E.g. the API Reference or code snippets need updates.
[ ] Yes

If yes, link a related docs PR and add a docs maintainer as a reviewer. Their approval is required.

Hey @Yotrail, looks like you pulled in a whole bunch of commits you likely didn't mean to in this PR. Could you create a new PR with just the changes you intended?


adapter-static: specifying a kit.paths.base causes a 500 on build

Describe the bug
The title basically describes it all. I have two commits on this example repository where the only thing I change is uncommenting the paths directory. Otherwise, it's the default template that is created when using npm init.
Reproduction
You can see the offending line change here: hhhapz/kit-adapter-fail
Here's the commands I ran:
pnpm init svelte@next static-example
pnpm install -D @sveltejs/adapter-static@next
<edit svelte.config.js, as seen in the above repo>
pnpm build

Logs
Here is the full stack trace:
> Using @sveltejs/adapter-static
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at Proxy.<anonymous> (file:///home/h/code/web/static-example/.svelte-kit/output/server/app.js:1203:29)
    at async load (file:///home/h/code/web/static-example/.svelte-kit/output/server/entries/pages/todos/index.svelte.js:13:23)
    at async load_node (file:///home/h/code/web/static-example/.svelte-kit/output/server/app.js:1224:14)
    at async respond$1 (file:///home/h/code/web/static-example/.svelte-kit/output/server/app.js:1348:22)
    at async render_page (file:///home/h/code/web/static-example/.svelte-kit/output/server/app.js:1475:20)
    at async resolve (file:///home/h/code/web/static-example/.svelte-kit/output/server/app.js:1591:103)
    at async Object.handle (file:///home/h/code/web/static-example/.svelte-kit/output/server/app.js:1676:20)
    at async respond (file:///home/h/code/web/static-example/.svelte-kit/output/server/app.js:1561:22)
    at async visit (file:///home/h/code/web/static-example/node_modules/.pnpm/@sveltejs+kit@1.0.0-next.252_svelte@3.46.3/node_modules/@sveltejs/kit/dist/chunks/index5.js:534:20)
> 500 /todos
    at file:///home/h/code/web/static-example/node_modules/.pnpm/@sveltejs+kit@1.0.0-next.252_svelte@3.46.3/node_modules/@sveltejs/kit/dist/chunks/index5.js:407:11
    at visit (file:///home/h/code/web/static-example/node_modules/.pnpm/@sveltejs+kit@1.0.0-next.252_svelte@3.46.3/node_modules/@sveltejs/kit/dist/chunks/index5.js:581:5)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
 ELIFECYCLE  Command failed with exit code 1.

System Info
System:
    OS: Linux 5.16 Arch Linux
    CPU: (12) x64 AMD Ryzen 5 2600 Six-Core Processor
    Memory: 10.55 GB / 15.62 GB
    Container: Yes
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 17.3.0 - /sbin/node
    Yarn: 1.22.17 - /sbin/yarn
    npm: 8.4.0 - /sbin/npm
  Browsers:
    Chromium: 97.0.4692.99
    Firefox: 96.0.3
  npmPackages:
    @sveltejs/adapter-auto: next => 1.0.0-next.17
    @sveltejs/adapter-static: ^1.0.0-next.26 => 1.0.0-next.26
    @sveltejs/kit: next => 1.0.0-next.252
    svelte: ^3.46.0 => 3.46.3

Severity
blocking all usage of building SvelteKit, with the option enabled.
Additional Information
No response

Is this still an issue? I couldn't reproduce it on<EMAIL_ADDRESS>and<EMAIL_ADDRESS>I got a different error but it was for a legitimate 404.