SCA fixes for BN in EC

This PR addresses 2 related but independent side-channel issues
Fix a SCA leak using BN_bn2bin()
BN_bn2bin() is not constant-time and leaks the number of bits in the processed BIGNUM.
The specialized methods in ecp_nistp224.c, ecp_nistp256.c and ecp_nistp521.c internally used BN_bn2bin() to convert scalars into the internal fixed length representation.
This can leak during ECDSA/ECDH key generation or handling the nonce while generating an ECDSA signature, when using these implementations.  The amount and risk of leaked information useful for a SCA attack varies for each of the three curves, as it depends mainly on the ratio between the bitlength of the curve subgroup order (governing the size of the secret nonce/key) and the limb size for the internal BIGNUM representation (which depends on the compilation target architecture).
To fix this, we replace BN_bn2bin() with BN_bn2binpad(), bounding the output length to the width of the internal representation buffer: this length is public.
Internally the final implementation of both BN_bn2binpad() and BN_bn2bin() already has masking in place to avoid leaking bn->top through memory access patterns.
Memory access pattern still leaks bn->dmax, the size of the lazily allocated buffer for representing the BIGNUM, which is inevitable with the current BIGNUM architecture: reading past bn->dmax would be an out-of-bound read.
As such, it's the caller responsibility to ensure that bn->dmax does not leak secret information, by explicitly expanding the internal BIGNUM buffer to a public value sufficient to avoid any lazy reallocation while manipulating it: this is already done at the top level alongside setting the BN_FLG_CONSTTIME.
Finally, the internal implementation of BN_bn2binpad() indirectly calls BN_num_bits() via BN_num_bytes(): the current implementation of BN_num_bits() can leak information to a SCA attacker, and is addressed in the next commit.
Make BN_num_bits() consttime upon BN_FLG_CONSTTIME
This issue was partially addressed by commit 972c87dfc7e765bd28a4964519c362f0d3a58ca4, which hardened its callee BN_num_bits_word() to avoid leaking the most-significant word of its argument via branching and memory access pattern.
The commit message also reported:

There are a few places where BN_num_bits is called on an input where
the bit length is also secret. This does not fully resolve those
cases as we still only look at the top word.

BN_num_bits() is called directly or indirectly (e.g., through BN_num_bytes() or BN_bn2binpad() ) in various parts of the crypto/ec code, notably in all the currently supported implementations of scalar multiplication (in the generic path through ec_scalar_mul_ladder() as well as in dedicated methods like ecp_nistp{224,256,521}.c and ecp_nistz256.c).
Under the right conditions, a motivated SCA attacker could retrieve the secret bitlength of a secret nonce through this vulnerability, potentially leading, ultimately, to recover a long-term secret key.
With this commit, exclusively for BIGNUMs that are flagged with BN_FLG_CONSTTIME, instead of accessing only bn->top, all the limbs of the BIGNUM are accessed up to bn->dmax and bitwise masking is used to avoid branching.
Memory access pattern still leaks bn->dmax, the size of the lazily allocated buffer for representing the BIGNUM, which is inevitable with the current BIGNUM architecture: reading past bn->dmax would be an out-of-bound read.
As such, it's the caller responsibility to ensure that bn->dmax does not leak secret information, by explicitly expanding the internal BIGNUM buffer to a public value sufficient to avoid any lazy reallocation while manipulating it: this should be already done at the top level alongside setting the BN_FLG_CONSTTIME.
Benchmarks
I ran some benchmarks to make sure the second fix does not have an excessively negative effect on performance: the results suggest that the performance loss is negligible for RSA, DSA and ECDSA (ecp_smpl (p384), ec2_smpl (b283), ecp_nistp224, ecp_nistz256)
Before
version: 3.0.0-dev
built on: built on: Thu Aug  1 23:56:53 2019 UTC
options:bn(64,64) rc4(16x,int) des(int) aes(partial) idea(int) blowfish(ptr)
compiler: ccache clang-3.9 -fPIC -pthread -m64 -Wa,--noexecstack -Qunused-arguments -Wall -O3 -g -DDEBUG_UNUSED -DPEDANTIC -pedantic -Wno-long-long -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wswitch -Wsign-compare -Wshadow -Wformat -Wtype-limits -Wundef -Werror -Wmissing-prototypes -Wstrict-prototypes -Wswitch-default -Wno-parentheses-equality -Wno-language-extension-token -Wno-extended-offsetof -Wconditional-uninitialized -Wincompatible-pointer-types-discards-qualifiers -Wno-unknown-warning-option -Wmissing-variable-declarations -Qunused-arguments -Wno-string-plus-int -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DNDEBUG
                  sign    verify    sign/s verify/s
rsa 2048 bits 0.000601s 0.000018s   1664.4  56525.3
                  sign    verify    sign/s verify/s
dsa 2048 bits 0.000263s 0.000221s   3804.4   4534.6
                              sign    verify    sign/s verify/s
 224 bits ecdsa (nistp224)   0.0000s   0.0001s  21518.9  11205.1
 256 bits ecdsa (nistp256)   0.0000s   0.0001s  40744.8  14774.5
 384 bits ecdsa (nistp384)   0.0010s   0.0008s    980.0   1316.3
 283 bits ecdsa (nistb283)   0.0006s   0.0011s   1721.0    881.4

After
version: 3.0.0-dev
built on: built on: Fri Aug  2 00:28:07 2019 UTC
options:bn(64,64) rc4(16x,int) des(int) aes(partial) idea(int) blowfish(ptr)
compiler: ccache clang-3.9 -fPIC -pthread -m64 -Wa,--noexecstack -Qunused-arguments -Wall -O3 -g -DDEBUG_UNUSED -DPEDANTIC -pedantic -Wno-long-long -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wswitch -Wsign-compare -Wshadow -Wformat -Wtype-limits -Wundef -Werror -Wmissing-prototypes -Wstrict-prototypes -Wswitch-default -Wno-parentheses-equality -Wno-language-extension-token -Wno-extended-offsetof -Wconditional-uninitialized -Wincompatible-pointer-types-discards-qualifiers -Wno-unknown-warning-option -Wmissing-variable-declarations -Qunused-arguments -Wno-string-plus-int -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DNDEBUG
                  sign    verify    sign/s verify/s
rsa 2048 bits 0.000603s 0.000018s   1657.9  56646.6
                  sign    verify    sign/s verify/s
dsa 2048 bits 0.000262s 0.000224s   3810.3   4454.5
                              sign    verify    sign/s verify/s
 224 bits ecdsa (nistp224)   0.0000s   0.0001s  21348.0  11186.0
 256 bits ecdsa (nistp256)   0.0000s   0.0001s  40201.4  14706.5
 384 bits ecdsa (nistp384)   0.0010s   0.0008s    974.0   1316.5
 283 bits ecdsa (nistb283)   0.0006s   0.0012s   1693.5    866.4

Acknowledgments
Thanks to David Schrammel and Samuel Weiser for reporting this issue through responsible disclosure.
Responsible disclosure
The initial security assessment evaluated the severity of this issue to be quite low, hence the fix is released with a public PR.

Backporting
I haven't tested it yet, but this should cherry-pick cleanly to 1.1.1, so I would ask the reviewers to approve for 1.1.1 as well.
Being a security issue, should this backported to 1.0.2 and 1.1.0 as well?

#6640 strikes again! :-)
Switching from BN_bn2bin to BN_bn2binpad is a good plan regardless. I'm dubious of the BN_num_bits change. That OpenSSL needs it in BN_bn2binpad is a symptom of #6640. 972c87d intentionally did not do this because, as I wrote, "those cases should not call BN_num_bits at all". I'm not aware of any crypto primitive whose correct implementation would require such an operation.
That's not to say you shouldn't add bandaids and partial workarounds where they make sense, but I don't think this is the right bandaid. Adding new dependencies on BN_FLG_CONSTTIME is undesirable, especially one that turns an O(1) operation to O(N). See #8865. And as this is only needed for a bandaid, you shouldn't entrench it in a public API.
Instead, I would recommend just having BN_bn2binpad do this internally, at which point you don't even need the full complexity of the bit computation. We did roughly this in our very early days. (We didn't figure out how BIGNUM could be saved until years later.)
https://boringssl.googlesource.com/boringssl/+/6887edb917ba9ecbed85f9d63ec36638b1d1dbb6^!/
Note, however, this is a partial workaround. It does not fix the root issue. The computation that filled in bn->top already leaked it. To fix the actual leak, you need to address #6640. The issue description has a roadmap. Once you do that, you can retire the bandaid, as we did:
https://boringssl.googlesource.com/boringssl/+/3b3e12d81ee52ee293b4a2dc4f416b209188e744^!/

I'm dubious of the BN_num_bits change. That OpenSSL needs it in BN_bn2binpad is a symptom of #6640. 972c87d intentionally did not do this because, as I wrote, "those cases should not call BN_num_bits at all". I'm not aware of any crypto primitive whose correct implementation would require computing a secret bit length.
That's not to say you shouldn't add bandaids and partial workarounds where they make sense, but I don't think this is the right bandaid. Adding new dependencies on BN_FLG_CONSTTIME is undesirable, especially one that turns an O(1) operation to O(N). See #6078. [edit: fixed the bug link... GitHub completion] And as this is only needed for a bandaid, you shouldn't entrench it in a public API.

As you mentioned, one should not call BN_num_bits at all on private input. So the O(N) slowdown should only affect vulnerable code shown below. Note that BN_num_bitsis not only called in BN_bn2binpad but also in
ec_scalar_mul_ladder
ec_GFp_nistp521_points_mul
ec_GFp_nistp521_points_mul
More instances are below, although they are probably not exploitable since all of them are on a word boundary:
ec_GFp_nistp224_points_mul
ec_GFp_nistp224_points_mul
ecp_nistz256_windowed_mul
ecp_nistz256_windowed_mul
ec_GFp_nistp256_points_mul
ec_GFp_nistp256_points_mul
rsa_sp800_56b_derive_params_from_pq

Note, however, this is a partial workaround. It does not fix the root issue.

Agree.

The computation that filled in bn->top already leaked it.

Not neessarily. bn->top leaks the word length, BN_bn2bin leaks the byte length.

I'm dubious of the BN_num_bits change. That OpenSSL needs it in BN_bn2binpad is a symptom of #6640. 972c87d intentionally did not do this because, as I wrote, "those cases should not call BN_num_bits at all". I'm not aware of any crypto primitive whose correct implementation would require computing a secret bit length.
That's not to say you shouldn't add bandaids and partial workarounds where they make sense, but I don't think this is the right bandaid. Adding new dependencies on BN_FLG_CONSTTIME is undesirable, especially one that turns an O(1) operation to O(N). See #6078. [edit: fixed the bug link... GitHub completion] And as this is only needed for a bandaid, you shouldn't entrench it in a public API.

@sam1013 listed the other places where BN_num_bits() is called directly rather than indirectly.
I would say that the need to pad the secret scalar by adding once or twice the modulus is a legitimate use for a constant time BN_num_bits(): we can do that either conditional to the BN_FLG_CONSTTIME or adding a separate BN_num_bits_consttime().
@davidben , maybe for these use cases you would prefer a separate dedicate function to prevent branch prediction attacks on the check for the flag inside BN_num_bits()?
That seems sensible to me, and I would gladly implement that change, a downside is that it will probably make backporting more cumbersome?
Addressing all the places where we do if (bn->flags & BN_FLG_CONSTTIME) to replace them with dedicated functions, instead feels like a completely separate PR that would overlap with the ongoing discussions on how to refactor the BIGNUM layer.

I went ahead and pushed another fixup commit that separates BN_num_bits_consttime() into its own (static inline) function.
This seems cleaner anyway and gives us more flexibility if we decide that in some branches we want to make BN_num_bits_consttime() libcrypto-internal/public and replace calls to BN_num_bits() with explicit consttime ones.

Not neessarily. bn->top leaks the word length, BN_bn2bin leaks the byte length.

I can't think of a scenario where the word length is public, but the byte length is secret. It is true that an attack on the word length is harder to exploit, which is why, as I said, it can make sense to apply bandaids like this PR, with the understanding that it does not fix the problem. It is merely a partial workaround. The root problem is still #6640.
BN_num_bits, however, does not actually leak the number of bytes. BN_bn2bin does and, as I wrote, switching to BN_bn2binpad makes sense. BN_bn2binpad also leaks bytes and, yes, as I said, a bandaid makes sense there. For an example bandaid that doesn't rely on BN_num_bits, see this commit: https://boringssl.googlesource.com/boringssl/+/6887edb917ba9ecbed85f9d63ec36638b1d1dbb6^!/. (The history here, if I recall, was OpenSSL added BN_bn2binpad after we pointed that commit out to a team member.)

As you mentioned, one should not call BN_num_bits at all on private input. So the O(N) slowdown should only affect vulnerable code shown below.

That is close, but not quite what I said. One should not call it at all when the bit length is secret. This is not the same as private input. 972c87dfc7e765bd28a4964519c362f0d3a58ca4's commit message details a problematic case and, indeed, was why the change was made: RSA primes have public bit length (used to setup Montgomery reduction) but the remaining bits are secret. Thus it is necessary for BN_num_bits to avoid leaking bits beyond the high bit.

ec_scalar_mul_ladder, [...], ec_GFp_nistp256_points_mul

Ah yes, those bits. That's a consequence of problems with the EC API (scalars should not be stored as BIGNUMs), but so it goes. We use a dedicated EC_SCALAR type to capture the necessary bounds, avoid EC_POINT_mul in ECDSA internals, and then reduce once in the wrapper code:
https://boringssl.googlesource.com/boringssl/+/303f1a86ac9992d11da14794adeded55f8d10d7f/crypto/fipsmodule/ec/p256-x86_64.c#205
https://boringssl.googlesource.com/boringssl/+/303f1a86ac9992d11da14794adeded55f8d10d7f/crypto/fipsmodule/ec/ec.c#858
https://boringssl.googlesource.com/boringssl/+/303f1a86ac9992d11da14794adeded55f8d10d7f/crypto/fipsmodule/ec/scalar.c#24
There are two important things to note here:

BN_num_bits() >= 256 is a much less general operation than BN_num_bits_consttime. You really only need to check that a bunch of bits are zero, which is more efficient than BN_num_bits_consttime.

The check really should compare against the group order, not merely do a bit size check. We made that change in https://boringssl.googlesource.com/boringssl/+/eb9232f06ffae855a9b8c5dfaffcaaf256dfbaac. This is because point addition formulas are often incomplete in the P + P case. Depending on the implementation, this may be mitigated by a branch or just not computing the right answer altogether (e.g. ecp_nistz256_avx2_point_add_affine). That means security and sometimes even correctness depend on proving the doubling case cannot happen. This proofs rely on the scalar being fully reduced, not just being the right bit length. This is related to #9239. (I suspect there is an input that makes nistz256 produce the wrong answer. I'll see if I can construct it...) A constant-time comparison should also be more efficient than BN_num_bits_consttime.

Of course, all that is a lot of things to fix all at once, so I'm not saying you should block this PR on it all. But it further underscores the temporariness of this PR, which is why you don't want to entrench this in public API.

I would say that the need to pad the secret scalar by adding once or twice the modulus is a legitimate use for a constant time BN_num_bits(): we can do that either conditional to the BN_FLG_CONSTTIME or adding a separate BN_num_bits_consttime().

A properly constant-time EC implementation should never need to do that. That padding is only needed as a workaround for #6640 and tends to cause problems of its own. See #8906, #7486, #4576, and several others I forgot. Indeed such padding conflicts directly with the unreduced scalar logic discussed above, both in triggering the variable time BN_nnmod logic and breaking all the doubling proofs.
Fundamentally, such padding cannot be constant-time because it needs to content with an already leaked and invalid bn->top. Padding the scalar is just moving the same core timing leak from one place to another.
Fix #6640, and that all goes away.

@davidben, maybe for these use cases you would prefer a separate dedicate function to prevent branch prediction attacks on the check for the flag inside BN_num_bits()?

Yeah, as this is purely a temporary hack to further kick the can down the road on fixing #6640, it should be a separate function and detached from BN_num_bits and BN_FLG_CONSTTIME entirely. (I see the PR still connects it to BN_num_bits.)
That way you can remove it when the complete fix is applied. (You can probably even remove it sooner. As noted, all the mentioned uses here, even for temporary workarounds, need much weaker, more efficient operations.)

Forgot to touch on rsa_sp800_56b_derive_params_from_pq. As with the EC version of the check, you can implement it more efficiently by comparing against a public value. But, fair enough. That one is one a bit count that is not purely a workaround for #6640.
(Doing an actual constant-time RSA keygen is rather challenging and requires looking at the whole process pretty carefully. See the changes here.)

(I suspect there is an input that makes nistz256 produce the wrong answer. I'll see if I can construct it...)

I stared at this a bit. I believe it's fine, though I didn't rework the entire proof. Were the top several bits of P-256's order not all ones, I believe it would be possible to construct a problematic input, but they're all ones.

I wonder if this is also leaking some secret, the private key's top:
https://github.com/openssl/openssl/blob/d5f854291336c96a3d2379ecc8c29f00ef516ad9/crypto/bn/bn_rand.c#L266-L278
This is called with the private key each time a ecdsa or dsa signature is created.

@romen looks like my comment above is still unresolved. It would be nice to get this in for the release on Tuesday. Also is backport to 1.1.0/1.0.2 appropriate?

@romen looks like my comment above is still unresolved. It would be nice to get this in for the release on Tuesday. Also is backport to 1.1.0/1.0.2 appropriate?

I'm trying to handle all my pending PRs before the freeze on Monday.
I believe this would be appropriate for backporting to 1.1.0 and 1.0.2, I am just afraid is going to be difficult to backport because the scope of the changes kept growing steadily!

Switching from BN_bn2bin to BN_bn2binpad is a good plan regardless. I'm dubious of the BN_num_bits change. That OpenSSL needs it in BN_bn2binpad is a symptom of #6640. 972c87d intentionally did not do this because, as I wrote, "those cases should not call BN_num_bits at all". I'm not aware of any crypto primitive whose correct implementation would require computing a secret bit length.
That's not to say you shouldn't add bandaids and partial workarounds where they make sense, but I don't think this is the right bandaid. Adding new dependencies on BN_FLG_CONSTTIME is undesirable, especially one that turns an O(1) operation to O(N). See #6078. [edit: fixed the bug link... GitHub completion] And as this is only needed for a bandaid, you shouldn't entrench it in a public API.
Instead, I would recommend just having BN_bn2binpad do this internally, at which point you don't even need the full complexity of the bit computation. We did roughly this in our very early days. (We didn't figure out how BIGNUM could be saved until years later.)

@davidben I have been thinking about the comment above.
I see your point, and I could agree that we could avoid calling BN_num_bits altogether inside the library on confidential scalars, but this would not really protect external users that could end up calling BN_num_bits() on confidential scalars.
I would argue that while "Adding new dependencies on BN_FLG_CONSTTIME is undesirable, especially one that turns an O(1) operation to O(N)." is in general true, the benchmarking I posted in the description suggests it does not really have a noticeable impact on performance.

This cherry-picks cleanly to 1.1.1 and 1.1.0, I am adding the 1.1.0 label and ask for explicit reconfirmation for backporting (@mattcaswell @bernd-edlinger )
I will open a 1.0.2 backport PR separately.

re-confirmed, thanks!

Thanks @bernd-edlinger and @mattcaswell , please have a look also at #9793 for the 1.0.2 backport.
I will merge this later today!

Also, Travis red cross seems unrelated to me (seems like a random timeout while running the tests), but I am not entirely sure as my browser does not process the travis log correctly for some reason!
Can any of you confirm this as well?

Travis issue is another instances of:
The job exceeded the maximum log length, and has been terminated.

Or, in other words, not an issue with this PR.

Merged!
Thanks everyone for all the feedback, it really improved my original fix a lot!


Move RandomNumberStreams to RowGenerators

RandomNumberStream objects must be unique to the result set that they
are generating so that concurrent table generations don't affect each
other's results
@petroav
I apologize for the size of this PR.  It couldn't really be split up into commits that could compile on their own.  But here's a brief tour of what's going on so it's easier to follow.

remove RandomNumberStream from the GeneratorColumns and replace it with seedsPerRow
Have all the RowGenerators extend AbstractRowGenerator.  AbstractRowGenerator's constructor takes a table, and from there creates a RandomNumberStream for each column in the table and puts it in a map
everything that used to call GeneratorColumn.getRandomNumberStream() now calls getRandomNumberStream(GeneratorColumn) to retrieve the column from the map.
There were a number of methods that took a column as an argument and in the method went and grabbed the RandomNumberStream from it (createNullBitMap(), makeAddressForColumn() generatePricingFor<Sales|Returns>Table(), generateJoinKey()).  These needed to be changed to take a RandomNumberStream directly.  This obviously affects all callers as well.
Remove all reset methods.  not relevant anymore since the RowGenerators and RandomNumberStreams are no longer reused.

Looks good. Can you paste what you wrote here in the commit message as well so that someone looking at this in the future can get context?

Done. And merged


Upgrade date-fns to v2

Now using<EMAIL_ADDRESS>(UPGRADE GUIDE)
Format string changes:

DD => d
YYYY => yyyy
A => a (for AM/PM)

Remove fecha and use date-fns for all parsing and formatting
parse() requires a 3rd argument of type Date to be used as a "reference date" which defines values missing from the parsed dateString
parse() now always returns a Date, but it might be Invalid Date, so I added a few calls to isValid() where necessary.

This effort is a bit more complicated. Will follow up with specific recommendations regarding date/time parsing, formatting and utilities.


Integration with blaze ecosystem numba python to llvm compiler?

Hi Tensorflow team,
Thanks for the open source work!
I'm always on the lookout to replace lower level work with python code.   I'm therefore really excited about Numba a JIT compiler that can turn a subset of pure python syntax into really fast multithreaded compiled code across multiple backends.  There are also other cool projects that provide different abstractions and graph symbolic representations to different data backends.
http://blaze.pydata.org/
https://github.com/libdynd/dynd-python
https://github.com/blaze/blaze
I don't have any specific use cases in mind (maybe to efface c++ use?), but just wanted to make the team aware.  As a pydata user, I would love to see any  cooperation and synergy between all these amazing projects in the ecosystem.

So I did some digging in the documentation. It would be convenient to be able to write a new op with Numba.

:+1:

letting you guys know that numba has recently opensourced their numbapro paid library, so everyone can use 'gpu' and 'parallel' targets now! I'd be really excited to see if anyone would use any of those in tensorflow

I think that 'close' belongs over in tensorflow/skflow #47

Oops, sorry.

Our team has recently released a library for python-native TensorFlow operator definition and optimization: opveclib. It is not part of the numba ecosystem, but I believe could be an alternative solution to  @datnamer 's desired functionality.

Since we have py_func() you can utilize any python numpy based library, so this feature seems addressed. If you have a more targeted feature request, we'd be happy to talk about it. Thanks!


Remove Runtime shutdown hook when SDK instance is stopped

What does this PR do?
Otherwise it is possible to have a memory leak when SDK instance is stopped, because reference to the SDK instance still exists through the hook runnable.
Review checklist (to be filled by reviewers)

[ ] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
[ ] Make sure you discussed the feature or bugfix with the maintaining team in an Issue
[ ] Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

Codecov Report

Merging #1956 (fac1b96) into develop (0f0d085) will decrease coverage by 0.05%.
The diff coverage is 55.56%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1956      +/-   ##
===========================================
- Coverage    83.21%   83.16%   -0.05%
===========================================
  Files          483      483
  Lines        17584    17566      -18
  Branches      2605     2603       -2
===========================================
- Hits         14632    14608      -24
+ Misses        2257     2254       -3
- Partials       695      704       +9

Files
Coverage Δ

...ain/kotlin/com/datadog/android/core/DatadogCore.kt
81.25% <55.56%> (-2.60%)
:arrow_down:

... and 30 files with indirect coverage changes


FsMagic should avoid sign extension on i686

uint64(buf.Type) is ffffffff9123683e on i686 due to sign extension, so it is not equal to FsMagic(0x9123683E) and detection of btrfs using magic numbers always fails with prerequisites not met message

Can you please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
git commit --amend -s --no-edit && git push -f

@nponeccop what os are you running on I can't reproduce on my machine using btrfs

The problem only apprars if you build a 32-bit docker. Docker mostly works but the configuration is unsupported yet, mostly because registry infrastructure doesn't have a notion of image architecture.
My patch only helps with future portability to 32-bit architectures and helps those few people that run a patched docker on 32-bit machines despite unsupported. The HEAD doesn't run on 32 bits because of a line of protection code which has to be removed with a patch.
Here is the patch I use. Note "This will change in the future." I'm trying to bring the future closer.
--- a/daemon/daemon.go
+++ b/daemon/daemon.go
@@ -1177,9 +1177,11 @@ func (daemon *Daemon) ImageGetCached(imgID string, config *runconfig.Config) (*i

 func checkKernelAndArch() error {
        // Check for unsupported architectures
-       if runtime.GOARCH != "amd64" {
-               return fmt.Errorf("The Docker runtime currently only supports amd64 (not %s). This will change in the future. Aborting.", runtime.GOARCH)
-       }
+       /*
+               if runtime.GOARCH != "amd64" {
+                       return fmt.Errorf("The Docker runtime currently only supports amd64 (not %s). This will change in the future. Aborting.", runtime.GOARCH)
+               }
+       */
        // Check for unsupported kernel versions
        // FIXME: it would be cleaner to not test for specific versions, but rather
        // test for specific functionalities.

and build using
 export AUTO_GOPATH=1
./hack/make.sh dynbinary

this doesnt break existing functionality so LGTM

LGTM


Multiplylabs issue 1660

Ticket: #1660 #1661

Coverage decreased (-0.3%) to 94.155% when pulling d671b91706e6f549c0c91ba059d05bf51f2c43c9 on multiplylabs-issue-1660 into dcc32aaa9ed1febc13f98268816c35b0837b12e4 on master.

Coverage decreased (-0.3%) to 94.155% when pulling d671b91706e6f549c0c91ba059d05bf51f2c43c9 on multiplylabs-issue-1660 into dcc32aaa9ed1febc13f98268816c35b0837b12e4 on master.

Coverage decreased (-0.3%) to 94.155% when pulling d671b91706e6f549c0c91ba059d05bf51f2c43c9 on multiplylabs-issue-1660 into dcc32aaa9ed1febc13f98268816c35b0837b12e4 on master.


image backwards-compatibility, scalability tweaks

Seeing 404s from Twitter in particular in the logs,
putting this in place should resolve them.

Baseline against previous version of the site production-ci449-gf00a887-20220109t212938:
Requests      [total, rate, throughput]         15000, 50.00, 49.96
Duration      [total, attack, wait]             5m0s, 5m0s, 231.25ms
Latencies     [min, mean, 50, 90, 95, 99, max]  49.019ms, 127.397ms, 62.89ms, 270.863ms, 350.473ms, 824.69ms, 2.521s
Bytes In      [total, mean]                     152235000, 10149.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:15000
Error Set:

This branch at 80 concurrent users:
Requests      [total, rate, throughput]         15000, 50.00, 48.10
Duration      [total, attack, wait]             5m10s, 5m0s, 10.258s
Latencies     [min, mean, 50, 90, 95, 99, max]  49.804ms, 1.868s, 76.107ms, 10.423s, 10.66s, 11.255s, 30.002s
Bytes In      [total, mean]                     151453527, 10096.90
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           99.49%
Status Codes  [code:count]                      0:77  200:14923
Error Set:

This branch at 20 concurrent users:
Requests      [total, rate, throughput]         15000, 50.00, 49.99
Duration      [total, attack, wait]             5m0s, 5m0s, 56.644ms
Latencies     [min, mean, 50, 90, 95, 99, max]  48.75ms, 64.839ms, 58.943ms, 68.277ms, 79.533ms, 235.168ms, 772.179ms
Bytes In      [total, mean]                     152235000, 10149.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:15000
Error Set:

This branch at 30 concurrent users:
Requests      [total, rate, throughput]         15000, 50.00, 49.99
Duration      [total, attack, wait]             5m0s, 5m0s, 62.48ms
Latencies     [min, mean, 50, 90, 95, 99, max]  47.191ms, 335.719ms, 71.067ms, 800.147ms, 1.63s, 3.022s, 8.16s
Bytes In      [total, mean]                     152235000, 10149.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:15000
Error Set:

a 15 min run at 30 concurrent requests app engine config:
Requests      [total, rate, throughput]         45000, 50.00, 50.00
Duration      [total, attack, wait]             15m0s, 15m0s, 57.492ms
Latencies     [min, mean, 50, 90, 95, 99, max]  47.558ms, 66.052ms, 58.441ms, 67.756ms, 81.835ms, 279.275ms, 1.061s
Bytes In      [total, mean]                     456705000, 10149.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:45000
Error Set:

main site @80 for reference for 15 min:
Requests      [total, rate, throughput]         45000, 50.00, 50.00
Duration      [total, attack, wait]             15m0s, 15m0s, 61.561ms
Latencies     [min, mean, 50, 90, 95, 99, max]  49.343ms, 122.672ms, 60.776ms, 119.015ms, 316.099ms, 1.515s, 5.716s
Bytes In      [total, mean]                     456705000, 10149.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:45000
Error Set:

Testing on this branch again at 30:
Requests      [total, rate, throughput]         45000, 50.00, 50.00
Duration      [total, attack, wait]             15m0s, 15m0s, 50.797ms
Latencies     [min, mean, 50, 90, 95, 99, max]  48.486ms, 68.133ms, 59.609ms, 77.272ms, 100.83ms, 274.155ms, 1.029s
Bytes In      [total, mean]                     456705000, 10149.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:45000
Error Set:

Testing on this branch again at 40:
Requests      [total, rate, throughput]         45000, 50.00, 50.00
Duration      [total, attack, wait]             15m0s, 15m0s, 62.148ms
Latencies     [min, mean, 50, 90, 95, 99, max]  47.177ms, 75.981ms, 59.277ms, 96.452ms, 197.071ms, 338.729ms, 991.509ms
Bytes In      [total, mean]                     456705000, 10149.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:45000
Error Set:

Seems like a good number, calling that good enough!


Refresh tokens without Glimboi server

Glimesh fixed the refresh tokens requiring a secret token a while back. We should switch to refreshing it ourselves instead of being dependent on the server. We need to store the relevant auth info as before.

Added in v2.0


Support for multiple separators in the 'No proxy host' field: pipe (|), semicolon (;), newline, and comma (,)

[ ] This pull request is created in the jfrog/jenkins-jfrog-plugin repository.

Fix #106
The fix will update the no_proxy list by changing the separators from semicolons to commas, as required by the JFrog CLI.

@Nathan770,
Would you like to resolve the conflicts?


BUGFIX-#130: Navigation Errors

Description
Changed "a" tags with Link from "react router dom" which stops page reload.
Resolves: #130
Checklist

Before submitting this pull request, kindly verify that the ensuing checkpoints have been reached.

[Yes ] Have you adhered to the repository's defined coding convention rules?
[ Not req.] Have you updated the 'documentation.md' file with the method/function documentation?
[ Not required] Have you sent a message along with the result or response?
[ Not required] Have you used the try-catch technique?
[ Not req.] Has the method/class been added to the documentation (md file)?

I certify that I have carried out the relevant checks and provided the requisite screenshot for validation by submitting this pull request.
I appreciate your contribution.

@hereisSwapnil  I have made the changes.

@hereisSwapnil  Please merge it, it has no conflicts with current branch.


Request::getParsedBody returns false but must null|array|object

I send POST request from jQuery via XMLHttpRequest. In case of empty request body there is no Content-Type header. In such case Request::getMediaType returns null and Request::getParsedBody returns false.
This was broke after 3.2.0 update (https://github.com/slimphp/Slim/pull/1734)

Can you explain what you're trying to do please?

I send POST request from jQuery. It may contain body or not, depending on user data:
$.ajax({url: '/' + CHAMP.vote.id + '/vote/send/' + voteId + '/', method: 'POST', data: requestData, dataType: 'json'});

In controller I want to validate data from this request and use getParsedBody function for this:
public function sendVote(Request $request, Response $response, array $args)
{
        $validator = [
            'g-recaptcha-response' => Validator::optional(Validator::regex("/^[\w-]*$/"))
        ];

        try {
            $params = Functions::validate($validator, $request->getParsedBody());
        } catch (NestedValidationException $e) {
            throw new NotFoundException($request, $response);
        }

I expect to receive array or null, but receive false in case of empty POST body and no Content-Type header.

Tested with 3.2.2 and it returns false. PSR-7 says getParsedBody() says return value should be null|array|object. So it should not return false in any case?
$app->post("/test", function ($request, $response, $args) {
    var_dump($request->getParsedBody());
});

%curl --include --request POST http://<IP_ADDRESS>:8080/test
HTTP/1.1 200 OK
Host: <IP_ADDRESS>:8080
Connection: close
X-Powered-By: PHP/5.6.17
Content-Type: text/html; charset=UTF-8
Content-Length: 103

boolean false

What content-type would that request be @tuupola ?
I am guessing there's a logic error in here.
https://github.com/slimphp/Slim/blob/3.x/Slim/Http/Request.php#L972-L982
We should have a unit test for this as well.

May be add else for this (https://github.com/slimphp/Slim/blob/3.x/Slim/Http/Request.php#L972-L982):
else {
    return null;
}

I think we could return null, or set parsedBody to null.
The unit test that will fail:
public function testGetParsedBodyNoContentType()
    {
        $method = 'GET';
        $uri = new Uri('https', 'example.com', 443, '/foo/bar', 'abc=123', '', '');
        $headers = new Headers();
        $cookies = [];
        $serverParams = [];
        $body = new RequestBody();
        $request = new Request($method, $uri, $headers, $cookies, $serverParams, $body);

        $this->assertNull($request->getParsedBody());
    }

@p-golovin  The if can't have an else that returns null, then nothing would get parsed ;)

:+1: @mathmarques
//cc @akrabat @silentworks @codeguy
I think either an empty array or null is fine in this situation. I would prefer the array but that is more for my preference than what it actually needs to be. Unfortunately the spec is vague in this area.

@geggleto

The if can't have an else that returns null, then nothing would get parsed ;)

Yes, in case of non-existent mediaType. But I agree, that it is bad solution.

I would appreciate review and test of #1808 to see if this solves the problem.

Works fine now. Thanks.


Masking Tomcat Version

Masking Tomcat Version to reduce information disclosure.
Signed-off-by: Davis Benny<EMAIL_ADDRESS>

Is there a blanket setting to turn all versions off?

showReport="false" showServerInfo="false" property
ensures that Tomcat-related version or error report is not displayed.


cleanup(ci): pin deps to hashes

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind cleanup

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area CI

Does this PR require a change in the driver versions?
As per title. According to supply chain security standards we currently employ.

What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

NONE

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: LucaGuerra
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [LucaGuerra]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, LucaGuerra
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,LucaGuerra]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 5f9c9c601c5af3f7da80ca941c52b4736911c255

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP, LucaGuerra
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP,LucaGuerra]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


What is the current status?

What is the current status of this project? It looks really useful. Is anyone using it? Is it production-ready?

More here: https://groups.google.com/forum/#!topic/debezium/G0CDUKdXSVQ

Awesome, thanks


RFC: rms_current : switch to 3 decimals ?

I had a local patch in my homeassistant docker build to switch ElectricalMeasurement to 3 decimals
https://github.com/zigpy/zha/blob/dev/zha/application/platforms/sensor/__init__.py#L555
I added _decimals = 3
 homeassistant/components/zha/sensor.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/homeassistant/components/zha/sensor.py b/homeassistant/components/zha/sensor.py
index 9e98060667..e5664b9c76 100644
--- a/homeassistant/components/zha/sensor.py
+++ b/homeassistant/components/zha/sensor.py
@@ -461,6 +461,7 @@ class Battery(Sensor):
 class ElectricalMeasurement(PollableSensor):
     """Active power measurement."""

+    _decimals = 3
     _use_custom_polling: bool = False
     _attribute_name = "active_power"
     _attr_device_class: SensorDeviceClass = SensorDeviceClass.POWER

This let me get the RMS current from my Nous A1Z (zigbee 3) power plug at the milliamp level (even though it shows power below 3W as 0W).
Could at least the amps be given with 3 decimals in ZHA? ie ElectricalMeasurementRMSCurrent? https://github.com/zigpy/zha/blob/dev/zha/application/platforms/sensor/init.py#L635C7-L635C38
Do you know how to add a modified zha library to a local homeassistant docker build?

I think you can choose the number of decimal places to display within Home Assistant's frontend, no?

@puddy no you misunderstood the issue.
I can choose the number of "decimals to display" but "the stored by homeassistant decimals" below the "_decimals =1" set in zha are all "0" without this zha change ...
I am all for Zha setting 3 decimals to store and letting homeassistant "decimals to display" to the current one decimal and let the user decide if he wants to display more. Though if we could display 3 decimals for current in amps in homeassistant that would be good to, but not a priority.
The issue is that currently only one decimal is stored so changing the "decimals to display" is useless.
Should I send a PR with _decimals 3 for the ElectricalMeasurement superclass or only the current derivatives? (is there any other than rms_current?)
And how could I test my PR before sending it?

@puddly @TheJulianJES Should we remove all rounding in the lib completely? We can push this concern to the presentation tier... which in this case is HA. Then we won't have requests like this trickle in on a cluster by cluster or attribute by attribute case. thoughts?


Got error while trying float8

[rank0]:   File "/home/dungnt/tt/lingua/lingua/float8.py", line 23, in matmul
[rank0]:     output = torch._scaled_mm(
[rank0]:              ^^^^^^^^^^^^^^^^^
[rank0]:   File "/home/dungnt/tt/lingua/lingua/probe.py", line 404, in __torch_dispatch__
[rank0]:     out = func(*args, **kwargs)
[rank0]:           ^^^^^^^^^^^^^^^^^^^^^
[rank0]:   File "/home/dungnt/miniconda3/lib/python3.12/site-packages/torch/_ops.py", line 716, in __call__
[rank0]:     return self._op(*args, **kwargs)
[rank0]:            ^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: RuntimeError: cutlass cannot initialize

Did I forgot to install any required lib? Thanks.

It looks like the error might be due to FP8 (float8) support, which is available only on newer NVIDIA GPUs. We use FP8 rowwise matmul which is only supported specifically on NVIDIA Hopper. I've found a similar issue in pytorch/pytorch#130359 that might give a you a bit more context.


Avoid passing SHAMapNodeID's to sync filters

In version 2 SHAMap's, we sometimes have to call a sync filter without knowing the correct node ID. Since no sync filter uses the node ID, the simplest solution is to just stop passing the node IDs to the sync filters.
This is a trivial change. It doesn't change behavior.
Howard's SHAMapV2 code does not conflict with this, and I've already made a branch with this change ahead of his.

Current coverage is 64.49%

Merging #1577 into develop will increase coverage by +0.01% as of 5dbe5e2

@@            develop   #1577   diff @@
=======================================
  Files           895     895
  Stmts         63899   63899
  Branches          0       0
  Methods           0       0
=======================================
+ Hit           41206   41209     +3
  Partial           0       0
+ Missed        22693   22690     -3

Review entire Coverage Diff as of 5dbe5e2

Powered by Codecov. Updated on successful CI builds.

This is fine as is. :+1:
I left a comments on how we could simply the SHAMapSyncFilter interface. You may implement them at your discretion.

:+1:


Add a "Preferences" window and rebindable hotkeys

This PR will add a preferences window, along with a way to rebind hotkeys (and actually see the keybindings available in the designer).
~There are WPF KeyBinding, KeyGesture and MouseGesture classes available, but the KeyBinding and KeyGesture classes do not allow binding single key hotkeys - they throw an exception - hence the custom implementation in PolyGesture. Additionally, I am not sure how we would make sure to call InvalidateVisual() if using the commands directly in XAML, or map them to a key to make them rebindable.~
~There is an InputBinding class used to bind an InputGesture (the base class for MouseGesture and KeyGesture to an ICommand, but when attempting to add a PolyGesture, WPF threw an error - expecting either a KeyGesture or a MouseGesture, hence the need to implement my own class that derives from InputBinding.~ No longer needed. See comments below for reasoning.
~EDIT: This PR is now dependent on PR #206 (Localization data binding), due to the number of new localizations added within the new windows.~ (#206 has been merged).

[x] Implement HotkeyCommandManager
[x] Write tests for HotkeyCommandManager
[x] Implement ManageKeybindingsViewModel
[x] Create UI for ManageKeybindings.xaml
[x] Refactor hardcoded commands in AnnoCanvas

[x] Key commands (copy, paste, rotate, delete)

[x] Load Hotkeys from Settings

[x] Serialize to Settings
[x] Deserialize from Settings

[x] Localize all the categories and settings

[x] Manage Keybindings localized

[x] Update SerializationHelper - required to be able to serialize Types

[x] Move to Newtonsoft.Json
[x] Update Implementation

[x] Modify MainWindow UI

[x] Create Tools Menu Item
[x] Create Tools -> Preferences... Menu Item
[x] Remove "Extras" Menu Item and move under "Tools"
[x] Localize "Tools", "Preferences"

[x] License window for ManageKeybindings page icons

[x] Window Created
[x] ViewModel Created
[x] Localize Licenses Window

[X] Write tests

[x] Licenses window tests (if required, may be quite simple)
[x] Settings tests (loading/saving hotkeys)

This PR is already huge, with that in mind I'm going to leave the following ideas for separate PRs:

Create UI and models for other settings categories (Updates, Extras etc.).
Use mouse images in the window for recording hotkeys.
Refactoring mouse commands out of Anno Canvas - this is probably quite a large job to undertake. Specifically, it could be useful to separate out zoom (mouse wheel), and copy item (middle click). The default MouseBinding class only uses a MouseAction, which does not have handling for mouse wheel scrolling.
Multiple keybindings - allow the user to choose from a list of commands and add multiple hotkey configurations per command.

I have not checked it in detail, but I would propose to combine it with the introduction of a tabbed interface.
At first there would be one tab with the canvas and one tab with the settings.
Later it would be possible to have more tabs with more canvas open. So a user could edit multiple layouts. This was a request/discussion in the past.

That would be an even more major change - probably would want to go into a separate PR. The only things I'm moving about from the main window is a few menu items.

Ok, maybe you are right. I will add a new issue for the tabbed interface.
You say there are erros when binding to a single key. I made a test project and can bind to everything I want.
Can you explain why the code below is not possible for the designer?
    <Window.InputBindings>
        <!--Binding to single key-->
        <KeyBinding Key="F5"
                    Command="{Binding ShowMessageCommand}"
                    CommandParameter="You pressed F5" />
        <KeyBinding Key="R"
                    Command="{Binding ShowMessageCommand}"
                    CommandParameter="You pressed the key R" />
        <!--Binding to key + modifier-->
        <KeyBinding Key="F5"
                    Modifiers="Ctrl"
                    Command="{Binding ShowMessageCommand}"
                    CommandParameter="You pressed Ctrl+F5" />
        <!--Binding to multiple modifiers-->
        <KeyBinding Key="F5"
                    Modifiers="Ctrl+Shift"
                    Command="{Binding ShowMessageCommand}"
                    CommandParameter="You pressed Ctrl+Shift+F5" />
        <!--you can also use "Gesture" to combine Key + Modifiers in a string-->
        <KeyBinding Gesture="Control+Shift+F2"
                    Command="{Binding ShowMessageCommand}"
                    CommandParameter="You pressed Ctrl+Shift+F2" />
        <!--The same is true for bindings to the mouse-->
        <MouseBinding  MouseAction="LeftDoubleClick"
                       Command="{Binding ShowMessageCommand}"
                       CommandParameter="You double clicked the left mouse button" />
        <MouseBinding  Gesture="Control+RightClick"
                       Command="{Binding ShowMessageCommand}"
                       CommandParameter="You clicked the right mouse button and pressed Ctrl" />
    </Window.InputBindings>

    public class MessageCommand : ICommand
    {
        public void Execute(object parameter)
        {
            MessageBox.Show(parameter.ToString());
        }

        public bool CanExecute(object parameter)
        {
            return true;
        }

        public event EventHandler CanExecuteChanged;
    }

2 things -

Try binding to something like R, or Shift + any letter.
How we call InvalidateVisual() if binding like that - without calling InvalidateVisual, the canvas feels sluggish if we're calling something like Rotate (although it will update once the mouse is moved).

I did a binding to the key R (second example)
The call to InvalidateVisual() would be done by the class/ViewModel which contains the command. Maybe this class then needs a reference to IAnnoCanvas or a new interface which only contains the method InvalidateVisual().
So no matter who calls the command, the logic would be the same and separated from the UI.
There could be separate classes for each command and the command does not care who executes it, be it a key or a mouse button or a key + modifier.
I think the main difficult is where to add/bind those commands. To the main window, the canvas or somewhere else?
The remap could be done by offering several ComboBoxes with the available Keys and Modifiers. So the user selects them to his preference and then the binding would be done via code. So maybe the Views need an interface with a method like AddCommandBinding.
Then there would be the separation of UI and logic again.
Something that has a reference to these interfaces and the concrete implementations with their ViewModels would also be needed.

I think this needs some more thinking and discussion.
But in my opinion there is no need for the custom PolyGesture and PolyBinding. At least at the moment I don't see their value.

Yep - you're right - no issues binding to a single key - although weirdly the R key binding doesn't work when the XAML code is moved into the MainWindow, but does work from AnnoCanvas - so not a major issue, just something in MainWindow that must be capturing and handling the single key keypresses.
One more thing I've thought about - how do we know from the code which binding represents which command?
My current idea is that we'd need to create the input bindings through code, rather than in the XAML, using the AddCommandBinding system or whatever we end up going with using information from settings. That then means we can then map the instance of the binding to a key, and go from there.
Apparently we can't create single key keybindings from code, so that must have been where I got my information from:
new KeyGesture(Key.R, ModifierKeys.None); throws
NotSupportedException: 'None+R' key and modifier combination is not supported for KeyGesture.

From https://referencesource.microsoft.com/#PresentationCore/Core/CSharp/System/Windows/Input/Command/KeyGesture.cs,208
All normal and digit keys with no modifiers are apparently invalid:
///<summary>
/// Is Valid Keyboard input to process for commands
///</summary>
internal static bool IsValid(Key key, ModifierKeys modifiers)
{
    //
    //  Don't enforce any rules on the Function keys or on the number pad keys.
    //
    if (!((key >= Key.F1 && key <= Key.F24) || (key >= Key.NumPad0 && key <= Key.Divide)))
    {
        //
        //  We check whether Control/Alt/Windows key is down for modifiers. We don't check
        //  for shift at this time as Shift with any combination is already covered in above check.
        //  Shift alone as modifier case, we defer to the next condition to avoid conflicing with
        //  TextInput.

        if ((modifiers & (ModifierKeys.Control | ModifierKeys.Alt | ModifierKeys.Windows)) != 0)
        {
            switch (key)
            {
                case Key.LeftCtrl:
                case Key.RightCtrl:
                case Key.LeftAlt:
                case Key.RightAlt:
                case Key.LWin:
                case Key.RWin:
                    return false;

                default:
                    return true;
            }
        }
        else if ((key >= Key.D0 && key <= Key.D9) || (key >= Key.A && key <= Key.Z))
        {
            return false;
        }
    }
    return true;
}

XAML:
<UserControl x:Class="AnnoDesigner.AnnoCanvas"
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
             xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
             xmlns:d="http://schemas.microsoft.com/expression/blend/2008" xmlns:models="clr-namespace:AnnoDesigner.Models"
             mc:Ignorable="d"
             d:DesignHeight="200"
             d:DesignWidth="200"
             Focusable="True"
             ClipToBounds="True">
    <UserControl.InputBindings>
        <!--Binding to single key-->
        <KeyBinding  Key="R"
                    Command="{Binding ShowMessageCommand}"
                    CommandParameter="You pressed R">
        </KeyBinding>
    </UserControl.InputBindings>
    <Grid Name="StatisticsPanel"
          HorizontalAlignment="Right"
          Background="White">

    </Grid>
</UserControl>

Actual created gesture:

Code from https://referencesource.microsoft.com/#PresentationCore/Core/CSharp/System/Windows/Input/Command/KeyGestureConverter.cs,88

So KeyGesture is a little strange. Therefore don't use it from code behind.
Following does work:
    public partial class MainWindow : Window
    {
        private MainViewModel _viewModel;

        public MainWindow()
        {
            InitializeComponent();
        }

        private void Window_Loaded(object sender, RoutedEventArgs e)
        {
            _viewModel = DataContext as MainViewModel;

            //exception -> do not use Gesture
            //InputBindings.Add(new KeyBinding(_viewModel.ShowMessageCommand, new KeyGesture(Key.D, ModifierKeys.None)));

            //exception -> do not use Gesture
            //InputBindings.Add(new KeyBinding(_viewModel.ShowMessageCommand, Key.D, ModifierKeys.None));

            //exception -> do not use Gesture
            //InputBindings.Add(new InputBinding(_viewModel.ShowMessageCommand, new KeyGesture(Key.D, ModifierKeys.None)));

            //binding to sinlge key
            InputBindings.Add(new KeyBinding
            {
                Command = _viewModel.ShowMessageCommand,
                CommandParameter = "you pressed the key 'D'",
                Key = Key.D
            });

            //binding to sinlge key with multiple modifiers
            InputBindings.Add(new KeyBinding
            {
                Command = _viewModel.ShowMessageCommand,
                CommandParameter = "you pressed Ctrl + Alt + the key 'D'",
                Key = Key.D,
                Modifiers = ModifierKeys.Control | ModifierKeys.Alt
            });

            //Modifiers.None works when set as property
            InputBindings.Add(new KeyBinding
            {
                Command = _viewModel.ShowMessageCommand,
                CommandParameter = "you pressed the key 'F'",
                Key = Key.F,
                Modifiers = ModifierKeys.None
            });

            //MouseBinding works also
            InputBindings.Add(new MouseBinding
            {
                Command = _viewModel.ShowMessageCommand,
                CommandParameter = "you clicked the middle mouse button",
                MouseAction = MouseAction.MiddleClick
            });
        }
    }

Cool - we'll go with  that then, and I'll remove the PolyBinding and PolyGesture classes. I might just add a comment with a link to this comment thread as an explanation as to why we have to create them that way.

The LicensesWindow has been completed. I've included some screenshots below. Everything should be fully localized where needed.

View images

The chevron animates between states (it took me way too long to add this in Blend, but it looks nice)

I'll get those changes added in.
As the to Preferences window missing an OK/Save button, as we don't have a way of rolling the changed settings back, I guess I could add a Close button? I just think that could cause confusion, especially if certain settings pages might have a rollback feature in future (so, for example, could have an Apply/Save button and a Cancel button.
The Update Settings page I will just delete - moving other settings to the window will be a separate PR, as this one already so huge.

A Close button would be sufficient.
Maybe don't add it to the preferences window, but to the page as a workaround?

I don't think thats possible, unless I add the close command as another depedency to the view model. I guess maybe the best solution given the situation would be to put the close button button beneath the categories list - a little bit of an odd place, but then it will not interfere with anything added to a page, and responsibility of closing the window stays with window.

[ ]  Is it possible to select the first (and currently only one) ListViewItem/Page by default?
So opening the Preferences window directly displays the ManageKeyBindingsPage.
[ ] When closing the preferences window, the main window is put to the back of all other windows.

open Explorer
open AnnoDesigner (above Explorer window)
open Preferences
switch to Explorer
close Preferences -> Explorer is shown

[ ] Is it possible to fix the start/default position of the preferences window? Like always open in the center of the app?
[ ] Should the preferences window made modal?

I'll look into selecting the first item by default. It might just be a question of manually doing it in the view model, or there might be a way to do it by reading the first ListViewItem.
Not sure why this is occuring, I'll investigate. I'm not doing anything different to other windows, I'm just calling Close().
It should be opening in the center of the app already.
Not sure about this - I think my initial thought was that if we eventually put other settings in there there may be a case where you might want to check something on the MainWindow (shortcut, tweak in functionality etc.), which would not be possible if this was made Modal.

Just looking at the middle-click issue. Its not the command does not refresh, its that MiddleClick is hardcoded to clone and rotate an object within AnnoCanvas.xaml.cs. I could remove that functionality, but then that would be a breaking change, and could disrupt users' experiences within the app.
For now, I propose to leave it, as refactoring the hardcoded mouse commands in AnnoCanvas is a major piece of work, and is currently outside the scope of this PR.


RunByLine cannot work completely without a couple of debugpy fixes

THis one is a definite:
https://github.com/microsoft/debugpy/issues/226
This one is less necessary (just makes the user have to step twice)
https://github.com/microsoft/debugpy/issues/224

Note there's work to do on our side once 226 is fixed. We have to change the code that counts as user code to have an exclude list so we exclude other cells.
This glob pattern seems to work:
'**/*'

This was fixed


when using partial body matching, fetchMock.getOption is not there

I'm so excited about your changes in the library recently. partial body matching is a great one 🎊
I updated library to 9.10 to use that feature but my tests are broken now.
TypeError: Cannot read property 'getOption' of undefined error is this one.
How it's occurred?
Basically I'm writing some tests like this one:
expect(
        fetchMock.called(/v1\/comments/, {
          method: "POST",
          body: {
            comment: {
              body: comment.body,
            }
          }
        })
      ).toBeTruthy();

If I remove body, it works because the problem is finding fetchMock instance: https://github.com/wheresrhys/fetch-mock/blob/master/src/lib/generate-matcher.js#L225
I've no idea how can I affect this generateMatcher code and put default fetchMock instance there as second argument:
https://github.com/wheresrhys/fetch-mock/blob/master/src/lib/inspecting.js#L10
What do you recommend for this case?
fetchMock.filterCalls is in same situation while it's related to same generateMatcher function

Fixed in v9.1.1


Cannot connect to running instance of chrome

I run chrome with :
start chrome.exe --remote-debugging-port=9222 --no-first-run --no-default-browser-check
Then I run this script
const puppeteer = require('puppeteer-core');
(async()=> {
    const browser = await puppeteer.connect({
        browserURL : "http://localhost:9222"
    });

})()

Got an error:
connect ECONNREFUSED <IP_ADDRESS>:9222
I also try to open localhost:9222 or <IP_ADDRESS>:9229 in browsers: nothing happens.
Any idea ?
Regards

What do you see when you run netstat -a -o in your cmd terminal?
I've had this issue on a few work machines intermittently and it seems to be an issue with their firewall. You should check if <IP_ADDRESS>:9222 is displayed in the output in a listening/established state


Can't reset from inside of nested navigato

Current Behavior
Current definition of my navigators
const AppNavigator = StackNavigator({
  Home: { screen: SigninScreen },
  Tabs: { screen: TabNavigator }, // A Tab Navigator
});

const TabScreen = TabNavigator({
  Saldos: { screen: SaldosScreen },
  Operaciones: { screen: OperacionesNavigator }, // a Stack Navigator
  Retenciones: { screen: RetencionesScreen },
  Ejecutivo: { screen: EjecutivoScreen },
});

const OpNavigator = StackNavigator(
  {
    Operaciones: { screen: OperacionesScreen },
    OperacionesDetails: { screen: OperacionesDetails },
  })

The app start with the Home screen, after user enter the login details the app redirects to the TabNavigator and the user is only able to navigate through that set of screens. (back button is disabled).
I add a logout button
const logout = NavigationActions.reset({
  index: 0,
  actions: [NavigationActions.navigate({ routeName: 'Home' })],
});

const LogoutButton = ({ dispatch }) => (
  <TouchableOpacity
    onPress={() => dispatch(logout)}
    style={{
      flexDirection: 'row',
      justifyContent: 'flex-start',
      alignItems: 'flex-start',
    }}
  >
    <Icon
      name="log-out"
      style={{ color: 'white', marginRight: 10, marginTop: 5 }}
    />
  </TouchableOpacity>
);

LogoutButton.propTypes = {
  dispatch: PropTypes.func.isRequired,
};

export default connect()(LogoutButton);

But onPress the button this errors raise

There is no route defined for key Home.
Must be one of:
'Operaciones, OperacionesDetails'

That errors shows only tow possible routes. That routes belongs to the OpNavigator but the current active screen is Saldos that belongs to TabNavigator.
What is the correct way to setup the navigators and perform the stack reset?.
The desired stack is.
|- Home (always the first screen to be loaded)
|- LoadingScreen (used in the transition from Home to Tabs)
|-- Tabs
|---- Operaciones (is a tab that holds two other screens as a Stack Navigator)
Your Environment

software
version

react-navigation
git+https://github.com/react-community/react-navigation.git

react-native
https://github.com/exponent/react-native/archive/sdk-16.0.0.tar.gz

node
v7.7.2

npm or yarn
yarn 0.21.3 npm 4.1.2

Hi ! I've the same problem with the DrawerNavigator component. When I reset the router state the drawer menu is replaced by the nested route screen.

+1

Hi everybody, I solved my problem with the last release v1.0.0-beta.9.
I cannot reproduce the bug.
Be carefull the navigatorOptions have been flatten, you should rewrite all your options.

Ok I think this has been fixed

It hasn't been fixed, I having this issue on v1.0.0-beta.10

👍  how to reset it?

I agree with @dabit1 it still doesn't work with v1.0.0-beta.10 version.

This has been working for me since v9. Not a full reset, but it gets me to the right tab? Some experimentation with levels of Stack/Tab navigators -> routeNames was required:
onPress={() => (navigation.state.index === 0 && navigation.state.routes[0].routes[0].index === 1) ?
            navigation.navigate('DrawerClose') : navigation.dispatch(
              NavigationActions.navigate({
                routeName: 'Summary',
                params: {},
                action: NavigationActions.navigate({ routeName: 'TravelTab' })
              })
            ) }


Stop using CG types outside renderers

Using CGFloat/CGPoint/CGRect, and especially AffineTransform/CGAffineTransform in a cross-platform context seems inadvisable.
Resolves #5.
Checklist

[X] I've read the Contribution Guidelines
[X] My contributions are licensed under the Swift license.
[X] I've followed the coding style of the rest of the project.
[X] I've added tests covering my changes (if appropriate).
[X] I've verified that my change compiles and works correctly, and does not break any existing tests.
[ ] I've updated the documentation (if appropriate).

@swift-ci test

@swift-ci test


Bikeshedding syntax for named vs list vs raw arguments and imports

Putting discussion on this here for after #21 is finished and gets merged.
Previously suggested:
Give them all separate names
this is the option that binread started with, since I made raw stuff be args_tuple = ... and import_tuple(name: ty)
i don't like this option in the long run, but i'm willing to accept import_raw(name: ty) as a special case because i'm having a hard time coming up with other syntax for it that doesn't conflict with e.g. list syntax
Same name, different syntax
this is what @csnover proposed:

args { ... } -> named arguments
args(...) -> list arguments
args = ... -> raw arguments

this doesn't have an easy equivalent for import, because we need both a name and a type.

@jam1garner has proposed making args { ... } and args(...) both be named arguments, and list arguments become handled by raw syntax. Personally, i'm wary that this will cause even more issues for finding ways to make import work.
something i've thought of: if we make it import = pattern, import_ty = ty then it would work, but it feels really awkward to me. guess that shouldn't matter as much as we should be dealing with writing types less often than calling them, but still.

I just realized that we already support import_tuple = args: Args. regardless of how i don't like it, i guess that means that it's totally feasible syntax to use.


Added role checks in data source

Description

Added role checks in data source
Users with admin role can upload, delete documents
user with user role can only view the existing file

Issues
List the issue or RFC link this PR is working on. If there is no such link, please mark it as n/a.
Type of change
List the type of change like below. Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds new functionality)
[ ] Breaking change (fix or feature that would break existing design and interface)
[ ] Others (enhancement, documentation, validation, etc.)

Dependencies
List the newly introduced 3rd party dependency if exists.
Tests
Describe the tests that you ran to verify your changes.

@jaswanth8888
This feature is useful.
Are you still working on this?

Known issue: https://github.com/opea-project/GenAIExamples/issues/1270


VM/VMSS: reuse existing extension instance name on update

Otherwise, vm agent might reject for more than 2 handlers
There is subtle difference on extension API between VM and VMSS, and this change handles both

Codecov Report

Merging #2395 into master will decrease coverage by -0.02%.
The diff coverage is 57.57%.

@@            Coverage Diff             @@
##           master    #2395      +/-   ##
==========================================
- Coverage   72.19%   72.18%   -0.02%
==========================================
  Files         323      323
  Lines       18100    18113      +13
  Branches     2655     2661       +6
==========================================
+ Hits        13068    13075       +7
- Misses       4205     4208       +3
- Partials      827      830       +3

Impacted Files
Coverage Δ

...zure-cli-vm/azure/cli/command_modules/vm/custom.py
73.3% <57.57%> (-0.3%)
:x:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 964f569...6b8a025. Read the comment docs.


(rules): S3_BUCKET_SSL_REQUESTS_ONLY checks to literally

What is the problem?
S3_BUCKET_SSL_REQUESTS_ONLY fails even in cases where it should't
Reproduction Steps
I'm using
docker run \
  --rm \
  -v $(pwd):/src:rw,Z \
  --workdir /src \
  public.ecr.aws/r7q6h7y6/guard:1.0.1 \
  cfn-guard validate -r /wa-Security-Pillar.guard -d test.yaml

to test these three templates
# template 1
Resources:
  Bucket:
    Metadata:
      guard:
        SuppressedRules:
          - S3_BUCKET_LEVEL_PUBLIC_ACCESS_PROHIBITED
          - S3_BUCKET_PUBLIC_READ_PROHIBITED
          - S3_BUCKET_PUBLIC_WRITE_PROHIBITED
          - S3_BUCKET_LOGGING_ENABLED
          - S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED
          - S3_DEFAULT_ENCRYPTION_KMS
          - S3_BUCKET_VERSIONING_ENABLED

    Type: AWS::S3::Bucket
  BucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !Ref Bucket
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Deny
            Action: "s3:*"
            Principal: "*"
            Resource: "*"
            Condition:
              Bool:
                "aws:SecureTransport": false

# template 2
Resources:
  Bucket:
    Metadata:
      guard:
        SuppressedRules:
          - S3_BUCKET_LEVEL_PUBLIC_ACCESS_PROHIBITED
          - S3_BUCKET_PUBLIC_READ_PROHIBITED
          - S3_BUCKET_PUBLIC_WRITE_PROHIBITED
          - S3_BUCKET_LOGGING_ENABLED
          - S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED
          - S3_DEFAULT_ENCRYPTION_KMS
          - S3_BUCKET_VERSIONING_ENABLED

    Type: AWS::S3::Bucket
  BucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !Ref Bucket
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Sid: EnforceTls
            Effect: Deny
            Action: "s3:*"
            Principal: "*"
            Resource: "*"
            Condition:
              Bool:
                "aws:SecureTransport": false

# template 3
Resources:
  Bucket:
    Metadata:
      guard:
        SuppressedRules:
          - S3_BUCKET_LEVEL_PUBLIC_ACCESS_PROHIBITED
          - S3_BUCKET_PUBLIC_READ_PROHIBITED
          - S3_BUCKET_PUBLIC_WRITE_PROHIBITED
          - S3_BUCKET_LOGGING_ENABLED
          - S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED
          - S3_DEFAULT_ENCRYPTION_KMS
          - S3_BUCKET_VERSIONING_ENABLED

    Type: AWS::S3::Bucket
  BucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !Ref Bucket
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Sid: DeleteProtection
            Action: s3:Delete*
            Effect: Deny
            Principal: '*'
            Resource:
              - !GetAtt Bucket.Arn
              - !Sub "${Bucket.Arn}/*"
          - Sid: EnforceTls
            Effect: Deny
            Action: "s3:*"
            Principal: "*"
            Resource: "*"
            Condition:
              Bool:
                "aws:SecureTransport": false

What did you expect to happen?
I'd expect all commands to run without errors, as the TLS enforcement is present in all three
What actually happened?
Only the first one runs without errors, the others give
# template 2
Other
test.yaml Status = FAIL
FAILED rules
wa-Security-Pillar.guard/S3_BUCKET_SSL_REQUESTS_ONLY                    FAIL
---
Evaluating data test.yaml against rules wa-Security-Pillar.guard
Number of non-compliant resources 1
Resource = BucketPolicy {
  Type      = AWS::S3::BucketPolicy
  Rule = S3_BUCKET_SSL_REQUESTS_ONLY {
    ALL {
      Check =  %s3_buckets_policies_ssl_requests_only[*].Properties.PolicyDocument.Statement.* EQUALS  {"Action":"s3:*","Effect":"Deny","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}} {
        ComparisonError {
          Message          {
            Guard Rule Set: wa-Security-Pillar
            Controls: SEC-9.3,SEC-9.4
            Violation: Bucket policies must feature a statement to enforce TLS usage.
            Fix: Set a bucket policy statement to '"Action":"s3:*","Effect":"Deny","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}' .
          }
          Error           = Check was not compliant as property value [Path=/Resources/BucketPolicy/Properties/PolicyDocument/Statement/0[L:22,C:15] Value={"Sid":"EnforceTls","Effect":"Deny","Action":"s3:*","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}}] not equal to value [Path=[L:0,C:0] Value={"Action":"s3:*","Effect":"Deny","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}}].
          PropertyPath    = /Resources/BucketPolicy/Properties/PolicyDocument/Statement/0[L:22,C:15]
          Operator        = EQUAL
          Value           = {"Sid":"EnforceTls","Effect":"Deny","Action":"s3:*","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}}
          ComparedWith    = {"Action":"s3:*","Effect":"Deny","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}}
          Code:
               20.        Version: '2012-10-17'
               21.        Statement:
               22.          - Sid: EnforceTls
               23.            Effect: Deny
               24.            Action: "s3:*"
               25.            Principal: "*"

        }
      }
    }
  }
}

and
# template 3
Other
test.yaml Status = FAIL
FAILED rules
wa-Security-Pillar.guard/S3_BUCKET_SSL_REQUESTS_ONLY                    FAIL
---
Evaluating data test.yaml against rules wa-Security-Pillar.guard
Number of non-compliant resources 1
Resource = BucketPolicy {
  Type      = AWS::S3::BucketPolicy
  Rule = S3_BUCKET_SSL_REQUESTS_ONLY {
    ALL {
      Check =  %s3_buckets_policies_ssl_requests_only[*].Properties.PolicyDocument.Statement.* EQUALS  {"Action":"s3:*","Effect":"Deny","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}} {
        ComparisonError {
          Message          {
            Guard Rule Set: wa-Security-Pillar
            Controls: SEC-9.3,SEC-9.4
            Violation: Bucket policies must feature a statement to enforce TLS usage.
            Fix: Set a bucket policy statement to '"Action":"s3:*","Effect":"Deny","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}' .
          }
          Error           = Check was not compliant as property value [Path=/Resources/BucketPolicy/Properties/PolicyDocument/Statement/0[L:22,C:15] Value={"Sid":"DeleteProtection","Action":"s3:Delete*","Effect":"Deny","Principal":"*","Resource":[{"Fn::GetAtt":"Bucket.Arn"},{"Fn::Sub":"${Bucket.Arn}/*"}]}] not equal to value [Path=[L:0,C:0] Value={"Action":"s3:*","Effect":"Deny","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}}].
          PropertyPath    = /Resources/BucketPolicy/Properties/PolicyDocument/Statement/0[L:22,C:15]
          Operator        = EQUAL
          Value           = {"Sid":"DeleteProtection","Action":"s3:Delete*","Effect":"Deny","Principal":"*","Resource":[{"Fn::GetAtt":"Bucket.Arn"},{"Fn::Sub":"${Bucket.Arn}/*"}]}
          ComparedWith    = {"Action":"s3:*","Effect":"Deny","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}}
          Code:
               20.        Version: '2012-10-17'
               21.        Statement:
               22.          - Sid: DeleteProtection
               23.            Action: s3:Delete*
               24.            Effect: Deny
               25.            Principal: '*'

        }
      }
      Check =  %s3_buckets_policies_ssl_requests_only[*].Properties.PolicyDocument.Statement.* EQUALS  {"Action":"s3:*","Effect":"Deny","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}} {
        ComparisonError {
          Message          {
            Guard Rule Set: wa-Security-Pillar
            Controls: SEC-9.3,SEC-9.4
            Violation: Bucket policies must feature a statement to enforce TLS usage.
            Fix: Set a bucket policy statement to '"Action":"s3:*","Effect":"Deny","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}' .
          }
          Error           = Check was not compliant as property value [Path=/Resources/BucketPolicy/Properties/PolicyDocument/Statement/1[L:29,C:15] Value={"Sid":"EnforceTls","Effect":"Deny","Action":"s3:*","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}}] not equal to value [Path=[L:0,C:0] Value={"Action":"s3:*","Effect":"Deny","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}}].
          PropertyPath    = /Resources/BucketPolicy/Properties/PolicyDocument/Statement/1[L:29,C:15]
          Operator        = EQUAL
          Value           = {"Sid":"EnforceTls","Effect":"Deny","Action":"s3:*","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}}
          ComparedWith    = {"Action":"s3:*","Effect":"Deny","Principal":"*","Resource":"*","Condition":{"Bool":{"aws:SecureTransport":false}}}
          Code:
               27.              - !GetAtt Bucket.Arn
               28.              - !Sub "${Bucket.Arn}/*"
               29.          - Sid: EnforceTls
               30.            Effect: Deny
               31.            Action: "s3:*"
               32.            Principal: "*"

        }
      }
    }
  }
}

CloudFormation Guard Version
cfn-guard 2.1.0
OS
MacOS / Docker
OS Version
No response
Other information
No response

@benbridts curious, what happens if you remove the SID for EnforceTls? Does it still throw same error?

I didn't include the output, but I believe what you're asking is template 1, and that one works as expected

I also believe template 1 is invalid when actually implemented

Recent merge included the fix to the too literal issue. All tests referenced above were added into the guard test file to validate fix.


cannot build on Gradle 7.3.3

I hit the following error:
git clone ssh://github.com/MartinWitt/laughing-train.git
cd laughing-train
./gradlew compile

* What went wrong:
A problem occurred evaluating project ':code-transformation'.
> No signature of method: static java.util.List.of() is applicable for argument types: (String) values: [MartinWitt]
  Possible solutions: pop(), is(java.lang.Object), join(java.lang.String), min(), sum(), any()

Thoughts?

What JDK have you used for building it? I believe currently it needs jdk 16 because of https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/stream/Stream.html#toList() but i can change it to jdk11 if this program has a real use now.

I changed the minimal needed JDK version to 11.

Thanks.
However, with both JVM I get the same error:
export JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-amd64/
./gradlew -version
./gradlew build

export JAVA_HOME=/usr/lib/jvm/java-1.17.0-openjdk-amd64/
./gradlew -version
./gradlew build

The problem comes from build.gradle itself, https://github.com/MartinWitt/laughing-train/pull/28 is the fix

When I "gradle compile" in folder code-transformation, it works well, however, when I build from the root of the project I get the following idea.
Thoughts?
FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':rest-api:distZip'.
> Could not resolve all task dependencies for configuration ':rest-api:runtimeClasspath'.
   > Could not resolve project :code-transformation.
     Required by:
         project :rest-api
      > No matching variant of project :code-transformation was found. The consumer was configured to find a runtime of a library compatible with Java 8, packaged as a jar, preferably optimized for standard JVMs, and its dependencies declared externally but:
          - Variant 'apiElements' capability spoon-code-solver:code-transformation:0.0.4 declares a library, packaged as a jar, and its dependencies declared externally:
              - Incompatible because this component declares an API of a component compatible with Java 11 and the consumer needed a runtime of a component compatible with Java 8
              - Other compatible attribute:
                  - Doesn't say anything about its target Java environment (preferred optimized for standard JVMs)
          - Variant 'runtimeElements' capability spoon-code-solver:code-transformation:0.0.4 declares a runtime of a library, packaged as a jar, and its dependencies declared externally:
              - Incompatible because this component declares a component compatible with Java 11 and the consumer needed a component compatible with Java 8
              - Other compatible attribute:
                  - Doesn't say anything about its target Java environment (preferred optimized for standard JVMs)

Can you try adding to the build.gradle in /rest-api/ the following:
{
	toolchain {
		languageVersion = JavaLanguageVersion.of(11)
	}
}

Thanks.
It does not work out of the box.
Could not find method toolchain() for arguments
@.*** on project ':rest-api' of type
org.gradle.api.Project.

I had 2 edits in my post because of a mistake and I guess if you are answering GitHub issues by mail you are reading them inside a mail client?  As far as I known Github doesn't send edits by mail. In the code snippet was the javamarked by arrow missing.
--->java {
	toolchain {
		languageVersion = JavaLanguageVersion.of(11)
	}
}

Does the current master work for you, where I applied the toolchain fix?

Yes, it does work, thanks a lot.
What service is provided by the REST API?

The Rest API was/is just for demonstration purpose in the GitHub readme, but is still missing automatic deployment. See here https://github.com/MartinWitt/laughing-train#try-it-yourselfoften-out-of-date

ok, thanks.


[LFT] Lucky cat on a chain

Introduction
Hi, I first got into Eth way back when, when we started the first meetups in Berlin and I researched it and similar models - anybody remember Mastercoin? ^^- for a research job. Have been writing or involved withit since then in various capacities and intensities. Now I want to build something again and find out what's new that I don't know about.
Idea description

I have a lucky cat that I will probably bring, that has RGB led eyes that can broadcast messages (in morse) and can move with the arm. A speaker or similar could potentially be added too. It was always the plan to bring it to blockchain in some way. However, right now, it runs on an ESP8266 and that is not powerful enough to run a full light client for example. Something like Elk might work much better, if it fits, but it is not available yet either way. But in any case it could call to apis and do stuff with that. So far my ideas were to allow  for the broadcast of messages by sending transactions to a specific contract or maybe have a special action if you send a crypto kitty friend to play (if possible). Nyancat sound?
Furthermore, for Android there exist apps that decode morse (with varying success) but none any more for iOS.
So any ideas and implementations that make the cat cooler or more useful are welcome, as well as app devs that help with iOS development.
However this is only a side project or in case I don't find anything else to do, I do not want to keep people from more useful projects. But I still like the cat :)
Skillset
Ideas, embedded programmers, smart contract developers, crypto kitty experts,
What skills do you need, or think you might need to implement the idea?
Communication
@fl0_id:matrix.org /0x042b18530d605c52d3d26e6d5590efaad2e14ab52d53735af9271fec3e721a64ae7ddaf1388148e351720b9d3f5d89b2aae9bfd187da12f2555cacf42d32125777

It could optionally also compete for the Raiden IoT challenge if people are interested. https://github.com/ethberlinzwei/Bounties/issues/7


export JestAssertionError from expect

🚀 Feature Proposal
export JestAssertionError from expect to allow other packages to import it.
Motivation
Given that the expect package is modular and published as a standalone package on npm, I'm using it outside of jest and outside of tests. I'm currently trying to use it for implementing a custom mock server for a dev environment. To properly implement this, I need to distinguish expect exceptions from other exceptions, and thus need to do
if (err instanceof JestAssertionError) {}

However this is not currently possible seen as I can't import JestAssertionError from expect
Example
if (err instanceof JestAssertionError) {
  res.status(400).end()
} else {
  res.status(500).end()
}

Pitch
Why does this feature belong in the [Jest core platform]
It can't come from anywhere else

Yeah, we should do this. It might be a bit weird that it's on the global object, but maybe not? Might make sense for Jest 26 to have something like import { expect, JestAssertionError } from 'expect';. That should work fine


raftstore: use readable format for leader lease in logs

Hi,
This PR use a readable output format for leader lease in log messages, so that it's easier to debug problems related with leader lease, such as https://travis-ci.org/pingcap/tikv/jobs/217020787.
PTAL @zhangjinpeng1987 @BusyJay @siddontang

LGTM

LGTM


Encode/decode anim version 1.2 buffers

Hello SMG, are there any progress made on this? I have a game called Gundam Versus (PS4, 2017) that utilizes version 1.2 .nuanmb for every animation.
I've tried working by my own by looking at the c# code on how compressed buffers are parsed, but made no headway, since the way they store the bit count and stuff is quite different. May I ask are there any documentation regarding how the v1.2 parse the compressed bytes? If you'd like I could post the JSON files here.

Hello SMG, are there any progress made on this? I have a game called Gundam Versus (PS4, 2017) that utilizes version 1.2 .nuanmb for every animation.
I've tried working by my own by looking at the c# code on how compressed buffers are parsed, but made no headway, since the way they store the bit count and stuff is quite different. May I ask are there any documentation regarding how the v1.2 parse the compressed bytes? If you'd like I could post the JSON files here.

I don't have many examples of version 1.2 animations, so I haven't looked into it much. The only examples I have are uncompressed. I currently only have compression implemented for version 2.0 and 2.1. I'm not aware of any C# implementations that correctly implement anim compression for any version. The primary goal is to support tooling and applications for Smash Ultimate, but I'm open to supporting any other games that use the SSBH/HBSS format.
Compression Headers (2.1+)
https://github.com/ultimate-research/ssbh_lib/blob/6d03d979b6a5bfa03564c8f609057fdcc25eddbd/ssbh_data/src/anim_data/compression.rs#L29-L44
Example Compressed Buffer (2.1+)
https://github.com/ultimate-research/ssbh_lib/blob/6d03d979b6a5bfa03564c8f609057fdcc25eddbd/ssbh_data/src/anim_data/buffers.rs#L1651-L1705

I don't have many examples of version 1.2 animations, so I haven't looked into it much. The only examples I have are uncompressed. I currently only have compression implemented for version 2.0 and 2.1. I'm not aware of any C# implementations that correctly implement anim compression for any version. The primary goal is to support tooling and applications for Smash Ultimate, but I'm open to supporting any other games that use the SSBH/HBSS format.
Compression Headers (2.1+)
https://github.com/ultimate-research/ssbh_lib/blob/6d03d979b6a5bfa03564c8f609057fdcc25eddbd/ssbh_data/src/anim_data/compression.rs#L29-L44
Example Compressed Buffer (2.1+)
https://github.com/ultimate-research/ssbh_lib/blob/6d03d979b6a5bfa03564c8f609057fdcc25eddbd/ssbh_data/src/anim_data/buffers.rs#L1651-L1705

Edit: I also forgot to mention that I did decompile the function that parse the compressed bytes in IDA, but I have no idea what it is trying to do, if you need some reference on that let me know.

I'm not sure if reverse engineering the decompiled code is any easier than reverse engeineering the binary files themselves. Let me know if you find anything interesting.

Edit: I also forgot to mention that I did decompile the function that parse the compressed bytes in IDA, but I have no idea what it is trying to do, if you need some reference on that let me know.

I'm not sure if reverse engineering the decompiled code is any easier than reverse engeineering the binary files themselves. Let me know if you find anything interesting.

Thanks for the reply, I'll just provide an example of the v1.2 nuanmb file.
So here's my finding:
Please note that in this game most of the animation are done on Rotation except for the main control and base bones, and scale is nonexistent since it is mostly controlled by the script file (good old msc).
Also I have the .anim counterpart that was extracted from the older game (it's the same animation) for cross referencing.

I'll just do a header breakdown here:
0x00 - 0x04 - Flags, I'll delve into 0x0934 and 0x0944 for the most part since these two contains most of the data.
0x04 - 0x08 - Frame count (e.g. 40 frame)
0x08 - 0x0C - Unknown Float 1, always 1
0x0C - 0x10 - Unknown Float 2
0x10 - 0x12 - Unknown Short, always 2
0x12 - 0x14 - Unknown Short, I am suspecting this is the bitcount that's used in decompression
For 0x0934:
0x14 - 0x20 - XYZ keys for translation in Float, at the first key (e.g. 0)
0x20 - 0x2C - XYZ keys for translation in Float, at some random key between first and final (e.g. 34)
0x2C - 0x38 - XYZ keys for translation in Float, at the final key (e.g. 40)
0x38 - End - The compressed data
For 0x0944:
0x14 - 0x44 - 12 different Float that's related to rotation, could be radian as the range is always between 0 - 2.
0x44 - End - The compressed data
I am not entirely sure how do I apply the same decompression logic that we have in 2.0, since there's a lot of information missing that 2.0 decompression relies on (e.g. bit count, start and end for lerp function, default values etc.)
I have added you on Discord if you don't mind me disturbing you there.
Thank you for your time, and just let me know if you have any insight on these based on your expertise dealing with other versions.
Front.zip

Thanks for the reply, I'll just provide an example of the v1.2 nuanmb file.
So here's my finding:
Please note that in this game most of the animation are done on Rotation except for the main control and base bones, and scale is nonexistent since it is mostly controlled by the script file (good old msc).
Also I have the .anim counterpart that was extracted from the older game (it's the same animation) for cross referencing.

I'll just do a header breakdown here:
0x00 - 0x04 - Flags, I'll delve into 0x0934 and 0x0944 for the most part since these two contains most of the data.
0x04 - 0x08 - Frame count (e.g. 40 frame)
0x08 - 0x0C - Unknown Float 1, always 1
0x0C - 0x10 - Unknown Float 2
0x10 - 0x12 - Unknown Short, always 2
0x12 - 0x14 - Unknown Short, I am suspecting this is the bitcount that's used in decompression
For 0x0934:
0x14 - 0x20 - XYZ keys for translation in Float, at the first key (e.g. 0)
0x20 - 0x2C - XYZ keys for translation in Float, at some random key between first and final (e.g. 34)
0x2C - 0x38 - XYZ keys for translation in Float, at the final key (e.g. 40)
0x38 - End - The compressed data
For 0x0944:
0x14 - 0x44 - 12 different Float that's related to rotation, could be radian as the range is always between 0 - 2.
0x44 - End - The compressed data
I am not entirely sure how do I apply the same decompression logic that we have in 2.0, since there's a lot of information missing that 2.0 decompression relies on (e.g. bit count, start and end for lerp function, default values etc.)
I have added you on Discord if you don't mind me disturbing you there.
Thank you for your time, and just let me know if you have any insight on these based on your expertise dealing with other versions.
Front.zip

I've updated the ssbh_data code to print out more of the anim 1.2 buffer types based on what you posted. You can run it as cargo run -p ssbh_data_json front.nuanmb out.json to print the information to the console. I've added three new header values that seem to use some sort of compression. The next step is to figure out how many items are in the compressed buffers. You can calculate the size per element since the buffer size is known. The floats at the beginning of the buffer may serve the same purpose as the min, max, and default values in version 2.0.

Thanks, I can visualize it better with the printout.
For header 0x4308 (V12Test3), there's another 9 float after the keyframe (unk2) and before the start of the compression buffers.
Interestingly, I think 0x4409's unk5 is definitely float related to XYZW, since on UDE_R originally does not have any rotation in XY, and the values show the same.
unk5:
0.0, 0.0, -0.416197, 0.909275,
0.0, 0.0, -0.314448, 0.949275,
0.0, 0.0, -0.186356, 0.982482
As for figuring out how many items are there in the compressed buffers, could you elaborate more? I cannot find a pattern between the buffer size and the number of keyframe.
I do notice that the buffer size will always align to 8 byte sizes, was this the case for v2.0+ anims as well?

I do see a pattern with unk4 and the number of non constant float components. The bits per entry for version 2.0 is summed over the bits used to represent each field. If a value does not change for the track, it takes 0 bits. Quaternions are assumed to be normalized, so the fourth component can be inferred using only a single bit for the sign. It looks like version 1.2 could be using similar logic for calculating the bits per entry.
I'm not sure how to calculate the number of elements in the compressed buffer. Version 2.0+ uses offsets into a single buffer, and I don't believe there are any alignment requirements on the data size for each track.

Thanks for the reply, as for the compression buffer parsing I think I'll do more investigation on it when I have access to my PS4 by next week, the decompiled code should give us some clue as how the compressed data is read and parsed.
That being said, I've already honed in on one subroutine that reads from the compressed buffer, and have decompiled the assembly code. I am not sure if the decompiled subroutine is the common decompression logic for every 1.2 header, but weirdly enough it reads 8 compressed bytes for each item.
I'll post the code here just in case you recognize some of the logic that is similar with v2.0+ way of decompression.
As for the animation data that's stored in the nuamb that I've sent you, the decompressed values should correspond to the animation data that I've attached here. (Same animation from older game).
I'll keep you posted if I find anything new, but I'll probably need to wait until next week before I can dive deeper into the decompiled code since I don't have access to my PS4 for now.
Really thank you for your time in helping out, this has cleared a lot of confusion for me and now I have a better direction to do more investigation.

As for the animation data that's stored in the nuamb that I've sent you, the decompressed values should correspond to the animation data that I've attached here. (Same animation from older game).

Assuming these values are accurate, you should be able to approximate what the decompressed values should be for all the transform tracks. The collada file is using transformation matrices that can be decomposed to a translation vector, scale vector, and rotation quaternion. Here's the result using glam in Rust. This seems to match up with the values stored in the header. The first value seems to be the first frame, the second value is somewhere in the middle, and the last value is the last frame. There's going to be a lot of rounding errors after converting to DAE and then decomposing the matrix again, so I wouldn't expect any of the values to match exactly.
use glam::Mat4;
// transforms is an array of matrix values from the collada DAE
for values in transforms {
    let transform = Mat4::from_cols_slice(&values).transpose();
    println!("{:?}", transform.to_scale_rotation_translation().1);
}

The result for the above code on the UDE_R transformation from the DAE.


Error compiling for board NodeMCU 1.0 (ESP-12E Module).

whenever I try to compile the code and upload it to my nodeMCU it shows this error code. and I am a newbie at this.please help me. Sorry for my weak English.
Error:
Arduino: 1.8.5 (Windows 10), Board: "NodeMCU 1.0 (ESP-12E Module), 80 MHz, Flash, 4M (1M SPIFFS), v2 Lower Memory, Disabled, None, Only Sketch, 115200"
D:\ESP8266\botton_2\botton_2.ino:88:25: fatal error: SimpleTimer.h: No such file or directory
#include <SimpleTimer.h>
                     ^

compilation terminated.
exit status 1
Error compiling for board NodeMCU 1.0 (ESP-12E Module).
This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.

Posted two times