[WIP] Improve SCons AddOption option handling

Currently, an option-with-arg added with AddOption has some problems. If the form --opt=arg is used, handling is correct, but if the form --opt arg is used (whitespace separator), arg ends up considered as one of the targets supplied on the command line.  If the option is specified with nargs > 1, there will always be whitespace as separators, and so these always fall into the hole, and depending on the rest of the command line, probably will trigger an error that the option required nargs arguments.
Work in progress: test cases added, solution still needed.
Closes #2748
Closes #2805
Closes #2977
Signed-off-by: Mats Wichmann<EMAIL_ADDRESS>Contributor Checklist:

[X] I have created a new test or updated the unit tests to cover the new/changed functionality.
[ ] I have updated master/src/CHANGES.txt directory (and read the README.txt in that directory)
[ ] I have updated the appropriate documentation

I've rejiggered reparse_local_options in SConsOptions.py to change the flow and pick the following arguments out of the "leftover arguments" list.  That fixes the part of the problem where the option gets separated from the arguments and so not processed properly, but it leaves the problem where the option-args still go on the targets list.  That happens in Main, and I don't see how to avoid it: the logic there is if it doesn't start with a dash, and contains an = sign, it goes on the value arguments list, if it doesn't it goes on the targets list.

I actually think that disallowing space separators and nargs >1 is the cleanest fix.  (for the latter case, --opt=foo,bar,baz and parse the single "word" of option-argument yourself would work).

Some more poking today and I think this is very difficult problem to solve because of the chicken-and-egg situation involved.  The command line is processed once, before the SConscripts are read, because you have to. (a) you might have an argument that tells you a different file to use than SConstruct, and (b) you have to gather up the targets and command-line Variables settings, because both of those things need to be available for checking/acting on in SConscripts.  Currently the  SCons-overridden _process_long_opt just ignores an option it doesn't recognize, instead of erroring; that defers the processing until later. When you're reading the SConscripts, and you see an AddOption, you call a function which tries to see if that option was on the command line, and if so collects it.  But because at this point the two other classes of unused arg words had to already be made available for the SConscript context, it's already too late: if you are going to add --foo, and --foo something was on the command line, by the time you know to process that, something has already been presented as part of COMMAND_LINE_TARGETS and BUILD_TARGETS. The only thing that actually works is if the option-argument was attached to the option word by the = sign.
This doesn't actually seem to have broken hundreds of users as one might expect if there were lots of users of the functionality, but there are several filed issues on the topic - three noted in the original message here. If we want to make it consistent I'm more convinced than before that the only clean way to do that is what is in the previous comment. I've confirmed the no-more-than-one-option-argument case needs only a two-line patch in SConsOptionParser.add_local_option and the other one is probably a two-line patch in reparse_local_options in the same class.  Of course there would be a doc impact as well.
I can't see how this could break many people: the more-than-one option-argument case can't work (the tests added here show that); the one-option-argument-with-space case can work, although it's basically by accident - in some configurations it will fail but in some it works - and we still have that option-argument made available as a target as well, which is wrong. But is it conceivable there are users who depend on the one-arg case who don't look at the *TARGET variables and so are proceeding.  So some thought is needed.

This one is force-updated to keep from going completely stale. I'm going to close the PR presently, and keep the tracking branch.

I'm going to close this one. I'll resubmit the testcases so we have them around.


WebView fixes

Summary of the Pull Request
This PR fixes the blank Settings window when the PowerToysSettings.exe is running elevated.
Improved error handling for the WebView control initialization.

PR Checklist

[x] Solves one of the two bugs that are reported in https://github.com/microsoft/PowerToys/issues/243
[x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

Detailed Description of the Pull Request / Additional comments
At startup, checks if PowerToysSettings.exe is running elevated and in that case drop the access token admin privileges.
If a fatal error occur during the WebView initialization, shows an error message instead of showing a blank window.
Code refactoring for easier integration of the upcoming WebView2 support.
Fix some warnings.
Coding style.
Note for the reviewers: I made several small commits to isolate the important changes from the code refactoring/variables and functions renaming.

Validation Steps Performed
Manual test.

@bzoz thanks for the review!


Fix - Globe Component with Memory leak

The memory leak was fixed by removing the resize event listener when the component is unmounted.
In the original code, the resize event listener was added within the onMount function, but it wasn't removed. This means that even after the component is unmounted, the listener would still be active, which can cause memory leaks if the component is mounted and unmounted repeatedly.
I tried to comment on the changes I made, and I also added select-none to the word 'Globe' in the component.

For future reference, or now, I would recommend just using svelte:window as it takes care of removing the listeners for you, and more.
See: https://svelte.dev/docs/special-elements#svelte-window

The memory leak was fixed by removing the resize event listener when the component is unmounted.
In the original code, the resize event listener was added within the onMount function, but it wasn't removed. This means that even after the component is unmounted, the listener would still be active, which can cause memory leaks if the component is mounted and unmounted repeatedly.
I tried to comment on the changes I made, and I also added select-none to the word 'Globe' in the component.

Thanks


feat) Add 'opRef is valid' message to pkg validate cmd when valid; cl…

…oses #153
feat) Run pkg validate on the op before running the op; closes #148
feat) Make sub ops errors more clear from parent ops; closes #106

approve

approv

approved

approv

approv

approv

approved

approve

approv


Not available in anaconda or conda-forge sources

Please could you add this to anaconda and/or conda-forge sources? Thank you.

Hi @ibby-dev thanks for reporting. I have initiated the process in https://github.com/conda-forge/staged-recipes/pull/18187

@ibby-dev can you check now if it is available?

Its part of conda-forge repo now.


Fix V3041, V3105 warnings from PVS-Studio Static Analyzer

Fix V3041, V3105 warnings from PVS-Studio Static Analyzer
Hello. I'm a member of the Pinguem.ru competition on finding errors in open source projects. A bug, found using PVS-Studio.
About your warnings: V3041, V3105.
Warnings with High priority:
V3105 The 'Topic' variable was used after it was assigned through null-conditional operator. NullReferenceException is possible. Message.cs 22
V3041 The expression was implicitly cast from '?' type to 'double' type. Consider utilizing an explicit type cast to avoid the loss of a fractional part. An example: double A = (double)(X) / Y;. ChatMessage.cs 270
V3041 The expression was implicitly cast from '?' type to 'double' type. Consider utilizing an explicit type cast to avoid the loss of a fractional part. An example: double A = (double)(X) / Y;. ChatMessage.cs 274
V3041 The expression was implicitly cast from '?' type to 'double' type. Consider utilizing an explicit type cast to avoid the loss of a fractional part. An example: double A = (double)(X) / Y;. ChatMessage.cs 278
V3041 The expression was implicitly cast from '?' type to 'double' type. Consider utilizing an explicit type cast to avoid the loss of a fractional part. An example: double A = (double)(X) / Y;. ChatMessage.cs 282
V3041 The expression was implicitly cast from '?' type to 'double' type. Consider utilizing an explicit type cast to avoid the loss of a fractional part. An example: double A = (double)(X) / Y;. ChatMessage.cs 284

@swiftyspiffy, @prom3theu5
About the licensing of the use of PVS wrote in the previous PR.
In more detail about our action is written on https://pinguem.ru (though in Russian).

Hi, I understand now. Sorry about the previous closure. Looks good, thanks :D


Added "Verify Email Address"

Added "Verify Email Address". This is used for the subject in the verify-mail

Thank you


Validation of statements upon data entry by actant slot entity type

Check whether the actant slot occupied by an allowed entity type (as per definition of the given A type entity in the allowed entity types for that A type.) Includes: is not a required actant slot, i.e. one which does not allow NULL, empty? Is not an actant slot requiring only NULL occupied? On this, cf. #684.
The only problem is when to trigger the check.

Here is the documentation
https://docs.google.com/document/d/1pmsCjNLHAOA58gVrwV64Qfsp-9wO4kZoXbtyjViMuQA/edit

implementation + tests done according to the docs


Restart Tor

I am using tor and generating onion url and associating it with a http server. Whenever my internet connection is lost the tor circuits break. I want to restart everything at this point. For doing so, i stop my http server and close my onion and tor by tor.Close() and onion.Close(). After that i try to repeat all the process from start to start tor, onion and http service. At this moment i am unable to connect to tor. Everyting context time out occurs.
If i close the program and re-run it seems to work fine. But when i try to restart within the running program it doesnot works.
Is there some issue of tor.Close() or onion.Close()? I am unable to figure this out.
How to restart existing running tor service without stop existing running program?

I honestly haven't the slightest clue what might be wrong, but this is a use case I want to actively use myself too in the following days, so I'll get back to you if I figure anything out.

Just hit this myself too. Funky, will try to figure it out. Seems that the tor bootstrap never finishes when restarting. At least it never gives back control to the caller, even though it says "100%" booted. I've tried nuking the datadir and restarting like that and it also doesn't seem to work, so it might be something off either in the lib or in the control lib.

I think the (or an) issue is in the controller lib: https://github.com/cretz/bine/blob/62912bff1c531f4cb731cc86d38e78bc94aea2fd/tor/tor.go#L370
Onion creation uses tor.EnableNetwork(ctx, true), blocking until the circuit bootstraps itself. However, if the network was enabled already, the above line bail out of the method, even if waiting was requested and the circuit is not yet done. Not sure if this is the cause for the hang, but it's a weird race. Will try to fix and see.

I knew until now that the Tor library is not capable of running two instances in the same process. Unfortunately it also seems that it's not capable of restarting itself without restarting the entire process. My hunch is that some global variable is ruined on shutdown.
Either way, Tor does support enabling and siabling networking, so you can get away with not killing the in-process Tor, just shutting the network off and back on and it should work:
You can enable networking on request via.
tor.EnableNetwork(context.Background(), false)

Turning off the networking is not exposed in the bine library, but you can do it via:
tor.Control.SetConf(control.KeyVals("DisableNetwork", "1")...)

The lack of the latter is tracked in https://github.com/cretz/bine/issues/41


use tiled images download from bingmap

Hello,
I have some tiled images download from bingmap, how can I use them? I have read the tutorial in detail, but still haven't found the way.
Thanks

Tiled imagery is used in VTS as bound layer.
Here is a tutorial: https://vts-geospatial.org/tutorials/bing-maps.html?highlight=bound
Feel free to ask if you have any further questions - be as specific as possible.


Metadata ingestion for source table RAW datatype vs. OM data type

Context
We currently map source table data types to OpenMetadata types. This is becoming hard to scale, and issues arise when we don't properly map a source type to OM type. These issues arise on the Profiler, where we use the OM DataType to prepare the ORM objects. If the types do not properly match the source types, our queries can be wrong.
Ingestion
We need to validate that our data type can properly cover the raw types of the sources. Right now we add "Varchar" to unknown types, which is not great. At least, use unknown type, but we should improve the coverage here.
Profiler changes
In the profiler ORM converter (link), we can use sqlalchemy dialects to map the RAW types to sqlalchemy types: RAW -> SQLAlchemy instead of RAW -> OM -> SQLAlchemy.
Discussion
Let's discuss possible changes here and their PROs / CONs:

Update table.json#/datatype to string -> This would have many major changes when running the metadata ingestion
Add a new table.json#/rawDataType as string and use that in the ORM converter. We would need to show this one in the UI -> Question, how do we then depracate the dataType change?
Add a new prop in the column systemDataType to store the raw data type

Tasks

[ ] Introduce system data type
[ ] Update profiler ORM converter

For This Issue removing using sqlalchemy dialects to map the RAW types to sqlalchemy types: RAW -> SQLAlchemy instead of RAW -> OM -> SQLAlchemy would not be possible at this point of time because of the following reasons.

The dialect not necessarily support all the data types, for ingestion we had to update this map to add the types which were not supported by dialect
The dialect might not necessarily depend on the map itself to get the sqlalchamy type (for ex. see here  how trino dialect handles some complex types) so to eliminate om type we might have to replicate this logic which could be tedious
OM type also helps to identify types in case of non-sqa sources.

54c5a4916926e0bc1a4f80ba23b89db9459e8f25 commit contains the changes done while exploring these thing which might be useful in case we revisit this issue!


Determine batch size from labels instead of inputs

Fix issue #57, and related tests are added.

/flake8-lint

LGTM


Packaging microk8s with a deployment in a VM image - results in very long startup time due to (most likely) hostname change

I'm trying to create a VM image that includes a single node (none HA) microk8s with some simple app deployed on it.
When the hostname stays the same it takes k8s around a minute and a half starting from when k8s is up (microk8s status --wait-ready returns) and till the node is actually serving the app.
When I spin up a VM based on that image with a different hostname (like most times will happen...) it takes k8s more than 8 minutes to get out of its limbo and start serving the application.
The hostname change is done automatically by the cloud-init script  before microk8s is even started (I stop microk8s and only then create the image).
I've tried adding --hostname-override to the kubelet config as part of my cloud-init script (and before starting microk8s) - but that doesn't seem to help and if you look at the attached logs it's still looking for the previous hostname node.
Seems to me like a bug that microk8s doesn't detect correctly the hostname change and takes forever (more than 8 minutes...) to get out of this loop.
When I run microk8s.kubectl get nodes - I see 2 node records.
The old hostname (stays in NotReady state) and the new hostname.
The new hostname appears as ready - but as I mentioned - only 8 minutes after startup (after microk8s status --wait-ready returns) microk8s actually starts to serve the application.
Any idea how can I overcome this long delay?
I thought as a last resort disabling the hostname change of the cloud-init script  - but that isn't a nice solution at all.
Inspect attached -
inspection-report-20220508_103506.tar.gz
Thanks in advance for any assistance!

Hello @bondib.
According to this answer and this answer on Stack Overflow, changing node names is rather an uncommon  use case for k8s.
As of microk8s, you could try running microk8s reset and re-deploying your application on the new VM, although I'm not sure this will help much time-wise.
Also note that in single node deployments microk8s will detect network changes and reconfigure all k8s services, but on multinode clusters changing IPs/hostnames will surely cause nodes to become unreachable.

Hello @lferran, so --hostname-override is not actually supported for multinode clusters?

Hi @ThanKarab

so --hostname-override is not actually supported for multinode clusters?

--hostname-override is supported for multinode clusters. You can configure it like this on each host:
echo '--hostname-override='<hostname>' | sudo tee -a /var/snap/microk8s/current/args/kubelet

NOTE: The discussion in the previous messages does not revolve around setting the hostname reported by kubelet manually using the --hostname-override flag, but rather how MicroK8s handles situations where the hostname of the machine changes (e.g. because of DHCP configs, or due to installing MicroK8s on a base image).

Also, for @bondib, not sure if this issue is still relevant, but my suggestion for this would be to include the MicroK8s snap in the base image, and then install it when the machine starts up for the first time. This would look like this:
# during image build (for channel, look at `snap info microk8s`)
snap download microk8s --channel 1.25 --target-directory /opt --basename microk8s
snap download core18 --target-directory /opt --basename core18

# during initial image boot
snap ack /opt/core18.assert && snap install /opt/core18.snap
snap ack /opt/microk8s.assert && snap install /opt/microk8s.snap --classic

Had a similar issue while changing a machine hostname after microk8s installation. Issuing sudo microk8s reset did NOT help, because it said a multi-node cluster cannot be reset 🙃 What did help is this:
# have the node leave the cluster
sudo microk8s leave

# (re)start the cluster
sudo microk8s start
sudo microk8s status --wait-ready

Takes a few minutes for everything to settle (even after the wait-ready), but even if you enable microk8s services like dns straight after, it will ultimately come ready after a bit 👌
It's as if changing the machine hostname does force microk8s to create a new node, which works in my case, but the older node is stuck in limbo (calico-node in Terminating for ever).

It's as if changing the machine hostname does force microk8s to create a new node, which works in my case, but the older node is stuck in limbo (calico-node in Terminating for ever).

This happens because calico-node is a daemonset. The hostname-override will in fact have the effect of kubelet registering again under the new name, but the previous node registration will stay there.
From the Kubernetes control plane point of view, this looks like a node that is not reconnecting, so it waits before deleting all daemonset pods on the "old" node.


Setting a /32 or /128 IP blockSize on a pool not possible

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
Similar to https://github.com/cilium/cilium/issues/24351 but in multi-pool mode we sometimes use /32 (or /128 for that matter) single-IP pools for stuff like reserved IPs or IPs with special use. However cilium does not use the first and last IP of the pool, meaning the smallest is /30 and still wastes at least two IPs per block. We use eBGP so the "first and last IPs" are of no concern to us.
Since the patch for LB-IPAM is in the works to improve this, the code should be reusable for normal pod IPAM as well.
Cilium Version
1.15.0 branch
Kernel Version
5.15.0
Kubernetes Version
1.27.1
Sysdump
No response
Relevant log output
No response
Anything else?
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Duplicate of #24351

I think it's very clear that this is something we want, as it has been requested quite often. It's less clear how to implement it. To summarize the discussion from Slack, there are a few concerns about using the current LB IPAM allocator as-is:

The new LB IPAM allocator works very fast when allocating "any" IP, but its way slower when allocating specific IPs, the more fragmented the IP space the longer it takes. Allocating specific IPs is performed during endpoint restoration at startup. However,  @dylandreimerink believes even in with heavy fragmentation allocating a specific IP is sub millisecond
The new LB IPAM allocator allocates IPs sequentially, and tries to re-use IPs if that reduces fragemenation. Where as the current multi-pool allocator allocates pod IPs at random. This means that the new allocator makes IP allocation much more predictable, allowing potential attackers to predict or reuse a specific pod IP. Random IP allocation does not fully solve that problem (after all, the IP space is typically small with ~250 IPs at most), but is probably still the preferred policies for pod IP allocation

To clarify for number 1, I just ran the benchmarks in the IP allocator. At 50000 blocks, it takes on average 0.3 ms to allocate a random specific IP, so not terrible but way slower than allocating "any" IP from the range.
goos: linux
goarch: amd64
pkg: github.com/cilium/cilium/pkg/ipalloc
cpu: 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz
BenchmarkHashAlloc_AllocPerStage/pre-alloced-50000-16         	    3985	    312188 ns/op	     166 B/op	       2 allocs/op

Allocating any IP takes roughly 200 nano seconds
goos: linux
goarch: amd64
pkg: github.com/cilium/cilium/pkg/ipalloc
cpu: 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz
BenchmarkHashAlloc_AllocAny-16    	 6240414	       196.7 ns/op	       3 B/op	       0 allocs/op
PASS
ok  	github.com/cilium/cilium/pkg/ipalloc	1.495s

The reason behind this is that with any IP we just pick the first free block we can find and get an IP, so O(1). But for a specific IP we have to iterate over the list of free blocks to find the block containing the given IP, which is an O(n) op where n is the number of fragments/blocks of free IPs.
Lets say 0.3ms is the worst case, then we can still restore about 3000 IPs per second which would be a lot of pods for 1 node. So we might be good here. Even so, I think we can improve on this by batching a bunch of IPs if need be.
With regards to number 2. I suggest making a Allocator[T].AllocRandom for the use case of smaller ranges such as a /24. Max amount of fragments in that case is 128 which is likely not a huge deal.
For larger ranges such as IPv6 /48 (assuming we want to support that) we can make a third method that does a middle ground. It takes a target amount of fragments, lets say a 1000. If below the target we pick a random IP from the range which will likely increase the fragment count by 1. If at or above the fragment count we pick a random block, so 1 in the 1000, then take its start or end IP, so we have a randomness of 1 in 2000. Allocating from the start or end of a fragment doesn't add new fragments and sometimes removes one. We can change the target to get a different performance/security tradeoff. (not sure what to call this, semi-random?)

out of curiosity, 200ns seems darn fast considering the allocation code is not called every second but only when a new pod spawns, no? I think that k8s scheduler itself is way slower than the alloc code.
And blocks are not /48 per ip pool I presume :) we use something like /104 at most I think? What about others?

out of curiosity, 200ns seems darn fast considering the allocation code is not called every second but only when a new pod spawns, no?

Yes, its very fast. That is because its basically 2 pointer lookups, 1 number increment and 1 write to a hash map. The random number allocation will be slower.

And blocks are not /48 per ip pool I presume :) we use something like /104 at most I think? What about others?

/104 is also 256 addresses so that would be great. When I hear people talk about IPv6 its always stupidly large ranges, since /48 is the smallest size you are allowed to announce on the internet. /104 is 16777216 IPs, so here having the semi-random allocation would be advisable to keep the fragments at a few thousand at most

Is there any news about this? As we have the same problems.

Is there any news about this? As we have the same problems.

Not supported yet.

Hi all,
I am using this configuration for the moment.
I created an ippool with this configuration and selector based on label:
apiVersion: "cilium.io/v2alpha1"
kind: CiliumLoadBalancerIPPool
metadata:
name: "blue-pool"
spec:
cidrs:

cidr: "<IP_ADDRESS>/30"
serviceSelector:
matchLabels:
color: blue

and I am adding the annotation in the svc
annotations:
io.cilium/lb-ipam-ips: <IP_ADDRESS>
labels:
color: blue
In this case, I am controlling with  (  serviceSelector: matchLabels: colour: blue ) to which service the IP will be assigned and which IP will be assigned.
I know that this is a bit annoying if the loadbalancer's IP changes, but maybe you can trick the configuration into adapting if the IP changes via IaC.

We have "developed" a small patch that we keep in our repo internally. Basically it removes the "do not use first and last IP from pool" logic, it removes "ip pool is 2^n-2" logic and several other things (I think it was something like 4 places it needed to be patched out) so we are fine right now and happily using it. Hacky but it works and allows us to actually use cilium in production clusters and do things we would not be able to do without it.
@fsbaraglia Our problem is not with ingress to the pod but with egress from that pod :) we have tied special ACLs on routers and firewalls to specific ranges and do not want 99.99% of our pods use these ranges by default.
(And no, we cannot connect those routers and firewalls into clustermesh :D)

Ah ok, good to know @CallMeFoxie
I guess I missed the egress context in the previous messages.

We have "developed" a small patch that we keep in our repo internally. Basically it removes the "do not use first and last IP from pool" logic, it removes "ip pool is 2^n-2" logic and several other things (I think it was something like 4 places it needed to be patched out) so we are fine right now and happily using it. Hacky but it works and allows us to actually use cilium in production clusters and do things we would not be able to do without it.

@CallMeFoxie We are working on a unified operator for multiple databases, the project is called KubeBlocks, we also encountered the same problem as some databases assuming that each IP address has a network device, they rely on the network device (like UUID) as their identity, it is useful in data replication(like paxos) for telling others in the replication group whether it is a new candidate or an old one just recovered from a failure through their identity, in such case, the static 32/128 pod IP seems the  possible and easy way to solve the problem without changing the codes of replication. I am interested that if your fixed-ip cilium branch is open-source or will you push it back to the upstream, or anything else...   Thank you for your attention and glad to see your progress.

not stale

@nayutah sorry, we have not made it yet into upstream with @CallMeFoxie.. we still keep our "modification" locally. @gandro sorry we've not been active much for the past 2 months... have you discussed anything internally regarding this issue?

just to sum it up.. we are talking about multipool IPAM, but being able to specify /32 or /128 address as maskSize in CiliumPodIPPool

@gandro sorry we've not been active much for the past 2 months... have you discussed anything internally regarding this issue?

I don't think there has been any update on this recently. I believe the above comments still hold true. I'm not aware of anyone actively working on this.

noooo not stale!

@CallMeFoxie mind sharing the patches required for Cilium to work in that way?

@CallMeFoxie mind sharing the patches required for Cilium to work in that way?

cc @oblazek can you pull the required commits from the git? :)

this is what we've got:
diff --git a/pkg/ipam/allocator/multipool/pool_allocator.go b/pkg/ipam/allocator/multipool/pool_allocator.go
index f0ae28eff5..d22fe75b92 100644
--- a/pkg/ipam/allocator/multipool/pool_allocator.go
+++ b/pkg/ipam/allocator/multipool/pool_allocator.go
@@ -71,9 +71,6 @@ func addrsInPrefix(p netip.Prefix) *big.Int {
        // compute number of addresses in prefix, i.e. 2^bits
        addrs := new(big.Int)
        addrs.Lsh(big.NewInt(1), uint(p.Addr().BitLen()-p.Bits()))
-       // subtract network and broadcast address, which are not available for
-       // allocation in the cilium/ipam library for now
-       addrs.Sub(addrs, big.NewInt(2))
        if addrs.Sign() < 0 {
                return big.NewInt(0)
        }
diff --git a/pkg/ipam/service/ipallocator/allocator.go b/pkg/ipam/service/ipallocator/allocator.go
index bd79d352b7..da9c4b7336 100644
--- a/pkg/ipam/service/ipallocator/allocator.go
+++ b/pkg/ipam/service/ipallocator/allocator.go
@@ -68,11 +68,11 @@ type Range struct {
 // the backing store.
 func NewCIDRRange(cidr *net.IPNet) *Range {
        base := bigForIP(cidr.IP)
-       max := maximum(0, int(RangeSize(cidr)-2)) // don't use the network broadcast,
+       max := maximum(0, int(RangeSize(cidr)))

        return &Range{
                net:   cidr,
-               base:  base.Add(base, big.NewInt(1)), // don't use the network base
+               base:  base.Add(base, big.NewInt(0)),
                max:   max,
                alloc: allocator.NewAllocationMap(int(max), cidr.String()),
        }

Is there an actual deployment method where the first and last IPs are not usable?  I would expect a more robust method of marking IPs as unusable if that were the case, as one might have to mark an IP as reserved for a router or similar.

FYI: PR by @juliusmh will remove the use of "first and last" IPs for /32 or /128 CIDRs: https://github.com/cilium/cilium/pull/34618 -
It will continue to reserve the first and last IP for larger CIDRs (i.e. there is no allowFirstLastIPs option like in CiliumLoadBalancerIPPool), but since /32 and /128 are unusable otherwise, this seems like a reasonable stop-gap solution even without changing the CRD.


log4stash v3.0.0 doesn't work with ElasticSearch 6.x

I faced issue logging to ElasticSearch in version 6.x after upgrade log4stash to v3.0.0
In the latest version requests that are send to Elastic doesn't contain field "_type" in body. it's not mandatory for Elastic version 7.x but without it requests fails for 6.x and lower.
Are there any plans for backward compatibility with older Elastic versions? Or maybe is there some way to make log4stash 3.0 working with older ES?

Hey, if you just add the "IndexType" in the configuration it will work for you.
Before v3.0.0 the type had a default value, now you have to specify it in order for it to be sent.
FYI, since v3.0.0 every version is tested for usage with ElasticSearch v5.x, 6.X and 7.X.


Fix/cs 43854 sync update

Hi @ishaileshmishra , I've made the requested changes. Please check and review.


We should have a test later for the double encoding of 0 in signed.

We should have a test later for the double encoding of 0 in signed repr.
See https://github.com/o1-labs/proof-systems/pull/1410

Can you add more detail here? AFAIK there is no explicit sign bit: i32 uses 2's complement, so 0 has a unique encoding.


Added Carthage install in README.md

Fixes #20

Current coverage is 93.74% (diff: 100%)

No coverage report found for master at 943d3f3.

Powered by Codecov. Last update 943d3f3...88c4a1f


Some sensors missing

Inexplicably, my MCU_LOAD, MCU_RPI_LOAD and SYSTEM_LOAD sensors have gone missing.  Fluidd shows their stats but the HA sensors don't exist.  There are 6 others missing as I previously had a total of 9 more, but I am not using those.
Moonraker integration 1.2.3
Moonraker v0.9.2-0-gb40751b
Home Assistant 2024.7.3
K1 Max
What causes them to be added to HA?
I have removed and reinstalled the integration, restarted HA and the printer to no avail.
I see nothing helpful in the logs

Yes, these sensors were causing issue, they are temporary removed until we can find a solution that does not break the integration for the others.
sorry about that

Ah, probably in the changelog which I rarely read.
Thanks
On Tue, Aug 13, 2024 at 6:28 AM Marc-Olivier Arsenault <
@.***> wrote:

Yes, these sensors were causing issue, they are temporary removed until we
can find a solution that does not break the integration for the others.
sorry about that
—
Reply to this email directly, view it on GitHub
https://github.com/marcolivierarsenault/moonraker-home-assistant/issues/388#issuecomment-2286013756,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHEX2GE36ZDCN5QIDMFH7KDZRHUUFAVCNFSM6AAAAABMOBFWUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBWGAYTGNZVGY
.
You are receiving this because you authored the thread.Message ID:
<marcolivierarsenault/moonraker-home-assistant/issues/388/2286013756@
github.com>

--
Bruce Morgan
@.***

please update to 1.3.1, it should be fixed


Update query_annotations.md

If I'm not mistaken, the name of the query in the first example is authors_from_country.
I was just going through the book and noticed it. Feel free to disregard if annoying. Appreciate the crate!

Thank you @cmarkh


Stork emoji

Storks are large birds associated with delivering babies or spring in some cultures. Otherwise they are a bit similar to cranes or flamingos #92 except for the color and beak.

"The Stork" is one of the cards in the Lenormand oracle (from circa 1799).
This emoji would also be helpful for representing the symbols in those cards.


Unable to load video backend

I tried the latest version of this program but when I run it I got this error: '[ERR] Unable to load video backend'

I think this is caused because of the removal of add_definitions("-DOL_CMAKE_USE_X11"), since the code will not build the X11 video backend if it's not defined :)
See if the latest commit fixes it! (as I wrote in the other issue, you might have to run git reset --hard HEAD before running git pull, in order to ensure that there are no file conflicts)

I tried it, the compilation works with your latest commit and I didn't have the error. But.. my screen is not updated when I run it.
I tried: 'openlux -k 2400 -b 40' and 'openlux -k 10000' even with sudo. I will see if bsbhro get this issue too, now he can run the program.

There was someone else with a similar issue, he was running Kubuntu 15.10. Are you running the same? If not, what version of ubuntu, and what desktop environment are you running?

I use Ubuntu 15.04 with the Unity Desktop.

I'll download a build of it then, and see if I can replicate the issue :)

Thank you for your time :)

I downloaded the x86 build of the desktop version of ubuntu 15.04. Are you using unity 8? (if you don't know, then you're probably using unity 7)
Also, could you run openlux under valgrind, strace, and ltrace, like such:
valgrind -v ./src/openlux
strace ./src/openlux
ltrace ./src/openlux
and then upload what they output to a pastebin? I'm mostly interested in ltrace, but valgrind or strace might have something useful too (tbh, I doubt that valgrind will, but just in case)

I'm using Unity 7 as my desktop environment. I uploaded what you request:
ltrace: http://pastebin.com/VMmeJ3Ct
strace: http://pastebin.com/cDA66Ugx
valgrind: http://pastebin.com/xCPPCKYn

Everything seems okay (apart from the addresses, basically everything is identical in the results of my computer, and yours)... There aren't any other programs changing the gamma ramp (e.g. redshift, f.lux)?
Just in case, try running openlux -k 0 or openlux -k 99999999 (doesn't matter the exact number of 9's, just put more than 5 =p).

Hi, Good Morning,
on my notebook with Xubuntu 15.10 the program works like expected (last night i read a book gently to eyes) and you closed issue #4. But on my PC with Ubuntu 15.04 the 'make' worked, and the command call is doing nothing. Is this expected?

Assuming that your case is the same as Tankypon's, what graphics card, and, more importantly, what driver (and driver version) are you using for it? Could you try updating the driver (if possible ... you might have to enable the proposed and backports repositories)?

I did this. No change. And I installed it on an old notebook with Lubuntu 15.04, i386, where it works.

I think this has to do with the graphics card driver then. If it's possible, could you try running an ubuntu 15.10 livecd, and see if it works there? (if you're compiling on the livecd, just run sudo apt-get install cmake build-essential libx11-dev libxxf86vm-dev git;git clone https://github.com/MiJyn/openlux.git;cd openlux;mkdir build;cd build;cmake ..;make I used that command quite a few times today to test it on various ubuntu builds ... just saves time to copy and paste =p)
Which graphics card are you using? And, if applicable (AMD or NVIDIA), are you using the proprietary or the open-source driver? Both drivers have worked for me on my PC (NVIDIA GTX 660 TI), using 352.09 for the proprietary one, and both the latest versions that come with ubuntu 15.04 and 15.10 of nouveau (the open-source one).
I'm thinking of releasing a software-based video backend for openlux for situations like this... I'll look into the possibility of this tomorrow :)

It's a "00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor Graphics Controller (rev 09)", but I'm not able to get a driver name and version. No proprietary used on Gnome-Ubuntu-15.04.
On this PC I made it with Xubuntu-15.10-Live-Stick: worked. I made it with Debian 8.2 Xfce: worked. And I made it on a 'fresh' installed Gnome-Ubuntu-15.04: worked. I have a driver problem on my Work-Ubuntu :-) - I'm not a programmer, but I thought since yesterday, 'He is too deep in hardware'. And you think so. Stay tuned.

@MiJyn My graphic card is Nvidia Geforce 740M. I use it with the proprietary drivers Nvidia 355.11 and my processor is Intel i7-4700MQ. When I will have the time I will download a live usb of Ubuntu 15.04 (to test on a fresh system) your program with my GPU

Cool!! So this almost certainly has to do with the graphics card driver then.
I'm currently working on a software-based solution (similar in concept to xcompmgr or compton) that will kind of emulate what the graphics card does. It's definitely not ideal, both in terms of performance and the resulting image, but it's better than nothing :)

I just got this error after updating my X11 from 1.18.6 to 1.19.3 in ubuntu 16.04.2 . I also got another error:
[ERR] 0-bit depths are not supported at this time
for the time being i'll just roll back

After looking a bit into this, it appears Ubuntu disabled Xf86VidMode, which is what openlux requires to change the gamma (see here: https://anonscm.debian.org/cgit/pkg-xorg/xserver/xorg-server.git/tree/debian/rules?id=b0943284842f2ed71a41b5abae2d77fff7c85baf#n152 ).
I'll have to add an xrandr backend soon to fix this...


x/build/cmd/coordinator: improve error message when build fails due to a module not being tidy (and causing failed requests to sum.golang.org)

The fix to #32528 was implemented by finding nested modules, and then invoking go test once for each one, in sequence.
This isn't compatible with GO_DISABLE_OUTBOUND_NETWORK=1 being set and converted to a permanent outbound network restriction as soon as the first set of tests run, because the second go test invocation may need to talk to the module mirror and/or checksum database to fetch/verify additional modules.
Edit: More accurately, the network gets restricted early on, before any of the tests run. The network restriction only affects the default checksum database at https://sum.golang.org. The module proxy is not affected because it's treated specially. See https://github.com/golang/go/issues/34356#issuecomment-532373912 for more details.
/cc @bradfitz @bcmills

The go.sum files are stale.
This doesn't have anything to do with it being the first or second test in a run. The firewall is applied before the command execution regardless.

I've looked, and indeed, the firewall is being restricted very early on, when the bc.Exec call is made to invoke go env -json GOMOD here:
https://github.com/golang/build/blob/da876aaec3965bc391792c76f5ccca997637a4ee/cmd/coordinator/coordinator.go#L2521-L2527
However, it is not a problem for fetching modules as I originally thought, those go through the non-restricted internal proxy (i.e., here).
But it is a problem if some go.sum files are missing entries and the Go 1.13 command decides to check with the default checksum database at https://sum.golang.org, it will be blocked:
:: Running [...] "go" "test" "-short" "golang.org/x/build/maintner/cmd/maintserve/..." [...]

verifying<EMAIL_ADDRESS><EMAIL_ADDRESS>Get<EMAIL_ADDRESS>dial tcp <IP_ADDRESS>:443: connect: no route to host
verifying<EMAIL_ADDRESS><EMAIL_ADDRESS>Get<EMAIL_ADDRESS>dial tcp <IP_ADDRESS>:443: connect: no route to host

In a way, this is a feature, because it helps find missing go.sum entries. But maybe we should be detecting and reporting that in a more user-friendly way? Or maybe we should be tunneling the checksum database through the module proxy?
Moving this to NeedsDecision to figure out how to proceed here. It's certainly not as big an issue as I originally mistakenly thought.

It's failing in a pretty readable way, IMO:

Get<EMAIL_ADDRESS>dial tcp <IP_ADDRESS>:443: connect: no route to host

But if there's an easy way to detect that and fail with a different message, that's fine too.

The current error message can be very confusing when one is not familiar with implementation details of x/build (which is most of Go contributors). For a recent example, see https://golang.org/cl/221102.
I've retitled this issue to make it more clear we want to improve the error text to be more readable.

I'm not working on this right now, so unassigning, but we should fix this.

If you expect the modules to be tidy (or at least complete), it may suffice to set GOFLAGS=-mod=readonly in the builder environment.

If you expect the modules to be tidy (or at least complete), it may suffice to set GOFLAGS=-mod=readonly in the builder environment.

That has become the default in Go 1.16 (#40728), so the error message there looks like this:
... in golang.org/x/tools/gopls

go: downloading github.com/sergi/go-diff v1.1.0
go: downloading honnef.co/go/tools v0.0.1-2020.1.5
go: downloading mvdan.cc/gofumpt v0.0.0-20200802201014-ab5a8192947d
go: downloading mvdan.cc/xurls/v2 v2.2.0
go: downloading github.com/BurntSushi/toml v0.3.1
go: updates to go.sum needed, disabled by -mod=readonly

(From CL 262765.)

CL 262341 should substantially improve the error message generated by cmd/go.


Replace max number of rows parameters with a single --max-rows parameter.

Can we please add a test for MetadataDumper where this argument is/is not present?

@ishmum123 I don't think there is an easy way to test MetadataDumper class by adding JUnit tests, because:

in case of an error a message is logged, so there is not much to assert (besides checking that no exception was thrown),
in case of success, the dumper will fail due to a lack of the connection to the database.

Did you have some particular scenario in mind?

Can we please add a test for MetadataDumper where this argument is/is not present?

@ishmum123 I don't think there is an easy way to test MetadataDumper class by adding JUnit tests, because:

in case of an error a message is logged, so there is not much to assert (besides checking that no exception was thrown),
in case of success, the dumper will fail due to a lack of the connection to the database.

Did you have some particular scenario in mind?

Agreed. It would be difficult to add tests for MetadataDumper atm. Let's skip this for now


Netherite Gear Tags

On 1.21.1 Neoforge Netherite Gears have the forge:gears/netherite item tag which isn't used anymore.
Instead they should have the c:gears and c:gears/netherite item tags.

Fixed in version 1.1.0


PLUTO QAQC page: Fix Value Differences for Zoning table

Per GIS https://github.com/NYCPlanning/data-engineering/issues/976#issuecomment-2372401914, need to de-duplicate value differences for zoning:

@sf-dcp good to close this? I think your PR didn't auto close it because it says and #1160 instead of fixes #1160


Indexing just stops

Hi,
I'm using ES 1.4.2, and a cluster of 2 data nodes, 6 shards, and about 1m docs in an index I'm trying to reindex.
I invoke this reindexing plugin, and it runs for a while, then stops. there is no output in the log at all.
It does not stop at the same place each time, sometimes it manages 72000docs, sometimes it makes 200k docs, but it never succeeds with the entire index. a subsequent
GET _reindex
yields
{
"acknowledged": true,
"names": []
}
I have also noticed that if I invoke this plugin on a cluster that has shard allocation disabled, and I target an as yet non existent index, ES creates the new index properly, but the reindexing plugin cannot write anything until shard allocation is enabled again (of course) - but after that it still fails. it just sits there doing nothing at all.
Hope these issues can be identified and fixed.
this plugin would be very useful if I could get it working properly.
Cheers

Does reindexing work on a cluster that has shard allocation enabled if you do not change the status for shard allocation when invoking reindexing?
Could you check a debug log for elasticsearch?

No. As I said it does not work even if I don't change the allocation status. Also as I said there are zero entries in the log.  It just stops.

Could you provide a step to reproduce it?
I do not have any problems for reindexing...

I wish I could.  All I do is start indexing with post from/_reindex/to
It starts.
Then it stops sometime later as though it finished properly.
My documents are quite large if that makes any difference.

so I resolved this,
after checking the source I added scroll=10m&size=200 to the end of my reindex url.
the time was the likely cause as it defaults to 1000 documents and 1minute max scroll time.
there are some places that errors can occur that will never be logged,
nor anyone alerted to, and I had to change a few things in order to run the tests locally too.
I'll make the changes available when I get a chance if you're interested

Hello,
I suffer from the same effect, but adding ?scroll=10m&size=200 didn't make it work.

Hi. I think this might be related to what we are also having on our production servers.
e.g. we have found this in our elasticsearch log,

Here is the full traceback:

[2015-12-15 14:53:25,814][ERROR][action.bulk              ] [Surge] unexpected error while replicating for action [indices:data/write/bulk[s]]. shard [[hep_v2][4]].
org.codelibs.elasticsearch.reindex.exception.ReindexingException: failure in bulk execution:
[95]: index [hep_v2], type [record], id [1398802], message [MergeMappingException[Merge failed with failures {[mapper [publication_info.recid] of different type, current_type [string], merged_type [lo
ng]]}]]
       at org.codelibs.elasticsearch.reindex.service.ReindexingService$ReindexingListener$2.onResponse(ReindexingService.java:214)
       at org.codelibs.elasticsearch.reindex.service.ReindexingService$ReindexingListener$2.onResponse(ReindexingService.java:209)
       at org.elasticsearch.action.bulk.TransportBulkAction$2.finishHim(TransportBulkAction.java:358)
       at org.elasticsearch.action.bulk.TransportBulkAction$2.onResponse(TransportBulkAction.java:330)
       at org.elasticsearch.action.bulk.TransportBulkAction$2.onResponse(TransportBulkAction.java:319)
       at org.elasticsearch.action.support.replication.TransportReplicationAction$ReplicationPhase.doFinish(TransportReplicationAction.java:983)
       at org.elasticsearch.action.support.replication.TransportReplicationAction$ReplicationPhase.doRun(TransportReplicationAction.java:836)
       at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
       at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.finishAndMoveToReplication(TransportReplicationAction.java:530)
       at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.performOnPrimary(TransportReplicationAction.java:608)
       at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase$1.doRun(TransportReplicationAction.java:452)
       at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

but meanwhile querying for the status at _reindex/<name> was simply returning:
{
  "acknowledged": true,
  "name": "6bedbfdd-228b-4822-bfcf-a437ff394a76",
  "found": true
}

(This is with ES 2.1.0)
In our case the error was simply due to the fact that we initalized the new index with a mapping that was slightly incompatible with the previous index (which was using dynamic mapping for certain fields).
Could it be that @derjohn's error is similar?
Nevertheless I think it would be great if the error state would be, not only logged in, but also reported in the JSON response to _reindex/<name>: in this way it would be possible to pass on to clients the responsibility to deal with the failure.
Currently we have not found a way to distinguish between a very long reindexing and a failed one.
cc: @jalavik

Merge failed with failures {[mapper [publication_info.recid] of different type, current_type [string], merged_type [long]]}]]

The mapping handling for ES 2.x became strict.
"recid" property must be string or long in all _type.

Yep. Indeed we fixed it locally for our own specific problem. The only issue with the elasticsearch-reindexing, though, was that there was no report of an error via the REST interface. The error is currently only logged in the Elasticsearch logs.

Did you run it with wait_for_completion=true?

I can't because it would timeout the REST request. (the index is several gigabyte of size). So what I am doing is a simple polling of the /_reindex/<name> handler.


Fix rule one shot issue when no time specified

It's good to see you fix the issue.
On the other hand, do you think one shot rule is also a typical usage to support?


Develop

What does this PR do?
This PR refactors the functions and classes and configure application for Heroku
Description of Task to be completed?
To ensure that classes and functions have single responsibility and also the application can be run after been deployed remotely
How should this be manually tested?
Gulp Test
Any background context you want to provide?
No
What are the relevant pivotal tracker stories?
Checkpoint 1: Inverted Index
Screenshots (if appropriate)
Questions:
No

Changes Unknown when pulling 56a3e6ed78542a43650495f1bab086a714c0ac72 on develop into ** on master**.

Changes Unknown when pulling b151cf228980c31788e37b4e21697ea7e8d124e3 on develop into ** on master**.

Changes Unknown when pulling<PHONE_NUMBER>7a7d0f50b983037c538a54ac202703 on develop into ** on master**.

Changes Unknown when pulling fe85507339472ac558131b5fac86877832796b0c on develop into ** on master**.

Changes Unknown when pulling f0043d051e352b0cce2224708379da58ce866771 on develop into ** on master**.


Re-enable integration tests on PRs

This attempts to re-enable integration tests on PRs that were disabled here.

Fixes #6575

ruh roh. Maybe we can't yet :(

The fact that it doesn't totally burn down from assembly issues is at least progress.

@ryanbrandenburg: Can this be closed now? Integration tests seem to be working on PRs.


URL Reputation Analysis

Potentially a type of URL Analysis, we may need to tweak the definition on URL Analysis to be broader if we subclass it.
Alternatively, we could have an Identifier Reputation analysis to apply to all identifier types including hashes or other fingerprints.

Identifier Analysis

Identifier Reputation Analysis

URL Reputation Analysis
Domain Reputation Analysis
IP Reputation Analysis
File Hash Reputation Analysis

added in release 0.11.0-BETA-1


visjs-community repo

I found out about your repo here https://github.com/almende/vis/issues/4259
and I thought you merged your work into https://github.com/visjs-community/timeline-plus but it seem that this one is the up to date one. Having multiple repos of the same library seem suboptimal. Is the merge going to happen in the end ?

I have still not moved my project there. It will be updated there soon.


fix(@schematics/angular): addProviderToModule bug: add providers into inner Object

Fix bug when add providers into module.
Fixes: #13080

@alan-agius4 Thanks for checking! I adopted your suggestions and updated this pr.

@bsdfzzzy can you please fix the commit msg as you have a typo?
Thanks

@alan-agius4 emmm...Which one please...

with ut should be with it

@alan-agius4 emmm... I mean Unit Test.

@alan-agius4 By the way, could you tell me when can this issue be solved if you know?


( DELTA 3  ) Update Gameplay Overlay to Ver. X

new version of Gameplay overlay it's out it's a simple Gameplay UI But still clean
Concept

done


Add streaming audio quality "kbps" alongside the stations

Hi,
You could add the radio streaming quality next to the radio station in the search box and now playing, something like > Radio AAA - 128 kbps or Radio BBB - 320 kbps. Sometimes I wanna know the transmission quality and I can't find it anywhere.

Function was implemented.

Thanks a lot. It's much better now. I think you should put this option in the now playing screen for radios already saved in the favorites next to the radio url as well.

Hello,
There are many stations that display 0kbps, which is incorrectly captured by the radio-browser.info API.
Since it is only displayed in the search for some stations, this is acceptable.

Hey,
so I added it.
But this is only displayed if it was retrieved from the radio-browser.info API.
Otherwise there is nothing there.

Yeah! Great! It helps a lot to search and listen to radios with high quality broadcasting. Thank you so much for adding this option since I love audio high quality.


Body tracking depth limit?

Hi,
I'm not actually sure of the maximum number of tracked bodies (is this known?) but in certain situations there may be a crowd of people watching a single person interact and I can see situations where it tracks X people in the crowd before failing to track the intended user.
There are obvious physical restrictions that could be put in place (camera angle, crowd control etc..) but it would be good if we could tell the tracker to ignore anything past a certain depth (X meters) to further control situations such as this.
I'm not missing any features that allow this currently? And is there any form of hard depth limit as is? (I'm using 0.9.2 as 0.9.3 seems to be less stable for our tracking usage) If not, is this something that could be considered? (And maybe further down the road cropped regions of interest, masking etc..)
Cheers,
Richard
P.S. Sorry, I see this has auto-labelled as a bug when it's more of a question - please fix as appropriate and I guess 'enhancement' may be a better route in the future?

I have done try it. It may be helpful for you.
The body tracking process are detects 2d human pose from infrared image in the first step.
Therefor, the body tracking area can be limited by setting Infrared to 0 (It is not detected area) according to the depth value, image area, and other. (NOTE: It is necessary to edits the buffer directly. And, it needs to be done before enqueue.)
https://twitter.com/UnaNancyOwen/status/1167443753365192704

Hi Tsukasa,
That's exactly what I was looking for, thanks! Didn't spot that we could manipulate the actual buffer data fed into the tracker (thought it was a copy). So far seems to work great.
Cheers,
Richard

Is there any further insight here as to the total amount of tracked bodies per device?


Two no-functional-change cleanups combined

This combines:

Uses of strconv.Itoa(int(x)) replaced with strconv.FormatInt() or strconv.FormatUint()
Tests that read values set by atomic.StoreInt64() use atomic reads as well.

The first would be needed to show system counters correctly on a 32-bit platform.
The second is part of #25 required to pass tests with -race.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

31 of 34 (91.18%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.1%) to 98.538%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

server/internal/packets/packets.go
0
3
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.1%

Covered Lines:
2629

Relevant Lines:
2668

💛  - Coveralls


SKMicrophone.Initialize() is throwing a native exception in HoloLens 2 UWP ARM Application

Description
I am using Microsoft Psi's Mixed Reality Microphone class which uses SKMicrophone => Microphone.cs.
There are times when I receive the following stack trace error when SKMicrophone.Start() is called. In my experience, it's fairly inconsistent as to when this native exception is thrown, but usually gets thrown by the 4th or 5th run-through of Start->Stop->Start... for SKMicrophone. Though, sometimes it can even throw after the first call to this function after redeploying the app.
As you'll see in the code excerpt and its output log below, I've isolated the error to SKMicrophone.Start(). The exception does not come from any of the other native API calls in SKMicrophone (e.g, getting the number of unread samples and if it's recording).
I notice your comment that Start may fail if the app does not have mic permissions or if it is unplugged. I am always using the default microphone so it is never unplugged and ideally is never still in use. It also always has the right permissions.
The call stack below is the native exception being thrown:

Please let me know if you'd like a .dmp file and I can send that along as well.
My next step is to build StereoKit from source to retrieve any more information and attach it to this issue. However, if you have a clue as to what might be going on here before then, that would be very helpful.
Platform / Environment

UWP ARM HoloLens 2
Nuget package => StereoKit: 0.3.7-preview.2
HoloLens 2 is up to date

Logs or exception details
A code excerpt with logging information to show where the crash is likely happening:
 /// <inheritdoc/>
public override bool Initialize()
{
    System.Diagnostics.Trace.WriteLine($"{nameof(this.Initialize)}: A0");

    if (!SKMicrophone.Start())
    {
        System.Diagnostics.Trace.WriteLine($"{nameof(this.Initialize)}: A0E");
        throw new Exception("Failed to access the system's default microphone.");
    }

    System.Diagnostics.Trace.WriteLine($"{nameof(this.Initialize)}: A1");

    this.active = true;

    return true;
}

/// <inheritdoc/>
public override void Shutdown()
{
    System.Diagnostics.Trace.WriteLine($"{nameof(this.Stop)}: A0");
    SKMicrophone.Stop();
    System.Diagnostics.Trace.WriteLine($"{nameof(this.Stop)}: A1");
}

/// <inheritdoc/>
public void Start(Action<DateTime> notifyCompletionTime)
{
    System.Diagnostics.Trace.WriteLine($"{nameof(this.Start)}: A0");

    /*this.active = true;*/
    notifyCompletionTime(DateTime.MaxValue);
}

/// <inheritdoc/>
public void Stop(DateTime finalOriginatingTime, Action notifyCompleted)
{
    this.active = false;
    notifyCompleted();
}

/// <inheritdoc/>
public unsafe override void Step()
{
    System.Diagnostics.Trace.WriteLine($"{nameof(Microphone)}: Apre");

    if (!this.active)
    {
        return;
    }

    System.Diagnostics.Trace.WriteLine($"{nameof(Microphone)}: A0");

    int unreadSamples = SKMicrophone.Sound.UnreadSamples;
    var originatingTime = this.pipeline.GetCurrentTime();

    System.Diagnostics.Trace.WriteLine($"{nameof(Microphone)}: A1");

    // Check if there are samples to capture.
    // Note that we wait for more than 1 unread sample to be available, otherwise the
    // ReadSamples() method below does not work as expected.
    if (this.active && SKMicrophone.IsRecording && unreadSamples > 1)
    {
        System.Diagnostics.Trace.WriteLine($"{nameof(Microphone)}: A2");

        // Ensure that the sample buffer is large enough
        if (unreadSamples > this.buffer.Length)
        {
            this.buffer = new float[unreadSamples];
        }

        System.Diagnostics.Trace.WriteLine($"{nameof(Microphone)}: A3");

        // Read the audio samples
        int samples = SKMicrophone.Sound.ReadSamples(ref this.buffer);

        System.Diagnostics.Trace.WriteLine($"{nameof(Microphone)}: A4");

        if (samples < unreadSamples && samples < this.buffer.Length)
        {
            System.Diagnostics.Trace.WriteLine($"{nameof(Microphone)}: A4E");
            throw new Exception(
                "Error reading audio samples from the microphone.\n" +
                $"Expected at least {Math.Min(unreadSamples, this.buffer.Length)} samples but obtained {samples}.");
        }

        // Convert to bytes and post the AudioBuffer
        byte[] audio = new byte[samples * this.audioFormat.BitsPerSample / 8];

        System.Diagnostics.Trace.WriteLine($"{nameof(Microphone)}: A5");

        fixed (void* src = this.buffer, dst = audio)
        {
            Buffer.MemoryCopy(src, dst, audio.Length, audio.Length);
        }

        System.Diagnostics.Trace.WriteLine($"{nameof(Microphone)}: A6");

        this.Out.Post(new AudioBuffer(audio, this.audioFormat), originatingTime);

        System.Diagnostics.Trace.WriteLine($"{nameof(Microphone)}: A7");
    }
}

A working instance's log:
Initialize: A0
Microphone: Apre
Microphone: Apre
The thread 0x2774 has exited with code 0 (0x0).
Microphone: Apre
The thread 0x242c has exited with code 0 (0x0).
Microphone: Apre
Microphone: Apre
Initialize: A1
Microphone: Apre
Microphone: A0
Microphone: A1
Microphone: A2
Microphone: A3
Microphone: A4
Microphone: A5
Microphone: A6
Microphone: A7

A failing instance's log:
Initialize: A0
Microphone: Apre
Microphone: Apre
Microphone: Apre
Microphone: Apre
Microphone: Apre
Microphone: Apre
The thread 0x2850 has exited with code 0 (0x0).
Microphone: Apre
The thread 0x167c has exited with code 0 (0x0).
Exception thrown at 0x71DBEADE (MMDevAPI.dll) in XXXX.exe: 0xC0000005: Access violation reading location 0x00000008.

The program '[5392] XXXX.exe' has exited with code 0 (0x0).

Thanks for the detailed report! :) I'm looking into this now, and I believe I was able to reproduce the problem. I'm not really sure what the problem is likely to be, but I'll report back here when I find out more!

Hey Nick, thanks for the updated version of MiniAudio in your latest preview build! Unfortunately, I still receive the same error, but have isolated the crash further to the ma_device_init function, when called from StereoKit's native code: https://github.com/StereoKit/StereoKit/blob/54ada8fe991aeef2e74eb74fd2404e69105bfdd2/StereoKitC/systems/audio.cpp#L296
I'll try debugging the MiniAudio header file next to see where this happening and will keep you updated!

I have still been poking at this bug, I was trying to fix it before the preview.3 release, but no luck quite yet! I figured I'd update miniaudio while I was at it, but didn't see anything in the changelog that looked like it'd be relevant.
I've been somewhat blocked from getting quality debug data by certain build issues, which is what I'm currently focused on. UWP has been exciting for me lately 😬 So yeah, any extra details are helpful, I'm still definitely working on this one!

Are there any updates on the bug or is there anything I can help with debugging? Since this issue is unfortunately regularly breaking our app, it is very high priority for us.

I unfortunately do not have any interesting updates here yet! Thank you for bumping this with a severity, I'll put this back at the top of my list.

Thank you!
Here are some steps on how to reproduce. I figure it's easier to test if it inherits IStepper since you can plug that in for any sample project.

Add the modified DemoMicrophone.cs Stepper (the only changes are in the ShowMicDeviceWindow() function where I simply Start and Stop the microphone):

class DemoMicrophone : IStepper
{
    Matrix descPose = Matrix.TR(-0.5f, 0, -0.5f, Quat.LookDir(1, 0, 1));
    string description = "Sometimes you need direct access to the microphone data! Maybe for a special effect, or maybe you just need to stream it to someone else. Well, there's an easy API for that!\n\nThis demo shows how to grab input from the microphone, and use it to drive an indicator that tells users that you're listening!";
    Matrix titlePose = Matrix.TRS(V.XYZ(-0.5f, 0.05f, -0.5f), Quat.LookDir(1, 0, 1), 2);
    string title = "Microphone";

    Sprite micSprite;
    Material micMaterial;

    public bool Initialize()
    {
        micSprite = Sprite.FromFile("mic_icon.png", SpriteType.Single);
        micMaterial = Default.MaterialUnlit.Copy();
        micMaterial.Transparency = Transparency.Blend;

        bool micSuccess = Microphone.Start();

        return micSuccess;
    }

    public void Shutdown()
    {
        Microphone.Stop();
    }

    bool StartMic()
    {
        if (!Microphone.IsRecording)
            return Microphone.Start();
        return false;
    }

    void StopMic() => Microphone.Stop();

    /// :CodeSample: Microphone Sound Microphone.Sound Microphone.IsRecording Sound.UnreadSamples Sound.ReadSamples
    /// ### Getting streaming sound intensity
    /// This example shows how to read data from a Sound stream such as the
    /// microphone! In this case, we're just finding the average 'intensity'
    /// of the audio, and returning it as a value approximately between 0 and 1.
    /// Microphone.Start() should be called before this example :)
    float[] micBuffer = new float[0];
    float micIntensity = 0;
    float GetMicIntensity()
    {
        if (!Microphone.IsRecording) return 0;

        // Ensure our buffer of samples is large enough to contain all the
        // data the mic has ready for us this frame
        if (Microphone.Sound.UnreadSamples > micBuffer.Length)
            micBuffer = new float[Microphone.Sound.UnreadSamples];

        // Read data from the microphone stream into our buffer, and track
        // how much was actually read. Since the mic data collection runs in
        // a separate thread, this will often be a little inconsistent. Some
        // frames will have nothing ready, and others may have a lot!
        int samples = Microphone.Sound.ReadSamples(ref micBuffer);

        // This is a cumulative moving average over the last 1000 samples! We
        // Abs() the samples since audio waveforms are half negative.
        for (int i = 0; i < samples; i++)
            micIntensity = (micIntensity * 999.0f + Math.Abs(micBuffer[i])) / 1000.0f;

        return micIntensity;
    }
    /// :End:

    /// :CodeSample: Microphone Microphone.GetDevices Microphone.Start
    /// ### Choosing a microphone device
    /// While generally you'll prefer to use the default device, it can be
    /// nice to allow users to pick which mic they're using! This is
    /// especially important on PC, where users may have complicated or
    /// interesting setups.
    ///
    /// ![Microphone device selection window]({{site.screen_url}}/MicrophoneSelector.jpg)
    ///
    /// This sample is a very simple window that allows users to start
    /// recording with a device other than the default. NOTE: this example
    /// is designed with the assumption that Microphone.Start() has been
    /// called already.
    Pose micSelectPose = new Pose(0.5f, 0, -0.5f, Quat.LookDir(-1, 0, 1));
    string[] micDevices = null;

    public bool Enabled => true;

    void ShowMicDeviceWindow()
    {
        // Let the user choose a microphone device
        UI.WindowBegin("Available Microphones:", ref micSelectPose);

        // User may plug or unplug a mic device, so it's nice to be able to
        // refresh this list.
        if (UI.Button("Refresh") || micDevices == null)
            micDevices = Microphone.GetDevices();
        UI.HSeparator();

        Vec2 size = V.XY(0.25f, UI.LineHeight);
        if (Microphone.IsRecording)
        {
            if (UI.Button("Stop Recording", size))
            {
                Microphone.Stop();
            }
        }
        else
        {
            if (UI.Button("Start Recording", size))
            {
                Microphone.Start();
            }
        }

        UI.WindowEnd();
    }
    /// :End:

    public void Step()
    {
        ShowMicDeviceWindow();

        if (Microphone.IsRecording)
        {
            // Squaring a 0-1 value gives an extra slow initial response, but
            // squaring the inverse of a 0-1 value and then inverting it back
            // gives a fast initial response! Try graphing "y = 1-(1-x)^2"
            // versus "y = x^2" for the x 0-1 range.
            float intensity = GetMicIntensity();
            intensity = 1 - intensity;
            intensity = 1 - (intensity * intensity);

            float scale = 0.1f + 0.06f * intensity;
            Color color = new Color(1, 1, 1, Math.Max(0.1f, intensity));
            Default.MeshSphere.Draw(micMaterial, Matrix.TS(0, 0, -0.5f, scale), color);
            micSprite.Draw(Matrix.TS(0, 0, -0.5f, 0.06f), TextAlign.Center);
        }
        else
        {
            // Draw it in red if we're not recording
            Default.MeshSphere.Draw(micMaterial, Matrix.TS(0, 0, -0.5f, 0.1f), new Color(1, 0, 0, 0.1f));
            micSprite.Draw(Matrix.TS(0, 0, -0.5f, 0.06f), TextAlign.Center);
        }

        Text.Add(title, titlePose);
        Text.Add(description, descPose, V.XY(0.4f, 0), TextFit.Wrap, TextAlign.TopCenter, TextAlign.TopLeft);
    }
}

Build the UWP StereoKit application for ARM.
Deploy on the HL2 via Debug/Release mode with Managed or Native debugging (hopefully MSFT adds Mixed Debugging for ARM apps soon). Be sure the Microphone capability is enabled in the APPX manifest.
I find it's random as to when the app crashes based on the number of button presses you make to Start and Stop the microphone. To reproduce, turn on/off the microphone several times (sometimes it takes me 5-15 button presses).

DMP file with heap information. ARM64 + HL2:
https://drive.google.com/file/d/1HLkPFDdKKhAUOjCcKRj6PJ_hOJfvamQJ/view?usp=sharing

Some additional details! Kinda just keeping track of things here so I don't lose 'em.
The call stack resolves a bit further down here.

Miniaudio will also spit out some additional log messages when its debugging features are activated. Nothing super obvious, but perhaps interesting threads to tug on. This is what gets printed every time recording starts, and the crash happens after this. Logs look identical regardless of crash or not.
 Attempting recovery. mappedBufferCaptureCap=768
[SK diagnostic] MA: [WASAPI] IAudioClient3_GetSharedModeEnginePeriod failed. Falling back to IAudioClient.

Some Issues that have a little bit of overlap, possibly some insight and tricks to try:
https://github.com/mackron/miniaudio/issues/81

Thank you for taking an in-depth look into this!
I took your suggestion on using MediaCapture as an audio source and I hope to have a simple follow-up question:
I wrote the following SK program: https://gist.github.com/austinbhale/329ecb46c8748741716dac1643164611 which basically initializes a MediaCapture component for audio and feeds that into a sound instance in SK. I believe the only available audio source profile for the HoloLens 2's microphone is 2 channels (top-left mic & top-center mic) @ 48khz sampling frequency with 32 bits per sample. StereoKit seems to take one channel as input, so we can just ignore the data for the second channel.
The audio plays in real time; however, the pitch is super low (I feel like Darth Vader is behind me!). Do you have a clue as to why the processed audio data is different than the results from the output from MiniAudio?
If you get a brief chance to test out the script, I'd greatly appreciate it. Maybe it'd be useful as an alternative example to the SKMicrophoneDemo with some more polishing. Thanks again!

I haven't tried running your script yet, but my guess might be that the 2 audio channels are interlaced? If you're reading interlaced data like that, I could imagine it sounding like a 2x slowdown.

Ok, I tried out the script, it totally was interlaced :)
I did this:
var stream = new MemoryStream(audio, 0, audio.Length);
var reader = new BinaryReader(stream);
var count = (audio.Length / 4) / 2;
var samples = new float[count];
for (var i = 0; i < count; i++)
{
	samples[i] = reader.ReadSingle();
	reader.ReadSingle();
}

And in the function above, doubling the samples there:
uint sampleCount = (frameDurMs * sampleRate) / 1000 * 2;

Sweet! Thanks Nick
I made a revision to the script to check the number of channels for interlaced data earlier on. Works like a charm!


HNC: webhooks broken in K8s 1.19 due to Golang 1.15

cc @mbtamuli. See https://github.com/kubernetes-sigs/multi-tenancy/issues/1096#issuecomment-690868907.
/assign @yiqigao217
{"level":"info","ts":1599798100.9179494,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/validate-hnc-x-k8s-io-v1alpha2-hierarchyconfigurations"}
{"level":"info","ts":1599798100.9179764,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/validate-objects"}
{"level":"info","ts":1599798100.9179943,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/validate-hnc-x-k8s-io-v1alpha2-hncconfigurations"}
{"level":"info","ts":1599798100.9180129,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/validate-hnc-x-k8s-io-v1alpha2-subnamespaceanchors"}
{"level":"info","ts":1599798100.9180274,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/validate-v1-namespace"}
{"level":"info","ts":1599798100.9180343,"logger":"controller-runtime.builder","msg":"skip registering a mutating webhook, admission.Defaulter interface is not implemented","GVK":"hnc.x-k8s.io/v1alpha2, Kind=HNCConfiguration"}
{"level":"info","ts":1599798100.91805,"logger":"controller-runtime.builder","msg":"skip registering a validating webhook, admission.Validator interface is not implemented","GVK":"hnc.x-k8s.io/v1alpha2, Kind=HNCConfiguration"}
{"level":"info","ts":1599798100.9180915,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/convert"}
{"level":"info","ts":1599798100.9180963,"logger":"controller-runtime.builder","msg":"conversion webhook enabled","object":{"name":""}}
{"level":"info","ts":1599798100.9181077,"logger":"controller-runtime.builder","msg":"skip registering a mutating webhook, admission.Defaulter interface is not implemented","GVK":"hnc.x-k8s.io/v1alpha2, Kind=HierarchyConfiguration"}
{"level":"info","ts":1599798100.918112,"logger":"controller-runtime.builder","msg":"skip registering a validating webhook, admission.Validator interface is not implemented","GVK":"hnc.x-k8s.io/v1alpha2, Kind=HierarchyConfiguration"}
{"level":"info","ts":1599798100.9181347,"logger":"controller-runtime.builder","msg":"conversion webhook enabled","object":{"name":""}}
{"level":"info","ts":1599798100.91815,"logger":"controller-runtime.builder","msg":"skip registering a mutating webhook, admission.Defaulter interface is not implemented","GVK":"hnc.x-k8s.io/v1alpha2, Kind=SubnamespaceAnchor"}
{"level":"info","ts":1599798100.9181566,"logger":"controller-runtime.builder","msg":"skip registering a validating webhook, admission.Validator interface is not implemented","GVK":"hnc.x-k8s.io/v1alpha2, Kind=SubnamespaceAnchor"}
{"level":"info","ts":1599798100.9181774,"logger":"controller-runtime.builder","msg":"conversion webhook enabled","object":{"name":""}}

I tried deploying the latest version using
HNC_VERSION=v0.5.2
kubectl apply -f https://github.com/kubernetes-sigs/multi-tenancy/releases/download/hnc-${HNC_VERSION}/hnc-manager.yaml

Even then the logs from the manager container shows some errors. Logs -
manager.log

Could it be a version issue of Kind or Kubernetes?
 They're all about defaults that are set on the API types themselves, but we don't use that feature. Our webhooks are registered explicitly and aren't affected by these message. (This is all detailed controller-runtime stuff).
I think this is the same as https://github.com/open-policy-agent/gatekeeper/pull/811, and we need to port that here. That is: HNC is current broken on K8s 1.19, because that's built with Go 1.15 which changes the way we handle certificates. Updating the comment to better describe this.
/assign @adrianludwin

We should fix this in the 0.5 branch. I'll do it.

That is: HNC is current broken on K8s 1.19, because that's built with Go 1.15 which changes the way we handle certificates. Updating the comment to better describe this.

Do you mean it should work with k8s <1.19?

That's right - I've personally tested HNC up to 1.18 (on GKE) but not yet
on 1.19, I'll try to get to this by the end of the week.
On Wed, Sep 16, 2020 at 10:52 PM Mriyam Tamuli<EMAIL_ADDRESS>wrote:

That is: HNC is current broken on K8s 1.19, because that's built with Go
1.15 which changes the way we handle certificates. Updating the comment to
better describe this.
Do you mean it should work with k8s <1.19?
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes-sigs/multi-tenancy/issues/1100#issuecomment-693776485,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AE43PZAX7O6UJZEU25WOU4TSGF2VBANCNFSM4RH4CATA
.

Hey @adrianludwin,
I was trying to run the tests. I am always getting errors no matter the kubernetes version I'm using.
I've tested with GKE - v1.15, v1.16 and v1.17, the ones you mentioned here - in Testing Signoff
Here are the logs
test-e2e-v115.log
test-e2e-v116.log
test-e2e-v117.log

Am I running the e2e tests wrong? I was getting even errors without the HNC_REPAIR not set.
I did the following -
gcloud container clusters create hnc-test-v117 --preemptible --num-nodes 1 --cluster-version 1.17 --release-channel rapid
HNC_REPAIR=true make test-e2e 2>&1 | tee test-e2e-v117.log

gcloud container clusters create hnc-test-v116 --preemptible --num-nodes 1 --cluster-version 1.16 --release-channel regular
HNC_REPAIR=true make test-e2e 2>&1 | tee test-e2e-v116.log

gcloud container clusters create hnc-test-v115 --preemptible --num-nodes 1 --cluster-version 1.15 --release-channel stable
HNC_REPAIR=true make test-e2e 2>&1 | tee test-e2e-v115.log

@mbtamuli  - I just discovered that we're broken at HEAD. Did you try this on v0.5.2? See here for details. We hadn't noticed this because we were testing on clusters that previously had HNC installed.
If you're not testing on HEAD, can you please attach the logs from HNC itself, and not just the tests? You can get the logs via kubectl logs or make deploy-watch.

I'm actively working on this, but I likely will only be able to fix this on Monday.

Did you try this on v0.5.2?

Yes I did. Updated the previous comment with the deployment steps. Just followed the deployement steps mentioned in the release.

Oh, you need to set HNC_REPAIR to https://github.com/kubernetes-sigs/multi-tenancy/releases/download/hnc-${HNC_VERSION}/hnc-manager.yaml (with the env var filled in). The test runs kubectl apply -f ${HNC_REPAIR} after it (deliberately) breaks the HNC installation.
If you leave the variable unset, it will skip some of the tests but it will run enough other things to be confident that you've at least installed it correctly.
Sorry, I should document that env var better.

I don't know how to run Kind on Github Actions. We use Prow and we have some pointers on how to run Kind in Prow, so we'll likely be setting that up soon-ish.

Oh, you need to set HNC_REPAIR to https://github.com/kubernetes-sigs/multi-tenancy/releases/download/hnc-${HNC_VERSION}/hnc-manager.yaml (with the env var filled in). The test runs kubectl apply -f ${HNC_REPAIR} after it (deliberately) breaks the HNC installation.

Ah, I should've taken a better look at the Makefile or the existing e2e tests. Will try setting HNC_REPAIR to the manifest path.

We use Prow and we have some pointers on how to run Kind in Prow, so we'll likely be setting that up soon-ish.

Can you mention one version of HNC(or commit), that you've tried with one version of KIND and one version of kubernetes, where the e2e tests finished successfully?
For example, you've tested successfully with
HNC - 0.5.2 AND Kind - 0.9.0 AND Kubernetes - 1.16

Hey @adrianludwin, I got these logs. I tested these on GKE 1.17
gcloud container clusters create hnc-test-v117 --preemptible --num-nodes 1 --cluster-version 1.17 --release-channel rapid

HNC - v.0.5.2

hnc-v117-0.5.2.log
test-e2e-v117-0.5.2.log

HNC - v.0.5.1

hnc-v117-0.5.1.log
test-e2e-v117-0.5.1.log

I think I saw the same thing recently and it was because I had the wrong version of kubectl-hns. Can you confirm you have the v0.5.x version of kubectl-hns on your path?

Hey, I did try with the correct version of kubectl-hns. This time a few of the tests passed.
FAIL! -- 20 Passed | 8 Failed | 0 Pending | 0 Skipped
HNC - v.0.5.1

hnc-v117.log
test-e2e-v117.log

It seems like you're running some weird combination of branch (0.5) and
master (will be 0.6) based on this error message:
      s: "Error: exit status 1\nOutput: \nCould not update the HNC

Configuration: admission webhook "hncconfigurations.hnc.x-k8s.io" denied
the request: Unrecognized mode 'Ignore' for /v1, Kind=Secret\n",
The "Ignore" mode (upper-case) was introduced after 0.5, so 0.5 won't
recognize it. In 0.5, it was called "ignore" (lower-case). Are you running
the master version of the e2e tests? If so, make sure you switch to the
hnc-v0.5 branch and run the tests via "make e2e-test" (and not "make
test-e2e," as it was renamed after 0.5).
On Tue, Sep 22, 2020 at 10:23 AM Mriyam Tamuli<EMAIL_ADDRESS>wrote:

Hey, I did try with the correct version of kubectl-hns. This time a few of
the tests passed.
FAIL! -- 20 Passed | 8 Failed | 0 Pending | 0 Skipped
HNC - v.0.5.1

hnc-v117.log
https://github.com/kubernetes-sigs/multi-tenancy/files/5262066/hnc-v117.log
test-e2e-v117.log
https://github.com/kubernetes-sigs/multi-tenancy/files/5262067/test-e2e-v117.log

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes-sigs/multi-tenancy/issues/1100#issuecomment-696755125,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AE43PZESCJVYGJSV6CUYAQDSHCXMLANCNFSM4RH4CATA
.

Fixed in v0.5 by #1125, and will be fixed in master by #1128. Closing because it's fixed in v0.5 (we should release v0.5.3 shortly).
/close


SAML Groups/Roles

On Discord I asked for having Roles in the SAML IDP implementation.
"I've tried to setup Zitadel as IDP for Rancher as SP by using it's Keycloak SAML implementation, seems Rancher requires a Groups field, the SamlResponse from Zitadel doesn't include that: zitadel/saml/.../attributes.go:59 are there plans to add a "Roles/Groups" Field? "
As discussed in a call with @stebenz I'll provide a SamlResponse that includes them.

Hi, this issue is blocking me to do a SSO with Amazon quicksights. I have a customer who would like to use zitadel to SSO Amazon quicksight.

@jochumdev never received an example, but I created a follow-up issue, which should cover this with the possibility to edit the attribute names themselves.
@harshana5 this should also cover your blocking situation
I will close this issue in regard to #5014 if any more information or questions come up, please fill it in there.


GH-35498: [C++] Fix source node batch realignment

Rationale for this change
Currently, source node uses too high a value for realignment, which caused a performance degradation.
What changes are included in this PR?
The source node batch realigns an array to its byte width (if it is more than 1).
Are these changes tested?
The changes are tested for correctness by the existing tests. The performance is checked by regression jobs.
Are there any user-facing changes?
Only performance.
This PR contains a "Critical Fix".

Closes: #35498

@ursabot please benchmark

I see a bunch of CI job (here, here, here and here) are failing due to OOM, presumably on realignment, but the allocation sizes are not large. More concerning is this failure, which shows an invalid alignment of 3. Presumably, the data type of the value being realigned is some kind of struct. @westonpace, what are your thoughts? Perhaps there is an API (or idiom) to get the byte width of the largest field of the data type, which I think would be the correct alignment in such a case?

Structs do not have any buffers themselves beyond the validity buffer.  The validity buffer will have a bit width of 1.  I don't think we need to worry about aligning any buffers with a width of 8 or less (e.g. only worry about it once the width is 16)
One problem with this PR is that it doesn't address offsets buffers.  For example, in a string / list / binary array there is a 32-bit offsets buffer which should be aligned to 32.  These arrays won't show up as fixed width arrays.
I was working on this a bit yesterday as well.  I ended up with something like this...
bool CheckMallocAlignment(const ArrayData& array) {
  auto type_id = array.type->storage_id();
  if (type_id == Type::DICTIONARY) {
    // The values array will be checked separately
    type_id = ::arrow::internal::checked_pointer_cast<DictionaryType>(array.type)
                  ->index_type()
                  ->id();
  }
  switch (array.type->id()) {
    case Type::NA:
    case Type::FIXED_SIZE_LIST:
    case Type::FIXED_SIZE_BINARY:
    case Type::BOOL:
    case Type::INT8:
    case Type::UINT8:
    case Type::DECIMAL128:
    case Type::DECIMAL256:
    case Type::SPARSE_UNION:
    case Type::RUN_END_ENCODED:
    case Type::STRUCT:
      // These have no buffers or all buffers need only byte alignment
      return true;
    case Type::INT16:
    case Type::UINT16:
    case Type::HALF_FLOAT:
      return CheckValuesAlignment(array, 16);
    case Type::INT32:
    case Type::UINT32:
    case Type::FLOAT:
    case Type::STRING:
    case Type::BINARY:
    case Type::DATE32:
    case Type::TIME32:
    case Type::LIST:
    case Type::MAP:
    case Type::DENSE_UNION:
    case Type::INTERVAL_MONTHS:
      return CheckValuesAlignment(array, 32);
    case Type::INT64:
    case Type::UINT64:
    case Type::DOUBLE:
    case Type::LARGE_BINARY:
    case Type::LARGE_LIST:
    case Type::LARGE_STRING:
    case Type::DATE64:
    case Type::TIME64:
    case Type::TIMESTAMP:
    case Type::DURATION:
    case Type::INTERVAL_DAY_TIME:
      return CheckValuesAlignment(array, 64);
    case Type::INTERVAL_MONTH_DAY_NANO:
      return CheckValuesAlignment(array, 128);
    default:
      return true;
  }
}

It's not pretty.  I also tried using a visitor pattern but it was ending up to be more complicated (though arguably a bit more future proof were we to add more fixed-width types).  CheckValuesAlignment checks the buffer at index 1.  Fortunately, it seems that in all cases, the buffer that needs alignment seems to be at that position.

I was thinking along the lines of your work; you're clearly ahead of me on this.

I don't think we need to worry about aligning any buffers with a width of 8 or less

We do. In failures I observed internally, the cause was misalignment of buffer addresses (which were even numerically odd) that were expected to be 4- or 8-byte aligned.
I think the switch-statement should be refactored into type_traits.h, if it ends up as part of the solution.
I think the correct idea here is that alignment applies to buffers, not arrays. That is, the format of the buffer is what determines the required alignment, if any, and I guess this format is determined by the buffer index within the array and the array's storage type.
@westonpace, should you or I take this forward?

We do. In failures I observed internally, the cause was misalignment of buffer addresses (which were even numerically odd) that were expected to be 4- or 8-byte aligned.

Sorry, I meant 8bits or less (e.g. we can safely assume that everything has at least 1 byte of alignment)

I think the correct idea here is that alignment applies to buffers, not arrays. That is, the format of the buffer is what determines the required alignment, if any, and I guess this format is determined by the buffer index within the array and the array's storage type.

Yes, unfortunately.

@westonpace, should you or I take this forward?

I think I should have time to make a PR today.  Sorry to steal a task, I should've marked yesterday that I was working on this.

Sorry, I meant 8bits or less (e.g. we can safely assume that everything has at least 1 byte of alignment)

Agreed. I included this condition in my recent commit here.

I think the correct idea here is that alignment applies to buffers, not arrays.

Yes, unfortunately.

I'd suggest considering adding something like DataType::GetBufferAlignment(int index) to return the required alignment of  buffer at index that must be a power-of-2 or otherwise be 0 for when no alignment is required. This would force any future data type to provide this info.

Sorry to steal a task

OK, you owe me one :)

Also, consider using {Check/Ensure}Alignment(Buffer, int64_T) from util/align_util.cc.

Is this obsoleted by #35565?

Is this obsoleted by #35565?

Yes. Closing this.


5 months and still some todo's

Any chance you are working on this project still? Looks like it has potential.

Thanks, but I'm not working on it at the moment. I might be able to squeeze in a couple of weekends of work sometime next month, but I'm currently swamped with work.
Feel free to pitch-in if you want to enhance it.


Request for example/documentation on signing transactions with a custom UTXO set

Thanks for adding the ability to add foreign utxos (https://github.com/LtbLightning/bdk-flutter/issues/86). Would it be possible to include an example of how to use this correctly?
I am currently working on an app that has a database of it's own unspent UTXO set (previous transaction id, output index and sats amount).
Is it possible to use this library to sign a transaction with only this data? I've only found documentation on how to fetch UTXO's via an electrum server. This however does not work for my use case.
Any advice or an example would be greatly appreciated.

Hi @BitcoinZavior do you have any code samples for this to work. I tried this but couldn't make it work. Seeking for some help here. Understanding there are lot more things to be looked upon. Hopefully we can see some code snippets or documentation update anything works.
Thanks a lot.


Update Kitchen Sink project

Code update/cleanup, refactor as needed.
Follow Angular Style guide.
Fix all errors during build.
Goal to separate each Angular feature to stand by themselves when using Augury to demo usage.
Possible idea is to make use of feature NgModules.

Update to Angular 2.2.0 - https://github.com/rangle/augury/issues/792

Dup, closing as this work will be broken out.


GATEWAY_EUI set to 00:00:00:00:00:00:00:00

I wanted to have the ability to remotely access my hotspots so I setup Balena with the RockPi 4B.
The problem I am having appears to be the packet-forwarder using 00:00:00:00:00:00:00:00 and 0 as GATEWAY_EUI.
2022-10-01 07:29:37.095 25 [info] <0.1646.0>@miner_mux_port:dispatch_port_logs:{118,13} [ gwmp-mux ] New packet forwarder client: 00:00:00:00:00:00:00:00, <IP_ADDRESS>:43552
2022-10-01 07:29:37.098 25 [info] <0.1646.0>@miner_mux_port:dispatch_port_logs:{118,13} [ gwmp-mux ] Connecting to server <IP_ADDRESS>:1681 on behalf of 00:00:00:00:00:00:00:00
2022-10-01 07:29:37.098 25 [info] <0.1646.0>@miner_mux_port:dispatch_port_logs:{118,13} [ gwmp-mux ] Connecting to server <IP_ADDRESS>:1682 on behalf of 00:00:00:00:00:00:00:00
2022-10-01 07:29:37.099 25 [info] <0.1753.0>@miner_lora_light:handle_udp_packet:{350,5} PULL_DATA from 0 on 49329
2022-10-01 07:29:37.102 25 [info] <0.1637.0>@miner_gateway_port:dispatch_port_logs:{146,13} [ gateway-rs ] new packet forwarder client: 00:00:00:00:00:00:00:00, <IP_ADDRESS>:47788

Bluetooth also appears to have issues and only works during initial boot/300 second sequence. I hold the button and nothing happens.
gateway-config  Failed to register GATT advertisement: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
I presume I have missed something - is there a copy of the original RockPi 4B firmware for balena so I can revert back (I really should have done this before hand)

@arduino43 you will need to email<EMAIL_ADDRESS>for the firmware for your specific device.
For the packet-forwarder having that EUI it seems you are using the wrong config files or something.


Remove hardcoded API GUID from staticwebapp.config.json

Must either set the API GUID during CI/CD deployment (like AZURE_TENANT_ID) or (better) go back to using AAD v2 endpoint, which shouldn't require that info in the config file.

Completed by switching to AAD v2 endpoint.


IDEA imports the Gradle project, and all the introduction of the package in a implementation way has become a Runtime

IDEA imports the Gradle project, and all the introduction of the package in a implementation way has become a Runtime

build.gradle
...
implementation ('xxxxx:xxxx:xxx')
...

IDEA version: 2018.2           Gradle version: 5.4.1

I'm not sure if you're saying that the import used to work with 4.10.3 and it no longer works in 5.4.1 or if you're saying the import hasn't worked with 4.10.3 or 5.4.1.  If you could provide a small example demonstrating the problem you're seeing, we're happy to take a look.


Annyang does not work on Jetson Nano?

Hi everyone
I am trying to use Annyang for a project on a jetson nano, but it doesn't seem to work. Even on the annyang website, the demo doesn't seem to do anything (I am using the chromium webbrowser). My mic is definitely working, because the demo works if I plug it into a different computer. My audio input settings are also set to my mic.
So is this just a common problem that this does not work with jetson products? Anyone else maybe with a jetson that could verify?
Thanks

Hi, it works in my side, Jetson Nano, I use Apache2 include. You can had some checks to verify if the issue is due to mic, network,browser... It's helpful for debug, add alert for showing a message.


Add assertion messages for all assertion statements

Work by @marcjansen and @bartvde

Build passes on Travis. Please merge.


Clarify node resolution ordering

Explain why we should process nodes by index order rather than random selection, etc.

conflicts exist

@esgoode following up on this? Are you able to fix the conflict?


Add possibility of using actions in DataObjects

It sometimes become handy to be able to execute actions on extracted data.
This can be the case if you have links or buttons in tables or any kind of input fields.
I am currently not 100% sure how to handle return types.
It makes sense for some actions to return the next PageObject. This is for example the case if a click leads to another page to be opened. Other interactions like writing data to a field should return the DataObject.

This is possible now...
Please check the 'Google Search' example : https://github.com/toolisticon/pogen4selenium/blob/develop/pogen4selenium-example/src/main/java/io/toolisticon/pogen4selenium/example/googleseach/GoogleSearchResult.java


Unable to run app and unable to generateJava code (kotlin.sequences.SequencesKt)

Hello! We are unable to run Java and Kotlin apps (also we cannot use generateJava Gradle task) in 5.1.8 (and 5.1.9 too) version. The error is not happening in 5.1.7.
Below, you will find the error traces in Java and Kotlin (very similar).
JAVA
Unable to find method 'kotlin.sequences.SequencesKt.flatMapIterable(Lkotlin/sequences/Sequence;Lkotlin/jvm/functions/Function1;)Lkotlin/sequences/Sequence;'
kotlin.sequences.SequencesKt.flatMapIterable(Lkotlin/sequences/Sequence;Lkotlin/jvm/functions/Function1;)Lkotlin/sequences/Sequence;

Gradle's dependency cache may be corrupt (this sometimes occurs after a network connection timeout.)

Re-download dependencies and sync project (requires network)
The state of a Gradle build process (daemon) may be corrupt. Stopping all Gradle daemons may solve this problem.

Stop Gradle build processes (requires restart)
Your project may be using a third-party plugin which is not compatible with the other plugins in the project or the version of Gradle requested by the project.

In the case of corrupt Gradle processes, you can also try closing the IDE and then killing all Java processes.

KOTLIN
Unable to find method ''kotlin.sequences.Sequence kotlin.sequences.SequencesKt.flatMapIterable(kotlin.sequences.Sequence, kotlin.jvm.functions.Function1)''
'kotlin.sequences.Sequence kotlin.sequences.SequencesKt.flatMapIterable(kotlin.sequences.Sequence, kotlin.jvm.functions.Function1)'

Gradle's dependency cache may be corrupt (this sometimes occurs after a network connection timeout.)

Re-download dependencies and sync project (requires network)
The state of a Gradle build process (daemon) may be corrupt. Stopping all Gradle daemons may solve this problem.

Stop Gradle build processes (requires restart)
Your project may be using a third-party plugin which is not compatible with the other plugins in the project or the version of Gradle requested by the project.

In the case of corrupt Gradle processes, you can also try closing the IDE and then killing all Java processes.

This is most likely a problem of your local maven cache.

Our latest release is 5.1.10, so you could try again with that as well.


Have an icon for DWARF extension

Testing #194071
https://marketplace.visualstudio.com/items?itemName=ms-vscode.wasm-dwarf-debugging
Maybe @hbons could help?

Used this for now:


IDBValidKey does not exist in lib.webworker.d.ts

This type exists in DOM types, but does not exist in Web Worker types. It should exist in both, as IndexedDB spec and implementation is the same for both.
It is defined here.

Types in worker file come from https://github.com/Microsoft/TSJS-lib-generator/blob/master/inputfiles/knownWorkerTypes.json. PRs welcomed adding the missing type.