Add refresh_cycles parameter to from_kafka_batched to accommodate adding Kafka topic partitions on the fly.

Streamz should be able to handle addition of Kafka topic partitions on the fly. The refresh_cycles parameter polls Kafka every N cycles to get the latest Kafka partition metadata and accommodates any added partitions on the fly.
[Some more discussion in #358]

@martindurant Do review when you get time.
Would like your opinion on:

The name refresh_cycles parameter. Maybe kafka_metadata_refresh_cycles is a better name, but too long?
There are too many if self.refresh_cycles is not None clauses, maybe there's a cleaner way to go about this?
Is the test case good enough to test the functionality?

I think the design is fundamentally fine, and I am not worried about too many if is None statements. It could be more compact, but probably not worth doing this.
if self.refresh_cycles:
    if cycles % self.refresh_cycles == 0:
        ...
    cycles += 1

or even
cycles = self.refresh_cycles

if self.refresh_cycles:
    if cycles < 1:
        ...
        cycles = self.refresh_cycles
    cycles -= 1

My main quibble is the name "refresh_cycles" in the signature - what it actually does is check the number of partitions every N polls. So why not call it something more specific like "check_npartitions_every". Within the function, cycles is fine.

Is there a cost to having more partitions than the topic really has? I wonder whether we want to, perhaps at a future date, allow the number of partitions to decrease too.

There's no way to make this work so that it happens automatically by the way the library talks with Kafka? It may be confusing to add this extra parameter. It's not something I've seen when working with other Kafka clients.

Is there a cost to having more partitions than the topic really has?

No, I'm not aware of any cost associated. In fact, it is recommended to add more partitions to balance any increase in load in the Kafka topic and scaling out improving performance.

I wonder whether we want to, perhaps at a future date, allow the number of partitions to decrease too.

I don't think Kafka supports or recommends decreasing number of partitions, because that would lead to data loss. It would be best in this case to just restart the stream.

There's no way to make this work so that it happens automatically by the way the library talks with Kafka? It may be confusing to add this extra parameter. It's not something I've seen when working with other Kafka clients.

We already talked a little about this in #358. An option is to just refresh the partition count every N cycles by default (so the exact same logic of this PR without exposing it via an additional parameter), without providing an option to the user to configure what N would be. We don't need N to be 1 necessarily, since a concern was that not all users might want this extra call or functionality to be executed in each poll. Also, some users have a really small poll interval, so having this in every poll cycle might not be worth it. But Martin thought it would be best to allow the user to configure this functionality somehow.

@jsmaupin , by "extra parameter" in your comment, were you referring to the kwarg (currently refresh_cycles), or the extra config you need to pass to the kafka setup?

were you referring to the kwarg (currently refresh_cycles), or the extra config you need to pass to the kafka setup?

@martindurant, we don't need to pass any extra config in the Kafka setup. I think Jarod was referring to refresh_cycles (now check_npartitions_every :))

I'm referring to the refresh_cycles parameter.

@chinmaychandak , are you able to do some kind of benchmark to see the effect of polling for the number of topic partitions on every cycle?

@chinmaychandak , are you able to do some kind of benchmark to see the effect of polling for the number of topic partitions on every cycle?

Okay, I can try some benchmarking. Do you have something specific in mind?
Although, this is what I've seen with a few things I've tried out:
The size of the metadata returned for a specific topic (this is important, else the metadata for the entire Kafka cluster can be really large) by list_topics(streamz-topic) is 64 bytes (as returned by sys.getsizeof). This is smaller than a single typical average Kafka message (anywhere b/w 100 bytes to a few KBs). This also means that the latency of retrieving this metadata, once every poll cycle, would be much lesser than that for retrieving a single Kafka message (and <<< than the latency of reading an entire batch of messages from multiple partitions as intended by from_kafka_batched).
I ran a small test, and the time taken for list_topics(streamz-topic) is an average of 25ms (across 100 cycles) for a topic with 8 partitions, and if I increase the number of partitions to 16, the time still remains the same. This is really minimal compared to the time taken to read an entire batch of messages (which can easily be in order of 10s of seconds).
But, as I said before, if the poll interval is ridiculously small (although I can't think why someone would want it to be less than at least 2-5 seconds, otherwise they're effectively not taking advantage of batching), calling list_topics on every cycle can be a problem.

We clearly have very different idea of the use case, as we found in our last discussion about this :)
Although the high-throughput, big-batch use case may be very important, in which any latency and overhead from streamz (and dask) are eaten by the downloading of kafka and further processing. That will be the case indeed when you have poll times >1s.
However, streamz is developed to be a real-time processing system. It is perfectly reasonably to have kafka polling many times a second, perhaps with only a few or even no events per batch. Perhaps the most important thing is to have output as quickly after an event appears as possible - but still need batch to deal with spiky traffic.

size of the metadata returned for a specific topic by list_topics(streamz-topic) is 64 bytes

This is probably unimportant: the data transfer is likely a very small compared to the connection time. It may well be that listing partitions, and polling offsets amounts to double the latency compared to polling offsets alone.

the time taken for list_topics(streamz-topic) is an average of 25ms

On a local cluster, the typical time to submit, execute and return a value from dask is ~5ms; submit alone is <<1ms and typical task overheads are <<1ms. The per-node latency in streamz is ... tiny (really should benchmark this! perhaps the new metadata stream could include time-of-execution...). So whether or not 25ms seems like a lot depend on the use case. How long does getting offsets take? Is this with kafka on a separate container within the same host?

By the way, since streamz is an async application (on tornado), https://www.confluent.io/blog/kafka-python-asyncio-integration/ might be interesting, as well as maybe aiokafka (which seems unmaintained). I don't know to what extent multiple kafka calls could be made concurrently, but if that works, maybe the various latencies can be amortized.

Okay, I understand the use case you're talking about now. I did feel that calling list_topics on every cycle could be problematic when the poll interval is really small.

On a local cluster, the typical time to submit, execute and return a value from dask is ~5ms; submit alone is <<1ms and typical task overheads are <<1ms.

What kind of executions/processing are we talking about here?

Is this with kafka on a separate container within the same host?

25ms was with Kafka on AWS, and the consumer on my local machine. With Kafka within the same host, the latency seems to be <<1ms, and within the same VPC but from a separate machine, the latency seems to be ~1ms.

How long does getting offsets take?

Getting a single message seems to take <<<1ms.
I feel that considering the numbers above ^ it should be safe to call list_topics on every cycle, although I'm really okay with keeping the PR as is, if you feel that would be more comfortable with all kinds of use cases.

Maybe this was already discussed and I missed it, but I understand that the Kafka consumer generally has a "re-balance trigger" where a trigger is activated when something happens like a new partition is added. There's no way we can tie into this triggering mechanism to schedule a metadata re-download?

What kind of executions/processing are we talking about here?

The simplest of functions, i.e., this is all overhead/latency. Obviously, real workloads take longer,
but the blocking part from the client's point of view should not change much.

Getting a single message seems to take <<<1ms

I don't mean consuming a message, but polling the offsets, which is what the streamz node spends its time on.
Note that the streamz event loop may be watching several inputs, e.g., from sockets or files, so having the kafka callback fast is important.

I don't mean consuming a message, but polling the offsets, which is what the streamz node spends its time on.

I confused this with the poll method which consumes the actual messages.
Okay, so without list_topics on every cycle, the avg. time to poll offsets for 100 (or 10000) cycles is around 21ms. With list_topics on every cycle, it is around 23ms.

Maybe this was already discussed and I missed it, but I understand that the Kafka consumer generally has a "re-balance trigger" where a trigger is activated when something happens like a new partition is added. There's no way we can tie into this triggering mechanism to schedule a metadata re-download?

I can't seem to find a trigger config anywhere. There is the topic.metadata.refresh.interval.ms config which I've used in the test added in this PR and it defaults to 5mins.

around 21ms. With list_topics on every cycle, it is around 23ms

Oh! That would suggest not bothering with the parameter, and only providing yes/no to checking. I am surprised. However, I am also surprised that 21ms is as high. Seems I don't understand the technicalities of the TCP connection that ck keeps open. :|

That would suggest not bothering with the parameter, and only providing yes/no to checking.

So are you suggesting a boolean parameter (refresh_partitions? :)) instead defaulting to True?

However, I am also surprised that 21ms is as high.

Yes, I was surprised too. Profiling at a little more granularity, get_watermark_offsets is taking about 2.5ms per call.

Right, so the actual latency for a call to kafka is in the 2ms range - so checking for partitions is still significant in the case of few starting partitions.

So are you suggesting a boolean parameter (refresh_partitions? :)) instead defaulting to True?

I would have said False :). The only reason would be, that this is simpler to understand; but given that the code is already there, this is fine too, but the docstring needs to be precise. I think the current version will do, but may need some clarification later (perhaps in examples).

Done!

@martindurant, is this good to go? Just one more PR in the pipeline after this. :)

@martindurant can you please help cut a release sometime this week? Would love to use the fixes/features of #358 and this PR in our existing workloads. :)

Remind me next week

@martindurant, bump ^

On pypi


libvirtd restarts in cycles each 10 minutes with error message in system logs

Description
libvirtd on the hypervisor restarts every 10 minutes under the user 'oneadmin,' while there is already a process running under the root user.
root 8590 0.6 0.0 1632864 47432 ? Ssl 10:22 0:09 /usr/sbin/libvirtd
oneadmin 63387 0.0 0.0 1547460 38376 ? Sl 10:46 0:00 /usr/sbin/libvirtd --timeout=120

When the process is initiated as 'oneadmin,' the following message also appears in the syslog.
Jan 4 10:46:57 cloud-testbed001 libvirtd[63387]: libvirt version: 8.0.0, package: 1ubuntu7.7 (Michal Maloszewski <[michal.maloszewski@canonical.com](mailto:michal.maloszewski@canonical.com)> Fri, 04 Aug 2023 10:42:25 +0200)
Jan 4 10:46:57 cloud-testbed001 libvirtd[63387]: hostname: cloud-testbed001
Jan 4 10:46:57 cloud-testbed001 libvirtd[63387]: Failed to open file '/sys/kernel/security/apparmor/profiles': Permission denied
Jan 4 10:46:57 cloud-testbed001 libvirtd[63387]: Failed to read AppArmor profiles list '/sys/kernel/security/apparmor/profiles': Permission denied
Jan 4 10:46:57 cloud-testbed001 libvirtd[63387]: Failed to open file '/sys/kernel/security/apparmor/profiles': Permission denied
Jan 4 10:46:57 cloud-testbed001 libvirtd[63387]: Failed to read AppArmor profiles list '/sys/kernel/security/apparmor/profiles': Permission denied
Jan 4 10:46:57 cloud-testbed001 libvirtd[63387]: Failed to open a VPD file '/sys/bus/pci/devices/0000:45:00.0/vpd': Operation not permitted
Jan 4 10:46:57 cloud-testbed001 libvirtd[63387]: Failed to open a VPD file '/sys/bus/pci/devices/0000:45:00.1/vpd': Operation not permitted
Jan 4 10:46:57 cloud-testbed001 libvirtd[63387]: Failed to open a VPD file '/sys/bus/pci/devices/0000:45:00.2/vpd': Operation not permitted
Jan 4 10:46:57 cloud-testbed001 libvirtd[63387]: Failed to open a VPD file '/sys/bus/pci/devices/0000:45:00.3/vpd': Operation not permitted

To Reproduce
Install latest 6.8.x OpenNebula version (e.g. via minione) and check on hypervisor node a system logs for error messages as above.
Expected behavior
No error messages should be like ones listed in that issue.
Details

Affected Component: libvirtd 8.0.0 (8.0.0-1ubuntu7.7)
Hypervisor: KVM
Version: 6.8.x
OS: Ubuntu 22.04.3 LTS

Additional context
https://opennebula.zendesk.com/agent/tickets/48971
https://forum.opennebula.io/t/libvirtd-starts-in-cycles-of-10-minutes/11733

Progress Status

[ ] Code committed
[ ] Testing - QA
[ ] Documentation (Release notes - resolved issues, compatibility, known issues)

I would like to add a few things I noticed.
We started seeing this behaviour, the multiple libvirtd processes, after upgrading OpenNebula from 6.6.x to 6.8.x, the behaviour described here seems to be a correct out of the box experience using libvirtd with systemd on Ubuntu 22.04 and also 24.04 when accessing libvirt as oneadmin or any other user for that matter.
If you access libvirt, with oneadmin or any user for that matter, a libvirtd process is spawned. Below an example on what you see if userX does a simple virsh list:
host4:~# ps uax|grep libvirtd
root     4007562  5.8  0.0 6575520 46688 ?       Ssl  jun11 3818:44 /usr/sbin/libvirtd

host4:~# sudo -u userX virsh list --all
 Id   Name   State
--------------------
root@host4:~# ps uax|grep libvirtd
userX    2697238 49.0  0.0 1547440 27648 ?       Sl   10:47   0:00 /usr/sbin/libvirtd --timeout=120
root     4007562  5.8  0.0 6575520 46688 ?       Ssl  jun11 3818:44 /usr/sbin/libvirtd

Though I have not deepdived into libvirt on Ubuntu to see if this is the behaviour they want it too be, it is the behaviour that comes out of the box on 22.04 and 24.04, older versions I have not checked.
A small part from the config, /etc/default/libvirtd that is installed by the package libvirt-daemon-system, seems to indicate it is expected bahviour:
# The default upstream behavior is for libvirtd.service to
# start on boot, perform VM autostart and shutdown again if
# nothing was started; later on, systemd socket activation
# is used to start it again when some client app connects.

To figure out then why we are seeing 'restarts' in a 10 minute interval the perception might have better been switched to who are what is accessing libvirt on a 10 min interval as user oneadmin.
From there we quickly come to the SYSTEM_HOST interval for the monitor probe.
When I configure the SYSTEM_HOST interval down from 600 to less 120 in the monitord config,  the process keeps running as it will not trigger the default timeout of 120 seconds.
I then guessed that pre 6.8.x sudo was always used in the probes and since 6.8.x somewhere a command is ran without sudo.
Looking into the relation of the timeout with the SYSTEM_HOST interval I suspected the cause being in the 'im/kvm-probes.d/host/system/cpu_features.sh' where the script contains:
FEATURES=$(virsh capabilities | grep '<feature name' | sed -e "s/^.*='//;s/'\/>$//" | xargs | tr ' ' ',')

This script is not present in 6.6. If anyone wants to get rid of the multiple libvirtd processes you can add a sudo entry for the command on the hypervisor and update the command in the probe to make use it. But in the end it doesn't really seem to be an issue, just a change in behaviour between 6.6 en 6.8 which got noticed because of the oneadmin user running some virsh command now. The moment you will run anything as userX or userY you will get extra libvirtd processes and see the same thing.

Hi team
We had same issue with libvirtd 8.0.0 (8.0.0-1ubuntu7.6). There is a a new compilation of the package available on ubuntu 22.04 repos (8.0.0-1ubuntu7.10) and apparently the release notes said something about fixing this issue. We will watch this cluster where we had to upgrade the libvirtd and share any information if issue comes back again.

Add a note to the known issues


sort: don't rely on serde-json for extsort

It is much faster to just write the lines to disk, separated by \n
(or \0 if zero-terminated is enabled), instead of serializing to json.
external_sort now knows of the Line struct instead of interacting with
it using the ExternallySortable trait. Similarly, it now uses the
crash_if_err! macro to handle errors, instead of bubbling them up.
Some functions were changed from taking &[Line] as the input to taking
an Iterator<Item = Line>. This removes the need to collect to a Vec
when not necessary.
@electricboogie I think you're most familiar with the code here, would you mind reviewing this? Sorry that this is such a huge change.

Do you have some perf wins?

Yes, this should be about 80% faster, but it is hard to say because we previously used more memory than what the flag told us to. I updated BENCHMARKING.md with benchmarking instructions.
We're still multiple times slower than GNU sort, especially on large files.

in benchmark, could you please provide some hyperfine examples to reproduce the improvement? thanks

Conceptually, this works, but I think you won't get the benefit of any cache as you do a final compare_by as you iterate(call next()) through the returned iter (the final pass).  So, it may perform worse on certain inputs, but I doubt it.
However, I think we may be prematurely optimizing here.  It may be the case that we find that we should write the entire struct to disk as, say, it is fastest for much larger inputs, or we may find another use case in the Line struct that is worth the bytes.
My take is -- I think the real problem is the memory usage.  We are still consuming 3 to 4x more memory than GNU because of the size of the Line struct.  Maybe we get our data structures/the Line struct to a place we are happy with first?  My opinion is that you were right the first time -- we need to push out the NumCache and Field/Other stuff to a separate structure.  I suggested a few possible (naive!) solutions on the issue page.  I'd be pleased to assist anywhere you'd need me.

I'm reasonably sure that reconstructing the cache at the last pass is much faster than serializing it to json. We also need to reduce memory usage, that's for sure. I promise I'll look into that :) let's see when I have time for this though 😅
I'll have to find out how to reliably measure memory usage, because at some point it looked like we're using less memory than GNU, which doesn't make much sense to me.
There's one bug that I'm fixing here that made the current version unusable though (not related to the json stuff), which is that we were always reading all lines before sorting, making all extsort useless (everything was loaded into memory anyways). You can't sort a big (something like 2GB) file with the current implementation. I also made the estimation on how much memory we're using more correct.
We could also leverage the current implementation for -m for the final merge step (to reduce code duplication, and what we have there (a BinaryHeap) should be faster than the algorithm in extsort when considering the algorithmic complexity)

Totally agree extsort is not yet ready for prime time -- memory usage looks like a V shape in Valgrind.  I was going to do some work to module-ize the extsort::sort_by implementation to accept chunks.  But after screwing around with that for awhile, I thought 2 things should probably come first: 1) we break input into chunks when feeding from the buffer, and 2) we make a sure we are happy with our data structures, because who wants to rework the extsort again.


Remove version: Jackett.Jackett version 0.21.1212

404 - release deleted
Microsoft Reviewers: Open in CodeFlow

URL: https://github.com/Jackett/Jackett/releases/download/v0.21.1212/Jackett.Installer.Windows.exe
Response Error: The remote server returned an error: (404) Not Found.
(Automated message - build 918)


Usage of Storage Provisioner for k3s confusing

Internal reference: SURE-4008
Reported in 2.6.3
K3s version: 1.21
(Wording gets tricky as there's the initial report and then subsequent investigation that points to UI.)
Issue description:
There is some confusion with respect to the Azure File Share Usage in Rancher 2.6.0 for storage class.
There are two provisioners and the user wanted to know the difference in usage and use case.

provisioner: kubernetes.io/azure-file
provisioner: file.csi.azure.com

When using K3S cluster, there is option to select the provisioner.
Business impact:
Without knowing the impact, unable to proceed the further installation. The user is stuck at K3S provisioning on Azure without knowing the impact of using different storage provisioner for K3S. If you could validate the approach and provide the feedback can be the solution here and we don't have much visibility due to the lack of documentation for same.
Two approaches considered:
Approach 1:
The Storage class provisioner: kubernetes.io/azure-file which is also in the Rancher UI in any imported cluster.
With the above approach and K3S as the imported cluster, failed to use azure file share. ( as k3s doesn’t have storage driver in it).
And we are not sure which particular driver is needed for this provisioner : kubernetes.io/azure-file
Approach 2:
In this approach , installed the Azure File Share CSI driver (as helm chart) in K3S cluster and azure secret from  storage account credentials.
The Storage class in this case has provisioner as: provisioner: file.csi.azure.com and we were able to get the file share created dynamically in the Azure Storage account with a file share.
Installing CSI driver Helm chart:
https://github.com/kubernetes-sigs/azurefile-csi-driver/tree/master/charts#install-a-specific-version
Creating azure-secret and storage class and then deploying this custom application which successfully created file share:
https://github.com/kubernetes-sigs/azurefile-csi-driver/blob/master/deploy/example/e2e_usage.md#option2-bring-your-own-storage-account-only-for-smb-protocol

Backend already looked into this and it was determined it tied into the UI; they looked into this issue to see where the UI gets the information for the CSI driver and to see how Rancher might be able to display this properly.
From their understanding, this is a UI issue, and needs to be updated here: https://github.com/rancher/dashboard/blob/master/shell/models/storage.k8s.io.storageclass.js as well as a few other places.
Repro steps:
There is no relevant documentation which is giving the hint to approach this scenario.
Workaround: none
Actual behavior:
No options to select storage provisioner from Rancher UI for K3S imported cluster.
Expected behavior:
The storage provisioner should be selected from Rancher UI
Files, logs, traces:
No Logs
Additional notes:
Kubernetes project in using CSI driver for Kubernetes instead of traditional in tree Volume plugin

The crux of the issue is that one of the provisioners is deprecated upstream and may not work, but is marked as supported by the UI. The provisioner that is currently recommended (and should work if deployed properly) is marked as unsupported by the UI. Most likely we just need to update the documentation and UI to track upstream changes to how they recommend deploying the azure-file CSI provisioner.

This would be question for @snasovich or @jakefhyde

According to this, I think marking current default as deprecated is the right choice, given 1.26's expected release day of 12/6.

Test Environment:
Rancher version: v2.7-head a2ef50b
Rancher cluster type: HA
Docker version: 20.10
Downstream cluster type: imported k3s cluster

Testing:
Tested this issue with the following steps:

Import a k3s cluster into Rancher
Wait for cluster to be imported successfully
Go to explore cluster
Install CSIDriver using link in the description https://github.com/kubernetes-sigs/azurefile-csi-driver/tree/master/charts#install-a-specific-version
After installation is successful, go to Storage section
Create a storage class and verify you can select provisioner

Result
User is able to select provisioner when creating storage class whereas before was unable to


Add 505 by Arctic Monkeys

Song Name
505
Artist Name
Arctic Monkeys
Source
Youtube
Youtube Link
https://youtu.be/1Jd2DysfCW8?si=p7XSWwiiFQNTmvEr
Direct File Link
No response
Song ID
48
Start Offset [ms]
0
Extra
Created from Auto Nong.
Created for level: 107553957
plz

accept


NPCs

Special NPCs with extra special abilities. Interacted with through dialogue. Each costs 1 prestige coin to unlock.
The Pirate
Two ideas so far for this one, either or:

Finds extra special deals, or items that otherwise don't spawn in the normal shop
Show (sell? give) him a ship (or any other item!?). After a while, he will add it to a list of "favourite items", which you can buy from him at any time, for a premium

The Mechanic
Can apply ship upgrades to any ship at any time, for money. Needs some kind of restriction... cooldown?
The Painter
A talented artist who can't design! Give them ship skin designs, and they will remember them forever, letting you apply the skin to any ship at any time, for credits.
The Diplomat
If we ever have faction reputation, this guy could fix it for you. Possibly with other benefits too.
The Scientist
A Grey. Give me three of the same item, and i will transmute them into one item of the next tech level up.
The Miner
Hire a midorian to go off and mine for you. Passive credit generation. Will need balancing for tech level - perhaps when you level up he is able to find rarer ores?

Azloxion's design: https://discord.com/channels/711548456019296289/721311036891725856/946905910389932043
Levelled NPCs that are based in specific systems, perhaps that you can hire in your own 'home' system

Azloxion suggests making the skin of a ship overwritable, not removable: https://discord.com/channels/711548456019296289/721311036891725856/947252517845209158

Azloxion suggests being able to teach the painter skins from a ship, perhaps by temporarily losing the ship, instead of just from skin tools: https://discord.com/channels/711548456019296289/721311036891725856/947252841515470898

Azloxion suggests placing NPCs inside of specific systems (#799):
https://discord.com/channels/711548456019296289/721311036891725856/946905910389932043
Questions:

Are there NPCs in fixed (or random?) locations across the galaxy? Or are they only in the kaamo club?

Bringing this forward to v2 because currently there arent really any rewards for prestiging
I'll probably only do one, maybe 2 npcs


Powershell script to rename the project to something different

#JustJust in case anyone needs it...
put the following into something like renameProject.ps1,  then put it in the root directory of the project
then just do ./renameProject.ps1 NextBigThing
Note the script will modify itself so you can rename it again if need be.
param(
[string]$projectName
)

$files = Get-ChildItem . *.* -rec -Attributes !D
foreach ($file in $files)
{
    (Get-Content $file.PSPath) |  Foreach-Object { $_ -replace "Vue2Spa", $projectName } |  Set-Content $file.PSPath
}
Rename-Item .\Vue2Spa.csproj $projectName".csproj"
Rename-Item .\Vue2Spa.sln $projectName".sln"

Does it also update the namespaces within the c# files (just asking).

You can create the project with dotnet new vuejs -n YourName. It's a standard way to give name to your projected based on a template.


when push to pivotal cloud foundry , pop up : "cp: cannot stat '/tmp/app/ui/core/assets': Permission denied"

Hi  guys:
I really can not find where these error occurs. below is the error log when we execute "cf push".
Could you please help check it , is this a bug?or where we could change the code to fix it?
Thanks
David
Context

go: finding github.com/spf13/cast v1.3.0
go: finding github.com/docker/distribution v2.7.1+incompatible
go: finding github.com/docker/go-connections v0.4.0
go: finding github.com/containerd/continuity v0.0.0-20181203112020-004b46473808
go: finding github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d
go: finding sigs.k8s.io/kustomize v2.0.3+incompatible
go: finding github.com/morikuni/aec v0.0.0-20170113033406-39771216ff4c
go: finding github.com/docker/go-metrics v0.0.0-20181218153428-b84716841b82
go: finding github.com/hashicorp/golang-lru v0.5.3
go: finding github.com/google/go-cmp v0.3.1
go: finding github.com/go-openapi/spec v0.19.4
go: finding github.com/MakeNowJust/heredoc v0.0.0-20171113091838-e9091a26100e
go: finding github.com/russross/blackfriday v1.5.2
go: finding github.com/go-openapi/jsonpointer v0.19.3
go: finding github.com/mitchellh/go-wordwrap v1.0.0
go: finding github.com/emicklei/go-restful v2.11.1+incompatible
go: finding github.com/go-openapi/jsonreference v0.19.3
go: finding github.com/go-openapi/swag v0.19.5
go: finding github.com/docker/spdystream v0.0.0-20181023171402-6480d4af844c
go: finding github.com/PuerkitoBio/purell v1.1.1
go: finding github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578
Build complete ...
Disk usage for cache and app folders:
All done
-----> Stratos build complete [03:38:38]
-----> Build time:
-----> 1m 58s
cp: cannot stat '/tmp/app/ui/core/favicon.ico': Permission denied
cp: cannot stat '/tmp/app/ui/core/assets': Permission denied
Failed to copy compiled droplet: exit status 1
Exit status 1
Cell ee8959ec-6546-4ae1-a34d-cea7e0b68687 stopping instance 3298a294-0c51-4edb-a356-deee4cbe8846
Cell ee8959ec-6546-4ae1-a34d-cea7e0b68687 destroying container for instance 3298a294-0c51-4edb-a356-deee4cbe8846
Error staging application: Staging error: staging failed
FAILED

Hi @david0718 , thanks for raising this issue. Could you confirm which version of Stratos you're using, which version of CF you're pushing to and whether you ran any commands before cf push (for instance npm run prebuild-ui)?

Hi @richard-cox
The cf is 2.6.9, and before cf push , I could run "npm install" and "npm run prebuild-ui" successfully.
Not quite sure if the security policy is consistent between pivotal cloud foundry and cloud foundry.
You know we could see
-----> Stratos build complete [03:38:38]
-----> Build time:
-----> 1m 58s
means the compile phase is good.
So the issue appears in staging phrase. :(
Do not know how to do with this.
I just use another way, to build a image, and push to harbor, I could cf push the docker version manifest file to pivotal cloud foundry. Just do not want to waste any of time.
Thanks
David

Yes this is strange. It's not something we've ran into in the past, though we haven't used PCF. Which version/branch of Stratos are you pushing? Is it possible something locally is changing the permissions of the files before being pushed?

@david0718  There seems to be a similar issue when using Windows (#4428), could you confirm what OS you're using?


Vulnerability check reported failure on main - Wed Jul 20 00:36:39 UTC 2022

https://github.com/nodejs/nodejs-dependency-vuln-assessments/actions/runs/2701477230
Invalid search criteria syntax: <Response [403]>

I also get this error sometimes (randomly). It looks like either a problem with the NVD server, or the script making too many requests in a short amount of time (triggering the rate limit). If it's the second one, it should improve when we start using an API key (which increases the rate limit for queries). If it's the first one, we need to account for it in the script.

@facutuesca it seems to be consistent today with all runs attempted hitting that error.  I still get it after merging your PR to improve the captured message.

This is the failure from recent failures:
Invalid search criteria syntax: <Response [403]>
Attempted search criteria: {'keyword': 'undici'}
Traceback (most recent call last):
  File "/home/runner/work/nodejs-dependency-vuln-assessments/nodejs-dependency-vuln-assessments/node/tools/dep_checker/main.py", line 168, in <module>
    exit(main())
  File "/home/runner/work/nodejs-dependency-vuln-assessments/nodejs-dependency-vuln-assessments/node/tools/dep_checker/main.py", line 151, in main
    nvd_vulnerabilities = query_nvd()
  File "/home/runner/work/nodejs-dependency-vuln-assessments/nodejs-dependency-vuln-assessments/node/tools/dep_checker/main.py", line [12](https://github.com/nodejs/nodejs-dependency-vuln-assessments/runs/7420307944?check_suite_focus=true#step:5:13)4, in query_nvd
    for cve in searchCVE(cpeMatchString=dep.get_cpe(), keyword=dep.keyword)
  File "/opt/hostedtoolcache/Python/3.9.[13](https://github.com/nodejs/nodejs-dependency-vuln-assessments/runs/7420307944?check_suite_focus=true#step:5:14)/x64/lib/python3.9/site-packages/nvdlib/cve.py", line 307, in searchCVE
    raw = __get('cve', parameters, limit, key, verbose)
  File "/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/nvdlib/get.py", line 41, in __get
    totalResults = raw['totalResults']
TypeError: 'Response' object is not subscriptable
Error: Process completed with exit code 1.

@facutuesca it seems to be consistent today with all runs attempted hitting that error. I still get it after merging your PR to improve the captured message.

@mhdawson The merged PR was not related to this issue. This error should improve once we start using the NVD API key (PR: https://github.com/nodejs/node/pull/43909)

Resolved now that PR landed and we added the NVD api token, closing.


Fix the ODE example

This removes the dependency on cassette for this example and harmonizes it with the other examples

Merged as part of #134


[BUG] Car bumper recognized as face. Need a way to teach that something recognized as a face is not one.

Describe the bug
Detecting car as face. No way to "teach" the detector to ignore this particular "face".
Version of Double Take
1.6.0
Expected behavior
It's saying with 98% confidence that a car is a face... this is clearly wrong. Is there a way to teach detectors to ignore "faces"? Basically a function to say "learn this image is not a face".
Screenshots

Hardware

OS: Docker on Debian

Browser (if applicable) Not applicable

Additional context
There are so many false positives and also detections of faces that aren't the same person but with high confidence. There should be a function (API permitting) to ask the detectors to learn "this image is NOT this person" so that it would improve detection by ignoring that image. On the screenshot above I have literally hundreds of detections all above 90% confidence because it's night time so the noise in the image triggers detection. It's spamming the matches screen.

Can second that. My cat is not a man 👍

The problem with this is not in the double take app, but in the deepstack or compreface applications or whatever you use..

The problem with this is not in the double take app, but in the deepstack or compreface applications or whatever you use..

This is why I said "API permitting". Do you know if there is an API call for "untraining" a specific "thing" against a name on the database? If there is then DoubleTake should offer that for cases like these when you want to give the API a set of images to tell it that it ISN'T what it thinks it is.

Are you using Deepstack by any chance?
If so I would recommend giving Compreface instead if you are able to, since moving over I haven't had a single silly detection.

I am on Compare face. It still thinks my cat is a person....
No false training done.
Problem is it is a general problem that you cant train negatives. Almost all AI staks support that feature.

Was there ever a resolution to this? Compare this my mailman is me all the time.


choose file button not taking file input after first upload is done

Can you assign this task to me

done

Can I do it on two pull requests

I would like to work on this issue, can you assign this task to me


profileManager.get(false) returns Optional.empty

Hi
I'm using
Play 2.7,
play-pac4j 7.0
HeaderClient and JWT
I can authorize request or reject unauthorized request properly but only having problem with getting the profile from the request.

The headers in the request does contain "authorization Bearer ....."

Any idea what might go wrong?
Thank you

Please use the mailing list for questions: https://groups.google.com/forum/?fromgroups#!forum/pac4j-users


feat: sha input

When the head SHA is different from the SHA that triggered the workflows (say, because there's a workflow that bumps the release and commits after the merge), the action will fail to find the upstream workflows; this adds a parameter called sha which can be set to ${{ github.event.workflow_run.head_commit.id }}.

not sure what to do about the linter errors, some seem unrelated to my changes (markdown), one doesn't seem resolvable (the await thing), and the assignment error is because I got confused by the syntax and didn't want it to explode (this kind of javascript is not exactly my first language)

dont worry about the linter ... it complains a lot :P

this is a good find ... thank you! left some reviews to address before I can merge

btw, I think the debug logging can be toned down a bit (https://github.com/ahmadnassri/action-workflow-run-wait/blob/master/action/lib/deduplicate.js#L18 tends to be very noisy and not very useful, resulting in logs being truncated randomly (?) for some reason) and the deduplication seems to not work, but I'll probably poke at it more next week


Model Modelica.Media.Examples.Tests.Components.PartialTestModel changed in a non-backwards compatible way

Reported by otter on 19 Apr 2013 09:04 UTC
The models Modelica.Media.Examples.Tests.Components.[wiki:PartialTestModel] and PartialTestModel2 have been changed in a non-backwards compatible way by utilizing components from Modelica.Fluid instead of Tests.Components and these components have slightly different parameters. As a result, existing models that change these parameters (like ambient.p_ambient) will result in an error (as it happens for MoMoLib).

Migrated-From: https://trac.modelica.org/Modelica/ticket/1080

Comment by otter on 19 Apr 2013 09:09 UTC
Fixed in 30bd5c300784097a9c95da932f609c3231413c76: Renamed PartialTestModel to PartialTestModel3 and partialTestModel2 to PartialTestModel3. Re-introduced the original PartialTestModel and PartialTestModel2 from MSL3.2 and marked them obsolete. Replaced the usage of PartialTestModel in all examples by PartialTestModel3

Comment by otter on 22 Apr 2013 14:29 UTC
Changed fixed in 431ad409935752dc5a35726f002362a1f1b33c1e:
There is still the (very small) risk that users use the test models in Modelica.Media.Examples.Tests.[wiki:MediaTestModels] and then this is not backwards compatible. Therefore, re-introduced the test models from MSL3.2.

Modified by otter on 20 Dec 2016 15:09 UTC


Benq EW2780U not supported

Checklist

[x ] I have searched for existing issues
[x ] I have looked through the Monitor Troubleshooting wiki
[ x] I have updated MonitorControl to the latest version

Describe the issue
The Benq EW2780U has DDC switched on but the monitor appears as not supported and none of the actions (brightness or volume) work.
Expected behavior
The Benq EW2780U to be recognised as Benq EW2780U and all the actions (brightness or volume) to work.
Screenshots

Other information
Having run ddcctl as suggested in the the Monitor Troubleshooting wiki, I found the following failures on each command (here setting the brightness to 1):
$ ddcctl -d 1 -b 1
D: CGDisplay D9F5565C-F61A-F34E-3D19-FC1626C9B248 dispID(#1) (1920x1080 0°) HiDPI
I: found 1 external display
E: Failed to parse WindowServer's preferences! (/Library/Preferences/com.apple.windowserver.plist)
E: Failed to acquire framebuffer device for display

Environment Information (please complete the following information):

macOS version: 11.1
MonitorControl version: 2.1.0 (Build 721)
Monitor(s): Benq EW2780U
Mac model: Mac mini (M1, 2020)
Monitor Cable(s)/Connection(s): HDMI 2.0

Okay, just realised this is most likely down to this issue around M1 support in general #323.

Hi all
For your info
I have a MBP TouchBar (Intel) and the same monitor that I ca'nt control
The monitor is connected via : usb-c hub --> miniDP to DP cable
Connecting via HDMI does not allow to access full resolution of the monitor, thus an option for me
I'm waiting for a usb-c to DP cable for another try
Rgds

#323


fix(docs/customization): Add django.db.models import in the example

This is a minor doc fix in one of the examples where fields from django.db.models are used but it is not imported in the first place.
The proposed change fixes this.

Thanks @CuriousLearner! Merged in f9753f1f2328940231e40927e54249aa708adb62.

Thanks @gasman :)


Configure CMake Graphviz feature.

CMake can generate Graphviz files showing the dependencies between the targets
in a project, as well as external libraries which are linked against.
This configuration removes the clutter from the dependency graphs of OrbitQt
and OrbitService.

Hi,
how does this interact with CMake? I would assume this needs to be somehow included in the main CMakeLists.txt, doesn't it?

Hi,
how does this interact with CMake? I would assume this needs to be somehow included in the main CMakeLists.txt, doesn't it?

Good point. I intended to share the doc with the context but forgot. Done that now.
tl;dr
cd build_default_relwithdebinfo
cmake --graphviz=dependencies_orbit_qt.dot ..
dot -Tpng -o dependencies_orbit_qt.png dependencies_orbit_qt.dot

The filename CMakeGraphVizOptions.cmake is magic to CMake and the config gets picked up automatically.

Hi,
how does this interact with CMake? I would assume this needs to be somehow included in the main CMakeLists.txt, doesn't it?

Good point. I intended to share the doc with the context but forgot. Done that now.
tl;dr
cd build_default_relwithdebinfo
cmake --graphviz=dependencies_orbit_qt.dot ..
dot -Tpng -o dependencies_orbit_qt.png dependencies_orbit_qt.dot

The filename CMakeGraphVizOptions.cmake is magic to CMake and the config gets picked up automatically.

Ah I see. I had a look at the code. It seems CMakeGraphVizOptions is a CMake module  which means it is resolved via the CMAKE_MODULE_PATH. Our cmake/ subdirectory is in the module path, so my guess is that it would work fine when the file lives there. It would be great if you could give that a try since I'm currently trying to remove all unnecessary files from the repo root.

Ah I see. I had a look at the code. It seems CMakeGraphVizOptions is a CMake module which means it is resolved via the CMAKE_MODULE_PATH. Our cmake/ subdirectory is in the module path, so my guess is that it would work fine when the file lives there. It would be great if you could give that a try since I'm currently trying to remove all unnecessary files from the repo root.

Okay, that did not work.
https://cmake.org/cmake/help/latest/module/CMakeGraphVizOptions.html

This file is first searched in CMAKE_BINARY_DIR, and then in CMAKE_SOURCE_DIR.

CMAKE_SOURCE_DIR is the repo root. CMAKE_BINARY_DIR are the ephemeral build... directories I guess.

Ah I see. I had a look at the code. It seems CMakeGraphVizOptions is a CMake module which means it is resolved via the CMAKE_MODULE_PATH. Our cmake/ subdirectory is in the module path, so my guess is that it would work fine when the file lives there. It would be great if you could give that a try since I'm currently trying to remove all unnecessary files from the repo root.

Okay, that did not work.
https://cmake.org/cmake/help/latest/module/CMakeGraphVizOptions.html

This file is first searched in CMAKE_BINARY_DIR, and then in CMAKE_SOURCE_DIR.

CMAKE_SOURCE_DIR is the repo root. CMAKE_BINARY_DIR are the ephemeral build... directories I guess.

I see. Thanks for pointing that out. That's definitely a weird CMake module.


feature suggestion: "raw mode" toggle

Type: Feature Request
When I type a character in a text-based file, a lot of things can happen: a letter can trigger an intellisense popup, a bracket can insert a matching closing bracket, pressing ENTER can indent the following line and add text (e.g. \item inside a TeX enumeration).
Most of the time, this is useful, but sometimes - especially when editing existing code or TeX source - it gets in the way. I could manually go into settings and turn all this off, then turn it back on again when I'm done importing and cleaning up code and back to writing my own - but laziness is a programmer virtue.
I suggest adding an option to enable/disable what I'll call "raw mode" (in analogy to the terminal) where any kind of auto-completion, auto-brackets, auto-formatting, auto-anything is off, so for example typing a { adds a single { character and nothing else. Obviously there should be key combinations assignable to enable/disable/toggle raw mode quickly, and some kind of indication in the status bar.
VS Code version: Code 1.88.0 (5c3e652f63e798a5ac2f31ffd0d863669328dc4c, 2024-04-03T13:26:18.741Z)
OS version: Windows_NT x64 10.0.19045
Modes:

We try to keep VS Code lean and we think the functionality you're asking for is great for a VS Code extension. Maybe you can already find one that suits you in the VS Code Marketplace. Just in case, in a few simple steps you can get started writing your own extension. See also our issue reporting guidelines.
Happy Coding!


Fix string comparisons

Similar to @jcbf 's fix in #107 fix also following ones (or at least the latter of them):
$ git grep -E '\$[^"]+ ='
include/despf.inc.sh:  test $# = 2 && echo $2 | grep -wq $1
tests/test-unit.sh:  test $1 = "-n" && { shift; > $tmp-out; } || cat > $tmp-out

$# is a number, should use -eq for comparison.
$1 can be quoted and x added as a placeholder.


Add FrontPress package to jsDeliver

Just like cdnjs, jsDeliver is a cdn for hosting front-end projects.
In order to add FrontPress to jsDeliver is necessary to follow the jsDeliver contributing guidelines.
The initial included version would be 1.0.7, but the package needs to be auto updated.

Submitted to jsDeliver and waiting for approval

It's now hosted by jsDelivr https://www.jsdelivr.com/projects/frontpress


kernel panic

Summary:
The first time I run Kitematic, it crashes my Mac.
Trace:
https://gist.github.com/mcandre/7ef03027fa10ada902db
System:

Kitematic 0.8.7
DockerToolbox 1.8.3
brew cask 0.57.0
brew 0.9.5
Mac OS X El Capitan 10.11
15" mid-2012 MacBook Pro

Looks like Virtualbox didn't play nice with your airport card drivers. Brew shouldn't be in there as Docker Toolbox does all of the work of installing the tools for you.
Can you confirm your Virtualbox version? As of 10/02/2015 Virtualbox release 5.0.6 which has a kernel panic fix added: https://forums.virtualbox.org/viewtopic.php?f=1&t=73652

Lol, I'm using VirtualBox 5.0.6!
$ brew cask info virtualbox
virtualbox: 5.0.6-103037
VirtualBox
https://www.virtualbox.org
/opt/homebrew-cask/Caskroom/virtualbox/5.0.6-103037 (4 files, 87M)
https://github.com/caskroom/homebrew-cask/blob/master/Casks/virtualbox.rb
==> Contents
  /Applications/VirtualBox.app/Contents/MacOS/VBoxManage (binary)
  /Applications/VirtualBox.app/Contents/MacOS/VBoxHeadless (binary)
  VirtualBox.pkg (pkg)

@mcandre If you're up for it - you can try their test builds, see if it solves your problem?
https://www.virtualbox.org/wiki/Testbuilds

Closing due to no further details provided


⚠ SolarOscp has degraded performance

In 388a53c, SolarOscp ($SOLAROSCP_URL) experienced degraded performance:

HTTP code: 200
Response time: 316 ms

Resolved: SolarOscp performance has improved in 7579d0e after 5 minutes.


Display of the bit accounting for motion vectors is broken in the analyzer.

Since this commit http://git.xiph.org/?p=daala.git;a=commit;h=043c91e1 the bit accounting of the motion vectors in the analyzer is broken.

Fixed by https://github.com/xiph/daala/commit/95f8af549b9f5c343db84e363cff6b34a637b515


added correct expectation for exKxz. all tests pass

Hi everybody,
This is a pull request for computing the expectation <x_t k(x_t, Z)>_q(X). Currently exKxz computes <x_{t-1}k(x_t, Z)>_q(X) which is basically the pair-wise expectation of two consecutive time steps. I believe this is useful in some dynamic systems that need to compute the expectation between two time steps. However the name of the function is a bit misleading and also the functionality of it is not in alignment with the general expectation terms.
This pull request handles both cases. The old function has been renamed to exKxz_pairwise and now the exKxz computes the <x_t k(x_t, Z)>_q(X). I could have opted for the reverse naming, e.g., exKxz and exKxz_non_pairwise but that would be misleading. Tests are modified in order to pass with both cases now.
Currently there seems to be a repetition of code but I tried to be the least intrusive possible.
Let me know what do you think.
Cheers

Codecov Report

Merging #513 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master    #513      +/-   ##
=========================================
+ Coverage   99.79%   99.8%   +<.01%
=========================================
  Files          28      28
  Lines        2498    2528      +30
=========================================
+ Hits         2493    2523      +30
  Misses          5       5

Impacted Files
Coverage Δ

gpflow/ekernels.py
100% <100%> (ø)
:arrow_up:

gpflow/kernels.py
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 71a57d9...00857db. Read the comment docs.

Hi @stefanosele, this looks great. Could you elaborate a bit on the exact use-case for these extra kernel expectations?
Otherwise I agree with the PR. The code repetition doesn't seem too severe, and it's hard to see how general this is exactly. I think some of the kernel expectation code can be improved in terms of both clarity and speed (@ialong has opinions about this, I believe).

Hi @markvdw,
A clear use-case is when you want to do moment matching for propagating uncertainty towards future samples from the following dynamic system:
f_t = x_{t-1} + f(x_{t-1})
For computing Var[f_t] you will need to compute at some point the term <x_{t-1} f(x_{t-1})>_q(X) which would require computing
<x_{t-1} k(x_{t-1}, z)>_q(X).
Does this make sense to you?

Great, I think that's an important use case. Again, I do think that some of this code can be cleaned, sped up and compactified, but that's a bigger job.

@stefanosele @markvdw hey guys,
the PR looks good, though I feel it is important to point out that the old exKxz can be used to calculate the new one (at the same computational cost - which as you noted, Mark, can be improved upon, I'll get around to it at some point).
Here's how.
The old exKxz essentially has two extra parameters for each index t-1:
\mu_t and \Sigma_{t, t-1}. The second one is passed in the second index of Xcov's first dimension.
Simply set:
\mu_t                  <------- \mu_{t-1}
\Sigma_{t, t-1}   <------- \Sigma_{t-1}
Before passing them to exKxz, and you should get the same values as if you used the new exKxz.
Intuitively this is the same as saying that q(x_t, x_{t-1}) is a degenerate Gaussian where x_t is really just x_{t-1} (and thus their cross-covariance is just the covariance of x_{t-1}).
All that would be needed would be to have separate arguments for \mu_{t-1} and \mu_t (and \mu_t would default to \mu_{t-1} if not passed, so no code would have to change). I.e. you would have:
def exKxz(self, Z, Xmu, Xcov, Xmu_next=None):

This would also be super useful when stitching together multiple (variable length) trials as it could handle boundaries between trials, rather than simply looking at the next entry in Xmu.
So in general I'm in favour of having just one, slightly more general, method, and I'd be happy to change the code to do it if you agree.

Hi @ialong,
I agree with your post, this is how I did it in the beginning and I guarantee that it gives same results.
The only annoying thing is that you need to duplicate Xcov for making it of rank 4 before calling the exKxz.
Another option (if we want to stick with the existing arguments) we can check the rank of the Xcov and if 4 proceed with the pairwise case, otherwise with the normal one.

@stefanosele yes I like this solution, it saves a lot of code reuse since the logic for exKxz and exKxz_pairwise is very similar.
The extra argument for multiple trials belongs to another discussion/PR I guess.

For future reference here's how one could merge the two functionalities into one method (don't think it's worth changing stuff yet as I need to refactor the kernexps anyway so I'll just do one PR for everything).
Just add these lines:
is_cross_exKxz = tf.rank(Xcov) > 3
Xmu = tf.cond(is_cross_exKxz, Xmu[:-1], Xmu)
Xmu_next = tf.cond(is_cross_exKxz, Xmu[1:], Xmu)
Xcov = tf.cond(is_cross_exKxz, Xcov, tf.tile(Xcov[None, ...], [2, 1, 1, 1]))

and replace occurrences of Xmu[:-1] with Xmu and of Xmu[1:] with Xmu_next in the old code.
@stefanosele @markvdw does that seem reasonable or would you prefer two separate methods?

Interesting, I think this looks elegant. Is there any downside to passing a rank 3 tensor? Like memory usage?

@ialong This is more or less the solution I had in mind but the other way around.
use the condition to check whether you need to compute the additional tensors that you require for the pairwise case and modify appropriately the call to mvnquad.
That way you don't need to pass the rank 3 tensors around in the case you want to compute the simpler expectation as @markvdw is worried about.

@stefanosele do you mean rank 4? Xcov is at least TxDxD and potentially 2xTxDxD.
@stefanosele @markvdw you guys are probably right that it's worth worrying about the extra memory overhead involved with an unnecessary rank 4 tensor. There's an easy fix for that of course (just use tf.cond where Xcov is used rather than to define Xcov). My post was about clarifying the logic for the RBF exKxz, I would need to check the mvnquad-based exKxz's if there can be an equally parsimonious refactoring.

@ialong Yes, I meant rank 4.


WIP: Update initial theme.json variables refactor

Description of the Change
Refactoring theme.json to make more agnostic - passing variables into other settings
I have the base styles connected to theme.json in via the plugin for

default
outline
text
colors for default (fill), outline - core - need colors for text and for all partials output

We still need:

show different button colors (partial and core button) - see .tui-content-container or teamwork for what is still needed
core styles moved to plugin
need to move custom button block to plugin (in teamwork)

Cc: @tlovett1 @devinle

@devinle
All current work has been pushed.
I have the base styles connected to theme.json in via the plugin for

default
outline
text
add conditional of theme enabled to add the custom button style
remove / provide direction for the custom file inputs targeting plugin files (in Teamwork tasks)

We still need:

show different button colors (partial and core button)
custom button block

I think this is a good spot to review / merge into the plugin branch before it gets too big.
CC: @fabiankaegy

@devinle
@tlovett1
This is now ready for review - works with this PR from the plugin: https://github.com/10up/wp-scaffold-ui/pull/14


only mark scenario as run once it finished running successfully [MRXN23-280]

Only mark a scenario as ranAtLeastOnce: true after it has run fully and successfully.
Testing instructions
Create new scenario, set it to run with a high enough number of iteration to give some time to check if the Scenario.ranAtLeastOnce flag is flipped from the initial default (false) to true while the Marxan solver is running.
Without this PR, the property should be flipped almost instantly after pressing the Run button (or triggering a marxan run via the API directly). With this PR, the property should only be flipped to true only after Marxan has run fully and successfully (so, it should stay false if the Marxan run is interrupted in flight or fails for whatever reason.
Feature relevant tickets
https://vizzuality.atlassian.net/browse/MRXN23-280

Checklist before submitting

[ ] Meaningful commits and code rebased on develop.
[ ] If this PR adds feature that should be tested for regressions when
deploying to staging/production, please add brief testing instructions
to the deploy checklist (docs/deployment-checklist.md)
[ ] Update CHANGELOG file

@yulia-bel don't forget a yarn format once done with everything here 😉

@yulia-bel thanks! let's merge 🚀


[fix][client] Fix race condition that leads to caching failed CompletableFutures in ConnectionPool

Motivation
There is a race condition while creating a Connection and putting it into the cache.
The operation may fail before that the CompletableFuture is put into the pool map and so cleanupConnection fails to remove the new connection from the map (because it is not there yet)
This is very likely to happen on k8s while a broker is not "ready"
The problem is that UnknownHostException fails very quickly (probably on the same thread), and the "exceptionally" block is executed before we put the CompletableFuture into the pool map, and so the "remove" operation fails
Modifications
Eagerly remove the CompletableFuture from the cache if it is "failed".
There is no need to retry, the client eventually will retry, and also retrying immediately will probably fail.
Verifying this change
I am not adding tests, this race condition is hard to reproduce. But it happens sometimes on some k8s clusters during fault injection tests.
Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

[ ] Dependencies (add or upgrade a dependency)
[ ] The public API
[ ] The schema
[ ] The default values of configurations
[ ] The threading model
[ ] The binary protocol
[ ] The REST endpoints
[ ] The admin CLI options
[ ] The metrics
[ ] Anything that affects deployment

Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

Matching PR in forked repository
PR in forked repository: https://github.com/eolivelli/pulsar/pull/21

/pulsarbot rerun-failure-checks

/pulsarbot rerun-failure-checks

As discussed on the mailing list https://lists.apache.org/thread/w4jzk27qhtosgsz7l9bmhf1t7o9mxjhp, there is no plan to release 2.9.6, so I am going to remove the release/2.9.6 label


keystoken / bid.glass

Some new ads on 4chan are pointing to keystoken.io with images served by bid.glass
for example https://bid.glass/creatives/2/m/c/c/2mccsd8rf2mViC.gif
I don't know which domain should be blocked, if not both, so just wanted to submit an issue for consideration.

OK, turns out bid.glass is entirely an advertisement serving platform, so I think it's the one we want.

so I think it's the one we want

It's already listed in shady-hosts:

Yep, my cache was too old. Things move quick! Thanks.


Change ignored

Hey all,
I'm having some weird issue, when I change the variable in .env file from one value to another, it doesn't get updated. When I add a new variable, it gets recongized. Is there any caching or what?
I'm having one main module which has a default .env file, and submodules, having their own .env which can override variables from the main module. Here's a code sample:
<?php

use ActiveRecord\Config;

$dotenv = \Dotenv\Dotenv::createMutable(__DIR__, null, false);
$dotenv->load();

Are you using getenv to read the variable?

Yes, exactly

Then you either need to use unsafe mode or change your code to read from $_SERVER, instead. getenv is not suitable for use on php-fpm, because it is not thread safe, so may leak variables from other apps on the same server, or return null.

See https://github.com/vlucas/phpdotenv#putenv-and-getenv.


IBMMobileFirstPlatformFoundationHybrid.framework is excluded, and MFP 7.1 doesn't recreate it

@nasht00 This ignore rule was added by you.  Did you verify that the framework is recreated by MFP if missing?

According to http://www-01.ibm.com/support/knowledgecenter/SSHS8R_7.1.0/com.ibm.worklight.dev.doc/devref/r_integrating_with_source_contro.html this file should not be ignored.


connection reset by peer

Installed bluealsa with the stretch repo and seems to work fine, I'm only using the a2dp side.
However after a file the BT connection between my speaker drops.  Is there any way to keep the connection alive.  I'm trusted and paired with bluetoothctl all fine.
If I'm constantly streaming the connection stays alive no problem
I've set the below to 0 using the /etc/bluetooth/main.conf file
# How long to stay in pairable mode before going back to non-discoverable
# The value is in seconds. Default is 0.
# 0 = disable timer, i.e. stay pairable forever
PairableTimeout = 0

which as I suspected wont do anything as the default is already 0
I've tried pulseaudio which doesnt suffer any such issues the sink just stays alive although change was needed in the pulseaudio config files.....
I have managed to compile the release on github using --debug and on connection the below error occurs
never the less it still is connected:
bluealsa: Unsupported AT message: TEST: command:+CHLD, value:(null)
bluealsa: Unsupported AT message: SET: command:+CMEE, value:1
bluealsa: Unsupported AT message: SET: command:+CLIP, value:1
bluealsa: Unsupported AT message: SET: command:+CCWA, value:1
bluealsa: Unsupported AT message: SET: command:+NREC, value:0

pi@raspberrypi:~ $ bt-device -i 42:15:BD:04:08:55
[42:15:BD:04:08:55]
  Name: LunaBox SoundBot
  Alias: LunaBox SoundBot [rw]
  Address: 42:15:BD:04:08:55
  Icon: audio-card
  Class: 0x240408
  Paired: 1
  Trusted: 1 [rw]
  Blocked: 0 [rw]
  Connected: 1
  UUIDs: [AudioSink, AVRemoteControlTarget, AdvancedAudioDistribution, AVRemoteControl, Handsfree]

Hi,
If you are using A2DP only, run bluealsa wih --disable-hsp and --disable-hfp. This will disable HFP connection negotiation, which might be the case here (my implementation is kind a naive, and when the other side does not respond "properly" connection is terminated).
It might be more informative if you manage to past logs from the disconnection event. Also please tell me, if you are connecting phone or a headset?

okay tried that here are the logs
This is a fresh install of stretch lite
Connection is of a normal Luna bluetooth speaker
like this one
https://www.amazon.co.uk/LunaBox-SoundBot-Bluetooth-Dual-Driver-Distortion-Rose-Gold/dp/B01N0NBXKM/ref=sr_1_1?ie=UTF8&qid=1509318301&sr=8-1&keywords=luna+bluetooth+speakers
pi@raspberrypi:~ $ ps -ef | grep blue
root       433     1  0 22:43 ?        00:00:00 /usr/lib/bluetooth/bluetoothd
root       565   529  0 22:47 pts/0    00:00:00 sudo bluealsa --disable-hsp --disable-hfp
root       569   565  0 22:47 pts/0    00:00:00 bluealsa --disable-hsp --disable-hfp
pi         628   529  0 22:53 pts/0    00:00:00 grep --color=auto blue

LunaBox SoundBot]# connect 42:15:BD:04:08:55
Attempting to connect to 42:15:BD:04:08:55
[LunaBox SoundBot]# bluealsa: ../../src/bluez.c:539: Endpoint method call: org.bluez.MediaEndpoint1.SelectConfiguration()
bluealsa: ../../src/bluez.c:539: Endpoint method call: org.bluez.MediaEndpoint1.SetConfiguration()
bluealsa: ../../src/bluez.c:471: A2DP Source (SBC) configured for device 42:15:BD:04:08:55
bluealsa: ../../src/bluez.c:473: Configuration: channels: 2, sampling: 48000
bluealsa: ../../src/transport.c:616: State transition: 0 -> 0
bluealsa: ../../src/ctl.c:571: +-+-
bluealsa: ../../src/bluez.c:593: Endpoint already registered: /A2DP/MPEG24/Source/1
bluealsa: ../../src/bluez.c:593: Endpoint already registered: /A2DP/MPEG24/Sink/1
bluealsa: ../../src/bluez.c:610: Registering endpoint: /A2DP/SBC/Source/2
bluealsa: ../../src/bluez.c:593: Endpoint already registered: /A2DP/SBC/Sink/1
Connection successful
[CHG] Device 14:BB:6E:30:61:D1 RSSI: -93
[CHG] Device 14:BB:6E:30:61:D1 RSSI: -55
[CHG] Device 14:BB:6E:30:61:D1 RSSI: -96
[CHG] Device 14:BB:6E:30:61:D1 RSSI: -58
[CHG] Device 14:BB:6E:30:61:D1 RSSI: -96
[CHG] Device 14:BB:6E:30:61:D1 RSSI: -55
[LunaBox SoundBot]# exit

The below turn up on the screen when the device disconnects:

pi@raspberrypi:~ $ bluealsa: ../../src/bluez.c:539: Endpoint method call: org.bluez.MediaEndpoint1.ClearConfiguration()
bluealsa: ../../src/transport.c:3

Managed some further tests - I have tried 2 other BT speakers utilising only a2dp and the same observations are noted.  The speaker disconnects after 3/4 mins of inactivity even if paired.  If audio is continuously played then the connection is kept alive.  The 3/4 mins of disconnection is not issued by bluealsa but by the peer.  It seems bluealsa perhaps doesn't keep the connection "alive" if not audio is being transmitted???

I've added this to
/etc/dbus-1/system.d/bluetooth.conf:

And since I've had no disconnects once paired....?
bluealsa is started as root but root should have access to bluez so why does this work?


Proposal: Allow the config file to be written in Typescript

It looks like the filename static.config.js is hardcoded into utils.js::getConfig. Are there any plans to allow the config file to be written in typescript similar to how you can write Webpack configs?
https://webpack.js.org/configuration/configuration-languages/#typescript

It uses babel register to allow es6 in the config. Not sure how this would
work though. Any ideas?
On Wed, Feb 7, 2018 at 6:04 PM Rachel Church<EMAIL_ADDRESS>wrote:

It looks like the filename static.config.js is hardcoded into
utils.js::getConfig. Are there any plans to allow the config file to be
written in typescript similar to how you can write Webpack configs?
https://webpack.js.org/configuration/configuration-languages/#typescript
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/nozzle/react-static/issues/358, or mute the thread
https://github.com/notifications/unsubscribe-auth/AFUmCY5UMKZJh55u_XNLzr31nZeMj_3Mks5tSkgngaJpZM4R9pGp
.

I have almost gotten it to work by editing utils.js to look for either static.config.js or static.config.ts. Using a sample project I added ts-node as a dependency made a static.config.ts and a tsconfig.json file. It builds just fine until I try and import other *.ts files into static.config.ts (such as a webpack config function).

It think this would mean the config file would need to be fully precompiled using a the standard typescript cli or the webpack typescript loader.

At this point, I think I've discovered the best way to do this would be to write your own miniscript that runs before react-static that compiles a static.config.ts file into a static.config.js file, then runs react-static.

As of babel 7, it now compiles typescript no problem. Might it make sense to reopen this?

Yeah. I'm not sure what it would take to support, but sounds viable. What would it take to make this happen?

It's quite simple, it just needs
{
    "presets": [
        "@babel/env",
        "@babel/preset-typescript"
    ],
    "plugins": [
        "@babel/proposal-class-properties",
        "@babel/proposal-object-rest-spread"
    ]
}

You then need to run babel with --extensions .ts,.tsx
That should do it.

This would essentially be running typescript for everyone by default right? Is it possible to just add that to your own .babelrc and call it good?

We do use babelRegister under the hood to do this though.

We do use babelRegister under the hood to do this though.

Yeah that's what I though. Just adding it for me would be simple enough. I am just afraid there are some internals that expect it to be js. For example, right now, if I just rename it to ts, it falls apart saying it can't find it. I know I could create build step that would first compile ts to js. But that's slightly barbaric. Especially when react-static config is not per say "build only". It touches actual app stuff. Props and what have you. That's the main reason why it seems like it would benefit from proper ts support. Also I must say, after checking out some of your presentations (which were very good) I totally got you framed like a typescript guy :) I was surprised rs is not entirely written in ts. It's so amazing for coding productivity, I can't even stand plain js anymore :p

Haha, well I'm flattered! I wish was a typescript guy, but sadly haven't invested the time to migrate my brain and projects yet. I'm well aware of the benefits, but haven't necessarily felt them first-hand yet. I suspect in the next year I will make the move, but for now, I need to be productive with the muscle memory I have... :(
I'm more than happy to merge any ideas that would make what you're attempting possible, as long as it doesn't change the experience or incur any overhead for non-TS users.

Haha, okay that makes sense then! Just consider my voice another one of those that will hopefully one day lure you to the typescript camp. Awesome doesn't even begin to describe it. I spit out 50% more code this way and save another 50% while debugging as most of the time, I find bugs while still in editor.
Roger that, I'll be happy to contribute. Especially the d.ts definitely needs some love. I can do that first.

That sounds like a great plan. Can't wait to be converted ;)

It was very easy for me, came from c++ / c# so I was praying for ts even before it existed. Made the switch right away then. Ok as per our discussion on the second issue, I'll be happy to contribute once v6 lands. Things I can do is:

Improve d.ts
Fix and improve ts example (the one we have right now is not rly showing the full potential imho)
Look into supporting config as ts file

I agree, the current TS example is not very fleshed out. Would love to upgrade that and get the typescript plugin for v6 working.

actually @sw-yx -- I was able to get this working with ts-node to parse my static.config.ts sucessfully.

yarn add ts-node
npx ts-node ./node_modules/react-static/bin/react-static {start|build|export|...} works if you have a static.config.ts.  No need to precompile to js anymore

I just added the variations of the above to my package.json scripts and everything seems to work now thanks to #978 .

sly dog! good to know.

actually @sw-yx -- I was able to get this working with ts-node to parse my static.config.ts sucessfully.

yarn add ts-node
npx ts-node ./node_modules/react-static/bin/react-static {start|build|export|...} works if you have a static.config.ts.  No need to precompile to js anymore

I just added the variations of the above to my package.json scripts and everything seems to work now thanks to #978 .

This way just got broken with the 7.0.6 release :(

This has been temporarily disabled. Will be added back shortly.