Many blackholed requests cause Envoy to consume excessive ram/cpu

Bug description
A program that makes occasional requests to services outside of the mesh combined with the cluster having outboundTrafficPolicy:REGISTRY_ONLY set will result in the Envoy process accumulating an unbounded number of stats entries (? stuff available on /stats) eventually causing Envoy to consume lots of ram and in some cases stops functioning all together.
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[X] Networking
[X] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Expected behavior
Ram/CPU stays bounded and Envoy remains functional
Steps to reproduce the bug
Important context: Istio has outboundTrafficPolicy:REGISTRY_ONLY set.
The Knative autoscaler acquired a feature which attempts to scrape statistics directly from individual pods. This direct scraping happens on autoscaler startup, and on occasion when some internal state gets reset. This results in periodic bursts of 1-10ish requests being attempted to individual pods to grab stats. After the autoscaler attempts to make these fetches, and finds out it isn't possible, it proceeds to scrape via the service (good!). As pods move around, and the autoscaler's internal state gets reset, it goes back to attempting to talk to the pods directly. This causes the number of /stats entries in Envoy to continuously grow because each attempt to scrape a pod results in an attempt to contact a unique IP. Eventually Envoy struggles to operate correctly, and seems to either lock up or becomes so slow as to be non-functional. With the Knative autoscaler it seems to take roughly a week to cause full breakdown of Envoy. The direct pod probes happen at a rate of about 10ish every 15 minutes (really slow burn). Lots of details in https://github.com/knative/serving/issues/8761 for the curious.
https://github.com/nairb774/experimental/blob/28f76fcc2db73ed30c46aff7ce4b25a47515d25c/http-prober/main.go is a simplified and accelerated reproduction of the behavior of the Knative autoscaler component. This can be deployed/run with ko to simulate a simplified Knative autoscaler behavior. Within a minute or two of running the http-prober, the /stats/prometheus page, on the Envoy dashboard (istioctl dashboard envoy), takes forever to return, and the response size is well over 100MiB (in the one test I ran) with about 140k rows. The memory usage of Envoy also ballooned to about 500MiB in that minute time.
Ideally the slow burn version of the Knative autoscaler wouldn't cause Envoy to topple over (though it is just as easy to say the autoscaler is as much at fault).
Version (include the output of istioctl version --remote and kubectl version and helm version if you used Helm)
$ istioctl version --remote
client version: 1.6.5
control plane version: 1.6.4
data plane version: 1.6.4 (57 proxies)
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.6-beta.0", GitCommit:"e7f962ba86f4ce7033828210ca3556393c377bcc", GitTreeState:"clean", BuildDate:"2020-01-15T08:26:26Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.8-eks-e16311", GitCommit:"e163110a04dcb2f39c3325af96d019b4925419eb", GitTreeState:"clean", BuildDate:"2020-03-27T22:37:12Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}

Knative v0.16.0 (for what it is worth)
How was Istio installed?
Istio Operator (version 1.6.4)
Environment where bug was observed (cloud vendor, OS, etc)
AWS EKS 1.16
AWS Managed Node Groups (1.16.8-20200618 / ami-0d960646974cf9e5b)
Here is an operator config for Istio: istio-system.yaml.gz which is from the cluster exhibiting the issue both via the Knative autoscaler as well as the minimal program above. I'm a little apprehensive dumping so much information from the cluster and making it publically accessible. I'm happy to pull specific logs/info that might be useful. I'm even happy to hop on a VC and do some live debugging/reproductions if that would help out.

I suspect this is because of metrics exposed by istio stats filter. Not very familiar with Knative set up. But does it rely on istio metrics? If not, can you add the following install options to disable stats emitted by Istio.
  values:
    telemetry:
      enabled: false
      v2:
        enabled: false

or you can simply delete stat-filter-1.6 envoyfilter: kubectl delete -n istio-system envoyfilter stats-filter-1.6.
If your system do rely on istio metrics, you can edit stats filter configuration to disable host header fallback:
kubectl edit -n istio-system envoyfilter stats-filter-1.6, and make the following change to all filter configuration block:
                 {
                   "debug": "false",
                   "stat_prefix": "istio",
+                  "disable_host_header_fallback": true
                 }

Interesting. I'll play with this tomorrow (time willing) and report back. Knative has its own metrics pipelines for its own operations, but I have a hunch (need to test) that DataDog's Istio integration makes use of the Ist io metrics. I'll turn it off and see what happens.

I got some time to try out the suggestions. Turning off telemetry is a little painful as it seems to break how Datadog is getting data. We are using their default integration, and I have not dug into any sort of configuration that might exist there.
Removing disable_host_header_fallback doesn't seem to help. Assuming I'm looking at the right code (https://github.com/istio/proxy/blob/1.6.4/extensions/stats/plugin.cc#L157 and https://github.com/istio/proxy/blob/1.6.4/extensions/common/context.cc#L106-L110), I would assume it to work. I restarted the istiod pods and then restarted the problematic Knative pod. I've attached all of the envoyfilter objects in the istio-system namespace. As a last ditch effort, I even edited the 1.4 and 1.5 objects as well, with no success. Stats lines like the following are still showing up for every errant attempt of the Knative autoscaler to try to reach individual pods:
reporter=.=source;.;source_workload=.=autoscaler;.;source_workload_namespace=.=knative-serving;.;source_principal=.=unknown;.;source_app=.=autoscaler;.;source_version=.=unknown;.;source_canonical_service=.=autoscaler;.;source_canonical_revision=.=latest;.;destination_workload=.=unknown;.;destination_workload_namespace=.=unknown;.;destination_principal=.=unknown;.;destination_app=.=unknown;.;destination_version=.=unknown;.;destination_service=.=<IP_ADDRESS>:9090;.;destination_service_name=.=BlackHoleCluster;.;destination_service_namespace=.=unknown;.;destination_canonical_service=.=unknown;.;destination_canonical_revision=.=latest;.;request_protocol=.=http;.;response_code=.=502;.;grpc_response_status=.=;.;response_flags=.=-;.;connection_security_policy=.=unknown;.;_istio_requests_total: 1
reporter=.=source;.;source_workload=.=autoscaler;.;source_workload_namespace=.=knative-serving;.;source_principal=.=unknown;.;source_app=.=autoscaler;.;source_version=.=unknown;.;source_canonical_service=.=autoscaler;.;source_canonical_revision=.=latest;.;destination_workload=.=unknown;.;destination_workload_namespace=.=unknown;.;destination_principal=.=unknown;.;destination_app=.=unknown;.;destination_version=.=unknown;.;destination_service=.=<IP_ADDRESS>:9090;.;destination_service_name=.=BlackHoleCluster;.;destination_service_namespace=.=unknown;.;destination_canonical_service=.=unknown;.;destination_canonical_revision=.=latest;.;request_protocol=.=http;.;response_code=.=502;.;grpc_response_status=.=;.;response_flags=.=-;.;connection_security_policy=.=unknown;.;_istio_requests_total: 1
reporter=.=source;.;source_workload=.=autoscaler;.;source_workload_namespace=.=knative-serving;.;source_principal=.=unknown;.;source_app=.=autoscaler;.;source_version=.=unknown;.;source_canonical_service=.=autoscaler;.;source_canonical_revision=.=latest;.;destination_workload=.=unknown;.;destination_workload_namespace=.=unknown;.;destination_principal=.=unknown;.;destination_app=.=unknown;.;destination_version=.=unknown;.;destination_service=.=<IP_ADDRESS>:9090;.;destination_service_name=.=BlackHoleCluster;.;destination_service_namespace=.=unknown;.;destination_canonical_service=.=unknown;.;destination_canonical_revision=.=latest;.;request_protocol=.=http;.;response_code=.=502;.;grpc_response_status=.=;.;response_flags=.=-;.;connection_security_policy=.=unknown;.;_istio_requests_total: 1
...
reporter=.=source;.;source_workload=.=autoscaler;.;source_workload_namespace=.=knative-serving;.;source_principal=.=unknown;.;source_app=.=autoscaler;.;source_version=.=unknown;.;source_canonical_service=.=autoscaler;.;source_canonical_revision=.=latest;.;destination_workload=.=unknown;.;destination_workload_namespace=.=unknown;.;destination_principal=.=unknown;.;destination_app=.=unknown;.;destination_version=.=unknown;.;destination_service=.=<IP_ADDRESS>:9090;.;destination_service_name=.=BlackHoleCluster;.;destination_service_namespace=.=unknown;.;destination_canonical_service=.=unknown;.;destination_canonical_revision=.=latest;.;request_protocol=.=http;.;response_code=.=502;.;grpc_response_status=.=;.;response_flags=.=-;.;connection_security_policy=.=unknown;.;_istio_request_bytes: P0(nan,1200.0) P25(nan,1225.0) P50(nan,1250.0) P75(nan,1275.0) P90(nan,1290.0) P95(nan,1295.0) P99(nan,1299.0) P99.5(nan,1299.5) P99.9(nan,1299.9) P100(nan,1300.0)
reporter=.=source;.;source_workload=.=autoscaler;.;source_workload_namespace=.=knative-serving;.;source_principal=.=unknown;.;source_app=.=autoscaler;.;source_version=.=unknown;.;source_canonical_service=.=autoscaler;.;source_canonical_revision=.=latest;.;destination_workload=.=unknown;.;destination_workload_namespace=.=unknown;.;destination_principal=.=unknown;.;destination_app=.=unknown;.;destination_version=.=unknown;.;destination_service=.=<IP_ADDRESS>:9090;.;destination_service_name=.=BlackHoleCluster;.;destination_service_namespace=.=unknown;.;destination_canonical_service=.=unknown;.;destination_canonical_revision=.=latest;.;request_protocol=.=http;.;response_code=.=502;.;grpc_response_status=.=;.;response_flags=.=-;.;connection_security_policy=.=unknown;.;_istio_request_duration_milliseconds: P0(nan,0.0) P25(nan,0.0) P50(nan,0.0) P75(nan,0.0) P90(nan,0.0) P95(nan,0.0) P99(nan,0.0) P99.5(nan,0.0) P99.9(nan,0.0) P100(nan,0.0)
reporter=.=source;.;source_workload=.=autoscaler;.;source_workload_namespace=.=knative-serving;.;source_principal=.=unknown;.;source_app=.=autoscaler;.;source_version=.=unknown;.;source_canonical_service=.=autoscaler;.;source_canonical_revision=.=latest;.;destination_workload=.=unknown;.;destination_workload_namespace=.=unknown;.;destination_principal=.=unknown;.;destination_app=.=unknown;.;destination_version=.=unknown;.;destination_service=.=<IP_ADDRESS>:9090;.;destination_service_name=.=BlackHoleCluster;.;destination_service_namespace=.=unknown;.;destination_canonical_service=.=unknown;.;destination_canonical_revision=.=latest;.;request_protocol=.=http;.;response_code=.=502;.;grpc_response_status=.=;.;response_flags=.=-;.;connection_security_policy=.=unknown;.;_istio_response_bytes: P0(nan,54.0) P25(nan,54.25) P50(nan,54.5) P75(nan,54.75) P90(nan,54.9) P95(nan,54.95) P99(nan,54.99) P99.5(nan,54.995) P99.9(nan,54.999) P100(nan,55.0)
...

Any thoughts on next steps? In the meantime, I should update the cluster to 1.6.7 (from 1.6.4).

No you should not remove it, You should add that line "disable_host_header_fallback": true to all filter configs.

disable_host_header_fallback was already set to true (state prior to edits) which is likely what got me turned around. Your initial diff was clear, I mentally broke when trying to apply.

Yeah, but did you add it to other sections? https://github.com/istio/istio/blob/1.6.7/manifests/charts/istio-control/istio-discovery/templates/telemetryv2_1.6.yaml#L194 and https://github.com/istio/istio/blob/1.6.7/manifests/charts/istio-control/istio-discovery/templates/telemetryv2_1.6.yaml#L146 I was meant to add this option to those two blocks.

Apparently I need to slow down and read, because once I did that things look like they will be in a better state.
reporter=.=source;.;source_workload=.=autoscaler;.;source_workload_namespace=.=knative-serving;.;source_principal=.=unknown;.;source_app=.=autoscaler;.;source_version=.=unknown;.;source_canonical_service=.=autoscaler;.;source_canonical_revision=.=latest;.;destination_workload=.=unknown;.;destination_workload_namespace=.=unknown;.;destination_principal=.=unknown;.;destination_app=.=unknown;.;destination_version=.=unknown;.;destination_service=.=unknown;.;destination_service_name=.=BlackHoleCluster;.;destination_service_namespace=.=unknown;.;destination_canonical_service=.=unknown;.;destination_canonical_revision=.=latest;.;request_protocol=.=http;.;response_code=.=502;.;grpc_response_status=.=;.;response_flags=.=-;.;connection_security_policy=.=unknown;.;_istio_requests_total: 17
reporter=.=source;.;source_workload=.=autoscaler;.;source_workload_namespace=.=knative-serving;.;source_principal=.=unknown;.;source_app=.=autoscaler;.;source_version=.=unknown;.;source_canonical_service=.=autoscaler;.;source_canonical_revision=.=latest;.;destination_workload=.=unknown;.;destination_workload_namespace=.=unknown;.;destination_principal=.=unknown;.;destination_app=.=unknown;.;destination_version=.=unknown;.;destination_service=.=unknown;.;destination_service_name=.=BlackHoleCluster;.;destination_service_namespace=.=unknown;.;destination_canonical_service=.=unknown;.;destination_canonical_revision=.=latest;.;request_protocol=.=http;.;response_code=.=502;.;grpc_response_status=.=;.;response_flags=.=-;.;connection_security_policy=.=unknown;.;_istio_request_bytes: P0(nan,1200.0) P25(nan,1225.0) P50(nan,1250.0) P75(nan,1275.0) P90(nan,1290.0) P95(nan,1295.0) P99(nan,1299.0) P99.5(nan,1299.5) P99.9(nan,1299.9) P100(nan,1300.0)
reporter=.=source;.;source_workload=.=autoscaler;.;source_workload_namespace=.=knative-serving;.;source_principal=.=unknown;.;source_app=.=autoscaler;.;source_version=.=unknown;.;source_canonical_service=.=autoscaler;.;source_canonical_revision=.=latest;.;destination_workload=.=unknown;.;destination_workload_namespace=.=unknown;.;destination_principal=.=unknown;.;destination_app=.=unknown;.;destination_version=.=unknown;.;destination_service=.=unknown;.;destination_service_name=.=BlackHoleCluster;.;destination_service_namespace=.=unknown;.;destination_canonical_service=.=unknown;.;destination_canonical_revision=.=latest;.;request_protocol=.=http;.;response_code=.=502;.;grpc_response_status=.=;.;response_flags=.=-;.;connection_security_policy=.=unknown;.;_istio_request_duration_milliseconds: P0(nan,0.0) P25(nan,0.0) P50(nan,0.0) P75(nan,11.2917) P90(nan,11.7167) P95(nan,11.8583) P99(nan,11.9717) P99.5(nan,11.9858) P99.9(nan,11.9972) P100(nan,12.0)
reporter=.=source;.;source_workload=.=autoscaler;.;source_workload_namespace=.=knative-serving;.;source_principal=.=unknown;.;source_app=.=autoscaler;.;source_version=.=unknown;.;source_canonical_service=.=autoscaler;.;source_canonical_revision=.=latest;.;destination_workload=.=unknown;.;destination_workload_namespace=.=unknown;.;destination_principal=.=unknown;.;destination_app=.=unknown;.;destination_version=.=unknown;.;destination_service=.=unknown;.;destination_service_name=.=BlackHoleCluster;.;destination_service_namespace=.=unknown;.;destination_canonical_service=.=unknown;.;destination_canonical_revision=.=latest;.;request_protocol=.=http;.;response_code=.=502;.;grpc_response_status=.=;.;response_flags=.=-;.;connection_security_policy=.=unknown;.;_istio_response_bytes: P0(nan,54.0) P25(nan,54.25) P50(nan,54.5) P75(nan,54.75) P90(nan,54.9) P95(nan,54.95) P99(nan,54.99) P99.5(nan,54.995) P99.9(nan,54.999) P100(nan,55.0)

Now I need to find a way to prevent the Istio Operator from overwriting the values. I can see a few options, but I think this looks to be a much better state.

I was meant to add this option to those two blocks.

Is this something you think might be changing in a future release?
Thank you immensely for your patience in the face of my apparently lacking comprehension skills.

So we did not provide a way to override stats filter configuration in the hope that we will have telemetry API by 1.7 but looks like that is not going to happen in short term.
@douglas-reid @mandarjog @kyessenov wdyt about adding config override into stats filter installation option? Without this, I think it would be hard for user to manage any patch on stats filter like this issue or any metric customization.

Although the configuration will only control root namespace envoyfilters, so user will still need to maintain per namespace envoyfilter, but at least it won't be overridden when upgrading Istio.

yeah, I would support pulling out filter configs out of envoy filter resource. wdyt @mandarjog ?

Adding config override into install seems like a decent start. FWIW, our goals for 1.7 were to design a new telemetry API, but we had a separate task to improve metrics customization.

https://github.com/istio/istio.io/pull/7952 added guide on how to use istioctl to customization metrics. It is in preliminary doc now: https://preliminary.istio.io/latest/docs/tasks/observability/metrics/customize-metrics/


Fix Fiat deposits

This is mostly a fix to add fiat deposits no matter what they are (native or non-native), which is something I overlooked when I first coded it.
I still need to add documentation to Bitbank supplemental plugin for this.
There is also a minor fix for filtering out zero'd entries from Pionex.
I need to compute, file taxes, and work out any bugs involved with that for now. I'll try to double back and refactor the RP2 JP report code once I'm finished.

I'm having issue getting the console_scripts to work.  I added dali_jp to setup.cfg, but it doesn't recognize the command.

Can you share the exact error you're getting? The procedure should be:

add a new jp country plugin in DaLI (see dali/plugin/country/us.py for an example)
add a new dali_jp line in the console_scripts section of setup.cfg

When you install DaLI it installs a local copy of rp2: perhaps that copy is stale? You can try to make clean and then reinstall DaLI.

That (reinstall dali) did it. Thanks! I'm still rusty with general programming building/installing.

Is it all right if we just add a check at line #268 after the two transactions are assigned to see if both transactions have the same exchange? Like this:
if transaction1.exchange == transaction2.exchange:
    continue

Is there a problem with that? Transfers will always be between two different exchanges, right?

No, the exchange field could be the same. E.g. two people filing together could have two separate accounts on binance and move coins between the two accounts.

Ok, how about at line#595, we check holders:
if from_exchange == to_exchange and from_holder == to_holder:
    return None

Of course, this means you have to change the return signature and check for None. It's a little sloppy I guess.

Ok, how about at line#595, we check holders:
if from_exchange == to_exchange and from_holder == to_holder:
    return None

Of course, this means you have to change the return signature and check for None. It's a little sloppy I guess.

If there is no major advantage to doing so, I would leave the logic unchanged (it seems a corner case to me).

This seems good to me.

Thanks for pinging me on this one: will do a final review in the next few days.


output folder created without the current date

For me it created an output folder without the current date: DerivedCrossParticipants/product_run.
This folder is empty.
Also, the run has an error message:

You have to install graphviz on windows.
Qu
2019年4月19日 -0400 PM2:25 Aditya Ponnada<EMAIL_ADDRESS>
For me it created an output folder without the current date: DerivedCrossParticipants/product_run.
This folder is empty.
Also, the run has an error message:
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

But that is only for the workflow pdf right?

Yes, maybe I should skip that error.
Qu
2019年4月19日 -0400 PM2:28 Aditya Ponnada<EMAIL_ADDRESS>
But that is only for the workflow pdf right?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

the output folder is empty nevertheless. it is also created without the [CURRENT_DATE]

The [CURRENT_DATE] description is out of date. I have removed that.

great! When I run the script without the scheduler, the output folder is empty

I'll check that. What was the error message when you use the scheduler

This is the error:

BTW, I get the same error when trying with the sampling-rate=80 to compute features:

It seems the syntax of your input argument has some problem

You need the --scheduler='processes'

And without quote. I'll update the README

(base) C:\Users\Dharam\MUSS>pipenv run python prepare_class_set.py ./muss_data/muss-data/ sampling-rate=80
prepare_class_set.py: Received extra arguments: sampling-rate=80
Usage: prepare_class_set.py [OPTIONS] input-folder

The same. Options should have -- before it.

the folder name duplicates because I ended up copying the compressed file. So it created a subfolder but that is not the problem.

there shouldn't be an extra muss_data in muss_data. It should be the folder structure of a mhealth dataset.

I copied the muss_data compressed folder. So the decompressed one had the muss_data as the sub folder. That is why I added ./muss_data/muss_data/

No, don't do that. Move that one level up.


I cannot make post requests

Hello Philipo,
I'm to use the League Route 5.
I have installed XAMPP with php7.4.
Whatever I do - all post requests are forbidden for me. I can handle the get requests.
As I have read in internet - it should be because of CSRF security measures. I suppose - I should add a CSRF token in my view template, but the program does not go to the controller. So, the view is not rendered at all.
I'm attaching some pictures of my code. I hope they will help you for faster review.

Please, help me with an advice.
Thank you
Hristo

It looks to me like you’re making a GET request to a POST route
Sent from ProtonMail for iOS
On Wed, Jul 21, 2021 at 06:45, Hristo Hristov @.***> wrote:

Hello Philipo,
I'm to use the League Route 5.
I have installed XAMPP with php7.4.
Whatever I do - all post requests are forbidden for me. I can handle the get requests.
As I have read in internet - it should be because of CSRF security measures. I suppose - I should add a CSRF token in my view template, but the program cannot go to the controller. So, the view is not rendered at all.
I'm attaching some pictures of my code. I hope they will help you for faster review.
LeagueRouteComposer json
LeagueRouteIndex php
LeagueRouteRouteFactory php
LeagueRouteRoutes php
LeagueRouteMethodNotAllowedException
Please, help me with an advice.
Thank you
Hristo
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Ok - How can I change the request then?
The post request is set in the html form which is not loaded.
What else can I do?

You need a GET route/request to display the form, then you need the form to submit a POST request to a POST route that will process the form

Very helpful answer, Filipo.
Thank you!

I'm going to close this as it's not an "issue" as such, more implementation advice.


Parameters name, address in GET /journeys

As the POI/address format requires latitude and longitude parameter values, could you kindly explain me the sense of supplying additionally parameter values for name and address. Thank you!
Code example:
vbb.journeys('900000003201', {
    type: 'location',
    name: 'Joe\'s place',
    address: 'Torfstraße 17',
    latitude: 52.5416823, longitude: 13.3491223
  })
.then(console.log)
.catch(console.error)

hafas-client (and therefore vbb-hafas, and therefore vbb-rest) returns data in the Friendly Public Transport Format 1.0.1.
The HAFAS endpoint being queried distinguishes stations, POIs and addresses. Therefore hafas-client needs to know what kind of "thing" you're passing in. The specific format for these can been in the journeys() docs.


Add Dockerfile

Make it easier to run the online ebook.

Refer to https://github.com/GitbookIO/gitbook/issues/527

Resolve by https://github.com/tobegit3hub/understand_linux_process/commit/bf631de840e506fbf91087ecb66641086c20e8f6 :smile:


Adding coding style checker like eslint?

Here is example.
// toLint.js
1  console.log("Hello World");
2    var hi = "HHIHI";
3
4  function ji(){
5    return 123;
6  }

By using eslint, the example set style to google
$ eslint index.js

/tmp/index.js
  1:13  error  Strings must use singlequote              quotes
  2:3   error  Unexpected var, use let or const instead  no-var
  2:7   error  'hi' is assigned a value but never used   no-unused-vars
  2:12  error  Strings must use singlequote              quotes
  4:1   error  Missing JSDoc comment                     require-jsdoc
  4:10  error  'ji' is defined but never used            no-unused-vars
  4:14  error  Missing space before opening brace        space-before-blocks

Q1: Do we need this?
Q2: What style should be used?

airbnb
google
standard

Any suggestion?

Maybe we can just use jscs like this:
https://github.com/cdnjs/cdnjs/commit/312a5abde77286bc211d0209196d6e8e872b4b09


Getting errors while using it with fastify-nextjs

server.js
const cacheableResponse = require("cacheable-response");
const next = require("next");
const fastify = require("fastify");

const port = parseInt(process.env.PORT, 10) || 3000;
const dev = process.env.NODE_ENV !== "production";
const app = next({ dev });

const handle = app.getRequestHandler();

const ssrCache = cacheableResponse({
  ttl: 1000 * 60 * 60, // 1hour
  get: async ({ req, res, pagePath, queryParams }) => ({
    data: await app.renderToHTML(req, res, pagePath, queryParams)
  }),
  send: ({ data, res }) => res.send(data)
});

app.prepare().then(() => {
  const server = fastify({ logger: { level: "error" } });

  server.get("/", (req, res) => ssrCache({ req, res, pagePath: "/" }));

  server.get("/blog/:id", (req, res) => {
    const queryParams = { id: req.params.id };
    const pagePath = "/blog";
    return ssrCache({ req, res, pagePath, queryParams });
  });

  server.get("*", (req, res) => handle(req, res));

  server.listen(port, err => {
    if (err) throw err;
    console.log(`> Ready on http://localhost:${port}`);
  });
});

Error Log
{"level":50,"time":1566390512702,"pid":31997,"hostname":"tss-H110M-H","reqId":1,"req":{"method":"GET","url":"/blog/first","hostname":"localhost:3000","remoteAddress":"<IP_ADDRESS>","remotePort":60968},"res":{"statusCode":500},"err":{"type":"NodeError","message":"The \"url\" argument must be of type string. Received type undefined","stack":"TypeError [ERR_INVALID_ARG_TYPE]: The \"url\" argument must be of type string. Received type undefined\n    at Url.parse (url.js:154:11)\n    at urlParse (url.js:148:13)\n    at Url.resolve (url.js:663:29)\n    at urlResolve (url.js:659:40)\n    at _getKey (/home/tss/ssr-caching-app/node_modules/cacheable-response/index.js:24:15)\n    at /home/tss/ssr-caching-app/node_modules/cacheable-response/index.js:87:17\n    at Object.server.get (/home/tss/ssr-caching-app/server.js:28:12)\n    at preHandlerCallback (/home/tss/ssr-caching-app/node_modules/fastify/lib/handleRequest.js:112:30)\n    at preValidationCallback (/home/tss/ssr-caching-app/node_modules/fastify/lib/handleRequest.js:101:5)\n    at handler (/home/tss/ssr-caching-app/node_modules/fastify/lib/handleRequest.js:70:5)"},"msg":"The \"url\" argument must be of type string. Received type undefined","v":1}

The "url" argument must be of type string. Received type undefined

Looks like at some point the url is not being passed as expected.

Closing because it's a cacheable-response bug.

The "url" argument must be of type string. Received type undefined

Looks like at some in your code point the url is not being passed as expected 🤔 .
Can you try to add some traces?
{"level":50,"time":1566713431011,"pid":13196,"hostname":"DESKTOP-B1VH7H0","reqId":1,"req":{"method":"GET","url":"/","hostname":"localhost:3000","remoteAddress":"<IP_ADDRESS>","remotePort":58397},"res":{"statusCode":500},"err":{"type":"NodeError","message":"The "url" argument must
be of type string. Received type undefined","stack":"TypeError [ERR_INVALID_ARG_TYPE]: The "url" argument must be of type string. Received type undefined\n    at Url.parse (url.js:154:11)\n    at urlParse (url.js:148:13)\n    at Url.resolve (url.js:663:29)\n    at urlResolve (url.js:659:40)\n    at _getKey (C:\Users\jibin\dev\ssr-caching-app\node_modules\cacheable-response\index.js:24:15)\n    at C:\Users\jibin\dev\ssr-caching-app\node_modules\cacheable-response\index.js:87:17\n    at Object.server.get (C:\Users\jibin\dev\ssr-caching-app\server.js:22:33)\n    at preHandlerCallback (C:\Users\jibin\dev\ssr-caching-app\node_modules\fastify\lib\handleRequest.js:112:30)\n    at preValidationCallback (C:\Users\jibin\dev\ssr-caching-app\node_modules\fastify\lib\handleRequest.js:101:5)\n    at handler (C:\Users\jibin\dev\ssr-caching-app\node_modules\fastify\lib\handleRequest.js:70:5)"},"msg":"The "url" argument must be of type
string. Received type undefined","v":1}
url.js:154
throw new ERR_INVALID_ARG_TYPE('url', 'string', url);
^

TypeError [ERR_INVALID_ARG_TYPE]: The "url" argument must be of type string. Received type undefined
at Url.parse (url.js:154:11)
at Object.urlParse [as parse] (url.js:148:13)
at Server.handleRequest (C:\Users\jibin\dev\ssr-caching-app\node_modules\next-server\dist\server\next-server.js:87:31)
at Object.server.get (C:\Users\jibin\dev\ssr-caching-app\server.js:30:33)
at preHandlerCallback (C:\Users\jibin\dev\ssr-caching-app\node_modules\fastify\lib\handleRequest.js:112:30)
at preValidationCallback (C:\Users\jibin\dev\ssr-caching-app\node_modules\fastify\lib\handleRequest.js:101:5)
at handler (C:\Users\jibin\dev\ssr-caching-app\node_modules\fastify\lib\handleRequest.js:70:5)
at handleRequest (C:\Users\jibin\dev\ssr-caching-app\node_modules\fastify\lib\handleRequest.js:19:5)
at onRunMiddlewares (C:\Users\jibin\dev\ssr-caching-app\node_modules\fastify\lib\middleware.js:22:5)
at middlewareCallback (C:\Users\jibin\dev\ssr-caching-app\node_modules\fastify\lib\route.js:358:5)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>start: NODE_ENV=production node server.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR!      C:\Users\jibin\AppData\Roaming\npm-cache_logs\2019-08-25T06_10_31_212Z-debug.log
Will this help?


App Passwords / Tokens for legacy WebDAV clients

After removing support for Basic Auth (see https://github.com/owncloud/ocis/issues/196), legacy WebDAV clients would need App Passwords / Tokens to connect, similar to the feature in oC10:
https://doc.owncloud.org/server/user_manual/personal_settings/security.html#app-passwords-tokens

see https://github.com/owncloud/ocis/issues/196#issuecomment-748197877

see https://github.com/owncloud/ocis/issues/196#issuecomment-748197877

see #196 (comment)

I don't get it? Basic auth will be disabled in prod environment? Can't use 3rd party WebDAV clients with with prod oCIS?

see #196 (comment)

I don't get it? Basic auth will be disabled in prod environment? Can't use 3rd party WebDAV clients with with prod oCIS?

@michaelstingl my bad, had a fast finger on the trigger.
AFAIK basic auth is intended only for testing, so work on getting basic auth production-ready needs to happen to also account for app passwords / tokens.

@michaelstingl my bad, had a fast finger on the trigger.
AFAIK basic auth is intended only for testing, so work on getting basic auth production-ready needs to happen to also account for app passwords / tokens.

To clarify (similar to oC 10 implementation), real account credentials should not work with basic auth, but users can create additional credentials that allow basic auth with 3rd party WebDAV clients. Perfect would be, if those additional credentials can only be used for file operations, and don't grant access to user or admin settings.

To clarify (similar to oC 10 implementation), real account credentials should not work with basic auth, but users can create additional credentials that allow basic auth with 3rd party WebDAV clients. Perfect would be, if those additional credentials can only be used for file operations, and don't grant access to user or admin settings.

thanks for the input @michaelstingl ! I'm unfamiliar with the feature, but it looks like this needs a concept for an implementation in OCIS. Will bring it up when I finish this grooming round with the team.

thanks for the input @michaelstingl ! I'm unfamiliar with the feature, but it looks like this needs a concept for an implementation in OCIS. Will bring it up when I finish this grooming round with the team.

closed accidentally?

yes, closed accindentally?

Rclone supports OIDC -> see our docs here: https://owncloud.dev/clients/rclone/webdav-sync-oidc/

Closing this as now handled in #7711


Accessing output variable more than once renders empty string

I'm trying to access an output variable from one stage in multiple consecutive stages.
The first time I access the variable I can access the value, but the next time I try it's an empty string.
In the example below B1 prints 'production' but C1 prints ''
trigger:
  - yaml-testing

pool:
  vmImage: windows-2019

stages:
- stage: A
  jobs:
  - job: A1
    steps:
      - task:  InlinePowershell@1
        name: setvar
        inputs:
          Script: |
            Write-Host "##vso[task.setvariable variable=Environment;isOutput=true]production"

- stage: B
  dependsOn: A
  variables:
    Environment: $[ stageDependencies.A.A1.outputs['setvar.Environment'] ]
  jobs:
  - job: B1
    steps:
      - task:  InlinePowershell@1
        name: printvar1
        inputs:
          Script: |
            Write-Host 'B1'
            Write-Host $(Environment)

- stage: C
  dependsOn: B
  variables:
    Environment: $[ stageDependencies.A.A1.outputs['setvar.Environment'] ]
  jobs:
  - job: C1
    steps:
      - task:  InlinePowershell@1
        name: printvar2
        inputs:
          Script: |
            Write-Host 'C1'
            Write-Host $(Environment)

Logs look like this:
// Stage: B
Job preparation parameters
Variables:
  Environment:
    Parsing expression: <stageDependencies.A.A1.outputs['setvar.Environment']>
    Evaluating: stageDependencies['A']['A1']['outputs']['setvar.Environment']
    Result: 'production'

// Stage: C
Job preparation parameters
Variables:
  Environment:
    Parsing expression: <stageDependencies.A.A1.outputs['setvar.Environment']>
    Evaluating: stageDependencies['A']['A1']['outputs']['setvar.Environment']
    Expanded: Null
    Result: ''

I just stumbled over the same issue. It looks like stageDependencies only contains variables from direct dependencies. Your stage C depends on B, but not on A directly. Try adding A as additional direct dependency:
- stage: C
   dependsOn:
     - A
     - B
...

B is already dependent on A so this should not be necessary, but for accessing variables via stageDependencies it is.


fix import path

Codecov Report

Merging #44 into master will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master      #44   +/-   ##
=======================================
  Coverage   92.78%   92.78%
=======================================
  Files          15       15
  Lines         943      943
=======================================
  Hits          875      875
  Misses         46       46
  Partials       22       22

Impacted Files
Coverage Δ

role/processor.go
93.61% <ø> (ø)
:arrow_up:

role/option.go
100% <ø> (ø)
:arrow_up:

policy/daemon.go
87.5% <ø> (ø)
:arrow_up:

authorizerd.go
94.2% <ø> (ø)
:arrow_up:

policy/signed_policy.go
83.33% <ø> (ø)
:arrow_up:

policy/option.go
94.11% <100%> (ø)
:arrow_up:

option.go
100% <100%> (ø)
:arrow_up:

jwk/option.go
100% <100%> (ø)
:arrow_up:

pubkey/option.go
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4bf70b5...2538b1d. Read the comment docs.

duplicated


NSVue - Unknown custom element:

Hi!
I followed the documentation and it seems that docs are incomplete.
According to my personal experience with your other plugins, I have to do this in my main.js file:
import Vue from 'nativescript-vue'
import CanvasPlugin from '@nativescript/canvas/vue'

Vue.use(CanvasPlugin)

The fact is that there is no vue folder with version 0.9.9 installed with npm (ns plugin add @nativescript/canvas). So I ended up doing:
import Vue from 'nativescript-vue'

Vue.registerElement('Canvas', () => require('@nativescript/canvas').Canvas)

I wanted to make a PR to update the docs, but maybe you'll want to fix the missing vue folder before.

Thanks will do

Interesting because that directory exists so it seems to be a build issue

Exactly! The vue folder is in repo but not in the npm package. It's weird because there are other similar folders like react or svelte in the npm package 🤔

fixed in https://github.com/NativeScript/canvas/commit/9f842dfe4b7bd3881d9c0a35797a27ca2b77533f


ktlint update

https://github.com/pinterest/ktlint/releases/tag/0.48.2
hi, new version is out

Hi @sanyarnd, sorry for the delay, it was an awkward upgrade, version 1.16.0 (supporting ktlink 0.48.2) is now available.


Error on console parsing fsx file

Describe the bug
With a fresh install following this tutorial: https://docs.microsoft.com/en-us/dotnet/fsharp/get-started/get-started-vscode
I get tons of errors on console when opening an empty fsx file.
Trace error states: "Error opening binary file 'C:\Program Files\dotnet\shared/Microsoft.NETCore.App\3.1.1\Microsoft.DiaSymReader.Native.amd64.dll': C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.1.1\Microsoft.DiaSymReader.Native.amd64.dll: bad cli header, rva 0"
Use sdk scripts is checked
It works properly for regular fs files in a project.
Expected behaviour
Script files should work as fs files.
Environment (please complete the following information):

OS: Windows 10 professional build  18362
Ionide version: 4.5.0
VSCode version: 1.41.1
dotnet SDK version: 3.1.101

After updating to version 4.6 it is working properly


Can’t find global settings in property inspector .html

Thanks much for all the hard work.
I’m a Python guy with very little JavaScript experience trying to figure out the JavaScript side of writing a plug-in.  The lorumflickr example was a great help, but it doesn’t show any global settings usage.  I’m pretty sure I’m setting them properly on the Python side, the values show up in the events in the log. But, I can’t figure out how to access them from the JavaScript in property inspector .html.
Care to point me in the right direction?

It's funny how all it takes is asking for the answer to show itself...
I gave my head one more good scratching and here's what I came up with.
    let global_settings;

    $PI.onDidReceiveGlobalSettings(jsn => {
        console.log('Property Inspector global settings', jsn);

        global_settings = jsn.payload.settings;

        if (global_settings?.some_global) {
            some_global_el.value = global_settings.some_global;
        }
    });

Agree... putting the getGlobalSettings() call in onConnected() first was a big part of the picture.  Your example code really helped a lot too.
If you have an example that integrates with asyncio, I'd love to see that.  My plug-in needs an event loop of its own.  A minor modification to sdk's StreamDeck.run() has me going with rel for now.  I added a 'dispatcher' parameter to pass to WebSocket's run_forever() method.
I'm not complaining though... I have a working plug-in now that wouldnt exist if not for your efforts.

@dBeau Please open a new issue with a question about the event loop and provide an example code.  It will also be good if you describe the task in order to understand exactly how the event loop and the plugin will interact.  I will try to help you and possibly change the library code so that tasks like yours are easier to solve.

The question required more examples. They have been provided.


updates to support all GCOL and PLOT “modes”

supports all of Acorn's "basic" GCOL modes, 0-7 (0-4 from the Beeb, 5-7 from the Archimedes)
also support the "invert pixels" PLOT "mode", which operates the same as GCOL mode 4
WIP - in principle now works in all colour depths, but needs testing.  so far work only done for pure graphics drawing operations - no text support

tested behaviour of GCOL modes in a 64-colour mode seems to essentially match Acorn
exact pixel positioning on PLOTs are slightly difficult to judge whether they match properly, owing to differences in screen resolutions and coordinate rounding.  Agon's relative PLOT coordinate calculations are pre-rounded - this can cause mis-matches in positioning, with Agon losing pixels when relative move values get repeatedly rounded down.  if Agon tracked positions of graphics cursor points using OS coordinates this would resolve that issue, but making such a change may be tricky
Acorn systems also seem to track two GCOL modes - one for foreground, the other background, and picks the relevant mode according to the PLOT command (whether it's using foreground or background colour)

tests in all depths work - Acorn's 16 colour modes by default use flashing colours tho for 8 of the colours - lolz
relative positioning issue fixed

line-fill plots didn't quite match up with respect to where cursor position was left after plot operations - now fixed
"move" plot on Acorn systems does a "set" operation to background colour, ignoring whatever gcol paint mode may have been set for bg colour plots, so that's now behaviour on Agon too


Surepet care flaps / pets / feeders broke with update

The problem

Previously & as per the documentation, feeders, pets, and flaps are supposed to be able to be aliased with a name (not just an ID)...
The new version appears to only supports IDs as integers.
Environment

Home Assistant release with the issue: 0.106.0
Last working Home Assistant release (if known): 0.105.5
Operating environment (Hass.io/Docker/Windows/etc.): Docker
Integration causing this issue: Sure Petcare
Link to integration documentation on our website: https://www.home-assistant.io/integrations/surepetcare/

Problem-relevant configuration.yaml

surepetcare:
  username: !secret SurePetCareUser
  password: !secret SurePetCare
  flaps: [{id: !secret SurePetFlapLevel1, name: 'Level 1'}]
  pets: [{id: !secret SurePetPetBiscuit, name: Biscuit}]

Traceback/Error logs

expected int @ data['surepetcare']['flaps'][0]. Got OrderedDict([('id', !secret SurePetFlapLevel1), ('name', 'Level 1')])
expected int @ data['surepetcare']['pets'][0]. Got OrderedDict([('id', !secret SurePetPetBiscuit), ('name', 'Biscuit')]).

Additional information

I should also point out that I also tried the following (which should be the same):
```yaml
surepetcare:
  username: !secret SurePetCareUser
  password: !secret SurePetCare
  flaps:
    - id: !secret SurePetFlapLevel1
      name: 'Level 1'
  pets:
    - id: !secret SurePetPetBiscuit
      name: Biscuit

The config schema was changed last release and this part was unfortunately missed in the breaking change notes and the documentation update. Sorry about that.

@MartinHjelmare - So how do we specify a name for our pets, flaps etc?..
I hope this isn't removed.. as I really think a useful name is a good thing.

It's possible to change name and entity_id in the entity settings in the frontend.

I would personally prefer if this could be set in the config :( ... how is reducing options a good thing?

oh sorry, my fault! I will make a PR documenting this. Sorry!

I know it is possible to do that ... but ... I would personally prefer if this could be set in the config (so I have all my settings visible in one place), otherwise how do I know that flap 12345 is "Back_Door_Flap" by looking at the config? ... How is reducing options a good thing?? ..

Editing config files is not the target which home assistant is heading to... I am also more comfortable with yaml files than the ui, but afaik, in the best case, integration should be configured without editing a config file at all (via config flow thing blabla). "Custom" options like the name or an icon should be configured via the UI nowadays (please correct me if I am wrong)

I would personally prefer a good name in the first place based on the config ... it could still be renamed in the interface later if someone wishes.
I think that HA should be moving to "do more in the interface" ... but I personally think that doing things in interface (e.g. renaming entities) should edit the config file... but I imagine that the internals would need to be drastically altered to allow this to be done :P.

Ok... I updated my config ... not as bad as I thought :P ... ill just comment what they are ... I thought their name may be something bad like "Flap{ID}" ... but it appears that it gets the names from SurePet... so I don't find this that bad 👍

@i00 has this issue been solved?

@springstan ... documentation needs updating as stated ... I like being able to name my entities ... but I guess since it gets them from the web I am * kind of * ok with that...
I am curious to see what happens though if it can't get the names when HA starts ... if the details were entered with a name and ID it would be possible to check the flap name ... but how does the sensor know to call itself: binary_sensor.pet_flap_level_1_cat_flap_connectivity to see if its available ... I haven't really looked through the code to check though.

@i00 Home assistant prefers to not have the name in the config file - it will not come back, regardless what discussed here...
about your fear of crazy named entities... take a look at https://developers.home-assistant.io and read about what the entity registry is and how it works
@springstan from my site it can be closed when the mentioned PRs are merged

@springstan from my site it can be closed when the mentioned PRs are merged

Sure thing 👍

@benleb ... I have have other 3rd components change names that have looked up names from online services ... I guess they don't use unique_id.

The documentation has been updated therefore this issue is solved and can be closed.


Multi GPU and Multi Node solution

I wanted to know how to use Multi-GPUs and Multi-Node solutions with the current Punica code and how to use.
Also wanted to know about the runner and scheduler code which is mentioned in the paper, is it implemented can you guide me about that.

@abcdabcd987 Your paper states Punica implementations consist of two parts: a Python li- brary on top of PyTorch that runs large language models on a single GPU and other system components to support model serving across a GPU cluster.
Was looking for solution with a GPU cluster, Multi GPU or Multi GPU Nodes

Hey @abcdabcd987 any update on this?

+1

Sorry, I still hasn't got time to clean up the code. But here's some old code if you need it right now:

Tensor-parallel: https://github.com/punica-ai/punica/tree/stash-tensor-parallel
Cluster scheduler and streaming API server: https://github.com/abcdabcd987/punica-server-experimental

Both are not usable out-of-box.


Added GCP Labels

Few things to keep in mind:

Does labels apply to all resources?
we should call cleanup at the end of all the resource ingestion. to avoid duplicate cleanup calls.


control-network-access-to-your-resources : "exceptions" might require a higher visibility.

https://learn.microsoft.com/en-us/azure/azure-monitor/logs/private-link-design#control-network-access-to-your-resources
the exception section in this article might need a bit more "visibility". as it is not clear at first glance what significant impact those exceptions might have. maybe add a colored note with an emphasis so it is more obvious that the "exceptions" are important aspects to the overall understanding of the AMPLS concept.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 5b6378de-a16e-25f9-910b-12d2f43a04d5
Version Independent ID: a38cba02-f02c-a3a4-46d9-2f6e0b8c87d6
Content: Design your Azure Private Link setup - Azure Monitor
Content Source: articles/azure-monitor/logs/private-link-design.md
Service: azure-monitor
Sub-service: logs
GitHub Login: @guywi-ms
Microsoft Alias: guywild

updated the core of this issue and re-opened as the original assumption was wrong, but rooted in the updated cause.

@phwecker
Thanks for your feedback! We will investigate and update as appropriate.

@phwecker We have verified and updated the document as required. The update will go live in less than 24 hours. Thanks again!


Adds DevContainer configuration

Updates instructions for client-server exercise
TECH-583

What are we trying to achieve with the client-server-architecture exercise/assignment?

@tanyapowell The first exercise is to implement a basic HTTP client, second requires implementation of a server and client. What context would you expect to see?

The majority of the other READMEs are quite detailed and this one isn't

Can you add a task in JIRA, then? Rewriting the docs is out of scope for this PR


Update repositories.json

adding repo
I hope this is all correct......

I added to the Master Repo BUT your manifest is defective.
https://raw.githubusercontent.com/tmastersmart/hubitat-code/main/packages/repository.json <-- contains errors/incomplete. Please use JsonLint.com to validate your JSON files. I didn't check further, so I'd encourage you to validate them all. :)


Configure Azure Pipelines

This configuration builds the application and produces builds for both Debug and Release configurations.

fffff

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: mmallona sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


rehex: init at 0.3.1

Motivation for this change
Init rehex, the "Reverse Engineers' Hex Editor" at version 0.3.1
(website is https://github.com/solemnwarning/rehex)
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.


Error: No amount found, or it is not a whole number

Why Worldpay enforce whole numbers for payment?
This is just absurd... :(
https://github.com/Worldpay/worldpay-lib-php/blob/df4c9fede9be9c4931784c90d21ee52274052174/lib/AbstractOrder.php#L46-L48
https://github.com/Worldpay/worldpay-lib-php/blob/df4c9fede9be9c4931784c90d21ee52274052174/lib/Utils.php#L13-L16

Sorry for the issues.
I can't find it now in WorldPay API but:

The payout amount. This is a whole number with an exponent of 2 e.g. 250 would be 2.50.


Consider using SetValue() instead of "Add"-method for binding to dictionaries

The following test fails with an "ArgumentException" ("An item with the same key has already been added") because BindDictionary uses the Add method.
[Fact]
public void CanOverrideExistingDictionaryKey()
{
    var input = new Dictionary<string, string>
    {
        {"abc", "override"}
    };

    var configurationBuilder = new ConfigurationBuilder();
    configurationBuilder.AddInMemoryCollection(input);
    var config = configurationBuilder.Build();

    var options = new Dictionary<string, string>
    {
        {"abc", "default"}
    };

    config.Bind(options);

    Assert.Equal(1, options.Count);
    Assert.Equal("override", options["abc"]);
}

How about using the Item setter instead to just always use the last value?
var setter = typeInfo.GetDeclaredProperty("Item");
setter.SetValue(dictionary, item, new object[] { key });

My scenario: I am playing around with combining static default values and user-supplied overrides with IOptions<T>. I'm using a custom IConfigureOptions class to set some default values and want to have it combined with a regular call to services.Configure(IConfiguration config).
Not sure if you see this as a behavioral breaking change?

We should change this to override--matches the config pattern more closely.


Fix delete tests on Windows

In tests not targeting deletes, just drop the remaining handle earlier. In the delete test, skip the check that will fail.
Also add a check that deleting non-existent files returns an error.

Coverage increased (+0.006%) to 91.915% when pulling 2b5a4bbde2f181ad7a6a42f922038b0ec4dfee0c on wip/delay-test-deletes into 66d6e4e2464d5ca5a7c51d2266b21d25d26f628d on master.

Coverage increased (+0.006%) to 91.916% when pulling b12a97abe4558b46be7291f2c0b37bc4c42f4ae7 on wip/delay-test-deletes into 66d6e4e2464d5ca5a7c51d2266b21d25d26f628d on master.


Tube Status update

Adding support for all london tube lines, DLR and overground
The user can press up/down to change selection and middle to
force a refresh of the selected line
This pull request now doesn't have my other un-related changes in it.

there's no support for saving state yet, so when it's in widget mode it will display 'All lines' as that is the first in the list.  This behaviour is acceptable to me.  We can add saving state later.
It might also be an idea to delay the fetching after selection for a second or so while the user cycles though the list OR only update when the user presses the middle button.  Right now it refreshes as soon as the selection changes.

Thanks, Dom!  I'd been idly thinking about doing something like this, so I'm glad you took it on.  If it breaks showing the Victoria Line in widget mode, you're breaking the most important widget that I use on my MetaWatch every day. :]  Would you mind adding a persistent setting for the selected line before I merge?  If you're not familiar with Android SharedPreferences, you can look at the code in the quicktext sample to see how you might save & load the line identifier to persistent storage.
Also, do all the line names actually fit on the screen?  If not, consider either using abbreviated name strings.


How can i define multiple helpers in codeceptjs conf file.

Requirements:

I want to create a single project for testing of both mobile and web.
How can I define both web driver and appium helper and switch on basis of test?
How will the conf file be structured?

In this case use only Appium helper. Itis extended from WebDriverIO helper so it can be used both for web apps and mobile apps.
To change configuration based on a test use profiles http://codecept.io/configuration/#profile


fix: added missing ssl configuration missing from helm chart

Issue
https://gravitee.atlassian.net/browse/APIM-4891
https://github.com/gravitee-io/issues/issues/9726
Description
The helm chart currently does not support propagating required SSL configurations for Redis to the gateway configmap and ultimately underlying deployed pods. This causes pods to enter a crash loop, as Gravitee is unable to locate required SSL configurations for Redis in the gravitee.yml config file and establishes invalid connections with the Redis Instance/Cluster.
Evidence
log

gravitee.yml

gateway-configmap.yaml

Additional context
This fix was tested and<EMAIL_ADDRESS>was informed of the tested fix in a Graviteeio client environment.
Steps to Reproduce
Simply enable SSL in the helm chart and point to the correct port, Gravitee pods will immediately enter a crashloop once the configmap is reloaded.

Thanks again @gh0stsrc for your contribution!


Tensorflow inference interface cannot be compiled from build.gradle

When I enter the dependency named compile 'org.tensorflow:tensorflow-android:+' it produces the error named

Error:(44, 0) Could not find method compile() for arguments [org.tensorflow:tensorflow-android:+] on object of type org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler

Even after adding all the required libraries and dependencies, the error still persists.

Nagging Awaiting Response: It has been 14 days with no activityand the awaiting response label was assigned. Is this still an issue?

Nagging Awaiting Response: It has been 14 days with no activityand the awaiting response label was assigned. Is this still an issue?

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?


Fix metrics and kubelet args

Fix rolebinding for Hawkular
Give the capability of deleting metrics
Improve the way of setting kubelet options for nodes
Add deprecated warnings for old ways of setting kubelet options

Hi @ishentras I'm going on holidays I'll review tonight if PR still open.

Kitchen test failures:
http://jenkins.meirionconsulting.tk/job/cookbook-openshift3/job/fix-metrics_kubelet-args/3/console
     ∅  get sa/metrics-deployer -n openshift-infra --template '{{.metadata.name}}' exit_status should eq 0

     expected: 0
          got: 1

     (compared using ==)

     ∅  get sa/metrics-deployer -n openshift-infra --template '{{.metadata.name}}' stdout should match /metrics-deployer/
     expected "" to match /metrics-deployer/
     Diff:
     @@ -1,2 +1,2 @@
     -/metrics-deployer/
     +""

  Command oc
     ∅  get secret/metrics-deployer -n openshift-infra --template '{{.metadata.name}}' exit_status should eq 0

     expected: 0
          got: 1

     (compared using ==)

     ∅  get secret/metrics-deployer -n openshift-infra --template '{{.metadata.name}}' stdout should match /metrics-deployer/
     expected "" to match /metrics-deployer/
     Diff:
     @@ -1,2 +1,2 @@
     -/metrics-deployer/
     +""

  Command oc
     ✔  get pods -n openshift-infra --no-headers | egrep -q '^(metrics|heapster|hawkular)' exit_status should eq 0
  Command oc

I'll update the test kitchen as the way of deploying metrics is quite different than before.

Let me check the build was not broken by some other thing.

It was a rubocop failure - have corrected and kicked off a test.

Kitchen failure (I will re-run to see if it's transient):
[cookbook-openshift3-test-5] Running shell script
+ kitchen converge standalone-ose12-centos-72
-----> Starting Kitchen (v1.16.0)
-----> Converging <standalone-ose12-centos-72>...
       Preparing files for transfer
       Preparing dna.json
       Resolving cookbook dependencies with Berkshelf 5.6.4...
       Removing non-cookbook files before transfer
       Preparing environments
       Preparing roles
       Preparing validation.pem
       Preparing client.rb
-----> Chef Omnibus installation detected (12.17.44)
       Transferring files to <standalone-ose12-centos-72>
       Starting Chef Client, version 12.17.44
       resolving cookbooks for run list: ["cookbook-openshift3::default"]
       Synchronizing Cookbooks:
         - iptables (4.2.0)
         - cookbook-openshift3 (1.10.56)
         - selinux_policy (2.0.1)
         - compat_resource (12.19.0)
       Installing Cookbook Gems:
       Compiling Cookbooks...
       Converging 114 resources
       Recipe: cookbook-openshift3::default
         * service[origin-master] action nothing (skipped due to action :nothing)
         * service[origin-master-api] action nothing (skipped due to action :nothing)
         * service[origin-master-controllers] action nothing (skipped due to action :nothing)
         * execute[daemon-reload] action nothing (skipped due to action :nothing)
         * service[httpd] action nothing (skipped due to action :nothing)
         * service[docker] action nothing (skipped due to action :nothing)
         * service[NetworkManager] action nothing (skipped due to action :nothing)
         * service[openvswitch] action nothing (skipped due to action :nothing)
         * service[haproxy] action nothing (skipped due to action :nothing)
       Recipe: iptables::_package
         * yum_package[iptables-services] action install (up to date)
       Recipe: iptables::default
         * execute[rebuild-iptables] action nothing (skipped due to action :nothing)
         * directory[/etc/iptables.d] action create (up to date)
         * template[/usr/sbin/rebuild-iptables] action create (up to date)
         * template[/etc/network/if-pre-up.d/iptables_load] action create (skipped due to only_if)
         * file[/etc/sysconfig/iptables] action create_if_missing (up to date)
         * template[/etc/sysconfig/iptables-config] action create (up to date)
         * template[/etc/sysconfig/ip6tables-config] action create (up to date)
         * service[iptables] action enable (up to date)
         * service[iptables] action start (up to date)
       Recipe: cookbook-openshift3::common
         * iptables_rule[firewall_jump_rule] action enable
           * template[/etc/iptables.d/firewall_jump_rule] action create (up to date)
            (up to date)
         * yum_repository[centos-openshift-origin] action create
           * template[/etc/yum.repos.d/centos-openshift-origin.repo] action create (up to date)
           * execute[yum clean metadata centos-openshift-origin] action nothing (skipped due to action :nothing)
           * execute[yum-makecache-centos-openshift-origin] action nothing (skipped due to action :nothing)
           * ruby_block[yum-cache-reload-centos-openshift-origin] action nothing (skipped due to action :nothing)
            (up to date)
         * service[firewalld] action stop (up to date)
         * service[firewalld] action disable (up to date)
         * yum_package[deltarpm] action install (up to date)
         * yum_package[libselinux-python] action install (up to date)
         * yum_package[wget] action install (up to date)
         * yum_package[vim-enhanced] action install (up to date)
         * yum_package[net-tools] action install (up to date)
         * yum_package[bind-utils] action install (up to date)
         * yum_package[git] action install (up to date)
         * yum_package[bash-completion] action install (up to date)
         * yum_package[dnsmasq] action install (up to date)
         * yum_package[docker] action install (up to date)
         * bash[Configure Docker to use the default FS type for origin-centos-72] action run (skipped due to not_if)
         * template[/etc/sysconfig/docker-storage-setup] action create (up to date)
         * template[/etc/sysconfig/docker] action create (up to date)
         * ruby_block[Change HTTPD port xfer] action nothing (skipped due to action :nothing)
       Recipe: cookbook-openshift3::etcd_cluster
         * yum_package[httpd] action install (up to date)
         * directory[/etc/etcd/ca] action create (up to date)
         * directory[/etc/etcd/ca/certs] action create (up to date)
         * directory[/etc/etcd/ca/crl] action create (up to date)
         * directory[/etc/etcd/ca/fragments] action create (up to date)
         * template[/etc/etcd/ca/openssl.cnf] action create (up to date)
         * execute[ETCD Generate index.txt origin-centos-72] action run (up to date)
         * file[/etc/etcd/ca/serial] action create_if_missing (up to date)
         * execute[ETCD Generate CA certificate for origin-centos-72] action run (up to date)
         * directory[/var/www/html/etcd] action create (up to date)
         * directory[/var/www/html/etcd/generated_certs] action create (up to date)
         * directory[/var/www/html/etcd/generated_certs/etcd] action create (up to date)
         * remote_file[/var/www/html/etcd/generated_certs/etcd/ca.crt] action create (up to date)
         * remote_file[/var/www/html/etcd/generated_certs/etcd/ca.key] action create (up to date)
         * openshift_add_etcd[Add additional etcd nodes to cluster] action add_node (skipped due to only_if)
         * openshift_add_etcd[Remove additional etcd nodes to cluster] action remove_node (skipped due to not_if)
       Recipe: cookbook-openshift3::master
         * yum_package[origin] action install (up to date)
         * yum_package[httpd] action install (up to date)
         * iptables_rule[firewall_master] action enable
           * template[/etc/iptables.d/firewall_master] action create (up to date)
            (up to date)
         * directory[/etc/origin/master] action create (up to date)
         * template[/etc/origin/master/session-secrets.yaml] action create_if_missing (up to date)
         * remote_directory[/usr/share/openshift/examples] action create (skipped due to only_if)
         * remote_directory[/usr/share/openshift/hosted] action create
         Recipe: <Dynamically Defined Resource>
           * cookbook_file[/usr/share/openshift/hosted/logging-deployer.yaml] action create (up to date)
           * cookbook_file[/usr/share/openshift/hosted/metrics-deployer.yaml] action create (up to date)
            (up to date)
       Recipe: cookbook-openshift3::master_standalone
         * execute[Create the master certificates] action run (up to date)
         * yum_package[origin-master] action install (up to date)
         * template[/etc/systemd/system/origin-master.service] action create (skipped due to only_if)
         * template[/etc/sysconfig/origin-master] action create (up to date)
         * execute[Create the policy file] action run (up to date)
         * template[/etc/origin/master/scheduler.json] action create (up to date)
         * yum_package[httpd-tools] action install (up to date)
         * template[/etc/origin/openshift-passwd] action create (up to date)
         * openshift_create_master[Create master configuration file] action create
           * template[/etc/origin/master/master-config.yaml] action create (up to date)

         * service[origin-master] action start (up to date)
         * service[origin-master] action enable (up to date)
       Recipe: cookbook-openshift3::master
         * directory[/root/.kube] action create (up to date)
         * execute[Copy the OpenShift admin client config] action run (up to date)
       Recipe: cookbook-openshift3::nodes_certificates
         * directory[/var/www/html/node] action create (up to date)
         * directory[/var/www/html/node/generated-configs] action create (up to date)
         * execute[Generate certificate directory for origin-centos-72] action run (up to date)
         * execute[Generate certificate for origin-centos-72] action run (up to date)
         * execute[Generate the node server certificate for origin-centos-72] action run (up to date)
         * execute[Generate a tarball for origin-centos-72] action run (up to date)
       Recipe: cookbook-openshift3::node
         * ruby_block[Update ca trust] action nothing (skipped due to action :nothing)
         * file[/usr/local/etc/.firewall_node_additional.txt] action create (up to date)
         * iptables_rule[firewall_node] action enable
           * template[/etc/iptables.d/firewall_node] action create (up to date)
            (up to date)
         * directory[/etc/origin/node] action create (up to date)
         * template[/etc/sysconfig/origin-node] action create (up to date)
         * yum_package[origin-node] action install (up to date)
         * yum_package[origin-sdn-ovs] action install (up to date)
         * remote_file[Retrieve certificate from Master[origin-centos-72]] action create_if_missing (up to date)
         * execute[Extract certificate to Node folder] action nothing (skipped due to action :nothing)
         * directory[Fix permissions on /etc/origin/node] action create (up to date)
         * file[Fix permissions on /etc/origin/node/ca.crt] action create (up to date)
         * remote_file[/etc/pki/ca-trust/source/anchors/openshift-ca.crt] action create (up to date)
         * template[/etc/dnsmasq.d/origin-dns.conf] action create (up to date)
         * cookbook_file[/etc/NetworkManager/dispatcher.d/99-origin-dns.sh] action create (up to date)
         * template[/etc/origin/node/node-config.yaml] action create (up to date)
         * selinux_policy_boolean[virt_use_nfs] action setpersist
           * execute[selinux-setbool-virt_use_nfs-on] action run (skipped due to not_if)
            (up to date)
         * service[origin-node] action start

           ================================================================================
           Error executing action `start` on resource 'service[origin-node]'
           ================================================================================

           Mixlib::ShellOut::ShellCommandFailed
           ------------------------------------
           Expected process to exit with [0], but received '1'
           ---- Begin output of /bin/systemctl --system start origin-node ----
           STDOUT:
           STDERR: Job for origin-node.service failed because the control process exited with error code. See "systemctl status origin-node.service" and "journalctl -xe" for details.
           ---- End output of /bin/systemctl --system start origin-node ----
           Ran /bin/systemctl --system start origin-node returned 1

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/cookbook-openshift3/recipes/node.rb

           165:   service "#{node['cookbook-openshift3']['openshift_service_type']}-node" do
           166:     retries 5
           167:     retry_delay 2
           168:     action :start
           169:   end
           170: end

Repeated test - same problem.

Ok will test again but we need to drop support for 1.2...

I think that would be OK.

@ianmiell I have found the issue :+1:
v1.2.x does not support some kubelet args (Ex: pods-per-core)
Currently checking against v1.3.x

@ianmiell Can you run another test please?

Something's up following a jenkins upgrade. Looking at it.
On Tue, Jul 11, 2017 at 8:26 PM, William Burton<EMAIL_ADDRESS>wrote:

@ianmiell https://github.com/ianmiell Can you run another test please?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IshentRas/cookbook-openshift3/pull/142#issuecomment-314546584,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGrczeWY3_dkCGQx7VDQgXAH7zTOByGAks5sM8xKgaJpZM4ORrS4
.

--
My book on Docker in Practice: Get 39% off with the code 39miell
http://manning.com/miell/?a_aid=zwischenzugs&a_bid=e0d48f62

Passed tests. 👍  If 3.2 is not supported, a reminder to add that fact to the changelog.


Why this issue closed? Please have a look on my last comment

Why this issue closed? Please have a look on my last comment
Originally posted by @novavision in https://github.com/firebase/quickstart-unity/issues/657#issuecomment-629688028
This is not solved... how come you closed it?

I've reopened the original issue and so I'm closing this one.


caller or token owner should be able to approve self as operator for …

…own tokens.
Motivation:
As discussed, the EIP doesn't explicitly state that someone can't be their own operator for their own tokens. It has no effect per se, other than clearing any existing operator for the same tokens. An address being its own approver does no harm, and does not seem to be explicitly forbidden by the EIP.
approve(my_address, my_token_id) -> results in getApproved(my_token_id) returning the address of the owner. Now the token owner has all the same rights that they already had anyway.
Same for setApprovalForAll(my_address) -> the token owner is now the operator for all of its own tokens.
Changes

removed check for (operator == sender) in setApprovalForAll.
removed error ApproveToCaller (no longer being used)
modified unit test that was testing for that error, to check instead that the call is not reverted.
and then modified that same test to check that isApprovedForAll returns true for the token owner
added a new test in "setApprovalForAll" to check that similarly, an address can call approve to set itself as operator for its own tokens
added a further test that, once an address has called approve to set itself as operator, transferring will clear the approval (as it would for any other operator)

@jrkosinski Psst, can you help fix the linter issues for the test file?

Hi Vectorized,
Yeah, I saw that; I will fix those issues tonight (in a few hours in my
timezone)
Thanks,
John
On Fri, Aug 12, 2022 at 3:16 PM Vectorized @.***> wrote:

@jrkosinski https://github.com/jrkosinski Psst, can you help fix the
linter issues for the test file?
—
Reply to this email directly, view it on GitHub
https://github.com/chiru-labs/ERC721A/pull/407#issuecomment-1212851437,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AATG4NR5VKPKBW5PZFOFWXTVYYB4BANCNFSM56I2SS6Q
.
You are receiving this because you were mentioned.Message ID:
@.***>

@cygaar LGTM, what do you think?

👍


Moving sql(13) resources to v1beta1 version

Moving sql(13) resources to v1beta1 version

[x] azurerm_mssql_database_vulnerability_assessment_rule_baseline
[x] azurerm_mssql_elasticpool
[x] azurerm_mssql_job_agent
[x] azurerm_mssql_job_credential
[x] azurerm_mssql_server_vulnerability_assessment
[x] azurerm_sql_managed_database
[x] azurerm_sql_managed_instance
[x] azurerm_sql_managed_instance_active_directory_administrator
[x] azurerm_sql_managed_instance_failover_group
[x] azurerm_sql_active_directory_administrator
[x] azurerm_sql_database
[x] azurerm_sql_elasticpool
[x] azurerm_sql_firewall_rule

managed resources take >24h to be created.
azurerm_mssql_job_agent is not working with any type of configuration (even IdentifyFromProvider). Resource unable to locate sql server name.

azurem_sql_* resources are all deprecated and moving to ignore list


typing: Remove ViewFuncT.

This removes ViewFuncT and all the associated type casts with ParamSpec
and Concatenate. This provides more accurate type annotation for
decorators at the cost of making the concatenated parameters
positional-only. This change does not intend to introduce any other
behavioral difference.
Self-review checklist

[ ] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[ ] Each commit is a coherent idea.
[ ] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

Merged, thanks @PIG208!


imwheel: init at 1.0.0pre12

Motivation for this change
Add imwheel package, allowing configuration of mouse scroll wheel for X11 users.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

Thanks for the tip, updated and squashed.

@GrahamcOfBorg build imwheel

Success on x86_64-linux (full log)
Attempted: imwheel
Partial log (click to expand)
make[1]: Leaving directory '/build/imwheel-1.0.0pre12'
post-installation fixup
moving /nix/store/ck235im4w9jgs9hfahjg6g4skrx4pbjf-imwheel-1.0.0pre12/man to /nix/store/ck235im4w9jgs9hfahjg6g4skrx4pbjf-imwheel-1.0.0pre12/share/man
shrinking RPATHs of ELF executables and libraries in /nix/store/ck235im4w9jgs9hfahjg6g4skrx4pbjf-imwheel-1.0.0pre12
shrinking /nix/store/ck235im4w9jgs9hfahjg6g4skrx4pbjf-imwheel-1.0.0pre12/bin/imwheel
gzipping man pages under /nix/store/ck235im4w9jgs9hfahjg6g4skrx4pbjf-imwheel-1.0.0pre12/share/man/
strip is /nix/store/rpbg8gmqxhz8g61p1plz5d2srs84pvmv-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/ck235im4w9jgs9hfahjg6g4skrx4pbjf-imwheel-1.0.0pre12/bin
patching script interpreter paths in /nix/store/ck235im4w9jgs9hfahjg6g4skrx4pbjf-imwheel-1.0.0pre12
checking for references to /build in /nix/store/ck235im4w9jgs9hfahjg6g4skrx4pbjf-imwheel-1.0.0pre12...

Success on aarch64-linux (full log)
Attempted: imwheel
Partial log (click to expand)
post-installation fixup
moving /nix/store/bdkqpcy260mn2mg1d3d4ad860jx1k8d7-imwheel-1.0.0pre12/man to /nix/store/bdkqpcy260mn2mg1d3d4ad860jx1k8d7-imwheel-1.0.0pre12/share/man
shrinking RPATHs of ELF executables and libraries in /nix/store/bdkqpcy260mn2mg1d3d4ad860jx1k8d7-imwheel-1.0.0pre12
shrinking /nix/store/bdkqpcy260mn2mg1d3d4ad860jx1k8d7-imwheel-1.0.0pre12/bin/imwheel
gzipping man pages under /nix/store/bdkqpcy260mn2mg1d3d4ad860jx1k8d7-imwheel-1.0.0pre12/share/man/
strip is /nix/store/6dpnd5aniypn8124mmy8f88s4mq2zl07-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/bdkqpcy260mn2mg1d3d4ad860jx1k8d7-imwheel-1.0.0pre12/bin
patching script interpreter paths in /nix/store/bdkqpcy260mn2mg1d3d4ad860jx1k8d7-imwheel-1.0.0pre12
checking for references to /build in /nix/store/bdkqpcy260mn2mg1d3d4ad860jx1k8d7-imwheel-1.0.0pre12...
/nix/store/bdkqpcy260mn2mg1d3d4ad860jx1k8d7-imwheel-1.0.0pre12


test_besttrack_ensemble tests failing

Hi @zacharyburnettNOAA ,
I merged the latest upstream main into my fork and tried running all the tests. I ran into issues again with the besttrack_ensemble tests (running out of the box after installing):
One of them in the monovariate test (the reference json has everything on just one list but the output json has the curly braces on their own lines):
>                   assert len(test_lines) == len(reference_lines), message
E                   AssertionError: "/home/wpringle/EnsemblePerturbation/tests/data/output/test_monovariate_besttrack_ensemble/vortex_1_variable_perturbation_1.json" != "/home/wpringle/EnsemblePerturbation/tests/data/reference/test_monovariate_besttrack_ensemble/vortex_1_variable_perturbation_1.json"
E                     - {
E
E                     -   "max_sustained_wind_speed": -1.0
E
E                     ? ^^                                ^
E
E                     + {"max_sustained_wind_speed": -1.0}
E                     ? ^                                ^
E
E                     - }
E                   assert 3 == 1
E                     +3
E                     -1

__init__.py:44: AssertionError

The other for the multivariate test doesn't know about "mocker":
______________________ ERROR at setup of test_multivariate_besttrack_ensemble ______________________
file /home/wpringle/EnsemblePerturbation/tests/test_track_ensemble.py, line 52
  def test_multivariate_besttrack_ensemble(mocker):
E       fixture 'mocker' not found
>       available fixtures: _socket_marker, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, socket_disabled, socket_enabled, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

ah ok, these are simple fixes, I'll fix these
it looks like I just had the output files on my machine so it was passing the tests automatically

also I added pytest-mock to the requirements, so you might need to pip install that as well

@WPringle fixed with 984703e4d1f9952f4ddc18163165c7596be5c9d1, also you don't need to install pytest-mock after all as I didn't actually use it


Allow periodicity for very basic meshes

Codecov Report
Attention: Patch coverage is 53.96825% with 58 lines in your changes are missing coverage. Please review.

Project coverage is 67.59%. Comparing base (b980b2b) to head (7e1606c).

Files
Patch %
Lines

src/Smesh.jl
53.96%
58 Missing :warning:

Additional details and impacted files
@@             Coverage Diff              @@
##              main      #16       +/-   ##
============================================
- Coverage   100.00%   67.59%   -32.41%
============================================
  Files            2        2
  Lines           55      179      +124
============================================
+ Hits            55      121       +66
- Misses           0       58       +58

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@sloede Do you have any idea why the tests don't appear to have started, but have actually already run, as you can see if you look at the Checks tab or press on the red X on the last commit?
I mean here it it no problem and I will merge the PR anyway, but I'm just wondering.

@sloede Do you have any idea why the tests don't appear to have started, but have actually already run, as you can see if you look at the Checks tab or press on the red X on the last commit? I mean here it it no problem and I will merge the PR anyway, but I'm just wondering.

No, but I've seen this weird behavior before in other PRs in the trixi framework org. Dunno what's causing this...

@sloede Do you have any idea why the tests don't appear to have started, but have actually already run, as you can see if you look at the Checks tab or press on the red X on the last commit? I mean here it it no problem and I will merge the PR anyway, but I'm just wondering.

No, but I've seen this weird behavior before in other PRs in the trixi framework org. Dunno what's causing this...

Okay, thanks anyway.


Blosc2 Codec?

I recently noticed that Blosc2 2.0 has been released: https://twitter.com/Blosc2/status/1605529031780311041. This made me wonder whether we should revisit the idea of adding blosc2 support to Numcodecs and Zarr.
Obviously blosc2 has gone in the direction of adding more features--it's not much more than just a compression codec, and includes I/O, metadata, plugins, etc.--such that there is no clear boundary between Zarr's features and blosc2's features. So we would first want to decide which parts of blosc2 would be advantageous to expose as a codec in numcodecs. The main question would be whether there is a benefit to using the blosc2 superchunk feature to store multiple chunks in a single shard. If so, we will quickly resurrect the discussion about Caterva in Zarr (https://github.com/zarr-developers/zarr-python/issues/713).

Ooh, yes, I'd be really interested to see blosc2 integrated with numcodecs / Zarr.
For now, I believe that imagecodecs supports blosc2. But deep integration with numcodecs & Zarr would be very interesting.

@FrancescAlted and I discussed adding Blosc2 support to Zarr during the 2022 NumFOCUS Summit. Think what we concluded it should be possible. Note this is a different approach than what is outlined in the blogpost above.
The relevant thing for this discussion is the Blosc2 chunk format. This is the thing (I think) we would want to interact with.
The Blosc2 chunk format has blocks, which in Zarr we would call shards. AFAICT these are the same, but the terminology is different. Will use the term shards (as that is what we are familiar with), but keep this in mind when reading the spec.
In Blosc2 it tracks starts/offsets into shards, which we would likely want to extract and add to the metadata. This overlaps a bit with the sharding work @jstriebel has been doing ( https://github.com/zarr-developers/zarr-python/pull/1111 ) so potentially could work with that approach. Think we would want to remove the header as this would already be stored in Zarr metadata. As a first pass this would likely entail some copying to add/remove the header. Longer term we might want the option to pass in the header separately (or something like this).
Anyways this is my recollection of that conversation, which is not as fresh as it was. I may very well have forgotten/misunderstood things.

Perhaps another way to go about this would be look at using Kerchunk with Blosc2

Yes, kerchunk is interested in accessing the chunks within a compressed stream. You could regard the compressed blocks as chunks, but they would presumably not be equal length, so additional logic would be needed.
With the release of indexed_gzip, we may be able to something similar across all implementations. There is some tradeoff here between writing lots of references for individual chunks versus storing "shard" information elsewhere versus just requesting the exact matrix offsets from the storage and having the compression layer figure out what to actually read (I don't know if the third is actually possible)

Numcodecs adopting Blosc2 would be great.  BTW, what we recently released as 2.0 is Python-Blosc2, not C-Blosc2 (whose 2.0 release happened 1,5 years ago).
For what is worth, we have just merged Caterva into the main branch of C-Blosc2, so the later has gained multidimensional capabilities.  During the merge, the API has changed a bit (mainly to adapt to the Blosc way of doing things), but the functionality in the new C-Blosc2 is the same. We will let the new API to rest a bit, and when the dust would be settled, we will proceed with releasing C-Blosc2 (probably 2.7.0) pretty soon.

Just so the situation is clear, Blosc2 compressed data is not decompressable by Blosc1. On the other hand, Blosc1 compressed data can be decompressed by Blosc2.
https://github.com/Blosc/hdf5-blosc/issues/29#issuecomment-1030773468
For this reason Blosc1 and Blosc2 are registered as separate HDF5 filter plugins:
https://portal.hdfgroup.org/display/support/Filters#Filters-32026
I suspect numcodecs will need to support both Blosc1 and Blosc2 compression, simultaneously, for the sake of backwards compatibility.
You may also want to consider deprecating Blosc1 compression in favor of Blosc2 compression.

Wow, thanks for the info, @fschwar4. (And of course @FrancescAlted for the PR! :wink:)


Error while deploying chalice project in python3.6

I have python=3.6.1(installed using anaconda), virtualenv=15.1.0, and chalice=0.9.0.
I have created a new project in my virtual environment as mentioned in the chalice documentation. It runs fine on local but when I am trying to deploy it to AWS using chalice deploy, I get the following error:
$ chalice deploy
Initial creation of lambda function.
Creating role
Creating deployment package.
dyld: Library not loaded: @rpath/libpython3.6m.dylib
  Referenced from: <path to project>/pytools/.chalice/venv/bin/python
  Reason: image not found
ERROR: The executable <path to project>/pytools/.chalice/venv/bin/python is not functioning
ERROR: It thinks sys.prefix is '<path to project>/pytools' (should be '<path to project>/pytools/.chalice/venv')
ERROR: virtualenv is not compatible with this system or executable

Any suggestions on how to resolve this?
Thanks.

I don't think virtualenv and anaconda installed python are compatible. You can either use something like: https://github.com/conda-forge/virtualenv-feedstock (which I have not used so I can't vouch for since I have not personally used it) to get a virtualenv working with anaconda. Or use virtualenv and a normally installed python version. I personally I use pyenv to manage my environments.

I got a similar error using the version of python from homebrew (i.e., not anaconda):
:) 10:43:14 [Eric-Hanchrows-MacBook-Pro tmp]$ type -a python3
python3 is /usr/local/bin/python3
:) 10:50:58 [Eric-Hanchrows-MacBook-Pro tmp]$ python3 --version
Python 3.6.1
:) 10:51:04 [Eric-Hanchrows-MacBook-Pro tmp]$ python3 -m venv venv.chalice
:) 10:51:17 [Eric-Hanchrows-MacBook-Pro tmp]$ . venv.chalice/bin/activate
(venv.chalice) :) 10:51:22 [Eric-Hanchrows-MacBook-Pro tmp]$ pip install chalice
Collecting chalice
Collecting virtualenv<16.0.0,>=15.0.0 (from chalice)
  Using cached virtualenv-15.1.0-py2.py3-none-any.whl
Collecting click==6.6 (from chalice)
  Using cached click-6.6-py2.py3-none-any.whl
Collecting six<2.0.0,>=1.10.0 (from chalice)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting typing==<IP_ADDRESS> (from chalice)
Collecting botocore<2.0.0,>=1.5.40 (from chalice)
  Using cached botocore-1.5.75-py2.py3-none-any.whl
Collecting docutils>=0.10 (from botocore<2.0.0,>=1.5.40->chalice)
  Using cached docutils-0.13.1-py3-none-any.whl
Collecting jmespath<1.0.0,>=0.7.1 (from botocore<2.0.0,>=1.5.40->chalice)
  Using cached jmespath-0.9.3-py2.py3-none-any.whl
Collecting python-dateutil<3.0.0,>=2.1 (from botocore<2.0.0,>=1.5.40->chalice)
  Using cached python_dateutil-2.6.0-py2.py3-none-any.whl
Installing collected packages: virtualenv, click, six, typing, docutils, jmespath, python-dateutil, botocore, chalice
Successfully installed botocore-1.5.75 chalice-0.10.0 click-6.6 docutils-0.13.1 jmespath-0.9.3 python-dateutil-2.6.0 six-1.10.0 typing-<IP_ADDRESS> virtualenv-15.1.0
(venv.chalice) :) 10:51:31 [Eric-Hanchrows-MacBook-Pro tmp]$ chalice new-project helloworld && cd helloworld
(venv.chalice) :) 10:51:47 [Eric-Hanchrows-MacBook-Pro helloworld]$ chalice deploy
Initial creation of lambda function.
Updating IAM policy.

The following actions will be added to the execution policy:

logs:PutLogEvents
logs:CreateLogGroup
logs:CreateLogStream

Would you like to continue?  [Y/n]: y
Creating deployment package.
ERROR: The executable /private/tmp/helloworld/.chalice/venv/bin/python3 is not functioning
ERROR: It thinks sys.prefix is '/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6' (should be '/private/tmp/helloworld/.chalice/venv')
ERROR: virtualenv is not compatible with this system or executable
(venv.chalice) :( 100 10:52:00 [Eric-Hanchrows-MacBook-Pro helloworld]$

I was also running into this when using Homebrew's python3 on macOS.  However, if I made my virtualenv with the --copies option then it works fine.

@jonypawks can you give this another try on the current beta version of chalice? It no longer uses virtualenvs to build the deployment package so I imagine this would no longer be an issue.
http://chalice.readthedocs.io/en/latest/quickstart.html#quickstart-and-tutorial

The beta version works for me now.  Thanks!

Good to hear.  Let us know if you run into anything else.  Closing.


Create basic init code for TorahApp.org link fetching

Please note this is "basic" code in that it doesn't include unit tests.

Had to make a new branch over in #12 , please take a look and let me know, the new version is a fully self contained singular file for new extractors.


[mobile] Check boxes are rendering at different sizes

On the Roadmap page, not all the check boxes are rendering as square depending on the length of the line. See attached screenshot.

Pixel 6 Pro, Firefox 129.0 (Build #2016035991)

Fixed in latest release 0.2.0


Add Ruby language support

Thank you for the contribution @santry !
Normally I add the changelog updates myself because otherwise parallel PRs interfere with each other. But I will just fix this while merging.


Median function does not work

Median function does not work?
This is the fiddle.
http://jsfiddle.net/m187t5vk/
["Error: TypeError: alasql.fn.median is not a function
    at alasql.Query.eval as selectfn (eval at yy.Select.compileSelect2 (https://cdn.jsdelivr.net/npm/alasql@0.4/dist/alasql.js:9730:9), <anonymous>:3:51)
    at doJoin (https://cdn.jsdelivr.net/npm/alasql@0.4/dist/alasql.js:7452:27)
    at doJoin (https://cdn.jsdelivr.net/npm/alasql@0.4/dist/alasql.js:7536:8)
    at queryfn3 (https://cdn.jsdelivr.net/npm/alasql@0.4/dist/alasql.js:6954:2)
    at queryfn2 (https://cdn.jsdelivr.net/npm/alasql@0.4/dist/alasql.js:6928:9)
    at source.datafn (https://cdn.jsdelivr.net/npm/alasql@0.4/dist/alasql.js:8363:20)
    at https://cdn.jsdelivr.net/npm/alasql@0.4/dist/alasql.js:6876:19
    at Array.forEach (<anonymous>)
    at queryfn (https://cdn.jsdelivr.net/npm/alasql@0.4/dist/alasql.js:6872:16)
    at statement (https://cdn.jsdelivr.net/npm/alasql@0.4/dist/alasql.js:8020:14)"]

This is what happened on my machine.
SELECT median(`total_count`) FROM CSV('/myproject/a893312d-9d41-4491-966f-f6a237a7a2e6.csv') AS `out_csv`
undefined:3
var y;var r={'(total_count)':(y=alasql.fn["median"](p['out_csv']['total_count']),y===y?y:undefined)};return r
                                                   ^

TypeError: alasql.fn.median is not a function
    at alasql.Query.eval [as selectfn] (eval at yy.Select.compileSelect2 (/myproject/node_modules/alasql/dist/alasql.fs.js:9665:9), <anonymous>:3:52)
    at doJoin (/myproject/node_modules/alasql/dist/alasql.fs.js:7459:27)
    at doJoin (/myproject/node_modules/alasql/dist/alasql.fs.js:7545:8)
    at queryfn3 (/myproject/node_modules/alasql/dist/alasql.fs.js:6966:2)
    at queryfn2 (/myproject/node_modules/alasql/dist/alasql.fs.js:6940:9)
    at parseText (/myproject/node_modules/alasql/dist/alasql.fs.js:17574:10)
    at /myproject/node_modules/alasql/dist/alasql.fs.js:3496:7
    at FSReqCallback.readFileAfterClose [as oncomplete] (node:internal/fs/read_file_context:68:3)
Debugger listening on ws://<IP_ADDRESS>:9232/c85e54b8-16f7-4449-bdfd-05c556ca648c

Is median part of the SQL definition?

BTW, on the server it will throw a process.uncaughtExeption, breaking the server. There's no way to catch it. I believe we should be able to catch those in the callback error

Released as part of v2.1.5


Can't build on Windows: Plugin or its dependencies couldn't be resolved

The first one error message when i'm running build.bat just like it's been told in build tutorial:
[INFO] Scanning for projects...
[INFO]
[INFO] ----------------------< org.sqlite:sqlite-server >----------------------
[INFO] Building SQLite database server 0.3.30
[INFO] --------------------------------[ jar ]---------------------------------
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.2.0/maven-resources-plugin-3.2.0.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.701 s
[INFO] Finished at: 2024-05-26T05:20:03+12:00
[INFO] ------------------------------------------------------------------------
[ERROR] Plugin org.apache.maven.plugins:maven-resources-plugin:3.2.0 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-resources-plugin:jar:3.2.0: Could not transfer artifact org.apache.maven.plugins:maven-resources-plugin:pom:3.2.0 from/to central (https://repo.maven.apache.org/maven2): Received fatal alert: protocol_version -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException

i'm using apache maven 3.6.1 and jdk 7
i've tried removing .m2 folder nothing changed

This solution from Stack Overflow question helped me:

For a permanent solution(mostly required in Java 7) - in you build directory(where you do the mvn command from) add directory:
.mvn (in cmd mkdir .mvn)
and in it create file
jvm.config
and in put the following line:
-Dhttps.protocols=TLSv1.2