[stable/prometheus-adapter] Allow the resource-reader-cluster-role to access Deployments

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
Feature Request
Version of Helm and Kubernetes:
Helm Client: v2.9.1
Helm Server: v2.9.1
Kubernetes Version: v1.10.6-gke.2
Which chart:
Prometheus Adapter
What happened:
When implementing autoscaling for a queue based workload, I found it easiest to associate a jobs_per_worker metric with the deployment that works that queue. The metrics are exported by a single pod and associated via correctly labeling the metric.
When configuring my HPA, I used something similar to the following:
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: "queue-one-hpa"
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: "queue-one-workers"
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Object
    object:
      target:
        kind: Deployment
        name: queue-one-workers
      metricName: jobs_per_worker
      targetValue: 100

When attempting to scale in this fashion, I received errors in the logs of the adapter that indicated that it the cluster role did not have access to deployments and therefore could not fetch the metric.
What you expected to happen:
Since Object metrics can be associated with Services, Deployments, etc... I would expect this adapter to be able to scale correctly with object metrics for those resources.
How to reproduce it (as minimally and precisely as possible):
Try to scale on an Object metric using a Deployment object as the target.
Anything else we need to know:
I'd be happy to work on this change. In my fork I simply added extensions to the apiGroups and deployments to the resources configuration in https://github.com/helm/charts/blob/master/stable/prometheus-adapter/templates/custom-metrics-resource-reader-cluster-role.yaml
We could make the same change here as well but I wanted to solicit some feedback to see if this makes sense or has other ramifications.

I'm able to associate a metric with a Deployment in my HPA with the existing chart. prometheus-adapter doesn't fetch metrics from a deployment, it fetches it from prometheus and uses your rule to group it by the deployment and expose it via its own REST API to be consumed by HPA. Thus, it shouldn't really need access to Deployment.
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: edge-worker
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: worker
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Object
    object:
      metricName: foo_rate
      targetValue: 0.05
      target:
        apiVersion: apps/v1
        kind: Deployment
        name: worker

One thing I noticed with your HPA is that it doesn't have "apiVersion: apps/v1" in target like mine. Not sure if it makes a difference. Can you post your custom rule as well? Here's mine:
rules:
  custom:
  - seriesQuery: '{__name__=~"^foo_seconds_count$"}'
    resources:
      overrides:
        app:
          group: apps
          resource: deployments
        kubernetes_namespace:
          resource: namespace
    name:
      matches: ""
      as: "foo_rate"
    metricsQuery: sum(rate(<<.Series>>{<<.LabelMatchers>>}[5m])) by (<<.GroupBy>>)

Ahh I see where I was going wrong now... And feel a little sheepish.
So I reinstalled the chart without my patch, and my HPAs still worked. I then tried to reproduce the stack trace that I had been seeing to see why I was under the impression that they didn't. Turns out it was from when I was first trying to debug my metrics that I encountered the issue.
In particular, I was using the FAQ, and running kubectl get commands to verify that things worked.
> kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 | jq
{
  "kind": "APIResourceList",
  "apiVersion": "v1",
  "groupVersion": "custom.metrics.k8s.io/v1beta1",
  "resources": [
    {
      "name": "namespaces/jobs_per_worker",
      "singularName": "",
      "namespaced": false,
      "kind": "MetricValueList",
      "verbs": [
        "get"
      ]
    },
    {
      "name": "deployments.extensions/jobs_per_worker",
      "singularName": "",
      "namespaced": true,
      "kind": "MetricValueList",
      "verbs": [
        "get"
      ]
    }
  ]
}```

I then tried to see if they were being associated with their deployments correctly:

kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1/deployments/*/jobs_per_worker | jq
Error from server (InternalError): Internal error occurred: unable to list matching resources

When I checked the logs of the server,

I0925 15:56:39.012448       1 handler.go:143] prometheus-custom-metrics-adapter: GET "/apis/custom.metrics.k8s.io/v1beta1/deployments/*/jobs_per_worker" satisfied by gorestful with webservice /apis/custom.metrics.k8s.io/v1beta1
I0925 15:56:39.012677       1 round_trippers.go:386] curl -k -v -XGET  -H "Accept: application/json" -H "User-Agent: adapter/v0.0.0 (linux/amd64) kubernetes/$Format" -H "Authorization: Bearer " 'https://<IP_ADDRESS>:443/apis/extensions/v1beta1/deployments'
I0925 15:56:39.033201       1 round_trippers.go:405] GET https://<IP_ADDRESS>:443/apis/extensions/v1beta1/deployments 403 Forbidden in 20 milliseconds
I0925 15:56:39.033247       1 round_trippers.go:411] Response Headers:
I0925 15:56:39.033253       1 round_trippers.go:414]     X-Content-Type-Options: nosniff
I0925 15:56:39.033258       1 round_trippers.go:414]     Content-Length: 393
I0925 15:56:39.033261       1 round_trippers.go:414]     Date: Tue, 25 Sep 2018 15:56:39 GMT
I0925 15:56:39.033266       1 round_trippers.go:414]     Audit-Id: 4c231987-ee5d-4c3d-bd91-afb7cb76a0ef
I0925 15:56:39.033270       1 round_trippers.go:414]     Content-Type: application/json
I0925 15:56:39.033485       1 request.go:897] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"deployments.extensions is forbidden: User "system:serviceaccount:kube-monitoring:custom-metrics" cannot list deployments.extensions at the cluster scope: Unknown user "system:serviceaccount:kube-monitoring:custom-metrics"","reason":"Forbidden","details":{"group":"extensions","kind":"deployments"},"code":403}
E0925 15:56:39.033683       1 provider.go:202] unable to list matching resource names: deployments.extensions is forbidden: User "system:serviceaccount:kube-monitoring:custom-metrics" cannot list deployments.extensions at the cluster scope: Unknown user "system:serviceaccount:kube-monitoring:custom-metrics"
I0925 15:56:39.034612       1 wrap.go:42] GET /apis/custom.metrics.k8s.io/v1beta1/deployments/%2A/jobs_per_worker: (50.635746ms) 500
goroutine 2864 [running]:
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).recordStatus(0xc42038d2d0, 0x1f4)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:207 +0xd2
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).WriteHeader(0xc42038d2d0, 0x1f4)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:186 +0x35
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).WriteHeader(0xc4204bb400, 0x1f4)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:192 +0xac
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/metrics.(*ResponseWriterDelegator).WriteHeader(0xc42020be60, 0x1f4)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/metrics/metrics.go:307 +0x45
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters.SerializeObject(0x168f342, 0x10, 0x7f2b25d87290, 0xc4209b2100, 0x17fc5c0, 0xc4204bc440, 0xc4209fed00, 0x1f4, 0x17e9080, 0xc420b28000)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters/writers.go:95 +0x8d
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters.WriteObjectNegotiated(0x17fd7c0, 0xc4201561e0, 0x1695338, 0x15, 0x168621b, 0x7, 0x17fc5c0, 0xc4204bc440, 0xc4209fed00, 0x1f4, ...)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters/writers.go:123 +0x33f
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters.ErrorNegotiated(0x17e0d80, 0xc42041bef0, 0x17fd7c0, 0xc4201561e0, 0x1695338, 0x15, 0x168621b, 0x7, 0x17fc5c0, 0xc4204bc440, ...)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters/writers.go:142 +0x149
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers.(*RequestScope).err(0xc420808f00, 0x17e0d80, 0xc42041bef0, 0x17fc5c0, 0xc4204bc440, 0xc4209fed00)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/rest.go:69 +0xd7
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers.ListResource.func1(0x17fc5c0, 0xc4204bc440, 0xc4209fed00)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/handlers/get.go:264 +0xc34
github.com/directxman12/k8s-prometheus-adapter/vendor/github.com/kubernetes-incubator/custom-metrics-apiserver/pkg/apiserver/installer.restfulListResource.func1(0xc42020bdd0, 0xc42031d490)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/github.com/kubernetes-incubator/custom-metrics-apiserver/pkg/apiserver/installer/installer.go:286 +0xc4
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/metrics.InstrumentRouteFunc.func1(0xc42020bdd0, 0xc42031d490)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/metrics/metrics.go:225 +0x208
github.com/directxman12/k8s-prometheus-adapter/vendor/github.com/emicklei/go-restful.(*Container).dispatch(0xc420766a20, 0x7f2b25b7e660, 0xc42000cba0, 0xc4209fed00)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/github.com/emicklei/go-restful/container.go:282 +0xa57
github.com/directxman12/k8s-prometheus-adapter/vendor/github.com/emicklei/go-restful.(*Container).Dispatch(0xc420766a20, 0x7f2b25b7e660, 0xc42000cba0, 0xc4209fed00)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/github.com/emicklei/go-restful/container.go:199 +0x57
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server.director.ServeHTTP(0x16a5252, 0x21, 0xc420766a20, 0xc4200b6a10, 0x7f2b25b7e660, 0xc42000cba0, 0xc4209fed00)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/handler.go:146 +0x4ec
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuthorization.func1(0x7f2b25b7e660, 0xc42000cba0, 0xc4209fed00)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/filters/authorization.go:64 +0x4bd
net/http.HandlerFunc.ServeHTTP(0xc42049e440, 0x7f2b25b7e660, 0xc42000cba0, 0xc4209fed00)
/usr/local/go/src/net/http/server.go:1947 +0x44
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters.WithMaxInFlightLimit.func1(0x7f2b25b7e660, 0xc42000cba0, 0xc4209fed00)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters/maxinflight.go:160 +0x3fc
net/http.HandlerFunc.ServeHTTP(0xc420235680, 0x7f2b25b7e660, 0xc42000cba0, 0xc4209fed00)
/usr/local/go/src/net/http/server.go:1947 +0x44
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithImpersonation.func1(0x7f2b25b7e660, 0xc42000cba0, 0xc4209fed00)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/filters/impersonation.go:49 +0x1ce4
net/http.HandlerFunc.ServeHTTP(0xc42049e600, 0x7f2b25b7e660, 0xc42000cba0, 0xc4209fed00)
/usr/local/go/src/net/http/server.go:1947 +0x44
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuthentication.func1(0x7f2b25b7e660, 0xc42000cba0, 0xc4209fed00)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/endpoints/filters/authentication.go:75 +0x2b3
net/http.HandlerFunc.ServeHTTP(0xc42049e640, 0x7f2b25b7e660, 0xc42000cba0, 0xc4209fec00)
/usr/local/go/src/net/http/server.go:1947 +0x44
github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1(0xc42050a600, 0x17feb80, 0xc42000cba0, 0xc4209fec00, 0xc420952c00)
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:97 +0x8d
created by github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP
/go/src/github.com/directxman12/k8s-prometheus-adapter/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:96 +0x1b5
logging error output: "{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Internal error occurred: unable to list matching resources","reason":"InternalError","details":{"causes":[{"message":"unable to list matching resources"}]},"code":500}\n"
[[kubectl/v1.11.3 (darwin/amd64) kubernetes/a452946] <IP_ADDRESS>:47136]

It's the `*` in the request that I did before. In order to fulfill that request, the custom metrics server queries K8s to find all the deployments that are available in order to query for those metrics in Prometheus. When in use via the HPA, the request is made with the deployment specified in the API and thus has no need to go ask K8s for the deployments.

So I guess its a little bit of a bummer for debugging that I can't get all the metrics for each deployment in one query, but I don't think that necessarily warrants the change made in https://github.com/helm/charts/pull/7926

Thanks for chiming in @steven-sheehy! I think we can close the PR and this issue.


[FEATURE] BulkIndexer - ensure that subsequent requests on the same documentID are consistently sent to the same worker

Is your feature request related to a problem?
The BulkIndexer allows the configuration of multiple workers via NumWorkers. Which fans out bulk requests to N go routines which manage bulk buffers and API requests separately.
There is no allocation logic for the fan out so there is no guarantee that multiple actions to the same document will exist in the same request. And more importantly it could break ordering as subsequent actions may be flushed before previous ones depending on the state of the what buffer is for that particular worker.
This can cause issues with systems processing a lot of messages for the same items such as streams from kafka where many item updates could occur for the same item. It makes it possible to trigger document not found exceptions in scenarios where 1 message creates and the next message is an update, because the update worker flushes but the create message is still on the buffer of another worker.
If document IDs can be guaranteed to be on the same worker they will either be in the same bulk request or at least in the correct order of bulk requests.
What solution would you like?

Replace the existing internal queue channel with an array of channels up to the NumWorkers.
Create a numerical hash of the DocumentID.
Use that numerical hash to determine what queue the request should be added to on the Add function.

What alternatives have you considered?

Consuming code could handle this by setting NumWorkers to 1, but this remove the horizontal scaling of this library and can cause throughput issues.
Additionally consuming code could create N BulkIndexer instances all with 1 NumWorkers and handle this hashing process there to add messages to different bulk indexers based on the hash. But it would be better if this was wrapped up in this library as it's generic.

Do you have any additional context?
N/A

This is a good idea. Want to try to implement this @mikesmitharoo?
Catch All Triage - 1 2 3 4 5


Why does OTA require embedded SSL cert and AT HTTPS access does not?

I would like to know what the rationale for the following is:

AT based commands that access HTTPS resources can simply be called by passing the URL and whatever parameters, if needed, such as the command [AT+HTTPCLIENT].  With these commands we can make the ESP32 access ANY website that is in HTTPS.

Why is it that OTA firmware downloads to the ESP require an embedded SSL key as part of the process? Why cannot the OTA upgrade just use the same HTTPS methodology of accessing SSL protected websites to download firmware, instead of us having to embed an SSL cert into the ESP32?  What difference does it make to the ESP32 whether I am fetching the ESP32 OTA firmware upgrade or just a standard HTTPS website?

it seems inconsistent that the ESP32 requires an embedded SSL to get an OTA update when the same ESP32 using AT commands can get any website data on the planet without needing the embedded SSL.
The MAIN REASON I am asking is this:
a) if I embed an SSL cert for a specific website for my ESP32 OTA firmware updates, that SSL is only good for a max of 1 year.  Once that SSL expires, and my ESP32 tries to do an OTA Update, it will fail -0x7200 due to the SSL being expired.  At that point the ESP32 cannot get any more OTA updates via SSL, due to the embedded key being expired.
If I ship 1000 boards with an ESP32 chip and the embedded SSL which expires (for example) 7/1/23, any unit that tries to do an OTA upgrade after 7/1/23 will fail to connect to the website due to the embedded SSL cert being expired.
b) on the other hand, if we do NOT have to embed an SSL and the ESP32 simply uses the same method of accessing HTTPS websites that the various AT commands use (none of which require us to embed an SSL cert), this would make the ESP32 OTA firmware updates future-proof, and not cut our units off of updates after 1 year.  The ESP32 would simply do the SSL handshake on the website and download the OTA firmware.  No embedded cert necessary.
There does not seem (to me) to be any upside to requiring an SSL embedded to get an OTA firmware update as opposed to using the same methods as used in the AT commands to access SSL sites.
At minimum I'd like to see a check box in MENUCONFIG to allow the user to OPTIONALLY embed an SSL for OTA, versus it just using the standard HTTPS commands in the ESP32 to fetch the firmware.

@heatsiphon AFAIK, ESP-AT OTA does not require a certificate and key as a must.
Could you please tell us where we wrote a wrong terrible document about it.


check prometheus if it sends alerts to old and new alertmanagers

check if prometheus sends alerts to old and new alertmanagers

It is


sigal build does not always re-enable terminal echo

I'm not sure what's triggering this (possibly the error outputs?), but whenever I run sigal build my terminal's echo is disabled. This means I cannot see subsequent commands I enter until I run reset. Here is an example:
u@host:~/share/Pictures$ sigal build
Collecting albums, done.
Collecting files  [####################################]  100%
Processing files  [####################################]  14751/14751
ERROR: Some files have failed to be processed:
ERROR:   - 2008/New Year’s Eve, December 31, 2008/._Photos - 1817 of 32274.webm
ERROR:   - 2010/10-31 Porsche Skills Day/VIDEO0020.webm
ERROR:   - 2010/10-31 Porsche Skills Day/VIDEO0021.webm
ERROR: You can run "sigal build" in verbose (--verbose) or debug (--debug) mode to get more details.
   Writing files  [####################################]  100%

Done, processed 14086 images and 665 videos (3 failed) in 41723.54 seconds.
u@host:~/share/Pictures$ /home/u/share/Pictures
u@host:~/share/Pictures$  11:32:13 up 2 days, 22:10,  5 users,  load average: 0.39, 0.44, 0.50
u@host:~/share/Pictures$

In the above trace, just after sigal exits I blind-entered pwd and uptime.

This has been reported once, but I have no idea where it comes from. The progress bars are managed with click, so that could be an option as well, hard to say without a way to reproduce the problem.


Publish to NPM

Publish to NPM as @blockstack/cli and update installation instructions in the Readme.

May I waltz in with the suggestion we rename the package to cli, so on npm it's @blockstack/cli ?

@blockstack/cli sounds good to me.
I believe the reason for the weird naming of this repo was due to a previous, now deprecated package called blockstack-cli

I'd caution against putting this on NPM.  This is a power-users only tool, and is unsafe for use with the general public or even with developers who aren't familiar with the inner workings of Blockstack.  For example, the CLI requires users to enter raw private keys for a lot of operations, and doesn't really stop you from shooting yourself in the foot if you enter the wrong private key(s) or wrong address(es), or enter private keys in the wrong order.

We are targeting developers with this tool though. The reason to publish it to NPM is to make it a little easier to install. I don't think we'll see non-developers flock to the tool by publishing it. I think the only thing we're doing here is improving our support for the tool.


'Dependencies is currently shipped as two binaries'

The README says

Dependencies is currently shipped as two binaries

Yet, I downloaded Dependencies_x64_Release.zip and saw this within my zip-handling program:

Still: thank you for the software!

Just read more:

Dependencies is currently shipped as two binaries (no installer present) : Dependencies.exe as a CLI tool and DependenciesGui.exe for its GUI counterpart (see screenshot). Just click on one of the release numbers above (preferably the latest), download and uncompress the archive and run DependenciesGui.exe.
Since the binary is not signed, SmartScreen might scream at runtime. Dependencies also bundle ClrPhTester.exe, a dumpbin-like executable used to test for non-regressions.

https://github.com/lucasg/Dependencies/blob/1997a40000b77bd3326cbc33672a7b9f78bb23f3/README.md#installation-and-usage

The further text gives one to understand that there are three tiles in total. My screenshot showed dozens of files.


Check repo for bad state first

Addresses issue #5
I'm attempting to test this locally, but python setup.py install seems to install the old version. If you have another option I'm all ears.

pushed a fix

I speculate CI will pick this up and re-run


First pass on Smart detector migration docs

Here's a first pass. I still have to look more closely at the graphics.  And we need a Next Steps.
Some changes I made

Present verb tense whenever possible.
Making lists and logic easier to read using bullets.

Also, the links might seem broken now, but Brian is making changes in the structure of the docs so I'm anticipating those.  Alerts will go into an /alerts directory.  App insights will still be in /app.  We always use relative links so they don't break in air grapped environments (like JEDI)


Add prototype chaos visualizer

Kate wrote various standalone visualizers outside the numberscope (https://math.katestange.net/numberscope/).  The chaos visualizer is not computationally intensive (no factoring etc), and fairly simple (not too much code), so this is a good first choice to try to port into Numberscope.  Note:  a particularly beautiful implementation of the pure chaos game (not based on an arbitrary sequence) is available at (https://experiments.withgoogle.com/the-chaos-game).

Nice! This suggests to me these question: is there an overall project goal of at some point incorporating all of the visualizers on that numberscope page you link above? More specifically, is there an intention to replace the current ShiftCompare visualizer with the Self-Similarity visualizer?

Yes, I think so, if they are up to snuff.  But the user interface will probably change (right now I coded a pop-up menu into each visualizer, etc.), so they'll be changed as they are incorporated.

OK, then I'd recommend when you have a chance filing a placeholder issue for porting other standalone visualizers to Numberscope, that could in the fullness of time be replaced by a series of issues, one for each visualizer that should be incorporated; and the one for Self-Similarity would make it clear if it replaces ShiftCompare.


Cepheid guide

First version of the Cepheid guide

Okay. I removed *png and taxonomy.html and changed toward --> near in the description


[Codelab] Add better instructions in "Write a Flutter desktop application"

After writing the main.dart code and completing OAUTH2 implementation, the app needs to be stopped and then be started again because a plugin is added and simply hot restarting wont work.
I was getting this error . This should be mentioned in the codelab.
Unhandled Exception: MissingPluginException(No implementation found for method canLaunch on channel plugins.flutter.io/url_launcher)

Hi @Umar-Waseem, Thanks for filing the issue. Can you please point to the codelab you are referring to?

Hi @Umar-Waseem, Thanks for filing the issue. Can you please point to the codelab you are referring to?

https://codelabs.developers.google.com/codelabs/flutter-github-client#0
This is the codelab im reffering to.

cc: @domesticmouse

Fix is in flight.
Googlers: cl/513415878


Azure rsource manager tools issues in vscode

when install ARM tools in vscode.when  you want to create a storage account suggestion for storage account is not showing.in the notification it is showing loading ARM schema and it is still loading. I disable the extension and restarted it even go with uninstalling vscode and reinstalling it .but the issue still persist.

@vishnuk47 Just to make sure...  The number of schemas available has grown much larger and it takes a minute or two for the extension to finish loading.  After several minutes, do any resource suggestions show up?  Thx.

And does it ever get past the Loading stage for Intellisense?  Also take a look at the Output pane, "Azure Resource Manager Tools" selected in dropdown, and see if there are any errors reported there.

Do note this is being fixed: https://github.com/microsoft/vscode-azurearmtools/issues/1789


use docker images in docker/trustlines

please note that we use the latest tag, just like we did previously by using the
develop branch.
see https://github.com/trustlines-network/relay/issues/166

Codecov Report

Merging #168 into develop will decrease coverage by 0.29%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           develop     #168     +/-   ##
==========================================
- Coverage    60.17%   59.88%   -0.3%
==========================================
  Files           42       42
  Lines         2702     2702
==========================================
- Hits          1626     1618      -8
- Misses        1076     1084      +8

Impacted Files
Coverage Δ

relay/blockchain/currency_network_proxy.py
80% <0%> (-3.81%)
:arrow_down:

relay/blockchain/proxy.py
76.19% <0%> (-3.58%)
:arrow_down:

relay/network_graph/graph.py
75.1% <0%> (-0.43%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update febd8ee...5b1a004. Read the comment docs.


feat: integrate forc-wallet and forc-deploy

Description
closes #4743.
This PR integrates forc-wallet and forc-deploy together so that forc-deploy can offer to sign your transactions without running a second terminal window. Below there are some inputs and outputs after this change.
forc-deploy --random-salt

Contract id: 0x77e0c380dff45759d1a16b6add83d0fd38e8d86ddd0c96cd5117c82b6d958edb

Please provide the password of your encrypted wallet vault at "/Users/kayagokalp/.fuel/wallets/.wallet":

At this point if the user enters wrong password:
Contract id: 0x77e0c380dff45759d1a16b6add83d0fd38e8d86ddd0c96cd5117c82b6d958edb

Please provide the password of your encrypted wallet vault at "/Users/kayagokalp/.fuel/wallets/.wallet":
Error: Failed to access forc-wallet vault. Please check your password

If the user enters correct password:
Contract id: 0x77e0c380dff45759d1a16b6add83d0fd38e8d86ddd0c96cd5117c82b6d958edb

Please provide the password of your encrypted wallet vault at "/Users/kayagokalp/.fuel/wallets/.wallet":
Do you accept to sign this transaction with fuel15f7mn9vue4eks9znjqf5tg6ylv8vkfh9xcdh3v6z9s7yagewgwasaxpxem [y/N]:

If the user enters y or Y and hits enter:
Contract id: 0x77e0c380dff45759d1a16b6add83d0fd38e8d86ddd0c96cd5117c82b6d958edb

Please provide the password of your encrypted wallet vault at "/Users/kayagokalp/.fuel/wallets/.wallet":
Do you accept to sign this transaction with fuel15f7mn9vue4eks9znjqf5tg6ylv8vkfh9xcdh3v6z9s7yagewgwasaxpxem [y/N]:y
contract 98f158f121c10bdfea5ffbdda904d3e7b61c7f9d80ae2beff75b5d04f0f3547b deployed in block 0x1dd25813f6a82a69347079feb265eeeb37e89f992cea65ad1373b378b80c1ecb

If the user enters anything else:
Contract id: 0x77e0c380dff45759d1a16b6add83d0fd38e8d86ddd0c96cd5117c82b6d958edb

Please provide the password of your encrypted wallet vault at "/Users/kayagokalp/.fuel/wallets/.wallet":
Do you accept to sign this transaction with fuel15f7mn9vue4eks9znjqf5tg6ylv8vkfh9xcdh3v6z9s7yagewgwasaxpxem [y/N]:N
Error: User refused to sign

Checklist

[ ] I have linked to any relevant issues.
[ ] I have commented my code, particularly in hard-to-understand areas.
[ ] I have updated the documentation where relevant (API docs, the reference, and the Sway book).
[ ] I have added tests that prove my fix is effective or that my feature works.
[ ] I have added (or requested a maintainer to add) the necessary Breaking* or New Feature labels where relevant.
[ ] I have done my best to ensure that my PR adheres to the Fuel Labs Code Review Standards.
[ ] I have requested a review from the relevant team or maintainers.

Also will keep this waiting, won't merge right away because I need a release for fuel.nix stuff with #4750 merged. I want to omit releasing a version with this (before updating the UX). So my current plan looks like:

Merge #4750
Cut a sway repo release
Merge this one
Merge UX improvements
Cut another release

So that if anything comes up (very bad bugs with this etc) we do not block fuel.nix resolutions any longer, and have a release  that both works for fuel.nix and without this changes.

Instead of typing forc-deploy --random-salt can --random-salt just be on by default? Perhaps this is already the case was just confused that it was in the description.

Opened more granular issues such as:

#4759
#4758
#4757
#4756
#4754


Rename repo and NPM package

To keep up with best practices for naming conventions of Angular libraries this repo and the NPM package should be renamed.  This will also keep it in line with the angular-esri-loader module that was recently changed.
Right now I'm thinking that the new name should be angular-esri4-components.  It seems to be that esri4 still needs to be specified since this package is incompatible with ArcGIS API for JavaScript v3 and chances are that it would be incompatible with a v5 if that were to ever exist.
Thoughts?
@tomwayson

That's an interesting question. My gut response is yes, keep the 4.
However, having been on the receiving end of the whole "it's just anuglar" fiasco, I can imagine a future where you'll regret having the 4 in there.
Renaming things is hard. I would think you'd want to avoid doing it again. I'd say drop the 4 too and just make it clear in the readme that it only supports v4. If someone really wants to write a component lib for v3, they can use angular-esri3-components.

Changes that need to be made:

[] rename repo to angular-esri-components
[] publish<EMAIL_ADDRESS>[] update links

esri-loader
jsapi-resources
esri-angular-cli-example

[] update esri-angular-cli-example
[] update ng-cli-electron-esri

Great work!
I'll add the update to jsapi-resources to my existing PR over there for angular-esri-loader.

FYI - https://github.com/Esri/jsapi-resources/pull/73/commits/bb4cd5f163484111e1275a87c4b4b468519fb48d


fatal error: esp_crt_bundle.h: No such file or directory (IDFGH-9060)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
4.4.3
Operating System used.
Windows
How did you build your project?
Command line with Make
If you are using Windows, please specify command line type.
None
What is the expected behavior?
After updating ESP-IDF from version 4.4.1 to version 4.4.3, the project stopped building:  fatal error: esp_crt_bundle.h: No such file or directory
What is the actual behavior?
C:/PlatformIO/libs/clouds/reTgSend/src/reTgSend.cpp:63:12: fatal error: esp_crt_bundle.h: No such file or directory

Looking for esp_crt_bundle.h dependency? Check our library registry!

CLI  > platformio lib search "header:esp_crt_bundle.h"
Web  > https://registry.platformio.org/search?q=header:esp_crt_bundle.h

#include "esp_crt_bundle.h"
^~~~~~~~~~~~~~~~~~
compilation terminated.
*** [.pio\build\esp32dev\libdc6\reTgSend\reTgSend.o] Error 1
Steps to reproduce.
In PlatformIO
Update ESP-IDF from 4.4.1 to 4.4.3
Build or installation Logs.
No response
More Information.
No response

@kotyara12 Can you please confirm that you have CONFIG_MBEDTLS_CERTIFICATE_BUNDLE enabled in your sdkconfig?


ggml_tallocr_alloc: not enough space in the buffer

Hi,
First of all, thanks for your amazing project!
After some time I came back to playing with llama, but trying to work with gguf I got stuck with 7B version, as if I try to load anything bigger, I get:
ggml_tallocr_alloc: not enough space in the buffer (needed 442368, largest block available 290848) GGML_ASSERT: <PATH_TO_ggml-alloc.c>:116: !"not enough space in the buffer"
I tried on two different machines with heaps of RAM, and it just won't go away.
I've seen someone hitting this issue when trying to fine-tune localGPT and llama, and it helped them to modify the batch size, but I get it when I try to initialize it and get it to answer simple question.
Best regards,
Marian

@mariansiwiak @Aimless-Wonder does this also come up when trying to run the model with llama.cpp?

Actually I was using llamacpp through langchain (
from langchain.llms import LlamaCpp
...
llm = LlamaCpp(
...
)
I am only able to run 7b models

I have an identical situation as @Aimless-Wonder .

I was able to load the 13 b model by using llama.cpp without langchain, but now I can't call it in a chain object:
from llama_cpp import Llama
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate
...
llm=Llama(
model_path='//home/alex/VSC files/TaxEnv/SWHARDEN/models/llama-2-13b.Q4_K_M.gguf'
)
...
llm_chain = LLMChain(prompt=prompt, llm=llm)
...
output includes:
pydantic.v1.error_wrappers.ValidationError: 2 validation errors for LLMChain
llm
instance of Runnable expected (type=type_error.arbitrary_type; expected_arbitrary_type=Runnable)
llm
instance of Runnable expected (type=type_error.arbitrary_type; expected_arbitrary_type=Runnable)

I had this issue when Chroma were running embedding. I am using mistral-7b-instruct-v0.1.Q5_K_S.gguf, which is 5Gb-ish in size, on CPU with plenty of RAM. On most recent versions setting bigger context size helped for me. Debug log stated that model was trained with 32k context size, while llama.cpp sets 512 by default. I increased it to 32k and error is gone.
BTW, downgrading also helped ) Hope it helps.

@mariansiwiak and @alazarchuk I believe I am running into the same issue with mistral-7b-instruct-v0.1.Q4_0.gguf
@alazarchuk how are you instantiating the model? Using langchain? can you reproduce the debug log here and the code you use to instantiate and call the model?

@i-am-mike-davis Here is an example https://github.com/alazarchuk/LangChain-Chat-with-Your-Data-Locally/blob/main/lecture_03/03_vectorstores_and_embeddings.ipynb. Scroll to Embeddings section.

Thanks @alazarchuk.
I resolved this by setting the context size to ~32k.

If someone still struggles, there is a solution:
https://github.com/langchain-ai/langchain/pull/10908

@mariansiwiak Hello. Can you tell me how to solve this problem?

Same problem.
Reproducing the problem:
Code from the test.py file:
from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.chains import LLMChain
from langchain.llms import LlamaCpp
from langchain.prompts import PromptTemplate

template = """Question: {question}

Answer: Ignore all previous instructions.
1. You are to provide clear, concise, and direct responses.
2. Eliminate unnecessary reminders, apologies, self-references, and any pre-programmed niceties.
3. Maintain a casual tone in your communication.
4. Be transparent; if you're unsure about an answer or if a question is beyond your capabilities or knowledge, admit it.

Translate the following text into Russian:
"""

prompt = PromptTemplate(template=template, input_variables=["question"])
# Callbacks support token-wise streaming
callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])

n_gpu_layers = 100
n_batch = 8
n_ctx = 4096

llm = LlamaCpp(
    model_path="./TEMP/ggml-model-q6-k.gguf",
    n_gpu_layers=n_gpu_layers,
    n_batch=n_batch,
    callback_manager=callback_manager,
    verbose=True,
)

llm_chain = LLMChain(prompt=prompt, llm=llm)
question = "How to tell if a customer segment is well segmented? In 3 bullet points."
llm_chain.run(question)

Run the test.py file with the following contents from bash:
$ python test.py

/home/artyom/.local/lib/python3.10/site-packages/langchain/llms/__init__.py:548: LangChainDeprecationWarning: Importing LLMs from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

`from langchain_community.llms import LlamaCpp`.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
ggml_init_cublas: GGML_CUDA_FORCE_MMQ:   no
ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes
ggml_init_cublas: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
llama_model_loader: loaded meta data with 24 key-value pairs and 435 tensors from ./TEMP/ggml-model-q6-k.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.name str              = LLaMA v2
llama_model_loader: - kv   2:                       llama.context_length u32              = 4096
llama_model_loader: - kv   3:                     llama.embedding_length u32              = 4096
llama_model_loader: - kv   4:                          llama.block_count u32              = 48
llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 14336
llama_model_loader: - kv   6:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv   7:                 llama.attention.head_count u32              = 32
llama_model_loader: - kv   8:              llama.attention.head_count_kv u32              = 8
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  10:                       llama.rope.freq_base f32              = 10000.000000
llama_model_loader: - kv  11:                          general.file_type u32              = 18
llama_model_loader: - kv  12:                       tokenizer.ggml.model str              = llama
llama_model_loader: - kv  13:                      tokenizer.ggml.tokens arr[str,32000]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...
llama_model_loader: - kv  14:                      tokenizer.ggml.scores arr[f32,32000]   = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv  15:                  tokenizer.ggml.token_type arr[i32,32000]   = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
llama_model_loader: - kv  16:                tokenizer.ggml.bos_token_id u32              = 1
llama_model_loader: - kv  17:                tokenizer.ggml.eos_token_id u32              = 2
llama_model_loader: - kv  18:            tokenizer.ggml.unknown_token_id u32              = 0
llama_model_loader: - kv  19:            tokenizer.ggml.padding_token_id u32              = 2
llama_model_loader: - kv  20:               tokenizer.ggml.add_bos_token bool             = true
llama_model_loader: - kv  21:               tokenizer.ggml.add_eos_token bool             = false
llama_model_loader: - kv  22:                    tokenizer.chat_template str              = {% for message in messages %}{% if me...
llama_model_loader: - kv  23:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:   97 tensors
llama_model_loader: - type q6_K:  338 tensors
llm_load_vocab: special tokens definition check successful ( 259/32000 ).
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 32000
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 4096
llm_load_print_meta: n_embd           = 4096
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 8
llm_load_print_meta: n_layer          = 48
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 4
llm_load_print_meta: n_embd_k_gqa     = 1024
llm_load_print_meta: n_embd_v_gqa     = 1024
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff             = 14336
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx  = 4096
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: model type       = 34B
llm_load_print_meta: model ftype      = Q6_K
llm_load_print_meta: model params     = 10.73 B
llm_load_print_meta: model size       = 8.20 GiB (6.56 BPW)
llm_load_print_meta: general.name     = LLaMA v2
llm_load_print_meta: BOS token        = 1 '<s>'
llm_load_print_meta: EOS token        = 2 '</s>'
llm_load_print_meta: UNK token        = 0 '<unk>'
llm_load_print_meta: PAD token        = 2 '</s>'
llm_load_print_meta: LF token         = 13 '<0x0A>'
llm_load_tensors: ggml ctx size =    0.33 MiB
llm_load_tensors: offloading 48 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 49/49 layers to GPU
llm_load_tensors:        CPU buffer size =   102.54 MiB
llm_load_tensors:      CUDA0 buffer size =  8294.05 MiB
....................................................................................................
llama_new_context_with_model: n_ctx      = 512
llama_new_context_with_model: freq_base  = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init:      CUDA0 KV buffer size =    96.00 MiB
llama_new_context_with_model: KV self size  =   96.00 MiB, K (f16):   48.00 MiB, V (f16):   48.00 MiB
llama_new_context_with_model: graph splits (measure): 3
llama_new_context_with_model:      CUDA0 compute buffer size =     1.14 MiB
llama_new_context_with_model:  CUDA_Host compute buffer size =     0.14 MiB
AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 |
Model metadata: {'tokenizer.chat_template': "{% for message in messages %}{% if message['role'] == 'system' %}{% if message['content']%}{{'### System:\n' + message['content']+'\n\n'}}{% endif %}{% elif message['role'] == 'user' %}{{'### User:\n' + message['content']+'\n\n'}}{% elif message['role'] == 'assistant' %}{{'### Assistant:\n'  + message['content']}}{% endif %}{% if loop.last and add_generation_prompt %}{{ '### Assistant:\n' }}{% endif %}{% endfor %}", 'tokenizer.ggml.add_eos_token': 'false', 'tokenizer.ggml.padding_token_id': '2', 'tokenizer.ggml.unknown_token_id': '0', 'tokenizer.ggml.eos_token_id': '2', 'general.architecture': 'llama', 'llama.rope.freq_base': '10000.000000', 'llama.context_length': '4096', 'general.name': 'LLaMA v2', 'tokenizer.ggml.add_bos_token': 'true', 'llama.embedding_length': '4096', 'llama.feed_forward_length': '14336', 'llama.attention.layer_norm_rms_epsilon': '0.000010', 'llama.rope.dimension_count': '128', 'tokenizer.ggml.bos_token_id': '1', 'llama.attention.head_count': '32', 'llama.block_count': '48', 'llama.attention.head_count_kv': '8', 'general.quantization_version': '2', 'tokenizer.ggml.model': 'llama', 'general.file_type': '18'}
/home/artyom/.local/lib/python3.10/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The function `run` was deprecated in LangChain 0.1.0 and will be removed in 0.2.0. Use invoke instead.
  warn_deprecated(
ggml_tallocr_alloc: not enough space in the buffer (needed 458752, largest block available 339968)
GGML_ASSERT: /tmp/pip-install-rniw_as1/llama-cpp-python_cb8d30060705490ea24801c108882464/vendor/llama.cpp/ggml-alloc.c:114: !"not enough space in the buffer"
Aborted

If anyone knows how to solve the problem or if I'm wrong somewhere, please advise how to solve it. Thank you!

@blademoon : You need to pass in the n_ctx to set it.
As you can see from the output:
llama_new_context_with_model: n_ctx      = 512

it still uses 512.
This should help:
n_ctx = 4096

llm = LlamaCpp(
    model_path="./TEMP/ggml-model-q6-k.gguf",
    n_gpu_layers=n_gpu_layers,
    n_batch=n_batch,
    n_ctx=n_ctx,
    callback_manager=callback_manager,
    verbose=True,
)

@lnist Thank you. I'll test it and return with feedback.

@lnist Seems to have worked. Thank you!


Locations showing unavailable/fttn only, but nbnco site showing as under construction (selective properties in same area)

As an example, in marian QLD.
2 RIDGWAY COURT MARIAN 4753 Current tech: FTTN Upgrade available: No
4 RIDGWAY COURT MARIAN 4753 Current tech: FTTN Upgrade available: Soon Tech Change Status: Committed Program Type: On-Demand N2P SDU/MDU Simple Target Eligibility Quarter: Jun 2024
Im not sure if this is a BUG, Something that just needs another sweep to update etc.
Have noticed with a few properties

This project gets the data straight from the same API that the NBN website uses.  It takes about 3 weeks to complete a full refresh of all 15117 suburbs.
Marian, QLD was last refresh about 10 days ago, so is likely to be refreshed again in about another 10-12 days.
https://lukeprior.github.io/nbn-upgrade-map/?suburb=marian&state=QLD&commit=main

I don't think there's any bug here?  (unless something failed to refresh, or the data is wrong (after refresh))

I have manually processed this suburb:
❯ ./code/main.py --suburb Marian --state QLD -H address_principals.sqlite
2023-11-16 08:54:10,543 INFO MainThread Checking for externally updated geojson results...
2023-11-16 08:54:15,956 INFO MainThread ...done
2023-11-16 08:54:15,987 INFO MainThread Selecting explicit Marian, QLD
2023-11-16 08:54:15,988 INFO MainThread Processing Marian, QLD
2023-11-16 08:54:15,988 INFO MainThread Fetching all addresses for Marian, QLD
2023-11-16 08:54:15,988 INFO MainThread Executing query:
            SELECT gnaf_pid, address, postcode, latitude, longitude
            FROM address_principals
            WHERE locality_name = ? AND state = ?
            LIMIT 100000
2023-11-16 08:54:16,154 INFO MainThread Fetched 1816 addresses from database
2023-11-16 08:54:16,159 INFO MainThread Loaded 1809 addresses from output file
2023-11-16 08:54:16,159 INFO MainThread Submitting 1816 requests to add NBNco data...
2023-11-16 08:54:20,406 INFO nbn_9 Completed 16 requests
2023-11-16 08:54:28,438 WARNING nbn_8 No suggestions for DUPLEX 1, 42 STAYTS ROAD MARIAN 4753
2023-11-16 08:54:28,668 WARNING nbn_8 No suggestions for DUPLEX 2, 42 STAYTS ROAD MARIAN 4753
2023-11-16 08:54:28,914 WARNING nbn_8 No suggestions for HOUSE 1, 38 STAYTS ROAD MARIAN 4753
2023-11-16 08:54:29,159 WARNING nbn_8 No suggestions for HOUSE 2, 38 STAYTS ROAD MARIAN 4753
2023-11-16 08:54:38,450 WARNING nbn_8 No suggestions for LOT 151 STAYTS ROAD MARIAN 4753
2023-11-16 08:54:41,391 WARNING nbn_8 No suggestions for LOT 2 J SHEWS ROAD MARIAN 4753
2023-11-16 08:54:49,901 WARNING nbn_8 No suggestions for LOT 37 STAYTS ROAD MARIAN 4753
2023-11-16 08:54:54,963 INFO nbn_8 Completed 1056 requests
2023-11-16 08:55:14,966 INFO nbn_5 Completed 1536 requests
2023-11-16 08:55:20,286 INFO nbn_2 Completed 1696 requests
2023-11-16 08:55:22,049 INFO nbn_3 Completed 1716 requests
2023-11-16 08:55:22,095 INFO nbn_7 Completed 1736 requests
2023-11-16 08:55:22,150 INFO nbn_6 Completed 1756 requests
2023-11-16 08:55:22,337 INFO nbn_1 Completed 1776 requests
2023-11-16 08:55:24,206 INFO nbn_0 Completed 1796 requests
2023-11-16 08:55:25,134 INFO nbn_4 Completed 1816 requests
2023-11-16 08:55:25,141 INFO MainThread Completed. Tally of tech types: {'WIRELESS': 488, 'FTTN': 1058, 'FTTP': 236, 'NULL': 12, 'SATELLITE': 15, None: 7}
2023-11-16 08:55:25,142 INFO MainThread Location ID types: {'LOC': 1599, 'Other': 210, 'None': 7}
2023-11-16 08:55:25,142 INFO MainThread Writing results to results/QLD/marian.geojson
2023-11-16 08:55:25,485 INFO MainThread Updating progress.json

Which are included in this PR https://github.com/LukePrior/nbn-upgrade-map/pull/314.  Once approved, you will see newer data (and will go to back of the queue for refresh)

You can see the changes from this refresh at https://github.com/LukePrior/nbn-upgrade-map/pull/314/files

You can see the changes from this refresh at https://github.com/LukePrior/nbn-upgrade-map/pull/314/files

Thanks for that, i wasnt quite sur e 100% as its out of my understanding, but that makes more sense its just a delay in updating vs a bug. I assumed the data would have all come down together as a suburb update so thought that it would be "all or nothing" vs selective. I appreciate you taking the time to do that. Will close this as "me being dumb" haha.

Each suburb is updated "all at once", but even NBN update their data progressively within suburbs, as they release more and more addresses.

Each suburb is updated "all at once", but even NBN update their data progressively within suburbs, as they release more and more addresses.

Yep, probably badly worded in my response, but yes, thats where i had assumed wrongly that it was grouped as suburb vs individuals or smaller lots etc.

You can see the changes from this refresh at https://github.com/LukePrior/nbn-upgrade-map/pull/314/files
Neither of the two addresses you quoted has changed.
NBNco has problems for "2 RIDGWAY COURT MARIAN 4753" with no direct match. The first suggestion also shows FTTN, no upgrade.

While "4 RIDGWAY COURT MARIAN 4753" does even worse

Oh, in this one you have a typo missing the "e" in Ridgeway, but on nbn co site ive found it only works properly with the "lots" included.


Fix grid data generation

An hdf5 file with grids should be structured like the following (example of one data point):

residue xxx Group

edge_features Group

features Datasets

...
mapped_features Group

features Datasets

While is it like the following:

So the mapped_features Group contains Groups instead of directly containing Datasets with the features. We need to allineate mapped_features Group to the others.

[x] Modify the code to generate Datasets (directly) in mapped_features instead of Groups
[x] Regenerate test file for grids (deeprank-core/tests/data/hdf5/1ATN_ppi.hdf5)

Additionally, by generating an example hdf5 file with the following script:
from pathlib import Path
import pkg_resources as pkg
PATH_DEEPRANK_CORE = Path(pkg.resource_filename("deeprankcore", ""))
ROOT = PATH_DEEPRANK_CORE.parent
PATH_TEST = ROOT / "tests"
from deeprankcore.query import (
    QueryCollection,
    ProteinProteinInterfaceResidueQuery)
from deeprankcore.tools.target import compute_targets
from deeprankcore.utils.grid import MapMethod, GridSettings
import glob
import os
import re
import sys
import h5py
import numpy as np
import pandas as pd

points_counts = [10, 10, 10]
grid_sizes = [30.0, 30.0, 30.0]

ref_path = str(PATH_TEST / "data/ref/1ATN/1ATN.pdb")
pssm_path1 = str(PATH_TEST / "data/pssm/1ATN/1ATN.A.pdb.pssm")
pssm_path2 = str(PATH_TEST / "data/pssm/1ATN/1ATN.B.pdb.pssm")
chain_id1 = "A"
chain_id2 = "B"
pdb_paths = [
    str(PATH_TEST / "data/pdb/1ATN/1ATN_1w.pdb"),
    str(PATH_TEST / "data/pdb/1ATN/1ATN_2w.pdb"),
    str(PATH_TEST / "data/pdb/1ATN/1ATN_3w.pdb"),
    str(PATH_TEST / "data/pdb/1ATN/1ATN_4w.pdb")]

queries = QueryCollection()

for pdb_path in pdb_paths:
    # Append data points
    targets = compute_targets(pdb_path, ref_path)
    queries.add(ProteinProteinInterfaceResidueQuery(
        pdb_path = pdb_path,
        chain_id1 = chain_id1,
        chain_id2 = chain_id2,
        targets = targets,
        pssm_paths = {
            chain_id1: pssm_path1,
            chain_id2: pssm_path2
        }
    ))

# Generate graphs and save them in hdf5 files
output_paths = queries.process(
    cpu_count=1,
    prefix='1ATN_ppi_grid',
    grid_settings=GridSettings(points_counts, grid_sizes),
    grid_map_method=MapMethod.FAST_GAUSSIAN)

I can't open the Datasets contained in the Groups in mapped_features parent Group. I get the following error for all the features:

I'm not sure what software you're using. When I run your script, I can retrieve the output from the file using h5py.

hdfview doesn't work on my linux desktop. When I run your script, I can retrieve the output from the file using h5py.

Okay I'll run it again after having merged your PR, and let's see what happens.

This may be related:
https://forum.hdfgroup.org/t/cannot-read-hdf5-data/6714

@gcroci2 I installed the latest hdfview and now I can reproduce your problem.
It seems uncompressed unchunked datasets can be viewed. But chunked compressed grid data cannot.

@gcroci2 I installed the latest hdfview and now I can reproduce your problem.
It seems uncompressed unchunked datasets can be viewed. But chunked compressed grid data cannot. I do think compression is necessary. Grid data is quite sparse.

Are the grid data compressed in original deeprank?

Are the grid data compressed in original deeprank?
No, they use a different system to remove sparsity:
https://github.com/DeepRank/deeprank/blob/master/deeprank/tools/sparse.py

That's why in the original deeprank, data is stored under ../value and ../index .

https://github.com/DeepRank/deeprank/blob/master/deeprank/tools/sparse.py

I see, but here we didn't have /index in the mapped_features groups, only compressed values. I also think is fine as they are, so compressed.


add possible solution

@savbace You are exactly correct on both count. Very clean code and I appreciate the comments. For now I am going to leave this PR open for other to see (at some point I am going to set this up with a separate branch showing the solution).

Great! And thank you, @Keboo, for the nice brain teaser.

Closing this pull request. I have created a new solution branch and merged your solution into it. I have also cited you as the source in the updated README.


Localization.StarterWeb example in 'Set The Culture Programmatically' missing a step

It took me a while to figure out why, but when following the example code for setting the culture programmatically, it never picks up the IRequestCultureFeature from the Context.Features.Get<IRequestCultureFeature>(). The sample code is missing app.UseRequestLocalization(); to enable this to work. Would be worth adding to avoid some of the confusion I faced coming to this topic for the first time.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d1833676-ff1b-c587-cc9c-81afffa8ef68
Version Independent ID: ad989395-3f49-4d55-521e-cf6fe56281ee
Content: Globalization and localization in ASP.NET Core
Content Source: aspnetcore/fundamentals/localization.md
Product: aspnet-core
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

@hishamco please review.

@Issafalcon if you mean Localization.StarterWeb that is available in the entropy repo the app.UseRequestLocalization() is already there at this line  https://github.com/aspnet/Entropy/blob/master/samples/Localization.StarterWeb/Startup.cs#L109?!! furthermore you can have a look to my LanguageSwitcherTagHelper which showing the same use case for switching & setting the languages

@hishamco - I do apologise. You're right, the entropy repo shows the code. And indeed, the doc shows the UseRequestLocalization further up the page. Thank you for taking the time to respond. Next time I'll spend more time looking at the rest of the docs / sample code rather than focusing in on just one part of them!

You're welcome, any be free to ask anytime

@Rick-Anderson I think we need to close this, because there's no further action needed here


Bug: executed other match group route

PHP Version
8.3
CodeIgniter4 Version
4.5.5
CodeIgniter4 Installation Method
Composer (using codeigniter4/appstarter)
Which operating systems have you tested for this bug?
Windows
Which server did you use?
apache
Database
No response
What happened?
Call other match ainto Routes
Steps to Reproduce
My Routes are write this kind
//app/config/Route.php

$routes->group('admin/(:any)',["filter" => "authGuard"], function ($routes) {

    $routes->group('course/inquiry', function ($routes) {
        $routes->group('reminder', function ($routes) {
            First Route =>   //$routes->get('list', 'Admin\CourseInquiry::reminderList');

        });
    });

    $routes->group('reminder', function ($routes) {
        Second Route =>$routes->get('list', 'Admin\Reminder::index');
    });
});

My Url is =>http://domain.com/index.php/admin/521d7941a1f28640069792fbb7dedd8e/course/inquiry/reminder/list
if We comment first one route of this route its execute second one Route .
Expected Output
it should be show route not found but its second one route are executed. that route are outside of that group and that route are different and outside side of group. If we want to execute first one route un comment that work perfectly.
Anything else?
No response

Not a bug. Please check what :any placeholder stands for: https://codeigniter.com/user_guide/incoming/routing.html#placeholders

let's you explain what happen whenever i am calling bellow url.
My url is "http://domain.com/index.php/admin/521d7941a1f28640069792fbb7dedd8e/course/inquiry/reminder/list"
here we are devided above url in 3 part

"http://domain.com/index.php/"
"admin/521d7941a1f28640069792fbb7dedd8e/"
"/course/inquiry/reminder/list"

and my routes are there
First Group =>
$routes->group('admin/(:any)',function ($routes) {
Second Group=>
   $routes->group('course/inquiry', function ($routes) {
           inside first Group=>
	$routes->group('reminder', function ($routes) {
		$routes->get('list', 'Admin\CourseInquiry::reminderList');
	});
});

   Third Group=>
$routes->group('reminder', function ($routes) {
	$routes->get('list', 'Admin\Reminder::index');
});

});
when we enter url on addressbar
first routes are enter because we added wild card(:any) so this path enter my first route of group like in this
$routes->group('admin/(:any)',function ($routes) {
});
then it should be execute 2 group of my route because i am defined route is   "/course/inquiry/reminder/list"
$routes->group('course/inquiry', function ($routes) {
$routes->group('reminder', function ($routes) {
$routes->get('list', 'Admin\CourseInquiry::reminderList');
});
});
it's executed perfectly but whenever i am committed above route of group which is  "$routes->get('list', 'Admin\CourseInquiry::reminderList'); " then it will executed third one routes.
$routes->group('reminder', function ($routes) {
$routes->get('list', 'Admin\Reminder::index');
});

Please use code blocks when you paste code samples, because it's hard to read.
In your initial report, you wrote:

if We comment first one route of this route its execute second one Route .... it should be show route not found but its second one route are executed.

The result is correct, because (:any) will match multiple segments, thereforce, the second route is correct match.
If you want it to work differently, you have to use (:segment) placeholder instead or structure your routes differently.


GEODE-8179: gfsh query cmd returns incorrect results if '=' sign is missing

@davebarnes97 please take a look. As we cannot upgrade spring shell version I agreed with @pivotal-eshu to add a note in the --query option.

Im going to close this PR by the moment, I want to check the code again. I tested with a simple spring-shell 1.2.0 app trying to confirm the problem exists there, but I saw that the = sign between the attribute name and value seems not been accepted. Thanks for the suggestion anyway @agingade


Doesn't work with Unreal Engine 4.1

Got error while compiling new version of Unreal Engine under Linux. The error is:
In file included from /home/shiko/Soft/UnrealEngine-promoted/Engine/Plugins/Developer/QtCreatorSourceCodeAccess/Intermediate/Build/Linux/x86_64-unknown-linux-gnu/UE4Editor/Development/QtCreatorSourceCodeAccess/Module.QtCreatorSourceCodeAccess.cpp:2:
In file included from /home/shiko/Soft/UnrealEngine-promoted/Engine/Plugins/Developer/QtCreatorSourceCodeAccess/Source/QtCreatorSourceCodeAccess/Private/QtCreatorSourceCodeAccessModule.cpp:25:
/home/shiko/Soft/UnrealEngine-promoted/Engine/Plugins/Developer/QtCreatorSourceCodeAccess/Source/QtCreatorSourceCodeAccess/Private/QtCreatorSourceCodeAccessModule.h:36:31: error: field type 'FQtCreatorSourceCodeAccessor' is an abstract class
Workaround: add new function member to FQtCreatorSourceCodeAccessor class:
virtual void RefreshAvailability() override; //this into QtCreatorSourceCodeAccessor.h file
void FQtCreatorSourceCodeAccessor::RefreshAvailability(){} /*this line into QtCreatorSourceCodeAccessor.cpp file */
I'm not sure if the function should be empty but at least this allows the UnrealEditor to compile.

There is actually a similar change made for the other plugin https://github.com/fire/SensibleEditorSourceCodeAccess/pull/9 Will investigate.

Please try latest.

It still doesn't work, but shikolad's fix made it compile and work

Try newest version, if it doesn't work open a bug.


Possibility to list all ongoing trainings & kill

Hi,
I am using the Training web API to get a new model. I have started separate processes with different input files. However, now I would like to kill some of them. Would it be possible to kill a training process in docker using its token? I have already tried to delete my container and re-create it hoping to see not any training process. But when I check the status of a previously started training process (curl -v -d "token=PdchtN8Cyv" localhost:8070/api/trainingResult), I still see that its status is 'ongoing'.
Thanks in advance!

Hi @malee1382 !
It's a very good point and I will see how to implement it reliably.
If I remember well, there would be two points:

kill a training process which is launched in a java thread
update the training task status correctly after an interrupted or failing training tasks


[BUG] Guest agent install configuration is covered

Describe the bug

The guest agent install configuration will be covered if users check the box for installing the guest agent before choosing a cloud-init template.

To Reproduce
Steps to reproduce the behavior:

Go to the create Virtual Machine page.
Go to the advanced options tab.
Check the box for installing the guest agent.
Choose the cloud-init template.

Expected behavior

The guest agent install configuration is always not be covered.
Support bundle

Environment:

Harvester ISO version: v0.2.0
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630): Baremetal with Dell PowerEdge R630

Additional context
Add any other context about the problem here.

There is a PR related to this issue. Let's check if it has been fixed yet. If not, we can punt it to v1.0.0.

After checking, this issue still exists on v0.3.0-rc1.
if users check the box "installing guest agent" before choosing a predefined user data template.
The guest agent install configuration will be covered
For replication steps please refer to short video clip
Guest_agent_config_covered

Move to v1.0.0 due to validation failed in v0.3.0.

the above UX is looking good to me, please feel free to close it @TachunLin .

Close this issue since we have finalized agreement to the UX design


Evaluation of well position effect

@Zitong-Chen-16 @shntnu @AnneCarpenter : the concerning well position analysis results. Would love to hear your thoughts!
Background
The team generated a few batches that contain many repeated controls to try understand well position effects. There are two batches, each with four plates. Each plate contains 48 wells with ALK_REF, and 48 well with ALK_VAR (referred to as "REF" and "VAR" from here on). The 'ALK' locus was chosen because it is a positive protein mis-localization control.
The objective here was to determine how well our production data analysis strategy detects phenotypic effects (morphological/localization differences between REF and VAR) compared to technical well position effects.
Our current method for calling variants with significant phenotypic effects is to train a binary classifier to distinguish a given VAR from its WT using three plates within a batch, and to test on the fourth plate. We measure success using an F1 score. Since we expect there to be some well position effect, we also train binary classifiers to differentiate cells from pairs of repeated control wells (the well position null). The REF-VAR F1 scores are compared to the distribution of control F1 scores to decide which pairs of REF-VAR wells outperform the well position null. This well position null is imperfect, because repeated controls are usually closer together than REF-VAR pairs (need to confirm this). Therefore, we did this larger analysis described here.
Methods
Features were split into 3 sets: protein features, non-protein fluorescent features (3 channels), and brightfield features (3 channels). Considering the 48 REF wells and 48 VAR wells, we define every possible pair of two well positions: REF-REF pairs (48 choose 2 = 1128), VAR-VAR pairs (48 choose 2 = 1128), and REF-VAR pairs (48*48 = 2304).
For each feature set (protein, non-protein, brightfield) and comparison (REF-REF, VAR-VAR, REF-VAR) we train a binary XGBoost classifier with three plates from one batch and test with the fourth plate from that batch. We also train one null for reference - the REF-REF pairs are used to train another classifier after shuffling the well position labels of the training data. The F1 score and feature importance scores are saved from each classifier.
Since this is all quite computationally intensive, this analysis considers only the four plates in batch 4 (ignoring batch 6) and only looks at one training & test set (there are 4 possible sets, one with each of the 4 plates as the test plate).
Results
Figure 1: the distribution of F1 scores across all classifiers, faceted by comparison type and feature set.

The takeaways here:

The distribution of F1 scores are virtually identical between REF-VAR, REF-REF, and VAR-VAR pairs
The brightfield features are best at predicting well position, next best are non-protein features, and next protein features
The null is centered on 0.5 and has a relatively narrow spread. For both the protein and non-protein fluorescent channels, there are classifiers that do significantly better and significantly worse than the NULL. Sam and I have talked through a potential explanation for this.
The distributions of protein and non-protein F1 scores unfortunately look suspiciously similar to the distribution of F1 scores in our production classifiers from this notebook (comparing all REF-VAR pairs across all loci in our real data) - look at the orange 'allele' distributions and compare to the protein and non-protein distribution shapes above:

Figure 2: a comparison of feature importance scores across comparison types. Feature importance (FIP) scores were ranked for each classifier (1 = best, highest = worst) and averaged across all classifiers with the same comparison type and feature set. Here are pairwise scatterplots of mean FIP scores from the protein feature classifiers for all comparison types. Each point corresponds to a single feature.

Key takeaways:

The classifiers are learning to use the same features regardless of whether we are looking at REF-REF, VAR-VAR, or REF-VAR pairs of wells. We appear to always be learning well position and not morphology (😱)
The null prioritizes random features (as expected from a shuffled null)
These takeways are the same for the non-protein and brightfield classifiers

Figure 3: a scatterplot comparing the F1 score for the classifier for each pair of wells to the physical distance between those two wells on the plate.

Key takeways:

Only the protein feature REF-REF, VAR-VAR, and REF-VAR classifiers have a significant correlation between F1 score and physical distance (corr r = 0.16, 0.22, 0.24, corr p-value = 10-8, 10-14, 10-35 respectively).

Updating this for our records: this is solved for the protein channel ('localization') but not for the non-protein channel ('morphology'). I will open a new issue to discuss the non-protein classifiers.
The key step was removing the well position effect correction in the pre-processing pipeline. We ported this over from the JUMP pipeline where the mean of each feature for each well position is subtracted from the data. For JUMP this is fine because there are many, many platemaps and so if there is any well position pattern, we are highly confident that it is technical. However, for VarChamp there are only ~5 platemaps and so by deleting the well position pattern, we are actually deleting a ton of the phenotypic signal. Removing this step and reprocessing the control scramble data shows separation between the WT-VAR and CTRL-CTRL distributions for the protein data for both the ALK WT-VAR and LPAR1-MAPK9 comparisons:
LPAR1-MAPK9 - protein channel

ALK WT-VAR - protein channel

Notebook with these plots is here.


Feature or documentation request: null, undefined and {} in the JS API

I was able to determine through trial and error that:

When there are no URL query parameters, jatos.urlQueryParameters is {}
When the batch JSON input is empty, jatos.batchJsonInput is null
When the key a is not found in the batch session data, jatos.batchSession.find("/a") is undefined

I think this and other similar cases should be documented clearly (I can do a PR), but I was also wondering, could it be changed to be more consistent? I was particularly surprised by the second one, I would have expected {}. {} is convenient because you can call Object.hasOwn on it.

I agree that those cases should be documented more clearly. I would appreciate any PR on https://github.com/JATOS/JATOS_docs or/and https://github.com/JATOS/JATOS/blob/master/modules/publix/public/javascripts/jatos.js.
Then about changing the actual return values I think we can't / it wouldn't make sense:

jatos.urlQueryParameters - This {} means that there potentially can be some parameters but in this study run there were none. I wouldn't use null or undefined here since it has more the meaning that there isn't even the possibility of parameters.
jatos.batchJsonInput - This is literally the value of the field in the batch properties that the user put there. If the user set {a:1} it's {a:1}, if they set {} it's {} and if they leave the field empty it's null. I actually think null is the proper value in this case since it means that nothing was assigned. One could argue that the empty string "" would be a better fit but not {} since it has the meaning that the user put {} in the JSON input field.
3.jatos.batchSession.find("/a") - The value actual has to be undefined since anything else, e.g. a null, would mean that the field exists in the batch session and it just has the value null assigned to it. The proper response to a non-existing field is undefined. Beside this I couldn't change the response since it's done by a library (https://github.com/alexeykuzmin/jsonpointer.js).

Continued at https://github.com/JATOS/JATOS/pull/277.

This has been solved by #277 and https://github.com/JATOS/JATOS_docs/pull/17.


Integrate Wallbox datapoints

There are more Variables in the vars.html according to Wallbox informations.
It looks liked, that the system supports up to 4 Wallboxes. In this case there are 4 indizies (0,1,2,3).
Maybe it is possible to build up a structure in this way:
senec.0.WALLBOX.0.variables => for Wallbox 1
senec.0.WALLBOX.1.variables => for Wallbox 2
etc.
Here are the Variables from the section "Wallbox" from vars.html
Section		Variable						Index
====
WALLBOX	ADDITIONAL_ERROR				0
WALLBOX	ADDITIONAL_ERROR				1
WALLBOX	ADDITIONAL_ERROR				2
WALLBOX	ADDITIONAL_ERROR				3
WALLBOX	ALLOW_INTERCHARGE				0
WALLBOX	APPARENT_CHARGING_POWER			0
WALLBOX	APPARENT_CHARGING_POWER			1
WALLBOX	APPARENT_CHARGING_POWER			2
WALLBOX	APPARENT_CHARGING_POWER			3
WALLBOX	CS_ENABLED						0
WALLBOX	CS_ENABLED						1
WALLBOX	CS_ENABLED						2
WALLBOX	CS_ENABLED						3
WALLBOX	DETECTION_MODE					0
WALLBOX	EV_CONNECTED					0
WALLBOX	EV_CONNECTED					1
WALLBOX	EV_CONNECTED					2
WALLBOX	EV_CONNECTED					3
WALLBOX	HW_TYPE							0
WALLBOX	HW_TYPE							1
WALLBOX	HW_TYPE							2
WALLBOX	HW_TYPE							3
WALLBOX	ID								0
WALLBOX	ID								1
WALLBOX	ID	2
WALLBOX	ID	3
WALLBOX	L1_CHARGING_CURRENT	0
WALLBOX	L1_CHARGING_CURRENT	1
WALLBOX	L1_CHARGING_CURRENT	2
WALLBOX	L1_CHARGING_CURRENT	3
WALLBOX	L1_USED	0
WALLBOX	L1_USED	1
WALLBOX	L1_USED	2
WALLBOX	L1_USED	3
WALLBOX	L2_CHARGING_CURRENT	0
WALLBOX	L2_CHARGING_CURRENT	1
WALLBOX	L2_CHARGING_CURRENT	2
WALLBOX	L2_CHARGING_CURRENT	3
WALLBOX	L2_USED	0
WALLBOX	L2_USED	1
WALLBOX	L2_USED	2
WALLBOX	L2_USED	3
WALLBOX	L3_CHARGING_CURRENT	0
WALLBOX	L3_CHARGING_CURRENT	1
WALLBOX	L3_CHARGING_CURRENT	2
WALLBOX	L3_CHARGING_CURRENT	3
WALLBOX	L3_USED	0
WALLBOX	L3_USED	1
WALLBOX	L3_USED	2
WALLBOX	L3_USED	3
WALLBOX	LOAD_IMBALANCE_DETECTED	0
WALLBOX	LOAD_IMBALANCE_DETECTED	1
WALLBOX	LOAD_IMBALANCE_DETECTED	2
WALLBOX	LOAD_IMBALANCE_DETECTED	3
WALLBOX	LOAD_IMBALANCE_ENABLED	0
WALLBOX	LOAD_IMBALANCE_ENABLED	1
WALLBOX	LOAD_IMBALANCE_ENABLED	2
WALLBOX	LOAD_IMBALANCE_ENABLED	3
WALLBOX	MAJOR_REV	0
WALLBOX	MAJOR_REV	1
WALLBOX	MAJOR_REV	2
WALLBOX	MAJOR_REV	3
WALLBOX	MAX_CHARGING_CURRENT_DEFAULT	0
WALLBOX	MAX_CHARGING_CURRENT_DEFAULT	1
WALLBOX	MAX_CHARGING_CURRENT_DEFAULT	2
WALLBOX	MAX_CHARGING_CURRENT_DEFAULT	3
WALLBOX	MAX_CHARGING_CURRENT_IC	0
WALLBOX	MAX_CHARGING_CURRENT_IC	1
WALLBOX	MAX_CHARGING_CURRENT_IC	2
WALLBOX	MAX_CHARGING_CURRENT_IC	3
WALLBOX	MAX_CHARGING_CURRENT_ICMAX	0
WALLBOX	MAX_CHARGING_CURRENT_ICMAX	1
WALLBOX	MAX_CHARGING_CURRENT_ICMAX	2
WALLBOX	MAX_CHARGING_CURRENT_ICMAX	3
WALLBOX	MAX_CHARGING_CURRENT_RATED	0
WALLBOX	MAX_CHARGING_CURRENT_RATED	1
WALLBOX	MAX_CHARGING_CURRENT_RATED	2
WALLBOX	MAX_CHARGING_CURRENT_RATED	3
WALLBOX	MAX_PHASE_CURRENT_BY_GRID	0
WALLBOX	MAX_TOTAL_CURRENT_BY_GRID	0
WALLBOX	METER_ENABLED	0
WALLBOX	METER_ENABLED	1
WALLBOX	METER_ENABLED	2
WALLBOX	METER_ENABLED	3
WALLBOX	METHOD_EN1	0
WALLBOX	METHOD_EN1	1
WALLBOX	METHOD_EN1	2
WALLBOX	METHOD_EN1	3
WALLBOX	MINOR_REV	0
WALLBOX	MINOR_REV	1
WALLBOX	MINOR_REV	2
WALLBOX	MINOR_REV	3
WALLBOX	MIN_CHARGING_CURRENT	0
WALLBOX	MIN_CHARGING_CURRENT	1
WALLBOX	MIN_CHARGING_CURRENT	2
WALLBOX	MIN_CHARGING_CURRENT	3
WALLBOX	PHASES_USED	0
WALLBOX	PHASES_USED	1
WALLBOX	PHASES_USED	2
WALLBOX	PHASES_USED	3
WALLBOX	PROHIBIT_USAGE	0
WALLBOX	PROHIBIT_USAGE	1
WALLBOX	PROHIBIT_USAGE	2
WALLBOX	PROHIBIT_USAGE	3
WALLBOX	SERIAL_NUMBER	0
WALLBOX	SERIAL_NUMBER	1
WALLBOX	SERIAL_NUMBER	2
WALLBOX	SERIAL_NUMBER	3
WALLBOX	SERIAL_NUMBER_INTERNAL	0
WALLBOX	SERIAL_NUMBER_INTERNAL	1
WALLBOX	SERIAL_NUMBER_INTERNAL	2
WALLBOX	SERIAL_NUMBER_INTERNAL	3
WALLBOX	SET_ICMAX	0
WALLBOX	SET_ICMAX	1
WALLBOX	SET_ICMAX	2
WALLBOX	SET_ICMAX	3
WALLBOX	SET_IDEFAULT	0
WALLBOX	SET_IDEFAULT	1
WALLBOX	SET_IDEFAULT	2
WALLBOX	SET_IDEFAULT	3
WALLBOX	SMART_CHARGE_ACTIVE	0
WALLBOX	SMART_CHARGE_ACTIVE	1
WALLBOX	SMART_CHARGE_ACTIVE	2
WALLBOX	SMART_CHARGE_ACTIVE	3
WALLBOX	SOCKET_ENABLED	0
WALLBOX	SOCKET_ENABLED	1
WALLBOX	SOCKET_ENABLED	2
WALLBOX	SOCKET_ENABLED	3
WALLBOX	STATE	0
WALLBOX	STATE	1
WALLBOX	STATE	2
WALLBOX	STATE	3
WALLBOX	UID	0
WALLBOX	UID	1
WALLBOX	UID	2
WALLBOX	UID	3
WALLBOX	UTMP	0
WALLBOX	UTMP	1
WALLBOX	UTMP	2
WALLBOX	UTMP	3

If you implement these variables I can test it for you before rollout in iobroker.
Thanks
Michael

Added a couple of datapoints (including WALLBOX) to low_prio_pulling (for now).
You may want to update your adapter via github.
Please check for wrong / missing units. I don't have a SENEC Wallbox so I can only guess if I am right about units and measurements.

I installed v1.2.0 of the adapter. The Wallbox datapoints are looking great. The units are looking good, too.
I currently have only the Wallbox but car for loading on the Wallbox. End of May I will have a test car when I will test the Wallbox, too. Then I can check the units and measurements again. But maybe an other user has a Wallbox AND a car and can check the values ;-)

SET_ICMAX should be Ampere.

Ok - I don't see a SET_ICMAX Datapoint (but that could be due to not having a wallbox).
Able to supply the full path to the datapoint (with the copy-button) so I can make sure it gets Amps as unit.

OK .. here are the datapoints for the ICMAX:
senec.0.WALLBOX.SET_ICMAX.0 senec.0.WALLBOX.SET_ICMAX.1 senec.0.WALLBOX.SET_ICMAX.2 senec.0.WALLBOX.SET_ICMAX.3
I will close the issue. I will install the new version when it is in the repo. Maybe I will find some issues when I have the test car.
Thanks for the addition.

I have some additional informations about the wallbox. I have looked in the source-code were alle the states are defined (https://ip/js/DE-de.js). There I have found these informations / definitions for the wallbox:
/*wallbox_info.html*/ 	"lWallboxTitle0": "Wallbox 1:", 	"lWallboxTitle1": "Wallbox 2:", 	"lWallboxTitle2": "Wallbox 3:", 	"lWallboxTitle3": "Wallbox 4:", 	"lWallboxMaxCurrent0": "MAX. LADESTROM RESULTAT (A):", 	"lWallboxMaxCurrent1": "MAX. LADESTROM RESULTAT (A):", 	"lWallboxMaxCurrent2": "MAX. LADESTROM RESULTAT (A):", 	"lWallboxMaxCurrent3": "MAX. LADESTROM RESULTAT (A):", 	"lWallboxMaxCurrentRated0": "MAX. LADESTROM AB WERK (A):", 	"lWallboxMaxCurrentRated1": "MAX. LADESTROM AB WERK (A):", 	"lWallboxMaxCurrentRated2": "MAX. LADESTROM AB WERK (A):", 	"lWallboxMaxCurrentRated3": "MAX. LADESTROM AB WERK (A):", 	"lWallboxMaxCurrentDefault0": "MAX. LADESTROM INSTALLIERT (A):", 	"lWallboxMaxCurrentDefault1": "MAX. LADESTROM INSTALLIERT (A):", 	"lWallboxMaxCurrentDefault2": "MAX. LADESTROM INSTALLIERT (A):", 	"lWallboxMaxCurrentDefault3": "MAX. LADESTROM INSTALLIERT (A):", 	"lWallboxMaxCurrentMax0": "MAX. LADESTROM USER (A):", 	"lWallboxMaxCurrentMax1": "MAX. LADESTROM USER (A):", 	"lWallboxMaxCurrentMax2": "MAX. LADESTROM USER (A):", 	"lWallboxMaxCurrentMax3": "MAX. LADESTROM USER (A):", 	"lWallboxState0": "ZUSTAND:", 	"lWallboxState1": "ZUSTAND:", 	"lWallboxState2": "ZUSTAND:", 	"lWallboxState3": "ZUSTAND:", 	"lWallboxTotalPower0": "GESAMT-LEISTUNG (VA):", 	"lWallboxTotalPower1": "GESAMT-LEISTUNG (VA):", 	"lWallboxTotalPower2": "GESAMT-LEISTUNG (VA):", 	"lWallboxTotalPower3": "GESAMT-LEISTUNG (VA):", 	"lWallboxTemperature0": "TEMPERATUR:", 	"lWallboxTemperature1": "TEMPERATUR:", 	"lWallboxTemperature2": "TEMPERATUR:", 	"lWallboxTemperature3": "TEMPERATUR:", 	"lWallboxRevision0": "REVISION:", 	"lWallboxRevision1": "REVISION:", 	"lWallboxRevision2": "REVISION:", 	"lWallboxRevision3": "REVISION:", 	"lWallboxHardware0": "HARDWARE (pcba-Typ):", 	"lWallboxHardware1": "HARDWARE (pcba-Typ):", 	"lWallboxHardware2": "HARDWARE (pcba-Typ):", 	"lWallboxHardware3": "HARDWARE (pcba-Typ):", 	"lWallboxCurrent0": "LADESTROM JE PHASE (A):", 	"lWallboxCurrent1": "LADESTROM JE PHASE (A):", 	"lWallboxCurrent2": "LADESTROM JE PHASE (A):", 	"lWallboxCurrent3": "LADESTROM JE PHASE (A):", 	"lWallboxNoConfig": "Keine Wallbox konfiguriert!",
The "translation" for the datapoints senec.0.WALLBOX.STATE.0/1/2/3 (the datapoints are decimal, but the definition is in hex):
var wallbox_state_name = {     0xA1: "Warte auf E-Fahrzeug", (161)     0xB1: "E-Fahrzeug fragt Ladung an", (177)     0xB2: "E-Fahrzeug kann geladen werden", (178)     0xC2: "E-Fahrzeug lädt momentan", (194)     0xC3: "E-Fahrzeug lädt mit verringertem Strom (Fehler F16, F17)", (195)     0xC4: "E-Fahrzeug lädt mit verringertem Strom (Schieflast F15)", (196)     0xE0: "Wallbox deaktiviert durch EN1/Schlüsselschalter oder für Konfigurationsupdate", (224)     0xE1: "Test Production", (225)     0xE2: "Steuereinheit kann programmiert werden", (226)     0xE3: "Warte auf Kommunikation mit SENEC-Speicher", (227)     0xF1: "Unerwartet geschlossener Kontakt (verschweißt)", (241)     0xF2: "Interner Fehler", (242)     0xF3: "DC Fehlerstrom erkannt", (243)     0xF4: "Fehler Kommunikation", (244)     0xF5: "Sperren der Ladebuchse fehlgeschlagen", (245)     0xF6: "Kabelfehler", (246)     0xF7: "E-Fahrzeug meldet Übertemperatur", (247)     0xF8: "Kommunikationsfehler zum E-Fahrzeug", (248)     0xF9: "Überstrom erkannt", (249)     0xFA: "Temperaturgrenzwert überschritten", (250)     0xFB: "Unerwartet geöffneter Kontakt", (251)     0xFC: "Reserved State", (252)     0xFD: "Reserved State" (253) };
In my case the 0xA1 = 161 = "Warte auf E-Fahrzeug". I wrote the decimal values in brackets () behind the definition.
Maybe this helps.
And: It lookes like, that the data aren't pulled.

And: It lookes like, that the data aren't pulled.

The matter isn't about pulling them :-)
If the Datapoint shows up in iobroker it is sent via lala.cgi but that doesn't mean any data is supplied at all :-(
Unless you expect datapoints to get an "update" every time a request is sent to lala: Only datapoints with updates are updated.
So if they are 0 for now they won't get an update unless the value changes.
Problem with "human readable status texts" is that they are different for each language so that is something I need to find a feasible way to implement in the future.

Unless you expect datapoints to get an "update" every time a request is sent to lala: Only datapoints with updates are updated.
So if they are 0 for now they won't get an update unless the value changes.

I understand. But with restarting the instance the new values are updated. I will have a look, if the datapoints are getting updated when a car is loading on the wallbox. I will test it end of may when I have a test car.

Problem with "human readable status texts" is that they are different for each language so that is something I need to find a feasible way to implement in the future.

I admit, but you have the datapoint senec.0.ENERGY.STAT_STATE_Text. There you translate the state from senec.0.ENERGY.STAT_STATE or senec.0.ENERGY.STAT_STATE_DECODE into human readable state. Maybe you can integrate a similiar function and make a multi-language option as described in Issue#8

translation added see #18


Intents never recognized

**Rasa NLU version**:

(MoodEnv) mike@mike-thinks:~/Programing/Rasa/Moodbot$ pip list :
Package             Version
------------------- -----------
...
rasa-core           0.9.0
rasa-nlu            0.12.3
...

Operating system :
Linux 16.04
Content of model configuration file:
slots:
  location:
    type: text
  mood:
    type: text
  information:
    type: text
  adjective:
    type: text

entities:
 - location
 - mood
 - information
 - adjective

templates:
  utter_greet:
    - 'Hello! How can I help?'
  utter_goodbye:
    - 'Talk to you later.'
    - 'Bye bye :('
  utter_ask_location:
    - 'In what location?'
  utter_joy:
    - 'Am so happy for you'
  utter_fear:
    - 'you shouldn be afraid'
  utter_anger:
    - 'Calm down, everything gonna be alright'
  utter_sadness:
    - 'be a tough guy'
  utter_disgust:
    - 'I understand, this is disgusting, we should get over it'
  utter_shame:
    - 'Ridicule does not kill'
  utter_guilt:
    - 'You must not feel guilty, feel proud'

actions:
 - utter_greet
 - utter_goodbye
 - utter_ask_location
 - actions.ActionWeather
 - utter_joy
 - utter_fear
 - utter_anger
 - utter_sadness
 - utter_disgust
 - utter_shame
 - utter_guilt

Issue:
Having problems with the tutorial, I followed the climate chatbot rasa tutorial provided by Justina Petraityte, you can find the GitHub repository here. However, my chatbot never recognizes the intent I tried to give it when I try with run_online.py, which has to be the location,
Indeed, for instance when testing the following sentence "I had a bad day today" with :
from rasa_nlu.training_data import load_data
from rasa_nlu import config
from rasa_nlu.model import Trainer
from rasa_nlu.model import Metadata, Interpreter

def train_nlu(data, configs, model_dir):
	training_data = load_data(data)
	trainer = Trainer(config.load(configs))
	trainer.train(training_data)
	model_directory = trainer.persist(model_dir, fixed_model_name = 'weathernlu')

def run_nlu():
	interpreter=Interpreter.load('./models/nlu/default/weathernlu/test')
	print(interpreter.parse(u"I am planning my holiday to Lithuania. I wonder what is the weather out there."))

if __name__ == '__main__':
	#train_nlu('./data/data.json', 'config_spacy.json', './models/nlu')
	run_nlu()

He wasn't able to recognize it :
(MoodEnv) mike@mike-thinks:~/Programing/Rasa/Moodbot$ python3 nlu_model.py
/home/mike/Programing/Rasa/Moodbot/MoodEnv/lib/python3.5/site-packages/rasa_nlu/extractors/entity_synonyms.py:85: UserWarning: Failed to load synonyms file from './models/nlu/default/weathernlu/test/entity_synonyms.json'
  "".format(entity_synonyms_file))
{'intent_ranking': [], 'text': 'I am planning my holiday to Lithuania. I wonder what is the weather out there.', 'entities': [], 'intent': None}

You can find the training_data.json file here and metadata.json there

Quite bizarre but after reinstalling the backend, spacy, it seems it solved the problem:
pip install rasa_nlu[spacy]
python -m spacy download en_core_web_md
python -m spacy link en_core_web_md en

Let's assume I forget to download it :/


Respond 200 on empty HTTP request to support GCE ingress health check

https://github.com/ripple/rippled/issues/3307

For https://github.com/ripple/rippled/issues/3307

Why wouldn't you do your health checks on an actual health check endpoint instead?

Having a public HTTP health check endpoint would be a great feature. It should use some heuristic (like info from server_info) to return a meaningful response in addition to a 200.
This 200 is just an indication that the rippled server is running and accepting connections on a particular port. It is not meant to denote health wrt network synchronization/load/etc.
This was a suggestion from @nbougalis as a one-off build for a particular use case I have. My biggest motivation for opening the PR is really to have Travis run and see if this unexpectedly breaks anything.

A non 200 response would also indicate that it is running and accepting connections.

Responding with a 200 allows this dumb endpoint to be used with a GKE ingress https://cloud.google.com/kubernetes-engine/docs/concepts/ingress#health_checks
The ingress allows changing the path of the health check, but not the accepted response code. The response code must be exactly 200.

You can set a path for readiness probes there, the /crawl endpoint for example would be a good workaround for this patch.

See https://github.com/ripple/rippled/issues/2809 for some discussion on this topic by the way.

Markus, that's a good idea. Docs are here: https://xrpl.org/peer-crawler.html
Example request: https://s1.ripple.com:51235/crawl

@MarkusTeufelberger  @intelliot  I really appreciate the feedback. I will prioritize #2809 to be discussed in the near future.  This fix is simply to provide a 200 response for the load balancer. Lets review this PR while #2809 undergoes discussion.

If this is for a load balancer, I think it might make sense to return 200 only if the server is synced with the network.
Sidenote: the /crawl endpoint does provide a 200 response:
% curl -I https://s1.ripple.com:51235/crawl
HTTP/1.1 200 OK
Server: rippled-1.4.0
Content-Type: application/json
Connection: close
Transfer-Encoding: chunked

Removing myself as reviewer.  The C++ looks fine.  But I'm not qualified to say if this is something that should be done for rippled.

@nbougalis do we want this as part of 1.6?

I think we should fold the idea behind this into #3365 and consider following the guidelines provided by @MarkusTeufelberger in that thread. I'd suggest that we just close this.


bug(mat-form-field): input placeholder overlapping input text, SASS depreciation warning

Reproduction
Use StackBlitz to reproduce your issue: https://stackblitz.com/fork/components-issue
Steps to reproduce:

Create a form using
Compare local build versus production build

Expected Behavior
What behavior were you expecting to see?

Actual Behavior
What behavior did you actually see?
After building my project with ng build --prod --configuration=production --aot, my input fields look like this:

DEPRECATION WARNING: As of Dart Sass 2.0.0, !global assignments won't be able to
declare new variables. Consider adding `$mat-form-field-outline-dedupe: null` at the root of the
stylesheet.

     ╷
4645 │   $mat-form-field-outline-dedupe: $mat-form-field-outline-dedupe + 0.00001 !global;
     │   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     ╵
<EMAIL_ADDRESS>4645:3  -mat-form-field-outline-label-floating()
<EMAIL_ADDRESS>4682:9  mat-form-field-outline-typography()
<EMAIL_ADDRESS>4951:3  mat-form-field-typography()
<EMAIL_ADDRESS>5005:3  angular-material-typography()
<EMAIL_ADDRESS>5033:3  mat-core()
    src/assets/styles/modules/_nav-search.scss 2:1       @import
    src/assets/styles/partials/_theme.scss 2:9           @import
    src/assets/styles/_partials.scss 2:13                @import
    stdin 1:9                                            root stylesheet

Environment
OS: macOS Catalina 10.15.4
Browser: Chrome 81.0.4044.113
     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/

Angular CLI: 9.1.1
Node: 13.11.0
OS: darwin x64

Angular: 9.1.2
... animations, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router, service-worker
Ivy Workspace: Yes

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.901.1
@angular-devkit/build-angular     0.901.1
@angular-devkit/build-optimizer   0.901.1
@angular-devkit/build-webpack     0.901.1
@angular-devkit/core              9.1.1
@angular-devkit/schematics        9.1.1
@angular/cdk                      9.2.1
@angular/cli                      9.1.1
@angular/flex-layout              9.0.0-beta.29
@angular/material                 9.2.1
@ngtools/webpack                  9.1.1
@schematics/angular               9.1.1
@schematics/update                0.901.1
rxjs                              6.5.5
typescript                        3.7.5
webpack                           4.42.0

I just tried it out against a new CLI project, but I didn't get the error. Can you post a link to your project?

Unfortunately,
I cannot provide a link to the project as it's an internal project. I can try to provide as many details as I can.
Here are some logs from the build:
DEPRECATION WARNING: As of Dart Sass 2.0.0, !global assignments won't be able to
 declare new variables. Consider adding `$mat-form-field-dedupe: null` at the root of the
 stylesheet.
      ╷
 4844 │   $mat-form-field-dedupe: $mat-form-field-dedupe + 0.00001 !global;
      │   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      ╵
<EMAIL_ADDRESS>4844:3  -mat-form-field-label-floating()
<EMAIL_ADDRESS>4911:7  mat-form-field-typography()
<EMAIL_ADDRESS>5005:3  angular-material-typography()
<EMAIL_ADDRESS>5033:3  mat-core()
     src/assets/styles/partials/_theme.scss 16:1          @import
     src/assets/styles/_partials.scss 2:13                @import
     stdin 1:9                                            root stylesheet
 DEPRECATION WARNING: As of Dart Sass 2.0.0, !global assignments won't be able to
 declare new variables. Consider adding `$mat-form-field-legacy-dedupe: null` at the root of the
 stylesheet.
      ╷
 4449 │   $mat-form-field-legacy-dedupe: $mat-form-field-legacy-dedupe + 0.00001 !global;
      │   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      ╵
<EMAIL_ADDRESS>4449:3  -mat-form-field-legacy-label-floating()
<EMAIL_ADDRESS>4499:9  mat-form-field-legacy-typography()
<EMAIL_ADDRESS>4948:3  mat-form-field-typography()
<EMAIL_ADDRESS>5005:3  angular-material-typography()
<EMAIL_ADDRESS>5033:3  mat-core()
     src/assets/styles/partials/_theme.scss 16:1          @import
     src/assets/styles/_partials.scss 2:13                @import
     stdin 1:9                                            root stylesheet
 DEPRECATION WARNING: As of Dart Sass 2.0.0, !global assignments won't be able to
 declare new variables. Consider adding `$mat-form-field-fill-dedupe: null` at the root of the
 stylesheet.
      ╷
 4347 │   $mat-form-field-fill-dedupe: $mat-form-field-fill-dedupe + 0.00001 !global;
      │   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      ╵
<EMAIL_ADDRESS>4347:3  -mat-form-field-fill-label-floating()
<EMAIL_ADDRESS>4377:9  mat-form-field-fill-typography()
<EMAIL_ADDRESS>4950:3  mat-form-field-typography()
<EMAIL_ADDRESS>5005:3  angular-material-typography()
<EMAIL_ADDRESS>5033:3  mat-core()
     src/assets/styles/partials/_theme.scss 16:1          @import
     src/assets/styles/_partials.scss 2:13                @import
     stdin 1:9                                            root stylesheet
 DEPRECATION WARNING: As of Dart Sass 2.0.0, !global assignments won't be able to
 declare new variables. Consider adding `$mat-form-field-outline-dedupe: null` at the root of the
 stylesheet.
      ╷
 4645 │   $mat-form-field-outline-dedupe: $mat-form-field-outline-dedupe + 0.00001 !global;
      │   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      ╵
<EMAIL_ADDRESS>4645:3  -mat-form-field-outline-label-floating()
<EMAIL_ADDRESS>4682:9  mat-form-field-outline-typography()
<EMAIL_ADDRESS>4951:3  mat-form-field-typography()
<EMAIL_ADDRESS>5005:3  angular-material-typography()
<EMAIL_ADDRESS>5033:3  mat-core()
     src/assets/styles/partials/_theme.scss 16:1          @import
     src/assets/styles/_partials.scss 2:13                @import
     stdin 1:9                                            root stylesheet

I can make an educated guess that we probably don't need the !global in there, but I'd rather avoid committing it to the codebase without verifying that it actually fixes the issue. Can you try going into the node_modules, removing the !global and trying to see if it fixes the warning?

@crisbeto Yes, that seemed to work. The warnings are gone AND the fields are fixed.

It looks like that ends up breaking the output that we were going for when we first added the workaround. I set up a small project with a Sass file that looks like this to try and reproduce it:
@import './node_modules/@angular/material/theming';
@include mat-core();

I wasn't able to get it to fail with any of the recent versions of sass or node-sass. Do you know which version you're running on your end? Also I'm wondering whether there isn't some config option in your build that's surfacing the warning.

Seems I've got it working now. Some of the sass imports were in the wrong order I guess. 🤷🏻♂Apologies for the confusion. Appreciate your help!


Removed custom version of AtmosphereSigmaFactory in diagnostics

Description
This PR removes the custom version of AtmosphereSigmaFactory in all affected diagnostics.
Corresponding PR in the core: ESMValGroup/ESMValCore#1382

Related to ESMValGroup/ESMValCore#1380 and #2354

Before you get started

[x] ☝ Create an issue to discuss what you are going to do

Checklist
It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

[x] 🛠 This pull request has a descriptive title
[x] 🛠 Code is written according to the code quality guidelines
[x] 🛠 Documentation is available
[x] 🛠 Tests run successfully
[x] 🛠 The list of authors is up to date
[x] 🛠 Any changed dependencies have been added or removed correctly
[x] 🛠 All checks below this pull request were successful

New or updated recipe/diagnostic

[x] 🧪 Recipe runs successfully
[x] 🧪 Recipe is well documented
[x] 🧪 Figure(s) and data look as expected from literature
[x] 🛠 Provenance information has been added

To help with the number of pull requests:

🙏 We kindly ask you to review two other open pull requests in this repository

@esmvalbot please run recipe_ecs_constraints.yml

@esmvalbot please run recipe_ecs_constraints.yml

Does the bot use the current main of ESMValCore? If not this will fail :grimacing:

sure it will, but I thought this bit of the diag will go through? Ignore the run if I'm wrong

@esmvalbot please run recipe_ecs_constraints.yml

Does the bot use the current main of ESMValCore? If not this will fail 😬

Yes it does. The bot does not install the Core in dev mode, just the Tool. Let's wait for the outcome of this run before merging this.

It needs the changes in the core (in particular the changed fixes for some models) to work, so it will only work with the current main of ESMValCore. I tested the recipes recipe_ecs_constraints.yml and recipe_schlund20esd.yml locally, both run fine now! :tada:

well the bot pulled a fast one and surprised us by not finding data - not even close to running the diagnostic :laughing: Merge at will by me :+1:

cheers @remi-kazeroni - safer that way :grin: