[CDA-91] As a Developer I can't invoke Jobs or Transformations from "Initial Transformation"

[CDA-91] As a Developer I can't invoke Jobs or Transformations from "Initial Transformation"
-fixed variable with class name for subclasses
-fixed message for exception loading sub transformation

Build Completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -f engine/build.xml checkstyle publish-local

Cleanup Commands
rm -r ~/.ivy2/local || echo no publish local to remove

Changed files
  engine/src/org/pentaho/di/trans/StepWithMappingMeta.java
  engine/src/org/pentaho/di/trans/messages/messages_en_US.properties
  engine/src/org/pentaho/di/trans/messages/messages_it_IT.properties
  engine/src/org/pentaho/di/trans/messages/messages_ja_JP.properties
  engine/src/org/pentaho/di/trans/steps/mapping/MappingMeta.java
  engine/src/org/pentaho/di/trans/steps/simplemapping/SimpleMappingMeta.java
  engine/src/org/pentaho/di/trans/steps/singlethreader/SingleThreaderMeta.java
  engine/src/org/pentaho/di/trans/steps/transexecutor/TransExecutorMeta.java

Newly Broken Tests:
org.pentaho.di.trans.steps.textfileinput.TextFileInputMetaLoadSaveTest.testSerialization:
:red_circle: java.lang.RuntimeException
java.lang.RuntimeException: Error validating dateFormatLocale
	at org.pentaho.di.trans.steps.loadsave.LoadSaveTester.validateLoadedMeta(LoadSaveTester.java:176)
	at org.pentaho.di.trans.steps.loadsave.LoadSaveTester.testXmlRoundTrip(LoadSaveTester.java:223)
	at org.pentaho.di.trans.steps.loadsave.LoadSaveTester.testSerialization(LoadSaveTester.java:182)
	at org.pentaho.di.trans.steps.textfileinput.TextFileInputMetaLoadSaveTest.testSerialization(TextFileInputMetaLoadSaveTest.java:167)
Caused by:

Still Broken Tests:
org.pentaho.di.trans.steps.syslog.SyslogMessageConcurrentTest.concurrentSyslogMessageTest:
:red_circle: org.junit.runners.model.TestTimedOutException
org.junit.runners.model.TestTimedOutException: test timed out after 2000 milliseconds
	at org.mockito.cglib.core.CollectionUtils.bucket(CollectionUtils.java:30)
	at org.mockito.cglib.core.EmitUtils.string_switch_hash(EmitUtils.java:260)
	at org.mockito.cglib.core.EmitUtils.string_switch(EmitUtils.java:171)
	at org.mockito.cglib.proxy.MethodInterceptorGenerator.generateFindProxy(MethodInterceptorGenerator.java:234)
	at org.mockito.cglib.proxy.MethodInterceptorGenerator.generate(MethodInterceptorG

Unit test coverage change
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.
org.pentaho.di.job.entries.evalfilesmetrics.JobEntryEvalFilesMetrics

Branch Change: -.78%:small_red_triangle_down:
Complexity Change: -1.03%:small_red_triangle_down:
Instruction Change: -.16%:small_red_triangle_down:

org.pentaho.di.job.entries.sql.JobEntrySQL

Instruction Change: -3.28%:small_red_triangle_down:
Line Change: -3.87%:small_red_triangle_down:

org.pentaho.di.job.entries.waitforsql.JobEntryWaitForSQL

Branch Change: -.66%:small_red_triangle_down:
Complexity Change: -.93%:small_red_triangle_down:
Instruction Change: -.13%:small_red_triangle_down:

org.pentaho.di.trans.step.StepMeta

Instruction Change:  + .19%
Line Change:  + .26%

org.pentaho.di.trans.steps.checksum.CheckSumMeta

Instruction Change:  + .11%

org.pentaho.di.trans.steps.combinationlookup.CombinationLookupMeta

Instruction Change: -.04%:small_red_triangle_down:

org.pentaho.di.trans.steps.csvinput.CsvInputMeta

Instruction Change:  + .05%

org.pentaho.di.trans.steps.datagrid.DataGridMeta

Instruction Change:  + .09%

org.pentaho.di.trans.steps.dimensionlookup.DimensionLookupMeta

Branch Change:  + .31%
Instruction Change:  + .06%
Line Change: -.23%:small_red_triangle_down:
Method Change: -1.02%:small_red_triangle_down:

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change: -1.04%:small_red_triangle_down:
Complexity Change: -1.05%:small_red_triangle_down:

org.pentaho.di.trans.steps.insertupdate.InsertUpdateMeta

Instruction Change:  + .05%

org.pentaho.di.trans.steps.ldapoutput.LDAPOutputMeta

Instruction Change:  + .06%

org.pentaho.di.trans.steps.mapping.MappingMeta

Line Change: -.18%:small_red_triangle_down:

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change: -3.53%:small_red_triangle_down:
Complexity Change: -4.05%:small_red_triangle_down:
Instruction Change: -.18%:small_red_triangle_down:
Line Change: -.48%:small_red_triangle_down:

org.pentaho.di.trans.steps.replacestring.ReplaceStringMeta

Instruction Change: -.07%:small_red_triangle_down:

org.pentaho.di.trans.steps.setvalueconstant.SetValueConstantMeta

Instruction Change:  + .14%

org.pentaho.di.trans.steps.simplemapping.SimpleMappingMeta

Line Change: -.28%:small_red_triangle_down:

org.pentaho.di.trans.steps.singlethreader.SingleThreaderMeta

Instruction Change:  + .10%
Line Change: -.19%:small_red_triangle_down:

org.pentaho.di.trans.steps.sort.SortRowsMeta

Instruction Change:  + .08%

org.pentaho.di.trans.steps.textfileinput.TextFileFilter

Complexity Change: -9.09%:small_red_triangle_down:
Instruction Change: -9.26%:small_red_triangle_down:
Line Change: -8.33%:small_red_triangle_down:
Method Change: -9.09%:small_red_triangle_down:

org.pentaho.di.trans.steps.textfileinput.TextFileInputMeta

Branch Change: -12.63%:small_red_triangle_down:
Complexity Change: -5.18%:small_red_triangle_down:
Instruction Change: -30.18%:small_red_triangle_down:
Line Change: -22.72%:small_red_triangle_down:
Method Change: -3.21%:small_red_triangle_down:

org.pentaho.di.trans.steps.transexecutor.TransExecutorMeta

Line Change: -.06%:small_red_triangle_down:

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta

Instruction Change: -.18%:small_red_triangle_down:

Checkstyle violations
The code submitted is not complying with one or more of our guidelines. To perform these checks locally, please use this tool.
/home/buildguy/project/engine/src/org/pentaho/di/trans/StepWithMappingMeta.java

:heavy_exclamation_mark: Unused import - org.pentaho.di.trans.steps.transexecutor.TransExecutorMeta. at Line: 37 Column: 8

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -f engine/build.xml checkstyle publish-local

Cleanup Commands
rm -r ~/.ivy2/local || echo no publish local to remove

Changed files
  engine/src/org/pentaho/di/trans/StepWithMappingMeta.java
  engine/src/org/pentaho/di/trans/messages/messages_en_US.properties
  engine/src/org/pentaho/di/trans/messages/messages_it_IT.properties
  engine/src/org/pentaho/di/trans/messages/messages_ja_JP.properties
  engine/src/org/pentaho/di/trans/steps/mapping/MappingMeta.java
  engine/src/org/pentaho/di/trans/steps/simplemapping/SimpleMappingMeta.java
  engine/src/org/pentaho/di/trans/steps/singlethreader/SingleThreaderMeta.java
  engine/src/org/pentaho/di/trans/steps/transexecutor/TransExecutorMeta.java

Still Broken Tests:
org.pentaho.di.trans.steps.syslog.SyslogMessageConcurrentTest.concurrentSyslogMessageTest:
:red_circle: org.junit.runners.model.TestTimedOutException
org.junit.runners.model.TestTimedOutException: test timed out after 2000 milliseconds
	at org.pentaho.di.trans.Trans$$EnhancerByMockitoWithCGLIB$$3ddc455c.CGLIB$STATICHOOK6(<generated>)
	at org.pentaho.di.trans.Trans$$EnhancerByMockitoWithCGLIB$$3ddc455c.<clinit>(<generated>)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.objenesis.instantiator.sun.SunReflectionFactoryInstantiator.newInstance(SunReflectionFactoryInstantiator.java:45)
	at org.objenesis.ObjenesisBase.n

Unit test coverage change
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.
org.pentaho.di.job.entries.evalfilesmetrics.JobEntryEvalFilesMetrics

Branch Change:  + .39%
Complexity Change:  + .51%
Instruction Change:  + .08%

org.pentaho.di.job.entries.sql.JobEntrySQL

Instruction Change: -3.28%:small_red_triangle_down:
Line Change: -3.87%:small_red_triangle_down:

org.pentaho.di.trans.step.StepMeta

Instruction Change:  + .19%
Line Change:  + .26%

org.pentaho.di.trans.steps.combinationlookup.CombinationLookupMeta

Instruction Change: -.04%:small_red_triangle_down:

org.pentaho.di.trans.steps.constant.ConstantMeta

Instruction Change:  + .10%

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change:  + 3.12%
Complexity Change:  + 3.16%
Instruction Change:  + .28%
Line Change:  + .74%

org.pentaho.di.trans.steps.ldapoutput.LDAPOutputMeta

Instruction Change: -.06%:small_red_triangle_down:

org.pentaho.di.trans.steps.mapping.MappingMeta

Line Change: -.18%:small_red_triangle_down:

org.pentaho.di.trans.steps.replacestring.ReplaceStringMeta

Instruction Change: -.07%:small_red_triangle_down:

org.pentaho.di.trans.steps.rowgenerator.RowGeneratorMeta

Instruction Change: -.08%:small_red_triangle_down:

org.pentaho.di.trans.steps.setvalueconstant.SetValueConstantMeta

Instruction Change:  + .14%

org.pentaho.di.trans.steps.simplemapping.SimpleMappingMeta

Line Change: -.28%:small_red_triangle_down:

org.pentaho.di.trans.steps.singlethreader.SingleThreaderMeta

Instruction Change: -.10%:small_red_triangle_down:
Line Change: -.19%:small_red_triangle_down:

org.pentaho.di.trans.steps.sort.SortRowsMeta

Instruction Change: -.08%:small_red_triangle_down:

org.pentaho.di.trans.steps.sortedmerge.SortedMergeMeta

Instruction Change:  + .18%

org.pentaho.di.trans.steps.tableoutput.TableOutputMeta

Branch Change: -.91%:small_red_triangle_down:
Complexity Change: -.82%:small_red_triangle_down:

org.pentaho.di.trans.steps.transexecutor.TransExecutorMeta

Line Change: -.06%:small_red_triangle_down:

org.pentaho.di.trans.steps.webservices.WebServiceMeta

Branch Change:  + 1.79%
Complexity Change:  + 1.19%
Instruction Change:  + .07%

@mbatchelor Could you please review and merge it?

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -f engine/build.xml checkstyle publish-local

Cleanup Commands
rm -r ~/.ivy2/local || echo no publish local to remove

Changed files
  engine/src/org/pentaho/di/trans/StepWithMappingMeta.java
  engine/src/org/pentaho/di/trans/messages/messages_en_US.properties
  engine/src/org/pentaho/di/trans/messages/messages_it_IT.properties
  engine/src/org/pentaho/di/trans/messages/messages_ja_JP.properties
  engine/src/org/pentaho/di/trans/steps/mapping/MappingMeta.java
  engine/src/org/pentaho/di/trans/steps/simplemapping/SimpleMappingMeta.java
  engine/src/org/pentaho/di/trans/steps/singlethreader/SingleThreaderMeta.java
  engine/src/org/pentaho/di/trans/steps/transexecutor/TransExecutorMeta.java

Still Broken Tests:
org.pentaho.di.trans.steps.syslog.SyslogMessageConcurrentTest.concurrentSyslogMessageTest:
:red_circle: org.junit.runners.model.TestTimedOutException
org.junit.runners.model.TestTimedOutException: test timed out after 2000 milliseconds
	at java.lang.Class.getDeclaredMethods0(Native Method)
	at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
	at java.lang.Class.getDeclaredMethods(Class.java:1975)
	at org.mockito.cglib.core.ReflectUtils.addAllMethods(ReflectUtils.java:349)
	at org.mockito.cglib.proxy.Enhancer.getMethods(Enhancer.java:422)
	at org.mockito.cglib.proxy.Enhancer.generateClass(Enhancer.java:457)
	at org.mockito.cglib.core

Unit test coverage change
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.
org.pentaho.di.core.auth.core.impl.DefaultAuthenticationConsumerFactory

Branch Change: -7.14%:small_red_triangle_down:
Complexity Change: -7.69%:small_red_triangle_down:

org.pentaho.di.job.entries.getpop.MailConnectionMeta

Branch Change: -1.00%:small_red_triangle_down:
Complexity Change: -1.41%:small_red_triangle_down:
Instruction Change: -.30%:small_red_triangle_down:

org.pentaho.di.job.entries.waitforsql.JobEntryWaitForSQL

Branch Change: -.66%:small_red_triangle_down:
Complexity Change: -.93%:small_red_triangle_down:
Instruction Change: -.13%:small_red_triangle_down:

org.pentaho.di.trans.steps.calculator.CalculatorMetaFunction

Branch Change: -8.70%:small_red_triangle_down:
Complexity Change: -3.08%:small_red_triangle_down:
Instruction Change: -.75%:small_red_triangle_down:
Line Change: -.89%:small_red_triangle_down:

org.pentaho.di.trans.steps.checksum.CheckSumMeta

Branch Change:  + 1.35%
Complexity Change:  + 1.45%
Instruction Change:  + .32%

org.pentaho.di.trans.steps.constant.ConstantMeta

Instruction Change:  + .10%

org.pentaho.di.trans.steps.csvinput.CsvInputMeta

Instruction Change: -.05%:small_red_triangle_down:

org.pentaho.di.trans.steps.datagrid.DataGridMeta

Instruction Change:  + .09%

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change: -2.08%:small_red_triangle_down:
Complexity Change: -2.11%:small_red_triangle_down:

org.pentaho.di.trans.steps.ifnull.IfNullMeta

Branch Change:  + 2.00%
Complexity Change:  + 1.89%
Instruction Change:  + .17%

org.pentaho.di.trans.steps.mapping.MappingMeta

Line Change: -.18%:small_red_triangle_down:

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change:  + 1.18%
Complexity Change:  + 1.35%

org.pentaho.di.trans.steps.setvalueconstant.SetValueConstantMeta

Instruction Change: -.14%:small_red_triangle_down:

org.pentaho.di.trans.steps.simplemapping.SimpleMappingMeta

Line Change: -.28%:small_red_triangle_down:

org.pentaho.di.trans.steps.singlethreader.SingleThreaderMeta

Instruction Change:  + .10%
Line Change: -.19%:small_red_triangle_down:

org.pentaho.di.trans.steps.transexecutor.TransExecutorMeta

Line Change: -.06%:small_red_triangle_down:


[Server] Components: Consider identifying CRD default property values

Current Behavior
User experience of configuring a component is enhanced when CRDs define default property values.
Desired Behavior
Model generator to identify or be cognizant of any default.
Implementation

Confirm validation behavior of the JSON Schema library being used. The "default" value is not automatically used to fill in missing values during validation. Whether or not a default value is applied depends on the specific JSON Schema library or tool being used.

[ ] i. Confirm golang server library behavior.
[ ] ii. Confirm rjsf json schema behavior.

Method of default value identification

Model Generator locates the property, and looks for the specific property within component JSON Schema to check for the presence of a default value.
Check for "default" keyword: Within that property's definition, look for the "default" keyword. If it's present, the value associated with it is the designated default.

Model generator to populate the Component's configuration with property value defaults.

Acceptance Tests

Components with JSON Schema properties that have the keyword "default"

Example of a need for this
A Service to Pod relationship is matched based on label selector. The Service has a port defined, but no protocol. The protocol to be used is assumed, since "tcp" is the default value for protocol. When the relationship evaluation occurs, "tcp" can be provided as the value for protocol, if "default" property values are considered.
In this specific example, once the Pod and Service are deployed, Kubernetes will add the default value(s) irrespective of whether the design contains them or not. A better UX is to show this to users prior to deployment.
Mockups
{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "age": { "type": "integer", "default": 25 }
  }
}

Contributor Guides and Handbook

🛠 Meshery Build & Release Strategy
📚 Instructions for contributing to documentation

Meshery documentation site and source

🎨 Wireframes and designs for Meshery UI in Figma
🙋🏾🙋🏼 Questions: Discussion Forum and Community Slack

I would like to work on this.


Endpoint /api/metadata/validate - some wrong request status codes or validation statuses

Occurred during exploration endpoint in https://github.com/IntersectMBO/govtool/issues/876
To reproduce use endpoint url = 'https://test-sanchonet.govtool.byron.network/api/metadata/validate'
with input as for specific case:
NOT OK, TBC - response is 200, status is null but hash is not for provided json, for case: URL to other json (drep format), hash ok - expected: INCORRECT_JSONLD
Inputs:
 {
    "url": "https://github.com/mpawel79/testrepo/blob/master/PawelDrep.jsonld",
    "hash": "eebfff0e2213ff0c5331e6d9d99d47a5a825dce7053671b6ed4daae4b4e0df5e"
}
Response code: 200
{
    "status": null,
    "valid": false,
    "raw": {
        "valid": true
    }
}
--------------------

NOT OK - response not ok, status not ok, for case: valid URL to corrupted json, hash ok - expected: INCORRECT_JSONLD
Inputs:
 {
    "url": "https://raw.githubusercontent.com/mpawel79/testrepo/master/test_INVALID_JSON_FORMAT.jsonld",
    "hash": "eebfff0e2213ff0c5331e6d9d99d47a5a825dce7053671b6ed4daae4b4e0df5e"
}
Response code: 200
{
    "status": null,
    "valid": false,
    "raw": {
        "valid": true
    }
}
--------------------

NOT OK - response ok, status not given - some other error msg, for INCORRECT_HASH not resturned,  URL ok, corrupted hash to long: expected: INCORRECT_HASH
Inputs:
 {
    "url": "https://raw.githubusercontent.com/mpawel79/testrepo/master/InfoJune03.jsonld",
    "hash": "CORRUPTED_eebfff0e2213ff0c5331e6d9d99d47a5a825dce7053671b6ed4daae4b4e0df5e"
}
Response code: 400
{
    "combinator": "ReqBody'",
    "error": "Error in $.hash: mzero"
}
--------------------

That endpoint is not to be used by govtool. We can close that issue as not planned.


Serving UI behind a reverse proxy with a URL prefix

Problem Description
I'm trying to serve the Fabio UI behind an nginx reverse proxy with a URL prefix of /fabio. This means i would like the URL of the UI to be https://server.domain.tld/fabio/routes and the API to be https://server.domain.tld/fabio/api. The health check should be https://server.domain.tld/fabio/api and the manual should be https://server.domain.tld/fabio/manual.
I have the following nginx config:
server {
  listen       *:443 ssl;

  ssl on;

  ssl_certificate           /etc/ssl/node.crt;
  ssl_certificate_key       /etc/ssl/node.key;
  ssl_session_cache         shared:SSL:10m;
  ssl_session_timeout       5m;
  ssl_protocols             TLSv1 TLSv1.1 TLSv1.2;
  ssl_ciphers               EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:ECDHE-RSA-AES128-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA128:DHE-RSA-AES128-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA128:ECDHE-RSA-AES128-SHA384:ECDHE-RSA-AES128-SHA128:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA128:DHE-RSA-AES128-SHA128:DHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:AES128-GCM-SHA384:AES128-GCM-SHA128:AES128-SHA128:AES128-SHA128:AES128-SHA:AES128-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4;
  ssl_prefer_server_ciphers on;

  access_log            /var/log/nginx/ssl.access.log combined;
  error_log             /var/log/nginx/ssl.error.log;

  add_header              Front-End-Https on;
  add_header              X-Content-Type-Options nosniff;

  location /consul/ {
    rewrite ^/consul/(.*)  /$1 break;

    proxy_pass       http://<IP_ADDRESS>:8500/;
    proxy_redirect   / /consul/;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Host $host:$server_port;
    proxy_set_header X-Forwarded-Server $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  }
  # consul hard codes their API endpoint URL in their web UI, have to reverse proxy this also
  location /v1/ {
    proxy_pass       http://<IP_ADDRESS>:8500/v1/;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Host $host:$server_port;
    proxy_set_header X-Forwarded-Server $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  }

  location /fabio/ {
    proxy_pass       http://<IP_ADDRESS>:9998/;
    proxy_redirect   / /fabio/;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Host $host:$server_port;
    proxy_set_header X-Forwarded-Server $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $host;
  }
}

Using this config i can hit /fabio/ and it redirects me to /fabio/routes, however the screen that is rendered has a blank routes list.

Looking at the code it appears that the UI has a hard coded path that it's using for the API in Javascript
https://github.com/fabiolb/fabio/blob/master/admin/ui/route.go#L123-L139
So /api/routes and /api/paths are hard coded. To get around this i had to add another reverse location for /api/:
  location /api/ {
    proxy_pass       http://<IP_ADDRESS>:9998/api/;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Host $host:$server_port;
    proxy_set_header X-Forwarded-Server $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $host;
  }

Ideas
It would be great if we could specify a custom prefix in the config so i could serve everything from the /fabio/ prefix with a single reverse proxy location.
FYI: it appears that Consul has the same problem with their API endpoint (it's hard coded as /v1/)

Same issue here, we have to add the following routes on the ALB side:

/api/
/routes
/logo.svg

If we could just route /fabio-ui/ or something like this, it would be much easier.

Hitting the same wall, used -ui-content-path to serve Consul UI behined Traefik with PathPrefix rules.. Is there any progress with this?

If using ALB, this will work:

Why do you people want to hide one proxy behind another? What is the good reason to use nginx with fabio?


[WIP][SPARK-29048] Improve performance on Column.isInCollection() with a large size collection

What changes were proposed in this pull request?

The Column.isInCollection() with a large size collection will generate an expression with large size children expressions. This make analyzer and optimizer take a long time to run.
In this PR, in isInCollection() function, directly generate InSet expression, avoid generating too many children expressions.
Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Test build #110470 has started for PR 25754 at commit 5d79149.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15449/
Test PASSed.

Test build #110470 has finished for PR 25754 at commit 5d79149.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110470/
Test PASSed.


Is it banable?

Can you get banned using an xp bot?
I really dont wanna lose my account

Obviously. Seems like you don't care about your account if you are using this.


yaml-cpp: disable test & util

Both the tests and utilites are neither run nor installed.

The change of the commit itself looks good to me. I have also successfully tested the commit on this build configuration:
Build Configuration:
BB_VERSION        = "1.33.4"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "ubuntu-14.04"
TARGET_SYS        = "i586-oe-linux"
MACHINE           = "qemux86"
DISTRO            = "nodistro"
DISTRO_VERSION    = "nodistro.0"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              = "master:bafb7e221d40d7a87a02cec8a97d98eec7c23438"
meta-oe
meta-multimedia
meta-python       = "master:b063789560bfb9c60a7a15277b5b3a9839b5ba74"
meta-ros          = "rapyuta-disable_test:658dad33c68876572dbeb8645ed5739350a44854"

However, I have some remarks on the commit message and the context and intention (first the possibly trivial points, then the harder ones):

The commit itself has two different emails for the author and the committer. Is there a specific reason for using two different emails in those different roles, or is just a coincidence due to use of different machines with different default git user configurations? I would prefer to just have one email, which can be used to reach the author in case of questions. If you need help to change this with git, just let me know.
The commit message contains a typo, utilites should be utilities.
In general, I would like to understand and the commit message should explain the following points: What is the reason for this commit (reducing image size, reducing dependencies, avoiding some runtime failure)? How was this discovered, e.g., some review of the artefacts, some failure, another activity for another specific goal? Why is the yaml-cpp recipe modified, are there possibly more recipes that will be addressed in future with the same intention? What are the drawbacks of removing the tests and utilities, will this deactivate some functionality on the existing image?

I modified the author and commit message. Let me know if this is ok. As for the reason why I started to look into this recipe was that it did not build for me due to an error with gmock. Disabling the test & util fixes this and I believe this is an issue with my particular setup and unrelated to the recipe itself.

Maybe you can simply add the error message that occurred with gmock in your build setup, and a comment, such as: In my particular setup, i.e., with gmock installed on the host system/image (?), building yaml-cpp failed with: ... Rather than addressing the issue in yaml-cpp, this commit deactivates the tests and utilities that fail with the setup. As both the tests and utilities are not installed and not packaged, this commit should not cause any build failures for any other unrelated setup.
Does this commit message reflect your situation and why you came up with that patch?

Unfortunately, I'm unable to reproduce the error (I no longer have access to the setup which produced this error). So, I'm not longer able to say what exactly the error was.

So let us document that state in the commit message with: In a specific build setup that we are not able to reproduce anymore, building yaml-cpp failed when compiling the utilities and tests. Rather than addressing the issue in yaml-cpp, this commit deactivates the tests and utilities that failed with that setup. As both the tests and utilities are not installed and not packaged, this commit should not cause any build failures for any other unrelated setup.

When we then potentially revisit this commit in a couple years, this commit message will clearly point out that its intention was to improve the situation for a very specific build setup, while hoping not to break all other existing build setups.

Change already appears in commit d17b8e1e055d89c4ebd0cf5f3045ee5abcb43971.


[MINOR][PYTHON][TESTS] Remove unnecessary hack imports

What changes were proposed in this pull request?
Remove unnecessary hack imports
Why are the changes needed?
it should be no longer needed after the introduction of parent Column class
Does this PR introduce any user-facing change?
no, test only
How was this patch tested?
ci
Was this patch authored or co-authored using generative AI tooling?
no

merged to master


⚠ Bofrost.lu has degraded performance

In 364ebca, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1266 ms

Resolved: Bofrost.lu performance has improved in d9180ec after 19 minutes.


Dependabot removing platform specific gems from Gemfile.lock

Is there an existing issue for this?

[X] I have searched the existing issues

Package ecosystem
Bundler
Package manager version
2.3.14
Language version
Ruby 3.1.2
Manifest location and content before the Dependabot update
/Gemfile.lock
    - google-protobuf (3.21.7)
    + google-protobuf (3.21.7-x86_64-linux)
    - mini_portile2 (2.8.0)
    - nokogiri (1.13.8)
    -    mini_portile2 (~> 2.8.0)
    + nokogiri (1.13.8-x86_64-linux)
      sorbet (0.5.10346)
        sorbet-static (= 0.5.10346)
      sorbet-runtime (0.5.10346)
    - sorbet-static (0.5.10346-universal-darwin-14)
    - sorbet-static (0.5.10346-universal-darwin-15)
    - sorbet-static (0.5.10346-universal-darwin-16)
    - sorbet-static (0.5.10346-universal-darwin-17)
    - sorbet-static (0.5.10346-universal-darwin-18)
    - sorbet-static (0.5.10346-universal-darwin-19)
    - sorbet-static (0.5.10346-universal-darwin-20)
    - sorbet-static (0.5.10346-universal-darwin-21)
    - sorbet-static (0.5.10346-universal-darwin-22)
      sorbet-static (0.5.10346-x86_64-linux)
      sorbet-static-and-runtime (0.5.10346)
        sorbet (= 0.5.10346)
    - syntax_tree (3.6.2)
    + syntax_tree (3.6.3)

dependabot.yml content
version: 2
updates:
  - package-ecosystem: 'bundler'
    pull-request-branch-name:
      separator: '-'
    directory: '/'
    schedule:
      interval: 'daily'
    labels:
      - 'dependencies'
      - 'bundler'
      - 'auto-merge'
      - 'auto-approve'
    ignore:
      # https://github.com/dependabot/dependabot-core/issues/5420 Try again after next release of dependabot
      - dependency-name: "sorbet-static-and-runtime"
  - package-ecosystem: 'npm'
    pull-request-branch-name:
      separator: '-'
    directory: '/'
    schedule:
      interval: 'daily'
    labels:
      - 'dependencies'
      - 'npm'
      - 'auto-merge'
      - 'auto-approve'
    ignore:
      # https://github.com/axios/axios/issues/5026 Try again after next release of dependabot
      - dependency-name: "axios"

Updated dependency
syntax_tree from 3.6.2 to 3.6.3
What you expected to see, versus what you actually saw
I would have expected dependabot not to introduce platform-specific dependencies where there was none (google-protobuf and nokogiri), and expected to keep the existing ones (sorbet-static)
Native package manager behavior
On a linux workstation everything is fine as expected since it's the same architecture as dependabot. On a macos workstation, the bundler modifies the manifest to add again those missing platforms.
Images of the diff or a link to the PR, issue, or logs

Smallest manifest that reproduces the issue
not tested yet :
ruby '3.1.2'

gem 'bundler', '2.3.14'
gem 'sorbet-static-and-runtime', '0.5.10346'
gem 'syntax_tree'
gem 'google-protobuf', '3.21.7'
gem 'nokogiri', '1.13.8'

Uugggh, this issue keeps chasing me, both upstream and here. I will have a look at this, thanks for reporting!

@thomasnemer Ok now I remember. So this is due to a recent upstream change (note that Dependabot is already at Bundler 2.3.22).
Basically, your lock file is not resolvable using the "RUBY" platform only (by the "RUBY" platform we mean, using only gems that are not platform specific). The reason is that the lock file includes sorbet-static, which does not have any versions that are NOT platform specific.
This was previously tolerated, but in other to fix other platform issues, we had to start being strict about it. But instead of failing, we decided to automatically fix the problematic lock files, locking them to a proper specific platform where the lock file is valid.
I think there's an issue here though, and that is that Bundler should be not remove your existing platform specific gems. So the expectation here would be a diff like this:
diff --git a/Gemfile.lock b/Gemfile.lock
index caf5c86..c70d427 100644
--- a/Gemfile.lock
+++ b/Gemfile.lock
@@ -2,9 +2,11 @@ GEM
   remote: https://rubygems.org/
   specs:
-    google-protobuf (3.21.7)
-    mini_portile2 (2.8.0)
-    nokogiri (1.13.8)
-      mini_portile2 (~> 2.8.0)
+    google-protobuf (3.21.7-x86_64-darwin)
+    google-protobuf (3.21.7-x86_64-linux)
+    nokogiri (1.13.8-arm64-darwin)
+      racc (~> 1.4)
+    nokogiri (1.13.8-x86_64-darwin)
+      racc (~> 1.4)
+    nokogiri (1.13.8-x86_64-linux)
       racc (~> 1.4)
     prettier_print (1.0.0)
     racc (1.6.0)
@@ -28,10 +30,18 @@ GEM
       prettier_print

 PLATFORMS
-  ruby
+  universal-darwin-14
+  universal-darwin-15
+  universal-darwin-16
+  universal-darwin-17
+  universal-darwin-18
+  universal-darwin-19
+  universal-darwin-20
+  universal-darwin-22
+  x86_64-linux

 DEPENDENCIES

It would be nice if Dependabot showed some message in the PR when it replaces the Ruby platform with something else, and why, but it's not something it can currently do.

@deivid-rodriguez I think this is a known issue, and were going to solve it in a bundler update. Did that not happen with 2.3.22?

Which problem are you having, exactly the same? Here Dependabot is behaving as expected, and Bundler is having an issue where it's removing too many platforms. Yet, removing the RUBY one in particular is correct. I plan to move this issue to the Bundler repo.

@deivid-rodriguez Do you happen to know what the upstream fix was regarding this:

Ok now I remember. So this is due to a recent upstream change (note that Dependabot is already at Bundler 2.3.22).

Basically, your lock file is not resolvable using the "RUBY" platform only (by the "RUBY" platform we mean, using only gems that are not platform specific). The reason is that the lock file includes sorbet-static, which does not have any versions that are NOT platform specific.

This was previously tolerated, but in order to fix other platform issues, we had to start being strict about it. But instead of failing, we decided to automatically fix the problematic lock files, locking them to a proper specific platform where the lock file is valid.

Our team is investigating something similar, we're seeing that dependabot is throwing away the ruby platform and removing sorbet-static-* gems and only adding ones that match our platform (in our case, it's arm64-darwin-21 which seems to resolve to 0.5.10263-universal-darwin-21).

It sounds like that is the expected behavior based on the change you're mentioning because sorbet-static doesn't satisfy ruby platform as well, but I'd love to understand the full picture.

@danielvdao Of course. The relevant changes are https://github.com/rubygems/rubygems/pull/5495, later amended by https://github.com/rubygems/rubygems/pull/5807.

@deivid-rodriguez are there any plans to update dependabot to 2.3.24 to solve the bundler taking too long to resolve issue? We keep our bundler version in sync with Dependabot via GH Actions, as otherwise we tend to forget to update 😅

We just got that proposed at #6042, so we will move it forward soon!

Can this be closed now that #6042 is merged/deployed?

Yeah, there's still another issue that I mentioned at https://github.com/dependabot/dependabot-core/issues/5917#issuecomment-1282272289, but I'll move that upstream. Thanks for the remainder!


Fixes #1655: Don't sync SW matrels if nothing is reading their output…

… stream

Can't we just move sync_sw_matrel_groups(state, matrel); from L920 to L953?

Actually yea, that should work.


Specifying nodes for the ALB ingress controller

In our cluster, we have setup 2 deployment services to register
an Application LB each. However, since our usage requires one of the service
to use GPU enabled AMI, we were surprised to notice that instances from
both node groups (CPU and GPU) were registering on the Application LBs
of both services.
Is there a way by which we can specify nodes which should be registered for
load balancer for a service?
We need this to implement an autoscaling rule which should step up/down
our GPU node group as a function of active connections to the ALB for the CPU
based service, and the current number of healthy hosts in the GPU dependent
service's ALB.
However, if all the nodes of a cluster are registering to all ALB; our metrics for
fetching the current count of healthy instances in a specific target group get
skewed.
Kindly suggest a way to limit registration of all instances to all ALB target groups;
and have it specified similar to how deployments in a k8s cluster have the
nodeSelector property.

We are looking for this exact same thing. We have different classes of machines and would like to control traffic via the ALB a little better.

I raised this issue with the AWS support team as well. The suggestion they have for this is adding the following to manifest:
metadata:
  annotations:
    alb.ingress.kubernetes.io/target-type: ip


some messages are not objects but instead scalar values

Here is the latest export for ExecuteMsg (in Typescript)
export type ExecuteMsg = {
  mint: {
    [k: string]: unknown;
  };
} | {
  set_whitelist: {
    whitelist: string;
    [k: string]: unknown;
  };
} | {
  update_start_time: Timestamp;
} | {
  update_per_address_limit: {
    per_address_limit: number;
    [k: string]: unknown;
  };
} | {
  mint_to: {
    recipient: string;
    [k: string]: unknown;
  };
} | {
  mint_for: {
    recipient: string;
    token_id: number;
    [k: string]: unknown;
  };
} | {
  withdraw: {
    [k: string]: unknown;
  };
};

I noticed that update_start_time is the only one that doesn't take an object. Every other message uses either an empty object or an object with props.
update_start_time: Timestamp

Is there a convention? Or should we account for this in the cosmwasm-typescript-gen

from @grod220 (after closing #67 in favor of combining issues)
When you have a query type such as:
#[cw_serde]
#[derive(QueryResponses)]
pub enum QueryMsg {
    #[returns(String)]
    AllPreviousOwners(String),

The generated typescript will be missing the String argument required for this query:
  allPreviousOwners = async (): Promise<String> => {
    return this.client.queryContractSmart(this.contractAddress, {
      all_previous_owners: {}
    });
  };

Reproducible example here: https://github.com/grod220/ts-codegen-example/blob/master/contracts/account-nft/src/msg/query.rs#L32-L33

example in our code https://github.com/CosmWasm/ts-codegen/commit/acbb8339cd8c8a9cb7311548832e08239516cfa8

[ ] get examples for the various outputs we support — what does that look like for scalar values?

Timestamps in typescript work as a nanosecond int wrapped as a string. We're explicitly using Timestamp instead of block time (time or height) to avoid potential confusion with block height. https://github.com/public-awesome/stargaze-tools/blob/main/scripts/minter.ts#L277


Rate limit fix to intelligently calculate duration for the rate limited task.

Pull Request type

[ ] Bugfix
[X] Feature

Changes in this PR

Closing this since it will be merged to another repo.


Fixes #260: Add support for url imports in jsonnet

Fixes issue #260
Proposed Changes

[x] import URL's content in jsonnet
[x] add check for IOError when no jsonnet file is found
[x] write test case for jsonnet's url import feature

I'm not totally this is a good approach as it introduces external dependencies which will make a compilation non deterministic.
Ideally we'd support this via dependency management.


Vital signs must have finite values - NaN is invalid

What happened?
I got this exception java.lang.IllegalArgumentException: Vital signs must have finite values - NaN is invalid multiple times with a build from commit 6192ed703d8517e7bfab801d9fbee622ec4e8161 (latest as of 2023-10-07). Symptomatically it's identical to #1038.
A sample command line was:
java -Xmx24G -jar ./build/libs/synthea-with-dependencies.jar --exporter.fhir.export false --exporter.csv.export true --exporter.baseDirectory '/synthea_data/Arizona Phoenix 706' --generate.log_patients.detail none --exporter.clinical_note.export false -p 781512 Arizona Phoenix
Relevant output from the process:
Running with options:
Population: 781512
Seed:<PHONE_NUMBER>458
Provider Seed:1696672364458
Reference Time:<PHONE_NUMBER>458
Location: Phoenix, Arizona
Min Age: 0
Max Age: 140

As I mentioned, the exception is observed in many runs, for different cities / areas. For example, I ran all the cities in California, the setVitalSign / NaN exception is seen in the following cities. How impactful is this exception? Is the entire generated dataset suspect, or will the impact be localized only to specific individual?
California Anaheim (2 occurrences)
California Arden-Arcade
California Bakersfield
California Blythe
California Concord
California Davis
California El Monte
California Fresno
California Inglewood
California Irvine
California Menlo Park
California Moreno Valley
California Orinda
California Oxnard
California San Diego (2 occurrences)
California San Jose
California Santa Ana
California Santa Monica
California Soledad
California Stanton
California Vallejo
California Westmont
California Windsor

Environment
OS:
Linux version 4.18.0-425.10.1.el8_7.x86_64<EMAIL_ADDRESS>(gcc version 8.5.0 20210514 (Red Hat 8.5.0-15) (GCC)) #1 SMP Thu Jan 12 16:32:13 UTC 2023

Java:
openjdk 11.0.20 2023-07-18 LTS
OpenJDK Runtime Environment (Red_Hat-<IP_ADDRESS>.8-1) (build 11.0.20+8-LTS)
OpenJDK 64-Bit Server VM (Red_Hat-<IP_ADDRESS>.8-1) (build 11.0.20+8-LTS, mixed mode, sharing)

Relevant log output
java.lang.IllegalArgumentException: Vital signs must have finite values - NaN is invalid
        at org.mitre.synthea.world.agents.Person.setVitalSign(Person.java:486)
        at org.mitre.synthea.modules.WeightLossModule.manageYearTwoThroughFive(WeightLossModule.java:165)
        at org.mitre.synthea.modules.WeightLossModule.process(WeightLossModule.java:82)
        at org.mitre.synthea.engine.Generator.updatePerson(Generator.java:712)
        at org.mitre.synthea.engine.Generator.createPerson(Generator.java:668)
        at org.mitre.synthea.engine.Generator.generatePerson(Generator.java:478)
        at org.mitre.synthea.engine.Generator.lambda$run$3(Generator.java:383)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

Hey @arvindshmicrosoft, thanks for reporting this. I'm looking into it.
As far as the exception goes, that crashes the executor for the single patient. It should not impact the rest of the population.

Hi @arvindshmicrosoft! I'm having a hard time reproducing this issue. Two questions:

Are you only seeing the NaN issue? I have seen a few other stack traces while running simulations, but not this one.
Could you run unzip -p synthea-with-dependencies.jar META-INF/MANIFEST.MF and put the output in a comment? I want to make sure we are working from the exact same jar file.

Hi @arvindshmicrosoft! I'm having a hard time reproducing this issue. Two questions:

Are you only seeing the NaN issue? I have seen a few other stack traces while running simulations, but not this one.
Could you run unzip -p synthea-with-dependencies.jar META-INF/MANIFEST.MF and put the output in a comment? I want to make sure we are working from the exact same jar file.

Sure, here we go. Note that I have built from source - commit https://github.com/synthetichealth/synthea/commit/6192ed703d8517e7bfab801d9fbee622ec4e8161 (latest as of 2023-10-07).
Manifest-Version: 1.0
Main-Class: App
Build-Timestamp: 2023-10-06T21:17:14.165+0000
Build-Version:
Created-By: Gradle 8.2.1
Build-Jdk: 11.0.20 (Red Hat, Inc. 11.0.20+8-LTS)
Build-OS: Linux amd64 4.18.0-477.27.1.el8_8.x86_64

Unfortunately, I don't seem to have the old logs available. Let me try to re-run afresh and post back if I see any other exceptions in addition to the above one as well.

@arvindshmicrosoft thanks for the update. The log for Wauwatosa, Wisconsin was especially helpful because it allowed me to reproduce the issue within a reasonable sized population.
The root of the issue is a method that converts BMI percentile into an actual BMI for an individual. For the case causing the issue in Wauwatosa, the individual has a BMI in the ~99.74th percentile. Given other parameters in the equation, this high value caused a component to flip from positive to negative, which causes the entire method to return NaN. The issue is somewhat rare because the code path also requires that the simulated individual undergoes successful weight loss in their late teenage years and then regress in weight management where their period of weight regression overlaps their 20th birthday. This is where the code switches from pediatric to adult weight management algorithms.
I should be able to put together a PR to fix this particular issue next week.

@eedrummer Thank you for fixing this issue. I no longer see this issue with the current code.


TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/102813

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/103047

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/103249

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/105369


Added ISS support of Sanger data format

Codecov Report

Merging #75 (953ce0a) into main (755d475) will increase coverage by 0.07%.
Report is 22 commits behind head on main.
The diff coverage is 36.84%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #75      +/-   ##
==========================================
+ Coverage   41.92%   42.00%   +0.07%
==========================================
  Files          16       17       +1
  Lines         854      900      +46
==========================================
+ Hits          358      378      +20
- Misses        496      522      +26

Files Changed
Coverage Δ

src/spatialdata_io/readers/merscope.py
25.00% <8.33%> (-2.03%)
:arrow_down:

src/spatialdata_io/readers/iss.py
53.33% <53.33%> (ø)

src/spatialdata_io/__init__.py
100.00% <100.00%> (ø)

src/spatialdata_io/_constants/_constants.py
100.00% <100.00%> (ø)

Thank for the PR!
@giovp the PR enables the support for ISS data. ISS data doesn't come as a commercial format or as the output of a pipeline. I discussed with Tong about supporting readers for non-standard techs, and I think that even if this reader would not be guaranteed to work out-of-the-box like for the other techs, I can see it being helpful for the users as a starting point to build a customize reader for the data.
Wdyt, do you think it could be merged (maybe after some generalization via extra parameters) or should be keep the repository strict to documented techs and maybe ask to move this as a notebook or example code, like for the docs?

Yeah looks good to me! I honestly don't have strong opinion, I guess an idea would be to add it to a separate module called something like "experimental" or so but maybe we don't need to do that yet, and just go with this for now?

experimental does make more sense to me. And it also correctly refects my understanding of SpatialData :)

Hi @BioinfoTongLI, getting back to this, sorry quite some time passed. I have added my review.

[ ] Do you have some public data that I could use to test this?

A few additional quick comments, after this we could merge. If you can share the data I can also quickly cover the following tasks.

[ ] Can you please remove change the __init__.py files so that we use from spatialdata_io.experimental import iss and not from spatialdata_io import iss.
[ ] Please update the changelog to reflect the new added function of this PR.
[ ] Please update api.md with a new section so the docs also show the experimental readers.

Thanks!

@LucaMarconato What is the status?

Checking this today. @BioinfoTongLI is it ok if I rehost the data that you gave me?
The data has a T channel (in the screenshot I show the labels). We currently don't support T, so I need to remove it to be able to test the reader. I would upload the data for T=0.
Also, in the future it will be useful to parse the dims directly from the omero metadata when available. It's something that we want to do and when done this reader would benefit from it by become more general.

@BioinfoTongLI there is a problem with the data you shared. These is the AnnData table:
Out[15]:
           celltype               region
61793  PAX3+MyoProg  region_labels_image
61147  PAX7+MyoProg  region_labels_image
59104      TransMes  region_labels_image
58640          Mes1  region_labels_image
62107  PAX7+MyoProg  region_labels_image
61720  PAX7+MyoProg  region_labels_image
61020     InterZone  region_labels_image
59316       ProxMes  region_labels_image

And these are the unique IDs from the labels object.
Out[6]: Index([0, 28222, 28462, 28574, 29455, 29524, 29816, 29852, 30020], dtype='uint32')

Therefore the table is not annotating the labels. I will merge the PR anyway, but please when you have time it would be convenient if you could point to/share some public data (or data that I can upload to our public S3 storage) with correct table <> labels matching, so that in the future we can keep testing the dataset.

Hi @LucaMarconato , sorry for being silent for a while. I've uploaded another similar dataset onto Zenodo. I've checked the label image and the h5ad is matching this time. Hope that's what you're expecting?
please lmk if there's any other issue.
Best,
Tong

Thanks for sharing the new data. I have tried the reader with the new data and noticed that some small fixes were required in the reader (already merged here https://github.com/scverse/spatialdata-io/pull/164). I think the reader works correctly.
I have used this code to read and visualize the data:
##
from spatialdata_io.experimental import iss

f = '/Users/macbook/Downloads/10887810'

sdata = iss(f, raw_relative_path='raw.ome.tif', labels_relative_path='label.ome.tif', h5ad_relative_path='iss.h5ad')

from napari_spatialdata import Interactive

Interactive(sdata)

This is the plot that I get in napari (I am showing the instance_id column).

The black parts of the plot are the background, while the gray areas indicate pixels that have no corresponding annotation in the table (maybe cells that have been filtered out).
Precisely, we have that this code
from spatialdata import get_element_instances

labels_ids = get_element_instances(sdata['region_labels_image'])
table_ids = sdata['table'].obs['instance_id']

assert set(table_ids).issubset(set(labels_ids))
print(set(labels_ids).difference(set(table_ids)))

gives this:
{0, 43026, 38421, 42526, 42530, 37925, 35385, 37955, 37444, 40004, 41030, 40538, 42586, 39002, 36444, 38493, 35427, 38503, 35953, 41074, 36994, 43650, 36997, 43655, 42641, 40084, 40087, 35996, 40608, 39601, 35520, 41670, 35528, 40145, 37588, 40162, 36583, 40679, 39665, 36084, 40700, 41725, 43276, 42777, 43807, 35623, 41267, 39223, 35641, 39737, 40250, 41790, 39236, 43851, 37708, 42835, 43867, 42334, 42846, 37728, 38755, 38253, 35694, 43375, 38264, 38784, 39811, 39317, 37782, 43415, 37271, 42389, 35738, 42906, 38814, 36261, 36265, 38313, 41901, 43454, 39881, 42443, 39894, 43480, 38875, 37343, 43489, 41960, 39402, 38380, 40941, 37362}

Do you expect some cells not to be annotated by the table? (In any case, the reader seems to work correctly).

Yes, those are the cells been discarded in the single-cell processing steps. I forgot to mention that!
Very nice! Thanks for checking!


Slow performance with multiple joins

CrateDB version
4.6.4
CrateDB setup information
Number of nodes: 3
Heap: 4gb
CPU Cores: 2
Observed behavior
SELECT r.name as region, count(c.id)
FROM core.regions r
LEFT JOIN core.projects p ON (r.name = p.region)
LEFT JOIN core.clusters c ON (c.project_id = p.id)
WHERE (c.is_deleted is null or c.is_deleted = false)
AND r.name NOT LIKE '%.edge' GROUP by r.name limit 100;

Takes anywhere from 5 to 20 seconds.
All 3 tables have < 500 entries.
EXPLAIN
Eval[name AS region, count(id)]
  └ Limit[100::bigint;0]
    └ GroupHashAggregate[name | count(id)]
      └ Filter[((is_deleted IS NULL) OR (is_deleted = false))]
        └ NestedLoopJoin[LEFT | (project_id = id)]
          ├ NestedLoopJoin[RIGHT | (name = region)]
          │  ├ Rename[region, id] AS p
          │  │  └ Collect[core.projects | [region, id] | true]
          │  └ Rename[name] AS r
          │    └ Collect[core.regions | [name] | (NOT (name LIKE '%.edge'))]
          └ Rename[id, is_deleted, project_id] AS c
            └ Collect[core.clusters | [id, is_deleted, project_id] | true]

Steps to Reproduce
CREATE TABLE IF NOT EXISTS "core"."regions" (
   "description" TEXT,
   "name" TEXT,
   "agent_url" TEXT,
   "prometheus_url" TEXT,
   "basic_auth_password" TEXT,
   "organization_id" TEXT,
   "aws_region" TEXT,
   "aws_bucket" TEXT,
   "deprecated" BOOLEAN,
   "provider" TEXT,
   "status" TEXT,
   "last_seen" TIMESTAMP WITH TIME ZONE,
   "dc" OBJECT(STRICT) AS (
      "created" TIMESTAMP WITH TIME ZONE,
      "modified" TIMESTAMP WITH TIME ZONE
   ),
   "cert_type" TEXT,
   PRIMARY KEY ("name")
)
CLUSTERED BY ("name") INTO 1 SHARDS
WITH (
   "allocation.max_retries" = 5,
   "blocks.metadata" = false,
   "blocks.read" = false,
   "blocks.read_only" = false,
   "blocks.read_only_allow_delete" = false,
   "blocks.write" = false,
   codec = 'default',
   column_policy = 'strict',
   "mapping.total_fields.limit" = 1000,
   max_ngram_diff = 1,
   max_shingle_diff = 3,
   number_of_replicas = '1',
   refresh_interval = 1000,
   "routing.allocation.enable" = 'all',
   "routing.allocation.total_shards_per_node" = -1,
   "store.type" = 'fs',
   "translog.durability" = 'REQUEST',
   "translog.flush_threshold_size" = 536870912,
   "translog.sync_interval" = 5000,
   "unassigned.node_left.delayed_timeout" = 60000,
   "write.wait_for_active_shards" = '1'
)

CREATE TABLE IF NOT EXISTS "core"."projects" (
   "dc" OBJECT(DYNAMIC) AS (
      "created" BIGINT,
      "modified" BIGINT
   ),
   "friendly_name" TEXT,
   "id" TEXT,
   "name" TEXT,
   "organization_id" TEXT,
   "region" TEXT,
   "is_deleted" BOOLEAN,
   PRIMARY KEY ("id")
)
CLUSTERED BY ("id") INTO 1 SHARDS
WITH (
   "allocation.max_retries" = 5,
   "blocks.metadata" = false,
   "blocks.read" = false,
   "blocks.read_only" = false,
   "blocks.read_only_allow_delete" = false,
   "blocks.write" = false,
   codec = 'default',
   column_policy = 'strict',
   "mapping.total_fields.limit" = 1000,
   max_ngram_diff = 1,
   max_shingle_diff = 3,
   number_of_replicas = '1',
   refresh_interval = 1000,
   "routing.allocation.enable" = 'all',
   "routing.allocation.total_shards_per_node" = -1,
   "store.type" = 'fs',
   "translog.durability" = 'REQUEST',
   "translog.flush_threshold_size" = 536870912,
   "translog.sync_interval" = 5000,
   "unassigned.node_left.delayed_timeout" = 60000,
   "write.wait_for_active_shards" = '1'
)

CREATE TABLE IF NOT EXISTS "core"."clusters" (
   "id" TEXT,
   "name" TEXT,
   "username" TEXT,
   "dc" OBJECT(STRICT) AS (
      "created" TIMESTAMP WITH TIME ZONE,
      "modified" TIMESTAMP WITH TIME ZONE
   ),
   "num_nodes" INTEGER,
   "hardware_specs" OBJECT(STRICT) AS (
      "cpus_per_node" REAL,
      "disk_size_per_node" TEXT,
      "disk_size_per_node_bytes" BIGINT,
      "disk_type" TEXT,
      "disks_per_node" INTEGER,
      "heap_ratio" REAL,
      "heap_size" TEXT,
      "heap_size_bytes" BIGINT,
      "memory_per_node" TEXT,
      "memory_per_node_bytes" BIGINT
   ),
   "crate_version" TEXT,
   "project_id" TEXT,
   "product_name" TEXT,
   "product_tier" TEXT,
   "product_unit" INTEGER,
   "channel" TEXT,
   "origin" TEXT,
   "is_deleted" BOOLEAN DEFAULT false,
   "cluster_version" SMALLINT,
   "subscription_id" TEXT,
   "external_ip" TEXT,
   "deletion_protected" BOOLEAN,
   "ip_whitelist" ARRAY(OBJECT(IGNORED)),
   "status" TEXT,
   "last_seen" TIMESTAMP WITH TIME ZONE,
   PRIMARY KEY ("id")
)
CLUSTERED BY ("id") INTO 1 SHARDS
WITH (
   "allocation.max_retries" = 5,
   "blocks.metadata" = false,
   "blocks.read" = false,
   "blocks.read_only" = false,
   "blocks.read_only_allow_delete" = false,
   "blocks.write" = false,
   codec = 'default',
   column_policy = 'strict',
   "mapping.total_fields.limit" = 1000,
   max_ngram_diff = 1,
   max_shingle_diff = 3,
   number_of_replicas = '1',
   refresh_interval = 1000,
   "routing.allocation.enable" = 'all',
   "routing.allocation.total_shards_per_node" = -1,
   "store.type" = 'fs',
   "translog.durability" = 'REQUEST',
   "translog.flush_threshold_size" = 536870912,
   "translog.sync_interval" = 5000,
   "unassigned.node_left.delayed_timeout" = 60000,
   "write.wait_for_active_shards" = '1'
)

With https://github.com/crate/crate/pull/16780 all joins are executed as Hash-Join:
cr> explain SELECT r.name as region, count(c.id)
    FROM core.regions r
    LEFT JOIN core.projects p ON (r.name = p.region)
    LEFT JOIN core.clusters c ON (c.project_id = p.id)
    WHERE (c.is_deleted is null or c.is_deleted = false)
    AND r.name NOT LIKE '%.edge' GROUP by r.name limit 100;
+----------------------------------------------------------------------------------------------+
| QUERY PLAN                                                                                |
+----------------------------------------------------------------------------------------------+
| Eval[name AS region, count(id)] (rows=0)                                                     |
|   └ Limit[100::bigint;0] (rows=0)                                                            |
|     └ GroupHashAggregate[name | count(id)] (rows=0)                                          |
|       └ Filter[((is_deleted IS NULL) OR (is_deleted = false))] (rows=0)                      |
|         └ HashJoin[LEFT | (project_id = id)] (rows=unknown)                                  |
|           ├ HashJoin[LEFT | (name = region)] (rows=unknown)                                  |
|           │  ├ Rename[name] AS r (rows=unknown)                                              |
|           │  │  └ Collect[core.regions | [name] | (NOT (name LIKE '%.edge'))] (rows=unknown) |
|           │  └ Rename[region, id] AS p (rows=unknown)                                        |
|           │    └ Collect[core.projects | [region, id] | true] (rows=unknown)                 |
|           └ Rename[id, is_deleted, project_id] AS c (rows=unknown)                           |
|             └ Collect[core.clusters | [id, is_deleted, project_id] | true] (rows=unknown)    |
+----------------------------------------------------------------------------------------------+

and with https://github.com/crate/crate/pull/15760 all filters can be pushed down to the source if the query is rewritten to:
cr>  explain SELECT r.name as region, count(c.id)
        FROM core.regions r
        LEFT JOIN core.projects p ON (r.name = p.region)
        LEFT JOIN core.clusters c ON (c.project_id = p.id) and (c.is_deleted is null or c.is_deleted =
    false)
        WHERE r.name NOT LIKE '%.edge' GROUP by r.name limit 100;
+-----------------------------------------------------------------------------------------------------------------------+
| QUERY PLAN                                                                                |
+-----------------------------------------------------------------------------------------------------------------------+
| Eval[name AS region, count(id)] (rows=unknown)                                                                        |
|   └ Limit[100::bigint;0] (rows=unknown)                                                                               |
|     └ GroupHashAggregate[name | count(id)] (rows=unknown)                                                             |
|       └ HashJoin[LEFT | (project_id = id)] (rows=unknown)                                                             |
|         ├ HashJoin[LEFT | (name = region)] (rows=unknown)                                                             |
|         │  ├ Rename[name] AS r (rows=unknown)                                                                         |
|         │  │  └ Collect[core.regions | [name] | (NOT (name LIKE '%.edge'))] (rows=unknown)                            |
|         │  └ Rename[region, id] AS p (rows=unknown)                                                                   |
|         │    └ Collect[core.projects | [region, id] | true] (rows=unknown)                                            |
|         └ Rename[id, project_id] AS c (rows=unknown)                                                                  |
|           └ Collect[core.clusters | [id, project_id] | ((is_deleted IS NULL) OR (is_deleted = false))] (rows=unknown) |
+-----------------------------------------------------------------------------------------------------------------------+


Size of response with no 'content-length' header

Is it possible for a tag to be added that uses 'content-length' is the type of the data sent is not in chunks and a chunk size indicator when the data is sent in chunks?

It probably could. Please create a pull request and we'd be happy to take a look & accept!

I would like to raise a pr if no ones working on this
any rough idea on how the tag to be named?

computed-size?

hmm, but how computed-size can help differentiate between a response is in chunks or not?
may I didn't get your point on this could you please explain
::1 - - [28/Mar/2020:16:19:54 +0000] "GET / HTTP/1.1" 200 - "-" "insomnia/7.1.1"
having this as the response how can we differentiate the response is chunked or not
thanks!

Probably, I was more thinking of how to get the information of the computed size to the user

@dougwilson any pointers on this?

Just raise the PR; if there is some issue with the choosen token name it can be addressed in review :+1:


NPM Install, Unable to verify first certificate

Can you share the logs ?

2019-06-15T07_00_06_000Z-debug.log

What node version do you have installed? Check with node --version
Then, do you have windows-build-tools installed? https://www.npmjs.com/package/windows-build-tools


V2 fixes

[I accidentally closed this PR, and don't seem to be able to reopen it, so I'm submitting it again. I don't think Volker had any comment on it yet.  Sorry about the problem.  Also messed up the table PR where there were comments, but I will fix the problems raised and resubmit that PR as well.]
This fixes a number of small issues with attribute inheritance in MmlNodes, and with the bounding boxes in CHTML.

Although the mathsize attribute is only allowed on token elements, we want it to affect things like fractions, roots, tables, etc., so that when \large and the other sizing macros are used, everything scales appropriately.  Similarly, <math mathsize="200%">...</math> should scale everything up, not just the token elements.

When inheriting values check for default attribute values only in the actual default list, not the chained global defaults (now that the global defaults are in the chain).

The <math> element should set the displaystyle as an inherited value, since its default value is computed from several other values, when not explicitly given.

Fix the handling of displaystyle in <mstyle>

In CHTML, the bounding boxes are now computed in place by computeBBox(), rather than creating new BBox structures each time.  But that means that when getBBox() is called with save = false, then the box is saved anyway.  So computeBBox() was changed to accept the BBox to be used, and getBBox() passes it either the original box (when the value is to be saved), or a new zeroed box (if the box isn't being saved).  The CHTMLWrapper computeBBox() implementations are modified to include the BBox parameter.  The return value is no longer needed, since the BBox is passed to it.

Finally, super.computeBBox() is replaced by this.childNode[0].getBBox(), since that way you get the cached bounding box of the inferred row (the only child), rather than creating a new BBox and appending one BBox to it (extra work for nothing).

Changes made


[BUG] Updates and publishing QuestionAnswering APIs too slow

Library name and version
Azure.AI.Language.QuestionAnswering 1.1.0-beta.2
Describe the bug
We are using Azure QuestionAnswering for our product and recently we saw that all QuestionAnswering's APIs we used to update QnA, Source, and publish Knowledgebase become too slow, previously it usually takes 5 seconds but now takes ~30 seconds to complete.
APIs which we notice the slowness in QuestionAnsweringProjectsClient:
UpdateSourcesAsync
UpdateQnasAsync
DeployProjectAsync
This is a sample result of API call:
{{
  'createdDateTime': '2022-08-03T08:31:27+00:00',
  'expirationDateTime': '2022-08-03T14:31:27+00:00',
  'jobId': '9cbe1191-248d-422f-9736-162b6af11d58',
  'lastUpdatedDateTime': '2022-08-03T08:31:35+00:00',
  'status': 'succeeded',
  'resultUrl': '/knowledgebases/TrabbleKnowledgebase'
}}

It looks like azure takes 7 seconds to complete but my side only receives this 30 seconds after trigger request. This happens even with our staging environment on azure so it is not an internet connectivity issue.
The issue also happened with Azure.AI.Language.QuestionAnswering library version 1.1.0-beta.1
Time running the benchmark:

BenchmarkDotNet=v0.13.1, OS=Windows 10.0.22000
11th Gen Intel Core i7-11800H 2.30GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK=6.0.201
  [Host]     : .NET 6.0.3 (6.0.322.12309), X64 RyuJIT
  DefaultJob : .NET 6.0.3 (6.0.322.12309), X64 RyuJIT

Method
Mean
Error
StdDev

UpdateSourcesAsync
30.44 s
0.082 s
0.069 s

UpdateQnasAsync
30.44 s
0.111 s
0.098 s

DeployProjectAsync
30.36 s
0.048 s
0.037 s

Expected behavior
APIs should complete in ~5 seconds
Actual behavior
APIs take ~30 seconds to complete
Reproduction Steps
I created  a minimal project for reproducing the issue
https://github.com/tung238/QuestionAnsweringIssue
Environment
.NET SDK (reflecting any global.json):
Version:   6.0.201
Commit:    ef40e6aa06
Runtime Environment:
OS Name:     Windows
OS Version:  10.0.22000
OS Platform: Windows
RID:         win10-x64
Base Path:   C:\Program Files\dotnet\sdk\6.0.201\
Host (useful for support):
Version: 6.0.3
Commit:  c24d9a9c91
.NET SDKs installed:
3.1.416 [C:\Program Files\dotnet\sdk]
5.0.405 [C:\Program Files\dotnet\sdk]
6.0.102 [C:\Program Files\dotnet\sdk]
6.0.200 [C:\Program Files\dotnet\sdk]
6.0.201 [C:\Program Files\dotnet\sdk]
.NET runtimes installed:
Microsoft.AspNetCore.App 3.1.22 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.22 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.22 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.14 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Storage:0.3783393,Extensions:0.05948581,Tables:0.051196944'

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

@rokulka can you take a look? Our default polling is every 1s if the service doesn't send back a Retry-After header, so any change to timing is not likely the SDK.

I have verified that the service is sending back a Retry-After: 00:00:30 header, which the newer SDKs honor. @peytonfraser @rokulka this might be something for the service to consider.

This is a major issue for our products that use QuestionAnswering as well. And ideas if/when this may be resolved? @peytonfraser @rokulka


chore: add new configuration object to the ApiReference component

Marc and I had this idea to pass a configuration object to the  component instead of using all those props. That universal configuration object can then be used in Vue.js, React, Fastify and with the CDN version.
It also makes it easier to introduce new props/options to other platforms, without adding to every single integration.
Example Snippets
// Universal configuration object
const configuration = {
  theme: 'moon',
  proxy: 'http://localhost:5051',
  isEditable: true,
  tabs: {
    initialContent: 'Swagger Editor',
  },
}

// Fastify plugin
await fastify.register(require('@scalar/fastify-api-reference'), {
  routePrefix: '/reference',
  apiReference: {
    theme: 'moon',
    spec: {
      content: { … }
    },
  },
})

Good point! I’ve added the old props and marked them as deprecated. They are still used as fallbacks for the configuration object, but trigger a console warning now.


Set OpenTelemetry::Exporters::Jaeger::Exporter with opentelemetry-collector

Hey guys,
I'm having some trouble with setting the OpenTelemetry::Exporters::Jaeger::Exporter with opentelemetry-collector.
This is how I configure it:
collector_host = 'otel-collector.opentelemetry.svc.cluster.local'
collector_port = 14268

OpenTelemetry::SDK.configure do |c|
  c.add_span_processor(
    OpenTelemetry::SDK::Trace::Export::SimpleSpanProcessor.new(
      OpenTelemetry::Exporters::Jaeger::Exporter.new(
        service_name: 'ruby-app', host: collector_host, port: collector_port
      )
    )
  )
end

it seems like the collector host succesfully resolved because I DON'T
get this error anymore: /usr/local/bundle/gems/opentelemetry-exporters-jaeger-0.4.0/lib/opentelemetry/exporters/jaeger/transport.rb:14:in** 'connect': getaddrinfo: Name or service not known (SocketError)
But I can't see any logs related to this ruby-app in the collector nor see any traces in Jaeger.
Am I missing something?

the opentelemetry-collector expects traces in /api/traces so I tried the Jaeger collector directly, but the problem persists. keep getting Unable to connect to Jaeger Agent

I won't get to this this week, so punting it to Alpha 0.6. In reality, it should be done well before then, and can be released as a 0.5.1 version of just the Jaeger gem.


You can no longer construct multiple windows at once

Description
What it says on the tin.
Reproduction

Make a few directional reinforced plasmaglass windows and click on multiple at once
See the plasmaglass fall on the ground and the do_after restart

https://github.com/space-wizards/space-station-14/pull/14609

this was caused by the latest doafter fix patch, it added something that disables multiple doafters at once which should probably be rethought or reverted

There is an option to do multiple doafters at once. See: Mining, Stripping.


DOC: add examples for special.eval_legendre()

Reference issue
Contributes to gh-7168
What does this implement/fix?
Add example code to the docstring for scipy.special.eval_legendre()

build doc in CircleCI was failed. Cloud you please merge master branch to your branch, then push your branch again?

Thank you! That does all make sense, and yeah I goofed up range the first time around.
I can rebase later today to squash those commits. It will be good practice, my experience with rebase is limited and I want to get to know it better


Linting fails to load eslint-plugin-unused-imports

What version of Turborepo are you using?
1.1.2
What package manager are you using / does the bug impact?
npm
What operating system are you using?
Windows
Describe the Bug
When installing the eslint-plugin-unused-imports and running npm run lint, TurboRepo fails to find the plugin. However, other plugins (like eslint-plugin-sort-imports or eslint-plugin-prettier) work fine.
This is reproduceable on the basic Turbo repository.
config/eslint-preset.js:
module.exports = {
  extends: ["next", "prettier"],
	plugins: ["unused-imports"],
  settings: {
    next: {
      rootDir: ["apps/*/", "packages/*/"],
    },
  },
  rules: {
    "@next/next/no-html-link-for-pages": "off",
    "semi": ["error", "always"],
  },
};

Console output:
>>> npm run lint

><EMAIL_ADDRESS>lint
> turbo run lint

• Packages in scope: config, docs, tsconfig, ui, web
• Running lint in 5 packages
web:lint: cache miss, executing 33cea277c05ce63e
docs:lint: cache miss, executing 8a0d1c7d88d761bc
docs:lint:
docs:lint: ><EMAIL_ADDRESS>lint
docs:lint: > next lint
docs:lint:
web:lint:
web:lint: ><EMAIL_ADDRESS>lint
web:lint: > next lint
web:lint:
web:lint: Failed to load plugin 'unused-imports' declared in '.eslintrc.js': Cannot find module 'eslint-plugin-unused-imports'
web:lint: Require stack:
web:lint: - C:\Users\Florian\Documents\myapp\apps\web\.eslintrc.js
docs:lint: Failed to load plugin 'unused-imports' declared in '.eslintrc.js': Cannot find module 'eslint-plugin-unused-imports'
docs:lint: Require stack:
docs:lint: - C:\Users\Florian\Documents\myapp\apps\docs\.eslintrc.js

Expected Behavior
It should find the module and run the fix.
To Reproduce
Create a new project with npx create-turbo@latest. Add "eslint-plugin-unused-imports": "^2.0.0" to the dependencies of packages/config.
In the root folder, run npm i and npm run lint.

Fixed it by installing using npm i --workspace config eslint-plugin-unused-imports.


A way to block specific phrases

Sometimes people have "spam" retweets that follow specific patterns. I know that you can put -keyword into the API.search text field, however can you make a seperate field that you can enter in phrases or keywords you don't want to show up, to clear up the clutter in the section with words that you want to show up.

@Zen00 , this sounds like an easy to do fix. I'll take a look over this soon. Sorry for not being very cooperative, but I have a lot on my schedule right now. If you know anyone that is willing to give a hand, that'd be great.


getTessel fails if there are no USB or no LAN Tessels

There are a couple of issues with getTessel (that, frankly, I should have caught in my last code review).
First, I believe the lan.findConnections and usb.findConnection functions in the Promise.join need to be invoked with parenthesis.
Second, Promise.join returns two arrays and those need to be consolidated.

Current pull request that is open uses promise.all but yes if one fails
they a will. I'll find a way to fix that.
On Mon, Mar 30, 2015 at 20:41 Jon<EMAIL_ADDRESS>wrote:

Assigned #53 https://github.com/tessel/t2-cli/issues/53 to @HarleyKwyn
https://github.com/HarleyKwyn.
—
Reply to this email directly or view it on GitHub
https://github.com/tessel/t2-cli/issues/53#event-269157323.


preview2 cannot be minified by create-react-app

When trying to run npm run build using create-react-app? I got this error:
Creating an optimized production build...
Failed to compile.

Failed to minify the code from this file:

        ./node_modules/@aspnet/signalr/dist/esm/Loggers.js:4

Read more here: http://bit.ly/2tRViJ9

With preview1 everything works fine.

As per https://github.com/facebook/create-react-app/blob/master/packages/react-scripts/template/README.md#npm-run-build-fails-to-minify it looks like we have to transpile everything for ES5, even the "modern" ECMAScript Modules version. That's super unfortunate. We'll look at it for RC


Events and delegates

Hello. Thank you for this roadmap. Why are events and delegates not mentioned?

That is a great question, and I did consider both in the initial design. As I went through initial revising, I narrowed down the list to language concepts that a new developer would interact with directly. In that list, Events and Delegates were not of them for the following reasons:
Events
Events are a core part of Pub/Sub code architectures commonly found within game and user interface development. However, during research for this roadmap I found these were complex enough that a new developer would typically interact with Events through an abstraction in the framework they are using such as UnityEvent or WinForms's code generation tooling. The reason for this is that there are some notable gotchas with Events that also make it not so much of a beginner concept, namely:

Events can create memory leaks if not unsubscribed once its lifetime has completed.
Architecting code to use Events, while not too difficult in practice, can be a bit difficult for a beginner. Deciding to couple your entities vs using a separate Orchestrator type of class can be a bit tricky, and requires knowledge of architectural patterns to prevent adding unnecessary layers of abstraction.

Delegates
For Delegates, while an important concept, developers usually use the modern wrappers such as Action and Func types instead. Each of these handles a bit of the boilerplate usually associated with setting up delegates manually.

Action is a delegate type with a return type of void
Func is a delegate type with a return type specified as its last argument in the generic parameters (e.g. Func<bool>)

Side note: I highly recommend Tarodev's video on Events and Delegates, which if you're interested explains these two things well in greater detail!

Thanks for the quick response!
I was in the process of learning WPF, but I was unable to advance due the fact that I couldn't work with events and delegates. That's why I went looking for a roadmap that would show me where have I missed these two concepts along the way.


Make sure server Service actually implements ReadyService

Services that route to multiple endpoints cannot implement back-pressure. As such, they should implement ReadyService.
I'm not sure if this is already done, but I thought I would file an issue to remind myself to check.

ReadyService is no longer a thing.


Use btn-$variant-color variables for outlined btns

Despite now having the ability to override outline-variant color, it's not actually being used so each are still #fff
I've set $btn-primary-color to #000 but noticed it was still #fff for outlined buttons because the var isn't passed into @mixin button-outline-variant
Related: twbs#20734

👍 Nice @intellix! I had forgotten about this.
I recall that I hesitated in committing this change because it does have one visual change… and so I committed the smallest change with the goal of getting it merged, then coming back to this… but I got distracted. So, I'm glad you've made this commit!
The hover color for the secondary outline button is currently white, this commit changes it to the $gray-dark color. Arguably, this change of color makes the behavior more consistent to the default button normal and hover states (see below).
Outline buttons
Default:

Hover (before change):

Hover (after change):

While this does contain a visual change… it makes the secondary outline button hover state more visually similar to that of the default secondary button, adding more contrast to the text color vs background color:
Buttons
Default:

Hover:

I approve this change… however it may have minor affect on projects when they upgrade to use this code.


Product Attributes not populating

Tried to update to 1.0.4 but the Product.Attributes array does not populate for me. Has attribute handling changed?

I got the same error and found the problem. Product.StartTracking method inits the dictionary of attributes after inserting all the values in it, meaning the dictionary of attributes is always empty.

Could this be a fix?
 public override void StartTracking()         {             Attributes = Attributes ?? new Dictionary<string, string>(); ...


calculate sway from angular velocity

Hi,
Thanks for providing GT7Proxy!
I was experiencing some erratic behavior in sway...
After some research and testing I found that I could calculate the sway force based on the telemetry.angularvelocity_y
I don't know if it is 100% correct, but it feels good to me.
Feel free to take what you like and share this code.
-William Williams

could you explain what erratic behavior you experienced ?  I'm wondering if its the same thing I noticed.

could you explain what erratic behavior you experienced ? I'm wondering if its the same thing I noticed.

The sway values would start in the correct direction, but the suddenly shift to the opposite direction.
It seems unpredictable.  Initially, I thought perhaps just the sign was wrong, but correcting the sign alone did not fix the problem; the values were just not correct.

        Hello,This is what can happen when going from -180 to 180…. Simtools has an option for this mitigation. I’ll double check and make comparisons with your approach which makes sense anyway. You noticed it on a specific track ?Cheers    ---- On ven., 16 févr. 2024 20:06:00 +0100<EMAIL_ADDRESS>wrote ----

could you explain what erratic behavior you experienced ? I'm wondering if its the same thing I noticed.
The sway values would start in the correct direction, but the suddenly shift to the opposite direction.
It seems unpredictable.  Initially, I thought perhaps just the sign was wrong, but correcting the sign alone did not fix the problem; the values were just not correct.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

Hello,This is what can happen when going from -180 to 180…. Simtools has an option for this mitigation. I’ll double check and make comparisons with your approach which makes sense anyway. You noticed it on a specific track ?

I tried the SimTools "Anti-rollover" toggle to mitigate this, and although it may have behaved differently, the values were still erratic and it did not feel good.
For testing, I use a Mazda MX5 on Tsukuba... but this sway issue can be experienced on any turn on any track.

I'm using Yaw 2 with Sim Racing Studio's GT7 profile , and I'm convinced they are not calculating the roll properly.  Check this video I made. youtube video,  and tell me if this is what you noticed. (You need to only focus on the Back view)
Notice at 27 seconds I'm on a straight and swerving left and right and the roll is in the opposite direction as expected.  But then (here's the problem) at 37 seconds I'm turning a left corner and its tilted in the same direction (it should be opposite)
If this script solves this issue,  what I am going to do on my fork of your script, is add SRS as an additional output and feed your correct values directly the rig, similar to what this guy did grandturismo-srs-proxy](https://github.com/gt7coder/grandturismo-srs-proxy/blob/main/gt_packet_definition.py)
at least I'm pretty sure it will work.

For me, calculating a force based on weight and angles is not correct. Sway is a force, which can be calculated based on lateral speed and only lateral speed. No matter if your weight is 10Kg or 1000Kg, the force is the same. We have the local velocity vector based on quaternion so it is pretty easy to compute the local forces vector. You seem to calculate a mix of traction loss with a mass pultiplier.
I have made a new short video focusing on roll/pitch/yaw/heave/sway/surge/traction loss (by the way, roll/pitch/yaw were inverted before the release I made today) : https://youtu.be/y-9C2CPViKQ
As you can see, sway behaves pretty accurately, you can notice it when pulling the hand brake : the driver is "pushed" on left or right side. You can notice it also when "climbing" the road going from flat to 20° roll angle.
Simtools filtering can be usefull, especially on heave to make it less "jumpy", depending also on you min/max settings.

I see in the video... on the left turn and turning left, the sway moves to
the right, but then also to the left.
Would you agree that the sway force felt while turning around a constant
curve should be constant?
If you drive in tight circles in a car, you would feel the sway force the
entire time.
On my SimRig, I would feel the sway push to the right (correctly) but then
immediately shift left while I am still turning.
I didn't know how to calculate sway based on the math you had, so I just
had to figure out the physics/math for myself.
I'm sure your method can work as well, but it doesn't feel right at the
moment...
(The inversions you had were reversed via SimTools)
On Thu, Feb 22, 2024 at 2:47 PM vthinsel @.***> wrote:

For me, calculating a force based on weight and angles is not correct.
Sway is a force, which can be calculated based on lateral speed and only
lateral speed. No matter if your weight is 10Kg or 1000Kg, the force is the
same. We have the local velocity vector based on quaternion so it is pretty
easy to compute the local forces vector. You seem to calculate a mix of
traction loss with a mass pultiplier.
I have made a new short video focusing on
roll/pitch/yaw/heave/sway/surge/traction loss (by the way, roll/pitch/yaw
were inverted before the release I made today) :
https://youtu.be/y-9C2CPViKQ
As you can see, sway behaves pretty accurately, you can notice it when
pulling the hand brake : the driver is "pushed" on left or right side. You
can notice it also when "climbing" the road going from flat to 20° roll
angle.
Simtools filtering can be usefull, especially on heave to make it less
"jumpy", depending also on you min/max settings.
—
Reply to this email directly, view it on GitHub
https://github.com/vthinsel/GT7Proxy/pull/3#issuecomment-1960288379, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AGFDMVIW7Z2JBYGXHN2GD6LYU6VHFAVCNFSM6AAAAABDDHYEP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRQGI4DQMZXHE
.
You are receiving this because you authored the thread.Message ID:
@.***>


Remove Old Param

Fixed Issues
$ https://github.com/Expensify/Expensify/issues/236683
Tests
See the W-E PR here: https://github.com/Expensify/Web-Expensify/pull/36262
QA
N/A

@Gonals - ready for review.


fixed ipify url

#30

Why would an ipv6 fix the issue ?

Why would an ipv6 fix the issue ?

It's not an ipv6. The old url simply doesn't work and times out...

But It do works for me

But It do works for me

Just tried with a VPN. My European IP got blocked even on the first request. An American IP goes through.
Maybe if the request fails with the first IP it could try with the second one?

the commit 4235315ed080688101c7677164ebfbec3bcc308c solved the issue


fixed sensors for dashboard + fast polling

Hello! Great work on this custom integration.
I have two initial suggested changes:
The first is simple - shortening the polling interval. From some testing, the EO seems perfectly capable of providing data once per minute and this helps with seeing what's happening, especially with short charges. Maybe long term, this could be a config item?
The second is more complex but more importantly, I have changed the sensor behavior in the following ways. These are all aimed at having the sensors behave in the expected ways in Home Assistant, particularly the Energy Dashboard:

Changed the StateClass to be 'total_increasing' for both sensors. This is a match for the Home Assistant documentation on a sensor like this which increases then goes through a reset back to 0 periodically - from the documentation for 'total_increasing' : "Similar to total, with the restriction that the state represents a monotonically increasing positive total which periodically restarts counting from 0, e.g. a daily amount of consumed gas, weekly water consumption or lifetime energy consumption." . The previous setting total is only suitable if the value decreasing has some sort of information e.g. a net energy meter.

Changed the logic for when 'last_reset' is registered, and when to record a 0 value. Previously the sensor reset every time it got a reading/data from the API. Instead, it should only reset when the value for the session drops back to 0 / it's a new session. This, combined with the StateClass drop it back to 0.

My apologies in advance that although I have a developer background, Python is new to me and I may have made some basic mistakes, hopefully, the principle of what I think the code should do is at least clear. I think there are quite a few error states to manage, which this does not yet do (like a potential race condition on never getting to 0, although the 1-minute scan interval in my testing seems to solve this).
To illustrate the outcome of these changes, this is how a charge session now registers in Home Assistant:

This compares to how one could look previously:

With these changes, data correctly accumulates on the energy dashboard too (still testing to validate this further):

Just realised I should say that the first picture illustrates two charge sessions - one ending, a short gap (value stays at high number) then a second session starting with the value starting again from 0.
Please do get in touch to discuss any of this (can do Zoom etc too if useful). Would love to collaborate on this over time.

Wow, thank you so much for contributing, I'll merge now! Yes I'd be happy to collaborate more on further improvements. Lets open issues for the things which could be better, and we can find a way to work it out.

@hallidaydaniel FYI, I updated this to have a default poll interval of 5 min, and made it a 'configure' option on the integration.

@twhittock - Just been looking to see what other improvements could be made. Is there a document or anything 'official' on the EO API that the integration uses? Just trying to sketch out what else could be possible.

No official documentation that I could find. There is another repo with a library https://github.com/bfayers/pyEOCharging and a thread on the HA forum where some people have been sharing postman collections. (https://community.home-assistant.io/t/ev-charger-api-collection/378432/18)

Thanks for these @twhittock  - useful work by others!
Having a look through, I think some useful goals (in my rough order of value - yours may differ!) are:

Get current session power - useful for understanding the makeup of power load on house. Seems to require a call to an additional URL, api/session/detail. Could be done following the confirmation of the existing session?
Get the current Volts and Amps (actually required for the first point, but some more work to present as sensors I guess). Less directly useful, but there so could be handy.
Enable and Disable Charger - looks like it's possible to lock/unlock with a call. This could be presented as a binary sensor in HA - Useful for being able to automate locking when you are away from home
Pause and Unpause Session - looks like also possible, again could be presented as a binary sensor or two buttons. Could be helpful for automating when the charger actually runs (for Solar / PV setups or economy overnight electricity rates
Session History - this looks to be the same as you get in the app (makes sense). Don't think it has that much value as an integration for HA.

Let me know what you think / which one to try first - I will open an issue for it and we can get started on it.

If I may make a suggestion related to number 4, I think a sensor indicating whether the charger is "live" (i.e. in a mode where it would provide power to the car if one was plugged in) or not would every useful, ideally along with some kind of override to make it live. The play / pause button is currently greyed out in my app, and calling them via the API returns a non-error RESTful response, but doesn't seem to do anything.
From what I can see, the charger is live if:

any one of the modes (solar, op or time) returned in ChargerOpts is 1, AND the schedule for the current half hour timeslot / type of day (weekend or weekday) matches (e.g. for off-peak charging between 00:30-04:30 and no other charging, the schedule string is
0PPPPPPPP00000000000000000000000000000000000000000

It's always 50 chars, one for each half hour block, starting at 00:00, and always terminating with two additional zeroes - the P in position 2 indicates start off peak charging at 00:30, as long as opMode is also 1). Other codes are used in the schedule string to indicate solar / scheduled charging.
OR

all modes are 0 AND there is a T in the relevant timeslot in the string.

I think there are a few other possibilities, and I also haven't yet tried overlapping schedules (what letter is used /does the app allow  a timeslot to be both off-peak and scheduled/solar?), but this might be good enough to start with.
My testing so far has confirmed that by replacing chars in the string with T, except the terminating zeros, and disabling the off peak and other modes, all via a call to set the charger options (see my respository version of pyEOCharging, which will hopefully soon be accepted by the upstream owner), the charger is set to live and this in itself means that a simple set of automations in HA could initially be made along the lines of:
IF (car is plugged in and car is at home)
and (
(time = <planned departure time - X mins>
OR time = off-peak start time
)
THEN override off-peak settings and set charger to live
(the X mins before departure allows for a call to the car's defrost/preheat service)
and a similar automation to disable the charger when the car is unplugged / leaves home / offpeak time ends.
I am currently trying to teach myself Python in my spare time, but I'm starting from a very low point. I'm a lot more familiar with API calls and spotting patterns in strings etc.
I'd be happy to help with this project where I can, but with my experience, that might be in the testing department until (if!) I get up to speed with Python. I may also be able to help with deciphering the results of the API calls.
It might also be worth keeping an eye on the original pyEOCharging repository and the discussion around my pull request.
@bfayers has managed to connect to the charger's private WiFI AP - and found some additional functionality via that route (so far all I've managed to do is connect to the AP - I've asked for more info) and, if you check some of the other discussions in that repository, his aim is also some form of HA integration. I'm going to suggest that we all collaborate to work on this project.


Feature Request: man pages.

Hi.
If i install gcc over homebrew for example i don't get the man-page entry for it.
-> man gcc says no manual entry for gcc
Alex

Hi; thanks for getting in touch. We install the gcc manpages with the same names as the executables; try man gcc-5. If there's anything else you think we should have man pages for, please open a ticket for that package at Homebrew/homebrew-core. Let us know if we can help with anything else.

man gcc-5 also gives "no manual entry for gcc"

That's odd; it works for me. brew info gcc shows that it's installed? Can you share brew config -- if you have Homebrew installed somewhere besides /usr/local, you probably have to set up MANPATH.

huh.. I know i tried to install it.
Just tried again and it died at "make bootstrap" had to ctrl-c and remebered, that the first installation went the same way.
I just didn't notice because OS X has the gcc-pseudo frontend to clang...

In that case, please post this as a build failure bug report over at https://github.com/homebrew/homebrew-core/issues, with build logs (from brew gist-logs) included. (Or if you're trying to build it --without-multilib, there's an existing issue https://github.com/Homebrew/homebrew-core/issues/234 that you can chime in on (again, with your brew gist-logs from it).)
For future reference, brew ls -v <formula-name> | grep man will give you a list of the man pages installed with a formula, if you need to verify them.


Implement language button

Description
We need to implement the language button to allow users to select their preferred language for the app.
Please, locate the button bellow the Report an issue feature:

Please, also add an icon next to the text to keep the consistency across this section. For example:

Languages that needs to be enable:

French
Portuguese
Spanish

@romicolman
@Faithkenny
If it's a button not a drop-down it also needs a dialog design


Update samples and readmes to use DefaultAzureCredential

This change updates all samples and READMEs that reference EnvironmentCredential to use DefaultAzureCredential instead.  We want to advertise this as the standard credential so that it will work for most users in most cases without extra configuration.  This also allows us to update the default credentials used inside of its chain over time.

/azp run js - mgmt - pr


Pebbled: - might be a good idea to set battery level warning from a seting as 15% might be too low.

Hi there @RomanistHere.  I see my Pebble watch has been Pebbled :)
Just a note about setting the battery warning to 15%.
The battery charge tends to fall off a cliff after about 30%.  In the space of 1 hour you can be in 30% the next minute your watch has shutdown as your are out of charge.  I'm wondering if the warning level % should be set from a settings menu ?

Hi :)
I haven't experienced it yet. If that's the case I'd suggest another way to inform the user about, like making some parts of screen or elements change colors.
For example, boot and calendar icons become yellow at 30%, and red at 15%.
Or we could move kilometers to the left corner and add battery widget at right one.

dont think this is ever going to get done.  Closing.


fix(textarea): support margin bottom

:tada: This PR is included in version 1.11.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release
v1.11.1

Your semantic-release bot :package::rocket:


Chess: Move B4 to D3

Please do not change the title. Just click "Submit new issue". You don't need to do anything else :D

@JayantGoel001 done! Successfully played move b4d3 for current game. Thanks for playing!

nice one

@JayantGoel001 done! Successfully played move b4d3 for current game. Thanks for playing!

nice one

thanks 👍😃