feat: improved error message when pom dependency version not found

I'm not sure how useful this change is? shouldn't you be able to figure out the pom based on the dependency name and the property name that is being accessed?
I'm not a heavy Java user, so would appreciate if you could expand on the situation you're dealing with where the current warning is not sufficient.
(also @oliverchang or @another-rex could you approve the CI workflows to run?)

I'm not sure how useful this change is? shouldn't you be able to figure out the pom based on the dependency name and the property name that is being accessed?
I'm not a heavy Java user, so would appreciate if you could expand on the situation you're dealing with where the current warning is not sufficient.

I guess it depends on how the library is being used. I came to this from scorecard, and there for example you would currently see:
$ scorecard --repo=github.com/apache/camel-examples
Starting [SAST]
Starting [License]
Starting [Code-Review]
Starting [CII-Best-Practices]
Starting [Maintained]
Starting [Fuzzing]
Starting [Dangerous-Workflow]
Starting [Pinned-Dependencies]
Starting [Security-Policy]
Starting [CI-Tests]
Starting [Dependency-Update-Tool]
Starting [Contributors]
Starting [Binary-Artifacts]
Starting [Vulnerabilities]
Starting [Packaging]
Starting [Signed-Releases]
Starting [Branch-Protection]
Starting [Token-Permissions]
Failed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.derby:derby: property "derby-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.junit.jupiter:junit-jupiter: property "junit-jupiter-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of ch.qos.logback:logback-core: property "logback-version" could not be foundFailed to resolve version of ch.qos.logback:logback-classic: property "logback-version" could not be foundFailed to resolve version of ch.qos.logback:logback-core: property "logback-version" could not be foundFailed to resolve version of ch.qos.logback:logback-classic: property "logback-version" could not be foundFailed to resolve version of ch.qos.logback:logback-core: property "logback-version" could not be foundFailed to resolve version of ch.qos.logback:logback-classic: property "logback-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.camel:camel-test-infra-kafka: property "project.version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.camel:camel-test-infra-kafka: property "project.version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.camel:camel-test-infra-kafka: property "project.version" could not be foundFailed to resolve version of org.apache.camel:camel-test-main-junit5: property "camel.version" could not be foundFailed to resolve version of org.apache.camel:camel-test-infra-aws-v2: property "project.version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of com.fasterxml.woodstox:woodstox-core: property "woodstox-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.commons:commons-lang3: property "commons-lang3-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.camel:camel-test-infra-couchbase: property "project.version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.camel:camel-test-infra-aws-v2: property "project.version" could not be foundFailed to resolve version of org.apache.camel:camel-test-infra-postgres: property "project.version" could not be foundFailed to resolve version of org.apache.camel:camel-test-infra-cassandra: property "project.version" could not be foundFailed to resolve version of org.awaitility:awaitility: property "awaitility-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.junit.jupiter:junit-jupiter: property "junit-jupiter-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.ftpserver:ftpserver-core: property "ftpserver-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.commons:commons-lang3: property "commons-lang3-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-jul: property "log4j2-version" could not be foundFailed to resolve version of org.springframework:spring-jdbc: property "spring-version" could not be foundFailed to resolve version of org.apache.derby:derby: property "derby-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of junit:junit: property "junit-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.hsqldb:hsqldb: property "hsqldb-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.camel:camel-test-infra-kafka: property "project.version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of io.rest-assured:kotlin-extensions: property "rest-assured-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.camel:camel-test-infra-minio: property "project.version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.camel:camel-test-infra-mongodb: property "project.version" could not be foundFailed to resolve version of io.rest-assured:rest-assured: property "rest-assured-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.camel:camel-log: property "project.version" could not be foundFailed to resolve version of org.apache.camel:camel-caffeine: property "project.version" could not be foundFailed to resolve version of org.apache.camel:camel-aws2-kinesis: property "project.version" could not be foundFailed to resolve version of org.apache.camel.example:resume-api-common: property "project.version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.camel:camel-test-infra-aws-v2: property "project.version" could not be foundFailed to resolve version of org.apache.camel:camel-caffeine: property "project.version" could not be foundFailed to resolve version of org.apache.camel:camel-cassandraql: property "project.version" could not be foundFailed to resolve version of org.apache.camel.example:resume-api-common: property "project.version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.camel.example:resume-api-common: property "project.version" could not be foundFailed to resolve version of org.apache.camel.example:resume-api-common: property "project.version" could not be foundFailed to resolve version of org.apache.camel.example:resume-api-common: property "project.version" could not be foundFailed to resolve version of org.apache.camel.example:resume-api-common: property "project.version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.activemq:activemq-broker: property "activemq-version" could not be foundFailed to resolve version of org.apache.activemq:activemq-client: property "activemq-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.junit.jupiter:junit-jupiter-api: property "junit-jupiter-version" could not be foundFailed to resolve version of org.junit.jupiter:junit-jupiter-engine: property "junit-jupiter-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.camel:camel-test-infra-pulsar: property "project.version" could not be foundFailed to resolve version of org.springframework.security:spring-security-web: property "spring-security-version" could not be foundFailed to resolve version of org.springframework:spring-web: property "spring-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.jboss.arquillian.container:arquillian-tomcat-embedded-9: property "arquillian-tomcat-version" could not be foundFailed to resolve version of org.apache.tomcat.embed:tomcat-embed-jasper: property "tomcat-version" could not be foundFailed to resolve version of org.junit.jupiter:junit-jupiter: property "junit-jupiter-version" could not be foundFailed to resolve version of org.jboss.arquillian.junit5:arquillian-junit5-container: property "arquillian-version" could not be foundFailed to resolve version of io.rest-assured:rest-assured: property "rest-assured-version" could not be foundFailed to resolve version of org.apache.activemq:activemq-broker: property "activemq-version" could not be foundFailed to resolve version of org.apache.activemq:activemq-client: property "activemq-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.springframework:spring-context: property "spring-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-core: property "log4j2-version" could not be foundFailed to resolve version of org.apache.logging.log4j:log4j-slf4j2-impl: property "log4j2-version" could not be foundFinished [Vulnerabilities]
Finished [Packaging]
Finished [Signed-Releases]
Finished [Branch-Protection]
Finished [Token-Permissions]
Finished [SAST]
Finished [License]
Finished [Code-Review]
Finished [CII-Best-Practices]
Finished [Maintained]
Finished [Fuzzing]
Finished [Dangerous-Workflow]
Finished [Binary-Artifacts]
Finished [Pinned-Dependencies]
Finished [Security-Policy]
Finished [CI-Tests]
Finished [Dependency-Update-Tool]
Finished [Contributors]

It's not super obvious those messages have anything to do with pom.xml's in that repo in the first place, let alone which ones. While there's of course only so much you can do from the osv-scanner side, and ultimately perhaps something should change on the scorecard side, I'd say the change in this PR would be a nice improvement already.

(on an aside, I am annoying I forgot to add the \n at the end of that output 🙄🤦)
Yeah I agree that output is not the best, though I think the problem is more with these outputs being done inefficiently because at the time I didn't want to commit to a specific way to surface them given it would impact the library function signatures and be just a whole "thing"...
(I've explored a solution to this in #186 - any thoughts you'd have would be appreciated!)
My concern with this change is that we're now expecting and promising more info out of the file, and that typically means more cases to handle (i.e. what happens if it's not there) - however I assume these two properties are always present in a pom.xml anyway (+ the way we're using them shouldn't actually cause a panic) and I agree that this is an improvement at least until something like #186 is landed (which'll allow scorecard to actually handle these warnings)

(on an aside, I am annoying I forgot to add the \n at the end of that output roll_eyesfacepalm)

:)

Yeah I agree that output is not the best, though I think the problem is more with these outputs being done inefficiently because at the time I didn't want to commit to a specific way to surface them given it would impact the library function signatures and be just a whole "thing"...
(I've explored a solution to this in #186 - any thoughts you'd have would be appreciated!)

Yeah, something more 'controlled' like that would be great long-term, but coming up with what exactly that should look like is indeed not so easy :) .

My concern with this change is that we're now expecting and promising more info out of the file, and that typically means more cases to handle (i.e. what happens if it's not there) - however I assume these two properties are always present in a pom.xml anyway (+ the way we're using them shouldn't actually cause a panic)

I actually tested this with some poms that didn't have a GroupID and that indeed looked pretty OK.

I agree that this is an improvement at least until something like #186 is landed (which'll allow scorecard to actually handle these warnings)

Jup - it should probably still print it, but for example prefixing it with the "check" it belongs to and perhaps the file it's processing would be really useful to be able to do, and looks like it might be fairly easy to plug into that structure (Famous Last Words).

another-rex removed this pull request from the merge queue due to the queue being cleared

Does this mean there is action required on my part?

another-rex removed this pull request from the merge queue due to the queue being cleared

Does this mean there is action required on my part?

Ah no, was just trying out the merge queue, turns out still doesn't quite work for our repo. Going to merge this in soon.


add a "do you like the app? you can review"

@nicolasburtey mentioned in a separate chat that we should add this to increase our ratings in the app store.  We should probably look to gain feedback when the user has completed an important action e.g. made a successful transaction. I found a good blog post which talks about collecting user feedback and I also found a library which ties into the native android/ios rating systems called react-native-rate.

The flow I had in my last app was

Asking the user if they are enjoying the app.

If yes:

Ask if they want to review.
If yes, prompt for a review.

If no:
2. Ask if they need help from support.
3. If yes, bring them to support.

This feature request has now a bounty on it. More info here

I'm interested in solving this issue.


inject-dockerfile tekton task

A tekton task which creates a dummy dockerfile and injects it into a workspace

in this option we are creating the Dockerfile manually using the echo command,
it may be better to create the Dockerfile in the task's repo and copy it into the workspace instead, please tell me what you think.

Also, @ShakedAviv50313. Please rebase main and try adding your task to the ./tekton/rhtap-consoledot-frontend-build-tests-pull-request.yaml so we can have a way of testing our tasks directly in RHTAP for a frontend example app


Add support for making MITMing selective depending on peer host/port

This is helpful when needing to deal with certificate pinned applications at the same time as the intended MITMable traffic, when both must use the same proxy.
Note: This pull request should be rejected if pull request #402 is merged first. #402 upgrades the java version of this project to 8, at which point it would be better to make shouldMITMPeer a default method inside MitmManager instead of in the extension interface SelectiveMitmManager.

The travis ci build for java 8 failed for this pull request, but succeeded in my repository (https://travis-ci.org/ScottDennison/LittleProxy/jobs/350944465), I'm not knowledgeable enough with travis ci to know why.
The travis ci build for java 7 fails in master already, and is also fixed by #402.


Feature/55 non printable characters

Going to merge this as it is urgent. @felipemmelo, if you find something I can fix it later.


Electron Bug - Titlebar missing buttons in Wayland (Fixed upstream)

A few weeks ago the new version no longer had minimize/maximize/close buttons in the titlebar under GNOME wayland.  I looked into it then and it was electron bug: https://github.com/electron/electron/issues/38786
Anyways its fixed now: https://github.com/electron/electron/pull/39003
So would be nice for a new build of webcord to fix this bug cuz it's really annoying to use without a close button.
Thanks.

An Electron 25 release was made with the fix, see https://github.com/electron/electron/releases/tag/v25.3.0

I'm not sure about updating, I might need to downgrade to v24 if v25 seems to be an issue for running WebCord on Windows and macOS.

If you installed WebCord with Flatpak, a workaround I found is to use Flatseal and select 'WebCord' then go the 'Socket' section and disable 'Wayland windowing system', WebCord will now run on Xwayland now with window buttons.
I tested screen sharing with the software running in Xwayland and screen sharing works as expected. Experience may vary.

Closing as Electron was updated to v26.0.0…


[WIP] Speed up codegen and take clippy lints

Taking care of some low-hanging fruit to try to speed up codegen.  Mostly around ensuring we use buffered file IO and reducing the amount of temporary strings we use without getting too invasive.
Changes to generated code are not yet included and won't be until I get clippy to be happier. 😉

I did more performance analysis using callgrind and confirmed Rustfmt is the cause of codegen performance woes. :crying_cat_face: This matches the rough output from the stopwatch timing added in https://github.com/rusoto/rusoto/pull/811 .

Cumulative changes show a 10% decrease in compile times on Linux (stable and beta).  This four or five minutes should keep us going for a bit more. 😉
Next up: verify all integration tests pass and then check in the generated code.

Integration tests pass with the following exceptions that are all known:

CloudHSM: https://github.com/rusoto/rusoto/issues/822
S3: https://github.com/rusoto/rusoto/issues/820
XRay: https://github.com/rusoto/rusoto/issues/823
SQS: https://github.com/rusoto/rusoto/issues/821

The S3 and SQS tests have PRs pending as fixes (https://github.com/rusoto/rusoto/pull/812 and https://github.com/rusoto/rusoto/pull/824).

Gonna get this in so we can have better compilation times. 😄


Optional deps 228

Description
This PR sets up the framework for optional dependencies in caikit and refactors the existing dependency management into the following sets:
User Deps

caikit: Dependencies for caikit.core
caikit[runtime-grpc]: Dependencies for the grpc runtime
caikit[all]: All dependencies for user functionality in caikit (currently core + runtime-grpc)

Developer Deps

dev-test: Dependencies for running unit tests
dev-docs: Dependencies for building documentation
dev-fmt: Dependencies for formatting, linting, and other stylistic checks
dev-build: Dependencies for building the wheel
all-dev: All developer dependencies (including all user dependencies)

NOTE: The choice of - as a delimiter seems to be required when using flit as our build tool. I had originally used . which seemed a little cleaner to me, but the flit build commands used to build the temporary wheel that tox then used to create the venvs was not being built correctly.

NOTE: I was a bit concerned about using caikit[...] inside the all deps as references, so I went digging. When the wheel gets built locally, the extras metadata looks like the following:
Requires-Dist: alchemy-config>=1.1.1,<2.0.0
Requires-Dist: alchemy-logging>=1.0.4,<2.0.0
Requires-Dist: anytree>=2.7.0,<3.0
Requires-Dist: docstring-parser>=0.14.1,<0.16.0
Requires-Dist: grpcio>=1.35.0,<2.0,!=1.55.0
Requires-Dist: ijson>=3.1.4,<3.3.0
Requires-Dist: munch>=2.5.0,<4.0
Requires-Dist: numpy>=1.20,<2
Requires-Dist: protobuf>=3.19.0,<5
Requires-Dist: py-to-proto>=0.3.0,<0.4.0,!=0.2.1
Requires-Dist: PyYAML>=6.0,<7.0
Requires-Dist: semver>=2.13.0,<4.0
Requires-Dist: six>=1.16.0,<2.0.0
Requires-Dist: tqdm>=4.59.0,<5.0.0
Requires-Dist: caikit[runtime-grpc] ; extra == "all"
Requires-Dist: caikit[all, dev-test, dev-docs, dev-fmt, dev-build] ; extra == "all-dev"
Requires-Dist: flit==3.8 ; extra == "dev-build"
Requires-Dist: sphinx>=4.0.2,<8.0 ; extra == "dev-docs"
Requires-Dist: sphinx-autoapi>=2.1.0 ; extra == "dev-docs"
Requires-Dist: sphinx-rtd-theme~=1.2.1 ; extra == "dev-docs"
Requires-Dist: pre-commit>=3.0.4,<4.0 ; extra == "dev-fmt"
Requires-Dist: pylint>=2.16.2,<3.0 ; extra == "dev-fmt"
Requires-Dist: pydeps==1.12.3 ; extra == "dev-fmt"
Requires-Dist: pytest>=6.2.5,<7.0 ; extra == "dev-test"
Requires-Dist: pytest-cov>=2.10.1,<3.0 ; extra == "dev-test"
Requires-Dist: pytest-html>=3.1.1,<4.0 ; extra == "dev-test"
Requires-Dist: tls_test_tools>=0.1.1 ; extra == "dev-test"
Requires-Dist: wheel>=0.38.4 ; extra == "dev-test"
Requires-Dist: grpcio-health-checking>=1.35.0,<2.0 ; extra == "runtime-grpc"
Requires-Dist: grpcio-reflection>=1.35.0,<2.0 ; extra == "runtime-grpc"
Requires-Dist: prometheus_client>=0.12.0,<1.0 ; extra == "runtime-grpc"
Requires-Dist: py-grpc-prometheus>=0.7.0,<0.8 ; extra == "runtime-grpc"
Project-URL: Source, https://github.com/caikit/caikit
Provides-Extra: all
Provides-Extra: all-dev
Provides-Extra: dev-build
Provides-Extra: dev-docs
Provides-Extra: dev-fmt
Provides-Extra: dev-test
Provides-Extra: runtime-grpc

In particular, the fact that you see Requires-Dist: caikit[all, dev-test, dev-docs, dev-fmt, dev-build] ; extra == "all-dev" worried me that a pip install caikit[all-dev] would go back out to the internet to resolve caikit[all], thus breaking installs from local wheel files.
According to the internet, however, this is actually safe. Specifically:

Since packagename is already in the set of dependencies to be resolved, mentioning it in its own dependencies doesn’t cause a cycle.


Preventing path traversal by plugins?

Hi! sqlc looks neat and I'm playing around with it.
The docs make this statement:

WASM plugins are fully sandboxed; they do not have access to the network, filesystem, or environment variables.

This caused me to look into how filesystem writes are performed. Am I misinterpreting the code, or does the use of path.Join allow for path traversal attacks? e.g. if a plugin emits a path beginning with ../../../. I don't speak any Go, but my read of the Clean function (which appears to be applied automatically to the result of Join) is that it only normalizes .. inside a path and preserves it at the front.
Apologies in advance if I've missed the defense against this.

Excellent question! As the documentation states, WASM plugins do not have any filesystem access. The output of a plugin is specified in the out parameter. I need to investigate if plugin output can escape that directory using "..".

@cbiffle and I were chatting about this before he filed the issue, and I was able to confirm they can (as he was writing it up!). I applied this diff to the Rust plugin demo:
--- a/src/main.rs
+++ b/src/main.rs
@@ -21,7 +21,7 @@ pub fn serialize_codegen_response(resp: &plugin::CodeGenResponse) -> Vec<u8> {

 pub fn create_codegen_response() -> plugin::CodeGenResponse {
     let mut file = plugin::File::default();
-    file.name = "hello.txt".to_string();
+    file.name = "../../../../../../../../../tmp/escaped.txt".to_string();
     file.contents = "Hello World".as_bytes().to_vec();

     let mut resp = plugin::CodeGenResponse::default();

and after running it:
cat /tmp/escaped.txt
Hello World

WASM plugins do not have any filesystem access

It appears that they don't have filesystem access in the WASI sense, but that's a narrower statement. FWIW I also can't work out how to read the filesystem from a plugin, only write it, so, that's good.

@jgallagher that looks conclusive to me. I'll get a PR up to fix the behavior. This could be a breaking change, so I'll need to decide if it can write to locations next to and under sqlc.json or the specified out directory.
Are you using sqlc at Oxide?

Assuming this gets fixed, I might suggest rephrasing that statement to "plugins can only write files into their output directory," since "no access to the filesystem" suggests to me that they couldn't, say, deposit malware to it or fill it up -- and to be useful plugins the writing part is important!

Are you using sqlc at Oxide?

Not currently, but John and I both think it's a really neat idea.

Yeah, it's a bit confusing to say that the plugins can write files, as they can't due to the sandbox. sqlc takes the output from the plugin and uses that to write the output files. I'm sure we can figure out some language that makes it clear what's happening.

nod Something like "plugins can send commands to sqlc to have files created in the output directory, but can't directly access the filesystem"

This doesn't include the necessary tests yet, but I believe https://github.com/sqlc-dev/sqlc/pull/3195 should fix the issue.

Confirmed:
% sqlc-dev generate
# package sqlc-gen-rust
error generating code: invalid file output path: /tmp/escaped.txt

Thanks!


test/e2e/suite/issuers/acme: fix dropped error

This fixes a dropped error in test/e2e/suite/issuers/acme.
NONE

/ok-to-test
/approve
/lgtm
Good catch! Thanks for this!

/kind cleanup


Move native API docs to their own folder for easier updating

This will allow us to make release cutting simpler by simply deleting all of the contents in the native-api folder and copying the new docs over, rather than needing to merge them with the JS docs and figure out which docs were supposed to be deleted.

[X] validated that links can be followed
[ ] validate the release cutting scripts work ok

Microsoft Reviewers: Open in CodeFlow

You might want to check by locally "snapping" a new version, seeing what files get created, and verify running the site locally that the pages show up as you expect in the browser.


Installing from repo on Arch throws hooks error

I wanted to give this one a shot and followed the instructions, but I get an error with nvidia.hook
error: failed to commit transaction (conflicting files)
phyOS-sysconf: /etc/pacman.d/hooks/nvidia.hook exists in filesystem
I moved nvidia.hook to nvidia.hook.bkp and the installation proceeded, but it created a bunch of files in /etc/pacman.d/hooks. Is this necessary?
$: paru -S phyOS-dwm-beta

community/gum                    0.6.0-1         26,19 MiB
phyOS-repo/phyOS-sysconf         2.0-03          14,94 MiB
phyOS-repo/phyOS-system-scripts  2.7-02           0,08 MiB
phyOS-repo/phyOS-dwm-beta        1.2.5_beta-01    0,17 MiB

Total Installed Size:  41,38 MiB

:: Proceed with installation? [Y/n]
(4/4) checking keys in keyring                                                 [--------------------------------------------] 100%
(4/4) checking package integrity                                               [--------------------------------------------] 100%
(4/4) loading package files                                                    [--------------------------------------------] 100%
(4/4) checking for file conflicts                                              [--------------------------------------------] 100%
error: failed to commit transaction (conflicting files)
phyOS-sysconf: /etc/pacman.d/hooks/nvidia.hook exists in filesystem
Errors occurred, no packages were upgraded.

Removed the hook, can you try again?

Now it does not create any

Does it install correctly? Just removed the nvidia hook.


docs: Japanese guide

add Japanese guide.
Here is what we discussed.
https://github.com/edmundhung/conform/pull/507

This is now available on both https://ja.conform.pages.dev and https://ja.conform.guide


[Feature Request] Unicode (other special character) support

As mentioned in #18, we are very restrictive in our character sets today. We had discussed relaxing this late last year, but made no changes yet. We should open up the character sets and give warnings where we think it might be applicable

I will do that! I have a parser written for unicode chars!

@tnowacki checkout, I have filtered most useless unicode in my last parser → https://github.com/sap-lang/sap-lang/blob/master/src/utils.rs

We probably don't want unicode characters anywhere but in comments. I'll take a look though when we get to this, thanks!


chrono-tz feature: Implement GraphQLType for chrono-tz

aims to fix https://github.com/graphql-rust/juniper/issues/516

Test book code examples via skeptics fails. The errors in the console seem unrelated though?

Looking at current open PRs after the new CI and hooks were added, this particular step seems to be failing for all

:( broke all the tests, fixing this

@LegNeato while all the tests are passing now, the chrono-tz feature is neither part of documentation nor part of CI pipeline. How can I remedy this?

@LegNeato while all the tests are passing now, the chrono-tz feature is neither part of documentation nor part of CI pipeline. How can I remedy this?

@LegNeato could you review this again?


TenantId value returns WorkspaceId

Describe the bug
When you query data in Sentinel the TenantId value displays the workspaceId which is incorrect and misleading.
I have checked this in several Sentinel workspaces and find the same result.
To Reproduce
query that shows a quick summary of all tables and the two fields in question and their values (if you have data in them).
you can see that the table 'Anomalies' and 'SentinelHealth' also shows the WorkspaceId which matches the TenantId!
search *
| where TimeGenerated > ago(1d)
| summarize by TenantId, WorkspaceId, $table
| sort by WorkspaceId
Expected behavior
TenantId value should reflect the actual Azure TenantId and not WorkspaceId
Screenshots

Hello @Gareze, thanks for flagging this, we will soon get back to you on this.

Hi @Gareze, we have updated the concerned team about this issue, will update you once we hear back from them, thanks!

Hello @Gareze, as per the documentation, the " TenantId" column in 'Anomalies' and 'SentinelHealth' are supposed to contain the log analytics workspace id for reference sharing the documentation link - https://learn.microsoft.com/en-us/azure/azure-monitor/reference/tables/anomalies Thank you.

Hi @v-sudkharat,
But it is not just these two tables, the "TenantId" value for the other tables in my example is aslo reflecting the workspace ID. I also think that this is very confusing and we had someone on our team reporting the incorrect Tenant ID due to this field value which caused a lot of short lived confusion.
Why call the field TenanId when you insert the workspace ID???? just makes no sense...

Hi @Gareze, I apologize for the confusion. The "TenantId" in the context of Log Analytics workspace is not the same as the Azure Active Directory Tenant ID. The "TenantId" in Log Analytics refers to the workspace ID, which is a unique identifier for the Log Analytics workspace.
If you need to retrieve the Azure Active Directory Tenant ID, you can use the Azure CLI or Azure Portal. Here's an example using Azure CLI command: az account show --query tenantId -o tsv

Hi @v-sudkharat,
I understand what you referring too but I still think it is confusing to anyone if "TenantId" means differened things in different contexts especially when the "WorkspaceId" field also exists.
The TenantId lookup which was carried out by a team member was related to finding out the TenantId in which the Sentinel alert triggered using KQL since we have many different Tenant's connected.
Now that I understand the rather strange logic on this we know not to trust that field output and will find other means, I just thought it is a mistake to be honest so reported it.


Ft periodic reports 148476449

What does this pull request do:-

It enables the user to set whether they will receive reports daily, weekly or monthly.

Changes Unknown when pulling d21e6f970f9660240d8b2a4367760be0da8e62f9 on ft-periodic-reports-148476449 into ** on master**.


GPU Acceleration

Is it possible to use NVIDIA CUDA acceleration?
In order to use body-pix or deeplab with 15+ FPS performance.

From the documentation here: https://www.tensorflow.org/lite/performance/delegates I don't think there is a Linux/x86 targetted GPU delegate that could be used, but happy to be proven wrong by someone!
Sticking to CPU, we may be able to improve by explicitly building the XNNPACK backend: https://blog.tensorflow.org/2020/07/accelerating-tensorflow-lite-xnnpack-integration.html

Oho, neat! The main reason for picking TFLite in the first place was that it has a workable CPU backend, "regular" Tensorflow basically requires a Nvidia GPU (at least as of ~ 1 year ago). And I guess part of the appeal of backscrub is that it doesn't require anything fancy in terms of hardware.

I have just discovered that there is an OpenCL delegate (https://github.com/tensorflow/tensorflow/tree/v2.4.1/tensorflow/lite/delegates/gpu), which can be built via CMake (only tested by Google on Android cross-builds but should work on Linux), however it's fighting me at the moment courtesy of untested Linux headers, and yet-another-flatbuffers-schema that needs compiling..

XNNPACK results you've got are very promising!
I was looking through internet and found that tflite models are not so good for GPU acceleration.
It is much better in TF no the LITE version. But since the whole project is based on tflite it probably will take a lot of effort to refactor to TF. Or am I wrong?

btw: https://github.com/mikan3rd/react-body-pix-sample

I was looking through internet and found that tflite models are not so good for GPU acceleration.

Thanks for the feedback and research @SamSpiri. Given tflite is built for use on mobile devices and specifically for their GPUs I'm a little surprised you found tflite models are 'not so good for GPU' (perhaps it's that desktop GPUs might have support for 64bit floats, mobile tends not to?) - got any links to what you discovered please?
As @floe notes above, this project uses tflite so it can operate without a GPU, and switching to full-fat Tensorflow will kill that off, so for now I think we're likely to stay with tflite. Code-wise, it wouldn't be particularly difficult/different to use full TF, and as/when we get a more pluggable structure I would expect someone to write the appropriate plugin :smile:

Hacked together GPU support now in my tree: https://github.com/phlash/backscrub/tree/xnnpack-test (yes, this is now a badly named branch!) Unable to test on my T430 as it does not support double precision (64bit) floats and crashes when loading any model :disappointed: If you wish to crash your own machine.. set BACKSCRUB_GPU=yes in your environment before running.
This branch must be built with cmake, as there is no makefile support for GPU and XNNPACK in Tensorflow Lite tree.
I also discovered that adding in a custom build rule to compile flatbuffer schemas to headers magically also builds  the flatc compiler and uses it... CMake is a bit scary at times!

That works just amazing! Even without GPU it works super quick with just 20% cpu usage on deeplab and body-pix.
But when run with GPU than CPU usage drops to 10% and GPU is barely used (only 80Mb ram and few % compute)
I'll share later my dockerfile and docker-compose to build and run the whole thing that I used.

https://github.com/floe/backscrub/pull/81

@phlash : I have tested your code with XNNPACK and it's simply amazing. With the selfie model I go from more than 200% CPU to around 100%. Very nice since my hardware an old iMac i7 2.93Ghz running Ubuntu cannot do GPU.

Now that we have CMake builds in, I'll get the XNNPACK stuff into main, so this should be default performance for everyone.

As a side comment, it's also running on ARM Cortex 53 (Pinephone running Mobian) with around 5FPS with the selfie model and the gain with XNNPACK is nearly the same going down from 252% CPU to 185% CPU. I had though to compile with Tensorflow 2.5 since there is a bug in XNNPACK on version 2.4.1 which prevents compiling on ARM. So it would be nice to go towards that version if possible.

I had though to compile with Tensorflow 2.5 since there is a bug in XNNPACK on version 2.4.1 which prevents compiling on ARM. So it would be nice to go towards that version if possible.

Can you open up a new issue for this, so we have something to track this? As API of TF shouldn't change too much, this might be easily doable.

TF v2.5.0 now awaiting test in PR #88

main branch builds nicely with TF 2.5 and XNNPACK, so this can be closed?


Some Chinese symbols are grabled, when you're using PdfAsync() method;

Description
Some of Chinese letters are grabled after PDF creating.
Complete minimal example reproducing the issue
Please try with these chinese symbols 屯門區 entered on your page in browser. And after that use Puppeteer and take PdfDataAsync method.
// Arrange
await using var browser = await Puppeteer.LaunchAsync(new LaunchOptions
{
Headless = true
});
await using var page = await browser.NewPageAsync();
if (headers != null && headers.Any())
{
await page.SetExtraHttpHeadersAsync(headers);
}
            await page.SetJavaScriptEnabledAsync(true);

            var navigationOptions = new NavigationOptions {
                WaitUntil = new[]
                {
                    WaitUntilNavigation.Load,
                    WaitUntilNavigation.Networkidle2
                }
            };
            await page.GoToAsync(url, navigationOptions);

// Act
var pdfData = await page.PdfDataAsync(new PdfOptions
{
PrintBackground = true,
FooterTemplate = footerTemplate,
DisplayHeaderFooter = true,
MarginOptions = new PuppeteerSharp.Media.MarginOptions
{
Bottom = "100px"
}
});
// Assert
Expected behavior:
屯門區 instead of those squres with crosses shown on screenshot
Actual behavior:

Versions
Version of PuppeteerSharp is 2.0.4
.NET runtime and version is .NET Core 3.1.

@rodion-andreev-d8 do you get that if you print to PDF using Chrome?

@kblok

all internally

But, do you get that if you print to PDF using Chrome navigating the page manually?

@kblok More info. When i run this app locally using Visual Studio all is working well. When a build it using docker i catch that issue.
this is a part of dockerfile

Hi @kblok I feel since we are using Convert.FromBase64String() we may encounter this problem can we try Encoding.Unicode.GetBytes() in  ProtocolStreamReader.cs ReadProtocolStreamByteAsync()
Guide me if I'm on right path, if we can do this to fix this issue

I'm not quite sure @venky8951. We are getting the content of the PDF there, not the text itself.

@kblok @venky8951 SOLVED. We had to use new fonts for asian symbols.
we/ve updated like here

Closed due to inactivity. Feel free to reopen it if needed.


Multi select does not respect the boundaries of the canvas

Discussed in https://github.com/wbkd/react-flow/discussions/3233

Originally posted by varghesethomase July 17, 2023
I would like to understand if there is a way to limit dragging of the nodes towards the canvas boundary when a multi select is performed.
Currently, when two widgets are selected, and is moved to the canvas boundary, it forces the farthest element from the canvas boundary to move towards it till they overlap. What is more desirable should be that when multi select is performed, the selected boundaries of the "multi selected" area is respected when moved towards the canvas boundary and retains intact how it is when the canvas boundary is touched by the selected element closest to it. I am attaching a video of how it behaves currently. I would appreciate your support on how to achieve it.
https://github.com/wbkd/react-flow/assets/5429756/214b3e98-69b3-457a-9dfa-36fe6fc9b6c6

I'm curious about the reference to "boundaries of the canvas". By this, are you referring to nodeExtent / translateExtent? In working with reactflow, I never found a clearly documented option to actually have hard "boundaries" or a "size" for the canvas. There's a finite viewport for it, but the ReactFlow itself seems always to be an infinitely panning surface (of course subject to translateExtent and nodeExtent) like Google Maps or similar.

fixed in v11.8.0
@bcrudolp this refers to nodeExtent.


SSNTP version checking

We do not check the SSNTP version for doing backward compatibility.
We need to e.g. keep track of supported commands for a given version.

FWIW, there are quite a few options for battle tested messaging/RPC mechanisms. protobuf  and gRPC seems to be quality options packed with features including message versioning and optional fields to be able to evolve messages over time keeping backward compatibility.
gRPC uses HTTP2, which could be an opportunity to reduce the number of certificates needed as well (along with all the niceties from HTTP2 and probably a better behaviour with corporation proxies in case clients/servers need to talk across those boundaries).
https://developers.google.com/protocol-buffers/
https://developers.google.com/protocol-buffers/docs/proto3
http://www.grpc.io/
Sorry for the random remark out of nowhere, was just idly looking at the list of issues.

@dlespiau thanks for the input.
We use Go's gob encoding  for the current SSNTP implementation. I'd qualify gob encoding as a Go optimized protobuf even though I assume the Go protobuf bindings may use gob encoding.
We actually do not want to use HTTP (1.1 or 2) for Ciao inter components IPC because we want a fully asynchronous protocol. SSNTP is a very simple application protocol built on top of TLS, so it'll be as good with corporate proxies as any other TLS based protocol (like e.g. HTTPS).

Ah! that explains things, I was sure I was missing something. Thanks!


Caddy 0.9 compatibility

This has been requested by @onodera-punpun for his/her photo sharing app.
Caddy 0.9 has changed how plugins have to register with Caddy.

[x] write integration tests (in addition to the already existing unit tests) to demonstrate that the plugin works as intended
[x] create a new branch that works with Caddy 0.8.x
[x] switch to 0.9 on master
[x] mark by which commit compatibility with 0.9 had been established

Assessment of point in time readiness: By the end of this Wednesday.

Please note that I did not receive a PR from Caddy's authors (see: https://caddyserver.com/blog/caddy-0_9-released#breaking-changes-to-the-package-structure ), a favour that would be much appreciated given that I contributed to Caddy.

The new branch for Caddy 0.8 is for-caddy-0.8.
Unit tests and integration tests are part of a »build pipeline«, and its status page has a link in the README.
For example: https://hub.blitznote.com/src/caddy.upload/pipelines/103

Closed by wmark/caddy.upload@82c3e475be7b1afeccdc293c6b9bad2edb5f7003
See also https://hub.blitznote.com/src/caddy.upload/pipelines/109

@wmark I am still not able to see caddy.upload @ https://caddyserver.com/download. Do I need to raise an issue in some other repo?

It's up to Caddy's author, mholt (mholt6 on Twitter), to list and include the plugin again. (He has unlisted the plugin because it's been made compatible not before, but after the launch of Caddy 0.9.)
In the meanwhile you can inject the plugin like done here when compiling Caddy: https://github.com/wmark/caddy.upload/commit/82c3e475be7b1afeccdc293c6b9bad2edb5f7003

You could download the build artifact from the CI server if you want to test the plugin:

Go to the latest build. Click on the icon build: success that is displayed with the README.
It's currently: https://hub.blitznote.com/src/caddy.upload/pipelines/109
Scroll down to section build and click on the button passed next to any build.
(Without a suffix like _aarch64 or _amd64_sse4 it's for architecture amd64.)
Click on download or browse on the top right side.
(There's no difference between debug and release builds.)

Please note that I did not receive a PR from Caddy's authors (see: https://caddyserver.com/blog/caddy-0_9-released#breaking-changes-to-the-package-structure ), a gesture that would be much appreciated given that I contributed to Caddy.

I couldn't find this repo based on the import path :P - I didn't know where your code was stored in a way that I could contribute.

Looks like cert from blitznote.com/src/caddy.upload was not accepted by go get. Only if -insecure flag is passed, go is able to get & import it :(


	Checking Ruby installation
	Ruby installation found.

	Checking Home Brew installation
	Home Brew installation found.

	Checking dependencies.
	Checking automake ... 		 automake found
	Checking Libtool ... 		 Libtool found
	Checking OpenSSL ... 		 OpenSSL found
	Checking llvm ... 		 llvm found
	Checking wget ... 		 wget found
	Checking CMake ... 		 CMake NOT found.
	Checking Boost ... 		 Boost found
	Checking GMP ... 		 GMP found
	Checking gettext ... 		 gettext found
	Checking MongoDB ... 		 MongoDB found
	Checking Doxygen ... 		 Doxygen found
	Checking Graphviz ... 		 Graphviz found
	Checking Python3 ... 		 Python3 found

	The following dependencies are required to install EOSIO.

	1. CMake

Do you wish to install these packages?
1) Yes
2) No
#? 1
	Updating Home Brew.
Already up-to-date.
	Installing Dependencies.
==> Downloading https://homebrew.bintray.com/bottles/cmake-3.11.0.high_sierra.bottle.tar.gz
######################################################################## 100.0%
==> Pouring cmake-3.11.0.high_sierra.bottle.tar.gz
==> Caveats
Emacs Lisp files have been installed to:
  /usr/local/share/emacs/site-lisp/cmake
==> Summary
🍺  /usr/local/Cellar/cmake/3.11.0: 2,363 files, 32.9MB
Unlinking /usr/local/Cellar/cmake/3.11.0... 452 symlinks removed
Linking /usr/local/Cellar/cmake/3.11.0... 452 symlinks created

	Checking MongoDB C++ driver installation.
	Mongo C++ driver found at /usr/local/lib/libmongocxx-static.a.

	Checking secp256k1-zkp installation.
	secp256k1 found at /usr/local/lib/

	Checking LLVM with WASM support.
	WASM found at /usr/local/wasm/bin/

>>>>>>>> ALL dependencies sucessfully found or installed . Installing EOS.IO

-- Using custom FindBoost.cmake
-- Boost version: 1.66.0
-- Found the following Boost libraries:
--   thread
--   date_time
--   filesystem
--   system
--   program_options
--   signals
--   serialization
--   chrono
--   unit_test_framework
--   context
--   locale
--   iostreams
-- Configuring Eos on OS X
-- binaryen building with -std=c++11
-- binaryen building for platform x86-64
-- binaryen building with -Wall
-- binaryen building with -Werror
-- binaryen building with -Wextra
-- binaryen building with -Wno-unused-parameter
-- binaryen building with -fno-omit-frame-pointer
-- binaryen building with -fPIC
-- binaryen building with -O2
-- binaryen building with -UNDEBUG
-- Using WASM clang => /usr/local/wasm/bin/clang
-- Using WASM llc => /usr/local/wasm/bin/llc
-- Using WASM llvm-link => /usr/local/wasm/bin/llvm-link
-- Found Secp256k1: /usr/local/lib/libsecp256k1.a
-- Configuring fc to build on Unix/Apple
-- zlib found
-- bzip2 found
-- Configuring SoftFloat
-- Using custom FindBoost.cmake
-- Boost version: 1.66.0
-- Found the following Boost libraries:
--   thread
--   date_time
--   filesystem
--   system
--   chrono
--   unit_test_framework
--   locale
-- Configuring ChainBase on OS X
-- Using custom FindBoost.cmake
-- Boost version: 1.66.0
-- Found the following Boost libraries:
--   thread
--   date_time
--   filesystem
--   system
--   chrono
--   program_options
--   unit_test_framework
--   locale
-- Configuring AppBase on OS X
-- Found bsoncxx headers: /usr/local/include/bsoncxx/v_noabi
-- Found bsoncxx library: /usr/local/lib/libbsoncxx-static.a;/usr/local/lib/libbson-static-1.0.a;-D_THREAD_SAFE
-- Found mongocxx headers: /usr/local/include/mongocxx/v_noabi;/usr/local/include/bsoncxx/v_noabi
-- Found mongocxx library: /usr/local/lib/libmongocxx-static.a;/usr/local/lib/libbsoncxx-static.a;/usr/local/lib/libbson-static-1.0.a;-D_THREAD_SAFE;/usr/local/lib/libmongoc-static-1.0.a;/usr/local/lib/libbson-static-1.0.a;-D_THREAD_SAFE;-lsasl2;-framework Security;-framework CoreFoundation;-lz;-lresolv
-- Git commit revision: 124c62d0
-- Git commit revision: 124c62d0
CMake Error at /usr/local/Cellar/cmake/3.11.0/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Failed to find Gettext libintl (missing: Intl_INCLUDE_DIR)
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.11.0/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/Cellar/cmake/3.11.0/share/cmake/Modules/FindIntl.cmake:47 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  programs/cleos/CMakeLists.txt:29 (find_package)

-- Configuring incomplete, errors occurred!
See also "/Users/liu/eosio/eos/build/CMakeFiles/CMakeOutput.log".

	>>>>>>>>>>>>>>>>>>>> CMAKE building EOSIO has exited with the above error.

It a problem with brew gettext package.  See https://github.com/EOSIO/eos/issues/2174.

sudo ./eosio_build.sh

the same issue,  but I have installed gettex according to the output of the command brew list | grep gettext.

brew unlink gettext && brew link --force gettext, it works for me.  #2028

I had the same issue. But https://github.com/EOSIO/eos/issues/2240#issuecomment-396309884 seems to have fixed it.


[Bug?] .env file not being read in worker mode

EDIT - It now looks like this is a .env file reading issue rather than a cache warmup one - I've fixed the warm-up problem but I'm still getting errors.
I'm currently moving my API-Platform dev / CI stack from Apache to FrankenPHP.  If I make a request to the API soon after the container starts, I typically get a response like this:

Most of the time, it eventually starts to work, so I assume it's Symfony cache warm up related?  (When I try run with a single worker, it never seems to warm up, but I suspect that's a config error by me.  I think I need single-worker mode for xdebug to be reliable?) In Apache, the early request just waits while the cache warms up and returns the response once it's finished (unless it times out).
I assume I'm seeing what you would expect?  So the questions are probably:

How can I warm up the cache correctly so that the container gets to a working state "instantly" on deployment.
How should I check that the warm up is complete?  It returns a 200 response even with that failure message.
And is there's a way to change things so that, at least in production, it can never leak that failure message even if the cache isn't warm?

This might be related to #220, but it feels a bit more Symfony and less Caddy deployment related than that.  It's also entirely possible that I'm simply using things wrong.  Once I understand what's going on I might be able to PR some docs to help others.

Moving from #314, since that seems to have been a different issue.
It looks pretty consistently like it's not reading the .env file for me in worker model on my project.  Starting the container with FRANKENPHP_CONFIG undefined works.  But when I add -e FRANKENPHP_CONFIG="worker ./public/index.php 1" I get errors consistent with missing .env-defined variables.
Based on the 1.0.0-php8.3-bookworm image, using Symfony 7.0.1 and API Platform 3.2.7.  Running with:
ENV APP_RUNTIME=Runtime\\FrankenPhpSymfony\\Runtime
ENV CADDY_GENERAL_OPTIONS="auto_https off"
ENV SERVER_NAME="http://main-api"
ENV CADDY_GLOBAL_OPTIONS=debug

I have https turned off since I'm serving the site via an Nginx reverse proxy (in another container) which handles TLS and talked http to the FrankenPHP container.  But when I saw the issue I mentioned above, I was running it standalone with auto HTTPS on.
For legacy reasons, my app is being served from /var/www/main-api in the container rather than /app (I've changed the WORKDIR).  But if I copy the .env file to /app/ it still doesn't get read (checking that the worker isn't hard-coded to expect it in /app).   Defining them as "real" environment variables in the Dockerfile does work, so it looks like it's specifically something about reading .env files.
I'm not sure it's helpful, but I'm getting this in the FrankenPHP container logs:
{"level":"debug","ts":1702213534.1945422,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_ip":"<IP_ADDRESS>","remote_port":"34208","client_ip":"<IP_ADDRESS>","proto":"HTTP/1.0","method":"GET","host":"main-api","uri":"/","headers":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"X-Real-Ip":["<IP_ADDRESS>"],"X-Forwarded-Proto":["https"],"X-Nginx-Proxy":["true"],"Connection":["close"],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Ch-Ua-Platform":["\"macOS\""],"Upgrade-Insecure-Requests":["1"],"Pragma":["no-cache"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36"],"Sec-Fetch-Mode":["navigate"],"Cache-Control":["no-cache"],"Sec-Ch-Ua":["\"Not_A Brand\";v=\"8\", \"Chromium\";v=\"120\", \"Google Chrome\";v=\"120\""],"Sec-Fetch-Site":["none"],"Sec-Fetch-Dest":["document"],"Accept-Encoding":["gzip, deflate, br"],"Accept-Language":["en-GB,en-US;q=0.9,en;q=0.8"],"X-Forwarded-For":["<IP_ADDRESS>"],"X-Forwarded-Port":["443"],"Sec-Fetch-User":["?1"]}},"method":"GET","uri":"/index.php"}
{"level":"debug","ts":1702213534.1946356,"msg":"request handling started","worker":"/var/www/main-api/public/index.php","url":"/index.php"}
{"level":"debug","ts":1702213535.6216128,"msg":"request handling finished","url":"/index.php"}
{"level":"debug","ts":1702213535.6216726,"msg":"request handling finished","url":"/index.php"}
2023/12/10 13:05:35.621	ERROR	http.log.access.log0	handled request	{"request": {"remote_ip": "<IP_ADDRESS>", "remote_port": "34208", "client_ip": "<IP_ADDRESS>", "proto": "HTTP/1.0", "method": "GET", "host": "main-api", "uri": "/", "headers": {"Pragma": ["no-cache"], "User-Agent": ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36"], "Sec-Fetch-Mode": ["navigate"], "Cache-Control": ["no-cache"], "Sec-Ch-Ua": ["\"Not_A Brand\";v=\"8\", \"Chromium\";v=\"120\", \"Google Chrome\";v=\"120\""], "Sec-Fetch-Site": ["none"], "Sec-Fetch-Dest": ["document"], "Accept-Encoding": ["gzip, deflate, br"], "Accept-Language": ["en-GB,en-US;q=0.9,en;q=0.8"], "X-Forwarded-For": ["<IP_ADDRESS>"], "X-Forwarded-Port": ["443"], "Sec-Fetch-User": ["?1"], "Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"], "X-Real-Ip": ["<IP_ADDRESS>"], "X-Forwarded-Proto": ["https"], "X-Nginx-Proxy": ["true"], "Connection": ["close"], "Sec-Ch-Ua-Mobile": ["?0"], "Sec-Ch-Ua-Platform": ["\"macOS\""], "Upgrade-Insecure-Requests": ["1"]}}, "bytes_read": 0, "user_id": "", "duration": 1.427297632, "size": 4902, "status": 500, "resp_headers": {"X-Debug-Token": ["55236e"], "Content-Type": ["application/problem+json; charset=utf-8"], "Vary": ["Accept"], "Content-Location": ["/errors/500"], "X-Debug-Token-Link": ["http://main-api/_profiler/55236e"], "Server": ["Caddy"], "X-Content-Type-Options": ["nosniff"], "Link": ["<http://www.w3.org/ns/hydra/error>; rel=\"http://www.w3.org/ns/json-ld#error\",<http://main-api/docs.jsonld>; rel=\"http://www.w3.org/ns/hydra/core#apiDocumentation\""], "X-Previous-Debug-Token": ["5d0937"], "Date": ["Sun, 10 Dec 2023 13:05:35 GMT"], "Pragma": ["no-cache"], "Expires": ["-1"], "X-Robots-Tag": ["noindex"], "X-Powered-By": ["PHP/8.3.0"], "X-Frame-Options": ["deny"], "Cache-Control": ["private, must-revalidate"]}}
{"level":"debug","ts":1702213535.8155124,"msg":"waiting for request","worker":"/var/www/main-api/public/index.php"}

(<IP_ADDRESS> is the Nginx proxy, <IP_ADDRESS> is the IP address the containers see for the web browser on the host machine)
The browser response tells me that the worker is at least finding most of the files to serve:

If there isn't an obvious solution @dunglas - something I've mis-configured - I'll try to build a simple producer project.
EDIT: Oh and I do have variables_order = EGPCS in my phpinfo output

I've been trying to dig into this some more and I'm still struggling.  I've tried running the demo project in as close to the same FrankenPHP config as my project, with all the same extensions, including trying (in both):

serving HTTP and HTTPS;
serving at localhost and another domain name;
serving from the /app directory as the WORKDIR and from another directory;

The demo project consistently reads the .env file (I set the API title as an env variable to test) in worker mode.  And my project reads it in non-worker mode but never in worker mode.
I've run it with caddy debug turned on, but it's not telling me much.  Kevin, can you suggest anything else I can try to dig into the issue, please, since I've completely failed to make a simple reproducer?
Log output from my project (worker mode, http only, modified WORKDIR)
2023-12-31 16:02:16 {"level":"info","ts":1704038536.18503,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000411a80"}
2023-12-31 16:02:16 {"level":"debug","ts":1704038536.184318,"logger":"http.auto_https","msg":"adjusted config","tls":{"automation":{"policies":[{}]}},"http":{"servers":{"srv0":{"listen":[":80"],"routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"vars","root":"public/"},{"encodings":{"gzip":{},"zstd":{}},"handler":"encode","prefer":["zstd","gzip"]}]},{"handle":[{"handler":"static_response","headers":{"Location":["{http.request.orig_uri.path}/"]},"status_code":308}],"match":[{"file":{"try_files":["{http.request.uri.path}/index.php"]},"not":[{"path":["*/"]}]}]},{"handle":[{"handler":"rewrite","uri":"{http.matchers.file.relative}"}],"match":[{"file":{"split_path":[".php"],"try_files":["{http.request.uri.path}","{http.request.uri.path}/index.php","index.php"]}}]},{"handle":[{"handler":"php","split_path":[".php"]}],"match":[{"path":["*.php"]}]},{"handle":[{"handler":"file_server"}]}]}],"terminal":true}],"automatic_https":{"disable":true},"logs":{"logger_names":{"main-api":"log0"}}}}}}
2023-12-31 16:02:16 {"level":"debug","ts":1704038536.186272,"logger":"http","msg":"starting server loop","address":"[::]:80","tls":false,"http3":false}
2023-12-31 16:02:16 {"level":"info","ts":1704038536.1867058,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
2023-12-31 16:02:16 {"level":"debug","ts":1704038536.191244,"msg":"starting","worker":"/var/www/main-api/public/index.php"}
2023-12-31 16:02:16 {"level":"info","ts":1704038536.1937191,"logger":"tls","msg":"cleaning storage unit","storage":"FileStorage:/data/caddy"}
2023-12-31 16:02:16 {"level":"info","ts":1704038536.1943705,"logger":"tls","msg":"finished cleaning storage units"}
2023-12-31 16:02:16 {"level":"debug","ts":1704038536.36632,"msg":"waiting for request","worker":"/var/www/main-api/public/index.php"}
2023-12-31 16:02:16 {"level":"info","ts":1704038536.366437,"msg":"FrankenPHP started 🐘","php_version":"8.3.1"}
2023-12-31 16:02:16 {"level":"info","ts":1704038536.3673,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
2023-12-31 16:02:16 {"level":"info","ts":1704038536.3673437,"msg":"serving initial configuration"}
2023-12-31 16:02:20 {"level":"debug","ts":1704038540.9129424,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_ip":"<IP_ADDRESS>","remote_port":"39992","client_ip":"<IP_ADDRESS>","proto":"HTTP/1.1","method":"GET","host":"main-api","uri":"/","headers":{"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"Accept-Encoding":["gzip, deflate"],"If-None-Match":["\"4fda40eadde691538290ce9597c93be5\""],"Connection":["keep-alive"],"Accept-Language":["en-GB,en;q=0.9"],"Cache-Control":["max-age=0"],"Upgrade-Insecure-Requests":["1"]}},"method":"GET","uri":"/index.php"}
2023-12-31 16:02:20 {"level":"debug","ts":1704038540.9132617,"msg":"request handling started","worker":"/var/www/main-api/public/index.php","url":"/index.php"}
2023-12-31 16:02:22 {"level":"debug","ts":1704038542.336652,"msg":"request handling finished","url":"/index.php"}
2023-12-31 16:02:22 {"level":"debug","ts":1704038542.3367305,"msg":"request handling finished","url":"/index.php"}
2023-12-31 16:02:22 2023/12/31 16:02:22.337     ERROR   http.log.access.log0    handled request {"request": {"remote_ip": "<IP_ADDRESS>", "remote_port": "39992", "client_ip": "<IP_ADDRESS>", "proto": "HTTP/1.1", "method": "GET", "host": "main-api", "uri": "/", "headers": {"User-Agent": ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36"], "Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"], "Accept-Encoding": ["gzip, deflate"], "If-None-Match": ["\"4fda40eadde691538290ce9597c93be5\""], "Connection": ["keep-alive"], "Accept-Language": ["en-GB,en;q=0.9"], "Cache-Control": ["max-age=0"], "Upgrade-Insecure-Requests": ["1"]}}, "bytes_read": 0, "user_id": "", "duration": 1.424004133, "size": 4939, "status": 500, "resp_headers": {"X-Previous-Debug-Token": ["c3c1fd"], "Vary": ["Accept"], "X-Frame-Options": ["deny"], "Server": ["Caddy"], "X-Powered-By": ["PHP/8.3.1"], "X-Content-Type-Options": ["nosniff"], "Content-Location": ["/errors/500"], "Cache-Control": ["no-cache, private"], "Link": ["<http://www.w3.org/ns/hydra/error>; rel=\"http://www.w3.org/ns/json-ld#error\",<http://main-api/docs.jsonld>; rel=\"http://www.w3.org/ns/hydra/core#apiDocumentation\""], "X-Debug-Token": ["66579e"], "X-Robots-Tag": ["noindex"], "Content-Type": ["application/problem+json; charset=utf-8"], "Date": ["Sun, 31 Dec 2023 16:02:22 GMT"], "X-Debug-Token-Link": ["http://main-api/_profiler/66579e"]}}
2023-12-31 16:02:22 {"level":"debug","ts":1704038542.552087,"msg":"waiting for request","worker":"/var/www/main-api/public/index.php"}

This seems more like a symfony issue than a frankenphp issue (frankenphp doesn't read any .env files to my knowledge). .env files are an application level thing that needs support from the application.
If you wanted to have Caddy/FrankenPHP load the env file and pass it on, I suspect this might work since we do support the env directive (pretty sure anyway).

Sorry Kévin - I just realised I would have pinged you when I copied my message from the other issue.  I was trying to avoid doing that on a weekend / holiday.
Thanks @withinboredom.  The fact that it works with Apache and with FrankenPHP in non-worker mode, but not in worker mode suggests to me that there's something FrankenPHP-specific going on.  But it's certainly possible it's a Symfony or (most likely) user configuration issue - especially since the demo project, which seems to work, is a Symfony/Api-Platform project just like mine.  I'll have a look at that --envfile option and see if it's at least a work around - thank you.


:books: added docs from trello

Description
Trello Link
https://trello.com/c/qIMbVtSR/8-js-basic-part-2
Goals accomplished

[x]  Added JS objects as markdown
[x]  Added JS arrays as markdown
[x]  Added JS functions as markdown

 JS Basic (Part - 2)


HIVE-26208: Exception in Vectorization with Decimal64 to Decimal casting

The exception was happening because the code that casts the Decimal64 to
Decimal was not being added in filtered expressions.
Once this code was added, it caused a regression in check_constraint.q. The
reason for this was because we do not want to convert Decimal64 to Decimal
if the expression explicitly handles decimal64 types. A method was added
to these classes that will prevent the conversion in these cases.

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

+1. LGTM


XMap.from_xml is going to freeze state for large XML String

For the file large_file.txt.gz when tried to parse using the following, it is going to freeze state.
$ iex -S mix
Erlang/OTP 20 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

Interactive Elixir (1.6.5) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> File.read!("large_file.txt.gz") |> :zlib.gunzip() |> XMap.from_xml

BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
       (v)ersion (k)ill (D)b-tables (d)istribution

@krishna-goteti I've spend a few minutes trying to improve the performance but none of my experiments improved the performance. Depending on your use case, you might want to use :xmerl directly :)


HBOS (and probably others) model doesn't need decision_scores_ and labels_ attributes

Hello,
I'm working on a scenario where I've to train HBOS model on past data, save model and then use that to found anomalies on new (unseen) data.
So during the HBOS training I don't need to save the decision_scores_ and labels_ attributes of the training data inside the model obj.
My suggestion is to skip the decision_scores_ and labels_ attributes initialization and if a user needs them he will be able to run predict on training data (and save them in another structure outside the model).
This new approach enable us to save very little model (in term of memory).
Here is an example (HBOS trained on 600k data)
decision_scores_ are 5MB as well as labels_ -> 10MB pkl file
Removing them you'll have a 2KB pkl file
Furthermore:

labels_ is used only in the fit_predict method which is deprecated.
in the predict method there's check_is_fitted(self, ['decision_scores_', 'threshold_', 'labels_']) but neither decision_scores_ or labels_ are used.

Thanks for the note. This will be put on the todo list. We will go through all the models to see which ones do not need the decision_scores_ and labels_ to be pre-calculated.
This is in place now for all models as most algorithms are unsupervised and users often finish the work by calling fit only.


Any information on using Newtonsoft to deserialize the results

I would like to be able to display colorized source but I am having trouble turning the Json result test into an object I can work with.

Sorry I was doing something wrong in my invocation.


reverse input of +output of #1114

⚠: Please review #1136 and #1137 first, as I merged the changes!
The only new thing is that changes from commit 4ef1c31 are working now, due to updates from the other PRs.
The last two commits add term tracker item (2c02240) and CHANGELOG.md-updates (4332c06)
Elements to be addressed

Type
Element
Description

[_B_]
input of aka RO_0002352 + has input aka RO_0002233
add Inverse Of-relation

[_B_]
output of aka RO_0002353 + has output aka RO_0002234
add Inverse Of-relation

[_B_]
participates in aka RO_0000056 + has participant aka RO_0000057
add Inverse Of-relation

Please ignore changes to 'oeo.omn'. Protégé likes to rearrange things 😤
Related discussions
This PR closes #1114

Sorry, this is hard to review. Why do we have two pull requests that do (almost?) the same.

Ah, I thought the PR-description might be explanatory, but seems like I missed some points, so I'll try to explain:
Those are resulting from the interdependencies of the related issues, which had to be solved in a specific order.
Each PR adresses one issue, which resolution was made possible by the previous PR.
Think of it as an incremental process. Changes were merged, as they were needed to address the next issues.

PR #1136  introduced relations asked for in issue  #1079
PR #1137  fixes the wrong axioms by replacing relations with the new ones noticed in issue  #1023
PR #1138  adds the Inverse Of-attributes to the relations formerly used in the wrong axioms and fixes issue #1114

When 1st and 2nd step are reviewed, the only additions are

4ef1c31 = ontology content-wise change (see PR description for details)
2c02240 = term tracker items
4332c06 = CHANGELOG.md-updates

So, I'd do the #1137 review first, so that the review order reflects the issue-chain.
In the "Files Changed"-Tab you can also filter by specific commits/a range of commits, maybe this makes it easier for you.
If I can assist you, just let me know.

Usually, PRs should be independent of each other. I don't understand what is happening here and thus cannot approve this.

I'd be happy to assist you, but it would be beneficial, if you were specific concerning the aspects that are unclear to you.
Please go to #1137 as that one is the prerequiste, as stated above.
As the issues were interdependent, the solution cannot be fully independent.
I'll close this PR, wait until #1137 is done and merge the remaining commits again, opening a new PR

Git-Magic 🧙♂

Huh, that was a quick merge, I was still reviewing...

I'm sorry. I just saw GitHub's "ready to merge"-notification.
Next time I'll wait ...
Shall I revert it?

Is there  reson why we didn't add a range to has participant? It's clearly given in the definition.

That's a good point.
It was neither in the treated issue (#1114) nor in the one where the other issue originated from (#979).
I consider it good style, not mixing to many aspects in one commit/PR, so I only focused on the elements which were part of the issues.
Shall I open a follow-up issue and take care of it?
Otherwise we can add it to the list in #973, but I'm not sure whether it fits there content-wise
(concerning OEO-defined vs imported object properties)

Better open a new issue. Thanks @markus-rothkoetter


STOP_ALL_CALLS fails when thread count is 1

So we can address your issue, please include the following:
Version of MarkLogic Java Client API
5.3.0
Version of MarkLogic Server
10.0-4.x and 10.0-5
Java version
Doesn't matter
OS and version
Doesn't matter
Input: Some code to illustrate the problem, preferably in a state that can be independently reproduced on our end
Create an ErrorListener that returns STOP_ALL_CALLS as its disposition, and attach it to a BulkInputCaller which runs with a single thread (i.e. single CallContext). Write a record using that will throw an exception, thus triggering the ErrorListener. It will get a null-pointer on line 238 in InputOutputEndpointImpl:
case STOP_ALL_CALLS:
  getCallerThreadPoolExecutor().shutdown();
``

The thread pool executor is null since there's only one thread, so the call to shutdown throws the NPE.

Expectation is that this shouldn't fail if the thread count is one - seems the logic should just say - if there's no thread pool executor, then no need to call shutdown on it.

NPE no longer seen.

There's no server-side change in this issue.


Added and styled hero section

Added hero section to handlebars homepage
Added custom css to to hero section in styles.css folder

Fixed merge confilicts for tailwind css output file


generate db if not exist

I.m.h.o it would be nice, if the database would be generated if missing; what do you think?

Good ideas thanks.

works fine! Thanks a lot!


Get-AtwsService - No Unit Count?

Hola Everyone,
I'm able to query pretty much everything but the main field I'd need for the project I'm working on... Does anyone know if I'm missing something or does the API not have the ability to return service/bundle unit counts? I see unitPrice and unitCost but no count :/ any help would be appreciated!

Hi @s0nders
The entity you need is ContractService instead of Service. As services are added to contracts, a new entity is created unique to that account and contract. The contract service is what units are added or removed from over the course of the contract's lifespan.
See the section on the ContractService entity in the API doc to get started.
https://www.autotask.net/help/Content/LinkedDOCUMENTS/WSAPI/T_WebServicesAPIv1_6.pdf
I created a custom function to layout the contract services for a given client in a nice table for review when questions arise about units. You'll have to cross-reference several entities to get all of this info in one place.

Thank you! I really appreciate the quick response,  I love the project btw!!
Get Outlook for Androidhttps://aka.ms/ghei36
From: Jason Wood
Sent: Monday, July 22, 11:23 AM
Subject: Re: [officecenter/Autotask] Get-AtwsService - No Unit Count? (#40)
To: officecenter/Autotask
Cc: Patrick Rufino, Mention
Hi @s0ndershttps://github.com/s0nders
The entity you need is ContractService instead of Service. As services are added to contracts, a new entity is created unique to that account and contract. The contract service is what units are added or removed from over the course of the contract's lifespan.
See the section on the ContractService entity in the API doc to get started.
https://www.autotask.net/help/Content/LinkedDOCUMENTS/WSAPI/T_WebServicesAPIv1_6.pdf
I created a custom function to layout the contract services for a given client in a nice table for review when questions arise about units. You'll have to cross-reference several entities to get all of this info in one place.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/officecenter/Autotask/issues/40?email_source=notifications&email_token=AMVYUMK2PHOZUA4XZEVHMLTQAXGHRA5CNFSM4IFZRRIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2QJAWQ#issuecomment-513839194, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AMVYUMNYYTZZTBDZ3MJUAU3QAXGHRANCNFSM4IFZRRIA.

Hi @s0nders
The entity you need is ContractService instead of Service. As services are added to contracts, a new entity is created unique to that account and contract. The contract service is what units are added or removed from over the course of the contract's lifespan.
See the section on the ContractService entity in the API doc to get started.
https://www.autotask.net/help/Content/LinkedDOCUMENTS/WSAPI/T_WebServicesAPIv1_6.pdf
I created a custom function to layout the contract services for a given client in a nice table for review when questions arise about units. You'll have to cross-reference several entities to get all of this info in one place.

BTW are you willing to share the function? I'm looking to take these queries and append the results in a tempDB for some reporting functions and that sounds like it would be a great place to start!

Hi @JsonWud, thank you for helping out! If you can share your script in any way, please do. The autogenerated functions only provide bare bones access to the API, so this kind of more advanced functionality would be very welcome.
I am closing this issue now.

Hi @s0nders,
So sorry for the delay. I would not be able to share the script that outputs the table above but would be happy to help you create your own if you haven't done so already. This script is in need of refactoring as it is quite old (in software years).
Basically you want to create an advanced function that takes an account name, or whatever you want to use as an identifier, as a parameter. Inside the function determine the contract from the accountname/id, then pull the contractservices associated with the contract. Once you have the contractservices for that contract, you can pull the contractserviceunits for the date you want to specify as the second parameter. I pull the services as well so that I can cross reference the service name. The rest is just selecting the fields and outputting to Format-Table (I add another optional switch parameter for this because I want to output full objects by default).
At the command line I would call this advanced function like so:
New-ContractUnitReport -AccountName 'XYZ Client' -Date '7/1/2019' -TableView
Let me know if you need more help!
Jason

Hi @s0nders,
So sorry for the delay. I would not be able to share the script that outputs the table above but would be happy to help you create your own if you haven't done so already. This script is in need of refactoring as it is quite old (in software years).
Basically you want to create an advanced function that takes an account name, or whatever you want to use as an identifier, as a parameter. Inside the function determine the contract from the accountname/id, then pull the contractservices associated with the contract. Once you have the contractservices for that contract, you can pull the contractserviceunits for the date you want to specify as the second parameter. I pull the services as well so that I can cross reference the service name. The rest is just selecting the fields and outputting to Format-Table (I add another optional switch parameter for this because I want to output full objects by default).
At the command line I would call this advanced function like so:
New-ContractUnitReport -AccountName 'XYZ Client' -Date '7/1/2019' -TableView
Let me know if you need more help!
Jason

Morning JsonWud!
Thank you for that response! I'm current at the the point where I can pull the units from the contractserviceunits but I'm having an issue narrowing it down to the current specific month like you stated. Would you be able to share how your accomplishing this? I think once I have that piece I should be good front that point. Thank you again for taking the time to assist, I really appreciate it.

@s0nders
Glad you figured it out. Don't be hard on yourself. Dealing with time in Autotask is not always a walk in the park. ;)

@s0nders
Autotask has a whole section dedicated to explaining ContractUnits in the documentation, but the key takeaway is that for any given $Date you'll want the ContractServiceUnit that begins before (or on) and ends after (or on) the date you are interested in:
$Units = Get-AtwsContractServiceUnit -ContractID $Contracts.id -StartDate $Date -LessThanOrEquals StartDate -EndDate $Date -GreaterThanOrEquals EndDate
$BundleUnits = Get-AtwsContractServiceBundleUnit -ContractID $Contracts.id -StartDate $Date -LessThanOrEquals StartDate -EndDate $Date -GreaterThanOrEquals EndDate

For some reason that is quite possibly rational (I think) you have to get units for service bundles in a separate query. The only difference between the two is that ContractServiceBundleUnit does not have a VendorId field (after all, you could bundle services from different vendors) AND the external reference field is called ContractServiceBundleId, not ContractServiceId. Important differences, those.
It really would be nice if we had a nice, static function that got both entities for any given contract or client, @JsonWud 😉

Happy to contribute after I get out from under some looming deadlines. :D
On Mon, Jul 29, 2019 at 4:19 PM Hugo Klemmestad<EMAIL_ADDRESS>wrote:

@s0nders https://github.com/s0nders
Autotask has a whole section dedicated to explaining ContractUnits in the
documentation, but the key takeaway is that for any given $Date you'll
want the ContractServiceUnit that begins before (or on) and ends after
(or on) the date you are interested in:
$Units = Get-AtwsContractServiceUnit -ContractID $Contracts.id -StartDate $Date -LessThanOrEquals StartDate -EndDate $Date -GreaterThanOrEquals EndDate
$BundleUnits = Get-AtwsContractServiceBundleUnit -ContractID $Contracts.id -StartDate $Date -LessThanOrEquals StartDate -EndDate $Date -GreaterThanOrEquals EndDate
For some reason that is quite possibly rational (I think) you have to get
units for service bundles in a separate query. The only difference
between the two is that ContractServiceBundleUnit does not have a VendorId
field (after all, you could bundle services from different vendors) AND the
external reference field is called ContractServiceBundleId, not
ContractServiceId. Important differences, those.
It really would be nice if we had a nice, static function that got both
entities for any given contract or client, @JsonWud
https://github.com/JsonWud 😉
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/officecenter/Autotask/issues/40?email_source=notifications&email_token=AJKHNXFV6YTJPLMFPMHOKGTQB5GFZA5CNFSM4IFZRRIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3B4KZA#issuecomment-516146532,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJKHNXDG4BAGSG4NP3IVLITQB5GFZANCNFSM4IFZRRIA
.

Oh, I know about those - issues on real bugs tend to pop up around those 😉 Best of luck!


Feat/tbrom pyvista 24r2 part2 (stacked on feat/tbrom_pyvista_24r2)

follow up of abandoned PR #126 (part 2) :
This PR introduces new capabilities regarding 3D field visualization

it adds new APIs (TBROM, TwinModel) to project field data onto given mesh, and update the resulting field with respect to current Twin states
associated unit testing
new examples illustrating the mesh projection workflow for CFD and FEA models

It adds the necessary dependencies and updates the CI workflow to install a DPF server when building the documentation and running the examples

Codecov Report
Attention: 19 lines in your changes are missing coverage. Please review.

Comparison is base (1dce0b8) 87.45% compared to head (5948d1b) 87.28%.

Additional details and impacted files
@@                     Coverage Diff                     @@
##           feat/tbrom_pyvista_24r2     #143      +/-   ##
===========================================================
- Coverage                    87.45%   87.28%   -0.18%
===========================================================
  Files                           11       11
  Lines                         2464     2556      +92
===========================================================
+ Hits                          2155     2231      +76
- Misses                         309      325      +16


Building API's with Django and GraphQL

NOTE: We tend to stray away or tend not to publish reviews/comparisons of commercial product offerings.
Brief Summary:
GraphQL is an open-source data query and manipulation language for APIs, and a runtime for fulfilling queries with existing data. GraphQL used on the frontend and backend. In this tutorial, I will be creating a Web API in GraphQL using Graphene-Python, Graphene-Python is a library for building GraphQL APIs in Python easily.
Key Takeaways:
1 - What is GraphQL.
2 - Integrating GraphQL in Django.
3 - Build a Simple Note API using GraphQL.
References:
Please list links to any published content/research that you intend to use to support/guide this article. (If none, please indicate 'N/A'.)

Thank you for your topic suggestion. Per the recent message posted in Slack, we are pausing acceptance of new articles until further notice while we work through the backlog of submitted drafts. We will notify you when we are ready to accept new topic suggestions. Thank you for your patience.


Feature request: calculate transaction hash of a raw transaction

It would be nice if starkli supported calculating the transaction hash of a raw transaction.

What do you mean a raw transaction? Starkli currently doesn't expose any raw transaction format. What experience do you have in mind?

Yeah, makes sense. Maybe Starkli is not the right place for this.


update panel group nunjuck macro

fixed the panel group to contain 2 panels within the nhsuk-grid-row nhsuk-panel-group div and not 4.
Description
Checklist

[ ] SCSS
[ ] Nunjucks macro
[ ] A standalone example
[ ] README/Documentation with HTML snippet
[ ] Tested against our testing policy (Resolution, Browser & Accessibility)
[ ] Visual tests (BackstopJS)
[ ] Print stylesheet considered
[ ] CHANGELOG entry

Looks good, thanks @nnekaO :)


Export Environment Variables in a Process's Context when an event is captured

Is there an existing issue for this?

[X] I have searched the existing issues

Is your feature request related to a problem?
I want to enhance monitoring for a process captured by Tetragon by looking at the environment variables available to the captured process.
Describe the feature you would like
To further enhance observability within a process's context, this feature will allow specified environment variables to be exported alongside other metadata when an event is captured. Such that, when specifying a Tetragon event, you can specify which environment variables you want to export when a Tetragon event is captured. For example:
kprobes:
- call: <some_call_here>
  env: ["ENV1", "ENV2"]

would yield the following entry in the json logs:
{
  "env": {
      "ENV1": "<some_string_value_here>",
      "ENV2:  "<some_string_value_here>"
  }
}

Describe your proposed solution
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Are there any updates on this?