Recipe UpgradeSpringBoot_3_3 misses upgrading wro4j-maven-plugin and the latest version of jakarta.xml.bind-api when the origin is Java8 and Spring Boot 1.5.X

What version of OpenRewrite are you using?
I am using:

Maven/Gradle plugin 5.34.0
rewrite-spring 5.13.2

How are you running OpenRewrite?
I'm using maven plugin on my own project.
You can check the project here:
https://github.com/mgvinuesa/spring-petclinic-openrewrite
It is a fork of spring-petclinic and you might use the branch with the name migrate_with_open_rewrite, I started with the TAG 1.5.x to migrate boot Spring Boot and Java using OpenRewrite.
What is the smallest, simplest way to reproduce the problem?
Execute the mvn command with the current configuration with Java 8.
mvn rewrite:run
What did you expect to see?
The wro4j-maven-plugin might upgrade from 1.8.0 to 2.X due to Java17 compatibility.
The recipe that adds  jakarta.xml.bind-api must include the latest version instead of 2.3.3.
What did you see instead?
Even there is a recipe to work with wro4j:
org.openrewrite.java.migrate.UpgradeToJava17
[WARNING]                         org.openrewrite.java.migrate.Java8toJava11
[WARNING]                             org.openrewrite.java.migrate.javax.AddJaxbDependencies
[WARNING]                                 org.openrewrite.java.dependencies.AddDependency: {groupId=jakarta.xml.bind, artifactId=jakarta.xml.bind-api, version=2.3.x, onlyIfUsing=javax.xml.bind..*, acceptTransitive=true}
[WARNING]                             org.openrewrite.java.migrate.cobertura.RemoveCoberturaMavenPlugin
[WARNING]                                 org.openrewrite.maven.RemovePlugin: {groupId=org.codehaus.mojo, artifactId=cobertura-maven-plugin}
[WARNING]                             org.openrewrite.java.migrate.wro4j.UpgradeWro4jMavenPluginVersion
[WARNING]                                 org.openrewrite.maven.UpgradePluginVersion: {groupId=ro.isdc.wro4j, artifactId=wro4j-maven-plugin, newVersion=1.10.1}

Nothing happens, but the version is for Java11, not for Java17.
On the other side, about jakarta, the recipe AddDependency
[WARNING]                     org.openrewrite.java.migrate.UpgradeToJava17
[WARNING]                         org.openrewrite.java.migrate.Java8toJava11
[WARNING]                             org.openrewrite.java.migrate.javax.AddJaxbDependencies
[WARNING]                                 org.openrewrite.java.dependencies.AddDependency: {groupId=jakarta.xml.bind, artifactId=jakarta.xml.bind-api, version=2.3.x, onlyIfUsing=javax.xml.bind..*, acceptTransitive=true}

Add the 2.3.3 dependency but it is not enough due to packaging error with Java17, this dependency still use javax.xml.bind package.
Apart from that, I have tried to add more recipes to solve these issues:
---
type: specs.openrewrite.org/v1beta/recipe
name: com.example.MigrateToSpringBoot3_3
displayName: Upgrade Wro4j Maven plugin version
causesAnotherCycle: true
recipeList:
    - org.openrewrite.java.spring.boot3.UpgradeSpringBoot_3_3
    - org.openrewrite.maven.UpgradePluginVersion:
        groupId: ro.isdc.wro4j
        artifactId: wro4j-maven-plugin
        newVersion: 2.1.1
    - org.openrewrite.maven.RemoveProperty:
        propertyName: cobertura.version
    - org.openrewrite.maven.UpgradeDependencyVersion:
        groupId: jakarta.xml.bind
        artifactId: jakarta.xml.bind-api
        newVersion: 4.0.2

But the last recipe, UpgradeDependencyVersion, does not execute although I have tried to put causesAnotherCycle = true. So if you put this recipe com.example.MigrateToSpringBoot3_3 instead of org.openrewrite.java.spring.boot3.UpgradeSpringBoot_3_3 you can see that jakarta.xml.bind-api` is not upgraded.
What is the full stack trace of any errors you encountered?
No errors, only the result is not the proper one.
Are you interested in [contributing a fix to OpenRewrite]
Currently im writing a post in spanish about it, it is pretty interesting, I have time i would like to fix this problem.

hi! Thanks for the detailed report, and bringing OpenRewrite to your Spanish audience 🙏🏻
We just had some PRs related to bind-api merged last week; would you mind trying our latest snapshot versions?

https://github.com/openrewrite/rewrite/pull/4273
https://github.com/openrewrite/rewrite-hibernate/pull/29
https://github.com/openrewrite/rewrite-migrate-java/pull/499
https://github.com/openrewrite/rewrite-spring/pull/485
https://github.com/openrewrite/rewrite-spring/pull/543

The wro4j dependency should be updated, as part of our Java 11 migration, as seen in rewrite-migrate-java:
https://github.com/openrewrite/rewrite-migrate-java/blob/724a2442755b5bde6f8cdd225db7eab531108a70/src/main/resources/META-INF/rewrite/wro4j-maven-plugin.yml#L17-L28
Also: welcome to let me know if there's anything you'd like be to proofread. Happy to help!

Hello @timtebeek
I just update the plugin with the following SNAPSHOT versions:
rewrite-maven-plugin: 5.35.5-SNAPSHOT
rewrite-spring: 5.14.0-SNAPSHOT
The result is the following:

The jakarta xml api dependency still appears with the wrong version

    <dependency>
      <groupId>jakarta.xml.bind</groupId>
      <artifactId>jakarta.xml.bind-api</artifactId>
      <version>2.3.3</version>
    </dependency>

[INFO] --- rewrite:5.35.0-SNAPSHOT:run (default-cli) @ spring-petclinic ---
org.openrewrite.java.migrate.UpgradeToJava17
[WARNING]                         org.openrewrite.java.migrate.Java8toJava11
[WARNING]                             org.openrewrite.java.migrate.javax.AddJaxbDependencies
[WARNING]                                 org.openrewrite.java.dependencies.AddDependency: {groupId=jakarta.xml.bind, artifactId=jakarta.xml.bind-api, version=2.3.x, onlyIfUsing=javax.xml.bind..*, acceptTransitive=true}

I think that this recipe org.openrewrite.java.migrate.jakarta.JavaxMigrationToJakarta  executed after must include the group and the artifact id related with the previous dependency.
Regarding wro4j I see the recipe that you linked, but this recipe is not enough due to this version is for Java11,
org.openrewrite.java.migrate.Java8toJava11
[WARNING]                             org.openrewrite.java.migrate.javax.AddJaxbDependencies
[WARNING]                                 org.openrewrite.java.dependencies.AddDependency: {groupId=jakarta.xml.bind, artifactId=jakarta.xml.bind-api, version=2.3.x, onlyIfUsing=javax.xml.bind..*, acceptTransitive=true}
[WARNING]                             org.openrewrite.java.migrate.cobertura.RemoveCoberturaMavenPlugin
[WARNING]                                 org.openrewrite.maven.RemovePlugin: {groupId=org.codehaus.mojo, artifactId=cobertura-maven-plugin}
[WARNING]                             org.openrewrite.java.migrate.wro4j.UpgradeWro4jMavenPluginVersion
[WARNING]                                 org.openrewrite.maven.UpgradePluginVersion: {groupId=ro.isdc.wro4j, artifactId=wro4j-maven-plugin, newVersion=1.10.1}

we would need upgrade to 2.X. But anyway, it does not work (dont change the property version to 1.10.1) but I dont know why.
I just update the project with the SNAPSHOT versions if you want to try.
Thanks in advance,
Manu

We've since done two full release trains with related fixes. Are you seeing any improvement with the latest?
https://github.com/openrewrite/rewrite-recipe-bom/releases/tag/v2.15.0

Hello, sorry for the delay.
I just update the following versions:

rewrite-maven-plugin to 5.36.0
rewrite-spring to 5.15.1

The result is different from the previous report. To remember it, the main problems were two:

The wro4j-maven-plugin might upgrade from 1.8.0 to 2.X due to Java17 compatibility.
The recipe that adds jakarta.xml.bind-api must include the latest version instead of 2.3.3.

Now the changes are:

The wro4j-maven-plugin now is the version 1.10.1 instead 1.8.0 (the original one) but not the needed one (2.x)
About jakarta, now the dependency is not forced to an specific version (2.3.3) and the plugin adds a new dependency management:

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>jakarta.validation</groupId>
        <artifactId>jakarta.validation-api</artifactId>
        <version>3.1.0</version>
      </dependency>
    </dependencies>
  </dependencyManagement>

So the problem regarding jakarta is solved!!! but the plugin is not, now the error when build the project is:
[ERROR] Failed to execute goal ro.isdc.wro4j:wro4j-maven-plugin:1.10.1:run (default) on project spring-petclinic: Execution default of goal ro.isdc.wro4j:wro4j-maven-plugin:1.10.1:run failed: Plugin ro.isdc.wro4j:wro4j-maven-plugin:1.10.1 or one of its dependencies could not be resolved: Failed to collect dependencies for ro.isdc.wro4j:wro4j-maven-plugin:jar:1.10.1 (): Could not resolve version conflict among [ro.isdc.wro4j:wro4j-maven-plugin:jar:1.10.1 -> ro.isdc.wro4j:wro4j-extensions:jar:1.10.1 -> org.webjars.npm:jshint:jar:2.11.0 -> org.webjars.npm:minimatch:jar:[3.0.2,3.1), ro.isdc.wro4j:wro4j-maven-plugin:jar:1.10.1 -> ro.isdc.wro4j:wro4j-extensions:jar:1.10.1 -> org.webjars.npm:jshint:jar:2.11.0 -> org.webjars.npm:cli:jar:[1.0.0,1.1) -> org.webjars.npm:glob:jar:[7.1.1,8) -> org.webjars.npm:minimatch:jar:[3.1.1,4)] -> [Help 1]

It looks like a problem between some transitive dependencies for the plugin but I think that the problem is related with this version is not compatible with JDK17 ,
https://stackoverflow.com/questions/74347674/mvn-issue-with-an-old-java-codebase-clashing-dependencies
https://community.sap.com/t5/crm-and-cx-questions/wro4j-is-throwing-version-conflict-for-java-11-with-2105/qaq-p/12591404

Thanks again! Wasn't aware there was a 2.x version of the plugin available, or that it's required for Java 17. Fixed just now.


Gstreamer example code doesn't work with vcpkg Gstreamer; can't create any element

Describe the bug
When using gstreamer from vcpkg with cmake, no pipeline can be created because gstreamer is unable to construct any element. Example code from gstreamer's basic tutorial doesn't work, but it does work with the system installation of gstreamer (I'm on Linux) My guess is that it can't find any plugins whatsoever but I'm not very knowledgeable about gstreamer either
Environment

OS: Arch Linux
Compiler: GCC 13.2.1 20230801

To Reproduce
Setup a project with vcpkg cloned into a subdirectory and bootstrapped there
CMakeLists.txt:
cmake_minimum_required(VERSION 3.20)
project(gstreamer-test)

find_package(PkgConfig REQUIRED)
pkg_check_modules(gstreamer-1.0 REQUIRED IMPORTED_TARGET gstreamer-1.0)

add_executable(test main.c)
target_link_libraries(test PkgConfig::gstreamer-1.0)

vcpkg.json:
{
	"dependencies": [
		"gstreamer"
	]
}

main.c: any code from gstreamer's basic tutorials
Run cmake with appropriately set -DCMAKE_TOOLCHAIN_FILE, obviously
Expected behavior
The code should work.
Failure logs
Obviously these are logs from gstreamer as there aren't any visible errors from vcpkg (I wouldn't be able to compile the program otherwise)
Output from the hello world example:
$ GST_DEBUG=2 ./test
0:00:00.000650393 541967 0x555b092cf6d0 WARN            GST_REGISTRY gstregistrybinary.c:551:gst_registry_binary_check_magic: Binary registry magic version is different : 1.3.0 != 1.23.90
0:00:00.001519358 541967 0x555b092cf6d0 ERROR           GST_PIPELINE subprojects/gstreamer/gst/parse/grammar.y:570:gst_parse_element_make: no element "playbin"

(test:541967): GStreamer-CRITICAL **: 23:23:43.518: gst_element_set_state: assertion 'GST_IS_ELEMENT (element)' failed

(test:541967): GStreamer-CRITICAL **: 23:23:43.518: gst_element_get_bus: assertion 'GST_IS_ELEMENT (element)' failed

(test:541967): GStreamer-CRITICAL **: 23:23:43.518: gst_bus_timed_pop_filtered: assertion 'GST_IS_BUS (bus)' failed
zsh: segmentation fault (core dumped)  GST_DEBUG=2 ./test

Output from the manual hello world example:
$ GST_DEBUG=2 ./test
0:00:00.000639124 550754 0x568d6a9236d0 WARN            GST_REGISTRY gstregistrybinary.c:551:gst_registry_binary_check_magic: Binary registry magic version is different : 1.3.0 != 1.23.90
0:00:00.001522758 550754 0x568d6a9236d0 WARN     GST_ELEMENT_FACTORY gstelementfactory.c:765:gst_element_factory_make_valist: no such element factory "videotestsrc"!
0:00:00.001529478 550754 0x568d6a9236d0 WARN     GST_ELEMENT_FACTORY gstelementfactory.c:765:gst_element_factory_make_valist: no such element factory "autovideosink"!
Not all elements could be created.

I have reproduced this problem locally, I will locate the problem and try to fix it.


Test: Specify hash64

The proposed change specifies what the good hash code must be.
With the current test, any change in "hash" would still make the test pass, incl. the changes that would result in an inefficient hash.

Thanks for the PR. The purpose of the test, however, is not to verify "good" hash, but rather that the overloaded hash methods return the same value. In that regard, the existing tests are valid. That said, I think verifying the returned hash value is a valid test, so you are welcome to submit another PR to add the new test. If you did that, I'd suggest that you add some comments on how to obtain the baseline hash value that the test is checking against, e.g. a known good implementation of murmur hash.


How to compile notable for arm?

I compiled it by "env GOOS=linux GOARCH=arm GOARM=5 make".
When I run it, it crashed with below output.
panic: could not locate box "static"
goroutine 1 [running]:
github.com/jmcfarlane/notable/vendor/github.com/GeertJohan/go.rice.MustFindBox(0x40f6a2, 0x6, 0x664ca861)

It would be great if @jmcfarlane provides an arm binary in the release

I successfully compiled v0.1.2 arm builds. Binaries here

Get latest go from https://golang.org/dl/
Install decencies in binary-deps.sh
Install other decencies by running go get github.com/jmcfarlane/notable
Compile by running following commands:

TAG="v0.1.2"
goos="linux"
arch="armv7"

buildArch="-X main.buildArch"
export GOARCH="arm"
export GOARM="7"

mkdir -p target/notable-${TAG}.${goos}-${arch}
cp LICENSE target/notable-${TAG}.${goos}-${arch}

export GOOS=$goos
export CGO_ENABLED=0
go build -o target/notable-${TAG}.${goos}-${arch}/notable

Change arch and GOARM as you need, for a modern Raspberry Pi is armv7


Activesupport requires ruby >= 2.2.2

Ruby support for 2.1 is broken by activesupport 5.0.x.
Please pin to activesupport to 4.2.6 if ruby < 2.2

Or we just get rid of it, see #29

can't bundler solve it?

👍 I just ran into this as well. Seems like a good short term fix is to update the dependency to < 5


Question: importing .proto messages

Hello,
First of all thank you for this framework and all your work in it!
This issue is only to find out how to go about with the following issue: I have 3 proto files, related in terms of use case (let's call them: Proto A, Proto B and Proto C).
Both Proto A and Proto B make use of a message format in Proto C, so in both files I included the following line:
import "<golang_path>/assets/rpc/C/C.proto";

Which generates this output:
import _ "<golang_path>/assets/rpc/C/C.proto"

the problem is that these proto files are also generated for JS. Which in turn return the following error:
<golang_path>/assets/rpc/C/C.proto: File not found.

I've tried setting the --proto_path parameter and removing the golang_path, but to no avail since the generation of the go import only mirrors the import in the proto file.
Is there a workaround for this?

Yep, you can use option go_package to set the import path that generated go code should use when referencing your Proto C file.
This is laid out in https://github.com/twitchtv/twirp/blob/master/protoc-gen-twirp/generator.go#L280-L290:
importPath := path.Dir(goFileName(def.File))

The goFileName function will prefer to use the go_package option's value for the import path, and will fall back to using the path to the file on disk (as you found).
The right way to do this is for Proto C to include a line like this:
syntax = "proto3";
package pcruz7.protoc; // or whatever

// replace with whatever import path you want:
option go_package = "github.com/pcruz7/somerepo/protoc";

This will at least decouple the import lines in generated .twirp.go and .pb.go files from how you invoke protoc and how you write imports in .proto files, so you can maybe work things into better shape for the JS generator.


 I've gotten 10 emails in the past 5 minutes with all the commits lol

Ever heard of condensation? I've gotten 10 emails in the past 5 minutes with all the commits lol

I'm testing some pytest on github actions, and I need update something after each run. You can unsubscribe from this PR in the panel on the right :)

No, it's fine I just thought i might take the chance to mess with you a little :)

How did you manage to make this happen? 😂

Never mind its from the removing of tkdnd

Lol

I won't describe all the things I've done
@rdbende from the description

Yeah bro you can even say the commits speak for itself like Hans Niemann said for his (rather stupid, reasonless, and costly) $400m lawsuit. ("My lawsuit speaks for itself" - Hans)

Oh dear the branch name 😂

 How did you manage to make this happen? :joy:

Yupp. Eight thousand lines of Tcl code removed. I hope I can reduce that by rewriting the Tcl part in Python, while making it more maintainable (I love how much more readable Serif has become, when I translated it to Python).

Yes, Tukaan is already in usable state and you can use it to build application, however some features are still missing, and there might be bugs that I haven't discovered. Documentation is also missing as it's not a priority atm, but you can probably figure out things from the source code. Altogether, I recommend you to wait until the next release, 'cause a LOT OF thing will change, but then I'd be happy to see people use it and give feedback on how usable it is.

And @rdbende try using commitizen.

Tbh, I'm happy with my commit messages, they are really descriptive, lolz


Update spl-token-2022 to v1.0.0 on anchor-spl

spl-token-2022 1.0.0 introduces new ExtensionTypes, such as GroupPointer, GroupMemberPointer, TokenGroup, TokenGroupMember. These extensions are already being used in mints. Currently, anchor-spl cannot deserialize these extensions since it is pinned to v0.9. The solution here would be to bump spl-token-2022 dependency to 1.0.0 on anchor-spl.

Yes, will do. Does the accounts not deserialize e.g. when using TokenAccount or is it the exported spl_token_2022 module that doesn't have the functionality to deserialize the new extensions? If it's the latter, you should be able to specify
spl-token-2022 = "1"

in your Cargo.toml to use the latest version.

It is the former case. It happens when checking the extensions on the mint account before initializing the token account:
 let mint_extensions = mint_state.get_extension_types()?;

This uses the import from:
use ::anchor_spl::token_2022::spl_token_2022::extension::{
  BaseStateWithExtensions, ExtensionType, StateWithExtensions,
};

So it will require a version bump.

FYI, there is a problem with the 1.18 release which I believe is currently being worked on. The upgrade is blocked until that problem has some sort of a resolution.


api

有api吗？

None for now, since this is an open-source research project instead of a service.

有提供api的计划吗？

Do you have plans to provide API?


Update crate names and rand deps

This PR updates the names crate that was depedending on a very old version of rand.

bot merge


Query fromN1ql take very long time in android

Hi,
I have this probleme in my project using cbl
environment:
  sdk: ">=2.15.1 <3.3.5"

  cbl: ^2.1.2
  cbl_flutter: ^2.0.2
  cbl_flutter_ee: ^2.1.2

The result of the below query is fine but taking a very long time only in the android version while in iOs it's  very fast.
Is there a problem using the changes() function in android ?
Here's my code

  print("TAG BEFORE Q1 "+DateTime.now().toString());
    Query query = await _source.fromN1ql('''  MY QUERY      ''');

    print("TAG AFTER  Q1 "+DateTime.now().toString());
    yield* query.changes().asyncMap((change) async {
      print("TAG INSIDE change "+DateTime.now().toString());
        return change.results.asStream().map((el) => CbRoute(el).toRoute()).toList();
      }
    );

And here is the display as you can see it takes more than 30 seconds to display results
TAG BEFORE Q1 2022-11-30 11:47:05.519142
TAG AFTER  Q1 2022-11-30 11:47:07.581359
TAG INSIDE change 2022-11-30 11:47:31.394575

Any idea ?

I don't really have an explanation as to why there is a big difference between iOS and Android.
Setting up an index might speed it up significantly, though, if you are not already using one. Query.explain returns an explanation of how the query will be executed, including whether an index will be used. Sometimes a query won't use an index unexpectedly. The query execution plan can help to figure out which index will actually help the query. Could you provide the query and the result of explain?

Even when I reduce my query to the minimum with indexes it takes very long time :
I/flutter: TAG BEFORE Q1 2022-12-01 11:46:08.731113
I/flutter: SELECT fl_result(rte.key) FROM kv_default AS rte WHERE (fl_value(rte.body, 'type') = 'route' AND fl_value(rte.body, 'company_id') = 'company-1HnFQt4GqkT') AND (rte.flags & 1 = 0)

    3|0|0| SEARCH TABLE kv_default AS rte USING INDEX companyid (<expr>=?)

    {"FROM":[{"AS":"rte","COLLECTION":"_"}],"WHAT":[["_.",["meta()","rte"],".id"]],"WHERE":["AND",["=",[".rte.type"],"route"],["=",[".rte.company_id"],"company-1HnFQt4GqkT"]]}
I/flutter: TAG INSIDE change 2022-12-01 11:46:34.128203

Hm, interesting. The stream returned from Query.changes should execute the query immediately when it is being listened to. Could you try running the query through Query.execute to verify that executing the query is what takes so long and not some timing issue in the live query?

Hi i simplify the query and use query.execute and it takes a long time too
    Query query = await _source.fromN1ql('''
        SELECT  meta(rte).id
        FROM _ as rte
        WHERE rte.type='route'
        AND rte.company_id='company-1HnFQt4GqkT'
        ''');

    Future(query.explain).then(print);

    print("TAG AFTER  Q1 "+DateTime.now().toString());

    final result = await query.execute();

    print('TAG After execute '+DateTime.now().toString());

Results console :
I/flutter (17638): TAG AFTER  Q1 2022-12-01 14:20:12.220447
I/flutter (17638): TAG After execute 2022-12-01 14:20:31.476102
I/flutter (17638): SELECT fl_result(rte.key) FROM kv_default AS rte WHERE (fl_value(rte.body, 'type') = 'route' AND fl_value(rte.body, 'company_id') = 'company-1HnFQt4GqkT') AND (rte.flags & 1 = 0)
I/flutter (17638):
I/flutter (17638): 3|0|0| SEARCH TABLE kv_default AS rte USING INDEX companyid (<expr>=?)
I/flutter (17638):
I/flutter (17638): {"FROM":[{"AS":"rte","COLLECTION":"_"}],"WHAT":[["_.",["meta()","rte"],".id"]],"WHERE":["AND",["=",[".rte.type"],"route"],["=",[".rte.company_id"],"company-1HnFQt4GqkT"]]}

Thanks for trying that.

What Android device are you using?
How many documents are in the database?

I use multiple device :  A redmi note 11 or A oppo and in the emulator i have same behavior

There are 3796612 documents in the DB

If i adjust the query to return 1 documents it's almost instantaneous

@regissio have you made an index with all the fields in the where clause into a single index?
 final documentTypeExpression = Expression.property('type');
 final companyIdExpression = Expression.property( 'companyId');
    final valueIndexItems = {
      ValueIndexItem.expression(documentTypeExpression),
      ValueIndexItem.expression(companyIdExpression)
    };
final index = IndexBuilder.valueIndex(valueIndexItems);
...

Also, note that database performance relies heavily on Disk I/O performance.   This is usually why people say it works great on iPhone, which Apple puts in extremely fast memory, and then some cheaper-built Android phones struggle with performance.   I don't know the speed of your devices, but in my experience with. Android this is a big reason why performance is so different between different devices.

@regissio also, it's not a secrete that CBL uses Sqlite for its storage engine.  When I have problems, I usually use these links:

https://www.sqlite.org/queryplanner.html
https://www.sqlite.org/eqp.html

Order matter in an index.
SELECT * FROM DB WHERE country = "US" AND state IS NOT MISSING AND city = "Austin";
Try to put the most unique value in the front of the index.


Errors parsing vvvv-Time_0.5.5

Something went wrong:
'>' is an unexpected token. The expected token is '='. Line 35, position 91.

This refers to

data-pjax-transient

which should end with

="true"

I suspect.
Filling those in the next problem was that the link and meta tags in the header are not all closed with

/>

So I fixed those manually too. Finally I got this error

Something went wrong:
The 'div' start tag on line 128 position 12 does not match the end tag of 'option'. Line 129, position 43.

This refers to

  </option></form><form accept-charset="UTF-8" action="/vvvvpm/vpdb/search" class="js-site-search-form" data-scoped-search-url="/vvvvpm/vpdb/search" data-unscoped-search-url="/search" method="get">
<input name="utf8" type="hidden" value="✓" />

Might count for others too packages too (I have the same initial error from "contrib.vpm1.nodx11.vpack", too. Should I try an older pack or an older vpm?

Qualifier: I downloaded the https://github.com/vvvvpm/vpdb repo and opened them from my desktop.
https://vvvvpm.github.io/ opening and running these packs directly seemed to work as advertised. Will there be an option to choose between 32 and 64bit pack or does that depend upon the pack writers to create the options?


[🐞] useVisibleTask$: "Error: Internal assert, this is likely caused by a bug in Qwik: close not found"

Which component is affected?
Qwik Runtime
Describe the bug
Using useVisibleTask$ inside a root-level slot component:
export const MyProvider = component$(() => {
  useVisibleTask$(() => {
    console.log("VISIBLE")
  }, { strategy: 'document-ready' });
  return <Slot />
});

export default component$(() => {
  return (
    <MyProvider>
      <QwikCityProvider>
        <head>
          <meta charSet="utf-8" />
          <link rel="manifest" href="/manifest.json" />
          <RouterHead />
          <ServiceWorkerRegister />
        </head>
        <body lang="en">
          <RouterOutlet />
        </body>
      </QwikCityProvider>
    </MyProvider>
  );
});

doesn't work and causes the error:
Error: Internal assert, this is likely caused by a bug in Qwik: close not found
Reproduction
https://codesandbox.io/p/github/robisim74/qwik-bug
Steps to reproduce
Inspect the console in DevTools after the app has started.
System Info
System:
    OS: Linux 5.15 Debian GNU/Linux 11 (bullseye) 11 (bullseye)
    CPU: (2) x64 AMD EPYC
    Memory: 965.60 MB / 2.01 GB
    Container: Yes
    Shell: 5.1.4 - /bin/bash
  Binaries:
    Node: 18.17.0 - ~/.local/state/fnm_multishells/1372_1690558720040/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 9.6.7 - ~/.local/state/fnm_multishells/1372_1690558720040/bin/npm
    pnpm: 7.1.0 - /usr/local/bin/pnpm
    Watchman: 20220528.183901.0 - /usr/local/bin/watchman
  npmPackages:
    @builder.io/qwik: ^1.2.6 => 1.2.6
    @builder.io/qwik-city: ^1.2.6 => 1.2.6
    undici: 5.22.1 => 5.22.1
    vite: 4.4.0 => 4.4.0

Additional Information
No response

I had the same Issue. It seems to me, that useVisibleTask$ can only be used within the body of the root component.
At least in my case, it worked after I moved MyProvider to within the body tag.

This is working as expected so I'm closing this issue for now.

Feel free to re-open a new one if it's still an issue for you.
Thanks


Multi Select: Select All but 2 options and/or deselect all but 1 option

Hello,
I wonder if it's possible to Select all items except two (2) and also to deselect all items except one (1).
The thing is, I have a option list which includes an option that should not be part of multi select. If that option is selected, all other options should be deselected.
There is also another option which if selected, all other options should be selected except the one option that triggers the multi select.
Thanks in advance,

Environment

Windows 7
jQuery 1.12.4
Bootstrap 3.3.7

Detail:
When Select/deselect all options customize menu is enabled, and you press the Select All button, will select all option without considering data-max-options property of the <optgroup> as shown in this jfiddle. When the Select All button is pressed, is should only select one (preferrably the first <option> of that <optgroup>), as inicated by data-max-options property.


Missing documentation for top-level resolve

For example:
<EMAIL_ADDRESS>
This usage of resolve isn't documented - we only document require.resolve in the README of this project.

Never mind - deprecated.


filecache: create parent directories as well

First off, thank you for the very nice project!
Running this on Windows I initially got an error because %USER%/.cache did not exist, so here's a simple fix. :)

Ah, sorry for the lazy drive-by commit directly from the GitHub UI. Should be fixed now. :)

Codecov Report

Merging #69 (3ea7ffe) into main (68bdcdc) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #69      +/-   ##
==========================================
- Coverage   97.67%   97.64%   -0.04%
==========================================
  Files          12       11       -1
  Lines         431      425       -6
  Branches       76       66      -10
==========================================
- Hits          421      415       -6
  Misses          6        6
  Partials        4        4

Impacted Files
Coverage Δ

httpx_cache/cache/file.py
100.00% <100.00%> (ø)

httpx_cache/__init__.py

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Deployed as 0.6.1 !


Pass in data to disallow

Is it possible to pass in an object of data, for example - names, date of birth? It would be great to stop users using their own details.
I understand zxcvbn warns against common names, but it might be nice to stop people using anything related to them.
Great work! Thanks

ignore me, I should have read the docs correctly
user_inputs is already there!


Min-max algorithm bug fixes and example notebook

Fix all bugs listed in #43 .
Update notebook with comparison of instantaneous/min-max algorithms.

 This led to some doubling of code changes. I resolved these conflicts locally and committed the changes here. I also made some slight changes to the notebook.
Can you please quickly check that everything works as intended?
If you are surprised that I can push to your PR, this is the default behavior on github. I was also surprised. But it makes a lot of sense that the default is that it is possible for maintainers to push to a PR that points to their repository.

@cchwala I ran the new instantaneous vs min-max notebook and everything looks OK.

@DanSereb Thanks!


Visual artifacts when paging using PageView on Android

Test device: Pixel XL
The PageView contents are rendered correctly on the initial render but once you start scrolling they become corrupted. They will remain this way even after the scrollable "settles".
The following are examples of text and a Widget near the edges of a DecoratedBox:

From @abarth (on Gitter): "it sounds like a problem in the raster cache that once we cache the texture for the page we draw the ugly version".

/cc @chinmaygarde Any idea why the text might be rendering like this in the raster cache?

Not really. The way we setup the raster cache entry hasn't changed at all recently. Will try to reproduce this issue. I have a Pixel XL I can test on.

I see the issue. No updates yet.

There is at least one app in development shipping mid April which will be blocked by this.  We have time.  Just making note. :)

Are you actively working on this @chinmaygarde ?

Yes, I am.

Any status updates?

Any news @chinmaygarde?

The underlying issue causing https://github.com/flutter/flutter/issues/9276 affects this as well.

@chinmaygarde I presume this is not going to make the M2 alpha cut today and should get bumped to May?

Update: https://github.com/flutter/engine/pull/3717 addresses this issue. Unfortunately, I am still dealing with the edge cases.

@lukef was asking me about this this morning (blocks shipping his next app, but not development currently).  Looks like good progress is being made, just needs a few more rounds of bug fixing?

The fix for this issue has landed in https://github.com/flutter/engine/pull/3717.

Thank you @chinmaygarde!


Perform tuning of phenix-like prioritization

Is your feature request related to a problem? Please describe.
Currently, the performance of our phenix-like prioritization is not the same as for exomiser-phenix. This is shown by the gene-ranking-shootout. Also, the stored resnik values are quite big.
Describe the solution you'd like
For the performance, we should have a look at the decompiled phenix JAR. For the storage size, we should look into storing f32 instead of f64 and whether this degrades performance.
Describe alternatives you've considered
N/A
Additional context
N/A

Here is an issue that I found:
https://github.com/anergictcell/hpo/issues/44
I can work around this by adding a pheno prune command that manually prunes the association TXT file.

Pruning is not really necessary when using the genes_to_phenotype.txt file rather than vice versa. This can be addressed in varfish-db-downloader.
Beyond this, it probably makes sense to remove non-child terms as recommended in https://github.com/anergictcell/hpo/issues/44.
However, the issue for ARID1B (and probably others) remains when using the Phenix-like approach. Peter Robinson, the author of Phenix and other tools, recommends to drop Phenix as "not state-of-the-art" and adopt HiPhive or Lirical.
I will keep in the phenix-like tool as it can serve as a baseline being relatively simple and we can implement it easily enough in Rust. We will need to look into other directions, including making it easy to add HiPhive.


LVS error in LDO OpAmp circuit

@854768750 , @Lastdayends, please see the failing test. There is an open in vccx.

@kkunal1408 , annotation misses CMC_S_B, please see the failing test: test_ldo_opamp_annotation_pass

@kkunal1408, @854768750, @Lastdayends, there are four tests using LDO OpAmp circuit. Three of these fail due to connectivity, annotation or routing problems. The fourth one did not generate a placement in 50 minutes (@srini229 , this might be of interest to you for enumeration of grouped blocks).
Could you please review these issues?
@kkunal1408 , the connectivity issue is due to a subcircuit in user_templates.sp. If the subcircuits in user_templates.sp are not validated, let's retire/delete this file until there are tests in place.
FAILED test_circuits.py::test_ldo_opamp_1 - AssertionError: Broken connectivity: {'layer': 'M1', 'netName': 'mp103_mp6_mp5/B', 'rect': [64500, 19600, 65100, 24500], 'net...
FAILED test_circuits.py::test_ldo_opamp_2 - AssertionError: ckt_ldo_opamp_2 (ckt_ldo_opamp_2_0):Number of DRC errorrs: 1
FAILED test_circuits.py::test_ldo_opamp_3 - AssertionError: CMC_S_PMOS_B_plplvt_w2160_m1_nf12 not found in placement

test_circuits.py::test_ldo_opamp_4 does not finish in 50 mins.

@soneryaldiz to follow up

@soneryaldiz ldo_4 example constraints look wrong. Can you please fix them?

@kkunal1408 , this branch is too old to resurrect. I suggest to close this PR and re-open a new one with latest changes.

works for me. We can close this branch. Let me know if any of the circuit tests you want to keep or close altogether.

This branch is too far behind master. LDO amplifier will be added in a future PR.


Get lowest and highest "visible" adjacent month dates

So I'm making some calculations on a week by week basis. And there's obviously some weeks where the previous or next month "bleed" into the current month. This week is an example:
Sun March 29th - Sat April 4th
So the CLNDR does a good job of showing the adjacent months, but I want to pass the "first day" of the visible month to my ajax call to do the calculations I need. So for the month of April 2015, the begin date would need to be "03/29/15" and the end date needs to be "05/09/15".
Now obviously I could just load the CLNDR with no data, and then Jquery the values for the first "day" showing in the calendar and last day showing out of the generated Html, but I was wondering if CLNDR had a built-in way to do this.

This is interesting. Just to clarify, do you mean some way to get the first and last rendered day from the .clndr() object? Normally I'd just say exactly what you suggest: $('#cal').first('.day')... or something, but it's not nice. Since you have full control over the view, you could also set this info into the data- attribute on the calendar from within the view. Otherwise, I don't see a real need here to add this to the main library.


Implement metadata forms

AWS DataZone has a feature called: Metadata Forms. Essentially it's a dynamic form creator that can be attached to datasets.
I think this is an excellent feature and it would help data.all solve some problems.
One problem is that at the moment we don't know what metadata users want to keep on their datasets. Currently data.all hardcodes Confidentiality and Topics. For our use case for example Topics make no sense and for Confidentiality we use our own levels.
I propose the following:

Create a new Play module Metadata Forms
Forms are owned by teams who created them
Forms should be visible to teams who created them but should also have an option to be shared with everyone
Forms should allow to require them to be used on all data.all datasets without exception. These would be very useful for organization wide metadata like confidentiality levels. This option should only be available to DA admins.
Datasets should allow to attach from 1 to N metadata forms.
Metadata forms should allow to create fields: strings, integers, booleans and most importantly using Glossary Terms as this allows to create specific dropdowns. Using glossary terms I can create a required form for my own confidentiality levels.
(Optional) Allow to specify required forms on Organization level or Environment level. This way an environment or organization can enforce certain metadata forms are always used on their datasets.
Form fields which were set as required by DAAdmin should be searchable as that way we can search by things like confidentiality level.
Remove confidentiality and topics dropdowns from datasets. Offer a migration path to users who already use it.
Refactor confidentiality with new checkboxes on datasets: 1) hide schema without having access etc.. Instead of hiding the meaning behind a confidentiality level make it explicit with checkboxes what is allowed on a dataset.

Hi @zsaltys - I think the above idea is one that we would want to pick up for a future release of data.all as it would be very helpful for more dynamic metadata associations on data.all resources. I think this would be a good candidate to extend the Discover Section of the UI rather than the Play section which focuses more on data consumption use cases
Additionally, I would be curious to hear your thoughts on if you see a benefit of extending this Metadata Forms concept to more than just Datasets but also to other resources in data.all, such as Notebooks, Worksheets, etc. I think offering this type of flexibility everywhere could have value but curious to hear other opinions here
We will add this issue to our backlog for now and determine when we can prioritize

For 2.6 we will work on the design, implementation will be carried out in 2.7

Overview
Purpose: Create, manage, and attach metadata forms to datasets.
Ownership: Forms are owned by the teams that create them.
Visibility: Forms are visible to the creating team with options to share them environment-wide, organization-wide, global.
Enforcement: DA admins can enforce the use of specific forms environment-wide, organization-wide, global.

Form Name | Type | Description | Example Value
-- | -- | -- | --
Form Name | String | The name of the metadata form. | Confidentiality Level
Form Description | String | A brief description of the form’s purpose. | Form to specify confidentiality levels.
Owner Team | String | The team responsible for the form. | Data Governance Team
Visibility | Dropdown | Visibility options for the form. Values: Team Only, Organization-Wide. | Team Only
Enforce Use | Checkbox (Admin) | Enforce the form on all datasets organization-wide. | Checked

Form Name
Type
Description
Example Value

Form Name
String
The name of the metadata form.
Confidentiality Level

Form Description
String
A brief description of the form’s purpose.
Form to specify confidentiality levels.

Owner Team
String
The team responsible for the form.
Data Governance Team

Visibility
Dropdown
Visibility options for the form. Values: Team Only, Environment-Wide, Organization-Wide, Global.
Team Only

Enforce Use
Checkbox (Admin)
Enforce the form on all datasets organization-wide.
Checked

@anmolsgandhi I absolutely think that this should be extended further than just datasets. This would be very useful for environments too. But I think yes we should make it extendable to any entity. I don't personally use other things in the Play area besides worksheets.. I don't see how it could be useful for worksheets though.
But I would definitely want this for datasets, environments and probably organizations too.

@zsaltys Thanks for the above guidance, i do think this feature should be extended further and we will ensure to design it that way. At this moment, we are all hands on deck for v2.6 release but we will work with you more closely on this one. cc @SofiaSazonova

Below there is a resume of the discovery phase.
I focused on the implementation of Metadata Forms (MF), and left out of the scope for now, requirements for the data.all changes: Refactor confidentiality and etc.

Main goal: Create an effective instrument for data consistency improvement.
Requirements

Metadata Forms must be developed as Play Module  Discovery module for Data.all
Data.all users should have opportunity to create, edit and delete metadata form.
Metadata Forms must have different visibility levels: from private to global.
Metadata forms can be attached to:

Organizations
Organization Teams
Environments
Environment Teams
Datasets
Worksheets
Dashboards
Consumption roles
Notebooks
ML Studio entities
Pipelines

Metadata Forms can be obligatory to use
For each entity there can be multiple forms attached
Form fields should be searchable

Structure
Table S.1. Metadata Form Properties

Field
Description
Type
Possible Values

Metadata Form URI
Identifier of metadata form
String
URI of metadata form

Form Name
The name of the metadata form.
String

Form Description
A brief description of the form’s purpose.
String

Owner Team
The team responsible for the form.
String
Team URI

Visibility
Visibility options for the form.
String
Team Only, Environment-Wide, Organization-Wide, Global.

Home Entity
URI of Org/Env. If Visibility is Global or Team-only, this field is None
String?
URI of Org/Env or None

Table S.2. Metadata Form Field Configuration

Field
Description
Type
Possible Values

Metadata Form
Identifier of metadata form
String
URI of metadata form

Field URI
Identifier of the field
String

Field Name
The name of the field.
String

Field Type
The type of field.
String
String, Integer, Boolean, Glossary Term.

Required
Whether the field is required.
Boolean

Glossary Term
Allows selection of a glossary term for dropdown values. Available only if Field Type is Glossary Term.
String
URI of Glossary Term

Possible Values
Possible values. If not set, the value can be any. Glossary term values populated automatically.
Array?

Table S.3. Filled Metadata Form

Field
Description
Type
Possible Values

Metadata Form
Identifier of metadata form
String
URI of metadata form

Filled form URI
Identifier of filled form
String
URI

Entity
Identifier of Entity that is form attached to
String
URI

Entity Type
Type of entity
String
Table E.2

Table S.4. Filled Metadata Form Filed

Field
Description
Type
Possible Values

Field URI

String

Filled form URI

String

Value
String representation of filed value
String

Enforcement
Metadata forms can be obligatory to fill in on different levels. User can select the metadata form and entity types, that should have this form attached.  Enforcement affects selected entity types on all lower levels hierarchically.
Table E.1. Metadata form enforcement rule

Field
Description
Type
Possible Values

Metadata Form
Identifier of metadata form, that is enforced
String
URI of existing form

Level
Level of affected entities. All lower levels will be affected.
String
From table E.2

Entity Types
Entity type affected by this enforcement
Array [String]
Identifiers of data.all entities types (table E.2)

Severity

String
Obligatory/Recommended

Table E.2. Metadata form levels’ hierarchy

Who can enforce:

Data.all admins can enforce any form on any level across the platform. They have full control over metadata form enforcement.
Owners/admins of the data can enforce forms for this levels and levels below in the hierarchy. For example, an org admin can enforce a form for the org, all teams in that org, all environments in the org, all datasets in those environments, etc.
Share approvers and requestors can enforce forms for a specific share they are involved with. However, they can only delete enforcement rules they created themselves - they cannot delete rules created by others

So in summary, enforcement capabilities cascade along with administrative privileges in the hierarchy. Global admins have full control, org/env admins can enforce for their sphere and below, dataset admins for the datasets and items in it, and share approvers for a specific share.

Re-opening this issue - think was accidently auto closed with the merge of #1422

@SofiaSazonova I like everything I see, I like the enforcement mechanisms. One I would like to bring up potentially as next enhancement later on is default values.
Let us imagine a use case where I as DA admin define a global metadata form with 30 fields I want all my datasets to answer.  Such as: Is this PII data? Is this GDRP data? JIRA link? DEV contact? DevOps Contact? Domain? Subdomain? Now many of these on the organization level would be very repetetive to define. For example let's say I have organization Sales and I have 30 datasets. Well the domain for us is always SALES so there's really no point to keep asking every dataset creator to define this on every import in fact maybe I want to enforce that in my organization the domain is ALWAYS sales. I would imagine this feature as "metadata form defaults". The way it would work if a metadata form is attached to an org for all their datasets then that org should be able to provide their defaults for that required metadata form.
@anmolsgandhi fyi

@zsaltys @TejasRGitHub @anushka-singh
Here is the demo. Not all of this is merged, but it will soon)
https://github.com/user-attachments/assets/cb4e1aed-e75a-4c9d-8086-9d417b026935

Thanks for this demo video @SofiaSazonova
It looks really good and user friendly!
I just had one comment. I believe metadata forms should be attached at dataset level and not org level because we are filling in metadata for datasets right? One org can have multiple datasets and each dataset can have a different metadata form.
Also is there a way to make metadata form mandatory to be filled out? It might be a useful feature for some customers.

@anushka-singh

I believe metadata forms should be attached at dataset level and not org
By design (in previous messages) it can be attached at many levels. As we discussed above, we shouldn't restrict MFs only to datasets.

Also is there a way to make metadata form mandatory to be filled out
Yes, It is also described in the design. It will be implemented in next step.


Disable RTTI on x-plat

My RTTI change "just worked" on Linux, which seemed wrong. Turns out, we never disabled RTTI to begin with. This change brings a release build of libChakraCoreStatic.a down from 484MB to 463MB.

That build break doesn't happen when i build locally in Linux... Looking into it now.

Update: apparently on Linux and macOS, debug and test builds require RTTI to be enabled throughout the codebase (anywhere that uses Allocator.h) because of a call to typeid() in the allocation tracking code. MSVC seems to optimize this out because our calls to typeid are always on static types rather than runtime objects, but clang doesn't seem to perform this optimization (for better or worse).


Add button for exporting plot information from visualization interface

As discussed in #80. See comments there for implementation details w/r/t the sample plot.
Could also add a button for exporting rank plot information (it'd just be a list of the features included, I guess).
Code should detect if the currently used metadata field is Sample ID or rankratioviz_balance—if so, then the output data should only be two columns (since these fields would already be in the output anyway).

Very cool. I think your ideas discussed in #80 are spot on. The three values that I would love to have in a .tsv are Sample ID, current metadata value, and log ratio. I do wonder about how to arrange the currently selected feature IDs. If someone is filtering on a taxon the numerator and denominator could include very many features IDs that could be clunky dropped into a column header. This probably won't cause much trouble in a pandas df anyhow.
I would say maybe a second .txt with the feature IDs could work, but I'd hate to separate outputs like that if it isn't necessary.

That's a good point re: the log ratio feature IDs. I don't think it'd break anything for pandas, but it might make things look ugly if there's a super-long column name for the log ratio.
One option would be putting all of the numerator feature IDs in the first line of the TSV file, all of the denominator feature IDs in the second line of the file, and then including all of the data afterwards (you could use something like the skiprows option in pandas.read_csv() to ignore these when reading the file).
However, I'm not so sure I like that option (having a normal TSV file would make parsing easier). I can probably get rankratioviz' visualization interface to output two files at once, one being a normal TSV for the sample plot (with just the log ratio column name being feature_log_ratio or something) and one being a list of feature IDs in the rank plot. This would work similarly to the screenshot functionality I just added in, where it tries to download the two files after you press a button once.
...That being said, this is thankfully a pretty minor problem—I'd imagine that once the initial functionality for this is ready, it'll be possible to change up the output details to be as useful as possible :)

@ssteiche The latest version of rankratioviz should have controls to let you export a .tsv file of all currently-drawn points in the sample plot, with the three fields mentioned (sample ID, current log ratio, and current metadata value).
There are two caveats to this: 1) I don't have a solution for exporting the currently-selected ranked features ready yet, and 2) I haven't been able to write any unit tests to verify that the exporting functionality is working 100%. (To make a long story short, there's another issue (#85) which is causing me problems with testing certain parts of the visualization interface. I'll try to get to addressing that eventually.)
In the meantime, to fix 1) above—if you want to know which features have been selected, you can just copy the text in the big text boxes on the left of the display with all the feature IDs (the top box shows all selected "numerator" features, and the bottom box shows all selected "denominator" features).
Hope this helps!

This is working excellently, thanks again Marcus! I might need to add you as an author on the manuscript for all the time you've save me here :).
One thing that might make the output slightly more portable would be to label the Log Ratio field in snake case Log_Ratio. I know it's silly, but it can make things really interface more smoothly. I read the .tsv into R to reproduce the plot with regression stats (below), but had a heck of a time figuring out what was throwing errors until I simply added in that underscore. Just one more example of how inexperienced users such as myself will butcher your program!
Thanks again, and this had been a tremendous help.

Haha I'm glad to hear things are working well! I'm super happy that this has been useful for your research.
Thanks for letting me know about the issue with spacing in the TSV file! The latest version of rankratioviz should solve the problems with the TSV file headers. Sample ID and Log Ratio are both now written as Sample_ID and Log_Ratio, and any metadata field values or sample IDs with whitespace in them are enclosed in double quotes (""). So the output should work well with most libraries (R included), hopefully! Again, thanks a ton for the heads up here—I'm glad I found out about this problem early so I could fix it up front :)
One more thing, speaking of problems: I recently realized that there was a bug in my code where samples with non-numeric values in the x-axis metadata field (e.g. Missing: Not provided) were being displayed on the chart with x = 0. Part of what led me to figuring this out was seeing your screenshot yesterday in #80, where I noticed a sort of "clustering" of points around x = 0. I documented this issue in #91, and just fixed it in the latest version of rankratioviz. (Now, these points will be filtered from the chart based on whatever the current x-axis is, although they'll still be included in the exported TSV data.)
It looks like the chart you generated doesn't have this problem—I'm guessing R filtered out the non-numeric values. However, I figured I should let you know so you can update to the latest version (which doesn't exhibit this bug).
Thanks again for your suggestions and for using rankratioviz!  I was staring at the interactive plot for a bit and wondering to myself if I really was seeing a vertical line of points at zero. You're also correct that I filtered the null values out for the R plot, which seems like the appropriate decision so I'm glad rankratioviz will handle them this way now.


[Feature] Better support for annotated text extraction

Currently, the Obsidian's native PDF viewer can extract annotated text (= text that is marked up by a text markup annotation (highlight etc) written inside the PDF file) only when textContentItem.char is present and nonempty (char is a property of TextContentItem present only in Obsidian's version of PDF.js).
This is due to how the Obsidian team implements PDFViewerChild.getTextByRect.
The two PDFs appearing in #223 are good examples. The first one has its text content items with non-empty char, whereas in the latter it's empty.
Since PDF++ uses the getTextByRect method as-is, this restriction also applies to PDF++.

[!note]
So in short, this is a problem of Obsidian itself, and PDF++ is not to be blamed for this. It just inherits the issue from the Obsidian core.

However, it's probably possible to fix this on PDF++'s end by measuring the bounding box of text divs using Range.getBoundingClientRect and monkey-patching the method.
It will be beneficial for some users.

Basically we can just reuse this snippet:
https://github.com/RyotaUshio/obsidian-pdf-plus/blob/d9f9d20084d794c928343ae6860322dcc9f7442a/src/vim/text-structure-parser.ts#L201-L225
But we need to fix the issue that it assumes textDiv.childNodes[0] is a text node, which can be wrong when search matches are rendered.

Released in 0.40.8.


Translate pins to other card

I get this error when i connect it and Not changing board:
20:49:30 | [W] | [p1reader:214] | CRC buffer overrun. Resetting.
-- | -- | -- | --
20:49:36 | [I] | [ota:109] | Boot seems successful, resetting boot loop counter.
20:49:40 | [W] | [p1reader:214] | CRC buffer overrun. Resetting.
20:49:50 | [W] | [p1reader:214] | CRC buffer overrun. Resetting.
20:50:00 | [W] | [p1reader:214] | CRC buffer overrun. Resetting.
20:50:10 | [W] | [p1reader:214] | CRC buffer overrun. Resetting.
20:50:20 | [W] | [p1reader:214] | CRC buffer overrun. Resetting.
20:50:30 | [W] | [p1reader:214] | CRC buffer overrun. Resetting.
20:50:40 | [W] | [p1reader:214] | CRC buffer overrun. Resetting.
20:50:50 | [W] | [p1reader:214] | CRC buffer overrun. Resetting.
20:51:00 | [W] | [p1reader:214] | CRC buffer overrun. Resetting.

I am getting this error when i change board in Yaml-file and tries to install it, and would like help to resolve the correct code/pins for my card.
I use this card: https://i0.wp.com/randomnerdtutorials.com/wp-content/uploads/2019/05/ESP8266-NodeMCU-kit-12-E-pinout-gpio-pin.png?quality=100&strip=all&ssl=1
And i have it connected:
1 = 5v -> VIN
2 = RTS -> D2 (GPIO04)
3+6= GND/DGND -> left GND
4= Not used
5= TX -> Right GPIO03
INFO Reading configuration /config/esphome/p1mini.yaml...
Failed config

switch.gpio: [source /config/esphome/p1mini.yaml:50]
  platform: gpio
  id: p1_cts
  pin:

    Cannot resolve pin name 'D2' for board esp01_1m.
    number: D2
switch.gpio: [source /config/esphome/p1mini.yaml:54]
  platform: gpio
  id: status_led
  pin:

    Cannot resolve pin name 'D4' for board esp01_1m.
    number: D4
    inverted: True
binary_sensor.gpio: [source /config/esphome/p1mini.yaml:61]
  platform: gpio
  id: secondary_p1_rts
  pin:

    Cannot resolve pin name 'D0' for board esp01_1m.
    number: D0
    mode: INPUT_PULLDOWN
    inverted: False

Try changing "number: D2" to "number: GPIO4", "number: RX" to "number: GPIO3" and "number: TX" to "number: GPIO1". Maybe TX and RX work already and you can keep them as is.
I don't know if your board has an onboard led and which gpio controls it, but change "number: D4" into "number: GPIOn" and if you don't have an led, just use a random free gpio.
D0/secondary_p1_rts is only needed if you want to pass the signal along to a secondary P1 reader, which I assume you do not, so set it to any free gpio. Not all gpio:s support "INPUT_PULLDOWN", but if you try to use one that does not, you will probably get a warning telling you which ones do.

Still the original error:

22:07:39 | [W] | [p1reader:214] | CRC buffer overrun. Resetting.
-- | -- | -- | --
22:07:39 | [D] | [switch:017] | 'p1_cts' Turning OFF.
22:07:39 | [D] | [switch:037] | 'p1_cts': Sending state OFF
22:07:40 | [D] | [switch:017] | 'status_led' Turning OFF.
22:07:40 | [D] | [switch:037] | 'status_led': Sending state OFF
22:07:40 | [D] | [switch:013] | 'p1_cts' Turning ON.
22:07:40 | [D] | [switch:037] | 'p1_cts': Sending state ON
22:07:40 | [D] | [switch:013] | 'status_led' Turning ON.
22:07:40 | [D] | [switch:037] | 'status_led': Sending state ON
22:07:42 | [D] | [sensor:126] | 'p1mini WiFi Signal': Sending state -73.00000 dBm with 0 decimals of accuracy

22:07:39 | [W] | [p1reader:214] | CRC buffer overrun. Resetting.
-- | -- | -- | --
22:07:39 | [D] | [switch:017] | 'p1_cts' Turning OFF.
22:07:39 | [D] | [switch:037] | 'p1_cts': Sending state OFF
22:07:40 | [D] | [switch:017] | 'status_led' Turning OFF.
22:07:40 | [D] | [switch:037] | 'status_led': Sending state OFF
22:07:40 | [D] | [switch:013] | 'p1_cts' Turning ON.
22:07:40 | [D] | [switch:037] | 'p1_cts': Sending state ON
22:07:40 | [D] | [switch:013] | 'status_led' Turning ON.
22:07:40 | [D] | [switch:037] | 'status_led': Sending state ON
22:07:42 | [D] | [sensor:126] | 'p1mini WiFi Signal': Sending state -73.00000 dBm with 0 decimals of accuracy

Aidon 6534

Oh, then you are in luck! Aidon 6534 uses an older binary format instead of the ascii format used by most meters.
However, the binary format was just implemented, so you only have to update to the latest version and everything will work!

That is good news!
I will try in a couple of hours and write again then!

YES!  It's working!  Thank you for the binary update! =)


Consider changing the ruleset naming

Problem is we have use /all as the only ruleset we had in azure-core and arm.
However this is confusing as with the addition of cannonical ruleset all isn't actually all its more required.
Proposing we change to the following:

all (Actually every single rule)
required or recommended (Only required rules)
<any-optional-ruleset>

This of course would need some change in the tspconfig validation of the spec repos as well as replacing.

Should be done after this has been rollout #854

https://github.com/Azure/typespec-azure/issues/654 is done, made a pr to automatically generate the all ruleset microsoft/typespec#3462 after this is merged we can remove all the explicit all rulesets and that will respect the established rules


fix: type in kes cef

:tada: This PR is included in version 2.16.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Get Location from the Home Zone in hass.config

Describe the feature
This is related to #46 which is a request for a templating integration.
This conversation on the HA Community may be useful.
https://community.home-assistant.io/t/how-does-an-integration-access-has-set-location/734850
Read Location from Home Assistant's Location
This feature is a simpler request which is for the integration to use the Home Zone as the source of the location. This is exposed on the HASS object under hass.config.
On every run, this integration should read the latitude and longitude from hass.config and that would solve the mainline stationary and mobile cases.
This should be the default behavior for new installs.
For existing installs, the existing specified location can be stored as the advanced config.
Advanced Configuration
Location
One might want to use Pirate Weather to see the weather at some other location, in that case the integration can expose manually setting location in advanced configuration which is normally hidden from the user.
Language
Hass.config exposes language and advanced setting could be used an override, or the HASS language could be used instead. The README says only English is supported though, so maybe just remove this and use the HA one if is needed in the future.
Units
Hass.config exposes units. It may be that the HA Units are not specific enough, and you would want and override to enable "SI". I believe UK, CA, US would be covered.
Related: Updating Location using homeassistant.set_location
Mobile uses can use homeassistant.set_location to set the location. Below is an example of using Peplink Mobile Router's GPS (captured from API calls) to do so.
alias: update_peplink_gps
trigger:
  - platform: state
    entity_id: sensor.peplink_gps
action:
  - service: device_tracker.see
    data:
      dev_id: van_luna
      gps:
        - "{{ state_attr('sensor.peplink_gps', 'latitude') }}"
        - "{{ state_attr('sensor.peplink_gps', 'longitude') }}"
  - service: homeassistant.set_location
    data_template:
      latitude: "{{ state_attr('sensor.peplink_gps', 'latitude') }}"
      longitude: "{{ state_attr('sensor.peplink_gps', 'longitude') }}"

Home Assistant version
latest
Integration version
latest
Acknowledgements

[X] I have searched this repository and API Repository to see if the feature has already been requested.
[X] This featre request is sepecific to the HA Integration.
[X] I have written an informative title.

Thanks for asking that question and bringing it to my attention! I was thinking about this a little over the weekend and might have something that could work here. The basic idea is that if someone puts a lat/lon in during setup, use that. Otherwise, the update coordinator could check the location on every update and use that.
The issue here is how to identify the update coordinator to make sure an entity knows what data to read from. This is currently based on location, but there's no real reason we couldn't use the entity name to define it instead. It'll take me a couple weeks to get to this, but adding it to the docket!

I'll add that displaying the location being used on the card would be nice to have too.

When you say location being used do you mean the coordinates of the current location or the name of the current location? Pirate Weather doesn't have a data point for the current location name (you can always request this as a feature in the API repo) but showing the coordinates should be doable.

I'd live with coordinates, I'm frequently not near a named place anyway.

It should be possible to show the location coordinates somewhere but I'll defer to @alexander0042. I also created a feature request for a location name being returned in the API.

Any ETA on this? Dying to get dynamic weather based on location of my RV (I have the pep wave integration)

People following this Issue should also go put a 👍 on the linked API issue.  https://github.com/Pirate-Weather/pirateweather/issues/260

Thanks for tagging that issue- if I can find a way to cache the geonames data, then it's very possible to return a location name.
With respect to the weather card, it's based off the integration name, so I don't know if it would be possible to update it dynamically. Maybe there's another card that might work better for this?
In the meantime, I'm testing a new update that lets the latitude and longitude be left blank during the initial setup, which will then let it pull the data from hass on each update!

V1.5.5 has just been released which allows you to enter in 0 as the latitude and longitude to pull in the current location from HA. It doesn't include a location name or output the current coordinates at the moment.

Would outputting the current coordinates (as sensors I guess) be helpful here? It'd be easy enough to do, but I'm not sure the value beyond debugging

Would outputting the current coordinates (as sensors I guess) be helpful here? It'd be easy enough to do, but I'm not sure the value beyond debugging

I know we used to have debug statements which printed the co-ordinates that it called the API with and I remember we removed it as it was caught by CodeQL.
It could maybe be added back and just ignore the CodeQL warning or if having a sensor would make more sense then it could be setup instead.

Just as an FYI on the sensor thing...
If you are dynamically updating the location of your HA instance, you must already have some sensor in order to do that.
I'm using a USB GPS Attached to my Victron Cerbo GX which I'm then collecting over MQTT. Other might be using the GPS in their cellular router (like a Peplink) or their Starlink.


How to reference env variables

Sorry for my ignorance but I am trying to us an ENV variable set by kubernetes in a replication container, but all the kubenetes ENV variables come up empty:
bash-3.2$ kubectl exec curlrc-mgu06 -- printenv | grep API
API_PORT=tcp://<IP_ADDRESS>:8080
API_PORT_8080_TCP_PROTO=tcp
API_PORT_8080_TCP_ADDR=<IP_ADDRESS>
API_PORT_8080_TCP_PORT=8080
API_PORT_8080_TCP=tcp://<IP_ADDRESS>:8080
API_SERVICE_HOST=<IP_ADDRESS>
API_SERVICE_PORT=8080
bash-3.2$ kubectl exec curlrc-mgu06 -- echo $API_SERVICE_HOST

bash-3.2$

whereas echo $HOME works as expected.
What is the correct way to reference these?

@cobrowserAlex I'm not sure what are you asking there

@rimusz i want to use the ENV variable API_SERVICE_HOST inside a docker image but the value is always empty.
So for instance sed -i "s/api_host/${API_SERVICE_HOST}/g;" /etc/nginx/conf.d/location-api.conf or any other variant i tried doesn't work since API_SERVICE_HOST is empty.

@cobrowserAlex I do not see the variable API_SERVICE_HOST even on GKE cluster inside the docker container. Try to use KUBERNETES_SERVICE_HOST

@rimusz I was reading this documentation and trying something similar, like using   NGINXSVC_SERVICE_HOST in a shell script.
I expect the ENV variable API_SERVICE_HOST to exist since i have a service named api. it seems to exist because I see the ENV variable in any container when I run printenv inside the container, like in my op.
API_PORT=tcp://<IP_ADDRESS>:8080
API_PORT_8080_TCP_PROTO=tcp
API_PORT_8080_TCP_ADDR=<IP_ADDRESS>
API_PORT_8080_TCP_PORT=8080
API_PORT_8080_TCP=tcp://<IP_ADDRESS>:8080
API_SERVICE_HOST=<IP_ADDRESS>
API_SERVICE_PORT=8080

The problem is that when I reference the exposed hostname ENV variable in a shell script, it comes up empty.

But the doc you referred I do not see any API_PORT referenced there,  only KUBERNETES_SERVICE_HOST is:
$ k exec nginx-s40au -- printenv | grep SERVICE
KUBERNETES_SERVICE_HOST=<IP_ADDRESS>
KUBERNETES_SERVICE_PORT=443

@rimusz ok I am not clear. Let me try again with a simple example.
I have a replication controller setting an ENV variable like this:
apiVersion: v1
kind: ReplicationController
metadata:
  name: php-fpm-api
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: php-fpm-api
    spec:
      containers:
      - name: php-fpm
        image: some-php-fpm-image
      ports:
        - containerPort: 9000
        env:
          - name: MONGO_MAIN_DB
            value: cobrowser_v3

sorry hit the wrong button.

sorry, that has noting to do with the Kubernetes cluster setup.
this is how the php scripts handles env variables.
I'm not the php expert, sorry

@rimusz ok, thanks anyway.
by the way strange that referencing $HOSTNAME does work:
kubectl exec php-fpm-api-zmbb5 -- sh -c 'echo $HOSTNAME'
php-fpm-api-zmbb5

@rimusz thanks!
As usual trying to figure out a way to explain the problem led me to the solution as well as your answer:
kubectl exec php-fpm-api-zmbb5 -- php -r 'echo getenv("MONGO_MAIN_DB");'

will yield the correct result.

cool, you got it sorted

For anyone else finding this, I paste the shell variant to get the environment variables. The key is to open a new shell.
sh -c 'sed -i "s/domain_placeholder/${DOMAIN_NAME}/g;" /var/www/config/config.js'


master build broken for some time

Hi there,
I administrate Compiler Explorer and we've been trying to build daily SPIRV tooling. It's been broken for us for a while and I wondered if someone could help us. We build with this script and it fails with this error :
  /root/SPIRV-Tools/source/opcode.cpp: In function 'int32_t spvOpcodeGeneratesType(spv::Op)':
  /root/SPIRV-Tools/source/opcode.cpp:345:19: error: 'OpTypeHitObjectNV' is not a member of 'spv::Op'
    345 |     case spv::Op::OpTypeHitObjectNV:
        |                   ^~~~~~~~~~~~~~~~~
  [ 18%] Building CXX object source/CMakeFiles/SPIRV-Tools-static.dir/val/validate_constants.cpp.o

Is it expected that the master should build?

Confirm on Android as well
make: Entering directory 'D:/Development/Vulkan-ValidationLayers/build-android/SPIRV-Tools/android_test'
"[arm64-v8a] Generate language specific header for DebugInfo: headers <= grammar"
"[arm64-v8a] Generate language specific header for OpenCLDebugInfo100: headers <= grammar"
"[arm64-v8a] Generate language specific header for NonSemanticShaderDebugInfo100: headers <= grammar"
"[arm64-v8a] Generate enum<->string mapping <= grammar JSON files"
"[x86_64] Generate language specific header for DebugInfo: headers <= grammar"
"[x86_64] Generate language specific header for OpenCLDebugInfo100: headers <= grammar"
"[x86_64] Generate language specific header for NonSemanticShaderDebugInfo100: headers <= grammar"
"[x86_64] Generate enum<->string mapping <= grammar JSON files"
"[armeabi-v7a] Generate language specific header for DebugInfo: headers <= grammar"
"[armeabi-v7a] Generate language specific header for OpenCLDebugInfo100: headers <= grammar"
"[armeabi-v7a] Generate language specific header for NonSemanticShaderDebugInfo100: headers <= grammar"
"[armeabi-v7a] Generate enum<->string mapping <= grammar JSON files"
"[x86] Generate language specific header for DebugInfo: headers <= grammar"
"[x86] Generate language specific header for OpenCLDebugInfo100: headers <= grammar"
"[x86] Generate language specific header for NonSemanticShaderDebugInfo100: headers <= grammar"
"[x86] Generate enum<->string mapping <= grammar JSON files"
[arm64-v8a] Compile++      : SPIRV-Tools-opt <= aggressive_dead_code_elim_pass.cpp
In file included from ../source/opt/aggressive_dead_code_elim_pass.cpp:18:
In file included from ../source/opt/aggressive_dead_code_elim_pass.h:30:
In file included from ../source/opt/basic_block.h:29:
In file included from ../source/opt/instruction.h:33:
../source/opt/reflect.h:57:29: error: no member named 'OpTypeHitObjectNV' in 'spv::Op'
         opcode == spv::Op::OpTypeHitObjectNV;
                   ~~~~~~~~~^
1 error generated.
make: *** [D:/Programs/Android/Sdk/ndk/25.1.8937393/build/../build/core/build-binary.mk:424: D:\Development\Vulkan-ValidationLayers\build-android\SPIRV-Tools\build\app/local/arm64-v8a/objs/SPIRV-Tools-opt/source/opt/aggressive_dead_code_elim_pass.o] Error 1
make: Leaving directory 'D:/Development/Vulkan-ValidationLayers/build-android/SPIRV-Tools/android_test'

SPIRV-Tools builds needs a newer SPIRV-Headers to build.
In this case it needs at least https://github.com/KhronosGroup/SPIRV-Headers/commit/c214f6f2d1a7253bb0e9f195c2dc5b0659dc99ef which was merged in SPIRV-Headers in Nov 16.

It looks like the SPIRV-Tools code was updated to use a more recent feature in SPIRV-Headers, but the DEPS file was not updated.  Unfortunately the SPIRV-Tools CI bots didn't discover the problem, and I've filed #5014 to make the SPIRV-Tools CI bots use the DEPS file.  It's only fair.

Thanks @dneto0

Landed the systematic fix #5031 to #5028


Correcting timezone advice about CronJob

This PR is closes issue #19268  and also reverts #18715. Timezone is not hardcoded as UTC, so corrected note section.

@sftim Can you please have a look at this.

@rajeshdeshpande02 could you perhaps instead review https://github.com/kubernetes/website/pull/19269?

Closing this one as the issue is already fixed with different PR


Websocket function not working (Websocket error object event)

Hi, I want to ask.
When  I run python manage.py runserver to run the program,
And prompt to <IP_ADDRESS>/dsserver to run deepspeech server,
The http protocol is working fine , but websocket is throwing error, Object event.
Is the websocket functioning?
Thank you.
Regards,
Asyraf

Websocket should be functioning if you have channels==1.1.8. If that doesn't work, I might need detailed stack trace


Skill

Fixes

Fixes #109 by @[khushigarg1]
Types of changes

[ ] Bug/Hot fix (non-breaking change which fixes an bug)
[x] New feature (non-breaking change which adds a functionality/feature)
[ ] Refactoring Old Code ( non-breaking change replacing already existing code)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Description

Preview Link

https://khushigarg1.github.io/Terminal-Portfolio/
Checklist:

[x] I have read the CONTRIBUTING guidelines in repository.
[x] My pull request has a descriptive title and is attached to an existing issue.
[x] I haven't repeated the code and it follows established code style of repository.
[x] My branch is up-to-date with upstream branch.

@TechSpiritSS  Please check now!

Great 🚀


Consider adding Azure Pipelines badge

Follow up to #2299. Would probably be good to have if we're deciding to keep Pipelines around.

I totally meant to do this but got bogged down by the Coveralls stuff and totally forgot.  Thanks!


Possible? Run on embedded system using gdb function calls

Hi,
I'm coming from an embedded background and have just decided to start using unit tests for my development and found that catch is really nice (easy to use!).
I'm using STM32 mcus and program them in C/C++ using eclipse with gnuarm plugin.
So far, the consensus for embedded unit testing seems to be that there are two options to do it:

test only hardware independent code on development pc
use QEMU or similar for hardware simulation

I wonder whether it would be possible to do unit testing of the actual code running on the mcu by using gdb to call the functions under test?
I've just done a quick proof of concept experiment and it seems to work:
http://imgur.com/a/0nm7W
I've basically created a simple function and called it using varying parameters during a debug. I think it should be possible to automate it and implement it into an existing framework.
The process could be:

use no optimization so no unused code gets deleted during compilation
halt cpu at known point after HW init
then run all tests from host PC, call all functions and check all returns and variable contents using gdb

Result:

No code overhead on mcu
code under test is compiled for and running on target hw

I'd be interested in an opinion about this. Where would I have to look in catch if I wanted to reroute all function calls to gdb? Or is there something I'm missing?
Thanks!

Sounds like an interesting idea.
There's (currently) nothing in Catch to explicitly support this. But if you can put the calls out to gbd (and be able to receive and interpret the output) within a Catch TEST_CASE you could do that manually (made more manageable with some helper functions, I suspect).
One thing on the wishlist for Catch in the future is to be able to run out-of-process. The main idea there is to run the Catch executable itself out-of-proc (for full isolation or death-tests) - but such a facility could probably be used to execute GDB. However that's not currently a work in progress, so at this stage you'd have to handle that part yourself.
If you do decide to take this up I'd be interested to hear how it goes, and whether there are some parts (such as the out-of-proc runner) that it makes sense to incorporate into Catch itself in the future.


[stable/efs-provisioner] Upgrade EFS provisioner

What this PR does / why we need it:
Upgrade the provisioner to the latest version (v2.2.0-k8s1.12).
Add myself to OWNERS.
Checklist

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/chart])

/assign @hareku

@srueg Thank you for your PR.
I think this PR is not necessary. Users can specify efs-provisioner tag from values.yaml, and default tag should be latest.
If you want to be an owner, please make another PR.

But the currently configured tag is v2.1.0-k8s1.11 and the Chart references an appVersion. I would much prefer to provide a sane default.

https://github.com/helm/charts/blob/14dcbf62bb04a9ae15be1f91c3f5db9067cd948b/stable/efs-provisioner/templates/deployment.yaml#L49
It seems like a tag references .Values.image.tag.
Is this understanding mistaken?

Yes, you're right. I just meant currently the values.yaml file references v2.1.0-k8s1.11 and not the latest tag.
It's also my understanding to be considered best practice to specify an image tag instead of using latest.

/approve

/lgtm
/ok-to-test


Why could a multiclass dice loss function solve the class imbalance problem?

@FabianIsensee Hello, sir. I notice that in the paper of this method the class imbalacnce issue was disposed by formulating a multiclass dice loss function. I cannot figure it out why the loss function could play the role. Could you detailed it, thank you!

In crossentropy, each pixel has the same weight irrespective of the class. by using a Dice loss, the weight of a pixel is different. If the CE tumor is small for example, then false positives or false negatives will impact the dice loss more and will thus intrinsically be weighted more.

I have understood it from your answer, thank you very much!,


fix: fix bugs in two examples

This PR fixes bugs that caused two examples to not properly work. After merging this PR they will work again.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.1%) to 89.685%

Files with Coverage Reduction
New Missed Lines
%

lib/cache.js
2
94.74%

Totals

Change from base Build<PHONE_NUMBER>:
-0.1%

Covered Lines:
1075

Relevant Lines:
1168

💛  - Coveralls


Call to a member function setRows() on null

Hey,
I'm getting this error when trying to read csv file.
There is a support for null values?
Thanks!

Please show us code. Impossible to help like this


Adds carflag-v0 env

Car Flag tasks a car with driving across a 1D line to the correct flag. The car must first drive to the oracle flag and then to the correct endpoint. The agent's observation is a vector of three floats: its position on the line, its velocity at each timestep, and the goal flag's location when it reaches the oracle flag. The agent's actions alter its velocity: it can accelerate left, perform a no-op (maintain current velocity), or accelerate right.

Thank you for the detailed comments and support. I have added the easy, medium, and hard levels and updated the list of environments accordingly.
While writing the tests, I noticed a bug in the DTQN codebase. The env currently returns only a done state instead of differentiating between truncated and terminated states.
To address this, I observed that the minesweeper env code uses a combination of the timestep and board dimensions to calculate the maximum time it could take, which works well with the varying game levels since the dimensions change. Could you suggest a scalable method to calculate the truncated state across all levels in the carflag env?

Yeah this is tricky. The minimum time required would be the time to reach maximum velocity, drive to the oracle, slow down to 0 velocity, accelerate to max velocity in the direction of the goal, then drive to the goal.
Do you know how many timesteps they used in the DTQN paper?

The DTQN paper has set the maximum_time_steps to be 200 for the easy mode.

Would it be better to set an arbitrary value or one based on the environment conditions?

Ideally, I'd say it should be set based on environment conditions. How many timesteps would it take an optimal policy to reach the oracle and then heaven?

Hey, I trained the DTQN policy and it took 85 timesteps for it to reach the oracle and then to heaven.

Oh if you have a working DTQN implementation it's even easier! How many timesteps does it take on medium and hard?

85-137 for easy, 125-175 for medium and 165-210 for hard.

How does easy: 200, medium: 300, hard: 400 timesteps sound?

Okay sure :+1:

Hey @smorad, I am getting this error in the testcases but unable to figure out what's wrong:
=================================================================== short test summary info ===================================================================
FAILED tests/test_wrappers.py::test_markovian_state_space_full[CarFlagEasy] - ValueError: space Box([-1.1  -0.07 -1.  ], [1.1  0.07 1.  ], (3,), float32) does not contain data (array([-0.4428458,  0.       ,  0.       ], dtype=float...
FAILED tests/test_wrappers.py::test_markovian_state_space_full[CarFlagMedium] - ValueError: space Box([-1.1  -0.07 -1.  ], [1.1  0.07 1.  ], (3,), float32) does not contain data (array([-0.3615731,  0.       ,  0.       ], dtype=float...
FAILED tests/test_wrappers.py::test_markovian_state_space_full[CarFlagHard] - ValueError: space Box([-1.1  -0.07 -1.  ], [1.1  0.07 1.  ], (3,), float32) does not contain data (array([1.020816, 0.      , 0.      ], dtype=float32), 0...
FAILED tests/test_wrappers.py::test_markovian_state_space_partial[CarFlagEasy] - ValueError: space Box([-1.1  -0.07 -1.  ], [1.1  0.07 1.  ], (3,), float32) does not contain data [ 0.32249273 -0.0015     -1.        ]
FAILED tests/test_wrappers.py::test_markovian_state_space_partial[CarFlagMedium] - ValueError: space Box([-1.1  -0.07 -1.  ], [1.1  0.07 1.  ], (3,), float32) does not contain data [ 0.73226799 -0.0015      0.        ]
FAILED tests/test_wrappers.py::test_markovian_state_space_partial[CarFlagHard] - ValueError: space Box([-1.1  -0.07 -1.  ], [1.1  0.07 1.  ], (3,), float32) does not contain data [-0.7669738  0.0015     0.       ]
FAILED tests/test_wrappers.py::test_markovian_state_space_info_dict[CarFlagEasy] - ValueError: space Box([-1.1  -0.07 -1.  ], [1.1  0.07 1.  ], (3,), float32) does not contain data (array([0.10640713, 0.        , 0.        ]), 0.5, 1.0, ...
FAILED tests/test_wrappers.py::test_markovian_state_space_info_dict[CarFlagMedium] - ValueError: space Box([-1.1  -0.07 -1.  ], [1.1  0.07 1.  ], (3,), float32) does not contain data (array([-0.65530812, -0.0015    ,  0.        ]), 0.5, -1...
FAILED tests/test_wrappers.py::test_markovian_state_space_info_dict[CarFlagHard] - ValueError: space Box([-1.1  -0.07 -1.  ], [1.1  0.07 1.  ], (3,), float32) does not contain data (array([0.25203142, 0.        , 0.        ]), 0.5, -1.0,...
FAILED tests/test_wrappers.py::test_state_space_full_and_partial[CarFlagEasy] - ValueError: space Box([-1.1  -0.07 -1.  ], [1.1  0.07 1.  ], (3,), float32) does not contain data (array([0.50059146, 0.        , 0.        ], dtype=float...
FAILED tests/test_wrappers.py::test_state_space_full_and_partial[CarFlagMedium] - ValueError: space Box([-1.1  -0.07 -1.  ], [1.1  0.07 1.  ], (3,), float32) does not contain data (array([-1.0373735,  0.       ,  0.       ], dtype=float...
FAILED tests/test_wrappers.py::test_state_space_full_and_partial[CarFlagHard] - ValueError: space Box([-1.1  -0.07 -1.  ], [1.1  0.07 1.  ], (3,), float32) does not contain data (array([0.86538035, 0.        , 0.        ], dtype=float...
=================================================== 12 failed, 441 passed, 6 skipped, 417 warnings in 5.21s ===================================================

I tried to np.clip the values as well yet I dont see how the values are out of bound. Could you please help?

Ah I think this might be related to our Markovian state space requirement. So with POMDPs, it is useful to have the state available for debugging. I think you might need to add the state_space member variable to your class. See here for an example. You should also implement a get_state method that returns a Markov state (see here).
For heaven/hell, I think the state is probably just the agent position/velocity and the positions of hell and heaven.

Thank you for the advice. I’ll look into the dimensions of get_state.

Changing the get_state to return just the observation (of size 3) also gives the same error.
 def get_state(self):
        # Return the position of the car, oracle, and goal
        return self.state

yay it passes the tests now!
@smorad could you review the code quality?
lastly, should I add new tests or would the existing ones suffice?

Great work! Your code is good. I think I have just one more recommendation:
To be in-line with the other popgym environments, I think it would be best if the episodic reward (non discounted return) was always between -1.0, 1.0. Maybe you could rescale the reward to be
max_reward = self.max_steps - self.heaven_position / self.max_velocity
reward = reward / max_reward

You might also want to add some tests just to ensure that the episode ends after you reach the boundary, and that the episode return is always bounded in [-1, 1].


Typo correction in document

Typo correction in document
netfix -> netflix

change branche p/r
origin : https://github.com/spring-cloud/spring-cloud-commons/pull/692


feature: Headless sync log feedback

Did you check the docs?

[X] I have read all the lazy.nvim docs

Is your feature request related to a problem? Please describe.
Working with lazy.nvim from the command line is possible, as described in the README:
nvim --headless "+Lazy! sync" +qa

The problem is that there is no feedback.
When performing actions from the commandline (maybe automated), I would expect to see output of those actions and check for errors.
Describe the solution you'd like
I would be nice if some kind of log could be printed to stdout.
Describe alternatives you've considered
Right now, lazy.nvim shows its info in a popup, if some logging was made, it could be redirected to stdout.
Additional context
No response

Just don't use headless, or do something like:
nvim +"lua require('lazy').sync({wait=true}); vim.cmd('qa!')"
I won't add logging for now. The ui makes it clear if there's errors. If you don't want to use the UI, then it's up to you


any example about  WMI   class StdRegProv?

I can't figure out how to access the StdRegProvider class to read values from the registry. It is necessary through WMI and not through the remote registry or WinRM. Are there any more examples?

hello @fardeadok, thanks for your interest!
I was trying to make things work as per this manual: https://learn.microsoft.com/en-us/windows/win32/wmisdk/wmi-tasks--registry.
after improving examples/dcom.go, I was able to run certain WMI registry commands:
$ go run examples/wmiexec.go --class StdRegProv  --method EnumValues --args '{"hDefKey":2147483650, "sSubKeyName":"SYSTEM\\CurrentControlSet\\Services\\Tcpip"}'
{
  "ReturnValue": 0,
  "Types": [
    4,
    1,
    1,
    4,
    1,
    2,
    4,
    4,
    4,
    4,
    4,
    4,
    4
  ],
  "sNames": [
    "BootFlags",
    "Description",
    "DisplayName",
    "ErrorControl",
    "Group",
    "ImagePath",
    "Start",
    "Tag",
    "Type",
    "NdisMajorVersion",
    "NdisMinorVersion",
    "DriverMajorVersion",
    "DriverMinorVersion"
  ]
}

$ go run examples/wmiexec.go --class StdRegProv  --method GetStringValue --args '{"hDefKey":2147483650, "sSubKeyName":"SYSTEM\\CurrentControlSet\\Services\\Tcpip", "sValueName": "DisplayName"}'
{
  "ReturnValue": 0,
  "sValue": "@%SystemRoot%\\system32\\drivers\\tcpip.sys,-10001"
}

Thank you. Super!
Is it possible to use that without AD?
SERVER =<IP_ADDRESS>                            // remote comp with WMI
USERNAME=DESKTOPALEX\Alex        // my username on remote comp
PASSWORD=XXXXX                               // my password on remote comp
TARGET=host/DESKTOPALEX              // remote comp  name

when trying to connect to a remote computer (it is not in the AD domain), it gives this error that the KDC cannot find:
"new_object_exporter:  bind: could not bind the selected transport: bind: init security context: security provider: krb5: init: apreq: affirm login: could not get valid TGT for client's realm: [Root cause: Networking_Error] Networking_Error: AS Exchange Error: failed sending AS_REQ to KDC: failed to communicate with KDC. Attempts made with UDP ( dok: DNS lookup KDC... no KDCs defined in configuration for realm DESKTOPALEX) and then TCP ( dok: DNS lookup KDC... no KDCs defined in configuration for realm DESKTOPALEX)"

@fardeadok I guess you need to use NTLM authentication in this case, just comment out following lines:
https://github.com/oiweiwei/go-msrpc/blob/main/examples/wmiexec.go#L83-L84


Adds Python 3.11 support

Also removes EOL 3.x versions of Python

A couple questions:

Python 3.7 EOL's in less than four months. Should we remove that too?
Do we also want to remove all 2.x references?

This work was sponsored by https://metify.io

Thanks for the PR !


Integration with AAD through env variables doesn't work (missing needed parameter state)

Be informative
DD v. 2.33.5 (release mode)
I built a DefectDojo instance on ECS AWS with one container per component.
As I understand things, the UWSGI component is the one responsible for Azure AD integration.
Bug description
The button for connecting through Azure AD doesn't appear, instead the highlighted message appears at the address https://my-defectdojo/complete/azuread-tenant-oauth2/ (I can't give you the actual address as it contains sensitive data, sorry in advance).
I followed the documentation : https://defectdojo.github.io/django-DefectDojo/integrations/social-authentication/#azure-active-directory and have set 4 environment variables for the UWSGI container which he is properly reading (if that is of any help, I am also using this environment system for the database connection and linking to the celery broker, etc..).
Steps to reproduce
Steps to reproduce the behavior:

Set DD_SOCIAL_AUTH_AZUREAD_TENANT_OAUTH2_KEY
DD_SOCIAL_AUTH_AZUREAD_TENANT_OAUTH2_SECRET
DD_SOCIAL_AUTH_AZUREAD_TENANT_OAUTH2_TENANT_ID
DD_SOCIAL_AUTH_AZUREAD_TENANT_OAUTH2_ENABLED as environment variables for UWSGI to their values on your end of Azure (True for ENABLED)
Launch the DefectDojo instance
Go to 'https://my-defectdojo/complete/azuread-tenant-oauth2/'
See error

Deployment method (select with an X)

[ X] Docker

Logs from uwsgi
GET /complete/azuread-tenant-oauth2/ => generated 0 bytes in 84 msecs (HTTP/1.1 302) 9 headers in 473 bytes (1 switches on core 1)
Screenshots

Please let me know if you have any idea of how to fix this issue

Hello, we are expieriencing the same problem but on version 2.35.4 after adding local_settings.py like we should.

@Raphaaaaaugh hey, have you been able to solve this problem?


SPIKE: Investigate GitHub Actions Reporter

Jest 28+ comes with support for a Github Actions Reporter for better GH integration of jest test results. Investigate/implement this new reporter. May require enhancing jest.config.ts to support specialized github config.
See documentation: https://jestjs.io/docs/configuration#github-actions-reporter

Vitest uses the GH reporter automatically (reacts to GITHUB_ACTIONS env var that is set in GH workflows), and playwright will soon use a GH reporter with the upcoming FE PR workflow.


add back generate hooks

Reproduction
npx nuxi generate
Describe the bug
In nuxt3 genrated hooks are never called
We have no control when the generation started and when it ended
https://github.com/nuxt/framework/blob/main/packages/schema/src/types/hooks.ts#L157-L173
Additional context
I need to use the generate:done hook to output a file. For example in the robots module

Also missing this feature. My workaround is to create an npm run generate script which just executes another script after, like nuxt generate && do-more-stuff.sh..

We now have nitro:build:public-assets which is called after prerender is complete and generated assets have been copied to output dir: https://github.com/nuxt/nuxt/pull/19638.

@danielroe generate:done would give you an array of errors if there was errors that were generated during the pre-render, this hook doesnt appear to contain that info or am i not seeing something? I need to be able to determine if the nuxt application was able to generate things successfully before executing certain code, what is the equivalent for this hook?

You can use the nitro hook prerender:route which will be called with the result of each route generation, including any error.

You can use the nitro hook prerender:route which will be called with the result of each route generation, including any error.

@danielroe
Im not sure this contains errors, is this the hook you are referring to:

https://nuxt.com/docs/api/advanced/hooks#nuxt-hooks-build-time
If so the ctx object doesnt contain any errors from what i can tell:
Its just a an object with a set with the routes:
{ routes: Set(4) { '/', '/test', '/200.html', '/404.html' } }
Am i missing something?

export default defineNuxtConfig({
  nitro: {
    hooks: {
      'prerender:generate' (route) {
        console.log(route.error)
      }
    }
  }
})

@danielroe Perfect.
Apologies, i misunderstood which the type of configuration hook. 👍🏽

generate:done also allowed you access to the builder, which in turn gave you access to the dist directory after it was generated. I'm still looking for a workaround for this, since I need to be able to modify .dist after it's generated. It doesn't look like nitro:build:public-assets is the same.

nitro:build:public-assets should give you access to the dist directory and allow modifying files - it's exactly for that purpose. But you will need to modify them within the nitro output directory rather than the dist directory as they have already been copied.

nitro:build:public-assets should give you access to the dist directory and allow modifying files - it's exactly for that purpose. But you will need to modify them within the nitro output directory rather than the .nuxt/dist directory as they have already been copied.

Thank you @danielroe, this is exactly what I needed, just didn't realize it right away! For a solid day or two I thought this concerned the ./public directory instead of ./output/public 🤦♂


bazel test failure fix for arm64 and code optimize

fix expected memory size fault for ARM64
optimising arch detect logic
Release note:
NONE

@rmohr

@zhlhahaha if you put a NONE into the release-note field instead of just deleting it, then you get rid of the release-note-label-needed label.

Hi @rmohr, I am rewriting this PR now. I have some questions about template_test.go and look forward your suggestion.
Can we have a function like UpdateFakeClusterConfigWithCPUArch to update cpuArch for clusterconfig? Otherwise, if use NewFakeClusterConfigWithCPUArch to set the cpuArch, we can not use BeforeEach and AfterEach in the tests. And there are lots of pre-setting code in BeforeEach, we have to write it in every test.

Can we have a function like UpdateFakeClusterConfigWithCPUArch to update cpuArch for clusterconfig? Otherwise, if use NewFakeClusterConfigWithCPUArch to set the cpuArch, we can not use BeforeEach and AfterEach in the tests. And there are lots of pre-setting code in BeforeEach, we have to write it in every test.

I did not look too close, but how about something like this?
type configFactory func(arch string) (*virtconfig.ClusterConfig, cache.SharedIndexInformer)

func BeforeEach() {
	configFactory = func(arch string) (*virtconfig.ClusterConfig, cache.SharedIndexInformer) {
	       // add additional setting here which you need
		config, _, _, kvInformer := testutils.NewFakeClusterConfigUsingKV(kv, arch)
		return config, kvInformer
	}
}

It("something", func() {
   config := configFactory("myarch")
})

table.Describe("something", func(arch string){
   config := configFactory("myarch")
})

Basically delaying the actual config creation til the test, but still providing all config details in the BeforeEach.

/retest

/retest

/retest

/retest

/retest

@zhlhahaha sorry I missed the latest update on the PR. Could you rebase so that I can approve?

@zhlhahaha sorry I missed the latest update on the PR. Could you rebase so that I can approve?

Ok, np.

/retest

/lgtm
/approve
Thanks!