[SPARK-31553][SQL] Fix isInCollection for collection sizes above the optimisation threshold

What changes were proposed in this pull request?
The InSet expression expects input collections of internal Catalyst types, for example hset must contain elements of UTF8String for child of string type. So, it means isInCollection must convert users values to internal Catalyst values but currently it doesn't perform the conversion. That leads to incorrect results for collection sizes above the threshold spark.sql.optimizer.inSetConversionThreshold.
Why are the changes needed?
The changes fix incorrect behaviour of isInCollection. For example, if the SQL config spark.sql.optimizer.inSetConversionThresholdis set to 10 (by default):
val set = (0 to 20).map(_.toString).toSet
val data = Seq("1").toDF("x")
data.select($"x".isInCollection(set).as("isInCollection")).show()

The function must return 'true' because "1" is in the set of "0" ... "20" but it returns "false":
+--------------+
|isInCollection|
+--------------+
|         false|
+--------------+

Does this PR introduce any user-facing change?
Yes
How was this patch tested?

By existing test suite ColumnExpressionSuite
Add new test to ColumnExpressionSuite
Manually benchmark it by the code from #25754:

def testExplainTime(collectionSize: Int) = {
        val df = spark.range(10).withColumn("id2", col("id") + 1)
        val list = Range(0, collectionSize).toList
        val startTime = System.currentTimeMillis()
        df.where(col("id").isInCollection(list)).where(col("id2").isInCollection(list)).explain()
        val elapsedTime = System.currentTimeMillis() - startTime
        println(s"cost time: ${elapsedTime}ms")
}

Then test on collection size 5, 10, 100, 1000, 10000, test result is:

collection size
explain time (before)
explain time (after)

5
64ms
65ms

10
68ms
64ms

100
41ms
162ms

1000
98ms
406ms

10000
654ms
2579ms

@cloud-fan @HyukjinKwon Please, review the PR.

Test build #121756 has started for PR 28328 at commit 47a1e44.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26434/
Test PASSed.

Test build #121756 has finished for PR 28328 at commit 47a1e44.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121756/
Test FAILed.

The build https://github.com/apache/spark/pull/28328#issuecomment-618992335 fails on
org.apache.spark.sql.hive.thriftserver.CliSuite.SPARK-11188 Analysis error reporting
I don't think it is related to the changes

jenkins, retest this, please

Test build #121762 has started for PR 28328 at commit 47a1e44.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26439/
Test PASSed.

This is a nice catch, @MaxGekk . As I wrote in the comment, it would be great if we can proceed two PR separately. Thanks!

Test build #121762 has finished for PR 28328 at commit 47a1e44.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121762/
Test PASSed.

I merged https://github.com/apache/spark/pull/28343 first. Could you rebase this PR, @MaxGekk ? Thanks.

@dongjoon-hyun I have rebased but the test from this PR starts failing. The function convertToScala(elem, child.dataType) doesn't convert UTF8String to String because child.dataType is NullType when InSet is created from isInCollection.
The test passed here because I don't wrap the result of convertToScala by Literal.
Probably, we need to revert https://github.com/apache/spark/commit/7d8216a6642f40af0d1b623129b1d5f4c86bec68 . Sorry, my bad.

@dongjoon-hyun @cloud-fan Regarding to this PR, WDYT of reverting the optimization https://github.com/apache/spark/pull/25754 instead of to fix it by this PR?

It's possible. I guess that we need @gatorsmile 's opinion since he was the merger of that.

I have fixed the test failure after rebasing on https://github.com/apache/spark/pull/28343 by passing element type from the place where the type is known https://github.com/apache/spark/pull/28328/commits/67f34a10a6bafe3a7da9480fc3f79cd041cb12b0 . I could open a follow-up PR for #28343 @dongjoon-hyun Let me know if you are ok with that.

Test build #121815 has started for PR 28328 at commit 67f34a1.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26490/
Test PASSed.

Test build #121815 has finished for PR 28328 at commit 67f34a1.

This patch fails to build.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class InSet(

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121815/
Test FAILed.

Test build #121818 has started for PR 28328 at commit dd69aa6.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26492/
Test PASSed.

Test build #121818 has finished for PR 28328 at commit dd69aa6.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121818/
Test FAILed.

retest this please

Test build #121826 has started for PR 28328 at commit dd69aa6.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26500/
Test PASSed.

Test build #121826 has finished for PR 28328 at commit dd69aa6.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121826/
Test FAILed.

jenkins, retest this, please

Test build #121835 has started for PR 28328 at commit dd69aa6.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26509/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26514/
Test PASSed.

Test build #121840 has started for PR 28328 at commit 05ce50a.

Test build #121840 has finished for PR 28328 at commit 05ce50a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121840/
Test FAILed.

Failures of CliSuite annoy a lot https://github.com/apache/spark/pull/28328#issuecomment-619520623 @dongjoon-hyun @cloud-fan @gatorsmile This is the PR to repeat tests from CliSuite https://github.com/apache/spark/pull/28329

jenkins, retest this, please

Test build #121844 has started for PR 28328 at commit 05ce50a.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26518/
Test PASSed.

Test build #121835 has finished for PR 28328 at commit dd69aa6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121835/
Test PASSed.

Test build #121844 has finished for PR 28328 at commit 05ce50a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121844/
Test PASSed.

cc @viirya , this is another instance that merging InSet and Set can fix the issue.

Actually this PR shows we still need InSet, to make the analyzer fast...

Actually this PR shows we still need InSet, to make the analyzer fast...

What that means? We optimize In with InSet in optimizer, right?

A In with many values is slow to analyze, as the type coercion rules or In.resolved are very slow.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26570/
Test PASSed.

Test build #121899 has started for PR 28328 at commit 4bc0e26.

I have update PR's description and added a column w/o optimization. I got the numbers by running the code:
  test("isInCollection benchmark") {
    def testExplainTime(collectionSize: Int) = {
      val df = spark.range(10).withColumn("id2", col("id") + 1)
      val list = Range(0, collectionSize).toList
      val startTime = System.currentTimeMillis()
      df.where(col("id").isInCollection(list)).where(col("id2").isInCollection(list)).explain()
      val elapsedTime = System.currentTimeMillis() - startTime
      println(s"cost time: ${elapsedTime}ms")
    }
    withSQLConf(SQLConf.OPTIMIZER_INSET_CONVERSION_THRESHOLD.key -> "100000000") {
      testExplainTime(1)
      testExplainTime(5)
      testExplainTime(10)
      testExplainTime(100)
      testExplainTime(1000)
      testExplainTime(10000)
    }
  }

Test build #121899 has finished for PR 28328 at commit 4bc0e26.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121899/
Test PASSed.

A In with many values is slow to analyze, as the type coercion rules or In.resolved are very slow.

That's a pain point. But when we merge In and InSet, we can have a constructor similar to current InSet? We can have both list and hset in In.  For example, in isInCollection, we still can construct In which has empty children but only hset?

Okay, the more I look, the more it makes me to think we should revert #25754 rather than adding bandaid fixes. Shall we revert?

After offline discussion with @gatorsmile @cloud-fan @HyukjinKwon, we decided to revert https://github.com/apache/spark/pull/25754 . I will open a PR for that and close this PR.


Add hazelcast store to readme

I've added my store for hazelcast (https://github.com/marudor/node-cache-manager-hazelcast) to the README.

Coverage remained the same at 99.259% when pulling b52d93b31b4a504f4e431c7d486e0945506bfa4a on marudor:master into 79572f9974c2319da3d1972da5d5a876577cbfc7 on BryanDonovan:master.

Thanks!


Unable to select a Device

Hey! :-)
I am trying to connect my Node-Red-Flow with the Echo-Dot following your installation guide and documentation on your product website. I am currently not able to select a previously added device in the Alexa-Home-Node (see attached screenshot).
Still refreshing, reboot and re-installation doesn't work for me.
Is there an issue with the way I tried to do so or may there be any different issue?
Thanks in advance for your help.
Regards. Hendrik.

First guess is you've got the password wrong when you added the account to Node-RED.
Is there anything in the Node-RED logs, especially when you hit the refresh button?

Thx for your fast reply offering your help. It was caused by an old Nodered-Version. So I upgraded and now works fine!

Thx for your fast reply offering your help. It was caused by an old Nodered-Version. So I upgraded and now works fine!


Issue_4

The report for this issue is in "Security Bug" Folder with txt file name Issue4.txt which contains doc file link of this report.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Refactor Ghostscript command line args building

Remove unnecessary embed fonts step, simply embed the fonts in the conversion. Tried this with a webfont from a :url and it still works. Not sure what made me initially use a separate step for this. Cargo cult programming.
Instead moved the relevant arguments to a new list of default pdfwrite arguments. Some of them are the default, anyway (like dEmbedAllFonts).
Refactored lowest-level GhostscriptImpl wrapper

Drop interface and & Impl suffix, align with ChromeRunner
Takes care of command line arguments only
Replace convert_to_pdfa function with pdfwrite function that takes a list of input files and optionally applies the PDF/A
Remove join function and use pdfwrite instead
Dropped conditional handling of safer mode after GS 1.50 for unconditional dSAFER flag

Made sure we apply escaping to paths everywhere

@ammancilla once this is merged, we could release another 1.5.1 or 1.6.0 or so and then start breaking API for a 2.0

Looks great


Not working with 107.5+ ???

Been fighting this for hours.  Any ideas?  I had this working long time ago and removed it.  As of recent I had a use for it and couldn't get it working :(  Adding the resource as a JS Module.  Tried copying raw code as well but no luck.
[frontend.js.latest.202003181] https://domain.duckdns.org/local/custom-lovelace/entity-attributes-card.js:7:1 Uncaught SyntaxError: Unexpected token '<'

had to append ?=v2 to the location of the component.


chore: change some errors to warns/infos

If the error is retryable or expected to happen then we shouldn't log it as an error. Better use warn/info.

Successfully created backport PR for releases/v0.2:

#3886

Git push to origin failed for releases/v0.2 with exitcode 1


Breaking change found related to streams from slbeta2 -> slbeta3

Description:
Most of the connector daily builds are failing in slbeta3 SNAPSHOT
There is a compilation error related to the usage of streams.
These connectors were working fine in slbeta2
Sample Error logs:
ERROR [endpoints.bal:(56:43,56:57)] incompatible types: expected '(record {| ballerinax/googleapis.calendar:0.1.6-SNAPSHOT:Calendar value; |}|error)', found '(record {| ballerinax/googleapis.calendar:0.1.6-SNAPSHOT:Calendar value; |}|error)?'
ERROR [endpoints.bal:(178:40,178:51)] incompatible types: expected '(record {| ballerinax/googleapis.calendar:0.1.6-SNAPSHOT:Event value; |}|error)', found '(record {| ballerinax/googleapis.calendar:0.1.6-SNAPSHOT:Event value; |}|error)?'
error: compilation contains errors

Sample link to failing daily build workflow:
https://github.com/ballerina-platform/module-ballerinax-googleapis.calendar/runs/3226806087?check_suite_focus=true
Reason for this breaking change:
The fixes done to solve https://github.com/ballerina-platform/ballerina-lang/issues/31464 might have created this breaking change in slbeta3
Affected Versions:
Ballerina slbeta3-SNAPSHOT
OS, DB, other environment details and versions:
JDK 11
Ubuntu

@anupama-pathirage Should we do the changes in connector side? or can we get it fixed from ballerina side?
It seems like if we have do the changes in connector side, we have to change the return type of the remote functions.
Example:
returns stream<Calendar,error>|error to returns stream<Calendar,error?>|error
If we do a method signature change in connector, this can break existing choreo-samples and custom integrations built by users.
Or else, is there any other way to fix this without changing the connector remote function signutures?
Please share your thoughts.

@hasithaa / @gimantha Please check on this and can you share more thoughts on this please. Did we introduce any breaking change?

Yes, we had to introduce https://github.com/ballerina-platform/ballerina-lang/pull/31674 breaking change.
With beta2 there was a bug where we had considered, stream<T, E> as equivalent to stream<T, E?>. We had handled the nil completion type internally. Due to this, we encountered crashes at runtime eg: https://github.com/ballerina-platform/ballerina-lang/issues/31671
Even though we warned this change is coming before beta2, seems we have missed this change in some modules as the compiler restriction was not there.
https://github.com/ballerina-platform/ballerina-lang/pull/31674

Closing since this change is done all all the connectors


(PUP-3742) Smoke test gems

Creates a rake task ci:test:gem and an acceptance test for installing
puppet as a gem and smoke testing the install against a predefined set
of hosts windows (ruby x86 and x64), osx and redhat. The osx case is
needed because we have a darwin-specific facter gem that has a
dependency on CFPropertyList.
The ci:test:gem task relies on the existing git setup step that
installs git and ruby on each host, even though we really only need
ruby.
Also on Windows, we're using the our vendored ruby, which already
includes some gems, e.g. win32-service, ffi. So the test setup step
wipes the current set of installed gems. Alternatively, it would be nice
to only install our vendored ruby, but we do not current publish that as
a build artifact.

$ env SHA=4.2.2 bundle exec rake ci:test:gem
...
Begin setup/gem/pre-suite/010_GemInstall.rb

Install puppet gem

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) 14:32:58$ curl -s -o build_data.yaml http://builds.puppetlabs.lan/puppet/4.2.2/artifacts/4.2.2.yaml

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) executed in 0.12 seconds

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) 14:32:58$ env PATH="/usr/bin:${PATH}" ruby -ryaml -e 'puts YAML.load_file("build_data.yaml")[:gemversion]'
4.2.2

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) executed in 0.22 seconds

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) 14:32:58$ env PATH="/usr/bin:${PATH}" ruby -rrbconfig -e 'puts Gem.default_dir'
C:/cygwin64/lib/ruby/gems/2.1.0

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) executed in 0.16 seconds

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) 14:32:58$ rm -rf 'C:/cygwin64/lib/ruby/gems/2.1.0'

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) executed in 0.34 seconds

  * Download puppet gem from http://builds.puppetlabs.lan/puppet/4.2.2/artifacts/puppet-4.2.2-x86-mingw32.gem

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) 14:32:58$ curl -s -o puppet.gem http://builds.puppetlabs.lan/puppet/4.2.2/artifacts/puppet-4.2.2-x86-mingw32.gem

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) executed in 0.33 seconds

  * Install puppet.gem

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) 14:32:59$ env PATH="/usr/bin:${PATH}" cmd /c gem install puppet.gem
unable to convert "\xF1" from ASCII-8BIT to UTF-8 for extra/Console.rdoc, skipping
unable to convert "\xF1" from ASCII-8BIT to UTF-8 for extra/Console_ANSI.rdoc, skipping
Successfully installed win32console-1.3.2-x86-mingw32
Successfully installed ffi-1.9.10-x86-mingw32
Successfully installed win32-security-0.2.5
Successfully installed win32-dir-0.4.9
Successfully installed facter-2.4.4-x86-mingw32
Successfully installed json_pure-1.8.2
Successfully installed hiera-3.0.1-x86-mingw32
Successfully installed win32-eventlog-0.6.3
Successfully installed win32-process-0.7.5
Successfully installed win32-service-0.8.7
Successfully installed minitar-0.5.4
Successfully installed puppet-4.2.2-x86-mingw32
Parsing documentation for facter-2.4.4-x86-mingw32
Installing ri documentation for facter-2.4.4-x86-mingw32
Parsing documentation for ffi-1.9.10-x86-mingw32
Installing ri documentation for ffi-1.9.10-x86-mingw32
Parsing documentation for hiera-3.0.1-x86-mingw32
Installing ri documentation for hiera-3.0.1-x86-mingw32
Parsing documentation for json_pure-1.8.2
Installing ri documentation for json_pure-1.8.2
Parsing documentation for minitar-0.5.4
Installing ri documentation for minitar-0.5.4
Parsing documentation for puppet-4.2.2-x86-mingw32
Installing ri documentation for puppet-4.2.2-x86-mingw32
Parsing documentation for win32-dir-0.4.9
Installing ri documentation for win32-dir-0.4.9
Parsing documentation for win32-eventlog-0.6.3
Installing ri documentation for win32-eventlog-0.6.3
Parsing documentation for win32-process-0.7.5
Installing ri documentation for win32-process-0.7.5
Parsing documentation for win32-security-0.2.5
Installing ri documentation for win32-security-0.2.5
Parsing documentation for win32-service-0.8.7
Installing ri documentation for win32-service-0.8.7
Parsing documentation for win32console-1.3.2-x86-mingw32
Installing ri documentation for win32console-1.3.2-x86-mingw32
Done installing documentation for facter, ffi, hiera, json_pure, minitar, puppet, win32-dir, win32-eventlog, win32-process, win32-security, win32-service, win32console after 66 seconds
12 gems installed

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) executed in 100.25 seconds

  * Verify it's sane

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) 14:34:39$ cmd.exe /c puppet --version
4.2.2

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) executed in 1.84 seconds

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) 14:34:41$ cmd.exe /c puppet apply -e "notify { 'hello': }"
Notice: Compiled catalog for qg4vc34e6ld86wm.delivery.puppetlabs.net in environment production in 0.47 seconds
Notice: hello
Notice: /Stage[main]/Main/Notify[hello]/message: defined 'message' as 'hello'
Notice: Applied catalog in 0.03 seconds

qg4vc34e6ld86wm.delivery.puppetlabs.net (win-2012r2-rubyx86) executed in 16.12 seconds

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) 14:34:57$ curl -s -o build_data.yaml http://builds.puppetlabs.lan/puppet/4.2.2/artifacts/4.2.2.yaml

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) executed in 0.28 seconds

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) 14:34:57$ env PATH="/usr/bin:${PATH}" ruby -ryaml -e 'puts YAML.load_file("build_data.yaml")[:gemversion]'
4.2.2

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) executed in 0.25 seconds

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) 14:34:57$ env PATH="/usr/bin:${PATH}" ruby -rrbconfig -e 'puts Gem.default_dir'
C:/cygwin64/lib/ruby/gems/2.1.0

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) executed in 0.14 seconds

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) 14:34:58$ rm -rf 'C:/cygwin64/lib/ruby/gems/2.1.0'

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) executed in 0.30 seconds

  * Download puppet gem from http://builds.puppetlabs.lan/puppet/4.2.2/artifacts/puppet-4.2.2-x64-mingw32.gem

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) 14:34:58$ curl -s -o puppet.gem http://builds.puppetlabs.lan/puppet/4.2.2/artifacts/puppet-4.2.2-x64-mingw32.gem

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) executed in 0.22 seconds

  * Install puppet.gem

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) 14:34:58$ env PATH="/usr/bin:${PATH}" cmd /c gem install puppet.gem
Successfully installed ffi-1.9.10-x64-mingw32
Successfully installed win32-security-0.2.5
Successfully installed win32-dir-0.4.9
Successfully installed facter-2.4.4-x64-mingw32
Successfully installed json_pure-1.8.2
Successfully installed hiera-3.0.1-x64-mingw32
Successfully installed win32-eventlog-0.6.3
Successfully installed win32-process-0.7.5
Successfully installed win32-service-0.8.7
Successfully installed minitar-0.5.4
Successfully installed puppet-4.2.2-x64-mingw32
Parsing documentation for facter-2.4.4-x64-mingw32
Installing ri documentation for facter-2.4.4-x64-mingw32
Parsing documentation for ffi-1.9.10-x64-mingw32
Installing ri documentation for ffi-1.9.10-x64-mingw32
Parsing documentation for hiera-3.0.1-x64-mingw32
Installing ri documentation for hiera-3.0.1-x64-mingw32
Parsing documentation for json_pure-1.8.2
Installing ri documentation for json_pure-1.8.2
Parsing documentation for minitar-0.5.4
Installing ri documentation for minitar-0.5.4
Parsing documentation for puppet-4.2.2-x64-mingw32
Installing ri documentation for puppet-4.2.2-x64-mingw32
Parsing documentation for win32-dir-0.4.9
Installing ri documentation for win32-dir-0.4.9
Parsing documentation for win32-eventlog-0.6.3
Installing ri documentation for win32-eventlog-0.6.3
Parsing documentation for win32-process-0.7.5
Installing ri documentation for win32-process-0.7.5
Parsing documentation for win32-security-0.2.5
Installing ri documentation for win32-security-0.2.5
Parsing documentation for win32-service-0.8.7
Installing ri documentation for win32-service-0.8.7
Done installing documentation for facter, ffi, hiera, json_pure, minitar, puppet, win32-dir, win32-eventlog, win32-process, win32-security, win32-service after 67 seconds
11 gems installed

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) executed in 93.33 seconds

  * Verify it's sane

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) 14:36:31$ cmd.exe /c puppet --version
4.2.2

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) executed in 1.17 seconds

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) 14:36:33$ cmd.exe /c puppet apply -e "notify { 'hello': }"
Notice: Compiled catalog for mz2hy2oanoymy47.delivery.puppetlabs.net in environment production in 0.50 seconds
Notice: hello
Notice: /Stage[main]/Main/Notify[hello]/message: defined 'message' as 'hello'
Notice: Applied catalog in 0.03 seconds

mz2hy2oanoymy47.delivery.puppetlabs.net (win-2012r2-rubyx64) executed in 15.39 seconds

r8m3y3ocg1whuuy.delivery.puppetlabs.net (osx-1010) 14:36:48$ curl -s -o build_data.yaml http://builds.puppetlabs.lan/puppet/4.2.2/artifacts/4.2.2.yaml

r8m3y3ocg1whuuy.delivery.puppetlabs.net (osx-1010) executed in 0.03 seconds

r8m3y3ocg1whuuy.delivery.puppetlabs.net (osx-1010) 14:36:48$ env PATH=":${PATH}" ruby -ryaml -e 'puts YAML.load_file("build_data.yaml")[:gemversion]'
4.2.2

r8m3y3ocg1whuuy.delivery.puppetlabs.net (osx-1010) executed in 0.09 seconds

  * Download puppet gem from http://builds.puppetlabs.lan/puppet/4.2.2/artifacts/puppet-4.2.2.gem

r8m3y3ocg1whuuy.delivery.puppetlabs.net (osx-1010) 14:36:48$ curl -s -o puppet.gem http://builds.puppetlabs.lan/puppet/4.2.2/artifacts/puppet-4.2.2.gem

r8m3y3ocg1whuuy.delivery.puppetlabs.net (osx-1010) executed in 0.17 seconds

  * Install puppet.gem

r8m3y3ocg1whuuy.delivery.puppetlabs.net (osx-1010) 14:36:48$ env PATH=":${PATH}" gem install puppet.gem
unable to convert "\xFF" from ASCII-8BIT to UTF-8 for lib/puppet/vendor/plist/test/assets/example_data.jpg, skipping
unable to convert "\xB5" from ASCII-8BIT to UTF-8 for lib/puppet/vendor/rgen/test/testmodel/ea_testmodel.eap, skipping
Successfully installed facter-2.4.4-universal-darwin
Successfully installed json_pure-1.8.2
Successfully installed hiera-3.0.1
Successfully installed puppet-4.2.2
Parsing documentation for facter-2.4.4-universal-darwin
Installing ri documentation for facter-2.4.4-universal-darwin
Parsing documentation for json_pure-1.8.2
Installing ri documentation for json_pure-1.8.2
Parsing documentation for hiera-3.0.1
Installing ri documentation for hiera-3.0.1
Parsing documentation for puppet-4.2.2
Installing ri documentation for puppet-4.2.2
4 gems installed

r8m3y3ocg1whuuy.delivery.puppetlabs.net (osx-1010) executed in 73.63 seconds

  * Verify it's sane

r8m3y3ocg1whuuy.delivery.puppetlabs.net (osx-1010) 14:38:02$ puppet --version
4.2.2

r8m3y3ocg1whuuy.delivery.puppetlabs.net (osx-1010) executed in 0.54 seconds

r8m3y3ocg1whuuy.delivery.puppetlabs.net (osx-1010) 14:38:02$ puppet apply -e "notify { 'hello': }"
Notice: Compiled catalog for r8m3y3ocg1whuuy.delivery.puppetlabs.net in environment production in 0.42 seconds
Notice: hello
Notice: /Stage[main]/Main/Notify[hello]/message: defined 'message' as 'hello'
Notice: Applied catalog in 0.03 seconds

r8m3y3ocg1whuuy.delivery.puppetlabs.net (osx-1010) executed in 3.43 seconds

o32veldr4wmbv47.delivery.puppetlabs.net (redhat-7) 14:38:06$ curl -s -o build_data.yaml http://builds.puppetlabs.lan/puppet/4.2.2/artifacts/4.2.2.yaml

o32veldr4wmbv47.delivery.puppetlabs.net (redhat-7) executed in 0.18 seconds

o32veldr4wmbv47.delivery.puppetlabs.net (redhat-7) 14:38:06$ env PATH="/usr/bin:${PATH}" ruby -ryaml -e 'puts YAML.load_file("build_data.yaml")[:gemversion]'
4.2.2

o32veldr4wmbv47.delivery.puppetlabs.net (redhat-7) executed in 0.06 seconds

  * Download puppet gem from http://builds.puppetlabs.lan/puppet/4.2.2/artifacts/puppet-4.2.2.gem

o32veldr4wmbv47.delivery.puppetlabs.net (redhat-7) 14:38:06$ curl -s -o puppet.gem http://builds.puppetlabs.lan/puppet/4.2.2/artifacts/puppet-4.2.2.gem

o32veldr4wmbv47.delivery.puppetlabs.net (redhat-7) executed in 0.34 seconds

  * Install puppet.gem

o32veldr4wmbv47.delivery.puppetlabs.net (redhat-7) 14:38:07$ env PATH="/usr/bin:${PATH}" gem install puppet.gem
unable to convert "\xFF" from ASCII-8BIT to UTF-8 for lib/puppet/vendor/plist/test/assets/example_data.jpg, skipping
unable to convert "\xB5" from ASCII-8BIT to UTF-8 for lib/puppet/vendor/rgen/test/testmodel/ea_testmodel.eap, skipping
Successfully installed facter-2.4.4
Successfully installed json_pure-1.8.2
Successfully installed hiera-3.0.1
Successfully installed puppet-4.2.2
Parsing documentation for facter-2.4.4
Installing ri documentation for facter-2.4.4
Parsing documentation for json_pure-1.8.2
Installing ri documentation for json_pure-1.8.2
Parsing documentation for hiera-3.0.1
Installing ri documentation for hiera-3.0.1
Parsing documentation for puppet-4.2.2
Installing ri documentation for puppet-4.2.2
4 gems installed

o32veldr4wmbv47.delivery.puppetlabs.net (redhat-7) executed in 67.20 seconds

  * Verify it's sane

o32veldr4wmbv47.delivery.puppetlabs.net (redhat-7) 14:39:14$ puppet --version
4.2.2

o32veldr4wmbv47.delivery.puppetlabs.net (redhat-7) executed in 0.53 seconds

o32veldr4wmbv47.delivery.puppetlabs.net (redhat-7) 14:39:14$ puppet apply -e "notify { 'hello': }"
Notice: Compiled catalog for o32veldr4wmbv47.delivery.puppetlabs.net in environment production in 0.45 seconds
Notice: hello
Notice: /Stage[main]/Main/Notify[hello]/message: defined 'message' as 'hello'
Notice: Applied catalog in 0.02 seconds

o32veldr4wmbv47.delivery.puppetlabs.net (redhat-7) executed in 2.27 seconds
setup/gem/pre-suite/010_GemInstall.rb passed in 378.92 seconds
      Test Suite: tests @ 2015-09-10 14:32:58 -0700

      - Host Configuration Summary -

              - Test Case Summary for suite 'tests' -
       Total Suite Time: 378.92 seconds
      Average Test Time: 378.92 seconds
              Attempted: 1
                 Passed: 1
                 Failed: 0
                Errored: 0
                Skipped: 0
                Pending: 0
                  Total: 1

      - Specific Test Case Status -

Failed Tests Cases:
Errored Tests Cases:
Skipped Tests Cases:
Pending Tests Cases:

No tests to run for suite 'post_suite'
Cleanup: cleaning up after successful run
Warning: ssh connection to win-2012r2-rubyx86 has been terminated
Warning: ssh connection to win-2012r2-rubyx64 has been terminated
Warning: ssh connection to osx-1010 has been terminated
Warning: ssh connection to redhat-7 has been terminated
Handing 'qg4vc34e6ld86wm.delivery.puppetlabs.net' back to vmpooler for VM destruction
Handing 'mz2hy2oanoymy47.delivery.puppetlabs.net' back to vmpooler for VM destruction
Handing 'r8m3y3ocg1whuuy.delivery.puppetlabs.net' back to vmpooler for VM destruction
Handing 'o32veldr4wmbv47.delivery.puppetlabs.net' back to vmpooler for VM destruction

Checks haven't completed due to [skip ci]

Otherwise :+1:

@kylog mentioned it'd be great to get this in 3.x. I'll update with Michael's comments, and if it works in 3.x, retarget there

I tried backporting to 3.x, and first we're missing commit 2e5b5e9b6bdf884adae970e4382ea9d0827e213e, which added the ruby/gem helper command methods. But then when running against 3.8.2, the puppet gem doesn't create the necessary directories until you do an agent run first:
cmd.exe /c puppet apply  -e "notify { 'hello': }"
Last 10 lines of output were:
	Warning: /File[C:/ProgramData/PuppetLabs/puppet/var/client_yaml]: Skipping because of failed dependencies
	Warning: /File[C:/ProgramData/PuppetLabs/puppet/var/clientbucket]: Skipping because of failed dependencies
	Warning: /File[C:/ProgramData/PuppetLabs/puppet/var/facts.d]: Skipping because of failed dependencies
	Warning: /File[C:/ProgramData/PuppetLabs/puppet/etc/ssl]: Skipping because of failed dependencies
	Warning: /File[C:/ProgramData/PuppetLabs/puppet/etc/ssl/certificate_requests]: Skipping because of failed dependencies
	Warning: /File[C:/ProgramData/PuppetLabs/puppet/etc/ssl/private]: Skipping because of failed dependencies
	Warning: /File[C:/ProgramData/PuppetLabs/puppet/etc/ssl/certs]: Skipping because of failed dependencies
	Warning: /File[C:/ProgramData/PuppetLabs/puppet/etc/ssl/public_keys]: Skipping because of failed dependencies
	Warning: /File[C:/ProgramData/PuppetLabs/puppet/etc/ssl/private_keys]: Skipping because of failed dependencies
	Error: Got 2 failure(s) while initializing: File[C:/ProgramData/PuppetLabs/puppet/etc]: change from absent to directory failed: Cannot create C:/ProgramData/PuppetLabs/puppet/etc; parent directory C:/ProgramData/PuppetLabs/puppet does not exist; File[C:/ProgramData/PuppetLabs/puppet/var]: change from absent to directory failed: Cannot create C:/ProgramData/PuppetLabs/puppet/var; parent directory C:/ProgramData/PuppetLabs/puppet does not exist>
`

We don't have this problem in stable, because puppet was modified to create ancestor directories. So I'm inclined to leave this at stable.

Added a check that puppet apply outputs the notify message.

Yeah @joshcooper looks like it's more trouble than it's worth to make it work on 3.x at this point.  Oh well - valiant effort!

+1 on leaving it at stable.

Verified with env SHA=4.2.2 bundle exec rake ci:test:gem

Merged up to master in https://github.com/puppetlabs/puppet/commit/af5a0261352ffe569547d4087f8528b36c6c30c1


Support gym without configure

This works with gym master (which has a small fix from gym 0.8.0)

Pushed a tweak to tox dependencies, that will hopefully fix travis


Added link to .redscr

.redhbl maybe? I'd like to keep it short and naming it that opens up creating .errhbl as well as a sort of sister command.


GUI

Is there a GUI implementation of this software already, or do you know of any plan to do so in future?

I'm aware that various GUIs have been developed (Java, QGIS plugins, C#) for internal/project-specific purposes across Australia, but none are available for release. I started a C++ Qt based project several years ago, using OpenGL for the graphics, but didn't get very far. I understand a Web service to provide a simple SAAS solution is under development, but I'm not sure where this is at. I am not aware of any plans to develop a GUI for release on GitHub.  If you are interested in developing a GUI, I would be very glad to discuss this and to gauge user need.

Dear Roger:
For the GUI of DynAdjust, "C++ Qt based using OpenGL" is really good choice.
If no 3D graphics involved, using "TypeScript plus Electron framework" to develop a Cross-Platform Desktop app may also is  a good choice for your consideration.
https://www.electronjs.org/docs
https://itnext.io/develop-a-cross-platform-desktop-application-with-electron-forge-react-webpack-typescript-ac2c7452b71f
https://www.sitepoint.com/build-a-desktop-application-with-electron-and-angular/


Orange Pi 5 Max Performance improvement

Is your feature request related to a problem? Please describe
The main issue is that at first it was running perfectly, with of course a low fps (23 to 25 fps). The thing is, the Orange Pi 5 processor is double the performance of the RPi5, and I know of people running this fine in their RPi5. Perhaps it requires a little bit of coding or research, but no one has any info on the net currently.
Whenever I was playing with my cousin, the system tended to lag a little, but it wasn't much. After a while playing and starting our bases, a lot of lag started happening, and traspasing floors or walls wasn't uncommon, which is frustrating, and I believe it might have something to do with the emulation or some kind of setting I put.
Describe the solution you'd like
As such, if there is any way that you could add to the ARM64_DEVICE some kind of setting for the rk3588 processor, which includes multiple SOC, it would be great; if it cannot be done, no problem. And if you could make a template for making it run in something like Puferpanel, which would allow me to see in live the issues presented in the Docker, it might also help (tried with pterodactyl and it had many issues with the panel so i was unable to install it).
Describe alternatives you've considered
I have attempted turning off the multithreading and turning it on, also changing the settings for the "BOX64_DYNAREC_STRONGMEM",
"BOX64_DYNAREC_BIGBLOCK", "BOX64_DYNAREC_SAFEFLAGS", "BOX64_DYNAREC_FASTROUND", "BOX64_DYNAREC_FASTNAN",
"BOX64_DYNAREC_X87DOUBLE".
Additional context
env.txt
json-code.txt
The env.text file is the .env file I used after changing the settings a little to see if there was any improvement, and the json-code.txt is a code I made to use as a template in pufferpanel that, unfortunately, didn't get farther than pulling the Docker image.
Feature Report Checklist

[ ] I am willing to implement this feature

I've added ARM64_DEVICE=rk3588 in the arm64 base image. You'll be able to test it as soon as @thijsvanloef updates the image tag. I'm not familiar with Puferpanel (and it's out of the "arm64-exclusive" scope) so I'll leave that to him as well :)

@sonroyaalmerol @DummyTitan You should be able to test ARM64_DEVICE=rk3588 with the thijsvanloef/palworld-server-docker:dev image :)

I reset my Box64 env config just in case and made sure to use the ARM64_DEVICE=rk3588, but did not use multithreading just in case and the lag and glitches are as always, the fps didnt see much improvement. Just in case i will try again with the multithreading on to see if there is any diffrence.

The multithreading helped with the waiting time in the load screen, but the fps became actually a bit more erratic; when I first get in it's at 4 fps, and after a while it goes to 21, 22, and can go down to 18 or less. Just in case this is the Docker compose code I'm using.
services:    palworld:       image: thijsvanloef/palworld-server-docker:dev       restart: unless-stopped       container_name: palworld-server       stop_grace_period: 30s # Set to however long you are willing to wait for the container to gracefully stop       ports:         - 8212:8212/udp         - 27015:27015/udp         # - 8212:8212/tcp  # Port for REST API if REST_API_ENABLED: true       env_file:          -  /srv/dev-disk-by-uuid-044ded6a-869c-4981-ad61-d131a8ab5a5d/Storage_and_Cache/palworld/enviorment/.env       volumes:          - /srv/dev-disk-by-uuid-044ded6a-869c-4981-ad61-d131a8ab5a5d/Storage_and_Cache/palworld:/palworld/       devices:       - '/dev/dri:/dev/dri'       - '/dev/dma_heap:/dev/dma_heap'       - '/dev/mali0:/dev/mali0'       - '/dev/rga:/dev/rga'       - '/dev/mpp_service:/dev/mpp_service'
The devices I put are for the internal GPU, just in case, because I didn't know if it might make a difference since it does at least when transcoding in jellyfin. I just changed the .env file and started the Docker container again after. I don't know if I needed to delete something or not, but when I do that, OMV just rebuilds it.  Game servers, in general, almost never use the GPU as they're not rendering any graphics.
Unfortunately, providing a Box64 build for the rk3588 is all I can do here. I'm not a Box64 contributor, nor an expert at it.
Try playing around with the Box64 dynarec env vars again. Also, you can try setting BOX64_DYNAREC_PAUSE to either 0, 1, 2, or 3 and see if either of those makes any difference.
See the Box64 docs for more options you can experiment with: https://github.com/ptitSeb/box64/blob/main/docs/USAGE.md
Considering the system requirements of running a Palworld server + emulation overhead, I'd say that's more or less the performance I'd expect from single board computers like your Orange Pi based on my exp.


Introduction of SwitchEffect()

Added SwitchEffect() with a callback inbetween the two animations and another callback at the end of the animations. @gskinner I ask you to update the documentation since I am not a native speaker.

See also Issue #119


Spelling, grammar, syntax improvement suggestions

Re-wrote some parts so that it flows better IMO; keep whatever you like :)

ACK


Failed in nopython mode pipeline (step: nopython mode backend)

Cannot cast array(bool, 1d, C) to bool: %".1784" = load {i8*, i8*, i64, i64, i8*, [1 x i64], [1 x i64]}, {i8*, i8*, i64, i64, i8*, [1 x i64], [1 x i64]}* %"$222.6"
File "", line 31:
def stabilitySelect(lambdaNeed,x,y,numIter,mask):

for i in prange(0,b.shape[0]):
if(b[i]>0):
^
[1] During: lowering "branch $222.6, 236, 305" at  (31)
This should not have happened, a problem has occurred in Numba's internals.
Please report the error message and traceback, along with a minimal reproducer
at: https://github.com/numba/numba/issues/new
If more help is needed please feel free to speak to the Numba core developers
directly at: https://gitter.im/numba/numba
Thanks in advance for your help in improving Numba!

Thanks for the report. Whilst there is no minimal working reproducer present I can guess from the lowering error that this is a case of trying to do this:
from numba import njit
import numpy as np

@njit
def foo():
    a = np.ones(1, np.bool_)
    if a > 0:
        print('truebr')
    else:
        print('falsebr')

foo()

and the root cause is that a > 0 is not a predicate but rather a conditional. https://github.com/numba/numba/pull/3901 is WIP to fix.

Hi,
I'm sorry I forgot to add the minimum work. The function acted more like:
@njit(parallel=True, fastmath=True)
Def foo():
A = np.ones(1,10)
If(A[0,0]<0):
Print()
I write the statement like this because I was only passing one value of the array into the statement which is why I couldn't figure out the issue. No python mode has been very difficult to deal with especially in the case of machine learning which was my intended use. Great system but still having some issues on this if statement.
From: stuartarchibald
Sent: Wednesday 3 April, 09:52
Subject: Re: [numba/numba] Failed in nopython mode pipeline (step: nopython mode backend) (#3932)
To: numba/numba
Cc: Mainz, Adam, Author
Thanks for the report. Whilst there is no minimal working reproducer present I can guess from the lowering error that this is a case of trying to do this:
from numba import njit import numpy as np @njit def foo(): a = np.ones(1, np.bool_) if a > 0: print('truebr') else: print('falsebr') foo()
and the root cause is that a > 0 is not a predicate but rather a conditional. #3901https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnumba%2Fnumba%2Fpull%2F3901&data=01|01|adam.mainz%40kcl.ac.uk|c65255550e654db2d27908d6b811bebf|8370cf1416f34c16b83c724071654356|0&sdata=vKm9B9DY5ICEyOpb%2BZeLD5IJ4w%2B%2FTzlLan4%2F%2Frjkt%2BA%3D&reserved=0 is WIP to fix.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnumba%2Fnumba%2Fissues%2F3932%23issuecomment-479400145&data=01|01|adam.mainz%40kcl.ac.uk|c65255550e654db2d27908d6b811bebf|8370cf1416f34c16b83c724071654356|0&sdata=sxM%2BpNZozX8rhM1fJXbBSE%2F5OPo3OaWK0V9iTFDFVQk%3D&reserved=0, or mute the threadhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAs_lFt4iKzUDhkzGjxk7Dibn_odawtLdks5vdGvdgaJpZM4cZNvb&data=01|01|adam.mainz%40kcl.ac.uk|c65255550e654db2d27908d6b811bebf|8370cf1416f34c16b83c724071654356|0&sdata=Y3wPu4owJic%2BRCqz4%2Bz7byT%2BRkoLzZtrY7MylTM3SS4%3D&reserved=0.

Thanks for the update. If you could produce a sample that reproduces the error I'd be happy to take a look. The sample is invalid without Numba being involved as the np.ones calls is incorrect (shape should be a tuple):
import numpy as np

def foo():
    A = np.ones(1, 10)
    if(A[0,0]<0):
        print()

foo()

and if that is fixed then:
from numba import njit
import numpy as np

@njit(parallel=True, fastmath=True)
def foo():
    A = np.ones((1, 10))
    if(A[0,0]<0):
        print()

foo()

compiles and executes fine.

Closing this issue due to lack of information. If more information becomes available please add it and reopen the issue. Thanks.


Added CallableTypeBuilder

🔍 What was changed

🤔 Why?

📝 Checklist

Closes #
How was this tested:

[ ] Tested manually
[ ] Unit tests added

📃 Documentation

Сool idea 👍🏻


RenderFlex overflow indicator is styled differently than mobile

I have this code in a Flutter Mobile project:
class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Scaffold(
        appBar: AppBar(
          title: Text('My title'),
        ),
        body: Center(
          child: Row(
            children: [
              Text('kljasdfljkahsdfjklahdfjklshaldfhjsdhkldhskladhjsfkladhjsfkladhjsfkajhsdfkljahsdfkljahsdlfjkh',
                maxLines: 1,
              ),
            ],
          ),
        ),
      ),
    );
  }
}

void main() {
  runApp(MyApp());
}

When I run the app, the Text widget overflows the space available to the Row.  graphic:

When running the same code in the flutter_web version of DartPad, though, I get this:

It's missing the yellow and black lines. Not a huge deal, I admit, but I miss them.

testing the code above with the latest master
screenshot

doctor
[✓] Flutter (Channel master, 1.21.0-6.0.pre.129, on Mac OS X 10.15.5 19F101, locale en-GB)
    • Flutter version 1.21.0-6.0.pre.129 at /Users/nevercode/development/flutter_master
    • Framework revision 30aef0a3b9 (12 hours ago), 2020-07-30 14:20:21 -0700
    • Engine revision a9910e409c
    • Dart version 2.10.0 (build 2.10.0-1.0.dev 24c7666def)

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
    • Android SDK at /Users/nevercode/Library/Android/sdk
    • Platform android-29, build-tools 29.0.2
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.5)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.5, Build version 11E608c
    • CocoaPods version 1.9.0

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 46.0.2
    • Dart plugin version 193.7361
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)

[✓] VS Code
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.12.2

[✓] Connected device (3 available)
    • macOS (desktop)  • macos      • darwin-x64     • Mac OS X 10.15.5 19F101
    • Web Server (web) • web-server • web-javascript • Flutter Tools
    • Chrome (web)     • chrome     • web-javascript • Google Chrome 84.0.4147.105

• No issues found!

the issue still occurs
thank you

Fixed with https://github.com/flutter/engine/pull/22597. Test on latest master.

Fixed with https://github.com/flutter/engine/pull/22597. Test on latest master.


⚠ Turnos 24x7 has degraded performance

In 6321c93, Turnos 24x7 (https://reservas-dev.alternativasinteligentes.com/) experienced degraded performance:

HTTP code: 200
Response time: 1005 ms

Resolved: Turnos 24x7 performance has improved in e111530.


CoreML conversion

Hi!
Thank you for your job!
Were you trying to convert this model to coreml?
I faced some issues while handling this.

Hi @heorhiikalaichev , thank you for your interest in DocShadow-ONNX-TensorRT.
I believe you were referred here from https://github.com/CXH-Research/DocShadow-SD7K/issues/6, but CoreML conversion does not need ONNX anymore. It is now recommended to convert directly from PyTorch via JIT tracing. Although I'm by no means an expert in CoreML, I think these are the issues you faced when naively tracing and converting the current DocShadow implementation to CoreML, in order:

PyTorch convert function for op 'var' not implemented. (https://github.com/apple/coremltools/issues/1203)
PyTorch convert function for op 'pythonop' not implemented. (https://github.com/apple/coremltools/issues/1135)
PyTorch convert function for op 'upsample_bicubic2d' not implemented.

I provided a simple export script which patches these issues and converts successfully:
https://github.com/fabio-sim/DocShadow-ONNX-TensorRT/blob/11be80d051bcfe372e5fb0d511bf5610db701e3f/export_coreml.py#L1-L59
Please do confirm whether the exported models run correctly on CoreML.

Thank you so much for your answer.
I just made the same changes as you and was able to convert this model.
I will test results and inform you in case they are different from pytorch model results.


Help Me: Suddenly all active hot wallet become inactive and gone from pm2. please help.  I have tried closed issues https://github.com/thebrumby/HotWalletClaimer/issues/246#issuecomment-2379057391 but can not resolve problem.

Introduction:
Before raising a help request, please ensure you've followed the necessary setup and troubleshooting steps. This will help us assist you more effectively and efficiently.
Which self-help guides have you tried to follow? (required):
Please list the guides or resources you've consulted so far.
Describe Your Issue (required):
Please provide a clear and concise description of the help you need.
Have you checked the FAQ (required):
Before submitting, please confirm that you've reviewed the FAQ linked above.
Steps Taken So Far (optional):
Describe what you have tried so far to resolve your issue.
Additional Information:
Include any other information that might help us assist you, such as screenshots, logs, or system details.


feat: put, get and delete functionality

- What I did

feat: Exception handling and response standardization
feat: Update Verb and put functionality using SelfKey, PublicKey and SharedKey
feat: get functionality using Self, Shared and Public keys
feat: Delete functionality

- How to verify it

Tests should pass.

I've marked this as draft for now, as the tests aren't passing in Actions. @shahumang19 please hit Ready for review once the tests are good.

Hey @cpswan , All test cases passed. Feel free to merge the PR whenever you are ready.


Gui

Tested and works, @ProgrammerDan do you want to look over this or should I just merge

@ProgrammerDan merge?

Just those comments, I'm assuming you have tested, so assuming those reasons are great I have no objection to merging. Lots more cleanup to do but at least some decent feature adds is a big help

I'm getting a Null Pointer Exception on line 48 of ConfigManager (this.config = plugin.getConfig();) when I try to run the latest version of JukeAlert.
I tried deleting my old config file and updating CivModCore, but that doesn't seem to have helped. Is there something else I should try?

 Should be fixed now.


More documentation Required

Hi,
Thanks for this plugin. I have installed and configured Versions: 0.5.5 from jmeter plugin. I tried to build this code but somehow found issues when installing in JMeter.
The Downloaded Versions: 0.5.5 is installed successfully. But ICA is not downloading properly. It would be great if you can provide step by step documentation for this plugin.

Hello,
See those 2 links which contain all steps:

https://github.com/Blazemeter/CitrixPlugin/blob/master/PRE_REQUISITES.md
https://github.com/Blazemeter/CitrixPlugin/blob/master/README.md

Anything else needed ?
Regards

I have gone through all these and everything is working as expected now. But ICA is not downloading. I will be thankful to you if you can help me to download ICA as i am not able to do this.
secondly, It would be great if you can provide step by step documentation for How to use this plugin. currently, README contains high level overview. I would like to request you to please provide Step By Step documentation. Thanks. Also your help regarding ICA downloading would be great.

Check in View Results Tree which step among HTTP steps is failing. It should be in red.
Probably a slight variation in URL parameters.

Did you enter the correct host, user, password using the Template , wether site is secured or not ?
If yes, second step will be to record using regular JMeter recorder and once it works put the working part in the existing download Test Fragment

Readme contains step by step guide.

Thank you Very much @pmouawad  for your help and response.
I would like to know one more thing. I am using Web Portal of Citrix & Desktop application. Will this plugin work to implement performance testing? also what is Context Path?
I am sorry , I am new to this, so may need your help to understand this plugin.

Check in View Results Tree which step among HTTP steps is failing. It should be in red.
Probably a slight variation in URL parameters.

Did you enter the correct host, user, password using the Template , wether site is secured or not ?
If yes, second step will be to record using regular JMeter recorder and once it works put the working part in the existing download Test Fragment

Readme contains step by step guide.

I am using following details.
Details: I am using following details to connect to Citrix
Providing following credentials
Getting following errors
Let me know what i am missing . All above details are working as expected for me manually using browser.

@aqiftekhar @pmouawad
This issue is in closed state but I trying to find final update from @aqiftekhar on this case. I am encountering similar issue in ICA download after providing all required details in template.
error log says "test expected to contain ClientSettings"

Thanks

Hi @ANURAG-CHOURASIA-BY
The current issue was closed.
Create a new issue with this data provided, so we can review it independently.


Add Versioned Docker Image Tags to Docker Hub

Issue submitter TODO list

[X] I've searched for an already existing issues here
[X] I'm running a supported version of the application which is listed here and the feature is not present there

Is your proposal related to a problem?
I am frustrated when there are no tags in the public Docker Hub images that match release versions and have to manually map commit refs to version numbers in order to mirror the images into my own infrastructure.
Describe the feature you're interested in
Please publish version number tags for image builds (e.g. provectuslabs/kafka-ui:0.7.1) to Docker Hub.
Describe alternatives you've considered
No response
Version you're running
0.7.1
Additional context
No response

https://hub.docker.com/r/provectuslabs/kafka-ui/tags?page=1&name=v

Apologies, I had anticipated that the image tags in Docker would match the branch tags in GitHub. It seems there is a v prefix 👍


ignore_row_if not adding to test query in expect_compound_columns_to_be_unique

https://github.com/calogica/dbt-expectations/blob/6bd8b64d0978e59fc36dd36972beb37ce18d780a/macros/schema_tests/multi-column/expect_compound_columns_to_be_unique.sql#L21
We noticed that when we set the ignore_row_if argument to any of the options (any_value_is_missing or all_value_is_missing or never). It does not add a where statement to the validation_errors CTE.
We are not seeing the expected behavior for this macro/test.

Ah good catch @prachishah48 ! I think it's b/c row_condition_ext is set but never used 😭 .
Should be able to fix that by tomorrow and cut a new patch release soon after. Is that bug blocking you from something?

Hi @clausherther,
Thank you for getting back to me :D and Nope, it is not blocking me from something but do let me know when it gets produced so I can see how the tests look

I have PR out (#171 ), and the generated SQL in the test suite now looks like this for all_values_are_missing case
with validation_errors as (

    select
        date_col,col_string_b
    from data_test
    where
        1=1
        and
        (
            date_col is not null and
            col_string_b is not null

        )

    group by
        date_col,col_string_b
    having count(*) > 1

)
select * from validation_errors

and like this for the any_value_is_missing case
with validation_errors as (

    select
        date_col,col_string_b
    from data_test
    where
        1=1
        and
        (
            date_col is not null or
            col_string_b is not null

        )

    group by
        date_col,col_string_b
    having count(*) > 1

)
select * from validation_errors

And like this is an additional row_condition: 2=2 is specified:
with validation_errors as (

    select
        date_col,col_string_b
    from postgres.dbt_expectations_integration_tests.data_test
    where
        1=1
        and
        2=2
        and
        (
            date_col is not null and
            col_string_b is not null

        )

    group by
        date_col,col_string_b
    having count(*) > 1

)
select * from validation_errors

Oh great! Thank you!!


Package and load assets from application package instead of firebase

Ensemble fetches all artifacts for an app from firebase. However, assets like fonts can be packaged with the app to avoid refetching and improve performance. We should be able to fetch remote artifacts and read assets from the local app package.
Technical notes:
We will be adding a manifest for studio to map artifacts and local file names. This may be leveraged to read local app assets as well.
Build process will need to put the assets into the right place and update manifest where to find them

cc: @kmahmood74


OTU filter reskin

The intended functionality is partially complete but can be merged in with future issues treated as enhancements.

Codecov Report

Merging #2302 (214aae1) into development (51717a1) will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff              @@
##           development    #2302   +/-   ##
============================================
  Coverage        66.47%   66.47%
============================================
  Files             1316     1316
  Lines            35829    35829
============================================
  Hits             23817    23817
  Misses           12012    12012

Impacted Files
Coverage Δ

app/controllers/otus_controller.rb
54.72% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 51717a1...214aae1. Read the comment docs.

Closing for #2652 tracking.


vbucks lol

hello my lil nigga
my internet doesnt work so can you make a offline version
also can you dualhook it please because i hate my data
deez

la LA la li lu le


IPv6 addresses are compared using case-sensitive string comparisons

Given a domain ipv6.example resolving to the following address:
2001:DB8::1

The following checks should pass, but do not:
check-dns.rb --domain ipv6.example --type AAAA --result 2001:db8::1
check-dns.rb --domain ipv6.example --type AAAA --result 2001:0db8:0000:0000:0000:0000:0000:0001
check-dns.rb --domain ipv6.example --type AAAA --result 2001:0DB8:0000:0000:0000:0000:0000:0001
check-dns.rb --domain ipv6.example --type AAAA --result 2001:db8:0:0:0:0:0:1
check-dns.rb --domain ipv6.example --type AAAA --result 2001:DB8:0:0:0:0:0:1
check-dns.rb --domain ipv6.example --type AAAA --result 2001:db8::<IP_ADDRESS>
check-dns.rb --domain ipv6.example --type AAAA --result 2001:DB8::<IP_ADDRESS>

The only check that passes is the short form, in uppercase:
check-dns.rb --domain ipv6.example --type AAAA --result 2001:DB8:1

Debug output looks like:
$ /opt/sensu/embedded/bin/check-dns.rb --domain ipv6.example --type AAAA --result 2001:db8:0:0:0:0:0:1 --debug
Entries: ;; Answer received from <IP_ADDRESS> (86 bytes)
;;
;; Security Level : UNCHECKED
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35419
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

OPT pseudo-record : payloadsize 512, xrcode 0, version 0, flags 32768

;; QUESTION SECTION (1  record)
;; ipv6.example.	IN	AAAA

;; ANSWER SECTION (1  record)
ipv6.example.	252	IN	AAAA	2001:DB8::1
ipv6.example.	252	IN	AAAA	2001:DB8::1
DNS CRITICAL: Resolved ipv6.example AAAA did not include 2001:db8:0:0:0:0:0:1

Once PR #5 is cut as a release, you can solve the case-sensitivity issue using a regular expression search (or, in fact, account for all the scenarios in this specific case) using a sufficiently thorough regular expression like the following:
check-dns.rb --domain ipv6.example --type AAAA -R '(?i-mx:(2001:(0)?db8:((((000)?0:){5}(000)?1)|:(000)?1|:<IP_ADDRESS>)))'

Granted that this is probably not ideal, but the problem is that the check isn't comprehending the DNS query in any meaningful way. It just compares strings. Here's a simple demonstration of your original example using case-insensitive regular expression searching:
check-dns.rb --domain ipv6.example --type AAAA -R '(?i-mx:(2001:db8::1))'

This will match both 2001:DB8::1 and 2001:db8::1, but of course won't match 2001:0db8::1 with its extra 0. I'm not sure if this helps any, but it might solve some of the least troublesome case-sensitivity issues.
As for making the check actually comprehend DNS answers such that it knows that 2001:db8::1 is the same as 2001:0DB8:0000:0000:0000:0000:0000:0001, and so on, that seems like it would take an extensive rewrite using the standard library's ipaddr library:
require 'ipaddr'
IPAddr.new('2001:db8::1') == IPAddr.new('2001:0DB8:0000:0000:0000:0000:0000:0001')
=> true

This sort of works...
My example is actually a little more complex than I gave in the original description. I've got a fact on each of my nodes (stamped out into my sensu client configuration) representing that node's public IPv6 address, so my actual sensu check has an interpolated value instead of a hardcoded IPv6 address - i.e. my check definition looks like check-dns.rb --domain ":::name:::" --type AAAA --result ":::public_ipaddress6:::".
If I change the parameter to be --regex-match "(?i):::public_ipaddress6:::" it seems to match for my use cases - this may be only because dnsruby and my fact happen to output the IPv6 address in a format that only differs by case, but it does make the check pass for all of my hosts.

I think we can certainly ignore the case for string comparison on this.

@nhinds can you check out #18 and validate this works for you?

It fixes half of the problem: instead of checking via case-sensitive string comparison, this is now via a case-insensitive string comparison.
As stated in the description, string comparison is not sufficient for comparing IPv6 addresses. For example, the addresses 2001:db8::1, 2001:0DB8:0000:0000:0000:0000:0000:0001, and 2001:DB8::<IP_ADDRESS> are all equivalent, but none of them compare as equal to each other using a string comparison.

gotcha let me see what I can cook up.

Honestly I don't really do much with ipv6 so I would like to check the different scenarios please let me know if I am missing anything or if something is not really valid.

[ ] ignore case when comparing ipv6 addresses
[ ] expand ipv6 address comparison to account for shorthand vs longhand
[ ] support scope id? saw this in windows... (${SOME_VALID_IPV6_ADDR}%${SCOPE_ID}
[ ] loopback

I feel like we only really need to solve the top 2.

I don't believe you can store scope IDs in AAAA records - https://tools.ietf.org/html/rfc3596 says it's just the address.
Loopback ipv6 addresses aren't really different from other addresses - they can just get expanded to their canonical form like any other address.
The best way to support this is to use an IP address library to deal with the comparisons, e.g. IPAddr. The lib can handle normalising addresses, ignoring case, expanding addresses, etc.
irb(main):001:0> require 'ipaddr'
=> true
irb(main):002:0> IPAddr
=> IPAddr
irb(main):003:0> IPAddr.new('2001:0DB8:0000:0000:0000:0000:0000:0001')
=> #<IPAddr: IPv6:2001:0db8:0000:0000:0000:0000:0000:0001/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff>
irb(main):004:0> a=_
=> #<IPAddr: IPv6:2001:0db8:0000:0000:0000:0000:0000:0001/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff>
irb(main):005:0> IPAddr.new('2001:DB8::<IP_ADDRESS>')
=> #<IPAddr: IPv6:2001:0db8:0000:0000:0000:0000:0000:0001/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff>
irb(main):006:0> b=_
=> #<IPAddr: IPv6:2001:0db8:0000:0000:0000:0000:0000:0001/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff>
irb(main):007:0> c=IPAddr.new('2001:db8::1')
=> #<IPAddr: IPv6:2001:0db8:0000:0000:0000:0000:0000:0001/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff>
irb(main):008:0> a==b
=> true
irb(main):009:0> a==c
=> true
irb(main):010:0> b==c
=> true

I seem to remember suggesting something like IPAddr. :-)
If that's the goal, I can try to take on the work of refactoring the plugin to use IPAddr wherever an IP address shows up, unless someone else is interested.

@nhinds yup thats what I was looking at using (I got far enough with the ipaddr lib that I can validate it will work for us but I broke some stuff in the process and need to spend some time refactoring it safely). Thanks for clarifying the scope_id, regarding loopback I meant it's usually only tied to localhost so I was wondering if this was even a valid use case to worry about not that it really adds much if any more complexity. IMHO a lot of the plugins are not in a great state and there has been a lack of maintainer involvement (not blaming anyone just 169 plugin repos and 14 people to moderate and most of us don't work for sensu so it's in our spare time) so I have been trying to prioritize by looking for the lowest hanging fruit first.
@jgnagy certainly not gonna turn away a sane community contribution.

@nhinds @jgnagy had some time to get around to this, can you check updated pr?


chore: Initial website for demo added

Initial single page website for toast demo

:tada: This PR is included in version 1.1.4 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


how to use proxy?

hi there, im a bit of a noob and don't quite understand how to use the proxy feature.
i've installed tor and when i type in for example main.py -user username -proxy this is what i get
2023-09-30 17:29:56,437 - INFO - Using proxy: socks5://XXX.X.X.X:XXXX
2023-09-30 17:29:58,442 - ERROR - SOCKSHTTPSConnectionPool(host='www.tiktok.com', port=443): Max retries exceeded with url: /@username /live (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x00000206FB5DE210>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))
2023-09-30 17:29:58,443 - INFO - Waiting 1 minute

Make sure the tor proxy is not only installed but also running before using this feature.


⚠ Samsung has degraded performance

In efaab02, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 9904 ms

Resolved: Samsung performance has improved in 581b04b.


[python] Upgrade the imports in examples

Pyupgrade recommends:
import asyncio
from typing import Annotated, Literal, TypedDict

import dotenv
from typing_extensions import Doc

As everything but Doc has been in typing since 3.8 or 3.9:
https://docs.python.org/3/library/typing.html#typing.Literal
Apparently Doc is non-standard, interesting discussion on the PEP here:
https://discuss.python.org/t/pep-727-documentation-metadata-in-typing/32566/172
It seems unlikely to get accepted, but I assume its the only way to document when using TypedDict? That's what I saw in the examples.

Ah but then there's a Pydantic error in 3.11:
Please use typing_extensions.TypedDict instead of typing.TypedDict on Python < 3.12
Dang! I'll just ignore my linter then / add a noqa. That might be worth mentioning in documentation about using TypeChat with TypedDict.

I assume its the only way to document when using TypedDict

So if you use Annotated[SomeType, Doc(...)] we recognize that. Some folks working on Python recommended we future-proof on that.
But we also just recognize Annotated[SomeType, "Some comment string."] because that's so common anyway (and it just feels better to use).

Ah okay, I saw that mentioned in the Discourse thread but wasnt sure if that was also a standard. I recommend documenting both of those.


Absence of VZA , SZA

Hi,
I am trying to read the S2 and S3 to xarray format. there is no problem with the reflectance dataset but got the metadata like VZA, SZA they do not include anything.

Hello @saeedojaghi , and best wishes for 2022. All the available metadata should be available in the search results (EOProduct.properties).
Metadata is not included in the returned xarray.DataArray but this is something that can be implemented, updating xarray.DataArray.attrs with EOProduct.properties. Does that sound good to you ?

Hello @saeedojaghi , and best wishes for 2022. All the available metadata should be available in the search results (EOProduct.properties). Metadata is not included in the returned xarray.DataArray but this is something that can be implemented in eodag-cube, updating xarray.DataArray.attrs with EOProduct.properties. Does that sound good to you ?

Happy new year sbrunato,I wish the best for you.
Maybe it is not true to say that view zenith angle(VZA) is not metadata because it is associated data with raw of Sentinle3, which is recorded for each pixel.But metadata is about the all image.I need geometrical information like as SZA VZA for each pixel. They should be like addaitional band behind the reflectance bands.
But I checed all of the provider they just provide reflectance or radiance values as bands and nothing about VZA or SZA

The problem here is that the search results you obtain with eodag do not contain the data you want.
The productType that matches your needs may not be configured in eodag yet.
For example, if you search on creodias, the configured product types are:
>>> [pt for pt in dag.list_product_types("creodias") if pt["platform"] == "SENTINEL3"]

[{'ID': 'S3_EFR',
  'abstract': '',
  'instrument': 'OLCI',
  'platform': 'SENTINEL3',
  'platformSerialIdentifier': 'S3A,S3B',
  'processingLevel': 'L1',
  'keywords': 'OLCI,SENTINEL,SENTINEL3,S3,S3A,S3B,L1,EFR',
  'sensorType': 'OPTICAL',
  'license': 'proprietary',
  'title': 'SENTINEL3 EFR',
  'missionStartDate': '2016-02-16T00:00:00Z'},
 {'ID': 'S3_ERR',
  'abstract': '',
  'instrument': 'OLCI',
  'platform': 'SENTINEL3',
  'platformSerialIdentifier': 'S3A,S3B',
  'processingLevel': 'L1,L2',
  'keywords': 'OLCI,SENTINEL,SENTINEL3,S3,S3A,S3B,L1,L2,ERR',
  'sensorType': 'OPTICAL',
  'license': 'proprietary',
  'title': 'SENTINEL3 ERR',
  'missionStartDate': '2016-02-16T00:00:00Z'},
 {'ID': 'S3_LAN',
  'abstract': 'LAN or SR_2_LAN___ (peps)',
  'instrument': 'SRAL',
  'platform': 'SENTINEL3',
  'platformSerialIdentifier': 'S3A,S3B',
  'processingLevel': 'L2',
  'keywords': 'SRAL,SENTINEL,SENTINEL3,S3,S3A,S3B,L2,LAN',
  'sensorType': 'RADAR',
  'license': 'proprietary',
  'title': 'SENTINEL3 SRAL Level-2 LAN',
  'missionStartDate': '2016-02-16T00:00:00Z'},
 {'ID': 'S3_OLCI_L2LFR',
  'abstract': 'The OLCI Level-2 Land Full Resolution (OL_2_LFR) products contain land and atmospheric geophysical products at Full\nresolution with a spatial sampling of approximately 300 m. The products are assumed to be computed in Near Real\nTime (NRT) (i.e. delivered to users less than 3 hours after acquisition), in Non-Time Critical (NTC) (i.e.\nwithin 1 month after acquisition) or in re-processed NTC.\nDetails at https://sentinel.esa.int/web/sentinel/user-guides/sentinel-3-olci/product-types/level-2-land\n',
  'instrument': 'OLCI',
  'platform': 'SENTINEL3',
  'platformSerialIdentifier': 'S3A,S3B',
  'processingLevel': 'L2',
  'keywords': 'OLCI,SENTINEL,SENTINEL3,S3,S3A,S3B,L2,L2LFR,LFR',
  'sensorType': 'OPTICAL',
  'license': 'proprietary',
  'title': 'SENTINEL3 OLCI Level-2 Land Full Resolution',
  'missionStartDate': '2016-02-16T00:00:00Z'},
 {'ID': 'S3_OLCI_L2LRR',
  'abstract': 'The OLCI Level-2 Land Reduced Resolution (OL_2_LRR) products contain land and atmospheric geophysical products\nat Reduced resolution with a spatial sampling of approximately 1.2 km. The products are assumed to be computed\nin Near Real Time (NRT) (i.e. delivered to users less than 3 hours after acquisition), in Non-Time Critical\n(NTC) (i.e. within 1 month after acquisition) or in re-processed NTC.\nDetails at https://sentinel.esa.int/web/sentinel/user-guides/sentinel-3-olci/product-types/level-2-land\n',
  'instrument': 'OLCI',
  'platform': 'SENTINEL3',
  'platformSerialIdentifier': 'S3A,S3B',
  'processingLevel': 'L2',
  'keywords': 'OLCI,SENTINEL,SENTINEL3,S3,S3A,S3B,L2,L2LRR,LRR',
  'sensorType': 'OPTICAL',
  'license': 'proprietary',
  'title': 'SENTINEL3 OLCI Level-2 Land Reduced Resolution',
  'missionStartDate': '2016-02-16T00:00:00Z'},
 {'ID': 'S3_SLSTR_L1RBT',
  'abstract': 'SLSTR Level-1 observation mode products consisting of full resolution,\ngeolocated, co-located nadir and along track view, Top of Atmosphere\n(TOA) brightness temperatures (in the case of thermal IR channels) or\nradiances (in the case of visible, NIR and SWIR channels) from all\nSLSTR channels, and quality flags, pixel classification information\nand meteorological annotations\n',
  'instrument': 'SLSTR',
  'platform': 'SENTINEL3',
  'platformSerialIdentifier': 'S3A,S3B',
  'processingLevel': 'L1',
  'keywords': 'SLSTR,SENTINEL,SENTINEL3,S3,S3A,S3B,L1,L1RBT,RBT',
  'sensorType': 'OPTICAL',
  'license': 'proprietary',
  'title': 'SENTINEL3 SLSTR Level-1',
  'missionStartDate': '2016-02-16T00:00:00Z'},
 {'ID': 'S3_SLSTR_L2LST',
  'abstract': 'The SLSTR Level-2 LST product provides land surface parameters generated on the wide 1 km measurement grid.\nIt contains measurement file with Land Surface Temperature (LST) values with associated parameters (LST\nparameters are computed and provided for each pixel (re-gridded or orphan) included in the 1 km measurement grid)\n',
  'instrument': 'SLSTR',
  'platform': 'SENTINEL3',
  'platformSerialIdentifier': 'S3A,S3B',
  'processingLevel': 'L2',
  'keywords': 'SLSTR,SENTINEL,SENTINEL3,S3,S3A,S3B,L2,L2LST,LST',
  'sensorType': 'OPTICAL',
  'license': 'proprietary',
  'title': 'SENTINEL3 SLSTR Level-2 LST',
  'missionStartDate': '2016-02-16T00:00:00Z'},
 {'ID': 'S3_SRA',
  'abstract': '',
  'instrument': 'SRAL',
  'platform': 'SENTINEL3',
  'platformSerialIdentifier': 'S3A,S3B',
  'processingLevel': 'L1',
  'keywords': 'SRA,SRAL,SENTINEL,SENTINEL3,S3,S3A,S3B,L1',
  'sensorType': 'RADAR',
  'license': 'proprietary',
  'title': 'SENTINEL3 SRAL Level-1',
  'missionStartDate': '2016-02-16T00:00:00Z'},
 {'ID': 'S3_SRA_A_BS',
  'abstract': '',
  'instrument': 'SRAL',
  'platform': 'SENTINEL3',
  'platformSerialIdentifier': 'S3A,S3B',
  'processingLevel': 'L1',
  'keywords': 'SRA,SRAL,SENTINEL,SENTINEL3,S3,S3A,S3B,L1,BS',
  'sensorType': 'RADAR',
  'license': 'proprietary',
  'title': 'SENTINEL3 SRAL Level-1',
  'missionStartDate': '2016-02-16T00:00:00Z'},
 {'ID': 'S3_SRA_BS',
  'abstract': '',
  'instrument': 'SRAL',
  'platform': 'SENTINEL3',
  'platformSerialIdentifier': 'S3A,S3B',
  'processingLevel': 'L1',
  'keywords': 'SRA,SRAL,SENTINEL,SENTINEL3,S3,S3A,S3B,L1,BS',
  'sensorType': 'RADAR',
  'license': 'proprietary',
  'title': 'SENTINEL3 SRAL Level-1',
  'missionStartDate': '2016-02-16T00:00:00Z'},
 {'ID': 'S3_WAT',
  'abstract': '',
  'instrument': 'SRAL',
  'platform': 'SENTINEL3',
  'platformSerialIdentifier': 'S3A,S3B',
  'processingLevel': 'L2',
  'keywords': 'SRAL,SENTINEL,SENTINEL3,S3,S3A,S3B,L2,WAT',
  'sensorType': 'RADAR',
  'license': 'proprietary',
  'title': 'SENTINEL3 SRAL Level-2 WAT',
  'missionStartDate': '2016-02-16T00:00:00Z'}]

If none of them matches your needs, can you please give us some information on this missing product type, going to https://finder.creodias.eu/ and report us which instrument, processing level, product type, etc would give you the data you want.
Thanks
related to #168

Hi again, thak you for your reply.
I am looking for S3_EFR, which is provided by creodias and onda.when you download through API, they include everything and you have a full package, however when you want to convert them directly to xarray dataset without downloading in that case you just have access to reflectance datatset.

This data seems to be included in tie_geometries. Does this work getting data for this band on you side ?

Hi Sylvain,
Thank you for your answer,I tried to run the same code that you provided

#25 and #26 will be needed for this.
For the moment a new release has not been published yet (but it will soon), so you'll need to use develop branch to test it.
Install following https://github.com/CS-SI/eodag-cube#contribute or
pip install git+https://github.com/CS-SI/eodag-cube.git@develop

I really appreciate it, Sylvain,
Finally, I fixed it. But as you know, to get the data, we have to download it first.
How is it possible without downloading the data, we reach the data through the cloud API in the form of xarray ??
It is strange for me also, why when we get "S2_MSI_L2A_COG" from "earth_search_cog", it is not necessary to download at first. But for S3_EFR from the provider  'creodias' , it is required to download data and then post process.
Does it because of the COG format of S2_MSI_L2A_COG??
Thank you in advance

I think there is another problem when you want to put the extent parameter in get_data and crop the data, the output is empty.
Do you have any comments on this problem?
Thank you

This data seems to be included in tie_geometries. Does this work getting data for this band on you side ?

As I said before it works now, nut I have another problem when I want to get_data from 'instrument_data' it does not work and give the following error:

But as you know, to get the data, we have to download it first.
How is it possible without downloading the data, we reach the data through the cloud API in the form of xarray ??

For the moment, zipped products have to be downloaded first for providers like onda or creodias because they do not provide a direct access to bands.
Using STAC providers (astraea_eod, usgs_satapi_aws, earth_search) data can be accessed per band without downloading the full product.

I think there is another problem when you want to put the extent parameter in get_data and crop the data, the output is empty.

This is because you get an error. If you look in the logs, you will see:
[ERROR   ] (_product         ) The transformation is already "north up" or a transformation between pixel/line and georeferenced coordinates cannot be computed for /path/to/Oa08_radiance.nc. There is no affine transformation and no GCPs. Specify transformation option SRC_METHOD=NO_GEOTRANSFORM to bypass this check.

Projection information is lacking as is for this data to enable on-the-fly cropping with get_data()

when I want to get_data from 'instrument_data' it does not work and give the following error:.

I also encounter the same issue with this band and opened a separate issue, see #28

STAC providers

Do you have any suggestion for this problem?

Do you have any suggestions for this problem?
I think we have to georeference the downloaded NetCDF file and then crop it

eodag-cube cannot georeference the data. So yes, you should get the whole NetCDF and georeference then crop it out of eodag-cube


Register proto files to buf.build

Summary

Register proto files of lbm-sdk to buf.build
Problem Definition

Every time we should explain developers what is changed, how to use gRPC, etc.
But I think we reduce the effort if we register our gRPC information to buf.build.
Proposal

The proto file of lbm-sdk need to be registered in buf.build so that users using the chain can refer to the gRPC information.

For Admin Use

[x] Not duplicate issue
[ ] Appropriate labels applied
[ ] Appropriate contributors tagged
[ ] Contributor assigned/self-assigned

I'll close this issue.


merge back the one other change and follow nick with moving to version 1.2 for feature add.

note we need to add some stuff to type the TS  ;(

Note see syonfox-dev for actual fixes switching to a real editor


Fix background polling element in table view blocks scrolling after appearing

Fixes #5175
ping @xavijam @iriberri

:+1:

:revolving_hearts:


breakpad: 2016-03-28 -> 2017-11-13, resolve glibc 2.26 compat

Motivation for this change
Alternative way to resolve breakpad compatibility (#31696), conflicts with #31746.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Oops, I guess the deletion has been already merged...

Ouch, I was faster >_>. Still, let's not leave an untested package in nixpkgs -- we have a lot of this dead weight now already.

OK.


Feature log control

Re-creating because travis-ci is stuck in #651, it doesn't recognise the conflict fixing and also not the new travis config that solves the build process. So pushing again
What current issue(s) does this address, or what feature is it adding?
Addresses #626
How did you solve this problem?

Changed logging from a single logger instance for neo-python wide logging, to 1 logger per component (i.e. network, db, vm and a generic default for all others).
Introduced a LogManager class that manages these loggers and allows them to be configured individually i.e. set network to INFO level, while having vm on DEBUG. At the same time the setup allows configuring the stdout of the CLI to certain levels, while having file logging output at other levels (previously not possible as logzero uses the same levels for all).

How did you make sure your solution works?
write tests
Are there any special changes in the code that we should be aware of?
Yes, logzero was mostly replaced by the LogManager. Note that I kept the LogFormatter from logzero so the look and feel is the same.
Why not use logzero?
I actually intended to keep logzero but it has some unexpected defaults/behaviour as described here.
In summary, different handlers always have the same levels, propagate on loggers is set to False from the default True. Looking at the time period these PRs and some issues related on the repository have been open/not addressed, made me believe the intention is probably to keep the behaviour as is. For neo-python it did not match the intended features/requirements I had in mind, being:

turn logging on or off for independent components
set independent logging levels for components
set independent logging levels for stdout (StreamHandler) and other handlers like RotatingFileHandler

Finally, I'm aware that neo-python-core uses logzero at some places. These will not be filtered by the logmanager. I believe this is not an issue, because as some point these log messages should be either removed or replaced by exceptions such that they can be properly responded to.
Please check the following, if applicable:

[X] Did you add any tests?
[X] Did you run make lint?
[X] Did you run make test?
[X] Are you making a PR to a feature branch or development rather than master?
[X] Did you add an entry to CHANGELOG.rst? (if not, please do)

Coverage increased (+0.2%) to 82.458% when pulling a21178e0cbaaa3be75810bf26ebcb3beb5e4488a on ixje:feature_log_control_recreate into 53e20b124e21cbcf85f5e38a29272f428bc74149 on CityOfZion:development.


fill in some of the docs

PR Summary
Working on filling in some of the gaps in the documentation. I'm still working on this.

PR Checklist

[x] Adds a test for any bugs fixed. Adds tests for new features.
[x] Format your changes by using the make format command after configuring with cmake.
[x] Document any new features, update documentation for changes made.
[x] Make sure the copyright notice on any files you modified is up to date.

This is now ready for review. Some comments:

I think this is a good time to resubmit the docs through RASSTI. I have submitted.
The docs are now roughly complete, IMO. This means the repo, IMO, is ready for a JOSS paper submission.
@chadmeyer @jdolence please double check me on the PTE solver section
RAMP has no documentation at this time. I don't feel qualified to describe it. @dholladay00 @chadmeyer or @jhp-lanl your help there would be welcome.
The fortran API is not documented. Let's wait until it solidifies a bit.
I now have the github tests install the Python interface so the docs build properly for Python

Thanks @jhp-lanl ! I fixed the text issues you found. Regarding the sudo and the location of the constructors, I'd rather leave those.

Makes sense!


Plugin claims invalid discord url

Hi!
I've installed the plugin with BRAT
I've created a new discord webhook and copy-pasted the URL in the settings of the plugin. However, when using the share nothing appears in discord. Looking in the obsidan console shows that the plugin is throwing an error "plugin:obsidian-discord-share:108 Uncaught (in promise) Error [WebhookURLInvalid]: The provided webhook URL is not valid."
I have created several webhooks but the result is the same each time
Have I missed something about how the webhook needs to be setup?

Thanks for writing up an issue. I'm not able to reproduce this, even with newly created webhooks so I don't believe anything has changed on my (or Discord's) end. But let's dive a little deeper.
If you go to your data.json file for the plugin (vault/.obsidian/plugins/obsidian-discord-share/data.json), it should look something like this. Does your webhook's url have a similar structure?

Thanks for the quick response! The format looks similar but the url I get is discordapp.com rather than discord.com for some reason:
https://discordapp.com/api/webhooks/xxxxx/yyyyyyyyyyyyyyyyyy_yyyy

Interesting. That was their old domain but I'd still expect it to work if they're generating that URL for you. Try sending something to your webhook from https://discohook.org/ and let me know if that works.

The url works with discohook. However, I found a solution - if I change the url to discord.com instead of discordapp.com
the plugin works. Seems that some internal vetting of the url in the plugin code doesn't like discordapp.com as a pattern
Thanks for your assistance!

Another person ran into this. Will look into a fix soon.


Safety Levels

When running APT2, I am receiving several messages about modules being disabled because the safety level is not 4.  Where do I find more information about resolving these messages?
Thanks

Nevermind.  I just learned to read!
Later


cmd/cgo: invalid option -E passed to ccache

What version of Go are you using (go version)?

$ go version
go version go1.16 linux/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/abhisek/.cache/go-build"
GOENV="/home/abhisek/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/abhisek/.gvm/pkgsets/go1.16/global/pkg/mod"
GONOPROXY="bitbucket.org/xxxxxxxxxx"
GONOSUMDB="bitbucket.org/xxxxxxxxxx"
GOOS="linux"
GOPATH="/home/abhisek/.gvm/pkgsets/go1.16/global"
GOPRIVATE="bitbucket.org/xxxxxxxxxx"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/abhisek/.gvm/gos/go1.16"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/abhisek/.gvm/gos/go1.16/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16"
GCCGO="gccgo"
AR="ar"
CC="ccache"
CXX="ccache"
CGO_ENABLED="1"
GOMOD="/home/abhisek/go/src/influxdb/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="gcc -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3058954447=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I installed ccache. From then on I'm getting the issue. I'm not sure how to resolve this.
I've uninstalled ccache, tried to even build ccache from src and install. All in vain.
I also tried reinstalling go.
The current installation is using gvm.
Even I tried reinstalling and using system wide go but I'm always seeing this issue when I run go via cmdline. I do not see this error while running using my IDE. My primary is zsh but I also tried using bash. Same issue.
I also reset my .zshrc file but still the same issue.
What did you expect to see?
▶ go run main.go
<NORMAL OUTPUT>

What did you see instead?
▶ go run main.go
# runtime/cgo
ccache: invalid option -- 'E'
Usage:
    ccache [options]
    ccache compiler [compiler options]
    compiler [compiler options]          (via symbolic link)

Common options:
    -c, --cleanup              delete old files and recalculate size counters
                               (normally not needed as this is done
                               automatically)
    -C, --clear                clear the cache completely (except configuration)
        --config-path PATH     operate on configuration file PATH instead of the
                               default
    -d, --directory PATH       operate on cache directory PATH instead of the
                               default
        --evict-older-than AGE remove files older than AGE (unsigned integer
                               with a d (days) or s (seconds) suffix)
    -F, --max-files NUM        set maximum number of files in cache to NUM (use
                               0 for no limit)
    -M, --max-size SIZE        set maximum size of cache to SIZE (use 0 for no
                               limit); available suffixes: k, M, G, T (decimal)
                               and Ki, Mi, Gi, Ti (binary); default suffix: G
    -X, --recompress LEVEL     recompress the cache to level LEVEL (integer or
                               "uncompressed") using the Zstandard algorithm;
                               see "Cache compression" in the manual for details
    -o, --set-config KEY=VAL   set configuration item KEY to value VAL
    -x, --show-compression     show compression statistics
    -p, --show-config          show current configuration options in
                               human-readable format
    -s, --show-stats           show summary of configuration and statistics
                               counters in human-readable format
    -z, --zero-stats           zero statistics counters

    -h, --help                 print this help text
    -V, --version              print version and copyright information

Options for scripting or debugging:
        --checksum-file PATH   print the checksum (64 bit XXH3) of the file at
                               PATH
        --dump-manifest PATH   dump manifest file at PATH in text format
        --dump-result PATH     dump result file at PATH in text format
        --extract-result PATH  extract data stored in result file at PATH to the
                               current working directory
    -k, --get-config KEY       print the value of configuration key KEY
        --hash-file PATH       print the hash (160 bit BLAKE3) of the file at
                               PATH
        --print-stats          print statistics counter IDs and corresponding
                               values in machine-parsable format

See also the manual on <https://ccache.dev/documentation.html>.

I'm not sure I understand the problem (then again, I don't know ccache very well, beyond what it is). That looks like a bad flag passed to ccache, but is it passed by Go? Is the problem that that the Go command and cgo are invoking the C compiler with a flag that ccache doesn't understand? I've updated the title based on this understanding, but if that doesn't seem correct feel free to fix it.

FWIW, I'm not sure if you're going to find the expertise on the issue tracker for working with ccache. It might be more productive (and get more visibility from more Go users) if you ask on the golang-nuts Google group. If there's a bug or a feature request or a change you think we should make, then we can follow-up here in the issue tracker.

When ccache is run as cc you should see the usual options including -E. Your go env output shows that CC and CXX are incorrectly set to ccache. cc is typically a symbolic link to ccache and it wraps the underlying compiler.
Some Linux distrbutions use ccache as the default compiler, and I've found it generally works fine with the Go toolchain.

I agree: the ccache command is not a compiler, and you should not set the environment variable CC to ccache.  Closing because this is not a bug in Go.


Scrape with Prometheus operator result in random 503

What did you do?
We are using kube-prometheus-stack and have ServiceMonitor defined for various services.
Name:         pg-grafana-pg-metrics
Namespace:    user-grafana
Labels:       app.kubernetes.io/component=metrics
              app.kubernetes.io/name=pg-grafana-pg-metrics
Annotations:  argocd.argoproj.io/sync-wave: 15
              argocd.argoproj.io/tracking-id: user-grafana:monitoring.coreos.com/ServiceMonitor:user-grafana/pg-grafana-pg-metrics
API Version:  monitoring.coreos.com/v1
Kind:         ServiceMonitor
Spec:
  Endpoints:
    Path:    /metrics
    Port:    http-patroni
    Scheme:  http
  Selector:
    Match Labels:
      App:  pg-grafana-pg-metrics
---
Name:         pg-keycloak-db-pg-metrics
Namespace:    security
API Version:  monitoring.coreos.com/v1
Kind:         ServiceMonitor
Spec:
  Endpoints:
    Path:    /metrics
    Port:    http
    Scheme:  https
    Tls Config:
      Ca File:               /etc/prometheus/istio-certs/root-cert.pem
      Cert File:             /etc/prometheus/istio-certs/cert-chain.pem
      Insecure Skip Verify:  true
      Key File:              /etc/prometheus/istio-certs/key.pem
    enableHttp2:             false
    Path:                    /metrics
    Port:                    http-patroni
    Scheme:                  https
    Tls Config:
      Ca File:               /etc/prometheus/istio-certs/root-cert.pem
      Cert File:             /etc/prometheus/istio-certs/cert-chain.pem
      Insecure Skip Verify:  true
      Key File:              /etc/prometheus/istio-certs/key.pem
  Selector:
    Match Labels:
      App:  pg-keycloak-db-pg-metrics

What did you expect to see?
Every time when we roll new nodes or when postgres pods are recreated, prometheus should be able to scrape the targets.
What did you see instead? Under which circumstances?
Every time when we roll new nodes or when Postgres pods are recreated, we see 503 Error in Prometheus Target status for random Postgres pods. upon restart of Prometheus pods it gets fixed and we see healthy target status Postgres pods. Verified that all the targets details are correct, and services are working. Also, On doing curl to Postgres pod scrape endpoint from Prometheus pod I can see the metric, but the Prometheus target status still shows 503 until we restart Prometheus.
System information
No response
Prometheus version
prometheus, version 2.54.1
kube-prometheus-stack - 62.3.1

Prometheus configuration file
No response
Alertmanager version
No response
Alertmanager configuration file
No response
Logs
No response

Maybe some things change and Prometheus is not made aware of them (maybe some certs are rotated etc.)
To rule out the prom operator, could you pin its version? or make a no-op change to the corresponding ServiceMonitor (to its labels/annotations) so the operator triggers a sync...

Yeah, without looking too closely, I'm not sure whether this is a Prometheus bug or an Operator bug (e.g. if the Operator just doesn't reload / reconfigure Prometheus correctly). In the latter case, the best place to file an issue would be over at https://github.com/prometheus-operator/prometheus-operator/issues.

@machine424 @juliusv I tried a no op change (added label) in Service Monitor and there was no config reload , validated in logs of config reloader for prometheus.
and also checked a spec change in Service Monitor , triggers the config reload as it should.

If the issue disappears after any change to the ServiceMonitor or any other CR watched by the prom operator (I was proposing a no-op change only to limit side effects), you should indeed reach out to the prom operator community.


Reading in and writing out bookmarks.html exported by major Browsers fails

Example: Brave browser. Export Bookmarks.
Document document = Jsoup.parse(input);
Files.write(output, document.outerHtml().getBytes());
The written out bookmarks.html breaks when I re-import it in Brave. Especially Folders and all the bookmarks they contain are gone.
I played around with different Parsers and Settings but nothing helped.

Can you add a snippet of the original HTML? What is causing the error, is it a parse error, or what?

One quick thought - are your character set inputs and outputs correct? Using .getBytes() is going to be platform dependant and may not be the same as what the content was initially saved in. Generally I would suggest specifying the charset on output (in the call to getBytes, or use a Writer).
Would be good if you can add a simple and minimal testcase to verify.

(Closing, insufficient detail provided to make progress)


Align lv_micropython to Micropyton v1.18

https://github.com/micropython/micropython/releases/tag/v1.18

Done.