Error: Unexpected state in sparklyr backend, terminating connection

Hi there,
I am getting this error when using rsparkling and h2o and cannot figure out why - I swear that the same configuration had worked just a few weeks ago.
I can connect to spark using just sparklyr without any issues.
When I load rsparkling and h2o, I can connect tot he spark cluster, but when I try to check the h2o status using h2o_context(sc) I get the following error:
Error: Unexpected state in sparklyr backend, terminating connection: failed to invoke spark command
17/11/09 14:12:25 INFO SparkContext: Invoking stop() from shutdown hook
17/11/09 14:12:25 INFO SparkUI: Stopped Spark web UI at http://<IP_ADDRESS>:4041
17/11/09 14:12:25 INFO StandaloneSchedulerBackend: Shutting down all executors
17/11/09 14:12:25 INFO CoarseGrainedSchedulerBackend$DriverEndpoint: Asking each executor to shut down
17/11/09 14:12:25 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/11/09 14:12:25 INFO MemoryStore: MemoryStore cleared
17/11/09 14:12:25 INFO BlockManager: BlockManager stopped
17/11/09 14:12:25 INFO BlockManagerMaster: BlockManagerMaster stopped
17/11/09 14:12:25 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/11/09 14:12:25 INFO SparkContext: Successfully stopped SparkContext
17/11/09 14:12:25 INFO ShutdownHookManager: Shutdown hook called
17/11/09 14:12:25 INFO ShutdownHookManager: Deleting directory /private/var/folders/gl/vgw262w9227cwqvzk595rbvjygdzh8/T/spark-f7a3c7b8-932b-4a8e-9507-390c46cfeb4c

And then any h2o commands fail with Error in writeBin(con, backend) : invalid connection.
The setup I was using which worked previously and now no longer works is:
Cluster/local Spark version: 2.2
R: 3.4.2
RStudio: 1.0.153
Sparklyr: 0.6.2
h2o: <IP_ADDRESS>
rsparkling: 2.2
I have now also tried every combination of updated packages, including the latest h2o (<IP_ADDRESS>), and the latest stable and master version of sparklyr (0.6.4, 0.7.0-9104). I get the same error.

Hi @renegademonkey ,
Can you send over more logs? Related issue might be here: https://github.com/rstudio/sparklyr/issues/933

Hi again @navdeep-G. That's the full log from RStudio.
In spark, the only log details I could see before sparklyr disconnected were:
org.apache.spark.rdd.RDD.collect(RDD.scala:935)
org.apache.spark.h2o.backends.internal.SpreadRDDBuilder.collectNodesInfo(SpreadRDDBuilder.scala:105)
org.apache.spark.h2o.backends.internal.SpreadRDDBuilder.build(SpreadRDDBuilder.scala:61)
org.apache.spark.h2o.backends.internal.SpreadRDDBuilder.build(SpreadRDDBuilder.scala:43)
org.apache.spark.h2o.backends.internal.InternalH2OBackend.init(InternalH2OBackend.scala:69)
org.apache.spark.h2o.H2OContext.init(H2OContext.scala:121)
org.apache.spark.h2o.H2OContext$.getOrCreate(H2OContext.scala:352)
org.apache.spark.h2o.H2OContext$.getOrCreate(H2OContext.scala:387)
org.apache.spark.h2o.H2OContext.getOrCreate(H2OContext.scala)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
sparklyr.Invoke$.invoke(invoke.scala:102)
sparklyr.StreamHandler$.handleMethodCall(stream.scala:97)
sparklyr.StreamHandler$.read(stream.scala:62)
sparklyr.BackendHandler.channelRead0(handler.scala:52)
sparklyr.BackendHandler.channelRead0(handler.scala:14)
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)

I just tried this with the latest sparklyr master (0.7.0-9105), as well as rsparkling 2.2. and h2o <IP_ADDRESS> and now it seems to be working again.

OK, I spoke too soon. Followed the EXACT SAME protocol as yesterday, and now the same errors has come up. Same install versions and nothing else changed. I really don't understand where this is coming from.

Not sure if this helps, but I've just turned on the spark log console and have the following output when i run h2o_context(sc):
> h2o_context(sc)
17/11/16 09:39:43 WARN H2OContext: Method H2OContext.getOrCreate with an argument of type SparkContext is deprecated and parameter of type SparkSession is preferred.
17/11/16 09:39:43 WARN InternalH2OBackend: Increasing 'spark.locality.wait' to value 30000
17/11/16 09:39:43 WARN InternalH2OBackend: Due to non-deterministic behavior of Spark broadcast-based joins
We recommend to disable them by
configuring `spark.sql.autoBroadcastJoinThreshold` variable to value `-1`:
sqlContext.sql("SET spark.sql.autoBroadcastJoinThreshold=-1")
17/11/16 09:39:43 WARN InternalH2OBackend: The property 'spark.scheduler.minRegisteredResourcesRatio' is not specified!
We recommend to pass `--conf spark.scheduler.minRegisteredResourcesRatio=1`
17/11/16 09:39:43 INFO InternalH2OBackend: Starting H2O services: Sparkling Water configuration:
  backend cluster mode : internal
  workers              : None
  cloudName            : sparkling-water-user_app-20171116093826-0333
  flatfile             : true
  clientBasePort       : 54321
  nodeBasePort         : 54321
  cloudTimeout         : 60000
  h2oNodeLog           : INFO
  h2oClientLog         : WARN
  nthreads             : 60
  drddMulFactor        : 10
17/11/16 09:39:43 INFO SparkContext: Starting job: collect at SpreadRDDBuilder.scala:105
17/11/16 09:39:44 INFO DAGScheduler: Got job 0 (collect at SpreadRDDBuilder.scala:105) with 61 output partitions
17/11/16 09:39:44 INFO DAGScheduler: Final stage: ResultStage 0 (collect at SpreadRDDBuilder.scala:105)
17/11/16 09:39:44 INFO DAGScheduler: Parents of final stage: List()
17/11/16 09:39:44 INFO DAGScheduler: Missing parents: List()
17/11/16 09:39:44 INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[1] at mapPartitionsWithIndex at SpreadRDDBuilder.scala:102), which has no missing parents
17/11/16 09:39:44 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 2.1 KB, free 2.0 GB)
17/11/16 09:39:44 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 1356.0 B, free 2.0 GB)
17/11/16 09:39:44 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on <IP_ADDRESS>:53141 (size: 1356.0 B, free: 2.0 GB)
17/11/16 09:39:44 INFO SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:1006
17/11/16 09:39:44 INFO DAGScheduler: Submitting 61 missing tasks from ResultStage 0 (MapPartitionsRDD[1] at mapPartitionsWithIndex at SpreadRDDBuilder.scala:102) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
17/11/16 09:39:44 INFO TaskSchedulerImpl: Adding task set 0.0 with 61 tasks
17/11/16 09:39:44 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, XXX.XXX.182, executor 3, partition 0, PROCESS_LOCAL, 4717 bytes)
...
17/11/16 09:39:44 INFO TaskSetManager: Starting task 59.0 in stage 0.0 (TID 59, XXX.XXX.241.183, executor 0, partition 59, PROCESS_LOCAL, 4717 bytes)
17/11/16 09:39:48 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on XXX.XXX.241.184:40304 (size: 1356.0 B, free: 9.5 GB)
17/11/16 09:39:48 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on XXX.XXX.241.185:37568 (size: 1356.0 B, free: 9.5 GB)
17/11/16 09:39:49 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on XXX.XXX.241.182:45675 (size: 1356.0 B, free: 9.5 GB)
17/11/16 09:39:49 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on XXX.XXX.241.181:46767 (size: 1356.0 B, free: 9.5 GB)
17/11/16 09:39:49 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on XXX.XXX.241.183:35955 (size: 1356.0 B, free: 9.5 GB)
17/11/16 09:39:49 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on XXX.XXX.241.186:34021 (size: 1356.0 B, free: 9.5 GB)
17/11/16 09:39:49 INFO BlockManagerInfo: Added rdd_0_7 in memory on XXX.XXX.241.184:40304 (size: 32.0 B, free: 9.5 GB)
...
17/11/16 09:39:49 INFO TaskSetManager: Finished task 26.0 in stage 0.0 (TID 26) in 4767 ms on XXX.XXX.241.185 (executor 5) (13/61)
17/11/16 09:39:49 INFO BlockManagerInfo: Added rdd_0_4 in memory on XXX.XXX.241.186:34021 (size:
...
1
17/11/16 09:39:49 INFO TaskSetManager: Finished task 40.0 in stage 0.0 (TID 40) in 4828 ms on XXX.XXX.241.186 (executor 2) (37/61)
...
17/11/16 09:39:49 INFO TaskSetManager: Finished task 35.0 in stage 0.0 (TID 35) in 4880 ms on XXX.XXX.241.183 (executor 0) (61/61)
17/11/16 09:39:49 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
17/11/16 09:39:49 INFO DAGScheduler: ResultStage 0 (collect at SpreadRDDBuilder.scala:105) finished in 5.038 s
17/11/16 09:39:49 INFO DAGScheduler: Job 0 finished: collect at SpreadRDDBuilder.scala:105, took 5.418373 s
17/11/16 09:39:49 INFO ParallelCollectionRDD: Removing RDD 0 from persistence list
17/11/16 09:39:49 INFO BlockManager: Removing RDD 0
17/11/16 09:39:49 INFO SpreadRDDBuilder: Detected 6 spark executors for 6 H2O workers!
17/11/16 09:39:49 INFO InternalH2OBackend: Launching H2O on following 6 nodes: (0,XXX.XXX.241.183,-1),(1,XXX.XXX.241.181,-1),(2,XXX.XXX.241.186,-1),(3,XXX.XXX.241.182,-1),(4,XXX.XXX.241.184,-1),(5,XXX.XXX.241.185,-1)
17/11/16 09:39:49 INFO BlockManagerInfo: Removed broadcast_0_piece0 on <IP_ADDRESS>:53141 in memory (size: 1356.0 B, free: 2.0 GB)
17/11/16 09:39:49 INFO BlockManagerInfo: Removed broadcast_0_piece0 on XXX.XXX.241.185:37568 in memory (size: 1356.0 B, free: 9.5 GB)
17/11/16 09:39:49 INFO BlockManagerInfo: Removed broadcast_0_piece0 on XXX.XXX.241.186:34021 in memory (size: 1356.0 B, free: 9.5 GB)
17/11/16 09:39:49 INFO BlockManagerInfo: Removed broadcast_0_piece0 on XXX.XXX.241.183:35955 in memory (size: 1356.0 B, free: 9.5 GB)
17/11/16 09:39:49 INFO BlockManagerInfo: Removed broadcast_0_piece0 on XXX.XXX.241.182:45675 in memory (size: 1356.0 B, free: 9.5 GB)
17/11/16 09:39:49 INFO BlockManagerInfo: Removed broadcast_0_piece0 on XXX.XXX.241.181:46767 in memory (size: 1356.0 B, free: 9.5 GB)
17/11/16 09:39:49 INFO SparkContext: Starting job: collect at InternalBackendUtils.scala:163
17/11/16 09:39:49 INFO DAGScheduler: Got job 1 (collect at InternalBackendUtils.scala:163) with 6 output partitions
17/11/16 09:39:49 INFO DAGScheduler: Final stage: ResultStage 1 (collect at InternalBackendUtils.scala:163)
17/11/16 09:39:49 INFO DAGScheduler: Parents of final stage: List()
17/11/16 09:39:49 INFO DAGScheduler: Missing parents: List()
17/11/16 09:39:49 INFO DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[3] at map at InternalBackendUtils.scala:100), which has no missing parents
17/11/16 09:39:49 INFO BlockManagerInfo: Removed broadcast_0_piece0 on XXX.XXX.241.184:40304 in memory (size: 1356.0 B, free: 9.5 GB)
17/11/16 09:39:49 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.2 KB, free 2.0 GB)
17/11/16 09:39:49 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 2023.0 B, free 2.0 GB)
17/11/16 09:39:49 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on <IP_ADDRESS>:53141 (size: 2023.0 B, free: 2.0 GB)
17/11/16 09:39:49 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:1006
17/11/16 09:39:49 INFO DAGScheduler: Submitting 6 missing tasks from ResultStage 1 (MapPartitionsRDD[3] at map at InternalBackendUtils.scala:100) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5))
17/11/16 09:39:49 INFO TaskSchedulerImpl: Adding task set 1.0 with 6 tasks
17/11/16 09:39:49 INFO TaskSetManager: Starting task 2.0 in stage 1.0 (TID 61, XXX.XXX.241.186,
...
17/11/16 09:39:50 INFO DAGScheduler: Parents of final stage: List()
17/11/16 09:39:50 INFO DAGScheduler: Missing parents: List()
17/11/16 09:39:50 INFO DAGScheduler: Submitting ResultStage 2 (InvokeOnNodesRDD[2] at RDD at InvokeOnNodesRDD.scala:27), which has no missing parents
17/11/16 09:39:50 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 1952.0 B, free 2.0 GB)
17/11/16 09:39:50 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 1221.0 B, free 2.0 GB)
17/11/16 09:39:50 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on <IP_ADDRESS>:53141 (size: 1221.0 B, free: 2.0 GB)
17/11/16 09:39:50 INFO SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:1006
17/11/16 09:39:50 INFO DAGScheduler: Submitting 6 missing tasks from ResultStage 2 (InvokeOnNodesRDD[2] at RDD at InvokeOnNodesRDD.scala:27) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5))
17/11/16 09:39:50 INFO TaskSchedulerImpl: Adding task set 2.0 with 6 tasks
17/11/16 09:39:50 INFO TaskSetManager: Starting task 5.0 in stage 2.0 (TID 67, XXX.XXX.241.185, executor 5, partition 5, NODE_LOCAL, 4822 bytes)
17/11/16 09:39:50 INFO TaskSetManager: Starting task 2.0 in stage 2.0 (TID 68, XXX.XXX.241.186, executor 2, partition 2, NODE_LOCAL, 4822 bytes)
17/11/16 09:39:50 INFO TaskSetManager: Starting task 3.0 in stage 2.0 (TID 69, XXX.XXX.241.182, executor 3, partition 3, NODE_LOCAL, 4822 bytes)
17/11/16 09:39:50 INFO TaskSetManager: Starting task 0.0 in stage 2.0 (TID 70, XXX.XXX.241.183, executor 0, partition 0, NODE_LOCAL, 4822 bytes)
17/11/16 09:39:50 INFO TaskSetManager: Starting task 4.0 in stage 2.0 (TID 71, XXX.XXX.241.184, executor 4, partition 4, NODE_LOCAL, 4822 bytes)
17/11/16 09:39:50 INFO TaskSetManager: Starting task 1.0 in stage 2.0 (TID 72, XXX.XXX.241.181, executor 1, partition 1, NODE_LOCAL, 4822 bytes)
17/11/16 09:39:50 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on XXX.XXX.241.185:37568 (size: 1221.0 B, free: 9.5 GB)
17/11/16 09:39:50 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on XXX.XXX.241.186:34021 (size: 1221.0 B, free: 9.5 GB)
17/11/16 09:39:50 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on XXX.XXX.241.181:46767 (size: 1221.0 B, free: 9.5 GB)
17/11/16 09:39:50 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on XXX.XXX.241.183:35955 (size: 1221.0 B, free: 9.5 GB)
17/11/16 09:39:50 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on XXX.XXX.241.182:45675 (size: 1221.0 B, free: 9.5 GB)
17/11/16 09:39:50 INFO TaskSetManager: Finished task 1.0 in stage 2.0 (TID 72) in 44 ms on XXX.XXX.241.181 (executor 1) (1/6)
17/11/16 09:39:50 INFO TaskSetManager: Finished task 2.0 in stage 2.0 (TID 68) in 47 ms on XXX.XXX.241.186 (executor 2) (2/6)
17/11/16 09:39:50 INFO TaskSetManager: Finished task 0.0 in stage 2.0 (TID 70) in 47 ms on XXX.XXX.241.183 (executor 0) (3/6)
17/11/16 09:39:50 INFO TaskSetManager: Finished task 3.0 in stage 2.0 (TID 69) in 49 ms on XXX.XXX.241.182 (executor 3) (4/6)
17/11/16 09:39:51 INFO TaskSetManager: Finished task 5.0 in stage 2.0 (TID 67) in 137 ms on XXX.XXX.241.185 (executor 5) (5/6)
17/11/16 09:39:51 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on XXX.XXX.241.184:40304 (size: 1221.0 B, free: 9.5 GB)
17/11/16 09:39:51 INFO TaskSetManager: Finished task 4.0 in stage 2.0 (TID 71) in 299 ms on XXX.XXX.241.184 (executor 4) (6/6)
17/11/16 09:39:51 INFO TaskSchedulerImpl: Removed TaskSet 2.0, whose tasks have all completed, from pool
17/11/16 09:39:51 INFO DAGScheduler: ResultStage 2 (foreach at InternalBackendUtils.scala:175) finished in 0.303 s
17/11/16 09:39:51 INFO DAGScheduler: Job 2 finished: foreach at InternalBackendUtils.scala:175, took 0.311864 s
17/11/16 09:39:51 INFO InternalH2OBackend: Starting H2O client on the Spark Driver (<IP_ADDRESS>): -name sparkling-water-user_app-20171116093826-0333 -nthreads 60 -ga_opt_out -quiet -log_level WARN -log_dir /Users/xxx/xxx/h2ologs/app-20171116093826-0333 -baseport 54321 -client -ip <IP_ADDRESS> -flatfile /var/folders/gl/vgw262w9227cwqvzk595rbvjygdzh8/T/1510825191226-0/flatfile.txt
17/11/16 09:39:51 INFO NativeLibrary: Loaded XGBoost library from lib/osx_64/libxgboost4j.dylib (/var/folders/gl/vgw262w9227cwqvzk595rbvjygdzh8/T/libxgboost4j8313698606636459355.dylib)
Found XGBoost backend with library: xgboost4j
Your system supports only minimal version of XGBoost (no GPUs, no multithreading)!
IP address not found on this machine
17/11/16 09:39:52 INFO SparkContext: Invoking stop() from shutdown hook
17/11/16 09:39:52 INFO SparkUI: Stopped Spark web UI at http://<IP_ADDRESS>:4041
17/11/16 09:39:52 INFO StandaloneSchedulerBackend: Shutting down all executors
17/11/16 09:39:52 INFO CoarseGrainedSchedulerBackend$DriverEndpoint: Asking each executor to shut down
17/11/16 09:39:52 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/11/16 09:39:52 INFO MemoryStore: MemoryStore cleared
17/11/16 09:39:52 INFO BlockManager: BlockManager stopped
17/11/16 09:39:52 INFO BlockManagerMaster: BlockManagerMaster stopped
17/11/16 09:39:52 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/11/16 09:39:52 INFO SparkContext: Successfully stopped SparkContext
17/11/16 09:39:52 INFO ShutdownHookManager: Shutdown hook called
17/11/16 09:39:52 INFO ShutdownHookManager: Deleting directory /private/var/folders/gl/vgw262w9227cwqvzk595rbvjygdzh8/T/spark-d6322370-0361-4d33-ab2d-9ea2155fae24
Error: Unexpected state in sparklyr backend, terminating connection: failed to invoke spark command (unknown reason)
In addition: Warning message:
In file(con, "r") :
  cannot open file '/var/folders/gl/vgw262w9227cwqvzk595rbvjygdzh8/T//RtmpklcGmh/file7d53470c36f_spark.log': No such file or directory

The issue seems to be with IP address not found on this machine - is this a firewall issue? Why would it be intermittent?

It seems like it could be a communication issue the between the spark master / executors / driver when starting the h2o context. But this is strange as I can run regular spark jobs on this cluster without any communication issues.

OK, after days of debugging, I've figured out that that IP address not found was arising because the h2o InetAddress.isReachable call was returning false with the default timeout on my system. I managed to get it to return true by manually setting the timeout to a higher value in scala:
System.setProperty("sys.ai.h2o.network.ip.ping.timeout", "6000")
And it works in scala! However, I now need to set the same value for sparkling water in R - any ideas on the syntax to use in the config file for use be sparkling water / h2o?


changes to fields validation in Edit / Publish

Fixes #1244 .

Changes proposed in this PR:

changes to Yup validation on edit form
changes to Yup validation on publish form

Validation Based on Schema
WIP:

[X] make sure edit and publish validation work the same way (right now there's a BUG on the Edit Form, it's not validating the Files field)

Redundant as differently done in #1685


[feat] Add support for proxy on AWS

Will run squid as proxy using https://www.redhat.com/sysadmin/quadlet-podman

https://gitlab.com/CentOS/cloud/sagano-examples/-/tree/main/autonomous-podman-hello?ref_type=heads


fix: Refactoring the code of the Upgrade flow for OIDC and SAML

Description

Refactoring the code of the Upgrade flow for OIDC and SAML to fix the issue of intercom popup not opening on click of upgrade button.

Fixes #20241
Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Tested the function call locally and it is now getting triggered.

Manual

Checklist:
Dev activity

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[ ] PR is being merged under a feature flag

QA activity:

[ ] Test plan has been approved by relevant developers
[ ] Test plan has been peer reviewed by QA
[ ] Cypress test cases have been added and approved by either SDET or manual QA
[ ] Organized project review call with relevant stakeholders after Round 1/2 of QA
[ ] Added Test Plan Approved label after reveiwing all Cypress test

/ok-to-test sha=a1cdf49


feat: impl GetContractEnv of cosmwasmAPI

Description

Closes https://github.com/line/cosmwasm/issues/144, https://github.com/line/cosmwasm/pull/151, https://github.com/line/wasmvm/pull/53
It's a poc implementation.
Gas consumption, security checks, unit tests, CI, etc. need to be checked.
I will change from a draft PR to a full PR once production work is complete.

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

[ ] Targeted PR against correct branch (see CONTRIBUTING.md)
[ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
[ ] Code follows the module structure standards.
[ ] Wrote unit and integration tests
[ ] Updated relevant documentation (docs/) or specification (x/<module>/spec/)
[ ] Added relevant godoc comments.
[ ] Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
[ ] Re-reviewed Files changed in the Github PR explorer
[ ] Review Codecov Report in the comment section below once CI passes

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: brew0722:x: Woosang SonWoosang Son seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

I changed the target branch and registered it as a new PR.
https://github.com/line/lbm-sdk/pull/427


Jekyl Install on Mac (sudo -E)

Took me a while to overcome this, so might be worth including in docs.
'gem install' on my mac requires sudo.   Sudo doesn't preserve env variables by default.
sudo -E gem install bundler jekyl
did the trick...sigh.
At least this will be searchable in the issues list for the next naive mac user.

Updated in the doc here.  Thanks for the suggestion!


adjust the source url for the registry server

What this PR does / why we need it:
After the creation of the new Kubevirt cluster, the images registry address has been changed. This PR adds the new images server address to the tests templates for Kubevirt cloud provider.
Optional Release Note:
None

/retest

/retest

/retest


john: build with recent gcc

Motivation for this change
john the ripper contains a patch for gcc >= 5 and builds fine. I am not sure why this was pinned in the first place.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Fits CONTRIBUTING.md.

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: john
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: john
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Failure on aarch64-linux (full log)
Attempted: john
Partial log (click to expand)
 extern size_t strlen (const char *__s)
               ^~~~~~
truecrypt_fmt_plug.c:314:95: error: 'num_iterations' undeclared (first use in this function)
    pbkdf2_sha512((const unsigned char*)key_buffer[i], strlen(key_buffer[i]), psalt->salt, 64, num_iterations, key, sizeof(key), 0);
                                                                                ^~~~~~~~~~~~~~
make[1]: *** [Makefile:520: truecrypt_fmt_plug.o] Error 1
make[1]: Leaving directory '/build/john-1.8.0-jumbo-1/src'
make: *** [Makefile:185: default] Error 2
builder for '/nix/store/h4w2nw75xh967jsq8pi7x3hhbmla5aml-john-1.8.0-jumbo-1.drv' failed with exit code 2
error: build of '/nix/store/h4w2nw75xh967jsq8pi7x3hhbmla5aml-john-1.8.0-jumbo-1.drv' failed

Failure on aarch64-linux (full log)
Attempted: john
Partial log (click to expand)
 extern size_t strlen (const char *__s)
               ^~~~~~
truecrypt_fmt_plug.c:314:95: error: 'num_iterations' undeclared (first use in this function)
    pbkdf2_sha512((const unsigned char*)key_buffer[i], strlen(key_buffer[i]), psalt->salt, 64, num_iterations, key, sizeof(key), 0);
                                                                                ^~~~~~~~~~~~~~
make[1]: *** [Makefile:520: truecrypt_fmt_plug.o] Error 1
make[1]: Leaving directory '/build/john-1.8.0-jumbo-1/src'
make: *** [Makefile:185: default] Error 2
builder for '/nix/store/h4w2nw75xh967jsq8pi7x3hhbmla5aml-john-1.8.0-jumbo-1.drv' failed with exit code 2
error: build of '/nix/store/h4w2nw75xh967jsq8pi7x3hhbmla5aml-john-1.8.0-jumbo-1.drv' failed

Success on x86_64-linux (full log)
Attempted: john
Partial log (click to expand)
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/dmg2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/sipdump2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/kwallet2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/keystore2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/openssl2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/7z2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/cracf2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/office2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/htdigest2john.py'...
/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1

Success on x86_64-linux (full log)
Attempted: john
Partial log (click to expand)
/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1

aarch64 failure is unrelated, I couldn't find any successful hydra build on aarch64 (looking back to March 2017). So let's exclude aarch64 for now.

@GrahamcOfBorg build john

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: john
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Attempted: john
Partial log (click to expand)
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/kwallet2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/ecryptfs2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/encfs2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/cracf2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/efs2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/pcap2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/ml2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/netscreen.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/openssl2john.py'...
/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1

Failure on aarch64-linux (full log)
Attempted: john
Partial log (click to expand)
 extern size_t strlen (const char *__s)
               ^~~~~~
truecrypt_fmt_plug.c:314:95: error: 'num_iterations' undeclared (first use in this function)
    pbkdf2_sha512((const unsigned char*)key_buffer[i], strlen(key_buffer[i]), psalt->salt, 64, num_iterations, key, sizeof(key), 0);
                                                                                ^~~~~~~~~~~~~~
make[1]: *** [Makefile:520: truecrypt_fmt_plug.o] Error 1
make[1]: Leaving directory '/build/john-1.8.0-jumbo-1/src'
make: *** [Makefile:185: default] Error 2
builder for '/nix/store/h4w2nw75xh967jsq8pi7x3hhbmla5aml-john-1.8.0-jumbo-1.drv' failed with exit code 2
error: build of '/nix/store/h4w2nw75xh967jsq8pi7x3hhbmla5aml-john-1.8.0-jumbo-1.drv' failed

why did it try aarch64 again?

Failure on aarch64-linux (full log)
Attempted: john
Partial log (click to expand)
 extern size_t strlen (const char *__s)
               ^~~~~~
truecrypt_fmt_plug.c:314:95: error: 'num_iterations' undeclared (first use in this function)
    pbkdf2_sha512((const unsigned char*)key_buffer[i], strlen(key_buffer[i]), psalt->salt, 64, num_iterations, key, sizeof(key), 0);
                                                                                ^~~~~~~~~~~~~~
make[1]: *** [Makefile:520: truecrypt_fmt_plug.o] Error 1
make[1]: Leaving directory '/build/john-1.8.0-jumbo-1/src'
make: *** [Makefile:185: default] Error 2
builder for '/nix/store/h4w2nw75xh967jsq8pi7x3hhbmla5aml-john-1.8.0-jumbo-1.drv' failed with exit code 2
error: build of '/nix/store/h4w2nw75xh967jsq8pi7x3hhbmla5aml-john-1.8.0-jumbo-1.drv' failed

@GrahamcOfBorg build john

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: john
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: john
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Attempted: john
Partial log (click to expand)
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/openssl2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/office2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/blockchain2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/sxc2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/lotus2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/cracf2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/strip2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/7z2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/openbsd_softraid2john.py'...
/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: john
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: john
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Attempted: john
Partial log (click to expand)
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/mcafee_epo2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/1password2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/efs2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/cracf2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/strip2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/ecryptfs2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/odf2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/openbsd_softraid2john.py'...
wrapping `/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1/bin/htdigest2john.py'...
/nix/store/bh75rbc1i4r53inpryqlp0svjp7vy3ib-john-1.8.0-jumbo-1


Skip some validation checks when BUNDLE group value is found

Resolves #2621

Thank you for the PR!
I think it's a step in the right direction. But I think we should not enforce exactly one fingerprint and icefrag, but at least one of each (and they don't need to be equal). Because that is what the Bundle RFC says essentially "ignore all the other m-sections".

I apologize, I mis-read the PR. Please ignore my previous comment.
This PR tries to enforce a single fingerprint and ice ufrag for non-bundled connections. If Pion will ever support non-bundled connections this would cause problems.


Performance Problem on a PC Laptop

Someone I know with a PC laptop is having a performance problem with this class.  Here's the code I'm using:
Set json = JsonConverter.ParseJson(response)
where response is in the attached file.  This guy says parsing the attached file is taking over 30 seconds.  It takes less than a second on my desktop computer though.  Does anyone have any ideas on what the issue could be?
test.txt

A recent Windows update caused a big slowdown, so that is most likely the issue. It was fixed very recently in v2.3.0, so try updating to that version. If that doesn’t work, please feel free to reopen the issue.


Confused over inlined dependencies

Hi scalameta folk,
Looking at https://repo1.maven.org/maven2/org/scalameta/semanticdb-scalac_2.13.8/4.7.7/ the jar file contains versions of Google's Protobuf, JLine, scalapb, sourcecode, and RelaxNG in the root of the jar.
Shouldn't this cause a user concern that the jar file is going to cause dependency problems when used?

Thanks for the question! It's a compiler plugin, so it needs to have everything included in a single jar. You should use https://repo1.maven.org/maven2/org/scalameta/semanticdb-scalac-core_2.13.8/4.7.7/ as a dependency


[CWS] Allow computing the difference between 2 activity dumps

What does this PR do?
security-agent runtime activity-dump diff --source /tmp/mydump.protobuf  --target /tmp/mydump.protobuf  --format dot > ...

Motivation

Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

Reviewer's Checklist

[ ] If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
[ ] Use the major_change label if your change either has a major impact on the code base, is impacting multiple teams or is changing important well-established internals of the Agent. This label will be use during QA to make sure each team pay extra attention to the changed behavior. For any customer facing change use a releasenote.
[ ] A release note has been added or the changelog/no-changelog label has been applied.
[ ] Changed code has automated tests for its functionality.
[ ] Adequate QA/testing plan information is provided if the qa/skip-qa label is not applied.
[ ] At least one team/.. label has been applied, indicating the team(s) that should QA this change.
[ ] If applicable, docs team has been notified or an issue has been opened on the documentation repo.
[ ] If applicable, the need-change/operator and need-change/helm labels have been applied.
[ ] If applicable, the k8s/<min-version> label, indicating the lowest Kubernetes version compatible with this feature.
[ ] If applicable, the config template has been updated.

Works fine with dot as output format to identify diffs:

For json is more a merge than a diff, but could be usefull too :)
LGTM


Support SPIFFE Server Side

Welcome!

[X] Yes, I've searched similar issues on GitHub and didn't find any.
[X] Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you expect to see?
Context and Feature Description
This is a follow up of #9376 (SPIFFE between backend servers and Traefik). I would like to propose to allow Traefik to serve its x509-SVID server side.
The use case I have in mind for this is with SPIRE federation in a multi-cluster setup over a public network. This would allow having a workload running in the first cluster establishing a secure mTLS connection with a Traefik instance running in the other cluster seamlessly, but I believe there could be many more usages to this, maybe also Maesh could benefit from something like this?
The alternative would be to either bypass Traefik or do a TLS passthrough to establish the mTLS connection with a spiffe aware backend server, which could also work.
How to implement this Feature
We could imagine three use cases here:

Traefik serving its x509-SVID but not requiring any client cert.
Traefik serving a non SPIFFE certificate, but requiring a valid x509-SVID from the client
Traefik serving its x509-SVID and requiring an x509-SVID from the client (mTLS)

I would tend to think that the latter case (mTLS) should be the default case, but still maintain the possibility of turning off either serving Traefik SVID or validating a client SVID.
Also, this certificate setup should be scoped per Router, so I would go for adding a spiffe section under the TLSOptions struct.
tls:
  options:
    someOption:
      spiffe:
        servesSVID: true # Tells traefik to serve its x509-SVID (default true)
        validateClientCert: true # Tells traefik to validate the client cert using the SPIFFE CA (default true).
        ids:  # List of allowed client SPIFFE IDs (optional)
           - spiffe://trustdomain.com/some/spiffeid
        trustDomain: trustdomain.com # Trusted domain (optional)

The configuration could look like this in a k8s CRD, where I want to enable SPIFFE mTLS on a router for a single caller.
---
apiVersion: traefik.containo.us/v1alpha1
kind: TLSOption
metadata:
  name:  echo-gateway-internal
  namespace: echo-gateway
spec:
  spiffe: # Turns on SPIFFE for this router, by default is is mTLS.
    ids:   # List of allowed client SPIFFE IDs
      - "spiffe://demo.voiapp.io/ns/traefik-external/sa/traefik-external"
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: echo-gateway-internal
  namespace: echo-gateway
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: spiffe
    traefik.ingress.kubernetes.io/router.tls.options: echo-gateway-echo-gateway-internal@kubernetescrd
spec:
  ingressClassName: traefik-internal
  rules:
  - host: echo-internal.localhost
    http:
      paths:
      - backend:
          service:
            name: echo-gateway
            port:
              name: https
        path: /echo
        pathType: Prefix

Proof of concept and code changes
I've extended my previous PoC to test this case, it sets up a Traefik instance behind a Traefik instance and have them establish a mTLS connection  between them using SPIFFE. This use case doesn't make any sense in real life.

PoC runtime where you can find all the configuration.
Code Changes

Let me know what do you think about this.

Hey @jlevesy, we never got back to you on this (sorry 🙏 ), but it looks like a great idea :)

Hey. I'll see what I can do then.


Simplify examples in README.

Simplify the examples in the README
Make some other minor changes to the README

This PR affects the readme only. Merging.


batch inference is as slow as single image inference in tensorflow c++

OS:Ubuntu 16.04
version:Tensorflow c++ 2.0-beta1 (compiled with all optimization flag:AVX AVX2 SSE4.1 SSE4.2 FMA XLA)
IDE:eclipse
With CUDA:No (just CPU in my prediction)
I have test the time of single image inference with tensorflow c++ api is 0.02 seconds which is so long that i just can not believe with my own eyes because i have compiled tensorflow c++ shared library with all the optimizations such as AVX/AVX2/FMA/SSE4.1/SSE4.2/FMA. However,i have to find the solution to decrease the cost time in prediction.Someone tells me the time can hugely decrease if i use batch inference instead of single image inference.Unfortunately,the time is 0.7 seconds in batch inference when the batch size is 32.In another word,0.7/32=0.02,it is as slow as single image inference.
This issue is related to tensorflow c++ performance so i ask it here and any help will be much appreciated.I have also put detailed information here stackoverflow .

Is this still an issue? Can you check with TF2.0 and let us know whether the issue persists with latest TF version. Thanks!

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?

Closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

I meet the same problem, I used tensoflow-gpu = 1.10.0, 1.12.0 and 1.14.0, the time of inference with batch_size = 1 is 80ms, and time of inference with batch_size=2 is 155ms. So in my opinion，batch inference is so fast compared to single image inference,Is that right?


BROKERS: HttpBroker SendRequestAsync

Closes: #30

Why does it need to be partial and should HttpBroker be renamed?


Add "fish" operators

Based on https://github.com/isovector/polysemy/pull/24.

After I add some example with interpreter, it should be ready for merging :smile:

Fantastic, thanks!!!


docker file created

Remove the commits with Merge branch 'main' of github.com:aamritbistaa/2023-summer to be approved.


Port Thai2Rom from Keras to PyTorch

See issue: #202
This is the pytorch version of Thai2Rom (seq2seq no attention mechanism).
there is still room for improvement, but you guys can test it.

Hello @c4n! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file pythainlp/transliterate/thai2rom.py:

Line 18:12: W291 trailing whitespace
Line 19:9: E265 block comment should start with '# '
Line 24:46: W291 trailing whitespace
Line 26:23: E261 at least two spaces before inline comment
Line 26:24: E262 inline comment should start with '# '
Line 26:54: W291 trailing whitespace
Line 27:23: E261 at least two spaces before inline comment
Line 27:24: E262 inline comment should start with '# '
Line 28:27: E261 at least two spaces before inline comment
Line 28:28: E262 inline comment should start with '# '
Line 30:25: E225 missing whitespace around operator
Line 31:25: E225 missing whitespace around operator
Line 32:32: E225 missing whitespace around operator
Line 33:32: E225 missing whitespace around operator
Line 34:9: E265 block comment should start with '# '
Line 36:80: E501 line too long (91 > 79 characters)
Line 38:68: E231 missing whitespace after ','
Line 38:80: E501 line too long (104 > 79 characters)
Line 40:1: W293 blank line contains whitespace
Line 41:34: E231 missing whitespace after ','
Line 44:12: W291 trailing whitespace
Line 55:1: W293 blank line contains whitespace
Line 56:5: E303 too many blank lines (2)
Line 56:22: E231 missing whitespace after ','
Line 60:12: W291 trailing whitespace
Line 64:80: E501 line too long (89 > 79 characters)
Line 65:54: E231 missing whitespace after ','
Line 65:56: E231 missing whitespace after ','
Line 65:80: E501 line too long (138 > 79 characters)
Line 65:82: E231 missing whitespace after ','
Line 65:110: E231 missing whitespace after ','
Line 65:112: E231 missing whitespace after ','
Line 67:25: E261 at least two spaces before inline comment
Line 67:26: E262 inline comment should start with '# '
Line 70:43: E231 missing whitespace after ','
Line 70:80: E501 line too long (108 > 79 characters)
Line 85:1: E303 too many blank lines (4)
Line 90:73: E251 unexpected spaces around keyword / parameter equals
Line 90:75: E251 unexpected spaces around keyword / parameter equals
Line 90:80: E501 line too long (80 > 79 characters)
Line 95:80: E501 line too long (89 > 79 characters)
Line 100:55: E231 missing whitespace after ','
Line 100:80: E501 line too long (86 > 79 characters)
Line 101:5: E128 continuation line under-indented for visual indent
Line 101:43: E231 missing whitespace after ','
Line 103:1: E302 expected 2 blank lines, found 1
Line 112:1: W293 blank line contains whitespace
Line 113:5: E303 too many blank lines (2)
Line 113:33: E231 missing whitespace after ','
Line 113:40: E231 missing whitespace after ','
Line 114:1: W293 blank line contains whitespace
Line 116:26: W291 trailing whitespace
Line 120:34: E261 at least two spaces before inline comment
Line 120:34: E262 inline comment should start with '# '
Line 120:80: E501 line too long (92 > 79 characters)
Line 120:93: W291 trailing whitespace
Line 124:50: E203 whitespace before ','
Line 124:51: E231 missing whitespace after ','
Line 124:80: E501 line too long (82 > 79 characters)
Line 125:5: E128 continuation line under-indented for visual indent
Line 125:38: E231 missing whitespace after ','
Line 127:1: E305 expected 2 blank lines after class or function definition, found 1
Line 130:39: W291 trailing whitespace
Line 131:1: W293 blank line contains whitespace
Line 132:1: W293 blank line contains whitespace
Line 132:1: W391 blank line at end of file


Hope that system call of uname get removed

Dash.vim would add nearly 15ms  to my vim start time, it's my second slowest plugin.
When I get the unname check removed,  the loading time would be less than 1ms.
I think it's not necessary to avoid loading the script by check the uname, it should be fine if the script get loaded but could not works.
This check slows down vim instead of make vim faster.

Removed.


[CodeCompletion] Add XFAIL to swift-ide-test

In -batch-code-completion mode, add a token parameter xfail={reason}. When 'FileCheck' succeeds on the token, it is considered "unexpected pass", and the test fails.
rdar://problem/71021285

@swift-ci Please smoke test

@swift-ci Please smoke test


Two versions of Control Center

There is a threat that users in the future might get confused and scared with all those terminals and code lines.
Therefore, there is way to make two versions of Control Central:

version for the user with no-technical background (all the processes would be done only on the background with no terminals open.
version for the users with technical background.
So, is there a need for us to implement it?

I would not make this a two version thing but make it modal. You can hide things in the background and have a setting to control that. By default things can be hidden to keep things simple and not confuse users. The option can be there to enable it in the foreground where users can see them like the peer building terminal etc.

Agreed, one version. I would so that the default state (the "simple" version) shows steps in the process, progress bars, countdown timer if possible, so that user knows that things are moving forward, is informed of progress, does not panic if it takes too long. Then an "advanced" control would expand the dialog to show more info - not sure if this is possible, but is there something like a terminal viewer where they could watch the action scroll past, and scroll back up to check things if needed?

@Erkin97 what do you think?

I've sketched out that idea. It's in the updated Axure link I just sent to you on Slack, Alex. (The second link, for some reason.)

Discussion is closed.

Some comments from Slack:
Also this Components thing should change where it is modal: (1) current implementation is advanced mode, (2) default simple mode should show capabilities that you can install to have groups of things like (web client, desktop client, peer management). Checking off one will install the set of things needed for it. This will be much easier for users.


Remove already generated files if installation fails halfway through

Maybe it should depend on how the installation fails? Not sure yet about this one.

Won't fix


Converting from poetry.lock to requirements.txt provides weird results

One of my workflows that I'm hoping to improve with dephell is generating a requirements.txt file from my poetry.lock which includes custom warehouse (with auth).  This makes it much easier to set up docker images (copy over this requirements.txt so I don't need to install/configure dephell or poetry in the image).
I gave this command a shot using my existing poetry.lock (Poetry made this, not dephell) and it gave me some bizarre results.
Config:
[tool.dephell.docker]
from = "poetry.lock"
to = "requirements.txt"

[[tool.poetry.source]]
name = "custom"
url = "https://my.custom.domain/simple/"

poetry.lock (sample)
[[package]]
category = "main"
description = "A database migration tool for SQLAlchemy."
name = "alembic"
optional = false
python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
version = "1.0.10"

[package.dependencies]
Mako = "*"
SQLAlchemy = ">=0.9.0"
python-dateutil = "*"
python-editor = ">=0.3"

[package.source]
reference = "custom"
type = "legacy"
url = "https://my.custom.domain/simple"

$ dephell deps convert --env docker
INFO converted

Output requirements.txt (sample)
git+https::/my.custom.domain/simple#egg=alembic; python_version != "3.0.*" and python_version != "3.1.*" and python_version != "3.2.*" and python_version != "3.3.*" and python_version >= "2.7"

Note a few issues here:

It's trying to use git for some reason?
It has "https::/" instead of "https://"
It's not providing my auth anywhere, so this requirements.txt will fail with pip
The package is not pinned properly from the lock file.  Shoudl show == 1.0.10 somewhere

Works fine with 0.7.4.  More PEBKAC


tpb-ultra-lite

Hi again,
This PR slims down tpb-lite even more, by removing the extrenuous purl dependency and replacing beautifulsoup4 with the lighter lxml.
I think this is an improvement, let me know what you think.

Hey there,
Thank you for your continued work!
I'm not against switching to lxml but I'm not overly familiar with it and how it compares to beautifulsoup4. Looking through the commit, the expressions used to filter the necessary attributes seem a little more complicated.
Why do you think we should switch to lxml? Why would you consider it lighter?
Thanks
Matt

Yeah the expressions do need a bit of getting used to, but on the other hand they are standard xpath expressions so there is a lot of examples and documentation online, and it's a good skill to learn for webscraping.
Well I think the main advantage of lxml is performence as it uses some C libraries to do the parsing/querying. Also, beautifulsoup4 is designed to parse just about any badly or weirdly formed html found in the wild west of the internet, with tpb-lite we are sticking to basically a single website which lxml doesn't seem to have any problems parsing. So I don't think beautifulsoup's leniency is required.
Here is an interesting discussion comparing the two: https://stackoverflow.com/questions/4967103/beautifulsoup-and-lxml-html-what-to-prefer#4967121
They know much than me about both lxml and beautifulsoup4 and I encourage you to read the discussion, but I think the main takeaway is that they recommend lxml unless the html is malformed and lxml can't parse it.
But I understand if you want to stick with beautifulsoup4, it's a matter of preference.

Yeah I mean I'm happy to switch, I don't hold much emotional attachment to BeautifulSoup and given the performance boost it seems like a good choice.
Before I merge the request though I will be writing some unittests so that further development can become more streamline.


Support Resource Manager API

Let's add support for the Resource Manager API to gcloud. I'm pretty excited about this one!
What is Resource Manager?
Resource Manager API

Given that Resource Manager is a lengthy term... and the concept is really just "projects"... and a project is so close to the core concept of using a gcp client library... I'm wondering if this is the first thing we should put directly onto the gcloud instance:
var gcloud = require('gcloud');

gcloud.createProject('new-project-123', function(err, project) {});

gcloud.getProjects(function(err, projects) {});

var project = gcloud.project('grape-spaceship-123');

project.delete(function(err) {});
project.restore(function(err) {});

project.getMetadata(function(err, metadata) {});
project.setMetadata({/*...*/}, function(err) {});

// @jgeewax @callmehiphop thoughts?

Is it likely that the resource manager api will ever do more than just projects?

It looks like there are Organizations as well (https://developers.google.com/apis-explorer/?hl=en_US#p/cloudresourcemanager/v1beta1/)
Maybe it's best to just keep resources:
var gcloud = require('gcloud');
var resources = gcloud.resources({...});

resources.getProjects();
var project = resources.project('grape-spaceship-123');

I like that, projects.getProjects(...) read kinda funny to me.

Vote!
// a
var resources = gcloud.resources();
resources.getProjects();
resources.project('grape-spaceship-123');

// b
var resourceManager = gcloud.resourcemanager();
resourceManager.getProjects();
resources.project('grape-spaceship-123');

// c
var resource = gcloud.resource();
resource.getProjects();
resource.project('grape-spaceship-123');

// d
??

I like c.
a sounds kinda funny to me with back to back plural nouns
b is maybe a little too lengthy for my tastes

C it is!
@jgeewax My project (that has resource manager API enabled) can list my projects and get their metadata. However, I get a 403 when I try to create one:
REQUEST
  method: 'POST'
  uri: 'https://cloudresourcemanager.googleapis.com/v1beta1/projects/'
  json: { projectId: 'gcloud-tests-1441915137564' }
  headers
   { Authorization: 'Bearer /*token*/',
     'User-Agent': 'gcloud-node/0.21.0' }

RESPONSE
  vary: 'X-Origin, Referer, Origin,Accept-Encoding'
  'content-type': 'application/json; charset=UTF-8'
  date: 'Thu, 10 Sep 2015 20:02:52 GMT'
  server: 'ESF'
  'cache-control': 'private'
  'x-xss-protection': '1; mode=block'
  'x-frame-options': 'SAMEORIGIN'
  'x-content-type-options': 'nosniff'
  'alternate-protocol': '443:quic,p=1'
  'alt-svc': 'quic=":443"; p="1"; ma=604800'
  'accept-ranges': 'none'
  connection: 'close'

I'm not sure how to auth this one; so far, gcloud lets users control one project's sub-services using a service account for it. What do we need to get to have control over all of their projects?

Yea, creating right now can only be done by a "person". In Python, this works by first running gcloud auth login on the command-line.
Does gcloud-node pick up the Cloud SDK's credentials in the same way? If so, these will work nicely.
Eventually, service accounts will be able to create projects, but that isn't the case yet.

Replied over at #864

Is it possible falling back to the gcloud sdk token doesn't work here because it doesn't request the scopes needed for this API? (nothing about Logging or the all encompassing auth/cloud-platform scope).


Add ability to process directories

Right now rflint only accepts filenames on the command line. It should accept directories, too, and process all robot files in that folder.

Is it possible to pass a directory to rflint and to process the files with '.resource' extension too? Or should the resource files be passed one by one?


Bug: width problem in AppSideBar CSS causing layout issues

Description:
There is an issue with the AppSideBar component where the width is incorrectly set in the CSS, leading to layout problems.

Suggested Fix:
I have corrected this issue in a pull request. The CSS width properties in  AppSideBar.js have been adjusted to match the intended design specifications.
Related Pull Request:
#4030

Thanks @mouad-dadda. Please follow the contributing guidelines and do not submit a PR until the issue has been triaged and assigned.

It looks like the styling regressed as it's no longer the same color. Resizing the app sidebar is not the fix.

@mouad-dadda I removed the needs triage label. As Nick said, it's a styling regression. If you do want to make the update, go ahead and assign yourself the issue.

.take

@mouad-dadda if you look back a few release or do a git bisect, you should be able to figure out when the styling broke and put it back to what it was.

@mouad-dadda if you look back a few release or do a git bisect, you should be able to figure out when the styling broke and put it back to what it was.

Thank you, @nickytonline, for the guidance. I’ll review previous releases and perform a git bisect to identify when the styling regression occurred .


Milestone 1 Tracker - Hello World Triangle

[x] All enums, structs, classes, and functions have some form of implementation
[x] Various API improvements

new MTLDevice(MTLDevice.MTLCreateSystemDefaultDevice()); -> MTLDevice.CreateSystemDefaultDevice();

[x] Fix MTLSize crash
[x] Remove unnecessary Get...() methods that are covered by property getters
[x] Fix inheritance
[x] Build up other foundational boilerplate to get a window open to render in
[ ] Fix some mismatched method selectors
[ ] Fix NSError-centric APIs
[ ] Easy NSString <-> String interop
[ ] Easy NSArray indexing

The first major milestone has been hit. Clear color is rendering.

Hello World Triangle achieved!


WIP: Review of smart contracts

General

[ ] Tests: move constants to a central file (e.g. btcHash)

OptionPairFactory

[x] Have a test file for the OptionPairFactory to test generating different option pairs -> what happens if the decimals() implementation is not an uint8 as in USDT?
[x] Add the collateral address to the CreatePair event
[ ] windowSize need to find a good minimum depending on blockchain. Suggestion: 24 hours for BTC

European

[x] expiryTime and windowSize not 0 check: move to European contract.

Obligation

[ ] remove poolSupply mapping. This way the user would have to withdraw collateral across multiple pools if they exist. However, we can handle that more simply by just taking the difference of total locked collateral vs total obligations without accounting for the exact pool balances. As long as there is an equal amount of collateral as obligations, we don't need to do special accounting for all the possible pools. That has an impact on the withdraw function and transfer function.

OptionLib

[x] Add a comment that amountA is both the amount of option tokens and the amount of collateral
[ ] tokenB and collateral should be the same?
[ ] Replace manual transfer and mint in lockAndWrite with addLiquidity to minimize slippage when adding liquidity: https://uniswap.org/docs/v2/smart-contracts/router02/#addliquidity
[ ] Replace manual transfer and swap in lockAndBuy by swapExactTokensForTokens to reduce slippage: https://uniswap.org/docs/v2/smart-contracts/router02/#addliquidity

Merging, leftover comments added as tickets.


Fixes #7 - Case Statements

TODO: add a type computation since case statements should resolve to a type per @Lotes review from prior (now deleted) PR

We have an UNKNOWN type if nothing clean comes out of the type computation. Do you need help with something?


Console error on missing optional styles.css

styles.css is optional in plugins, isn't it? Console is producing error when it's not found...
error in grabReleaseFileFromRepository https://github.com/codyburleson/litmojo/releases/download/0.0.4/styles.css Error: Request failed, status 404
    at new t (app.js:1:699319)
    at Xb (app.js:1:699511)
    at app.js:1:700188
    at app.js:1:235201
    at Object.next (app.js:1:235306)
    at a (app.js:1:234045)

This is not a big deal because it still seems to work, but maybe you want to know anyway?

Thank you for reporting this. In fact this is by design. BRAT is mostly designed for testing. So its alerting the dev that a core file is missing. It is true, Obsidian ignores this if the file is missing, but it still look for it.
I include even and empty styles.css in every release just to show that this component has been thought through.
If its getting annoying though, let me know, we could create a toggle for the feature.

This should be a toggle that is by default disabled. Considering its optional nature, having a hard-coded dependency simply causes false positives to arise when using this plugin. Likewise, it also pollutes the console log with unnecessary error messages.


gamenetworkingsockets: 1.3.0 -> 1.4.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/GameNetworkingSockets/versions.
meta.description for gamenetworkingsockets is: "GameNetworkingSockets is a basic transport layer for games"
meta.homepage for gamenetworkingsockets is: "https://github.com/ValveSoftware/GameNetworkingSockets"
meta.changelog for gamenetworkingsockets is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 1.4.0 with grep in /nix/store/l557i02hbbmc2nvn057j7hmp83q05hzs-GameNetworkingSockets-1.4.0

found 1.4.0 in filename of file in /nix/store/l557i02hbbmc2nvn057j7hmp83q05hzs-GameNetworkingSockets-1.4.0

Rebuild report (if merged into master) (click to expand)

3 total rebuild path(s)

2 package rebuild(s)

2 x86_64-linux rebuild(s)
1 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
gamenetworkingsockets
soldat-unstable

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/l557i02hbbmc2nvn057j7hmp83q05hzs-GameNetworkingSockets-1.4.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A gamenetworkingsockets https://github.com/r-ryantm/nixpkgs/archive/38ead05dd943894adb77e3cf4ae76fcfb8bea1e1.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/l557i02hbbmc2nvn057j7hmp83q05hzs-GameNetworkingSockets-1.4.0
ls -la /nix/store/l557i02hbbmc2nvn057j7hmp83q05hzs-GameNetworkingSockets-1.4.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package failed to build:

soldat-unstable

1 package built:

gamenetworkingsockets

Maintainer pings
cc @sternenseemann for testing.

@ofborg build soldat-unstable

#179883


Add .tif extension to FileSystemFileResolver input_types

Adds support for .tif input files in Riiif::FileSystemFileResolver. From the Library of Congress TIFF, Revision 6.0:

The TIFF 6.0 standard recommends the use of tif (or TIF) as extension. Macintosh Filetype is TIFF and tiff is sometimes found as a filename extension.

New specs for 'tif', 'tiff', 'jpg' and 'png' inputs in file_system_file_resolver_spec.rb
Fixes #69

Hi @jcoyne I hope that commit addresses your concerns. I totally agree about not wanting to add binaries to specs. Thanks for the feedback!

Ah good catch! I think I was sidetracked looking into the "jp" extension. Haven't seen that before, is that an abbreviation of "jpg" or "jp2"?

@bmquinn I think that .jp extension was a mistake and I've removed that. Would you be so good as to rebase your change, and remove the test for the .jp extension?

Right on, I removed the jp spec and squashed the commits. Hope that does the trick, but let me know if there's anything else to do. Thanks @jcoyne!


Revert PR #4258: "Swagger changes for connectors: Google AdWords, Oracle Service Cloud, OData, Rest Service, Dynamics AX and support for wildcardPath."

Revert PR (#4258): "Swagger changes for connectors: Google AdWords, Oracle Service Cloud, OData, Rest Service, Dynamics AX and support for wildcardPath."
UX is not ready and swagger changes should come after UX. So revert swagger changes.
This reverts commit c7b860297ddf53c1f9ef501583c824fdd00cbc40.
This checklist is used to make sure that common issues in a pull request are addressed. This will expedite the process of getting your pull request merged and avoid extra work on your part to fix issues discovered during the review process.
PR information

[x] The title of the PR is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For information on cleaning up the commits in your pull request, see this page.
[x] Except for special cases involving multiple contributors, the PR is started from a fork of the main repository, not a branch.
[x] If applicable, the PR references the bug/issue that it fixes.
[x] Swagger files are correctly named (e.g. the api-version in the path should match the api-version in the spec).

Quality of Swagger

[x] I have read the contribution guidelines.
[x] My spec meets the review criteria:

[x] The spec conforms to the Swagger 2.0 specification.
[x] The spec follows the guidelines described in the Swagger checklist document.
[x] Validation tools were run on swagger spec(s) and have all been fixed in this PR.

Automation for azure-sdk-for-ruby

Can one of the admins verify this patch?

Automation for azure-sdk-for-python

Automation for azure-sdk-for-node

Automation for azure-sdk-for-go

Automation for azure-sdk-for-js

@msxzq I'm not clear what is the relation between your UX and swagger? Are you talking about Azure Portal? We in general do not let this kind of reverts unless there is some critical issue with it.

@msxzq I'm not clear what is the relation between your UX and swagger? Are you talking about Azure Portal? We in general do not let this kind of reverts unless there is some critical issue with it.

Hi @hovsepm , first of all, sorry for the inconvenience.
UX is the app model of connectors in ADF.
On the one hand, according to the internal developing process, we should first release UX and then swagger. There may be breaking changes if swagger is released before UX is ready and UX changes afterwards. In my previous PR, I violated the order so I want to revert the commit to avoid potential risks.
On the other hand, I haven't generated the sdk for the swagger changes in the previous PR so the rollback won't bring customer impacts.
Could you please help me rollback the commit ? Thanks!


Working set seem to be shared between containers

I use heapster to get info from cadvisor to influxdb. What I've noticed is that memory/working_set_bytes_gauge is approximately the same, no matter which container I choose. But memory/usage_bytes_gauge does seem to represent what cadvisor shows as a "hot" memory. I.e. working set is greater than usage bytes for some containers.
Is there some misunderstanding?

Please check if the problem persists in the new Heapster version candidate gcr.io/google_containers/heapster:v0.20.0-alpha10 and reopen if needed.


Look into Visual studio project templates

Create a new repo for it if needed

Also look into https://www.youtube.com/watch?v=2hpNFrY_faI

Also https://docs.microsoft.com/en-in/dotnet/core/tools/custom-templates


Added error logging to OAuth signin link generation flow.

Fixes #1728
stay tuned for tests
===
Before, if the Emulator failed to spin up an ngrok instance for the OAuth signin flow (due to a misconfigured ngrok path, or some other ngrok related error), it would swallow the error and fall back to generating an emulated OAuth token.
This change will now log any errors from that flow to the log panel, and tell the user that it is falling back to generating an emulated OAuth token.

Coverage decreased (-0.5%) to 64.756% when pulling e03bd3b5928dec40339a509f7e5c4b7c85fe4834 on toanzian/oauth-logging into 508bd9581bbc7aa869bfaade0e4674d878a86476 on master.


Where would you add dummy datasets?

Issue: In order to run certain functions in your repository you need some basic data. It would be beneficial if users had by default a minimal dataset to test functions. this would be a "default" option in the functions.

Why don't we put it under tests/tools//data (e.g. tests/tools/tici/data) and then also implement a (integration)-test that actually uses the dummy data. If we want to use unit and integration tests we could do the same folder pattern (tests/tools/tici/unit and tests/tools/tici/integration). What do you think?😊

uh I actually like this idea a lot @alho94

Sounds really good @jaegglic 😃

I propose to move the vianu/tools/x one level up to vianu/x as there will be no further directory of the level tools.

We stay with the recommendation from @jaegglic for the dummy data and the integration tests
We stay with the recommendation from @alho94 for removing tools folder


Fix websocket protocol violation whitespace in the begining of the response

I can't merge the whole pull request because you put also your personal modifications. I am however correcting the whitespace.


API to restart kafka brokers

Thinking forward about testing Kroxylicious edge cases - network disconnection.
As the author of a component like Kroxylicious, I want to be able to write integration tests that ensure my component correctly handles broker restarts correctly.  To this end I would value an API that permitted me to restart one or more brokers within a cluster.  It is important that the broker comes back on the same ports (as it would in production).  I have this requirement for both the in-VM and container use-case.
I'm imagine a API that looks like:
#restart(int nodeId) => CompletionStage
with a CompletionStage that completes when the broker restart is finished and all the brokers see one and other.
I don't envisaged needing functionality to do things like roll the brokers of a cluster (waiting for ISRs etc).  If a test needs that, it could easily construct it itself.
I don't envisage needing the ability to restart any kafka broker that is solely in the controller role.

@SamBarker @racheljpg @devguyio @franvila @robobario  comments/edits welcome..

all the brokers see one and other.

This is actually tricky to do using the Admin client. You have no control over which brokers get sent the Metadata/DescribeCluster request, nor do you know for sure that the metadata you receive isn't from before the restart. If might be necessary to observe the broker logging to do this robustly, if you're wedded to this semantic.

@tombentley so the approach that @SamBarker and I landed on was to create a single partition topic with RF = <number of brokers> and wait for that to be reported as having replicas on all brokers.   That seems to have stabilized the tests.

@franvila wondering if you'd have capacity to take a look at this?   being able to demonstrate kroxy's robustness will be key to first production use-cases.

I'm picking this feature up.


Spec counter not registering first spec after weapon switch on same enemy

As the title suggests, with today's change to the special attack counter plugin, it appears to sometimes not register the first special attack on the same enemy should it be successful; more specifically, with soloing Corp (wherein I do 2 DWH, then 20 Arclight, then 200 BGS damage), after switching from one weapon to another, the first special attack with that weapon doesn't always get added to the counter (though it sometimes does, I haven't noticed a pattern to this inconsistency). I'm guessing it does this because all the attacks are done consecutively on the same enemy.

I was messing around with this and posted some results on discord, will leave them here as I think this is most likely also what is happening here (interacting changing)
so basically, with the spec counter currently you can change interacting before the special damage is applied
if you interact with nothing, it will fail to track that spec
if you interact with another npc, and hit that npc after, it will track damage you dealt on that npc as spec
so if you hit npc A with spec but click on npc B as you spec (on npc A), then hit npc B normally, it applies that damage as spec
thats the interacting issue

and then the issue where it doesn't count 0 as spec hit and will take the first red hitsplat instead

fixed in 844a85392bdf77993e72e2997d8fe5e3a05ae6a8


Dependency tracking/renovate

We need a dependency tracker, maybe renovate as we did in v1

I created a Fork for onboarding / testing purposes.
From what i understand so far, this will only apply to the helm dependencies defined in cluster-addon, as Docker / Kubernetes is defined through the directory structure deliberately.
I am also unsure if there are relations between the versions of deps in the cluster-addon to other versions and how this all relates to the structure ("providers/openstack/alpha/1-28"). If yes, we should consider pinning ranges in the helm dependencies Chart.yaml.


New version: Microsoft.Azd version 1.4.400

[ ] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] This PR only modifies one (1) manifest
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.5 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

Package version 1.4.3 is manifest version 1.4.400 because Windows Installer packages only support 3 version fields (the 4th is ignored, which is documented). So to pack semver major.minor.patch(-preview) into 3 numbers, we - like many other products - use major.minor.(patch*100 + preview level).


Implement formvalidation repo

Alaska Airlines Pull Request
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Resolves: #210
Summary:
Please summarize the scope of the changes you have submitted, what the intent of the work is and anything that describes the before/after state of the project.
Implement auro-formvalidation repo and update dependencies
Type of change:
Please delete options that are not relevant.

[x] New capability
[x] Revision of an existing capability
[ ] Infrastructure change (automation, etc.)
[ ] Other (please elaborate)

Checklist:

[ ] My update follows the CONTRIBUTING guidelines of this project
[ ] I have performed a self-review of my own update

By submitting this Pull Request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Pull Requests will be evaluated by their quality of update and whether it is consistent with the goals and values of this project. Any submission is to be considered a conversation between the submitter and the maintainers of this project and may require changes to your submission.
Thank you for your submission!
-- Auro Design System Team

@jordanjones243 please rebase this branch to ensure that this update is still compatible due to the large refactor recently released.

Tests will show until failing until the formvalidation PR is released and this datepicker PR is updated with that latest version:

https://github.com/AlaskaAirlines/auro-formvalidation/pull/11


[Bug] CREATE-JOB syntax not supported

Search before asking

[X] I had searched in the issues and found no similar issues.

Version
2.0.3
What's Wrong?
in 2.0 document, it supports the CREATE-JOB syntax, but in practice, it prompts a syntax error when used. error like this:
Failed to execute sql...
Encountered : JOB
https://doris.apache.org/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-JOB
What You Expected?
Is it supported, or is it used correctly, or when can it be supported?
How to Reproduce?
CREATE JOB my_job ON SCHEDULE AT '2020-01-01 00:00:00' DO INSERT INTO db1.tbl1 SELECT * FROM db2.tbl2;
Anything Else?
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

2.0 not support

https://github.com/apache/doris/pull/29136


[Automation] Generate Fluent Lite from scheduler#package-2016-03

[Automation] Generate Fluent Lite from scheduler#package-2016-03

Azure Logic Apps is replacing Azure Scheduler, which is being retired. To continue working with the jobs that you set up in Scheduler, please migrate to Azure Logic Apps as soon as possible.


Fixes to Editor

Updates / fixes to editor so it works properly

Integrated PyLint into the editor. Now any Python syntax errors will be caught and displayed by the editor.
Now the editor is able to detect more than one error at a time. Previously, the editor navigated the AST tree of the code and stopped at the first error it saw (it couldn't navigate past an error). All errors are now shown.
Fixed tooltips not displaying. Now when the user mouses over a gutter marker (on lines with errors), a tooltip will appear with the line number, character number, and error message pertaining to the error.
Fixed Codemirror CSS / JavaScript plugins not loading in properly.

Note: This doesn't pertain to the filter, just the editing experience. Dealing with intermediate variables being used to bypass the filter still needs to be handled.

Yay exciting!!


Cannot use Console: AttributeError: 'str' object has no attribute 'decode

Trying to run DahuaConsole on a Debian12.8 system with python3.11.
Console seems to be able to login as usual, but unfortunately as soon as I issue a command it crashes:
root@ldap:/etc/DahuaConsole# /etc/DahuaConsole/bin/python3 Console.py --rhost <IP_ADDRESS> --rport 5000 --auth admin:passw --proto dhip
[*] [Dahua Debug Console 2019-2021 bashis <mcw noemail eu>]
[*] logon type "default" with proto "dhip" at <IP_ADDRESS>:5000
[+] Opening connection to <IP_ADDRESS> on port 5000: Done
[+] Dahua Debug Console: Success
[+] Login: Success
[+] keepAlive thread: Started
[*] [Active Users]
<EMAIL_ADDRESS>since 2024-11-24 10:30:01 with "Console" (Id: 21)
[*] Remote Model: VTH5221D, Class: VTH, Time: 2024-11-24 10:30:01
[Console]# config all
Traceback (most recent call last):
  File "/etc/DahuaConsole/Console.py", line 888, in <module>
    main()
  File "/etc/DahuaConsole/Console.py", line 882, in main
    DebugConsole(dargs=dargs)
  File "/etc/DahuaConsole/Console.py", line 32, in __init__
    self.main_console()
  File "/etc/DahuaConsole/Console.py", line 223, in main_console
    msg = sys.stdin.readline().strip().decode('ascii')
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'decode'
[*] Closed connection to <IP_ADDRESS> port 5000
root@ldap:/etc/DahuaConsole#

Any idea why this happens?
BTW: Using Console to e.g. dump config with parameter "--dump config" works fine.

deleting .strip().decode('ascii') entirely works for me

Thx. As I am not a python guy I just followed strict what you suggested:
             #msg = sys.stdin.readline().strip().decode('ascii')
             msg = sys.stdin.readline()

But this leads to following error msg:
  File "/etc/DahuaConsole/Console.py", line 224     msg = sys.stdin.readline()                               ^

check if there is extra space or tab beside the .readline()
or try replace your code with this, see if it helps
try:
self.prompt()
#msg = sys.stdin.readline().strip().decode('ascii')
msg = sys.stdin.readline()
if not self.dh or not self.dh.remote.connected():
log.failure('No available instance')
return False
cmd = msg.split()
i have python 3.12.3


[PHP Provider] Ensure that PHP-FPM workers output is streamed to Nginx

This PR allows errors and warnings from PHP-FPM workers to be streamed to Nginx output. This is critical to be able to get access to the application log, which is usually streamed on php://stderr in production.
Reference: https://serverfault.com/questions/417102/nginx-not-logging-php-errors

@coffee-cup Sorry, I'm a bit ignorant in Rust 😇 .
I did run cargo snapshot, but it didn't update anything:
running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

done: no snapshots to review

I'm sure I'm missing something 😅

Nevermind, I figured it out :-)


The start_new_chapter callback should always execute when a chapter is started

The start_new_chapter callback should always be executed when a chapter is started. Currently, this callback is not executed if the cursor is at the top of the page when the chapter begins. While this may be okay for the built-in behavior, it's a severe limitation for extensions that must perform other work at the chapter transitions.
The start_new_chapter callback will now be responsible for checking if the cursor is already at the top of a new page, including the built-in implementation.


06-design-report-in-power-bi-desktop.md

English:
Task 5: Design page 2
Current Translation:
Tâche 4 : Conception de page 2
Suggested Translation:
Tâche 5 : Conception de la page 2
Description:
change made for stylistic global consistency

Hello contributor,
Thank you for your contribution! We are processing your suggestions and we will update you shortly.
Kind regards,
Microsoft Worldwide Learning Team

Run bugfix tool and waiting for result.


prompt about disabled gps

when you will enter settings and gps is disabled.

it can use some popup.


Update CartLinePrice to use Storefront API's CartLineEstimatedCost

The CartLinePrice component is manually calculating the line price via the merchandise/productVariant price and quantity. The SF API now exposes a CartLineEstimatedCost object (https://shopify.dev/api/storefront/2022-04/objects/CartLineEstimatedCost#fields) and we should use this instead to ensure consistency in cart data.

Blocked by #755

#755 is closed, is this now unblocked? Is it still relevant?

Probably not blocked anymore as per https://github.com/Shopify/shopify/pull/351919, but the logic is still odd as documented here: https://github.com/Shopify/shopify/issues/351913

Probably not blocked anymore as per https://github.com/Shopify/shopify/pull/351919, but the logic is still odd as documented here: https://github.com/Shopify/shopify/issues/351913


runtime(filetype): Add detection for Angular HTML templates

Since Angular 17, the new Control Flow Syntax is not valid HTML. This PR adds a new filetype detection for the HTML templates of Angular.
It first checks the filename. The Angular convention is to use *.component.html for the template. However, this is not mandatory.
If the filename does not match, it will check the contents of the file if it contains:

One of the Control-Flow blocks: @if, @for, @switch, @defer
A structural directive: *ngIf, *ngFor, *ngSwitch, *ngTemplateOutlet
Builtin Angular elements: ng-template or ng-content
String interpolation: {{ something }}

This enables the Angular LSP to attach only to htmlangular filetypes, as well as language parsers, such as tree-sitter.
I've added a simple ft test, but only on filename. I'd like to add a testcase to check the contents of the file, but I do not know how. Tips are welcome.

I've added a simple ft test, but only on filename. I'd like to add a testcase to check the contents of the file, but I do not know how. Tips are welcome.

Have a look https://github.com/vim/vim/blob/4a7a4a3675b6ad90a525524ba4684925df212325/src/testdir/test_filetype.vim#L2485-L2491
Nevermind, I'll add a test while merging.

As mentioned in https://github.com/vim/vim/pull/13594#issuecomment-1834465890 detecting angular files by filename is not desired. I think this is still the case.

Okay, we can revert looking at the filename and just make it inspect the file, when doing so, we can also fix the  comment. @dlvandenberg is that fine for you?

Okay, we can revert looking at the filename and just make it inspect the file, when doing so, we can also fix the comment. @dlvandenberg is that fine for you?

Yeah, sure.


Ability to specify custom RouteHandler

This patch adds the ability to specify a route handler at configuration time. I'm using it to integrate with the ResourcesOverMvc project. Example:
routes.MapAttributeRoutes(config =>
{
    config.ScanAssembly(System.Reflection.Assembly.GetExecutingAssembly());
    config.UseRouteHandler(() => new ResourcesOverMvc.Web.Mvc.RomRouteHandler());
});

Cool! Thanks for the patch. I will merge it and release it with some other things that are cooking. Will let you know when you can get this feature via nuget.
Cheers!


feat(MatDialog): Implement hide/show methods

Feature Description
This feature was previously discussed on #5162.
The proposed solution was to use the addPanelClass method to hide the dialog. But it doesn't work if you are using a backdrop. I built a demo to show the problem (TestDialog01Component).  There is no way to get the current overlay of the dialog to dispose of it manually  (See #15840).
I could find a workaround (TestDialog02Component) by not using the built-in backdrop and creating a custom overlay. But it feels too hacky.
The MdDIalog should have a built-in hide/show method or expose the overlayRef.
If the team accepts it, I could implement it and send a PR.
Use Case
I think in at least two use cases:

An expensive operation to render the dialog.
Using Optimistic UI to hide the modal as soon as the user clicks the button, but it needs to run some async operations that depend on each other, so you cannot destroy the modal until all the operations are complete (my use case).

The solution here seems to be saving the data entered on the dialog so you can re-create it when the dialog is opened. I don't imagine we will add an API to hide the dialog without destroying it

Hey @andrewseguin, what about exposing the  OverlayRef (#15840)? It would solve the problem as well.


Images blurry in the viewer for some M series items, but not when zoomed in.

What
Reported at https://wellcome.slack.com/archives/C8X9YKM5X/p1655209629677859:

Not sure if this is a wider issue, or just affecting me/this record, but I'm looking at one of the M series registers: https://wellcomecollection.org/works/ppq9nwss/items?canvas=65 This morning the main images were all blurry, but were proper resolution when I clicked to zoom in. Now it's the other way around: the main image is fine, but when I click to zoom in it's all blurry and doesn't refocus.

https://wellcomecollection.org/works/ppq9nwss/items?canvas=65

Where

WT/D/1/20/1/1/1
Within this archive I'm seeing the same on other items at WT/D/1/20/1/1/1 but not other random places I've looked at in WT/D/1/20/1. For example, these are not blurry:
WT/D/1/20/1/2/1
WT/D/1/20/1/3/1
WT/D/1/20/1/7/62
I've only seen the main image be blurry, but clear when clicked on for more detail. I've not seen the blurriness swapped around as described above.
I've not seen it on other random items I've looked at elsewhere in the catalogue.

Another example, not from the M series:
WA/HMM/TR/Inv/A.8

Digirati:

the issue is with images that required users to agree to the "content advisory" notification.

the blurry image is because the largest thumbnail for these images is 200px wide, this is being resized too far and stretched. I'll fix that now

https://wellcome.slack.com/archives/CBT40CMKQ/p1655391928349849

To expand on above comment:
A number of "thumbnail" jpgs are generated when an image is added to the DLCS. The largest of these being 1024px on longest edge.
An optimisation was added on Monday 13/06 to make more use of these thumbnails by scaling them down to fulfil requests that are <1000px.
However, this introduced a bug with "content advisory" images as the largest thumbnail generated for those images is 200px, which resulted in blurry images as these were incorrectly being upscaled.
The works page requests a different sized image depending on viewport resolution, which is why some may be blurry and others not.
Deep zoom will never use thumbnails so these won't have been affected by the above - if deep zoom appears blurry it's generally because images aren't being loaded quick enough for the zoom operation.