Why encrypted jobs?

One of the jobs in the config is encrypted.
    {       "type": "encrypted",       "args": {         "format": "json",         "data":  ...
Base64 decoding shown following header:
age-encryption.org/v1 -> scrypt yKFNmP9oNK38ps7grEA12A 18
I'm curious why? That does not help to build the trust, I'm not going to recommend this tool to anyone (despite fully supporting its mission until) we figure this out.

@Fahrenheit2539 you have an option to skip those jobs completely via --skip-encrypted commandline flags (see implementation) specifically so that you don't have to blindly trust me or other maintainers but there are multiple benefits to having encrypted jobs. I cannot really talk about all of the benefits (for various reasons) but we can speculate that those jobs could use vulnerabilities that are not publicly disclosed yet (and thus exposing them could pose a risk at innocent companies that are also using affected software) or that it's just better if the receiving end of those attacks don't have a quick and easy way to examine what's targeting them and thus have harder time building their defense
We could also setup a process where external independent security experts would validate both the code, the executable, and the encrypted config. For the config we'd need to validate that those sources can be trusted before we provide them with the content of those configs - let me know if you know anyone who's interested and I'll try to put them in contact with appropriate people

Thanks, really appreciate quick response! Your arguments make sense, don't think you want to add more bureaucracy in this fast-moving process.
What would help is updating docs to briefly mention this. I'll send a PR in the evening (US time)
Sent from Outlookhttps://aka.ms/qtex0l Mail for Windows Mobile 11

From: Bohdan @.>
Sent: Tuesday, March 22, 2022 8:25:46 AM
To: Arriven/db1000n @.>
Cc: Alex @.>; Mention @.>
Subject: Re: [Arriven/db1000n] Why encrypted jobs? (Issue #394)
@Fahrenheit2539https://github.com/Fahrenheit2539 you have an option to skip those jobs completely via --skip-encrypted commandline flags (see implementationhttps://github.com/Arriven/db1000n/blob/06d290b70c1b8818254fd5fcac53720cad8417ac/src/jobs/utils.go#L94-L96) specifically so that you don't have to blindly trust me or other maintainers but there are multiple benefits to having encrypted jobs. I cannot really talk about all of the benefits (for various reasons) but we can speculate that those jobs could use vulnerabilities that are not publicly disclosed yet (and thus exposing them could pose a risk at innocent companies that are also using affected software) or that it's just better if the receiving end of those attacks don't have a quick and easy way to examine what's targeting them and thus have harder time building their defense
We could also setup a process where external independent security experts would validate both the code, the executable, and the encrypted config. For the config we'd need to validate that those sources can be trusted before we provide them with the content of those configs - let me know if you know anyone who's interested and I'll try to put them in contact with appropriate people
—
Reply to this email directly, view it on GitHubhttps://github.com/Arriven/db1000n/issues/394#issuecomment-1075322362, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAZZBREC5TOWXXBHARY6PCTVBHQ7VANCNFSM5RLC42NQ.
You are receiving this because you were mentioned.Message ID: @.***>

I don't think that it would require a lot of bureaucracy, especially on my side. I would most likely delegate the trust establishment to someone else (whom I already trust).
And it would be beneficial to get some feedback from experts anyway at some point - I plan to continue working on this project even after the war ends, haven't seen any open tools that allow orgs to easily stress test their DDoS protection at scale (legally it's the projects purpose from the very beginning, we can just say that we're outsourcing this stress test right now)

Can send me an email to fahrenheit2539 at hotmail.com? Interested to discuss how can I help.


Fix for CAMEL-8452: Camel route model - Preserve {{ }} placeholders

We preserve just the uri property of ProcessorDefinitions before we create the associated Processor and restore the original value after the the processor is created.
We can probably target more properties to be preserved/restored as the need arrises.

There would be more work to do as I think a general approach is needed and not only for uri attributes.
But the idea of restoring after creating processor is good. So IMHO we should be able to

remember which we have changed
restore back

Ad 1)
By letting resolvePropertyPlaceholders return a Map<String, Object> with the original options that was replaced.
And the same for this guy
https://github.com/apache/camel/blob/master/camel-core/src/main/java/org/apache/camel/model/ProcessorDefinition.java#L430
Ad 2)
And then after the processor is created, then restore them back on the model
Mind that resolvePropertyPlaceholders is used a few places in the ProcessorDefinition so you need to do this in both places.
And also in DataFormatDefinition and RouteDefinitionHelper

Hey @davsclaus, i've just push my latest version of the fix.  It has one problem, it's that you have to use replaced route id's in the getRouteDefinition() calls and show up replaced in the id attribute of the route element of the dumpModelAsXml().  See:
https://github.com/chirino/camel/commit/7de51fd5ff33dc83dc58894b38b9ecc87619cf14#diff-2f0c2e22792869e439b2459cbc483a29R32
If you uncomment a few lines in DefaultCamelContext, then you can use the un-replaced values in those tests.  But then a bunch of other tests start failing.  So I decided to keep it commented for now.

Its a bit ugly to restore after camel context has been started, and doing that around staring camel context. I think this is the taking it too far in a wrong direction.
Okay so lets for now not restore route ids, as that can cause issues (we can look at those later)
And instead implement this where it belongs in the logic of processor definition / prepare route.

We chatted about this, let me add the chat log here
[13:57:59]  	 davsclaus:  CAMEL-8452?
[14:00:12]  	 ya its only the commented out in the doStart i think its the wrong direction
[14:00:19]  	 doing stuff after the context has been started
[14:00:23]  	 but that is not enabled
[14:00:44] 	 mhautman<EMAIL_ADDRESS>left IRC. (Ping timeout: 20 seconds)
[14:00:54]  	 having a runnable / callable as the redo action is great/clever
[14:01:19]  	 you may want to add a little bit of javadoc to the RestoreAction class to desribe a bit of its purpose
[14:01:29]  	 so when others come along the code that have a better idea what it does
[14:01:35]  	 because we dont do this elsewhere
[14:01:51]  	 also the code should pass checkstyle
[14:01:55]  	 so from cmd run
[14:02:01]  	 mvn install -Dsourcecheck
[14:02:09]  	 in camel-core
[14:02:33]  	 and also create this unit test DumpModelAsXmlPlaceholdersTest
[14:02:35]  	 in camel-spring
[14:02:44]  	 so we test it when using spring xml dsl as primary
[14:02:49]  	 as there can be subtle difference

Thanks for the PR this has been merged. Do you mind closing this


fix(install): add missing python package

Should address one of the issues mentioned in the following tickets:

https://github.com/MiczFlor/RPi-Jukebox-RFID/issues/1618
https://github.com/MiczFlor/RPi-Jukebox-RFID/issues/1788

Hi, thanks for the PR.
that’s a little strange, because evdev should be installed here:
https://github.com/MiczFlor/RPi-Jukebox-RFID/blob/5084ace1f1077f04d4fae4640545e7df9b97f9d7/scripts/installscripts/buster-install-default.sh#L849
of course your PR works, but why does the pip install not work? Did you see something during a failed installation in the install logs?

Maybe there’s a problem with Bullseye:
https://github.com/MiczFlor/RPi-Jukebox-RFID/issues/1653#issuecomment-985765521
seems to work with Buster though

Maybe there’s a problem with Bullseye: #1653 (comment)
seems to work with Buster though

saw that comment but actually for me the problem occurred also with Buster. I can try to reproduce it and see if I see any error in the log.

I assume that’s the underlying issue #1791

@petschni can you check, if #1791 solves your issue?

I think while this fixes the issue, the underlying issue is #1791

@s-martin #1792 actually fixes the problem. However the following error is still thrown - but does not seem to impact the feature:
#
# INSTALLATION FINISHED
#
#####################################################

If you are using an RFID reader, connect it to your RPi.
(In case your RFID reader required soldering, consult the manual.)
Have you connected your RFID reader? [Y/n]
Please select the RFID reader you want to use
1) USB-Reader (e.g. Neuftech)
2) RC522
3) PN532
4) Manual configuration
5) Multiple RFID reader
#? 1
Choose the reader from list
0 HXGCoLtd Keyboard
Device Number: 0
Exception ignored in: <function BaseEventLoop.__del__ at 0x761e08a0>
Traceback (most recent call last):
  File "/usr/lib/python3.7/asyncio/base_events.py", line 622, in __del__
    warnings.warn(f"unclosed event loop {self!r}", ResourceWarning,
  File "/usr/lib/python3.7/asyncio/base_events.py", line 390, in __repr__
    f'<{self.__class__.__name__} running={self.is_running()} '
  File "/usr/lib/python3.7/asyncio/base_events.py", line 1794, in get_debug
    return self._debug
AttributeError: '_UnixSelectorEventLoop' object has no attribute '_debug'

DONE. Let the sounds begin.


[SPARK-44687][BUILD] Fix mima check for Scala 2.13 after SPARK-44198 merged

What changes were proposed in this pull request?
This pr aims add a new ProblemFilters to MimaExcludes.scala to fix mima check for Scala 2.13 after SPARK-44198 merged.
Why are the changes needed?
Scala 2.13's daily tests have been failing the mima check for several days:

https://github.com/apache/spark/actions/runs/5765663964

Does this PR introduce any user-facing change?
No
How was this patch tested?

Pass GitHub Actions
Manual verification：

The mima check was passing before SPARK-44198.

// [SPARK-44425][CONNECT] Validate that user provided sessionId is an UUID
git reset --hard a3bd477a6d8c317ee1e9a6aae6ebd2ef4fc67cce
dev/change-scala-version.sh 2.13
dev/mima -Pscala-2.13

[success] Total time: 129 s (02:09), completed 2023-8-5 14:21:06

The mima check failed after SPARK-44198 was merged

// [SPARK-44198][CORE] Support propagation of the log level to the executors
git reset --hard 5fc90fbd4e3235fbcf038f4725037321b8234d94
dev/change-scala-version.sh 2.13
dev/mima -Pscala-2.13

[error] spark-core: Failed binary compatibility check against org.apache.spark:spark-core_2.13:3.4.0! Found 1 potential problems (filtered 4013)
[error]  * the type hierarchy of object org.apache.spark.scheduler.cluster.CoarseGrainedClusterMessages#SparkAppConfig is different in current version. Missing types {scala.runtime.AbstractFunction4}
[error]    filter with: ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.scheduler.cluster.CoarseGrainedClusterMessages$SparkAppConfig$")
[error] java.lang.RuntimeException: Failed binary compatibility check against org.apache.spark:spark-core_2.13:3.4.0! Found 1 potential problems (filtered 4013)
[error] 	at scala.sys.package$.error(package.scala:30)
[error] 	at com.typesafe.tools.mima.plugin.SbtMima$.reportModuleErrors(SbtMima.scala:89)
[error] 	at com.typesafe.tools.mima.plugin.MimaPlugin$.$anonfun$projectSettings$2(MimaPlugin.scala:36)
[error] 	at com.typesafe.tools.mima.plugin.MimaPlugin$.$anonfun$projectSettings$2$adapted(MimaPlugin.scala:26)
[error] 	at scala.collection.Iterator.foreach(Iterator.scala:943)
[error] 	at scala.collection.Iterator.foreach$(Iterator.scala:943)
[error] 	at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
[error] 	at com.typesafe.tools.mima.plugin.MimaPlugin$.$anonfun$projectSettings$1(MimaPlugin.scala:26)
[error] 	at com.typesafe.tools.mima.plugin.MimaPlugin$.$anonfun$projectSettings$1$adapted(MimaPlugin.scala:25)
[error] 	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] 	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:63)
[error] 	at sbt.std.Transform$$anon$4.work(Transform.scala:69)
[error] 	at sbt.Execute.$anonfun$submit$2(Execute.scala:283)
[error] 	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:24)
[error] 	at sbt.Execute.work(Execute.scala:292)
[error] 	at sbt.Execute.$anonfun$submit$1(Execute.scala:283)
[error] 	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:65)
[error] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] 	at java.lang.Thread.run(Thread.java:750)
[error] (core / mimaReportBinaryIssues) Failed binary compatibility check against org.apache.spark:spark-core_2.13:3.4.0! Found 1 potential problems (filtered 4013)
[error] Total time: 82 s (01:22), completed 2023-8-5 14:23:49

with this pr, mima check pass

gh pr checkout 42358
dev/change-scala-version.sh 2.13
dev/mima -Pscala-2.13

[success] Total time: 157 s (02:37), completed 2023-8-5 14:31:05

Merged into master. Thanks @srowen


Missing selected items when using the select all first first row option

Hi,
Again I am loving this pod. Thanks for keeping it well maintained!!
I think I found a bug today.
If you add a first row with type .All. In the callback to setSelectedItem. The selected items array is returning with zero items.
It would make sense to me if there is a select all option available the setSelectedItem closure should return an array of all the items.
If this is not a bug and it is the intentional behaviour I am sorry. I just thought I would raise it with you.
Thanks.

Hi @heliumdice
Thanks for the feedback.
You'll get an event when you select the first row whether it is "All", "None" or any text you've set.
// array of data objects
var dataArray = []

//  array of selected items
var selectedItems = []

// var to maintain first row selection
var firstRowSelected = true

selectionMenu.addFirstRowAs(rowType: .All, showSelected: firstRowSelected) { (text, isSelected) in
       firstRowSelected = isSelected

      // Here we'll get the text of the row and based on the text we can decide if it is none or all.
      // Ex.
     if (text == "All"){
          // do something ( selectedItems = dataArray)
     }
}

I hope you got my point.


add missing tests for symlink and symlinkat

those tests were originally added in https://github.com/nh2/hatrace/pull/31/ and removed in https://github.com/nh2/hatrace/pull/32/files

This change is

Thanks a lot!


didcom service url over https

Is this a regression?
Yes
Description
when service is on https as didcom service static configured on http for agent runner we are getting browser security error of using http request over https:
xhr.js:258 Mixed Content: The page at 'https://wallet.socious.io/connect/?_oob=eyJpZCI6IjUzNzIzNDVkLWY1ODItNDFlYy1iMTQ2LTdiMjEyN2JjYTRhOCIsInR5cGUiOiJodHRwczovL2RpZGNvbW0ub3JnL291dC1vZi1iYW5kLzIuMC9pbnZpdGF0aW9uIiwiZnJvbSI6ImRpZDpwZWVyOjIuRXo2TFNtZGF0a1RrODdZSDFSQWpHWm5ud1dNeVBiRHBnR0dFWkZ1VGt3eVZjNHJLUS5WejZNa2p0Z2lYQ2FHZWpXeW5OaGlzc29XZFJyazJXVEh4SHJBM1U2VnZvZ0RMTEJDLlNleUowSWpvaVpHMGlMQ0p6SWpvaWFIUjBjRG92THpVMExqZzFMakl3TVM0eU9qZ3dNREF2Wkdsa1kyOXRiU0lzSW5JaU9sdGRMQ0poSWpwYkltUnBaR052YlcwdmRqSWlYWDAiLCJib2R5Ijp7ImdvYWxfY29kZSI6ImlvLmF0YWxhcHJpc20uY29ubmVjdCIsImdvYWwiOiJFc3RhYmxpc2ggYSB0cnVzdCBjb25uZWN0aW9uIGJldHdlZW4gdHdvIHBlZXJzIHVzaW5nIHRoZSBwcm90b2NvbCAnaHR0cHM6Ly9hdGFsYXByaXNtLmlvL21lcmN1cnkvY29ubmVjdGlvbnMvMS4wL3JlcXVlc3QnIiwiYWNjZXB0IjpbXX19' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://<IP_ADDRESS>:8000/didcomm'. This request has been blocked; the content must be served over HTTPS.
Please provide the exception or error you saw
No response
Please provide the environment you discovered this bug in
No response
Anything else?
No response

@jeyem, I apologize for the delay in replying to this tread
Our infrastructure uses the API Gateway (APISIX) to manage TLS and expose the Mediator and the Agent REST API endpoints.
Managing TLS at the proxy layer is generally considered a best practice in modern distributed architectures. It abstracts the complexity of secure communications from the services themselves and allows centralized control over security policies and certificates. This approach is particularly beneficial in microservices architectures where numerous services need secure exposure to the outside world. The gateway can efficiently handle security, allowing service developers to focus on business logic and service functionality without compromising on security.
Moreover, managing TLS at the gateway simplifies certificate management, as you only need to manage certificates at the gateway level rather than for each service. This can also be more cost-effective and reduce the administrative overhead of certificate renewal and compliance checks.
TLS can also be implemented at the application level to simplify local development and prototyping. I'm not sure this feature will be a priority as we have tight resources and an extensive roadmap with other features.
Currently, we are looking for the right place to document TLS management and usage of the API Gateway in the deployments.
Also, you can use any proxy for this purpose: APISIX, Caddy, Traefik, Nginx, etc.

This issue has been addressed and is now considered solved. If you have further questions or related concerns, please open a new issue.


Skip VISA VIs verification for LabVIEW 2020

[x] This contribution adheres to CONTRIBUTING.md. (Required)
[ ]  Automatically post PR comments with images for G code changes? (Recommended for small changes)

What does this Pull Request accomplish?
This is the missed out case from the PR https://github.com/ni/measurementlink-labview/pull/434. The testcase test_VerifyVIsNotBroken.vi checks MeasurementLink VISA folder VIs which fails the test when run in LabVIEW 2020. Thus skipped this folder verification alone for LV 2020.
Why should this Pull Request be merged?
To pass test_VerifyVIsNotBroken.vi in LV 2020.
What testing has been done?
Manually ran the test in LV2020

The Diffs are:

I'm surprised that the VIs are broken. I'm not necessarily opposed to the change you're making, but I want to better understand what is happening. We went out of our way to make sure the VIs are not broken, even when the driver is not installed. However, you might get a loading dialog if the driver isn't installed and that may be reason enough to skip these tests.

@pbirkhol-ni I'm running this test in environment with VISA and LV 2020 installed. I see some of the VIs in MeasurementLink VISA folder are trying to load and fail. Here I have shared a small video on the test run in LV
Finally, The following VIs are broken: MeasurementLink VISA.lvlib:Session Factory.lvclass:To VISA Init and Close Enum.vi is displayed in test result.
It seems like vi.lib\NI VISA\VISA grpc Open Close Behavior.ctl is failing to load.

Okay. Thanks for clarifying. To VISA Init and Close Enum.vi is broken because the VISA enum isn't installed. That breakage will not propagate up to a customer's diagram, though, because we wrap that call in a Call by Reference node. So everything is working as intended, I just overlooked that VI. So this change is fine. Let me just give it a once over and I'll approve.


Fixed KeyShared routing when messages are sent in batches of 1

~Note: this is based on top of #6791, #7104, #7105 and #7106. Once these are merged, I'll rebase here. For the sake of this review, check commit 6ca3a75d20~
Motivation
When messages are published in batches of 1 message (eg: from Go native client lib), the KeyShared dispatcher is failing to extract the key from the first message in batch, instead everything is routed using NONE_KEY, landing on 1 single consumer.

/pulsarbot run-failure-checks


chore: update zeego & clear up code

Why
update the Zeego version to fix some hacks code and clear up some design-system packages.
How

Test Plan

The icons are reversed now on web:

Also, not sure if related, but the twitter icon is also misaligned in that branch.


Set az conversion

This PR is a replacement of #105 which was closed accidentally.
Most of the comments / suggestions were applied in the previous PR.
Below is my comment regarding the suggestion to separate enu2rgaz into enu2rg and en2az:
https://github.com/opera-adt/COMPASS/pull/105#discussion_r1157843023

@seongsujeong, while I was testing this PR, I noticed that the azimuth component of the Solid Earth Tides (SET) is not allocated inside the HDF5 CSLC product. However, it seems that the azimuth component of the correction is allocated inside the temporary product corrections in the scratch directory. Have you tested the PR? Can you reproduce the issue on your side?

@seongsujeong, while I was testing this PR, I noticed that the azimuth component of the Solid Earth Tides (SET) is not allocated inside the HDF5 CSLC product. However, it seems that the azimuth component of the correction is allocated inside the temporary product corrections in the scratch directory. Is this intentional? If not, I believe you should modify this function here
https://github.com/opera-adt/COMPASS/blob/8aeecda5f43b15869a0d044b78df9d7b9864d5b3/src/compass/utils/h5_helpers.py#L583
and include the azimuth SET.

@vbrancat Thanks for catching this. I've made the changes on the function to add the azimuth component of SET LUT be written into the output HDF5.

@scottstanie it would be great if you can have a look at this PR. Thanks

Looks like changes to isce v0.12 means it's erroring because of azimuth_carrier being the wrong name for an argument:

  File "/u/aurora-r0/staniewi/repos/sweets/src/sweets/_geocode_slcs.py", line 59, in run_geocode
    s1_geocode_slc.run(cfg)
  File "/u/aurora-r0/staniewi/repos/COMPASS/src/compass/s1_geocode_slc.py", line 158, in run
    isce3.geocode.geocode_slc(geo_burst_raster, rdr_burst_raster,
TypeError: geocode_slc() got an unexpected keyword argument 'azimuth_carrier'

any chance of adding a unit test for some of the new/changed functionality?

@scottstanie I'm not sure if the errors you reported in the comment above is related to this PR. I know it can be a bit tricky to test this PR with the most up-to-date ISCE3, but if the change look good to you overall?


Feat: Alpha discard

I've added alpha discard to be able to have grids and stuff per texture. The art department requested it asap.
So here it is

Well I was waiting to the lead of graphics to see it because yesterday he commented that maybe there is a better way than the discard...

Sorry!! I though that as there were no pending reviewers it was good to go.


Compilation error on file lib/hedwig_xmpp.ex

OS X 10.11.6
elixir 1.3.2
Erlang/OTP R19
Also failed with elixir 1.2.5/OTP R18.
===> Compiling p1_utils
===> Compiling fast_xml
==> hedwig_xmpp
Compiling 1 file (.ex)

== Compilation error on file lib/hedwig_xmpp.ex ==
** (CompileError) lib/hedwig_xmpp.ex:4: module Hedwig.Adapter is not loaded and could not be found
    (elixir) expanding macro: Kernel.use/1
    lib/hedwig_xmpp.ex:4: Hedwig.Adapters.XMPP (module)
    (elixir) lib/kernel/parallel_compiler.ex:116: anonymous fn/4 in Kernel.ParallelCompiler.spawn_compilers/1

could not compile dependency :hedwig_xmpp, "mix compile" failed. You can recompile this dependency with "mix deps.compile hedwig_xmpp", update it with "mix deps.update hedwig_xmpp" or clean it with "mix deps.clean hedwig_xmpp"

Add the following line in your dep
[ {:hedwig_xmpp, github: "hedwig-im/hedwig_xmpp"}]

@RuthraiahThulasi Thanks!


Refactor hashing and fingerprint functions

Fixes #271.
With newly added classes for fingerprinting, now it is easy to make a
fingerprint from various types of data.
FingerprintBuilder can be used to build a fingerprint from data.
FileBasedRelation.signature now takes it as an argument, so the
implementations don't have to do the hashing themselves. They can just
use the provided builder.
For unorderd combining, one can use bitwise XOR to combine multiple
fingerprints. This way, the order becomes irrelevant and can be used for
unordered collections like Set.

What is the context for this pull request?

Tracking Issue: #271
Parent Issue: N/A
Dependencies: N/A

What changes were proposed in this pull request?

The signature related API is changed. Instead of using plain String, a dedicated class (Fingerprint) is introduced.
The issue #271 is fixed by removing the call to sortBy.

Does this PR introduce any user-facing change?

The JSON format remains the same as well as MD5. However, the way the signatures are computed is changed to accommodate more efficient computation. Therefore, existing signatures won't match the new one. This is a breaking change as #268 was.
For library authors and developers who want to extend Hyperspace, the API is incompatible and must be adapted.
How was this patch tested?

Unit test

@clee704, @sezruby
If this is a breaking change that will make the already created indexes useless, I would like to throw in some common sense questions that should be addressed in a way or another:

Any migration tool to migrate the existing indexes to the new fingerprinting method?
If the answer is NO to the question above, is there a plan to have such a tool?
If the answer is NO do we at least know how many are using OSS Hyperspace right now and let them know before about this fact?
In the case of a user trying to use an old index with this new version how would he know that the problem is this breaking change? Is there an exception thrown or a message or something?
Is there a process defined somewhere for cases when breaking changes are introduced?
How will this breaking change be documented for the users?

I know these are a lot of questions and as good engineers we need to have a bit of "client-centric" mind to create a useful and easy-to-use product.
@rapoth, @apoorvedave1, @pirz, @imback82 WDYT?

@clee704, @sezruby
If this is a breaking change that will make the already created indexes useless, I would like to throw in some common sense questions that should be addressed in a way or another:

Any migration tool to migrate the existing indexes to the new fingerprinting method?
If the answer is NO to the question above, is there a plan to have such a tool?
If the answer is NO do we at least know how many are using OSS Hyperspace right now and let them know before about this fact?
In the case of a user trying to use an old index with this new version how would he know that the problem is this breaking change? Is there an exception thrown or a message or something?
Is there a process defined somewhere for cases when breaking changes are introduced?
How will this breaking change be documented for the users?

I know these are a lot of questions and as good engineers we need to have a bit of "client-centric" mind to create a useful and easy-to-use product.
@rapoth, @apoorvedave1, @pirz, @imback82 WDYT?

Thanks @andrei-ionescu for rasing these questions! Here's my take on this:

Any migration tool to migrate the existing indexes to the new fingerprinting method?
If the answer is NO to the question above, is there a plan to have such a tool?

My thinking is that we should not block on a migration utility for this upcoming release. However, I requested @clee704 to write-up a design proposal for how such a migration utility would be designed along with all the dimensions we would have to cover. @clee704 should be able to get to that soon.  Once it passes vote, it will be open for one of the folks to take up in the coming weeks.
That being said, if you are affected by this breaking change or one of your users/customers is, please let us know. Currently, I have already provided a heads-up to the customers I know who are using Hyperspace in production and they were fine with it.

In the case of a user trying to use an old index with this new version how would he know that the problem is this breaking change? Is there an exception thrown or a message or something?

@clee704 @sezruby Could we verify what happens and add the appropriate error message + steps they would have to take to overcome the problem?

Is there a process defined somewhere for cases when breaking changes are introduced?

No but now is probably a good time to discuss and document this. In general, as long as we do not announce a v1.0, we do not guarantee backward/forward compatibility and do not recommend the code to be used in production. This may not have anything to do with the quality of the code but rather stability (as customers provide us with more and more feature requests, we are trying to establish a good foundation so we can avoid major breaking changes in the future).
As we get closer to v1.0, we will follow semantic versioning guidelines (i.e., everything within a major version will be forward/backward compatible but across major versions, we may break things but we will provide a migration utility by then).

How will this breaking change be documented for the users?

It will be documented in the release notes of this specific version and the frequently-asked-questions section. Suggestions are welcome!

@rapoth
Thanks for putting up your thoughts.

My thinking is that we should not block on a migration utility for this upcoming release.

I agree that we should not block this PR by not having the migration tool - this is acceptable now as Hyperspace is at the beginning. It would be great though to have at least a ticket/issue/proposal/feature to describe the gap and put together a plan to fill the gap. I’m glad we are on the same page.

if you are affected by this breaking change or one of your users/customers is, please let us know

Currently no major impact. I do have a few indexes created on big datasets 15TB and more or which I will be forced to recreate when you upgrading.

My thinking is that we should not block on a migration utility for this upcoming release.

I agree that we should not block this PR by not having the migration tool - this is acceptable now as Hyperspace is at the beginning. It would be great though to have at least a ticket/issue/proposal/feature to describe the gap and put together a plan to fill the gap. I’m glad we are on the same page.

Thanks @andrei-ionescu! I've requested @clee704 to do this. Once he wraps up his current work, he will write up a proposal . Here's the feature request I just opened #420 - please feel free to add anything else I may have missed directly in that issue.

If this signature change is the only breaking change of v0.5, then it should be easy to update the signatures of indices from old systems as we have a version string in the serialized index log entries. This will be addressed as part of #420. Perhaps a simple migration utility to address this could be part of v0.5.

Spark was too slow to create such many files, so I did measurement for the signature method of DefaultFileBasedRelation with fake files.
Below are the elapsed times of the signature method with varying number of files (averages of 10 repetitions).

# Files
Before
After
Speedup

10,000
65.7 ms
32.4 ms
51%

100,000
581 ms
249 ms
57%

1,000,000
7.59 s
2.32 s
69%

The results can vary depending on many other factors such as file path lengths and the format of file names. In this measurement I assumed a single root directory whose path length is 64 and random file names of length 67 (which is the default length of parquet files in Spark).
Here's the source for the test:
import scala.util.Random

import org.apache.hadoop.fs.{FileStatus, Path}
import org.apache.log4j.{Level, Logger}
import org.apache.spark.SparkFunSuite
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.execution.datasources.{HadoopFsRelation, LogicalRelation, PartitioningAwareFileIndex}
import org.apache.spark.sql.execution.datasources.parquet.ParquetFileFormat
import org.apache.spark.sql.types.StructType

import com.microsoft.hyperspace.index.sources.default.DefaultFileBasedRelation
import com.microsoft.hyperspace.util.fingerprint.MD5FingerprintBuilderFactory

class PerformanceTest extends SparkFunSuite {
  test("DefaultFileBasedRelation.signature performance") {
    for (numFiles <- Seq(10000, 100000, 1000000)) {
      val repetition = 10
      val measurements = (1 to repetition).map(_ => runTest(numFiles))
      println(s"numFiles = $numFiles")
      println(s"max = ${measurements.max / 1000000.0} ms")
      println(s"min = ${measurements.min / 1000000.0} ms")
      println(s"avg = ${measurements.sum / measurements.length / 1000000.0} ms")
    }
  }

  def runTest(numFiles: Int): Long = {
    val fileBasedRel = getDefaultFileBasedRelation(numFiles)
    val start = System.nanoTime()
    val sig = fileBasedRel.signature(fb)
    // val sig = fileBasedRel.signature
    val end = System.nanoTime()
    assert(sig.isDefined)
    // assert(sig.nonEmpty)
    end - start
  }

  def getDefaultFileBasedRelation(numFiles: Int): DefaultFileBasedRelation = {
    val files = (1 to numFiles).map(_ => getRandomFileStatus())
    val location = FakeFileIndex(files)(spark)
    val parquet = new ParquetFileFormat()
    val st = new StructType()
    val rel = HadoopFsRelation(location, st, st, None, parquet, Map())(spark)
    val logicalRel = LogicalRelation(rel)
    new DefaultFileBasedRelation(spark, logicalRel)
  }

  def getRandomFileStatus(): FileStatus = {
    val length = rand.nextLong()
    val modificationTime = rand.nextLong()
    val name = getRandomAlphanumericString(67)
    val path = new Path(s"/$dir", name)
    new FileStatus(length, false, 3, 128 * 1024 * 1024, modificationTime, path)
  }

  def getRandomAlphanumericString(length: Int): String = {
    val chars = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
    val s = new Array[Char](length)
    for (i <- 0 until length) s(i) = chars(rand.nextInt(chars.length))
    new String(s)
  }

  Logger.getLogger("org").setLevel(Level.OFF)
  lazy val spark = SparkSession.builder().master("local").getOrCreate()
  lazy val rand = new Random(42)
  lazy val dir = getRandomAlphanumericString(64)
  lazy val fb = new MD5FingerprintBuilderFactory().create

  case class FakeFileIndex(files: Seq[FileStatus])(sparkSession: SparkSession)
    extends PartitioningAwareFileIndex(sparkSession, Map(), None) {
    override def allFiles(): Seq[FileStatus] = files
    override def partitionSpec() = ???
    override def leafFiles = ???
    override def leafDirToChildrenFiles = ???
    override def rootPaths = ???
    override def refresh() = ???
  }
}


separate AI into class

or even in separate file. This would be for later usage.

fixed by commit https://github.com/zordsdavini/vaskici_tui/commit/8bb6fe31c2901c47d7d1a2944000a2967b1c5ed4


Incompatibility Issue with Timestamp Precision when Using to_deltalake()

Problem Description:
The issue I'm encountering is related to the casting of timestamp columns in a DataFrame when utilizing the to_deltalake() function. Presently, when I pass a dtype containing timestamp columns, the columns are casted to nanoseconds (ns) precision. However, Deltalake exclusively supports microsecond (us) precision for timestamps. As a result of this discrepancy in precision, the deltalake.write_deltalake operation throws an error with the message: "Schema error: Invalid data type for Delta Lake: Timestamp(Nanosecond, None)."
Desired Solution:
I propose that necessary modifications be implemented in the athena2pyarrow function of the AWS SDK for Pandas. Specifically, when handling timestamp columns in this function, it should be adjusted to use microseconds (us) precision instead of nanoseconds (ns). This adjustment will ensure that the timestamp columns align with Deltalake's precision requirements, fostering consistency in precision. By incorporating this alteration, we can guarantee that Deltalake can effectively parse the timestamp column, thus preventing any compatibility issues stemming from precision disparities.

Hi @DimitarSirakov I've double checked and it is possible to write datetime objects in microseconds (us):
from datetime import datetime
import pandas as pd
import awswrangler as wr

df = pd.DataFrame({"c0": [1, 2, 3], "c1": ["foo", None, "bar"], "c2": [3.0, 4.0, 5.0], "c3": [True, False, None]})
df["c4"] = pd.to_datetime(datetime.now().timestamp())
# Cast to microseconds
df["c4"] = df["c4"].astype("datetime64[us]")
# Validate dtype
df.dtypes

Out[4]:
c0             int64
c1            object
c2           float64
c3            object
c4    datetime64[us]
dtype: object

wr.s3.to_deltalake(path=<REDACTED>, df=df)

If I inspect the output parquet file, the timestamp in the file is in microseconds (us).
> parquet-tools inspect <REDACTED>
...
############ Column(c4) ############
name: c4
path: c4
max_definition_level: 1
max_repetition_level: 0
physical_type: INT64
logical_type: Timestamp(isAdjustedToUTC=false, timeUnit=microseconds, is_from_converted_type=false, force_set_converted_type=false)
converted_type (legacy): NONE

Can you provide a reproducible example of the issue, please?


Add column name metadata to tabmat matrices

Checklist

[ ] Added a CHANGELOG.rst entry

Summary
Add column names and term names metadata to all matrix classes, accessible via the column_names and term_names properties, or via the get_names and set_names functions.
Details
The main idea is that only DenseMatrix, SparsMatrix and CategoricalMatrix instances have these attributes, and other types construct theirs on the fly. This reduces complexity and the amount of cases we have to consider. Please see details in comments.
Column and terms:- I propose to store store two attributes: column and term names. While it may seem weird from a dataframe perspective, it makes sense when used in a regression setting, especially with a formula interface. A term is an element of a formula (e.g. poly(x) or C(x), while columns are the set of variables they imply. In the case of categoricals, a term is a categorical and a column is one of its dummy columns.
Names are  optional: Each name is stored as a vector, can contain None as their elements. When using the column name getter function, those columns will have some dynamically generated name signifying column index (e.g. _col_1 ). These names are not persistent over e.g. matrix concatenation or column reordering. Normal name are.
Matrix operations: column names are propagated when concatenating via SplitMatrix or using column indexing.
Formula interface: this way of implementing column and term names would streamline the formula materializer a significantly, and a couple hundred lines worth of overloaded functions could be removed.
Issues
The propagation of these new attributes when using certain methods is inconsistent at best and somewhat incorrect at worst. This inconsistency arises from the fact that dense and sparse matrices are subclassed differently from their respective base classes.

Inconsistencies in column name propagation
Take a DenseMatrix and a SparseMatrix with some names:
>>> import numpy as np
>>> import scipy.sparse as sps
>>> from tabmat import DenseMatrix, SparseMatrix

>>> Xd = DenseMatrix(np.ones((5, 2)), column_names=["d1", "d2"], term_names=["d", "d"])
>>> Xs = SparseMatrix(sps.csc_matrix(np.ones((5, 2))), column_names=["c1", "c2"], term_names=["c", "c"])

Indexing correctly propagates column names (explicitly implemented) ✅
>>> Xd[:, [1, 0]].column_names, Xs[:, [1, 0]].column_names
(['d2', 'd1'], ['c2', 'c1'])

Transpose and reshape incorrectly propagates column names for DenseMatrix ❌. For SparseMatrix, the transpose is not even a SparseMatrix (better).
>>> Xd.T.column_names
['d1', 'd2']

Other functions correctly propagate column names for DenseMatrix ✅, but not for SparseMatrix ❌
>>> (Xd * 2).column_names, (Xs * 2).column_names
(['d1', 'd2'], [None, None])

So ATM this PR is fine for the purposes of getting column names for regressions if the matrices are not really manipulated after creating them, but can lead to some nasty surprises if one is not careful.

I've rebased this branch to the densematrix-refactor branch (the proposed refactoring effort to make the API more well-defined and consistent). This solved the issues from my previous comment (inconsistent column name propagation), and column name handling became more explicit and straightforward.

Can you add an entry to changelog please.


Revising the encoding of BoxBitVec_n

Thanks to @bollu for reporting that this simple program fails to check:
let bv_trivial (input_x : BV.bv_t 3) = assert (BV.bvdiv input_x 1 == input_x)

The problem is that when encoded to SMT, the goal looks something like
(BoxBitVec_3 (bvdiv (UnboxBitVec_3 x) (bv2int 1))) = x)

This fails, because the encoding doesn't let us prove that (BoxBitVec_3 (UnboxBitVec3 x) == x)
I have now added a typing inversion axiom of the form, for constants N:
(forall ((x Term)) (implies (HasType x (FStar.BV.t N)) (is-BoxBitVec_N x)))

I also removed what seems like a dangerous axiom saying that the BoxBitVec constructor is fresh, i.e., we had Term_constr_id (BoxBitVec_N x) == 12 + n, where 12 was an arbitrary offset from some known constants. This is now gone.
Along the way, I refactored, without any semantic change, the API for encoding constructors to SMT.

everest green


Module Federation eager shared libs break anytime a remote build is requested from a host build

Bug report

What is the current behavior?
Module Federation eager shared libs break anytime a remote build is requested from a host build
If the current behavior is a bug, please provide the steps to reproduce.
example: https://github.com/rubixibuc/broken-mf-eager

create two module federation builds
in the first build, import any lib without first creating an async chunk. don't use import('./bootstrap')
in the first build, declare that lib as eager: true
in the second build, expose a module using module federation exposes
import([second build module]) somewhere inside the first MF build

everything works without step 5
technically everything works if build 1 and build 2 have no overlapping shareScope, but when the possibility exists that build 1 and build 2 can share a dependency nothing can successfully be eagerly loaded in build 1 - which seems to defeat the whole point of eager: true

It looks like eagerly loaded libs fail when there is a promise pending, and step 5 creates a pending promise that doesn't resolve in time.

What is the expected behavior?
I would expect that eagerly loaded lib to either be shared or superseded for the pending remotes version so that regardless of the two copies only one is actually executed.

Other relevant information:
webpack version:  5.72.1
Node.js version: 16.15.0
Operating System: MacOS 12.3.1
Additional tools: ModuleFederation Plugin

Do you get a warning in browser like “failed to init external” or something along those lines?

Do you get a warning in browser like “failed to init external” or something along those lines?

This is the generated error: Uncaught Error: Shared module is not available for eager consumption: webpack/sharing/consume/default/react/react

try removing unneeded plugins, try disabling webpack dev server package

I went through the source code when I originally reported this. There was a bug in the module resolving code. It could be fixed now, so I'll test with the latest version of webpack and report back

@rubixibuc Hello, did you test it? Thank you

@alexander-akait I haven't, and Ty for the reminder, will test this week

Still happening after bumping library versions
"@babel/core": "^7.21.3",
"@babel/preset-react": "^7.18.6",
"babel-loader": "^8.3.0",
"external-remotes-plugin": "^1.0.0",
"html-webpack-plugin": "^5.5.0",
"webpack": "^5.76.2",
"webpack-cli": "^4.10.0",
"webpack-dev-server": "^4.12.0"

@ScriptedAlchemy Are you able to confirm this config is either valid or invalid in the example repo:
https://github.com/rubixibuc/broken-mf-eager

I am still running into the same issue:
"webpack": "^5.85.1",
"webpack-bundle-analyzer": "^4.4.1",
"webpack-cli": "^5.1.3",
"webpack-dev-server": "^4.15.0"
I am attaching {eager: true} to every library it gives the error for.
For eg: I got the same error for "babel/ployfill". So I edited the webpack config as follows:
 new ModuleFederationPlugin({
name: "host",
filename: "remoteEntry.js",
remotes: {
   	remote:"remote@http://localhost:5500/remoteEntry.js"
},
 shared:{
 ...deps,
"babel-polyfill": { eager: true }
 }
 })

This fixed the issue for "babel-polyfill" library.
But then again, I got same error for "redux", "react", "prop-types" .. etc etc and I keep applying the above fix for all libraries.
Has anyone found the solution..? Please help!

Sorry, don't look at this currently, you can debug and try to find the root of the problem

so,  Anyone  can fix it , I also face the same issue with  vue,

Still happening after bumping library versions
"@babel/core": "^7.21.3",
"@babel/preset-react": "^7.18.6",
"babel-loader": "^8.3.0",
"external-remotes-plugin": "^1.0.0",
"html-webpack-plugin": "^5.5.0",
"webpack": "^5.76.2",
"webpack-cli": "^4.10.0",
"webpack-dev-server": "^4.12.0"

@ScriptedAlchemy Are you able to confirm this config is either valid or invalid in the example repo:
https://github.com/rubixibuc/broken-mf-eager

@rubixibuc Hello, did you test it? Thank you

@alexander-akait Are you running into a similar issue?

hi ,did you fix it , I also face the same issue like yours.  Although use by vue

Okay, I looked at https://github.com/rubixibuc/broken-mf-eager, and sorry you can't do it, because of Shared module is not available for eager consumption, shorty:
Here import:
import React from 'react'

but it can't be here, because:
new ModuleFederationPlugin({
  name: "app1",
  remotes: {
    app2<EMAIL_ADDRESS>  },
  shared: {react: {singleton: true, eager: true}, "react-dom": {singleton: true, eager: true}},
}),

webpack can't init shared module(s) in such case - module should be loaded once and share own state beween different application, there is an answer and solution https://webpack.js.org/concepts/module-federation/#troubleshooting:
You need to use and load them in bootstrap.js
import('./bootstrap.js');

we can't do it in the sync mode, sorry
It is not a bug, it works as expected
Also you have ExternalTemplateRemotesPlugin, but it is only for externals (module.constructor.name === 'ExternalModule' https://github.com/module-federation/external-remotes-plugin/blob/main/index.js#L17), there are not externals, you can write:
 externals: {
    react: "React",
    "react-dom": "ReactDOM",
  },

But in this case you need to have such externals.
So I don't see any bugs here, everything works as expected.
Anyway feel free to feedback


upgrade cypress 4.9.0 and image

Description
Warning, this cypress version will not support Node 8 anymore

update to new cypress 490: https://docs.cypress.io/guides/references/changelog.html#4-9-0
update image

Type of Change

Breaking change ('break', 'deprecate'): deprecate Node 8

I am seeing 'Cannot read property 'split' of undefined' on the icon tests. Is that a known issue, or potentially something wrong with my environment?

@Jskobos is correct, will require separate PR

The tests seem unchanged. Does the deep link smoke test take into account the new navbar position/markup? Some of those tests are failing, though they're failing on develop too.


Dynamic Max Height

Hi,
Is there any way to get a dynamic max height?
I would like my gantt to fill available space, and to use the gantt scrool bars to navigate it if it overflows.
If I set max height to false, the chart overflows and I get scrollbars on the window. This is inconvenient because I cant see the dates anymore when scrolling down,
Thank you.
Louis

using ResizeSensor and some javascript to adjust CSS height dynamically, it should be possible, but it's currently not supported.


Add some notes and examples to the MacOS build instructions.

This PR includes some additional information based on trying out the MacOS build instructions on my iMac platform.

@kgerheiser I am a software engineer in JCSDA working on @climbfuji's JEDI infrastructure team. I am helping test the spack-stack release candidate and want to add a few things to the MacOS instructions that I found during my testing.

Sorry, I just noticed that I have to go now. Thank you both for the reviews. I'll finish up my changes tomorrow and push them to github.

Let's remove the git-flow package from this PR and then it is good to go.

Done!


Text wrapping when word should be passed to a new line

I am seeing some unexpected text wrapping behaviour. I could not see any setting on how to control this (I was expecting the wrapped word to be moved down to the next line only wrapping words that can't fit on a single line). Have I missed something or is this a bug?

Not sure why this is happening, but line break specifications are in CSS.
Have you checked the CSS for the element to see what the value is?


 ›   Error: No assignees specified. in chgm

it seems the command pd incident:assign -i ${PD_ALERT} -u $PD_SILENT_TEST_USER fails :(
I'll investigate later

Solved upstream


Generate requirements.txt from IronPython does not work.

VS Version: 28523.274.d16.0stg
Python Tool Version: 16.0.19011.1
Repro steps:

Install IronPython (http://ironpython.net/download/ )
Create Python Application project
Add existing IronPython environment to python environment node
Right-click IronPython environment node and select Generate requirements.txt

Expect result:
requirements.txt can be generated in PE node and the file should include the packages and it's versions, like:

Actual result:
requirements.txt was been generated , but the contents of the file are incorrect

Closing this issue; we are not investing any more in IronPython support due to low usage and high cost.


Project setup

In this pull I set up the repo:

Create React/Redux App
Setup linters
Install MUI and MUI-Icons

Kindly Review it and share your feedback.

@Donard97  Thanks for the approval.


Performance : make CSV export way faster by not re-allocating and handling huge string

Types of changes
The change is to improve massively the problematic performance of the CSV node while exporting CSV from JS object
I consider this a bug fix due to the poor performance of the current implementation on fairly big payloads

[x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)

I did not take the time to create an actual ticket before-hand and apologize for that.
Proposed changes
By not using string.slice(0,-1) but rather joining smaller array that includes the separator only when required, the performance of the node is improved at by 2 to 3 order when handling big JS object
Checklist

[x] I have read the contribution guidelines
[ ] For non-bugfix PRs, I have discussed this change on the forum/slack team.
[x] I have run grunt to verify the unit tests pass
[ ] I have added suitable unit tests to cover the new/changed functionality

Performance number
Since this PR is about improving the node performance, we need to understand the actual performance of the node, and how it now compares before/after the fix.
Here is the flow used for the following data :
[[{"id":"f1d31f03.48da08","type":"inject","z":"4e2ac6b5.d48208","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"500","payload":"","payloadType":"date","x":360,"y":3360,"wires":[["c12bdabb.f06d7"]]},{"id":"c12bdabb.f06d7","type":"function","z":"4e2ac6b5.d48208","name":"build long array","func":"let payload = [\n    {\n        \"dbd5\": \"23080413430901\",\n        \"dat2\": \"ASTRG1\",\n        \"fhl9\": 1\n    },\n    {\n        \"dbd5\": \"23080413430901\",\n        \"dat2\": \"ASTRG2\",\n        \"fhl9\": \"010031\"\n    },\n    {\n        \"dbd5\": \"23080413430901\",\n        \"dat2\": \"ASTRG4\",\n        \"fhl9\": \"230804134309\"\n    },\n    {\n        \"dbd5\": \"23080413430901\",\n        \"dat2\": \"ASTRG5\",\n        \"fhl9\": 672\n    },\n    {\n        \"dbd5\": \"23080413430901\",\n        \"dat2\": \"ASTRG6\",\n        \"fhl9\": 1310\n    },\n    {\n        \"dbd5\": \"23080413430901\",\n        \"dat2\": \"ASTRG7\",\n        \"fhl9\": 0\n    },\n    {\n        \"dbd5\": \"23080413430901\",\n        \"dat2\": \"ASTRG8\",\n        \"fhl9\": \"C\"\n    },\n    {\n        \"dbd5\": \"23080413430901\",\n        \"dat2\": \"ASTRG3\",\n        \"fhl9\": 1\n    },\n    {\n        \"dbd5\": \"23080413431002\",\n        \"dat2\": \"ASTRG1\",\n        \"fhl9\": 2\n    },\n    {\n        \"dbd5\": \"23080413431002\",\n        \"dat2\": \"ASTRG2\",\n        \"fhl9\": \"010031\"\n    },\n    {\n        \"dbd5\": \"23080413431002\",\n        \"dat2\": \"ASTRG4\",\n        \"fhl9\": \"230804134310\"\n    },\n    {\n        \"dbd5\": \"23080413431002\",\n        \"dat2\": \"ASTRG5\",\n        \"fhl9\": 672\n    },\n    {\n        \"dbd5\": \"23080413431002\",\n        \"dat2\": \"ASTRG6\",\n        \"fhl9\": 1310\n    },\n    {\n        \"dbd5\": \"23080413431002\",\n        \"dat2\": \"ASTRG7\",\n        \"fhl9\": 0\n    },\n    {\n        \"dbd5\": \"23080413431002\",\n        \"dat2\": \"ASTRG8\",\n        \"fhl9\": \"C\"\n    },\n    {\n        \"dbd5\": \"23080413431002\",\n        \"dat2\": \"ASTRG3\",\n        \"fhl9\": 1\n    },\n    {\n        \"dbd5\": \"23080413431103\",\n        \"dat2\": \"ASTRG1\",\n        \"fhl9\": 3\n    },\n    {\n        \"dbd5\": \"23080413431103\",\n        \"dat2\": \"ASTRG2\",\n        \"fhl9\": \"010031\"\n    },\n    {\n        \"dbd5\": \"23080413431103\",\n        \"dat2\": \"ASTRG4\",\n        \"fhl9\": \"230804134311\"\n    },\n    {\n        \"dbd5\": \"23080413431103\",\n        \"dat2\": \"ASTRG5\",\n        \"fhl9\": 672\n    }\n]\n\nconst mult = msg.topic;\n\nlet array = [];\n\nfor (let index = 0; index < mult; index++) {\n    array.push(...payload)\n}\n\nmsg.payload = array;\nmsg.start = Date.now();\nnode.send(msg);","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":560,"y":3360,"wires":[["c0355eb0.128b88"]]},{"id":"48ae2f1b.c4f84","type":"comment","z":"4e2ac6b5.d48208","name":"500x","info":"","x":310,"y":3320,"wires":[]},{"id":"f0d849d8.3c5f","type":"inject","z":"4e2ac6b5.d48208","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"5000","payload":"","payloadType":"date","x":360,"y":3280,"wires":[["c12bdabb.f06d7"]]},{"id":"3222da4e.06b0f6","type":"comment","z":"4e2ac6b5.d48208","name":"5000x","info":"","x":310,"y":3240,"wires":[]},{"id":"c0355eb0.128b88","type":"csv","z":"4e2ac6b5.d48208","name":"","sep":",","hdrin":"","hdrout":"none","multi":"one","ret":"\\n","temp":"","skip":"0","strings":true,"include_empty_strings":"","include_null_values":"","x":730,"y":3360,"wires":[["95421023.bf6f78"]]},{"id":"95421023.bf6f78","type":"function","z":"4e2ac6b5.d48208","name":"to result","func":"msg.payload = {\n    duration : Date.now() - msg.start,\n    outputSizeMB : msg.payload.length / (1024*1024) \n}\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":880,"y":3360,"wires":[["72045619.792198"]]},{"id":"72045619.792198","type":"debug","z":"4e2ac6b5.d48208","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1050,"y":3360,"wires":[]}]

Note : additionnal multiplier of payload was used to have a more precise/broader view
The flow output nice informations :
{"duration":56,"outputSizeMB":2.5527000427246094}

duration is in ms and outputSizeMB is the length of the resulting string (in MB).
I know we're working with big payload here, but this never was an issue for really small payload (few hundreds bytes/kb are mostly "fine")
Here are the performance difference, the scale is log in both X and Y axis.
X is the size of the file in MB. (first point is at 0.23 which is 230kb)
Y is the duration reported in ms.

What this means is that for the same computationnal cost, we used to output a 230kb csv in around half a second (~380ms)
Now we can instead do it way faster (in the 10s of ms) or export a 23mb csv for the same duration.
In the graph we clearly see the scaling is different, that is because the slice is O(bytes) for each lines, meaning we're O(bytes*line), where the join is somewhat constant in speed because it depends only on the byte on the same line count, meaning the scaling is O(line)
Old node scaling being O(n^2):

New node scaling being mostly linear (and noisy cause the function is to fast)

Hi, thanks for taking the time to raise a PR around this.  Your discovery and work does a good job of pointing out the issue with recreating immutable strings and improves on that with the use of a string builder (an array).

I did not take the time to create an actual ticket before-hand and apologize for that.

It is quite unfortunate that I have plans to replace the CSV parsing with a much MUCH faster library that is also RFC 4180 compliant (this parser is has a number of compliance issues especially around sections 2.6 and 2.7) however this work could be a breaking change so is being held back for V4.x
I will leave this PR in open and pointed to the master branch as it may make it into a maintenance release.

Just for providing some context, at the company I work for we have a custom fork of node-red that is not open-source and I've done that fix 2 weeks ago (the graphs and all are from there, it just took me some time to have a flow that can be shared and to write down the stuff here), we are also not rebased upon recent releases. Like you've pointed out, this one is pretty nice cause it's totally a drop-in replacement with no breaking change, but I understand if it does not make its way to a proper release if you're already working on a better replacement.
We plan on sharing more of our changes if they properly fit back into node-red (and maybe discuss about the one who does not)
Plan is important there because we're pretty busy, we'll see how it goes :)
Anyway thanks for your time and have a nice day

Looks good, hopefully there will be a v3.1.1 fix release soon to pick up a few small fixes.
@Steve-Mcl can we put this on the list to include if/when please ? Are we using dev for 3.1.1 ? or master ?
Thanks

Dev is 4.0
Master for 3.1.x

Any performance improvement is welcome. Also gives us a better base line for any future rework of the node.


Image data

Hi, do you provide the original image data? Currently the hugging face only mentions the names of the image files, but it is not clear where to download the images. (https://huggingface.co/datasets/SNUMPR/realfred_json?row=0)
{ "high_idx": 0, "image_name": "000000000.png", "low_idx": 0 }

Hi @chiayewken
Thank you for having an interest in our work!
We currently do not provide the original image data. If you are looking to use the dataset for training purposes, such as ABP or MOCA, you can find the pre-generated ResNet features here.
If you need the images, you might want to check the code augment_trajectories.py, which could help you generate RGB images from traj_data.json files.
Please let me know if you have any further questions.
Thanks.

Closing this issue due to inactivity. Feel free to reopen this if you have any questions.


Added the same setting used in Crocoddyl manuscript + added building option for examples + fixed problem with building benchmarks - [merged]

In GitLab by @cmastalli on Sep 20, 2019, 13:41
Merges topic/clean-icra20-branch -> devel
This PR tackles the issues reported in #249, #244 and #248. Now we can run make commands for examples with and without display option, e.g.
make examples-quadrupedal-gaits

or
make examples-quadrupedal-gaits-display

fyi: @nim65s and @wxmerkt

In GitLab by @cmastalli on Sep 20, 2019, 13:43
The current PR points to my fork cmake submodule. I will switch to the main repo after merging this PR: https://github.com/jrl-umi3218/jrl-cmakemodules/pull/238

In GitLab by @cmastalli on Sep 20, 2019, 13:43
enabled an automatic merge when the pipeline for e0fdee54df8412add3cc034c74855d7d0bb2bc74 succeeds

In GitLab by @nim65s on Sep 20, 2019, 13:47
The CI won't pass until your commit is in the jrl-cmakemodule main repository. Meanwhile, you can update the url in .gitmodules

In GitLab by @nim65s on Sep 20, 2019, 13:47
Hum… Some jobs still succedded, you might just try again those who failed x)

In GitLab by @cmastalli on Sep 20, 2019, 15:01
marked as a Work In Progress

In GitLab by @cmastalli on Sep 20, 2019, 15:01
changed the description

In GitLab by @cmastalli on Sep 20, 2019, 15:02
No, they don't pass. I market as WIP. I will switch to the main repo whenever it's possible

In GitLab by @cmastalli on Sep 20, 2019, 17:51
added 3 commits
41204a1e - [bugfix] Fixed problem with logger612ee5ef - [utils] Added functions to plot the OC solution and solver convergence6d889a36 - [notebooks] Updated the introduction to crocoddyl notebook
Compare with previous version

In GitLab by @cmastalli on Sep 23, 2019, 14:10
added 7 commits
30d6868d - [solver] Do not compute gains when nu <= 05b06318e - [utils] Adde the impulse model option for the quadruped case1dd9fd68 - [utils] Included the jumping problem generator for the quadruped cases4417d280 - [examples] Updated the quadruped example, using FDDP solver + lateral jumpdbdefa59 - [cmake] Needed Pinocchio 2.1.7 for examplescddfc724 - [log] Renamed properly this log filea7f450d8 - [cleanup] Reformatted the code
Compare with previous version

In GitLab by @cmastalli on Sep 23, 2019, 14:12
added 1 commit
5c79b184 - [cmake] Updated cmake module
Compare with previous version

In GitLab by @cmastalli on Sep 23, 2019, 16:37
added 5 commits
53f6da3f - [notebook] Solved an issue, changed x, xdot to y and ydot9b66243d - [utils] Created the jumping problem and added the impulse1456f5df - [cleanup] Renamed names by impulse in quadruped utilsd6f924ca - [cleanup] Tiny clean up of FDDP bindingsb153d687 - [efficiency][solver] Used Eigen dot and noalias in expected model computation for FDDP solver
Compare with previous version

In GitLab by @cmastalli on Sep 24, 2019, 12:08
added 2 commits
19d96598 - [utils] Added D gain in quadruped example (Baugarmter stabilization)eea933b7 - [solver][bugfix] It was used the backward and forward from DDP
Compare with previous version

In GitLab by @cmastalli on Sep 24, 2019, 12:34
added 3 commits
6763514a - [Examples] show display / plot with environment variablesc86baf16 - [Readme] use fenced code blocksbfb0527e - Merge branch 'topic/clean-icra20-branch' into 'topic/clean-icra20-branch'
Compare with previous version

In GitLab by @cmastalli on Sep 24, 2019, 12:55
added 1 commit
5a454176 - [bugfix] Error in the dimension of q in legacy unittest + use StdVec_Force
Compare with previous version

In GitLab by @cmastalli on Sep 24, 2019, 18:44
added 6 commits
ffec5d1d - [solver] Relinearize Vxx as done in the FDDP (efficiency)7d325880 - [solver] Missed a step in the relinearizationd03c738d - [solver] Do not need to rewrite computeDirection (same for DDP and FDDP)4da1d5cd - [bindings] Updated the SolverFDDP bindings591fcb08 - [cleanup] Formatted the code7c90279f - [bindings] Exposed parameters of FDDP and DDP solvers
Compare with previous version

In GitLab by @cmastalli on Sep 25, 2019, 08:26
added 1 commit
c3552271 - [unittest] The previous backward method wrongly doesn't update Vx since this...
Compare with previous version

In GitLab by @cmastalli on Sep 25, 2019, 15:16
added 13 commits
9c9b3b96 - [bindings] Used iterator in vector to list converter37e1f0ab - [bindings] Added Python list to std::vector<double> converter (missed the other way around)5eedfea3 - [cleanup] Removed explanation lines inside Python utils code for DDP2e0061e2 - [unittest] Added unittest for FDDP solverac1fc98a - [solver] Added computeDirection in FDDP since the calc function is different93b2bc68 - [utils] FDDPDerived is derived from DDPDerived340d965a - [example] Updated the log file for the quadrupedal example32639217 - [utils] Renamed Callbacks functions (removed Solver keyword)e4bdd761 - [utils] Renamed function as plotConvergence9c02f2b0 - [solver] Created a function to retrieved the callback functions in each solver0a96d40e - [notebook] Adapted the unicycle scripta2f8dc31 - [notebook] Fixed the bipedal walking notebook9293c299 - [notebooks] Renamed the files
Compare with previous version

In GitLab by @cmastalli on Sep 25, 2019, 16:39
added 6 commits
9f1a70bd - [utils] Accepted nu=0 for plotting in quadruped and biped scripts25cc5d29 - [utils] Render plots as done in the biped case3b711642 - [cmake] Remove command to display examplesdef8a714 - [utils] Added option to pass figure title in plotConvergence4e9339fa - [examples] Plotting also the convergence of each optimization problem4affee53 - [readme] Fixed grammar mistakes + added info about benchmarks + added credits and acknowledge
Compare with previous version

In GitLab by @cmastalli on Sep 25, 2019, 17:13
added 1 commit
a7cfe3ce - [utils] Used the proper robot
Compare with previous version

In GitLab by @cmastalli on Sep 25, 2019, 17:56
Ok, I enlisted the performed task in the commits:

Fixed an bug in the SolverFDDP
Fixed problems in various notebooks
Updated description in README
Exposed parameters in Python of FDDP and DDP solvers
Cleaned up and renamed examples and notebooks
Improved the plot functions
Added Python unit-test for FDDP

There is still needed extra work for merging humanoid jumps.

In GitLab by @cmastalli on Sep 25, 2019, 17:56
unmarked as a Work In Progress

In GitLab by @cmastalli on Sep 25, 2019, 17:56
merged

In GitLab by @cmastalli on Sep 25, 2019, 17:56
mentioned in commit 47f7b7cf10470ca66ef156499f8da4dc91c44699


Allow to remove items from recent project list

I'm trying to add remove buttons for 'Recent Project' list and I want to show some variants for UI. Also I want to add 'Clear list' label at the end of the list, for remove all projects from the list.
First variant is just labels without any alignment (default)

Second variant is labels with left alignment

Third variant is button without alignment for project name label

And fourth variant is button with left alignment for project name label

Let me know if is there some better solutions.

Fourth looks nice, I would also remove background drawable from close
button (so only X icon is visible), over and and down drawable should be
left. I also have one suggestion, all those buttons kinda clutter UI, it
would be nice if X button would only appear if you hover mouse over project
name.
sob., 12 gru 2015, 10:11 Pavel Puchok użytkownik<EMAIL_ADDRESS>napisał:

I'm trying to add remove buttons for 'Recent Project' list and I want to
show some variants for UI. Also I want to add 'Clear list' label at the end
of the list, for remove all projects from the list.
First variant is just labels without any alignment (default)
[image: label_center_aligment]
https://cloud.githubusercontent.com/assets/4688310/11761168/9cb48e7a-a0c8-11e5-8ac6-c55ccaf86ddd.png
Second variant is labels with left alignment
[image: label_left_aligment]
https://cloud.githubusercontent.com/assets/4688310/11761169/a27dabb6-a0c8-11e5-9a80-55655d5d0cc9.png
Third variant is button without alignment for project name label
[image: button_label_center_aligment]
https://cloud.githubusercontent.com/assets/4688310/11761159/1e7bc924-a0c8-11e5-87d1-28272344d999.png
And fourth variant is button with left alignment for project name label
[image: button_label_left_aligment]
https://cloud.githubusercontent.com/assets/4688310/11761171/aa13da58-a0c8-11e5-9d87-f6266ab3a2e7.png
Let me know if is there some better solutions.
—
Reply to this email directly or view it on GitHub
https://github.com/kotcrab/VisEditor/issues/67#issuecomment-164127643.

How I can remove background for up button state? This is can be done without change uiskin?
I found some places where background of up state is not visible (for ex: toolbar icons), but in all this places background of parent has the same color with up state of buttons. So is invisible just for this reason.

Something like this should work:
VisImageButtonStyle style = new VisImageButtonStyle(VisUI.getSkin().get("close", VisImageButtonStyle.class));
style.up = null;
add(new VisImageButton(style));

This works, but button begin jumping under mouse cursor.
It seems like button in Up state have different size with Over/Down states. While state changes from Up to Over, "X" icon moved little bit to right and up, also all parent table cells change own sizes.

I see, well then don't add it. It will probably require new drawables so I'll play with it once it's merged.

Style fixed, all done.


[HELP]

Is this still working as of today? cant seem to get it to work despite it injecting

Hello, yes it’s still working today. Can you provide more informations about your problem and send also some screen. Ensure to activate logs and title injection and also show me the console by pressing f12 then console.

Hello. The code doesn't inject in Quiz Review. In early April It did, but now it doesn't. I think moodle changed something. Injecting in review allows to test the accuracy the GPT. The real quiz works in test moodle. I tried on my real institution moodle and test one. The console is blank, so I can't share anything.

Hello, yes I removed the fact you can inject it in review.

It ended up being a conflict between other chrome extentions i had enabled, once i disabled them all it started working properly. Thank you very much for the response and the extension

As a developer it is your choice to decide. However, the preview inject allowed to test some functions with specific questions. I do not rely on moodle test page. For example, this time I needed to test some features, as not titling(inject title), not cursor indicating. I wish all will be working correctly my next quiz.

Thx. I’ll see if I’ll implement it back :)
If the issues is complete for you then close it please.

Is there any way to make the code be less than 3 letters?

Not for now but I can make it in the next update.

It'd be great if it could be 1-3 characters. Thanks for the response

To avoid any problems I’ll set it to 2


Ensure license headers are applied to all files

Per https://github.com/kubernetes/community/blob/master/github-management/kubernetes-repositories.md#rules-for-donated-repositories
We can add something like https://github.com/kubernetes-sigs/metrics-server/blob/f0ba7343dfe50be509b419b21db014e4949d73d9/Makefile#L196-L212 to the Makefile and include it in a verify target for CI.

/kind cleanup
ht @dgrisonnet

/triage accepted
/assign @ehashman


opaquely paged blob listing

Provide a blobstore iterator over blob keys that can be continued using an opaque page token and marker (blob key).
Token handing is backgrounded unless explicit continuation is needed.
Iterator will visit every key (as filtered by prefix/delimiter).

@xbrianh can you respond to the comment?  after that, I'll merge.


Add Dockerfile

Bypass the custom build script and use a Dockerfile to build the service.
The docker file is based on Alpine and it's much smaller image than golang:1.6-wheezy

The main point of the build script is that the binary only is what ends up in the final docker-image. The problem with the alpine and other containers is you end up with a multilayered image which contains the entire golang build infrastructure as well.

The new docker build changes + CI of the image with travis take care of this.


Creating bulks

Dear Open-Catalyst team,
First, I want to thank you for your nice datasets available and for the workflows you published here. I understood that this workflow here is able to get me the input files for VASP calculations, whose results are in the OC20 dataset. However, I am struggling to understand, how to copy your workflow for a completely different bulks, which are not in the dataset and which I also want to explore (using the same qualities as your datasets ... ).
As far as I can understand the already RPBE optimized bulks are stored in one of the pickle files (e.g.  https://dl.fbaipublicfiles.com/opencatalystproject/data/input_generation/bulk_db_nelems_2021sep20.pkl ). But I cannot see here any automatized workflow how to start from whatever you start for the bulk (mp number/bulk coordinates/ase object ...), through bulk optimization to a new pkl file. Please, could you provide me with some more details?
Thank you very much! Ondrej

With the new version (at the moment in the refactor-v2). This passing ASE objects is working fine. Here is one of my examples:
from ase.io import read
from ocdata.core import Adsorbate, Slab, Bulk, AdsorbateSlabConfig

adsorbate = Adsorbate(adsorbate_smiles_from_db = "*H")
bulk_atoms = read('new_materials_rpbe/ZnO_mp-2133/OUTCAR')
bulk = Bulk(bulk_atoms=bulk_atoms)
slabs100 = Slab.from_bulk_get_specific_millers(specific_millers = (1,0,0), bulk=bulk)


Issue with k8s.io/docs/reference/command-line-tools-reference/kubelet/

This is a...

[ ] Feature Request
[ ] Bug Report

Problem:
Proposed Solution:
Page to Update:
https://kubernetes.io/...

@kushortiz does this PR fix your issue? https://github.com/kubernetes/website/pull/9344 - please provide more information if not. Thanks!

Closing in favor of issue #9413 to avoid confusion.


Add server-side validation to conditional fields

…if a containing checkbox is selected (i.e. present in form.cleaned_data), contained subfields become required so that it's not only JS that can ensure they work correctly
TODO: perhaps these conditional fields need to remove their subfields' values from self.cleaned_data if not selected in case javascript is not enabled. The situation being: something is entered into a subfield but the containing checkbox is not checked. With javascript, the subfields become disabled so are not sent to the backend so are effectively treated as empty. Without javascript, the subfields remain enabled so backend will treat it as though the checkbox was selected.

this was just a rebase off master branch

yup that's all i did


Install and compile<EMAIL_ADDRESS>package com.alibaba.json.bvtVO.???????? does not exist

see:
localhost:~ # spack install --keep-stage -v fastjson
[+] /home/spack/opt/spack/linux-sles15-aarch64/gcc-7.4.0/openjdk-1.8.0_191-b12-efygw4r7e2ocitlwttcp2yn6beizzz5c
[+] /home/spack/opt/spack/linux-sles15-aarch64/gcc-7.4.0/maven-3.6.3-nlllqprafuwb2ob66z6e2edp6nlsbvnx
==> Installing fastjson-1.2.76-z7mzhzejy3apulb24jqt6oqixpvlqkwv
==> No binary for fastjson-1.2.76-z7mzhzejy3apulb24jqt6oqixpvlqkwv found: installing from source
==> Using cached archive: /home/spack/var/spack/cache/_source-cache/archive/c7/c707eb3d3c809c30051583e46a38e9dc06c8941219161bd1bb7a9d6bb6ad1fbf.tar.gz
==> fastjson: Executing phase: 'build'
==> [2021-04-06-17:37:57.253972] '/home/spack/opt/spack/linux-sles15-aarch64/gcc-7.4.0/maven-3.6.3-nlllqprafuwb2ob66z6e2edp6nlsbvnx/bin/mvn' 'package' '-DskipTests'
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------< com.alibaba:fastjson >------------------------
[INFO] Building fastjson 1.2.76
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ fastjson ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 6 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ fastjson ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 190 source files to /home/stage/root/spack-stage-fastjson-1.2.76-z7mzhzejy3apulb24jqt6oqixpvlqkwv/spack-src/target/classes
[INFO] /home/stage/root/spack-stage-fastjson-1.2.76-z7mzhzejy3apulb24jqt6oqixpvlqkwv/spack-src/src/main/java/com/alibaba/fastjson/util/TypeUtils.java: Some input files use or override a deprecated API.
[INFO] /home/stage/root/spack-stage-fastjson-1.2.76-z7mzhzejy3apulb24jqt6oqixpvlqkwv/spack-src/src/main/java/com/alibaba/fastjson/util/TypeUtils.java: Recompile with -Xlint:deprecation for details.
[INFO] /home/stage/root/spack-stage-fastjson-1.2.76-z7mzhzejy3apulb24jqt6oqixpvlqkwv/spack-src/src/main/java/com/alibaba/fastjson/parser/DefaultJSONParser.java: Some input files use unchecked or unsafe operations.
[INFO] /home/stage/root/spack-stage-fastjson-1.2.76-z7mzhzejy3apulb24jqt6oqixpvlqkwv/spack-src/src/main/java/com/alibaba/fastjson/parser/DefaultJSONParser.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ fastjson ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 98 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ fastjson ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2806 source files to /home/stage/root/spack-stage-fastjson-1.2.76-z7mzhzejy3apulb24jqt6oqixpvlqkwv/spack-src/target/test-classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/stage/root/spack-stage-fastjson-1.2.76-z7mzhzejy3apulb24jqt6oqixpvlqkwv/spack-src/src/test/java/com/alibaba/json/bvt/issue_3100/Issue3132.java:[4,39] package com.alibaba.json.bvtVO.???????? does not exist
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  17.871 s
[INFO] Finished at: 2021-04-06T17:38:16+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project fastjson: Compilation failure
[ERROR] /home/stage/root/spack-stage-fastjson-1.2.76-z7mzhzejy3apulb24jqt6oqixpvlqkwv/spack-src/src/test/java/com/alibaba/json/bvt/issue_3100/Issue3132.java:[4,39] package com.alibaba.json.bvtVO.???????? does not exist
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
==> Error: ProcessError: Command exited with status 1:
    '/home/spack/opt/spack/linux-sles15-aarch64/gcc-7.4.0/maven-3.6.3-nlllqprafuwb2ob66z6e2edp6nlsbvnx/bin/mvn' 'package' '-DskipTests'

12 errors found in build log:
     23    [INFO] Copying 98 resources
     24    [INFO]
     25    [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ fastjson ---
     26    [INFO] Changes detected - recompiling the module!
     27    [INFO] Compiling 2806 source files to /home/stage/root/spack-stage-fastjson-1.2.76-z7mzhzejy3apulb24jqt6oqixpvlqkwv/spack-src/target/test-classes
     28    [INFO] -------------------------------------------------------------
  >> 29    [ERROR] COMPILATION ERROR :
     30    [INFO] -------------------------------------------------------------
  >> 31    [ERROR] /home/stage/root/spack-stage-fastjson-1.2.76-z7mzhzejy3apulb24jqt6oqixpvlqkwv/spack-src/src/test/java/com/alibaba/json/bvt/issue_3100/Issue3132.java:[4,39] package com.alibab
           a.json.bvtVO.???????? does not exist
     32    [INFO] 1 error
     33    [INFO] -------------------------------------------------------------
     34    [INFO] ------------------------------------------------------------------------
     35    [INFO] BUILD FAILURE
     36    [INFO] ------------------------------------------------------------------------
     37    [INFO] Total time:  17.871 s
     38    [INFO] Finished at: 2021-04-06T17:38:16+08:00
     39    [INFO] ------------------------------------------------------------------------
  >> 40    [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project fastjson: Compilation failure
  >> 41    [ERROR] /home/stage/root/spack-stage-fastjson-1.2.76-z7mzhzejy3apulb24jqt6oqixpvlqkwv/spack-src/src/test/java/com/alibaba/json/bvt/issue_3100/Issue3132.java:[4,39] package com.alibab
           a.json.bvtVO.???????? does not exist
  >> 42    [ERROR]
  >> 43    [ERROR] -> [Help 1]
  >> 44    [ERROR]
  >> 45    [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
  >> 46    [ERROR] Re-run Maven using the -X switch to enable full debug logging.
  >> 47    [ERROR]
  >> 48    [ERROR] For more information about the errors and possible solutions, please read the following articles:
  >> 49    [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

See build log for details:
  /home/stage/root/spack-stage-fastjson-1.2.76-z7mzhzejy3apulb24jqt6oqixpvlqkwv/spack-build-out.txt

When I install and compile<EMAIL_ADDRESS>in the sppack, an error occurs: package com.alibaba.json.bvtVO.???????? does not exist. What can I do?
Information on your system
localhost:~ # spack debug report

Spack: 0.16.0-77-98460b6f4
Python: 3.6.5
Platform: linux-sles15-aarch64
Concretizer: original


Edge to edge in main branch

PR to test status bar transparency.
Approved by SME.

Added new E2E API

Multiple issues still exist that were noted on other PRs and not addressed.
As discussed please review E2E specific issues.


Rescue exception in worker (now with fixed rubocop warning)

The commit for issue #741 did not fix the problem of delayed_job worker process crashing on exceptions of type Exception for me.
If I change the sample job to raise an Exception instead of StandardError the delayed_job tests break...
I think the worker has also to rescue exceptions of class Exception.

Coverage decreased (-0.22%) to 90.7% when pulling 703d28c3ac973d2a5c6ebeb076046f2f04dc5a8a on elitau:rescue_exception_in_worker into ce88693429188a63793b16daaab67056a4e4e0bf on collectiveidea:master.

The remaining rubocop offenses were present before my changes.

Please URGENTLY merge this.
This is an absolutely critical change to add. DelayedJob MUST rescue Exception, this is very bad whoever made this change. Our delayed job broke in production due to this, 1000+ jobs and lots of clients were affected. Not cool at all.

@albus522 Now that you’ve merged this can we release version 4.1.0?

@sferik go for it

I think this PR should be merged: https://github.com/collectiveidea/delayed_job/pull/846


Configuring silent-refresh asset on Angular 4.4.6's webpack

I'm getting this error for silent refresh 'silent_refresh_timeout'.
"assets": [
  "src/silent-refresh.html"
],

How I can configure the same for Angular 4.4.6 version which doesn't have .angular-cli.json?

I recommend turning to Stack Overflow or similar for more generic questions (e.g. how to serve a static asset in an older version of Angular), where there's more people with time to answer questions.

@jeroenheijmans As you told I have searched a lot, I'm not able to find a such configuration for static html file in Angular 4. If you have noticed any such configuration guide lines in online please share the link.

You need to put it in the "assets", example config file:
{
  "$schema"<EMAIL_ADDRESS>  "project": {
    "name": "app"
  },
  "apps": [
    {
      "root": "src",
      "outDir": "dist",
      "assets": [
        "i18n",
        {"glob": "**/i18n/*.json", "input": "./app"},
        "favicon.ico",
        "silent_refresh.html"
      ],
      "index": "index.html",
      "main": "main.ts",
      "polyfills": "polyfills.ts",
      "test": "test.ts",
      "tsconfig": "tsconfig.app.json",
      "testTsconfig": "tsconfig.spec.json",
      "prefix": "jsvs",
      "styles": [
        "styles.scss"
      ],
      "scripts": [],
      "environmentSource": "environments/environment.ts",
      "environments": {
        "dev": "environments/environment.ts",
        "prod": "environments/environment.prod.ts"
      }
    }
  ],
  "lint": [
    {
      "project": "src/tsconfig.app.json",
      "exclude": "**/node_modules/**"
    },
    {
      "project": "src/tsconfig.spec.json",
      "exclude": "**/node_modules/**"
    }
  ],
  "test": {
    "karma": {
      "config": "./karma.conf.js"
    }
  },
  "defaults": {
    "styleExt": "scss",
    "component": {},
    "serve": {
      "port": 8080
    }
  }
}

Hoping/assuming things got resolved in the mean time. Please open up a fresh issue with a repro or perhaps consider posting a Stack Overflow question to get further assistance.

This config file never pat of angular 4 version, latest version of angular has .angular-cli.json. And I don't think so there is  a proper documentation for implementing  angular-oauth2-oidc on Angular 4 version


Node Capacity/Allocatable of VF num NOT correct

What happened?
Node Capacity/Allocatable of VF num NOT correct
Capacity:
  cpu:                        128
  ephemeral-storage:          256981444Ki
  hugepages-1Gi:              0
  hugepages-2Mi:              0
  memory:<PHONE_NUMBER>Ki
  openshift.io/ibcx6vfnuma0:  7
  openshift.io/ibcx6vfnuma1:  7
  pods:                       110
Allocatable:
  cpu:                        127600m
  ephemeral-storage:          256981444Ki
  hugepages-1Gi:              0
  hugepages-2Mi:              0
  memory:<PHONE_NUMBER>794
  openshift.io/ibcx6vfnuma0:  7
  openshift.io/ibcx6vfnuma1:  7
  pods:                       110

$ kubectl get SriovNetworkNodePolicy ibcx6vfnuma0 -n sriov-network-operator -oyaml
apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodePolicy
metadata:
  annotations:
  generation: 1
  name: ibcx6vfnuma0
  namespace: sriov-network-operator
spec:
  deviceType: netdevice
  isRdma: true
  linkType: IB
  nicSelector:
    deviceID: 101b
    rootDevices:
    - "0000:12:00.0"
    vendor: 15b3
  nodeSelector:
    feature.node.kubernetes.io/ib-network-sriov.capable: "true"
  numVfs: 8
  priority: 10
  resourceName: ibcx6vfnuma0

What did you expect to happen?
Expected numVfs is : 8
What are the minimal steps needed to reproduce the bug?
Step 1: Deploy sriov-network-operator 1.2.1
Step 2: Deploy a payload:
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: myubt
  name: myubt
  namespace: test1
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: myubt
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      annotations:
        k8s.v1.cni.cncf.io/networks: sriov-network-operator/ibcx6vfnuma0,sriov-network-operator/ibcx6vfnuma1
      creationTimestamp: null
      labels:
        app: myubt
    spec:
      containers:
      - args: [ "while true; do sleep 30; done;" ]
        command: [ "/bin/bash", "-c", "--" ]
        image: ubuntu:20.04
        imagePullPolicy: IfNotPresent
        name: myubt
        resources:
          limits:
            cpu: "1"
            memory: 4Gi
            openshift.io/ibcx6vfnuma0: "1"
            openshift.io/ibcx6vfnuma1: "1"
          requests:
            cpu: "1"
            memory: 4Gi
            openshift.io/ibcx6vfnuma0: "1"
            openshift.io/ibcx6vfnuma1: "1"
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      hostPID: true
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: default
      serviceAccountName: default
      terminationGracePeriodSeconds: 30

Step 3: Delete the sriov-device-plugin-7wkm8 pod of node on which step 2 payload is running on.
kubectl delete po sriov-device-plugin-7wkm8 -n sriov-network-operator

Then, the sriov-device-plugin daemonset will auto launch a new pod you just deleted.
Step 4: Now the node Capacity/Allocatable of VF num NOT correct, show me 7, NOT 8. Why?
Anything else we need to know?
Yeah!
I add some debug code to sriov-network-device-plugin,
It's filterd by linkTypes list, code location: pkg/netdevice/netDeviceProvider.go#GetFilteredDevices()
and, pkg/resources/deviceSelectors.go#func (s *linkTypeSelector) Filter, linkType := dev.(types.PciNetDevice).GetLinkType(), linkType is empty.
And, the final root cause is pkg/netdevice/pciNetDevice.go#func NewPciNetDevice(...)
	pciAddr := pciDev.GetPciAddr()
	glog.Infof("...pciAddr: %s", pciAddr)
	netDevs, _ := utils.GetNetNames(pciAddr)
	if len(netDevs) == 0 {
		ifName = ""
	} else {
		ifName = netDevs[0]
	}

The len(netDevs) == 0, because the /sys/bus/pci/devices/0000:12:00.5/net directory is empty.
# pwd
/sys/bus/pci/devices/0000:12:00.5/net
# ls -al
total 0
drwxr-xr-x  3 root root 0 Dec 10 22:06 .
drwxr-xr-x 12 root root 0 Dec 10 22:06 ..

and other vf director is seems OK, like below:
# pwd
/sys/bus/pci/devices/0000:12:00.4/net
# ls
ibs2v3
# ls ibs2v3/
addr_assign_type  broadcast        carrier_down_count  delete_child  device   flags              ifindex    mode              netdev_group  phys_port_name  power       set_mac     subsystem     uevent
addr_len          carrier          carrier_up_count    dev_id        dormant  gro_flush_timeout  iflink     mtu               operstate     phys_switch_id  proto_down  speed       tx_queue_len  umcast
address           carrier_changes  create_child        dev_port      duplex   ifalias            link_mode  name_assign_type  phys_port_id  pkey            queues      statistics  type

And if I removed the spec.template.metadata.annotations
k8s.v1.cni.cncf.io/networks: sriov-network-operator/ibcx6vfnuma0,sriov-network-operator/ibcx6vfnuma1

And repeat the step 1,2,3,4..., Everything is OK, the vf number is 8, NOT 7.
So, My Question is, what's the difference between add k8s.v1.cni.cncf.io/networks to annotations and NOT.
So, My Question is, what's the difference between add k8s.v1.cni.cncf.io/networks to annotations and NOT.
So, My Question is, what's the difference between add k8s.v1.cni.cncf.io/networks to annotations and NOT.
Component Versions
Please fill in the below table with the version numbers of components used.

Component
Version

SR-IOV Network Device Plugin
3.5.1

SR-IOV CNI Plugin
sriovCni:v2.6.3; ibSriovCni: v1.0.2

Multus
Not sure

Kubernetes
v1.23.10

OS
Ubuntu 20.04 Kernel 5.4.0-42-generic

sriov-operator
v1.2.0

Config Files
Config file locations may be config dependent.
Device pool config file location (Try '/etc/pcidp/config.json')
{
  "resourceList": [
    {
      "resourceName": "ibcx6vfnuma0",
      "selectors": {
        "vendors": [
          "15b3"
        ],
        "devices": [
          "101c"
        ],
        "rootDevices": [
          "0000:12:00.0"
        ],
        "linkTypes": [
          "infiniband"
        ],
        "IsRdma": true,
        "NeedVhostNet": false
      },
      "SelectorObj": null
    },
    {
      "resourceName": "ibcx6vfnuma1",
      "selectors": {
        "vendors": [
          "15b3"
        ],
        "devices": [
          "101c"
        ],
        "rootDevices": [
          "0000:33:00.0"
        ],
        "linkTypes": [
          "infiniband"
        ],
        "IsRdma": true,
        "NeedVhostNet": false
      },
      "SelectorObj": null
    }
  ]
}

Multus config (Try '/etc/cni/multus/net.d')
CNI config (Try '/etc/cni/net.d/')
Kubernetes deployment type ( Bare Metal, Kubeadm etc.)
Kubeconfig file
SR-IOV Network Custom Resource Definition
Logs
SR-IOV Network Device Plugin Logs (use kubectl logs $PODNAME)
Multus logs (If enabled. Try '/var/log/multus.log' )
Kubelet logs (journalctl -u kubelet)

yep that was what I was going to comment that we fix that image around the link type

can I close this issue?

can I close this issue?

Yeah, thanks a lot.


SQLSRV - output parameters are not returned from stored procedures with PRINT commands inside

Just started to migrate my projects from PHP 5.6/Freetds driver to PHP 7.3.8/SQLSRV laters driver (5.6.1), Odbc 17. The SQL Server version is 2008 R2 SP2.
The output parameter variables are not populated, if there are print functions inside an executed stored procedure, although the sqlsrv_query command succeeds.
 Also, there is a great chance that programmers leave print commands uncommented. Other drivers (freetds, mssql ado and ado.net drivers etc.) discard print messages and return output parameter values correctly.

Please provide a repro script.
Also, did you call sqlsrv_next_result?

No, but the procedures do not return any result sets (select data). Are print messages treated as result sets?

What I have figured out is:

for numeric parameters: output parameter values are not returned from a stored procedure with print messages if the parameter variable was initialized with the null value. They are returned either if there are no print messages or the parameter variable was initialized with a value other than null.
for varchar parameters: output parameter values are not populated if there are print messages. The sqlsrv_next_result function does not help.

I have figured it out. I had to put "SET NOCOUNT ON" first after a parameter declaration. Without it, if I use the PRINT command inside a procedure, it interferes with the sqlsrv varchar output parameters. If I put "SET NOCOUNT ON", output varchar parameters are returned correctly.


can't upload to disks with s3 driver like minio

when i try to upload to a s3 compatible disk i receive the above error, i've been looking all over for any additional config that has to be done to make it work but to no avail.
this is my disk:
'minio' => [             'driver' => 's3',             'key' => env('MINIO_ACCESS_KEY'),             'secret' => env('MINIO_SECRET_KEY'),             'region' => env('MINIO_REGION', 'us-east-1'),             'bucket' => env('MINIO_DEFAULT_BUCKET', 'images'),             'endpoint' => env('MINIO_ENDPOINT', 'http://minio:9000'),             'url' => env('MINIO_URL', 'http://minio:9000'),             'use_path_style_endpoint'=>true,             'throw'=>true         ],
and in my panel i call the library like this:
 FilamentImageLibraryPlugin::make()                 ->addAllowedDisk('minio','Thư viện hình ảnh')                 ->navigationSort(14),
any help will be very appreciated.

Hi @tungnguyenls2008 ,
I will try to replicate this issue and get to a fix as soon as possible.


add Mysql gui

ℹ Repository information
The repository has:

[x] At least three issues with the good first issue label.
[ ] At least 10 contributors. (new project for new comers)
[x] Detailed setup instructions for the project.
[x] CONTRIBUTING.md
[x] Actively maintained.

Hi @sanket-deepsource / @jai-deepsource / @aman-deepsource / @ishan-deepsource / @rahul-deepsource / @rohan-deepsource,
Any update on merging this request, Can you please let me know if anything more is required. Thank you for you time and consideration

Thank you @jai-deepsource


Allow Coercion in Set() for numbers.

-Fixes #1659 #1669

✅ No public API change.

✅ No public API change.

✅ No public API change.

✅ No public API change.

✅ No public API change.

✅ No public API change.


Making sinopia setup easier for the user

Simply installing sinopia does not create a config file (or even a
directory when it should be placed). Running sinopia for the first time
generates a default config file so it’s easier for the user to
configure it properly.

@mkonicek LGTM, but leaving the meta stuff for your approval.

Thanks @radko93. Small nit.

Ok, resolved @mkonicek!

@facebook-github-bot shipit

@mkonicek "an unpleasant but necessary task." makes 💯 sense


Failed Permanently

Every time the server runs the "flush message queue" script the queue record status is "2:Failed Permantly" - but if I manually run the script it works fine.  Anyone have any suggestions here?

Got it...missed this issue in another thread.  account name and password issue.


Bump redis from 3.5.3 to 4.4.4 in /lambda_functions/v1/requirements

Bumps redis from 3.5.3 to 4.4.4.

Release notes
Changelog
Commits

updated-dependencies:

dependency-name: redis dependency-type: direct:production ...

Moved from dependabot PR.

@dependabot rebase

dependabot no longer has permissions to build this. I have done a new PR as myself : https://github.com/ministryofjustice/opg-data-lpa/pull/103. which does the same as this


More Spanner investigation

This enables the three parts of the tool (database creation, table
creation, row insertion) to be independently configured to be
synchronous or asynchronous.

Thinking about this more, I could probably make a single method in each case which optionally runs asynchronously (but is always an async method). Preferences?

Leave it as it is for now. It's simple to change later if it becomes easier to do so.


First non-regression tests

Motivations:

Since PR #62 we now have a simple mechanism to save output data and to check whether 2 instances are identical
The basis are now available to start implementing non-regression tests (non-regression in terms of output data)
Principle: we include a set of reference output, for all models and solvers in the unit tests, and we use them to check that the output does not change as the code evolves (to make sure we did not introduce a bug)
If the output changes for a good reason (e.g.: because we introduced an improvement in the accuracy / stability of a solver), the unit tests will fail (which will indicate the improvement is significant), and we will simply update the reference output.

Proposition:

create a output_ref/ folder in the unit tests to hold the reference output, and use it for checking

They are coded, but the system is not ready yet (solvers not working properly). This will be reactivated once #58 is done.


Add support for PRs with more than 30 reviews

This PR adds support for pagination which is required for this action to work properly on PRs with 31+ reviews.

@ekimia Thank you for your contribution! It would be helpful if you could fix the lint error.

my pleasure! I'll take care of the lint errors next week.

@ekimia
I'll merge it once here, and I'll fix any lint errors, etc. on my end. Thank you for your contribution!


Fix automatic upgrades

Use the docker_image resource to notify osl_dockercompose to rebuild/restart the
containers if there is a new image.
Signed-off-by: Lance Albertson<EMAIL_ADDRESS>

!bump patch ~