Validate and document Hardened Imported RKE2 Cluster upgrade from 1.24 to 1.25

As part of validating support for k8s 1.25 support we want to ensure hardened imported RKE2 clusters can be upgraded from k8s <=1.24 to 1.25.
The purpose of this "task" issue is to verify/refine/document the steps outlined below. The "document" part can be done as a comment on this issue and will likely be turned into a separate documentation issue.

Create standalone RKE2 hardened cluster on k8s 1.24 per the existing hardening recommendations
Import this cluster into Rancher
Prepare the cluster for the upgrade. This step definitely needs refinement! See the documentation for the standalone RKE2 upgrade procedure prepared per https://github.com/rancher/rke2-docs/issues/51 as a starting point. However, we need to add a configuration of cluster-level PSA similar to rancher-restricted PSACT to these steps as updating the namespace where SUC runs as well as other Rancher-specific namespaces may not be a feasible option (this is to be evaluated).
After preparing the cluster per the above, edit the cluster in Rancher UI, change k8s version to 1.25 and save the changes.
Ensure the cluster is upgraded successfully and is otherwise healthy

Reference "k8s 1.25 support" umbrella issue: https://github.com/rancher/rancher/issues/38701

Following the steps mentioned in the issue, an imported hardened rke2 cluster is upgraded from 1.24 to 1.25 and passes the cis scan.
The rke2 cluster runs Kubernetes v1.24.10+rke2r1 with three server nodes and is configured following the instruction from RKE2 Docs - HA; it is hardened by following RKE2 Hardening Guide.
The cluster is imported into rancher and upgraded to v1.25.6+rke2r1 via Rancher UI.
The rancher-cis-benchmark app version 4.0.0-rc3 is used to run the CIS scan with the profile rke2-cis-1.23-profile-hardened. There is no failure when the cluster version is 1.24; when the cluster version is 1.25, there are only known expected failures.
To prepare the cluster for the upgrade, instead of following the steps in the RKE2 docs we need to do the following:
First, on each node, create the file rancher-psact.yaml at the directory /etc/rancher/rke2  with the following content from Rancher's builtin rancher-restricted PSACT.   Both the filename and the directory can be customized.

 /etc/rancher/rke2/rancher-psact.yaml
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- configuration:
    apiVersion: pod-security.admission.config.k8s.io/v1
    defaults:
      audit: restricted
      audit-version: latest
      enforce: restricted
      enforce-version: latest
      warn: restricted
      warn-version: latest
    exemptions:
      namespaces:
      - ingress-nginx
      - kube-system
      - cattle-system
      - cattle-epinio-system
      - cattle-fleet-system
      - longhorn-system
      - cattle-neuvector-system
      - cattle-monitoring-system
      - rancher-alerting-drivers
      - cis-operator-system
      - cattle-csp-adapter-system
      - cattle-externalip-system
      - cattle-gatekeeper-system
      - istio-system
      - cattle-istio-system
      - cattle-logging-system
      - cattle-windows-gmsa-system
      - cattle-sriov-system
      - cattle-ui-plugin-system
      - tigera-operator
    kind: PodSecurityConfiguration
  name: PodSecurity
  path: ""

Then, on each node, edit the RKE2 config file with the following two changes:

switch the profile from cis-1.6 to cis-1.23
set the path for the pod security admission config file to the above file

for example
profile: "cis-1.23"
pod-security-admission-config-file: /etc/rancher/rke2/rancher-psact.yaml

After finishing those two steps, we can edit the cluster to upgrade the Kubernetes version in Rancher UI.

This is moved to to-test for QA to validate if they can upgrade the cluster with the provided steps.
Once we get the confirmation from QA, we will add those steps to rancher's documentation.
A separate issue in the rancher-docs repo will be linked to this one.

QA TEST PLAN
Scenarios

Scenario
Test Case

1
As a standard user, validated that the instructions in the RKE2 hardening guide successfully upgrades an v1.24 -> v1.25 imported RKE2 cluster

Reopening issue as I am seeing some unexpected behavior. See details below:
ENVIRONMENT DETAILS

Rancher install: Helm
Rancher version: v2.7-head https://github.com/rancher/rancher/commit/211409c9a689d8ee4d801ed1bce1d887b0fb4b00
Browser: Chrome

TEST RESULT

Scenario
Test Case
Result

1
As a standard user, validated that the instructions in the RKE2 hardening guide successfully upgrades an v1.24 -> v1.25 imported RKE2 cluster
:x:

VALIDATION STEPS
Scenario 1

In a terminal, setup an RKE2 cluster with 3 servers, 1 worker and K8s version set at v1.24.
Followed the RKE2 hardening guide to ensure the cluster is hardened before importing into Rancher.
Navigated to Rancher > Cluster Management > Import Existing.
Imported my RKE2 cluster into Rancher; validated that the cluster and nodes came up as Active in Cluster Management and Cluster Explorer.
Validated that the pods in the system project page are up and Active.
Installed the CIS benchmark chart into the cluster.
Ran a CIS scan with the rke2-profile-1.23-hardened profile and validated that it passed:

Navigated back to my terminal where I created my RKE2 cluster.
In each node, created file rancher-psact.yaml at the /etc/ranhcer/rke2 directory. The file contains the following:

apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- configuration:
    apiVersion: pod-security.admission.config.k8s.io/v1
    defaults:
      audit: restricted
      audit-version: latest
      enforce: restricted
      enforce-version: latest
      warn: restricted
      warn-version: latest
    exemptions:
      namespaces:
      - ingress-nginx
      - kube-system
      - cattle-system
      - cattle-epinio-system
      - cattle-fleet-system
      - longhorn-system
      - cattle-neuvector-system
      - cattle-monitoring-system
      - rancher-alerting-drivers
      - cis-operator-system
      - cattle-csp-adapter-system
      - cattle-externalip-system
      - cattle-gatekeeper-system
      - istio-system
      - cattle-istio-system
      - cattle-logging-system
      - cattle-windows-gmsa-system
      - cattle-sriov-system
      - cattle-ui-plugin-system
      - tigera-operator
    kind: PodSecurityConfiguration
  name: PodSecurity
  path: ""

In each node, ran the following commands in the /etc/rancher/rke2 directory:

sed -i 's+1.6+1.23+g' config.yaml
echo 'pod-security-admission-config-file: /etc/rancher/rke2/rancher-psact.yaml' >> config.yaml

Navigated back to Rancher and upgraded the cluster from v1.24 to v1.25.
Noted that I am seeing the following:

The upgrade never finishes and I am stuck in this state.

Spoke with @jiaqiluo offline and we narrowed down the issue I reported to be a configuration issue. I am no longer blocked and will move this back to test.

Retested and validated that this is addressed. See below:
ENVIRONMENT DETAILS

Rancher install: Helm
Rancher version: v2.7-head https://github.com/rancher/rancher/commit/211409c9a689d8ee4d801ed1bce1d887b0fb4b00
Browser: Chrome

TEST RESULT

Scenario
Test Case
Result

1
As a standard user, validated that the instructions in the RKE2 hardening guide successfully upgrades an v1.24 -> v1.25 imported RKE2 cluster
:white_check_mark:

VALIDATION STEPS
Scenario 1

In a terminal, setup an RKE2 cluster with 3 servers, 1 worker and K8s version set at v1.24.
Followed the RKE2 hardening guide to ensure the cluster is hardened before importing into Rancher.
Navigated to Rancher > Cluster Management > Import Existing.
Imported my RKE2 cluster into Rancher; validated that the cluster and nodes came up as Active in Cluster Management and Cluster Explorer.
Validated that the pods in the system project page are up and Active.
Installed the CIS benchmark chart into the cluster.
Ran a CIS scan with the rke2-profile-1.23-hardened profile and validated that it passed:

Navigated back to my terminal where I created my RKE2 cluster.
In each node, created file rancher-psact.yaml at the /etc/ranhcer/rke2 directory. The file contains the following:

apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- configuration:
    apiVersion: pod-security.admission.config.k8s.io/v1
    defaults:
      audit: restricted
      audit-version: latest
      enforce: restricted
      enforce-version: latest
      warn: restricted
      warn-version: latest
    exemptions:
      namespaces:
      - ingress-nginx
      - kube-system
      - cattle-system
      - cattle-epinio-system
      - cattle-fleet-system
      - longhorn-system
      - cattle-neuvector-system
      - cattle-monitoring-system
      - rancher-alerting-drivers
      - cis-operator-system
      - cattle-csp-adapter-system
      - cattle-externalip-system
      - cattle-gatekeeper-system
      - istio-system
      - cattle-istio-system
      - cattle-logging-system
      - cattle-windows-gmsa-system
      - cattle-sriov-system
      - cattle-ui-plugin-system
      - tigera-operator
    kind: PodSecurityConfiguration
  name: PodSecurity
  path: ""

In each node, ran the following commands in the /etc/rancher/rke2 directory:

sed -i 's+1.6+1.23+g' config.yaml
echo 'pod-security-admission-config-file: /etc/rancher/rke2/rancher-psact.yaml' >> config.yaml

Navigated back to Rancher and upgraded the cluster from v1.24 to v1.25.
Validated that the cluster and nodes upgraded to v1.25 and is in an Active state.
Validated that the pods in the system project page are up and Active.
Ran a new CIS scan and noted that it failed as expected.
Created a test deployment with nginx:latest as the container image; validated that it failed due to rancher-restricted being the PSACT:


Kinmapchi2 for python nnls

Chi2 based on the kinematic maps (v, sigma, h3, h4,...), kinmapchi2, can now be calculated for the python NNLS (was available for the LegacyWeightSolver only).
For this, calculating kinmapchi2 was completely re-implemented for all weight solvers in method WeightSolver.chi2_kinmap(weights). Probably because v and sigma are determined via fitting a Gaussian to the respective histograms (just like in kinematic maps plotting), the kinmapchi2 values are a bit different from the ones read from the LegacyWeightSolver output (about 2.7% based on test_nnls.py and runs with both weight solvers).
Also added a weights table output feature to the LegacyWeightSolver so that now all weight solvers write the same ecsv output.
If you have time, please test and comment on the acceptability of the 2.7% difference ;-)

Comparison of test_nnls.py results:
Previous method (kinmapchi2 values read & calulated from the LegacyWeightSolver output nn_kinem.out, only available for LegacyWeightSolver):
  m-bh    a-bh  c-dh  f-dh q-stars p-stars u-stars  ml        chi2             kinchi2           kinmapchi2         time_modified      orblib_done weights_done all_done which_iter        directory
-------- ----- ------ ---- ------- ------- ------- --- ------------------ ------------------ ----------------- ----------------------- ----------- ------------ -------- ---------- -----------------------
100000.0 0.001 1000.0 10.0    0.54    0.99  0.9999 5.0<PHONE_NUMBER>5521018<PHONE_NUMBER>1505871<PHONE_NUMBER>107143 2023-08-10T21:39:35.000        True         True     True          0 orblib_000_000/ml05.00/
100000.0 0.001 1000.0 10.0    0.54    0.99  0.9999 1.0 42418.835768625584   18223.0332977481 34721.57131534838 2023-08-10T21:39:35.000        True         True     True          1 orblib_000_000/ml01.00/
100000.0 0.001 1000.0 10.0    0.54    0.99  0.9999 9.0<PHONE_NUMBER>2152033<PHONE_NUMBER>381026 2525562.934382647 2023-08-10T21:39:35.000        True         True     True          1 orblib_000_000/ml09.00/

New method (using WeightSolver.chi2_kinmap(weights)):
LegacyWeightSolver
  m-bh    a-bh  c-dh  f-dh q-stars p-stars u-stars  ml        chi2             kinchi2           kinmapchi2          time_modified      orblib_done weights_done all_done which_iter        directory
-------- ----- ------ ---- ------- ------- ------- --- ------------------ ------------------ ------------------ ----------------------- ----------- ------------ -------- ---------- -----------------------
100000.0 0.001 1000.0 10.0    0.54    0.99  0.9999 5.0<PHONE_NUMBER>5521018<PHONE_NUMBER>1505871 1005436.8699955629 2023-08-10T21:32:41.000        True         True     True          0 orblib_000_000/ml05.00/
100000.0 0.001 1000.0 10.0    0.54    0.99  0.9999 1.0 42418.835768625584   18223.0332977481  36040.63890041469 2023-08-10T21:32:41.000        True         True     True          1 orblib_000_000/ml01.00/
100000.0 0.001 1000.0 10.0    0.54    0.99  0.9999 9.0<PHONE_NUMBER>2152033<PHONE_NUMBER>381026 2581953.2688042363 2023-08-10T21:32:41.000        True         True     True          1 orblib_000_000/ml09.00/

scipy NNLS
  m-bh    a-bh  c-dh  f-dh q-stars p-stars u-stars  ml        chi2             kinchi2           kinmapchi2          time_modified      orblib_done weights_done all_done which_iter        directory
-------- ----- ------ ---- ------- ------- ------- --- ------------------ ------------------ ------------------ ----------------------- ----------- ------------ -------- ---------- -----------------------
100000.0 0.001 1000.0 10.0    0.54    0.99  0.9999 5.0<PHONE_NUMBER>301296<PHONE_NUMBER>531478 1000011.3059815513 2023-08-10T21:34:16.000        True         True     True          0 orblib_000_000/ml05.00/
100000.0 0.001 1000.0 10.0    0.54    0.99  0.9999 1.0  42453.69372127462  18223.81866655845  35912.95977085516 2023-08-10T21:34:16.000        True         True     True          1 orblib_000_000/ml01.00/
100000.0 0.001 1000.0 10.0    0.54    0.99  0.9999 9.0<PHONE_NUMBER>0540892<PHONE_NUMBER>1994907  2602700.254474954 2023-08-10T21:34:16.000        True         True     True          1 orblib_000_000/ml09.00/

Hi @prashjet and @sthater,
as discussed today, it would be great could you test this PR with a 'non-trivial' galaxy (>1 non-fixed parameter, etc.).
Sabine: please use config file setting
weight_solver_settings:
    type: "LegacyWeightSolver"
    nnls_solver: 1

Prash: please use config file setting
weight_solver_settings:
    type: "NNLS"
    nnls_solver: 'scipy'

Testing procedure:

Run a few iterations, inspect the kinmapchi2 values in the all_models table.
Add one or more iterations with a second DYNAMITE run and verify that this works.
Inspect the all_models table and save it somewhere else.
Run the same iterations with the code from the master branch and verify that the kinmapchi2-values are close.

Many thanks and cheers,
Thomas

Hi @sthater and @prashjet,
I could not let go - the last commit should make it possible to continue old runs with the LegacyWeightSolver, only 4 more lines of code... The method should be fully compatible with the Model.get_weights() method (used by the Analysis and Plotter classes) because it calls the WeightSolver.solve() method which in turn reads existing data and if the weights file is not existing, it will create it upon the first call.
The only limitation is that old LegacyWeightSolver runs cannot be continued with the scipy NNLS. Workaround: calling Model.get_weights() on all existing models to create the weights files, but that's probably stretching it...
Let me know what you think, perhaps we won't need to rush releasing two releases after all...
Cheers,
Thomas

I more or less did the following:

Testing procedure:

Run a few iterations, inspect the kinmapchi2 values in the all_models table.
Add one or more iterations with a second DYNAMITE run and verify that this works.
Inspect the all_models table and save it somewhere else.
Run the same iterations with the code from the master branch and verify that the kinmapchi2-values are close.

Setup: 2 free params (ml & f_dm). Start with 4 models produced by FullGrid then switching to LegacyGridSearch to add 4 more models. In all cases below, identical parameter lists are generated.
Here are the resulting kinmapchi2 values. Using LegacyWeightSolver:
    kinmapchi2
------------------
 74980.33136167636
 49772.92890019686
 54768.18645254749
<PHONE_NUMBER>887669
58798.604915884694
 81266.34115377656
 42504.44156228304
 23857.45130716322

with NNLS:
    kinmapchi2
------------------
 73427.31437091822
 49831.96099365194
 54548.93128352338
<PHONE_NUMBER>160483
 58500.33809891473
 80118.76168753469
41965.936167707805
 23879.67772101332

I'm satisfied that these are a close enough match.
Everything above was run from the kinmapchi2_for_python_nnls branch. Re-running from branch master as suggested, with LegacyWeightSolver gives the following:
    kinmapchi2
------------------
 74999.38826898485
49766.507882430786
54753.495638351655
180208.76262994218
 58800.72603149484
 99728.29449456796
 42481.00604266759
23851.807296894774

which is largely similar except for the third row from the bottom. Re-running again from master but with a different random seed gives
    kinmapchi2
------------------
 76491.51881890604
 49704.88693633562
54904.454892487054
180843.72283326794
 58880.00862570841
100864.12901310157
 43496.23295194898
24026.798580051007

which again is similar except for the third row from the bottom. So this model has a larger natural stochasticity.
I think safe to conclude the two weight solvers give consistent values of kinmapchi2.


Failed to fetch information about the app

hey guys,
i upgraded blockstack.js to 19.2.2 and uploaded the latest version of TimeStack to S3.
Now, i’m no longer able to login due to a “Failed to fetch information about the app requesting authentication. Please contact the app maintainer to resolve the issue.”
Console log states: Access to fetch at ’https://my.timestack.app/manifest.json' from origin ‘http://localhost:8888’ has been blocked by CORS policy: No ‘Access-Control-Allow-Origin’ header is present on the requested resource. If an opaque response serves your needs, set the request’s mode to ‘no-cors’ to fetch the resource with CORS disabled.
CORS is/was already enabled on S3 (& worked before updating) - what else could be the issue here?
Thanks!

@coboltDE still having this issue? Closing for now and happy to reopen if you still need help.


Added license url to nuspec

Would be nice to link the license for automated tools that check used nuget package licences :P

Yep, thanks!


Feature/leanient validation javadoc amends

Sorry, but this is not how open source works. This isn't your personal playground here.
First, an issue is created and the maintainers of this library will decide whether a new feature is needed or not. Then, maybe, eventually a PR will be created. This might be someone from the outside or, more often, an internal committer.

Sorry for the inconvenient, my intention was not to create this PR, it was incomplete at that moment and happened accidentally, that is the reason I closed almost immediately. And, thanks for the guidelines about the process to follow.

Hi @cesarcastillob, thanks for reaching out to me!
I think I should also clarify it in the contribution docs.
Ideally, there would be only one Validation type instead of N over time.
It would be nice to first to  what is missing in the existing Validation. That is what I call the „requirement space“. If and how it is solved is part of the „solution space“.


Add EC2 Task Runner

Feature description
We should add support for running tasks on Amazon EC2 instances directly (without Batch).
The user will need to specify (we can set sensible defaults e.g. to use Ubuntu AMI and query private subnet ID for the default VPC):

the containerImage
instanceType e.g. t2.micro
imageId - AMI e.g. ami-0abcdef1234567890
subnetId
list of tags for cost tracking

Kestra should:

output insensitive instance details like confirmation of the InstanceId, instanceType etc.
capture execution logs
terminate the instance after the task finishes and confirm that in the logs

Note that if the user chooses an instance type without Docker preinstalled, we can't run it as a container.
We should then log a friendly error message: "Your chosen AMI doesn't support Docker runtime. Please choose a different AMI for your instance. See the Amazon EC2 documentation for more details."

You could have the server alway choose the image for this type of task is an extension of the current server and not arbitrary task running.

no no, actually we ideally want to enable running arbitrary commands -- for an extension of the current server, we have a feature called worker groups


Using SunEditor to draft emails

Is there any "Email drafting mode" in SunEditor? Here is the issue, there may be other areas where this behavior is similar: if I use the editor to write an email -- similar to the browser interface on Gmail, let's say -- and I want to insert a link, SunEditor's default option seems to be inserting a relative path before the link I provide. For example, if I insert "https://www.abc.com", the resulting link will be something like "/mypage/https://www.abc.com". I think I can change this behavior by choosing the "Download link" option, but this option's name is misleading for my use case. What I'd like to have is an option to turn off this behavior, as I want SunEditor to take any text I provide and use it as the link. This is especially relevant as I am using substitution codes in my backend to produce the links, for instance, the user might enter "[[LinkToDepartmentList]]" which then gets written out at a full link when the email is generated.
Many thanks for any tips.

I'm encountering the same problem. I need the href to become "{DeepLink}", not localhost/{Deeplink}.

@BJBigler I found a quick fix by editing the files:

Link.js line 178 from selectionATag.href to selectionATag.getAttribute('href');
_anchor.js line 127 change to const href = contextAnchor.linkAnchor.getAttribute('href');
_anchor.js line 297 change to context.linkValue = preview.textContent = value;
_anchor.js line 318 change to contextAnchor.linkValue = anchor.getAttribute('href');

https://user-images.githubusercontent.com/47681665/144294738-966041ce-1cf0-48b2-adaa-61f11f834a96.mp4

Oh wow! This is great. Thanks! I've been using the CDN for the JS files, so I'll need to rope them in.

You can use a linkProtocol option.
suneditor.create("#id", {
  linkProtocol: "http://"
})


Added an additional page called DonorLists

I have added an additional file called DonorLists in the pages folder. This will allow projects to get a list of their donors. No other changes have been made.

Hi, just saw this reply(somehow missed it in my email). I'll try make those changes asap.

Hey @realguy33 just checking in here how are things going ser?

Hi, Sorry for the delay! i had some previously unfinished freelance work whose deadline was approaching. I have started working on this. Shouldn't take long.

beatful

very very good

🚀

Gm

Thanks for changes bro

Gm

Happy new year


D435i not Recongized Properly

Currently my D435i is only being recognized as a USB2.1 device and therefore I can't use some of the functions properly - i.e. the rs-data-collect
I found this below about how it has to be recognized as a USB3 to work- how do I make it be recognized properly?? (I am working with a jetson nano dev kit)

According to the log you published, the device was connected as a USB 2 and it is behavior in this case is not well defined.

Assuming you connect it to a USB 3 jack, you should try to reconnect again and check how it was connected using linux tools, like this:

lsusb -d 8086: -v | grep -i bcdUSB

If it shows:

bcdUSB               2.10 That's not good.

bcdUSB               3.20 is good.

If you can run realsense-viewer, it also shows if it was connected as a usb2.1 or 3.2

Originally posted by @doronhi in https://github.com/IntelRealSense/realsense-ros/issues/635#issuecomment-467754466

If the camera is in a USB 3 port but is being detected as USB 2, please first try inserting the camera into the port quickly and firmly instead of slowly.  This should increase the chances of it being correctly identified as USB 3.

I just tried and it is still recognized only as "Intel Realsense USB2" with USB Descriptor of 2.1

Are you using the official short-length cable supplied with the camera please?  Some users have found that they could only get USB 2 when using their own choice of cable, and solved the problem by changing to a higher-quality cable.

Just went back and found the original cable it came with, it has the blue inside on the USB, so I'm sure its a USB3. But now when I plug it in, the camera isn't recognized at all. I receive the "No device detected. Is it plugged in?" message.

The 'No device detected' message sometimes occurs when using the D435i with an Nvidia Jetson TX2 board.  Are you using a TX2, please?

Nope, its the Jetson Nano Developer Kit

Ah yes, I re-read your first message and saw that.  My apologies.
I located another case where a user was also having problems with their D435i being detected as USB 2.0 on Nano.
https://github.com/IntelRealSense/librealsense/issues/3759#issuecomment-483862292

thank you!
I've looked through that and it seems as though it could be an issue with the patches I installed, I will try again.

This did not work- I installed the patches from an open source (jetson hacks) that was suppose to make the jetson nano compatible with the D435i camera, and yet no change. Does intel have any patches for this camera that I can use?

@RealSenseCustomerSupport can I please have some assistance on this matter
I have tried so many different ways to get the D435i recognized on the jetson nano, I followed the jetson hacks installation and kernal patches, yet still nothing. I have also updated the FW on the D435i to see if that would help but to no avail

@MartyG-RealSense @christopherbate @dorodnic

@kat-mcg-lin , what is the FW version installed ? (rs-enumerate-devices -s)
Note that D435i had only recently received USB2 support with firmware <IP_ADDRESS>+.
With the older versions it is expected to be recognized as "Intel Realsense USB2" device with no RGB or IMU sensors available.

@ev-mp that command doesn't work because the device isn't recognized on the jetson at all

It is version <IP_ADDRESS> and I just upgraded it yesterday

the only available firmware on the intel website is 5.11.4- how does one get the 5.11.6?

@kat-mcg-lin hello, USB2 enumerations issues are often produced by hw setup - and trying different cables/USB ports may help. Adding USB3 self-powered hub is also worth mentioning, especially on Low-power systems such as Odroid.
BTW, can you share the platform/hardware you're using as well as the SDK's version?
Regarding fw5.11.6  - it is temporally unavailable (#4038), but resolving the USB2 issue would allow you to utilize the device properly without additional fw upgrades.

@ev-mp hi!
how do I gather the information about my platform/hardware and SDK version?
I've also tried changing ports and cords, older cords would allow it to be recognized but only as USB 2.1

@kat-mcg-lin,

The Jetson Nano kit comes with  4 USB3 ports, so it is strange - have you tried to connect any non-Realsense USB3 camera to these ports, and if so - were they recognized as USB3 ?
Can you share the kernel log when the camera is plugged in ? (dmesg -wT)

I would also check the vendor's support site and verify USB BIOS settings.
And in case the configuration is correct then self-powered hub is still a viable option.
At the same time, I'd cross-check that D435i on another preferably desktop PC to rule out unit-related issues.

I currently don't have a non-Realsense camera, but I have the T265 Realsense tracking camera that is also USB 3 and it gets properly recognized as USB 3.1
the output for dmesg -wT is attached
dmesg -wT output.docx
As well, the D435i camera is recognized on the host computer I am using, which is a Linux system.

I have D435i working reliably on the Jetson Nano. I'm using firmware <IP_ADDRESS> with librealsense 2.21.0, and kernel patches are required: https://github.com/jetsonhacksnano/installLibrealsense.
Most cables I've tested don't work, including the one in the box. This one is reliable: https://www.amazon.com/gp/product/B07D7RZ3WV
Hotplugging does not work.

Hello @kat-mcg-lin , according to the log the camera is properly recognized as USB3 device:

[Wed May 22 13:07:05 2019] usb 2-1.3: new SuperSpeed USB device number 4 using tegra-xusb
[Wed May 22 13:07:05 2019] usb 2-1.3: New USB device found, idVendor=8086, idProduct=0b3a
[Wed May 22 13:07:05 2019] usb 2-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Wed May 22 13:07:05 2019] usb 2-1.3: Product: Intel(R) RealSense(TM) Depth Camera 435i
[Wed May 22 13:07:05 2019] usb 2-1.3: Manufacturer: Intel(R) RealSense(TM) Depth Camera 435i
[Wed May 22 13:07:05 2019] usb 2-1.3: SerialNumber:<PHONE_NUMBER>74
[Wed May 22 13:07:05 2019] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 435i (8086:0b3a)

In case you haven't changed the configuration then most certainly the inconsistency in USB 2/3 enumeration is due to the HW setup.
Make sure you connect D435i firmly and fast  - slow plug-in may also result in device being enumerated as USB2.


Update README.md

Hello!
Fixed markdown in README.md.
Markdown rules: https://github.com/markdownlint/markdownlint/blob/main/docs/RULES.md

I'll close this one if you don't mind :)
The README needs a bigger update first, the formatting changes look good but the facts are no longer entirely up to date.


Skv1

xx

xx

Codecov Report

Merging #238 into develop will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             develop     #238   +/-   ##
==========================================
  Coverage      90.49%   90.49%
  Complexity       575      575
==========================================
  Files             42       42
  Lines           1725     1725
  Branches         344      344
==========================================
  Hits            1561     1561
  Misses           104      104
  Partials          60       60

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 52959a5...af7c347. Read the comment docs.


differences between how microtubule and microfilament motors are described

despite analogous activities the terms for
ATP-dependent microtubule motor activity
and
"ATP-dependent actin filament motor activity" (i.e myosin)
are quite different graphs:

I just came across this myself, only to find (again!) that Val has already reported it.
Right now, to accurately annotate a myosin, you need to know to add both the "microfilament motor activity" term and the "actin-dependent ATPase activity" - the myosin annotations I've seen tend to be either one or the other, so searches using either one term or the other only give partial lists....
So, I would also like to see the "microfilament motor activity" branch made comparable to the "microtubule motor activity", for the same reasons Val states. And I would also support changing the term name and def of the current "actin-dependent ATPase activity" to mirror the "ATP-dependent microtubule motor activity" term.
FYI, here's how things currently look (changed a bit since Val's original post):
==
id: GO:0003777
name: microtubule motor activity
namespace: molecular_function
def: "Catalysis of movement along a microtubule, coupled to the hydrolysis of a nucleoside triphosphate (usually ATP)." [GOC:mah, ISBN:0815316194]
is_a: GO:0003774 ! motor activity
==
id: GO:1990939
name: ATP-dependent microtubule motor activity
namespace: molecular_function
def: "Catalysis of movement along a microtubule, coupled to the hydrolysis of ATP." [PMID:19686686]
xref: Reactome:R-HSA-9610627 "KIF17 transports GluN1:GluN2B (GRIN1:GRIN2B) NMDA receptors to the plasma membrane"
is_a: GO:0003777 ! microtubule motor activity
is_a: GO:0016887 ! ATPase activity
==
id: GO:0000146
name: microfilament motor activity
namespace: molecular_function
def: "Catalysis of movement along a microfilament, coupled to the hydrolysis of a nucleoside triphosphate (usually ATP)." [GOC:mah, ISBN:0815316194]
is_a: GO:0003774 ! motor activity
==
id: GO:0030898
name: actin-dependent ATPase activity
namespace: molecular_function
def: "Catalysis of the reaction: ATP + H2O = ADP + phosphate. This reaction requires the presence of an actin filament to accelerate release of ADP and phosphate." [GOC:mah]
synonym: "actin-activated ATPase activity" EXACT []
is_a: GO:0016887 ! ATPase activity
==

Is this now out. of date?
This term request might also impact
https://github.com/geneontology/go-ontology/issues/20563

Actually more info here. I will close
https://github.com/geneontology/go-ontology/issues/18664
as duplicate

@ValWood @sjm41
Do we need both terms in those pairs:
GO:1990939 ATP-dependent microtubule motor activity vs   GO:0003777 microtubule motor activity
and
GO:0030898 actin-dependent ATPase activity  vs GO:0000146 microfilament motor activity
I was proposing to merge, see #20877
Thanks, Pascale

I think they can merge?

I agree.
I think it would make sense for the final two terms to be named: "ATP-dependent microtubule motor activity" and "ATP-dependent actin microfilament motor activity".
And the final terms would each have two parents, 'motor activity' and 'ATPase activity', right?
I'm sure you're aware, but there will also be some child terms to sort out/merge here:

Update these child term names to match final parent name (maybe this is automatic?):
GO:2000574 regulation of microtubule motor activity
GO:2000575 negative regulation of microtubule motor activity
GO:2000575 negative regulation of microtubule motor activity

Merge/update names of children of GO:0030898 and GO:0000146:
GO:1904621 regulation of actin-dependent ATPase activity (regulates GO:0030898)
GO:1904623 positive regulation of actin-dependent ATPase activity (positively_regulates GO:0030898)
GO:1904622 negative regulation of actin-dependent ATPase activity (negatively_regulates GO:0030898)

GO:0120079 regulation of microfilament motor activity (regulates GO:0000146)
GO:0120081 positive regulation of microfilament motor activity (positively_regulates GO:0000146)
GO:0120080 negative regulation of microfilament motor activity (negatively_regulates GO:0000146)
GO:0060002 plus-end directed microfilament motor activity (is_a GO:0000146)
GO:0060001 minus-end directed microfilament motor activity (is_a GO:0000146)

Is the ATP-dependent useful to the users, or did we just get used to it ? (I can never tell). I would have done simply 'microtubule motor activity' and 'actin microfilament motor activity'

I think since all motors are ATP-dependent it would be OK not to mention in the term name, but it should be part of the definition (and can be exact synonym)

Dumping the ATP-bit in the term name (but maintaining the ATPase parentage!) is fine with me.

Just realised that #19738 is relevant to reclassifying ATP-depedent motors.
That is, it looks like the primary enzymatic function of these motor proteins should be an isomerase activity (rather than a ATPase/hydrolase activity) - maybe change parent of  "motor activity" (GO:0003774) from "nucleoside-triphosphatase activity (GO:0017111 JSON)" to "GO:0016853 ! isomerase activity"?

I was thinking to change the main parent to 'motor activity' - the isomerase activity seems odd, as @kaxelsen points out, in EC it means "Isomerases altering macromolecular conformation"; while GO is using isomerase in the more 'classical/chemical' meaning.
I was planning to add the missing ECs for #19738  but not add the isomerase parent. Unless you think this is useful?

I guess there are 2 (related) points here:

EC now clearly defines these motor proteins as isomerases (EC:5) rather than hydrolases (EC:3), whereas GO defines them as hydrolases (via the current parentage of 'motor activity'). Do we want to align with EC on this?

The current parent of 'motor activity' is 'nucleoside-triphosphatase activity' - I guess that reflects that the 3 children of 'motor activity' are coupled to an ATPase or GTPase activity:
GO:0061791 GTPase motor activity (this has only 2 annotations!)
GO:0003777 microtubule motor activity (this is 'ATP dependent')
GO:0000146 microfilament motor activity (this is 'ATP dependent')
So based on the other discussion about classifying MF that use ATP, having the 'nucleoside-triphosphatase activity' parent here seems inconsistent?  From that viewpoint, and we want an alternative catalytic activity parent for 'motor activity', then maybe 'isomerase activity' is best??

note that all of these:
Update these child term names to match final parent name (maybe this is automatic?):
GO:2000574 regulation of microtubule motor activity
GO:2000575 negative regulation of microtubule motor activity
GO:2000575 negative regulation of microtubule motor activity
Merge/update names of children of GO:0030898 and GO:0000146:
GO:1904621 regulation of actin-dependent ATPase activity (regulates GO:0030898)
GO:1904623 positive regulation of actin-dependent ATPase activity (positively_regulates GO:0030898)
GO:1904622 negative regulation of actin-dependent ATPase activity (negatively_regulates GO:0030898)
GO:0120079 regulation of microfilament motor activity (regulates GO:0000146)
GO:0120081 positive regulation of microfilament motor activity (positively_regulates GO:0000146)
GO:0120080 negative regulation of microfilament motor activity (negatively_regulates GO:0000146)
GO:0060002 plus-end directed microfilament motor activity (is_a GO:0000146)
GO:0060001 minus-end directed microfilament motor activity (is_a GO:0000146)
are BP terms in the "regulation of molecular function" branch. There is talk of getting rid of these terms because they usually represent MF in the BP ontology. Indeed, these  terms are used for myosin/dynein/kinesin light chain /regulatory subunits.
So it might be good to establish the MF regulator terms and figure out if these processes are really required
(or merge them with the appropriate MF terms)

@thomaspd
@pgaudet
This should be done at the same time as
https://github.com/geneontology/go-ontology/issues/21619

[x] Merged  'GO:1990939 ATP-dependent microtubule motor activity'  into superclass  'GO:0003777 microtubule motor activity'
[x] Added logical defs for

'GO:0003777 microtubule motor activity'
intersection_of: GO:0003774 ! motor activity
intersection_of: has_participant GO:0005874 ! microtubule
and
microfilament motor activity:
GO:0003774 ! motor activity
intersection_of: has_participant GO:0005884 ! actin filament

[x]  added relationship: has_part GO:0016887 ! ATP hydrolysis activity to both, but not in the logical definition, since this is not what distinguishes the activity from their common superclass.

Also, added pubmed references to microtubule terms, PMID:15659646, PMID:32842864

@sjm41
For your point 2 in https://github.com/geneontology/go-ontology/issues/18664#issuecomment-780718994
Now both
'microfilament motor activity'
'microtubule motor activity'
are children of 'ATPase-dependent activity'.
Is that OK? Should we create a grouping term, ATP-dependent motor activity?
@ValWood

@ValWood @sjm41
AFAIK everything was done that was requested here  - expect the regulation issue (https://github.com/geneontology/go-ontology/issues/18664#issuecomment-781384093) - I will create a new obsoletion ticket.
Thanks, Pascale

Is that OK? Should we create a grouping term, ATP-dependent motor activity?

Personally I  don't think we need an ATP-dependent grouping. They will all group under
motor activity. They also group naturally because all the GTPase motors are scission motors as far as I am aware, all the microtubule motors and ATPases.
I think it's OK if they are annotated to the type of motor and ATP or GTPase without this orthogonal grouping.

Perfect - I also dont like too many grouping terms - we only have 2 classes here so far. We can do it later if needed.

@pgaudet

I agree we don't need the additional 'ATP-dependent motor activity' grouping term.
Can you clarify whether 'motor activity' is still a child of 'nucleoside-triphosphatase activity' (GO:0017111)? That bit still seems odd to me given we're specifying this via the separate 'ATPase-dependent activity' parent for the 'microfilament motor activity' and 'microtubule motor activity' terms.

Can you clarify whether 'motor activity' is still a child of 'nucleoside-triphosphatase activity' (GO:0017111)

No, I have moved it directly under MF. So, 'microfilament motor activity' and 'microtubule motor activity' are direct children of 'ATP(ase)'dependent activity, and 'GTPase motor activity' is_a 'GTPase activity'.  There is also another sibling, 'proton motive force-driven motor activity'.
OK ?
Thanks, Pascale

Yes, that makes more sense, thanks!


Issue/359/dsps v3.0.0

I pulled in the changes from main, so that there aren't conflicts anymore.
But we probably want to figure out something else to do about the files in
src/rail/examples_data/creation_data/data/dsps_default_data  it looks like those are 59 MB, which seems really big for a code repo.
We have some examples use setting up examples to curl data from NERSC, we could do that here.

Hi @eacharles , I followed your suggestions and implemented the following changes:

pull latest changes from main
curl ssp templates from public lsst directory and removed the templates in the examples_data folder
fix the issues with the unit tests
removed the dsps_reqs.txt file, since now dsps is pip installable and does not require diffstar and diffmah to be git cloned
If you think that everything is alright, then you can approve the review :)
Grazie mille!


Create client.py

This is intentionally left out
https://github.com/nortxort/tclib/blob/master/.gitignore#L6


some question about the releases page

I found that https://github.com/EmergingTechnologyAdvisors/node-serialport/releases has some .gz file
what the meaning such as electron-v47.
v47 is the version? but the electron vesion is 1.xxx
What is the use of these?

@reconbot how can I use this prebuild?
but the way, have a easy way to use serialport under electron?
I use a new NoteBook, I find that is difficulty  to rebuild the serialport

@reconbot i now :cry:


Customized generation - insert before message (structs)

Please kindly review. I have detailed the issue - https://github.com/stepancheg/rust-protobuf/issues/675
Note: This feature is not expected to break existing behavior.

Thanks!


mix beacon.install generator

Similar to phx.gen.auth and others, a generator to inject beacon into an existing app, either single or umbrella. That could lower the barrier to adoption.
It should perform most of the steps described in README providing a sane default to get started.

Closed by https://github.com/BeaconCMS/beacon/pull/102


Revision

This is just a reminder to handle in the future.
Several parts of the new code were copied from the old one, So I believe we should revise it to make sure it has no bugs hidden, or if it could be done better. I'm specially concerned with the _update_plot() and how the zoom/pan is working.

Great idea!  python _update_plot is a core piece and it is quite possilbe it is not optimized or could be done in a better way.
Just a comment on Zoom/Pan, I have changed the ToolSelect to no longer be used and have instead added a python self.zp that will allow a more straighforward way of keeping track here.  It should be noted that when Zoom/Pan is enabled, the arrow keys are blocked.

And just to update this thread, The tool setup has changed to now be put into a list.  This was done to hopefully keep track of instances as the functionality grows and makes the checks easier.
@gamaanderson Do you feel better about _update_plot now with the change to V variables (adding strong and weak)?

It is an improvement, but I will not fell better till I sit down and understand every line of that code, which I have not done jet. From what I have seen there is a gap between self.display and self.canvas, I

So on line 575 we're using the PyArt library to create figure in matplotlib that is associated to the python self.ax.  This Axis instance was established in the python _set_fig_ax function, with fig and ax being associated.  The python _set_figure_canvas function uses the matplotlib backend that connects the matplotlib figure instance and Qt.  So once the figure has been made, L614 is used to actually update the Qt canvas with the matplotlib figure.
Does that make sense?

Actually it does, I will reflect over that.


Easier way to resume earlier ELT pipeline by discovering or providing initial replication key values per stream

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/2734
Originally created by @ers81239 on 2021-04-28 21:46:12

Problem to solve
(Summarize the problem we are trying to solve in the form of we need to do [A], so that [B] can [C])
Target audience
(For whom are we doing this? Include a persona)
Further details
As a Meltano user, I want to resume ELT for existing target tables so that I can replace existing ELT tools with Meltano.  With Incremental loading, this boils down to telling the extractor to start at the max_replication_value in the destination table.  Today, this is very difficult and involves either using elt --dump and elt --state or else manually editing the state in meltano.db.
Proposal

Whole enchilada:  Stop tracking the max_replication_value in the state and instead read it from the loader on every execution.
Much easier:  Allow max_replication_value (and for completeness min_replication_value) as metadata settings for incremental extractors.  This supports all kind of use cases where the user wants to sync a portion of a source for resource/time management reasons.

What does success look like, and how can we measure that?
Increased adoption due to lower barrier to entry.
Regression test
N/A?
Links / references
Please note that this was taken from GitLab, to be changed accordingly

View 10 previous comments from the original issue on GitLab

Still very relevant


Incorrect examples for Powershell queries that not utilize graph filtering

Examples for PowerShell does not utilize filters in graph, which leads to suboptimal results, and overcomplicates things.
Fe. in my organization we have well over 100k groups, and only 10 licensing groups.
Even MSOL based page have proper filters assigned.
Get all groups with licenses
Is 16 lines:
# Import the Microsoft.Graph.Groups module
Import-Module Microsoft.Graph.Groups
# Get all groups and licenses
$groups = Get-MgGroup -All
$groupsWithLicenses = @()
# Loop through each group and check if it has any licenses assigned
foreach ($group in $groups) {
    $licenses = Get-MgGroup -GroupId $group.Id -Property "AssignedLicenses, Id, DisplayName" | Select-Object AssignedLicenses, DisplayName, Id
    if ($licenses.AssignedLicenses) {
        $groupData = [PSCustomObject]@{
            ObjectId = $group.Id
            DisplayName = $group.DisplayName
            Licenses = $licenses.AssignedLicenses
        }
        $groupsWithLicenses += $groupData
    }
}

Should be 4:
# Import the Microsoft.Graph.Groups module
Import-Module Microsoft.Graph.Groups
# Get all groups and licenses
Get-MgGroup -All  -filter 'AssignedLicenses/any()' -property Id,DisplayName,AssignedLicenses | Select-Object id, DisplayName, AssignedLicenses

Get all groups with license errors
Is:
# Import User Graph Module
Import-Module Microsoft.Graph.Users
# Authenticate to MS Graph
Connect-MgGraph -Scopes "Group.Read.All"
# Get all groups in the tenant with license assigned and with errors
$groups = Get-MgGroup -All -Property LicenseProcessingState, DisplayName, Id, AssignedLicenses | Select-Object  displayname, Id, LicenseProcessingState, AssignedLicenses | Select-Object DisplayName, Id, AssignedLicenses -ExpandProperty LicenseProcessingState | Select-Object DisplayName, State, Id, AssignedLicenses | Where-Object {$_.State -eq "ProcessingFailed" -and $_.AssignedLicenses -ne $null }
# Display the results and format output
$groups | Format-Table -AutoSize

Should Be:
# Import User Graph Module
Import-Module Microsoft.Graph.Users
# Authenticate to MS Graph
Connect-MgGraph -Scopes "Group.Read.All"
# Get all groups in the tenant with license assigned and with errors
Get-MgGroup -All -Property LicenseProcessingState, DisplayName, Id, AssignedLicenses -filter "AssignedLicenses/any()" | Where-Object {$_.LicenseProcessingState.State -eq "ProcessingComplete"} | Select-Object -ExpandProperty LicenseProcessingState -ExcludeProperty AdditionalProperties  | Select-Object DisplayName, Id, AssignedLicenses, State

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 62f978ec-9661-d804-bcc5-d4af11287414
Version Independent ID: beba2bc9-c45b-1d0f-7bbe-bcb7c0f5d8e7
Content: Microsoft Graph PowerShell examples for group licensing - Microsoft Entra
Content Source: articles/active-directory/enterprise-users/licensing-powershell-graph-examples.md
Service: active-directory
Sub-service: enterprise-users
GitHub Login: @barclayn
Microsoft Alias: barclayn

@pkrzysz
Thanks for your feedback! We will investigate and update as appropriate.

For the section Get all users with license errors in a group, this brings more value

# Import User Graph Module
Import-Module Microsoft.Graph.Users

# Authenticate to MS Graph
Connect-MgGraph -Scopes "Group.Read.All", "User.Read.All"

#get all groups in distionary object for getting firendly displaynames
$groups=Get-MgGroup -All  -filter 'AssignedLicenses/any()' -property Id,DisplayName | Select-Object id, DisplayName
$groupsDict=@{}; $groups|ForEach-Object {$groupsDict.Add($_.id,$_.DisplayName)}

#get users with assigned licenses
#this will be faster is majority of users are assigned per group, as we only read directory once
$users=Get-MgUser -All -Filter 'assignedLicenses/$count ne 0' -ConsistencyLevel eventual -CountVariable licensedUserCount -Property userPrincipalName,licenseAssignmentStates

#get users with problems with licenses assigned by group
$badlicense=$users | Where-Object {$_.licenseAssignmentStates |
                     Where-Object error -ne 'None' |
                     Where-Object AssignedByGroup -ne $null}

#get only bad licenses with userprincipalname
$badlicensedetails = $badlicense | Select-Object userprincipalname -ExpandProperty LicenseAssignmentStates|
                                   Select-Object userprincipalname, AssignedByGroup,@{n='AssignedByGroupName';e={$groupsDict[$_.AssignedByGroup]}},
                                                 DisabledPlans , Error, LastUpdatedDateTime, SkuId |
                                   Where-Object error -ne 'None' |
                                   Where-Object AssignedByGroup -ne $null

#get Statistics for errors per group
if ($badlicense.count -gt 0){
"Found {0} accounts with license problems" -f $badlicense.count
$badlicensedetails | Group-Object AssignedByGroupName, Error |
                     Select-Object name, count |
                     Sort-Object count -Descending
}else {"No license errors"}

And for the section of Get all users with license errors in the entire organization outputting the affected group, error and username makes the output better:

# Import User Graph Module
Import-Module Microsoft.Graph.Users

# Authenticate to MS Graph
Connect-MgGraph -Scopes "Group.Read.All", "User.Read.All"

#get all groups in distionary object for getting firendly displaynames
$groups=Get-MgGroup -All  -filter 'AssignedLicenses/any()' -property Id,DisplayName | Select-Object id, DisplayName
$groupsDict=@{}; $groups|ForEach-Object {$groupsDict.Add($_.id,$_.DisplayName)}

#get users with assigned licenses
#this will be faster is majority of users are assigned per group, as we only read directory once
# we can filter by 'assignedLicenses/$count ne 0', but this forces eventual consistency, and requires much more permissions. Both assignedLicenses and licenseAssignmentStates are not indexed and cannot be used in complex queries
$users=Get-MgUser -All -Property userPrincipalName,licenseAssignmentStates

#get users with problems with licenses assigned by group
$badlicense=$users | Where-Object {$_.licenseAssignmentStates |
                     Where-Object error -ne 'None' }

#get only bad licenses with userprincipalname
$badlicensedetails = $badlicense | Select-Object userprincipalname -ExpandProperty LicenseAssignmentStates|
                                   Select-Object userprincipalname, AssignedByGroup,@{n='AssignedByGroupName';e={$groupsDict[$_.AssignedByGroup]}},
                                                 DisabledPlans , Error, LastUpdatedDateTime, SkuId |
                                   Where-Object error -ne 'None'

#output the problem with the license
$badlicensedetails | Select-Object AssignedByGroupName, Error, UserPrincipalName | Sort-Object AssignedByGroupName, Error, UserPrincipalName

@pkrzysz
I'm going to assign this to the document author so they can take a look at it accordingly.

@pkrzysz thank you for calling this out. We are in the process of updating the article. #please-close


Unable to write

Hi im trying to use this lib
but when i try to do for example
$FB->set('/lotto',array('data'=>'valore'));
only the root will be changed with the value to '/lotto' and not as  a key

Thanks for pointing it out! @boardmain


Made some changes to the README file

🛠 Fixes issue #
📇 Changes proposed
✅ Check List (Mark all the applicable boxes)

[x] My code follows the code style of this project.
[x] This PR does not contain plagiarized content.
[x] This PR contains changes in the documentation
[x] This PR contains changes in the Code of the project
[x] The title of my pull request is a short description of the requested changes.

📄 Note to reviewers

📷 Screenshots

I would like to enhance the look of the README.md file by adding some colorful borders between sections to make it more readable

Please raise an issue for the same

Okay


Shadow variable in 'catch' should be an Error

Consider the following code:
	var x = 3;
	try{ throw "boom"; }
	catch (x) { }
	console.log(x);

According to JS spec catch should have its own scope. This behaviour can be observed in almost every browser, since they will all print 3 at the end. The exception is naturally IE which will print "boom" (tested up to IE9, would be interesting to see how 10 does with this, considering its high test262 score).
Right now as you can see, you can get your nice numeric vars turned into strings, or anything else for that matter. It doesn't matter whether x is in a function or global scope. This is extremely hard to notice and debug, you just wouldn't think about this.

:+1:


Add a doctor command to check prereqs

I got this idea from flutter. A doctor command is extremely helpful in scenarios when the users are troubleshooting issues. Since there are a few dependencies(k3s kubectl) that are expected to be installed before k3ai can work, a doctor command can be used to list errors if the deps are missing. This can evolve into a more detailed diagnostic with auto-remediation capabilities in the future. The output would also be helpful when someone wants to file defects as more diagnostic information can be built into this command as needed.

Didn't quite get the need here for a docker command which leads to a prereq of having docker itself.
When you have k3s installed, k3s kubectl will automatically be there.

Closing due inactivity


Map mongodb Ids to accessionIds

Proposed solution
Mongodb now creates its own ids on top of accessionIds we're creating. I think it would be it easier to check for previous accessionIds and return correct objects from db, if we just use accessionIds as mongodb:s internal ids.
Any thoughts about this?

Probably better to leave internal Id as it is, since it's optimized for mongo queries, sorting etc.


Bugs: hide-and-seek

Hello @AbdusSattar-70
I have reviewed your repository related to the local-storage task and found a bug there in line number 16 which required the variable to be added.

Hello @RHK-MICROVERSE,
Thank you for reviewing for giving me a good suggestion. I have solved issue #39


Add onBeforeInput listener

To handle a special case in element web (regarding pasting stuff from an iPhone to a Mac, see issue https://github.com/vector-im/element-web/issues/25327)  we require a before input listener to be added to the rich text editor.
This PR adds that listener and ensures it passes the event through when required to handle the case listed in the issue above. This listener will be required anyway to implement IME behaviour correctly (ie allowing composition of characters in certain foreign languages)

Codecov Report
Patch coverage has no change and project coverage change: +1.50 :tada:

Comparison is base (cfda115) 88.28% compared to head (22d1916) 89.79%.

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #710      +/-   ##
============================================
+ Coverage     88.28%   89.79%   +1.50%
============================================
  Files           145       79      -66
  Lines         16652    13932    -2720
  Branches        769        0     -769
============================================
- Hits          14702    12510    -2192
+ Misses         1765     1422     -343
+ Partials        185        0     -185

Flag
Coverage Δ

uitests
?

uitests-android
?

uitests-ios
?

unittests
89.79% <ø> (+1.98%)
:arrow_up:

unittests-android
?

unittests-ios
?

unittests-rust
89.79% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 66 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


python:slim does not contain _dbm

python:slim
$ docker run python:slim python -c 'import dbm.ndbm'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.11/dbm/ndbm.py", line 3, in <module>
    from _dbm import *
ModuleNotFoundError: No module named '_dbm'

$ docker run python:slim find /usr/local/lib/python3.11/lib-dynload/ -name '*dbm*' -print
/usr/local/lib/python3.11/lib-dynload/_gdbm.cpython-311-x86_64-linux-gnu.so

python
$ docker run python python -c 'import dbm.ndbm'

$ docker run python find /usr/local/lib/python3.11/lib-dynload/ -name '*dbm*' -print
/usr/local/lib/python3.11/lib-dynload/_dbm.cpython-311-x86_64-linux-gnu.so
/usr/local/lib/python3.11/lib-dynload/_gdbm.cpython-311-x86_64-linux-gnu.so

Is it possible to include it? Or could you please advise me how to use it?

Hmm, interesting -- I'm not sure what we're missing -- I've tried looking in Debian's python3.X package sources for clues, and I'm coming up empty. :confused:
Given that the non-slim is working, it's likely that we need to install some additional package from https://github.com/docker-library/buildpack-deps/blob/65d69325ad741cea6dee20781c1faaab2e003d87/debian/bullseye/Dockerfile (or one of the dependencies of those), but it's difficult to narrow down which one. :sweat_smile:

Thank you for your response.


fix selected crazy driver

FIxed bug with crazy driver when we selected a car

will still do crazy driver if there are more cars in the level
else there is no crazy driver

clearly don't need student.py in this PR :)
also check #4


E1120:No value for argument 'labelvalues' in constructor call?

Hi
The example provided can't be compiled due to "E1120:No value for argument 'labelvalues' in constructor call" for Summary class at this line.
REQUEST_TIME = Summary('request_processing_seconds', 'Time spent processing request')
Constructor is defined as
    def __init__(self, name, labelnames, labelvalues):
What value should be provided for the labelvalues?

Are you sure that's a compiler error?

Actually it comes from pylint.

Dupe of #210


Feature/ci add sementic release

Codecov Report

Merging #33 (e05d32c) into main (9df25ae) will not change coverage.
The diff coverage is 100.00%.

:exclamation: Current head e05d32c differs from pull request most recent head 8519d70. Consider uploading reports for the commit 8519d70 to get more accurate results

@@           Coverage Diff           @@
##             main      #33   +/-   ##
=======================================
  Coverage   91.95%   91.95%
=======================================
  Files          18       18
  Lines         696      696
  Branches       50       50
=======================================
  Hits          640      640
  Misses         51       51
  Partials        5        5

Flag
Coverage Δ

samples
78.84% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

apps/latte/src/runner.ts
94.25% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9df25ae...8519d70. Read the comment docs.


Dockerfile not building.

I am trying to build the image with:
docker buildx build --output type=docker . -t sdk-rust-contract-builder:next -f ./Dockerfile
like it says in the documents, but I get the following error:
ERROR: failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to read dockerfile: open /var/lib/docker/tmp/buildkit-mount2384260769/Dockerfile: no such file or directory
I haven't found any good resources for solving issues trying to build on multiversx, please help, thanks.

Hello,
Instead of building the image, you can pull it from here:
https://hub.docker.com/r/multiversx/sdk-rust-contract-builder/tags
However, if you'd like to build it locally, simply use docker build (instead of the more powerful docker buildx):
docker build . -t sdk-rust-contract-builder:next -f ./Dockerfile

Let us know how it works :pray:
Also, what OS are you using & what is the Docker version?

Perfect, that was the quickest response ever, thanks! It is building now. I might have messed up and been in a directory outside of the project. I am on Windows 10 currently, but am trying to learn the smart contracts on Multiversx at home on Ubuntu 22.04, but I can't get past an error 'error[E0554] #![feature] may not be used on the stable release channel-sc-derive,...', the main recommendation is to  run mxpy deps install rust --overwrite , cargo clean && cargo build , and rustup default nightly. Which none seem to work, I was thinking about uninstalling and re-installing everything. Docker version 20.10.22, build 3a2c30b

Glad to hear this, thank you!
@johnkirch123, for the second issue - note that we have an experimental setup for Visual Studio Dev Containers:
https://github.com/multiversx/mx-template-devcontainers
E.g. see the button "Dev Containers|Open", or the Code (green, upper-right) > Codespaces > Create codespace on main. It will set up an environment containing the necessary dependencies (rust and mxpy, as well). Note thought that the setup is still experimental / work in progress.


Rule proposal: JSDoc relaxation

ESLint has the valid-jsdoc rule which allows us to validate that certain JSDoc elements are present, when relevant. However, it complains when the React render method does not provide a return value in the JSDoc. i.e.
/**
 * React render
 */
render: function() {
  //...
}

This is just a nit and I have no idea how much power, if at all, eslint-plugin-react has towards the application of this rule by eslint, but is there any conceivable way in which it would prevent valid-jsdoc from emitting that error, there, for all React components? Sort of like, toggling the requireReturn to false if the method is called render.
Thank you!

Sadly, as far as I know, a plugin don't have the power to silent an error reported by an ESLint rule, the only way we have to do something approaching is to replicate the ESLint rule, "patch" it to fit our needs and use it instead of the ESLint one. It is not very clean and it can be time consuming to maintain the rule in sync with the ESLint one.
Also, about your case, I do not understand why you do not want to document the return of render. In my opinion it should be documented like any other function.

it can be time consuming to maintain the rule in sync with the ESLint one.

Agree.
It seems that adding simple @returns smth line to render() is much simpler solution.

Also, about your case, I do not understand why you do not want to document the return of render. In my opinion it should be documented like any other function.

You are right, but I find it would be cumbersome and repetitive to always document React's render as it's always gonna be the same. That being said, I think the fault is mine, I shouldn't have put that doc-block-looking comment immediately before the render method, I'll add an empty space in-between like I have done for other sections of my components. ("React properties", "React lifecycle", "Own methods", etc.)
Finally, thanks for confirming my doubts regarding the limitations of the plugin system!


Explanation for "#Params" value of 7 on ogbg-molhiv submission ( https://github.com/sangyx/gtrick/blame/a87f225ae851d1fb0c4330712c731723ebf95332/benchmark/pyg/README.md#L14  )

Hello @sangyx !
Thanks very  much for sharing this work.
For the ogbg-molhiv entry at https://web.archive.org/web/20240324173558/https://ogb.stanford.edu/docs/leader_graphprop/ for the GIN entry which links back to this repository (#24 , GIN, Yunxin Sang(SJTU)) the "#Params" column has a value of 7.

Can you confirm that value is correct, and if so, elaborate how the GIN model is run with only 7 parameters?
It appears per https://github.com/sangyx/gtrick/blame/a87f225ae851d1fb0c4330712c731723ebf95332/benchmark/pyg/README.md#L7 that the command would be python graph_pred.py --model gin to reproduce the ogbg-molhiv benchmark score.
Specifically, can you confirm whether it was EGNN with EGINConv layers ( https://github.com/sangyx/gtrick/blob/4491cc30f0c8810d8d2cf834fc5f5a8e84e0efc9/benchmark/pyg/model.py#L124 ) which appear to default to 5 layers with 300 channels ( https://github.com/sangyx/gtrick/blame/4491cc30f0c8810d8d2cf834fc5f5a8e84e0efc9/benchmark/pyg/graph_pred.py#L152C7-L153C68 ) now and 256 channels originally ( https://github.com/sangyx/gtrick/blob/108e956a6f7ecbf12202caecf7c7bb4f82757ab3/benchmark/graph_pred.py#L140 )
used for ogbg-molhiv 0.7778 ROCAUC as listed in https://github.com/sangyx/gtrick/blame/4491cc30f0c8810d8d2cf834fc5f5a8e84e0efc9/benchmark/pyg/README.md#L24 , and how many parameters per sum(p.numel() for p in model.parameters()) this has or had?
Even for a very simple model type like logistic regression, 7 parameters seems to be fewer than the number of atom and bond dimensions in the dataset, so would seem to require looking at only a subset of the dimensions to achieve that number. But for a GIN with multiple layers and nontrivial hidden dimension (regardless of whether 256 or 300 or other changes in repo since initial commit), how this model would only have 7 parameters seems unclear. Weihua Hu (OGB team)'s first demonstration GIN on the leaderboard for this problem had 1,885,206 parameters, by comparison. I do believe it could be done with orders of magnitude fewer, but 7 seems extreme for a GIN.
Thanks in advance for explaining this!

Thanks for your feedback, @willy-b.
You're right: the #Params on ogbg-molhiv are actually #hyperparams. If possible, could you help modify it?

tl;dr: only modification needed to get the current parameter count is add a print of OGB's recommended expression sum(p.numel() for p in model.parameters()), inserted right before run_graph_pred(args, model, dataset) at https://github.com/sangyx/gtrick/blob/4491cc30f0c8810d8d2cf834fc5f5a8e84e0efc9/benchmark/pyg/graph_pred.py#L182 to get these values (specifically, you can add print(f"model parameter count: {sum(p.numel() for p in model.parameters())}")).
For me to help get parameter counts for you, which I am happy to do or double check, I would need to confirm the exact configuration you used in your submission (e.g. whether hidden_channels was 300 or 256) as I have not yet reproduced your result. (I do try to guess below.)
For reference, per https://web.archive.org/web/20240324175343/https://ogb.stanford.edu/docs/leader_overview/ , it is the parameters of the model (not hyperparameters), to be listed:
#Params: The number of parameters of your model,
which can be calculated by `sum(p.numel() for p in model.parameters())`.
If you use multi-stage training (e.g., apply node2vec and then MLP),
please sum up all the parameters (both node2vec and MLP parameters).

They also say:
Please [contact<EMAIL_ADDRESS>if there is anything unclear or you have found any problematic submission on the leaderboards.

If you prefer (were busy or something) I can contact them once we agree on an update to the parameter count to get it updated but probably it would be smoothest if you contacted them there to update it. I am not a member of the OGB team (just took Stanford XCS224W whose professor's group helps manage OGB and which had a homework assignment related to ogbg-molhiv is why I was reviewing the entries) but would be happy to help you get this number corrected.
Using your default configuration ( https://github.com/sangyx/gtrick/blame/4491cc30f0c8810d8d2cf834fc5f5a8e84e0efc9/benchmark/pyg/graph_pred.py#L152C7-L153C68 ), it appears (shown later in this comment)
that applying their provided parameter counting expression to your unofficial implementation of the GIN model causes it to have more similar parameter counts to the OGB team's official GIN model entry for ogbg-molhiv, where theirs has 1,885,206 parameters listed in the #Params field (entry #33 at https://web.archive.org/web/20240324173558/https://ogb.stanford.edu/docs/leader_graphprop/#ogbg-molhiv ) (without virtual node or other tricks). Makes sense as OGB team also uses 5 layers, 300 emb dim for their GNN https://github.com/snap-stanford/ogb/blob/f631af76359c9687b2fe60905557bbb241916258/examples/graphproppred/mol/main_pyg.py#L72 by default same as default this project has at https://github.com/sangyx/gtrick/blame/4491cc30f0c8810d8d2cf834fc5f5a8e84e0efc9/benchmark/pyg/graph_pred.py#L152C7-L153C68 ). Note it is possible to reduce the parameter count potentially without reducing the performance by reducing the depth (e.g. from 5 to 2, there is evidence from GCNs, see discussion in https://arxiv.org/pdf/1806.03536 that two hops may be enough for many networks) and/or reducing the embedding dimension, but that is a separate discussion from what you are doing here.

To get some idea of the numbers expected,
I tried printing this for the default configuration of your graph_pred.py with GIN for ogbg-molhiv:
For graph_pred.py GIN, ogbg-molhiv with num_layers=5, hidden_channels=300 (your current default; https://github.com/sangyx/gtrick/blame/4491cc30f0c8810d8d2cf834fc5f5a8e84e0efc9/benchmark/pyg/graph_pred.py#L152C7-L153C68 ),
the number of parameters appears to be 1,885,506 (close to the value reported by the OGB team for their own GIN model; may make sense as OGB team also uses 5 layers,
300 emb dim for their GNN https://github.com/snap-stanford/ogb/blob/f631af76359c9687b2fe60905557bbb241916258/examples/graphproppred/mol/main_pyg.py#L72 by default):
# in google colab jupyter notebook
%cd /content/
!git clone https://github.com/sangyx/gtrick.git
%cd /content/gtrick/benchmark/pyg
!python graph_pred.py  --model gin --dataset ogbg-molhiv --device 0 --dataset_path /content/dataset # provided this dataset path is an empty writable folder

#Namespace(dataset='ogbg-molhiv', dataset_path='/content/dataset', device=0, log_steps=1, num_layers=5,
#hidden_channels=300,
#dropout=0.5, lr=0.001,
#batch_size=32,
#num_workers=0,
#model='gin', epochs=500, runs=10, patience=30)

#model parameter count: 1885506

# printed model object as well
#model:EGNN(
#  (node_encoder): AtomEncoder(
#    (atom_embedding_list): ModuleList(
#      (0): Embedding(119, 300)
#      (1): Embedding(5, 300)
#      (2-3): 2 x Embedding(12, 300)
#      (4): Embedding(10, 300)
#      (5-6): 2 x Embedding(6, 300)
#      (7-8): 2 x Embedding(2, 300)
#    )
#  )
#  (convs): ModuleList(
#    (0-4): 5 x EGINConv() # each has 366001 parameters, not tied/shared
#  )
#  (bns): ModuleList(
#    (0-4): 5 x BatchNorm1d(300, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
#  )
#  (out): Linear(in_features=300, out_features=1, bias=True)
#)
#model.convs[0-4]: EGINConv(), parameters: 366001 per layer
#model.convs[0-4].edge_encoder: BondEncoder(
#  (bond_embedding_list): ModuleList(
#    (0): Embedding(5, 300)
#    (1): Embedding(6, 300)
#    (2): Embedding(2, 300)
#  )
#)
#model.convs[0-4].mlp: Sequential(
#  (0): Linear(in_features=300, out_features=600, bias=True)
#  (1): BatchNorm1d(600, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
#  (2): ReLU()
#  (3): Linear(in_features=600, out_features=300, bias=True)
#)

# This may not be the configuration used by @sangyx as for 10 runs output from author's code on Google Colab L4 GPU,
# in initial checks I get 0.75 ROCAUC
# (a bit far from 0.7778 ROCAUC for ogbg-molhiv reported on the leaderboard and
# https://github.com/sangyx/gtrick/blame/4491cc30f0c8810d8d2cf834fc5f5a8e84e0efc9/benchmark/pyg/README.md#L24 )
# --------------------------------------------------------------------------------
# All runs:
# Best Train: 0.9240 ± 0.0208
# Best Valid: 0.8325 ± 0.0103
# Final Train: 0.8765 ± 0.0300
# Final Test: 0.7505 ± 0.0144
# --------------------------------------------------------------------------------

Earlier it appeared graph_pred.py had been configured for a default of num_layers=5, hidden_channels=256, ( https://github.com/sangyx/gtrick/blob/108e956a6f7ecbf12202caecf7c7bb4f82757ab3/benchmark/graph_pred.py#L140 )
so I checked the parameter count there as well and it appears to use 1,383,686 parameters in that case:
# in google colab jupyter notebook
%cd /content/
!git clone https://github.com/sangyx/gtrick.git
%cd /content/gtrick/benchmark/pyg
!python graph_pred.py --hidden_channels 256 --model gin --dataset ogbg-molhiv --device 0 --dataset_path /content/dataset

# Namespace(dataset='ogbg-molhiv', dataset_path='/content/dataset', device=0, log_steps=1,
#num_layers=5,
#hidden_channels=256,
#dropout=0.5,
#lr=0.001,
#batch_size=32,
#num_workers=0,
#model='gin',
#epochs=500,
#runs=3,
#patience=30)

# model parameter count: 1383686
# (performance measured in ROCAUC pending, takes long time to generate 10 runs; please let me know what configuration you ran so I can count parameters and reproduce the score you submitted to ogbg-molhiv for that configuration, thanks!)

Thanks!

Hey @willy-b, I've been quite busy lately. If you're interested, you could update the GIN record on OGB with the correct #params and your reproduction performance (And remember to add your name to this record). Feel free to experiment with different hyperparameters (lr, hidden_channels, dropout) and report the best results. The record I'm submitting to the leaderboard aims to demonstrate that even a simple model with deliberate tuning can outperform many so-called state-of-the-art GNN models.

Hey @sangyx , I would be happy to collaborate while fixing the misreported "#Params".
Since we are updating your OGB entry and collaborating, we may as well increase the reported score and rank with a tiny change.
I have proposed https://github.com/sangyx/gtrick/pull/11 which adds optional support for weight_decay, choice of add pooling instead of only mean pooling for converting node embeddings to graph embeddings, and option for a MLP after pooling node embeddings. These are off by default, controlled by argument to gtrick/benchmark/pyg/graph_pred.py only, so should not disrupt any existing usecases you have.
More details in the PR, but I give example hyperparameters using these features (no hyperparameter search was done, I just used the ones I found in my own work, 2 layers, embed dimension of 64, 1e-6 weight decay, add pooling, use MLP after pooling node embeddings; though I did confirm with just 2 layers and 64 embed dimension no code changes the network in this project did not perform as well) that improve on your currently reported score per the leaderboard methodology. See example sets of 10 runs here using your unmodified training/evaluation code to report the stats:  https://colab.research.google.com/drive/1MluOH3pOafwGdt9TKMqOBLfvLyeeDMX1?usp=sharing  (link public). You would remain the smallest parameter count for a GIN on the leaderboard and the 2nd smallest model despite removing the invalid reported "#Params" of 7. The example I tested is 50,947 parameters. Still much smaller than 1.8M actual parameters in your current default configuration and in that of the OGB team's GIN entry (as we reduce layers from 5 to 2 and embedding dimension to 64 from 300).
The reason to do this would be that as I had difficulty reproducing the previous ROCAUC score of this repo, instead of potentially replacing your score with a slightly lower one when fixing the "#Params" reported, the PR above appears to consistently (many sets of 10 runs, not seeing a single worse mean; though it is possible it rarely could do worse with new seeds) outperform the existing reported score, so we could actually improve your ranking while fixing the parameter count.
In doing this I am donating findings I was going to use for my own submission ( https://github.com/willy-b/tiny-GIN-for-ogbg-molhiv ) which I am happy to do as we don't need so many GINs on the leaderboard and I like the reusable structure of your project for enabling others to be more productive.
Let me know what you think of https://github.com/sangyx/gtrick/pull/11 , thanks!

Hi @willy-b, thanks for your contribution. This is an excellent improvement. I've merged your PR and added a note to the readme.

Thanks!
I am emailing OGB shortly to try to get the entry updated to correct the "#Params" column and update to latest score and will CC you.

Hello,
As you know (but for the record here), I emailed them at 2024-07-18 23:12:50 -0700 (PT) notifying them you approved a correction of the parameter count in your solution from 7 to 50,947  (that is you accidentally reported the hyperparameter count on Apr 30, 2022 instead of parameter count) and did not receive a follow up on that yet:
Date: Thu, 18 Jul 2024 23:12:50 -0700
Subject: Fwd: Update to ogbg-molhiv entry #24 from Yunxin Sang (Per https://github.com/sangyx/gtrick/issues/10 )
From: Willy Bruns<EMAIL_ADDRESS>To<EMAIL_ADDRESS>Cc: Yunxin Sang<EMAIL_ADDRESS>
(note our emails are already visible at https://ogb.stanford.edu/docs/leader_graphprop/ )
I also mentioned it in another email to them about a different topic on 2024-08-23. They did take a couple of months to get back to me about something else so this may just be because the project is not very active and understaffed. The Google Group is essentially silent this year for example.
Thanks!


inprove your readme , and add mit license

Good contribution.

Currently you can see the images uploaded by other users, it is a future plan to implement a fully interactive system for the user.


Retransmit 0-RTT data with FW security as soon as possible

We stop accepting the non-FW encryption after receiving the first FW-secured packet in the server. The client should therefore retransmit 0-RTT data using FW encryption, not always with the same level as the initial transmission.

We don't have 0-RTT implemented for the client, so I don't think this is an issue.

Okay, but I'd still like to keep this issue around as it's a non-obvious part of the change required for 0-RTT (#438).

This is obsolete now that we moved to TLS.


Add AIP download from AM Storage Service

Draft, the actual download to the storage service is not implemented,
this is just a proof of concept for how it might work.

#820

Replaced by https://github.com/artefactual-sdps/enduro/pull/831.


(PUP-3323) Ensure we don't check gpg

When testing the yum utilities, we are creating an empty rpm package so
that we can install from a repo without hitting any external repos.
However, we aren't bothering to sign that empty package. When we go to
do the actual install, yum bails out because the package isn't signed.
This is great, but not in this instance. We don't care that the package
isn't signed, we just want to make sure the install works. This commit
adds in a flag to the install to make sure yum doesn't check the
signature.

Nice, that's easy. Did you try running it and see that it passes?


Single relation DDL, DML

I work a moderate size web application where most of its tables are modeled and stored in Postgres.
We have one table that needs to be customized per user. Ideally, we would use an off the shelf Haskell library to implement all DDL and DML operations for a single table in memory, and then persist the relation definition and relation contents to postgres whenever they changed.
Would you recommend forking this library to accomplish that? Are there other libraries in this space that you would recommend?

The use-case you describe is certainly possible and we have at least one user doing something similar, however, the implementor will need to be keenly aware of the feature differences between postgresql and Project:M36. For example, postgresql does not support algebraic data types, nested relations, or a transaction graph and Project:M36 does not support SQL or indexing. The implementor would also need to create the Project:M36<->postgresql mapping which, depending on your scenario, could be non-trivial.
You might also evaluate SQLite and postgresql temporary tables.
In any case, let us know how we can help to inform your design decision.

This library seems like the only option for having a definition of the relation in memory, where there can be DDL operations like adding a column, while still being able to hijack the DML logic to write to one fat table with all columns nullable. I don't need anything fancier like nested relations.
I could have used something like sqlite, if there was a Haskell implementation of the sqlite engine (not just bindings to it's query interface).
It will be sometime until my use case matures enough to actually use m36 instead of a simple imitation fork, but if you take interest in this use case, I would want only the following:

data types for expressing definition of a relation, and it's contents
operations for modifying the relation definition, which operating on an in memory dictionary
interface for expressing queries, DML, but not tied to any implementation

Then I would take this skeleton and implement as desired.

In that case, I think Project:M36 would suit your use-case just fine.

The Tupleable interface will allow you to marshal Haskell ADTs to tuples and back.
Project:M36 certainly supports transaction-safe DML.
Both DML and DDL interfaces (also Haskell ADTs) could be sent to a secondary, SQL-based engine, though not all the features would make sense.

Be aware that Project:M36 does not support NULL, however, Maybe as a value is supported and can approximate NULL but without three-valued logic.

Let me know if you have any more questions; I will close this ticket for now.

Thanks for your help.


Add ability to store and get user defined settings from taf config

as TAF already has it’s own config, it will be good idea to make it customizable by user settings to eliminate creating additional configs for user settings

done in f98ebe9


Have you thought about HAproxy

Is your feature request related to a problem? Please describe.
Using this in front of an exchange server and authentication is an issue due to lack of NTLM support
Describe the solution you'd like
I'd live to see a for or option to use HAproxy engine
Describe alternatives you've considered
N/A
Additional context
Love this as its been setup for coming up 6 months with no issues

Auth request related to #159
As for haproxy, is it that you just require load balancing that's already built in to nginx?


Wrong order of invoking completion provider and change event

VSCode Version: 1.13.0
OS Version: macOS  10.12.5

The order in which code completion and change events are triggered is currently wrong. When a user types a letter which triggers code completion extensions must first be able to process this new input (e.g. parse the document) to find proper completion candidates. Otherwise the new input is not considered and wrong candidates are listed. Hence the order should be switched. If that is not possible or feasible it should be possible to trigger code completion from the text change handler and suppress the standard invocation.
Steps to Reproduce:

Use an extension with both a completion provider and the workspace.onDidChangeTextDocument event.
Open a document which triggers this extension.
Add breakpoints in provideCompletionItems in the completion provider and workspace.onDidChangeTextDocument.
Type a letter and watch the order both breakpoints are hit (completion comes first).

With the code below I cannot reproduce this. The document is properly updated as I type and the change event comes before the completion request is send
    vscode.workspace.onDidChangeTextDocument(e => {
        console.log('onDidChangeTextDocument', e.document.getText());
    });

    vscode.languages.registerCompletionItemProvider('abc', {
        provideCompletionItems(doc, pos) {
            console.log('provideCompletionItems', doc.getText());
            return [
                new vscode.CompletionItem('Hello'),
                new vscode.CompletionItem('World'),
            ];
        }
    });

Oh, you are right. I'm sorry, the out-of-order handling is my fault. I have a timer in place to coalesc multiple quick changes into one to avoid time costly parsing actions while typing quickly.
It would be very nice if vscode would help here and not send a change event on every key stroke (if within a certain timeout) and consequently trigger follow-up actions like code completion later as well. As you can see I can handle keystrokes, but it's very tricky to align that with calls made by vscode then.

It would be very nice if vscode would help here and not send a change event on every key stroke (if within a certain timeout) and consequently trigger follow-up actions like code completion later as well.

That we cannot do because there are other extensions and state relying on the fact that events aren't debounced. I understand that heavy operations like parsing etc need that and our recommendation is to work with the TextDocument#version-property is such cases. Thanks and Happy Coding

Thanks for the hint and the prompt handling of my inquiry.


Make fromBytes() public.

External modules (like HTTP transport) need to convert byte streams
into composites: for instance, when decoding error messages.
Signed-off-by: Greg Bean<EMAIL_ADDRESS>

Good point.  I will switch it over to using fromObject at relevant points.


Ability to specify filename for helm based git write back target

We would like to use argocd image updater to automatically update the development branch of our application config repo and write back the version to git.  This works well.  We would then like to promote to test by merging this file into our test branch (which does not use image updater) and eventually prod branch which also does not use image updater.  The issue is, each of our applications is named differently, myapp-dev, myapp-test, myapp-prod and the file written to git is ".argocd-source-<appname>.yaml".  I would like to make image updater use the ".argocd-source.yaml" instead so it would be honored in all environments like it is documented in Argo docs, https://argo-cd.readthedocs.io/en/stable/user-guide/parameters/#store-overrides-in-git
The solution I am thinking is to make this similar to the kustomize configuration so you could specify:
argocd-image-updater.argoproj.io/write-back-target: "helm:/.argocd-source.yaml"
I think the above gives the most flexibility in the case that Argo starts accepting different filenames in future releases.  You could also implement it as:
argocd-image-updater.argoproj.io/write-back-target: "helm:exclude-appname"
or something similar to denote that the templated name of .argocd-source-<appname>.yaml should not be used which aligns with ArgoCD's implementation of either having app name or not.

After writing this up and discussing with a colleague, we came up with a work-around until this can be implemented.  You can create a symlink from the environment file that you use argocd image updater on to the generic, non app specific filename.  We just tested this out and it works as expected.
ln -s .argocd-source-myapp-dev.yaml .argocd-source.yaml
Commit symlink file