Broken cni config

Bug description
For unknown reason Istio deletes a configuration file of Cilium, and 'containerd' can't finish network plugin initialization. Looks like Istio tried read config when them not ready yet, if add some init container with sleep(example below) everything up successfully
initContainers:
- command:
  - sh
  - -c
  - sleep 10s
  image: busybox:1.32
  name: give-time-for-cilium

Also if I create file manually, everything work fine
cat <<EOF > /etc/cni/net.d/05-cilium.conflist
{
  "cniVersion": "0.3.1",
  "name": "k8s-pod-network",
  "plugins": [
    {
      "enable-debug": false,
      "name": "cilium",
      "type": "cilium-cni"
    },
    {
      "kubernetes": {
        "cni_bin_dir": "/opt/cni/bin",
        "exclude_namespaces": [
          "istio-system"
        ],
        "kubeconfig": "/etc/cni/net.d/ZZZ-istio-cni-kubeconfig"
      },
      "log_level": "info",
      "name": "istio-cni",
      "type": "istio-cni"
    }
  ]
}
EOF

systemctl restart containerd

Logs:
Containerd:
level=error msg="failed to reload cni configuration after receiving fs change event(\"/etc/cni/net.d/05-cilium.conf.tmp.716574100\": WRITE)" error="cni config load failed: failed to load CNI config file /etc/cni/net.d/05-cilium.conf: err
level=error msg="failed to reload cni configuration after receiving fs change event(\"/etc/cni/net.d/05-cilium.conf\": RENAME)" error="cni config load failed: no network config found in /etc/cni/net.d: cni plugin not initialized: failed
level=error msg="failed to reload cni configuration after receiving fs change event(\"/etc/cni/net.d/05-cilium.conflist\": REMOVE)" error="cni config load failed: no network config found in /etc/cni/net.d: cni plugin not initialized: fai
level=error msg="failed to reload cni configuration after receiving fs change event(\"/etc/cni/net.d/ZZZ-istio-cni-kubeconfig.tmp.664862221\": WRITE)" error="cni config load failed: no network config found in /etc/cni/net.d: cni plugin n
level=error msg="failed to reload cni configuration after receiving fs change event(\"/etc/cni/net.d/ZZZ-istio-cni-kubeconfig.tmp.664862221\": RENAME)" error="cni config load failed: no network config found in /etc/cni/net.d: cni plugin

Pod istio-cni-node, Container install-cni:
info    install cni with configuration:
CNINetDir: /etc/cni/net.d
MountedCNINetDir: /host/etc/cni/net.d
CNIConfName:
ChainedCNIPlugin: true
CNINetworkConfigFile:
CNINetworkConfig: {
  "cniVersion": "0.3.1",
  "name": "istio-cni",
  "type": "istio-cni",
  "log_level": "info",
  "kubernetes": {
      "kubeconfig": "__KUBECONFIG_FILEPATH__",
      "cni_bin_dir": "/opt/cni/bin",
      "exclude_namespaces": [ "istio-system" ]
  }
}
LogLevel: warn
KubeconfigFilename: ZZZ-istio-cni-kubeconfig
KubeconfigMode: 0600
KubeCAFile:
SkipTLSVerify: false
K8sServiceProtocol:
K8sServiceHost: <IP_ADDRESS>
K8sServicePort: 443
K8sNodeName: ip-100-100-100-101.eu-central-99.compute.internal
UpdateCNIBinaries: true
SkipCNIBinaries: [[]]

2021-07-12T13:21:42.055425Z     info    Copied istio-cni to /host/opt/cni/bin.
2021-07-12T13:21:42.068778Z     info    Copied istio-iptables to /host/opt/cni/bin.
2021-07-12T13:21:42.068820Z     info    Directory /host/secondary-bin-dir is not writable, skipping.
2021-07-12T13:21:42.069043Z     info    write kubeconfig file /host/etc/cni/net.d/ZZZ-istio-cni-kubeconfig with:
# Kubeconfig file for Istio CNI plugin.
apiVersion: v1
kind: Config
clusters:
- name: local
  cluster:
    server: https://[<IP_ADDRESS>]:443
    certificate-authority-data: <CA cert from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt>
users:
- name: istio-cni
  user:
    token: "<redacted>"
contexts:
- name: istio-cni-context
  context:
    cluster: local
    user: istio-cni
current-context: istio-cni-context

2021-07-12T13:21:42.069411Z     info    Using CNI config template from CNI_NETWORK_CONFIG environment variable.
2021-07-12T13:21:42.069435Z     info    CNI config: {
  "cniVersion": "0.3.1",
  "name": "istio-cni",
  "type": "istio-cni",
  "log_level": "info",
  "kubernetes": {
      "kubeconfig": "/etc/cni/net.d/ZZZ-istio-cni-kubeconfig",
      "cni_bin_dir": "/opt/cni/bin",
      "exclude_namespaces": [ "istio-system" ]
  }
}
2021-07-12T13:21:44.581124Z     info    CNI config file /host/etc/cni/net.d/05-cilium.conf exists. Proceeding.
2021-07-12T13:21:44.581527Z     info    Renaming /host/etc/cni/net.d/05-cilium.conf extension to .conflist
2021-07-12T13:21:44.581570Z     info    Created CNI config /host/etc/cni/net.d/05-cilium.conflist
2021-07-12T13:21:44.581775Z     info    Invalid configuration. CNI config file removed: /host/etc/cni/net.d/05-cilium.conflist
2021-07-12T13:21:44.582000Z     info    Restarting...
2021-07-12T13:21:44.623193Z     info    Copied istio-cni to /host/opt/cni/bin.
2021-07-12T13:21:44.637836Z     info    Copied istio-iptables to /host/opt/cni/bin.
2021-07-12T13:21:44.637874Z     info    Directory /host/secondary-bin-dir is not writable, skipping.
2021-07-12T13:21:44.638122Z     info    write kubeconfig file /host/etc/cni/net.d/ZZZ-istio-cni-kubeconfig with:
# Kubeconfig file for Istio CNI plugin.
apiVersion: v1
kind: Config
clusters:
- name: local
  cluster:
    server: https://[<IP_ADDRESS>]:443
    certificate-authority-data: <CA cert from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt>
users:
- name: istio-cni
  user:
    token: "<redacted>"
contexts:
- name: istio-cni-context
  context:
    cluster: local
    user: istio-cni
current-context: istio-cni-context

2021-07-12T13:21:44.638389Z     info    Using CNI config template from CNI_NETWORK_CONFIG environment variable.
2021-07-12T13:21:44.638412Z     info    CNI config: {
  "cniVersion": "0.3.1",
  "name": "istio-cni",
  "type": "istio-cni",
  "log_level": "info",
  "kubernetes": {
      "kubeconfig": "/etc/cni/net.d/ZZZ-istio-cni-kubeconfig",
      "cni_bin_dir": "/opt/cni/bin",
      "exclude_namespaces": [ "istio-system" ]
  }
}

Journalctl:
Jul 12 15:41:23 ip-100-100-100-101.eu-central-99.compute.internal kubelet[5043]: E0712 15:41:23.582025    5043 kubelet.go:2195] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Jul 12 15:41:24 ip-100-100-100-101.eu-central-99.compute.internal kubelet[5043]: W0712 15:41:24.126910    5043 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
Jul 12 15:41:28 ip-100-100-100-101.eu-central-99.compute.internal kubelet[5043]: E0712 15:41:28.594592    5043 kubelet.go:2195] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Jul 12 15:41:29 ip-100-100-100-101.eu-central-99.compute.internal kubelet[5043]: W0712 15:41:29.127235    5043 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d

[ ] Docs
[X] Installation
[X] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
[ ] Upgrade
Expected behavior
Istio will create file /etc/cni/net.d/05-cilium.conflist. Validation of this file must be passed. After that containerd must be up and ready
Steps to reproduce the bug
Install cilium and istio to the cluster
Version (include the output of istioctl version --remote and kubectl version --short and helm version --short if you used Helm)
istioctl version --remote
client version: 1.10.0
control plane version: 1.10.0

kubectl version --short
Client Version: v1.18.0
Server Version: v1.18.16-eks-7737de

helm version --short
v3.2.4+g0ad800e

Cilium: 1.9.7
How was Istio installed?
Helm chart
Environment where the bug was observed (cloud vendor, OS, etc)
Cloud vendor: AWS EKS
OS:
cat /etc/os-release
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"

@khorn7sk could you paste original /host/etc/cni/net.d/05-cilium.conf file here?

@bianpengyuan
cat /etc/cni/net.d/05-cilium.conf
{
  "cniVersion": "0.3.1",
  "name": "cilium",
  "type": "cilium-cni",
  "enable-debug": false
}

Actually look at the log,  what Invalid configuration. CNI config file removed: /host/etc/cni/net.d/05-cilium.conflist suggests is that the conflist is removed by others, instead of Istio CNI. Istio CNI only does periodic check on the config file, complains with that message if it cannot find the file, and retry installation.
Not sure how cilium installation is done, but I guess the race condition between Istio and cillium CNI installation somehow makes cillium installer consider the node CNI configuration is not done properly and does a clean up? Is there log about cillium installation? If possible, I think you can taint the node to prevent Istio CNI being installed before cillium is ready.

@bianpengyuan I compared logs(from all pod) from 2 nodes, healthy and broken. And didn't found any difference(I mean in cilium logs), everything is clear. Can you explain you idea about taint?

It is just a way to mark the node to repel pods until becomes ready (i.e. until cilium proper runs) https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/. This is just a possible way, any method that guarantees cilium ready before other pods starting up on the node would work (probably cilium already provide a solution, not sure). k8s vendor typically provide such kind of guarantee.


Swap @if burguers statement into a @mixins

Hello, nice burguers 👍 🍔  !
Usually each project has just one burguer.
And of course we can go into the files and comment the @import, but this means touching files in node_modules ( that problably will rely on a problem if we try a package version update ) 😢 .
Wouldn't be a better approach just have @mixins for each burguer ( and have them listed in the DOC) so i can @include just the one that i want ?
Keeping your files clean 😃.

@HectorLS Hey Hector, thanks!
This was designed in a way that you won’t have to mess with anything in node_modules and also include just the ones you want.
In the Customization section of the README.md there are instructions on how to do so.
 For example, if you only want the squeeze variant, you would put the following in your settings file or one of your .scss files for your project:
$hamburgers-types: (
  squeeze
);

When the CSS is compiled, only that type will be included.
Hope that helps.


v17.3.0 implement Kahlan tests

implement Kahlan tests

[Current coverage][cc-pull] is 100%

Merging [#431][cc-pull] into [master][cc-base-branch] will not change coverage

@@           master   #431   diff @@
====================================
  Files          24     24
  Lines         358    358
  Methods        81     81
  Messages        0      0
  Branches        0      0
====================================
  Hits          358    358
  Misses          0      0
  Partials        0      0

Powered by Codecov. Last updated by [fee4124...d1f34c6][cc-compare]
[cc-base-branch]: https://codecov.io/gh/eugene-matvejev/battleship-game-api/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/eugene-matvejev/battleship-game-api/compare/fee41243124f1f2471e047f226d4255dc03a4292...d1f34c60119205834de3a9e5c6a8d52002c8b1f0
[cc-pull]: https://codecov.io/gh/eugene-matvejev/battleship-game-api/pull/431?src=pr


Hello!I found the conflict Node!!!

I found the conflict node with N-sidebar,please tell others!!
https://github.com/christian-byrne/comfyui-search-navigation
this node cause the N-sidebar invaild!!I try 300 nodes and found it!

Thanks! I have slightly modified the code so that it can also coexist with that extension!

Thanks for your work again and again


[$250] RBR displayed on expense preview but no error in expense details on Employee 1 side

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 9.0.23-0
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail: n/a
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: Applause internal team
Slack conversation:
Action Performed:
Pre-requisite: Create a Control workspace, set "Manually approve all expenses over" to $10 and set Advanced approval in OD as:

Boss owner (admin) submits to Mini owner
Mini owner (admin) submits Boss owner
Manager (employee) submits to Mini owner, forwards to Mini owner, over limit forwards (limit 100) to Boss owner
Employee 1 (employee) submits to Manager

Expected Result:
RBR on expense preview should be displayed if there is an error displayed on expense details page.
Actual Result:
RBR is displayed on expense preview but there is no error on expense details page.
Workaround:
unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
https://github.com/user-attachments/assets/aa34e55b-30cc-4863-b7c2-2c331ce4de89

View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01c80d6113db7d8a4e
Upwork Job ID:<PHONE_NUMBER>681672456
Last Price Increase: 2024-08-30

Issue OwnerCurrent Issue Owner: @ntdiary

I can't reproduce this one reliably, going to assign External to see if C+ can!

Any luck @ntdiary?

Ok, thanks for trying! Lets close this one then.


support building private repos

Is it possible to plumb through private GitHub repo building using ocluster at the moment?
I tried modifying (a local) ocurrent-deployer with a GitHub app that has access to a private repository. That lets the deployer query the repository, but then the actual build when submitted to a ocluster pool fails to clone the repository.
As far as I can tell, the best way to do that is to pass the GH access token through to the git clone in the clone URL.  This probably needs support in ocluster to hide the token from the logs...

@tmcgilchrist - had you experimented with this a while back? The major concern - at least for the “main” cluster, is that the trust model for the workers changes as soon as we have them receiving secrets.

It’s possible to get it working I had a prototype working last year.
There’s a feature request on ocaml-ci for it as well
https://github.com/ocurrent/ocaml-ci/issues/323
The two concerns were the security of private repos (as David mentioned) we
cache everything in a shared directory and don’t really care about sharing
git repos, and no one was strongly asking for it at the time.
What is your use case for it?
On Sat, 14 May 2022 at 8:52 pm, David Allsopp @.***>
wrote:

@tmcgilchrist https://github.com/tmcgilchrist - had you experimented
with this a while back? The major concern - at least for the “main”
cluster, is that the trust model for the workers changes as soon as we have
them receiving secrets.
—
Reply to this email directly, view it on GitHub
https://github.com/ocurrent/ocluster/issues/179#issuecomment-1126794102,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABJXOPAFFFLQ2SDKMDJBT3VJ7Y6FANCNFSM5V55D6IA
.
You are receiving this because you were mentioned.Message ID:
@.***>

My usecase is pretty straightforward: got a bunch of private repos that need building and deploying. The builds are pushed to the github container registry (which has the advantage that the access token used can both clone the repository and push to the ghcr.io container registry with the same access privs as the source repository).
The deployer works great; the only hitch is that I can't farm out the builds to an ocluster instance.

If you trust the ocluster instance then we could just plumb through the right information and get it working, however that is somewhat of a hack but perhaps is enough in this case. For my original use case, I just wanted to be able to build private GitHub repos and didn't care too much about whether the source was available on the cluster. The only security was not having the repo as a public repo on GitHub.
A better solution would be making the private git repos truely private, maybe by having a private git cache store per GH user or some other segregation of the repositories. We haven't really investigated this area of the design space.

Agreed that plumbing through the auth info would be enough in the first instance.  Can do more sophisticated ocluster reservation later.


Run CI on Julia v1.10

Since Julia v1.11 will be out probably soon and v1.10 will be the new LTS, I would like to run CI also on v1.10.

Sounds reasonable. However, we still support Julia v1.9 - and I would like to keep it this way unless you have a very good reason not to do so. My motivation for this is mainly that there are people out there who are not power users of Julia like we, so they will likely have older versions of it. I would like to support them. That's something I have experienced with BSeries.jl.
Thus, we could for example keep one job on Julia v1.9 and move the others to Julia v1.10.

Thus, we could for example keep one job on Julia v1.9 and move the others to Julia v1.10.

Sounds good. I added a test, which runs with v1.9 on Ubuntu.

What about the Downgrade job? I would run it on v1.10. Do you agree?

What about the Downgrade job? I would run it on v1.10. Do you agree?

I don't have a strong opinion there. One could argue that the oldest versions of all dependencies will likely be available on the oldest Julia versions, though. But as I said, I'm fine either way

Ok, let's keep it on v1.9 for now.

Sorry, I mixed up the PRs. I wanted to merge the other one...

Are you fine with this PR @ranocha? If yes, I would create a new release. If not, I would first consider your suggestion before creating a new release.
Sorry again for not waiting for your review :grimacing:


Compilation failure with rust-nightly because of external_doc feature

https://github.com/zkcrypto/subtle-ng/blob/73b8a16e5aed0ddccecfd3410c9323c6a0cd35cd/src/lib.rs#L12
The current version (2.4.1) fails to compile with rustc 1.54.0-nightly (35fff69d0 2021-06-06)  because the "external_doc" feature has been removed. For more information, see this PR.

Not sure what the best way to fix this is.
Switching to
#![cfg_attr(feature = "nightly", doc = include_str!("../README.md"))]

doesn't work on stable, because the right hand side of the cfg_attr has unstable syntax, even though it's not taken.
This is supposed to be stable in 1.54, but I think that doesn't land for three weeks or so.


Fix genesis api's fork version

While testing for withdrawals devnet4/6, it surfaced that in getGenesis api response, genesis fork version is expected to be the config's GENESIS_FORK_VERSION constant in spite of other forks being scheduled at genesis:

This PR fixes the same

:tada: This PR is included in v1.6.0 :tada:


Possible race conditions in PFIO multi group server

User @mmanyin has reported the following symptoms. When using the multi-group PFIO server they will occasionally find the that the run crashes very infequently and throws an error in the server when opening the file (in the NetCDF4_FileFormatter) for writing on the server side. Restarting the segment runs through just fine so does seem like a possible race condition.
The other piece is that this only seems to occur when the user has a collection that is writing multiple time steps to the file for example:
  bot_inst_1hr_g1440x721_x1.format:       'CFIO' ,
  bot_inst_1hr_g1440x721_x1.descr:        '2d,1-Hourly,Instantaneous,Single-Level,Bottom Layer Diagnostics' ,
  bot_inst_1hr_g1440x721_x1.conservative: 1
  bot_inst_1hr_g1440x721_x1.nbits:        12 ,
  bot_inst_1hr_g1440x721_x1.template:     '%y4%m2%d2_%h2%n2z.nc4' ,
  bot_inst_1hr_g1440x721_x1.archive:      '%c/Y%y4/M%m2' ,
  bot_inst_1hr_g1440x721_x1.mode:         'instantaneous' ,
  bot_inst_1hr_g1440x721_x1.grid_label: PC1440x721-DC
  bot_inst_1hr_g1440x721_x1.chunksize:    1440 721 1 1 ,
  bot_inst_1hr_g1440x721_x1.deflate:      1 ,
  bot_inst_1hr_g1440x721_x1.frequency:    010000 ,
  bot_inst_1hr_g1440x721_x1.duration:     240000 ,
  bot_inst_1hr_g1440x721_x1.ref_time:     000000 ,
  bot_inst_1hr_g1440x721_x1.levels:       72
  bot_inst_1hr_g1440x721_x1.fields:       'PS'          , 'DYN'     ,

One possibility is that if somehow the file were still being written due to disk slowness and this causes some sort of NetCDF error but this is all speculation.
I will attempt to reproduce this by running ExtDataDriver.x with the multi-group server and producing multi-step files. If this can't be reproduced it will be hard to fix.
In theory, if this really is an issue appending to an existing file in the multi-group server is would be possible to have some sort of protection in theory, when the "coordinator" process assigns work, if the file name in the request matches the file a worker is still writing, well something I suppose could be done although would require some thought.

well that was easy to reproduce. I already reproduced this, just told ExtDataDriver.x to use the multigroup server, told it to write an hourly file with a duration of 24 hours and it crashed after a while so at least we can reproduce, now to see if there's any reasonable fix.

It was fixed.


ReferenceError: SecretKey is not defined

How do you import the SecretKey constructor?
It's used in the examples.

SecretKey is part of an older version of the crypto-utils library. You can install the proper version via the command:
npm install<EMAIL_ADDRESS>then import via:
import { SecretKey } from '@cmdcode/crypto-utils'

Ok thank you! I'm still getting "TypeError: SecretKey is not a constructor" even with 2.0.2.

Would love to see some updated code as well!

Ok thank you! I'm still getting "TypeError: SecretKey is not a constructor" even with 2.0.2.
runnin into this as well..

and then the previous util.getPublicKey() function doesn't exist in crypto-utils either.

In the latest version of @cmdcode/crypto-utils:
import * as ecc from '@cmdcode/crypto-utils'

const seckey = ecc.keys.gen_seckey()

Everything is typed so you should be able to explore the API for ecc and ecc.keys using vscode or any code editor that supports completions with typescript.
I am still working on updating the documentation for @cmdcode/crypto-utils and @cmdcode/tapscript in my free time. I apologize for the delay! Let me know if you have any further questions on this.


Corrected a typo, which broke the rule in line 21

Corrected a typo, which broke the rule in line 21.

I opened issue #74 for this pull request.

Thanks for the fix!


:smiling_imp:修改 Windows 下 npm start  启动报 Module not found: Error: Cannot resolve 'file' or 'directory' 错误

@eitel Thanks :+1:  #4 resolved.


Add missing fields

Please add the following missing fields to the HppResponse object:
CARD_PAYMENT_BUTTON
MERCHANT_RESPONSE_URL
HPP_LANG
SHIPPING_CODE
SHIPPING_CO
BILLING_CODE
BILLING_CO

Never mind. Can see they are accessible via getSupplementaryData method

Thanks for the feedback. We'll add specific methods in the next release.
Seán MacDomhnall
Realex Payments


Mesh Index Message Correction

The Mesh class supports both triangle and quad meshes. The type is inferred through the "index" array being of type vec3ui or vec4ui. However, when no data is present, the error message only mentions vec4ui. This change is related to an issue from Ospray Anari.
I've added a check to see if any data is present beforehand. If not, the error message now states that vec3ui or vec4ui is allowed.
Old message:
[ERROR] ospray::Mesh must have 'index' 1D array with element type vec4ui
New message:
[ERROR] ospray::Mesh must have 'index' 1D array with element type vec3ui for triangle meshes or vec4ui for quad meshes.

superseded by 2fc5e250a29045a75d0eea3c5b98b8e12efd3605


Implement GetSmartContractSubState API

GetSmartContractSubState allows us to return the substate, variable or structure of a large subset. The API is not currently supported.
for eg:
curl --request POST \
  --url http://localhost:4201/ \
  --header 'Content-Type: application/json' \
  --data '{
    "id": "1",
    "jsonrpc": "2.0",
    "method": "GetSmartContractSubState",
	"params": ["zil15tjx2l0gzzxaxucwk50stgw5smthhckl8qcqmf","bnum_req", []]
}'

cc @DrZoltanFazekas @bzawisto @JamesHinshelwood

@chetan-zilliqa Isn't a duplicate of https://github.com/Zilliqa/zq2/issues/1545

Duplicate of #1545

oh yeah, sorry,  I should have checked before. Thanks @saeed-zil and @JamesHinshelwood


Reworked LandmarkMapEvaluator

The LandmarkMapEvaluator now works similar to the LandmarkPointEvaluator.

The unspecifc PixelImage[Double] were changed to a  class LandmarkMap.
The class can be used directly to correct for the false-positive and false-negative ratios.
A method is provided to precalculate the map with added noise model using the GeneralMaxConvolution.

I am concerned mainly with the Unit return type in LandmarkMapEvaluator.apply

The updated pull-request addresses all comments from the reviewers. It should solve most comments and some are now outdated.

Looks good


1.0.0-beta.3 Update

This project is long overdue for an update, so I'm giving it a shot.
The main goal of this update is to get it on Storybook v8 and the latest version of SolidJS.
Work done so far:

Removed old artifacts (like Lerna git heads)
Updated yarn
Made bundling experience closer to main repo
Fixed out-of-date code
Updated dependencies
Added .editorconfig

Things to do:

[x] Make sure reactivity is preserved (when splitting the render.tsx into renderToCanvas.tsx)
[x] Investigate the use of delay (there appears to be a 20ms delay to take care of a race condition in async code; smells really bad)
[x] Investigate use of npx to run jiti on other platforms (jiti doesn't appear on Windows, so I call it through npx)
[x] Make sure exports are set up correctly
[x] Update templates
[x] Add in documentation for future devs
[x] Update README
[x] Investigate Vite's deprecation of the CJS build of the Node API
[x] Investigate race condition in bundler
[x] Add CI for formatting and linting
[x] Refactor CD actions
[x] Add bundling check script

Right now, my branch appears to work (with a small project linked via yarn). I say appears to work, since I never used Storybook (came from React Cosmos and my own Vite tool), so I have a limited frame of reference to use.

@webblocksapp Hey, is there a reason for delaying rendering in docs mode? It seems to work properly without it (at least in Storybook v8).

Hello @JonahPlusPlus , I’m currently not very active in contributing to this library. I added this delay because the Storybook documentation mode took a while to render, and sometimes the screen would remain blank. If you find that this delay is unnecessary and everything works fine without it, feel free to discuss the PR with the Storybook team through their official Discord channel. Best regards.

@JonahPlusPlus Exciting, thanks so much for picking this up! If you have any Storybook q's please give us a holler here or in our Discord and we'll do our best to help

@webblocksapp Understood, thanks for the super quick reply! I suppose I'll try creating more stories until I get some lag and see if the problem remains.
@shilman No problem, I need something like this in my projects and I would rather not write an entire library from scratch!

@JonahPlusPlus If you're coming from Cosmos and have a few minutes to spare, I'd love to ask you some questions about things we might do better in Storybook.

@shilman Sure, we can discuss it on the Discord server.

Everything is almost done, but I ran into that blank canvas bug when running hundreds of stories via MDX in docs mode. I'm going to have to figure out how to wait for Solid to re-render before returning (if that is the case).

I improved the workaround for the race condition. My working theory is that it's a scheduling issue, probably with Storybook and the browser, due to assumptions with how many other frameworks render. Instead of using a delay, it's enough to space out render calls by using a semaphore (I noticed that calling timeout with 0 delay was enough to get just one story to render, which means it's likely related to DOM rendering).
Now I tested with rendering ~700 stories with no issue (200 was enough to get the bug). I also noticed the bug showed up in both docs and normal mode, so this change applies to both.

Refactored the CD scripts to use yarn idiomatically.

Ready for review! That took longer than expected (I was really busy with midterms and school projects), but it's finally at a point where I think it's acceptable.
Future work:

Investigate more robust solutions to the race condition (either in Storybook or Solid).
Add features listed in README

@JonahPlusPlus Thanks so much for reviving this!!!! This is an impressive set of changes 🙌
Can we please add:

[ ] A script for building the repo
[ ] A "hello world" Solid project in the monorepo with a storybook, for testing purposes

This might be a good reference project -- another framework developed outside of the monorepo that has a test app & a test lib: https://github.com/literalpie/storybook-framework-qwik

Also instead on using common js, it would be better to use esm in preset.js in packages/frameworks/solid-vite since:

Node has good support for ESM so running it shouldn't be a problem
package.json has "type": "module"
Storybook crashes if using require on node v23.1.0 with ReferenceError: module is not defined

@shilman okay, I completed the requested changes (wish I could have done it faster, but school projects kept getting in my way)
@TheComputerM I wasn't able to get it working. Perhaps there was a fault in my testing (I used a project that worked with the CJS version, but it kept giving errors about not finding the module, so maybe I didn't properly name the files). I think I want to hold off on this for now, just so I can get this update out.

Actually, I suppose I can mess around more with Vite's ESM API.

Okay, the example now uses Vite's ESM API. I switched over to my laptop (I use Arch btw), and it became apparent that I just need to set the project type to "module" for the example. I don't see any need to use ESM for the presets (since the CJS works fine without any warning, but if there is some other reason that's undocumented, I would love to hear it).
Lesson learnt: don't use yarn on Windows. It breaks too often, which makes stuff like this confusing.

TypeScript types seem to be broken in this version. The only contents of index.d.ts are now:
// dev-mode
export * from '../src/index';

but there is no src/ directory in the published package (likely stripped out as a pre-publish step).


Article explaining neural network activation functions added.

Link to  Medium article explaining neural network activation functions by Sagar Sharma added to the document.

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

Hi @menpj
Thank you for your PR but currently, we do not merge PRs that only add to the More Information section.
You can refer to the Contributing Guide here to see why we close certain PRs.


The orbital overlap is still not handled correctly by the AngularMomentum operator

Environment

Qiskit Nature version: main @ fa75ce1f337672e5f4255137efba9aeed3e956ae (also part of 0.7.1)
Python version: Any
Operating system: Any

What is happening?
#1273 unfortunately has not been fixed properly.
This issue resurfaced while I was looking into some more extended tests. Specifically, using the overlap matrix which I obtained for a (6,4) active space of O2:
ovlpab = np.asarray(
    [
        [0.987256, -0.001123, 0.00006, -0.0],
        [-0.001123, -0.987256, -0.0, -0.00006],
        [0.000019, 0.000055, -0.3195, -0.931662],
        [0.000056, -0.000019, -0.931662, 0.3195],
    ],
)

Basically, comparing the computed S^2 values for different occupations obtained from PySCF's FCI module and Qiskit Nature still produce vastly different results.
How can we reproduce the issue?
import numpy as np

from qiskit.primitives import Estimator
from qiskit_algorithms.observables_evaluator import estimate_observables
from qiskit_nature.second_q.circuit.library import HartreeFock
from qiskit_nature.second_q.mappers import JordanWignerMapper
from qiskit_nature.second_q.properties import AngularMomentum, ElectronicDensity

norb = 4
nelec = (4, 2)
ovlpab = np.asarray(
    [
        [0.987256, -0.001123, 0.00006, -0.0],
        [-0.001123, -0.987256, -0.0, -0.00006],
        [0.000019, 0.000055, -0.3195, -0.931662],
        [0.000056, -0.000019, -0.931662, 0.3195],
    ],
)

hf_density = ElectronicDensity.from_particle_number(norb, nelec, include_rdm2=True)
dm1a = hf_density.alpha["+-"]
dm1b = hf_density.beta["+-"]
dm2aa = hf_density.alpha["++--"]
dm2bb = hf_density.beta["++--"]
dm2ab = hf_density.alpha_beta["++--"]

ovlpaa = np.eye(ovlpab.shape[0])
ovlpbb = np.eye(ovlpab.shape[0])
ovlpba = ovlpab.T

# manual computation just like done inside of `pyscf.fci.spin_op`
ssz = (
    np.einsum("ijkl,ij,kl->", dm2aa, ovlpaa, ovlpaa)
    - np.einsum("ijkl,ij,kl->", dm2ab, ovlpaa, ovlpbb)
    + np.einsum("ijkl,ij,kl->", dm2bb, ovlpbb, ovlpbb)
    - np.einsum("ijkl,ij,kl->", dm2ab, ovlpaa, ovlpbb)
    + np.einsum("ji,ij->", dm1a, ovlpaa)
    + np.einsum("ji,ij->", dm1b, ovlpbb)
) * 0.25

dm2abba = -dm2ab.transpose(0, 3, 2, 1)  # alpha^+ beta^+ alpha beta
dm2baab = -dm2ab.transpose(2, 1, 0, 3)  # beta^+ alpha^+ beta alpha
ssxy = (
    np.einsum("ijkl,ij,kl->", dm2abba, ovlpab, ovlpba)
    + np.einsum("ijkl,ij,kl->", dm2baab, ovlpba, ovlpab)
    + np.einsum("ji,ij->", dm1a, ovlpaa)
    + np.einsum("ji,ij->", dm1b, ovlpbb)
) * 0.5
spin_square = ssxy + ssz

s = np.sqrt(spin_square + 0.25) - 0.5
mult = s * 2 + 1
print("FCI", spin_square, mult)

ang_mom = AngularMomentum(norb, ovlpab).second_q_ops()

mapper = JordanWignerMapper()
qubit_op = mapper.map(ang_mom)

hf_state = HartreeFock(norb, nelec, mapper)

result = estimate_observables(Estimator(), hf_state, qubit_op)
new_ss = result["AngularMomentum"][0]
new_s = np.sqrt(new_ss + 0.25) - 0.5
new_mult = new_s * 2 + 1
print("Qiskit", new_ss, new_mult)

This results in:
FCI 2.050648651787 3.0335778557914086
Qiskit 2.0 3.0

What should happen?
Qiskit should produce the same result as the PySCF implementation.
Any suggestions?
Basically, there are multiple problems:

the formula $S^2 = S^- S^+ + S^z (S^z + 1)$ assumes that there are more alpha than beta spin particles. One needs to take the symmetric version to allow either case: $S^2 = 0.5 * (S^- S^+ + S^+ S^-) + S^z S^z$

on top of that, the logic that was added in #1275 is actually flawed. Through mostly trial and error, I found a patch last night which I admittedly did not understand completely but it made all my test cases pass :eyes: Essentially, I ended up overwriting the coefficients of the one-body terms which result from normal-ordering the $S^+ S^-$ (and v.v.) product to all be 1 (instead of the coefficient that resulted from the overlap squared).

Today, I found out why (thanks also to @AlbertoBaiardi for the extensive discussions). The problem is, that $S^+$ and $S^-$ do NOT need to the include the overlap as coefficients! Rather, we need to adapt our anti-commutation relation and weight this with the overlap. This is explained quite nicely in sections 4.1 and 4.2 of this paper of which I will include the relevant screenshots below:

Long story short, to fix this issue we need to:

[ ] use the symmetric formula for $S^2$
[ ] revert the inclusion of the overlap as coefficients inside $S^+$ and $S^-$ (and $S^x$ and $S^y$ by extension)
[ ] integrate the non-unity anti-commutation relation directly into the FermionicOp

It turns out, that there are more things at play here...
The good news
The Qiskit calculation is mostly correct. In fact, all that is needed is to use a symmetric formula for $S^2$ which also allows the number of beta-spin particles to be larger than the alpha ones (rather than vice versa).
The "bad" news
The problem here is rather of conceptual nature. The problem is, that using unrestricted-spin orbitals for the active space, can result in the active alpha- and beta-spin orbitals to span different spaces. This then result in a non-unitary alpha-beta overlap matrix, which in turn actually breaks an assumption on which PySCF's implementation is based (which is where I obtained those numpy.einsum equations from). Fixing it is possible by simply changing the following two lines:
    + np.einsum("ji,ij->", dm1a, ovlpaa)
    + np.einsum("ji,ij->", dm1b, ovlpbb)

to become:
    + np.einsum("ji,ij->", dm1a, ovlpab @ ovlpba)
    + np.einsum("ji,ij->", dm1b, ovlpba @ ovlpab)

This properly accounts for the non-unitary nature of ovlpab. I tested this in a more extensive example in the snippet below.

A code snippet to compute the active and inactive $S^2$ expectation values in different active spaces of $O_2$.
from copy import copy
import numpy as np

from pyscf import gto, scf
from qiskit.primitives import Estimator
from qiskit_algorithms.observables_evaluator import estimate_observables
from qiskit_nature.second_q.circuit.library import HartreeFock
from qiskit_nature.second_q.mappers import JordanWignerMapper, ParityMapper
from qiskit_nature.second_q.properties import AngularMomentum, ElectronicDensity
from qiskit_nature.second_q.properties.s_operators import s_z_operator
from qiskit_nature.second_q.operators import FermionicOp

def diff_operators(total_op, active_op, padding):
    """Takes the difference of two operators by padding the latter to the size of the former."""
    ops = {}
    for name in total_op.keys():
        padded_op = FermionicOp(
            {
                " ".join(
                    f"{term[0]}_{term[1] + padding * (1 + (term[1] // (active_op[name].num_spin_orbitals // 2)))}"
                    for term in terms
                ): coeff
                for terms, coeff in active_op[name].terms()
            },
            num_spin_orbitals=total_op[name].num_spin_orbitals,
        )

        ops[name] = (total_op[name] - padded_op).simplify()

    return ops

def compute_s2_qiskit(norb, nelec, ops_2nd):
    """Computes S^2 on the HF state using Qiskit (Nature)."""
    mapper = JordanWignerMapper()
    qubit_op = mapper.map(ops_2nd)

    hf_state = HartreeFock(norb, nelec, mapper)

    result = estimate_observables(Estimator(), hf_state, qubit_op)
    ss = result["AngularMomentum"][0]
    ssz = result["S_z^2"][0]
    return ss, ssz

def compute_s2_numpy(hf_density, ovlpab, ovlpaa=None, ovlpbb=None):
    """Computes S^2 on the HF density and overlap using numpy."""
    dm1a = hf_density.alpha["+-"]
    dm1b = hf_density.beta["+-"]
    dm2aa = hf_density.alpha["++--"]
    dm2bb = hf_density.beta["++--"]
    dm2ab = hf_density.alpha_beta["++--"]

    if ovlpaa is None:
        ovlpaa = np.eye(ovlpab.shape[0])
    if ovlpbb is None:
        ovlpbb = np.eye(ovlpab.shape[1])

    ovlpba = ovlpab.T

    ssz = (
        np.einsum("ijkl,ij,kl->", dm2aa, ovlpaa, ovlpaa)
        - np.einsum("ijkl,ij,kl->", dm2ab, ovlpaa, ovlpbb)
        + np.einsum("ijkl,ij,kl->", dm2bb, ovlpbb, ovlpbb)
        - np.einsum("ijkl,ij,kl->", dm2ab, ovlpaa, ovlpbb)
        + np.einsum("ji,ij->", dm1a, ovlpaa)
        + np.einsum("ji,ij->", dm1b, ovlpbb)
    ) * 0.25

    dm2abba = -dm2ab.transpose(0, 3, 2, 1)  # alpha^+ beta^+ alpha beta
    dm2baab = -dm2ab.transpose(2, 1, 0, 3)  # beta^+ alpha^+ beta alpha
    ssxy = (
        np.einsum("ijkl,ij,kl->", dm2abba, ovlpab, ovlpba)
        + np.einsum("ijkl,ij,kl->", dm2baab, ovlpba, ovlpab)
        + np.einsum("ji,ij->", dm1a, ovlpab @ ovlpba)  # ovlpaa)
        + np.einsum("ji,ij->", dm1b, ovlpba @ ovlpab)  # ovlpbb)
    ) * 0.5
    ss = ssxy + ssz

    return ss, ssz

# we use the triplet O2 as a test case
m = gto.M(atom="O 0 0 0; O 0 0 1.3", basis="sto3g", spin=2)

# UHF has a little bit of spin contamination for this system:
h = scf.UHF(m).run()
# converged SCF energy = -147.620355111774  <S^2> = 2.0033214  2S+1 = 3.0022135

ref_ss, _ = h.spin_square()

def verify_results(spin_square, source, label):
    """Verifies the result against the reference values."""
    if np.isclose(spin_square, ref_ss):
        print(f"    The {label} <S^2> computed by {source} is correct!")
        return True
    else:
        print(f"    ERROR: The {label} <S^2> computed by {source} is WRONG!")
        print(f"           <S^2> = {spin_square:.5f}  Δ = {ref_ss - spin_square:.5f}")
        return False

mo_a = h.mo_coeff[0]
mo_b = h.mo_coeff[1]
occ_a = h.mo_occ[0]
occ_b = h.mo_occ[1]
ovlp = h.get_ovlp()
ovlpab_tot = mo_a.T @ ovlp @ mo_b
nelec_tot = (int(sum(occ_a)), int(sum(occ_b)))
norb_tot = ovlpab_tot.shape[0]

print("\n===== References ======")

# as a reference, we (re-)compute the total S^2 values
tot_2nd_ops = AngularMomentum(norb_tot, ovlpab_tot).second_q_ops()
tot_s_z = s_z_operator(norb_tot)
tot_2nd_ops["S_z^2"] = tot_s_z @ tot_s_z
tot_ss_qiskit, tot_ssz_qiskit = compute_s2_qiskit(norb_tot, nelec_tot, tot_2nd_ops)
_ = verify_results(tot_ss_qiskit, "Qiskit", "total")

density_tot = ElectronicDensity.from_particle_number(norb_tot, nelec_tot, include_rdm2=True)
tot_ss_numpy, tot_ssz_numpy = compute_s2_numpy(density_tot, ovlpab_tot)
_ = verify_results(tot_ss_numpy, "Numpy", "total")

# we now look towards active spaces and gradually shrink it by removing more and more inner orbitals
for ninact in range(10):

    print(f"\n--- {ninact} Inactive Orbitals ---", flush=True)

    # lists of inactive and active orbital indices
    inact = list(range(ninact))
    act = list(range(ninact, norb_tot))

    # active parameters
    norb_act = len(act)
    nelec_act = (int(sum(occ_a[act])), int(sum(occ_b[act])))
    ovlpab_act = mo_a[:, act].T @ ovlp @ mo_b[:, act]

    # compute the active <S^2> value with Qiskit
    active_2nd_ops = AngularMomentum(norb_act, ovlpab_act).second_q_ops()
    act_s_z = s_z_operator(norb_act)
    active_2nd_ops["S_z^2"] = act_s_z @ act_s_z
    act_ss_qiskit, act_ssz_qiskit = compute_s2_qiskit(norb_act, nelec_act, active_2nd_ops)

    # we can compute the missing <S^2> based on the difference of the total and active operators
    diff_ops = diff_operators(tot_2nd_ops, active_2nd_ops, ninact)
    diff_ss_qiskit, diff_ssz_qiskit = compute_s2_qiskit(norb_tot, nelec_tot, diff_ops)

    # we can now sum the active and inactive to verify against the reference
    sum_ss_qiskit = act_ss_qiskit + diff_ss_qiskit
    _ = verify_results(sum_ss_qiskit, "Qiskit", "active + inactive")

    # now we compute the active <S^2> value with Numpy
    density_act = ElectronicDensity.from_particle_number(norb_act, nelec_act, include_rdm2=True)
    act_ss_numpy, act_ssz_numpy = compute_s2_numpy(density_act, ovlpab_act)

    # to obtain the inactive <S^2> contribution while also including the active+inactive terms, we
    # must remove the active+active sector from the overlap used by Numpy
    ovlpaa_diff = np.eye(norb_tot)
    ovlpaa_diff[ninact:, ninact:] = 0
    ovlpbb_diff = np.eye(norb_tot)
    ovlpbb_diff[ninact:, ninact:] = 0
    ovlpab_diff = copy(ovlpab_tot)
    ovlpab_diff[ninact:, ninact:] = 0

    diff_ss_numpy, diff_ssz_numpy = compute_s2_numpy(density_tot, ovlpab_diff, ovlpaa_diff, ovlpbb_diff)

    sum_ss_numpy = act_ss_numpy + diff_ss_numpy
    correct = verify_results(sum_ss_numpy, "Numpy", "active + inactive")
    if not correct:
        print("\tThe error lies in the <S_z^2> value:")
        print(f"\t  total    Qiskit <S_z^2> = {tot_ssz_qiskit}")
        print(f"\t    active   Qiskit <S_z^2> = {act_ssz_qiskit}")
        print(f"\t    inactive Qiskit <S_z^2> = {diff_ssz_qiskit}")
        print(f"\t  total    Numpy  <S_z^2> = {tot_ssz_numpy}")
        print(f"\t    active   Numpy  <S_z^2> = {act_ssz_numpy}")
        print(f"\t    inactive Numpy  <S_z^2> = {diff_ssz_numpy}")
        print(f"\tThe number of   active electrons is: {nelec_act}")
        nelec_inact = (nelec_tot[0] - nelec_act[0], nelec_tot[1] - nelec_act[1])
        print(f"\tThe number of inactive electrons is: {nelec_inact}")

One final problem remains, which is for the case where both, the inactive and active sector include unpaired electrons. Here, the numpy.einsum code does NOT produce the correct expectation value of $S_z$. It is possible to get the correct inactive value from the difference of the total and active terms, but computing the inactive value directly I have still not figured out yet. However, this is not really a problem because such a choice of active space is not really relevant.

Output of the above snippet
converged SCF energy = -147.620355111774  <S^2> = 2.0033214  2S+1 = 3.0022135

===== References ======
    The total <S^2> computed by Qiskit is correct!
    The total <S^2> computed by Numpy is correct!

--- 0 Inactive Orbitals ---
    The active + inactive <S^2> computed by Qiskit is correct!
    The active + inactive <S^2> computed by Numpy is correct!

--- 1 Inactive Orbitals ---
    The active + inactive <S^2> computed by Qiskit is correct!
    The active + inactive <S^2> computed by Numpy is correct!

--- 2 Inactive Orbitals ---
    The active + inactive <S^2> computed by Qiskit is correct!
    The active + inactive <S^2> computed by Numpy is correct!

--- 3 Inactive Orbitals ---
    The active + inactive <S^2> computed by Qiskit is correct!
    The active + inactive <S^2> computed by Numpy is correct!

--- 4 Inactive Orbitals ---
    The active + inactive <S^2> computed by Qiskit is correct!
    The active + inactive <S^2> computed by Numpy is correct!

--- 5 Inactive Orbitals ---
    The active + inactive <S^2> computed by Qiskit is correct!
    The active + inactive <S^2> computed by Numpy is correct!

--- 6 Inactive Orbitals ---
    The active + inactive <S^2> computed by Qiskit is correct!
    The active + inactive <S^2> computed by Numpy is correct!

--- 7 Inactive Orbitals ---
    The active + inactive <S^2> computed by Qiskit is correct!
    The active + inactive <S^2> computed by Numpy is correct!

--- 8 Inactive Orbitals ---
    The active + inactive <S^2> computed by Qiskit is correct!
    ERROR: The active + inactive <S^2> computed by Numpy is WRONG!
           <S^2> = 1.50332  Δ = 0.50000
	The error lies in the <S_z^2> value:
	  total    Qiskit <S_z^2> = 1.0
	    active   Qiskit <S_z^2> = 0.25
	    inactive Qiskit <S_z^2> = 0.75
	  total    Numpy  <S_z^2> = 1.0
	    active   Numpy  <S_z^2> = 0.25
	    inactive Numpy  <S_z^2> = 0.25
	The number of   active electrons is: (1, 0)
	The number of inactive electrons is: (8, 7)

--- 9 Inactive Orbitals ---
    The active + inactive <S^2> computed by Qiskit is correct!
    The active + inactive <S^2> computed by Numpy is correct!

Finally, this means that the implementation of #1275 was largely correct. This is also good news, because integrating the overlap-matrix into the FermionicOp would not really work as I had originally suggested. A PR to make some final improvements to fix this issue will follow shortly.


Support for PostgreSQL name type

Describe your feature request
Currently querying the usename column on pg_user (select usename from pg_user) fails with:
thread '<unnamed>' panicked at 'not implemented: name', /Users/runner/work/connector-x/connector-x/connectorx/src/sources/postgres/typesystem.rs:100:22

https://github.com/sfu-db/connector-x/blob/main/connectorx/src/sources/postgres/typesystem.rs#L101
That column is using the postgres name type:

https://www.postgresql.org/docs/current/datatype-character.html#DATATYPE-CHARACTER-SPECIAL-TABLE
https://dba.stackexchange.com/questions/217533/what-is-the-data-type-name-in-postgresql

Hi @nhurden I've fixed similar issues and will try to take a stab at yours over the weekend. Cheers

@nhurden I've opened a PR here https://github.com/sfu-db/connector-x/pull/504. Feel free to checkout the feature branch and test it yourself while the PR is reviewed.

@nhurden The PR has been merged, you should be able to use name types relatively soon

Can confirm this is now working in 0.3.2. 👍


Commit Farming

Build capabilities for creators to allocate a percentage of creator tokens to reward maintainers who have commits in master. This will add an incentive model for maintainers to keep developing OSS projects, receive a portion of creator yield, and have voting power in governance models.

That feature is already in place.
By transferring a portion of the Property tokens, The right to earn a portion of the reward is also mapped to the transferee.
@defi-er

Sorry, if I didn't explain the concept correctly. I'm exploring the possibility for creators to transfer creator tokens via "commit farming" when maintainers have commits in master for the OSS project they receive creator tokens for the work they've done.  @Akira-Taniguchi

I see, that's what you mean.
It's a new concept of distribution using GitHub.
@defi-er


Will plugin support be there any time soon?

Hi there! I'd like to know whether or not this project will support plugins any time soon. I'm a maintainer of Impulse and I'm currently implementing plugin support for all the servers I can find. Impulse relies on plugins for reporting back to a swarm instance (basically a component which keeps track of everything), that the server has been started successfully.
An answer would be much appreciated.
Sincerely, ShadowAria.

Hi ShadowAria,
we currently dont have any eta when there will be a Plugin System. If you want I can ping you when we have one :)

Sure! Thank you very much.


az command "--query flag" inconsistency in windows vs linux

Describe the bug
There is a bug in the az cli command when running on Windows.
Accessing the indices in arrays is not allowed in windows..
I see the error message [0] was unexpected at this time.
Related command
az appconfig credential list -g <resource-group> -n <account-name> --query "([?name=='Primary'].connectionString)[0]"
Works in linux

"Endpoint=https://account-name.azconfig.io;Id=;Secret="

Fails in windows

[0] was unexpected at this time.
C:\path>  "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\..\python.exe" -IBm azure.cli appconfig credential list -g  -n  --query ([?name=='Primary'].connectionString)[0]

Errors
[0] was unexpected at this time.
C:\path>  "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\\..\python.exe" -IBm azure.cli appconfig credential list -g <resource-group> -n <account-name> --query ([?name=='Primary'].connectionString)[0]

Issue script & Debug output
[0] was unexpected at this time.
C:\path>  "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\\..\python.exe" -IBm azure.cli appconfig credential list -g <resource-group> -n <account-name> --query ([?name=='Primary'].connectionString)[0]

Expected behavior
Expected the command to work the same way in both linux and windows.
Expected to not fail in windows.
Environment Summary
azure-cli                         2.49.0
core                              2.49.0
telemetry                          1.0.8
Dependencies:
msal                              1.20.0
azure-mgmt-resource               22.0.0
Python location 'C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python.exe'
Extensions directory 'C:\Users\sanallur.azure\cliextensions'
Python (Windows) 3.10.10 (tags/v3.10.10:aad5f6a, Feb  7 2023, 17:05:00) [MSC v.1929 32 bit (Intel)]
Legal docs and information: aka.ms/AzureCliLegal
Your CLI is up-to-date.
Additional context
Originally found at https://github.com/Azure/azure-sdk-for-js/issues/26034

Thank you for opening this issue, we will look into it.


[gstreamer] gstCamera::Capture() -- an error occurred retrieving the next image buffer`

I am getting the following error in "Coding your own Object Detection Program in Python for live camera stream" as I am following this video link:https://www.youtube.com/watch?v=obt60r8ZeB0&list=PL5B692fm6--uQRRDTPsJDp4o0xbzkoyf8&index=13
 [cuda]   cudaGraphicsGLRegisterBuffer(&interop, allocDMA(type), cudaGraphicsRegisterFlagsFromGL(flags)) [cuda]      invalid OpenGL or DirectX context (error 219) (hex 0xDB) [cuda]      /jetson-inference/utils/display/glTexture.cpp:360 [cuda]   cudaGetLastError() [cuda]      invalid OpenGL or DirectX context (error 219) (hex 0xDB) [cuda]      /jetson-inference/utils/cuda/cudaYUV-YV12.cu:119 [cuda]   cudaI420ToRGB(input, (uchar3*)output, width, height) [cuda]      invalid OpenGL or DirectX context (error 219) (hex 0xDB) [cuda]      /jetson-inference/utils/cuda/cudaColorspace.cpp:53 [cuda]   cudaConvertColor(latestYUV, mFormatYUV, nextRGB, format, mOptions->width, mOptions->height) [cuda]      invalid OpenGL or DirectX context (error 219) (hex 0xDB) [cuda]      /jetson-inference/utils/codec/gstBufferManager.cpp:445 [gstreamer] gstBufferManager -- unsupported image format (rgb8) [gstreamer]                     supported formats are: [gstreamer]                        * rgb8 [gstreamer]                        * rgba8 [gstreamer]                        * rgb32f [gstreamer]                        * rgba32f [gstreamer] gstCamera::Capture() -- an error occurred retrieving the next image buffer

@AnilSarode do you have a physical display attached to your Jetson?   If not, the GUI won't work over SSH with X11 forwarding because of the OpenGL/CUDA interop.  Instead, see some of the streaming options from here like RTP/RTSP if you are running headlessly and want to view the video remotely:
https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-streaming.md

@dusty-nv Thanks for your quick response. I am also accessing the Jetson Nano through Physical Displays (i.e. physical display attached) and also by using SSH. I am running a Python script on the terminal of the attached monitor display and not through the SSH terminal.

OK gotcha @AnilSarode - can you run sudo apt-get install mesa-utils and then glxinfo ?   I wonder if somehow you got another OpenGL driver installed at some point.  You can also test glxgears that way.

@dusty-nv This the glxinfo : glxinfo.txt
and glxgears works correctly.

@AnilSarode are you able to run any of the other examples from jetson-inference like imagenet, detectnet, video-viewer, ect?

@dusty-nv Sorry for the late response. any of the other examples from jetson-inference like imagenet, detectnet, video-viewer, etc also give me the same error as above.

@AnilSarode Have you made any progress on this issue? It's quite the same problem i meet.

@GITsologun this typically happens when the NVIDIA OpenGL driver gets overwritten by another graphics driver, or you are attempting to use SSH with X11 forwarding.   If you have another SD card, I would quickly try a fresh install of JetPack to see if the issue re-occurs just to eliminate that possibility.

@GITsologun this typically happens when the NVIDIA OpenGL driver gets overwritten by another graphics driver, or you are attempting to use SSH with X11 forwarding. If you have another SD card, I would quickly try a fresh install of JetPack to see if the issue re-occurs just to eliminate that possibility.

Appreciate for your quick reply. I didn't use SSH or X11. I bought this Jetson Orin NX few days ago, and the only thing I did is following the basic tutorial with the default system.  Maybe you are right that I can try a fresh install of JetPack, however is there any other quick solution?

Hmm okay, since it's already a fresh install, hopefully you needn't do that again.  Do the graphics work if you use the jetson-inference docker container instead?  Or are you already using container?

Hmm okay, since it's already a fresh install, hopefully you needn't do that again. Do the graphics work if you use the jetson-inference docker container instead? Or are you already using container?

Not yet. I am trying to run the docker container by "docker/run.sh", suffering the very slow downloading of "pulling fs layer"...

Hmm okay, since it's already a fresh install, hopefully you needn't do that again. Do the graphics work if you use the jetson-inference docker container instead? Or are you already using container?

I just tested the docker container, and it didn't work. See the attached picture.

Hmm okay, since it's already a fresh install, hopefully you needn't do that again. Do the graphics work if you use the jetson-inference docker container instead? Or are you already using container?

The problem has been solved by fresh install of both Jetson Linux and Jetpack. The experience is that one should be very careful on the screen info, in case of missing any errors.  Thank you for your help, and good luck to the other guys.


Add report command to influx_inspect

Required for all non-trivial PRs

[x] Rebased/mergable
[x] Tests pass
[ ] CHANGELOG.md updated

This adds a report sub-command to influx_inspect to print out series cardinality estimates for a shard directory.  It removes the info command which was never safe to run since it would start the engine and kick off compactions and other mutations to the data dir.  The report command is safe to run while a server is live and is read-only.
Sample output w/ influx_stress
influx_inspect report -detailed ~/.influxdb/data/stress/autogen/4
File			Series	Load Time
000000004-000000003.tsm	100000	10.317458ms
000000005-000000001.tsm	100000	5.898216ms

Statistics
  Series:
    Total (est): 99545
  Measurements (est):
    cpu: 99545 (100%)
  Fields (est):
    cpu: 1
  Tags (est):
    location: 1
    host: 100927
Completed in 358.155511ms

The first section just list the file nam, number series in that file and how long it took to load it.  The Statistics section, which is only shown if run w/ -detailed, shows estimates on the total series, measurements, fields per measurement and values per tag.  They are estimates because getting an exact count could consume a lot of memory.  I've used this command on shards with 500M+ series to help identify high cardinality tags.
The output and stats collected could be improved, but it is useful in it's current state IMO.
The command can also help identify problem TSM files that are affecting startups (e.g. slow to load, fail to load, etc..)

cc @gunnaraasen @beckettsean

@corylanou

👍


feat: create -rgb suffixed variables for all rgb() values

Description
This PR adds -rgb suffixed variables with the comma separated list of individual color values. This enables usage such as:
--spectrum-assetcard-overlay-background-color: rgba(var(--spectrum-global-color-blue-400-rgb), 0.20);

I think this is a working solution, but should we consider an alpha postcss function?

@GarthDB a PostCSS solution would not work with themes where colors change (i.e. current Express and Spectrum) -- at build time, we would not know the value of the color since it could be Express or Spectrum. It would be fine once we unify colors.
I think this is the best workable solution for now, and it's really nice to use native CSS functionality!


Stylistic and Structural Change for Fields and Fieldsets

Issue Description
While looking through forms, UX has requested for two change to be made to Forms.

For fieldsets, they would like the help-text and error message to come before the fields and inputs
For all form elements, they want to reduce the breakpoint for inline wrapping to 544px.

Resolved in #472


Ensure ./ == ./${PREFIX}/index.html when developing locally

Using the unbundled local versions of files should behave the same as
when using the bundled versions.

Can you give an example of something that is broken without this patch? It's not clear to me what this fixes.

It's the local asset version of #2.
I threw together a repo to demonstrate: https://github.com/2sidedfigure/esc-pr5-demo
Clone it, run go generate to create the asset file, and then build and run with the -local flag.
Without the submitted patch, curl -vL localhost:8080 (even curl -vL localhost:8080/index.html) returns a 404. With the submitted patch, it returns the index file, as expected.


Convention for naming unit tests

Actual behavior
The documentation mentions on several places that the unit tests should be named as test_unit_*, but on some other places as unit_test_*. It is not clear which naming is the correction one and which to chose when writing new tests.
To Reproduce
$ git grep unit_test_
docs/source/repository-dir-layout.md:            unit_test_actorname.py              # should contain the actor name
docs/source/test-actors.md:- unit_test_*.py
docs/source/test-actors.md:- [Example of unit tests](https://github.com/oamg/leapp-repository/blob/master/repos/system_upgrade/el7toel8/actors/checkbootavailspace/tests/unit_test_checkbootavailspace.py)
docs/source/unit-testing.md:            unit_test_my_actor.py
docs/source/unit-testing.md:            unit_test_my_actor.py
tests/data/workflow-tests/tags/unittestworkflow.py:    name = 'unit_test_workflow'
$ git grep test_unit_
docs/source/deprecation.md:repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py::test_process PASSED
docs/source/deprecation.md:repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py::test_process
docs/source/deprecation.md:  /tmp/leapp-repository/repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py:10: _DeprecationWarningContext: Usage of deprecated Model "Foo"
docs/source/deprecation.md:repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py::test_process FAILED
docs/source/deprecation.md:repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py::test_process
docs/source/deprecation.md:  /tmp/leapp-repository/repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py:13: _DeprecationWarningContext: Usage of deprecated Model "Foo"
docs/source/deprecation.md:repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py::test_process
docs/source/deprecation.md:  /tmp/leapp-repository/repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py:10: _DeprecationWarningContext: Usage of deprecated Model "BaseFoo"
docs/source/unit-testing.md:`test_*_{actor_name}.py`. For example: `test_unit_sctpconfigread.py` or

Expected behavior
Documentation suggesting only one option.

OS and version: (e.g. Fedora 29 or $ cat /etc/system-release)
master

@Jakuje  thanks for the report. Both cases should be ok, but you are right that documentation should be consistent.


Improve HTTPS certificate generation, plus drive-by's and Makefile comments

Hello friend!
I don't think it's useful to encrypt the https.key file with AES-256 and passphrase, especially because dealing with a passphrase is hard in Camp. I think that by default, make https && SECURE=yes make start fails because Camp feeds the encrypted key directly to Node's TLS module, making it choke.
Also, I bumped the key bits from 1024 to 4096, and forced the CSR digest to be SHA-256 (instead of the default SHA-1 I believe).
Also, I added some comments to the Makefile, and replaced some hard-coded web/ references with $(WEB)/ so that the sc-specific commands still work if you rename your web/ folder.

As a variant for τ-day, we could also use the end of signed 32-bit time: january 19th, 2038.

Thanks!

Thanks for merging!


Introduce @apollo/utils.logger package with generic Logger interface

@glasser let me know what you think about publishing this interface from here. This interface also exists in gateway so I think it makes sense to publish it as a shared util. This expands on our existing interface a bit by allowing n args to the various log functions.

(the readme will also be good to give https://github.com/apollographql/apollo-server/pull/6262 something to link to)


fix(upgrade): add retries for curl downloads

Signed-off-by: Zespre Chang<EMAIL_ADDRESS>IMPORTANT: Please do not create a Pull Request without creating an issue first.
Problem:

Failures might occur around curl commands in detect_repo and many other places in the upgrade scripts during node upgrades due to network issues. This leads to a total failure of the node upgrade Pod because we have set -e in the upgrade scripts.
Solution:

Add retry for file downloading using curl.
Related Issue:
#3079
Test plan:

Prepare a v1.0.3 Harvester cluster with any number of nodes
Kickstart the upgrade to v1.1.0 which contains the changes in this PR
Wait until the upgrade entering phase 4: node upgrade
Copy the output of the upgrade repo VM's Service resource with kubectl -n harvester-system get svc upgrade-repo-hvst-upgrade-lbwg6 -o yaml > upgrade-repo-svc.yaml for example
Remove the Service resource with kubectl -n harvester-system delete svc upgrade-repo-hvst-upgrade-lbwg6 for example
Check the logs of the running post-drain Pod, there should be some curl retrying logs
Resume the upgrade by putting back the Service resource with kubectl apply -f upgrade-repo-svc.yaml
Check the logs of the running post-drain Pod, the retry should end, and the upgrade proceeds

Updated from endless retry to limited retry since we're gonna have job-level retry.


[dev-tool] Running samples in an order

samples publish step generates the readme content for the samples and puts them in an order of their weights assigned in the samples.
Similarly, asking for a new feature for check-node-versions to allow running the samples in a particular order so that a sample that is pre-req to another sample would be executed first/.
We can either have

a samples-setup script so that samples don't have pre-requisites among themselves
or take help from the ordering of the samples to run them

Example:

Service Bus

browseMessages sample is run before sendMessages sample but expects the latter as a pre-requisite.

@deyaaeldeen based on the discussion should we close this issue?

@witemple-msft I am not sure but I am proposing that published samples should be self-contained so that it is easy for customers to copy and paste them. However, there are currently samples that are not, e.g., Service Bus ones listed in the issue description.

I agree that samples should be self-contained.  I logged #20698.  @HarshaNalluru do you know any other samples that have similar issue?

Same with receiving as well.
The session receiving sample does not have the send counterpart.
Better to run the check-node-versions script to see what needs prerequisites.


Remove non-functioning path, domain params from eraseCookies

Resolves #718

This is a breaking change that removes 2 options which may come in handy for particular use cases, and as such I will have to reject this PR.


file aren't uploading when creating a question

after submitting the form, the files aren't uploaded

fix :)


Additional repos from EC mid-year update

@WHYTEWYLL could I get a review here when you have a chance before the run starts next week? 🙏


Crash if no image loaded and CTRL-R is pressed and then collapsed

Titel says it all.
Version 37-18bfc38

I've removed the opportunity to crash, but I'll try to do it even better soon, as it is starting to become messy more that I like.


Authoring guidelines: description

Discussion: #143
Deploy preview: here

Green check from kazk and ggorlen, so merging. If you think something still needs to be fixed let me know or prepare a PR.


Non‐null type field with a default value is required although it should not be

Describe the bug
According to: https://facebook.github.io/graphql/June2018/#sec-Input-Object-Required-Fields
"An input field is required if it has a non‐null type and does not have a default value. Otherwise, the input object field is optional.". I am seeing different behavior. I defined ArticleInputType and configured it with default value using IntValueNode and I still get 'id' is required message.
To Reproduce
Steps to reproduce the behavior:
class ArticleInputType : InputObjectType<Article>
    {
        protected override void Configure(IInputObjectTypeDescriptor<Article> descriptor)
        {
            //base.Configure(descriptor);

            descriptor.Name("Article");

            descriptor.Field(f => f.Id)
                .Type<IntType>()
                .DefaultValue(new IntValueNode(0));
        }
    }

public abstract class MediaItem
    {
        public int Id { get; set; }
        public string Title { get; set; }
        public string Description { get; set; }
        public IEnumerable<Image> Images { get; set; }
    }
    public class Article : MediaItem
    {

    }

class MutationType : ObjectType<Mutation>
    {
        protected override void Configure(IObjectTypeDescriptor<Mutation> descriptor)
        {
            descriptor.Field(f => f.CreateArticle(default(Article)))
                .Type<NonNullType<ArticleType>>()
                .Argument("article", a => a.Type<NonNullType<ArticleInputType>>());
        }
    }

Expected behavior
When i execute:
mutation createArticle{
  createArticle(article: {}){

  }
}

I shouldn't get this:
{
  "errors": [
    {
      "message": "`id` is a required field and cannot be null.",
      "locations": [
        {
          "line": 29,
          "column": 27
        }
      ]
    }
  ]
}

Desktop (please complete the following information):

OS: Windows 10
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Hi,
I will write a little test and check this one out. Give me a view minutes.

No problem. This is my first time using this library and I spend some time on this and I still cannot figure it out.

I'd like to not have property Id: int be required when I create my Articles.

I updated my report with MutationType.cs

Btw.... you can just provide an int here DefaultValue(0); The input value node is more for the schema first world. But that has nothing to do with the error :)

Yeah, that was the first thing I did but after I got the error above I tried multiple things including
DefaultValue(0) and DefaultValue(default(int))

There must be something with your schema setup. Could you provide me with your full schema? I think that the type is somewhere not wrapped by a schema type and then the schema will infer the type. Since int is not nullable it would then become a nonnulltype.

Are you by any chance registering your schema like the following?
Schema.Create(c => c.RegisterMutationType<Mutation>());

Since then. we would not use the schema types and their description. The following would be how to provide us with the schema types:
Schema.Create(c => c.RegisterMutationType<MutationType>());

That's probably it! Let me try.

The first registration is if you want to work with poco types. We will then basically infer everything from your types. You can then annotate your types the attributes to give us more hints.
The second variant is code-first with schema types, which keeps your .net types clean from any attributes.

We also have a slack group. Feel free to join us there for any help:
https://join.slack.com/t/hotchocolategraphql/shared_invite/enQtNTA4NjA0ODYwOTQ0LTBkZjNjZWIzMmNlZjQ5MDQyNDNjMmY3NzYzZjgyYTVmZDU2YjVmNDlhNjNlNTk2ZWRiYzIxMTkwYzA4ODA5Yzg

Does it work?

I am getting an exception when I try to run the application after
registering MutationType cs Mutation. Once I figure that out I'll let you
know. Thank you for helping out!
On Fri, Mar 1, 2019 at 2:46 PM Michael Staib<EMAIL_ADDRESS>wrote:

Does it work?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ChilliCream/hotchocolate/issues/610#issuecomment-468837589,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFtJv9PznfAtSHuvSGhqVb5ZN-hdogsPks5vSa3RgaJpZM4bZ4wZ
.

OK, great! If you need any further help to get it running just let me know.
BTW, if it is a schema exception then you can look up the schema violations in the errors collection of the exception.
There is also a Star Wars example here:
https://hotchocolate.io/docs/example-star-wars-code-first
and some other examples for more advance concepts here:
https://hotchocolate.io/docs/example-other-code-first

After I resolved couple of Schema exceptions I have it working now! I'll
make sure to join Slack group you sent me. Thank you!

I am closing this issue as it's resolved.


Fix #20

The default behaviour of DepTrim is analyzing the dependencies and creating a folder libs-deptrim in the root of the project, which contains all the specialized dependencies.

Oh, okay.

I've renamed the title of this PR to clarify this.

Thank you. It makes it clearer.


Strange behavior when placing a ion-range inside an ion-item

Ionic version:  (check one with "x")
[X] 2.x
I'm submitting a ...  (check one with "x")
[X] bug report
Current behavior:
I get strange behavior when placing an ion-range inside a ion-item, hide almost all the elements inside
Expected behavior:
show all the elements inside the ion-item, included the h2 and p elements
Steps to reproduce:
Plunker example
<ion-item>
  <ion-thumbnail item-left>

  </ion-thumbnail>

Title

Artist • Album

  <button ion-button clear item-right large color="primary" (click)="togglePlay()">
    <ion-icon *ngIf="!playing" name="play"></ion-icon>
    <ion-icon *ngIf="playing"  name="pause"></ion-icon>
  </button>
  <ion-range class="audio-progress"
    [(ngModel)]="audioPos"
    color="primary"
    step="1" max="100"></ion-range>
</ion-item>

Ionic info: (run ionic info from a terminal/cmd prompt and paste output below):
Cordova CLI: 6.5.0
Ionic Framework Version: 2.0.1
Ionic CLI Version: 2.2.1
Ionic App Lib Version: 2.2.0
Ionic App Scripts Version: 1.1.0
ios-deploy version: Not installed
ios-sim version: Not installed
OS: Windows 8.1
Node Version: v6.10.0
Xcode version: Not installed

i think this is expected . ion items can only have one component in them. range, toggle, checkbox vs..
you should place range in a different ion item

@comfortme I see what you say, and it make sense, at the end I used a html5 progress  element and make it looks like material design with css. I just wanted make a simple audio player, but since there is not a progress component in Ionic #6669,  I thought to use a range component was a good idea, my bad
<progress class="progress" max="{{ audioDuration }}" value="{{ audioPos }}"></progress>

Is there any other workaround for this? I really need ion-range with other elems in one ion-item-sliding.

@slzr There will be an ionic progress component soon ;)

I believe you have to place the ion-range inside a


Binding: Collapse authorize and authorizeUser and remove default identity

Once #43 is complete, this functionality may be collapsed into one exposed method from the binding layer and the default identity may then be removed.

Closed by #49


Add support for headers in errors

Resolves #571. I'd like to document this, but I'm not quite sure where.

yeah, hmmm... we need a doc just on error handling

looks good to me.
people who would find interest in this, would you like to review? @niftylettuce @menems @dead-horse @tejasmanohar @cesarandreu @pensierinmusica @felixfbecker @ruimarinho @fixe

hi , I think the ability to add new header on error is a good point , but i am not totally agree about keeping already set header in that way. May be In addition we can have a global var like app.keepHeaders = true that keep all headers on error.
As I said, the problem with that approach is,  the module responsible of keeping header have to know which headers are already set. (it's not always the case) and the goal is remove some boilerplate i think .
But I can't judge that because i haven't make a pull request for that issue (missing time sorry)
So maybe other have another opinion?

@menems Yeah this is a bit hacky. Maybe have err.headers and err.keepHeaders?

I'd rather remove keep header hack like err.headers.kept = true, people can just set err.headers.kept = this.response.get('kept').
Keep this feature simple and obvious.

@dead-horse Sounds good. I'll implement that in a bit. Would you like something to solve menems' problem, that the error thrower might not know which headers to keep? I think that a ctx.keepHeadersOnError object would be a good idea. Maybe a new name though, because that sounds like a boolean.

I also think that a ctx.keepHeadersOnError would be a good idea because it could help when errors are thrown in another library.

not really sure how much should we do in the default error handler, because lots of people already use third party error handlers that don't implement like this may cause some unexpected behavior. so we'd better keep the rules clearly enough and make third party error handlers follow these rules.
ctx.keepHeadersOnError maybe good, but I can't think any use case that we really need this for now.

@dead-horse my use case, and i think it 's a standard one. I have a cors middleware and an authentication behavior, if i use ctx.throw(401). the headers of the response will be reseting and all cors headers too, so the client will scold me :) because the client want the cors headers even if you don't have permission to access a resource. that because on context error function headers is by default reset.
My only solution is to catch the error and set status to bypass the context.onerror.
app.use( (ctx, next) => next.catch(e => ctx.status = e.status))
```
It's a little annoying i think because status will be set by default ctx.onerror handler.

@menems the default error handler already sets the status: https://github.com/koajs/koa/blob/master/lib/context.js#L135

@PlasmaPower Yes I know but it remove headers too https://github.com/koajs/koa/blob/master/lib/context.js#L121

@menems Oh, I thought that you were then throwing the error again after setting the status. Speaking of that, that might actually eliminate the use case for ctx.keepHeadersOnError:
app.use(function*(next){
  try {
    yield next;
  } catch (err) {
    err.headers = err.headers || [];
    err.headers.push('x-my-header');
    throw err;
  }
})

We seem to have a wiki page on error handling, should I make an edit there once this merges to document this?
I'm not quite sure on why we have koajs.com, the github wiki, and /docs/. Should we try to consolidate these?

koajs.com is built from the docs. we should put it in the docs!

hahaha good call

@jonathanong Oh, I see. I thought they weren't the same since a recent change wasn't in there. I'll add a docs page for error handling then.

I've added documentation for this.

@PlasmaPower : ok so if I understand the goal of this pr is to pass header on error and throw it again?
ok no problem i'm going to continu to catch my error like this for keeping headers on response :D
app.use( (ctx, next) => next.catch(e => ctx.status = e.status))

ps: I'm still thinking this middleware should be unnecessary with a global var that keep headers for status != 5XX

@menems this is more aimed at koa libraries I think. They shouldn't implement their own error handlers. In your case, you should probably just make an app.on('error') error handler that does what you want. It will override ctx.onerror.

@PlasmaPower : nop, i tried that before but you can override effectively app.onerror, but not ctx.onerrorbecause ctx.onerrorjust delegate the error to app.onerrorbut continue to remove headers and other stuff. other solutions are to extends Context object and override the ctx.onerror or, context can be like app ans extends eventEmitter and have the same behaviour of application like here: https://github.com/koajs/koa/blob/v2.x/lib/application.js#L120

I see. Yeah since ctx.onerror is basically a try-catch at the start of the chain, then your middleware is practically an error handler. I guess that makes this the recommended way to make an error handler:
app.on(async (ctx, next) => {
  try {
    await next();
  } catch (err) {
    err.status = err.status || 500;
    ctx.body = 'An error occured: ' + err.status;
    ctx.status = err.status;
  }
})

If you do that, aren't you deoptimizing the code?

@thelinuxlich What do you mean?

try/catch deoptimizes V8 code

@thelinuxlich I wasn't aware of that. However, doing a bit of research, it would appear that it will not apply to the next call, so the impact is minimal. Is this the case? Also, if try-catch is to be avoided, you can always just do .catch on a promise:
app.on((ctx, next) => {
  return next().catch(function(err){
    err.status = err.status || 500;
    ctx.body = 'An error occured: ' + err.status;
    ctx.status = err.status;
  })
})

koa-compose@next try-catches middleware anyway, so I'm not sure any of that matters. https://github.com/koajs/compose/blob/next/index.js#L40

hmmm interesting

tried to see if removing the try/catch was faster... it was not. https://github.com/koajs/compose/pull/49

i believe it only deoptimizes that function, anyway

it's actually faster w/ try/catch!

I propose adding 5 more try-catches for optimal performance /s

+1 on more try/catches. Maybe add a finally block aswell?

anyways, this LGTM

I need to fix the docs before merging, it should be done soon. I was a big confused on ctx.onerror and app.onerror.

Okay they should be good now.

LGTM

Small change to reflect current implementation: it('should keep headers specified in the error' -> it('should set headers specified in the error'.


gnutls doesn't clone

Building for iOS and the dependencies cannot progress past this lib
Cleaning source in gnutls
Cloning into 'gnutls'...
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
./build-gnutls.sh: line 19: cd: gnutls: No such file or directory
fatal: ambiguous argument 'gnutls_3_2_4': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
ERROR: Failed to install sources and configure source git

See this is duped in another issue...


refactor: Adopt consistent structure for vars

:tada: This PR is included in version 2.0.0-next-major.11 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add inputs.txt to handle internal file dependencies

This issue is part of an effort to implement substantive improvements to the lab template, as discussed in https://github.com/gentzkow/GentzkowLabTemplate/issues/16.
In this issue, the goal is to work on the way the template handles input files.  The approach to be adopted per the decision in plans for next steps is:

Each module should have a input.txt file in which users should list the files that are required as inputs
make.sh will read through these files and add symlinks to the input folder

I am assigning myself to work on this. I will consider whether the creation of the symlinks should be done in make.sh itself or in an additional script stored in lib/shell. I will also investigate whether there is a way to scan over all scripts and automate the identification of paths.

I have created the inputs.txt files in 276cb0c. There should also be changes in make.sh. To prevent conflicts with the changes in other branches, I will list the changes here, but will wait to push them until #18 and #20 have been closed.
The changes in make.sh should look as follows:
- # Copy and/or symlink input files to local /input/ directory
- # (Make sure this section is updated to pull in all needed input files!)
+ # Add symlink input files to local /input/ directory
+ # (Make sure inputs.txt is updated to pull in all needed input files!)
rm -rf "${MAKE_SCRIPT_DIR}/input"
mkdir -p "${MAKE_SCRIPT_DIR}/input"
- # cp my_source_files "${MAKE_SCRIPT_DIR}/input/"
+ if [[ -f "inputs.txt" ]]; then # check if inputs.txt exists
+ links_created=false
+ while IFS= read -r file_path; do
+ if [[ -n "$file_path" && "$file_path" != \#* ]]; then  # skip empty or commented out lines
+       if [[ -f "$file_path" ]]; then  # check if the file_path is valid
+       file_name=$(basename "$file_path")
+       abs_path=$(realpath "$file_path") # get absolute path
+       ln -sf "$abs_path" "${MAKE_SCRIPT_DIR}/input/$file_name" # create symlink in the input folder
+       links_created=true
+       else
+         echo "Error: $file_path does not exist or is not a valid file path." >&2
+       false # trigger error handler
+       fi
+       fi
+       done < "inputs.txt"
+       if [[ "$links_created" == true ]]; then
+       echo -e "\nAll input links were created!"
+       else
+         echo -e "\n\033[0;34mNote:\033[0m There were no input links to create."
+       fi
+       else
+         echo -e "\nError: No inputs.txt file found in the module." | tee -a "${LOGFILE}"
+       false # trigger error handler
+       fi

I have also done some research regarding whether it would be possible to implement automated scanning of file dependencies. One alternative is using a DAG structure with software like SCons, Snakemake, or Nextflow, but this seems like a more complex step. We could also write ad hoc scripts to do it manually, but it would require some creative combination of regex and searching for functions such as "load" or "read_csv", which does not seem very practical and would increase the complexity of the template without large returns. My sense is that we can stick with manually adding input files for now, either with the inputs.txt approach introduced in this issue, or with the previous approach in which users manually added the files in make.sh with cp my_source_files "${MAKE_SCRIPT_DIR}/input/".

@lucamlouzada Thanks for the great work here.
If we were to go this route, we'd want to abstract the code for scanning inputs.txt into a helper function rather than having it directly in make.sh. It's too clunky, plus it would be a lot of redundancy to have this repeated across directories.
More fundamentally, I'm a little worried looking at it that writing our own parser like this is going to create a potential source of errors/complexity down the line. E.g., cases where people get the syntax in inputs.txt wrong.
Here are two possible alternatives.
(1) link_inputs.sh: Instead of inputs.txt, we have a shell file that contains the bash simlink commands for the various input directories directly.
(2) input_paths.do, input_paths.r, etc.: An even simpler solution would be to have separate scripts by language that define paths that can then be called as inputs by the respective scripts. E.g., input_paths.r might look something like
raw_data <- "../../0_raw/"
clean_data <- "../../1_data/output"

These could be stored in /source/.

Thanks @gentzkow. Agreed on making this a separate helper script if we go down this route.

More fundamentally, I'm a little worried looking at it that writing our own parser like this is going to create a potential source of errors/complexity down the line.

What kind of other errors/complexity did you have in mind? The current approach implements error handling for the paths in the shell script, so that errors in the txt files will be shown to the users as warning messages. But these alternatives are both good options, my sense is it depends a bit on what our priorities will be.

(1) link_inputs.sh: Instead of inputs.txt, we have a shell file that contains the bash simlink commands for the various input directories directly.

By this, do you mean users would edit the shell file instead of the txt file? We could have a section in the shell script for users to edit (similarly to what is done in the current template version with the cp source_files lines) and handle errors within the same script, removing the need for a separate txt file. Is this what you had in mind?  I think it would still be possible to have issues if users get the syntax wrong - and my sense is that users would be more used to editing txt files than shell files.

(2) input_paths.do, input_paths.r, etc.: An even simpler solution would be to have separate scripts by language that define paths that can then be called as inputs by the respective scripts.

This is indeed simpler and could be easy to implement. The downside is having users edit more than one file if inputs for different languages are not concentrated in the same script.

What kind of other errors/complexity did you have in mind?

Just that it's another failure point. I know you've implemented some good error handling, but it's always possible that there's a gap in that or that a user does something unexpected.

This is indeed simpler and could be easy to implement. The downside is having users edit more than one file if inputs for different languages are not concentrated in the same script.

I'm leaning toward this approach. It would remove one big layer of complexity. I think the case where a given module has scripts in multiple languages is relatively rare, and in those cases it's not obviously a bad thing to make clear which inputs are being used by the different categories of scripts. Let's proceed w/ this approach.

Hi @gentzkow, looking back into this now.

input_paths.do, input_paths.r, etc.: An even simpler solution would be to have separate scripts by language that define paths that can then be called as inputs by the respective scripts.

I realized that this approach would require users to add code on top of their scripts to source the paths from the input_paths scripts. For example, users would have to add source("source/input_paths.r") on top of their wrangle_data.r script. I think this is something that we discussed when we were debating the alternative for make_externals.sh and we concluded we did not want users to always have to add something at the beginning of their scripts.
Two potential solutions (let me know if you think of others; I can also keep looking for alternatives):

We have the run_xxx.sh scripts automatically source the input_paths.xx files for each language before running, if they exist.
We have the input_paths.xx scripts create symlinks or copy the files instead of declaring paths as global variables.

The problem with (1) is that it makes the run_xxx scripts a little more complex (and could still lead to issues if users get the syntax wrong). The problem with (2) is that it is basically the same as the previous approach using inputs.txt, only with extra files.
I think (1) is preferable, but still don't know whether it is better thaninputs.txt- it is essentially very similar, with the difference that the complexity is in run_xx instead of in make.sh and users edit different scripts in each language instead of a single inputs.txt file.

Thanks @lucamlouzada. I'm actually not strongly opposed to having those source commands at the top of the scripts. That's actually what I was imagining.
One downside of this approach is that there's no way to tell from within a given r script what global variables are being created. So someone could come upon a command like load(file.path(myfolder, "mydata.rdat")) and not be able to figure out where the myfolder global variable came from. That seems like a fairly small cost, though.
This approach still seems more robust to me than either (1) or (2), since it preserves the property that someone can clone the repo and then run the individual r etc. scripts out of the box.

@gentzkow

This approach still seems more robust to me than either (1) or (2), since it preserves the property that someone can clone the repo and then run the individual r etc. scripts out of the box.

Fair enough. It does not keep the property, however, that the modules are structured with individual input and output folders, right? Essentially with this approach we could remove the input folder because all input paths will be defined in the scripts themselves as global variables (no need to copy or symlink files). While this makes the repo simpler, it removes a layer of transparency.
I still think we should reconsider the inputs.txt approach. I feel like having a chance users will get the syntax wrong is a relatively small cost that can be mitigated with documentation - and any of these approaches is subject to user error (it doesn't really matter if the error is in inputs.txt or in input_paths.r). It preserves all desired properties, especially transparency by keeping an input folder with symlinks to the files.
Does this point make sense? Sorry for this back and forth but I think these are relatively important changes, so I want to make sure we are 100% happy with the option we choose.

OK. I hear you.
What do you think about just having a shell script called get_inputs.sh that creates /input/ rather than inputs.txt. Do you see big advantages to the latter? If not, I think the shell script version would be more transparent to the user.

Thanks @gentzkow !
I think both can work well. The one advantage of inputs.txt I can think of is that the user would have to edit only the clean txt file and not have to edit .sh files, which people may be less familiar with.
But I do think get_inputs.sh creating symlinks would work well. We could have one of these in each module, in which users edit the input paths. Then make.sh can call this script each time the repo is run, and if we do create something like setup.sh it could also call these scripts in each module allowing users to run the modules without running make.sh.
Let me know if you confirm and I'll work on this approach.

Confirmed! This sounds like a good plan.

Great, thanks @gentzkow.
I just implemented this new approach in 2592137. I deleted the inputs.txt file in each module (in 6c3247b) and replaced it with a get_inputs.sh file, in which users should add the file paths for symlinks to be created. This script will be sourced by make.sh, and can also be sourced in a setup.sh script should we really decide to create one.
Again, I have not yet edited make.sh as I want to avoid conflicts with changes in #19. This approach is simpler than the previous one so all that make.sh needs to do is source the new get_inputs.sh script (and this script also recreates the input directory so we can remove these lines from make.sh).
The changes in make.sh should look as follows:
- # Copy and/or symlink input files to local /input/ directory
+ # Add symlink input files to local /input/ directory
- # (Make sure this section is updated to pull in all needed input files!)
+ # (Make sure get_inputs.sh is updated to pull in all needed input files!)
- rm -rf "${MAKE_SCRIPT_DIR}/input"
- mkdir -p "${MAKE_SCRIPT_DIR}/input"
- # cp my_source_files "${MAKE_SCRIPT_DIR}/input/"
+ source "${MAKE_SCRIPT_DIR}/get_inputs.sh"

I have made some updates in this branch following the make.sh changes in #19. First, I updated make.sh per https://github.com/gentzkow/GentzkowLabTemplate/issues/24#issuecomment-2412592632 in 4cb97f0 to reflect the new get_inputs.sh approach.
Importantly, I noticed that since get_inputs.sh can be called by different scripts (i.e. it could be called by the setup.sh script per #26, it could be called directly in each module, or finally by make.sh which itself can be called alone or by run_all.sh), it is important that it is robust to different working directories. To that end I slightly updated the example instructions for relative paths in cf48179.
I also removed a line in make.sh that changed the working directory after calling the scripts, in
4f99c9e. In this way we'll always stick with the same working directory (since make.sh runs scripts in a subshell).

Small update: In 6481b86 I just changed the word "Error" to "Warning" in the case where get_inputs.sh has invalid references.

Summary:
In this issue, we introduced a new approach for handling internal file dependencies. After some discussion in the issue thread, we decided to go with a file called get_inputs.sh , which is present in each module. It is called by make.sh , such that make.sh also had to be updated. Details about these changes and the discussion about them can be found in the issue thread, in particular in https://github.com/gentzkow/GentzkowLabTemplate/issues/24#issuecomment-2412592632 and https://github.com/gentzkow/GentzkowLabTemplate/issues/24#issuecomment-2460550024, as well as in the associated PR #27.
Changes were merged to main in d530d89. The stable issue branch is here.
Thanks @ShiqiYang2022 and @gentzkow for review and feedback!


User Story 3.2 : Notification link is not directing to any page

Given in the user story 3.2 , as an individual if i apply for an org , a notification has to be popped out , no email confirmation will be sent . When i try to click the notification , is that i will be directed to a separate page or link .
https://user-images.githubusercontent.com/51660881/120706193-ca157e00-c486-11eb-8d78-17d3ffd56586.mp4

Issue fixed, Notification link will not be taken to any page and will remain in same org page


Updated the version of Attestation service for Intel SGX API

Signed-off-by: Marikkannu, SureshX<EMAIL_ADDRESS>

@suresh-marikkannu provide a more verbose description in the commit message and update the first comment in this PR

Intel DCAP tracked here https://github.com/omec-project/c3po/issues/22

retest this please

When this will be merged, see https://github.com/omec-project/omec-project-ci/issues/8

retest this please

@somnathchakrabarti  could you please approve this so that we can take care of omec-project/omec-project-ci#8?

resolve #18

@omec-project/c3po-maintainers could you please merge this?

retest this please