Investigate shellhook and makeWrapper for helmsman-aws image

We need a way to export the environment variable HELM_PLUGINS from the helm-diff derivation. The current workaround is setting it in the Nebulaworks docker image.
Investigate using shellhook or makeWrapper to set the environment variable and make it available to the nix profile.

helm-diff was added to upstream nixpkgs: https://github.com/NixOS/nixpkgs/pull/106901
Going to close this now, since its no longer needed.


Add some more logging in the ML pipeline.

Context

Add a tiny bit more logging to ML pipeline to help us debug. I think logging should be on on dev/pre-prod.
Changes proposed in this pull request

Guidance to review

Link to JIRA ticket

N/A
Things to check

[X] I have added any new ENV vars in all deployed environments and updated the .env.example and .env.test files in the repo

Superseded by #236


More configurable health checks for GCLB

To do this we'd pipe a richer datastructure through the Add method: https://github.com/kubernetes/contrib/blob/master/ingress/controllers/gce/backends/backends.go#L140, something like:

type healthCheck {
    path string
    interval time.Duration
}

type backendInfo struct {
  nodePort int
  hc healthCheck
}

And then read those values from annotations here: https://github.com/kubernetes/contrib/blob/master/ingress/controllers/gce/controller/controller.go#L238 (similar method called ListBackendInfo) and pipe it through to https://github.com/kubernetes/contrib/blob/master/ingress/controllers/gce/controller/controller.go#L271.
Essentially if any Ingress specifice ingress.beta.kubernetes.io/health-check-path: "/foo" and/or ingress.beta.kubernetes.io/health-check-interval: "duraiton", we should program the health check appropriately.

I've been thinking through the implications of this change.
Currently health checks are created per backend with the name k8s-be-<node-port>--<cluster-uid>. These are then reused for any Ingress resources which make use of the same backend service, or more specifically node_port.
With this change we'll opening the door to health checks which are unique per Ingress.
Questions

Do we still try and dedup the healthchecks?

If so what naming convention would be best, so would we read all k8s healthchecks and look for a match?

If not do we simply create per ingress health checks which can be named k8s-be-<ing-name>-<be-name>--<cluster-uid>

My personal thoughts are given that backends are likely not going to be reused in multiple ingress resources we move to a per ingress naming scheme.

My personal thoughts are given that backends are likely not going to be reused in multiple ingress resources we move to a per ingress naming scheme.

Resource sharing is required for 2 things:

Backends: because the quota is really low (3 by default), and it's very common to have a single kube Service for, eg: login and only need a single backend for that (so all urlmaps have a /login going to a single backend).
Instance groups: because you can't add a single vm to > 1 loadbalanced instance group and get accurate utilization information.

So the core problem seems to be: when you crate a backend, you need to give it a  health check, but that makes the backend unusable by some other Ingress that has asked for a different health check.

If not do we simply create per ingress health checks which can be named k8s-be----

If we could avoid this it would be best (because of the reason stated above), but it's software, so I'm sure we can make it work somehow if we need to :)

Do we still try and dedup the healthchecks?
If so what naming convention would be best, so would we read all k8s healthchecks and look for a match?

Yeah I'm voting for this. You can still use a single backend in multiple url maps with a different health check. The health checks can be specific to the Ingress name/ns, I don't think we'll need multiple backends to share a health check.
I think this also takes us one step closer to health checking straight to endpoints.


Automated Package Upgrade

:tada: This PR is included in version 7.0.75 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Unable to select scan types in scan configuration ui screen

Description
Unable to select scan types in scan configuration ui screen

Affected Version
main
Steps to Reproduce
Create new scan config
Checklist

[X] I have read the contributing guidelines
[X] I have verified this does not duplicate an existing issue

Fixed as part of https://github.com/openclarity/vmclarity/pull/466


typescript: token function always wants mode as second argument

After the fix here https://github.com/drwpow/cobalt-ui/issues/14 Typescript now complains about a second argument modeName is missing.

Thanks for filing another one! You are now our top bug reporter 🏆
What version of TypeScript are you using? I am having a little trouble reproducing that locally.

I use Typescript 4.8.4
I'm not an TS expert. After some tinkering, this gives me correct typechecking and intellisense (used your apple example):
export declare function token(tokenID: keyof typeof tokens): typeof tokens[typeof tokenID];

export declare function token<T extends keyof typeof modes, K extends keyof typeof modes[T]>(
  tokenID: T,
  modeName: K,
): typeof modes[typeof tokenID];

Thank you! Will give that a try. And I’ll see if there’s a test I can write that catches this. I find typechecking tests like this a little tricky to write, but they’re worth having now.

Maybe you can look into how @mattpocock is doing type "checking" in his courses. eg here: https://github.com/total-typescript/type-transformations-workshop/blob/eaba59ce5d28c3217076c64496e39cc4f6a81b8f/src/04-conditional-types-and-infer/20-type-helpers-pattern.solution.ts#L5
By using these helpers: https://github.com/total-typescript/type-transformations-workshop/blob/main/src/helpers/type-utils.ts

https://github.com/SamVerschueren/tsd
This is also great

Fixed via https://github.com/drwpow/cobalt-ui/commit/c72ea6ffff3e30ec4d21f0b556e0ea218c97180a! 🎉


Golang example on OpenShift 4.3 cannot produce deployments (finalizer RBAC error)

Bug Report

What did you do?
Followed https://sdk.operatorframework.io/docs/golang/quickstart/
What did you expect to see?
Memcached pods appearing
What did you see instead? Under which circumstances?

2020-07-21T08:47:07.870Z        INFO    controllers.Memcached   Creating a new Deployment       {"memcached": "default/memcached-sample", "Deployment.Namespace": "default", "Deployment.Name": "memcached-sample"}
2020-07-21T08:47:07.877Z        ERROR   controllers.Memcached   Failed to create new Deployment {"memcached": "default/memcached-sample", "Deployment.Namespace": "default", "Deployment.Name": "memcached-sample", "error": "deployments.apps \"memcached-sample\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>"}

No new pods. Above error comes from the logs of my operator.

Go
Possible Solution
Missing rbac in role.yaml surely, missing annotations somewhere to allow finalizer permissions?
Here's my generated role.yaml:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  creationTimestamp: null
  name: manager-role
rules:
- apiGroups:
  - apps
  resources:
  - deployments
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - cache.example.com
  resources:
  - memcacheds
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - cache.example.com
  resources:
  - memcacheds/status
  verbs:
  - get
  - patch
  - update
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
  - list

My reconcile method as per the example:
// +kubebuilder:rbac:groups=cache.example.com,resources=memcacheds,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups=cache.example.com,resources=memcacheds/status,verbs=get;update;patch
// +kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups=core,resources=pods,verbs=get;list;

func (r *MemcachedReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {

@a-roberts can you try adding the following marker and running make deploy again?
// +kubebuilder:rbac:groups=apps,resources=deployments/finalizers,verbs=get;create;update;patch;delete

HI @a-roberts,
It shows duplicated of https://github.com/operator-framework/operator-sdk/issues/3590. See my comment in https://github.com/operator-framework/operator-sdk/issues/3590#issuecomment-665645542.
I am closing this one and I'd like to ask for you follow up the  https://github.com/operator-framework/operator-sdk/issues/3590 if possible. However, if you do you think that it should be re-opened for any reason please feel free to ping and let us know.

Reopening after I hit the same error while testing on Openshift 4.5.6:
ERROR   controllers.Memcached   Failed to create new Deployment {"memcached": "default/memcached-sample", "Deployment.Namespace": "default", "Deployment.Name": "memcached-sample", "error": "deployments.apps \"memcached-sample\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>"}

The fix for me was to give the operator's ClusterRole permissions to update the memcached CR's finalizers, with the following marker in the controller:
// +kubebuilder:rbac:groups=cache.example.com,resources=memcacheds/finalizers,verbs=get;update;patch

- apiGroups:
  - cache.example.com
  resources:
  - memcacheds/finalizers
  verbs:
  - get
  - patch
  - update

@estroz @camilamacedo86 @joelanford  Sorry if I missed it but did we discuss why we don't have the above in our docs and sample memcached controller to make the quickstart guide work by default on openshift (or any cluster with OwnerReferencesPermissionsEnforcement set).
https://sdk.operatorframework.io/docs/building-operators/golang/tutorial/#specify-permissions-and-generate-rbac-manifests
I know there's similar discussion for the same bug for Helm operators in https://github.com/operator-framework/operator-sdk/issues/3767#issuecomment-679123829 which I think is for the same reason.

Wouldn't call this a bug but definitely something to fix in the docs to make operators work on openshift by default.

This came up for the Helm operator as well in https://github.com/operator-framework/operator-sdk/issues/3767
What is it about OpenShift that causes this error that doesn't happen in vanilla Kubernetes?
I'm curious if this is an OpenShift-specific API server customization, or if its an extra admission plugin that is not enabled by default in vanilla Kubernetes?
The reason I ask: Should we push to get this permission added to the default scaffold in Kubebuilder?

Found it: https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#ownerreferencespermissionenforcement
Given that this is built into upstream Kubernetes, I think we could make a case for including this in Kubebuilder's scaffolding. WDYT?

Yeah that admission controller seems to be a reasonable default to have in any non-vanilla cluster.
So, not unreasonable to have the kubebuilder controller scaffold have the <kind>/finalizers marker comment as well.
But in the meanwhile we should probably update our docs and sample controller to include a note on this so that our example works on openshift or any non-vanilla cluster.

https://github.com/kubernetes-sigs/kubebuilder/issues/1654

Hello, just ran into it  generating/installing the Go MemcachedStatus operator provided in https://github.com/operator-framework/operator-sdk/tree/master/testdata with.
operator-sdk version operator-sdk version: "v1.1.0", commit: "9d27e224efac78fcc9354ece4e43a50eb30ea968", kubernetes version: "v1.18.2", go version: "go1.15 linux/amd64", GOOS: "linux", GOARCH: "amd64"
and using oc version Client Version: 4.5.17 Server Version: 4.5.17 Kubernetes Version: v1.18.3+45b9524

Hi @fckbo,
The testdata/go/memcached operator has not the finalizer permission required for it works in OCP:

// +kubebuilder:rbac:groups=cache.example.com,resources=memcacheds/finalizers,verbs=get;update;patch

See:
https://github.com/operator-framework/operator-sdk/blob/master/testdata/go/memcached-operator/controllers/memcached_controller.go#L45-L48
We added this permission in upstream https://github.com/kubernetes-sigs/kubebuilder/pull/1688 but for v3+ plugin which means that it will ONLY be available to SDK when this plugin version is supported here whcih is not the current case.
So, you can add the permission and run make manifests to gen the RBAC, build the project and test it in OCP.  However, I think we can mock the permission in the example as well. See; https://github.com/operator-framework/operator-sdk/pull/4162/files.

Hi @camilamacedo86,
thx for your answer, this is what I had done....it worked... and sorry, I actually did not realise that the fix would be included only in a future release. Thx for clarifying.

It is fine and really thank you for your collaboration 🥇


unknown named module: 'react-native-google-signin'

Had this problem when launch react app:
unknown named module: 'react-native-google-signin'
Any idea?

I am also getting this problem. @peterchen16 any solution.

Try the following:

Delete the node_modules folder - rm -rf node_modules && npm install
Reset packager cache - rm -fr $TMPDIR/react-* or node_modules/react-native/packager/packager.sh --reset-cache
Clear watchman watches - watchman watch-del-all


Question: Autoscale plots when legend item is turned off

I cant determine if this is a pyqtgraph function or PGlive.
A graph with multiple plots and a legend.
If you click an item in the legend box its plot is turned off in the graph. (its still collecting data)
What I have is three plots with what can be large variations in Y axis values.
I was hoping to autorange the remaining plots when a plot is turned off from the legend.
For example below. Turn off Watts and have Volts and Amps auto scale for best viewable resolution.

This is issue of pglive and LiveAxisRange. Issue was, that plot was used in autorange calculation when it was hidden.
This should be fixed in version 0.5.4.

Confirmed Fixed!
PGlive gets more exciting all the time.
Thanks for the quick fix :-)


tests.references: use runNixOSTest instead of nixosTest

Description of changes
~As #282401 makes NixOS tests available on MacOS, it is now possible to run NixOS tests on all the platforms OfBorg would test against. Packages tests built with testers.runNixOSTest should now benefit from such cross-platform support.~ Only NixOS test running on MacOS is brought by #282401 so far. See https://github.com/NixOS/nixpkgs/issues/294725#issuecomment-1987281287 for detail.
This PR migrates tsets.references from nixosTest to runNixOSTests and makes it available on all platforms. See also #293891.
See also #273183 for exposing test.references.testScriptBin on non-Linux platforms.

Cc: @roberth
Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[X] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[X] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[X] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

@ofborg build tests.trivial-builders.references

build tests.trivial-builders.references

@ofborg build testers.runNixOSTest-example

I just hard-code the attribute path (tests.trivial-builders.references.testScriptBin). Changing attribute name / attribute path itself is expected to involve manual fixes. Let's keep it simple.

@ofborg build tests.trivial-builders.references

Maybe my suggestion was more complicated than needed.
It looks pretty good now!

Your suggestion was valid, as I initially wanted to make the attribute path relocatable/overridable, and your suggestion was a cleaner approach to achieve it.
Later I found that such desire not only complicate the implementation but also compromise overridabilty of the test case itself, so here it is.

@roberth #273183 proposes a way to forward the interface of this test case onto non-Linux platforms. I turned that PR to draft to wait for #178717.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/1756

Thanks!


Upgrade clap and clap_complete version

The PR contains all the changes required for the cli to work with the latest clap version. I also took the liberty to upgrade clap_complete. Resolves #152
Comment
test_get_start_and_uds_client_command had to be removed because clap::Command no longer implements PartialEq, Eq. If you have any other suggestions, let me know.
Suggestion

clap::error has render().ansi() function now. Enabling it can do this: image1 to image 2
I also noticed here that in some cases because of both print and the Err() return, the error message gets printed twice. Remove the print and only return the Err() here?

Thank you, @24seconds!

[About the suggestions]

Good. Nice!
Good! Thank you for catching this!

I just added both suggestions that I mentioned. Also realized, due to the shell completion approach, it is possible to support all 5 shells that clap_complete support so add all 5 of them. Let me know your thoughts.

[About the comment]
I looked the code to understand the purpose of test_get_start_and_uds_client_command.
This test exists to check that get_start_and_uds_client_command should contain exact subcommands - what get_common_subcommands provides. This is because preventing the discrepancy between what get_main_command provides and what get_start_and_uds_client_command provides when the new subcommand is introduced.

I will be back on weekend 🙇

Let me know what you think of the updated test.


Raster.to_xarray() not masking NaN nodata values

It seems like NaNs are not masked when using Raster.to_xarray(). The Raster object contains the nodata attribute which is set via Raster.to_rio_dataset(), so not sure what the issue is and if that if that is a common issue, or only happens in my case.
I solved this by manually masking NaN values slope_xr.where(slope_xr != slope_xr.rio.nodata).
%matplotlib inline
import pdemtools
import xdem
from geoutils import Raster
from pyproj import CRS
import matplotlib.pyplot as plt

# download some ArcticDEM data
bounds = (-2789094.0, 708249.0, -2789044.0, 708299.0)
dem = pdemtools.load.mosaic(
    dataset='arcticdem',
    resolution=2,
    bounds=bounds,
    version='v4.1'
)

# Xarray to Raster
crs = CRS.from_wkt(dem['spatial_ref'].attrs['crs_wkt'])
epsg = crs.to_epsg()
affine = dem.rio.transform()

raster = Raster.from_array(
    data=dem.data,
    crs=epsg,
    transform=affine,
    nodata=-9999.           # set nodata value
)

# Raster to xDEM
x_dem = xdem.DEM(raster)
print(type(x_dem))
print(x_dem.nodata)

# Compute terrain parameter
slope = xdem.terrain.slope(x_dem)

# Convert back to Xarray
slope_xr = slope.to_xarray()
print(slope_xr.data[0,:3,:3])

This code outputs:

Thanks @geograFie, I also noticed this behaviour yesterday when adding the from_xarray() function to solve #519. Both are now fixed in #521 with more tests, awaiting peer-review before merging and making a new release. :wink:


[zh] fix the beta summary indentation in using-api

https://kubernetes.io/zh/docs/reference/using-api/#api-版本控制 has wrong indentation of beta summary:

Trivial one.
/lgtm
/approve


Failed TLS connection.  Unexpected error before accepting TLS connection by OpenSSL addr="<IP_ADDRESS>" host="<IP_ADDRESS>" port=34680 error_class=OpenSSL::SSL::SSLError error="SSL_accept SYSCALL returned=5 errno=0 state=before SSL initialization"

Describe the bug
I am receiving unexpected error before accepting TLS connection.
-- Using transport encryption for the input TCP plugin.
-- Using letsencrypt certificate.
-- I am using the FQDN to reach the Fluentd server.
To Reproduce

Create a letsencrypt certificate.
Add it to the stores and in the config file.
Send inbound traffic to it.

Expected behavior
Expecting it to not throw any errors
Your Environment
- Fluentd version: 14.4.4
- TD Agent version: td-agent 4.3.0 fluentd 1.14.3
- Operating system: VERSION="20.04.3 LTS (Focal Fossa)"
- Kernel version: 5.11.0-1025-azure

Your Configuration
<source>
  @type tcp
  tag tcp.events # required
  <parse>
    @type json
  </parse>
  port 9111   # optional. 5170 by default
  bind <IP_ADDRESS> # optional. <IP_ADDRESS> by default
  <transport tls>
    cert_path /etc/td-agent/certs/fullchain.pem
    private_key_path /etc/td-agent/certs/privkey.pem
  </transport>
  delimiter "\n" # optional. "\n" (newline) by default
</source>

Your Error Log
2022-01-19 22:16:24 +0000 [warn]: #0 unexpected error before accepting TLS connection by OpenSSL addr="SANITIZED" host="SANITIZED" port=34680 error_class=OpenSSL::SSL::SSLError error="SSL_accept SYSCALL returned=5 errno=0 state=before SSL initialization"

Additional context
No response

Could you describe the detail of the client side?

SSL_accept SYSCALL returned=5 errno=0 state=before SSL initialization

A typical cause of this error is non-TLS client is reached to this port.
See also #2952


Segfaults on Python exit with py 3.5 using petsc4py

Cython 0.27 results in segfaults in PyImport_Cleanup when exiting Python when it is used to compile petsc4py (https://bitbucket.org/petsc/petsc4py) on Python 3.5.3.  In particular, when clearing the dict of modules:
#0  _Py_ForgetReference (op=op@entry=0x7ffff5f85160 <__pyx_moduledef>) at Objects/object.c:1766
#1  0x00000000004a2f6a in _Py_Dealloc (op=0x7ffff5f85160 <__pyx_moduledef>) at Objects/object.c:1794
#2  0x00000000004921a9 in free_keys_object (keys=keys@entry=0xcc7db0) at Objects/dictobject.c:351
#3  0x00000000004976c1 in PyDict_Clear (op=op@entry=0x7ffff7f68448) at Objects/dictobject.c:1388
#4  0x00000000005601ee in PyImport_Cleanup () at Python/import.c:415
#5  0x000000000041ff62 in Py_Finalize () at Python/pylifecycle.c:576
#6  0x0000000000435249 in Py_Main (argc=argc@entry=3, argv=argv@entry=0x904020) at Modules/main.c:788
#7  0x000000000041c799 in main (argc=3, argv=0x7fffffffd938) at ./Programs/python.c:65

And op->_ob_prev is NULL, which presumably should not happen.
git bisect blames 9f2471dba3.
Any idea as to where to start looking properly?

Looks like this is indeed a petsc4py problem (which was indeed playing tricks during import), so no bug here, thanks.

I've disabled PEP-489 support by default for 0.27.1. It's too risky for now.


Add additional null checks to Avalonia.Native.WindowBaseImpl.

What does the pull request do?
In some places in WindowBaseImpl we were checking for a null _native reference, but in other places we were assuming it's non-null. Add checks for a null reference there in all places except the Init method (which I assume it can't be null).
Fixed issues
Fixes #7231

You can test this PR using the following package version. 0.10.999-cibuild0017676-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]


Can't open existing database

I am not able to open an existing database, I keep getting the following error:
AggregateError: All promises were rejected
Creating e new database works fine but I can't reopen it with the same address. Any idea on how to fix this?

You may need to close the db and also explicitly close the underlying leveldb. Can you share a code example?

You may need to close the db and also explicitly close the underlying leveldb. Can you share a code example?

Here's the code sample below:
const Libp2pOptions = {
  services: {
    pubsub: gossipsub({
      // neccessary to run a single peer
      allowPublishToZeroPeers: true
    }),
    identify: identify()
  }
}

class JuvelutionDB {

  orbitdb;
  ipfs;

  async init() {

    try {

      const entropy = bip39.mnemonicToEntropy("seed_phrase_here");
      const seed = Buffer.from(entropy, 'hex');

      OrbitDBIdentityProviderDID.setDIDResolver(KeyDidResolver.getResolver())
      useIdentityProvider(OrbitDBIdentityProviderDID)
      const didProvider = new Ed25519Provider(seed)
      const provider = OrbitDBIdentityProviderDID({ didProvider });

      const libp2p = await createLibp2p({ ...Libp2pOptions })
      this.ipfs = await createHelia({ libp2p })

      const identities = await Identities({ ipfs: this.ipfs });
      const identity = await identities.createIdentity({ provider })

      this.orbitdb = await createOrbitDB({ ipfs: this.ipfs, identities, identity });

      console.log("Done initialising node");

    } catch (error) {
      console.log("ERROR", error);
    }
  }

  async close() {
    await this.ipfs?.stop();
    await this.orbitdb?.stop()
  }

  async open(address) {

    try {
      const db = await this.orbitdb.open(address);
      console.log("DATABASE: ", db);
      console.log(await db.all());
    } catch (error) {
      console.log("Error opening database: ", error)
    }
  }
}

Thanks. Can you provide a full working copy which I can run and that demonstrates the problem?

Thanks. Can you provide a full working copy which I can run and that demonstrates the problem?

import { createHelia } from 'helia'
import { createOrbitDB, useIdentityProvider, Identities, useAccessController } from '@orbitdb/core'
import { gossipsub } from "@chainsafe/libp2p-gossipsub";
import { identify } from "@libp2p/identify";
import { createLibp2p } from 'libp2p'
import * as bip39 from 'bip39'
import OrbitDBIdentityProviderDID from '@orbitdb/identity-provider-did'
import KeyDidResolver from 'key-did-resolver'
import { Ed25519Provider } from 'key-did-provider-ed25519'

const Libp2pOptions = {
  services: {
    pubsub: gossipsub({
      // neccessary to run a single peer
      allowPublishToZeroPeers: true
    }),
    identify: identify()
  }
}

class AppDB {

  orbitdb;
  ipfs;

  async init() {

    try {

      const entropy = bip39.mnemonicToEntropy("enter your seed phrase or generate one");
      const seed = Buffer.from(entropy, 'hex');

      OrbitDBIdentityProviderDID.setDIDResolver(KeyDidResolver.getResolver())
      useIdentityProvider(OrbitDBIdentityProviderDID)
      const didProvider = new Ed25519Provider(seed)
      const provider = OrbitDBIdentityProviderDID({ didProvider });

      const libp2p = await createLibp2p({ ...Libp2pOptions })
      this.ipfs = await createHelia({ libp2p })

      const identities = await Identities({ ipfs: this.ipfs });
      const identity = await identities.createIdentity({ provider })

      useAccessController(CustomAccessController)

      this.orbitdb = await createOrbitDB({ ipfs: this.ipfs, identities, identity });

      console.log("Done initialising node");

    } catch (error) {
      console.log("ERROR", error);
    }
  }

  async close() {
    await this.ipfs?.stop();
    await this.orbitdb?.stop()
  }

  async open(address) {

    try {
      const db = await this.orbitdb.open(address, { type: 'documents' });
      console.log("DATABASE ADDRESS: ", db.address);
      console.log(await db.all());
    } catch (error) {
      console.log("Error opening database: ", error)
    }
  }
}

async function main() {
  try {
    const db = new AppDB();

    // initialise database
    await db.init();

   // open a database
   await db.open("my-db")
  } catch (error) { console.log(error) }
}

main();

Running without a database address works fine, but using the address to open again return the error.
Thanks.

The problem is caused by not having a permanent blockstore in Helia (IPFS).
When you try and open the database by name, it will create it if it doesn't exist. If you try and open a database by address, it will attempt to retrieve the database from IPFS's blockstore. However because your blockstore is transient (I.e. it is only available in memory while your script is running), the blocks that make up your database are no longer available and so the open db times out with an error.
By adding a permanent blockstore, the "open db by address" is able to retrieve the blocks from the underlying IPFS blockstore.
If you add a permanent blockstore, this should solve your problem.
E.g.
Import the level blockstore:
import { LevelBlockstore } from 'blockstore-level'

Initialize it and pass it to Helia:
const blockstore = new LevelBlockstore('./ipfs/blocks')
this.ipfs = await createHelia({ libp2p, blockstore })

Hope this helps.

Thank you very much. Now working 💯


Blank login redirect no response

Bug description
Lol I file this under bug report, but basically have you seen a case where when you hit login it just pops up a window, doesn't return and goes nowhere?
https://gitpod.spicpod.tech/api/login?host=gitlab.com&returnTo=https%3A%2F%2Fgitpod.spicpod.tech%2Fcomplete-auth%3Fmessage%3Dsuccess its trying that but never gets a response, I'm not sure what backend service its trying to hit to debug it and the logs I've glanced at don't show anything obvious.
API is up though cause https://gitpod.spicpod.tech/api/version returns something so its not entirely dead.
 :)
Steps to reproduce
Install Gitpod like a complete cowboy and then wonder why it doesn't work.
Expected behavior
No response
Example repository
No response
Anything else?
No response

Yeah, I've seen this loads. It's a pain in the backside to debug. From memory, it's in the server component.
Usually, the issue is that the GitHub/GitLab creds are incorrectly set up - usually the URL is wrong, but check clientId and clientSecret. Did you add the credentials with a secret, or did you do it manually via the GUI?

If the former, switch to the latter.
If the latter, check you've set it correctly.

.... The GUI??
What's this thing you speak of?
On Thu, 16 Feb 2023, 15:41 Simon Emms, @.***> wrote:

Yeah, I've seen this loads. It's a pain in the backside to debug. From
memory, it's in the server component.
Usually, the issue is that the GitHub/GitLab creds are incorrectly set up

usually the URL is wrong, but check clientId and clientSecret. Did you
add the credentials with a secret, or did you do it manually via the GUI?

If the former, switch to the latter.
If the latter, check you've set it correctly.

—
Reply to this email directly, view it on GitHub
https://github.com/mrsimonemms/gitpod-self-hosted/issues/15#issuecomment-1433290091,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAZI6EHIZRB2RHEJZ7UFJTWXZDDBANCNFSM6AAAAAAU6KBT2E
.
You are receiving this because you authored the thread.Message ID:
@.***>

--
Spicule Limited is registered in England & Wales. Company Number:
09954122. Registered office: First Floor, Telecom House, 125-135 Preston
Road, Brighton, England, BN1 6AF. VAT No. 251478891.
All engagements
are subject to Spicule Terms and Conditions of Business. This email and its
contents are intended solely for the individual to whom it is addressed and
may contain information that is confidential, privileged or otherwise
protected from disclosure, distributing or copying. Any views or opinions
presented in this email are solely those of the author and do not
necessarily represent those of Spicule Limited. The company accepts no
liability for any damage caused by any virus transmitted by this email. If
you have received this message in error, please notify us immediately by
reply email before deleting it from your system. Service of legal notice
cannot be effected on Spicule Limited by email.

The main web dashboard.

Isn't this chicken and egg? how does one login without being able to login via github or gitlab?

oh this localhost:8800 forward thing I found digging around on the website?

Ah, missing settingsUrl that I pulled out of someone elses github ticket and the redirect has sprung into life...

At the very least I'll be able to write you a morons guide to getting started at the end of all of this!

Morons of the world unite!!! 🥕 🏴


429 Too Many Requests: You have reached maximum request limit

Where did you see the issue in our documentation?
https://docs.polygon.technology/cdk/build/quickstart/deploy-stack/
Issue description
how to modify the  maximum request limit?

Hi @wade-liwei can you raise an issue on the kurtosis repo for technical problems please: https://github.com/0xPolygon/kurtosis-cdk

I want to do stress test.

The error you are experiencing seems to indicate that you are reaching a limit on an RPC node.

yea, I have changed the rate limit parameter.
do you stress test documents?
which stage does the project in currently?


[RFC] Should we mandate a common serial baud rate?

Right now the baud rate is as provided by upstream projects.
For most boards it's 115200, except for Rockchip where it would be 1500000. About an order of magnitude more.
The most common default baud rate would be used for all boards.
Benefits

Common instructions for all boards
Assuming 115200, the most common speed out there
Some distros have their getty default to 115200

Drawbacks

Does not match defaults from other projects
Does not match vendor defaults (meh!)

Huh...
TF-A already sets it to 115200

https://github.com/ARM-software/arm-trusted-firmware/blob/8078b5c5a0c2a47710df96412d88df53486e2b29/plat/rockchip/rk3399/rk3399_def.h#L20


uemacs: init at 2014-12-08

Motivation for this change
Low on dependencies and quick to compile.
Things done

[x] Tested using sandboxing
(nix.useChroot on NixOS,
or option build-use-chroot in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] OS X
[x] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

updated and tested.

Thank you

Thanks :)


make_keychain uses wrong wif prefix for testnet

Describe the bug
make_keychain -t generates a WIF key with an "L" prefix which corresponds to mainnet and not testnet.
Instead, a testnet WIF key should begin with a "c".
See https://en.bitcoin.it/wiki/List_of_address_prefixes
$ stx make_keychain -t
{
    "mnemonic": "poet crumble alpha soft boss gadget venture roast dance update awake unhappy skirt crunch gown cake deal surround supreme load crystal traffic maze crumble",
    "keyInfo": {
        "privateKey": "da210b07f0f00edd5dc4b0e862afc61a80f4be360f29a4e0a21dad5f80ccd83e01",
        "address": "ST1ED1MJ1QC18W53N710BBK1VBHK43Y2KMRHKEPA7",
        "btcAddress": "moyicrwHUVdc5Mg9roBiMH6GUaw4okhJPX",
        "wif": "L4Xj2A5CoGhdN32VW5nbwZeCP7dAAGUjxviycq3BLxxZTdjcLRBk",
        "index": 0
    }
}

What version of Stacks.js are you using?
$ npm list -g
+--<EMAIL_ADDRESS>
Is the bug present in the Stacks.js CLI, Gaia hub, Connect or Blockstack Browser?
Stacks.js CLI
To Reproduce
Steps to reproduce the behavior:
$ stx make_keychain -t
Expected behavior
Expect to generate a testnet WIF key with a "c" prefix.
Screenshots
If applicable, add screenshots to help explain your problem.
Console log
If applicable, add console log to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Good catch 🙏 Thanks for the report — we'll get right on that!

In addition, I was not able to import the generated testnet private key, WIF or mnemonic into testnet Electrum so the private key generation may have issues as well.


Pgcopy stream metrics

In order to measure the behavior of the copy endpoints (following https://github.com/CartoDB/cartodb-platform/issues/4414)
Things that are interesting to be measured:

query time,
size of payload,
format (text, csv, or binary)

Reviewers, take into account ---> https://github.com/CartoDB/cartodb-platform/issues/4414#issuecomment-390932252


Save Editor: Time Controls can cause Gameovers

[✓] I have checked the known issues and searched Github to ensure this issue has not already been reported.
[✓] This is not a feature request, I know those belong on Discord/Github Discussions.

Description of the bug:
Rewinding time past the dividing mark (6am/6pm) will progress to the next. Which can mean that if you're on the evening of the final day, such as say 6:10pm, and press the -15 minute or -1 hour buttons or even the slider, and the resulting time would be under 6PM it will go straight to the moon crashing.
Can you reproduce? If so please list the steps:

[✓] Yes, here are the steps to reproduce the bug:

Open Save Editor

Set it to the third day

Set time to be after 6pm/18:00

using the slider or the buttons rewind time before 6pm/18:00

Moon Fall

Expected behavior:
Rewinding to the previous time segment. IE: 1st Day Nighttime to 1st Day Daytime. Or 2nd Day to 1st night. Or 3rd night to 3rd day.
Screenshots or videos:
Via Button:
https://github.com/HarbourMasters/2ship2harkinian/assets/18604504/7eb6703b-d364-4ae7-9dd7-f58b1bc32c10
Via Slider:
https://github.com/HarbourMasters/2ship2harkinian/assets/18604504/c930e1d2-c285-45af-b140-2f8c5d4ec894
Going from day one to game over just by sliding back and forth over the 6AM diving line, in about 30 seconds:
https://github.com/HarbourMasters/2ship2harkinian/assets/18604504/c67856c2-9686-4b97-9021-808fd65c4931
System Information:

OS: Windows 10 64bit
GFX Backend: DirectX & OpenGL (Tested both by coincidence) *
Game Version: 1.0.0
Rom Version: USA 1.0

Additional Information:
* The coincidence was that I ran into it originally on DirectX last night, and the recordings above were done in OpenGL. And the only reason I switched was to check performance differences between the two.
Was slightly annoyed when I ran into it by accident, it was the cycle after getting link out of the deku form, and when I wanted to double check I didn't miss an event I thought to rewind it. Instead I got moon fall followed by having to sit through Happy Mask reteaching the song of healing a second time :P

My best guess is that since this is a developer tool this kind of glitch is expected. One option would be to decrement the day by 1 instead of time and then increase time to the time you wanted.
I messed around with this same behavior in the Recomp project and had the same results, it's probably due to the fact that 06:00 and 18:00 both increment the stage of day (Day 1 daytime, Day 1 Nighttime, Day 2 Daytime, etc.).
If you want a feature for user-friendly rewinding of time, I'd consider adding it to discussions. I'm not sure if they want to remove this behavior from the developer tool or not since it is the exact behavior you'd expect from moving the time past those 2 values...

Fix is available in Rika Charlie 1.0.2


ModuleConcatenationPlugin transform re-export module to WEBPACK_IMPORTED_MODULE_0_lodash[true]

Do you want to request a feature or report a bug?
Maybe a bug, some behavior I don't understand

What is the current behavior?
Transformed with undefined error.
If the current behavior is a bug, please provide the steps to reproduce.

When I re-export variable from all of some module, And use like below
// ./module.js
import * as lodash from "lodash";
export { lodash };

// ./main.js
import { lodash } from "./module"

// webpack will transform here to `__WEBPACK_IMPORTED_MODULE_0_lodash__[true].keys({})`
lodash.keys({})

I don't understand why __WEBPACK_IMPORTED_MODULE_0_lodash__[true]
When I change ./module.js to
// ./module.js
import * as l from "lodash";
export const lodash = l;

it works again.
What is the expected behavior?
first codes of ./module should be transformed well.
If this is a feature request, what is motivation or use case for changing the behavior?
Please mention other relevant information such as the browser version, Node.js version, webpack version and Operating System.

<EMAIL_ADDRESS><EMAIL_ADDRESS>

Seem to be an issue with exporting namespace objects.

I attempted to reproduce this myself locally in multiple varieties on latest and I was unable to do so.
Is this version of lodash technically a CJS package (regular lodash vs lodash-es)?
Also, @morlay are you using any plugins such as babel-plugin-lodash or lodash-webpack-plugin (or both?). Could you please provide a minimal reproducible repo for this issue? If so I could probably tackle this myself too.

haven't use plugin.
codes with lodash here just show the errors, other CJS package have same issue, like React, moment.
Repo is here https://github.com/morlay/webpack-issue-5443
the error result https://github.com/morlay/webpack-issue-5443/blob/master/built/app.js#L81
Without ts-loader will got same result.

Add this:
if(exportName === true) return importedVar; // namespace object

before this line: https://github.com/webpack/webpack/blob/master/lib/optimize/ConcatenatedModule.js#L44
Because exportName is set to true here: https://github.com/webpack/webpack/blob/master/lib/optimize/ConcatenatedModule.js#L233-L236

And send a PR

And write a test

I'll do it myself.

Hahaha! Thanks @sokra
Just returning to this. @morlay if you'd like to verify you are welcome to test against the local branch.

@TheLarkInn Cool, It is fixed !
https://github.com/morlay/webpack-issue-5443/blob/master/__built__/app.js#L83
Thanks @sokra  ~


Add Binder support for example notebooks

Here's a version of the Binder button that will work for this PR (the one in README.md points to main):

Nice, this looks like a much cleaner way to do things. It seems like Binder has enough support that it's an okay medium-term solution (i.e., likely to stay free)?

Yes, I hope it will stay that way...


Scan mode not working in panel view, while working in preview

Checklist:

[x ] I updated to the latest version available
[ x] I cleared the cache of my browser

Release with the issue:
latest
Last working release (if known):
not known
Browser and Operating System:
Safari/Edge, Windows/IOS

Description of problem:
HASS panel view with one big panel on the screen: Scan mode does not switch between cameras
Edit the panel, edit the card, do nothing watching preview: cameras are switching ok on the preview

Frigate card diagnostic information

{
  "ha_version": "2024.5.5",
  "card_version": "5.2.0",
  "browser": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36 Edg/<IP_ADDRESS>",
  "date": "2024-06-15T12:41:28.813Z",
  "frigate_version": {
    "bd6fcb1a93ed9f2702999df48d3984d4": "5.2.0/0.13.2-6476f8a"
  },
  "lang": "en",
  "timezone": "Europe/Moscow",
  "git": {
    "build_version": "5.2.0-HEAD+g69249b6",
    "build_date": "Fri, 23 Jun 2023 15:26:26 GMT",
    "commit_date": "Thu, 22 Jun 2023 09:21:26 -0600"
  },
  "config": {
    "type": "custom:frigate-card",
    "cameras": [
      {
        "camera_entity": "camera.gates",
        "live_provider": "go2rtc",
        "go2rtc": {
          "modes": [
            "webrtc"
          ],
          "stream": "gates_hq"
        },
        "dependencies": {
          "all_cameras": false
        }
      },
      {
        "camera_entity": "camera.door",
        "live_provider": "go2rtc",
        "go2rtc": {
          "modes": [
            "webrtc"
          ],
          "stream": "door_hq"
        }
      },
      {
        "camera_entity": "camera.dvor_gates",
        "live_provider": "go2rtc",
        "go2rtc": {
          "modes": [
            "webrtc"
          ],
          "stream": "dvor_gates_hq"
        }
      },
      {
        "camera_entity": "camera.dom2",
        "live_provider": "go2rtc",
        "go2rtc": {
          "modes": [
            "webrtc"
          ],
          "stream": "dom2_hq"
        }
      }
    ],
    "dimensions": {
      "aspect_ratio_mode": "dynamic"
    },
    "media_viewer": {
      "layout": {
        "position": {
          "x": 65
        }
      }
    },
    "view": {
      "default": "live",
      "camera_select": "live",
      "dark_mode": "auto",
      "scan": {
        "enabled": true,
        "untrigger_seconds": 3
      }
    },
    "menu": {
      "position": "right"
    },
    "performance": {
      "profile": "high"
    },
    "live": {
      "transition_effect": "slide"
    }
  }
}

Javascript errors shown in the web inspector (if applicable):

Additional information:
Somehow, once, i made it working after hard refreshing the page, but after switching pages it stopped working again.

[Issue cleanup] Please update to the latest build and let me know if you still have this issue:
https://github.com/dermotduffy/frigate-hass-card/releases/tag/v6.0.0-beta.1

Please re-open if still an issue with new builds. Thanks and good luck!


Build for AWS

Hello there.
Do you know someone that uses AWS with Agora? either with linux or windows.
All our testing with the different plugins/templates you have is to solve the server crashing problem which is: SERVER_PROCESS_CRASHED
"Server process exited without calling ProcessEnding(), exitCode(-1073741515),...
The build deploy could run in AWS just enabling the plugin, but it failed when we implemented your Agora classes as a module (we didn't even migrate the template blueprints in this step).
But enabling the plugin without using it may be the reason why is not failing.
Our current solution is to try to disable Agora modules and everything when compiling the server. But we hope that you can guide us or give us a reference.
Thank you for your help!.

你是aws 的windows服务器吗？我们没有在服务器上尝试部署过。我们sdk目前只支持windows，不支持linux。不过我认为在windows上部署不会用问题，因为windows机器本地是能够运行的。

And if the windows server crashes, you can grab a windows dump and send it to us. My email is<EMAIL_ADDRESS>you can send the dump to us, and we will pay attention to this issue.


Right click action

#191
Here is an example implementation to solve #191

[x] Add support for middle click
[x] Add some default right click actions

[x] Wifi menu on right click
[x] Spotify next track
[x] Remap space hover action to right click

[x] Add support for user widgets

There was a mix up with the commits because we worked in parallel. You can just squash them when merging

I thought about your middle click note.
I think we should allow middle click options for user widgets for sure, and also consider supporting optional middle click options for default widgets, but not set them by default, since most users won't have access to them

I merged everything in master! The middle click action can also be triggered with cmd+click. Let me know if I missed anything.

I merged everything in master! The middle click action can also be triggered with cmd+click. Let me know if I missed anything.

Wow that's a great idea! really like it

Just one small thing, I don't mind it, but it's customary to merge PRs or work from them, so that people who work on them will be credited appropriately.
Again, I'm cool with it and you added a lot in this case, but it would be nice if next time you could either start the work from the PR as a base, so other contributors will get the appropriate credit (it appears both on the user profile and in the repo history)
I know from my experience managing open source projects, some people care about it more than others, but it's a good habit to have :)

Indeed! This time I squashed all the commits so Github isn't automatically recognising this pull request. I usually do not proceed that way (you can see there is a lot of contributors), I'm sorry I was a little too fast.
Anyway I'm always making sure to give credits at the end of the readme each time I merge a pull request (by squashing or not).
I'll do better next time! :)

And of course thank you for your work, I'm glad other people always bring some fresh ideas :)


adds ghost ipfs image

This adds an IPFS compatible image, for deploy on Akash.

Need to update the README.md in the source of the repo!

@88plug ghost is listed on the main README.md, and then this PR adds the third image to the end of the Ghost README.md , is there a different expectation? Thanks!


Automator: update common-files@master in istio/istio@master

Generated by Automator - 2023-08-10T20:18:37+00:00

@istio-testing: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-pilot-multicluster_istio
21fda905e0ec0a17449db54385bb21b4b87b9c8d
link
true
/test integ-pilot-multicluster

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-pilot-multicluster

@istio-testing: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-pilot-multicluster_istio
21fda905e0ec0a17449db54385bb21b4b87b9c8d
link
true
/test integ-pilot-multicluster

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-pilot-multicluster


EDM-984: CI for e2e tests with RH images

Summary by CodeRabbit

CI/CD Improvements

Added a new GitHub Actions workflow for daily automated end-to-end testing
Implemented comprehensive testing process with log collection and artifact upload
Added support for manual workflow triggering

Infrastructure

Enhanced testing environment with Docker authentication and Kubernetes cluster setup
Automated image building and deployment for testing purposes

Move to konflux instead


Shake mouse to find it could be better

Description of the new feature / enhancement
Allow transparent for both colors (I only need a circle around the mouse pointer, don't dim the entire screen).
Remove the circle after 100ms (setting) - it's only needed for a very short time, now it stays on to long.
Scenario when this would be used?
Everyday work on multiple monitors.
Supporting information
No response

This is multiple asks.  I think both others are tracked
Shake has been present for a while.


prometheus: generate config deterministically

Because maps have a non-deterministic order when newly instantiated, we need to use means to make sure that the prometheus yaml configuration we generate is deterministic.
This changes any use of a map in the config to use the yaml.MapSlice, which ensures deterministic map behavior, because the yaml.MapItems are sorted by their keys before marshaling to yaml.
Note that this does rely on Kubernetes not changing the order or arrays in objects, as the Alertmanagers and Endpoints lists are reflected 1 to 1 in the generated config. If this does turn out to be a problem we'll have to find a way to sort those items as well, but that will likely require individual sorting for each of the lists.
Let me know if you want to keep the test that I wrote to make sure that it is deterministic or not.
@fabxc @mxinden

LGTM in general.
In hindsight, one could have probably written a recursive function transforming our previous map[string]interface{} nestings to the equivalent yaml.MapSlice :) but now that it's all converted, that's fine too.

👍 For keeping the tests! Especially to ensure this when upgrading k8s versions.


Update rpc-nodes.txt

@lightiv can you give us some more information around the node itself?

This node specs are:
VPS Node
2 CPU
8 MB RAM
320 GB Hard Drive
Note I updated the url:
http://akash-sentry01.skynetvalidators.com:26657

@lightiv 8GB ram I'm guessing?

haven't heard back, feel free to reopen this if it's still a goal paul.

Hi, I did not respond because I though it was a give that I meant 8 GB instead of 8 MBs.  I don't have the re-open button so I will resubmit a new Pull request.


Undefined values ???

The library encode undefined as value 2 for BOOLNULL tag while nothing defined in spec.
Note that seems a deviation from JSON compatibility and is not very portable as differentiation between undefined and null does not exist in most language.

I agree it's weird, and probably not very necessary in software that currently uses BIPF.

I made change proposal on the spec https://github.com/ssbc/bipf-spec/pull/3.  In practice, the side effect is that "undefined" is compliant...

Any plan to resolve ? There must be reason that you introduced that. In Manyverse ? In ssb stack components ? If I can help fixing it, just tell me. And drive me towards the component that depends on it and I can propose alternative to avoid it

If I remember correctly (and I don't always do) it was not my choice to included undefined. BIPF was originally written by Dominic Tarr and there's few modifications that arj and I made in its core logic.

https://github.com/ssbc/bipf/commit/fc226e8fc9536af243adf5c97d606239e0c79dac

Yes I found it too...  Will be hard determine what it can impact to realign to the spec...
Note, in parallel, I found that in a base stack setup with ssb-db2 , there are some message with such path having a undefined value...   Maybe I will be able to find out where come from by tracing back.

Yeah I bet there are malformed SSB messages with undefined in them. Not by design, probably a mistake. But we still don't want the application to crash upon reading those. At least coercing them to null.


fix: DAAL_EXPORT macros for vc compiler(rls branch)

The changes are the same as in PR #2910 which is already merged to main.

please, don't merge until I solve the issue with nightly gh actions build


Problems encountered in building/deploying a release web-app

Problem
According to the documentation, you just need to run trunk build --release ( included  index.html with basic content due to Error: error getting canonical path to source HTML file "index.html") but it did not render the app that I made.
Tried using dioxus-cli and removed the index.html. Works fine but how do I modify the index.html before build?
Is there an updated guide on how to deploy/build a released version, should we use trunk or should we use dioxus-cli?
If via trunk:

why doesn't it render the application I made? (served it via caddy and shows only white/blank page)

If via dioxus-cli:

how do I modify the information in the generated index.html. If I place a basic index.html in the root directory, it just copies it in the dist without modification regarding the release build assets.

Steps To Reproduce
Steps to reproduce the behavior:

place index.html with basic content
trunk -v build --release

or

place index.html with basic content
dioxus build

Expected behavior

via trunk: render the app
via dioxus-cli: modify the index.html upon build, render the app

Environment:

Dioxus version: 0.3.2
Rust version: 1.68.1
OS info: win11
App platform: web

Questionnaire

[ ] I'm interested in fixing this myself but don't know where to start
[ ] I would like to fix and I have a solution
[x] I don't have time to fix this right now, but maybe later

For trunk and the Dioxus CLI you need to include a div with an id of "main" for Dioxus to render into (example). Trunk also has a great guide that provides information about configuration options here.
For the Dixous CLI, you currently also need to include some template boilerplate to include the WASM and Javascript files in the output (example).

Thank you for the fast reply. Closing this issue.
Note: It might be better to add this to the current documentation so that newbies like me won't be surprised by these additional steps, whether using trunk or dioxus-cli.


Separate fft examples

Split of the FFT examples into c2c and r2c. Tested on gnu/mpich, intel and nvhpc with all working. Only annoyance is the following warning from Intel
mpiifort  -fpp -std08 -O3 -mavx2 -march=core-avx2 -mtune=core-avx2 -module mod -Imod  -DDOUBLE_PREC -DVERSION=\"e8f2cc6\"  -c src/log.f90 -o obj/log.o

src/log.f90(85): warning #7416: Fortran 2008 does not allow this intrinsic procedure.   [COMPILER_VERSION]
    write (io_unit, *) 'Compiled with ', compiler_version()
-----------------------------------------^

src/log.f90(86): warning #7416: Fortran 2008 does not allow this intrinsic procedure.   [COMPILER_OPTIONS]
    write (io_unit, *) 'Compiler options : ', compiler_options()
----------------------------------------------^

mpiifort  -fpp -std08 -O3 -mavx2 -march=core-avx2 -mtune=core-avx2 -module mod -Imod  -DDOUBLE_PREC -DVERSION=\"e8f2cc6\"  -c src/io.f90 -o obj/io.o

src/io.f90(1411): warning #7346: The CHARACTER* form of a CHARACTER declaration is an obsolescent feature in Fortran 2008.
    character*(*), intent(in) :: varname
---------------^

Hello,
The last warning :

src/io.f90(1411): warning #7346: The CHARACTER* form of a CHARACTER declaration is an obsolescent feature in Fortran 2008.
    character*(*), intent(in) :: varname```

this is probably just a typo @pbartholomew08 ?

The other warnings

mpiifort  -fpp -std08 -O3 -mavx2 -march=core-avx2 -mtune=core-avx2 -module mod -Imod  -DDOUBLE_PREC -DVERSION="e8f2cc6"  -c src/log.f90 -o obj/log.o
src/log.f90(85): warning #7416: Fortran 2008 does not allow this intrinsic procedure.   [COMPILER_VERSION]
write (io_unit, *) 'Compiled with ', compiler_version()
-----------------------------------------^
src/log.f90(86): warning #7416: Fortran 2008 does not allow this intrinsic procedure.   [COMPILER_OPTIONS]
write (io_unit, *) 'Compiler options : ', compiler_options()
----------------------------------------------^```
Maybe we just need to replace use iso_fortran_env, only : output_unit, compiler_version, compiler_options with use iso_fortran_env ?

The last warning :

src/io.f90(1411): warning #7346: The CHARACTER* form of a CHARACTER declaration is an obsolescent feature in Fortran 2008.
    character*(*), intent(in) :: varname

this is probably just a typo @pbartholomew08 ?

Not a typo - it was the kind ADIOS2 wanted, it's just an old-style of Fortran, will make a note to check whether we can change to something else, but I think leave as is for now

and the warnings on the iso_fortran_env functions really don't make sense: https://www.intel.com/content/www/us/en/develop/documentation/fortran-compiler-oneapi-dev-guide-and-reference/top/language-reference/program-units-and-procedures/intrinsic-modules/iso-fortran-env-module/intrinsic-module-procedures-iso-fortran-env.html


feat: Add ContextRelevanceEvaluator to RAG eval harness

Proposed Changes:

This PR adds support for using the ContextRelevanceEvaluator component with the RAG evaluation harness.
How did you test it?

Unit tests
Checklist

I have read the contributors guidelines and the code of conduct
I have updated the related issue with new insights and changes
I added unit tests and updated the docstrings
I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
I documented my code
I ran pre-commit hooks and fixed any issue

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

98 of 99 (98.99%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.2%) to 97.696%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

test/evaluation/harness/rag/test_harness.py
95
96
98.96%

Totals

Change from base Build<PHONE_NUMBER>:
0.2%

Covered Lines:
721

Relevant Lines:
738

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

104 of 105 (99.05%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.7%) to 98.118%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

test/evaluation/harness/rag/test_harness.py
101
102
99.02%

Totals

Change from base Build<PHONE_NUMBER>:
0.7%

Covered Lines:
730

Relevant Lines:
744

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

99 of 99 (100.0%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.8%) to 98.238%

Totals

Change from base Build<PHONE_NUMBER>:
0.8%

Covered Lines:
725

Relevant Lines:
738

💛  - Coveralls


Why is this listed under properties?

This is not really a property in traditional sense, is it? Usually properties occur in {} sections and have the form name = value; This is not the case here, so it looks strange that this is described as a property.

Dokumentoplysninger
⚠ Undlad at redigere denne sektion. Det er påkrævet til docs.microsoft.com ➟ GitHub-problemlinkning.

ID: bb594c36-2d2d-fb42-491b-1cc626354ce5
Version Independent ID: 3d48588d-cb49-4bed-d066-0bf44a5d5ecb
Content: Field No. Property - Business Central
Content Source: dev-itpro/developer/properties/devenv-field-no.-property.md
Service: dynamics365-business-central
GitHub Login: @SusanneWindfeldPedersen
Microsoft Alias: solsen

Hello, thanks for your feedback. And you are absolutely right, it is not a property. In order to track this issue, can you specify how you found the topic? Thanks!

We were checking property parsing completeness for our app, Prism for AL (https://stati-cal.com), and therefore reviewed all properties documented in GitHub: https://github.com/MicrosoftDocs/dynamics365smb-devitpro-pb/blob/main/dev-itpro/developer/properties/devenv-field-no.-property.md.
We then found the published documentation page with a google search: business central field-no. property

Thanks so much for the quick response! We will fix this with the next doc refresh.


Disjoint Fields

Hello and thank you for your excellent crate, I have been reading through it for a while before I start a project and I wanted to get your opinion about how to best tackle an issue in my project.
I am planning on writing some crates to support Garmin's ANT spec and most of their data is LE and contiguous fields... except one legacy profile which has the following layout.

Bit
0-7
8-11
12-15
16-23

Field
LSB Field A
LSN Field B
MSN Field A
MSB Field B

Does your library have any mechanism to resolve this legacy issue or do you know if there are any tricks I can employ to get around this?
P.S. I am new to rust, and am planning to use this project to get better with the language. Also I am avoiding building anything too big as the plan is to support embedded targets.

I think my best bet is to use private fields and just write the getters and setters manually.


capture err message when vim exiting

Is your feature request about something that is currently impossible or hard to do? Please describe the problem.
i am trying to capture err message when vim exiting for debug purpose
Describe the solution you'd like
what i do currently is:
au vimleave * call writefile(split(execute('mess'), '\n')

Describe alternatives you've considered
but looks somehow sometime no all err message would be capture/wrote totally.
// i mean some err message just happening when vim exiting.
// some random err just happened while vim exiting occasionally.
Additional context
i am not sure if there were some better ways or if this was a bug..
if not, the feature request wish all err msg can be captured by this way or some others alternative.

i am trying to capture err message when vim exiting for debug purpose
...
if not, the feature request wish all err msg can be captured by this way or some others alternative.

Try this:
:debug quitall!
>step (or >next)

See:

:help :debug
:help >step
:help >next

Alternatively, try this:
:set verbose=2 verbosefile=/tmp/log | quitall!

Then, read /tmp/log.  If there is not enough info, you can increase the verbosity level up to 20.
See:

:help 'verbose'
:help 'verbosefile'

thx.
but some err message just happening when vim exiting, it happened randomly or occasionally,
which meant it more like log, not debug. however, seems not all err msg (when exiting) would wrote into mess at that timing.

it happened randomly or occasionally, which meant it more like log, not debug

Then an autocmd which increases the verbosity level and logs the messages into a file:
autocmd VimLeavePre set verbose=2 verbosefile=/tmp/log

Here is an example to test:
$ vim -Nu NONE -S <(tee <<'EOF'
    vim9script
    delete('/tmp/log')
    autocmd VimLeavePre * set verbose=1 verbosefile=/tmp/log
    autocmd VimLeave * Error()
    def Error()
        silent! eval [][0]
    enddef
    quitall!
EOF
)

$ grep '^E[0-9]\+:' /tmp/log
E684: List index out of range: 0

Notice that the error was logged even though it was given after the options were set.

ok, let me try verbose=20 (not sure which value is proper so 20) if can get sth or not.

even though it was given after the options were set.

that's ok, i just want the detail when just vim exiting, besides looks the verbose content would be very large (if 20).

Is your feature request about something that is currently impossible
or hard to do? Please describe the problem.
i am trying to capture err message when vim exiting for debug purpose
Describe the solution you'd like
what i do currently is:
au vimleave * call writefile(split(execute('mess'), '\n')

Describe alternatives you've considered
but looks somehow sometime no all err message would be capture/wrote
totally.
Additional context
i am not sure if there were some better ways or if this was a bug..
if not, the feature request wish all err msg can be captured by this
way or some others alternative.

It depends on why Vim is exiting.  I suppose you can't predict,
otherwise you could give a message there.
Have a look at ch_logfile().  It adds some overhead, but it might work
for your purpose.
--
A real patriot is the fellow who gets a parking ticket and rejoices
that the system works.
/// Bram Moolenaar -- @.*** -- http://www.Moolenaar.net   \
///                                                                      \
\\        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

yes, just trying to find out some unpredictable err msg.
// guess should be good enough, i will try. closing this for now. thx.


keep-test: Contract data bucket

This was done a very long time ago in a branch that's about to get closed.
Came from https://github.com/keep-network/keep-core/pull/1005

Any reason we wouldn't want to merge this now? Looks good to me.

No reason.

Salibtc


Bugtool: Fix gops commands

Due the changes made in 4fa2bcad0d1c92e12dffdf89513fb59aff915003
bugtool cannot retrieve gops stacks correctly. The main issue is that
bugtool use os.exec and never pass that information to bash, so the
output of gops stack was not correct. More info in #3981
With this change all commands run on bash, so parameters can be
retrieved without issues.
Fix #3981
Signed-off-by: Eloy Coto<EMAIL_ADDRESS>

test-me-please

Attached the bugtool output  from  my test, these two files are:

cilium-runtime: Run cilium-bugtool inside runtime vagrant box. (With sudo)
ciluum-k8s: Run cilium-bugtool inside cilium pod and retrieve the info from pod:

vagrant@k8s1:~$ kubectl exec -n kube-system -ti cilium-4lpbt -- cilium-bugtool
vagrant@k8s1:~$ kubectl  -n kube-system cp cilium-4lpbt:/tmp/cilium-bugtool-20180503-080709.834+0000-UTC-556457265.tar .
tar: Removing leading `/' from member names

cilium-bugtool.zip

Please provide an example of manual testing of bugtool to show that this has been fixed.

Test failure is unrelated as this change affects bugtool only, so I'm going to merge this.

@eloycoto @ianvernon do we intend to backport this?

backport in https://github.com/cilium/cilium/pull/4009


infinite retries

add support for infinite retries somehow? I can of course do
ConstantBackoff(math.MaxInt32)

but it'll allocate a huge array
perhaps an option to simply repeat last duration
or ability to pass a function for lazy evaluation instead of []Duration
or make use of rangefuncs of go 1.23 to provide a lazy evaluated infinite range of durations

I'm not sure that there's much value in this library supporting infinite retries, that's basically just a for { } loop which you don't need a library for?

In general, the cases where you need infinite retries are not "resiliency" cases so much.

I agree with eapache.
Infinite retries means no plan, with no change of recovery.
Basically, It's more constructive to have a plan to cover the failed process as a result of consuming all retries than to have it retry for an unrealistic amount of time.
Also, there is not much the library can guarantee in an infinite retries.
Then it is reasonable to deal with it at your own risk.


pytest

The current test suite needs several improvements:

[ ] Move from Python unittest to pytest to allow parameterization.
[ ] Organize tests logically. At the least, we should split them into dppy_kernel_tests and njit_tests.
[ ] Tests should run for realistic problem sizes.
[ ] We need to run the test cases for multiple dpctl.device_context (opencl:gpu, opencl:cpu, level0:gpu).
[ ] The test cases should be run for multiple data types. To start with maybe float32, float64 are sufficient, but later we should test for integer types, complex types, and reduced precision types.

@PokhodenkoSA @1e-to
I have updated the description of the ticket to capture several improvements that are needed for our test suite.
I had to partially start addressing these improvemnets in #179 as I need to run our test cases for both OpenCL and LevelZero. Look at the modified test_numpy_trigonometric_functions.py for an example of what I am proposing to do. Since, I did not know anything about pytest before today afternoon I may not be doing things quite right, so feel free to suggest changes.

@PokhodenkoSA @1e-to
I have updated the description of the ticket to capture several improvements that are needed for our test suite.
I had to partially start addressing these improvemnets in #179 as I need to run our test cases for both OpenCL and LevelZero. Look at the modified test_numpy_trigonometric_functions.py for an example of what I am proposing to do. Since, I did not know anything about pytest before today afternoon I may not be doing things quite right, so feel free to suggest changes.

@PokhodenkoSA why was this ticket closed? The sub-tasks that are outlined above are not yet addressed. Do we plan to open another ticket for those tasks?

@PokhodenkoSA why was this ticket closed? The sub-tasks that are outlined above are not yet addressed. Do we plan to open another ticket for those tasks?

It was closed by PR. But PR completed a part of work.

It was closed by PR. But PR completed a part of work.

All the sub-task are complete except the broadcast operation. Numba-dppy does not support broadcast operations yet. I am closing the ticket, we can open more topical issues for test suite improvement in the future.


Host online demo

Interactive, live demo online at http://demo.styleguide.sc5.io/

Good stuff! Sharing all over!


Prepare release 0.5.0

Summary
Move release notes to 0.5 folder.
Update version to 0.5.0
To be released on Monday, June 20.
Details and comments
Fixes #

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 66.058%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2676

Relevant Lines:
4051

💛  - Coveralls


[folding] folding indicators missing for folded code

Issue Type: Bug
For some time the visual studio code has been blocked, it does not display the fragments of code.
Suppose I have a code fragment inside an if, when it is blocked accessing this code inside the if is impossible, it is necessary to close the visual code and re-enter sometimes it happens up to 10 times in the day

access the points marked with the red arrow when it is blocked you can not
In this image you can see how the code can not be accessed

VS Code version: Code 1.35.1 (c7d83e57cd18f18026a8162d042843bda1bcf21f, 2019-06-12T14:30:02.622Z)
OS version: Windows_NT x64 10.0.17134
Extensions (18)

Extension
Author (truncated)
Version

html-snippets
abu
0.2.1

bracket-pair-colorizer
Coe
1.0.61

angular2-snippets
dan
1.0.21

githistory
don
0.4.6

vscode-html-css
ecm
0.2.0

tslint
eg2
1.0.43

auto-close-tag
for
0.5.6

auto-rename-tag
for
0.1.0

Angular2
joh
8.1.1

vscode-extension-auto-import
mop
1.3.3

vscode-language-pack-es
MS-
1.35.1

atom-keybindings
ms-
3.0.6

csharp
ms-
1.20.0

powershell
ms-
2019.5.0

al-formatter
ras
3.0.0

vscode-icons
vsc
8.7.0

JavaScriptSnippets
xab
1.7.2

tslint-snippets
zuo
1.1.0

Very strange...
What about the folding indicators (+, -) next to the line number?
What about changing the settings: editor.folding, editor.showFoldingControls ?

I also have this issue. I'm using 1.35.1. I like to use the keyboard shortcuts. Sometimes, after using them, I won't be able to unfold the code. It's a sporadic issue. Sometimes it resolves itself after a few minutes. Other times I have to restart VS Code.

Any solution for this?

I am also having this issue. This happens almost every single time I open a python file in the VSCode editor. The only solution (temporary) I have found is to restart VSCode (however after a few minutes of use, the arrows to unfold/fold the code disappear). I have enabled editor.folding and editor.showFoldingControls is set to always. I have also tried using the keyboard shortcuts but they don't work. When I try placing the cursor after the functions that are folded (and that I can't unfold), and deleting, it deletes code inside the function but does not unfold it (I can tell its deleting because the line numbers decrease).
Note: I am using VSCode to connect to a remote server (cluster)
Extensions:
Pylance: v2022.7.20
GitHub Copilot: v1.31.6194
Excel Viewer: v4.2.55
Python: v2022.10.0
R: v2.5.0
Jupyter: v2022.6.1001902341
Jupyter Notebook Renderers: v1.0.8
Jupyter Keymap: v1.0.0
Version: 1.69.0
OS: Windows_NT x64 10.0.19043

Same issue, I'm on a remote VueJS project
VS code v1.69.0
Microsoft Windows [Version 10.0.19044.1806]
Remote Linux: Ubuntu 20.04.2; Kernel 5.4.0-107-generic

Same issue, remote-ssh
Version: 1.69.2
Commit: 3b889b090b5ad5793f524b5d1d39fda662b96a2a
Date: 2022-07-18T16:32:39.000Z
Electron: 18.3.5
Chromium: 100.0.4896.160
Node.js: 16.13.2
V8: <IP_ADDRESS>-electron.0
OS: Darwin arm64 21.6.0

Same issue here. I use code folding a lot in vim but in vscode it's virtually impossible to use. Not only folding controls disappear - hotkeys don't work either.
It happens in any language.
I use it in remote mode over ssh.

Can confirm this:
Version: 1.70.1
Commit: 6d9b74a70ca9c7733b29f0456fd8195364076dda
Date: 2022-08-10T06:08:46.959Z (1 day ago)
Electron: 18.3.5
Chromium: 100.0.4896.160
Node.js: 16.13.2
V8: <IP_ADDRESS>-electron.0
OS: Darwin arm64 21.6.0

Will observe projects/used extensions.

I have the same issue with python files in vscode:
Version: 1.73.1
Commit:<PHONE_NUMBER>f055b99068d3688932416f2346dd3b
Date: 2022-11-09T03:54:53.913Z
Electron: 19.0.17
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Linux x64 5.15.0-56-generic
Sandboxed: No

isort
ms-
2022.8.0

python
ms-
2022.18.2

jupyter
ms-
2022.9.1303220346

jupyter-keymap
ms-
1.0.0

jupyter-renderers
ms-
1.0.12

vscode-jupyter-cell-tags
ms-
0.1.6

vscode-jupyter-slideshow
ms-
0.1.5

cpptools
ms-
1.12.4

LiveServer
rit
5.7.9

ayu
tea
1.0.5

very much an issue for me in js, i've tried re-installing vs code to no avail
It will work for the first few 20 (ish) seconds before it stops and all code folding functions don't work. If i fold a block of code before it happens i cannot unfold


Add parser for Anchore Grype scan

This pull request add a new parser for Anchore Grype, the scanner for Anchore.
https://github.com/anchore/grype
Grype is a vulnerability scanner for container images and filesystems. Easily install the binary to try it out.

so you don't like the unit tests for empty reports anymore? :-D
I think they were created to make sure that if empty reports are coming in, the parser doesn't crash. Similar to that we want to make sure we don't crash on missing nodes / elements / attributes.

I can try to generate an empty one. Let me add it.

So I have not tried or looked hard, but per your previous parser refactoring, that's all that's needed now to have parser? We probably need to update the parser doc.

yes. I tested it with ptvsd configuration and some real reports of the tool (version 0.7.0).
I'm working on the second part of the dynamic infrastructure. after this part is done, the factory will not need modifications when adding a new parser.

There's a couple changes that need to be made that were missed. See #2287 for details.

Actually, i think grype will/is leveraged at the decentralized way to gather findings too in their enterprise v3?
Good job anyways :)

@Maffooch the whole parser stuff has been completely overhauled in https://github.com/DefectDojo/django-DefectDojo/pull/3689
Also need to update the doc. @damiencarol could you amend eventually?

Are the docs and sample repo updated?

@valentijnscholten for the samples https://github.com/DefectDojo/sample-scan-files/pull/71 . Doc is comming


Sometimes the "beam" processes can't stop completely when stop the rabbit_node_ng job

When running "bosh deploy" to upgrade the rabbit_node_ng job, bosh will stop all the jobs on the vm first and then unmount the persistent disk. But sometimes stopping the jobs can't stop the "beam" processes (created by rabbit_node job), so the persistent disk is still used by "beam" and can't be unmounted. It will cause the bosh deploy fail. To fix it, we need to log in to the rabbit_node_ng vm and kill all the beam processes and re-run bosh deploy. With this problem, we can't run bosh deploy fully automatically without manual interference.
We investigated that the problem is caused by some warden processes (the father process of beam processes) can't be stopped by stopping rabbit_node job (/var/vcap/jobs/rabbit_node_ng/bin/rabbit_node_ctl stop). Normally, when bosh stops rabbit_node job, the  warden processes (like wshd: 19gdipma38k) will be killed so the beam processes will be killed along with them. But sometimes the warden processes failed to be killed so the beam processes keep alive and occupy the persistent disk.
One way to fix this problem we can think of is to add the function to "/var/vcap/jobs/rabbit_node_ng/bin/rabbit_node_ctl stop" command to check if there are some warden processes still alive after "kill_and_wait $PIDFILE 60". If yes, kill them.

Is there any update?
/cc @maximilien


SampleSet.fromList should be renamed to PointSet.fromList

Description of suggestion or shortcoming:
Right now, SampleSet.fromList generates a pointset distribution!
We should probably have an actual SampleSet.fromList function, that returns a SampleSet dist. (Maybe it requires the sample length to be the same as the active samples, or something?)

Wait, actually it is creating sample set dists, my bad.


DELIA-60079: Coverity Security Issues require fix in hdmicec

Initialization fixed for m_deviceInfoStatus

Please review the changes @anand-ky


Need to figure out how to color code the bar wicks correctly based on dynamic data.

Hello,
I'm attempting to color code the bar and wick based on custom logic.
In scenegraph.ts I set dColor based on custom properties I'm passing which sets the fill and stroke to a custom color.
export function getBarConfig(
  d: any,
  x: string,
  y: string,
  y2: string,
  width: number,
  fill: string | null,
  stroke: string | null,
  lineWidth: number | null,
  innerPadding = PADDING_INNER,
  maxPaddingInPixels = MAX_PADDING,
  pixelsToTime = 1000,
) {
  let base = 0;

  if (y2) {
    base = d[y2];
  }

  const dColor = d.sell ? '#FF5252' : d.buy ? '#24FF00' : d.hold ? '#24FF00' : '#FF5252'
  const strokesWidth = stroke && lineWidth ? lineWidth * 2 * pixelsToTime : 0;
  const calculatedPadding = Math.min(
    width * innerPadding,
    maxPaddingInPixels * pixelsToTime + strokesWidth,
  );

  return {
    x: d[x],
    y: Math.max(d[y] as number, base),
    height: Math.abs(base - (d[y] as number)),
    width: width - calculatedPadding,
    fill: dColor,
    stroke: dColor,
    lineWidth: lineWidth ?? 1,
  };
}

The fill color of the candles sets correctly while the wick remains gray. I'm having trouble tracking down where the wick is generated and how to dynamically change the color.
Thoughts?

Fixed!!


[MLLIB]Support specifying features via multiple columns

What changes were proposed in this pull request?
This PR will mainly introduce two new APIs in both “Predictor” and “PredictionModel”, and a new parameter named “featuresCols” in “sharedParams”.
APIs:
def setFeaturesCol(value: Array[String]): M = ...
protected def isSupportMultiColumnsForFeatures: Boolean = false
Parameter:
final val featuresCols: StringArrayParam = new StringArrayParam(this, "featuresCols",   ...)
Why are the changes needed?
There are always more features than one in a classification/regression task, however the current API to specify features columns in Predictor of Spark MLLib only supports one single column, which requires users to assemble the multiple features columns into a "org.apache.spark.ml.linalg.Vector" before fitting to Spark ML pipeline. So users have to spend some extra time to search and learn other transformers that can do this vectorization thing, then pick one in their applications. It looks like somewhat complicated.
With this PR, things become much more simple to run a machine learning task. Users no longer need the vectorization step for features columns, just specify these features columns by our new API, then fit the loaded the dataset directly to Spark ML Pipeline. It not only makes the ML application code more concise, but also saves a lot of time to learn other transformers for vectorization to run a classification/regression task , especially for novices at Spark ML.
Does this PR introduce any user-facing change?
Yes, it is adding a new API setFeaturesCol(value: Array[String] for users to specify multiple columns for features directly, without assembling them into a vector.
How was this patch tested?
Added unit test, and tested implementation on XGBoost-Spark.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

I don't know about MLLIB but this is clearly not a minor change, so leaving general comment to guide on contribution.
Could you please file an issue on JIRA and update the title of your PR? You're encouraged to take a look at the doc how to contribute via http://spark.apache.org/contributing.html
Thanks!

I don't know about MLLIB but this is clearly not a minor change, so leaving general comment to guide on contribution.
Could you please file an issue on JIRA and update the title of your PR? You're encouraged to take a look at the doc how to contribute via http://spark.apache.org/contributing.html
Thanks!

Thanks for reivew, filed an issue and updated the title.

ok to test

Test build #111691 has started for PR 25983 at commit 1146a85.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16684/
Test PASSed.

Test build #111691 has finished for PR 25983 at commit 1146a85.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
trait HasFeaturesCols extends Params

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111691/
Test PASSed.

Overall change makes sense to me, had one minor question on the test.  It would be nice to get someone more familiar with ML lib stuff to review though.  @WeichenXu123  are you more familiar with this code?

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16936/
Test PASSed.

@zhengruifeng saw you doing some multi-column work in another PR, pinging in case you are familiar with this code as well

test this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17370/
Test PASSed.

Test build #112400 has started for PR 25983 at commit b3c68e0.

Test build #112400 has finished for PR 25983 at commit b3c68e0.

This patch fails SparkR unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112400/
Test FAILed.

Test build #112406 has started for PR 25983 at commit b3c68e0.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17377/
Test PASSed.

Test build #112406 has finished for PR 25983 at commit b3c68e0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112406/
Test PASSed.

Yes the benefit is you don't have to create the vector, you can just pass in the multiple feature columns directly. An easier API to use.  It is being used with XGBoost.

@zhengruifeng  do you have specific concerns?

We already have a long-established component for this, VectorAssembler. One way or the other the assembly has to be done. I get that a utility setFeatures method saves a little code, but, it means changing all the implementations, while we still can't remove VectorAssembler. I'm not sure it's worth it.

right it can be done today, but this makes it  simpler to use and possibly more performant.  I think it especially helps the cases where your data is already in columnar format.  VectorAssembler has to make a pass over the data and merge multiple columns.   Many ML algorithms prefer columnar data and this allows the algorithm to determine what it wants to do with the columns. if it doesn't need to merge them, that is just extra processing.   If it already knows the format it can take advantage of that.
In this case you actually don't have to change the implementations until you want to, which is what this PR does.  Its just introducing the api, xgboost is going to use it, then other algo's can support as they want to.

The xgboost implementation can also just do this if desired, take n columns instead of 1 vector-valued one. Is this a real possibility or just theoretical? right now this doesn't take advantage of it anywhere in Spark ML; what would be an example?

What do you mean by can take n columns instead of 1?  The api currently allows for one parameter.
yes its being used in practice in XGBoost on GPU.   Any of the algorithms could either do the vector assembly internally (thus making it less complex for the user), or if the algorithm knows what to do with the feature columns it can skip the creation of the vector altogether.   We had some CPU code for XGboost that did this as well.  @firestarman can you provide an example.

You're suggesting that some implementation would prefer to be passed n columns, rather than 1 vector column of n values, because it might be more optimal in some cases. I'm asking what the use case is, either in theory or in practice. Are you saying the xgboost impl already effectively allows this?

yes we have an actual implementation that is using this for XGBoost running on GPUs.  This version takes advantage of the columnar layout.  @firestarman should have the code for the XGBoost CPU side as well that could side step creating the vector, which he should be able to point us to.
We thought it would be a good api addition for convenience and allow the  benefits I stated above.

Sure, but xgboost can do that itself without adding an API that isn't (yet) otherwise used by Spark MLlib. I guess I'd wait until there's more than 1 implementation that can take advantage, to make it an API method for everyone. How does this improve xgboost, out of curiosity -- is this in the repo already?

Yes we have an external XGBoost repo you should be able to see the gpu code @firestarman  can you point to it and explain a bit?    I get what your saying but I also think sometimes its the chicken and egg problem and other external algorithms outside mllib could benefit but don't want to hassle with modifying spark. we would have to investigate if other mllib algo's would be able to skip the vector assembly or if this api would just be an easier to use api and hide the vector creation from the user.

VectorAssembler has to make a pass over the data and merge multiple columns.
VectorAssembler only trigger a first() job to get the sizes of input vectors.

Many ML algorithms prefer columnar data and this allows the algorithm to determine what it wants to do with the columns.
Do you mean column-based parallelism used in distributed tree building? Such function is not exposed to end users, and what you need to do is only to set params like (..., updater=distcol).
If some alg will benefit from column-based parallelism, I guess it is better to split the features internally. No alg in MLLibs is designed to fit/transform with column-based datasets for now, so I do not prefer to add this feature.

It is being used with XGBoost.
I cannot find any related docs in XGBoost Parameters. Could you please provide a link for this?

Yes we have an external XGBoost repo you should be able to see the gpu code @firestarman can you point to it and explain a bit?

Here are the links of gpu code

Column names => indices
https://github.com/rapidsai/xgboost/blob/rapids-spark/jvm-packages/xgboost4j-spark/src/main/scala/ml/dmlc/xgboost4j/scala/spark/XGBoost.scala#L484
indices => handles of column data
https://github.com/rapidsai/xgboost/blob/rapids-spark/jvm-packages/xgboost4j-spark/src/main/scala/ml/dmlc/xgboost4j/scala/spark/XGBoost.scala#L917

You can see features column names are used directly to transform to the handles of columns data,  then build the columnar based DMatrix for XGBoost.

VectorAssembler only trigger a first() job to get the sizes of input vectors.
And I think we can futhuer optimize it so that this first() job is not triggered when all columns are double values.
Do you mean column-based parallelism used in distributed tree building? Such function is not exposed to end users, and what you need to do is only to set params like (..., updater=distcol).
If some alg will benefit from column-based parallelism, I guess it is better to split the features internally.

Thanks for review.
Actually this is not only for alg that benefit from column-based parallelism, but also for spark ML application users, with this PR and its implemention in MLLibs, they don't need VectorAssembler any more for ML features column things, but simply and directly set feature column names to Predictor instead, just like what you do with label and prediction.
What's more, the original column names are required for MLLibs to parse the info of the column-based datasets. But currently there is no API to save the original column names in a Predictor .

No alg in MLLibs is designed to fit/transform with column-based datasets for now, so I do not prefer to add this feature. I cannot find any related docs in XGBoost Parameters. Could you please provide a link for this?

XGBoost should support column-based datasets now.  you can find clues from code or PRs, like
https://github.com/dmlc/xgboost/pull/4816
https://github.com/dmlc/xgboost/pull/4850
https://github.com/dmlc/xgboost/pull/4902
One more thing, would it be more reasonable to define the APIs firstly in Spakr ML interface for requirements from users, then have them implemented in Spark MLLibs and other ML implementations ?

@zhengruifeng

Is the lack of response because people are ok with the explanation?
There are also lots of algorithms outside the spark core ml lib code that use these classes and so we were thinking this would be convenience method that doesn't add much overhead to the spark core code.  It seems like pure overhead to create the VectorAssembler if the algorithm doesn't need it.

So, xgboost's impl can already do this. It doesn't need to be an API in Spark. If Spark implementations can benefit from it, then it makes sense to add this API. But if there aren't any other impls, I don't think it adds anything, and adds a second way to do something you can do already. I'd rather not until it buys more.
Am I missing something... why not just expose this in xgboost right now?

If current algs in MLLIB will not benefit from this design (execpt skip VectorAssembler), I do not prefer it. SKLearn is also widely used, it do not support this feature.

There are also lots of algorithms outside the spark core ml lib code that use these classes and so we were thinking this would be convenience method that doesn't add much overhead to the spark core code.

Yes, but I think current design in MLLIB do not pose an obstacle to impl third-party libs.

Right but if its not in Spark then multiple algos/libs can't use it and will likely re-invent the same thing. People are not going to see it in xgboost and then decide to add it to Spark.  This is why I encouraged Liangcai to propose it.  To me its useful to XGBoost so the question is then could it be useful to other algorithms in the future. To me I thought that answer was yes for the following reasons:

Simpler, more user friendly API

more obvious to user how to send in multiple features
user doesn't have to create VectorAssembler
algorithm could behind the scene create the Vector if required
algorithm could skip if its not required
don't lose the information about original column names
give the algorithm more flexibility

more performant if can skip the VectorAssembler
If GPU algorithm could be a lot more performant if data is already columnar

Then on top of that, its really not much code so I didn't think it introduced much maintenance on the Spark side.  We could mark it with Experimental as well and remove it if really not used by anyone else.
I'm by no means an expert in all ML algorithms, so if that doesn't make sense and you don't think matches way everyone is doing it then we can just leave it in XGboost then.

@firestarman  please close this


Python: package with dependency on grpcio fails to install

I can't get packages that have a dependency on grpcio to install; they fail with the message
...
third_party/zlib/gzwrite.c: In function ‘gzclose_w’:
third_party/zlib/gzwrite.c:573:5: warning: implicit declaration of function ‘close’ [-Wimplicit-function-declaration]
     if (close(state->fd) == -1)
     ^
zip_safe flag not set; analyzing archive contents...
No eggs found in /tmp/easy_install-PByQ05/grpcio-0.14.0/egg-dist-tmp-qwx9F0 (setup script problem?)
error: The 'grpcio>=0.14.0' distribution was not found and is required by [...]

Strangely, it seems possible to work around this by installing grpcio explicitly using pip.
I am able to reproduce the error with a minimal Python package on a clean virtual machine. That package is available here, with steps to reproduce in the README: https://github.com/geigerj/temp-repo
The full error message is available here: https://gist.github.com/geigerj/951aa1db2fdd732f3feaa68549e83cfd

@bjwatson

Looks like this might be a key piece?
https://gist.github.com/geigerj/951aa1db2fdd732f3feaa68549e83cfd#file-install_output-txt-L43

Looks like this might have been related?
https://github.com/grpc/grpc/issues/6075

So I noticed that in the pre-release 0.14.1 that there are some updates for grpcio.
See: https://github.com/grpc/grpc/releases/tag/release-0_14_1
I haven't been able to get it to build the bdist_egg.

Note to self: set the zip_safe flag on distributions.

@soltanmm
In #6439, we decided we did not want to support egg distributions.  I had a quick fix #6437 for this, but we decided to not integrate it.
In light of this use case, do we want to re-evaluate?

Was this fixed by https://github.com/grpc/grpc/pull/6437?

Yes.

Is there an estimate for when 0.15.0 will be released?

@bjwatson: we think it will be in the next seven days.

Thanks.


Ensure apiQueryURL ends in slash in the entire app, not just when sending a query

in neurobagel/query-tool#212 we added an automatic adding of a slash to the query URL:
https://github.com/neurobagel/query-tool/blob/81a690fd74e0c08a95079a26622f4d4ed018517a/components/QueryForm.vue#L153-L158
but this only affects the sending of the queries, not the overall apiQueryURL (e.g. for listing the nodes and options):
https://github.com/neurobagel/query-tool/blob/81a690fd74e0c08a95079a26622f4d4ed018517a/nuxt.config.js#L55-L58
We should make sure we add a slash if one is missing at the deepest level (maybe even immediately when we read the .env file)

Following up on the previous pull request, what additional requirements need to be addressed for this issue? 👨🍳

Hi @samadpls,
This issue has been addressed in the new tool but we currently don't have a test for it. If you're interested in writing a test for it please see https://github.com/neurobagel/query-tool/issues/61


Dev/coze demo

Purpose
Add coze graph, agent web page can connect with custom coze settings
Task

[x] After coze related graph selected, corresponding settings tab will show in settings dialog;
[x] If coze settings are empty and coze graph is selected, corresponding settings dialog will show;
[x] All settings are stored in browser localstorage;
[x] Customized error toast;
[ ]  Add coze label/logo to tab title

Screenshots

when will this branch merged into main?