Parallel MOF combilation

Adding support for parallel MOF compilation. The number if jobs can be defined with parameter 'MofCompilationTaskCount'. If this parameter is not used, standard serial build is used

Starting more than 2 processes loading the same modules can cause issues. This is why the 'RandomWait' parameter is required in parallel build to let the new processes wait to a random number of seconds.

Adding support for parallel Meta MOF compilation.
Throwing if the compilation does not succeed so InvokeBuild get the failure status
Removing 'FilterNode' and 'FilteredNode' references as it is not used and redundant with 'Filter'.
Simple code reorg and code style alignments

The MOF build still works successful.

Failed build?
Flying today, won't have time to look.

Working now, there was still a case sensitivity issue with the roles in the config files.

@gaelcolas, any comments on this PR?

Right...

This took quite some time:
PSDesiredStateConfiguration\node : Specified argument was out of the range of valid values.
Parameter name: times
At C:\Git\Aldi DSC\Operations\RootConfiguration.ps1:16 char:5
+     node $ConfigurationData.AllNodes.NodeName {
+     ~~~~
    + CategoryInfo          : OperationStopped: (:) [PSDesiredStateConfiguration\node], ArgumentOutOfRangeException
    + FullyQualifiedErrorId : System.ArgumentOutOfRangeException,PSDesiredStateConfiguration\node

The error was actually caused in RootConfiguration.ps1::51 if "55 -$okMessage.Length" results in a negative number. Fix is on the way...

Ping...

All good and tested (finally).
Merged


If data dir exists already, just use performance options. If not, use sync type (instead of config.Hypersync)

Current dependencies on/for this PR:

main

PR #934   👈

This stack of pull requests is managed by Graphite.


Skip tests that fail so we can merge antoniois PR to fail when tests …

will allow us to merge  #401 i think

Also wait a second @jayunit100 Now we're not running ANY e2e tests....
[ PASSED ] kpng-e2e-ipv4-iptables is operational.
[ PASSED ] Removed daemonset.apps/kube-proxy.
[ PASSED ] Loaded kpng:test container image.
[ PASSED ] Created service account kpng.
[ PASSED ] Created clusterrolebinding kpng.
[ PASSED ] Created configmap kpng.
[ INFO ] Enabling Metrics
[ PASSED ] Installation of kpng is done.

I think what we really need to do is maintain SKIP lists of explicit test's which are failing for each backend so we have TODOs to go and fix those specific tests ("rather then just skipping everything")

i found a bunch of NFT tests and others fail now.  ill double check with this CI run. but ... something like

Summarizing 37 Failures:
  [FAIL] [sig-network] Services [It] should be able to switch session affinity for NodePort service [LinuxOnly] [Conformance]
  test/e2e/network/service.go:3858
  [FAIL] [sig-network] Conntrack [It] should be able to preserve UDP traffic when initial unready endpoints get ready
  vendor/github.com/onsi/ginkgo/v2/internal/suite.go:605
  [FAIL] [sig-network] Conntrack [It] should be able to preserve UDP traffic when server pod cycles for a ClusterIP service
  vendor/github.com/onsi/ginkgo/v2/internal/suite.go:605
  [FAIL] [sig-network] Services [It] should be able to create a functioning NodePort service [Conformance]
  test/e2e/network/service.go:1286
  [FAIL] [sig-network] Services [It] should preserve source pod IP for traffic thru service cluster IP [LinuxOnly]
  test/e2e/network/util.go:133
  [FAIL] [sig-network] Services [It] should be possible to connect to a service via ExternalIP when the external IP is not assigned to a node
  test/e2e/network/service.go:1321
  [FAIL] [sig-network] Services [It] should be able to change the type from ExternalName to ClusterIP [Conformance]
  test/e2e/network/service.go:1432
  [FAIL] [sig-network] Services [It] should have session affinity timeout work for NodePort service [LinuxOnly] [Conformance]
  test/e2e/network/service.go:3775
  [FAIL] [sig-network] Services [It] should allow pods to hairpin back to themselves through services
  test/e2e/network/service.go:1129
  [FAIL] [sig-network] Conntrack [It] should drop INVALID conntrack entries [Privileged]
  vendor/github.com/onsi/ginkgo/v2/internal/suite.go:605
  [FAIL] [sig-network] Services [It] should be able to change the type from NodePort to ExternalName [Conformance]
  test/e2e/network/service.go:1557
  [FAIL] [sig-network] Services [It] should be able to connect to terminating and unready endpoints if PublishNotReadyAddresses is true
  test/e2e/network/util.go:175
  [FAIL] [sig-network] Services [It] should have session affinity work for NodePort service [LinuxOnly] [Conformance]
  test/e2e/network/service.go:3858
  [FAIL] [sig-network] Services [It] should implement service.kubernetes.io/service-proxy-name
  test/e2e/network/service.go:2267
  [FAIL] [sig-network] Services [It] should have session affinity timeout work for service with type clusterIP [LinuxOnly] [Conformance]
  test/e2e/network/service.go:3775
  [FAIL] [sig-network] Services [It] should have session affinity work for service with type clusterIP [LinuxOnly] [Conformance]
  test/e2e/network/service.go:3858
  [FAIL] [sig-network] Conntrack [It] should be able to preserve UDP traffic when server pod cycles for a NodePort service
  vendor/github.com/onsi/ginkgo/v2/internal/suite.go:605
  [FAIL] [sig-network] Services [It] should be rejected for evicted pods (no endpoints exist)
  test/e2e/network/service.go:2477
  [FAIL] [sig-network] Services [It] should be rejected when no endpoints exist
  test/e2e/network/service.go:2386
  [FAIL] [sig-network] Services [It] should be able to up and down services
  test/e2e/network/service.go:1148
  [FAIL] [sig-network] Services [It] should implement service.kubernetes.io/headless
  test/e2e/network/service.go:2319
  [FAIL] [sig-network] Services [It] should be able to change the type from ClusterIP to ExternalName [Conformance]
  test/e2e/network/service.go:1513
  [FAIL] [sig-network] DNS [It] should resolve DNS of partial qualified names for services [LinuxOnly] [Conformance]
  test/e2e/network/dns_common.go:455
  [FAIL] [sig-network] DNS [It] should provide DNS for services  [Conformance]
  test/e2e/network/dns_common.go:455
  [FAIL] [sig-network] DNS [It] should resolve DNS of partial qualified names for the cluster [LinuxOnly]
  test/e2e/network/dns_common.go:455
  [FAIL] [sig-network] Services [It] should be able to switch session affinity for service with type clusterIP [LinuxOnly] [Conformance]
  test/e2e/network/service.go:3858
  [FAIL] [sig-cli] Kubectl client Guestbook application [It] should create and stop a working application  [Conformance]
  test/e2e/kubectl/kubectl.go:403
  [FAIL] [sig-network] Services [It] should create endpoints for unready pods
  vendor/github.com/onsi/ginkgo/v2/internal/suite.go:605
  [FAIL] [sig-network] Services [It] should serve a basic endpoint from pods  [Conformance]
  test/e2e/network/service.go:824
  [FAIL] [sig-network] DNS [It] should provide DNS for pods for Subdomain [Conformance]
  test/e2e/network/dns_common.go:455
  [FAIL] [sig-auth] ServiceAccounts [It] ServiceAccountIssuerDiscovery should support OIDC discovery of service account issuer [Conformance]
  test/e2e/auth/service_accounts.go:634
  [FAIL] [sig-network] DNS [It] should provide DNS for the cluster  [Conformance]
  test/e2e/network/dns_common.go:455
  [FAIL] [sig-network] Services [It] should be able to update service type to NodePort listening on same port number but different protocols
  test/e2e/network/service.go:1365
  [FAIL] [sig-network] Services [It] should be able to change the type from ExternalName to NodePort [Conformance]
  test/e2e/network/service.go:1471
  [FAIL] [sig-network] DNS [It] should provide DNS for ExternalName services [Conformance]
  test/e2e/network/dns_common.go:455
  [FAIL] [sig-network] Networking Granular Checks: Services [It] should function for endpoint-Service: http
  vendor/github.com/onsi/ginkgo/v2/internal/suite.go:605
  [FAIL] [sig-network] Networking Granular Checks: Services [It] should update endpoints: udp
  vendor/github.com/onsi/ginkgo/v2/internal/suite.go:605


[Bug] Installing plugins fails

Describe the bug
Installing a plugin via the admin console fails. There is no specific error message in the UI, but the logfile says Failed to install plugin" error="failed to create temporary file: open /tmp/1141914888.zip: read-only file system. I suppose every K8s has a read-only filesystem. I haven't found any other way to install plugins, e. g. setting GF_INSTALL_PLUGINS fails for the same reason.
Version
5.0.0-rc0
To Reproduce
Steps to reproduce the behavior:
see above
Expected behavior
I would expect it to be possible to install a plugin with the Operator. The docs do not provide any insight on this, except for a situation where a dashboard-plugin can be installed with a dashboard. But in my case it is about a datasource plugin.
Runtime (please complete the following information):

OS: Linux
Environment: Kubernetes (k3s)

Hi @realulim you're using a fairly old release of the operator, the RC was only intended for initial and experimental deployments, do you also see the same behaviour with the newest release? v5.4.1 ?

5.0.0-rc1 was the current version at the time I installed the cluster. I have now upgraded to 5.4.1 and could install the plugins via the Admin user. However, in additional to the success message, an error message popped up:
Plugin not found, no installed plugin with that id
The plugin is also not listed under the "installed plugins" section. When I go to its page again, I again get the button to install the plugin. The log file said this:
logger=installer.fs t=2023-09-12T12:48:34.002837457Z level=info msg="Downloaded and extracted marcusolsson-json-datasource v1.3.6 zip successfully to /var/lib/grafana/plugins/marcusolsson-json-datasource" logger=plugin.loader t=2023-09-12T12:48:34.032928166Z level=info msg="Plugin registered" pluginID=marcusolsson-json-datasource logger=context userId=1 orgId=1 uname=nxgadmin t=2023-09-12T12:48:34.232519424Z level=info msg="Request Completed" method=GET path=/api/plugins/marcusolsson-json-datasource/settings status=404 remote_addr=<IP_ADDRESS> time_ms=48 duration=48.428622ms size=64 referer=https://***/plugins/marcusolsson-json-datasource handler=/api/plugins/:pluginId/settings logger=context userId=1 orgId=1 uname=nxgadmin t=2023-09-12T12:48:34.368251292Z level=info msg="Request Completed" method=GET path=/public/plugins/marcusolsson-json-datasource/module.js status=404 remote_addr=<IP_ADDRESS> time_ms=48 duration=48.41061ms size=31 referer=https://***/plugins/marcusolsson-json-datasource handler=public-assets
Perhaps the referrer (which I have asterisked out) is the culprit, because that URL is certainly not available on the Internet.
On another note, when the Grafana containers start, there appears the following message in the log file:
logger=provisioning.plugins t=2023-09-12T12:46:12.458269324Z level=error msg="Failed to read plugin provisioning files from directory" path=/etc/grafana/provisioning/plugins error="open /etc/grafana/provisioning/plugins: no such file or directory"
Perhaps this directory should actually be present?

Oops, I'm sorry, I misspoke in my previous comment. The ***-ed URL is actually available on the Internet (if I am logged into Grafana as Admin) and leads to the plugin page.

No, I am installing via the Admin UI. The example you point to is for installing dashboard plugins, but here we have datasource plugins.
Also, we are managing our dashboards not through the operator, but via the Grafana API (as they have to created and removed at runtime).

I have now created my own Grafana image based on grafana/grafana-oss:9.5.9, which adds the environment variable for installing plugins like so:
ENV GF_INSTALL_PLUGINS=marcusolsson-json-datasource,marcusolsson-csv-datasource
This works fine, when I start the image via docker-compose, but when I use it with the Grafana Operator, then this environment variable is ignored. It seems the operator sets it to empty.
So what is actually the correct way to install Grafana plugins globally in a cluster managed by the Grafana Operator?

The problem appears to be that the Plugin is installed only in one pod (I have 3 Grafana pods running). So occasionally, when this pod will be hit, the plugin appears to be installed and with the next request another pod is hit and the plugin isn't there.

@realulim the env is supposed to be populated by the operator itself. Based on the resources you have, it prepares the env, and then it's up to grafana to install everything. Some of the examples can be found here.

@weisdd the example you point to is for installing a dashboard (and a plugin to go with it). However, I need to install a plugin (no dashboard). Doing this via the Web UI fails, when more than one pod is running - it is only ever installed on one pod. Perhaps it is not supported to install plugins via the Web UI, but I haven't found any docs on that.
What did work eventually was to set GF_INSTALL_PLUGINS on the deployment descriptor. It does not work to set it in the image, which is what I tried first and what worked with Docker. But it seems the operator ignores the ENV variables set in the image.

@realulim on the same page, you can find an example for installing plugins through a datasource CR, so GrafanaDashboard is not the only option.
grafana-operator is meant to manage only those things that are explicitly specified in corresponding CRs. Thus, if you manually make an API call through Web UI to install a plugin, it gets installed on whatever instance the request happen to be forwarded to. It has nothing to do with the operator's behaviour.
As I pointed out earlier, GF_INSTALL_PLUGINS is supposed to be populated by the operator based on CRs. Other scenarios are not supported, and not sure why you would need that in the first place. If you still think you have a compelling use-case, feel free to raise a feature request, it'll be reviewed during one of the next meetings, we're open to new ideas.

@weisdd I've seen those examples, but unfortunately they don't apply to my use case. I want to install a plugin. Not a datasource or a dashboard or anything else - just a plugin. Since I found nothing in the docs on how to do that programmatically or via an API, I tried to do it through the Web UI. If that is not supported, then is there any other supported way of installing a plugin?
As I said, setting GF_INSTALL_PLUGINS seems to work, when setting it on the deployment. But if this is not the correct way to install plugins, then what is?
Or do you mean to say that provisioning Grafana with a number of plugins already installed is not supported at all?

@realulim If you want to install only a plugin, the correct way to do so is still to reference it in any GrafanaDashboard / GrafanaDatasource CR.
Any Grafana instance exists to show dashboards while getting data from datasources, so I can't think of a scenario where you want to use the operator, but don't want to deploy GrafanaDashboard / GrafanaDatasource. Thus, I don't really understand your concern in the first place.

@weisdd I have an application, where users can create dashboards dynamically based on the data they have. These dashboards are then created at runtime via the Grafana API. But of course the plugin has to be present at that time. And there is no way to install a plugin via the Grafana API, so I have to make sure the plugin is already present once the dashboards (and datasources) get created.
Thus: I need to provide a Grafana instance with a certain number of plugins at install time. Then the dashboards and datasources get created at runtime by the users.

@realulim But you still have several options:

Update the tool that generates dashboards or patch its manifests (e.g. with kustomize or anything else that fits your deployment model) to include information on which plugins need to be installed (so it becomes part of Dashboard/Datasource CR);
Create a dashboard/datasource yourself with those plugins that you're aware of;
Prepackage plugins in the image you use.

With option 1 or 2, the operator will recreate grafana instance with updated GF_INSTALL_PLUGINS env, and then Grafana will install the plugins during its boot-up process.

The tool uses the Grafana API to create dashboards. But there is no way to install plugins via the Grafana API.
Same story. Everything is created via the Grafana API. The tool is a userland tool, so it has no access to the K8s cluster.
I tried that. I set GF_INSTALL_PLUGINS as an environment variable in the image and tested the image with docker and it worked. But the same image did not work with the operator, the plugins weren't installed.

The only thing that did work was setting GF_INSTALL_PLUGINS in the deployment descriptor for the Operator.

@realulim I think we're going in circles here :)
So, you're saying that it's you who installs plugins by setting the environment variable.
Instead, why can't you simply create a dummy GrafanaDashboard or GrafanaDatasource with references to the required plugins to make the operator install them for you? - That's the way to do that with the operator. And, in the end, you need to have at least one datasource, so it won't hurt to create to create that datasource with operator and reference all required plugins in the same CR.
If you avoid that path just because you don't want to have any CRs deployed along with the operator, probably, you don't need the operator in the first place, because the operator is meant to work with CRs.

Creating a dummy datasource is something I haven't tried. Maybe that could work if I could find a way to hide it from the users.
The application works in a way that datasources / dashboards are installed, once a user is given permission to view the data. Once permission is taken away (or the data deleted), the datasource / dashboard is removed again. So there is a seperate datasource / dashboard lifecycle that is managed by the users themselves.
Therefore a dummy datasource would be confusing to users and, to be fair, it seems like a pretty convoluted strategy to install a datasource that nobody needs just to get a plugin. From a security point of view a datasource needs special permissions, whereas a plugin does not. Plugins, on the other hand, are code and have versions and need to be updated seperately. So in my eyes it is not a good idea to lump those concepts together.

It can be a dashboard instead, doesn't really matter. If you experiment with this and still think you want to have plugin management at grafana CR level in addition to what we already have, feel free to create a separate feature request explaining why you think it's a good idea. If majority of maintainers gets convinced by your use-case, the feature will get added at some point. If not, you can still fallback to either datasource or dashboard CR.

I think all required instructions have been provided, thus closing the issue.

I still think it's a bug that the Operator does allow the manual installation of a plugin, but then only installs it on one node. I believe if manual installation cannot be made to work with multiple nodes, there should be an error message instead. Or at the very least a bold warning in the docs: do not use the Grafana UI to install plugins, when you have multiple nodes.

I may not have enough knowledge about this operator. But in general K8s operators are for managing multiple nodes of a stateful system. That obviously includes managing state between the nodes and I've seen K8s operators do that.
And to some extent this operator seems to manage state, too. For example, I have installed dashboards via the Grafana API and they seem to work on all nodes, not just the one I sent the API request to. So it is unexpected that using the Grafana API via the UI works differently. A useful word of warning could be "Don't manage Grafana via its UI, when using the operator, because every command you send will go to only one node".

@realulim If you want to install only a plugin, the correct way to do so is still to reference it in any GrafanaDashboard / GrafanaDatasource CR. Any Grafana instance exists to show dashboards while getting data from datasources, so I can't think of a scenario where you want to use the operator, but don't want to deploy GrafanaDashboard / GrafanaDatasource. Thus, I don't really understand your concern in the first place.

But how can we auto enable the plugin? I successfully installed a plugin, but I can't enable it automatically.

@realulim But you still have several options:
1. Update the tool that generates dashboards or patch its manifests (e.g. with kustomize or anything else that fits your deployment model) to include information on which plugins need to be installed (so it becomes part of Dashboard/Datasource CR);

2. Create a dashboard/datasource yourself with those plugins that you're aware of;

3. Prepackage plugins in the image you use.

With option 1 or 2, the operator will recreate grafana instance with updated GF_INSTALL_PLUGINS env, and then Grafana will install the plugins during its boot-up process.

@weisdd
Our grafana instances have no internet access, therefore, we can't use the GF_INSTALL_PLUGINS option.
I have packaged the plugin in the docker image I use, but this one seems to be removed by the operator.
Using the same image locally (docker run) or with the bitnami helm  chart (no operator) works, does the operator empty the GF_INSTALL_PLUGINS folder on start ?


Bruno installed via snap on Xubuntu is crashing

Problem
Bruno installed via snap on Xubuntu is crashing on startup.
Bruno version:
snap list  | grep bruno
bruno                      0.25.0                      3      latest/stable  helloanoop        -

Gtk version:
gtk-launch --version
3.24.33

Xubuntu version:
lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.3 LTS
Release:	22.04
Codename:	jammy

Error message:
A JavaScript error occurred in the main process
Uncaught Exception:
Error: Cannot find module<EMAIL_ADDRESS>Please verify that the package.json has a valid "main" entry
    at tryPackage (node:internal/modules/cjs/loader:360:19)
    at Module._findPath (node:internal/modules/cjs/loader:573:18)
    at Module._resolveFilename (node:internal/modules/cjs/loader:926:27)
    at n._resolveFilename (node:electron/js2c/browser_init:249:1105)
    at Module._load (node:internal/modules/cjs/loader:785:27)
    at c._load (node:electron/js2c/asar_bundle:5:13339)
    at Module.require (node:internal/modules/cjs/loader:1012:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/snap/bruno/3/resources/app.asar/node_modules/@usebruno/js/src/utils.js:2:17)
    at Module._compile (node:internal/modules/cjs/loader:1120:14)
Gtk-Message: 11:29:12.811: Failed to load module "xapp-gtk3-module"

(bruno:24205): Gtk-WARNING **: 11:29:12.857: Theme parsing error: gtk.css:1374:85: 'font-feature-settings' is not a valid property name

BTW your .deb package works perfect, thank you for building this great product.
dpkg -i bruno_0.25.0_amd64_linux.deb
Selecting previously unselected package bruno.
(Reading database ... 309523 files and directories currently installed.)
Preparing to unpack bruno_0.25.0_amd64_linux.deb ...
Unpacking bruno (0.25.0) ...
Setting up bruno (0.25.0) ...

@FrontEndStudio
Glad that you like Bruno!
The snap build has been fixed in the latest release v0.26.0 . Can you try again ?

Closing this issue, please re-open if snap builds still continue to fail.


[ACR] az acr connected-acr create/update/delete/list/show: new feature in beta.

Description
Adding Connected Registry code to the CLI
History Notes
[Component Name 1] az acr connected-acr: New beta feature

This checklist is used to make sure that common guidelines for a pull request are followed.

[x] The PR title and description has followed the guideline in Submitting Pull Requests.

[x] I adhere to the Command Guidelines.

ready for upstream fork


cannot use CloneFlags

For some reason all CloneFlags constants and type are not found at all, as if they are not declared public
error[E0425]: cannot find value `CLONE_NEWNET` in module `nix::sched`
  --> src/main.rs:13:67
   |
13 |     let p = nix::sched::clone(Box::new(hello), stack, nix::sched::CLONE_NEWNET, None).unwrap();
   |                                                                   ^^^^^^^^^^^^ not found in `nix::sched`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0425`.

the compiler version is
rustc 1.31.1 (b6c32da9b 2018-12-18)

stupid me, I apologize


Remove OpenDayLight plugin

Integration tests: https://beta-jenkins.mcc.schubergphilis.com/job/cosmic/job/0002-tracking-repo-branch-build/121/
Merge together with: https://github.com/MissionCriticalCloud/cosmic-client/pull/60 and https://github.com/MissionCriticalCloud/cosmic-plugin-hypervisor-kvm/pull/29

All green:


Jupyter plugin hides some custom kernelspecs when the executable defined in the argv cannot be found

Environment data

VS Code version: 1.79.2
Jupyter Extension version (available under the Extensions sidebar): v2023.4.1001091014
Python Extension version (available under the Extensions sidebar): v2023.10.1
OS (Windows | Mac | Linux distro) and version: Linux Ubuntu 22.04 (same on CentOS7)
Python and/or Anaconda version: n/a
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): n/a
Jupyter server running: Local (same container as vscode)

Expected behaviour
All kernel spec visible in JupyterLab interface should appear in VSCode/Jupyter Kernels list as well. Eventually explicit error message (or warning) is shown if kernelspec is incomplete or broken in some way.
Extend documentation in https://github.com/microsoft/vscode-jupyter/wiki/General-overview-of-Kernel-Discovery-&-Execution-in-Jupyter-(&-extension)#1-global-kernelspecs and/or https://code.visualstudio.com/docs/datascience/jupyter-kernel-management#_jupyter-kernels to explicitly state what paths are searched and how broken or incomplete kernelspec are handled.
I was giving up trying jupyter plugin after not being able to use my kernel with it until I realized why it skips my kernel spec.
Actual behaviour
Some kernels are silently skipped during detection.
Steps to reproduce:
Put a kernel spec into ~/.local/share/jupyter/kernels/custom/kernel.json  with a command on system path:
{
    "display_name": "CUSTOM",
    "language": "python-custom",
    "argv": [
        "mycommand",
        "{connection_file}"
    ]
}

The kernel will be picked up by JupyterLab and will work as expected.
KernelPicker will not show the kernel in the list.
Resolution on my side was to change "mycommand" into "/full/path/mycommand" and it worked! Thanks for the great plugin!
EDIT: Per discussion below it's language set to be different from python (like "python-custom") that does the trick. With language set to "python", which is the default, the code makes some assumptions that are not valid for custom kernels.

Thank you for filing this issue and sorry you are running into this.
I've tested this at my end (with the exact same kernelspec json) and it works as expected. please can you test this with the latest version of Jupyter exetnsion
You do not have the latest version.
Also, please can you sahre the logs from the Jupyter extension

Open your command palette and select the command Jupyter: Show Output
Copy all of the output and paste that here
This generate contains errors or warnings when kernels are hidden from the list

Ok, my bad. It appears that I changed "language" also and that was what affected things. My kernel spec that was not detected:
{
    "display_name": "CUSTOM",
    "language": "python",
    "argv": [
        "custom-start",
        "{connection_file}"
    ],
    "interrupt_mode": "message"
}

Thanks for the tip about "Jupyter: Show Output":
20:50:10.735 [warn] Kernel Spec for 'CUSTOM' (~/.local/share/jupyter/kernels/custom/kernel.json) hidden, as we cannot find a matching interpreter argv = 'custom-start'

The message is a bit cryptic though.
As per: https://jupyter-client.readthedocs.io/en/stable/kernels.html

language: The name of the language of the kernel. When loading notebooks, if no matching kernelspec key (may differ across machines) is found, a kernel with a matching language will be used. This allows a notebook written on any Python or Julia kernel to be properly associated with the user’s Python or Julia kernel, even if they aren’t listed under the same name as the author’s.

It feels weird that the field affects how kernel is getting detected, don't you think? My use case is similar to "docker kernel" as in https://stackoverflow.com/a/63715102, i.e., it is python kernel but not locally installed.

It feels weird that the field affects how kernel is getting detected, don't you think?

Thanks, I agree.
However we found that most users end up with just python in that path or pyspark or the like, and when using VS Code, such commands are not in the path (env variable) of VS Code, as a result these kernels cannot be launched from within VS Code.
Outside vscode users open terminals and have conda activated or the like, hence those executables are in the current path
But go into vscode they fail, and thats why we hide them.
I understand that this doesn't work for you, i just wanted you to understand why we decided to exclude them (to avoide users frolm running into issues where the kernel might not start)
I'll discuss this with the team

Thanks for getting back to this!
Just to be clear: I'm completely fine that VSCode requires full path in the kernelspec file. It was just not clear for someone like me trying it out that:
(a) kernel spec was found but hidden [ since I didn't know I should look into the "output" and even then message was pretty cryptic - it should have said that absolute path is required ]
(b) not a word about the need to specify the full path in the docs about "generic kernelspec", nor about specifics for the "python" kernels, nor about potential "hiding" of the kernelspec (I did read all the docs before reporting, sorry if I missed that)

Like @iakovn, myself (and actually multiple other physicists in my collaboration but I'm the one debugging it on our behalf) are hitting this. We'd like to be able to use VS Code's remote explorer to interact with Jupyter on remote environments but given some requirements for the software that we need in the environment (we need to be able to see and interact with the CernVM File System) we need to mutate the default kernelspec generated from something like
(lcg-example) [23:01] login01.af.uchicago.edu:~ $ python -m ipykernel install --user --name="lcg-example" --display-name="Example LCG view + venv kernel"
Installed kernelspec lcg-example in /home/feickert/.local/share/jupyter/kernels/lcg-example
(lcg-example) [23:10] login01.af.uchicago.edu:~ $ cat ~/.local/share/jupyter/kernels/lcg-example/.bak.kernel.json
{
 "argv": [
  "/home/feickert/.venvs/lcg-example/bin/python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Example LCG view + venv kernel",
 "language": "python",
 "metadata": {
  "debugger": true
 }
}(lcg-example) [23:10] login01.af.uchicago.edu:~ $

to something that would allow for the environment manipulation suggested by the Jupyter team in https://github.com/jupyterhub/jupyterhub/issues/847#issuecomment-260152425
(lcg-example) [23:11] login01.af.uchicago.edu:~ $ cat ~/.local/share/jupyter/kernels/lcg-example/kernel.json
{
 "argv": [
  "/home/feickert/.local/share/jupyter/kernels/lcg-example/startup.sh",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Example LCG view + venv kernel",
 "language": "python",
 "metadata": {
  "debugger": true
 }
}
(lcg-example) [23:11] login01.af.uchicago.edu:~ $ cat ~/.local/share/jupyter/kernels/lcg-example/startup.sh
#!/usr/bin/env bash

export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
# Allows for working with wrappers as well
source "${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh" --quiet || echo "~~~ERROR: setupATLAS failed!~~~"

lsetup 'views LCG_103 x86_64-centos7-gcc11-opt'

. ~/.venvs/lcg-example/bin/activate

exec python -m ipykernel_launcher $@
(lcg-example) [23:11] login01.af.uchicago.edu:~ $

Resolution on my side was to change "mycommand" into "/full/path/mycommand" and it worked!

@iakovn, can you elaborate on this? My experience with the valid kernelspec above is failure with VS Code failing to find the kernel at all.
@DonJayamanne while I appreciate that the VS Code team might not want to make things visible by default, can it be made possible so that there is at least some dev option to still give a full path to a valid kernel?

Environment:

VS Code version: 1.79.2
Jupyter Extension version (available under the Extensions sidebar): v2023.6.1101941928
Python Extension version (available under the Extensions sidebar): v2023.12.0
OS (Windows | Mac | Linux distro) and version: Linux Ubuntu 22.04 (remote server is CentOS7)
Python and/or Anaconda version: CPython 3.9.12
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): venv
Jupyter server running: Remote

Using /bin/bash as the first element in argv and changing "language" to "python-custom" should solve the problem for you as far as I understand.

changing "language" to "python-custom" should solve the problem for you as far as I understand.

Wow, indeed this works! Amazing and thank you!

Using /bin/bash as the first element in argv

I had previously run chmod +x on my startup script, but you're right that explicitly putting this in the kernel is probably a better approach to be safe.
Thanks for your help @iakovn.

@DonJayamanne Are there any official docs on using python-custom?

EDIT: Per discussion below it's language set to be different from python (like "python-custom") that does the trick

@iakovn Sorry one final question, from this do I correctly understand that python-custom" is just an arbitrary string choice and the only thing that matters for the kernel to get picked up is that "language" is not "python"? That is, "python-custom" isn't something from IPython, correct?

EDIT: Per discussion below it's language set to be different from python (like "python-custom") that does the trick

@iakovn Sorry one final question, from this do I correctly understand that python-custom" is just an arbitrary string choice and the only thing that matters for the kernel to get picked up is that "language" is not "python"? That is, "python-custom" isn't something from IPython, correct?

That's my understanding, yes. Seems that the plugin has some undocumented heuristics for "python" aiming to automatically handle venvs.
@DonJayamanne would be great if you explain the logic either here or in the docs.

That's my understanding, yes. Seems that the plugin has some undocumented heuristics for "python" aiming to automatically handle venvs.

Thanks. Yes, from my limited understanding of kernelspecs from the jupyter_client kernel sepcs docs (which is where I've found the best user docs on IPython kernelspecs so far)

language: The name of the language of the kernel. When loading notebooks, if no matching kernelspec key (may differ across machines) is found, a kernel with a matching language will be used. This allows a notebook written on any Python or Julia kernel to be properly associated with the user’s Python or Julia kernel, even if they aren’t listed under the same name as the author’s.

there's no constraint on "language". So we're just use "language": "python-custom" to make a human readable note that we're getting to Python by another means.
But very much agreed that user docs from the VS Code team on this would be amazing!

Hi everyone, I'm sorry about this issue, I will review the current workflow next iteration.
Perhaps what the Jupyter extension can do is,

Not hide items that have paths such as /bin/bash or the like
And if we do hide items (which have ambiguous paths such as python), then we display a more informative message in the logs and how it can be resolved.

Perhaps what the Jupyter extension can do is,

Not hide items that have paths such as /bin/bash or the like
And if we do hide items (which have ambiguous paths such as python), then we display a more informative message in the logs and how it can be resolved.

Thanks for these suggestions @DonJayamanne. I think both of these sound reasonable, but even just documentation on what the expected user behavior is would be great.

@DonJayamanne - are the original repro steps valid to verify this, and what is the expected behavior?

We continue to hide them but have a better message in the logs

It seems to now attempt to start the kernel, and fail:
 kernelConnectionMetadata: Cs {
    kind: 'startUsingLocalKernelSpec',
    kernelSpec: {
      specFile: 'c:\\Users\\<username>\\AppData\\Roaming\\jupyter\\kernels\\incompleteSpec\\kernel.json',
      name: 'incompleteSpec',
      argv: [Array],
      language: 'python',
      executable: 'custom-startup',
      display_name: 'incomplete spec',
      metadata: [Object]
    },
    interpreter: undefined,
    id: '.incompleteSpec.custom-startup.\\.'
  },

It seems to now attempt to start the kernel, and fail:

@amunger I'm not sure i understand the issue,
is the problem that the kernel appears in the list and it does not work, or that jupyter appears to launch this process and  it fails?

oh sorry, we will still display these custom python kernel specs, and if it fails, its on the user, basically you have a bogus kernel spec, and we have no idea its bogus or not.
In the issue you can see users have custom kernel.specs and just because we fail, doesn't mean its invalid, the language can be python but they are not really pyton kernels, e.g. they can be launched through some custom executable.
So its correct to display these, thats whats expected.


Retry-After for last_operation endpoints?

Hello, we would like to set the "Retry-After" in reposnse Header of */last_operation , like described in the spec: https://petstore.swagger.io/?url=https://raw.githubusercontent.com/openservicebrokerapi/servicebroker/v2.16/openapi.yaml#/ServiceInstances/serviceInstance.lastOperation.get
How can we do this?
Thanks :-)

I've added a new property LastOperationResource.RetryAfter for this purpose in d3d3e698fb592c976c9084bea9c0befce195d398.
This is available in OpenServiceBroker.Server 0.5.0.

Many thanks!


cpp.Void prioritized over Void

When targeting C++ target in Haxe, autocomplete prioritizing cpp.Void completion over Void, resulting in autogeneration of cpp.Void import, and breaking compilation as cpp.Void is a return type.
Current workaround:
Write Void and the press escape to cancel autocomplete.
Visual representation of the problem:
https://i.imgur.com/WNgcehA.gif

Maybe you would like to try the build for this PR and give your feedback to include it or further improve it:
https://ci.appveyor.com/project/FlashDevelop/flashdevelop-devel/build/<IP_ADDRESS>-hsflhfjf/artifacts

@Neverbirth Yes, this PR do fix that problem. I've talked with Slava about this problem last year, what actually resulted in that PR. Talked with him right now, and he said he'll fix merge conflicts soon.


SAK-41197 Allow log job to include a stacktrace.

The test logging job doesn't include a stacktrace, this adds the ability
to include a stacktrace in the test logging event.

@buckett can you fix the conflict

Conflict resolved.


chore: Rename frontend folder to skore-ui

part of #345

Linked with https://github.com/probabl-ai/skore/issues/345


maxLength dosen't work with integer when combined

Attmeting to validate with rules: ['integer','maxLength:5'] will always return false saying "The field must not exceed 5 characters long" however the value is "1234"

Surely the rule you're looking for is: 'max:999'.


Add integration test for secondary user-store user account lock

Add integration test to test issue https://github.com/wso2/product-is/issues/16625

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/6194907251

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/6194907251
Status: failure

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/6196543865

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/6196543865
Status: cancelled

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/6197048418

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/6197048418
Status: cancelled

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/6197707748

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/6197707748
Status: success


BZ1275325: WAS: indexing fails with ContextNotActiveException

See https://bugzilla.redhat.com/show_bug.cgi?id=1275325
Implementation of EJB was flawed; it was neither an "no interface" bean, nor a "local interface" bean.

Will this work on Tomcat, where there is no EJB implementation?

@psiroky Hi, I've not changed the underlying mechanism to create an Asynchronous Executor. We try to lookup a EJB Asynchronous Bean here if it fails - as it would on Tomcat - we fallback to our own thread pool here.

@manstis thanks for the explanation. I just saw "EJB" and immediately though about Tomcat....

Thanks.
master: 064be4dcd4cb322341d160e01fb9f458d59c4175

@jhrcek this is another good example where only an integration test would help guard against regressions on WAS. We have usage against the new interface in tests in kie-wb-common, but only an integration test can catch real problems here. Anyways, something for our list of ToDos.


docs: set heroku button to target tag v1.10.3

This sets the Heroku Deploy button to target the v1.10.3 tag. Previously, it would deploy the master branch which can sometimes be broken, see #794. We would also have to update the link manually every time we make a new release.

I actually just wanted to do that as well LOL. Thank you 👍


Sensor Grouping

Dear team,
Firstly, many thanks for your fantastic work on SSPOC and also for providing this package. This is indeed a beneficial resource.
I am currently using PySensors to find the minimal sensors required for a classification task. However, instead of running this directly on the actual sensor count, I intend to input the extracted features and then select the number of sensors.
Specifically, I have data from 20 sensors, and I calculate 100 features/sensor. This results in a matrix with 2000 columns. In other words, the SSPOC algorithm treats this as a problem with 2000 sensors. While the 'selected_sensors' lists the particular feature that can be traced back to the corresponding sensor. I would be grateful if you could please link me to any inbuilt function for grouping the sensors or any other voting mechanism that I may use or implement?
Thanks a lot for your time.

Unfortunately we don't currently have any built-in methods with the functionality you're suggesting. I'm also unaware of any functions for grouping sensors. I think your best bet is probably to implement a custom method. You could write a function that maps from the expanded sensor index (0 to 1999) to the original sensor index (0 to 19). Then you could apply this function to selected_sensors. A few ideas for how to proceed from there:

Pick the first k unique sensors to appear
Of the top n features, choose the k which appear most often
Use the individual sensor "saliences" (from the sensor_coef_ attribute) to help decide which sensors to use, e.g. sum up the saliences across features for each sensor and select the k sensors with the highest scores. See the update_sensors source code for examples of how to work with sensor_coef_.

Unfortunately we don't currently have any built-in methods with the functionality you're suggesting. I'm also unaware of any functions for grouping sensors. I think your best bet is probably to implement a custom method. You could write a function that maps from the expanded sensor index (0 to 1999) to the original sensor index (0 to 19). Then you could apply this function to selected_sensors. A few ideas for how to proceed from there:

Pick the first k unique sensors to appear
Of the top n features, choose the k which appear most often
Use the individual sensor "saliences" (from the sensor_coef_ attribute) to help decide which sensors to use, e.g. sum up the saliences across features for each sensor and select the k sensors with the highest scores. See the update_sensors source code for examples of how to work with sensor_coef_.


Bug/ui fixes 2

This is meant to be merged after https://github.com/caciviclab/disclosure-frontend/pull/160 because it is based on that branch. There were a few things in the UI fix list related to #160.

Scroll to main content on state transition

Fix the page title and page metadata

[x] #160

@KyleW just rebased this one too, ready for review when you get a chance.

I think this is mergeable since #160 shipped, but didn't click the button since I wasn't sure. @adborden Merge if you're ready.


reentrency (read file while write)

Hello,
I am using littlefs with freertos (nxp sdk).
Is it allowed to read file while another task is writing into the same file ?
Thank you,
ranran

Ranran,
I am using littleFS with FreeRTOS.
My experience are, Multible readers are OK, but writers must be alone.
If reads are occuring while a file are open for write, you will have bad data written into your file.
Regards
Carsten


Permission for connection with sample-kotlin

Hi,
I use yours sample-kotlin to try understand how to code with your library but I face an issues.
The scan work perfectly fine but when I try "service discovery" I have this message : "Connection error: java.lang.SecurityException: Need android.permission.BLUETOOTH_CONNEC..."
I had "uses-permission android:name="android.permission.BLUETOOTH_CONNECT " in AndroidManifest.xml but the issue is the same.
My mobile is an OppoReno 6 5g with 12 android version
Thanks for any help you can provide

Hmmm... I see that there is a problem in the sample app and the library. You should be able to workaround it by giving a BLUETOOTH_CONNECT runtime permission to the app.

Thanks I was able to fix it with the runtime permission it's work perfectly fine now


feat: Add support for percent of totals in MetricsViewAggregation

To add support for percent of totals based filter criteria in alerts, we need to add percent of totals as a variant of a measure.
This PR adds that support to MetricsViewAggregation by using existing window functions.
Note: There is an issue with limits with this approach. Percent of total applies after limit (2nd added test fails)

On reflection, I think this might be simpler to solve with an additional query.
I think there might actually be a simple way to achieve that. The new metricsview package supports Query-level and AST-level rewrites. This might be achieved with a Query-level rewrite that looks for percent-of-total measures, and if it finds any, computes the total and rewrites the compute measure to contain the actual float value to divide by.
For inspiration, you can see this rewriter which turns relative time ranges into fixed time ranges, sometimes making a separate query to get the min/max timestamps of the metrics view: https://github.com/rilldata/rill/blob/b67474d7b4fd3770f21999f0c3e96a3e86c216b7/runtime/metricsview/executor_rewrite_time.go#L14-L16


57 flexible configuration

Proposed changes
Implemented "scrutineer2" to run verification with a pair of stream configurations (existing bin/scrutineer is still supported)
scrutineer/target/appassembler/bin/scrutineer2 \
    --primary-config=jdbc.properties \
    --secondary-config=elasticsearch7.properties \
    --numeric

**This addresses issue #57 **
Checklist

[ x] I've read the CONTRIBUTING doc
[x ] All tests pass locally with my changes
[x] I've added unit tests and integration tests.
[x] I've added necessary documentation (if appropriate)

Further comments
Let's start the review/discussion first, I have updated the README, CHANGELOG

Also if you can rebase as I've added security checks to the Github workflows set that you can bring in.

I have updated README AND CHANGELOG with instructions to run scrutineer2


Getting an error while running redis-commander

Hi All,
I could not able run the redis-commaner, it is throwing the below error.
Could you some faced this type of error, please help.

The port 8081 is already in use. Do you have another instance of redis-commander running somewhere else or another application listening on port 8081?

This is no error in redis commander but another app running at the same tcp port at the same time.
Its expected behaviour for this case - only one app at a time can use the port 8081


I2S:  left/right channels are swapped for read (IDFGH-4826)

Environment

Development Kit: Lolin D32 Pro
Kit version: NA
Module or chip used:  ESP32-WROVER-32
IDF version: v4.0.2
Build System:   CMake
Compiler version: xtensa-esp32-elf-gcc (crosstool-NG esp-2020r3) 8.4.0
Operating System:     Windows 10
(Windows only) environment type:  WSL2
Using an IDE?: No
Power Supply:   USB

Problem Description
When reading a mono left-channel microphone using ONLY_LEFT, all the audio samples are zero.  Changing to ONLY_RIGHT results in the expected samples being read from the I2S microphone.
Cause:  A bugfix that was applied to other ESP-IDF releases has been missed in the v4.0.2 release.  The fix commit is:
https://github.com/espressif/esp-idf/commit/ff325e84b65e88547ed5508ae1de82232f2d8f43
Expected Behavior
Audio samples are read
Actual Behavior
Audio samples are all-zero
Steps to reproduce

configure a one-channel I2S microphone to put sample data onto the left channel
configure I2S to read data on the left channel.  e.g:  ONLY_LEFT

Code to reproduce this issue
code is not needed in this case.  The bug will be clear when the development team looks at the bug referenced above (https://github.com/espressif/esp-idf/commit/ff325e84b65e88547ed5508ae1de82232f2d8f43) and compares to code in the function i2s_param_config() in file i2s.c
Debug Logs
Other items if possible

@miketeachman Thanks for reporting and sorry for late reply, we have back ported the fix to release/4.0, will update once the fix is available on GitHub. Thanks.

Thanks for reporting and sorry for late reply, the fix on release/4.0 is available https://github.com/espressif/esp-idf/commit/4e5805dec3d956bd2c4c095b5457f203dd3b4347, feel free to reopen.


Update to new helm stable repo

Switch out deprecated helm repo for new stable repo.

https://www.cncf.io/blog/2020/11/05/helm-chart-repository-deprecation-update/
https://helm.sh/docs/faq/#i-am-getting-a-warning-about-unable-to-get-an-update-from-the-stable-chart-repository

I don't have write access so I cannot merge.


No opacity slider in colour selection?

I'm not seeing an option to adjust opacity in either of the layers or term info colour selection tools.  This would mostly be useful for templates and domains so that their colours do not interfere too much with the neurons and expression patterns shown within these volumes. Might also be nice to have the colour selection tools always be the same to make things easier for the user.

https://casesandberg.github.io/react-color/
As discussed at the sprint call this is the library we use at the moment, feel free to choose another picker and we can swap that in VFB.
Thx!

I would vote for the chrome colour picker, maybe without the hex colour code section if possible, as it's simliar to what's already there but with the opacity slider.


quantized BERT model save fail

Hello,
Firstly, thank you for making this resource available! Learned about it via Twitter https://twitter.com/peter_izsak/status/1190896532515696640
I tried the quantized BERT training example from your docs: http://nlp_architect.nervanasys.com/quantized_bert.html
The training halts without an error message. I do get an FP32 model in the output directory (pytorch_model.bin), an eval_results file, and checkpoint sub-directories, but not a quantized model (expecting quant_pytorch_model.bin)
Is there something further I should do / some way to diagnose what's going wrong?
thank you! Andrew

Hi @cainesap
Are you using the last code? git cloned and installed from there?
The quantized export is not available yet if you installed using pip install nlp-architect

Ah! No I'm not, thank you for the quick response, I'll try that :)

I installed from github but now see an error on 1st epoch:
2019-11-06 15:59:01,601 INFO ***** Running training *****
2019-11-06 15:59:01,625 INFO   Num examples = 3668
2019-11-06 15:59:01,626 INFO   Num Epochs = 3
2019-11-06 15:59:01,626 INFO   Instantaneous batch size per GPU/CPU = 8
2019-11-06 15:59:01,626 INFO   Total train batch size (w. parallel, distributed & accumulation) = 8
2019-11-06 15:59:01,627 INFO   Gradient Accumulation steps = 1
2019-11-06 15:59:01,627 INFO   Total optimization steps = 1377
Epoch:   0%|                                                                                | 0/3 [00:00<?, ?it/s]Killediteration:   0%|▎                                                                                | 1/459 [01:51<14:14:24, 111.93s/it]

Can I do anything else to diagnose?
thank you, Andrew

I installed and ran the quantized BERT training example. I encounter the following error at the outset. Any pointers would be much appreciated! Or let me know if I can provide further info..
2019-11-06 15:59:01,601 INFO ***** Running training *****
2019-11-06 15:59:01,625 INFO   Num examples = 3668
2019-11-06 15:59:01,626 INFO   Num Epochs = 3
2019-11-06 15:59:01,626 INFO   Instantaneous batch size per GPU/CPU = 8
2019-11-06 15:59:01,626 INFO   Total train batch size (w. parallel, distributed & accumulation) = 8
2019-11-06 15:59:01,627 INFO   Gradient Accumulation steps = 1
2019-11-06 15:59:01,627 INFO   Total optimization steps = 1377
Epoch:   0%|                                                                                | 0/3 [00:00<?, ?it/s]Killediteration:   0%|▎                                                                                | 1/459 [01:51<14:14:24, 111.93s/it]

Hi @cainesap , I guess its a backend configuration. Are you running on GPU?
Can you provide any other logs to diagnose the error?

Thanks @peteriz for the reply -- it prompted me to check again that I had all the right dependencies. I updated pytorch (CPU) and all is fine! Model trains and saves a quantized version :)) thanks again


Uncaught TypeError: Storage.removeItem is not a function

Describe the bug
The error is displayed after a successful upload using Chrome browser.
To Reproduce
Steps to reproduce the behavior:

Select the file, using file html element in the form.
Start upload, on successful upload the following error is displayed.
Uncaught TypeError: Storage.removeItem is not a function at XMLHttpRequest.xhr.onload (tus.js)

** Error source **
if (offset == this._size) {
if (this.options.removeFingerprintOnSuccess && this.options.resume) {
// Remove stored fingerprint and corresponding endpoint. This causes
// new upload of the same file must be treated as a different file.
Storage.removeItem(this._fingerprint);
}
** Fix **
if (offset == this._size) {
if (this.options.removeFingerprintOnSuccess && this.options.resume) {
// Remove stored fingerprint and corresponding endpoint. This causes
// new upload of the same file must be treated as a different file.
this._storage.removeItem(this._fingerprint);
}

Moved to https://github.com/tus/tus-js-client/issues/151.


Add code coverage

references:
https://adamwathan.me/2014/12/28/test-coverage-code-climate-travis-ci/

Tried with Istanbul, but it was showing up all the transpiled code as untested. There are few alternative approaches to deal with transpiled code, but I don't see it being worth the effort.


fix DefaultClient may not release connection when sync wipe cache

fix DefaultClient may not release connection when sync wipe cache
here  is error log which I added to find why sync ofthen fail
Thread failed during wipe cache at java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.

so I add some code to make sure connection to be released

Thank you for the PR. Could you follow the patch process, with Kylin code style? please check: http://kylin.apache.org/development/howto_contribute.html


Search can't find map

I think there is something wrong with using params with axios.
Raw request URL https://api.beatsaver.com/search/text/0?q=Bad%20Romance%20Lady%20Gaga&sortOrder=Relevance
Using
(await axios.get('https://api.beatsaver.com/search/text/0', {
  params: { q: 'Bad%20Romance%20Lady%20Gaga', sortOrder: 'Relevance' },
})).data

yields
{ docs: [] }

Using
(await axios.get(
  'https://api.beatsaver.com/search/text/0?q=Bad%20Romance%20Lady%20Gaga&sortOrder=Relevance'
)).data

yields
{
  docs: [
    {
      id: 'fcad',
      name: 'Lady Gaga - Bad Romance (Short Ver.)',
      description: 'Lady Gaga - Bad Romance (Short Ver.)\n' +
        'Preview : https://www.bilibili.com/video/BV1A541177Qb/',
      uploader: [Object],
      metadata: [Object],
      stats: [Object],
      uploaded: '2020-10-10T06:42:40.442Z',
      automapper: false,
      ranked: false,
      qualified: false,
      versions: [Array],
      createdAt: '2020-10-10T06:42:40.442Z',
      updatedAt: '2020-10-10T06:42:40.442Z',
      lastPublishedAt: '2020-10-10T06:42:40.442Z'
    },
    {
      id: '1f571',
      name: 'Sweet but Bad Romance',
      description: 'Remix of Sweet but Psycho (Ava Max) and Bad Romance (Lady Gaga)\n',
      uploader: [Object],
      metadata: [Object],
      stats: [Object],
      uploaded: '2021-12-29T05:36:10.101380Z',
      automapper: false,
      ranked: false,
      qualified: false,
      versions: [Array],
      createdAt: '2021-12-29T05:34:55.176899Z',
      updatedAt: '2021-12-30T07:00:51.591270Z',
      lastPublishedAt: '2021-12-30T07:00:51.591270Z'
    },
    {
      id: '106b0',
      name: 'Lady Gaga: Bad Romance',
      description: 'Updated (Fixed a few bad beats). This is my first map. Took me about 12 hours. Any feedback I would greatly appreciate or if you want to improve it, message me.',
      uploader: [Object],
      metadata: [Object],
      stats: [Object],
      uploaded: '2020-10-31T02:40:51.745Z',
      automapper: false,
      ranked: false,
      qualified: false,
      versions: [Array],
      createdAt: '2020-10-31T02:40:51.745Z',
      updatedAt: '2020-10-31T02:40:51.745Z',
      lastPublishedAt: '2020-10-31T02:40:51.745Z'
    }
  ]
}

Not sure but I think this might help resolving it: https://axios-http.com/docs/urlencoded

Hmm sounds correct here
https://axios-http.com/docs/req_config

// params are the URL parameters to be sent with the request
// Must be a plain object or a URLSearchParams object
// NOTE: params that are null or undefined are not rendered in the URL.
params: {
ID: 12345
},

// paramsSerializer is an optional function in charge of serializing params
// (e.g. https://www.npmjs.com/package/qs, http://api.jquery.com/jquery.param/)
paramsSerializer: function (params) {
return Qs.stringify(params, {arrayFormat: 'brackets'})
},

The problem seems to be with the encodeURIComponent. Using encodeURI also doesn't work. I think axios automatically does this and therefore this creates errors like this


I don't know how to use to join channel

python3 bot.py
and then? and how to send a command?

You can set the default channel in the settings. After it is connected, you can tell it to join other channels by giving it the .join #channel command.


ecdsa-sha2-nistp256 is a weak host key algorithm?

Looking at your tool I see ecdsa-sha2-nistp256 gets flagged as weak and I can't find a reason why. I can't find any discussion of it being a weak algorithm. Why is it flagged in your tool?

Can you please update your script to include the latest ciphers as per the below IETF page.
https://tools.ietf.org/id/draft-ietf-curdle-ssh-kex-sha2-09.html#rfc.section.3
def get_output(rawlist): 	if rawlist: 		ciphers = ['3des-cbc','aes128-cbc','aes192-cbc','aes256-cbc','aes128-ctr','aes192-ctr','aes256-ctr','aes128-gcm@openssh.com','aes256-gcm<EMAIL_ADDRESS>	strong_ciphers = ['chacha20-poly1305@openssh.com','aes256-gcm@openssh.com','aes128-gcm@openssh.com','aes256-ctr','aes192-ctr','aes128-ctr'] 		weak_ciphers = [] 	   	macs = ['hmac-md5','hmac-md5-96','hmac-ripemd160','hmac-sha1','hmac-sha1-96','hmac-sha2-256','hmac-sha2-512','umac-64','hmac-md5-etm@openssh.com','hmac-md5-96-etm@openssh.com','hmac-ripemd160-etm@openssh.com','hmac-sha1-etm@openssh.com','hmac-sha1-96-etm@openssh.com','hmac-sha2-256-etm@openssh.com','hmac-sha2-512-etm@openssh.com','umac-64-etm<EMAIL_ADDRESS>	strong_macs = ['hmac-sha2-512-etm@openssh.com','hmac-sha2-256-etm@openssh.com','umac-128','umac-128-etm<EMAIL_ADDRESS>	weak_macs = [] 		kex = ['curve25519-sha256', 'curve25519-sha256@libssh.org','diffie-hellman-group1-sha1','diffie-hellman-group14-sha1','diffie-hellman-group-exchange-sha1','diffie-hellman-group-exchange-sha256','ecdh-sha2-nistp256','ecdh-sha2-nistp384','ecdh-sha2-nistp521','ecdsa-sha2-nistp256-cert-v01@openssh.com','ecdsa-sha2-nistp384-cert-v01<EMAIL_ADDRESS>	strong_kex = ['curve25519-sha256'<EMAIL_ADDRESS>'diffie-hellman-group-exchange-sha256'] 		weak_kex = [] 		hka = ['ecdsa-sha2-nistp256-cert-v01@openssh.com','ecdsa-sha2-nistp384-cert-v01@openssh.com','ecdsa-sha2-nistp521-cert-v01@openssh.com','ssh-ed25519-cert-v01@openssh.com','ssh-rsa-cert-v01@openssh.com','ssh-dss-cert-v01@openssh.com','ssh-rsa-cert-v00@openssh.com','ssh-dss-cert-v00@openssh.com','ecdsa-sha2-nistp256','ecdsa-sha2-nistp384','ecdsa-sha2-nistp521','ssh-ed25519','ssh-rsa','ssh-dss'] 		strong_hka = ['ssh-rsa-cert-v01@openssh.com','ssh-ed25519-cert-v01@openssh.com','ssh-rsa-cert-v00@openssh.com','ssh-rsa','ssh-ed25519'] 		weak_hka = [] 		dmacs = []

I'm not actually using this tool anymore, but I'm still curious of the design choice if you'd like to comment on that.

The Kex algorithms (weak, strong, mac etc..) are hard coded. I would prefer it to be fetched from the openssh site if possible.
The reference I could find of the Kex alogrithms/ciphers was an RFC link is below:
https://tools.ietf.org/id/draft-ietf-curdle-ssh-kex-sha2-09.html#rfc.section.3

Ok, but in that reference ecdsa-sha2-nistp256 is listed as a SHOULD implement host key algorithm. Hence, it's strange to me that such a designation from the ietf would lead to a hard coding of the algorithm as weak.

hiya, the reasoning was explained in a blog post which is embedded in the old source code: https://github.com/evict/SSHScan/commit/4be9a2b8f466eff16e6306c3e83633a5ab1cbb1c.
https://stribika.github.io/2015/01/04/secure-secure-shell.html
It's not configurable, if you still wish to use it. Check config.yml


fix(consume join identity authorization): pass correct issuer pays arg

:tada: This PR is included in version 3.0.0-alpha.10 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 3.0.0-beta.1 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 3.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


RHEL prerequisite not found

this error is on RHEL9, also we should switch to dnf command from yum
[rocky@dirk1 ~]$ sudo dnf install lib32gcc-s1 unzip fuse3 python3-devel
Last metadata expiration check: 0:56:17 ago on Fri Jun 21 11:35:13 2024.
No match for argument: lib32gcc-s1
Package unzip-6.0-56.el9.x86_64 is already installed.
Package fuse3-3.10.2-6.el9.x86_64 is already installed.
Error: Unable to find a match: lib32gcc-s1

Solved in froster v0.14.7


Debbugger not starting

Output:
Traceback (most recent call last):   File "/Users/d0sse/Library/Application Support/Sublime Text 3/Packages/Debugger/modules/debugger.py", line 69, in get     instance = Debugger(window, skip_project_check=skip_project_check)   File "/Users/d0sse/Library/Application Support/Sublime Text 3/Packages/Debugger/modules/debugger.py", line 118, in __init__     self.console: ConsoleOutputPanel = ConsoleOutputPanel(self)   File "/Users/d0sse/Library/Application Support/Sublime Text 3/Packages/Debugger/modules/console.py", line 22, in __init__     super().__init__(debugger, 'Debugger Console', name='Console', show_tabs=True, remove_last_newline=True)   File "/Users/d0sse/Library/Application Support/Sublime Text 3/Packages/Debugger/modules/output_panel.py", line 137, in __init__     scaled_font = settings.get('font_size') * (Settings.minimum_console_height + 1.75) / 5 TypeError: unsupported operand type(s) for +: 'NoneType' and 'float'

Now works!

Now works!

Did you have to do anything to fix this? Restart Sublime or something?

I ran into the same issue.
Fixed it by setting my "font_size" to the default in my user settings, then restarting sublime.

Now works!

Did you have to do anything to fix this? Restart Sublime or something?

I changed Font_size, then I restarted Sublime!


Update Tekton Pipeline v0.35.1 -> v0.36.0

Update Tekton Pipeline dependency to v0.36.0
https://github.com/tektoncd/pipeline/releases/tag/v0.36.0

Fixed by: https://github.com/buildsec/frsca/pull/237


Input filtering in pb/http interfaces for things that can crash the Riak cluster [JIRA: RIAK-1619]

Passing in a string/binary instead of an integer in an n-val for bucket properties can lead to some hard to troubleshoot seemingly random crashes.
For example: https://gist.github.com/marksteele/c2002b44b005ea1fda63
A user specified a binary as the n-val, which causes things using coverage queries to crash.
All input should be type checked to prevent misbehaving clients from bringing down the system. I suspect that n-vals are simply one of many places where introducing type checking will prevent hard to troubleshoot issues.

This seems only appropriate for HTTP interface (PB already has strict types on properties), but should already be caught by bucket validators. The exception is potentially that a bad default property could be set and not caught by the validators if the user does not explicitly create buckets/bucket-types.
[posted via JIRA by Sean Cribbs]


Minimal or empty docs

This is a list of pages which have little or no real documentation in them

https://strands.readthedocs.io/en/latest/aaf_deployment/index.html
https://strands.readthedocs.io/en/latest/fremen/fremengrid.html
https://strands.readthedocs.io/en/latest/g4s_deployment/index.html
https://strands.readthedocs.io/en/latest/g4s_deployment/docs/network.html
https://strands.readthedocs.io/en/latest/g4s_deployment/tsc_bringup/checklist.html
https://strands.readthedocs.io/en/latest/mongodb_store/index.html
https://strands.readthedocs.io/en/latest/planning_tutorial/index.html
https://strands.readthedocs.io/en/latest/robblog/data.html latin text
https://strands.readthedocs.io/en/latest/scitos_apps/index.html
https://strands.readthedocs.io/en/latest/scitos_2d_navigation/wiki/Home.html
https://strands.readthedocs.io/en/latest/scitos_drivers/flir_pantilt_d46.html
https://strands.readthedocs.io/en/latest/scitos_robot/wiki/Home.html
https://strands.readthedocs.io/en/latest/semantic_segmentation/index.html nothing useful
https://strands.readthedocs.io/en/latest/strands_3d_mapping/strands_sweep_registration.html
https://strands.readthedocs.io/en/latest/strands_3d_mapping/dynamic_object_retrieval/k_means_tree.html
https://strands.readthedocs.io/en/latest/strands_3d_mapping/observation_registration/index.html
https://strands.readthedocs.io/en/latest/strands_3d_mapping/observation_registration/observation_registration_launcher.html
https://strands.readthedocs.io/en/latest/strands_3d_mapping/wiki/Home.html
https://strands.readthedocs.io/en/latest/strands_apps/door_pass.html
https://strands.readthedocs.io/en/latest/strands_apps/index.html
https://strands.readthedocs.io/en/latest/strands_apps/pose_extractor.html
https://strands.readthedocs.io/en/latest/strands_apps/ramp_climb.html
https://strands.readthedocs.io/en/latest/strands_apps/topological_roslaunch.html
https://strands.readthedocs.io/en/latest/strands_executive_behaviours/automated_routine.html
https://strands.readthedocs.io/en/latest/strands_exploration/index.html
https://strands.readthedocs.io/en/latest/strands_hri/index.html
https://strands.readthedocs.io/en/latest/strands_hri/strands_hri_launch.html
https://strands.readthedocs.io/en/latest/strands_hri/strands_simple_follow_me.html
https://strands.readthedocs.io/en/latest/strands_hri/strands_visualise_speech/pulse.html
https://strands.readthedocs.io/en/latest/strands_morse/bham/architecture.html
https://strands.readthedocs.io/en/latest/strands_morse/bham/real_TO-BE-MOVED.html
https://strands.readthedocs.io/en/latest/strands_morse/wiki/Home.html
https://strands.readthedocs.io/en/latest/strands_navigation/index.html
https://strands.readthedocs.io/en/latest/strands_navigation/joy_map_saver.html
https://strands.readthedocs.io/en/latest/strands_navigation/wiki/Home.html
https://strands.readthedocs.io/en/latest/strands_qsr/index.html
https://strands.readthedocs.io/en/latest/strands_qsr/wiki/Home.html
https://strands.readthedocs.io/en/latest/strands_qsr_lib/index.html
https://strands.readthedocs.io/en/latest/strands_recovery_behaviours/backoff_behaviour.html
https://strands.readthedocs.io/en/latest/strands_recovery_behaviours/index.html
https://strands.readthedocs.io/en/latest/strands_social/fake_camera_effects.html
https://strands.readthedocs.io/en/latest/strands_social/index.html
https://strands.readthedocs.io/en/latest/trajectory_behaviours/relational_learner/docs.html

Please exclude these for now.


How to realize pull to refresh and load more

hello, I have a question that is how to realize pull to refresh and load more, Does anyone know?

Note: you are more likely to get a response for general databinding questions on somewhere like StackOverflow


ImportError: cannot import name 'Etherscan'

I encountered this error when I tried to use the package.

Hi, could you please provide a code snippet that replicates the exact problem?

Hello, It's solved. Thanks for your reply. I appreciate you.

ImportError Traceback (most recent call last)
in ()
----> 1 from etherscan import Etherscan
ImportError: cannot import name 'Etherscan

Kindly make sure that the package is installed and up-to-date by running pip install etherscan-python==2.0.2


Add a lot of element helpers

Just an idea as adding all individual elements as tests seem trivial and bloated for what they do: Write one huge test which has all of the elements in one "test page" using a lot of linefeeds to make them on separate lines. So one can simply add them all into that test. When the test fails it could show the line number of the first non-matching line, which would lead to the problem quite reliably.

Just an idea as adding all individual elements as tests seem trivial and bloated for what they do: Write one huge test which has all of the elements in one "test page" using a lot of linefeeds to make them on separate lines. So one can simply add them all into that test. When the test fails it could show the line number of the first non-matching line, which would lead to the problem quite reliably. Probably too late :)

Yeah, you're probably right. Or something like table-driven tests. I'm usually not a fan of those, but here they might make sense.
That, or auto-generated tests. ;)


Now, I think the point is, with cutechess the order the games are finished doesn't affect the order in which the games are listed. The games are listed in the order they are started.

id prefer not doing this as i don't really see and advantage or use case and only downsides (if something goes wrong i.e. pc crash you only have a few pgns depending on when the games finish instead of all leading up to the pc crash)
and this would require some bookkeeping for pgns and saving them in memory to be written at a later point

I'll leave this open for now until this becomes an issue for someone relying on this behaviour

I can give an example where this would be useful.. I now test if all games have the same PlyCount between cutechess and fast-chess, but I can't easily do a 1-to-1 comparison since the order is different. Hence, I sort the PlyCount's before comparison. This might miss some bugs (i.e. two games have different PlyCount, but when sorted it is the same), even though it will probably be few.
I don't think it is critical though, more a nice to have.


Add Appointment Scheduling usecase

Purpose
This PR contains the following.

Front-end implementation of the appointment-scheduling
Ballerina medconnect service
README.md files

Are these images subject to copyrights? or generated?

Did you check copyrights for these images? or generated?

One image was generated. I removed the other image and added a generated image.


Fix FileProducer passing an empty slice as input

I found a bug where FileProducer will always return Input::Eof(Some(_)) even when the slice is empty.
Also, just to let you know, the travis test is reporting the build as passing even when the test stream::tests::seeking_consumer is failing because it can't find a file named testfile.txt.

I see, thanks! SO now, your flac parser uses the 1.0 beta, right? What issues did you encounter? Your repository triggered a few fixes in the beta, around error types, and a refactoring of bit level parsing.
What causes the hangs you refer to in your commit messages? Adding a function to grow the producer's buffer should be easy enough.

Haven't encountered that many, but the ones I've noticed are from Producers. Beside the one that got fix with this merge, the buffer size not growing in FileProducer is the other. And that is what is causing things to hang, functions using a FileProducer inside of it end up being stuck in a loop of the handle waiting for more than the current size of the buffer and the slice being passed is always to same size.

It makes sense. If you need anything else in the producers, please tell me. It looks like I won't release the 1.0 until next week, and I'd like to fix as many issues as possible in the meantime.


Struct field access fails if expression type is not yet concrete

I was playing around with the new type system and came across this issue:
>>> struct F { a: Current, b: Current }
>>> head([F { a: 1A, b: 2A }]).b
error: while type checking
  ┌─ <input:24>:1:28
  │
1 │ head([F { a: 1A, b: 2A }]).b
  │ -------------------------- ^ Can not access field 'b' of non struct type 'T185'
  │ │
  │ T185

My guess is there just needs to be a zonking step performed before looking up the struct type for field accesses?

Thank you for testing my implementation and reporting this!
This is something I completely missed. It might be a bit harder to fix. It's not related to lists. Here's an even simpler example that also fails:
fn id(x) = x
struct F { x: Scalar }
id(F { x: 1 }).x

The problem is with the polymorphic function call (head or id), because we create a fresh type variable for the instantiation of that function (e.g T0 -> T0 for id). We then add a T0 ~ F constraint matching the parameter type with the argument type, but as you pointed out, that will only get solved after we look up the type for the .x field access (the return type T0 in this case), leading to a *"Can not access field 'x' of non struct type 'T0'" error.

My guess is there just needs to be a zonking step performed before looking up the struct type for field accesses?

Hm. Are there general rules for when this is okay? We are currently solving constraints (and then zonking unification variables) only after we have looked at (and elaborated) the whole statement. We are not doing classical Hindley-Milner Algorithm W, where constraints are solved immediately when they are discovered. Instead, I tried to implement a strategy called "constraint typing", where constraints are first collected and only solved after looking at the full statement. I'm not a type theory expert, so I might have made some mistakes. But it is my understanding that this formulation is more suited (or even needed) for Numbat because (1) we need to support dimension types and (2) we want to support type class constraints.
From my understanding of the literature, the constraint typing approach is not strictly needed for (1), but Algorithm W needs some rather complex adaptations to support dimension types directly, while the formulation is more natural using constraint typing. And type class constraints (2), like the Dim constraint that we already have, also nicely fit into this formulation. And I think they can't be integrated into HM directly(?).
Something that doesn't fit well into the current type system are the nominally typed records that you implemented. Looking back at this, I was recently thinking if structurally typed records would have been the right call after all 😄. I think they fit better into functional programming languages, and they can fully support type inference (for example: PureScript has a powerful record system using row polymorphism). But nominally type structs still feel a bit more natural to me.
And I thought that we could get away with this by simply requiring type annotations for usage of structs. So something like fn f(x) = x.foo would simply not be supported. I thought this was a reasonable limitation. But this bug shows that I didn't think this through...
Maybe the solution would be an additional constraint? I quickly thought about this when I encountered a similar thing that you reported here... but then I found another solution for that case (whenever we have closed types, i.e. without any unification variables, we now try to pass them from the inside to the outside as far as possible. This didn't just solve my problem with struct field accesses, but also leads to much better error messages. Otherwise, we would only see constraint solver errors, which are horrible). The additional constraint could be something like HasField(Type, Field). The way this could work looks like this:
When we encounter a field access expr.x, we would not require direct knowledge about the type of expr (and force it to be of the right struct type). Instead, we would only generate a new HasField(expression-type, field-name, field-type) constraint that we could later solve. We wouldn't know the field-type just yet, so we would generate a fresh type variable, use that in the constraint, and return it as the type of the field-access expression. For the id(...).x example, we would generate the constraints
T0 ~ F
HasField(T0, "x", T1)

and return a type of T1. We would then solve the first constraint with a T0 := F substitution, turning the second constraint into
HasField(F, "x", T1)

Whenever the first argument in the HasField constraint has turned into a concrete type, we can attempt to solve it by looking up whether or not that type (F) has a field with name "x". If it does, we solve that constraint by substituting T1 with the looked-up field type.
If someone knows more about type systems then I do, I would really appreciate some help 😄. But right now, I'm thinking this might be a way out for this particular problem.

Oh, haha: https://ghc.gitlab.haskell.org/ghc/doc/users_guide/exts/hasfield.html
Need to read it in detail, but sounds very much related to the discussion here

HasField constraints are a good solution and I've used them often in haskell. But there's some considerations I can think of:

If we allow users to use HasField then we will need to support dictionary passing or monomorphisation in order to allow typing:fn foo<Y, X: HasField<"y", Y>>(x: X) -> Y = x.y

So that this function is able to access y it can either be passed an extra accessor function at runtime, monomorphised, or we can change the runtime representation of structs to Map<String, Value>.
However if we don't expose the HasField constraints, then I think it should always be possible to solve the HasField constraint into the concrete struct type.

Rust I think is a good example to look into here, there is no exposed HasField constraint, yet a field access always has to resolve a concrete type.

#461 implements a simple version of HasField constraints.

If we allow users to use HasField then [...]

Right. Since we don't have other types where field access would be useful (except for some of the builtin types like DateTime which are not customizable), I think I would keep that constraint internal for now.

However if we don't expose the HasField constraints, then I think it should always be possible to solve the HasField constraint into the concrete struct type.

:+1:

Rust I think is a good example to look into here, there is no exposed HasField constraint, yet a field access always has to resolve a concrete type.

I think this is what we have now with #461 as well.

This discussion here also made me realize that there is a similar problem with one another builtin type (DateTime), that sparks a whole discussion around multi-parameter type classes / traits with associated types, for which I opened a new ticket here: #462


Convert toggles into docsify tabs

Description
Changes proposed in this pull request:

Convert toggles into docsify tabs.
Use lowercase for 'custom resource'.

Related issue(s)
See also #18379

/retest

/retest


Enable FHIR Data Pipes to write FHIR resources directly to BigQuery using SQL-on-FHIR schema

A ticket (https://github.com/google/fhir-data-pipes/issues/455) already exists for this, hence closing this


33090 extend sensitive post parameter filtering

I created this branch to fix the issue of sensitive post parameter filtering not marking the request until the view is executed, that is, filtering cannot be applied to reports  generated by exceptions in the middleware for ticket. Carlton Gibson already suggested a solution which I worked on and also wrote a test for it.

Closing due to inactivity.


Polymorphic and Generic Support

Description
Adds support for sealed classes as well as generics
Closes #40
Closes #58
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[x] This change requires a documentation update

How Has This Been Tested?
Added several new tests, old tests still pass
Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have updated the CHANGELOG and bumped the version
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

update README


Hardware overheating issue

For more information
https://www.youtube.com/watch?v=dsa8JWZiugA&feature=youtu.be

done


Add modifiers, more special marks to cheatsheet

Closes #873, closes #636, partially addresses #667. Will check a few boxes in #840.

All the changes look good to me 👍


Fix the pendingCI message that is sent to keybase

I am only going to have the stale PRs run for now while troubleshooting the pending ci PRs


Consumer benchmark test for paused partitions

For details about this new Kafka Consumer benchmark test see Jira issue KAFKA-8814.  Original PR and Jira:

PR: KAFKA-7548: KafkaConsumer should not throw away already fetched data for paused partitions (v2)
Jira: KAFKA-7548

To recreate this test from the Jira issue:
# Run on trunk
TC_PATHS="tests/kafkatest/benchmarks/core/benchmark_test.py::Benchmark.test_consumer_throughput" bash tests/docker/run_tests.sh
# Rebase onto tag 2.3.0
git rebase --onto 2.3.0 trunk
# Run on 2.3.0
TC_PATHS="tests/kafkatest/benchmarks/core/benchmark_test.py::Benchmark.test_consumer_throughput" bash tests/docker/run_tests.sh

@ijuma @hachikuji Please review at your convenience.

@ijuma I don't see benchmark related test results in the PR-triggered Jenkins build.  Is there a benchmark build that can be run with this branch?

@hachikuji Thanks for the reply.  When I first started exploring the way to benchmark this work I had some reservations about modifying the consumer performance tool as well.  It makes sense that the existing benchmarks use this tool, but it does place limits on the types of consumer scenarios that can be tested.
There does seem to be precedent to modify the tools for system testing.  Some of the apps in org.apache.kafka.tools appear to exist just for this purpose (VerifiableConsumer, VerifiableLog4jAppender, VerifiableProducer).  In TransactionalMessageCopier there's an argument called --enable-random-aborts which is only used for testing:

Whether or not to enable random transaction aborts (for system testing)

I like your idea about testing how the partition pauses affect Kafka Streams, but I'm not very familiar with the use case or if this fix has much impact for it.  I can speak to the Alpakka Kafka project.  The consumer Source always polls on a set interval, but would pause partitions when there was no demand for records downstream.  We would poll anyway to handle any offset commit acknowledgements that might be outstanding, but this would cause the consumer to throw away data pre-fetched data.  Also, IIRC the original issue was reported by LinkedIn WRT how Samza pauses partitions, but I'm not familiar with this use case either.  I think there's value in demonstrating the performance gain with a low level test like this one because it's easy to understand, but I agree that maybe it should avoid modifying the consumer performance tool.
Perhaps I could modify VerifiableConsumer instead to support this use case since it's only used for system testing?  I could also create a new tool.

I looked at Kafka Streams partition pausing use cases, but I'm not sure how to use Kafka Streams in a way that would trigger lots of partition pause/resumes to demonstrate the issue like I have in this PR, or with external projects that use the KafkaConsumer.  @mjsax @guozhangwang Do you have any ideas on how to structure a Kafka Streams perf test that would demonstrate the performance improvement from #6988 ?
I looked at org.apache.kafka.tools.VerifiableConsumer.  It could be modified to support partition pausing like I've done with ConsumerPerformance, but it doesn't feel like an appropriate place to add it since it is generally used to assert consumer state rather than performance.
I considered making a copy ConsumerPerformance and stripping it down to only support partition pausing so that it's not exposed to end users through kafka-consumer-perf-test.sh, but this wouldn't be a very DRY implementation.
I think there is precedent for modifying the public-facing perf tools for system tests, as I mentioned in this comment: https://github.com/apache/kafka/pull/7221#issuecomment-523057390
@ijuma Do you have any suggestions?


static file documented as endpoint

I have a static route as follows
app.static('/robots.txt', './static/robots.txt')

It gets documented with swagger and there is no way to set @doc.exclude(True) for it.

Yes, I think it is better not to document those static file


illogical oneOf detection missing

Describe the bug
I found an impossible oneOf definition.
I'm constantly trying to educate people about oneOf vs anyOf. In any case, it would be better if the lint tool could catch impossible definitions.
To Reproduce
Most simple reproduction:
oneOf:
  - type: 'object'
  - type: 'object'

Here is the actual place I spotted it:

Given this https://github.com/Rebilly/api-definitions/blob/main/openapi/components/schemas/Subscription.yaml#L119-L132 with a oneOf: $ref to InvoiceTimeShift object, null.
The problem is that the InvoiceTimeShift object itself has a type of object or null.
See error (that there are no lint catch of this problem).

Given that Rebilly is likely to change that before we finish this, here is how to reproduce it.
Excerpt from a schema:
  invoiceTimeShift:
    description: reduced to shorten example
    example: null
    oneOf:
      - $ref: ./InvoiceTimeShift.yaml
      - type: 'null'

Now, let's look at InvoiceTimeShift.yaml (only the first couple of lines needed):
type:
  - 'object'
  - 'null'

Why is this impossible? oneOf needs to match one and only only option. In this case, InvoiceTimeShift can be null, and the second option is also null (both the same). There is no way to distinguish which one was intended. Therefore, this is an impossible oneOf.
Expected behavior
I expected to be alerted to an incorrect oneOf definition. oneOf requires each item must be able to be distinguished from the other items (as opposed to anyOf).
Logs
n/a
OpenAPI description
See link above
Redocly Version(s)
latest (n/a)
Node.js Version(s)
n/a
Additional context
Noticed while debugging something unrelated.

no-ambiguous-schemas or no-ambiguous-oneOf could be a potential name for the rule.

Thanks for reporting! This looks more like an enhancement though.

@adamaltman a question based on the next example from the link you provided:
{
  "oneOf": [
    { "type": "number", "multipleOf": 5 },
    { "type": "number", "multipleOf": 3 }
  ]
}

Wouldn't it be also considered an illogical use of oneOf?  The number 15 will be technically valid against both of the subschemas. Is there a way to distinguish an intentional use of oneOf to exclude certain numbers from an example of illogical use?

I don't know what you mean by the intentional use of oneOf to exclude certain numbers?

The same example - all multiples of 3 and 5 will be valid against that oneOf except for multiples of 3 AND 5 at the same time. (e.g. 3, 5 - valid, 15 - invalid). So technically a similar  trick could be used intentionally.

I'm not sure if there are many of this multiple of 3 and 5 type cases in the real world

Most likely not too much. And the vast majority should be using anyOf instead. But how about the following:
oneOf:
- type: object
  properties:
    foo: { type: string }
- type: object
  properties:
    bar: { type: number }

Technically any object is invalid against this schema since additionalProperties are true by default and any field will be valid against both schemas. However, I suppose it's pretty common to write schemas like that. Do we want to error on such cases as well?

@tatomyr I think the schema is written incorrectly for "I want 15 to be invalid" and the user would need to compose of more complex objects with not multiple of 15. So, I do believe it is still illogical. I'm not sure we need to check for all aspects of illogicality.
Your example with additionalProperties is a common illogical situation. I believe that I would want to know it is illogical, but there are likely people who would like to assume additionalProperties was false even though it isn't false.
I think the rule should take into consideration required properties. Sometimes a missing required property can make the case logical even without additionalProperties set to false.

Makes sense. Thanks for the explanation!

  It would be amazing!!  Certainly, there are a million configurable rules you can create for these types of checks, similar to #1318
There has never really existed a good JSON Schema linter, which is something the JSON Schema core group has discussed many times, but without the budget to produce something. I would definitely consider discussing at least some of the bigger schema authoring mistakes they encounter to try and build some rules around them.


Change how top row works

Try to rework top row to work with a service class and events to update the top row contents instead of the cascading value.
The cascading value requires prerender to be turned on (as far as I know right now) in every view that wants to access the top row which causes it to load data twice for those views.

fixed with #6


lease should not be added to the content when using client.write()

Hey,
There is no reason the lease value goes into the content of the secret
>client.write('secret/test/cyril/item1', username='u1', password='p1', cont='blabla', lease='2h')
>client.read('secret/test/cyril/item1')['data']
{u'cont': u'blabla',
 u'lease': u'2h',
 u'password': u'p1',
 u'username': u'u1'}

It's because you use kwargs as the json dictionary input when you pass send it to the API

This seems kinda related to #133 in that it is a side effect of kwargs...

right, same cause, different consequences

Going to track all variants of this woe in the other issue...


Fix for issue #48

Coverage decreased (-0.4%) to 92.778% when pulling 8caa2e25e2af97249535d21ac4fce86fff8156af on edgarmueller:master into 8f542cb91ca6bd71852e3ae6f676c95facca6479 on eclipsesource:master.

Coverage increased (+0.3%) to 93.525% when pulling 338ba134ec54d128d0200af1977f5e345463e960 on edgarmueller:master into 8f542cb91ca6bd71852e3ae6f676c95facca6479 on eclipsesource:master.


Cli argument -lvs is broken

Description
Trying to only run LVS on a design that was previously hardened breaks the flow. command used:
./flow.tcl -lvs -design APU -gds designs/APU/runs/RUN_2022.12.19_10.20.21/results/signoff/APU.gds -net designs/APU/runs/RUN_2022.12.19_10.20.21/tmp/signoff/29-APU.pnl.v

Expected Behavior
Run LVS on gds vs netlist
Environment report
Kernel: Linux v5.15.0-52-generic
Distribution: ubuntu 20.04
Python: v3.8.10 (OK)
Container Engine: docker v20.10.21 (OK)
OpenLane Git Version: improve-yosys-naming-dev
pip: INSTALLED
python-venv: INSTALLED
---
PDK Version Verification Status: OK
---
Git Log (Last 3 Commits)

78015a1 2022-12-14T21:12:51+08:00 scripts: yosys: generate readable names for instances - Sean Cross -  (HEAD -> improve-yosys-naming)
90d369b 2022-12-13T22:59:28+02:00 Iterate over `$libs` in read_libs (#1570) - Kareem Farid -  (tag: 2022.12.14)
0d13487 2022-12-13T19:05:19+02:00 Add `report_parasitic_annotation` to sta (#1569) - Kareem Farid -  ()
---
Git Remotes

origin	git@github.com:The-OpenROAD-Project/OpenLane.git (fetch)
origin	git@github.com:The-OpenROAD-Project/OpenLane.git (push)

Reproduction material
./flow.tcl -lvs -design APU -gds designs/APU/runs/RUN_2022.12.19_10.20.21/results/signoff/APU.gds -net designs/APU/runs/RUN_2022.12.19_10.20.21/tmp/signoff/29-APU.pnl.v

Relevant log output
[ERROR]: during executing: "openroad -exit -no_init -python /openlane/scripts/odbpy/power_utils.py write_powered_def --output /openlane/designs/APU/runs/RUN_2022.12.19_10.36.55/tmp/signoff/1-APU.p.def --input-lef /openlane/designs/APU/runs/RUN_2022.12.19_10.36.55/tmp/merged.nom.lef --power-port VDD --ground-port VSS --powered-netlist {} 0 |& tee /dev/null /openlane/designs/APU/runs/RUN_2022.12.19_10.36.55/logs/signoff/2-write_powered_def.log"
[ERROR]: Exit code: 1
[ERROR]: Last 10 lines:
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/openlane/scripts/odbpy/power_utils.py", line 84, in write_powered_def
    print(f"Top-level design name: {reader.name}")
AttributeError: 'OdbReader' object has no attribute 'name'
child process exited abnormally

@marwaneltoukhy The flow.tcl script uses the <design>/config.tcl (or maybe <design>/config.json) file to retrieve many of the parameters for LVS. It was not designed for standalone LVS of gds and verilog outside of the openlane flow.
Is there an APU/config.* file with the correct parameters?

@d-m-bailey Yes it does, this is a design in openlane. The documentation for this option can be found here

@marwaneltoukhy I believe I'm the one that originally added the lvs subroutine. There were changes later.
The error messages look like it's trying to add the power nets which it will attempt to do if LVS_INSERT_POWER_PINS is set. If your verilog already has the power nets included, you want to set this to 0 or delete it (maybe).
See the run_lvs routine of scripts/tcl_commands/lvs.tcl