discord-screenaudio: init at 1.8.0

Description of changes
Added discord-screenaudio package, a 3rd part discord client that allows screen sharing with audio on Linux.
Currently, does not build, so I'm creating a draft while I try to get it working.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Currently fails build with:
/build/discord-screenaudio-9faabe1/submodules/rohrkabel/src/metadata.cpp:7:10: fatal error: pipewire/extensions/metadata.h: No such file or directory
    7 | #include <pipewire/extensions/metadata.h>

As far as I can tell this file should be present with the Pipewire package, not sure what I'm missing.

Currently fails build with:
/build/discord-screenaudio-9faabe1/submodules/rohrkabel/src/metadata.cpp:7:10: fatal error: pipewire/extensions/metadata.h: No such file or directory
    7 | #include <pipewire/extensions/metadata.h>

As far as I can tell this file should be present with the Pipewire package, not sure what I'm missing.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/fail-to-import-pipewire-metadata-h-when-building-package/23000/1

Builds but fails to stream with [userscript] Did not find 'discord-screenaudio-virtmic', trying again in 100ms

discord-screenaudio-virtmic is visible in Helvum.

I've updated this to 1.7.1 with a personal derivation, seems to build fine, but I'm getting this error after starting a screenshare:
[userscript] Did not find 'discord-screenaudio-virtmic', trying again in 100ms

There seem to be similar issues reported on their github but I can't seem to find one that works for me, is there any NixOS issue here?

Yes it is an issue with this package right now. It is for some reason unable to see available sources and devices. You are unable to change your default input and output and the streaming fails with this error. I haven't figured out why yet. Probably a missing dependency.

just tried building this on latest version, same issue with audio devices, logs flooded with [userscript] Did not find 'discord-screenaudio-virtmic', trying again in 100ms when streaming. would you have any idea if there is a missing dependency or something @maltejur?

It seems to differ from distro to distro depending on how QtWebEngine/Chromium is compiled, but I think sometimes pipewire-pulse is still needed. If that doesn't work let me know, I could try to spin up a nixos vm and maybe find another fix.

Hmmm, perhaps I'm using it wrong, but I've added pipewire.pulse to buildInputs on my personal derivation, and it still doesn't seem to work.

I've bumped the build to the most recent version and rebased on master.

As an alternative, I got xwaylandvideobridge working. It allows you to stream audio if you run it before starting your screen share. Unsure if it works for X11, but did for me on Wayland. I packaged it here in case anyone wants to try it.

Going to close this, not working on it anymore.


Actions fail with Unable to resolve action failure

Hi,
We have seen since last Friday that our github actions started to fail with the following error:
Error: Unable to resolve action `snapcore/action-build@v1`, unable to find version `v1`

Is there something that we can do to resolve this issue?
Here is also our github action config:
name: Build and release arm64 snap

on:
  release:
    types:
      - released

env:
  SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}

jobs:
  build:
    runs-on: raspberry-pi-arm64
    outputs:
      snap-file: ${{ steps.build.outputs.snap }}
    steps:
      - uses: shimataro/ssh-key-action@v2
        with:
          key: ${{ secrets.SSH }}
          name: id_rsa
          known_hosts: ${{ secrets.KNOWN_HOSTS }}

      - uses: actions/checkout@v4

      - name: 🚢 Build arm64 snap
        uses: snapcore/action-build@v1
        id: build
        with:
          snapcraft-args: |
            --build-for=arm64
            --bind-ssh

      - uses: actions/upload-artifact@v4
        with:
          name: build_output
          path: ${{ steps.build.outputs.snap }}

  release:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - uses: actions/download-artifact@v4
        id: download
        with:
          name: build_output
          path: build_output/

      - name: Get File
        id: get_file
        run: |
          echo "download-path=$(find build_output/ -name '*.snap' -type f)" >> $GITHUB_STATE

      - uses: snapcore/action-publish@v1
        env:
          SNAPCRAFT_STORE_CREDENTIALS: ${{ secrets.STORE_LOGIN }}
        with:
          snap: ${{ steps.get_file.outputs.download-path }}
          release: edge

  sent_sentry:
    runs-on: ubuntu-latest
    needs: release
    steps:
      - uses: shimataro/ssh-key-action@v2
        with:
          key: ${{ secrets.SSH }}
          name: id_rsa
          known_hosts: ${{ secrets.KNOWN_HOSTS }}

      - uses: actions/checkout@v4

      # Setup Flutter
      - uses: kuhnroyal/flutter-fvm-config-action@v2
        id: fvm-config-action
      - uses: subosito/flutter-action@v2
        with:
          flutter-version: ${{ steps.fvm-config-action.outputs.FLUTTER_VERSION }}
          channel: ${{ steps.fvm-config-action.outputs.FLUTTER_CHANNEL }}
          cache: true

      - name: "Add env file to all projects"
        run: mkdir -p assets && touch assets/.env
      - name: "Install dependencies"
        run: flutter pub get
      - name: Upload source to Sentry
        run: |
          set -e # fail if any command fails
          VERSION=`grep 'version:' pubspec.yaml | awk '{print $2}'`
          SENTRY_RELEASE=${VERSION}
          echo "Uploading source to Sentry using SENTRY_RELEASE=$SENTRY_RELEASE"
          flutter packages pub run sentry_dart_plugin

Same here.

Same here, should we replace all snapcore/action-build@v1 by canonical/action-build@v1?

+1

Using
        uses: canonical/action-build@v1

instead of
        uses: snapcore/action-build@v1

fixes the issue.


Let's work on removing nodeunit completely

Okay, I'm done. I'd probably squash all the migration commits and just leave the infrastructure changes extra.

Oh! Remind me: Will need to enable browser tests on-branch before merging because those are completely untested right now.

Aaand browser test pass too 🎉


[Enhancement][UI] Add URL to Problem when a ressource having an Problem

Is your feature request related to a problem? Please describe.
One of my Resources is having an problem. In the Cluster-Dashboard is an red indicator.
Describe the solution you'd like
It would be nice, if the red indicator is a URL which brings me to the related resource

.

This Featurerequest is still relevant

This Featurerequest is still relevant


Grab project logs from start of file and not end with the fluentd collector

What kind of request is this (question/bug/enhancement/feature request):
Feature Request/Enhancement
Steps to reproduce (least amount of steps as possible):

Setup Project Logging via Rancher to remote collector.  Doesn't matter type.
Set flush interval to 1 second.
Deploy a workflow that generates lots of output ( I used my test image rayisbadat/infloop )

Result:
There is a delay before the fluentd log collector starts forwarding logs.  Delay ranges from 10-60 seconds, although had some outliers that went two minutes.   This causes us to lose the boot output, which we think will be critical to roll out and production.
Other details that may be helpful:
Adding two settings into auto generated /opt/rke/var/lib/kubelet/pods/UUID/volumes/kubernetes.io~secret/config/project.conf  will fix this problem in the auto generated  fluentd project.conf file.
skip_refresh_on_startup true Speeds up the collection and forwarding
read_from_head true Causes fluentd collector to scrape entire file and not start with the end of file.
PR: https://github.com/rancher/rancher/pull/25049
Timing from some simple tests (time in seconds)

setting\Run #
1
2
3
4
5
min
avg
mean
max

default
58.203
30.385
23.806
37.457
13.32
13.32
32.6342
30.385
58.203

skip_refresh_on_startup true
48.933
39.976
10.573
43.578
38.641
10.573
36.3402
39.976
48.933

read_from_head true
55.677
58.089
56.444
12.87
30.588
12.87
42.7336
55.677
58.089

Both
15.166
19.32
26.926
33.81
34.274
15.166
25.8992
26.926
34.274

Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI):v2.3.4
Installation option (single install/HA): Single install

Cluster information

Cluster type (Hosted/Infrastructure Provider/Custom/Imported): Custom(Self Hoststed)

Machine type (cloud/VM/metal) and specifications (CPU/memory): VM (Virtual Box), 2 Cores, 4GB of Ram

Kubernetes version (use kubectl version): v1.17.0

Docker version (use docker version):

Client: Docker Engine - Community
 Version:           19.03.4
 API version:       1.40
 Go version:        go1.12.10
 Git commit:        9013bf583a
 Built:             Fri Oct 18 15:49:05 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Rancher version: 2.4 master-head (02/28/2020) commit id: e59944099

tailed the external fluentd docker container aggregator logs to monitor the logs.
Enabled project logging in the Rancher System project with an external fluentd aggregator.
Checked a couple System project containers logs to verify the start of file logs
Exported the fluentd docker container aggregator logs to a local file
Compared the logs in the fluentd aggregator local file with the System project containers logs

Now the logs are getting sent and aggregated by the external fluentd from the beginning of the logs in the rancher containers.

Further testing was done today in Rancher version: 2.4 master-head (03/02/2020) commit id: fa96c0c3d

Setup the external Fluentd aggregator to use @type file  to redirect the logs to a log file
Deploy the provided rayisbadat/infloop image for generating logs.
Setup the project level logging to start collecting the logs and send to the external Fluentd.
Monitored the logs size until I got more than 50 Mb in file size.
Deleted the infloop workload from rancher
Started analizing the logfile.

I was able to see the start of the logs of the infloop image {"log":"Output> Iteration: 1... even after it got very big in size. Validating that the logs is collected from start of file and not tailing the log.

I was also able to reproduce this issue in Rancher v2.3.5  instead of starting on log iteration 1 as my previous test in 2.4 we only got the part of the logs in the aggregator the first one being: {"log":"Output> Iteration: 73777...

Upgrading from Rancher v2.3.5 to latest 2.4 master-head (03/03/2020) commit id: c921ba250 I see the new configuration getting applied and working as expected.

@izaac can you test this in 2.3-hea

Rancher version: v2.3-head (03/13/2020) commit id: cdd0acc06
The issue was previously reproduced in 2.3.5.
I performed the following test scenarios with positive results:

Deploy the provided rayisbadat/infloop image for generating logs.
Setup the external Fluentd aggregator to use @type file to redirect the logs to a log file
Setup the project level logging to start collecting the logs and send to the external Fluentd.
Monitored the logs size until I got more than 50 Mb in file size.
Deleted the infloop workload from rancher
Started analyzing the log file.

Upgrade was made from v2.2.9 to v2.3-head I see the new configuration getting applied and working as expected.


$setValidationErrors not updating view

great package, im just trying to reduce duplication in my code right now, and currently i have 2 sets of errors in the view.

the errors returned via a post request when the forms submitted
the errors provided by this package.

naturally im trying to reduce this down to one master "error in errors" template to prevent this:
<template v-for="error in errors.email">
    @{{ error }}
</template>
<template v-for="error in $login.email.errors">
    @{{ error.message }}
</template>

and instead just have:
<template v-for="error in $login.email.errors">
    @{{ error.message }}
</template>

in the promise returned from server ive added the $setValidationErrors() function call as described in the docs:
//map server provided errors to the validation object (our server returns a simple array of messages so we need to format it correctly for the validator)
var errors = [];
response.data.errors.email.forEach(function(error){
    errors.push({field: 'email', message: error});
});
this.$setValidationErrors(errors);

//this is the data im trying to eliminate from the view and how they are currently sent back
this.errors.email = response.data.errors.email || [];

only problem is this isnt updating the view with the errors?
done a bit of debuging and it seems the data i pass IS being added to the correct locations (under errors, and under each fields errors object which can be seen by adding this to the template:
{{$login | json}}
so what am i doing wrong?
i can see from the debugging the pristine, modified, etc variables all seem to be set as if the validator hasnt tried to perfom validation yet after the ajax request. i suspect the validator resets itself after passing and adding the errors at this point simply adds them as data and not passes them to the view?
any help would be gratefully recieved, i love this plugin and hope it can work in this way.

Thank you for reporting!!
Can you provide the minimum reproduction code with jsfiddle please ?

Ive created a fiddle, however its not quite behaving the same as my actual code, as im getting errors even before the problem im having.
the fiddle is setup to simply add errors via the this.$setValidationErrors(); call, but strangly jsfiddle is reporting that function is undefined???
also the "pre" response validator errors arent even showing up in the dom, which IS working on my project.
I can tell the validation is happening as the login button is disabled until a valid email and min 6 char password is added, and the errors can be seen using {{ $login | json }}.
im more of a php guy so cant be sure if the jsfiddle problems are because of me, or what.
It shows what im trying to do but could be confusing.
in jsfiddle = $setValidationErrors is failing (works on my project)
in my project $setValidationErrors is not failing, but the new error messages arent being reported to the dom.
does that make sense?
https://jsfiddle.net/2p2tyjLd/26/

Thank you for your jsfiddle code providing !!
I checked your code.

the fiddle is setup to simply add errors via the this.$setValidationErrors(); call, but strangly jsfiddle is reporting that function is undefined???

Hmm ...
Jsdeliver is strange ...
https://cdn.jsdelivr.net/vue.validator/2.0.0-alpha.21/vue-validator.min.js
jsdelivr is delivering the v2.0.0-alpha.20 as it should be v2.0.0-alpha.21 ...
I forked your jsfiddle, and I fixed.
https://jsfiddle.net/kazupon/rv83j7Lt/4/

only problem is this isnt updating the view with the errors?

I also confirmed the your reporting issues at the above forked jsfiddle code.
I think that the behavior of matrial.js is affecting the vue-validator.
By the inspect element of brower debugging tools using, You can check the error message that is rendered with v-for.
(When you remove the mdl-textfield__error, You can confirm it)
To avoid this issues, I found the error message is rendered correctly by using validator-errors component.
https://github.com/vuejs/vue-validator#error-message-enumeration-component
I recommend the it use.

appologies, yes it was some stylign within the css hiding the errors, thanks for your help.


Get-HuduCompanies returns a company object when Id is specified

Cmdlet: Get-HuduCompanies
Issue
At least as of Hudu V2.24, the Hudu API returns a companies object at the root GET /companies/{id} call.  This is not illustrated in the API docs.
This results in an inconsistent response from Get-HuduCompanies depending if you're calling by ID, name, or for all companies. A small change on Line 57 of Get-HuduCompanies.ps1 would resolve this inconsistency for both cases:
Current
$Company = Invoke-HuduRequest -Method get -Resource "/api/v1/companies/$Id"
return $Company

Potential Fix
$Company = Invoke-HuduRequest -Method get -Resource "/api/v1/companies/$Id"
if ($Company | Get-Member -Name company) { return $Company.company } else { return $Company }

This has been fixed and will be live in the next version.


Created generic coordinator for cri

Relates to #303

Created an interface "Coordinator" (in cri/coordinator/types.go), that coordinators need to implement in order to integrate with the vhive-cri.
Renamed and restructured firecracker-coordinator, which implements the coordinator-interface
Changed service, so it works on the coordinator interface (takes a specific coordinator-type when creating)
Commented out some lines on cri/container_create.go, which I think don't have an effect (besides maybe for testing). If there needed, we would have to add the function "InsertActive" to the Coordinator interface.

All tests (with make test-cri) are successful.

practical advice: avoid changing the "main" branch in your forks, keep main in sync with upstream and use per-feature branches

I don't understand why the vhive-cri-tests are timing out when creating a service. It is running fine on my server-instance (xl170). Is there maybe a known issue with that?

I don't understand why the vhive-cri-tests are timing out when creating a service. It is running fine on my server-instance (xl170). Is there maybe a known issue with that?

there are sporadic issues that pop up in the CI. I'll be restarting such tests

please sign off your commits to fix the linter problem (git commit -s <..>)

I added all your requested changes, and test-cri still runs through.
So if it still doesn't run through in the CI, I'll have a look at the logs of the CI-tests.

@plamenmpetrov can you please take a look at this refactoring and let us know what you think?

@nathanieltornow the PR looks good. let’s wait a bit for another approval before merging but you can go ahead with your other changes.

Im so sorry, this was a mistake, I'm fixing it right now

Hello, thank you for your interest in the project!
What should the interface look like?
It could be beneficial to have an interface for the coordinator. However, it could be better to go one abstraction level higher and have an interface as follows which different sandboxing coordinators can implement:
interface {
CreateContainer(ctx context.Context, r *criapi.CreateContainerRequest) (*criapi.CreateContainerResponse, error)
RemoveContainer(ctx context.Context, r *criapi.RemoveContainerRequest) (*criapi.RemoveContainerResponse, error)
}

To understand this better from my perspective: The coordinator is tightly coupled with firecracker. I am not confident to foresee how much of the coordinator interface that you propose will translate well for gVisor or other sandboxing technologies. But perhaps this is not important. What really matters is the implementation of CreateContainer and RemoveContainer (currently inside container_create.go and container_remove.go.
With the interface I propose, container_create.go will be look more or less as follows:
func (s *Service) CreateContainer(ctx context.Context, r *criapi.CreateContainerRequest) (*criapi.CreateContainerResponse, error) {
    return s.sandboxingtechnology.CreateContainer(ctx, r)
}

It will be analogous for container_remove. Then, for each sandboxing technology we can implement CreateContainer and RemoveContainer, which may or may not need to involve a coordinator.
How should the code be strcutured if we support multiple sandboxing technologies?
The firecracker coordinator relies on the firecracker-containerd orchestrator (ctriface.Orchestrator). In the same way, the gVisor coordinator will have to rely on some sort of gVisor ochestrator. If we decide to place all coordinators in the cri package, where do we implement the gVisor orchestrator? Will it be in vhive/gvisoriface? What about if we add additional virtualization technologies? It might make more sense to have 1 package for firecracker, 1 for gVisor, 1 for any other new supported we decide to support. All of these will have their own ctriface, own coordinator, etc. Then, in vhive.go you could specify whether you want to use firecracker-containerd, gVisor, or anything else.
Some more concrete comments:

Currently, it looks like the cri/coordinator/gvisor/coordinator.go file contains a dummy implementation. I see from #303 that the plan is to implement an orchestrator for gVisor as well, so could we leave the creation of this file and the gVisor package as a whole for that future PR?
The cri_test.go file is in the firecracker coordinator package. Apart from TestMain (which can be moved to coordinator_test.go, the tests in cri_test.go should not be specific to a particular coordinator implementation (or orchestrator implementation for that matter). Could you move it outside of the firecracker coordinator package?

Hi @plamenmpetrov, thank you for your comments.
Regarding "What should the interface look like"
Yes, I think your suggestion would also be a great approach.
The only problem I see there is that you might have to change kubernetes/knative configurations when switching sandboxing techniques. In my understanding, the current cri will create either QP-containers or VMs, depending on the call.
So the benefit of doing the abstraction at the coordinator-level would be, that the functionality of the QP-container stays the exactly same,  and on StartSandbox() just one sandbox (either gvisor or fc) will be created, returning an IP-address to the sandbox, that the QP uses to reach the sandbox.
Regarding "How should the code be structured"
I'm open to the overall structure of the code. I think having completely separate packages for each sandboxing technique would be definitely the cleanest approach.
Regarding your concrete comments
Yes, I will make those changes.

Thanks a lot, folks! I appreciate the discussion.
I agree with @plamenmpetrov that the interface approach is desirable. I also agree that we should move the tests to the cri/ folder so that we can just fire the same tests with a different runtime option.
Some clarification for gVisor integration, compared to Firecracker. We are going to follow the same dual-containerd setup, as before, with one for function instance sandboxes and one for regular containers.
gVisor is going to use a stock containerd with a gVisor plugin. Hence, to support gVisor, we just need to set up an extra containerd daemon and a CRI coordinator to forward all commands from k8s to that containerd-with-the-plugin. For example, the cri module doesn't need to implement the IP addresses management. However, we would like to have a possibility to have some sandbox management inside the CRI coordinator (e.g., cgroups limits).
@nathanieltornow could you please implement the interface-based architecture? also, could you please move the tests a level up under the cri/ folder? it would also require adding a runtime argument for the tests to either run the Firecracker and the gVisor coordinators?
Regarding the dummy gVisor code, I would leave it as is. We are not going to merge the code into main before gVisor support is completely ready. Instead, we are going to push the code to a feature/gvisor branch (as a copy of Nathaniel's branch) and open PRs against it. I am going to give @nathanieltornow write permissions to push his branch directly into upstream. @nathanieltornow please push your current branch as a feature/gvisor branch to upstream.

@nathanieltornow Thank you for taking care of the small changes so quickly. Yes, separate packages for each runtime, e.g. firecracker-containerd, gVisor makes the most sense to me as well.
@ustiugov Thanks for clarifying. It seems altogether that the gVisor support should not present major issues.
As far as the coordinator/interface question is concerned, each runtime (firecracker-containerd, gVisor, containerd, etc.) needs to implement the CRI interface to work with Kubernetes. In vHive right now, the service struct does that for firecracker-containerd. It proxies most CRI calls to stock containerd and intercepts CreateContainer and RemoveContainer.
It is possible that for other runtimes, like gVisor, a different implementation of CRI is more suitable. For example, we might not want to make the distinction for queue-proxy and user-container. (@ustiugov , I know you said you want to do that, but I am giving an example. As a result, implementing the interface at the coordinator level might be restrictive for some other runtimes. For this reason, the right interface might be actually CRI. This will be the most liberal approach. To see this, for each runtime there will be a service (like the one for firecracker-containerd), which will implement CRI. Then, the CRI implementation for the runtime can afford to make changes to other CRI calls, not only CreateContainer and RemoveContainer.
Let me know what you think.

Thanks for all your comments!
@ustiugov I created the feature/gvisor branch with the current code of this PR and will make my next PRs to this branch.
I implemented an interface as suggested by @ustiugov and pushed it to nathanieltornow/feature/gvisor, and I think it is a clearer approach to before.

Interface is in cri/types.go
Firecracker implementation in cri/firecracker/service.go

@plamenmpetrov I agree, introducing the interface at the CRI-level might be the best solution, since it introduces the most separation between unrelated components. The only concern I'm having with that solution is, that you won't be able to change between different sandboxing-techniques with setting a flag to ./vhive, but maybe also have to change knative-settings as well.

@plamenmpetrov I think that we should keep the dual-containerd architecture and build upon it. Proxying CRI requests gives us more control that might be useful in future, and running everything in gVisor might lead to problems with regular stateful services. Keeping only functions in gVisor sandboxes would be more localized and easier to maintain.

please re-open this PR from feature/gvisor

@ustiugov It is true that it might not work out of the box, but having the possibility to do it in the future without other changes is probably desirable. Anyways, I am not arguing strongly.
@nathanieltornow Having the interface at the CRI level will not impact vhive/vhive.go. Depending on the flag (fccd, gVisor, etc.), we can create the respective service, which implements CRI.


Sort unique cameras/types/optics to generate stable indices in hdf5

Possible solution for #2063
However, I am not sure we really want to introduce this requirement

I don't think it should be a requirement (or relied on), but it is nice just from a user-readablity standpoint that the order doesn't vary unexpectedly.  However, I still wonder why it would for files from the same prod and subarray...

@kosack see my comment here: https://github.com/cta-observatory/ctapipe/issues/2063#issuecomment-1238105227


Feature/show item paginated list

[x] I have tested my component
[x] I have updated the documentation or there is no documentation needed

:tada: This PR is included in version 2.0.0-rc.15 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Improve the footer .

What feature?
@Vaishnavi-Patil2211  , I would like the improve the footer of the BIT page given below.So,please assign this issue to me.
Show us the magic with screenshots

Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSoC'23 contributor
[X] I want to work on this issue

@Vaishnavi-Patil2211 ,please assign this issue to me.


refactor(sumologicexporter): use golang.org/x/exp/slices for sorting fields

This PR uses generic slice sorting algorithm and saves yet another allocation in fields.string().
Related article: https://eli.thegreenplace.net/2022/faster-sorting-with-go-generics/
benchcmp old.txt new.txt
benchcmp is deprecated in favor of benchstat: https://pkg.go.dev/golang.org/x/perf/cmd/benchstat
benchmark              old ns/op     new ns/op     delta
BenchmarkFields-16     646           618           -4.32%
BenchmarkFields-16     642           603           -6.03%
BenchmarkFields-16     636           614           -3.35%
BenchmarkFields-16     640           616           -3.72%
BenchmarkFields-16     648           612           -5.56%

benchmark              old allocs     new allocs     delta
BenchmarkFields-16     11             10             -9.09%
BenchmarkFields-16     11             10             -9.09%
BenchmarkFields-16     11             10             -9.09%
BenchmarkFields-16     11             10             -9.09%
BenchmarkFields-16     11             10             -9.09%

benchmark              old bytes     new bytes     delta
BenchmarkFields-16     368           344           -6.52%
BenchmarkFields-16     368           344           -6.52%
BenchmarkFields-16     368           344           -6.52%
BenchmarkFields-16     368           344           -6.52%
BenchmarkFields-16     368           344           -6.52%

EDIT: the linter fails because we don't have full generics support in golangci-lint just yet: https://github.com/golangci/golangci-lint/issues/2649

I'm not sure if I'm reading this right, but from https://github.com/golangci/golangci-lint/issues/2649 it sounds like the linter should work if we explicitly set go version to 1.18 in the config? The issue description claims that they automatically disable linters which can't handle generics for go 1.18. @pmalek-sumo ?


I am unable to do an In Place Upgrade from 2016 to 2019

When I mount the iso and execute the setup I am unable to select the keep personal files and app option. It is grayed out.
This is the iso I am using for the 2019 Version:
en_windows_server_2019_updated_april_2020_x64_dvd_12d6dc63.iso
The Windows Server 2016 Version that is currently installed is: Windows Server 2016 Standard 10.0.14393 Build 14393 RETAIL Channel

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 505fe07b-6fdd-4887-3c8b-15405c5d8308
Version Independent ID: 25645df2-0ffb-4cc3-a8d2-5466edd309fb
Content: Overview about Windows Server upgrades
Content Source: WindowsServerDocs/upgrade/upgrade-overview.md
Product: windows-server
Technology: server-general
GitHub Login: @RobHindman
Microsoft Alias: robhind

Me too, getting exactly the same thing

I have the same issue. Have Windows 2016 Standard and with Windows 2019 Standard iso can't upgrade.

Thank you for your contribution to Windows Server documentation.  This appears to be a support request or can be address in our community forums.  #please-close

@clgz2002 Please open a support ticket.  GitHub issues for this repo are for reporting problems with our documentation. #please-close


[DCOS-51247] fix: gm-scroll-view does not trigger onScroll events

Overview
reactjs-components Table was expecting .gm-scroll-view to raise onScroll events in order to update it's virtual list of items. These events were not being raised because this is not the scrollable element in gemini. The new selector is the one added to <GeminiScrollbar> component in our FluidGeminiScrollbar wrapper.
There is another separate issue of the entire <tbody> getting replaced on the DOM when the event is raised, but this fixes the "giant blank screen" issue after scrolling ends and reveals a "janky scroll because the browser is too busy manipulating the DOM" issue.
Testing
Go to the jobs history page on SOAK to see a long list and start scrolling.

Update:
I dove deep into “window” rendering (react-virtualized etc) and determined that our reactjs-components’ VirtualList (used by our Table component) has a fatal flaw: It does not render absolutely positioned elements with a diffable style attribute and therefore imposes too much DOM activity on React. You can see an ‘overscroll’ behavior in the simplest examples which is simply ridiculous. I sought to replace VirtualList with react-window, but unfortunately ran out of time. Plus, I have a feeling react-window is fundamentally incompatible with <table> positioning anyway, which is imposed by the Table component. If so much work is needed, it might best to replace with the ui-kit table.

Hey Brandon, looking at soak with your branch doesnt seem to improve a lot, am I doing something wrong or is it the reason for your follow-up comment? Do you think that replacing the table actually help? My impression was more like there is not "that one" thing that improves performance drastiaclly, but its more like the general amount of load thats on chrome during scrolling 🤔
https://cl.ly/972399115c0b

@juliangieseke It's still possible to overscroll with this table (even in the documentation example) and I believe the reason to be what I mentioned in the update... But I think things are still improved because the table no longer has to wait for a polling response to receive an update, as opposed to a scroll event.

I also don't see much of a difference.

:tada: This PR is included in version 2.77.4 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


wrong times for events after DST change

I am in the CEST (UTC+2) time zone. We are on Daylight Saving Time until the last Sunday in October, when we will switch to CET (UTC+1). (Google) Calendar events planned after the time switch are shown in CEST time; so, for instance, I have a meeting at 2:00 PM on 5th November which calendar.vim shows at 3:00 PM. I think the event time should rather appear in the time zone which will be in use on that day.

This is same as #95 and is a known issue. But I'm not planning to support DST switching in the near future, sorry.

Would you accept a PR about this functionality?

The solution to this problem is probably rather simple. You can define timezone as a +/- hour:minute or as "Europe/Amsterdam". The first definition is fixed and doesn't keep track of DST. If you use the second definition google calendar automatically takes care of DST.
You can verify with calendar.vim that this works by changing the definition of set_timezone here https://github.com/itchyny/calendar.vim/blob/8d4ab7a8d9120a3538216056047d10df3947789b/autoload/calendar/google/calendar.vim#L591 to the following:
function! s:set_timezone(calendarId, obj) abort
  let a:obj.timeZone = 'Europe/Brussels'
endfunction

Found where the time is corrected. You could change this line and make timezone (defined +/- hour:minute) dependent one the date and time to be converted:
https://github.com/itchyny/calendar.vim/blob/8d4ab7a8d9120a3538216056047d10df3947789b/autoload/calendar/time.vim#L135
From vim v8.1.2326 there is the strptime function that can be used for this:
let time_zone = strftime("%z",strptime("%Y-%m-%dT%H:%M%z",a:str))

In short, with this fix you'll probably want to remove this line:
https://github.com/itchyny/calendar.vim/blob/8d4ab7a8d9120a3538216056047d10df3947789b/autoload/calendar/setting.vim#L46
and each time 'calendar#setting#get('time_zone')' is invoked check whether it is set. If it's not run 'time_zone = strftime("%z",strptime("%Y-%m-%dT%H:%M%z",a:str))' to check the correct offset according to DST.

Basically not that terribly difficult. You need two changes:
One here:
https://github.com/itchyny/calendar.vim/blob/8d4ab7a8d9120a3538216056047d10df3947789b/autoload/calendar/time.vim#L28
change this function first two lines in
function! calendar#time#time_zone(str) abort   let time_zone = strftime("%z",strptime("%Y-%m-%dT%H:%M:%S%z",a:str))
and add code to pass the original string from the datetime function. I've attached the changed file. You would probably also have to add code to check that a:str is in the correct format to be read by strptime.
And a oneliner here:
https://github.com/itchyny/calendar.vim/blob/8d4ab7a8d9120a3538216056047d10df3947789b/autoload/calendar/google/calendar.vim#L592
into
  let timezone = strftime("%z",strptime("%Y-%m-%dT%H:%M:%S",a:obj.dateTime))
I would propose that you check first whether calendar#setting#get('time_zone') returns something, if not do the strftime/strptime thing with the passed string.
time.vim.gz

Been using this solutions for a month now. It seems to work without any problems.

It would be very cool if we could get this in.

The issue is more serious than just a matter of display:
I just added today (in February, when where I am is in the EST timezone), an event in late March, which is after the DST change to EDT, by adding the event by pressing i and writing 3/27 10:00:17:00 eventname, but the event shows up in google calendar for 11 to 18 on March 27. This is quite confusing.

The problem seems more serious than just a matter of display.
I added today (well before DST change) an event in late March (after the DST change for where I am) by pressing i and writing 3/27 10:00-17:00, but the event was added to google as being between 11:00 and 18:00. This seems quite confusing, and certainly would lead to errors...


Entrust::routeNeedsRole ignores Route::group?

Attempting to setup multiple domains like so:
www.domain - Front guest domain, no auth.
login.domain - Sub domain, handles normal authentication (basically all Confide routes).
go.domain - Logged in domain. All requests must be authenticated.
//filters.php
Route::group(array("domain" => 'go.domain'), function()
{
    Entrust::routeNeedsRole( '/*', 'GoAccess', function(){
     if ( Auth::guest() ) // If the user is not logged in
        {
        Session::put( 'loginRedirect', Request::url() );
        return Redirect::to( 'http://login.domain' );
        }
    } );
});

No matter what domain I access, the Entrust::routeNeedsRole seems to be running, causing an obvious redirect loop, as there's no way to login.
Ideas/Workarounds?

Was this bug solved?


cmd/compile: the compiler should probably recognize runtime.Goexit

What version of Go are you using (go version)?

$ go version
go version go1.14rc1 linux/amd64

Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN="/home/manlio/.local/bin"
GOCACHE="/home/manlio/.cache/go-build"
GOENV="/home/manlio/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/manlio/.local/lib/go:/home/manlio/src/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/manlio/sdk/go1.14rc1"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/manlio/sdk/go1.14rc1/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build423783759=/tmp/go-build -gno-record-gcc-switches"
GOROOT/bin/go version: go version go1.14rc1 linux/amd64
GOROOT/bin/go tool compile -V: compile version go1.14rc1
uname -sr: Linux 5.5.2-arch1-1
/usr/lib/libc.so.6: GNU C Library (GNU libc) stable release version 2.31.
gdb --version: GNU gdb (GDB) 8.3.1

What did you do?
https://play.golang.org/p/SZZ4gDss0DK
What did you expect to see?
The program to be compiled.
What did you see instead?
/prog.go:13:1: missing return at end of function

The Go compiler does not know that runtime.Goexit causes the function to exit, so the return statement is unreachable and not required.
Of course there are similar functions like os.Exit, but Goexit is part of the runtime, so it should probably be recognized by the compiler.
On the other hand, vet should know about runtime.Goexit and should report that the statements after Goexit() are not reachable.
The compile or vet commands should also report an error if a function used in the go statement has a non empty result in the signature, but this is a different issue and probably there is a reason why it does not report an error.

The relevant bit of the spec is https://golang.org/ref/spec#Terminating_statements. I’d personally be reluctant to add runtime.Goexit; everything else in that list is lower level.
It is important for vet to use the same set of terminating statements as the compiler. Otherwise you’ll be forced to add a terminating statement in some cases (like this) to make your code compile but vet will then complain about it.

The Goexit function really acts as a low level API: https://github.com/golang/go/issues/29226

/cc @randall77 @griesemer per owners.

The definition of "terminating statements" is based strictly on properties of the language spec (which are unlikely to change), not APIs (which may change).
Also, even if we added runtime.Goexit it wouldn't be satisfactory anyway: It's not unlikely that runtime.Goexit is factored out into a function that does cleanups or perhaps prints additional error information; thus it is that function that will be called (possibly from multiple places) in a program. And unless we propagate the "exit" information to the caller, knowing that runtime.Goexit doesn't return won't be very helpful - it'll save a single return statement. That doesn't seem worth it.
More precisely, runtime.Goexit doesn't terminate the function, it terminates the goroutine that calls it. If we wanted to capture that correctly, we'd need to propagate that up the call stack, with extra rules. I am not convinced this is worth the extra complexity or the mixing of language spec and library APIs.
Not really a proposal, but leaving open for a decision by the proposal review commitee.

I've found a bunch of times where I call t.Fatal in a testing utility but I still need to return, so instead I currently add switch{} after the fatal calls,
adding some attribute(like C++'s noreturn) or even type (like Rust's Never type(!)) would be really useful even if it would be limited to the standard library only


Improve flexibility + adaptability of banner

Just opening a PR so that we can review changes / add comments etc :)
Will fix https://github.com/beyonk-adventures/gdpr-cookie-consent-banner/issues/2

Just found some time to try this out today. Looking fantastic!


mpl circ drawer fails with idle_wires=False

Environment

Qiskit Terra version: latest
Python version:
Operating system:

What is happening?

qc = QuantumCircuit(6, 5)
qc.x(5)
qc.h(range(6))
qc.cx(range(5),5)
qc.h(range(5))
qc.measure(range(5), range(5))

backend = provider.get_backend('ibm_geneva')
trans_qc = transpile(qc, backend, optimization_level=3, approximation_degree=0)
trans_qc.draw('mpl', idle_wires=False)

gives
Traceback (most recent call last):
Input In [9] in <cell line: 2>
trans_qc.draw('mpl', idle_wires=False)
File /opt/miniconda3/envs/qiskit/lib/python3.10/site-packages/qiskit/circuit/quantumcircuit.py:1907 in draw
return circuit_drawer(
File /opt/miniconda3/envs/qiskit/lib/python3.10/site-packages/qiskit/visualization/circuit_visualization.py:231 in circuit_drawer
image = _matplotlib_circuit_drawer(
File /opt/miniconda3/envs/qiskit/lib/python3.10/site-packages/qiskit/visualization/circuit_visualization.py:601 in _matplotlib_circuit_drawer
return qcd.draw(filename)
File /opt/miniconda3/envs/qiskit/lib/python3.10/site-packages/qiskit/visualization/matplotlib.py:323 in draw
max_anc = self._get_coords(n_lines)
File /opt/miniconda3/envs/qiskit/lib/python3.10/site-packages/qiskit/visualization/matplotlib.py:579 in _get_coords
q_indxs.append(self._wire_map[qarg])
KeyError: Qubit(QuantumRegister(27, 'q'), 0)
How can we reproduce the issue?
Do above
What should happen?
should not fail
Any suggestions?
No response

A self-contained reproducer for assisting debugging:
from qiskit import QuantumCircuit, transpile
from qiskit.providers.fake_provider import FakeKolkata

qc = QuantumCircuit(6, 5)
qc.x(5)
qc.h(range(6))
qc.cx(range(5),5)
qc.h(range(5))
qc.measure(range(5), range(5))

trans_qc = transpile(qc, FakeKolkata(), optimization_level=3, approximation_degree=0)
trans_qc.draw('mpl', idle_wires=False)

It has to do with barrier and idle_wires. This simple circuit fails as well. Fix is forthcoming.
qc = QuantumCircuit(4, 4)
qc.x(2)
qc.barrier()
qc.draw('mpl', idle_wires=False)


Pull in rummager's search documentation

This PR adds pulls in the rummager documentation page for the search API.
https://github.com/alphagov/rummager/blob/master/docs/search-api.md
I think this is probably the most useful page for rummager. I've considered also pulling in pages like indexing docs, but they seem less complete at the moment. Perhaps we can start with this page and see how it's used?
Thoughts @MatMoore?

Was there a problem linking out to more project focused documentation?
http://alphagov.github.io/rummager was serving documentation for a while, and I found it really useful! What I like about github pages is the docs live really close to the code, and updating them is really low maintenance.
I'm not a huge fan of the search-api.md page, which is why I've been looking at swagger-based tools to automatically generate it, but it's the most complete thing we have for now. Tara's blog post might be a better introduction though, it's got lots of examples: https://gdsdata.blog.gov.uk/2016/05/26/use-the-search-api-to-get-useful-information-about-gov-uk-content/
https://github.com/alphagov/rummager/blob/master/docs/adding-new-fields.md is another one I've referred back to recently (even though I think I wrote it)


Sorting on decimal and numbers in general

Is this functional in the current release? Haven't looked through the change logs much, but the version I'm using doesn't seem to have it.

Are your numbers actual numbers or strings?
orderBy

Note: if you notice numbers are not being sorted as expected, make sure they are actually being saved as numbers and not strings.


Not updated Total caches size after deletion

Describe the bug
gh actions-cache list show Total caches size. It seems not to be updated after gh actions-cache delete.
Steps to reproduce the behavior
$ gh extension install actions/gh-actions-cache
✓ Installed extension actions/gh-actions-cache
$ gh actions-cache list
Total caches size 319.32 MB

Showing 1 of 1 cache entries in peaceiris/private-repo

setup-go-Linux-go--3b25b2e3990931c5fa648090fe97a4fdaf9b42dba07ace5b62ec0b9b2d4751a3                                                [319.32 MB]     refs/pull/530/merge                       an hour ago
$ gh actions-cache delete setup-go-Linux-go--3b25b2e3990931c5fa648090fe97a4fdaf9b42dba07ace5b62ec0b9b2d4751a3
You're going to delete 1 cache entry

setup-go-Linux-go--3b25b2e3990931c5fa648090fe97a4fdaf9b42dba07ace5b62ec0b9b2d4751a3                                                [319.32 MB]     refs/pull/530/merge                       an hour ago

? Are you sure you want to delete the cache entries? Delete

✓ Deleted 1 cache entry with key 'setup-go-Linux-go--3b25b2e3990931c5fa648090fe97a4fdaf9b42dba07ace5b62ec0b9b2d4751a3'
$ gh actions-cache list
Total caches size 319.32 MB

There are no Actions caches currently present in this repo or for the provided filters

Expected vs actual behavior
gh actions-cache list shows the following message after deleting the cache.

Total caches size 0.0 MB

$ gh actions-cache list
There are no Actions caches currently present in this repo or for the provided filters

Hi @peaceiris There is a 5-6 min delay in updating Total caches size


feat(clerk-js): Add default Allowed redirect origins

Description
This PR introduces default values for the allowedRedirectOrigins option, the current implementation does not provide any defaults, with this change if there is no option provided the default will be similar to the example below.
Let's say the host of the application is test.host, the origins will be

https://test.host/
https://test.host/*
https://*.yourawesomeapp.clerk.accounts.dev/
https://*.yourawesomeapp.clerk.accounts.dev/*

Checklist

[ ] npm test runs as expected.
[ ] npm run build runs as expected.
[ ] (If applicable) JSDoc comments have been added or updated for any package exports
[ ] (If applicable) Documentation has been updated

Type of change

[ ] 🐛 Bug fix
[X] 🌟 New feature
[X] 🔨 Breaking change
[ ] 📖 Refactoring / dependency upgrade / documentation
[ ] other:

Packages affected

[ ] @clerk/backend
[ ] @clerk/chrome-extension
[X] @clerk/clerk-js
[ ] @clerk/clerk-expo
[ ] @clerk/fastify
[ ] gatsby-plugin-clerk
[ ] @clerk/localizations
[ ] @clerk/nextjs
[ ] @clerk/clerk-react
[ ] @clerk/remix
[ ] @clerk/clerk-sdk-node
[ ] @clerk/shared
[ ] @clerk/themes
[ ] @clerk/types
[ ] build/tooling/chore

!preview

Hey @octoper, your preview is currently deploying and should be available shortly.

Status
Preview
Updated (UTC)

:cook: Deploying...
Inspect
Nov 15, 2023 10:48 AM

🔧 Based on the definition of the origin from MDN docs, I think that we should only allow passing origin values to the allowedRedirectOrigins option (eg ${protocol}://${domain}:${port}). Doing that will also allow us to drop the /* path default origins. If we want to match paths in allowedRedirectOrigins i would advise we rename the options. cc: @nikosdouvlis

The /* part should not be required for someone to use allowedRedirectOrigins successfully. These tests cover almost all valid use cases (https://github.com/clerk/javascript/blob/d37d44a68e83b8e895963415f000c1aaef66e87e/packages/clerk-js/src/utils/tests/url.test.ts#L421) without /*
Are you suggesting we manually remove /* if the user passes it in? Would that offer any benefit DX-wise?


feat(ui): update pipeline status UI

Closes #7127
Describe your changes :

I worked on the ..... because ...

Type of change :

[x] Bug fix
[x] Improvement
[x] New feature
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation

Frontend Preview (Screenshots) :
For frontend related change, please link screenshots of your changes preview! Optional for backend related changes.

Checklist:

[x] I have read the CONTRIBUTING document.
[ ] I have commented on my code, particularly in hard-to-understand areas.
[ ] I have added tests that prove my fix is effective or that my feature works.
[x] All new and existing tests passed.

Reviewers

@chirag-madlani what is remaining here and why didn't we merge this yet?

@chirag-madlani what is remaining here and why didn't we merge this yet?

@harshach Cypress was failing. Will be merged today

@chirag-madlani what is remaining here and why didn't we merge this yet?

Sorry, @harshach I mixed this up with @ulixius9's PR. We discussed this in the standup that we will be doing it in next release as ingestion is doing some changes around this


gulp build fails

I found the build command fails, prompts:
[18:46:56] Task 'lint' is not in your gulp file

Please use
$ gulp csscopy
$ gulp jscopy
$ gulp js

I will fix it and the README later. Thanks.

@qcgm1978 README was updated.


Move InCluster auth in kubernetes example config

CA and Bearer Token are config of kubernetes_sd_configs, not the
scrape_config. Also updated misleading top-level comment and removed
unnecessary global config.

Thanks!

tls_config is valid at the scrape config level - this is to allow the actual scrape targets to be secured.
When used in cluster, the CA & bearer token settings on the SD config are not normally required: the mounted secrets will be used for that.

The original example file could be misleading as it uses the service account token to do the actual scrape too. Perhaps just removing the scrape level config would be a better idea.

@keegancsmith Would you be able to just check that the tls_config in the SD config isn't required for your set up when running in cluster? It shouldn't be, but just want to make sure there isn't a bug here.

@jimmidyson That makes a lot of sense, and yes I was mislead when reading the config file. I was thinking of removing the scrape level config, but if it helps make scraping more secure maybe we should move it back with a comment?

@keegancsmith I think if we remove it from the scrape level config that should be covered by other docs? It's generic scrape config so unrelated to kubernetes SD per se.

Looking at this again, due to the separation of scrape & discovery, the CA cert & bearer token is required on the scrape config to be able to scrape the master. Perhaps it makes more sense to separate out the SD configs for the master & services, which are likely to have different auth requirements for scrape configs?

Agreed. Yes I don't require it, but I am scraping my own services. I just did a quick test with the example config, and something I didn't notice before was that that I could scrape all k8s services except one. The one failing was the only one behind https. I have just sent #1124 to re-include the auth lines, since that fixes the issue. Sorry about that.

Yes, that would make sense. In my case I use something like
  relabel_configs:
  - source_labels: [__meta_kubernetes_service_name]
    action: keep
    regex: ^foo$
  - source_labels: [__address__]
    action: replace
    target_label: __address__
    regex: ^(.+)(?::\d+)$
    replacement: $1:6060

anyways. I think that would likely be more obvious to new users to select by service name. I have the port rewriting since I don't expose the metrics on the same endpoint as the usual service.

@keegancsmith You can also use an annotation on the service (the example config uses annotations prometheus.io/* to be able to relabel scheme, port & path.

Yes I went down that path at first, but in our case it is more straightforward to make the config explicit than to tag the service. I can imagine when the number of services we want to scrape grows I'll switch to annotations.


fix: spec.KubeContext does not fallback to HelmDefaults.KubeContext

fixes: #1782

Hi, @mumoshu
I made a fix of the issue that I opened yesterday,
but I'm not sure this fix works with no side effect.
Let me know if you have any concerns, thanks.

I have tested the fix by running make test, then I fixed the failed tests,
but I'm not adding any tests for this fix.
Where should I put the test of this fix if it's necessary?

@katsew Hey! Thanks for the PR. This looks great so far.
I'm still figuring that out, but could you start by adding a single test case "without the default kubecontext", in pkg/app/app_apply_test.go? That would help us clarifying what are differences when you've specified kubecontext or not, and also help us to not introduce any regression on cases that we don't set the default kubecontext.

Hi, @mumoshu . Thanks for the response.
I'll add a test for the case you provided.

@mumoshu
I copy-n-pasted existing test cases which depend on the constants helmV2ListFlags and helmV3ListFlags ,
then I added new constants helmV2ListFlagsWithoutKubeContext, helmV3ListFlagsWithoutKubeContext to test the fix.
If it's ok, then I'll squash the commit.
Please review it, thanks.

@katsew Thank you so much for your effort! Everything looks great. Let me squash it and merge this.


Update sbt-avro to 3.3.0

Updates com.cavorite:sbt-avro from 3.2.0 to 3.3.0.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.cavorite", artifactId = "sbt-avro" } ]

labels: sbt-plugin-update, semver-minor

Codecov Report

Merging #177 (a394350) into main (595f1ad) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #177   +/-   ##
=======================================
  Coverage   70.51%   70.51%
=======================================
  Files           4        4
  Lines          78       78
  Branches        4        4
=======================================
  Hits           55       55
  Misses         23       23

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 595f1ad...a394350. Read the comment docs.


No objects shown by s3 ls , in case metadata of one file is corrupted

Environment info
[root@ocp-neha-1-inf ~]# oc version
Client Version: 4.7.0
Server Version: 4.7.0
Kubernetes Version: v1.20.0+bd9e442
[root@ocp-neha-1-inf ~]# noobaa version
INFO[0000] CLI version: 5.8.0
INFO[0000] noobaa-image: noobaa/noobaa-core:5.8.0-20210418
INFO[0000] operator-image: noobaa/noobaa-operator:5.8.0
[root@ocp-neha-1-inf ~]#
Actual behavior

Corrupting  inode metadata inside NooBaa endpoint pods. This causes an s3 list for all objects to fail.
Inside endpoint

Expected behavior

s3 list should report  objects ("happy" and "object_abc.txt") , because their owners/ACLs are good

Steps to reproduce

Corrupt object of the bucket and try to list object

More information - Screenshots / Logs / Other output
sh-4.4# ls -lrt
ls: cannot access 'happy1': Permission denied
total 1
-?????????? ? ?    ?     ?            ? happy1
-rw-------. 1 root root 11 Apr 28 11:00 happy
-rw-r--r--. 1 3001 3003 12 Apr 28 11:51 object_abc.txt
sh-4.4#
Listing the respective s3 bucket results in the following output:
[root@fyreauto-x-app1 ~]# s3 ls s3://trial-buc
urllib3/connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
urllib3/connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
urllib3/connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
An error occurred (InternalError) when calling the ListObjectsV2 operation (reached max retries: 2): We encountered an internal error. Please try again.
Endpoint Logs:
ep_corrupt.log

@nehasharma5 in order to repro and fix, how exactly did you corrupt the specific file?

@nimrod-becker I had selinux setting disabled on Storage Cluster and did noobaa setup. After that perform any operation on object like appending , copy, delete from SC, getting the above issue

@nehasharma5 as I couldn't reproduce this issue locally. Can you try to reproduce and provide the logs - I don't see anything with the logs you provided - not even a single S3 command. Can you also run with higher debug when doing so, then I will have the whole info. And if the cause is that the endpoint is getting restarted in the middle please provide the previous endpoint logs as well: oc logs <endpoint-pod> -p Thanks

Hi @romayalon , @jackyalbo : I can define the metadata of an object while uploading but what could be a way to corrupt it AFTER it is uploaded. Also, when a read is performed on an object, would it matter if metadata is not the same as that when it was originally uploaded?
Also, it is mentioned that inode metadata is corrupted inside endpoint pod. I don't understand what it means. Please explain.

@akmithal according to Neha's comment - I had selinux setting disabled on Storage Cluster and did noobaa setup.
I'm not sure but I think it's not relevant any more

Closing, HPO is discontinued.


Federated tags

Cross-server tags, currently you can't see repositories from other servers (example: https://git.nixnet.xyz/explore/repos?q=ntp&topic=1)

@theel0ja it's not very clear about your requirements. Could you give more details?


Fix Magic issues

Fixed some strange Magic issues. Also fixed issues with Ace:
Ace was taking the following styles:
#code-editor{
  position:absolute;
  top:0;
  bottom:0;
  left:0;
  bottom:0;
}

Absolute positioning takes it out of the regular document flow. I think it's a bit strange that Ace recommend users to do this, as it only makes sense if the editor is to be used full-screen.
I've turned Ace back into a static-ly positioned element, with a fixed height (420px). Other issues have been fixed, including forcing the footer to the bottom of the page. Visually, the page is bug-free. There is still potential for fonts not to work, although I'm not sure why.
If you find any further issues with Magic components or confusing terminology/lack of description in Magic docs, please submit them to https://github.com/palkerecsenyi/magic/issues, and I'll fix it ASAP.

Thanks.


Company Page - Contract Quick Settings - Design Changes

Following changes are suggested.

We need click on "Total Annually Fees" to add service.

We need "Clear All" Button to remove all present schedule.
In "Quick Setting" When we select a particular date then all monthly or Quaterly schedule should be repeated on selected date.

https://www.berrycast.com/conversations/5031b7f7-a2dc-509d-9ad6-f219f4cbad4a
Originally posted by @sureshsirvi in https://github.com/Fammeo/EasyCloudBooks/issues/1520#issuecomment-1228054896

Creating Same thing on different button makes things difficult to manage in terms of development, as future changes ke time ye nai yad rehta

over to @vinpahilwani and @sureshsirvi

agar date 10/06/2012 aisa liya hai and phir woh monthly kar raha hai toh kya hoga? 12 mahine kaise pure honge?

uski jagah aisa karo ki upar se jab woh monthly, quarterly, half year aisa click kare toh udhar hi next step puche data and once he does that automatically hum dates create kare

Initiated by @sureshsirvi

Requirement:

We need selection of date to repeat scheduler for each month.
When we select monthly option then irrespective of date or month already passed the scheduler must be created for all 12 months. Example if we select date 10/09/2022 and select option monthly then schedular must be created from 10 April to 10 March.

Over to @vinpahilwani

Yes nice. but only issue is mujhe invoice next month me karna hai toh? matlab april ka may, may ka june tab. Likewise first quarter ka invoice mujhe next quarter me toh? @sureshsirvi

I agree with vinod sir..
As we Select particular start date  then ther after all 12 months schedule to be created for the year.

Clear All Button Required?

Yes clear required


new bypass site

Link
https://ay.live/quSz
Version
Manifest-v2
What browser(s) are you using?
Chrome browsers (includes brave, edge, opera, etc)
What platform are you using?
Computer (Windows, Linux, MacOS)
(Optional) Anything else?
No response

Dupe #1229


Working with a list of phishing domains

Hi,
I was wondering if it's possible to work with a list of phishing domains, i've tried adding the list as a dictionary and tld but i think i need it for ssdeep, afaik it only takes 1 url with --ssdeep-url, is there a way to provide a list ? am i missing something?
I couldn't manage to provide a list of variations it will consider apart from the original domain name

The tool can only work with single domain name at a time. Have you considered using a script with simple loop?

Actually no i haven't, i was so sure i was doing something wrong and there was a better way that i didn't try anything else yet.
Now that i tried what you suggested i can see it didn't produce the results i wanted so i will describe my use case better

i'm looking for domain keywords that are different from the original name i.e. word1.com - word2.com but the content is the same or very similar, is there a way to compare by content or is that ssdeep?
the original domain is a two word hyphened domain i.e. word1-word2.com i can see the tool only checks the first word, is there a way to make it check the second one as well?

I'm not sure I follow. Can you provide a specific example of input domain and desired output?

Sure,
Lets take:
fast-refund.org

What i'm looking for is a way to check both words so i expected the results to provide something like:
fast-{words}.org

{words}-refund.org

{words}.org

{words}{words}.org

and with different suffix as well

What actually happens is only the first word and the suffix get searched
The other case is when there is a different domain that is very similar \ copied the original so i want to find those too, for this example that would be
instant-refund.world

But because it uses different words it isn't covered in the search
I'm looking to cover or mitigate both cases

This obviously works only when the dictionary fuzzer is enabled with -d <file> argument. If input domain contains hyphens, the fuzzer splits it and replaces the first and last chunk with provided dictionary words.
I did not use double hyphens in the code. I used .join() method of '-' string object.


Property cursor

It is from https://github.com/redboltz/mqtt_cpp/issues/297#issuecomment-527242751.
@jonesmz wrote:
Now that mqtt_cpp is using MQTT_NS::buffer to store all of the items in the properties, maybe we can revisit this discussion.
What do you think about doing just-in-time parsing?
When the packet is received, we can store the data for the properties in an MQTT_NS::buffer the same way we currently do.
That MQTT_NS::buffer can be owned by an MQTT_NS::property_cursor object
(In this situation, when I say cursor i mean the word in the same sense that some database engines use it. As a handle to data that will be fetched and/or parsed on demand, but until the parsing is triggered, the data is in an unspecified state).
The MQTT_NS::property_cursor object provides functions like:
struct property_cursor
{
    std::vector<optional<v5::property_varient>> parse_all_properties() { ... };
    optional<v5::property_varient> parse_next() { ... };
    struct iterator
    {
        // Some implementation that parses the property data only when the value is accessed.
    };
    iterator begin();
    iterator end();
    buffer data_;
}

This allows us to completely avoid needing to allocate storage space for the std::vector object.
For the test_broker, it also means that the broker never needs to parse the properties for publish messages, and can just send them to the subscribers directly. No deserialization / parsing, but also no serialization. Just moving blobs of data around (Much more efficient!!)
It also means that code that uses mqtt_cpp can decide if they want to parse the properties or not, on their own.
For example: I have several MQTTv5 messages that do transmit properties (because all of my message send the same list of properties, always), but the code that receives the properties never use them. Right now, mqtt_cpp is parsing the properties, and allocating a std::vector, even though I don't want it to, and don't use the result.
It also means that I would be able to do something like this (note, c++17 syntax, but can achieve the same result in C++14, too):
v5::property_cursor cursor(data from boost::asio);
std::optional<v5::correlation_data> corrData;
for(auto property : cursor)
{
    corrData = std::visit[](auto & prop) -> std::optional<v5::correlation_data>
    {
        if constexpr(std::is_same_v<decltype(prop), v5::correlation_data>)
        { return prop; }
        else
        { return std::nullopt(); }
    }
    if(corrData)
    {
        // We're only looking for one specific property. Exit early.
        break;
    }
}

In the above code, parsing only happens when the property is evaluated, or the iterator is dereferenced (and returns std::nullopt, if it can't be parsed, to indicate failure).
If I'm only interested in a specific property, I don't need to pay the runtime cost of 1) Allocating storage for std::vector, or 2) Parsing all properties. I only have to pay for parsing the properties until I find the one I want.

And, of course, it also means that the debug function signatures is much much smaller.

I will take a look this. Please wait a moment.

I think that it is nice to have. Avoiding parse conditionally is valuable.
I'm not sure how this mechanism work with buffer life time management.
Packet format is like as follows:
+-----------+---------------+---+--------------+--------+---
|FixedHeader|RemainingLength|...|PropertyLength|Props...|...
+-----------+---------------+---+--------------+--------+---

If RemainingLength is smaller than packet_bulk_read_limit_, then buffer is allocated once.
And properties are refer to the buffer. Each property might have buffers, it is sharing the lifetime of the buffer.
If RemainingLength is NOT smaller than packet_bulk_read_limit_, PropertyLength is compared to props_bulk_read_limit_.
If PropertyLength is smaller than props_bulk_read_limit_, then buffer is allocated once.
If PropertyLength is NOT smaller than props_bulk_read_limit_, then use buf_ for property_id.
When property_id is read, then read property_body using buf_ or newly allocated buffer . It depends on property_id.
The mechanism should work well with this process.

I think that there are three cases:

Case
Condition
Property Cursor Friendly

1
Read whole packet all at once
yes

2
Read all properties all at once
yes

3
Read individual property step by step
no

I think that most of clients need to parse all properties but broker sometimes doesn't need to parse (e.g. publish message delivery).
For broker, we can introduce force avoid case 3 option for endpoint.
class endpoint {
    void set_property_read_all_at_once(bool b = true);

If the flag is set, then enforce reading properties all at once even if PropertyLength is very big.
It is just an idea.

If RemainingLength is smaller than packet_bulk_read_limit_, then buffer is allocated once.
And properties are refer to the buffer. Each property might have buffers, it is sharing the lifetime of the buffer.
If RemainingLength is NOT smaller than packet_bulk_read_limit_, PropertyLength is compared to props_bulk_read_limit_.
If PropertyLength is smaller than props_bulk_read_limit_, then buffer is allocated once.
If PropertyLength is NOT smaller than props_bulk_read_limit_, then use buf_ for property_id.
When property_id is read, then read property_body using buf_ or newly allocated buffer . It depends on property_id.
The mechanism should work well with this process.

The case of PropertyLength > props_bulk_read_limit_ is complicated.
This situation requires that we allocate multiple mqtt::buffer objects, which partially defeats the purpose of using the concept of a PropertyCursor.
It's still an improvement because 1) No need to parse / deserialize unless requested by user. 2) smaller stack traces, which is good for debug experiences.

Case
Condition
Property Cursor Friendly

1
Read whole packet all at once
yes

2
Read all properties all at once
yes

3
Read individual property step by step
no

Another data structure that we can consider is a memory list.
Here's a simplified version of it.
template<typename Alloc>
struct memory_list
{
    using list_t = std::list<mqtt::buffer, Alloc>;
    size_t read_at(size_t idx, size_t count, std::uint8_t * pOutput);
    size_t write_at(size_t idx, size_t count, std::uint8_t * pOutput);

    struct read_cursor
    {
        list_t::iterator currentBuffer;
        size_t currBufferIdx;
        mqtt::buffer::iterator begin();
        mqtt::buffer::iterator end();
    };
    read_cursor
    std::list<mqtt::buffer, Alloc> storage_;
}

With a memory pool of mqtt:buffer objects that stores the mqtt::buffers that are no longer in use by user code, this implementation needs to allocate very rarely, and allows for mqtt_cpp::endpoint to do case 3 every time without any performance concerns.

I'm not sure where is the goal of your idea.
But I want to keep the mechanism v5::property_variant. I think that your idea is NOT removing it.

No need to parse / deserialize unless requested by user.

It has an advantage for implementing broker but I guess that most of clients want to parsed properties.
Another point, if properties don't contain the property that have string/binary, then endpoint doesn't need to any memory allocation for property buffer. endpoint (re)uses pre-allocated buf_. *1.
Of course, std::vector<v5::property_variant> allocation is required.
I'm not sure your approach keeps *1 advantage ? Or even if the property doesn't contain string/binary, then allocate buffer? If it does, I think that it is not good.
I think that I don't understand your idea well yet.

smaller stack traces, which is good for debug experiences.

If you are using gdb, then something like https://github.com/ruediger/Boost-Pretty-Printer could help.
The current C++ is going to use std::variant more because it is standardized. If std::variant has many entries, the problem is happened. I think that tool help is natural approach.
I want to discuss two goals separately. Let's focus on the performance improvement (conditional property parse) here.

I'd like to know the impact of existing APIs. e.g. publish() family, handlers.

Apologies for short answer with bad formatting. Am typing on phone.
The property cursor can have an MQTT::variant.
If case 1 then v5::properties (vector of variant)
If case 2 then MQTT::buffer
If case 3 then vectorMQTT::buffer
If the entire packet is so small that it fits into buf_ then parsing the data will be very very low cost, anyway.

Functions called by end user to send message with properties would take a variant. Instead of v5::properties, they would take variant<v5::properties, v5::property_cursor>
But if v5::property_cursor can be implicitly constructed from v5::properties, then instead it can just take v5::property_cursor. End user won't see a difference.
Handlers would need to change to accept a v5::property_cursor instead of v5::properties.

Indeed, I understand your approach.
Keeping the interface of existing APIs is nice. And breaking change of handlers is acceptable.

If the entire packet is so small that it fits into buf_ then parsing the data will be very very low cost, anyway.

I don't understand this part yet.
What happens the case as follows:

Properties contains one very big string/binary (A), and many non string/binary properties (B).
PropertyLength is not less than props_bulk_read_limit_

The current implementation is allocating one buffer for A and (re)use internal buf_ (10bytes) for B.
I'm afraid that allocating buffers for B instead of using buf_.
buf_ is reused again and again. Because if each B is parsed, then the buf_ is no longer needed. How to keep this mechanism?


Fix failing integration tests.

We do two things here:

Add a rate limiter reset to the diag web host tests.
Run tests in serial.  We need to do this as the tests depend on the rater limiter being reset

I also call this method in TestTrace in the same file. Is that a problem/still necessary?

(I'll leave Mike to review this as I'm away for a couple of days. :)

Thanks Mike!


Installation Error

python setup.py install
Traceback (most recent call last):
File "setup.py", line 4, in
from setuptools import setup
ImportError: No module named setuptools

I just tried to install it in a Python 2.7.6 virtualenv on Xubuntu 14.04 and it there was no problem during the setup.
What are your system details?

That did the trick for the install!  Sorry I missed it. FYI, I'm using Linux Mint Mate 17.3.
I have Samsung Model SM-G360V.  Is Vendor and Model case insensitive?  So, do I use -V samsung  -M sm-g360v?
Thank you for this software!  I tried PureSync on Windozzzzee.  Incredibly slow!!!

No problem.
I am ignoring case when searching for the device, so it doesn't matter as long as model and vendor are correct.


Manually initializing a destroyed connection pool did not change database schema name

Summary
I am setting up test script to create multiple database schema to utilize AVA parallel test run. As a result, I need to create the schema first, then switch the application connection pool to use the new created schema. However I encountered a bug in doing so, and would like to ask the official proper way to change the database schema name.
Environment
Knex version: 0.21.0
Database + version: MySQL 5.7
OS: Windows 10
Bug
According to the documentation:

If you ever need to explicitly teardown the connection pool, you may use knex.destroy([callback]). You may use knex.destroy by passing a callback, or by chaining as a promise, just not both. To manually initialize a destroyed connection pool, you may use knex.initialize([config]), if no config is passed, it will use the first knex configuration used.

I expect I can change the database name by knex.destroy and knex.initialize(newConfig) with new configuration. This does not work.
Reduced test case as follows:
const knex = require('knex')

function getConfig () {
  return {
    client: 'mysql',
    connection: {
      host: 'localhost',
      user: 'root',
      password: ''
    }
  }
}

const db = knex(getConfig()).on('query', function (data) {
  console.log(data.sql)
})

const method = 1

;(async () => {
  // Randomly (in my case it is based on server listening port) assigned a database name for AVA parallel testing
  const dbName = 'TMP_DB_' + (Math.floor(Math.random() * 999) + 9001)
  try {
    await db.raw(`CREATE DATABASE ${dbName}`)
    await db.raw(`USE ${dbName}`)
    await db.schema.createTable('tmp1', (table) => {
      table.increments()
      table.string('val1', 10).notNullable()
    })
    if (method === 1) {
      // I tried the documented way, but the `connection.database` is not changed with the following code
      await db.destroy()
      const newConfig = getConfig()
      newConfig.connection.database = dbName
      db.initialize(newConfig)
    } else if (method === 2) {
      // Is this the proper way to set the `connection.database`?
      db.client.connectionSettings.database = dbName
    }
    // This will result in `Error: ER_NO_DB_ERROR: No database selected` if database name is not set
    await Promise.all([
      db('tmp1').insert({ val1: 'a' }),
      db('tmp1').insert({ val1: 'b' }),
      db('tmp1').insert({ val1: 'c' })
    ])
  } finally {
    try {
      await db.raw(`DROP DATABASE ${dbName}`)
    } catch (ignore) {}
    await db.destroy()
  }
  console.log('End')
})().catch(e => console.log(e.stack))

Question
My additional question together with this bug, is how to change the database schema of the connection pool? Is the const method = 2 the proper way to do so? Or it can only be treated as undocumented way and may change in the future?

You could just destroy and throw away old knex instance and create completely new instance with updated config. That would be at least working official way.
Anyways this might be also a real bug. I didn't find any tests for passing new configuration for initialize. https://github.com/knex/knex/blob/e37aeaa31c8ef9c1b07d2e4d3ec6607e557d800d/test/integration/suite.js#L48

Thanks a lot. The main reason why I am avoiding throw away old knex instance is because I am abstracting my database implementation in separated files as shown below. Some complexity will be introduced if I need to handle throw away of knex instance for testing ☹
// In file `db.js`
const knex = require('knex')

const db = knex({
  client: 'mysql',
  connection: {
    host: 'localhost',
    user: 'root',
    password: ''
  }
}).on('query', function (data) {
  console.log(data.sql)
})

module.exports = db

// In file `index.js`
const db = require('./db')
// ...

hmm.. I don't see how that works for testing if you are trying to run multiple tests concurrently against multiple databases without having multiple knex instances there which are connected to different database 🤔

Good catch. This is process isolation provided by AVA which run each test file in separated process. Therefore even I defined one single global knex instance, it only apply to one process and another separated process will have it's own global knex instance pointing to another database.

I just hit the exact same problem and was disappointed to see that client.config does not get updated (basically the entire flow happening inside the knex constructor does not execute again for the new config), and the main reason I tried avoid recreating an instance is for the exact same reason @VeryCrazyDog mentioned :(

@MehdiSaffar Would you like to send a PR implementing support for this use-case?

I will try my hand at it

I opened a PR please take a look :) @VeryCrazyDog @kibertoad

@VeryCrazyDog  Here is a temporary approach with ES6 proxies you could use if you want, not sure how correct it is but it seems to work on my side. Example:
const defaultConfig = YOUR_KNEX_CONFIG

const makeKnex = require('knex')

const wrapper = { instance: makeKnex(defaultConfig) }

async function reconnect (config) {
  await wrapper.instance.destroy()
  wrapper.instance = makeKnex(config)
}

const knexInstance = new Proxy(wrapper, {
  get: (target, prop) => {
    if (prop === 'reconnect') {
      return reconnect
    }

    return target.instance[prop]
  }
})

module.exports = knexInstance

@MehdiSaffar I haven't thought of using Proxy at that time (maybe I didn't even know Proxy at that time). Thanks a lot for sharing, though I am no longer working with project using RDBMS now.


QLC+ 4.10 not finding ENTTEC USB DMX Pro

I've been using 4.9.1 for some time now, using the DMX USB i/o, with no issues. I'm using Lubuntu 15.04.
When 4.10 came around, I uninstalled 4.9.1, and installed 4.10-qt5.
Now, it is sending zilch to the DMX universe. The USB-DMX box (ENTTEC USB DMX Pro) doesn't even turn the light on that shows that it's receiving data. One of my lights is stuck in "Presentation Mode", because it hasn't received anything over DMX ever since I first opened QLC+ 4.10.
I ran it in Debug Mode, and it gave me this on startup:

bool QLCFixtureDefCache::loadMap(const QDir&) Unknown Fixture Map tag:  "FixturesMap"
bool QLCFixtureDefCache::loadMap(const QDir&) Unknown Fixture Map tag:  ""
virtual bool LibFTDIInterface::open() "DMX USB PRO" device not found
virtual bool LibFTDIInterface::clearRts() "DMX USB PRO" USB device unavailable
virtual bool LibFTDIInterface::open() "DMX USB PRO" device not found
virtual bool LibFTDIInterface::clearRts() "DMX USB PRO" USB device unavailable

I have no idea what's going on. The device is plugged in exactly like before. The I/O page says that the device is operating correctly, and it's using libFTDI.
Could someone please give me a hand? I have a big event coming up in two days, and I'll probably have to rollback to the previous version until this is sorted out.

So, yeah, I rolled back to 4.9.1 (not before giving 4.10-qt4 a try, just for kicks and giggles), and it's now functioning correctly again. The I/O config is exactly the same, minus the changes in what it displays, i.e. 4.9.1 doesn't mention libFTDI anywhere.

By the way (sorry for the triple post) another oddity I experienced with 4.10 was that it wasn't displaying any of my Audio devices in the I/O tab. 4.9.1 displays all four fine, even though it only half-works (audio spectrum RGB matrices don't work at all).

Fixed by 1b863bf98a2e8d41dc1923dae6b6445aea5bdeb7

Are you planning on making a 4.10.1 version from this commit? Because this is a pretty severe issue

http://www.qlcplus.org/forum/viewtopic.php?f=2&p=39745#p39745

Perfect, thank you!!


Connection timeout message during automatic EGF download

I am using the automatic ESGF download option with ESMValTool. I want to download all available ensemble members for a given model (recipe below). I get a message saying TimeoutError: [Errno 110] Connection timed out. I am not sure if this means I should increase the timeout number in the esgf-pyclient.yml file. I'd like some help understanding what I am doing wrong so I can download the neccessary/available data successfully.
I am running ESMValTool on JASMIN using the module load esmvaltool option to run the recipe. I have an interactive logon option set in my esgf-pyclient.yml file. Here are the contents of that file:
logon:
  hostname: "esgf.ceda.ac.uk"
  interactive: true
search_connection:
  urls:
    - 'https://esgf.ceda.ac.uk/esg-search'
    - 'https://esgf-node.llnl.gov/esg-search'
    - 'https://esgf-data.dkrz.de/esg-search'
    - 'https://esgf-node.ipsl.upmc.fr/esg-search'
    - 'https://esg-dn1.nsc.liu.se/esg-search'
    - 'https://esgf.nci.org.au/esg-search'
    - 'https://esgf.nccs.nasa.gov/esg-search'
    - 'https://esgdata.gfdl.noaa.gov/esg-search'
distrib: true
timeout: 600  # seconds
cache: '~/.esmvaltool/cache/pyesgf-search-results'
expire_after: 86400  # cache expires after 1 day

My recipe is :
# ESMValTool
# recipe_download_esgf_data.yml
---
documentation:
  description: |
    This is a recipe to download data sets from ESGF nodes.

  authors:
    - swaminathan_ranjini

  title: |

    Recipe to download data from ESGF nodes.

  maintainer:
    - swaminathan_ranjini

datasets:

  - {dataset: MPI-ESM1-2-LR, project: CMIP6, exp: historical, ensemble: r[1:10]i1p1f1, start_year: 1995, end_year: 2014, grid: gn}
  - {dataset: MPI-ESM1-2-LR, project: CMIP6, exp: ssp370, ensemble: r[1:10]1i1p1f1, start_year: 2081, end_year: 2100, grid: gn}

preprocessors:
  preproc_extract_region_land:
    extract_shape:
      shapefile : IPCC-AR6-shapefiles/IPCC-WGI-reference-regions-v4.shp
      decomposed : False
      method : contains
      crop: True
      ids:
        - 'S.Asia'
    mask_landsea:
      mask_out : sea

diagnostics:
  day_pr:
    description: extract region
    variables:
      pr:
        preprocessor: preproc_extract_region_land
        project: CMIP6
        mip: day
    scripts: null

And finally this is the content of the  file ~/.esmvaltool/cache/esgf-hosts.yml :
esgf-node2.cmcc.it:
  duration (s): 2
  error: false
  size (bytes): 3098826
  speed (MB/s): 1.9
esgf3.dkrz.de:
  duration (s): 0
  error: false
  size (bytes): 1812797
  speed (MB/s): 5.0

@rswamina are you able to download anything at all or this is what you get at all times? @bouweandela is the specialist #1 when it comes to ESGF download issues, so am CC-ing him in here; also, depending on what the troubleshooting leads to, I may have to move this issue to ESMValCore :beer: PS: note that JASMIN has had some pretty nasty issues in the past 24 hours

@rswamina are you still seeing the reported behaviour? :beer:

Hi @rswamina,
Searching for files and downloading them are two separate steps that make use of separate servers.

Searching for files is done using the index servers (or nodes as they are known on ESGF), those are listed in the configuration file you posted. This step makes use of the timeout listed there. If you're having issues with the search, you can move another server to the top of the list, as ESMValCore will use the first index node that is online from that list.
The next step is downloading, this is done from the servers where the data is stored. This can be one or more servers, depending on where the data is hosted. The URLs are retrieved as part of the search. If you prefer not to use a particular host for downloading, you can set duration to a very large number for that server in the file ~/.esmvaltool/cache/esgf-hosts.yml. ESMValCore keeps track of how fast servers are and will prefer faster ones, so if you set duration to a high number it will think that server is slow and only use that one as a last resort. Note that this file is automatically updated, so your changes might get overwritten at some point.

Could you post the full stack trace to see which URL is giving you the timeout error? You can ask about the status of servers at the ESGF user mailing list.

Thanks. I will see if I can get that information out.

Hi @bouweandela -  On further investigation of the cache file, I noticed this was an older file from a previous run. Per instruction in the Readthedocs page I deleted it and find that a new one is not created now. I wonder if that means that connection was not made successsfully. This is the stack trace:

2023-01-21 19:32:11,049 UTC [11715] ERROR   Program terminated abnormally, see stack trace below for more information:
Traceback (most recent call last):
  File "/apps/jasmin/community/esmvaltool/miniconda3/envs/esmvaltool/lib/python3.10/site-packages/esmvalcore/_main.py", line 499, in run
    fire.Fire(ESMValTool())
  File "/apps/jasmin/community/esmvaltool/miniconda3/envs/esmvaltool/lib/python3.10/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/apps/jasmin/community/esmvaltool/miniconda3/envs/esmvaltool/lib/python3.10/site-packages/fire/core.py", line 466, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/apps/jasmin/community/esmvaltool/miniconda3/envs/esmvaltool/lib/python3.10/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/apps/jasmin/community/esmvaltool/miniconda3/envs/esmvaltool/lib/python3.10/site-packages/esmvalcore/_main.py", line 428, in run
    logon()
  File "/apps/jasmin/community/esmvaltool/miniconda3/envs/esmvaltool/lib/python3.10/site-packages/esmvalcore/esgf/_logon.py", line 28, in logon
    manager.logon(**cfg['logon'])
  File "/apps/jasmin/community/esmvaltool/miniconda3/envs/esmvaltool/lib/python3.10/site-packages/pyesgf/logon.py", line 184, in logon
    creds = c.logon(username, password,
  File "/apps/jasmin/community/esmvaltool/miniconda3/envs/esmvaltool/lib/python3.10/site-packages/myproxy/client/__init__.py", line 1453, in logon
    self.getTrustRoots(writeToCACertDir=True,
  File "/apps/jasmin/community/esmvaltool/miniconda3/envs/esmvaltool/lib/python3.10/site-packages/myproxy/client/__init__.py", line 1609, in getTrustRoots
    conn.connect((self.hostname, self.port))
  File "/apps/jasmin/community/esmvaltool/miniconda3/envs/esmvaltool/lib/python3.10/site-packages/OpenSSL/SSL.py", line 2022, in connect
    return self._socket.connect(addr)
TimeoutError: [Errno 110] Connection timed out
2023-01-21 19:32:11,057 UTC [11715] INFO
If you have a question or need help, please start a new discussion on https://github.com/ESMValGroup/ESMValTool/discussions
If you suspect this is a bug, please open an issue on https://github.com/ESMValGroup/ESMValTool/issues
To make it easier to find out what the problem is, please consider attaching the files run/recipe_*.yml and run/main_log_debug.txt from the output directory.

Hi @bouweandela  - I did not need the logon option for downloading CMIP6 data. However when doing an ESGF download, I could not use the generic option to download multiple ensemble members. Am I correct in understanding that I cannot provide an option like r[1:10] for ESGF downloads?

The syntax for specifying multiple ensemble members in a recipe is r(1:10)  (note the different brackets). Does it work if you use that?

No it does not. Here is how I specify my datasets:
  - {dataset: MPI-ESM1-2-HR, project: CMIP6, exp: historical, ensemble: "(r1:10)i1p1f1", start_year: 1995, end_year: 2014, grid: gn}

  - {dataset: MPI-ESM1-2-HR, project: CMIP6, exp: ssp370, ensemble: "(r1:10)i1p1f1", start_year: 2081, end_year: 2100, grid: gn}

The log message includes something like :
2023-01-26 14:25:11,244 UTC [32380] DEBUG   Starting new HTTPS connection (1): esgf.ceda.ac.uk:443
2023-01-26 14:25:11,290 UTC [32380] DEBUG   https://esgf.ceda.ac.uk:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&latest=True&project=CMIP6&source_id=MPI-ESM1-2-HR&variant_label=%28r1%3A10%29i1p1f1&experiment_id=ssp370&grid_label=gn&table_id=fx&variable=sftlf HTTP/1.1" 200 1218
2023-01-26 14:25:11,291 UTC [32380] DEBUG   Found the following files matching facets {'project': 'CMIP6', 'source_id': 'MPI-ESM1-2-HR', 'variant_label': '(r1:10)i1p1f1', 'experiment_id': 'ssp370', 'grid_label': 'gn', 'table_id': 'fx', 'variable': 'sftlf'}: none
2023-01-26 14:25:11,291 UTC [32380] WARNING Missing data for fx variable 'sftlf' of dataset ScenarioMIP
2023-01-26 14:25:11,291 UTC [32380] DEBUG   For fx variable 'sftof', found table 'Ofx'
2023-01-26 14:25:11,291 UTC [32380] DEBUG   Looking for files matching ['sftof_Ofx_MPI-ESM1-2-HR_ssp370_(r1:10)i1p1f1_gn*.nc'] in []

It seems like the files are not being retrieved. I do know that individual files exist and can be retrieved.  For instance if I try just to get ensemble member r2i1p1f1 in the above case that works for the historical and ssp370 experiments.

The letter r needs to be outside the brackets. Can you try again with datasets
  - {dataset: MPI-ESM1-2-HR, project: CMIP6, exp: historical, ensemble: "r(1:10)i1p1f1", start_year: 1995, end_year: 2014, grid: gn} in
  - {dataset: MPI-ESM1-2-HR, project: CMIP6, exp: ssp370, ensemble: "r(1:10)i1p1f1", start_year: 2081, end_year: 2100, grid: gn}

This worked. Thanks @bouweandela . I was staring so hard at the screen that I did not see the r was outside the quotes. Sorry. At this time, I am able to connect and download data from ESGF for CMIP6 data. If I have other problems, I will open a different issue. Happy to close this now.


support integration with VS code in gradle plugin

Task aggregateJsonSchema should configure JSON schema mappings for vscode (yaml.schema in .vscode/settings.json) when .vscode folder is detected in project.

helm-values-shared should be packaged as a webserver in a GraalVM native image (Unix, MacOS and Windows).
helm-values VS code extension can then run webserver in background and call its API to reuse business logic from shared Java library.


Update dependencies.

I've filed https://github.com/gimli-rs/addr2line/issues/311 on the test failure and will skip the addr2line/gimli update for now.