Behaviour of SeqAnParser with a truncated fastq

This is a bit of a strange question, but here goes.
While making a test for #877, I made the following FASTQ file:
@1:1:1:1:1 1:N:0:NNNNN
CAGGCGCCCA
+
][aaX__aa[
@1:1:1:1:2 1:N:0:NNNNN
CAGGCGCCCA

I.e., a valid record, followed by a record truncated after the sequence.
From my understanding of the format, and certainly from a pragmatic point, this is an invalid file. But the seqan parser silently accepts it, leaving read.quality := "".
And there's no indication of an invalid record (i.e., seqan::readRecord returns 0, no exceptions are thrown).
I'm fairly certain that it's me who is wrong and this is some corner case of the format, but it smells strange to me somehow.
Cheers,
Kevin

Further testing indicates the same happens with
@1:1:1:1:1 1:N:0:NNNNN
CAGGCGCCCA
+
][aaX__aa[
@1:1:1:1:2 1:N:0:NNNNN

which has to be invalid, surely :)

And even this:
@1:1:1:1:1 1:N:0:NNNNN
CAGGCGCCCA
+
][aaX__aa[
@

So I gather this is a bug @ctb? If so, would you like me to dig further and/or contact seqan?

Further digging is welcome! Looks like a bug with SeqAn 1.4.x. It'll be interesting to see if it also exists in 2.x.

@ctb Shall we add a check for FASTQ files that have missing quality records and throw our own exception?

[ ] Add to known issues list

On Mon, Mar 30, 2015 at 03:03:41AM -0700, Michael R. Crusoe wrote:

@ctb Shall we add a check for FASTQ files that have missing quality records and throw our own exception?

[ ] Add to known issues list

Sure.

OK, I'm happy to a) investiage w/ SeqAn 2, and b), raise a khmer exception when the quality is "" if it's a fastq file.

Thanks!
On Mon, Mar 30, 2015 at 1:42 PM, Kevin Murray<EMAIL_ADDRESS>wrote:

OK, I'm happy to a) investiage w/ SeqAn 2, and b), raise a khmer exception
when the quality is "" if it's a fastq file.
—
Reply to this email directly or view it on GitHub
https://github.com/ged-lab/khmer/issues/884#issuecomment-87631210.

OK, so w/ seqan 2, using the following works, throwing seqan::UnexpectedEnd. Which sounds about right!
#include <seqan/seq_io.h>

using namespace seqan;

int main(int argc, char **argv)
{
    std::string id, seq, qual;

    if (argc < 2) {
        return 1;
    }

    SeqFileIn seqFileIn(argv[1]);

    while (1) {
        readRecord(id, seq, qual, seqFileIn);
        std::cout << id << '\t' << seq << '\t' << qual << '\n';
    }

    return 0;
}

Huzzah. We should still file an issue with Seqan to document the problem for 1.4.x users.

OK, I'll double check the issue with a vanilla 1.4.2 tarball first, and raise an issue on their github. Tomorrow though, tis late down under :)

Thanks @kdmurray91, I appreciate it

Interestingly, seqan 1.4 does something different:
#include <seqan/seq_io.h>

int main(int argc, char **argv)
{
    std::string id, seq, qual;

    if (argc < 2) {
        return 1;
    }

    seqan::SequenceStream seqFileIn;
    seqan::open(seqFileIn, argv[1]);
    size_t seqs = 0;

    while (!seqan::atEnd(seqFileIn)) {
        seqan::readRecord(id, seq, qual, seqFileIn);
        std::cout << seqs++ << ' ' <<  id << '\t' << seq << '\t' << qual << '\n';
    }

    return 0;
}

Output:
~/src/khmer$ ./seqan14 tests/test-data/truncated.fq
0 1:1:1:1:1 1:N:0:NNNNN	CAGGCGCCCA	][aaX__aa[
1

So, it makes a new record, but doesn't fill ANY fields.
I can confirm that the same happens with the khmer headers under ./third-party.
So a "fix" would be to check the length of the id/seq/qual strings and raise an error if it's empty? Or should we just silently skip it?

Gist links for future reference:
seqan 1.4
seqan 2.0

Intriguing. What version of Seqan were you initially testing with, 1.4.2?
On the first record only: set flag based on presence of qual string. If that flag is set then the length of seq and qual string must be equal; throw an appropriate khmer exception if not.

Ah, it was khmer::SeqAnParser. I'll investigate further and make sure that my reported output was accurate. It could have been a bug in my debugging code (yo dog, i hear you like debugging...).
I'll get back to you tomorrow.

So, with: https://gist.github.com/7f8b2178764483fbd294 (which uses khmer::read_parsers::SeqAnParser), I get the same as with vanilla seqan 1.4, i.e. a blank record at the end. I'll implement your suggestion above @mr-c

Thank you for the follow up.

@mr-c what should we do with the case of an empty sequence?

The original bug is fixed by #897, maybe @mr-c can re-open if you want this to linger as motivation for the SeqAn2  PR/bug

Though, a "known issue" we didn't discuss is that the following will be interpreted as a sequence only file, which it, a) kinda is, and b) well, this is never going to happen.
@seq
ACGCTAGCTA

I checked, and the following is correctly found to be erroneous:
@seq1
ACGT
@seq2
ACGT
+
IIII

So it's only for single reads.

I don't see our job to be a correctness checker all the time but I guess that can go in the known issues list.

:+1:

[ ] add to known issues list and close

documented in #1222


feat(djlint): respect local buffer tabstop config

Formatter now able to use local buffer tabstop config. One of the case is when the tabstop is set by editorconfig.
Close #457

Additionally I would like to add a section in the docs to mention that user could config their formatter to respect local buffer ts like this if the formatter support indentation config through argument.

LGTM thanks!


Explanation of origin O in Tutorial 1 still not clear

Hey there, nice package!
Just wanted to mention that I still got confused by the shortcut O in the Tutorial 1. I saw that you added an explanation recently in #301 but it did not really help me in that moment as I read O as 0 without even thinking.
I had to look at a diff between copypasted code and the code I typed to find out, what was wrong.
I think it is better to write "the letter O" and not "the character O". That's also how Luxor.jl does it in their Tutorial.

Hi @adediego , do you mind opening a PR with that? :wink:
It's definitely welcome :smile:


Free Drop: Error: Prefix not found

Is it just me or are anyone also getting this error: “Error: Prefix not found”
I’m using the latest version of Nova 😊

Looks like Nova Wallet and Free drop

I’ve tried with both drops available

Is it just me or are anyone also getting this error: “Error: Prefix not found”
I’m using the latest version of Nova 😊

Looks like Nova Wallet and Free drop

I think it's related to the paid minting instead of the free drop. As far as I know, the functionality for paid minting on statemint network is not ready.

It's ready checkout polkadot-punks.vercel.app

closing as there will be issue for paid mints


Allow customization of normal mode mappings

I think it can be useful to allow the user to add/remove mappings to the calendar window.
It can be useful if the plugin mappings clashes with some user mappings.
For example, I use the plugin mainly on small windows
nmap <silent> <Leader>cal :Calendar -view=year -split=vertical
         \ -position=topleft -width=27<CR>
nmap <silent> <Leader>caL :Calendar -view=year -split=horizontal
         \ -position=below -height=12<CR>

While it works fine when I check something and just close the calendar window, I can't use some mappings for window (Ctrl-[hjkl]) and tab page navigation (Ctrl-[np]).

I think user can already configure the mappings freely. See :h calendar-problem-4.

Thanks for the speedy answer. Indeed, that approach is far better.
My only regret is that I spent some time writing this alternative solution.
Before starting it I checked :h calendar-key-mappings, then I searched for mappings on that file.
Maybe a reference for calendar-problem-4 could be added at the end of the key mappings section to help someone else searching for that information.

Sorry for that. Thank you for your suggestion. I'll add the link later.


Environment variables

How can we pass environment variables using the same syntax as for "script" jobs?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8ad03c70-4014-135f-ffd3-245c08b3ce32
Version Independent ID: 42a84b54-9ab7-89b2-00b1-efcfc6ba6eef
Content: SSH Deployment task - Azure Pipelines
Content Source: docs/pipelines/tasks/deploy/ssh.md
Product: devops
Technology: devops-cicd
GitHub Login: @RoopeshNair
Microsoft Alias: ronai

@davidstaheli for ssh task

@mboutet are you using the SSH task with a script file, or inline script text?  We don't currently have a way to pass environment variables to the remote machine where the script executes unless you use inline script and set the variables dynamically in the script text.

I was using an inline script like so (note: values modified with dummy values):
- job: Deploy
  displayName: Deploy
  variables:
  - group: SSH keys
  - name: API_IMAGE
    value: myiimage:latest
  - name: AZURE_AGENT_DOCKER_REGISTRY_USER
    value: azureagent
  - name: DOCKER_REGISTRY
    value: myorg.azurecr.io
  pool:
    vmImage: 'ubuntu-latest'
  steps:
  - task: SSH@0
    inputs:
      sshEndpoint: deploymentserver
      runOptions: inline
      inline: |
        docker login -u $AZURE_AGENT_DOCKER_REGISTRY_USER -p $AZURE_AGENT_DOCKER_REGISTRY_PASSWORD $DOCKER_REGISTRY
        # Deployment commands ommitted
        docker logout $DOCKER_REGISTRY
    env:
      AZURE_AGENT_DOCKER_REGISTRY_PASSWORD: $(AZURE_AGENT_DOCKER_REGISTRY_PASSWORD)

So if I use the following instead, it should work?
- job: Deploy
  displayName: Deploy
  variables:
  - group: SSH keys
  - name: AZURE_AGENT_DOCKER_REGISTRY_USER
    value: azureagent
  - name: DOCKER_REGISTRY
    value: myorg.azurecr.io
  pool:
    vmImage: 'ubuntu-latest'
  steps:
  - task: SSH@0
    inputs:
      sshEndpoint: deploymentserver
      runOptions: inline
      inline: |
        export AZURE_AGENT_DOCKER_REGISTRY_USER=$AZURE_AGENT_DOCKER_REGISTRY_USER
        export AZURE_AGENT_DOCKER_REGISTRY_PASSWORD=$AZURE_AGENT_DOCKER_REGISTRY_PASSWORD
        export DOCKER_REGISTRY=$DOCKER_REGISTRY

        docker login -u $AZURE_AGENT_DOCKER_REGISTRY_USER -p $AZURE_AGENT_DOCKER_REGISTRY_PASSWORD $DOCKER_REGISTRY
        # Deployment commands ommitted
        docker logout $DOCKER_REGISTRY
    env:
      AZURE_AGENT_DOCKER_REGISTRY_PASSWORD: $(AZURE_AGENT_DOCKER_REGISTRY_PASSWORD)

@davidstaheli
from OP quesiton, will this work. ```
env:
AZURE_AGENT_DOCKER_REGISTRY_PASSWORD: $(AZURE_AGENT_DOCKER_REGISTRY_PASSWORD)

What is the difference between command and line in a multiline option and use case?

I was able to get this to work by changing the format of the variables in the inline script.
Call the pipeline variable but wrap it in double quotes. Unsure if this would work properly for non-strings.
I'm doing something similar by passing AKV variables and it retains the secret nature of the values correctly in the logs even with debug enabled.
    docker login -u "$(AZURE_AGENT_DOCKER_REGISTRY_USER)" -p "$(AZURE_AGENT_DOCKER_REGISTRY_PASSWORD)" "$(DOCKER_REGISTRY)"

I tried wrapping in double quotes to no avail.
I'm doing something like this:
          inline: |

            export AZP_TOKEN="$(ServiceAccountPAT)"

            echo $AZP_TOKEN

the result of the echo is three asterisks which it has to be passing to my agent. The agent works fine if I hardcode my token in place of the variable and gives me a invalid token when using the variable.

to me it seems the SSH@0 task requires the inline script to explicitly include export HOST_SECRET=$(AZURE_PIPELINE_SECRET) as one of the inline bash commands.
to me it seems that to explicitly include the export statement in this way, it will result in the secret value being passed unmasked, and the secret will appear as readable in logs on the host.
if we adopt the recommended method for passing secrets to the to the generic Azure Pipelines "Task", wherein we pass the env parameters such as -
`

task:
<...>
env:
HOST_SECRET: $(AZURE_PIPELINE_SECRET)
`

to me this pattern seems not to present the secret from env as required for use on the Host.


Fixing CVE updating springframework to 6.1.13

Short description outlining key changes/additions
Fixing CVE updating springframework to 6.1.13
JIRA Ticket Number
Type of change

[ ] Bug fix
[ ] New feature
[ ] Breaking change
[ ] Infrastructure change

Pull request checklist

[ ] I have added unit tests for new code that I have added
[ ] I have added/updated functional tests where appropriate
[ ] The code follows our coding standards

An exhaustive list of peer review checks can be read here

CI: Security warnings found!


Error when running in https://github.com/NixOS/nixpkgs/pull/112625

I've run hammering on https://github.com/NixOS/nixpkgs/pull/112625 (6d2451626d6aa17cec17db1ee8f26835698ab26d) in nixpkgs-review and got the following error: It does not happen on master and I am not sure whats going on here.
nixpkgs-hammer --exclude attribute-ordering --exclude explicit-phases python38Packages.aws-adfs
error: --- TypeError ------------------------------------------------------------------------------------------------------------------------------------------------------- nix-instantiateat: (18:69) in file: /nix/store/0b124ysqxb9bg3r5zfwdzflf44gvm54h-nixpkgs-hammer/overlays/python-include-tests.nix

    17|           hasDoCheckFalse = (drvArgs ? doCheck) && (drvArgs.doCheck == false);
    18|           hasPytestCheckHook = drvArgs ? checkInputs && lib.any (n: n.name == "pytest-check-hook") drvArgs.checkInputs;
      |                                                                     ^
    19|           hasPythonImportsCheck = drvArgs ? pythonImportsCheck;

value is null while a set was expected
(use '--show-trace' to show detailed location information)
Traceback (most recent call last):
  File "/nix/store/0b124ysqxb9bg3r5zfwdzflf44gvm54h-nixpkgs-hammer/bin/.nixpkgs-hammer-wrapped", line 295, in <module>
    main(args)
  File "/nix/store/0b124ysqxb9bg3r5zfwdzflf44gvm54h-nixpkgs-hammer/bin/.nixpkgs-hammer-wrapped", line 223, in main
    overlay_data = nix_eval_json(all_messages_nix, args.show_trace)
  File "/nix/store/0b124ysqxb9bg3r5zfwdzflf44gvm54h-nixpkgs-hammer/bin/.nixpkgs-hammer-wrapped", line 65, in nix_eval_json
    result = subprocess.check_output(
  File "/nix/store/4jf77mhzlwhk92rh32ba0qff0nc7vj3i-python3-3.8.7/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/nix/store/4jf77mhzlwhk92rh32ba0qff0nc7vj3i-python3-3.8.7/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['nix-instantiate', '--strict', '--json', '--eval', '-']' returned non-zero exit status 1.

That means a package has null in checkInputs. Which is discouraged in nixpkgs but it the checks should handle it. I cannot reproduce with  https://github.com/NixOS/nixpkgs/pull/112625 (746c3368eee at the time I checked it out) nor with https://github.com/fabaff/nixpkgs/archive/6d2451626d6aa17cec17db1ee8f26835698ab26d.tar.gz but the following patch can reproduce it:
diff --git a/pkgs/development/python-modules/aws-adfs/default.nix b/pkgs/development/python-modules/aws-adfs/default.nix
index ce4c6f78926..1f381f62901 100644
--- a/pkgs/development/python-modules/aws-adfs/default.nix
+++ b/pkgs/development/python-modules/aws-adfs/default.nix
@@ -23,7 +23,7 @@ buildPythonPackage rec {
   # Required for python3 tests, along with glibcLocales
   LC_ALL = "en_US.UTF-8";

-  checkInputs = [ glibcLocales pytest pytestrunner pytestcov mock ];
+  checkInputs = [ glibcLocales pytest pytestrunner pytestcov mock null ];
   propagatedBuildInputs = [ botocore lxml requests requests-kerberos click configparser fido2 ];

   meta = with lib; {


Trt version upgrade

Description
Upgrade TensorRT to latest version. I also updated the docker container for JNI build in the action, and added some documentation/resources for local tensorrt + djl development help.

Codecov Report

Merging #1758 (21a107e) into master (bb5073f) will decrease coverage by 1.61%.
The diff coverage is 70.42%.

@@             Coverage Diff              @@
##             master    #1758      +/-   ##
============================================
- Coverage     72.08%   70.47%   -1.62%
- Complexity     5126     5550     +424
============================================
  Files           473      527      +54
  Lines         21970    24697    +2727
  Branches       2351     2680     +329
============================================
+ Hits          15838    17405    +1567
- Misses         4925     5958    +1033
- Partials       1207     1334     +127

Impacted Files
Coverage Δ

api/src/main/java/ai/djl/modality/cv/Image.java
69.23% <ø> (-4.11%)
:arrow_down:

...rc/main/java/ai/djl/modality/cv/MultiBoxPrior.java
76.00% <ø> (ø)

...rc/main/java/ai/djl/modality/cv/output/Joints.java
71.42% <ø> (ø)

.../main/java/ai/djl/modality/cv/output/Landmark.java
100.00% <ø> (ø)

...main/java/ai/djl/modality/cv/output/Rectangle.java
72.41% <0.00%> (ø)

...i/djl/modality/cv/translator/BigGANTranslator.java
21.42% <ø> (-5.24%)
:arrow_down:

...odality/cv/translator/BigGANTranslatorFactory.java
33.33% <0.00%> (+8.33%)
:arrow_up:

...nslator/InstanceSegmentationTranslatorFactory.java
14.28% <0.00%> (-3.90%)
:arrow_down:

.../cv/translator/StyleTransferTranslatorFactory.java
40.00% <ø> (ø)

.../ai/djl/modality/cv/translator/YoloTranslator.java
8.33% <0.00%> (-0.50%)
:arrow_down:

... and 412 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3fbf2de...21a107e. Read the comment docs.


Update ink to 4.2.0

With this new version we can use the gap flexbox param to add spacing among blocks.

Good job 👏🏼


Xref editing

@oliver---- Pls review and merge. There's one bigger glitch related to the modal closing as you toggle reference targets. I wanted to discuss a some other UX options but we can do that in a separate iteration.
The basic UX principle so far is that if you open the ref target editor the selected targets are shown on top. As you select other targets they will be where they are until you close the dialog and reopen (then they'll appear on top again).

Need to take a close look.


Signatures of Artifacts in Maven Central Reference Unpublished PGP Key

What version of OpenCensus are you using?
0.31.0
What JVM are you using (java -version)?
It's not really applicable, but we are using...
openjdk version "17.0.2" 2022-01-18 LTS
OpenJDK Runtime Environment Zulu17.32+13-CA (build 17.0.2+8-LTS)
OpenJDK 64-Bit Server VM Zulu17.32+13-CA (build 17.0.2+8-LTS, mixed mode)

What did you do?
We are using dependency verification in Gradle and we ran into an issue. Gradle cannot find your public key on any of the default key servers.
I am looking at the signature of the POM file via gpg -vv and it references a key with an id of AC7A514BC9F9BB70.
What did you expect to see?
I see that your documentation for releasing instructs maintainers to publish their public key, but I manually checked all the common key servers and none of them have AC7A514BC9F9BB70.
It should have been published on at least one (ideally all) of Gradle's default key servers and ideally on your website as well.
In the meantime, it is easy to workaround this issue in Gradle by explicitly ignoring the key.
<ignored-keys>
   <ignored-key id="ac7a514bc9f9bb70" reason="Key couldn't be downloaded from any key server"/>
</ignored-keys>

Thanks for flagging this. I was a bit surprised, because I thought I'd uploaded the key and that OSSRH needed to be able to access the key in order to validate the release. In any case, I uploaded the key to keys.openpgp.org (again?) just now, and verified that https://keys.openpgp.org/search?q=AC7A514BC9F9BB70 returns a meaningful result.


python3Packages.lmnotify: init at 0.0.4

Motivation for this change
Add missing python package needed for home-assistant LaMetric integration.
This project has not tests.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Relase notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[X] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 128468 at df4894bf run on aarch64-linux 1

2 packages built successfully:

python38Packages.lmnotify
python39Packages.lmnotify


Remove react-router-redux in favor of connected-react-router

react-router-redux is deprecated and no longer maintained. (https://github.com/ReactTraining/react-router/tree/master/packages/react-router-redux)
They recommend using connected-react-router (https://github.com/supasate/connected-react-router) instead.

Can you rebase this against v0.16.0. Will make it easier to review


What is KProcessHacker

The installer asks me if I want to install KProcessHacker and that it's not recommended.
What is KProcessHacker?

What is KProcessHacker?

The kernel driver.

that it's not recommended.

It's not recommended because some games and programs scan for our driver and forcefully reboot your machine. There are also companies using it as an end-run around GDPR and keeping logs about users with the driver installed.
So it's not recommended for now.

if I want to install KProcessHacker

The stable version always installs the kernel driver. That option is for allowing connections from processes without debug privileges such as when Process Hacker hasn't been elevated to administrator.
The nightly version recently started disabling the driver by default. The option is no longer available without administrator privileges.
I'd recommend using the nightly version since it has more features to avoid interference by third party software:
https://processhacker.sourceforge.io/nightly.php

Shame that we can't modify our own kernel without games making a fuss over it


DeleteStreamAsync Not Exposed to Package Consumers

Describe the issue
Currently the DeleteStreamAsync() functionality is not exposed to the package consumer. It is unclear whether this exclusion is intentional or an oversight.
Suggestion 1
Move DeleteStreamAsync() with implementation from IEventStoreManagementClient to IEventStoreClient.
Suggestion 2
Make IEventStoreManagementClient public and move the internal access modifier to the unimplemented methods.
I'm happy to help with code changes. I just need to know which approach is preferred :)

I think suggestion 1 is the optimal way, and we should remove the IEventStoreManagementClient from the package entirely.
@Anders-Toegersen, I would be grateful if you would implement the changes.

@TomMalow this would also help you with the event-store data migration.

Cheers and happy holidays


Fix cell left side actions display

Hello,
In some displays, such as in my asus zenbook's 13.3'' display or the one of my samsung galaxy s8+,  the left side actions (the plus sign to create a new cell, the arrow to show or hide a cell, and the pin to pin cells) are almost hidden and are very difficult to click or tap

Hi Vicente! Thanks for the feedback. If you want to discuss the user interface, please head over to our forums: https://talk.observablehq.com/
We’re aware of some of the current user interface limitations on small screens and are working to improve the design. (On very small screens, you can click the blue pencil icon in the bottom-right corner to cause the left margin to slide out for easier clicking.)

Thank you very much for your quick reply. I'll go to the forum. Thanks for
the workaround!
On Mon, 19 Nov 2018, 12:50 Mike Bostock<EMAIL_ADDRESS>wrote:

Hi Vicente! Thanks for the feedback. If you want to discuss the user
interface, please head over to our forums: https://talk.observablehq.com/
We’re aware of some of the current user interface limitations on small
screens and are working to improve the design. (On very small screens, you
can click the blue pencil icon in the bottom-right corner to cause the left
margin to slide out for easier clicking.)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/observablehq/notebook-stdlib/issues/74#issuecomment-439940267,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AK5qKy6_BImXUx9V1DnqCCG6cxYy-V15ks5uwtM3gaJpZM4YpX4Y
.


fix: Bash functions should exit on error

The run command wrapper was silently ignoring errors. This commit ensures that a failed command will exit, ensuring that we don't accidentally ignore errors anymore.

Run shouldn't fail rather the script including utils should have set -eu -o pipefail as default.

That's not what was observed in #1559
You can easily reproduce by applying this diff:
diff --git a/manifests/k8s/config/exporter/kustomization.yaml b/manifests/k8s/config/exporter/kustomization.yaml
index 55514604..ffcbfa12 100644
--- a/manifests/k8s/config/exporter/kustomization.yaml
+++ b/manifests/k8s/config/exporter/kustomization.yaml
@@ -3,7 +3,7 @@ resources:
   # - openshift_scc.yaml
   - exporter.yaml
 #  uncomment this line if prometheus deployed
-#  - prometheus_common_service_monitor.yaml
+# - prometheus_common_service_monitor.yaml
 #  - prometheus_common_rules.yaml
 #  - prometheus_high_granularity_rules.yaml

Then running make build-manifest OPTS="CI_DEPLOY PROMETHEUS_DEPLOY"
This will make the run kustomize edit ... commands exit with error code 1... but somehow that error is being ignored 🤔
In other words, applying set -eu -o pipefail (which is present in hack/build-manifests.sh) doesn't appear to be operating as expected.

My main objection to this patch is that with die being called from run the script can exit at any point. The only purpose of run is to print the command it runs and nothing more.
So ... as a quick test, I removed all run from hack/build-manifest.shfunctionbuild_manifest` and then ran the script and it exhibits the same erroneous behaviour.
	build_manifest || {
		fail "Fail to build the manifests"
		return 1
	}

Doing a lot more than what is intended. Please allow me some time to look into this.

@dave-tucker I guess the fix really is to just not use any if or 'fn || failconstruct inmain` . IOW,
main() {
	export PATH="$PROJECT_ROOT/tmp/bin:$PATH"

	local opts=$1
	shift
	ensure_all_tools

	for opt in ${opts}; do
		info "Setting $opt as True"
		eval "$opt=true"
	done

	info "move to untrack workspace ${MANIFESTS_OUT_DIR}"
	run rm -rf "${MANIFESTS_OUT_DIR}"
	run mkdir -p "${MANIFESTS_OUT_DIR}"
	run cp -r manifests/k8s/config/* "${MANIFESTS_OUT_DIR}"/
	build_manifest
}

should fix it


[Do Not Merge] feat: Managed Cluster Cache

Fixes #113
Modifications
A live cluster cache when query always ensured its up to date before returning the state.
Verification
Unit test. (WIP)
Note this PR is only for reference ATM. It needs to be merged together with the other automated syncing refactor PRs.

Closed with https://github.com/numaproj-labs/numaplane/pull/124


Add MaskWriteRegister

This is an implementation of the masked write register function code (0x16).

Thank you! Would you mind adding a changelog entry? I guess this could be released as v0.5.5.
LGTM. Anyone else?

Thank you! Would you mind adding a changelog entry?

Done!

Thanks for the hint. This is indeed a breaking change, so we better release it as v0.6.0. Version numbers are not a scarce resource ;)


Wrong info log

The XsuaaServicesParser log a wrong info log @line 64:
if (attributeString.isPresent()) {
logger.info("XSUAA VCAP_SERVICES has no attribute with name '{}'.", name);
}
... which means that if the attribute is present the log states the opposite

Dear Ralph, thanks for reporting! It is already fixed in 1.7.0version.


[release/0.1] Add missing commits

@dgn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_sail-operator_release-0.1
f5df7fb8a190161728c381289d224380dd187bc4
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

/test gencheck

@dgn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_sail-operator_release-0.1
f5df7fb8a190161728c381289d224380dd187bc4
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Maybe https://github.com/istio/test-infra/pull/5464 will fix the gen issue

fix is in https://github.com/istio-ecosystem/sail-operator/pull/269

@dgn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_sail-operator_release-0.1
f5df7fb8a190161728c381289d224380dd187bc4
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.


[FormState] Adds externalErrors prop

What does this PR do?
This PR adds an externalErrors prop to FormState. This allows you to initialize the form with external errors. This API also enables the possibility of async errors.
Example
Here's how you would initialize the form with external errors. Notice that external errors are merged with remote errors and are returned by the errors array.
External errors will also be passed down to corresponding fields if a field is specified.
Form state will not manage your external errors, you will need to clear them yourself if necessary.
class MyComponent extends React.Component {
    state = {
        externalErrors: [{message: 'Example error', field: 'foo'}]
    }

  render() {
    const {externalErrors} = this.state;

    return (
      <FormState
        initialValues={{foo: ''}}
        externalErrors={externalErrors}
        onSubmit={async (formDetails) => {
          const response = await submitFormMutation(formDetails);

           if (response.errors) {
               // we still need to clear the errors ourselves because formstate
              // doesn't know what they represent or when to clear them
               this.setState({externalErrors: null});
               return response.errors;
           }
        }}
      >
        {({fields, errors}) => {
           return (
             <Banner>
              {/* the externalErrors get seamlessly added to the errors array */}
               {errors.map(error =>
{error.message}
)}
             </Banner>
               {/* and passed down to matching fields */}
              <TextField {...fields.foo} />
           )
        }}
      </FormState>
    )
  }
}

Here's how this API can enable async errors.
class MyComponent extends React.Component {
    state = {
        externalErrors: []
    }

  render() {
    const {externalErrors} = this.state;

    return (
      <FormState
        initialValues={{foo: ''}}
        externalErrors={externalErrors}
        onSubmit={async (formDetails) => {
          const response = await submitFormMutation(formDetails);

           if (response.errors) {
               return response.errors;
           }
        }}
      >
        {({fields, errors}) => {
           return (
             <Banner>
               {errors.map(error =>
{error.message}
)}
             </Banner>
              <TextField {...fields.foo} />
              {/* this would return the same type of error objects */}
               <RemoteValidation
                   fields={fields}
                   url={someUrl}
                  onFail={(externalErrors) => this.setState({externalErrors}}
                />
           )
        }}
      </FormState>
    )
  }
}

Cases that I've tried to consider

Initializing form with external errors
Mixing remote errors with external errors
Updating external errors when remote errors are present

Proof it works 😄
This screenshot shows a server error and an external error coexisting.

🎉


Keyboard issues

Fixes #110 and #108.

Keyboard shortcuts using letters are now declared using actual characters (vs keycode) and a flag to accept shift key
Letter-based shortcuts are not impacted by caps lock anymore
Save has been moved to keydown and works on both mac and windows with CTRL/CMD
Removed duplicate code in handling the save notifications

Had a play around and looks great. Thanks!


Sort Prizes/gifts order

For all times that prizes/gifls are listed (on volunteers page), sort Group -> Value -> Name
Discussion:
User can put in the groups that are "most important" first and use value/name to sort the rest

Promo ( free with a given number of hours items) where do those go? First I presume?


CASMCMS-8968: Add retries of most commands in IMS import/export tools

Adds limited retries to appropriate commands in the IMS import/export tools.
Backports forthcoming

/backport release/1.3 release/1.5 release/1.6


Feature add agenda

Add a border on the left and apply styles to tasks and deadlines

Add a border on the left and apply styles to tasks and deadlines

Ми зразу в main пушим, чи це випадково і треба в dev?

By chance, my fault, it was supposed to be in the development


"ctr images pull" debugging should show the underlying hosts where layers are being pulled from

What is the problem you're trying to solve
We are seeing slow image pulls from "ctr images pull" from certain global locations to ghcr.io (Github Container Registry) hosted images. But IPv4 and IPv6 ping times to ghcr.io look fine from all locations.
It turns out that some ghcr.io assets, specifically layers, are served through a CDN at pkg-containers.githubusercontent.com . Inspection of IPv4 routes to that host look fine globally, but a number of locations have "scenic routes" to IPv6 addresses for that content. The resulting bad routing leads to very slow or stopped downloads.
None of the debug options I could figure out for "ctr images pull" exposed the "pkg-containers.githubusercontent.com" destination for image pulls, which made debugging much harder.
The feature request is to add a flag that would print out this detail to assist in debugging.
Describe the solution you'd like
Add a debug flag that prints out the domain name and IPv4 or IPv6 address of the host from which image layers are being retrieved.
Additional context
Discussion here:
https://hachyderm.io/@w8emv/111942173630607597
https://mastodon.online/@karppinen/111958791252380328

We added both trace and dump options from the HTTP Go API that you can use with ctr i pull:
   --http-dump                       dump all HTTP request/responses when interacting with container registry
   --http-trace                      enable HTTP tracing for registry interactions

I've used them before and gained a lot of insight into the HTTP transactions; can you let me know if either of those are valuable for what you are trying to accomplish?

Thanks @estesp .
I see this header in my trace
INFO[0003] Host: pkg-containers.githubusercontent.com
which starts to look helpful, but there's nothing in the trace that I can recognize that would tell me whether that host is being reached by IPv4 (where routing is find and everything is peachy) or IPv6 (where routing is fubar and nothing is right).

Might have to get the Go DNS debug resolver involved then:
export GODEBUG=netdns=2

That should dump the name->IP resolver decisions from Golang and help you find out where it is doing IPv4 vs. IPv6 resolution?


Change build/target SDK ver (26->28), min SDK ver (15->16), and other dependency versions.

This PR is the followup to pull request #229 : "Prep work for updates of SDK (v26->v28) and other dependencies". It also follows chronologically after my other pull requests #230 : 'Solve "leaked Intent" exception.' and #231 : "Fix for APK gen exception: com.android.dex.DexException".  Please merge those first to avoid merge conflicts, build errors, and other potential problems (or perhaps just merge this to pull all changes and ignore+close the others if you don't need the metadata).
Notable caveat from the commit message:

This will bump the minSdkVersion from 15 to 16. According to
https://developer.android.com/about/dashboards/index.html
this will make CarStream no longer work on Android 4.0.4 and
earlier devices. 4.1.x and later should still work. This
represents about 0.6% of the active devices in the last
7 days (0.3% for the 4.0 family plus 0.3% for the 2.3 family).
One of those is probably mine, and I can't imagine actually
wanting to use the thing to connect to Android Auto; maps
would be sooooo slow, assuming any of it would work at all.

Hope this helps.
Thank you for making this app!

@chadjoan Can you please update the .travis.yml too to 28.0.3. Without that travis is not able to build.

I tried.
The error mentioning missing licenses is gone now. But...
The Travis script is now producing AAPT errors about the " inner element", which should have already been fixed by the commit that created the override_id_bodies.xml file. However, in this case the error message is slightly different. It is mentioning aauto-sdk-v4.5 specifically instead of just some generic project directory for merging machine generated text/code. I followed the parent commits from the commit that Travis CI tested to confirm that my earlier fix for this type of error message is still present in the codebase. And it is. So I don't think it's a problem with Travis CI building the wrong code/commit.
Prompted by the error message and noticing that build.gradle has a line "compile 'com.github.martoreto:aauto-sdk:v4.5'" in it, I went to https://github.com/martoreto/aauto-sdk to look at the res/values/ids.xml file to see what things I might need to add to override_id_bodies.xml (the error message in the Travis output seems to lack this information, unfortunately). Well, apparently that project was taken down, and a message was left: "This was an unofficial SDK for Android Auto. ... Google asked me to take it down, citing the resulting abuse." So I'm not really sure how this works in the first place: CarStream appears to depend on something that no longer exists.
Google searching aauto-sdk (along with other terms that I forget now) brought me to bluenazgul/aauto-sdk on github. This one does seem to be populated with code, but I have no idea if this is an accurate representation of what Travis CI is trying to integrate with. I suspect that it is different, actually: instead of res/values/idx.xml it has res-extra/values/ids.xml (note the "-extra" after "res") and the contents of that file seem like they would be perfectly compatible with the newer version of the build system.
I can't reproduce the error on my system.
I am rebuilding (successfully) at commit 34ee553, the latest commit in this PR as of this writing. Travis CI  seems to be building a different commit, 5a9e614, which is a child of 34ee553 and some other commit. I don't know how to retrieve Travis CI's commit, so right now I can't test it directly.
Given that I can't repro Travis CI's new error, and the error messages lack the identifiers that I would need for implementing the typical workaround, I am not sure how to proceed at the moment.
Let me know if you have any ideas.
In case it helps, here are a few things I could use help with:

How to retrieve files from the Travis CI test instance, or even better, just have access more detailed build logs. Ultimately I want to get to the  identifiers/names that are involved in the problem.
Why was I able to build CarStream when it depends on a project, martoreto/aauto-sdk, that seems to have been removed from the internet? Do we need this project specifically, or are there feasible alternatives?
How to fetch the commit/branch that Travis CI is using (so that I can attempt to repro with my own system/environment).

It seems like you've been away from this project for a long time, so thank you for responding to my pull requests!  And, of course, for writing this app in the first place :)

How about cloning one of the repos from https://github.com/martoreto/aauto-sdk/network/members and then making changes to it and adding that as a dependency ?

Good idea. I'll try that when I get back to working on this.
I didn't know about that view in github. That's pretty handy.

Update: I'm having issues getting aauto-sdk to build as a submodule.
I made a Stack Overflow question in the hopes that someone more experienced with Gradle can help:
https://stackoverflow.com/questions/56535100/with-gradle-how-do-i-use-a-dependency-subproject-that-defines-a-bunch-of-tasks


use App_Packages convention

fixes #10
i also removed ExampleApiApprovalTest from the package. i think doco is a better place for that and it will break for anyone not using xunit

note the version is now hard coded in a diff single location.

also your website is case sensitive and doesnt do redirects

@JakeGinnivan bump :)

Been on a cruise ship so had no internet. Will probably sort when we get to Miami, otherwise early next week when back in London.
Sent from my Windows Phone

From: Simon<EMAIL_ADDRESS>Sent: ý04/ý03/ý2015 21:17
To<EMAIL_ADDRESS>Cc: Jake<EMAIL_ADDRESS>Subject: Re: [ApiApprover] use App_Packages convention (#11)
@JakeGinnivanhttps://github.com/JakeGinnivan bump :)
—
Reply to this email directly or view it on GitHubhttps://github.com/JakeGinnivan/ApiApprover/pull/11#issuecomment-77293655.

LOL. no worries. have fun


Collect and document implementors of the Provided Service Duck Type

Collect and document implementors of the Provided Service duck type.

One implementor we're interested in is Strimzi.  @AndrewJSchofield had started a proposal, which we should update to be a duck type if possible.

Created a feature request to MongoDB: https://feedback.mongodb.com/forums/924355-ops-tools/suggestions/44030625-support-service-binding-specification-for-kubernet

Created a feature request to Redis: https://github.com/RedisLabs/redis-enterprise-k8s-docs/issues/186

@arthurdm and @baijum Are you aware of any implementors yet?  I've got one on my side and want to make sure any that have been successfully implemented are included.

I know that both of the Runtime Component Operator and Open Liberty Operator teams are aiming for an update this month to be able to expose a workload (app) as a bindable service endpoint via the duck type.  Will need to circle back on this.


Get pencil to write new setting

Ref #129861
By adding the ignoreDirtyFile flag, when following the steps in the issue above, changing a setting with the pencil while the file is dirty results in the file saving and then the selected value being applied.

changing a setting with the pencil while the file is dirty results in the file saving

I don't think this should save the file automatically. Can it write into the dirty file and keep it dirty?

Yeah, we shall not be saving automatically. The action that is broken here is Save & Retry. This action shall save the file and retry the operation which used to work before.

@sandy081 the current implementation makes it so that the new value is written to the file, but the file remains unsaved.
With this implementation, we don't get the previous issue where the Save & Retry dialog would pop up.

I am not sure if force saving the file is expected. May be user might have some unsaved changes which he/she does not want to save. Hence we show this prompt to save the file first and retry. User can instead ignore this prompt and revert the changes in the file and retry the operation.

@sandy081 I updated the PR contents and https://github.com/microsoft/vscode/pull/137765#issue-787450051 to hopefully be clearer. I'm not force-saving the file.

@rzhao271 I am seeing following behavior with this fix and it seems Save and Retry action is just updating the dirty model without saving.

Save and retry shall save the file and retry the operation as below

This behavior was broken by me when redirecting user settings writes to main process. Created a PR that brings back this behaviour. Please take a look - https://github.com/microsoft/vscode/pull/138657

Closing in favour of https://github.com/microsoft/vscode/pull/138657


Don't add the internal publication of the original event twice

When one duplicates an event, the duplicated event contains two internal publications. One of them references the original event. (see attached screenshot)

I guess this is relevant for the releases 8.x and 7.x  either

If this is relevant for 7.x, can you re-direct your pull request towards r/7.x?

@lkiesow Not sure what to with the failed Travis build

@lwillmann you should be able to restart the failed job. I've restarted it for now, but the error looks to me like something that would happen again.

@gregorydlogan thanks for the restart. It happened again as predicted. Is there something I can do to fix this?

Take a look at the failed job, right here.  You can also get to it via Details on the Travis failure.

and the issue basically seems to be that Travis uses the build instructions from develop. Likely because the patch was pushed when this pull request was still pointing at develop. My guess would be that you can just touch (git commit --amend --no-edit) and push it again to fix this. Or Greg can ignore the test ;-P

Looks like this worked.

@lwillmann You've succeeded in keeping the duplicate from keeping the original publication, but in testing against r/7.x I'm seeing the duplicate with no publications at all :)

@gregorydlogan Hmmm ... I've tested it again and for me it seems to work. There is only one internal publication as you can see in the screenshot.

External publications of a duplicated event have never been published automatically as described in the documentation of the WOH: If the original event was already published, the duplicate won't be published right away. The user will have to publish it manually when he is done editing it.


Implement tokens payment

Creation and sending of txs using tokens
Minor fixes
Set version 2.2.0

Pull Request Test Coverage Report for Build 460

240 of 256 (93.75%)  changed or added relevant lines in 21 files are covered.
3 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.7%) to 81.65%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

safe_relay_service/relay/models.py
7
8
87.5%

safe_relay_service/relay/tasks.py
5
6
83.33%

safe_relay_service/tokens/urls.py
0
1
0.0%

safe_relay_service/tokens/models.py
5
7
71.43%

safe_relay_service/relay/views.py
51
54
94.44%

safe_relay_service/gas_station/urls.py
0
4
0.0%

safe_relay_service/relay/serializers.py
20
24
83.33%

Files with Coverage Reduction
New Missed Lines
%

safe_relay_service/tokens/urls.py
3
0.0%

Totals

Change from base Build 429:
0.7%

Covered Lines:
1455

Relevant Lines:
1782

💛  - Coveralls


Make Dan's quote database exportable to an html or php or text file

This would be added to your web panel, and could export it to a php or html or text file.
if it were php or html it would look like:
< p >Check markiplier's twitter< /p >
< p > just get these robotic appliances and hook $user onto the api< /p >
Or if text file separate each quote by linebreak
Check markiplier's twitter
just get these robotic appliances and hook $user onto the api
You could also have it add a number to each quote so

Check markiplier's twitter
just get these robotic appliances and hook $user onto the api

Or maybe even the date the quotes were added into the database.
You could also have it order by first to last by date added or maybe last to first, or maybe allow the user to choose the sorting, so many possibilities!

..I'm failing to see how this would be handy, other than to reuse it in other bots.

Poetic Readings?
[Maybe add a way to turn his gibberish into a poem?]
Exporting it to other bots like you said.
Maybe for reading for fun, or displaying on a webpage on dan stats?


Fix bool and null handling

This PR fixes bool and null handling.

Disclaimer: This review was made by a crew of AI Agents.
Code Review Comment for PR #1771 - Fix Boolean and Null Handling
Overview
This PR modifies the _validate_tool_input method in src/crewai/tools/tool_usage.py, enhancing the handling of boolean and null values during tool input validation.
Positive Changes

Separation of Handling: The logic for handling null ("null") and boolean values ("true" and "false") is separated, improving code clarity and semantic correctness.
Pythonic Adjustments: Converts "null" to Python's None, aligning with Python conventions.
Capitalization of Booleans: Boolean values are capitalized ("true" → "True" and "false" → "False"), adhering to Python's built-in bool type.

Suggestions for Improvement
1. Use a Mapping for Boolean and Null Values
To enhance clarity and maintainability, consider using a dictionary for mapping string inputs to their Python equivalents:
BOOLEAN_MAPPING = {
    "true": "True",
    "false": "False",
    "null": "None"
}

elif value.lower() in BOOLEAN_MAPPING:
    value = BOOLEAN_MAPPING[value.lower()]

2. Improve Comment Accuracy
Update comments to accurately reflect the logic and intent of the code:
elif value.lower() in ["true", "false"]:  # Check for boolean values only
    value = value.lower().capitalize()
elif value.lower() == "null":  # Handle null values separately
    value = "None"

3. Consider Type Hints
Adding type hints can improve documentation and code clarity:
def _validate_tool_input(self, tool_input: str) -> str:
    # ... existing logic ...
    value: str = value.lower().capitalize()
    # ... continue with the function ...

Potential Issues to Watch For

Input Validation: Ensure that the input value is not None before calling .lower() to avoid potential AttributeError.
Error Handling: Implement error handling for edge cases in string manipulation to improve robustness.

Testing Recommendations
To ensure the robustness of the changes, consider adding unit tests for the following cases:

Mixed case inputs (e.g., "True", "FALSE", "nUll").
Edge cases such as empty strings or special characters within boolean strings.
Explicit tests for handling None values.

Conclusion
The modifications in this PR lead to a more Pythonic and semantically correct approach to handling boolean and null values. However, the suggested improvements could enhance the maintainability and robustness of the input validation process. Overall, this update is a step in the right direction for improving the codebase.

This comment encapsulates detailed insights from the provided analysis with actionable recommendations and emphasizes the importance of testing the new changes effectively.


use semaphore instead of inmemory queue for processing system tasks

Pull Request Test Coverage Report for Build 3920

88 of 120 (73.33%)  changed or added relevant lines in 9 files are covered.
20 unchanged lines in 5 files lost coverage.
Overall coverage decreased (-0.1%) to 68.967%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

core/src/main/java/com/netflix/conductor/core/events/SimpleActionProcessor.java
0
2
0.0%

core/src/main/java/com/netflix/conductor/core/utils/SemaphoreUtil.java
11
14
78.57%

core/src/main/java/com/netflix/conductor/metrics/Monitors.java
0
3
0.0%

core/src/main/java/com/netflix/conductor/core/execution/tasks/SystemTaskWorkerCoordinator.java
31
38
81.58%

core/src/main/java/com/netflix/conductor/core/execution/tasks/SystemTaskExecutor.java
27
44
61.36%

Files with Coverage Reduction
New Missed Lines
%

core/src/main/java/com/netflix/conductor/core/execution/tasks/IsolatedTaskQueueProducer.java
1
81.82%

core/src/main/java/com/netflix/conductor/core/execution/tasks/SystemTaskWorkerCoordinator.java
1
83.33%

core/src/main/java/com/netflix/conductor/core/utils/QueueUtils.java
2
90.32%

cassandra-persistence/src/main/java/com/netflix/conductor/dao/cassandra/CassandraEventHandlerDAO.java
8
69.86%

cassandra-persistence/src/main/java/com/netflix/conductor/dao/cassandra/CassandraMetadataDAO.java
8
55.79%

Totals

Change from base Build 3913:
-0.1%

Covered Lines:
11732

Relevant Lines:
17011

💛  - Coveralls


Build fix

@lubeltra,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


Lint code automatically

Use a linter that runs automatically for all code push to this origin.
This is dependent on #1.

Automatic linting is working with commit 1be9fc5e803def42d985fb422b2522eeb3b8c4a2.


Add Metro configuration instructions for React Native

Summary
This PR updates the README.md file in the matter.js-react-native package to include necessary Metro bundler configuration instructions for React Native projects.
Changes

Added a section to README.md detailing the required Metro configuration for resolving matter.js packages in React Native.

Why
Without this configuration, users may encounter module resolution errors when integrating matter.js with React Native. This update aims to provide clear guidance to prevent such issues.
Related Issues

Fixes module resolution error as discussed in the project’s discussions.

Checklist

[x] Updated documentation
[x] Verified changes locally

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.MAGLY_ZHR seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Uncaught (in promise): ReferenceError: WebMidi is not defined

I just installed the latest magentamusic npm package for my Angular project.
I've got same error; I don't use the ts files directly. In Angular typescript project, I cannot use magentamusic.js script javascript file because Angular needs Typescript and d.ts type file.
ERROR in node_modules/@magenta/music/es5/core/player.d.ts(51,14): error TS2503: Cannot find namespace 'WebMidi'. node_modules/@magenta/music/es5/core/player.d.ts(52,32): error TS2503: Cannot find namespace 'WebMidi'. node_modules/@magenta/music/es5/core/recorder.d.ts(34,22): error TS2503: Cannot find namespace 'WebMidi'. node_modules/@magenta/music/es5/core/recorder.d.ts(35,24): error TS2503: Cannot find namespace 'WebMidi'. node_modules/@magenta/music/es5/core/recorder.d.ts(39,32): error TS2503: Cannot find namespace 'WebMidi'.
Even though magentajs music core sources are written in Typescript having d.ts, the output files built doesn't look up d.ts correctly. IDK . Maybe WebMidi package hasn't been installed after installing npm @magenta/music package.
I think Angular cannot find the package installed.
If I install "@types/webmidi", this error is gone. But another error comes out !
"core.js:15723 ERROR Error: Uncaught (in promise): ReferenceError: WebMidi is not defined
ReferenceError: WebMidi is not defined"
I have to install webmidi package separately although magenta music package uses it;
import WebMidi from 'webmidi'; Also I added the following line in angular.json config file
"scripts": [
              "./node_modules/webmidi/webmidi.min.js"
            ],

Please check out the how magentajs music package exports webmidi and types in the package.json of @magenta/music!
I am using pretty much latest Chrome browser and magenta package on MacOSx

I don't think we publish any types with magenta (and honestly, I don't know enough about types to check if we do). @cannoneyed do you have any ideas?

I'm also running into this problem. As of now, I cannot use @magenta/music in an Angular app

I've made some changes to the es6 modules in https://github.com/tensorflow/magenta-js/pull/365 and i think it should work better now. I've tested it out in this repo: https://github.com/notwaldorf/example-magenta-in-ts. I know it's not Angular, but it is using Typescript, and I could repro this problem in that repo. Give it a try and let me know!

Closing this, since I think it was fixed and there's been no updates on this issue


Astrobee 3-DOF Compliant Perching Arm

Hi, My simple question.
Will Astrobee 3-DOF Compliant Perching Arm data be open source?

Thank you for your reply.
Can we use 3D-Printable data? For example, STL.

Sorry, we are unable to share the STL files.

I understood. Thank you for reply.


Use map[string]struct{} for compression map in Len

map[string]int requires 8 bytes per entry to store the unused position information.
~I couldn't get any clean benchmarks from this, but map[string]struct{} definitely uses less memory than map[string]int.~
This does unfortunately make the related test cases slightly weaker as they no longer test that the Len code is finding compression points in the correct spot.
Updates #709.
Edit: I was able to get some better benchmarks that show a very notable improvement in memory use.
$ benchstat {old,new}.bench
name                       old time/op    new time/op    delta
MsgLength-12                  329ns ± 0%     328ns ± 0%   -0.27%  (p=0.040 n=9+9)
MsgLengthNoCompression-12    9.37ns ± 1%    9.34ns ± 2%     ~     (p=0.099 n=10+10)
MsgLengthPack-12             2.26µs ± 9%    2.36µs ±23%     ~     (p=0.780 n=9+10)
MsgLengthMassive-12          63.1µs ± 8%    53.0µs ± 3%  -15.89%  (p=0.000 n=10+10)

name                       old alloc/op   new alloc/op   delta
MsgLength-12                  32.0B ± 0%     32.0B ± 0%     ~     (all equal)
MsgLengthNoCompression-12     0.00B          0.00B          ~     (all equal)
MsgLengthPack-12               896B ± 0%      896B ± 0%     ~     (all equal)
MsgLengthMassive-12          20.2kB ± 0%    14.8kB ± 0%  -26.74%  (p=0.000 n=8+9)

name                       old allocs/op  new allocs/op  delta
MsgLength-12                   1.00 ± 0%      1.00 ± 0%     ~     (all equal)
MsgLengthNoCompression-12      0.00           0.00          ~     (all equal)
MsgLengthPack-12               8.00 ± 0%      8.00 ± 0%     ~     (all equal)
MsgLengthMassive-12             138 ± 0%       138 ± 0%     ~     (all equal)

I would have sworn this length was somehow needed, guess not.

sorry, went to the wrong way through the PRs, created a merge conflict /o\

lgtm

These were going to create a merge conflict regardless of ordering. I've manually rebased.


[WIP] Follow on cleanup to #1275

Description
Clean up from some of the changes in #1275
None required

Kicking semaphore tests.


TypeError: Alphafold3.forward() got an unexpected keyword argument 'molecule_ids'

Seems the latest codebase does not have molecule_ids in the forward function?
https://github.com/lucidrains/alphafold3-pytorch/blob/main/alphafold3_pytorch/alphafold3.py#L3213
The new codebase doesnt match with the example usage in the README.md.

it is here? https://github.com/lucidrains/alphafold3-pytorch/blob/main/alphafold3_pytorch/alphafold3.py#L3220

seems I might have accidentally modified the source file. Thanks for the prompt reply!


Document table_metadata in Auth API

Related to: https://github.com/CartoDB/cartodb/issues/14998
Preview: http://auth-api-table-metadata.developers.carto-staging.com/developers/auth-api/

retest this please


Additional caching in the arvo kernel

The +slym on each vane activation and each invocation of +slur should cache the type calculation.
(Over the shoulder coded with Ted.)

@philipcmonk We'd like your review on this when you get a chance, mostly to double-check that we didn't introduce any space leaks by using +wa more.

@philipcmonk friendly ping


🎮 Game Request
The world is burning, Protect and Escort a group of wild ibex away from the inferno.
On the path there are sleeping ibex to rescue, wake them and they will join the group.
You will have to use the 4 elements (Air, Earth, Fire, Water)
to make your animal progress safely in the environnment.
The 4 elements are primary elements but there are also secondary elements at play:

The Volcano (creates fire)
The Source (drops water)
The Forest (created by Earth and Water)
The Wind (left wind and right wind is created randomly in Air)

Point down the features
All elements interact with each other and the ibex also react to elements in different ways.

Earth is a platform for the ibex
Fire scares the ibex
Water attracts the ibex
Forest makes the ibex running faster

Controls
Keyboard: SPACE to draw an element, ARROWS to move, W/Z + X + C + V to switch between Air + Earth + Fire + Water.
Keyboard is recommended but playing only with mouse is also possible (click on elements and DRAG the inner cursor).
Select program in which you are contributing
GSSoC23
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

Hey @S-ishita ! Thank you so much for your raising the issue💗
It’s all yours, you can come anytime again and make some contributions! 🚀
Alone, we can do little, but together we can do so much! 😇


Ubuntu/Debian yarn add firebase error + fix

(myenv) marc@penguin:~/citi-map$ yarn add firebase
yarn add v1.22.17
[1/4] Resolving packages...
[2/4] Fetching packages...
error<EMAIL_ADDRESS>The engine "node" is incompatible with this module. Expected version ">=12". Got "10.24.0"
error Found incompatible module.
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.
To resolve must grab the latest node version from Nodesource: curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash -
To install: sudo apt-get install -y nodejs
Some dependency errors and package build info
warning " ><EMAIL_ADDRESS>has unmet peer dependency "<EMAIL_ADDRESS>warning " ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>|| ^16.0.0".
warning " ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>|| ^16.0.0".
warning "material-ui ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>warning "material-ui ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>|| ^16.0.0".
warning "material-ui ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>|| ^16.0.0".
warning "material-ui ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>|| ^15.0.0 || ^16.0.0".
warning " ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>warning " ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>warning "react-scripts ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>warning "react-scripts > eslint-config-react-app ><EMAIL_ADDRESS>has unmet peer dependency "<EMAIL_ADDRESS>warning "react-scripts > eslint-config-react-app ><EMAIL_ADDRESS>has unmet peer dependency "<EMAIL_ADDRESS>warning "react-scripts > react-dev-utils ><EMAIL_ADDRESS>has unmet peer dependency "typescript@>= 2.7".
warning "react-scripts > eslint-config-react-app > @typescript-eslint/eslint-plugin ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>|| >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta".
[4/4] Building fresh packages...
success Saved lockfile.
success Saved 43 new dependencies.
info Direct dependencies
└─<EMAIL_ADDRESS>info All dependencies
├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>├─<EMAIL_ADDRESS>└─<EMAIL_ADDRESS>

Closing...with prejudice :-)

Hey @mashcode, glad you found a solution, in general, we recommend to use node 16.0.0 or higher

Funny. The latest stable node release from ubuntu/debian is 10.x. Of course
this is the detached Chromebook linux so it's not at all surprising :-)
On Wed, Mar 16, 2022 at 5:24 PM Marco Berlot @.***>
wrote:

Hey @mashcode https://github.com/mashcode, glad you found a solution,
in general, we recommend to use node 16.0.0 or higher
—
Reply to this email directly, view it on GitHub
https://github.com/dingaaling/citi-map/issues/13#issuecomment-1069648937,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAWKXCYHT6UY75ILEPP6BTVAJGQFANCNFSM5QZ5XR7Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***>


invariant-style type guard

TypeScript Version:  2.5.3
function invariant(condition: any, message: string): void {
    if (!condition) {
        throw new Error(message);
    }
}

function double(a: number | string) {
    invariant(typeof a === 'number', 'Sorry, strings not supported yet');
    return a * 2;
}

(Playground)
Expected behavior:
invariant should establish a type guard for the code following the invariant statement.
Actual behavior:
TypeScript errors with error TS2362: The left-hand side of an arithmetic operation must be of type 'any', 'number' or an enum type.
This is because a still resolves to type string | number.
Suggestion:
invariant ensures that its first argument condition is truthy, and throws otherwise, i.e. when the code following the invariant statement is reached it is safe to make all the same assumptions as if that code were wrapped in if (condition) { … }, or equivalently, the invariant were replaced with its literal implementation.
Although the latter works as a regular type guard (if (typeof a !== 'number') { throw … }), it makes the code unnecessarily verbose. I think there should be a way of declaring that a given function guarantees that some guard is active, and never returns otherwise, i.e. something like this (pseudo code):
declare function invariant(condition: any, message: string): void if testValue;

I would expect the syntactic form of any such thing to be more like this:
function invariant(condition: boolean, message: string): condition is true {
    if (!condition) {
        throw new Error(message);
    }
    return true;
}

This doesn't work as you want it to (TS doesn't seem to infer from condition is true to mean that typeof a !== 'number' and carry that inference forward), but something would certainly need to be in the signature of invariant to indicate that it is has this behavior.
That said, I don't see a lot of advantages to using invariant over simply saying
if (typeof a !== 'number') throw 'Sorry, strings not supported yet';

This will work exactly as you want invariant to work, without any separate utility function that you need to define. If you want to augment message somehow, you could easily define a never-returning function that does so:
function crash(message: string): never {
    throw `Something crashed: ${message}.`;
}

function double(a: number | string) {
    if (typeof a !== 'number') return crash('Sorry, strings not supported yet');
    return a * 2;
}

The project I work on uses this crash function quite liberally.

See #10421, #8655

@RyanCavanaugh Thanks for the reference. Though I tried looking for existing issues, I could not find any that had the invariant application that I was looking for, at least not under that name. :)
@krryan I hadn't thought about splitting the check and message formatting/post-processing etc. into a separate function. Thanks for the suggestion. I like your example of crash and will look into that.


WAV sound durations is always 4 seconds?

Hi, thanks for your work, I have one question, corpus database is always 4 seconds, can I use wav samples with different duration?
I have same samples with 4, 5 and 6 seconds of duration...

You can. You can understand this parameter.
https://github.com/yeyupiaoling/VoiceprintRecognition-Pytorch/blob/e68069ae427d0f0bc5b39ae86ea567a97e47483e/train.py#L31


Updates codecov yaml

Welcome to Codecov :tada:
Once merged to your default branch, Codecov will compare your coverage reports and display the results in this comment.
Thanks for integrating Codecov - We've got you covered :open_umbrella:


style: DatasourceControl dropdown z-index issue

When datasource name is long or browser window is small, DatasourceControl dropdown menu cut in half on narrow screens.
Expected results
The dropdown menu should be able to overflow from the side bar.
This should be fixable by adding z-index or rendering the dropdown in portal.
Another option is to right-align the dropdown menu.
Actual results

How to reproduce the bug

Go to Explore page
Select a datasource with very long name
Narrow your browser window
Click on the "..." dropdown.

Environment
(please complete the following information):

superset version: superset version
python version: python --version
node.js version: node -v
npm version: npm -v

Checklist
Make sure these boxes are checked before submitting your issue - thank you!

[x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[x] I have reproduced the issue with at least the latest released version of superset.
[ x I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context
Not sure if this is a regression, but should be somewhat related to #10971
cc @mistercrunch

@kgabryje 🙏

I think it's going to solve itself when I refactor react-bootstrap dropdowns to Antd

Similar to https://github.com/apache/incubator-superset/issues/11377 (these are the only two cosmetic / overflow issues I am aware of).
cc @junlincc

@ktmud @junlincc https://github.com/apache/incubator-superset/pull/11395 should fix the issue.

Merged, and confirmed this is fixed in master:


Beta 9 for 1.21.1 crashes on fabric loader 0.16.0 and higher

Was working fine earlier. Modrinth just pushed beta 9 to me, game no longer starts. made test pack with just the game, this mod plus dependencies, and tested on different versions of fabric loader.  Loader 0.15.11 works, 0.16.0 and up do not, always giving me this error. Rolling back to beta 7 is working just fine.
https://pastebin.com/6KrFJDMj

Yea, I released an update version of beta 9 as the WrapMethod injection was not remapping so I released a silent update for the version.

@TheHitman2772 Try re downloading beta 9 from Modrinth or curseforge

Loading fine now, thanks for the quick update!


Python2 crashes with non-ascii caracters.

Hi,
When using PyInquirer with python2 I face a problem when the options include non-ascii characters.
Python3 works fine though.
Here is a minimal working example that demonstrates this behaviour.
# -*- coding: utf-8 -*-
from __future__ import print_function, unicode_literals
from PyInquirer import prompt

def main():
    choice = 'John’s Pizza'
    questions = [
        {
            'type': 'list',
            'name': 'pizza',
            'message': 'Pizzeria:',
            'choices' : [choice]
        }
    ]
    answer = prompt(questions)
    pizza = answer['pizza']
    print('Answer is |' + pizza + '|' )

if __name__ == '__main__':
    main()

Python3 execution:
❯ python3 encoding-test.py
? Pizzeria:  John’s Pizza
Answer is |John’s Pizza|

Python2 execution:
❯ python2 encoding-test.py
? Pizzeria:  (Use arrow keys)
Traceback (most recent call last):
  File "encoding-test.py", line 20, in <module>
    main()
  File "encoding-test.py", line 15, in main
    answer = prompt(questions)
  File "/usr/local/lib/python2.7/dist-packages/PyInquirer/prompt.py", line 71, in prompt
    eventloop=eventloop)
  File "/usr/local/lib/python2.7/dist-packages/prompt_toolkit/shortcuts.py", line 625, in run_application
    result = cli.run()
  File "/usr/local/lib/python2.7/dist-packages/prompt_toolkit/interface.py", line 413, in run
    self._redraw()
  File "/usr/local/lib/python2.7/dist-packages/prompt_toolkit/interface.py", line 358, in _redraw
    self.renderer.render(self, self.layout, is_done=self.is_done)
  File "/usr/local/lib/python2.7/dist-packages/prompt_toolkit/renderer.py", line 429, in render
    extended_height=size.rows,
  File "/usr/local/lib/python2.7/dist-packages/prompt_toolkit/layout/containers.py", line 142, in write_to_screen
    sizes = self._divide_heigths(cli, write_position)
  File "/usr/local/lib/python2.7/dist-packages/prompt_toolkit/layout/containers.py", line 177, in _divide_heigths
    dimensions = [get_dimension_for_child(c, index) for index, c in enumerate(self.children)]
  File "/usr/local/lib/python2.7/dist-packages/prompt_toolkit/layout/containers.py", line 175, in get_dimension_for_child
    return c.preferred_height(cli, write_position.width, write_position.extended_height)
  File "/usr/local/lib/python2.7/dist-packages/prompt_toolkit/layout/containers.py", line 1652, in preferred_height
    return self.content.preferred_height(cli, width, max_available_height)
  File "/usr/local/lib/python2.7/dist-packages/prompt_toolkit/layout/containers.py", line 1000, in preferred_height
    cli, width - total_margin_width, max_available_height, wrap_lines),
  File "/usr/local/lib/python2.7/dist-packages/prompt_toolkit/layout/controls.py", line 254, in preferred_height
    content = self.create_content(cli, width, None)
  File "/usr/local/lib/python2.7/dist-packages/prompt_toolkit/layout/controls.py", line 259, in create_content
    tokens_with_mouse_handlers = self._get_tokens_cached(cli)
  File "/usr/local/lib/python2.7/dist-packages/prompt_toolkit/layout/controls.py", line 239, in _get_tokens_cached
    cli.render_counter, lambda: self.get_tokens(cli))
  File "/usr/local/lib/python2.7/dist-packages/prompt_toolkit/cache.py", line 37, in get
    value = getter_func()
  File "/usr/local/lib/python2.7/dist-packages/prompt_toolkit/layout/controls.py", line 239, in <lambda>
    cli.render_counter, lambda: self.get_tokens(cli))
  File "/usr/local/lib/python2.7/dist-packages/PyInquirer/prompts/list.py", line 98, in _get_choice_tokens
    append(i, choice)
  File "/usr/local/lib/python2.7/dist-packages/PyInquirer/prompts/list.py", line 92, in append
    tokens.append((T.Selected if selected else T, str(choice[0]),
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 4: ordinal not in range(128)

The apostrophe unicode character is what is breaking the python2 execution.

Currently, though I haven't figured out how to work around that without escaping the ascii codec. Though you can use unicode as below to get through it without error.
unicode('John’s Pizza').encode("unicode_escape")

which will give out what we dont really want
>>> John\u2019s Pizza

So its better to avoid it. In case you were able to fix it, do share how you are able to. I'll be working on it in my free time to get it fixed.

I have not tested it thoroughly, but it seems that removing the str() call from PyInquirer/prompts/list.py in line 92 solves the issue for now. See diff below:
@@ -89,7 +89,7 @@ class InquirerControl(TokenListControl):
                 tokens.append((T.Selected if selected else T,
                                '- %s (%s)' % (choice[0], choice[2])))
             else:
-                tokens.append((T.Selected if selected else T, str(choice[0]),
+                tokens.append((T.Selected if selected else T, choice[0],
                                select_item))
             tokens.append((T, '\n'))

Maybe this will guide you in the right direction even if it won't solve the issue yet.
Thanos

Oh lemme check that :+1: . Thanks

Just ran into the same issue - would be great if we could merge that fix.

I have added a fix to the dev branch. I'll push it up to the master with some other fix.

So I am gonna close this issue.

Sounds great, we want to use it as part of https://github.com/rasahq/rasa_core and since we still need to support py2 & py3 that would make this a lot easier :rocket:

Nice one, :+1: . Thanks for your time

Do you have any plans when the next release will happen?

No, not anytime soon. Cos I am planning to add editor and some other fix. I need to carry out test against some environments. Just too busy these days in my company. Also I wanted to add some of the TODO feature. But, I can create a pre release version, so people can use most recent fix with no new feature. :+1:

Yes that would be great, if you tag it with 1.1.0a1 (or whatever your next release is) "normal" people wont get it if they install from pip only if you specifically use the version number, that would be best :+1:


ZEPPELIN-383 CLASSPATH_OVERRIDES environment variable (2)

Prepend CLASSPATH_OVERRIDES environment variable when building CLASSPATH in the shell scripts.
This PR replaces the closed #386

Would ZEPPELIN_CLASSPATH_OVERRIDES be more consistent? while all other Zeppelin defined env variable starts with ZEPPELIN_

I deleted by mistake my fork... Creating #412 to further update this PR with ZEPPELIN_CLASSPATH_OVERRIDES.


Automatic error report

Problem Description
What steps will reproduce the problem?

Ipython terminal isn't opening
Traceback
  File "C:\Anaconda3\envs\deep-learning\lib\site-packages\spyder\plugins\ipythonconsole.py", line 1038, in create_new_client
    self.connect_client_to_kernel(client)
  File "C:\Anaconda3\envs\deep-learning\lib\site-packages\spyder\plugins\ipythonconsole.py", line 1073, in connect_client_to_kernel
    kc.start_channels(shell=True, iopub=True)
  File "C:\Anaconda3\envs\deep-learning\lib\site-packages\qtconsole\kernel_mixins.py", line 43, in start_channels
    super(QtKernelClientMixin, self).start_channels(*args, **kw)
  File "C:\Anaconda3\envs\deep-learning\lib\site-packages\jupyter_client\threaded.py", line 212, in start_channels
    self.shell_channel._inspect = self._check_kernel_info_reply
  File "C:\Anaconda3\envs\deep-learning\lib\site-packages\jupyter_client\client.py", line 143, in shell_channel
    socket, self.session, self.ioloop
  File "C:\Anaconda3\envs\deep-learning\lib\site-packages\jupyter_client\threaded.py", line 205, in ioloop
    self._ioloop = ioloop.IOLoop()
  File "C:\Anaconda3\envs\deep-learning\lib\site-packages\tornado\util.py", line 312, in __new__
    instance.initialize(*args, **init_kwargs)
  File "C:\Anaconda3\envs\deep-learning\lib\site-packages\zmq\eventloop\ioloop.py", line 139, in initialize
    super(ZMQIOLoop, self).initialize(impl=impl, **kwargs)
  File "C:\Anaconda3\envs\deep-learning\lib\site-packages\tornado\ioloop.py", line 845, in initialize
    super(PollIOLoop, self).initialize(**kwargs)
  File "C:\Anaconda3\envs\deep-learning\lib\site-packages\tornado\ioloop.py", line 357, in initialize
    self.make_current()
  File "C:\Anaconda3\envs\deep-learning\lib\site-packages\tornado\ioloop.py", line 315, in make_current
    assert asyncio is None
AssertionError

Package Versions

Spyder 3.2.7 ``
Python 3.5.4
Qt 5.6.2
PyQt5 5.6
Windows 10

Dependencies
pyflakes >=0.6.0 :  1.6.0 (OK)
pycodestyle >=2.3:  2.3.1 (OK)
pygments >=2.0   :  2.2.0 (OK)
pandas >=0.13.1  :  0.22.0 (OK)
numpy >=1.7      :  1.14.1 (OK)
sphinx >=0.6.6   :  1.7.1 (OK)
rope >=0.9.4     :  0.10.7 (OK)
jedi >=0.9.0     :  0.11.1 (OK)
nbconvert >=4.0  :  5.3.1 (OK)
sympy >=0.7.3    :  None (NOK)
cython >=0.21    :  None (NOK)
qtconsole >=4.2.0:  4.3.1 (OK)
IPython >=4.0    :  6.2.1 (OK)
pylint >=0.25    :  1.8.2 (OK)

Thanks for reporting. This is a duplicate of #6627 ; and in any case outside of Spyder's control. To fix for now, run conda install tornado=4.5.3 from the Anaconda Prompt/Terminal/shell (Windows/Mac/Linux) in the conda environment that Spyder is installed in.


Get required image info for UpgradeLog from cluster repo

IMPORTANT: Please do not create a Pull Request without creating an issue first.
Problem:

We hardcoded the UpgradeLog-related image repos and tags in the controller code base.
Solution:

Determine the image repos and tags by querying the corresponding charts from the cluster repo container at runtime.
Related Issue:
#3595
Test plan:

Build a new ISO image containing this PR and provision a single node cluster with it
Upgrade the cluster with the same ISO image
Check the UpgradeLog CR to see if InfraReady condition is True
Try to download the log archive

@starbops Could you rebase the latest master?

@mergifyio backport v1.2


WebSocket.send() of very large string closes the WebSocket unexpectedly

git clone git@github.com:oxygens/jsonrpc-bidirectional.git jsonrpc-bidirectional
cd jsonrpc-bidirectional
npm install
node --expose-gc --max-old-space-size=3072 tests/bug_uws_send_large_string_connection_closed.js

Windows 10, 64 bit
Works fine with ws.

[20348] callRPCMethodSiteB
[20348] [2017-04-25T20:32:39.200Z] Sent JSONRPC request, ping(). Larger than 1 MB, not logging.

[20348] Rejecting all Promise instances in WebSockets/JSONRPCClientPlugin.
[20348] Rejecting all Promise instances in WebSockets/JSONRPCClientPlugin.
[20348] Rejected 1 Promise instances in WebSockets/JSONRPCClientPlugin.
[20348] Unhandled Rejection at: Promise Promise {
  <rejected> Error: WebSocket closed. Code: 1006. Message: ""
    at WebSocketWrapper._webSocket.on (C:\Users\oxygen\Desktop\jsonrpc-bidirectional\src\Plugins\Client\WebSocketTransport.js:197:29)
    at emitTwo (events.js:111:20)
    at WebSocketWrapper.emit (events.js:194:7)
    at WebSocketClient._webSocket.on.theArgs [as internalOnClose] (C:\Users\oxygen\Desktop\jsonrpc-bidirectional\src\WebSocketAdapters\uws\WebSocketWrapper.js:86:11)
    at process.nextTick (C:\Users\oxygen\Desktop\jsonrpc-bidirectional\node_modules\uws\uws.js:63:19)
    at _combinedTickCallback (internal/process/next_tick.js:73:7)
    at process._tickCallback (internal/process/next_tick.js:104:9) } reason Error: WebSocket closed. Code: 1006. Message: ""
    at WebSocketWrapper._webSocket.on (C:\Users\oxygen\Desktop\jsonrpc-bidirectional\src\Plugins\Client\WebSocketTransport.js:197:29)
    at emitTwo (events.js:111:20)
    at WebSocketWrapper.emit (events.js:194:7)
    at WebSocketClient._webSocket.on.theArgs [as internalOnClose] (C:\Users\oxygen\Desktop\jsonrpc-bidirectional\src\WebSocketAdapters\uws\WebSocketWrapper.js:86:11)
    at process.nextTick (C:\Users\oxygen\Desktop\jsonrpc-bidirectional\node_modules\uws\uws.js:63:19)
    at _combinedTickCallback (internal/process/next_tick.js:73:7)
    at process._tickCallback (internal/process/next_tick.js:104:9)

It is relatively easy to reproduce the issue with the given steps.
There's also a related issue: https://github.com/uWebSockets/uWebSockets/issues/534

I'll reopen when you have properly filled the template out.

const WebSocket = require("uws");
const WebSocketServer = WebSocket.Server;

(async () => {
	const webSocketServer = new WebSocketServer({port: 8080});
	webSocketServer.on("error", console.error);
	webSocketServer.on("connection", console.log);
	webSocketServer.on("message", console.log);

	const webSocket = new WebSocket("ws://localhost:8080/api");
	await new Promise((fnResolve, fnReject) => {
		webSocket.on("open", fnResolve);
		webSocket.on("error", fnReject);
	});

	let str10MBPayload = "";
	let nIterator = 0;
	while(str10MBPayload.length < 10 * 1024 * 1024 /*10 MB*/)
	{
		str10MBPayload += str10MBPayload + "_" + (++nIterator);
	}

	webSocket.on("close", process.exit);
	webSocket.send(str10MBPayload);
})();

There.

Right, you see I have a template to fill out so that I can early catch what the user is doing.
You have several issues in your code:

You are actually sending more than 16 mb, which is why the server terminates the connection as is the design.
You are not even registering to the message events of the server properly. You don't have a "message" event on the server, you have it on the websockets the server creates.

This is a message event on the server: 	webSocketServer.on("message", console.log);
It also should be irrelevant if a message event is registered on the server or not.
"which is why the server terminates the connection as is the design."
I will not argue with your design, yet you should properly document these things and send a proper reason for closing the connection (currently an empty string is sent).
The code works with no issues with ws.
It doesn't matter how much I'm sending, your server closes a connection without specifying a reason.
"By design" means your design, or official spec?

Clearly the Autobahn test suite is wrong when it shows µWS 100% compliant. I would suggest opening an issue in their repo so that you can share your standards expertise with the world.

The connection close reason parameter is there for ... a reason. And the reason is: help developers find out why the connection was closed so they can debug.
Clearly the Autobahn is not testing for this scenario, and clearly it doesn't care about the connection close reason beeing an empty string.

See ya.

Dude, the only mistake is that you have lube in your eyes.
I repeat: there is a message handler on the server.
And you're sloppy: you're closing the connection with an empty string as reason. Must be because of all that lube on your hands.


ModSecurity DoS protection on IIS

We require a DoS protection on our web site that is running on IIS, so we decided to try out ModSecurity. We installed ModSecurity 2.9.2 with OWASP CRS 3.0.2. After doing a bit of digging we opted to set PL to level 3. Now, at this PL if the burst counter (IP:DOS_BURST_COUNTER) is greater equal 1, then the blocking flag is being set (IP:DOS_BLOCK). The blocking flag (IP:DOS_BLOCK) expires within a timeout period (TX:DOS_BLOCK_TIMEOUT).
We didn't change the default values, but, anyway, here they are:
SecAction \
 "id:900700,\
  phase:1,\
  nolog,\
  pass,\
  t:none,\
  setvar:'tx.dos_burst_time_slice=60',\
  setvar:'tx.dos_counter_threshold=100',\
  setvar:'tx.dos_block_timeout=600'"

The issue is that by default ModSecurity is trying to drop connection which, as it say in wiki, "is currently not available on Windows based builds.". So, it floods the EventLog with messages like "ModSecurity: Access denied with code 500 (phase 1) (Error: Connection drop not implemented on this platform. Operator EQ matched 1 at IP:dos_block." which is unacceptable for us and returns code 500 which is not expected behavior.
We tried to overwrite the action by adding these rules:
SecRuleUpdateActionById 912120 "t:none,deny,status:403"
SecRuleUpdateActionById 912130 "t:none,deny,status:403"

However, in this case the first request from a user is always blocked returning status 403. The following request is fine though.
So, is there a way to set up DoS protection on IIS the way it doesn't flood the EventLog and block the first user request?

@AnryCryman
It might be due to the fact that 912120 contains a chain action.
See note at SecRuleUpdateActionById: "Note : If the target rule is a chained rule, you must currently specify chain in the SecRuleUpdateActionById action list as well. This will be fixed in a future version."
Try adding the chain action to your SecRuleUpdateActionById directive.

Yeah, that worked. Thanks for the help.

@AnryCryman
It might be due to the fact that 912120 contains a chain action.
See note at SecRuleUpdateActionById: "Note : If the target rule is a chained rule, you must currently specify chain in the SecRuleUpdateActionById action list as well. This will be fixed in a future version."
Try adding the chain action to your SecRuleUpdateActionById directive.

I´m with the same problem of AnryCryman.
How to add chain in 912120 rule?
BR.,
Thiago


Introduce a Subdirectory for Non-Plugin, Non-Executable Binaries

Summary
In the course of issue #574, it became apparent that the NuGet update will introduce a large number of additional binaries to the Duality runtime. Keeping them all in the main directory is not very nice to work with, but as the project grows and matures, this might not remain restricted to NuGet only. To fix this, introduce a new binary subfolder next to Plugins where all non-plugin, non-executable binaries are stored.
Analysis

Update project settings to the build output is matching the new folder structure. Maybe also fix the OpenTK library location to match the actual binary installation (currently in Plugins, isn't actually in there in a binary install).

Could be a bit tricky since the regular build thing is to have all output assemblies and their dependencies in the same folder.
The test setup matching the real one would be nice, but it's not vital as long as the implementation is done in a way that it doesn't matter. So try it, but don't get stuck trying.

The DefaultAssemblyLoader will need to be updated with regard to the assembly search paths.
All PluginManager implementations need to be updated so they don't try to load all available assemblies as plugins, but only those in the Plugins folder, if they're not already doing that.
The existing AssemblyResolve handler needs to be extended to search in the new subfolder as well.

Make sure to load .pdb files too, where available.

Update the package manager code and tests with the new file mappings.
Need to update the Publish Game dialog / publish script.
Update the plugin project template, also check the code that updates it at runtime.
Potential names:

Assemblies, too generic. Plugins and executables are assemblies too.
Binaries, too generic. Plugins and executables are binaries too.
Libraries, probably the best one so far. Maybe shorten to Libs?
?

CoreLibraries? As this doesn't seem to be something the user himself will be managing but something that's needed for duality to function.

CoreLibraries? As this doesn't seem to be something the user himself will be managing

If any of the installed packages contain non-Duality assemblies, or depend on them, they'll go in there as well, so it won't be Duality-only libraries in there.

Progress

Created a new develop-3.0-libs-dir branch to work on this.
Introduced a new Libs subdirectory to be searched for static, non-plugin dependencies in the same way as regular Assembly search paths by the runtime.
Chose to implement this transparently in the DefaultAssemblyLoaders resolve handler before doing anything else. That way, the entire change is invisible to plugin loading code and no adjustments need to be made there.

Immediate ToDo

Update project settings to the build output is matching the new folder structure.

Might need to introduce an intermediate build output directory for all libraries, combined with postbuild copy and adjusted debug settings.
Postbuild commands previously caused problems when building in Xamarin Studio on non-Windows systems. Avoid them.
Instead, use msbuild copy commands via csproj to achieve the same without any command lines.

Update the package manager code and tests with the new file mappings.
Update the Publish Game dialog / publish script.
Update the plugin project template, also check the code that updates it at runtime.

Progress

Updated project files to create a build output that matches the new folder structure using msbuild copy commands.
Fixed startup problems due to attempting to resolve Duality assemblies before the resolve handler was in place.

Immediate ToDo

As it turns out, this new setup has several problems.

Using any dependency will crash the application when accidentally done before the assembly resolve handler is in place. This can happen quite easily and reduces the overall stability of the project. The alternative of depending on a config file to add another probing path is kind of ugly, and it is unclear whether this config file is evaluated consistently across runtimes.
The VS Test Explorer can no longer run tests as-is, as there is no easy and robust way to change the working directory they're executed in, and requiring to do so makes it more difficult to run tests in VS, NUnit and AppVeyor environments.

Both of the above have a noticeable negative impact on maintenance while the overall issue is mostly about convenience and "look and feel" of the Duality folder structure.

Consider closing the issue as wont-fix and discarding the work that has been done so far.
Consider alternatives.

Salvaged the explicit output aggregate changes for a develop-3.0 commit in order to clean up the output directory and make sure no assemblies end up in Plugins by accident.


Default values for config fields?

Writing https://github.com/SchemaStore/schemastore/pull/2394 I was in doubt for the default values of these config, can you help me?
[config]
skip_core_tasks = true # ?
skip_git_env_info = true # ?
skip_rust_env_info = true # ?
skip_crate_env_info = true # ?
default_to_workspace = false # ?

[config]
skip_core_tasks = false
skip_git_env_info = false
skip_rust_env_info = false
skip_crate_env_info = false
default_to_workspace = true

its the opposite :)
https://sagiegurari.github.io/cargo-make/api/cli/types/struct.ConfigSection.html

Fixed in https://github.com/SchemaStore/schemastore/pull/2398.


Debugging is completely broken

vsce package creates a partially broken package.
Im not sure that the debug adapter is even called.
All debugging is broken until this is fixed.

I think it has something to do with package dependencies. But it is fixed in a horrible way right now

It was because the await-notify package was not listed under dependencies and so it was not included in the package. This would completely break everything unless you had the package globally installed. Versions 0.0.2 and 0.0.3 included the entire project and were not ideal. Versions 0.0.4+ do not have this issue.


Fix param sharing

Once an object gets created via POST, pass its ID to subsequent GET/PUT/DELETE requests

Thanks for the fix.


count() query of data in pool 4x slower than count() of same data via zq

(When this issue was first opened on behalf of @philrz, it was incorrectly framed as being a perf problem specific to when the Zed service is run in the cloud, but in fact the perf issue is entirely reproducible when everything is run on the same local host. @philrz has now morphed the issue to represent that & re-repro'ed with current Zed commit 9c0f097.)

Doing a count() on a large data set is consistently slower when run against a pool stored in a Zed lake when compared when queried in its file form via zq.
Here's a repro with a ~3 GB, gzip'ed ZNG file, which is ~4.3 GB after being gunzip'ed.
$ ls -l fdns-a.zng.gz
-rw-r--r--@ 1 phil  staff<PHONE_NUMBER> Jun 28 14:46 fdns-a.zng.gz

$ gzcat fdns-a.zng.gz | wc -c
<PHONE_NUMBER>

$ zed -version
Version: v1.2.0-49-g9c0f0973

$ curl http://localhost:9867/version
{"version":"v1.2.0-49-g9c0f0973"}

$ zed create -orderby value:asc fdns
pool created: fdns 2EjlcsW7X6NOV8M1ldIJWAdjecd

$ zed use fdns
Switched to branch "main" on pool "fdns"

$ zed load fdns-a.zng.gz
(1/1) 3.00GB/3.00GB 0B/s 100.00%
2EjnTG8vKFsdi3nppDh2AwpjfmT committed

$ time zed query -z 'count()'
{count:342541391(uint64)}

real	3m42.805s
user	0m0.079s
sys	0m0.036s

$ time zed query -z 'count()'
{count:342541391(uint64)}

real	3m41.223s
user	0m0.076s
sys	0m0.034s

$ zq -version
Version: v1.2.0-49-g9c0f0973

$ time zq -z 'count()' fdns-a.zng.gz
{count:342541391(uint64)}

real	0m48.988s
user	1m12.660s
sys	0m2.189s

$ time zq -z 'count()' fdns-a.zng.gz
{count:342541391(uint64)}

real	0m51.672s
user	1m16.811s
sys	0m2.234s

Actually @mattnibs, the zed query was done against a Zed lake on localhost on my Mac, not on the cloud service. (Slack)

Update: The description of #4210 gave me a sense that the change might improve the perf of what's shown in this issue, but a quick re-run shows not.
$ zed -version
Version: v1.2.0-130-g8dd27d98

$ curl http://localhost:9867/version
{"version":"v1.2.0-130-g8dd27d98"}

$ time zed query -z 'count()'
{count:342541391(uint64)}

real	3m1.506s
user	0m0.062s
sys	0m0.026s

The improvement from 3:40 to just over 3 minutes had already been logged at the prior commit bcec331 so that help came from something else we'd done.

The gap continues to close, but there's still a significant difference.
$ zed -version
Version: v1.7.0-17-g3a98e7d8

$ time zq -z 'count()' fdns-a.zng.gz
342541391(uint64)
real	0m46.000s
user	1m5.618s
sys	0m2.161s

$ time zed query -z 'count()'
342541391(uint64)
real	1m59.889s
user	0m0.050s
sys	0m0.025s


Reduce boost applied to coronavirus content

The default boost for anything is 1, which is used as a multiplier to promote or demote certain content.
The boost here is cumulative with other boosts in this file, so any covid landing pages also have another 2.5 applied to them, transactions have 1.5 etc.
Our supposition is that artificially boosting all the coronavirus content by such a lot is suppressing other relevant content.
The ideal scenario is that no taxon has any particular precedence over any other, particularly with such a strong "brand" (the inclusion of "covid" or "coronavirus" in the query is an excellent indicator), it should be able to stand on its own two feet with the other settings we have applied.
With that in mind, we don't want to remove it completely, for fear that we might actively hurt coronavirus searches. So we'll start off by reducing it a bit, and monitor the overall click through rate
If the rate doesn't change, and the coronavirus team are happy, then I'll subsequently remove this boost altogether.

Thanks @leenagupte 🙇