guestbook example is broken

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.1", GitCommit:"1dc5c66f5dd61da08412a74221ecc79208c2165b", GitTreeState:"clean", BuildDate:"2017-07-14T02:00:46Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.5", GitCommit:"17d7182a7ccbb167074be7a87f0a68bd00d58d97", GitTreeState:"clean", BuildDate:"2017-09-11T21:52:19Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

$ kedge create -f examples/guestbook/db.yaml
persistentvolumeclaim "mongodb-data" created
service "mongodb" created
secret "mongodb-admin" created
secret "mongodb-user" created
configmap "mongodb-user" created
deployment "mongodb" created

$ kubectl get pods
NAME                       READY     STATUS    RESTARTS   AGE
mongodb-2659439370-6k4zr   0/1       Error     2          19s

$ kubectl logs mongodb-2659439370-6k4zr
=> sourcing 10-check-env-vars.sh ...
=> sourcing 20-setup-wiredtiger-cache.sh ...
=> sourcing 30-set-config-file.sh ...
=> sourcing 35-setup-default-datadir.sh ...
ERROR: Couldn't write into /var/lib/mongodb/data
CAUSE: current user doesn't have permissions for writing to /var/lib/mongodb/data directory
DETAILS: current user id = 184, user groups: 997 0
stat: failed to get security context of '/var/lib/mongodb/data': No data available
DETAILS: directory permissions: drwxr-xr-x owned by 0:0, SELinux: ?

Ummm. Confused what example you're using.
It says: kedge create -f examples/guestbook/db.yaml
Yet our folder-name is guestbook-demo ?
https://github.com/kedgeproject/kedge/tree/master/examples/guestbook-demo

Nvm, I am getting the same error too:
▶ kubectl logs mongodb-2659439370-1vmwl
=> sourcing 10-check-env-vars.sh ...
=> sourcing 20-setup-wiredtiger-cache.sh ...
=> sourcing 30-set-config-file.sh ...
=> sourcing 35-setup-default-datadir.sh ...
ERROR: Couldn't write into /var/lib/mongodb/data
CAUSE: current user doesn't have permissions for writing to /var/lib/mongodb/data directory
DETAILS: current user id = 184, user groups: 997 0
stat: failed to get security context of '/var/lib/mongodb/data': No data available
DETAILS: directory permissions: drwxr-xr-x owned by 0:0, SELinux: ?

@containscafeine
The old-example works (using the bitnami container). We'll have to trouble-shoot this and fix up the current container we use.
▶ ./kedge apply -f backend.yaml -f frontend.yaml -f db.yaml
service "backend" configured
deployment "backend" configured
service "frontend" configured
deployment "frontend" configured
persistentvolumeclaim "mongodb-data" configured
service "mongodb" configured
secret "mongodb-admin" configured
secret "mongodb-user" configured
configmap "mongodb-user" configured
deployment "mongodb" configured

github.com/kedgeproject/kedge  master ✗                                                                                18h55m ◒  ⍉
▶ kubectl get po -w
NAME                        READY     STATUS    RESTARTS   AGE
backend-2776521642-zf5kl    1/1       Running   0          2m
frontend-2675451940-nq6d6   1/1       Running   0          2m
mongodb-3720339234-1njhr    1/1       Running   0          34s

hmm, that is strange it works on OpenShift but not on Kubernetes (minishift/minikube). We should report it so it can be fixed. Those CentOS images are in really bad shape.

@cdrage @kadel @containscafeine , I  have started working on it.
so @cdrage , I am following this official example https://kubernetes.io/docs/tutorials/stateless-application/guestbook/

Okay! Push to a fork or GitHub PR if you need help. I started on it for a
bit before I realized that you had already began.
On Dec 14, 2017 06:33, "Suraj Narwade"<EMAIL_ADDRESS>wrote:
@cdrage https://github.com/cdrage @kadel https://github.com/kadel
@containscafeine https://github.com/containscafeine , I have started
working on it.
so @cdrage https://github.com/cdrage , I am following this official
example https://kubernetes.io/docs/tutorials/stateless-
application/guestbook/
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kedgeproject/kedge/issues/520#issuecomment-351686103,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGH-oB9AxTicQ6RqdoEBxY49L_3UErxrks5tAQeHgaJpZM4Q5DxZ
.


When Template is updated, immediately reconcile any matching workloads

I want updating a template to fix workload issues without waiting for the exponential backoff.
Given a StarTemplate whose inner template is malformed in a way that prevents proper object submission
And a workload that is in exponential backoff from erroring on this component submission
When the template is patched
Then the workload immediately begins the reconciliation process

Example
Before
kind: Workload
status:
  conditions:
    - type: ComponentsUnableToSubmit
      status: "True"
      reason: TemplateRejectedByAPIServer
    - type: Ready
      status: "False"
      reason: ComponentsUnableToSubmit

After
In no more than 10 seconds after submitting new template:
status:
  conditions:
    - type: ComponentsUnableToSubmit
      status: "False"
      reason: ComponentSubmissionComplete
    - type: Ready
      status: "True"
      reason: Ready

Resolved in #304


Fix *T

Fix *T type so it now implements TestingFT:

previous Run() method renamed to RunT(),
un-shadow Run(string, func(t*testing.T)),
NewT() handles *T as argument,

Resolve #22 but UseEqual() instead of EnableEqual()
Only export needed types/funcs
Code() & Smuggle() refuse variadic function as parameter
tdhttp.New{,JSON,XML}Request() now accept headers definition

Pull Request Test Coverage Report for Build 372

121 of 121 (100.0%)  changed or added relevant lines in 27 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.004%) to 99.647%

Totals

Change from base Build 366:
0.004%

Covered Lines:
4795

Relevant Lines:
4812

💛  - Coveralls


[BUG] Error migrating pre-v0.6.0 dataset to v0.6.2

I tried to load a persistent dataset that I had created with a pre-video version of FiftyOne using the latest develop, and experienced the following error:
The dataset is an image dataset with content that looks like the screenshot posted below:
import fiftyone as fo

dataset = fo.load_dataset("bdd100k-validation")

Migrating dataset 'bdd100k-validation' to the current version (0.6.2)
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-2-ae96572947cd> in <module>
----> 1 dataset = fo.load_dataset("bdd100k-validation")

~/dev/fiftyone/fiftyone/core/dataset.py in load_dataset(name)
     88         ValueError: if no dataset exists with the given name
     89     """
---> 90     return Dataset(name, _create=False)
     91
     92

~/dev/fiftyone/fiftyone/core/singleton.py in __call__(cls, name, _create, *args, **kwargs)
     34         ):
     35             instance = cls.__new__(cls)
---> 36             instance.__init__(name=name, _create=_create, *args, **kwargs)
     37             name = instance.name  # `__init__` may have changed `name`
     38             cls._instances[name] = instance

~/dev/fiftyone/fiftyone/core/dataset.py in __init__(self, name, persistent, _create)
    201                 self._sample_doc_cls,
    202                 self._frame_doc_cls,
--> 203             ) = _load_dataset(name)
    204
    205         self._deleted = False

~/dev/fiftyone/fiftyone/core/dataset.py in _load_dataset(name)
   2051                 % (dataset_doc.name, VERSION)
   2052             )
-> 2053             runner.run(dataset_names=[dataset_doc.name])
   2054         dataset_doc.reload()
   2055         dataset_doc.version = VERSION

~/dev/fiftyone/fiftyone/migrations/runner.py in run(self, dataset_names)
     45             for revision, module in self._revisions_to_run:
     46                 fcn = etau.get_function(self.direction, module)
---> 47                 fcn(conn, dataset_name)
     48
     49     @property

~/dev/fiftyone/fiftyone/migrations/revisions/v0_6_2.py in up(db, dataset_name)
     22         dataset_dict["media_type"] = "image"
     23
---> 24     for field in dataset_dict["sample_fields"] + dataset_dict["frame_fields"]:
     25         if "media_type" in field:
     26             del field["media_type"]

KeyError: 'frame_fields'

@benjaminpkane Also, I can't seem to reproduce this at the moment, but I've also seen a bug where I get an error when exiting my Python process with a dataset loaded. The error is a missing version key somewhere.
I will try to track this down in more detail. I think it was on develop with https://github.com/voxel51/fiftyone/pull/576 merged.

Looking into it

@benjaminpkane Also, I can't seem to reproduce this at the moment, but I've also seen a bug where I get an error when exiting my Python process with a dataset loaded. The error is a missing version key somewhere.
I will try to track this down in more detail. I think it was on develop with #576 merged.

This is likely from using a v0.6.2. dataset in a prior version of fiftyone (pre-migrations). I don't think there is much we can do about that. Or it is at least not worth the engineering, imo.

Resolved by #601


with Protractor, Controller setTimeout function not run

In My Controller , when test with  Protractor,interval or setTimeout function not to be invoke
$interval(function(){
console.log("aaaaa");
},100,1);

Please direct general support questions like this one to an appropriate support channel, see https://github.com/angular/protractor/blob/master/CONTRIBUTING.md#questions
Your question is better suited for StackOverflow with the 'protractor' tag or the Gitter channel.
Thanks!


[SPARK-25132][SQL] case-insensitive field resolution when reading from Parquet/ORC

What changes were proposed in this pull request?
Spark SQL returns NULL for a column whose Hive metastore schema and Parquet schema are in different letter cases, regardless of spark.sql.caseSensitive set to true or false. This applies not only to Parquet, but also to ORC. Following is a brief summary:

ParquetFileFormat doesn't support case-insensitive field resolution.
native OrcFileFormat supports case-insensitive field resolution, however it cannot handle duplicate fields.
hive OrcFileFormat doesn't support case-insensitive field resolution.

https://github.com/apache/spark/pull/15799 reverted case-insensitive resolution for ParquetFileFormat and hive OrcFileFormat. This PR brings it back and improves it to do case-insensitive resolution only if Spark is in case-insensitive mode. And field resolution will fail if there is ambiguity, i.e. more than one field is matched. ParquetFileFormat, native OrcFileFormat and hive OrcFileFormat are all supported.
How was this patch tested?
Unit tests added.

Can one of the admins verify this patch?

Can one of the admins verify this patch?


Help a n00b to install in an non internet env..

Summary
Im sorry, this is not a bug but i can't get it to install. Help needed. I hope someone can point me in the correct direction...
Issue Type
Bug Report
Ansible Version
ansible [core 2.16.2]

Steps to Reproduce
- name: Deploy RKE2
  hosts: all
  become: yes
  vars:
    rke2_ha_mode: true
    rke2_ha_mode_keepalived: false
    rke2_kubevip_svc_enable: false
    rke2_airgap_mode: true
    rke2_airgap_implementation: copy
    rke2_version: 1.29.1+rke2r1
    rke2_cni:
      - multus
      - calico
    rke2_artifact:
      - sha256sum-{{ rke2_architecture }}.txt
      - rke2.linux-{{ rke2_architecture }}.tar.gz
      - rke2-images.linux-{{ rke2_architecture }}.tar.zst
    rke2_airgap_copy_additional_tarballs:
      - rke2-images-multus.linux-{{ rke2_architecture }}
      - rke2-images-calico.linux-{{ rke2_architecture }}
  roles:
     - role: lablabs

Expected Results
Where do i specify the target dir for my downloaded RKE2 files? Or where should i put the files?
I have a directory with the files: rke2-images.linux-amd64.tar.zst + rke2.linux-amd64.tar.gz + sh256sum-amd64.txt
I'm trying to install in air-gap mode, no internet access. I just want the playbook to install RKE2 on the target nodes in the inventory-file and fetch the files from my directory with the RKE2 files.
We have our own LB.
Actual Results
I get the following error:
I don't find any rke2.sh file in the role.

TASK [lablabs : Copy local RKE2 installation script] **********************************************************************************************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: If you are using a module and expect the file to exist on the remote, see the remote_src option
fatal: [master-01]: FAILED! => {"changed": false, "msg": "Could not find or access 'local_artifacts/rke2.sh'\nSearched in:\n\t/home/pingo/Downloads/rke2/roles/lablabs/files/local_artifacts/rke2.sh\n\t/home/pingo/Downloads/rke2/roles/lablabs/local_artifacts/rke2.sh\n\t/home/pingo/Downloads/rke2/roles/lablabs/tasks/files/local_artifacts/rke2.sh\n\t/home/pingo/Downloads/rke2/roles/lablabs/tasks/local_artifacts/rke2.sh\n\t/home/pingo/Downloads/rke2/files/local_artifacts/rke2.sh\n\t/home/pingo/Downloads/rke2/local_artifacts/rke2.sh on the Ansible Controller.\nIf you are using a module and expect the file to exist on the remote, see the remote_src option"}
An exception occurred during task execution.

aha, you had to copy the install.sh script from rke2 and rename it to rke2.sh. And point it to rke2_artifact_path and rke2_airgap_copy_sourcepath

works now


adding new contentTypes related to TVB pipeline

relevant for https://search.kg.ebrains.eu/instances/4edef2ff-e653-40a5-a222-0a5759fdaaaf

json (specific for old MINDS, as far as formerly specified)
tsv (specific for TVB)

@jagru20 / @apdavison : I do not know who of you has more time at the moment ;) (one of you for the review is enough I guess)

@apdavison yes you are right. I'll change it

done :)


Thank you!

Thank you very much. Works perfectly! saved me a lot of time and money.

You're welcome 😄


Fix trivy slack channel

What this PR does / why we need it:
Fixes the Slack channel to which Trivy notifications are sent for the latest release
cc @neil-hickey

The problem with that script is it just scans source code, we're scanning images


Gateways with multiple node pubkeys

I was talking with someone from an LSP this weekend. They mentioned how they rotate nodes periodically. They could give us multiple node pubkeys. We could generate invoices with route hints for both of these node pubkeys. This might increase incoming payment success if senders could route through 2 different nodes.

Hi, I would like to work on this issue, could you please give me more details on how you expect multiple node public keys to be implemented?
I am new to contributing to open source projects and any additional guidance would be greatly appreciated 👍

Too hard of an issue for someone unfamiliar with the project, let alone new to OS dev.

Hi @elsirion, thanks for the comments.
I understand that this topic may be too complex for someone new to the project, could you recommend a simpler problem or some additional resources to help me familiarize myself with the project before tackling more complex tasks?
I appreciate any guidance you can provide!


chore(): bump version ionic-native latest

Thanks for the PR! I'm assigning @danbucholtz to take a look at this. We have to make some tweaks to ionic-native. :smile:

Close reopened in #252

Close reopened in #252


Project Name

This project needs a name.
Suggestions?

Project name has been change to "PolarBear". No clue why but it's the name now.
Other name suggests are welcome, however.


Email validation on sign up

Hello,
I see there is no email validation on sign up. Is there a standard way to do I have to do it myself? It can be useful to have an option.
What you think?

There is not currently a built-in option for doing email confirmation. There used to be an option for this a number of years ago, but we've found it to be a cargo-culted practice that doesn't actually give most sites anything of value compared to the hassle it causes users. It's one of the opinions Clearance holds.
That said, I have worked on sites where this was a necessity, so we've been improving the story on how to handle this type of functionality yourself. I've also been writing a lot of documentation lately, which may help.

You will need to add two columns to your user model or to a join model: email_confirmation_token and email_confirmed_at (or a boolean if you'd rather).
On registration of a new user, be sure to generate an email_confirmation token and also deliver an email with the confirmation link.
Write the email confirmation controller that marks the user as confirmed and blanks the token.
Write and configure a Sign In Guard that disallows sign in by users that are not confirmed.
For extra credit: handle this process again when the user changes their email addrress.

We attempt to provide the controller extension points that might make sending that email possible and the Sign In Guards that allow you to block sign in for any number of reasons.
I do think that a well done email confirmation process might make for an interesting gem available as an extension to clearance.

Thanks @derekprior for your help. I think it may cause a security issue because we have to check if email addresses are real. I will check if I create a gem.

we do validate emails for proper format, using the email_validator gem. We just don't validate that it's deliverable or that the user owns them.
You could potentially monitor bounces for marking a user as having an invalid email. Another approach I took on a project once was to have every single email we ever sent a user contain a confirmation parameter in the links. If the user ever once clicked a link from an email we'd confirm that address using an application controller before_filter. So it was more of a passive email confirmation.

Thanks for suggestions. I think the simplest is to have a confirmation email. It's pretty common and, I think, users are accustomed.

Here are some old numbers from a product we ran with email confirmation:

We have numbers that show you lose customers this way. In one example, we have a consumer app with 9,204 email sign ups, but only 4,607 confirmations.

Source: https://robots.thoughtbot.com/email-and-password-confirmation-bias
That source is old and the result anecdotal rather than scientific, but it mirrors my experience. I'm not trying to disuade you in particular from adding this feature to your specific app -- mostly just trying to document the entire conversation since this request comes up somewhat frequently.


Mullvad/VPN

Mullvad/VPN
Originally posted by @Macyoutuber in https://github.com/oras-project/oras/issues/987

If you have a specific issue or request for this repo, please just let us know. Thank you!


save sending messages in client side

i know this idea is a lot crazy. but it may useful.
let me describe the detail.

save messages to tiny local mq and get position instead of sending messages to server directly.
after that, read messages from tiny local mq by position and send messages to rocketmq (server side).
position will not change if meet any exception (eg: network error/broker busy/system busy)
The tiny local mq should be implemented by mmap and restore messages if client restart.

I don't know whether this idea can be implemented in DefaultMQProducerImpl.

Allowing caching on the client side means allowing data loss and unreliability, which may be very useful in some special network scenarios, or to send OneWay messages while collecting logs, without server confirmation.

Allowing caching on the client side means allowing data loss and unreliability, which may be very useful in some special network scenarios, or to send OneWay messages while collecting logs, without server confirmation.

so, the tiny local mq should be implemented by mmap (like rocketmq) and restore messages if client restart.
data will not loss since data can be save in file (mmap)

It will greatly increase the complexity of the client, and most business systems hope to use a lightweight SDK in the production environment. Client may runs in non-persistent container

it can be a optional config in client side. I think many people will like this function beacuse it's a way to help them to solve exception.
We can mount a phyical disk for non-persistent container (eg: docker) to save data.
It will greatly increase the complexity of the client, but it's useful.

This function exists in metaq, it's indeed useful in some cases, such as log report and some data insensitive cases.

In the production environment，some cases are that first expand then shrink, the client consumer will be replaced, Is this meant to be loss more msgs？


BatchInstanceNorm1d

Hello,
I have been working on a project using a WAE with BatchNorm in the encoder and InstanceNorm in the decoder to have both representation/classification power for the encoder and the capacity of AdaIN for style conditioning in the decoder.
I choose this empirically throughout my experiments and then found out your paper which generalizes the possibility of mixing both.
I would like to try it and then make reference to your work. It was not a major feature of my research to separate the normalizations, just an implementation detail that I could replace with yours.
I replaced every 2d normalization in my networks by BatchInstanceNorm2d, no problem, it processes without error the 4D tensors (batch,chan,H,W).
However, I also had to change every 1d normalization by BatchInstanceNorm1d, which is for the 2D tensors output of linear layers. And there I have an error ...
I double checked the data, given a 2D tensor (90,1024) output of linear layer which is followed with BatchInstanceNorm1d(1024), I get the following:
/fast-2/adrien/virtualenv_p3/lib/python3.5/site-packages/torch/nn/functional.py in batch_norm(input, running_mean, running_var, weight, bias, training, momentum, eps)
1617             size_prods *= size[i + 2]
1618         if size_prods == 1:
-> 1619             raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size))
1620
1621     return torch.batch_norm(
ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 92160])
could you help fixing this please ?
ran on python 3.5 and pytorch 1.0.0

I also tried switchable normalization from
https://github.com/switchablenorms/Switchable-Normalization
replacing normalization layers from my models with SwitchNorm1d and SwitchNorm2d
from https://github.com/switchablenorms/Switchable-Normalization/blob/master/devkit/ops/switchable_norm.py
I did not have issues with this implementation and I am currently training new models with SwitchNorm
If we can fix BatchInstanceNorm1d, then I can also train them and make comparison of both BatchInstanceNorm and SwitchNorm against the initial model with BN encoder and IN decoder
Thanks !

I made a few modifications in the forward after # Instance norm
if case to not reshape 2D tensors that did not forward in BatchInstanceNorm1d
now it forwards so I will give it a try ... are these modifications correct to you ?
`class _BatchInstanceNorm(_BatchNorm):
def init(self, num_features, eps=1e-5, momentum=0.1, affine=True):
super(BatchInstanceNorm, self).init(num_features, eps, momentum, affine)
self.gate = Parameter(torch.Tensor(num_features))
self.gate.data.fill(1)
setattr(self.gate, 'bin_gate', True)
def forward(self, input):
    self._check_input_dim(input)

    # Batch norm
    if self.affine:
        bn_w = self.weight * self.gate
    else:
        bn_w = self.gate
    out_bn = F.batch_norm(
        input, self.running_mean, self.running_var, bn_w, self.bias,
        self.training, self.momentum, self.eps)

    # Instance norm
    b, c  = input.size(0), input.size(1)
    if self.affine:
        in_w = self.weight * (1 - self.gate)
    else:
        in_w = 1 - self.gate

    # input = input.view(1, b * c, *input.size()[2:])
    # this did not forward in the following F.batch_norm(input,...) for 2D input tensors and 1d norm ..
    if input.dim() != 2:
        input = input.view(1, b * c, *input.size()[2:])

    out_in = F.batch_norm(
        input, None, None, None, None,
        True, self.momentum, self.eps)

    # out_in = out_in.view(b, c, *input.size()[2:])
    # same as before
    if input.dim() != 2:
        out_in = out_in.view(b, c, *input.size()[2:])
        out_in.mul_(in_w[None, :, None, None])
    else:
        out_in.mul_(in_w)

    return out_bn + out_in`

I made a few modifications in the forward after # Instance norm
if case to not reshape 2D tensors that did not forward in BatchInstanceNorm1d
now it forwards so I will give it a try ... are these modifications correct to you ?
class _BatchInstanceNorm(_BatchNorm):
    def __init__(self, num_features, eps=1e-5, momentum=0.1, affine=True):
        super(_BatchInstanceNorm, self).__init__(num_features, eps, momentum, affine)
        self.gate = Parameter(torch.Tensor(num_features))
        self.gate.data.fill_(1)
        setattr(self.gate, 'bin_gate', True)

    def forward(self, input):
        self._check_input_dim(input)

        # Batch norm
        if self.affine:
            bn_w = self.weight * self.gate
        else:
            bn_w = self.gate
        out_bn = F.batch_norm(
            input, self.running_mean, self.running_var, bn_w, self.bias,
            self.training, self.momentum, self.eps)

        # Instance norm
        b, c  = input.size(0), input.size(1)
        if self.affine:
            in_w = self.weight * (1 - self.gate)
        else:
            in_w = 1 - self.gate

        # input = input.view(1, b * c, *input.size()[2:])
        # this did not forward in the following F.batch_norm(input,...) for 2D input tensors and 1d norm ..
        if input.dim() != 2:
            input = input.view(1, b * c, *input.size()[2:])

        out_in = F.batch_norm(
            input, None, None, None, None,
            True, self.momentum, self.eps)

        # out_in = out_in.view(b, c, *input.size()[2:])
        # same as before
        if input.dim() != 2:
            out_in = out_in.view(b, c, *input.size()[2:])
            out_in.mul_(in_w[None, :, None, None])
        else:
            out_in.mul_(in_w)

        return out_bn + out_in

Hi, @adrienchaton
We didn't consider 1d case because Instance Normalization loses its effect in that case.
I.e., for a 2d tensor of (batch size, channels), Instance Normalization makes all elements become zero because the instance-wise mean equals the input tensor itself.
Nevertheless, it is worth making BIN capable of 1d case by disabling instance normalization (which is what Switchable Normalization did), which I will update soon.
Thank you!

That is correct, I didn't take this into consideration.
Thank you for your answer, this update makes sense as for SN.


SPARC INITIATIVE data submition NIH OT2-025340

This Github issue is synchronized with Zendesk:
Ticket ID: #32673
Group: Support
Requester<EMAIL_ADDRESS>Issue escalated by: Anita Bandrowski
URL:
Original ticket description:

Dear Dr. Pelot ,
My name is Anna Pilko and I am very excited to be the new Research
Coordinator in the SPARC Initiative. I am sending this email to introduce
myself and to thank you for your contribution to the project.
Having ~20 years of experience in molecular biology, I understand that your
time is extremely valuable so I will try to minimize what the curation team
asks of you. However, some actions on your part might be required once I
review all of the submitted files.
This email is to let you know that I will be looking at your files in the
coming 1-2 weeks.
Please expect to hear from me either if I encounter any problems or once
the process is complete. Should you have any questions and/or concerns
please do not hesitate to reach me at<EMAIL_ADDRESS>I appreciate your understanding and apologize for delays in data processing
as we are dealing with a backlog caused by the implementation of new
templates and changes in staffing at UCSD.
Sincerely,
Anna (Anka) Pilko, Ph.D
Research Data Coordinator
UC San Diego, Atkinson Hall, 9500 Gilman Dr, La Jolla, CA 92093
https://orcid.org/0000-0003-2353-6310

Comment made from Zendesk by Anna Pilko on 2019-08-29 at 19:12:

Good afternoon Dr. Pelot,
I have verified receipt of 9 data sets uploaded to the BlackFlynn portal.
The unique dataset ID associated with your award  that I recognized in the
systems are the following:
N:dataset:3951ecbd-e20f-4650-8385-1b45f4a9b7fb
<https://app.blackfynn.io/N:organization:618e8dd9-f8d2-4dc4-9abb-c6aaab2e78a0/datasets/N:dataset:3951ecbd-e20f-4650-8385-1b45f4a9b7fb>
N:dataset:55c5b69c-a5b8-4881-a105-e4048af26fa5
<https://app.blackfynn.io/N:organization:618e8dd9-f8d2-4dc4-9abb-c6aaab2e78a0/datasets/N:dataset:55c5b69c-a5b8-4881-a105-e4048af26fa5>
N:dataset:6fa2666c-aa3d-4e27-a405-7848fc061b04
<https://app.blackfynn.io/N:organization:618e8dd9-f8d2-4dc4-9abb-c6aaab2e78a0/datasets/N:dataset:6fa2666c-aa3d-4e27-a405-7848fc061b04>
N:dataset:217920c9-08cc-4680-99ff-d321b0484098
<https://app.blackfynn.io/N:organization:618e8dd9-f8d2-4dc4-9abb-c6aaab2e78a0/datasets/N:dataset:217920c9-08cc-4680-99ff-d321b0484098>
N:dataset:be6c03c9-e427-488a-9dea-f4f64fca1f14
<https://app.blackfynn.io/N:organization:618e8dd9-f8d2-4dc4-9abb-c6aaab2e78a0/datasets/N:dataset:be6c03c9-e427-488a-9dea-f4f64fca1f14>
N:dataset:c082215b-6765-4cc8-a965-92773b4c18c1
<https://app.blackfynn.io/N:organization:618e8dd9-f8d2-4dc4-9abb-c6aaab2e78a0/datasets/N:dataset:c082215b-6765-4cc8-a965-92773b4c18c1>
N:dataset:92f31530-41e6-4844-8ed8-6d1d534254f5
<https://app.blackfynn.io/N:organization:618e8dd9-f8d2-4dc4-9abb-c6aaab2e78a0/datasets/N:dataset:92f31530-41e6-4844-8ed8-6d1d534254f5>
N:dataset:6d6818f2-ef75-4be5-9360-8d37661a8463
<https://app.blackfynn.io/N:organization:618e8dd9-f8d2-4dc4-9abb-c6aaab2e78a0/datasets/N:dataset:6d6818f2-ef75-4be5-9360-8d37661a8463>
N:dataset:bec4d335-9377-4863-9017-ecd01170f354
<https://app.blackfynn.io/N:organization:618e8dd9-f8d2-4dc4-9abb-c6aaab2e78a0/datasets/N:dataset:bec4d335-9377-4863-9017-ecd01170f354>
You should hear back from me within a 3-4 week time frame as I process
files gradually.
Best,
*Anna (Anka) Pilko, Ph.D*
Research Data Coordinator, FDI Lab
UC San Diego, Atkinson Hall, 9500 Gilman Dr, La Jolla, CA 92093
https://orcid.org/0000-0003-2353-6310
On Wed, Aug 28, 2019 at 11:01 AM anita bandrowski<EMAIL_ADDRESS>wrote:
> Thank you Nikki,
>
> Just chiming in here. We are trying to work with BF to ensure that the
> basic metadata is in one place and needs to be entered only one time by
> investigators and that it is clear where that needs to be entered and
> checked. For ISAN, we took some shortcuts, that are not appropriate for the
> long term project.  We are trying now to "un-shortcut" some of those. Thank
> you for your continued support and patience, with Anna on board, we are
> fully staffed and should get back to you much more quickly.
>
> Best,
> anita
>
> On Wed, Aug 28, 2019 at 10:56 AM Nikki Pelot<EMAIL_ADDRESS>wrote:
>
>> Hi Anna,
>>
>> My name is Nikki Pelot, and I'm Research Program Director for the Grill
>> SPARC award at Duke University. Thank you for the intro email.
>>
>> Our team has uploaded 9 histology datasets (plus metadata and protocols).
>> We have not received any feedback on them, so hopefully they are correct
>> and complete.
>>
>> Moving forward, we will be uploading computational modeling data, which
>> does not fit the BIDS/metadata/protocol templates. However, we will be
>> uploading our model inputs & outputs, with clear documentation; we will be
>> uploading model code at a later date to accompany manuscripts, as per
>> discussions with the NIH SPARC PMs.
>>
>> Lastly, I've been in discussions with Anita, Joost, and Andrew about the
>> metadata describing a dataset. The datasets display differently in
>> Blackfynn vs. Blackfynn Discover vs. SPARC portal. Those result pages
>> should be clear in the dataset description (title, ordered authors with
>> affiliations, abstract, DOI, tags/keywords ...). Further, there are
>> metadata uploaded in the dataset (e.g., dataset_description.xlsx), but also
>> metadata fields on Blackfynn. So before the datasets go public, the
>> investigators should have an opportunity to ensure that their dataset
>> descriptions in particular are complete, accurate, and displaying property.
>>
>> Thanks,
>>
>> Nikki
>>
>> -------------------------------------------------------
>> Nikki Pelot, PhD
>> Research Director
>> Duke University | Biomedical Engineering | Grill Lab
>><EMAIL_ADDRESS>>><PHONE_NUMBER>
>> <http://nikkipelot.weebly.com>nikkipelot.weebly.com
>> ------------------------------
>> *From:* Anna Pilko<EMAIL_ADDRESS>>> *Sent:* Monday, August 26, 2019 8:54 PM
>> *To:* Nikki Pelot<EMAIL_ADDRESS>>> *Cc:* Anita Bandowski<EMAIL_ADDRESS>Gabi Pine <
>><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>>> *Subject:* SPARC INITIATIVE data submition NIH OT2-025340
>>
>>
>> Dear Dr. Pelot ,
>>
>> My name is Anna Pilko and I am very excited to be the new Research
>> Coordinator in the SPARC Initiative. I am sending this email to introduce
>> myself and to thank you for your contribution to the project.
>>
>> Having ~20 years of experience in molecular biology, I understand that
>> your time is extremely valuable so I will try to minimize what the curation
>> team asks of you. However, some actions on your part might be required once
>> I review all of the submitted files.
>>
>> This email is to let you know that I will be looking at your files in the
>> coming 1-2 weeks.
>>
>> Please expect to hear from me either if I encounter any problems or once
>> the process is complete. Should you have any questions and/or concerns
>> please do not hesitate to reach me at<EMAIL_ADDRESS>>>
>> I appreciate your understanding and apologize for delays in data
>> processing as we are dealing with a backlog caused by the implementation of
>> new templates and changes in staffing at UCSD.
>>
>>
>>
>> Sincerely,
>>
>> Anna (Anka) Pilko, Ph.D
>>
>> Research Data Coordinator
>>
>> UC San Diego, Atkinson Hall, 9500 Gilman Dr, La Jolla, CA 92093
>>
>> https://orcid.org/0000-0003-2353-6310
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__orcid.org_0000-2D0003-2D2353-2D6310&d=DwMFaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=h4OsZ729ciZBRM1NoCsLs8PVKHz05TCTQHVOCTMH4nE&m=u9ihg3v0lrOvNwr33BCcNZnsQKGR2kKMFdK0RvhLT7Y&s=G9NCB5Pp3LKDc4J6JgXzkuXDmMZKkiMIl-kMFtochLA&e=>
>>
>>
>>
>
> --
> All key biological entities deserve an #RRID!
> orcid.org/0000-0002-5497-0243
>
>


Change EC2 to RDS and point to correct example

The rds example looks like it has a few copy/paste artifacts. Cleaning those up to ensure consistency. I'm unable to actually get this test running as it continually times out at 10 mins.

The relevant tests passed (there were some failures, but they are unrelated to this and are known to be brittle). Going to merge. Thanks again for your contribution!


use .env for account id and target region.

Use .env for AWS_ACCOUNT_ID and TARGET_REGION.
Change to use TARGET_REGION.
Change to-do-list/user-service to use .env.
Remove "user-service" from remove.sh shell script.  Change to manually remove.

OK


[Feature Request] Fallback to run from arun

I prefer async LLM calls in my code, but need fallbacks for LLMs that do not support. My code lets users supply their own LLMs.
It would be nice to automatically do this fallback to run if arun will not work with the given LLM. Something like mychain.arun(..., fallback=True).
The work around I use is below:
class FallbackLLMChain(LLMChain):
    """Chain that falls back to synchronous generation if the async generation fails."""
    async def agenerate(
        self,
        input_list: List[Dict[str, Any]],
        run_manager: Optional[AsyncCallbackManagerForChainRun] = None,
    ) -> LLMResult:
        """Generate LLM result from inputs."""
        try:
            return await super().agenerate(input_list, run_manager=run_manager)
        except NotImplementedError as e:
            return self.generate(input_list, run_manager=run_manager)

It might be useful in repo though.

@agola11 @nfcampos thoughts on adding something like this base chain class? could raise a specific AsyncNotImplementedError or something?

We really should not be doing blocking calls in async coroutines as it blocks the main runloop. A better approach would be loop.run_in_executor. I think instead of having a dedicated fallback chain, we can just make the default implementation of agenerate use run_in_executor to call the sync method. This is what I did with vector stores for example: https://github.com/hwchase17/langchain/blob/f3ec6d2449f3fe0660e4452bd4ce98c694dc0638/langchain/vectorstores/base.py#L145
Ideally, all most LLMs/blocking stuff move towards having actual async implementations as that is usually way better than awaiting run_in_executor (no worries about thread safety, easer to debug, better error propagation, etc).

lets make that the default method then! better than breaking


CAMEL-11497: Migrate FAQ - remaining questions

https://issues.apache.org/jira/browse/CAMEL-11497
All FAQs are migrated by this commit.

Thanks :1st_place_medal:


[Agent] Add plugin for Activemq 5.x

Please answer these questions before submitting pull request

Why submit this pull request?

[ ] Bug fix

[x] New feature provided

[ ] Improve performance

Related issues

Bug fix

Bug description.

How to fix?

New feature or improvement

Describe the details and related test reports.

@WithLin CI is still broken.
@ascrutae Please run this pr with test case pr: https://github.com/SkywalkingTest/skywalking-agent-testcases/pull/20

hi, I have solved the project file, but CI still reported the same error.

Coverage decreased (-0.005%) to 24.585% when pulling 910d03863003c6857dd13c2c9e4fc41ac0d048a2 on WithLin:master into a5d261039278019b6d288d72a760ba3beda7444a on apache:master.

Here is the test report and log

The newest test report looks very good.
Just 2 version(5.13.4, 5.10.0) tested fail. @WithLin Do you want to fix them or after this PR merge?

I suggest you send a new pr to change the number to Constant with meaning name. Such as destination type.


Managing case when several row groups per file for 'remove_row_groups()'

Hi @martindurant
I would like to make following proposal to manage the critical flaw you identified.
Here is the general logic:

if file written by fastparquet and file scheme different than flat, proceed 'normally' (no check)
if file with file scheme flat, this corresponds to the case you highlight: several files written with file scheme = simple, and then 'merged' together in a single ParquetFile object. In this case, check if the row group removal will lead to removing files that contain both row group to be kept and row group to be removed
if file not written by fastparquet, same as situation 2 above (when file scheme is flat)

In situations 2 and 3, a map of all row group is created. It is a dict in which file names are the keys, and values are the list of row group each file contain.
A similar map for row groups to be removed is also created, and a comparison of row group lists is done file per file (for file to be removed.
It seems to me that this manages this critical flaw.
I have added 2 test cases for testing this:

with data written by pyarrow (through pandas)
with file scheme is flat

For each case, files contain 2 row groups.
I am testing that an error is correctly raised if removing a single group, or that the removal proceeds when removing 2 row groups in the same file.
Please, there is one thing in the code you implemented with default_remove.
There is an except IOError both in  default_remove and in remove_row_groups.
There is redundancy here. Shouldn't one be removed?
Also, there should be no more error now with new code, because file removal is done once per file, on the opposite to previous code, when file was trying to be removed as many times as there are row groups to be removed within.
Bests

Thanks @yohplala


cdb release failed

Currently the cdb release fails with:
"Error validating package: Error while reading /mods/vacuum-moon/mod.conf: Line 3: Expected line to contain \'=\'"

This here is the cause:
https://github.com/FreeLANMan/RealMoon2021/blob/418d3628992fa2afd93bc16ddeb102ff8c332b06/mods/vacuum-moon/mod.conf#L3-L4
Those 2 lines are settings and don't really belong in a mod.conf

Thanks


working version of atdx calculation with a small tool_parser

Closed because we discussed on Zoom and will use another PR for addressing the feedback.


Set default behavior of rejects

Type of change

Refactoring

Description

Ensure that MODIFIED outcome is used for transient errors
Ensure that REJECT outcome causes a retry instead of ending up on the
global DLQ

@k-wall Should we make this option (especially the REJECT behavior) configurable in the infra configs?
Checklist

[ ] Update/write design documentation in ./documentation/design
[ ] Write tests and make sure they pass
[ ] Update documentation
[ ] Check RBAC rights for Kubernetes / OpenShift roles
[ ] Try your changes from Pod inside your Kubernetes and OpenShift cluster, not just locally
[ ] Reference relevant issue(s) and close them after merging
[x] Update CHANGELOG.md

@enmasse-ci run tests profile=shared

:x: Test Summary :x:
TEST_PROFILE: shared
TEST_CASE:
TOTAL: 139
PASS: 126
FAIL: 1
SKIP: 12
BUILD_NUMBER: 229
:heavy_exclamation_mark: Test Failures :heavy_exclamation_mark:

testWebSocketSendReceiveQueue in io.enmasse.systemtest.shared.standard.web.ChromeWebSocketBrowserTest

@enmasse-ci run tests profile=shared

:heavy_check_mark: Test Summary :heavy_check_mark:
TEST_PROFILE: shared
TEST_CASE:
TOTAL: 139
PASS: 127
FAIL: 0
SKIP: 12
BUILD_NUMBER: 232

@enmasse-ci run tests profile=isolated

:x: Test Summary :x:
TEST_PROFILE: isolated
TEST_CASE:
TOTAL: 98
PASS: 89
FAIL: 3
SKIP: 6
BUILD_NUMBER: 233
:heavy_exclamation_mark: Test Failures :heavy_exclamation_mark:

testConnectorTLS in io.enmasse.systemtest.isolated.bridging.ConnectorsTest
testReceiveThroughConnector2 in io.enmasse.systemtest.isolated.bridging.ConnectorsTest
testForwardToFullQueue in io.enmasse.systemtest.isolated.bridging.ForwardersTest

@enmasse-ci run tests testcase=ForwardersTest#testForwardToFullQueue

:heavy_check_mark: Test Summary :heavy_check_mark:
TEST_PROFILE: pr
TEST_CASE: ForwardersTest#testForwardToFullQueue
TOTAL: 1
PASS: 1
FAIL: 0
SKIP: 0
BUILD_NUMBER: 241


minCount/maxCount for number/date length.

Would be nice if minCount and maxCount would be available to check for length of number/date as string length, where you'd probably also convert strings to count validators instead reserving min/max for actual ranges.

I'm not sure I understand this. Can you post some example schemas and data?

as in when u measure the length of string/number/integer it should
logically be minCount/maxCount.
On Tue, Apr 17, 2018 at 3:44 AM, Eric Dobbertin<EMAIL_ADDRESS>wrote:

I'm not sure I understand this. Can you post some example schemas and data?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/aldeed/simple-schema-js/issues/253#issuecomment-381759955,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKz-ga6OoOypsxyf6rMcUtQSQQgNrrbUks5tpRCsgaJpZM4TVD9X
.

The reasons for this might go back to when you could define array and array item in the same definition, which isn't possible anymore. So it might be possible to use a clearer name like minLength for string character length.
I'm not sure what you mean by measuring the "length" of a number. The number of digits? That would be a new feature, but I can't think of why you would want to do that.

Some numbers are better expressed in length.


components bom-ref scanning issues

Trying to run cyclonedx analyze, or grype , trivy to check for vulnerabilities on the exported SBOM file you'll receive errors.
The reason is that the bom-ref contains the purl and not a hash.
Example grype output:
failed to catalog: unable to decode sbom: unable to decode cyclonedx json document: json: cannot unmarshal object into Go struct field Component.components.hashes of type []cyclonedx.Hash
Changing the component bom-ref to a hash would solve that issue. Any plans to do that?

@vchaindz, you can use depscan since not sure if trivy and grype supports CycloneDX 1.5 specification properly.

thanks @prabhu good to know. depscan works. I'll add blint in combination with depscan then soon to https://sbom.sh.
Therefore I'm also interested in having streaming support in python, to produce and upload SBOMs in a simple way.
Just FYI - I changed the bom-ref to a hash instead of a purl and grype and trivy work. Seems they definitely have an issue when bom-ref is not a hash.

Thank you. Will keep this issue open and add validation to blint generated json. Best to file an issue against syft or cyclonedx-go to find out more about the bom-ref issue.

2.1.0 includes a fix. For some reason, hash was presented as an object. Strongly suspect some pydantic bug must have gotten fixed since when I made this particular change, hashes were getting dumped as a nested array breaking validations.

Great - thanks for the quick fix!

Have release 2.1.1 which includes a fix for dotnet. It was an extra comma that was resulting in validation failures prompting me to change the hash logic.


Avoid too large time steps in end time selection

I am not sure if this is a expected behaviour or not.
In my current app i want to set a minimum rental period of 2 hours and if i set the timeBlock value to 120, the end time goes in 2 hour steps. I would expected it would take the timeStep value.
State of master:
Code:
TimeRange(
...
timeStep: 60
timeBlock: 120
...
),

Result:
Start:
6:00 | 7:00 | 8:00 | 9:00
End:
9:00 | 11:00 | 13:00 | 15:00
*Bold start time is the selected time.
Expected Result:
Start:
6:00 | 7:00 | 8:00 | 9:00
End:
9:00 | 10:00 | 11:00 | 12:00 | 13:00

Hi @mfrischbutter, thanks for you contribution, but The widget is designed to select one or more time blocks, where each block must have what is defined in timeBlock. If we apply the change you propose, a user could select a range that does not contain a multiple of the time defined in timeBlock.
I think one way for the widget to cover this need is to have a new parameter that indicates the minimum range:
TimeRange(
...
timeStep: 60
timeBlock: 60
minimumRange: 120
...
),

The result of this would be:
Start:
6:00 | 7:00 | 8:00 | 9:00
End:
9:00 | 10:00 | 11:00 | 12:00 | 13:00


when applying this lua require('lspsaga.rename').rename() on non-identifiers or builtin keywords  it doesn't warn me and does nothing

I encountered this issue in gopls server,
I expected something like throw an error in status line

Gopls did not return any check messages, so the rename of other lsp plugins should also behave consistently, such as the built-in codename, and it did not throw any errors.
The solution to this problem I can think of is to maintain a Go keyword table to check whether the current cursor word matches


Some internal links written in Reference JSDoc comments are broken

Relative links written in JSDoc comments are broken with the new output structure. These links typically take this format #/p5/noFill, work was done to make them relative with "./" instead of "#/" but the new structure won't handle that properly. We should address this at the building script

Partially addressed by #64
Closing in favor of a more direct request for changes in upstream sources


PC-904 Include Shell referrals in Octopus csv

closes PC-904
though the ticket implies this is backend work, the actual substitution takes place on the frontend
follows the same format as the other converter logic, where on submission the 'supplier' value is changed
note the 'user_selected_supplier' field on the csv will always show the supplier selected, fulfilling the requirement to be able to differentiate pure octopus referrals
includes a new test to verify

One thing on the ticket that I want to check is that we have the requirement that Shell referrals should have "Shell" noted against them somewhere in the export CSV. Does that already happen automatically, or is it missing?

@jdgage yes - the export csv always has a 'user selected supplier' field which is always what the user selects on the supplier select page


Unified addTestDevice - no fill error

This PR resolve #34.
After adding [FBAdSettings addTestDevice:[FBAdSettings testDeviceHash]]; I eventually was able to display ads on iOS simulator. Previously I always got 1001 no fill error.
Unfortunately, Android SDK does not have testDeviceHash method so I had to check how AdSettings works from decompiled code.

Great! We did the same in Exponent.
CC: @satya164

@grabbou @satya164 I've just pushed fixes.

Is this why I'm not seeing ads on iOS? https://github.com/callstack-io/react-native-fbads/issues/45#issuecomment-283134703
How do I get this to work on 3.1.1 (I'm using React Native 0.37).

Yes, @booboothefool, that was the reason. Should be gone when we release a new version.


Light and contrast

Added QStyle icon instead of icononbuttonhandler. The button handler code is still there but in comments right now in case we need it again or still wanna use it.

We wait with this until we know which icons we need.


TST: fix test failures from Matplotlib 2.1 update

Closes gh-7997.

Ugh forgot that there are also a bunch of refguide check errors that need to be fixed too.

Thanks, merging. Added some stopwords to make refguide check pass.
OSX still fails, but apparently due to some different reason (pip or homebrew failure) :/

I wonder if @matthew-brett would understand the failure better?

Not sure. the osx failure is probably sporadic, seems to have appeared by itself and maybe goes away in the same way

Or, maybe there was some recent change in multibuild we should adapt to?

You know what, hombrew no longer works on Travis OSX. That might be it.

Any more info about that? E.g. is it gone for good or just temporary?

travis-ci/travis-ci#8552

added HOMEBREW_NO_AUTO_UPDATE=1 to travis settings, which seems to have fixed it

backported to 1.0.x in gh-8041


readme example is not working

expected result: resp including a vertice id
but the real result: [SERVER SERIALIZATION ERROR]
// replace map[string]string{"x": "1234"} with this:
map[string]string{"x": "'1234'"},
// but got
// [<nil>]

gremgo simply set the binding and rebinding data to the Args of the request to gremlin server.
	req.Args["bindings"] = bindings
	req.Args["rebindings"] = rebindings

It works on my side with Azure Cosmos DB.


feat: add support for secure TLS bootstrapping to CSE

What type of PR is this?

/kind feature
What this PR does / why we need it:
Adds support for secure TLS bootstrapping on agent nodes - for now we only install the plugin at runtime since we're using an alpha version of the client-go plugin. Will add a step to the VHD build in a subsequent PR.
Which issue(s) this PR fixes:

Fixes #
Requirements:

[ ] uses conventional commit messages

[ ] includes documentation
[ ] adds unit tests
[ ] tested upgrade from previous version

Special notes for your reviewer:
Release note:
none

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.008%) to 79.178%

Totals

Change from base Build<PHONE_NUMBER>:
-0.008%

Covered Lines:
2236

Relevant Lines:
2824

💛  - Coveralls


addition to describe ARM's per instance throttling

@rjmax,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

@rjmax : Thanks for your contribution to the Azure documentation! The author, @tfitzmac, has been notified to review your proposed change.

@rjmax - thanks, Ryan!

#sign-off


Document is specifically calling out Project 2013

Why does this page specifically call out Project 2013? This is confusing for customers the way it is worded.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 28e6f1e8-57dc-a912-63ca-ac19ffea6857
Version Independent ID: 5d3a8a46-0e3b-3f76-32b3-396f30c1f2ed
Content: Task pane add-ins for Project - Office Add-ins
Content Source: docs/project/project-add-ins.md
Service: project
Sub-service: add-ins
GitHub Login: @o365devx
Microsoft Alias: o365devx

Hi @mheisenberg,
Thanks for flagging this obsolete reference in the Project add-in overview article. I'll update that soon.


[Doc] search result links don't work

When using the documentation's search box, one gets 404 errors when clicking the result links. Example at https://docs.scipy.org/doc/scipy/reference/search.html?q=sum_duplicates&check_keywords=yes&area=default

None of those search results seems to 404 for me. Can you copy a link here that 404s?

For example:
https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.bsr_matrix.sum_duplicatesundefined?highlight=sum_duplicates#scipy.sparse.bsr_matrix.sum_duplicates
I checked in two different browsers (Chromium, Firefox)

Ok, I see now after clearing cache. I think the problem is gh-11971

Hmm, I see. Annoying.

Hotfix applied.

Wow, thank you!


work around showing map on modals

Hi everyone,
Just in case, if you'll use a map over a modal it doesn't show correctly by default, indeed displays a "Grey" area's when it is placed inside a dynamic element. So do you need the next work around:
var currentCenter = GoogleMaps.maps.yourMap.instance.getCenter(); google.maps.event.trigger(GoogleMaps.maps.yourMap.instance, "resize"); GoogleMaps.maps.loadCenterMap.instance.setCenter(currentCenter); // Re-set previous center
The first and the third lines are just for reset the center because the resizing affects the center.
I lost a lot of time thinking that the problem was in api or something like that.

Also see #58


Webmentions from Bluesky posts with GIFs contain no content

I got a couple webmentions for a post of mine that I syndicated to Bluesky. Two replies to that post were just GIFs, and the webmentions I got for them don't appear to have any content:

https://bsky.app/profile/evanphx.dev/post/3lb4k6ek7ys2e

Webmention

https://bsky.app/profile/wafer.baby/post/3lb4kkb7pmk2t

Webmention

If you check out the bridg.gy source URLs, you'll see that the h-entry contains no content at all. Just the p-uid, dt-published, p-author, u-url and the u-in-reply-to URLs.

Interesting, good point! GIFs are just external embeds in Bluesky records, I thought we handled those, but we don't do anything special for GIFs, so maybe we need to.
Example for your first post, at://evanphx.dev/app.bsky.feed.post/3lb4k6ek7ys2e (pruned):
{
  "$type": "app.bsky.feed.post",
  "createdAt": "2024-11-17T04:14:18.328Z",
  "embed": {
    "$type": "app.bsky.embed.external",
    "external": {
      "description": "ALT: a man with long hair and a beard has the words that 's a lot on his face",
      "thumb": {
        "$type": "blob",
        "ref": {"$link": "bafkreiec76e7hexvvi2qx57e74kci7i6jffl5kdwadnv7uml4quvy6sohm"},
        "mimeType": "image/jpeg",
        "size": 179303
      },
      "title": "a man with long hair and a beard has the words that 's a lot on his face",
      "uri": "https://media.tenor.com/m-uAEtLAfgsAAAAC/thats-a-lot-samus-paulicelli.gif?hh=498\u0026ww=498"
    }
  },
  "text": ""
}

wait do they really serve that blob with a MIME type of image/jpeg 😭

Ah, that blob isn't the GIF itself, it's the thumbnail, which is a JPEG. 😁


Creat Account very slow

Xcode Version 9.4.1 (9F2000)
Simulator Version 10.0 (SimulatorApp-851.2 CoreSimulator-518.22) iPhone 6
System version iOS 11.4
TrustKeystore Verision 0.4.1
  pod 'TrustKeystore', :git=>'https://github.com/TrustWallet/trust-keystore', :branch=>'master'

I use KeyStore to create an account, the code is as follows
//keyStore
public let documentDir = NSSearchPathForDirectoriesInDomains(FileManager.SearchPathDirectory.documentDirectory, .userDomainMask, true).first ?? ""
public let keyStore = try? KeyStore(keyDirectory: URL(fileURLWithPath: documentDir).appendingPathComponent("ne"))

    @available(iOS 10.0, *)
    func createAccount(with password: String, completion: @escaping (Result<Account, KeystoreError>) -> Void) {
        DispatchQueue.global(qos: .userInitiated).async {
            let account = self.createAccout(password: password)
            DispatchQueue.main.async {
                completion(.success(account))
            }
        }
    }

    func createAccout(password: String) -> Account {
        let account = try! keyStore?.createAccount(password: password, type: .encryptedKey)
        return account!
    }

But the creation time is very long,there are even 10s. The same,Trust only takes 1s,I use Instruments to check the time.Test results as shown, first one is my project, second is Trust.
I found that there is a "specialized" in front of the thread of Trust starting from "KeyStoreKey.init()".
I want to know why this is and What can I do to be the same as Trust?
Thank you  very much.

You need to enable compiler optimizations.

@alejandro-isaza-hs It does not work , still thank you.

I found the most time-consuming code

Surprisingly, in the Trust this code is skipped and there is no explicit execution
what is this？

@vikmeup Can you provide some suggestions？

@alejandro-isaza-hs has more knowledge on this.

@vikmeup thank you very much!

@alejandro-isaza It works,really thank you so much.


Cannot read properties of null (reading 'insertBefore')

Version
3.2.18
Reproduction link
github.com
Steps to reproduce
I created this project to reproduce it. https://github.com/Guervyl/vueJs3-not-updated. But, I could not be able to get the error. From this test repository, the value just not updated.
And this error is weird. It's nothing like the others. I made a video so that you can see what's going on. Before you comment anything please watch the video. There is no error like this. Here is the link: https://drive.google.com/file/d/1cP-4a4rZ69bBxQphzs7hhPQgN0Ozol4m/view?usp=sharing .
It's hard to explain what's going on. When I access a property on the object elementObj.type === 'link' I got the error. When I remove the code, I don't get the error. And this only happens when I after I roughly removed some existed property from the object outside of vuejs.
What is expected?
It should not throw error and the view was supposed to be updated.
What is actually happening?
It throws an error.

Please watch the 5 minutes video to be able to understand what the problem is.

You are essentially updating the original non-reactive object (outside Vuejs) and expect Vue's reactive proxy for that object to update. That doesn't work, Vue doesn't get informed about changes to the nonreactive original opject.
Consequently, when removing a property from the original object, Vue's reactive proxy for that object thinks it's still there and all sorts of funky stuff can go wrong.

For Vue 3, reactivity works through proxies. So essentially: Do NOT set an external object on this then expect the original object to become "reactive". This worked in Vue 2 but won't in Vue 3. You can only work through the reactive proxy of the object (through this.object)
This will be more clearly documented in the new docs, but if you are coming from Vue 2, make sure to understand the difference.

It's great that vue3 does not touch the original object.
The thing is: I'm not modifying the external/original object and expect this.object (this.elementObj) to be reactive. I:

modify the original object outside of Vue3.
then, assign or set the original object to this.object (this.elementObj) inside Vue3.

Every time I change the original object, I assign it to this.object.
Assigning the external object to Vue3, Vue3 was supposed to create a new reactive object or proxy. Because I assign a new object reference.
this.object is not gonna hold one single object for the hole entire application lifetime. this.object is gonna have (will be assign) a new object based on the element that the user selects.
Here is a presentation of what I'm doing: https://drive.google.com/file/d/16MONa-7LFF_2imALWW7DxrE5cFd5LvXY/view?usp=sharing

The weird thing about it is the fact that the error is thrown only if I access a property from the object. It assigns the new object with no problem. I just can't figure out what is going wrong. Even if I try to check if the property exists 'type' in this.elementObj ? this.elementObj.type : undefined I got an error because of this.elementObj.type, because the property type exists at the time I assign the new object to this.object.
I even try to invalidate this.object by assigning it to null this.object = null, I get the error because I set it to null and I even check v-if="object" to not render the component if it's null or undefined.
The output is:
[Vue warn]: Unhandled error during execution of scheduler flush. This is likely a Vue internals bug. Please open an issue at https://new-issue.vuejs.org/?repo=vuejs/vue-next
  at <App>
warn @ runtime-core.esm-bundler.js:6673
logError @ runtime-core.esm-bundler.js:6847
handleError @ runtime-core.esm-bundler.js:6839
callWithErrorHandling @ runtime-core.esm-bundler.js:6793
flushJobs @ runtime-core.esm-bundler.js:7026

Uncaught (in promise) TypeError: Cannot read properties of null (reading 'insertBefore')
    at insert (runtime-dom.esm-bundler.js:277)
    at processCommentNode (runtime-core.esm-bundler.js:3833)
    at patch (runtime-core.esm-bundler.js:3785)
    at patchBlockChildren (runtime-core.esm-bundler.js:4108)
    at processFragment (runtime-core.esm-bundler.js:4169)
    at patch (runtime-core.esm-bundler.js:3796)
    at ReactiveEffect.componentUpdateFn [as fn] (runtime-core.esm-bundler.js:4409)
    at ReactiveEffect.run (reactivity.esm-bundler.js:195)
    at callWithErrorHandling (runtime-core.esm-bundler.js:6790)
    at flushJobs (runtime-core.esm-bundler.js:7026)
insert @ runtime-dom.esm-bundler.js:277
processCommentNode @ runtime-core.esm-bundler.js:3833
patch @ runtime-core.esm-bundler.js:3785
patchBlockChildren @ runtime-core.esm-bundler.js:4108
processFragment @ runtime-core.esm-bundler.js:4169
patch @ runtime-core.esm-bundler.js:3796
componentUpdateFn @ runtime-core.esm-bundler.js:4409
run @ reactivity.esm-bundler.js:195
callWithErrorHandling @ runtime-core.esm-bundler.js:6790
flushJobs @ runtime-core.esm-bundler.js:7026

It's great that vue3 does not touch the original object. I wasn't modifying the external object to expect Vue3 to react. I was assigning the external object to this.object after I modified the external object.
I found the problem. Somewhere in the code it was destroying the Vue html container.
Thanks for answers.


[HAC-2112] Common details page header component

Jira ticket: https://issues.redhat.com/browse/HAC-2112
This PR contains components for displaying the header section of a details page (breadcrumbs, title, labels, action buttons and action menu)
Test with yarn storybook

/retest

@vojtechszocs FYI this is another PR that is timing out in the test phase:
PASS lib-utils packages/lib-utils/src/k8s/hooks/useK8sWatchResources.XXXX.ts (10.837 s, 399 MB heap size)
PASS lib-utils packages/lib-utils/src/k8s/hooks/useK8sWatchResource.XXXX.ts (432 MB heap size)
PASS lib-utils packages/lib-utils/src/app/redux/ReduxExtensionProvider.XXXX.tsx (471 MB heap size)
{"component":"entrypoint","file":"k8s.io/XXXX-infra/prow/entrypoint/run.go:165","func":"k8s.io/XXXX-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Process did not finish before 30m0s timeout","severity":"error","time":"2022-09-16T14:13:13Z"}
{"component":"entrypoint","file":"k8s.io/XXXX-infra/prow/entrypoint/run.go:255","func":"k8s.io/XXXX-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Process did not exit before 15s grace period","severity":"error","time":"2022-09-16T14:13:28Z"}
{"component":"entrypoint","error":"process timed out","file":"k8s.io/XXXX-infra/prow/entrypoint/run.go:80","func":"k8s.io/XXXX-infra/prow/entrypoint.Options.Run","level":"error","msg":"Error executing XXXX process","severity":"error","time":"2022-09-16T14:13:28Z"}
error: failed to execute wrapped command: exit status 127
INFO[2022-09-16T14:13:36Z] Step all-packages-all-packages failed after 31m20s.
INFO[2022-09-16T14:13:36Z] Step phase XXXX failed after 31m20s.

/retest

For now marking /lgtm and /approve as this component is WIP and is not currently used by any plugins

/lgtm

Codecov Report
Base: 17.75% // Head: 22.64% // Increases project coverage by +4.88% :tada:

Coverage data is based on head (c2fcebb) compared to base (937d9aa).
Patch coverage: 62.56% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #156      +/-   ##
==========================================
+ Coverage   17.75%   22.64%   +4.88%
==========================================
  Files          44       57      +13
  Lines        1543     1753     +210
  Branches      412      464      +52
==========================================
+ Hits          274      397     +123
- Misses       1269     1355      +86
- Partials        0        1       +1

Impacted Files
Coverage Δ

.../details-page-header/DetailsPageHeader.stories.tsx
0.00% <0.00%> (ø)

...-utils/src/components/details-page-header/index.ts
0.00% <0.00%> (ø)

...etails-page-header/utils/ActionButtons.stories.tsx
0.00% <0.00%> (ø)

...s/details-page-header/utils/ActionMenu.stories.tsx
0.00% <0.00%> (ø)

.../details-page-header/utils/Breadcrumbs.stories.tsx
0.00% <0.00%> (ø)

...mponents/details-page-header/DetailsPageHeader.tsx
80.00% <80.00%> (ø)

...onents/details-page-header/utils/ActionButtons.tsx
92.30% <92.30%> (ø)

...omponents/details-page-header/utils/ActionMenu.tsx
96.55% <96.55%> (ø)

...mponents/details-page-header/utils/Breadcrumbs.tsx
100.00% <100.00%> (ø)

.../src/components/details-page-header/utils/index.ts
100.00% <100.00%> (ø)

... and 5 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


feat(note): add circleOfFifthsDistance getter

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

3 of 3 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
448

Relevant Lines:
448

💛  - Coveralls


click.delegate bindings stop working

I'm submitting a bug report

Library Version:
2.2.1 and above

Please tell us about your environment:

Operating System:
OSX 10.14 | Linux (CentOS)

Node Version:
10.15.2

NPM Version:
6.4.1

JSPM OR Webpack AND Version
JSPM 0.16.54

Browser:
Tested in: Chrome 73 | Firefox 66

Language:
ESNext

Current behavior:
With version 2.2.1 or 2.3.0 of aurelia-binding, all our click.delegate bindings stop working after navigating to a few different routes in our aurelia application. There is no console error, they simply stop working silently - the function call never happens.
We can load the app on a particular route/VM and the click.delegate bindings work fine. Then we navigate to a different route, come back to the original one, and they have stopped working.
Expected/desired behavior:
With version 2.2.0 of aurelia-binding, no problems occur - the click.delegate bindings continue to work after navigation.

@gavinaiken There is no change related to click handling between two versions.
Beside this, what else did you upgrade while upgrading binding to 2.3? Also please note that click.delegate works with bubbling, so if there's something stopping the propagation of event before it reaches the document, click delegate won't be able to work

I will see if I can make a simple repro but so far I have only been able to see it with our large aurelia application so I don't know if that will be possible. Will report back.
I'm sure it is something in this library because I updated all our aurelia dependencies at once (essentially all the core aurelia modules), observed the issue, then tested rolling them back one by one until the issue was resolved. I was able to upgrade everything else again so the situation we have now is that all the aurelia modules in our app are on the latest released versions except for aurelia-binding.
Looking at the diffs between 2.2.0 and 2.2.1 my suspicion falls on this commit: https://github.com/aurelia/binding/commit/56ba36daa0429f1164fb78208c3d082147f7f6bb but I'm not sure how I can test 2.2.0 plus that change to see if that exhibits the issue. Will look into that today.

Hmm. Is the README out of date for how to build the module? I did the following:
git clone git@github.com:aurelia/binding.git
cd binding
git checkout 2.2.0
npm i
gulp build

and got the following error:
n$ gulp build
internal/util/inspect.js:31
const types = internalBinding('types');
              ^

ReferenceError: internalBinding is not defined
    at internal/util/inspect.js:31:15
    at req_ (/Users/gavin/repos/aurelia/binding/node_modules/natives/index.js:137:5)
    at require (/Users/gavin/repos/aurelia/binding/node_modules/natives/index.js:110:12)
    at util.js:25:21
    at req_ (/Users/gavin/repos/aurelia/binding/node_modules/natives/index.js:137:5)
    at require (/Users/gavin/repos/aurelia/binding/node_modules/natives/index.js:110:12)
    at fs.js:42:21
    at req_ (/Users/gavin/repos/aurelia/binding/node_modules/natives/index.js:137:5)
    at Object.req [as require] (/Users/gavin/repos/aurelia/binding/node_modules/natives/index.js:54:10)
    at Object.<anonymous> (/Users/gavin/repos/aurelia/binding/node_modules/graceful-fs/fs.js:1:99)

gulp build works in master branch, but if I check out 2.2.0 I get the above. What am I doing wrong!?

The commit which breaks our app is this one: https://github.com/aurelia/binding/commit/a5d2b330d68b104ebd3503d97d52027bb4d9bc83
I tracked it down by checking out the master branch of aurelia-binding, symlinking the dist/amd file into my jspm folder, and reverting single commits from the diffs between 2.2.0 and 2.2.1 https://github.com/aurelia/binding/compare/2.2.0...2.2.1 until our app worked ok. To be sure it was just that commit I then went back to master again, and reverted just that single commit, and verified that it works ok.

It feels like you are hitting this https://github.com/nodejs/node/issues/25132
But i'm not sure why. I don't hit that.

I managed not to get derailed by the internalBinding error by working from the master branch, where gulp build works fine for me. I tracked down the commit which breaks my app, as per the above https://github.com/aurelia/binding/issues/747#issuecomment-481216664. Trying to dig further into why that is causing problems now.

@gavinaiken Cool, nice that you figured that out. Could you help with a repro?

Haha yeah the repro seems the hardest bit. If i can understand why that commit breaks my app I might be able to put a repro together...

Better than a repro, I have a PR with a fix: https://github.com/aurelia/binding/pull/748

Updated PR: https://github.com/aurelia/binding/pull/749

@gavinaiken sorry to have caused you that trouble.
(-:
And now I stepped into it as well..

Ha no worries, when you write software, sometimes things go wrong :)

well I obviously copy pasted w/o adding a significant test
which I intend to write now (-:


Use IANA as the vocabulary base for did:jwk

This PR removes the context dependency on JWS 2020.

Note that this will generate a "different RDF type" for JsonWebKey2020.... but does that really matter?
Maybe not, since DID Core did not define the RDF type for JsonWebKey2020.


feat: Add automatic generation of BTF archives

This PR will add automatic generation of BTF archives within a GitHub Action.
The push will be done to the btfhub repo if changes are found, automatically.
Sample run: https://github.com/simar7/btfhub-tools/runs/4270159313?check_suite_focus=true

One more point, we'll probably need to add lynx as a package to the runner:

Closing in favour of https://github.com/aquasecurity/btfhub/pull/18


undefined reference to `UNICHARSET::CleanupString[abi:cxx11](char const*, unsigned long)'

os: centos7
leptonica: 1.78.0
tesseract : master branch
run these below，every thing is ok：

git clone https://github.com/tesseract-ocr/tesseract.git
./autogen.sh
./configure --prefix=/usr/local/ --with-extra-libraries=/usr/local/lib

but，run make  an error raised like this：

......
/bin/sh ../../libtool  --tag=CXX   --mode=link g++  -g -O2 -std=c++17   -o tesseract tesseract-tesseractmain.o libtesseract.la -L/usr/local/lib -llept   -fopenmp    -lrt -lpthread
libtool: link: g++ -g -O2 -std=c++17 -o .libs/tesseract tesseract-tesseractmain.o -fopenmp  ./.libs/libtesseract.so -L/usr/local/lib /usr/local/lib/../lib64/libstdc++.so /usr/local/lib/liblept.so -lm -lrt -lpthread -fopenmp -Wl,-rpath -Wl,/usr/local/lib -Wl,-rpath -Wl,/usr/local/lib/../lib64
./.libs/libtesseract.so: undefined reference to UNICHARSET::CleanupString[abi:cxx11](char const*, unsigned long)' collect2: error: ld returned 1 exit status make[2]: *** [tesseract] Error 1 make[2]: Leaving directory /tmp/tesseract/tesseract/src/api'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/tmp/tesseract/tesseract'
make: *** [all] Error 2

Please provide full output of configure process and make process.
-std=c++17 and [abi:cxx11] indicate that something strange must happen to at your system.

i'm sorry to reply you so late, i resolve this issue by reinstall once again. I think it's my c++ compiler did not work.


Manually triggered inline completions do not show up

VS Code Insiders, Copilot
F1 > trigger inline completions
Nothing happens

Found by @hstaudacher

Thanks for the fix. Works for me fine in latest insiders -> verified


Nintendo 3DS PRUDP V1

Hi
I tried to create "server" for Metroid Prime FF to look into the packets just for fun.
But I have a problem with the packet signature in PRUDP V1.
The connection with the 3DS seems a little bit simpler as opposed to the Wii U. It just connects to nasc.nintendowifi.net, sends a request and gets a response that inlucdes an IP (Key named locator in the response) that is base64 encoded and a token that I have no idea how it is encoded. After it closes the connection to nasc it directly sends a SYN PRUDP V1 packet to the Game Server. The game seems to just accept my SYN packet from my server without a signature but the CONNECT packet that follows it will not accept. My CONNECT response is the exactly the same as the one from the real server, except the packet signature and the session ID. But the session ID seems to be a random number anyway so it shouldn't cause the game to not accept my packet.
I guess the 3DS doesn't use a secure key to calculate the signature because there is no such thing as a secure server, except it is hidden in the token from the nasc.
So the real problem I'm facing is how can I get the access key of Metroid Prime FF. Do you have any idea?

Hello,
The 3DS connection is actually pretty similar to the Wii U. The only difference is that 3DS games retrieve the address of the authentication server from nasc.nintendowifi.net and Wii U games request it from account.nintendo.net.
The token in the nasc response probably contains secret data that can only be decrypted by Nintendo's servers. You shouldn't need to worry about it. Wii U games pass the token to the authentication server as parameter in the LoginEx method. I guess 3DS games do the same. LoginEx then returns the address of the secure server and some other information. This is how the 3DS knows where it can find the secure server.
You are right that the session id is just a random number and it shouldn't cause any issues, but keep in mind that the packet signature depends on the content of the packet. If you use a different session id than the real server, you can't blindly copy the packet signature from the real server.
The only way to find the access key is by looking at the executable of the game. I don't know how much experience you have with this kind of things, but a relatively easy way to find it is to load the executable into IDA and look for a string that consists of exactly 8 lowercase hex digits.

There must be an access key somewhere, and I'm pretty sure they only contain hex digits in 3DS games too. The access key is not the only thing of the friend server that's different from other 3DS/Wii U servers, and (for example) the access key of Mario Kart 7 is 6181dff1.
The access key of SSB4 does indeed seem to be 2869ba38, although I haven't tested it myself either. If you give me your code.bin I can see if I can find the access key myself. Or maybe you can try to brute force it if you have a packet capture with the real server, because there are only 2^32 possible access keys and the HMAC calculation can be done pretty fast.

I guess I found the access key now. The game doesn't have the access key directly written into its code instead its saved in a plain text .ini file that is located in romfs/ini/skuinfo/.
This file also includes the CTR Game ServerID and a bunch of other stuff.
Here is the complete text of the file
CTR ProductCode = CTR-P-BCAP
CTR UniqueID = 0x01752
CTR OtherUniqueIDForDSDownload = 0x01753
CTR GameServerID = 0x0016CE00
CTR GameServerAccessKey = f79fb3c5
CTR AccessibleSaveDataId1 = 0x01753
Yes just 6 lines of plain text no more

Interesting. I'll add it to the Game Server List.
Feel free to reopen this issue if you have any further questions.

I don't see anything wrong with your python code, but I think you meant you stripped the first 4 bytes instead of the first 6 bytes. Could you send me your packet dump? Maybe I can find out why the packet signature is different.

I've found it out myself. Don't know if this is just different for 3DS games or if this applies to Wii U games too but in the SYN packet when the consoles doesn't know the connection signature of the server it doesn't use it for calculation at all. In other words commenting the connection signature update line should fix the problem.

Nice finding! I checked my client code (prudp.py), and you're right: if it hasn't received the connection signature yet, it doesn't use it for the packet signature calculation at all. I don't know how this happened, but I must have made a mistake while writing that part of the wiki page. I'll fix it. Thanks!
I also see why you said that you removed the first -6- bytes from the packet now. I completely forgot about the magic number 😅.

Now I have a problem with the Kerberos key.
Your wiki states you need a NEX password to generate a kerberos key but I don't get such a thing. The nasc just returns a locator, retry, returncd, token and datetime value. And if I read your code right you use the nintendo network account password to generate the key. But a 3DS doesn't have this because it doesn't use a nintendo network account. Giving the game nothing (buffer length 0) or just 00 bytes as kerberos data gives an error in game with states that it couldn't connect to the server.
And to just check if I understood the wiki page right
the whole kerberos Ticket (140 bytes length) consists of the secure key, server pid and ticket data, where the first 32 bytes are the secure key, the next 4 bytes the server pid and the rest the ticket info. The ticket info is practically not needed because the game can't read it anyway.

The key is not derived from the nintendo network account password but from the NEX password. NEX username/password are automatically generated by the server the first time you log in, and can not be changed manually. The Wii U account server returns your NEX username/password when you request a NEX token, but I have no idea how to obtain your NEX password from 3DS servers unfortunately.
The first 32 bytes of the kerberos ticket are indeed the secure key, the next 4 bytes the server pid, and the rest the ticket info. But the ticket is smaller than 140 bytes, normally, where did you get that number?

That is the LoginEx Response from the MPFF auth server from nintendo (already decrypted)
400100000A01060000000280000001000100806BEC078C000000F682DBE86C306DC68F5DF461E21CA86841D4560B18F1BE5552B81841AADB5B5C310918A046F9E4B29C2949748F4B38396A008032A0FF27AA0E7C770B493B8A30AC85980250DE2BE36BC75ACBF02C0A76A769EA2AE998F6FD1637A0378B696B2E043FB473A2A5455C9FBA8C5365F39C240B11508814416C0F78075D2C8F222216FF7882874DC82E7791A796BD015E0000004D007072756470733A2F616464726573733D33342E3230382E3136362E3230323B706F72743D34303436313B4349443D313B5049443D323B7369643D313B73747265616D3D31303B747970653D320000000000010000B378A08A1F00000039006272616E63683A6F726967696E2F72656C656173652F6E67732F332E392E782E32303078206275696C643A335F395F31375F323030355F3000
and if I read this right than this part must be the kerberos part
F682DBE86C306DC68F5DF461E21CA86841D4560B18F1BE5552B81841AADB5B5C310918A046F9E4B29C2949748F4B38396A008032A0FF27AA0E7C770B493B8A30AC85980250DE2BE36BC75ACBF02C0A76A769EA2AE998F6FD1637A0378B696B2E043FB473A2A5455C9FBA8C5365F39C240B11508814416C0F78075D2C8F222216FF7882874DC82E7791A796BD
with the 4 bytes in front of this (0x8C) the buffer length which is 140 in decimal

That's interesting. Looks like it uses the updated ticket data format I've only seen on Nintendo Switch servers so far. I guess that's because this game is quite new, as it was released only two years ago.
By the way, there's no problem with sharing your user id. It can be easily requested from the account server anyway (see /admin/mapped_ids). You should never tell anyone your NEX password though.
I'll tell you if I ever find out how you can retrieve your NEX password. I tried to reverse engineer the 3DS friends system module, but haven't found anything useful so far.

I guess I found something interesting that has something to do with the kerberos ticket
I looked in the RAM around my user_pid and found an interesting string. I froze the beginning of this string to zero and the game told me that it can't connect to the server after receiving the LoginEx Response from the official server. So I guess this string has something todo with decrypting this kerberos ticket.
I don't if I should post this string here because I don't know if this is some private password or some static value which is for everyone the same. This same string is also found in the friends module RAM at address 0x00198DA8.

Someone from Pretendo (a group that tries to write a replacement for nintendo network servers when they go down) made a 3DS homebrew app that dumps your NEX password. I haven't tested it myself, but if it works you can use this to get your NEX password.
https://cdn.discordapp.com/attachments/408723233030799362/495588775389560835/Get_PID_Passwrd.7z

Tested the homebrew and it works.
and the Password is the same that I found in the RAM and I could successfully decrypt the kerberos part and check the HMAC.
You probably already know that but this longer kerberos part is made of
32 bytes that represent the secure key
4 bytes integer (could be the server pid; 1 in my case)
4 bytes integer (are probably the remaining bytes of the kerberos part; 84 in may case)
4 bytes integer that is the length in bytes of the next part (16 in my case)
"the next part" (16 bytes long in my case)
4 bytes integer that is the length of the kerberos ticket which is probably always 60
60 bytes kerberos ticket
Strangely the the 3DS password doesn't look close to one of the Wii U. The Wii U password just contains lowercase and uppercase letters and numbers but the 3DS password also contains special characters like ? and =
and BTW thank you for your help

I don't know if this is a problem in the documentation or I'm reading this wrong
CE0000006D0112000000048000000100000011004D617463686D616B6553657373696F6E00A9000000A500000000270000009F420C00331D1C6C331D1C6C010004005F00000000000000300200000000000005004C756973000374000000020000000600000000000000030000000000000000000000000000000000000001010000000C000000310000C951A83C51281C000001000000640000000000000000001C00000002000000040040535200030105004047495200010300000000000000A884AE8A1F000000010000000000000000
This is a BrowserMatchmakeSession Response all fits well except for the very last value (https://github.com/Kinnay/NintendoClients/wiki/Match-Making-Types#matchmakesession-structure)
A884AE8A1F000000 startedTime
010000 UserPassword
00000000 ReferGid
00 userPWEnabled
00 systemPWEnabled
Is this normal that the last value is just missing or do I read it wrong

Nintendo often changes structures when they update the NEX library (especially the MatchmakeSession structure has received lots of changes), and some games even have their own patches. Without looking at the game's code, there's no easy way to determine the NEX version or the structure format. The "only present in" column on the wiki merely consists of guesses based on the games I've looked into so far. If I remember correctly, I extracted most match making types from Super Mario Odyssey, so it's possible that your game uses a slightly different structure format.
I don't think you read it wrong, it's probably an error in the documentation in this case.

Ok thank you. Now I know the last value does porbably not exist in my version

Sorry I'm misusing this issue to just that it seems that the documentation about the Acknowledgment packets in V0 is not correct. At least for the friend server the ack packet is build like that. The multi ack flag is never set instead its just a normal data packet with the ack flag (3DS always sends a 1 but the Nintendo server sends a 9 with a payload size of 0). The sequence id is not that of the sending machine but instead is the sequence id of the packet that gets acknowledged. And I'm just guessing that the packet signature tells the other end that when it's 0x12345678 this packet is an Acknowledgment.

The MPFF Server seems to use to only aggregate acknowledgment. Also it seem to have specify every packet that it acknowledges. If it has to acknowledge 2 packets at once it writes both sequence ids behind each other.
But I have another question. Do you know what Call ID (RMC Protocol) the server uses when he sends a request? This seems to be a special case because I only saw this with the InitiateProbe Request. Also this request gets only acknowledged by the client but it doesn't send an empty response unlike the server does for every request that has no actual response data. Maybe this has something todo with the Call ID or this is just normal behaivior by the client when it gets a response

The call id shouldn't matter, but this isn't normal behavior either. There's probably something wrong with your RMC request, but I can't tell you what's wrong without seeing the packet you sent to your 3DS.

This isn't from my server but from the official server. I try to understand how the UDP hole punching works. And after the Client sends a RequestProbeInitiationExt for every StationURL he gets from GetSessionURLs to the server the server sends to the other client (the Host) a InitiateProbe Request with Call IDs like 0x02C345A7 (frist Probe) or 0x02C345A9 (second Probe)

So I got it to work that 3 3DS could join a game in MP FF (4 should probably work too but couldn't test it). The Call ID in the InitiateProbe Request seems to be of no matter. I sent a static 50 in all my requests and the game didn't care. But what I found out is that the RVCID does matter. A 3DS refuses to connect to a device with the exact same RVCID.
BTW the access Key of Pokemon Ultra Sun is 086f9d28 and the Game Server ID is 00164800.

I looked a little bit at the packets of Pokemon and found some differences with your documentation. In the MatchmakingExtension(0x6D) Protocol it seems the packets after WithdrawMatchmakingAll are different than in your documentation. I looked at the code and I guess after WithdrawMatchmakingAll follows ClearMyPreviouslyMatchedUserCache instead of FindMatchmakeSessionByGatheringId. If you want to look at the code yourself I can give you the file and the offset where I found it.

My documentation is based on Super Mario Odyssey, but there are many different versions of the NEX library, and some games even have their own patches. The "common protocols" should be the same in all games and versions, but there may be some differences in the "nintendo only" protocols (for example, the MatchmakeExtension protocol).

Don't know if you want to add that to your documentation but Luigi's Mansion 2 seems to be a special canditate when it comes to packet signature calculation. First it doesn't use the the connection signature in the CONNECT packet and second instead of using the connection signature of the other end it uses it's own connection signature.
Also it seems that something is missing in the LoginEx Request. "Newer" (are games that were released 2013 really that new?) games that use the new Kerberos format also use 10 more bytes in the request. Between the Any Data Holder's second length and the token length there are 10 undocumented bytes. The first 6 bytes were always 0 in my received requests. The other 4 seem to be some u32. Luigi's Mansion 2 uses PRUDP1 but uses the old kerberos format. Kid Icarus Uprising uses PRUDP0. The other games in the table are using the PRUDP1 with the new format and these 10 extra bytes.
BTW if you want them to add it, here are the Access Keys of some games. I did verify them.
|Kid Icarus Uprising|00030000|58a7e494|
|Luigi's Mansion 2|00076400|3861a9f8|
|Pokémon X/Y|00055D01|876138df|
|TLoZ Tri Force Heroes|00176E00|c1621b84|

Thanks, I've added the games to the game server list page.
If you can figure out the exact meaning and format of those unknown fields I'll add them to the wiki too. I don't really want to add incomplete information to the PRUDP/NEX pages.


Code-server as chrome app not functional (keys or clicks not working)

OS/Web Information

Web Browser: Chrome
Local OS: Mac OS
Remote OS: Debian
Remote Architecture: x86
code-server --version: 3.9.1 e0203f2a36c9b7036fefa50eec6cf8fa36c5c015

Steps to Reproduce

Run Code-Server
Install Chrome App by clicking 3 dots -> more tools -> create shortcut, or the install icon if it's available to you
The new code-server app is read only

Expected
I should be able to interact with the code server app as if it was in a browser tab.
Actual
The cursor is blinking, but nothing else works. No keys or clicks are responding.
Screenshot

Notes
This issue can be reproduced in VS Code: No

Can you please try upgrading to the latest version of code-server?
I tried to reproduce locally (macOS + Chrome) but didn't have any luck.
Screenshot
https://user-images.githubusercontent.com/3806031/120369498-339b5e00-c2c8-11eb-9412-267b596679b1.mov


JAR that contains grpc in its named shown as vulnerable

Is this a request for help?:
no
Is this a BUG REPORT or a FEATURE REQUEST? (choose one):
BUG REPORT

Version of Anchore Engine and Anchore CLI if applicable:
https://ci-tools.anchore.io/inline_scan-v0.6.0
What happened:
I have a Java project that contains the Google Cloud Trace jar. When Anchore scans the JARs inside the fat-jar it reports a vulnerability related to "pure" grpc.
vulnerabilities        package        CRITICAL Vulnerability found in non-os package type (java) - /app/businessmanager.jar:BOOT-INF/lib/grpc-google-cloud-trace-v2-1.0.1.jar (CVE-2017-7860 - https://nvd.nist.gov/vuln/detail/CVE-2017-7860)        stop
vulnerabilities        package        CRITICAL Vulnerability found in non-os package type (java) - /app/businessmanager.jar:BOOT-INF/lib/grpc-google-cloud-trace-v2-1.0.1.jar (CVE-2017-7861 - https://nvd.nist.gov/vuln/detail/CVE-2017-7861)        stop
vulnerabilities        package        CRITICAL Vulnerability found in non-os package type (java) - /app/businessmanager.jar:BOOT-INF/lib/grpc-google-cloud-trace-v2-1.0.1.jar (CVE-2017-8359 - https://nvd.nist.gov/vuln/detail/CVE-2017-8359)        stop
vulnerabilities        package        CRITICAL Vulnerability found in non-os package type (java) - /app/businessmanager.jar:BOOT-INF/lib/grpc-google-cloud-trace-v2-1.0.1.jar (CVE-2017-9431 - https://nvd.nist.gov/vuln/detail/CVE-2017-9431)        stop

What did you expect to happen:
The engine shouldn't link the old issue, related to a different jar, to the one that's in my project.

Thanks for filing this issue. Please show the packages output or vulnerability listing output for this image so I can better the specific package name being detected.

Hey @zhill
the complete output I get is:
Gate                   Trigger        Detail                                                                                Status
vulnerabilities        package        MEDIUM Vulnerability found in non-os package type (java) - /app/businessmanager.jar:BOOT-INF/lib/okhttp-3.12.0.jar (CVE-2018-20200 - https://nvd.nist.gov/vuln/detail/CVE-2018-20200)                           warn
vulnerabilities        package        CRITICAL Vulnerability found in non-os package type (java) - /app/businessmanager.jar:BOOT-INF/lib/grpc-google-cloud-trace-v2-1.0.1.jar (CVE-2017-7860 - https://nvd.nist.gov/vuln/detail/CVE-2017-7860)        warn
vulnerabilities        package        CRITICAL Vulnerability found in non-os package type (java) - /app/businessmanager.jar:BOOT-INF/lib/grpc-google-cloud-trace-v2-1.0.1.jar (CVE-2017-7861 - https://nvd.nist.gov/vuln/detail/CVE-2017-7861)        warn
vulnerabilities        package        CRITICAL Vulnerability found in non-os package type (java) - /app/businessmanager.jar:BOOT-INF/lib/grpc-google-cloud-trace-v2-1.0.1.jar (CVE-2017-8359 - https://nvd.nist.gov/vuln/detail/CVE-2017-8359)        warn
vulnerabilities        package        CRITICAL Vulnerability found in non-os package type (java) - /app/businessmanager.jar:BOOT-INF/lib/grpc-google-cloud-trace-v2-1.0.1.jar (CVE-2017-9431 - https://nvd.nist.gov/vuln/detail/CVE-2017-9431)        warn
vulnerabilities        package        MEDIUM Vulnerability found in os package type (dpkg) - apt (CVE-2020-3810 - https://security-tracker.debian.org/tracker/CVE-2020-3810)                                                                          warn
vulnerabilities        package        MEDIUM Vulnerability found in os package type (dpkg) - libapt-pkg5.0 (CVE-2020-3810 - https://security-tracker.debian.org/tracker/CVE-2020-3810)                                                                warn
vulnerabilities        package        CRITICAL Vulnerability found in non-os package type (java) - /app/businessmanager.jar:BOOT-INF/lib/grpc-google-cloud-trace-v2-1.0.1.jar (CVE-2017-7860 - https://nvd.nist.gov/vuln/detail/CVE-2017-7860)        stop
vulnerabilities        package        CRITICAL Vulnerability found in non-os package type (java) - /app/businessmanager.jar:BOOT-INF/lib/grpc-google-cloud-trace-v2-1.0.1.jar (CVE-2017-7861 - https://nvd.nist.gov/vuln/detail/CVE-2017-7861)        stop
vulnerabilities        package        CRITICAL Vulnerability found in non-os package type (java) - /app/businessmanager.jar:BOOT-INF/lib/grpc-google-cloud-trace-v2-1.0.1.jar (CVE-2017-8359 - https://nvd.nist.gov/vuln/detail/CVE-2017-8359)        stop
vulnerabilities        package        CRITICAL Vulnerability found in non-os package type (java) - /app/businessmanager.jar:BOOT-INF/lib/grpc-google-cloud-trace-v2-1.0.1.jar (CVE-2017-9431 - https://nvd.nist.gov/vuln/detail/CVE-2017-9431)        stop

We're looking at how to improve vuln matches on NVD specifically so I'll take this into account. This looks like a miss on the fuzzy part of the vendor, product, version prediction that we use to try to match java with CPEs in the NVD data.
For a workaround you can whitelist the match in a custom policy json bundle.


Sync CompareInfo.cs with dotnet/runtime

#7913 brought in an odd change.

Thanks! I got this mixed up with the globalization fixes.


Tests: Prepare reference scores once for all env. configs

🚀 Feature
If we use the reference metric only for generating static results, these results will be loaded and used in all testing environments?
The proposed flow is semi-online; still, the references are generated during CI test time but only once and shared to remaining tests... so consider using some JSON or YAML universal format.
TIP1: The references can be shared among jobs as artifact:rabbit:
TIP2: To speed up we can use some caching with limited time as we do for pip/package caching
For example, the case with all Condas:

create all testing references with frozen env that is supported by all reference metrics
run full matric of all configurations PT v{1.4, ...1.12}

Motivation
this may help to unblock various configurations and also:

guarantee that the results are identical regardless of env. and if needed, they can be inspected...
testing could be faster as references are not computed for each config...
we have not locked with testing for env that the reference packages support...

Pitch
Make tests more predictable and scalable.
Alternatives
Still generating on-the-fly results, BUT let's discuss this option a bit... :)
Additional context
part of #678
pls comment @PyTorchLightning/core-metrics

I support this option in general, I just request that this is implemented with proper caching to ensure that the reference creation is fast most of the time and frequent cache-invalidation that we are always up to date.
@Borda knows all this already since we discussed offline, but mentioning it here for completeness :)


Feature Request - Option to save git credentials

Rancher versions:
rancher/server: 1.6.2
rancher/agent: 1.2.2
Problem:
At the moment, there is no option to save Git credentials (Password or SSH Key) somewhere in Rancher. This is particularly useful while adding a custom Rancher Catalog.
Workaround:
Very insecure. Save credentials in the Git URL of Rancher Catalog
<EMAIL_ADDRESS>

Duplicate of #3248 / #7638