API "describe Gateway" does not show correct HA and connections status

For example, when using libreswan cable-driver, which has failed to establish connections,
the HA status was shown as "passive", while the actual problem was in connections status.
In addition, Status showed "Failure:  ", but without specifying "Status Message:  ...".
$ oc describe Gateway -n submariner-operator

11:25:28 Name:         default-cl1-9vgwj-worker-bq8rt
11:25:28 Namespace:    submariner-operator
11:25:28 Labels:       <none>
11:25:28 Annotations:  update-timestamp:<PHONE_NUMBER>
11:25:28 API Version:  submariner.io/v1
11:25:28 Kind:         Gateway
11:25:28 Metadata:
11:25:28   Creation Timestamp:  2020-06-15T08:23:44Z
11:25:28   Generation:          1
11:25:28   Resource Version:    2834031
11:25:28   Self Link:           /apis/submariner.io/v1/namespaces/submariner-operator/gateways/default-cl1-9vgwj-worker-bq8rt
11:25:28   UID:                 c5a419dd-ab09-44fb-9665-689ad0aca164
11:25:28 Status:
11:25:28   Connections:
11:25:28   Ha Status:  passive
11:25:28   Local Endpoint:
11:25:28     Backend:      libreswan
11:25:28     cable_name:   submariner-cable-nmanos-cluster-b-10-166-0-26
11:25:28     cluster_id:   nmanos-cluster-b
11:25:28     Hostname:     default-cl1-9vgwj-worker-bq8rt
11:25:28     nat_enabled:  true
11:25:28     private_ip:   <IP_ADDRESS>
11:25:28     public_ip:    <IP_ADDRESS>
11:25:28     Subnets:
11:25:28       <IP_ADDRESS>/19
11:25:28   Status Failure:
11:25:28   Version:         v0.4.0-rc2
11:25:28 Events:            <none>

What you expected to happen:
oc describe Gateway should show correct HA status and connections status, including correct message on each component failure, if any.
Environment:

Submariner version: 0.4.0 RC2
Kubernetes version: OCP 4.4.6
Cloud provider or hardware configuration: OSP & AWS
OS (e.g: cat /etc/os-release):

Actually, the Status Failure:   is also wrong, as it always being printed, even when HA status is active, and all connections are working.
Do you want a new bug for it ?

Actually, the Status Failure:  is also wrong, as it always being printed, even when HA status is active, and all connections are working.
Do you want a new bug for it ?

Well the fact that it's always printed is not really wrong - Status Failure is a field and may have an empty value.

The Ha Status  field is only set to Passive if engine.driver was not initialized which means it wasn't elected the leader, at least yet. I assume you only have one gateway node labelled. I suspect it was initially set to Passive on startup and then gained leadership but never updated the Gateway object thereafter. There's several code paths that could lead to this scenario - all but one are due to API server failure. The other is due to failure from engine.ListCableConnections where we intend to set gateway.Status.StatusFailure but instead return nil thus eliding the update - this is a bug. I suspect this is the case you're hitting, ie the libreswan cable driver is returning an error. It would be helpful to see the logs to verify. In any case, I'll submit a PR to fix this.

@nyechiel @mangelajo I'm reopening bug:
What should I expect to see now in case of a connections error ?
Can you replace the "Status Failure:" field with just "Status:" + error message if there's one (like in following example) ?
See output of subctl latest (build devel-1-g6dc3413) with libreswan on OSP-AWS env:
14:23:03 (conda-test) subctl join --kubecontext nmanos-cluster-b --kubeconfig /mnt/skynet-data/ocp-install/merged_kubeconfig   --clusterid nmanos-cluster-b ./broker-info.subm --cable-driver libreswan   --ikeport 501 --nattport 4501 --subm-debug
14:23:03 * ./broker-info.subm says broker is at: https://api.nmanos-cluster-a.devcluster.openshift.com:6443
14:23:04 * There are 1 labeled nodes in the cluster:
14:23:04   - default-cl1-9vgwj-worker-bq8rt
14:23:04  ��� Discovering network details  ...
14:23:04     Discovered network details:
14:23:04         Network plugin:  OpenShift
14:23:04         Service CIDRs: [<IP_ADDRESS>/16]
14:23:04         Cluster CIDRs: [<IP_ADDRESS>/14]
14:23:04  ��� Discovering network details
14:23:04  ��� Discovering multi cluster details  ...
14:23:04     Discovered global network details for Cluster nmanos-cluster-a:
14:23:04         ServiceCidrs: [<IP_ADDRESS>/16]
14:23:04         ClusterCidrs: [<IP_ADDRESS>/14]
14:23:04         Global CIDRs: [<IP_ADDRESS>/19]
14:23:04  ��� Validating Globalnet configurations  ...
14:23:04  ��� Validating Globalnet configurations
14:23:04  ��� Assigning Globalnet IPs  ...
14:23:04  ��� Assigning Globalnet IPs
14:23:04  ��� Allocated GlobalCIDR: <IP_ADDRESS>/19
14:23:04  ��� Discovering multi cluster details
14:23:04  ��� Deploying the Submariner operator  ...
14:23:19  ��� Deploying the Submariner operator
14:23:19  ��� Created operator CRDs
14:23:19  ��� Created operator namespace: submariner-operator
14:23:19  ��� Created operator service account and role
14:23:19  ��� Created lighthouse CRDs
14:23:19  ��� Created lighthouse service account and role
14:23:19  ��� Created Lighthouse service accounts and roles
14:23:19  ��� Deployed the operator successfully
14:23:19  ��� Deploying multi cluster service discovery  ...
14:23:19  ��� Deploying multi cluster service discovery
14:23:19  ��� Updated existing Lighthouse entry in the openshift-dns role
14:23:19  ��� Creating SA for cluster  ...
14:23:20  ��� Creating SA for cluster
14:23:20  ��� Deploying Submariner  ...
14:23:20  ��� Deploying Submariner
14:23:20  ��� Submariner is up and running
14:23:20

4 Minutes later, the Gateway shows "Status: connecting" (due to libreswan non-default ports bug),
And "Status Failure:" shows empty message, while there should have been indication of the problem already.
14:27:58 (conda-test) /mnt/skynet-data/ocp-install/GOBIN/oc describe Gateway -n submariner-operator
14:27:58 Name:         default-cl1-9vgwj-worker-bq8rt
14:27:58 Namespace:    submariner-operator
14:27:58 Labels:       <none>
14:27:58 Annotations:  update-timestamp:<PHONE_NUMBER>
14:27:58 API Version:  submariner.io/v1
14:27:58 Kind:         Gateway
14:27:58 Metadata:
14:27:58   Creation Timestamp:  2020-06-29T11:23:30Z
14:27:58   Generation:          2
14:27:58   Resource Version:    25777886
14:27:58   Self Link:           /apis/submariner.io/v1/namespaces/submariner-operator/gateways/default-cl1-9vgwj-worker-bq8rt
14:27:58   UID:                 415214e5-5a53-4734-994b-1d11c13e3d9f
14:27:58 Status:
14:27:58   Connections:
14:27:58     Endpoint:
14:27:58       Backend:      libreswan
14:27:58       cable_name:   submariner-cable-nmanos-cluster-a-10-166-84-92
14:27:58       cluster_id:   nmanos-cluster-a
14:27:58       Hostname:     ip-10-166-84-92
14:27:58       nat_enabled:  true
14:27:58       private_ip:   <IP_ADDRESS>
14:27:58       public_ip:    <IP_ADDRESS>
14:27:58       Subnets:
14:27:58         <IP_ADDRESS>/19
14:27:58     Status:          connecting
14:27:58     Status Message:
14:27:58   Ha Status:         active
14:27:58   Local Endpoint:
14:27:58     Backend:      libreswan
14:27:58     cable_name:   submariner-cable-nmanos-cluster-b-10-166-0-26
14:27:58     cluster_id:   nmanos-cluster-b
14:27:58     Hostname:     default-cl1-9vgwj-worker-bq8rt
14:27:58     nat_enabled:  true
14:27:58     private_ip:   <IP_ADDRESS>
14:27:58     public_ip:    <IP_ADDRESS>
14:27:58     Subnets:
14:27:58       <IP_ADDRESS>/19
14:27:58   Status Failure:
14:27:58   Version:         v0.4.1
14:27:58 Events:            <none>
14:27:58
14:27:58 (conda-test) submariner_status=UP
14:27:58 (conda-test) subctl info
14:27:59     Discovered network details:
14:27:59         Network plugin:  OpenShift
14:27:59         Service CIDRs: [<IP_ADDRESS>/16]
14:27:59         Cluster CIDRs: [<IP_ADDRESS>/14]
14:27:59 (conda-test) /mnt/skynet-data/ocp-install/GOBIN/oc describe cm -n openshift-dns
14:27:59 Name:         dns-default
14:27:59 Namespace:    openshift-dns
14:27:59 Labels:       dns.operator.openshift.io/owning-dns=default
14:27:59 Annotations:  <none>
14:27:59
14:27:59 Data
14:27:59 ====
14:27:59 Corefile:
14:27:59 ----
14:27:59 # lighthouse
14:27:59 supercluster.local:5353 {
14:27:59     forward . <IP_ADDRESS>
14:27:59 }
14:27:59 .:5353 {
14:27:59     errors
14:27:59     health
14:27:59     kubernetes cluster.local in-addr.arpa ip6.arpa {
14:27:59         pods insecure
14:27:59         upstream
14:27:59         fallthrough in-addr.arpa ip6.arpa
14:27:59     }
14:27:59     prometheus :9153
14:27:59     forward . /etc/resolv.conf {
14:27:59         policy sequential
14:27:59     }
14:27:59     cache 30
14:27:59     reload
14:27:59 }
14:27:59
14:27:59 Events:  <none>
14:28:00 (conda-test) /mnt/skynet-data/ocp-install/GOBIN/oc get pods -n submariner-operator --show-labels
14:28:00 NAME                                             READY   STATUS    RESTARTS   AGE     LABELS
14:28:00 submariner-gateway-qrcsm                         1/1     Running   0          4m27s   app=submariner-engine,controller-revision-hash=5c6b846694,pod-template-generation=1
14:28:00 submariner-globalnet-v45m5                       1/1     Running   0          4m25s   app=submariner-globalnet,component=globalnet,controller-revision-hash=6bcbdc4876,pod-template-generation=1
14:28:00 submariner-lighthouse-agent-75dd874f44-nbgpd     1/1     Running   0          4m25s   app=submariner-lighthouse-agent,component=submariner-lighthouse,pod-template-hash=75dd874f44
14:28:00 submariner-lighthouse-coredns-6897c85cc4-86jmx   1/1     Running   0          4m23s   app=submariner-lighthouse-coredns,component=submariner-lighthouse,pod-template-hash=6897c85cc4
14:28:00 submariner-lighthouse-coredns-6897c85cc4-sdgfr   1/1     Running   0          4m23s   app=submariner-lighthouse-coredns,component=submariner-lighthouse,pod-template-hash=6897c85cc4
14:28:00 submariner-operator-5d88f98b4c-rh4tx             1/1     Running   0          4m56s   name=submariner-operator,pod-template-hash=5d88f98b4c
14:28:00 submariner-routeagent-24dw8                      1/1     Running   0          4m25s   app=submariner-routeagent,component=routeagent,controller-revision-hash=5c5f78b485,pod-template-generation=1
14:28:00 (conda-test) /mnt/skynet-data/ocp-install/GOBIN/oc get clusters -n submariner-operator -o wide
14:28:01 NAME               AGE
14:28:01 nmanos-cluster-a   2m28s
14:28:01 nmanos-cluster-b   2m28s
14:28:01 (conda-test) /mnt/skynet-data/ocp-install/GOBIN/oc describe cluster "nmanos-cluster-b" -n submariner-operator
14:28:02 Name:         nmanos-cluster-b
14:28:02 Namespace:    submariner-operator
14:28:02 Labels:       <none>
14:28:02 Annotations:  <none>
14:28:02 API Version:  submariner.io/v1
14:28:02 Kind:         Cluster
14:28:02 Metadata:
14:28:02   Creation Timestamp:  2020-06-29T11:23:30Z
14:28:02   Generation:          1
14:28:02   Resource Version:    25776500
14:28:02   Self Link:           /apis/submariner.io/v1/namespaces/submariner-operator/clusters/nmanos-cluster-b
14:28:02   UID:                 5eea7d07-551e-4680-abd0-3852885b5f5e
14:28:02 Spec:
14:28:02   cluster_cidr:
14:28:02     <IP_ADDRESS>/14
14:28:02   cluster_id:  nmanos-cluster-b
14:28:02   color_codes:
14:28:02     blue
14:28:02   global_cidr:
14:28:02     <IP_ADDRESS>/19
14:28:02   service_cidr:
14:28:02     <IP_ADDRESS>/16
14:28:02 Events:  <none>

@nyechiel @mangelajo I'm reopening bug:

This appears to be a different issue, ie the libreswan driver not reporting a specific connection failure - in the original issue, it wasn't even reporting the connection at all due to the bug I outlined.  It seems you're reporting that the specific connection should have reported its Status  is failed with a specific Status Message. If so, can you please open a new issue?
The Status Failure (currently) reports an overall failure, specifically if 'ListConnections' fails. In this case, 'ListConnections' itself succeeded hence no Status Failure.  It doesn't provide a roll-up of the specific connection statuses (there could be multiple connections with some successful, some connecting, some failed etc). This would be s separate issue.

Opened new issue, and closing this one.


fill in additional text with autocomplete

IF the autocomplete is A,B,C
how can i fill in D? i just want to use autocomplete as an aid, not as a limit.
besides, the arrow keys isn't quite like those in  EXCEL, which can change focus during input. you can only change focus when you are not inputting. is it possible to improve it ?
thx!

Sorry, I am not sure if I understood your question. But, it is not possible to move from a cell during edition with arrows. Because the arrows will be used to navigate in the editor.


UI-Kit 2.0

Explore/dev/research UI-Kit version 2

@dominikwilkowski Can I close this — is idea here held in other tix/trello cards?

Ideas are in my head :)


[Python] How to avoid overflow error.

I know that the argument safe=False in pa.Table.from_pandas ignores overflow errors, but it does not ignore overflow in list or in struct.
It works.
import pyarrow as pa
import pyarrow.parquet as pq
import pandas as pd
import json

test_json = [
    {
        "name": "taro",
        "id":<PHONE_NUMBER>,
        "points": [2, 2, 2],
        "groups": {
            "group_name": "baseball",
            "group_id": 1234
        }
    },
    {
        "name": "taro",
        "id": 1234,
    }
]

schema = pa.schema([
    pa.field('name', pa.string()),
    pa.field('id', pa.int32()),
    pa.field("points", pa.list_(pa.int32())),
    pa.field('groups', pa.struct([
        pa.field("group_name", pa.string()),
        pa.field("group_id", pa.int32()),
    ])),
])

writer = pq.ParquetWriter('test_schema.parquet', schema=schema)
df = pd.DataFrame(test_json)
table = pa.Table.from_pandas(df, schema=schema, safe=False)
writer.write_table(table)
writer.close()

table = pq.read_table("test_schema.parquet")
print(table)

name: [["taro","taro"]]
id: [[-1248285164,1234]]
points: [[[2,2,2],null]]
groups: [ – is_valid: [
true,
false
] – child 0 type: string
[
"baseball",
null
] – child 1 type: int32
[
1234,
null
]]

However, the following two do not work.
test_json = [
    {
        "name": "taro",
        "id": 2,
        "points": [2,<PHONE_NUMBER>, 2],
        "groups": {
            "group_name": "baseball",
            "group_id": 1234
        }
    },
    {
        "name": "taro",
        "id": 1234,
    }
]

Traceback (most recent call last):
File "test_pyarrow.py", line 35, in <module>
table = pa.Table.from_pandas(df, schema=schema, safe=False)
File "pyarrow/table.pxi", line 1782, in pyarrow.lib.Table.from_pandas
File "/home/s0108403058/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 594, in dataframe_to_arrays
arrays = [convert_column(c, f)
File "/home/s0108403058/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 594, in <listcomp>
arrays = [convert_column(c, f)
File "/home/s0108403058/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 581, in convert_column
raise e
File "/home/s0108403058/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 575, in convert_column
result = pa.array(col, type=type_, from_pandas=True, safe=safe)
File "pyarrow/array.pxi", line 312, in pyarrow.lib.array
File "pyarrow/array.pxi", line 83, in pyarrow.lib._ndarray_to_array
File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: ('Value<PHONE_NUMBER> too large to fit in C integer type', 'Conversion failed for column points with type object')

test_json = [
    {
        "name": "taro",
        "id": 2,
        "points": [2, 2, 2],
        "groups": {
            "group_name": "baseball",
            "group_id":<PHONE_NUMBER>
        }
    },
    {
        "name": "taro",
        "id": 1234,
    }
]

Traceback (most recent call last):
File "test_pyarrow.py", line 35, in <module>
table = pa.Table.from_pandas(df, schema=schema, safe=False)
File "pyarrow/table.pxi", line 1782, in pyarrow.lib.Table.from_pandas
File "/home/s0108403058/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 594, in dataframe_to_arrays
arrays = [convert_column(c, f)
File "/home/s0108403058/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 594, in <listcomp>
arrays = [convert_column(c, f)
File "/home/s0108403058/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 581, in convert_column
raise e
File "/home/s0108403058/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 575, in convert_column
result = pa.array(col, type=type_, from_pandas=True, safe=safe)
File "pyarrow/array.pxi", line 312, in pyarrow.lib.array
File "pyarrow/array.pxi", line 83, in pyarrow.lib._ndarray_to_array
File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: ('Value<PHONE_NUMBER> too large to fit in C integer type', 'Conversion failed for column groups with type object')

How can I resolve this error?
pyarrow==7.0.0

@kato1208 The preferred way to solve overflow is to change your schema to use types that are appropriate for the data. Look at your first example: The "id" value starts as<PHONE_NUMBER>, but when converted to Arrow (with safe=False) comes out as -1248285164. I doubt that's what you actually want.
That id value is just too large to fit into an int32. If you change the schema to use int64 you can run this without any overflow errors and without needing safe=False. And it will give you correct data.
schema = pa.schema([
    pa.field('name', pa.string()),
    pa.field('id', pa.int64()),
    pa.field("points", pa.list_(pa.int32())),
    pa.field('groups', pa.struct([
        pa.field("group_name", pa.string()),
        pa.field("group_id", pa.int64()),
    ])),
])

@wjones127
Thank you for your comment !
Unfortunately there are reasons why the schema cannot be changed and we would like to allow for overflow error.
How can I ignore overflow error?
We apologize for the inconvenience and thank you in advance for your time.

How can I ignore overflow error?

Well it sounds like there is an issue where safe isn't being pushed down into nested arrays, so I don't think this is currently possible. You are welcome to file an issue on the Apache Jira to implement that pushdown.

Indeed it seems the safe keyword is ignored if we are using the python_to_arrow.cc code path (in contrast to the numpy_to_arrow.cc code path). This is similar as a recent issue that was opened with a variant on this (using list arrays): https://issues.apache.org/jira/browse/ARROW-16491
A simplified example (even without using a nested type):
# input is a python list -> safe keyword doesn't work
>>> pa.array([3046682132], pa.int32(), safe=False)
...
ArrowInvalid: Value<PHONE_NUMBER> too large to fit in C integer type

# input is a numpy array -> safe=False has effect
>>> pa.array(np.array([3046682132]), pa.int32(), safe=False)
<pyarrow.lib.Int32Array object at 0x7f40e939f040>
[
  -1248285164
]

So going to close this, and add the example at https://issues.apache.org/jira/browse/ARROW-16491

We created an "umbrella" issue that includes all the issues opened and connected to the Python to array conversion following the codepath in python_to_arrow.cc. See: https://github.com/apache/arrow/issues/36202


Update DeepTTC definition file.

          Issue occurs with pandas 2.0. Added requirement.txt file that specifies acceptable pandas version

Originally posted by @AlexandrNP in https://github.com/JDACS4C-IMPROVE/DeepTTC/issues/7#issuecomment-1716146723

depends on https://github.com/JDACS4C-IMPROVE/DeepTTC/issues/7#issuecomment-1716146723

Fixed


Improve API

Should be better defined:

Default main
others?

Default main is taken care of in b27c1ec


forgot to actually install the export formatter last week

@spra85 @kand Forgot to actually install this properly in the editor in my last pull request.

:+1:


Add some way to display isocontours on an image in the plot

This has been requested.

I vote for this feature. Contour lines are very useful, for example, when dealing with Resonant Inelastic X-ray Scattering (RIXS) planes or raytracing simulations. Below a RIXS plane example with  matplotlib.pyplot.imshow  (left) and adding matplotlib.pyplot.contour on top (right).

I am aware of silx.examples.findContours, but it would be very nice having something like a wrapper to matplotlib.pyplot.contour accessible in PlotWidget.

How do you use matplotlib.pyplot.contour? You only uses it as a layer? Do you custom things?
I suppose using the colormap state (min/max linear/log) and displaying around 10/20 contours is enough to provide a good result for a first step. And easy to do.
Or you expect a user interface with a slider to pick a value for the isocontour. Or custom few things.

How do you use matplotlib.pyplot.contour? You only uses it as a layer? Do you custom things?

Yes, I use contour as an additional layer to the image to permit the user better visualizing the (weak) features in the image. Nothing more.

I suppose using the colormap state (min/max linear/log) and displaying around 10/20 contours is enough to provide a good result for a first step. And easy to do.

Yes, the user should be able to choose how many contours to plot and possibly the color and the line widths.

Or you expect a user interface with a slider to pick a value for the isocontour. Or custom few things.

No interface is needed. I would propose to simply extend the API of PlotWidget to let the user decide if to show the contour on top of the image or not (by default), for example, this way:
PlotWidget.addImage(... , contour=True, contour_levels=20, contour_colors='black', contour_linewidths=0.5)
Here a complete minimal example showing the behaviour I have in mind:
import numpy as np
from scipy.stats import multivariate_normal
import matplotlib.pyplot as plt
from matplotlib import cm
#
contour_levels = 20
contour_colors = 'gray'
contour_linewidths = 0.5
#
#dummy data
x = np.arange(-1, 1, 0.01)
y = np.arange(-1, 1, 0.01)
xx, yy = np.meshgrid(x, y)
zz0 = multivariate_normal.pdf(np.dstack((xx, yy)), mean=[0.5, -0.2], cov=[0.1, 0.05])
zz1 = multivariate_normal.pdf(np.dstack((xx, yy)), mean=[-0.25, 0.35], cov=[0.25, 0.35])
zz = zz0+zz1
#normalize colors from min to max
zzmin, zzmax = np.min(zz), np.max(zz)
norm = cm.colors.Normalize(vmin=zzmin, vmax=zzmax)
extent = (x.min(), x.max(), y.min(), y.max())
levels = np.linspace(zzmin, zzmax, contour_levels)
#
plt.close('all')
fig, axs = plt.subplots(1,2)
#no contours
axs[0].imshow(zz, origin='lower', extent=extent, cmap=cm.Reds, norm=norm)
#with contours
axs[1].imshow(zz, origin='lower', extent=extent, cmap=cm.Reds, norm=norm)
axs[1].contour(x, y, zz, levels, colors=contour_colors, linewidths=contour_linewidths)
#
plt.subplots_adjust()
plt.show()

As far as we do not end up with something like this ...

To me it was extending the plot item API (either by extending the image item or adding a new one) to enable configuring the contours and a small add-on in the PlotWidget to initialize it.
I would not go into configuration widgets.

@t20100 I vote for the extension of the plot API to enable (+ minimum configuration) the overlay of the contour lines. The default behaviour will still be without contour, as it is now.
@vasole your example goes beyond just simply having the possibility to overlay the contour lines. How to find the contours in noisy data should be probably tackled with a specific widget, as is in silx.examples.findContours.


Docs: use Promise.fromCallback instead of Promise.defer().callback

The docs make use of defer.callback, but from running the code and looking at the source code for Promise.defer it is apparent that Promise.defer().callback is undefined. Instead, use Promise.fromCallback().

Thanks for the doc fix!


Node 10 support on linux

I understand that the main purpose of this package is to bring support for macOS Mojave but I would like to ask you if you could consider depending on https://github.com/abandonware/noble which fixes Node 10 support for linux. This is the root issue https://github.com/noble/node-bluetooth-hci-socket/issues/84.
This would enable people who have to support both macOS Mojave and Node >=10 on Linux to use this library.

@geovie looks like abandonware/noble is interested in integrating. Please see https://github.com/abandonware/noble/issues/4.
That fork is pretty popular with 1600 weekly downloads.
https://www.npmjs.com/package/@abandonware/noble
Compare that to the original package with has 1800 weekly downloads.
https://www.npmjs.com/package/noble

Are there any patches from this fork that are pending upstream ?
If yes they could be merged into @abandonware/noble
Again efforts to track changes are appreciated:
https://github.com/abandonware/noble/issues/2

@rzr No @abandonware/noble already contains the latest changes https://github.com/abandonware/noble/pull/8

Feel free to rebase on @abandonware/noble if you want, I plan to keep it based on upstream too


Expose ModuleType to Singletons

In order to implement some build statistics in Soong, expose the module
type (the string used to define the module) to singletons.

LGTM


Resolve

Based on the fix though, I checked a few packages that may have the same issue with aiohttp and came up with a list

azure-communication-chat
azure-communication-identity
azure-communication-networktraversal
azure-communication-phonenumbers
azure-communication-sms

EDIT: they are fine.
So after this PR I'm going to check them for the same error.
@tjprescott here ya go!

Thank you @scbedd! You are my hero!


Unable to use Dingo Pipe with 1 detector networks due to bad size of nflows training array

When doing inference with a 1 detector network, one needs to train an unconditional density estimator on the set of 1D GNPE samples. However, for some reason nflows does not like this as it thinks the training vector is the wrong shape. This occurs during the sampling step of dingo_pipe. The error message is the following:
12:53 dingo_pipe INFO    : Command line arguments: Namespace(ini='outdir_GW190620_030421/GW190620_030421_config_complete.ini', verbose=False, calibration_model=None, spline_calibration_envelope_dict=None, spline_calibration_nodes=10, spline_calibration_curves=1000, idx=0, data_dump_file=None, event_data_files=['outdir_GW190620_030421/data/GW190620_030421_data0_1245035079-3_generation_event_data.hdf5'], proposal_samples_file=None, importance_sampling_generation=False, ignore_gwpy_data_quality_check=True, shift_segment_for_psd_generation_if_nan=True, timeslide_file=None, trigger_time='1245035079.3', data_dict=None, data_format=None, allow_tape=True, channel_dict='{H1:GWOSC, L1:GWOSC}', detectors=['L1'], duration=8.0, psd_dict=None, psd_fractional_overlap=0.0, post_trigger_duration=2.0, sampling_frequency=4096.0, psd_length=128, psd_maximum_duration=1024, psd_method='median', psd_start_time=None, maximum_frequency='1024.0', minimum_frequency='20.0', tukey_roll_off=0.4, resampling_method='lal', injection_dict=None, injection_file=None, injection_random_seed=None, asd_dataset=None, use_psd_of_trigger=True, injection_waveform_approximant=None, zero_noise=False, num_noise_realizations=100, accounting='dingo', accounting_user=None, label='GW190620_030421_data0_1245035079-3_sampling', local=False, local_generation=True, local_plot=False, outdir='/work/nihargupte/analysis/dingo/events/O3/GW190620_030421/eccentric_hm_v2/outdir_GW190620_030421', overwrite_outdir=False, request_disk=0.5, request_memory=120.0, request_memory_generation=8.0, request_cpus=1, request_cpus_importance_sampling=32, sampling_requirements=['TARGET.CUDAGlobalMemoryMb>20000'], extra_lines=['None'], simple_submission=False, conda_env=None, scheduler='condor', scheduler_args=None, scheduler_module=[None], scheduler_env=None, scheduler_analysis_time='7-00:00:00', submit=False, condor_job_priority=0, transfer_files=True, disable_hdf5_locking=True, log_directory=None, osg=False, desired_sites=None, analysis_executable=None, analysis_executable_parser=None, plot_data=False, plot_spectrogram=False, plot_corner=True, plot_weights=True, plot_log_probs=True, create_summary=False, email=None, notification='Never', queue=None, existing_dir=None, webdir=None, summarypages_arguments=None, result_format='hdf5', final_result=True, final_result_nsamples=20000, prior_dict=None, reference_frequency=None, waveform_approximant='SEOBNRv4EHM_opt', model='/data/nihargupte/projects/dingo-devel/tutorials/09_eccentricity/train_dirs/production_train_dirs/train_dir_SEOBNRv4EHM_5/model_latest.pt', model_init='/data/nihargupte/projects/dingo-devel/tutorials/09_eccentricity/train_dirs/train_dir_settings_SEOBNRv4EHM_opt_O3_L1_time/model_latest.pt', recover_log_prob=True, device='cuda', num_gnpe_iterations=30, num_samples=500000, batch_size=50000, density_recovery_settings='ProxyRecoveryDefault', importance_sample=True, importance_sampling_settings='Default', importance_sampling_updates='{}', n_parallel=20)
12:53 dingo_pipe INFO    : Unknown command line arguments: []
12:53 dingo_pipe INFO    : GNPE network does not provide log probability. Generating samples and training a new network to recover it.
/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/torch/nn/init.py:405: UserWarning: Initializing zero-element tensors is a no-op
  warnings.warn("Initializing zero-element tensors is a no-op")
/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/torch/nn/modules/linear.py:114: UserWarning: An output with one or more elements was resized since it had shape [4096, 256], which does not match the required output shape [256]. This behavior is deprecated, and in a future PyTorch release outputs will not be resized unless they have zero elements. You can explicitly reuse an out tensor t by resizing it, inplace, to zero elements with t.resize_(0). (Triggered internally at ../aten/src/ATen/native/Resize.cpp:26.)
  return F.linear(input, self.weight, self.bias)
Traceback (most recent call last):
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/bin/dingo_pipe_sampling", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/work/nihargupte/src/dingo-devel-aei/dingo/pipe/sampling.py", line 240, in main
    analysis.run_sampler()
  File "/work/nihargupte/src/dingo-devel-aei/dingo/pipe/sampling.py", line 183, in run_sampler
    prepare_log_prob(
  File "/work/nihargupte/src/dingo-devel-aei/dingo/gw/inference/inference_pipeline.py", line 244, in prepare_log_prob
    unconditional_model = result.train_unconditional_flow(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/src/dingo-devel-aei/dingo/core/result.py", line 468, in train_unconditional_flow
    unconditional_model = train_unconditional_density_estimator(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/src/dingo-devel-aei/dingo/core/density/unconditional_density_estimation.py", line 103, in train_unconditional_density_estimator
    model.train(
  File "/work/nihargupte/src/dingo-devel-aei/dingo/core/models/posterior_model.py", line 336, in train
    train_loss = train_epoch(self, train_loader)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/src/dingo-devel-aei/dingo/core/models/posterior_model.py", line 463, in train_epoch
    loss = -pm.model(data[0], *data[1:]).mean()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/src/dingo-devel-aei/dingo/core/nn/nsf.py", line 242, in forward
    return self.log_prob(y)
           ^^^^^^^^^^^^^^^^
  File "/work/nihargupte/src/dingo-devel-aei/dingo/core/nn/nsf.py", line 215, in log_prob
    return self.flow.log_prob(y)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/glasflow/nflows/distributions/base.py", line 40, in log_prob
    return self._log_prob(inputs, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/glasflow/nflows/flows/base.py", line 39, in _log_prob
    noise, logabsdet = self._transform(inputs, context=embedded_context)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/glasflow/nflows/transforms/base.py", line 56, in forward
    return self._cascade(inputs, funcs, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/glasflow/nflows/transforms/base.py", line 50, in _cascade
    outputs, logabsdet = func(outputs, context)
                         ^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/glasflow/nflows/transforms/base.py", line 56, in forward
    return self._cascade(inputs, funcs, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/glasflow/nflows/transforms/base.py", line 50, in _cascade
    outputs, logabsdet = func(outputs, context)
                         ^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/glasflow/nflows/transforms/coupling.py", line 84, in forward
    transform_params = self.transform_net(identity_split, context)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/glasflow/nflows/nn/nets/resnet.py", line 98, in forward
    temps = block(temps, context=context)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/glasflow/nflows/nn/nets/resnet.py", line 42, in forward
    temps = self.batch_norm_layers[0](temps)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/torch/nn/modules/batchnorm.py", line 138, in forward
    self._check_input_dim(input)
  File "/work/nihargupte/environments/micromamba/envs/eccentricity5/lib/python3.11/site-packages/torch/nn/modules/batchnorm.py", line 301, in _check_input_dim
    raise ValueError(
ValueError: expected 2D or 3D input (got 1D input)

I've tracked it down to the following line: https://github.com/uofgravity/nflows/blob/glasflow/nflows/transforms/coupling.py#L84
When training running this transform on a [n_batch, 1] shape tensor this identity_split is a size [n_batch, 0] tensor. Whereas if we have a 2D flow identity_split is a [n_batch, 1] size tensor.
But actually, this is a conceptual problem now that I think about it. In a coupling transform one needs at least a dimension 2 vector so that one can swap the identity portion (0th column) and the transform portion (1st column) of the vector after the transform.
A lazy workaround could be to pass a vector of 0's to the NDE as a second dummy column. I'm not sure how well this would do performance-wise since we would effectively be learning a delta-function cross the GNPE proxy distribution. (And learning a delta function for a flow is always hard). But I don't have a good intuition on this.
Otherwise, we could use a simpler set of transforms for the 1D case. That is something that doesn't use a coupling transform.
@stephengreen @jonaswildberger thoughts?


Update Android Studio to <IP_ADDRESS>

Version name, version code, and SHA-256 checksum pulled from the Android
Studio website. SHA-256 is for the Mac DMG image.

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} reports no offenses.
[x] The commit message includes the cask’s name and version.

Additionally, if adding a new cask:

[ ] Named the cask according to the token reference.
[ ] brew cask install {{cask_file}} worked successfully.
[ ] brew cask uninstall {{cask_file}} worked successfully.
[ ] Checked there are no open pull requests for the same cask.
[ ] Checked the cask was not already refused in closed issues.
[ ] Checked the cask is submitted to the correct repo.

Thank you.


Add BasePath to IConfigurationSourceRoot

Decouple the configration from dependency injection. To resolve a file
based configuration, user should either feed a full path or given a base
path to the Configuration constructor. The base path is usually resolved
from application base directory.
/cc @davidfowl @dougbu @Praburaj @glennc
Issue: https://github.com/aspnet/Configuration/issues/159

Ping. /cc @muratg

:watch:

Updated.
/cc @dougbu :watch:

This change will break templates. We also need to update the user secrets API.

/cc @rustd
This will break templates. https://github.com/aspnet/Templates/blob/dev/src/BaseTemplates/StarterWeb/Startup.cs#L22

:shipit: when @davidfowl is happy

Update to the UserSecrets https://github.com/aspnet/UserSecrets/pull/11

This looks like it addresses all the things I cared about :shipit:

OK lets see how big of a break this is :shipit:

@davidfowl @glennc some one take a look the change UserSecrets as well? https://github.com/aspnet/UserSecrets/pull/11

@Praburaj just sign off the UserSecrets change. I'll merge this PR shortly.

@troydai yeah I gave some comments on it already.

Can this be removed - https://github.com/aspnet/Configuration/blob/dev/src/Microsoft.Framework.ConfigurationModel/project.json#L12?

Ah yes it can be!


Load nullables from dask

[ ] Closes #9631
[ ] Depends on #9617
[ ] Tests added / passed
[ ] Passes pre-commit run --all-files

This builds on #9617 , adding a config option to handle setting nullable string dtypes, and handles a round trip from Pandas.  Noting that docs are still needed for the config.

This gist shows handling of https://github.com/dask/dask/issues/9631


let some fixed values configurable

Check list

[x] I wrote CHANGELOG.md if the pull request adds/modifies features.

Summary
This p-r make some fixed values specific for certain sensors configurable.

@ishiy1993 @eguchi1904 do you know any other sensor specific constants used in UnicamIspCapture?

:+1:


Throw a error on intel mac 10.15.7 19H1824 Command failed

Error while obtaining machine id:
Error: Command failed: ioreg -rd1 -c IOPlatformExpertDevice
    at ChildProcess.exithandler (node:child_process:406:12)
    at ChildProcess.emit (node:events:390:28)
    at maybeClose (node:internal/child_process:1064:16)
    at Socket.<anonymous> (node:internal/child_process:450:11)
    at Socket.emit (node:events:390:28)
    at Pipe.<anonymous> (node:net:687:12)

Yup, seeing this as well

Any updates on this issue team? Even I am seeing the same error on MAC 12.4 Intel chipset.
I am getting this error during the auto update.
Log print i see as - Checking auto update for rosetta environment.


Timer Issue Fix

I identified the error, seems like you have to have a history to make the timer start, it doesnt make the history row as it should in the database, thats the error :) made one manually and the timer started right away on first deposit :)
# Check if this is the first, second, or another deposit
$stmt = $db->query('SELECT * FROM history ORDER BY id DESC');
$mostRecentHistory = $stmt->fetch();

# If items is not empty, then this is the first deposit
$mostRecentItems = $mostRecentHistory['allItemsJson'];
if (strlen($mostRecentItems) > 0) {
	# Put a new row in, starting the next pot/round. However, don't start the timer yet.
	$stmt = $db->query('INSERT INTO history (endTime) VALUES (0)');
	$startTimer = 0;
}

# If there is already a round going but the timer hasn't started yet, then check to see if there are multiple people in the round
if ($mostRecentHistory['endTime'] === '0') {
	if (strlen($mostRecentItems) === 0) {
		$stmt = $db->query('SELECT * FROM currentPot');
		$allItemsCurPot = $stmt->fetchAll();
		$steamId = $allItemsCurPot[0]['ownerSteamId64'];

		$startTimer = 0;

		foreach ($allItemsCurPot as $item) {
			if ($steamId !== $item['ownerSteamId64']) {
				# There are multiple people in the pot, start the timer
				$endTime = round(microtime(true) * 1000 + 120000.0);
				$stmt = $db->prepare('UPDATE history SET endTime = :endtime');
				$stmt->bindValue(':endtime', $endTime);
				$stmt->execute();

				$startTimer = 1;
			}
		}
	}
} else { # This is if the timer is already running. It doesn't matter what $startTimer is
	$startTimer = 0;
}

So the code for depositing is expecting a row with endTime 0 in the Database, and it made it work to manually make a row with the following SQL
INSERT INTO history (endTime) VALUES (0)

What i did in steps is

Execute the sql
Deposit an item and let the timer run out
Succes??

It seems like this is the same as with the chat, where it expects the site to have a history like the chat expects a message to already be there

Well for me it did put a new history line tho but it didn't start the timer...

This is totally not needed ... you only need 2 different people to enter else it wont work.

@TheAnthonyNL Actually without following this step my website wouldn't start the timer when two people enter. After adding values to the history table as this step states it all worked flawlessly.

@TheAnthonyNL The point isnt that YOUR site wont work unless 2 enters, the point is that the first time you "install" the website and database, just like the chat not working without manually entering a message in the database, the timer/pot system wont start or work before you put in a manual row with the SQL i provided and is provided in deposit.php

Hmm i see well i will fix this for you guys if someone can help me with it

@TheAnthonyNL This is a fix for first time installs of the site :) and the website is supposed only to start the timer when 2 different steamId's enter :)

I know

I can confirm adding the first line myself to history fixed the same issue for me. Two different players - timer doesn't start; added line and made additional few items deposit - timer is now fully working.

@Stuffa1991  i only meant to update the deposit.php  so that you don't have to manual add the sql after you installed the site ¬¬/ but nobody cares so i won't update it.

Thx guys ! i luv you :p


Fieldset disabled no longer disables Select2

Field set disabled no longer disables select 2. This worked in the prior version 3.2.

This should be fixed in 4.0.
If it is not fixed, please add a :+1: to the ticket to bring it to our attention. Please also include an example that we can test to verify that it is not actually working.

Using jQuery 2.1.3 and Select2 4.0.0, it's not working for me on Chrome 45
http://jsfiddle.net/2fu4ws2y/

Yup, looks like this is still an issue. We are going to need to figure out what check Sizzle used to use to determine the disabled status of an element. Clearly it's not prop('disabled'), as that returns false in Chrome.

I believe .is('.disabled') is quite reliable.

Hey we just updated 4.0.0 and it still seems to be an issue. I was just wondering if there were any plans for an update with a fix. Thanks.

It's a fundamental problem with Internet explorer, that said there is a jquery workaround in the works.
On Feb 18, 2016 5:39 PM, Aryeh Stiefel<EMAIL_ADDRESS>wrote:Hey we just updated 4.0.0 and it still seems to be an issue. I was just wondering if there were any plans for an update with a fix. Thanks.
—Reply to this email directly or view it on GitHub.

For me it seems to not work even in Chrome

I can also confirm that disabling a fieldset fails to disable a select2 within the fieldset in Chrome on v4.0.0.

I'm experiencing this as well.
I did notice the fieldset disabled selector issue was recently fixed in jquery:
https://github.com/jquery/sizzle/pull/283
https://github.com/jquery/sizzle/issues/174
Seems it will roll out in JQ2.3.

Any luck fixing the issue? It still is present
Chrome 54
Select2 4.0.3
jQuery 2.2.3

This issue is not stale.  It still exists in 4.0.12
Also, IE doesn't matter much anymore.  All the other browsers support disabled fieldset including Edge:
https://caniuse.com/#feat=fieldset-disabled
Please re-open.

I ran into this issue yesterday and woke up early to hack together a fix for it. Unfortunately, I'm not familiar with the unit test framework used by the project, and I don't have time to figure it out right now - I'll try to get to it this weekend. I'll open a PR shortly in case anybody wants to review the changes in the meantime, or can assist with the tests.

Any news on this ?

Any news?

Any news ?


Ignore graphql module due to<EMAIL_ADDRESS>errors

This is a temporary fix and we should remove the graphql stub once a working version is published.

!merge


Pressing cmd+alt+c doesn't copy path in explorer.

VSCode Version: version 1.26.0-insider
OS Version: 10.13.4 (17E199)

Steps to Reproduce:

click on file in explorer
press cmd+alt+c
press cmd+v in editor
no path printed

Does this issue occur when all extensions are disabled?: Yes/No yes

sorry it was a typo
All ext were disabled, non were installed


Regions bounds extend further beyond the bottom of the text

As above, the bounds of regions seem to extend a little further beyond the bottom of where they should. This causes some issues in detection in the following examples:
Video showing two examples
To investigate further, I drew the bounds of regions to the screen. This is what led me to believe it maybe the issue, as you're still technically hovering over the other region:
Image showing Bounds drawn to screen
Let me know if you'd like anymore details! Thank you :)
Edit: The font I'm using in the above example is "Teko", found here:
Google Fonts: Teko

Not possible to fix this fully as GameMaker is giving me some UVs for each character so I can't get the actual visual bounding box. Instead, if I scan backwards then this mostly works around the issue. Work around is here and will be released in v8.2: https://github.com/JujuAdams/Scribble/commit/d54104d05afaab0e1804b7b14429caab5b3b25ed


[docs] add diagrams and fix broken links

Description
Added some diagrams that explains the escrow concepts better and fixed broken code sample link
Test Plan
How did you test the new or updated feature?
Visual inspection

If your changes are not user-facing and do not break anything, you can skip the following section. Otherwise, please briefly describe what has changed under the Release Notes section.
Type of Change (Check all that apply)

[ ] protocol change
[ ] user-visible impact
[ ] breaking change for a client SDKs
[ ] breaking change for FNs (FN binary must upgrade)
[ ] breaking change for validators or node operators (must upgrade binaries)
[ ] breaking change for on-chain data layout
[ ] necessitate either a data wipe or data migration

Release notes

I will look into drawing the diagram with mermaid


DO NOT MERGE: Sentry testing

Description

Checklist:

[ ] Have you documented your changes in the pull request description?
[ ] Does the change present any security considerations?
[ ] Is any gem functionality overloaded? Eg: Devise controller methods being overloaded.
[ ] Has acceptance criteria been tested by a peer?
[ ] Has the CHANGELOG been updated? (If change is worth telling users about.)

General testing (author)

[ ] Test without JavaScript
[ ] Test on small screen

Accessibility testing (author)

[ ] Works keyboard only
[ ] Tested with one screen reader
[ ] Zoom page to 400% - content still visible
[ ] Disable CSS - does content make sense and appear in a logical order?

Codecov Report

Merging #1845 (0a5b35c) into master (065bbac) will decrease coverage by 65.92%.
The diff coverage is 0.00%.

@@             Coverage Diff             @@
##           master    #1845       +/-   ##
===========================================
- Coverage   89.62%   23.70%   -65.93%
===========================================
  Files         399      399
  Lines        8665    10994     +2329
  Branches     1257      669      -588
===========================================
- Hits         7766     2606     -5160
- Misses        718     8310     +7592
+ Partials      181       78      -103

Impacted Files
Coverage Δ

app/jobs/product_export_job.rb
0.00% <0.00%> (-66.67%)
:arrow_down:

app/models/contact.rb
0.00% <0.00%> (-100.00%)
:arrow_down:

app/models/accident.rb
0.00% <0.00%> (-100.00%)
:arrow_down:

app/models/incident.rb
0.00% <0.00%> (-100.00%)
:arrow_down:

app/forms/alert_form.rb
0.00% <0.00%> (-100.00%)
:arrow_down:

app/forms/comment_form.rb
0.00% <0.00%> (-100.00%)
:arrow_down:

app/jobs/send_alert_job.rb
0.00% <0.00%> (-100.00%)
:arrow_down:

app/policies/team_policy.rb
0.00% <0.00%> (-100.00%)
:arrow_down:

app/helpers/search_params.rb
0.00% <0.00%> (-100.00%)
:arrow_down:

app/forms/remove_user_form.rb
0.00% <0.00%> (-100.00%)
:arrow_down:

... and 296 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 065bbac...0a5b35c. Read the comment docs.


⚠ Bofrost.lu has degraded performance

In 13c2027, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 897 ms

Resolved: Bofrost.lu performance has improved in 1aa3f28 after 6 minutes.


Add footer and header and fix button width

This pull request adds a footer and header to the Chat component. The header displays the name of the bot, and the footer displays the text "Powered by dbBot". Additionally, the width of the ChatButton component has been fixed to 120%. This resolves issue #26.

Images? @ShlomiShitrit


TRITON-1943 add Jenkinsfiles to Triton repositories

make check failure:
00:00:41.570  deps/jsstyle/jsstyle -f tools/jsstyle.conf lib/identity.js lib/formats/rfc4253.js lib/formats/pkcs1.js lib/formats/openssh-cert.js lib/formats/x509-pem.js lib/formats/dnssec.js lib/formats/auto.js lib/formats/putty.js lib/formats/pem.js lib/formats/ssh.js lib/formats/ssh-private.js lib/formats/x509.js lib/formats/pkcs8.js lib/ed-compat.js lib/utils.js lib/algs.js lib/key.js lib/private-key.js lib/certificate.js lib/errors.js lib/ssh-buffer.js lib/fingerprint.js lib/dhe.js lib/signature.js lib/index.js
00:00:41.570  Unescaped left brace in regex is illegal here in regex; marked by <-- HERE in m/\S{ <-- HERE / at deps/jsstyle/jsstyle line 666.
00:00:41.570  make: *** [tools/mk/Makefile.targ:183: check-jsstyle] Error 255

is that something you've seen before?

Codecov Report

Merging #64 into master will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #64      +/-   ##
==========================================
+ Coverage   87.11%   87.14%   +0.02%
==========================================
  Files          25       25
  Lines        4013     4013
==========================================
+ Hits         3496     3497       +1
+ Misses        517      516       -1

Impacted Files
Coverage Δ

lib/signature.js
88.94% <0%> (+0.48%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e97ab59...c5bb75e. Read the comment docs.


feat: add HTTP client with exponential backoff policy

The existing HTTP client tries requests only once.
This new client will ensure that:

server initialization won't fail if the OIDC keys endpoint is slow to respond
functional tests won't fail if the dummy OIDC server is slow to start. E.g. https://github.com/openfga/openfga/runs/6839424195?check_suite_focus=true#step:7:28

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@c86993b). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main       #3   +/-   ##
=======================================
  Coverage        ?   53.38%
=======================================
  Files           ?       25
  Lines           ?     2465
  Branches        ?        0
=======================================
  Hits            ?     1316
  Misses          ?     1053
  Partials        ?       96

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c86993b...195f408. Read the comment docs.

@miparnisari After thinking about this a little more. Would it be possible to not introduce a whole new HTTP client library with retries and just use the backoff.Retry(...) approach to wait for the mock OIDC server to start up, just like we do with the OpenFGA server ensureServiceUp.
I think it'd be nice if we could avoid introducing all of the HTTP client code into the OpenFGA package just to wait for an HTTP server's healthiness. If we're going to introduce it though, it would at least be nice to put it in an openfga/pkg repository that has shared packages like this so we can share and reuse it elsewhere. If we're going to introduce it into our software maintenance cycle, we may as well be able to share it and evolve across other projects in the OpenFGA organization on Github.

I think it'd be nice if we could avoid introducing all of the HTTP client code into the OpenFGA package just to wait for an HTTP server's healthiness.

That's not the only use case, I also want to retry when fetching the JWKs in the OIDC authenticator.

it would at least be nice to put it in an openfga/pkg repository that has shared packages like this so we can share and reuse it elsewhere

I'm confused, we don't need to put it in a brand new repository to reuse it, no?

I'm confused, we don't need to put it in a brand new repository to reuse it, no?

You're right, we don't have to. But for a shared library that expands beyond the scope of just this project it would make sense for it to live elsewhere. An import from openfga just for a shared retry http client is what would seem odd to me. I would expect an import like that to come from a shared repository, not openfga. That's what I meant.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

I'm confused, we don't need to put it in a brand new repository to reuse it, no?

You're right, we don't have to. But for a shared library that expands beyond the scope of just this project it would make sense for it to live elsewhere. An import from openfga just for a shared retry http client is what would seem odd to me. I would expect an import like that to come from a shared repository, not openfga. That's what I meant.

👍 i think when we have more things that we want to reuse we can create a new repo


2 out of 3 bbs acting as master

CF v 270
Diego v 1.23.1
IaaS: AWS
2 AZs
Tonight there was a failover of the rds db (5m) and after that the system was kinda down (no push, no delete, etc). While investigating, we found out that:

bbs1 had "too many open files" error (but was still acting as master)
bbs2 took over as master but didn't do anything
bbs3 was idle

I guess something went wrong on the bbs side because I don't think it's normal to have 2 masters :)
What can I do to help you debug this issue?

Thanks for letting us know about this issue, @paolostivanin. Would you be able to provide the stdout and stderr logs from all the BBS instances in this deployment, as well as the times when the RDS instance failed over or times when you noticed or resolved these degradations of service?
Some additional questions:

Are the BBS instances configured to acquire their locks in Consul, Locket, or both?
Could you clarify what you mean when you say that BBS #1 was still acting as master and that BBS #2 took over as master but didn't do anything? Which instance or instances were handling API requests? Which instance or instances were performing convergence operations? If you happen to know it, which instances held the BBS lock during which periods of time?

Thanks,
Eric, CF Diego PM

@emalm yes, I do have the stderr and stdout logs for the 2 bbs that were acting as master. Can I send them to you as they are of do I have to "clean" them before? AFAIK these logs shouldn't contain credentials, tokens, etc, right?
The following times are in UTC.

23.8. 01:33-01:34 - RDS used for CC DB did a fail over (took ~ 1min), this happened during the maintenance window specified for this RDS instance so it is a normal situation

23.8. 01:33-01:40 - CF API (/v2/info) was not available, RDS downtime triggered restart of cloud controller process (on api nodes), unclear why this took so long

23.8. 01:33-09:04 - Diego BBS (aka database) not available

02:40 - Diego BBS came back automatically but with 2 masters which means it is still not working
09:04 - manually repaired Diego BBS via bosh recreate, Diego BBS is available again (1 master, 2 slaves)

BBS instances acquire their locks using consul (locket is still marked as experimental)

I mean that prior to the outage only one BBS was reporting to grafana but after the outage 2 BBS were reporting at the same time. CF api and any lifecycle operation (including repair of crashed app instances) were not available. Prior to the outage, database_z1 83efe73d-0b03-47eb-a74e-52b2a2bc59df was the master. After the outage (and the manual repair) database_z2 865e9a1e-0dfe-44be-89e8-52c570d08823 became the master.

Thanks for the additional information, @paolostivanin, especially about the timing. I believe the BBS logs would be safe to send without any scrubbing to remove sensitive credentials or environmental details.
In the hopefully unlikely event that this issue occurs again before we identify the root cause and introduce any relevant fixes, could you also try killing the stuck BBS process with kill -QUIT $(pidof bbs), and then capture the goroutine dump that the process emits to the stderr log?
Thanks again,
Eric

Hi,
please find attached the stderr and stdout logs for bbs_z1/0 and bbs_z2/0 and /1. If you need I also have the api_worer and (partial)
api logs.
bbs_all.tar.xz.zip

@paolostivanin What type of RDS database are you using, postgres or MySQL?

postgres!

@paolostivanin In your timeline above, you have two events for the BBS. One at 02:40 where the BBS came back and two at 09:04 when you manually repaired the BBS via BOSH recreate. At what point did CF API operations start working?

@crhino CF api and any lifecycle operations (including repair of crashed app instances) were not available for ~7.5h. API started working again ~9:04 UTC

@paolostivanin Is it possible for you to get the cell logs from that time period? That would help diagnose the downtime issue.

Unfortunately no. We have a very high load and a hundred of cells, so we don't keep the logs too long.

We think we have figured out why the deployment wasn't working. Looking at the BBS logs for the node that did not shut down, you can see that there are still bbs.request.cells being logged starting soon after the BBS loses its lock. We believe this is the auctioneer attempting to fetch the CellReps from the BBS, as it retries every second (which the auctioneer is configured to do on failure), and started right as the new BBS took the lock and started to do convergence processes. Our theory is that the auctioneer had an idle TCP connection to the old BBS hanging around, and when it received work, reused that idle connection to the old BBS. That bypasses any DNS lookup, so it would not see a DNS change to the new BBS node.
The root cause of this is that the postgres driver does not provide any timeouts associated with a backing postgres database, causing the BBS to hang during shutdown. You can see here that others have experienced the problem of hanging database connections. We have written this story to implement postgres timeouts.
Hope that helps, we will close this issue since we have a story to track the work.


Add HOMEBREW_UPGRADE_GREEDY="auto-updates" and "latest"

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[x] Have you written new tests for your changes? Here's an example.

[x] Have you successfully run brew typecheck with your changes locally?
[x] Have you successfully run brew tests with your changes locally?

Fixes: #16735

This should also be supported in brew outdated, which also has these options.

Added in latest update.

@reitermarkus If you disagree: let's discuss here instead.

As I mentioned in the issue, I prefer one variable since it avoids the case where multiple can be specified. For the user it's a choice between 1, latest and auto-updates.

@reitermarkus Feels a bit weird to have 1 be one of these options and also there's few variables that take multiple values like that so may be more confusing than not.

FWIW, the existing code does handle when multiple options are specified (e.g. brew upgrade --greedy --greedy-auto-updates --greedy-latest).
https://github.com/Homebrew/brew/blob/7848bd32269d5fcfecc00aeb142db426b9896ed6/Library/Homebrew/cask/upgrade.rb#L79-L90
If separate env vars is preferred I have a patch ready to go for that too (https://github.com/jck112/Homebrew-brew/commit/ea119ed07f3d943d0bb3fdf2f0f7b0c150f77a33).

Thanks @jck112! Will let you know when we've decided.

@Homebrew/maintainers @Homebrew/cask any thoughts on one variable vs. many? We don't have many variables which have differing behaviours due to setting the same variable to different values so feels unintuitive but I'm open to other viewpoints.

Thanks @jck112! Will let you know when we've decided.
@Homebrew/maintainers @Homebrew/cask any thoughts on one variable vs. many? We don't have many variables which have differing behaviours due to setting the same variable to different values so feels unintuitive but I'm open to other viewpoints.

I would prefer just one variable.

My general opinion (to anything):

If the behaviour of multiple at the same time is well defined and supported (and not just a case of one overrides the other), then multiple variables makes sense
Otherwise just a single variable is best

Feels a bit weird to have 1 be one of these options

We can call it true or always if that makes it less weird, but the code currently accepts any value anyways.

We don't have many variables which have differing behaviours due to setting the same variable to different values

That's because we also don't have that many flags that have the same prefix with different behaviours like --greedy, --greedy-latest, --greedy-auto-updates.
The one I could find is --linux, --linux-self-hosted and --linux-wheezy in dispatch-build-bottle, but that doesn't really need a global setting.

If the behaviour of multiple at the same time is well defined and supported

That's actually a good question. I don't think it is well defined and more of an implementation detail. Is --greedy-latest + --greedy-auto-updates equal to just --greedy?

Is --greedy-latest + --greedy-auto-updates equal to just --greedy?

Yeah, it is.

Yeah, it is.

Okay, if it wasn't, separate variables would make sense to me.
Given that it is equivalent, one variable makes more sense to me.

I think following the flags and having three variables makes sense to me.

Personally, I think it would actually make more sense for the flags to be --greedy, --greedy latest and --greedy auto-updates so only one option can be specified. For now, they should at least have conflicts with each other.

I recall we passed on this in #15164, but I'm glad this is discussed. Regardless, I agree with the 3 variable approach if we are keeping the flags as they are.

I think it would actually make more sense for the flags to be --greedy, --greedy latest and --greedy auto-updates

I like this approach a lot more, with the one caveat that --greedy itself is confusing and not descriptive enough to differentiate it from the other options. Perhaps having something along the lines of --greedy all as an alias for --greedy?

@jck112 Sounds right to me! Don't need to be all in this PR if you don't want but, if they are, that might be nice!

I spent some time working on this but ran into a few implementation details that require some input:

Which format is preferred: "--greedy=all" or "--greedy all"? The former is an OptionalArgument type and the latter is a PlacedArgument type, which is determined internally by OptionParser using the format "--arg=[ARG]" or "--arg [ARG]" respectively. The parser.rb code requires some modifications to support PlacedArgument options.
A side-effect of changing --greedy to a flag, is that the short version -g can also accept an argument (e.g. -gall, -gauto-updates, -glatest). The only way to avoid this with OptionParser is to declare -g as a separate switch, which would also mean it would be listed seperately in documentation (e.g. brew help upgrade).

Are you okay getting rid of -g and --greedy (i.e. force users to switch to --greedy all)? Doing flag   "--greedy=" will create an option with RequiredArgument type, which means OptionParser will throw an error if the user provides --greedy without an argument:
$ brew upgrade --greedy
Error: missing argument: --greedy

Are you okay getting rid of -g and --greedy (i.e. force users to switch to --greedy all)?

@jck112 We can make it hidden: true but cannot remove it just yet (unfortunately) as we'll need to go through a deprecation cycle.

Doing flag "--greedy=" will create an option with RequiredArgument type, which means OptionParser will throw an error if the user provides --greedy without an argument:

Ah. Annoying. @homebrew/brew any thoughts on how best to address this temporarily?


Multiline comment folding doesn't work in 1.22.1

Issue Type: Bug

Create new .js file.
Paste text bellow.

/**
 * This is not folding.
 * @api {get} /user/:id Request User information
 * @apiName GetUser
 * @apiGroup User
 *
 * @apiParam {Number} id Users unique ID.
 */

let fn = () => {
    // This folds nicely
    return 1;
}

Comment is not offering folding option, function does. This is standard apidoc format for writing comments. Same happens with code --disable-extensions. It used to work before the latest update.
VS Code version: Code 1.22.1 (950b8b0d37a9b7061b6f0d291837ccc4015f5ecd, 2018-04-06T00:21:13.607Z)
OS version: Darwin x64 17.5.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz (8 x 2900)

Load (avg)
3, 2, 2

Memory (System)
16.00GB (0.10GB free)

Process Argv
/Applications/Visual Studio Code.app/Contents/MacOS/Electron

Screen Reader
no

VM
0%

Extensions (9)

Extension
Author (truncated)
Version

xml
Dot
1.9.2

vscode-nsp
ada
1.0.1

dakara-foldplus
dak
1.3.0

vscode-eslint
dba
1.4.8

githistory
don
0.4.0

rainbow-csv
mec
0.2.1

vscode-exec-node
mir
0.5.0

code-spell-checker
str
1.6.6

local-history
xyz
1.6.2

Duplicate of #46651 -> https://github.com/Microsoft/TypeScript/issues/22732
Set "typescript.experimental.syntaxFolding": false to use the old indentation based folding

Thanks @mjbvz !


Update admin-documentation.md

Updated Project Service Automation to Project Operations

@stsporen We reviewed this and are holding on to this change until we get a broader guidance from the product team and branding. Closing this wothout mergin.
Thanks for your contribution.


SqlServer for Ubuntu: DbContextPooling & SequenceEndToEnd tests hits frequent timeouts

Disabled tests on Ubuntu
Double_dispose_concurrency_test
Concurrency_test
Can_use_sequence_end_to_end
Can_use_sequence_end_to_end_from_multiple_contexts_concurrently_async
Using SqlServer 2017 on Ubuntu 16.04

Stack trace from Can_use_sequence_end_to_end_from_multiple_contexts_concurrently_async
System.Data.SqlClient.SqlException : Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
---- System.ComponentModel.Win32Exception : Unknown error 258
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParserStateObject.ReadSniError(TdsParserStateObject stateObj, UInt32 error)
   at System.Data.SqlClient.TdsParserStateObject.ReadSniSyncOverAsync()
   at System.Data.SqlClient.TdsParserStateObject.TryReadNetworkPacket()
   at System.Data.SqlClient.TdsParserStateObject.TryPrepareBuffer()
   at System.Data.SqlClient.TdsParserStateObject.TryReadByte(Byte& value)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
   at System.Data.SqlClient.SqlDataReader.get_MetaData()
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite, String method)
   at System.Data.SqlClient.SqlCommand.ExecuteScalar()
   at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.Execute(IRelationalConnection connection, DbCommandMethod executeMethod, IReadOnlyDictionary`2 parameterValues) in /home/entityframework/BuildAgent/work/46872f3c7fa72c0c/src/EFCore.Relational/Storage/Internal/RelationalCommand.cs:line 224
   at Microsoft.EntityFrameworkCore.ValueGeneration.Internal.SqlServerSequenceHiLoValueGenerator`1.GetNewLowValue() in /home/entityframework/BuildAgent/work/46872f3c7fa72c0c/src/EFCore.SqlServer/ValueGeneration/Internal/SqlServerSequenceHiLoValueGenerator.cs:line 54
   at Microsoft.EntityFrameworkCore.ValueGeneration.HiLoValueGeneratorState.Next[TValue](Func`1 getNewLowValue) in /home/entityframework/BuildAgent/work/46872f3c7fa72c0c/src/EFCore/ValueGeneration/HiLoValueGeneratorState.cs:line 66
   at Microsoft.EntityFrameworkCore.ValueGeneration.ValueGenerator`1.NextValueAsync(EntityEntry entry, CancellationToken cancellationToken) in /home/entityframework/BuildAgent/work/46872f3c7fa72c0c/src/EFCore/ValueGeneration/ValueGenerator`.cs:line 49
   at Microsoft.EntityFrameworkCore.ValueGeneration.ValueGenerator.NextAsync(EntityEntry entry, CancellationToken cancellationToken) in /home/entityframework/BuildAgent/work/46872f3c7fa72c0c/src/EFCore/ValueGeneration/ValueGenerator.cs:line 40

@saurabh500 @geleems These are all tests which hit the database pretty hard with multiple connections. Do you think these failures might be related to the Linux/SQL Server issues that are being investigated?

@ajvikers this is the next item we are picking up for perf investigation.
@smitpatel @ajcvickers Do you know if EF Core on Windows against SQL server on Linux works OK?
I am trying to understand if its the SqlClient networking or could this be server networking that needs to be looked into.

@saurabh500 We only test everything on Linux or everything on Windows for this case so we don't have any info if the tests also fail when running Windows pointing at Linux or vice versa.

@smitpatel, @ajcvickers
Could you please proved sample repro for me to reproduce the issue in my local environment?

@smitpatel, @ajcvickers
Could you please provided sample repro for me to reproduce the issue in my local environment?

@geleems It's my understanding that you guys are able to run EF tests--at least, that was the case last time I talked with @saurabh500. Given that, you should be able to run the given tests on Linux and reproduce the issue. Or have you tried this and are not finding it to repro?

@geleems You need to run the EF tests.
You can build the EF Clone and run
~/.dotnet/dotnet test --framework netcoreapp2.0 -v d   --no-build
The above command runs all the tests.
You may want to run this particular failing test using following instructions
Running tests selectively using dotnet test is done via “filter” arg. From help printed out in console. --filter <EXPRESSION>                 Run tests that match the given expression.                                         Examples:                                         Run tests with priority set to 1: --filter "Priority = 1"                                         Run a test with the specified full name: --filter "FullyQualifiedName=Namespace.ClassName.MethodName"                                         Run tests that contain the specified name: --filter "FullyQualifiedName~Namespace.Class"                                         More info on filtering support: https://aka.ms/vstest-filtering
After this you run the test, which may or may not reproduce the issue based on the configuration of the VM.
To extract a SQLClient specific test, I would attach a SQL Profiler to the Sql server and see what queries are being executed and try to repro the issue using SqlClient.
I would start by running the tests on Linux against a local Linux server vs a Windows SQL Server to see if the server is acting up in this case. If there is a difference in the server side behavior, then this may be a server issue. Else we need to isolate the problem on the SqlClient.
If you figure that this is a SqlClient perf issue, you could enable managed SNI on Windows and run the tests and collect tracing using PerfView.exe That should give you insights on CPU hotspots if any. Else you may want to capture the thread time in PerfView to figure out if we are waiting for any thread scheduling.
As a first step, I recommend setting up the tests on Linux and run this specific test against the two server SKUs to find what component is a bottle nech

I am testing out with code on Ubuntu and server on widows.

@smitpatel
What is the new stack trace you got after testing with .NET Core that includes those fixes I mentioned above?
at System.Data.SqlClient.TdsParserStateObject.ReadSniSyncOverAsync() should not appear in your new stack trace since ReadSniSyncOverAsync() method was removed by the fixes I mentioned above.

@geleems - Stack trade did not change. Can you tell minimum version of SqlClient to be targeted to get the fix?
Also, running on Linux targeting SqlServer on windows did work without error.

@smitpatel
Sorry I was confused with ConnectSyncOverAsync() method that was removed. Please ignore my last comment.
Does the issue only happen when it targets local Linux SQL Server? Or it does not matter whether it targets local SQL Server or remote SQL Server?

@smitpatel @ajcvickers I am looking into the EFCore code  and I see that the SqlClient version it depends on is from 12/16/2017
Can you update the version to the latest and run the tests again? Since then there have been 2 more performance fixes. If this upgrade can be easily done and validations be run in your CI, that would be quite nice before we dive deeper into this issue.

@saurabh500 - What is the latest version of SqlClient? It is not straight fwd to upgrade dependency in the code but i can try to upgrade locally and run the test on linux

The latest would be 4.5.0-preview2-26202-05
https://dotnet.myget.org/feed/dotnet-core/package/nuget/System.Data.SqlClient

@saurabh500 - I tested with 4.5.0-preview2-26202-05 but it still failed. Even a different test started failing which did not fail in past.

@smitpatel what is the other test failing?
Thanks for trying out the updated version.

new test failed after updating
MonsterFixupChangedChangingSqlServerTest.Can_build_monster_model_and_seed_data_using_principal_navigations
Same stack trace.

Moving this to backlog to re-enable the test when the external issues are fixed.

We should try this again with the new SqlClient


Remove headers.accept from server middleware

Because files don't need to compile on each load, don't need the header accept if overhead.
Also replace filter with find.

Resolved by https://github.com/Snugug/vite-plugin-eleventy/commit/2045b8cd6a42fa44d7dd266a4867376ac28d657d


corretto,adoptopenjdk: cleanup

Description of changes
Split from #286267
CCing @thiagokokada @philiptaron as previous reviewers
Small cleanup bits from the original PR that didn't fit into the other split PRs.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Folded into #314380


Fix fallback logo being detected by chromes image description feature

role="none"/"presentational" does not do enough to hide the child fallback image used in older browsers.
We have found that Chrome's new "Get image descriptions from Google" feature will notice this fallback image.
This means when screenreaders such as NVDA visit the logo, it'll announce "Unlabelled graphic, To get missing image descriptions, open the context menu." for this fallback image.
By using aria-hidden="true" we can ensure that the SVG and it's children are all hidden from the accessibility tree, which I have confirmed in NVDA + Chrome.
I have also made this change to the other places we're using role="presentation" in this way for consistency.

Get image descriptions on Chrome
The Difference Between role=”presentation” and aria-hidden=”true”

Closes https://github.com/alphagov/govuk-frontend/issues/1636

Makes sense to me, thanks Nick 👏 How confident are you that this will work consistently everywhere? Do you think we need to test this change across all of the assistive technologies we support?

@36degrees I have not done a full run through testing with all assistive technologies and browsers as I have confidence that this aria attribute has very broad support and we use it elsewhere (for example the accordion component).
I can do some testing soon to double check.

Well done Nick, proud of you.

@36degrees I've tested this in JAWS, NVDA in Edge, Chrome, IE11, Firefox and VoiceOver in Safari and it dont hear any icons being announced when tabbing to the GOV.UK link in the header.

@36degrees I've tested this in JAWS, NVDA in Edge, Chrome, IE11, Firefox and VoiceOver in Safari and it dont hear any icons being announced when tabbing to the GOV.UK link in the header.

If you're confident with the changes to the other components as well then I think this is good to merge 👍

@36degrees I also tested the start link button

Needs a changelog,oops


Fix bug in draw_line() if y2 < y1

Here is a quick solution. Hope this help ;)

GitHub Gist is here https://gist.github.com/armel/c45ab7e2632b6e6788c090074ae21cce for a bug example. With
@M5Stack Glass Unit, there is a bug with draw_line(x1, y1, x2, y2, mode).
If y2 < y1, we obtain an horizontal line 🤔 Hope this help.

Thanks!

Thanks for this contribution!


YOLACT Multi-scale Training

Does YOLACT implementation support multi-scale training? It raises a dimension error while converting labels from image level to feature level.
Thank you for your support.
Baris.

Hi @cancam Sorry, no support at the moment.


MM-26329 Fix Channel info guests info in landscape

Summary
on iOS in landscape mode the channel info screen wasn't applying the padding to the guest subtitle (actual fix https://github.com/mattermost/mattermost-mobile/commit/be19a050159550309b3a8419521a9ded7add6539)
This PR also refactors the Channel Info screen and splits every option into its own component (this helps with the issue we were having on toggling the favorite, mute and ignore mentions options)
Ticket Link
https://mattermost.atlassian.net/browse/MM-26329

Building app in separate branch.

Successfully building:
https://circleci.com/gh/mattermost/mattermost-mobile/9480
https://circleci.com/gh/mattermost/mattermost-mobile/9479

Artifact links:
https://9480-70265724-gh.circle-artifacts.com/0/~/mattermost-mobile/Mattermost_Beta.ipa
https://9479-70265724-gh.circle-artifacts.com/0/~/mattermost-mobile/Mattermost_Beta.apk

@josephbaylon is the option available on previous versions/web? It should render if you have permissions, but if you can confirm I’ll take another look

@enahum Yes it is on v1.34 build 318 (see image)

@josephbaylon believe it or not, it was cause of a type 🤦

Building app in separate branch.

Successfully building:
https://circleci.com/gh/mattermost/mattermost-mobile/9504
https://circleci.com/gh/mattermost/mattermost-mobile/9503

Artifact links:
https://9504-70265724-gh.circle-artifacts.com/0/~/mattermost-mobile/Mattermost_Beta.apk
https://9503-70265724-gh.circle-artifacts.com/0/~/mattermost-mobile/Mattermost_Beta.ipa


register eventshub store first to prevent loss of events

Fixs eventing flaky tests,like

https://github.com/knative/eventing/runs/2888678781?check_suite_focus=true#step:12:5784
https://github.com/knative/eventing/runs/2885623770?check_suite_focus=true#step:12:5596

These flaky tests are caused by the EventsHubStore not receiving the probe-sender's event which leads to the failure of the step of checking whether the probe-sender is completed.
After add logs to probe-sender pod and list all k8s event in the end. It is found that the probe-sender sent the event successfully. The problem is that the EventsHubStore did not receive this event.
The problem may be in the registered listener section,
Currently, the EventsHubStore is registered(registerEventsHubStore()) after the probe-sender pod is running/fail(k8s.WaitForPodRunningOrFail()). However, there may be a situation where the probe-sender pod has finished sending events and then the EventsHubStore starts to register EventListener. This will cause some events to be lost.
I ran kind-e2e tests more than 20 times on my forked repo, found that after applying this fix, these flaky tests did not occur

Changes

:bug: change the order of registerEventsHubStore and k8s.WaitForPodRunningOrFail(), to prevent loss of probe-sender‘s event.

/kind bug

Release Note

Docs

/close  Fixed by https://github.com/knative-sandbox/reconciler-test/pull/218


Update amazon-s3-bucket.md

Added space at the end for testing
Purpose of this pull request
This pull request (PR) ...
Issue number:

Select the type of change:

[x] Minor Changes - Typos, formatting, slight revisions
[ ] Update Content - Revisions and updating sections
[ ] New Content - New features, sections, pages, tutorials
[ ] Site and Tools - Updates, maintenance, and new packages for the site, Gatsby, React, etc

@jschwegler-sumo , thank you so much for completing this UAT use case!
I've added you as an internal contributor. Once this PR has been merged (~30 mins from now), you'll see able to merge your request.
If you have questions, please let us know.
cc: @stacykor

Thanks for helping with UAT and successfully submitting a PR!


Wma/parallel improvements

The problem here is that we're trying to use FastClosures, let's move to just use polyester for now, and make an issue for fastclosures (which is the better overall option going forward)


File descriptor safety for program fd + prog attaching

This PR is divided into two commits:

Use OwnedFd inside ProgramData to ensure file descriptor safety.
Use AsFd when attaching programs externally and save them as OwnedFd internally.

This PR tries to limit the amount of breaking changes as much as possible, in that the signatures are changing but in practice none or very little changes would be required as a user. This is highlighted by the fact that none of the tests and only one of the doctests had be be touched. The one doctest that was touched was due to the fact that the doctest was explicitely calling for as_raw_fd() on a TcpStream for attaching to a socket filter program but this function now accepts anything that implemented AsFd so the change was just to remove the explicit call to as_raw_fd() and instead pass a reference to the stream directly.
This PRs reveal some existing unsafeties/leaks but does not address them as that would require a larger api change that I'd need some feedback from the maintainers to know what you all would want to do. Namely:

SockMapFd wraps a RawFd but there is no compile time nor runtime check that the file descriptor remains valid when passed back to attaching it to a SkMsg or a SkSkb. This is now highlighted by the fact that SockMapFd now implements AsFd which required use of unsafe { BorrowedFd::borrow_raw }. Adding a lifetime to this struct would fix the issue but it'd require some thinking on how to handle  mutably borrowing a Program while borrowing a Map at the same time.

ProgramFd wraps a RawFd but there is no compile time nor runtime check that the file descriptor remains valid when passed back to a method in an aya program. This is now highlighted by the fact that this struct now implement AsFd which required use of unsafe { BorrowedFd::borrow_raw }. Adding a lifetime to this struct would fix the issue but it'd require some thinking on how to handle  mutably borrowing two programs at the same time.

LircMode2::query gets a vector of owned file descriptors internally and then returns a list of LircLinks with those file descriptors in them. However, LircLink does not drop its file descriptor on close however because the other way to make it is from LircMode2::attach by re-using the file descriptor from the already loaded program. Possible solutions:

We could make LircMode2::attach duplicate the file descriptor so that the link has an OwnedFd that it will close on drop by default. This wouldn't be a breaking change from an API perspective but I am not sure if cloning a file descriptor would be a nono in this case for some reason I don't know about (I don't think it matters much performance-wise fwiw).
We could also make ProgramData use an Arc<OwnedFd> instead that we can clone very cheaply but that to me feels like a weird bandaid since in this case there should only be ONE owner and also affecting all programs when this is only an issue for LircMode2 feels like too big of a hammer.
We could add a lifetime to LircLink and then have it internally store some sort of Cow<BorrowedFd> thing so it can have an Owned or BorrowedFd based on whether we got it from the attach or the query. I am not sure how LircLinks are used so I don't know if this would turn out awkward to use
Completely rework or split query into multiple functions so it can return OwnedFd of the programs separately from making the LircLink.

Relates to: https://github.com/aya-rs/aya/issues/612

This change is

always calling try_clone_to_owned on any ProgramFd's that get stored in a Link type. This ensures the program fd can always be closed when the link is disposed of + shouldn't have any weird side-effects around program lifecycle.

N.B: In all cases where we're calling try_clone_to_owned() we should add comments...
As an optimization later we might want to come back and replace with BorrowedFd instead - which would likely involve adding lifetimes per the PR description.

I took a shallower cut at this problem in https://github.com/aya-rs/aya/pull/744 and made ProgramFd borrowed from the program.

Hopefully this is simpler now that #744 is in.

I've fixed all the conflicts in the latest rebase but I am not sure what's the deal with the macos integration test nor the public api diff (I don't see that diff when I run it locally). Could someone help me get that resolved and then try to push to get this PR merged soon-ish? My tank for rebasing + merge conflicts is beginning to run low 😪

The public API diff just needs to be generated using nightly.

I'm fixing the mac failures in https://github.com/aya-rs/bpf-linker/pull/103.

@alessandrod @dave-tucker any other changes I need to do before merging this? I am trying to avoid further messy conflicts, these rebases are burning me out.


vle-assignmentgraded-template.json: incorrect success item

vle-assignmentgraded-template.json has:
"result": {
		"score": {
			"scaled": "**SCORE_SCALED**",
			"raw": "**SCORE_RAW**",
			"min": "**SCORE_MIN**",
			"max": "**SCORE_MAX**"
		},
		"success": false,
		"completion": true,
		"response": "**SCORE_RESPONSE**",
		"extensions": {
			"http://xapi.jisc.ac.uk/grade": "**GRADE**"
		}

I'm pretty sure that '"success": false' is wrong (if used, it should be a variable, not fixed). In any case:

it's deprecated in common structures
it's not in the TSV template.

I suggest just remove this from the JSON template, as it's potentially confusing.
We may have to leave it as deprecated in the common structures, as it may be used elsewhere.

+1 for removal
+1 for leaving deprecated

From call:
Remove


feat: (bb) 128-bit challenges

This PR modifies our Transcript class to achieve the following:
every time a hash function is used to generate challenges, the output is split into two 128-bit field elements to generate 2 challenges per hash
This change gives us the following benefits:

the amount of hashing required to fold/verifier proofs is reduced
where challenges map to Verifier scalar multiplications, those scalar muls are now half-width and can be more efficiently evaluated (requires additional code to support)

Looks mostly fine. Left some questions/comments. Also, does this actually provide much benefit vs just truncating challenges to 128 bits? (Edit: ah, maybe we try to generate all the alphas in a row so it cuts down hashing by almost 50% there..., but otherwise maybe not too much impact elsewhere...)

yeah it's mostly the alpha challenges. There are over 40 of them IIRC, so removing 20 hashes is nontrivial. There are a few other hashes it cuts out IIRC we estimated the saving could be up to 3k constraints

Benchmark results
Metrics with a significant change:

protocol_circuit_simulation_time_in_ms (private-kernel-tail-to-public): 760 (-45%)
avm_simulation_time_ms (Token:mint_public): 50.2 (-89%)
avm_simulation_time_ms (Token:assert_minter_and_mint): 454 (+55%)
l2_block_processing_time_in_ms (8): 508 (-23%)
node_history_sync_time_in_ms (5): 3,793 (-36%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Proof generation
Each column represents the number of threads used in proof generation.

Metric
1 threads
4 threads
16 threads
32 threads
64 threads

proof_construction_time_sha256_ms
5,764
1,556
713
776 (+3%)
780

proof_construction_time_sha256_30_ms
11,551 (+1%)
3,072
1,384
1,417 (-1%)
1,463

proof_construction_time_sha256_100_ms
44,062
11,765 (-1%)
5,823 (+3%)
6,012 (-1%)
5,644 (+5%)

proof_construction_time_poseidon_hash_ms
78.0
34.0
34.0
57.0 (-2%)
86.0 (-1%)

proof_construction_time_poseidon_hash_30_ms
1,530
422
202
223 (-1%)
270 (-1%)

proof_construction_time_poseidon_hash_100_ms
5,638
1,512
677
725 (-1%)
741 (-2%)

L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
4 txs
8 txs
16 txs

l1_rollup_calldata_size_in_bytes
4,356
7,876
14,884

l1_rollup_calldata_gas
50,244
92,896
178,072

l1_rollup_execution_gas
843,085
1,576,633
3,361,865

l2_block_processing_time_in_ms
237 (-7%)
:warning: 508 (-23%)
808 (+2%)

l2_block_building_time_in_ms
10,871 (+1%)
21,471 (+1%)
42,441

l2_block_rollup_simulation_time_in_ms
10,870 (+1%)
21,471 (+1%)
42,441

l2_block_public_tx_process_time_in_ms
9,278 (+1%)
19,835 (+1%)
40,793

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 8 txs.

Metric
3 blocks
5 blocks

node_history_sync_time_in_ms
3,845 (-8%)
:warning: 3,793 (-36%)

node_database_size_in_bytes
12,447,984
16,441,584

pxe_database_size_in_bytes
16,254
26,813

Circuits stats
Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit
simulation_time_in_ms
witness_generation_time_in_ms
input_size_in_bytes
output_size_in_bytes
proving_time_in_ms

private-kernel-init
71.9 (-5%)
378 (+1%)
21,373
44,898
N/A

private-kernel-inner
157 (+3%)
680 (+1%)
72,354
45,020
N/A

private-kernel-reset-tiny
276 (+2%)
665
65,018
44,886
N/A

private-kernel-tail
150 (+1%)
128 (-2%)
50,589
52,921
N/A

base-parity
5.55 (-1%)
N/A
160
96.0
N/A

root-parity
35.4 (+1%)
N/A
73,948
96.0
N/A

base-rollup
2,744 (+7%)
N/A
189,183
664
N/A

block-root-rollup
41.1
N/A
58,205
2,448
N/A

public-kernel-setup
78.7 (-1%)
N/A
105,085
71,222
N/A

public-kernel-app-logic
92.0
N/A
104,897
71,222
N/A

public-kernel-tail
804 (+1%)
N/A
390,582
16,414
N/A

private-kernel-reset-small
271 (+1%)
N/A
66,341
45,629
N/A

private-kernel-tail-to-public
:warning: 760 (-45%)
580 (+1%)
445,634
1,825
N/A

public-kernel-teardown
77.9 (-1%)
N/A
105,349
71,222
N/A

merge-rollup
20.2 (+1%)
N/A
38,174
664
N/A

undefined
N/A
N/A
N/A
N/A
88,094 (+4%)

Stats on running time collected for app circuits

Function
input_size_in_bytes
output_size_in_bytes
witness_generation_time_in_ms

ContractClassRegisterer:register
1,344
11,731
345 (+1%)

ContractInstanceDeployer:deploy
1,408
11,731
18.1 (+1%)

MultiCallEntrypoint:entrypoint
1,920
11,731
403

FeeJuice:deploy
1,376
11,731
394 (+3%)

SchnorrAccount:constructor
1,312
11,731
62.4

SchnorrAccount:entrypoint
2,336
11,731
369

FeeJuice:claim
1,344
11,731
34.6 (-5%)

Token:privately_mint_private_note
1,280
11,731
74.3 (+2%)

FPC:fee_entrypoint_public
1,344
11,731
26.1 (+9%)

Token:transfer
1,312
11,731
208 (-1%)

Benchmarking:create_note
1,344
11,731
74.7 (-1%)

SchnorrAccount:verify_private_authwit
1,280
11,731
26.3

Token:unshield
1,376
11,731
498

FPC:fee_entrypoint_private
1,376
11,731
656 (-3%)

AVM Simulation
Time to simulate various public functions in the AVM.

Function
time_ms
bytecode_size_in_bytes

FeeJuice:_increase_public_balance
45.6 (-9%)
2,307

FeeJuice:set_portal
15.0 (+10%)
2,680

Token:constructor
74.9 (-7%)
13,809

FPC:constructor
49.2
8,523

FeeJuice:check_balance
36.5 (+1%)
2,026

Token:mint_public
:warning: 50.2 (-89%)
4,557

Token:assert_minter_and_mint
:warning: 454 (+55%)
2,991

AuthRegistry:set_authorized
36.2 (-27%)
923

FPC:prepare_fee
71.6 (-3%)
3,925

Token:transfer_public
13.2 (-27%)
9,482

FPC:pay_refund
40.6 (-12%)
4,856

Benchmarking:increment_balance
1,164
3,170

Token:_increase_public_balance
38.7 (+3%)
2,284

FPC:pay_refund_with_shielded_rebate
51.1 (-1%)
4,856

Public DB Access
Time to access various public DBs.

Function
time_ms

get-nullifier-index
0.158 (-4%)

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
1 leaves
16 leaves
64 leaves
128 leaves
256 leaves
512 leaves
1024 leaves

batch_insert_into_append_only_tree_16_depth_ms
2.18
3.90
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_count
16.8
31.7
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_ms
0.113
0.110
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_32_depth_ms
N/A
N/A
11.1
17.9 (+2%)
31.2 (+2%)
59.0
114 (+2%)

batch_insert_into_append_only_tree_32_depth_hash_count
N/A
N/A
95.9
159
287
543
1,055

batch_insert_into_append_only_tree_32_depth_hash_ms
N/A
N/A
0.106
0.104 (+3%)
0.102 (+2%)
0.101
0.102 (+2%)

batch_insert_into_indexed_tree_20_depth_ms
N/A
N/A
14.3 (+1%)
25.7 (+2%)
43.9 (+2%)
82.3
163 (+3%)

batch_insert_into_indexed_tree_20_depth_hash_count
N/A
N/A
108
207
355
691
1,363

batch_insert_into_indexed_tree_20_depth_hash_ms
N/A
N/A
0.109 (+1%)
0.104 (+2%)
0.106 (+6%)
0.102
0.104 (+4%)

batch_insert_into_indexed_tree_40_depth_ms
N/A
N/A
16.4 (+1%)
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_count
N/A
N/A
132 (+1%)
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_ms
N/A
N/A
0.106
N/A
N/A
N/A
N/A

Miscellaneous
Transaction sizes based on how many contract classes are registered in the tx.

Metric
0 registered classes
1 registered classes

tx_size_in_bytes
64,838
668,997

Transaction size based on fee payment method
| Metric |  |
| - |  |


ci/e2e: fix issue with stable/latest

Should fix this issue: https://github.com/rancher/elemental/actions/runs/3266797237/jobs/5371069900, which was introduced by PR https://github.com/rancher/elemental/pull/426.
Verification run: https://github.com/ldevulder/elemental/actions/runs/3267658765.

Shouldn't this refer to Rancher 2.6.9-rc4 instead of 2.6.8 ?

:+1: