Implementation of dashboards within IoGT platform to visualize survey/poll/quiz data, built on Apache Superset

To be able to visualize individual surveys, polls, and quizzes.

Some user stories based on our discussions:

As an ADMIN I need to be able to use the existing permissions structure to allow users to create and access dashboards for specific forms

As a USER I need to be able to publicly share the dashboards that I create - by URL or ideally also embed

As an ADMIN I need to be able to give Groups permission to publicly share dashboards

As a USER I need a new Superset dashboard to be automatically populated with a basic set of visualizations based on data from the single form I select. The types of visualizations and types of data included can be informed by U-Report's dashboards, example here. The visual design can be taken from the private Power BI dashboard which was shared

As a USER I need to be able to adjust the automatically populated Superset dashboard within the limits of data I have permission for

Is it possible to set a Group of users in wagtail admin to be passed into superset
Superset to be set up as a single instance.

I sent a proposed SQL query which should work with Jinja to accomplish by-Page permissions based on Wagtail users and their Group membership, still a draft.

Admin users will need to be able to create a new dashboard for a questionnaire. The charts within the dashboard will need to be built up dynamically depending on the questions.

[ ] Dashboard to be created via a button on the results page - no restrictions to how many they can create for one questionnaire.
[ ] Create dashboard for Questionaire using /api/v1/dashboard/{pk}
[ ] Create chart for the each question in Questionaire using: /api/v1/chart/{pk}

The following charts will need to be created for each dashboard

[ ] Total submissions - Big number

[ ] Field Type: Checkbox, Checkboxes, drop down, radio buttons

Values are to be the percentage of answers to responders

All values to be added even if the value is 0

If only 2 answer options: pie chart

if more than 2 answer options: bar chart

[ ] Field Type: Number, Positive Number

Median value as big number

Extend to allow users to decide: Mean, Mode, Min, Max instead

Extend to display table showing distribution

[ ] Field Type: Date, Date/Time

Heat map calendar with sub-category being time series, month or day depending on range of answers

[ ] Field Type: Email, URL

Table with all answers and a number showing occurrences

[ ] Field Type: Single line text, multi line text

no chart

I have gone through superset APIs and are able to generate following

dashboards
datasets
charts

need the following to generate things mentioned above in superset

superset username and password for currently logged in wagtail user
database name used in superset for current site

the flow will be as follows:

user opens form data listing
click the generate dashboard button
it will redirect user to an intermediate page
user credentials

if no token is available or token and refresh tokens are expired then ask for credentials
if token is available and not expired then don't ask for credentials

user will adjust charts parameters (if necessary)
upon submission user will be redirected to newly generated dashboard on superset

rough sketch

I think we should divide this task into 2 iterations
1st iteration

no extend feature to change chart type and aggregation metric (defaults will be used)

2nd iteration

extend feature to change chart type and aggregation metric

CC: @cbunicef @istride

I think the first iteration suggestion is fine.
Thanks Sher for the proof of concept with the dashboard setup in Superset. We have one critical change we need to make - we can't have each question be a separate dataset. This is both for clutter reasons and because as it is the data cannot interact with each other, ie we cannot filter charts by submission date because each chart is a different dataset.
In terms of naming Datasources can we change
survey#1229
to
[site url]autodashboard[survey/poll/quiz/etc][page number][page title]_[date dashboard created in YYYY-MM-DD hh:mm:ss format]
eg
www.internetofgoodthings.org_autodashboard_survey_1229_All in survey_2022-01-01 00:00:00
And finally can we have a row at the top of the dashboards that has

a markdown box with this text: #[survey/poll/quiz/etc] - [page title]
the number of submissions chart

I've shown an example here:
https://superset-sandbox.iogt.site/superset/dashboard/p/N8B9LJ4dyLx/

@cbunicef

from your last comment I gather you want to add submission time filter on dashboards, right?
I was able to create all charts for a single survey from a single dataset except time-series chart for question type “Date/Datetime”
reason being superset uses “Time Column” to generate visualization and this same “Time Column” is used for filtering on dashboard too
for question type “Date/Datetime” I believe we need to generate time-series chart from user inputs of date/datetime, if we specify user inputs in “Time Column” then filtering on submission time can’t be applied
for reference I have created following for sandbox

survey
submissions
dataset
chart
dashboard

Very happy to hear that the single dataset was worked out. Let's talk more about the Time Column tomorrow.
I'd love to see an example of a dashboard with the other question types from a single Dataset source


Change sequencer_node_handle type

Stack:

#2636 ⬅
#2635
#2632

⚠ Part of a stack created by spr. Do not merge manually using the UI - doing so may have unexpected results.

This change is


[feature request] Allow queries to have wildcards and fields

This PR https://github.com/influxdb/influxdb/pull/3580 imposes a restriction to not allow queries as follows:
select *, foo from bar
select foo, * from bar
select *, * from bar
select * from bar group by time(1m), *

As the query engine does not currently support them.
We should add support for those types of queries and remove this restriction.

I opened a ticket about this months ago, but @pauldix closed it. I still think it's a good idea though.
https://github.com/influxdb/influxdb/issues/1634

This wildcard functionality is now supported in the new query engine so I'm going to close this.

Sorry to ask this here, but do we allow things like?
select *_field from bar


Cert-Manager (v0.15.1) Certificate generate error

kubectl apply -f cert.yaml
namespace/cert-manager-test unchanged
Error from server (InternalError): error when creating "cert.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=30s": x509: certificate is valid for k8s-pi-master, not cert-manager-webhook.cert-manager.svc
Error from server (InternalError): error when creating "cert.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=30s": x509: certificate is valid for k8s-pi-master, not cert-manager-webhook.cert-manager.svc
vinay@pramukha:~$ cat cert.yaml
apiVersion: v1
kind: Namespace
metadata:
name: cert-manager-test
apiVersion: cert-manager.io/v1alpha2
kind: Issuer
metadata:
name: test-selfsigned
namespace: cert-manager-test
spec:
selfSigned: {}
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: selfsigned-cert
namespace: cert-manager-test
spec:
commonName: example.com
secretName: selfsigned-cert-tls
issuerRef:
name: test-selfsigned
vinay@pramukha:~$
ubuntu@k8s-pi-master:/storage$ cat <<EOF | kubectl apply -f -

apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
email<EMAIL_ADDRESS>server: https://acme-staging-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-staging
solvers:
- http01:
ingress:
class: nginx
EOF
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post "https://cert-manager-webhook.kube-system.svc:443/mutate?timeout=30s": x509: certificate is valid for k8s-pi-master, not cert-manager-webhook.kube-system.svc
inspection-report-20200707_162032.tar.gz

Its strange i dont see the cert managers pod running.

ubuntu@k8s-pi-master:~$ kubectl get pods -A
NAMESPACE            NAME                                        READY   STATUS             RESTARTS   AGE
container-registry   registry-5d5b754549-m8zl6                   1/1     Running            2          33h
default              echo1-bfb568c8b-7fchh                       0/1     CrashLoopBackOff   287        24h
default              echo1-bfb568c8b-phc29                       0/1     CrashLoopBackOff   287        24h
default              echo2-5f44d9d965-hznqh                      0/1     ImagePullBackOff   0          24h
default              nginx-deployment-6b474476c4-5sm7t           1/1     Running            1          23h
default              nginx-deployment-6b474476c4-btzdq           1/1     Running            1          23h
ingress              nginx-ingress-microk8s-controller-247ws     1/1     Running            6          33h
ingress              nginx-ingress-microk8s-controller-8gzgf     1/1     Running            1          26h
ingress              nginx-ingress-microk8s-controller-qwtxr     1/1     Running            1          28h
kube-system          cert-manager-7747db9d88-7gjvf               1/1     Running            1          11m
kube-system          cert-manager-cainjector-87c85c6ff-pm8xf     1/1     Running            3          11m
kube-system          cert-manager-webhook-55fcfdfd7c-9gnc6       1/1     Running            1          11m
kube-system          coredns-588fd544bf-9nmds                    1/1     Running            2          33h
kube-system          dashboard-metrics-scraper-59f5574d4-wcrll   1/1     Running            1          24h
kube-system          hostpath-provisioner-9b7695c6b-x5p27        1/1     Running            2          33h
kube-system          kubernetes-dashboard-6d97855997-tmqmf       1/1     Running            1          24h
kube-system          metrics-server-79749d858b-bhzl6             1/1     Running            2          33h
kube-verify          kube-verify-5f976b5474-4vz5s                1/1     Running            2          33h
kube-verify          kube-verify-5f976b5474-7fg52                1/1     Running            2          33h
kube-verify          kube-verify-5f976b5474-tjl5v                1/1     Running            2          33h
kube-verify1         kube-verify-5f976b5474-6t7pv                1/1     Running            1          28h
kube-verify1         kube-verify-5f976b5474-7blz8                1/1     Running            1          28h
kube-verify1         kube-verify-5f976b5474-mprc4                1/1     Running            1          28h
metallb-system       controller-5f98465b6b-gd82s                 1/1     Running            2          33h
metallb-system       speaker-6gs2w                               1/1     Running            1          28h
metallb-system       speaker-cml76                               1/1     Running            1          26h
metallb-system       speaker-ncs2p                               1/1     Running            2          33h
monitoring           alertmanager-main-0                         2/2     Running            4          33h
monitoring           arm-exporter-522xf                          2/2     Running            2          26h
monitoring           arm-exporter-mcw44                          2/2     Running            4          33h
monitoring           arm-exporter-ptshf                          2/2     Running            2          28h
monitoring           grafana-676bcb5687-d6hl2                    1/1     Running            1          31h
monitoring           kube-state-metrics-96bf99844-6tlrq          3/3     Running            6          33h
monitoring           node-exporter-9gkhm                         2/2     Running            4          33h
monitoring           node-exporter-cjjgs                         2/2     Running            2          28h
monitoring           node-exporter-hbg8s                         2/2     Running            2          26h
monitoring           prometheus-adapter-f78c4f4ff-zvcr7          1/1     Running            2          33h
monitoring           prometheus-k8s-0                            3/3     Running            4          31h
monitoring           prometheus-operator-6b8868d698-pt9rw        2/2     Running            4          33h
monitoring           smtp-server-5c7c8d77f8-q592k                1/1     Running            2          33h
nextcloud            nextcloud-857cd47bdf-xrnm4                  1/1     Running            1          24h
ubuntu@k8s-pi-master:~$ cat <<EOF | kubectl apply -f -

apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
email<EMAIL_ADDRESS>server: https://acme-staging-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-staging
solvers:
- http01:
ingress:
class: nginx
EOF
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post "https://cert-manager-webhook.kube-system.svc:443/mutate?timeout=30s": x509: certificate is valid for k8s-pi-master, not cert-manager-webhook.kube-system.svc
ubuntu@k8s-pi-master:~$ cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
email<EMAIL_ADDRESS>server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- http01:
ingress:
class: nginx
EOF
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post "https://cert-manager-webhook.kube-system.svc:443/mutate?timeout=30s": x509: certificate is valid for k8s-pi-master, not cert-manager-webhook.kube-system.svc
ubuntu@k8s-pi-master:~$
inspection-report-20200708_154023.tar.gz

Please find the latest inspect report tarball @balchua - Thank you for the respnse and kindly look into this

Im totally unfamiliar with cert-manager, but google led me to this. https://github.com/jetstack/cert-manager/issues/2752#issuecomment-605883456
It seems to suggest using cert-manager namespace , instead of another can lead to better result.

vinay@pramukha:~$ kubectl get namespace
NAME                 STATUS   AGE
cert-manager         Active   9h
cert-manager-test    Active   2d1h
container-registry   Active   2d10h
default              Active   2d10h
ingress              Active   2d10h
kube-node-lease      Active   2d10h
kube-public          Active   2d10h
kube-system          Active   2d10h
kube-verify          Active   2d10h
kube-verify1         Active   2d5h
metallb-system       Active   2d10h
monitoring           Active   2d10h
nextcloud            Active   2d1h
vinay@pramukha:~$
vinay@pramukha:~$ kubectl get pods -A
NAMESPACE            NAME                                        READY   STATUS             RESTARTS   AGE
cert-manager         cert-manager-66bbb47c56-lh229               1/1     Running            0          9h
cert-manager         cert-manager-cainjector-5579468649-8jwvg    1/1     Running            0          9h
cert-manager         cert-manager-webhook-746cf468-xzb24         1/1     Running            0          9h
container-registry   registry-5d5b754549-m8zl6                   1/1     Running            2          2d10h
default              echo1-bfb568c8b-7fchh                       0/1     CrashLoopBackOff   575        2d
default              echo1-bfb568c8b-phc29                       0/1     CrashLoopBackOff   575        2d
default              echo2-5f44d9d965-hznqh                      0/1     ImagePullBackOff   0          2d
default              microbot-65bc8bdd7c-sgrd6                   1/1     Running            0          8h
default              microbot-65bc8bdd7c-zzdqt                   1/1     Running            0          8h
default              nginx-deployment-6b474476c4-5sm7t           1/1     Running            1          2d
default              nginx-deployment-6b474476c4-btzdq           1/1     Running            1          2d
ingress              nginx-ingress-microk8s-controller-247ws     1/1     Running            6          2d10h
ingress              nginx-ingress-microk8s-controller-8gzgf     1/1     Running            1          2d3h
ingress              nginx-ingress-microk8s-controller-qwtxr     1/1     Running            1          2d5h
kube-system          coredns-588fd544bf-9nmds                    1/1     Running            2          2d10h
kube-system          dashboard-metrics-scraper-59f5574d4-wcrll   1/1     Running            1          2d1h
kube-system          hostpath-provisioner-9b7695c6b-x5p27        1/1     Running            2          2d10h
kube-system          kubernetes-dashboard-6d97855997-tmqmf       1/1     Running            1          2d1h
kube-system          metrics-server-79749d858b-bhzl6             1/1     Running            2          2d10h
kube-verify          kube-verify-5f976b5474-4vz5s                1/1     Running            2          2d10h
kube-verify          kube-verify-5f976b5474-7fg52                1/1     Running            2          2d10h
kube-verify          kube-verify-5f976b5474-tjl5v                1/1     Running            2          2d10h
kube-verify1         kube-verify-5f976b5474-6t7pv                1/1     Running            1          2d5h
kube-verify1         kube-verify-5f976b5474-7blz8                1/1     Running            1          2d5h
kube-verify1         kube-verify-5f976b5474-mprc4                1/1     Running            1          2d5h
metallb-system       controller-5f98465b6b-gd82s                 1/1     Running            2          2d10h
metallb-system       speaker-6gs2w                               1/1     Running            1          2d5h
metallb-system       speaker-cml76                               1/1     Running            1          2d3h
metallb-system       speaker-ncs2p                               1/1     Running            2          2d10h
monitoring           alertmanager-main-0                         2/2     Running            4          2d10h
monitoring           arm-exporter-522xf                          2/2     Running            2          2d3h
monitoring           arm-exporter-mcw44                          2/2     Running            4          2d10h
monitoring           arm-exporter-ptshf                          2/2     Running            2          2d5h
monitoring           grafana-676bcb5687-d6hl2                    1/1     Running            1          2d7h
monitoring           kube-state-metrics-96bf99844-6tlrq          3/3     Running            6          2d10h
monitoring           node-exporter-9gkhm                         2/2     Running            4          2d10h
monitoring           node-exporter-cjjgs                         2/2     Running            2          2d5h
monitoring           node-exporter-hbg8s                         2/2     Running            2          2d3h
monitoring           prometheus-adapter-f78c4f4ff-zvcr7          1/1     Running            2          2d10h
monitoring           prometheus-k8s-0                            3/3     Running            4          2d7h
monitoring           prometheus-operator-6b8868d698-pt9rw        2/2     Running            4          2d10h
monitoring           smtp-server-5c7c8d77f8-q592k                1/1     Running            2          2d10h
nextcloud            nextcloud-857cd47bdf-xrnm4                  0/1     Running            2          2d1h
vinay@pramukha:~$ cat <<EOF | kubectl apply -f -

apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
email<EMAIL_ADDRESS>server: https://acme-staging-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-staging
solvers:
- http01:
ingress:
class: nginx
EOF
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=30s": x509: certificate is valid for k8s-pi-master, not cert-manager-webhook.cert-manager.svc
vinay@pramukha:~$

Issue still persists even after with dedicated namespace "cert-manager"

What does it look like when you do this?
kubectl get certificate -o wide --namespace cert-manager.
If you don't mind, can you use markdown code formatting? It is difficult to read them when its not in markdown code format.
Thanks.

`ubuntu@k8s-pi-master:~$ kubectl get namespace
NAME                 STATUS   AGE
cert-manager         Active   18h
cert-manager-test    Active   2d10h
container-registry   Active   2d19h
default              Active   2d20h
ingress              Active   2d19h
kube-node-lease      Active   2d20h
kube-public          Active   2d20h
kube-system          Active   2d20h
kube-verify          Active   2d19h
kube-verify1         Active   2d15h
metallb-system       Active   2d19h
monitoring           Active   2d19h
nextcloud            Active   2d11h

ubuntu@k8s-pi-master:~$ kubectl get certificate -o wide --namespace cert-manager

No resources found in cert-manager namespace.

ubuntu@k8s-pi-master:~$ kubectl get certificate -o wide --namespace kube-system

No resources found in kube-system namespace.

ubuntu@k8s-pi-master:~$ kubectl get certificate -o wide -A

No resources found

ubuntu@k8s-pi-master:~$
`
There are no certificates in cert-manager and all other namespaces.

That could be the reason why the apiserver couldn't talk to the cert manager webhook.
I think its best to check with the cert-manager project team.


Question: Will Python Dependency Injection framework be supported?

Is it possible to add support for https://python-dependency-injector.ets-labs.org/ ?
Or maybe there are some concerns about why it is not used?

Hello!
It is definitely possible to implement. Will try to add to the upcoming release.

Hello! It is definitely possible to implement. Will try to add to the upcoming release.

Thanks! Great!
Also, have a question: Is diator is async only because of di package? Because I do not see any async operations except of resolving dependencies via async executor in di package. Is it possible to port diator to async mode?
If there is something I can do, please, just tell me.

Also, have a few more questions not related to this topic, just do not wanna create many issues. Is there any road map for development? Very interesting the content of the next releases. There are interesting components (like NotificationEvent) which are cannot be used now (if I clearly understood everything from docs and code), and it would be nice to know when they are planned to be supported.

Sure, you can read CONTRIBUTING.MD to understand how to setup and modify project.
All types of events are fully supported and ready to use. Is there any particular issue you faced?

Yeah, got typing issue during creating handler for NotificationEvent. Mypy throws an error, because of generic of EventHandler event type is bound to DomainEvent, while NotificationEvent is just Event. The same for ECSTEvent.
EventHandler requires only DomaintEvent based events.

Unfortunately, there are several places where mypy throws this kind of error. Thanks for sharing, I will include it to the next release.
Planning to start work on it on the weekends!

Sorry, I read your comment second time.
EventHandler accepts only DomainEvent because it is designed in order to handle only business logic. Whereas, notification and ECST events are sent asynchronously to the message broker. Afterwards, it could be handled by other services (consumers).
Hope I described properly :)

Oh, yeah, got it! Thanks for the explanation.

Also, could you please answer this one - https://github.com/akhundMurad/diator/issues/26#issuecomment-1708943177

Diator is async only. Probably, in the far future it will have support of the sync mode


Add ZeroOp as neutral elements of addition, make IdentityOp neutral element of composition

Add ZeroOp (returns zeros)
Makes ZeroOp neutral element of OperatorAddion, IdentityOp of composition, i.e.
Op + zeroop = op
op@identityop = op

I am considering changing the default of the ZeroOp to always return a scalqr integer 0 in forward and adjoint.
This is the most useful form of the operator ( as a neutral element of addition), for example in the matrix operator. On its own, it does not pass the adjointess test, but ZeroOp+any linearoperator does.
I cannot think of a use case where retuning a scalar tensor does not work.

I am considering changing the default of the ZeroOp to always return a scalar integer 0 in forward and adjoint.
This is the most useful form of the operator ( as a neutral element of addition), for example in the matrix operator. On its own, it does not pass the adjointess test, but ZeroOp+any linearoperator does.
I cannot think of a use case where retuning a scalar tensor does not work.
And whenever somebody needs a zeroop, they actually need this broadcasting form..

@ckolbPTB @schuenke

I am considering changing the default of the ZeroOp to always return a scalar integer 0 in forward and adjoint.

sounds reasonable to me


WatchStream Stream error: 'Unavailable: Network connectivity changed'

When login into iCloud account, the final step fails with:
WatchStream (6000006dc718) Stream error: 'Unavailable: Network connectivity changed'
Sign in with Apple errored: The operation couldn’t be completed. (com.apple.AuthenticationServices.AuthorizationError error 1001.)
This is from base code on this commit.

Can you provide more details? Did you run this on the Simulator or on a device? Which versions of Xcode, iOS were you using? If possible, share a stack trace and/or log output, as well as exact steps to repro.


VLDC Code of Conduct

@C-Pro I believe you know much more about this.

[ ] main points

[ ] ru
[ ] en

[ ] md
[ ] raw txt for posting CoC by bot on demand (or me may just post the link to GitHub page with CoC

I think we can share it with GDG Vladivostok:
https://devfest.gdgvl.ru/code-of-conduct/
Markdown source en/ru:
https://github.com/C-Pro/site-devfest-2019/blob/master/content/code-of-conduct.md
https://github.com/C-Pro/site-devfest-2019/blob/master/content/code-of-conduct.ru.md

But I think we should have short one for the bot with a link to fullsize COC EULA  MEGA PRO XXL MAX

GDG COC is good enough

Any progress?


trace-dispatcher / node: metric registration clash

We see the dreaded ExceptionInLinkedThread "ThreadId 33" The name ""slotsMissed"" is already taken by a metric. once more.
This time found by @fmaste, on a recent master, while running inside Docker:
node-4    | cardano-node: ExceptionInLinkedThread "ThreadId 33" The name ""slotsMissed"" is already taken by a metric.
node-4    | CallStack (from HasCallStack):
node-4    |   error, called at ./System/Metrics.hs:214:5 in ekg-core-<IP_ADDRESS>-LyPC4OzssyMIbAlQK2bJI4:System.Metrics

We have to clarify if the code base used is the current version. The name "slotMissed" is not used any more for a metric, and the general problem should have been fixed in this patch https://github.com/input-output-hk/cardano-node/commit/ad316eabbe8da3b2ab72085387f835461c9e9f04

I found out that commit https://github.com/input-output-hk/cardano-node/commit/ad316eabbe8da3b2ab72085387f835461c9e9f04 which was merged into master on 28. of June is missing for Release 1.35 branch.

Already fixed.


Resizing an ANF volume via azure CLI and REST API doesn't work. Documentation is wrong.

There's no command in the azure cli or REST documentation to resize a volume.
We need to either update the documentation with a real example, or we should change the documentation to say it's not supported.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 4dbc9649-08ca-0d28-7c17-c096cd9b58eb
Version Independent ID: ee93ce1a-469e-405a-74ed-0a04fd0917f7
Content: Resize the capacity pool or a volume for Azure NetApp Files
Content Source: articles/azure-netapp-files/azure-netapp-files-resize-capacity-pools-or-volumes.md
Service: azure-netapp-files
GitHub Login: @b-hchen
Microsoft Alias: anfdocs

@matt-chan
Thanks for your feedback! We will investigate and update as appropriate.


ThreadedWebsocketManager start_futures_socket never uses testnet url

Firstly thanks for the good work.
I noticed the ThreadedWebsocketManager::start_futures_socket never sets the futures testnet url for receiving user data.
This is because it calls just futures_socket  in the BinanceSocketManager which just defaults to FSTREAM_URL  (rather than switch to FSTREAM_TESTNET_URL)
   def futures_socket(self):
        """Start a websocket for futures data

            https://binance-docs.github.io/apidocs/futures/en/#websocket-market-streams

        :returns: connection key string if successful, False otherwise

        Message Format - see Binance API docs for all types
        """
        return self._get_account_socket('futures', stream_url=self.FSTREAM_URL)

It would be nice if it automatically selected the testnet if specified in the client.
I've hacked mine work it to work, by adding new start_testnet_socket / futures_testnet_socket functions to the ThreadedWebsocketManager / ThreadedWebsocketManager, but it's not pretty.
Seems like it should really something along the lines of the  _get_futures_socket to get the correct url.
Thanks, Jamie

It's easy to solved this issued, just added:
        if self.testnet:
            return self._get_account_socket('futures', stream_url=self.FSTREAM_TESTNET_URL)

before the line:
        return self._get_account_socket('futures', stream_url=self.FSTREAM_URL)

It's easy to solved this issue:
IN steams.py::: def futures_socket(self)
add:
        if self.testnet:
            return self._get_account_socket('futures', stream_url=self.FSTREAM_TESTNET_URL)

before the line:
return self._get_account_socket('futures', stream_url=self.FSTREAM_URL)


Carthage-Issues running for iOS

Hi there,
I am running into the problem that
react-native run-ios
leads to
Failed to read file or folder at [...]/node_modules/react-native-ble-plx/ios/BleClientManager/Carthage/Build/iOS/BleClientManager.framework Command PhaseScriptExecution failed with a nonzero exit code
I have carthage installed with brew and it's accessible.
I tried the following:

rm node-modules -> npm install
react-native unlink react-native-ble-plx -> react-native link react-native-ble-plx

Nothing helped.
My dependencies within the package.json looks like this:
  "dependencies": {
    "react-native-ble-plx": "^1.1.0",
    "base64-arraybuffer": "^0.1.5",
    "react": "16.8.3",
    "react-native": "0.59.10",
    "react-native-gps-state": "^2.0.3",
    "react-native-modal-overlay": "^1.0.1",
    "react-native-svg": "^9.13.3",
    "react-native-timer": "^1.3.6",
    "styled-components": "^3.3.3"
  }

The directory
[...]/node_modules/react-native-ble-plx/ios/BleClientManager
never contains any Carthage directory.
What might be the problem? How can I solve this? BTW: For Android everything is working fine.

Carthage is not used since 0.10.0. Check the changelog and new integration instructions


Improve warning message on mismatch between framework/compiler versions

The previous message did not reveal what the mismatch was and gave the wrong recommendation if the installed framework/compiler version was lower than the dependencies stated in the Manifest.

Thanks @cboulanger !


Pass location of kernel's Cargo.toml to bootloader

As discussed in #44.

Published as version 0.7.7


[C#] Value of inherited export variable of a 'Tool' resets every recompile

Godot version
Godot_v3.3.3-stable_mono_win64
System information
Windows 10
Issue description
I want to make two C# classes: S1 and S2, and make S2 inherits S1.
S1 has an export variable S1_Int, and S2 has an export variable S2_Int (and also S1_Int that is inherited from S1).
If I add the [Tool] attribute to S1 and S2, an issue arise, when I try to change the value of the export variable S1_Int in S2 (that was inherited from S1) and then compile the C# code, the value of S1_Int resets to the default value.
Example with pictures:
The scene Level.tscn contains S1 and S2:

1)At first, S1_Int and S2_Int have their default values (in the example 5 and 20):

2)Change S1_Int to 6, and S2_Int to 21:

3)Make "Some change" in a cs file to force a recompile:

4)Run the project, and notice that S1_Int resets while S2_Int stays with the same value:

For some reason, this issue only happens if the scene that contains S1 and S2 is opened, if it's not the value won't reset.
Also, if after the change I close and open Level.tscn again, S1_Int will contain the right value.
Moreover, if I print the values using GD-Print, in the first run the right value will be printed, however in the second run the default value will be printed.
Steps to reproduce
1)Create 2d Project using C# with the mono version.
2)Create 2 scenes, S1 and S2 where S2 inherits S1.
3)Add a protected export variable to S1, and compile the project (in order to view the variable in S2).
4)Change the default value of this variable.
5)Make a small change in the C# code (to force a recompile), and run the project.
6)The value of the inherited variable will reset.
Minimal reproduction project
ToolBugTest.zip

I suppose I can confirm that on linux version of 3.3.3. mono x64(kinda, my bug looks like alot like this but is simpler in a way):
After closing and reopening of the project all of the overrided exported variables are "reverted" to default values in the start scene. But closing the scene and opening it again and all the overrided values are intact.

Duplicate of https://github.com/godotengine/godot/issues/37812


pullRequest resource uploading step failing

Expected Behavior
upload step should correctly discover any required information from the given /workspace/<resource>/ folder and successfully update the given PR.
Actual Behavior
following error occurs:
{"level":"info","ts":1575932806.2572246,"logger":"fallback-logger","caller":"logging/config.go:69","msg":"Fetch GitHub commit ID from kodata failed: open /var/run/ko/refs/heads/pr: no such file or directory"}
{"level":"info","ts":1575932806.257303,"logger":"fallback-logger","caller":"pullrequest-init/main.go:56","msg":"RUNNING UPLOAD!"}
{"level":"info","ts":1575932806.2580793,"logger":"fallback-logger","caller":"pullrequest-init/api.go:107","msg":"Syncing resource: &{PR:0xc000212000 Status:0xc000224420 Comments:[] Manifests:map[comments:map[] labels:map[]]}"}
{"level":"info","ts":1575932806.5881653,"logger":"fallback-logger","caller":"pullrequest-init/api.go:144","msg":"Current labels: map[]"}{"level":"info","ts":1575932806.5882623,"logger":"fallback-logger","caller":"pullrequest-init/api.go:155","msg":"Creating labels [] for PR 10"}{"level":"info","ts":1575932806.8430235,"logger":"fallback-logger","caller":"pullrequest-init/api.go:177","msg":"Setting comments for PR 10 to: []"}{"level":"fatal","ts":1575932807.101696,"logger":"fallback-logger","caller":"pullrequest-init/main.go:62","msg":"1 error occurred:\n\t* Ref not found\n\n","stacktrace":"main.main\n\t/workspace/go/src/github.com/tektoncd/pipeline/cmd/pullrequest-init/main.go:62\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:200"}

Steps to Reproduce the Problem

create a pullRequest resource
use pullRequest resource as both input and output resource

Additional Info

download step is successful to populate the `/workspace// directory
tekton v0.9.1

/kind bug

It'd be nice if this could open a PR in addition to updating existing ones.

/milestone Pipelines 0.9.2

This should be fixed now at head, at in v0.9.2. If you find this not to be the case, please reopen with repro steps.

@castlemilk are you able to verify that this fixes your issue? (not to alarm you but I'm asking b/c I saw some odd behaviour when I was trying to use this recently, hoping it was just me 🤞 )

@bobcatfish this looks to be working for me now on v0.9.2

Glad to hear it!! Thanks @castlemilk :D


Why does MinIO not support overwriting the Response Header value when calling GetObject?

MinIO's GetObject Option only supports passing in Request Headers, not response-header parameters to override Response Headers, whereas Amazon S3's documentation indicates that such parameters are supported.
MinIO's GetObject Documentation:

Amazon S3 Documentation:
https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html

Request Syntax:
GET /Key+?partNumber=PartNumber&response-cache-control=ResponseCacheControl&response-content-disposition=ResponseContentDisposition&response-content-encoding=ResponseContentEncoding&response-content-language=ResponseContentLanguage&response-content-type=ResponseContentType&response-expires=ResponseExpires&versionId=VersionId HTTP/1.1

Now, let's look into the code of MinIO Go SDK, we can find that only the parameters of versionId and PartNumber are supported while building HTTP Request, while other parameters are not supported.

What considerations are behind this? When will this feature be supported?
MinIO Go SDK Version: v7.0.45

@reedchan7 Seems like we have an xy problem
If you describe what you are trying to achieve, instead of a solution to an unknown problem it would be a lot easier to evaluate.
response-content-type, response-content-language, response-content-disposition, response-content-encoding does not affect anything, since it would only make sense in a browser. You are receiving data back as a stream, so none of these matter.
If you are concerned about the headers returned, why not just ignore them, instead of manually overriding?

@klauspost
We will provide services both on the command-line terminal and the web browser, so even though these Headers may not be effective on the command-line terminal, they are still useful on the browser, including Expires and Content-Disposition, etc.
Moreover, we have connected to several object storage services, such as Aliyun OSS, Tencent Cloud COS, and MinIO, we may extend to Amazon S3 and others. Therefore, we have abstracted the APIs of these services, however, this API behaves differently, hence the confusion above.
However, I don't think this is a big problem now. If we have the requirement, I have forked a copy and implemented the coverage of these response-* headers. Later, if we think these headers are not important anymore, we can remove the related logic and switch back to your official implementation.
Thank you for your reply and assistance!

Response changes for standard header types are provided via presigned URLs for GetObject this has been supported since day 1.
Use presignedGet API for what you want.

@reedchan7 Makes sense. I don't see any issue with implementing them. They can easily be added without breaking compatibility.
The easiest would probably be to just add them as supportedHeaders
Then (*GetObjectOptions).Set("Response-Content-Type", "text/plain") can be used to set it. You are welcome to send a PR that adds those fields.
@harshavardhana This is something else.

@klauspost I did this by adding a reqParams to GetObjectOptions and then adding some methods, similar to the presignedGet API. Is this okay? Support passing in a URL query string in GetObjectOptions

@klauspost I did this by adding a reqParams to GetObjectOptions and then adding some methods, similar to the presignedGet API. Is this okay? Support passing in a URL query string in GetObjectOptions

@reedchan7 that is the correct way to do it.

@klauspost @harshavardhana Do you need a PR?

@reedchan7 Yes, though un-export ToQueryValues and remove the pointer change on func (o *GetObjectOptions) Header() - that is a potentially breaking change.

@klauspost @harshavardhana Here is the PR: https://github.com/minio/minio-go/pull/1756
I set ToQueryValues to private, and rollback func (o *GetObjectOptions) Header() to func (o GetObjectOptions) Header()  (Although this would warn: "Struct Client has methods on both value and pointer receivers. Such usage is not recommended by the Go Documentation.").
I add a supportedQueryValues just like your supportedHeaders.
When SetReqParam or AddReqParam method is invoked, the isStandardQueryValue function would be called to check if the key passed in by the user is supported.
And then I wrote an example.
Please review, thanks.


Crash in Architectury Dev Environment

Describe the bug
Crash on running forge:runClient
Steps to reproduce

...
Logs
https://gist.github.com/ReMelon/223a64f88f38a9be08fcd767c5c9bd8f
Additional context
Im developing a mod in architectury and depending on forgified fabric api for forge
Forge Version is 47.1.3

Please provide the necessary reproduction steps

Please provide the necessary reproduction steps

Sorry, I think seeing the code would help more https://github.com/ReMelon/crystallized


After merging the change constifying EVP_PKEY_get0 return value external gost test fails

The gost engine uses EVP_PKEY_get0*() functions and does not expect the return value being const. This makes the external tests build to fail.
A temporary workaround could be to make the gost engine build without -Werror. Otherwise this needs fixing in the gost-engine code.

@beldmit Would you please look at this on the gost-engine side?

@t8m, will do.

Example of the failure:
https://github.com/openssl/openssl/runs/2057812906

No, switching of the -Werror makes non-working build :(

Oh. It looks like we've got to the situation I tried to avoid as much as possible.
I'd suggest temporary switch off the gost tests until I deal with this stuff. I'm afraid there is no relevant API to replace it for the GOST engine.

It sounds real mysterious with a build that breaks when you remove -Werror...

The build doesn't, but the tests do

I am curious why this breaks. Shouldn't the Gost engine work with legacy keys only?

2021 19:05, 19:05, Dmitry Belyavskiy<EMAIL_ADDRESS>napsal/a:

Oh. It looks like we've got to the situation I tried to avoid as much
as possible.
I'd suggest temporary switch off the gost tests until I deal with this
stuff. I'm afraid there is no relevant API to replace it for the GOST
engine.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/openssl/openssl/issues/14466#issuecomment-792957747

It should. I didn't investigate yet, but it may fall because of trying to modify the constant EVP_PKEY object

If the EVP_PKEY object has legacy origin it shouldn't be truly const. I.e., the key should be update-able although the get0 call returns it as const. If we broke that somehow, we need to fix that. @mattcaswell ?

2021 20:02, 20:02, Dmitry Belyavskiy<EMAIL_ADDRESS>napsal/a:

It should. I didn't investigate yet, but it may fall because of trying
to modify the constant EVP_PKEY object
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/openssl/openssl/issues/14466#issuecomment-792995461

If the EVP_PKEY object has legacy origin it shouldn't be truly const.

Right...if legacy origin then it shouldn't be a problem.

I'm thinking we could even provide a new function EVP_PKEY_get0_legacy() that would return a non-const pointer but would return NULL if the key doesn't have legacy origin.

2021 20:09, 20:09, mattcaswell<EMAIL_ADDRESS>napsal/a:

If the EVP_PKEY object has legacy origin it shouldn't be truly const.

Right...if legacy origin then it shouldn't be a problem.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/openssl/openssl/issues/14466#issuecomment-793001082

Or just use the get1 functions which are already non-const.

Or....actually maybe there isn't an EVP_PKEY_get1....'cos its void *...:-(

Or just use the get1 functions which are already non-const.

It was the first idea but I didn't find it immediately

Or just use the get1 functions which are already non-const.

Hmmm, except that functionality that's part of an EVP_PKEY_ASN1_METHOD must be able to manipulate the internal legacy key, by design.  And that's exactly what an engine does.

Functions that are designed for EVP_PKEY_ASN1_METHOD will have to unconst the return value of EVP_PKEY_get0()

An idea is to make EVP_PKEY_get0() return a non-const value, which the type specific counterparts (EVP_PKEY_get0_RSA() et al) return const values, with the argument that EVP_PKEY_get0() is more likely to be used from EVP_PKEY_METHOD and EVP_PKEY_ASN1_METHOD functionality.

Yes, it's a great idea

Then we should also make it to return NULL if the key origin is not legacy.

2021 20:38, 20:38, Richard Levitte<EMAIL_ADDRESS>napsal/a:

An idea is to make EVP_PKEY_get0() return a non-const value, which
the type specific counterparts (EVP_PKEY_get0_RSA() et al) return const
values, with the argument that EVP_PKEY_get0() is more likely to be
used from EVP_PKEY_METHOD and EVP_PKEY_ASN1_METHOD functionality.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/openssl/openssl/issues/14466#issuecomment-793019476

Then we should also make it to return NULL if the key origin is not legacy.

Sure


{locked: false} body in HEAD requests

The HEAD request seem to now have a body, with the value {locked: false}, and this shouldn't be the case, as HEAD requests should not have a body.
This issue causes this error:
Error while running request handler: Request constructor: HEAD or GET Request cannot have a body.

when trying to re-use a request, like in the case of rewrites: https://developer.fastly.com/learning/compute/migrate/#modify-a-request-url-path
ℹ This issue is reproducible in v0.7.0 and v1.0.0, everything seems to work fine in v0.6.0 and below.
An example of a problematic HEAD request:

the same request but with<EMAIL_ADDRESS>
You can see that the body property in the first screenshot has a value of {locked: false} while it should be null.

Thank you for reporting this @ilyasmez - I've been able to replicate the issue and know where the bug was introduced, I'm working on resolving this now 👍

Awesome! Thank you @JakeChampion.


Tenant Deploy fails with Unable to retrieve Arn attribute for AWS::Cognito::UserPool

while deploying the SIF,
SIF Deployment, i'm getting this error
Running this command fails with missing permission on CongnitoUserPool
npm run cdk -- deploy \  -c tenantId=<tenantId> \  -c environment=<environment> \  -c administratorEmail=<administratorEmail> \  -c enableDeleteResource=true \  -c includeCaml=true \  -c deleteBucket=true \  -r arn:aws:iam::<ACCOUNT ID>:role/sif-deploy-role  --all --require-approval never --concurrency=10
Error Details:
The error indicates that the sif-deploy-role is not authorized to perform the cognito-idp:DescribeUserPool action on the specified Cognito User Pool.
This might be due to missing permissions in the IAM policy attached to the sif-deploy-role
The complete error message is as follow:
Stack deploy failed (the stack disappeared while we were deploying it): Unable to retrieve Arn attribute for AWS::Cognito::UserPool, with error message User: arn:aws:sts::<ACCOUNT ID>:assumed-role/sif-deploy-role/AWSCloudFormation is not authorized to perform: cognito-idp:DescribeUserPool on resource: arn:aws:cognito-idp:us-east-1:<ACCOUNT ID>:userpool/us-east-1_excjjgI46 because no identity-based policy allows the cognito-idp:DescribeUserPool action (Service: CognitoIdentityProvider, Status Code: 400, Request ID: fc4aa047-561e-4154-8c36-1c4d9fe5e778), Unable to retrieve Arn attribute for AWS::Cognito::UserPool, with error message User: arn:aws:sts::<ACCOUNT ID>:assumed-role/sif-deploy-role/AWSCloudFormation is not authorized to perform: cognito-idp:DescribeUserPool on resource: arn:aws:cognito-idp:us-east-1:<ACCOUNT ID>:userpool/us-east-1_excjjgI46 because no identity-based policy allows the cognito-idp:DescribeUserPool action (Service: CognitoIdentityProvider, Status Code: 400, Request ID: 334a5872-4282-4c2d-a5bf-fe9e578ffc33)
I have created the IAM Role using

Apologies, we will update this in a future release. The role is missing the following permissions:
"cognito-idp:DescribeUserPool",
"cognito-idp:AdminGetUser",
"cognito-idp:AdminListGroupsForUser"

You will also need to add the following LakeFormation policy to the deploy role:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": "lakeformation:*",
      "Resource": "*"
    }
  ]
}

now i'm getting this error,
 ❌ Deployment failed: Error [ValidationError]: Unable to fetch parameters [/sif/shared/dev/caml/inferenceEndpointName] from parameter store for this account.     at Request.extractError (/Users/satishvanga/Desktop/data-fabric/guidance-for-aws-sustainability-insights-framework-main/common/temp/node_modules/.pnpm/aws-cdk@2.87.0/node_modules/aws-cdk/lib/index.js:307:46430)     at Request.callListeners (/Users/satishvanga/Desktop/data-fabric/guidance-for-aws-sustainability-insights-framework-main/common/temp/node_modules/.pnpm/aws-cdk@2.87.0/node_modules/aws-cdk/lib/index.js:307:89761)     at Request.emit (/Users/satishvanga/Desktop/data-fabric/guidance-for-aws-sustainability-insights-framework-main/common/temp/node_modules/.pnpm/aws-cdk@2.87.0/node_modules/aws-cdk/lib/index.js:307:89209)     at Request.emit (/Users/satishvanga/Desktop/data-fabric/guidance-for-aws-sustainability-insights-framework-main/common/temp/node_modules/.pnpm/aws-cdk@2.87.0/node_modules/aws-cdk/lib/index.js:307:195751)     at Request.transition (/Users/satishvanga/Desktop/data-fabric/guidance-for-aws-sustainability-insights-framework-main/common/temp/node_modules/.pnpm/aws-cdk@2.87.0/node_modules/aws-cdk/lib/index.js:307:189303)     at AcceptorStateMachine.runTo (/Users/satishvanga/Desktop/data-fabric/guidance-for-aws-sustainability-insights-framework-main/common/temp/node_modules/.pnpm/aws-cdk@2.87.0/node_modules/aws-cdk/lib/index.js:307:154175)     at /Users/satishvanga/Desktop/data-fabric/guidance-for-aws-sustainability-insights-framework-main/common/temp/node_modules/.pnpm/aws-cdk@2.87.0/node_modules/aws-cdk/lib/index.js:307:154505     at Request.<anonymous> (/Users/satishvanga/Desktop/data-fabric/guidance-for-aws-sustainability-insights-framework-main/common/temp/node_modules/.pnpm/aws-cdk@2.87.0/node_modules/aws-cdk/lib/index.js:307:189595)     at Request.<anonymous> (/Users/satishvanga/Desktop/data-fabric/guidance-for-aws-sustainability-insights-framework-main/common/temp/node_modules/.pnpm/aws-cdk@2.87.0/node_modules/aws-cdk/lib/index.js:307:195826)     at Request.callListeners (/Users/satishvanga/Desktop/data-fabric/guidance-for-aws-sustainability-insights-framework-main/common/temp/node_modules/.pnpm/aws-cdk@2.87.0/node_modules/aws-cdk/lib/index.js:307:89929) {   code: 'ValidationError',   time: 2024-10-03T21:52:18.951Z,   requestId: '4dbb41dd-64c7-44b6-82e1-c628f0b982d3',   statusCode: 400,   retryable: false,   retryDelay:<PHONE_NUMBER>815742 }
Unable to fetch parameters [/sif/shared/dev/caml/inferenceEndpointName] from parameter store for this account.

Hello. I see above that you deployed the tenant stack with the includeCaml=true context variable. Can you verify you also set this variable when deploying the platform stack? This is where the CaML endpoint is created.

sorry, I didn't, can I deploy with it again (includeCaml=true)

but, here you are saying like Note: This only needs to be run once per environment.

Hello. Sorry for the confusion. CaML is a feature of SIF for using ML to match products to their EIO (economic input-output) category for Scope 3 emissions calculations. The CaML feature is used by SDF when SDF is deployed (see demo here).
This feature in SDF supports a demo of CaML. If you do not plan to demonstrate or use this feature you can skip deploying SDF.
The CaML SageMaker endpoint is a shared resource that is deployed when the SIF platform stack is created. You can re-deploy the SIF platform stack and include the CaML flag, repeat the tenant SIF deployment with the CaML flag, and then deploy SDF to use that feature. You do not need to tear down any of the stacks - just re-deploy the CDK.

while trying to re-deploy without deleting the existing stack, it is give this
No stacks match the name(s)
npm run cdk -- deploy \     -c environment=dev \     -c deleteBucket=true \     -c clusterDeletionProtection=false \     -c includeCaml=true \         -r arn:aws:iam::<account_id>:role/sif-deploy-role     --all


Update TrustFrameworkKeySet Policy Keys Document for keys_v2

In TrustFrameworkPolicies, secrets are stored which allow the framework to access different services on behalf of the application. Introducing new functionality using keys_v2 to support entity type operations like get keys and patch the key status for a key within keyset.

@gyanendersinghgithub , FYI, this is the public repo for external contributions. For new API docs, please use the private authoring repo so that you can follow our workflow: https://github.com/microsoftgraph/microsoft-graph-docs/pulls.
For more information, see the Get started topic: https://dev.azure.com/msazure/One/_wiki/wikis/Microsoft Graph Partners/614277/Get-started


Stack overflow for recursive grammar

I am getting stack overflow for next grammar:
clause
    : clause AND clause
    | action
    ;

action  : 'action' ;

AND : 'AND' ;

with action AND  input.
Debug info:
TOKENS FOUND IN FIRST PASS:
[@-1,0:5='action',<1>,1:0]
[@-1,7:9='AND',<2>,1:7]
UNTOKENIZED:
Parser rule names: clause, action
  State: 0 (type: RuleStartState)
    State: 4 (type: BasicState)
      State: 5 (type: BasicState)
        State: 2 (type: RuleStartState)
          State: 15 (type: BasicState)
            State: 16 (type: BasicState)
              State: 3 (type: RuleStopState)
                State: 6 (type: BasicState)
                  State: 12 (type: StarLoopEntryState)
                    State: 10 (type: StarBlockStartState)
                      State: 7 (type: BasicState)
                        State: 8 (type: BasicState)
                          State: 9 (type: BasicState)
Suggesting tokens for rule numbers: 1
SUGGEST: tokenSoFar= remainingText=  lexerState=1
SUGGEST: tokenSoFar= remainingText=  lexerState=5
NONMATCHING LEXER TOKEN: a remaining=
                    State: 13 (type: LoopEndState)
                      State: 1 (type: RuleStopState)
                        State: 11 (type: BlockEndState)
                          State: 14 (type: StarLoopbackState)
                            State: 12 (type: StarLoopEntryState)
                              State: 10 (type: StarBlockStartState)
                                State: 7 (type: BasicState)
                                  State: 8 (type: BasicState)
                                    State: 9 (type: BasicState)
Suggesting tokens for rule numbers: 1
SUGGEST: tokenSoFar= remainingText=  lexerState=1
SUGGEST: tokenSoFar= remainingText=  lexerState=5
NONMATCHING LEXER TOKEN: a remaining=
                              State: 13 (type: LoopEndState)
                                State: 1 (type: RuleStopState)
                                  State: 11 (type: BlockEndState)
                                    State: 14 (type: StarLoopbackState)
                                      State: 12 (type: StarLoopEntryState)
                                        State: 10 (type: StarBlockStartState)
                                          State: 7 (type: BasicState)
                                            State: 8 (type: BasicState)
                                              State: 9 (type: BasicState)
...

Fix is in the newly-built version 0.0.28


netdev/ipv6: Move xxx_ipv6multicast from arch to common code

Summary
The xxx_ipv6multicast function in each driver is not adapted to multiple IPv6 addresses yet, and they're redundant, so try to take them into common code.
Change:

Add MAC g_ipv6_ethallnodes and g_ipv6_ethallrouters in netdev_register
Add multicast MAC for Neighbor Solicitation when adding any IPv6 address, and remove them when IPv6 address is removed
Select NET_MCASTGROUP when NET_ICMPv6 because now we need d_addmac when we have ICMPv6

Note:
We want modules outside net stack to call functions like netdev_ipv6_add and never touch the related MAC address, so these MAC functions are added as internal functions to net/netdev/netdev.h
Impact
d_addmac & d_rmmac logic of each net driver when CONFIG_NET_ICMPv6 is enabled.
Testing
Not yet.

I gave commit 9b06eba a quick test run, issues:

The main IPv6 address is not added to MAC filter if it is configured before ifup call. It looks like netdev_ipv6_mcastmac() never gets called for it.
Ping works to address added with ifconfig eth0 inet6 add fd0f::e200 but gives warning:

PING fd0f::e200(fd0f::e200) 56 data bytes
64 bytes from fd0f::e100: icmp_seq=2 ttl=64 time=0.171 ms (DIFFERENT ADDRESS!)
64 bytes from fd0f::e100: icmp_seq=3 ttl=64 time=0.187 ms (DIFFERENT ADDRESS!)

TCP connections do not work because the response comes from wrong IP address.

Problems 2 and 3 exist prior to this commit. I think I'll try to make a fix for those, because TCP connections not working is a blocker for my application. I can assign IP addresses so that 4 last bytes are equal, so I'm fine with taking our time with the MAC filter fix.

I gave commit 9b06eba a quick test run, issues:

The main IPv6 address is not added to MAC filter if it is configured before ifup call. It looks like netdev_ipv6_mcastmac() never gets called for it.
Ping works to address added with ifconfig eth0 inet6 add fd0f::e200 but gives warning:

PING fd0f::e200(fd0f::e200) 56 data bytes
64 bytes from fd0f::e100: icmp_seq=2 ttl=64 time=0.171 ms (DIFFERENT ADDRESS!)
64 bytes from fd0f::e100: icmp_seq=3 ttl=64 time=0.187 ms (DIFFERENT ADDRESS!)
3. TCP connections do not work because the response comes from wrong IP address.
Problems 2 and 3 exist prior to this commit. I think I'll try to make a fix for those, because TCP connections not working is a blocker for my application. I can assign IP addresses so that 4 last bytes are equal, so I'm fine with taking our time with the MAC filter fix.

@PetteriAimonen Are you using netinit to initialize your main IPv6 address? I forgot to change the IOCTL SIOCSLIFADDR which is also left for backward compatibility and is used by netinit. You may try 430289b107 to see if it's correct now.

Commit 430289b now works quite well for me.
Our codebase actually is cutting some corners and writing directly into dev->d_ipv6addr before bringing up the interface.
I realize that it should use ioctl instead, and that is something I'm going to fix.
But even that seems to work ok now.

Commit 430289b now works quite well for me.

@PetteriAimonen Thanks for checking this commit, I'll polish this PR to a ready state this week.

Our codebase actually is cutting some corners and writing directly into dev->d_ipv6addr before bringing up the interface. I realize that it should use ioctl instead, and that is something I'm going to fix. But even that seems to work ok now.

If you can access dev->d_ipv6addr directly, I think you might be working under kernel space, then just calling netdev_ipv6_add would be enough. If you're really under user space, I suggest you use existing netlib functions like netlib_add_ipv6addr or netlib_set_ipv6addr, wish you success.


Annotate tree_util

Nice! Thanks as always, Neil.
I have suggestions which fall into two main categories:

whitespace

Of course, I'm happy to fix all whitespace to match your style 😄

having more specific output type annotations (lists rather than sequences).

I'm interested if there are any strong opinions against 2, but it seems preferable to be specific about output types (unless we thought they were likely to change).

I don't have any strong opinion, but I should have commented on this in the change.   I initially wrote the annotations using List, but it turns out that the underlying jaxlib functions are already annotated using Sequence.  This caused typing errors in the tree_util code.
I feel like jaxlib and tree_util should match: either jaxlib should change its annotations to List, or else tree_util should use Sequence to match.  What do you think?  Do you agree?
Promising sequence means that you could one day choose a different sequence type than list.  For example, you might provide some alternative structure that supports easier tree traversal.  Maybe that was the motivation of whoever annotated jaxlib:  keep options open.  If I had to choose, I think promising less is usually better.
Please let me know what you prefer.

I agree with the jaxlib author that promising less is usually better.

I'm not so sure about this in general. Why not promise even less and say it's just an Iterable? We want to promise the right amount, and IMO there's no harm in continuing to say it's a list, as our docs already did. Or at least, there wouldn't be except for the other issue you mentioned.

but it turns out that the underlying jaxlib functions are already annotated using Sequence. This caused typing errors in the tree_util code.

Ah I see. Let's keep it how your PR does it then! I don't mean to add any extra work.

I'm not so sure about this in general. Why not promise even less and say it's just an Iterable? We want to promise the right amount,

Yes, you're absolutely right of course.

Ah I see. Let's keep it how your PR does it then! I don't mean to add any extra work.

Okay, thanks!  I'll make the change above, squash, and let you know when it's done.

You're so generous and easy to work with, Neil! It's always a pleasure, and I always learn something. You're the best.

Unfortunately, internal typechecks (i.e. typechecks spanning all google code which uses these functions!) ran into some issues with this.
As one example, jraph does addition on the leaves outputs of tree_flatten calls. But that doesn't typecheck against Sequence, since you can't add Sequences together!
Maybe we could put these annotations under if TYPE_CHECKING? Got a better idea?

As one example, jraph does addition on the leaves outputs of tree_flatten calls. But that doesn't typecheck against Sequence, since you can't add Sequences together!

That makes sense.  That's why I had to change a couple instances of concatenation in the Jax code to use generalized unpacking.

Maybe we could put these annotations under if TYPE_CHECKING? Got a better idea?

Unfortunately, that won't solve the type checking errors since those run under TYPE_CHECKING too.  I can see a few options:

Change tree-util annotations to use lists, and change jaxlib to match,
Change tree-util annotations to use lists, and add casts throughout so that jaxlib doesn't complain, or
Change the client code to cast results to list where necessary.

Option 1 means changing Jaxlib (is that hard?)  Option 2 adds extra casts that weren't there before, and is awkward having two mismatched interfaces.  Option 3 depends on how many type errors you ended up with.
Another question goes back to your point above.  What do you want in an ideal world?  Would you rather promise list and people can easily concatenate them or would you rather promise sequence and you're free to use another container down the road?
There may be other options, but those are the obvious ones to me.  Please let me know what you decide and I'll repair this change.

Thanks for that super clear analysis.
I think Option 3 would be pretty annoying.
I like Option 1 but I wonder if @hawkinsp has a different opinion. Peter, WDYT?

@hawkinsp If you like, I can make a corresponding pull request to tensorflow's jaxlib to make the interface promise lists?

Sure, that's fine. Just tag me in the PR so I see it.

This should pass when Tensorflow 2.10's version of jaxlib is used (whenever that is)

@mattjj Would it be possible to get this merged now that the tensorflow type annotations are in jaxlib?

(taking over review/merge, since this is related to #12049)


add model factory

WIP to retrieve a model instance from the avsc file path.
https://github.com/godatadriven/pydantic-avro/issues/37

@jonykalavera Nice change to add this. Can you add some testing for this to be sure that this stays working?

Codecov Report

Merging #36 (c7d1628) into main (5388e41) will decrease coverage by 2.89%.
The diff coverage is 14.28%.

@@            Coverage Diff             @@
##             main      #36      +/-   ##
==========================================
- Coverage   89.08%   86.18%   -2.90%
==========================================
  Files           2        2
  Lines         174      181       +7
==========================================
+ Hits          155      156       +1
- Misses         19       25       +6

Impacted Files
Coverage Δ

src/pydantic_avro/avro_to_pydantic.py
77.77% <14.28%> (-4.84%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5388e41...c7d1628. Read the comment docs.


pull request for e2e test at 01/26/2017 20:26:09

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


i get this when i try to route to Route::get('generator_builder', '\InfyOm\GeneratorBuilder\Controllers\GeneratorBuilderController@builder');

Route [io_field_template] not defined. (View: C:\xampp\htdocs\advertisement\vendor\infyomlabs\generator-builder\views\builder.blade.php)

@sanamshrestha123 You should run php artisan infyom.publish:generator-builder
It will publish routes in routes.php with named routes.

Updated docs to publishing routes in generator builder.
you can check here: https://labs.infyom.com/laravelgenerator/docs/6.0/gui-interface#routes-publish-command


storage::RawClient member functions should return kUnimplemented

We think pure virtual functions are a recipe to break backwards compatibility as we add member functions. The class should:

Return a status of kUnimplemented for simpler RPCs.
Return a satisfied future with a status of kUnimplemented for asynchronous and polling RPCs.
Return a range containing a single value with status kUnimplemented for iteration APIs.

What should be done about virtual ClientOptions const& client_options() const = 0; when ClientOptions has an explicit constructor? Do we just return CreateDefaultClientOptions()?

Good question.  Short answer is I don't know. We would probably design the class differently these days.
One possibility is to throw an exception via google::cloud::Throw*, that will crash with exceptions disabled, not great, but easy to explain.
Another possibility is to leave this always pure virtual, which is ugly because one function would be pure virtual and the others would not.
I think I like the second one better. /cc @devjgm in case he has better ideas.

For anyone following along, the code we're talking about is here
https://github.com/googleapis/google-cloud-cpp/blob/f9d9b2498d30f83a84f206e71c3dcb721ee00d3f/google/cloud/storage/internal/raw_client.h#L53
Good question @remyabel. I don't have any better ideas. I think my vote would be to agree with @coryan 's inclination and leave that one method pure. Yes, it's a little strange that the one method is pure and all the others have a default implementation. However, that method looks different than the others since it doesn't return a StatusOr anyway, so it doesn't bother my aesthetic sensibilities too much.

do we think this class should be concrete or abstract? I think the latter, and making the destructor pure-virtual would be breaking at this point, so leaving this pure also solves that problem.
The real thing we're trying to accomplish is avoid breaking changes when we add methods, making the existing ones non-pure is basically an optional thing for consistency, so I don't think it's a problem. Removing a method is always going to be breaking.

@mr-salty do you mind elaborating on what changes you want to see (or don't want to see)?

Keeping open, but can't commit to it right now.

Closing, we just need to make new functions non-pure virtual.


Suspect button element type not render properly during page load

Category

[x] Bug

Describe the bug
Suspect button element type not render properly during page load
{
  "schema": "https://developer.microsoft.com/json-schemas/sp/view-formatting.schema.json",
  "debugMode": false,
  "hideSelection": true,
  "hideColumnHeader": true,
  "rowFormatter": {
    "elmType": "button",
    "customRowAction": {
      "action": "defaultClick"
    },

Steps to reproduce

Click on any item in the list when first launch, nothing happened
Click preview in format current view
Click on item in the list - form launched

Expected behavior
Launch of list item form at the right

Post at wrong section.


Changed PUB_SECTOR_PATTERN to fix validation on IPA code

#147
Scope of the pull request
Fix validation on IPA code for schools

Please bring this to 5
https://github.com/italia/spid-compliant-certificates-python/blob/main/spid_compliant_certificates/__init__.py#L31
that would be our release


[BUG]: running mysql migration with multiple queries results in syntax error

What version of drizzle-orm are you using?
0.23.4
Describe the Bug
Running mysql migration results in syntax error.
I wrote a simple schema file with 2 tables, generated initial migrations by running npx drizzle-kit generate:mysql --out migrations/ --schema app/db/schema.ts. The generated sql is all valid. I can successfully run it myself if I simply copy paste it into a mysql prompt. But if I run the migration using drizzle, I get a syntax error.
My guess is that the method used to send sql to mysql does not support multiple queries. If I edit the migration file to contain only 1 query, the migration succeeds.
Details
perfectly valid migration file that generates a syntax error
CREATE TABLE `teams` (
	`name` varchar(256)
);

CREATE TABLE `users` (
	`id` serial AUTO_INCREMENT PRIMARY KEY NOT NULL,
	`name` varchar(256)
);

code to run migration files (which generates a syntax error)
import { migrate } from 'drizzle-orm/mysql2/migrator';
import { drizzle } from 'drizzle-orm/mysql2';
import mysql from 'mysql2/promise';

export const getDb = () =>
  drizzle(mysql.createPool(process.env.DATABASE_URL as string), {
    logger: true,
  });

export async function doMigration() {
    await migrate(getDb(), { migrationsFolder: './migrations' });
}

migration logs reporting syntax error
Query: CREATE TABLE IF NOT EXISTS `__drizzle_migrations` (
			id SERIAL PRIMARY KEY,
			hash text NOT NULL,
			created_at bigint
		)
Query: SELECT id, hash, created_at FROM `__drizzle_migrations` ORDER BY created_at DESC LIMIT 1
Query: BEGIN
Query: CREATE TABLE `teams` (
	`name` varchar(256)
);

CREATE TABLE `users` (
	`id` serial AUTO_INCREMENT PRIMARY KEY NOT NULL,
	`name` varchar(256)
);

Query: ROLLBACK
Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'CREATE TABLE `users` (
	`id` serial AUTO_INCREMENT PRIMARY KEY NOT NULL,
	`name`' at line 5
    at PromisePool.query (/Users/jonsmithers/git/ragnar-next/node_modules/mysql2/promise.js:351:22)
    at MySql2PreparedQuery.eval (webpack-internal:///(sc_server)/./node_modules/drizzle-orm/mysql2/session.js:71:36)
    at Generator.next (<anonymous>)
    at eval (webpack-internal:///(sc_server)/./node_modules/drizzle-orm/mysql2/session.js:26:71)
    at new Promise (<anonymous>)
    at __awaiter (webpack-internal:///(sc_server)/./node_modules/drizzle-orm/mysql2/session.js:8:12)
    at MySql2PreparedQuery.execute (webpack-internal:///(sc_server)/./node_modules/drizzle-orm/mysql2/session.js:66:16)
    at MySql2Session.execute (webpack-internal:///(sc_server)/./node_modules/drizzle-orm/mysql-core/session.js:14:88)
    at MySqlDialect.eval (webpack-internal:///(sc_server)/./node_modules/drizzle-orm/mysql-core/dialect.js:91:51)
    at Generator.next (<anonymous>)
    at fulfilled (webpack-internal:///(sc_server)/./node_modules/drizzle-orm/mysql-core/dialect.js:11:32)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  code: 'ER_PARSE_ERROR',
  errno: 1064,
  sql: 'CREATE TABLE `teams` (\n' +
    '\t`name` varchar(256)\n' +
    ');\n' +
    '\n' +
    'CREATE TABLE `users` (\n' +
    '\t`id` serial AUTO_INCREMENT PRIMARY KEY NOT NULL,\n' +
    '\t`name` varchar(256)\n' +
    ');\n',
  sqlState: '42000',
  sqlMessage: "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'CREATE TABLE `users` (\n" +
    '\t`id` serial AUTO_INCREMENT PRIMARY KEY NOT NULL,\n' +
    "\t`name`' at line 5"
}

@jonsmithers
Reason
In MySQL you can't run >1 DDL queries against database
Solution
There are 2 ways to handle this case
First one using mysql2 driver connection patch. You need to add multipleStatements: true to mysql2 connection config and it should look like this:
also here are some docs for mysql migrations
const poolConnection = mysql.createPool({
  host: 'localhost',
  user: 'root',
  database: 'test',
  multipleStatements: true,
});

I also would advice to use single connection rather than pool
Second option would be using drizzle-kit breakpoints features
You can regenerate migrations using --breakpoints if you are using cli options for drizzle kit or just add "breakpoints": true to drizzle.config.json file
In this case you'll see extra --> statement-breakpoint after each DDL statement, that will help drizzle-orm to separate them properly
We don't have second option documented, but we will have soon

Ahhh, I missed those bits in the documentation. Sounds like that will resolve my issue. Thank you!


Add commit policy and code of conduct

I tweaked the language slightly, to make it clear that PR's authored by Googlers can be approved by non-Google collaborators. If the new language seems reasonable, I'll merge.

LGTM.


Update node example to use the prebuilt binary

Summary
Unfortunately I can't tell if the version for nodePackages.pnpm is prebuilt or not. Once we have that information in https://nixhub.io I will update the examples again.
How was it tested?
devbox install

This is the command to check if a package is in the cache:
nix path-info --store 'https://cache.nixos.org' --json nixpkgs/<commit>#<attr_path>

For example:
nix path-info --store 'https://cache.nixos.org' --json nixpkgs/3c614fbc76fc152f3e1bc4b2263da6d90adf80fb#nodejs_18


2.0 planning

The work for 2.0 is really taking root.  After getting recurrent neural networks in v1 (not yet released) I could see a great deal how we could make it simpler, but I also wanted to address the entire architecture as a whole.  We can simplify the strategies of a recurrent neural network and a feedforward neural network, so that more advanced concepts, like convolution could easily be used.  Also I really want to make the api's for both feedforward and recurrent to be as close as possible, so you could simply change out the underlying network type, and recurrent is what you'd have.  After a lot of careful consideration, this is what I've got:
Nets will have at least two means of composition, imperative and functional. They can be mixed.
This is an imperative defined feedforward convolutional neural network:
new brain.FeedForward({
  inputLayer: () => input({ width: 24, height: 24 }),
  hiddenLayers: [
    (input) => convolution({ filterCount: 8, filterWidth: 5, filterHeight: 5, padding: 2, stride: 1 }, input),
    (input) => relu(input),
    (input) => pool({ padding: 2, stride: 2 }, input),
    (input) => convolution({ padding: 2, stride: 1, filterCount: 16, filterWidth: 5, filterHeight: 5 }, input),
    (input) => relu(input),
    (input) => pool({ width: 3, stride: 3 }, input),
    (input) => softMax({ width: 10 }, input)
  ],
  outputLayer: (input) => output({ width: 10 }, input)
});

This is a functional defined feedforward convolutional neural network:
new brain.FeedForward({
  inputLayer: () => input({ width: 24, height: 24 }),
  hiddenLayers: [
    (input) =>
      softMax({ width: 10 },
        pool({ width: 3, stride: 3 },
          relu(
            convolution({ padding: 2, stride: 1, filterCount: 16, filterWidth: 5, filterHeight: 5 },
              pool({ padding: 2, stride: 2 },
                relu(
                  convolution({ filterCount: 8, filterWidth: 5, filterHeight: 5, padding: 2, stride: 1 },
                    input
                  )
                )
              )
            )
          )
        )
      )
  ],
  outputLayer: (input) => output({ width: 10 }, input)
});

Both of these create the exact same type of network, the reason for the two strategies is that I really like to essentially not have to learn a language, and the functional approach matches up to math nearly one to one.  The imperative version too, sometimes you just don't want to think in reverse, and lists are nice.  For recurrent nets I think the functional will shine more as you start the mind-bending process of recursion.  I'm open to suggestion!
Later we can apply the same composition to do:
const net = new brain.Recurrent({ inputLayer, hiddenLayers, outputLayer });

I'm sure there will be more, but this is a start.

Json input and output is currently tested and is:
toJSON():
net.toJSON();

note: no change, see next for json definition.
static fromJSON(json, function):
const net = FeedForward.fromJSON({
        layers: [
          {
            type: 'TestLayer',
            foo: true
          },
          {
            type: 'TestLayer',
            foo: true,
            inputLayerIndex: 0
          },
          {
            type: 'TestLayer',
            foo: true,
            inputLayerIndex: 1
          },
          {
            type: 'TestLayer',
            foo: true,
            inputLayerIndex: 2
          }
        ]
      }, (jsonLayer, input) => {
        switch (jsonLayer.type) {
          case 'TestLayer':
            return new TestLayer(jsonLayer, input);
          default:
            throw new Error(`unknown layer ${ jsonLayer.type }`);
        }
      });

Note: inputLayerIndex isn't terribly useful here, but will be when recurrent happens.  The definition is FeedForward.fromJSON(json, (jsonLayer, input) => { return new ALayer(jsonLayer, input); });
The idea is that we can accept any layer, your own are fine, if you like.

Just a quick note.  Recurrent neural networks proposal:
new brain.Recurrent({
  inputLayer:
  hiddenLayers: [],
  outputLayer:
});

Because a LSTM neural network is essentially architecturally the same as an RNN, but with different hidden layers, we can have a layer be the definition for the LSTM.  All the math is encapsulated.  Easy to understand, and easy to extend.  Picture the whole underlying method like this:
//this will be part of our lib
brain.LSTM = class LSTM extends brain.Recurrent {}
LSTM.defaults = {
  brain.Recurrent.defaults...,
  hiddenLayers: layer.lstm
};

//this will be outside of lib, in real world:
new brain.LSTM();

Also: the layer is in charge of how the layer learns, period.  This sounds vague and probably a bit weird, but picture the forward propagate (predict) and back propagate (compare, and learn) as managed inside the layer so all the math that affects the layer is in one location.  The order in which layers predict, compare, and learn, is only managed inside the lib.  So we get this super nice and clean means of composing a layer:
import Base from 'brain/layer/base';
import makeKernel from 'brain/utilities/make-kernel';
class MyLayer extends Base {
  //attach to GPU
  createKernels() {
    this.predictKernel = makeKernel({ size: [this.width, this.height, this.depth] }, predict);
    this.compareKernel = makeKernel({ size: [this.width, this.height, this.depth] }, compare);
    this.learnKernel = makeKernel({ size: [this.width, this.height, this.depth] }, learn);
  }

  //attach GPU to our class, and special operations of class
  predict() {
    this.outputs = this.predictKernel(this.weights, this.inputLayer.outputs);
  }
  compare() {
    this.biases = this.compareKernel(...);
  }
  learn() {
    this.deltas = this.learnKernel(...);
  }
}

//math here
function predict(weights, inputs) {
  return weights[this.thread.z][this.thread.y][this.thread.x] * inputs[this.thread.z][this.thread.y][this.thread.x];
}
function compare() {}
function learn() {}

This runs on gpu, node, and js.

So here is the latest in trends for LSTM and recurrent behavior.  We need to define a single set of programs that can be reused, and a simplified set of terminologies that describe them.
Here is the commit: https://github.com/BrainJS/brain.js/commit/d43b1ee8644bfffa7eb27da3fde4ef2e32c819dd#diff-b84ad68a55b45bd31698c5356097abf3R15
The idea behind it:
We need a way to define groups of layers that compose a single layer.  LSTM, for example, is very simple in that it uses the same technology as RNN but it is a bit bigger.  If we describe each step in the LSTM as a "layer" it becomes very hairy.  However, if we describe each mathematical operation as part of the layer, the "layer" is actually composed of many "layers".  But don't think about that.  Just think of that if we want a complex layer, it can have multiple mathematical operations.  To simplify (ha!) the understanding of this, we have the concept of "group".  I'll try my best to describe it:
export default class SuperLayer extends Group {
  constructor(settings) {
    super(settings);
    //setup layer here
  }

  static kernel(settings) {
    return (layer, inputLayer, previousOutputs) => {
      //layer passed in above ^
      //tons of layers and math
      return result;
    };
  }
}

So the concept here is that the kernel gets created once, however the layer can be composed any number of times.  The recurrent behavior stacks the layers, the layers can use add, multiply, sigmoid, relu, etc. inside the kernel, and then out comes the result.  So we have one program to manage the operations.
Feedback welcomed!

Now that GPU.js is released as v1.0.0, we are full on!  For reference: https://gist.github.com/robertleeplummerjr/4aaf8afb177c9c80f8452d5025117e26


Localhost HTTPS issues

Hi @unclecheese just wanted to check if you'd have any way around to get https working locally. I've setup a custom server to start up nextjs and have made a few tweaks on the https globalAgent to rejectUnauthorized, setting that to false as suggested in a few places.
Here's my server setup
const { createServer: httpsCreateServer, globalAgent } = require("https");
const { parse } = require("url");
const fetch = require("node-fetch");
const next = require("next");
const fs = require("fs");
const { devPort, isUsingHTTPS, secureLocalDomain } = require('./server.env');

if (isUsingHTTPS) {
    globalAgent.options.rejectUnauthorized = false;
    globalAgent.options.key = fs.readFileSync(secureLocalDomain.key);
    globalAgent.options.cert = fs.readFileSync(secureLocalDomain.cert);
}

const dev = process.env.NODE_ENV !== "production";
const app = next({ dev });
const handle = app.getRequestHandler();

const httpsOptions = isUsingHTTPS && {
    key: fs.readFileSync(secureLocalDomain.key),
    cert: fs.readFileSync(secureLocalDomain.cert),
    rejectUnauthorized: false
};

app.prepare().then(() => {
    httpsCreateServer(httpsOptions, (req, res) => {
        const parsedUrl = parse(req.url, true);
        handle(req, res, parsedUrl);
    }).listen(devPort, (err) => {
        if (err) throw err;
        console.log("ready - started server on url: https://localhost:" + devPort);
    });
});

But even though I've done the update to the globalAgent, that doesn't get picked up on createClient and I still get errors on the StaticBuild query.
FetchError: request to https://ss-nextjs.local.test/graphql failed, reason: unable to verify the first certificate
If I go to the createClient on node modules and add the https globalagent configs there, then things work fine.
Just wondering if you'd know of away I could get https to work without these gql issues and the need to update anything on this npm module.

Hey, thanks for jumping in and testing, @rafaeldsousa .. You should be able to configure the client as you need in the ss.config.ts file. Have you tried adding the required parameters in there?

@unclecheese thanks a lot for your reply.
I see what you mean and that's probably the best approach, but I believe as is we wouldn't be able to include the agent config in the options, as the Types available for ClientConfig are on String|StringMap and when passing the agent config we need to pass new Agent from https. Not sure if any casting here would work.
export interface ClientConfig {
    endpoint: string
    options: {
        [key: string]: string | StringMap
        headers: StringMap
    }
}


Function may be static in optical_flow_test.py

This is to progressively get rid of the @run_all_in_graph_and_eager_mode. I try no to take everything out of the class at once to avoid a big diff.

@failure-to-thrive
You are owner of some files modified in this pull request.
Would you kindly review the changes whenever you have the time to?
Thank you very much.

Thank you for the review!


Make OrderPlacementMixin more customizable

This PR changes the signature of the get_message_context method to make it useful for various event codes that could be provided to the send_confirmation_message method.

Hello @solarissmoke,
I made few changes on this PR.
Could you please review it again?
Thanks!


jbuf: trace data for plot

Thanks! Now I'll rebase and update

https://github.com/baresip/re/pull/963
https://github.com/baresip/baresip/pull/2732


Web app: Implement dark mode

Resolves #206
Could also have a setting for it to override it (e.g. "System" / "Light" / "Dark"), but this just automatically uses the system  mode.

a
b
c

Codecov Report
Patch and project coverage have no change.

Comparison is base (d294a69) 65.85% compared to head (4828e3a) 65.85%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #790   +/-   ##
=======================================
  Coverage   65.85%   65.85%
=======================================
  Files          53       53
  Lines        7646     7646
=======================================
  Hits         5035     5035
  Misses       1775     1775
  Partials      836      836

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Render crash upon adding this mod

Not sure why, but as soon as I added this mod, Minecraft decided working was too good for me. Related crashlog here: http://pastebin.com/fDWCFk5C
As far as I can tell, it's AE2Stuff, which is strange considering how a) I haven't even started AE yet, b) I was in the Twilight Forest and my base is in the overworld, but the only mod referenced is ae2Stuff and adding this mod (and the dependency) was literally the only thing changed when I went from stable to crash.

Please try this build and report if it fixes the crash (or not) - http://jenkins.bdew.net/job/ae2stuff-1.7.10/50/artifact/build/libs/ae2stuff-<IP_ADDRESS>-mc1.7.10.jar

Gave it a try - near as I can tell, that's fixed it. Played around in-game for ten minutes, checked the same areas I was in before, mobs, etc, no crashes occurred.


Don't instrument Servlet API < 3.0

When the application uses Servlet API 2.5, for example, we should not instrument it as it causes errors at runtime.
Instead, we should log a warning on startup that we don't support this version.

Getting the version is quite straightforward using ServletContext.getMajorVersion(), however we might not get access to this class until a servlet or filter is being actually used or initialized.
Then, does it means servlet instrumentation will be active by default, but could self-disable at runtime if relevant ?

We could filter out class loaders that don't contain types that were introduced in 3.0.
@Override
public ElementMatcher.Junction<ClassLoader> getClassLoaderMatcher() {
    return CustomElementMatchers.classLoaderCanLoadClass("javax.servlet.annotation.WebServlet");
}


deps: remove dvc dependency

dvc-http does not really depend on dvc. This also removes a circular dependency
closes https://github.com/iterative/dvc/issues/8897

Thanks a lot for the quick fix! Appreciate it 👍


We don't get notification when a session is bookmarked from list

alarm is not set when a session is bookmarked from SessionList directly.

@utkarshnath Please let the mentors approve the issue before you start working on it. Maybe things were intended this way due to some reason. So, please get an pproval before starting to work on an issue. Thanks!

@mananwason Sorry I'll take care of it from now


The Name field, Element Documentation Field, etc on elements is pasting rich text rather than plain text

Copy some rich text from a website and do a CMD+V (macos 10.12.6) using Modeler 1.11.2

This problem has existed for a long time.
Identified Fields with the problem:

Name
Element Documentation
Listener: Field Injection: Value
Input: Text: Value
Output: Text: Value

Related to https://github.com/camunda/camunda-modeler/issues/961.

seems to be fixed


getAnonymousSignature(): add null handling for TreePath return [phab:D1855]

Created by jrtom at 2017-08-10 18:48:56:

schroederc accepted this revision at 2017-08-10 18:54:10

jrtom closed via commit b22b09bbec92bb25753762e278bbb8357dfd2b8d at 2017-08-10 18:55:28


Umami on Netlify stop working since v1.37.0 upgrade

Hello,
My Umami stopped to worked and collect data since the 1.37.0 upgrade (it's works smoothly from March 2022).
On the Dashboard the only things i have an error is the realtime view on console log :
react_devtools_backend.js:4026 TypeError: Failed to fetch
    at r (_next/static/chunks/pages/_app-3f39117fa6aacd01.js:1:434499)
    at o (_next/static/chunks/pages/_app-3f39117fa6aacd01.js:1:434853)
    at _next/static/chunks/398-8eb46fcd2cf61446.js:1:19669
    at s (_next/static/chunks/main-f1f37299d5d02885.js:1:104683)
    at Generator._invoke (_next/static/chunks/main-f1f37299d5d02885.js:1:104471)
    at forEach.e.<computed> [as next] (_next/static/chunks/main-f1f37299d5d02885.js:1:105106)
    at a (_next/static/chunks/pages/_app-3f39117fa6aacd01.js:1:454515)
    at u (_next/static/chunks/pages/_app-3f39117fa6aacd01.js:1:454718)
    at _next/static/chunks/pages/_app-3f39117fa6aacd01.js:1:454777
    at new Promise (<anonymous>)

issue on fetch .. _app-3f39117fa6aacd01.js:1          GET https://hello-laurent.netlify.app/api/realtime/init 500
I have no issue during the deploy on Netlify.
My Post-gre DB run on Supabase
If i made  on local
yarn prisma migrate resolve --rolled-back "02_add_event_data"
yarn prisma migrate deploy
yarn check-db
I ve got this issue
2 migrations found in prisma/migrations

Applying migration `02_add_event_data`
Error: P3018

A migration failed to apply. New migrations cannot be applied before the error is recovered from. Read more about how to resolve migration issues in a production database: https://pris.ly/d/migrate-resolve

Migration name: 02_add_event_data

Database error code: 42501

Database error:
ERROR: must be owner of table event

DbError { severity: "ERROR", parsed_severity: Some(Error), code: SqlState(E42501), message: "must be owner of table event", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("aclchk.c"), line: Some(3583), routine: Some("aclcheck_error") }

Someone have an idea about it or possibility to run Migration manually from Supabase manager ?

I had the same issues with Supabase when tried running migrations locally and while trying to fix it, ended up breaking lots of stuff so had to create a new db, migrate all the data to the new db, but I'm still stuck with the issues and endpoints throwing 5xx errors 😞

Update: https://github.com/umami-software/umami/discussions/1434#discussioncomment-3439712
Post this, I am able to fix the issues and getting no errors when running check-db and update-db commands locally.

Thanks for the help @AkashRajpurohit !!!
i will look in quickly.

Same error after clean install

Sorry @mikecao to mention you.
But do you think the next release could fix 1.37 issue upgrade ?

Thanks a lot @mikecao for the hard work !!!
Have a good day from France


Lens not starting on Arch from AppImage

When trying to start Lens from the AppImage on my Arch install, I am presented with the following logs:
info: 📟 Setting Lens as protocol client for lens://
info: 📟 failed ❗
info: 🚀 Starting Lens from "/home/wgoeman/.config/Lens"
info: 🐚 Syncing shell environment

After this output, the application hangs and nothing happens. No GUI is being started.
I would gladly attach some more logs, but I have no idea on how to get more verbose output on what is going on.

Do you have something in your shell startup scripts (i.e. .bash_profile) that might be blocking?

Can you direct me to where the AppImage is built? If the AppImage tries to start a bash shell, which should then launch Lens, then that could be the reason. My bash init script switches the process (exec) to fish at the end. That could be causing issues, but then again it would be strange for Lens to launch bash.

Just tried it using the latest AppImage of Lens 5 Beta on a relatively new Arch installation with Gnome, Wayland, PipeWire, and ZSH - works like a charm.
Have you ever considered setting your login shell to fish instead of calling it whenever bash should be started? Fish is not fully compatible to bash. You shouldn't assume everything to run just as expected with bash

Fish is intentionally not POSIX-compatible and as such some of the things you are used to work differently

Source: https://fishshell.com/docs/current/fish_for_bash_users.html)

I have been considering to just set fish as my main shell in the past. I did have an issue at some point though where after an update, fish failed to start. That would not be great when it is my 'main' shell. Note that I have been using fish this way for years and Lens is the first application actually having issues with that.
Anyway, I suppose I can make the switch now as being able to use Lens is a good motivation to do so. It is still just a workaround imho though.
I just find it curious that Lens requires a shell to be able to start. Is that by design? No way to get around that?

I don't know about the internals of the AppImage. I haven't had problems using other shells like zsh as my login shell. I don't wanna remove bash from my os to test the implications of not having it :D
I'd consider this as an edge case rather than a bug because you did use a shell that's not compatible to bash as a bash replacement in a way that's not officially supported nor intended in any way.
Have you tried it yet? Did it work?


Document has helper

All row columns except for id are nullable by default. A column can be null for two reasons. It's not loaded yet, or it's obsolete. That's why the has helper is so useful, and that's why it belongs to the application itself.
TODO: Add an example to README.md.
const nodes = pipe(
  useQuery((db) =>
    db
      .selectFrom("node")
      .select(["id", "md"])
      .where("isDeleted", "is not", model.cast(true))
      .where("id", "in", ids)
  ),
  (a) => ({ ...a, rows: a.rows.filter(has(["md"])) })
);

Closed because of https://github.com/evoluhq/evolu/issues/69


Feature Request: Notify user if Assemblyline has already seen a sample

Is your feature request related to a problem? Please describe.
Our analysts frequently encounter the issue of redundancy, submitting a malware sample to Assemblyline that has already been analysed. While the current system allows users to query and identify such instances, it is not proving to be an intuitive or streamlined process. As a result, our team often misses opportunities for collaboration and shared learning with analysts who have previously studied the same sample.
To address this issue, we propose an automatic alert system in Assemblyline similar to the functionality offered by VirusTotal. Upon submission of a sample, if Assemblyline recognizes the sample as one that has been analysed previously, it should immediately trigger a notification. This alert would inform the user about the sample's existing presence in the system.

Implementation could tie into #52

How would you implement this?

Dialog box before actually performing the submission to give the user the chance not to submit the file and only view the old results?
Submit anyway but in the submission view have a list of other ppl that have submitted the file (This is somewhat already implemented because this is in the file detail view already)
Full blown notification system that would notify all affected user that are logged in that someone is looking at a file they've previously submitted with notification popups?

From a notification perspective, I was thinking email.

Email, and a popup would be ideal. As you said #2 is somewhat already implemented, but has been our experience that users do not really look at this.


As a teacher, I want to be able to create a class for a module

so that I can assign students to the class


WithJavaScript AngleSharp.Services.IScriptingProvider could not found

'Could not load type 'AngleSharp.Services.IScriptingProvider' from assembly 'AngleSharp, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=e83494dcdc6d31ea'.'
var config = Configuration.Default.WithJavaScript();

Sorry the provided information is insufficient. You'll need to provide a MWE to reproduce the issue. What's your configuration like? I assume there is a problem with your appconfig.


the application crashes when copying files

this occurs in all directories and file types
v 1.2.2

@kleuton05 thanks for this report could you post the debug files?

debug.txt
debug_fulltrust.txt

@kleuton05 is the crash happening when copying files, pasting files or navigating to a folder? Did you get a notification when the app crashed?

The moment I paste a file, the app freezes and only unfreezes again after the transfer is completed.

If the app is not crashing (closing unexpectedly) than this is the same as #3355

@kleuton05 could you confirm whether the app is crashing (closing unexpectedly) or freezing? Thanks!

freezing

Thanks fixed in version 1.2.3 (#3350)