Fix a problem with Import Transition List / Assay Library, wherein user isn't given the chance to deal with errors after click on OK if errors were previously reviewed…

… (unintended side effect of AssociateProteins work trying to avoid redundant work).
Updated AbstractFunctionalTest.ImportAssayLibraryOrTransitionList() to check this scenario handling transition lists with errors:
In transition list import column select dialog, hit OK
Get the error dialog, hit cancel - takes you back to column select dialog as it should
In column select, hit "Check for Errors"
Get the error dialog, hit OK - takes you back to column select as it should
In column select dialog, hit OK - skips right over the expected error check dialog, which is bad

The issue centers around the "IsCheckForErrorsNecessary(isAssociateProteins)" call - I guess there's more than one set of code for associate proteins.

Yeah, oops, not your work. I'll get a different reviewer.


Missing env vars don't fail gracefully

If we want to crash, I think we should
match get_env e with
| Some x -> x
| None ->
  eprintf "A nice message here";
  exit 1

This is the run-time getenv, or the ppx?
Is there a link to code where this issue occurs?


⚠ Bofrost.be has degraded performance

In 9e114a6, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1689 ms

Resolved: Bofrost.be performance has improved in 98da4a3 after 47 minutes.


⚠ ClamAV has degraded performance

In 272546a, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 55 ms

Resolved: ClamAV performance has improved in 4243392 after 5 minutes.


[Bug] Zero count on resolved cells after Baysor

Cells that are resolved after baysor segmentation on patches may have zero count.
The issue still needs to be reproducible.

Actually, it is due to map_partition concurrently writing on the same adata object

Fixed in v1.0.4


Refactor/Reimplement Helm templating via Clusterclass templating

As mentioned in https://github.com/SovereignCloudStack/k8s-cluster-api-provider/issues/666 the helm-chart that reflects the current state of https://github.com/SovereignCloudStack/k8s-cluster-api-provider now exists. Because the cluster stack operator can not be used to template a helm chart we will need to reimplement the templating using the clusterclass templating mechanism.

We can take the current helm values.yaml as a starting point for the cluster-class variables. Currently we have the following values(now)-variables(then) in the helmchart:

external_id
dns_nameservers
controller_flavor
worker_flavor
prefix
testcluster_name
node_cidr
etcd_unsafe_fs
http_proxy
no_proxy
containerd_registry_files
etcd_defrag
tweak_kubeapi_memlimit
worker_machine_gen
control_plane_machine_gen
openstack_security_groups
images

A 1:1 translation of "helm-templating" to "cluster-class-patching" will not be possible everywhere for example the cluster-class resource can not be templated as this is the template source for cluster-class-patching.

all helm values are included as cluster class variables now, creating the patches is the next step


Email notification arrives with PST time zone

I see that the email notification is hard coded to send the time in PST:
https://github.com/ks888/LambStatus/blob/ba950df3241ac9143e03411d6c1a06d126cc0180/packages/lambda/src/api/sendEmails/index.js#L230
Wouldn't it make more sense to make it configurable, or at least hardcode it to UTC which is more universal than PST?
Thanks.

Thank you for your suggestion. Yes, it makes sense to make it configurable, though personally I can't find the enough time to support it.
Also, UTC hardcoding is certainly better than PST. Fortunately we already have the formatDateTime function in utils/datetime and so the changes should be not so difficult. It's great if you have a chance to contribute the code.

Sorry for my late reply. If we make it configurable, I think it's better if a user can change the time zone via GUI. In the demo page below, it will be useful if the Notification tab contains the time zone setting item.
https://demo-admin.lambstatus.org/settings
Env variable may be possible, but most users can't change it because they launch their stack using CloudFormation.

I would also love to see this implemented, and seems like an option in the UI is more along the lines of how LambStatus is used and deployed.
However - regardless of this implementation - I believe the default should be changed from PST, to UTC.


[FEATURE] Ice autoban filter

Honestly ice is so predictable now that we could just humor him and code an autoban into the bot to kick him out.
Things to check for ice:

Pings pome, aurelian, mander and sometimes, wooosh
Account creation date less than 3 days old
Set list of swear words known to be used by him in conjunction in pings

I don't think we should add specific code just for dealing with Ice. Maybe we can have the bot auto-ban anyone who is new and gets muted for spamming (or says something from the blocklist).

 At least then you can contact mods in case of misfire. Also let's you be more lenient with the algorithm.

I think what should be done is:-

Mute for 10(?) mins if mass spam. Mutes for 30 mins if the account is new (>= 3 days) and spams
If they get unmuted and spam again, within a set amount of time, they get muted for longer (eg: infraction-break time: 30mins; punishment: 1 hour)
The mute time gets longer and longer and the infraction-breaking time threshold can get shorter/longer whatever needs to be implemented.
3 chances and you're banned.

We could create a custom role for people who spam and have a new account to have permanent slow-mode but that would be too much hassle.
Anything is flexible here, of course, but imo we shouldn't be auto banning.

Ice sees this github issue too you can't stop him


Create Brave Browser Topic

[x] I followed the contributing guidelines: https://github.com/github/explore/blob/main/CONTRIBUTING.md.
[x] I have no affiliation with the project I am suggesting (as a maintainer, creator, contractor, or employee).

Which change are you proposing?

[ ] Suggesting edits to an existing topic or collection
[x] Curating a new topic or collection
[ ] Something that does not neatly fit into the binary options above

Curating a new topic or collection

[x] I've formatted my changes as a new folder directory, named for the topic or collection as it appears in the URL on GitHub (e.g. https://github.com/topics/[NAME] or https://github.com/collections/[NAME])
[x] My folder contains a *.png image (if applicable) and index.md
[x] All required fields in my index.md confirm to the Style Guide and API docs: https://github.com/github/explore/tree/main/docs

Brave is becoming one of the major web browsers with a large number of users. Adding this topic will help people to collect some basic information about it.

I'm not entirely sure what is causing this issue.

I'm not entirely sure what is causing this issue.

It looks like there is a carriage return character (\r) instead of a new line. If you accept my suggested change in my review it should fix the issue.

topics/brave/index.md

@Nihal-Raj Please feel free to open back up this PR whenever you're ready! Thank you.


ESP8266: Watchdog Timer Reset when using v4.0.0-rc0 release

Hi, I'm trying to use a basic example on a NodeMCU ESP8266 board, but I am getting Watchdog Timer Resets.
Error:
 ets Jan  8 2013,rst cause:4, boot mode:(1,6)

wdt reset

Code:
#include <Arduino.h>

#include <jled.h>

JLed led = JLed(LED_BUILTIN).On().DelayBefore(2000);

void setup() {
}

void loop() {
  led.Update();
}

Am I doing something wrong?
Thanks!

I can reproduce the problem and am working on a fix - thanks for the report.

Thanks 😃

I located the problem in the newly introduced placement new, which needs to use an aligned buffer on the ESP8266. I'll provide a fix tomorrow.

This is working perfectly, thanks


Add Interspersed proposal

Motivation
Add a new evolution proposal for the interspersed algorithm.

This depends on #258 to be merged

@swift-ci test

Merging over known CI failure


feat: port Analysis.NormedSpace.Multilinear

Porting notes:

Some declarations are slow and need extra heartbeats
Needed to simp often before applying mkContinuous_norm_le, otherwise Lean wouldn't find the argument H automatically.

Two questions that I was also faced with:
why do we have to increase heartbeats?
why does open Set not work to make referring to things like preimage work without having to do Set.preimage?

bors merge


Shadow or outlined color on text

Hi, I wanna set a narrow outlined color to the text for its better visibility, can any body help me???

@jenkhefter you'd probably want to use text Stroke. Search for "drawing text with stroke", probably this link would help: http://stackoverflow.com/questions/9044769/how-to-draw-text-with-different-stroke-and-fill-colors


Need a way to reset locally stored data

Expected behavior:
If there was a need to reset the data at the server level, any data stored locally (offline) in user's browsers would need to be invalidated/flushed from the local pouchdb storage
Actual behavior:
If the couchdb database is deleted and recreated as an empty database, user's offline data ends up repopulating the data because of the sync between pouch and couch
Steps to reproduce:

A user navigates to a hospitalrun site that has records in it.
A couchdb admin deletes the main database and creates a new (empty) main database.
The user from #1 navigates to the hospitalrun site which causes the pouch/couch sync to run which in turn pushes the locally stored records back to couch, thus repopulating the database that was intentionally emptied.
Screenshots (if applicable):
N/A
OS and Browser:
All

The solution here is to write a script that issues deletes for all the records in the database vs deleting the database and recreating the database as new.
The problem with this is that you cannot then reload the same data dump, because the load DB function in the app does a replication.  Load DB should be changed to instead of doing replication it should load the records in the dump as new records/revs.

@tangollama  would it be possible to give this a try?

Please do.

On Oct 18, 2017, at 12:10 PM, Darryl Pargeter<EMAIL_ADDRESS>wrote:
@tangollama would it be possible to give this a try?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.


two different version of mixer pod instances running after istio upgrade

Describe the bug
istio is upgraded from 1.0.3 to 1.1.0-snapshot6, after upgrade, two pods of istio-policy running, one has version 1.0.3 and another pod has version 1.1.0-snapshot6.
Expected behavior
Only new version 1.1.0-snapshot6 of mixer should exists.
Steps to reproduce the bug

install istio with version: 1.0.3
upgrade to 1.1.0-snapshot6

helm upgrade istio install/kubernetes/helm/istio --namespace istio-system

check istio-stystem pods:

kubectl -n istio-system get po
NAME                                      READY     STATUS      RESTARTS   AGE
istio-citadel-676c58584b-tpx6z            1/1       Running     0          1h
istio-egressgateway-54477c6569-rlhrv      1/1       Running     0          1h
istio-galley-58b7c6b6bb-sq2gl             1/1       Running     0          1h
istio-ingressgateway-f6c4b779b-s882d      1/1       Running     0          1h
istio-init-crd-10-zgsfd                   0/1       Completed   0          3d
istio-init-crd-11-4hz6d                   0/1       Completed   0          3d
istio-init-crd-certmanager-10-khd8d       0/1       Completed   0          3d
istio-pilot-7d586bdf7d-gppcd              2/2       Running     0          1h
istio-policy-688f99c9c4-l4sww             2/2       Running     0          31m
istio-policy-755cf49c4f-6ccvd             2/2       Running     0          1h
istio-sidecar-injector-6d7586f8cd-fq9pg   1/1       Running     0          1h
istio-telemetry-69b794ff59-sq4rq          2/2       Running     0          1h
istio-telemetry-7c7ff645cf-r62tb          2/2       Running     0          1h
prometheus-849b9cddff-h8krh               1/1       Running     0          1h

check deployment for istio-policy:

 kubectl -n istio-system get deployment istio-policy
NAME           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
istio-policy   1         1         1            1           3d

kubectl -n istio-system describe deployment istio-policy
Name:                   istio-policy
Namespace:              istio-system
CreationTimestamp:      Fri, 15 Feb 2019 16:03:13 -0800
Labels:                 app=istio-mixer
                        chart=mixer
                        heritage=Tiller
                        istio=mixer
                        release=istio
                        version=1.1.0
Annotations:            deployment.kubernetes.io/revision=1
Selector:               app=policy,chart=mixer,heritage=Tiller,istio=mixer,istio-mixer-type=policy,release=istio,version=1.1.0
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  0 max unavailable, 1 max surge
Pod Template:
  Labels:           app=policy
                    chart=mixer
                    heritage=Tiller
                    istio=mixer
                    istio-mixer-type=policy
                    release=istio
                    version=1.1.0
  Annotations:      scheduler.alpha.kubernetes.io/critical-pod=
                    sidecar.istio.io/inject=false
  Service Account:  istio-mixer-service-account
  Containers:
   mixer:
    Image:       docker.io/istio/mixer:1.1.0-snapshot.5
    Ports:       9093/TCP, 42422/TCP
    Host Ports:  0/TCP, 0/TCP
    Args:
      --monitoringPort=9093
      --address
      unix:///sock/mixer.socket
      --configStoreURL=mcp://istio-galley.istio-system.svc:9901
      --configDefaultNamespace=istio-system
      --useAdapterCRDs=true
      --trace_zipkin_url=http://zipkin:9411/api/v1/spans
    Requests:
      cpu:     10m
    Liveness:  http-get http://:9093/version delay=5s timeout=1s period=5s #success=1 #failure=3
    Environment:
      GODEBUG:  gctrace=2
    Mounts:
      /etc/certs from istio-certs (ro)
      /sock from uds-socket (rw)
   istio-proxy:
    Image:       docker.io/istio/proxyv2:1.1.0-snapshot.5
    Ports:       9091/TCP, 15004/TCP, 15090/TCP
    Host Ports:  0/TCP, 0/TCP, 0/TCP
    Args:
      proxy
      --domain
      $(POD_NAMESPACE).svc.cluster.local
      --serviceCluster
      istio-policy
      --templateFile
      /etc/istio/proxy/envoy_policy.yaml.tmpl
      --controlPlaneAuthPolicy
      NONE
    Requests:
      cpu:  10m
    Environment:
      POD_NAME:        (v1:metadata.name)
      POD_NAMESPACE:   (v1:metadata.namespace)
      INSTANCE_IP:     (v1:status.podIP)
    Mounts:
      /etc/certs from istio-certs (ro)
      /sock from uds-socket (rw)
  Volumes:
   istio-certs:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  istio.istio-mixer-service-account
    Optional:    true
   uds-socket:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Progressing    True    NewReplicaSetAvailable
  Available      True    MinimumReplicasAvailable
OldReplicaSets:  <none>
NewReplicaSet:   istio-policy-755cf49c4f (1/1 replicas created)
Events:          <none>

check version for policy pods
(The following pod has version 1.0.3)

k -n istio-system describe pod istio-policy-688f99c9c4-l4sww
Name:           istio-policy-688f99c9c4-l4sww
Namespace:      istio-system
Node:           docker-for-desktop/<IP_ADDRESS>
Start Time:     Tue, 19 Feb 2019 15:19:13 -0800
Labels:         app=policy
                istio=mixer
                istio-mixer-type=policy
                pod-template-hash=2449557570
Annotations:    scheduler.alpha.kubernetes.io/critical-pod=
                sidecar.istio.io/inject=false
Status:         Running
IP:             <IP_ADDRESS>
Controlled By:  ReplicaSet/istio-policy-688f99c9c4
Containers:
  mixer:
    Container ID:  docker://8a6e4e2be58542838a3eb40a12f7d298224df9c9ec0b13e0d25c147c51e2c435
    Image:         docker.io/istio/mixer:1.0.3
    Image ID:      docker-pullable://istio/mixer@sha256:8eda1ea2dba1c299d98edac38f4c98fb39f87ebf9af13161b1e2ac07db8f8386
    Ports:         9093/TCP, 42422/TCP
    Host Ports:    0/TCP, 0/TCP
    Args:
      --address
      unix:///sock/mixer.socket
      --configStoreURL=k8s://
      --configDefaultNamespace=istio-system
      --trace_zipkin_url=http://zipkin:9411/api/v1/spans
    State:          Running
      Started:      Tue, 19 Feb 2019 15:19:14 -0800
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:     10m
    Liveness:  http-get http://:9093/version delay=5s timeout=1s period=5s #success=1 #failure=3
    Environment:
      GODEBUG:  gctrace=2
    Mounts:
      /sock from uds-socket (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from istio-mixer-service-account-token-dx2k8 (ro)
  istio-proxy:
    Container ID:  docker://85e64006a94caae0146eb3fc7bb3a881c76f66c49a72d685aa9f920dedf7b3f0
    Image:         docker.io/istio/proxyv2:1.0.3
    Image ID:      docker-pullable://istio/proxyv2@sha256:947348e2039b8b0e356e843ae263dc0c3d50abbf6cfe9d676446353f85b9ccb7
    Port:          15090/TCP
    Host Port:     0/TCP
    Args:
      proxy
      --serviceCluster
      istio-policy
      --templateFile
      /etc/istio/proxy/envoy_policy.yaml.tmpl
      --controlPlaneAuthPolicy
      NONE
    State:          Running
      Started:      Tue, 19 Feb 2019 15:19:15 -0800
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:  10m
    Environment:
      POD_NAME:       istio-policy-688f99c9c4-l4sww (v1:metadata.name)
      POD_NAMESPACE:  istio-system (v1:metadata.namespace)
      INSTANCE_IP:     (v1:status.podIP)
    Mounts:
      /etc/certs from istio-certs (ro)
      /sock from uds-socket (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from istio-mixer-service-account-token-dx2k8 (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          True
  PodScheduled   True
Volumes:
  istio-certs:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  istio.istio-mixer-service-account
    Optional:    true
  uds-socket:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
  istio-mixer-service-account-token-dx2k8:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  istio-mixer-service-account-token-dx2k8
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason                 Age                From                         Message
  ----     ------                 ----               ----                         -------
  Normal   Scheduled              35m                default-scheduler            Successfully assigned istio-policy-688f99c9c4-l4sww to docker-for-desktop
  Normal   SuccessfulMountVolume  35m                kubelet, docker-for-desktop  MountVolume.SetUp succeeded for volume "uds-socket"
  Normal   SuccessfulMountVolume  35m                kubelet, docker-for-desktop  MountVolume.SetUp succeeded for volume "istio-mixer-service-account-token-dx2k8"
  Normal   SuccessfulMountVolume  35m                kubelet, docker-for-desktop  MountVolume.SetUp succeeded for volume "istio-certs"
  Normal   Pulled                 35m                kubelet, docker-for-desktop  Container image "docker.io/istio/mixer:1.0.3" already present on machine
  Normal   Created                35m                kubelet, docker-for-desktop  Created container
  Normal   Started                35m                kubelet, docker-for-desktop  Started container
  Normal   Pulled                 35m                kubelet, docker-for-desktop  Container image "docker.io/istio/proxyv2:1.0.3" already present on machine
  Normal   Started                35m                kubelet, docker-for-desktop  Started container
  Normal   Created                35m                kubelet, docker-for-desktop  Created container
  Warning  DNSConfigForming       4m (x29 over 35m)  kubelet, docker-for-desktop  Search Line limits were exceeded, some search paths have been omitted, the applied search line is: istio-system.svc.cluster.local svc.cluster.local cluster.local

(The following pod has version 1.1.0-snapshot5)
k -n istio-system describe pod istio-policy-755cf49c4f-6ccvd
Name:           istio-policy-755cf49c4f-6ccvd
Namespace:      istio-system
Node:           docker-for-desktop/<IP_ADDRESS>
Start Time:     Tue, 19 Feb 2019 14:30:36 -0800
Labels:         app=policy
                chart=mixer
                heritage=Tiller
                istio=mixer
                istio-mixer-type=policy
                pod-template-hash=3117905709
                release=istio
                version=1.1.0
Annotations:    scheduler.alpha.kubernetes.io/critical-pod=
                sidecar.istio.io/inject=false
Status:         Running
IP:             <IP_ADDRESS>
Controlled By:  ReplicaSet/istio-policy-755cf49c4f
Containers:
  mixer:
    Container ID:  docker://7573c342827107c28958a6f6ff07e19f0e234c826fcf1887437c1563e087ed8b
    Image:         docker.io/istio/mixer:1.1.0-snapshot.5
    Image ID:      docker-pullable://istio/mixer@sha256:80b2506ed8bde3ca5402445ae5eca78b15a6469db8da3f2fd0b48db8480a6fd4
    Ports:         9093/TCP, 42422/TCP
    Host Ports:    0/TCP, 0/TCP
    Args:
      --monitoringPort=9093
      --address
      unix:///sock/mixer.socket
      --configStoreURL=mcp://istio-galley.istio-system.svc:9901
      --configDefaultNamespace=istio-system
      --useAdapterCRDs=true
      --trace_zipkin_url=http://zipkin:9411/api/v1/spans
    State:          Running
      Started:      Tue, 19 Feb 2019 14:30:39 -0800
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:     10m
    Liveness:  http-get http://:9093/version delay=5s timeout=1s period=5s #success=1 #failure=3
    Environment:
      GODEBUG:  gctrace=2
    Mounts:
      /etc/certs from istio-certs (ro)
      /sock from uds-socket (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from istio-mixer-service-account-token-dx2k8 (ro)
  istio-proxy:
    Container ID:  docker://a8879e4b0f758fd592391e1df716ec14ac90eaae39b5920764ee807fa61af5b3
    Image:         docker.io/istio/proxyv2:1.1.0-snapshot.5
    Image ID:      docker-pullable://istio/proxyv2@sha256:2329bed32fde5d3ed0c4d3f7f0594e8258573226c50406e7d25d0298cd119685
    Ports:         9091/TCP, 15004/TCP, 15090/TCP
    Host Ports:    0/TCP, 0/TCP, 0/TCP
    Args:
      proxy
      --domain
      $(POD_NAMESPACE).svc.cluster.local
      --serviceCluster
      istio-policy
      --templateFile
      /etc/istio/proxy/envoy_policy.yaml.tmpl
      --controlPlaneAuthPolicy
      NONE
    State:          Running
      Started:      Tue, 19 Feb 2019 14:30:40 -0800
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:  10m
    Environment:
      POD_NAME:       istio-policy-755cf49c4f-6ccvd (v1:metadata.name)
      POD_NAMESPACE:  istio-system (v1:metadata.namespace)
      INSTANCE_IP:     (v1:status.podIP)
    Mounts:
      /etc/certs from istio-certs (ro)
      /sock from uds-socket (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from istio-mixer-service-account-token-dx2k8 (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          True
  PodScheduled   True
Volumes:
  istio-certs:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  istio.istio-mixer-service-account
    Optional:    true
  uds-socket:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
  istio-mixer-service-account-token-dx2k8:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  istio-mixer-service-account-token-dx2k8
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason            Age               From                         Message
  ----     ------            ----              ----                         -------
  Warning  DNSConfigForming  3m (x67 over 1h)  kubelet, docker-for-desktop  Search Line limits were exceeded, some search paths have been omitted, the applied search line is: istio-system.svc.cluster.local svc.cluster.local cluster.local

now delete the pod  istio-policy-688f99c9c4-l4sww (having version 1.0.3), a new pod will be allocated and up, but the version still is 1.0.3.

if delete the pod istio-policy-755cf49c4f-6ccvd  (having version 1.1.0-snapshot.5), a new pod will be allocated and up, and the version is 1.1.0-snapshot.5

Version
kubectl version: 1.10.3
istioctl version: 1.0.3 / 1.1.0-snapshot6
Installation
1.0.3 was installed by:
helm template install/kubernetes/helm/istio --name istio --namespace istio-system > $HOME/istio.yaml
$ kubectl create namespace istio-system
$ kubectl apply -f $HOME/istio.yaml

Hi, it's not possible to upgrade from template install using "helm upgrade" - the upgrade command only works when you have Tiller installed:
https://istio.io/docs/setup/kubernetes/helm-install/#option-2-install-with-helm-and-tiller-via-helm-install
You can choose one of the following:

Use template install process, then upgrade by repeating the same steps for the target version.
Use helm tiller to install the source version, then you can use "helm upgrade" command.

@ostromart Just tried with option2 (helm tiller to install and upgrade) on a different cluster and the issue still exist.

Could you share your entire install and upgrade command history so I can try to repro?

Actually same command "helm upgrade" was used for the install and upgrade:
install
/bin/bash -c ./helm upgrade --tiller-namespace=kube-system --version=1.0.5 --namespace=istio-system  --install istio wmt/istio

upgrade:
/bin/bash -c ./helm upgrade --tiller-namespace=kube-system  --version=1.1.0-snapshot.6  --namespace=istio-system  --install istio-init ./roles/istio/files/1.1.0-snapshot.6/istio-init

/bin/bash -c ./helm upgrade --tiller-namespace=kube-system  --version=1.1.0-snapshot.6  --namespace=istio-system  --install istio ./roles/istio/files/1.1.0-snapshot.6/istio

@linsun this bug was discovered when testing the pod disruption budget in https://github.com/istio/istio/issues/9951 . can you take a look?

@linsun  actually nevermind.  my mistake

I tried again on a clean cluster, and the issue consistently happening:
Steps:
kubenetes version: v1.12.4
install istio 1.0.5 by:
/bin/bash -c ./helm upgrade --tiller-namespace=kube-system  --version=1.0.5 --values=./roles/istio/files/1.0.5/wmt-values-isti... --namespace=istio-system --tls --install istio ./roles/istio/files/1.0.5/istio

upgrade to istio 1.1.0-rc.0 by:
/bin/bash -c ./helm upgrade --tiller-namespace=kube-system  --version=1.1.0-rc.0 --values=./roles/istio/files/1.1.0-rc.0/wmt-values...  --namespace=istio-system --tls --install istio-init ./roles/istio/files/1.1.0-rc.0/istio-init

/bin/bash -c ./helm upgrade --tiller-namespace=kube-system  --version=1.1.0-rc.0 --values=./roles/istio/files/1.1.0-rc.0/wmt-values...  --namespace=istio-system --tls --install istio ./roles/istio/files/1.1.0-rc.0/istio

Then I see 3 istio-policy pods with version 1.0.5 which is old and 1 istio-policy pod running with version 1.1.0-rc.0.

P0 for @ostromart to investigate.  We'll adjust from there.   @ostromart says he can look into it this week

I'm not sure if this is something that's started to occur in recent builds but with 1.1-rc0 and rc1 I keep hitting
https://github.com/istio/istio/issues/11990, specifically the first problem with the PodDisruptionBudget, so I haven't been able to complete a tiller upgrade to repro the problem in this issue. Going to ping that issue to see if there's a workaround.
helm upgrade istio install/kubernetes/helm/istio --namespace istio-system
Error: UPGRADE FAILED: no PodDisruptionBudget with the name "istio-galley" found
kc get PodDisruptionBudget -n istio-system
...
istio-galley           1               N/A               0                     1h
...

Actually @linsun could you take a look at this?  There is an issue with the pod disruption budget introduced in
#9951 that's blocking @ostromart from reproducing this issue.

I replied in #9951 it is an issue of helm: https://github.com/helm/helm/issues/2914
Is this only impact downgrade?  If so, maybe we should not even supportdowngrade using helm upgrade @duderino @ostromart @sdake @morvencao

https://github.com/istio/istio/pull/12177 should fix this issue.

hmmmm, the last nightly doesn't have this fix yet:  https://gcsweb.istio.io/gcs/istio-prerelease/daily-build/release-1.1-20190301-09-16/
@utka I wonder if it's time for rc2?

on it

I replied in #9951 it is an issue of helm: helm/helm#2914
Is this only impact downgrade? If so, maybe we should not even supportdowngrade using helm upgrade @duderino @ostromart @sdake @morvencao

@linsun not sure why we would support downgrade using helm upgrade when there is a helm rollback operation.
I am more concerned with the automated MTLS configuration removal that has broken upgrade.  I am not sure how to fix this problem without a revert.  @morvencao any ideas?
Cheers
-steve

@sdake Do you mean this issue: https://github.com/istio/istio/issues/12154?

@sdake Do you mean this issue: #12154?

yes I was jammed up behind that in https://github.com/istio/istio/issues/11990#issuecomment-468891090.
Cheers
-steve

@zhqworld @ostromart @morvencao @linsun @sdake please try https://github.com/istio/istio/releases/tag/1.1.0-rc.2

@duderino I think we need https://github.com/istio/istio/pull/12173 merged before verifying this. Or we will get the follow error with helm upgrade
root@morven1:~/istio-1.0.3# helm install ./install/kubernetes/helm/istio --name istio --namespace istio-system
root@morven1:~/istio-1.0.3# cd ../istio-1.1.0-rc.2/
root@morven1:~/istio-1.1.0-rc.2# helm upgrade istio-init --install --namespace istio-system install/kubernetes/helm/istio-init --tls
root@morven1:~/istio-1.1.0-rc.2# helm list --tls | grep istio
istio                 	1       	Sun Mar  3 17:59:32 2019	DEPLOYED	istio-1.0.3                 	istio-system
istio-init            	1       	Sun Mar  3 18:02:02 2019	DEPLOYED	istio-init-1.1.0            	istio-system
root@morven1:~/istio-1.1.0-rc.2# helm upgrade istio --namespace istio-system install/kubernetes/helm/istio --tls
Error: UPGRADE FAILED: no MeshPolicy with the name "default" found
root@morven1:~/istio-1.1.0-rc.2# helm history istio --tls
REVISION	UPDATED                 	STATUS  	CHART      	DESCRIPTION
1       	Sun Mar  3 17:59:32 2019	DEPLOYED	istio-1.0.3	Install complete
2       	Sun Mar  3 18:04:00 2019	FAILED  	istio-1.1.0	Upgrade "istio" failed: no MeshPolicy with the name "defa...

And after patch the fix in https://github.com/istio/istio/pull/12173, the helm upgrade/rollback is smooth AND old policy and telemetry pods killed after new ones start running:
root@morven1:~/istio-1.0.3# helm install --name istio --namespace istio-system install/kubernetes/helm/istio --tls
NAME:   istio
LAST DEPLOYED: Sun Mar  3 18:18:07 2019
NAMESPACE: istio-system
STATUS: DEPLOYED
...
root@morven1:~/istio-1.0.3# helm list --tls | grep istio
istio                 	1       	Sun Mar  3 18:18:07 2019	DEPLOYED	istio-1.0.3                 	istio-system
root@morven1:~/istio-1.0.3# kubectl -n istio-system get pod
NAME                                     READY     STATUS    RESTARTS   AGE
istio-citadel-78df8c67d9-gbtwj           1/1       Running   0          1m
istio-egressgateway-674686c846-5gq7l     1/1       Running   0          1m
istio-galley-58f566cb66-rz8xz            1/1       Running   0          1m
istio-ingressgateway-6bbdd58f8c-t4k5j    1/1       Running   0          1m
istio-pilot-7fc859bf4f-khcxl             2/2       Running   0          1m
istio-policy-68797d879-jrwjq             2/2       Running   0          1m
istio-sidecar-injector-b88dfb954-rq2p5   1/1       Running   0          1m
istio-telemetry-68787476f4-k5skh         2/2       Running   0          1m
prometheus-76db5fddd5-8zftq              1/1       Running   0          1m
root@morven1:~/istio-1.0.3# cd ../istio-1.1.0-rc.2/
root@morven1:~/istio-1.1.0-rc.2# helm upgrade istio-init --install --namespace istio-system install/kubernetes/helm/istio-init --tls
Release "istio-init" does not exist. Installing it now.
NAME:   istio-init
LAST DEPLOYED: Sun Mar  3 18:20:13 2019
NAMESPACE: istio-system
STATUS: DEPLOYED
...
root@morven1:~/istio-1.1.0-rc.2# helm list --tls | grep istio
istio                 	1       	Sun Mar  3 18:18:07 2019	DEPLOYED	istio-1.0.3                 	istio-system
istio-init            	1       	Sun Mar  3 18:20:13 2019	DEPLOYED	istio-init-1.1.0            	istio-system
root@morven1:~/istio-1.1.0-rc.2# helm upgrade istio --namespace istio-system install/kubernetes/helm/istio --tls
Release "istio" has been upgraded. Happy Helming!
LAST DEPLOYED: Sun Mar  3 18:21:07 2019
NAMESPACE: istio-system
STATUS: DEPLOYED
...
root@morven1:~/istio-1.1.0-rc.2# helm history istio --tls
REVISION	UPDATED                 	STATUS    	CHART      	DESCRIPTION
1       	Sun Mar  3 18:18:07 2019	SUPERSEDED	istio-1.0.3	Install complete
2       	Sun Mar  3 18:21:07 2019	DEPLOYED  	istio-1.1.0	Upgrade complete
root@morven1:~/istio-1.1.0-rc.2# kubectl -n istio-system get pod
NAME                                     READY     STATUS      RESTARTS   AGE
istio-citadel-78857bd5fb-b2vgb           1/1       Running     0          48s
istio-galley-5dd775669d-n6h8s            1/1       Running     0          48s
istio-ingressgateway-844bc594bb-vdnmp    1/1       Running     0          48s
istio-init-crd-10-kkmqp                  0/1       Completed   0          1m
istio-init-crd-11-kl92v                  0/1       Completed   0          1m
istio-init-crd-certmanager-10-x76xw      0/1       Completed   0          1m
istio-init-crd-certmanager-11-5xj5l      0/1       Completed   0          1m
istio-pilot-b45fbf6f4-wh8zk              2/2       Running     0          48s
istio-sidecar-injector-bbd48c989-5wmjr   1/1       Running     0          47s
istio-telemetry-6cb48cd948-j8b2j         2/2       Running     1          48s
prometheus-89bc5668c-qdgx9               1/1       Running     0          48s

@morvencao I pulled your helm mtls revert which should shortly merge in release-1.1, and then rebased on release-1.1 which has this PR: https://github.com/istio/istio/pull/12177.  I receive duplicate mixer and pilot instances in that scenario.
Cheers
-steve

OK, so https://github.com/istio/istio/pull/12173 has been merged.  Now @sdake says need someone to tackle the duplicate pilot and mixer instances (see also https://github.com/istio/istio/pull/12173#issuecomment-469362977)

@morvencao could you take a look at this today?  Let me know where things are at the end of your day.  Thanks.

This problem was fixed. Verified the fix works for 1.0.6 -> release-1.1@6662fc7
cd ../istio-1.0.6/
helm install install/kubernetes/helm/istio --name istio --namespace istio-system --tls
git checkout release-1.1
helm upgrade --install istio-init install/kubernetes/helm/istio-init --namespace istio-system --tls
helm upgrade istio install/kubernetes/helm/istio --namespace istio-system --tls

the result is:
root@cyang1:~/go/src/istio.io/istio# kubectl get po
NAME                                      READY     STATUS      RESTARTS   AGE
istio-citadel-5bf5488468-6sgjv            1/1       Running     0          5m
istio-galley-96d888b56-hhj7q              1/1       Running     0          5m
istio-ingressgateway-665ffb67b8-nn9v7     1/1       Running     0          5m
istio-ingressgateway-665ffb67b8-pbqjd     1/1       Running     0          3m
istio-init-crd-10-nmqnv                   0/1       Completed   0          6m
istio-init-crd-11-59z9m                   0/1       Completed   0          6m
istio-init-crd-certmanager-10-24k4f       0/1       Completed   0          6m
istio-init-crd-certmanager-11-7dsw7       0/1       Completed   0          6m
istio-pilot-7c78c7dddb-sswnq              2/2       Running     0          5m
istio-sidecar-injector-744f68bf5f-v5d55   1/1       Running     0          5m
istio-telemetry-577955c48d-4hr6h          2/2       Running     0          5m
prometheus-89bc5668c-x9dkt                1/1       Running     0          5m

There is no duplicated mixer pod instances.

Thanks @clyang82 , the same results from my local test.
@sdake The fix https://github.com/istio/istio/pull/12177 should be already in release 1.1 and RC2, while this one https://github.com/istio/istio/pull/12173 not.
Anyway, the latest release-1.1 branch should contains the two fix,  if that goes well, we should have the fix for next RC.

@morvencao everything is gtg - upgrade and rollback work fine.  Thanks for your contributions!
This issue can be closed by the PMs if they wish.

Thank you much @sdake @clyang82 @morvencao this is awesome, closing it as you all have validated it.

@linsun @sdake @clyang82 @morvencao thanks a ton!

still seeing it when upgrading from 1.0.6 to 1.1.4


support for std.uni store / load trie entry table

remi.thebault (@rtbo) reported this on 2016-12-28T15:20:28Z
Transfered from https://issues.dlang.org/show_bug.cgi?id=17038
Description
Use case:
  - efficiently map a dchar to some unicode properties not supported in std.uni

example of workflow:
 - in a separate program:
    1. build a AA of the required mapping
    2. build a std.uni.Trie out of this AA (e.g. using std.uni.codepointTrie)
    3. store the trie entry tables in a generated D module

 - in the application/library code:
    4. rebuild the Trie from the entry tables.

Steps 1 to 3 are already supported by std.uni, though not documented (I had to dive into std.uni code to understand how it must be done).

Step 4 is currently not directly feasible as the Trie ctor that take the entry tables is private.

I could workaround by copying the following 'TrieEntry' struct and 'asTrie' function in my own code (which still happen to call the private ctor, but both dmd and ldc seem to accept it somehow)

struct TrieEntry(T...)
{
    size_t[] offsets;
    size_t[] sizes;
    size_t[] data;
}

@safe pure nothrow auto asTrie(T...)(in TrieEntry!T e)
{
    return const(CodepointTrie!T)(e.offsets, e.sizes, e.data);
}

remi.thebault commented on 2017-01-08T09:57:57Z
In addition, it should be supported to store the table for both 32 bits and 64 bits.

Currently the Trie.store method outputs D code for the running architecture only.
64 bits code output cannot compile in 32 bits.
32 bits code output will compile in 64 bits, but will likely give false results.

And what about endianness?


Draft: Version 2 refactor, breaking changes but much better plugin and API

How to Test

Ensure all your keymaps/commands/autocmds/functions still work (after updating your config with any API changes that might be needed)
Ensure documentation is accurate for new APIs

Testing for Regressions
I have tested the following:

[ ] Triggering keymaps from legendary.nvim in all modes (normal, insert, visual)
[ ] Creating keymaps via legendary.nvim, then triggering via the keymap in all modes (normal, insert, visual)
[ ] Triggering commands from legendary.nvim in all modes (normal, insert, visual)
[ ] Creating commands via legendary.nvim, then running the command manually from the command line
[ ] augroup/autocmds created through legendary.nvim work correctly

@olimorris would you mind using this branch for a while and helping me file issues for anything that doesn't work and any documentation that is not accurate on this branch?
I think I've covered everything documentation-wise but it's hard to be sure.
Note that there are some breaking changes on this branch, I'll do my best to summarize them:

Some config options have moved, see new config.lua or the README.md changes regarding config in this PR
The top-level API functions have changes like bind_keymaps() changed to just keymaps(), others following the same patterns
The helpers have moved to legendary.toolbox
kind was removed, instead you can use require('legendary.toolbox').is_keymap() (there's a function for all the types)
the which-key integration functions are no longer on the top-level require('legendary'), those are at require('legendary.integrations.which-key'), however if config.which_key.auto_register == true, they will still be auto-registered

These changes cut my legendary.nvim startup time basically in half too ❤
I hope it does the same for you, I'm curious if you'll see the same results.

Alright. A pretty smooth move as you can see by 935e2b3e62f141a186915bc22ce121012cb056f4. Only issue was I had actually messed up an autocmd. The old version of Legendary seemed to be forgiving of that. The new version cried and I was able to figure it out.
One thing I've noticed, I get a lot of LSP duplicates now:

I'd even borrowed your helper function:
  if
    #vim.tbl_filter(
      function(keymap) return (keymap.desc or ""):lower() == "lsp: find references" end,
      vim.api.nvim_buf_get_keymap(bufnr, "n")
    ) > 0
  then
    return {}
  end

LSP duplicates should be resolved now, the issue was I forgot to filter by buffer by default. It's now only showing items that either don't have any buffer option set, or have the buffer option set to current buffer.

Also curious if your startup time is better 👀
If you've got packer.nvim profiling enabled you can run :PackerProfile to get data like this:

Ah, I took a look at your commit in your dotfiles repo -- I'm not sure how that autocmd layout worked before either actually 😂
The augroups are supposed to be a list of autocmds. Looks like you've got it worked out now though. The current behavior is the intended behavior.

Ah, I took a look at your commit in your dotfiles repo -- I'm not sure how that autocmd layout worked before either actually 😂
The augroups are supposed to be a list of autocmds. Looks like you've got it worked out now though. The current behavior is the intended behavior.

Hahaha. How that ended up in that state, I don't know. #LateNightDotfileEditing
Shall test the startup time later too.

What on earth have you done to it?! Speed increase is incredible:
Version 2:
Oli.plugins.legendar   3.26    1.97 ▊
legendary              0.69    0.42 ▏
legendary.api.cmds     0.25    0.15
legendary.data.state   0.21    0.12
legendary.data.iteml   0.17    0.10
legendary.ui.scratch   0.15    0.09
legendary.ui           0.12    0.07
legendary.lua          0.10    0.06
legendary.data.augro   0.09    0.06
legendary.api.cache    0.06    0.03

Version 1:
Oli.plugins.legendar   7.98    4.72 █▊
legendary              4.19    2.48 ▉
legendary.bindings     2.39    1.42 ▌
legendary.cmds         1.14    0.68 ▎
legendary.config       0.61    0.36 ▏
legendary.compat.whi   0.60    0.36 ▏
legendary.utils        0.59    0.35 ▏
legendary.formatter    0.57    0.34 ▏
legendary.types        0.57    0.33 ▏
legendary.scratchpad   0.56    0.33 ▏
legendary.helpers      0.56    0.33 ▏

👏🏼 👏🏼 👏🏼

LSP duplicates should be resolved now, the issue was I forgot to filter by buffer by default. It's now only showing items that either don't have any buffer option set, or have the buffer option set to current buffer.

I'm still getting duplicates btw

What on earth have you done to it?! Speed increase is incredible

Different architecture. Keymaps are now parsed based on a schema upfront and then stored internally in a very simple format, basically tables like, for keymaps for example:
{
  -- your actual keys
  keys = string,
  description = string,
  -- implementation is converted to mode_mappings
  -- even if there's only 1, makes processing a lot
  -- simpler and faster
  mode_mappings = {
    n = { implementation = string|function, opts = table|nil }
    v = { implementation = string|function, opts = table|nil }
    x = { implementation = string|function, opts = table|nil }
    c = { implementation = string|function, opts = table|nil }
    s = { implementation = string|function, opts = table|nil }
    t = { implementation = string|function, opts = table|nil }
    i = { implementation = string|function, opts = table|nil }
  }
}

and for commands:
{
  cmd = string,
  implementation = function|string|nil,
  description = string,
  opts = table,
  unfinished = boolean,
}

This makes the initial processing a lot less complex, and a lot faster, and specifically makes validation a lot simpler, enough so that I'm just using vim.validate now.

I'm still getting duplicates btw

I'll check out your config and work on this now

Oh interesting, I'm getting duplicates too. I didn't realize.

lol. I was checking for buf but the actual option is buffer 🙈
are you still getting duplicates now after latest commit?

Any plans to change the visual selection part anytime soon?

Assuming you mean what we talked about in the GitHub discussion. Does this solve your problem, or no?
  if require('legendary.toolbox').is_visual_mode() then
    -- do visual mode stuff
  else
    -- do normal mode stuff
  end

or alternatively, using per-mode mappings?

So that's complete parity of features compared to V1

A couple new features too 😉
Namely:

Option to put scratchpad in a floating window
Command to toggle the scratchpad open and closed (mostly useful if you're using a floating window)
Ability to define command which accept arguments
Sorting most recently used items to the top (previously just the single most recent would be at the top)

Any plans to change the visual selection part anytime soon?

Assuming you mean what we talked about in the GitHub discussion. Does this solve your problem, or no?
  if require('legendary.toolbox').is_visual_mode() then
    -- do visual mode stuff
    -- there is also this if you need it
    local marks = require('legendary.toolbox').get_marks()
  else
    -- do normal mode stuff
  end

or alternatively, using per-mode mappings?

No this is great. I've now updated for this.

@olimorris I’d like to get some input on the best way to communicate these changes to users, here are some ideas I came up with:

Just merge the breaking changes, users will check GitHub where I’ll have the pinned issue for breaking changes (definitely not ideal, I’d prefer to be more gently and considerate to users)
Keep the bind_* function variants, and delegating internally to the new versions, and display a message via vim.deprecate, tag v2.0.0, then remove them and tag v2.0.1. The message will instruct users to either pin to v1.0.0 tag (which I’ll create before merging this PR), or update to v2-compatible configs
Something else?

I don’t have a Reddit account anymore so I can’t post on the subreddit, unfortunately I think that’s where the most people would see it.

Speaking from experience, I can definitely recommend adding deprecation notices. I think people are fine with being pressured (by way of messages when they start Neovim) into updating their config to a newer api, so long as their config isn't broken. That feels like the plugin author has actively made a decision to burn their config (🔥) without their permission. I've been hugely guilty of that in the past (😿).
I defo think Reddit is the place to show case the plugin again. The crowd can be very hit and miss but it pulls in the numbers! I estimate I received about 30+ stars from my last Reddit post and I lost tracked of total clones.

Agreed 100%.
Went ahead and added deprecation notices in 5d46c0c and did my best to remap things to their new equivalents.

Also, just received this error after updating:

Could this be handled gracefully?

That message is using vim.deprecate and it is copying the value to the correct config value in the new setup.
How would you suggest I handle it more gracefully?

Nice, I like that a lot. Updated.

I think I wanna re-record the demo gif in the README.md and then this PR is lookin pretty good 😎

Planning to release this to master tomorrow some time, probably in the morning. Trying to decide if I should make a Reddit account just to post an update there lol

Just read the new README and had some thoughts (feel free to discard every single one of them):
In Safari, I find that mp4 videos do not autoplay in GitHub, whilst they do in Chrome. Perhaps that is an issue on my end but worth testing to make sure. Because if I come to Legendary for the first time, I'm not sure I would totally understand what we mean by a Legend (especially if English is not a primary language). In VS Code I see Cmd+Shift+P being called the Command Palette. Is there a way to rephrase the introductory sentence to be more explicit about what Legendary does? Something crude like:
"Define your keymaps, commands, and autocommands as simple Lua tables, building a legend (or command palette) at the same time."
I have a bit of a dislike for videos as the main media in a README as well because it requires me to watch 10+ seconds of someones plugin to try and workout what it does (try watching the leap.nvim one). Videos are absolutely useful for demonstrating features once the reader has grasped the elements of the plugin but upfront, I'd love a nice logo incorporating a few snapshots of Legendary.

In Safari, I find that mp4 videos do not autoplay in GitHub, whilst they do in Chrome

Yeah, I was having a hard time keeping the gif file size small enough. I can give it another shot as a gif.
As far as the tagline, I can do something like: "Define your keymaps, commands, and autocommands as simple Lua tables, building a legend (like VS Code's Command Palette) at the same time."
Any specific thoughts on what a logo should look like? I had played with the idea but I'm bad at designing stuff like that 😂

Okay, got a gif working with appropriate file size.

As far as the tagline, I can do something like: "Define your keymaps, commands, and autocommands as simple Lua tables, building a legend (like VS Code's Command Palette) at the same time."

Personally, I think anyone new to the plugin will grasp what it does rig away. Also, worth updating awesome-neovim with the new tagline?

Any specific thoughts on what a logo should look like? I had played with the idea but I'm bad at designing stuff like that 😂

Haha I should not be giving any advice out on this 🤣. For my tmux-pomodoro-plus plugin I found a cool stock image and stuck it over the tmux logo and made it a banner. Could always be an option in place of just a cool font and some screenshots placed together in a banner.

Also, worth updating awesome-neovim with the new tagline?

Good call, PR: https://github.com/rockerBOO/awesome-neovim/pull/599

reddit post: https://www.reddit.com/r/neovim/comments/yqiefg/legendarynvim_now_much_faster_with_new_features/

Congratulations @mrjones2014! A huge milestone.

Thanks! I'm really happy with the way it turned out, and tbh I'm happy to be free of Teal. Rewriting to Teal about 6 months ago really did not have the payoff I'd hoped. Sure, static types are nice, but using EmmyLua annotation comments with the Lua language server is close enough and the tooling surrounding Teal (LSP, editor support, compiler, etc.) are pretty sub-par developer experience IMHO.

It will be so so so much easier to reason about when fixing bugs and adding new features now as well. The architecture before was getting to be a bit of a mess.


Add an API that combines autocomplete with arbitrary cypher queries

Background: The current SciGraph API breaks down functionalities into separate areas: lexical, annotation, graph. This is a useful high level breakdown but frequently we need to combine these. This is one thing the old oboedit tool got right that has never been duplicated in Protege, the ability to combine advanced lexical queries with graph queries (cc @dosumis).
In this particular case, we would like to be able to filter auto-complete results based on a cypher query. A simple but common example is an autocomplete field that is restricted to be only transitive reflexive subclasses of a given ontology class, where the members are not in a given subset. See https://github.com/geneontology/noctua/issues/295 for example. A more sophisticated example would use taxon constraints: e.g. restrict this autocomplete to classes applicable for mammalia.
Because this is potentially expensive we may want to restrict the set of cypher queries that can be used to something pre-defined by the SciGraph admin, specified using a yaml file analogous to the current dynamic option. The yaml would specify the name of the autocomplete (e.g. 'biological process mammalia') and map this to a cypher query. There could also be a hint as to whether the cypher should be applied after the search to filter out entries, or prior to the search, to create a whitelist.
I like the idea of a power api user being able to specify arbitrary cypher but don't know the implications in terms of killer queries.

Another use case for this capability is in the ObOne browser experiment that Mark P and I have been working with. Specifically, we are currently using the /graph/neighbors endpoint to get all neighbors of distance 1 from the currently active node. However, this endpoint has no ability to have paged results (e.g., count, startIndex and limit parameters). It would be nice to be able to give the user the ability to filter these neighbors based on the same criteria used for autocomplete, so that as the user types into a Filter By field, the set of neighbors seen is pruned to match the filter.
So basically, an optional parameter to /autocomplete that indicated that candidates should be neighbors of distance=1 would be useful. Alternatively, adding a filterBy expression to /neighbors would work.

On 2 Jun 2016, at 6:22, Dan Keith wrote:

Another use case for this capability is in the ObOne browser
experiment that Mark P and I have been working with. Specifically, we
are currently using the /graph/neighbors endpoint to get all
neighbors of distance 1 from the currently active node. However, this
endpoint has no ability to have paged results (e.g., count,
startIndex and limit parameters).

sounds like it would be useful to have this

It would be nice to be able to give the user the ability to filter
these neighbors based on the same criteria used for autocomplete, so
that as the user types into a Filter By field, the set of neighbors
seen is pruned to match the filter.

I'm not really sure about the utility of this, as to do anything useful
that isn't hardcoded to any one use case you'd need the full power of a
query language like cypher
But for the text filtering case, you could piggy back this off the
autocomplete call, but that feels an odd mixing of concerns

So basically, an optional parameter to /autocomplete that indicated
that candidates should be neighbors of distance=1 would be useful.
Alternatively, adding a filterBy expression to /neighbors would work.

You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
https://github.com/SciGraph/SciGraph/issues/177#issuecomment-223289487


Export to TensorRT engine on a Jetson Nano gives 'Killed'

Hi,
When I tried to export to the tensorrt .plan engine file on my nano, it gives:
Building engine...
Building FP16 core model...
Building accelerated plugins...
Applying optimizations and building TRT CUDA engine...

Killed

Here are the things I did:

I cloned the 19.10 branch.
I retrieved one of the pretrained models from here.
I exported it to onnx via retinanet export commands on my docker container. I double-checked that the model is valid by running the commands on my dev computer (which has a Geforce RTX 2070) and it gave very good results (and it's suuper fast, good job guys!!)
I followed the instructions for the Jetson xavier but on my nano.

Everything worked so far but it failed on the export command at the end to generate the .plan engine
Is it due to the architecture of the nano that's different from the xavier? or a memory issue? (Should I decrease workspace_size?)
I saw in #140 that you succeeded to make it run on the nano and you mentionned a sm number..
What are the changes to make in order to make it run on nano? (even if it's slow)

What's your Jetpack version? 4.3? Use this: cat /etc/nv_tegra_release
What model are you using?
The sm architecture is for DeepStream CMakelists.txt file, but for your case the export step is failing for which you don't need to change any variables.

I get this:
# R32 (release), REVISION: 3.1, GCID: 18186506, BOARD: t210ref, EABI: aarch64, DATE: Tue Dec 10 06:58:34 UTC 2019

19.10 should work fine for this, make sure that the branch being used on the Nano and 2070 is 19.10. Try to use ResNet18/34

I double checked everything and actually used your trained retinanet18 model for conversion... I don't see what's wrong it just killed when running the c++ export command... Didn't you change anything to make it work on the nano? looks like there is a memory issue Idk because when I ran the onnx2trt and I decreased the workspace it succeeds but gives wrong results.. What do you suggest me to do?

Let's try to debug this:

Add this set(CMAKE_CUDA_FLAGS "-g -G") here

cmake and make -j again

sudo apt install gdb valgrind on Nano

Run gdb ./export

Inside GDB: r model.onnx engine.plan (model.onnx should be in same directory as export executable)

Wait for it to fail, and when it fails enter backtrace

Paste the output of backtrace here

Not sure if that'll help but did you select max performance option on Nano and/or add swap memory?

@yashnv, I followed the steps above so when I insert set(CMAKE_CUDA_FLAGS "-g -G") as you mentionned here and run cmake -DCMAKE_CUDA_FLAGS="--expt-extended-lambda -std=c++14" .. and make -j it throws me an error like:
error: __host__ or __device__ annotation on lambda requires --expt-extended-lambda nvcc flag
sorry, I don't know much with c++.. I don't understand why the flag was not passed successfully
I tried to run the gdb without your instructions and it gives:
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
Building engine...
[New Thread 0x7f83b671c0 (LWP 11580)]
Building FP16 core model...
Building accelerated plugins...
Applying optimizations and building TRT CUDA engine...
[New Thread 0x7f5cc351c0 (LWP 11629)]
[Thread 0x7f5cc351c0 (LWP 11629) exited]
[Thread 0x7f83b671c0 (LWP 11580) exited]

Program terminated with signal SIGKILL, Killed.
The program no longer exists.
(gdb) backtrace
No stack.

@Sushil-Thapa is there a way to select the performance on Nano? does that affect memory? Idk about adding swap mem... @yashnv did you guys add swap memory to make it work?

change set(CMAKE_CUDA_FLAGS "-g -G") to set(CMAKE_CUDA_FLAGS "--expt-extended-lambda -std=c++14 -g -G"), and try again.

Actually, I managed to export it, just needed to reboot the nano. Mem issues.. Closing it. Thank you very much!

I was talking about this option. (details here)
$ sudo nvpmodel -m [mode]
Great that you got it solved with the reboot.
I managed to export it but didn't see much improvement with the TensorRT in terms of speed.
@lafius What was the max speed(fps) that you were able to achieve in your detection on the nano?

I had the same problem, I just found out that converting engine would cost about 50% of memory. so this might be the memory issue.
after decreasing the workspace size, the issue solved.
btw, is anyone know how to configure nano to use much more memory until it kills the process? I believe using about 50% of memory is very common in applications.


archlinux support + tool updates + misc improvements

There are many improvements in this pull request.

multi-distribution support

archlinux support (although some tools are not guaranteed to work or still lacking install-root script)
differentiate between debian and ubuntu if needed

installation of python based tools inside ctftools virtualenv

using the new ctf-tools-pip command as a wrapper
some pip installable tools are already converted
would fix #13

possibility for installation of multiple different gdb plugins
Added archlinux based Dockerfile
Additional tools, updates to tools
misc fixes

Thanks! That is an incredible amount of changes. It looks like TravisCI is failing before even getting to the testing -- looks like a Travis issue. I'll look into it soon!

Ok, I give up on Travis. I'm going to sync it and figure it out later :-)

Thanks for all the work!!!


moving 'nub' for efficiency

I saw
catMaybes (nub x)

in student code.
Due to the non-linear runtime complexity of nub, the linear runtime complexity of catMaybes, and the fact that catMaybes never makes the list longer, performing these two operations the other way around should always be better.
The cases with lefts and rights are analogous.

There are a few further, similar potential rules that could be added:
With reverse:
    - warn: {lhs: catMaybes (reverse x), rhs: reverse (catMaybes x)}
    - warn: {lhs: lefts (reverse x), rhs: reverse (lefts x)}
    - warn: {lhs: rights (reverse x), rhs: reverse (rights x)}

(not necessarily big performance wins here, but at least the reverse might get to work on a shorter list).
With cycle:
    - warn: {lhs: catMaybes (cycle x), rhs: cycle (catMaybes x)}
    - warn: {lhs: lefts (cycle x), rhs: cycle (lefts x)}
    - warn: {lhs: rights (cycle x), rhs: cycle (rights x)}

(where an additional benefit is that catMaybes etc. need to process fewer elements after the rule applied).
With sort:
    - warn: {lhs: catMaybes (sort x), rhs: sort (catMaybes x)}
    - warn: {lhs: lefts (sort x), rhs: sort (lefts x)}
    - warn: {lhs: rights (sort x), rhs: sort (rights x)}

(seems correct given the Ord instances for Maybe and Either, and could be very beneficial again due to the non-linear runtime complexity of sorting).

I was also thinking about whether there could be similar rules for functions like delete :: Eq a => a -> [a] -> [a], but it seems less clear what to do there.

Another possible direction would be to also consider functions from the containers and/or extra packages, like nubOrd :: Ord a => [a] -> [a] and nubSort :: Ord a => [a] -> [a]

Moving things like reverse/sort to work on fewer elements makes a lot of sense, as does nub. Happy to merge this PR as is, or let you extend it for the sort/reverse cases too - let me know.
In most cases I think avoiding nub is the right thing to do, using nubOrd or similar. But it would be easier if there was a non-quadratic nub-like in the base library, since I'm a little wary about recommending another dependency. I think that's definitely a wart in the standard library (nub should probably be deleted entirely, it's such a massive foot-gun).

I've added the corresponding rules for reverse and sort.
I've not added the rules for cycle since you did not mention/comment on them.
I see what you mean about "nub shouldn't even exist". Also why you are not currently having Hlint propose to replace nub by nubOrd or nubSort (them not being in base). Actually, my thinking about them here wasn't even to propose such a replacement. Only to also add rules like
    - warn: {lhs: catMaybes (nubOrd x), rhs: nubOrd (catMaybes x)}
    - warn: {lhs: lefts (nubOrd x), rhs: nubOrd (lefts x)}
    - warn: {lhs: rights (nubOrd x), rhs: nubOrd (rights x)}

to the section https://github.com/ndmitchell/hlint/blob/11898f6371ea6175f11e1a7049e85d2d8c1f7e96/data/hlint.yaml#L919-L923 That is, when people are already using the extra dependency.

You can add the rules about nubOrd to the normal section - the idea is that the extra rules are opt-in since you are wanting to say you are explicitly OK with adding a dependency of extra. However, if you've already used nubOrd in the LHS, you must be OK with that, so they can go in the normal enabled as standard section.

That reasoning makes sense!
However, I am now a bit confused/puzzled, probably since I don't know how exactly hlint resolves the function names it sees:

If I add a rule like - warn: {lhs: catMaybes (nubOrd x), rhs: nubOrd (catMaybes x)} to the default group, how will hlint know that by nubOrd I mean the function from the extra package? (I guess one might add that as a package import to the group, but the 2. item below has me puzzled on how that works.)
In the existing extra group there is no import of the extra package. So what is going on there? How does hlint resolve nubOrd in a rule like - warn: {lhs: "nubOrdBy compare", rhs: "nubOrd"} to the function of that name in Data.List.Extra?

All this has me wondering particularly since nubOrd exists in many places, and not all of them are the exact same function presumably (https://hoogle.haskell.org/?hoogle=nubOrd&scope=set%3Astackage). So how does this work out at all?

 If it knows nubOrd came from Data.List.Extra it will see if there are any imports of that and make a best guess. If it doesn't (which is the case if you don't give it an explicit import) it makes the guess that it probably is the nubOrd you are thinking about. It's not a particularly satisfactory aspect of HLint, and there are a bunch of bugs associated with that relaxed approach that we've accumulated under a tag: https://github.com/ndmitchell/hlint/labels/scope

Thank you for the explanation. That really makes some aspects of hlint's rule application clearer!
I've now added the three rules for nubOrd as well, in the default group.


Terraform is trying to poll Master node before creation

Hey there 👋
Just stumbled on your module and thought it was interesting.
Found an issue where you are polling the master VM for the kubeconfig before it's creation.
Fixed it by adding a
   depends_on = [       proxmox_vm_qemu.k3s-master,     ]
block here https://github.com/fvumbaca/terraform-proxmox-k3s/blob/main/master_nodes.tf#L93
Have a great day

Hey! Thanks for the issue! This should now be fixed in v0.0.3. Hope your enjoying the module! 🙂


Make DefaultAzureCredential as default credential method instead of anonymous access?

Hi there
what do you think of making DefaultAzureCredential as the default credential method instead of anonymouns access because I think that is the most common use case.

+1, that would be great.

xref https://github.com/fsspec/adlfs/pull/250, where this was implemented but then reverted. I think we can do this, but we'll need a deprecation cycle.
I was somewhat worried that this would be too noisy of a deprecation, but if we think that the majority of users are specifying credential anyway (which I think they are) then the only people affected are those relying on credential="anon".
So +1 to deprecating the current default with a FutureWarning and then moving to DefaultAzureCredential in a later future.

Is there any way to move this forward? There are some libraries that do not expose fsspec storage options because they assume the anon=False logic like in s3, and I suppose azure is not their main target audience. As a workaround would it be possible to add an env option to disable anon access before we transition to anon=False by default?

For those interested in this https://github.com/fsspec/filesystem_spec/pull/1194 provides a generic alternative solution (make sure to update fsspec as it's pretty recent addition). You can pass a default argument to FSSPEC for specific implementations through an env variable as follows:
os.environ["FSSPEC_ABFS"] = """
{
  "anon": false
}
"""

I'll be closing my PR (https://github.com/fsspec/adlfs/pull/393) as it's not getting any traction.


Is media type text/calendar supported?

https://datatracker.ietf.org/doc/html/rfc5545
Are there plans to support media type text/calendar? Or is there an existing media type that satisfies this?

It's supported!

@webron
I couldn't find any examples of it. Is this right? I don't see any errors with this but I am not 100% sure.
responses:
  "200":
    description: A successful response
    content:
      text/calendar:
        schema:
          description: Calendar response.

Yup, that works. You don't even need the schema in this case.

@webron
What is the correct syntax without the schema? The swagger editor gives me a should be object error with the following specification on text/calendar.
responses:
  "200":
    description: A successful response
    content:
      text/calendar:

This should work:
responses:
  "200":
    description: A successful response
    content:
      text/calendar: {}

@webron
I see that works. Thank you for your help!


⚠ otzivi-tut has degraded performance

In 168a3ff, otzivi-tut (https://отзывы-тут.рф/) experienced degraded performance:

HTTP code: 200
Response time: 13209 ms

Resolved: otzivi-tut performance has improved in 6c44d05.


Add rune calculation utility, test suite for rune calculation, useEff…

…ect for level change and setting runes needed/runes held

Molto bene

Grazie mille


Crash when rcl_timer_reset is called

Describe the bug
When I call rcl_timer_reset on a timer, microros will crash
Additional context
I tracked the issue to the "Static operation" commit https://github.com/micro-ROS/rmw_microxrcedds/commit/cb4f5220b5666be270ac29820ccd3e7a19f267f1.
And I tracked it down the crash to rmw_trigger_guard_condition at line https://github.com/micro-ROS/rmw_microxrcedds/blob/5de3d7687fff033dffa62594c3ead55ced76c724/rmw_microxrcedds_c/src/rmw_trigger_guard_condition.c#L34
Before the "static operation" commit rmw_create_guard_condition() would allocate the data field dynamiclly and doing so again, does fix the crash. so the function looks, like this:
rmw_guard_condition_t *
rmw_create_guard_condition(
  rmw_context_t * context)
{
  (void)context;

  rmw_uxrce_mempool_item_t * memory_node = get_memory(&guard_condition_memory);
  if (!memory_node) {
      // POI - it fails here.. it retuns here NULL
    RMW_SET_ERROR_MSG("Not available memory node");
    return NULL;
  }
  rmw_uxrce_guard_condition_t * aux_guard_condition =
    (rmw_uxrce_guard_condition_t *)memory_node->data;
  aux_guard_condition->hasTriggered = false;
  aux_guard_condition->rmw_guard_condition.context = context;
  aux_guard_condition->rmw_guard_condition.implementation_identifier =
    rmw_get_implementation_identifier();

+  aux_guard_condition->rmw_guard_condition.data = (bool *)rmw_allocate(sizeof(bool));
+  bool * hasTriggered = (bool *)aux_guard_condition->rmw_guard_condition.data;
+  *hasTriggered = false;

  return &aux_guard_condition->rmw_guard_condition;
}

Although I understand it's not the appropriate fix, but I hope it will be enough information for you to confirm this bug.

Hello @amfern, thanks for this. Let me know if using this patch this is solved: https://github.com/micro-ROS/rmw_microxrcedds/pull/209

Hi @pablogs9, i see #209 merged into main branch. so i tried to compile main branch but i get compilation error
/uros_ws/firmware/mcu_ws/uros/rmw_microxrcedds/rmw_microxrcedds_c/src/rmw_node.c:237:13: error: dereferencing pointer to incomplete type 'rmw_context_impl_t' {aka 'struct rmw_context_impl_t'}
  237 |     &context->graph_guard_condition;
      |             ^~
make[2]: *** [CMakeFiles/rmw_microxrcedds.dir/build.make:274: CMakeFiles/rmw_microxrcedds.dir/src/rmw_node.c.obj] Error 1

It has been backported to galactic and foxy, please use the branch corresponding to your ROS 2 distro. I'm not sure this error is related to the PR but with a distro mismatch.

Sorry, for that.
I compiled the galactic branch, and it fixed the crash. thank you

Cool, thanks a lot for the feedback!
Closing this


OPC Timestamp Format

When reading data from an OPC server I am getting the timestamp formatted as such:

timestamp: "Sun, 31 Dec 1600 23:57:55 GMT

On Matrikon Explorer the date is ok.

Greetings, I'll take a look. Probably a problem with one of our libs.
Ty

Would you mind sharing your Node-RED flow with us?
Thx

Here you go.
opcda.import.flow.txt

Greetings
The node-opc-da lib was updated. Try updating to the latest version and see if the timestamp problem persists.
[]'s

Hi,
The problem is now fixed.
Ty


MB-13754: Working client logging

Jira ticket
Summary

Rework milmoveLogger to be compliant with @opentelemetry/api
add componentDidCatch to SystemAdmin page to log errors

Verification Steps for the Author
These are to be checked by the author.

[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] Have the Jira acceptance criteria been met for this change?

Verification Steps for Reviewers
These are to be checked by a reviewer.

Warnings

:warning:

Files located in legacy directories (src/shared or src/scenes) have
been edited. Are you sure you don’t want to also relocate them to the new file structure?
View the frontend file org ADR for more information

:warning:
This PR does not include changes to unit tests, even though it affects app code.

Messages

:book:
:link: MB-13754

  Generated by :no_entry_sign: dangerJS against 9c073c46b1cf06a12aedc0ec08f325c7e5dac086

Test coverage on client failed.
Last run: Tue 08 Aug 2023 06:14:38 PM UTC
Refer to this report to see coverage details for the client app.
Documentation for test coverage can be found at this link.

Test coverage on server failed.
Last run: Thu 10 Aug 2023 03:11:55 PM UTC
Refer to this report to see coverage details for the server app.
Documentation for test coverage can be found at this link.

Also, would this be useful to test in exp?

@YanZ777 this is currently running in EXP now in this PR, #11208. Feel free to add commits if you want to further test the client-side errors.


Console output from SFTP post upload command

Hi,
Not sure if the underlying protocol supports it, but would be great if as part of a sftp deployment, we could get the console output from the commands that can be run after upload.
i.e. upload a file and run a remote test suite on it.
Suspect we can do that with a combination of batch. sftp and app, but would be good if we could wrap it all up cleanly.
Thoughts ?

@enly1
Sorry for the late answer.
I have released a new version 9.30.0 which is now able to output the results of a command that is executed on a SFTP server.
An example:
{
    "deploy": {
        "targets": [
            {
                "type": "sftp",

                // ...

                "noCommandOutput": false,

                "connected": [
                    {
                        "command": "whoami",
                        "writeOutputTo": "current_bash_user",
                        "executeBeforeWriteOutputTo": "$others['current_bash_user'].trim()"
                    }
                ],

                "uploaded": [
                    {
                        "command": "npm test",
                        "verbose": true
                    },

                    {
                        "command": "echo \"You have uploaded as ${current_bash_user}\"",
                        "verbose": true
                    }
                ]
            }
        ]
    }
}

Here the updated wiki article, which describes the new feature in detail.

Thanks, will give it a try. Very well timed. Thanks as always.


[FLINK-23890] improve cep operator timer register policy

What is the purpose of the change
(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them on each deployment (during recovery).)
Brief change log
(for example:)

The TaskInfo is stored in the blob store on job creation time as a persistent artifact
Deployments RPC transmits only the blob storage reference
TaskManagers retrieve the TaskInfo from the blob cache

Verifying this change
Please make sure both new and modified tests in this PR follows the conventions defined in our code quality guide: https://flink.apache.org/contributing/code-style-and-quality-common.html#testing
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (100MB)
Extended integration test for recovery after master (JobManager) failure
Added test that validates that TaskInfo is transferred only once across recoveries
Manually verified the change by running a 4 node cluster with 2 JobManagers and 4 TaskManagers, a stateful streaming program, and killing one JobManager and two TaskManagers during the execution, verifying that recovery happens correctly.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@mayuehappy, thanks for your contribution to reduce the frequency of the timer registration. I left the minor comments for your changes. BTW, @mayuehappy, could you fix the typo in the tile, which is impove->improve? cc @dawidwys, could you please take a look at this improvement?

@SteNicholas Sorry for the late reply , i've your comment and made some replies. PTAL, thx.


Update kustomization files to latest standard

This removes deprecation warning when applying kustomize to the files

/test pipeline-service-upgrade-test

/test pipeline-service-upgrade-test

were you able to confirm the deprecation warnings were gone before the pipelineruns got pruned @Roming22 ?
otherwise,
/lgtm

@gabemontero There are some warning left because they come from tekton-results resources. I've submitted a change for that.
I ran kustomize build locally, I didn't look at the logs, as there's no reason it should behave differently.


Label projection != label transfer

This benchmark does not benchmark a method's performance on labelling a totally new dataset, given an existing reference dataset. Instead, it takes a large dataset and then uses one batch as test holdout.

We should create a new task task_label_transfer which maps cell type labels from a reference dataset to a query dataset. This benchmark will then also have to deal with having to match different matrix shapes.
To avoid confusion, we should update the description of this task to reflect the distinction between label projection and label transfer

Hi Robrecht! Your comment brings up two questions:

This task was set up like this, because the correct labels on the test part were easily available. How would we get ground truth (or just correct labels) for label_transfer task between two datasets?
When transferring labels from one dataset to another, there can be a situation where some (or all) query cell types are not represented in the reference. I was thinking of emulating this by withholding some cell types in label_projection task. How do you think this should be handled in the label_transfer?


Add session API endpoints

This adds all the session API endpoints. I also did a grammar check by running everything through LanguageTool and did some minor formatting tweaks. Please especially check over the local sync endpoint, I was sort of confused about that one.

Looks good to me. Thanks!

I'm not sure why the tests are failing. These were setup automatically by slate and I haven't looked into how all that works.
I think before I made that update it was passing.

I'm not sure if you can rerun the checks or not, but it should pass them. Regardless, it should be okay to merge it since it looks like just the 2.7 version failed. Though I'm not sure what version you are using for deploying the site.


Transform stream output

Adds onto #6 to try to lessen data written to disk again
Doesn't work currently

Note that #6 had to be reverted, may need reconsideration how to do this properly....streams are hard


Check silent renew

check if silent renew is set up and if so what expiry timescales it has on it i.e. likely ppl might want to be doing a thing for longer than 1 hour -->auto-refresh token

When investigating this @sHtev has said:
// we don’t save session in hydra, just in profile, so normally you just redirect to /login on profile and pick up the session token, but silent renew expects hydra to have the session stored, it does not know how to redirect or get a login prompt, hydra just says “I don’t have a session, please log in” and the OIDC client basically just goes :shrug:
so we either

work around sessions expiring and not lose state
or we talk about getting hydra to mirror profile session handling (aka ask Core if it's in their backlog)
we can increase the access token TTL to match profile session length which currently defaults to 1 hour (https://raspberrypifoundation.slack.com/archives/C02D2T8JMQU/p1670607176670669)//

Expect that the next steps is for Maddy/Patrick to check with Greg if we do something interim, but ideally/longterm my view = this should be standardised/fixed in Hydra by sounds of things, given speaking to Projects site people this is also an issue currently for Project site session expiry, not just Editor.
Front-end I'd expect the desired user behaviour is it boots them after 1 hr of actual idleness or a max time of 24 hrs or something.

@grega 's steer (TY!) = this should get handled in the specific product application aka we call Hydra and ask for a new token. This = how it works for Projects site.
@patch0 please let me know your thoughts when you can --> and can then raise a backlog issue for us to make a change (ideally based on not-idle? --> fetch new token) at a later date.

I believe Hydra v1 is handling sessions (and stores session data in the table hydra_oauth2_access). Haven't spent any time looking at this recently however so will refresh (ha) my memory...
As mentioned in the Slack thread, the refresh_token grant type will need adding to the Editor's Hydra client config.

Further testing confirms silent renew works iff:

Client has refresh_token grant type as above
Identity provider (profile) sets remember and remember_for on both acceptLoginRequest and acceptConsentRequest API calls (so that hydra saves authentication success in its session data)

Token TTL can be set on a per-client basis - see https://www.ory.sh/docs/hydra/guides/client-token-expiration#lifespan-configuration-for-specific-clients


Alternative with aligned texture

Re-made the challenge with interesting formulas for geographic to cartesian transforms
Texture aligned to actual coordinates
Corrected box positioning
Loader shown while data is being fetched from Internet
Using PeasyCam

Hey this is fantastic! This repo, however, is meant to have the code that matches the video exactly so others can start from that code and make their own improvements. I was thinking of putting READMEs in the challenge folders then people could link out to their improved code. Would you like to pull request that?

Hey,
Sorry if I did this wrong. I was just looking for a way to share around
your great work.
Readmes are a good idea. I'll get involved if I find a little time, which
is becoming a scarce resource for me these days I'm afraid.
Keep the happy coding and thank you
Le 22 févr. 2017 15:06, "Daniel Shiffman"<EMAIL_ADDRESS>a
écrit :
Hey this is fantastic! This repo, however, is meant to have the code that
matches the video exactly so others can start from that code and make their
own improvements. I was thinking of putting READMEs in the challenge
folders then people could link out to their improved code. Would you like
to pull request that?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/CodingTrain/Rainbow-Code/pull/73#issuecomment-281678211,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AYu4SDbegHunyTbB9s4tn6Yw4grazGThks5rfEDkgaJpZM4MHlkT
.

You're not doing it wrong at all, on the contrary, we're all learning here!  I'll add the README and link to your repo now!

Ok, it's here!
https://github.com/tadimon/Rainbow-Code/tree/master/challenges/CC_58_EarthQuakeViz3D
I might suggest just making your own repo rather than having the variation inside a fork of all of these examples, but there's no rush of course!


[timer decomp]

Pull Request Test Coverage Report for Build 553112469

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 67.856%

Totals

Change from base Build 551863480:
0.0%

Covered Lines:
27939

Relevant Lines:
41174

💛  - Coveralls


Pull Request labels

Add pull request labels (as mentioned in #1138) for the change-log script.

We have that already. Suggestions for further categories are welcome.


Add placeholder CBOR encoder/decoder as (unreleased) extra

CBOR is a good alternative to JSON for byte efficient serialization, and may be a replacement for the custom dvalue format in the debugger protocol in the future. Experiment a bit with binding CBOR to Duktape; the pull provides C bindings to encode/decode arbitrary Ecmascript (Duktape) values into CBOR and back. There's also a command line tool for doing JSON-to-CBOR and CBOR-to-JSON conversions. The extra is not yet included in the distributable because it'll need some more work.
Tasks:

[ ] CBOR encoding
[ ] CBOR decoding
[ ] Testcases
[ ] README
[ ] JSON/CBOR conversion tool

Re: Bindings - It might be nice to eventually provide JavaScript bindings for CBOR too.  It would compliment the built-in standard JSON bindings nicely.
For what it's worth, I think CBOR might be somewhat heavy for use in the debugger protocol ("heavy" conceptually, not referring to footprint).  When I switched miniSphere from Duktape to ChakraCore, I needed to implement a new debugger protocol to communicate with SSj; unlike Duktape, CC has a JSON-based debugging API that runs directly on the target.
I researched CBOR and almost implemented it for debug messages, but ultimately decided to adapt the Duktape protocol instead as it was simpler and still met all my needs (plus I didn't need to rewrite as much code immediately, a nice bonus :-)
https://github.com/fatcerberus/minisphere/blob/master/src/shared/ki.c
https://github.com/fatcerberus/minisphere/blob/master/src/shared/ki.h

Re: Bindings - It might be nice to eventually provide JavaScript bindings for CBOR too. It would compliment the built-in standard JSON bindings nicely.

Agreed, having some binary serialization format would be nice.
Though one major reservation I have with that is that eventually I'd like to have a fast binary serialization format that would be able to serialize object graphs efficiently. Ultimately I'd like to be able to serialize a heap and revive it back from flash, with enough metadata to allow the application to revive native objects like files, sockets, etc in the process.
So that will need some memory efficient format that will be able to represent all Duktape types reasonably well. And it will need to be able to represent object graphs with reference cycles.
CBOR doesn't do that directly (and it's indeed a bit challenging) but it can still be done over CBOR by using some sort of object reference scheme, e.g. serialize the heap as indexed values and resolve the indices when reading back.
So maybe it would work out, maybe not. Using a standard would be nice, but it's not a showstopper requirement if it's not a reasonably good fit without a lot of elbow grease.

By heavy I meant in the sense of being overly generic.  For example CBOR objects are key/value pairs where the key can be any type, even entire objects!  So you d necessarily be using a very limited subset for the debugger protocol which feels kind of awkward.
The genericity would be nice for serializing actual Ecmascript objects though so maybe it wouldn't be so bad after all.  I don't know, I'm just picky about bloat I guess :)

For example CBOR objects are key/value pairs where the key can be any type, even entire objects!

But that's basically what one needs to extend better (versioning-wise) than a flat list of primitive values.

Ah, misread what you said :) Yeah, keys can be objects but nothing forces one to create such objects. I would stick with the JSON compatible subset so that a CBOR-JSON mapping would be straightforward.

So for concreteness, if one encountered an object whose keys were objects in the protocol, one would simply declare a syntax error and detach. It wouldn't be fully in keeping with the usual parsing principles, i.e. one would normally parse such an object, and if the key wasn't interesting, just ignore it. But, it'd be an OK way to deal with the situation IMO because nothing forces one to send such objects.
It's also not actually clear whether one might actually need such objects. For example, when encoding property tables, the keys are not actually strings but duk_hstring references. It might be better to encode the property table keys as string references, retaining as much of the internal representation as possible, than to encode them as plain strings. (Although in this particular case one might need to use an array for the key/value pairs to preserve ordering, something that CBOR doesn't guarantee; key/value ordering of any CBOR/JSON object might change in transit.)
One motivation for writing a basic CBOR binding is to get a better grasp of what works and what doesn't.

@fatcerberus A small sidenote re: objects-as-keys, Lua allows objects as table keys and it's actually IMO a good feature: it makes the object model more regular (keys and values have the same space), allows tables to be used for tracking e.g. visited objects conveniently, and objects can be used as unique keys similarly to Symbols.
In fact, if one were able to control enumeration, placeholder empty objects would serve the role of Symbols. Without access to the object (assuming they could be protected from enumeration) they would be inaccessible to a call site which didn't have a reference to the object.
The downside is that objects are a bit heavier in terms of representation but depending on the tagged value representation this may be an issue or not. For example, with tagged pointers strings and arbitrary tagged values are the same size and it's a non-issue; in other representations like current duk_tval it matters and string pointers are smaller than general tagged values.
Another downside is that many other contexts assume keys are strings (or at least primitive); in particular, JSON does that. So if you actually take advantage of that representation, you'll then run into problems with JSON serialization. But that'd also be the case if one wanted to serialize a Map.

Merging a work-in-progress which has some remaining work (half-floats, etc) but is nevertheless useful to some extent.


lsm: refactor compaction

Preface:
I've been hacking on this code for a past couple of days, and, while I am not yet bold enough to hit ./zig/zig build check button, I am at the point where I desperately need feedback!
I think the overall shape of code should be somewhat clear at this point, even if it is missing a detail or two (I see that I've removed release_table_blocks, I probably should be calling that somewhere).
Right now, I am at the stage where I am pretty convinced in this new approach, but that's probably simply because I've been staring at this code for quite some time and it makes too much sense to me. So curious WDYT here!

Refactor compaction to:

make the code easier to follow,
make it less unobvious how the compactions are actually schedules (that is, what work can be in-flight at any given point in time),
make scheduling a bit more aggressive inside a single compaction (e.g. start merge when the first two value blocks are in, separate IO and block resources),
make it easier to run several compactions at the same time.

Code Changes

Fewer types
All tricky logic is centralized in compaction_dispatch
Many FIFOs are replaced with fewer RingBuffers
Flatten Beat and Bar into compaction

CompactionPipeline no longer implements pipelining. Now, it only deals with scheduling compactions themselves. Pipelining is the job of each individual compaction. That is, CompactonPipeline kicks off a particualar compaciton, and recieves the control back only when that compaciton's beat is done.
CompactionPipeline should be rename to CompactionSchedule, or maybe just merged with a forest.
A new type, CompactionPool, is introduced. This is a bag of all resources shared by compactions. Previously, blocks, reads and writes were lumped together. Now, CompactionPool holds separate IOPS for reads, writes, blocks, and CPUS.
CompactionPool is a forest-level object, but it doesn't depend on the type of the forest. So, all compactions share the same CompactionPool type, which allows them to share the actual object.
The actual pipelining logic is moved to Compaction.compaction_dispatch function. This function no longer tries to explicitly encode a pipeline. Rather, it is just a loop that spawns a bunch of jobs, and the pipelining happens as a side-effect of that. The pseudo-code for compaction_dispatch is:
for each job:
    if have resources for the job AND
       job's dependencies are fulfilled AND
       it makes sense to start the job:

       start the job

Here, the jobs are reading the index blocks, reading the value blocks, merging/copying, writing the data blocks, writing the index blocks. Resources are blocks (memory) and grid reads&writes. Dependencies are blocks read previously by other jobs (reading a value block needs an index block). The "makes sense" part coves:

spreading resources across jobs to avoid deadlocks,
avoiding overreads when finishing a beat-worth of work.

There isn't some kind of comptime meta programming framework for defining jobs, it's just a good old pile of ifs (pushed up to the compaction_dispatch cental location).
Dependency tracking is done via RingBuffers. For example, there's a RingBuffer of level_b value blocks. Blocks are added to this queue in order when we start the corrsponding read. When we finish the read, we flip the status of block (which is in the queue at this point) to read_done.
So, when we think about launching a merge, we check that:

there's a head value block in the queue
the status of this block is read_done

If the merge ends up fully consuming the block, it pops it off the queue and adds back to the shared pool.
This setup is similar to a bunch of fifos we used before, but:

With ring-buffers, so that it's statically obvious what's the upper bound for the size of the queue.
Queues are matched to block kind (value block/index block/output block) rather than to it's lifetime stage (pending/reading etc).
There are just fewer queues (I think, didn't actually count).

Behavior Changes

Separate limits for memory, CPU, and IO resources.
Aggressive scheduling -- start merge as soon as the first two blocks are in
Running two compactions just work, given enough resources.
A different "winding down" approach.

Previously, allocating a block for compaction also allocated a grid read and a grid write. In the new approach, those are allocated separately. Additionally, besides available resources there are static limits on the amount of outstanding work. That is, RingBuffers for blocks are statically sized, so you can't read more than X blocks at the same time even if you have spare blocks.
For simplicity, it is assumed that there's actually enough reads and writes to fill the queues fully, but the approach should work even if you restrict all compactions to just a single read and a single write.
The blocks are distributed in a priority round-robin fashion. That is, we first give block to the table builder, and then alternate between level b and level a. This way:

We avoid deadlocks: if the number of blocks is larger than a minimum, than the jobs will be scheduled in the right order to avoid running out of blocks completely
We give equal resources to level a and level b (the approach should be extensible to fair allocation, proportional to level "size")
The schedule ends up being maximally pipelined (the first four reads are: level_a index, level_b index, level_b value, level_a value, which allows starting the merge ASAP).

I think this schedule allows even for a simplistic strategy for running several compactions: if you have N*min blocks, it would be correct to start N compactions at the same time. At least one of them will be guaranteed to be able to proceed. This scheme is simplistic, because:

a single compaction can monopolize an unfair share of resources. Compactions are greedy.
there's no code path for "if compaction A frees resource X, wake up compaction B waiting on X".

Both points should be fixable I think though!
Another difference (I think) happens in the "winding down" phase. That is, in what happens when the compaction consumed the required quota of inputs, but haven't yet produced a full value table of output. This is a tricky situation, because, depending on the tombstones, you might have to read quite a bit more tables still. The worst case here is that you have a full table on level A, a full table on level B, but their merge is an empty set.
In the previous approach, due to explicit pipelining, we'll just loop over:
loop {
    read fixed number of blocks from level
    run merge
}

In the new approach, this is handled by AND it makes sense to start the job part of the pseudocode --- if we are in the winding-down phase, we just artificially limit the depth of our RingBuffers.
Such overreads are tracked via values_wasted_count.

cc @cb22 @sentientwaffle @batiati

It is getting there!
This now basically works, and passes non-trivial amount of VOPR seeds.
I am yet to restore logging&tracing though...

Was pretty green before the last round of cleanups,

the benchmark results are not terrible (though I haven't properly benchmarked this yet), and I think I did everything I wanted to in the scope of this PR!
Ready for review!

Rebased to pick up the VOPR fix for "all clients evicted" issue