Improve “Resource metrics pipeline” topic

https://kubernetes.io/docs/tasks/debug-application-cluster/resource-metrics-pipeline/ is a bit too basic. A diagram would really help, and we should try to avoid mentioning developer details like hack/kube-up.sh
/language en

/triage accepted

/assign

hey @verma-kunal, I saw this issue closed. I would like to re-open and then hack a PR. Will you be working this?

/reopen
/assign

Not sure if this will do the trick (can do it manually if needed):
/reopen

hey @verma-kunal, I saw this issue closed. I would like to re-open and then hack a PR. Will you be working this?

Really sorry! Got busy with some university stuff here
You can surely create the PR for this & that would even help me understand the issue and the fix 🙂

Notes:
 ~ minikube kubectl -- get --raw "/apis/metrics.k8s.io/v1beta1/nodes" | jq '.'
{
  "kind": "NodeMetricsList",
  "apiVersion": "metrics.k8s.io/v1beta1",
  "metadata": {
    "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes"
  },
  "items": [
    {
      "metadata": {
        "name": "minikube",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes/minikube",
        "creationTimestamp": "2021-12-16T19:43:54Z"
      },
      "timestamp": "2021-12-16T19:43:43Z",
      "window": "30s",
      "usage": {
        "cpu": "550637108n",
        "memory": "1006636Ki"
      }
    }
  ]
}
➜  ~

  ~ minikube kubectl -- get pods -n kube-system
NAME                               READY   STATUS    RESTARTS   AGE
coredns-78fcd69978-f6pqv           1/1     Running   0          22h
etcd-minikube                      1/1     Running   3          22h
kube-apiserver-minikube            1/1     Running   3          22h
kube-controller-manager-minikube   1/1     Running   3          22h
kube-proxy-pmm26                   1/1     Running   0          22h
kube-scheduler-minikube            1/1     Running   3          22h
metrics-server-77c99ccb96-mdjhd    1/1     Running   0          22h
storage-provisioner                1/1     Running   0          22h
➜  ~

  ~ minikube kubectl -- top node
NAME       CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
minikube   479m         23%    984Mi           49%

some Notes:
add minikube metrics server addon
~ minikube addons enable metrics-server
    ▪ Using image k8s.gcr.io/metrics-server/metrics-server:v0.4.2
🌟  The 'metrics-server' addon is enabled

some Notes:
➜  ~ minikube kubectl -- get --raw "/apis/metrics.k8s.io/v1beta1/pods" | jq '.'
{
  "kind": "PodMetricsList",
  "apiVersion": "metrics.k8s.io/v1beta1",
  "metadata": {
    "selfLink": "/apis/metrics.k8s.io/v1beta1/pods"
  },
  "items": [
    {
      "metadata": {
        "name": "coredns-78fcd69978-f6pqv",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/coredns-78fcd69978-f6pqv",
        "creationTimestamp": "2021-12-17T17:58:01Z"
      },
      "timestamp": "2021-12-17T17:57:42Z",
      "window": "30s",
      "containers": [
        {
          "name": "coredns",
          "usage": {
            "cpu": "7338891n",
            "memory": "14520Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "etcd-minikube",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/etcd-minikube",
        "creationTimestamp": "2021-12-17T17:58:01Z"
      },
      "timestamp": "2021-12-17T17:57:44Z",
      "window": "30s",
      "containers": [
        {
          "name": "etcd",
          "usage": {
            "cpu": "58846045n",
            "memory": "53836Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "kube-apiserver-minikube",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube-apiserver-minikube",
        "creationTimestamp": "2021-12-17T17:58:01Z"
      },
      "timestamp": "2021-12-17T17:57:38Z",
      "window": "30s",
      "containers": [
        {
          "name": "kube-apiserver",
          "usage": {
            "cpu": "172408299n",
            "memory": "244196Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "kube-controller-manager-minikube",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube-controller-manager-minikube",
        "creationTimestamp": "2021-12-17T17:58:01Z"
      },
      "timestamp": "2021-12-17T17:57:46Z",
      "window": "30s",
      "containers": [
        {
          "name": "kube-controller-manager",
          "usage": {
            "cpu": "84375531n",
            "memory": "43312Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "kube-proxy-pmm26",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube-proxy-pmm26",
        "creationTimestamp": "2021-12-17T17:58:01Z"
      },
      "timestamp": "2021-12-17T17:57:48Z",
      "window": "30s",
      "containers": [
        {
          "name": "kube-proxy",
          "usage": {
            "cpu": "3349042n",
            "memory": "14716Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "kube-scheduler-minikube",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube-scheduler-minikube",
        "creationTimestamp": "2021-12-17T17:58:01Z"
      },
      "timestamp": "2021-12-17T17:57:46Z",
      "window": "30s",
      "containers": [
        {
          "name": "kube-scheduler",
          "usage": {
            "cpu": "8237321n",
            "memory": "20108Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "metrics-server-77c99ccb96-mdjhd",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/metrics-server-77c99ccb96-mdjhd",
        "creationTimestamp": "2021-12-17T17:58:01Z"
      },
      "timestamp": "2021-12-17T17:57:49Z",
      "window": "30s",
      "containers": [
        {
          "name": "metrics-server",
          "usage": {
            "cpu": "13778968n",
            "memory": "12720Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "storage-provisioner",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/storage-provisioner",
        "creationTimestamp": "2021-12-17T17:58:01Z"
      },
      "timestamp": "2021-12-17T17:57:47Z",
      "window": "30s",
      "containers": [
        {
          "name": "storage-provisioner",
          "usage": {
            "cpu": "5055900n",
            "memory": "11172Ki"
          }
        }
      ]
    }
  ]
}

some Notes:
k8s/metrics/../metrics/type.go
k8s/metrics/../metrics/v1beta1/generated.proto
Kubelet summary API moved to staging
Kubelet summary API in staging folder

related issue https://github.com/kubernetes/website/issues/31035

https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/2371-cri-pod-container-stats/README.md

/lifecycle frozen

The description of the Summary API Source section does not match the KEP and code implementation.
Suggested modification:
The Kubelet gathers stats at node, volume, pod and container level, and outputs their statistics in the Summary API for consumers to read.

see #31035

@sftim @serathius
Initial hack resource-metrics pipeline diagram. Included pre-1.23 cAdvisor and 1.23 CRI-O. Feedback needed.

diagram live editor

(nit) Typo in cAdivsor.
This is a big topic so well done for helping tackle it @chrismetz09. It's an important page.
The HPA talks to the Kubernetes API. Behind the scenes, part of that API is served via an API extension - the metric server. The HPA (or VPA, potentially) then writes back to the Kubernetes API to update the thing it's managing, such as a Deployment.
I'd be tempted to show that part of the detail first. Once that's explained, we can then focus on covering what happens within a node that allows the metric server to serve the extension API.
That second diagram should show that cAdvisor runs on (usually) all nodes, via a DaemonSet, and that cAdvisor exposes an HTTP service that the Metrics Server connects to. In the text I'd explain how the Metrics Server discovers nodes (ie, via the Kubernetes API).
Going further, a third diagram could show that you could use a different API extension to server your own custom MetricValueList responses. For example, to integrate Prometheus metrics and use those in addition to Metrics Server. I'd draw the third diagram and then use an SVG editor to make a version that hides the extra bits, so that the rest of the layout stays the same between diagram 1 and diagram 2.
Some existing diagrams that we might learn from (but not plagiarize!) are in:

https://www.superawesome.com/blog/how-we-scale-our-kid-safe-technology-using-auto-scaling-on-kubernetes/
https://medium.com/kubernetes-tutorials/introduction-to-kubernetes-monitoring-architecture-98a265e0917d
https://blog.itaysk.com/2019/01/15/Kubernetes-metrics-and-monitoring

https://github.com/kubernetes-sigs/metrics-server#readme might also be relevant (I hope it is).

@sftim @serathius
Revised resource metrics hack diagram:

mermaid live editor diagram link

In terms of networking (TCP, HTTP etc): the metrics server makes a connection to cAdvisor, and the API server connects to the metrics server. I think of it like this:

mermaid live editor diagram link

For the dataflow, that could be
mermaid live editor diagram link

Whoa! Cooking on the diagrams :-). Several questions and clarifications:

I see kubelet API endpoints here: https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/server/server.go#L84-L93

I understand role of cAdvisor (soon to be legacy?) does. Call to ../metrics/cadvisor returns massive amounts of data. Help jq! Too much to parse :-)

Does the metrics server call the Kubelet ../stats/summary API for each Kubelet? Or does the metric server call the ../resources/cadvisor API?

Here is my diagram pertaining to question 3.

mermaid live editor diagram link

BTW the metrics-server is also a client of the Kubernetes API (for things like discovering nodes).
As far as I can see, metrics-server connects to /metrics/resource on the kubelet - see https://kubernetes.io/docs/concepts/cluster-administration/system-metrics/ and https://github.com/kubernetes-sigs/metrics-server/blob/eebdfa3ec19b8397541b5b814f09ce6b2c16ade1/pkg/scraper/client/resource/client.go#L91
I might have been wrong about the direction of the edge between cAdvisor and the kubelet at the connection level. It looks like the kubelet is an HTTP client of a cAdvisor that runs embedded within the kubelet, but I'm not sure.

BTW the metrics-server is also a client of the Kubernetes API (for things like discovering nodes).

Yes and also pods. Metrics server watches for all pods and nodes in cluster. Nodes for metrics collection discovery, pods for metadata. Pod metadata is used to endure pod is in ready state and to handle queries with label selectors. For example query for HPA, metrics server needs to identify which pods fulfill the label selectors from Deployment.

As far as I can see, metrics-server connects to /metrics/resource on the kubelet - see https://kubernetes.io/docs/concepts/cluster-administration/system-metrics/ and https://github.com/kubernetes-sigs/metrics-server/blob/eebdfa3ec19b8397541b5b814f09ce6b2c16ade1/pkg/scraper/client/resource/client.go#L91

This is true for next release v0.6.0+, for versions 0.5.2 and older metrics-server uses /stats/summary.

I might have been wrong about the direction of the edge between cAdvisor and the kubelet at the connection level. It looks like the kubelet is an HTTP client of a cAdvisor that runs embedded within the kubelet, but I'm not sure.

cAdvisor is compiled in into Kubelet, communication is just a function call.

@serathius would you be willing to tweak the diagram until it looks right to you?

Also relevant: https://github.com/kubernetes/website/issues/27876

More context, albeit archived: https://github.com/kubernetes/design-proposals-archive/blob/main/instrumentation/monitoring_architecture.md

As per this comment, I am unassigning to you.
/unassign @verma-kunal

Would anyone like to open a PR to add a diagram to that page? It'd be welcome.

@chrismetz09 I don't see any updates; so unassigning you. Please feel free to assign, if you come back here again and are willing to work on 🙂
/unassign @chrismetz09

/triage accepted
/help wanted


Sam/feat/start of week

Added choice of start day of week. You can choose to set Monday as the first day of the week instead of Sunday. Added tests for the new functionality.

Hello, any plans on accept the two pending pull requests?.

with your update I get this error:
RangeError (end): Invalid value: Only valid value is 0: 7


fail early if target-dir exists without --clean option

fail early if target-dir exists without --clean option, or it's not in a writable directory.

unit tests on progress.


fix exceptions for write-option variable_names/2

#188

Codecov Report

Merging #220 (ff3206f) into main (59c5001) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #220   +/-   ##
=======================================
  Coverage   96.86%   96.87%
=======================================
  Files          21       21
  Lines        6320     6336   +16
=======================================
+ Hits         6122     6138   +16
  Misses        163      163
  Partials       35       35

Impacted Files
Coverage Δ

engine/builtin.go
96.33% <100.00%> (+0.02%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 59c5001...ff3206f. Read the comment docs.


Crash during Vite build when using react-animated-slider

I encountered an issue where using react-animated-slider in my Vite project is causing the build process to crash. The issue occurs specifically when using react-animated-slider component in my code.

My React Version : 18

I have the same issue when I using Vite


Please invite me to the GitHub Community Organization

Name
Don Louie - @donlouie
Discord Username (if applicable)
n/a
Additional Context
I would like to improve my development and communication skills by joining EddieHubCommunity. I discovered this group by browsing the Github global campus website.

Welcome to the community!


default implementation for enums that have a raw string value

It would be awesome for enum's that have a rawValue String to have implicit conformance to Decodable. If you think it's a good idea @ikesyo, I can go ahead and start a PR 😄

This is it: https://github.com/ikesyo/Himotoki/blob/3.0.0/Sources/RawRepresentable.swift. You just need to do that:
enum YourEnum: String, Decodable {
    case A, B, C
}


Add video support

videos in /first-rows and /rows will be like
{"src": "https://huggingface.co/datasets/username/dataset_name/resolve/commit/hash/path/to/video.mp4"}

when possible, otherwise the back-ends store videos in the assets on S3 as usual (e.g. if they come from parquet or webdataset data or zipped data)
TODO: support hf:// urls in URLPreparator

I remember you mentioned at one point that some video dataset have one video, and each row refers to a range [start, end], which would be accessed as "media fragments" in HTML: https://developer.mozilla.org/en-US/docs/Web/Media/Audio_and_video_delivery#specifying_playback_range. Is it still a thing, or do we just have one row = one video?

one row = one video (we'll see later if we need to support other cases)


All proxies unchecked except one

Proxies(good: 1, dead: 0, unchecked: 19999, reanimated: 0, mean backoff time: 0s)
Proxies(good: 0, dead: 1, unchecked: 19999, reanimated: 0, mean backoff time: 572s)
I have the list of 20,000 proxies in this format:
username1:password1@host:port
username2:password2@host:port
username3:password3@host:port
username4:password4@host:port
username5:password5@host:port
username6:password6@host:port
username7:password7@host:port
All ips and ports are the same, but usernames and passwords are different.
With this format only one ip is used and other are unchecked even if I have 20,000 proxies.
Can you please help me?

Getting the same issue,

@darshanlol is right, it involves proxy authorization, but not directly.
The problem is in the get_proxy method from expire.py: if you are using authorized IP addresses, this method will retrieve only the root domain from the proxy. The Scrapy driver removes the authorization part of the proxy address and it messes up the good/dead/unchecked marking.
I just added another key to the request.meta with the original proxy IP and it worked. It's ultimately a workaround to the aforementioned issue, but it works.
    def process_request(self, request, spider):
        if 'proxy' in request.meta and not request.meta.get('_rotating_proxy'):
            return
        proxy = self.proxies.get_random()
        if not proxy:
            if self.stop_if_no_proxies:
                raise CloseSpider("no_proxies")
            else:
                logger.warn("No proxies available; marking all proxies "
                            "as unchecked")
                self.proxies.reset()
                proxy = self.proxies.get_random()
                if proxy is None:
                    logger.error("No proxies available even after a reset.")
                    raise CloseSpider("no_proxies_after_reset")

        request.meta['proxy'] = proxy
        request.meta['download_slot'] = self.get_proxy_slot(proxy)
        request.meta['_rotating_proxy'] = True
        request.meta['_original_proxy_url'] = proxy    # adding new variable here

...

    def _handle_result(self, request, spider):
        proxy = request.meta.get("_original_proxy_url", None)      # changing proxy variable to grab from request.meta
        if not (proxy and request.meta.get("_rotating_proxy")):
            return
        self.stats.set_value(
            "proxies/unchecked",
            len(self.proxies.unchecked) - len(self.proxies.reanimated),
        )
        self.stats.set_value("proxies/reanimated", len(self.proxies.reanimated))
        self.stats.set_value("proxies/mean_backoff", self.proxies.mean_backoff_time)
        ban = request.meta.get("_ban", None)
        if ban is True:
            self.proxies.mark_dead(proxy)
            self.stats.set_value("proxies/dead", len(self.proxies.dead))
            return self._retry(request, spider)
        elif ban is False:
            self.proxies.mark_good(proxy)
            self.stats.set_value("proxies/good", len(self.proxies.good))

I can confirm this one.
It seems that requests go out with the different usernames & passwords initially even though the reporting doesn't work.
However retries tend to use the same bad proxy again and again essentially making them useless. So this one is not just about reporting. I'm not sure whether the above workaround also helps with that.
Maybe it's because [1] removes user & password from request.meta['proxy'], which is then later returned with the response and hits [2] incl. the _retry().
[1] https://docs.scrapy.org/en/latest/_modules/scrapy/downloadermiddlewares/httpproxy.html#HttpProxyMiddleware
[2] https://github.com/TeamHG-Memex/scrapy-rotating-proxies/blob/master/rotating_proxies/middlewares.py#L161

I guess this one is not too uncommon as rotating proxy providers tend to implement APIs via the usernames.


(SIMP-2231) Create timezone module

Timezone used to be managed in simplib.
Updated saz/timezone for Puppet 4, including strongly typed modules and
data in modules.
SIMP-2231 #close
SIMP-1679 #comment created timezone module

This is a forked module and doesn't need our module assets.


Auto Update Art - Single Click for All Systems

Navigating to each individual system is time consuming.  If at all possible, could a feature be added for a one-click Auto Update Art for all systems so that every ROM is queued and can be left alone to update?

This is now in the latest release!


Android crash - error couldn't find key cache in dynamic object

Hi
Im getting an error while trying to open docs (www) throw android,
Error says error couldn't find key cache in dynamic object
you can see it with the readme example of openDo method

Load  the new Version 2.5.8

Still the same error couldn't find key cache in dynamic object
Installed version 2.5.8 and recompiled all.
I have tried both my own http pdf and the example one, both give the same error
OpenFile.openDoc([{
url:"http://www.snee.com/xml/xslt/sample.doc",
fileName:"sample",
fileExt:"",
fileType:""
}], (error, url) => {
if (error) {
console.error(error);
} else {
console.log(url)
}
})

Sorry the readme only includes the ios part, i have uploaded the android part. Version 2.6.0

Thanks,
Now i got a new issue
When trying to get http pdf i get this error :
ExceptionsManager.js:73 Activity not found to handle: content://com.reactlibrary.provider/name/PP_1206215886.pdf (application/pdf)
this is the way i try to get the file
OpenFile.openDoc(
[
{
url:
'http://che.org.il/wp-content/uploads/2016/12/pdf-sample.pdf',
fileName: 'test',
cache: false,
fileType: 'pdf'
}
],
(error, url) => {
if (error) {
console.error(error)
} else {
console.log(url)
}
}
)

Please read the readme, at the moment it is not possible to open a pdf In the app. You have to install a external application to open the pdf.
Von meinem iPhone gesendet

Am 22.11.2017 um 19:22 schrieb zivchen<EMAIL_ADDRESS>Thanks,
Now i got a new issue
When trying to get http pdf i get this error :
ExceptionsManager.js:73 Activity not found to handle: content://com.reactlibrary.provider/name/PP_1206215886.pdf (application/pdf)
this is the way i try to get the file
OpenFile.openDoc(
[
{
url:
'http://che.org.il/wp-content/uploads/2016/12/pdf-sample.pdf',
fileName: 'test',
cache: false,
fileType: 'pdf'
}
],
(error, url) => {
if (error) {
console.error(error)
} else {
console.log(url)
}
}
)
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.

@philipphecht Thank you releasing this project to the open source community.
I'm using the 2.6.3 version and the issue still persists.
After some googling the solution seems to be the cache key must be present in the config object passed to OpenFile.openDoc now it is working for me after adding cache key.
In short terms if you get the error couldn't find key xxx in dynamic object go make sure you have the key xxx defined in OpenFile.openDoc config object.
Dropped my comment here to help future questions about this issue.


Make sure all cirq gates decompose to XPow/YPow/ZPow/CZPow + Measurement

As part of https://tinyurl.com/cirq-organize-decompose-rfc, we want to make sure that all cirq gates decompose to the XPow/YPow/ZPow/CZPow + Measuremet gateset.
This issue is to track all the gates where we need to explicitly add a _decompose_ method to make sure all decompositions end on the above target gateset.
We should also discuss what happens for gates where trivial decomposition is not present?

MatrixGates : Should we add an analytical decomposition to the _decompose_ method of matrix gates?
Same question as above for ControlledGates.

P1 - I need this no later than the next release (end of quarter)

Should we add an analytical decomposition to the decompose method of matrix gates

How would this work? Is it possible to decompose any arbitrary MatrixGate into a target gateset?

@vtomole Yes. We have utilities in Cirq to decompose any arbitrary 2q Matrix Gate into the CZ gateset (see See https://github.com/quantumlib/Cirq/blob/master/cirq-core/cirq/transformers/analytical_decompositions/two_qubit_to_cz.py) and there's an open issue https://github.com/quantumlib/Cirq/issues/4022 to add compilation of any arbitrary unitary to 1/2q gates

I also think we should have MeasurementGate with invert_mask decompose into a series of pre- and post-measurement Xs that represent the invert mask (and the subsequent uninversion), so that "measurement" can be a uniquely defined thing.


input bam file has always the same name

input bam file has always the same name in the MarkDuplicates process
before, it was called runId.bam for single pair fastq and sampleId.bam for multiple pairs fastq

PR checklist

[X] PR is made against dev branch
[X] This comment contains a description of changes (with reason)
[X] Ensure the test suite passes (./scripts/test.sh -p docker -t ALL).
[x] CHANGELOG.md is updated

I guess we never really checked the names of the metrics file from this process.
But Pontus did notice, so I figured it was an easy fix ;-)


#16: Change the Row component to a FlexBox component

Change the name of the component and add an option for a row or column FlexBox

FlexBox has replaced Row in 0.2.6

FlexBox has replaced Row in 0.2.6


blog/avr/asm/include/ files are executable?

Hi,
I have just downloaded your collection of inc files, and after extracting they seem to be executable (755), all except one - tn13def.inc, which has mod bits 664.
I didn't even know that git manages such attributes, but that's worth fixing.
Take care!

Hi. Ooops, thanks for reporting it! Will send a fix soon.
Take care, L


Unified Console not working on IE11

Trying to launch Unified console on IE 11 browser, getting an error on console:
"Exception Thrown and not caught".

Will be fixed.


Proxy issue

fix proxy issue with
var HttpsProxyAgent = require('https-proxy-agent');
...
fetch(url,{ agent:new HttpsProxyAgent('http://proxy_address:port')}).then((resp) => {

We're not using https-proxy-agent anywhere in the code, not sure what you mean by this.

it's example for those, who work behind proxy with this Application. Need to add some config property for this.

I had the same problem, working behind a firm proxy. It saved my time, thank you @geminixandroid .


SimpleImpersonation in Linux

Hello,
I was wondering if there were any plans to support Linux with SimpleImpersonation? I'm receiving the following error when trying to run it:
Unable to load DLL 'advapi32.dll': The specified module or one of its dependencies could not be found.
My company is migrating away from Windows to Linux and this library has been supremely awesome to use. I'd really prefer to try and continue with it as opposed to having to look for workarounds.
Thanks,
Victor

See "Platform Support" in the readme.
https://github.com/mj1856/SimpleImpersonation#platform-support
Such an effort would require a deeper understanding of authentication models in these environments than I possess.
Another way to think about it is that under the hood we are working with WindowsIdentity objects and the Win32 LogonUser API.  These don't exist outside of Windows.

I read it on the readme, but was more curious about future plans. The reason for developing .NET Core was to allow for devs to use .NET in a Linux setting, particularly with AWS in play. If this library allow for us to interact with our Windows machines from Linux via Active Directory, it would be tremendously useful and fill a niche in the .NET ecosystem.

I'd recommend exploring the various client libraries for Azure AD start here.  Or for on-prem AD, you'll need to look at enabling ADFS, or using old-school tech like LDAP.
SimpleImpersonation is really just a simple library to work with auth on the local machine.  The only remote auth it can offer is when it is provided by the local OS.  For instance, there's nowhere here where you specify the IP address or DNS name of a domain server to authenticate against.  If the local Windows computer can't find the domain you provide, then it's not going to work.
If there ever were a Linux story for this library, it would be around similar concepts - whatever the Linux equivalents are.  Or perhaps it would work with the Linux VM extension for AAD.  But it wouldn't do anything beyond interacting with the local OS or something installed on the local OS.


Cannot define order when defining filter fields in a dictionary

Hi,
When defining filters in a dictionary as in the following example, we loose the ability to define the order of fields. Maybe it is better to use an ordered dictionary instead, or a list of lists (tuple of tuples)?
    fields = {'price': ['lt', 'gt'],
              'release_date': ['exact'],
             }

Example taken from: http://django-filter.readthedocs.org/en/latest/usage.html#the-filter

Yeah. The dictionary syntax is a convenience here — most useful for quickly putting together FilterSets, and inspired from tie-in usage when creating APIs (where order isn't important) with Django Rest Framework.
If order is important to you, drop back to defining the filter attributes individually and provide fields as a list.
(Of course I'm open to a Pull Request that handles using an OrderedDict but it's not something I'm going to work on per se.)

I've also stumbled upon this issue and the following seems to work for me:
        fields = OrderedDict([
            ('subtype', ['exact']),
            ('price', ['lt', 'gt']),
            ('description', ['icontains',]),
            ('created_at', ['year__gt'])
        ])

@carltongibson, I understand this is not exactly guaranteed behaviour since it's not supported but do you think it's likely to break at some point?


hostman: options: remove k8s_cluster_cidr

这个 PR 实现什么功能/修复什么问题:
Remove instead of deprecation.  No user of this option anymore

是否需要 backport 到之前的 release 分支:

release/3.2

/area host
/cc @wanyaoqi @zexi @swordqiu

Codecov Report

Merging #6580 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #6580   +/-   ##
=======================================
  Coverage    7.16%    7.16%
=======================================
  Files         647      647
  Lines      109213   109213
=======================================
  Hits         7829     7829
  Misses     100783   100783
  Partials      601      601

Flag
Coverage Δ

#aFlag
7.16% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 48fb78f...34027b1. Read the comment docs.

/lgtm


CLI tool to inspect stores

Summary
I am adding a new CLI tool that fetches information from all stores (discovered through DNS SRV records) and display which stores have blocks that match a set of label pairs.
I am just laying the foundation here for other sub-commands to come, I don't want to add all of them in a single PR to avoid having a HUGE PR.
Note for reviewers: I did not know where to put auxiliary code for this tool. So I added it to cmd, let me know where it would be better to put this.
One other idea I had was to move these tools to a repository in the community org, then we could just import runnable functions here. This we could decouple lots of logic that lives inside of thanos but are not related to thanos directly.
Related to #5399

[ ] I added CHANGELOG entry for this change.
[ ] Change is not relevant to the end user.

Preview
Here is one example execution of the command in a deployment with over 200 store-gw instances:

Changes

Verification

Hi @pedro-stanaka, could you please also attach some screenshots of how this tool looks like?

Hi @pedro-stanaka, could you please also attach some screenshots of how this tool looks like?

Added to the PR body.

Hello folks. Is this still of interest? Otherwise I can close the PR.

/remind me in 1 week


Replace akashnet.net rpc with proxy

We're depedent on this node to run 100% of the time for the api and other places in the code. We need to replace it with the new RPC proxy that will be built,

Hey @troian and @baktun14 ,
Public endpoints tend to enforce rate limits per IP address. As far as I know, using a proxy server would result in all users sharing the same rate limits if no caching is in place. Due to this, I recently created a REST API endpoint for another frontend that checks the health of public RPCs/RESTs and provides an updated list. Putting the RPC selection on the client side would ensure availability even if usage of the endpoints increases significantly.
You can check it out here: https://9c0pbpbijhepr6ijm4lk85uiuc.ingress.europlots.com/api/rpcs/akash

Hey, we currently have client side rpc selection. This is mostly for our api that relies on the akashnet.net rpc and we need a proxy for reliability. Thanks for the list of rpcs.


Cursor is not in correct location

After typing anything in chat, the cursor becomes inaccurate if trying to relocate

I am using tabbychat 2.0.0-beta-snapshot.92.litemod, mnm-util-0.3-snapshot.92.litemod and macros-compat-1.0.1.litemod

dfa595eb478f8d7fec6805798552ad96ec64b139
I think that's the issue you're talking about, but I'm not sure. Is it with the actual chat or the chat input?

Sorry for not answering back earlier. Quite busy with work and family. It is the input chat, anytime you type it slowly starts to go offtrack of the next character.

I've already fixed it. Can you test?
On Fri, Nov 20, 2015 at 2:31 PM Josiah Jones<EMAIL_ADDRESS>wrote:

Sorry for not answering back earlier. Quite busy with work and family. It
is the input chat, anytime you type it slowly starts to go offtrack of the
next character.
—
Reply to this email directly or view it on GitHub
https://github.com/killjoy1221/TabbyChat-2/issues/35#issuecomment-158502721
.

Somewhat related.
a38e15908320d40b013df85d930a4ce1c708585f


Automated cherry pick of #2043: guest export keys add zone

Cherry pick of #2043 on release/2.11.
#2043: guest export keys add zone

Codecov Report

:exclamation: No coverage uploaded for pull request base (release/2.11@05650ca). Click here to learn what that means.
The diff coverage is n/a.

@@              Coverage Diff               @@
##             release/2.11   #2044   +/-   ##
==============================================
  Coverage                ?   7.18%
==============================================
  Files                   ?     445
  Lines                   ?   69272
  Branches                ?       0
==============================================
  Hits                    ?    4980
  Misses                  ?   63878
  Partials                ?     414

Flag
Coverage Δ

#aFlag
7.18% <ø> (?)

Impacted Files
Coverage Δ

pkg/compute/models/guests.go
0.69% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 05650ca...5ecc490. Read the comment docs.

/lgtm
/approve


datepicker: "Maximum call stack size exceeded`

Reproduction
https://formkit.link/b7647b1d60174c0788f2f3da4b1fedc7
Describe the bug
An infinite loop is triggered when advancing to the year part of the date picker (after first selecting a day and month, and you start typing a year).

Environment
• OS: Windows
• Browser: Chrome
• Version: 126.0.6478.127

Hmm. perhaps this is a windows thing? On chrome on macOS im unable to duplicate it.

@justin-schroeder Looks like it. Doesn't reproduce on my Mac, not even in Windows running under Parallels. But on my "real" Windows-PC it reproduces in both Chrome, Edge and Firefox.

Having same issue on Mac OS. Any updates?

@justin-schroeder same issue here
Reproduction : https://formkit.link/40ab8b24282190ae90c0a49b490ee818

Hmm, still cannot duplicate this issue. @TarSen99, @kevcomparadise, @hognevevle — using @kevcomparadise reproduction (https://formkit.link/40ab8b24282190ae90c0a49b490ee818) can someone confirm these details:

Your browser (and OS)

Your timezone

Your locale

Browser Chrome
Timezone GTM+1
Locale : FR

@kevcomparadise Yeah, seems to be be throwing the error in certain time zones. Will take a look at this.

@hognevevle @kevcomparadise @TarSen99 Update: an issue for this has been filed at @formkit/tempo https://github.com/formkit/tempo/issues/67.


Add support for running files with #delimit ;

If code is like this:
di 2 +
    2;

and a "#delimit-mode" is turned on, then select all code (except for comments) up until a ;. Don't look for a ; that comes after ///.

Closing as this can be done with the Stata Jupyter kernel


ftp or ftps

I installed:

git clone https://github.com/evertramos/docker-wordpress-letsencrypt.git and setup my web site
git clone https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion.git

It works fine! ;-) Thanks a lot!!!
I login to my wordpress site, and try to add contactform 7 and pagebuilder. However, both require a connection ftp or ftps. How do I add this into that system?
I tried to add a user and the invitation email is not sent. What do I need to do to get that to work?

I fixed that by chown www-data:www-data of wp-content directory.
Mail I could not fix yet. Any hint?

I still have not looked to the emails... if you find a solutions please come back.


updates docs for 1.5

cf. https://www.pivotaltracker.com/story/show/162097213

Good stuff @originalhat @iplay88keys


Sink to a Web Socket server?

I am trying to implement a web socket sink.
I an running https://hub.docker.com/r/jmalloc/echo-server as a simple server (the "host" and "port" in the code below) to test delivery of messages.
Below is the simple (and probably naive) code I have used as a test:
import os
import time
from streamz import Stream
# pip install websocket-client
import websocket
from websocket import create_connection

def increment(x):
    return str(x * x)

def wss(x, url, websocket):
    while not websocket.connected:
        print('reconnecting')
        websocket.connect(url)
    try:
        print('SND:', x)
        r = web_socket.send(x)
        print('RCV', r)
    except WebSocketTimeoutException as err:
        print(err)
    except Exception as err:
        print(err)

host = os.getenv("WEBSOCKET_HOST", "<IP_ADDRESS>")
port = os.getenv("WEBSOCKET_PORT", "8345")

web_socket = create_connection(
    "ws://{}:{}/".format(host, port),
    timeout=30)
source = Stream()
source.map(increment).sink(wss)

for j in range(1, 4):
    print('\nstarting loop:', j)
    for i in range(1, 10):
        source.emit(i * j)
    time.sleep(60)

web_socket.close()

This seems to work, but if I change this setup to use an asynchronous source instead, e.g. data from a message broker which arrives intermittently, then very soon (after a dozen messages) the WebSocketTimeoutException is raised with each subsequent message being sent.
Is there a documented and straightforward way to reliably sink messages into a web socket?
If not, are there guidelines to follow - especially as we'd like to extend the range of sinks; and it seems possible there is a more generic approach that will apply when connecting to similar dynamic end-points?

I think indeed you will need to subclass Sink and implement async def update. I agree that there should be async examples, and your websocket would be a perfect case.

@martindurant Thanks for the suggestion.
On a related note, I'd be very interested to know what the developers have planned for this project going forward. As I mentioned elsewhere, it would be great if external sources/sinks, based on widely-used data streaming technologies including MQTT, AMQP and Web Sockets, become standard, first-class citizens in streamz. This would significantly increase its applicability in a wide range of scenarios.

I think it's fair to say that this project is not under heavy development right now. It fills an interesting niche in python-land and could have a variety of use-cases, but currently not too many examples that can be shared publicly. For those that use it, it seems to perform well for either event-by-event or mini-batch realtime processing.

@martindurant Thanks for the reply.  I am not quite sure how to interpret this.  Do the developers not plan any more work for the foreseeable future? Or if they do have plans then they are not going to tell anyone?
Am I better off forking and developing it myself, or submitting pull requests (on the assumption that any changes will be acceptable)?

Do the developers not plan any more work for the foreseeable future?

I meant to say that there are no specific new features in development that I am aware of. That doesn't mean that there won't be smaller addition (we continue to handle PRs and releases). If there are bigger ideas around or show-cases that come to light, we'd be happy to take part, time and effort permitting.

Unfortunately, I am no longer going to be working on streaming data in the foreseeable future.
It would be really great though if someone could work on including this code (with improvements and tests as required of course).  Websockets are widely and extensively used in real-world applications and having this as a "first class" feature would help streamz gain wider adoption.  AFAIK there is no other Python library tackling this domain.


Remove integration tests code coverage

Current coverage is 63.07% (diff: 100%)

No coverage report found for master at ad6708f.

Powered by Codecov. Last update ad6708f...f62d493


Aggregate error is shadowed

No need to wrap it otherwise hard to inspect details.

WebPubSub JS Client issue @zackliu

Fixed


bridge Webfail results in Error 500

Sadly the Webfail bridge does not work anymore, anything what I do results in an error 500.
rss.xxx.xx:443 <IP_ADDRESS> - xxxx [02/Mar/2017:21:07:24 +0100] "GET /bridge/?action=display&bridge=Webfail&language=en&type=%2F&format=Html HTTP/1.1" 500 370 "https://rss.schmidt.lc/bridge/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
Nothing more in the log files :/

Fixed in 100f3cd56d8a972af107c0f4e63514601c916b83.

Sadly it does not fix the problem, still getting just Error 500 :(

Very weird. It seems that they blacklist weird user-agents, but not immediately.
I changed it again to using Firefox user agent. Should no longer cause troubles.

Yeah! It is working now, thanks :D


add stream support

Add stream support according to event_stream_format

Thank you for your suggestion. Using callback function is a great idea to deal with async response in for example JavaScript, however, I'd like to take different approach like chan in this Go code.

relate #9

resp, err := client.HTTPClient.Do(req)
check err and close body

maybe try generic function

Maybe set header X-Accel-Buffering: no for nginx

伙计们，谢谢你的评论。我创建了一个关于选项的拉取请求stream。#11 有时间请看一看并发表评论。

Current version non-streaming requests do not go swich default
Maybe try
type Streamble[T any] interface {
	Stream() <-chan T
}
	v, ok := (response).(Streamble[any])
	if ok {
		if v.Stream() != nil {
			go handle(v.Stream(), httpres.Body)
			return nil
		}
	}
	return decode(response, httpres.Body)

Thank you @xiao333ma and @Naist4869.
After taking some time to write a bunch of code based on my idea with chan, for more safety and usability, I came to the conclusion that using callback is better.
Let me respect your commit and I'll merge it into develop branch later. Please stay tuned and thank you for your initiative.


Very slow on long lists

It takes ages to check a file with very long statement (for example https://github.com/WeblateOrg/language-data/blob/master/weblate_language_data/check_languages.py takes minute and half to check).
The problem seems to be in:
https://github.com/sobolevn/flake8-eradicate/blob/a9866301eb145be5b2f17ae58532b549d97c53a0/flake8_eradicate.py#L126-L129
The _tokens accumulate until the statement parsing is complete and flake8-eradicate checks all the tokens in the list for every line.

@nijel PRs are welcome! I would love to release a version with the fix.

Unfortunately, my knowledge of flake8 plugins is exactly zero, otherwise the PR would be already there...


Various VPI examples

In this PR four VPI usage examples are added.
VPI examples
The directory contains some VPI usage examples:

helloworld: minimal VPI code example that runs a simulation, prints a message and exits.
access: signal read/write example on an adder component using vpi_put_value and vpi_get_value.
list: example on signal hierarchy iteration using vpi_iterateand vpi_scan.
timestep: shows how to run a simulation for an arbitrary number of timesteps.

to run a test, just go in an example directory and execute
sh run.sh

Thank you for the feedback! I am waiting for your response :)

I don't understand why I cannot reply to some of the previous (outdated) comments, but I can reply to some other. Hence, I think that the review and this comment might be a little messy. Hope we can understand each other...

Sorry, but is this really helpful to make the example easy to understand from people with basic knowledge? I can change it but adding a generic just adds unnecessary information and details for the purpose of the example.

As commented in https://github.com/ghdl/ghdl-cosim/issues/1#issuecomment-622045044, we target people with basic knowledge about using GHDL and VPI at the same time. But we expect users to know VHDL and VPI, or we provide them the references to go and learn.
Apart from that, the purpose of this examples is to learn. An important part of learning a new technology is being able to play with it. Hence, we are ok with using features which add very little complexity from VHDL's perspective, and which make it easier for users to play with this, our toy. Think e.g. of a user that wants to test what would happen with signals which are larger than 32 or 64 bits.
The question might or might not make sense, but in any case we don't want him/her to stop doing it because of how cumbersome it is to find and change hardcoded values.
From this point of view, a generic or a constant can be used, but it is general bad practice (in any language) to hardcode sizes. Of course, I don't mean to criticize your coding; just to draw the limit for oversimplification.

Is it possible to pass multiple *.vpi arguments? Or do all VPI features need to be built into a single module?

The answer is I don't know.

I will ask.

I can put it out of common. I don't think that it can be added to vpi_user.h since vpi_user.h should contain only VPI calls/definitions that are defined by the IEEE PLI 2.0 standard. I don't know what is the GHDL policy about including such kind of code. So I cannot give any opinion on that :P Sorry.

Let's keep it as vpi/common.c and vpi/common.h for now, and I'll ask Tristan about providing a vpi_user_ghdl.h.

So, do you suggest to put every common vhd file in the same directory?

The structure that we are following is that each new file that is added to the repo, is required for a reason. If two examples use a source which is almost the same except for a line or a few lines, we consider whether it can be shared by adding some parameter. Should doing so involve too much complexity, it is ok to duplicate very similar files.

Even if some examples contain similar or same code, I would not recommend moving them out of the particular example folder to limit the necessity to look around to the minimum (and IMHO it is very important in a documentation project).

Also, in this way the user loses the ability to compile/run a single test, so I discourage this solution. For code examples IMHO it is more important self-containment rather than code reuse.

We need to balance complexity and verbosity. On the one hand, we want intermediate and advanced users to pick any isolated example and run/use it directly. On the other hand, we want unexperienced users which are reading the docs in order (as a tutorial), not to be continuously diffing many files that look similar and which might or might not be the same.
We are currently handling this as subsections (groups of examples). So, in this PR the "atomic unit" is vpi/quickstart. The script that is guaranteed to work alone is vpi/quickstart/run.sh. Then, vpi/quickstart/access/run.sh might exist, and it might be completely independent, but this is not guaranteed. In some cases, you need to call the parent script first. In other cases, the parent script does execute the child explicitly.

Please, give an example to do this in a proper way.

See, for example, https://github.com/ghdl/ghdl-cosim/tree/master/vhpidirect/quickstart/sharedvar or https://github.com/ghdl/ghdl-cosim/tree/master/vhpidirect/arrays/intvector.
Overall, the docs define which examples exist and how they are named. Sources might or might not match the structure.

I didn't get what you say when you write

Make access be a case of helloworld

Since they are two distinct examples I wouldn't recommend mixing them together.

Both are subexamples of "quickstart":

vpi/quickstart/

ent.vhd
tb.vhd
vpi_hello.c
vpi_helloworld.c
vpi_access.c
vpi_access_hierarchy.c
vpi_timestep.c

Or, to make the distinction more clear:

vpi/quickstart/

ent.vhd
vpi_hello.c
vpi_helloworld.c
vpi_timestep.c
access/

tb.vhd
vpi_access.c
vpi_access_hierarchy.c

Moreover, if the entity and testbench will be shared with other examples:

vpi/

ent.vhd
tb.vhd
quickstart/

vpi_hello.c
vpi_helloworld.c
vpi_timestep.c
access/

vpi_access.c
vpi_access_hierarchy.c

The point is that 5 examples exist in the docs, independently of which of these structures is used for the sources.
It is easier to understand for new users that they need to focus on different *.c files only, because the "already know" the content of others. Intermediate/advanced users can quickly cherry-pick the subset of files they need. They are not going to reuse the provided run.sh anyway.

The thing is that to demonstrate very basic usage of VPI, the VHDL component under test do not matter so much. So, for helloworld I can, for example, just put a complete void top module without any signals, because at the end the VPI code is interacting with the simulator but not with the simulation. If you think it is better, I can try to make the ent.vhd and tb.vhd even more concise in each test.

This. In VHPIDIRECT examples, many C sources are common but VHDL sources need to be different. Here, as you said, VHDL sources are almost irrelevant (as long as they contain the minimum features that we want to test). Hence, instead of making ent.vhd and tb.vhd more concise in each test, I would suggest to make them general enough so that they can used in any VPI example (last structure above). If some example in the future requires some weird feature in VHDL, we will consider adding different VHDL sources to the corresponding subdir. But, for now, it seems unnecessary.

Hi,
I think that I won't work anymore on this PR. Either because I don't believe that, given the scope of the PR, implementing more changes will lead to any quality improvement, and (more important) because I don't have more time to spend on this. So, take it as it is and do what you want with the PR. You can merge it and do the changes you prefer later, you can reject it or you can make it as a pending PR indefinitely. I am fine with whatever decision you will take.


Scaffold-DbContext failed while using C# 7.0 above (exclude 7.0)

Exception message: build failed

Steps to reproduce
dotnet new mvc --auth Individual
use identity scaffolder to scaffold identity code
edit csproj and add
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
  <LangVersion>latest</LangVersion>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'">
  <LangVersion>latest</LangVersion>
</PropertyGroup>

then edit Program.cs:
change Main method to
public async static Task Main(string[] args)
{
    await CreateWebHostBuilder(args).Build().RunAsync();
}

and build it to make sure it can be built successfully.
then use Scaffold-DbContext or dotnet ef dbcontext, it shows "build failed"
ALSO, IF YOU CHANGE THE MAIN METHOD TO ASYNC METHOD BEFORE SCAFFOLD IDENTITY, THE IDENTITY SCAFFOLDER WILL FAIL AND SHOWS:

What's more, I found a new scaffolder bug that, it will keep original namespace the same as folder name "New folder" but not "New_folder", which means the generated code can not be compiled successfully unless user replace the blank space with "_" manually.
Further technical details
EF Core version: 2.1 rc1 (found in project.csproj or packages.config)
Database Provider: Microsoft.EntityFrameworkCore.Sqlite (e.g. Microsoft.EntityFrameworkCore.SqlServer)
Operating system: Windows 10 1803 x64
IDE: Visual Studio 2017 15.7 (e.g. Visual Studio 2017 15.4)

@bricelam I've updated the steps to repo this problem

I hit this today too.
I also found that if you've moved the SqlDbContext in to another project so it can be referenced by other projects then it can't find the SqlDbContext in the drop down.

@hez2010 @AndyCJ The issue was reported on a 2.1 RC release. Does this still happen with the latest 2.1 releases of the SDK and EF Core?

@ajcvickers This problem still exists when scaffolding identity pages in .NET Core SDK 2.1.403. But Scaffold-DbContext and dotnet ef dbcontext works fine.
And also I found that the scaffolded identity pages miss a "ShowRecoveryCodes" page.

I gave it a quick test and it seems fine for me now. I'm not sure what I messed up. I must have been on an old version as you say.

Created new mvc project via dotnet "dotnet new mvc --auth Individual" (.NET Command Line Tools (2.1.402)).
Changed project language version to "latest minor version" via VS (15.8.6).
Restored NuGet packages via VS.
Updated NuGet packages via VS.
Changed Program.cs to have "public static async Task Main(string[] args)" as the entry point.
Ran "Scaffold-DbContext -connection "DataSource=app.db" Microsoft.EntityFrameworkCore.Sqlite" via the package manager console.


Definition of a template to describe OSS communities

SC-TEI-1

Added template management in layer definition

what is "template management in layer definition"?

is this the same that issue #59? Is something new?

ok, we need to mark that is the same than issue #59. But it is not closed yet, we didn't test it

This is a placeholder to mark that we took into account TEI requirement SC-TEI-1


Set up ONAP OpenStack environment on Packet for VNF demo

Goal: Set up ONAP OpenStack environment on Packet for VNF demo
Tasks:

[x]  Create new VM on Packet or Digital Ocean:

[x]  Ubuntu 16.04
[x]  64bit
[x]  125G memory
[x]  1T disk.

[x]  Clone Gerrit repo => https://gerrit.onap.org/r/p/demo.git , https://github.com/onap/demo/

[x]  Use Vagrant scripts to run demo scripts

[x]  Fix issues with Vagrant

[x]  Fix issues with OpenStack

OpenStack setup is documented in https://github.com/cncf/onap-demo/issues/1

Setting up OpenStack Ocata in Virtualbox on Ubuntu 17.10
Provision base dev software and pull down onap-demo code:
git clone git@github.com:cncf/onap-demo.git -b amsterdam-openstack-ocata

cd onap-demo

./vagrant/run_demo.sh

Deploy the router to new packet server:
cd router-node ; vagrant plugin install vagrant-reload ; vagrant up

Deploy openstack to new packet server:
cd ../vagrant ; vagrant up control ; vagrant up control-1 ; vagrant up control-2

Register compute nodes as resources for OpenStack to use:
vagrant ssh control -- source /vagrant/openrc \; nova-manage cell_v2 simple_cell_setup


DrupalVM seems pretty slow by default

`: -->
Issue Type

Bug Report / Support Request

Your Environment

VBox 5
Ansible 2.2
vagrant 1.9.3

Your OS
windows 10
Full console output

Summary
after vagrant up/provision the drupalvm.dev seems sluggish. is it optimised with cacheing/memcache/opcode etc by default or do i need to enable all of that? is there a set of recommended optimisations?
comparing this against a drupal docker container installed today which seems to run 5-10 times faster on the same machine running from an Ubuntu docker host...

@laurencefass - On Windows, the primary reason is because the default (and safe, but terribly slow) synced folder type is vboxsf. See http://docs.drupalvm.com/en/latest/getting-started/installation-windows/#troubleshooting-vagrant-synced-folders (especially the performance section) for more info.
Closing this in favor of more centralized discussion in https://github.com/geerlingguy/drupal-vm/issues/1242


Recover Files vs. Version - different naming in context menu and synch view

In the synchronization view, the context menu item is labeled as "Recover File" while in the Shell context menu it says "Versions" - we might want to have the same name in both locations. I think "Recover Files" sounds more intuitive.

I changed the commands to:

Hard Delete
Recover File Version
Share Folder

It should be easier to understand each command now.


Repeated calls with new parameters produce still the same result

    // Generate a GraphQL query
    const query1 = UserQuery({
      id: 1337,
      count: 10
    })

    console.log(query1.user.params)  // => {id: 1337}

    const query2 = UserQuery({
      id: 243,
      count: 5
    })

    console.log(query2.user.params)  // => {id: 1337}

Nice catch. I'll fix this tomorrow.

Should be ok now. I've added a test case for that. Thanks!


Hamaka\Tasks namespace

Hamaka\Tasks namespace to reflect class name used for config in README and also actually used directory.

Thanks!


Frontend main parts

Authentication

Login(admin/user)
Registration
Forgot password

User Profile(s)

Profile view
Edit profile(settings)
Delete profile(settings)

Dashboard(admin/user)

Table list(users list)(admin) (crud*4)
Table list(pages list)(admin/user) (crud*4)
CRUD (page list) (4pages)
Info (chart)


Hyphen in package name (in install_requires) causes confusion in some cases (eg SQLAlchemy / SQLAlchemy-Utils)

Full details below.
In short, if the install_requires contains 'sqlalchemy','sqlalchemy-utils' in that order then setup gives the error:
error: The 'sqlalchemy' distribution was not found and is required by ...
But if the order of those two packages is swapped, it works just fine.

I am on Centos 7, using Python 3.6 and running the following script to install a package:
$ cat sh/make-env.sh
#!/bin/bash

if [ ! -d py ] || [ ! -d sh ] || [ ! -d db ]; then
    echo "run script from root directory"
    exit 1
fi

rm -fr env
python3 -m venv env
source env/bin/activate
pip install --upgrade pip setuptools wheel
pushd py
python setup.py develop

If I use this setup.py:
import setuptools

with open("../README.md", "r") as fh:
    long_description = fh.read()

setuptools.setup(
    name="stp-db",
    version="0.0.1",
    author="ISTI (Andrew Cooke)",
<EMAIL_ADDRESS>    description="Load STP data into a Postgres database",
    long_description=long_description,
    long_description_content_type="text/markdown",
    url="https://code.usgs.gov/EEW/stp-db",
    packages=setuptools.find_packages(),
    install_requires=[
        'psycopg2',
        'sqlalchemy',
        'sqlalchemy-utils',
        'configargparse',
        'colorlog'
    ],
    entry_points={
        'console_scripts': [
            'load_stp=stp_db.load_results:main'
        ]
    },
    classifiers=[
        "Programming Language :: Python :: 3",
        "Operating System :: OS Independent",
    ],
    python_requires='>=3.6',
)

then i see this output:
$ sh/make-env.sh
Collecting pip
  Using cached https://files.pythonhosted.org/packages/5a/4a/39400ff9b36e719bdf8f31c99fe1fa7842a42fa77432e584f707a5080063/pip-20.2.2-py2.py3-none-any.whl
Collecting setuptools
  Using cached https://files.pythonhosted.org/packages/c3/a9/5dc32465951cf4812e9e93b4ad2d314893c2fa6d5f66ce5c057af6e76d85/setuptools-49.6.0-py3-none-any.whl
Collecting wheel
  Using cached https://files.pythonhosted.org/packages/a7/00/3df031b3ecd5444d572141321537080b40c1c25e1caa3d86cdd12e5e919c/wheel-0.35.1-py2.py3-none-any.whl
Installing collected packages: pip, setuptools, wheel
  Found existing installation: pip 9.0.3
    Uninstalling pip-9.0.3:
      Successfully uninstalled pip-9.0.3
  Found existing installation: setuptools 39.2.0
    Uninstalling setuptools-39.2.0:
      Successfully uninstalled setuptools-39.2.0
Successfully installed pip-20.2.2 setuptools-49.6.0 wheel-0.35.1
~/stp-db/py ~/stp-db
running develop
running egg_info
writing stp_db.egg-info/PKG-INFO
writing dependency_links to stp_db.egg-info/dependency_links.txt
writing entry points to stp_db.egg-info/entry_points.txt
writing requirements to stp_db.egg-info/requires.txt
writing top-level names to stp_db.egg-info/top_level.txt
reading manifest file 'stp_db.egg-info/SOURCES.txt'
writing manifest file 'stp_db.egg-info/SOURCES.txt'
running build_ext
Creating /home/andrew/stp-db/env/lib/python3.6/site-packages/stp-db.egg-link (link to .)
Adding stp-db 0.0.1 to easy-install.pth file
Installing load_stp script to /home/andrew/stp-db/env/bin

Installed /home/andrew/stp-db/py
Processing dependencies for stp-db==0.0.1
Searching for colorlog
Reading https://pypi.org/simple/colorlog/
Downloading https://files.pythonhosted.org/packages/2a/81/12d77537c82c5d46aa2721dfee25a0e873ef5920ebd0827152f411effb57/colorlog-4.2.1-py2.py3-none-any.whl#sha256=43597fd822ce705190fc997519342fdaaf44b9b47f896ece7aa153ed4b909c74
Best match: colorlog 4.2.1
Processing colorlog-4.2.1-py2.py3-none-any.whl
Installing colorlog-4.2.1-py2.py3-none-any.whl to /home/andrew/stp-db/env/lib/python3.6/site-packages
Adding colorlog 4.2.1 to easy-install.pth file

Installed /home/andrew/stp-db/env/lib/python3.6/site-packages/colorlog-4.2.1-py3.6.egg
Searching for configargparse
Reading https://pypi.org/simple/configargparse/
Downloading https://files.pythonhosted.org/packages/bb/79/3045743bb26ca2e44a1d317c37395462bfed82dbbd38e69a3280b63696ce/ConfigArgParse-1.2.3.tar.gz#sha256=edd17be986d5c1ba2e307150b8e5f5107aba125f3574dddd02c85d5cdcfd37dc
Best match: ConfigArgParse 1.2.3
Processing ConfigArgParse-1.2.3.tar.gz
Writing /tmp/easy_install-wdtdnxpk/ConfigArgParse-1.2.3/setup.cfg
Running ConfigArgParse-1.2.3/setup.py -q bdist_egg --dist-dir /tmp/easy_install-wdtdnxpk/ConfigArgParse-1.2.3/egg-dist-tmp-7pag0gw3
zip_safe flag not set; analyzing archive contents...
Moving ConfigArgParse-1.2.3-py3.6.egg to /home/andrew/stp-db/env/lib/python3.6/site-packages
Adding ConfigArgParse 1.2.3 to easy-install.pth file

Installed /home/andrew/stp-db/env/lib/python3.6/site-packages/ConfigArgParse-1.2.3-py3.6.egg
Searching for sqlalchemy-utils
Reading https://pypi.org/simple/sqlalchemy-utils/
Downloading https://files.pythonhosted.org/packages/14/68/e5301c4c960c79a32333b8805e52cb69d3d237aa869a773b4157ccb3eb26/SQLAlchemy-Utils-0.36.8.tar.gz#sha256=fb66e9956e41340011b70b80f898fde6064ec1817af77199ee21ace71d7d6ab0
Best match: SQLAlchemy-Utils 0.36.8
Processing SQLAlchemy-Utils-0.36.8.tar.gz
Writing /tmp/easy_install-6rqth3f2/SQLAlchemy-Utils-0.36.8/setup.cfg
Running SQLAlchemy-Utils-0.36.8/setup.py -q bdist_egg --dist-dir /tmp/easy_install-6rqth3f2/SQLAlchemy-Utils-0.36.8/egg-dist-tmp-d_va4wps
warning: no previously-included files matching '*.pyc' found under directory 'tests'
warning: no previously-included files matching '*.pyc' found under directory 'docs'
no previously-included directories found matching 'docs/_build'
warning: no previously-included files found matching 'docs/_themes/.git'
creating /home/andrew/stp-db/env/lib/python3.6/site-packages/SQLAlchemy_Utils-0.36.8-py3.6.egg
Extracting SQLAlchemy_Utils-0.36.8-py3.6.egg to /home/andrew/stp-db/env/lib/python3.6/site-packages
Adding SQLAlchemy-Utils 0.36.8 to easy-install.pth file

Installed /home/andrew/stp-db/env/lib/python3.6/site-packages/SQLAlchemy_Utils-0.36.8-py3.6.egg
Searching for sqlalchemy
Downloading https://files.pythonhosted.org/packages/14/68/e5301c4c960c79a32333b8805e52cb69d3d237aa869a773b4157ccb3eb26/SQLAlchemy-Utils-0.36.8.tar.gz#sha256=fb66e9956e41340011b70b80f898fde6064ec1817af77199ee21ace71d7d6ab0
Best match: SQLAlchemy Utils-0.36.8
Processing SQLAlchemy-Utils-0.36.8.tar.gz
Writing /tmp/easy_install-6pcjaun4/SQLAlchemy-Utils-0.36.8/setup.cfg
Running SQLAlchemy-Utils-0.36.8/setup.py -q bdist_egg --dist-dir /tmp/easy_install-6pcjaun4/SQLAlchemy-Utils-0.36.8/egg-dist-tmp-nm8c14br
warning: no previously-included files matching '*.pyc' found under directory 'tests'
warning: no previously-included files matching '*.pyc' found under directory 'docs'
no previously-included directories found matching 'docs/_build'
warning: no previously-included files found matching 'docs/_themes/.git'
removing '/home/andrew/stp-db/env/lib/python3.6/site-packages/SQLAlchemy_Utils-0.36.8-py3.6.egg' (and everything under it)
creating /home/andrew/stp-db/env/lib/python3.6/site-packages/SQLAlchemy_Utils-0.36.8-py3.6.egg
Extracting SQLAlchemy_Utils-0.36.8-py3.6.egg to /home/andrew/stp-db/env/lib/python3.6/site-packages
SQLAlchemy-Utils 0.36.8 is already the active version in easy-install.pth

Installed /home/andrew/stp-db/env/lib/python3.6/site-packages/SQLAlchemy_Utils-0.36.8-py3.6.egg
error: The 'sqlalchemy' distribution was not found and is required by stp-db

but if the lines are reversed
$ grep sqlal py/setup.py
        'sqlalchemy-utils',
        'sqlalchemy',

then the output is:
$ sh/make-env.sh
Collecting pip
  Using cached https://files.pythonhosted.org/packages/5a/4a/39400ff9b36e719bdf8f31c99fe1fa7842a42fa77432e584f707a5080063/pip-20.2.2-py2.py3-none-any.whl
Collecting setuptools
  Using cached https://files.pythonhosted.org/packages/c3/a9/5dc32465951cf4812e9e93b4ad2d314893c2fa6d5f66ce5c057af6e76d85/setuptools-49.6.0-py3-none-any.whl
Collecting wheel
  Using cached https://files.pythonhosted.org/packages/a7/00/3df031b3ecd5444d572141321537080b40c1c25e1caa3d86cdd12e5e919c/wheel-0.35.1-py2.py3-none-any.whl
Installing collected packages: pip, setuptools, wheel
  Found existing installation: pip 9.0.3
    Uninstalling pip-9.0.3:
      Successfully uninstalled pip-9.0.3
  Found existing installation: setuptools 39.2.0
    Uninstalling setuptools-39.2.0:
      Successfully uninstalled setuptools-39.2.0
Successfully installed pip-20.2.2 setuptools-49.6.0 wheel-0.35.1
~/stp-db/py ~/stp-db
running develop
running egg_info
writing stp_db.egg-info/PKG-INFO
writing dependency_links to stp_db.egg-info/dependency_links.txt
writing entry points to stp_db.egg-info/entry_points.txt
writing requirements to stp_db.egg-info/requires.txt
writing top-level names to stp_db.egg-info/top_level.txt
reading manifest file 'stp_db.egg-info/SOURCES.txt'
writing manifest file 'stp_db.egg-info/SOURCES.txt'
running build_ext
Creating /home/andrew/stp-db/env/lib/python3.6/site-packages/stp-db.egg-link (link to .)
Adding stp-db 0.0.1 to easy-install.pth file
Installing load_stp script to /home/andrew/stp-db/env/bin

Installed /home/andrew/stp-db/py
Processing dependencies for stp-db==0.0.1
Searching for colorlog
Reading https://pypi.org/simple/colorlog/
Downloading https://files.pythonhosted.org/packages/2a/81/12d77537c82c5d46aa2721dfee25a0e873ef5920ebd0827152f411effb57/colorlog-4.2.1-py2.py3-none-any.whl#sha256=43597fd822ce705190fc997519342fdaaf44b9b47f896ece7aa153ed4b909c74
Best match: colorlog 4.2.1
Processing colorlog-4.2.1-py2.py3-none-any.whl
Installing colorlog-4.2.1-py2.py3-none-any.whl to /home/andrew/stp-db/env/lib/python3.6/site-packages
Adding colorlog 4.2.1 to easy-install.pth file

Installed /home/andrew/stp-db/env/lib/python3.6/site-packages/colorlog-4.2.1-py3.6.egg
Searching for configargparse
Reading https://pypi.org/simple/configargparse/
Downloading https://files.pythonhosted.org/packages/bb/79/3045743bb26ca2e44a1d317c37395462bfed82dbbd38e69a3280b63696ce/ConfigArgParse-1.2.3.tar.gz#sha256=edd17be986d5c1ba2e307150b8e5f5107aba125f3574dddd02c85d5cdcfd37dc
Best match: ConfigArgParse 1.2.3
Processing ConfigArgParse-1.2.3.tar.gz
Writing /tmp/easy_install-kal09a6m/ConfigArgParse-1.2.3/setup.cfg
Running ConfigArgParse-1.2.3/setup.py -q bdist_egg --dist-dir /tmp/easy_install-kal09a6m/ConfigArgParse-1.2.3/egg-dist-tmp-nb7yhr5u
zip_safe flag not set; analyzing archive contents...
Moving ConfigArgParse-1.2.3-py3.6.egg to /home/andrew/stp-db/env/lib/python3.6/site-packages
Adding ConfigArgParse 1.2.3 to easy-install.pth file

Installed /home/andrew/stp-db/env/lib/python3.6/site-packages/ConfigArgParse-1.2.3-py3.6.egg
Searching for sqlalchemy
Reading https://pypi.org/simple/sqlalchemy/
Downloading https://files.pythonhosted.org/packages/26/27/18dbd04360651192ec84940190d660c7bdb6bc73d50ebad9cabbea3c606c/SQLAlchemy-1.3.19-cp36-cp36m-manylinux1_x86_64.whl#sha256=8280f9dae4adb5889ce0bb3ec6a541bf05434db5f9ab7673078c00713d148365
Best match: SQLAlchemy 1.3.19
Processing SQLAlchemy-1.3.19-cp36-cp36m-manylinux1_x86_64.whl
Installing SQLAlchemy-1.3.19-cp36-cp36m-manylinux1_x86_64.whl to /home/andrew/stp-db/env/lib/python3.6/site-packages
Adding SQLAlchemy 1.3.19 to easy-install.pth file

Installed /home/andrew/stp-db/env/lib/python3.6/site-packages/SQLAlchemy-1.3.19-py3.6-linux-x86_64.egg
Searching for sqlalchemy-utils
Reading https://pypi.org/simple/sqlalchemy-utils/
Downloading https://files.pythonhosted.org/packages/14/68/e5301c4c960c79a32333b8805e52cb69d3d237aa869a773b4157ccb3eb26/SQLAlchemy-Utils-0.36.8.tar.gz#sha256=fb66e9956e41340011b70b80f898fde6064ec1817af77199ee21ace71d7d6ab0
Best match: SQLAlchemy-Utils 0.36.8
Processing SQLAlchemy-Utils-0.36.8.tar.gz
Writing /tmp/easy_install-awt4608h/SQLAlchemy-Utils-0.36.8/setup.cfg
Running SQLAlchemy-Utils-0.36.8/setup.py -q bdist_egg --dist-dir /tmp/easy_install-awt4608h/SQLAlchemy-Utils-0.36.8/egg-dist-tmp-0_nshz3i
warning: no previously-included files matching '*.pyc' found under directory 'tests'
warning: no previously-included files matching '*.pyc' found under directory 'docs'
no previously-included directories found matching 'docs/_build'
warning: no previously-included files found matching 'docs/_themes/.git'
creating /home/andrew/stp-db/env/lib/python3.6/site-packages/SQLAlchemy_Utils-0.36.8-py3.6.egg
Extracting SQLAlchemy_Utils-0.36.8-py3.6.egg to /home/andrew/stp-db/env/lib/python3.6/site-packages
Adding SQLAlchemy-Utils 0.36.8 to easy-install.pth file

Installed /home/andrew/stp-db/env/lib/python3.6/site-packages/SQLAlchemy_Utils-0.36.8-py3.6.egg
Searching for psycopg2
Reading https://pypi.org/simple/psycopg2/
Downloading https://files.pythonhosted.org/packages/a8/8f/1c5690eebf148d1d1554fc00ccf9101e134636553dbb75bdfef4f85d7647/psycopg2-2.8.5.tar.gz#sha256=f7d46240f7a1ae1dd95aab38bd74f7428d46531f69219954266d669da60c0818
Best match: psycopg2 2.8.5
Processing psycopg2-2.8.5.tar.gz
Writing /tmp/easy_install-ewj8wtpl/psycopg2-2.8.5/setup.cfg
Running psycopg2-2.8.5/setup.py -q bdist_egg --dist-dir /tmp/easy_install-ewj8wtpl/psycopg2-2.8.5/egg-dist-tmp-pno4p5ow
zip_safe flag not set; analyzing archive contents...
psycopg2.__pycache__._psycopg.cpython-36: module references __file__
creating /home/andrew/stp-db/env/lib/python3.6/site-packages/psycopg2-2.8.5-py3.6-linux-x86_64.egg
Extracting psycopg2-2.8.5-py3.6-linux-x86_64.egg to /home/andrew/stp-db/env/lib/python3.6/site-packages
Adding psycopg2 2.8.5 to easy-install.pth file

Installed /home/andrew/stp-db/env/lib/python3.6/site-packages/psycopg2-2.8.5-py3.6-linux-x86_64.egg
Searching for six
Reading https://pypi.org/simple/six/
Downloading https://files.pythonhosted.org/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl#sha256=8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced
Best match: six 1.15.0
Processing six-1.15.0-py2.py3-none-any.whl
Installing six-1.15.0-py2.py3-none-any.whl to /home/andrew/stp-db/env/lib/python3.6/site-packages
Adding six 1.15.0 to easy-install.pth file

Installed /home/andrew/stp-db/env/lib/python3.6/site-packages/six-1.15.0-py3.6.egg
Finished processing dependencies for stp-db==0.0.1

also noticed this on another project that was working fine not so long back (my life seems to revolve around sqlalchemy projects right now), so i suspect it's a regression.

You've encountered #196. Because you're using setup.py develop instead of pip install -e, you're triggering the deprecated easy_install to install the dependencies, and easy_install has a known bug resolving dependencies that share a prefix followed by a dash. It installs sqlalchemy-utils, then when it goes to install sqlalchemy, it thinks it already has the "utils" version of "sqlalchemy" installed, which satisfies the requirement.
Just replace python setup.py develop with python -m pip install -e . or just pip install -e . and you should be happy.

ok, thanks (sorry for the dup).


Homepage points to GitLab issue tracker

https://todoman.readthedocs.io/en/latest/ points to https://gitlab.com/hobarrera/todoman/issues

Thanks!


Updated newsfeed.js - improved fullscreen iframe

Had much better performance using 100vw (viewport width) than 100% (why - idk), but with 100% about 5% of my screen (1080x1920) to the right of the scroll bar was left black/blank, with 100vw I legitimately takes up the whole screen/viewport width.
With the 10000 height the articles would always load about half way scrolled down. So I reduced the height and style.height to 3000. At my resolution at least, which I assume is fairly common, I had much better results. Unfortunately 3000 also isn't perfect - this still requires some tweaking. The article loads perfectly at the top of the iframe at 2500, but 2500 is much too small for most articles. 3000 seemed a good compromise, I could scroll far enoguh to read most articles on Reuters, and also load far enoguh up to read the beginning of the article.
And finally I added a "scroll back up" button notification. This seems to work flawlessly.

Please send your pull requests the develop branch.
Don't forget to add the change to CHANGELOG.md.

Note: Sometimes the development moves very fast. It is highly
recommended that you update your branch of develop before creating a
pull request to send us your changes. This makes everyone's lives
easier (including yours) and helps us out on the development team.
Thanks!

Does the pull request solve a related issue?
If so, can you reference the issue?
What does the pull request accomplish? Use a list if needed.
If it includes major visual changes please add screenshots.

Warnings

:warning:

Please include an updated CHANGELOG.md file.This way we can keep track of all the contributions.

  Generated by :no_entry_sign: dangerJS


"The requested transaction is being used concurrently by another request" exception

As suggested I'm using one singleton client.
When running multiple concurrent requests with transaction I'm getting The requested transaction is being used concurrently by another request exception.
Basically I have a method which called concurrently.
  public async Task ConcurrentTransactionRequest()
  {
       using (var scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
       {
           var query = client.Cypher.....;

            await query.ExecuteWithoutResultsAsync();

            scope.Complete();
       }
   }

If I create new client for this particular scope and use it everything works as expected.
What you suggest in this situation?

Have you tried the overload for new TransactionScope() which takes TransactionScopeOption and TransactionScopeAsyncFlowOption??

Yes, I did. Below is my test which runs two concurrent transactions.
	[Test]
	public async Task ConcurrentRequestsWithinTransaction()
	{
		const int numberOfUsers = 10;

		var users = new List<User>();

		for (var i = 0; i < numberOfUsers; i++)
		{
			users.Add(new User
			{
				UserId = i
			});
		}

		var createQuery = client.Cypher
			.ForEach("(user IN {users} | CREATE (u:User) SET u = user)")
			.WithParam("users", users);

		await createQuery.ExecuteWithoutResultsAsync();

		var tasks = new List<Task<int>>();

		for (var i = 0; i < 2; i++)
		{
			tasks.Add(CreateRequest());
		}

		await Task.WhenAll(tasks);

		var results = tasks.Select(x => x.Result).ToArray();

		Assert.That(results.All(x => x == numberOfUsers));
	}

	private async Task<int> CreateRequest()
	{
		int result;

		using (var scope = new TransactionScope(TransactionScopeOption.RequiresNew, TransactionScopeAsyncFlowOption.Enabled))
		{
			var query = client.Cypher
				.Match("(u:User)")
				.With("COUNT(*) AS C")
				.Return<int>("C");

			result = (await query.ResultsAsync).FirstOrDefault();

			scope.Complete();
		}

		return result;
	}

@cskardon any suggestions?

Not as yet, I've not had time to add in appropriate unit tests to establish what the cause would be - sorry :(

A few notes.
Sometimes instead of The requested transaction is being used concurrently by another request exception, Unrecognized transaction id. Transaction may have timed out and been rolled back. thrown.
Plus if a number of concurrent requests is big enough the whole test will hang and eventually will abort with timeout.
Hope this helps. This issue looks pretty big to me as transaction behaviour is hard to predict.

Hi, I've been trying to replicate this in the codebase, and I can't get either of those errors to occur, I'm having to whip out the ol'  console apps to see if there's anything off. A couple of questions;

What version of Neo4j are you using
How big is User? I presume pretty average size...?

Riiight, got it happening - and the error is from the DB, not the client, ok, that gives me more to work on....

Hi,
Neo4j is 2.3.1 and client version is <IP_ADDRESS>. I'm running my test against clean database creating 10 nodes each contain id attribute only.
Cheers for looking into that.

Could you use the ITransactionGraphClient instead?
ITransactionalGraphClient txClient = (ITransactionalGraphClient) Client;

using (var tx = txClient.BeginTransaction(Neo4jClient.Transactions.TransactionScopeOption.RequiresNew))
{
    var query = Client.Cypher
        .Match("(u:User)")
        .With("COUNT(*) AS C")
        .Return<int>("C");

    result = (await query.ResultsAsync).FirstOrDefault();
    tx.Commit();
}

Does that work for you?

Transactional client works for small number of requests. But for 50 concurrent requests it fails with
System.ApplicationException : Received an unexpected HTTP status when executing the request.

The response status was: 404 Not Found

The response from Neo4j (which might include useful detail!) was: {"results":[],"errors":[{"code":"Neo.ClientError.Transaction.UnknownId","message":"Unrecognized transaction id. Transaction may have timed out and been rolled back."}]}

   at Neo4jClient.Execution.ResponseBuilder.Execute(String commandDescription, TaskFactory taskFactory) in D:\temp\d298ce3\Neo4jClient\Execution\ResponseBuilder.cs:line 218
   at Neo4jClient.Transactions.Neo4jTransaction.Rollback() in D:\temp\d298ce3\Neo4jClient\Transactions\Neo4jTransaction.cs:line 125
   at Neo4jClient.Transactions.Neo4jTransaction.Dispose() in D:\temp\d298ce3\Neo4jClient\Transactions\Neo4jTransaction.cs:line 276
   at Neo4jClient.Transactions.TransactionScopeProxy.Dispose() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionScopeProxy.cs:line 56
   at Neo4jClient.Transactions.TransactionManager.EndTransaction() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionManager.cs:line 212
   at Neo4jClient.Transactions.TransactionScopeProxy.Dispose() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionScopeProxy.cs:line 48
   at Neo4jClient.Transactions.TransactionManager.EndTransaction() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionManager.cs:line 212
   at Neo4jClient.Transactions.TransactionScopeProxy.Dispose() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionScopeProxy.cs:line 48
   at Neo4jClient.Transactions.TransactionManager.EndTransaction() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionManager.cs:line 212
   at Neo4jClient.Transactions.TransactionScopeProxy.Dispose() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionScopeProxy.cs:line 48
   at Neo4jClient.Transactions.TransactionManager.EndTransaction() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionManager.cs:line 212
   at Neo4jClient.Transactions.TransactionScopeProxy.Dispose() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionScopeProxy.cs:line 48
   at Neo4jClient.Transactions.TransactionManager.EndTransaction() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionManager.cs:line 212
   at Neo4jClient.Transactions.TransactionScopeProxy.Dispose() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionScopeProxy.cs:line 48
   at Neo4jClient.Transactions.TransactionManager.EndTransaction() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionManager.cs:line 212
   at Neo4jClient.Transactions.TransactionScopeProxy.Dispose() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionScopeProxy.cs:line 48
   at Neo4jClient.Transactions.TransactionManager.EndTransaction() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionManager.cs:line 212
   at Neo4jClient.Transactions.TransactionScopeProxy.Dispose() in D:\temp\d298ce3\Neo4jClient\Transactions\TransactionScopeProxy.cs:line 48

Or with transaction was closed or rolled back exception.

What number are you looking to work with?

I'm using Neo4j as part of public Web API. So it could be any number of concurrent requests.
Is this issue related to one single GraphClient?

TBH I'm not sure, possibly.... seems like the server isn't happy with the numbers.. I'm looking to see if it's a delayed response or actually missing data

@cskardon any news? I'm thinking about creating a client for each neo request. I know that you will make a request to obtain neo server configuration each time, but that way I get working transaction scenario.
It looks like there is problem with TransactionManager and TransactionPromotableSinglePhaseNotification when you have concurrent threads accessing single GraphClient instance.

Hi, no, at the moment no progress, it's odd as it's not consistent, I can get it to run 50 concurrent tx's no problems sometimes, then the next time it doesn't work.
The response from the server isn't consistent from my POV, and it's taking a while to look into I'm afraid, sorry for the lack of updates :(

Any updates or ETA when this issue might be resolved?

The problem using ITransactionalClient explicitly is distributed transactions. If you have an ambient transaction that wraps calls to SQL and Neo you would probably need to wrap explicitly Neo calls in ITransactionalClient.BeginTransaction(). Consider below.
using (var scope = TransactionScope())
{
    // SQL data modification

    using (var tx = ((ITransactionalClient)graphClient).BeginTransaction())
    {
        // Neo data modification

        tx.Commit();
    }

    scope.Complete();
}

At the moment I can create new GraphClient for every request. The only downside of this each GraphClient.Connect() will call neo to obtain configuration each time. Currently there is no way of caching configuration and initializing GraphClient with it. I mean RootApiResponse object. I don't like connect method semantics as it not an actual connect but rather initialize logic.
The whole situation with transactions is worrying as you cannot predict how neo calls might be wrapped in transaction scope. Currently I have to wrap all neo calls into TransactionScope.Supress at bottom level in order to avoid transaction related issues.
I hope you understand my concerns.

Hi - firstly - sorry - I only just realised my draft response hadn't been sent for your last comment, so apologies for looking like I was ignoring you.
So, for your first question - I don't have an ETA - TBH I haven't even got to the route of the problem, I'm not sure if it's the codebase, or the DB doing funny things. As I'm sure you can appreciate - it's a complicated debug process - which takes time :/
In terms of the semantics of Connect - personally - it does Connect to the database, I could see your argument that something like Initialize might be added so you could initialize a client from the RootApiResponse and there would be no objection to you doing a pull request around that and getting it in.
From what you're saying you certainly use the TransactionScope over and above what I do - and I don't pretend to be an expert in any way shape or form about it. My best suggestion would be that as you know the problem space in far more depth than I do - and that comes into pull request territory.
Which (and I fully accept) is a totally rubbish thing to say - I don't really like saying 'do a pull request' as it comes across as grumpy and shirty about the whole situation - in this instance I genuinely think you will have a better idea at what needs to be fixed than I will.
If TransactionScope wasn't there as an option - and the whole TX thing was done in a more bespoke (albeit) simpler way - how would you approach this? Would you need it to be usable in the TransactionScope?

Hi, don't get me wrong. I'm not complaining. I do realize transactions implementation complexities.
It would be good to have some notes on the Wiki around transactions, scenarios and possible issues.
In terms of getting neo server configuration and having one config and multiple clients per app. This  looks quite natural to me. This would definitely help with transaction scenario as having client per request eliminates issues with concurrent transactions and need to use transactional client. You can safely wrap requests with default TransactionScope.
In my fork I added NeoServerConfiguration class which encapsulates RootApiResponse. GraphClient.Connect() now has optional NeoServerConfiguration param which used to initialize client. If it's null default logic will execute. In order to get server configuration you need to call
GraphClient.Connect(NeoServerConfiguration configuration = null);

NeoServerConfiguration.GetConfiguration(Uri rootUri, string username = null, string password = null);

I can create a pull request for you to review if you up to.
I appreciate all response provided.
Thanks

So I guess as a basic Tx user myself it's not something that has occurred, so in terms of Wiki updates - I have to rely on people like yourself to find the cases so it can be updated :)
So code-wise are you doing this?
var config = NeoServerConfiguration.GetConfiguration(new Uri("blah", user, pass);
var gc = new GraphClient(new Uri("blah"), user, pass);
gc.Connect(config);

Once I finish load balance testing with different transaction scenarios in our staging environment I will add notes on transaction usage to the Wiki. There might be other issue I suspect.
In terms of server config, this is exactly how it works.

OK, great, I'd be interested in looking at a pull request if it is working as you hope.

Here is pull request https://github.com/Readify/Neo4jClient/pull/162. Have a look. Cheers.

Cheers buddy, I'll have a look as soon as I can!

Hey guys, I'm having a similar problem. I have a scenario where the client can fire two simultaneous requests to perform an update on the node. Without transaction mechanisms generate an inconsistency in the bank.
When I put the TransacionScope, I have the following error: Neo.ClientError.Transaction.ConcurrentRequest "," message ":" The requested transaction is being used concurrently by another request. "
I will test the pull sent by @drgraduss. You have an example of how to use this solution?
This problem seems to be simple to occur.
Thanks

If you can write a test that can generate this, I would be super pleased to
see what I can do.
So a few questions:
Are you using the TransactionScope in the same way as @drgraduss? i.e.
with a SQL transaction as well?
Have you tried the ITransactionGraphClient?
What exactly is the update? (setting things? adding relationships?)
The message you're getting seems to be different, how is your client doing
the updates? Do you have some simple sample code / project I could look at?
My current thoughts are around there being 2 main areas:

It's the client that is messing up (this is the BAD scenario)
The DB is going squiff (not unheard of - but I would probably say
unlikely)

I appreciate this is painful, but we'll hopefully get out the other side -
please bear with me!
On 19 February 2016 at 19:41, jefersonmanjos<EMAIL_ADDRESS>wrote:

Hey guys, I'm having a similar problem. I have a scenario where the client
can fire two simultaneous requests to perform an update on the node.
Without transaction mechanisms generate an inconsistency in the bank.
When I put the TransacionScope, I have the following error:
Neo.ClientError.Transaction.ConcurrentRequest "," message ":" The requested
transaction is being used concurrently by another request. "
I will test the pull sent by @drgraduss https://github.com/drgraduss.
You have an example of how to use this solution?
This problem seems to be simple to occur.
Thanks
—
Reply to this email directly or view it on GitHub
https://github.com/Readify/Neo4jClient/issues/156#issuecomment-186372855
.

@cskardon @jefersonmanjos I updated pull request with usage example + the reason behind this change. Hope this helps. Let me know if there is any issues.

@drgraduss @jefersonmanjos - does this pull fix it for you?
I'm wanting to make sure before I pull it in - I don't want to make it worse!

Hey guys, sorry for the delay. At this time I believe solved my problem using ITransactionalGraphClient. I followed the example here: http://neo4j.com/blog/integration-testing-neo4j-c-sharp/
Thank you all. If another scenario untested appear, I comment here

@jefersonmanjos you will have issues with ITransactionalGraphClient when you run into certain number of concurrent connections as discussed above.
@cskardon my pull request deals with ability to cache server configuration and use client per request. it works for scenario with multiple concurrent transactions as each client has it's own transaction manager. works with both standard TransactionScope and ITransactionalGraphClient. I encourage you to have a look at pull request as this feature might be helpful.

Oh hey, no worries, I just wanted to make sure it was all good, I'll get a version done tomorrow!
-----Original Message-----
From: "drgraduss"<EMAIL_ADDRESS>Sent: 24/02/2016 21:38
To: "Readify/Neo4jClient"<EMAIL_ADDRESS>Cc: "Chris Skardon"<EMAIL_ADDRESS>Subject: Re: [Neo4jClient] "The requested transaction is being usedconcurrently by another request" exception (#156)
@jefersonmanjos you will have issues with ITransactionalGraphClient when you run into certain number of concurrent connections as discussed above.
@cskardon my pull request deals with ability to cache server configuration and use client per request. it works for scenario with multiple concurrent transactions as each client has it's own transaction manager. works with both standard TransactionScope and ITransactionalGraphClient. I encourage you to have a look at pull request as this feature might be helpful.
—
Reply to this email directly or view it on GitHub.

@cskardon I just want you to have a look and review pull. no pressure :) Let me know what you think.

I'm pretty sure the issue is:
        [ThreadStatic] private static Stack<TransactionScopeProxy> scopedTransactions;

Thread static doesn't play well with async. See my post here. I suggest you either float the current transaction around in the framework by using the CommittableTransaction directly, use logical call context (has its caveats as well), or upgrade to 4.6 and use AsyncLocal.

Hey @danielmarbach - by float - you mean pass it around, instead of storing in the scopedTransactions bit?

Yes. For example you can pass around commitable tx or you own tx wrappers

Am 20.04.2016 um 13:11 schrieb Chris Skardon<EMAIL_ADDRESS>Hey @danielmarbach - by float - you mean pass it around, instead of storing in the scopedTransactions bit?
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub

So @danielmarbach you are right. The  [ThreadStatic] cause issue in Neo4jClient when we use transaction with asyc method.
I change this line:
[ThreadStatic] private static Stack<TransactionScopeProxy> scopedTransactions;
for
static AsyncLocal<Stack<TransactionScopeProxy>> scopedTransactions;
And is working perfect. But, I have doubt if is the correct way to use AsyncLocal.

Where are your doubts comming from? Based on the littlw code I saw it is used how it should be used. The tricky part is to initialize the value properly since it doesn't provide a value factory like ThreadLocal

Am 07.09.2016 um 21:58 schrieb jefersonmanjos<EMAIL_ADDRESS>So @danielmarbach you are right. The [ThreadStatic] cause issue in Neo4jClient when we use transaction with asyc method.
I change this line:
[ThreadStatic] private static Stack scopedTransactions;
for
static AsyncLocal<Stack> scopedTransactions;
And is working perfect. But, I have doubt if is the correct way to use AsyncLocal.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

https://github.com/Readify/Neo4jClient/issues/156#issuecomment-180306546
I'm getting this behavior too. Been looking around quite a bit, testing various workarounds etc.
BUT using Bolt, with a small patch (one AsyncLocal, not destroyed per manager), this issue is gone.
https://github.com/simendsjo/Neo4jClient/commit/6d1867418e66b5f7c21ce760adf8fc64504e901d
Unfortunately, we cannot quite migrate to Bolt yet, so we would like Rest to be fixed too.
I notice that this issue is nearly three years old, so I guess other people have done various workarounds already. How has other people solved this issue?


cors: refactor loop to if

Commit Message: cors: refactor loop to if
Additional Description:
PR #36079 updated the policies of the CORS filter to be a vector of references that must exist.
This followup changes the original for-loop to an if statement that returns the first element, if exists.
Risk Level: low
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

/assign @wbpcode


Calculate the SpinBox value using the Expression class

This closes #31780.

Thanks!

After thinking some more, I think there might be issues with this change, doesn't it break the previous logic about prefix?
And I couldn't figure out if Expression gives access to the scene tree last I tried; if it does, this might be problematic as people could hack games through SpinBoxes :)

And I couldn't figure out if Expression gives access to the scene tree last I tried; if it does, this might be problematic as people could hack games through SpinBoxes :)

Regarding this concern, I did some tests and it seems like Expression does have same access to the scene tree, but only if base_instance is provided, and in this PR it's set to the NULL default, so it's fine.

After thinking some more, I think there might be issues with this change, doesn't it break the previous logic about prefix?

Indeed, I tried in the current master branch and the feature is not working properly if there is a prefix, you'd now be expected to delete the prefix to be able to have the string parsed properly.


refactor: import getMessaging from firebase/messaging/sw instead

In order to use FCM in a service worker, getMessaging should be imported from firebase/messaging/sw.
Originally reported on StackOverflow

thanks for catching this! FYI @egilmorez @kroikie


refactor: Change proposal submission notification message

Description
Changed the message in the notification bar: appended 'You will receive a confirmation email in due course.' to the original message, so users know that they will receive a confirmation email.

Motivation and Context
Confirmation emails can be delayed, so users often submit multiple proposals thinking their proposal was not submitted as they did not receive the email when they think they should. This change informs users they will 'in due course' receive an email, instead of leaving them to wonder if they are being sent one.

How Has This Been Tested
Ensured the edited string appears in the notification bar.

Fixes
Part of UserOfficeProject/user-office-project-issue-tracker#652

Changes

Depends on

Tests included/Docs Updated?

[ ] I have added tests to cover my changes.
[ ] All relevant doc has been updated

Can we change "Your proposal has been submitted successfully. You will receive a confirmation email in due course." -> "Proposal has been submitted successfully. You will receive a confirmation email."  as the email is sent immediately

Can we change "Your proposal has been submitted successfully. You will receive a confirmation email in due course." -> "Proposal has been submitted successfully. You will receive a confirmation email." as the email is sent immediately

@TickleThePanda - After talking to Simon, we thought it may be best to keep the "in due course" part in the notification as on our end emails are not always sent immediately - which was the cause of opening this issue in the first place. He advised I ask you for your opinion, so what do you think?


Add HasDisability base component

Short Description
Similar to the PauschbetragPages and FahrkostenPauschalePages this refactors the HasDisability component in such a way that we have one base component and then two different components for person A and B.
Feedback

Do you see any problem with that?
Did I miss something?

How to review this Pull Request
Link to Confluence (internal)

Closed because wrong branch used


Unify the notation of RubyGems

This PR is mainly replacing Rubygems with RubyGems.
Rubygems is a Ruby module name (e.g. Bundler::Source::Rubygems) , whereas RubyGems is the proper notation as below.
https://github.com/rubygems/rubygems/blob/master/README.md
In this PR, it changed the notation mainly used in console output and documentation  (and some source code comments) .

@bundlerbot r+

@segiddins Thanks always.

Thank you for these great contributions!


[chat] add unit tests

📌 Checklist before creating the PR

[x] I have created an issue for this PR for traceability
[x] The title follows the standard format: [doc/gemini/tensor/...]: A concise description
[ ] I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge
e.g. fixed #1234, closed #1234, resolved #1234

Close #4133, #4193, #4135.
📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

💥 Checklist before requesting a review

[x] I have linked my PR to an issue (instruction)
[x] My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
[x] I have performed a self-review of my code
[x] I have added thorough tests.
[ ] I have added docstrings for all the functions/methods I implemented

⭐ Do you enjoy contributing to Colossal-AI?

[ ] 🌝 Yes, I do.
[ ] 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

There are still several errors. DO NOT merge until they are resolved.

[ ] Gemini in tests/test_checkpoint
[ ] Zero2 in tests/test_checkpoint
[ ] Zero2 + Lora in tests/test_train's stage 1 & 2
[ ] Lora in tests/test_train's stage 3