CrashLoop when deploying to non-default namespace

What happened:
Splunk connect for k8s container goes into Crashloop when deploying to a namespace other than default.  For example, we need to deploy to a namespace called 'observability'.  Below is the error from the container log:
kubectl logs -f -n observability release-name-splunk-kubernetes-logging-jqmj2
2020-02-28 21:07:35 +0000 [info]: parsing config file is succeeded path="/fluentd/etc/fluent.conf"
2020-02-28 21:07:35 +0000 [info]: gem 'fluentd' version '1.9.1'
2020-02-28 21:07:35 +0000 [info]: gem 'fluent-plugin-concat' version '2.4.0'
2020-02-28 21:07:35 +0000 [info]: gem 'fluent-plugin-jq' version '0.5.1'
2020-02-28 21:07:35 +0000 [info]: gem 'fluent-plugin-kubernetes_metadata_filter' version '2.4.2'
2020-02-28 21:07:35 +0000 [info]: gem 'fluent-plugin-prometheus' version '1.7.0'
2020-02-28 21:07:35 +0000 [info]: gem 'fluent-plugin-splunk-hec' version '1.2.1'
2020-02-28 21:07:35 +0000 [info]: gem 'fluent-plugin-systemd' version '1.0.2'
2020-02-28 21:07:35 +0000 [warn]: thread exited by unexpected error plugin=Fluent::Plugin::KubernetesMetadataFilter title=:pod_watch_thread error_class=Fluent::ConfigError error="start_pod_watch: Exception encountered setting up pod watch from Kubernetes API v1 endpoint https://<IP_ADDRESS>:443/api: pods is forbidden: User "system:serviceaccount:observability:release-name-splunk-kubernetes-logging" cannot list resource "pods" in API group "" at the cluster scope ({"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User \"system:serviceaccount:observability:release-name-splunk-kubernetes-logging\" cannot list resource \"pods\" in API group \"\" at the cluster scope","reason":"Forbidden","details":{"kind":"pods"},"code":403}\n)"
2020-02-28 21:07:35 +0000 [warn]: thread exited by unexpected error plugin=Fluent::Plugin::KubernetesMetadataFilter title=:namespace_watch_thread error_class=Fluent::ConfigError error="start_namespace_watch: Exception encountered setting up namespace watch from Kubernetes API v1 endpoint https://<IP_ADDRESS>:443/api: namespaces is forbidden: User "system:serviceaccount:observability:release-name-splunk-kubernetes-logging" cannot list resource "namespaces" in API group "" at the cluster scope ({"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"namespaces is forbidden: User \"system:serviceaccount:observability:release-name-splunk-kubernetes-logging\" cannot list resource \"namespaces\" in API group \"\" at the cluster scope","reason":"Forbidden","details":{"kind":"namespaces"},"code":403}\n)"
#<Thread:0x000055e388a75b68@namespace_watch_thread@/usr/share/gems/gems/fluentd-1.9.1/lib/fluent/plugin_helper/thread.rb:70 run> terminated with exception (report_on_exception is true):
/usr/share/gems/gems/fluent-plugin-kubernetes_metadata_filter-2.4.2/lib/fluent/plugin/kubernetes_metadata_watch_namespaces.rb:79:in rescue in start_namespace_watch': start_namespace_watch: Exception encountered setting up namespace watch from Kubernetes API v1 endpoint https://<IP_ADDRESS>:443/api: namespaces is forbidden: User "system:serviceaccount:observability:release-name-splunk-kubernetes-logging" cannot list resource "namespaces" in API group "" at the cluster scope ({"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"namespaces is forbidden: User \"system:serviceaccount:observability:release-name-splunk-kubernetes-logging\" cannot list resource \"namespaces\" in API group \"\" at the cluster scope","reason":"Forbidden","details":{"kind":"namespaces"},"code":403} (Fluent::ConfigError) )                 from /usr/share/gems/gems/fluent-plugin-kubernetes_metadata_filter-2.4.2/lib/fluent/plugin/kubernetes_metadata_watch_namespaces.rb:70:in start_namespace_watch'
from /usr/share/gems/gems/fluent-plugin-kubernetes_metadata_filter-2.4.2/lib/fluent/plugin/kubernetes_metadata_watch_namespaces.rb:30:in set_up_namespace_thread'                 from /usr/share/gems/gems/fluent-plugin-kubernetes_metadata_filter-2.4.2/lib/fluent/plugin/filter_kubernetes_metadata.rb:283:in block in configure'
from /usr/share/gems/gems/fluentd-1.9.1/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
2020-02-28 21:07:35 +0000 [error]: config error file="/fluentd/etc/fluent.conf" error_class=Fluent::ConfigError error="start_namespace_watch: Exception encountered setting up namespace watch from Kubernetes API v1 endpoint https://<IP_ADDRESS>:443/api: namespaces is forbidden: User "system:serviceaccount:observability:release-name-splunk-kubernetes-logging" cannot list resource "namespaces" in API group "" at the cluster scope ({"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"namespaces is forbidden: User \"system:serviceaccount:observability:release-name-splunk-kubernetes-logging\" cannot list resource \"namespaces\" in API group \"\" at the cluster scope","reason":"Forbidden","details":{"kind":"namespaces"},"code":403}\n)"
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Generate helm manifest with modified values.yaml then use kubectl apply --recursive command to deploy the charts
Anything else we need to know?:
Environment: Development

Kubernetes version (use kubectl version):
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.2", GitCommit:"59603c6e503c87169aea6106f57b9f242f64df89", GitTreeState:"clean", BuildDate:"2020-01-23T14:21:36Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.9", GitCommit:"500f5aba80d71253cc01ac6a8622b8377f4a7ef9", GitTreeState:"clean", BuildDate:"2019-11-13T11:13:04Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}
Ruby version (use ruby --version):
ruby 2.3.7p456 (2018-03-28 revision 63024) [universal.x86_64-darwin18]
OS (e.g: cat /etc/os-release):
macbook mojave running minikube
Splunk version:
8.0.1
Others:

logging pod now needs service account created with permission to read from kubernetes API server. please edit your values.yaml file to create service account. or please grant right permission to system:serviceaccount:observability:release-name-splunk-kubernetes-logging this service account

logging pod now needs service account created with permission to read from kubernetes API server. please edit your values.yaml file to create service account. or please grant right permission to system:serviceaccount:observability:release-name-splunk-kubernetes-logging this service account

Below is the output of the two service account yamls I see in my namespace.  What is not correct with the rolebinding?
kubectl get sa -n monitoring
NAME                                     SECRETS   AGE
default                                  1         3d17h
splunk-1-4-0-splunk-kubernetes-logging   1         3d17h
kubectl get sa splunk-1-4-0-splunk-kubernetes-logging -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"ServiceAccount","metadata":{"annotations":{},"labels":{"app":"splunk-kubernetes-logging","chart":"splunk-kubernetes-logging-1.4.0","heritage":"Tiller","release":"splunk-1-4-0"},"name":"splunk-1-4-0-splunk-kubernetes-logging","namespace":"monitoring"}}
creationTimestamp: "2020-02-28T22:14:00Z"
labels:
app: splunk-kubernetes-logging
chart: splunk-kubernetes-logging-1.4.0
heritage: Tiller
release: splunk-1-4-0
name: splunk-1-4-0-splunk-kubernetes-logging
namespace: monitoring
resourceVersion: "1320"
selfLink: /api/v1/namespaces/monitoring/serviceaccounts/splunk-1-4-0-splunk-kubernetes-logging
uid: 9f5cf8b9-5a77-11ea-afea-6675893171b6
secrets:

name: splunk-1-4-0-splunk-kubernetes-logging-token-lfq8l

kubectl get sa -n monitoring default -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
creationTimestamp: "2020-02-28T22:06:23Z"
name: default
namespace: monitoring
resourceVersion: "561"
selfLink: /api/v1/namespaces/monitoring/serviceaccounts/default
uid: 8e7cd458-5a76-11ea-afea-6675893171b6
secrets:

name: default-token-m7dxj

oh i am sorry, let me be correct myself. not only the service account, we need cluster role and that role to be bound to this service account. please refer here: https://github.com/splunk/splunk-connect-for-kubernetes/tree/develop/manifests/splunk-kubernetes-logging

oh i am sorry, let me be correct myself. not only the service account, we need cluster role and that role to be bound to this service account. please refer here: https://github.com/splunk/splunk-connect-for-kubernetes/tree/develop/manifests/splunk-kubernetes-logging

I am deploying version 1.4.0 as is by downloading the 1.4.0 tgz file.  Do I need to add something extra in my values.yaml file to take care of the binding?  I see the clusterRole.yaml, ClusterRoleBinding.yaml and ServiceAccount.yaml files in the tgz, which I thought would take care of all this when I apply the chart.

by default, it should be taken care of automatically. Do you see cluster role created in your cluster?

by default, it should be taken care of automatically. Do you see cluster role created in your cluster?

Below is the output of get clusterrole for my custom namespace (monitoring) where the splunk logging container is deployed.
kubectl get clusterrole -n monitoring
NAME                                                                   AGE
admin                                                                  3d20h
cluster-admin                                                          3d20h
edit                                                                   3d20h
kubernetes-dashboard                                                   3d20h
splunk-1-4-0-splunk-kubernetes-logging                                 3d20h
system:aggregate-to-admin                                              3d20h
system:aggregate-to-edit                                               3d20h
system:aggregate-to-view                                               3d20h
system:auth-delegator                                                  3d20h
system:aws-cloud-provider                                              3d20h
system:basic-user                                                      3d20h
system:certificates.k8s.io:certificatesigningrequests:nodeclient       3d20h
system:certificates.k8s.io:certificatesigningrequests:selfnodeclient   3d20h
system:controller:attachdetach-controller                              3d20h
system:controller:certificate-controller                               3d20h
system:controller:clusterrole-aggregation-controller                   3d20h
system:controller:cronjob-controller                                   3d20h
system:controller:daemon-set-controller                                3d20h
system:controller:deployment-controller                                3d20h
system:controller:disruption-controller                                3d20h
system:controller:endpoint-controller                                  3d20h
system:controller:expand-controller                                    3d20h
system:controller:generic-garbage-collector                            3d20h
system:controller:horizontal-pod-autoscaler                            3d20h
system:controller:job-controller                                       3d20h
system:controller:namespace-controller                                 3d20h
system:controller:node-controller                                      3d20h
system:controller:persistent-volume-binder                             3d20h
system:controller:pod-garbage-collector                                3d20h
system:controller:pv-protection-controller                             3d20h
system:controller:pvc-protection-controller                            3d20h
system:controller:replicaset-controller                                3d20h
system:controller:replication-controller                               3d20h
system:controller:resourcequota-controller                             3d20h
system:controller:route-controller                                     3d20h
system:controller:service-account-controller                           3d20h
system:controller:service-controller                                   3d20h
system:controller:statefulset-controller                               3d20h
system:controller:ttl-controller                                       3d20h
system:coredns                                                         3d20h
system:csi-external-attacher                                           3d20h
system:csi-external-provisioner                                        3d20h
system:discovery                                                       3d20h
system:heapster                                                        3d20h
system:kube-aggregator                                                 3d20h
system:kube-controller-manager                                         3d20h
system:kube-dns                                                        3d20h
system:kube-scheduler                                                  3d20h
system:kubelet-api-admin                                               3d20h
system:node                                                            3d20h
system:node-bootstrapper                                               3d20h
system:node-problem-detector                                           3d20h
system:node-proxier                                                    3d20h
system:persistent-volume-provisioner                                   3d20h
system:public-info-viewer                                              3d20h
system:volume-scheduler                                                3d20h
view                                                                   3d20h

how about cluster role binding?

how about cluster role binding?

kubectl get clusterrolebinding -n monitoring
NAME                                                   AGE
cluster-admin                                          3d21h
kubeadm:kubelet-bootstrap                              3d21h
kubeadm:node-autoapprove-bootstrap                     3d21h
kubeadm:node-autoapprove-certificate-rotation          3d21h
kubeadm:node-proxier                                   3d21h
kubernetes-dashboard                                   3d21h
minikube-rbac                                          3d21h
splunk-1-4-0-splunk-kubernetes-logging                 3d21h
storage-provisioner                                    3d21h
system:aws-cloud-provider                              3d21h
system:basic-user                                      3d21h
system:controller:attachdetach-controller              3d21h
system:controller:certificate-controller               3d21h
system:controller:clusterrole-aggregation-controller   3d21h
system:controller:cronjob-controller                   3d21h
system:controller:daemon-set-controller                3d21h
system:controller:deployment-controller                3d21h
system:controller:disruption-controller                3d21h
system:controller:endpoint-controller                  3d21h
system:controller:expand-controller                    3d21h
system:controller:generic-garbage-collector            3d21h
system:controller:horizontal-pod-autoscaler            3d21h
system:controller:job-controller                       3d21h
system:controller:namespace-controller                 3d21h
system:controller:node-controller                      3d21h
system:controller:persistent-volume-binder             3d21h
system:controller:pod-garbage-collector                3d21h
system:controller:pv-protection-controller             3d21h
system:controller:pvc-protection-controller            3d21h
system:controller:replicaset-controller                3d21h
system:controller:replication-controller               3d21h
system:controller:resourcequota-controller             3d21h
system:controller:route-controller                     3d21h
system:controller:service-account-controller           3d21h
system:controller:service-controller                   3d21h
system:controller:statefulset-controller               3d21h
system:controller:ttl-controller                       3d21h
system:coredns                                         3d21h
system:discovery                                       3d21h
system:kube-controller-manager                         3d21h
system:kube-dns                                        3d21h
system:kube-scheduler                                  3d21h
system:node                                            3d21h
system:node-proxier                                    3d21h
system:public-info-viewer                              3d21h
system:volume-scheduler                                3d21h

I am noticing one thing.
serviceAccount used in the logging daemonset should match the one that is created by this helm chart.
splunk-1-4-0-splunk-kubernetes-logging is different from
release-name-splunk-kubernetes-logging

I am noticing one thing.
serviceAccount used in the logging daemonset should match the one that is created by this helm chart.
splunk-1-4-0-splunk-kubernetes-logging is different from
release-name-splunk-kubernetes-logging

When I deployed the chart I passed in my own release name like below:
helm template
-n splunk-1-4-0
--values ./values/values.yaml
--output-dir ./manifests ./charts/splunk-connect-for-kubernetes/charts/splunk-kubernetes-logging

i do not know why your release name didn't get rendered. you should be able to fix it by editing daemonset to use service account name that is created

i do not know why your release name didn't get rendered. you should be able to fix it by editing daemonset to use service account name that is created

Even if I deploy it without passing in my own release name I get the same result.  The container goes into Crashloop with the same error in logs.  It only works if I deploy it to the default namespace.

which namespace did you deploy this to?
observability or monitoring?

please make sure all resources are deployed in one namespace and the service account name matches in the daemonset
kubectl edit daemonset -n NAMESPACE release-name-splunk-kubernetes-logging

which namespace did you deploy this to?
observability or monitoring?

The namespace observability and monitoring are interchangeable.  I deployed it to monitoring namespace first and then blew away my cluster and started fresh with observability namespace.  I deployed to a single namespace in both cases.

please make sure all resources are deployed in one namespace and the service account name matches in the daemonset
kubectl edit daemonset -n NAMESPACE release-name-splunk-kubernetes-logging

nodeSelector:
beta.kubernetes.io/os: linux
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: release-name-splunk-kubernetes-logging
serviceAccountName: release-name-splunk-kubernetes-logging
terminationGracePeriodSeconds: 30
tolerations:

so change serviceAccount: release-name-splunk-kubernetes-logging serviceAccountName: release-name-splunk-kubernetes-logging
to actual service account created and right role assigned.

so change serviceAccount: release-name-splunk-kubernetes-logging serviceAccountName: release-name-splunk-kubernetes-logging
to actual service account created and right role assigned.

I just deployed with all defaults to the observability namespace:
kubectl apply --recursive --filename ./manifests -n observability
clusterrole.rbac.authorization.k8s.io/release-name-splunk-kubernetes-logging created
clusterrolebinding.rbac.authorization.k8s.io/release-name-splunk-kubernetes-logging created
configmap/release-name-splunk-kubernetes-logging created
daemonset.apps/release-name-splunk-kubernetes-logging created
secret/splunk-kubernetes-logging created
serviceaccount/release-name-splunk-kubernetes-logging created
kubectl edit daemonset -n observability release-name-splunk-kubernetes-logging
Edit cancelled, no changes made.
(py3) NY013JJLVCG:tmp vinit_masaun$ kubectl get pods -n observability
NAME                                           READY   STATUS             RESTARTS   AGE
release-name-splunk-kubernetes-logging-szm9m   0/1     CrashLoopBackOff   4          2m22s

Edit cancelled, no changes made.
you didn't save any changes.
please use helm to deploy.
we don't support deploying with manifests. it is just reference

Edit cancelled, no changes made.
you didn't save any changes.
please use helm to deploy.
we don't support deploying with manifests. it is just reference

we cant use helm2 as it depends on tiller, which is not allowed by our security policy.  As far as I understand it, helm 3 is not supported to deploy this app.

Edit cancelled, no changes made.
you didn't save any changes.
please use helm to deploy.
we don't support deploying with manifests. it is just reference

we cant use helm2 as it depends on tiller, which is not allowed by our security policy. As far as I understand it, helm 3 is not supported to deploy this app.

you didn't save any changes.
I ran the helm template command before the kubectl apply command to generate the manifests with values.yaml

helm template --values  /tmp/values/values.yaml --output-dir /tmp/manifests /tmp/charts/splunk-connect-for-kubernetes/charts/splunk-kubernetes-logging

we don't officially support helm3 yet, but last time i used it, it works just as well.

@vinitmasaun were you able to resolve this issue? I am running into the same error

@vinitmasaun were you able to resolve this issue? I am running into the same error

yes, I used helm3 install method, which resolved my issue.  According to the app developer, helm template and manifest method is not supported with version 1.4.0.

@vinitmasaun What version did you use? 1.3?

@vinitmasaun What version did you use? 1.3?

1.4.0

@vinitmasaun  i have the same issue. can you copy paste the helm3 oneliner that you use ?

@vinitmasaun  nevermind i found it in the doc.


Still getting runtime error SYSTEM_NO_ROLL in background mode

This is a continuation of #6727. The fix previously implemented wasn't enough to fix the memory issues for good.
I suspect that MT_OBJECTS needs to be cleared as I mentioned in #6728, but I'll do some additional analysis first

False positive. The problem wasn't in abapGit's background mode, but in one of my own applications that use abapGit.


[FEATURE] Add Preview in Box Shadow

Description
Just like any other generators, it'd be great to add a preview in box shadow generator. It'll show you how the shadow will look as you'll play with the values (sliders)
Screenshots
No response

It's great having you contribute to this project by creating an issue Make sure you are assigned to this before you work on it and you read the Contributing.md file, Thank you! Welcome to the community :nerd_face:

i could work on this

@Coding-Hashira do you have an idea of what to do?

@Coding-Hashira do you have an idea of what to do?

Maybe we can place a preview box, with box shadow, same as result, but it'll get updated when you'll play with sliders.

@Coding-Hashira do you have an idea of what to do?

Maybe we can place a preview box, with box shadow, same as result, but it'll get updated when you'll play with sliders.

Maybe place a tiny div at the top right corner of the window.

@Coding-Hashira do you have an idea of what to do?

Maybe we can place a preview box, with box shadow, same as result, but it'll get updated when you'll play with sliders.

Maybe place a tiny div at the top right corner of the window.

Yep, though position totally depends on you. Also, if someone else wants to work on this issue, I'm completely okay with that!

@Dun-sin Can I work on this?

@IamGideonIdoko try to place the preview at the top left corner

:tada: This issue has been resolved in version 2.17.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[Dense-Urban-ODD] Improve cruise planner safe distance when cruising a front npc

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and no duplicate issues were found.
[X] I've agreed with the maintainers that I can plan this task.

Description
Part of:

#7485

Dense Urban ODD Characteristics

Low-Speed Driving: Typical speeds around 15 km/h.
Complex Traffic Elements: Includes dense traffic with various dynamic interactions.
Dynamic Agents: Frequent encounters with moving pedestrians and cyclists.
Bidirectional Movement: Vehicles operate in lanes supporting bidirectional flow.
Traffic Rule Non-compliance: Other road users may exhibit behaviors such as:

Driving in the wrong direction.
Cutting in unexpectedly from off-road areas.
Jaywalking pedestrians.
Improper / sudden overtaking by other road users.

In the context of Dense Urban ODD, currently obstacle_cruise_planner keeps a very long safe distance between ego and front npc which makes the driving scenario not natural. or potentially develop new features to improve the slowing down of ego vehicle to overtaking road user maintaining both passengers safety and comfortability.
Purpose
Based on description above, the purpose of this issue is to improve  obstacle_cruise_planner  safe distance calculations to have a more near human driving scenarios while cruising a front npc.
Cruising front npc "UC-NTR-005-0001"
The following image describes the scenario where ...

an npc is driving in in front of ego vehicle
the npc slows down from 14 km/h to 10 km/h
ego vehicle is expected to have a safe distance of 2-seconds (follow distance / separation distance) with front npc.

The purpose of of this improvement is to mitigate any unnecessary distance (very long distance between ego and npc) that makes the driving scenario not natural / human-like.
Npc slowing down from 14 km/h to 10 km/h
https://github.com/user-attachments/assets/781ebdf0-32dc-4d79-951d-8094d66faa07
For reproducing the issue using scenario simulation, you can use the these scenario and map files.
Possible approaches
The obstacle_cruise_planner  already depends on a function named calcRSSDistance() to get the target_dist_to_obstacle
This function basically depends on this equation :
d_safe =  v_ego * t_reaction + (v_ego^2) / (2 * a_max) - (v_obstacle^2) / (2 * a_min) + d_margin

This yields to a very long distance in the afortmentioned scenario between ego and front npc.
A possible approach is to provide another function with needed parameters to implemented a time-based safe distance (for example a 2-seconds or 3-seconds separation distance)
Definition of done

[ ] Implement a new function for time-based safe distance for better cruising with front npc.
[ ] Add needed parameters.
[ ] Ensure the target scenario is passing.

@brkay54 Could you provide the reasons behind the safety distance of 2 seconds?
Could you post the link to the reference you used to define this?


[Bug]: get_or_create collection can alter metadata on an existing collection

What happened?
When we get_or_create a collection, we can supply a metadata argument. This makes sense on the create path, but is ambiguous on the get path.
This isn't strictly a bug, but we should think carefully about what we want the actual behavior we want is.
Versions
All
Relevant log output
No response

Dupe https://github.com/chroma-core/chroma/issues/478


Implement SQL connector with batch operation and procedure calls

Description:
$subject
Design doc can be found in,
https://docs.google.com/document/d/1J_5gzUDopvqMJRGz04tCre79lwKLj0DtKumE7S34nlw/edit#
The design is still under review.

As discussed in the design review, the procedure call implementation is moved to next quarter


:sparkles: Writer realm should not have rights on App attributes

Writer Realm should not have rights on App attributes. This role should be replaced by a Realm admin role.
Plus app manager should inherit reader
A new Role should be App Manager that can manage all apps on the realm

Je crée une issue à part pour " A new Role should be App Manager that can manage all apps on the realm"


No toolbar on either Chrome or firefox

I added the flask debug toolbar to my app. It looks working because it gave me the redirect warning  like below:
_Redirect (302)
Location: /app/bokeh
The Flask Debug Toolbar has intercepted a redirect to the above URL for debug viewing purposes. You can click the above link to continue with the redirect as normal. If you'd like to disable this feature, you can set the config variable DEBUG_TB_INTERCEPT_REDIRECTS to False._
But there is no FDT tool bar shown up after I got the response. I tried both Chrome and Firefox, same result.  Do I need to do any configuration on the browser side to display the toolbar?

It is related to the dynamic routing of docker container. Does anybody know how to configure the host IP address of docker container to bypass the problem? Thanks!

Finally, we found the root cause, our server is a hybrid of both cgi-bin and flask. the invocation url is not "/" but "/app/". The flask debug toolbar by default try to load the toolbar from "https://xyz.com/_debug_toolbar/static/js/toolbar.js",
instead of the correct url:
"https://xyz.com/app/_debug_toolbar/static/js/toolbar.js"
Is there any configuration parameter we can use to set this correct?

I don't know if you fixed this, but I run a flask app under a prefix and the proxy supplies the right prefix, and Flask respects it by default so the toolbar works. (There is a bug in the SQLAlchemy EXPLAIN/SELECT sub-panels that I have submitted a pull request for.) You could try setting SCRIPT_NAME in the Flask config if your proxy is not doing it right now?

Just merged @mattaw 's pull request.
@johnxpan what does Flask think it's home url/prefix is? Ie, is the redirect to /app prefix happening within Flask or externally? If external, then you need to fix your proxy to also redirect the javascript files. Otherwise if within Flask, how are you setting that prefix? Is it for the whole app, or just a sub-blueprint?


Update README.md

provide direct link to the wiki, which is otherwise not trivial to find.

This change is

Merged with Robert's changes to README


Fix PrimaryKey substitution in LaravelReadQuery

When applying a non-trivial orderBy object that contained PrimaryKey
field, getResourceSet was ignoring previously-retrieved copy of target
model's primary key and trying to dig it out anew.  This doesn't work
too well on query builders, as getKeyName doesn't exist.
The simple and obvious fix is to simply cut the orderBy handling over
to use the previously-retrieved copy of keyName, and let it worry about
whether we're hanging off a model, relation, etc.

Coverage remained the same at 100.0% when pulling 5d59d27eb0de32614ac91cb7a5592ebcda5133b5 on CyberiaResurrection:FixExpandAndOrder into e18320b05ec4572bd62a4b6db901a2a4bfcae314 on Algo-Web:master.

Coverage remained the same at 100.0% when pulling 279db29345674e3dc044e026fef521ff86376476 on CyberiaResurrection:FixExpandAndOrder into e18320b05ec4572bd62a4b6db901a2a4bfcae314 on Algo-Web:master.


White screen??!!

Hello, I have tried to install this xcode project on my phone.
It shows the splashscreen then goes into a white screen mode forever, it won't show anything after that.

@mayed505  yes, it is showing white screen because we didn't made any UI still. I will push some UI soon and then you can start working.

Ahh, Thank you.
Can't wait to reskin that baby up!

@mayed505  good to know that!

@imjog Would you help me with custom development? I own a hosting business and we can do some trades :)
I've some idea for you
I'll join your slack

gitter*

The project is done?

@mayed505 please do not spam here. I have to block you from commenting.

@imjog How am I spamming? I'm asking about my issue which is "Is the project done?" because I've seen you update stuff 3 hours ago..

Which project is completed in 3 hours?

No, I'm saying you have made updates to THIS project 3 hours ago. I thought you have finished it.
https://gyazo.com/597a32446c020230862735a7c6e44e4b

@mayed505 if you owning hosting business then you must also have learned to be professional, not cheap. Github issues is not place to talk about business.

Facepalm you're going off topic.
https://gyazo.com/f570c71304261e8b1eb3388a3a6a387b
You aren't answering my question, I'm asking about this project if you have fixed that white screen issue this has nothing to do with my last request.


MO1: choose sats or bitcoin mode

This is part of MO1, please refer to MO1: https://trello.com/c/1SUC3KXe
This issue covers the end-to-end implementation including screens, navigation based on UX flow, actions, reducers, Sagas as well as update to storage, wallet image and upgrade script.

In the currency section, the user can now also chose between sats mode and BTC mode. By default sats mode is on but when BTC mode is turned on, the Home Screen currency selection changes between the local currency and BTC (not sats). The conversion also changes across all the screens - transactions, accounts, send, receive, etc.

Screens names with xd links to follow

https://xd.adobe.com/view/6e89fcde-5787-4d1a-a1ba-c284e770fdad-3bda/
Settings – Currency & Language


New Release v6.0.0

⚠ A couple of significant breaking changes in this release ⚠
https://github.com/terraform-aws-modules/terraform-aws-eks/pull/468
https://github.com/terraform-aws-modules/terraform-aws-eks/pull/424
Resolves https://github.com/terraform-aws-modules/terraform-aws-eks/issues/512

I get this error when running plan against my EKS
Terraform v0.12.7
+ provider.aws v2.22.0
+ provider.kubernetes v1.8.1

Error: Unsupported attribute

  on .terraform/modules/eks/outputs.tf line 48, in output "cluster_oidc_issuer_url":
  48:   value       = aws_eks_cluster.this.identity.0.oidc.0.issuer

This object has no argument, nested block, or exported attribute named
"identity".

map_users also breaks but that seems expected given https://github.com/terraform-aws-modules/terraform-aws-eks/pull/424

map_users also breaks but that seems expected

Yep. You need to adjust a variable.

I get this error when running plan against my EKS

Thanks! I've added requirement for the latest AWS provider that includes the oidc output.
Can you retest? source = "git::https://github.com/max-rocket-internet/terraform-aws-eks.git//?ref=f4762acafcef994f7ad8d36ecfbaa8547ff61fae"

Updated my map_users to the new way and used that version. Plan output was

Added k8s.io/cluster/ to ASG
Added tag Environment =  to IAM worker role
Worker launch config updated
config_map_aws_auth updated so all values in quotes
module.eks.random_pet.workers

I get this error when running plan against my EKS
Terraform v0.12.7
+ provider.aws v2.22.0
+ provider.kubernetes v1.8.1

Error: Unsupported attribute

  on .terraform/modules/eks/outputs.tf line 48, in output "cluster_oidc_issuer_url":
  48:   value       = aws_eks_cluster.this.identity.0.oidc.0.issuer

This object has no argument, nested block, or exported attribute named
"identity".

Same here trying to upgrade existing cluster using latest module commit above from version 5.1.0

FWI the EKS identity attribute has been added in the AWS provider v2.28.0 https://github.com/terraform-providers/terraform-provider-aws/blob/master/CHANGELOG.md#2280-september-12-2019

Thanks, just stumbled on that.
On Tue, Sep 17, 2019 at 2:06 PM Thierno IB. BARRY<EMAIL_ADDRESS>wrote:

FWI the EKS identity attribute has been added in the AWS provider v2.28.0
https://github.com/terraform-providers/terraform-provider-aws/blob/master/CHANGELOG.md#2280-september-12-2019
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/terraform-aws-modules/terraform-aws-eks/pull/514?email_source=notifications&email_token=ABHPPPSQXTZOKRLFG3BVL7TQKETEZA5CNFSM4IXP4ZBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD65SPSY#issuecomment-532359115,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABHPPPW2AQKH3J7ZJFFKY43QKETEZANCNFSM4IXP4ZBA
.

Thanks for the effort @jjshanks! I think that all looks good and as expected.

I am trying to update a 1.12 cluster to 1.13 using this module:

bumped the module from v5.0.0 to v6.0.0
bumped the EKS version from 1.12 to 1.13

I created a cluster with the old v5.0.0 version using Kubernetes 1.12 and I am trying to upgrade to the v6.0.0 version and to 1.13 by doing a tf apply -- that's where I get an error regarding the cluster_oidc_issuer_url:
Error: Invalid index

  on .terraform/modules/k8s.eks/outputs.tf line 48, in output "cluster_oidc_issuer_url":
  48:   value       = aws_eks_cluster.this.identity.0.oidc.0.issuer
    |----------------
    | aws_eks_cluster.this.identity is empty list of object

The given key does not identify an element in this collection value.

I seem to be using the latest version of the AWS provider:
tf version
Terraform v0.12.9
+ provider.archive v1.2.2
+ provider.aws v2.28.1
+ provider.kubernetes v1.9.0
+ provider.local v1.3.0
+ provider.null v2.1.2
+ provider.random v2.2.0
+ provider.template v2.1.2

I think that since this is an EKS 1.12 cluster there's no identity so things get weird. There is a note about this output not being available: Only available on Kubernetes version 1.13 and 1.14 clusters created or upgraded on or after September 3, 2019.
Any ideas? I guess I could upgrade to 1.13 first and then bump the module version.
Should I create an issue for this?

I can confirm the 2-step upgrade works:

using the module at v5.0.0 upgrade cluster from 1.12 to 1.13
upgrade module from v5.0.0 to v6.0.0

Hmmm strange. So not all clusters have this attribute yet.

I can confirm the 2-step upgrade works

OK makes sense, thanks for testing! Will be annoying for some users though...

@max-rocket-internet @Vlaaaaaaad I have an 1.13 cluster (created before that date, though), and I am facing the same issue. I am going to attempt the upgrade to 1.14 before bumping to 6.0.0 and report back.

The 1.12 cluster I tested on was created today for this specific test so looks like this will affect:

all 1.12 clusters
1.13 clusters created before September 3, 2019 -- I guess AWS may be doing a rolling update of those maybe?

The provider return an empty map identity (aws_eks_cluster.this.identity is  {}) when the EKS cluster doesn't have one https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_eks_cluster.go#L460. So aws_eks_cluster.this.identity.0.oidc.0.issuer won't exists.
We should test aws_eks_cluster.this.identity is not empty.
One of these (or all) this should works:
output "cluster_oidc_issuer_url" {
  description = "The URL on the EKS cluster OIDC Issuer"
  value       = concat(aws_eks_cluster.this.identity.*.oidc.0.issuer, [""])[0]
}

# Or

output "cluster_oidc_issuer_url" {
  description = "The URL on the EKS cluster OIDC Issuer"
  value       = join(aws_eks_cluster.this.identity.*.oidc.*.issuer, "")
}

# Or

output "cluster_oidc_issuer_url" {
  description = "The URL on the EKS cluster OIDC Issuer"
  value       = length(aws_eks_cluster.this.identity) > 0 ? aws_eks_cluster.this.identity.0.oidc.0.issuer : ""
}

I'll send a small PR for this.
@bmcstdio @Vlaaaaaaad Can you please test this ?

@barryib: sure. Gimme a branch name when it's done and I can test it.

For the middle proposal( which looked the best in my opinion) Terraform fails with:
Error: Nested splat expression not allowed

On .terraform/modules/eks/outputs.tf line 48: A splat expression (*) cannot be
used inside another attribute-only splat expression.

The last option which I put in Vlaaaaaaad/terraform-aws-eks@test-barryib-oidc-12 works on the test case I reported above! 🎉
@barryib looking forward to your PR!

Here is the PR. @Vlaaaaaaad @bmcstdio https://github.com/terraform-aws-modules/terraform-aws-eks/pull/516 can you have a look please ?

@anuragkapur This fix is included in https://github.com/terraform-aws-modules/terraform-aws-eks/releases/tag/v6.0.1

Thank you @barryib  . I am now upgrading from 5.1.0 to 6.0.1 .
I see now:
Plan: 4 to add, 3 to change, 3 to destroy.


Migrate tests/apps/data_model to pytest

What is the context of this PR?
Migrates all unittest tests to pytest in tests/apps/data_model.
How to review
Does it make sense; do tests pass?
Checklist

[ ] New static content marked up for translation
[ ] Newly defined schema content included in eq-translations repo

 Migrate data_model module to pytest framework (3 SP)


refactor: cache always true

What kind of change does this PR introduce?

Did you add tests for your changes?

Does this PR introduce a breaking change?

What needs to be documented once your changes are merged?

Thanks


IIF Manifests

This allows you to add /manifest to the end of an item record page to
get its IIIF Presentation Manifest. Requires that Loris is running in
order to get the height/width of the JP2 derivatives.
Closes #21

A note - this may be slow. May want to do some work to make it not depend on Fedora at all

+1--there's nothing in there manifest that requires Fedora, right? I wonder if it's worth even going a step further and caching the manifest??

@jpstroop I'm not sure it's worth figuring out cache invalidation given the speed of solr

I was thinking more about having to hit t he image server once per page whenever the manifest is requested. We could index the width and height and not use IIIF::Presentation::ImageResource#create_image_api_image_resource, I suppose.

Looks fine to me--I can make a separate issue re :arrow_up:. Are you ready to have it merged?

@jpstroop Wanted to get it running from solr, since Fedora is basically a non-starter - it would have failed us the moment we put in a real book.

@jpstroop @flyingzumwalt Ready for review

No complaints here.


Update git protocol and PR protocol; #trivial

Describe rules for public and private SDK.

@Mourad-Aly
I have updated what you suggest in f2b76e36b0648a75f70ad5c06d94d5c196bafd17.


execute didn't work when SQL ends with a comment

Calling execute with SQL like the following didn't work:
INSERT INTO t (x, y, z) VALUES (7, 7, 7);
INSERT INTO t (x, y, z) VALUES (8, 8, 8);
-- this is a comment at the end

Executing that SQL results in a Extralite::Error: not an error error because prepare_multi_stmt_impl can set stmt to NULL when SQL ends with a comment. The documentation for sqlite3_prepare_v2 states:

If the input text contains no SQL (if the input is an empty string or a comment) then *ppStmt is set to NULL.

Great fix, thanks @gschlager !


Switch np.math.ceil to int(np.ceil) as the former is deprecated as of NumPy 1.25

This is just a minor deprecation fix PR that I saw while running the test suite.

@samuelgarcia, I know you worked on this rawio so just FYI-ing you here for this deprecation fix PR.


Can anyone help to get the string for Dedao Max( Note Air)?

I am trying to flash the Dedao firmware to my Note air through the update menu, but it didn't allow. If I get the zip
file, maybe there is other way I can flash it in.
I get the file from here https://pan.baidu.com/s/18yacVkce64ecwOi6YA_LXw   code: f2UA

Or is it possible to modify the upx file to fake a note air version, maybe that also can pass the model verification.

I am trying to flash the Dedao firmware to my Note air through the update menu, but it didn't allow. If I get the zip
file, maybe there is other way I can flash it in.
I get the file from here https://pan.baidu.com/s/18yacVkce64ecwOi6YA_LXw code: f2UA

As far as I know, to get the encryption key (called "strings" in this project), firstly a device with the firmware installed already is needed because there is not other way to get OnyxOtaService.apk for this firmware.

This issue is very old. If you are still interested, please reissue it.


Add smtp output binding

Description
This PR contains an SMTP output binding for sending email directly via SMTP.
See https://github.com/dapr/components-contrib/pull/646
Issue reference
https://github.com/dapr/components-contrib/issues/645
Checklist
Please make sure you've  completed the relevant tasks for this PR, out of the following list:

[x] Code compiles correctly
[x] Created/updated tests
[x] Unit tests passing
[x] End-to-end tests passing
[x] Created issue in the https://github.com/dapr/docs/ repo: https://github.com/dapr/docs/issues/1145

@EdwinVW please run:
go get -u github.com/dapr/components-contrib@master
go mod tidy
And push the changes.

@EdwinVW please run:
go get -u github.com/dapr/components-contrib@master
go mod tidy
And push the changes.

@yaron2 Done.
I did receive an error that the smtp package could not be found. But I recon this makes sense because the smtp binding package is not yet merged and published.

@EdwinVW please run:
go get -u github.com/dapr/components-contrib@master
go mod tidy
And push the changes.

@yaron2 Done.
I did receive an error that the smtp package could not be found. But I recon this makes sense because the smtp binding package is not yet merged and published.

Sorry, I jumped the gun. didn't notice we didn't merge it in contrib yet. It's merged now, please re-do the steps and push the changes.

Congrats and thanks @EdwinVW


Add Swift support

Add FBSnapshotTestVerifyView:indentifier: and FBSnapshotTestVerifyLayer:identifier as macros don't work that well in Swift one has to workaround and use methods instead.
Current limitation is that the failure description is not displayed inline when a test fails. One has to look into the console.
But maybe @modocache has an answer to that?

I have a few things to add here:

FB_REFERENCE_IMAGE_DIR seems not to work so my thinking is to make a configurable variable
I would like to have both FBSnapshotVerifyView(view, withIdentifier: "something") and FBSnapshotVerifyLayer(view)

Between Swift compatibility and fuzzy matching in #86, looks like we're about to have a very exciting 2.0 release!

All tasks of this PR are done. Waiting for #88 for the Travis build to get green.

So, CocoaPods succeeds. I have to figure out why the framework target is not working, but that I will do tomorrow.

I'm not seeing a full expansion of $(SOURCE_ROOT) when passing this param as an environment variable. I get "./" where I would expect an absolute path to the source.

The other thing I've noticed so far is that when I run tests from a test target like MyLibraryApplicationTests, the environment variable doesn't get MyLibraryApplicationTests for $(PROJECT_NAME). Instead, it get the name of the buildable reference in the LaunchAction of the scheme. I'm still poking around and tweaking things, so I'll keep on reporting what I find as I go.

So $(SOURCE_ROOT) is the absolute path to the project most of the time, but I'm finding some cases where it's just "." (which resolves to the data folder on the simulator). $(PROJECT_DIR) expands consistently so far, so I think that solves it.
$(PROJECT_NAME) expands to just the launch target project name, where previously it was the test target name. That might trip up a few people, but it's workable.
So far these are the gotchas I've found that people will hit when migrating to 2.0 and switching from macros to env vars. I have a few more tests to run, will update shortly.

thanks @pietbrauer!

Will hold off on tagging to give #86 a few more days to squeeze it into a 2.0 release.

Thanks @dstnbrkr!


[Bug] CSS Classes Only Applied To 1 Pane

The cssclasses seem to only be applied to 1 pane currently, it'd be nice if it applied to any panes that apply to the set cssclass if possible 🙏🏽 I always write with preview available and it'd be great if it applied to both.
Example of issue:

Hmm this is one of the scenarios I made sure to try and support. Maybe I broke something in an update. Will investigate.

This specifically seems to be an issue with linked panes, which I definitely did not test (or know about at all, actually 😅. Should be able to support this, just need to figure out the right event to hook into.

@SlRvb fixed in #14, available in v1.3.0. Let me know if you see any more issues.


Migrate all permission requests in Call screen to an activity result api

Jira issue:
https://glia.atlassian.net/browse/MOB-3211
What was solved?
Release notes:

[x] Feature
[ ] Ignore
[ ] Release notes (Is it clear from the description here?)
[ ] Migration guide (If changes are needed for integrator already using the SDK - what needs to be communicated? Add underneath please)

Additional info:

[ ] Tests updated? Added? Unit, acceptance, snapshots?
[ ] Logging for future troubleshooting of client issues added?

Screenshots:

Maybe I'm missing something, I see that you removed onActivityResult and onRequestPermissionsResult but don't see activity result API usage. 🤔

It turned out that there was no permission request on the Call screen. Everything was moved to OperatorRequestController when I was refactoring media-upgrade and screen-sharing related code. The other permissions are handled inside PermissionManager that are using the activity result api under the hood.


Revert to non-master image for Rook-Ceph in latest CSV

Signed-off-by: Jose A. Rivera<EMAIL_ADDRESS>

/hold

/lgtm
we need a stable rook image. ci isn't passing because noobaa is failing regardless of this change.

/hold cancel


Do KTweak changes remain after uninstalling it?

I installed it and I had issues at shut down -had to keep shutdown button pressed to turn off.
I uninstalled it and I noticed my SSD driver led keeps working -even when I run nothing.
Do I need to manually uninstall everything, and how?

Must be something else

I think you're right.
I reinstalled KTweak, enabled it via systemctl, and next shutdown was fine -I shut down again after that and again it closed fine.

I found the culprit in the journal.
I had "/usr/bin/xset s off -dpms &" in my OpenBox autostart, and journal mentioned upower service found DPMS disabled so, shut down didn't finish.
Using "/usr/bin/xset s off  &" instead took care of the issue.

It was the last update of ananicy-cpp-git that caused all this.


suppressWarnings doesn't suppress highlighting in editor

suppressWarnings does work to the extent that it hides the warnings in the Ceylon Problems and Messages windows, which is good, but it doesn't suppress the yellow highlighting in the editor. Perhaps that's desirable, but it doesn't seem quite right too me.

Isn't the typechecker supposed to not add the warnings when it sees the annotation?

Nope. That's not how it works.

Look at WarningSuppressionVisitor in typechecker/util.

Oh I see, it's used by CeylonProjectBuild for the global model.


Add main entrypoint

This PR defines a main function and an entry point named as the package "easyfermi".
After this PR it should be possible to launch easyfermi from the command line typing simply easyfermi, without the need of importing the whole package.
I updated the README and the documentation accordingly.
Unfortunately at the moment I can't yet test the software rigorously to see if I broke something because of #37 (and #45).
Closes #20.

I merged the bugfix of #37 and the error disappeared.
Now this version is still not optimal for two reasons:

while downloading or running the GUI freezes (but this might be not that critical since you wouldn't use the GUI anyway)
I still see the XML error, so I cannot further test if everything works


py3 requires you to cast keys and values into a list

Coverage remained the same at 72.597% when pulling 165ee7c6d1f08c646d3e2b500812578f73fefa09 on py3 into cb9ecd4bce0294354584a5b7587a3406eda6c383 on develop.

Coverage remained the same at 61.015% when pulling 35561f30175dfabbed9590e9d3af75219e5b2725 on py3 into 0bc3eef17bb5bd940f8998170695611f71004459 on develop.


keyboard navigation in main menu

I would like to use the keyboard in the main menu to select singleplayer, multiplayer or the highscore

@sengmann can be done with "TAB"


Dev Test: Collision Lab 1.1.0-dev.2

@arouinfar @ariel-phet, collision-lab/1.1.0-dev.2 is ready for dev testing. The intention is for this to be a quicker dev test administered by @arouinfar @ariel-phet before rc testing begins.
Document issues in https://github.com/phetsims/collision-lab/issues and link to this issue.

General Dev Test

What to Test

Click every single button.
If there is sound, make sure it works.
Make sure you can't lose anything.
Play with the sim normally.
Try to break the sim.
Try to include browser version numbers
If there is a console available, check for errors and include them in the Problem Description.
Run through the string tests on at least one platform, especially if it is about to go to rc.
Check the Game Up harness on one platform.

Focus

Glaring performance issues/

Inelastic collisions! (things like https://github.com/phetsims/collision-lab/issues/150, https://github.com/phetsims/collision-lab/issues/139, https://github.com/phetsims/collision-lab/issues/132, etc)

General Dev Test Platforms

[ ] Latest macOS, Chrome and Safari
[ ] Latest iOS, Safari
[ ] Windows 10, all browsers (IE dropped)
[ ] Latest Chrome OS, Chrome

Link(s)

Simulation

FAQs for QA Members

There are multiple tests in this issue... Which test should I do first?
Test in order! Test the first thing first, the second thing second, and so on.

How should I format my issue?

We typically assign the developer who opened the issue in the QA repository.

My question isn't in here... What should I do?
You should:

Consult the QA Book.
Google it.
Ask Katie.
Ask a developer.
Google it again.
Cry.

@brandonLi8 I've tested this quite a bit on latest macOS/Chrome and I'm not seeing any physics issues.
I plan on doing a bit more testing on macOS/Safari tomorrow.

macOS/Safari looked good to me. I did find one non-platform specific bug (https://github.com/phetsims/collision-lab/issues/167) and suggested a small pixel polish (https://github.com/phetsims/collision-lab/issues/168).

Closing this dev test - @jonathanolson has made several improvements to the performance of the sim and we should test the latest version when available.


Unexpected token

I added this code to my React project.
import { pipeline } from "@xenova/transformers";

async function sentimentAnalysis() {
  // Allocate a pipeline for sentiment-analysis
  let pipe = await pipeline("sentiment-analysis");
  let out = await pipe("I love transformers!");
  console.log(out);
}

sentimentAnalysis();

I am surprised the docs don't tell me to download a model, so I think this code will auto-download it... anyway I get this issue...
<EMAIL_ADDRESS>38:84
Module parse failed: Unexpected token (38:84)
File was processed with these loaders:

./node_modules/babel-loader/lib/index.js
You may need an additional loader to handle the result of these loaders.
|
| var RUNNING_LOCALLY = FS_AVAILABLE && PATH_AVAILABLE;

var __dirname = RUNNING_LOCALLY ? path.dirname(path.dirname(url.fileURLToPath(import.meta.url))) : './';
|
| // Only used for environments with access to file system

Seems like I need access to the filesystem... but that can't be right because this runs in the browser ... ?

Seems like I need access to the filesystem... but that can't be right because this runs in the browser ... ?

This is just a check to see if you're running in development mode (which is useful for me) - but this won't affect you.

Module parse failed: Unexpected token (38:84)

Regarding this error, could you note what version of node and npm you are using? Also, I am aware of people that use create-react-app experiencing issues with newer JS libraries. If you are using CRA, I would advise you migrate to a better build tool like Vite (see previous issues for more information on this).

node v16.17.0
yes I am using CRA

I would advise that you update to node >= 18.x (v16 is reaching EOL soon) and use Vite instead of CRA. I don't think I will be adding backwards compatibility support for older versions.


cfs-cli: no valid master

I already run config set command. And then run cluster info. But it always say no valid master. I checked the file .cfs-cli.json in root. It did exist and masterAddr was just like what I set. I modified it manually like the image upon. config info also show the result. But cluster info still got no valid master
Version Info: v2.1.0 in Release Binary

The reason already found.
The master server config format should be masterIp:Port.
I will close this issue.


Windows GPU accuracy extremely bad

Hey i'm quite new to mxnet, I followed the installation instructions and succeeded in installing it on windows 8.1 64 bit, I then ran the train_mnist.py --network lenet without a problem, quite slow but the accuracy at the end is good at around 99.2, but when I run it as --network lenet --gpus 0 to use my gpu its definitely a lot faster but the accuracy never gets above 10% which is terrible, there must be something wrong theoretically it should be the same accuracy right? I installed cuda 7.5 and also extracted cuddn v3 just as indicated, everything runs without a problem except the accuracy is terrible, i'm running on a laptop with a nvidia 660m graphics card, it has compute capability 3.0.
After running the file I get Train-accuracy=0.098825

Here is my output from train_mnist.py:
2016-01-09 12:48:47,622 Node[0] start with arguments Namespace(batch_size=128, data_dir='mnist/', gpus=None, kv_store='local', load_epoch=None, lr=0.1, lr_factor=1, lr_factor_epoch=1, model_prefix=None, network='mlp', num_epochs=10, num_examples=60000)
[12:48:51] src/io/iter_mnist.cc:91: MNISTIter: load 60000 images, shuffle=1, shape=(128,784)
[12:48:52] src/io/iter_mnist.cc:91: MNISTIter: load 10000 images, shuffle=1, shape=(128,784)
2016-01-09 12:48:52,053 Node[0] Start training with [cpu(0)]
2016-01-09 12:48:53,105 Node[0] Epoch[0] Batch [50]	Speed: 6447.52 samples/sec	Train-accuracy=0.686719
2016-01-09 12:48:53,829 Node[0] Epoch[0] Batch [100]	Speed: 8836.63 samples/sec	Train-accuracy=0.793828
2016-01-09 12:48:54,660 Node[0] Epoch[0] Batch [150]	Speed: 7707.90 samples/sec	Train-accuracy=0.836302
2016-01-09 12:48:55,366 Node[0] Epoch[0] Batch [200]	Speed: 9064.13 samples/sec	Train-accuracy=0.858555
2016-01-09 12:48:56,192 Node[0] Epoch[0] Batch [250]	Speed: 7749.72 samples/sec	Train-accuracy=0.873969
2016-01-09 12:48:57,027 Node[0] Epoch[0] Batch [300]	Speed: 7662.28 samples/sec	Train-accuracy=0.885052
2016-01-09 12:48:57,808 Node[0] Epoch[0] Batch [350]	Speed: 8206.58 samples/sec	Train-accuracy=0.893951
2016-01-09 12:48:58,552 Node[0] Epoch[0] Batch [400]	Speed: 8606.22 samples/sec	Train-accuracy=0.900723
2016-01-09 12:48:59,377 Node[0] Epoch[0] Batch [450]	Speed: 7758.36 samples/sec	Train-accuracy=0.906563

It looks fine. Did you try pulling the newest change and make clean && make?

here is mine:

C:\mxnet\nocudnn\python\image-classification>D:\Python27\python.exe train_mnist.
py --network lenet --gpus 0
2016-01-09 20:52:15,706 Node[0] start with arguments Namespace(batch_size=128, d
ata_dir='mnist/', gpus='0', kv_store='local', load_epoch=None, lr=0.1, lr_factor
=1, lr_factor_epoch=1, model_prefix=None, network='lenet', num_epochs=10, num_ex
amples=60000)
[20:52:17] D:\chhong\mxnet\src\io\iter_mnist.cc:94: MNISTIter: load 60000 images
, shuffle=1, shape=(128, 1, 28, 28)
[20:52:18] D:\chhong\mxnet\src\io\iter_mnist.cc:94: MNISTIter: load 10000 images
, shuffle=1, shape=(128, 1, 28, 28)
2016-01-09 20:52:18,315 Node[0] Start training with [gpu(0)]
2016-01-09 20:52:20,598 Node[0] Epoch[0] Batch [50]     Speed: 4719.76 samples/s
ec      Train-accuracy=0.096719
2016-01-09 20:52:21,969 Node[0] Epoch[0] Batch [100]    Speed: 4668.13 samples/s
ec      Train-accuracy=0.098203
2016-01-09 20:52:23,334 Node[0] Epoch[0] Batch [150]    Speed: 4688.64 samples/s
ec      Train-accuracy=0.100625
2016-01-09 20:52:24,688 Node[0] Epoch[0] Batch [200]    Speed: 4723.25 samples/s
ec      Train-accuracy=0.100039
2016-01-09 20:52:26,042 Node[0] Epoch[0] Batch [250]    Speed: 4726.74 samples/s
ec      Train-accuracy=0.098344
2016-01-09 20:52:27,424 Node[0] Epoch[0] Batch [300]    Speed: 4634.32 samples/s
ec      Train-accuracy=0.099635
2016-01-09 20:52:28,793 Node[0] Epoch[0] Batch [350]    Speed: 4671.53 samples/s
ec      Train-accuracy=0.099955

As you can see the accuracy remains at the 9% range, and even after the 10 epochs it remains the same, as far as the make part, I downloaded and installed pre-built package for gpu from here
https://github.com/dmlc/mxnet/releases

My output with exactly the same command on linux:
python train_mnist.py --network lenet --gpus 0
2016-01-09 14:18:41,245 Node[0] start with arguments Namespace(batch_size=128, data_dir='mnist/', gpus='0', kv_store='local', load_epoch=None, lr=0.1, lr_factor=1, lr_factor_epoch=1, model_prefix=None, network='lenet', num_epochs=10, num_examples=60000)
[14:18:43] src/io/iter_mnist.cc:94: MNISTIter: load 60000 images, shuffle=1, shape=(128, 1, 28, 28)
[14:18:43] src/io/iter_mnist.cc:94: MNISTIter: load 10000 images, shuffle=1, shape=(128, 1, 28, 28)
2016-01-09 14:18:43,402 Node[0] Start training with [gpu(0)]
2016-01-09 14:18:46,866 Node[0] Epoch[0] Batch [50]	Speed: 2515.84 samples/sec	Train-accuracy=0.810000
2016-01-09 14:18:49,499 Node[0] Epoch[0] Batch [100]	Speed: 2431.10 samples/sec	Train-accuracy=0.876484
2016-01-09 14:18:52,040 Node[0] Epoch[0] Batch [150]	Speed: 2518.40 samples/sec	Train-accuracy=0.903073
2016-01-09 14:18:54,563 Node[0] Epoch[0] Batch [200]	Speed: 2537.25 samples/sec	Train-accuracy=0.918750
2016-01-09 14:18:57,251 Node[0] Epoch[0] Batch [250]	Speed: 2380.75 samples/sec	Train-accuracy=0.928750
2016-01-09 14:18:59,741 Node[0] Epoch[0] Batch [300]	Speed: 2570.31 samples/sec	Train-accuracy=0.936120
2016-01-09 14:19:02,343 Node[0] Epoch[0] Batch [350]	Speed: 2459.97 samples/sec	Train-accuracy=0.941897
2016-01-09 14:19:04,880 Node[0] Epoch[0] Batch [400]	Speed: 2523.58 samples/sec	Train-accuracy=0.946660
2016-01-09 14:19:07,560 Node[0] Epoch[0] Batch [450]	Speed: 2387.78 samples/sec	Train-accuracy=0.950122

This seems to be a windows specific issue. @hjk41 Could you look into it?

  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]]

Could you try to run cuda's sample code for matrix multiply and see if the
results are normal?
On Jan 9, 2016 6:18 PM, "jonathanponce"<EMAIL_ADDRESS>wrote:

It returns an array of zeros, seems as if the operations are not taking
place or are all returning zero

print x.asnumpy()
[[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]]

—
Reply to this email directly or view it on GitHub
https://github.com/dmlc/mxnet/issues/1228#issuecomment-170301104.

I ran the sample code and everything seems to be ok
[Matrix Multiply Using CUDA] - Starting...
GPU Device 0: "GeForce GTX 660M" with compute capability 3.0

MatrixA(320,320), MatrixB(640,320)
Computing result using CUDA Kernel...
done
Performance= 4.40 GFlop/s, Time= 29.805 msec, Size= 131072000 Ops, WorkgroupSize
= 1024 threads/block
Checking computed result for correctness: Result = PASS

NOTE: The CUDA Samples are not meant for performance measurements. Results may v
ary when GPU Boost is enabled.

The results are as expected

I can't reproduce the problem locally so I can't think of anything now.
You can try git bisect https://git-scm.com/docs/git-bisect to see if it's a recently introduced bug.

I tried out the previous Windows build and it worked without a problem, so that means windows binary build 20160106 has a bug in the gpu computation section, there have been 29 commits since then so its possible that it has been fixed already.

Even if it is just to back @jonathanponce, I have exactly the same problem. Running train_mnist.py without the --gpus 0 command gives an accuracy of about 0.97, but running with --gpus 0 gives an accuracy of about 0.07
I use Windows 7 64bit with Python 2.7 and have tried windows binary build 20160120 and windows binary build 20160113. Both have the same problem for me.

@hjk41 Looks like gpu code is not running but not reporting error on windows with their cards. Could you look into it?

@piiswrong I watched the gpu load with GPU-Z when running the mxnet code and the gpu load is around 25%, so the code is using my gpu.

This post reports on the same issue:
https://www.kaggle.com/c/second-annual-data-science-bowl/forums/t/18079/end-to-end-deep-learning-tutorial-0-0392/105458#post105458
I ran into the same situation as well. Not sure yet if the earlier releases solve the problem.

Same issue here with mxnet and python. I installed the latest windows build 20160202 and while training a network the accuracy wasn't increasing. The computation was taking placing on the gpu because I checked it with gpu-z....
Did the simple arithmetic tests on gpu  mentioned by @piiswrong and it gave me zeroes.
So I switched to the 20151228 build and now it works ok.
So definately the bug from 20160106 still exists in 20160202. Hope it helps.....

@piiswrong @Quares @JohanManders @gpapadop79
Sorry it take me so long to respond. I was fully occupied with an internal conference last few weeks. I just tried with 20160202 and simple test seems to work alright for me. I guess it must be something in the system configuration side. I am using Windows Server 2012 Datacenter, Python 2.7.10 x64. I will try to switch to some other platform and see if it works there.
Meanwhile, could you help me narrow down the problem a little bit? Here are some speculations:

run "where libmxnet.dll" and see if you are using the right version of libmxnet.dll
run matrixMulCuBLAS from nvidia CUDA samples and see if it works
try building mxnet from source and do the test again

Python 2.7.10 (default, May 23 2015, 09:44:00) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import mxnet as mx
OpenCV is unavailable.
>>> a = mx.nd.ones((2,3), mx.gpu(0))
>>> a.asnumpy()
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]], dtype=float32)
>>> x = mx.nd.zeros((10,10), ctx=mx.gpu(0))
>>> x[:] = 1
>>> x = x*2
>>> print x.asnumpy()
[[ 2.  2.  2.  2.  2.  2.  2.  2.  2.  2.]
 [ 2.  2.  2.  2.  2.  2.  2.  2.  2.  2.]
 [ 2.  2.  2.  2.  2.  2.  2.  2.  2.  2.]
 [ 2.  2.  2.  2.  2.  2.  2.  2.  2.  2.]
 [ 2.  2.  2.  2.  2.  2.  2.  2.  2.  2.]
 [ 2.  2.  2.  2.  2.  2.  2.  2.  2.  2.]
 [ 2.  2.  2.  2.  2.  2.  2.  2.  2.  2.]
 [ 2.  2.  2.  2.  2.  2.  2.  2.  2.  2.]
 [ 2.  2.  2.  2.  2.  2.  2.  2.  2.  2.]
 [ 2.  2.  2.  2.  2.  2.  2.  2.  2.  2.]]

Just tried on another machine with Windows Server 2012R2, Python 2.7.10 x64, it also works fine. :-(
I think I need some help here. It would be great if someone is willing to share a machine that can reproduce the problem.

Looks like it's caused by low cuda compute capability GPUs.

Could be. I am running Titan. Does this also occur for low compute capability GPUs on Linux?

I have a GTX 670 and when I boot into Ubuntu, mxnet works fine. In Windows I cannot get it to work.
I ran some tests on my Windows 7 64bit, using windows binary build 20160216. Using a build earlier, does the same for me.

libmxnet.dll is in the right place

C:\Users\XXXXX>where libmxnet.dll
C:\Anaconda\Lib\site-packages\mxnet-0.5.0-py2.7.egg\mxnet\libmxnet.dll

matrixMulCuBLAS passes

C:\ProgramData\NVIDIA Corporation\CUDA Samples\v7.5\bin\win64\Release>matrixMulC
UBLAS.exe
[Matrix Multiply CUBLAS] - Starting...
GPU Device 0: "GeForce GTX 670" with compute capability 3.0`

MatrixA(640,480), MatrixB(480,320), MatrixC(640,320)
Computing result using CUBLAS...done.
Performance= 1059.89 GFlop/s, Time= 0.185 msec, Size= 196608000 Ops
Computing result using host CPU...done.

but mxnet gives me 0. 0. 0.

Python 2.7.11 |Anaconda 2.3.0 (64-bit)| (default, Jan 29 2016, 14:26:21) [MSC v.
1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import mxnet as mx
>>> a = mx.nd.ones((2,3), mx.gpu(0))
>>> a.asnumpy()
array([[ 0.,  0.,  0.],
       [ 0.,  0.,  0.]], dtype=float32)
>>> x = mx.nd.zeros((10,10), ctx=mx.gpu(0))
>>> x[:] = 1
>>> x = x*2
>>> print x.asnumpy()
[[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]]
>>>

@jonathanponce So it is not related to compute capability, since both GTX670 and Titan have compute capability 3.0.
Could you try to run a C++ program? You can try to this one:
https://github.com/hjk41/MxNet.cpp.git
Checkout the test branch and copy libmxnet.lib/libmxnet.dll to lib/windows/, then build the solution in windows/vs/MxNetTestApp/MxNetTestApp.sln with x64. The program just creates an NDArray on GPU, populate it with ones and then print it out. This is pretty much what mx.nd.ones((2,3), mx.gpu(0)) does.

@piiswrong @tqchen @antinucleon
I just find that both Python and R invokes NDArray::SetValueOp() directly in zeros() and ones(). I think calling C++ member functions from dynamic libraries are pretty dangerous. I am not sure whether it is related to the problem we are facing here. But it will bite us someday. Can we add a C api for SetValueOp so we can avoid the problem?

@hjk41 Did you want me to do the test? If so, I cloned the test branch, copied the dll and lib file (also needed the lib file) and build the solution successfully. I don't know what should happen or how long it should take, but running the program seems to do nothing.

@JohanManders The program should output a series of digits from 0 to 5. If it prints nothing, then there must b something wrong. It means the problem also occurs for c++ programs.

@hjk41 Mmm... Strange... Building CUDA samples like marchingCubes, matrixMulCUBLAS and particles seem to be no problem and run perfectly.

I also ran matrixMulCUBLAS and it passes.
My environment is Windows 7 x64 python 2.7.11 (Anaconda 2.5.0) and GTX 960 (which has compute capability 5.2)

Thanks guys. I think I will have to reinstall one of my machines to use Windows 7 to reproduce the problem, which will need some time. Meanwhile, if someone can try to debug the problem, it would be great. With the C++ program, it shouldn't be too hard.

So I assume the new (7th) release doesn't solve the issue yet? How is it with you @JohanManders? I haven't had time to work on my desktop to test it yet.

@Quares I have tried the latest build, Windows binary build 20160216, and I still have the problem.

I just found that I have the same problem, tried both mnist and cifar10 examples. I am using GTX980 and Windows 10.
I tried different builds and found that all snapshots after build20151228 do not work, also spotted that the file size has been reduced very much since build 20151228 --- are there changes in compilation config?

I tried link to the provided CUDA/CuDNN DLL files and also my own DLLs (same version) via different PATH, did not work either.
Perhaps it maybe a compiler / OS level issue.

@thyu Could you try the C++ program in the test branch of https://github.com/hjk41/MxNet.cpp.git
I have recreated the problem with a Windows 10 machine in Python, but the C++ program runs just fine

Seems fine?
$ ./MxnetTestApp.exe
0 1 2 3 4 5

Yes. So it seems to be something in Python/R binding or how they use the
library.
On Fri, Feb 19, 2016 at 3:34 AM, thyu<EMAIL_ADDRESS>wrote:

@hjk41 https://github.com/hjk41
Seems fine?
$ ./MxnetTestApp.exe
0 1 2 3 4 5
—
Reply to this email directly or view it on GitHub
https://github.com/dmlc/mxnet/issues/1228#issuecomment-185882232.

--
HONG Chuntao
System Research Group
Microsoft Research Asia

@thyu @jonathanponce @JohanManders @Quares @piiswrong
Could you help me check with the latest binary build here? https://github.com/dmlc/mxnet/releases/tag/20160223
I think it is the problem with CUDA library. Windows Server 2012 and Windows 10/8 uses different CUDA binaries so I assume there is some difference between the libraries we link. The libmxnet.dll compiled on Windows 2012 does not work on Windows 10/8, and vice versa.
The latest binary was compiled on Windows 10 and it works well on my machine. But I don't have another Windows 10/8/7 machine to test on. So could you help me validate this?

@hjk41 Your latest builds seems to work perfectly! Thanks man, this helps me a lot! The mnist example now outputs a train accuracy of 0.999 and validation accuracy of 0.991.

@JohanManders Great! I will use Windows 10 in the future for building the binary distribution.

Great news! I will test it in the evening and will let you know.

@hjk41 The latest build works!!! Thanks!    You rock!!!!

@JohanManders @Quares @hjk41
Did anyone else notice a small decrease in performace on the latest release?
When I trained a model with the 20151228 release, needed about 15.5 sec/epoch. Now with the latest release training the exact same model takes 18.5 sec/epoch.

I am happy that it works, but I also see a big speed difference between Windows and Ubuntu. For Windows I downloaded the latest pre-built package 20160223. For Ubuntu I just downloaded the latest version and build it.
I did two tests on my dual-boot i7 system with a GTX 670:
train_mnist.py
Windows 7 | Build 20160223                        : ~  6750 samples / sec
Ubuntu    | Downloaded en build a few minutes ago : ~ 20000 samples / sec

Training on other data
Windows 7 | Build 20160223                        : ~ 49 sec / epoch
Ubuntu    | Downloaded en build a few minutes ago : ~ 31 sec / epoch

Darn! I must switch to linux! :-P
My speed difference is on win7 between pre-built 20151228 and 20160223. I also tried CuDNN 4 but had no difference.

The new release (20160223) works on my Windows10 machine. Great work guys!
Side note: I also noticed (in my case substantial) performance decrease in terms of speed, but that's probably related to various other things happening.

The new release works on my machine as well, awesome!
I also observe that Linux is faster than Windows for quite a while, I have Linux box with GTX970 which runs around 700 images per sec something over train_cifar10, but in my company's Windows machine it is only slightly higher than 620 img per sec. It might not be just a single-factor issue and perhaps we can improve it afterwards...

Interesting. I will take a look into it. I expect there to be some
performance difference between Windows and Linux, but didn't expect it to
be so huge.
On Wed, Feb 24, 2016 at 7:33 AM, thyu<EMAIL_ADDRESS>wrote:

The new release works on my machine as well, awesome!
I also observe that Linux is faster than Windows for quite a while, I have
Linux box with GTX970 which runs around 700 images per sec something over
train_cifar10, but in my company's Windows machine it is only slightly
higher than 620 img per sec. It might not be just a single-factor issue and
perhaps we can improve it afterwards...
—
Reply to this email directly or view it on GitHub
https://github.com/dmlc/mxnet/issues/1228#issuecomment-187965807.

--
HONG Chuntao
System Research Group
Microsoft Research Asia

Here is my results:
python train_mnist.py:
GTX980, Windows 10:               20000 samples/sec
Titan, Ubuntu 14.04:                  40000 samples/sec

Also, CPU runs much faster in this case, at around 50000 samples/sec.
python train_cifa10.py:
GTX980, Windows 10:               396 samples/sec
Titan, Ubuntu 14.04:                  445 samples/sec

So my guess is that Windows has higher overhead with regard to small GPU operations. In MNIST, the computation is so light that this overhead dominates, and thus CPU > GPU-Linux > GPU-Windows. In Cifa10, there are much less operations and hence the difference is much smaller. Has anyone tried running heavier workloads like ImageNet?

@Quares
According to this: https://github.com/dmlc/mxnet/pull/1449
cudnn 4 is supported

@gpapadop79 cool! great to know! I was under the impression about cuddn3 because the documentation doesn't reflect on cuddn4 yet. I am interested to see how the performance changes between cudnn3 and cudnn4. I am running both GTX660Ti and GTX980Ti on two seperate machines so have a nice overview of the performance upgrade between the two cards.

Hi,
Installed the GPU enabled R library (R version: 3.2.3) on Windows 7 today. Looks like it's working on the CPU but not on my GPU. The code seems to execute on the GPU (confirmed with GPU-Z), but error improvement stalls on the second round using example code (as above merged issue), and mx.nd.ones(c(2,3), mx.gpu()) generates a table of 0's not 1's.
I'm using the latest files for everything, and the precompiled GPU package for R. I read in this thread: https://github.com/dmlc/mxnet/issues/250, that indicates 'remove USE_CUDNN' to compile for Cuda Compute GPUs 2.1 (and lower). I'm using a 2.1 GPU. Could this be the problem?
Could using an earlier 2015 release be a solution? Also might I have to compile my own GPU-enabled files without USE_CUDNN to fix this? I'm hoping I don't have to upgrade my computer to get his working as I'm only doing preliminary testing.
Any help would be appreciated.
Thanks, Gavin.

I haven't tested it on GPU with with compute capability 2.1. I guess the
pre-built binary may not work for you, since it is compiled with compute
capability 3.5. Could you try to compile from source and see if it works?
On Sun, Mar 6, 2016 at 11:49 PM, xenmind<EMAIL_ADDRESS>wrote:

Hi,
Installed the GPU enabled R library (R version: 3.2.3) on Windows 7 today.
Looks like it's working on the CPU but not on my GPU. The code seems to
execute on the GPU (confirmed with GPU-Z), but error improvement stalls on
the second round using example code (as above merged issue), and
mx.nd.ones(c(2,3), mx.gpu()) generates a table of 0's not 1's.
I'm using the latest files for everything, and the precompiled GPU package
for R. I read in this thread: #250
https://github.com/dmlc/mxnet/issues/250, that indicates 'remove
USE_CUDNN' to compile for Cuda Compute GPUs 2.1 (and lower). I'm using a
2.1 GPU. Could this be the problem?
Could using an earlier 2015 release be a solution? Also might I have to
compile my own GPU-enabled files without USE_CUDNN to fix this? I'm hoping
I don't have to upgrade my computer to get his working as I'm only doing
preliminary testing.
Any help would be appreciated.
Thanks, Gavin.
—
Reply to this email directly or view it on GitHub
https://github.com/dmlc/mxnet/issues/1228#issuecomment-192919248.

--
HONG Chuntao
System Research Group
Microsoft Research Asia

Will do, thanks.

Just wanted to report that compiling the shared dll, with USE_CUDNN turned off worked.

The error is back in the latest release the accuracy remain terrible, the 20160223 build works fine, but the rest appear to have the error again

Gosh... Which windows version are you using?
On Tue, Mar 22, 2016 at 8:40 PM, jonathanponce<EMAIL_ADDRESS>wrote:

The error is back in the latest release the accuracy remain terrible, the
20160223 build works fine, but the rest appear to have the error again
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/dmlc/mxnet/issues/1228#issuecomment-199791777

--
HONG Chuntao
System Research Group
Microsoft Research Asia

Windows 8.1

I found the 20160419 binary release has the same problem on my computer, then I compiled mxnet from the latest source and it now works fine.
My hardware and software configurations:
GPU: GTX850M
OS: Windows 8.1 x64
Compiler: Visual Studio 2013 Update 5
3rdparty Software: CUDA 7.5, CUDNN V4, OpenCV 3.1, OpenBLAS 0.2.14
With the default options it produces a 63,658KB libmxnet.dll，with Link-Time Code Generation turned on the size is reduced to 19,900KB, both work fine in GPU mode and the speed is about the same.

Hi wdx04,
Would you mind sharing the compiled libmxnet.dll (63,658kb) or update the instruction. I have same setting but can't make it work.
Thank you.

Hi @piiswrong @hjk41
I found the GPU-based mxnet didn't work well on my laptop. I have the same problem about mxnet mentioned above. The performance was very good when using CPU-based mxnet, while the GPU-based method had a low accuracy around 10% on mnist dataset.
My OS is win 10 64-bits, and the device graphic is nividia GTX 960M,  CUDA 7.5. I used 20160531-win 10-baniry-release to install the mxnet on my laptop. In addition, I have tested the CUDA with C and pycuda, both of them were Ok.
So, I would like ask how to deal with it? Any help could be appreciated. Thank you.

Hi @juanlp ,
You may download my version of libmxnet.dll here(built on 5/22):
http://pan.baidu.com/s/1bzxnvK
I'm facing compiling errors with the latest source so I can't provide the latest libmxnet.dll binary.

Compiling errors are fixed, here is the latest binary:
http://pan.baidu.com/s/1dFrzQet

I'm having the same issue. Windows 10 64bit, GTX 1080, VS2013 + CUDA 8.0 + CuDNN v5, Python 2.7.11 x64, compiling from the source code. CPU works fine but GPU fails - remains at 10% for MNIST example. CUDA samples work OK, but a = mx.nd.ones((2,3), mx.gpu(0)) returns all zeros instead of ones. I can't find a solution in the thread above.

Hi @wdx04,
I downloaded your libmxnet-0612.dll and unpacked it, and put it into the original mxnet file instead of the previous libmxnet.dll. But aftering install it using python setup.py install, there are some mistakes when runing import mxnet. What is up ? Thank you

Hi @pablozhang,
Probably you don't have OpenCV 3.1 and/or cuDNN v4 dlls, my libmxnet.dll was dynamically linked to these libraries, as opposed to the official one. You can download them here:
http://pan.baidu.com/s/1bpbURXt

Hi @wdx04 ,
I have a confuse about it whether or not I unpack the file and put the two .dll into the 3rdparty file of mxnet is Ok?

Hi @wdx04,
Thank you for your help, my mxnet does work now. The error has been fixed. Thank you.

Hi @pablozhang,
You are welcome. I think placing the two .dll in any directory listed in your PATH environment variable will be OK.

Really appreciate the job by @wdx04 . This can be a temporary fix for the problem.

@piiswrong @hjk41
The problem persists for CUDA 8.0. Is there a known cause for GPU to fail like this? Is there a known solution?

I use pre-builded mxnet 20160531_win10_x64_gpu package and I install cuda_8.0.27_win10,but I still have the same problems. DId anyone know the reason ?

@cemkeskin  have you solve you problem? I have the same question with you?

Here is the problem:
GTX 1080 + CUDA 8.0 = mxnet returns zeros for mx.nd.ones()
GTX 1080 + CUDA 7.5 = curand() doesn't work (NVIDIA issue) and mxnet relies on curand for some tasks. But training on CPU works
So you either need to remove all references to curand (e.g. initialize your arrays manually), or switch back to an older card.

Hi, I got the same problem about training accuracy lower than 10%, I got the same issue like @JohanManders.
I use R 3.3.0, Windows10, cudnn v3, CUDA 7.5, GPU card is GTX 1080.
I tried to create the table like what you did on R, and this might be the reason. And switch the gpu version to cpu works fine.
> a = mx.nd.ones(c(2,3), mx.gpu(0))
> a
     [,1] [,2] [,3]
[1,]    0    0    0
[2,]    0    0    0
> a+1
     [,1] [,2] [,3]
[1,]    0    0    0
[2,]    0    0    0

> a = mx.nd.ones(c(2,3), mx.cpu(0))
> a
     [,1] [,2] [,3]
[1,]    1    1    1
[2,]    1    1    1
> a+1
     [,1] [,2] [,3]
[1,]    2    2    2
[2,]    2    2    2

I switch From W10 to Ubnutu16.04 ,train minist both on cpu and gpu  work  correctly. My Environment is  GTX 080 + CUDA 8.0  + Ubuntu16.04 ,

Hey, @yuantangliang thanks for sharing. I will try it on ubuntu. I have checked the mxnet C++  version on windows with CUDA 7.5, cudnn v3. It seems works fine. So the problem might be how R using the library of CUDA on windows.

@yuantangliang Hi, thanks for your tips, but I am a totally newbie of ubuntu, while I was trying to compile mxnet of ubuntu for R, I got this message:
$ make -j$(nproc)
make: Nothing to be done for `all'.

I followed this page to edit some lines of the config.mk .
USE_CUDA = 1
USE_CUDA_PATH = /usr/local/cuda
USE_BLAS = atlas

I don't know what make this problem. May you help me with it?

Did you run cmake first ? you should first run cmake to generate MakeFile.

@yuantangliang Hey, Finally I successfully run mxnet package on R with Ubuntu without the low accurate problem and correctly ones matrix of counting on GPU. Thanks for helping. My environment is GTX1080, Ubuntu 14.04, CUDA 8.0, R 3.3.1. This is a really nice experience to try it on Ubuntu. I learned a lot. :D

Has anyone made R to work with GPU on Windows?

@rfcv not yet, but it works pretty well on ubuntu.

I can confirm that it works fine on Windows and R on the latest version
with CUDA 7.5
On 12 Jul 2016 08:39, "long-jian"<EMAIL_ADDRESS>wrote:

@rfcv https://github.com/rfcv not yet, but it works pretty well on
ubuntu.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/mxnet/issues/1228#issuecomment-231906667, or mute
the thread
https://github.com/notifications/unsubscribe/AChJ6DHGmWrIMdcL4h9asv06ANqTNysTks5qUuJXgaJpZM4HBwc2
.

@juanlp can't make sure what make my problem, what is your GPU environment?

I think it is related to cudnn version.
I tested cudnn4+cuda8 in ubuntu 16.04 with gtx1080. I got very bad accuracy.
But I get very good accuracy when change to cudnn5

I'm having exactly same problem with windows 10 64bit, Visual Studio 2013, CUDA 8.0 RC, CUDNN5, built from source as of 8/10. Has anyone figured out what is the root cause of this issue?
It appears NDArray assignment by numerical value doesn't work, but setting by numpy array is fine, as in this short example.
import mxnet
arr = mxnet.ndarray.empty( (2,2), mxnet.gpu(0) )
arr[:] = 1.
arr.asnumpy()
array([[ 0.,  0.],
       [ 0.,  0.]], dtype=float32)
import numpy
arr[:] = numpy.ones((2,2))
arr.asnumpy()
array([[ 1.,  1.],
       [ 1.,  1.]], dtype=float32)

Also, when I built mxnet debug version the gpu is working fine, e.g. mnist example converges quickly.

Same Problem with pre-built20160531 on my win7 64bit and CUDA8.0 RC, CUDNN5, VS2013. Also passed cuda samples' matrix multiply.
I'm going to back to 20160223 and see if it works.

same issue as everyone, CUDA 7.5, CUDNN3, VS2013, with the latest windows build for GPU, the accuracy is very low, and the matrix test only gives 0.
Reverted to the 20160223 build and it works, 0.9911 on MNIST.

Using GTX 1080, CUDA 8.0, CuDNN v5.1, VS2015, Win10, libmxnet.dll built from the source code. Release version still fails with mx.nd.ones, but the debug version works. So the problem is only with the release version.

The problem still exists in release 20160531_win10_x64_gpu.7z and v0.7.0. Using GT 740, CUDA 7.5, cuDNN v3, in Win7.

I have the same problem on release 20160531_win10_x64_gpu.7z:

a=mx.nd.ones((2, 3), mx.gpu())
a.asnumpy()
array([[ 0.,  0.,  0.],
[ 0.,  0.,  0.]], dtype=float32)
b=mx.nd.ones((2, 3), mx.cpu())
b.asnumpy()
array([[ 1.,  1.,  1.],
[ 1.,  1.,  1.]], dtype=float32)

I tested the windows binary build 20160223,it hasn't this problem.

I tested 20160531_win2012_x64_gpu.7z,it hasn't this bug too

@qggjonny  Can you share your cudnn version, cuda version and GPU hardware?
I tested 20160223_win10_x64_gpu.7z on cudnn-7.0-win-x64-v3.0-prod.zip, cuda 8.0 and GTX1080.
Still not working.
In [1]: import mxnet as mx
...: a=mx.nd.ones((2, 3), mx.gpu())
...: print(a.asnumpy())
...:
...: b=mx.nd.ones((2, 3), mx.cpu())
...: print(b.asnumpy())
...:
[[ 0.  0.  0.]
[ 0.  0.  0.]]
[[ 1.  1.  1.]
[ 1.  1.  1.]]
Still did not work.

My cuda version is 7.5,GPU is GT730。
Perhaps you can try 20160531_win2012_x64_gpu.7z

@qggjonny  When I installed win2012 version and call import mxnet in python console, it tells me that missing cudnn64_70.dll, but I have put my cudnn in the correct folder and cudnn v5 & v5.1 are both named cudnn64_5.dll, can you look at the 3rdparty folder and see what is your cudnn version?

I put cudnn64_70.dll in mxnet\3rdparty\cudnn and mxnet\3rdparty\cudnn\bin.

I have the same problem with 20160531_win10_x64_gpu and also with 20160531_win2012_x64_gpu:
In [3]: (mxnet.nd.ones((2,2), mxnet.cpu())*100).asnumpy()
Out[3]:
array([[ 100.,  100.],
       [ 100.,  100.]], dtype=float32)

In [4]: (mxnet.nd.ones((2,2), mxnet.gpu())*100).asnumpy()
Out[4]:
array([[ 0.,  0.],
       [ 0.,  0.]], dtype=float32)

I return the 20160223_win10_x64_gpu.7z, the GPU works also.  But in the newest verision, it does not work...

@jf003320018 I also tried 20160223_win10_x64_gpu.7z, but not work. I use cuda8.0 with cudnn3.
What is your cuda and cudnn?

@auroralinan Maybe you can try cudnn v3, it has cudnn64_70.dll

@yunzhou I have the same environment with you and mxnet does not work either.

@MaticsL  I tried 20160223_win10_x64_gpu.7z + cuda7.5 + cudnn3. The gpu ones functions returns 0
Also my hardware is gtx1080.
I also notice that even if I put nothing under 3rdparty\cudnn 20160223_win10_x64_gpu, it still runs, but the gpu ones function returns 0.
I will try cuda 7.0 later

@yunzhou my CUDA is 8.0 and cudnn is V3.  Just following the readme 20160223_win10_x64_gpu.7z, it will work.

@jf003320018 thanks. Since CUDA 7.0 can not recognize gtx 1080. I will return to CUDA8.0 and try 20160223_win10_x64_gpu.7z.

i used to compile mxnet with CUDA 8.0 RC  cudnn 5.1 opencv 3.0  mkl in windows 10 and had the this problem too
now, i change  CUDA 8.0 RC to CUDA 8.0 and find no problem
P.S i use gtx 1060

Using R-package, I have this exact problem with:
Windows 10
CUDA 8.0
cudnn V3
GTX 1060
20160531_win10_x64_gpu as well as 20160223_win10_x64_gpu
Does anyone have a reliable solution ?

@sanson87 You can use prebuild version at https://github.com/dmlc/mxnet/issues/2813

@MaticsL Thanks, do you know how I can build my GPU R-package from those? The folder 3rdparty seems to be missing from the 20161101_mxnet_x64_gpu for instance. Sorry if it's a dumb question.

I have the same problem on windows 10 using
CUDA 8.0
cudnn V3
GeForce 840M
I have tried both 20160531_win10_x64_gpu and 20161104_win10_x64_gpu
but running on GPU the train accuracy remaine always fixed.
Which version can I try to solve this problem?
Thanks.

It seems like this problem can't be solved...

Actually compiling from the latest source code with the new CUDA 8.0.44, cuDNN 5.1 and VS2015 worked for me finally. The issue seems to be solved, most likely due to the new CUDA release. With CUDA 8.0.27 only the Debug version was working correctly.

Thanks cemkeskin, but the problem still persists..
I'm using the same versions you mentioned (CUDA 8.0.44, cuDNN 5.1, VS2015), but using the R-Package folder from the GitHub repository (precisely, this: https://github.com/dmlc/mxnet/) and editing with the windows binary build 20160531 (see the first attachment) i get this error when try to installing (see the second attachment). I tried to editing the "NAMESPACE" file removing those names that can't be found, and the installation seems to work, but when i run the models in R under the GPU the train accuracy remains fixed after the second epoch.
Any suggestion? What do you mean with "compiling from the source code"?

I don't want to be annoying, but can someone help me? I really need a solution

To build MxNet from source, please follow the instructions here:
http://mxnet.io/get_started/setup.html#build-mxnet-on-windows
The prebuilt binary sometimes have strange problems with different OS/CUDA
configurations. Building from source usually solves the problem
On Wed, Nov 9, 2016 at 5:49 PM, No41Name<EMAIL_ADDRESS>wrote:

I don't want to be annoying, but can someone help me? I really need a
solution
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/mxnet/issues/1228#issuecomment-259373691, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABFI4Zv4-sAyjuVyF59GgkeqUHK0w9K-ks5q8ZcYgaJpZM4HBwc2
.

When I use the 20160531_win10_x64_gpu.7z, I have same problem on :
Windows8.1, CUDA 8.0, cuDNN v3 or v5, VS 2015, GTX 860m.
But I do solve this problem by using this https://github.com/yajiedesign/mxnet/releases  version, I follow actions:

Uninstall mxnet from python. (if you already install it.)
Download the 20161125_mxnet_x64_gpu.7z , unpack it, copy all files into path you previous unpack 20160531_win10_x64_gpu.7z and replace those there.
Do download the cuDNNv4 and unpack it into .../3rdparty/cudnn/
Run the setupenv,cmd or  set the enviornment path.
cd to .../python and type python setup.py install.

There is one thing to note that it requires cudnn64_4.dll which belongs to cuDNNv4.

I met the same problem in Ubuntu16.0.4 LTS , CUDA 8.0.44, CuDNN 5.1. And my GPU is Tesla K40c
But this doesn't happen all the time.
I want to know how to solve this problem, thanks.

I am facing the simliar issue which is even more interesting. I have single binary runs twice and get totally different result.
D:\mxnet\example\image-classification>python train_mnist.py --gpus 0 --network lenet
INFO:root:start with arguments Namespace(batch_size=64, disp_batches=100, gpus='0', kv_store='device', load_epoch=None, lr=0.1, lr_factor=0.1, lr_step_epochs='10', model_prefix=None, mom=0.9, network='lenet', num_classes=10, num_epochs=20, num_examples=60000, num_layers=None, optimizer='sgd', test_io=0, top_k=0, wd=0.0001)
WARNING:root:[Deprecation Warning] mxnet.model.FeedForward has been deprecated. Please use mxnet.mod.Module instead.
INFO:root:Start training with [gpu(0)]
INFO:root:Epoch[0] Batch [100]  Speed: 1418.34 samples/sec      Train-accuracy=0.116719
INFO:root:Epoch[0] Batch [200]  Speed: 1402.49 samples/sec      Train-accuracy=0.101875
INFO:root:Epoch[0] Batch [300]  Speed: 1404.02 samples/sec      Train-accuracy=0.092188
INFO:root:Epoch[0] Batch [400]  Speed: 1404.78 samples/sec      Train-accuracy=0.097656
INFO:root:Epoch[0] Batch [500]  Speed: 1400.95 samples/sec      Train-accuracy=0.107188
I run again, without touch any file or reboot. even in same cmd promot:
D:\mxnet\example\image-classification>python train_mnist.py --gpus 0 --network lenet
INFO:root:start with arguments Namespace(batch_size=64, disp_batches=100, gpus='0', kv_store='device', load_epoch=None, lr=0.1, lr_factor=0.1, lr_step_epochs='10', model_prefix=None, mom=0.9, network='lenet', num_classes=10, num_epochs=20, num_examples=60000, num_layers=None, optimizer='sgd', test_io=0, top_k=0, wd=0.0001)
WARNING:root:[Deprecation Warning] mxnet.model.FeedForward has been deprecated. Please use mxnet.mod.Module instead.
INFO:root:Start training with [gpu(0)]
INFO:root:Epoch[0] Batch [100]  Speed: 1418.85 samples/sec      Train-accuracy=0.823750
INFO:root:Epoch[0] Batch [200]  Speed: 1407.11 samples/sec      Train-accuracy=0.915937
INFO:root:Epoch[0] Batch [300]  Speed: 1405.28 samples/sec      Train-accuracy=0.939063
INFO:root:Epoch[0] Batch [400]  Speed: 1405.67 samples/sec      Train-accuracy=0.948281
INFO:root:Epoch[0] Batch [500]  Speed: 1390.74 samples/sec      Train-accuracy=0.953594
INFO:root:Epoch[0] Batch [600]  Speed: 1344.73 samples/sec      Train-accuracy=0.957031
INFO:root:Epoch[0] Batch [700]  Speed: 1345.86 samples/sec      Train-accuracy=0.955781
INFO:root:Epoch[0] Batch [800]  Speed: 1380.60 samples/sec      Train-accuracy=0.960781
INFO:root:Epoch[0] Batch [900]  Speed: 1350.45 samples/sec      Train-accuracy=0.962344
I bet this is due to some error related to initialization of weights. if i got another repro, i will try to debug.

This issue is closed due to lack of activity in the last 90 days. Feel free to reopen if this is still an active issue. Thanks!


Parallelism problem with the E2E test execution accessing the different resources

I've delved into the issue a bit #28, and it appears to be a parallelism problem stemming from all test cases utilizing the same binary files. If memory serves me right, I encountered a similar issue with FullTeaching, and I resolved it by employing distinct folders.
+ /home/jenkins/workspace/ugins-maven-javadoc-plugin-3.6.3/retorchfiles/scripts/tjoblifecycles/tjob-textexecution.sh tjobeshopa 0 5028 'CatalogTests#FilterProductsByBrandType,LoggedUserTest#loginTest,OrderTests#testCancelOrder'
The HOST_IP is: <IP_ADDRESS>, and PORT 5028 for the TJOB tjobeshopa
[INFO] Scanning for projects...
[INFO]
[INFO] ----------< io.github.giis-uniovi:retorch-st-eShopContainers >----------
[INFO] Building retorch-st-eShopContainers 1.0.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ retorch-st-eShopContainers ---
[INFO] skip non-existing resourceDirectory /home/jenkins/workspace/ugins-maven-javadoc-plugin-3.6.3/src/main/resources
[INFO]
[INFO] --- compiler:3.11.0:compile (default-compile) @ retorch-st-eShopContainers ---
[INFO] No sources to compile
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ retorch-st-eShopContainers ---
[INFO] Copying 2 resources from src/test/resources to target/test-classes
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  3.694 s
[INFO] Finished at: 2023-12-09T00:44:40+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.3.1:testResources (default-testResources) on project retorch-st-eShopContainers: copying /home/jenkins/workspace/ugins-maven-javadoc-plugin-3.6.3/src/test/resources/test.properties to /home/jenkins/workspace/ugins-maven-javadoc-plugin-3.6.3/target/test-classes/test.properties failed with FileAlreadyExistsException -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Some necessary improvements include:

Implement the usage of different source folders to prevent conflicts of this nature.
Save the logs of each TJob into separate files. In this particular execution, I'm unable to retrieve the logs due to the this issue.

@augustocristian Is there any update on this?

Close this issue