fix to build system to limit number of threads #115

Now uses at most 8 threads.

@Luthaf what do you think about this PR?
Part of me feels that Marcel should have just requested more memory on his HPC node

I think this is worth it: anything that include torch takes a lot of memory to compile, and sometime the front node has limits on how much memory a single user can use. Also, we don't have that many files to compile, so using more than 8 threads won't really help much.
I'm not sure how much RAM compilation with 8 threads would take though, do we know that this is low enough to fix the issue?

I've added two environment variables that modify the pip build process:
see: https://github.com/lab-cosmo/sphericart/issues/115#issuecomment-2225053275

Are we sure it's a good idea to have an undocumented environment variable? I think min(cpu_count, 8) is sufficient and simple enough

I think min(cpu_count, 8) is sufficient

I'm not sure. We barely have 8 files to compile, so if the build fails with make -j it will also fail with make -j8. We could do a lower default limit (2-3) if you prefer, or query available RAM and use min(cpu_count, ram/4GiB) (4GiB is close to what I saw a single compiler process take when building code that #include <torch/...>.

Querying the RAM is probably the proper fix for the original issue, so I will try to implement it

EDIT: it seems that there isn't a reliable way to get the RAM from the Python standard library, we're now doing two jobs


Review of the guide for basic usage of Pandas module

The guide for dynamic systems and its utilization is to be reviewed.

link to source_code:
https://cocalc.com/projects/cdcf3856-3e8b-4d1c-bcb2-5c42ba749d83/files/UCZESTNICY/Guide_Source/Source/[PL][Guide]Podstawy używania biblioteki Pandas oraz DataFrame.ipynb#id=7246f9


Apply Made Tech frontend to productionisation.io

I think @lukemorton has a package that will wrap up all of this stuff, which is near readiness. I might be wrong though.

I copied what luke to for learn.madetech.com

Screenshots? 😄

Craig has followed the static asset route, which is the most appropriate for GitHub Pages unless you want to commit node_modules or Ruby Gems to the repo.

What was the thing you were working on @lukemorton that you mentioned to me?


After deleting a (newly created) namespace it is stuck in the "Terminating state" and wont go away until removing the "kubernetes" finalizer.
What did you expect to see?
Kubernetes properly removing the namespace
What did you see instead? Under which circumstances?
The namespace is stuck in "Terminating" state.
Environment

operator-lifecycle-manager version: 0.14.1

Kubernetes version information:

kubectl version Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.2", GitCommit:"59603c6e503c87169aea6106f57b9f242f64df89", GitTreeState:"clean", BuildDate:"2020-01-23T14:21:54Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.9-eks-c0eccc", GitCommit:"c0eccca51d7500bb03b2f163dd8d534ffeb2f7a2", GitTreeState:"clean", BuildDate:"2019-12-22T23:14:11Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}

Kubernetes cluster kind: 1.14 on AWS EKS

Possible Solution
Got help on the Slack Channel kubectl get apiservice gave
NAME                                   SERVICE               AVAILABLE   AGE ... v1.packages.operators.coreos.com       olm/v1-packages-operators-coreos-com   False (ServiceNotFound)   6d23h ...
After deleting the apiservice, namespaces are properly terminating again.
Additional context

This is a really big problem.

Workaround for now:
https://stackoverflow.com/questions/52369247/namespace-stuck-as-terminating-how-do-i-remove-it/59667608#59667608

@abhinavdahiya thanks, but unfortunately it wouldn't work in an automated pipeline

The issue seems to be that apiservice is still present, after doing kubectl delete apiservices.apiregistration.k8s.io v1.packages.operators.coreos.com namespace is successfully deleted. This resource should be deleted by operator itself. I tried setting ownerReference on apiservice on namespace, but that does not seem to work, as ownerReferences does not seem to work on global resources.
There was issue opened on kubernetes github https://github.com/kubernetes/kubernetes/issues/60807, but this is not really an issue, as apiservice is still depending on namespace and thus finalizer never completes.

If you have sufficiently powerful orchestration for kubernetes, you can split packageserver clusterserviceversion from deployment and make it so it is removed before operator is removed. This is a workaround, and what kubernetes should really do is implement garbage collection (ownerReferences) for global resources.
Example of workoround using pulumi: https://gist.github.com/offlinehacker/856b64ec5ad5ab3829bf01f1fb29958d

FWIW I captured a workaround for this issue and a few others in a quick shell script, manage.sh

Is this still an issue?

NOTE: This is still an issue as of Feb 2024


resin CLI seems to delete itself after trying to perform an update

When i ran "resin apps" the cli said it was performing a minor update, this failed and now when i run resin apps, it says command not found.
here is the console log:
resin-io:resin-ftp hobochild$ resin whoami
shaun
resin-io:resin-ftp hobochild$ resin apps
Performing patch update, hold tight...
Command failed: /bin/sh -c npm install --silent --global resin-cli

resin-io:resin-ftp hobochild$ sudo resin apps
Password:
sudo: resin: command not found
resin-io:resin-ftp hobochild$ resin apps
-bash: /usr/local/bin/resin: No such file or directory
resin-io:resin-ftp hobochild$ which resin
resin-io:resin-ftp hobochild$

This is an issue with NPM. In case of an installation error, they should automatically rollback to the previously installed version, which they don't.
As an improvement, the CLI will no longer perform a self update, and will instead notify the user to run the update command by his own: https://github.com/resin-io/resin-cli/issues/72.

Let's discuss this again if prevent the self update still confuses users. In that case we could add an informative message to resin update explaining the possible problem caused by NPM.


W+Click in Inventory should always put it in the crafting grid

This is already the case in 1.11.2