Using Nvidia CUDA with repo2docker

Related to https://github.com/jupyterhub/team-compass/issues/52 and https://github.com/jupyterhub/team-compass/issues/85 and the fork from @spMohanty that includes CUDA in the base image
@choldgraf was curious about how we could vary the base image in r2d. @minrk thought we could create an additional configuration file to specify the base image eg runtime.txt we should probably require ubuntu as the OS?
This was mentioned in this month's meeting

Should we re-title this issue to "Uses-cases for changing the base image"? Otherwise this will become a mixture of "how to do GPU support" and "base images for X and Y and Z".

@jzf2101 : Good to seem some momentum around this.
repo2docker with GPU support is something we have been actively using on crowdAI. We managed to collect about 800 repositories (and numerous tags in each of the repositories), amounting to ~1TB of code, that are repo2docker (the crowdai-fork) compatible , for two of the NIPS competitions this year that I was co-organizing. And we have a few more challenges on crowdAI coming up which will aggresively use the same setup.
The way we do it, is very hacky, and non-standard. We simply change the base image to nvidia/cuda:9.0-cudnn7-runtime-ubuntu16.04. Which ties all submissions to cuda-9.0, and cudnn7, while in many cases, users would want to choose which cuda version and cudnn version they want to build against.
Unfortunately, given the licensing situations with cuda and cudnn, the only way we can have cuda/cudnn in built images, if we build on top of the official base images released by nvidia : https://hub.docker.com/r/nvidia/cuda/tags/
All of them are built on top of Ubuntu16.04/18.04.
We have seen some weird behaviour in terms of reproducibility with building dockerimages with these images as the base image, as for some reason, they abruptly update the old tags, sometimes breaking production code, or some weird dependencies. Wish they versioned the images better. But this would be an important consideration from repo2dockers point of view.
So ideally, in context of GPU support, the configurations that we would need inruntime.txt would be the cuda version, and the cudnn version, when present, the buildpack should try to build a GPU compatible image (with an appropriate message ofcourse).
And we would really love to see this released as a part of repo2docker. our fork is already 324 commits behind, and as a small team focused on quite a few things, it might not be possible to keep catching up with repo2docker all the time :D
But as I mentioned, its great to see some momentum around GPU support here, and would love to see this shipped with repo2docker soon :D

The Dockerfile shown in https://github.com/jupyterhub/zero-to-jupyterhub-k8s/issues/994#issue-373992464 suggests that we do not need to switch base images to be able to support running on  a Nvidia GPU. Is there something missing from the linked Dockerfile (it has only been tested on GKE)?
If that Dockerfile also works on a local machine (maybe with nvidia-docker) then this isn't a use case that motivates adding the ability to switch base images.

@betatim : I quickly skimmed through it, but I think I understand whats hapenning with it.
The k8 cluster on GKE, which I think is still in beta mode, mounts the cuda drivers from the host nodes. GKE ensures that the cuda drivers are all accessible at a particular location (usually /usr/local/nvidia) on the host machine, so that the k8 pods can mount them on the go.
But thats something, which would be hard for us  (and many other repo2docker users) to use in the long run, because there you basically pin a whole cluster to a particular cuda version. And this might have many unintended side effects, for instance the tensorflow-gpu pip packages was not supported with cuda-9.1, and users might want to explicitly choose the cuda and cudnn versions they want to run their code against, which might have huge impacts on the performance of their code.
Hence, in an ideal case, we would want each repository to specify its own cuda/cudnn versions, and build off from a base image which have the necessary drivers in the image itself.
And that Dockerfile will not work on local machine, unless you ensure that cuda is present at the correct location, and is mounted onto the container during runtime at the appropriate location(s).

@betatim suggests we may need to call:

repo2docker --no-run https://myrepo.com
will build an image from the repo
that you can then run with nvidia-docker run <name-of-image-here>

@jzf2101 : Ahh ! well nvidia-docker will install the basic nvidia drivers depending on the gpu on the host machine, but not the full cuda toolkit. The nvidia drivers enable the container to "see" the GPUs on the host machine, but they still expect a CUDA capable container to be able to use the GPUs in any sensible way.
And the licensing of CUDA (and cudnn, etc), makes it very non trivial to efficiently package them into prebuilt images, hence most people just start off the official nvidia/cuda base images. Installing cuda through scripts etc, would require you to expressly agree to the terms of usage by nvidia during the installation, which cannot be abstracted away in automated build processes (at least in a legal way).

So you're not calling the nvidia runtime when you run crowdai-repo2docker?

@jzf2101 : No, when we use crowdai-repo2docker, then we do not even need nvidia-docker, the goal there is to build a CUDA capable docker image.
When we run the docker image, then we need nvidia-docker if running locally, or the nvidia-gpu-device-plugin daemonset on a k8 cluster (while requesting for nvidia/gpu:1).

FWIW I think we should provide documentation on how to also avoid swapping the image in the case that you DO want CDA drivers fwiw.

To use your GPU there are two things that need doing: install various things inside your container and install various things on the host.
repo2docker can't help you with installing things on the host. In the comment I linked to above they are running on GKE and use some GKE specific way to install things on all the nodes in the cluster. I'd expect users to take care of all that "somehow".
The next question is how do we get stuff into the docker image. This is where a lot of people use the Nvidia image as a base image.
Taking a look at the Dockerfile for nvidia/cuda it appears to install some apt packages and set some environment variables. What I'd like to confirm is if  the Dockerfile in this comment does the same as the Nvidia Dockerfile does. My guess is the answer is yes, but I don't have access to a GPU to check.
# For the latest tag, see: https://hub.docker.com/r/jupyter/datascience-notebook/tags/
FROM jupyter/datascience-notebook:f2889d7ae7d6

# GPU powered ML
# ----------------------------------------
RUN conda install -c conda-forge --yes --quiet \
    tensorflow-gpu \
    cudatoolkit=9.0 && \
    conda clean -tipsy && \
    fix-permissions $CONDA_DIR && \
    fix-permissions /home/$NB_USER

# Allow drivers installed by the nvidia-driver-installer to be located
ENV LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/nvidia/lib64
# Also, utilities like `nvidia-smi` are installed here
ENV PATH=${PATH}:/usr/local/nvidia/bin

Instead of installing apt packages it uses conda-forge packages. This is easier for us to do in repo2docker as we currently do not support adding extra PPAs before installing user specified apt packages.
This would give the user control over which version of CUDA to install inside the image no? It would have to match the drivers installed on the host but that is also the case when you use the Nvidia provided image.

@parente can you confirm?  @betatim https://github.com/jupyter/docker-stacks/issues/745 discusses a lot of the things you have observed

For the neurips.mybinder.org deployment (Binderhub with GPUs) we managed to deploy several repos without making any change to repo2docker. Instead we installed a few extra libraries via existing mechanisms.
This made me think: can we support the CUDA use-case via a build pack that installs additional packages and sets environment variables? We'd need a way to trigger the build pack.

I think we can use the neurips blog post to structure the stuff that we did in a way that'll make it easier to decide what changes could be made to repo2docker (e.g. a buildpack) to accomodate this. One of the reasons I'd like to get that post ready sooner than later is so that we don't forget all the stuff we did to make it work :-)

Hey all, I'm interested in using repo2docker to build a cuda enabled image from an existing github repo. Any updates on this issue?

You can build GPU enabled images with repo2docker by installing the cudatoolkit conda package. Check out the demo at https://github.com/jzf2101/GAN_tutorial/tree/gpu-binder which we used at NeurIPS. No changes to repo2docker were required.

Ah awesome, thanks @betatim

You can build GPU enabled images with repo2docker by installing the cudatoolkit conda package. Check out the demo at https://github.com/jzf2101/GAN_tutorial/tree/gpu-binder which we used at NeurIPS. No changes to repo2docker were required.

This solution doesn't work for me. For my pods to work with pytorch they need the cuda compatibility packages installed from Nvidia's private package source.
My solutions are changing the base image of repo2docker
Or somehow add additional sources (as in the Nvidia cuda docker image)
Or run a postbuildadmin script (doesn't exist yet?) inrepo2docker
Or use the crowdai repo2docker image instead of repo2docker (currently doesn't work)
Or use a Dockerfile and circumvent most of repo2docker (less desirable)
Do you guys have anymore ideas?

To use your GPU there are two things that need doing: install various things inside your container and install various things on the host.
repo2docker can't help you with installing things on the host. In the comment I linked to above they are running on GKE and use some GKE specific way to install things on all the nodes in the cluster. I'd expect users to take care of all that "somehow".
The next question is how do we get stuff into the docker image. This is where a lot of people use the Nvidia image as a base image.
Taking a look at the Dockerfile for nvidia/cuda it appears to install some apt packages and set some environment variables. What I'd like to confirm is if the Dockerfile in this comment does the same as the Nvidia Dockerfile does. My guess is the answer is yes, but I don't have access to a GPU to check.
# For the latest tag, see: https://hub.docker.com/r/jupyter/datascience-notebook/tags/
FROM jupyter/datascience-notebook:f2889d7ae7d6

# GPU powered ML
# ----------------------------------------
RUN conda install -c conda-forge --yes --quiet \
    tensorflow-gpu \
    cudatoolkit=9.0 && \
    conda clean -tipsy && \
    fix-permissions $CONDA_DIR && \
    fix-permissions /home/$NB_USER

# Allow drivers installed by the nvidia-driver-installer to be located
ENV LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/nvidia/lib64
# Also, utilities like `nvidia-smi` are installed here
ENV PATH=${PATH}:/usr/local/nvidia/bin

Instead of installing apt packages it uses conda-forge packages. This is easier for us to do in repo2docker as we currently do not support adding extra PPAs before installing user specified apt packages.
This would give the user control over which version of CUDA to install inside the image no? It would have to match the drivers installed on the host but that is also the case when you use the Nvidia provided image.

@betatim I have a gpu enabled binderhub I'm testing on, I could give you access if you would like to play? Running this Dockerfile now.

`FROM tensorflow/tensorflow:latest-gpu-jupyter
--- Jupyter
install the notebook package
RUN pip install --no-cache --upgrade pip &&
pip install --no-cache notebook
create user with a home directory
ARG NB_USER
ARG NB_UID
ENV USER ${NB_USER}
ENV HOME /home/${NB_USER}
RUN adduser --disabled-password
--gecos "Default user"
--uid ${NB_UID}
${NB_USER}
WORKDIR ${HOME}
USER ${USER}
RUN conda install pip --yes
COPY . .
RUN pip install --no-cache-dir -r requirements.txt`
Also to note that this does work

@betatim
Would you be interested in testing your solution on a gpu enabled binderhub?

I ran into similar questions those days, I'll try to contribute to this discussion.

Should we re-title this issue to "Uses-cases for changing the base image"? Otherwise this will become a mixture of "how to do GPU support" and "base images for X and Y and Z".

It looks like there are multiple subjects:

host configuration
spawner configuration
image definition

1. Host configuration
The host configuration is what is installed bare metal. Depending on the use case, you may have to handle that configuration by yourself or even be unable to change anything on the host depending on what solution you're using:

infrastructure/platform as a service from a cloud provider

k8s GPU nodes may be pre-configured or need some extra action (for example https://cloud.google.com/kubernetes-engine/docs/how-to/gpus#installing_drivers)
depends on the provider

bare metal servers, VMs, k8s clusters or local instances: everything as to be installed

What has to be installed on the host is basically:

nvidia drivers
nvidia-container-runtime or similar (see documentation for k8s)

2. Spawner configuration
To keep JHub terminology, let's call a spawner the engine/tool used to run a container from an image. It has to give access to the host GPU(s) resource(s).
repo2docker offers capability to run images using docker engine. By default, docker is not exposing GPU devices to the containers (docker run ubuntu:focal nvidia-smi won't work on a host having a GPU device)

using docker CLI, one can pass --gpus extra option as documented

without passing --gpus <value>, NVIDIA_VISIBLE_DEVICES env var is unset/empty making nvidia-container-runtime working as runc. See nvidia-container-runtime documentation
alternatively, nvidia images take care of setting this, but also provide CUDA libraries (see the next section).

using repo2docker CLI, I just created a PR to implement it (using docker Python API) https://github.com/jupyterhub/repo2docker/pull/1079

For k8s use cases, it probably has to be handled on downstream repos such as jupyterhub or binderhub with:

respect to specific use cases
they may require some feature upstream (in this repo)
an effort to find common/shared solutions to the same issues, re-use code, etc.

3. Image definition
Images need to be capable of running code (e.g. from a notebook) on GPU device(s).
Depending on the library used (pytorch, tensorflow, etc.), you may need extra dependencies to be installed to be able to execute your code. Those deps should be pinned by your project (or your dependencies), and installed by the package manager associated to your manifest/spec file (selecting the appropriate BuildPack) at build time.
CUDA particular case
Available libraries
Depending on which API level is required (driver, runtime or CUDA libs), you may need various extra packages to be installed:

it's pretty well explained in this thread, to summarize:

driver API is the low-level API, available from CUDA drivers libcuda
runtime API is the high-level API, available from CUDA runtime libcudart
CUDA libraries such as cuFTT/cuBLAS/etc., available from cudatoolkit

for example, pytorch installed with conda needs cudatoolkit to run on GPU

it's up to the package manager to install it

Base image
This final image will be pretty similar (but surely not equivalent) using:

nvidia base image then installing dependencies from a manifest/spec files for a given BuildPack

note: it will also have an impact on the spawner, as mentioned previously, because those base images are already setting the right OCI env vars

using that manifest/BuildPack to install both (providing the required tool chain is packaged with it)

Even when using a minimal base image, relying on cudatoolkit (which is pretty extensive) leads to large images.
In term of resources consumption, it would be tempting to think that re-using same layers through nvidia base images is more efficient than storing a lot of huge layers for every package manager installation.

what if we did the following:

Added a new parameter to repo2docker, something like --base-image that for now would accept something like org/image:tag.
This image, if provided, would replace what's in the buildpack-deps image: https://github.com/jupyter/repo2docker/blob/master/repo2docker/buildpacks/base.py#L13
We provide a warning if this is provided that says something likeWarning: Alternate base image provided. Reproducibility or compatibility with repo2docker build-packs is not guaranteed. repo2docker will only work with base images running <LIST-OF-REQUIRED-THINGS>.

Add a documentation page that more explicitly lays out the requirements in the base image.

What do folks think?

cc @choldgraf
If you are using conda eco-system/BuildPack upon nvidia base image it would result in having the same large CUDA libraries installed twice:

at system level (using apt) through the base image
at env level (using conda/mamba)

To actually save space re-using the same layers from nvidia base images, it would probably need more/too complex/risky strategies:

install from sources and/or install some packages/libraries without theirs dependencies to rely on system ones
have different base images for each BuildPack, with cuda libs already installed in base environment
etc.

But then, AFAIK it would also mean doing crazy things like:

interpreting a spec file to select base image (e.g. cudatoolkit found as a conda requirement translate into selecting some nvidia/cuda-jupyter:runtime image)
splitting the recipe to prevent installing libs twice
only first level dependencies could be handled
etc.

Maybe some of you found much more efficient strategies!

I have a working example of installing cuda all the requisite libraries using just the conda buildpack at
GitHub.com/ctr26/ZeroCostDl4Mic
My gpu k8s is currently down but it works locally atm.

FWIW I manually replace this line in the template with
FROM nvidia/cuda:11.2.0-cudnn8-runtime-ubuntu18.04

Or whatever cuda/cudnn combination is installed on the destination host.  We have to add a couple apt installs using that image instead of the default bionic but once that's done all I have to do is toggle that line to build with or without GPU.
It would make sense that some sort of enable_gpu+ cuda_version parameters be used to implement that toggling directly in base.py using the nvidia base images of the Ubuntu version repo2docker is using, for example.  Would that be too simplistic ?

https://github.com/ctr26/basic-gpu-binder

This works for me on my k8s binderhub deployment

On 9 Dec 2021, at 20:30, Yves Moisan @.***> wrote:

FWIW I manually replace this line in the template with
FROM nvidia/cuda:11.2.0-cudnn8-runtime-ubuntu18.04
Or whatever cuda/cudnn combination is installed on the destination host. We have to add a couple apt installs using that image instead of the default bionic but once that's done all I have to do is toggle that line to build with or without GPU.
It would make sense that some sort of enable_gpu+ cuda_version parameters be used to implement that toggling directly in base.py using the nvidia base images of the Ubuntu version repo2docker is using, for example. Would that be too simplistic ?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.


Add support for handful of HTML tags

Implemented <mark>, <sup>, <u> and <q> HTML tags.

What is the solution to use underlines when using Redcarpet after Reverse Markdown? I use Reverse Markdown to store WYSWYG into database and then Redcarpet to show the content to a web page.


I checked and it seems like there is nothing specific to iOS 8.3 and up that this library needs. Can we lower the minimum deployment target to iOS 8.0 instead?

I guess that should not be a problem. Can you send a PR?

Sure, no problem!


No dependencies listed in package.json

This package would not work with other installed modules (which were working fine) in XCode unless I specifically cd'd into node_modules/react-native-device-uuid/, ran npm install invariant --save to add the invariant dependency to the package.json, and changed the first line of node_modules/react-native-device-uuid/DeviceUUID.ios.js from var RCTDeviceUUID = require('NativeModules').DeviceUUID; to var RCTDeviceUUID = require('react-native').NativeModules.DeviceUUID;.

Ah, just realized that the last commit has the changes I mentioned above. I installed this with npm (as opposed to git cloning), so I'm assuming the last commit has not yet been published to npm?

You're right. I haven't had time to publish the latest version to npm. I'll
do it soon, probably on this weekend.
Thanks for reminding!
Esther S. Weon<EMAIL_ADDRESS>2015年11月19日 週四，上午6:31寫道：

Ah, just realized that the last commit has the changes I mentioned above.
I installed this with npm (as opposed to git cloning), so I'm assuming the
last commit has not yet been published to npm?
—
Reply to this email directly or view it on GitHub
https://github.com/lazywei/react-native-device-uuid/issues/9#issuecomment-157886095
.

--
Chih-Wei (Bert)

@esthersweon, I have published the latest commit as v1.2.0. Please check if it solves your issue. And always feel free to reopen this issue if it doens't solve your issue.
Thanks for reminding :smile:


Add two expressions to expression reference page

Add conditional and emptyStringMap to expression reference page.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: bianpengyuan
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: sdake
Assign the PR to them by writing /assign @sdake in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

content/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@bianpengyuan can you please force push a rebase.  This will trigger a rerun of circleci which had a partial outage.
Cheers
-steve

@douglas-reid I cannot add reviewers. ptal?

@sdake Mind to help merge this?


Extract retina atlas changes back into their own PR

The retina atlas has been approved and updated in CELLxGENE.
We should be able to release it after one final review!
Please move the retina information to its own PR and I will deploy this branch to dev for approval.
(the organoid atlases may need more time to complete QC of the CELLxGENE collection  so those will come later, separately)

@frano-m can you update the retina text as per this document please.
https://docs.google.com/document/d/1XRH3Tet8R0_x0mOSjp9kIX0CQRMMqYhM/edit
Thanks!
D


Add release notes for v0.3.1 to changelog

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
144

Relevant Lines:
144

💛  - Coveralls


Usage

Hi and thank for such useful tool.
But as a noob, i still have questions how to use it.
Where it should be run?
a. From Home Assistant
b. From a PC with Python installed?
c. From specific Docker/LXC container?
Could you please give a bit more details about how-to use it?

Hi! Thanks for your kind words :).
As the project connects to the Mosquitto broker, it can be executed anywhere that can connects to the broker. The easiest approach is executing it in the same machine as the Mosquitto broker.
If you have more questions, maybe you can explain here your setup and try to figure out what works the best for you :)

Hi! Thanks for your kind words :).
As the project connects to the Mosquitto broker, it can be executed anywhere that can connects to the broker. The easiest approach is executing it in the same machine as the Mosquitto broker.
If you have more questions, maybe you can explain here your setup and try to figure out what works the best for you :)

Thanks for your reply.
I have an MQTT broker running in LXC container.
In additional i have LXC container with Zigbee2MQTT.
On same host also running VM with HA OS.
There also PC running Windows on same network, with Python installed.
So I'd like to run your tool on a PC to generate a file, afterwards it will be copied to HA OS machine with SmartIR installed.

according to my previous answer, is it possible to run this tool on windows PC with python installed?


Equations command modification

This PR modifies the equation command to automatically generate the corresponding law with variables over Fin n (in addition to the law with variables over Nat that is already being generated).
This is used to show that the automatically generated laws are equivalent to the original equation in terms of satisfiability.
Additionally, this PR introduces an environment extension to store all the equational laws, which can be used to export laws as described in #142.
This PR also modifies a result transporting laws from Fin n to Nat to omit the n != 0 hypothesis.

this looks good to me!

Thank you very much @0art0!
Please make sure the project builds correctly by locally running scripts/run_before_push.sh.

Thanks. I seem to be getting a strange build error in one of the generated files:
✖ [3337/3343] Building equational_theories.Generated.EquationSearch.theorems.Run5
trace: .> LEAN_PATH=././.lake/packages/batteries/.lake/build/lib:././.lake/packages/Qq/.lake/build/lib:././.lake/packages/aesop/.lake/build/lib:././.lake/packages/proofwidgets/.lake/build/lib:././.lake/packages/Cli/.lake/build/lib:././.lake/packages/importGraph/.lake/build/lib:././.lake/packages/LeanSearchClient/.lake/build/lib:././.lake/packages/mathlib/.lake/build/lib:././.lake/packages/checkdecls/.lake/build/lib:././.lake/packages/MD4Lean/.lake/build/lib:././.lake/packages/UnicodeBasic/.lake/build/lib:././.lake/packages/BibtexQuery/.lake/build/lib:././.lake/packages/doc-gen4/.lake/build/lib:././.lake/build/lib LD_LIBRARY_PATH= /home/art/.elan/toolchains/leanprover--lean4---v4.13.0-rc3/bin/lean -Dpp.unicode.fun=true -DautoImplicit=false -DrelaxedAutoImplicit=false ././././equational_theories/Generated/EquationSearch/theorems/Run5.lean -R ./././. -o ././.lake/build/lib/equational_theories/Generated/EquationSearch/theorems/Run5.olean -i ././.lake/build/lib/equational_theories/Generated/EquationSearch/theorems/Run5.ilean -c ././.lake/build/ir/equational_theories/Generated/EquationSearch/theorems/Run5.c --json
error: Lean exited with code 137
Some required builds logged failures:
- equational_theories.Generated.EquationSearch.theorems.Run5

I can't immediately think of a reason why the changes introduced in my code would be causing this, but I'll continue to track down the issue.

Ok @0art0. I will take a look at it later.
Can you please resolve the merge conflicts?

@pitmonticone Thank you so much. I've merged conflicts on both PRs.

@0art0 : Is this PR ready for reviewing? I believe there is another PR that is dependent on this one as well.

Yes, it's ready for reviewing.

awaiting-review

I think the MagmaNat theorems can be added to the blueprint. The EquationalLawConversion file is more of an implementation detail, in my opinion; it was a quick hack on my part to bypass the complicated meta-programming that would otherwise be required to convert maps from Fin n to a curried sequence of quantifiers.

@0art0 : which task does this PR close btw?
About the blueprint: Seems sensible. I'll change this PR to a draft, once the blueprint is ready we can get this merged.

This closes #428. Alright, I can add a note about the conversion between MagmaLaws over Fin n and Nat. I think it would be good for @goens to have a look at it when it's done, since it's the machinery he formalized in #368.

The action didn't timeout, but it still took 53 minutes to run the project. Is this expected?
I'm inclined to merge both PRs (#427 and #429) because they look fine to me (even if the branch of #429 seem to have been created from the branch of #427 which isn't considered best practice), but I may be missing something.
See discussion here https://leanprover.zulipchat.com/#narrow/stream/458659-Equational/topic/Do.20we.20need.20to.20build.20EquationalSearch.20files.3F/near/475682423

Let's wait for the blueprint to be updated

I've added a section to the blueprint.

Great, the cache mechanism worked properly and now it took just a few minutes.

Thanks again @0art0!

The blueprint looks good to me too! (I presume @teorth would have written down the proof as 'trivial', but I think this level of detail doesn't hurt either)


⚠ Spryr has degraded performance

In f7389e9, Spryr (https://spryr.com) experienced degraded performance:

HTTP code: 200
Response time: 18034 ms

Resolved: Spryr performance has improved in 62265b4 after 7 minutes.


refactor!: migrate to typescript

continue #95
Resolve todos and fix some issues, also changed the compiler to tsc because this is how I usually do, @react-native-community/bob was removed.
This is a huge change, before public release, I think we should have a beta version first

the conflict has been resolved

Ironically I just spent most of my "open source" time today reviewing another module's typescript re-write, so I don't have time just this moment - but similar to that one I cannot overstate how happy I am to see this, and how happy I will be to have it in typescript. So thank you
One quick question - the switch from bob to tsc. using bob let's us easily generate mutliple derivative JS targets, commonjs and modules, at the same time. So is kind of useful. Do you feel strongly about raw tsc, or even if bob is unfamiliar to you, would it be okay to leave it and preserve our ability to intermediate between our typescript source and various/possibly-multiple JS transpiles?

@mikehardy The main point I remove bob is I didn't see why we need multiple kinds of js output, and I kind of want to reduce the package size on npm (including not uploading the src folder) to just make it neat, but maybe the whole thing (bob removal and size reduce) should be fully discussed before I go further, I'm a little bit too fast on this without consideration, let me put bob back on this PR, thanks for the reminder.

Just about including source or not: uploading src is a strict requirement for me. The majority of pull requests come from people that just reach into node_modules and just fix something, the PR is an afterthought. If you remove the src and build machinery you lose a huge percentage of contributions in my experience. I do not see uploading a few extra text files in the npmjs pack binary as being any sort of problem, compared to the general problem that in open source, people that actually contribute are far too infrequent...

@mikehardy agree with you uploading a few extra text files isn't a big deal, I've added bob back and this PR I think is ready to merge if no further issue.

great - it's the weekend here in the holidays so review might be a bit delayed but please don't take that as any signal - I'm really excited to get this done. Thanks again for posting it

It will be wonderful if there's a beta version on npm to test it properly, currently I am just pushing it to my repo and yarn add from there.
None is it necessary, just consider it as a nice to have

It will be wonderful if there's a beta version on npm to test it properly, currently I am just pushing it to my repo and yarn add from there.
None is it necessary, just consider it as a nice to have

A strict requirement of the migration for me is that it is buildable from source, from the installed version or from a version that is in git (so people may use a repo or branch or commit ref) then they should be able to change into the node_modules/react-native-fbsdk-next directory and run yarn (or yarn && yarn build or whatever) and it works. If that's okay then it's okay to alpha test at least, you can put out a call for testers on the roadmap issue perhaps? I can test it as well. I'm still behind deadline on one project and now it's the holidays so I haven't had time personally but this is critical for the repository's future I think, so it will get reviewed and merged soon I hope :-), thanks for your patience

I've created a pre-release named v7.0.0-alpha.1 including the recent feature I've made in my repo rnike/react-native-fbsdk#v7.0.0-alpha.1, but what I really want to do is to publish v7.0.0-alpha.1 to npm for a proper testing, @mikehardy any idea how can I achieve that?

I can try after I review it. The holidays mean I'm quite short on time though, apologies

@mikehardy That's ok, this is no hurry. Thank you for still replying during your holidays.

status for me:

I've pulled the branch and run the example app successfully, as a basic criteria for merge - everything is :100:
I want to review the actual changes but may just not have time - so the strategy I will actually use is exactly as @rnike suggests - I'll see about getting a beta version out, then people may test integrate it. We can put a time box around how long we wait prior to full release perhaps? Maybe at the end of the first week of January? In the end people do not have to upgrade and we'll mark it as a major version just in case so it should be okay either way

I'm having a really frustrating time getting the version and tag lined up the way I like.
I have so far been unable to trigger a forced breaking change, and I think I know why but I am out of time for now on holidays to fix it.
The release is officially out there on the beta tag on npmjs.com so anyone may install it. It is still in the v6 series though, which is unwanted. It is released from the beta branch here until merged back to master. beta branch is strictly ahead of master at the moment so that part is easy to manage.
Future steps:

add configuration to semantic release on beta branch so it understands the ! syntax in commit messages for breaking changes (I did this in some other repo, I think react-native-google-ads so can copy from there)
make another ! commit on beta branch and trigger release action on beta branch so beta tag on npmjs.com lines up with a v7 beta release
test integrate beta package with various work projects and my react-native-authentication-example-template to make sure it's good
let it sit until january then merge to master and release there ?

This is now successfully out as v7.0.0-beta.1 on npmjs.com and on the beta branch, got it all working.
Now it can incubate - and this is a call for testers, any interested parties should test this in their projects and report problems before end of first week of January!

Thanks


controllercmd: allow configuring topology detection

In a HyperShift context, the toplogy that a component is expected to adhere to is not the management-cluster-wiide infrastructure, but the tenant-specific value. This commit allows consumers to detect topology in less rigid ways.

name fix needed.
/approve

/lgtm

/lgtm


html query_all test is flaky on DDC / chrome canary

This test occasionally flakes and only on chrome canary:
https://travis-ci.org/dart-lang/sdk/builds/174739421

Oddly, it's got a static error too (at least against the regular SDK):
[error] A value of type 'ElementList<Element>' can't be assigned to a variable of type 'List<CanvasElement>'. (tests/html/queryall_test.dart, line 48, col 33)

This doesn't appear strong mode specific.  @alan-knight do you know if the DDC API is different?

I should note there are also several deprecation hints on the test:
[hint] 'queryAll' is deprecated and shouldn't be used. (/Users/vsm/dart/sdk/tests/html/queryall_test.dart, line 32, col 24)

queryAll is marked deprecated, but "because it's semantics will be changing in the future". I'm not sure what that means, especially because it just forwards to querySelectorAll, which isn't deprecated.
The static error isn't different in the SDK version. I'm not sure what is and isn't allowed. It seems that
List<Sub> foo = new List<Super>();
is statically valid but a runtime error. However
List<Sub> foo = new ElementList<Super>();
is a static error, where
class ElementList<T> extends ListBase<T> {}
I don't know how to see the failure from that link. What happens when it flakes? And if it has a static error how does it run at all?

Ahh, I was looking at the wrong version.  It looks like this test has strong mode errors fixed in the DDC copy:
diff test/codegen/lib/html/queryall_test.dart  ../../tests/html/queryall_test.dart
4a5,7
> library NodeListTest;
> import 'package:unittest/unittest.dart';
> import 'package:unittest/html_config.dart';
7,8d9
< import 'package:expect/minitest.dart';
<
9a11,12
>   useHtmlConfiguration();
>
45c48
<       var all = queryAll('canvas');
---
>       List<CanvasElement> all = queryAll('canvas');
61c64
<       var canvases = all.where((e) => e is CanvasElement);
---
>       Iterable<CanvasElement> canvases = all.where((e) => e is CanvasElement);

So, it seems to just timeout, and only occasionally.
See the log here: https://api.travis-ci.org/jobs/174739424/log.txt?deansi=true
It's not terribly informative though.  I think this covers all the recent errors I see here:
https://travis-ci.org/dart-lang/sdk/builds

The test is simple enough that this seems like it might just be a canary bug.

Shall we skip for now?:
https://codereview.chromium.org/2492903002

this is no longer skipped


E2E: Also dumb objects and logs from Hypershift namespace

Currently, CI artifacts do not include the kube objects and pod logs
from the Hypershift namespace, which makes diagnosing issues where
something in the Hypershift operator went wrong difficult.

Do we want to make it a DumpOption? Or just do it all the time?  My vote is just do it all the time.
Also typo in the commit msg s/dumb/dump/

/lgtm

/hold
sorry missed your comment @sjenning

Do we want to make it a DumpOption? Or just do it all the time? My vote is just do it all the time.

@sjenning the reason I gated it is that dump is a per-cluster cmd and these are the logs for all clusters, WDYT? I am ok either way

/hold cancel

/lgtm


thanks

Expected behavior
Actual behavior
Information about the Issue
Steps to reproduce the behavior

...
...

hey

🌮
No information provided or steps to replicate - Closing 😞


Add RxAnimated

RxAnimated
Project URL
https://github.com/RxSwiftCommunity/RxAnimated
Description
Animated RxCocoa bindings
Why it should be included to awesome-ios (optional)
Checklist

[x] Only one project/change is in this pull request
[x] Addition in chronological order (bottom of category)
[ ] Supports iOS 9 / tvOS 10 or later
[ ] Supports Swift 4 or later
[x] Has a commit from less than 2 years ago
[x] Has a clear README in English

          1 Warning

:warning:
Found 1 link issue, a project collaborator will take care of these, thanks :)

Link issue by awesome_bot

Line
Status
Link

1270
301
https://creativesdk.adobe.com/ redirects tohttps://www.adobe.io/apis/creativecloud/creativesdk.html

  Generated by :no_entry_sign: Danger

thx


Create Dockerfile to install ANCM msi

It could be useful to test the ANCM msi

Closing in favor of what Mike's been working on.


gitlab: 14.6.2 -> 14.6.3

Motivation for this change
https://about.gitlab.com/releases/2022/01/17/gitlab-14-6-3-released/
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Please re-run the update script with ./update.py update-all --rev v14.6.3-ee instead of the commit hash.

Fixed that and deployed to my instance for testing

Please re-run the update script with ./update.py update-all --rev v14.6.3-ee instead of the commit hash.

whoops, thanks for fixing it :3


Install on Windows

I'm running Jekyll on windows with Ruby and getting the following error:
[!] There was an error parsing Gemfile:
[!] There was an error while loading jekyll-theme-type.gemspec: No such file or directory - git ls-files -z. Bundler cannot continue.
from C:/Users/zheng/Documents/type-theme/jekyll-theme-type.gemspec:13
-------------------------------------------
spec.add_development_dependency "rake", "~> 10.0"

end

# coding: utf-8
-------------------------------------------
. Bundler cannot continue.
from C:/Users/zheng/Documents/type-theme/Gemfile:2
-------------------------------------------
source "https://rubygems.org"

gemspec

source "https://rubygems.org"
-------------------------------------------

I had to update the file to make versions >=


[SDK] Perf improvements with Promise.all

Don't merge this
I'm aware this is a lot of changes. I will probably split it up into smaller PRs so that we have more control
Problem solved
We have a lot of Promise.all that are not calling the inner requests concurrently, rather sequencially.
Changes made

[ ] Public API changes: list the public API changes made if any
[x] Internal API changes: explain the internal logic changes

How to test

[x] Automated tests: link to unit test file
[ ] Manual tests: step by step instructions on how to test

Self note: The following Pormise.all in this file can be improved however it seems too complex dont wanna touch that rn

[ ] packages\sdk\src\evm\core\classes\drop-erc1155-claim-conditions.ts
[ ] packages\sdk\src\evm\common\claim-conditions\processClaimConditionInputs.ts
[ ] packages\sdk\src\evm\common\any-evm-utils\computeDeploymentInfo.ts

Self note: The following files are to be cleaned up/improved

Most changes are published. Happy to close this


chore(master): release 1.14.23

:robot: I have created a release beep boop
1.14.23 (2023-09-17)
Bug Fixes

deps: update dependency pnpm to v8.7.6 (cbfa244)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/wayofdev/docker-node/releases/tag/v1.14.23 :sunflower:


index.d.ts isn't shipped with 3.3.0

moved to somewhere else?

Shoot. My bad, looks like this happened from #77 which helped flowtype work.
I think we just need the .d.ts file from src to end up in the lib folder. Can you add --copy-files to the babel build step? That should do it.


Update favicon with logo

Replaced the current favicon files with OVN logo equivalents. Deployed and they are working now.


util:RemoveFolderEx

https://www.hass.de/content/wix-how-use-removefolderex-your-xml-scripts
I follow this to remove program.
After install the program with msi, I add some file into the folder.
Then I remove it with msi , the files that I add still exist.
It can not be removed.
Do you know why?
thank

Hello @alicera,
without some sample code it is difficult to say what the problem is.
Remarks at https://wixtoolset.org/documentation/manual/v3/xsd/util/removefolderex.html suggest that:

"for a typical use case of removing a folder during uninstall is to write the directory path to the registry and to load it during uninstall" (using RememberPropertyPattern)

In the sample article you linked APPLICATIONFOLDER was indeed read from registry. Are you doing the same?
Alternatively, if you just want to delete some files and not the entire folder, you can instead use https://wixtoolset.org/documentation/manual/v3/xsd/wix/removefile.html

hi @kurtanr
Thanks for  your reply.
I try install it with msi and create a folder 0 by myself
Then remove the app, the folder can not clear up.
There is a 0 folder exist
Here is the code that I reproduce and I reference this
https://www.hass.de/content/wix-how-use-removefolderex-your-xml-scripts
https://www.codeproject.com/Tips/105638/A-quick-introduction-Create-an-MSI-installer-with
you can get the icon with https://www.codeproject.com/favicon.ico
I create the example.exe (fake).
I build it with the command
"C:/Program Files (x86)/WiX Toolset v3.11/bin/candle.exe" -ext WixUtilExtension example.wxs && "C:/Program Files (x86)/WiX Toolset v3.11/bin/light.exe" -ext WixUtilExtension example.wixobj

example.wxs
<?xml version="1.0"?>
<?define ProductVersion = "0.0.1"?>
<?define ProductUpgradeCode = "12345678-1234-1234-1234-111111111111"?>
<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi"
  xmlns:util="http://schemas.microsoft.com/wix/UtilExtension">
   <Product Id="*" UpgradeCode="$(var.ProductUpgradeCode)"
            Name="Example Product Name" Version="$(var.ProductVersion)" Manufacturer="Example Company Name" Language="1033">
      <Package InstallerVersion="200" Compressed="yes" Comments="Windows Installer Package"/>
      <Media Id="1" Cabinet="product.cab" EmbedCab="yes"/>
      <Icon Id="ProductIcon" SourceFile="favicon.ico"/>
      <Property Id="ARPPRODUCTICON" Value="ProductIcon"/>
      <Property Id="ARPHELPLINK" Value="http://www.exampleproduct.com"/>
      <Property Id="ARPURLINFOABOUT" Value="http://www.examplecompany.com"/>
      <Property Id="ARPNOREPAIR" Value="1"/>
      <Property Id="ARPNOMODIFY" Value="1"/>
      <Upgrade Id="$(var.ProductUpgradeCode)">
         <UpgradeVersion Minimum="$(var.ProductVersion)" OnlyDetect="yes" Property="NEWERVERSIONDETECTED"/>
         <UpgradeVersion Minimum="0.0.0" Maximum="$(var.ProductVersion)" IncludeMinimum="yes" IncludeMaximum="no"
                         Property="OLDERVERSIONBEINGUPGRADED"/>
      </Upgrade>
      <Condition Message="A newer version of this software is already installed.">NOT NEWERVERSIONDETECTED</Condition>

      <Property Id="APPLICATIONFOLDER">
        <RegistrySearch Key="Software\Example Company Name\Example Product Name" Root="HKLM" Type="raw" Id="APPLICATIONFOLDER_REGSEARCH" Name="Path" />
      </Property>

      <Directory Id="TARGETDIR" Name="SourceDir">
         <Directory Id="ProgramFilesFolder">
            <Directory Id="INSTALLDIR" Name="Example">
               <Component Id="ApplicationFiles" Guid="12345678-1234-1234-1234-222222222222">
                  <File Id="ApplicationFile1" Source="example.exe"/>
               </Component>
            </Directory>
         </Directory>

         <Directory Id="ProgramMenuFolder">
            <Directory Id="ProgramMenuSubfolder" Name="Example">
               <Component Id="ApplicationShortcuts" Guid="12345678-1234-1234-1234-333333333333">
                  <Shortcut Id="ApplicationShortcut1" Name="Example Shortcut Name" Description="Example Product Name"
                            Target="[INSTALLDIR]example.exe" WorkingDirectory="INSTALLDIR"/>
                  <RegistryValue Root="HKCU" Key="Software\Example Company Name\Example Product Name"
                            Name="installed" Type="integer" Value="1" KeyPath="yes"/>
                  <RemoveFolder Id="ProgramMenuSubfolder" On="uninstall"/>
               </Component>
            <Component Id="CleanupMainApplicationFolder" Guid="0ffde969-e17c-4708-a7fd-3e0dfacab6b0">
                <RegistryValue Root="HKCU" Key="Software\Example Company Name\Example Product Name" Name="Path" Type="string" Value="[APPLICATIONFOLDER]" KeyPath="yes" />
                <!-- We need to use APPLICATIONFOLDER variable here or RemoveFolderEx
                        will not remove on "install". -->
                <util:RemoveFolderEx On="uninstall" Property="APPLICATIONFOLDER" />
            </Component>
            </Directory>
         </Directory>
      </Directory>

      <InstallExecuteSequence>
         <RemoveExistingProducts After="InstallValidate"/>
      </InstallExecuteSequence>

      <Feature Id="DefaultFeature" Level="1">
         <ComponentRef Id="ApplicationFiles"/>
         <ComponentRef Id="ApplicationShortcuts"/>
           <ComponentRef Id="CleanupMainApplicationFolder" />

      </Feature>
   </Product>
</Wix>

install it with msi and create a folder 0 by myself

_
uninstall it

0 exist , not remove

Hello @alicera,
one thing I've noticed is that RegistrySearch targets Root="HKLM" and in RegistryValue you are targeting Root="HKCU".
Also, your code does not really follow the example from https://www.hass.de/content/wix-how-use-removefolderex-your-xml-scripts where XML structure is quite different (e.g. RemoveFolderEx is under the DirectoryRef component).
I also suggest you install / uninstall the installer with logging enabled (/L*V) and you should get more information regarding the problem.

Can I use the Root="HKCU" to RegistryValue ( installed path )  not  Root="HKLM" ?
https://www.hass.de/content/wix-how-use-removefolderex-your-xml-scripts  the sample have some problem, (1) The "APPLICATIONFOLDER" of `    <util:RemoveFolderEx On="install" Property="APPLICATIONFOLDER" /> is reference from A or B ?

A :
B:

A <Property Id="APPLICATIONFOLDER">

B <DirectoryRef Id="APPLICATIONFOLDER">

Hello @alicera,

Both HKCU and HKLM are fine - the important thing is that you use the same one both in RegistrySearch and in RegistryValue.
Don't think the linked example is wrong - I saw similar code samples in other places also.

Anyway, since this issue is not really related to any of the examples / articles in this repository, I am closing the issue.

Thanks your help
Do you meet the status?
When I upgrade my program with wix, I found the   <util:RemoveFolderEx On="uninstall" Property="INSTALLDIR" />  will remove the folder.
I expect the uninstall step not the upgrade.
Do you have any idea?

Hello @alicera,
during major upgrade previous version of the application if first uninstalled and then the new version is installed ('Schedule' property has some more information about that). This is why folders are deleted on upgrade.

I reference the major upgrade.
I set Schedule as afterInstallInitialize.
At the same condition, I try to use
<RemoveFile Id="FILE" Directory="log" Name ="*.*" On ="uninstall"/>
<RemoveFolder Id="FOLDER" Directory="log" On="uninstall"/>

and
<util:RemoveFolderEx On="uninstall" Property="INSTALLDIR"/>

The util:RemoveFolderEx will be trigger  when  uninstall and upgrade.
The  RemoveFolder and RemoveFile will be trigger when uninstall.


Adding 'E' marker to active enterprise context in 'pachctl config list context'

This PR is a response to feature request: https://github.com/pachyderm/pachyderm/issues/6161

On second look, I actually think I also need to clarify some expected behavior around whether we should present the 'E' marker if enterprise isn't activated. Would it be bad or undesired to show an open-source user the 'E*' markers next to their active context?
Internally, when making enterprise calls without an explicit active_enterprise_context in a config, the active_context is used instead. This means that the absence of active_enterprise_context doesn't mean that enterprise isn't being used. I need to look more through the uses of active_enterprise_context to get an idea of whether there may be a simple semantic solution here.

Would it be bad or undesired to show an open-source user the 'E*' markers next to their active context?

My inuition is that it would be better not to, but I see how that introduces extra complexity, and potentially requires pachctl config list context to make an RPC, when I would otherwise expect it to be a client-only command.
My intuition for a UX:

If active_enterprise_context is set, print an E next to it no matter what
Otherwise, make an RPC to get the current enterprise status. Print the list of contexts, and if the RPC succeeds and indicates that enterprise is active, print E next to the active_context. Otherwise, don't print an E. In the specific case that the RPC fails/times out, also print an error (e.g. could not reach cluster) prior to the list of contexts.

Alternatively, if that's too complicated to be worth implementing:

simply print an E if and only if active_enterprise_context is set?

WDYT?

My intuition:

If active_enterprise_context is set, print an E next to it no matter what
Otherwise, make an RPC to get the current enterprise status. Print the list of contexts, and if the RPC succeeds and indicates that enterprise is active, print E* next to the active_context. Otherwise, don't print an E (only *). In the specific case that the RPC fails/times out, also print an error (e.g. could not reach cluster) prior to the list of contexts.

I actually like this approach a lot. To me, it yielding the most accurate results seems to be worth the small extra cost of complexity and RPC.
I think my thinking was too boxed into reflecting the enterprise status with the information stored on the local config.


whether or not the dataset is handled by the dataset_converter to the hdf5 files and is also supplied for the research community?

Hi Sir,
whether or not the dataset is handled by the dataset_converter to the hdf5 files and is also  supplied for the research community?
Or else, there is still few researcher who can use the nilmtk and nilmtk-contrib for research.
And the phenomenon will continue to exist, "few pubilication presenting algorithmic contributions within the field went on to contribute implementations back to the toolkit".

I have converted three dataset by using the dataset_converters python scripts. The preprocessed dataset by me is as follows,
redd.hdf5, ukdale, ampds,
But the dataport and some other datasets cannot be preprocessed by us easily.
I greatly appreciate all your comments, help and suggestions about the dataset.
The successful train and test of REDD snapshot is as the figure shows.

@oneway3124  Can you share the preprocessed redd dataset with me? I am working on a research and I noticed in some work they use redd dataset that preprocessed in some way. I tried to reproduce their results with the redd dataset (that is in nilmtk format) but always fail.
Also, there is a big gap in the data of house 1 main while in that same period there exist data for the aplliances. Similar gaps can be found in the main power in the other houses.

If you want to get the preprocessed redd dataset,  I can give you via the email or pan.baidu.com?
Thanks a lot!
Best Regards,
Wireless Sensor Networks, School of Computer Science, Sichuan University.
Wang Wei(王伟)，  |mobile: +86-159-0810-6107 | email<EMAIL_ADDRESS><EMAIL_ADDRESS>

I'll try to look into this but unfortunately we don't own most of the datasets, we can't just rehost them unless they have a clear license (lots of them don't). I opened https://github.com/nilmtk/nilmtk/issues/909 to track and discuss this in 2021.

But the dataport...

See https://github.com/nilmtk/nilmtk/issues/873 (and https://github.com/Pecan-Street/DataPort-Examples/issues/1 -- no replies so far) -- we probably won't support Dataport in the future, at least not fully. Besides a small selection of CSV that is freely available under some terms, it's a commercial service that changes without notice. It's more fair to ask them to support NILMTK instead. NILMTK is a maintained only by volunteers.
It's not hard to find discussions on how copyright issues are a real threat nowadays. I recommend anyone reading on that before hosting someone else's files.
As a final note:

Or else, there is still few researcher who can use the nilmtk and nilmtk-contrib for research.
And the phenomenon will continue to exist, "few pubilication presenting algorithmic contributions within the field went on to contribute implementations back to the toolkit".

I disagree that's the main reason. Most people that can't grab a single dataset to work won't be able to contribute significant features unless they're on the topic for the long term. The issue is more of a lack of culture of collaboration, which is something severely lacking in the fields like this. Note that NILMTK did and does have collaboration over the years, but that's a tiny set of individual compared to the whole set of users. But I digress...


[5.x branch] Add a possibility to define the search query when adding visualizations

I will do a pull request for it.


fix(numericality): missing configuration argument

the numericality validation is not working for the fluent api as the configuration is not passed through the call stack.

@EisenbergEffect CLA confirmation request (I think I remember searus contributing elsewhere but couldn't find anything yet to confirm)

@searus If you don't mind could you add the config for https://github.com/aurelia/validatejs/blob/master/src/validator.js#L35-L58 and https://github.com/aurelia/validatejs/blob/master/src/validator.js#L75-L78 also?  If not no biggie I can add they just all need the same thing.

CLA Confirmed.

Travelling at the moment but will do it later today

Added them in now


"SyntaxError: not a PNG file" in maps.map() call with 204 response

I'm getting the following traceback (trimmed from pytest run) when pygbif gets a HTTP 204 error back from the GBIF API. According to the documentation, maps.map() returns a GbifMap object and doesn't raise any exceptions.  It appears that the API is returning a 204 when there's no data to return for a query. Unfortunately, there's no upstream documentation on what HTTP status codes are allowed for the mapping API, so I puzzled out that this is probably when there aren't any occurrences of the taxon to show, but I also seemed to get it in possible error cases.
Versions:
Python: 3.8.12 (old, needed for another dependency)
pygbif: 0.6.0
pillow: 8.3.2

The monkey patch I ended up applying to work around #87 also removes the attempt to plot, so the allows me to properly get the "empty" response.


Question: stick position

Hi @SovGVD
I've been combing through your repo trying to get something in JS to use the low-level packet API.
I had a question about this bit in your code:
// TODO... this string manipulation is soooo slow!
var tmp_roll     = "00000000000"+(parseInt(1024 + 660 * this.stickData.roll) & 0x07ff).toString(2); tmp_roll=tmp_roll.substr(-11);
var tmp_pitch    = "00000000000"+(parseInt(1024 + 660 * this.stickData.pitch) & 0x07ff).toString(2); tmp_pitch=tmp_pitch.substr(-11);
var tmp_throttle = "00000000000"+(parseInt(1024 + 660 * this.stickData.throttle) & 0x07ff).toString(2); tmp_throttle=tmp_throttle.substr(-11);
var tmp_yaw      = "00000000000"+(parseInt(1024 + 660 * this.stickData.yaw) & 0x07ff).toString(2); tmp_yaw=tmp_yaw.substr(-11);
//var tmp_boost    = "00000000000"+(parseInt(1024 + 660 * 0)).toString(2);

Why are you adding 11 0's and then substr'ing them right off immediately after?
I realise this is code ported from go & python and whatever else is out there, but I'm a bit confused about the stick position packets and that's the bit I need to get working :)
Cheers

Hi. If I remember it right, stick values is 11 (or 12) bits, so we need to have 11 values all the time to contact it correctly later, that is why we need to have leading zero bites at least the same size as value length. I'm not happy with that strings solution, but final value should be really long and JS INT can't handle it.

That makes sense; working with large numbers in base-2 strings is never fun (and slow).
I'm not sure what platform you were targeting but node 10.4+ has support for BigInt. I'm trying to write some code for a Tessel 2, which is stuck at node 8.x.

That was another issue about it https://github.com/SovGVD/nodetello/issues/1

@SovGVD I commented on that issue, but I was able to replicate the logic using BigInt and the code more or less copied straight over from TelloLib. I hope this helps a bit.

Thank you @ruswerner ! I will try to check that code with my drone and hopefully merge it.


fix: Change association_defaults type from map(string) to any to support tags

Description
var.association_defaults suppose to accept map of tags.
Note: Changing type to any because map(any) does not work with mix type values:
variables.tf:79,1-32: all map elements must have the same type.

Motivation and Context
fixes: https://github.com/terraform-aws-modules/terraform-aws-eks-pod-identity/issues/14
Breaking Changes
no
How Has This Been Tested?

[x] I have updated at least one of the examples/* to demonstrate and validate my change(s)
[x] I have tested and validated these changes using one or more of the provided examples/* projects
[x] I have executed pre-commit run -a on my pull request

This PR is included in version 1.4.1 :tada:


JUnit 4 gets published as compile dependency, i.e. becomes a transitive dependency in projects using mockwebserver

This is not a bug in the functionality of mockwebserver, but a bug in packaging the pom.xml in the JAR (I suppose, haven't used Gradle).
Step to reproduce

Create a (Maven) project and add JUnit 5 as test dependency (or open an existing project that uses JUnit 5) in your IDE of choice.
Add the mockwebserver as a test dependency to the project.
Create a test class, type in "Test" and invoke intellisense.

Expected behavior
Intellisense offers org.junit.jupiter.api.Test from JUnit 5, as that is the testing library specified in the project.
Actual behavior
Intellisense offers both org.junit.Test (JUnit 4) and org.junit.jupiter.api.Test. The former one is the first in the pick list so I need to be very careful when doing this.
Some help
Notice the JUnit compile dependency in the https://repo1.maven.org/maven2/com/squareup/okhttp3/mockwebserver/4.9.3/mockwebserver-4.9.3.pom file.
I suppose the POM is generated by Gradle.

Upgrade to the latest 5.0 alpha. Should be fixed there, as we've created a new MockWebServer artifact with no JUnit dependency.

I just checked the 5.0 POM (https://repo1.maven.org/maven2/com/squareup/okhttp3/mockwebserver/5.0.0-alpha.6/mockwebserver-5.0.0-alpha.6.pom), JUnit is compile dependency too.

This might be an issue with the Gradle POM plugin (or whatever this is):
https://github.com/square/okhttp/blob/master/build.gradle.kts#L213

The dependency is correct. It's a test artifact for JUnit 4. You'll want to use mockwebserver-junit5.

OK, will https://mvnrepository.com/artifact/com.squareup.okhttp3/mockwebserver3-junit5 a try. Thanks.

Yup, that resolves the issue 😄 Thanks.


[FLINK-27013][hive] Hive dialect supports is_distinct_from

What is the purpose of the change
To support is_distinct_from with Hive dialect.
Brief change log
Add the implementation for is_distinct_from in ExprCodeGenerator.scala.
Verifying this change
Added test involving with is_distinct_from in HiveDialectQueryITCase.java#testDistinctFrom
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving):  no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper:  no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? N/A

@flinkbot run azure

@luoyuxia  Thanks for your contribution, LGTM

@flinkbot run azure


ValueError in augmenter 'UnnamedAllChannelsHistogramEqualization'

Hi, so I basically use this function like all the others
    aug = iaa.AllChannelsHistogramEqualization()         img_aug, bbs_aug = aug(image=img, bounding_boxes=bbs)
The input image is a numpy array of dtype float32 and bbs is the boundingbox object supported by the library.
Then I get:
ValueError: Got dtype 'float32' in augmenter 'UnnamedAllChannelsHistogramEqualization' (class 'AllChannelsHistogramEqualization'), which is a forbidden dtype (bool, uint16, uint32, uint64, uint128, uint256, int8, int16, int32, int64, int128, int256, float16, float32, float64, float96, float128, float256).

Hello there!
I was able to reproduce the error and by looking at the documentation it would seem like the function supports only unsigned 8 bit ints unit8
link to the api documentation:
https://imgaug.readthedocs.io/en/latest/source/api_augmenters_contrast.html#imgaug.augmenters.contrast.AllChannelsHistogramEqualization
Additional chunk to reproduce the error

import numpy as np
import imgaug as ia
import imgaug.augmenters as iaa
import cv2

ia.seed(1)

img = ia.quokka(size=(64, 64))

bbs = [ia.augmentables.bbs.BoundingBox(x1=10, y1=20, x2=20, y2=30),
       ia.augmentables.bbs.BoundingBox(x1=25, y1=50, x2=30, y2=60)]
bbs_oi = ia.augmentables.bbs.BoundingBoxesOnImage(bbs, shape=image.shape)

aug = iaa.AllChannelsHistogramEqualization()

img_aug, bbs_aug = aug(image=img, bounding_boxes=bbs)
# Works

img_aug, bbs_aug = aug(image=img.astype(np.float32), bounding_boxes=bbs)
# Traceback (most recent call last):
#   File "<stdin>", line 1, in <module>
#   File "/home/jspaezp/.local/lib/python3.8/site-packages/imgaug/augmenters/meta.py", line 2027, in __call__
#     return self.augment(*args, **kwargs)
#   File "/home/jspaezp/.local/lib/python3.8/site-packages/imgaug/augmenters/meta.py", line 1998, in augment
#     batch_aug = self.augment_batch_(batch, hooks=hooks)
#   File "/home/jspaezp/.local/lib/python3.8/site-packages/imgaug/augmenters/meta.py", line 643, in augment_batch_
#     batch_inaug = self._augment_batch_(
#   File "/home/jspaezp/.local/lib/python3.8/site-packages/imgaug/augmenters/contrast.py", line 1386, in _augment_batch_
#     iadt.allow_only_uint8(images, augmenter=self)
#   File "/home/jspaezp/.local/lib/python3.8/site-packages/imgaug/dtypes.py", line 535, in allow_only_uint8
#     return gate_dtypes_strs(
#   File "/home/jspaezp/.local/lib/python3.8/site-packages/imgaug/dtypes.py", line 349, in gate_dtypes_strs
#     return _gate_dtypes(dtypes, allowed, disallowed, augmenter=augmenter)
#   File "/home/jspaezp/.local/lib/python3.8/site-packages/imgaug/dtypes.py", line 476, in _gate_dtypes
#     raise ValueError(
# ValueError: Got dtype 'float32' in augmenter 'UnnamedAllChannelsHistogramEqualization' (class 'AllChannelsHistogramEqualization'), which is a forbidden dtype (bool, int8, uint64, int16, float16, uint16, float32, uint32, int32, int64, float64, float128).

As a workaround you could try scaling your images to the 0-255 range from what you have and convert them temporarly to unit8. Then, after the augmentation convert back to floats, this would loose some precision but ... might be good enough for your purpose.
If not I think you might need to implement your own augmenter to deal with the ranges you are using.
Best,
Sebastian


text/template: The ability to output an io.Reader

Having the ability to templatize and concatenate multiple streams through a template would be highly useful. One can think of this as a generalization of the functionality provided by MultiReader, TeeReader, etc, except we'd have the ability to store stubs in files, and have more granular control of intermediate text and formatting.
For example, I'd like to be able to write something similar to the following:
r := // some predefined io.Reader...
t, _ := template.New("").Parse(`
<html>
    <head>
        <meta charset="UTF-8"><meta name="viewport" content="width=device-width, maximum-scale=1.0">
        <link href="/static/blog.css" rel="stylesheet">
    </head>
    <body>
        {{ . }}
    </body>
</html>
`)
t.Execute(os.Stdout, r)

Currently, I have a workaround using channels, however the overhead of channel communication doesn't make it worth using.
copyReader := func(c chan<- string, r io.Reader) {
	p := make([]byte, 100)
	for {
		n, err := r.Read(p)
		c <- string(p[:n])
		if err != nil {
			close(c)
			break
		}
	}
}
r := // some predefined io.Reader...
t, _ := template.New("").Parse(`
<html>
    <head>
        <meta charset="UTF-8"><meta name="viewport" content="width=device-width, maximum-scale=1.0">
        <link href="/static/blog.css" rel="stylesheet">
    </head>
    <body>
        {{range .}}{{.}}{{end}}
    </body>
</html>
`)
c := make(chan string)
go copyReader(c, r)
t.Execute(os.Stdout, c)

Why not use a func in the template like ioutil.ReadAll?

ioutil.ReadAll would require me to buffer all of the stream before outputting it. I'm looking for the ability to output parts of the stream, one after the other, like io.Copy.

The ioutil.ReadAll solution is probably fine for most users, so it's okay to close this issue. That being said, having a more traditional looping construct would open up the template package to more stream-like use-cases. Right now, the range directive is restricted to iterable types.


Cut Prometheus 2.32.0-rc.0

Signed-off-by: Julius Volz<EMAIL_ADDRESS>

Hmm, Windows tests failing in a way that I think I've seen before:
--- FAIL: TestQueryLog/rule_queries,_[::1]:50203,_enabled_at_start (6.57s)

Didn't #9664 add exemplar support to the Agent?

Didn't #9664 add exemplar support to the Agent? And if so I think we should add that to the changelog.

Ah, didn't know that, will add.

When we create the GitHub release, I think it would be a good idea to include both changelogs (agent beta, 2.32), as most people will be upgrading from the previous major release, not the beta.

When we create the GitHub release, I think it would be a good idea to include both changelogs (agent beta, 2.32), as most people will be upgrading from the previous major release, not the beta.

That's a good idea, will do. I think the release CI process will automatically  only the rc.0 changelog, but I can edit it afterwards...

Just a question: IIUC, we will merge the changelog from beta and rc into a single 2.32.0 at the end, right?

@codesome Yes, that's the plan!

I'm going to go ahead and merge this as it has one approval and the comments have been addressed.

/prombench v2.31.1

/prombench cancel


Fixed replacement of as on aarch64-darwin when cross-compiling

Motivation for this change
Cross-compiling for iOS currently fails on aarch64-darwin due to this rm failing; the bare file as does not exist because it's prefixed with the target.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Wrong base, closing, will open a new PR


APISnoop Sunburst Graph error related to Only show conformance eligible endpoints function

What did you expect to happen?
When opening APISnoop and selecting the  Only show conformance eligible endpoints option this is expected:

What actually happened?

The image seems to be a mix of Alpha / Beta / Stable without `ineligible endpoint.

The bug was experienced by @hh and @Riaankl

Once changing the view by selecting Stable and then  Only show conformance eligible endpoints it is not possible to repeat the behavior. Possible due to caching page?

Fixed in https://github.com/cncf/apisnoop/pull/511


DetailActivity3 enterTransition and returnTransition Listeners.

Hello,
First of all, thanks for this useful code!
I've found that the listeners set to enterTransition and returnTransition in DetailActivity3#setupEnterAnimations() and DetailActivity3#setupExitAnimations() are both called when DetailActivity3 starts and also when back is pressed.
In fact, returnTransition is called first.
Using your layout no problem were observed, but using a different layout (more complex) or concatenating different effects may cause problems, for example when setting the visibility/invisibility of the child views.
The only solution I've found is to call setupExitAnimations() when enterTransition ends:
        final Transition enterTransition = getWindow().getSharedElementEnterTransition();
        enterTransition.addListener(new Transition.TransitionListener() {
            ...
            @Override
            public void onTransitionEnd(Transition transition) {
                animateRevealShow(bgViewGroup);
                enterTransition.removeListener(this);
                setupExitAnimations();
            }
            ...

@RlagoMan
The solution is easy: remove enterTransition listener on onTransitionEnd callback:
public void onTransitionEnd(Transition transition) {
                transition.removeListener(this);

                animateRevealShow(bgViewGroup);
                enterTransition.removeListener(this);
                setupExitAnimations();
            }

There is a full working example here: https://github.com/lgvalle/Material-Animations/blob/master/app%2Fsrc%2Fmain%2Fjava%2Fcom%2Flgvalle%2Fmaterial_animations%2FRevealActivity.java

Hi,
Thanks for the reply. That's what I've made and it is working.
Best regards!
El 13/9/2015 22:46, "Luis G. Valle"<EMAIL_ADDRESS>escribió:

@RlagoMan https://github.com/RlagoMan
The solution is easy: remove enterTransition listener on onTransitionEnd
callback:
public void onTransitionEnd(Transition transition) {
transition.removeListener(this);
            animateRevealShow(bgViewGroup);
            enterTransition.removeListener(this);
            setupExitAnimations();
        }

There is a full working example here:
https://github.com/lgvalle/Material-Animations/blob/master/app%2Fsrc%2Fmain%2Fjava%2Fcom%2Flgvalle%2Fmaterial_animations%2FRevealActivity.java
—
Reply to this email directly or view it on GitHub
https://github.com/lgvalle/Material-Animations/issues/12#issuecomment-139916165
.


Auto generated css files font problem

Hello,
I try to learn amp how is it work and thank you for this great module!!!
I face below problem when I build a project or run on the local.
"http://localhost:3000/amp:4:3674 The attribute 'href' in tag 'link rel=stylesheet for fonts' is set to the invalid value '/_nuxt/styles.css'."
This is the style.css on the html
<link rel="stylesheet" href="/_nuxt/styles.css">
Is there any idea why can it be?
Thank you

Link to stylesheet does not allow in AMP pages, I think you are using nuxt extractCSS. If so try disabling extractCSS in nuxt.config.js

Yes, you are right @farnabaz this problem is solved when I removed extractCSS. Now I have a different problem but we can close this.
Thanks for prompt response :)

Hi. How i can disable extractCSS only for amp pages?

It works for me. Try:
nuxt.config.js
{
  build: {
    extractCSS: true,
  }

  hooks: {
    // This hook is called before generatic static html files for SPA mode
    'generate:page': ({ page, path }) => {
      if (/^\/amp\//gi.test(path)) {
        page.html = modifyHtml(page.html)
      }
    },
    // This hook is called before rendering the html to the browser
    'render:route': (url, page) => {
      if (/^\/amp\//gi.test(url)) {
        page.html = modifyHtml(page.html)
      }
    }
  }

}

modifyHtml
import { LOCALES } from './config/locales'

const modifyHtml = (html) => {
  const styleConcat = renderSync({ file: 'assets/css/amp.scss', outputStyle: 'compressed' }).css.toString()
  html = html.replace('</head>', `<style amp-custom>${styleConcat}</style></head>`)
  html = html.replace(/<link[^>]*rel="stylesheet" href=".*\.css[^>]*>/gi, '')
  html = html.replace(/<link[^>]*rel="preload" href=".*\.js[^>]*>/gi, '')
  return html
}


Disable netrw

In the terminal if I do $ nvim /path/to/directory/ as the docs suggest, netrw opens instead of lir.
Do I need to configure something special or is a bug?

This did the trick:
vim.g.loaded_netrw = 1
vim.g.loaded_netrwPlugin = 1

Thank you for reporting the problem!
I've just added a FAQ to the doc.


Components don't render when imported as reexport from another package

Describe the bug
So I am trying to setup shadcn in a components package in my monorepo. This components package has been created with npx sv create so its a standard svelte 5 project. I ran pnpm dlx shadcn-svelte@next init to setup shadcn and added a tooltip with pnpm dlx shadcn-svelte@next add tooltip.
I then re-exported this tooltip from the packages `index.ts
export * as Tooltip from './components/ui/tooltip/index.js';

and ran the pnpm package command.
When I try to use the tooltip like so:

<Tooltip.Provider delayDuration={200}>
	<Tooltip.Root>
		<Tooltip.Trigger>{@render children?.()}</Tooltip.Trigger>
		<Tooltip.Content
			class="hidden lg:flex max-w-[10rem] items-center justify-center hyphens-auto break-words"
		>

{title}

		</Tooltip.Content>
	</Tooltip.Root>
</Tooltip.Provider>

the component doesn't get rendered and doesn't even appear in the Dom.
Reproduction
.
Logs
No response
System Info
System:
    OS: Windows 10 10.0.19045
    CPU: (16) x64 AMD Ryzen 7 5800X 8-Core Processor
    Memory: 13.25 GB / 31.93 GB
  Binaries:
    Node: 20.15.0 - C:\Program Files\nodejs\node.EXE
    npm: 10.8.1 - C:\Program Files\nodejs\npm.CMD
    pnpm: 9.4.0 - ~\AppData\Roaming\npm\pnpm.CMD
  Browsers:
    Edge: Chromium (130.0.2849.68)
    Internet Explorer: 11.0.19041.4355

Severity
blocking an upgrade

Did you setup your tailwind config so that your internal component library's css is included?

Did you setup your tailwind config so that your internal component library's css is included?

I am not sure, I basically just followed the instructions. Do you mean the   packages/components/src/app.css needs to be imported into the packages/components/tailwind.config.ts file?

I believe you need to setup the content.
https://tailwindcss.com/docs/content-configuration

I believe you need to setup the content.
https://tailwindcss.com/docs/content-configuration

Yes the shadcn cli set up a tailwind.config.ts file that already includes content: ['./src/**/*.{html,js,svelte,ts}'],

I think you need to setup your content in the project consuming your ui library to point to the ui library

So if your structure is
- ui
- site
  - tailwind.config.ts

You need to setup the tailwind.config.ts to parse all the svelte html and ts files in ui.


🚀 - Add maskitoWithGuide processors

Which package(s) are relevant/related to the feature request?
@maskito/core
Description
See this discussion: https://github.com/Tinkoff/maskito/issues/179
We need to add possibility to add guide.
Proposed solution
import { Maskito, maskitoPipe, maskitoWithGuide } from '@maskito/core';

const { guidePreprocessor, guidePostprocessor } = maskitoWithGuide('dd/mm/yyyy')

const maskedInput = new Maskito(element, {
  // ...
  preprocessor: maskitoPipe(
    guidePreprocessor,
    // ...,

  ),
  postprocessor: maskitoPipe(
    // ...,
    guidePostprocessor,
  ),
  overwriteMode: 'replace', // ONLY THIS
});

Additional tasks:

[ ] Add page With Guide to Recipes section.
[ ] Add Phone with guide example to this new page
[ ] Add Date with guide example to this new page

Let's just return MaskitoOptions from this utility so that overwriteMode is also present and people can write:
const mask = {
  ...maskitoWithGuide(...)
  mask: ...
}

Let's just return MaskitoOptions from this utility so that overwriteMode is also present and people can write:
const mask = {
  ...maskitoWithGuide(...)
  mask: ...
}

The problem of this approach that it is diffucult to extend it with other processors 🤔

But if you need it you can destruct it earlier.


WIP: Send raw tx when signing

Coverage remained the same at 94.753% when pulling 73d764995232ab2349b821fe8c3265053c5ac37a on isocolsky:log-raw-tx-on-sign into 82458dfd1487371a2bf450565ecdea7b37e5da1c on bitpay:master.

Coverage increased (+0.01%) to 94.763% when pulling 39b8cf7f71ae73ab509b2d07f03b22172da59b18 on isocolsky:log-raw-tx-on-sign into 82458dfd1487371a2bf450565ecdea7b37e5da1c on bitpay:master.

No longer needed.


"Builds > List Builds for a Property" demos the wrong endpoint

The DEFINTION, EXAMPLE REQUEST, and EXAMPLE RESPONSE are for GET /libraries/:id/builds.
The required scenario does not exist in api_specfication.json.

There is no property.builds.
There are only library.builds and environment.builds.
This is a typo.
I'll change List Builds for a Property to List Builds for a Library.

https://github.com/Adobe-Marketing-Cloud/reactor-developer-docs/pull/16


Check dropbox for filename conflicts on EBI submit

The way EBI works is by having a single, flat directory where all our files are uploaded.  This means that if we have two studies and there are samples names that are shared between studies (which is common for BLANK samples), the submit will fail because the there will be a filename collision.  Therefore, we should check the dropbox before attempting submission to make sure that there will be no collisions.

I think this issue doesn't apply anymore as we are prepending all samples with study_id. Thus, the only time there is going to be a collision is if we resubmit the study, which is fine. Please reopen if you think this is not solved.


Randomize Unit Tests

Randomize unit tests
Ensure unit tests are independent for randomly running the specs

/test ?

/test 4.12-openshift-e2e

lgtm
but did we actually agree that we want to have this? I remember some discussion in chat... 🤔

but did we actually agree that we want to have this? I remember some discussion in chat... 🤔

IIRC we agreed not to do a repetition, but the concept of randomization was accepted.

/retest

/retest


lpdf for Hidden Markov models with discrete latent states

Description
Consider a hidden Markov model with observations y, a discrete latent state x and model parameters psi. To fit this model in Stan, we need to integrate x out, and then compute gradients for the marginal density p(y | psi). Doing this by hand is lengthy and doesn't guarantee optimal computation of the gradients. Deriving an algorithm for the optimal derivatives is not trivial: it can be done as a byproduct of the EM algorithm or by constructing an adjoint method, a discretized version of what we do with ODEs.
I propose to implement an hmm_marginal_density_lpdf() function with optimal gradient calculations. In addition there will be two complementary functions: hmm_hidden_state_probs() which computes the marginal probabilities of the hidden states and hmm_hidden_state_rng() which simulates these hidden states.
Example
x is discrete and takes values over a finite set with K elements. Suppose we have N observations y, corresponding to N transitions. The user then defines

the K x K transition matrix Gamma, with (i, j)th entry being the probability that x(n) = i given x(n - 1) = j
the K x N observational density omega, with for each observation the probability density p(y | x) over each possible value of x.
The K-vector initial state, rho.

The user then calls
target += hmm_marginal_lpdf(log_omegas, Gamma, rho);

Matrix[K, N + 1] alphas = hmm_hidden_state_prob(log_omegas, Gamma, rho);

int x[N] = hmm_hidden_state_rng(log_omegas, Gamma, rho);

Expected Output
This feature will facilitate and optimize the use of hidden markov models in Stan.
Current Version:
v3.1.0

On Jan 30, 2020, at 4:49 PM, Michael Betancourt<EMAIL_ADDRESS>wrote:

Why is omega on the log scale but not Gamma (stochastic matrix) or rho (initial distribution?).

Because the Stan language exposes log density functions where as Gamma and rho
will be simplices on the linear scale for simpler models.

OK.  That makes sense.  I made the same decision with log_mix, but have been uncomfortable with that ever since since the implementation for log_mix(alpha, lp1, lp2) is just log_sum_exp(log(alpha) + lp1, log1m(alpha) + lp2), so you always wind up taking the logs anyway.  The problem with linear is that it doesn't give you as much dynamic range.  But then that's probably not such a big deal if there are vanishingly small mixture components.

Will there be a way to set the initial distribution to be the stationary distribution of Gamma?

Because Gamma and rho are customizable the user can compute the eigenvectors of Gamma and
set rho accordingly external to the HMM density.

I think eventually it'd be nice to have a version with one fewer argument that does that automatically.  I can do that once the first one's built.

No, it won't be too expensive to check that Gamma is a stochastic matrix and rho is a simplex; that time will be dwarfed by the rest of the computation of the HMM.
rho will be cheap, but for time-dependent Gamma the checks could become substantial. For an
initial implementation, however, we should be safe and worry about optimizations later.

I completely agree we should check.  The plan's to rewrite checks so that they can be turned off wholesale.

I've always seen the rows of Gamma be the simplexes, but it appears that there are two conventions for stochastic matrices in the literature (right stochastic and left stochastic). Conventionally HMMs have used the one where rows are simplexes so that Gamma[i, j] is the probability of transitioning from state i to state j. If you're going to do this the other way you'll have to make it very clear that's the convention. And you'll definitely need tests either way.

We were working off of stats/machine learning textbooks that defined Gamma with simplex columns
mirroring the linear algebraic action of the transition matrix on the previous state probabilities.

I'm not sure which texts you're looking at, but Bishop sets them up with rows being simplexes (p. 610).  The Qin et al. paper on chemistry I really liked also made the rows simplexes.  The Langrock et al. paper "Flexible and practical modeling of animal telemetry data" also sets up the stochastic matrix row-wise (that paper's the basis of the popular moveHMM package in R).  It's also the way I set it up in the manual, although that says more about me than the external world.  Who sets up the transition matrix with columns as simplexes?

I'm fine changing the conventions for the gammas, and having the rows as simplexes.
For the unit test for the log density evaluation, I'll construct a simple case where there are only two states. It's straightforward to do this calculation analytically.

The more general thing to do would be to test 1-state, 2-state, and 3-state examples to make sure all the boundary conditions on the loops work.
Then tests inputs of size 0, 1, 2, and 3.
You can at least test that finite diffs match autodiff without any analytic work.

On Feb 28, 2020, at 10:54 AM, Charles Margossian<EMAIL_ADDRESS>wrote:
I'm fine changing the conventions for the gammas, and having the rows as simplexes.
For the unit test for the log density evaluation, I'll construct a simple case where there are only two states. It's straightforward to do this calculation analytically.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

I agree with Bob. Having the rows as simplexes for the transition probability matrix is definitely how I'd also prefer to have it set up.

Closed in #1778


AttributeError: 'STrack' object has no attribute '_count

Traceback (most recent call last):
File "tools/demo_track.py", line 372, in
main(exp, args)
File "tools/demo_track.py", line 365, in main
imageflow_demo(predictor, vis_folder, current_time, args)
File "tools/demo_track.py", line 263, in imageflow_demo
online_targets = tracker.update(outputs[0], [img_info['height'], img_info['width']], exp.test_size)
File "/content/ByteTrack/yolox/tracker/byte_tracker.py", line 268, in update
track.activate(self.kalman_filter, self.frame_id)
File "/content/ByteTrack/yolox/tracker/byte_tracker.py", line 48, in activate
self.track_id = self.next_id()
File "/content/ByteTrack/yolox/tracker/basetrack.py", line 36, in next_id
self._count += 1
AttributeError: 'STrack' object has no attribute '_count'

Something seems to be wrong with the code...I was able to run the code yesterday. But today this error message popped out. I have tried creating a new conda environment and installed the dependencies again but is facing the same problem. Please help!

For some reason, the inheritance doesn't work on this class,  You can get around it by adding a reference In byte_tracker.py to _count in the STrack.init like so
class STrack(BaseTrack):
    shared_kalman = KalmanFilter()
    def __init__(self, tlwh, score):
        self._count = 0

Hello, may I ask whether this problem has been solved? I have also encountered this problem.

I was able to work around it by git cloning from an older commit (The one on May 13, 2022). Please see this post for your reference. The command to git clone from an older commit can be found at https://stackoverflow.com/questions/3555107/git-clone-particular-version-of-remote-repository.

Thank you very much!

I was able to work around it by git cloning from an older commit (The one on May 13, 2022). Please see this post for your reference. The command to git clone from an older commit can be found at https://stackoverflow.com/questions/3555107/git-clone-particular-version-of-remote-repository.

THX！

you need to replace yolox with the previous version

you need to replace yolox with the previous version
how can i do that？  thx！

You have to add  the super().init in the STrack so it can access the baseclass variables

I think this issue cannot be simply resolved by adding the super().init() method in the STrack class. Doing so seems to lead to issue #219. Checking out at the previous commit seems to be a better option in the current stage. My script works fine at the previous commit 2c082be

I think this issue cannot be simply resolved by adding the super().init() method in the STrack class. Doing so seems to lead to issue #219. Checking out at the previous commit seems to be a better option in the current stage. My script works fine at the previous commit 2c082be

hello I have met the same problem that only single box is output, could you please tell me some details about your solution? since I checkout the commit 2c082be, but it seems can't work, still return _'STrack' object has no attribute 'count'

Hi @Breezewrf the _count variable is not useful..what I did was I created a self.id_count for the bytetracker class and then passed it as an argument to the function call where we creat the new tracks and assigned that self.id_counter to the track_ID and then incremented the variable

Yes @jasons1425 you are right I faced the same issue but I added a new variable for counting the objects so that got resolved.

Hi @Breezewrf the _count variable is not useful..what I did was I created a self.id_count for the bytetracker class and then passed it as an argument to the function call where we creat the new tracks and assigned that self.id_counter to the track_ID and then incremented the variable

Thanks a lot! problem got resolved

Thank you @PranvQT. The problem is likely because each STrack object has its own track_id counting instead of the class-level one, so different tracks for the same ByteTracker object are sharing the same track_id.
To count the number of tracks in each ByteTracker object without relying on the BaseTrack class attributes, I have added an itertools.count() as the STrack counter inside the ByteTracker class. This argument is passed to the BaseTrack class so that whenever the next_id() method is called, the latest track id (count) can be fetched from this generator.

I'm new to bytetrack. Can I see your code to see what changes should be made to get it to run over a video

I'm new to bytetrack. Can I see your code to see what changes should be made to get it to run over a video

Thank you @PranvQT. The problem is likely because each STrack object has its own track_id counting instead of the class-level one, so different tracks for the same ByteTracker object are sharing the same track_id.
To count the number of tracks in each ByteTracker object without relying on the BaseTrack class attributes, I have added an itertools.count() as the STrack counter inside the ByteTracker class. This argument is passed to the BaseTrack class so that whenever the next_id() method is called, the latest track id (count) can be fetched from this generator.

I didn't make many changes honestly. Just added a counter to count the track ids for each ByteTracker. You can find the changes in this pull request.
The modified scripts work fine in my local environment, but I would recommend you to just pull the main branch which has been recently reverted to get rid of this issue :)

Hi Yifu,
Thank you so much for your prompt reply!
I tried to add the line “self._count = 0” to byte_tracker.py as you have suggested. The code can now run, but the algorithm can only detect and track only one person (ID 1) in a scene full of people.
@.***
May I know what other steps I need to take if I want to detect and track all people in the scene?
Thanks again for your kind help!
Sincerely,
Feiyang
From: sr99622 @.>
Date: Sunday, June 26, 2022 at 12:50 AM
To: ifzhang/ByteTrack @.>
Cc: feiyang-hku @.>, Author @.>
Subject: Re: [ifzhang/ByteTrack] AttributeError: 'STrack' object has no attribute '_count (Issue #210)
For some reason, the inheritance doesn't work on this class, You can get around it by adding a reference In byte_tracker.py to _count in the STrack.init like so
class STrack(BaseTrack):
shared_kalman = KalmanFilter()

def __init__(self, tlwh, score):

    self._count = 0

—
Reply to this email directly, view it on GitHubhttps://github.com/ifzhang/ByteTrack/issues/210#issuecomment-1166324063, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALV55KGVV2XY4FECOUBJHODVQ42DJANCNFSM5Z2KSRZA.
You are receiving this because you authored the thread.Message ID: @.***>


Add container-related words

https://docs.docker.com/engine/install/ubuntu/#uninstall-old-versions

Thanks!


Tab.Pane components do not re-render properly when switching without a key prop, when the root components of the panes are equal

Bug Report
Tab.Pane Components do not render their content when the root component of one Tab.Pane is the same as another Tab.Pane's root component, unless the key prop is provided for the tab components. It should be fixed or the documentation and examples should reflect that tab.key prop is required so that it does not become an issue.
Steps
https://codesandbox.io/s/3qjnx5r6xm
Switch between the first and the second tab, and the switch between the first and the third tab, while watching the console output.
Expected Result
Tab panes must re-render their components when switching among them
Actual Result
Tab panes do not re-render content properly when the root components of the Tab.Panes are identical
Version
0.80.1
Testcase
https://codesandbox.io/s/3qjnx5r6xm

I'm having this same issue. docs are of no help. hope you can find a solution.

yup just make sure to add the key prob to your tab panes


Convert the CFG to an on-demand analysis - NFC.

This fixes some TODOs by moving the CFG into the IRContext as an
analysis.

Re-based, re-tested and pushed as 9f20799fb4129f4b1174fb168fb4508f46f64c5b


Change Add Apps to Manage Apps and support translation.

Hmmm. This one has me in a quandary. I actually think that 'Add Apps' is the better description given the now critical role of this in accessing the site and site features/components.
Many people looking at this menu dropdown aren't interested in managing anything. They just want to navigate to somewhere on the site. If the location they want isn't there, they would probably consider adding it.
However, if it is to be "Manage Apps" we probably should do away with the plus symbol. I'd probably wait for @git-marijus for advice. Though I agree it should be translate-able. No problem there.

It's probably not worth over-analyzing. I wanted to capture that it's an "add" and an "edit" button, but "Add/Edit Apps" looked too clunky. I also think "Select Apps" might be okay.

Select works for me and I don't have a problem with that appearing next to a plus sign. I'd still give Mario the final word.

The only reason this link is there is to provide a possibility to "Add" new apps to the tray. Anything else would be completely misleading. I kindly ask you to keep it that way.
When clicking that link at first you can't manage, edit or select anything. You can add or remove apps to the bin. For any other action you have to click through to edit or create.

OK I'm going to pull it so we get the other fix and then change the words back.

Thanks


libqi: 2.1.3-1 in 'indigo/distribution.yaml' [bloom]

Increasing version of package(s) in repository libqi to 2.1.3-1:

upstream repository: https://github.com/aldebaran/libqi.git
release repository: https://github.com/ros-naoqi/libqi-release.git
distro file: indigo/distribution.yaml
bloom version: 0.5.20
previous version for package: null

should be renamed to naoqi-libqi


Add some download integrity.

Hey,
Hopefully this is of use to you. I have no previous experience with Ant and welcome all criticism. Please check that you get the same checksums.

Sorry, I'm working on other projects now and can't afford the time. Feel free to use the commit, though.

Thanks @haphut and @llwp This PR is not longer useful, we have a new building script (gradle) with a proper dependency management