RFC: Update 'west blob' to support source files with binary data

Introduction
Zephyr supports pulling in binary blobs using 'west blob'. This mechanism supports .bin (and similar files) as well as pre-compiled libraries (i.e. .a files). This proposal extends the blob support to allow for C/C++ files containing only binary data encoded as an array of bytes (or similar mechanisms).
Problem description
Acquiring .bin files and encoding them in C/C++ code is a use case that is already supported by the tooling. However, some silicon vendors have already produced libraries with the binary data in a C/C++ file. If we don't support the acquisition of these files, then vendors will have to re-implement something they already have, resulting in additional testing and risk of defect introduction.
Proposed change
The proposal is to update https://github.com/zephyrproject-rtos/zephyr/blob/main/doc/contribute/bin_blobs.rst to allow for C/C++ files that encode binary data. These files would be required to follow the same rules as documented for pre-compiled libraries: https://github.com/zephyrproject-rtos/zephyr/blob/main/doc/contribute/bin_blobs.rst#precompiled-library-specific-requirements. Source files would be considered to be the 'bin' type already supported by 'west blob'.
Detailed RFC
We need to change the 'west blob' documentation to add the rule allowing C/C++ files.
Proposed change (Detailed)
See 'Proposed change'.
Dependencies
None.
Concerns and Unresolved Questions
Known concerns are around developers abusing this by adding pre-compiled code that provides and API and violates security or other similar concerns.
Alternatives
In addition to the document changes, we could add a 'src' type to 'west blob' to differentiate binary files from source files encoding binary data.
Acquiring .bin files and encoding them in C/C++ code is a use case that is already supported by the tooling. However, some silicon vendors have already produced libraries with the binary data in a C/C++ file. If we don't support the acquisition of these files, then vendors will have to re-implement something they already have, resulting in additional testing and risk of defect introduction.

This proposal extends the blob support to allow for C/C++ files containing only binary data encoded as an array of bytes (or similar mechanisms).

But why?  If we can have tools take a binary and expose it via a named symbol is that not functionally equivalent?

@quic-egmc, my primary concern is that the files Infineon has are coming from a separate team as a self-contained and validated package. Changing where the files live, such as moving binary files and their C encoding to another repository and fetching them via the blob mechanism doesn't introduce many concerns. However, throwing away the C file and regenerating it using a different tooling system and then revalidating introduces risk that something is broken in a previously working package.
One way I can ameliorate this concern is to use the Zephyr binary conversion tools and then do a md5 or equivalent to be sure that the regenerated C file is functionally equivalent to what is generated by the other team. I will take a look at that prior to next TSC.

Here are some points against this proposal:

We have tools for dealing with binary files, dumping them in hex, extracting strings, copying, etc. Dealing with a binary file in C format seems like a pain. How do I get the binary out?
There is no need to convert a binary file to C just to include it in the build. We can use 'incbin' in an assembly file to do this, for example, which allows symbols to be added. I'm sure there are other ways.
The C format is a derived format, not the original file, since as mentioned above there is tooling needed to convert from binary to C. We should try to have (closer to) original files in Zephyr, not things processed by tools that we don't have. Vendors don't really need to re-implement something, just stop running that extra step.
Different vendors may format the C code differently, complicating any effort to ingest / convert things to another format later, but there is no ambiguity as to the format of a binary file
This adds complexity since the only way to obtain the binary file is to compile a board within Zephyr! I am not suggesting that the file would contain C code or preprocessor elements, but that format is far more complex than is needed for a simple binary file
People might use #ifdef to change the content of the binary depending on the board being built. I hope not!
This 'binary blob' thing is provided to some extent as a courtesy to people who don't want to publicise the source code for the blobs. We should make this accommodation as easy as possible on the Zephyr project and its contributors. A plain binary file seems best.

BTW there is 'binman firmware packager' if we want to add more sophistication in this area to Zephyr.

@sjg20 We do use #ifdef to change the firmware image, since there are two choices, normal image or manufacturing test image. See here for instance: https://github.com/Infineon/wifi-host-driver/blob/master/WiFi_Host_Driver/resources/firmware/COMPONENT_43012/resources.h.
I assume making this decision (normal vs mfg image) could be configured via KConfig.

C encoded bin blobs
To me, from the contents nature it clearly is of "Allowed Types:Firmware images" encoded in C or even double encoded in ihex or srec or other and encoded in C and not a pre-compiled library.
From that perspective it is rather a special case (subtype) of "Allowed Types:Firmware images" which would have to be described/regulated there and not a new type.
I can see the advantage of allowing "C encoded binaries" especially when your receive it in that format from a third-party. On the other hand regulating all kind of special cases makes the handling complicated and therefore I would rather vote for keeping the interface small.
@ifyall: I feel, you should convince your separate team to provide the images as C encoded files (if still used somewhere else) and as *.bin files to be included via "incbin" in your package. I guess they must have them somewhere as bin.
Licensing
It is very important that, as a user of Zephyr, I do not run into the situation that I have to disclose my companies IP, because I did not recognize linking against non-permissively licensed code / binary parts.
Hence what ever change we do, that should be "guaranteed".
To my understanding that is covered by

as described in "Tainting" and
"Precompiled library-specific requirements:Interface header files"

already.

@ifyall: I feel, you should convince your separate team to provide the images as C encoded files (if still used somewhere else) and as *.bin files to be included via "incbin" in your package. I guess they must have them somewhere as bin.

This.
Asking zephyr to open up arbitrary source files as .bin files just because you do not want to perform the conversion of a single array in a C file to a .bin file and validate the results for yourself is asking rather too much of upstream in my opinion.

Asking zephyr to open up arbitrary source files as .bin files just because you do not want to perform the conversion of a single array in a C file to a .bin file and validate the results for yourself is asking rather too much of upstream in my opinion.

I agree.

@sjg20 We do use #ifdef to change the firmware image, since there are two choices, normal image or manufacturing test image. See here for instance: https://github.com/Infineon/wifi-host-driver/blob/master/WiFi_Host_Driver/resources/firmware/COMPONENT_43012/resources.h.
I assume making this decision (normal vs mfg image) could be configured via KConfig.

That makes it even worse, IMO. We have to decode the file with a C preprocessor to get the contents!

It's also worth noting outside of this use case, there are Machine Learning models that often come in C Array format. You look at various partners who do ML Training and AutoML in the TinyML community, they often output a C Array

It's also worth noting outside of this use case, there are Machine Learning models that often come in C Array format. You look at various partners who do ML Training and AutoML in the TinyML community, they often output a C Array

That's not an issue unless you'll need to ship one of these models in order to support a SoC in upstream Zephyr (unlikely).

Ping from the Process WG: @ifyall - Is this still something you would like Zephyr to adopt?  Or can we close this?

Warning: I'm quoting different people.

Vendors don't really need to re-implement something, just stop running that extra step.

Even simpler: stop omitting the original .bin file from their validated release package. The generated C file can stay in the package for any non-Zephyr user if needed.

However, throwing away the C file and regenerating it using a different tooling system and then revalidating introduces risk that something is broken in a previously working package.

Oh, wait: so the original binary is already in the release package in your case? So there's actually nothing to ask from your vendor?

We have tools for dealing with binary files, dumping them in hex, extracting strings, copying, etc.

Could you share some pointers?

...  is to use the Zephyr binary conversion tools and then do a md5 or equivalent to be sure that the regenerated C file is functionally equivalent to what is generated by the other team. I will take a look at that prior to next TSC.

Good idea: you could find and report bugs in the vendor's tooling :-)
Maybe the easiest is to compile the two generated C files together and compare them with memcmp().

Different vendors may format the C code differently, complicating any effort to ingest / convert things to another format later,

The list of restrictions in https://github.com/zephyrproject-rtos/zephyr/blob/main/doc/contribute/bin_blobs.rst makes it indeed pretty clear that binaries are more "tolerated" than welcome for a number of maintenance reasons. Example:

Precompiled library blobs must be in a data format which is compatible with and can be linked by a toolchain supported by the Zephyr Project. This is required for maintainability and usability.

There is also a very important introduction at the top of this document:

It is important to note that all the information in this section applies only to upstream (vanilla) Zephyr.
There are no limitations whatsoever (except perhaps license compatibility) in the support for binary blobs in forks or third-party distributions of Zephyr.

As soon as you distribute your own Zephyr branch, you can add to it absolutely whatever you want.

This was discussed previously and voted against. It is no longer relevant.

Just a comment on this closed RFC, I'm assuming the vote was to not allow a hex array C source file as a binary blob. Unless I'm missing this, shouldn't we document this point in the blobs section of the documentation?
We list the allowed types and the second one, firmware images, doesn't specify the actual format. It just says:

"An image containing the executable code for a secondary processor or CPU".

When advising an internal team, they also have been delivering a C hex array as part of their deployment for the last several years. My initial feedback was that it wasn't explicitly called out that they needed to convert this to a binary file (my reading of the above line). But now I found  this discussion thread from the process working group. I'm guessing I might have missed this set of meetings but with the benefit of some hindsight, I think we should clearly articulate the rule/restriction around a source file for both libraries and firmware images submitted as a binary blob.


fix: Temporarily disable editing metric title

SUMMARY
Temporarily disable editing metric title until displaying custom title is supported.
Closes https://github.com/apache/superset/issues/12375
Closes https://github.com/apache/superset/issues/12379
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

CC: @junlincc @villebro

@kgabryje why is this closed? I think it's reasonable to disable custom label edits for Saved Metrics until we add support for adhoc overrides.

@kgabryje why is this closed? I think it's reasonable to disable custom label edits for Saved Metrics until we add support for adhoc overrides.

@ktmud This PR disables edits for every metric as I misunderstood the requirement. Disabling edits just for saved metrics would require some more refactoring and we decided that we should wait for the overrides feature instead of introducing changes that would soon need a revert. However, if you think it's vital to disable edits for saved metrics until the feature is implemented, I could take it on again

@ktmud This PR disables edits for every metric as I misunderstood the requirement. Disabling edits just for saved metrics would require some more refactoring and we decided that we should wait for the overrides feature instead of introducing changes that would soon need a revert. However, if you think it's vital to disable edits for saved metrics until the feature is implemented, I could take it on again

I think it's worthwhile to do some refactoring and add a "overridable" flag or something as even if we enabled overrides for saved metrics some companies may not want that because of consistency.

I think it's worthwhile to do some refactoring and add a "overridable" flag or something as even if we enabled overrides for saved metrics some companies may not want that because of consistency.


Support configuration for SDKs to talk to remote Dapr APIs

Today, all SDKs assume sidecar to be running locally. In addition, they should all allow connection to a remote API with TLS over HTTP or gRPC. SDKs should agree on some common configurations:

Validate the SDK can connect to a remove Dapr API endpoint over TLS via the app hardcoding the endpoint and passing it via a constructor method or builder class.
~DAPR_HTTP_ADDRESS~ DAPR_HTTP_ENDPOINT environment variable should be accepted and parsed by the factory method/class on each SDK to use that endpoint in case of HTTP protocol (e.g. https://dapr.mycompany.com)
~DAPR_GRPC_ADDRESS~ DAPR_GRPC_ENDPOINT environment variable should be accepted and parsed by the factory method/class on each SDK to use that endpoint in case of GRPC protocol. (e.g. https://dapr-grpc.mycompany.com)
~DAPR_SSL_CA_PINNING_PATH environment variable containing the CA certs accepted for SSL pinning.~

SDKs should accept (1) or (2) (both if both protocols are supported). Also, (1) and (2) should take precedence over DAPR_SIDECAR_IP + (DAPR_HTTP_PORT or DAPR_GRPC_PORT) when DAPR_SIDECAR_IP is supported.
Assessment on each SDK:

Java SDK

Does not provide a way to override the endpoint apart from IP/hostname and port.
DAPR_SIDECAR_IP env var already exists and is used in conjunction with DAPR_HTTP_PORT or DAPR_GRPC_PORT. See https://github.com/dapr/java-sdk/blob/806ddc00bd4e5fb1531c3746ccc2ae295043b59b/sdk/src/main/java/io/dapr/config/Properties.java

.Net SDK

Offers a way to override grpc and http endpoints in builder class. See https://github.com/dapr/dotnet-sdk/blob/f42b690f4c67e6bb4209932f660c46a96d0b0457/src/Dapr.Client/DaprClientBuilder.cs#L33
Defaults to <IP_ADDRESS> and user can override via code. See https://github.com/search?q=repo%3Adapr%2Fdotnet-sdk httpEndpoint&type=code

Update:
SSL pinning is out of scope for this issue.

Most if not all Dapr SDKs today support 1 and 2, albeit there's no uniform format and some only allow code override. are you suggesting here that SDKs standardize on an agreed upon env var?

DAPR_SSL_CA_PINNING_PATH environment variable containing the CA certs accepted for SSL pinning.

I would recommend not encouraging pinning a specific certificate. Use the trust store of the host or container instead. Certificate pinning is considered deprecated and for example has been dropped by all web browser. (some more info)

I definitely think we should have a TRUST_ANCHOR_PATH (or otherwise named) option for configuring the root CAs used for connections.
Particularly in private PKIs, root CAs should be considered runtime configuration and is often not desired to be part of the container image itself.

It would also be nice if these did not solely rely on environment variables, as this requires the process to restart in order to pick up new values.

I agree. We should just prefix it and make it DAPR_TRUST_ANCHOR_PATH

I have a different opinion on that, as I mentioned in the call. I think users should set the CA certificate by modifying the trust store of their app's container, rather than setting DAPR_TRUST_ANCHOR_PATH.
My arguments are:

DAPR_TRUST_ANCHOR_PATH is a Dapr-specific thing, so users who use the SDK would see a different behavior compared to users who do not.

More specifically, if an app that uses the Dapr SDK then decides to make a call to Dapr directly (for example because the SDK uses gRPC and the apps wants to use HTTP service invocation instead - and gRPC->HTTP is being deprecated), then they would have to manage the certificates differently in that case.

Relying on the trust store is a well-known, established pattern. It's common for large organizations to add or remove certs from that, and it's very likely users already have practices and processes to manage that. In this case we'd be "reinventing the wheel" both for our clients and us.
Certain stacks already have other ways to set what trust store to use. For example, there's the standard SSL_CERT_FILE and SSL_CERT_DIR that are used by things like OpenSSL and even Go
By adding an env var that is specific to Dapr, we are making this a Dapr problem which it doesn't necessarily have to be.

That said, I can see the arguments for adding this too, but wanted to have my .02 written down for posterity :)

Regarding the endpoint configuration, I think we should have ENDPOINT instead of ADDRESS because some users might have a url rewrite on the loadbalance or API gateway.
runs app without sidecar but SDK will communicate securely with remote Dapr API
DAPR_HTTP_ENDPOINT=https://my-company.com/dapr make run

Adding a couple of comments on this thread:

Is The possibility for MTLS with client cert verification in the horizon? asking as if it is then we possibly also need for one environment variable to indicate where the client cert will be at and a convention on naming to avoid two env vars, maybe using same k8s convention (tls.key and tls.crt).  This might be low priority or leave for later if we don't foresee needing it in the near future.
I like the DAPR_HTTP_ENDPOINT, one comment in regards to convention to mean when a connection is secured by using https scheme as shown on the top. Usually gRPC connections do not use the scheme as part of the connection string so we should let users knows that this is part of the conventions to mean a secure channel will be established on their behalf. Also if the scheme is missing we should raise an error explaining it.
Josh mentioned about issues with ca folder in environment variable, but I don't think that is the problem with having these path as environment variables.  Probably what would be needed is a watch against the folders, maybe for a future task.  The same would happen in k8s pods, if the path changes, you would get a reload of the pod to use the new volume mount.

Adding a couple of comments on this thread:

Is The possibility for MTLS with client cert verification in the horizon? asking as if it is then we possibly also need for one environment variable to indicate where the client cert will be at and a convention on naming to avoid two env vars, maybe using same k8s convention (tls.key and tls.crt).  This might be low priority or leave for later if we don't foresee needing it in the near future.
I like the DAPR_HTTP_ENDPOINT/DAPR_GRPC_ENDPOINT env var naming, but one comment in regards to convention to mean when a connection is secured by using https scheme as shown on the top. Usually gRPC connections do not use the scheme as part of the connection string so we should let users knows that this is part of the conventions to mean a secure channel will be established on their behalf. Also if the scheme is missing we should raise an error explaining it.
Josh mentioned about issues with DAPR_TRUST_ANCHOR_PATH folder in environment variable, but I don't think that that is the problem with having these path as environment variables.  Probably what would be needed is a watch against the folders, maybe for a future task.  The same would happen in k8s pods, if the path changes, you would get a reload of the pod to use the new volume mount.

At least in the .NET SDK, it accepts the entire schema: https://learn.microsoft.com/en-us/aspnet/core/grpc/client?view=aspnetcore-7.0#configure-grpc-client

FYI the JavaScript SDK already has this configuration embedded into the API. For instance, one could do
const daprHost = "<IP_ADDRESS>"; // Dapr Sidecar Host, could also be https://example.com
const daprPort = "3500"; // Dapr Sidecar Port of this Example Server

// HTTP Example
const client = new DaprClient({ daprHost, daprPort });

Having extra environment variables is not required, but it can be there and the API options can take precedence over env var.

FYI the JavaScript SDK already has this configuration embedded into the API. For instance, one could do

const daprHost = "<IP_ADDRESS>"; // Dapr Sidecar Host, could also be https://example.com

const daprPort = "3500"; // Dapr Sidecar Port of this Example Server

// HTTP Example

const client = new DaprClient({ daprHost, daprPort });

Having extra environment variables is not required, but it can be there and the API options can take precedence over env var.

Thanks. Exactly, this issue is about consistency and to avoid configuration to be hardcoded in the app.

Since we're talking about this, would it be possible to also expose DAPR_APP_PROTOCOL which contains the value of the app-id flag?
This would allow SDKs to know if they need to use HTTPS or H2C rather than "regular" HTTP (starting with 1.10, using HTTPS or H2C in 1.11 offers better perf due to HTTP/2 support). Could also allow the SDKs to even automatically select between gRPC or HTTP possibly in the future.

Since we're talking about this, would it be possible to also expose DAPR_APP_PROTOCOL which contains the value of the app-id flag?
This would allow SDKs to know if they need to use HTTPS or H2C rather than "regular" HTTP (starting with 1.10, using HTTPS or H2C in 1.11 offers better perf due to HTTP/2 support). Could also allow the SDKs to even automatically select between gRPC or HTTP possibly in the future.

This is a good idea

DAPR_APP_PROTOCOL does not impact the protocol the SDK uses to talk to the sidecar. The app implements this protocol to receive calls from sidecar, so it is the sidecar that needs this information and not the other way around.
Now, we can change that if the SDKs will implement the callback API on behalf of the app.

DAPR_APP_PROTOCOL does not impact the protocol the SDK uses to talk to the sidecar. The app implements this protocol to receive calls from sidecar, so it is the sidecar that needs this information and not the other way around.
Now, we can change that if the SDKs will implement the callback API on behalf of the app.

I think this should be a separate issue discussion IMO.

I think this should be a separate issue discussion IMO.

Sure. I made an implementation (it was simple) here: https://github.com/dapr/dapr/pull/6512 We can discuss there

PR for Python: https://github.com/dapr/python-sdk/pull/578

@berndverst maybe the port would be included if dealing with the format as @artursouza mentioned in https://github.com/dapr/dapr/issues/6035#issuecomment-1529913155 (ie DAPR_HTTP_ENDPOINT=https://my-company.com/dapr)

We must not include the protocol or port in the URL - that would completely break the Python implementation @filintod
As a SDK maintainer (and contrib maintainer) I am not in favor of this. I prefer using our standard DAPR_GRPC_PORT combined with DAPR_GRPC_ENDPOINT.
Please also note that in the Python SDK we only use grpc.insecure_channel. We have no implementation for grpc.secure_channel. This is probably similar in other SDKs.
It is unclear how in the SDKs we should switch between secure and insecure channel. Today the Python SDK takes no protocol prefix for the HOST or ENDPOINT as only insecure channel are supported.

It is unclear how in the SDKs we should switch between secure and insecure channel. Today the Python SDK takes no protocol prefix for the HOST or ENDPOINT as only insecure channel are supported.

See #6512

We must not include the protocol or port in the endpoint - that would completely break the Python implementation today @filintod
As a SDK maintainer (and contrib maintainer) I am not in favor of this. I prefer using our standard DAPR_GRPC_PORT combined with DAPR_GRPC_ENDPOINT.
Please also note that in the Python SDK we only use grpc.insecure_channel. We have no implementation for grpc.secure_channel. This is probably similar in other SDKs.
It is unclear how in the SDKs we should switch between secure and insecure channel. Today the Python SDK takes no protocol prefix for the HOST or ENDPOINT as only insecure channel are supported.
The grpc libraries cannot automatically choose between secure and insecure channel. There is no such method.
Either way, it is clear that we need a more formal proposal writeup -- the issue here leaves too many open question.
@artursouza can you please write up a more formal proposal and circulate it with all SDK maintainers?

I don't think this needs a formal proposal, other SDKs are implementing this. Each SDK has a different library and how to use a secure channel is unique on each - which no formal proposal can change. The goal is simple, the user should set the endpoint + token and be ready to go. This issue is not asking maintainers to do the work, it can be done by the community - which includes me. I am volunteering to make these changes.

As we agreed in the call, here is the proposal: https://github.com/dapr/proposals/pull/27

@artursouza Is this issue closed with work done?

It was done in most SDKs. We can keep it closed.


[TEST] Exclude training offloading when gcov is enabled

Because of cyclic dependency between ml-service::training_offloading and nntrainer, exclude training offloading when gcov option is enabled.

:memo: TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #508. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.


nixos-minimal-17.09 fails to boot from USB

nixos-minimal-17.09.3205.12512f25e9e-i686-linux.iso
fails to boot from two different USB sticks, tried on two different laptops.
Disk label for stick is NIXOS_ISO.

Can't reproduce. I downloaded the same iso from the website (with sha256sum 83b052831d82b421cf6bc6e639df0a8b4f7a7de5e7b2dd5d036711faa058af3b), booted in QEMU with qemu-system-x86_64 -boot d -cdrom nixos-minimal-17.09.3205.12512f25e9e-i686-linux.iso -m 512 and it boots normally all the way.

I have the same sha256sum on my end. I was booting on real hardware, one laptop I tried on was native 32-bit, the other was a 64-bit machine. Same behavior on both machines.

@clayt0nk Have you tested the sticks for reliability?
I have several sticks that I disposed of because with the time they started to wear out and produce errors on transfer.
I usually do this test prior to using the stick for images:

mount the sticks into the system and format them with new fs
lsblk  (for the size of the sticks)
dd if=/dev/zero of=~/testimg.img bs=1M count=xxxK  ;where xxx=<amount of gigabytes your stick has, minus few. For example, if lsusb reports your stick as 16G, try 14K there.
sync  ;and wait.

after this:

md5sum of the testimg.img on your sdd/hdd.
cp the file into the usb and md5sum it there, compare
rm the file from ~/testimg.img and cp it from usb to your  ~. md5sum again.
all three md5sums should match. If they don't match, then stick can be disposed of.

I guess the simple thing to try with the sticks is after the copying to unplug it, mount the ISO filesystem and run sha256sum nix-store.squashfs. I get 1b8ae9de66f02eda87e98967c7c01c85e11669f503a7fd7feef336120fe78593.

 I've been thinking we should do that too.

@bjornfor That would be nice idea, would save from lots of problems in the future. An install is done once and checksumming is a read operation, thus should not impact flash life expectancy.

Thanks for the encouragement and suggestions, guys, I am further along. The boot problem was procedural, I neglected to do a sync before removing the USB sticks and the result (per @dezgeg suggestion) was two different USB sticks with an incorrect nix-store.squashfs sha256sum. I think few people do OS installs so often as for it to become routine. I would suggest the documentation needs to list essential steps like doing a sync after writing the USB stick. Since most of the stick image is one file, a debugging footnote suggesting a sha256sum comparison of nix-store.squashfs on disk and on USB is a really elegant self-check.

Next issue I think is also documentation, so I will keep it here: per docs I used fdisk to create a very simple partitioning scheme for my first try: one swap partition and one partition for everything else. I then started the install and everything was great (if very, very slow) until near the end when I hit a grub error. I did not preserve the actual error text, but the gist was, by accepting fdisk's default suggestions for where to put the partitions, I ended up with no (or not enough) space between the MBR and the first partition. Apparently grub2 needs a bit of empty space there, and the install failed there. This also needs to be in the docs.

I live in China and the connectivity to your default repo through the Great FireWall (GFW) is very, very poor. It took literally all day yesterday to get a very minimal SSH-only install all the way through to the grub2 error above. I changed the config file to grub1 and re-invoked nixos-install and that seems to have provoked(?) a complete rebuild of everything(?) which ran all night and is still not done this morning. Most other Linux distros I am familiar with have multiple mirrors, including some inside the GFW. It is currently not clear to me that NixOS has any other repo mirrors, in or outside of the GFW, and how to select and use one if it exists.

@clayt0nk I'm having a similar problem when I try to make a bootable NixOS USB stick. Both the graphical and minimal iso(s) fail when I try to write the iso to my usb.
I tried both of these commands to write the graphical (and minimal) iso(s). Both resulting USB sticks simply immediately exit out of the boot process without any error message. Just jumping back to the boot loader.
What did you mean by "... do a sync before removing the USB"? What extra command did you do?
sudo dd if=~/Downloads/nixos-graphical-18.03.132847.aec217852f2-x86_64-linux.iso of=/dev/sda
sudo dd bs=1024 status=progress if=~/Downloads/nixos-graphical-18.03.132847.aec217852f2-x86_64-linux.iso of=/dev/sda

Literally, in a terminal, execute a "sync" command. It seems modern hard disks and their drivers do a lot of caching, so dd may have exited, but some of what it just "wrote" still lingers in the cache and has not actually made it onto the disk. If you pull the USB out before that cache is written out, the result is corruption. The sync command simply causes all disk caches to be flushed immediately. When it returns, you should be good to pull out the stick.

Not stale, i have the same issue on latest (20/06/2021):
eefe915f68cf6a4ec519d9940d7c533b9b2c76d2d8c893cd279659e927419cfa  nixos-minimal-21.05.1555.6b940846e99-x86_64-linux.iso

Solved by using different USB flask disk


Update readme to allowing incoming NTLM traffic

Since NLA is now supported, setting this to Deny all accounts will prevent NLA authentication, and prevent RDP access altogether if NLA is enforced.
Unless of course I'm actually wrong about this and this issue still exists, but #32 implied that the instructions were updated to support NLA when that doesn't seem to be the case, at least for the readme...?
Also, the two following settings after this one only apply to the client/should be set on client and are not at all required.

I don't think this is needed. Those secondary setup instructions (the bullet list) are not needed as long as the first secpol instruction is followed.
Enabling all accounts for NLA will allow login attempts that don't actualy show the ip address in the event viewer.

I thought the idea as per #32 was that NLA can now be used - and I've tested with enforcing NLA. Unless that's not the case? Then what did you mean by your response to #32 which asks if NLA can be used?

So in other words there's still no way to enforce NLA and manage to log IP addresses of failed attempts still...?

You should be able to enable NLA and still get ip addresses as long as you do the first step, but I think NLA is on by default in Windows so no need to mess with it. I could be wrong though. And it would be off for old installs of IPBan.

Doesn't seem to be the case, at least when I tested with ipban 1.2.1. Was only able to resolve "DNS" addresses to IP, which is the computer hostname, meaning it only works for a LAN.
NLA, if not configured, is on by default for Windows 8/Server 2012 machines. I don't recall if it's enforced however.
I guess I'll test again with 1.2.3


Only client login page instead of page for order

Hello,
i today installed boxbilling v 4.17 on my server together with i-MSCP 1.2.2 .
The problem is i do not get the order page, i allways (be forwarded ?) get a client
login page.
Even if i use "Products" "links" the view button i get client login page.
Same with "Order page with ID" and "Order page with slug".
I do not get anything else than login page "billing/index.php?_url=/login"
What am i doing wrong ?
Kind regards
Harry
P.S.:
The server is running on Ubuntu 14.04 (up-to-date) with kernel 3.13.0-53-generic
PHP version is :
PHP 5.5.9-1ubuntu4.9 (cli) (built: Apr 17 2015 11:44:57)
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies
with the ionCube PHP Loader + Intrusion Protection from ioncube24.com (disabled) v5.0.7, Copyright (c) 2002-2015, by ionCube Ltd.
with Zend OPcache v7.0.3, Copyright (c) 1999-2014, by Zend Technologies
Apache 2.4.7-1ubuntu4.4
libapache2-mod-fastcgi
License.php is replaced.

Hello ;
It's the expected behavior with newest Boxbilling theme. You can switch back to old behavior by using old boxbilling theme. See theme settings in admin area.

Hello,
Thank you.
That's what i wanted to have.
Kind regards

Huraga default theme does not have guest interface because it's main purpose is to easily integrate with any existing theme.


Playing Catch

Hey Stone! I mentioned this way back in the thread regarding the door and joint contraints...
...but catching objects you toss up is difficult the higher it is. Naturally IRL, you're able to get your hand in front of a falling object so you can (ideally) catch it. Right now with your current implementation, grabbing only occurs when the controller is touching the object. Trying to catch a falling object where you have to pull the trigger right at the moment they're touching is largely impossible. (see: Why you can't catch a falling dollar).
What I proposed, and is more than likely how every other VR dev is doing it, is to allow the user to preload the grab, or anticipate it, by pulling the trigger when the object is within X units. If the falling object makes contact with the controller within that threshold (X), then they 'Catch' the object. If they pull the trigger too soon or of course too late and they miss.
For a practical example fire up The Lab, or pretty much any VR game/demo.
Cheers!
-Steven

I'll take a look at this maybe tomorrow.
I think I'll add it as an extra script you add to the controller that adds another trigger collider that's larger, then if the object falls into that collider then it will be set as touching, even though you're not touching it.
Maybe need some extra work on it where it only activates if you've thrown an item. Otherwise it may be weird for items that are stationary, so perhaps if the object you're colliding with has a velocity then it's catchable

Well this really needs no work beyond simply allowing a window of time where when the trigger is pulled and held, when an object comes in contact with the controller it will be Grabbed. In other words the moment the trigger/grip/whatever is pulled:
canGrabFallingObject -= Time.deltaTime;
if (canGrabFallingObject < 0)
{
     //Nothing happens!
}
else if (canGrabFallingObject > 0  &&  collidersTouch)
{
     //Grab that shit!
}

A larger collider would allow the falling object to be grabbed when it wasn't near the controller.
Finally Rigidbody.Velocity > 0.0?

Yeah I can give that way a go and see if it produces a good output.

Thinking about it, this way may not work.
It relies on the controller still touching the object at some point in the near future.
The problem may be being caused by collision tunnelling (where the object is moving so fast that the physics engine is moving it at too large a step for the collision to ever take place)
One solution to that is to enable continuous collision detection but it's a bit of a processor hog if it's on for lots of objects.

Well Continuous on the moving object(s) only?
Also to be clear, I can't see many instances where the thrown object is moving that quickly. If you were trying to catch a bullet fired at you sure, but tossing a ball up?

Say this becomes an issue as you say. We could combine your idea for a larger Collider around the controller when the trigger is pulled and < Time.deltaTime, and using OverlapSphere when the object is overlapping, turn on Continuous and check for contact with the controller grab collider.

The collision tunneling is a slight issue in that the object's velocity is moving so fast that the rigidbody connects and is grabbable but the velocity takes it into an ungrabbed state so it gets dropped
I think i've solved it by just setting the object's velocity to zero on grab.
Not sure if this is a good idea to do all the time anyway (would you ever want to hold an object that keeps it's velocity? seems weird)

Well right...it's being caught so Velocity should be zero no? Don't get me wrong...I'm not looking for authenticity in the physics sim, just in resultant interaction.
Well either way I'll be ready to test! :)

Yeah I agree, if you catch it, then it should lose all velocity.
Like in the real world if you caught a fast moving heavy object, then you'd be pulled with that object till it's momentum was gone.
But in a VR game if you catch (let's say a tron disc) and it pulls your controller or play rig across the floor, then that's really jarring and doesn't work well.
I'll zero the velocity (and angularVelocity) whenever you grab anything (not just in the cooldown period catch)

Ok latest version has this in now. set the Grab Precognition variable on the Interact Grab script to set your window of grab time

Perfect! As per usual I'll peep it when I'm back in front of the VR rig!

Ugh I'm sorry I completely blanked on this!
Works fantastically, though not until I entered values around 100, which seems largely arbitrary from an end-user perspective (vs. using seconds lets say).
One odd thing that happened, is at times the object would stay attached to the controller without my having to hold the grip button. To release it, I had to grip again and then release. I can't say if this occurred only after playing catch with objects, but it happened nonetheless.
Also for clarity, I was tossing around the two cubes (green/red) in 021 - GrabbingObjectsWithJoints, and any of the subsequent files that included that 'scene' with the drawer, box with lid and door.
Thanks Stone!
-Steven


request-transformer plugin missing parameters when client request body is larger than client_body_buffer_size

Is there an existing issue for this?

[X] I have searched the existing issues

Kong version ($ kong version)
Kong 3.2.2
Current Behavior
When I upload a file ,Trigger Kong to write temporary file, upstream can only receive request-transformer body,  not receive the file data
kong error.log:
2023/10/25 16:15:49 [warn] 125097#0: *5859 a client request body is buffered to a temporary file /apps/svr/kong/client_body_temp/0000000122,
Expected Behavior
No response
Steps To Reproduce
1、kong 3.2.2 ,dbless mode
2、kong_customer.yml

name: test-sit
host: test-sit
connect_timeout: 10000
protocol: http
routes:

strip_path: false
preserve_host: true
name: test-sit
hosts:

test.com
methods:
GET
POST
paths:
/
plugins:
name: request-transformer
config:
add:
body:
- sysFlag:test
3、upload a file , the file size larger than client_body_buffer_size，kong write temporary file ,upstream only receive body sysFlag:test

Anything else?
No response

@kanbo thank you for opening this issue. I believe this to be a duplicate of: https://github.com/Kong/kong/issues/10584
The behavior you are observing should be expected, in the logs you should find the following warning:

request body did not fit into client body buffer, consider raising 'client_body_buffer_size'

can you confirm?
At this time, in order for the request-transformer plugin to successfully apply body transformations it is required that the whole body fits in the memory buffer. If the current size of client_body_buffer_size is not enough for the body to fit, you can adjust its value in order to ensure the body transformation is applied correctly.


Table.body missing valid rowKey for Migration plan wizard

gotta fix this other unbecoming notification

Note, this only happens when you import via csv

So we're gonna chalk this up to, if your csv is invalid/malformed, yeah, you'll get errors


Show current date in textbox

Hi,
Is it possible to show current date in text box initially?
Is it possible to use this as date converter  to/from Jalali to Georgian?
Thanks in advance.

Yes these are good features to add

@aebrs  I recommend using moment-jalaali and in near future this component will start using that for conversion as well.

Although I said that we will have this features but now they seem redundant. You can just the variable of v-model you are passing to component to today's date


cannot time_offset half an hour

RainLoop <IP_ADDRESS>, chrome, centos7
RainLoop version, browser, OS:
auth_logging_format = "[{date:Y-m-d H:i:s}] Auth failed: ip={request:ip} user={imap:login} host={imap:host} port={imap:port}"
Expected behavior and actual behavior:
time_offset = "3.30"
Steps to reproduce the problem:
there is no option to add half an hour to log files ...
Logs or screenshots:

Fixed


CFLint inclusion

What is the reason for having CFLint included? I see that Issue #8 removed the ability to define the path as configuration. If I understand correctly, Issue #10 explains why it needed to be forked, but if those changes are part of core CFLint, is it still necessary? It seems to me that it's more of a dependency.
That said, the roadmap indicates you are looking to "remove the dependency on Java" so perhaps you are considering a complete rewrite which would justify the lack of dependency on the original project.
For now, it would be great if the path or at least file name were left to be from a configuration file instead of having to modify main.js when upgrading since the included jar is not kept up-to-date.

Ease of use is important.  But I don't think you want to tackle "remove dependence on Java".   CF itself runs on java --  we should be able to reuse the CF java install without needing to install java somewhere else. that would make sense.

@ryaneberly Since you maintain the CFLint project, these questions would best be directed to you. Do you see the project introducing any breaking changes? If so, would those be reserved for major version changes?
I can see the "ease of use" point, but based on the answers above I wonder if it would be possible to dynamically pull the latest compatible version then. Either way, my suggestion for storing the location as configuration remains.

I don't see any breaking changes in the near future.  the CLI interface and output have stabilized.  I think it's far to expect reasonable compatibility on all future 1.x.x  releases

Following up with this:

I found node-github-releases which seems to be under atom's organization. There's no documentation, but it looks like it might work to keep CFLint updated if its GitHub releases are kept up-to-date, which it doesn't seem that they are at the moment. I expect the ability to filter by version numbers exists as well.
I also found an NPM CFLint wrapper, which should be easy to integrate, and while it does depend on Java, you would at least be removing the direct dependency since that seems to be one of your goals. It unfortunately doesn't add much at the moment, but if properly maintained I think letting it manage keeping CFLint updated makes more sense than in an editor plugin.


⚠ BeyondHD has degraded performance

In 29c32f1, BeyondHD ($BHD) experienced degraded performance:

HTTP code: 200
Response time: 2476 ms

Resolved: BeyondHD performance has improved in 8147a2c.


How the language is detected by Azure AD upon login. Is it user attribute, computer locale, or some other method?

Please update the article https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/customize-branding and include the following information:
1- Company Branding is required for Windows 10 Autopilot is not clear if some features will not work without it. https://docs.microsoft.com/en-us/windows/deployment/windows-autopilot/windows-autopilot-requirements#configuration-requirements Please verify that this is the case and a list of what features will not work without branding.
2- How the language is detected by Azure AD upon login. Is it user attribute, computer locale, or some other method?
3- Include a preview showing how should look the sign in page after Company Branding is applied, indicating the chances. The final result in other features differtent from the Azure/Office 365 sign in page, such as Windows Autopilot.
4- What do I win? What do I lose? if I enable Company Branding.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 72f2e675-436d-dcd4-fa27-49408b201158
Version Independent ID: 4d821732-fc10-f4a6-3c7d-77bce75299d7
Content: Add branding to your organization's sign-in page - Azure Active Directory
Content Source: articles/active-directory/fundamentals/customize-branding.md
Service: active-directory
Sub-service: fundamentals
GitHub Login: @eross-msft
Microsoft Alias: lizross

@juquesa Thanks for your comment. We are actively investigating and will get back to you shortly.

@eross-msft Could you please have the recommendations provided by the customer , reviewed by engineering and update the article as needed ?

@shashishailaj  I will look into adding this content to the backlog. I'm adding the item to my task backlog and closing this issue.
#please-close


support multi-platform images

What would you like to be added:
kmesh related images should not have 'arch' suffix, ref: https://github.com/kmesh-net/kmesh/blob/main/kmesh_compile.sh#L22
Why is this needed:
docker already support building multi-platform images, ref: https://docs.docker.com/build/building/multi-platform/

@lec-bit is this issue fixed?

done
/close


Move cloud init to ovirt

Move cloud-init attachment out of ManageIQ state machine and into Ovirt Gem.  Refactor and improve related tests.
Depends on a new version of Ovirt gem with PR https://github.com/ManageIQ/ovirt/pull/31
https://bugzilla.redhat.com/show_bug.cgi?id=1238390

@Fryguy @gmcculloug Please review

@brandondunne Title has [WIP], but no wip label.  Which one is it for this PR?

@chessbyte Just discussed with @brandondunne and this is no longer WIP.

Sorry, meant to remove the [WIP] but got distracted

:+1: Looks good to me!


Initial working build. Needs runtime testing

I've successfully built the snap which installs and runs faas-cli version successfully. Still needs further testing for all functions to verify everything is sound.
Signed-off-by: Daniel Llewellyn<EMAIL_ADDRESS>

Thank you for help @diddledan it much appreciated!


Xmake generates invalid compilation database

Xmake Version
v2.7.3+202211090100
Operating System Version and Architecture
macOS Monterey
Describe Bug
When being used to compile 墨干 on macOS, xmake generates invalid compilation database files:
[
{
  "directory": "/Users/poscat/Projects/mogan",
  "arguments": ["/usr/bin/xcrun -sdk macosx clang", "-c", "-Qunused-arguments", "-target", "arm64-apple-macos12.6", "-isysroot", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.0.sdk", "-g", "-O0", "-std=c++17", "-Itests/Base", "-I/Users/poscat/Projects/mogan/build/.gens/url_test/macosx/arm64/debug/rules/qt/ui", "-Isrc/Data/Convert", "-Isrc/Data/Document", "-Isrc/Data/Drd", "-Isrc/Data/History", "-Isrc/Data/Observers", "-Isrc/Data/Parser", "-Isrc/Data/String", "-Isrc/Data/Tree", "-Isrc/Edit", "-Isrc/Edit/Editor", "-Isrc/Edit/Interface", "-Isrc/Edit/Modify", "-Isrc/Edit/Process", "-Isrc/Edit/Replace", "-Isrc/Graphics/Bitmap_fonts", "-Isrc/Graphics/Colors", "-Isrc/Graphics/Fonts", "-Isrc/Graphics/Gui", "-Isrc/Graphics/Handwriting", "-Isrc/Graphics/Mathematics", "-Isrc/Graphics/Pictures", "-Isrc/Graphics/Renderer", "-Isrc/Graphics/Spacial", "-Isrc/Graphics/Types", "-Isrc/Kernel/Abstractions", "-Isrc/Kernel/Containers", "-Isrc/Kernel/Types", "-Isrc/Plugins", "-Isrc/Plugins/Pdf", "-Isrc/Plugins/Pdf/PDFWriter", "-Isrc/Plugins/Pdf/LibAesgm", "-Isrc/Plugins/Qt", "-Isrc/Plugins/UniversalStacktrace", "-Isrc/Scheme", "-Isrc/Scheme/S7", "-Isrc/Scheme/Scheme", "-Isrc/Style/Environment", "-Isrc/Style/Evaluate", "-Isrc/Style/Memorizer", "-Isrc/System", "-Isrc/System/Memory", "-Isrc/System/IO", "-Isrc/System/Boot", "-Isrc/System/Classes", "-Isrc/System/Files", "-Isrc/System/Language", "-Isrc/System/Link", "-Isrc/System/Misc", "-Isrc/Texmacs", "-Isrc/Texmacs/Data", "-Isrc/Typeset", "-Isrc/Typeset/Bridge", "-Isrc/Typeset/Concat", "-Isrc/Typeset/Page", "-ITeXmacs/include", "-Isrc/Plugins/MacOS", "-DQTTEXMACS", "-DQTPIPES", "-DUSE_QT_PRINTER", "-DQT_QML_DEBUG", "-DQT_DEPRECATED_WARNINGS", "-DQT_GUI_LIB", "-DQT_WIDGETS_LIB", "-DQT_PRINTSUPPORT_LIB", "-DQT_SVG_LIB", "-DQT_TEST_LIB", "-DQT_CORE_LIB"<EMAIL_ADDRESS>"-framework", "QtTest", "-framework", "DiskArbitration", "-framework", "IOKit", "-framework", "CoreGraphics", "-framework", "OpenGL", "-framework", "Carbon", "-framework", "AppKit", "-framework", "QtGui", "-framework", "QtWidgets", "-framework", "QtPrintSupport", "-framework", "QtSvg", "-framework", "QtMacExtras", "-framework", "QtCore", "-framework", "CoreFoundation", "-framework", "Foundation", "-framework", "Security", "-framework", "SystemConfiguration", "-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-I"<EMAIL_ADDRESS>"-fPIC", "-o", "build/.objs/url_test/macosx/arm64/debug/tests/Base/base.cpp.o", "tests/Base/base.cpp"],
  "file": "tests/Base/base.cpp"
},
... (omitted)

(full file)
As you can see, the first argument, "/usr/bin/xcrun -sdk macosx clang" is invalid as it does not point to an executable and should be broken up into four arguments instead.
Expected Behavior
The generated compilation database file should be valid
Project Configuration
No response
Additional Information and Error Logs
To reproduce, just clone mogan and run xmake project -k compile_commands

As you can see, the first argument, "/usr/bin/xcrun -sdk macosx clang" is invalid as it does not point to an executable and should be broken up into four arguments instead.

no, it's valid and it works for me.

According to the spec of compilation database, the first argument should be the executable name.

which leads to errors when using CLion.

which errors? it works for vscode on my machine and I cannot get any errors in CLion.

CLion complains about not being able to find "/usr/bin/xcrun -sdk macosx clang" in path (which I think is 100% justified)

I can't reproduce the problem at the moment, so you can verify that it works by manually changing it to something like the following.
  "arguments": ["/usr/bin/xcrun", "-sdk", "macosx", "clang", "-c", "-Qunused-arguments", "-target", "arm64-apple-macos12.6", ...

If this works, I will try to improve it.

Yes, breaking up the first argument does work.

try it again, xmake update -s dev

but clangd does not support ["/usr/bin/xcrun", "-sdk", "macosx", "clang"
https://github.com/xmake-io/xmake/issues/3286

I improved it again, it will convert to "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang"
xmake update -s dev


Track new CSS Text Level 4 wrapping property values

CSS Text level 4 has new values for the word-wrap and friends. It would be nice to track those.
I'm especially on the lookout for pretty and balance

Duplicate of #1064


Remove script directory

To run local dev spr, use 'make bin' and add dist directory to path.

commit MERGED in pull request #47


feat(release): Update SDK to use API released on 2024-11-13

NEW FEATURES

Support for cluster network
Support for cluster network attachments in Instance and Instance Template
Introduction of ResourceGroupId filter on ListKeys

BREAKING CHANGES

All pagelinks(First, Next) on collection is changed to PageLink

CHANGES

Deprecation of ClassicAccess for Vpc (Instead, use a Transit Gateway to connect VPC to Classic Infrastructure)
api version 2024-11-12

BUG FIXES

None

PASSED [100%]
===================================================== 3130 passed in 17.10s =====================================================

:tada: This PR is included in version 0.25.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Update ES6 arrow function compiling support

This can solve issue #103

Closing because this project is now deprecated.


Feature/serializer

Hello @alevy, due to discussion in this pr i made changes in another branch, 'cause we are using my fork.
So I implement seriliazer object which can do stuff what we need.
But i'd like to discuss implementation with you before adding new tests.
Please take a look on it and let me know what you think about it.

@alevy Hello, that's cool thanks! I have some ideas how to simplify it.

@alevy i made changes, when you will have some time please take a look on it.

@denis-vanat Looks much better. Can you explain the rationale behind adding the opcode to the (de)serialize functions?

@saschat By adding opcode to (de)serialize we can decide what to do for each case. ie we can ignore (de)serialize values for append/prepend methods or implement it in another way.

@saschat @alevy any thoughts?

@alevy Merge and publish?


Pre Macomb robot tested&working code

Ignore the mess that is Underbot Subsystem, I'll clean it up after comp 💀

trust is good code!

@UserKorns1202 pls


CI: Automate the fetching of SimpleCov HTML results

Our automated, GitHub Actions-based CI runs are currently stitching together SimpleCov results from multiple independent runs of our tests, involving different Ruby versions and test suites. These aggregated results are used to a) pass/fail the overall CI run based on coverage thresholds, and b) comment on the PR with a markdown table of current coverage values.
Locally, devs can very easily have SimpleCov produce an HTML based overview of coverage, but it is less easy to conduct all of the various testing combinations that a single one of our automated CI runs takes care of.
It would be great if we could have an HTML summary for the CI runs, AND be able to access it as easily as we currently are for local dev.
We'll need to do two things:

Update the CI's SimpleCov report generation to include HTML output (DONE)
Automated the fetching of the GHA artifact that contains the HTML

For 1), see #1377.
For 2), perhaps a Rake task could:

Accept a branch or PR number or default to "latest in dev"
Leverage the GHA API endpoint for listing artifacts, https://docs.github.com/en/rest/actions/artifacts#list-artifacts-for-a-repository
Leverage the GHA API endpoint for downloading artifacts, https://docs.github.com/en/rest/actions/artifacts#download-an-artifact
Explode the artifact to a git ignored location
Invoke open /path/to/index.html

@fallwith  - Has this been addressed with Stoat?

This has been addressed with stoat perf! https://github.com/fallwith/stoat


Codegen cache

This is a PR for caching the compiled objects in codegen, so that if the same query is requested, the codegen checks whether or not the plan has already been compiled by comparing each item in the two plans.  This is another PR which is cherry picked from #656, and is based on the PR #675.
This cache could be enhanced further in the future by mixing eviction policies, configuring the cache size, manually pinning some queries in the cache, making them persistent in the cache, etc. In terms of software engineering point of view, the plan comparator could be embedded with plan classes rather than having it separately in codegen.

Coverage decreased (-5.8%) to 65.682% when pulling 605ab74c6aba6f1ab049644be3c690def24cecb3 on seojungmin:codegen-cache into 03c1764d2cbb1087c3909d9b13dd95ec8d304ad1 on cmu-db:master.

The plan comparison scheme in this PR may not work for some corner cases. The concept is fine, but more careful review on the plan comparison is required.  I am closing the PR, and resubmit the PR once it is more ready.


updated readme

good job! :) When we resolve problem with tests.. I'll update our geolocation at pypi.


Added the Shortest Path Finder gssoc'21

Description
Path-Finder is a Shortest Path Finding GUI application made with Pygame. It finds the shortest path between two points having some obstacles and thus avoiding them. It also shows the area of the board tracked to find the shortest path.
Fixes #496
Have you read the Contributing Guidelines on Pull Requests?

[x] Yes
[ ] No

Type of change

[x] New feature (non-breaking change which adds functionality)

Checklist:

[x] My code follows the style guidelines(Clean Code) of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have created a helpful and easy to understand README.md
[x] My documentation follows Template for README.md
[x] My changes generate no new warnings
[x] I have added tests/screenshots(if any) that prove my fix is effective or that my feature works.

Please review @santushtisharma10 @tamanna-verma @antrikshmisri as I don't have much experience in this algo. PIng me when it's ready to be merged, will assign the tags

@SpecTEviL Looks good , just add handle this error which occurs at boundaries

Its not occurring in my local machine. Can you retry that? Or else elaborate on the error? @antrikshmisri

@SpecTEviL Looks good , just add handle this error which occurs at boundaries

Its not occurring in my local machine. Can you retry that? Or else elaborate on the error? @antrikshmisri

@SpecTEviL Looks good , just add handle this error which occurs at boundaries

Also, can you recheck if it's my project only? Because the file you're running has a different name than mine. So just check that once and if the error occurs again do let me know. I'll debug it. @antrikshmisri
Thanks!

@SpecTEviL Yes , it is your script just under different name. I think the problem is in your grid placement.

As you can see near the black block there is still some space to place a block and as soon as I place a block there an IndexError is thrown

@SpecTEviL I agree with @antrikshmisri .The problem is in your grid placement.

@SpecTEviL I agree with @antrikshmisri .The problem is in your grid placement.

@SpecTEviL I agree with @antrikshmisri .The problem is in your grid placement.

Ok, I'll debug it and get back to it.
Thank you @antrikshmisri and @santushtisharma10 for letting me know about this error.
I'll debug it ASAP!

@SpecTEviL I agree with @antrikshmisri .The problem is in your grid placement.

@SpecTEviL I agree with @antrikshmisri .The problem is in your grid placement.

Ok, I'll debug it and get back to it.
Thank you @antrikshmisri and @santushtisharma10 for letting me know about this error.
I'll debug it ASAP!

Hey, @santushtisharma10 and @antrikshmisri I've made the required changes. You can review it now. If the error still persists do let me know.
Thanks!

@antrikshmisri @tamanna-verma

@all-contributors please add @SpecTEviL for code and documentation


Twemsentinel : supervisor restart error : nutcracker: ERROR (no such process)

Hi,
I have just configure the twemsentinel but facing below error while restart the sentinel.
restart
nutcracker: ERROR (no such process)
nutcracker: ERROR (no such process)
restart error
config.yml :-
sentinel_ip: "<IP_ADDRESS>"
sentinel_port: "26380"
twemproxy_config_file: "nutcracker.yml"
nutcracker_restart_command: "supervisorctl restart nutcracker"
log_file: "twemsentinel.log"
Please help to resolve the issue.

Hi
You should install nutcracker ( https://github.com/twitter/twemproxy ) and set nutcracker_restart_command in config.yml
I used supervisord for restart nutcracker (http://supervisord.org/)

I have already installed the nutcracker and run through with below command.
cache    15407     1  0 16:20 ?        00:00:05 /home/cache/.gem/ruby/gems/nutcracker-<IP_ADDRESS>/ext/nutcracker/src/nutcracker -c ../conf/nutcracker.yml -p /var/run/nutcracker/nutcracker.pid
I am using supervisorctl to restart the nutcracker service but it gives below error
nutcracker: ERROR (no such process)
nutcracker: ERROR (no such process)
But it unable to find the process id of nutcracker to restart the process.
Also please share the command how to restart the nutcracker through supervisord.

The problem is not about twemsentinel
Your supervisorctl cant find nutcracker process. Did you try "supervisorctl reload" and "supervisorctl update"
Try in terminal "supervisorctl restart all" or "supervisorctl restart nuctracker(or you program name)"

how to run it through supervisord. We are getting error while running below command.
supervisord restart nutcracker
Error :- positional arguements are not supported.

can you paste supervisord program conf file

; Sample supervisor config file.
;
; For more information on the config file, please see:
; http://supervisord.org/configuration.html
;
; Note: shell expansion ("~" or "$HOME") is not supported.  Environment
; variables can be expanded using this syntax: "%(ENV_HOME)s".
[unix_http_server]
file=/tmp/supervisor.sock   ; (the path to the socket file)
chmod=0777                  ; socket file mode (default 0700)
chown=cache:cache           ; socket file uid:gid owner
;username=user              ; (default is no username (open server))
;password=123               ; (default is no password (open server))
;[inet_http_server]         ; inet (TCP) server disabled by default
;port=<IP_ADDRESS>:9001        ; (ip_address:port specifier, *:port for all iface)
;username=user              ; (default is no username (open server))
;password=123               ; (default is no password (open server))
[supervisord]
logfile=/home/cache/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB        ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10           ; (num of main logfile rotation backups;default 10)
loglevel=info                ; (log level;default info; others: debug,warn,trace)
pidfile=/home/cache/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false               ; (start in foreground if true;default false)
minfds=1024                  ; (min. avail startup file descriptors;default 1024)
minprocs=200                 ; (min. avail process descriptors;default 200)
;umask=022                   ; (process file creation umask;default 022)
user=cache                   ; (default is current user, required if root)
;identifier=supervisor       ; (supervisord identifier, default is 'supervisor')
;directory=/tmp              ; (default is not to cd during start)
;nocleanup=true              ; (don't clean up tempfiles at start;default false)
;childlogdir=/tmp            ; ('AUTO' child log dir, default $TEMP)
;environment=KEY="value"     ; (key value pairs to add to environment)
;strip_ansi=false            ; (strip ansi escape codes in logs; def. false)
; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket
user=cache
;serverurl=http://<IP_ADDRESS>:9001 ; use an http:// url to specify an inet socket
;username=chris              ; should be same as http_username if set
;password=123                ; should be same as http_password if set
;prompt=mysupervisor         ; cmd line prompt (default "supervisor")
;history_file=~/.sc_history  ; use readline history if available
; The below sample program section shows all possible program subsection values,
; create one or more 'real' program: sections to be able to control them under
; supervisor.
;[program:theprogramname]
;command=/bin/cat              ; the program (relative uses PATH, can take args)
;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
;numprocs=1                    ; number of processes copies to start (def 1)
;directory=/tmp                ; directory to cwd to before exec (def no cwd)
;umask=022                     ; umask for process (default None)
;priority=999                  ; the relative start priority (default 999)
;autostart=true                ; start at supervisord start (default: true)
;autorestart=unexpected        ; whether/when to restart (default: unexpected)
;startsecs=1                   ; number of secs prog must stay running (def. 1)
;startretries=3                ; max # of serial start failures (default 3)
;exitcodes=0,2                 ; 'expected' exit codes for process (default 0,2)
;stopsignal=QUIT               ; signal used to kill process (default TERM)
;stopwaitsecs=10               ; max num secs to wait b4 SIGKILL (default 10)
;stopasgroup=false             ; send stop signal to the UNIX process group (default false)
;killasgroup=false             ; SIGKILL the UNIX process group (def false)
user=cache                     ; setuid to this UNIX account to run the program
;redirect_stderr=true          ; redirect proc stderr to stdout (default false)
;stdout_logfile=/a/path        ; stdout log path, NONE for none; default AUTO
;stdout_logfile_maxbytes=1MB   ; max # logfile bytes b4 rotation (default 50MB)
;stdout_logfile_backups=10     ; # of stdout logfile backups (default 10)
;stdout_capture_maxbytes=1MB   ; number of bytes in 'capturemode' (default 0)
;stdout_events_enabled=false   ; emit events on stdout writes (default false)
;stderr_logfile=/a/path        ; stderr log path, NONE for none; default AUTO
;stderr_logfile_maxbytes=1MB   ; max # logfile bytes b4 rotation (default 50MB)
;stderr_logfile_backups=10     ; # of stderr logfile backups (default 10)
;stderr_capture_maxbytes=1MB   ; number of bytes in 'capturemode' (default 0)
;stderr_events_enabled=false   ; emit events on stderr writes (default false)
;environment=A="1",B="2"       ; process environment additions (def no adds)
;serverurl=AUTO                ; override serverurl computation (childutils)
; The below sample eventlistener section shows all possible
; eventlistener subsection values, create one or more 'real'
; eventlistener: sections to be able to handle event notifications
; sent by supervisor.
;[eventlistener:theeventlistenername]
;command=/bin/eventlistener    ; the program (relative uses PATH, can take args)
;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
;numprocs=1                    ; number of processes copies to start (def 1)
;events=EVENT                  ; event notif. types to subscribe to (req'd)
;buffer_size=10                ; event buffer queue size (default 10)
;directory=/tmp                ; directory to cwd to before exec (def no cwd)
;umask=022                     ; umask for process (default None)
;priority=-1                   ; the relative start priority (default -1)
;autostart=true                ; start at supervisord start (default: true)
;autorestart=unexpected        ; whether/when to restart (default: unexpected)
;startsecs=1                   ; number of secs prog must stay running (def. 1)
;startretries=3                ; max # of serial start failures (default 3)
;exitcodes=0,2                 ; 'expected' exit codes for process (default 0,2)
;stopsignal=QUIT               ; signal used to kill process (default TERM)
;stopwaitsecs=10               ; max num secs to wait b4 SIGKILL (default 10)
;stopasgroup=false             ; send stop signal to the UNIX process group (default false)
;killasgroup=false             ; SIGKILL the UNIX process group (def false)
user=cache                     ; setuid to this UNIX account to run the program
;redirect_stderr=true          ; redirect proc stderr to stdout (default false)
;stdout_logfile=/a/path        ; stdout log path, NONE for none; default AUTO
;stdout_logfile_maxbytes=1MB   ; max # logfile bytes b4 rotation (default 50MB)
;stdout_logfile_backups=10     ; # of stdout logfile backups (default 10)
;stdout_events_enabled=false   ; emit events on stdout writes (default false)
;stderr_logfile=/a/path        ; stderr log path, NONE for none; default AUTO
;stderr_logfile_maxbytes=1MB   ; max # logfile bytes b4 rotation (default 50MB)
;stderr_logfile_backups        ; # of stderr logfile backups (default 10)
;stderr_events_enabled=false   ; emit events on stderr writes (default false)
;environment=A="1",B="2"       ; process environment additions
;serverurl=AUTO                ; override serverurl computation (childutils)
; The below sample group section shows all possible group values,
; create one or more 'real' group: sections to create "heterogeneous"
; process groups.
;[group:thegroupname]
;programs=progname1,progname2  ; each refers to 'x' in [program:x] definitions
;priority=999                  ; the relative start priority (default 999)
; The [include] section can just contain the "files" setting.  This
; setting can list multiple files (separated by whitespace or
; newlines).  It can also contain wildcards.  The filenames are
; interpreted as relative to this file.  Included files cannot
; include files themselves.
;[include]
;files = relative/directory/*.ini

write this in your supervisord conf file and reload supervisord
[program:nutcracker]
command=(your nutcracker path)/nutcracker -c  (twemsentinel path)/nutcracker.yml
stderr_logfile = /var/log/supervisord/nutcracker-err.log
stdout_logfile = /var/log/supervisord/nutcracker-out.log

Hi,
Its working now but give below error.
nutcracker: ERROR (not running)
nutcracker: ERROR (abnormal termination)

Hi,
After restart through supervisor, server value updated in single quotes like ' <IP_ADDRESS> ' in nutcracker.yml file . I have tried to resolved but nut able to fix yet.
If you have any idea please help me to resolve.


Problems with $push

So, I'm trying to push a mongo ObjectId into an array. The model is the following:
const videos = new Schema({
    name: { type: String, required: true },
    likes: [{ type: ObjectId, ref: 'users', default: [] }],
    dislikes: [{ type: ObjectId, ref: 'users', default: [] }],
    views: { type: Number, default: 0 },
  }, {
    timestamps: true
  });

Then hooks are the following:
before: {
    patch: [
      authenticate('jwt'),
      isValidAction(),
      doLikeAction()
    ]
  }

the doLikeAction is the function not working
const doLikeAction = function(){
  return function(hook) {
    return new Promise((resolve, reject) => {
      hook.app.service('users').find({ query: { _id: hook.params.user._id }})
        .then(res => {
          hook.app.service('videos').update(hook.id, { $push: { likes: res.data[0]._id } })
          .then(res => {
            console.log(res)
            resolve(res)
          })
          .catch(err => {
            console.log(err)
            reject(new errors.BadRequest(constants.BAD_PARAMETERS));
          })
        })
    })
  }
}

It doesn't say which error it is. The mongo ID are valid. The project is created using the yo generator. Hope you can help! Thank you :)

The problem is probably that you are resolving with the response instead of the hook object (hooks require certain return values). This causes an error which you will probably surface if you fix the promise chain by using the promises that Feathers already returns:
const doLikeAction = function(){
  return function(hook) {
    return hook.app.service('users').find({
      query: { _id: hook.params.user._id}
    })
    .then(res => {
      return hook.app.service('videos').update(hook.id, {
        $push: { likes: res.data[0]._id }
      });
    }).then(res => {
      console.log(res);
      return hook;
    });
  }
}

Everything works fine with update, but it's not what I meant to use. My bad.
If instead of update I use patch (which is what I want to use), then it always fail.
.then(res => {
      return hook.app.service('videos').patch(hook.id, {
        $push: { likes: res.data[0]._id }
      });
    })

The point is that I'm not understanding why it fails and the error is not helping :(
Thank for your reply hope you can help! :)

@Fraccaman , are you able to resolve the patch issue ..? I am still not able to make it working
userDetails.patch(contextrecords.receiver_id, {
query: {
$push: { approved_friends: contextrecords.sender_id }
}
});

@Fraccaman ,
Sorry my fault , the below code worked for patch
userDetails.patch(contextrecords.receiver_id, {
      $push: { approved_friends: contextrecords.sender_id }
    });


Honor Ordering of DI for xml

There were changes made in bpmn-js 6.3.4 https://github.com/bpmn-io/bpmn-js/blob/develop/CHANGELOG.md#634 fixing an issue from https://github.com/bpmn-io/bpmn-js/issues/985
This broke our export due to the relationship between participants and their multiple DIs.
We disabled this reordering, however, in the long run this is something we should fix to suit the standard.

I think the core problem is in this line.
It calls getDi for each element, which obviously does not work for participants.
So we have to somehow overwrite that with the following exceptions:

If the element is a Participant, return nothing (undefined).
If the element is a ChoreographyActivity, also return the participant band DIs ([di1, di2, ...]).
Otherwise, return the singular di (di).

We should also check whether this works for our re-implementation of Messages, their selfAndAllChildren function might not be aware of the fact that we use them differently.


Remove pod-level metrics from telemetry pipeline

Blocked by #261
Stop writing to the source and target labels in Prometheus.

Note that we cannot remove the source_ip field from the request context in the telemetry report proto because the proxy does not know which deployment an IP belongs to.  It is the controller that must determine this.

I believe this was closed in #304 as well:
https://github.com/runconduit/conduit/blob/2015d992cc9cdc6f32bb132add5118b7862cd455/controller/telemetry/server.go#L407-L412


Revert "Upload exported CSV file to S3 rather than attaching it"

Reverts alphagov/support-api#411 which introduces a bug; corrected PR forthcoming.

I agree it might have been easier to fix forward here, Support API doesn't get released very often, and it's possible to add messages in the Release app telling other people not to deploy it, but oh well!


Request: Light theme tweaks

I love the Dark mode of Night Owl and it's my primary theme!
In certain lighting conditions I've been looking for a Light theme, and Night Owl Light (No Italics) is pretty close.
Here are some things I've been tweaking for what it's worth.
I can do a PR for some of this if you're interested.
- Active tab is hard to distinguish

Adjusted to #fefefe

- Selected text in "Find in Files" is hard to read

Adjusted to #c2c7c78f

Syntax highlighting contrast ratios aren't as high as Dark mode
Dark = 7.62
https://contrast-ratio.com/#%23c792ea-on-%23011627
Light = 4.85
https://contrast-ratio.com/#%23994cc3-on-%23fbfbfb
To increase the ratio I changed all of the #994cc3 TextMate rules to #7c11b7 which has a 7.7 ratio
https://contrast-ratio.com/#%237c11b7-on-%23fbfbfb
And the #c789d6 for JSON Property values (string) to #b63dd4.
Haven't gone any further yet, because I'm not great at picking colors.
Thanks for bringing this theme into the world!

@cdeutsch I was having the same problem with the active tab contrast. Your fix is great! Thanks!

@cdeutsch I was having the same problem with the active tab contrast. Your fix is great! Thanks!

I've found this combination, when combined with @cdeutsch's active tab color, gives a really nice tab appearance:
    "[Night Owl Light]": {
      "tab.activeBackground": "#fefefe",
      "tab.activeBorder": "#994cc3"
    }

I've found this combination, when combined with @cdeutsch's active tab color, gives a really nice tab appearance:
    "[Night Owl Light]": {
      "tab.activeBackground": "#fefefe",
      "tab.activeBorder": "#994cc3"
    }


Uniform mesh conversion fixes

Fixes issues where multiple uniform mesh conversions happen in the same timestep (e.g. gamma-transport corrected global flux calculations with detailed axial expansion and critical control rod insertion).

Ah dang the commit from the other PR was not supposed to be in here. Will remove.

K I made a subclass at least to acknowledge the fact that that one was really neutronics specific.


Voice changing issue (CUDA out of memory)

Hi I am new to RVC, and I have little knowledge about it.
I was trying out how powerful the application, and it was really cool. I was able to separate the voice and instrumental.
But when it comes to changing voice, I getting these error:

I assume that because my gpu ran out of memory:

I was using a random trained voice (the .pth file) from this repo
And I sure it wasn’t the problem (I haven’t try the other voice yet), and it was because of my gpu limitations. I also haven’t try train a voice yet but I assume that it will also give out of memory error.
So my question is:

Can I solve the out of memory issue? Was it related to the “config.py”? Because I did read the faq and I think I need to change some settings in the file.
If not, can I use cpu processing for the voice changer? I am fine with the slow processing speed as long as it came with results!

Pc specs:
Cpu: Core i3 gen 12th (4 core 8 threads)
Ram: 16GB
Gpu: GTX 950 (2GB vram)
I also forgot to mention that I used the installation script from this video (or the powershell iex script: iex (irm rvc.tc.ht)) so that I saved my time installing manually.
Thanks!
哈咯哈咯，我的华文会很差，只是希望可以让你们明白点。
我刚接触这个变音器，所以我不是很会用。
我只是来看看这个东西有多好玩，就觉得这个分唱歌的和伴奏的很成功。
但是到变音的时候，就开始有问题。我是觉得因为out of memory的问题，所以就求方法搞这个问题。

所以我想问的是请问那个out of memory的问题可以解决吗？是不是关系到“config.py”？
如果不能的话我可以用CPU操作吗？

我电脑的配件:
Cpu: intel core i3 gen 12th
Ram: 16GB
Gpu: GTX 950 (2GB vram)
謝謝!

Update: nevermind, in the "config.py", just lower the x_pad, x_query, x_center, x_max

so it won't show cuda timeout, and the problem is not even related to gpu vram, it was fine.
Let me know if i can actually improve this, otherwise problem solved! Thanks!

gtx950 is not within the supported range. Please use a Niadiv graphics card of gtx10xx series or higher

2G GPU memory is not enough for inference.
If you want to use CPU to inference,  you can modify this line
self.device = "cuda:xxx"
to
self.device = "cpu"

Thanks! It works very stable for now.


Check release exists

For a release, if the version before and after are the same, still create a release if the releases does not already exist.
Also update tests to allow for a certified image tests use being removed from certified images in pyxis.

LGTM


Fix(#88): Adding an option for use local IP as host

Added Ethernet address along with configuration. I have prepared this PR for #88 but without using node-ip.

Thank you sooooo much for the PR  💟


Fix for [Bug] PlutoGrid Unexpected null value #88

Fix: https://github.com/doonfrs/pluto_grid_plus/issues/88

Hi @stan-at-work
Why does the whole file look changed? There is something wrong with the formatting. What do you think?

Hi @stan-at-work
Why does the whole file look changed? There is something wrong with the formatting. What do you think?

Idk probly this has something to do with it:
https://www.baeldung.com/linux/line-breaks-types#:~:text=CR LF%2C which stands for,cursor to the next line.

@doonfrs This is just a white space isse, as discribed in: https://github.com/doonfrs/pluto_grid_plus/pull/122 and https://github.com/doonfrs/pluto_grid_plus/pull/102#issuecomment-2547222298
Turn the Hide whitespace toggle and then you see the changes.
https://github.com/user-attachments/assets/a9445006-d388-43b6-b3dd-06c14ef85c0c


sed: cannot rename /etc/sedLNnnDc: Device or resource busy

Hi, i have big issues trying to complete cluster configuration on docker.
Using pai release v0.8.y
docker version 17.06.2-ce
I have 5 different docker containers, Dev-Box, Master and three Member nodes.
My quick-start.yaml looks like;
machines:
  - <IP_ADDRESS>
  - <IP_ADDRESS>
  - <IP_ADDRESS>
  - <IP_ADDRESS>

ssh-username: pai
ssh-password: pai-password

ssh-keyfile-path: /root/.ssh/id_rsa
ssh-secret-name: pai-password

Generating configuration works as expected, the Dev box can access all nodes via ssh
python paictl.py config generate -i ./deployment/quick-start/quick-start-example.yaml -o /cluster-configuration -f
/usr/local/lib/python2.7/dist-packages/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown. warnings.warn(warning, RequestsDependencyWarning)
2018-11-05 10:38:56,366 [INFO] - paramiko.transport : Connected (version 2.0, client OpenSSH_7.2p2) /usr/lib/python2.7/dist-packages/Crypto/Cipher/blockalgo.py:141: FutureWarning: CTR mode needs counter parameter, not IV self._cipher = factory.new(key, *args, **kwargs)
2018-11-05 10:38:56,562 [INFO] - paramiko.transport : Authentication (publickey) successful!
2018-11-05 10:38:56,649 [INFO] - deployment.k8sPaiLibrary.maintainlib.common : Executing the command on host [<IP_ADDRESS>]: cat /etc/resolv.conf | grep nameserver | cut -d ' ' -f 2 | head -n 1 <IP_ADDRESS>
2018-11-05 10:38:56,664 [INFO] - paramiko.transport : Connected (version 2.0, client OpenSSH_7.2p2)
2018-11-05 10:38:56,860 [INFO] - paramiko.transport : Authentication (publickey) successful!
2018-11-05 10:38:56,947 [INFO] - deployment.k8sPaiLibrary.maintainlib.common : Executing the command on host [<IP_ADDRESS>]: hostname 48cff661a0ae
2018-11-05 10:38:56,962 [INFO] - paramiko.transport : Connected (version 2.0, client OpenSSH_7.2p2)
2018-11-05 10:38:57,156 [INFO] - paramiko.transport : Authentication (publickey) successful!
2018-11-05 10:38:57,242 [INFO] - deployment.k8sPaiLibrary.maintainlib.common : Executing the command on host [<IP_ADDRESS>]: hostname 9ca63035a91b
2018-11-05 10:38:57,257 [INFO] - paramiko.transport : Connected (version 2.0, client OpenSSH_7.2p2)
2018-11-05 10:38:57,528 [INFO] - paramiko.transport : Authentication (publickey) successful!
2018-11-05 10:38:57,604 [INFO] - deployment.k8sPaiLibrary.maintainlib.common : Executing the command on host [<IP_ADDRESS>]: hostname 67807992a189
2018-11-05 10:38:57,641 [INFO] - paramiko.transport : Connected (version 2.0, client OpenSSH_7.2p2)
2018-11-05 10:38:57,884 [INFO] - paramiko.transport : Authentication (publickey) successful!
2018-11-05 10:38:57,958 [INFO] - deployment.k8sPaiLibrary.maintainlib.common : Executing the command on host [<IP_ADDRESS>]: hostname f8baef49f2c8

But starting the cluster causes an error;
python paictl.py cluster k8s-bootup -p /cluster-configuration

/usr/local/lib/python2.7/dist-packages/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
  warnings.warn(warning, RequestsDependencyWarning)
2018-11-05 10:47:36,517 [INFO] - __main__ : Begin to initialize PAI k8s cluster.
2018-11-05 10:47:36,574 [WARNING] - deployment.k8sPaiLibrary.maintainlib.deploy : Begin to deploy a new cluster to your machine or vm.
2018-11-05 10:47:36,575 [INFO] - deployment.k8sPaiLibrary.maintainlib.deploy : Begin to deploy k8s on host <IP_ADDRESS>, the node role is [ master ]
2018-11-05 10:47:36,644 [INFO] - paramiko.transport : Connected (version 2.0, client OpenSSH_7.2p2)
/usr/lib/python2.7/dist-packages/Crypto/Cipher/blockalgo.py:141: FutureWarning: CTR mode needs counter parameter, not IV
  self._cipher = factory.new(key, *args, **kwargs)
2018-11-05 10:47:36,842 [INFO] - paramiko.transport : Authentication (publickey) successful!
2018-11-05 10:47:36,928 [INFO] - deployment.k8sPaiLibrary.maintainlib.common : Executing the command on host [<IP_ADDRESS>]: getent passwd pai | cut -d: -f6
/home/pai
2018-11-05 10:47:36,963 [INFO] - paramiko.transport : Connected (version 2.0, client OpenSSH_7.2p2)
2018-11-05 10:47:37,161 [INFO] - paramiko.transport : Authentication (publickey) successful!
pai-password
2018-11-05 10:47:37,277 [INFO] - paramiko.transport : Connected (version 2.0, client OpenSSH_7.2p2)
2018-11-05 10:47:37,446 [INFO] - paramiko.transport : Authentication (password) successful!
2018-11-05 10:47:37,547 [INFO] - paramiko.transport.sftp : [chan 0] Opened sftp connection (server version 3)
2018-11-05 10:47:37,552 [INFO] - paramiko.transport.sftp : [chan 0] sftp session closed.
2018-11-05 10:47:37,564 [INFO] - paramiko.transport : Connected (version 2.0, client OpenSSH_7.2p2)
2018-11-05 10:47:37,760 [INFO] - paramiko.transport : Authentication (publickey) successful!
2018-11-05 10:47:37,847 [INFO] - deployment.k8sPaiLibrary.maintainlib.common : Executing the command on host [<IP_ADDRESS>]: tar -xvf master-deployment.tar
master-deployment/docker-daemon.json
master-deployment/docker-ce-install.sh
master-deployment/docker-config-update.py
master-deployment/kubelet.sh
master-deployment/hosts-check.sh
master-deployment/kubelet-start.sh
master-deployment/etc/kubernetes/config
master-deployment/etc/kubernetes/manifests/scheduler.yaml
master-deployment/etc/kubernetes/manifests/controller-manager.yaml
master-deployment/etc/kubernetes/manifests/apiserver.yaml
master-deployment/etc/kubernetes/manifests/etcd.yaml
2018-11-05 10:47:37,880 [INFO] - paramiko.transport : Connected (version 2.0, client OpenSSH_7.2p2)
2018-11-05 10:47:38,075 [INFO] - paramiko.transport : Authentication (publickey) successful!
2018-11-05 10:47:38,169 [INFO] - deployment.k8sPaiLibrary.maintainlib.common : Executing the command on host [<IP_ADDRESS>]: sudo ./master-deployment/hosts-check.sh <IP_ADDRESS>
pai-password
sed: cannot rename /etc/sedlXim0b: Device or resource busy
sed: cannot rename /etc/sedPAzZKb: Device or resource busy
sed: cannot rename /etc/sedAC3fMc: Device or resource busy
sed: cannot rename /etc/sedLNnnDc: Device or resource busy

So looking at the script on the cluster master it is trying to replace the localhost address in /etc/hosts file to the hostname of the container.
# set host ip
hostip=$1

# Change <IP_ADDRESS> line to "<IP_ADDRESS> localhost $hostname"
grep -q "<IP_ADDRESS>" /etc/hosts && \
    sed -i "/<IP_ADDRESS>/c\<IP_ADDRESS> localhost $(hostname)" /etc/hosts || \
    sed -i "\$a<IP_ADDRESS> localhost $(hostname)" /etc/hosts

# Comment <IP_ADDRESS> line
sed -i "/<IP_ADDRESS>/s/^/# /" /etc/hosts

# Comment hostip line
sed -i "/$hostip/s/^/# /" /etc/hosts

So I have tried many times and search widely to try to sort this help. I am assuming of course that each node of the cluster (Master and Member nodes) can be docker containers too.
Any help would be very appreciated.

Hi, OpenPai based on kubernetes and docker, so I wonder whether what you are doing can work or not. You can remove the  modify of /etc/hosts is grantee the  working of hadoop. I think you can remove those line pasted in your issue.

Thanks @ydye I may be tripping up on the install instructions for cluster. Should kubernetes be on physical host that I am running as Dev-Box?

Hi @leonobrien
You'd better not run the dev-box container on the physical machine where you wanna deploy pai. And here is the answer. Following are 2 way to deploy OpenPai. And if you wanna achieve the DinD (Docker in Docker), that means running container in a container. Maybe you should refer to this link first. But we haven't test this scenario before.
Why dev-box shouldn't be on the same physical machine  (Or the node runs docker-daemon?) of kubernetes, when deploy kubernetes by OpenPai.
This script is responsible for installing docker-ce and enable log-rotation. So the docker-daemon will be restart. If the dev-box container is running on one of the node
The reason to enable log-ration is because that hadoop will ouput a large amount of log which may run out of the disk space.
Two different way.
Single-box
In this mode, you only need to deploy pai on the same machine. Those machine should be labeled with ```
pai-master = "true"
pai-worker = "true"
k8s-role = master

When deploying pai, please install the following dependency package on your host environment (Not in the container) and then instal OpenPai on the host environment.

RUN apt-get -y update &&
apt-get -y install
nano
vim
joe
wget
curl
jq
gawk
psmisc
python
python-yaml
python-jinja2
python-paramiko
python-urllib3
python-tz
python-nose
python-prettytable
python-netifaces
python-dev
python-pip
python-mysqldb
openjdk-8-jre
openjdk-8-jdk
openssh-server
openssh-client
git
bash-completion
inotify-tools
rsync
realpath
net-tools &&
mkdir -p /cluster-configuration &&
git clone https://github.com/Microsoft/pai.git &&
pip install python-etcd docker kubernetes GitPython

More detailed information, please refer to the [link](https://github.com/Microsoft/pai/blob/master/docs/pai-management/doc/cluster-bootup.md#singlebox)

###### distributed mode.

In this mode, you should have at least 2 node. And more detailed information, please refer to this [Link](https://github.com/Microsoft/pai/blob/master/docs/pai-management/doc/cluster-bootup.md#customizeddeploy)

thanks @ydye you are most helpful. I'm up and running with Kubernetes and progressing.


[Feature]: Modifiable completion default

What would you like to see added?
Currently if there is no rule to make a completion, DefaultCompleter defaults to files in the current directory. Make this modifiable.

I think we will need to do some more design work on priority levels for completions, and how they should be merged. Having a single base completion rule for default that is configurable is a good place to start for now.


[Build plan] Use the provided file system consistently

When writing an output file map and using the integrated Swift
driver, use the provided file system rather than the local file
system.

@swift-ci please smoke test


Update a word

In the context (hehe) of the sentence, are made more sense to me. Know how to use a state management lib? Then avoid using context.

I believe the gist of the sentence is that if you shouldn't try to solve the problems Redux solves using context. I.e., if you want to manage state globally, try Redux first.
Sorry that that isn't clear! Does that make sense or do you still think the wording should be changed?

@alexkrolick That makes sense, although it sounds like we are saying the same thing! I still think the word needs changing

Not quite - what it's trying to say is:

Don't use context for shared global state, because you will run into the shouldComponentUpdate issue. Redux and similar libraries have solved/worked around this issue, so try those out.

I'm not disagreeing with the intention of the message.  This as a sentence structure doesn't make sense:

If you aren't familiar with A, don't use B.

There's no implication of what to use instead of B because of the double negative.

If you are familiar with A, don't use B, use A instead.

Since this, context has become a stable API, so this isn't relevant any more. But thank you for the PR!


HA utility meter helper does not pick up device_class and unit_of_measurement

Question
Hi! I have made utility meter helpers for daily sums of my kWh consumption on various devices. These are "total kWh" states, just like the lifetime energy state from this integration.
The utility meters for the other sensors work as expected, they all show the meter values as kWh, and I can place them in the same history graph card. The utility meter sensors also show correct attributes for unit_of_measurement: kWh and device_class: energy.
However, this does not work when I make a utility meter for the lifetime energy. The utility meter sensor does not display the value as kWh, and the attributes for unit_of_measurement and device_class are just not there.
I did notice one difference between the input sensors though, but I couldn't find any meaningful documentation on this. All the sensors working correctly also has the two following attributes on them meter_type: 1 and meter_type_name: ELECTRIC
I also noticed they have the lightning bolt icon by default, while the lifetime energy has the counter icon by default.
Perhaps these values should be added to the lifetime energy state in this integration?
What version of the integration are you using?
v0.9.43
Anything in the logs that might be useful for us?
No response
Additional information
No response

I cannot reproduce the problem. It works as expected in my system.
The utility meter integration needs at least one full time cycle (hour, day or whatever you have selected) to pick up device_class and unit_of_measurement from the primary sensor.

How strange, but thank you for your input. The meter has been running for four days, with a daily cycle, but I did just notice I haven't actually charged my car for six days. I will update this thread as soon as I have charged my car, to see if that helps the situation.

Here are my sensors prior to charging tonight:

Now after charging, it looks better on the sensors, with unit_of_measurement and device_class on the daily meter:

Also, the graph was fine for a while, while I was charging. After charging, however, the graphs are getting weird. Although the sensors seems to have the right info, I now get two graphs on the meter sensor. Notice how they even overlap in time. I am still a bit of a HA newbie, so maybe someone understands what's going on here?

I realize this might not be an issue with the integration, but I haven't seen this happen to any of my other daily meters. So I still hope there's something that can be done here to fix it. Or perhaps someone with general HA knowledge can help me sort it out on my end?

This is most likely related to the Utility meter integration.
You will probably get better help with this issue by asking in the HA forum

It works as expected im my systems.
However, I don't see the point by adding the daily meter to the dashboard. The energy dashboard can read data from the  lifetime sensor directly and will calculate the consumed energy for the time interval selected.

Hmm you're right, not sure why I thought I needed to add the daily meter to the dashboard.
So it seems the solution was that I needed to have some actual consumption on the charger, and then wait another day or so for the graphs and data to sort themselves out :smiley:
Thank you for all your help!


Added NEXENTRO Switching Actuator Fingerprints

Added Nexentro Actuator:
NEXENTRO Switching Actuator

Hey @JuniorInsta I am seeing that there are some conflicts on this PR can you please resolve then we can merge? Thank you .

Hey @JuniorInsta I am seeing that there are some conflicts on this PR can you please resolve then we can merge? Thank you .

Done.


[BUG] TDVT action add_ds corrupts connection element of tds files

Describe the bug
When executing tdvt action --add_ds, the attributes of the connection element in the tds files are getting corrupted. In our case, the connection attribute class='actian_odbc' is being removed from the connection element as the attribute tdvtconnection=dsn123 is added. The tdvtconnection value (shown here as dsn123) doesn't seem to matter.
Research
To isolate the problem, I extracted the functions mangle_tds and updated_tds_as_str from setup_env.py into a simple test program parse2.py.txt. There were changes to this area of code in PR 1208.
Console Output of Test Program
C:\test> copy test.tds.orig test.tds

C:\test> python parse2.py
Modifying ./test.tds

C:\test> diff test.tds.orig test.tds
1c1
< <connection class='actian_odbc' dbname='testv1' one-time-sql='' port='21064' schema='abcschema' server='localhost' username='abctest' />
---
> <connection tdvtconnection='dsn123' dbname='testv1' one-time-sql='' port='21064' schema='abcschema' server='localhost' username='abctest' />

Desktop

OS: Windows 10
Tableau Desktop Versions: 2023.2.4 (20232.23.21215.0350) and 2023.3.1 (20233.23.1227.1336) both tested

Python & package versions
C:\test> python -V
Python 3.10.7

C:\test> pip list
Package    Version
---------- -------
defusedxml 0.7.1
pip        23.3.2
setuptools 63.2.0
tdvt       2.13.4

About you:
Name: Michael Habiger
Company: Actian Corporation

Thanks, I'll take a look.
Internal tracking:  W-14959403

@lukewrites I'm curious if you were able to reproduce the problem with the given test case.

We appreciate the updates @lukewrites