java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no leveldbjni64-1.8 in java.library.path

I am running opendaylight beryllium verison on windows 7 x64,  I and got such error when startup karaf framework.    I think leveldbjni crash because of  lack of some system dlls,  see the logs and screen capture.
E:\odl\distribution-karaf-0.4.3-Beryllium-SR3\bin>karaf
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; sup
port was removed in 8.0
________                       ________                .__  .__       .__

__
_____  \ ______   ____   ____ ______ \ _____  ..|  | || __ |  |_
/  |
/   |   ____ / __ \ /    \ |    |  _  <   |  ||  | |  |/ |  |
\   __
/    |    \  |> >  ___/|   |  |    `   / __ _  ||  ||  / //  >   Y
\  |
_______  /   / _  >|  /____  (____  / ||/_  /||
/|
/|**|        /     /        /     //            /__**/
/
Hit '' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '' or type 'system:shutdown' or 'logout' to shutdown OpenDaylight.
opendaylight-user@root>
opendaylight-user@root>
opendaylight-user@root>
opendaylight-user@root>
opendaylight-user@root>Uncaught error from thread [opendaylight-cluster-data-akk
a.persistence.dispatchers.default-plugin-dispatcher-22] shutting down JVM since
'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[opendaylight-cluster-d
ata]
java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no leveldbjni6
4-1.8 in java.library.path, no leveldbjni-1.8 in java.library.path, E:\odl\distr
ibution-karaf-0.4.3-Beryllium-SR3\data\cache\org.eclipse.osgi\bundles\178\1.cp
META-INF\native\windows64\leveldbjni.dll: Can't find dependent libraries, E:\odl
\distribution-karaf-0.4.3-Beryllium-SR3\data\tmp\leveldbjni-64-1-534075808401160
7636.8: Can't find dependent libraries]
at org.fusesource.hawtjni.runtime.Library.doLoad(Library.java:187)
at org.fusesource.hawtjni.runtime.Library.load(Library.java:143)
at org.fusesource.leveldbjni.JniDBFactory.(JniDBFactory.java:48)
    at akka.persistence.journal.leveldb.LeveldbStore$class.leveldbFactory(Le

veldbStore.scala:36)
at akka.persistence.journal.leveldb.LeveldbJournal.leveldbFactory(Leveld
bJournal.scala:20)
at akka.persistence.journal.leveldb.LeveldbStore$class.preStart(LeveldbS
tore.scala:112)
at akka.persistence.journal.leveldb.LeveldbJournal.preStart(LeveldbJourn
al.scala:20)
at akka.actor.Actor$class.aroundPreStart(Actor.scala:472)
at akka.persistence.journal.leveldb.LeveldbJournal.aroundPreStart(Leveld
bJournal.scala:20)
at akka.actor.ActorCell.create(ActorCell.scala:580)
at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263)
at akka.dispatch.Mailbox.run(Mailbox.scala:219)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:617)
at java.lang.Thread.run(Thread.java:745)
E:\odl\distribution-karaf-0.4.3-Beryllium-SR3\bin>

Hi any fix for the same ?
Thanks.

I ran into the same issue, after some investigation I (well mostly @vdmdmv) found that the problem was that the native source is dynamically linked against the Visual Studio 2010 runtime. It seems like a decent solution to this problem would be to package whatever runtime is being used like the native leveldb code gets packaged. The workaround @vdmdmv and I found was to just download and install the VS2010 redistributable package, after that everything worked fantastically

How to resolve this on Linux

@following

have you tried unzipping the jar and using ldd to figure out what runtime the jni library is linked against?

I instead used RocksDb which has the same API and was working fine. See here:
https://github.com/dalab/pboh-entity-linking/issues/4#issuecomment-318285695

I hit the same problem and the answer was that I included wrong library from maven. Instead
org.fusesource.leveldbjni:leveldbjni-all:1.8 it was org.fusesource.leveldbjni:leveldbjni:1.8. The latter doesn't contain any native libraries.

I hit the same problem and the answer was that I included wrong library from maven. Instead
org.fusesource.leveldbjni:leveldbjni-all:1.8 it was org.fusesource.leveldbjni:leveldbjni:1.8. The latter doesn't contain any native libraries.

should we give this

org.fusesource.leveldbjni
leveldbjni-all
1.8

in settings.xml of maven folder

For me, installing the Microsoft Visual C++ 2010 redistributable package resolved this issue


Feature Request / Question of File Templates and Images

Hi
Images
It would be very nice to add images into generated package.
As example we have to add our company logo and a base product logo into every odoo module which is generated. And this is forgotten to many times...
File generation
Another problem that I have is to generate the __init__.py files based on selected folders.
An odoo Module has the following structure:

/module

__init__.py
/models/__init__.py
/controllers/__init__.py
/wizards/__init__.py

Now when I do not need e.g. the wizards folder, I disable this folder in package template Popup.
But in root __init__.py I have to add (or remove) all (not) needed imports for sub packages.
Is there a possibility to automize the generation of __init__.py in root by selected folders?
Regards
Renzo

Hi, thx for feedback. I will try to develop some features for your request. P.S. What you mean by imports for sub packages, TextInjections?

I mean the root __init__.py file should look like:
import models
import controllers
import wizards

or when e.g. controllers folder is not used:
import models
import wizards

Something like a live template which can loop over subfolders...

Oh.. i miss this feature too and already have some ideas about possible solution. Currently you can use dummy variables + Velocity engine (Supported in FileTemplates by default) as temporary solution.
e.g. global var ADD_WIZZARD
then in FileTemplate check value of variable:
#if (${ADD_WIZZARD} && ${ADD_WIZZARD} != "")import wizards#end

If ADD_WIZZARD isn't empty import will be added, but approach with variables still independent from selecting folders, so you should manually control both conditions.

Oh nice to read that you already have some ideas...
The usage of variables is a solution but not not realy one that I will use... I don't like to set twice to not use a folder...
So I will wait for another solution of you ;-)))
Thanks a lot for this time saving plugin!


14.0-CURRENT client guide is failing

There have been issues with the uploading of new images to GCE (which is what Cirrus CI uses), see PR 272354.
Let's wait until the dust settles and re-run the failed tests.

Re-run failed tests.
All passing.


Allthemodium

1.20.1 allthemodium not working .

Version 1.20.1 currently does not support allthemodium, I will support it in the future.


add evergreen integration

As discussed, a minimal working evergreen setup suffices for now, as long as it conducts static analysis and unit testing

I assume you weren't able to test this, since there is no Evergreen project. If so, let's just commit this and then iterate, once we see what the actually build looks like.

I could test locally by choosing the branch name (together with evergreen config file path) as below in the project settings page:

That is how I tested it previously. will try to investigate whether we could see the CI effect here by github integration option in the project settings as well.

I tested this branch indirectly by creating a separate temporary PR at https://github.com/mongodb/mongo-hibernate/pull/5. Now testing succeeded and I closed that testing PR. Latest commits in this PR reflected the issues flushed out during the testing.

I rebased on the main branch and made the following changes:

change palantir change to palantirJavaFormat().formatJavadoc(true)
delete the '.editorconfig' file for it was based on google-java-format. There is no .editorconfig file for palantir, so no bother

FYI, if you force push like this, reviewers are not able to see what has changed since the last review.

yeah, a separate PR is justified. Will create one.
On Mon, Oct 21, 2024 at 4:09 PM Jeff Yemin @.***> wrote:

@.**** requested changes on this pull request.
In gradle/libs.versions.toml
https://github.com/mongodb/mongo-hibernate/pull/4#discussion_r1809452262
:

@@ -4,7 +4,6 @@
[versions]
junit-jupiter = "5.10.3"
spotless = "6.25.0"
-palantir = "2.50.0"

Why don't we need this? Is it because it's picked up from Spotless?
In build.gradle.kts
https://github.com/mongodb/mongo-hibernate/pull/4#discussion_r1809454369
:

@@ -62,7 +62,7 @@ spotless {

     removeUnusedImports()

   palantirJavaFormat(libs.versions.palantir.get()).style("GOOGLE").formatJavadoc(true)

   palantirJavaFormat().formatJavadoc(true)

As this doesn't have anything to do with Evergreen, can we do a separate
PR for it?
—
Reply to this email directly, view it on GitHub
https://github.com/mongodb/mongo-hibernate/pull/4#pullrequestreview-2383208500,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB6UYAR76EDHZ2EHNJYPJCLZ4VNPBAVCNFSM6AAAAABP76HEFGVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDGOBTGIYDQNJQGA
.
You are receiving this because you authored the thread.Message ID:
@.***>


Can (and should):mng be based on an image built from hilbert-ui's Dockerfile?

Eric's suggestion:

Isn'it it better if the Dockerfile in hilbert-ui specifies it's requirements, volumes, port, etc. and then the Dockerfile for mng just uses
FROM hilbert-ui
???
That way the Dockerfile in hilbert-ui specifies its specific requirements, way of building, etc. and the one in mng only any extra things needed to interact with the rest of the system that are beyond the scope of hilbert-ui (if any).
Isn't this the idea of basing images on other images? Like a form of class inheritance?

We currently provide / share

built infrastructure: DockerHub automatic builds and local Makefiles for local development
Execution run-time via docker-compose
Therefore i would not like some completely different official image for no good reason.

Separation of a single :mng image was already discussed:
it would be nice to have separate

:backend with hilbert-cli-server and Dashboard's Backend server and
:dashboard - just the Web UI part
if easily possible.

As for the hierarchy: AFAIK Dashboard's Back-End server requires hilbert-cli-server management tool not the other way around. Therefore i would propose the following (new official images inside hilbert-docker-images):

:backend-cli with hilbert-cli-server for direct testing of hilbert-cli-server and low-level configuration management
:backend-dashboard - base on above, with Dashboard's Backend server.

I don't understand how what you're saying relates to my question and I don't care to discuss the structure of this repository, so I'll ask it differently:
Since:

Building the mng image has lines specific to hilbert-ui (npm install, exposed ports, etc.) that might change with future versions of hilbert-ui
I understand that an image can be based on another image adding extra things on top of it, as a form of inheritance.

Isn't it a good idea to:

Base the mng image on a hilbert-ui image built through the hilbert-ui Dockerfile?
Perhaps base the hilbert-ui image on a hilbert-cli image that would be ubuntu + hilbert-cli?

Sorry, i really don't like the idea of diverging from existing & working approach to managing docker images! Too many downsides due to using something directly unmanageable in the middle of image inheritance chain!

Base the mng image on a hilbert-ui image built through the hilbert-ui Dockerfile?

Note that currently mng image is based on hilbert-ui's Dockerfile but extends hilbert-ui with hilbert-cli-server tools and is consistent with our run-time!
So why not stick to hilbert-docker-image/images/mng and update it whenever necessary.

Perhaps base the hilbert-ui image on a hilbert-cli image that would be ubuntu + hilbert-cli?

Yes. Possible (but hilbert-cli should better be based on nodejs).
May we consider a separate image, say backends, which is based on nodejs combining

dashboard's back-end server (e.g. without nginx and WebUI content) with
hilbert-cli-server tools ?

Then the remaining client parts of hilbert-ui (e.g. nginx and static WebUI content) would
require no special basement and can just link with backends though network in run-time.
What do you think about such a separation of hilbert-ui into 2 loosely connected parts?
Would it be 1. possible 2. sensible 3. simple thing to do?


Add an interface for retrieving block data

Something like
impl SystemContextHandle { // or SystemContext
  // None: definitively unknown/missing/unrecoverable block, e.g. could be a block commitment that hasn't been sequenced
  // Err: something went wrong when trying to fetch, you should try again later, e.g. network error
  async fn recover_block(comm: Commitment<Block>) -> Result<Option<Block>, Error>;
}

This should automatically try all the layers of Tiramisu:

query the CDN
query any nodes that were known to be on the DA committee for this block (if it helps, the caller could also provide the leaf or view number or DAC so that HotShot can figure out who was on the committee)
initiate VID

Reason for an active query rather than a new event type is that not all nodes will actually want the full block, so we don't want to initiate this unless asked for it. We do not want to automatically, passively recover every block in order to emit events.

We will do this at the application level


Add support to async and await

Coverage remained the same at 100.0% when pulling f8e397bfa12d23fbc387a215e0e59ff4b8863e02 on feature/add-async-support into e089ab6ea7485d2793e6c8948473e392aaf0cf0f on master.


WIP: Add scale test

Add a distributed dns with multiple namespaced dns operator deployments scale test using kube burner.

@jsmolar @trepel this is how we are looking at approaching automatic scale testing


"Unknown error" when writing .img.xz file on Windows

Etcher version: 1.0.0-beta.10
Operating system and architecture: Windows 10 x64, Insider Preview build 14385
Do you see any meaningful error information on DevTools? Do I see anything? Yes. Meaningful? No. Just an error object with the same information as shown below.

When I attempt to flash an image (.img.xz) to an SD card, the operation often opens the UAC dialog for verification and then sits there with the message "Starting..." until I quit the application. When it gets past that, it progresses to around the 80% mark and then slows, before throwing an error (pictured below). The image that was written is corrupted, so Etcher did not successfully complete the imaging operation by the time the error was thrown.
I can't find any useful info in the dev console and there are no other logs as far as I can tell, so past this screenshot I don't have anything to go off of.

Hi @WasabiFan ,
Thanks for reporting this issue and the valuable information. Can you share a link to that exact same image you're trying to flash to see if I can reproduce? Also, you mention a .img.xz, but in the screenshot, I see an uncompressed .img instead.
Does this happen only with this image? If so, do you have another USB Drive or SD Card to try burning this exact same image to it, and see if the error still happens?
Regarding DevTools, is there a stack trace when expanding the Error object?

Can you share a link to that exact same image you're trying to flash to see if I can reproduce?

Yep, it's here. It is the one called ev3-nightly-2016-07-13-bootroot.img.xz. The purpose of burning these images is to test this new nightly build system, so it is possible that the image is invalid; keep that in mind. Obviously, regardless of its validity, the burner should never throw these errors if it can be helped.

Also, you mention a .img.xz, but in the screenshot, I see an uncompressed .img instead.

Ah, unfortunate window placement :wink: It is a .img.xz.

Does this happen only with this image? If so, do you have another USB Drive or SD Card to try burning this exact same image to it, and see if the error still happens?

I tried burning an image file that I know has worked with Etcher before, but I hit that "Starting..." bug 5 times in a row and gave up. Trying again, it happened another 4 times on that image and is continuing to happen on the original image as well. It got past the "Starting..." issue once after reinstalling Etcher (using the original image that I linked to), then started failing with various other errors which I believe there are already issues for. When I run it as administrator manually, I get this:

After reinstalling just now, I got this (which is again covered by #552):

None of it is reproducible or consistent, but it's definitely very broken. I have not yet gotten the original error on a known-good image, but I haven't gotten to the burn step either, so that doesn't tell us much. It does seem that the chance of getting to the 80% mark increases if Etcher has been freshly installed and was launched by the installer, but that is only conjecture. I have yet to get past the "Starting..." stage if I was prompted by UAC previously.
I can't get to the stage that would let me repro the original bug on a different storage device, either.
Side note: The bug where the button just says "Starting..." seems to be caused by a race condition or other oversight that is making the main application wait for the child process indefinitely.

Regarding DevTools, is there a stack trace when expanding the Error object?

There is not, however if I add some breakpoints I can get a bit more information on what's happening. This is as much useful context as I can get:

But that doesn't seem to give much insight into what is actually happening.

Hi @WasabiFan ,
Thanks a lot for the great amount of details about the issue.
I have no idea what could be causing this, but I see that just very few certain people are experiencing it, which makes me think there is a common system setting or something along those lines.
The application gets stuck at "Starting..." because it forks the writer process, and waits for it to emit progress state back to the parent process. I see you mentioned you were running the application as administrator. What happens if you run the application as a normal user, and go though the UAC elevation dialog (you do get it right?)?
I'll make a custom Etcher build for you from master with extra debugging information, to see if we can get to the bottom of it.

I'm uploading the custom build. In the mid-time, can you test v1.0.0-beta.9? If the previous version works we can try bisecting Etcher to try to find the culprit. You can download v1.0.0-beta.9 from https://resin-production-downloads.s3.amazonaws.com/etcher/1.0.0-beta.9/Etcher-win32-x64.exe

Since you are more developer-oriented, you might want to give the Etcher CLI a try, and see if that makes any difference. The Etcher CLI currently lives in this same repository, although is not being property released yet.
You can make use of it by cloning the repo and installing dependencies, as described in https://github.com/resin-io/etcher/blob/master/docs/CLI.md.

I have no idea what could be causing this, but I see that just very few certain people are experiencing it, which makes me think there is a common system setting or something along those lines.

I know that I am running Windows Insider Preview builds, which can be unstable and break existing applications. Although I'd doubt that that is the problem as Microsoft is currently in the final stages of stabilizing for their Anniversary Update, it is something to consider. It's also possible that they changed something small intentionally that has side-effects.

I see you mentioned you were running the application as administrator. What happens if you run the application as a normal user, and go though the UAC elevation dialog (you do get it right?)?

Yes, I've tried launching it from the installer, from the Start menu, from CMD, and from Start as administrator. Launching it as admin seems to completely break Etcher, running it from CMD is no different from Start, and running it from the installer seems to increase the likelihood of getting past the "Starting..." message.  The reason that I tried launching as administrator was because I noticed that I wasn't prompted when it was launched from the installer, and given my previous success on first install I posited that running as administrator may solve the problem. Interestingly, it seems to behave differently depending on whether it was run by the installer as administrator or by me as administrator. Again, this is all simply conjecture (and loosely-based conjecture at that) so don't consider these ramblings as fact.

I'll make a custom Etcher build for you from master with extra debugging information, to see if we can get to the bottom of it.

OK, sounds good! I'm happy to try that out and give you what you need.

In the mid-time, can you test v1.0.0-beta.9? If the previous version works we can try bisecting Etcher to try to find the culprit.

Yep, I'm working on testing the previous version now. However, assuming that you are referring to git bisect, I don't know how to run an Electron-based app from source; I'd need some tips on how to do that, but I'm willing to try the bisect if I can get some help with that build operation. Additionally, there are multiple issues here -- the two screenshotted most recently, the "Starting..." thing, and the nondescript error when it gets to 80%. Which would I be looking for?

Although I have no idea how Etcher and its modules are architected, it might simplify debugging in the future if the child process logged to a log file independent of the main Etcher application so that one could access more detailed textual logs even if the GUI was having problems or doesn't handle that information.

Since you are more developer-oriented, you might want to give the Etcher CLI a try, and see if that makes any difference. The Etcher CLI currently lives in this same repository, although is not being property released yet.

I'll try that later today and see how it goes.

I know that I am running Windows Insider Preview builds, which can be unstable and break existing applications. Although I'd doubt that that is the problem as Microsoft is currently in the final stages of stabilizing for their Anniversary Update, it is something to consider. It's also possible that they changed something small intentionally that has side-effects.

I doubt this is the issue. I remember a user reporting this one from Windows 10 Home.

Yep, I'm working on testing the previous version now. However, assuming that you are referring to git bisect, I don't know how to run an Electron-based app from source; I'd need some tips on how to do that, but I'm willing to try the bisect if I can get some help with that build operation. Additionally, there are multiple issues here -- the two screenshotted most recently, the "Starting..." thing, and the nondescript error when it gets to 80%. Which would I be looking for?

Bisecting Etcher yourself might be a pain, since you need to compile from source on every step (which requires Visual Studio 2013). I'm happy to assist you on it if you really want to do it, but otherwise lets hope my custom build reveals something worthy.

Although I have no idea how Etcher and its modules are architected, it might simplify debugging in the future if the child process logged to a log file independent of the main Etcher application so that one could access more detailed textual logs even if the GUI was having problems or doesn't handle that information.

Actually, the writer child process writes the progress to a temporary file, which is then tailed by the parent process. I'm logging the temporary file name on the custom build, so should be able to open it manually and see what's going on if needed.

Here's the custom build: https://www.dropbox.com/s/lao6mcn1k0gvjyx/Etcher-win32-x64.exe?dl=0

Downloading now. Anything special I should do (should I run the application from the command line?)?

Nothing special should be necessary. Just install and run as normally.

BTW, thanks a lot for all the help so far. You're being super helpful and valuable debugging this issue! Very appreciated :)

BTW, thanks a lot for all the help so far. You're being super helpful and valuable debugging this issue! Very appreciated :)

No problem! I know how hard it can be to attempt to provide project support to someone who can't stick with it to test things, and am willing to put the time in to make sure that this gets resolved. If it would be easier to chat in Gitter (as I see Etcher has a Gitter room), I'd be available for that in an hour or two.

After installing the test build, I allowed the installer to launch Etcher directly. I picked the image and it got to the 80% mark and then threw the original error, as expected. The UAC dialog did not come up, as presumably the installer launched it as admin. It doesn't look like there was much of interest that came out of the logs, but it does validate that the process is being elevated and seems to be running fine up until then. It's also worth noting that at least part of the image was successfully written, it's just not complete (the target device boots and displays the splash screen but then throws boot errors and panics).
You can find the output from both the dev console and the log file here.
I ran it again as a normal user and it threw the Command failed: diskpart /s... error. Running that command manually from an admin command prompt works without error. As one would assume, the console says STDOUT: Is elevated? false... tracker.js:10 STDOUT: Elevating! and I was prompted to allow it.
I ran it a third time, this time as administrator. It threw the "child process exited..." error.  The dev console looked like this:

And the child process log only had a single line:
{"command":"error","data":{"message":"Command failed: diskpart /s \"C:\\Users\\<username>\\AppData\\Local\\Temp\\_diskpart-1468446780855\"\n"}}

Interestingly, this seems to be the same internal error as occurred previously, but something masked the internal error and instead just displayed the message about the process dying.
One thing I notice here is that there's a \n at the end. Is that line feed included when the command is executed? Given that the command runs fine when I do it manually, one thing to consider is that Windows may be unhappy about the \n instead of \r\n. Moreover, is it possible that those characters (\ and n) are literally being sent?

I am unable to find any builds of beta 9. I have a previously-downloaded copy of the beta 7 installer, but beta 9 (or any of the older versions) doesn't seem to be available anymore. If you can point me to a place I can find those installers, I can test out the older versions too.

Etcher version: 1.0.0-beta.10
Operating system and architecture: Windows 10 x64,  Version 1511 (Build 10586.494)
same problem as op trying to write debian-8.5.0-amd64-netinst.iso to a sandisk cruze 4gb flashdrive.
if you need more info from me let me know and I'll post it, still noob to all of this stuff.

@jeffiscow Is it the same error in the developer console? Does it also stop at 80%?
The developer console can be accessed via Ctrl+Alt+I

@jviotti New piece of information: The same error, aside from slightly different messages as a result of more recent changes, occurs for me in beta 7. It seems like it might offer a slight bit more information -- you can see the error in the screenshot below. This does however show that this isn't a new issue, and judging by @jeffiscow's comment, it happens on vanilla WIndows 10 too.

@WasabiFan im sorry but where in the dev console do I look?

Ah, I see -- yep, that's one of the issues that I'm seeing.

Thank you very much for the huge help on this issue. You guys rock! This is all extremely weird, and I don't have anything in mind yet. I'll be focusing 100% of my time on this issue from now on, so we'll hopefully get it fixed very soon.
I'll try to get more clues about what's going on and will report back, and probably ask you some questions, or ask you to test anything soon.
Thanks a lot again!

If you really have some spare time (you don't have to do it), you could test every Etcher version (there are not many luckily yet :) ) to attempt to identify in which one the issue started appearing:

[ ] v1.0.0-beta.10: https://resin-production-downloads.s3.amazonaws.com/etcher/1.0.0-beta.10/Etcher-win32-x64.exe
[ ] v1.0.0-beta.9: https://resin-production-downloads.s3.amazonaws.com/etcher/1.0.0-beta.9/Etcher-win32-x64.exe
[ ] v1.0.0-beta.8: https://resin-production-downloads.s3.amazonaws.com/etcher/1.0.0-beta.8/Etcher-win32-x64.exe
[ ] v1.0.0-beta.7: https://resin-production-downloads.s3.amazonaws.com/etcher/1.0.0-beta.7/Etcher-win32-x64.exe
[ ] v1.0.0-beta.6: https://resin-production-downloads.s3.amazonaws.com/etcher/1.0.0-beta.6/Etcher-win32-x64.exe
[ ] v1.0.0-beta.5: https://resin-production-downloads.s3.amazonaws.com/etcher/1.0.0-beta.5/Etcher-win32-x64.exe
[ ] v1.0.0-beta.4: https://resin-production-downloads.s3.amazonaws.com/etcher/1.0.0-beta.4/Etcher-win32-x64.exe
[ ] v1.0.0-beta.3: https://resin-production-downloads.s3.amazonaws.com/etcher/1.0.0-beta.3/Etcher-win32-x64.exe
[ ] v1.0.0-beta.2: https://resin-production-downloads.s3.amazonaws.com/etcher/1.0.0-beta.2/Etcher-win32-x64.exe
[ ] v1.0.0-beta.1: https://resin-production-downloads.s3.amazonaws.com/etcher/1.0.0-beta.1/Etcher-win32-x64.exe

Earlier versions were very unstable, so I wouldn't be surprised if they don't even work at all or exhibit other bugs that were fixed in newer versions :) Testing down to v1.0.0-beta.5 is probably enough.

The error code we're getting is 4094. Sadly, Microsoft is really bad at giving more information about error codes, but this is what I've found:

The windows error code 4094 is generally generated due to errors reported by the driver itself meaning that this error is a software problem or compatibility issues between the hardware installed and the system being used. And it is very common and it is occurs when the operating system and the drivers installed are not companionable with each other.

I'd try opening the Device Manager and seeing if the operating system is reporting any errors for the drive you're trying to flash to. Did you test with other drives btw? Did the issue still happen?

4094 is not even documented in https://msdn.microsoft.com/en-us/library/windows/desktop/ms681387(v=vs.85).aspx :sigh:

If there was a weird I/O error with the drive, then diskpart failing might actually make sense. While the drive is connected, can you try the following?

Run diskpart.exe.

Run list disk to list the available drives. They'll have an ID number that should take note of.

Run select disk N, where N is the ID number above (make sure you type the right one to avoid wiping up your hard-drive!).

Run clean.

Run rescan.

Did diskpart complained in the process?

I've uploaded a new custom build based on the previous one that includes extra debugging information to attempt to find out if diskpart provides any information about the reason it fails: https://www.dropbox.com/s/6b8g9jrx4157iub/Etcher-win32-x64-14-July-DISKPART-DEBUG.exe?dl=0

I'l try the the things suggest above and get back to you when I get home in about 2 hours

I manage to reproduce the diskpart and the Child process exitted with code: 1 a couple of times. It seems the recipe is:

Clean the drive with diskpart.
Create a "New Simple Volume" using diskmgmt.msc.
Run Etcher as Administrator.

Any variation of those steps causes the error to not appear for me.
I didn't find much about the cause yet, but I got the Child process exitted with code: 1 one and by manually inspecting the writer process logs, I can see its actually the same diskpart issue.

The reason why the error gets sometimes masked seems to be because the child process emits the close event faster than the data coming from stderr at https://github.com/resin-io/etcher/blob/master/lib/src/child-writer/index.js#L92.
I believe this happens because of the interval when tailing the log file. We should ensure somehow that all data from the log file has been read and emitted before killing the process.

Damn, I can't reproduce anymore, even with the steps outlined above :/

beta 9: press "flash!" go to "starting..." the quickly back to "flash!" with nothing being written to flashdrive
beta 8: same as 9
beta 7: pressed the "flash!" button popup titled "Error" "An error ocurred! button is stuck on "Starting..."
beta 6: go to flash get error https://drive.google.com/open?id=0B-Pp_HBf1mVBRjJGak9id01ES0k
beta 5: press the "flash" button gets stuck on "Starting..."
14-July-DISKPART-DEBUG.exe: error https://drive.google.com/open?id=0B-Pp_HBf1mVBYnR4QzZ4OWxrWDA
hope this helps

If you really have some spare time (you don't have to do it), you could test every Etcher version (there are not many luckily yet :) ) to attempt to identify in which one the issue started appearing:

I can try out the older ones later today. Keep in mind that there are multiple issues we've mentioned here -- the "Starting..." thing, the "child process exited" error, and the generic "write" error at 80%. My guess is that the first two might be the same thing (and just the behavior is being differentiated by a race condition) but the latter one definitely isn't exactly the same.

I'd try opening the Device Manager and seeing if the operating system is reporting any errors for the drive you're trying to flash to. Did you test with other drives btw? Did the issue still happen?

Yes, I have tested with other devices; all of the issues previously mentioned also happen on other devices. Windows seems happy with the device based on the information in device manager.

If there was a weird I/O error with the drive, then diskpart failing might actually make sense. While the drive is connected, can you try the following?

I tried that and it worked without issue yesterday; I tried both running the exact command that Etcher runs (the one that passes the command file to diskpart) and running them manually, and it worked repeatedly without fail. When running the Etcher GUI or CLI though, I was seeing this error very frequently -- so much so that I couldn't get past it to test the error at 80%.

I manage to reproduce the diskpart and the Child process exitted with code: 1 a couple of times. It seems the recipe is...

I have been doing something similar; because Etcher (i.e. the utility that Etcher uses) only recognizes the drive if it already has a partition on it, I have been re-formatting the drive after each attempt to repro these issues because Etcher dies somewhere after wiping the drive. I haven't been running as administrator, but I have been doing the other two things.

The reason why the error gets sometimes masked seems to be because the child process emits the close event faster than the data coming from stderr at ...

Why don't you use stdout to receive progress events instead of the file? I imagine that would be more robust, but presumably you're doing it this way because that didn't work right.

I ran the new debug build and both the dev console and the log file are exactly the same as they were before. Where is the new debug information supposed to be?
On a whim, I decided to check the Windows event log to see if anything had been showing up there. Sure enough, I see a bunch of errors like this that correspond to me seeing the diskpart error:
Cannot zero sectors on disk \\?\PhysicalDrive3. Error code: 5@0101000F

This seems to be the most direct information on the root cause that we've seen. There are other mentions of that error string online, but based on my quick search, none of them seem applicable to our situation. I have tried running disk check, but it says both drives are fine. I still plan to do some more searching.

Based on what @jeffiscow posted (I haven't gotten a chance to try older versions myself) it appears that the diskpart error goes back to at least beta 6, maybe 5. That's interesting, as I was previously able to flash cards with Etcher without issue until I tried earlier this week.

@WasabiFan Windows updated last night could that be a cause?

It is possible that a Windows update is causing this, but I believe we are on different versions of Windows, so that seems unlikely.

I have been doing something similar; because Etcher (i.e. the utility that Etcher uses) only recognizes the drive if it already has a partition on it, I have been re-formatting the drive after each attempt to repro these issues because Etcher dies somewhere after wiping the drive. I haven't been running as administrator, but I have been doing the other two things.

Looks like drivelist considers a drive that doesn't have a file system as non-removable. I'll fix it very soon, thanks a lot for raising this up.

On a whim, I decided to check the Windows event log to see if anything had been showing up there. Sure enough, I see a bunch of errors like this that correspond to me seeing the diskpart error:
Cannot zero sectors on disk \\?\PhysicalDrive3. Error code: 5@0101000F

Amazing discoveries! I didn't know about the Windows Event Log thing (I'm a Mac guy), super helpful. I'll investigate why this is the case.

I've been doing some investigation, and it seems that you have to make Windows "repair" the drive so clean succeeds. I'm still struggling to manage to reproduce the issue again, but I wonder what'd happen if we run chkdsk <driveletter>: /f right before clicking "Flash". Does that solve the issue?
Alternatively, I'm thinking the clean operation fails because Windows doesn't acknowledge recent changes to the drive (e.g: a format change). Another thing to try would be to run the rescan diskpart command before clicking "Flash".

The chkdsk /R option might be worth testing as well:

/R Locates bad sectors and recovers readable information (implies /F).

According to this guy, running clean again after the error does the trick. We could run the command once, and if it fails, retry.

Looks like drivelist considers a drive that doesn't have a file system as non-removable. I'll fix it very soon, thanks a lot for raising this up.

Ah, that's what it is called! I remembered reading about that module somewhere in this repo but couldn't remember the name. The file system discovery issue isn't a big problem, but yeah it would probably make it a bit easier.

I didn't know about the Windows Event Log thing (I'm a Mac guy), super helpful. I'll investigate why this is the case.

Haha yeah... it's a place that one rarely wants to go, because seeing the event log breaks the illusion that your PC is running fine. It's often full of error messages that one really doesn't want to know about :laughing:
It's very odd that diskpart doesn't return any of that information automatically, and instead makes you go searching for it. Definitely doesn't make debugging any easier!

I've been doing some investigation, and it seems that you have to make Windows "repair" the drive so clean succeeds. I'm still struggling to manage to reproduce the issue again, but I wonder what'd happen if we run chkdsk : /f right before clicking "Flash". Does that solve the issue?

I tried repairing and it didn't report any issues, but I haven't tried Etcher since, so it may have done something anyway. I can try that soon.

Yesterday I tried several times to repro the error using diskpart on the command line to no avail. Just now, however, I was able to do it, so I did some testing. This seems to indicate that you are either correct in that the rescan fixed it or in that the second clean did. See below.
DISKPART> list disk

  Disk ###  Status         Size     Free     Dyn  Gpt
  --------  -------------  -------  -------  ---  ---
  Disk 0    Online          238 GB      0 B        *
  Disk 1    Online          931 GB  1024 KB
  Disk 2    Online          931 GB      0 B
  Disk 3    Online         1901 MB      0 B

DISKPART> select disk 3

Disk 3 is now the selected disk.

DISKPART> clean

DiskPart has encountered an error: Access is denied.
See the System Event Log for more information.

DISKPART> rescan

Please wait while DiskPart scans your configuration...

DiskPart has finished scanning your configuration.

DISKPART> list disk

  Disk ###  Status         Size     Free     Dyn  Gpt
  --------  -------------  -------  -------  ---  ---
  Disk 0    Online          238 GB      0 B        *
  Disk 1    Online          931 GB  1024 KB
  Disk 2    Online          931 GB      0 B
* Disk 3    Online         1901 MB  1900 MB

DISKPART> clean

DiskPart succeeded in cleaning the disk.

Looks promising! I'll quickly make a custom build that retries clean if an error happens, and we'll see how it goes.

I ran the new debug build and both the dev console and the log file are exactly the same as they were before. Where is the new debug information supposed to be?

Agh, I hot edited a dependency and then re-installed dependencies when packing, silly :)

I've just uploaded three custom builds:

Retry once if clean fails: https://www.dropbox.com/s/8tpmvlmmsqap0x3/Etcher-win32-x64-ONE-RETRY.exe?dl=0
Retry up to ten times if clean fails: https://www.dropbox.com/s/hkhn2cx9dn595k6/Etcher-win32-x64-TEN-RETRIES.exe?dl=0
Run rescan before clean: https://www.dropbox.com/s/ezaw74x1g41173z/Etcher-win32-x64-RESCAN-BEFORE-CLEAN.exe?dl=0

Hopefully one of these fixes the issue.

Wow, covering all your bases! I'll try them and report back.

Using the "One retry" version, I ran it 6 times and every time it got past the "Starting..." phase when it previously threw the diskpart errors. I tried multiple drives and various other configurations, and every time it worked as it should have up to that point. It did not fix the later error (the original one mentioned in this issue), but as far as I can tell the diskpart ones are resolved. Reinstalling the standard beta 10 brought the error back first try.
So, I think that's the fix to go with. However, it would be great if it warned you in the dev console that it was retrying so that it was clear that it had hit that problem. That way, we could be sure that the retry logic was in fact activating and future debugging would probably be easier as well.
I think that, based on what I saw with the one-time version, ten times is probably unnecessary. Would you like me to also try the one that rescans?

Now, I think the main thing that we have to figure out is the "unknown error, write" issue, which now happens every time for me. @jeffiscow would you mind installing the version that @jviotti linked to entitled "ONE RETRY"? I'd like to know if you now see an error there like I do or if it works without issue.

@WasabiFan im getting a "Command failed: diskpart /s" "C:Users\USERNAME\AppData\Local\Temp_diskpart-1468535173355"
https://drive.google.com/open?id=0B-Pp_HBf1mVBZ0k3eVZaRVdvRW8

Did you install Etcher from the link that @jviotti provided (https://www.dropbox.com/s/8tpmvlmmsqap0x3/Etcher-win32-x64-ONE-RETRY.exe?dl=0)?

reinstalled to double check
Error: Child process exitted with error code: 1
    at ChildProcess.child.on (C:\Program Files (x86)\Etcher\resources\app.asar\lib\src\child-writer\index.js:106:38)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:191:7)
    at maybeClose (internal/child_process.js:850:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:215:5) `undefined`

Interesting... I just tried it a few more times and I am still unable to get that error while running this new version.

@WasabiFan Thanks a lot for the testing. Its awesome to hear the bug is gone for you. Regarding the build that rescans, if you can test that one, that'd be amazing. If the rescan one works fine as well I'd definitely go with that, given its a more elegant solution to the problem.
@jeffiscow Hm, I guess one retry is not enough for you. Do you mind quickly testing the other two builds I uploaded above as well (one that retries up to 10 times, and one that rescans the drive before attempting to clean it)?

@WasabiFan

Now, I think the main thing that we have to figure out is the "unknown error, write" issue, which now happens every time for me.

Yeah, lets go for it now. Does it happen with every image/drive combination? I'd be really interested to hear if you can reproduce it with the CLI (https://github.com/resin-io/etcher/blob/master/docs/CLI.md).

if you can test that one, that'd be amazing. If the rescan one works fine as well I'd definitely go with that, given its a more elegant solution to the problem.

Unfortunately, after running it a few times, it looks like the rescan does not fix the problem; I got the diskpart error every time.

Yeah, lets go for it now. Does it happen with every image/drive combination?

Using an image that I know has worked with Etcher in the past, the flash completed successfully. However, running it using the image that I was trying originally, it consistently throws the error.
Doing a bit of digging, I think I have uncovered the problem: the image is 4GB instead of the ~0.5 that I was expecting. Burning a 4GB image to a 2GB card is obviously not going to work. So, switching to a 16GB card, Etcher finished successfully.
The takeaway here is that I think Etcher needs to validate that the image you are trying to use will fit. Can that be done easily? Or is it already supposed to be happening?

I guess it's time to open an issue on the project that created the image to figure out why the images have grown 😆

I do believe Etcher already guards against that case. I wonder why it
didn't in this case, and if it may be linked to the issue.
--
Alexandros Marinos
Founder & CEO, Resin.io
+1<PHONE_NUMBER>
@alexandrosm
On Fri, Jul 15, 2016 at 11:38 AM, Wasabi Fan<EMAIL_ADDRESS>wrote:

if you can test that one, that'd be amazing. If the rescan one works fine
as well I'd definitely go with that, given its a more elegant solution to
the problem.
Unfortunately, after running it a few times, it looks like the rescan does
not fix the problem; I got the diskpart error every time.
Yeah, lets go for it now. Does it happen with every image/drive
combination?
Using an image that I know has worked with Etcher in the past, the flash
completed successfully. However, running it using the image that I was
trying originally, it consistently throws the error.
Doing a bit of digging, I think I have uncovered the problem: the image is
4GB instead of the ~0.5 that I was expecting. Burning a 4GB image to a 2GB
card is obviously not going to work. So, switching to a 16GB card, Etcher
finished successfully.
The takeaway here is that I think Etcher needs to validate that the image
you are trying to use will fit. Can that be done easily? Or is it already
supposed to be happening?
I guess it's time to open an issue on the project that created the image
to figure out why the images have grown 😆
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/resin-io/etcher/issues/571#issuecomment-233033358,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABLUCMfdocWjcwqOHuvJQJt1JH4r3DzVks5qV9OagaJpZM4JLE0-
.

We indeed make that check. I'm thinking there is either some subtle way to bypass it accidentally or drivelist, which is module reporting the drive size, is reporting an incorrect size for that specific drives.
Can you run this script (https://github.com/resin-io-modules/drivelist/blob/master/scripts/win32.bat) when the drive is plugged in and report back that it outputs?
Also, did the problematic drive used to get auto-selected? If so, can show us what you get when clicking the "Change" label below the second drive? The label opens the drive selector modal, which should show a warning label if it detects the drive is not large enough.

Here's the drive-specific output for one of them:
device: "\\\\.\\PHYSICALDRIVE3"
description: "Generic Flash Disk USB Device"
size:<PHONE_NUMBER>
mountpoint: Null
name: Null
system: True

And the other one:
device: "\\\\.\\PHYSICALDRIVE3"
description: "Multi Flash Reader USB Device"
size:<PHONE_NUMBER>
mountpoint: "H:"
name: "H:"
protected: False
system: False

Etcher correctly determines that the drive is 2GB in the menus as well.

I'm glad to hear that you found the problem! Happy I could be helpful.


Improve CLI XML

This improves the XML specifications for BackgroundIntensity and SeparateStainsMacenkoPCA, namely by:

Specifying the input images with <image> rather than <file>
Removing the minimum constraint on --samplePath, allowing the special default value of -1 to be valid.

Codecov Report

Merging #314 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #314   +/-   ##
=======================================
  Coverage   45.15%   45.15%
=======================================
  Files          99       99
  Lines        2695     2695
  Branches       98       98
=======================================
  Hits         1217     1217
  Misses       1478     1478

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6a7d593...7930859. Read the comment docs.


Not installing on Win7 64 - "node scripts/install.js"

Getting the following error, not sure what's going on, but on Ubuntu/OS X installs with no problem.
npm ERR! Failed at the<EMAIL_ADDRESS>install script.
npm ERR<EMAIL_ADDRESS>install: `node scripts/install.js`
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script.
npm ERR! This is most likely a problem with the node-sass package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node scripts/install.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls node-sass
npm ERR! There is likely additional logging output above.
npm ERR! System Windows_NT 6.1.7601
npm ERR! command "C:\\Program Files\\nodejs\\\\node.exe" "C:\\Program Files\\nod
ejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "node-sass"
npm ERR! cwd E:\test
npm ERR! node -v v0.10.35
npm ERR! npm -v 1.4.28
npm ERR! syscall spawn
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! not ok code 0

I am having the same problem.  Below shows<EMAIL_ADDRESS>but the same error occurs when trying to install<EMAIL_ADDRESS>using the Zurb Foundation 5 CLI to create a Libsass project.
PS C:\temp> node -v
v0.10.36
PS C:\temp> npm -v
1.4.28
PS C:\temp> node -p process.arch
x64
PS C:\temp> cd /temp
PS C:\temp> npm install node-sass; node
|
><EMAIL_ADDRESS>install C:\temp\node_modules\node-sass
> node scripts/install.js

npm ERR<EMAIL_ADDRESS>install: `node scripts/install.js`
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script.
npm ERR! This is most likely a problem with the node-sass package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node scripts/install.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls node-sass
npm ERR! There is likely additional logging output above.
npm ERR! System Windows_NT 6.1.7601
npm ERR! command "C:\\Program Files\\nodejs\\\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js"
 "install" "node-sass"
npm ERR! cwd C:\temp
npm ERR! node -v v0.10.36
npm ERR! npm -v 1.4.28
npm ERR! syscall spawn
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! not ok code 0

Any idea?

I wasn't able to install those versions (2.0.0beta and 1.2.3). I used "node-sass": "~0.7.0" in my package.json based on another project that was working.

I am not sure about it. I built the Windows x64 binaries on Win 8.1.
I was under the impression than Windows binaries built with VS2013 do not require a separate configuration (like XP target) for seven?
Meanwhile you can test it by manually downloading the binary in vendor/<os-specific-folder>/binding.node from https://github.com/sass/node-sass-binaries/tree/v2.0.0-beta based on node -p process.arch. If it says ia32 then create a directory called win32-ia32 inside node-sass/vendor directory otherwise win32-x64 then test with node -p console.log(require('node-sass').info()).
If the above solution works, that would mean this issue is fixed, because we have already fixed the download binary machinery which was broken in beta. If not then it is a broken binary issue which needs digging for seven-target (which I really hope is not the case here)..

@am11, where is the node-sass/vendor directory?

@pamrichmond, when we run npm install node-sass, our install script creates a directory called vendor at node-sass root. This directory doesn't exist in the repo (which I think it should with a .gitkeep).

@am11 , I am new to node, npm, and node-sass, so I'm not sure where the folders should be placed. I tried creating these folders in several places unsuccessfully, but I'm guessing here. I'm willing to try again if you can be specific about the locations of the folders/files. Here is what I tried:
c:\temp\myprojectname\node_modules\node-sass\vendor\win32-x64\bindings.node
c:\temp\myprojectname\node_modules\node-sass\bin\win32-x64\bindings.node
   (because an older working version used bin instead of vendor)
c:\Program Files\nodejs\node_modules\npm\node_modules\node-sass\vendor\win32-x64\bindings.node
c:\Program Files\nodejs\node_modules\npm\node_modules\node-sass\bin\win32-x64\bindings.node

Here is the output:
c:\temp\myprojectname>node -p console.log(require('node-sass').info())

module.js:340
    throw err;
          ^
Error: Cannot find module 'node-sass'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at [eval]:1:13
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (module.js:456:26)
    at evalScript (node.js:559:25)
    at startup (node.js:80:7)
    at node.js:929:3

@pamrichmond, the first one is correct.
First, please execute these commands line by line: https://gist.github.com/am11/e5de3c49c219f0811e1d.
If it fails, please create a similar GH gist with the produced output of that script and share the link.

@am11, I ran the commands you gave. The results are at: https://gist.github.com/pamrichmond/5c7680d40ded1d1d2e0e
Note that I was installing in c:\temp\temp1, but the install error says ><EMAIL_ADDRESS>install c:\temp\node_modules\node-sass, which seems like it might be trying to install into the wrong folder.  Regardless, the install failed.  The install didn't create the node_modules folder, so I manually created it and the appropriate subfolders, as well as the binary.  The node command still generated an error.

@pamrichmond, this is strange that it didn't created the node_modules directory, but great that you created it manually. :)
c:\temp\temp1\node_modules\node-sass\vendor>node -p console.log(require('node-sass').info())

Can you try:
c:\temp\temp1\node_modules\node-sass\vendor>cd ..\..\..\

c:\temp\temp1> node -p console.log(require('node-sass').info())

Also:
c:\temp\temp1> node

# this is node.js shell
require('node-sass')

@am11 I tried the additional commands:
c:\temp\temp1>node -p console.log(require('node-sass').info())

module.js:340
    throw err;
          ^
Error: Cannot find module 'node-sass'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at [eval]:1:13
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (module.js:456:26)
    at evalScript (node.js:559:25)
    at startup (node.js:80:7)
    at node.js:929:3

c:\temp\temp1>node
> require('node-sass')
Error: Cannot find module 'node-sass'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at repl:1:2
    at REPLServer.self.eval (repl.js:110:21)
    at Interface.<anonymous> (repl.js:239:12)
    at Interface.emit (events.js:95:17)
    at Interface._onLine (readline.js:203:10)
    at Interface._line (readline.js:532:8)
>

Actually require() automagically resolves the relevance from current directory to <current-directory>\node_modules\<module-name> and then it looks for global installation of the module (npm -g ..).

@pamrichmond, does this help: http://stackoverflow.com/a/25877185/863980?

@am11 That fixed it for me, thanks!
I had to add c:\Windows\System32 to System variables.

@am11, that fixed it for me. I added 'C:\Windows\System32' to my PATH, and npm install node-sass now works. Thank you.

Great! Glad to hear that. :)
@davetgreen this may also fix the issue you are having as well.


What does the configured LoaderOptionsPlugin do in the webpack2 configuration?

What does this actually do, and why do I need it in my configuration?
new webpack.LoaderOptionsPlugin({
      options: {
        transforms: [
          function(file) {
            return through(function(buf) {
              this.queue(buf.split("").map(function(s) {
                return String.fromCharCode(127-s.charCodeAt(0));
              }).join(""));
            }, function() { this.queue(null); });
          }
        ]
      }
    })

@nicholascloud The LoaderOptionsPlugin was mainly necessary in the webpack 1 => webpack 2 transition phase, bc since webpack 2 it's disallowed to have custom props on the webpack.config.js {Object} directly, in this case transforms. I can't give you a guarantee for this 😛, but normally this should live in module.rules[i].options now.
webpack.config.js
{
   loader: 'transform-loader',
   options: {
      transforms: [
         function (file) {
            return through((buffer) => {
               return this.queue(
                 buffer
                    .split('')
                    .map((chunk) => String.fromCharCode(127-chunk.charCodeAt(0))))
                    .join('')
            }),
            () => this.queue(null)
         }
      ]
   }
}

@michael-ciniawsky Awesome, thanks for the clarification.

@nicholascloud if you are using this loader, could you please verify 😁 😛 ?

@michael-ciniawsky Here is my configuration, and everything works great:
// Transform Node.js fs lib calls to be inline strings.
// @see: https://github.com/webpack-contrib/transform-loader
{
    test: /\.js$/,
    exclude: /node_modules|vendor/,
    loader: 'transform-loader/cacheable',
    enforce: 'post',
    options: {
        brfs: true,
        // @see: https://github.com/webpack-contrib/transform-loader#webpack-2x-config-example
        // @see: https://github.com/webpack-contrib/transform-loader/issues/20
        transforms: [
            function (/*file*/) {
                return through((buffer) => {
                    return this.queue(
                        buffer.split('')
                            .map((chunk) => String.fromCharCode(127-chunk.charCodeAt(0))))
                            .join('')
                }, () => this.queue(null))
            }
        ]
    }
}

@nicholascloud Thx 👍 Mind sending a quick PR with docs update :D ? Otherwise I do it later 🙃

@michael-ciniawsky For the README file? Yeah, I'll set up a PR.

Yep, 💯  thx in advance 👍

Related PR: https://github.com/webpack-contrib/transform-loader/pull/21


"JS function to Java Interface"  Proxy leaking V8 native resources when GC-ed

Currently it's possible to provide Java functional interface for the cases where JS function is passed from JS to Java.
It's implemented using Proxy and V8FunctionInvocationHandler.
Currently .release() of native j2v8 objects is implemented:

in .finalize() method, which called during the GC.
If Proxy was not GC yet - during the call to V8JavaObjectUtils.releaseV8Resources(): Underlying V8Function function is stored in the v8Resources and thus released.

Issue: while 2nd option working well and tested in the unit tests, the 1st option is leaking j2v8 native objections (v8 function and v8 receiver).
It's hard to test 1st case as it involves GC, which runs at not determined period of time. Also following comment in the source code highlight possible problem as well:

// TODO: How do we release if this gets executed on a different thread?

I have tested it and indeed GC is run on the separate thread. E.g. on Nexus 5/Android 6 it's running on FinalizerDaemon. But all J2V8 objects requires .release() to be called on the same "v8 thread".
As result the Proxy object is released, but native j2v8 object are leaking.
Possible solution:

By default implement the proxy as call-back function, but not listeners. E.g. allow functions to be called once. This way V8Function could be released right after it's invoked.
If listener behaviour is required - introduce annotation like  @JsListener. I could be present either on java interface declaration or when 3rd party interface is used as method argument.
Listener implementation could be implemented in 2 ways:
3.1. Keep hard reference to all the call-back functions and thus they will be released in the V8JavaObjectUtils.releaseV8Resources() as mentioned above.
3.2. Provide "V8 executor" during V8-adapter initialisation. When .finalize() is invoked - this executor can run .release() of underlying V8Function on the proper v8 thread.

Implementing listeners as 3.1 will result in retaining all  V8Functionlisteners in memory until V8 is released.
Implementing listeners as 3.2 would require passing V8 executor, which leads to additional requirements of end users of v8-adapter lib.
Also different behaviour could be implement if V8 executor is present or not - but this sounds to much complicated for now.
Also as an option V8FunctionInvocationHandler and resulting proxy could implement Releasable  interface and released manually if they need to be called known amount of times. E.g. 2 or 3 , etc.
@crahda as for me, adding v8 executor does not make things to much complicated for end users as it would be need only for listeners (but not for call-backs). Instead it would allow proper memory clean-up.
Additionally if this behaviour is implemented - some default functional interface like JsFunction could be created. And if function is present inside of the JS object - it could be automatically converted to Map without worries of leaking memory.

@crahda I have created 2 pull requests:

https://github.com/alicorn-systems/v8-adapter/pull/25 is way to manually release JS proxy-based call-backs resources, which is currently not released at GC and leaking.
https://github.com/alicorn-systems/v8-adapter/issues/26 is proper fix to avoid memory leaks by making JS function to be call-backs by default + proper resources clean-up if it was not called. Also if JS v8 executor is provided - releases unused v8 resources at the time of Java GC.

Seconds PR contains changes from 1st one already. Also changes in 1st could be used to manually release listeners if needed.

@crahda I added 3rd PR https://github.com/alicorn-systems/v8-adapter/pull/27 - for reading V8Function as default Java interface (JsBasedCallBack) if exact Java interface is not provided.
This PR is related to #11 but can be implemented only now after PR https://github.com/alicorn-systems/v8-adapter/pull/26 for issues #24. It's also performance efficient as it's does not used Proxy.

@crahda just in case: if you like all 3 PR - last one https://github.com/alicorn-systems/v8-adapter/pull/27 contains changes from https://github.com/alicorn-systems/v8-adapter/pull/26 and https://github.com/alicorn-systems/v8-adapter/pull/25.

@AlexTrotsenko apologies for my radio silence --- I am traveling overseas and have been swamped in other projects. I have gone ahead and approved all of your PRs, and they are now deployed to Maven Central as version 1.59-SNAPSHOT! I will deploy them as a full release (1.59) in the morning once my new PGP keys finish synchronizing...
Thank you so much for your continued support of this project.

Hi @crahda, thanks for taking care of PRs and your kind words. It's clear, that each of us has many things to deal with it - don't worry about it.
Also would like to mention, that I have found today a small bug with "injection's override" api in specific condition and already created PR - please check it when you will have a time: https://github.com/alicorn-systems/v8-adapter/pull/29

@AlexTrotsenko Thanks so much -- I've pulled in your PR and it is now in Maven as 1.591-SNAPSHOT. Moving forward, I'll try to only push minor builds when we add new features...


  "type": "pwa-msedge",

Environment (please complete the following information):

OS (e.g., Windows 10):
Extension version (e.g., 1.2.3):

Describe the bug:

Repro steps:

Expected behavior:

Additional context:

  "type": "pwa-msedge",

  "type": "pwa-msedge",

Closing as incomplete. Please feel free to add more information if you have any and we can re-open the issue.


Invoke-PowerBIRestMethod fails with Service Principal

Invoke-PowerBIRestMethod does not function when being called from a Service Principal. I have configured the service principal in the Azure tenant with the required API permissions, but the service account can't call the Invoke-PowerBIRestMethod. The value returned says it's not authorized even though it has permissions. I also tried this with Get-PowerBIReport -Scope Organization and Invoke-RestMethod with no success.
Example script:
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$tenantId = "#####"
$AppId = "#####"
$PasswordClearText = "######"
$PW = ConvertTo-SecureString -String $PasswordClearText -AsPlainText -Force
$Credential = New-Object -TypeName "System.Management.Automation.PSCredential" -ArgumentList $AppId, $PW
Import-Module -name MicrosoftPowerBIMgmt
$Account = Connect-PowerBIServiceAccount -Credential $Credential -Tenant $tenantId -ServicePrincipal
if(!$Account) {
Throw "Could not connect to Power BI service account."
}
$Reportsresponse = Invoke-PowerBIRestMethod -Url 'reports' -Method Get
if(!$Reportsresponse){
Resolve-PowerBIError
}
$Reportsresponse.value | Out-Host
Disconnect-PowerBIServiceAccount
Message        : Response status code does not indicate success: 401 (Unauthorized).
StackTrace     :    at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at Microsoft.PowerBI.Commands.Profile.InvokePowerBIRestMethod.d__31.MoveNext()
Exception      : System.Net.Http.HttpRequestException
InvocationInfo : {Invoke-PowerBIRestMethod}
Line           : $Reportsresponse = Invoke-PowerBIRestMethod -Url 'reports' -Method Get
Position       :  - Doc\PowerShell\Azure Authentication Test - Az.ps1:46 char:20
+ ... Reportsresponse = Invoke-PowerBIRestMethod -Url 'reports' -Method Get
+                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 1

Is there any update here.. I`ve used "Invoke-PowerBIRestMethod" and "New-PowerBIReport" to publish a dashboard using Sp but i keep getting Unauthorized.

I am experiencing the same behavior, trying to pull tenant data out from the administration rest api's. What is the preferred authorization?

I was  able to do this by using Get-PowerBIAccessToken

$secpasswd = ConvertTo-SecureString "xxxxxxxxxxxxxxxxxx" -AsPlainText -Force
$mycreds = New-Object System.Management.Automation.PSCredential ("<service principal_Id/app_id>", $secpasswd)
Connect-PowerBIServiceAccount -ServicePrincipal -Credential $mycreds -Tenant $tenantId
$auth_token= Get-PowerBIAccessToken -AsString

Same here trying to deploy a PBI Pipeline
Connect-PowerBIServiceAccount -Tenant $tenantid -ServicePrincipal -Credential $Cred

$body = @{
    sourceStageOrder = 0 # The order of the source stage. Development (0), Test (1).
    datasets = @(
        @{sourceId = "yyyyyyy" }
    )
    reports = @(
        @{sourceId = "xxxxxxxx" }
    )
    options = @{
        # Allows creating new item if needed on the Test stage workspace
        allowCreateArtifact = $TRUE
        # Allows overwriting existing item if needed on the Test stage workspace
        allowOverwriteArtifact = $TRUE
    }
} | ConvertTo-Json

$url = "pipelines/{0}/Deploy" -f $pipelineId
Invoke-PowerBIRestMethod -Url $url -Method Post -Body $body

[x] Service principal with workspace permissions & Pipeline Access

[x] Service principal with API Permissions:

[x] Allowed SPN to use PBI APIs

But I still getting the following response:
99
{"error":{"code":"PowerBINotAuthorizedException","pbi.error":{"code":"PowerBINotAuthorizedException","parameters":{},"details":[],"exceptionCulprit":1}}}
0

Same here trying to deploy a PBI Pipeline
Connect-PowerBIServiceAccount -Tenant $tenantid -ServicePrincipal -Credential $Cred

$body = @{
    sourceStageOrder = 0 # The order of the source stage. Development (0), Test (1).
    datasets = @(
        @{sourceId = "yyyyyyy" }
    )
    reports = @(
        @{sourceId = "xxxxxxxx" }
    )
    options = @{
        # Allows creating new item if needed on the Test stage workspace
        allowCreateArtifact = $TRUE
        # Allows overwriting existing item if needed on the Test stage workspace
        allowOverwriteArtifact = $TRUE
    }
} | ConvertTo-Json

$url = "pipelines/{0}/Deploy" -f $pipelineId
Invoke-PowerBIRestMethod -Url $url -Method Post -Body $body

[x]  Allowed SPN to use PBI APIs

[x]  Service principal with workspace permissions & Pipeline Access
[x]  Service principal with API Permissions:

But I still getting the following response:
99
{"error":{"code":"PowerBINotAuthorizedException","pbi.error":{"code":"PowerBINotAuthorizedException","parameters":{},"details":[],"exceptionCulprit":1}}}
0

I solved it with that changes:

I added to the API Permissions of service principal (second pict) this: Gateway.Read.All
And I allow another check into the PowerBI Admin portal (first pict), Dataset Execute Queries REST API

The changes need some time to work it fine...


InfluxDB 2 - Telegraf Config URL should be configurable

Proposal:
The InfluxDB2 GUI generates URL's for Telegraf configs. The base URL should be configurable.
Current behavior:
InfluxDB2 generates a URL based on some magic.
Desired behavior:
Generate based on magic OR generate URL based on passed configuration/environmental variable.
Alternatives considered:
None
Use case:
The current assumption in design is that InfluxDB2 will be "access(ed) over local ip". We should be able to provide our own information as the InfluxDB2 instance may be behind a proxy. It may be behind a loadbalancer. The user may prefer using DNS resolution. The automagic detection may fail.

I just confirmed that the http-bind-address startup option should achieve what you're looking for.
On my mac:

bin/darwin/influxd --http-bind-address "MBP13-NISELY.local:9999"

https://v2.docs.influxdata.com/v2.0/reference/config-options/#http-bind-address


after redirect  getFlash()  is null? why?

What steps will reproduce the problem?
I am is chinese people at shanghai
my yii2 versions is yii2.08-dev
i use Yii::$app->getSession()->setFlash('success', "save is success"); in siteController/addAction
than  use $this->redirect("site/index");
i use  Yii::$app->getSession()->getFlash('success');
why??
What is the expected result?
i want use yii\bootstrap\Alert  in layout/main.php
What do you get instead?
it is "NULL";
i use Yii2.0.7   it  is good;
Additional info

Q
A

Yii version
2.0.8-dev

PHP version
5.6.19

Operating system
win10 professional edition

Duplicates #9187


add catcma

Contributor Agreements
Please read the contributor agreements and if you agree, please click the checkbox below.

[x] I agree to the contributor agreements.

Motivation

Add CatCMA
Description of the changes

This PR adds CatCMA.
See https://arxiv.org/abs/2405.09962 for the details.

@y0z
FYI:
While I haven't checked it carefully, this paper may be of interest to you.
Large-scale Benchmarking of Metaphor-based Optimization Heuristics, GECCO'24.
https://arxiv.org/abs/2402.09800


Made Random Profile Generator App

I have developed a Random Profile Generator app utilizing React and the randomuser API. I kindly request you to consider accepting this pull request under the Hacktoberfest 2023 label. Your approval would be greatly appreciated.

After Clicking Change User Button

As a newbie in open source and participating in Hacktoberfest for the first time, I'm having trouble understanding the specific changes you expect in the file. Could you please provide more guidance or clarification on the required modifications? Your assistance would be greatly appreciated.

@Haris-Mohanty, I have made new pull request after updating branch name as per your need. Please merge it under hactoberfest2023 label.


Type or tags by section instead of by each card

Great addon. Is there a way to quickly set a section of cards as a specific tag or as a type=Cloze without having to specify it to each card?

Unfortunately not at this time. You could set the type for card at the top of the file but this would then set it for all of the cards in the document

That's exactly what I would like. I've tried to set a global variable on the google docs like here at the top with # type = Cloze with this example document https://docs.google.com/document/u/1/d/18GGj3I_7rdQxOiW5X9MZd1ooCIyMnLsuR_2z3B_ib2I/pub and it doesn't seem to recognize the cards as Cloze format but rather as the default basic format?
Furthermore, would you be able to demonstrate how #fileType = topics works using the Google Docs? I'd like to be able to split my Google Doc into multiple decks. Thank you very much for your time.


Update tests to delete the ams account created at the beginning of ea…

…ch test.
Update recordings.

This checklist is used to make sure that common guidelines for a pull request are followed.

[x] The PR has modified HISTORY.rst describing any customer-facing, functional changes. Note that this does not include changes only to help content. (see Modifying change log).

[x] I adhere to the Command Guidelines.

Fixing precheck issues.


No. is blocked, call monitor stays open

If a number is blocked in the fritzbox and the number is calling, the call monitor stays open.
This happens only for blocked numbers.
Maybe because its kind of reverse in handling? It first disconnects the caller and then it shows incoming call.
09:53:14HOME Bridge FritzPlatform Callmonitor (group): Call disconnected with blocked
09:53:14HOME Bridge FritzPlatform Callmonitor (group): Incoming call from: blocked to private

I have the same issue. Problem is now that i have an open sensor in Apple Home which is still open. Therefore my Automation does not work, as the sensor shows still open. This sensor set my TV on mute when a call is incoming.
How can i close the sensor manually?

The only way is to make your self a phone call (just for 1 sec) to reset the monitor. :(

Ok, no smart but a workaround, thanks!

Thats not a workaround. I got the same automation like you, even to pause Apple TV if a movie is been watched. But I deactivated all those automations that are using the call monitor...

You are right, workaround means the automation can be further used which is not possible in this case. I hope @seydx can offer a solution for that issue?

Hey @seydx, kann mir vorstellen, dass es in dieser Zeit nicht so passt, jedoch wollte ich mal dezent anfragen, ob und wann (ca.) wir mit einem Fix rechnen können?
LG


GH-151: HostInfo is now extensible

Fixes #151 : HostInfo now has public constructors and is not marked as final

Not sure if there anything more to it :-)


Unable to open a PR

Hi. I would like to add cookie encryption support to express. I plan to add the encryption bit here and the decryption bit in the cookie-parser library. I have even created the symmetric-cipher.js npm package for cookie encryption and decryption, similar to the cookie-signature` library.
This is my first time contributing to open-source, so please go easy. When I attempt to open a PR, I run into this:

Kindly assist.

I am now able to open a PR. Must be a one-time thing.


ENOENT: no such file or directory.

ENOENT: no such file or directory, scandir '*\node_modules\htmlparser2\lib\node_modules\entities\maps'. I use with Expo. Please help me.

We are in the process of cleaning the issue tracker to make it manageable. Because your issue did not follow our contribution guidelines, it will be closed. But you can ask this question on stackoverflow. Don't forget to add the react-native-render-html  tag. We are notified on this tag, and will be happy to help you here.


Un-set unified title and tool bar on mac (Qt property)

Description
This one is a bit weird, but here goes. This fixes the background "behind" the QtViewerDockWidgets being the wrong color (something from the system theme). I think this was actually the root cause of the issues seen in #5529 and addressed separately in #5526. I could go either way then on whether to include the changes in #5526 as well.
Anyway this removes a property that was set to fix an intermittent problem quite a while ago. It seems the behavior of this property actually changed in Qt5, and it is supposedly not compatible with windows that contain Open GL content.
Before this change

After this change

Type of change

[x] Bug-fix (non-breaking change which fixes an issue)

References
https://github.com/napari/napari/pull/245
https://github.com/napari/napari/issues/239
https://github.com/napari/napari/issues/5529
#5526
How has this been tested?

[x] I checked my changes work with both PySide and PyQt backends on Qt6, and PyQt on Qt5

Final checklist:

[x] My PR is the minimum possible work for the desired functionality

For me this works perfectly and it's quite simple.
I wonder if this will mess something up on some sort of minimum requirements install?
But from reading the old threads, it's clear that the original issue wasn't easy to trigger, because none of the other mac users at the time had the issue.
One thing that makes this better is that the background when undocking is set by the napari theme and not OS theme. Now, I prefer to have napari and OS synced, but if you don't and you try to undock a widget it will look horrible. The other PR does not fix that on my end with pyside6.

One thing that makes this better is that the background when undocking is set by the napari theme and not OS theme.

Yeah this is what I set out to fix with this one.
I can go either way about including the changes from the other PR in addition. They still make sense to me, but I think with this change they're not strictly necessary. I'll happily defer to the resident Qt experts!

I wonder if this will mess something up on some sort of minimum requirements install?

We do have minreq in our tox matrix, so we should be good on that front :)

Maybe for an older Qt5 version like 5.6 or 5.9 or older MacOS versions was necessary to set that property for some reason? I think this solution makes sense :+1: (it's always nice to find a one liner solution for multiple problems and even better if it is done by removing code :) )

Man these tests failing pyqt5/python 3.8 are a bummer :(
At least the dims ones would pass if rerun!
When do we drop python 3.8? 👀

Man these tests failing pyqt5/python 3.8 are a bummer :(

this is not a problem with python 3.8 I observed it also on other python version.

Based on the above approvals and the comment from @dalthviz I think this is ready to merge.
Starting the clock—I use pyside6 in my dev env, so this is a QoL issue for me!


Add proc reaper for PID 1 containerized processes

On Linux pid 1 processes (container processes) must reap their children.
This library copied directly from openshift/origin.

No dependencies and trivial.  Prerequisite for moving router out of origin.


Audio interruption and audio route change handling

The player is gracefully paused on interruption and pauses audio playback on noisy route change, according to good behaviour expectations. Delegate callbacks let the client know when this occurs
Apple resources
https://developer.apple.com/documentation/avfaudio/avaudiosession/responding_to_audio_session_interruptions https://developer.apple.com/documentation/avfaudio/avaudiosession/responding_to_audio_session_route_changes

While I like this, I don't think this is needed. AVPlayer does this automatically but AVPlayer is built in a way to be reasonably easy to play something.
I believe clients that use AudioStreaming should handle AVAudioSession as per their needs.
Client should consider to abstract away the real implementation of the audio playing mechanism so they can also change it if they choose to.
My suggestions is to provide documentation or FAQ on how to respond to AVAudioSession changes and notifications instead of baking into this library.

I completely agree with @dimitris-c.
It's important for users who use this library  to handle AVAudioSession in accordance with their needs. Abstraction of the audio playing mechanism allows users to have the flexibility to change it if they choose to.

ok


Fix local cache metrics in log

What changes are proposed in this pull request?
Fix local cache metrics in log.
Why are the changes needed?
When there are multiple cache directories, the metrics of LocalCacheManager log are incorrect. For example
2023-06-22 16:04:38,437 INFO  LocalCacheManager - Restoring PageStoreDir (/mnt/disk/1/alluxio.local.cache/LOCAL)
2023-06-22 16:04:38,443 INFO  LocalCacheManager - PageStore (/mnt/disk/1/alluxio.local.cache/LOCAL) restored with 7 pages (6895353 bytes), discarded 0 pages (com.codahale.metrics.Counter@347d2634 bytes)
2023-06-22 16:04:38,443 INFO  LocalCacheManager - Restoring PageStoreDir (/mnt/disk/2/alluxio.local.cache/LOCAL)
2023-06-22 16:04:38,443 INFO  LocalCacheManager - PageStore (/mnt/disk/2/alluxio.local.cache/LOCAL) restored with 7 pages (6895353 bytes), discarded 0 pages (com.codahale.metrics.Counter@347d2634 bytes)

The second directory is empty, but it has the same metric with the first one.
With this PR, the log became the following version
2023-06-22 16:37:20,780 INFO  LocalCacheManager - Restoring PageStoreDir (/mnt/disk/1/alluxio.local.cache/LOCAL)
2023-06-22 16:37:20,787 INFO  LocalCacheManager - PageStore (/mnt/disk/1/alluxio.local.cache/LOCAL) restored with 7 pages (6895353 bytes), discarded 0 pages (0 bytes)
2023-06-22 16:37:20,787 INFO  LocalCacheManager - Restoring PageStoreDir (/mnt/disk/2/alluxio.local.cache/LOCAL)
2023-06-22 16:37:20,788 INFO  LocalCacheManager - PageStore (/mnt/disk/2/alluxio.local.cache/LOCAL) restored with 0 pages (0 bytes), discarded 0 pages (0 bytes)

Does this PR introduce any user facing changes?
NO

alluxio-bot, merge this please.

alluxio-bot, merge this please.


Update 'automated testing' hyperlink

Thanks, @paulcwarren! Could you please heed Dreddbot's comment and make your membership in a suitable GitHub org public? Then we can merge this pronto.
Best,
Eric

Tis done!


Wrong result when using new numpy ffi in deferred compute

Find it in CI result of https://github.com/apache/incubator-mxnet/pull/17958.
Reproduce:
Sample code, use master branch
import mxnet as mx
import mxnet._deferred_compute as dc
from mxnet import np, npx
npx.set_np()
with dc.context():
    a = np.ones((2, 2))
    b = np.tril(a, 1)
    c = np.tril(a, -1)

sym = dc.get_symbol([b, c], sym_cls=mx.sym.np._Symbol)
res = sym.bind(mx.context.current_context(), args={}).forward()
res

Results:
[
[[1. 1.]
 [1. 1.]]
<NDArray 2x2 @cpu(0)>,
[[1. 1.]
 [1. 1.]]
<NDArray 2x2 @cpu(0)>]

If replace _api_internal.tril by _npi.tril, the result is right.
@leezu Can you take a look at this?

@hzfan

Thanks for reporting the issue. I can confirm that disabling the new FFI for tril via below patch avoids the issue. I'll take a look why the two FFIs behave differently
diff --git a/python/mxnet/ndarray/numpy/_op.py b/python/mxnet/ndarray/numpy/_op.py
index 4820f56f3..8ff7ba073 100644
--- a/python/mxnet/ndarray/numpy/_op.py
+++ b/python/mxnet/ndarray/numpy/_op.py
@@ -2051,7 +2051,7 @@ def tril(m, k=0):
            [ 7.,  8.,  0.],
            [10., 11., 12.]])
     """
-    return _api_internal.tril(m, k)
+    return _npi.tril(m, k)

 def _unary_func_helper(x, fn_array, fn_scalar, out=None, **kwargs):

@leezu Have you been able to find the root cause for this bug?

@haojin2 sorry, I didn't get to dive deep on this issue yet. I'll work on this during our bug bash in the coming week.

It's due to common expression elimination. As workaround one could set MXNET_ELIMINATE_COMMON_EXPR=0.
https://github.com/apache/incubator-mxnet/blob/ab4f7f6a7335e88034edcf61402aec170cdca5fd/src/executor/eliminate_common_expr_pass.cc#L59-L94
For the old FFI, n->attrs.dict != m->attrs.dict and the two nodes are preserved.
But in the new FFI, n->attrs.dict == m->attrs.dict and the np.tril(a, -1) is replaced by np.tril(a, 1).
For the new FFI, the reason is
https://github.com/apache/incubator-mxnet/blob/7bef85ecbf9cb064d45479efa5cc46c6f8a4e947/src/api/operator/utils.h#L52-L57
only parses attributes if autograd is enabled, whereas the old FFI always parses attributes. This was an oversight when I rebased the DC PR on top of the FFI update PR.


Container: how to add event subscribers?

How to add event subscribers to Containers that are spawned dynamically?
Should newly spawned Containers push to the same handlers as the Container that spawned them?
pushing to handler for mouse_entered and mouse_exited here:
https://github.com/ioxu/boxer/blob/9d16528a92268d7a1a3d71ae1c3dd886549bbb14/boxer/containers.py#L363

arguments to EventDispatcher.push_handlers is a dictionary of key event_name as a string, and value of the callback handler function.
e.g. :
{'mouse_entered': <function mouse_entered_container at 0x00000252B3750700>}```

ah, brainbdead moment:
all I need to do is push mouse_entered and mouse_exited handlers on the first initialised Container
    c.push_handlers( mouse_entered = mouse_entered_container )
    c.push_handlers( mouse_exited = mouse_exited_container )

and then call the root_container to dispatch_event()
... just like the pattern that the rest of the events pushed to the Container do 🙄


How to use Tensorboard or Tensorflow Plot

As is mentioned,during the training phase, the samples including images, questions, ground truth answers, and predicted answers can be monitored on Tensorboard if the library Tensorflow Plot is installed,how could i operate to monitor this process???
I see tfplot and seaborn two modules in link"Tensorflow Plot",do i need to install seaborn either???
And also if i want to visualize the CNN internal network,what should i do???
Thanks for your guide!!!

Please install tfplot with this command
pip install git+https://github.com/wookayin/tensorflow-plot.git@master.
The detailed document can be found here tfplot. After installing it, you can view the training details on Tensorboard.


webpackPrefetch failed with webpack version >= 5.25.0

Bug report

What is the current behavior?
Server should get start without any error with webpackPrefetch in dynamic import. But it is failed to start if the webpack version is >= 5.25.0 and it is working fine if webpack version <= 5.24.0.
Got below error while starting server
/Users/Documents/dev/product/build/server.js:21367
/******/ webpack_require.O(0, ["server"], () => (), 5);
^

SyntaxError: Unexpected token ')'
at wrapSafe (internal/modules/cjs/loader.js:1053:16)
at Module._compile (internal/modules/cjs/loader.js:1101:27)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1157:10)
at Module.load (internal/modules/cjs/loader.js:985:32)
at Function.Module._load (internal/modules/cjs/loader.js:878:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
at internal/main/run_main_module.js:17:47

If the current behavior is a bug, please provide the steps to reproduce.
Issue is because of adding webpackPrefetch in below import
import loadable from '@loadable/component';

const Component = loadable(
    () =>
        import(
            /* webpackPrefetch: true */ /* webpackChunkName: "compChunk" */ '../component'
        ),
    { ssr: false }
);

What is the expected behavior?
Server should get start without any error with webpackPrefetch in dynamic import

Other relevant information:
webpack version:  5.38.0
Node.js version:  12.18.4
Operating System: Mac 10.14.6

Not reproducible with webpack version 5.38.1.
Please give us more info.
Also show us webpack config

Duplicate https://github.com/webpack/webpack/issues/12993, feel free to feedback or provide example how we can reproducible if it is no duplicate


Update reflect.ts to initialize errors array

fix: crash calling type.matchesValue when errors array is not initialized

Looks great, thanks!

Released in 0.7.4 (and deployed to typescript-rtti.org). Thanks again!


Creation of invalid python function sometimes completes with status READY

Bug Report
I have test that is trying to create Python function with invalid python code. Invalid python code is just a random characters i.e. not a python code at all. Not sure about the other runtimes (nodejs and etc).
Expected behavior
Status of the function creation should always be reported as  'ERROR'.
Current behavior
Most of the times function creation finishes with a proper status code of 'ERROR'. However it appears that sometimes after reporting the function status as INITIALIZED it moves to 'READY' state.
Steps to reproduce

Try to create python function with some invalid python code (some characters that can not be considered python code at all.)
Retry to create the invalid function several times.
Most of the times the status is reported properly as ERROR by sometimes the function status is reported as READY.

Impact

High - Blocker - READY state should 100% mean function is ready to run.

Fixed by dispatchframework/python3-base-image#11


topic.partitionMetadata.forEach is not a function

Within no-kafka/lib/client.js, Client.prototype.updateMetadata includes these lines:
_.each(response.topicMetadata, function (topic) {
    self.topicMetadata[topic.topicName] = {};
    topic.partitionMetadata.forEach(function (partition) {
        self.topicMetadata[topic.topicName][partition.partitionId] = partition;
    });
});

For me, though,topic.partitionMetadata is an object with keys '0', '1', ..., instead of an actual array, so it does not inherit the 'forEach' method. And so, [TypeError: topic.partitionMetadata.forEach is not a function] is thrown.
My kafka service is flozano/kafka:0.9.
Wrapping topic.partitionMetadata with _.toArray() seems to fix the problem. I've been using no-kafka for a while now, and just started having this problem recently, which is strange.

topic.partitionMetadata is an array. Check your code for something that mutates it.

I ran into the same issue this week:
Node.js Version: 6.11.0 (on Alpine Linux)
Kafka Version: <IP_ADDRESS>
Dockerfile: solsson/kafka-persistent:<IP_ADDRESS> (GitHub)
Topic-Configuration:
Partitions: 20
Replication: 3
Code:
const producer = new Kafka.Producer({
  connectionString: 'kafka:9092'
})

producer.init()

I wrapped the forEach method above in no-kafka/lib/client.js with Array.isArray(topic.partitionMetadata) and logged everything that didn't match:
typeof topic.partitionMetadata
// => object

topic.partitionMetadata
// => { '0':
   { error: null,
     partitionId: 17,
     leader: 1,
     replicas: { '0': 0, '1': 1, '2': 2 },
     isr: { '0': 1, '1': 2, '2': 0 } },
  '1':
   { error: null,
     partitionId: 8,
     leader: 1,
     replicas: { '0': 0, '1': 1, '2': 2 },
     isr: { '0': 1, '1': 2, '2': 0 } },
  '2':
   { error: null,
     partitionId: 11,
     leader: 1,
     replicas: { '0': 0, '1': 1, '2': 2 },
     isr: { '0': 1, '1': 2, '2': 0 } },
  …
  }

@oleksiyk Any idea what might cause this?

@fhemberger Are you able to make a standalone test that replicates the problem?

Okay, looks like I found the reason: It's not at all related to kafka at all.
We are using your library in a hapi web-service, with hapi-and-healthy for health checks. At first, it sounds really far off that this might be causing the issue, but as soon as I removed the plug-in, everything worked as expected again. I didn't went into too much detail, but there's some patching/extending lodash going on (for example using undermore), which might interfere with the lodash's _.each method.

@fhemberger thank you for adding this info here. We also use the hapi-and-healthy plugin.

@scmorse hapi-and-healthy has been updated already, so this issue shouldn't appear any longer.


Create a GitHub pages site with the bevy_ggrs example

Resolves #212
Code largely borrowed from bevy-shell-template
Supersedes: #244

Trying to see if I can get the deploy job to pass...
This is rather unhelpful:

Okay, it's deploying now, but seems I glossed over the issue with the panic:
panicked at 'Cannot find element: #bevy_ggrs_example.', /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bevy_winit-0.11.2/src/winit_windows.rs:137:21

Should also check why the artifact is 85 mb, maybe we're not building release builds?

Alternatively, we can do this with just wasm-opt/wasm-pack, and we can use the same launcher I use for all the embeds on my blog:
https://johanhelsing.studio/bevy-embed?app=extreme_4
Should allow the example to stay simple stupid (no trunk/yew or anything), and no hacks for canvas elements.

Should also check why the artifact is 85 mb, maybe we're not building release builds?

That would definitely be the case. My builds are usually <10MB in release mode when I deploy my entire game (which includes matchbox).

Closing in favor of #335


Make png to jpg converter (flatten to specified background color)

I think this would be a killer feature. Serving .png images is less than ideal compared to .jpg. But keeping the master images as large .png files is a must if you want transparency. Hence something like this would be preferred:
{{ $png := .Resources.GetMatch "image.png" }}
{{ $jpg := $png.toJPG "#FFFFFF" }}

or using filters as in #6255:
{{ $jpg := $png.Filter (images.Flatten "#FFFFFF") }}

I hope you get my point. We have an eCom site and our product images are large .png files with a transparent background. Right now we flatten these in a separate step into .jpg files with two different backgrounds depending on what the background on the underlying element is. But I think this kind of processing could (and should!) be included by default in hugo.

hey ! the code base is (very ) huge , can you point the specific  file,  where this feature  could be added directly . like template parser or similar file .

@roz3x we need to agree on the API first. I don't mind adding this "somehow", but we need to consider where/how.

is this something you wanna fix right now , or you want to add in future updates !?

I don't mind having it fixed now, but we need to know how before doing it. $img.ToJPG does not look "right" to me, but this is not something I plan to spend time on right now.

Speaking for my needs, something like this would feel "okay" to me:
{{ $png := ... }}
{{ $jpg := $png.Convert("jpeg bg=#ffffff") }}
// or: {{  $jpg := $png.Convert("jpeg #ffffff") }}
// or: {{  $jpg := $png.Convert("jpeg", (dict "bg" "#ffffff")) }}
{{ $webp := $png.Convert("webp") }}

My thinking is that the required parameters depend on the target image format. Required parameters have to be passed to Convert. If missing, Convert will raise an error.
For example, creating a JPG from an image with transparent background needs the background color. A solution with a flatten filter might be harder to get because converting a png to jpeg without flattening before will have to assume a default background color.
A generic Convert could be used for all sorts of image formats, i.e. we wouldn't need functions for each format, e.g. $img.ToPng, $img.ToJpeg, $img.ToWebP, $img.ToJpeg2000, etc.

This would be useful for our  use-case too. Is there any reason not to have the specification of image quality (e.g., "q75") be sufficient for automatic conversion of a png to jpg? Right now the quality indicator is ignored for png, but it seems that once this API exists it can now be respected via conversion to jpg.

@gnalck Quality is most probably not enough. There's a default quality setting in the config and it's (probably) not intuitive to output a jpeg if it's defined on the call and a png if the default quality is used. And there might be other formats in the future which support a quality setting.
I spend some time and created a wip PR, changes are at https://github.com/jansorg/hugo/commit/0a309db00fa681f3df5da3de75811f016a1383d9 for now.
This allows to do $jpg := $png.Resize "400x jpeg q70" and $jpg2 := $jpg.Resize "200x200", for example.
It's in an early stage, but seems to work well for me locally. It's not yet supporting the flattening background color.
I couldn't find a webp encoder in pure go. https://github.com/chai2010/webp contains a set of C files and https://github.com/nickalie/go-webpbin seems to link to a C library. Not sure if one of them would be okay for Hugo as a dependeny.
There's not Jpeg2000 encoder I could find.

C libraries is not a route we want to go. Or: We do it only if we really, really need the particular library (e.g. libsass). I'm a little surprised that no company have stepped up and created native encoders for stuff like WebP and progressive jpegs.

@jansorg we have the converter in place. What we need to close this particular issue is a way to set the background color. Webp is not in the scope of this particular issue. And I would suggest you take any discussions re. webp in another issue.


Allow adding a 2D image to a 3D space view

Based on https://discord.com/channels/1062300748202921994/1164158709103083612
Sometimes you want to show a 2D image with some 3D data next to it, like at 1:34 in this video: https://youtu.be/bKUAZ7IQcf0?si=x6VGZTB8usL90I2v&t=94
To accomplish this one should be able to do rr.log("3d/image", rr.Image(…)) + rr.log("3d/paths", rr.LineStrips3D(paths)), and then add both entities to the same 3D space view. However, this is currently not allowed:

Repro:
import numpy as np
import rerun as rr

rr.init("rerun_example_test", spawn=True)
image = np.zeros((8, 12, 3), dtype=np.uint8)
rr.log("world/image", rr.Image(image))
rr.log("world/points", rr.Points3D([[0, 0, 0], [1, 1, 1], [2, 2, 2]]))

Interestingly we only recently prevented this, as of: https://github.com/rerun-io/rerun/pull/3822
Rather than treating images as optionally being 3D, I'm inclined to require the explicit addition of an orthographic projection:
rr.log("world/image", rr.Image(image))
rr.log("world/points", rr.Points3D([[0, 0, 0], [1, 1, 1], [2, 2, 2]]))

Github misinterpreted my comment as a resolution


Can you release .jar file?

Is it possible to provide *.jar file(s)?

Jars are provided via Bintray https://bintray.com/sedmelluq/com.sedmelluq/download_file?file_path=com%2Fsedmelluq%2Flavaplayer%2F1.3.9%2Flavaplayer-1.3.9.jar