Add -dynamic back in; removal broke IHaskell

This commit removed -dynamic as a default flag https://github.com/IHaskell/IHaskell/commit/2891bc8498ce12ae2677d72c7252e2623158e378
The results of this change break IHaskell in two ways. First, it now links every package available statically on startup. This is fine for small projects. In a large project though, with 200 packages, I gave up after 15 minutes. Second, many of us use IHaskell with packages that rely on shared libraries. This stopped working entirely.
There's a related long discussion here
https://github.com/IHaskell/IHaskell/pull/1252/files#diff-30f4411ae6041ebe629c055ae5c4407e47989f719a5733416bfa23454270ba65R130
The suggested solution is that users should add dynamic in this case. But there is simply no way for users to figure this out.
I had to run strace and notice among its 100Mb of output that ihaskell is stuck loading one object file after another and look at gdb to confirm that it's stuck in the linker pinning the CPU to 100%. No one is going to do this and then realize that -dynamic is needed in IHaskell.
But it's even worse for a small project. You will inexplicably get errors about shared libraries. And they'll be lazy! So most of the code works, until you trip over some code that uses the shared library and then you get an incredibly cryptic error about symbols missing. Users have absolutely no way of tracking this back to IHaskell, never mind realizing that they need to add -dynamic.
This was done to make building docker images simpler. https://github.com/IHaskell/IHaskell/pull/1252
All of this is a horrible cost to pay for some convenience when building docker images. It breaks the common case for a slightly more convenient corner case. The downside of this pull request is it breaks building the Docker images, would you mind fixing that @brandon-leapyear? You seemed to have a version that copies over the right files.
I suggest we add -dynamic back in. @junjihashimoto suggested the same in the discussion above.

Hi please reference this account in the future; brandon-leapyear is my work account.
I haven't touched this project since the PR, so I don't really have any motivation for fixing this. Just like any OSS project, anyone should make the changes they want; just cuz I rewrote the Docker stuff, doesnt mean I'm forever the maintainer of the code. Of course, feel free to completely revert the PR if you dont want to bother with getting dynamic working with the new Docker implementation (which I suggested in the original PR too; I was never forcing this rewrite in)
I never had a version that copied over the right files. To get it working with the new multi-stage build, you'd have to add a COPY for all .so files linked to by ihaskell, which would be annoying now, and whenever ihaskell gets a new transitive dependency. I really dont think docker is the best solution for a dynamic library; I might recommend hiding the -dynamic flag behind a cabal flag and disable it in the docker build.
Just for my own knowledge, I still dont understand why ihaskell needs -dynamic, can you elaborate? My intuition would be that you build the ihaskell binary, you run it and specify a haskell project at runtime, and it shells out to ghc to compile and run the project. Why does running Haskell code at runtime depend on how the driver is linked?

Hi please reference this account in the future; brandon-leapyear is my work account.

Sure! The PR was done under that account that's why I referenced it.

I haven't touched this project since the PR, so I don't really have any motivation for fixing this. Just like any OSS project, anyone should make the changes they want ...

Sounds good! I think you're taking my comments in a very negative light. Sorry. They were never meant to be negative. We just want to use IHaskell :)

I might recommend hiding the -dynamic flag behind a cabal flag and disable it in the docker build.

Then IHaskell will be broken in this unfortunate way for everyone who uses docker. But it is a low-effort compromise.

My intuition would be that you build the ihaskell binary, you run it and specify a haskell project at runtime, and it shells out to ghc to compile and run the project. Why does running Haskell code at runtime depend on how the driver is linked?

Well, not quite. It doesn't 'shell out' to GHC, which would mean that GHC (well, GHCi) runs in a different process (although this would be one way of implementing IHaskell). It starts a GHC session within the IHaskell process.
Code for the installed packages needs to come from somewhere. They aren't linked into the ihaskell binary (otherwise you would need to recompile IHaskell every time you added a package). Without -dynamic every package is loaded with the GHC static linker, with -dynamic every package is loaded with the system dynamic linker. The latter is lazy and fast. And without the latter you don't get to load shared libraries.
Perhaps you're asking a different question: why do the compile-time flags of IHaskell determine its runtime behavior, couldn't the two be decoupled? Likely, yes, they could be. But it wouldn't help anything.
At runtime we want IHaskell to use the dynamic linker, nothing would be gained from having an IHaskell compiled without -dynamic that runs a GHC session that behaves as if -dynamic was turned on. You would still need to include all of those shared libraries for anything to work. Even if IHaskell was implemented by having GHCi run in a different process it wouldn't help, then we'd want that GHCi to use the dynamic linker and those shared libraries.

Hi please reference this account in the future; brandon-leapyear is my work account.

Sure! The PR was done under that account that's why I referenced it.

I haven't touched this project since the PR, so I don't really have any motivation for fixing this. Just like any OSS project, anyone should make the changes they want ...

Sounds good! I think you're taking my comments in a very negative light. Sorry. They were never meant to be negative. We just want to use IHaskell :)

I might recommend hiding the -dynamic flag behind a cabal flag and disable it in the docker build.

Then IHaskell will be broken in this unfortunate way for everyone who uses docker. But it is a low-effort compromise.

My intuition would be that you build the ihaskell binary, you run it and specify a haskell project at runtime, and it shells out to ghc to compile and run the project. Why does running Haskell code at runtime depend on how the driver is linked?

Well, not quite. It doesn't 'shell out' to GHC, which would mean that GHC (well, GHCi) runs in a different process (although this would be one way of implementing IHaskell). It starts a GHC session within the IHaskell process.
Code for the installed packages needs to come from somewhere. They aren't linked into the ihaskell binary (otherwise you would need to recompile IHaskell every time you added a package). Without -dynamic every package is loaded with the GHC static linker, with -dynamic every package is loaded with the system dynamic linker. The latter is lazy and fast. And without the latter you don't get to load shared libraries.
Perhaps you're asking a different question: why do the compile-time flags of IHaskell determine its runtime behavior, couldn't the two be decoupled? Likely, yes, they could be. But it wouldn't help anything.
At runtime we want IHaskell to use the dynamic linker, nothing would be gained from having an IHaskell compiled without -dynamic that runs a GHC session that behaves as if -dynamic was turned on. You would still need to include all of those shared libraries for anything to work. Even if IHaskell was implemented by having GHCi run in a different process it wouldn't help, then we'd want that GHCi to use the dynamic linker and those shared libraries.

I think you're taking my comments in a very negative light. Sorry. They were never meant to be negative.

No worries, I didn't take them as negative 🙂 My intent was simply to reply to your "do you mind fixing the Docker images" question with "I do mind because I was never intending to forever maintain the Dockerfile" with as little harsh-ness as possible, which I possibly failed in.

Code for the installed packages needs to come from somewhere. They aren't linked into the ihaskell binary (otherwise you would need to recompile IHaskell every time you added a package).

When you say "package", what do you mean here? Do you mean a dependency of IHaskell or a dependency of the project you're using IHaskell to run?

Even if IHaskell was implemented by having GHCi run in a different process it wouldn't help, then we'd want that GHCi to use the dynamic linker and those shared libraries.

Ok so let's pretend IHaskell is compiled statically linked, and implemented such that it receives Haskell code from you, then runs callProcess "ghc" ["-dynamic", "-e", <code>]. It sounds like you're saying this is still an issue because the shared libraries from the compilation of IHaskell itself don't exist to be reused. But my argument is that the build of IHaskell itself should be sandboxed from the runtime user environment. e.g. I should be able to build the IHaskell binary and send it to you, and you should be able to run it, with IHaskell building and linking third-party dependencies built in the usual project sandbox (e.g. ~/.stack or ~/.cabal/store). Whether IHaskell itself has the dependencies statically linked or is loading deps dynamically, the Haskell evaluator it's driving should be completely separate from that.
That last paragraph was me speaking generally; this all seems particularly ill-suited for the Docker use-case. If your project uses postgresql-libpq, how do you install the libpq libraries in the Docker image for IHaskell to compile/run your code against? One option could be that each of your (non-trivial) projects should have its own Dockerfile that extends the IHaskell one:
FROM ihaskell
RUN apt-get install postgresql-dev

That is, the IHaskell Docker image's sole responsibility is to provide the IHaskell binary (and an OS that can run the binary) and it's up to you to set up the environment for your particular project. And again, this should be the case whether or not IHaskell is dynamically linked

I think you're taking my comments in a very negative light. Sorry. They were never meant to be negative.

No worries, I didn't take them as negative 🙂 My intent was simply to reply to your "do you mind fixing the Docker images" question with "I do mind because I was never intending to forever maintain the Dockerfile" with as little harsh-ness as possible, which I possibly failed in.

Code for the installed packages needs to come from somewhere. They aren't linked into the ihaskell binary (otherwise you would need to recompile IHaskell every time you added a package).

When you say "package", what do you mean here? Do you mean a dependency of IHaskell or a dependency of the project you're using IHaskell to run?

Even if IHaskell was implemented by having GHCi run in a different process it wouldn't help, then we'd want that GHCi to use the dynamic linker and those shared libraries.

Ok so let's pretend IHaskell is compiled statically linked, and implemented such that it receives Haskell code from you, then runs callProcess "ghc" ["-dynamic", "-e", <code>]. It sounds like you're saying this is still an issue because the shared libraries from the compilation of IHaskell itself don't exist to be reused. But my argument is that the build of IHaskell itself should be sandboxed from the runtime user environment. e.g. I should be able to build the IHaskell binary and send it to you, and you should be able to run it, with IHaskell building and linking third-party dependencies built in the usual project sandbox (e.g. ~/.stack or ~/.cabal/store). Whether IHaskell itself has the dependencies statically linked or is loading deps dynamically, the Haskell evaluator it's driving should be completely separate from that.
That last paragraph was me speaking generally; this all seems particularly ill-suited for the Docker use-case. If your project uses postgresql-libpq, how do you install the libpq libraries in the Docker image for IHaskell to compile/run your code against? One option could be that each of your (non-trivial) projects should have its own Dockerfile that extends the IHaskell one:
FROM ihaskell
RUN apt-get install postgresql-dev

That is, the IHaskell Docker image's sole responsibility is to provide the IHaskell binary (and an OS that can run the binary) and it's up to you to set up the environment for your particular project. And again, this should be the case whether or not IHaskell is dynamically linked

I'm happy to change this back, since among other things the start-up time without -dynamic is a lot slower and restarting the kernel takes forever. In fact we default to doing it the old way when using Nix. However this change is currently breaking the Nix build so I need to fix whatever is happening there  before this can be merged.

I can't merge this in its current state, so closing for now.


feat: add photoswipe for image zoom

Files are formatted by the Astro and Biome VSCode plugins.
The modified PhotoSwipeLightbox instance works with all images inside #post-container, regardless of image size.
Every time a route completes rendering, a new PhotoSwipeLightbox instance is created.
Every time before a route change, the old PhotoSwipeLightbox instance is destroyed.

close #131

Thank you for your contribution!

It seems this does not work well for online images. If possible, could you please open another PR to address it?


Upgrade Webpack to latest version

The upgrade to webpack v4 seems to break the way the BannerPlugin is working. The header comments are necessary for the production build.
Originally posted by @Mardaneus86 in https://github.com/Mardaneus86/futwebapp-tampermonkey/pull/226#r499487488

When i upgrade to v4 i do see the banners when configuring it as follows:
new webpack.BannerPlugin({
      banner: header,
      raw: false,
      entryOnly: true
    }),

In this case the banner won't be wrapped in a comment and therefore wont be obfuscated.

Let's release for FIFA 21 first, then review how we can get this working and remove the default obfuscation in webpack v4 since it isn't necessary for this project.

Sure!

but for sniping should i use the old script?

use the latest release @totti240282 , thanks

sorry but I don't see the menu to snipe

This is a technical issue for the developers, has nothing to do with functionality so you won't notice this.
Then again: There will never be features in this script for sniping or any other means that might be considered cheating by the FIFA community.


Different p-value calculation

freezing-archer calculates p-values differently with calc_table_match_pval and calc_local_match_pval.
For calc_table_match_pval, freezing-archer adds one in both numerator and denominator, this may be due to continuity correction or avoiding division by zero?
https://github.com/bvernot/freezing-archer/blob/master/bin/s_star_fns.py#L267
For calc_local_match_pval, freezing-archer doesn't add one in both numerator and denominator.
https://github.com/bvernot/freezing-archer/blob/master/bin/s_star_fns.py#L377
https://github.com/bvernot/freezing-archer/blob/master/bin/s_star_fns.py#L378

It looks like different approaches to calculate p-values would finally affect the calculation for posterior probabilities, suggesting numerically instability in these methods???


Manual testing build#256

Initial set-up
Check that the IntelliJ Idea UTBot plugin can be successfully installed

[x] Choose appropriate workflow from the list (by default, filter by main branch and take the latest one) build#256
[x] Download plugin
[x] Check downloaded zip-file size < 100 MB
[x] Open IntelliJ IDE
[x] Remove previously installed UTBot plugin
[x] Clone or reuse UTBot project (https://github.com/UnitTestBot/UTBotJava.git)
[x] Open the project in the IDE
[x] Install the downloaded plugin
[x] Restore default settings of UTBot: Settings -> Tools -> UTBot

Go through manual scenarios
Manual scenario #1

[x] Use default plugin settings
[x] Open the utbot-sample/src/main/java/org/utbot/examples/algorithms/ArraysQuickSort.java file
[x] Generate tests for the class
[x] Remove results
[x] Generate tests for the methods
reproduced #709

Manual scenario #2

[x] Use default plugin settings
[x] Open the utbot-sample/src/main/java/org/utbot/examples/mock/CommonMocksExample.java file
[x] Generate tests with all available (mocking) options
bug found: #747

Manual scenario #3

[x] Create a new Gradle project
[x] Add a simple java file to test
[x] Generate a test with the suggested test root

Manual scenario #4

[x] Create a new Maven project
[x] Add a simple java file to test
[x] Generate a test with a new test root
reproduced #712

Manual scenario #5

[x] Create a new Idea project
[x] Add a simple java file to test
[x] Generate tests for several classes

@denis-fokin new bug: https://github.com/UnitTestBot/UTBotJava/issues/747


Nice job! (+ some suggestions)

I just wanted to say cheers for making this.
The idea was on my todo-list, so thanks for saving me a bunch of time :D
The preview of the node is a brilliant touch!
A few suggestions / features that would improve the usability:

Being able to change the padding / spacing of the nodes and groups. They seem excessively thick right now and could certainly use some "Compact Mode" options. Ideally optional of course.

Looking at the screenshot above you can see that I can only fit the "Adv-Controlnet" categories and "AnimateDiff" on my screen.

Having favorite (pinned nodes) is a good idea, but I wouldn't put them at the top as it is. I will easily put 20-50 nodes as favorite, and now my list will be even more cluttered. It's still a good idea, but what about making it a tab-like structure instead. So for your left-side-bar menu you still have the list as it is of Custom Nodes. These are just called "Nodes". And then there's a "Pinned" tab, where all my pins are.

And an even more relevant option for me would be to be able to build my own menu of nodes. Meaning I can create my own directory structure and pick nodes from all over the place, and then use that list as my master list of nodes. Essentially a "Pinned" tabs, but with more structuring options like categories/subcategories.

The largest issue I have with Comfy (aside from buggy UI), is that nothing is cohesive, since it's all extension driven. So there's no clean  way for me to sort all my "String related nodes" into one category. I want to pick bits and pieces of downloaded extensions, without having to manually copy the code into my own collection, that would be a nightmare to manage.
If Pinned nodes could be sorted into a tiered structure, or my #3 suggestion would work, then I think we're closer to having a reliable UI.
Sidenote:

It doesn't play well with the [https://github.com/11cafe/comfyui-workspace-manager](Workspace Manager extension).

Excellent!!!

If you want to lower the bas just use the "[Sidebar] Space Top" in the ComfyUI option! :p and you can use the [Sidebar] Node Size for the point 1
I like the idea of a tab-like structure and even the custom category/node, what i dont really have is time right now so i will definetly try to do that, I just don't know when I will finish it 🤣

If you want to lower the bas just use the "[Sidebar] Space Top" in the ComfyUI option! :p and you can use the [Sidebar] Node Size for the point 1 I like the idea of a tab-like structure and even the custom category/node, what i dont really have is time right now so i will definetly try to do that, I just don't know when I will finish it 🤣

One step ahead good sir!

Thanks, those settings worked perfectly for me :)
Don't stress about the feature suggestions.
But it could certainly add additional usability!
This has already made my week though <3

Can you tell me how you made the picture you posted on Mar 25, 2024

Now I can close this since I've implemented the custom category option!


Weird \0 bytes

Hey, I'm seeing weird things with WebSockets. Basically, in onMessage(), I receive the correct string - but with additional superfluous null bytes attached (string displayed as hex in UTF8 format for illustrative purposes):
2b4d6f6e204d61792030372032333a34303a353620434553542032303138207c203630303538206d730a00000000
These can be fewer or more null bytes, depending on... the weather I guess. (?!!!)
I think I've seen the same with a JavaScript WebSocket client too, just with... fewer null bytes.
If I just drop these null bytes, the application works fine.
What's going on here?
Simple example.

Hello @stefan-reich ,
Please use the Bug Report template provided at https://raw.githubusercontent.com/TooTallNate/Java-WebSocket/master/.github/ISSUE_TEMPLATE.md.
Without a full bug report this will be closed.
Best regards,
marci4

You are trying to waste my time. I can provide a test program you can run if you like.
I'm using Java 1.8 on Peppermint Linux 64 bit. Standard stuff.

Missing template.
Closing issue!
Best regards,
marci4


Can't link the library

Expected Behaviour
After install the acpcore with npm, run "react-native link react-native-acpcore" should link the library to the app.
Actual Behaviour
showing error message
"Error: Failed to get dependency config
at getDependencyConfig (appPath/node_modules/@react-native-community/cli/build/commands/link/getDependencyConfig.js:49:11)
at Object.link [as func] (appPath/node_modules/@react-native-community/cli/build/commands/link/link.js:97:61)
at Promise.resolve.then (appPath/node_modules/@react-native-community/cli/build/cliEntry.js:152:22)
at process._tickCallback (internal/process/next_tick.js:68:7)
at Function.Module.runMain (internal/modules/cjs/loader.js:745:11)
at startup (internal/bootstrap/node.js:283:19)
at bootstrapNodeJSCore (internal/bootstrap/node.js:743:3)"
Steps to Reproduce
1.Have an existing react native app
2.cd MyReactApp
3.npm install @adobe/react-native-acpcore --save
4.react-native link react-native-acpcore
Platform and Version
Node v10.15.0
Npm 6.4.1
react-native-cli: 2.0.1
react-native: 0.59.3

HI @maoesx, can you share the the full log output when you run "react-native link react-native-acpcore? I'm able to link in new projects without issue.

@maoesx it should be react-native link @adobe/react-native-acpcore. Do we have some incorrect docs somewhere?

Yes, there's an error in the readme:
2.1 Link
React Native 0.60+
CLI autolink feature links the module while building the app.
React Native <= 0.59
$ react-native link react-native-acpcore (should be react-native link @adobe/react-native-acpcore)
Anyway, the new command works for me, thanks!

@maoesx Thanks for pointing out the typo, I'll get that corrected. Glad the suggestion worked on your end.


NonPrintableCheck sniff

Non printable chars sniffs for css,js and php files, implementing the themecheck plugin rule.
Reference #94

@grappler @jrfnl I have added the requested changes. The build fails on PHP 7 as the T_BAD_CHARACTER is no longer available.

@selul Thanks for the update. I'll try and do another review run through tomorrow.

@jrfnl I checked with the Theme Check plugin and I am getting the same result which makes sense as the regex is the same but the in the theme check this is only a info, so it is not required.
It would be good to do some research on the subject. @selul would you be willing to help out. Do symbols like °, ′, ″ count as non-printable characters and what negative affect can they have?

I spoke to Otto and the reason this was added in the Theme Check was to check for Byte Order Mark. Explanation. This is now being checked with Generic.Files.ByteOrderMark
The check was made a bit looser to try to catch other issues too.
I did a bit of research, non printable characters are things like line breaks. In order for this sniff to work we would need to think of places where it does not make sense to have non printable characters. I am not sure how many places there are.
More of the ASCII codes can be found in the links 1-31 and 127-159 are the invisible characters.
http://ascii.cl/htmlcodes.htm
http://www.ascii-code.com/

Closing this PR in favour of continuing the discussion in the original issue #94.
Once that discussion has run its course, we can always reopen.


Change progress of characters in Ch. 3 Co-op vs Bitrey (Lose)

In this section, it looks like only the blue miner is doing work and progressing. We should split this up more evenly across contributors. Let's reference the % distribution in #579.
https://github.com/saving-satoshi/saving-satoshi/assets/32662508/023d45d4-ca7e-47fd-9bd3-4a7a1a80fb1b
Originally posted by @ecurrencyhodler in https://github.com/saving-satoshi/saving-satoshi/pull/570#pullrequestreview-1546364979

Just adding some more info, it looks like the "Blocks found" do not increment for some of the players. It stays set to 1, which in turn means the ContributionBar doesn't update properly.

So the other miners are fixed at 1 block because they each need at least 1 block to show up on the contribution bar, in the co-op lesson however, these miners are not supposed to win any blocks as this is the lesson where all miners are using the same nonce to mine so only the highest hashrate is finding blocks

Wow. 🤯 You're right. Great catch.
Can we move the green rabbit behind the blue then? It looks funky having the green in front. I'll change the title of this issue.

I actually think that with the introduction of #570 I can remove the need to give all of the other miners 1 block to have them be visible

Okay. Sounds good!


First relase to production

Implement the merge from the develo branch to the main branch.
Perform the Merge
Description:
Merge the develop branch into the main branch.
Instructions:
Make sure you are on the main branch:
git checkout main
Perform the merge of develop in main:
git merge develop


Add introduce command

This command is only available to admins and mods.

?introduce @mention: DM's the mentioned user with an introduction to Codewars and its Discord server. Falls back to mentioning the user in the channel if DM is not available.
?introduce @mention #help-solve: DM's the mentioned user with guidelines on asking for help on solving Codewars Kata. Falls back to mentioning the user in the channel if DM is not available.
?introduce @mention #off-topic: DM's the mentioned user with an introduction to the off-topic channel and general guidelines on participation. Falls back to mentioning the user in the channel if DM is not available.

I have implemented most of the suggestions from the previous pull request, except that the messages are currently hardcoded in the source code instead of in separate text files. It was mentioned that the text files containing the messages should be in a separate directory, but which directory should we put it in? Once we've agreed on that, I can move the message contents out into a separate directory and fix any remaining issues.

?introduce @mention #off-topic has been replaced with ?introduce @mention #anything in response to the renaming of the affected channel.

It was mentioned that the text files containing the messages should be in a separate directory, but which directory should we put it in?

Messages can be under text/ next to the command. I think that makes the most sense.

Thanks for the feedback! I suppose we just keep ?introduce @mention #help-solve and perhaps add a ?warn command in another pull request?

I removed ?introduce @mention and ?introduce @mention #anything and moved the message contents for ?introduce @mention #help-solve into a top-level directory text/. The reason I did not place text/ alongside the source file for the command introduce.ts is because the compiled file introduce.js lives in a different directory from introduce.ts so I cannot just do path.join(__dirname, 'text') to access that directory (though it can be fixed by replacing /src with /lib). Is that fine by you?

The reason I did not place text/ alongside the source file for the command introduce.ts is because ...

Oh I forgot about that, sorry. Yeah, that's fine.

Suggestions applied, please review

Thanks!


Crash in dmz_YCbCr_to_RGB(_IplImage*, _IplImage*, _IplImage*, _IplImage**)

Hi,
David from Uber here. I was emailing back and forth with Jake (PayPal Technical Account Manager) and he recommended I open a GitHub issue so we can better track this issue.
We are still seeing this elusive crash that was addressed in the following pull request https://github.com/card-io/card.io-iOS-source/pull/57. We see this across iOS versions 7, 8, 9 and iPhone devices 6s, 6, 6 Plus, 5s with around 1k crashes occurring in the past week ending 4 Apr 2016.
CardIO version 5.3.1 (https://github.com/card-io/card.io-iOS-SDK/tree/5.3.1)
Here is the stack trace:
Hardware Model:      iPhone6,1
Process:             UberClient [8093]
Path:                /var/mobile/Containers/Bundle/Application/3FCCECC0-3042-4168-8B0D-695136B95522/UberClient.app/UberClient
Identifier:          com.ubercab.UberClient
Version:             2.128.3 (2.128.3)
Code Type:           ARM-64
Parent Process:      unknown [1]
Date/Time:           2016-04-05 01:43:01.000 -0400
OS Version:          iPhone OS 9.2.1 (13D15)
Report Version:      104
Exception Type:      EXC_BAD_ACCESS (SIGBUS)
Exception Codes:     KERN_INVALID_TASK at 0x0000000000000010
Crashed Thread:      0

Thread 0 Crashed:
0   UberClient                     0x000000010085ba94 dmz_YCbCr_to_RGB() + 272
1   UberClient                     0x000000010085b9e4 dmz_YCbCr_to_RGB() + 96
2   UberClient                     0x0000000100840670 -[CardIOIplImage rgbImageWithY:cb:cr:] + 152
3   UberClient                     0x000000010083c04c -[CardIOVideoFrame imageWithGrayscale:] + 204
4   UberClient                     0x000000010087bd5c -[CardIOView didScanCard:] + 460
5   UberClient                     0x000000010087b924 -[CardIOView videoStream:didProcessFrame:] + 128
6   UberClient                     0x000000010087a2b4 -[CardIOCameraView videoStream:didProcessFrame:] + 484
7   UberClient                     0x0000000100847ba0 -[CardIOVideoStream sendFrameToDelegate:] + 100
8   Foundation                     0x00000001814ebe20 __NSThreadPerformPerform + 340
9   CoreFoundation                 0x0000000180ae0efc __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 24
10  CoreFoundation                 0x0000000180ae0990 __CFRunLoopDoSources0 + 540
11  CoreFoundation                 0x0000000180ade690 __CFRunLoopRun + 724
12  CoreFoundation                 0x0000000180a0d680 CFRunLoopRunSpecific + 384
13  GraphicsServices               0x0000000181f1c088 GSEventRunModal + 180
14  UIKit                          0x0000000185884d90 UIApplicationMain + 204
15  UberClient                     0x00000001007fd9ec main + 260
16  libdyld.dylib                  0x00000001805ae8b8 start + 4

Thread 0 crashed with ARM-64 Thread State:
  cpsr: 0x0000000060000000     fp: 0x000000016fd569c0     lr: 0x000000010085b9e4     pc: 0x000000010085ba94
    sp: 0x000000016fd56960     x0: 0x0000010e000001ac     x1: 0x000000018078a8f4    x10: 0x00000000000001ac
   x11: 0x0000000000000000    x12: 0x00000000000000ac    x13: 0x0000000101ff5020    x14: 0x0000000000000000
   x15: 0x0000000000000001    x16: 0xfffffffffffffff1    x17: 0x0000000100840910    x18: 0x0000000000000000
   x19: 0x0000000145886480     x2: 0x0000000000058000    x20: 0x0000000000000000    x21: 0x0000000000000000
   x22: 0x00000001459188b0    x23: 0x0000000000000003    x24: 0x000000018620c367    x25: 0x0000000000000000
   x26: 0x0000000000000000    x27: 0x000000019d24e000    x28: 0x000000010209c060    x29: 0x000000016fd569c0
    x3: 0x0000000000000000     x4: 0x0000000003000001     x5: 0x0000000000000000     x6: 0x0000000000000000
    x7: 0x0000000000000000     x8: 0x000000010cda8010     x9: 0x0000000000000001

We still cannot reproduce this reliably.
I have a hunch it may be to do with (CMSampleBufferRef)sampleBuffer disappearing before it hits [CardIOIplImage rgbImageWithY:cb:cr] (stack frame 2). According to the Apple documentation (https://developer.apple.com/library/mac/documentation/AVFoundation/Reference/AVCaptureVideoDataOutputSampleBufferDelegate_Protocol/#//apple_ref/occ/intfm/AVCaptureVideoDataOutputSampleBufferDelegate/captureOutput:didOutputSampleBuffer:fromConnection:), sampleBuffer is not guaranteed to stick around after method return, however its image data is being referenced well afterwards.
A direct reference to the pixel buffer of the sampleBuffer is stored in colocatedImage->imageData of CardIOIplImage instances. This buffer is accessed in a callback that is dispatched by performSelectorOnMainThread: in CardIOVideoStream, well after captureOutput:didOutputSampleBuffer:fromConnection: has returned. This memory might have been recycled by the system. Not sure how robust OpenCV is to corrupt image data, but could be worth a look.
if you need more information, feel free to reach out. Thanks for your help!
Cheers,
David

Thanks for the investigation David. We are taking a look.

Thanks for your detailed investigation. I wasn't able to reproduce the crash but based on your description and the Apple documentation, I made a few changes which should help in release 5.3.2. If you still see issues, please leave a comment.

Thanks @bluk , we'll integrate 5.3.2 and let you know our findings.

FWIW, we at SpotHero also saw this crash. Will also let you know if we see recurrence after upgrading to 5.3.2.

Just had a recurrence again today, after a release of the upgrade to v5.3.2 of the SDK. iPhone 6 running 9.3.1. Looks like the stack trace is largely the same as @nudge's pre-5.3.2 trace:
Crashed: com.apple.main-thread
EXC_BAD_ACCESS KERN_INVALID_ADDRESS 0x0000000000000010

0  SpotHero                       0x10027c7bc dmz_YCbCr_to_RGB(_IplImage*, _IplImage*, _IplImage*, _IplImage**) +<PHONE_NUMBER>
1  SpotHero                       0x1002613ec +[CardIOIplImage rgbImageWithY:cb:cr:] +<PHONE_NUMBER>
2  SpotHero                       0x10025cd50 -[CardIOVideoFrame imageWithGrayscale:] +<PHONE_NUMBER>
3  SpotHero                       0x1002a50a8 -[CardIOView didScanCard:] +<PHONE_NUMBER>
4  SpotHero                       0x1002a4c70 -[CardIOView videoStream:didProcessFrame:] +<PHONE_NUMBER>
5  SpotHero                       0x1002a3610 -[CardIOCameraView videoStream:didProcessFrame:] +<PHONE_NUMBER>
6  SpotHero                       0x100268964 -[CardIOVideoStream sendFrameToDelegate:] +<PHONE_NUMBER>
7  Foundation                     0x183f9bffc __NSThreadPerformPerform + 340
8  CoreFoundation                 0x18357d124 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 24
9  CoreFoundation                 0x18357cbb8 __CFRunLoopDoSources0 + 540
10 CoreFoundation                 0x18357a8b8 __CFRunLoopRun + 724
11 CoreFoundation                 0x1834a4d10 CFRunLoopRunSpecific + 384
12 GraphicsServices               0x184d8c088 GSEventRunModal + 180
13 UIKit                          0x188779f70 UIApplicationMain + 204
14 SpotHero                       0x100169de4 main (main.m:14)
15 libdispatch.dylib              0x1830428b8 (Missing)

Crashlytics shows no memory or disk space pressure for this crash, so that's probably not it. FWIW, it's the first crash we've seen for this in about a month.

@designatednerd Thanks for the update. From what you're saying, I'm going to assume that the v5.3.2 fix helped since there wasn't a crash for a month, but there may still be another memory issue. Will go over the code again and see if there's any other noticeable memory issues.

@bluk Very possible, but I think @nudge was seeing this crash significantly more often in the first place, so it may be worth seeing how they're doing before you go too far down the rabbit hole.

We're still seeing this crash after integrating 5.3.2 into our builds.
The crash rate looks roughly the same as with pre-CardIO 5.3.2. Seeing around 1k crashes for the week ending 16-Apr 2016. We're still seeing it spread across many device types (iPhone 5-6s Plus) and OS versions (8.4-9.3.1). Will continue to do some more investigation and monitoring.
Incident Identifier: ED252731-4EB1-48C3-9A30-99763F8343C1
CrashReporter Key:   b7a6443459f54b8289e898865e93cdd28fd6138c
Hardware Model:      iPhone6,2
Process:             UberClient [710]
Path:                /var/containers/Bundle/Application/AC23CF6D-268B-418E-98C3-8A1EDCC4B942/UberClient.app/UberClient
Identifier:          com.ubercab.UberClient
Version:             2.133.2 (2.133.2)
Code Type:           ARM-64
Parent Process:      ? [1]
Date/Time:           2016-05-17 18:36:54.000 +1000
OS Version:          iPhone OS 9.3.1 (13E238)
Report Version:      104
Exception Type:      EXC_BAD_ACCESS (SIGBUS)
Exception Codes:     KERN_INVALID_TASK at 0x0000000000000010
Crashed Thread:      0

Thread 0 Crashed:
0   UberClient                     0x0000000100948a3c dmz_YCbCr_to_RGB() + 272
1   UberClient                     0x0000000100948988 dmz_YCbCr_to_RGB() + 92
2   UberClient                     0x000000010092d66c -[CardIOIplImage rgbImageWithY:cb:cr:] + 152
3   UberClient                     0x0000000100928fd0 -[CardIOVideoFrame imageWithGrayscale:] + 204
4   UberClient                     0x0000000100971328 -[CardIOView didScanCard:] + 460
5   UberClient                     0x0000000100970ef0 -[CardIOView videoStream:didProcessFrame:] + 128
6   UberClient                     0x000000010096f890 -[CardIOCameraView videoStream:didProcessFrame:] + 456
7   UberClient                     0x0000000100934be4 -[CardIOVideoStream sendFrameToDelegate:] + 100
8   Foundation                     0x0000000181417ffc __NSThreadPerformPerform + 340
9   CoreFoundation                 0x00000001809f9124 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 24
10  CoreFoundation                 0x00000001809f8bb8 __CFRunLoopDoSources0 + 540
11  CoreFoundation                 0x00000001809f68b8 __CFRunLoopRun + 724
12  CoreFoundation                 0x0000000180920d10 CFRunLoopRunSpecific + 384
13  GraphicsServices               0x0000000182208088 GSEventRunModal + 180
14  UIKit                          0x0000000185bf5f70 UIApplicationMain + 204
15  UberClient                     0x00000001008e034c main + 260
16  libdyld.dylib                  0x00000001804be8b8 start + 4

Thread 0 crashed with ARM-64 Thread State:
  cpsr: 0x0000000060000000     fp: 0x000000016fd0a9d0     lr: 0x0000000100948988     pc: 0x0000000100948a3c
    sp: 0x000000016fd0a980     x0: 0x0000010e000001ac     x1: 0x000000018069e8f4    x10: 0x00000000000001ac
   x11: 0x0000000000000000    x12: 0x000000000000009e    x13: 0x0000000102174ed0    x14: 0x0000000000000000
   x15: 0x0000000000000001    x16: 0xfffffffffffffff1    x17: 0x000000010092d928    x18: 0x0000000000000000
   x19: 0x0000000149c8a970     x2: 0x0000000000058000    x20: 0x0000000000000000    x21: 0x0000000000000000
   x22: 0x000000014889e410    x23: 0x0000000000000003    x24: 0x00000001865a5b1d    x25: 0x0000000000000000
   x26: 0x0000000000000000    x27: 0x000000019da49000    x28: 0x0000000102198058    x29: 0x000000016fd0a9d0
    x3: 0x0000000000000000     x4: 0x0000000003000001     x5: 0x0000000000000000     x6: 0x0000000000000000
    x7: 0x0000000000000000     x8: 0x000000010a19c010     x9: 0x0000000000000001

@bluk: Any update on this?

@designatednerd Unfortunately, no update right now. Haven't been able to reproduce this on different devices but will do more audits of the memory management pieces.

From Fabric analysis I found that we only have this crash on the iPad, and I mean 100% iPad.
The iPad only represents about 5% of our iOS devices, so this seems relevant.
I hope it helps.

Hi guys.
From Fabric analysis we also constantly have the same issue and even for Card.io version 5.3.2.
Crashed: com.apple.main-thread
0  Rider                          0x1019514b8 dmz_YCbCr_to_RGB(_IplImage*, _IplImage*, _IplImage*, _IplImage**) +<PHONE_NUMBER>
1  Rider                          0x1019360e8 +[CardIOIplImage rgbImageWithY:cb:cr:] +<PHONE_NUMBER>
2  Rider                          0x101931a4c -[CardIOVideoFrame imageWithGrayscale:] +<PHONE_NUMBER>
3  Rider                          0x10197a024 -[CardIOView didScanCard:] +<PHONE_NUMBER>
4  Rider                          0x101979bec -[CardIOView videoStream:didProcessFrame:] +<PHONE_NUMBER>
5  Rider                          0x10197858c -[CardIOCameraView videoStream:didProcessFrame:] +<PHONE_NUMBER>
6  Rider                          0x10193d660 -[CardIOVideoStream sendFrameToDelegate:] +<PHONE_NUMBER>
7  Foundation                     0x1826c802c __NSThreadPerformPerform + 340
8  CoreFoundation                 0x181ca909c CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION + 24
9  CoreFoundation                 0x181ca8b30 __CFRunLoopDoSources0 + 540
10 CoreFoundation                 0x181ca6830 __CFRunLoopRun + 724
11 CoreFoundation                 0x181bd0c50 CFRunLoopRunSpecific + 384
12 GraphicsServices               0x1834b8088 GSEventRunModal + 180
13 UIKit                          0x186eba088 UIApplicationMain + 204
14 Rider                          0x100c89c68 main (AppDelegate.swift:20)
15 libdispatch.dylib              0x18176e8b8 (Missing)
Are there any news regarding this issue?

Any update on this? Still seeing multiple crashes.

We're getting multiple reports of crashes because of this. Any update?

It looks like -[CardIOIplImage rgbImageWithY:cb:cr:] could implement some sanity checks for NULL pointers, given that the dmz/opencv code it calls doesn't check anything.

Hello, do you guys have any chance to look closer into this? This is a consistent crash producer for our users.

The project seems dead (no commit in 5 months)


Update to Angular v9 automatically checks in the result!

🐞 Bug report
Command (mark with an x)

- [ ] new
- [ ] build
- [ ] serve
- [ ] test
- [ ] e2e
- [ ] generate
- [ ] add
- [x ] update
- [ ] lint
- [ ] xi18n
- [ ] run
- [ ] config
- [ ] help
- [ ] version
- [ ] doc

Is this a regression?

 Yes, the previous version in which this bug was not present was: ....
Description
 When running: `ng update @angular/cli @angular/core --next --allow-dirty`
It automatically checked in the result to my git repo. This is definitely NOT desired. In many use cases, this may be a "Trial" without a desire to check in. In other use cases, this may be a key dev environment that the developers would want to try out before committing.
In any case, this should NOT automatically check in without at least asking first.
UPDATE: In looking into this further, it does THREE commits during the update process.
🔬 Minimal Reproduction

Run: ng update @angular/cli @angular/core --next --allow-dirty

Check the Git change list, all changes are check in.

🔥 Exception or Error

No error

🌍 Your Environment

Was 8.1.3 prior to the update.

Now:
Angular CLI: 9.0.0-rc.0
Node: 10.16.0
OS: win32 x64
Angular: 9.0.0-rc.0
... animations, cli, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.900.0-rc.0
@angular-devkit/build-angular     0.900.0-rc.0
@angular-devkit/build-optimizer   0.900.0-rc.0
@angular-devkit/build-webpack     0.900.0-rc.0
@angular-devkit/core              9.0.0-rc.0
@angular-devkit/schematics        9.0.0-rc.0
@ngtools/webpack                  9.0.0-rc.0
@schematics/angular               9.0.0-rc.0
@schematics/update                0.900.0-rc.0
rxjs                              6.5.3
typescript                        3.6.4
webpack                           4.41.2

Anything else relevant?

@vikram @StephenFluin

This is especially hard because users that are not very proficient with git will have a hard time removing these commits.

This behavior can be disabled via the --skip-commits/-C option.
I believe the concept here was that an update is similar to the development of a PR for a feature or bug fix.  A user would create a branch and then make a set of changes.  After completion, adjustments would be made to ensure everything is working and then the results would be squashed/rebased and then pushed upstream for review and eventual merge.
The commit per migration also has the advantage of allowing the changes from each migration to be viewed in isolation and altered with a higher degree of confidence, if needed.  Explanations for the migrations are also contained within the corresponding commit messages.
cc @IgorMinar

We don't tell users to make a branch in the update guide though, I think.
I also wonder how these commits interact with projects that have git hooks for the commit messages. One of the projects I'm testing updates on uses husky for that.

This may work well for some users and use cases ... but not all developers using Angular are well experienced with git or have other standard processes and procedures.
Plus with the automatic check-in, it is much more difficult to see which files that the update modified.
Also, since it does at least 3 check ins, it makes it even harder for a developer to undo this.
We should NOT be doing this without explicit permission. Make the flag --skip commits true by default and/or ask a question. Then those devs with experience and who work the way you are expecting can CHOSE to check in.
Thanks!

That was just an example use case; although one that probably should be considered a recommended practice.
As to git hooks, they intentionally don't interact: git commit --no-verify -m "${message}"

Related to https://github.com/angular/angular-cli/issues/14600

I agree this can cause issues with those inexperienced with git and it is something that should be considered. That linked issue, however, does provide an additional use case for the current behavior.  Performing an update on top of an unknown quantity of uncommitted changes can have catastrophic results to a project.  Although, the linked situation could be mitigated by adding an additional message instructing the user to verify and commit the results of the update operation.
As an aside and for anyone curious, the command to remove the three last commits and keep the changes is git reset HEAD~3.  To remove the commits and the associated changes: git reset --hard HEAD~3

I agree that this is certainly a surprising, even startling, change for many Angular CLI users.
At minimum, if this is the default behavior, there need to be workspace options in angular.json to enable both --allow-dirty and --skip-commits for users who want the previous behavior.
I've tried a number of updates now across different apps with this behavior in place and I can tell you that it was never desirable in any of those cases.

The commit message needed to be reworded (not trivial for many Git users)
The commit needed to be amended to add fixes or changes so that tests would pass and the build would work. The repo doesn't want commits that don't build/pass tests.
The commits needed to be squashed in some cases (not trivial for many Git users)

I've had to use the --allow-dirty flag manually from the CLI almost one hundred times now and every time I wish that I could just set a flag to make that the default behavior.
In my experience doing updates to real world apps (which is significant), I've run into the above issues many times with this new behavior. I've never run into a case where I had a dirty branch with lots of unrelated changes and ended up with a catastrophic result.
I am perfectly happy with a prompt or warning that costs me one extra key press during ng update to verify that my current branch is in a good state.

Although --allow-dirty is related to the commit functionality, I don't think it should be conflated with the auto-commit facility as originally listed in this issue.

Perhaps a middle ground solution would be to ask after each update step (package changes, migration script) that makes a change to inform the user about the changes and ask if they wish to commit or continue?
The challenge of not committing is that its very easy for changes to happen by the user that break the migration, and then its hard to track what happened. Git history can be a big help here, but as others have said it depends on one's familiarity with git. When 100s of files are touched automatically, I need confidence where those changes came from.

The name --skip-commits is hard to associate with my desire to disable automatic committing of migration changes during ng update. It would expect something like ng update --auto-commit or ng update --auto-commit=false.

I suggest we make this a CHOICE:

Off by default
The options for automatic check in more clearly defined in the update instructions so those that want it can get the auto check ins by setting a flag.

As it is, the developer does not have an opportunity to see what is changed or easily undo the changes (minimum of THREE check ins) especially if they are not git experts.
If we do need to keep this feature ... I suggest we add LARGE warnings everywhere that this auto check in will happen and that developers cannot just "try out" an update.

Wow. All that’s missing is to skip the PR (why bother?) and push the branch (master of course). What could be easier? :wink:
I would definitely be AGAINST automatic commits. I am unpersuaded by the arguments in favor.
The following is a list of quotes (for and against) and my commentary from a slack channel thread.
“It means that the exact changes made are in history now and I don’t accidentally make other changes and lose track of them in the mess.”
Not convinced: The changes are in the working set (or maybe staged) so there is no history to lose. If I make more changes during the upgrade that is almost certainly because I had to make those changes as part of the update.
“There are a lot of migration steps that might run, it would be nice if those were individually committed”
Yes but will I be able to make sense of the intermediate commits? Are they distinct in a way that is meaningful to the developer? Could I undo any of them? Reorder? Squash? I don’t really know why one commit or three and I am not sure why I should care.
“there are errors that weren’t fixed automatically but they get lost in the stream of logs and easily missed.”
Is that an argument FOR auto-committing?  If an upgrade fails, after 2 previous commits, what is my recourse? Do I fix and restart? How? How do I roll back? Assume I’m a git tenderfoot who neglected to start a new branch (still in master) before upgrading. TBH, I am a person who has been trapped mid-update.  When that happens, all I want to do is cry, rollback, and post a message on the gde channel.
“The commit per migration [allows] changes from each migration to be viewed in isolation and altered with a higher degree of confidence, if needed. Explanations for the migrations are also contained within the corresponding commit messages. ”
This argument has some merit … although when you add that there can be “hundreds or more small changes across the entire codebase”, I think you’ve undermined that argument.
The commit message is boilerplate. Could emit the combined messages to console or to a dated v9_conversion.readme.txt file
“I personally would like to check the changes once again before it gets committed.”
I would like to do that too. While many people have tooling that makes it somewhat easy to review the changes wrought by recent commits, it sure is nice to see them in VS Code as pending changes.
This behavior can be disabled via the --skip-commits/-C option.
Seriously? Who knows about that … or will remember to add it?
“This is especially tough because users that are not very proficient with git will have a hard time removing these commits.”
I agree strongly. At least we should provide a mechanism to revert (perhaps display a git reset SHA --hard  advice in the console when it’s over.
Committing feels too paternalistic. If I know how to use git, you have kind of offended me. If I’m new to git, you’ve set me up for a potentially nasty fall as I do what the console invites me to do: npm push.
The slack thread had more arguments in favor but I think I've addressed the strongest of them.
Reasonable minds can (and do) differ on the value of auto-committing.
I’d like to have the choice: (no commit by default) and have the benefits of auto-commit explained … along with a stern warning against the git push reflex … both BEFORE and AFTER.

Just my 2 cents as well.
When I saw the autocommit during the upgrade I was surprised. For me personally it is not that much of a big deal, as I'm doing the upgrade in a separate branch anyway and I used to always create intermediate commits during the upgrade (like after upgrading core and cli, and then after material etc).
That said, I would never leave those commits as is, as they do not comply with my conventional commit messages. So I'd rather squash/rebase whatever those commits before merging them in.
But, I see this a bit problematic for the "average developer" where - unfortunately - I still see a lot of problems related to everyday Git usage (that's why I created a hugely successful Git course on Egghead). So they would be irritated and wouldn't necessarily know how to undo these automatic commits etc.
So

I'd rather ask users during the upgrade if they want to commit the changes
provide opt-in flags to pass to the ugprade
explicitly mention it in some docs

Just want to point out that the --skip-commits option is new to Angular CLI v9. So for the (typical) case when updating from v8 to v9 the workflow as per the migration guide (https://next.angular.io/guide/updating-to-version-9) looks like this:

run npm install --no-save<EMAIL_ADDRESS>&& ng update @angular/cli @angular/core --next. Note that trying to use the --skip-commits option here will fail as it doesn't exist in v8
ng update runs, updating @angular/cli to version 9. Then the new version (?) of the CLI is invoked to migrate @angular/core to v9, with the --skip-commits option now available. But the user can't really interrupt this migration to tell the CLI not to commit.

The only way around this I could find was to kill the update process at the point where the package.json had been changed but the migrations had not yet been run (ie. no auto-commits yet). Then I explicitly ran the migrations with --migrate-only --skip-commits --allow-dirty.

@benelliott There is no need to attempt to kill the process.  All options are passed during the update bootstrap process.
Executing the following will result in no commits being generated:
npm install --no-save<EMAIL_ADDRESS>&& ng update @angular/cli @angular/core --next --skip-commits

This was tested with the latest published packages against a CLI 8.3.0 & Framework 8.2.13 project.

@clydin Sorry about that - not sure what I did wrong initially!

I'm in favor of flipping the default to be skipping the commits and leaving the sum of all changes on disk. I updated a bunch of applications last night, and I believe the most common usecase for users is to run a command that they don't fully understand (Migration can't be applied one by one as I understand it), and then analyze what it did via git status / visual tools.

+1  for @DeborahK and the following! When you know there’s a new commit, this feature is more or less okay. But since there are schematics active that could update my code I want to check what they did before commiting this. This could be confusing for people who don’t know that a commit will be made. Of course everything is possible if you know what you're doing, I can undo a commit at any time… but anyway. I don’t see the real value of this change ;-)

In addition to the above where I certainly agree with @DeborahK and @wardbell,
and I want to see this as an optional future that is off by default as @StephenFluin does, I want to add the following.

this is a breaking change, for all the above reasons.
as it does multiple commits, undoing it needs more git knowledge as the average developer has. (not hard, but needs a lookup)
with multiple commits it needs multiple steps to undo!
breaks expectations from the past

the path forward could be 1 of 2:

Make this optional, so the old behavior is the default
automatically create a branch for this.

With option 2 you keep most of the benefits from the old way, and the new way.
at the cost of 1 extra developer step (merge in the branch)

Just a quick note on point 3,  undo-ing the update no matter the number of commits can be a single command if desired: git reset --hard <SHA before the update>.  In that same regard,  there is also a proposal for a future addition to provide fallback in the event of an update failure by querying the user and then executing the aforementioned along with a package install if the user answered in the affirmative.  (Assuming the user did not use the --allow-dirty option in which case such an action would be dangerous and could erase an untold amount of the users work.)

@clydin looking up the <SHA before the update> is a step in itself in my opinion.

I ran the ng update @angular/cli @angular/core command without knowing what to expect. I was going to look at changed files to learn about all the changes. But, if you hiding the changes by committing them, then that doesn't help me understand whats happening.
I too vote for what @StephenFluin is suggesting.

@SanderElias That is correct.  However, it would always be two steps regardless of the number of commits.  Many IDEs with git integration (highly recommended in general) will also show the git history graphically which can reduce the user actions to only one git command.  The outlined proposal above would also completely remedy the need for any lookup which, I agree, could be confusion for someone with limited git experience.

@clydin The current version of the CLI doesn't do any commit's. So reverting back to the last known good state is always:
git reset HEAD --hard

And done. 1 step, no second guesses.
Also, I could easily see all the changes to my app in the git diff in my vs-code, without having to do anything.
Now that also takes additional steps.
I really hope this is not set in stone just yet!

If auto-commit remains the default, can we print near the bottom of the console log

a notice explain that the upgrade changes have been committed
the git command that reveals those commits (git log --oneline -n) where n is the exact number of said commits
advice for rollback containing the exact git command that will reverse those commits (git reset --hard SHA with the actual SHA).

Such actionable information may go far to reduce anxiety.

Alternatively, can we meet the requirement of providing migration change information by adding comments in the code? Something like this:
  // MIGRATION: Static flag migration removes the `static` flag from dynamic queries.
  // As of Angular 9, the "static" flag defaults to false and is no longer required for your view and content queries.
  // Read more about this here: https://v9.angular.io/guide/migration-dynamic-flag
  @ViewChild('filterElement') filterElementRef: ElementRef;

The benefits of this are:

The developer can readily see the change where it happens instead of a (potentially large) set of console logs.
The developer will more easily see the helpful text (read more...) WHERE IT IS CHANGED, instead of having to read through the console logs. (This set of text is not in the commit message.)

The downside is:

The CLI is adding comments to the code!
Too big of a change this late in the RC process?

What about a dedicated log file in some exploitable format, yaml? Explaining which files have changed and why.
It can be quite annoying to have the same comment block on every migrated part of the code... or maybe we need a schematic to remove the comments 😅

2cents here
If it's going to be done automatically, it should be squashed into one commit. But then that opens a lot of possibilities for errors and doesn't really provide a clear way to see what was changed.
If it's going to be opt-in, then it could prompt the user asking if they wanted those changes to be committed.
TL;DR, don't automatically commit changes 😄

This was resolved by changing the default to not commit and instead provide it as an option via --create-commits.
Thanks for the feedback everyone.


translate img

Job d2l-zh/PR-1003/1 is complete.
Check the results at http://preview.d2l.ai/d2l-zh/PR-1003/

Job d2l-zh/PR-1003/2 is complete.
Check the results at http://preview.d2l.ai/d2l-zh/PR-1003/

Job d2l-zh/PR-1003/3 is complete.
Check the results at http://preview.d2l.ai/d2l-zh/PR-1003/

Job d2l-zh/PR-1003/4 is complete.
Check the results at http://preview.d2l.ai/d2l-zh/PR-1003/

Job d2l-zh/PR-1003/5 is complete.
Check the results at http://preview.d2l.ai/d2l-zh/PR-1003/

Job d2l-zh/PR-1003/6 is complete.
Check the results at http://preview.d2l.ai/d2l-zh/PR-1003/


Performance degradation with HF Trainer when training LayoutLMv3

System Info
peft == 0.4.0
transformers == 4.31.0
accelerate==0.21.0
Colab notebook shared below
python 3.10.6
Who can help?
@pacman100 @sayakpaul, (and potentially @NielsRogge as I've adapted the notebook from his examples)
Information

[X] The official example scripts
[X] My own modified scripts

Tasks

[X] An officially supported task in the examples folder
[X] My own task or dataset (give details below)

Reproduction
In this notebook, I train LayoutLMv3 with LoRa in two ways:

using the HuggingFace Trainer
using a manual training loop

The issue is that the HF Trainer yields far worse results (f1=0.68) than that of the manual training (f1=0.89), even when the Trainer trains for 3x more epochs than the manual training.
Note that I've adapted Sourab's training loop from the official example for token classification with LoRa using LayoutLM (v1)
Expected behavior
Using the HF Trainer yields similar results to the manual training loop

So just to ensure I understand correctly, you're using LayoutLMv3 with PEFT (LoRA) and the issue is that your manual training loop works better than HF Trainer? In that case, it sounds to me like this is not a PEFT issue.
In general, it is always quite hard to get the exact same results with two different training methods. Are you very sure that the hyper-parameters are 100% identical between the two? If yes, then I would suggest opening an issue in the transformers repo with code that shows the difference, people there might be able to help you.

@BenjaminBossan I tried to get the configurations/hyperparameters of the two training methods to be as close as possible to each other (to the best of my knowledge, of course).
I'll open an issue on the transformers side, thank you!


Save prompt crashes to api screen for keywords

Issues
When I entered values in for a keyword then click the exit button, prompt came up, I hit save then got this screen:

Seems no errors but did get this:
Will-change memory consumption is too high. Budget limit is the document surface area multiplied by 3 (1915396 px). Occurrences of will-change over the budget will be ignored.

No errors on git bash
Step 3.27 on script

Did you download a release or pull from Master? Which release?
release 22.0
What operating system are you running?10 pro
Are you running locally or in the cloud?
local
Which browser are you using?
FF

repeat the issue, confirmed this is crash

what is the result of docker-compose ps?

articulate-v0220_api_1             /bin/sh -c dockerize -wait ...   Exit 137
articulate-v0220_duckling_1        /bin/sh -c stack exec duck ...   Exit 137
articulate-v0220_elasticsearch_1   /usr/local/bin/docker-entr ...   Exit 143
articulate-v0220_rasa_1            ./entrypoint.sh start --pa ...   Exit 0
articulate-v0220_redis_1           docker-entrypoint.sh redis ...   Exit 0
articulate-v0220_ui_1              yarn start:prod                  Exit 1

Did you shut it down? That shows everything is down.
either way @dcalvom I can confirm this.

Create a keyword
Give it a name
Add some values
Click exit instead of save
in the exit prompt click save
page crashes.

@dcalvom @RussellSPG we should also check the other places we have those exit buttons (action screen? agent screen?)

Checked for Actions and no problem. Agent screen doesn't have an exit button.

Same problem for when hitting a delete for keyword

Error: An internal server error occurred main.c636e3c573fa6f0f0ce9.chunk.js:1:52865

Just to keep in mind, the keyword does delete, just crashes to the api screen and have to keep refreshing

Closed by https://github.com/samtecspg/articulate/commit/009545f16156fb8844af98763fa1f388f01cdb51


Add schemaName field to Channel

I'm proposing to add also the "schemaName" field, next to the "schemaVersion" that was added while ago.
It looks that apart from wildfly-channel YAML configuration files, there is also going to be a prospero YAML configuration file which defines other aspects of provisioning configuration. Currently it contains list of channels (specified via GAVs or URLs), and list of Maven repositories. Example of current prospero config:
channels:
    gav: "org.jboss.eap.channels:eap-8.0-beta:1.0.0.Beta-redhat-00001"
repositories:
  - "id": "central",
    "url": "https://repo1.maven.org/maven2/"
  - "id": "mrrc",
    "url": "https://maven.repository.redhat.com/ga/"
  - "id": "channel",
    "url": "file:///home/thofman/tmp/m2-repo"

It would help to avoid confusing one config file with the other if we specify schema names for each config. The proposal is to have e.g.:
schemaName: "wildfly-channel"
schemaVersion: "1.0.0" # already present
name: "Channel for org.jboss.eap:wildfly-ee-galleon-pack:pom:8.0.0.Beta-redhat-20220614\
  \ feature pack."
description: "Generated by org.wildfly.galleon-plugins:wildfly-galleon-maven-plugin\
  \ at 2022-06-14T16:21:17.793742Z"
streams:
  - groupId: "antlr"
    artifactId: "antlr"
    version: "2.7.7.redhat-7"
  - ...

and:
schemaName: "prospero"
schemaVersion: "1.0.0"
channels:
    gav: "org.jboss.eap.channels:eap-8.0-beta:1.0.0.Beta-redhat-00001"
repositories:
  - "id": "central",
    "url": "https://repo1.maven.org/maven2/"
  - ...

Cc @jmesnil

"unexpected attribute" -> is that an error from the wildly-channel.
afaict, if it reads an invalid YAML file, it should report something like "Invalid Channel" (with the underlying schema error).
What would change the addition of a schemaName to the YAML file wrt user reporting?

So I created a file that reads:
schemaVersion: "1.0.0"
repositories:
  - id: test
    url: test

and read it with ChannelMapper.from(file) and it simply returns empty Channel object, no exception at all. Is that the worst case of all? :)
If nothing else, schemaName would give us a simple way to validate that the file is a channel file. Similarly, prospero would be able to validate the the file given to it is a provisioning config file and not a wildfly channel file.
Sure, we can make wildfly-channel library report "Invalid channel", but reporting something more concrete like "Invalid channel: schemaName is expexted to be 'wildfly-config', not 'prospero'" would I think give me better hint of what I did wrong.
And there is still the human aspect. I have new provisioning tool to use with EAP 8 which I know nothing about. I get two YAML files from support with long filenames that I need to use together. If I open the YAMLs, will I get a hint about what they are?

schemaVersion: "1.0.0"
repositories:
  - id: test
    url: test

this file is indeed valid because the Channel schema only requires the schemaVersion and will ignore unknown properties such as repositories.

If I open the YAMLs, will I get a hint about what they are?

That's why the Channel spec defines name and description fields to let a human understand the content of the file.
name: "Channel for org.jboss.eap:wildfly-ee-galleon-pack:pom:8.0.0.Beta-redhat-20220614\
  \ feature pack."

In my opinion the schema validation is very loose. It seems to defeat the purpose of having "schemaVersion" defined. It opens a possibility that an invalid input will pass through the system, producing invalid input (component versions not updated as expected), without anybody noticing.

an invalid input will pass through the system, producing invalid input (component versions not updated as expected), without anybody noticing.

could you give an example of such invalid input?

Mistakenly supplying wrong kind of YAML file, as I described above, that happens to have "schemaVersion" attribute.
Typo:

schemaVersion: "1.0.0"
streamsBLAH:
  - groupId: org.jboss.universe
    artifactId: community-universe
    version: 1.2.0.Final
  - ...

All right, I'm closing this ticket then. Thanks for reviewing.


OLS-447: Unit test for checking provider type

Description
Unit test for checking provider type
Type of change

[ ] Refactor
[ ] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library
[x] Unit tests

Related Tickets & Documents

Related Issue #OLS-447
Closes #

Checklist before requesting a review

[x] I have performed a self-review of my code.
[x] PR has passed all pre-merge test jobs.
[ ] If it is a core feature, I have added thorough tests.

Testing

Run make test-unit as usual

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.57%. Comparing base (e5a828b) to head (a40aba8).
Report is 13 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #834      +/-   ##
==========================================
- Coverage   94.77%   94.57%   -0.20%
==========================================
  Files          55       55
  Lines        2143     2009     -134
==========================================
- Hits         2031     1900     -131
+ Misses        112      109       -3

see 5 files with indirect coverage changes

/squash

/tide/squash

/merge-method-squash

/retest

/lgtm
/approve


cigien: Watch d4rkshop.com

cigien requests the watch of the watch_keyword d4rkshop\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

d4rkshop\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

d4rkshop\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Jeff Schaller in Charcoal HQ

Approved by Jeff Schaller in Charcoal HQ


Minute data end at 3:59 p.m.

When I query historical minute data, the closing time for the day is 3:59 p.m. Why isn't 4:00 p.m. included? I need the data to be from 9:30 a.m. - 4:00 p.m. inclusive.

Regular trading hours end at 3:59:59pm. No regular trading data takes place during the 4:00pm minute. The 3:59pm minute close is the close for the day.


Show info about operator under the cursor

Not in master, sorry. I want to encourage people to remember what the operators do with cmd g, but having this part of the UI is too much.

Fair enough


chapter-03/aws/aws_create_windows_ec_instance.yaml  - ami id expired

As of now line 76 => image: ami-0f5761c546ea1265a

Hey @bramdokman,
I greatly appreciate you taking the time to file the issue. I'll get it updated shortly.
Thanks!

I've updated the windows ami, thank you again. :)


NOT TO BE MERGED: An example of 1.4 failure when hitting unknown content

Test to demonstrate that a simple creation of a DDS will trip 1.4 client, as ID compressor is involved in such case (if it's enabled in container).
The purpose of having ID compressor in runtime is that it can be used by runtime (and runtime customers) if it's present. DDSs & data store creation code will leverage that for creation of short IDs, as it's known that usage of long IDs results in substantial increase of summary sizes (1.5-2.0x compared to using short IDs, mostly due to GC blob) and snapshots (I'd estimate - 5% for Loop payloads).
We can disable such functionality directly (i.e. have an extra knob that instructs that particular path not to use ID compressor even if it's present in runtime), but that sort of defeats the purpose of having ID compressor at runtime (and controlling its availability via its own knob), and it does not protect against some other feature of taking dependency of ID compressor.
There is also a risk that someone will use 2.0 FF packages and will start using SharedTree (with ID compressor) but will accidently try to open such document in 1.4. This may result in a failure like this test shows. It may also result a corruption of a document if ID compressor ops are not present in an op tail - 1.4 summarizer will happily drop ID compressor blob from the summary, breaking eventual consistency of a file.
Test results (saving you some clicks):
https://dev.azure.com/fluidframework/public/_build/results?buildId=259650&view=ms.vss-test-web.build-test-results-tab&runId=5582815&resultId=101480&paneView=debug

Closing as that was just a demo


Missing \end{groupplot} for subplot

I use matplotlib2tikz to generate a tikz file for a figure with several subplots, and I notice that when the number of subplots is smaller than row*col, the final tikz file won't compile because of missing \end{groupplot}. A minimal working python template looks like this.
import matplotlib.pyplot as plt
import numpy as np

x = np.linspace(0, np.pi)

fig = plt.figure()
fignum = 8  # the problem goes away if fignum is 9
for subid in range(fignum):
    sub = fig.add_subplot(3, 3, subid + 1)
    y = np.sin(x)
    sub.plot(x, y)

from matplotlib2tikz import save as tikz_save
tikz_save('test.tikz')

But if fignum is 9 in the code above, the resulting tikz is compilable. I can work around this by either manually adding \end{groupplot} to the tikz file, or changing the row and column number. I think a quick patch will work.

Fix released, thanks for the report.


[Bug] calendar.silent is not useful

Version
5.4.3
Link to Minimal Reproduction
https://echarts.apache.org/examples/en/editor.html?c=calendar-heatmap
Steps to Reproduce
visualMap{
type: "piecewise"
}
series: {
type: "heatmap",
}
calendar{
slider:true
}
Current Behavior
slider is not useful
dayLabel & monthLabel always responding to mouse events.
Expected Behavior
how to cancel the mouse hover effect
Environment
- OS:
- Browser:
- Framework:

Any additional comments?
No response

"slider" or silent ?
could also use tooltip: {show:false} to disable tooltips.


[update][plugins_path]  mac broken

Fri Aug 19 09:26:26 BST 2016
Info:
originally installed:  20160630

After running:
sh -c "$(curl -fsSL https://raw.githubusercontent.com/guarinogabriel/mac-cli/master/mac-cli/tools/update)

mac is now broken with every command producing the following:
% mac list
/usr/local/bin/mac: line 207: /usr/local/bin/mac-cli/misc/help: No such file or directory
/usr/local/bin/mac: line 213: /usr/local/bin/mac-cli/plugins/brew: No such file or directory
/usr/local/bin/mac: line 214: /usr/local/bin/mac-cli/plugins/general: No such file or directory
/usr/local/bin/mac: line 215: /usr/local/bin/mac-cli/plugins/git: No such file or directory
/usr/local/bin/mac: line 216: /usr/local/bin/mac-cli/plugins/image: No such file or directory
/usr/local/bin/mac: line 217: /usr/local/bin/mac-cli/plugins/brew: No such file or directory
/usr/local/bin/mac: line 218: /usr/local/bin/mac-cli/plugins/iterm: No such file or directory
/usr/local/bin/mac: line 219: /usr/local/bin/mac-cli/plugins/lamp: No such file or directory
/usr/local/bin/mac: line 220: /usr/local/bin/mac-cli/plugins/magento: No such file or directory
/usr/local/bin/mac: line 221: /usr/local/bin/mac-cli/plugins/network: No such file or directory
/usr/local/bin/mac: line 222: /usr/local/bin/mac-cli/plugins/performance: No such file or directory
/usr/local/bin/mac: line 223: /usr/local/bin/mac-cli/plugins/search: No such file or directory
/usr/local/bin/mac: line 224: /usr/local/bin/mac-cli/plugins/ssh: No such file or directory
/usr/local/bin/mac: line 225: /usr/local/bin/mac-cli/plugins/web: No such file or directory
/usr/local/bin/mac: line 226: /usr/local/bin/mac-cli/plugins/xcode: No such file or directory
/usr/local/bin/mac: line 227: /usr/local/bin/mac-cli/plugins/volume: No such file or directory

Running mac from within the git repo:
rprimus@mbp /usr/local/src/mac-cli
 % ./mac help
./mac: line 207: ./mac-cli/misc/help: Permission denied
./mac: line 215: ./mac-cli/plugins/git: Permission denied
./mac: line 216: ./mac-cli/plugins/image: Permission denied
./mac: line 218: ./mac-cli/plugins/iterm: Permission denied
./mac: line 219: ./mac-cli/plugins/lamp: Permission denied
./mac: line 220: ./mac-cli/plugins/magento: Permission denied
./mac: line 221: ./mac-cli/plugins/network: Permission denied
./mac: line 222: ./mac-cli/plugins/performance: Permission denied
./mac: line 223: ./mac-cli/plugins/search: Permission denied
./mac: line 224: ./mac-cli/plugins/ssh: Permission denied
./mac: line 225: ./mac-cli/plugins/web: Permission denied
./mac: line 226: ./mac-cli/plugins/xcode: Permission denied
./mac: line 227: ./mac-cli/plugins/volume: Permission denied

Each of the above files have perms 600.

Fri Aug 19 09:52:32 BST 2016
UPDATE:
Have (re)installed version 2.  All seems fine now.
Closing.


Renamed file added by gg causes status to crash

Steps to Reproduce
TODO: ran gg add on a file that detected rename
Expected Behavior
gg shows status as rename
Actual Behavior
gg: read status entry: invalid code ' ' 'R'

Version Info
gg built from source at 6a7382d3be3b54f4210b7a662f2e23250497ad35 on 2018-05-31T17:04:04Z with local modifications
go: go1.10.2 gc linux/amd64
git version 2.17.0

I don't have time right now to construct the repro case, but I suspect that this state might require using git add to reproduce.

Figured it out and added steps. The Git version is significant: 2.7.4 does not reproduce this issue, whereas recent Git versions (2.17 and 2.18) do. I don't know the exact version of Git that introduces this behavior.
In Git 2.7.4, running git status --porcelain at the end yields:
AM bar.txt
 D foo.txt

whereas Git 2.18.0 yields:
 R foo.txt -> bar.txt


feat: flagd provider implementation

This PR builds upon the work that @benjiro did with regards to the Flagd implementation. In this PR, I added the Readme, added the remaining xml doc comments and added Unit tests.

I'll be looking at this today/tomorrow. Thanks @bacherfl !

wow @bacherfl and @benjiro ! Nice work. I think this looks good for a alpha release. I think the only thing I would like to see addressed in this PR would be changes to just throw in error conditions instead of manually returning the default - the SDK does that for us. Your tests look good to me as well. Quick question - did you manually test this as well with a small sample app of some kind? We will soon write integration tests using this provider in the dotnet SDK, so it will get tested rigorously eventually... I'm just wondering.
One other thing I will do is make sure to set this as a 0.0.1 release (if I don't do that, it will default to 1.0.0, which I'm sure we don't want at the moment). I can do this with changes to the release manifest, so don't worry and let me handle that part once we are otherwise set to merge!
The other things to do are:

automatically generate the grpc assets
add an LRU cache
automatic config via environment variables
Unix socket support
TLS support

I will create separate issues for these, and anyone can do them. Really great work, again!

Hi @toddbaert thanks a lot for the review! I will address your comments as soon as possible and will keep you updated when I have pushed my changes! Regarding your question: I did do a quick manual test by running flagd locally and using the SDK with the Flagd provider registered (i.e. the example from the readme) - will do that again once I have made all adaptations to make sure everything works as expected :)

@bacherfl I made some small changes:

pulled in the submodule in the CI as you mentioned
added conditional compilation for non NETSTANDARD2_0 platforms (see commit, and here for more info)
ran dotnet format

I think we are done here :sunglasses: . I can't thank you enough for your help.

@bacherfl Amazing work! Everything seems to work great. I made very some small changes (and one suggestion above):

pulled in the submodule in the CI as you mentioned
added conditional compilation for non NETSTANDARD2_0 platforms (see commit, and here for more info)
ran dotnet format

I think we are done here 😎 . I can't thank you enough for your help.

Thanks @toddbaert for the review and your help! I just pushed the suggested changes with regards to the .csproj file, so the PR should be good to go :)


Not compatible with Laravel 10

Description

this package turns out to be for laravel 8

and in version 10 monolog/AbstractProcessingHandler has changed its implementation of write().

that's why it gives an error.

you should add in the description that it is compatible with version 8.

or add versions for Laravel 10

Screenshot is provided

Could you check again, please

NC - great job, thank you


Email addresses shouldn't be flagged as titles

Currently, likes like<EMAIL_ADDRESS>get flagged as Title elements. They should be detected as EmailAddress elements or similar. That could be done similarly to how we detect Address.

Closes by #957


complete Discount resource

Implement (Discount)[https://docs.shopify.com/api/discount] with integration tests

issue #25


SimMotor is not importable without dev dependencies

I can't do: from ophyd_async.sim.demo import SimMotor in a project that depends on ophyd-async.
File ~/projects/bluesky/bluesky-stomp/venv/lib64/python3.10/site-packages/ophyd_async/sim/demo/_pattern_detector/_pattern_generator.py:4
      1 from pathlib import Path
      2 from typing import AsyncGenerator, AsyncIterator, Dict, Optional
----> 4 import h5py
      5 import numpy as np
      6 from bluesky.protocols import DataKey, StreamAsset

ModuleNotFoundError: No module named 'h5py'

h5py is a dev dependency, so it should move into primary dependencies?

I'm erring on the side of a ophyd_async[sim] extra that includes h5py

Should we re-organize imports to allow for importing SimMotor without needing h5py? As I understand h5py is only necessary for the pattern generator sim device. So SimMotor maybe should be importable w/o needing this library?

at the moment the main branch does have the ophyd_async[sim] extra
importing SimMotor without the h5py would only make sense if we restrict the methods --> if a method uses the h5py libarary we'd throw an error
import functools

def require_library(lib_name: str):
    """
    A decorator to ensure a specific library is available in the scope.
    If the library is not present, raises an ImportError.
    Caches the result for subsequent checks.
    """
    library_cache = {}

    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            # Check the cache first
            if lib_name not in library_cache:
                try:
                    # Attempt to import the library
                    __import__(lib_name)
                    library_cache[lib_name] = True
                except ImportError as e:
                    # Cache the failure and raise an error
                    library_cache[lib_name] = False
                    raise ImportError(f"Required library '{lib_name}' is not available.") from e

            # If library exists, run the decorated function
            if library_cache[lib_name]:
                return func(*args, **kwargs)

        return wrapper

    return decorator

I think we should organise sim into various subpackages that can each have their own deps, e.g.
sim\
  motor\
  detector\
  testing\

I'm inclined to agree with @stan-dot, I don't think it is too costly to require the user to install h5py in order to make use of the sim package. Extra dependencies is a nice way to avoid requiring everything for every user but it's possible to go too far in the other direction too. H5py is one library and not too heavyweight, so I'd throw in my vote for requiring it if you want to use the sim package.

Decision: make h5py a sim dependency and keep with demo


[Bug]: Unexpected token 'export' when Jest test imports d3-array (3.2.4)

Version
29.7.0
Steps to reproduce
Create an npm project for testing and initialize.
mkdir testing
cd testing
npm init -y

Install dependencies:
npm install --save-exact d3-array
npm install --save-exact --save-dev jest ts-jest @types/jest @types/d3-array

Create a test file, example.test.ts with these contents:
import { quantile } from 'd3-array';

describe('quantile', () => {
  it('should be a function', () => {
    expect(typeof quantile).toBe('function');
  });
});

Create a jest.config.js with these contents:
module.exports = {
  transform: { '^.+\\.ts?$': 'ts-jest' },
};

Invoke jest:
npx jest

Expected behavior
Example test should succeed:
$ npx jest
 PASS  ./example.test.ts
  quantile
    ✓ should be a function (2 ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        0.36 s
Ran all test suites.

Actual behavior
$ npx jest
 FAIL  ./example.test.ts
  ● Test suite failed to run

    Jest encountered an unexpected token

    Jest failed to parse a file. This happens e.g. when your code or its dependencies use non-standard JavaScript syntax, or when Jest is not configured to support such syntax.

    Out of the box Jest supports Babel, which will be used to transform your files into valid JS based on your Babel configuration.

    By default "node_modules" folder is ignored by transformers.

    Here's what you can do:
     • If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/ecmascript-modules for how to enable it.
     • If you are trying to use TypeScript, see https://jestjs.io/docs/getting-started#using-typescript
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/configuration
    For information about custom transformations, see:
    https://jestjs.io/docs/code-transformation

    Details:

    /home/jimbo/testing/node_modules/d3-array/src/index.js:1
    ({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,jest){export {default as bisect, bisectRight, bisectLeft, bisectCenter} from "./bisect.js";
                                                                                ^^^^^^

    SyntaxError: Unexpected token 'export'

      1 |
    > 2 | import { quantile } from 'd3-array';
        | ^
      3 |
      4 | describe('quantile', () => {
      5 |     it('should be a function', () => {

      at Runtime.createScriptFromCode (node_modules/jest-runtime/build/index.js:1505:14)
      at Object.<anonymous> (example.test.ts:2:1)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        0.369 s
Ran all test suites.

Additional context
WORKAROUND: Add the following to your jest.config.js to map all d3-* modules to their respective JavaScript implementations under node_modules:
  moduleNameMapper: {
    '^d3-(.+)$': '<rootDir>/node_modules/d3-$1/dist/d3-$1.js',
  },

Environment
System:
    OS: Linux 6.5 Ubuntu 22.04.4 LTS 22.04.4 LTS (Jammy Jellyfish)
    CPU: (16) x64 AMD Ryzen 9 4900HS with Radeon Graphics
  Binaries:
    Node: 20.11.1 - ~/.nvm/versions/node/v20.11.1/bin/node
    npm: 10.2.4 - ~/.nvm/versions/node/v20.11.1/bin/npm
  npmPackages:
    jest: 29.7.0 => 29.7.0

Note: I had wanted to add my workaround to this bug: #12036, but it is locked for contributions.

can you set up a reproduction without ts-jest? That's not maintained in this repo


Platform linux on some android devices

Versions

Kivy: master (2.0.0rc1)

Description
Last week I had to rebuild a kivy app that was working in many android devices. I normally do it with buildozer docker image.
I was able to create the apk again and it works as expected ... in some devices! In other ones the app crashes during splash screen due to a PermissionError.
After debugging with adb logcat I noticed kivy is not recognizing the platform correctly as it return 'linux' instead of 'android'.
This causes that ANDROID_APP_PATH is not used and it tries to write on /data/ path.
I'm using kivy==master on buildozer.spec.
Logs
04-27 14:54:56.946  7788  7807 I python  : Android kivy bootstrap done. name is main
04-27 14:54:56.946  7788  7807 I python  : AND: Ran string
04-27 14:54:56.946  7788  7807 I python  : Run user program, change dir and execute entrypoint
04-27 14:54:57.303  7788  7807 I python  :  Traceback (most recent call last):
04-27 14:54:57.303  7788  7807 I python  :    File "/home/user/hostcwd/.buildozer/android/app/main.py", line 10, in <module>
04-27 14:54:57.304  7788  7807 I python  :    File "/home/user/hostcwd/.buildozer/android/app/app.py", line 4, in <module>
04-27 14:54:57.305  7788  7807 I python  :    File "/home/use
r/hostcwd/.buildozer/android/platform/build-armeabi-v7a/build/python-installs/myapp/kivy/__init__.py", line 351, in <module>
04-27 14:54:57.306  7788  7807 I python  :  PermissionError: [Errno 13] Permission denied: '/data/.kivy'
04-27 14:54:57.306  7788  7807 I python  : Python for android ended.

yep, likely related! I guess maybe it would be safer to replace "and" with "or" in this test, maybe expecting all of these paths to exists is a bit much, any of them should be a good enough indicator that we are on android.

My proposal is to add a env variable on p4a bootstrappers like we do in kivy-ios, it seems more clean to me:
https://github.com/kivy/kivy/blob/26228b202e821824afd8b2536234b26c23ae1b9d/kivy/utils.py#L434

Well, on p4a we set an environ variable to let apps know that we are in p4a P4A_BOOTSTRAP:
https://github.com/kivy/python-for-android/blob/develop/pythonforandroid/bootstraps/common/build/jni/application/src/start.c#L86

@opacam Never seen that! So basically we could already detect if We're on android without any change on p4a.
I don't think that anyone is actually using kivy on Android without the aid of p4a, and also if there's such situation, the developer could easely set P4A_BOOTSTRAP=1 in it's startup routine.

Well, on p4a we set an environ variable to let apps know that we are in p4a P4A_BOOTSTRAP:

IMHO the environment variable described by @misl6 can unify the criterion in many OS builds and is more explicit.

Using an environment variable seems like a correct solution, I think the Kivy PR that broke things was misguided. It doesn't really matter what the env var is, arguably there's a case for making available both a KIVY_BUILD env var that the user can always override if they want, or if that isn't set making kivy fall back to the P4A_BOOTSTRAP env var.

FYI It works as expected in all devices. Thanks for quick response!


Paginator breaks feed on peer view when its longer than limit.

Paginator and peer view need to be checked out.

Removed paginator from subfeeds.


Fix missing library name prefix for cuda

In #465 the native import, and library search path was updated, the native import had the lib prefix removed, and the search path code was updated to prepend lib to the name when not running on windows.
However, there was a place in the cuda path generation (Skip checks) that was not using the prefix. This PR Fixes that missing prefix.

Could you briefly explain what this is fixing/adding? Thanks.

Could you briefly explain what this is fixing/adding? Thanks.

In #465 the native import, and library search path was updated, the native import had the lib prefix removed, and the search path code was updated to prepend lib to the name when not running on windows.
However, there was a place in the cuda path generation (Skip checks) that was not using the prefix. This PR Fixes that missing prefix.

Correction: Outside of Skip checks. SkipChecks was correct.


entity sensor.pure_cool_link_particulates state 'init' cannot be processed as a number

Hi,
I often get the following warning in my log file. Does anybody know what I can do about it?
Error in 'Woonkamer: Dyson aan en weer uit' trigger: In 'numeric_state' condition: entity sensor.pure_cool_link_particulates state 'init' cannot be processed as a number
Any help is appreciated
Best regards

Anyone else has this issue?


Add metrics for TiKV

https://github.com/tshlabs/cadence seems to be a good choice.
Would you like to do that ? @overvenus

Of course, but I need some time to familiar with tshlabs/cadence and etsy/statsd.

Cool!

I hope you can use https://prometheus.io, which I believe will be the default metrics collecting tool in the near future. However, it does not have a rust binding.

Find https://github.com/prepor/prometheus-rust but I don't know whether it can work or not.
Btw, I browser the prometheus doc and think it is not hard to support in rust, but this depends our need, if we decide to use prometheus, I think we can bind in rust, otherwise, not to do.

Thanks @xiang90
The rust implementation of prometheus is on early stage. Let's re-consider it later.
We may still use cadence right now.

@ngaut Hi!
Here is my implemention plan and details:

Add a new sub mod under mod utils, named statsd
New statsd wraps cadence.
Similar to crate log, statsd has a single static client, and may only be initialized once.

Pros:

It is convenient to change the underlying statsd implemention.
Simple API and less code, eg:

use utils::statsd;
statsd::count("foo", 1);

Cons:

Statsd's prefix is limited.
unsafe code block.

any suggestion?

Please do not import any more "unsafe".
@BusyJay will help you to remove unsafe block.
Others look good to me.

No unsafe :confused:
OK, I will work around, or make a PLAN B.


Question regarding cypress in the Electron app

I am curious if you have run Cypress from inside the electron app (in the electron-app workspace)?   I'm having the hardest time getting that working.    but basically using npm run start I can launch a cypress test from the app.    if I package it up and install it on a mac the app fails to start with
Uncaught Exception: Error: Cannot find module './lib/cypress'
which is coming from cypress/index.mjs
been trying to see if unpacking the asar will help but can't seem to get anything unpacked.

I guess I could ping every 29 days to keep it from getting stale.

Hi @gpinkham (sorry for the late response), do you still need assistance with this ?
If not could please share your findings ?

No worries.    so I honestly haven't anything with the Cypress part in a bit.      my goal was to generate test data using the the electron app I wrote (which submits it to a backend system via API) and then I wanted to use Cypress to run automated UI tests against the backend from within the electron app.     Basically use the results of the API calls to feed into the Cypress tests.    But yeah I could never get the App to load if I include the Cypress tests in the production build.    Only works when running in dev mode (npm run start).      If its not something you have done we can probably close this.    I'd hate for you to put any time into it as I don't honestly know if I want to run the tests any more.    Been using it without that feature for months now.


how to solved

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 51695048-f8f8-c2c1-89ff-dc3614f9d3d8
Version Independent ID: 372b149c-9a75-7db5-f8fd-4820e7b05b35
Content: Plan for Hyper-V scalability in Windows Server
Content Source: WindowsServerDocs/virtualization/hyper-v/plan/plan-hyper-v-scalability-in-windows-server.md
Product: windows-server
Technology: hyper-v
GitHub Login: @BenjaminArmstrong
Microsoft Alias: benarm

Hey nurersafa007,
Thanks again for contacting Microsoft. We value your feedback.
We're unable to determine what the intent of your question is. If there are any errors or areas of improvement with said documentation, please let us know so we may better assist. We're now closing this thread.
Thank you and have a most pleasant day.
#please-close


Jest async testing is not showing test status until the end

🐛 Bug Report
If you have a single file containing a number of async tests, the test status will not be printed until all the tests are executed and finished. This makes the stdout look frozen.
Video example:
https://drive.google.com/file/d/1BY12Jk53ywMdQt4wv8uSwdPausB0DNhk/view?usp=sharing
To Reproduce
Either run:
https://repl.it/@RomualdQuantin/jest-async-test
Or use jest async examples:

Download this directory: https://github.com/facebook/jest/tree/master/examples/async
run npm install
Replace file content __mocks__/request.js with (add a 1 second delay):

'use strict';

const users = {
  4: {name: 'Mark'},
  5: {name: 'Paul'},
};

export default function request(url) {
  return new Promise((resolve, reject) => {
    const userID = parseInt(url.substr('/users/'.length), 10);
    setTimeout(() => {
      return users[userID]
        ? resolve(users[userID])
        : reject({
            error: 'User with ' + userID + ' not found.',
          })
    }, 1000);
  });
}

run npm test

Expected behavior
I should see the test status appear one by one, instead of seeing all tests statuses at the very end of the test suits. Depending how many are executed, it makes the stdout looks frozen / broken.
I should see:

First test status: ✓ works with resolves (1008 ms)
A second later the second test status: ✓ works with promises (1004 ms)
Another second later the third test status: ✓ works with async/await (1005 ms)
And so on.

Link to repl or repo (highly encouraged)
https://repl.it/@RomualdQuantin/jest-async-test
envinfo

  System:
    OS: macOS Mojave 10.14.6
    CPU: (8) x64 Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz
  Binaries:
    Node: 12.15.0 - /usr/local/bin/node
    Yarn: 1.22.4 - /usr/local/bin/yarn
    npm: 6.13.4 - /usr/local/bin/npm
  npmPackages:
    jest: * => 26.6.3

Unfortunately this no longer works when you run more than one test file as they would mix into each other. We could of course implement this only for the case where just one test file is being run, but that doesn't appear worth it for the couple of seconds max

Hi @jeysal, I'm not sure what you mean. I've added 2 more files to the repl and no test status is displayed before the end of the 3 files.
Note that I tried a lot of different syntax:

added the done callback with async / await
use promise syntax instead of async / await
added the done callback with promise syntax

Nothing seems to change he behavior.

That is correct, I'm not saying the behavior you are looking for is already implemented when running more than 1 test file. I'm saying the behavior you are looking for would not work well with more than 1 test file, because outputs of different test file would be mixed.

@jeysal What about only one single file? It still does not show up anything and as I saw, runInBand does not support this anyway when trying to sequentially execute multiple test files (which I had before).

Correct, this feature request has still not been implemented, not even for a single file