ng add @scully/init

🐞 Bug report
Description
ng add @scully/init causes error, Angular 12 on Windows 10
🔬 Minimal Reproduction

ng add @scully/init

💻Your Environment
Windows 10
Angular Version:

D:\Angular12\my-blog>ng --version

     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/

Angular CLI: 12.1.4
Node: 16.5.0 (Unsupported)
Package Manager: npm 7.20.2
OS: win32 x64

Angular: 12.1.4
... animations, cli, common, compiler, compiler-cli, core, forms
... platform-browser, platform-browser-dynamic, router

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1201.4
@angular-devkit/build-angular   12.1.4
@angular-devkit/core            12.1.4
@angular-devkit/schematics      12.1.4
@schematics/angular             12.1.4
rxjs                            6.6.7
typescript                      4.3.5

Warning: The current version of Node (16.5.0) is not supported by Angular.

Scully Version:
1.1.4

D:\Angular12\my-blog>type package.json
{
  "name": "my-blog",
  "version": "0.0.0",
  "scripts": {
    "ng": "ng",
    "start": "ng serve",
    "build": "ng build",
    "watch": "ng build --watch --configuration development",
    "test": "ng test"
  },
  "private": true,
  "dependencies": {
    "@angular/animations": "~12.1.0",
    "@angular/common": "~12.1.0",
    "@angular/compiler": "~12.1.0",
    "@angular/core": "~12.1.0",
    "@angular/forms": "~12.1.0",
    "@angular/platform-browser": "~12.1.0",
    "@angular/platform-browser-dynamic": "~12.1.0",
    "@angular/router": "~12.1.0",
    "bootstrap": "^5.0.2",
    "rxjs": "~6.6.0",
    "tslib": "^2.2.0",
    "zone.js": "~0.11.4"
  },
  "devDependencies": {
    "@angular-devkit/build-angular": "~12.1.4",
    "@angular/cli": "~12.1.4",
    "@angular/compiler-cli": "~12.1.0",
    "@types/jasmine": "~3.8.0",
    "@types/node": "^12.11.1",
    "jasmine-core": "~3.8.0",
    "karma": "~6.3.0",
    "karma-chrome-launcher": "~3.1.0",
    "karma-coverage": "~2.0.3",
    "karma-jasmine": "~4.0.0",
    "karma-jasmine-html-reporter": "~1.7.0",
    "typescript": "~4.3.2"
  }
}

🔥 Exception or Error

i Using package manager: npm
√ Found compatible package version<EMAIL_ADDRESS>√ Package information loaded.

The package<EMAIL_ADDRESS>will be installed and executed.
Would you like to proceed?  Invalid response body while trying to fetch https://registry.npmjs.org/yargs: 6232:error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac:c:\ws\deps\openssl\openssl\ssl\record\ssl3_record.c:677:
npm ERR!

npm ERR!      C:\Users\xxxx\AppData\Local\npm-cache\_logs\2021-07-30T08_54_56_586Z-debug.log
× Package install failed, see above.

Hi @jam-one could you verify if this problem is fixed with the develop version of scully?

This problem has not been fixed.  Please update to ini >=1.3.6 to avoid a prototype pollution issue
npm WARN deprecated @@.: this package has been renamed to ts-evaluator.       C:\Users\xxxx\AppData\Local\npm-cache_logs\2021-08-01T02_45_50_588Z-debug.log
× Package install failed, see above.
The last 20 lines of the log file:
$ tail -20 /cygdrive/c/Users/xxxx/AppData/Local/npm-cache/_logs/2021-08-01T02_45_50_588Z-debug.log
1550 warn cleanup     [Error: EPERM: operation not permitted, unlink<EMAIL_ADDRESS>{
1550 warn cleanup       errno: -4048,
1550 warn cleanup       code: 'EPERM',
1550 warn cleanup       syscall: 'unlink',
1550 warn cleanup       path<EMAIL_ADDRESS>1550 warn cleanup     }
1550 warn cleanup   ]
1550 warn cleanup ]
1551 timing reify:rollback:createSparse Completed in 901088ms
1552 timing reify:rollback:retireShallow Completed in 358ms
1553 timing command:install Completed in 1108705ms
1554 verbose stack Error: 13596:error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac:c:\ws\deps\openssl\openssl\ssl\record\ssl3_record.c:677:
1555 verbose cwd D:\Angular12\my-blog
1556 verbose Windows_NT 10.0.19043
1557 verbose argv "C:\Program Files\nodejs\node.exe" "C:\home\xxxx\npm\node_modules\npm\bin\npm-cli.js" "install"<EMAIL_ADDRESS>"--quiet"
1558 verbose node v16.5.0
1559 verbose npm  v7.20.2
1560 error code ERR_SSL_DECRYPTION_FAILED_OR_BAD_RECORD_MAC
1561 error 13596:error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac:c:\ws\deps\openssl\openssl\ssl\record\ssl3_record.c:677:
1562 verbose exit 1
Cheers,
Jam One
On Saturday, July 31, 2021, 6:25:01 PM GMT+7, Jeffrey Bosch<EMAIL_ADDRESS>wrote:

Hi @jam-one could you verify if this problem is fixed with the develop version of scully?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

Any solutions?
On Saturday, July 31, 2021, 6:25:01 PM GMT+7, Jeffrey Bosch<EMAIL_ADDRESS>wrote:

Hi @jam-one could you verify if this problem is fixed with the develop version of scully?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

@jam-one When I try this, it works for me:

Can you verify there are no spaces or other non-ASCII characters in your path name. NodeJS on windows doesn't handle that combination properly.

I run this using Windows 10 cmd as administrator.
D:\Angular12\my-blog>where npm
C:\home\xxxx\npm\npm
C:\home\xxxx\npm\npm.cmd
C:\Program Files\nodejs\npm
C:\Program Files\nodejs\npm.cmd
D:\Angular12\my-blog>npm
npm
Usage:
npm install        install all the dependencies in your project
npm install   add the  dependency to your project
npm test           run this project's tests
npm run       run the script named
npm  -h   quick help on
npm -l             display usage info for all commands
npm help     search for help on  (in a browser)
npm help npm       more involved overview (in a browser)
All commands:
access, adduser, audit, bin, bugs, cache, ci, completion,
    config, dedupe, deprecate, diff, dist-tag, docs, doctor,
    edit, exec, explain, explore, find-dupes, fund, get, help,
    hook, init, install, install-ci-test, install-test, link,
    ll, login, logout, ls, org, outdated, owner, pack, ping,
    pkg, prefix, profile, prune, publish, rebuild, repo,
    restart, root, run-script, search, set, set-script,
    shrinkwrap, star, stars, start, stop, team, test, token,
    uninstall, unpublish, unstar, update, version, view, whoami
Specify configs in the ini-formatted file:
    C:\Users\xxxx.npmrc
or on the command line via: npm  --key=value
More configuration info: npm help config
Configuration fields: npm help 7 config
@.*** C:\home\xxxx\npm\node_modules\npm
D:\Angular12\my-blog>where node
C:\Program Files\nodejs\node.exe
D:\Angular12\my-blog>node
Welcome to Node.js v16.5.0.
Type ".help" for more information.

.help
.break    Sometimes you get stuck, this gets you out
.clear    Alias for .break
.editor   Enter editor mode
.exit     Exit the REPL
.help     Print this help message
.load     Load JS from a file into the REPL session
.save     Save all evaluated commands in this REPL session to a file

Press Ctrl+C to abort current expression, Ctrl+D to exit the REPL

.exit
There are no spaces or non-ASCII characters in my path name.
Cheers,Jam One

On Wednesday, August 4, 2021, 11:11:07 AM GMT+7, Sander Elias<EMAIL_ADDRESS>wrote:

@jam-one When I try this, it works for me:
Can you verify there are no spaces or other non-ASCII characters in your path name. NodeJS on windows doesn't handle that combination properly.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

In my nodejs directory I had a program named node.exe, I copied it and renamed the copy nodejs.exe. Then when I re-ran the command, it worked.
D:\Angular12\my-blog>ng add @scullyio/init
i Using package manager: npm
√ Found compatible package version: @@.***
√ Package information loaded.
The package @@.*** will be installed and executed.
Would you like to proceed? Yes
√ Package successfully installed.
@./init" schema is using the keyword "id" which its support is deprecated. Use "$id" for schema ID.
    Install ng-lib
    ✅ Added dependency
UPDATE src/polyfills.ts (3013 bytes)
UPDATE package.json (1225 bytes)
√ Packages installed successfully.
@./init:scully" schema is using the keyword "id" which its support is deprecated. Use "$id" for schema ID.
    ✅ Update package.json
UPDATE package.json (1299 bytes)
D:\Angular12\my-blog>ng generate @scullyio/init:markdown
@./init:markdown" schema is using the keyword "id" which its support is deprecated. Use "$id" for schema ID.
@./init:markdown" schema is using the keyword "id" which its support is deprecated. Use "$id" for schema ID.
? What name do you want to use for the module? posts
? What slug do you want for the markdown file? id
? Where do you want to store your markdown files? mdfiles
? Under which route do you want your files to be requested? posts
    ✅ Update scully.my-blog.config.ts
CREATE src/app/posts/posts-routing.module.ts (409 bytes)
CREATE src/app/posts/posts.component.html (153 bytes)
CREATE src/app/posts/posts.component.spec.ts (621 bytes)
CREATE src/app/posts/posts.component.ts (494 bytes)
CREATE src/app/posts/posts.component.scss (131 bytes)
CREATE src/app/posts/posts.module.ts (387 bytes)
UPDATE scully.my-blog.config.ts (297 bytes)
UPDATE src/app/app-routing.module.ts (707 bytes)
@.***/init:post" schema is using the keyword "id" which its support is deprecated. Use "$id" for schema ID.
    ✅ Blog ./mdfiles/2021-08-04-posts.md file created
CREATE mdfiles/2021-08-04-posts.md (99 bytes)
So can you please check if that is the problem? Because I think by default when you install NodeJS on Windows it creates only a node.exe.
Thanks for the help.
Cheers,Jam One
On Wednesday, August 4, 2021, 11:11:07 AM GMT+7, Sander Elias<EMAIL_ADDRESS>wrote:

@jam-one When I try this, it works for me:
Can you verify there are no spaces or other non-ASCII characters in your path name. NodeJS on windows doesn't handle that combination properly.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

So, it's somehow related to your environment.
Closing this as there seems nothing left todo ;)

Or you could upgrade your script to check for for node.exe in addition to nodejs.exe, since the default installation of NodeJS in windows only installs node.exe. This way other users won't face the same issue I experienced.
Just a thought.
Cheers,Jam One
On Thursday, August 5, 2021, 8:47:03 PM GMT+7, Sander Elias<EMAIL_ADDRESS>wrote:

So, it's somehow related to your environment.
Closing this as there seems nothing left todo ;)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

We are nowhere calling nodejs  in our code base. The text even doesn't exist in any of our source files. I don't think we are even calling out node itself directly.
All the invocations are done through npm scripts.

How weird. That was the only change I made and it worked. Well if anyone else has this error, it's an easy fix then.
Cheers,Jam One.
On Friday, August 6, 2021, 9:59:23 AM GMT+7, Sander Elias<EMAIL_ADDRESS>wrote:

We are nowhere calling nodejs in our code base. The text even doesn't exist in any of our source files. I don't think we are even calling out node itself directly.
All the invocations are done through npm scripts.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.


CPU vs GPU Inference

Thanks for the project - love keijiro's work and seeing it extended!
I was hoping for confirmation on which in this set of models are running on the CPU vs the GPU and what the path would look like to achieve inference purely on the GPU if not already there. I can see a heavy load on the CPU and so am unable to hit 30fps in testing so am curious if it'll be achievable in the future (by myself or others).
Cheers

BlazePoseBarracuda is already running most of the processing (including inference and any necessary pre/post processing) on the GPU. In my environment, the bulk of the CPU load comes from dispatching the compute shaders inside the Barracuda package.

What are the specs of your computer?
Does it reproduce with other keijiro's Barracuda projects?

After more investigation I am thinking that the Deep Profiling actually had a far bigger performance impact than the Barracuda operations themselves and was skewing my understanding of the bottlenecks. With Deep profiling Off I'm seeing much more expected results from the CPU and GPU. I've tested with an RTX 6000 and a GTX 960M now.


Can I host windows service using Nancy without .net framework?

Can I host windows service using Nancy without .net framework? I want to use .net core.
Prerequisites

[ ] I have written a descriptive issue title
[ ] I have verified that I am running the latest version of Nancy
[ ] I have verified if the problem exist in both DEBUG and RELEASE mode
[ ] I have searched open and closed issues to ensure it has not already been reported

Description

Steps to Reproduce

System Configuration

Nancy version:
Nancy host

[ ] Nancy.Hosting.Aspnet
[ ] Nancy.Hosting.Self
[ ] Nancy.Owin ()
[ ] Other:

Other Nancy packages and versions:
Environment (Operating system, version and so on):
.NET Framework version:
Additional information:

Sure, why not 😄

How? I am not finding enough resources. Basically I am using .net core 2.1 and .net standard for all the libraries in my project. I want to install it as windows service and host pages at localhost for browsing. Basically trying to achieve below result using Nancy
https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/windows-service?view=aspnetcore-2.1

Have you tried going through the guide? Try after you've set it up as a Windows service, it's just a matter of hooking Nancy up in your Startup class. Look at the Kestrel sample in this repo to see how it's done.

I have already seen Kestrel sample but it can not be hosted as windows service. I do not see "Start" and "Stop" methods or service manager class to override my functionalities

The one you linked; https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/windows-service?view=aspnetcore-2.1

Thank you. so if I understand currently, once I create windows service using .net core then we need to hook up Nancy. I have added Nancy.Hosting.Self nuget package , but I am getting below warning
Package 'Nancy.Hosting.Self 1.4.1' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETCoreApp,Version=v2.1'. This package may not be fully compatible with your project.
I am new bee to development. Is there any sample which shows hooking up Nancy to .net core 2.1 windows service?

Is there any sample which shows hooking up Nancy to .net core 2.1 windows service?

No, but as I mentioned, you need to combine two resources:

The tutorial you mentioned above to set up a Windows service
The Kestrel sample in this repo.

The first will show you how to host ASP.NET Core in a Windows service, the second will show you how to hook up Nancy in your Startup class.
Nancy.Hosting.Self is .NET Framework only (not .NET Core). In addition to the core Nancy package, you need to install the following packages:
https://github.com/NancyFx/Nancy/blob/dbdbe9428caea06c17d34a507c9216fe35abadc4/samples/Nancy.Demo.Hosting.Kestrel/Nancy.Demo.Hosting.Kestrel.csproj#L16-L18
Adding Nancy to your pipeline is done like this:
https://github.com/NancyFx/Nancy/blob/dbdbe9428caea06c17d34a507c9216fe35abadc4/samples/Nancy.Demo.Hosting.Kestrel/Startup.cs#L26

Thank you. we can close this ticket. Appreciated. I have updated my previous question. Great. Thanks

Awesome! Feel free to keep asking if you encounter problems 😄

You should be able to use https://github.com/NancyFx/Nancy.Bootstrappers.Autofac

I have added prerelease version
Install-Package Nancy.Bootstrappers.Autofac -Version 2.0.0-clinteastwood

I have managed to build the application without build errors from below code but unable to get the instances. Basically how do I inject?
       protected override void ConfigureApplicationContainer(ILifetimeScope container)
       {

        _services.Configure<Settings>(_configuration.GetSection("MySettings"));
        var deviceId = _configuration["Settings:DeviceId"];

        var containerFactory = new ContainerFactory();
        var containerBuilder = new ContainerBuilder();

        container.Update(builder =>
        {
            builder.RegisterInstance(containerFactory).As<IContainerFactory>();
            containerBuilder.Register(c => new Engine(containerFactory));
            containerBuilder.Register(i => new HubClient(deviceId, containerFactory)).As<HubClient> ().SingleInstance();
        });
        base.ConfigureApplicationContainer(container);
    }`

But now how to I inject it in my modules?
     internal class CustomWebHostService : WebHostService
      {
         public CustomWebHostService(IWebHost host) : base(host)
        {
          _engine = host.Services
                        .GetRequiredService<IContainerFactory>();  ? How do I get engine instance!
        }
      }

This fails!

I am completely blocked, as even Kestrel sample is showing just "Loading..." in page. something seems to be blocking it.


i18n(es): astro-components update

added note on dynamic vs reactive values

What kind of changes does this PR include?

New or updated content
Translated content

Description

Update Astro Components (JSX-like Expressions section) with a note on dynamic vs reactive values

Hey @glopzel, thanks for the amazing contribution! Welcome to our Docs Face Pile! 💜
Also, thank you @fredoist 🙌
OBS: We like to welcome first-time contributors with a pun in our Discord (#docs channel) and we sure would love to have you there as well! https://astro.build/chat


[SPARK-21396][SQL] Fixes MatchError when UDTs are passed through Hive Thriftserver

Signed-off-by: Atallah Hezbor<EMAIL_ADDRESS>What changes were proposed in this pull request?
This PR proposes modifying the match statement that gets the columns of a row in HiveThriftServer. There was previously no case for UserDefinedType, so querying a table that contained them would throw a match error. The changes catch that case and return the string representation.
How was this patch tested?
While I would have liked to add a unit test, I couldn't easily incorporate UDTs into the HiveThriftServer2Suites pipeline. With some guidance I would be happy to push a commit with tests.
Instead I did a manual test by loading a DataFrame with Point UDT in a spark shell with a HiveThriftServer. Then in beeline, connecting to the server and querying that table.
Here is the result before the change
0: jdbc:hive2://localhost:10000> select * from chicago;
Error: scala.MatchError: org.apache.spark.sql.PointUDT@2d980dc3 (of class org.apache.spark.sql.PointUDT) (state=,code=0)

And after the change:
0: jdbc:hive2://localhost:10000> select * from chicago;
+---------------------------------------+--------------+------------------------+---------------------+--+
|                __fid__                | case_number  |          dtg           |        geom         |
+---------------------------------------+--------------+------------------------+---------------------+--+
| 109602f9-54f8-414b-8c6f-42b1a337643e  | 2            | 2016-01-01 19:00:00.0  | POINT (-77 38)      |
| 709602f9-fcff-4429-8027-55649b6fd7ed  | 1            | 2015-12-31 19:00:00.0  | POINT (-76.5 38.5)  |
| 009602f9-fcb5-45b1-a867-eb8ba10cab40  | 3            | 2016-01-02 19:00:00.0  | POINT (-78 39)      |
+---------------------------------------+--------------+------------------------+---------------------+--+

LGTM
@atallahhezbor Could you submit another PR to address the comment from @liufengdb ?
This fix is nice to have in Spark 2.3. Let merge this now.
Thanks! Merged to master/2.3


using KNN instead of perturbing to produce neighborhood?

Would it be useful to produce the neighborhood from the training set with KNN instead of generating neighbors by perturbing? It might be handy when there is a big training-set to choose neighbors from, and there are relationships between the features that might be broken when perturbing every feature by itself - thus creating an 'illegal' neighbor

Not from the training set, as that is what is used to train the model (and thus predictions on it are not a good measure of how the model predicts elsewhere). Maybe from a validation set, but it's hard to imagine a problem where you would have enough data set apart for validation for this to make sense.

Agree - from the validation set. Suppose I have a big validation set, do you think its better to use KNN or simply random sample from it? As the notion of locality is already "handled" by the weights of the explainer model...

come to think of it - why is it wrong to use the training data for generating the neighborhood?
The current method of perturbation uses the distributions of the training data for generating the neighborhood. You could have claimed the same claim: "predictions on it are not a good measure of how the model predicts elsewhere" on that approach, and demand that the user provide the validation data for estimating the mean and scale of the features...

In answer to the first question: with a big validation set, I think simply sampling from it is a good idea for tabular data. As you mention, I think locality is already covered by the weighting step. For text, data is so sparse that the validation set would need to be enormous.
About using the training data: the current method of perturbation assumes that features are independent and samples according to training set proportions (in the case of categorical), or according to a normal (in the case of continuous). We would expect the validation set to have similar statistics, but it is a different thing to actually use the instances used for training.
Think of a classifier that completely overfits by creating a table where each individual example in the training set is assigned to its class (i.e. exact matching of training instances), and I think the reason will become clear.

Ok, thanks a lot for these answers! I think that the main thing I'm trying to understand is this:
Say I have a sample which is an outlier. My black-box classifier uses an esoteric combination of features to classify it, discerning it from its neighbors.
Now the current explainer perturbs around the center of mass of the training set, and then fits a distance-weighted simple model.
Wouldn't it be better if I would perturb around the explained sample itself and then fit a distance-weighted simple model?
It's just that in a sparse world, I'm worried that the explainer would not understand that given a set of features (e.g. age>70, bilirubin < 1.5 and BMI > 3.0) it is essential to classify using a usually-non-important feature (e.g. hemoglobin level)
Did you try that approach? I guess it can be easy to change and perturb around the sample and check if its indeed giving better explanations, but I think that there's something that I'm missing here since you didn't mention that approach at all :(

Notice that perturbing around the explained sample is not the same as using the k-nearest neighbors. I did try this - sampling from a normal centered at the feature value for the current instance (in the case of continuous attributes).
The problem you outline with outliers is a real one, but sampling around the explained sample has its own problems - imagine the explained sample has a really extreme value (e.g. age > 98). We may not see any instances with age < 60 if we sample around that point. If the classifier has a fixed threshold on age at 55, we will think that age is not an important factor (since perturbing it does not change the prediction).
Honestly, I don't know which is better. I thought perturbing around the sample would be less robust with a fixed width (i.e. there's an extra parameter to chose, how 'far' away we're sampling).

Maybe you could sample both from the center of the training data AND sample around the the instance of interest? Then combine the two samples and proceed as usual with the locally weighted kernel

Nice idea @christophM ! but again I stumble upon the problem of evaluating which method is better... I read the LIME paper, but I really don't have a feel how can I assess which approach is better quantitatively, so right now I'm just trying different flavors of neighborhood selection and looking at the features they selected and deciding if they make sense to me...

It will depend on the data and features used, I guess.
In the end you would like to evaluate which explanation is better, but as far as I know there is no good metric for that.


ImageEditingManager.cropImage does not work correctly after upgrading from 0.15 to 0.17

My App takes pictures with https://github.com/lwansbrough/react-native-camera (and the option Camera.constants.CaptureTarget.disk).
In 0.17 when I'm trying to crop the image via ImageEditingManager.cropImage(url, cropData), it doesn't work correctly anymore. It will result in a black or sometimes some different color image. For me, it looks like, it shows only one pixel of it. I tried all different resizeModes and imageSizes to display it, but it looks always the same. Images from the assets library still work fine.
url =  '/var/mobile/Containers/Data/Application/AA2AEEF1-85A1-4268-9129-D65C671330D2/Documents/F8749E02-6818-4CDB-8C15-83C96132C945.jpg'
Anybody know what changed here?
Thanks.

Hi there! This issue is being closed because it has been inactive for a while.
But don't worry, it will live on with ProductPains! Check out its new home: https://productpains.com/post/react-native/imageeditingmanagercropimage-does-not-work-correctly-after-upgrading-from-015-to-017
ProductPains helps the community prioritize the most important issues thanks to its voting feature.
It is easy to use - just login with GitHub.
Also, if this issue is a bug, please consider sending a PR with a fix.
We're a small team and rely on the community for bug fixes of issues that don't affect fb apps.


yubioath-desktop: 5.0.5 -> 5.1.0

Description of changes
New version released with support for Yubikey Bio.
Tried to add version tests also, unfortunately seems it fails without display, even though it is supposed to just print the version.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/869


PR pipeline failing on Smoke Test: Interactive Window on Linux

PR pipelines have been consistently failing yesterday and today on the Smoke Linux-Py3.7 step with the following:

  3 passing (24m)
  2 failing
  1) Smoke Test: Interactive Window
       Run Cell in interactive window:
     Error: Timeout of 180000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/vsts/work/1/s/out/test/smoke/datascience.smoke.test.js)
      at listOnTimeout (internal/timers.js:531:17)
      at processTimers (internal/timers.js:475:7)

  2) Smoke Test: Interactive Window
       Run Cell in native editor:
     Error: Timeout of 180000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/vsts/work/1/s/out/test/smoke/datascience.smoke.test.js)
      at listOnTimeout (internal/timers.js:531:17)
      at processTimers (internal/timers.js:475:7)

End Exit Reporter for Mocha.

Oldest failing PR pipeline I could find for yesterday is: https://dev.azure.com/ms/vscode-python/_build/results?buildId=52503&view=logs&j=1aba65ad-c92c-52be-9563-c48c052b759e&t=2c376755-b100-5ef8-d7f3-9dfae4234e44

Closing as a duplicate of https://github.com/microsoft/vscode-python/issues/8883


Dynamic storage ID in React Native

Storage instance can be created using a custom MMKV storage ID.
export const storage = new MMKV({
  id: `user-${userId}-storage`,
  path: `${USER_DIRECTORY}/storage`,
  encryptionKey: 'hunter2'
})

However, it is not clear from the documentation where this dynamic customization variables come from (${userId} & ${USER_DIRECTORY}).
Are this compile-time variables? If so, is there a way to set them as a runtime variables?
Example: setting ${userId} after user is logged in some online service and userId is an ID of that service.

Thank you. I'll check it out.
I guess the complete example would be:

define store dynamically with useEffectthat waits for userId
access the store with useMMKV({ id: ${userId}.storage }) hook


Distinct ports for zmqpubrawblock, zmqpubrawtx

lnd now needs distinct ports for zmqpubrawblock and zmqpubrawtx so just use the defaults

Thanks for this PR, much appreciated! 👍


Could not load file or assembly 'Unity.Abstractions'

I had my unity composite package before without a problem but now without modifying my code i get this exception "Could not load file or assembly 'Unity.Abstractions', Version <IP_ADDRESS>, Culture=neutral, PublicKeyToken=6d32ff45e0ccc69f' ...".
I just can start my application when i change my AssemblyInfo.cs, for example changing AssemblyFileVersion or other property, but after closing and starting my app again, the exception throws again.

I would recommend linking to individual packages. Just link to Unity.Container and NuGet will do the rest.


fix: error parsing JSONSchema in json-schema-validation function

Some users, as outlined in #270, are getting an error parsing the x-sdk-operations JSON Schema in the json-schema-validation function.
Instead of parsing the JSON in the json-schema-validation, it may resolve the problem to parse the JSON file up front, and load it into STATIC_ASSETS as an object, which in hindsight is the more sensible approach.
This simple change passes in our testing environment. However, the original implementation also works in our testing environment. Without replicating the users' environments who are experiencing this issue, I'm not sure this resolves the problem. I am, however, certain that parsing the JSON Schema is the root of the problem.

It looks like both the users that encountered the problem in #270 are running on Windows.  I think we need to test your fix on Windows to see if it works there.

Closing this PR until we move forward with more robust support for Windows compatibility.

Any updates on this?


Reaname mock_api/mock_api.py -> fake_api_data/cloud_status_data.py

Be more preciese of what we have in files

Coverage remained the same at 78.431% when pulling 90104280d78a886ef3430901502224e8e21ca6ad on rename_mock_api into 415a77f613fad402f9f7e5ce9da7edfa3f4baec4 on master.


Add sample dialog to the web app

Creating an issue based on feedback I received over email...

I found your Code about retail chatbot: https://developer.ibm.com/code/patterns/create-cognitive-retail-chatbot/. That's cool but I think the demo (try the app) could be improved a bit. It's not clear what is possible to do or what the shop sells, I think a couple of tweaks in the Dialog would make the demo more effective.

This can be solved by simply adding a new div before we wrap the body tag in this file: https://github.com/IBM/watson-online-store/blob/master/python-flask-server/static/index.html#L62-L63 and add in a sample transcript.

Feature requests are now tracked internally.


Support for GPS Info IFD

I have basically added a support for GEO-tags in EXIF.

Sorry this took so long to merge. The real world got in the way :)
Many thanks for the PR.
Dave


Hovered line indicator

It's me again.
As showed here https://github.com/Akifyss/obsidian-border#active-line-highlight-and-hovered-line-indicator, I'd like to see an indicator in the gutter editor (at the left of each line).
All is ok in the configuration (the 2 settings are not disabled).
How can I fix that please?

Hi, sorry I don't quite understand what "gutter editor" refers to, can you provide a screenshot or more information please?

Sorry, it's the used term for IDE 😉
In the screenshot, I mean the blue vertical line :

Sorry, I am not a proficient speaker of English😂. So you can't see the blue vertical line on your side?

That's right : I can't see it.

That's weird, can you report your system and theme version?
You could probably try downloading and reinstalling obsidian (as I use :has() for this function, only a reinstalled and downloaded obsidian has this feature).

New info: it works on my linux computer. Not on my macbook.

This feature should not be OS related, my guess is that this feature may require Electron 21:
https://github.com/obsidianmd/obsidian-releases/releases/tag/v1.1.8-E21

Ok i will have a look (it’s okay on my iPhone too)

I have Obsidian 1.1.12 on my macos.

I'd like to confirm that you have re-downloaded and installed your obsidian from the official website? If you have been using obsidian since before 1.1.8, you may need to re-install obsidian from the official website to upgrade to Electron 21.

It works :tada: Thank you!
My installer was still on 1.0.0 version un fact.


Unable to evaluate a namespaced xpath

Hello,
I have an error when I try to evaluate a namespaced xpath:
$ cargo run
   Compiling quick-error v1.1.0
   Compiling typed-arena v1.2.0
   Compiling peresil v0.3.0
   Compiling sxd-document v0.2.0
   Compiling sxd-xpath v0.4.0 (https://github.com/shepmaster/sxd-xpath.git#16b17d77)
   Compiling rust v0.1.0 (file:///home/sanpi/test/rust)
    Finished debug [unoptimized + debuginfo] target(s) in 8.74 secs
     Running `target/debug/rust`
thread 'main' panicked at 'No namespace for prefix', /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libcore/option.rs:715
note: Run with `RUST_BACKTRACE=1` for a backtrace.

extern crate sxd_document;
extern crate sxd_xpath;

fn main() {
    let package = ::sxd_document::parser::parse(r#"
<d:multistatus xmlns:d="DAV:" xmlns:cs="http://calendarserver.org/ns/">
    <d:response>
        <d:href>/</d:href>
        <d:propstat>
            <d:prop>
                <d:current-user-principal>
                    <d:href>/principals/users/johndoe/</d:href>
                </d:current-user-principal>
            </d:prop>
            <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
    </d:response>
</d:multistatus>"#).unwrap();

    let document = package.as_document();

    ::sxd_xpath::evaluate_xpath(&document, "/d:multistatus")
        .unwrap();
}

That's correct; there's nothing that ties the prefixes in the XPath to the prefixes in the document. Prefixes are meant as a localized shorthand for a namespace. You wouldn't want your program to stop working if the input document decided to start using D as the prefix.
As the documentation states:

For more complex needs, XPath parsing and evaluation can be split apart. This allows the user to specify namespaces, variables, extra functions, and which node evaluation should begin with. You may also compile an XPath once and reuse it multiple times.

The documentation for Context further shows how to configure this.

You wouldn't want your program to stop working if the input document decided to start using D as the prefix.

Furthermore, what would you expect for the XPath //d:* to return with the XML:
<d:root xmlns:d="uri://outer">
  <d:child xmlns:d="uri://inner" />
</d:root>

Thank you, that works fine with an appropriate context.

Thank you for providing an easy-to-follow example!

extern crate sxd_document;
extern crate sxd_xpath;

fn main() {
    let package = ::sxd_document::parser::parse(r#"
<d:multistatus xmlns:d="DAV:" xmlns:cs="http://calendarserver.org/ns/">
    <d:response>
        <d:href>/</d:href>
        <d:propstat>
            <d:prop>
                <d:current-user-principal>
                    <d:href>/principals/users/johndoe/</d:href>
                </d:current-user-principal>
            </d:prop>
            <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
    </d:response>
</d:multistatus>
"#).unwrap();

    let document = package.as_document();
    let root = document.root().children()[0];

    let mut context = ::sxd_xpath::Context::new();
    context.set_namespace("d", "DAV:");

    let factory = ::sxd_xpath::Factory::new();

    let xpath = factory.build("/d:multistatus")
        .expect("Could not compile XPath")
        .expect("No XPath was compiled");

    xpath.evaluate(&context, root)
        .unwrap();
}

I did forget to mention that it's possible to write code that does the original request. You'd have to take the document, grab the root, then the topmost element, iterate over all the namespaces, then register those URIs and prefixes on the context. However, you still run into the risks of a mis-matched.
I've thought about adding some code like this for the little XPath utility that ships with the library, as it's very useful when you are exploring an XML file on the command line.


Arch board feed back

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-search-documents


Received Data Size Issue

Not able to parse received data request if incoming data size is more than 2 mb.
Error message :
input message size 2927303 exceeds maximum message size 1048576
if let idx = data.firstIndex(where: {$0 == 0x1e})
After commenting above line I am able to parse received data but I am not able to understand above code.

Please collect logs at the debug level and attach to this issue. 0x1e is a record separator. TBH, I am not sure how this code can work without it.
Also, what is your scenario? 2MB feels like a lot for a message.

Closing as requested information has not been provided.


Add documentation for the Text section of TypedSvg

I initially had trouble implementing a text node. Following the pattern of other functions like circle and rect, I thought that the text content for text_ needs to be defined as an Attribute. However, no such Attribute exists. It took me a long time to figure out that text actually resides in TypedSvg.Core and that the text_ is for creating a <text> element. To help clear up this confusion, can we provide a brief explanation and an example code in the Text section of TypedSvg?

Yes, it probably does need better explanation.
If you submit a PR with some improved docs it will get merged in.

@rupertlssmith Made a PR.

PR merged.


pkl-server missing on Maven Central

pkl-cli is published to Maven Central, but depends on pkl-server which is not.
It should be published as well.

You need to bump to version 0.25.1, which fixes this.
pkl-server is not a library; it gets bundled into the other libraries that we ship (e.g. pkl-cli-java, and pkl-tools)


App crashes while searching images

Summary:
Because of null query(MwQueryResult) or null mwQueryResponse, the app crashes when accessing member variables of these references
System logs:
STACK_TRACE=java.lang.NullPointerException
at fr.free.nrw.commons.mwapi.OkHttpJsonApiClient.lambda$searchImages$5$OkHttpJsonApiClient(OkHttpJsonApiClient.java:245)
at fr.free.nrw.commons.mwapi.OkHttpJsonApiClient$$Lambda$5.call(Unknown Source)
at io.reactivex.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:44)
at io.reactivex.Single.subscribe(Single.java:3394)
at io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
at io.reactivex.Scheduler$DisposeTask.run(Scheduler.java:579)
at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)
at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
at java.util.concurrent.FutureTask.run(FutureTask.java:137)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
at java.lang.Thread.run(Thread.java:856)

Device and Android version:
What make and model device (e.g., Samsung J7) did you encounter this on?
ANDROID_VERSION=4.1.2
APP_VERSION_NAME=2.10.2
BRAND=samsung
PHONE_MODEL=SPH-D710
CUSTOM_DATA=
Commons app version:
2.10.2
Would you like to work on the issue?
Yes

If one of the null values inside the response was causing the crash, #2708 fixes it.

Duplicate of #2711. Created accidentally, closing this one


automatic epochs number selection?

We propose a "pre-defined" number of epochs as in the benchmarking paper by the Theis lab, as part of one recipe for example

In case it's relevant, I've experienced that the training curve usually flattens out if I set number of epochs = 1e6 / number_of_cells. It's handy because for most datasets inference always takes 12 minutes, so it's very predictable.

Great Valentine :) we proposed another yet very similar thing. I think the slight difference is that we max it out at 400 epochs (maybe the coefficient is also slightly different).
I have an interest in setting up an easy way to reproduce the benchmarking for different variants of scVI so that we can modify the "public code" and make sure we get the same performance (at least for the latent space). We could for example compare the two epoch numbers (though it might not matter that much).


Wrong Spectrum Computation

https://github.com/jamiebullock/LibXtract/blob/41273dd7a56ebe7f1a232cfa0b870ebb67eefd7d/src/vector.c#L323-L373
I was trying to implement a higher level API in Julia and I've found these two lines that seem wrong to me. Shouldn't it compute the euclidean distance between imag and real? The following two lines seems copied from the previous clause:
https://github.com/jamiebullock/LibXtract/blob/41273dd7a56ebe7f1a232cfa0b870ebb67eefd7d/src/vector.c#L367-L368
If you confirm, I can do a pull request

I noted that in master branch there is the following line:
*marker = sqrt(XTRACT_SQ(real) + XTRACT_SQ(imag)) / (double)N;
So, I think develop branch is wrong. I cannot compile master branch, though...


LayoutTest 4.0.2 crashes with autolayout failures

💥
Induce an autolayout failure for a simple test case and we see that the test suite terminates instead of failing the individual test. At least we know what test is having an issue.

Sorry to be slow to respond to this. This certainly isn't preferable. I'm going to be out of the country and off internet for 3 weeks, so won't have time to take a look at this.
A PR would be welcome. Otherwise, I'll take a look sometime in a month or so.
As you say, seems noncritical since it is a failure, but would definitely like to fix this.

Seems like this was an issue with our auto layout swizzler conflicting. We've resolved the issue. Not a problem with the lib. Thanks for the reply.

Ah, amazing. Thanks for following up on this :D


Adds DMA implementation

This PR adds a DMA implementation with the following features:

[x] DMA1 and DMA2 support
[x] DMAMUX1 support
[x] 16 Streams (8 for DMA1, 8 for DMA2)
[x] 16 Mux-Channels (8 for DMA1, 8 for DMA2)
[x] 8 Request Generators
[x] SafeTransfer for memory safe DMA transfers (Not completed yet)
[ ] Implementations of SafeTransfer for all peripherals

TODO

[x] Extension trait
[ ] Documentation
[ ] Support for other MCUs (currently only STM32H743 is supported)
[ ] Enable clippy warnings for the whole crate again

Discussion

As the DMA is not an official embedded-hal item, we should consider making this an opt-in feature.

Note: Write access on my fork is enabled for collaborators. Collaboration is appreciated.

There are edited files completely unrelated to the DMA. These are edits of cargo fmt. I know this isn't great, but opening a PR just for formatting some files correctly isn't ideal either.
Should we keep these changes in this PR or revert them at the end and merge them separately?

Looks good, at some point I will have a look and try this out.
There's some documentation of the register fields happening in stm32-rs which may be useful https://github.com/stm32-rs/stm32-rs/pull/322
Is there an existing DMA HAL that this is based on, maybe in one of the other stm32 crates?

This is currently a standalone implementation, not supported by any HAL I'm aware of, nor it is based on implementations of other HALs of the stm32-rs group.
My aim is to fulfill these requirements to make it as compatible as possible with embedded-hal, when the DMA api gets included (or not ...).
@richardeoin

There is a problem with my current SafeTransfer-Implementation: It requires buffers with static lifetimes. But such buffers (when created safely) get allocated in .bss, which itself gets allocated in DTCM-RAM. The problem is that the DMA can't access DTCM.
(Actually everything gets allocated in DTCM-RAM currently, maybe we should allocate the stack in SRAM1)
A possible fix would be to allow non-static lifetimes for the buffers and implement Drop implementations for SafeTransfer, which disables the stream, and for the buffers, which panics if the stream is still running so the buffer won't get deallocated.
Do you have other suggestions? Let me hear.
Merry Christmas btw :)

How is this progressing? There's a huge amount of functionality here, and it's interesting how it compares to other implementations (eg. F7)
More examples and more documentation are definitely needed. A paragraph at the top of each file (//!) would be very helpful.
I see the change to the memory map, and you're right that the 'memory' side of a DMA transfer will be the AXISRAM (or SRAM1,2,3). But moving the stack out of DTCM will hurt performance for all applications. The better option is to add an attribute the to static mut buffers as required, like this:
#[link_section = ".axisram.buffers"]

I was quite busy learning for exams for the last month so sadly i didnt make any progress since then.
But good news is that i have now semester vacation until end of april so i have now time to make progress here again.
The lack of documentation is for sure the first thing i'll address to enable collaboration and better reviewing.
I've never heard of this macro before, but what are u suggesting looks awesome. I'll look into it soon.

So quick question, as you probably noticed I have several effective_CONFIG methods defined for dma::Stream additionally to CONFIG. The reason for this is because the hardware forces these configs to a specific value when the stream gets enabled because otherwise the config would be invalid (see RM 0433 Rev 7 Chapter 15.3.18, 15.5.5 for details).
I personally dislike this behaviour, although its well documented in the reference manual.
A workaround would be to catch these situations and panic, forcing the software to manually fixing the config. Or we could add a Config builder, which guarantees valid configs at type level, but could end up bit messy and not easy to use. Or we leave it like this.

I see, the hardware sets the register value so it is always a valid configuration. For returning the current state, then try to read it from the hardware where possible.
For setting a valid configuration, I'd try prototyping a builder for a Config type. I don't think it will be that complicated, and one of the benefits of the builder type is that it allows you to write the logic to return None (or panic!) based on the values already set.

The safe_transfer module is currently not really usable, and not intended to be used directly. There will be implementations of SafeTransfer for all peripherals, which take part of the peripheral configuration of the channel. I recently pushed an example into serial::dma for demonstration purposes. Note that this module is configured as a test-module to exclude it from compilation, as the implementation is based on an old version of SafeTransfer and has very likely many bugs.

Closing as this PR is getting really cluttered with commits. When most of the work is done, I'll reopen / open an new PR.


Publisher API details top menu UX

Change the back to listing wording to a more intuitive one
UX issue in back to the listing button, People will click on it to goto API overview page

This is fixed with 3.2.
The Back to Listing link is removed and the api name is linked to overview page. Going to api listing requires to click the big left had side icon.


[IDEA] Global cache for Trending Projects

Problem to solve
Trending projects requires multiple graph queries to load all payments. This means

More GRT costs in paying for Graph queries
Increased loading time

Proposal

Fetching Trending projects will save the information to a JSON file, and upload it to IPFS with a TTL value.
Any user that loads the Trending projects page will initially fetch that file.
If an expired TTL value is received, that user repeats step 1.

Note: changing the contents of the JSON cache will change the CID, meaning the file can't be accessed via a URI. We should be able to use this https://docs.pinata.cloud/pinata-node-sdk#pinlist to find the cache file, if it's saved using a unique metadata attribute.
Other challenges may be related to multiple users loading the expired TTL cache and attempting to upload a new version simultaneously.

Done!


Save share URL

This commit will save the share URL (if any) to @share_url on FileSharingApi, if the URL is not present on the response body it will be an empty String

@dachinat I added pry-byebug since I find it really useful when developing gems, if you don't like it then I can revert it.
Last commit is basically bundle exec rubocop --auto-correct

@Quintasan Thanks a lot


Does it work on Macos?

I am so sorry I am new to this. Does this work on Macos? I know that Python has a version for Macos but not sure about Nvidia compontents?

It works, You can try standalone executable for Mac  from -> https://github.com/Purfview/whisper-standalone-win , it should run on Mac OS X v10.15 x86-64 and above.
Apple has officially dropped support for Nvidia GPUs in their newer operating systems, starting with macOS Mojave (10.14).
To run it on GPU you would need cuBLAS 11.x and cuDNN 8.x libs, I've no idea where to get them.

sorry, have to write this again. How do you get it to work on Macos from the link you provided? There is no file in that link, it just provides the codes but how do I execute them? Do I need to install something first? The link refers to the faster whisper but in that link (https://github.com/guillaumekln/faster-whisper) phyton is required so I dont understand how I will do it without Phyton in mac.

You already asking there: https://github.com/Purfview/whisper-standalone-win/discussions/90

The link refers to the faster whisper but in that link (https://github.com/guillaumekln/faster-whisper) phyton is required so I dont understand how I will do it without Phyton in mac.

It's ready to run standalone executable and contains Python interpreter inside the executable, so you don't need to install anything.
It refers this [faster-whisper] repo because it has compiled code from this repo.

@dazzng: If you look for a ready to use application for macOS, you might want to take a look at https://github.com/kaixxx/noScribe
It uses faster-whisper under the hood. (Disclaimer: I'm the author.)


Fields UUID not working on creation table

Hello,
In my model definition when I have a field like type: DataTypes.UUID
the table creation with the sync() function crashes with :
    { SequelizeDatabaseError: SQLITE_ERROR: near "BINARY": syntax error
The SQL creation table under sequelize-mocking is :
CREATE TABLE IF NOT EXISTS brands (id CHAR(36) BINARY NOT NULL UNIQUE PRIMARY KEY,
The field id stays as MySQL schema creation (CHAR(36) BINARY) that doesn't seems supported on sqlite (UUID normally).
When I change the field as VARCHAR the table creation is ok.
In the Sequelize documentation : http://docs.sequelizejs.com/manual/tutorial/models-definition.html
Sequelize.UUID
// UUID datatype for PostgreSQL and SQLite
// CHAR(36) BINARY for MySQL
Any idea ?
Regards,

Hi
Have you got this error when you use basically Sequelize ?
If not, could you use Sequelize and Sqlite (instead Mysql) and check if you have the error too ?
If so, that means you have an issue on Sequelize itself which not deal uuid and Sqlite
Have you got some issues onto the Sequelize bug tracker ?
Regards

When I use Sequelize with sqlite in my project I don't have this error, the table creation with db.sync() is OK.
The sync function seems KO with the Sequelize mocked object :

SequelizeMocking.createAndLoadFixtureFile()
SequelizeMocking.create()
ERROR on : return mockedSequelize.sync()

I fixed it by forcing the dialect as sqlite during the tests :
const db = new Sequelize(sqlite://...`
sequelize-mocking seems not override the dialect somewhere.
Thanks !

@SuperToma Interesting
Could you send me a tiny project to illustrate the issue ? With one test without the dialect forcing and another test with the dialect forcing ?
I think the Sequelize evolution has changed some deep logic and so check too the dialect into the URI (and not only the dialect property)
Many thanks for your feedback
Regards

Hi,
The bug is still here :/
I've been digging a little bit and thats what I found:
The problem is the DataType of a field, it is not reset to sqlite during copy. Except that a DataType is linked to a dialect.
I still don't know why it's not reset, but I think it's because of
this
function in sequelize:
normalizeDataType(Type) {
    let type = typeof Type === 'function' ? new Type() : Type;
    const dialectTypes = this.dialect.DataTypes || {};

    if (dialectTypes[type.key]) {
      type = dialectTypes[type.key].extend(type);
    }

    if (type instanceof DataTypes.ARRAY && dialectTypes[type.type.key]) {
      type.type = dialectTypes[type.type.key].extend(type.type);
    }
    return type;
  }

I hope it helps !
I'm just starting on sequelize so I don't totally know how it works, maybe it's unrelated :smile: ...

Hi @Embraser01
That probalby means you should add an issue into the sequelize backlog
I could check this, but if it needs a patch, it means we should fix the sequelize api
Regards

I don't think it's sequelize fault. We just have to create a new DataType depending on SQLite.
I write up an easy fix:
    static copyModel(sequelizeInstance, model) {
        class TempModel extends Sequelize.Model {
        }

        let newModel = TempModel.init(
            _.merge({ }, model.attributes),
            _.merge({ }, model.options, { 'sequelize': sequelizeInstance, 'modelName': model.name })
        );

        // Let's recreate a new instance of the datatype
        for (const att of Object.values(newModel.attributes)) {
            att.type = new Sequelize.DataTypes[att.type.key]();
        }
        return newModel;
    }

    ///////// Or after the mock

   sequelizeInstance.modelManager.all.forEach(model => {
        for (const att of Object.values(model.attributes)) {
            att.type = new DataTypes[att.type.key]();
        }
    });
```

Regards

I will have a look on the next days
Many thanks for your feedback
Regards

Just tried @Embraser01 fix and it works/passes tests


Add ability to customize toolbox titles (#3488)

Related Issue
Backport fix for https://github.com/microsoft/AdaptiveCards/issues/3487 into release/1.2
Description
Cherry-pick https://github.com/microsoft/AdaptiveCards/commit/4ea40b042ed8115327529c0031ffb77895534d4d into release/1.2
How Verified
Verified manually in adaptivecards-designer-app
Microsoft Reviewers: Open in CodeFlow

Just a heads up that I will merge ignoring the failures here which are totally unrelated to my changes. Let me know if you have concerns about this.

Just a heads up that I will merge ignoring the failures here which are totally unrelated to my changes. Let me know if you have concerns about this.

That's fine the fixes that @nesalang / @almedina-ms / @paulcam206 have pulled in are still sitting inmy staging branch.

Just a heads up that I will merge ignoring the failures here which are totally unrelated to my changes. Let me know if you have concerns about this.

That's fine the fixes that @nesalang / @almedina-ms / @paulcam206 have pulled in are still sitting inmy staging branch.

And they are in.

Bypassing non NodeJS checks.


[Dev] merge_arrow_pr.py uses 16.1.0 as the default fix version

Describe the bug, including details regarding any error messages, version, and platform.
Can we use 17.0.0 as the default fix version?
Component(s)
Developer Tools

Did we change that? I saw 17.0.0 when merging a PR? (at least a few days ago, didn't yet merge anything yesterday or today)

I saw this today.
The 16.1.0 milestone may be created yesterday or today. It may be related.

Yes, the 16.1.0 milestone was created afterwards.

Issue resolved by pull request 41305
https://github.com/apache/arrow/pull/41305


'React/RCTEventEmitter.h' file not found

I tried both the automatic and the manual installations for iOS, but I always get an error 'React/RCTEventEmitter.h' file not found in RNCGeolocation.h when building the project. I've cleaned the build folder, removed and reinstalled my pods, but nothing I do seems to make any difference 😬 when I undo all my changes, the project builds fine.
I'm on<EMAIL_ADDRESS>and<EMAIL_ADDRESS>

I undid all my changes, followed the automatic installation steps again and this time it seems to build and run fine, I don't think I actually changed anything 🤷♂

@AdrianMrn  Hi, how you solved this issue ? Mine is still not working

I honestly couldn't tell you @kukuandroid. Like I said, I just retried the installation, and it ended up working 🤷


⚠ GitHub has degraded performance

In ca01e91, GitHub (https://www.githubstatus.com/api/v2/status.json) experienced degraded performance:

HTTP code: 200
Response time: 166 ms

Resolved: GitHub performance has improved in afbba64 after 47 minutes.


Swift project: pod installation issue:

How to use SIAlertView in swift?
There is one question mark after the SIAlertView. And can not use the SIAlertView in the swift file.

I'm using SIAlertView in my swift project in it works fine. Whether you use Cocoa Pod or copy the files manually, you have to import #import "SIAlertView.h" in your Bridging-Header.
Then you can use it the following:
var alertView: SIAlertView = SIAlertView(title: "Hello World", andMessage: "Works!")
Which question mark do you mean?


PHP with HTML problem

VSCode Version: latest
OS Version: win 10 latest

The problem:
Why when i use php and use html outside php codes there is not autocomplete for html??

VS Code can basically only handle one language per file. This is covered in https://github.com/Microsoft/vscode/issues/2915


[do not merge] more CI playing

PR #257 has a revised/enhanced tmux session, not sure if this is needed anymore


Lid close action description

The Lid close action description "Specifies the action that occurs when the user closes the lid of a portable computer." is duplicated in several item settings. Is this a typo? Or, meant to indicate that the item setting also applies to or effects the Lid close action?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5fd918fb-7d8a-cc92-f8d9-339c2db98cb1
Version Independent ID: 9cb416d9-1927-eb95-4e3f-ef9f9684b14b
Content: Create and apply power plans - Configuration Manager
Content Source: sccm/core/clients/manage/power/create-and-apply-power-plans.md
Product: configuration-manager
Technology: configmgr-client
GitHub Login: @aczechowski
Microsoft Alias: aaroncz

Thanks @JustMatthew for catching this. I double-checked with the engineering team, it's definitely a copy/paste error. That sentence only applies to the lid close action, it should be removed from all other settings that have it.
I'll be honest, it's unlikely that we'll get to fixing this issue anytime soon, as there are no planned investments at this time in this feature area. So I've marked this issue with one of the community tags for help. If you want to make the change yourself, you can find instructions in How to contribute.

Thanks to @jpcast88 for submitting a change for this issue, which is why it closed. The change should publish live tomorrow (Thursday, April 18) around 11am Pacific time.


Update manual instruction for postgres and neo4j (tested)

Hi Jonathan,
as discussed in teams I am sending this pull request specific to the first part of the updates of task 2. The second part containing the update for mongodb and GraphDB should follow shortly after this week.
Kind regards,
C

Can you add more details about the scripts for Postgres and Neo4j? What are the parameters that need to be used? Maybe add some example parameterization? In Postgres, does it work out of the box, or do we need to create a custom schema?

I added some more description for both postgres and neo4j regarding the running of the script. Simplified as much as possible. And yes both scripts work out of the box.


When calling cmd script, comma in string is not preserved

Steps to reproduce
Create a batch file mybatch.cmd containing:
@echo off
echo %1
echo %2

In PowerShell call it with
PS C:\Users\me> .\mybatch.cmd "a,b"
a
b
PS C:\Users\me> .\mybatch.cmd 'a,b'
a
b
PS C:\Users\me> .\mybatch.cmd --% "a,b"
"a,b"
ECHO is off.  # this is expected in cmd
PS C:\Users\me> .\mybatch.cmd --% 'a,b'
'a
b'            # this is expected in cmd

Expected behavior
Comma inside the string should be preserved, like when calling it in cmd:
mybatch.cmd "a,b"
"a,b"
ECHO is off.

Actual behavior
Comma is parsed as array.
Environment data

$PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.2.3
PSEdition                      Core
GitCommitId                    6.2.3
OS                             Microsoft Windows 10.0.18363
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

$PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.0.0-preview.6
PSEdition                      Core
GitCommitId                    7.0.0-preview.6
OS                             Microsoft Windows 10.0.18363
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

It isn't PowerShell that parses the argument as an array, it is cmd that interprets unquoted token a,b as two arguments, because not only spaces but also , and ; can serve as argument separators.
PowerShell performs re-quoting as needed behind the scenes, and in the absence of spaces inside arguments, it currently decides that enclosing the value in double quotes is not necessary, so that both .\mybatch.cmd "a,b" and .\mybatch.cmd 'a,b' end up getting called as .\mybatch.cmd a,b, at which point cmd considers a and b separate arguments.
This re-quoting logic does not special-case calling cmd.exe or batch files, and is currently generally broken - see #1995
The workaround is to use embedded double quoting:
.\mybatch.cmd '"a,b"'
.\mybatch.cmd "`"a,b`""

You can use the iep function defined here to automate this quoting  and to compensate for the other ways in which re-quoting is broken.
Note that the planned fix for #1995, detailed in this RFC, will break the workaround (as it will break all other current workarounds) and - unless cmd.exe / batch-file calls are special-cased - would offer no alternative (other than the limited --% approach) - see this comment.


[BUG] CUDNN_STATUS_MAPPING_ERROR with cudnnSetStream

Describe the bug
Facing a CUDNN_STATUS_MAPPING_ERROR with cudnnSetStream while running MLCommons Training benchmark:
https://github.com/mlcommons/training_results_v3.1/tree/main/NVIDIA/benchmarks/dlrm_dcnv2/implementations/hugectr
To Reproduce

Followed the steps in this README.md for data preprocessing followed by training execution: https://github.com/mlcommons/training_results_v3.1/blob/main/NVIDIA/benchmarks/dlrm_dcnv2/implementations/hugectr/README.md
We ran this both on a single node and an 8 node cluster with each server containing 8 GPUs. We should source the appropriate config

Expected behavior
DLRM Reference implementation should start training on the cluster
Environment (please complete the following information):

OS: Ubuntu 22.04
Graphic card: NVIDIA DGX A100
Nodes - 8 nodes with 8 GPUs each
CUDA version: 12.2
Built an image using this Dockerfile(https://github.com/mlcommons/training_results_v3.1/blob/main/NVIDIA/benchmarks/dlrm_dcnv2/implementations/hugectr/Dockerfile)

Additional context

Able to reliably reproduce this issue when executing this code both on a single node and 8 node cluster
Stack trace:
terminate called recursively
what():  Runtime error: CUDNN_STATUS_MAPPING_ERROR
cudnnSetStream(cudnn_handle_, current_stream) (set_stream @ /workspace/dlrm/hugectr/HugeCTR/include/gpu_resource.hpp:80)
terminate called recursively
terminate called recursively
terminate called after throwing an instance of 'HugeCTR::core23::RuntimeError'
[A100-06:286040] *** Process received signal ***

@minseokl Could you help investigate this issue with running NVIDIA's MLPerf Training workload?

hi @rgandikota what's the configuration you are using? Is it this one? If yes, could you turn off cuda graph and overlap to see if there is more specific error message? The cuda graph and overlap can be turned off here

hi @rgandikota what's the configuration you are using? Is it this one? If yes, could you turn off cuda graph and overlap to see if there is more specific error message? The cuda graph and overlap can be turned off here

Hi @shijieliu. We ran the same training after turning off both cuda graphs and overlap. The error is still the same. Please find the full stack trace below:
=====================================================ModelFit===================================================== [HCTR][06:48:08.684][INFO][RK0][main]: Use non-epoch mode with number of iterations: 512110 [HCTR][06:48:08.684][INFO][RK0][main]: Training batchsize: 8192, evaluation batchsize: 16384 [HCTR][06:48:08.684][INFO][RK0][main]: Evaluation interval: 25605, snapshot interval: 2000000 [HCTR][06:48:08.684][INFO][RK0][main]: Dense network trainable: True [HCTR][06:48:08.684][INFO][RK0][main]: Use mixed precision: False, scaler: 1.000000, use cuda graph: False [HCTR][06:48:08.684][INFO][RK0][main]: lr: 0.005000, warmup_steps: 0, end_lr: 0.000000 [HCTR][06:48:08.684][INFO][RK0][main]: decay_start: 0, decay_steps: 0, decay_power: 2.000000 [HCTR][06:48:08.684][INFO][RK0][main]: Training source file: /data/train_data.bin [HCTR][06:48:08.684][INFO][RK0][main]: Evaluation source file: /data_val/val_data.bin :::MLLOG {"namespace": "", "time_ms":<PHONE_NUMBER>684, "event_type": "INTERVAL_END", "key": "init_stop", "value": null, "metadata": {"file": "/workspace/dlrm/mlperf_logger/callbacks.py", "lineno": 50}} :::MLLOG {"namespace": "", "time_ms":<PHONE_NUMBER>685, "event_type": "INTERVAL_START", "key": "run_start", "value": null, "metadata": {"file": "/workspace/dlrm/mlperf_logger/callbacks.py", "lineno": 50}} :::MLLOG {"namespace": "", "time_ms":<PHONE_NUMBER>685, "event_type": "INTERVAL_START", "key": "epoch_start", "value": null, "metadata": {"file": "/workspace/dlrm/mlperf_logger/callbacks.py", "lineno": 51, "epoch_num": 0}} terminate called after throwing an instance of 'HugeCTR::core23::RuntimeError'   what():  Runtime error: CUDNN_STATUS_MAPPING_ERROR 	cudnnSetStream(cudnn_handle_, current_stream) (set_stream @ /workspace/dlrm/hugectr/HugeCTR/include/gpu_resource.hpp:80) [735c14efcde7:00878] *** Process received signal *** [735c14efcde7:00878] Signal: Aborted (6) [735c14efcde7:00878] Signal code:  (-6) [735c14efcde7:00878] [ 0] /usr/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7ffba5754520] [735c14efcde7:00878] [ 1] /usr/lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c)[0x7ffba57a8a7c] [735c14efcde7:00878] [ 2] /usr/lib/x86_64-linux-gnu/libc.so.6(raise+0x16)[0x7ffba5754476] [735c14efcde7:00878] [ 3] /usr/lib/x86_64-linux-gnu/libc.so.6(abort+0xd3)[0x7ffba573a7f3] [735c14efcde7:00878] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2b9e)[0x7ffb9c807b9e] [735c14efcde7:00878] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xae20c)[0x7ffb9c81320c] [735c14efcde7:00878] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xad1e9)[0x7ffb9c8121e9] [735c14efcde7:00878] [ 7] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(__gxx_personality_v0+0x99)[0x7ffb9c812959] [735c14efcde7:00878] [ 8] /usr/lib/x86_64-linux-gnu/libgcc_s.so.1(+0x16884)[0x7ffba29f6884] [735c14efcde7:00878] [ 9] /usr/lib/x86_64-linux-gnu/libgcc_s.so.1(_Unwind_RaiseException+0x311)[0x7ffba29f6f41] [735c14efcde7:00878] [10] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(__cxa_throw+0x3b)[0x7ffb9c8134cb] [735c14efcde7:00878] [11] /usr/local/hugectr/lib/libhuge_ctr_shared.so(_ZN7HugeCTR11GPUResource10set_streamERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEi+0x345)[0x7ffaeec42895] [735c14efcde7:00878] [12] /usr/local/hugectr/lib/libhuge_ctr_shared.so(_ZN7HugeCTR25StreamContextScheduleable3runESt10shared_ptrINS_11GPUResourceEEb+0x46d)[0x7ffaeec41f8d] [735c14efcde7:00878] [13] /usr/local/hugectr/lib/libhuge_ctr_shared.so(_ZN7HugeCTR8Pipeline3runEv+0x10b)[0x7ffaeec40beb] [735c14efcde7:00878] [14] /usr/local/hugectr/lib/libhuge_ctr_shared.so(+0xe0383c)[0x7ffaeece483c] [735c14efcde7:00878] [15] /usr/lib/x86_64-linux-gnu/libgomp.so.1(+0x1dc0e)[0x7ffb9c738c0e] [735c14efcde7:00878] [16] /usr/lib/x86_64-linux-gnu/libc.so.6(+0x94b43)[0x7ffba57a6b43] [735c14efcde7:00878] [17] /usr/lib/x86_64-linux-gnu/libc.so.6(+0x126a00)[0x7ffba5838a00] [735c14efcde7:00878] *** End of error message ***

It will be helpful if you could make the ucp version compatible.
What's your command line arguments passing to the [training script]s(https://github.com/mlcommons/training_results_v3.1/blob/main/NVIDIA/benchmarks/dlrm_dcnv2/implementations/hugectr/train.py)?

@shijieliu - Here are the details (sharing it for single node)
Test run 1:  We set the environment as configured here
root@5795011ad9d8:/workspace/dlrm# source config_DGXH100_1x8x6912.sh
root@5795011ad9d8:/workspace/dlrm# python train.py
docker run --shm-size=1g --ulimit memlock=-1 --cap-add=sys_nice  --security-opt seccomp=unconfined --runtime=nvidia --rm -it  -v /mnt/weka/mlperf/data/dlrm/dataset/criteo_binary/:/data -v /mnt/weka/mlperf/data/dlrm/dataset/criteo_binary/:/data_val -it dlrm-mlperf:1
Test 2: with default values
docker run --shm-size=1g --ulimit memlock=-1 --cap-add=sys_nice  --security-opt seccomp=unconfined --runtime=nvidia --rm -it  -v /dlrm/dataset/criteo_binary/:/data -v /dlrm/dataset/criteo_binary/:/data_val -it dlrm-mlperf:
Test run 2: with default values as defined in code. code location
root@5795011ad9d8:/workspace/dlrm# python train.py

I think the warning may be fine. But it would be helpful if you can make the ucp version compatiable.
Could you share what's the command line arguments you are passing to training scripts

I run with the default value and it works well. This is my log.txt.
I am suspecting the data preprocessing maybe wrong which leads to illegal memory access because of overflow in embedding input. Could you use the following code to check your dataset?
import os
import numpy as np
import struct
import random
from tqdm import tqdm

alpha = 1.1
output_filename = '/data/train_data.bin'

dataset_info = [
    (1, 3, 40000000),
    (1, 2, 39060),
    (1, 1, 17295),
    (1, 2, 7424),
    (1, 6, 20265),
    (1, 1, 3),
    (1, 1, 7122),
    (1, 1, 1543),
    (1, 1, 63),
    (1, 7, 40000000),
    (1, 3, 3067956),
    (1, 8, 405282),
    (1, 1, 10),
    (1, 6, 2209),
    (1, 9, 11938),
    (1, 5, 155),
    (1, 1, 4),
    (1, 1, 976),
    (1, 1, 14),
    (1, 12, 40000000),
    (1, 100, 40000000),
    (1, 27, 40000000),
    (1, 10, 590152),
    (1, 3, 12973),
    (1, 1, 108),
    (1, 1, 36),
]
offset = np.cumsum([0] + [v[1] for v in dataset_info])
num_dense_features = 13
num_label = 1

total_samples =<PHONE_NUMBER>
n = 1024

num_cate_features = sum([num_table * hotness for (num_table, hotness, _) in dataset_info])
max_vocabulary_size = sum([v for _, _, v in dataset_info])
item_num_per_sample = 1 + num_dense_features + num_cate_features
sample_format = r"1I" + str(num_dense_features) + "f" + str(num_cate_features) + "I"
sample_size_in_bytes = 1 * 4 + num_dense_features * 4 + num_cate_features * 4

min_vocabulary_size = np.asarray([np.iinfo(np.int64).max for _ in range(26)])
max_vocabulary_size = np.asarray([np.iinfo(np.int64).min for _ in range(26)])

assert os.path.getsize(output_filename) == total_samples * sample_size_in_bytes
with open(output_filename, "rb") as file:
    for i in tqdm(range(100000)):
        samples_bytes = file.read(sample_size_in_bytes * n)
        samples = struct.unpack(sample_format * n, samples_bytes)
        samples = np.asarray(samples).reshape(n, -1)
        cate_features = samples[:, num_dense_features + num_label:].astype(np.int64)
        min_cate_features = np.asarray([np.min(cate_features[:, start: end].reshape(-1)) for start, end in zip(offset, offset[1:])])
        min_vocabulary_size = np.minimum(min_cate_features, min_vocabulary_size)
        max_cate_features = np.asarray([np.max(cate_features[:, start: end].reshape(-1)) for start, end in zip(offset, offset[1:])])
        max_vocabulary_size = np.maximum(max_cate_features, max_vocabulary_size)

print('min_vocabulary_size', min_vocabulary_size)
print('max_vocabulary_size', max_vocabulary_size)

The script prints the range of input category and in my side the output is
min_vocabulary_size [0 1 2 1 0 0 2 2 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2]
max_vocabulary_size [39999999    39059    17176     7421    20264        2     7085     1534  62 39999996  3067955   405281        9     2208    11937      154        3      973       13 39999999 39999999 39999999   590151    12972      107       34]

You can share your output so I can help check.

@shijieliu The script for validating our data fails on a size assertion by the looks of it. Please find the output below:
python test_preprocessing.py Traceback (most recent call last):   File "test_preprocessing.py", line 55, in <module>     assert os.path.getsize(output_filename) == total_samples * sample_size_in_bytes AssertionError
Here is the process we used to preprocess our data using NVTabular to speed up the preprocessing:
https://github.com/pytorch/torchrec/tree/main/torchrec/datasets/scripts/nvt
Is the script you shared specific to the output from this method of preprocessing mentioned in this ReadMe using CPU only?
https://github.com/mlcommons/training_results_v3.1/tree/main/NVIDIA/benchmarks/dlrm_dcnv2/implementations/hugectr

Is the script you shared specific to the output from this method of preprocessing mentioned in this ReadMe using CPU only?

Yes. This ReadMe should match with  our training scripts. There is at least one difference I can tell beteen the one using NVTabular and our ReadMe is that the one using NVTabular lacks converting one-hot dataset to multi-hot dataset which is the step 1.5 in our ReadMe.

Thanks much @shijieliu. Is there anyway we can get download pre-processed data. That will be a great help if we can download the data.
This will reduce a lot of time and confusion instead repeating the steps which may go wrong.

@shijieliu  - Thanks for providing the snippet. It was helpful in verifying the dataset.
We have executed and tested the code you provided on the dataset. You were right in pointing out that the conversion from a one-hot dataset to a multi-hot dataset was missing in a previous step.
However, we are currently facing another issue on the A100 (multi-node) platform with HUGECTR, specifically regarding communication.
fyi - we were able to successfully run BERT.
Any input or information would be greatly appreciated and will assist us in moving forward.
A100-02:2358430:2358556 [1] [proxy.cc:1495](http://proxy.cc:1495/) NCCL WARN [Service thread] Could not receive type from localRank 7, res=3, closed=0
A100-02:2358430:2358556 [1] [proxy.cc:1519](http://proxy.cc:1519/) NCCL WARN [Proxy Service 9] Failed to execute operation Connect from rank 15, retcode 3
A100-02:2358430:2358556 [1] NCCL INFO socketProgressOpt: abort called
A100-02:2358430:2358556 [1] NCCL INFO misc/socket.cc:805 -> 3
A100-02:2358430:2358556 [1] [proxy.cc:1495](http://proxy.cc:1495/) NCCL WARN [Service thread] Could not receive type from localRank 6, res=3, closed=0
A100-02:2358430:2358556 [1] [proxy.cc:1519](http://proxy.cc:1519/) NCCL WARN [Proxy Service 9] Failed to execute operation Connect from rank 14, retcode 3
Traceback (most recent call last):
File "/workspace/dlrm/train.py", line 368, in
model = hugectr.Model(solver, reader, optimizer)
RuntimeError: Runtime error: internal error - please report this issue to the NCCL developers
ncclGroupEnd() (all2all_warmup @ /workspace/dlrm/hugectr/HugeCTR/src/resource_managers/resource_manager_core.cpp:63)
/workspace/dlrm# python validate_dataset.py
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100000/100000 [28:09<00:00, 59.19it/s]
min_vocabulary_size [2 1 2 1 0 0 2 2 2 2 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2]
max_vocabulary_size [39999999    39059    15213     7421    20264        2     6776     1348
62 39999999  3067955   405281        9     2208    11937      154
3      973       13 39999999 39999999 39999999   590151    12972
98       34]

Hi @rgandikota glad to see the dataset seems right!
And for your questions, it seems that the nccl init failed which happens before traning starts. Could you try set NCCL_DEBUG=INFO NCCL_DEBUG_SUBSYS=INIT and see if there is more sepcific information? Thanks!

Please find the log attached with suggested configuration. (Anything more than 2 nodes results with an error)
Example run of 4 nodes: slurm-938.log
Error Snippet:
A100-08:3103687:3103857 [2] ibvwrap.c:115 NCCL WARN Call to ibv_reg_mr_iova2 failed
A100-08:3103687:3103857 [2] NCCL INFO ib_plugin.c:634 -> 2
A100-08:3103687:3103857 [2] NCCL INFO transport/net.cc:680 -> 2
A100-08:3103687:3103857 [2] NCCL INFO proxy.cc:1306 -> 2
A100-08:3103687:3103857 [2] NCCL INFO proxy.cc:1377 -> 2
Note: It works with 2 nodes [slurm-935_2nodes.log]
slurm-935_2nodes.log

A100-02:56101:56272 [3] ibvwrap.c:115 NCCL WARN Call to ibv_reg_mr_iova2 failed
A100-02:56101:56272 [3] NCCL INFO ib_plugin.c:634 -> 2
A100-02:56101:56272 [3] NCCL INFO transport/net.cc:680 -> 2
A100-02:56101:56272 [3] NCCL INFO proxy.cc:1306 -> 2
A100-02:56101:56272 [3] NCCL INFO proxy.cc:1377 -> 2

A100-02:56101:56272 [3] proxy.cc:1519 NCCL WARN [Proxy Service 11] Failed to execute operation Connect from rank 11, retcode 2

From the 4node log, it seems there is some problem in the IB connection on 4 nodes, so the setup of IB failed on ibv_reg_mr_iova2. I would suggest using nccl_test to double check if nccl can work properly and if the problem still exists, IB configuration needs to be checked.

@shijieliu  - Thank you.
Just to clarify, It works if we choose any two servers from the list. However, we encounter issues when selecting more than two servers.

Please find the mpirun results
date ;mpirun -x LD_LIBRARY_PATH -np 4 -host A100-01:1,A100-02:1,A100-03:1,A100-04:1, ./build/all_reduce_perf -b 8 -e 1M -f 2 -g 8;date
Wed Dec 13 10:51:17 AM PST 2023
nThread 1 nGpus 8 minBytes 8 maxBytes 1048576 step: 2(factor) warmup iters: 5 iters: 20 agg iters: 1 validation: 1 graph: 0

Using devices
Rank  0 Group  0 Pid 653913 on    A100-01 device  0 [0x07] NVIDIA A100-SXM4-80GB
Rank  1 Group  0 Pid 653913 on    A100-01 device  1 [0x0a] NVIDIA A100-SXM4-80GB
Rank  2 Group  0 Pid 653913 on    A100-01 device  2 [0x44] NVIDIA A100-SXM4-80GB
Rank  3 Group  0 Pid 653913 on    A100-01 device  3 [0x4a] NVIDIA A100-SXM4-80GB
Rank  4 Group  0 Pid 653913 on    A100-01 device  4 [0x84] NVIDIA A100-SXM4-80GB
Rank  5 Group  0 Pid 653913 on    A100-01 device  5 [0x8a] NVIDIA A100-SXM4-80GB
Rank  6 Group  0 Pid 653913 on    A100-01 device  6 [0xc1] NVIDIA A100-SXM4-80GB
Rank  7 Group  0 Pid 653913 on    A100-01 device  7 [0xc4] NVIDIA A100-SXM4-80GB
Rank  8 Group  0 Pid 617574 on    A100-02 device  0 [0x07] NVIDIA A100-SXM4-80GB
Rank  9 Group  0 Pid 617574 on    A100-02 device  1 [0x0a] NVIDIA A100-SXM4-80GB
Rank 10 Group  0 Pid 617574 on    A100-02 device  2 [0x44] NVIDIA A100-SXM4-80GB
Rank 11 Group  0 Pid 617574 on    A100-02 device  3 [0x4a] NVIDIA A100-SXM4-80GB
Rank 12 Group  0 Pid 617574 on    A100-02 device  4 [0x84] NVIDIA A100-SXM4-80GB
Rank 13 Group  0 Pid 617574 on    A100-02 device  5 [0x8a] NVIDIA A100-SXM4-80GB
Rank 14 Group  0 Pid 617574 on    A100-02 device  6 [0xc1] NVIDIA A100-SXM4-80GB
Rank 15 Group  0 Pid 617574 on    A100-02 device  7 [0xc4] NVIDIA A100-SXM4-80GB
Rank 16 Group  0 Pid 616085 on    A100-03 device  0 [0x07] NVIDIA A100-SXM4-80GB
Rank 17 Group  0 Pid 616085 on    A100-03 device  1 [0x0a] NVIDIA A100-SXM4-80GB
Rank 18 Group  0 Pid 616085 on    A100-03 device  2 [0x44] NVIDIA A100-SXM4-80GB
Rank 19 Group  0 Pid 616085 on    A100-03 device  3 [0x4a] NVIDIA A100-SXM4-80GB
Rank 20 Group  0 Pid 616085 on    A100-03 device  4 [0x84] NVIDIA A100-SXM4-80GB
Rank 21 Group  0 Pid 616085 on    A100-03 device  5 [0x8a] NVIDIA A100-SXM4-80GB
Rank 22 Group  0 Pid 616085 on    A100-03 device  6 [0xc1] NVIDIA A100-SXM4-80GB
Rank 23 Group  0 Pid 616085 on    A100-03 device  7 [0xc4] NVIDIA A100-SXM4-80GB
Rank 24 Group  0 Pid 623912 on    A100-04 device  0 [0x07] NVIDIA A100-SXM4-80GB
Rank 25 Group  0 Pid 623912 on    A100-04 device  1 [0x0a] NVIDIA A100-SXM4-80GB
Rank 26 Group  0 Pid 623912 on    A100-04 device  2 [0x44] NVIDIA A100-SXM4-80GB
Rank 27 Group  0 Pid 623912 on    A100-04 device  3 [0x4a] NVIDIA A100-SXM4-80GB
Rank 28 Group  0 Pid 623912 on    A100-04 device  4 [0x84] NVIDIA A100-SXM4-80GB
Rank 29 Group  0 Pid 623912 on    A100-04 device  5 [0x8a] NVIDIA A100-SXM4-80GB
Rank 30 Group  0 Pid 623912 on    A100-04 device  6 [0xc1] NVIDIA A100-SXM4-80GB
Rank 31 Group  0 Pid 623912 on    A100-04 device  7 [0xc4] NVIDIA A100-SXM4-80GB

out-of-place                       in-place
size         count      type   redop    root     time   algbw   busbw #wrong     time   algbw   busbw #wrong
(B)    (elements)                               (us)  (GB/s)  (GB/s)            (us)  (GB/s)  (GB/s)
       8             2     float     sum      -1    78.48    0.00    0.00      0    79.05    0.00    0.00      0
      16             4     float     sum      -1    78.40    0.00    0.00      0    81.24    0.00    0.00      0
      32             8     float     sum      -1    78.94    0.00    0.00      0    78.69    0.00    0.00      0
      64            16     float     sum      -1    78.57    0.00    0.00      0    79.54    0.00    0.00      0
     128            32     float     sum      -1    79.67    0.00    0.00      0    80.48    0.00    0.00      0
     256            64     float     sum      -1    81.17    0.00    0.01      0    79.20    0.00    0.01      0
     512           128     float     sum      -1    80.27    0.01    0.01      0    80.82    0.01    0.01      0
    1024           256     float     sum      -1    81.33    0.01    0.02      0    79.44    0.01    0.02      0
    2048           512     float     sum      -1    81.22    0.03    0.05      0    81.94    0.02    0.05      0
    4096          1024     float     sum      -1    85.09    0.05    0.09      0    83.84    0.05    0.09      0
    8192          2048     float     sum      -1    516.5    0.02    0.03      0    435.5    0.02    0.04      0
   16384          4096     float     sum      -1   1497.9    0.01    0.02      0    945.0    0.02    0.03      0
   32768          8192     float     sum      -1   2623.6    0.01    0.02      0   1584.1    0.02    0.04      0
   65536         16384     float     sum      -1   1122.5    0.06    0.11      0   2217.6    0.03    0.06      0
  131072         32768     float     sum      -1   2788.9    0.05    0.09      0   1144.4    0.11    0.22      0
  262144         65536     float     sum      -1    589.4    0.44    0.86      0   2597.6    0.10    0.20      0
  524288        131072     float     sum      -1   2098.7    0.25    0.48      0   1125.8    0.47    0.90      0
 1048576        262144     float     sum      -1    922.8    1.14    2.20      0   1659.6    0.63    1.22      0

Out of bounds values : 0 OK
Avg bus bandwidth    : 0.192288

Wed Dec 13 10:52:08 AM PST 2023

Thanks @jndinesh
The perf about allreduce and all2all in NCCL is very bad compared with the desired perf using IB. This is a perf issue however it may imply functional issue as well. Like the nccl_test does not using IB. Could you try setting NCCL_DEBUG=INFO NCCL_DEBUG_SUBSYS=INIT with nccl_test and share the log? Thanks!

According to NCCL, could you also try to check remove the user limits on registering pinned memory? For example, if you are using docker, try add --shm-size=1g --ulimit memlock=-1 when starting container.

@shijieliu - Thank you for your quick response and suggestion. Your attention to detail is much appreciated.
Please find attached log results.txt, which contains the environment variables you suggested while executing mpirun.
mpirun was executed on a host machine (Bare metal server) and not on a container environment.
However, I did observe RDMA statistics when mpirun was running.
Thanks,
Dinesh

@jndinesh The reason for your bad nccl_test perf may be the wrong configuration about numa. Could you try running nccl_test with mpirun -np 64 --bind-to numa --host A100-01:8,...(notic the :8) and remove -g 8 in all_reduce_perf argument? Let's see if this can solve your perf issue.
And another thing I want to check is your card type. Could you share the output of lspci? Thanks!

Please find below the attached lspci.txt log and mpirun output.
Please note that instead of using 64, it was configured to use 8 in mpirun -np 8.
Configuring with 64 instead of 8 will result in an error. For your reference, the log with that configuration is also attached.
mpirun -x NCCL_IB_GID_INDEX=3 -x LD_LIBRARY_PATH  -np 8 --bind-to numa --host A100-01:8  ./build/all_reduce_perf -b 8 -e 128M -f 2
# nThread 1 nGpus 1 minBytes 8 maxBytes 134217728 step: 2(factor) warmup iters: 5 iters: 20 agg iters: 1 validation: 1 graph: 0
#
# Using devices
#  Rank  0 Group  0 Pid 932421 on    A100-01 device  0 [0x07] NVIDIA A100-SXM4-80GB
#  Rank  1 Group  0 Pid 932422 on    A100-01 device  1 [0x0a] NVIDIA A100-SXM4-80GB
#  Rank  2 Group  0 Pid 932423 on    A100-01 device  2 [0x44] NVIDIA A100-SXM4-80GB
#  Rank  3 Group  0 Pid 932424 on    A100-01 device  3 [0x4a] NVIDIA A100-SXM4-80GB
#  Rank  4 Group  0 Pid 932425 on    A100-01 device  4 [0x84] NVIDIA A100-SXM4-80GB
#  Rank  5 Group  0 Pid 932426 on    A100-01 device  5 [0x8a] NVIDIA A100-SXM4-80GB
#  Rank  6 Group  0 Pid 932427 on    A100-01 device  6 [0xc1] NVIDIA A100-SXM4-80GB
#  Rank  7 Group  0 Pid 932429 on    A100-01 device  7 [0xc4] NVIDIA A100-SXM4-80GB
#
#                                                              out-of-place                       in-place
#       size         count      type   redop    root     time   algbw   busbw #wrong     time   algbw   busbw #wrong
#        (B)    (elements)                               (us)  (GB/s)  (GB/s)            (us)  (GB/s)  (GB/s)
           8             2     float     sum      -1   4450.3    0.00    0.00      0    137.8    0.00    0.00      0
          16             4     float     sum      -1    137.8    0.00    0.00      0    137.7    0.00    0.00      0
          32             8     float     sum      -1    137.9    0.00    0.00      0    138.0    0.00    0.00      0
          64            16     float     sum      -1    137.9    0.00    0.00      0    137.8    0.00    0.00      0
         128            32     float     sum      -1    138.3    0.00    0.00      0    138.3    0.00    0.00      0
         256            64     float     sum      -1    139.4    0.00    0.00      0    139.6    0.00    0.00      0
         512           128     float     sum      -1    139.5    0.00    0.01      0    139.8    0.00    0.01      0
        1024           256     float     sum      -1    140.9    0.01    0.01      0    140.6    0.01    0.01      0
        2048           512     float     sum      -1    141.0    0.01    0.03      0    141.4    0.01    0.03      0
        4096          1024     float     sum      -1    143.2    0.03    0.05      0    143.1    0.03    0.05      0
        8192          2048     float     sum      -1    146.1    0.06    0.10      0    144.5    0.06    0.10      0
       16384          4096     float     sum      -1    145.6    0.11    0.20      0    144.9    0.11    0.20      0
       32768          8192     float     sum      -1    150.8    0.22    0.38      0    149.7    0.22    0.38      0
       65536         16384     float     sum      -1    151.2    0.43    0.76      0    148.7    0.44    0.77      0
      131072         32768     float     sum      -1    149.8    0.87    1.53      0    147.8    0.89    1.55      0
      262144         65536     float     sum      -1    149.7    1.75    3.06      0    148.9    1.76    3.08      0
      524288        131072     float     sum      -1    154.5    3.39    5.94      0    154.8    3.39    5.93      0
     1048576        262144     float     sum      -1    158.6    6.61   11.57      0   3987.5    0.26    0.46      0
     2097152        524288     float     sum      -1    186.6   11.24   19.67      0    195.0   10.75   18.82      0
     4194304       1048576     float     sum      -1    207.8   20.19   35.33      0    207.6   20.21   35.36      0
     8388608       2097152     float     sum      -1   7664.4    1.09    1.92      0    376.6   22.27   38.98      0
    16777216       4194304     float     sum      -1   1514.5   11.08   19.39      0    591.8   28.35   49.61      0
    33554432       8388608     float     sum      -1   1302.6   25.76   45.08      0   2197.5   15.27   26.72      0
    67108864      16777216     float     sum      -1   3326.5   20.17   35.30      0   6675.0   10.05   17.59      0
   134217728      33554432     float     sum      -1    10766   12.47   21.82      0    16400    8.18   14.32      0
# Out of bounds values : 0 OK
# Avg bus bandwidth    : 8.32216
#

mpirun -x NCCL_IB_GID_INDEX=3 -x LD_LIBRARY_PATH  -np 64 --bind-to numa --host A100-01:8  ./build/all_reduce_perf -b 8 -e 128M -f 2
--------------------------------------------------------------------------
There are not enough slots available in the system to satisfy the 64
slots that were requested by the application:

  ./build/all_reduce_perf

Either request fewer slots for your application, or make more slots
available for use.

A "slot" is the Open MPI term for an allocatable unit where we can
launch a process.  The number of slots available are defined by the
environment in which Open MPI processes are run:

  1. Hostfile, via "slots=N" clauses (N defaults to number of
     processor cores if not provided)
  2. The --host command line parameter, via a ":N" suffix on the
     hostname (N defaults to 1 if not provided)
  3. Resource manager (e.g., SLURM, PBS/Torque, LSF, etc.)
  4. If none of a hostfile, the --host command line parameter, or an
     RM is present, Open MPI defaults to the number of processor cores

In all the above cases, if you want Open MPI to default to the number
of hardware threads instead of the number of processor cores, use the
--use-hwthread-cpus option.

Alternatively, you can use the --oversubscribe option to ignore the
number of available slots when deciding the number of processes to
launch.

After checking with @jndinesh @rgandikota, the issue is sovled by

setting --propagate=STACK when launching srun.
reinstall the environment.


For quality report use a cost function which takes into account by how much the prediction is wrong

This is another addition to the stats output.
Instead of just counting how often the rating system predicted the outcome of a game wrong, this PR takes into account by how much the prediction was wrong. This has the advantage of penalizing ratings more, when they are far away of the real skill, and ratings less if they are closer to the real rating.

In addition, when calculating stats, assume RD=0, which is the win probability in the Bradley-Terry model.

Thanks for all the work here @flovo, sorry it took so long to get these merged.

Thanks for all the work here @flovo, sorry it took so long to get these merged.


Implemented MediaElement ObjectDisposedException fix

Description of Bug
FormsVideoView instance was never set to null after disposing, which would sometimes lead to accessing the disposed object and throwing ObjectDisposedException.
I have also rearranged method calls in SetVideoURI to avoid calling the base method after the await, during which the instance could get disposed because the overridden method is async void. The only downside of this approach I could think of is that the metadata (video size and duration) may update after the video has already started. Maybe we could revert this and override Dispose and set a private field isDisposed = true and then after the await check if isDisposed == false to call the base method?
Issues Fixed

Fixes #1769
Fixes #1571

Behavioral Changes

PR Checklist

[x] Has a linked Issue, and the Issue has been approved
[ ] Has tests (if omitted, state reason in description)
[ ] Has samples (if omitted, state reason in description)
[x] Rebased on top of main at time of PR
[x] Changes adhere to coding standard
[ ] Updated documentation

@anvuks before we merge this you need to assign the CLA, can you do that please?

@pictos I already did when I submitted this PR, but for some reason the bot says it's not signed. Does it usually take this long or do I have to do something else? I've also clicked on recheck link but nothing happens.

@anvuks sometimes it gets crazy hehe
I'll ask someone on the top check if everything is good.

I have closed and reopened this PR to try to fix pending cla issue by triggering the PR webhook without any luck and just noticed that microsoft-cla bot's comment has been overwritten by the net-foundation-cla bot with the same CLA Signed message. I don't know how it missed, but there is nothing I can do about it besides opening another PR.
If it's okay with you, can you merge this PR?

As far as the CLA goes: one is green so that should be all good! :)

So when will this be approved to be merged?

I have closed and reopened this PR to try to fix pending cla issue by triggering the PR webhook without any luck and just noticed that microsoft-cla bot's comment has been overwritten by the net-foundation-cla bot with the same CLA Signed message. I don't know how it missed, but there is nothing I can do about it besides opening another PR.
If it's okay with you, can you merge this PR?

I have closed and reopened this PR to try to fix pending cla issue by triggering the PR webhook without any luck and just noticed that microsoft-cla bot's comment has been overwritten by the net-foundation-cla bot with the same CLA Signed message. I don't know how it missed, but there is nothing I can do about it besides opening another PR.
If it's okay with you, can you merge this PR?

when will this be merged?

@brettnguyen It should be merged as soon as a maintainer approves my changes. If this fix is critical to your app, you are going to have to use a nightly release until the next stable release.
@pictos CLA issues have been resolved, so if you have the time can you merge this please? 😁

@anvuks I'll take a look this week asap

@pictos Thanks! Don't worry about it, I know the CommunityToolkit maintainers are doing it all in their free time and life sometimes gets hectic. I'll be contributing more for sure 😀