Error to new install<EMAIL_ADDRESS>with npm

🐞 Bug report
Command (mark with an x or write it if not found the right command)

[sudo] npm i -g @angular/cli

Is this a regression?

I can install the latest version (7.3.9)
Description
When try install the last version of angular-cli npm package the command output a error and the installation process exited with status 130.
I tried install with and without sudo privileges.
🔬 Minimal Reproduction

Run this command: npm i -g @angular/cli
🔥 Exception or Error
The npm output when run the install command:

[        ..........] \ extract:genfun: sill extract<EMAIL_ADDRESS>/usr/local/bin/ng -> /usr/local/lib/node_modules/@angular/cli/bin/ng

><EMAIL_ADDRESS>postinstall /usr/local/lib/node_modules/@angular/cli
> node ./bin/postinstall/script.js

? Would you like to share anonymous usage data with the Angular Team at Google under
Google’s Privacy Policy at https://policies.google.com/privacy? For more details and
npm ERR! code ELIFECYCLE
npm ERR! errno 130
npm ERR<EMAIL_ADDRESS>postinstall: `node ./bin/postinstall/script.js`
npm ERR! Exit status 130
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR!

In case it helps I tried to update Nodejs (v12.3.1) and NPM (v6.9.0) and the problem not solve.

Hey! I finded a fast solution for solve this problem. I one of my attempts to know what happens, I see that if create a new user in the same os environment, the problem nothing happens.
My solution is remove all user data related with npm and angular cli, like .angular-config.json user file, .npm folder, .cache folder and others (I don't remember all files that I removed cry)
That is possible it angular save any state or any others files in user folder? Or that it npm save any state or other think?
Thank's!

I am having the same exact issue. Tried deleting .angular-config.json, .npm and .cache folders, but it's still not working.
@amoncusir could you check your command history and see what else you deleted? Thanks!

 Would you like to share anonymous usage data with the Angular Team at Google under
Google’s Privacy Policy at https://policies.google.com/privacy? For more details and
how to change this setting, see http://angular.io/analytics. No
(node:7812) UnhandledPromiseRejectionWarning: Error: EACCES: permission denied, open '/root/.angular-config.json'
at Object.openSync (fs.js:443:3)
at Object.writeFileSync (fs.js:1194:35)
at createGlobalSettings (/usr/lib/node_modules/@angular/cli/utilities/config.js:65:10)
at Object.getWorkspaceRaw (/usr/lib/node_modules/@angular/cli/utilities/config.js:73:26)
at setAnalyticsConfig (/usr/lib/node_modules/@angular/cli/models/analytics.js:288:43)
at Object.promptGlobalAnalytics (/usr/lib/node_modules/@angular/cli/models/analytics.js:326:9)
at process._tickCallback (internal/process/next_tick.js:68:7)
(node:7812) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:7812) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
+<EMAIL_ADDRESS>added 227 packages from 175 contributors in 36.211s

When installed node and npm with snap I had to run the following command in order to install @angular/cli globally:
sudo npm install -g @angular/cli --scripts-prepend-node-path

same here , all build using ng-cli were broken,
(the build do a  local temporary install of node & ng-cli )
added the following in  the build scripts and the issue was solved
npm config set script-prepend-node-path true

Nothing mentioned worked for me, additionally, everything I have found says not to install as root, or using sudo.
I followed some official documentation from npm manual and it worked for me.
https://docs.npmjs.com/resolving-eacces-permissions-errors-when-installing-packages-globally

I just ran into the same issue. Using yarn appears to work just fine until they fix this:
yarn add global @angular/cli

Run npm install -g @angular/cli without sudo. It worked for me.

Not sure if this helps, none of above mentioned worked for me.
Still getting
? Would you like to share anonymous usage data with the Angular Team at Google under
Google’s Privacy Policy at https://policies.google.com/privacy? For more details and
npm ERR! code ELIFECYCLE
npm ERR! errno 130
npm ERR<EMAIL_ADDRESS>postinstall: `node ./bin/postinstall/script.js`
npm ERR! Exit status 130

And if I tried with
yarn add global @angular/cli
Following error occurs
2019/06/14 13:54:54 [~]$ yarn add global @angular/cli
yarn add v1.16.0
info No lockfile found.
warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
error /home/ubuntu/node_modules/@angular/cli: Command failed.
Exit code: 1
Command: node ./bin/postinstall/script.js
Arguments:
Directory: /home/ubuntu/node_modules/@angular/cli
Output:
/home/ubuntu/node_modules/rxjs/internal/util/hostReportError.js:4
    setTimeout(function () { throw err; });
                             ^

UnexpectedEndOfInputException [Error]: Unexpected end of file.
    at _readValue (/home/ubuntu/node_modules/@angular-devkit/core/src/json/parser.js:624:19)
    at parseJsonAst (/home/ubuntu/node_modules/@angular-devkit/core/src/json/parser.js:721:17)
    at Object.parseJson (/home/ubuntu/node_modules/@angular-devkit/core/src/json/parser.js:753:16)
    at MapSubscriber.project (/home/ubuntu/node_modules/@angular-devkit/core/src/experimental/workspace/workspace.js:259:175)
    at MapSubscriber._next (/home/ubuntu/node_modules/rxjs/internal/operators/map.js:49:35)
    at MapSubscriber.Subscriber.next (/home/ubuntu/node_modules/rxjs/internal/Subscriber.js:66:18)
    at MapSubscriber._next (/home/ubuntu/node_modules/rxjs/internal/operators/map.js:55:26)
    at MapSubscriber.Subscriber.next (/home/ubuntu/node_modules/rxjs/internal/Subscriber.js:66:18)
    at Observable._subscribe (/home/ubuntu/node_modules/@angular-devkit/core/node/host.js:191:21)
    at Observable._trySubscribe (/home/ubuntu/node_modules/rxjs/internal/Observable.js:44:25)
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

Had the same issue when attempt to install<EMAIL_ADDRESS> UnhandledPromiseRejectionWarning: Error: EACCES: permission denied, open '/Users/MYHOMEDIR/.angular-config.json'
The following steps in this order, worked for me:
OSX Mojave 10.14.5, Installed node 12.4.0, npm 6.9.0
followed by the following commands:
sudo chown -R $USER:$(id -gn $USER) /Users/MYHOMEDIR/.config
npm config set script-prepend-node-path true note, without sudo
sudo npm install -g<EMAIL_ADDRESS>

My problem was the following:
npm install -g @angular/cli
npm ERR! path /Users/acme/.npm/_cacache/index-v5/48/99
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall mkdir
npm ERR! Error: EACCES: permission denied, mkdir '/Users/acme/.npm/_cacache/index-v5/48/99'
As it turned out, user 'acme' was given no permissions to folder '48'. Assigning 'everyone' 'Read & Write' permissions to folder 'index-v5' fixed the problem for now.

I got the same problem but I solved mine by running CMD as adminsitrator.


Update Spring boot to 3.2.8

@geoand : Last PR before releasing 3.2.0

Thanks!

@jacobdotcosta : Could you please plan a 3.2.0 release from master. Thanks!

Hi! Yes, I'll take care of this.

Done. 3.2.0 is released.
FYI @graben

Well, commit history looks weird. Missing maven release plugin commits especially for tag!

Not sure what happened. The release process ended OK but the artifacts aren't on sonatype.

I would suggest to reset main to the last real commit. And start release once again.

I've release 3.2.0.Beta1. It's on Nexus. Could you take a look at it @graben ?

@jacobdotcosta: Successfully build from tag and tested artifact from Maven Central.

Will do the 3.2.0 release now. 🤞🏽

Release failed as there's an outage on oss.sonatype.org - https://status.maven.org/

@jacobdotcosta: OSSRH seems to be up again.

3.2.0 has been released
FYO @graben


Error loading the plugin

Hi all,
When loading this plugin installed via zkg on source built Zeek, I get the following error when deploying via zeekctl
Duplicate &default attribute is ambiguous
The full output displays this on multiple lines of the plugin
error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 23: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 41: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 53: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 88: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 104: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 115: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 169: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 543: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 550: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 556: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 562: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 570: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 581: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 607: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 619: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 762: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 776: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 792: Duplicate &default attribute is ambiguous error in /opt/zeek/share/zeek/site/packages/./zeek-plugin-bacnet/./consts.zeek, line 1894: Duplicate &default attribute is ambiguous
Any idea why this may be, or what I can do to get this to work properly?
EDIT - I'm actually seeing this on all of the plugins installed from the amzn repos

Fixed by #29 and am working on merging for the rest of the repo's today.


Compile Error on Angular 15.1.1

Hi  @rbalet ,
I've tested your updated version but the problem persisted:
Error: node_modules/ngx-mat-input-tel/lib/ngx-mat-input-tel.component.d.ts:91:97 - error TS2344: Type '{ preferredCountries: { alias: "preferredCountries"; required: false; }; enablePlaceholder: { alias: "enablePlaceholder"; required: false; }; inputPlaceholder: { alias: "inputPlaceholder"; required: false; }; ... 10 more ...; disabled: { ...; }; }' does not satisfy the constraint '{ [key: string]: string; }'.
  Property '"preferredCountries"' is incompatible with index signature.
    Type '{ alias: "preferredCountries"; required: false; }' is not assignable to type 'string'.

I believe is related to the Angular version I have: 15.1.1
That equivalent code from the ngx-mat-intl-tel-input  lib is different:
static ɵcmp: i0.ɵɵComponentDeclaration<NgxMatIntlTelInputComponent, "ngx-mat-intl-tel-input", never, { "preferredCountries": "preferredCountries"; "enablePlaceholder": "enablePlaceholder"; "inputPlaceholder": "inputPlaceholder"; "cssClass": "cssClass"; "name": "name"; "onlyCountries": "onlyCountries"; "errorStateMatcher": "errorStateMatcher"; "enableSearch": "enableSearch"; "searchPlaceholder": "searchPlaceholder"; "describedBy": "describedBy"; "format": "format"; "placeholder": "placeholder"; "required": "required"; "disabled": "disabled"; }, { "countryChanged": "countryChanged"; }, never, never, true>;
Don't know if there is something else that can be done to make it work with my version of angular.
Thanks!

@hobojoe I cannot reproduce your error.
Would it be possible for you to provide a stackblitz example of it ?

@hobojoe I'm closing this issue as not reproducible


How to change the username and password in hygiea

Hi,
i want to change my username and password in hygiea dashboard is thr any way???

Outside of modifying the database entry you cannot. #1031 will add more auth features.

@vetrivelan25 https://github.com/capitalone/Hygieia/issues/909


feat: enable using existing resource group

Description
The latest Account Infrastructure Base DA supports using existing resources groups, this update adds passing a new optional  input existing_resource_group_name from the stack to the corresponding inputs of Account Infrastructure Base DA.
If existing_resource_group_name is supplied, the resource_group_name parameter value is ignored.
Note that Account Infrastructure Base DA does not use prefix parameter when the existing resource group names are supplied, so the existing_resource_group_name value in the stack needs to be the actual full resource group name.
Release required?

[ ] No release
[X] Patch release (x.x.X)
[ ] Minor release (x.X.x)
[ ] Major release (X.x.x)

Release notes content
If existing_resource_group_name is supplied, this resource group will be used instead of provisioning a new one. The resource_group_name parameter value is ignored  in this case.
Note that Account Infrastructure Base DA does not use prefix parameter when the existing resource group names are supplied, so the existing_resource_group_name value in the stack needs to be the actual full resource group name.
Run the pipeline
If the CI pipeline doesn't run when you create the PR, the PR requires a user with GitHub collaborators access to run the pipeline.
Run the CI pipeline when the PR is ready for review and you expect tests to pass. Add a comment to the PR with the following text:
/run pipeline

Checklist for reviewers

[ ] If relevant, a test for the change is included or updated with this PR.
[ ] If relevant, documentation for the change is included or updated with this PR.

For mergers

Use a conventional commit message to set the release level. Follow the guidelines.
Include information that users need to know about the PR in the commit message. The commit message becomes part of the GitHub release notes.
Use the Squash and merge option.

/run pipeline

:tada: This issue has been resolved in version 0.1.0 :tada:
The release is available on:

GitHub release
v0.1.0

Your semantic-release bot :package::rocket:


Refresh the routing when realtime segment is committed

Description
Related to issue #6029
When a consuming realtime segment is committed, it will refresh the segment ZK metadata (the partition metadata can change when the stream partition does not match the table config). The broker needs to refresh the routing for this segment in order to pick up the change in the segment ZK metadata.
This PR utilizes the SegmentRefreshMessage Helix message to refresh the broker routing when consuming realtime segment is committed.
Also add more tests in SegmentPartitionLLCRealtimeClusterIntegrationTest to verify the behavior.
With this change, when the stream partition changes for a partitioned table (which should be very rare), the result won't be accurate when the consuming segment contains records from multiple partitions, but will auto-recover when the segment is committed and the correct partition metadata is persisted to the segment ZK metadata.

@Jackie-Jiang Thanks for addressing this.


OmniSharp gone, F12 in VSCode no longer works

Type: Bug
Launch VSCode in the same manner as every time in the last several years.
Notice that OmniSharp has been replace by Language Server.
Realise that your ASP.NET (Web Forms, CLR4, .NET 4.8) solutions are unlikely to work in this new LSP world and you'll need to revert back to VS again.
Realise that even the most basic 'Go To Definition' stuff doesn't even work, even in plain vanilla C# projects and no amount of restart language server (like restart omni) will make it work.
Experiment with 'Switch to use the Omnisharp server' option.
Look at clock and realise your deadline is looming and real work needs to happen.

VS Code version: Code 1.80.2 (2ccd690cbff1569e4a83d7c43d45101f817401dc, 2023-07-27T20:40:28.909Z)
OS version: Windows_NT x64 10.0.19045
Modes:
Extensions (12)

Extension
Author (truncated)
Version

Bookmarks
ale
13.4.1

path-intellisense
chr
2.8.4

todo-tree
Gru
0.0.226

csdevkit
ms-
0.3.21

csharp
ms-
2.0.320

vscode-dotnet-runtime
ms-
1.6.0

vscode-edge-devtools
ms-
2.1.3

remote-ssh
ms-
0.102.0

remote-ssh-edit
ms-
0.86.0

remote-explorer
ms-
0.4.1

debugger-for-edge
msj
1.0.15

svelte-vscode
sve
107.9.0

Use 1.2.6.x version

Same issue.  No C# functionality at all.  No references, no navigation, no intellisense, most syntax tokens remain unhighlighted, no errors indicated.  All the .net logs in the output tab are completely empty.  I can't find any sign that vscode is even recognizing the language beyond the "C#" in the current language status indication.
This is the case for every C# project I try.  The projects I work with are on github, but unfortunately require shared libraries from a game plus a few third party mods so it is not the best reproduction example.
Just in case, here's one of them
https://github.com/RoboPhred/autoccultist

I had the same issue.  It seems all of those features have moved into the "C# Dev Kit" extension.  I did not try the omnisharp server option, but installing the dev kit extension got everything working again.

/causedByExtension


Error in Homebridge Log - Warning Characteristic

I am seeing the following error in the Homebridge log.  Wyze devices do seem to be running well.
This plugin generated a warning from the characteristic 'On': characteristic value expected boolean and received undefined. See https://homebridge.io/w/JtMGR for more info.
I have tried created a child bridge, and it did not solve the problem.

Update: the Wyze devices are super slow, laggy, and sometimes unresponsive.


URL below "ABC Sneak Peak" not available

Link does not appear to have anything associated

Fixed. Thanks!


Should we make an API?

I know of a project that would like to take the data of the labs and map them. I think that having an API would be a nice way to account for the projects that are using the data.
I found an open source platform: deployd that can maybe make it easier.
What do you think?

This should be something for waaaay down the line. Closing for the moment


LMS does not turn on even my training model broke down because of out of memory

I train a 3D U-Net model with batch_size=1 and input size=256256256. I use ubuntu 18 and RTX 6000 with 24 GB GPU RAM.
lms_cb = LMS()
lms_cb.batch_size = 1
os.environ['TF_LMS_SIMULATOR_MEM_RATIO'] = '0.4'  # bigger than 0.8 may tricger error OOM
lms_cb.sync_mode=3
lms_cb.autotune_plot = True
history = net.fit(x, y, batch_size=batch_sz, validation_data=valid_data, callbacks=[lms_cb, checker, saver, csv_logger])

but it shown that LMS is not necessary.
I0101 19:37:24.588294<PHONE_NUMBER>07616 lms.py:1275] [LMS][0] [Simulator] Available memory for simulation: 8.79 GiB (memory ratio: 0.4)
I0101 19:37:24.588538<PHONE_NUMBER>07616 lms.py:1275] [LMS][0] [Simulator] Available CPU memory for simulation: 64.0 GiB
I0101 19:37:24.589878<PHONE_NUMBER>07616 lms.py:1275] [LMS][0] [Simulator] Batch size used for simulation: 1
I0101 19:37:25.994517<PHONE_NUMBER>07616 lms.py:1275] [LMS][0] [Simulator] Found a parameter set: swapout_threshold 1558, swapin_ahead 1, swapin_groupby 0, sync_mode 0
I0101 19:37:25.996134<PHONE_NUMBER>07616 lms.py:1275] [LMS][0] LMS is not necessary. Turned LMS off.
I0101 19:37:26.055914<PHONE_NUMBER>07616 lms.py:1275] [LMS][0] Editing model for LMS
I0101 19:37:26.644204<PHONE_NUMBER>07616 lms.py:1275] [LMS][0] The graph has 30991 vertices and 37521 edges.
I0101 19:37:26.644354<PHONE_NUMBER>07616 lms.py:1275] [LMS][0] The graph has 225.41 MiB of learning parameters.
I0101 19:37:26.644531<PHONE_NUMBER>07616 lms.py:1275] [LMS][0] The largest GPU operation is conv3d_13/kernel/Initializer/random_uniform/mul consuming 0.01 GiB
I0101 19:37:33.928616<PHONE_NUMBER>07616 lms.py:1275] [LMS][0] Original categorized topological sort has 1558 levels.
I0101 19:37:34.083726<PHONE_NUMBER>07616 lms.py:1275] [LMS][0] LMS is not necessary. Turned LMS off.```

After that OOM errors occured.

I posted this on #6 as well:
TFLMSv2 (in WMLCE and PowerAI 1.6.0) uses an auto-tuning simulator to attempt to find the optimized tuning values. This simulator depends on accurately calculating your model tensor sizes. It uses static graph analysis to do this and therefore cannot account for some overhead that some of the operations used by your model may trigger. In addition, we have seen some models where the tensor size calculations are done by the simulator do not work well and it then leads to cases like yours, where the auto-tuning determines that no LMS swapping nodes are required, when in fact they are.

In these cases you can sometimes force the auto-tuning to work by setting mem ratio lower as you did, when that doesn't yield results you need to follow the manual tuning instructions in the WML CE documentation and not rely on automatic tuning.

In terms of your code and output above, I see that it is saying your largest op is only 0.01 GB. That is very small and makes me think your model is also not working well with the auto tuner.
Your code snippet has an error, but it likely will not affect the outcome of the auto-tuning not working:
lms_cb = LMS()
lms_cb.sync_mode=3 ### This is not the correct way to set sync_mode. sync_mode is a parameter to the LMS constructor

I suggest you try following the manual tuning instructions in the WMLCE documentation.
You could start with:
lms_cb = LMS(swapout_threshold=1, swapin_ahead=1, swapin_groupby=0)
history = net.fit(x, y, batch_size=batch_sz, validation_data=valid_data, callbacks=[lms_cb, checker, saver, csv_logger])

The maximum values for those 3 keyword arguments to LMS with your model are 1558, from this output line Original categorized topological sort has 1558 levels.

I also use a 3D U-Net for a lot of TFLMS testing and the model is here: https://github.com/smatzek/3DUnetCNN/
A lot of paper publishes, performance results, and blogs use this model with TFLMS and are linked from here:
https://developer.ibm.com/linuxonpower/2019/06/11/tensorflow-large-model-support-resources/

Thank you. I will try your advice. By the way, I wonder if it is because of my model. My model for multi task segmentation is composed with 2 3D U-Nets with a shared encoder:
shared_encoder = ...
decoder_1 = ...
decoder_2 = ...
model_1 = Model(shared_encoder, decoder_1)
model_2 = Model(shared_encoder, decoder_2)
model_1.compile()
model_2.compile()
for i in range(100):
    model_1.fit(x_1,y_1, callback=[lms_callbacks])
    model_2.fit(x_2,y_2, callback=[lms_callbacks])

The two models is compiled and trained seperately but with a shared encoder.
I want to know if it is the special design of my whole model that cause some unexpected errors? Or the model design has no effect?

Another question is: if tflms is compatible with mix precision? (mix precision is another technique which can reduce the GPU memory by store feature maps with 16 bits instead of 32 bits. )

While I have not tested it, it should work. TFLMS will insert swapping nodes before in the graph in the python layer before the mixed precision graph optimizer which is written in native code has a chance to run. This will probably make auto-tuning over estimate the amount of swapping needed if precision is selectively changed to FP16. However, in general the swapping nodes added by TFLMS should still function despite mix precision doing additional changes afterward.

The poor functioning of auto-tuning on your model(s) maybe due to the way you are training, but I wouldn't want to say for sure that is the case.

A new implementation of TensorFlow Large Model Support has been released. This implementation does not require tuning and is easier to enable in your code. Please see the current master branch version of the README.md for more information.


TTNN where and sub backward - incomplete documentation

For all TTNN backward ops - where and rsub:
The documentation does not cover supported data types (BFLOAT16/BFLOAT8_B) neither the supported layouts (ROW_MAJOR/TILE).
In practice, it seems that it does support BFLOAT16 and TILE layout.
However, ROW_MAJOR seems not to be supported for any of these ops.
BFLOAT8_B seems to be sometimes supported, but with low PCC.
Supported ranks are also not mentioned, but execution was successful for 2,3,4.

Hi @npetrovic-tenstorrent
Issue has been addressed and merged to main. Can we close this issue ?

Hi @npetrovic-tenstorrent Issue has been addressed and merged to main. Can we close this issue ?

Thank you, you can close it now.

@VirdhatchaniKN Thank you! It is not described in the documentation. You can close the issue.

Issue fixed


inprogress

testing for the project

[x]  Bug fiexd
[x]  proper validation
[x]  pipeline is correct

All the ting are properly done


Compiling with LLVM 10

Hi Pierre, it's been a while, I hope you're well!
I'm currently trying to get faustgen~ to work on Arch, which has LLVM 10.0.0 now. The Faust version included as a submodule in faustgen is pretty old (Aug 2018, somewhere around Faust 2.8.4), which doesn't compile with LLVM 10. This begins already with the regex used to parse LLVM version numbers in the Faust cmake configuration, which is completely thrown off by 2-digit version numbers. ;-)
I was able to get that version to compile by messing quite a bit with the Faust source, the required patch is included below. I could package up faustgen~ for Arch that way, by just applying that patch when building the package. But I guess that the proper way forward is to just update faustgen~ so that it uses a more recent Faust version.
2.20.2 is the most recent stable release, but unfortunately it seems that even that still needs to be patched up with the following two commits from Dec 2019 (one of these two is the LLVM 10 support which was contributed by Jean-Michaël Celerier, yeah, the guy from Université de Bordeaux who's organizing LAC this year ;-):
https://github.com/grame-cncm/faust/commit/0bbc3e4b605d3cec5e8fa17162b342e761ae9a1a
https://github.com/grame-cncm/faust/commit/10a35c5341c110b7dcdd5310f46bb357605420d1
(I know that because that's how David Runge builds the Faust package in the Arch community repositories, you can find his latest PKGBUILD here: https://github.com/archlinux/svntogit-community/blob/packages/faust/trunk/PKGBUILD.)
Anyway, we can always wait for the next stable Faust release, which should hopefully compile OOTB with LLVM 10. I just wanted to give you a quick heads up so that you know that an update of the Faust submodule may be in order rather sooner than later. What do you think?

And just for the record, here's the patch for the faust submodule (at https://github.com/grame-cncm/faust/commit/341bd3c562b4dcb9271a82bc939313613bc59244) that I needed to apply to the current HEAD to make the faustgen~ source build with LLVM 10.0.0. (GH doesn't allow me to attach diffs, so I'm just inlining it here.)
diff --git a/build/CMakeLists.txt b/build/CMakeLists.txt
index e4d12e08d..be0590395 100644
--- a/build/CMakeLists.txt
+++ b/build/CMakeLists.txt
@@ -181,13 +181,17 @@ if (INCLUDE_LLVM )
 		endif()
 	endif()

-	string (REGEX REPLACE "([0-9])\\.([0-9]).*" "LLVM_\\1\\2" LLVM_VERSION ${LLVM_PACKAGE_VERSION})
+	string (REGEX REPLACE "([0-9]+)\\.([0-9]+).*" "LLVM_\\1\\2" LLVM_VERSION ${LLVM_PACKAGE_VERSION})

 	# the declarations below are redundant but necessary to cope with different cmake behaviors on different platforms
 	set (TMP ${SRCDIR}/generator/llvm)
 	set_property(SOURCE ${TMP}/llvm_dsp_aux.cpp ${TMP}/llvm_dynamic_dsp_aux.cpp APPEND_STRING PROPERTY COMPILE_FLAGS "-fno-rtti ${LLVM_DEFINITIONS}")
 	set_property(SOURCE ${TMP}/llvm_dsp_aux.cpp ${TMP}/llvm_dynamic_dsp_aux.cpp APPEND_STRING PROPERTY CMAKE_CXX_FLAGS "-fno-rtti ${LLVM_DEFINITIONS}")
 	set (FAUST_LIBS ${FAUST_LIBS} ${LLVM_LD_FLAGS} ${LLVM_LIBS})
+
+	if("${LLVM_PACKAGE_VERSION}" VERSION_GREATER_EQUAL 10)
+         set (CMAKE_CXX_STANDARD 14)
+	endif()
 endif()

 ####################################
diff --git a/compiler/generator/llvm/llvm_code_container.cpp b/compiler/generator/llvm/llvm_code_container.cpp
index ae88a24d9..60af94672 100644
--- a/compiler/generator/llvm/llvm_code_container.cpp
+++ b/compiler/generator/llvm/llvm_code_container.cpp
@@ -68,7 +68,7 @@ LLVMCodeContainer::LLVMCodeContainer(const string& name, int numInputs, int numO

     // Set "-fast-math"
     FastMathFlags FMF;
-#if defined(LLVM_60)
+#if defined(LLVM_60) || defined(LLVM_70) || defined(LLVM_80) || defined(LLVM_90) || defined(LLVM_100)
     FMF.setFast();  // has replaced the below function
 #else
     FMF.setUnsafeAlgebra();
@@ -98,7 +98,7 @@ LLVMCodeContainer::LLVMCodeContainer(const string& name, int numInputs, int numO

     // Set "-fast-math"
     FastMathFlags FMF;
-#if defined(LLVM_60)
+#if defined(LLVM_60) || defined(LLVM_70) || defined(LLVM_80) || defined(LLVM_90) || defined(LLVM_100)
     FMF.setFast();  // has replaced the below function
 #else
     FMF.setUnsafeAlgebra();
@@ -220,7 +220,7 @@ void LLVMCodeContainer::generateComputeBegin(const string& counter)
         Function::Create(llvm_compute_type, GlobalValue::ExternalLinkage, "compute" + fKlassName, fModule);
     llvm_compute->setCallingConv(CallingConv::C);

-#if !defined(LLVM_50) && !defined(LLVM_60)
+#if !defined(LLVM_50) && !defined(LLVM_60) && !defined(LLVM_70) && !defined(LLVM_80) && !defined(LLVM_90) && !defined(LLVM_100)
     llvm_compute->setDoesNotAlias(3U);
     llvm_compute->setDoesNotAlias(4U);
 #endif
diff --git a/compiler/generator/llvm/llvm_dsp_aux.cpp b/compiler/generator/llvm/llvm_dsp_aux.cpp
index 725a5d248..4a3e077e4 100644
--- a/compiler/generator/llvm/llvm_dsp_aux.cpp
+++ b/compiler/generator/llvm/llvm_dsp_aux.cpp
@@ -37,7 +37,7 @@
 #include <llvm/Support/FileSystem.h>
 #include <llvm/Support/TargetSelect.h>

-#if defined(LLVM_40) || defined(LLVM_50) || defined(LLVM_60)
+#if defined(LLVM_40) || defined(LLVM_50) || defined(LLVM_60) || defined(LLVM_70) || defined(LLVM_80) || defined(LLVM_90) || defined(LLVM_100)
 #include <llvm/Bitcode/BitcodeReader.h>
 #include <llvm/Bitcode/BitcodeWriter.h>
 #else
diff --git a/compiler/generator/llvm/llvm_dynamic_dsp_aux.cpp b/compiler/generator/llvm/llvm_dynamic_dsp_aux.cpp
index df0b590f6..c20e7b7db 100644
--- a/compiler/generator/llvm/llvm_dynamic_dsp_aux.cpp
+++ b/compiler/generator/llvm/llvm_dynamic_dsp_aux.cpp
@@ -73,7 +73,7 @@
 #include <llvm/Target/TargetLibraryInfo.h>
 #endif

-#if defined(LLVM_40) || defined(LLVM_50) || defined(LLVM_60)
+#if defined(LLVM_40) || defined(LLVM_50) || defined(LLVM_60) || defined(LLVM_70) || defined(LLVM_80) || defined(LLVM_90) || defined(LLVM_100)
 #include <llvm/Bitcode/BitcodeReader.h>
 #include <llvm/Bitcode/BitcodeWriter.h>
 #include <llvm/Transforms/IPO/AlwaysInliner.h>
@@ -82,6 +82,11 @@
 #include <llvm/Support/raw_ostream.h>
 #endif

+#if defined(LLVM_100)
+#include <llvm/InitializePasses.h>
+#include <llvm/Support/CodeGen.h>
+#endif
+
 using namespace llvm;
 using namespace std;

@@ -118,7 +123,7 @@ static bool isParam(int argc, const char* argv[], const string& param)
     return false;
 }

-#if defined(LLVM_40) || defined(LLVM_50) || defined(LLVM_60)
+#if defined(LLVM_40) || defined(LLVM_50) || defined(LLVM_60) || defined(LLVM_70) || defined(LLVM_80) || defined(LLVM_90) || defined(LLVM_100)
 static Module* ParseBitcodeFile(MEMORY_BUFFER Buffer, LLVMContext& Context, string* ErrMsg)
 {
     using namespace llvm;
@@ -162,7 +167,11 @@ void llvm_dynamic_dsp_factory_aux::write(std::ostream* out, bool binary, bool sm
     string             res;
     raw_string_ostream out_str(res);
     if (binary) {
+#if defined(LLVM_70) || defined(LLVM_80) || defined(LLVM_90) || defined(LLVM_100)
+        WriteBitcodeToFile(*fModule, out_str);
+#else
         WriteBitcodeToFile(fModule, out_str);
+#endif
     } else {
         out_str << *fModule;
     }
@@ -174,7 +183,11 @@ string llvm_dynamic_dsp_factory_aux::writeDSPFactoryToBitcode()
 {
     string             res;
     raw_string_ostream out(res);
+#if defined(LLVM_70) || defined(LLVM_80) || defined(LLVM_90) || defined(LLVM_100)
+    WriteBitcodeToFile(*fModule, out);
+#else
     WriteBitcodeToFile(fModule, out);
+#endif
     out.flush();
     return base64_encode(res);
 }
@@ -183,7 +196,11 @@ void llvm_dynamic_dsp_factory_aux::writeDSPFactoryToBitcodeFile(const string& bi
 {
     STREAM_ERROR   err;
     raw_fd_ostream out(bit_code_path.c_str(), err, sysfs_binary_flag);
+#if defined(LLVM_70) || defined(LLVM_80) || defined(LLVM_90) || defined(LLVM_100)
+    WriteBitcodeToFile(*fModule, out);
+#else
     WriteBitcodeToFile(fModule, out);
+#endif
 }

 // IR
@@ -234,7 +251,7 @@ static void AddOptimizationPasses(PassManagerBase& MPM, FUNCTION_PASS_MANAGER& F
         }
         Builder.Inliner = createFunctionInliningPass(Threshold);
     } else {
-#if defined(LLVM_40) || defined(LLVM_50) || defined(LLVM_60)
+#if defined(LLVM_40) || defined(LLVM_50) || defined(LLVM_60) || defined(LLVM_70) || defined(LLVM_80) || defined(LLVM_90) || defined(LLVM_100)
         Builder.Inliner = createAlwaysInlinerLegacyPass();
 #else
         Builder.Inliner = createAlwaysInlinerPass();
@@ -317,7 +334,7 @@ bool llvm_dynamic_dsp_factory_aux::initJIT(string& error_msg)

         builder.setOptLevel(CodeGenOpt::Aggressive);
         builder.setEngineKind(EngineKind::JIT);
-#if !defined(LLVM_60)
+#if !defined(LLVM_60) && !defined(LLVM_70) && !defined(LLVM_80) && !defined(LLVM_90) && !defined(LLVM_100)
         builder.setCodeModel(CodeModel::JITDefault);
 #endif

@@ -348,7 +365,7 @@ bool llvm_dynamic_dsp_factory_aux::initJIT(string& error_msg)

         // -fastmath is activated at IR level, and has to be setup at JIT level also

-#if !defined(LLVM_50) && !defined(LLVM_60)
+#if !defined(LLVM_50) && !defined(LLVM_60) && !defined(LLVM_70) && !defined(LLVM_80) && !defined(LLVM_90) && !defined(LLVM_100)
         targetOptions.LessPreciseFPMADOption = true;
 #endif
         targetOptions.AllowFPOpFusion       = FPOpFusion::Fast;
@@ -357,7 +374,7 @@ bool llvm_dynamic_dsp_factory_aux::initJIT(string& error_msg)
         targetOptions.NoNaNsFPMath          = true;
         targetOptions.GuaranteedTailCallOpt = true;

-#if defined(LLVM_40) || defined(LLVM_50) || defined(LLVM_60)
+#if defined(LLVM_40) || defined(LLVM_50) || defined(LLVM_60) || defined(LLVM_70) || defined(LLVM_80) || defined(LLVM_90) || defined(LLVM_100)
         targetOptions.NoTrappingFPMath = true;
         targetOptions.FPDenormalMode   = FPDenormal::IEEE;
 #endif
@@ -410,7 +427,7 @@ bool llvm_dynamic_dsp_factory_aux::initJIT(string& error_msg)
             }

             if ((debug_var != "") && (debug_var.find("FAUST_LLVM1") != string::npos)) {
-#if defined(LLVM_60)
+#if defined(LLVM_60) || defined(LLVM_70) || defined(LLVM_80) || defined(LLVM_90) || defined(LLVM_100)
             // TargetRegistry::printRegisteredTargetsForVersion(std::cout);
 #else
             TargetRegistry::printRegisteredTargetsForVersion();
@@ -427,7 +444,7 @@ bool llvm_dynamic_dsp_factory_aux::initJIT(string& error_msg)
             pm.add(createVerifierPass());

             if ((debug_var != "") && (debug_var.find("FAUST_LLVM4") != string::npos)) {
-#if defined(LLVM_38) || defined(LLVM_39) || defined(LLVM_40) || defined(LLVM_50) || defined(LLVM_60)
+#if defined(LLVM_38) || defined(LLVM_39) || defined(LLVM_40) || defined(LLVM_50) || defined(LLVM_60) || defined(LLVM_70) || defined(LLVM_80) || defined(LLVM_90) || defined(LLVM_100)
             // TODO
 #else
             tm->addPassesToEmitFile(pm, fouts(), TargetMachine::CGFT_AssemblyFile, true);
diff --git a/compiler/generator/llvm/llvm_instructions.hh b/compiler/generator/llvm/llvm_instructions.hh
index c1147950d..b9f230f53 100644
--- a/compiler/generator/llvm/llvm_instructions.hh
+++ b/compiler/generator/llvm/llvm_instructions.hh
@@ -58,7 +58,7 @@
 #include <llvm/Support/raw_ostream.h>
 #include <llvm/Transforms/Scalar.h>

-#if defined(LLVM_40) || defined(LLVM_50) || defined(LLVM_60)
+#if defined(LLVM_40) || defined(LLVM_50) || defined(LLVM_60) || defined(LLVM_70) || defined(LLVM_80) || defined(LLVM_90) || defined(LLVM_100)
 #include <llvm/Bitcode/BitcodeReader.h>
 #include <llvm/Bitcode/BitcodeWriter.h>
 #else

In case it helps, we have a Faust 2.27.2 release with LLVM 10 support here https://github.com/grame-cncm/faust/releases.

Yes, @sletz I noticed this, too, but it has ha big sticker NOT TO BE PACKAGED all over it. ;-)

Well, we might use this for testing and then just switch to 2.27.2 proper when it comes out. But it seems that @pierreguillot is on vacation anyway, so let's just wait and see what his thoughts are.

Release 2.27.1 done: https://github.com/grame-cncm/faust/releases. Time for vacation.

So this has LLVM 10 support? There's nothing in the release notes about that.
In any case, @sletz, have a nice vacation! I'm on vacation as well. ;-)

Yes, and just fixed that.

@pierreguillot, for the time being I've uploaded an initial PKGBUILD to the Arch User Repositories (AUR) here: https://aur.archlinux.org/packages/pd-faustgen/. This is for the 0.1.1 version, so it includes the patch from above to make it compile with LLVM 10.
I've noticed that you already started working on a new branch, great! I will test that asap, and once it's finished, I'll update the AUR package.

Just tested the dev/v0.1.2 branch which seems to work pretty well already. Compilation goes through on Arch alright, but then it fails during linking with the following error message:
[ 98%] Linking C shared library "../external/faustgen~.pd_linux"
/usr/bin/cmake -E cmake_link_script CMakeFiles/faustgen_tilde_project.dir/link.txt --verbose=1
/usr/bin/cc -fPIC   -shared -Wl,-soname,faustgen~.pd_linux -o "../external/faustgen~.pd_linux" CMakeFiles/faustgen_tilde_project.dir/src/faust_tilde_io.c.o CMakeFiles/faustgen_tilde_project.dir/src/faust_tilde_options.c.o CMakeFiles/faustgen_tilde_project.dir/src/faust_tilde_ui.c.o CMakeFiles/faustgen_tilde_project.dir/src/faustgen_tilde.c.o  ../faust/build/lib/libfaust.a /usr/lib/libLLVMSupport.a /usr/lib/libLLVMCore.a /usr/lib/libLLVMIRReader.a -lLLVMjit /usr/lib/libLLVMX86CodeGen.a /usr/lib/libLLVMX86AsmParser.a /usr/lib/libLLVMX86Desc.a /usr/lib/libLLVMX86Disassembler.a /usr/lib/libLLVMX86Info.a /usr/lib/libLLVMX86Utils.a -lstdc++ -L/usr/lib -lLLVM-10 /usr/lib/libLLVMAsmParser.a /usr/lib/libLLVMAsmPrinter.a /usr/lib/libLLVMDebugInfoDWARF.a /usr/lib/libLLVMCFGuard.a /usr/lib/libLLVMGlobalISel.a /usr/lib/libLLVMSelectionDAG.a /usr/lib/libLLVMCodeGen.a /usr/lib/libLLVMBitWriter.a /usr/lib/libLLVMScalarOpts.a /usr/lib/libLLVMAggressiveInstCombine.a /usr/lib/libLLVMInstCombine.a /usr/lib/libLLVMTransformUtils.a /usr/lib/libLLVMTarget.a /usr/lib/libLLVMAnalysis.a /usr/lib/libLLVMProfileData.a /usr/lib/libLLVMObject.a /usr/lib/libLLVMBitReader.a /usr/lib/libLLVMCore.a /usr/lib/libLLVMRemarks.a /usr/lib/libLLVMBitstreamReader.a /usr/lib/libLLVMTextAPI.a /usr/lib/libLLVMMCParser.a /usr/lib/libLLVMMCDisassembler.a /usr/lib/libLLVMMC.a /usr/lib/libLLVMBinaryFormat.a /usr/lib/libLLVMDebugInfoCodeView.a /usr/lib/libLLVMDebugInfoMSF.a /usr/lib/libLLVMSupport.a -lz -lrt -ldl -ltinfo -lpthread -lm /usr/lib/libLLVMDemangle.a
/usr/bin/ld: cannot find -lLLVMjit
collect2: error: ld returned 1 exit status

Maybe an issue with the LLVM 10 cmake configuration?

Hello Albert and Stéphane,
I'm fine and I hope you're well too!
Indeed, I started to have a look at this issue and I encountered a problem when linking LLVM.  In the CmakeList.txt, I replaced
llvm_map_components_to_libnames(llvm_libs all) that doesn't work anymore (llvm_libs was empty) by llvm_map_components_to_libnames(llvm_libs support core irreader engine) (I'm not sure of the components to use so I put some of them just to check if the method was still working and if it is the all argument that is not valid anymore and indeed the all argument seems nott valid anymore - now I should ask Stephane which ones to use exactly...). Anyway, now, it seems that LLVMjitt is not found (and I guess that should be necessary for Faust), I will try to recompile LLVM manually (perhaps this lib is not shipped with the pre-built version?). For the moment, it was just a quick test but I have to further investigate the changes since LLVM 8...

Hi Pierre, great to hear from you! Well, you're pretty close already, getting the required LLVM libs sorted out shouldn't be that hard. In fact, I think that just linking against libLLVM-10.so should be enough (at least that's what /usr/lib/libfaust.so links against over here), and you already got that in your link line. We just need to get rid of all the rest. ;-) I'm not sure how well that will work on Mac and Windows, though, but we can start worrying about that once we've got the Linux version working.
I'm taking a look at the source myself right now, as there are also some things I'd like to add -- like dumping a list of all parameters and the metadata (not just printing them in the Pd console), as well as OSC, MIDI and polyphony support, which are still needed to bring pd-faustgen up to par with Stephane's Max/MSP faustgen and my own pd-faust. The dynamic Pd GUI generator is still something that only pd-faust offers right now, but I can probably replace that with something written in Lua (using Pd-Lua).
I'd really like to get rid of pd-faust at some point and use pd-faustgen instead. It's the way forward, as pd-faust still uses the ancient LLVM 3.5, which gives me a lot of trouble nowadays. (It just crashes Pd when trying to use any external that depends on the system LLVM in some way, such as Gem.)

But to get back to the linker issue, I don't see why the original line, llvm_map_components_to_libnames(llvm_libs all), would have to be changed, since it seems to work fine when linking the master branch against LLVM 10 (after applying the patch to bring LLVM 10 compatibility to the old Faust snapshot). Let me give that a try.

Works fine over here (using the LLVM 10 package from the official Arch repos, that is). More precisely, if I just take dev/v0.1.2 as it is, and change back that one line to:
llvm_map_components_to_libnames(llvm_libs all)

The build then runs through just fine, no hitches during linking. The binaries also work (tested by running faustgen~-help.pd in Purr Data).
Here's the link line reported by running make VERBOSE=1:
/usr/bin/cc -fPIC   -shared -Wl,-soname,faustgen~.pd_linux -o "../external/faustgen~.pd_linux" CMakeFiles/faustgen_tilde_project.dir/src/faust_tilde_io.c.o CMakeFiles/faustgen_tilde_project.dir/src/faust_tilde_options.c.o CMakeFiles/faustgen_tilde_project.dir/src/faust_tilde_ui.c.o CMakeFiles/faustgen_tilde_project.dir/src/faustgen_tilde.c.o  ../faust/build/lib/libfaust.a -lpthread -lstdc++ -L/usr/lib -lLLVM-10

Much tidier, it only links against libLLVM-10.so, as it should. So for me rev. 3d2d282a7ffedb0be28bd392e20eae4d76e91af9 actually broke things and should probably be reverted. Maybe there's an issue with your LLVM 10 installation in Travis which tripped the linker problems with rev. 0a5d578a8816c58a1427e1609f844e3e075b7d06?
Anyway, this means that I should be able to create a working package by just going back to rev. 0a5d578a8816c58a1427e1609f844e3e075b7d06.

Yep, rev. 0a5d578a8816c58a1427e1609f844e3e075b7d06 does it, so I updated the pd-faustgen-git package on the AUR accordingly: https://aur.archlinux.org/packages/pd-faustgen-git/
Anyway, once v0.1.2 is out, I'll see whether we can get @dvzrv from the Arch proaudio team on board to take over and improve my pd-faustgen package, so that it becomes officially available in the Arch community repositories.

On Travis CI, I use the LLVM pre-built 10.0.0 (https://github.com/llvm/llvm-project/releases/download/llvmorg-10.0.0/clang+llvm-10.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz and https://github.com/llvm/llvm-project/releases/download/llvmorg-10.0.0/clang+llvm-10.0.0-x86_64-apple-darwin.tar.xz). Then using llvm_map_components_to_libnames(llvm_libs all), the CI fails in CMakeList.txt because llvm_libs is empty when calling list(REMOVE_ITEM llvm_libs LTO). But if we just have to link against libLLVM, perhaps we could remove these lines. I only have to ensure that it works fine on mac and windows too. I'm on holidays so I'll have time to work on it the next weeks :)

For macOS, there is no libLLVM-10 in the pre-built package. I managed to link against 'all' again using llvm_map_components_to_libnames(llvm_libs ${LLVM_TARGETS_TO_BUILD})  (that seems to include all the libs) but there are still unresolved symbols at runtime:
tried .../pd-faustgen/external/faustgen~.pd_darwin and succeeded
.../pd-faustgen/external/faustgen~.pd_darwin: dlopen(/.../pd-faustgen/external/faustgen~.pd_darwin, 10): Symbol not found: __ZN4llvm11ObjectCache6anchorEv
  Referenced from: .../pd-faustgen/external/faustgen~.pd_darwin
  Expected in: flat namespace
 in .../pd-faustgen/external/faustgen~.pd_darwin

That's unfortunate. So maybe they always distribute LLVM as static libs only. But even then llvm_map_components_to_libnames(llvm_libs all) is supposed to work. At least I think so; what little documentation I could find about LLVM's cmake build system on llvm.org doesn't say so, it only talks about building llvm itself.
In any case, the component specifier all is recognized, I can see that looking at the definition of llvm_map_components_to_libnames, in LLVM-Config.cmake line 269. If it gives an empty list of libs, then IMHO that's a bug in the cmake configuration, so either they botched that release, or their cmake config is broken in some way, or both. Oh well. It's not the first botched LLVM package, I've seen many over the years. It's a bit disturbing, though, that something like that slips through LLVM's own CI.
One thing you might still try is to run the shell command llvm-config --libs all instead. At least on my system that gives me the proper link options (-lLLVM-10). But if llvm_map_components_to_libnames is broken, llvm-config might well be broken, too.
So LLVM 10 seems to be a no-go on Travis for now. I'd suggest to revert both rev. 3d2d282a7ffedb0be28bd392e20eae4d76e91af9 and c4bd14f73acd08bda13f850e40ebef601843ab0a, and just go back to the LLVM version that you were using before. AFAICT, the version in https://github.com/CICM/pd-faustgen/releases/tag/v0.0.0 should be LLVM 5, right? With a bit of luck, that should still be recent enough to build current Faust. And we already know that that version builds fine with LLVM 10 on Arch at least.

I've got that source in my testing branch over at https://github.com/agraef/pd-faustgen/tree/testing now, where I simply got rid of rev. 3d2d282 and c4bd14f entirely. You could just pull that branch over to your repo and check whether it compiles ok in Travis, check that it compiles in Windows, too, and release it as v0.1.2.

I dropped the commit but it fails on mac and {Windows](https://ci.appveyor.com/project/CICM/pd-faustgen/builds/34447214) and the link for LLVM-5 pre-built has changed so it doesn't compile on (Linux)[https://travis-ci.org/github/CICM/pd-faustgen/jobs/714406425]. This last problem can easily be fixed but it seems that the Faust 2.27.2 is not compatible with LLVM-5 on Windows and Mac...

Using LLVM-9.0.0 with Faust v2.27.2 seems to work on Linux and Mac so I'll try with this version. But I think I have to create a new LLVM package for Windows. Once it compiles on Windows I can create a new release v0.1.2. For the other requests can you create new issues and/or MRs?

You mean dump, OSC, MIDI and polyphony support? Yes sure, will do.

Great thanks! I managed to compile against LLVM-9.0.0 on Windows with Faust v2.27.2 so I will setup the CI with the new LLVM pre-built and prepare a new release 0.1.2. Then we could start a GitHub project for 0.2.0 with your requests, does it sound good to you?

Everything seems to work! If the branch dev/v0.1.2 is good for you I will merge it into master.

Sounds great! @sletz, is there some documentation about faustgen somewhere, specifically about its polyphony and OSC support?

Maybe here: https://github.com/grame-cncm/faust/tree/master-dev/embedded/faustgen

Oops, sorry, I should have been able to find that myself. ;-)

@pierreguillot I just tested again with the latest dev/v0.1.2 and it still builds fine with LLVM 10 on Arch. As soon as you've merged into master, I'll update my AUR packages.
For a proper Arch package, we'll probably need a complete tarball, though, submodules included. Do you know how to do this? Something like this: https://github.com/grame-cncm/faust/blob/HEAD/Makefile#L205 (that's the dist target in Faust's toplevel Makefile; sorry, I'm not sure how to add a target like this to the cmake config).

No, I've no idea of the proper way to do that. Do you really need a new target? Perhaps, we add a simple bash script to do that and create a new artefact on the Linux CI when there is a tag? Do you want to do that for v0.1.2? I close this issue and we can create another one for this specific question.

No, a bash script would be fine. I can write one and submit it as a PR. Stay tuned.

See https://github.com/CICM/pd-faustgen/pull/10.


fixes https://github.com/kongchen/swagger-maven-plugin/issues/659

@martinkalina Thank you for the PR. Could you please add some unit test coverage for this change?

Hi, I enhanced your tests to cover the change. I had to add Kotlin to the project.

@martinkalina If the Kotlin dependencies are for tests only, could you please add the test scope to the dependencies?

Thanks @martinkalina . The scoping change is appreciated.


UAT: Case Filing Inquiry Affidavit

on Respondent details page, during case filing, it says that if Pin code is different, need to file 202 inquiry affidavit, but the image upload placeholder below says 225 Inquiry affidavit. What's the correct Affidavit that needs to be filed?

Why do we only support JPG and PDF? Why don't we allow PNG?
Even with JPG, it doesn't allows other known extension JPEG?

Updated the localisation for note message.
Why do we only support JPG and PDF? Why don't we allow PNG?
-->We added these file support as per the PRD and figma screen.
Even with JPG, it doesn't allows other known extension JPEG?
--> In PRD and figma only JPG was mentioned not JPEG.

@Amityadav012 JPG and JPEG are same file formats. JPEG stands for Joint Photographic Experts Group and JPG is just a short form.
Please ensure that wherever JPG is supported, we do allow JPEG also and vice versa.
@manimaarans  please take note of the same.

@manimaarans  can you update and close the ticket or move to backlog based on the status of the ticket


Increase grafana-operator-controller-manager memory limit

On OpenShift with 22 dashboards, the manager pod peaks around 465Mi at startup and then goes down to 250-270Mi. Increase the limit otherwise pod is dies with OOMKilled error.
Increase all 3 types of deployment for consistency reasons even though the referenced issue was for OLM.
Fixes #1255

Apparently config/manager/manager.yaml don't generate the kustomize file (which might be an issue) but nonetheless, I don't see the need to increase the memory limit for kustomize users.
So far, no one have complained, and I think it's a general best practice to set your resource requirement as you see fit.
But as discussed in the issue, setting OLM memory limits is a pain and that is why we increase them.
Thanks for the PR @hugares


git-codereview: init at 2020-01-15

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

@edef1c Please use the complete pull request review template - https://github.com/NixOS/nixpkgs/blob/master/.github/PULL_REQUEST_TEMPLATE.md .

Thanks!


Test update

Passing a lot, and set up, but a couple of issues with getting and fixing/re-setting indexed values.

I figured out the syntax to get the indexed variable results. And I updated 'Consumer' so it can be fixed. Still need to figure out how to update an indexed Commodity, or to include a parameter that is a coefficient on just part of an indexed variable, so we can run that counter-factual.

I made Consumers fixable values. And I added a counter factual for the JPMGE model to the test that works (but for now, it re-runs the whole model in order to change the benchmark value for Y [see below])
Remaining issue is to be able to actually re-set/update values for variables like we do for parameters. Maybe we need to check JuMP for that?

I think we're good and done on this.


Add footer section in each of the services page

I would like to add a footer section in each service page .

@con-artist start working!


initial host.json basic

This is just a sample of what it could look like. Please disregard the format, the length or anything for that matter.
There are also many possibilities with the voice that is being used. I'm definitely going for an impersonal voice due to where it will be hosted.
Ideas, critics and praises are welcome.

@MaximRouiller,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

sandbox/101functions/basics/host.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

sandbox/101functions/basics/host.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

@anthonychu @BrianPeek who can sign-off and merge, if all this is okay?

@MaximRouiller In order to sync up over formatting & consistency, should we hold off on merging until after our status meeting?

I would like to wait until after we push live and discuss on the call tomorrow, if possible.  You can review the PR on the review site for now.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

sandbox/101functions/basics/host.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

sandbox/snippets/101functions/basics/host.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

As per @BrianPeek recommendation.  Moved the files to the right directory.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

sandbox/snippets/101functions/basics/host.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Support deletion of basic message records in messaging plug-in

The basicmessage_storage plug-in retains all basic message records so that they can be consumed through the Tenant UI. For privacy and security reasons, it should be possible to delete messages once received, or not retain them at all.
Acceptance Criteria:

Add endpoint to support programatic deletion of basic messages
Add setting to disable retention of messages received?

Note: this will have effects on Tenant Ui functionality, that might need to be revisited.

I can see how "delete all" might be desirable in some scenarios, unsure about the priority.
Another thing to look at is being able to disable storage on a per-tenant basis, to cater to deployments (like Traction) that might need different behaviour depending on their supported use-cases.

https://github.com/hyperledger/aries-acapy-plugins/pull/52 resolves the most basic of functionality.
What about future work?

[ ]  disable storage option (per tenant? per connection)
[ ]  delete all message (per tenant? per connection?)
[x]  Are there unit tests?

Where are the startup issues being tracked? I don’t recall seeing those.
I notice the issues above related to the use of the indy-sdk, which we shouldn’t be using...

Same question. We've built and deployed an 0.11 Traction to Dev (and PRs work), and I can build and start local docker on main.

Where are the startup issues being tracked? I don’t recall seeing those.

0.11.0 has some issues starting up as a dependency to plugins as the tenant based logging has some file permission issues. @shaangill025 is working on a resolution.
The basicmessage_storage plugin has a dependency for mergedeep that it's pyproject.toml file does not capture properly. I should add that as it's own pull request.

I notice the issues above related to the use of the indy-sdk, which we shouldn’t be using...

I changed the wallet-type to askar and get the error aries_askar.error.AskarError: The requested database path was not found so it appears that askar doesn't want to run in-memory, but I haven't confirmed that either.

The issue @shaangill025 is working on is related to starting ACA-Py on bare-metal (not Docker).
For Askar runnig in-memory: I am pretty sure I have run it in-memory before, make sure all of the other settings related to the type of storage, etc. have been removed otherwise it might be getting confused.

Completed, single message deletion enabled, and top level config and sub-wallet configurations to enable/disable.
This is a breaking change, if deployed acapy instances do not set the plugin config value appropriately, that acapy instance will no longer save messages (incoming or outgoing).


[Feature Request] Multi-Modality Support for More Comprehensive Applications

While Prompt Flow already offers impressive flexibility regarding data types within the overall stack, encompassing flow input/output and node intermediate data, it is crucial to address the limited support for visualization and tracing, which primarily revolves around text-oriented capabilities.
To further empower developers in creating comprehensive multi-modality applications, the following enhancements are highly recommended, although not exclusively limited to:

Media Type Specification: Extend the functionality of Prompt Flow to enable users to specify the media type (e.g., image, video, audio) for flow input/output, facilitating appropriate visualization and interpretation.

Expanded Testing/Chat Interface: Enhance the test/chat interface to accommodate inputting varying media types, ensuring seamless interaction with diverse modalities throughout the development process.

Media Type Specification for Node Intermediate Data: Provide developers with the ability to specify the media type for node intermediate data and ensure proper visualization within the Visual Studio Code (VSC) extension and portal, promoting a more integrated and holistic development experience.

Streaming Output for Multimedia Types: Particularly for media types such as video and audio, enable the streaming output capability, allowing for real-time and dynamic visualization that caters to the specific demands of multimedia applications.

Hi, Is there any updates for this feature request?


fflogs: 8.12.19 -> 8.13.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/RPGLogs/Uploaders-fflogs/releases.
meta.description for fflogs is: Application for uploading Final Fantasy XIV combat logs to fflogs.com
meta.homepage for fflogs is: https://www.fflogs.com/client/download
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 8.13.1 with grep in /nix/store/jpak0q5p9p5vakv1297zyx63dgszalb0-fflogs-8.13.1
found 8.13.1 in filename of file in /nix/store/jpak0q5p9p5vakv1297zyx63dgszalb0-fflogs-8.13.1

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

fflogs

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/jpak0q5p9p5vakv1297zyx63dgszalb0-fflogs-8.13.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A fflogs https://github.com/r-ryantm/nixpkgs/archive/62dcf47ab471bb6e0883f156ab10d2b5e97477bd.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/62dcf47ab471bb6e0883f156ab10d2b5e97477bd#fflogs

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/jpak0q5p9p5vakv1297zyx63dgszalb0-fflogs-8.13.1
ls -la /nix/store/jpak0q5p9p5vakv1297zyx63dgszalb0-fflogs-8.13.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

fflogs

Maintainer pings
cc @sersorrel for testing.

[!TIP]
As a maintainer, if your package is located under pkgs/by-name/*, you can comment @NixOS/nixpkgs-merge-bot merge to automatically merge this update using the nixpkgs-merge-bot.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 336212 run on x86_64-linux 1

1 package built:

fflogs

@NixOS/nixpkgs-merge-bot merge


[REVIEW]: Please review my Open Source Project

Project Link:
https://www.github.com/DesignSystemsOSS/eccentrictouch
You can also join the discord community here
Feel free to check out other cool repositories of EddieHub Community here

Thank you for sharing - we will review on a livestream when time allows :)

Nice one. Labels on issues I like it!
some suggestions:

remove the Version Releases in the Readme due those could be found also in releases https://github.com/DesignSystemsOSS/eccentrictouch/releases
remove the view of the package due ist not needed

Sure! Thanks for the suggestions @schmelto 👍🏽

Reviewing on a live stream https://youtu.be/tZJGPO432OQ
Looks great, some thoughts:

You have releases/tags 🎉
Suggest hiding packages if not using that section
Not sure about screenshots from the docs, could there be a screenshot from an example project
Suggest using conventional commit messages


Text-Question does not allow certain special characters such as !?* and {}

Text-Question does not allow certain special characters such as !?* and {}. This should work.
Instead it returns an error 400 on when pressing next.

Input is validated against a function treating the following charracters as illegal: /><;?*!&{}. PR #92 contains a debatable proposed solution.


merge package from arlac77/npm-package-template-esm-only

package.json

chore(package<EMAIL_ADDRESS>chore(scripts): cover@#overwrite c8 --temp-directory build/coverage ava && c8 report -r lcov --temp-directory build/coverage

Codecov Report

Merging #8 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master       #8   +/-   ##
=======================================
  Coverage   88.41%   88.41%
=======================================
  Files           4        4
  Lines         164      164
  Branches       22       22
=======================================
  Hits          145      145
  Misses         19       19

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 223e6eb...2876483. Read the comment docs.


[Bug]: Datepicker dropdown menu is cut off in the deployed mode

Is there an existing issue for this?

[X] I have searched the existing issues

Description
When I open a datepicker dropdown menu it will be cut off if it is at the top of the page.

This only happens in Appviewer mode, not while building the app.
Steps To Reproduce

Add a date picker to your app
Deploy the app
Scroll down so that the date picker is at the top page
Open date picker

Public Sample App
No response
Version
Cloud

@momcilo-appsmith please share the app unable to reproduce this

@dilippitchika For some reason, I can't find it and I can't reproduce it as well. I saw it in an app from one of our users and reproduced it that day. But it seems like it is solved now.


simplify typechain usage for queries and tx

currently the typechain return types are a massive pain. Can we make this better? probably.
E.g. having to handle ReturnNumber type currently

tackle this with the js side of sr25519

Part of ongoing typechain fork work

Refinement - can we get any info here into the tickety stuff for typechain proposal

And then close this


Request to unpublish this package

Hi WinkMeter
If you are not planning to support this package please unpublish.
Saves a lot of time for developers.
Thanks
Dev

Or transfer repository for other developer


e2e: retry creation of the RBD StorageClass

On occasion the creation of the StorageClass can fail due to an
etcdserver timeout. If that happens, the creation can be attempted after
a delay.
This has already been done for CephFS StorageClasses, but was missed for
RBD.
See-also: ceph/ceph-csi@8a0377ef02

Show available bot commands
These commands are normally not required, but in case of issues, leave any of
the following bot commands in an otherwise empty comment in this PR:

/retest ci/centos/<job-name>: retest the <job-name> after unrelated
failure (please report the failure too!)
/retest all: run this in case the CentOS CI failed to start/report any test
progress or results

/retest ci/centos/k8s-e2e-external-storage/1.22

/retest ci/centos/mini-e2e-helm/k8s-1.23

/retest ci/centos/k8s-e2e-external-storage/1.22

Failed to create Ceph cluster

/retest ci/centos/mini-e2e-helm/k8s-1.23

node-plugin went missing:
Apr 14 16:48:14.577: INFO: 1 / 1 pods ready in namespace 'cephcsi-e2e-be428dbb301f' in daemonset 'csi-rbdplugin' (0 seconds elapsed)
Apr 14 16:48:14.588: FAIL: failed to run uname cmd : podlist is empty, stdErr:

/retest ci/centos/mini-e2e-helm/k8s-1.23

/retest ci/centos/mini-e2e-helm/k8s-1.23

Rook deployment failure

@mergifyio rebase

@mergifyio requeue


Uploaded finding image 404s (surewould had mentioned this issue #1674)

I had not solve  this problem yet, and tried many way ,but no way .

quoted @surewould behind!

Bug description
I created a clean release docker build from the dev branch, created a finding, uploaded an image, and the image doesn't show in the finding after. Instead of seeing the expected image I get a 404.
Steps to reproduce
Steps to reproduce the behavior:

Add a test product
Add a test finding
Manage images on finding
Upload and save a picture
Image won't show in thumbnail when dropped back to the finding main page, and navigation into the picture will give you a proper 404

Expected behavior
I expected to see the uploaded images displayed with the finding
Deployment method (select with an X)

[ ] Kubernetes
[x] Docker
[ ] setup.bash / legacy-setup.bash

Environment information

Operating System: [e.g. CentOS 7.4]

Screenshots (optional)

Are you running from Dev branch or master branch ?

Are you running from Dev branch or master branch ?

master

same problem

It works fine when debug:"on" , but still 404 in debug:off .

I solved this problem (started with docker);
I modified docker-compose. Yml to mount the path to save the image locally and give read and write permissions, then it works.
Modified parts:
uwsgi:
... ...
volumes:
      - /usr/local/django-DefectDojo-<IP_ADDRESS>/media:/app/media/finding_images/
      - /usr/local/django-DefectDojo-<IP_ADDRESS>/media/cache:/app/media/CACHE/images/finding_images/

I tried to give read and write rights directly inside the container, but failed.
@dougmorato @Exploit-3389

I made the following changes to docker-compos.yml and nginx.conf,then it can successfully show  images when in debug:"False":
# vim docker-compose.yml

nginx:
  ... ...
  volumes:
      - /data/defectdojo/media/:/usr/share/nginx/html/media
      - /data/defectdojo/media/CACHE:/usr/share/nginx/html/media/CACHE
uwsgi:
  ... ...
  volumes:
      - /data/defectdojo/media/finding_images/:/app/media/finding_images/
      - /data/defectdojo/media/CACHE/images/finding_images/:/app/media/CACHE/images/finding_images/

# vim nginx.conf
... ...
location /media {
      alias /usr/share/nginx/html/media;
    }
... ...

@ClintCrawford

I made the following changes to docker-compos.yml and nginx.conf,then it can successfully show images when in debug:"False":
# vim docker-compose.yml

nginx:
  ... ...
  volumes:
      - /data/defectdojo/media/:/usr/share/nginx/html/media
      - /data/defectdojo/media/CACHE:/usr/share/nginx/html/media/CACHE
uwsgi:
  ... ...
  volumes:
      - /data/defectdojo/media/finding_images/:/app/media/finding_images/
      - /data/defectdojo/media/CACHE/images/finding_images/:/app/media/CACHE/images/finding_images/

# vim nginx.conf
... ...
location /media {
      alias /usr/share/nginx/html/media;
    }
... ...

thx , it really works  at first upload , when I delete  the picture ,and upload the same picture (same filename),the thumbnail will be broken ,but the original pic is ok .!

debug:"on"

do you know how to write it with debug:"on" ?

debug:"on"

do you know how to write it with debug:"on" ?

I made the following changes to docker-compos.yml and nginx.conf,then it can successfully show images when in debug:"False":
# vim docker-compose.yml

nginx:
  ... ...
  volumes:
      - /data/defectdojo/media/:/usr/share/nginx/html/media
      - /data/defectdojo/media/CACHE:/usr/share/nginx/html/media/CACHE
uwsgi:
  ... ...
  volumes:
      - /data/defectdojo/media/finding_images/:/app/media/finding_images/
      - /data/defectdojo/media/CACHE/images/finding_images/:/app/media/CACHE/images/finding_images/

# vim nginx.conf
... ...
location /media {
      alias /usr/share/nginx/html/media;
    }
... ...

could you paste the full docker-compose.yml you have ? or how to run it with debug:"on" ? I want to solve this images issue too, regards.

I made the following changes to docker-compos.yml and nginx.conf,then it can successfully show images when in debug:"False":
# vim docker-compose.yml

nginx:
  ... ...
  volumes:
      - /data/defectdojo/media/:/usr/share/nginx/html/media
      - /data/defectdojo/media/CACHE:/usr/share/nginx/html/media/CACHE
uwsgi:
  ... ...
  volumes:
      - /data/defectdojo/media/finding_images/:/app/media/finding_images/
      - /data/defectdojo/media/CACHE/images/finding_images/:/app/media/CACHE/images/finding_images/

# vim nginx.conf
... ...
location /media {
      alias /usr/share/nginx/html/media;
    }
... ...

could you paste the full docker-compose.yml you have ? or how to run it with debug:"on" ? I want to solve this images issue too, regards.


⚠ SceneTime has degraded performance

In 543bfcf, SceneTime ($ST) experienced degraded performance:

HTTP code: 200
Response time: 8047 ms

Resolved: SceneTime performance has improved in e598498 after 6 minutes.


Relative score, coverage and weighted coverage are missing

@lsabor  are all of these metrics still valid ?

Relative Score, Archived Relative Score, and coverage are all completed.
Weighted Coverage is imported into the Archived Relative Score, so any already resolved questions will display the weighted coverage correctly. Any future questions which resolved and have weighted coverage should be changed to no longer have weighted coverage as this will have no consequences on already known results.
I'll leave this ticket open until I've QA'd this.

Weighted coverage is the only thing that needs QA still, and effects nearly nothing. Moving to lower priority.

This looks correct to me. Weighted Coverage is essentially just the "Comboprediction.coverage" from the old database. It had two differences from current coverage

"weighted coverage" is over the "actual open period" not "scheduled open period", so a question resolving early didn't prevent users from getting 100% weighted coverage.
If a question has a hidden period coverage weighting, then that gets accounted for in the weighted coverage

Also worth noting that "Weighted Coverage" for any question resolving before the rewrite is actually taken directly from the old codebase and stored in the "ArchivedScore" table, which would incorporate and weighting that was going on back then. It was stored in the old "Comboprediction.coverage". Questions resolving after the rewrite no longer take hidden period coverage weighting into account, but the "weighted coverage" is still based on (1) above.
If this doesn't seem right to you, let's have a conversation about it.

Sorry. Yes you are correct, we can close this.


Go build is slow in Dagger

Problem
It is commonly reported that building a go program is slower in Dagger. There is no fundamental reason that this should be the case.
Solution
Help users identify the root cause of their slow go build, and help them fix it. If necessary, fix the root cause in the product and docs.
The most likely cause is the lack of a properly setup cache volume. As a result, each go build runs on an empty go cache, forcing it to scan and re-download all dependencies.

Thank you for opening this! I've been meaning to post in Discord but got sidetracked and when I heard that someone else also suspects their go builds are slow with the SDK I thought I'd bring it up. What I am doing is a multi-arch multi platform build for a go CLI. I will try my best to share as much code as I can while hiding the corporate bits.
package main

import (
	"context"
	"fmt"
	"os"
	"os/exec"
	"time"

	"dagger.io/dagger"
	platformFormat "github.com/containerd/containerd/platforms"
	"golang.org/x/sync/errgroup"
)

var (
	publishRepo    = "mycompany.docker.repo.com"
	dockerRepoEdge = "mycompany-edge.docker.repo.com"
	platforms      = []dagger.Platform{
		"linux/amd64",
		// "linux/arm64",
	}
	buildImage   = dockerRepoEdge + "/jenkinscore/ubn20/golang-node16:2.4.0-0"
	publishImage = publishRepo + "/myimagename:latest"
)

func main() {
	if err := os.RemoveAll("dist/"); err != nil {
		fmt.Println(err)
	}

	ctx := context.Background()

	client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stdout))
	if err != nil {
		fmt.Println(err)
	}
	defer client.Close()

	err, variants := build(ctx, client.Pipeline("Build"))
	if err != nil {
		fmt.Println(err)
	}

	if err := publish(ctx, client.Pipeline("Publish"), variants); err != nil {
		fmt.Println(err)
	}

}

// util that returns the architecture of the provided platform
func architectureOf(platform dagger.Platform) string {
	return platformFormat.MustParse(string(platform)).Architecture
}

// util that returns the OS of the provided platform
func platformOf(platform dagger.Platform) string {
	return platformFormat.MustParse(string(platform)).OS
}
func build(ctx context.Context, client *dagger.Client) (error, []*dagger.Container) {
	fmt.Println("Building with Dagger")

	err := dockerLogin(publishRepo, dockerRepoEdge)
	if err != nil {
		return err, nil
	}

	gocache := client.CacheVolume(time.Now().Weekday().String())

	g, ctx := errgroup.WithContext(ctx)

	platformVariants := make([]*dagger.Container, 0, len(platforms))
	for _, platform := range platforms {

		p := platform
		g.Go(func() error {

			golang := client.Container().
				From(buildImage).
				WithEnvVariable("https_proxy", os.Getenv("https_proxy")).
				WithMountedCache("/cache", gocache).
				WithEnvVariable("GOMODCACHE", "/cache").
				WithEnvVariable("GOOS", platformOf(p)).
				WithEnvVariable("GOARCH", architectureOf(p)).
				WithMountedDirectory("/src", client.Host().Workdir()).
				WithWorkdir("/src")

			golang = golang.
				WithExec([]string{"go", "mod", "download"}).
				WithExec([]string{"make", "build"}).
				WithExec([]string{"make", "test"})

			path := "dist"
			output := golang.Directory(path)

			binaryCtr := client.Container(dagger.ContainerOpts{Platform: p})

			binaryCtr = binaryCtr.WithDirectory("/dist", output).
				WithFile("/bin/app", output.File("app")).
				WithEntrypoint([]string{"./bin/app"})
			platformVariants = append(platformVariants, binaryCtr)
			return nil
		})

	}
	if err := g.Wait(); err != nil {
		return err, nil
	}
	return nil, platformVariants
}

func publish(ctx context.Context, client *dagger.Client, variants []*dagger.Container) error {
	img, err := client.Container().
		Publish(ctx, publishImage, dagger.ContainerPublishOpts{
			PlatformVariants: variants})
	if err != nil {
		return err
	}

	fmt.Println(img)
	return nil
}

func dockerLogin(repos ...string) error {
//.....login to company private docker registry
}

Note that adding the errGroup didn't really seem to make a difference in total time taken.
Also note that make build and make test aren't doing anything fancy. Just running go build and go test. When I run this locally outside of dagger those commands take only a few seconds (less than 10). But when I run within Dagger it takes almost 2 mins per command. I even tried only keeping 1 platform which essentially makes this a single build+test (no parallel). That didnt' seem to improve things.
When I run consecutively the cache kicks in and the build is almost instantaneous. But if I make any changes then again it takes all that time.
I also did a dagger-graph on this to see if I can identify where it's taking time. And it shows that the build and test steps are what's taking so long.
I tried increasing the amount of resources I allocate to my docker desktop but it didn't make a difference. I can allocate a lot because I have a M1 Max.
Maybe I am doing something silly that's making it so slow but I can't see it.

qemu issue?

I just added a cache mount for the go build cache (which is separate from the go mod cache) to our CI, which seems to make a significant difference (provided your runner isn't ephemeral of course): https://github.com/dagger/dagger/blob/main/internal/mage/util/util.go#L113
There we put it at /root/.cache/go-build but you can double check the value by running go env in your image and looking at the value of GOCACHE. Or I think (not 100% sure) if you set the GOCACHE env var to anything custom the go toolchain will honor that

qemu issue?
@nipuna-perera
is there a linux/arm64 of your buildImage? (Apple m1 need it to run without qemu)

That's a good point. We don't have a arm64 version of it. I can try with a separate image that is arm64

I just added a cache mount for the go build cache (which is separate from the go mod cache) to our CI, which seems to make a significant difference (provided your runner isn't ephemeral of course): https://github.com/dagger/dagger/blob/main/internal/mage/util/util.go#L113
There we put it at /root/.cache/go-build but you can double check the value by running go env in your image and looking at the value of GOCACHE. Or I think (not 100% sure) if you set the GOCACHE env var to anything custom the go toolchain will honor that

I will try again with the build cache. However, my builds run fast locally outside of Dagger even without a build cache.

I will try again with the build cache. However, my builds run fast locally outside of Dagger even without a build cache.

Go builds automatically use that build cache, so it should be used outside of dagger unless you are explicitly disabling it there.

It seems that the mismatch in architecture of my builder image is what's slowing things down. So I guess it's related to qemu. When I pull an image that's arm64, the build goes almost as fast as my local.
It might help to print a warning when the architecture doesn't match that the performance may be affected.

Now that we support multi-platform in the API, I wonder if we could implement in Dagger the cross-compile strategy used by https://github.com/crazy-max/goxx ?

It might help to print a warning when the architecture doesn't match that the performance may be affected.

It is a buildkit feature.
if you engine always run on same machine
it could be set like
client.Container(dagger.ContainerOpts{Platform: fmt.Sprint("linux/%s", runtime.Arch)}).From(buildImage)

But pls don't set it by default.
It will broke runing with remote engine.

Now that we support multi-platform in the API, I wonder if we could implement in Dagger the cross-compile strategy used by https://github.com/crazy-max/goxx ?

It will be better, is client could return the runner platform info
type Runner struct {
     Platform string
     OS string
     Arch string
}

Then we could use to define --platform=$BUILDPLATFORM in dagger.
client.Container(dagger.ContainerOpts{Platform: client.Runner().Platform})

FROM --platform=$BUILDPLATFORM crazymax/goxx:1.17 AS base

What should happen next with this? Were any PRs opened for this? If there is some local code, should a PR be created? cc @grouville

There are currently no pull requests (PRs) open for this issue. However, there are two proposals being discussed. The issue at hand involves two separate problems that are interconnected but can be solved independently:
1. Incorrectly configured cache volumes:
According to Solomon's statement, the issue seems to be stemming from incorrectly configured cache volumes.
The fix for this is relatively straightforward, as we need to ensure that people are aware of two key aspects:

State-of-the-art implementations.
The iterative debugging process to address this issue.

Currently, our documentation does not cover these points.
Therefore, the first short-term step towards resolving this issue would be to update the documentation and provide the necessary information. The new TUI will assist in identifying such issues, and a documentation upgrade on the example found here should suffice.
2. Mismatch in the architecture of the builder image:
The second issue is related to a mismatch in the architecture of the builder image.
When building a Go binary on a different architecture than the runner (e.g., building on Mac M1 and running on a different architecture), virtualization is forced without the users being aware of it.
It appears to be related to the use of qemu, as mentioned: "When I pull an image that's arm64, the build goes almost as fast as my local."
Morlay's proposal suggests adding the following code snippet to address this issue:
type Runner struct {
    Platform string
    OS       string
    Arch     string
}

// If client.Container.From() doesn't have an architecture == client.Runner().Platform(), show warning
client.Container(dagger.ContainerOpts{Platform: client.Runner().Platform}).From("")

Implementing this solution would enhance the developer experience (DX) by ensuring that all downloaded images preferably match the runner's architecture, thus avoiding virtualization.
If a mismatch occurs, a warning should be displayed to the user, as troubleshooting this behavior requires knowledge and experience in working with containers.
@gerhard:

First one can be done async, without consensus.
Second one is important too, and requires consensus on the DX part

Update on point 2
It turns out that we already retrieve the architecture of the builder. So, my proposal is unnecessary.
For that, Erik implemented the client.DefaultPlatform() API:
package main

import (
	"context"
	"fmt"
	"os"

	"dagger.io/dagger"
)

func main() {
	ctx := context.Background()
	client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stdout))
	if err != nil {
		panic(err)
	}
	defer client.Close()

	platform, err := client.DefaultPlatform(ctx)
	if err != nil {
		panic(err)
	}

	// initialize this container with the platform of the builder
	ctr := client.Container(dagger.ContainerOpts{Platform: platform})

	// this alpine image has published versions for each of the
	// platforms above. If it was missing a platform,
	// an error would occur when executing a command below.
	ctr = ctr.From("rocker/shiny-verse")

	// execute `uname -m`, which prints the current CPU architecture
	// being executed as
	stdout, err := ctr.
		WithExec([]string{"uname", "-m"}).
		Stdout(ctx)
	if err != nil {
		panic(err)
	}

	// this should print 3 times, once for each of the architectures
	// being executed on
	fmt.Printf("I'm executing on architecture: %s\n", stdout)
}

In this example, it does exactly what we want (aka. failure on mismatch):

rocker/shiny-verse is only linux/amd64. When trying to use it, we have this error:

panic: input:1: container.from no match for platform in manifest sha256:126430df11658bae8b5a93f425f731c820447ff92458eebddca4137f0f046032: not found

It seems to be, however, unnecessary:
package main

import (
	"context"
	"fmt"
	"os"

	"dagger.io/dagger"
)

func main() {
	ctx := context.Background()
	client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stdout))
	if err != nil {
		panic(err)
	}
	defer client.Close()

	// initialize this container with the platform of the builder
	ctr := client.Container()

	// this alpine image has published versions for each of the
	// platforms above. If it was missing a platform,
	// an error would occur when executing a command below.
	ctr = ctr.From("rocker/shiny-verse")

	// execute `uname -m`, which prints the current CPU architecture
	// being executed as
	stdout, err := ctr.
		WithExec([]string{"uname", "-m"}).
		Stdout(ctx)
	if err != nil {
		panic(err)
	}

	// this should print 3 times, once for each of the architectures
	// being executed on
	fmt.Printf("I'm executing on architecture: %s\n", stdout)
}

This gives the same failure output, and I didn't have to specify the platform, Dagger retrieved it

The only moment this mismatch could happen is when the user iterates on valid platform:
platforms      = []dagger.Platform{
		"linux/amd64",
		// "linux/arm64",
	}

Here, if the image doesn't have "linux/arm64", the user could still force using a virtualized container:
package main

import (
	"context"
	"fmt"
	"os"

	"dagger.io/dagger"
)

func main() {
	ctx := context.Background()
	client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stdout))
	if err != nil {
		panic(err)
	}
	defer client.Close()

	// initialize this container with the platform of the builder
	ctr := client.Container(dagger.ContainerOpts{Platform: "linux/amd64"})

	// this alpine image has published versions for each of the
	// platforms above. If it was missing a platform,
	// an error would occur when executing a command below.
	ctr = ctr.From("rocker/shiny-verse")

	// execute `uname -m`, which prints the current CPU architecture
	// being executed as
	stdout, err := ctr.
		WithExec([]string{"uname", "-m"}).
		Stdout(ctx)
	if err != nil {
		panic(err)
	}

	// this should print 3 times, once for each of the architectures
	// being executed on
	fmt.Printf("I'm executing on architecture: %s\n", stdout)
}

It runs on my Mac M1. If I were to build a Go binary on this architecture, it would also be longer.
TLDR: The current platform behavior warns users that the builder platform isn't present in the container image. A user would definitely be aware of the fact that virtualization is being used, as it had to force a specific architecture, different than the builder's one.
Following this: this issue can be closed when:

https://github.com/dagger/dagger/pull/5139
Gets merged


Add support for Puppet metadata-json-lint pre-commit hook

Hi,
Thanks for the speedy, thorough and educational review. Id didn't realize the framework already contained such rich functionality. I guess I should have read a bit more and do less :-).
I will apply the requested changes

Don't know why the Windows tests fail :-(

Fore pushed the last requested changes so the git history stays clean.


remove version info from docs and some small doc updates

switch to plain sbt-paradox plugin to remove Akka branding
needed to remove version-info for modules - it's Akka specific and not supported by sbt-paradox
removed the Akka specific release notes
removed docs for external connectors (in community maintained repos - see https://github.com/apache/incubator-pekko-connectors/issues/23)
some random doc updates

@jchapuis are you happy with my responses to the issues you raised - if so, I'll go ahead and merge this

@jchapuis are you happy with my responses to the issues you raised - if so, I'll go ahead and merge this

sure! sorry i missed it. lgtm


Multiclass loss calculation fails at evaluation

[Evaluation] We need to correctly format the data before passing it into the multiclass loss function.
To fix:
We can replicate this line
https://github.com/NRCan/geo-deep-learning/blob/8678cd900f12d2ded72395131b804c3babd74474/train_segmentation.py#L353
here:
https://github.com/NRCan/geo-deep-learning/blob/8678cd900f12d2ded72395131b804c3babd74474/train_segmentation.py#L445

Hi Vic, I've tested this case and haven't noticed any difference between the values in the first and second implementation:

Fixed with PR #496


Allow second-level precision on day calculations

Addresses https://github.com/zendesk/biz/issues/21 (thanks, @take!).
The underlying DayTime object used to power the day calculations, up to this point, did not have second-level precision as its former primary purpose was to translate timestamps (e.g. 09:50) into actionable times.
By extending second-level precision down to this object, we bring that precision to the day calculations.
@alex-stone

Thanks for the fix! :tada:

:+1: looks good to me!

Released in v1.2.1.


Fix up broken patches for ozone and indigo

This patch was broken - the line starting // Note should have had spaces in front of it, so it wasn't being applied
I also adapted it to the current code so it applies cleanly

I also added another fix to a indigo patch that wasn't applying cleanly and getting rejected

@davidfraser thank you for your PR.
I already updated patches individually, according to this PR.
for next PR, please push one PR for single diff file. because each diff file is separated by individual feature for easier maintenance.
about patch to ozone:
I didn't notice it was broken until your PR, thank you. I added acknowledgements to you in my commit log.
about indigo patch:
I'm maintaining this repository in best effort( at current, almost weekly but not daily or shorter).
the patch broken is occurred between periodical maintenance works by commits in upstream. so I (already) remade patch in maintenance before noticing your PR.
I'm patrolling upstream commits and trying my patches are apply-able in daily CI.
then I found patch broken for indigo ( patch command produced .rej file), but missed for ozone because of no .rej produced.

Thanks, it's helpful to understand your process!


nvidia gtx960 x265 failed with exit code: 1 (0x1)

Hello,
About a year ago, I posted how Handbrake takes too long to encode in x265.
http://www.tomsguide.com/answers/id-3051134/handbrake-hevc-h265-encoding-blu-ray-hours.html
I stopped my attempts to use handbrake with an nvidia card, after learning that Handbrake does not support Hardware x265 encoding. The last answer I got, was to try StaxRIP, since it supports x265 encoding with Nvidia cards. Now I started trying Stax Rip, but every time, it comes out with an error code(ATTACHED):
nvidia 265 error.txt
Having zero experience in StaxRip, I followed the settings from a youtube video for nvidia 10bit gpu encoding. And I thought that maybe those settings were the issue. So I selected"Automatic Workflow", loaded the Bluray folder, and the error is the same.
Can someone explain me what am I missing?
Stats:
Windows 10 home 64bits
AMD FX(tm)-4130 Quad-core Processor
Ram: 16gb
Video Card:
EVGA GeForce GTX 960 Super Clocked ACX 2.0 4GB GDDR5 128 Bit Gaming Graphic Card (04G-P4-3962-KR)
Thank you

bump?

Were you attempting to encode using NVIDIA H.265 (NVEnc)? NVENC HEVC Main10 10-bit hardware encoding is only supported on NVIDIA Pascal GP10x. Not supported on NVIDIA Maxwell.
Navigate to the NVEnc folder (\Apps\NVEnc), run the command NVEncC64.exe --check-features
You will notice the the output
10bit depth               no
Refer to
https://en.wikipedia.org/wiki/Nvidia_NVENC#Fourth_Generation_.28Pascal_GP10x.29

Well yes, I was trying.
I found some topics about the "Maxwell limitations", but I also found a number of people talking about encoding with hardware on a nvidia 900 series. So I thought it was possible.
Well if it is not, then my apologies and thank you for your answer.

I've been using NVIDIA NVENC to re-encode my video collections on GTX960. Just not on 10 bit.

Well yes, I was trying.
I found some topics about the "Maxwell limitations", but I also found a number of people talking about encoding with hardware on a nvidia 900 series. So I thought it was possible.
Well if it is not, then my apologies and thank you for your answer.

Interesting,
I got the same error, even when I set it up to 8 bit.
Any ideas?

Same error exists, even when trying 8 bit.
I have changed the container to h264, x264, nvidia 264, x265, h265, nvidia 265.
Also the container, from THD+ac3, to ac3, ma4, etc.
It always "Analyzes" and then the error.
Any ideas?
"
Error Demux M2TS using eac3to 3.31 x86
Demux M2TS using eac3to 3.31 x86 failed with exit code: 1 (0x1)
The exit code might be a system error code: STATUS_WAIT_1
The exit code might be a system error code: Incorrect function.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Demux M2TS using eac3to 3.31 x86
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
"G:\WIN10\USERS\Daniel\Libraries\Downloads\staxrip 1.6\Apps\eac3to\eac3to.exe" "G:\WIN10\USERS\Daniel\Libraries\Videos\Movie\BDMV\PLAYLIST" 1) 2: G:\WIN10\USERS\Daniel\Libraries\Videos\Movie\Movie.mkv 3: "G:\WIN10\USERS\Daniel\Libraries\Videos\Movie\Movie ID3 English.thd+ac3" 1: "G:\WIN10\USERS\Daniel\Libraries\Videos\Movie_chapters.txt" 6: "G:\WIN10\USERS\Daniel\Libraries\Videos\Movie\Movie ID6 English.sup" -progressnumbers
M2TS, 1 video track, 3 audio tracks, 1 subtitle track, 2:43:48, 24p /1.001
1: Chapters, 16 chapters
2: h264/AVC, 1080p24 /1.001 (16:9)
3: TrueHD/AC3 (Atmos), English, 7.1 channels, 48kHz
(embedded: AC3 EX, 5.1 channels, 448kbps, 48kHz, dialnorm: -27dB)
4: DTS Master Audio, English, 5.1 channels, 24 bits, 48kHz
(core: DTS, 5.1 channels, 1509kbps, 48kHz)
5: AC3, English, 5.1 channels, 640kbps, 48kHz, dialnorm: -27dB
6: Subtitle (PGS), English
Creating file "G:\WIN10\USERS\Daniel\Libraries\Videos\Movie_chapters.txt"...
v02 Extracting video track number 2...
a03 Extracting audio track number 3...
a03 Removing AC3 dialog normalization...
The h264 muxer doesn't support this stream type yet.v02
Please send a 20MB sample to<EMAIL_ADDRESS>s06 Extracting subtitle track number 6...
v02 Muxing video to Matroska...
Aborted at file position 1048576.
StaxRip.ErrorAbortException: Demux M2TS using eac3to 3.31 x86 failed with exit code: 1 (0x1)
The exit code might be a system error code: STATUS_WAIT_1
The exit code might be a system error code: Incorrect function.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Demux M2TS using eac3to 3.31 x86
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
"G:\WIN10\USERS\Daniel\Libraries\Downloads\staxrip 1.6\Apps\eac3to\eac3to.exe" "G:\WIN10\USERS\Daniel\Libraries\Videos\Movie\BDMV\PLAYLIST" 1) 2: G:\WIN10\USERS\Daniel\Libraries\Videos\Movie\Movie.mkv 3: "G:\WIN10\USERS\Daniel\Libraries\Videos\Movie\Movie ID3 English.thd+ac3" 1: "G:\WIN10\USERS\Daniel\Libraries\Videos\Movie_chapters.txt" 6: "G:\WIN10\USERS\Daniel\Libraries\Videos\Movie\Movie ID6 English.sup" -progressnumbers
M2TS, 1 video track, 3 audio tracks, 1 subtitle track, 2:43:48, 24p /1.001
1: Chapters, 16 chapters
2: h264/AVC, 1080p24 /1.001 (16:9)
3: TrueHD/AC3 (Atmos), English, 7.1 channels, 48kHz
(embedded: AC3 EX, 5.1 channels, 448kbps, 48kHz, dialnorm: -27dB)
4: DTS Master Audio, English, 5.1 channels, 24 bits, 48kHz
(core: DTS, 5.1 channels, 1509kbps, 48kHz)
5: AC3, English, 5.1 channels, 640kbps, 48kHz, dialnorm: -27dB
6: Subtitle (PGS), English
Creating file "G:\WIN10\USERS\Daniel\Libraries\Videos\Movie_chapters.txt"...
v02 Extracting video track number 2...
a03 Extracting audio track number 3...
a03 Removing AC3 dialog normalization...
The h264 muxer doesn't support this stream type yet.v02
Please send a 20MB sample to<EMAIL_ADDRESS>s06 Extracting subtitle track number 6...
v02 Muxing video to Matroska...
Aborted at file position 1048576.
at StaxRip.Proc.Start() in D:\Projekte\VS\VB\StaxRip\General\Proc.vb:line 334
at StaxRip.MainForm.OpenEac3toDemuxForm(String playlistFolder, Int32 playlistID) in D:\Projekte\VS\VB\StaxRip\Forms\MainForm.vb:line 5046
"
Thank you
Thank you

The thing you CAN try however is to try and encode h265 with EVERY setting set to rec709....in theory there shouldn't be anything wrong with that. But I can't try it myself to check if that might work.

ALSO DON'T forget to set the master-display string to rec709.