ProgressPlugin breaks with webpack 4.25.* due to incorrect option colors

Bug Report or Feature Request (mark with an x)
- [x ] bug report -> please search issues before submitting
- [ ] feature request

Command (mark with an x)
- [ ] new
- [*] build
- [*] serve
- [ ] test
- [ ] e2e
- [ ] generate
- [ ] add
- [ ] update
- [ ] lint
- [ ] xi18n
- [ ] run
- [ ] config
- [ ] help
- [ ] version
- [ ] doc

Versions

Repro steps

npm install
npm run build / serve
The log given by the failure

Progress Plugin Invalid Options
options['colors'] is an invalid additional property
options should pass "instanceof" keyword validation
options should match exactly one schema in oneOf
ValidationError: Progress Plugin Invalid Options
options['colors'] is an invalid additional property
options should pass "instanceof" keyword validation
options should match exactly one schema in oneOf
Desired functionality

In common.ts#L147 the ProgressPlugin is called with parameter colors. This does not meet the schema which is validated ProgressPlugin.js#L90
Mention any other details that might be useful

Having the same issue when a making fresh project.
Repro steps
ng new [project-name]
npm install && ng serve --open --prod
Expected result
Should open new window with fresh project opened
Result
λ ng serve -o
Progress Plugin Invalid Options

options['colors'] is an invalid additional property
options should pass "instanceof" keyword validation
options should match exactly one schema in oneOf

ValidationError: Progress Plugin Invalid Options

options['colors'] is an invalid additional property
options should pass "instanceof" keyword validation
options should match exactly one schema in oneOf

    at validateOptions (..\project-name\node_modules\schema-utils\src\validateOptions.js:31:11)
    at new ProgressPlugin (..\project-name\node_modules\webpack\lib\ProgressPlugin.js:90:3)
    at Object.getCommonConfig (..\project-name\node_modules\@angular-devkit\build-angular\src\angular-cli-files\models\webpack-configs\common.js:123:27)
    at BrowserBuilder.buildWebpackConfig (..\project-name\node_modules\@angular-devkit\build-angular\src\browser\index.js:74:31)
    at DevServerBuilder.buildWebpackConfig (..\project-name\node_modules\@angular-devkit\build-angular\src\dev-server\index.js:111:46)
    at MergeMapSubscriber.check_port_1.checkPort.pipe.operators_1.concatMap [as project] (..\project-name\node_modules\@angular-devkit\build-angular\src\dev-server\index.js:38:40)
    at MergeMapSubscriber._tryNext (..\project-name\node_modules\rxjs\internal\operators\mergeMap.js:65:27)
    at MergeMapSubscriber._next (..\project-name\node_modules\rxjs\internal\operators\mergeMap.js:55:18)
    at MergeMapSubscriber.Subscriber.next (..\project-name\node_modules\rxjs\internal\Subscriber.js:64:18)
    at TapSubscriber._next (..\project-name\node_modules\rxjs\internal\operators\tap.js:62:26)
    at TapSubscriber.Subscriber.next (..\project-name\node_modules\rxjs\internal\Subscriber.js:64:18)
    at MergeMapSubscriber.notifyNext (..\project-name\node_modules\rxjs\internal\operators\mergeMap.js:84:26)
    at InnerSubscriber._next (..\project-name\node_modules\rxjs\internal\InnerSubscriber.js:25:21)
    at InnerSubscriber.Subscriber.next (..\project-name\node_modules\rxjs\internal\Subscriber.js:64:18)
    at ForkJoinSubscriber.notifyComplete (..\project-name\node_modules\rxjs\internal\observable\forkJoin.js:79:25)
    at InnerSubscriber._complete (..\project-name\node_modules\rxjs\internal\InnerSubscriber.js:32:21)

Version
ng --version

     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/

Angular CLI: 6.2.5
Node: 10.13.0
OS: win32 x64
Angular:
...

Package                      Version
------------------------------------------------------
@angular-devkit/architect    0.8.5
@angular-devkit/core         0.8.5
@angular-devkit/schematics   0.8.5
@schematics/angular          0.8.5
@schematics/update           0.8.5
rxjs                         6.2.2
typescript                   2.9.2

npm -v
6.4.1

Having same issue while fresh npm install.
Progress Plugin Invalid Options

options['colors'] is an invalid additional property
options should pass "instanceof" keyword validation
options should match exactly one schema in oneOf

ValidationError: Progress Plugin Invalid Options

options['colors'] is an invalid additional property
options should pass "instanceof" keyword validation
options should match exactly one schema in oneOf

at validateOptions (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/schema-utils/src/validateOptions.js:31:11)
at new ProgressPlugin (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/webpack/lib/ProgressPlugin.js:90:3)
at Object.getCommonConfig (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/@angular-devkit/build-angular/src/angular-cli-files/models/webpack-configs/common.js:123:27)
at BrowserBuilder.buildWebpackConfig (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/@angular-devkit/build-angular/src/browser/index.js:74:31)
at MergeMapSubscriber.rxjs_1.of.pipe.operators_1.concatMap [as project] (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/@angular-devkit/build-angular/src/browser/index.js:31:38)
at MergeMapSubscriber._tryNext (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/@angular-devkit/build-angular/node_modules/rxjs/internal/operators/mergeMap.js:65:27)
at MergeMapSubscriber._next (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/@angular-devkit/build-angular/node_modules/rxjs/internal/operators/mergeMap.js:55:18)
at MergeMapSubscriber.Subscriber.next (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/@angular-devkit/build-angular/node_modules/rxjs/internal/Subscriber.js:64:18)
at TapSubscriber._next (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/@angular-devkit/build-angular/node_modules/rxjs/internal/operators/tap.js:62:26)
at TapSubscriber.Subscriber.next (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/@angular-devkit/build-angular/node_modules/rxjs/internal/Subscriber.js:64:18)
at MergeMapSubscriber.notifyNext (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/@angular-devkit/build-angular/node_modules/rxjs/internal/operators/mergeMap.js:84:26)
at InnerSubscriber._next (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/@angular-devkit/build-angular/node_modules/rxjs/internal/InnerSubscriber.js:25:21)
at InnerSubscriber.Subscriber.next (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/@angular-devkit/build-angular/node_modules/rxjs/internal/Subscriber.js:64:18)
at ForkJoinSubscriber.notifyComplete (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/@angular-devkit/build-angular/node_modules/rxjs/internal/observable/forkJoin.js:79:25)
at InnerSubscriber._complete (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/@angular-devkit/build-angular/node_modules/rxjs/internal/InnerSubscriber.js:32:21)
at InnerSubscriber.Subscriber.complete (/media/shabbir/Projects/SVN/production/hub/trunk/node_modules/@angular-devkit/build-angular/node_modules/rxjs/internal/Subscriber.js:76:18)

Having same with when creating a fresh project by command ng new project _name
Progress Plugin Invalid Options
options['colors'] is an invalid additional property
options should pass "instanceof" keyword validation
options should match exactly one schema in oneOf
ValidationError: Progress Plugin Invalid Options
options['colors'] is an invalid additional property
options should pass "instanceof" keyword validation
options should match exactly one schema in oneOf
at validateOptions

i have the same problem. after create a new project using angular console.
`Progress Plugin Invalid Options
options['colors'] is an invalid additional property
options should pass "instanceof" keyword validation
options should match exactly one schema in oneOf
ValidationError: Progress Plugin Invalid Options
options['colors'] is an invalid additional property
options should pass "instanceof" keyword validation
options should match exactly one schema in oneOf
at validateOptions (C:\dev\NGFullCalendar\node_modules\schema-utils\src\validateOptions.js:31:11)
at new ProgressPlugin (C:\dev\NGFullCalendar\node_modules\webpack\lib\ProgressPlugin.js:90:3)
at Object.getCommonConfig (C:\dev\NGFullCalendar\node_modules\@angular-devkit\build-angular\src\angular-cli-files\models\webpack-configs\common.js:123:27)
at BrowserBuilder.buildWebpackConfig (C:\dev\NGFullCalendar\node_modules\@angular-devkit\build-angular\src\browser\index.js:74:31)
at DevServerBuilder.buildWebpackConfig (C:\dev\NGFullCalendar\node_modules\@angular-devkit\build-angular\src\dev-server\index.js:111:46)
at MergeMapSubscriber.check_port_1.checkPort.pipe.operators_1.concatMap [as project] (C:\dev\NGFullCalendar\node_modules\@angular-devkit\build-angular\src\dev-server\index.js:38:40)
at MergeMapSubscriber._tryNext (C:\dev\NGFullCalendar\node_modules\rxjs\internal\operators\mergeMap.js:65:27)
at MergeMapSubscriber._next (C:\dev\NGFullCalendar\node_modules\rxjs\internal\operators\mergeMap.js:55:18)
at MergeMapSubscriber.Subscriber.next (C:\dev\NGFullCalendar\node_modules\rxjs\internal\Subscriber.js:64:18)
at TapSubscriber._next (C:\dev\NGFullCalendar\node_modules\rxjs\internal\operators\tap.js:62:26)
at TapSubscriber.Subscriber.next (C:\dev\NGFullCalendar\node_modules\rxjs\internal\Subscriber.js:64:18)
at MergeMapSubscriber.notifyNext (C:\dev\NGFullCalendar\node_modules\rxjs\internal\operators\mergeMap.js:84:26)
at InnerSubscriber._next (C:\dev\NGFullCalendar\node_modules\rxjs\internal\InnerSubscriber.js:25:21)
at InnerSubscriber.Subscriber.next (C:\dev\NGFullCalendar\node_modules\rxjs\internal\Subscriber.js:64:18)
at ForkJoinSubscriber.notifyComplete (C:\dev\NGFullCalendar\node_modules\rxjs\internal\observable\forkJoin.js:79:25)
at InnerSubscriber._complete (C:\dev\NGFullCalendar\node_modules\rxjs\internal\InnerSubscriber.js:32:21)

PS C:\dev\NGFullCalendar>`

I have the same issue but only in Ubuntu 16.04. On Windows 10 it works just fine.
Using Node 8.x

This problem does not happen when I do npm install on Windows 10. But on my MacBook the problem really does occur, just as reported above.
I am having to use Windows for now to be able to run my projects in Angular 6. I do not know how to solve this problem and I do not know why it is occurring only on Mac but in Windows not ...

I have the same issue but only in Ubuntu 16.04. On Windows 10 it works just fine.
Using Node 8.x

Windows 10 user here. Using Node 10 LTS

make ng update

I had the same issue on Ubuntu 18.10, Node 10.8.0, NPM 6.2.0 and Angular CLI 6.2.6.
Fixed it with removing colors option on this file:
<EMAIL_ADDRESS>on Line: 123
Changed this:
extraPlugins.push(new ProgressPlugin({ profile: buildOptions.verbose, colors: true }));
to:
extraPlugins.push(new ProgressPlugin({ profile: buildOptions.verbose }));
Hope this help. Cheers.

Can be fixed temporarily by adding this line to package.json
"devDependencies":{
...
"webpack": "4.24.0"
}

Can be fixed temporarily by adding this line to package.json
"devDependencies":{
...
"webpack": "4.24.0"
}

This worked for me.

I had the same issue on Ubuntu 18.10, Node 10.8.0, NPM 6.2.0 and Angular CLI 6.2.6.
Fixed it with removing colors option on this file:
<EMAIL_ADDRESS>on Line: 123
Changed this:
extraPlugins.push(new ProgressPlugin({ profile: buildOptions.verbose, colors: true }));
to:
extraPlugins.push(new ProgressPlugin({ profile: buildOptions.verbose }));
Hope this help. Cheers.

Its works! Thanks!

Can be fixed temporarily by adding this line to package.json
"devDependencies":{
...
"webpack": "4.24.0"
}

then
npm install

This worked for me as well! Thank you!

Can be fixed temporarily by adding this line to package.json
"devDependencies":{
...
"webpack": "4.24.0"
}

then
npm install

Worked for me! Thanks.
Deleted node_modules, then ran npm cache clean --force && npm install && npm install --open --prod

Can be fixed temporarily by adding this line to package.json
"devDependencies":{
...
"webpack": "4.24.0"
}

then
npm install

Can be fixed temporarily by adding this line to package.json
"devDependencies":{
...
"webpack": "4.24.0"
}

then
npm install
This worked for me too. Thanks

Can be fixed temporarily by adding this line to package.json
"devDependencies":{
...
"webpack": "4.24.0"
}

then
npm install

This worked for me. Thanks

Looks like the validation logic along with that validateOptions call in ProgressPlugin, was just added to<EMAIL_ADDRESS>here:
https://github.com/webpack/webpack/commit/aa8cb0fab74160caca59e1cfdd0758c0346313b0#diff-5d04086b4365820a456acec22b899956R86

ng update --all did the work!

"webpack": "4.24.0"

Worked for me! thanks!!

Can be fixed temporarily by adding this line to package.json
"devDependencies":{
...
"webpack": "4.24.0"
}

then
npm install

Worked for me...

ng update --all did the work!

works like a charm

Heya, we have a fix incoming in https://github.com/angular/angular-cli/pull/12870. As far as I can tell, this only affects 6.2.x. Neither 6.1.x nor 7.x should be affected.

I had the same issue on Ubuntu 18.10, Node 10.8.0, NPM 6.2.0 and Angular CLI 6.2.6.
Fixed it with removing colors option on this file:
<EMAIL_ADDRESS>on Line: 123
Changed this:
extraPlugins.push(new ProgressPlugin({ profile: buildOptions.verbose, colors: true }));
to:
extraPlugins.push(new ProgressPlugin({ profile: buildOptions.verbose }));
Hope this help. Cheers.

It works! Thanks a lot :)

Fixed in https://github.com/angular/angular-cli/releases/tag/v6.2.7

`Fixed it with removing colors option on this file:
<EMAIL_ADDRESS>on Line: 123
Changed this:
extraPlugins.push(new ProgressPlugin({ profile: buildOptions.verbose, colors: true }));
to:
extraPlugins.push(new ProgressPlugin({ profile: buildOptions.verbose }));`
This worked for me as well -- thanks!

Updating to Angular CLI 6.2.7 fixed it for me.

"webpack": "4.24.0"

Did the job also for me.

ng update --all did the work!

Worked for me as well.

"webpack": "4.24.0"

worked for me! =) Thanks!

I am facing same issue. None of the above solutions working for me . Any other suggestions ?

<EMAIL_ADDRESS>alone didn't make it, had to upgrade one more:
    "@angular-devkit/build-angular": "0.10.6",
    "@angular/cli": "6.2.7",

Another workaround is ng serve --progress=false until you can update angular-cli

I am facing same issue. None of the above solutions working for me . Any other suggestions ?
try do it :npm uninstall webapck,
then Can be fixed temporarily by adding this line to package.json

"devDependencies":{
...
"webpack": "4.24.0"
}

then
npm install

Same problem with me (Angular 6.1.10 and Angular Cli 6.2.7)
Remove package-lock.json and npm cache clean --force && npm install --force
Worked for me :)

update from
    "@angular-devkit/build-angular": "0.8.4",

to
    "@angular-devkit/build-angular": "0.11.0",

ng update @angular/cli worked for me


Cascade 2 Levels

Hello
Is there anyway to limit the cascading feature? So cascading would apply only to levels 1,2,3 but not 4?
Thank you for your time,
Arthur

What do you mean by cascading? If you mean to render all children flat after the fourth level - no that is not possible, you should convert your data to match what you need to achieve.
If I misunderstood - please clarify?

I'm referring to the checkbox plugin. By default checking one of the boxes will also check all the children & grandchildren, etc. of that node. I want it to only check the children, not the grandchildren or any deeper levels.


Partnership changes for 21.02

With recent changes to the Open Targets partnership, we need to update the following parts of the Angular Platform codebase:

[x] Home page "Partners" section
[x] "Partners" section in Footer
[x] webapp repo LICENCE file

Ticket closed as work completed and merged into 3.21.0 release


Issues with build on github actions

My build here shows the issues: https://github.com/singerbj/bag-tag/actions/runs/8810082004/job/24181887449#step:6:254
Run ./Godot.app/Contents/MacOS/Godot --headless --path . --quit-after 10
  ./Godot.app/Contents/MacOS/Godot --headless --path . --quit-after 10
  shell: /bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    PROJECT_NAME: BagTag
    IOS_APPLE_ID_USERNAME: ***
    IOS_APPLE_ID_PASSWORD: ***
    BUILD_CERTIFICATE_BASE64: ***

    P12_PASSWORD: ***
    BUILD_PROVISION_PROFILE_BASE64: ***
    KEYCHAIN_PASSWORD: ***
    VERSION: 0.0.1

2024-04-24 02:24:56.052 Godot[4178:17406] *** +[NSString stringWithUTF8String:]: NULL cString
Godot Engine v4.2.1.stable.official.b09f793f5 - https://godotengine.org

SCRIPT ERROR: Parse Error: Could not find base class "GameObject".
          at: GDScript::reload (res://scripts/people.gd:1)
ERROR: Failed to load script "res://scripts/people.gd" with error "Parse error".
   at: load (modules/gdscript/gdscript.cpp:2788)
ERROR: Unable to open file: res://.godot/imported/Bina.PNG-fbd58b18277f6de557dd4[238](https://github.com/singerbj/bag-tag/actions/runs/8810082004/job/24181887449#step:6:241)661bdab5.ctex.
   at: _load_data (scene/resources/compressed_texture.cpp:41)
ERROR: Failed loading resource: res://.godot/imported/Bina.PNG-fbd58b18277f6de557dd4238661bdab5.ctex. Make sure resources have been imported by opening the project in the editor at least once.
   at: _load (core/io/resource_loader.cpp:275)
ERROR: Failed loading resource: res://assets/art/Bina.PNG. Make sure resources have been imported by opening the project in the editor at least once.
   at: _load (core/io/resource_loader.cpp:275)
ERROR: res://scenes/people/Bina.tscn:11 - Parse Error: [ext_resource] referenced non-existent resource at: res://assets/art/Bina.PNG
   at: _parse_ext_resource (scene/resources/resource_format_text.cpp:163)
ERROR: Unable to open file: res://.godot/imported/Ben.PNG-c3ef107ad193f5feaa8dee7920f0d2ca.ctex.
   at: _load_data (scene/resources/compressed_texture.cpp:41)
ERROR: Failed loading resource: res://.godot/imported/Ben.PNG-c3ef107ad193f5feaa8dee7920f0d2ca.ctex. Make sure resources have been imported by opening the project in the editor at least once.
   at: _load (core/io/resource_loader.cpp:275)
ERROR: Failed loading resource: res://assets/art/Ben.PNG. Make sure resources have been imported by opening the project in the editor at least once.
   at: _load (core/io/resource_loader.cpp:275)
ERROR: res://scenes/people/Ben.tscn:11 - Parse Error: [ext_resource] referenced non-existent resource at: res://assets/art/Ben.PNG
   at: _parse_ext_resource (scene/resources/resource_format_text.cpp:163)
ERROR: Unable to open file: res://.godot/imported/Dad.PNG-adfb80ddf5e529491fa52e05ad0a55f1.ctex.
   at: _load_data (scene/resources/compressed_texture.cpp:41)
ERROR: Failed loading resource: res://.godot/imported/Dad.PNG-adfb80ddf5e529491fa52e05ad0a55f1.ctex. Make sure resources have been imported by opening the project in the editor at least once.
   at: _load (core/io/resource_loader.cpp:275)
ERROR: Failed loading resource: res://assets/art/Dad.PNG. Make sure resources have been imported by opening the project in the editor at least once.
   at: _load (core/io/resource_loader.cpp:275)
ERROR: res://scenes/people/Dad.tscn:11 - Parse Error: [ext_resource] referenced non-existent resource at: res://assets/art/Dad.PNG
   at: _parse_ext_resource (scene/resources/resource_format_text.cpp:163)
ERROR: Unable to open file: res://.godot/imported/Gigi.PNG-33e96f855f16[268](https://github.com/singerbj/bag-tag/actions/runs/8810082004/job/24181887449#step:6:272)3ac181c1a6faea79e.ctex.
   at: _load_data (scene/resources/compressed_texture.cpp:41)
ERROR: Failed loading resource: res://.godot/imported/Gigi.PNG-33e96f855f162683ac181c1a6faea79e.ctex. Make sure resources have been imported by opening the project in the editor at least once.
   at: _load (core/io/resource_loader.cpp:275)
ERROR: Failed loading resource: res://assets/art/Gigi.PNG. Make sure resources have been imported by opening the project in the editor at least once.
   at: _load (core/io/resource_loader.cpp:275)
ERROR: res://scenes/people/Gigi.tscn:11 - Parse Error: [ext_resource] referenced non-existent resource at: res://assets/art/Gigi.PNG
   at: _parse_ext_resource (scene/resources/resource_format_text.cpp:163)
ERROR: Unable to open file: res://.godot/imported/MagJake.PNG-5[270](https://github.com/singerbj/bag-tag/actions/runs/8810082004/job/24181887449#step:6:274)785fa7df55f24f3d0dd73c8d9dc8.ctex.
   at: _load_data (scene/resources/compressed_texture.cpp:41)
ERROR: Failed loading resource: res://.godot/imported/MagJake.PNG-5270785fa7df55f24f3d0dd73c8d9dc8.ctex. Make sure resources have been imported by opening the project in the editor at least once.
   at: _load (core/io/resource_loader.cpp:275)
ERROR: Failed loading resource: res://assets/art/MagJake.PNG. Make sure resources have been imported by opening the project in the editor at least once.
   at: _load (core/io/resource_loader.cpp:[275](https://github.com/singerbj/bag-tag/actions/runs/8810082004/job/24181887449#step:6:279))
ERROR: res://scenes/people/MagJake.tscn:11 - Parse Error: [ext_resource] referenced non-existent resource at: res://assets/art/MagJake.PNG
   at: _parse_ext_resource (scene/resources/resource_format_text.cpp:163)
SCRIPT ERROR: Parse Error: Could not find base class "GameObject".
          at: GDScript::reload (res://scripts/backpack.gd:1)

For whatever reason Godot won't build my project on the runner, despite it working locally and in my other actions that build windows, linux, web, and android. Any guidance on why this might be? Perhaps someone dealt with a similar issue already?

I forgot to update the tag v4 to the latest version v4.0.3
Now both tags are aligned and the build should work. You can simply restart the action and see if i builds correctly now.

Thanks for doing that, but for whatever reason I am now seeing that my export yields different failing results in each way I run it and then somehow corrupts my Godot project when I do it from my editor. I think I need to dig into it more - closing this for now.

met the same error: https://github.com/dionsradio/shanhai_merge/actions/runs/10993045802/job/30518689885
name: iOS upload

on:
  # push:
  #   branches:
  #     - 'main'
  #   paths:
  #     - "version"
  #     - ".github/workflows/upload-ios.yml"
  #     - "exportOptions.plist"
  #     - "export_presets.ios.example"
  workflow_dispatch:

env:
  PROJECT_NAME: Merge
  WORKING_DIRECTORY: game
  IOS_APPSTORE_TEAM_ID: ${{ secrets.IOS_APPSTORE_TEAM_ID }}
  IOS_APPLE_ID_USERNAME: ${{ secrets.IOS_APPLE_ID_USERNAME }}
  IOS_APPLE_ID_PASSWORD: ${{ secrets.IOS_APPLE_ID_PASSWORD }}

jobs:
  deploy:
    runs-on: macos-latest
    steps:
      - name: Checkout source code
        uses: actions/checkout@v4
      - name: Export and upload to Testflight
        uses: dulvui/godot-ios-upload@v4
        with:
          working-directory: $WORKING_DIRECTORY
          project-name: $PROJECT_NAME
          godot-version: 4.3
          provision-profile-path: ~/Library/MobileDevice/Provisioning\ Profiles/build_pp.mobileprovision
          apple-id-username: $IOS_APPLE_ID_USERNAME
          apple-id-password: $IOS_APPLE_ID_PASSWORD

@willin I can see in your screenshot the error about a missing path ../shanhai-merge-ios/ so you might need to create the directory first in the action. I guess this is the export path you defined in the export_preset.cfg file?
Creating the directory before the export could be the solution.
- name: create export directory
  run: mkdir shanhai-merge-ios

I can also see that you still use my default working directory game. Is your project also setup like this, with the project file in the game directoy?

Godot Engine v4.3.stable.official.77dcf97d8 - https://godotengine.org/
Error: Process completed with exit code 124.

WARNING: Export Icons: Icon (icons/app_store_1024x1024) must be opaque.
     at: add_message (editor/export/editor_export_platform.h:179)
Exporting launch screen storyboard
Exporting additional assets
	export: step 2: Code-signing dylibs
	export: step 3: Making .xcarchive
xcodebuild (.xcarchive):
Command line invocation:
    /Applications/Xcode_15.4.app/Contents/Developer/usr/bin/xcodebuild -project ../shanhai-merge-ios/shanghai-merge.xcodeproj -scheme shanghai-merge -sdk iphoneos -configuration Release -destination generic/platform=iOS archive -allowProvisioningUpdates -archivePath ../shanhai-merge-ios/shanghai-merge.xcarchive
User defaults from command line:
    IDEArchivePathOverride = /Users/runner/work/shanhai_merge/shanhai_merge/shanhai-merge-ios/shanghai-merge.xcarchive
    IDEPackageSupportUseBuiltinSCM = YES
Build settings from command line:
    SDKROOT = iphoneos17.5
Prepare packages
ComputeTargetDependencyGraph
note: Building targets in dependency order
note: Target dependency graph (1 target)
    Target 'shanghai-merge' in project 'shanghai-merge' (no dependencies)
GatherProvisioningInputs
CreateBuildDescription
ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -v -E -dM -arch arm64 -isysroot /Applications/Xcode_15.4.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS17.5.sdk -x c++ -c /dev/null
ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -v -E -dM -isysroot /Applications/Xcode_15.4.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS17.5.sdk -x c -c /dev/null
ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/usr/bin/ibtool --version --output-format xml1
ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/usr/bin/actool --print-asset-tag-combinations --output-format xml1 /Users/runner/work/shanhai_merge/shanhai_merge/shanhai-merge-ios/shanghai-merge/Images.xcassets
ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/usr/bin/actool --version --output-format xml1
ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -v -E -dM -arch arm64 -isysroot /Applications/Xcode_15.4.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS17.5.sdk -x c -c /dev/null
ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld -version_details
Build description signature: [291](https://github.com/dionsradio/shanhai_merge/actions/runs/10993600322/job/30520410232#step:4:297)c92e39a714b22e6ff899b6c257526
Build description path: /Users/runner/Library/Developer/Xcode/DerivedData/shanghai-merge-dvvdodvsjebmvfesuxkiqcmqmfdg/Build/Intermediates.noindex/ArchiveIntermediates/shanghai-merge/IntermediateBuildFilesPath/XCBuildData/291c92e39a714b22e6ff899b6c257526.xcbuilddata
/Users/runner/work/shanhai_merge/shanhai_merge/shanhai-merge-ios/shanghai-merge.xcodeproj: error: shanghai-merge has conflicting provisioning settings. shanghai-merge is automatically signed for development, but a conflicting code signing identity iPhone Distribution has been manually specified. Set the code signing identity value to "Apple Development" in the build settings editor, or switch to manual signing in the Signing & Capabilities editor. (in target 'shanghai-merge' from project 'shanghai-merge')
** ARCHIVE FAILED **
ERROR: Xcode Build: Xcode project build failed, see editor log for details.
export: end
   at: add_message (editor/export/editor_export_platform.h:182)
ERROR: Project export for preset "iOS" failed.
   at: _fs_changed (editor/editor_node.cpp:1028)
Error: Process completed with exit code 1.

yes. but still errored

@willin now the problem is related to the signing of the application. It seems that you have set automatic code signing in XCode ore similar, and in my action manual codes sign is used.
I don't have much experience with codesigning, but I see 2 possible options:

you set manual codesign
you copy the steps of my action manually into your actipn file, so you can change the archive step to automatic signing

I might add an option to choose between manual/automatic signing in a later version, as soon I know morr about the differences

Sorry to bother you again.
name: iOS upload

on:
  # push:
  #   branches:
  #     - 'main'
  #   paths:
  #     - "version"
  #     - ".github/workflows/upload-ios.yml"
  #     - "exportOptions.plist"
  #     - "export_presets.ios.example"
  workflow_dispatch:

env:
  PROJECT_NAME: Merge(Mountain and Sea)
  WORKING_DIRECTORY: game
  IOS_APPSTORE_TEAM_ID: ${{ secrets.IOS_APPSTORE_TEAM_ID }}
  IOS_APPLE_ID_USERNAME: ${{ secrets.IOS_APPLE_ID_USERNAME }}
  IOS_APPLE_ID_PASSWORD: ${{ secrets.IOS_APPLE_ID_PASSWORD }}
  BUILD_CERTIFICATE_BASE64: ${{ secrets.IOS_BUILD_CERTIFICATE_BASE64 }}
  P12_PASSWORD: ${{ secrets.IOS_P12_PASSWORD }}
  BUILD_PROVISION_PROFILE_BASE64: ${{ secrets.IOS_PROVISION_PROFILE_BASE64 }}
  KEYCHAIN_PASSWORD: ${{ secrets.IOS_KEYCHAIN_PASSWORD }}

jobs:
  deploy:
    runs-on: macos-latest
    steps:
      - name: Checkout source code
        uses: actions/checkout@v4

      - name: mkdir for iOS
        run: mkdir shanhai-merge-ios

      # https://docs.github.com/en/actions/deployment/deploying-xcode-applications/installing-an-apple-certificate-on-macos-runners-for-xcode-development
      - name: Install the Apple certificate and provisioning profile
        run: |
          # create variables
          CERTIFICATE_PATH=$RUNNER_TEMP/build_certificate.p12
          PP_PATH=$RUNNER_TEMP/build_pp.mobileprovision
          KEYCHAIN_PATH=$RUNNER_TEMP/app-signing.keychain-db

          # import certificate and provisioning profile from secrets
          echo -n "$BUILD_CERTIFICATE_BASE64" | base64 --decode -o $CERTIFICATE_PATH
          echo -n "$BUILD_PROVISION_PROFILE_BASE64" | base64 --decode -o $PP_PATH

          # create temporary keychain
          security create-keychain -p "$KEYCHAIN_PASSWORD" $KEYCHAIN_PATH
          security set-keychain-settings -lut 21600 $KEYCHAIN_PATH
          security unlock-keychain -p "$KEYCHAIN_PASSWORD" $KEYCHAIN_PATH

          # import certificate to keychain
          security import $CERTIFICATE_PATH -P "$P12_PASSWORD" -A -t cert -f pkcs12 -k $KEYCHAIN_PATH
          security list-keychain -d user -s $KEYCHAIN_PATH

          # apply provisioning profile
          mkdir -p ~/Library/MobileDevice/Provisioning\ Profiles
          cp $PP_PATH ~/Library/MobileDevice/Provisioning\ Profiles

      - name: Set version and short version
        run: echo "VERSION=$(cat version | cut -f1 -d-)" >> $GITHUB_ENV

      - name: Create and config export_presets.cfg
        run: |
          cp game/export_presets.ios.example game/export_presets.cfg
          sed -i -e "s|VERSION_CODE|$VERSION|g" $WORKING_DIRECTORY/export_presets.cfg
          sed -i -e "s|VERSION_NAME|$VERSION|g" $WORKING_DIRECTORY/export_presets.cfg
          sed -i -e "s|IOS_SHORT_VERSION|$VERSION|g" $WORKING_DIRECTORY/export_presets.cfg
          sed -i -e "s|IOS_VERSION|$VERSION|g" $WORKING_DIRECTORY/export_presets.cfg
          sed -i -e "s|IOS_APPSTORE_TEAM_ID|$IOS_APPSTORE_TEAM_ID|g" $WORKING_DIRECTORY/export_presets.cfg

      - name: Export and upload to Testflight
        uses: dulvui/godot-ios-upload@v4
        with:
          working-directory: $WORKING_DIRECTORY
          project-name: $PROJECT_NAME
          godot-version: 4.3
          provision-profile-path: ~/Library/MobileDevice/Provisioning\ Profiles/build_pp.mobileprovision
          apple-id-username: $IOS_APPLE_ID_USERNAME
          apple-id-password: $IOS_APPLE_ID_PASSWORD

almost the same, but uses dulvui/godot-ios-upload@v4 not v1.
==> Summary
🍺  /opt/homebrew/Cellar/coreutils/9.5: 477 files, 13.5MB
Run gtimeout 60 ./Godot.app/Contents/MacOS/Godot --headless --path $WORKING_DIRECTORY -e
  gtimeout 60 ./Godot.app/Contents/MacOS/Godot --headless --path $WORKING_DIRECTORY -e
  shell: /bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    PROJECT_NAME: Merge(Mountain and Sea)
    WORKING_DIRECTORY: game
    IOS_APPSTORE_TEAM_ID: ***
    IOS_APPLE_ID_USERNAME: ***
    IOS_APPLE_ID_PASSWORD: ***
    BUILD_CERTIFICATE_BASE64: ***
    P12_PASSWORD: ***
    BUILD_PROVISION_PROFILE_BASE64: ***

    KEYCHAIN_PASSWORD: ***
    VERSION: 0.0.1

Godot Engine v4.3.stable.official.77dcf97d8 - https://godotengine.org/
ERROR: Xcode Build: Xcode project build failed, see editor log for details.
xcodebuild (.xcarchive):
   at: add_message (editor/export/editor_export_platform.h:182)
Command line invocation:
ERROR: Project export for preset "iOS" failed.
    /Applications/Xcode_15.4.app/Contents/Developer/usr/bin/xcodebuild -project ../shanhai-merge-ios/shanghai-merge.xcodeproj -scheme shanghai-merge -sdk iphoneos -configuration Release -destination generic/platform=iOS archive -allowProvisioningUpdates -archivePath ../shanhai-merge-ios/shanghai-merge.xcarchive
   at: _fs_changed (editor/editor_node.cpp:1028)
User defaults from command line:
    IDEArchivePathOverride = /Users/runner/work/shanhai_merge/shanhai_merge/shanhai-merge-ios/shanghai-merge.xcarchive
    IDEPackageSupportUseBuiltinSCM = YES
Build settings from command line:
    SDKROOT = iphoneos17.5
Prepare packages
ComputeTargetDependencyGraph
note: Building targets in dependency order
note: Target dependency graph (1 target)
    Target 'shanghai-merge' in project 'shanghai-merge' (no dependencies)
GatherProvisioningInputs
CreateBuildDescription
ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -v -E -dM -arch arm64 -isysroot /Applications/Xcode_15.4.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS17.5.sdk -x c++ -c /dev/null
ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -v -E -dM -isysroot /Applications/Xcode_15.4.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS17.5.sdk -x c -c /dev/null
ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/usr/bin/ibtool --version --output-format xml1
ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/usr/bin/actool --print-asset-tag-combinations --output-format xml1 /Users/runner/work/shanhai_merge/shanhai_merge/shanhai-merge-ios/shanghai-merge/Images.xcassets
ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/usr/bin/actool --version --output-format xml1
ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -v -E -dM -arch arm64 -isysroot /Applications/Xcode_15.4.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS17.5.sdk -x c -c /dev/null
ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld -version_details
Build description signature: 346dd4b09efb0db14330cf2400ff0dd6
Build description path: /Users/runner/Library/Developer/Xcode/DerivedData/shanghai-merge-dvvdodvsjebmvfesuxkiqcmqmfdg/Build/Intermediates.noindex/ArchiveIntermediates/shanghai-merge/IntermediateBuildFilesPath/XCBuildData/346dd4b09efb0db14330cf2400ff0dd6.xcbuilddata
/Users/runner/work/shanhai_merge/shanhai_merge/shanhai-merge-ios/shanghai-merge.xcodeproj: error: shanghai-merge has conflicting provisioning settings. shanghai-merge is automatically signed for development, but a conflicting code signing identity iPhone Distribution has been manually specified. Set the code signing identity value to "Apple Development" in the build settings editor, or switch to manual signing in the Signing & Capabilities editor. (in target 'shanghai-merge' from project 'shanghai-merge')
** ARCHIVE FAILED **
export: end
Error: Process completed with exit code 1.

@willin everything seems correct to me. There might be a problem with the certificate and profiles itself. Did you convert them to base 64 etc. and fill username, password, team id and p12 password to github secrets?
I saw that you have also set application/code_sign_identity_debug="iPhone Developer", try with an empty value like this
application/code_sign_identity_debug=""
Here an exports presets config that worked for me with Godot 4, but I haven't yet tried with 4.3, but nothing should change:
https://github.com/dulvui/futsal-manager/blob/2c6c79ee25c047e01a7d11bc9086756796f0a607/game/export_presets.html.example

yes, in bash history:
❯ base64 -i Certificates.p12| pbcopy
❯ base64 -i  MergeDistribution.mobileprovision| pbcopy

and i also tried code_sign_identity_debug with Apple Developer and empty. all the same error

That seems correct to me. From experience I can tell that iOS build.with Github Actions can be really tricky, and error messages some times are simply wrong and misleading.
Do yoy have a macOS device where you can try everything locally? There you might find easier the problem, and save precious github action macos runner minutes.
It could also be something completly else, like missing AppIcons or missing exportOptions.plist file
Here an example for the plist file, if you haven't one already.
https://github.com/dulvui/futsal-manager/blob/2c6c79ee25c047e01a7d11bc9086756796f0a607/exportOptions.plist

I just saw that the exportsOptions.plist filemis missing in the readme, I'll add it as soon I have access to my laptop.

<key>provisioningProfiles</key>
    <dict>
        <key>com.merge</key>
        <string>com.merge</string>
    </dict>
    <key>signingCertificate</key>
    <string>iPhone Distribution</string>

i wonder whether signingCertificate cannot be iPhone Distribution?
/Users/runner/work/shanhai_merge/shanhai_merge/shanghai-merge.xcodeproj: error: shanghai-merge has conflicting provisioning settings. shanghai-merge is automatically signed for development, but a conflicting code signing identity iPhone Distribution has been manually specified. Set the code signing identity value to "Apple Development" in the build settings editor, or switch to manual signing in the Signing & Capabilities editor. (in target 'shanghai-merge' from project 'shanghai-merge')

If I remeber right, iPhone distribution is deprecated and works only with older versions of XCode.
Now Apple Distribution should be used

Regenerate Apple Distribution cert then:
/Users/runner/work/shanhai_merge/shanhai_merge/shanghai-merge.xcodeproj: error: shanghai-merge has conflicting provisioning settings. shanghai-merge is automatically signed for development, but a conflicting code signing identity iPhone Distribution has been manually specified. Set the code signing identity value to "Apple Development" in the build settings editor, or switch to manual signing in the Signing & Capabilities editor. (in target 'shanghai-merge' from project 'shanghai-merge')
** ARCHIVE FAILED **

then edit export_presets.ios.example:
application/code_sign_identity_debug="Apple Distribution"
application/code_sign_identity_release="Apple Distribution"

error changes:
Export: ARM64 simulator library, generating from device library.
WARNING: Export Icons: Icon (icons/app_store_1024x1024) must be opaque.
     at: add_message (editor/export/editor_export_platform.h:179)
Exporting launch screen storyboard
Exporting additional assets
	export: step 2: Code-signing dylibs
	export: step 3: Making .xcarchive
xcodebuild (.xcarchive):
ERROR: Xcode Build: Xcode project build failed, see editor log for details.
Command line invocation:
   at: add_message (editor/export/editor_export_platform.h:182)
    /Applications/Xcode_15.4.app/Contents/Developer/usr/bin/xcodebuild -project ../shanghai-merge.xcodeproj -scheme shanghai-merge -sdk iphoneos -configuration Release -destination generic/platform=iOS archive -allowProvisioningUpdates -archivePath ../shanghai-merge.xcarchive

User defaults from command line:
    IDEArchivePathOverride = /Users/runner/work/shanhai_merge/shanhai_merge/shanghai-merge.xcarchive
    IDEPackageSupportUseBuiltinSCM = YES

Build settings from command line:
    SDKROOT = iphoneos17.5

Prepare packages

ComputeTargetDependencyGraph
note: Building targets in dependency order
note: Target dependency graph (1 target)
    Target 'shanghai-merge' in project 'shanghai-merge' (no dependencies)

GatherProvisioningInputs

CreateBuildDescription

ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -v -E -dM -arch arm64 -isysroot /Applications/Xcode_15.4.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS17.5.sdk -x c++ -c /dev/null

ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -v -E -dM -isysroot /Applications/Xcode_15.4.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS17.5.sdk -x c -c /dev/null

ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/usr/bin/ibtool --version --output-format xml1

ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/usr/bin/actool --print-asset-tag-combinations --output-format xml1 /Users/runner/work/shanhai_merge/shanhai_merge/shanghai-merge/Images.xcassets

ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/usr/bin/actool --version --output-format xml1

ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -v -E -dM -arch arm64 -isysroot /Applications/Xcode_15.4.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS17.5.sdk -x c -c /dev/null

ExecuteExternalTool /Applications/Xcode_15.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld -version_details

Build description signature: 12eaa5cd650e6eeac40d448ddf7d0068
Build description path: /Users/runner/Library/Developer/Xcode/DerivedData/shanghai-merge-cdkuvhkuaahpeyhhsbaxitdjatbf/Build/Intermediates.noindex/ArchiveIntermediates/shanghai-merge/IntermediateBuildFilesPath/XCBuildData/12eaa5cd650e6eeac40d448ddf7d0068.xcbuilddata
/Users/runner/work/shanhai_merge/shanhai_merge/shanghai-merge.xcodeproj: error: "shanghai-merge" requires a provisioning profile. Select a provisioning profile in the Signing & Capabilities editor. (in target 'shanghai-merge' from project 'shanghai-merge')
** ARCHIVE FAILED **

ERROR: Project export for preset "iOS" failed.
   at: _fs_changed (editor/editor_node.cpp:1028)
export: end
Error: Process completed with exit code 1.

I have empty strings for both here
application/code_sign_identity_debug=""
application/code_sign_identity_release=""

Did you update also in exportsOptions.plist?
Tomorrow I will try to build my game with Godot 4.3 to make sure it works.

when this error requires a provisioning profile.
They have to be the exact same string

TEAM_ID.com.dionsradio.shanhaimerge should i add the team id prefix?

yes, the names must exactly match. I also had this issue once, took me days to understand what was the problem.
Let's hope the best 🤞

emmmm no use....

I tried now with my game and it worked, just the upload step didn't success, becasue i didn't increment the version name. But the export and build worked.
https://github.com/dulvui/futsal-manager/actions/runs/11033627841
The only difference I can still see here, is that I have empty strings here and not Apple Distribution.
application/code_sign_identity_debug=""
application/code_sign_identity_release=""

/Users/runner/work/shanhai_merge/shanhai_merge/shanghai-merge.xcodeproj: error: shanghai-merge has conflicting provisioning settings. shanghai-merge is automatically signed for development, but a conflicting code signing identity iPhone Distribution has been manually specified. Set the code signing identity value to "Apple Development" in the build settings editor, or switch to manual signing in the Signing & Capabilities editor. (in target 'shanghai-merge' from project 'shanghai-merge')

Sorry I don't know what could be still wrong.

@willin I just retired now to export with Godot 4.3 and I got the same error as you. I checked now and it seems that Godot can now archive and create the .ipa file by itself, so no manual archive is needed anymore.
I'll try to get it working on my side, and then let you know as soon as it works.

@willin Now I was finally able to build correctly a game built with Godot 4.3!
The export settings changed in version 4.3, and Godot can now do the building of the App directly, so no manual archiving etc. is needed anymore. So I created a new action called godot-ios-export, that only exports the game using Godot itself.
The exportOptions.plist file is no longer needed, and the exports_preset.cfg file had also some changes. Also the version can now be set in project settings and the value will be then taken from there.
You can find an working example here.
https://github.com/dulvui/futsal-manager/blob/main/.github/workflows/upload-ios.yml
Let me know if it finally worked!

i'll try.


WIP: add zendesk integration

Address: PURCHASE-2442
In general this PR adds Zendesk integration to Artwork pages (For Sale, BNMO, Inquiry) and the checkout flow. To accomplish it does a few things:

Adds react-zendesk to create a wrapper around the Zendesk widget.
Adds a new environment variable to store the Zendesk key.
Render the Zendesk component on the artwork page and within the BNMO flow.

TODO:

[ ] This doesn't yet take in account the display for works above 10kUSD or the equivalent in EUR, GBP etc. Acceptance Criteria. I know currency conversion at Artsy previously has been a bit tricky. Do we have any helpers for this?
[ ] Add the ZENDESK_KEY to the staging and production environments.

Artwork Page:

Order Page:

It's worth noting that library returns a script tag with the async property set to true. Ideally we might want to use defer to ensure the script doesn't block parsing. I'm not seeing a simple way to pass deter (i.e. I don't think the library supports it). This might be a legitimate reason to pull this out of the head and move it to the body. Wondering what others think?

It's worth noting that library returns a script tag with the async property set to true. Ideally we might want to use defer to ensure the script doesn't block parsing.

This would be a great PR.

@damassi great minds! I was thinking about tackling that if I got some time this weekend. Seems like a small enough change.

@starsirius do you think the CI build failure is related to environment variables?
Also do you have additional feedback for this PR?

The CI failure was probably because this PR was submitted from your fork instead of artsy repo.
Looks like there are 2 todos:

Appears after 20s on the page (I'm guessing this could probably be configured in Zendesk dashboard).
Any work above 10kUSD or the equivalent in EUR, GBP etc.

Since we use the environment variable to guard the release of this feature, I'm 👍  to separate them out to another PR if you prefer that.

Ahh that's right I think I recall running into this on my last force PR too. So I think I resolve this by creating a new PR that is pushed to upstream versus origin?

But it looks like this is now getting passed the build image failure I was seeing earlier. If the build is green can we merge this and address the remaining todo items in a follow up PR? I can create a ticket.

Yeah it seems the branch pushed to the artsy repo was linked to this PR (clicking the green ✔  next to the last commit takes me to that branch instead of your fork).


unable to train logistic regression model

I'm unable to train lr models due to out of index error in following code
`
models_per_condition = []
    for plane in planes:
        checkpoint_pattern = glob(f'{models_dir}/*{plane}*{condition}*.pt')
        checkpoint_path = sorted(checkpoint_pattern)[-1]
        checkpoint = torch.load(checkpoint_path, map_location=device)`

im getting error at sorted(checkpoint_pattern)[-1]
I ran the previous scripts as directed in readme of this repo.

problem is in glob i.e. file reading from directory.
its not reading files from directory.

I'm unable to train lr models due to out of index error in following code ` models_per_condition = []
    for plane in planes:
        checkpoint_pattern = glob(f'{models_dir}/*{plane}*{condition}*.pt')
        checkpoint_path = sorted(checkpoint_pattern)[-1]
        checkpoint = torch.load(checkpoint_path, map_location=device)`

im getting error at sorted(checkpoint_pattern)[-1] I ran the previous scripts as directed in readme of this repo.

I'm unable to train lr models due to out of index error in following code ` models_per_condition = []
    for plane in planes:
        checkpoint_pattern = glob(f'{models_dir}/*{plane}*{condition}*.pt')
        checkpoint_path = sorted(checkpoint_pattern)[-1]
        checkpoint = torch.load(checkpoint_path, map_location=device)`

im getting error at sorted(checkpoint_pattern)[-1] I ran the previous scripts as directed in readme of this repo.

You can try to move all .pt files under one folder.


GO 1.16 Program do not compile on raspberry pi

What version of Go are you using (go version)?

$ go version
go version go1.16 linux/arm

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
$ go env
GO111MODULE=""
GOARCH="arm"
GOBIN=""
GOCACHE="/home/pi/.cache/go-build"
GOENV="/home/pi/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="arm"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/pi/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/pi/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_arm"
GOVCS=""
GOVERSION="go1.16"
GCCGO="gccgo"
GOARM="6"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/pi/vaccin/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -marm -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build4049214872=/tmp/go-build -gno-record-gcc-switches"
What did you do?
go build my_program.go
What did you expect to see?
to finish without compilation error. I compile the same program on Unbuntu without any problem
What did you see instead?
# runtime
/usr/local/go/src/runtime/mgcsweepbuf.go:87:80: memstats.gc_sys undefined (type mstats has no field or method gc_sys)
/usr/local/go/src/runtime/mgcsweepbuf.go:106:102: memstats.gc_sys undefined (type mstats has no field or method gc_sys)
/usr/local/go/src/runtime/stubs32.go:14:6: call16 redeclared in this block
previous declaration at /usr/local/go/src/runtime/stubs.go:287:55

This issue is often caused by a corrupt Go installation. Did you delete the previous version of go before unpacking 1.16?

I reinstalled Go 1.16 and it is working now. Thanks for the quick reply.

@toothrot @dmitshur this class of error occurs after every point release. Do you think the installation instructions could be improved to highlight that previous installations should be removed first?

We can try, but we'll need to narrow down where the source of the problem was.
@jpmieville Which installation instructions did you follow when you ran into this issue? Was it https://golang.org/doc/install or somewhere else?

It always happens if you don't remove the existing installation first before installing the new one. The installation page just mentions a single line

If you have a previous version of Go installed, be sure to remove it before installing another.

I think it should be made part of the numbered list for better visibility so that users always go through it without fail.

Thank you for the feedback. I've filed #44426 about improving the Linux installation instructions.

We can try, but we'll need to narrow down where the source of the problem was.
@jpmieville Which installation instructions did you follow when you ran into this issue? Was it https://golang.org/doc/install or somewhere else?

On my raspberry pi, I used the following shell script
#!/bin/bash
if [ -d "/usr/local/go/bin" ] ; then
rm "/usr/local/go/bin"
echo "/usr/local/go/bin deleted"
fi
export GOLANG="$(curl https://golang.org/dl/|grep armv6l|grep -v beta|head -1|aw
k -F> {'print $3'}|awk -F< {'print $1'})"
echo "download $GOLANG"
wget https://golang.org/dl/$GOLANG
echo "install $GOLANG"
sudo tar -C /usr/local -xzf $GOLANG
rm $GOLANG
unset GOLANG
On my laptop, I did the installation manually by removing the previous installation

so it was an unclean removal (/usr/local/go/bin vs /usr/local/go)

Yes, it was because I deleted the folder and then reinstalled it, and then it worked fine.

how do i remove on ubuntu 20.04?

Please see the latest instructions at: https://golang.org/doc/install#install. And for further questions, please use a forum. We don't monitor closed issues for troubleshooting questions.


Improve scaffolding commands

All of us would agree that the scaffolding command is a great things in WinterCMS as it is into Laravel.
I've build many plugins this last weeks and used them widely, so I wanted to give my feedback on what could and should be improved from a "plugin-creator" point of view.
Also I'd like to get some other "plugin-creators" feedback here on how the features should be implemented

Translation keys into stubs
The first thing that should be improved is the translation keys into controllers. Most of them are already included into the core and translated in many language. I've modified the stubs here: https://github.com/wintercms/storm/pull/35
All of us need this:

many creator will create plugin for their personal use in their native-language so they are always forced to open and modify the generated controllers' views files which are only generated in english.
all the plugin-creators who aim to share their plugin on the marketplace want to make it usable by all the community without a regard to their language. Many plugins will be shared into english + the creator native language. I hope that the fact that the controllers actions will now be translated into the end-user native language directly from the core, it will ease its usage and invites the user to make a PR to the plugin to add his native-language translation.

[EDIT] @mjauvin pointed me out that a related feature has not been merged in the past due to the fact that it would be hard for beginners to read and modify the view file.
This should live as an option of the create:controller command:
php artisan create:controller AuthorName.PluginName ControllerName --use-translations

Add composer.json file to create:plugin command
Because the marketplace will widely rely on composer-installation (stop me if I'm wrong), the create:plugin file should create the composer.json file.
Most of us has never published a package, easing the first step is a nice-to-have and will encourage us to do so.
The composer.json file can't contain any comment. To guide the users into publishing his package on packagist, the command-line should display into the terminal a link to this page which include a "how-to" section.

Generate the lang folder from command-line
A thing which is missing into the scaffolding commands is definitely the lang folder, I keep creating it plugin after plugin.
Because many of us will use create:plugin command to create site-specific plugin which will never be translated or may not even need user-readable text (Eg: a tiny Plugin.php file just to register to some event and process some logic), this should probably live into an option of the create:plugin command:
artisan create:plugin AuthorName.PluginName --with-lang-files

maybe this option could be filled with some extra locales that he want to translate himself:
artisan create:plugin AuthorName.PluginName --with-lang-files=en,fr,de,it

the default would be the en and [current_installation_locale] files.
Using this option, it would generates the lang folder and locale files but also include keys into the Plugin.php file:
public function pluginDetails()
{
    return [
        'name'        => 'authorname.pluginname::lang.plugin.name',
        'description' => 'authorname.pluginname::lang.plugin.description'
        //...
    ];
}

The presence of the lang folder should also be detected intocreate:model, create:component and create:reportwidget commands to generate the stub files including translation keys.
This would ease and so increase the number of translated plugins at a long term.

Add a --settings option to create:model command
A thing that annoyed me is to create a basic model that I should almost modify totally to make it as a settings model.
The create:model command should accept a --settings option that would generate a basic settings model and not create a create_model_table.php file.
<?php namespace AuthorName\PluginName\Models;

use Winter\Storm\Database\Model;

class Settings extends Model
{
    use \Winter\Storm\Database\Traits\Validation;

    public $implement = ['System.Behaviors.SettingsModel'];

    public $settingsCode = 'authorname_pluginname_settings';

    public $settingsFields = 'fields.yaml';
}

if possible, passing this option would make the model class name optional and would default to Settings so this command would be valid:
php artisan create:model AuthorName.PluginName --settings

Add a create:tests command
It won't be the most used for sure but easing the creation of plugin-test will improve the community-plugins code quality at a long term and we'll all benefits from this.
This command line should generate the basic files including a passing test:

phpunit.xml
tests/unit/[PluginName]PluginTest.php

<?php namespace AuthorName\PluginNameTests\Unit

use PluginTestCase;

class PluginNamePluginTest extends PluginTestCase
{
    /**
     * A basic test example.
     *
     * @return void
     */
    public function testBasicTest()
    {
        $this->assertTrue(true);
    }
}

A shortcut to this could be included into create:plugin command, imagine ❤ :
php artisan create:plugin AuthorName.PluginName --with-lang-files --with-tests

Add a run:tests command
Additionally to the previous one, it was a bit annoying to go into each plugin folder to run the plugin's tests.

This command should be possible to be ran from the root:
php artisan test:plugin AuthorName.PluginName

End-note
I could work on all these features as soon as most of us agree on the need of these and how they should be implemented. So please plugin-creators: Give us your feedback!
Do you think about anything that you would see implemented that I'm missing or should the commands I described be implemented in another way / syntax?

@RomainMazB I'll convert this to a discussion in the Discussions tab. :)


feat:delete environment command

Currently there is no command to delete the chaos environment. This feature will aim to add command to delete chaos environments based on the id provided.

@Shivam you will have to add a check to see if there are any infras that are present for the environment, if yes you can just prompt that there are infras present in the environment and they need to be deleted before deleting the environment.

@SarthakJain26 @neelanjan00
Currently i am an  issue. If i delete a environment and try to create a new environment immediately then this error occurs
Chaos Environment connection failed: write exception: write errors: [E11000 duplicate key error collection: litmus.environment index: environment_id_1 dup key: { environment_id: "env1" }]
does this is not reflecting in the db?
also we are using LIST environment. But as i checked there already exist a handlers for get environment in graphlql. I suggest we should use that for get chaos-environment command rather than iterating through list of environments.This way we would provide more details and isolate both commands, this will also help in testing commands separately. We should make changes in the get chaos-environment command andlist chaos-environmentscommand

@SarthakJain26 @neelanjan00 Currently i am an issue. If i delete a environment and try to create a new environment immediately then this error occurs Chaos Environment connection failed: write exception: write errors: [E11000 duplicate key error collection: litmus.environment index: environment_id_1 dup key: { environment_id: "env1" }] does this is not reflecting in the db?
also we are using LIST environment. But as i checked there already exist a handlers for get environment in graphlql. I suggest we should use that for get chaos-environment command rather than iterating through list of environments.This way we would provide more details and isolate both commands, this will also help in testing commands separately. We should make changes in the get chaos-environment command andlist chaos-environmentscommand
	var isEnvExist = false
	for i := range Env.Data.ListEnvironmentDetails.Environments {
		if newInfra.EnvironmentID == Env.Data.ListEnvironmentDetails.Environments[i].EnvironmentID {
			utils.White_B.Println(Env.Data.ListEnvironmentDetails.Environments[i].EnvironmentID)
			isEnvExist = true
			break
		}
	}

This is other instance we are using the iteration which imo should not be needed.

@shivam-Purohit On deleting it does get reflected in the DB but the error you are seeing is because we are not deleting the document from the DB rather we are just marking it as removed, and since an mongo index is set on name field, therefore mongo does not allows the creation of a new environment with the same name. The solution to this is make the index as partial index and not consider the documents that are marked as removed. Anyways this is a separate issue which will have to be updated in graphql code.
Regarding using GET command instead of LIST, you are right, we should use GET where we want to fetch just one resource and we should be using LIST where multiple resources are being fetched. You can go ahead and update the implementation. Thanks

@SarthakJain26 so what should be the next step in this feature then?
Also I have pushed the changes I suggested if you can take a look.

@shivam-Purohit please also add the example usage of command in delete/delete.go and Usage_0.23.0.md file

done! can you take another look? thank you!

The solution to this is make the index as partial index and not consider the documents that are marked as removed.

I see a commit pushed in litmus regarding this. Should this issue now be fixed @SarthakJain26 ?

The solution to this is make the index as partial index and not consider the documents that are marked as removed.

I see a commit pushed in litmus regarding this. Should this issue now be fixed @SarthakJain26 ?

Yes Shivam! This will be fixed now.

@SarthakJain26 done!


App getting stuck while getting products.

Hi
Fetching products are working fine for me but now it stuck while fetching products in iOS.
if (await isAppPurchaseAvailable()) {
      ProductDetailsResponse productDetailResponse =
          await _connection.queryProductDetails(_productIds);
}

and after few seconds i am getting this error in logs.
[BackgroundTask] Background Task 2 ("SKProductsRequest"), was created over 30 seconds ago. In applications running in the background, this creates a risk of termination. Remember to call UIApplication.endBackgroundTask(_:) for your task in a timely manner to avoid this.
Can anyone please let me know why i am getting this error.

Hi @KuldeepAIP
It seems you are asking for assistance on personal code. Please see https://flutter.dev/community for resources and asking questions like this,
you may also get some help if you post it on Stack Overflow and if you need help with your code, please see https://www.reddit.com/r/flutterhelp/
Closing, as this isn't an issue with Flutter itself. If you disagree, please write in the comments and I will reopen it.
Thank you


GetItemLinqQueryable and ResourceToken does not work with cross partition query

Describe the bug
If you create Partition Permission  and create a cross-partition  Query on the container, it will throw an UnauthorizedException from IAuthorizationTokenProvider.GetUserAuthorizationToken
To Reproduce
var result = await tokenContainer.GetItemLinqQueryable<IUserDefinition>()
                        .Where(i => i.Id == userDefinition2.Id)
                        .ToStreamIterator();

Expected behavior
return empty result. The caller should not need Read permission on the container to execute the statement.
Actual behavior
UnauthorizedException
Environment summary
SDK Version: Source
OS Version (e.g. Windows 10

Does it work if you set the QueryRequestOptions.PartitionKey to the Partition you gave it permission to?

@fuocor fixed with PR #1143


Fix route to sign out

Sign out route: /signout -> /auth/signout

@cilwerner

Can you review this PR?
IMPORTANT: When this content is ready to merge, you must add #sign-off in a comment or the approval may get overlooked.

#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team


Need guidance on setting up LLM on Mac

Users are seeking assistance or guidance on how to properly set up and configure the LLM function to run on Mac systems. They may be facing difficulties in installing dependencies, configuring environment variables, or resolving any other setup-related issues specific to macOS.

Mac can not use CDDA but you can use mps for LLM training, you can use code like: to.('ops')

to.('mps')

thanks !


Adding retry logic to services in google-cloud-python

I will see if I can find it, but I think there was a thread somewhere that @tseaver and @dhermes were talking about having retry be an option that you passed into the call to the service.
I think some of the other libraries just retry by default but I'll try and dig a little bit.
Issues involving retry:

#3133 Core
https://github.com/GoogleCloudPlatform/google-cloud-python/issues/2684 [pubsub]
https://github.com/GoogleCloudPlatform/google-cloud-python/issues/2771 [pubsub]
https://github.com/GoogleCloudPlatform/google-cloud-python/issues/2693 [datastore]
#2813
#2896 [datastore]
#2683, #2699 [logging]
#2949, #3072 [storage]

That would be great, because retrying would also solve #2699 [logging]

@dhermes @daspecster To follow up on our offline discussion:

The GAPIC API object constructor takes as an optional parameter a retry configuration object (example in JSON; use a Python dict in the constructor instead).
This dict allows you to override the default retrying behavior for all GAPIC methods. You can pass a partial configuration (i.e., configure some but not all of the methods in an API).
Generally, by default, GAPIC methods retry automatically if the corresponding HTTP call uses a verb that is idempotent.
Every GAPIC package contains in the package data the default configuration (example).
You can change the retrying behavior on a per-method-call (as opposed to per-method) basis using the GAX CallOptions object. Every GAPIC method takes a CallOptions object as an optional parameter.

Another consideration is that we should have something for the REST implementations that we have.  AFAICT GAPIC is only for gRPC right?

@daspecster Yes, GAPIC currently supports gRPC only. But for whatever you build for REST, it would be ideal to keep it compatible with the GAPIC retry model so that if we "switch out the plumbing" at at later date and support gRPC for those APIs, you won't have to change the surface or duplicate GAPIC functionality.

#2583 is also relevant to the retry discussion; as we still cannot properly determine which datastore requests to retry, perhaps it can be covered here.

#2896 affects me as well, end of the stack trace I get is attached to that ticket.

Hi, I'm catching the same 503 ServiceUnavailable error when fetching data from the datastore.
google-auth==0.5.0
google-cloud-datastore==0.22.1
Traceback:
ServiceUnavailable: 503 {"created":"@1486373186.421925534","description":"Endpoint read failed","file":"src/core/ext/transport/chttp2/transport/chttp2_transport.c","file_line":1851,"grpc_status":14,"occurred_during_write":0,"referenced_errors":[{"created":"@1486373186.421924166","description":"Secure read failed","file":"src/core/lib/security/transport/secure_endpoint.c","file_line":166,"referenced_errors":[{"created":"@1486373186.421920569","description":"Socket closed","fd":8,"file":"src/core/lib/iomgr/tcp_posix.c","file_line":249,"target_address":"ipv4:<IP_ADDRESS>:443"}]}]}
I think that authorization expires, is there a way of authorizing the Client to avoid this kind of Exception?
Since catching it everywhere does not seems to be a nice solution.
Thanks

@geigerj What bugs are retried by default? (Looking at an internal bug where Datastore connections time out after 30 minutes, and auto-retry does not work.)

Assuming by "bugs" you mean status codes, by default we retry only on UNAVAILABLE and DEADLINE_EXCEEDED, and only for idempotent calls.
However, that configuration can be overridden at a per-service level (example: Pubsub).
It would be useful to know what status code Datastore is returning with the 30min timeout; we can add that to the GAPIC config in that case.

Now that tenacity is a dependency of google-cloud-core, it should be significantly easier to build in consistent retires for all libraries.

From my point of view the retry logic should be simple and build just into the core, something like "retry when HTTP 5xx". Otherwise it will be hard to make it consistent across Python, Go, PHP, Java,  C#, ... or a custom HTTP code. I understand it's not possible always, but the main logic should be in the core and the libraries should cover just some edge cases.
If you disagree, then I just hope all language libraries (Python, Go, PHP, Java,  C#, ...) have this retrying logic consistent :grin:
We use Python (googleapiclient,  google-cloud-python), C# (google-cloud-dotnet) and PHP (first custom HTTP, then google-cloud-php) and always spent some time digging the docs and code how the library supports retrying.

@xmedeko I'm not sure what you mean. Our libraries do not share a common core across languages. We have to implement retry in each language's core library.

Maybe I have misunderstood your post "it should be significantly easier to build in consistent retires for all libraries". I would expect the retry logic is just in the google-cloud-core and the libraries like google-cloud-storage, google-cloud-datastore, does not need implement it. And I just can set some core settings (e.g. NUM_RETRIES=6) to influence all libraries.
I understand there is not a shared core across languages libs. My concern is, that there should be share same logic (rules, principles) across  languages libs. I have found different HTTP/JSON request retry logic when I have investigated it a half year ago.
Also, I miss some general doc about the retry logic for all Google APIs. E.g. I have found some for gsutil, storage. But from my experience, these 5xx error happens in other APIs, too. I would expect some general Google Cloud API Retry Handling Strategy doc and the other API docs would just link that. I know the doc issues does not belong here. Just to explain my previous post.

I am closing this as @jonparrott is already working on retry as part of his google.api.core work.

Hello,
I guess that there has been no update on this issue, since it has been closed and I could not find any other reference to this problem in another issue or PR.
I just wanted to leave a code example that I will be using to work around this problem, based on the backoff library for the retry logic (retryable can be used as well, but it seems easier to use backoff):
import backoff
import google.cloud.storage
from google.api_core import exceptions

class _RobustConnection(google.cloud.storage._http.Connection):
    @backoff.on_exception(backoff.expo, exceptions.ServerError, max_tries=5)
    def api_request(self, *args, **kwargs):
        return super(_RobustConnection, self).api_request(*args, **kwargs)

class RobustGoogleStorageClient(google.cloud.storage.client.Client):
    """A robust client for google.cloud.storage

    This will only work on regular requests/operations.
    It will *not* work on batch operations.
    """
    def __init__(self, *args, **kwargs):
        super(RobustGoogleStorageClient, self).__init__(*args, **kwargs)
        # Change directly the _base_connection member because changing the
        # _connection property will fail: the connection is already initialized
        # in the parent constructor
        self._base_connection = _RobustConnection(self)

I cannot throughly test this because I cannot generate 5xx errors, but I think this could help.

Thank you for the work.  We also should have a new retry framework in the api_core module in most (if not all) updated versions.

Cool, I am looking forward for the retry framework.
Just an update: As I commented before, I've tried patching the client with relative success: it works for any request that uses the self._connection member, but other operations can still fail. Nevertheless, I think I solved 90% of my current problems with this approach.
My next candidate is Blob.upload_from_file, which uses a multipart or resumable upload.
This one is failing with a requests.exceptions.ConnectionError: ('Connection aborted.', BrokenPipeError(32, 'Broken pipe')), although it does not happen often.
I found that it would be just too much work to pin down and patch how requests are made internally on the Blob object. I managed to just wrap the only place where I am uploading files around a function that is decorated with a retry library, as my previous post.

@speedplane unless you need to overwrite the defaults for all methods, I don't think you need to do that. Each method should allow you to pass in a retry argument.

Yes, but at least in my project, this library is buried under multiple layers of middleware libraries, touching all of the calls is not so straightforward. Using global or a set config file would be nice. In the meantime I can just monkey-patch.

For instance, the datastore client doesn’t have a configurable retry parameter (https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/datastore/google/cloud/datastore/client.py) and I have further libraries that sit on top of that.

IMO it's nice to have a possibility to change the default value for all calls. I also increase default retries. For the old google-api-python-client I use own subclass of googleapiclient.http.HttpRequest with the overridden execute method.

After reading this tread I'm still confused on how to fix this problem. Is there a config / parameter where to set the "Retry Strategy"? or other issue where is discussed?

I ended up doing the same thing as @xmedeko, passing the subclass of googleapiclient.http.HttpRequest in teh requestBuilder kwarg of googleapiclient.discovery.build, something like this:
class MyHttpRequest(googleapiclient.http.HttpRequest):
    def execute(self, http=None, num_retries=20):
        return super().execute(http=http, num_retries=num_retries)

creds = ...
sheets = googleapiclient.discovery.build('sheets', 'v4', credentials=creds, requestBuilder=MyHttpRequest)


Ensure 412 error code from broker is handled correctly

per https://github.com/openservicebrokerapi/servicebroker/blob/master/spec.md#api-version-header

This needs to be done upstream https://github.com/pmorie/go-open-service-broker-client/issues/126

/remove-lifecycle rotten
/lifecycle frozen


How to plot Graph of waves against time

I am using VRX classic.
Like wind, one of the elements that affect the environment of VRX classic is a wave. I want to get the topic of the wave using
'rqt_plot /vrx/debug/wind/speed/data'
and output a graph of the wave over time. Do you know what to do in this case?

OK, so in that case I think the remaining piece of information you need is where on the surface of the water you would like to measure the amplitude. The nature of the simulation is that the amplitude is different in different places.
In any case, there is no built in functionality to do what you want, but depending on your needs you might be able to get a close approximation by putting a buoyant object on the service of the water and graphing its z position.

Closing this as it's not a bug or feature request.


Add custom cobra doc gen

Cobra has a builtin markdown generation feature. Here we are just using doc.GenMarkdownTree, but the problem is that the generated documents have a date part https://github.com/openshift/osd-utils-cli/blob/master/docs/command/osd-utils-cli_set.md#auto-generated-by-spf13cobra-on-17-jun-2020.
This is not very friendly for CI and new PRs. Maybe we can investigate a way to customize the document output using https://github.com/spf13/cobra/blob/master/doc/md_docs.md#customize-the-output

Seems there is a good example here https://github.com/openshift/moactl/blob/master/cmd/docs/cmd.go


Animations don't work during scrolling page

Hi, animations don't work with iphone (4th gen. the mine, but i think in every other gen.) and ipad if you browse the page with Chrome or Google Apps, during the scrolling of the page. it works only when scrolling stops and the finger is lifted from touching the screen...

This is a browser bug/quirk and outside the scope of this project.


i can be greater than 9

@aakash-mishra thanks for the PR. If the first argument to mulByDigit doesn't have to be a digit, then shouldn't we just call it mulByInt and stop using the word "digit" anywhere?

@aakash-mishra thanks for the PR. If the first argument to mulByDigit doesn't have to be a digit, then shouldn't we just call it mulByInt and stop using the word "digit" anywhere?

I agree. I'll make the changes.

I've made the required changes . Please review @lkuper


Backup data pipelining

I've come to realize how disconnected each stage of the backup currently is, and I wanted to open up a dialog on refactoring this gem so that we can stream data through each stage rather than perform each read/write separately to the disk.
Take this configuration as an example:
Backup::Model.new(:database_and_assets, 'full database and asset backup') do

  database MySQL do |database|
    database.host               = DB_CONFIG['host']
    database.name               = DB_CONFIG['database']
    database.username           = DB_CONFIG['username']
    database.password           = DB_CONFIG['password']
    database.additional_options = ['--single-transaction']
  end

  archive :assets do |archive|
    archive.add "/path/to/asset/data"
  end

  compress_with Gzip do |compression|
    compression.best    = true
    compression.fast    = false
  end

  encrypt_with OpenSSL do |encryption|
    encryption.password = "my_password"
    encryption.salt     = true
    encryption.base64   = false
  end

  store_with Local do |local|
    local.path          = "/some/mount/of/remote/device"
    local.keep          = 4
  end
end

When I trigger the backup, the following occurs:

Mysql dump to .sql file
Tar asset data
Tar results of 1 and 2 together
Gzip tar file from 3
Encrypt tar.gz from 4
Copy tar.gz.enc to destination path

It seems to me that we could easily combine steps 3, 4, 5, and 6 together, so that instead of performing a full read and write of the data 4 times, we only do it once.  This could be accomplished by:

Enabling the compress_with Gzip statement to inform the packager that the -z option should be used
Enabling the encrypt_with OpenSSL statement to inform the packager that it should be piped into the openssl command with the correct options
Enabling the store_with Local statement to inform the packager where it should be saved to instead of writing to a tmp file and copying.

These enhancements may become critical as backups get larger in size and disk/network I/O time can not be fully controlled.
I don't have any specific code change proposals yet, but I wanted to get a the rest of the communities thoughts on this problem.

Yup. This is what I've been wanting to do for a while now but wasn't sure what a good approach would be to take. Thanks for bringing it up. This is definitely something I want to have incorporated in to Backup, because as you said, the large the Backups the heavier I/O / CPU and Disk Usage will be.
If we could stream/convert File A -> File B without leaving a copy of File A behind that would be great. (And File B -> C -> D to stream through all the stages to get to the final result).
I am certainly willing to get something like that going, it has been bugging me for quite a while. Though it won't make it in to 3.0.20 (next release) if possible I'd like to implement it as soon as possible. It should be a seamless upgrade seeing as it doesn't change DSL or the end-result for the user so it can be incorporated at any time.
I'm currently waiting with 3.0.20 because I want to get the last few tickets resolved and bugs fixed so we have a clean base to work off of. I hope to get 3.0.20 out in the next few days, maybe this weekend if all goes well!
Cheers and thanks for the suggestion!

Note:
This would have to work with every compressor and encryptor, not just openssl and gzip, but that shouldn't be a problem I think?

As long as the tool supports reading and writing via stdin/stdout, then it should be easy to build a command pipeline with any of the tools involved.

Right, we'll have to see if bzip2 pbzip2 lzma (compressors) and gpg (encryptor) support it, if so then we should be able to get it going. Also there is another process you missed because it isn't in the latest gem yet (but in HEAD@develop) which is the Splitter, and the Splitter basically uses the split utility to split the archives in to multiple chunks, but I'm pretty sure that supports reading/writing from stdin/stdout.
https://github.com/meskyanichi/backup/blob/develop/lib/backup/splitter.rb
This will be added to Backup in 3.0.20 which is in a few days hopefully.

This is currently being worked on. This will not be in this next release, but will be in the following release - hopefully not too far off :)
As it stands, the new process will be:

Each archive configured will pipe it's tar output through the configured Compressor.
Each MySQL and PostgreSQL database configured will pipe it's dump output through the configure Compressor.
(I'm still looking at the other databases...)

So, from your example, you would end up with:
database_and_assets/archive/assets.tar.gz
database_and_assets/MySQL/#{DB_CONFIG['database']}.sql.gz
The final packaging tar command would be piped through the Encryptor, giving you:
#{time}.database_and_assets.tar.enc
Or, if the new Splitter is also used, the tar output will be piped through the Encyptor and split, resulting in:
#{time}.database_and_assets.tar.enc-aa
#{time}.database_and_assets.tar.enc-ab
...etc...

Pipeline changes have been merged to the develop branch. 9be16f7cf2ce942d75e16d93b9fbc8236aed6f1a


Continuous integration

I think we should set up CI using Travis (for linux) and AppVeyor (for Windows). I'm quite happy to do the work (I've just done it for another project); I just thought I'd create an issue before I start to make sure I'm not treading on anyone's toes!

I'm not opposed; I was thinking about setting up a Jenkins server, and letting anybody attach a slave with their particular tool chain to it. This would allow for a more diverse set of build systems to run the tests. But, anything would be better than what we have right now. If you want to invest the time I say go for it.

Wow, sorry I took so long with this - it got put on my todo list and then immediately buried by a hundred other things. I've set it up now - have a look at the linked PR. Should be good to go after you've set up accounts on both services - should only take 5 mins.
 And reliable too - Jenkins has always been a 99% automatic solution for me.


[incubator-kie-issues#1478] Register DRG callbacks to assigned models from DMNCompilerImpl

Closes incubator-kie-issues/1478

Hi @samuel-beniamin thanks for the PR.
I tried to understand better the overall context, and I'm sharing my thoughts: please correct me if/where I'm wrong: I may misread something.

by itself, I think the fix could be right
but those AfterProcessDrgElements / addCallback /  addCallbackForModel are used/needed only by the signavio module
but being defined/ implemented in the "core"  DMNCompilerImpl, those somehow "pollute" it with something that, IMO, has no reason to be there

So, I was thinking to

change DMNCompilerImpl.processDrgElements from private to protected, to be overridable
get rid of all signavio-specific stuff from it (double check also the tests)
create a signavio-specific subclass of DMNCompilerImpl that add the behavior required by signavio
put your modifications in that subclass

The idea would be to have DMNCompilerImpl dealing only with "core" responsibilities, and move implementation-specific ones (e.g. signavio) to dedicated subclasses
Wdyt ? Does this make sense ? Do you think you could do that ?
@baldimir @yesamer

Hi @samuel-beniamin thanks for the PR. I tried to understand better the overall context, and I'm sharing my thoughts: please correct me if/where I'm wrong: I may misread something.

by itself, I think the fix could be right
but those AfterProcessDrgElements / addCallback /  addCallbackForModel are used/needed only by the signavio module
but being defined/ implemented in the "core"  DMNCompilerImpl, those somehow "pollute" it with something that, IMO, has no reason to be there

So, I was thinking to

change DMNCompilerImpl.processDrgElements from private to protected, to be overridable
get rid of all signavio-specific stuff from it (double check also the tests)
create a signavio-specific subclass of DMNCompilerImpl that add the behavior required by signavio
put your modifications in that subclass

The idea would be to have DMNCompilerImpl dealing only with "core" responsibilities, and move implementation-specific ones (e.g. signavio) to dedicated subclasses
Wdyt ? Does this make sense ? Do you think you could do that ?
@baldimir @yesamer

Hello @gitgabrio,
Thank you for taking the time to review the code and for your suggestions.
I think your suggestions make sense, I believe I understood what need to be done, and I will be happy to make these changes in this PR as well, if that is okay.

Hello @gitgabrio
Well, I tried the approach of the a separate subclass for Signavio specific things, I have some observation and a conclusion:

I ended up not able to inject that Signavio specific compiler easily into the code.
DMNCompilerConfiguration is not Signavio specific, this is related to the core compiler for extensions in general.
drgCompilers are not Signavio specific, they are also used by other extensions, (e.g. trisotech is also using that).

This makes me think that AfterProcessDrgElements follows the same reasoning, is not Signavio specific (since drgCompilers are not), AfterProcessDrgElements is indeed used only now by Signavio, but that doesn't limit the need to Signavio only.

I have a question, do you know a way with which I can integrate the new compiler into the code by any means, I had the impression that the compiler is not intended to be changed but extended through the DMNCompilerConfiguration only, am I correct?
If I am correct on the previous point, I would actually close this ticket after rethinking this part and also close the ticket, since this is the way the compiler was meant to be extended actually.

First of all, many thanks @samuel-beniamin for all that.
I'll need a bit of time to double check, but it is very possible that you are right.
I've answered immediately to first of all thanks you again, I'll let you know ASAP!


fix css include

Hello. Please fix your bower dependencies, like
"main": ["dist/cropper.js", "dist/cropper.css"]
becouse wiredep isn't including css file.
"dist/cropper.css" - this line

OK~ Thanks~


CORS Issue

Hey guys, i have an issue.
I try to make get call
getSomeData(): Observable {
let url = here is my api URL;
let headers = new Headers({ 'Content-Type': 'application/json; charset=utf-8', "Access-Control-Allow-Origin": "*", 'dataType': 'json', });
let options = new RequestOptions({ headers: headers });
return this._http.get(url, options).map(res => res.json());
}
And I have next error
XMLHttpRequest cannot load (my URL) Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:3000' is therefore not allowed access. The response had HTTP status code 405.
This Api (It using firebird) returns XML by default, maybe this cause a problem.
How to make that always came JSON?
How to fix this problem with CORS?

This is outside of the scope of this repo. For more information please see the api you are using and the docs for CORS.


Why does parallel pipeline require a master

In my understanding, pipeline parallelism is decentralized, but why is a master needed in the example.
args.world_size = 5  # "This program requires exactly 4 workers + 1 master"

it's the main program that runs the model and joins the results


The problem when to run experiments using the Docker container

I met the problem when run bash run.sh $GPU python3 src/main.py --config=qmix --env-config=sc2 with env_args.map_name=2s3z.
Launching container named 'zkg_pymarl_GPU_python3_XIdE' on GPU 'python3' docker: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:459: container init caused: Running hook #1:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: device error: python3: unknown device: unknown. ERRO[0001] error waiting for container: context canceled
I don't know how to solve this error.

same problem here

same problem here

Hello, you can refer to this issue: #89. I posted my solution in the comments.

Hello, you can refer to this issue: #89. I posted my solution in the comments.

@reubenwong97  many thanks :)

@reubenwong97  many thanks :)

Try to use your GPU ID instead of '$GPU'. Like:bash run.sh 0 python3 src/main.py --config=qmix --env-config=sc2 with env_args.map_name=2s3z

This issue caused by '$GPU', use some numbers (like: '0', '1') can make this shell work.
However, the CUDA can not work with the docker (nvidia-docker has been installed)
Any ideas of the '$GPU'? @samvelyan
Thanks!
btw, CUDA can work when I run python3 src/main.py --config=qmix --env-config=sc2 with env_args.map_name=2s3z (do not use docker)

@4ever-Rain you may wanna run nvidia-smi to first, check the ids of your GPUs, which you can use in place of $GPU. I had experienced problems when the GPU was low on memory due to other tasks running. You can check if it has available memory with nvidia-smi.

@reubenwong97 Thanks for your advice. I'm sure my GPU is available and free. I have used GPU ids ('0') instead of '$GPU'. But CUDA still not work within docker.
Meanwhile，I'm sure torch.cuda_is_available is True in the docker.
Maybe there is something wrong about 'run.sh'?

@4ever-Rain I encounter the same problem. When I use the cuda() in the container, it will get stuck and can not work. Do you have solved the problem?

@4ever-Rain I encounter the same problem. When I use the cuda() in the container, it will get stuck and can not work. Do you have solved the problem?

Yep. It works for me now.
I finally run the code out of the container by installing all the necessary packages into a conda virtual environment.


Remove .babelrc from the published package (for using with React Native)

Installed from npm, this package doesn't work with React Native, because although the released code is transpiled, .babelrc is also published to npm.
Fixed by adding the files property in package.json to prevent .babelrc from being published.
See:
https://github.com/facebook/react-native/issues/4062

See #16

I understand why Babel should not be applied to node_modules, but in this case I can't do anything about the current behavior the React Native packager.
I wish you would consider taking this patch as a practical workaround to avoid everyone using redux-optimist with React Native having to publish their own fork to npm. The package published to npm doesn't include the src folder, so I'm quite certain .babelrc doesn't have a practical reason to be published there.

You could contribute a patch to fix the bug in react-native.  Alternatively you can rm node_modules/redux-optimist/.babelrc as a postinstall step.  I don't want to take this as there are other reasons running babel against node_modules will break things and merging this encourages people to think it's a good idea to do so.  I also don't want to have to make the same change to all my other packages or leave all the other module authors in the position of having to do the same thing.  If I (and every other package author) merge changes like this, there is no pressure on react-native to fix their bug.

That makes sense. Thanks for taking the time to explain this decision 👍


Guide for using Popper with R and RStudio

Draft of the computational research guide for R  and RStudio. Finally 🙂
@ivotron

@ivotron fixed most of this. Should be ready for re-review.
Note: I don't include a version for the alpine-wget image because latest is the only tag available for the image on Docker Hub.


Everything spazzes out when game is over sometimes

Sometimes the game just straight up spazzes out...

We need to find the source of this

I'm pretty sure its because of #45

I'm closing this provisionally in the hopes that it has been fixed because of #67. If not we will reopen and fix this separately later


Add a loader to show loading states

Components to have loader for loading states:

Select control
MultiSelect control
Search control

Fixed and tested for select control.


Get channel and user names by ID?

By reading messages on message API, i'm getting this object
{ type: 'message',
  channel: 'G0HBYU10R',
  user: 'U086E9068',
  text: 'hello !',
  ts: '1451320460.000007',
  team: 'T02C5T9ES',
  event: 'ambient' }

but i need to get additional data like channel name and user name, not only their ID's
is this possible with botkit?

found slack api calls like that
bot.api.channels.list({'exclude_archived' : 1}, function (err, res) {
    console.log(res);
});

which leads to
TypeError: Cannot read property 'channels' of undefined

okay, sorry, found the way out
to use bot api, you need to save spawn instance
let botApi = bot.spawn({
  token: cfg.slackBotToken,
}).startRTM();

after that this will work
botApi.api.channels.list({exclude_archived: 1}, function (err, res) {
  console.log(res);
});


Increase test coverage

Background
Relates to #400.
We would like better test coverage, as a baseline for assurance that our code is well tested and correct.
Proposed Solution

Identify any missing test modules which weren't brought over from cardano-sl to cardano-ledger
Look at modules which have low code coverage as reported by https://coveralls.io/github/input-output-hk/cardano-ledger

[ ] Compile a list of other modules which are lacking coverage, so address this whole category of problem, rather than doing them in a one-off fashion 🙂

Use Jordan's past PRs to see which tests have already been brought over.

Next Action

[ ] Compile list
[ ] Create issues for moving any modules over

OR just raise a PR moving the modules over

Another way to raise coverage % is to exclude derived instances which we can safely assume are correct.
It seems that we could either write dummy functions to exercise all instances, or we can try to mess around with the .tix files via overlaying:
https://stackoverflow.com/questions/46481633/how-can-i-generate-a-coverage-overlay-with-hpc-and-stack
https://stackoverflow.com/questions/25797720/how-to-exclude-derivation-of-show-from-hpcs-code-coverage

In general I'd like to exclude some functionality, such as Buildable instances that have nothing to test! I guess we could test that they don't crash, so just add eval $ sformat build exampleValue or something?


Indent Level not aligning correctly

Description
This is a defect I have introduced that partially is related to attributeChain (see #10) and occurs when a wrap based indentation is being used and the token is wrapped in a Liquid expression. It is a very minor issue.
Playground Reference
Issue can be visualized in the playground on line 10.
Context
This defect is occuring due to malformed level reference. Likely incurred during attribute parsing. This is a simple fix.

Note
This is actually not a defect but just standard behaviour. The playground sample is wrapping a self-closing void which is identified as such in the grammars of Prettify. Some additional though needs to be had here. Will revisit it soon


feat(jqLite): Add automatic prefixing to CSS properties

jQuery 1.8 added the automatic CSS property prefixing.
Less code to write when you don't have to worry about managing
prefixes.
Handles setting prefixed values so is backwards compat.

OK, so this PR is quite old now and many browsers now support many of these properties without prefixes. I don't think there is that much benefit in supporting it in jqLite. Given that jquery supports this idea, then it might be that one should just use that instead.

@ryanseddon do you still need this or can we close the PR?

No haven't used angular in quite some time, closing.


Multiple passwords for connecting two PostgreSQL database in a project.

Some users want to connect multiple PostgreSQL database in the same project.
It uses a different password in each database.
Due to the Digdag use single password in a single project,
It's can't create a task for connecting multiple databases.
@muga
What do you think to add an option for specifying connection password like the below?
pg>: test.sql
password: pg.devevelopment.password

Current implementation
Proposed modification.
secrets.getSecretOptional("password").or(() -> {
 String passSecName =  params.get("dev_password", String.class);
  secrets.get(passSecName);
})

References.

Twitter (Japanese)

Thank you for suggesting. and got your point. I don't have good idea so much to solve this for now. Let me think about it. @frsyuki do you have any ideas?

I'm taking care of this feature at https://github.com/treasure-data/digdag/pull/839

#839 is good idea. thank you! @komamitsu


Add release notes for 2.2.1 [#181259448]

Signed-off-by: Jeremy Alvis<EMAIL_ADDRESS>

We will notify here when we are ready for this to be merged. Keeping it as an active PR so that it is clear that we have release notes in progress for 2.2.1.