Gym got stuck at xcrun after successfully archiving and got "Xcode 7 provisioning_profile_path will be ignored" warning though the Xcode version is 8

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I have read the Contribution Guidelines

Issue Description
Although I am using Xcode 8, I got these warnings/errors after Archive Succeeded:
You're using Xcode 7, the `provisioning_profile_path` value will be ignored
Please follow the Code Signing Guide: https://codesigning.guide (for match) or https://docs.fastlane.tools/codesigning/GettingStarted/

and then I got stuck at xcrun
/usr/bin/xcrun /usr/local/lib/ruby/gems/2.3.0/gems/gym-1.10.0/lib/assets/wrap_xcodebuild/xcbuild-safe.sh -exportArchive -exportOptionsPlist '/var/folders/77/zxlvnwsj3dv3nc6kt_szls2w0000gn/T/gym_config20161003-2651-1jka4zc.plist' -archivePath /Users/archy/Library/Developer/Xcode/Archives/2016-10-03/LittleDragonPoC\ 2016-10-03\ 11.30.20.xcarchive -exportPath '/var/folders/77/zxlvnwsj3dv3nc6kt_szls2w0000gn/T/gym_output20161003-2651-qfr7ei'

more info
My iOS project 1. contains a custom nested framework (there are many custom frameworks inside one custom framework that the app uses) 2. is generated by Unity (the app project is exported by Unity, and we just do some code injection to get swift to take control and imported our custom nested framework to it). Before updating my xcode to xcode 8 I am able to build it and put it up on testflight just fine in xcode (after some tinkering). But after updating it to xcode 8 I get this error that I have never encountered in my life before (despite all these build errors I have encountered).
ERROR ITMS-90171: "Invalid Bundle Structure - The binary file 'MyApp.app/libswiftRemoteMirror.dylib' is not permitted. Your app can't contain standalone executables or libraries, other than the CFBundleExecutable of supported bundles. Refer to the Bundle Programming Guide at https://developer.apple.com/go/?id=bundle-structure for information on the iOS app bundle structure."

It complains about a file named libswiftRemoteMirror.dylib. And it exists in all my frameworks and the app.
After googling I found this SO post. So I decided to give gym a try to see if it can build it without the annoying libswiftRemoteMirror.dylib.
Complete output when running fastlane, including the stack trace and command used
INFO [2016-10-03 11:32:29.47]: ▸ Archive Succeeded
DEBUG [2016-10-03 11:32:29.62]: Stored the archive in: /Users/archy/Library/Developer/Xcode/Archives/2016-10-03/LittleDragonPoC 2016-10-03 11.30.20.xcarchive
ERROR [2016-10-03 11:32:29.62]: You're using Xcode 7, the `provisioning_profile_path` value will be ignored
ERROR [2016-10-03 11:32:29.62]: Please follow the Code Signing Guide: https://codesigning.guide (for match) or https://docs.fastlane.tools/codesigning/GettingStarted/
+-------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------+
|                                                                                Generated Package Command                                                                                |
+-------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------+
| Option                                                                                | Value                                                                                |
+-------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------+
| /usr/bin/xcrun /usr/local/lib/ruby/gems/2.3.0/gems/gym-1.10.0/lib/assets/wrap_xcodebuild/xcbuild-safe.sh -exportArchive |                                                                                |
| -exportOptionsPlist                                                                                | /var/folders/77/zxlvnwsj3dv3nc6kt_szls2w0000gn/T/gym_config20161003-2651-1jka4zc.plist |
| -archivePath /Users/archy/Library/Developer/Xcode/Archives/2016-10-03/LittleDragonPoC\ 2016-10-03\ 11.30.20.xcarchive   |                                                                                |
| -exportPath                                                                                | /var/folders/77/zxlvnwsj3dv3nc6kt_szls2w0000gn/T/gym_output20161003-2651-qfr7ei        |
+-------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------+
INFO [2016-10-03 11:32:29.63]: $ /usr/bin/xcrun /usr/local/lib/ruby/gems/2.3.0/gems/gym-1.10.0/lib/assets/wrap_xcodebuild/xcbuild-safe.sh -exportArchive -exportOptionsPlist '/var/folders/77/zxlvnwsj3dv3nc6kt_szls2w0000gn/T/gym_config20161003-2651-1jka4zc.plist' -archivePath /Users/archy/Library/Developer/Xcode/Archives/2016-10-03/LittleDragonPoC\ 2016-10-03\ 11.30.20.xcarchive -exportPath '/var/folders/77/zxlvnwsj3dv3nc6kt_szls2w0000gn/T/gym_output20161003-2651-qfr7ei'

Configuration Files
Fastfile:
lane :beta do
  cert
  sigh
  gym(scheme: "Unity-iPhone")
  testflight
end

Environment
fastlane version (run fastlane -v): 1.104.0
Do you use bundler to execute fastlane (i.e. bundle exec fastlane)? Nope.
Do you use a Ruby environment manager (e.g. chruby, rbenv, rvm)? Nope.

@fastlane-bot yeah went through it.

@account-archived-270316 Have you tried using PROVISIONING_PROFILE_SPECIFIER?

I have the some problem here.
Fastlane get stuck here:
INFO [2016-10-10 12:18:05.77]: $ /usr/bin/xcrun /Library/Ruby/Gems/2.0.0/gems/gym-1.11.0/lib/assets/wrap_xcodebuild/xcbuild-safe.sh -exportArchive -exportOptionsPlist '/var/folders/93/nkj7sgwd6x52_cg_bw4by7b98r07ws/T/gym_config20161010-17096-17r2lhy.plist' -archivePath /Users/myUser/Library/Developer/Xcode/Archives/2016-10-10/MyAppName\ 2016-10-10\ 12.16.41.xcarchive -exportPath '/var/folders/93/nkj7sgwd6x52_cg_bw4by7b98r07ws/T/gym_output20161010-17096-t3ni2o'

Hi @marcocattai Can you send over your full input and output? I'd love to look into this. Thanks so much!

@marcocattai  I have got the same problem as you! Did you manage to solve it?

@Sajjon Can you run fastlane run update_fastlane and then let me know if you still hit this error? If you do run fastlane env and share the output. Thanks!

The same error.
The most interesting part that if I call xcbuild directly - it works.
/usr/bin/xcrun /usr/local/lib/ruby/gems/2.3.0/gems/gym-1.11.3/lib/assets/wrap_xcodebuild/xcbuild-safe.sh -exportArchive -exportOptionsPlist '/var/folders/tr/_lmsx9210vscqg_hpdwb85000000gn/T/gym_config20161103-4654-cnxjdk.plist' -archivePath /Users/SinceTV/Library/Developer/Xcode/Archives/2016-11-03/CutCuts\ 2016-11-03\ 15.37.43.xcarchive -exportPath'
will produce valid ipa
/var/folders/tr/_lmsx9210vscqg_hpdwb85000000gn/T/gym_output20161103-4654-ngmb3h'`
✅ fastlane environment ✅
Stack

Key
Value

OS
10.11.6

Ruby
2.3.1

Bundler?
false

Xcode Path
/Applications/Xcode.app/Contents/Developer/

Xcode Version
8.1

Git
git version 2.9.3 (Apple Git-75)

Installation Source
/usr/local/bin/fastlane

Host
Mac OS X 10.11.6 (15G1004)

Ruby Lib Dir
/usr/local/Cellar/ruby/2.3.1_2/lib

OpenSSL Version
OpenSSL 1.0.2j  26 Sep 2016

fastlane files:
`./fastlane/Fastfile`
# Customise this file, documentation can be found here:
# https://github.com/fastlane/fastlane/tree/master/fastlane/docs
# All available actions: https://github.com/fastlane/fastlane/blob/master/fastlane/docs/Actions.md
# can also be listed using the `fastlane actions` command

# Change the syntax highlighting to Ruby
# All lines starting with a # are ignored when running `fastlane`

# If you want to automatically update fastlane if a new version is available:
# update_fastlane/Users/SinceTV/Documents/projects/cutcutz/fastlane/Appfile

# This is the minimum version number required.
# Update this, if you use features of a newer version
fastlane_version "1.98.0"

#default_platform :ios

platform :android do
  #before_all do
    # ENV["SLACK_URL"] = "https://hooks.slack.com/services/..."
  #end

  #desc "Runs all the tests"
  #lane :test do
  #  gradle(task: "test")
  #end

  desc "Submit a new Beta Build to Crashlytics Beta"
  lane :beta do
    ENV["ANDROID_HOME"]="/Users/SinceTV/Library/Android/sdk"
    ENV["JAVA_HOME"]="/Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home"

    gradle(project_dir: "platforms/android/", task: 'wrapper')
    #gradle(task: "assembleRelease")
    sign_apk(
      alias: "Beta",
      keystore_path: "./cckey.jks"
	)
    zipalign(apk_path: "platforms/android/build/outputs/apk/android-debug.apk")
    supply(track: "beta",
    skip_upload_metadata: true,
    skip_upload_images: true,
    skip_upload_screenshots: true
	)
	end

  desc "Deploy a new version to the Google Play"
  lane :deploy do
    gradle(task: "assembleRelease")
    supply
  end

  # You can define as many lanes as you want

#  after_all do |lane|
    # This block is called, only if the executed lane was successful

    # slack(
    #   message: "Successfully deployed new App Update."
    # )
#  end

  #error do |lane, exception|
    # slack(
    #   message: exception.message,
    #   success: false
    # )
  #end
end

platform :ios do
  #before_all do
    # ENV["SLACK_URL"] = "https://hooks.slack.com/services/..."
    # update_project_provisioning(
    #  xcodeproj: "platforms/ios/CutCutz.xcodeproj"#,
    # profile: "ClientDevices_Deploy.mobileprovision"
    #  )
  #end

  desc "Fetches the provisioning profiles so you can build locally and deploy to your device"
  lane :certs do
    match(app_identifier: "io.nodeart.cutcutz.debug", type: "development", force_for_new_devices: true)
    match(app_identifier: "io.nodeart.cutcutz.internalbeta", type: "appstore", force_for_new_devices: true)
    match(app_identifier: "io.nodeart.cutcutz.production", type: "appstore", force_for_new_devices: true)
  end

  desc "Runs all the tests"
  lane :test do
    scan
  end

  desc "Submit a new Beta Build to Apple TestFlight"
  desc "This will also make sure the profile is up to date"
  lane :beta do
    #ensure_git_status_clean
    #ensure_git_branch(branch: 'master')
    #git_pull

    build_number = Time.new.strftime("%Y.%m.%d.%H%M%S")
    increment_build_number(build_number: build_number, xcodeproj:"platforms/ios/CutCuts.xcodeproj")

    match(app_identifier: "com.cutcuts.app", type: "appstore") # more information: https://codesigning.guide
    gym(scheme: "CutCuts", sdk: "iphoneos", configuration: "Release", project: "platforms/ios/CutCuts.xcodeproj", clean: true) # Build your app - more options available
    pilot(app_identifier: "com.cutcuts.app")

    # sh "your_script.sh"
    # You can also use other beta testing services here (run `fastlane actions`)
  end

  desc "Deploy a new version to the App Store"
  lane :appstore do
    match(type: "adhoc", project: "platforms/ios/CutCuts.xcodeproj")
    # snapshot
    gym(scheme: "CutCuts", sdk: "iphoneos", project: "platforms/ios/CutCuts.xcodeproj") # Build your app - more options available
    deliver(force: true)
    # frameit
  end

  # You can define as many lanes as you want

  #after_all do |lane|
    # This block is called, only if the executed lane was successful

    # slack(
    #   message: "Successfully deployed new App Update."
    # )
  #end

  error do |lane, exception|
    # slack(
    #   message: exception.message,
    #   success: false
    # )
  end
end

# More information about multiple platforms in fastlane: https://github.com/fastlane/fastlane/blob/master/fastlane/docs/Platforms.md
# All available actions: https://github.com/fastlane/fastlane/blob/master/fastlane/docs/Actions.md

# fastlane reports which actions are used
# No personal data is recorded. Learn more at https://github.com/fastlane/enhancer```
</details>

<details><summary>`./fastlane/Appfile`</summary>

```ruby
app_identifier "com.cutcuts.app" # The bundle identifier of your app
apple_id<EMAIL_ADDRESS># Your Apple email address

team_id "2Y5LPLND37" # Developer Portal Team ID

# you can even provide different app identifiers, Apple IDs and team names per lane:
# More information: https://github.com/fastlane/fastlane/blob/master/fastlane/docs/Appfile.md

fastlane gems

Gem
Version
Update-Status

credentials_manager
0.16.2
💥 Check failed

spaceship
0.36.2
💥 Check failed

fastlane_core
0.53.0
💥 Check failed

deliver
1.14.5
💥 Check failed

snapshot
1.16.3
💥 Check failed

frameit
3.0.0
💥 Check failed

pem
1.4.0
💥 Check failed

cert
1.4.3
💥 Check failed

sigh
1.11.2
💥 Check failed

produce
1.2.1
💥 Check failed

gym
1.11.3
💥 Check failed

pilot
1.11.2
💥 Check failed

scan
0.13.1
💥 Check failed

supply
0.7.1
💥 Check failed

match
0.10.0
💥 Check failed

screengrab
0.5.5
💥 Check failed

fastlane
1.106.2
💥 Check failed

Loaded fastlane plugins:
No plugins Loaded*
Loaded gems

Gem
Version

did_you_mean
1.0.0

highline
1.7.8

commander
4.4.0

terminal-table
1.4.5

json
1.8.3

multipart-post
2.0.0

faraday
0.9.2

faraday_middleware
0.10.0

dotenv
2.1.1

net-ssh
3.2.0

net-sftp
2.1.2

plist
3.1.0

rubyzip
1.1.7

security
0.1.3

krausefx-shenzhen
0.14.10

slack-notifier
1.5.1

i18n
0.7.0

thread_safe
0.3.5

tzinfo
1.2.2

minitest
5.8.3

activesupport
<IP_ADDRESS>

CFPropertyList
2.3.3

claide
1.0.1

colored
1.2

nanaimo
0.2.0

xcodeproj
1.4.1

rouge
1.11.1

xcpretty
0.2.4

terminal-notifier
1.7.1

addressable
2.3.8

credentials_manager
0.16.2

multi_xml
0.5.5

unf_ext
<IP_ADDRESS>

unf
0.1.4

domain_name
0.5.20161021

http-cookie
1.0.3

faraday-cookie_jar
0.0.6

fastimage
1.6.8

xcode-install
2.0.8

word_wrap
1.0.0

multi_json
1.12.1

babosa
1.0.2

excon
0.54.0

gh_inspector
1.0.2

fastlane_core
0.53.0

bundler
1.13.6

mini_magick
4.5.1

xcpretty-travis-formatter
0.0.4

uber
0.0.15

representable
2.3.0

retriable
2.1.0

mime-types-data
3.2016.0521

mime-types
3.1

hurley
0.2

little-plugger
1.1.4

logging
2.1.0

jwt
1.5.6

memoist
0.15.0

os
0.9.6

signet
0.7.3

googleauth
0.5.1

httpclient
<IP_ADDRESS>

google-api-client
0.9.19

io-console
0.4.5

generated on: 2016-11-03

@TKBurner Hi thanks for your reply.
So I wanna use fastlane on my Mac Mini and start fastlane as part of CI (letting Mac mini act as Team City agent).
This is my Gemfile:
source 'https://rubygems.org'

gem 'fastlane'
gem 'match'
gem 'gym'
gem 'deliver'
gem 'plist'
gem 'json'
gem 'curb'

I am running fastlane with bundle exec fastlane ios beta (after having run sudo bundle update (directly via CLI on Mac mini, not using Team City)) and the exporting part of gym actually finishes but takes almost 20 minutes, whereas not using bundler or Gemfile at all, just plain old sudo gem install fastlane and fastlane ios beta is fast, just some seconds or mintue for this step.
Output for just the export part of gym:
[14:30:45]: ▸ Touching mycomp.app.dSYM
[14:30:46]: ▸ Archive Succeeded
[14:30:46]: Generated plist file with the following values:
[14:30:46]: ▸ -----------------------------------------
[14:30:46]: ▸ {
[14:30:46]: ▸   "method": "enterprise"
[14:30:46]: ▸ }
[14:30:46]: ▸ -----------------------------------------
[14:30:46]: $ /usr/bin/xcrun /usr/local/lib/ruby/gems/2.3.0/gems/gym-1.11.3/lib/assets/wrap_xcodebuild/xcbuild-safe.sh -exportArchive -exportOptionsPlist '/var/folders/xy/hg_sfr7d3njfjmqqnc5hkht40000gn/T/gym_config20161103-45954-1yb3h9.plist' -archivePath /Users/admin/Library/Developer/Xcode/Archives/2016-11-03/mycomp\ 2016-11-03\ 14.13.10.xcarchive -exportPath '/var/folders/xy/hg_sfr7d3njfjmqqnc5hkht40000gn/T/gym_output20161103-45954-c87x7b'
[14:50:38]: Compressing 15 dSYM(s)
[14:50:38]: $ cd '/Users/admin/Library/Developer/Xcode/Archives/2016-11-03/mycomp 2016-11-03 14.13.10.xcarchive/dSYMs' && zip -r '/Users/admin/Developer/iOS/mycomp-iOS/mycomp.app.dSYM.zip' *.dSYM

This is the output from fastlane env:
[14:37:32]: Generating fastlane environment output, this might take a few seconds...
✅ fastlane environment ✅
Stack

Key
Value

OS
10.12.1

Ruby
2.3.1

Bundler?
false

Xcode Path
/Applications/Xcode.app/Contents/Developer/

Xcode Version
8.1

Git
git version 2.9.3 (Apple Git-75)

Installation Source
/usr/local/bin/fastlane

Host
Mac OS X 10.12.1 (16B2555)

Ruby Lib Dir
/usr/local/Cellar/ruby/2.3.1_2/lib

OpenSSL Version
OpenSSL 1.0.2j  26 Sep 2016

fastlane files:
`./fastlane/Fastfile`
# Customise this file, documentation can be found here:
# https://github.com/fastlane/fastlane/tree/master/fastlane/docs
# All available actions: https://github.com/fastlane/fastlane/blob/master/fastlane/docs/Actions.md
# can also be listed using the `fastlane actions` command

# Change the syntax highlighting to Ruby
# All lines starting with a # are ignored when running `fastlane`

# If you want to automatically update fastlane if a new version is available:
# update_fastlane

require_relative 'utils/mycomp_utils'

# This is the minimum version number required.
# Update this, if you use features of a newer version
fastlane_version "1.104.0"

default_platform :ios

platform :ios do
	before_all do
		ensure_git_status_clean
	end

	desc "Runs all the tests"
	lane :certs do
		match_appstore
		match_development
		match_enterprise_distibution
	end

	desc "Release beta version of the app"
	desc "Available options are:"
	desc "	version_number_index"
	desc "	version_number_manual"
	desc "	version_number_force"
	desc "	allow_arbitrary_loads"
	desc "	release_notes"
	lane :beta do |options|
		begin
			match_enterprise_distibution # Make sure we are using latest certs and provisioning profile
			build_app(options.merge(:is_app_store_build => false))
			mission_control(
				s3_access_key: "some_access_key",
				s3_secret_access_key: "some_secret_access_key",
				release_notes: options[:release_notes]
			)

			slack_notify_new_version(is_beta: true)
		rescue Interrupt
			cleanup_on_interrupt
		end
	end

	desc "Release app to the App Store"
	desc "Available options are:"
	desc "	version_number_index"
	desc "	version_number_manual"
	desc "	version_number_force"
	desc " 	submit_for_review"
	desc " 	automatic_release"
	lane :store do |options|
		begin
			match_appstore # Make sure we are using latest certs and provisioning profile
			build_app(options.merge(:is_app_store_build => true))

			deliver( # Upload IPA to iTunes Connect (see 'Deliverfile' for more config)
				submit_for_review: options[:submit_for_review],
				automatic_release: options[:automatic_release]
			)

			add_and_push_tag
			slack_notify_new_version(is_beta: false)
		rescue Interrupt
			cleanup_on_interrupt
		end
	end

	after_all do |lane|
		# Only called if executed lane was successful
		cleanup
	end

	error do |lane, exception|
		cleanup
	end
end```
</details>

<details><summary>`./fastlane/Appfile`</summary>

```ruby
apple_id<EMAIL_ADDRESS>team_name 'mycomp AB' # yes... same team_name for `standard` and `enterprise` account
$app_identifier_beta = 'com.mycomp.app.developer'
$app_identifier_store = 'com.mycomp.app'
$team_id_enterprise = 'MY_ENTERPRISE_TEAM_ID'
$team_id_store = 'MY_TEAM_ID'

for_lane :beta do
	app_identifier $app_identifier_beta
	team_id $team_id_enterprise
end

for_lane :store do
	app_identifier $app_identifier_store
	team_id $team_id_store
end```
</details>

### fastlane gems

| Gem                 | Version | Update-Status |
| ------------------- | ------- | ------------- |
| credentials_manager | 0.16.2  | ✅ Up-To-Date  |
| spaceship           | 0.36.2  | ✅ Up-To-Date  |
| fastlane_core       | 0.53.0  | ✅ Up-To-Date  |
| deliver             | 1.14.5  | ✅ Up-To-Date  |
| snapshot            | 1.16.3  | ✅ Up-To-Date  |
| frameit             | 3.0.0   | ✅ Up-To-Date  |
| pem                 | 1.4.0   | ✅ Up-To-Date  |
| cert                | 1.4.3   | ✅ Up-To-Date  |
| sigh                | 1.11.2  | ✅ Up-To-Date  |
| produce             | 1.2.1   | ✅ Up-To-Date  |
| gym                 | 1.11.3  | ✅ Up-To-Date  |
| pilot               | 1.11.2  | ✅ Up-To-Date  |
| scan                | 0.13.1  | ✅ Up-To-Date  |
| supply              | 0.7.1   | ✅ Up-To-Date  |
| match               | 0.10.0  | ✅ Up-To-Date  |
| screengrab          | 0.5.5   | ✅ Up-To-Date  |
| fastlane            | 1.106.2 | ✅ Up-To-Date  |

### Loaded fastlane plugins:

**No plugins Loaded***

<details><summary><b>Loaded gems</b></summary>

| Gem                       | Version      |
| ------------------------- | ------------ |
| did_you_mean              | 1.0.0        |
| highline                  | 1.7.8        |
| commander                 | 4.4.0        |
| terminal-table            | 1.4.5        |
| json                      | 1.8.3        |
| multipart-post            | 2.0.0        |
| faraday                   | 0.9.2        |
| faraday_middleware        | 0.10.0       |
| dotenv                    | 2.1.1        |
| net-ssh                   | 3.2.0        |
| net-sftp                  | 2.1.2        |
| plist                     | 3.1.0        |
| rubyzip                   | 1.1.7        |
| security                  | 0.1.3        |
| krausefx-shenzhen         | 0.14.10      |
| slack-notifier            | 1.5.1        |
| i18n                      | 0.7.0        |
| thread_safe               | 0.3.5        |
| tzinfo                    | 1.2.2        |
| activesupport             | <IP_ADDRESS>      |
| CFPropertyList            | 2.3.3        |
| claide                    | 1.0.1        |
| colored                   | 1.2          |
| nanaimo                   | 0.2.0        |
| xcodeproj                 | 1.4.1        |
| rouge                     | 1.11.1       |
| xcpretty                  | 0.2.4        |
| terminal-notifier         | 1.7.1        |
| credentials_manager       | 0.16.2       |
| multi_xml                 | 0.5.5        |
| unf_ext                   | <IP_ADDRESS>      |
| unf                       | 0.1.4        |
| domain_name               | 0.5.20161021 |
| http-cookie               | 1.0.3        |
| faraday-cookie_jar        | 0.0.6        |
| fastimage                 | 1.6.8        |
| xcode-install             | 2.0.8        |
| word_wrap                 | 1.0.0        |
| multi_json                | 1.12.1       |
| babosa                    | 1.0.2        |
| excon                     | 0.54.0       |
| gh_inspector              | 1.0.2        |
| fastlane_core             | 0.53.0       |
| bundler                   | 1.13.6       |
| mini_magick               | 4.5.1        |
| xcpretty-travis-formatter | 0.0.4        |
| uber                      | 0.0.15       |
| representable             | 2.3.0        |
| retriable                 | 2.1.0        |
| mime-types-data           | 3.2016.0521  |
| mime-types                | 3.1          |
| hurley                    | 0.2          |
| little-plugger            | 1.1.4        |
| logging                   | 2.1.0        |
| jwt                       | 1.5.6        |
| memoist                   | 0.15.0       |
| os                        | 0.9.6        |
| signet                    | 0.7.3        |
| googleauth                | 0.5.1        |
| httpclient                | <IP_ADDRESS>      |
| google-api-client         | 0.9.19       |
| io-console                | 0.4.5        |
</details>

*generated on:* **2016-11-03**
</details>

This is the outout from `bundle exec fastlane env`
[14:38:32]: Generating fastlane environment output, this might take a few seconds...
<details><summary>✅ fastlane environment ✅</summary>

### Stack

| Key                 | Value                                       |
| ------------------- | ------------------------------------------- |
| OS                  | 10.12.1                                     |
| Ruby                | 2.3.1                                       |
| Bundler?            | true                                        |
| Xcode Path          | /Applications/Xcode.app/Contents/Developer/ |
| Xcode Version       | 8.1                                         |
| Git                 | git version 2.9.3 (Apple Git-75)            |
| Installation Source | /usr/local/bin/fastlane                     |
| Host                | Mac OS X 10.12.1 (16B2555)                  |
| Ruby Lib Dir        | /usr/local/Cellar/ruby/2.3.1_2/lib          |
| OpenSSL Version     | OpenSSL 1.0.2j  26 Sep 2016                 |

### fastlane files:

<details><summary>`./fastlane/Fastfile`</summary>

```ruby
# Customise this file, documentation can be found here:
# https://github.com/fastlane/fastlane/tree/master/fastlane/docs
# All available actions: https://github.com/fastlane/fastlane/blob/master/fastlane/docs/Actions.md
# can also be listed using the `fastlane actions` command

# Change the syntax highlighting to Ruby
# All lines starting with a # are ignored when running `fastlane`

# If you want to automatically update fastlane if a new version is available:
# update_fastlane

require_relative 'utils/mycomp_utils'

# This is the minimum version number required.
# Update this, if you use features of a newer version
fastlane_version "1.104.0"

default_platform :ios

platform :ios do
	before_all do
		ensure_git_status_clean
		ENV["SLACK_URL"] = "https://hooks.slack.com/services/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"
	end

	desc "Runs all the tests"
	lane :certs do
		match_appstore
		match_development
		match_enterprise_distibution
	end

	desc "Release beta version of the app"
	desc "Available options are:"
	desc "	version_number_index"
	desc "	version_number_manual"
	desc "	version_number_force"
	desc "	allow_arbitrary_loads"
	desc "	release_notes"
	lane :beta do |options|
		begin
			match_enterprise_distibution # Make sure we are using latest certs and provisioning profile
			build_app(options.merge(:is_app_store_build => false))
			mission_control(
				s3_access_key: "some_access_key",
				s3_secret_access_key: "some_secret_access_key",
				release_notes: options[:release_notes]
			)

			slack_notify_new_version(is_beta: true)
		rescue Interrupt
			cleanup_on_interrupt
		end
	end

	desc "Release app to the App Store"
	desc "Available options are:"
	desc "	version_number_index"
	desc "	version_number_manual"
	desc "	version_number_force"
	desc " 	submit_for_review"
	desc " 	automatic_release"
	lane :store do |options|
		begin
			match_appstore # Make sure we are using latest certs and provisioning profile
			build_app(options.merge(:is_app_store_build => true))

			deliver( # Upload IPA to iTunes Connect (see 'Deliverfile' for more config)
				submit_for_review: options[:submit_for_review],
				automatic_release: options[:automatic_release]
			)

			add_and_push_tag
			slack_notify_new_version(is_beta: false)
		rescue Interrupt
			cleanup_on_interrupt
		end
	end

	after_all do |lane|
		# Only called if executed lane was successful
		cleanup
	end

	error do |lane, exception|
		cleanup
	end
end```
</details>

<details><summary>`./fastlane/Appfile`</summary>

```ruby
apple_id<EMAIL_ADDRESS>team_name 'mycomp AB' # yes... same team_name for `standard` and `enterprise` account
$app_identifier_beta = 'com.mycomp.app.developer'
$app_identifier_store = 'com.mycomp.app'
$team_id_enterprise = 'MY_ENTERPRISE_TEAM_ID'
$team_id_store = 'MY_TEAM_ID'

for_lane :beta do
	app_identifier $app_identifier_beta
	team_id $team_id_enterprise
end

for_lane :store do
	app_identifier $app_identifier_store
	team_id $team_id_store
end```
</details>

### fastlane gems

| Gem                 | Version | Update-Status |
| ------------------- | ------- | ------------- |
| credentials_manager | 0.16.2  | ✅ Up-To-Date  |
| fastlane_core       | 0.53.0  | ✅ Up-To-Date  |
| spaceship           | 0.36.2  | ✅ Up-To-Date  |
| cert                | 1.4.3   | ✅ Up-To-Date  |
| deliver             | 1.14.5  | ✅ Up-To-Date  |
| frameit             | 3.0.0   | ✅ Up-To-Date  |
| gym                 | 1.11.3  | ✅ Up-To-Date  |
| sigh                | 1.11.2  | ✅ Up-To-Date  |
| match               | 0.10.0  | ✅ Up-To-Date  |
| pem                 | 1.4.0   | ✅ Up-To-Date  |
| pilot               | 1.11.2  | ✅ Up-To-Date  |
| produce             | 1.2.1   | ✅ Up-To-Date  |
| scan                | 0.13.1  | ✅ Up-To-Date  |
| screengrab          | 0.5.5   | ✅ Up-To-Date  |
| snapshot            | 1.16.3  | ✅ Up-To-Date  |
| supply              | 0.7.1   | ✅ Up-To-Date  |
| fastlane            | 1.106.2 | ✅ Up-To-Date  |

### Loaded fastlane plugins:

**No plugins Loaded***

<details><summary><b>Loaded gems</b></summary>

| Gem                       | Version      |
| ------------------------- | ------------ |
| did_you_mean              | 1.0.0        |
| bundler                   | 1.13.6       |
| io-console                | 0.4.5        |
| CFPropertyList            | 2.3.3        |
| i18n                      | 0.7.0        |
| json                      | 1.8.3        |
| minitest                  | 5.9.1        |
| thread_safe               | 0.3.5        |
| tzinfo                    | 1.2.2        |
| activesupport             | <IP_ADDRESS>      |
| addressable               | 2.3.8        |
| babosa                    | 1.0.2        |
| colored                   | 1.2          |
| highline                  | 1.7.8        |
| commander                 | 4.4.0        |
| security                  | 0.1.3        |
| credentials_manager       | 0.16.2       |
| excon                     | 0.54.0       |
| gh_inspector              | 1.0.2        |
| multi_json                | 1.12.1       |
| plist                     | 3.1.0        |
| rubyzip                   | 1.1.7        |
| terminal-table            | 1.4.5        |
| fastlane_core             | 0.53.0       |
| multipart-post            | 2.0.0        |
| faraday                   | 0.9.2        |
| unf_ext                   | <IP_ADDRESS>      |
| unf                       | 0.1.4        |
| domain_name               | 0.5.20161021 |
| http-cookie               | 1.0.3        |
| faraday-cookie_jar        | 0.0.6        |
| faraday_middleware        | 0.10.0       |
| fastimage                 | 1.6.8        |
| multi_xml                 | 0.5.5        |
| claide                    | 1.0.1        |
| curb                      | 0.9.3        |
| dotenv                    | 2.1.1        |
| mini_magick               | 4.5.1        |
| rouge                     | 1.11.1       |
| xcpretty                  | 0.2.4        |
| net-ssh                   | 3.2.0        |
| net-sftp                  | 2.1.2        |
| krausefx-shenzhen         | 0.14.10      |
| slack-notifier            | 1.5.1        |
| xcpretty-travis-formatter | 0.0.4        |
| jwt                       | 1.5.6        |
| little-plugger            | 1.1.4        |
| logging                   | 2.1.0        |
| memoist                   | 0.15.0       |
| os                        | 0.9.6        |
| signet                    | 0.7.3        |
| googleauth                | 0.5.1        |
| httpclient                | <IP_ADDRESS>      |
| hurley                    | 0.2          |
| mime-types-data           | 3.2016.0521  |
| mime-types                | 3.1          |
| uber                      | 0.0.15       |
| representable             | 2.3.0        |
| retriable                 | 2.1.0        |
| google-api-client         | 0.9.19       |
| terminal-notifier         | 1.7.1        |
| word_wrap                 | 1.0.0        |
| xcode-install             | 2.0.8        |
| nanaimo                   | 0.2.0        |
| xcodeproj                 | 1.4.1        |
</details>

*generated on:* **2016-11-03**
</details>

This is the output from bundle exec fastlane env
[14:38:32]: Generating fastlane environment output, this might take a few seconds...
✅ fastlane environment ✅
Stack

Key
Value

OS
10.12.1

Ruby
2.3.1

Bundler?
true

Xcode Path
/Applications/Xcode.app/Contents/Developer/

Xcode Version
8.1

Git
git version 2.9.3 (Apple Git-75)

Installation Source
/usr/local/bin/fastlane

Host
Mac OS X 10.12.1 (16B2555)

Ruby Lib Dir
/usr/local/Cellar/ruby/2.3.1_2/lib

OpenSSL Version
OpenSSL 1.0.2j  26 Sep 2016

fastlane files:
`./fastlane/Fastfile`
# Customise this file, documentation can be found here:
# https://github.com/fastlane/fastlane/tree/master/fastlane/docs
# All available actions: https://github.com/fastlane/fastlane/blob/master/fastlane/docs/Actions.md
# can also be listed using the `fastlane actions` command

# Change the syntax highlighting to Ruby
# All lines starting with a # are ignored when running `fastlane`

# If you want to automatically update fastlane if a new version is available:
# update_fastlane

require_relative 'utils/mycomp_utils'

# This is the minimum version number required.
# Update this, if you use features of a newer version
fastlane_version "1.104.0"

default_platform :ios

platform :ios do
	before_all do
		ensure_git_status_clean
		ENV["SLACK_URL"] = "https://hooks.slack.com/services/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"
	end

	desc "Runs all the tests"
	lane :certs do
		match_appstore
		match_development
		match_enterprise_distibution
	end

	desc "Release beta version of the app"
	desc "Available options are:"
	desc "	version_number_index"
	desc "	version_number_manual"
	desc "	version_number_force"
	desc "	allow_arbitrary_loads"
	desc "	release_notes"
	lane :beta do |options|
		begin
			match_enterprise_distibution # Make sure we are using latest certs and provisioning profile
			build_app(options.merge(:is_app_store_build => false))
			mission_control(
				s3_access_key: "some_access_key",
				s3_secret_access_key: "some_secret_access_key",
				release_notes: options[:release_notes]
			)

			slack_notify_new_version(is_beta: true)
		rescue Interrupt
			cleanup_on_interrupt
		end
	end

	desc "Release app to the App Store"
	desc "Available options are:"
	desc "	version_number_index"
	desc "	version_number_manual"
	desc "	version_number_force"
	desc " 	submit_for_review"
	desc " 	automatic_release"
	lane :store do |options|
		begin
			match_appstore # Make sure we are using latest certs and provisioning profile
			build_app(options.merge(:is_app_store_build => true))

			deliver( # Upload IPA to iTunes Connect (see 'Deliverfile' for more config)
				submit_for_review: options[:submit_for_review],
				automatic_release: options[:automatic_release]
			)

			add_and_push_tag
			slack_notify_new_version(is_beta: false)
		rescue Interrupt
			cleanup_on_interrupt
		end
	end

	after_all do |lane|
		# Only called if executed lane was successful
		cleanup
	end

	error do |lane, exception|
		cleanup
	end
end```
</details>

<details><summary>`./fastlane/Appfile`</summary>

```ruby
apple_id<EMAIL_ADDRESS>team_name 'mycomp AB' # yes... same team_name for `standard` and `enterprise` account
$app_identifier_beta = 'com.mycomp.app.developer'
$app_identifier_store = 'com.mycomp.app'
$team_id_enterprise = 'MY_ENTERPRISE_TEAM_ID'
$team_id_store = 'MY_TEAM_ID'

for_lane :beta do
	app_identifier $app_identifier_beta
	team_id $team_id_enterprise
end

for_lane :store do
	app_identifier $app_identifier_store
	team_id $team_id_store
end```
</details>

### fastlane gems

| Gem                 | Version | Update-Status |
| ------------------- | ------- | ------------- |
| credentials_manager | 0.16.2  | ✅ Up-To-Date  |
| fastlane_core       | 0.53.0  | ✅ Up-To-Date  |
| spaceship           | 0.36.2  | ✅ Up-To-Date  |
| cert                | 1.4.3   | ✅ Up-To-Date  |
| deliver             | 1.14.5  | ✅ Up-To-Date  |
| frameit             | 3.0.0   | ✅ Up-To-Date  |
| gym                 | 1.11.3  | ✅ Up-To-Date  |
| sigh                | 1.11.2  | ✅ Up-To-Date  |
| match               | 0.10.0  | ✅ Up-To-Date  |
| pem                 | 1.4.0   | ✅ Up-To-Date  |
| pilot               | 1.11.2  | ✅ Up-To-Date  |
| produce             | 1.2.1   | ✅ Up-To-Date  |
| scan                | 0.13.1  | ✅ Up-To-Date  |
| screengrab          | 0.5.5   | ✅ Up-To-Date  |
| snapshot            | 1.16.3  | ✅ Up-To-Date  |
| supply              | 0.7.1   | ✅ Up-To-Date  |
| fastlane            | 1.106.2 | ✅ Up-To-Date  |

### Loaded fastlane plugins:

**No plugins Loaded***

<details><summary><b>Loaded gems</b></summary>

| Gem                       | Version      |
| ------------------------- | ------------ |
| did_you_mean              | 1.0.0        |
| bundler                   | 1.13.6       |
| io-console                | 0.4.5        |
| CFPropertyList            | 2.3.3        |
| i18n                      | 0.7.0        |
| json                      | 1.8.3        |
| minitest                  | 5.9.1        |
| thread_safe               | 0.3.5        |
| tzinfo                    | 1.2.2        |
| activesupport             | <IP_ADDRESS>      |
| addressable               | 2.3.8        |
| babosa                    | 1.0.2        |
| colored                   | 1.2          |
| highline                  | 1.7.8        |
| commander                 | 4.4.0        |
| security                  | 0.1.3        |
| credentials_manager       | 0.16.2       |
| excon                     | 0.54.0       |
| gh_inspector              | 1.0.2        |
| multi_json                | 1.12.1       |
| plist                     | 3.1.0        |
| rubyzip                   | 1.1.7        |
| terminal-table            | 1.4.5        |
| fastlane_core             | 0.53.0       |
| multipart-post            | 2.0.0        |
| faraday                   | 0.9.2        |
| unf_ext                   | <IP_ADDRESS>      |
| unf                       | 0.1.4        |
| domain_name               | 0.5.20161021 |
| http-cookie               | 1.0.3        |
| faraday-cookie_jar        | 0.0.6        |
| faraday_middleware        | 0.10.0       |
| fastimage                 | 1.6.8        |
| multi_xml                 | 0.5.5        |
| claide                    | 1.0.1        |
| curb                      | 0.9.3        |
| dotenv                    | 2.1.1        |
| mini_magick               | 4.5.1        |
| rouge                     | 1.11.1       |
| xcpretty                  | 0.2.4        |
| net-ssh                   | 3.2.0        |
| net-sftp                  | 2.1.2        |
| krausefx-shenzhen         | 0.14.10      |
| slack-notifier            | 1.5.1        |
| xcpretty-travis-formatter | 0.0.4        |
| jwt                       | 1.5.6        |
| little-plugger            | 1.1.4        |
| logging                   | 2.1.0        |
| memoist                   | 0.15.0       |
| os                        | 0.9.6        |
| signet                    | 0.7.3        |
| googleauth                | 0.5.1        |
| httpclient                | <IP_ADDRESS>      |
| hurley                    | 0.2          |
| mime-types-data           | 3.2016.0521  |
| mime-types                | 3.1          |
| uber                      | 0.0.15       |
| representable             | 2.3.0        |
| retriable                 | 2.1.0        |
| google-api-client         | 0.9.19       |
| terminal-notifier         | 1.7.1        |
| word_wrap                 | 1.0.0        |
| xcode-install             | 2.0.8        |
| nanaimo                   | 0.2.0        |
| xcodeproj                 | 1.4.1        |
</details>

*generated on:* **2016-11-03**
</details>

@IvanProdaiko94 does your export step actually fail? Or is it extremely slow (~20 min) as mine?

Running the very same command on my Mac book pro (posting fastlane env below) it takes under 1 minute. So 1 (Macbook Pro) vs 20 (Mac mini) minutes
[15:19:54]: Generated plist file with the following values:
[15:19:54]: ▸ -----------------------------------------
[15:19:54]: ▸ {
[15:19:54]: ▸   "method": "enterprise"
[15:19:54]: ▸ }
[15:19:54]: ▸ -----------------------------------------
[15:19:54]: $ /usr/bin/xcrun /usr/local/lib/ruby/gems/2.3.0/gems/gym-1.11.3/lib/assets/wrap_xcodebuild/xcbuild-safe.sh -exportArchive -exportOptionsPlist '/var/folders/gk/qvl965nn1gg93sw1st939b3h0000gp/T/gym_config20161103-44829-1jr8quv.plist' -archivePath /Users/alexandercyon/Library/Developer/Xcode/Archives/2016-11-03/mycomp\ 2016-11-03\ 15.00.23.xcarchive -exportPath '/var/folders/gk/qvl965nn1gg93sw1st939b3h0000gp/T/gym_output20161103-44829-1lvv5o8'
[15:20:47]: Compressing 15 dSYM(s)
[15:20:47]: $ cd '/Users/alexandercyon/Library/Developer/Xcode/Archives/2016-11-03/mycomp 2016-11-03 15.00.23.xcarchive/dSYMs' && zip -r '/Users/alexandercyon/Developer/iOS/mycomp/App/mycomp.app.dSYM.zip' *.dSYM

The result of bundle exec fastlane env:
✅ fastlane environment ✅
Stack

Key
Value

OS
10.12.1

Ruby
2.3.1

Bundler?
true

Xcode Path
/Applications/Xcode.app/Contents/Developer/

Xcode Version
8.1

Git
git version 2.9.3 (Apple Git-75)

Installation Source
/usr/local/bin/fastlane

Host
Mac OS X 10.12.1 (16B2555)

Ruby Lib Dir
/usr/local/Cellar/ruby/2.3.1_2/lib

OpenSSL Version
OpenSSL 1.0.2j  26 Sep 2016

fastlane files:
Same fastfile...
fastlane gems

Gem
Version
Update-Status

credentials_manager
0.16.2
✅ Up-To-Date

fastlane_core
0.53.0
✅ Up-To-Date

spaceship
0.36.2
✅ Up-To-Date

cert
1.4.3
✅ Up-To-Date

deliver
1.14.5
✅ Up-To-Date

frameit
3.0.0
✅ Up-To-Date

gym
1.11.3
✅ Up-To-Date

sigh
1.11.2
✅ Up-To-Date

match
0.10.0
✅ Up-To-Date

pem
1.4.0
✅ Up-To-Date

pilot
1.11.2
✅ Up-To-Date

produce
1.2.1
✅ Up-To-Date

scan
0.13.1
✅ Up-To-Date

screengrab
0.5.5
✅ Up-To-Date

snapshot
1.16.3
✅ Up-To-Date

supply
0.7.1
✅ Up-To-Date

fastlane
1.106.2
✅ Up-To-Date

Loaded fastlane plugins:
No plugins Loaded*
Loaded gems

Gem
Version

did_you_mean
1.0.0

bundler
1.13.6

io-console
0.4.5

CFPropertyList
2.3.3

i18n
0.7.0

json
1.8.3

minitest
5.9.1

thread_safe
0.3.5

tzinfo
1.2.2

activesupport
<IP_ADDRESS>

addressable
2.3.8

mini_portile2
2.1.0

nokogiri
<IP_ADDRESS>

aws-sdk-v1
1.66.0

babosa
1.0.2

colored
1.2

highline
1.7.8

commander
4.4.0

security
0.1.3

credentials_manager
0.16.2

excon
0.54.0

gh_inspector
1.0.2

multi_json
1.12.1

plist
3.1.0

rubyzip
1.1.7

terminal-table
1.4.5

fastlane_core
0.53.0

multipart-post
2.0.0

faraday
0.9.2

unf_ext
<IP_ADDRESS>

unf
0.1.4

domain_name
0.5.20161021

http-cookie
1.0.3

faraday-cookie_jar
0.0.6

faraday_middleware
0.10.0

fastimage
1.6.8

multi_xml
0.5.5

claide
1.0.1

curb
0.9.3

dotenv
2.1.1

mini_magick
4.5.1

rouge
1.11.1

xcpretty
0.2.4

net-ssh
3.2.0

net-sftp
2.1.2

krausefx-shenzhen
0.14.10

slack-notifier
1.5.1

xcpretty-travis-formatter
0.0.4

jwt
1.5.6

little-plugger
1.1.4

logging
2.1.0

memoist
0.15.0

os
0.9.6

signet
0.7.3

googleauth
0.5.1

httpclient
<IP_ADDRESS>

hurley
0.2

mime-types-data
3.2016.0521

mime-types
3.1

uber
0.0.15

representable
2.3.0

retriable
2.1.0

google-api-client
0.9.19

terminal-notifier
1.7.1

word_wrap
1.0.0

xcode-install
2.0.8

nanaimo
0.2.0

xcodeproj
1.4.1

generated on: 2016-11-03

So maybe I should try to uninstall ruby somehow? And also uninstall bundle and fastlane... Will try to investigate that.

I do believe this thread about slow xcrun can be related, since it is the xcrun command that takes 20 minutes:
/usr/bin/xcrun /usr/local/lib/ruby/gems/2.3.0/gems/gym-1.11.3/lib/assets/wrap_xcodebuild/xcbuild-safe.sh -exportArchive -exportOptionsPlist...

I will investigate if the problem is related to scheme mentioned here.

Actually my schemes where in the correct place, in the XXX.xcodeproj/xcshareddata/xcschemes/ folder (added to git)

@TKBurner Any idea? Is it xcrun that's slow? Or is it the script xcbuild-safe? How can I determine that?

I tried a vanilla archiving and exporting on the Mac mini from Xcode, using the cert and PP downloaded via match (i.e. not using fastlane gym) and that was equally slow.
So the problem I am having is not related to gym, so I am very sorry for my misleading comments.
It seams like I am not the only one having problems with slow exporting in Xcode 8.

@Sajjon i hade same troubles on a few CI machines, after sierra upgrade, i ended up creating a new blank keychain, and setting this to default.

@hjanuschka I actually found out that OS X server had generated up 357 certificates in my keychain!
So indeed this StackOverflow answer was right
Deleting duplicate certificates decreased time from 21 minutes to less than 4 minutes.

Unfortunately it's not my issue. When I call xrun directly it works like a charm. And I tried to wait a lot, nearly a few hours, and it also didn't help :( will dig it bit later today

@IvanProdaiko94 How did it go? Let me know if you are still running into this issue!


bug

It seems   root_metadata not used in update_root,  parameter are mismatch in mcts.py.
line 81
eval_state = self.update_root(eval_state, env_state, root_metadata, params)
line 97
def update_root(self, tree: MCTSTree, root_embedding: chex.ArrayTree,
                 params: chex.ArrayTree, **kwargs) -> MCTSTree:

     key, tree = get_rng(tree)
     root_policy_logits, root_value = self.eval_fn(root_embedding, params, key)
     root_policy = jax.nn.softmax(root_policy_logits)
     root_node = tree.at(tree.ROOT_INDEX)
     root_node = self.update_root_node(root_node, root_policy, root_value, root_embedding)
     return set_root(tree, root_node)

Thank you for pointing this out!
All examples currently use the AlphaZero class, which implements update_root with the correct parameter ordering.
for MCTS parameter ordering is indeed incorrect

https://github.com/lowrollr/turbozero/commit/4e55c7291576966a5acd75df75ce34e4422d4e63

evaluate should be vmapped, try something like this:
eval_key, env_key, rng_key = jax.random.split(rng_key, 3)
eval_keys = jax.random.split(eval_key, batch_size)
env_keys = jax.random.split(env_key, batch_size)

env_state, env_state_metadata = jax.vmap(init_fn)(env_keys)

# template embedding should not have a batch dimension
template_env_state, _ = init_fn(jax.random.PRNGKey(0))
evaluator_init = partial(az_evaluator.init, template_embedding=template_env_state)
eval_state = jax.vmap(evaluator_init)(eval_keys)

evaluate = partial(az_evaluator.evaluate,
                   env_step_fn=env_step_fn,
                   params=param)

output = jax.vmap(evaluate)(
        eval_state=eval_state,
        env_state=env_state,
        root_metadata=env_state_metadata)

I recommend using the Trainer class as described here, https://github.com/lowrollr/turbozero/blob/main/notebooks/hello_world.ipynb
I haven't fully documented a lot of the underlying classes yet which do have their peculiarities -- Trainer should be more straightforward to work with.

thanks，now have a new problem.
File "/turbozero/core/evaluators/mcts/mcts.py", line 81, in evaluate
eval_state = self.update_root(eval_state, env_state, root_metadata, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/turbozero/core/evaluators/alphazero.py", line 56, in update_root
return set_root(tree, root_node)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/turbozero/core/trees/tree.py", line 121, in set_root
data=jax.tree_util.tree_map(
^^^^^^^^^^^^^^^^^^^^^^^
File "/turbozero/core/trees/tree.py", line 122, in
lambda x, y: x.at[tree.ROOT_INDEX].set(y),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/lib/python3.11/site-packages/jax/_src/numpy/array_methods.py", line 497, in set
return scatter._scatter_update(self.array, self.index, values, lax.scatter,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/lib/python3.11/site-packages/jax/_src/ops/scatter.py", line 80, in _scatter_update
return _scatter_impl(x, y, scatter_op, treedef, static_idx, dynamic_idx,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/lib/python3.11/site-packages/jax/_src/ops/scatter.py", line 115, in _scatter_impl
y = jnp.broadcast_to(y, tuple(indexer.slice_shape))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/lib/python3.11/site-packages/jax/_src/numpy/lax_numpy.py", line 1227, in broadcast_to
return util._broadcast_to(array, shape)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "//lib/python3.11/site-packages/jax/_src/numpy/util.py", line 425, in _broadcast_to
for arr_d, shape_d in safe_zip(arr_shape, shape_tail))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: safe_zip() argument 2 is shorter than argument 1

Could you share your code?

        def one_step(prev):
        """Execute one self-play move using MCTS.
        """
        env_state, rng_key, step, eval_state = prev
        del inputs
        rng_key, rng_key_next = jax.random.split(rng_key, 2)
        env_state_metadata = StepMetadata(
            rewards=env_state.rewards,
            action_mask=env_state.legal_action_mask,
            terminated=env_state.terminated,
            cur_player_id=env_state.current_player,
        )
        terminated = env_state.terminated

        output = jax.vmap(evaluate)(
            eval_state=eval_state,
            env_state=env_state,
            root_metadata=env_state_metadata)

        env_state = step_fn_move(env_state, output.action)

        eval_state = output.eval_state
        eval_state = az_evaluator.step(eval_state, output.action)

        return (env_state, rng_key_next, step + 1, env_new3, eval_state)

it use jax.lax.scan:
output = jax.lax.scan(single_move, (env_state, rng_key, step,eval_state), None, length=400, unroll=1)
maybe this place is incorrect?
eval_state = output.eval_state
eval_state = az_evaluator.step(eval_state, output.action)

yes，there a feature i need that's missing,  like go self-data process,  it need symmetries.

Could you describe how the feature should work? I can help you find the right spot to integrate it.

And i need use mult gpus, I don't see something like pmap in code. Need a example that how to train use mult gpus.

Supporting multiple GPUs should be straightforward to add but is not currently included. I can work on this and let you know when it is supported. Created a separate issue to track.

Could you describe how the feature should work? I can help you find the right spot to integrate it.

in go game, self-play data usually augmentation by something like np.rot90 and  np.fliplr.

about that error, could be this place need vmap?
eval_state = output.eval_state
eval_state = az_evaluator.step(eval_state, output.action)

about that error, could be this place need vmap?
eval_state = output.eval_state
eval_state = az_evaluator.step(eval_state, output.action)

Yes, every function that operates on an evaluator state assumes a singular input rather than a batch and should be vmapped.

Could you describe how the feature should work? I can help you find the right spot to integrate it.

in go game, self-play data usually augmentation by something like np.rot90 and np.fliplr.

If I were you, I would implement a custom class extending Trainer, and overwrite collect to place each of the symmetries into the replay buffer with self.memory_buffer.add_experience. You will need to make sure policy_mask and policy_weights are augmented consistently with your game state data.
here: https://github.com/lowrollr/turbozero/blob/main/core/training/train.py#L103-L140
You should only need to extend the behavior of collect.

If I were you, I would implement a custom class extending Trainer, and overwrite collect to place each of the symmetries into the replay buffer with self.memory_buffer.add_experience. You will need to make sure policy_mask and policy_weights are augmented consistently with your game state data.

class GoTrainer(Trainer):
   def collect(self,
        state: CollectionState,
        params: chex.ArrayTree
    ) -> CollectionState:
        step_key, new_key = jax.random.split(state.key)
        eval_output, new_env_state, new_metadata, terminated, rewards = \
            self.step_train(
                key = step_key,
                env_state = state.env_state,
                env_state_metadata = state.metadata,
                eval_state = state.eval_state,
                params = params
            )
        buffer_state = self.memory_buffer.add_experience(
            state = state.buffer_state,
            experience = BaseExperience(
                env_state=state.env_state,
                policy_mask=state.metadata.action_mask,
                policy_weights=eval_output.policy_weights,
                reward=jnp.empty_like(state.metadata.rewards)
            )
        )

       # generate symmetries here and add to replay memory just like above

        buffer_state = jax.lax.cond(
            terminated,
            lambda s: self.memory_buffer.assign_rewards(s, rewards),
            lambda s: s,
            buffer_state
        )

        return state.replace(
            key=new_key,
            eval_state=eval_output.eval_state,
            env_state=new_env_state,
            buffer_state=buffer_state,
            metadata=new_metadata
        )

yes, I'm going to try this, but if can't use mult gpus, training  will be slow.
I change to this:
eval_state = output.eval_state
eval_state = jax.vmap(az_evaluator.step)(eval_state, output.action)
But error occurred before the code reached that.
that error is in this:
output = jax.vmap(evaluate)(
            eval_state=eval_state,
            env_state=env_state,
            root_metadata=env_state_metadata)

and  i print(state.observation.shape) in eval_fn,  it shows  (19, 19, 17) ， without  batch dimension， is this normal?

I seems find why this error happened.

I am going to close this issue as the original problem has been resolved.
I hope to continue to expand the documentation of this project so that it is more clear as to how to approach unique use-cases.
Thank you for your questions and feedback! Please create another issue if you run in to problems and feel free to email me if you have more questions.

The speed that I test was slow. Then I test https://github.com/lowrollr/turbozero/blob/main/notebooks/hello_world.ipynb this example,  seems  also slow from gpu utilization.
what is max_nodes means?  I found that max_nodes has a significant impact on speed; the larger max_nodes is, the slower the speed.
mctx-az is also in this situation.  and mctx-az is faster than turbozero that in my test.

max_nodes reflects the maximum capacity of the tree. Trees cannot be sized dynamically so must have a maximum number of nodes set prior to collection. It makes sense that performance gets worse as max_nodes increases, this ultimately makes it so operations are on larger matrices.
I am aware that the backend is currently less performant than mctx. It is a priority for me to fix this.

I'm still confuse about max_nodes.
if max_nodes=100,  is it means every node have 100  child node?  or is it means that will search 100 node  at most in this evaluate?
why you suggest it larger than num_simulations？that will be very very slow.
yes, I look forward to it running faster.

I believe the most factor affecting speed lies in the computational aspect of the CPU.  GPU not fully utilized.

max_nodes refers to the maximum capacity of the tree -- so yes it does mean a tree with max_nodes=100 will at most evaluate 100 distinct game states.
I advise setting it higher than num_iterations because this implementation re-uses subtrees from a previous search -- so most of the time the tree will already be partially populated when a new search is started. Setting max_nodes higher than num_iterations means that there will be room for num_iterations nodes in the tree more often. I have yet to document out-of-bounds behavior but it works similarly to https://github.com/lowrollr/mctx-az.
Increasing max_nodes linearly increases the memory footprint of the search tree data structure. It's definitely a trade-off of speed vs. accuracy to set it higher/lower, and its value relative to 'num_iterations' should be problem-dependent (branching factor and # of iterations both matter). Setting max_nodes = num_iterations is fine if you're worried about speed.

I think the most factor affecting speed lies in the computational aspect of the CPU. GPU not fully utilized.

Do you have evidence of this? I'm not aware of any CPU-bound portion of the training loop and in my experiments I've had no issues with GPU utilization.

In my experiments,  keep num_simulations unchange,  when max_nodes = 32, it cost 460 seconds，
when max_nodes = 600, it cost 2200 seconds，gpu Pwr:Usage/Cap is more lower than when max_nodes = 32.  Test with mctx-az.   Turbozero is same situation.
In this situation, because num_simulations unchange, so the computational workload of the GPU remains unchanged, so problem is in cpu?

Some more details could be useful here.
What are you running, one call to search?
What is num_simulations set to?
What environment are you using?
Computational workload is higher when max_nodes is increased. Search operates on tensors of size [num_batches, max_nodes, ... ]

environment: go 19*19
model_size: just like alphazero paper, 40 block, 256 channl
num_simulations: 128
batch size: 50  per device
step: 410
There is another strange phenomenon： as the number of steps increases, the gpu utilization rate becomes lower and lower

Thank you for letting me know, I will look into what you are describing to see if I can replicate it and diagnose.
Are you running more than one call to search (MCTS.evaluate)? Some of the weird behavior you are describing could be down to JIT-compilation overhead on the first call.

It's use  jax.lax.scan，like this:  jax.lax.scan(one_step, (env_state, rng_key, step,eval_state), None, length=410, unroll=1)
JIT-compilation should only affect one times.
environment: go 19*19
model_size: just like alphazero paper, 40 block, 256 channl
num_simulations: 128
batch size: 50 per device
step: 410
when max_nodes = 32, it cost 460 seconds，
when max_nodes = 600, it cost 2200 seconds，
gpu Pwr:Usage/Cap is more lower than when max_nodes = 32.
Test with mctx-az. Turbozero is same situation.

Thanks for pointing this out, I will see if I can replicate.

Are you familiar with the MuZero algorithm? I have some questions and hope you can help me.

I haven't worked with MuZero specifically as much but have read the paper. Feel free to send me an email with your questions and I'll see if I can answer.
I still plan on looking into the issues you mention but have been very busy and have not had a chance yet.


updated typora (latest)

no idea why the appcast isn't being updated by the cask-repair script. I can close and resubmit properly if you'd like


fix django version to 1.2.5 in testsuite

updated buildout.cfg setup for newer djangorecipe

this project is dead. closing all issues/PRs. sorry for the notification.


onProgress callback of startUpload function gives 100% directly

I want to upload my getApplicationDocumentsDirectory() into iCloud. And for that I am using following code:
final storage = await ICloudStorage.getInstance('iCloud.com.***');
              final dir = await getApplicationDocumentsDirectory();
              await storage.startUpload(filePath: dir.path, onProgress: (value) {
                value.listen((event) {
                  print('PROGRESS--------$event');
                });
              });

Here event is printing 100. and size of getApplicationDocumentsDirectory() is around 40MB.
If I go to iCloud from settings then it will showing initially 13 to 14 MB and by time it increase.
So that mean backup still in progress but this function directly tell us that it is completed.

It won't upload all the contents inside the application documents directory. It will only create a folder in the iCloud container.

@deansyd If it's not upload all the contents of app documents directory, then why my app showing 40MB data in iCloud settings? My app documents directory size is same. 40MB

@deansyd Waiting for the solution or any temporary workaround.

I haven't tested the scenario on uploading the entire directory. If you are sure all the contents in the directory are uploaded, the progress is probably only for the folder creation. I will test this case when I got time. The work around would be uploading individual files or zip the folder and upload it.

Okay. thanks for your suggestion. please take your time and let me know your feedback

@deansyd uploading single file is working perfectly, tested with 50MB, 100MB, 800MB.
But uploading entire folder seems not to be working. please check this issue.

@nayanbabariya95, I just tested uploading a directory. It does upload all the contents of the directory but the progress only reports the status of the folder itself. The library wasn't intended for supporting uploading or downloading an entire directory. So this will not be fixed for now. You can use those workarounds I suggested earlier or you are welcome to contribute a PR to get his feature implemented.

@deansyd Thanks for review. I want to know for my case. I need to upload 3 to 4 files in iCloud, so how can I know all the four files are uploaded? because if I upload them separate, then onProgresscallback will also different. Any way to merge those all callback and listen on a single time?

You can merge the progress streams into a single stream using StreamZip https://pub.dev/documentation/async/latest/async/StreamZip-class.html


IntelliSense handles conditional references like non existing ones

Environment data
dotnet --info output:
.NET Command Line Tools (1.0.0)

Product Information:
 Version:            1.0.0
 Commit SHA-1 hash:  e53429feb4

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.14393
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\1.0.0

VS Code version: 1.12.1
C# Extension version: 1.9.0
Steps to reproduce
Add a conditional reference to your .csproj file like so
<Choose>
        <When Condition=" '$(Configuration)'=='Release' ">
          <ItemGroup>
            <PackageReference Include="SomeProject">
              <Version>1.0.6</Version>
            </PackageReference>
          </ItemGroup>
        </When>
        <When Condition=" '$(Configuration)'=='Debug' ">
           <ItemGroup>
            <ProjectReference Include="..\SomeProject\SomeProject.csproj" />
          </ItemGroup>
        </When>
  </Choose>

Expected  behavior
IntelliSense optimally uses the Debug reference (project) to do it's work
(works in Visual Studio but not in Visual Studio Code)
Alternatively it should support the <otherwise></otherwise> tag if assuming 'Debug' as default configuration is not an option.
https://msdn.microsoft.com/en-us/library/ms164282.aspx
Actual behavior
IntelliSense acts like there is no reference at all

There have been a lot of changes to the O# project system since this issue was open. If you are still experiencing this issue, please reopen.


[Question] How would I use this in running tests?

Hello! Thank you for this project, I see a great need for it!
I'm trying to run a test using this package and I'm having trouble understanding how to set the wasm-bindgen target to web. Here is the commend I tried -
wasm-pack test --node --test wasm --target web

Which gave me the following error -
Error loading target specification: Could not find specification for target "web"

My test is in the wasm file specified in the -- test flag. This test uses the #[wasm_bindgen_test] decorator, if relevant. The function it tests calls tokio::task::spawn_blocking and is wrapped in the #[wasm_bindgen] and #[no_mangle] decorators.
Thanks for your help!

Hi @dannytoomey , welcome :)
Could you try putting target in front of other arguments??
Like this:
wasm-pack test --node --target web --test wasm

If this doesn't work, please let me know.

Thanks so much for your prompt reply! That did not work, unfortunately. Here's the error I received:
Error: Must specify at least one of `--node`, `--chrome`, `--firefox`, or `--safari`
Caused by: Must specify at least one of `--node`, `--chrome`, `--firefox`, or `--safari`

FYI, I will share the command that actually works though it's not for tests.(It's being used in Rinf)
wasm-pack --quiet build ./my_crate --out-dir ./output --no-typescript --target web -- -Z build-std=std,panic_abort

I'm not really experienced in wasm-pack with test code, but I'll try to share the solution after searching for it.

Thanks! That does work for builds, though it does not appear to work for tests. In case it's useful, here's a little more context on what I'm trying to do.
I'm making a demo for learning purposes to understand how WebAssembly works. I'm testing performance with the following Fibonacci function, which I'm hoping to use tokio_with_wasm  to run asynchronously.
use tokio_with_wasm::tokio;
use wasm_bindgen::prelude::*;

#[no_mangle]
pub fn fib_rs(n: u32) -> u32 {
    if n <= 1 {
        return n;
    } else {
        return fib_rs(n - 1) + fib_rs(n - 2);
    }
}

#[wasm_bindgen]
#[no_mangle]
pub async fn get_fibs_tokio_wasm(n1: u32, n2: u32) -> u32 {
    let fut_1 = tokio::task::spawn_blocking(move || fib_rs(n1));
    let fut_2 = tokio::task::spawn_blocking(move || fib_rs(n2));
    let fib_1 = fut_1.await.unwrap();
    let fib_2 = fut_2.await.unwrap();
    return fib_1 + fib_2;
}

I'm able to test with with cargo test, but since it requires tokio to run, I'm hoping to stick to #[wasm_bindgen_test] so I can do everything in the wasm runtime. I'm trying to test the above function with the following test.
#[wasm_bindgen_test]
async fn test_get_fibs_tokio_wasm_exec() {
    let n = 35;
    let now = std::time::Instant::now();
    {
        let _ = fib_rs(n);
        let _ = fib_rs(n);
    }
    let single_elapsed = now.elapsed().as_secs_f32();
    let now = std::time::Instant::now();
    {
        let _ = js_sys::Promise::resolve(&JsValue::from(get_fibs_tokio_wasm(n, n).await));
    }
    let para_elapsed = now.elapsed().as_secs_f32();
    // if get_fibs is properly async, this should be ~2x as fast,
    // though the ratio won't be exactly the same between runs
    assert!(0.6 > para_elapsed / single_elapsed && para_elapsed / single_elapsed > 0.4);
}

The only way I'm aware of running the #[wasm_bindgen_test] is with wasm-pack, though if that's not a good tool to use for this purpose, I'd be eager to use something else! My main concern is being able to run the test function in a wasm runtime.


Search/Find zone should have a dropbox to recall my lasts searched strings

So I can just reselect a previous search

Simply press UpArrow

Yes, and there is another feature request for a dropdown


Literal elements should not be in union with their generic type

Something like Literal["a", "b"] | str is probably sub-optimal: if it can be any string, we can remove the Literal and if it can only be "a" or "b", we can remove str.

I just published flake8-pyi 22.8.2, which includes this check :) https://pypi.org/project/flake8-pyi/

Could theoretically also extend this to Protocols (and sub-classes): Hashable | slice | None | str | tuple could just be Hashable

That one feels a bit out of scope — similar to https://github.com/PyCQA/flake8-pyi/issues/246#issuecomment-1182023651, you're getting into the question of "is type X a subtype of type Y?", and that's a pretty complicated question that a proper type checker will always be far better equipped to answer than a humble linter :)

Could theoretically also extend this to Protocols (and sub-classes): Hashable | slice | None | str | tuple could just be Hashable

Relevant pyright discussion: https://github.com/microsoft/pyright/discussions/3949


Vim theme doesn't like the console black background

I'm use a 256 colors theme in Vim. It works on Linux and macOS but when I run it in Bash on Windows
I'm using an Ubuntu distro I got in the Windows Store:
~ $ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:        16.04
Codename:       xenial

When I open Vim, it looks like this:

I can't find a way to remove that black background.
Windows Build Number: Microsoft Windows [Version 10.0.17134.48]

Console development is coming along and it's possible to do some color customizations but they aren't easy to implement.
If you have specific theme requirements, I would probably recommend a third-party terminal app in the mean time that are just easier to set up.

Awesome, thanks @sirredbeard ! wsltty does the trick.


v0.3.0

Add the feature based on roadmap.md.

Units

[x] meters per second (metric)
[x] feet per second (english)
[ ] kilometers per hour (metric)
[ ] miles per hour (english)


Add rebar parser

Hi, this PR adds support for Erlang Rebar3 package manager; (spec: http://www.rebar3.org/docs/dependencies)
It's was big task due to the fact that rebar.config uses erlang code for format and there was not code parser for that;
It can reads dependencies from the :deps block and from each :deps block under each profile;

@timgluz I left 1 comment in the code, but otherwise it looks OK.

It is double counting,  check couple of lines up. And you don't like #todo
blocks, so I removed it
On Fri, Jul 21, 2017, 15:43 Robert Reiz<EMAIL_ADDRESS>wrote:

@reiz commented on this pull request.
In lib/versioneye/parsers/nuget_parser.rb
https://github.com/versioneye/versioneye-core/pull/122#discussion_r128764526
:

@@ -317,7 +317,6 @@ def parse_dependency_version( project, dependency )

 project.projectdependencies.push(dependency)
 project.out_number     += 1 if ProjectdependencyService.outdated?( dependency )

project.unknown_number += 1 if product.nil?

@timgluz https://github.com/timgluz Why did you remove that? And why is
this related to Erlang? That's the Nuget parser.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/versioneye/versioneye-core/pull/122#pullrequestreview-51477237,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABKs0aZYvrsuopWCjjS495AV2HDOhABcks5sQKsVgaJpZM4OfUbE
.


Update Docs and example of using new force-temp-security-group flag

Type of Change

Docs

What does this PR do? / Related Issues / Jira
Updates the Documentation and Gives an example of using the new --force-temp-security-group flag

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 15.09%. Comparing base (29329a9) to head (00ffaff).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #225   +/-   ##
=======================================
  Coverage   15.09%   15.09%
=======================================
  Files          17       17
  Lines        1305     1305
=======================================
  Hits          197      197
  Misses       1094     1094
  Partials       14       14


OPSAPS-51232. Optimized the mkdirs (not chowns)

Made the mkdirs optimal (one single command).

I guess this can be closed, right?


Restyle palette page

This is on top of #16. Looks like

/cc @dflock

And it's live at: http://simurai.com/duotone-syntax/

Might be neat to also add some code samples.


Update Yarn to v4.3.1

[x] ManageIQ/manageiq-providers-cisco_intersight#115
[x] ManageIQ/manageiq-providers-ibm_cloud#499
[x] ManageIQ/manageiq-providers-ibm_power_vc#112
[x] ManageIQ/manageiq-providers-lenovo#410
[x] ManageIQ/manageiq-providers-nuage#315
[x] ManageIQ/manageiq-providers-redfish#199
[x] ManageIQ/manageiq-providers-nsxt#121
[x] https://github.com/ManageIQ/manageiq/pull/23064
[x] https://github.com/ManageIQ/manageiq-ui-classic/pull/9206
[ ] ~https://github.com/ManageIQ/manageiq-ui-service/pull/1906~ (overwritten by 4.3.1 changes below)

[x] ManageIQ/manageiq-providers-cisco_intersight#118
[x] ManageIQ/manageiq-providers-ibm_cloud#501
[x] ManageIQ/manageiq-providers-ibm_power_vc#114
[x] ManageIQ/manageiq-providers-lenovo#412
[x] ManageIQ/manageiq-providers-nuage#317
[x] ManageIQ/manageiq-providers-redfish#202
[x] https://github.com/ManageIQ/manageiq-providers-nsxt/pull/123
[x] https://github.com/ManageIQ/manageiq/pull/23072
[x] https://github.com/ManageIQ/manageiq-ui-classic/pull/9213
[x] https://github.com/ManageIQ/manageiq-ui-service/pull/1906

This is done.


fix: tolerant no write permission for copy favicon to static dir

Pull Request Checklist
Note to first-time contributors: Please open a discussion post in Discussions and describe your changes before submitting a pull request.
Before submitting, make sure you've checked the following:

[x] Target branch: Please verify that the pull request targets the dev branch.
[x] Description: Provide a concise description of the changes made in this pull request.
[ ] Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
[ ] Documentation: Have you updated relevant documentation Open WebUI Docs, or other documentation sources?
[x] Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
[x] Testing: Have you written and run sufficient tests for validating the changes?
[x] Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
[x] Prefix: To cleary categorize this pull request, prefix the pull request title, using one of the following:

BREAKING CHANGE: Significant changes that may affect compatibility
build: Changes that affect the build system or external dependencies
ci: Changes to our continuous integration processes or workflows
chore: Refactor, cleanup, or other non-functional code changes
docs: Documentation update or addition
feat: Introduces a new feature or enhancement to the codebase
fix: Bug fix or error correction
i18n: Internationalization or localization changes
perf: Performance improvement
refactor: Code restructuring for better maintainability, readability, or scalability
style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc.)
test: Adding missing tests or correcting existing tests
WIP: Work in progress, a temporary label for incomplete or ongoing work

Changelog Entry
Description
The copy of favicon may fail when static dir in a read only file system (which may happened in such env which make the deploy files unwritable). I add a try catch to prevent the error to make the whole process break.
Added

[List any new features, functionalities, or additions]

Changed

[List any changes, updates, refactorings, or optimizations]

Deprecated

[List any deprecated functionality or features that have been removed]

Removed

[List any removed features, files, or functionalities]

Fixed

[List any fixes, corrections, or bug fixes]

Security

[List any new or updated security-related changes, including vulnerability fixes]

Breaking Changes

BREAKING CHANGE: [List any breaking changes affecting compatibility or functionality]

Additional Information

[Insert any additional context, notes, or explanations for the changes]

[Reference any related issues, commits, or other relevant information]

Screenshots or Videos

[Attach any relevant screenshots or videos demonstrating the changes]

LGTM, Thanks!


Feature merge dynamicaly +aot compatibility

Hi @jamesmcroft ,
Thank you for that cli.
Do you think it is possible to make it available as library compatible aot?
My need is for SlimFaas https://github.com/AxaFrance/SlimFaas it is a function as a service over kubernetes it act like a proxy.
I would like to merge many swagger in one from all function to one swagger given by SlimFaas.
Do you think it is possible? Do you have time to Do evolution here ?
If not i will try to build one inside SlimFaas.
Regards,
Guillaume

@guillaume-chervet I will take a look. I've not built AOT compatible libraries previously, I will do some research to understand what changes need to be made to the current Swagger Merge SDK to support this.

@guillaume-chervet a pre-release of this change has been pushed to NuGet - https://www.nuget.org/packages/SwaggerMerge.SDK/1.0.0-preview.240802.1
Let me know how you get on with this change and I will be happy to explore pushing this out officially.

Thank you so much @jamesmcroft , I test it as soon as I come back from holidays :)


3DuF does not flip control layer

The SVG of the control layer must be flipped for correct milling. Currently 3DuF does not do this.
Current 3DuF Control Output Example

Correct 3DuF Control Output Example

I'd like to bring this up again as an issue. Earlier versions of 3DuF used to flip the chip horizontally. That needs to be reimplemented again.

Implemented in the CNC outputs as of commit - 2545e9e5107388d6f9d0fec27bfa0b405c68d20f


Test-DbaLastBackup not working in dev

Is this:
Just place "x" to indicate selection (e.g. "[x]")

[ ] feature
[x] bug:

Details
Test-DbaLastBackup in dev is not working, perhaps because of changes to get-dbabackuphistory? Please help -at a loss. It just says Failure

Resolved


restApiRoot not recognized for login request

Thanks for putting together what seems to be a very promising project.
I have my Loopback API running at <IP_ADDRESS>:8000/v2, however, when I try to login to the CMS it's hitting http://<IP_ADDRESS>:8000/api/user/login?include=user. If I console.log restApiRoot in boot-cms.js, it's properly set to /v2. What do I need to do differently in order to tell the CMS to POST the login to the correct url?

Modify the CMS/Config.json with a property for public.apiBaseUrl and specify the /v2 path.

Thanks. Unfortunately that has no effect though.
It looks like index.js line 286 is passing options.basePath to the client. However, even when setting that, Config.js defaults to /api/ in all cases, whereas it seems like it should be looking to window.config.apiBaseUrl first and defaulting to /api/ if it's not set. If I switch that line around to the following, login posts to the expected url:
apiBaseUrl: window.config.apiBaseUrl || '/api/',

Addressed in f3700246a5aba65791e89c2bdea25552bd1cd3ed


cudnn batch norm

Just want  to confirm if CuDNN update the moving statistics in forward instead of backward, this might affect the advanced mode for the memory optimization, so maybe we want to disable it for now if there is no significant speed gain.

Currently it's registered as a separate op anyway because it's not compatible with the original interface.

@piiswrong In the advanced mode, bn forward will be called many times. BTW, I notice CuDNN Activation costs more memory and a little bit slower. We may consider a better way to make choice of CuDNN or our implementation.

Ok I get what you mean. But how can it cost more memory? It doesn't need out_affine, so should save half of the memory right?

Dropout has been added in blacklist, I think you need to add cudnn bn here https://github.com/dmlc/mxnet/blob/master/src/symbol/static_graph.cc#L255

Also mention the advanced mode is still experimental and disabled by default

@tqchen Let's merge it first, then add into blacklist?

OK, if the review passes, and it works for the testcases by replacing batchnorm let us merge it in


How can i change dot, line size. Remove axits Y

dot and line size bigger than i need.
And i want don't show axits Y like this image http://upfree.ssc.vn/images/905Dashboard.jpg
Thank you!

i'm sorry, it work
line.setStrokeWidth(1);
line.setPointRadius(3);
lineData.setAxisYLeft(new Axis().setHasLines(true).setHasSeparationLine(false));


add tombstone check

fix https://github.com/tikv/client-go/issues/659
The root cause is reResolve() there will return true, even if the store's state is tombstone.
https://github.com/tikv/client-go/blob/master/internal/locate/region_cache.go#L2442
The existing unit test assumes if pd client gets a tombstone store, it will return nil, but in the force scale-in scenario, it won't.
https://github.com/tikv/client-go/blob/master/internal/mockstore/mocktikv/pd.go#L202-L208

hi @Smityz thanks for the fix, but our community requires DCO for commits. (for more details about DCO  https://www.secondstate.io/articles/dco/)

hi @Smityz thanks for the fix, but our community requires DCO for commits. (for more details about DCO https://www.secondstate.io/articles/dco/)

done for it


Create an interface for event dispatcher observers

Not shure how to do this in js, but there should be rules which functions an observer has to implement. Depends on #31.

Done. Observers have to implement this.handleRelayData() to get events. They can expose their controllers when registering to a relay point.


RFC: Metaphysics PRs to include data generating code

Proposal:
We've all agreed that any time we PR any changes to MP schemas we have already merged and deployed changes to staging in upstream applications. I think we should consider including some ruby code in every MP PR that generates the stub data required to demonstrate all cases of the new MP code.
At its most basic it should just be some code that we can paste into the console, something like:
user = User.where(email:'someemail@whatever.com').first
show = Show.create!(whatever)
show2 = Show.create!(whatever2)
follow = FollowShow.create(user: user, show: show)
user.id

That's a simple example - it creates everything necessary to make some assertions, and it ends by logging out the ID you'd look up in your root query in metaphysics. The PR would then include various GraphQL queries against that exact data as well as their results, and anyone can independently verify it.
Reasoning
It can be a nontrivial enterprise to put Gravity into a state that exposes the exact nuances that some new MP feature requires. Logging into the console and tweaking stuff until it works is fine, if you're willing to have your changes blown away over the weekend. This also requires any PR reviewers to figure out how to generate gravity state that'll demonstrate the PR's correctness.
Exceptions:
If we're writing unit tests all of this just gets fabricated - we understand how important it is to actually test these things in a predictable, reliable way. This is specifically for those PRs that include features that cross service boundaries.
Additional Context:
Perhaps we can have some template for the ruby code that wraps it in a guard clause - regenerate this or return it if it already exists.
How is this RFC resolved?
Every metaphysics PR includes ruby code to (re)generate its upstream data.
I thought about potentially tracking these blocks of code in git, maybe adding a data_sets top-level folder, but that won't do well over time since we don't track full migrations.
I think I'd be happy with a comment in the PR containing a block of code?

I particularly eager to hear from members of the team on how they've seen "code boundaries" tested / otherwise verified at other organizations. cc @SamRozen

Thanks for your thoughts here @dleve123 you sound like you've got a really good sense of this problem space and I'm happy to defer to your proposed solution.
PACT sounds dope - I just looked briefly so I hope this understanding is correct, but basically it acts as a fake service boundary, providing stubbed data to e.g. metaphysics so that gravity stops being an explicit dev dependency but also verifies that stub data against gravity separately? That sounds like a solution worth exploring.

Thanks for your thoughts here @dleve123 you sound like you've got a really good sense of this problem space and I'm happy to defer to your proposed solution.

To be super transparent, I've done some research on this topic, but haven't implemented a solution for a distributed test suite in the past, so don't have the wisdom of experience here. My "sniff test" suggests that a test suite / some automation seems a missing ingredient of our service approach.

PACT sounds dope - I just looked briefly so I hope this understanding is correct, but basically it acts as a fake service boundary, providing stubbed data to e.g. metaphysics so that gravity stops being an explicit dev dependency but also verifies that stub data against gravity separately?

Yes, this is my understanding!

My sense is that resolving this RFC isn't going to be a workflow change, but a research project? Some priority for a team? Either way, I think it's good that you are bringing up this problem @mbilokonsky.

Yeah I'm increasingly realizing that the RFC was maybe not the right way to bring this up because low key I wasn't 100% sold on the actual RFC action item myself - I just wanted to get people who care about it talking about it.
I'd be totally interested in taking on a project like this if it's something we want to allocate resources to!

@mbilokonsky this sounds like something that could be talk about in the platform practice as a first step! Maybe worth pinging @joeyAghion to add it to the agenda for an upcoming week.

I'd rather not increase the "code" that's uncommitted or untested or targets only test/dev scenarios.
On the other hand, I think that traditional "seed data" is a practice we haven't exploited much, and could serve (via largely existing tooling) to fill gaps when we're doing significant development without there being representative data easily reference-able in staging environments. Even when a system (like Gravity or Positron) performs a periodic copy of data from production, it could supplement with a standing seed dataset.
When applicable, though, I think source systems should be the ones responsible for maintaining their own seed or minimal data, as opposed to Metaphysics. E.g., Gravity could supply a few partners and auctions, while Exchange supplies some orders in the various interesting states. Presumably Metaphysics development could target that data locally or in staging to confirm the expected shape of results.
It does seem like this aims to address some of the same needs as "contract testing," which is a worthy research project. It's hard to say if one or both are worthwhile without demonstrations of either.
Seed data might also be a useful artifact of coordination between teams. E.g., if a new product need depends on auctions or fairs in a particular state, it could serve as a very concrete deliverable between them and ease development across those boundaries.

@dleve123 thanks for the in person ping to check this discussion. One way Spotify tackled this was by relying heavily on data schema (mainly protobuf). Schemas were in their own separate repo and versioned so that each service could reference precisely what there were using. Then APIs were versioned too and backward compatibily has to be maintained for a while. Not sure if that really tranlates to our ecosystem here though.

It sounds like we want to keep this conversation going, but perhaps not in the exact format that this RFC proposes.
Going to close this for now but will follow up with @mbilokonsky to see what the next steps should be!


Fix trie lookup, added a test to ensure memoized function doesn't get…

… called too much

Much simpler, thanks! Could you please merge with master here as well?

Thanks!


Search - App crashes after deleting track expense in Search

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 9.0.58-1
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught on HybridApp, is this reproducible on New Expensify Standalone?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail: N/A
Email or phone of affected tester (no customers<EMAIL_ADDRESS>Issue reported by: Applause - Internal Team
Action Performed:

Go to staging.new.expensify.com
Go to self DM
Track a manual expense
Go to Search
Select the track expense via checkbox
Click on the dropdown > Delete expense
Delete the expense

Expected Result:
App will not crash
Actual Result:
App crashes after deleting track expense in Search
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Standalone
[x] Android: HybridApp
[x] Android: mWeb Chrome
[x] iOS: Standalone
[x] iOS: HybridApp
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos

Add any screenshot/video evidence

https://github.com/user-attachments/assets/0d106e5d-98ca-46ec-8467-570caad5ee98
logs (2).txt
View all open jobs on GitHub

It's a BE bug
Backend returns in DeleteMoneyRequestOnSearch API a report with a value is 6

Then the app crashes here
https://github.com/Expensify/App/blob/99802996a734b01bae3437d201f8e79e27223cf7/src/libs/ReportUtils.ts#L2721

I'll take this

@lanitochka17 can you please re-test this? I tried to reproduce in dev, staging and production on web and desktop and I failed to do so on multiple accounts. Maybe it has magically fixed itself.

Issue is not reproducible on the latest build 9.0.62-3
https://github.com/user-attachments/assets/83798093-0e6a-4529-ae0c-3dc3f8341687

Thanks, so let's close this as fixed.


Test262 treats passing tests as failures

I honestly don't know how to reproduce this issue, but sometimes the Test262 runner will list a test under the "new fails" list and mark it as a failure, but if you use node test262 ./test_path to run the test individually it says the test was a success.
My best guess is that it has something to do with concurrency, as this is so on and off

Actually, after running something with --threads=1 it still marks various tests as failures, so perhaps it doesn't have much to do with concurrency? There could still be issues definitely, but it should be able to keep up with only one thread

There are a few flaky timeout tests I'm aware of, could be those?

I'm pretty sure it's not that because it was over half of the results as failures and most of them were compile errors.
I think it might've been some issue about state transfering between runs, i.e. my fault. I think I might've fixed it, but I'm running test262 to check right now.
I'll close this issue if it ends up being that 😓


#2584 disable spark engine when not properly configured

Description
if dataprep.etl.spark option is disabled,
the spark option should be unselectable on creating snapshot popup
Related Issue :
#2584
How Has This Been Tested?

Do not give spark option (refer below):

Just delete spark: option and its children.
Import & wrangle s5k_1.csv.txt
Click "Snapshot" button
Click "Advanced settings"
Spark should be disabled.

Need additional checks?
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have read the CONTRIBUTING document.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have added tests to cover my changes.

Additional Context

run build
deploy to 5

This is the last PR of @dataprep before branching 3.3.x.


[QUERY] What does garbage collector mean / do during DeletePath ?

Library name and version
Azure.Storage.Files.DataLake
Query/Question
I came across this documentation for c# SDK for adls gen 2 sdk delete, that says

The DeleteAsync(DataLakeRequestConditions, CancellationToken) operation marks the specified path deletion. The path is later deleted during garbage collection which could take several minutes.

I couldn't find more documentation related to this anywhere else. Can you please help clarify how this garbage collector operates? Is this a C# SDK thing or a general implementation detail of ADLS? What should be the client side expectation while using this method from the SDK? Will a GET request immediately after the delete return 200 until this is garbage collected?
Environment
.NET Framework 4.8

Any updates ?

Any updates ?


fix: added startForeground to onCreate

resolve #620 #524 #473 #391
ANR graph changes since the fork was applied (released on 13 October 2020).

We've made this changed in the past and it caused other crashes, I'd like to see more testing in different types of apps before merging this.

@Bang9 how has this been working for you over this time?

@dcvz I attached ANR graph
ANR seems to be spiking, but most recent occurrences are caused by other modules.

We'll pull this in @Bang9 and we'll send it out with the next release v2.0.1 and we can see how the change performs in different apps.


Creating runtime module

Description
Issue: https://github.com/jenshaase/phel-composer-plugin/issues/2
In order to decouple the Runtime creation from its logic, I created a new module named Runtime, in which you can find its Factory, the Runtime interface, and the Runtime logic.
Changes

Created a Runtime Module with

RuntimeFactory <- it creates/initialize the Runtime Singleton
Runtime <- it contains the Runtime logic
RuntimeInterface <- to invert the Runtime dependencies

Updated the docu, tests, and namespaces in order to use the RuntimeFactory instead of the Runtime itself

QA & Release notes
In order to QA these changes, I changed the phel/phel-composer-plugin version branch to the latest one that is using the RuntimeFactory as well. Therefore, we need to:
1st) merge the other PR https://github.com/jenshaase/phel-composer-plugin/pull/3
2nd) update this PR: we need to point the plugin to master back again -> "phel/phel-composer-plugin": "dev-master" and update it (composer update phel/phel-composer-plugin), commit and put the changes.
3rd) once everything is green we are able to merge this PR into master.

Merged it!


Controller attacker

Pull request summary
Adds a controllerAttack directive
Description:

Added:

Adds a new directive "ControllerAttack" which will spawn as many creeps as walkable positions around a controller using a local SpawnGroup and coordinate their attack on the same tick.

Changed:

None

Removed:

None

Fixed:

None

Testing checklist:

[x] Changes are backward-compatible OR version migration code is included
[x] Codebase compiles with current tsconfig configuration
[x] Tested changes on {choose PUBLIC/PRIVATE} server OR changes are trivial (e.g. typos)

A replay of the attackers in action:
screeps.com/s/m2ifuE

Requested changes published.  Let me know what you think!
One note: RED/PURPLE was in use for DirectiveGuardSwarm, so I have that directive commented out in initializer.ts.  It will require a different color scheme.


Give warning of unsaved changes if user tries to navigate before saving content

If the user navigates while editing a post the user should be prompted with an alert to confirm the navigation so they don't lose unsaved changes.
If the user has saved all changes this prompt can be ignored.

I can pick this up!

Sweet! If you have any questions feel free to ask in the open source chat in our Discord!
Thanks for helping.
I had made a custom hook but never got it finished. I can try find the WIP if it's helpful

Sure, that would be great! I have in mind something that might work also. I can also take something else if you are close to finishing this one though. Im easy either way!!

I didn't go too far with it because there was a few kinks. I was just stealing code from this thread: https://stackoverflow.com/questions/63064778/next-js-warn-user-for-unsaved-form-before-route-change

great !! I will check it out! Thanks

UPDATE: Have a functioning version of this using confirm dialogs. It would be much nicer to get this working with the modal in the components folder. Will work on it during the week but most likely next weekend I will have time to work on it properly.

@xiaoniuniu89  @NiallJoeMaher  is this issue still required as the app has autosave functionality? The autosave seems to work fairly fast, the only case I can see this being required is if the autosave fails and you try navigate away from the page?

this issue is done, should be closed @NiallJoeMaher


[WathCa] The app quits unexpectedly after doing a couple of cases.

This issue was originally reported in #17 and reported by @dariadg
After doing a couple of cases, the App would quit:

It would always quit after I’ve loaded a new image, then pressed: “Segment Images”.
It would ask: “Do you have a segmentation to load”. I’d answer: “No”.
[Then] it shuts down completely.

As the tested, the issue is most likely due to failure of generating centrelines or clipper planes. A number of safeguards are now added to alert the user, if the process fails. It may still be necessary to reevaluate the meshing tool used at the moment for clipping the veins.

So, it seems that the crash can occur when the user clicks on clipper planes button in the PV clipper view. Debugging the code shows that segmentation fault is caused by the vtkvmtkPolyDataCenterlineSections filter:
vtkSmartPointer ctrLineSects;
ctrLineSects->SetInputData(surface->GetVtkPolyData());
ctrLineSects->SetCenterlines(line);
ctrLineSects->SetCenterlineSectionAreaArrayName("CentrelineSectionAreaArrayName");
ctrLineSects->SetCenterlineSectionMinSizeArrayName("CenterlineSectionMinSizeArrayName");
ctrLineSects->SetCenterlineSectionMaxSizeArrayName("CenterlineSectionMaxSizeArrayName");
ctrLineSects->SetCenterlineSectionShapeArrayName("CenterlineSectionShapeArrayName");
ctrLineSects->SetCenterlineSectionClosedArrayName("CenterlineSectionClosedArrayName");
ctrLineSects->Update(); //update causes the crash
And the filter fails most likely due to degenerate triangles of the decimated mesh. We need to find a way to clean the mesh after decimation or remove the decimation code all together and find a better meshing tool with control over meshing resolution.

I added polydata cleaning and recalculating normals filters to the code after the decimation step. It seems it has stopped the crashes in cases sent to us that had issues in the past. @alonsoJASL can you please run some tests on the new clipping code? If it works, I think we can close this issue.

By using vtkPolyDataConnectivityFilter and vtkWindowedSincPolyDataFilter after the decimation step, the problem seems to be fixed.