Spaceship - IAPList status returns nil when in "developerRemovedFromSale" status

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description
Here are log from Spaceship Playground
disabled_iap = iap.find("REDACTED")
=> <Spaceship::Tunes::IAPList
    application=<Spaceship::Tunes::Application
        apple_id="REDACTED",
        name="REDACTED",
        vendor_id="REDACTED",
        bundle_id="REDACTED",
        last_modified=1498800850000,
        issues_count=0,
        app_icon_preview_url="REDACTED",
        version_sets=[<Spaceship::Tunes::VersionSet
            type="APP",
            application=<Spaceship::Tunes::Application
            #<Object ...>>,
            platform="ios">]>,
    reference_name="REDACTED",
    product_id="REDACTED",
    family_reference_name=nil,
    duration_days=0,
    versions=[{"screenshotUrl"=>"REDACTED", "canSubmit"=>false, "issuesCount"=>0, "itunesConnectStatus"=>"developerRemovedFromSale"}],
    purple_apple_id=["REDACTED"],
    last_modfied_date=1500456859128,
    is_news_subscription=false,
    number_of_codes=0,
    maximum_number_of_codes=100,
    app_maximum_number_of_codes=1000,
    is_editable=false,
    is_required=false,
    can_delete_addon=true>
[5] pry(#<Spaceship::Playground>)> disabled_iap.status
=> nil

You can see "developerRemovedFromSale" in versions. (scroll to the right)
An Apple Developer document on Working with Your Product's Pricing and Availability shows the list of In-app purchase product statuses.
But in here, I found that only few of them are declared. "developerRemovedFromSale" status is not declared. I think that's because maybe the rest is not necessary in most cases, or just their actual values are not found yet.
I'm preparing a pull request for this.
Environment
✅ fastlane environment ✅
Stack

Key
Value

OS
10.12.3

Ruby
2.4.1

Bundler?
true

Git
git version 2.10.1 (Apple Git-78)

Installation Source
~/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bin/fastlane

Host
Mac OS X 10.12.3 (16D32)

Ruby Lib Dir
~/.rbenv/versions/2.4.1/lib

OpenSSL Version
OpenSSL 1.0.2j  26 Sep 2016

Is contained
false

Is homebrew
false

Is installed via Fabric.app
false

Xcode Path
/Applications/Xcode.app/Contents/Developer/

Xcode Version
8.2.1

System Locale

Variable
Value

LANG
ko_KR.UTF-8
✅

LC_ALL

LANGUAGE

fastlane files:
`./fastlane/Fastfile`
skip_docs

desc "Runs the unit tests and other verifications for the fastlane repo"
lane :test do |options|
  validate_repo
end

desc "Increment the version number of this gem"
lane :bump do |options|
  ensure_git_branch(branch: "master")
  ensure_git_status_clean

  github_api_token = ENV["FL_GITHUB_RELEASE_API_TOKEN"]
  UI.user_error!("Please provide a GitHub API token using `FL_GITHUB_RELEASE_API_TOKEN`") if github_api_token.to_s.length == 0

  path = "./fastlane/lib/fastlane/version.rb"

  # Verify everything is in a consistent state
  latest_version = current_version
  local_version = version_get_podspec(path: path)
  UI.user_error!("Version on RubyGems doesn't match local repo: #{latest_version} != #{local_version}") if latest_version != local_version

  changelog_text = show_changelog

  bump_type ||= 'minor' if prompt(text: "New feature, method or API?", boolean: true)
  bump_type ||= 'patch'

  slug = "fastlane/fastlane"
  new_version = version_bump_podspec(path: path, bump_type: bump_type)
  sh "git checkout -b 'version-bump-#{new_version}'"
  commit_message = "Version bump to #{new_version}"
  git_commit(path: path,
             message: commit_message)
  push_to_git_remote

  pr_body = ["Auto-generated by fastlane 🤖"]
  pr_body << "**Changes since release '#{latest_version}':**"
  pr_body << changelog_text
  pr_url = create_pull_request(
    api_token: github_api_token,
    repo: slug,
    title: commit_message,
    body: pr_body.join("\n\n")
  )

  if ENV['SLACK_URL']
    slack(
      channel: "action",
      default_payloads: [],
      message: "Version bump: #{pr_url} ✨",
      payload: {}
    )
  end

  # Revert to master branch
  sh "git checkout master"
end

desc "Does everything that's needed for a release"
desc "This includes running tests and verifying the GitHub release"
lane :release do
  slack_train_start(distance: 2,
                       train: "🚀",
           reverse_direction: true,
                        rail: "✨")

  update_fastlane
  slack_train

  # Git verification
  #
  ensure_git_status_clean
  ensure_git_branch(branch: 'master')
  git_pull

  validate_repo

  # Verifying RubyGems version
  #
  require "../fastlane/lib/fastlane/version.rb"
  version = Fastlane::VERSION
  old_version = current_version
  puts "Deploying #{version}"
  if Gem::Version.new(version) <= Gem::Version.new(old_version)
    UI.user_error!("Version number #{version} was already deployed")
  end

  # Then push to git remote
  #
  push_to_git_remote

  # Preparing GitHub Release
  #
  github_release = get_github_release(url: "fastlane/fastlane", version: version)
  if (github_release || {}).fetch('body', '').length == 0
    show_changelog(old_version: old_version)

    title = prompt(text: 'Title: ')
    description = prompt(text: "Please enter a changelog (make sure to rephrase changes to make it easy to understand, and remove non-important changes that don't affect the end-user): ",
                         multi_line_end_keyword: "END")

    github_release = set_github_release(
      repository_name: "fastlane/fastlane",
      name: [version, title].join(" "),
      tag_name: version,
      description: description,
      is_draft: false
    )

    # Actual release of the gem
    #
    sh "gem push ../pkg/fastlane-#{version}.gem"

    release_url = github_release['html_url']

    message = [title, description, release_url].join("\n\n")
    add_fastlane_git_tag(tag: "fastlane/#{version}", message: message)
  end

  # After publishing
  #
  if ENV['SLACK_URL']
    release_notes = github_release['body']
    # markup - turn github references into links
    release_notes.gsub!(/\(#([0-9]+)\)/, '(<https://github.com/fastlane/fastlane/issues/\1|#\1>)')
    slack_message = "Successfully released [fastlane #{version}](#{release_url}) :rocket:"
    slack(
      channel: "releases",
      default_payloads: [],
      message: slack_message,
      payload: {
        "New" => release_notes
      }
    )
    slack(channel: "action", default_payloads: [], message: slack_message)
  end

  clubmate

  puts "You can now tweet:".green
  releases_url = "https://github.com/fastlane/fastlane/releases/tag/#{version}"
  puts "[fastlane] #{github_release['name']} #{releases_url}"

  send_mac_app_ci_reminder

  update_docs
  slack_train
end

desc "Makes sure the tests on https://docs.fastlane.tools still work with the latest version"
lane :verify_docs do
  clone_docs do
    Bundler.with_clean_env do
      puts `sed -i -e "s/activate_bin_path/bin_path/g" $(which bundle)` # workaround for bundler https://github.com/bundler/bundler/issues/4602#issuecomment-233619696
      sh "bundle install"
      sh "bundle exec fastlane test skip_building:true" # skip_building since we don't have a proper python environment set up
    end
  end
end

desc "Generates the actions.md that will get used on https://docs.fastlane.tools"
desc "This will NOT also automatically submit a pull request to fastlane/docs"
desc "This SHOULD only be used for testing changes to the docs"
lane :test_generate_docs do
  ENV['USE_ENHANCE_TEST_DATA'] = "true"
  generate_markdown_docs
  ENV.delete("USE_ENHANCE_TEST_DATA")
end

desc "Update the actions.md on https://docs.fastlane.tools"
desc "This will also automatically submit a pull request to fastlane/docs"
lane :update_docs do
  verify_env_variables

  template_path = File.expand_path("./assets/render_plugin.md.erb")
  clone_docs do
    actions_md_path = generate_markdown_docs

    Bundler.with_clean_env do
      sh "bundle update"
    end

    if `git status --porcelain`.length == 0
      UI.success("No changes in the actions.md ✅")
    else
      # Create a new branch
      sh "git checkout -b 'update-actions-md-#{Time.now.to_i}'"
      plugins_path = "docs/plugins/available-plugins.md"
      plugin_scores(template_path: template_path,
                      output_path: plugins_path)

      Dir.chdir("fastlane") do # this is an assumption of fastlane, that we have to .. when shelling out
        # Commit the changes
        git_commit(path: [actions_md_path, "Gemfile.lock", plugins_path],
                message: "Update actions.md for latest fastlane release 🚀")
        # Push them to the git remote
        push_to_git_remote

        # Submit the pull request
        pr_url = create_pull_request(
          api_token: ENV["FL_GITHUB_RELEASE_API_TOKEN"],
          repo: "fastlane/docs",
          title: "[Bot] Update actions.md for latest fastlane release 🚀",
          body: "Auto-generated by _fastlane_ bot 🤖"
        )
        UI.success("Successfully submitted a pull request to fastlane/docs: #{pr_url} 🚀")
      end
    end
  end
end

def generate_markdown_docs
  require 'fastlane/documentation/markdown_docs_generator'
  actions_md_path = File.expand_path("docs/actions.md")
  Fastlane::MarkdownDocsGenerator.new.generate!(target_path: actions_md_path)

  return actions_md_path
end

def clone_docs
  require 'tmpdir'
  git_url = ENV['FASTLANE_DOCS_GIT_URL'] || "https://github.com/fastlane/docs"

  Dir.mktmpdir("fl_clone") do |tmp_dir|
    Dir.chdir(tmp_dir) do
      sh "git clone #{git_url} --depth=1"
      Dir.chdir("docs") do
        yield
      end
    end
  end
end

error do |lane, exception|
  if ENV['SLACK_URL']
    slack(channel: "testing", message: exception.to_s, success: false)
  end
  slack_train_crash
end

private_lane :send_mac_app_ci_reminder do
  if ENV['FABRIC_SLACK_URL']
    slack(
      slack_url: ENV['FABRIC_SLACK_URL'],
      channel: 'deployment-tools',
      default_payloads: [],
      message: "Please run the Fastlane Mac App Package CI job in TeamCity\n#{ENV['FABRIC_MAC_APP_CI_JOB_URL']}"
    )
  end
end

desc "Verifies all tests pass and the current state of the repo is valid"
private_lane :validate_repo do
  # Verifying that no debug code is in the code base
  #
  ensure_no_debug_code(text: "binding.pry", extension: ".rb", exclude: "playground.rb", exclude_dirs: ["\.bundle"]) # debugging code
  ensure_no_debug_code(text: "# TODO", extension: ".rb", exclude_dirs: ["\.bundle"]) # TODOs
  ensure_no_debug_code(text: "now: ", extension: ".rb", exclude_dirs: ["\.bundle"]) # rspec focus
  ensure_no_debug_code(text: "<<<<<<<", extension: ".rb", exclude_dirs: ["\.bundle"]) # Merge conflict

  # spaceship and credentials_manager don't have access to fastlane_core
  ensure_no_debug_code(text: " UI\\.", path: "spaceship/lib", extension: ".rb", exclude_dirs: ["\.bundle"])
  ensure_no_debug_code(text: " UI\\.", path: "credentials_manager/lib", extension: ".rb", exclude_dirs: ["\.bundle"])

  rubocop

  # Verifying the --help command
  #
  # tools_to_test = Fastlane::TOOLS
  tools_to_test = [:fastlane] # until we have the binaries for all tools included here
  tools_to_test.each do |tool_name|
    next if no_binary.include?(tool_name)
    Dir.chdir("..") do
      binary_path = File.join("bin", tool_name.to_s)
      content = sh("PAGER=cat #{binary_path} --help")
      ["--version", "https://fastlane.tools", tool_name.to_s].each do |current|
        UI.user_error!("--help missing information: '#{current}'") unless content.include?(current)
      end
    end
  end

  Dir.chdir("..") do
    # Install the bundle and the actual gem
    sh "bundle check || bundle install"
    sh "rake install"

    # Run the tests
    #
    sh "bundle exec rake test_all"

    # Verify shell code style
    sh 'find -E . -regex ".*\.(sh|bash)" -not -name "Pods-*" -name ".bundle" -exec shellcheck {} +'
  end

  test_generate_docs
  ensure_code_samples

  # Verify docs are still working
  verify_docs
end

desc "Get the version number of the last release"
private_lane :current_version do
  puts "Checking the latest version on RubyGems"
  download(url: "https://rubygems.org/api/v1/gems/fastlane.json")["version"]
end

desc "All repos that don't have a binary to test"
private_lane :no_binary do
  ['fastlane_core', 'spaceship', 'credentials_manager']
end

desc "Print out the changelog since the last tagged release and open the GitHub page with the changes"
lane :show_changelog do |options|
  old_version = options[:old_version] || current_version

  changes = sh("git log --pretty='* %s via %aN' #{old_version}...HEAD --no-merges ..", log: $verbose).gsub("\n\n", "\n")
  changes.gsub!("[WIP]", "") # sometimes a [WIP] is merged

  github_diff_url = "https://github.com/fastlane/fastlane/compare/#{old_version}...master"
  sh "open #{github_diff_url}"

  puts "Changes since release #{old_version}:\n\n#{changes.cyan}"
  changes # return the text without the modified colors
end

desc "Add a git tag in the fastlane repo for this release"
private_lane :add_fastlane_git_tag do |options|
  `git tag -am #{options[:message].shellescape} #{options[:tag].shellescape}`
  push_git_tags
end

# Run the code samples, yo
private_lane :ensure_code_samples do
  UI.message "<U+1F557>  Verifying all code samples work with the latest fastlane release"
  test_sample_code(path: File.expand_path("docs/Actions.md"))
  UI.success "<U+2705>  All fastlane code samples work as expected"
end

desc "Ensure all the requirement environment variables are provided"
desc "this way the deployment script will fail early (and often)"
private_lane :verify_env_variables do
  ["GITHUB_USER_NAME", "GITHUB_API_TOKEN"].each do |variable|
    next if ENV[variable].to_s.length > 0
    UI.user_error!("Missing ENV variables #{variable}, make sure to provide one for fastlane to be successful")
  end
end

No Appfile found
fastlane gems

Gem
Version
Update-Status

fastlane
2.54.4
✅ Up-To-Date

Loaded fastlane plugins:

Plugin
Version
Update-Status

fastlane-plugin-ruby
0.1.3
✅ Up-To-Date

fastlane-plugin-clubmate
0.1.0
✅ Up-To-Date

fastlane-plugin-slack_train
0.2.0
✅ Up-To-Date

Loaded gems

Gem
Version

did_you_mean
1.1.0

bundler
1.15.3

io-console
0.4.6

rake
11.3.0

CFPropertyList
2.3.5

public_suffix
2.0.5

addressable
2.5.2

ast
2.3.0

babosa
1.0.2

debug_inspector
0.0.3

binding_of_caller
0.7.2

builder
3.2.3

byebug
9.1.0

claide
1.0.2

colored2
3.1.2

cork
0.3.0

nap
1.1.0

open4
1.3.4

claide-plugins
0.9.2

coderay
1.1.1

colored
1.2

highline
1.7.8

commander-fastlane
4.4.5

json
2.1.0

docile
1.1.5

simplecov-html
0.10.2

simplecov
0.14.1

tins
1.15.0

term-ansicolor
1.6.0

thor
0.19.4

coveralls
0.8.21

safe_yaml
1.0.4

crack
0.4.3

multipart-post
2.0.0

faraday
0.13.1

faraday-http-cache
1.3.1

git
1.3.0

kramdown
1.14.0

sawyer
0.8.1

octokit
4.7.0

unicode-display_width
1.3.0

terminal-table
1.8.0

danger
4.3.5

ox
2.6.0

danger-junit
0.7.3

declarative
0.0.9

declarative-option
0.1.0

diff-lcs
1.3

unf_ext
<IP_ADDRESS>

unf
0.1.4

domain_name
0.5.20170404

dotenv
2.2.1

excon
0.58.0

fakefs
0.8.1

http-cookie
1.0.3

faraday-cookie_jar
0.0.6

faraday_middleware
0.12.2

fastimage
2.1.0

gh_inspector
1.0.3

jwt
1.5.6

little-plugger
1.1.4

multi_json
1.12.1

logging
2.2.2

memoist
0.16.0

os
0.9.6

signet
0.7.3

googleauth
0.5.3

httpclient
2.8.3

mime-types-data
3.2016.0521

mime-types
3.1

uber
0.1.0

representable
3.0.4

retriable
3.1.1

google-api-client
0.13.4

mini_magick
4.5.1

multi_xml
0.6.0

plist
3.3.0

rubyzip
1.2.1

security
0.1.3

slack-notifier
1.5.1

terminal-notifier
1.8.0

tty-screen
0.5.0

word_wrap
1.0.0

nanaimo
0.2.3

xcodeproj
1.5.1

rouge
2.0.7

xcpretty
0.2.8

xcpretty-travis-formatter
0.0.4

fastlane-plugin-clubmate
0.1.0

fastlane-plugin-ruby
0.1.3

fastlane-plugin-slack_train
0.2.0

hashdiff
0.3.6

interception
0.5

method_source
0.8.2

netrc
0.11.0

parallel
1.12.0

parser
<IP_ADDRESS>

powerpack
0.1.1

slop
3.6.0

pry
0.10.4

pry-byebug
3.5.0

pry-rescue
1.4.5

pry-stack_explorer
<IP_ADDRESS>

rack
1.6.8

rack-protection
1.5.3

rainbow
2.2.2

rb-readline
0.5.5

rest-client
2.0.2

rspec-support
3.5.0

rspec-core
3.5.4

rspec-expectations
3.5.0

rspec-mocks
3.5.0

rspec
3.5.0

rspec_junit_formatter
0.2.3

ruby-progressbar
1.8.1

rubocop
0.49.1

tilt
2.0.8

sinatra
1.4.8

webmock
2.3.2

xcode-install
2.2.1

yard
<IP_ADDRESS>

generated on: 2017-08-31

Please also add "Developer Action Needed" to the list.
Also, it would be great if IAPStatus does not return nil for any status from Apple that it doesn't recognize. It should return the raw status text from Apple in this case.

@luongm I am not sure what exactly iTunes Connect sends as a response for Developer Action Needed status; it could possibly be developerActionNeeded, but I don't know. I haven't encountered such situation since I used Spaceship.
I agree that it would be good and useful if IAPStatus returns raw text instead of nil. However, if we allow IAPStatus to return an arbitrary string, instead of predefined strings, I think codes that use those strings would be broken if Apple changes the strings for some reason.
Btw, I forgot to write close #10206 in my pull request (#10224) and now I close this issue.

I unfortunately did not capture what the status was when one of my IAP was marked like that. Now I don't have an IAP with developer action needed anymore, so I guess I'll update this or create a new issue later when Apple marks one again.


Expose Chai helpers for custom assertions

Chai exposes some utilities for creating plugins, e.g. for defining custom assertions. This change just propagates that exposure through the module shim.

The build failures are due to the bug fixed in https://github.com/switchfly/ember-test-helpers/pull/28.  We need to update ember-test-helpers to 0.4.1 for ember-mocha (should likely be another pull request though).

OK, #29 is merged, can you rebase to get this to pass?

All set -- thanks for getting to #29 so quickly :)


Meta issue: FFI attributes syntax

There are some FFI (and C FFI) specific attributes that we might want to add to Rust (e.g. see https://github.com/rust-lang/rust/pull/58327 and https://github.com/rust-lang/rust/pull/58315, amongst others).
We probably want to expose FFI attributes in a clean way.
Prefix vs no prefix
Many FFI attributes don't have a prefix, e.g., #[link_name], etc. Some like #[returns_twice] are required for correctness, and some like pure and const are optimization hints that might be ignored by the compiler. These do however have overloaded names that we might want to put into context to avoid confusion with, e.g., const fn.
Also, some attributes like pure and const have names with overloaded semantics that might make sense to disambiguate, e.g., instead of #[const] which is very similar to const fn, maybe #[ffi_const].

Should we use prefixes at all?
Should we do so for all FFI attributes or only for some?
If we only use prexies for some attributes, do we decide for which on a 1 by 1 basis, or how do we decide this?

FFI can be used to interface with any language, but some attributes might only make sense when interfacing with some particular language.

Should we take the language into account when prefixing attributes? (e.g. c_ffi or ffi::c)
Should we just use a general prefix, e.g., ffi? (independent of language)
Should we decide the prefix on a 1 by 1 basis? (should we define some rules for the prefixes? e.g. ffi and ffi_c vs ffi and c_ffi, etc.)

Option namespacing
Some FFI attributes might need to behave slightly different depending on the toolchain used to compile the code that's being interfaced via FFI.
For example, clang currently exposes both the [[const]] and [[gcc::const]] attributes. These currently are identical, but this separation is forward compatible with diverging semantics in the optimization hints that const has in the different toolchains. This is important both for interfacing clang binaries with gcc compiled libraries and vice-verse, and also, for not performing incorrect optimizations on programs that satisfy e.g. [[gcc::const]] requirements but not clang's [[const]].

Should we already plan a syntax for platform toolchain disambiguation? (e.g. #[ffi::c::const(clang)] vs #[ffi::c::const(cranelift)] ?

Even if this is not RFC'ed right now, it might be worth it to have some plan for this in the pipe line in case it becomes necessary, and to make sure that merged RFCs are forward compatible with this. Otherwise we might end with #[c_ffi_const_clang] and #[gcc_ffi_c_const] or some similar zoo of attributes.

Some notes:

ffi:: as a prefix seems like a clean way to deal with complexity by namespacing. Disambiguating further with ffi::c:: also seems reasonable and clear. These attributes are probably not going to be used all that often and so clarity seems paramount.

const is a keyword. Thus, #[ffi::const] and #[ffi(const)] are both illegal according to the attribute grammar. We could use a hack, but I'd strongly prefer not to.
This is an opportunity to rename const to something else both to avoid confusion with Rust and because it's not a good name. As I noted on https://github.com/rust-lang/rust/pull/58327, I think that GCC has named these attributes poorly and misleadingly. I'm not inclined to continue in their footsteps, especially since it's, from what I can tell, not part of the standard C definition.

Re. #[ffi::c::const(clang)] vs #[ffi::c::const(cranelift)], I think that while there are few drawbacks in taking them into account syntactically, I'm already on the edge wrt. #[ffi::c::pure], and so I am opposed to the unbounded complexity this can occur. I also think this encourages too toolchain-dependent code and I don't want to recognize that LLVM or GCC exist in the language specification itself.

not part of the standard C definition.

Restricting Rust foreign function interface to "standard C" was never a goal AFAIK. So I don't think that this arguments supports giving these different names.

I think that GCC has named these attributes poorly and misleadingly. I'm not inclined to continue in their footsteps

These attributes are very old. As a consequence, a huge amount of C code in the wild uses them, C programmers know them, and most C compilers support them.
The name is bad today, but "pure" wasn't really mainstream term back then.

This is an opportunity to rename const to something else both to avoid confusion with Rust and because it's not a good name.

For the user copy-pasting a C function declaration into Rust, giving it the same name has maximum discoverability. Giving them a different name, reduces this discoverability.
So it appears to me that giving these good names is at tension with making them discoverable.
The PR balances the trade-offs by using #[c_ffi_const] instead of #[const], but I'd like to hear different ways to balance these trade-offs (ideally concrete suggestions), and whether there is a way that could allow us to have our cake and eat it too (giving them good Rusty names, while making them very discoverable).

const is a keyword. Thus, #[ffi::const] and #[ffi(const)] are both illegal according to the attribute grammar. We could use a hack, but I'd strongly prefer not to.

Indeed :/ Underscores create new identifiers, so as a namespacing method, it doesn't have this problem. Alternatively, #[ffi("const")] wouldn't have this problem either.

and I don't want to recognize that LLVM or GCC exist in the language specification itself.

Maybe it would be better to handle this at the ABI level? For example, via: extern "C(clang)" { ... }.

#[ffi(const)] is ok.
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=8bb34a21ec0ce26cf99bf2c61b30ef35
With https://github.com/rust-lang/rust/pull/57367 the "no keywords" restriction is only active for built-in attributes due to using MetaItem as implementation detail and this can be overridden for individual attributes (e.g. cfg_attr already doesn't fit into MetaItem).

#[ffi(const)] is ok.
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=8bb34a21ec0ce26cf99bf2c61b30ef35

Oh right; we changed this... I should know this since I reviewed the PR 😅. Still, I think ffi::thing seems cleaner in terms of namespacing instead of having a monolithic attribute.

Restricting Rust foreign function interface to "standard C" was never a goal AFAIK. So I don't think that this arguments supports giving these different names.

What notable stable FFI things do we expose that are not standard C? I think it's an important consideration because it matters wrt. universality and because if something is in standard C it is unlikely to ever change and you don't get the potential for dialectal problems between clang, gcc, etc. The examples to come to my mind are packed and specifying alignment (but that's useful for non-FFI purposes as well).

and most C compilers support them.

But notably not MSVC.

The name is bad today, but "pure" wasn't really mainstream term back then.

I think it was misleading when it was first introduced in 2000 (which is not that old) and I expect the term "pure function" has been used for some time...

For the user copy-pasting a C function declaration into Rust, giving it the same name has maximum discoverability. Giving them a different name, reduces this discoverability.

I think it's relatively easy to write a sentence about the correspondence to GCC/clang/etc. and it will quite likely be the first thing you find if you search for "Rust pure". If you search for "Rust const" you'd find something entirely different anyways (at least in the first search results) so that doesn't seem optimal in terms of discoverability.

The PR balances the trade-offs by using #[c_ffi_const] instead of #[const], but I'd like to hear different ways to balance these trade-offs (ideally concrete suggestions), and whether there is a way that could allow us to have our cake and eat it too (giving them good Rusty names, while making them very discoverable).

I'll think of something.

Maybe it would be better to handle this at the ABI level? For example, via: extern "C(clang)" { ... }.

What is that repainting intended to achieve? You still get the multiplicative complexity of all these different toolchains as well as the encouragement of toolchain dependence in code (which is the opposite of what I think we should aim for).

Relevant paper: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0078r0.pdf

What notable stable FFI things do we expose that are not standard C?

That's offtopic, we can talk about this in Discord if you want, but let's stay on this issue.

But notably not MSVC.

That's mainly why I suggested using const(clang) or const(gcc), but MSVC is a bad example, and there is precedent in stable Rust of ignoring it (but that's offtopic to).

it will quite likely be the first thing you find if you search for

const and pure will be a pain to discover with online searches: as you mention const means something else in Rust and is widely used already, while pure used to mean something in Rust, and is a mainstream term.
But I have a solution.
This issue is about adding a consistent way of mapping C attributes in Rust. Let's assume the syntax is #[ffi(...)]. We would teach this as: "if you need to map a C attribute to Rust FFI, you use #[ffi(c_attr_name)] - that's all you need to know". If the name in Rust is the same as in C, that works and does the correct thing. If we changed the name, we reserve the C name, and if a user types it in, we error, with a suggestion "Maybe you meant the C FFI attribute rust_attr_name ?".
That way we don't have to use C names, and people writing code don't need to remember the mapping.

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0078r0.pdf

(note: that's a C++ paper, and there hasn't been a new revision since 2015)

This issue is blocking #2633 and it seems like @gnzlbg has a reasonable solution. What would help move this forward?

So I've had more time to think about this, and while this issue is worth resolving, I think #2633 needs extra work for unrelated reasons (will post there in a sec).


⚠ Bofrost.lu has degraded performance

In ff2452e, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 2170 ms

Resolved: Bofrost.lu performance has improved in 45f0445 after 6 minutes.


Proposal: Custom event for components

Let's talk about state lifting in powercycle.
This is official counter app example of powercycle.

I removed some boilerplate codes which are not important in this topic.
Let's focus on only components.

function main(sources) {
  const count$ = sources.sel.incrementButton.click
    .fold(count => count + 1, 0)
    .startWith(0)

  return (
    <>

Counter: {count$}

      <button sel='incrementButton' value={count$}>Increment</button>
    </>
  )
}

Nice code, but I want to add button to decrease count.
So, let's add some code.
function main(sources) {
  const increment$ = sources.sel.incrementButton.click.mapTo(1);
  const decrement$ = sources.sel.decrementButton.click.mapTo(-1);
  const change$ = xs.merge(increment$, decrement$);
  const count$ = change$
    .fold((count, change) => count + change, 0)
    .startWith(0);

  return (
    <>

Counter: {count$}

      <button sel='incrementButton'>Increment</button>
      <button sel='decrementButton'>Decrement</button>
    </>
  );
}

not bad, but we can find some code that could be abstracted.
that code is a button.
let's abstract it in clean way(in FP way).
function ChangeButton(sources) {
  const change$ = sources.sel.changeButton.click.mapTo(sources.props.delta); // how can we provide this stream to main?

  return (
    <button sel='changeButton'>{sources.props.children}</button>
  );
}

function main(sources) {
  const change$ = xs.merge(increment$, decrement$); // wait... how can we get these streams?
  const count$ = change$
    .fold((count, change) => count + change, 0)
    .startWith(0);

  return (
    <>

Counter: {count$}

      <ChangeButton delta={1}>Increment</button>
      <ChangeButton delta={-1}>Decrement</button>
    </>
  );
}

Oh, we got some problem here, how can we share stream of children with its parent?
sharing through props(or sources) via imitate is very verbose and ineffective way.
but sharing through returned value is also too hard to use(since we can't use shorthand of powercycle function and jsx well)!
and I know that updating state stream of parent in children is bad since it makes bidirectional data flow in application(and more reasons.. like "duplication of sink stream").
As I know, we don't have any certain solution for this problem.
So I'd like to propose my idea in here, a Custom Event sink.
this is very simple concept, just adding reserved sink event.
for example,
function ChangeButton(sources) {
  const change$ = sources.sel.changeButton.click.mapTo(sources.props.delta);

  return [
    <button sel='changeButton'>{sources.props.children}</button>,
    {
      event: {
        click: change$,
      } // I did consider to use merged stream of multiple event stream, but this would be better to read, understand.
    },
  ];
}

function main(sources) {
  const increment$ = sources.sel.incrementButton.click;
  const decrement$ = sources.sel.decrementButton.click;
  const change$ = xs.merge(increment$, decrement$);
  const count$ = change$
    .fold((count, change) => count + change, 0)
    .startWith(0);

  return (
    <>

Counter: {count$}

      <ChangeButton sel='incrementButton' delta={1}>Increment</button>
      <ChangeButton sel='decrementButton' delta={-1}>Decrement</button>
    </>
  );
}

feedbacks & questions are welcome!
thank you.

That's exactly what reducers are for:
import xs from 'xstream'
import { run } from '@cycle/run'
import { makeDOMDriver } from '@cycle/react-dom'
import { withState } from '@cycle/state'
import withPower from 'powercycle'
/** @jsx withPower.pragma */
/** @jsxFrag withPower.Fragment */

function ChangeButton(sources) {
  return [
    <button sel='btn'>{sources.props.children}</button>,
    { state: sources.sel.btn.click
      .map(value => prev => prev + sources.props.delta) }
  ]
}

function main(sources) {
  const reducer$ = xs.of(() => 0);

  return [
    <>

Counter: {sources.state.stream}

      <ChangeButton delta={1}>Increment</ChangeButton>
      <ChangeButton delta={-1}>Decrement</ChangeButton>
    </>,
    { state: reducer$ }
  ]
}

const drivers = {
  react: makeDOMDriver(document.getElementById('app'))
}

run(withState(withPower(main)), drivers)

Yes, but ChangeButton directly updates Its parents state.
this reduces reusability of code.

results of ChangeButton(state) is highly coupled with shape of state of its parent

if we change shape of state of main, then we should change ChangeButton.
what if want not to update main's state to less then 0? In your code, the only way is to change code of ChangeButton. doesn't this strange? we have to change code of ChangeButton to add feature in main!
summary: low reusability, high coupling -> bad abstraction

bidirectional data flow

bidirectional data flow is well-know bad practice, which makes debugging and composing hard.

implicit data flow

when we see main, we can't get any hint about updating it's state. Don't forget that FRP's strong point is explicit data flow.

Incorrect. There's no bi-direction, ChangeButton is a pure component. It receives a state and returns with a reducer. The parent gives the state and decides upon its return value. If you want to restrict its behavior, you can easily set it up with scoping and lens.
Here, the main state has a different structure and the parent prevents the counter from going negative. All the while ChangeButton remains the same as above:
import xs from 'xstream'
import { run } from '@cycle/run'
import { makeDOMDriver } from '@cycle/react-dom'
import { withState } from '@cycle/state'
import withPower from 'powercycle'
import { get, map } from 'powercycle/util'
/** @jsx withPower.pragma */
/** @jsxFrag withPower.Fragment */

function ChangeButton(sources) {
  return [
    <button sel='btn'>{sources.props.children}</button>,
    { state: sources.sel.btn.click
      .map(value => prev => prev + sources.props.delta) }
  ]
}

function main(sources) {
  const reducer$ = xs.of(() => ({ counter: 0 }));
  const preventNegativeLens = {
    get: state => state.counter,
    set: (state, childState) => ({ ...state, counter: Math.max(0, childState) })
  }

  return [
    <>

Counter: {get('counter')}

      <ChangeButton delta={1} scope='counter'>Increment</ChangeButton>
      <ChangeButton delta={-1} scope={{ state: preventNegativeLens }}>Decrement</ChangeButton>
    </>,
    { state: reducer$ }
  ]
}

const drivers = {
  react: makeDOMDriver(document.getElementById('app'))
}

run(withState(withPower(main)), drivers)

You can even break down the lens into a scope and a constraint, so it's clearer:
function main(sources) {
  const reducer$ = xs.of(() => ({ counter: 0 }));
  const preventNegativeLens = {
    get: state => state,
    set: (state, childState) => Math.max(0, childState)
  }

  return [
    <>
      <Scope scope='counter'>

Counter: {get()}

        <ChangeButton delta={1}>Increment</ChangeButton>
        <ChangeButton delta={-1} scope={{ state: preventNegativeLens }}>Decrement</ChangeButton>
      </Scope>
    </>,
    { state: reducer$ }
  ]
}

Okay, now I understand what you meant, thank you for information. 😄


MLB markets are unresolved

MLB markets for yesterday, 9/20, should've resolved overnight but did not.

Based on testing this morning,  this looks to be fixed. cc: @pgebheim


Fixed Errors in PR87(Command Handler + Parser PRs)

Parser moved from import files version to multiple inheritances.
Cleaned command handler
ToDo: Test cases for command handler

Merged using #127, Closing this.

Merged using #127, Closing this.


Update Netty to 4.1.99.final

References

https://mail.openjdk.org/pipermail/hotspot-gc-dev/2023-September/043696.html
https://netty.io/news/2023/09/21/4-1-99-Final.html

/cc @He-Pin

-1 from me. This is the umpteenth recent PR for netty upgrades. This is time consuming and of little benefit. We want to set up an automated approach based on Scala Steward. This is a much better way to do this and than random manual PRs.-1

The responsibility for that should be with #56, rather than with this issue. Automating such version upgrades is far superior to manual PRs, so why #56 has not made further progress?

@pjfanning you are one of Pekko PMC, you should take the responsibility to push on #56 and formulate a roadmap, Instead of being overly critical of the contributors' positive responses.

I think this pr is fine
Even the current pekko is not affected. Which only happens when the native epoll transport is been used.
But it always better to fix critical issues like, and let users feel safe.
We are happy to announce the release of netty 4.1.99.Final. This release fixes a bug which could lead to SIGSERV while running on JDK21+. As JDK21 was recently released and is considered an LTS release we did decided to do this emergency release. Because of the nature of this bug (crashing the JVM) you should upgrade ASAP.
Netty suggests upgrading ASAP.
But yes, the scala steward is better but we can have this in before that.
@Roiocam Would you like to reopen it?

Netty 4 support for Pekko has never had a release. We don't need to rush netty upgrades. All they affect are our CI runs. netty 4.1.100 will probably be out soon. We can just upgrade occasionally or once in the run up to a release.

Netty 4 support for Pekko has never had a release. We don't need to rush netty upgrades. All they affect are our CI runs. netty 4.1.100 will probably be out soon. We can just upgrade occasionally or once in the run up to a release.
Java 21 release is also a real mess because of a crazy last minute license change - see https://adoptium.net/en-GB/blog/2023/09/temurin21-delay/

Good to know, but yes, not to rush means we have a clean commits line too.

4.1.100.Final is out but this is a security release so we will want to update
https://nvd.nist.gov/vuln/detail/CVE-2023-44487

4.1.100.Final is out but this is a security release so we will want to update
https://nvd.nist.gov/vuln/detail/CVE-2023-44487

no doubt right


Confusing note about breaking changes

First and foremost thank you for creating this GitHub action! 💛 We've been using it for a while now and it's been a great help.
I was reviewing recent changes and saw in README:

Breaking change from v2.x and onwards

I checked the current version of the action - 1.25.2. Then I tried to find version 2.x released and couldn't. After looking at the recent commits I realised that 2.x most likely refers to version 2.0 of the image - https://github.com/koslib/helm-eks-action/commit/1e6fdfac3e06f9a6d14a2d1ac96a014de6a0974d and 1.25.2 already contains the breaking change.
Can you confirm if my reasoning is correct? If yes, I think that it would make sense to clarify the note in the README.

Hi @adamniedzielski and thanks for your kind words - also glad you find this action useful!
In retrospect, I find this indeed confusing. To be clear, in version v1.25.2 the base docker image tag has indeed been changed to 2.0. However the action itself does not contain any breaking changes. This is because the KUBE_CONFIG_DATA env var is still needed, and it doesn't matter if it's loaded from the repo secrets or if it's dynamically fed from a previous step setting an env variable key/value.
I see some options moving forward and I'd like your input:

Remove this note as it is causing confusion, and replace it with a note pinpointing to a more secure way of providing the kube context data without having to fill it base64-ed in the repo secrets.
Should the major version of the action and the base docker image be aligned? Now that I am thinking this is kinda makes sense - and I'd appreciate a lot your thoughts!

Btw if you're interested in helping out maintaining/extending the action lmk :)

Hi again @adamniedzielski I've went ahead and closed this issue after attempting to make the readme to look les confusing. Let me know if you think it still has room for improvement in this regard or feel free to submit a PR :) Thanks again!


Improve lottery plugin

Remove all unnecessary this.errorReply from this.can function.

Done @panpawn.

Thanks


fix(ui) use e-mail as label consistently, fixes #380

see #380

I think we misunderstood #380.
The correct spelling should be "Email" not "E-mail", so it should only be changed in the login screens:

Good point, thanks for adding this comment. I updated the PR accordingly.


Seperate Infrastructure with a jumpbox

Hello,
I'm following your RanchCast, and I found it very interesting :)
I have a question regarding this topic, how do you plan to setup the jumpbox? And more precisely, are you planning to split kubectl access vs ingress access?
Thanks for your answer.


Correction for raw affiliation CIRAD, UMR AGAP, Saint Louis, Senegal

Correction needed for raw affiliation CIRAD, UMR AGAP, Saint Louis, Senegal
raw_affiliation_name: CIRAD, UMR AGAP, Saint Louis, Senegal
new_rors: ;02w4exq36
previous_rors:
works_examples: W2054543534
contact: aef6da75ffa258ae292a4156141d30a2:f28a8fb667ffdeaf29c495e5deb7d2c6c7ff @ inrae.fr

This issue was accepted and ingested by the OpenAlex team on 2024-11-18. The new affiliations should be visible within the next 7 days.


Use UIScreen over keyWindow to get size, so that size can be used without host application

Before without a host app the keyWindow would be nil, resulting in the screen size being reported as "0x0", now the size will be correct.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Colton Schlosser seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

@alanzeino Can I get some feedback here? This is an improvement to using this framework for libraries which don't use host apps.

@reidmain Thanks for the review :) Now just need someone to merge!
The dimensions matter because without them the file names aren't unique, for example when using the following devices, here are the suffixes. Without the screensize the names would conflict.
'platform=iOS Simulator,name=iPhone 7,OS=11.3' -><EMAIL_ADDRESS>'platform=iOS Simulator,name=iPhone SE,OS=11.3' -><EMAIL_ADDRESS>
'platform=iOS Simulator,name=iPhone X,OS=12.2' -><EMAIL_ADDRESS>'platform=iOS Simulator,name=iPhone 8 Plus,OS=12.2' -><EMAIL_ADDRESS>'platform=iOS Simulator,name=iPhone Xs Max,OS=12.2' -><EMAIL_ADDRESS>
Long term the size of the view might be a better solution, but this works out fine :)

@reidmain Thanks for the review :) Now just need someone to merge!
The dimensions matter because without them the file names aren't unique, for example when using the following devices, here are the suffixes. Without the screensize the names would conflict.
'platform=iOS Simulator,name=iPhone 7,OS=11.3' -><EMAIL_ADDRESS>'platform=iOS Simulator,name=iPhone SE,OS=11.3' -><EMAIL_ADDRESS>
'platform=iOS Simulator,name=iPhone X,OS=12.2' -><EMAIL_ADDRESS>'platform=iOS Simulator,name=iPhone 8 Plus,OS=12.2' -><EMAIL_ADDRESS>'platform=iOS Simulator,name=iPhone Xs Max,OS=12.2' -><EMAIL_ADDRESS>
Long term the size of the view might be a better solution, but this works out fine :)

Ah of course. This is an option mask so if you don't include the name of the device you can't unique it. That was the assumption I was making.
Alright to not make any breaking changes I'll merge this right now and we can look into addressing the larger issue in the future. Thanks again @cltnschlosser.


Implement command 'sign-eif'

Issue #, if available:
#204
Description of changes:

Implemented 'sign-eif' which can be used to sign an existing EIF by using a private key or KMS.
Add new arguments for 'describe-eif' for extracting certificate information for images signed with KMS.
Add new arguments for 'run-enclave' for validating KMS signed images.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@pro-vlad Thanks for this! Are you still planning to get it merged? We're looking for this feature


Remove deprecated resources

Description
Remove all resources markes as deprecated (see #186)
Migrations required

Remove aws_zone variable. It's no longer in use. Delete it.
Replace runners_off_peak_* variables with runners_machine_autoscaling.
Remove enable_forced_updates variable. Use enable_asg_recreation instead.

Verification
Deployed the new version of the module into our production environment and ran some build jobs. Worked fine.
Documentation
We use pre-commit to update the Terraform inputs and outputs in the documentation via terraform-docs. Ensure you have installed those components.

We have no issues with dnd, so just would like to learn what the issue is with dind. See here an example workflow that just work fines:
build:
  image: docker:latest
  services:
    - docker:dind
  script:
    - docker version
    - docker build --pull -t blurk .
    - docker run blurk


uia APP.kill() bug

I'm not sure about this, but my observation is that APP.kill() depending on the state of the program throws an error or "halfly terminates" probably because of trying to press ESC on a non existing control/window.
Example 1 : windows magnifier. This only happens, when you click on the dropdown menu AND the window itself was focused.
Example 2 : OBS. This only happens, when the Stats dialog is open AND the OBS window itself was focused.
Example 3 : windows magnifier half termination -> simple example code is APP = pywinauto.Application(backend="uia").start("magnify.exe") and APP.kill(). If you have docked magnifier, the top of the screen remains "docked". Also I have high confidence in that the application still lingers in the background/memory (huge performance drop while writing this issue), have no idea if this is directly related to pywinatuo APP.kill().
Example 1 code (Make sure to rename "Nagyito" to Magnifier maybe, name of the window and "Nezetek" to Views maybe, the dropdown with 3 values):
import pywinauto
from pywinauto import actionlogger
actionlogger.enable()

APP = pywinauto.Application(backend="uia").start("magnify.exe")
APP.Nagyito.set_focus() # name of the window maybe Magnifier
APP.Nagyito.Nezetek.click_input() # name of the dropdown maybe Views
APP.kill()

Result of above code:
2018-08-15 11:48:47,341 INFO: Started magnify.exe application.
2018-08-15 11:48:48,710 INFO: Clicked Button "Nézetek" by left button mouse click at (267, 354)
2018-08-15 11:48:49,165 INFO: Typed text to the Menu: {ESC}
Traceback (most recent call last):
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\uia_defines.py", line 232, in get_elem_interface
    iface = cur_ptrn.QueryInterface(cls_name)
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\comtypes\__init__.py", line 1158, in QueryInterface
    self.__com_QueryInterface(byref(iid), byref(p))
ValueError: NULL COM pointer access

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\controls\uiawrapper.py", line 434, in close
    iface = self.iface_window
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\controls\uiawrapper.py", line 132, in __get__
    value = self.fget(obj)
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\controls\uiawrapper.py", line 322, in iface_window
    return uia_defs.get_elem_interface(elem, "Window")
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\uia_defines.py", line 234, in get_elem_interface
    raise NoPatternInterfaceError()
pywinauto.uia_defines.NoPatternInterfaceError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/oliver.horvath/PycharmProjects/pywinautogithub/main4.py", line 10, in <module>
    APP.kill()
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\application.py", line 1247, in kill
    win.close()
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\controls\uiawrapper.py", line 440, in close
    self.type_keys("{ESC}")
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\base_wrapper.py", line 882, in type_keys
    self.verify_actionable()
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\base_wrapper.py", line 626, in verify_actionable
    self.verify_visible()
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\base_wrapper.py", line 649, in verify_visible
    raise ElementNotVisible()
pywinauto.base_wrapper.ElementNotVisible

Example 2 is with OBS:

Enable the option "Open Stats dialog on startup" in General, 2nd option maybe.

import pywinauto
from pywinauto import actionlogger
actionlogger.enable()

APP = pywinauto.Application(backend="uia").start(work_dir=r"C:\Program Files (x86)\obs-studio\bin\64bit", cmd_line=r"C:\Program Files (x86)\obs-studio\bin\64bit\obs64.exe")
APP.Dialog.set_focus() # OBS window
APP.kill()

Result of above code:
Traceback (most recent call last):
  File "C:/Users/oliver.horvath/PycharmProjects/pywinautogithub/main4.py", line 11, in <module>
    APP.kill()
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\application.py", line 1247, in kill
    win.close()
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\controls\uiawrapper.py", line 434, in close
    iface = self.iface_window
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\controls\uiawrapper.py", line 132, in __get__
    value = self.fget(obj)
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\controls\uiawrapper.py", line 322, in iface_window
    return uia_defs.get_elem_interface(elem, "Window")
  File "C:\Users\oliver.horvath\venv\pywinautogithub\lib\site-packages\pywinauto\uia_defines.py", line 231, in get_elem_interface
    cur_ptrn = element_info.GetCurrentPattern(ptrn_id)
_ctypes.COMError: (-2147220991, 'Az esemény nem tudott meghívni előfizetőt', (None, None, None, 0, None))

Translation of last line: "The event was unable to invoke a subscriber".

For the first example I could reproduce only app.kill() bug. .set_focus() worked for me though I caught (740, 'CreateProcess', 'The requested operation requires elevation.') and had to run Magnifier manually and then connect to it. It's very strange because my Python process was running as Administrator for sure. Haven't figured it out yet. One more hidden level of elevation?
Didn't try OBS Studio yet. But this issue is going to be the next.

Addional elevation might be a feature of pretty new Windows 10 RS4 I use: Version 1803 (OS Build 17134.407).

@meshuggahtas Is it possible to test on this branch of my fork? It doesn't try to close anything now, it just kills the app. Nothing more.

Using that branch:
Example 1:

OK, works great, no null com pointer error in 20 runs.

Example 3:

Okay, closing works and no memory issue (in the original issue, it was like some part of magnify still ran).
Still remains docked when closing, which shouldn't be the case in normal operation (opening magnify from start menu for example and closing the application will remove the dock area from the top).

Method .kill() doesn't assume any chance for the app to perform any un-initialization like restoring window layout. If you want correct screen restore, use app.Magnifier.close() for particular window instead of app.kill() for the whole app.

You are right. Example 2 also works flawlessly, so the bugs written in this issue are solved by that app_kill_bug tree :) (Issue can be closed)

@meshuggahtas thanks for testing it! I'm going to close it after PR merge.

pywinauto 0.6.6 .kill(soft=True) sometimes throws this error:

ctypes.COMError: (-2147220991, 'An event was unable to invoke any of the subscribers',  (None, None, None, 0, None))

Full log:

Traceback (most recent call last):
File "C:/Users/oliver.horvath/PycharmProjects/pywinautogithub/main4.py", line 10, in
APP.kill(soft=True)
File "C:\Users\oliver.horvath\venv\pywinautogithub\Lib\site-packages\pywinauto-app_kill_bug\pywinauto\application.py", line 1248, in kill
win.close()
File "C:\Users\oliver.horvath\venv\pywinautogithub\Lib\site-packages\pywinauto-app_kill_bug\pywinauto\controls\uiawrapper.py", line 434, in close
iface = self.iface_window
File "C:\Users\oliver.horvath\venv\pywinautogithub\Lib\site-packages\pywinauto-app_kill_bug\pywinauto\controls\uiawrapper.py", line 132, in get
value = self.fget(obj)
File "C:\Users\oliver.horvath\venv\pywinautogithub\Lib\site-packages\pywinauto-app_kill_bug\pywinauto\controls\uiawrapper.py", line 322, in iface_window
return uia_defs.get_elem_interface(elem, "Window")
File "C:\Users\oliver.horvath\venv\pywinautogithub\Lib\site-packages\pywinauto-app_kill_bug\pywinauto\uia_defines.py", line 231, in get_elem_interface
cur_ptrn = element_info.GetCurrentPattern(ptrn_id)
_ctypes.COMError: (-2147220991, 'Az esemény nem tudott meghívni előfizetőt', (None, None, None, 0, None))

Example code where I get this error thrown after 1~10 tries (seems like a timing related problem):
import pywinauto, time
from pywinauto import actionlogger
actionlogger.enable()

for i in range(0,30):
    print(i)
    APP = pywinauto.Application(backend="uia").start("magnify.exe")
    APP.Nagyito.set_focus()
    APP.Nagyito.Nezetek.click()
    APP.kill(soft=True)

Otherwise the soft=True nicely closes the application (Magnify in my case),

Thanks for the concise report. We probably, just need to handle gracefully the disapperead windows.

Closing as implemented in atspi branch. It helps us to track progress for 0.7.0 milestone. Thanks for understanding!


Expose parameter to choice which format list to display

Description
We want to only display dot format list
Development
Expose constructor to specify which formats we want
Screenshot

Thanks for doing this. I want to double-check if the change has any impact on the menu in the Catalyst case, since its content is driven from ToolbarContent.


Multinomial parse

Adds new operation rand_uniform and refactors multinomial op. to accept dynamic shape inputs.  These changes will allow multinomial distributions to be created from random numbers at runtime.
The ONNX Multinomial instruction is implemented as a combination of rand_uniform and multinomial operations, since Migraphx' Multinomial doesn't generate a sequence of random numbers but requires them to be passed as an argument.
Use of random seed in rand_uniform is complicated, with a lot of command permutations.  This change allows four ways of seeding the random sequence:

Specify a seed when compiling the model.  This will always result in the same pseudo-random sequence every time the model is run, although it may not be apparent if inputs are of dynamic size.
Specify a seed at runtime.
Instruct the operation to auto-generate seeds with a runtime input.
Command auto-seeding at compile time.  This is the Onnx specified behavior when a seed isn't given.

Options #2 and #3 don't apply to Onnx, since it doesn't recognize separate compile and runtime steps for a model.  The advantage of these choices is that the same compiled model can either use a specified seed and thus be repeatable, or auto-generate a seed if none is given.
In order to support options #2 and #3, we would need to make the seed optional at runtime, not just at compile time.  This presents a potential conflict in that output-lens, the shape of the output, is also an optional argument because we might have to override the compiled shape in the case of dynamic inputs.  A language such as Migraphx with positional arguments can't have an operation with two optional arguments because if only one is provided at runtime, there is no (simple) way to distinguish which one it is.
The solution to this conflict in this implementation is that an output shape argument is optional but a seed argument is required.  If there is at least one value in the seed argument, it is used.  To command auto-seeding when running an inference, input an argument with a 0-size shape.  A shape with no rank (an empty lens() or dyn_dims() vector) is a runtime error, but a shape with rank one containing a length of 0 represents an empty data array and is legal.  For case #4, there is a flag use_auto_seed which, if set, causes auto-seeding no matter what seed value is passed.  This latter supports the Onnx requirement.
In Onnx parsing, then, use_auto_seed is set if no seed argument is present.

Merged the random_uniform and random_seed changes which are now in Develop branch, and updated tests to match.

Needs #2362  to be merged first


Update package.json

Fix path issue that will cause npm install to fail on Windows. See related issue here: https://github.com/npm/npm/issues/2576

I should add, I am not able to test this change on a Mac.

thats a bit annoying....silly windows. This approach won't work on the mac as the path won't resolve. I might need to pursue a different solution all together (fortunately i just saw one recently). I'll leave this here for reference until I can try this other solution.

Hi guys,
FYI, there is a PR in original repo to remove the need of bower:
https://github.com/chrisgladd/grunt-phantomcss/pull/26
You can give it a try. I used it and it was fine.

I've switched phantomcss to being pulled in via NPM, so there isn't a bower dependency any longer. This means I took the offending line on out. Hope this helps!

Great! Good news!


Full support for CUDA device attributes

Rounded out the device attribute enums to support all currently know CUDA device attributes.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@95ab2fa). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff            @@
##             master   #2867   +/-   ##
========================================
  Coverage          ?   86.1%
========================================
  Files             ?     324
  Lines             ?   67631
  Branches          ?    7597
========================================
  Hits              ?   58235
  Misses            ?    8181
  Partials          ?    1215

Thanks for the contribution!  Surprising to see how many CUDA device attributes have been added over the years.  :)


Enhancement: Possibility to set response body to object instead of string in the schema.json

Hi,
Use Case:
The response body to my requests is always json.
When the simulation.json file is created, just like it says in the schema.json, the body is treated as a string.
So the simulation.json file looks for example like this
...
"body": "{\"key\":\"value\"}"
...
It would be really nice if the body could be treated as an object instead (some setting somewhere).
Resulting in simulation.json to look like this
...
"body":{"key":"value"}
...
Then simulation.json is directly passed in whatever json viewer you are using and you can more easily analyize the payload without extra steps
Can this be something of interest?
Thanks!

We're aware that the JSON within JSON is quite difficult to reason about due to the escape characters. However, I am not sure about your suggestion as it would result in invalid JSON. What we are working on is a simulation editing UI which would do the unescaping for you. Does this sound like a viable alternative?

My suggestion with the schema.json set to object for body was more of a conceptual one.
At the same time data added to the simulation.json has to be in the right format of course.
I guess this is a matter of how hoverfly handles the response body in the first place.
Im happy and grateful with all your work and a UI does sound like a viable alternative.
thanks


chore: add semantic release github

:tada: This issue has been resolved in version 2.0.4 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


I am unable to download from Paramount Plus with yt-dlp.

Checklist

[X] I'm asking a question and not reporting a bug/feature request
[X] I've looked through the README
[X] I've read the guidelines for opening an issue
[X] I've searched the bugtracker for similar questions including closed ones

Question
I wanted to know what command to download Paramount Plus?

yt-dlp URL

yt-dlp URL

yt-dlp URL

Pass your cookies for authentication https://github.com/yt-dlp/yt-dlp#authentication-options


Automatic install of NoMoreGrind/KCT configs for RP-0?

Is it possible for CKAN to automatically install the NoMoreGrind/KCT config files present in the RP-0 download? I see in the registry file there's a section to describe what goes where, but I'm not sure if certain technical limitations prevent these configs from being applied, especially since they're not full mods...

The only issue would be if KCT's settings need to be overwritten. If KCT ships with a default settings file, then RP-0 cannot overwrite it and the installation will throw an error.
We can still work around that though using config packages like we are doing for RemoteTech.

Ah, that makes sense. Would this be the right place to request such a thing, or should I go to the forums?

No, github is definitely our preferred channel. I'll see to it as soon as I have the time.
In the meantime, if you'd like to help out you can take a look at how remote tech is doing it and try to do the same :)

All right, thanks! I'll see what I can do. I'm not very familiar with the github pull request model, though, so I may have to pop back for some questions later...

Alright, took a look at how RemoteTech works, as well as the files already present in No More Grind/KCT...
KCT seems to use the same strategy that RemoteTech does (generate default configs if none found), so I think I can make a config file for that. I'll try tomorrow, once I get some sleep.
No More Grind, though, provides its own config file, so (as I found out the hard way) CKAN won't copy the RP-0 config even if I delete the default one manually (I think). In this case, would I have to submit a pull request to the No More Grind author to do something more like KCT/RemoteTech? I could easily be missing something though since I'm not very familiar with CKAN...

Oh, also, what should I put for author/license/version numbers?

Well, I guess in this case NoMoreGrind has a conflict with RP-0. We should mark them as incompatible in the metadata, I believe.
As for the author, license, etc, I say you should just copy the ones from RP-0 (since the file still comes from the same download). Thanks for helping us out! :)

So tried a pull request, not sure if that's the right way to go about this?
How would marking RP-0/No More Grind as incompatible work, though, since it seems those two are updated automatically? Would I need to make the changes over at the NetKAN repo?

Sorry I haven't seen this earlier, I'm the author of No More Grind. I think a much better approach would be to incorporate the source into RP-0 as it's really 50ish lines of code. Here is the whole source, which I now release in the public domain so you can use it however.

Gosh dang it, I'm on both the CKAN and RP-0 projects. And yes, we definitely want CKAN to install the KCT and NMG files.
Given that NMG is so small, I've no problems compiling it directly into RP-0 as @AlexanderDzhoganov  suggests. Alternatively, rather than having NMG read a config file directly, we can tweak it to read from ConfigNodes, which means not having to worry about overwriting files (we can use ModuleManager stanzas to do config), although that does change existing behaviour. We can also do both, which feels like the best solution (we compile NMG into RP-0, but tweak it to read from ConfigNodes). cc'ing in @NathanKell , although I'm happy to do the work here.
KCT seems to only create config files if they don't already exist, so @aw94's metadata should work, although it'd be great to:

[ ] Have the KCT RP-0 config be a netkan file that triggers off RP-0 releases.
[ ] Have RP-0 metadata suggest or recommend the KCT config.

~ pjf

I'll see if I can figure out the netkan file for KCT.

That sounds good to me. I won't be doing coding until I get home though, so if you'd rather not wait a week do please go ahead and do that. I'm happy to help via IRC though. :)

Alright, @pjf, KCT stuff seems to be done.

I'm going to throw out a wild guess at this being done in our parts so I'm closing this in hopes of being correct. If not poke it open again :)


fixed trigger names and added the trigMenu includes to the validation jobs

fixed trigger names used in a few SelecEvents :[
switched to the trigger validation menu in the nightly jobs

Hi @gianipez,
You have proposed changes to files in these packages:

Validation
JobConfig

which require these tests: build.
@Mu2e/write, @Mu2e/fnalbuild-users have access to CI actions on main.
:hourglass: The following tests have been triggered for 7a388bb51c638670cbea4f9bb0b279b48fc1e80c: build  (Build queue has 6 jobs)
About FNALbuild. Code review on Mu2e/Offline.

:sunny: The build tests passed at 7a388bb51c638670cbea4f9bb0b279b48fc1e80c.

Test
Result
Details

test with
:white_check_mark:
Command did not list any other PRs to include

merge
:white_check_mark:
Merged 7a388bb51c638670cbea4f9bb0b279b48fc1e80c at 6852538baacf6880fed1d6dce93b0713be3ab3c2

build (prof)
:white_check_mark:
Log file. Build time: 11 min 27 sec

ceSimReco
:white_check_mark:
Log file.

g4test_03MT
:white_check_mark:
Log file.

transportOnly
:white_check_mark:
Log file.

POT
:white_check_mark:
Log file.

g4study
:white_check_mark:
Log file.

cosmicSimReco
:white_check_mark:
Log file.

cosmicOffSpill
:white_check_mark:
Log file.

ceSteps
:white_check_mark:
Log file.

ceDigi
:white_check_mark:
Log file.

muDauSteps
:white_check_mark:
Log file.

ceMix
:white_check_mark:
Log file.

rootOverlaps
:white_check_mark:
Log file.

g4surfaceCheck
:white_check_mark:
Log file.

FIXME, TODO
:white_check_mark:
TODO (0) FIXME (0) in 0 files

clang-tidy
:white_check_mark:
0 errors 0 warnings

N.B. These results were obtained from a build of this Pull Request at 7a388bb51c638670cbea4f9bb0b279b48fc1e80c after being merged into the base branch at 6852538baacf6880fed1d6dce93b0713be3ab3c2.
For more information, please check the job page here.
Build artifacts are deleted after 5 days. If this is not desired, select Keep this build forever on the job page.


Missing early ReferenceError for some invalid simple assignment targets

For example:
async function foo() {
  (await 1) = 1;
}

Expected: early ReferenceError
Actual: no error

To be clear, this is not just a bug with async functions. We are missing early errors for such things as:
function foo() {
  1++;
}

Expected: early ReferenceError
Actual: no error

@pleath tried making this an early error iirc but ran into problems? And Chrome tried too but also had problems?

We used to have an incomplete/inconsistent set of assignments to invalid LHS's that would result in early error. I completed the set, but we had compat issues, particularly with foo() = x. Maybe the best we can do right now is throw early error on all invalid LHS except function call result.

In master all cases mentioned in this issue have been fixed.


Boxstarter gets installed to top of PSModulePath and overwrites Write-Host

Checklist

[X] I have verified this is the correct repository for opening this issue.
[X] I have verified no other issues exist related to my problem.
[X] I have verified this is not an issue for a specific package.
[X] I have verified this issue is not security related.

What You Are Seeing?
Any usage of Write-Host leads to Boxstarter modules being loaded. (which slows down execution of arbitrary PowerShell scripts)
What is Expected?
Boxstarter modules should only be loaded during execution of Boxstarter/Chocolatey sessions.
How Did You Get This To Happen?

Install Boxstarter
use Write-Host in a PowerShell script
take a look at $env:PSModulePath / Get-Command Write-Host

System Details

Operating System:
Windows PowerShell version:
Chocolatey CLI Version:
Chocolatey Licensed Extension version:
Chocolatey License type:
Terminal/Emulator:

Installed Packages
Boxstarter v3.0

Output Log
-

Additional Context
No response

most likely it would be possible to temporary add Boxstarter modules into PSModulePath and remove it in Cleanup-Boxstarter

not so straight-forward as initially suggested 😞
removing Boxstarter.BaseDir altogether would be a breaking change - currently looking into moving it to the end of *path variables, - this way it would only jump into action if a Boxstarter module get's explicitly loaded (such as in Boxstarter-choco extension module).
@flcdrg any thoughts?

Is there a way to use the modules without adding them all to PSModulePath? (or maybe just add one or some of them)

Is there a way to use the modules without adding them all to PSModulePath? (or maybe just add one or some of them)

There might be. It's been a while since I was able to take a look at this.
I can confirm this change does what I wanted it to / fixes a problem.
If that's okay with you I'd like to get this trough in order to fix the Bug it addresses and create a follow-up issue/improvement that'll try to rip out the modules from PSModulePath and dynama-load them just from the Chocolatey extension/ BoxstarterShell script.

Yes, fine with me


macOS compatibility

Hi!
Loved playing around with this package. Made a quick update to allow for macOS as well, so GlobeView() is available there as well. Updated to iOS14 min requirement due to UIColor inits with SwiftUI's Color

Hey pexavc, this is great! Thank you for the update!


"Maritime Domain Awareness" web map references URL that I can't access

web map 995d354c59694c01bbd1ff877441526a references the url...
http://geo.weather.gc.ca/geomet/?lang=E
When I access this url I receive an error that this site can't be reached.

I can see the information from that WMS service in a web map. When trying to access this URL from a browser:
https://geo.weather.gc.ca/geomet/?lang=E&service=WMS&request=GetCapabilities
You cannot reach it, but adding the WMS url in a web map the information shows up properly.
Please test URL in a web map and close issue. Very similar to issue #1529

@jfrygeo Tried to add new wms service using this url ( https://geo.weather.gc.ca/geomet/?lang=E&service=WMS&request=GetCapabilities) in a web map and received the following error:
"The WMS service, https://geo.weather.gc.ca/geomet/?lang=E, cannot be added to the map. It is either not available or you have entered an invalid URL for the type of layer you want to reference."

@jfrygeo Haven't been able to add the WMS url to a web map. Each attempt has thrown errors. I'm going to go ahead and close this issue since you have been able to get this to work.


[DNS] zone import should strip trailing .@ from relative record names

The following is a legitimate DNS record from a BIND zone file:
_sip._tls.@	3600 	IN	SRV	100	1	443	sipdir.online.lync.com
Note the '.@' in the record name, which currently causes import to fail (Bad Request).
Where a record name ends in .@, we should strip it off (and then apply the current value of $ORIGIN).

@MyronFanQiu  add to S162.

Could you provide some related file contents? It would be helpful.


Leverage improvements in openjdk 16

Openjdk 16 is going to be released next month
https://openjdk.java.net/projects/jdk/16/
possible improvements
Unix domain socket
https://openjdk.java.net/jeps/380
https://inside.java/2021/02/03/jep380-unix-domain-sockets-channels/
Netty could revamp it's UnixDomainSocket support to leverage openjdk support. This might alter performance, feature completeness and OS support (Windows?)
native code binding
Netty make extensive use of the JNI
https://openjdk.java.net/jeps/389
https://inside.java/2020/12/21/podcast-010/
The foreign linker API (and the jextract tool) improve over the JNI in many ways and this could alter performance.
exploiting SIMD
Netty is a foundational, performance critical library, as such it could benefit from intra core parallelism (vector operations)
which are now exposed through the vector API
https://openjdk.java.net/jeps/338
older JDKs improvements to leverage:
Java 9 added the @HotSpotIntrinsicCandidate annotation. Methods annotated with that annotation will have intrinsics for them.
intrinsics functions are direct call functions that inline java calls as direct assembly
intrinsics

We can't rely on any incubator modules because they freely change their public API between releases. And @HotSpotIntrinsicCandidate is internal to the JDK.

@chrisvest well you could detect if the VM is openjdk 16, if it is > 16 then fallback to the old cold path like for pre 16.
Once jdk 17 is released, manually check if the breaking changes break the existing implementation, if not then add 17 to the accepted range.
Once the feature is out of preview, accept any super jdk version

JEP 380 is no longer incubating: https://openjdk.java.net/projects/jdk/16/
The NioSocketChannel could support Unix Domain Sockets without any additional library just by leveraging the existing corresponding socket channel in the JDK. When JDK < 16, just fallback to the previous limiations.

With JDK 16, one can do this:
var channel = SocketChannel.open(StandardProtocolFamily.UNIX);
channel.connect(UnixDomainSocketAddress.of("/var/run/some_socket.sock"));
writer = Channels.newWriter(channel, StandardCharsets.UTF_8);
// Write stuff to channel using writer ..

I happy to review a PR ...

Also it is a bit unfortunate as Netty's SocketChannel requires InetSocketAddress

Just as FYI, I have been running benchmarks of Netty (with WebFlux) using Unix Domain Sockets with Netty. I am observing a slight decrease in performance compared to HTTP2 for the same load tests. If there is a better way to implement Unix Domain Sockets with Java 16, it may have significant performance improvements.

Nobody else wanting this in Netty?

@murphye you might wanna test Netty unix domain support (not the jdk 16 one) that use the io_uring API.
https://github.com/netty/netty-incubator-transport-io_uring/pull/64
(althought it seems to be a WIP currently)
Io_uring can lead to significant performance improvements.

what's the status of this?

There is a new JEP that officially mention Netty, thought it might interest you:
https://openjdk.java.net/jeps/423

Hi, this is Thomas from the Vert.x team. I would like to try contributing support for Unix Domain Sockets for users of JDK16+.
Our users would benefit from it when deploying apps to clouds that simplify secure DB connectivity by providing a Domain Socket (e.g. Google Cloud with PostgreSQL or MySQL).
As part of the JDK, the API is also available to applications compiled with GraalVM's native-image.
Is this something you would consider integrating? I talked with @franz1981 offline and he advised to start the conversation here.
In this case, shall I create a separate issue to track the effort? This one is a mix of several RFEs.

@tsegismont We already have support in the Netty 5 alphas, because we've streamlined the channel types. The annoying thing with Netty 4.1 is how domain sockets have their own set of channel types per transport, so presumably this would be a new pair of channels for the Nio transport that only work on JDK 16+, but it would also be a lot of changes to the interfaces to make the domain socket APIs available to the transport module - APIs that are currently defined in the transport-native-unix-common module. It's a lot of changes to the public API surface, which is why we haven't done it.
Maybe there's a simpler way to do it, though as @normanmaurer mention above, SocketChannel requires InetSocketAddress. The anticipated size and complexity of the change is my main reservation, but I'm otherwise not against the idea.

I agree with @chrisvest ... I think the best bet is netty 5.

Thank you both for the update.

Hi, I am proposing a simpler way of adding UDS support in netty 4.x
Please have a look at #13965


An option to make napalm extendable

I briefly spoke to @ktbyers about this, and I do not expect this to go in as described here, just wanted to use it as the basis of discussion.
Napalm's ability to work with multiple vendors is amazing, but I run into situations where I simply know my ask does not fit into a vendor neutral model or deals with proprietary one-off protocols. This is made more difficult when using ansible for instance, as I only have access to the defined getters... without modifying and managing that code.
I was looking for a way to extend napalm in a somewhat predictable matter. Here is what I came up with. You would need to have a custom_napalm_* library and Custom* class to import anywhere you wanted to use the additional features.
respective driver update:
(napalm-dir) ken@kcelenza-ubuntu:/github/sample/1/napalm-ios$ git diff
diff --git a/napalm_ios/__init__.py b/napalm_ios/__init__.py
index a322d7f..2aba9d5 100644
--- a/napalm_ios/__init__.py
+++ b/napalm_ios/__init__.py
@@ -14,7 +14,10 @@

 """napalm_ios package."""
 import pkg_resources
-from napalm_ios.ios import IOSDriver
+try:
+    from custom_napalm.ios import CustomIOSDriver as IOSDriver
+except:
+    from napalm_ios.ios import IOSDriver

 try:
     __version__ = pkg_resources.get_distribution('napalm-ios').version
diff --git a/napalm_ios/ios.py b/napalm_ios/ios.py
index 2447e73..c027f70 100644
--- a/napalm_ios/ios.py
+++ b/napalm_ios/ios.py
@@ -25,7 +25,10 @@ import telnetlib
 import copy

 from netmiko import ConnectHandler, FileTransfer, InLineTransfer
-from napalm_base.base import NetworkDriver
+try:
+    from custom_napalm.base import CustomNetworkDriver as NetworkDriver
+except ImportError:
+    from napalm_base.base import NetworkDriver
 from napalm_base.exceptions import ReplaceConfigException, MergeConfigException, \
             ConnectionClosedException, CommandErrorException

(napalm-dir) ken@kcelenza-ubuntu:/github/sample/1/napalm-ios$

example extension
(napalm-dir) ken@kcelenza-ubuntu:~/.virtualenvs/napalm-dir/local/lib/python2.7/site-packages/custom_napalm$ ls
base.py  __init__.py  ios.py
(napalm-dir) ken@kcelenza-ubuntu:~/.virtualenvs/napalm-dir/local/lib/python2.7/site-packages/custom_napalm$ cat base.py
from napalm_base.base import NetworkDriver
class CustomNetworkDriver(NetworkDriver):

    def get_eigrp(self):
        raise NotImplementedError

    def get_trustsec(self):
        raise NotImplementedError

(napalm-dir) ken@kcelenza-ubuntu:~/.virtualenvs/napalm-dir/local/lib/python2.7/site-packages/custom_napalm$ cat ios.py

from napalm_ios.ios import IOSDriver
class CustomIOSDriver(IOSDriver):
    """NAPALM Cisco IOS Handler."""

    def get_eigrp(self):
        ntp_servers = {}
        command = 'show run | include ntp server'
        output = self._send_command(command)

        for line in output.splitlines():
            split_line = line.split()
            if "vrf" == split_line[2]:
                ntp_servers[split_line[4]] = {}
            else:
                ntp_servers[split_line[2]] = {}

        return ntp_servers

(napalm-dir) ken@kcelenza-ubuntu:~/.virtualenvs/napalm-dir/local/lib/python2.7/site-packages/custom_napalm$

Coverage increased (+0.1%) to 61.141% when pulling 6ca770ee41beae8cb82270c826c4e9b0a4787f9a on itdependsnetworks:napalm_extendable into e21f10f5f26b02d15ed0e73fe4e6476638d991e7 on napalm-automation:develop.

An example of this working.
>>> from napalm import get_network_driver
>>>
>>> ios_device='<IP_ADDRESS>'
>>> ios_user='ntc'
>>> ios_password='password'
>>> driver = get_network_driver('ios')
>>> device = driver(ios_device, ios_user, ios_password)
>>> device.open()
>>> device.get_eigrp()
{u'<IP_ADDRESS>': {}}
>>> device.get_trustsec()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ken/.virtualenvs/napalm-dir/local/lib/python2.7/site-packages/custom_napalm/base.py", line 8, in get_trustsec
    raise NotImplementedError
NotImplementedError

Note: I just copied ntp code over to have the method return something

I think the key aspect of this is to make it work with things that are layered on top of NAPAlM (like Ansible and potentially Salt).
What @itdependsnetworks did can probably be simplified. I might try to mock something up over the weekend.

@itdependsnetworks Okay, I think you can just do this:
$ cat napalm_ios_custom.py
from napalm_ios.ios import IOSDriver

class CustomIOSDriver(IOSDriver):
    """NAPALM Cisco IOS Handler."""

    def get_eigrp(self):
        ntp_servers = {}
        command = 'show run | include ntp server'
        output = self._send_command(command)

        for line in output.splitlines():
            split_line = line.split()
            if "vrf" == split_line[2]:
                ntp_servers[split_line[4]] = {}
            else:
                ntp_servers[split_line[2]] = {}

        return ntp_servers

$ cat test_x.py
from napalm_base import get_network_driver
from getpass import getpass

ios_device='cisco1.twb-tech.com'
ios_user='pyclass'
ios_password=getpass()
driver = get_network_driver('ios_custom')
device = driver(ios_device, ios_user, ios_password)
device.open()

print
print "-" * 80
print "EIGRP"
print device.get_eigrp()
print
print "-" * 80
print "ARP Table"
print device.get_arp_table()
print
print "-" * 80
print "Method that doesn't exist"
print device.get_trustsec()

$ python test_x.py
Password:

--------------------------------------------------------------------------------
EIGRP
{u'<IP_ADDRESS>': {}, u'<IP_ADDRESS>': {}}

--------------------------------------------------------------------------------
ARP Table
[{u'interface': u'FastEthernet4', u'ip': u'<IP_ADDRESS>', u'mac': u'00:62:EC:29:70:FE', u'age': 8.0}, {u'interface': u'FastEthernet4', u'ip': u'<IP_ADDRESS>', u'mac': u'C8:9C:1D:EA:0E:B6', u'age': 0.0}, {u'interface': u'FastEthernet4', u'ip': u'<IP_ADDRESS>', u'mac': u'1C:6A:7A:AF:57:6C', u'age': 113.0}, {u'interface': u'FastEthernet4', u'ip': u'<IP_ADDRESS>', u'mac': u'52:54:AB:BE:5B:7B', u'age': 138.0}, {u'interface': u'FastEthernet4', u'ip': u'<IP_ADDRESS>', u'mac': u'52:54:AB:71:E1:19', u'age': 49.0}, {u'interface': u'FastEthernet4', u'ip': u'<IP_ADDRESS>', u'mac': u'52:54:AB:C7:26:AA', u'age': 140.0}, {u'interface': u'FastEthernet4', u'ip': u'<IP_ADDRESS>', u'mac': u'52:54:AB:3A:8D:26', u'age': 11.0}, {u'interface': u'FastEthernet4', u'ip': u'<IP_ADDRESS>', u'mac': u'00:01:00:FF:00:01', u'age': 156.0}, {u'interface': u'FastEthernet4', u'ip': u'<IP_ADDRESS>', u'mac': u'00:02:00:FF:00:01', u'age': 83.0}, {u'interface': u'FastEthernet4', u'ip': u'<IP_ADDRESS>', u'mac': u'64:64:9B:E8:08:C8', u'age': 4.0}, {u'interface': u'FastEthernet4', u'ip': u'<IP_ADDRESS>', u'mac': u'00:1C:C4:BF:82:6A', u'age': 16.0}, {u'interface': u'FastEthernet4', u'ip': u'<IP_ADDRESS>', u'mac': u'00:1B:78:73:56:34', u'age': 13.0}]

--------------------------------------------------------------------------------
Method that doesn't exist
Traceback (most recent call last):
  File "test_x.py", line 22, in <module>
    print device.get_trustsec()
AttributeError: 'CustomIOSDriver' object has no attribute 'get_trustsec'

We would then in napalm-ansible just need to make this not be an enumerated choices (i.e. switch it to just be a string:
dev_os=dict(type='str', required=False, choices=os_choices),

I don't know if this applies at all in Salt or StackStorm2 (i.e. what would be need to augment NAPALM in those contexts and what pattern would work).

The trick is that what you load here:
driver = get_network_driver('ios_custom')

Most match a Python module named napalm_ios_custom that is somewhere on your Python sys.path.

My suggestion would be to just implement the method in napalm-base. The EIGRP is a straightforward one and I don't think having a get_eigrp method would be terrible. For more vendor specific stuff, for example, qfabric, we could have junos_get_qfabric and for things that are a mixed combination like chassis we could have get_chassis and then have some common stuff and then a vendor dictionary inside were we put things that are unique to the platform.

@dbarrosop I think the broader issue is that end-users need a way to augment our methods and still have it work with napalm-ansible (without modifying napalm-ansible). I think we should eliminate the choices argument or at least give a way to turn it off (from an Ansible playbook).

Well, what’s the problem then? If someone wants to subclass a driver they are free to do so. We can change the ansible modules to not have hardcoded drivers and just rely on ger_network_drivers to find the correct one. I don’t think we have to change anything else for that.

Ok, reread your last comment and I just realized I said exactly the same as you... sorry, I am very tired today :P

No worries, I will make this change in napalm-ansible.

This certainly makes sense from napalm perspective, and I don't expect this to be implemented, but I figured I would give it the "old college try." EIGRP is just an example, I have dozens of these that I have to figure out on short notice based on customer requirements. My current solution is to use other solutions, e.g. ntc-templates, custom filters, etc.... However I would love the ability to use the same napalm methodology for these cases.

I think the right thing to do is to remove the restrictions in the ansible modules. That way you can have your own subdrivers inheriting from the main drivers and upstream changes at your own pace without being blocked by us. Would that work? Am I even managing to get my point across?

Everything you said makes sense.
It's difficult when managing multiple environments/customers. They will not be able to upgrade without having to carry code over, which most likely they would not understand.

A way to do that is by having your own repo as entry point. So instead of installing napalm they install ntc-napalm which basically has napalm as requirement and include your own subclasses

Understood, though that comes with it's own set of complexity.
Thanks for your consideration :)

I would suggest revisiting this after the reunification as get_network_drivers will certainly change then so we might be able to do something there.

I'll just submit the same idea with a new marketing spin and hope to sneak it in :)

I had some further discussions with @itdependsnetworks on this as I wasn't seeing how we weren't meeting his requirements.
He wants something like the following:
Look for custom_napalm_ios.py first and if that exists use that.
If that doesn't exist fall back to napalm_ios.py.

Basically try to use the custom module first per platform and then fallback to the standard module if the custom module doesn't exist.

Yeah, I realized about it when checking his proposed code, that's why I prefer to revisit the idea after the reunification, get_network_drivers will change dramatically by then so I'd rather look at this then. What I don't like is that we are conflating business logic with module discovery, the whole purpose of get_network_drivers being dynamic was for people to implement their own drivers but then I was expecting people to call custom modules with custom names like ntc_ios or my_driver_icant_publish_because_legal rather than just ios and rely on some customer order but anyway. As mentioned, let's revisit after the reunification.
@itdependsnetworks add some buzzwords like "intent based" or "software defined class discovery" and we will be all in ;)


Interface for MapBuilder

Defines an interface for MapBuilder that can be used to implement a gRPC stub.
RFC=0002

Diffbase is 4f1e39d

@wohe The idea is that we have a gRPC-forwarding implementation of the MapBuilderInterface and the TrajectoryBuilder.


Port-Folio: Contact-Me Section

Contact Me section updated and completed.

Thanks so much wuletawwonte for your honest review. I got to learn a lot from the review. Thanks again.


Overhaul the agent install process

Hello Puppet People,
The Puppet Pipelines team is taking ownership of this code base and the following points motivated us to do a substantial rewrite of this puppet module:
[1] Much as the puppet agent can be installed with a curl | bash (on *NIX platforms) and curl | powershell (on Windows), the Distelli agent also has this capability. Therefore, the puppet code should use this capability rather than modeling the various files that should be downloaded... and if that capability doesn't work in any way, we should be fixing it on the https://pipelines.puppet.com/download/client and https://pipelines.puppet.com/download/client.ps1 download URLs.
[2] On Windows, the distelli.exe must be a symlink since any persistent processes prevent the update of the distelli.exe binary if it is not a symlink. The powershell script that does the default agent install already does this.
[3] Customers need ways to configure parameters that were previously not configurable. For example, the download location of the distelli executable and the "-data-dir" option passed into the agent which allows multiple agents to run on the same host. Also, having the ability to NOT start the agent.
[4] Various newer products don't allow downloading an arbitrary agent version, and using the default agent version for your server install is a recommended best practice. This also simplifies endpoint configuration (since using the AgentEndpoint configuration was deprecated many years ago), and allows one to properly download the agent from a PfA/PfC/CD4PE/etc installation.
I apologize that this change is a substantial departure from the existing code base, but it is necessary to make this module work properly in all the various environments it may be used in.
Special Thanks to:

Eric Williamson @ericwilliamson
Reid Vandewiele @reidmv
Ethan Brown @iristyle
Michael Lombardi @michaeltlombardi
Bill (aka William) Hurt @RandomNoun7
James Pogran @jpogran

I've manually tested this change on Windows 7 and CentOS 5, since I figure these are "lowest common denominator" platforms. I'll continue to test on other platforms and update the documentation with the latest configuration options (see agent.pp).
I've also ran the pdk update command with version 1.7.1 of the pdk, and ran the pdk validate and pdk test unit commands with success.
Are there other ways that I should be testing this change?
Thanks,
-Brian

Tested on: win-10-ent-i386, win-7-x86_64, centos-5-i386 with these commands:
WINDOWS:
curl -k -o install.ps1 'https://PE-HOST/packages/current/install.ps1'
cat install.ps1 | powershell -NoProfile -ExecutionPolicy Bypass -Command -
(approve certificate, add to test Classification)
/cygdrive/c/Program\ Files/Puppet\ Labs/Puppet/bin/puppet.bat agent -t --debug
UNIX:
curl -k -o /tmp/install.bash 'https://PE-HOST/packages/current/install.bash'
scp /tmp/install.bash iomn30gyynmas2r.delivery.puppetlabs.net:.
cat install.bash | sudo bash
(approve certificate, add to test Classification)
puppet agent -t --debug
(note that I'm scp'ing the install.bash for platforms where curl has a broken SSL, namely centos-5-i386)


#1791 - Enable Assessments tab view and NOA view for Public Institution Users - Part 3

Created E2E tests for, institution noa, award, history, request, appeal, and exception controllers.

Tests for decorator @IsBCPublicInstitution() and @HasStudentDataAccess("studentId") are mostly the same in all tests.
Application appeal DTOs were interchanged and updated the logic to use the correct DTO.
Moved MASKED_MSFAA_NUMBER and StudentAssessmentStatus to libs.
Created methods createFakeDisbursementReceiptValue and createFakeDisbursementReceipts.
Restricted the staff section of the application exception on the API side for institution users.
Confirmed with @JasonCTang about adding the confirmation of COE -> Institution doesn't need the feature to confirm the COE from the assessment page.

Next PR

Offering change and scholastic change related changes and related e2e

Thanks for all the hard work put into this effort @ann-aot . Please have a look at the comments.


Show Prefix/Suffix by default

I would like to know whether there is anyway to forcefully show prefix/suffix even without hovering or focus.

@mlhk2007 ,
set the clearMaskOnLostFocus option to false


Invalid expression in generated resolver when using keyField in named connections

Hi there,
When using a named connection for many-one models with the keyField specified, the generated resolvers do not use the keyField attribute, instead it uses the default column name
E.g. the many-one between Company and Blogs:
type Company @model {
  id: ID!
  name: String
  blogs: [Blog] @connection(name: "CompanyBlogs")
}

type Blog @model
   @key(fields: ["company","id"]) {
  company: ID!
  id: ID!
  Company: Company @connection(name: "CompanyBlogs", keyField: "company")
  name: String!
  posts: [Post] @connection(name: "BlogPosts")
}

type Post @model @key(fields: ["company","id"]) {
  company: ID!
  id: ID!
  Company: Company @connection(keyField: "company")
  title: String!
  blog: Blog @connection(name: "BlogPosts")
}

The generated Company.blogs.req.vtl uses blogCompanyId as the key expression instead of the specified company, so all queries return a DynamoDB invalid key error
#set( $query = {
  "expression": "#connectionAttribute = :connectionAttribute",
  "expressionNames": {
      "#connectionAttribute": "blogCompanyId"
  },
  "expressionValues": {
      ":connectionAttribute": {
          "S": "$context.source.id"
    }
  }
} )

Expected behavior
The connectionAttribute should use the value specified in the connection's keyField
#set( $query = {
  "expression": "#connectionAttribute = :connectionAttribute",
  "expressionNames": {
      "#connectionAttribute": "company"
  },
  "expressionValues": {
      ":connectionAttribute": {
          "S": "$context.source.id"
    }
  }
} )

amplify @ 1.7.0
node @ 10.15.3

@mdw2000 Thanks for the details. Can you try adding the keyField to the @connection on the Company.blogs field?
# ...
type Company @model {
  id: ID!
  name: String
  blogs: [Blog] @connection(name: "CompanyBlogs", keyField: "company")
}

That did the trick, thanks @mikeparisstuff!

To elaborate/clarify on this topic; a @connection keyField should exist on both sides of the named relationship.
A complete example:
type Tenant @model {
  id: ID!
  name: String
  companies: [Company] @connection(name: "Companies", keyField: "tenantId")
}

type Company @model {
  id: ID!
  name: String!
  tenant: Tenant! @connection(name: "Companies", keyField: "tenantId")
  tenantId: ID!
}```

I was trying to solve the problem where, when the `keyField` was not set one would automatically get created as `companyTenantId`.


Create directory if it doesn't exist

Create the output directory if it doesn't exist.

Fixed in <IP_ADDRESS>22.


fix: Upgrade SSL policy to restricted

What type of PR is this? (check all applicable)

fix

Description
Upgrades policy to use RESTRICTED which is compliant with PCI DSS and all modern browsers support it.

This PR is included in version 1.2.3 :tada:


Import Key function is throwing error "{"error":{"code":"BadParameter","message":"RSA key is not valid - cannot instantiate crypto service"}}"

Bug Report

import path of package - https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azkeys#Client.ImportKey
SDK version - latest

What happened?
- We followed below steps to use this ImportKey Function
- Generated rsakey ->ssh-keygen -t rsa -b 4096 -m PEM -f privatev2.key
- converted to .pem file -> openssl rsa -in privatev2.key -text > privatev2.pem
- Then converted to jet using the online link -> https://8gwifi.org/jwkconvertfunctions.jsp
- Then we copied the jwt objects into code and tried as shown below
- But when we run, we get "{"error":{"code":"BadParameter","message":"RSA key is not valid - cannot instantiate crypto service"}}"

What did you expect or want to happen?

Need to import the key to the keyvault

How can we reproduce it?

Please provide the exact go lang code sample to convert .pem to azkeys.JSONWebKey and import key using ImportKey Function.

Anything we should know about your environment. - NA

@lirenhe - Any update here?

@lirenhe - Can we have some updates here?

Hi @Roopan92. Because of the error message you received, the issue is most likely an error in the way you generated your RSA key. I would try a different and more direct means of generating your data.
I am also confused with the package name you used in line 481. I would expect it to be azkeys.ImportKey().

@Roopan92 how did you parse the PEM into a JWK? Also, you're passing base64 as a []byte slice, which is not valid. You need to base64 decode it into the original slice e.g., https://go.dev/play/p/Qr2FnDpV9gB

@gracewilcox  - You can ignore the name at 481. Thats something we internally handle it and at the end we are using azkeys.ImportKey only.
@heaths / @gracewilcox - If possible, can you please provide me an example go code for azkeys.Importkey function?. It would be very helpful

just like how we are uploading the .pem file into the keyvault

@Roopan92 I'm confused: how are you otherwise uploading the .pem file into Key Vault? Key Vault doesn't support importing keys as PEM-encoded data - only for certificates is that possible. Are you importing the PEM file as a secret? If so, keep in mind that the entirety of the PEM file - private key included - will be accessible to anyone with permissions to read secrets.

@heaths  - I need to implement the below functionality using Azure SDK - in the KEYS.

Hope my ask is straight forward.
In other ways, Give me an example for below function.

I see. That's a feature of the portal and not present in the Key Vault REST API. I imagine the portal is doing pretty much what we prescribed i.e., that you parse the PEM file into a JWK and import that. But in your example, you can't simply convert the base64 to a []byte slice. You have to decode the base64 to []byte using the encoding/base64 module or similar, or just parse the PEM like so: https://gist.github.com/heaths/d548f139404d438dc1369326a687e70e

Thanks @heaths. This is what we were expecting and there was no much details in the doc about this we got blocked, but now with your sample we are able to import the key into the key vault.
Thanks @gracewilcox for your support.
We can close this issue.

@gracewilcox did you want to repurpose this to consider a sample, or do you want to close it?

@gracewilcox did you want to repurpose this to consider a sample, or do you want to close it?

Yes! Going to add an example to the code base shortly, then will close the issue. Thanks! :)