runtime: avoid cleaning up extensions directory

When bundling gems via Git with native extensions with the --clean flag the extensions/ directory should be left around.
Lacks a test, I'll follow up with that.

(optional debugging fairy tale for the record)
It all started an innocent Friday morning with a production Rails console taking a long time to boot (~2m). I started investigating why. ps auxf quickly surfaced that something fishy was going on:
8192     12831  0.3  0.0<PHONE_NUMBER>6 pts/4    Ssl+ 15:48   0:00  \_ borginit:shopify/console/1
8192     12927  0.0  0.0   8316   668 pts/4    S+   15:48   0:00      \_ /sbin/ppidshim /bin/bash -c exec bin/rails console
8192     12929 48.0  0.3<PHONE_NUMBER>60 pts/4   Sl+  15:48   0:10          \_ ruby /app/bin/rails console
8192     16762  0.0  0.0   8568   816 pts/4    S+   15:49   0:00              \_ sh -c /usr/lib/shopify-ruby/2.1.6-shopify2/bin/ruby extconf.rb 2>&1
8192     16764  7.7  0.1<PHONE_NUMBER>36 pts/4   Sl+  15:49   0:00                  \_ /usr/lib/shopify-ruby/2.1.6-shopify2/bin/ruby extconf.rb
8192     23646  0.0  0.0   8568   820 pts/4    S+   15:49   0:00                      \_ sh -c make >/artifacts/ruby/2.1.0/bundler/gems/nokogiri-e2821bee7e21/ext/nokogiri/tmp/x86_64-unknown-linux-
8192     23647  0.0  0.0  10580  2492 pts/4    S+   15:49   0:00                          \_ make
8192     23652  0.0  0.0  10580  2776 pts/4    S+   15:49   0:00                              \_ make all-recursive
8192     23653  0.0  0.0  21696  3428 pts/4    S+   15:49   0:00                                  \_ /bin/bash -c fail=; \ if (target_option=k; case ${target_option-} in ?) ;; *) echo "am__make_ru
8192     23668  0.0  0.0  21700  2328 pts/4    S+   15:49   0:00                                      \_ /bin/bash -c fail=; \ if (target_option=k; case ${target_option-} in ?) ;; *) echo "am__mak
8192     23669  0.0  0.0  10580  2584 pts/4    S+   15:49   0:00                                          \_ make all-am
8192     24300  0.0  0.0  21660  3288 pts/4    S+   15:49   0:00                                              \_ /bin/bash -c echo "  CC      " tree.lo;/bin/bash ./libtool --silent --tag=CC   --mo
8192     24303  0.0  0.0  22420  4164 pts/4    S+   15:49   0:00                                                  \_ /bin/bash ./libtool --silent --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I
8192     24309  0.0  0.0  10852  2456 pts/4    S+   15:49   0:00                                                      \_ gcc -DHAVE_CONFIG_H -I. -I./include -I./include -D_REENTRANT -fPIC -pedanti
8192     24310  0.0  0.0  46200 16276 pts/4    R+   15:49   0:00                                                          \_ /usr/lib/gcc/x86_64-linux-gnu/4.8/cc1 -quiet -I . -I ./include -I ./inc

For some reason, nokogiri was compiling every time we booted. But only when booting the production console. rails runner, starting a Unicorn worker or firing up Resque all boot normally without doing any ridiculous compilation.
I gdb(1)'ed into the process and ran my dear colleague @csfrancis' MRI call stack script to obtain a Ruby backtrace to figure out where that compilation was happening, giving me:
/artifacts/ruby/2.1.0/gems/activesupport-4.2.4/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/ext/builder.rb:73:in `run'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/ext/ext_conf_builder.rb:38:in `block in build'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/tempfile.rb:324:in `open'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/ext/ext_conf_builder.rb:17:in `build'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/ext/builder.rb:161:in `block (2 levels) in build_extension'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/ext/builder.rb:160:in `block in build_extension'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/ext/builder.rb:159:in `build_extension'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/ext/builder.rb:198:in `block in build_extensions'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/ext/builder.rb:195:in `build_extensions'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/specification.rb:1436:in `block in build_extensions'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/user_interaction.rb:45:in `use_ui'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/specification.rb:1434:in `build_extensions'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/basic_specification.rb:56:in `contains_requirable_file?'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/specification.rb:915:in `block in find_by_path'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/specification.rb:883:in `block in each'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/forwardable.rb:183:in `each'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/specification.rb:882:in `each'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems/specification.rb:914:in `find_by_path'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/rubygems.rb:188:in `try_activate'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/irb/locale.rb:150:in `block in search_file'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/irb/locale.rb:158:in `block in each_localized_path'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/irb/locale.rb:167:in `each_sublocale'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/irb/locale.rb:157:in `each_localized_path'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/irb/locale.rb:145:in `search_file'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/irb/locale.rb:124:in `find'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/irb/locale.rb:108:in `load'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/irb/locale.rb:32:in `initialize'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/irb/init.rb:114:in `init_config'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/irb/init.rb:16:in `setup'
/usr/lib/shopify-ruby/2.1.6-shopify2/lib/ruby/2.1.0/irb.rb:380:in `start'
/artifacts/ruby/2.1.0/gems/railties-4.2.4/lib/rails/commands/console.rb:110:in `start'
/artifacts/ruby/2.1.0/gems/railties-4.2.4/lib/rails/commands/console.rb:9:in `start'
/artifacts/ruby/2.1.0/gems/railties-4.2.4/lib/rails/commands/commands_tasks.rb:68:in `console'
/artifacts/ruby/2.1.0/gems/railties-4.2.4/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
/artifacts/ruby/2.1.0/gems/railties-4.2.4/lib/rails/commands.rb:17:in `<top (required)>'
/artifacts/ruby/2.1.0/gems/bootscale-0.5.1/lib/bootscale/core_ext.rb:4:in `require'
bin/rails:5:in `<main>’

This confirms that it does indeed happen as part of starting the REPL, and thus is only a problem with the production console. This problem did not manifest in development either.
Looking into the irb source guided by the backtrace it's clear it attempts to pick up the file irb/encoding_aliases.rb. Eventually it makes it's way down to IRB::Locale#search_file to try and find irb/encoding_aliases.rb:
    def search_file(lib_paths, dir, file)
      each_localized_path(dir, file) do |lc_path|
        lib_paths.each do |libpath|
          full_path = File.join(libpath, lc_path)
          return full_path if File.readable?(full_path)
        end
        redo if defined?(Gem) and Gem.try_activate(lc_path)
      end
      nil
    end

The interesting code here is Gem.try_activate(..). If Rubygems is loaded, it'll attempt to call Gem#try_activate in an effort to expand the load path even further to try and find irb/encoding_aliases.rb. Eventually we end up in Gem::Specification#build_extensions as witnessed by the backtrace:
  def build_extensions # :nodoc:
    return if default_gem?
    return if extensions.empty?
    return if installed_by_version < Gem::Version.new('2.2.0.preview.2')
    return if File.exist? gem_build_complete_path
    return if !File.writable?(base_dir)
    return if !File.exist?(File.join(base_dir, 'extensions'))
    # code to build
  end

What seems to be happening is that nokogiri is not quitting this check, but simply proceeding down to build the extension once again.
I added the following line to get an overview of a) whether any other extensions were getting built and I just happened to only catch nokogiri, b) figure out if any of those values were wrong:
puts "name=#{name} installed_by_version=#{installed_by_version.inspect} gem_build_complete_path=#{gem_build_complete_path} build_complete=#{File.exist?(gem_build_complete_path)} whatever=#{ !File.writable?(base_dir) && !File.exist?(File.join(base_dir, 'extensions'))}"

There are three interesting cases here I wanted to inspect the output of:

Gem with rubygems as the source with extensions
Gem with git as the source with extensions

An example of 1) is Shopify/semian:
name=semian installed_by_version=#<Gem::Version "2.2.3"> gem_build_complete_path=/artifacts/ruby/2.1.0/extensions/x86_64-linux/2.1.0-static/semian-0.3.0/gem.build_complete build_complete=true whatever=false

This looks completely reasonable. For nokogiri as an example of 2):
name=nokogiri installed_by_version=#<Gem::Version "2.2.3"> gem_build_complete_path=/artifacts/ruby/2.1.0/bundler/gems/extensions/x86_64-linux/2.1.0-static/nokogiri-e2821bee7e21/gem.build_complete build_complete=false whatever=false

What was even more strange is that all other examples of 2) actually didn't compile. nokogiri is the only gem that actually compiles in the 2) group. The others look like this
name=json installed_by_version=#<Gem::Version "0"> gem_build_complete_path=/artifacts/ruby/2.1.0/bundler/gems/extensions/x86_64-linux/2.1.0-static/json-a41ce74ba367/gem.build_complete build_complete=false whatever=false

What's interesting about those two different examples of 2) is:

nokogiri has a proper installed_by_version, but json has it at the default of 0
Both json and nokogiri's gem.build_complete file doesn't exist
nokogiri compiles, but json does not

gem.build_complete is an empty file rubygems leaves around after compiling the extension.
The reason we're compiling nokogiri and not json is that json has installed_by_version at the default value of 0, which means the check above:
    return if installed_by_version < Gem::Version.new('2.2.0.preview.2')

Makes #build_extensions return early, avoiding the compilation. However, this is just a coincidence (and probably a bug in rubygems). The check that should make it terminate is:
    return if File.exist? gem_build_complete_path

So why is installed_by_version set for nokogiri and not json? Because our fork of nokogiri has installed_by_version set in its gemspec. It's the only example of a Gem with a git source that has this for Shopify:
s.installed_by_version = "2.2.2" if s.respond_to? :installed_by_version

(I haven't dug deeper into is why the input says 2.2.2 when the output above says 2.2.3)
So we now know why nokogiri is the only C-extension compiling: It doesn't return early on the version check. This starts to uncover the actual bug: gem_build_complete_path is not present for git-sourced gems and is not compiling by accident due to the version default.
Sure enough, running find vendor/bundler -name "*gem.build_complete" shows that gem.build_complete exists for all gems that are downloaded for rubygems. However, there are none for the gems with a Git source.
When running the same find(1) command in development "*gem.build_complete" are clearly present in vendor/bundler/bundler/extensions. This directory doesn't exist at all in production. However, when nokogiri compiles it creates the extensions/ directory with the gem.build_complete file in this directory as well as the shared object for the extension.
What's different between development and production? I suspected our Docker image building process
would be the cause, since it does a lot of liberal cleanups to save space:
prune_unwanted_files() {
  set +x
  # Sprockets cache; no value at runtime
  rm -rf ${artifacts_path}/cache/assets
  # Another cache; no value once all gems are installed
  rm -rf ${artifacts_path}/ruby/*/cache
  # git dirs of installed gems are useless once we no longer have to update them
  rm -rf ${artifacts_path}/ruby/*/bundler/gems/*/.git
  # final output is named rlibmemcached*; lib* are intermediate artifacts
  rm -rf ${artifacts_path}/ruby/*/bundler/gems/memcached-*/ext/lib*
  # Again, git dirs useless once we're done installing
  rm -rf ${artifacts_path}/ruby/*/gems/*/.git
  # Large source files, intermediate artifacts
  rm -rf ${artifacts_path}/ruby/*/bundler/gems/nokogiri-*/{ports,ext/nokogiri/tmp}
  # *.o, *.c, *.h, and *.a are intermediate object or source files not loadable
  # at runtime.
  find "${artifacts_path}/ruby" -name '*.{o,c,h,a}' -exec rm -f {} \;
  find "${artifacts_path}/ruby" -name 'gem_make.out' -exec rm -f {} \;
  set -x
}

Unfortunately bundler/extensions is not there. Something else must be removing the extensions/ directory and all its contents. I looked at our container build logs to try and see if anything suspicious was being run after bundle install:
Oct  2 15:15:22.014:	Bundle complete! 213 Gemfile dependencies, 292 gems now installed.
Oct  2 15:15:22.015:	Gems in the groups development, test and debug were not installed.
Oct  2 15:15:22.015:	Bundled gems are installed into /artifacts.
Oct  2 15:15:22.079:	Removing  (extensions)
Oct  2 15:15:22.083:	Removing billing (7570b0472fbe)

The second last line looked extremely suspicious. But where was it coming from? The removal goes back to the --clean flag to bundler(1) which ends up calling Bundler::Runtime#clean. This method is responsible for cleaning up unused gems. However, it doesn't consider the extensions/ directory inside the directory that stores all the gems with a Git source:
stale_git_dirs  = git_dirs - spec_git_paths

extensions/ would be part of git_dirs, but not spec_git_paths and thus would end up being part of the resulting array for stale_git_dirs.
If the extensions/ directory is left around, Rubygems would be able to pick up the gem.build_complete file and not compile the extension for the right reasons which resolves my bug.
Some things to consider at the tail of this:

Why are versions not set correctly for Git sourced gems in Rubygems?
Why was the version set to 2.2.3 on nokogiri even though the spec says 2.2.2?
Why is irb calling Gem.try_activate on every gem? Removing that would break backwards compatibility unfortunately.

@ibawt @csfrancis @camilo @byroot @arthurnn @sgrif @EiNSTeiN- @burke
@hone

:clap: This also explain why bundler keep recompiling native gems. You just cut our build time in half @Sirupsen

Woah @andremedeiros, this should not have been merged without a test

I've tested it myself. @sirupsen will push another PR with tests.

Still, changes like this should be tested before they hit master, and merged via @homu

@segiddins agree, this wasn't critical to get merged.

@Sirupsen great catch, also awesome debugging story! I'm very excited to learn why git extensions always recompile, since I've wondered about it for a long time.
@andremedeiros Thanks for reviewing this! In the future, write a comment that includes @homu r+ after you approve. Then the bot will take care of creating the merge, running the tests on the merge, and update master if the merged commit passes.
@segiddins it's totally my fault for not making sure @andremedeiros knew about @homu, sorry. :/
I don't think we've ever officially stated this before, but as part of the workflow improvements that I've been working on (like homu), from now on I'd like to only accept PRs that include tests that are red before the PR and green after. Bundler is so complex nowadays that regressions seem to happen every time something we care about doesn't have a test. :stuck_out_tongue_closed_eyes:
Thanks for the good work, everyone! :trophy:


AbstractRandomizerRegistry should be public

AbstractRandomizerRegistry has a lot of nice methods for interacting with field definitions. Since the class holds no state, it would be really nice to offer it as an extendable class to consumers.

Fixed in 432475c1ca8b687017b84f573187109450f9d599 and will be part of v3.8.0 release.


Update deps

Finch now allows the next version of nimble_options.
Updating this dep lets OpenTelemetry Phoenix use a different version of nimble_options

Bump @wojtekmach . What else can I do to help?

Thank you!

Updating this dep lets OpenTelemetry Phoenix use a different version of nimble_options

mix.lock is local to req, it is not taken into consideration when req is used as a dependency. But your change is useful to req developers for sure. Thanks!


Support only set prof.active

What is changed and how it works?

Issue Number: Close #xxx

What's Changed:

Related changes

[ ] PR to update pingcap/docs/pingcap/docs-cn:
[ ] Need to cherry-pick to the release branch

Check List
Tests

[ ] Unit test
[ ] Integration test
[ ] Manual test (add detailed scripts or steps below)
[ ] No code

Side effects

[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Breaking backward compatibility

Release note

[calvin@10-2-12-79 ~]$ curl "http://<IP_ADDRESS>:5765/debug/pprof/heap_activate"
set prof.active = true[calvin@10-2-12-79 ~]$
[calvin@10-2-12-79 ~]$
[calvin@10-2-12-79 ~]$ curl "http://<IP_ADDRESS>:5765/debug/pprof/heap_activate?period=1"
activate heap profile success[calvin@10-2-12-79 ~]$
[calvin@10-2-12-79 ~]$ curl "http://<IP_ADDRESS>:5765/debug/pprof/heap_list"
[calvin@10-2-12-79 ~]$ curl "http://<IP_ADDRESS>:5765/debug/pprof/heap_list"
/DATA/disk4/luorongzhen/tidb-data/tiflash-5761/flash/heap-F9tLc3/000001.heap

Introduced a period argument, which means periodically dump heap file.

If call http://<IP_ADDRESS>:5765/debug/pprof/heap_activate?period=1, prof.activate would be set to true, and a dedicated thread will dump heap file at a given interval. Which is the previous path

If call http://<IP_ADDRESS>:5765/debug/pprof/heap_activate?period=0, or http://<IP_ADDRESS>:5765/debug/pprof/heap_activate would be set to true.

If call http://<IP_ADDRESS>:5765/debug/pprof/heap_deactivate?period=1, we will only stop the periodical dump thread. However, prof.activate would be set to true, if there is a previous profiling. Or remain false, if there is no profiling.

If call http://<IP_ADDRESS>:5765/debug/pprof/heap_deactivate?period=0 or http://<IP_ADDRESS>:5765/debug/pprof/heap_deactivate,  prof.activate would be set to false. And the periodical dump thread will be stopped, if any.

set prof.active succeed[calvin@10-2-12-79 ~]$ curl "http://<IP_ADDRESS>:5765/debug/pprof/set_prof_active"
set prof.active succeedcurl "http://<IP_ADDRESS>:5765/debug/pprof/heap_activate"
activate heap profile scurl "http://<IP_ADDRESS>:5765/debug/pprof/heap_deactivate"
deactivate heap profiletail /data4/luorongzhen/tidb-deploy/tiflash-5761/log/tiflash_tikv.log
[2024/08/28 15:44:38.016 +08:00] [INFO] [raft_client.rs:830] ["resolve store address ok"] [addr=<IP_ADDRESS>:5721] [store_id=1] [thread_id=398]
[2024/08/28 15:44:38.016 +08:00] [INFO] [raft_client.rs:699] ["server: new connection with tikv endpoint"] [store_id=1] [addr=<IP_ADDRESS>:5721] [thread_id=398]
[2024/08/28 15:44:38.023 +08:00] [INFO] [raft_client.rs:873] ["connection established"] [try_count=1] [msg_count=2] [cost=17.76472ms] [addr=<IP_ADDRESS>:5721] [store_id=1] [thread_id=398]
[2024/08/28 15:44:38.989 +08:00] [INFO] [kv.rs:794] ["batch_raft RPC is called, new gRPC stream established"] [source_store_id=Some(1)] [thread_id=7]
[2024/08/28 15:44:38.991 +08:00] [INFO] [raft.rs:1386] ["received a message with higher term from 10006"] ["msg type"=MsgAppend] [message_term=25] [term=24] [from=10006] [raft_id=10007] [region_id=10005] [thread_id=442]
[2024/08/28 15:44:38.991 +08:00] [INFO] [raft.rs:1151] ["became follower at term 25"] [term=25] [raft_id=10007] [region_id=10005] [thread_id=442]
[2024/08/28 15:44:38.991 +08:00] [INFO] [region.rs:125] [" 122:10005 0, peer changed"] [is_replicated=false] [role=Follower] [leader_id=10006] [region_id=10005] [thread_id=442]
[2024/08/28 15:44:46.913 +08:00] [INFO] [client.rs:799] ["set cluster version to 8.4.0-alpha"] [thread_id=102]
[2024/08/28 15:45:53.543 +08:00] [INFO] [profile.rs:171] ["periodical heap profiling is started"] [has_active=true] [thread_id=669]
[2024/08/28 15:45:57.288 +08:00] [INFO] [profile.rs:189] ["periodical heap profiling is canceled"] [active=true] [thread_id=669]
[calvin@10-2-12-79 ~]$

[calvin@10-2-12-79 ~]$ curl "http://<IP_ADDRESS>:5765/debug/pprof/set_prof_inactive"
set prof.active succeedcurl "http://<IP_ADDRESS>:5765/debug/pprof/heap_activate"
activate heap profile scurl "http://<IP_ADDRESS>:5765/debug/pprof/heap_deactivate"_prof_inactive"
deactivate heap profiletail /data4/luorongzhen/tidb-deploy/tiflash-5761/log/tiflash_tikv.log
[2024/08/28 15:44:38.989 +08:00] [INFO] [kv.rs:794] ["batch_raft RPC is called, new gRPC stream established"] [source_store_id=Some(1)] [thread_id=7]
[2024/08/28 15:44:38.991 +08:00] [INFO] [raft.rs:1386] ["received a message with higher term from 10006"] ["msg type"=MsgAppend] [message_term=25] [term=24] [from=10006] [raft_id=10007] [region_id=10005] [thread_id=442]
[2024/08/28 15:44:38.991 +08:00] [INFO] [raft.rs:1151] ["became follower at term 25"] [term=25] [raft_id=10007] [region_id=10005] [thread_id=442]
[2024/08/28 15:44:38.991 +08:00] [INFO] [region.rs:125] [" 122:10005 0, peer changed"] [is_replicated=false] [role=Follower] [leader_id=10006] [region_id=10005] [thread_id=442]
[2024/08/28 15:44:46.913 +08:00] [INFO] [client.rs:799] ["set cluster version to 8.4.0-alpha"] [thread_id=102]
[2024/08/28 15:45:53.543 +08:00] [INFO] [profile.rs:171] ["periodical heap profiling is started"] [has_active=true] [thread_id=669]
[2024/08/28 15:45:57.288 +08:00] [INFO] [profile.rs:189] ["periodical heap profiling is canceled"] [active=true] [thread_id=669]
[2024/08/28 15:46:33.355 +08:00] [INFO] [profile.rs:171] ["periodical heap profiling is started"] [has_active=false] [thread_id=669]
[2024/08/28 15:46:35.881 +08:00] [INFO] [profile.rs:189] ["periodical heap profiling is canceled"] [active=true] [thread_id=669]
[2024/08/28 15:46:35.881 +08:00] [INFO] [profile.rs:178] ["disable prof.active after periodical heap profiling"] [thread_id=669]


Dino ServerDisplay view should only display service lists suitable for the server type

Fixed by #50

Fixed by #50


2.4.0 Loginfile Update Process - Unreliable gamebuild and loginfile versioncheck

Currently Launchbuddy is not performing game updates.  I have to launch the game normally and update it, then start launchbuddy.  This then requires me to repair at least 1 loginfile before use.
Anyone else having this issue?

Im haivng a similar issue , im waiting for a response for my post too

I've had a lot of issues with new releases lately, what I've done is just reverted back to v2.2.0. It works without any issues.

I've had a lot of issues with new releases lately, what I've done is just reverted back to v2.2.0. It works without any issues.

Same issue, hasnt updated the last few updates, I choose update on the popup and it just closes, same if I manually choose to update.

does anyone know how to manually create a login file

have to confirm more issues with 2.3.9:
I don't the need for ...

Increased loginfile swap timeout to 20 secs
..., because shouldn't LB find running GW2 (update) processes even each second and wait another second?

However, I begin everytime with a "GW2-64.exe -image" to update GW2 itself 1st.
The I launch LB.
It starts updating loginfile of my 1st account and does it successfully, but the LB popups show me, there is someting wrong with timing.
When it then tries to update loginfile of my 2nd account, it doesn't do it and a popup appears and warns me, that it didn't update and if I proceed, this could be erroneous.
So I pressed no and and the same popup appears again -> endless loop.
So I pressed yes and it skips this account, but says, that this loginfile is not good but also does not update it at later LB starts.
Then I have to "Repair loginfile" manually in LB.
This happens also with my next 2 accounts.
In summary:
loginfile of 1st account : OK
loginfile of 2nd account : FAIL
loginfile of 3rd account : OK
loginfile of 4th account : FAIL

Still an issue.

Still an issue, can confirm what Scrambled says about every other account.
1st Good
2nd Fails, needs repair
3rd Good
4th Fails, needs repair
5th Good
6th Fails, needs repair
7th Good
8th Fails, needs repair

For me it hasn't been consistent with "every other". I have 4 accounts and on average only 1 has required repair, athough lately it has been 2.

I've had a lot of issues with new releases lately, what I've done is just reverted back to v2.2.0. It works without any issues.

does it work with the current XML file? Or do I need to re-create all accounts again?

Still an issue, can confirm what Scrambled says about every other account.
Seems this is the case for me with the last couple updates.  It wasn't always so, but seems it is now.

Fix should be live with 2.4.0.
https://github.com/TheCheatsrichter/Gw2_Launchbuddy/commit/e152d3cced27dfb39cf9c343b5b84a4e7f42f1ac
Please report back if you still have persistent errors.

Fix should be live with 2.4.0.
e152d3c
Please report back if you still have persistent errors.

Still requires me to update the game outside of LB manually before starting LB.  LaunchBuddy only updates loginfiles.

The remaining loginfiles get correctly? (not only every second one)
My guess is that even after a faulty gameclient update the gw2 client seems to update the build version in its xml file.
LB reads that file to gather information and therefore seems to think that your game is allready updated correctly.
To shine some light in the process LB works as follows:

Load the gameclient xml file and get the current build version
Check the official gw2 API for the newest build version
If API version > local version --> update gameclient
3.1 Run the gameclient with "-image" to update the gameclient and automatically close it when its done
3.2 Reimport the game xml file and compare it to the previous version which was gathered from the xml file
Compare each account's loginfile build version to the current gameclient version --> update if version is lower

I will further investigate the gw2 gameclient update process and when it writes to its xml file. Maybe some additional indicator for gameclient versionchecking is needed. However I am currently not sure how to achieve this without reading the gameclients memory.

Just as i finished writing this comment I found something interesting:
At the moment something seems to be off with the gw2 API:

Current newest gameclient version : 117585
Current API build version: 115267

The API currently is not up to date, leads to the following problem:
Lets say your gameclient is on 116000.
You start Launchbuddy it checks the API which returns 115267 --> 116000 > 115267 --> true;  which means Launchbuddy thinks the game to be up to date!
If you now manually update the game you will update to 117585.
Rerunning Launchbuddy now also results in no gameupdate, however it will work as intended.
Currently my only option would be to force a gameclient update on every startup which is a sketchy fix...
Hopefully Anet fixes this API bug soon. :(

The issue with the API was already reported at https://github.com/TheCheatsrichter/Gw2_Launchbuddy/issues/237#issuecomment-849933640
Nicodemus also stated, that he wrote a post to Arenanet in https://github.com/TheCheatsrichter/Gw2_Launchbuddy/issues/237#issuecomment-849938519 , which is posted at https://en-forum.guildwars2.com/topic/96243-gw2-client-build-number-stuck-at-115267/ , but still unanswered and still not fixed by Arenanet.
So, please don't rely on API build version anymore!
--
I also realized, that the timing at updating the Accounts files works at my friend's machine, who has a M.2 disk (very fast).
I have a good old HDD disk for some tools like GW2LB and other stuff and only the Game files on my SDD (faster than HDD, but slower than M.2).
I think, that is the reason with such bad sleep timeouts waiting for updated login files at GW2LB update process.
I recommend at least to check, if login files were changed by the update process within last 5 minutes or so.
If not, don't change the version number in Accs.xml
Maybe also try another update run for that account, if user presses "OK" button or skip over if user presses "Skip" button.

I am currently creating some concepts to solve this nasty API bug. The quick fix will be the forced update which I likely will push with 2.4.1. However as I said this solution is time consuming and not really a clean way to handle this, at least in my opinion.
Currently the loginfile build number is only updated when:

the update actually was successfull and something changed in the loginfiles data.
the users manually accepts that the loginfile is valid anyway.

public bool ValidateUpdate(string OldHash)
        {
            Console.WriteLine($"OldHash= {OldHash} NewHash= {MD5HASH}");
            //Should be called after every Update
            if (OldHash == MD5HASH)
            {
                DialogResult dialogResult = MessageBox.Show(($"The loginfile for {AccountManager.GetAccountByID(int.Parse(this.Name)).Nickname} did not change when updating from {Gw2Build} to {EnviromentManager.GwClientVersion}. Proceed anyway?"), "Loginfile Update", MessageBoxButtons.YesNo);
                if (dialogResult == DialogResult.No)
                {
                    Valid = false;
                    return false;
                }
            }
            gw2build = EnviromentManager.GwClientVersion;
            Valid = File.Exists(EnviromentManager.LBLocaldatsPath+Name+".dat");
            return Valid;
        }

Force gameclient update is now live with https://github.com/TheCheatsrichter/Gw2_Launchbuddy/releases/tag/2.4.1.
This option can be changed in the LB settings tab if needed.
A third option for the loginfile validation process has currently not been added, as the case: loginfile update failed and you want to retry the same file can be achieved by manual validation and pressing loginfile repairing in the acc settings tab.

So far working well.  Thanks for the hard work.

Updates worked for me also today =)
Thanks for your work.

https://github.com/TheCheatsrichter/Gw2_Launchbuddy/commit/c422d5b94c27d93ecb7760ca791547db1c6a0096
Released with 2.4.2

@TheCheatsrichter : thanks for updating.
But I would not hardcode the old build version from API site in a check:
if(ClientBuild == "115267") { .. }
This API build version can change at every time and can get old again at every time later on.
This could also happen to the build version from the asset address.
I would read all build versions from all known sites (API, Asset, maybe more sometimes) at any time.
Then make the decision, which version should be returned:

If no build version could be read, set forcegameclientupdate = true and whatever else is needed to force a game update.
If 1 or more build versions could be read, return the highest build number and do the usual tests, if a game and / or login file update is needed.

Greetz.

@ScrambledBrain
True, true, I was busy fixing the versionswitcher which broke a few days ago and totally forgot to implement the cleaner build version check.
https://github.com/TheCheatsrichter/Gw2_Launchbuddy/commit/20247ca4248b744ed2dab54c82d87d70e3111ef8
https://github.com/TheCheatsrichter/Gw2_Launchbuddy/commit/3c99717ea35a80b6cf51f3a7d6ea1f29b4e956c3
These changes already released with 2.4.2 as its release has been delayed to include the versionsiwtcher fix.

No new reports since the fix.
Closing.


Ember warning after message removed from queue

Error in console:
WARNING: Message not found in message queue: undefined

Appears everytime a message is removed from the queue.
Considered a non-blocker as this only affects the dev environment - Em.warn does not output in production environments.

Fixed by #8


Add Smarter Backoffs for MPI ring connection

Hey, I am running many LightGBM training runs as part of LightGBM on Spark's CI process and i notice that it does alot of waiting because of bad network and this dramatically slows the tests:
https://github.com/microsoft/LightGBM/blob/0a4a7a86f5a1d3146c36c7d8c082154a193d4893/src/network/linkers_socket.cpp#L190
Could you potentially make it first wait for half a second, then wait for 10 seconds? Or make this configurable. Thanks for the help!

This could be configurable.
I will create a PR for it.


Allow for passing a custom http.Transport for the backend to use.

Great project here, would love to be able to pass a custom http.Transport for the backend's http.Client to use.
Why?:

Proxy support
Custom TLS config
etc

Will be submitting a PR shortly

added in #2


axios.create

Do you mind to release a new version on npm to get the "axios.create" ?
Thanks.

I'm working on fixing an issue first. Will release once that's done.


Position issues

Describe the bug
The picker works great when the click that triggers the open is near the middle or top of the screen, but if it happens near the bottom, the end of the picker is cut off due to the available screen space not being enough. I read the docs here and this is supposed to be true by default, however it's still cutting off the bottom of the picker.
Expected behavior
I would expect that when the picker is too close to the bottom of the viewport, it would move up. I have inline and inline-with-input active in my configuration. Please let me know if I can keep these while also having the picker place itself intelligently when needed.
Screenshots
The screenshot isn't super helpful because I cropped out the grid behind it (proprietary). But the screen / viewport ends where it cuts off.

Desktop & mobile (please complete the following information):

Chrome
"@vuepic/vue-datepicker": "^3.3.1"

Please update to a newer version, as this is fixed in the meantime. Playground


sokol_app.h: trigger a linker error if users forget -pthread

My suggestion to resolve #455

Looks like  good solution, thanks!

Looks like  good solution, thanks!

There's an error when compiled as C++:
/usr/include/pthread.h:470:12: error: conflicting declaration of ‘int pthread_yield()’ with ‘C’ linkage
 extern int pthread_yield (void) __THROW;
            ^~~~~~~~~~~~~

I'll include pthread.h instead and removed the manual declaration.

There's an error when compiled as C++:
/usr/include/pthread.h:470:12: error: conflicting declaration of ‘int pthread_yield()’ with ‘C’ linkage
 extern int pthread_yield (void) __THROW;
            ^~~~~~~~~~~~~

I'll include pthread.h instead and removed the manual declaration.

hmmmm it's not that simple... pthread.h doesn't contain pthread_yield() by default, instead _GNU_SOURCE must be defined before, but if there's already another include of pthread.h the _GNU_SOURCE define doesn't work. What a mess :/
My next attempt was calling pthread_self(), but for some reason this seems to be optimized away...

hmmmm it's not that simple... pthread.h doesn't contain pthread_yield() by default, instead _GNU_SOURCE must be defined before, but if there's already another include of pthread.h the _GNU_SOURCE define doesn't work. What a mess :/
My next attempt was calling pthread_self(), but for some reason this seems to be optimized away...

pthread_self (funny enough) doesn't actually require pthread to be linked. It's in libc.
And yeah, _GNU_SOURCE was the reason I just forward declared pthread_yield instead of including pthread.
As for C++, that's unpleasant. How about we keep the __THROW for now and in a couple of hours I'll find some time to submit a better solution in a PR?

pthread_self (funny enough) doesn't actually require pthread to be linked. It's in libc.
And yeah, _GNU_SOURCE was the reason I just forward declared pthread_yield instead of including pthread.
As for C++, that's unpleasant. How about we keep the __THROW for now and in a couple of hours I'll find some time to submit a better solution in a PR?

Er... by the way won't adding "C" fix things properly? C functions should be nothrow
extern "C" int pthread_yield();

Er... by the way won't adding "C" fix things properly? C functions should be nothrow
extern "C" int pthread_yield();

No worries, I found an okay-ish solution: pthread_attr_init/destroy works. It shows up in libsokol.a as external undefined symbol (unlike pthread_self()).

No worries, I found an okay-ish solution: pthread_attr_init/destroy works. It shows up in libsokol.a as external undefined symbol (unlike pthread_self()).

won't adding "C" fix things properly

Not sure, there's a lot of C++ specific stuff in /usr/include/pthread.h, and almost all functions are marked as __THROW.

won't adding "C" fix things properly

Not sure, there's a lot of C++ specific stuff in /usr/include/pthread.h, and almost all functions are marked as __THROW.

Ok, see: https://github.com/floooh/sokol/commit/5fbb6a25501e5478674ca0600a7539033592d749

Ok, see: https://github.com/floooh/sokol/commit/5fbb6a25501e5478674ca0600a7539033592d749


[Bug] Strange TS issues for imports coming from @storybook/addon-svelte-csf

Describe the bug
I am encountering TS issues with all imports from @storybook/addon-svelte-csf:
Argument of type 'typeof Meta' is not assignable to parameter of type 'ConstructorOfATypedSvelteComponent'.
  Type 'typeof Meta' provides no match for the signature 'new (args: { target: any; props?: any; }): ATypedSvelteComponent'.

Possible causes:
- You use the instance type of a component where you should use the constructor type
- Type definitions are missing for this Svelte Component. If you are using Svelte 3.31+, use SvelteComponentTyped to add a definition:
  import type { SvelteComponentTyped } from "svelte";
  class ComponentName extends SvelteComponentTyped<{propertyName: string;}> {}

The exact same message with a different typeof is repeated for Template and Story imports.
Expected behavior
No TS issues
Environment

OS: Ubuntu 23.04
Node.js version: v18.17.1
NPM version: pnpm 8.6.10
Browser (if applicable): Brave
Browser version (if applicable): Version 1.56.20 Chromium: 115.0.5790.171 (Official Build) (64-bit)
Device (if applicable): Desktop

Additional context
Story looks like this:
<script lang="ts">
  import { Meta, Template, Story } from '@storybook/addon-svelte-csf'
  import Space from '$lib/components/ui/Space.svelte'
</script>

<Meta title="Actions/Space" component={Space} />

<Template let:args>
  <Space {...args} />
</Template>

<Story name="Horizontal" />

<Story name="Vertical" />

Config is as follows:
import type { StorybookConfig } from '@storybook/sveltekit'

const config: StorybookConfig = {
  stories: [
    {
      directory: '../src/stories/**',
      files: '*.stories.@(ts|svelte)',
      titlePrefix: 'Design System',
    },
  ],
  addons: [
    '@storybook/addon-interactions',
    '@storybook/addon-links',
    '@storybook/addon-a11y',
    '@storybook/addon-svelte-csf',
    {
      name: '@storybook/addon-essentials',
      options: {
        backgrounds: true,
      },
    },
    {
      name: '@storybook/addon-styling',
      options: {},
    },
  ],
  framework: {
    name: '@storybook/sveltekit',
    options: {},
  },
  docs: {
    autodocs: false,
  },
  core: { disableTelemetry: true },
}

export default config

Are you using Svelte 3? I'm using Svelte 3 in my SvelteKit project and also encountered the same type errors.
This might be an issue with new types in Svelte 4. I created a new SvelteKit project with Svelte 4 and no errors showed up.

I already feared that this would be the case, thanks for confirming it!

Fixed by upgrading to Svelte 4.
This dependency needs to drop Svelte 3 from it's peer dependencies here since it's misleading: https://github.com/storybookjs/addon-svelte-csf/blob/main/package.json#L98

@hobbes7878 seems like Svelte 4 did break our typing system after all - I'm not sure there's an easy fix here.
My suggestion would be to release a new major which only allows Svelte 4 as a peer dependency.
Alongside that we could also release a new patch that that only allows Svelte 3 as a peer dependency, so we'd release two new versions that clearly defines the necessary dependencies.
Thoughts?


Improve application architecture to multi-tier

Right now the project is a monolith Django. Divide it into font end, back end.

Decided the issue would not be beneficial


feat(minifier) implmement define for member expressions

adds the ability to transform
process.env.NODE_ENV
into
"development"
when that config is passed in

#3959  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @camc314 and the rest of your teammates on  Graphite

Could you please add some test cases for this PR?

there should be some in here?
https://github.com/oxc-project/oxc/blob/1e77217321a64508774bac76faa904157aa6c6e8/crates/oxc_minifier/tests/oxc/replace_global_defines.rs#L26-L40
or do you mean more integration-esque? if so where do those live?

Could you please add some test cases for this PR?

there should be some in here?
https://github.com/oxc-project/oxc/blob/1e77217321a64508774bac76faa904157aa6c6e8/crates/oxc_minifier/tests/oxc/replace_global_defines.rs#L26-L40
or do you mean more integration-esque? if so where do those live?

Forgive me, I didn't see those! That's perfect, thanks.

I'll revisit this PR with the one in Rolldown together this week.

I'll revisit this PR with the one in Rolldown together this week.

sounds good. feel free to close if you don't think this is the right approach


502 Bad Gateway on fresh install - nginx & node

Hey - I set up a new Dokku instance using the Digital Ocean 1-click installer. My domain is added, SSH keys set up, and I can push apps.
I've got a very minimal app made with Node + Express that works fine on Heroku, but is giving me 502 Bad Gateway after deploying with Dokku.

there are no errors on deploying.
the app isn't crashed when I run docker ps
running docker logs CONTAINER_ID doesn't output anything
running dokku logs appname doesn't output anything either
cat ../var/log/nginx/error.log gives me the only discernable error messages - a bunch of entries like this:

2014/03/20 14:54:28 [error] 5128#0: *14 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: <IP_ADDRESS>, server: donethis.flatwhite.co, request: "GET / HTTP/1.1", upstream: "http://<IP_ADDRESS>:49154/", host: "donethis.flatwhite.co"

Any idea?

I'm having the very same problem with 3 apps (but not a 4th in the same machine). They've worked for various months now, and since today they started to return this error. After a simple ps:restart they work again, but only for some minutes, then they start to fail again.

@fiatjaf please open another issue instead of commenting on an 8 month old one. Thank you :)


Metrics problem while evaluating a list of lists

While trying the FAQ example with a different dataset, I found a problem with metrics. It prints 0.0 always.
What I did:

Added a new reader that supports a test set for question/answer
Added the evaluation target and metrics accuracy.

I see that is implemented as follows. The problem is that y_predicted is a list of lists and y _true is a list of strings. So, there is never a match.
I made a quick fix for this but I was wondering if this is a bug, or I am using some component incorrectly.
For example, lists are as follows:
y_true = ['answer 1', 'answer 2']
y_predicted = [['answer 1'], ['answer 2']]
    examples_len = len(y_true)
    correct = sum([y1 == y2 for y1, y2 in zip(y_true, y_predicted)])
    return correct / examples_len if examples_len else 0

This is the config:
    "metrics": [
      {
        "name": "accuracy",
        "inputs": ["y", "answer"]
      }
    ]

Hey @cuent , could you paste the entire config here.
make sure that the out value of chainer fits your y_true. Try leaving the metrics as  "metrics": ["accuracy"].


Stop restarting on crashing code

I have been using Supervisor for development, and it irks me that if I make a syntax error or something else that will make Node crash immediately when launched, Supervisor will just continue to restart the process, several times a second.
Wouldn't it be a safe assumption that a process that dies within its first second of runtime should not be restarted?

I agree, the instant infinite crash loop is a bit annoying. However, I always found it equally annoying when it bailed and I had to tab back into that terminal to restart supervisor just because I made a syntax error.
I've been thinking the best solution might be an exponential backoff, where the first crash within a given time period gets no delay, but subsequent crashes get longer and longer delays, up to 5s or something reasonable like that.


Added Cog(map) and its own evac shuttle

About the PR

Added Cog, a new map based on a mix of goonstation's cogmap and Citadel's cogmap and my own ideas. I also added a evac shuttle (emergency_wode.yml) to fit Cog's rather wide evac (Wode is a placeholder name that can be changed later)
new features (from goon's):

new evac shuttle
a reporter's office (with reporter job ofc)
a remade, larger brig
warden's office
a small medical room in brig
quartermaster's office
a shuttle construction bay
virology
interesting maints rooms
singo containment in the middle of engineering
routing room has be turned into teg
owl room has been turned into bee room
grav gen in (former) shuttle bay
tons of conveyer belts in disposals (these don't lead to routing systems)
disposal pipes leading from the kitchen to sec (to feed prisoners, it is locked behind a windoor to prevent chef from breaking into sec)
disposal pipes from botany to the freezer
disposal pipes from kitchen to the freezer
the animal ranch is now under service doors, allowing more service roles to work with the animals
clown and mime now have a nice room
gambling table
cargo dock has been pushed outward to allow cargo to dock
toxins lab has been turned into artifact lab
sec has their own evac pod (for prisoners mostly)
moved robotics to sci, turned chem to med

--OTHER THINGS TO NOTE--

no library or librarian job
the routing system has mostly been remove due to the fact we do not have mass launchers
the routers on the far edges of the station have been removed because they would block shuttle access for cargo, salvage, and other shuttles trying to fly to the station
the doors in security that look like normal glass doors have had their security access removed to allow people to emag to the brig without needing to recharge
With the solar panels, Singo, AME, and TEG, the power is superfluous
no lawyer room

Please give feedback, additionally I may have to add Cog to some files to get it in the map rotation but im not sure which ones those are, please tell me
Media

Cog

Wode evac shuttle

Requirements

[X] I have read and I am following the Pull Request Guidelines. I understand that not doing so may get my pr closed at maintainer’s discretion
[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

Changelog

:cl:

add: Added Cog, a new midpop map based on Cogmap from ss13
add: Added a evac shuttle for Cog

Coggers

that was quick

okay unironically triple layer armory and/or airlock it
remove the pipes in the teg room
add mapped trans pride cloak :godo:
and make all the beds in medical med beds and you have a good map

I really like the main design, no issues from what I see there, but the decoration and aesthetics in the lengthy hallways both in and out of departments are lacking

I should add: some areas feel way larger and spacious than they should be, not that they necessarily need to be shrunk but that they could have more physical objects, deco or props to make it feel and look better

These are my only notable nitpicks, They just feel rather empty, either due to their size or lack of deco

that evac shuttle looks like a real easy target for mass murder via bombing

This atmos looks packed Atmos but

Holy shit this hit me with memories...

4 SecTechs, 2 only blocked by one door of security? I uh... think the 2 at the front of sec seem excessive and insecure.
Maybe throw some vending machines, lockers, and chairs up here, seems like there is a good space for it too

Logger pro cartridge?

Maybe you should just turn this into a normal front desk? I see you kinda converted the Telesci room but this just looks silly.

There needs to be some magic mirrors somewhere, maybe in the barbershop?

This area looks horrid to work in to be honest, and I'm not sure if you're going to be able to get the ship parts out of there without having to deal with them bumping into doors and one-tile-wide spaces. Maybe merge the 2 rooms, remove the ship part stuff, move the gas tank inside, and flaten that wall. Knock the northern wall up for the storage room into space a little more so you don't bump into the PA every time you enter that room.

The Rad shields on the engi windows are pointless with plasma glass, it blocks all rads anyways. Maybe swap it for normal glass or remove the rad shutters(I would say make them normal glass for consistency with fland).

What I noticed is that there is no meteor protection anywhere except the telecoms satellite.

ngl 2X3 burn chamber ooof requesting 3X3 plzzzz

I will make changes later today

I dont know if the bot shows images for updated maps so ill get one

i will review tonight or tomorrow

posting image in comments as well as pr

btw, there is no need to add cl about adding evac shuttle, just simple
- add: Added Cog, a new midpop map based on Cogmap from ss13
would be enough

another day another image

im going to just post the updated images in the pr description

no misc printers/mechcomp replacement devices, time to die due to nukie armory

This mapping pr contains code changes, which prevents the map server from adding the map to the viewer


Fix dataframe module circular import problem

Adjust dataframe module to fix circular import problem

Codecov Report

Merging #73 (a38e3c0) into main (662efcf) will increase coverage by 5.94%.
The diff coverage is 83.33%.

@@            Coverage Diff             @@
##             main      #73      +/-   ##
==========================================
+ Coverage   53.71%   59.66%   +5.94%
==========================================
  Files          39       40       +1
  Lines         780      781       +1
  Branches       38       39       +1
==========================================
+ Hits          419      466      +47
+ Misses        350      303      -47
- Partials       11       12       +1

Impacted Files
Coverage Δ

towhee/dataframe/variable.py
50.00% <50.00%> (ø)

towhee/dataframe/_iterator.py
58.97% <77.77%> (+58.97%)
:arrow_up:

towhee/dataframe/__init__.py
100.00% <100.00%> (+100.00%)
:arrow_up:

towhee/dataframe/dataframe.py
73.33% <100.00%> (+73.33%)
:arrow_up:

towhee/tests/dataframe/test_dataframe.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 662efcf...a38e3c0. Read the comment docs.

/approve
/lgtm


Update user settings without oauth

[CI] Staging environment is at http://cap-staging-docs.codefresh.io.s3-website.us-east-1.amazonaws.com/csdp-user-settings-noauth/docs/

[CI] Staging environment is at http://cap-staging-docs.codefresh.io.s3-website.us-east-1.amazonaws.com/csdp-user-settings-noauth/docs/

[CI] Staging environment is at http://cap-staging-docs.codefresh.io.s3-website.us-east-1.amazonaws.com/csdp-user-settings-noauth/docs/


Add more quality documentation to drive adoption

Improve the quality of documentation with more coverage, better navigability and search
Add more hands on tutorials for both document store and event store
Add more real world use cases, usage and implementation using Marten.

I'm calling this one mission accomplished for now.


Sort out Github permissions

Project setup:

[x] Add Contributers to github repo
[ ] Bring in draft3d branch as pull request to this repo
[x] Setup github pages.

Pages aviliable at: https://iiif-commons.github.io/manifesto-3d/


express 3

Is this suppose to be supported in 3.x? Seems to just continuously be "starting..." but doesn't ever complete.

Fixed in 2.0.0 (just published) by PR #5.


Update SOCIETY.Schools_PreKto12

Benefit
It has been a number of years since the schools data has been updated. This update will allow us to show schools for the current 2023-24 school year and provide the Utah Board of Education feedback of incorrect addresses.
Acceptance Criteria

Updated school locations with relevant demographics added to the attributes.
Corrected addresses returned to the BoE so that geocoding schools in the future is less painful.

Notes
No response
Risks
None
Issue Reference
refs #

Hopefully progress can be made this November when 2024/2025 school locations are published be UBOE.


Mrc 4808 Add Base Select

Still won't be able to see anything in the demo yet, but we are very close. This is the base for both the single select and the multi select.
Thanks @EmmaLRussell for the suggestion to use mixins in #2. It uses a simple mixin to extract out the expand, collapse and flattenOptions logic. The structure of it is very similar to the single select component you guys reviews in #1 however note the slot in the c-dropdown-toggle, that will come in handy for when we have to display the tags for the multiselect! The other difference is that the dropdown-item component is now also given this conditional checked prop. Otherwise I believe everything in the template is the same as the single select so hopefully shouldnt be too bad to review.

Are you going to add a test harness here or something that we can run up and look at the different selects?

Just seen this in the next PR!


Mapper types aren't being  applied for all typescript-resolvers

Describe the bug
Hi everyone,
This is my first time creating a public issue (and I'm new to TS and Graphql), so please forgive me in advance for any mistake.
I believe that this issue is very similar to https://github.com/dotansimha/graphql-code-generator/issues/1041. It looks like the mapper isn't always mapping types correctly in the current version. I'm creating a new issue as it was requested in https://github.com/dotansimha/graphql-code-generator/issues/1041.
Here is the case that we have seen recently.

My GraphQL schema:

type MovieInsights {
  movieStarsPerMonth: [MonthlyCounts!]!
  movieStarsCumulativePerDay: [MovieCounts!]!
}

type MonthlyCounts {
  month: Date!
  movieCounts: [MovieCounts!]!
}

type MovieCounts {
  # The dynamically resolved Movie type
  movie: Movie!
  stars: Int!
  period: Date!
}

The generated mappers:
Movie: MappedMovie
MonthlyCounts: MonthlyCounts
MovieCounts: Omit<MovieCounts, 'movie'> & {
  movie: ResolversTypes['Movie']
}

:point_up: See that the mapper correctly omitted the movie from the mapping for MovieCounts as it has its own resolver. It doesn't do the same for MonthlyCounts that indirectly uses Movie - it does so through the MonthlyCounts. The generated typescript types:
export type MonthlyCounts = {
  month: Scalars['Date']
  movieCounts: Array<MovieCounts>
}

export type MovieCounts = {
  movie: Movie
  stars: Scalars['Int']
}

My GraphQL operations:

The problem comes when I'm implementing the resolver for movieStarsPerMonth (that uses MonthlyCounts). I keep getting the error: Type 'MappedMovie' is missing the following properties from type 'Movie': boxOffice, worldwideReleases, awards, and 15 more.ts(2322).
The implementation of each query:
// Works fine!
async movieStarsCumulativePerDay(
  { studioId },
   _args,
  { services }
){
  const starCounts = ... // get from service
  return await Promise.all(
    starCounts.map(async (dailyCounts) => {
      // movieService returns a MappedMovie
      const movie = await services.movieService.getMovie(s.movieId)
      return {
        movie,
        period: dailyCounts.period,
        stars: dailyCounts.users
      }
    })
  )
}

// This gives me the error shown above: Type 'MappedMovie' is missing the following properties from type 'Movie': boxOffice, worldwideReleases, awards, and 15 more.ts(2322)
async movieStarsPerMonth(
  { studioId },
  _args,
  { services }
) {
  const yearlyStars = ... //get from a service
  const counts = yearlyStars.map(
    async (monthCounts) => {
      return {
        month: monthCounts.month,
        movieCounts: await Promise.all(
          monthCounts.movies.map(async (s: StarCounts) => {
            // movieService returns a MappedMovie
            const movie = await services.movieService.getMovie(s.movieId)
            return {
              movie,
              period: s.period,
              stars: s.stars
            }
          })
        )
      }
    }
  )
  return await Promise.all(counts)
}

Expected behavior
The mappers correctly map the dynamic MappedMovie to Movie and the compiler is happy 💯 .
Environment:

OS: OSX
The library versions that we have:

<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

NodeJS: v8.11.4

Notably the same error exists after on @1.2.0 ☝

@mshwery @solon-aguiar I think that I fixed that, maybe you can assist me with it?
I added this test before applying the last fix: https://github.com/dotansimha/graphql-code-generator/blob/master/packages/plugins/typescript/resolvers/tests/mapping.spec.ts#L58-L98 , but maybe it's missing something?
Thank you!

I think the missing piece is one layer of indirection:
type Movie {
  id: ID!
  title: String!
}

type Book {
  id: ID!
  author: String!
}

union MovieLike = Movie | Book

type NonInterfaceHasNarrative {
  narrative: MovieLike!
  movie: Movie!
}

# This one will have the issue – it resolves `movies: Array<NonInterfaceHasNarrative>` which is the fully resolved type from the graphql SDL, instead of applying `movie: ResolverTypes['Movie']`
type LayerOfIndirection {
  id: ID!
  movies: [NonInterfaceHasNarrative!]!
}

It seems strange that the SDL types themselves don't refer to ResolverTypes['MappedType']. Is there a reason for that?

@mshwery the basic types are generated by typescript and they are the base types, ResolversTypes just refers to it and replaces fields when needed.
I'll take a look soon and try to solve it. Thanks!

@mshwery thank you for the reproduction, and sorry for the delay. I managed to understand the issue, reproduce it and fix it :)
Now for a given schema:
      type Movie {
        id: ID!
        title: String!
      }

      type Book {
        id: ID!
        author: String!
      }

      union MovieLike = Movie | Book

      type NonInterfaceHasNarrative {
        narrative: MovieLike!
        movie: Movie!
      }

      type LayerOfIndirection {
        id: ID!
        movies: [NonInterfaceHasNarrative!]!
      }

      type AnotherLayerOfIndirection {
        inner: LayerOfIndirection!
      }

you will get:
    export type ResolversTypes = {
      String: Scalars['String'],
      Boolean: Scalars['Boolean'],
      Movie: MovieEntity,
      ID: Scalars['ID'],
      Book: Book,
      MovieLike: ResolversTypes['Movie'] | ResolversTypes['Book'],
      NonInterfaceHasNarrative: Omit<NonInterfaceHasNarrative, 'narrative' | 'movie'> & { narrative: ResolversTypes['MovieLike'], movie: ResolversTypes['Movie'] },
      LayerOfIndirection: Omit<LayerOfIndirection, 'movies'> & { movies: Array<ResolversTypes['NonInterfaceHasNarrative']> },
      AnotherLayerOfIndirection: Omit<AnotherLayerOfIndirection, 'inner'> & { inner: ResolversTypes['LayerOfIndirection'] },
    };

Fix is in: https://github.com/dotansimha/graphql-code-generator/pull/2022

Fixed in 1.3.0 🎉


Revisions page - Revision details

Deployment status

Chart name
Chart revision and date
Tags: chart version, app version, namespace and cluster
Install status (the message from helm)
Action buttons: reconfigure, uninstall, check for new version

Uninstall will display a confirmation popup with the details (what the user is deleting and from where and which objects will
be deleted) and confirm button
Reconfigure – Will open a dialog where the users can:
Choose which version to install (from a dropdown)
View which version is currently installed, release name, namespace
View the cluster that the chart is installed on
View and edit the user-defined values in a textbox, on every edit (using debounce) the system will calculate and display
the changes under “manifest changes” area

in case of an error the user will see details about it
in case of success the user will see side by side the YAML changes
View the chart values reference
Action buttons – scan for problem and confirm
The reconfigure button will be displayed only if the user have the latest version installed, in case there isn’t they will see
“upgrade to <version_number>” button instead
Check for new version button will check if there is a new version available and will change the “reconfigure” button to the
“upgrade” button

Suggestion: We can auto-check for new version on every time interval and remove this button

@chad11111
I would love to take this one.


Cow: Watch<PHONE_NUMBER>1322(?# facebook ID Physiotherapy-Home-Care)

Cow requests the watch of the watch_keyword<PHONE_NUMBER>1322(?# facebook ID Physiotherapy-Home-Care). See the MS search here and the Stack Exchange search in text, in URLs, and in code.

Rejected by tripleee in Charcoal HQ. Reason: 'bad timing, duplicate submitted at the same time'


Draft PR for PostgreSQLHSC

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[x] I commit to follow the Breaking Change Policy of “no breaking changes
[x] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[x] Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

Adding new API(s)
Adding a new API version
Adding a new service

[x] Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If there are following updates in the PR, ensure to request an approval from API Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in stable version
[ ] Removing properties in stable version
[ ] Removing API version(s) in stable version
[ ] Updating API in stable version with Breaking Change Validation errors
[ ] Updating API(s) in preview over 1 year

Please follow the link to find more details on PR review process.

/azp run automation - sdk

/azp run unifiedPipeline

/azp run unifiedPipeline

/azp run unifiedPipeline

/azp run unifiedPipeline

/azp run unifiedPipeline

/azp run unifiedPipeline

/azp run unifiedPipeline

/azp run unifiedPipeline

/azp run unifiedPipeline

/azp run unifiedPipeline

/azp run unifiedPipeline

@msyyc @kairu-ms Is ok for python and ruby SDK using same namespace as another spec.

@msyyc Yuchao Yan FTE @kairu-ms Kai Ru FTE Is it ok for python and ruby SDK using same namespace as another spec?

It's ok for Ruby SDK

@enescakir Is your PR is not ready yet? If so, please add DoNotMerge label. And I'll ask ARM review again after your PR is ready. Thanks

@ChenTanyi We are ready now, you can ask ARM review.

@j5lim There are lots of API changes after your singedOff. Could you please review it again from ARM side?

@enescakir Please let me know when it ready to merge.

@ChenTanyi it's ready to merge.

/azp run


Add n8n + simple GHA workflow

Added n8n
Added a simple GHA workflow to start, wait and then stop all the services

Do the docker resource minimum requirements need to be increased?


Add delete option in Notes Taking App

Add a delete option to delete a particular note here

Hey @swapnilsparsh please assign me this task under IWOC'23.

@ananyakushal You are already assigned with 2 tasks so finish them off then get it assign on new task.

Sure.

@swapnilsparsh this is my first project but please assign me to this task


Update to Kotlin 1.8

Updates the project to Kotlin 1.8, along with a few other dependencies.

[x] Unmark as draft once kotlinpoet has a release of 1.13.0 with Kotlin 1.8 updates, currently using snapshot

Fixes #119

Rebased on top of main for new set of checks to take effect.


Improve FunctionPointer class

Currently the existing APIs are difficult to work through when composing classes with callbacks. Currently function pointers must be duplicated to pass through callback APIs. This level of indirection needs to be duplicated across any classes that pass callbacks.

The callback interface is currently not standardized. This is notable in differences between APIs such as Ticker and Thread. The current FunctionPointer does not support the existing callback types and can’t be standardized as is. Additionally this has resulted in FunctionPointer being duplicated across the developer site (example).

FunctionPointer is not forward compatible with C++11. Even if we hold off for now, eventually compatibility with std::function will eventually become a requirement. Libraries have already started to use the FunctionPointerArgN classes. By introducing a forward and backward compatible function pointer class now, we can reduce the confusion caused by the required name change for  C++11 later.

Here are the proposed changes for the FunctionPointer class:

Adopt C++11 style template arguments, requires rename to Callback
Add constructor for C style callback functions
Add constructor for passing Callbacks
Add static function for passing to C style callbacks
typedef FunctionPointer/FunctionPointerArg1 for backwards compatibility

cc @pan-, @sg-, @c1728p9
mv #94, #131, #139

@mbed-bot: TEST
HOST_OSES=ALL
BUILD_TOOLCHAINS=ALL
TARGETS=ALL

Looks like there might be a problem with this patch and the async api.

Moved to #1793 since @geky is away


Here is my Gold price prediction by using LSTM model

I achieved 96% accuracy

@PRIYANSHU2026 You already added the same project in another repo kindly do not spam it's against the rule.
here


Unable to create order

Describe the bug
When I try to update the certificates I get this problem.
(Unable to create order.)
Expected behavior
He should simply update my certificates like he always did.
Platform:

OS: windows 2012 r2
Version: Win-acme 2.1.18 pluggable]

Additional context
It was working normally, I just updated these new certificates and now I get this error.

Could you run with --verbose and post a full log please?


ztpd exit code is always 0 regardless of exit condition

Describe the bug
The ztpd daemon always returns a 0 exit code from its main process, regardless of whether the process exited cleanly or not. This is due to ztpd_run() having a void return type.
To Reproduce
Steps to reproduce the behavior:

ztpd -c /path/to/ztpd/config.json -d
kill -9 $(pidof -s ztpd)

Expected behavior
ztpd exits with a non-zero exit code.
Desktop (please complete the following information):

OS: Azure Percept Device
ztp v0.4.0.

The event_loop_run() return type needs to be changed from void to int, and the return value should indicate the status of the event loop. 0 should be returned from event_loop_run() if the event loop exited cleanly, otherwise, an appropriate negative errno value (eg.  -EINVAL, -ENOSYS, etc. should be returned.
The return value should be specified using the event_loop_stop() function. Its signature should be changed to:
int
event_loop_stop(struct event_loop *loop, int retval)

Where retval is the even loop return status to be returned from event_loop_run(). This value can then be propagated to event_loop_run() (probably through the struct event_loop context with a new field/member).

Do you want the error code to only be propagated if we enter on_signal_received, or should we have the error code from other events, like dpp_start, to also be propagated up to main?
`on_signal_received(int fd, void *context)
{
struct signalfd_siginfo siginfo;
ssize_t bytes = read(fd, &siginfo, sizeof siginfo);
if (bytes < (ssize_t)(sizeof siginfo)) {
zlog_warning("siginfo read returned too little data");
return;
}
switch (siginfo.ssi_signo) {
    case SIGINT:
    case SIGTERM:
        event_loop_stop((struct event_loop *)context);
        break;
    default:
        break;
}

}`
It looks like this is the only place where event_loop_stop is being used.

I think it makes sense to only propagate from where event_loop_stop is currently used since that is the only proper/expected way to stop the event loop.

Ok, so I only have to worry about if ztpd received a SIGINT or a SIGTERM for this issue right?

Ok, so I only have to worry about if ztpd received a SIGINT or a SIGTERM for this issue right?

Yep, seems like that's the only place where the event loop is stopped.

#16


build: use moov-io/cryptfs instead of copied gpgx package

Codecov Report

Merging #73 (e262579) into master (5bdedb2) will decrease coverage by 6.21%.
The diff coverage is 0.00%.

:exclamation: Current head e262579 differs from pull request most recent head 3d854b3. Consider uploading reports for the commit 3d854b3 to get more accurate results

@@            Coverage Diff             @@
##           master      #73      +/-   ##
==========================================
- Coverage   19.14%   12.93%   -6.22%
==========================================
  Files          12       10       -2
  Lines         423      348      -75
==========================================
- Hits           81       45      -36
+ Misses        323      298      -25
+ Partials       19        5      -14

Impacted Files
Coverage Δ

pkg/filelist/bucket.go
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5bdedb2...3d854b3. Read the comment docs.


Updated Nil-Checking section of the README.

Updated the Nil-Checking section of the README with three new cases.
Included standard for the below cases:
-If we want to enter a scope if at least one object is non-nil
-If we want to enter a scope if the object is nil
-If we want to enter a scope if the an object is non-nil/nil and a bool is true/false.

Feels more like best practices, or even tips to me. Good tips. Yet not sure it should live there

I'm ok with either approach; Adding more examples for various use cases to the ruleset is perfectly valid IMO

Ok. I'm not against it either. And it makes more sense to have it right after the rule. But there is a difference between rule and examples, and as the guide grows it might get to a point it's too big. We'll see if/when we get there

@lpollard11 would you mind moving this to the Best Practices section and create a link from the style guide nil-checking rule?

Yeah for sure! @ThibaultKlein Do you want me to move all of it or just part of it?

Move it all please :)

@lpollard11 @ThibaultKlein I'd also maybe add a link in this section referring to it in the best practices so  we can point it out. I think it's valuable information and my worry is that if we don't directly link back to the best practices guide, no one will ever use it / know of it.

That's something we might do for other rules as well, would be good to have a format for it (the same way we have the rationale thing. Give it a try we'll see how it looks

Sounds good all, I'll move it all to the best practice and include a link to that in this nil-checking section in the actual style guide.

I left a comment, still approving since it's not a huge deal
:+1:

:+1:


compilation failed on mac

Dear Authors
I tried to install Nimbus on my personal computer (Mac Book pro) and it failed at the "make" step
please see the error below. I am not very familiar with C compilers but my searches on the web point out that the '-L/usr/lib64' option is not well recognized on Mac.
I checked for clang possible errors but didn't find anything relevant on stackOverFlow or similar forums.
Thank you by advance for your suggestions
Best regards
Nicolas
/Library/Developer/CommandLineTools/usr/bin/make -C cplusplus/trim
mkdir -p bin/
g++ build/adapter_trim.o -g -L/usr/lib64 -std=c++0x -pthread
-Irwwb
-Iinclude
-lboost_program_options
-o bin/nimbus_trim
ld: warning: directory not found for option '-L/usr/lib64'
/Library/Developer/CommandLineTools/usr/bin/make -C cplusplus/align
/Library/Developer/CommandLineTools/usr/bin/make -C lib/libnimbus
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/AdapterTrim.cpp -o build/AdapterTrim.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/Alignment.cpp -o build/Alignment.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/AlignmentBuilder.cpp -o build/AlignmentBuilder.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
src/AlignmentBuilder.cpp:40:29: warning: delete called on non-final
'Nimbus::alignment::Alignment' that has virtual functions but non-virtual
destructor [-Wdelete-non-virtual-dtor]
if( f_alignment != NULL ) delete f_alignment ;
^
src/AlignmentBuilder.cpp:41:29: warning: delete called on non-final
'Nimbus::alignment::Alignment' that has virtual functions but non-virtual
destructor [-Wdelete-non-virtual-dtor]
if( r_alignment != NULL ) delete r_alignment ;
^
2 warnings generated.
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/Amplicon.cpp -o build/Amplicon.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/AmpliconAlignment.cpp -o build/AmpliconAlignment.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/AmpliconIndex.cpp -o build/AmpliconIndex.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/Read.cpp -o build/Read.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/SAMrecord.cpp -o build/SAMrecord.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/Utils.cpp -o build/Utils.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/_DNANode.cpp -o build/_DNANode.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/io.cpp -o build/io.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/stdafx.cpp -o build/stdafx.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
ar -cvq libnimbus.a  build/AdapterTrim.o  build/Alignment.o  build/AlignmentBuilder.o  build/Amplicon.o  build/AmpliconAlignment.o  build/AmpliconIndex.o  build/Read.o  build/SAMrecord.o  build/Utils.o  build/_DNANode.o  build/io.o  build/stdafx.o
q - build/AdapterTrim.o
q - build/Alignment.o
q - build/AlignmentBuilder.o
q - build/Amplicon.o
q - build/AmpliconAlignment.o
q - build/AmpliconIndex.o
q - build/Read.o
q - build/SAMrecord.o
q - build/Utils.o
q - build/_DNANode.o
q - build/io.o
q - build/stdafx.o
/opt/local/bin/ranlib: file: libnimbus.a(_DNANode.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(stdafx.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(_DNANode.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(stdafx.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(_DNANode.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(stdafx.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(stdafx.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(_DNANode.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(_DNANode.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(stdafx.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(_DNANode.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(stdafx.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(_DNANode.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(stdafx.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(_DNANode.o) has no symbols
/opt/local/bin/ranlib: file: libnimbus.a(stdafx.o) has no symbols
rm -r build
/Library/Developer/CommandLineTools/usr/bin/make -C lib/libthreadutils
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/Signal.cpp -o build/Signal.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/TQueue.cpp -o build/TQueue.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
mkdir -p build
g++ -c -g -Wall -O4 -std=c++0x -Iinclude src/stdafx.cpp -o build/stdafx.o
clang: warning: -O4 is equivalent to -O3 [-Wdeprecated]
ar -cvq libthreadutils.a  build/Signal.o  build/TQueue.o  build/stdafx.o
q - build/Signal.o
q - build/TQueue.o
q - build/stdafx.o
/opt/local/bin/ranlib: file: libthreadutils.a(Signal.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(TQueue.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(stdafx.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(Signal.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(TQueue.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(stdafx.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(Signal.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(TQueue.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(stdafx.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(Signal.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(stdafx.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(TQueue.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(Signal.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(TQueue.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(stdafx.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(Signal.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(TQueue.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(stdafx.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(Signal.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(TQueue.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(stdafx.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(Signal.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(TQueue.o) has no symbols
/opt/local/bin/ranlib: file: libthreadutils.a(stdafx.o) has no symbols
rm -r build
mkdir -p bin/
g++  build/Manager.o  build/Reader.o  build/Worker.o  build/Writer.o  build/main.o  build/nimbus.o  build/nimbusheader.o  build/opt.o  build/preprocess.o -g -L/usr/lib64 -std=c++0x -pthread
-Iinclude
-Llib/libnimbus -lnimbus -Ilib/libnimbus/include
-Llib/libthreadutils -lthreadutils -Ilib/libthreadutils/include
-o bin/nimbus_align
ld: warning: directory not found for option '-L/usr/lib64'
ld: archive has no table of contents file 'lib/libthreadutils/libthreadutils.a' for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [nimbus] Error 1
make: *** [align] Error 2

Dear Nicolas,
unfortunately, we did not test Nimbus on a MAC and we do not have MAC computers available to us.
The quickest solution would be to install Nimbus in a Ubuntu virtual machine  On my windows PC, I use VirtualBox, but there are other options available on MACOS.
with kind regards,
Rutger Brouwer

Dear Rutger
Thank you for your answer and your suggestion. Actually, I was on the same way and I tried to install Nimbus on a VirtualBox. It seems to work (at least the make didn’t end with an error).
I would like to test if everything run correctly on a small dataset.  For now my searches did not point me any public dataset with amplicon design, suitable for Nimbus. Do you know where I can find such dataset ?
In any case, thank you for taking time to answer me.
Best
Nicolas
Nicolas Beaume, Ph.D
Bioinformatics & machine learning
social networks :
LinkedIn https://www.linkedin.com/in/nicolas-beaume-bioinfo/
Research gate https://www.researchgate.net/profile/Nicolas_Beaume
Facebook http://facebook.com/Nicolas.Beaume.BIS
Twitter https://twitter.com/Nicolas_beaume
web site https://braininsilico.com/

Le 13 sept. 2018 à 16:57, erasmus-center-for-biomics<EMAIL_ADDRESS>a écrit :
Dear Nicolas,
unfortunately, we did not test Nimbus on a MAC and we do not have MAC computers available to us.
The quickest solution would be to install Nimbus in a Ubuntu virtual machine On my windows PC, I use VirtualBox, but there are other options available on MACOS.
with kind regards,
Rutger Brouwer
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/erasmus-center-for-biomics/Nimbus/issues/6#issuecomment-421037043, or mute the thread https://github.com/notifications/unsubscribe-auth/AZgIqLNbzAcLjSxlw5r8VjAG242amNjlks5uanI_gaJpZM4WhzPg.


fix(wi-view-height): fix wi view height after applying filters

Fix List view and board view height after applying filters

@SMahil fabric8/fabric8-ui:SNAPSHOT-PR-2272-1 is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-2272-fabric8-planner.badger.fabric8.io

@SMahil fabric8/fabric8-ui:SNAPSHOT-PR-2272-2 is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-2272-fabric8-planner.badger.fabric8.io

@SMahil fabric8/fabric8-ui:SNAPSHOT-PR-2272-3 is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-2272-fabric8-planner.badger.fabric8.io

Last work item of the list not completely visible.

Unable to change WI type in WI quick add after selecting a filter


Error with Move and Order

Getting a repeating error in the log about an issue with the Move and Order part of the stats.  We changed it to "Move,Move" and it works fine.
https://i.imgur.com/3wQ7Zj5.png

It appears that the data you are reading is not in the correct format. Dnd
Beyond has been fiddling with their page and you may need to adjust the
provider to compensate.
Best Regards,
Lord Ashes
On Thu, Dec 1, 2022 at 3:34 PM dcsparx @.***> wrote:

Getting a repeating error in the log about an issue with the Move and
Order part of the stats. We changed it to "Move,Move" and it works fine.
https://i.imgur.com/3wQ7Zj5.png
—
Reply to this email directly, view it on GitHub
https://github.com/LordAshes/TaleSpire-BeyondLinkViaChrome/issues/8, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AGQ5Y6EQSFWWCMZHTBKEUY3WLEDXDANCNFSM6AAAAAASRGEACM
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Would you be willing to show/tell me where to change that?

Of course. The data extractor is in the injection.js file found in the
Chrome Extension folder. If you open it up, you will see a bunch of
functions for extracting data and then near the end of the file, the code
that actually calls those functions and formats the JSON object that is
sent to the server. To make things easier, I made functions like GetStat
and GetModifier that uses the "key" string that you provide it to extract
the desired piece of data. If the website changes a bit, those string may
not longer exist (e.g. may have been renamed) and thus you need to see what
they have been changed to.
You can do that by loading your DND change in DndBeyond and then using the
INSPECT browser option to look at the HTML DOM. Do not use the View Source
option, that will not work. Now in the INSPECT you need to find the section
of HTML code that surrounds your data and update the string "key" in the js
file accordingly.
Once you modify the js file, you will need to go to the extension in Chrome
and press the update button (the circular arrow button) to have Chrome
update to using the latest injection.js file.
The best way to test if your extract is correct is to run the server in
LEARN mode (see docs for more info). Don't run Talespire but instead go to
the BeyondLinkServer.exe folder and then run "BeyondLinkServer.exe 9100
LEARN". The LEARN parameter will tell the server to stay open even if
Talespire is not running.
With the server open, refresh your DndBeyond page and the data extractor
will stat sending data to the BeyondLinkServer. You will see changes being
written to the screen but you can also look at the content.json which
contains the whole JSON that was sent from the Chrome Extension. If you see
empty pieces of data or wrong looking data you know you have an issue with
the extractor.
That is a quick summary. If you need more help I suggest coming over to the
Unofficial Talespire Modding Discord and we can chat there to assist you.
Lord Ashes
On Fri, Dec 9, 2022 at 2:56 PM dcsparx @.***> wrote:

Would you be willing to show/tell me where to change that?
—
Reply to this email directly, view it on GitHub
https://github.com/LordAshes/TaleSpire-BeyondLinkViaChrome/issues/8#issuecomment-1344727502,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGQ5Y6BBJ3FUHB5OYG5T6L3WMOFITANCNFSM6AAAAAASRGEACM
.
You are receiving this because you commented.Message ID:
@.***>


update gp tutorial

@fehiepsi i've made a first pass at trying to improve the readability of this tutorial.
i'm more or less satisfied with the first part (before the sparse stuff). the second part (sparse stuff) still needs some work. i'd like to get some feedback from you before i finish editing the second part. i also wonder if some of the naming conventions in the GP code could be improved (see below)?
for the first part (non-sparse)
note that i've refactored the plot function. in some cases i'm a little confused about when the observation noise is/isn't included in the kernel. please make sure i haven't introduced anything wrong here. also let me know if i've removed  or distorted any explanatory points that you thought were worth making.

is there any reason we'd still need pyro.enable_validation(False) ?
if i understand it right, GPRegression can only do gaussian likelihoods. this seems like a strange naming convention: regression doesn't mean gaussian likelihoods it means continuous outputs. or am i missing something?
why did you want to use gpr.fix_param("noise", torch.ones(1) * 0.01) in the MAP section?

for the second part (sparse)

it seems like RBF$$$variance_MAP is constrained (and thus positive) while RBF$$$variance is not. isn't this confusing?
similar to the point above it seems that SparseGPRegression implies gaussian likelihood?
why fix the noise to 0.01 in the last model?

@martinjankowiak Thank you very much for revising gp tutorial! Here are some of my comments:

I think we can set_validation now because we have updated pyro to a pytorch commit which solved the instability of precision matrix computation.
Both GPy implementation and GPflow's one use Gaussian likelihood for regression. In GP literature (to my knowledge), GP regression also mean Gaussian likelihood. So I think that it is a good name here.
About naming, I am more concerned about the name: SparseVariationalGP and SparseMVN. To me (and from literature), it is better to call it VariationalSparseGP. I used SVGP because GPflow authors used that name for their implementation. About SparseMVN, I think it is better to call it LowRankMVN (to compatible with literature about low-rank approximation of a covariance matrix, e.g. Stein's).
About constraint, if we use pyro.get_param_store().named_parameters() then due to this, it will return unconstrained value. For all parameters of GP module, to me it is better to call model.get_param(param_name) or kernel.get_param(param_name). We can call pyro.param(param_name_with_module_name) but it will take extra effort to define param_name_with_module_name.
We can set a prior to noise with a small value. I remember that it is fixed there just to illustrate fix_param functionality, which is unnecessary now because we have a better doc now.
To include/eliminate noise in the prediction, you can control it through noiseless parameter (in the plot function). I have included noise in all plots but you can change that behavior to what is better for you.
For GPR and SGPR, kernel is still the original kernel. Noise is added into the kernel (to create new kernel) just to help computation.
In a recent commit, you can set name to kernels, models to avoid duplicated params (if you don't want to use pyro.clear_param_store()). E.g.
kernel2 = RBF(input_dim=1, variance=torch.tensor(0.1), lengthscale=torch.tensor(10.)), name="RBF2"
It is good to expose SVI steps in the first inference. For the later ones, you can simplify it by
gpmodel.optimize(optimizer=optim.Adam({"lr": 0.01}), num_steps=100) (0.001 and 1000 are default values).

re: regression, i'm afraid that's not the convention. for example see here:
https://papers.nips.cc/paper/3806-gaussian-process-regression-with-student-t-likelihood

@martinjankowiak Personally, I think of "GPR" as a combined name, rather than a GP model to do regression problem. The paper you mentioned said that "In the Gaussian process regression the observation model is commonly assumed to be Gaussian, which is convenient in computational perspective." in its abstract. The standard reference: GPML book assumes Gaussian and to my knowledge, other GP libraries also follow this convention. So to not make users confused, we should find an appropriate name. Do you have a better name for it? Maybe GaussianGP?

@fehiepsi thanks for your quick response!

i don't love the name GPRegression but i guess it's fine. still, the docs should be updated
https://github.com/uber/pyro/blob/dev/pyro/contrib/gp/models/gpr.py
to make it clear that the 'noise' is referring to the use of a gaussian likelihood with a particular variance. it should be very clear that it doesn't support other likelihoods.

i've made another pass. this is now ready for review. it would be great if you could take a close look!

@martinjankowiak I can't make a review directly from web due to large diff so I make comments here. Mostly they are just suggestions so you can select which ones are appropriate:

At this point

Here  x,x′∈X  are points in the input space and  y∈Y  is a point in the output space.  f  is a draw from the GP prior specified by the kernel  Kf  and represents a function from  X  to Y .

Because we use y = f + eps, I think y should be an output vector rather than a point here.

I guess no need to call "pyro.clear_param_store()" at the beginning

In the plot function, I think it is better to have comment: "# (iii) plots samples from the prior (with no conditioning on observed data, inducing points, and variational parameters)" and update the code to cover variational model (though we don't use it)

noise = model.noise if type(model) != SparseVariationalGP else model.likelihood.variance
cov = kernel.forward(Xtest) + noise.expand(n_test).diag()

In the later part of Inference section, do you feel that there are many "Note that" phrase? ^^
In Fit the model using MAP section, to avoid the unconstrained param problem, we can use:

for param_name in pyro.get_param_store().get_all_param_names():
    print(param_name, pyro.param(param_name).item())

In More Sparse GPs, I hope you can introduce the sv-dkl example when mentioned about other likelihoods.

"So we can compare to what we've done above, we're still going to use a gaussian likelihood." -> "To compare to..."

Across the tutorial, do you think that it is better to use "Gaussian" instead of "gaussian"?

About docs of GPRegression, I think we already have made it clear at this section.

@fehiepsi great, thanks, i'll incorporate your feedback and make another pass.
re: the docstring of GPRegression, nowhere does it say that the noise is gaussian. it just says 'noise'

Ah, got it. I will update its docs in a future pull request related to GP.

@fehiepsi  i've incorporated your feedback. thanks! this should now be good to go.
open to comments/suggestions from anyone else!

@martinjankowiak is this good to merge? if @fehiepsi is fine with it im happy to merge it and add additional adjustments in future PRs

Yes, I like it. :D


Access both member/column and row by passing argument values

To develop a function that retrieves a cell value by tablename, column, and row performance is affected by allocating an entire column with this:
function get_lt_byc(lt, c, r)
       lt[c][r]
end

 @btime get_lt_byc($lt,$val, $r)
  15.406 ns (1 allocation: 32 bytes)
10.0

versus embedding the column to be accessed in the function's definition with by accessing the row tuple, then the element:
function get_lta(lt,r)
       lt[r].a
end

@btime get_lta($lt, $r)
  1.500 ns (0 allocations: 0 bytes)
10.0

or by accessing the column in the function and then getting the row value from the column with this:
function get_a_then_r(lt, r)
       lt.a[r]
end

@btime get_a_then_r($lt,$r)
  1.500 ns (0 allocations: 0 bytes)
10.0

In the first case the column, c, must be passed as a symbol and there is an allocation for the entire column. The row, r, is passed by integer.  In the second case, we get the benefit of laziness by derefing the table to get a named tuple of a row and then accessing the element, a.   In the second case, the row is accessed and the member of the tuple is accessed with dot notation, so we must get some laziness benefit.  The third case is how both LazyTables and TypedTables work:  get the column with dot notation, and then the value at the row index.
So, why does this matter at all?  It's nice to be able to pass arguments for both the column (or named tuple member) and the row.  Seemingly, the only way to get "compiled away" speed is with dot notation.
I don't know if there is any way around this, but it sure could be nice.

It has to do with the Tables.jl implementation.
function get_row_then_column(lt, row::Integer)
    # lt[row] returns a single row (akin to a named tuple) and return type of column is known
    lt[row].column
end

and
function get_column_then_row(lt, row::Integer)
    # lt.column returns a single array of known type
    lt.column[row]
end

In both of these cases it is easy for the compiler to figure out the return value.
The issue is that the element type of an arbitrary column is not known until runtime.
If you do know the return value of col ahead of time you can help it out.
function get_lt_col_row(::Type{T}, lt, col, row::Integer) where T
    _row = lt[row]
    getproperty(_row, col)::T
end


[Communities] Add admin capability to edit community, including labels

Problem
Currently, only DevDAO developers with certain permissions have the ability to configure the relevant items for the community page, such as the name, description, and labels. This makes it hard to scale the number of communities and give the relevant users the permissions they need. This ticket introduces a new "Admin" tab on the community space, which will provide more control for managing the community page as well as visibility into the admins of each page.
User Stories

As a user with admin permissions, I want to edit the relevant items for my community so that I can make the necessary changes without relying on DevDAO developers.
As a general user, I want to know who are the admins of each page so that I can contact them if I have any questions, concerns, or feedback.

Acceptance Criteria

There is a new "admin" tab for each community
All users can access this tab using the /communities/community-name/admin URL and stay on that page when they click refresh
All users can see the top public section of the admin tab, which shows includes the list of labels and admin users
Only users with admin permissions to a specific community can see bottom private section of the admin tab, which shows all the fields that were entered during the new community creation process
Only users with admin permissions should have the ability to easily edit and update all of the relevant fields from the new community creation process

@nearmax @frol @ailisp – Please see this ticket for the proposed "admin" tab and let me know if you have any feedback or concerns. For this MPV, I am leaving the Jira/Kanban management out of scope, which we will solve in a separate ticket.

@ori-near let's change the title to the following:
[Communities] Store existing communities on chain & add permission control to enable data ownership for community members

Let's also move it to "In progress"


Update http4s-client, http4s-dsl, ... to 0.23.0

Updates

org.http4s:http4s-client
org.http4s:http4s-dsl
org.http4s:http4s-server

from 0.21.25 to 0.23.0.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Applied Migrations

github:http4s/http4s/v0_22?sha=v0.22.0

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.http4s" } ]

labels: library-update, semver-minor, scalafix-migrations

Superseded by #74.


Style fix to comply with dartfmt

I signed it!

Looks like someone else also had the same idea - the modern syntax is used today
https://github.com/dart-lang/tuple/blob/366318b7cb1d7c79dc00a6852684ff7af518a612/lib/tuple.dart#L61


Use CL-ENVIRONMENTS as fallback for DECLARATION-INFORMATION

For Lisp implementations that do not provide an implementation of CLtLv2's DECLARATION-INFORMATION, fallback to the portable implementation afforded by CL-ENVIRONMENTS.
This allows POLICY-COND to work with ABCL and ECL.

@easye Thanks so much! Would you mind also conditionalizing the #+sbcl with :feature?

@easye Thanks so much! Would you mind also conditionalizing the #+sbcl with :feature?

Implemented as https://github.com/stylewarning/policy-cond/pull/3/commits/351ae000b3d333298504d36c188a7866a2abe0d9.