Add testnet join flag for setting external_address

Is your feature request related to a problem? Please describe.
Right now, public testnet participants peer directly with the PL-run infra when running testnet join. By default, the nodes run by testnet partipicants won't peer with each other, because our tendermint config template sets external_address to any-address:
https://github.com/penumbra-zone/penumbra/blob/40f9de041c2c5cddaedebc05cd3696271f9c8cc7/testnets/tm_config_template.toml#L207-L215
Our pd testnet join logic carefully excises special addresses like "<IP_ADDRESS>" (#1821), so only the PL-run instances will have their addresses set in a new node's p2p.seeds section of the tm config.
Describe the solution you'd like
Allow public testnet participants to opt-in to setting an external_address when joining, e.g.:
pd testnet join --external-address <IP_ADDRESS>

Doing so will enable folks to declare their nodes as external accessible, and cause their external addresses to land in tm configs of subsequently joining nodes.
Describe alternatives you've considered
We've considered leveraging the remote_ip attribute, but decided against it (#1815). It makes more sense to permit testnet participants to opt in to serving as an externally accessible peer.
Additional context
While it's not a hard requirement, let's perform some housekeeping on the testnet code before tackling this change: #1826, #1827. Also, make sure to update the documentation with an explanation of how the --external-address flag functions.

I think we can do this now that #1827 is closed.

Testing in combination with the seed_mode=true setting described in #1847 works just peachy:

Look at all those peers!


TS ignores field_weights option

Hi,
I allways get best mach for field with higher weight. Basically I have 'case' model where I keep personal details like first and last name, cases are related to agencies where I keep agency details. It seems to work fine untill I search for case name that is part of agency name i.e.
when I search for "Emm" as a result I get all records related to "Emmersson" agency but thats not what I need, I need case name in first place so I've set weights but it seems it just doesnt work.
I use Sphinx 2.2.9 and ThinkingSphinx 3.1.4
Please help.
ThinkingSphinx::Index.define :person, :with => :active_record do
  # fields
   indexes [first_name, last_name, family_name ], :as => :name, :sortable => true
   indexes born_date, :sortable => true
   indexes death_date, :sortable => true
   indexes agency.name, :as => :agency
   indexes agency.city, :as => :city
   indexes advertisements.advertisement_media.medium.media_name, :as => :medium
   where "cases.deleted = 0 AND cases.status = 1"
   # attributes
   has created_at, updated_at
   has agency.group_agencies.agency_group.id, :as => :agency_group_ids
   has advertisements.advertisement_media.medium.medium_groups.id, :as => :medium_group_ids
end

I have set default sphinx sope on Case model
class Case < ActiveRecord::Base
  include ThinkingSphinx::Scopes

...
  sphinx_scope(:weights) {
    { :field_weights => {
      :name => 1,
      :agency => 10,
      :city => 15,
      :medium => 10
    }
  }
  default_sphinx_scope :weights
...
end

On Case controller I have:
@results = Case.search Riddle::Query.escape(params[:search]), :star => true, :page => params[:page], :per_page => 6

I've also tried with (same result):
@results = Case.search Riddle::Query.escape(params[:search]), :star => true, :page => params[:page], :per_page => 6, :field_weights => {
      :name => 1,
      :agency => 10,
      :city => 15,
      :medium => 10
    }

As result I get this query:
SELECT * FROM `case_core` WHERE MATCH('*emm*') AND `sphinx_deleted` = 0 LIMIT 0, 10 OPTION field_weights=(name=1, agency=10, city=15, medium=10)

As a result I get related agencies in first place insead of names.

Problem solved. :name weight should be higher than :agency weight.


Small changes to https://github.com/clementchadebec/benchmark_VAE/pull/100

Hi @paul-english,
This PR includes some tiny changes to https://github.com/clementchadebec/benchmark_VAE/pull/100 that I think would be needed before merging.

it fixes a small issue when using the TrainingPipeline with a RAE_L2 instance.
it includes warning messages when a dataloader is passed since this may override the training config
it adds some additional tests

Let me know your thoughts. If you are fine with those suggestions, do not hesitate to merge this PR so I can merge https://github.com/clementchadebec/benchmark_VAE/pull/100.
Thanks again :)
Best,
Clément

@paul-english Can you merge this in.


Webhook: elvis.config not found when creating PR from fork

When a PR is created from a fork, the webhook is looking for the elvis.config in the specific branch from the fork, but the name of the repo being used is always the upstream repo.
The repo used in the call to the elvis_webhook:repo_config/4 function needs to be the one in pull_request > head > repo > full_name, it can be fetched using ktn_maps:get/2.

elvis server is dead.


Negative values for account balance

Is your feature request related to a problem? Please describe.
About the fact that accounts can't have negative values, is it possible to overcome this in some way ?
(In the formance documentation: Accounts in Formance Ledger cannot go negative! (Except for the special @world account).)

Hey Agallan ! Yes it's possible to have account with negative values, there is a mapping system with some rules you can use to allow this, and soon you'll be able to do apply some rules 'per transaction' if you want to allow overdraft on some accounts ! I'll get back to you soon with the documentation about this :)

@jdupas22 we are interested in this too, do you have an example Numscript that shows how to allow overdrafting on a transaction level? Could not find it in the docs.

Hey Edward ! Sorry for the delay ! Yes of course !
send [USD/2 500] (
  source = @foo allowing overdraft up to [USD/2 100]
  destination = @world
)


[SWAP] Add CacheLoader class @open sesame 11/17 16:55

This patch adds cache loader class.
Cache loader supports loading cache elements methods in both synchoronous and
asynchronous manners. Asynchoronous methods are using task executor, and
loods whole cache elements with execution orders in other thread.
Signed-off-by: Jiho Chu<EMAIL_ADDRESS>

:memo: TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2041. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.

:octocat: cibot: @jihochu, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://ci.nnstreamer.ai/nntrainer/ci/repo-workers/pr-checker/2041-202211151619580.66334009170532-9c4997ccaca96cdccbbbf50a710a588c2302eeed/.

:octocat: cibot: @jihochu, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://ci.nnstreamer.ai/nntrainer/ci/repo-workers/pr-checker/2041-202211171655560.88194608688354-9c4997ccaca96cdccbbbf50a710a588c2302eeed/.

:octocat: cibot: @jihochu, test/unittest/memory/unittest_cache_loader.cpp does not include Doxygen tags such as @file @brief @author @bug. You must include the Doxygen tags in the source code. Please refer to a Doxygen manual at http://github.com/nnstreamer/TAOS-CI/blob/main/ci/doc/doxygen-documentation.md

:octocat: cibot: @jihochu, test/unittest/memory/unittest_cache_loader.cpp does not include Doxygen tags such as @file @brief @author @bug. You must include the Doxygen tags in the source code. Please refer to a Doxygen manual at http://github.com/nnstreamer/TAOS-CI/blob/main/ci/doc/doxygen-documentation.md

:octocat: cibot: @jihochu, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://ci.nnstreamer.ai/nntrainer/ci/repo-workers/pr-checker/2041-202212221649450.97625088691711-d8bfd7acc1296f237bc37f7023fb446f63d9197e/.


Restored real valued models that were missing softmax layer

@lisaong,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


dot abapgit: implement user interface for editing

after https://github.com/larshp/abapGit/pull/587

after https://github.com/larshp/abapGit/issues/630

In Advanced -> Repo Settings:


Add Cache Storage Size Limit Feature

Feature Request:
Please add a configurable cache storage size limit. This feature would allow to set a maximum cache size, preventing excessive storage usage.
Use Case:
In apps displaying many images, the cache can grow too large, consuming too much storage. A size limit would help manage storage more effectively.
Proposed Implementation:

Add a parameter for maximum cache size (in bytes) to the configuration:

  await FastCachedImageConfig.init(
    subDir: storageLocation,
    clearCacheAfter: const Duration(days: 7),
    maxCacheSize: 100 * 1024 * 1024, // Example: 100 MB
  );

Implement logic to remove the least recently used items when the cache exceeds this limit.

Benefits:

Prevents excessive storage consumption.

I think you can close this one as you created two issues


Remove ExternalPort from example in README

The externalport parameter has been removed from the server & launch script, so isn't needed in the example. See https://github.com/OpenRA/OpenRA/pull/15458

Thank you!


log file size limit

It might be a good idea to implement a log size rollover strategy in addition to the time-based one.

I think it should be more like "time/size-based flush interval" than a rollover - because we want to upload smaller files too rather than concatenating a ton of chunks and still sending them in one request.

@AndrewDryga Ah, I thought that was already the case (one PUT per log file). But yeah I think that makes sense!

Since we don't have rollover or uploading now, should we just stick a counter here and start failing at 100 MB or something? https://github.com/firezone/firezone/blob/c0f98ea8960841670a704ed2dfc6f8507f1e964b/rust/connlib/clients/shared/src/file_logger.rs#L159
And then at startup do a check of the disk usage / disk free for the log dir, and show an error dialog if we're using too much space, and quietly don't log if the computer is totally out of disk


linux-raspberrypi: Use the linux-raspberrypi.inc from meta-raspberrypi

The linux-raspberrypi.inc was copied from master when the kernel
recipe was added.
It had the BOOTFILES_DIR_NAME var which is not defined
elsewhere and which determined cmdline.txt not being installed
Fix this by using the inc file from meta-raspberrypi and
delete the copied one
Changelog-entry: use the linux-raspberrypi.inc from meta-raspberrypi
Signed-off-by: Vicentiu Galanopulo<EMAIL_ADDRESS>

@balena-ci retest

@balena-ci I self-certify!

@balena-ci retest


Disabling blog

Hi,
Thanks for the great work.
Sorry this is probably a noob question.
I would like to disable the blog form the website but I could not figure out how.
I tried to add blog and _posts to exclude: in my config file but it only worked partially. The posts do not show up anymore but the blog link is still in the navigation bar. How should I remove the blog link from the top of the page?
Thanks in advance

In _includes, comment the line in header.html that links to the blog blog

If anyone else comes looking for this, now the blog can be disabled on _config.yml by leaving the blog_name line empty

Hi,
related issue:
I would like to keep the blog but instead use a different title ("writings"), and accordingly also use the permalink /writings/ (instead of /blog/).
I have made the respective changes in config.yml, header.html, etc. My "writings" page does show on the webpage as intended, but I cannot get the posts + pagination links to consistently redirect to the updated permalinks. Whenever I restart my local setup, the original "blog" folders and links are generated from scratch and automatically inserted back into my site folder, and so the posts + pagination links then again link back to /blog/ instead of /writings/ (e.g., http://localhost:4000/blog/2021/distill/ instead of http://localhost:4000/writings/2021/distill/).
My current setup can be found in https://github.com/ottuba/github-website – I'd appreciate any help!

If anyone else comes looking for this, now the blog can be disabled on _config.yml by leaving the blog_name line empty

blog_name did not work for me, but empying blog_nav_title worked.

Answered in #627.


Fix nil-error when APN returns no tokens

This error was caught in the wild and happens when APN doesn't return any data, but just "data" that results in length being nil.
This adds a test case that reproduces the error and fixes it by stopping the fetching of data as soon as nil is encountered.
Also: an unused local variable is removed by replacing it with the _.

I also had to update the .travis.yml file since the Travis builds failed on too-old-ruby-versions. The guard dependency somehow pulled in ruby_dep, which crashes bundle install when it doesn't know the current Ruby platform. I did two things:

Remove guard from the Gemfile, since no active pling developer uses it.
Update the ruby-versions in .travis.yml


fix: enable use of pretokenized dataset with padding_free

Description of the change

Related issue number

How to verify the PR

Was the PR tested

[ ] I have added >=1 unit test(s) for every new method I have added.
[ ] I have ensured all unit tests pass

Cherry picked the fix


SAMZA-2227 : Fix to ensure that new topics detected by regex-monitor are picked up during next deployment of app

@jagadish-v0 Please take a look.

Addressed all comments.


The yum_package resource has a leaky behaviour with specific use cases

Description
Some yum_package resources exhibit a behaviour which kind of looks like a memory leak. Forcing a GC doesn't release the memory, however, if it's being run in a loop, it eventually stops increasing hence the reason for not classifying it as a true memory leak.
While the replication case mainly revolves around an RPM package which is downloaded from S3 due to the sheer size of used up memory, it may be reliably reproduced with something like yum_package[logrotate] action [:upgrade]. This resource is part of the logrotate cookbook from the Chef Supermarket.
Chef Version
12.18.31
Platform Version
CentOS Linux release 7.3.1611 (Core) / x86_64
Replication Case
The replication case requires the get_process_mem gem. Optionally, you may install GNU test to wrap the process for some basic profiling (i.e the test RPM package on CentOS).
This gem returns the RSS size which may not be the most accurate, however it is a good indicator when things go pear shaped. Chef runs on a schedule invoked from cron. While it doesn't have lasting effects, most of the time, when resident services are starting to thrash or the OOM killer takes out the one process which is probably more important than Chef when the Chef client is running, people tend to get bit upset.
# ssm.rb
require 'get_process_mem'

package = "#{Chef::Config[:file_cache_path]}/amazon-ssm-agent.rpm"

remote_file package do
  source 'https://amazon-ssm-eu-west-1.s3.amazonaws.com/latest/linux_amd64/amazon-ssm-agent.rpm'
  owner 'root'
  group 'root'
  mode '0644'
end

ruby_block 'rss_before' do
  block do
    Chef::Log.info "RSS before: #{GetProcessMem.new.mb}"
    Chef::Log.info "GC.stat before: #{GC.stat}"
  end
end

yum_package package

ruby_block 'rss_after' do
  block do
    GC.start
    Chef::Log.info "RSS after: #{GetProcessMem.new.mb}"
    Chef::Log.info "GC.stat after: #{GC.stat}"
  end
end

To invoke:
\time -v chef-apply ssm.rb --force-logger | uniq

The "\time -v" bit (GNU time) is optional. Piping through uniq is also optional, although you may want to get rid of the duplicate logging output (see #1695).
A second example where execute rpm replaces yum_package is also provided to observe the lack of leaky behaviour.
# ssm.rb
require 'get_process_mem'

package = "#{Chef::Config[:file_cache_path]}/amazon-ssm-agent.rpm"

remote_file package do
  source 'https://amazon-ssm-eu-west-1.s3.amazonaws.com/latest/linux_amd64/amazon-ssm-agent.rpm'
  owner 'root'
  group 'root'
  mode '0644'
end

ruby_block 'rss_before' do
  block do
    Chef::Log.info "RSS before: #{GetProcessMem.new.mb}"
    Chef::Log.info "GC.stat before: #{GC.stat}"
  end
end

execute "rpm -Uvh --oldpackage --replacepkgs #{package}"

ruby_block 'rss_after' do
  block do
    GC.start
    Chef::Log.info "RSS after: #{GetProcessMem.new.mb}"
    Chef::Log.info "GC.stat after: #{GC.stat}"
  end
end

This is the loop replication case where it eventually stops growing:
# package.rb
require 'get_process_mem'

package = "#{Chef::Config[:file_cache_path]}/amazon-ssm-agent.rpm"

ruby_block 'block' do
  block do
    GC.start
    Chef::Log.info "RSS: #{GetProcessMem.new.mb}"
  end
end

10.times do |t|
  t += 1

  yum_package "package#{t}" do
    source package
  end

  ruby_block "block#{t}" do
    block do
      GC.start
      Chef::Log.info "RSS#{t}: #{GetProcessMem.new.mb}"
    end
  end
end

Client Output
# leaky case
\time -v chef-apply ssm.rb --force-logger | uniq
[2017-04-06T15:44:56+00:00] INFO: Run List is []
[2017-04-06T15:44:56+00:00] INFO: Run List expands to []
[2017-04-06T15:44:56+00:00] INFO: Processing remote_file[/var/chef/cache/amazon-ssm-agent.rpm] action create ((chef-apply cookbook)::(chef-apply recipe) line 5)
[2017-04-06T15:44:56+00:00] INFO: Processing ruby_block[rss_before] action run ((chef-apply cookbook)::(chef-apply recipe) line 12)
[2017-04-06T15:44:56+00:00] INFO: RSS before: 76.828125
[2017-04-06T15:44:56+00:00] INFO: GC.stat before: {:count=>33, :heap_allocated_pages=>748, :heap_sorted_length=>761, :heap_allocatable_pages=>12, :heap_available_slots=>304885, :heap_live_slots=>295241, :heap_free_slots=>9644, :heap_final_slots=>0, :heap_marked_slots=>139986, :heap_swept_slots=>13402, :heap_eden_pages=>748, :heap_tomb_pages=>0, :total_allocated_pages=>748, :total_freed_pages=>0, :total_allocated_objects=>1430013, :total_freed_objects=>1134772, :malloc_increase_bytes=>5154344, :malloc_increase_bytes_limit=>23492560, :minor_gc_count=>27, :major_gc_count=>6, :remembered_wb_unprotected_objects=>2015, :remembered_wb_unprotected_objects_limit=>3422, :old_objects=>122454, :old_objects_limit=>204972, :oldmalloc_increase_bytes=>23583176, :oldmalloc_increase_bytes_limit=>20132659}
[2017-04-06T15:44:56+00:00] INFO: ruby_block[rss_before] called
[2017-04-06T15:44:56+00:00] INFO: Processing yum_package[/var/chef/cache/amazon-ssm-agent.rpm] action install ((chef-apply cookbook)::(chef-apply recipe) line 19)
[2017-04-06T15:45:07+00:00] INFO: Processing ruby_block[rss_after] action run ((chef-apply cookbook)::(chef-apply recipe) line 22)
[2017-04-06T15:45:07+00:00] INFO: RSS after:<PHONE_NUMBER>5
[2017-04-06T15:45:07+00:00] INFO: GC.stat after: {:count=>51, :heap_allocated_pages=>6981, :heap_sorted_length=>6982, :heap_allocatable_pages=>4, :heap_available_slots=>2845493, :heap_live_slots=>1810038, :heap_free_slots=>1035455, :heap_final_slots=>0, :heap_marked_slots=>1809970, :heap_swept_slots=>1035523, :heap_eden_pages=>6977, :heap_tomb_pages=>4, :total_allocated_pages=>6981, :total_freed_pages=>0, :total_allocated_objects=>10018957, :total_freed_objects=>8208919, :malloc_increase_bytes=>1984, :malloc_increase_bytes_limit=>24355549, :minor_gc_count=>40, :major_gc_count=>11, :remembered_wb_unprotected_objects=>2067, :remembered_wb_unprotected_objects_limit=>4134, :old_objects=>1806292, :old_objects_limit=>3612584, :oldmalloc_increase_bytes=>2432, :oldmalloc_increase_bytes_limit=>22319355}
[2017-04-06T15:45:07+00:00] INFO: ruby_block[rss_after] called
	Command being timed: "chef-apply ssm.rb --force-logger"
	User time (seconds): 11.78
	System time (seconds): 0.52
	Percent of CPU this job got: 90%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:13.55
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 240336
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 255813
	Voluntary context switches: 3755
	Involuntary context switches: 4474
	Swaps: 0
	File system inputs: 0
	File system outputs: 2840
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

# non-leaky case
\time -v chef-apply ssm.rb --force-logger | uniq
[2017-04-06T15:45:28+00:00] INFO: Run List is []
[2017-04-06T15:45:28+00:00] INFO: Run List expands to []
[2017-04-06T15:45:28+00:00] INFO: Processing remote_file[/var/chef/cache/amazon-ssm-agent.rpm] action create ((chef-apply cookbook)::(chef-apply recipe) line 5)
[2017-04-06T15:45:28+00:00] INFO: Processing ruby_block[rss_before] action run ((chef-apply cookbook)::(chef-apply recipe) line 12)
[2017-04-06T15:45:28+00:00] INFO: RSS before: 77.0703125
[2017-04-06T15:45:28+00:00] INFO: GC.stat before: {:count=>33, :heap_allocated_pages=>748, :heap_sorted_length=>761, :heap_allocatable_pages=>12, :heap_available_slots=>304877, :heap_live_slots=>295312, :heap_free_slots=>9565, :heap_final_slots=>0, :heap_marked_slots=>139988, :heap_swept_slots=>13360, :heap_eden_pages=>748, :heap_tomb_pages=>0, :total_allocated_pages=>748, :total_freed_pages=>0, :total_allocated_objects=>1430151, :total_freed_objects=>1134839, :malloc_increase_bytes=>5281528, :malloc_increase_bytes_limit=>23494240, :minor_gc_count=>27, :major_gc_count=>6, :remembered_wb_unprotected_objects=>2015, :remembered_wb_unprotected_objects_limit=>3422, :old_objects=>122451, :old_objects_limit=>204972, :oldmalloc_increase_bytes=>23819840, :oldmalloc_increase_bytes_limit=>20132659}
[2017-04-06T15:45:28+00:00] INFO: ruby_block[rss_before] called
[2017-04-06T15:45:28+00:00] INFO: Processing execute[rpm -Uvh --oldpackage --replacepkgs /var/chef/cache/amazon-ssm-agent.rpm] action run ((chef-apply cookbook)::(chef-apply recipe) line 20)
[2017-04-06T15:45:34+00:00] INFO: execute[rpm -Uvh --oldpackage --replacepkgs /var/chef/cache/amazon-ssm-agent.rpm] ran successfully
[2017-04-06T15:45:34+00:00] INFO: Processing ruby_block[rss_after] action run ((chef-apply cookbook)::(chef-apply recipe) line 22)
[2017-04-06T15:45:34+00:00] INFO: RSS after: 77.30859375
[2017-04-06T15:45:34+00:00] INFO: GC.stat after: {:count=>34, :heap_allocated_pages=>748, :heap_sorted_length=>761, :heap_allocatable_pages=>18, :heap_available_slots=>304877, :heap_live_slots=>140066, :heap_free_slots=>164811, :heap_final_slots=>0, :heap_marked_slots=>139998, :heap_swept_slots=>164893, :heap_eden_pages=>730, :heap_tomb_pages=>18, :total_allocated_pages=>748, :total_freed_pages=>0, :total_allocated_objects=>1431623, :total_freed_objects=>1291557, :malloc_increase_bytes=>1984, :malloc_increase_bytes_limit=>23024355, :minor_gc_count=>27, :major_gc_count=>7, :remembered_wb_unprotected_objects=>2019, :remembered_wb_unprotected_objects_limit=>4038, :old_objects=>131109, :old_objects_limit=>262218, :oldmalloc_increase_bytes=>6096296, :oldmalloc_increase_bytes_limit=>20132659}
[2017-04-06T15:45:34+00:00] INFO: ruby_block[rss_after] called
	Command being timed: "chef-apply ssm.rb --force-logger"
	User time (seconds): 1.95
	System time (seconds): 0.25
	Percent of CPU this job got: 26%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:08.34
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 79596
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 80542
	Voluntary context switches: 2572
	Involuntary context switches: 1531
	Swaps: 0
	File system inputs: 0
	File system outputs: 36536
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

# loop case
chef-apply package.rb --force-logger | grep RSS | uniq
[2017-04-06T16:08:21+00:00] INFO: RSS: 71.21484375
[2017-04-06T16:08:31+00:00] INFO: RSS1:<PHONE_NUMBER>
[2017-04-06T16:08:41+00:00] INFO: RSS2: 313.21875
[2017-04-06T16:08:52+00:00] INFO: RSS3:<PHONE_NUMBER>5
[2017-04-06T16:09:01+00:00] INFO: RSS4:<PHONE_NUMBER>5
[2017-04-06T16:09:12+00:00] INFO: RSS5: 374.28125
[2017-04-06T16:09:22+00:00] INFO: RSS6:<PHONE_NUMBER>5
[2017-04-06T16:09:31+00:00] INFO: RSS7: 371.734375
[2017-04-06T16:09:41+00:00] INFO: RSS8:<PHONE_NUMBER>
[2017-04-06T16:09:50+00:00] INFO: RSS9:<PHONE_NUMBER>
[2017-04-06T16:09:59+00:00] INFO: RSS10:<PHONE_NUMBER>

# few runs of the chef-client for yum_package[logrotate]
# this is part of an actual cookbook, running within an actual run_list
# the logging is made by a custom built tool to detect these leaky behaviours
# by hooking into Chef::Resource#run_action with bit of a monkey patch
# GC.start is invoked before taking any "after" measurement
# most resources of the 227 in the run_list don't modify the RSS size

# using a custom built Ruby 2.3.1 with the ruby-prof patches for memory profiling with the chef gem
run_action [:upgrade] on yum_package[logrotate]; before:<PHONE_NUMBER> MiB - after:<PHONE_NUMBER>5 MiB
run_action [:upgrade] on yum_package[logrotate]; before:<PHONE_NUMBER> MiB - after:<PHONE_NUMBER> MiB

# using the upstream Chef client from the Omnibus package
run_action [:upgrade] on yum_package[logrotate]; before: 150.84375 MiB - after:<PHONE_NUMBER>5 MiB
run_action [:upgrade] on yum_package[logrotate]; before:<PHONE_NUMBER> MiB - after:<PHONE_NUMBER>5 MiB

Stacktrace
Not applicable.

so, i'm going to rewrite the yum_package resource in the same form as the dnf_package resource and burn it all.
this also doesn't look like a memory leak.
the yum_provider has a yum cache here:
https://github.com/chef/chef/blob/master/lib/chef/provider/package/yum/yum_cache.rb
that is basically yum list dumped into memory along with all the provides stuff, so its huge.  it puts the ruby VM under a lot of memory pressure, so the fact that it grows the RSS a lot the first time a yum_provider is encountered is expected.  the further expansion as the ruby VM comes under additional memory pressure and starts expanding its heap is also expected behavior of the ruby VM.  its dynamic so as it comes under more pressure it makes decisions about expanding vs. garbage collecting and it starts expanding more in order to reduce the GC load until it finds a balance eventually.
the dnf_package provider goes a different direction and uses a python process which serves the same function as the in-memory yum_cache but re-uses all the python code.  then in order to re-populate the cache after installing software we just nuke the python process and restart it (flushing memory and giving it back).
so, dunno what to tell you, you've clearly done a bunch of work, but you're just adding ammunition to the fire that we need to burn it all.

at one point i tried doing this in order to setup some tweakables to force the RSS to grow slower and force more GCs, but which produces a tradeoff between RSS and CPU consumption and runtime:
https://github.com/chef/appbundler/pull/25/files
that didn't even work at all, but you can play around with those settings, i think taking the 1.2's down to 1.05 or something severely restricts the growth of RSS, but you'd have to do that in a wrapper around chef-client, you can't tweak it from within the running process.

Also the purpose of the yum_cache is to make chef runs faster.  While rpm -qa is pretty quick, forking off a yum command to query the yum cache to determine the available version is horribly slow (order of dozens of minutes of additional runtime on chef-client runs that do a lot of yum_package--so it could turn 1-2 minute runs into 10+ minute runs which becomes quickly unacceptable for most users).

This got me thinking about possible solutions. I guess the main keywords are avoiding shelling out as much as possible to avoid paying the shell out penalty. Came up with few:

slurp the output of rpm -qa into a data structure and use it as cache (memory - time trade-off). Don't know how much is this going to improve the memory usage issues, but probably it'll read better than little south of 400 MiB while it's a start-up cost only judging by the typical length of the output. Don't know how yum_cache deals with this though
use something like ruby-rpm to query the RPM database directly. ruby-rpm itself is unmaintained and GPLv2 which is problematic for Chef's license. ruby-rpm-ffi is pretty much alpha stage
query YumDB directly which is available since yum 3.2.26 (approximately 2010). This is probably the best of the bunch as it exposes the database as a filesystem structure. I think most, if not all, of the required information may be extracted from the file name: http://yum.baseurl.org/wiki/YumDB

I'm thinking whether I'm missing something relatively obvious as you've much more experience with this than I do.
I was looking upon your comments on the #4894 pull and have used the same set of packages to do few tests. Bear in mind, my netbook isn't the fastest of the bunch.
For each I have done two runs. The first on a cold container (no packages installed) and the second of a container which already has everything installed.
cat packages.sh
#!/bin/sh

for i in strace lsof tcpdump git dmidecode ltrace bc zsh vim-minimal systemtap telnet mtr subversion nmap gdb iptraf-ng tmux zlib-devel wget postfix sysstat
do
  yum -y -q install $i
done

\time -v ./packages.sh
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
install-info: No such file or directory for /usr/share/info/nettle.info
install-info: No such file or directory for /usr/share/info/wget.info.gz
	Command being timed: "./packages.sh"
	User time (seconds): 84.14
	System time (seconds): 55.29
	Percent of CPU this job got: 73%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 3:10.69
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 87772
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 621
	Minor (reclaiming a frame) page faults: 1805454
	Voluntary context switches: 49999
	Involuntary context switches: 23635
	Swaps: 0
	File system inputs: 320176
	File system outputs: 868968
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

\time -v ./packages.sh
Package strace-4.8-11.el7.x86_64 already installed and latest version
Package lsof-4.87-4.el7.x86_64 already installed and latest version
Package 14:tcpdump-4.5.1-3.el7.x86_64 already installed and latest version
Package git-<IP_ADDRESS>-6.el7_2.1.x86_64 already installed and latest version
Package 1:dmidecode-3.0-2.el7.x86_64 already installed and latest version
Package ltrace-0.7.91-14.el7.x86_64 already installed and latest version
Package bc-1.06.95-13.el7.x86_64 already installed and latest version
Package zsh-5.0.2-25.el7_3.1.x86_64 already installed and latest version
Package 2:vim-minimal-7.4.160-1.el7_3.1.x86_64 already installed and latest version
Package systemtap-3.0-7.el7.x86_64 already installed and latest version
Package 1:telnet-0.17-60.el7.x86_64 already installed and latest version
Package 2:mtr-0.85-7.el7.x86_64 already installed and latest version
Package subversion-1.7.14-10.el7.x86_64 already installed and latest version
Package 2:nmap-6.40-7.el7.x86_64 already installed and latest version
Package gdb-7.6.1-94.el7.x86_64 already installed and latest version
Package iptraf-ng-1.1.4-6.el7.x86_64 already installed and latest version
Package tmux-1.8-4.el7.x86_64 already installed and latest version
Package zlib-devel-1.2.7-17.el7.x86_64 already installed and latest version
Package wget-1.14-13.el7.x86_64 already installed and latest version
Package 2:postfix-2.10.1-6.el7.x86_64 already installed and latest version
Package sysstat-10.1.5-11.el7.x86_64 already installed and latest version
	Command being timed: "./packages.sh"
	User time (seconds): 19.84
	System time (seconds): 1.84
	Percent of CPU this job got: 99%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:21.72
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 39256
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 137069
	Voluntary context switches: 51
	Involuntary context switches: 381
	Swaps: 0
	File system inputs: 88
	File system outputs: 3032
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

Now that I've established a baseline, this is what I came up with for YumDB:
cat packages.rb
#!/opt/chef/embedded/bin/ruby

def install(pkg)
  if Dir["/var/lib/yum/yumdb/#{pkg[0]}/*-#{pkg}-*"].empty?
    `yum -y -q install #{pkg}`
  end
end

%w(strace lsof tcpdump git dmidecode ltrace bc zsh vim-minimal systemtap telnet mtr subversion nmap gdb iptraf-ng tmux zlib-devel wget postfix sysstat).each do |pkg|
  install pkg
end

\time -v ./packages.rb
	Command being timed: "./packages.rb"
	User time (seconds): 82.64
	System time (seconds): 54.70
	Percent of CPU this job got: 68%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 3:21.84
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 88288
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 524
	Minor (reclaiming a frame) page faults: 1793906
	Voluntary context switches: 51248
	Involuntary context switches: 21007
	Swaps: 0
	File system inputs: 287280
	File system outputs: 867928
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

\time -v ./packages.rb
	Command being timed: "./packages.rb"
	User time (seconds): 0.18
	System time (seconds): 0.02
	Percent of CPU this job got: 62%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.32
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 9024
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 28
	Minor (reclaiming a frame) page faults: 1279
	Voluntary context switches: 213
	Involuntary context switches: 44
	Swaps: 0
	File system inputs: 6720
	File system outputs: 0
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

While the initial run is very comparable to invoking yum in a shell loop, the second run which leverages YumDB is blazing fast as it completely avoids shelling out.
For comparison, this is the startup/teardown cost of Ruby itself:
cat empty.rb
#!/opt/chef/embedded/bin/ruby

\time -v ./empty.rb
	Command being timed: "./empty.rb"
	User time (seconds): 0.15
	System time (seconds): 0.01
	Percent of CPU this job got: 97%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.16
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 9032
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 1
	Minor (reclaiming a frame) page faults: 1273
	Voluntary context switches: 4
	Involuntary context switches: 7
	Swaps: 0
	File system inputs: 24
	File system outputs: 0
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

The version, release, and arch is available in the file name which is a fairly trivial string processing problem:
/opt/chef/embedded/bin/pry
[1] pry(main)> Dir['/var/lib/yum/yumdb/c/*-chef-*']
=> ["/var/lib/yum/yumdb/c/c76cd02f0ed2cc2eea4d8633a1628cd8d8e65bbe-chef-12.19.36-1.el7-x86_64"]

The only thing I see missing here is the package epoch which TBH, I don't know how often it is used in practice over the lifetime of a distribution, however it may be used by private repositories. I'll keep poking at YumDB to see whether this may be easily obtained.

Actually, I see now the epoch is bit more problematic.
rpm -qa | wc -l
259
rpm -qai | grep Epoch | wc -l
32

31 out of those are CentOS packages. 1 is custom built twice with no epoch and with epoch = 1 for testing purposes. The obvious bit I was missing from YumDB is the lack of epoch which actually proves to be important and it renders useless the above solution.

Wrapped up a crappy algorithm to save the interesting bits of rpm -qai. There's a startup penalty involved (roughly one second), it uses some memory (around 12 MiB), but nothing like what I've seen with the current yum_cache. Updating the data structure should be fairly easy when a package is changed, although it involves a trip to rpm -qi $pkg.
cat packages.rb
#!/opt/chef/embedded/bin/ruby

Encoding.default_external = Encoding::UTF_8

`rpm -qai > /tmp/rpm.txt`

rpm_cache = {}

name = nil
version = nil
release = nil
arch = nil
epoch = nil

File.readlines('/tmp/rpm.txt').each do |line|
  k, v = line.split(':')
  next if k.nil? || v.nil?
  k = k.strip
  v = v.strip

  if k == 'Name'
    if name.nil?
      name = v
      next
    else
      rpm_cache[name] = {
        'version' => version,
        'release' => release,
        'arch' => arch,
        'epoch' => epoch
      }

      name = v
      version = nil
      release = nil
      arch = nil
      epoch = nil
    end
  end

  version = v if k == 'Version'
  release = v if k == 'Release'
  arch = v if k == 'Architecture'
  epoch = v if k == 'Epoch'
end

rpm_cache[name] = {
  'version' => version,
  'release' => release,
  'arch' => arch,
  'epoch' => epoch
}

%w(strace lsof tcpdump git dmidecode ltrace bc zsh vim-minimal systemtap telnet mtr subversion nmap gdb iptraf-ng tmux zlib-devel wget postfix sysstat).each do |pkg|
  if rpm_cache[pkg].nil?
    `yum -y -q install #{pkg}`
  end
end

rpm_cache roughly looks like:
{
  # [...]
  "systemtap"=>
    {"version"=>"3.0", "release"=>"7.el7", "arch"=>"x86_64", "epoch"=>nil},
  "mtr"=>
    {"version"=>"0.85", "release"=>"7.el7", "arch"=>"x86_64", "epoch"=>"2"}
}

The profiling on a primed container looks promising:
\time -v ./packages.rb
	Command being timed: "./packages.rb"
	User time (seconds): 1.07
	System time (seconds): 0.07
	Percent of CPU this job got: 99%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.14
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 20976
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 6011
	Voluntary context switches: 8
	Involuntary context switches: 11
	Swaps: 0
	File system inputs: 0
	File system outputs: 480
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

Still wondering whether I'm missing something here.

So what you're doing with rpm -qa caching is just a stripped down version of the yum_cache, but you're going to need to build out the support for "whatprovides" syntax as well (ie.  yum_package "/usr/bin/perl" and i think we can or should even be able to do yum_package "/usr/bin/perl >= 5.12.0".  If you're doing that in ruby you also need all the support for doing version comparisons and handling the nerva (name, epoch, release, version, architecture) correctly and all that.
What i found simpler with the dnf_provider was to have a dnf_provider.py that i just do some stripped-down-to-the-bones rpc to in order to ask python what provides a given "package" and what is installed, and that is plenty fast and has the advantage that we don't have to maintain the same logic in ruby as exists in python.
There's a perf hit because the way i implemented it, after doing an install i just kill the python process to "flush the cache".  I went down the road of trying to optimize this case, but it turned out to not actually provide that much of a perf boost, the perf boost would only be during runs that actually do a lot of work (likely only first provisioning, not any idempotent case) and i was getting actually leaky problems trying to poke the python objects to make them refresh the case without recycling the python process (i also went the other way and tried having the python process doing the installation work so that presumably the cache would update since the same process holding the cache was doing the installation...  similarly didn't help much).
I'll have to take a look at YumDB since I didn't know about that, but we also considered ruby ffi bindings to dnf as well.

I think the "whatprovides" functionality is the culprit here. The memory usage is blowing up when the yum_package resource installs an RPM file i.e receives a disk path as argument instead of package name. The rest of the yum_package resources barely bump the memory usage.
I'll close this one down as it's unlikely to change until dnf takes over. I'll remember to avoid using yum_package with RPM files to avoid this particular issue. The only winning move is not to play.

Well, I'll take a stab at porting over the dnf_package approach to the yum_package provider sometime around chef 13.1-13.2 kind of timeframe, which may make things considerably better both from runtime, memory utilization, and bugginess perspectives.

@lamont-granquist Is this something worth keeping open? #5213 this was a red herring but still a problem. We would love to see memory usage go down by querying yumdb directly.

so in testing the work in #6540 the results which are relevant to this thread are:
# bundle exec ./apply.rb
root        13  0.0  0.0      0     0 ?        S<   Oct27   0:00 [khelper]
root     11122  0.0  0.0 110508     8 ?        S    Oct27   0:00 /sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-enp0s3.pid -lf /var/lib/NetworkManager/dhclient-5583cf67-b4c4-49f2-bd3d-b4aabcb87f28-enp0s3.lease -cf /var/lib/NetworkManager/dhclient-enp0s3.conf enp0s3
root     12398 34.0 11.6<PHONE_NUMBER>6 pts/0    Sl+  23:35   0:01 ruby /opt/chef/embedded/lib/ruby/gems/2.4.0/bin/chef-apply ./apply.rb
root     12603  0.0  0.2 113132  1376 pts/0    S+   23:35   0:00 sh -c ps xauwww | egrep '(chef-apply|helper)'
root     12605  0.0  0.1 112664   980 pts/0    S+   23:35   0:00 grep -E (chef-apply|helper)
RSS after: 57.1953125
GC.stat after: {:count=>55, :heap_allocated_pages=>548, :heap_sorted_length=>548, :heap_allocatable_pages=>0, :heap_available_slots=>223379, :heap_live_slots=>161712, :heap_free_slots=>61667, :heap_final_slots=>0, :heap_marked_slots=>161574, :heap_eden_pages=>548, :heap_tomb_pages=>0, :total_allocated_pages=>548, :total_freed_pages=>0, :total_allocated_objects=>1453729, :total_freed_objects=>1292017, :malloc_increase_bytes=>2032, :malloc_increase_bytes_limit=>16777216, :minor_gc_count=>47, :major_gc_count=>8, :remembered_wb_unprotected_objects=>837, :remembered_wb_unprotected_objects_limit=>1674, :old_objects=>150769, :old_objects_limit=>301538, :oldmalloc_increase_bytes=>1606648, :oldmalloc_increase_bytes_limit=>16777216}
Recipe: (chef-apply cookbook)::(chef-apply recipe)
  * yum_package[strace] action install
    - install version 0:4.12-4.el7.x86_64 of package strace
  * yum_package[lsof] action install
    - install version 0:4.87-4.el7.x86_64 of package lsof
  * yum_package[tcpdump] action install
    - install version 14:4.9.0-5.el7.x86_64 of package tcpdump
  * yum_package[git] action install (up to date)
  * yum_package[dmidecode] action install
    - install version 1:3.0-5.el7.x86_64 of package dmidecode
  * yum_package[ltrace] action install
    - install version 0:0.7.91-14.el7.x86_64 of package ltrace
  * yum_package[bc] action install
    - install version 0:1.06.95-13.el7.x86_64 of package bc
  * yum_package[zsh] action install (up to date)
  * yum_package[systemtap] action install
    - install version 0:3.1-3.el7.x86_64 of package systemtap
  * yum_package[telnet] action install
    - install version 1:0.17-64.el7.x86_64 of package telnet
  * yum_package[mtr] action install
    - install version 2:0.85-7.el7.x86_64 of package mtr
  * yum_package[subversion] action install
    - install version 0:1.7.14-11.el7_4.x86_64 of package subversion
  * yum_package[nmap] action install
    - install version 2:6.40-7.el7.x86_64 of package nmap
  * yum_package[gdb] action install
    - install version 0:7.6.1-100.el7.x86_64 of package gdb
  * yum_package[iptraf-ng] action install
    - install version 0:1.1.4-6.el7.x86_64 of package iptraf-ng
  * yum_package[tmux] action install
    - install version 0:1.8-4.el7.x86_64 of package tmux
  * yum_package[zlib-devel] action install
    - install version 0:1.2.7-17.el7.x86_64 of package zlib-devel
  * yum_package[wget] action install
    - install version 0:1.14-15.el7_4.1.x86_64 of package wget
  * yum_package[postfix] action install (up to date)
  * yum_package[sysstat] action install
    - install version 0:10.1.5-12.el7.x86_64 of package sysstat
  * yum_package[sudo] action install (up to date)
  0.100000   0.090000  11.870000 ( 21.560122)
root        13  0.0  0.0      0     0 ?        S<   Oct27   0:00 [khelper]
root     11122  0.0  0.0 110508     8 ?        S    Oct27   0:00 /sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-enp0s3.pid -lf /var/lib/NetworkManager/dhclient-5583cf67-b4c4-49f2-bd3d-b4aabcb87f28-enp0s3.lease -cf /var/lib/NetworkManager/dhclient-enp0s3.conf enp0s3
root     12398  4.9 12.4<PHONE_NUMBER>6 pts/0    Sl+  23:35   0:01 ruby /opt/chef/embedded/lib/ruby/gems/2.4.0/bin/chef-apply ./apply.rb
root     12882  6.0  4.3<PHONE_NUMBER>2 pts/0    S+   23:35   0:00 /bin/python /home/vagrant/chef/lib/chef/provider/package/yum/yum_helper.py 11 10
root     12885  0.0  0.2 113132  1376 pts/0    S+   23:35   0:00 sh -c ps xauwww | egrep '(chef-apply|helper)'
root     12887  0.0  0.1 112664   976 pts/0    S+   23:35   0:00 grep -E (chef-apply|helper)
RSS after: 61.01171875
GC.stat after: {:count=>57, :heap_allocated_pages=>548, :heap_sorted_length=>548, :heap_allocatable_pages=>0, :heap_available_slots=>223379, :heap_live_slots=>164358, :heap_free_slots=>59021, :heap_final_slots=>0, :heap_marked_slots=>164305, :heap_eden_pages=>548, :heap_tomb_pages=>0, :total_allocated_pages=>548, :total_freed_pages=>0, :total_allocated_objects=>1523952, :total_freed_objects=>1359594, :malloc_increase_bytes=>1144, :malloc_increase_bytes_limit=>16777216, :minor_gc_count=>48, :major_gc_count=>9, :remembered_wb_unprotected_objects=>860, :remembered_wb_unprotected_objects_limit=>1720, :old_objects=>162744, :old_objects_limit=>325488, :oldmalloc_increase_bytes=>1600, :oldmalloc_increase_bytes_limit=>16777216}

that's the new yum provider code.
# bundle exec ./apply.rb
root        13  0.0  0.0      0     0 ?        S<   Oct27   0:00 [khelper]
root     11122  0.0  0.0 110508     8 ?        S    Oct27   0:00 /sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-enp0s3.pid -lf /var/lib/NetworkManager/dhclient-5583cf67-b4c4-49f2-bd3d-b4aabcb87f28-enp0s3.lease -cf /var/lib/NetworkManager/dhclient-enp0s3.conf enp0s3
root     16882 48.0 11.7<PHONE_NUMBER>4 pts/0    Sl+  23:48   0:00 ruby /opt/chef/embedded/lib/ruby/gems/2.4.0/bin/chef-apply ./apply.rb
root     17183  0.0  0.2 113132  1376 pts/0    S+   23:48   0:00 sh -c ps xauwww | egrep '(chef-apply|helper)'
root     17185  0.0  0.1 112664   980 pts/0    S+   23:48   0:00 grep -E (chef-apply|helper)
RSS after: 57.62890625
GC.stat after: {:count=>51, :heap_allocated_pages=>581, :heap_sorted_length=>581, :heap_allocatable_pages=>0, :heap_available_slots=>236809, :heap_live_slots=>156867, :heap_free_slots=>79942, :heap_final_slots=>0, :heap_marked_slots=>156729, :heap_eden_pages=>581, :heap_tomb_pages=>0, :total_allocated_pages=>581, :total_freed_pages=>0, :total_allocated_objects=>1363501, :total_freed_objects=>1206634, :malloc_increase_bytes=>2032, :malloc_increase_bytes_limit=>16777216, :minor_gc_count=>44, :major_gc_count=>7, :remembered_wb_unprotected_objects=>818, :remembered_wb_unprotected_objects_limit=>1636, :old_objects=>145810, :old_objects_limit=>291620, :oldmalloc_increase_bytes=>2488, :oldmalloc_increase_bytes_limit=>16777216}
Recipe: (chef-apply cookbook)::(chef-apply recipe)
  * yum_package[strace] action install
    - install version 4.12-4.el7 of package strace
  * yum_package[lsof] action install
    - install version 4.87-4.el7 of package lsof
  * yum_package[tcpdump] action install
    - install version 4.9.0-5.el7 of package tcpdump
  * yum_package[git] action install (up to date)
  * yum_package[dmidecode] action install
    - install version 3.0-5.el7 of package dmidecode
  * yum_package[ltrace] action install
    - install version 0.7.91-14.el7 of package ltrace
  * yum_package[bc] action install
    - install version 1.06.95-13.el7 of package bc
  * yum_package[zsh] action install (up to date)
  * yum_package[systemtap] action install
    - install version 3.1-3.el7 of package systemtap
  * yum_package[telnet] action install
    - install version 0.17-64.el7 of package telnet
  * yum_package[mtr] action install
    - install version 0.85-7.el7 of package mtr
  * yum_package[subversion] action install
    - install version 1.7.14-11.el7_4 of package subversion
  * yum_package[nmap] action install
    - install version 6.40-7.el7 of package nmap
  * yum_package[gdb] action install
    - install version 7.6.1-100.el7 of package gdb
  * yum_package[iptraf-ng] action install
    - install version 1.1.4-6.el7 of package iptraf-ng
  * yum_package[tmux] action install
    - install version 1.8-4.el7 of package tmux
  * yum_package[zlib-devel] action install
    - install version 1.2.7-17.el7 of package zlib-devel
  * yum_package[wget] action install
    - install version 1.14-15.el7_4.1 of package wget
  * yum_package[postfix] action install (up to date)
  * yum_package[sysstat] action install
    - install version 10.1.5-12.el7 of package sysstat
  * yum_package[sudo] action install (up to date)
  0.520000   0.060000  12.830000 ( 23.822429)
root        13  0.0  0.0      0     0 ?        S<   Oct27   0:00 [khelper]
root     11122  0.0  0.0 110508     8 ?        S    Oct27   0:00 /sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-enp0s3.pid -lf /var/lib/NetworkManager/dhclient-5583cf67-b4c4-49f2-bd3d-b4aabcb87f28-enp0s3.lease -cf /var/lib/NetworkManager/dhclient-enp0s3.conf enp0s3
root     16882  6.0 17.1<PHONE_NUMBER>8 pts/0    Sl+  23:48   0:01 ruby /opt/chef/embedded/lib/ruby/gems/2.4.0/bin/chef-apply ./apply.rb
root     18203  0.0  0.2 113132  1376 pts/0    S+   23:48   0:00 sh -c ps xauwww | egrep '(chef-apply|helper)'
root     18205  0.0  0.1 112664   976 pts/0    S+   23:48   0:00 grep -E (chef-apply|helper)
RSS after: 83.83984375
GC.stat after: {:count=>64, :heap_allocated_pages=>1712, :heap_sorted_length=>1712, :heap_allocatable_pages=>0, :heap_available_slots=>697803, :heap_live_slots=>432540, :heap_free_slots=>265263, :heap_final_slots=>0, :heap_marked_slots=>432487, :heap_eden_pages=>1712, :heap_tomb_pages=>0, :total_allocated_pages=>1712, :total_freed_pages=>0, :total_allocated_objects=>2895749, :total_freed_objects=>2463209, :malloc_increase_bytes=>1144, :malloc_increase_bytes_limit=>16777216, :minor_gc_count=>54, :major_gc_count=>10, :remembered_wb_unprotected_objects=>844, :remembered_wb_unprotected_objects_limit=>1688, :old_objects=>431066, :old_objects_limit=>862132, :oldmalloc_increase_bytes=>1600, :oldmalloc_increase_bytes_limit=>16777216}

that's the old code.

so more concisely master with yum-dump.py currently has this kind of impact:
root     16882  6.0 17.1<PHONE_NUMBER>8 pts/0    Sl+  23:48   0:01 ruby /opt/chef/embedded/lib/ruby/gems/2.4.0/bin/chef-apply ./apply.rb
root 30303 57.0 15.8<PHONE_NUMBER>0 ? Rs 23:50 0:00 /usr/bin/python /home/vagrant/chef/lib/chef/provider/package/yum/yum-dump.py --options --installed-provides --yum-lock-timeout 30

while the rewritten yum provider behaves like:
root     12398  4.9 12.4<PHONE_NUMBER>6 pts/0    Sl+  23:35   0:01 ruby /opt/chef/embedded/lib/ruby/gems/2.4.0/bin/chef-apply ./apply.rb
root     12882  6.0  4.3<PHONE_NUMBER>2 pts/0    S+   23:35   0:00 /bin/python /home/vagrant/chef/lib/chef/provider/package/yum/yum_helper.py 11 10

That 83MB of RSS should be stable, and likely double-counts pages like glibc between both of them.  The 164MB of RSS in the current master case is transient at the peak of yum-dump.py running.  So I think I can argue that this cuts down peak memory pressure on the box by 50% and 80MB.
The way the VSZ jumps up to 400MB in the new code bugs me quite a bit and I can't explain it, although theoretically that is just all accounting and doesn't impact memory if it doesn't show up in RSS, but it is still odd and unexplained, and I suspect if I dig into it I'll learn horrible things I never wanted to know about Open.popen3 and Process.spawn in ruby since that is likely the difference.

Thanks for the update and your work for lowering the memory usage. Will be looking forward to this.


Update README.md

Updated path for Bugs and Feature proposal badges according to the new repository name

The issue has already fixed


node.fetch ESM import issue in getFileByURL.js

Link to reproduction
private repo
Describe the Bug
Getting the following error in Payload 2.4.0 while trying to build the types:
/.../node_modules/payload/dist/uploads/getFileByURL.js:121
undefined
      ^

Error [ERR_REQUIRE_ESM]: require() of ES Module /.../node_modules/node-fetch/src/index.js from /.../node_modules/payload/dist/uploads/getFileByURL.js not supported.
Instead change the require of index.js in /.../node_modules/payload/dist/uploads/getFileByURL.js to a dynamic import() which is available in all CommonJS modules.
    at Object.newLoader [as .js] (/.../node_modules/pirates/lib/index.js:121:7)
    at Object.<anonymous> (/.../node_modules/payload/dist/uploads/getFileByURL.js:11:59)
    at Object.newLoader [as .js] (/.../node_modules/pirates/lib/index.js:121:7) {
  code: 'ERR_REQUIRE_ESM'
}

Node.js v20.10.0

npm ERR! Lifecycle script `payload:types` failed with error:

To Reproduce
npm run payload:types is failing after upgrading to 2.4.0. All packages are the latest versions.
Payload Version
2.4.0
Adapters and Plugins
db-mongodb, bundler-webpack, live-preview

According to node-fetch the following should be used on CJS:
// mod.cjs
const fetch = (...args) => import('node-fetch').then(({default: fetch}) => fetch(...args));

https://www.npmjs.com/package/node-fetch
The CJS and ESM syntax is different and TypeScript code gets here transpiled to CJS syntax which is wrong.
General question: why not using Node.js' fetch on 20.x?

Fails here: https://github.com/payloadcms/payload/blob/48f1299fcba3e3811c6a7f31499f238537f9a5e3/packages/payload/src/uploads/getFileByURL.ts#L1

Monkey patched the transpiled getFileByURL.js file:
"use strict";
Object.defineProperty(exports, "__esModule", {
    value: true
});
Object.defineProperty(exports, "default", {
    enumerable: true,
    get: function() {
        return _default;
    }
});
//const _nodefetch = /*#__PURE__*/ _interop_require_default(require("node-fetch"));
const _path = /*#__PURE__*/ _interop_require_default(require("path"));
function _interop_require_default(obj) {
    return obj && obj.__esModule ? obj : {
        default: obj
    };
}
const getFileByURL = async (url)=>{
    if (typeof url === 'string') {
        const res = await fetch(url, {
            headers: {
                'Content-Type': 'application/json'
            },
            method: 'GET'
        });
        const data = await res.buffer();
        const name = _path.default.basename(url);
        return {
            name,
            data,
            mimetype: res.headers.get('content-type') || undefined,
            size: Number(res.headers.get('content-length')) || 0
        };
    }
};
const _default = getFileByURL;

See discussion here: https://github.com/node-fetch/node-fetch/issues/1279#issuecomment-1275933370

Having updated to Payload 2.4.0, I'm currently unable to run npm dev.
I'm currently on Node 20.8.0.
[nodemon] 2.0.22
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: ts
[nodemon] starting `ts-node src/server.ts -- -I`

/data/projects/ac/payload/node_modules/ts-node/dist/index.js:701
            return old(m, filename);
                   ^
Error [ERR_REQUIRE_ESM]: require() of ES Module /data/projects/ac/payload/node_modules/node-fetch/src/index.js from /data/projects/ac/payload/node_modules/payload/dist/uploads/getFileByURL.js not supported.
Instead change the require of index.js in /data/projects/ac/payload/node_modules/payload/dist/uploads/getFileByURL.js to a dynamic import() which is available in all CommonJS modules.
    at require.extensions.<computed> [as .js] (/data/projects/ac/payload/node_modules/ts-node/dist/index.js:701:20)
    at Object.<anonymous> (/data/projects/ac/payload/node_modules/payload/dist/uploads/getFileByURL.js:11:59)
    at require.extensions.<computed> [as .js] (/data/projects/ac/payload/node_modules/ts-node/dist/index.js:701:20)
    at Object.<anonymous> (/data/projects/ac/payload/node_modules/payload/dist/uploads/generateFileData.js:21:62)
    at require.extensions.<computed> [as .js] (/data/projects/ac/payload/node_modules/ts-node/dist/index.js:701:20)
    at Object.<anonymous> (/data/projects/ac/payload/node_modules/payload/dist/collections/operations/create.js:22:27)
    at require.extensions.<computed> [as .js] (/data/projects/ac/payload/node_modules/ts-node/dist/index.js:701:20)
    at Object.<anonymous> (/data/projects/ac/payload/node_modules/payload/dist/collections/requestHandlers/create.js:14:56)
    at require.extensions.<computed> [as .js] (/data/projects/ac/payload/node_modules/ts-node/dist/index.js:701:20)
    at Object.<anonymous> (/data/projects/ac/payload/node_modules/payload/dist/collections/buildEndpoints.js:21:56)
    at require.extensions.<computed> [as .js] (/data/projects/ac/payload/node_modules/ts-node/dist/index.js:701:20)
    at Object.<anonymous> (/data/projects/ac/payload/node_modules/payload/dist/collections/initHTTP.js:16:64)
    at require.extensions.<computed> [as .js] (/data/projects/ac/payload/node_modules/ts-node/dist/index.js:701:20)
    at Object.<anonymous> (/data/projects/ac/payload/node_modules/payload/dist/initHTTP.js:15:58)
    at require.extensions.<computed> [as .js] (/data/projects/ac/payload/node_modules/ts-node/dist/index.js:701:20)
    at Object.<anonymous> (/data/projects/ac/payload/node_modules/payload/dist/index.js:22:19)
    at require.extensions.<computed> [as .js] (/data/projects/ac/payload/node_modules/ts-node/dist/index.js:701:20)
    at Object.<anonymous> (/data/projects/ac/payload/src/server.ts:43:33)
    at m._compile (/data/projects/ac/payload/node_modules/ts-node/dist/index.js:708:29)
    at require.extensions.<computed> [as .ts] (/data/projects/ac/payload/node_modules/ts-node/dist/index.js:710:16)
    at main (/data/projects/ac/payload/node_modules/ts-node/dist/bin.js:154:20)
    at Object.<anonymous> (/data/projects/ac/payload/node_modules/ts-node/dist/bin.js:238:5)

Hi @cbratschi @DavidOliver this has been fixed and will be out in our next release. Thanks for reporting!

Hey, I wanna start a new project but I can't because it hasn't been released yet. Is there a way I can start using payload  before the relase ?

I'd also appreciate a new release. :) Thanks for fixing.
@AhmetHuseyinDOK , you could specify v2.3.1 in your package.json file and npm install?

I checked the latest Payload version and there is still an issue left: the getExternalFile() call is silently failing:
https://github.com/payloadcms/payload/blob/8015e999cd5834f532a200ef03fd392d04b3209f/packages/payload/src/uploads/generateFileData.ts#L76
I modified the code to get the thrown exception:
Error [ERR_REQUIRE_ESM]: require() of ES Module /.../node_modules/node-fetch/src/index.js from /.../node_modules/payload/dist/uploads/getExternalFile.js not supported.
Instead change the require of index.js in /.../node_modules/payload/dist/uploads/getExternalFile.js to a dynamic import() which is available in all CommonJS modules.
    at require.extensions.<computed> [as .js] (/.../node_modules/ts-node/dist/index.js:851:20)
    at mod.require (/.../node_modules/next/dist/server/require-hook.js:65:28)
    at /.../node_modules/payload/dist/uploads/getExternalFile.js:56:109
    at async getExternalFile (/U.../node_modules/payload/dist/uploads/getExternalFile.js:56:36) {
  code: 'ERR_REQUIRE_ESM'
}

https://github.com/payloadcms/payload/blob/main/packages/payload/src/uploads/getExternalFile.ts#L15
The transpiled TypeScript code is:
const { default: fetch } = await Promise.resolve().then(()=>/*#__PURE__*/ _interop_require_wildcard(require("node-fetch")));

This is still a require() call and not a dynamic import.
Could you please pass all caught exceptions as cause to a secondary exception: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Error/cause. In that case no exceptions get lost and it's mich easier to debug such issues.

Could you please re-open and patch according to my description below.
Error Handling
First log the error on console like on this line in the same file:
https://github.com/payloadcms/payload/blob/b974a2c042a1b4c58ca9a740d5be7f16bf66177f/packages/payload/src/uploads/generateFileData.ts#L256
Then modify it here:
https://github.com/payloadcms/payload/blob/b974a2c042a1b4c58ca9a740d5be7f16bf66177f/packages/payload/src/uploads/generateFileData.ts#L84
Here is a patch for the transpiled code:
        try {
            if (url && url.startsWith('/') && !disableLocalStorage) {
                const filePath = `${staticPath}/${filename}`;
                const response = await (0, _getFileByPath.default)(filePath);
                file = response;
                overwriteExistingFiles = true;
            } else if (filename && url) {
                file = await (0, _getExternalFile.getExternalFile)({
                    req,
                    data: data
                });
                overwriteExistingFiles = true;
            }
        } catch (err) {
            console.error(err);
            throw new _errors.FileUploadError(req.t);
        }

Now we see all errors.
Better would be to pass the error to FileUploadError:
https://github.com/payloadcms/payload/blob/b974a2c042a1b4c58ca9a740d5be7f16bf66177f/packages/payload/src/errors/FileUploadError.ts#L8
class FileUploadError extends APIError {
  constructor(t?: TFunction, cause: Error) {
    super(
      t ? t('error:problemUploadingFile') : 'There was a problem while uploading the file.',
      httpStatus.BAD_REQUEST,
    )

    this.cause = cause
  }
}

node-fetch Fix
Now we see the root cause of the error:
Error [ERR_REQUIRE_ESM]: require() of ES Module /.../node_modules/node-fetch/src/index.js from /.../node_modules/payload/dist/uploads/getExternalFile.js not supported.
Instead change the require of index.js in /.../node_modules/payload/dist/uploads/getExternalFile.js to a dynamic import() which is available in all CommonJS modules.
    at require.extensions.<computed> [as .js] (/.../node_modules/ts-node/dist/index.js:851:20)
    at mod.require (/.../node_modules/next/dist/server/require-hook.js:65:28)
    at /.../node_modules/payload/dist/uploads/getExternalFile.js:61:109
    at async getExternalFile (/...node_modules/payload/dist/uploads/getExternalFile.js:61:36) {
  code: 'ERR_REQUIRE_ESM'
}

Problem occurs in transpiled code here:
https://github.com/payloadcms/payload/blob/b974a2c042a1b4c58ca9a740d5be7f16bf66177f/packages/payload/src/uploads/getExternalFile.ts#L22
This is the patch for the transpiled code (replace node-fetch by fetch):
const getExternalFile = async ({ data, req })=>{
    const { filename, url } = data;
    if (typeof url === 'string') {
        let fileURL = url;
        if (!url.startsWith('http')) {
            const baseUrl = req.get('origin') || `${req.protocol}://${req.get('host')}`;
            fileURL = `${baseUrl}${url}`;
        }
        //Note: replaced node-fetch by fetch
        //const { default: fetch } = await Promise.resolve().then(()=>/*#__PURE__*/ _interop_require_wildcard(require("node-fetch")));
        const res = await fetch(fileURL, {
            credentials: 'include',
            headers: {
                ...req.headers
            },
            method: 'GET'
        });
        if (!res.ok) throw new _errors.APIError(`Failed to fetch file from ${fileURL}`, res.status);
        //const data = await res.buffer();
        const data = Buffer.from(await res.arrayBuffer());
        return {
            name: filename,
            data,
            mimetype: res.headers.get('content-type') || undefined,
            size: Number(res.headers.get('content-length')) || 0
        };
    }
    throw new _errors.APIError('Invalid file url', 400);
};

Basically these two lines are modified:
//const { default: fetch } = await Promise.resolve().then(()=>/*#__PURE__*/ _interop_require_wildcard(require("node-fetch")));
const data = Buffer.from(await res.arrayBuffer());

@cbratschi Some fixes were pushed in this area. I'd be curious if you still see this issue in 2.14.2.
Regarding replacing node-fetch with fetch. Unfortunately, this is not possible for us to do because Payload v2 supports node versions before 18 which was when the built-in fetch api was introduced.

@denolfe Still the same. The only change is that the error is now better handled and its message is shown in a toast.
Here is the full error message:
[22:08:46] ERROR (payload): FileRetrievalError: Es gab ein Problem während des Hochladens der Datei. require() of ES Module /.../node_modules/node-fetch/src/index.js from /.../node_modules/payload/dist/uploads/getExternalFile.js not supported.
Instead change the require of index.js in /.../node_modules/payload/dist/uploads/getExternalFile.js to a dynamic import() which is available in all CommonJS modules.
    at generateFileData (/.../node_modules/payload/src/uploads/generateFileData.ts:86:15)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async updateByID (/.../node_modules/payload/src/collections/operations/updateByID.ts:146:57)
    at async updateByIDHandler (/.../node_modules/payload/src/collections/requestHandlers/updateByID.ts:43:17)

The node-fetch ESM module is still being loaded by require(). This would only work with Node's 22 experimental loader.
You could downgrade node-fetch to v2. The TypeScript transpiler converts the async import() to a require() which is not easy to solve.

I am trying to recreate this on<EMAIL_ADDRESS>how can I repro? I have cloned down the public demo repository, I am on node v20.14.0 but when running pnpm generate:types it seems to work fine for me.

@JarrodMFlesch the error happens for instance after saving resized images using non-local image storage. In that case Payload uses node-fetch to retrieve the original image data.
This error does not occur while generating Payload types.
A easy test case would be to directly import getFileByURL() and pass any URL. In previous Payload versions the exception was quietly ignored. So double check you get all exceptions.

I ran into this exact error. I am saving my images to the AWS EFS file system, and the initial upload works fine but attempting to save a cropped image throws that node-fetch error.
[17:00:43] ERROR (payload): FileRetrievalError: There was a problem while uploading the file. require() of ES Module /home/node/app/node_modules/node-fetch/src/index.js from /home/node/app/node_modules/payload/dist/uploads/getExternalFile.js not supported.
Instead change the require of index.js in /home/node/app/node_modules/payload/dist/uploads/getExternalFile.js to a dynamic import() which is available in all CommonJS modules.
    at generateFileData (/home/node/app/node_modules/payload/dist/uploads/generateFileData.js:69:23)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async updateByID (/home/node/app/node_modules/payload/dist/collections/operations/updateByID.js:108:61)
    at async updateByIDHandler (/home/node/app/node_modules/payload/dist/collections/requestHandlers/updateByID.js:41:21)

We're facing exact same issue using Google Cloud Storage and in local development where we don't have external storage connected.
Crop works fine when uploading an image but after uploading it, it throws the error mentioned above
We're on Payload 2.23.1 and Node.js 20.11.0

Any solution to this yet? I am just starting out with the website template and I am getting this same error when I try to npm run dev in local development

same issue here
><EMAIL_ADDRESS>payload-cms /Users/Jesus/Documents/WJS/pnpm-monorepo
> pnpm --filter payload-cms "generate:types"

><EMAIL_ADDRESS>generate:types /Users/Jesus/Documents/WJS/pnpm-monorepo/apps/payload-cms
> payload generate:types

node:internal/modules/esm/loader:352
      throw new ERR_REQUIRE_ESM(url, true);
            ^

Error [ERR_REQUIRE_ESM]: require() of ES Module file:///Users/Jesus/Documents/WJS/pnpm-monorepo/node_modules/.pnpm/@payloadcms+db-postgres@3.0.0-beta.123_payload@3.0.0-beta.123_graphql@16.9.0_monaco-editor<EMAIL_ADDRESS>not supported.
    at Object.transformer [as .js] (file:///Users/Jesus/Documents/WJS/pnpm-monorepo/node_modules/.pnpm/tsx@4.19.1/node_modules/tsx/dist/register-DvZgvjmQ.mjs:2:748)
    at TracingChannel.traceSync (node:diagnostics_channel:315:14) {
  code: 'ERR_REQUIRE_ESM'
}

Node.js v22.11.0

but i downgrade to 20.18.0 node version
><EMAIL_ADDRESS>generate:types /Users/Jesus/Documents/WJS/pnpm-monorepo/apps/payload-cms
> payload generate:types

[21:06:36] INFO: Compiling TS types for Collections and Globals...
[21:06:36] INFO: Types written to /Users/Jesus/Documents/WJS/pnpm-monorepo/apps/payload-cms/src/payload-types.ts

Hi @jesusvallez! Just checking—did you add "type": "module" to your package.json? Please note that non-ESM projects aren’t officially supported at the moment.
Regarding this issue overall: in Payload 3.0, we’ve moved from CommonJS to ESM, so with "type": "module" set in your package.json, this issue should no longer occur. However, if you do run into anything similar in Payload 3.0 with "type": "module" configured, please feel free to open a new issue, and we’ll address it as soon as possible.
Since our efforts are now focused on version 3.0, we won’t be backporting this fix to 2.0. If upgrading to Payload 3.0 isn’t possible yet, you could try updating to the latest Node.js version—23.0.0 or higher—which should support ESM imports from CommonJS by default.

Hi @AlessioGr! thank you very much for your reply.
I am using this template
https://github.com/payloadcms/payload/blob/beta/templates/vercel-postgres/package.json
And indeed, if I add ‘type’: ‘module’ it works correctly with version 22.


Call recording not working on Red Mi mobiles

This apps sims work good but it's not recording on RED MI device.My simple question is
Why the call is not recording on RED MI devices ?

not working on OPPO either


Fix assigned to field

Codecov Report

Merging #60 (cbaac12) into main (04ca287) will increase coverage by 0.06%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #60      +/-   ##
==========================================
+ Coverage   92.96%   93.03%   +0.06%
==========================================
  Files          20       20
  Lines        3057     3059       +2
==========================================
+ Hits         2842     2846       +4
+ Misses        215      213       -2

Flag
Coverage Δ

Apex
93.03% <100.00%> (+0.06%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...ain/default/classes/ActionPlansTriggerHandlers.cls
90.64% <ø> (+0.49%)
:arrow_up:

...ain/default/classes/ActionPlanDetailController.cls
92.52% <100.00%> (+0.14%)
:arrow_up:

.../main/default/classes/ActionPlansTestUtilities.cls
94.27% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Added support for textDecorationLine

@ryangomba have we abandoned the even branch pattern?

Updated to include remove those unnecessary warning and invariant changes.


MediaMuxer

Hi,
With rtmp, you have MediaMuxer to mux the audio and video frames. What's about rtsp, which tool plays muxing role?
Thanks,

No, I haven't a MediaMuxer.
SrsFlvMuxer is not a MediaMuxer, this class connect to server and create flv packet of video or audio (separately) and send over the net. RtspClient do a similar roll in rtsp.

Hi,
You make some changes from yasea of Leoma, he convert RGBAToI420 and RGBAToNV12, but you convert YV12toYUV420 and NV21toYUV420.
Is there any reason behind this?
Thanks

Yes. Because Yasea use opengl with a glreadpixel and it get RGBA format. I'm using camera data directly that get data in nv21 and yv12.
nongpuimage brach of yasea use a similar way.

Wow, got it now, thanks

I think that convert RBGA to YUV420. If not i don't know


Run helmlint multiple times, with optional features enabled

Right now we run make helmlint once, and it checks the helm chart with just the default settings for each optional feature.
Instead we should be running it once with each separate feature enabled (plus maybe 2 more for all features on, and all features off). This might not cover all possible expansions of the templates, but that would suffer from combinatorial explosion.
We have code in our image_list.rb script to enumerate all available feature, and expand the templates for various combinations (it used to test all combinations, but now only runs all features on, and all features off). This code could probably be extended to cover the helm linting use case as well.

Could this be alleviated by adding a schema for the values.yaml?

Could this be alleviated by adding a JSON schema for the values.yaml?

@viccuad I don't think so. The JSON schema just validates the content of values.yaml. helm lint will render the templates with the current settings and makes sure the generated output is valid YAML. Since the templates will generate different content depending on the configured features, running helm lint against different combinations still makes sense to me.
However, this is still just a poor substitute for proper tests. Given the complexity of our templates, I think we should strive to have a broader suite of tests that generate charts for various configuration options, and then verify that the generated kube resource configurations match what we expect.
That will be a separate issue, that I plan on filing later this week. For one thing, we'll need a feature cf-operator util interpolate to apply all the generated ops files against the manifest. I'm going to file an issue for that one as well. 😄  We can get started by using bosh int for now, and there is code in imagelist.rb that does exactly that. But bosh doesn't support the additional qcopy and qmove operations that we added to quarks-operator. kubecf doesn't use them yet, but it will in the future, and then bosh int would no longer work.
Maybe if we can get started on proper tests sooner, then we don't need to bother with running helm lint for different configurations. I think that would be my preferred outcome.
PS and BTW, it just so happens that I wrote a schema for our values.yaml last week for my "multi-stack" work (still WIP): https://github.com/cloudfoundry-incubator/kubecf/blob/jandubois/1086-configurable-multi-stack-support/deploy/helm/kubecf/values.schema.yaml.

We can get started by using bosh int for now

I'm using my own dumb wrapper locally, but having it in cf-operator might be nice.  In the mean time that might be helpful.


Copyedit follow-up to PR #1497

This pull request is mostly just copy-editing of some documentation that changed a lot in pull request #1497.
It might seem like I completely re-wrote the page titled "Configure the Node" but I mostly just re-arranged it into three logical sections:

Edit config-map.yaml
Edit secret.yaml
Deploy Your config-map.yaml and secret.yaml

I also added some more explanatory comments to the files config-map.yaml and secret.yaml, so that someone reading those files after the data values have been entered will still have some help to understand what the values mean.
Also:

Expanded explanation of kubectl contexts.
In the section about the Kubernetes Dashboard (Web UI), I added the dashboard's URL (which ends in /ui, a fact that isn't obvious).
There were some Azure-specific paragraphs that looked like so:

Bold text.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Quisque iaculis urna vitae justo bibendum aliquam. Maecenas congue eu metus a dapibus. Praesent accumsan varius erat, eget vehicula erat porttitor id. Integer posuere arcu odio, in maximus diam accumsan non. Vestibulum elementum, nulla eget dictum mollis, eros nisl vestibulum lectus, ut molestie risus lorem sit amet metus.
but should look like below. (This is so it's clear that the bold heading is just for that one paragraph.):
Bold text. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Quisque iaculis urna vitae justo bibendum aliquam. Maecenas congue eu metus a dapibus. Praesent accumsan varius erat, eget vehicula erat porttitor id. Integer posuere arcu odio, in maximus diam accumsan non. Vestibulum elementum, nulla eget dictum mollis, eros nisl vestibulum lectus, ut molestie risus lorem sit amet metus.

I also made a few other minor changes.

Codecov Report

Merging #1526 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1526   +/-   ##
=======================================
  Coverage   98.41%   98.41%
=======================================
  Files          58       58
  Lines        2779     2779
=======================================
  Hits         2735     2735
  Misses         44       44


Sequential processing

For large folders of files, users may prefer slower overall output if they receive files to work with more quickly.

The code is in a good spot for this to be implemented.
A big roadblock is having react wait for each axios call to finish.  currently each file gets sent to the backend at the same time and it isn't handled properly.
I've mostly looked into using await/async, with no success.  I've followed code snippets from online exactly, but I've never gotten anything to actually wait (backend always receives all calls at once)

Completed the implementation of sequential processing within a new endpoint.
React waits for each axios call to finish by recursively making the next axios call within the .finally() block.
Sequential call can easily be configured to pass user's file to different endpoints, and expects to receive back a status code (0 for success), a transcript, and optionally a detected language


add region configuration support

@justindowning - a little more configuration capability (AWS cli requires a default region for configuration)
since I don't do too much puppet (or Ruby) code, got a little mixed up with the $title thingy, since I'm not sure if it was used only for naming the block, or also as input for the action...
would appreciate feedback,

+1 from me for this feature
It would be even better if the module didn't fail when access/secret keys aren't supplied.
See IAM roles - http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html

Great suggestion. I opted to implement this slightly differently (including some clean up work). You can check it out in #17. If there are no objections or suggestions, I'll merge soon. :+1:


fix(docs): rxjs path in CODING_STANDARDS.md

When PR #7276 gets in, this line will be erased.

So I'm closing this.


Test updates

Add "basic" test capability to the suite ... this allows a apache/php only test best as it does not install a db or ElkArte i (Elk files are available).  To use you would update the travis.yml and add in a case like:
- php: 5.4
  env: DB=none

then add tests suffixed with xyzBasic.php .. it will then run all xyzBasic.php tests that it finds in the standard test directory.
In addition

Moved from Fx to Chrome for selenium tests
PHP Unit updated to 4.6 with updated compose lock file
few minor tweaks

And not test related, also renamed the httpreq class so it autoloads.

/me goes testing! :smile:


How can I add the pets without the panel?

The panel takes up too much space and I actually want to see the pets running on my code. :)

+1 !

+1, feels pointless to allocate screen space for the pet.

I know everyone wants this feature, it's been requested many, many, times!
But it's not possible to implement in the VS Code extension API. The pets must be inside a panel, extensions cannot render graphics onto the code editor.

Hmmm I see...
Is there ways to render it in other pre existing panels? Like file explorer, or little nav bar on the top or the bottom?

nope.


File Dialogs adjustments (v2)

Made adjustments to open/save file dialogs according to notes from Akien (see comment https://github.com/godotengine/godot/pull/13252#issuecomment-347045849), plus a bit more:

Added "go to parent folder" (^) button to Save a File dialog.
Tree.cpp: "nothing_selected" signal has been re-made
Fixed issue in Project Export dialog: MODE_SAVE_FILE wasn't set when you click "Export".

See commit message for all details.

Note that there are style issues in your commit: https://travis-ci.org/godotengine/godot/jobs/308004043
See http://docs.godotengine.org/en/latest/community/contributing/code_style_guidelines.html

Merged manually as ad351a29d to fix style. Good job, it works fine :)

@akien-mga will fix at morning, if this can't be merged manually due to this style issue.

Oh, already merged. Thanks :)


Unhardcode HoLC launcher icon for DEB pkg

Since the icon already is installed to a location compliant with freedesktop.org standards you don't need to specify the full icon path or the file type extension in the launcher.
The icon will be found anyway.
Cf. this example .desktop launcher presented in the freedesktop.org documentation or your own launcher for the Hopsan GUI

Thank you.


Tekton integration refactoring and enhancements.

This is great! Much more flexible and in one place now.


Melt relational primops

! @tranma @jystic

#472

From the failed build:
=== prop_read_or_drop_entity from test/Icicle/Test/Sea/Zebra.hs:97 ===
--- many lines snipped ---
error: #error Failed during codegen (seaOfXValue: buffer elements should be converted to pushes by convertValues : (VBuf [VString \"bart\"],BufT 1 StringT)..)\n")

👍 on 🍏

I'm not sure nested Bufs can be a thing? Group in group will allow nested arrays but nested bufs probably can't exist.
:1st_place_medal: otherwise!

I'm not sure nested Bufs can be a thing? Group in group will allow nested arrays but nested bufs probably can't exist.

What about Buf (Array _)?

I think Buf (Array t) might be generated with the right kind of group

going to merge this hellhole

🍾


Use github.com/json-iterator/go instead of encode/json  for system-probe API

What does this PR do?
'encode/json' is not capable of marshalling 'map[interface{}]interface{}'.
Describe how to test/QA your changes
Test that the command system-probe config by-source works without returning an error when service_monitoring_config.http_replace_rules is configured.

/merge


jdk16-windows-aarch64-hotspot : IOException: Cannot run program "nohup"

https://ci.adoptopenjdk.net/job/build-scripts/job/utils/job/cross-compiled-version-out/115/console
Caused: java.io.IOException: Cannot run program "nohup" (in directory "C:\Users\George\Jenkins\workspace\build-scripts\utils\cross-compiled-version-out\OpenJDKBinary"): CreateProcess error=2, The system cannot find the file specified
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1142)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)

fixed, I didn't add git bash unix tools to the system PATH

fixed, I didn't add git bash unix tools to the system PATH

I didn't think we normally did that on our systems - shouldn't that be picked up through cygwin?

I didn't think we normally did that on our systems - shouldn't that be picked up through cygwin?

I don't have cygwin installed on that machine. In fact it has nothing installed on it other than Java and Git Bash.

I don't have cygwin installed on that machine. In fact it has nothing installed on it other than Java and Git Bash.


Fix undeclared identifier OPENSSL_ia32cap_P for e_rc4_hmac_md5.c.

When compiling with VS2019, the following error occurs in e_rc4_hmac_md5.c:
1>e_rc4_hmac_md5.c
1>C:\openssl\crypto\evp\e_rc4_hmac_md5.c(90,31): error C2065: 'OPENSSL_ia32cap_P': undeclared identifier
1>C:\openssl\crypto\evp\e_rc4_hmac_md5.c(90,33): error C2109: subscript requires array or pointer type

Please either send us a CLA - details here:
https://www.openssl.org/policies/cla.html
Or, if you consider your contribution trivial from a copyright perspective, then update your commit to include the text "CLA: trivial" on a line on its own in the body of the commit description.
For the record I consider this a trivial contribution.

CLA:trivial

@mattcaswell : is the cla required label to be removed manually in this case ?

We usuallly reopen the ticket to kick the cla-bot.

The bot still complains because only one of the three commits has the 'CLA: trivial' label. But that can trivially be resolved when squashing the pr. So well, let's just remove the label manually...

I noticed the bot still complaining and assumed it was because of the missing space ie "CLA:trivial" instead of "CLA: trivial". Could be resolved anyway.

You’re right, it’s in the subject line.

It is interesting that I did the same "git commit --ammend" to another PR and added a single line "CLA:trivial" (no space) to the comment and that one worked nicely. (Although it is a bit puzzling for me why that ended up 2 extra commits.) Anyway, the bot automatically fixed (so it seemed) the cla-check for that other PR but not for this one.
Was "git commit --ammend" not the right thing to do?

You seem to have added a merge commit as result of pulling master. We don't actually allow those - always make sure you use git pull --rebase when pulling a fresh master. That is where the confusion has come from I think.

Looks like its only needed for master.

merged, thanks.
master: https://github.com/openssl/openssl/commit/f702256d5514e54138279fd4637353e328262817


Update GGML dependency

I'm looking into the quantisation issue. Would you be able to update the models to use inplace? https://github.com/ggerganov/ggml/commit/3ce314550126a611bd6294da193380d51b8df86d

@danforbes are you OK with the state of the PR as-is? Might merge it and do the de/requant stuff in another PR to unblock this.


qeustion about No module named 'skill_chaining'

Dear Sir
I had tried install ViSkill, in Ubuntu 18.04, as web instruments. But
(1) I cannot find that file as

4.Then add one line of code at the top of gym/gym/envs/__ init__.py to register SurRoL tasks:
directory: anaconda3/envs/dex/lib/python3.8/site-packages/
import surrol.gym

So, I add line "import surrol.gym" in ～/anaconda3/envs/viskill/lib/python3.8/site-packages/gym/envs/__ init__.py
Is it correct?
(2) then, I try
python SurRoL/surrol/data/data_generation_bipegtransfer.py --env BiPegTransfer-v0 --subtask grasp
it show me the error of below.

Traceback (most recent call last):
File "SurRoL/surrol/data/data_generation_bipegtransfer.py", line 13, in
from skill_chaining.components.envrionment import BiPegTransferSLWrapper
ModuleNotFoundError: No module named 'skill_chaining'

It looks it cannot identify the skill_chaining. Did I make something wrong? I will be very appreciated if you ｗould kindly figure out the solution.
Thank you very much.

Hi, Jaujie,
(1): Yes.

Yes, and I had rebuild again.
I also try find ~ -name 'skill_chaining*', and it shows

/home/cmuh/.local/share/Trash/files/ViSkill/viskill/configs/skill_chaining.yaml
/home/cmuh/ViSkill/viskill/configs/skill_chaining.yaml

There is no skill_chaining.py for import? I also check your web file, but I also cannot find it. Did I miss something?

Yes, and I had rebuild again. I also try find ~ -name 'skill_chaining*', and it shows

/home/cmuh/.local/share/Trash/files/ViSkill/viskill/configs/skill_chaining.yaml
/home/cmuh/ViSkill/viskill/configs/skill_chaining.yaml

There is no skill_chaining.py for import? I also check your web file, but I also cannot find it. Did I miss something?

Hi, I have the same error here. Have you found a solution to this?

I just found out that the code is a bit wrong. In the file "SurRoL/surrol/data/data_generation_bipegtransfer.py", line 13, in
from skill_chaining.components.envrionment import BiPegTransferSLWrapper
we need to change it to
from viskill.components.envrionment import BiPegTransferSLWrapper
then it should work.

Thanks for catching it out! I used the original name of this repo here, which would work on my local computer. I will fix it soon.

Dear Tao,
Can you fix it asap? Thanks!
Regards,
Qi
Sent from iPhone
On Sat, 4 May 2024 at 11:50 PM, Tao Huang @.***> wrote:

Thanks for catching it out! I used the original name of this repo here,
which would work on my local computer. I will fix it soon.
—
Reply to this email directly, view it on GitHub
https://github.com/med-air/ViSkill/issues/1#issuecomment-2094271102, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ACLQIFFZRVZVO5VYLZ4FILLZAT7VDAVCNFSM6AAAAAA3W7U3JOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJUGI3TCMJQGI
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Specify BIG-IP account requirements for controllers

Bugs
Describe the bug in detail:

What doc are you reporting an issue with (provide URL)?

What information is inaccurate?

Is information missing?

Is a code sample incorrect?

Requests
Provide details regarding what you would like to see added to our documentation:
The documentation for each CC has an important box with the following information (http://clouddocs.f5.com/containers/v2/kubernetes/index.html):
Important:

- The BIG-IP Controller cannot manage objects in the /Common partition.
- The BIG-IP partition you want to manage must exist before you launch the BIG-IP Controller.
- The BIG-IP Controller does not create or destroy BIG-IP partitions.
- You can use multiple BIG-IP Controller instances to manage separate BIG-IP partitions.
- You can create one (1) BIG-IP virtual server per Service port. Create a separate virtual server F5 Resource ConfigMap for each Service port you wish to expose.

For all the controllers, we need to stress that the controller requires full access to the objects within the partition so it can perform CRUD operations on the objects (create, read, update, destroy).  Any reference to the managed resource objects from outside the partition can interfere with the controller's ability to do this.
Environment

[x ] Cloud Foundry
[ x] Kubernetes
[ x] OpenShift
[x ] Mesos/Marathon)

Use Case
(i.e., what are you trying to do that you can't find documentation for?)

duplicates #291


Update embedded-custom-layout.md

JSON corrections

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: johanflikweert sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@johanflikweert : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@davidiseminger : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Example Instagram package seeds error

I am getting this error when I tried to enable example-instagram package for the first time.
TypeError: Cannot read property '_id' of undefined W20170407-08:55:01.526(6.5)?
(STDERR)     at createPic (packages/example-instagram/lib/server/seed.js:35:13)
it seems the dummy users are not there in db before seeding pictures and so,user object is undefined.

I just tried, and I didn't notice any errors. Maybe do meteor reset and then try again?

This is the same as Issue #1669.
There is a PR #1769 aimed at fixing it.


Remove the dependency on requests

This removes the use of requests from the core functionality, which
results in requests being only used as extension, therefore not being
necessary as dependency anymore.
The motivation for this change is to get rid of all the dependencies of
requests, whose size sums up quite a bit, when not requiring to patch
requests. It follows the removal of requests from botocore, which
was done as part of v1.11.0.
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.

Thank you for your contribution. I have merged this PR and it will be out with our next release.


Bump node-sass to latest now that they have resolved npm audit warning

The latest version of node-sass now uses request 2.87, which apparently
resolves the npm audit warning.
I think we have to continue pinning node-gyp for now, but I am hoping
that node-gyp will similarly upgrade to request 2.87 soon. Once that
occurs, we should be able to remove it as a dev dependency.

Since this is just a dependency change, I'm going to go ahead and merge.


Dynamic td width while creating table

I am creating a table which includes two rows with dynamic numbers of td.
The important part is to set the width for each td and the width has been calculated and passed to view.
tdWidth: 100/numberOfChars => which gives a percentage for width
I am using following code but it is not working:

{{#each array}}
{{this}}
{{/each}}

any helps appreciated.

@nknapp please help!

@amirzandi could you please share more of your code?
Especially the part that creates the "tdWidth" value as well as how you link it to that part of your page.

this is the server side:
res.render('./game/phrase_box', {
layout: false,
incorrects: result.incorrects,
corrects: result.corrects,
numberOfChars: result.numberOfChars,
tdWidth: 100 / numberOfChars
});
this is the client side:
{{#if incorrects}}
{{#if corrects}}

                {{#each corrects}}
                    {{this}}
                {{/each}}

{{/if}}

{{else}}
{{/if}}

You're iterating in "corrects" in your "each" statement, but that object doesn't hold the "tdWidth" object.
I think you may need to change this:
{{tdWidth}}
To this:
{{../tdWidth}}
Ref: http://handlebarsjs.com/ (under "Handlebars Paths")

@magikstm ohhhhh you saved my day.... thank you very much... thanks a lot.


Traceback (most recent call last):
File "/home/xiaoduo/liu/ER-NeRF-main/data_utils/face_tracking/face_tracker.py", line 219, in
render_imgs = renderer(
File "/home/xiaoduo/miniconda3/envs/nerf/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/xiaoduo/liu/ER-NeRF-main/data_utils/face_tracking/render_3dmm.py", line 199, in forward
rendered_img = self.renderer(mesh)
File "/home/xiaoduo/miniconda3/envs/nerf/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/xiaoduo/miniconda3/envs/nerf/lib/python3.10/site-packages/pytorch3d/renderer/mesh/renderer.py", line 62, in forward
images = self.shader(fragments, meshes_world, **kwargs)
File "/home/xiaoduo/miniconda3/envs/nerf/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/xiaoduo/liu/ER-NeRF-main/data_utils/face_tracking/render_3dmm.py", line 73, in forward
images = softmax_rgb_blend(
File "/home/xiaoduo/miniconda3/envs/nerf/lib/python3.10/site-packages/pytorch3d/renderer/blending.py", line 205, in softmax_rgb_blend
alpha = torch.prod((1.0 - prob_map), dim=-1)
RuntimeError:
#define POS_INFINITY __int_as_float(0x7f800000)
#define INFINITY POS_INFINITY
#define NEG_INFINITY __int_as_float(0xff800000)
#define NAN __int_as_float(0x7fffffff)
Traceback (most recent call last):
File "/home/xiaoduo/liu/ER-NeRF-main/data_utils/process.py", line 407, in
save_transforms(base_dir, ori_imgs_dir)
File "/home/xiaoduo/liu/ER-NeRF-main/data_utils/process.py", line 276, in save_transforms
params_dict = torch.load(os.path.join(base_dir, 'track_params.pt'))
File "/home/xiaoduo/miniconda3/envs/nerf/lib/python3.10/site-packages/torch/serialization.py", line 699, in load
with _open_file_like(f, 'rb') as opened_file:
File "/home/xiaoduo/miniconda3/envs/nerf/lib/python3.10/site-packages/torch/serialization.py", line 230, in _open_file_like
return _open_file(name_or_buffer, mode)
File "/home/xiaoduo/miniconda3/envs/nerf/lib/python3.10/site-packages/torch/serialization.py", line 211, in init
super(_open_file, self).init(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'data/obama/track_params.pt'

Seems some errors have occurred in the function face_tracking. Please check this step

refer to issue https://github.com/Fictionarry/ER-NeRF/issues/31


[FLINK-36923] Bump flink version to 1.20.0, adapt CI workflow

Bump flink version to 1.19.1 to support Flink 1.20

Hi @ferenc-csaky
Could you help to review this? and I would greatly appreciate it if you could share your thoughts on this bump plan and the next release.

Hi, thanks for your contribution! I am planning to try to include at least JDK 17 (and possibly 21) support for the next release, which will require to bump HBase version and pretty much all Hadoop related dependencies. I spent some time on that before releasing 4.0, but it is not trivial and I ran into some runtime issues, that's where I decided to cut scope and release to only JDK8 and 11.
Including these version updates makes sense anyway, so I do not see a problem merging it on its own.

@lvyanquan Any specific reason to remove 1.19 as well? I am not necessarily against it, but since 1.19 is still supported I believe it would make sense to go with 1.19, 1.20 support as the default setup if there are no other things to consider. WDYT?

Hi, @ferenc-csaky.
The reason of directly bumping to 1.20 is because I want to speed up the adaptation to 1.20 and try to start integrating with Flink 2.0, and we've released flink-connector-hbase 4.0.0 that supported Flink 1.18 & Flink 1.19, so we can release a 4.0.1 for bug fix in Flink 1.19.
However, this is not a mandatory requirement for me, so I also want to confirm with you whether this is feasible.
If this is too radical, I will revert back to bumping 1.19 and support both 1.19 and 1.20.