Error loading plugin 'fastlane-plugin-firebase_app_distribution' in 0.4.0

Issue Description
I run fastlane only on Gitlab CI and yesterday it stopped working.
Fastlane is imported using just two words gem "fastlane" inside Gemfile, same for plugin gem 'fastlane-plugin-firebase_app_distribution' inside Pluginfile.
But once I set it to fixed version 0.3.8, it started working
Complete verbose output
...
Fetching fastlane-plugin-firebase_app_distribution 0.4.0
...
Installing fastlane-plugin-firebase_app_distribution 0.4.0
...
[08:27:45]: Error loading plugin 'fastlane-plugin-firebase_app_distribution': cannot load such file -- fastlane/plugin/firebase_app_distribution
[08:27:45]: It seems like you wanted to load some plugins, however they couldn't be loaded
[08:27:45]: Please follow the troubleshooting guide: https://docs.fastlane.tools/plugins/plugins-troubleshooting/
+-------------------------------------------+-----------+------------------+
|                               Used plugins                               |
+-------------------------------------------+-----------+------------------+
| Plugin                                    | Version   | Action           |
+-------------------------------------------+-----------+------------------+
| fastlane-plugin-firebase_app_distribution | undefined | No actions found |
+-------------------------------------------+-----------+------------------+
[!] No actions were found while loading one or more plugins
    Please use `bundle exec fastlane` with plugins
    More info - https://docs.fastlane.tools/plugins/using-plugins/#run-with-plugins

Library
Version

fastlane
[the latest]

fastlane-plugin-firebase_app_distribution
[the latest]

Thanks for filing the ticket! I wasn't able to reproduce it on my end. I'm using fastlane version 2.209.1, and I tried updating my Pluginfile to use both no specified version (gem 'fastlane-plugin-firebase_app_distribution') and specifying the latest version (gem 'fastlane-plugin-firebase_app_distribution', "0.4.0") - and both worked for me.
A few troubleshooting ideas:

Can you try running the command with the --verbose flag?
Can you send your fastlane version info and your Pluginfile setup?

Yes, for sure.
Here is Gemfile:
source "https://rubygems.org"

gem "fastlane"

plugins_path = File.join(File.dirname(__FILE__), 'fastlane', 'Pluginfile')
eval_gemfile(plugins_path) if File.exist?(plugins_path)

Pluginfile:
# Autogenerated by fastlane
#
# Ensure this file is checked in to source control!

gem 'fastlane-plugin-firebase_app_distribution'

And the gitlab job:
build_debug_and_deploy:
  stage: build_debug
  image: $REGISTRY_URL/$CI_PROJECT_NAME:v0.0.1
  only:
    - master
    - develop
    - merge_requests
  before_script:
    - bundle install
  script:
    - bundle exec fastlane test --verbose
    - bundle exec fastlane buildDebugAndDeploy
  artifacts:
    paths:
      - app/build/outputs/
    expire_in: 3 days
  allow_failure: false

Beginning of the logs with --verbose:
DEBUG [2023-01-21 04:18:17.04]: Checking if there are any plugins that should be loaded...
DEBUG [2023-01-21 04:18:17.04]: Loading 'fastlane-plugin-firebase_app_distribution' plugin
INFO [2023-01-21 04:18:17.04]: gem 'fastlane-plugin-firebase_app_distribution' is already installed
ERROR [2023-01-21 04:18:17.04]: Error loading plugin 'fastlane-plugin-firebase_app_distribution': cannot load such file -- fastlane/plugin/firebase_app_distribution
DEBUG [2023-01-21 04:18:17.05]: uninitialized constant Fastlane::FirebaseAppDistribution
ERROR [2023-01-21 04:18:17.05]: It seems like you wanted to load some plugins, however they couldn't be loaded
ERROR [2023-01-21 04:18:17.05]: Please follow the troubleshooting guide: https://docs.fastlane.tools/plugins/plugins-troubleshooting/
+-------------------------------------------+-----------+------------------+
|                               Used plugins                               |
+-------------------------------------------+-----------+------------------+
| Plugin                                    | Version   | Action           |
+-------------------------------------------+-----------+------------------+
| fastlane-plugin-firebase_app_distribution | undefined | No actions found |
+-------------------------------------------+-----------+------------------+

[!] No actions were found while loading one or more plugins
    Please use `bundle exec fastlane` with plugins
    More info - https://docs.fastlane.tools/plugins/using-plugins/#run-with-plugins

Successfully loaded Appfile at path '/builds/[Hidden]/fastlane/Appfile'
-------

And the ending:
+------+------------------+-------------+
|           fastlane summary            |
+------+------------------+-------------+
| Step | Action           | Time (in s) |
+------+------------------+-------------+
| 1    | default_platform | 0           |
| 2    | test             | 626         |
+------+------------------+-------------+
Looking for related GitHub issues on fastlane/fastlane...
Search query: Malformed version number string undefined
URL: https://api.github.com/search/issues?q=Malformed%20version%20number%20string%20undefined+repo:fastlane/fastlane
➡  java.lang.IllegalArgumentException: Unknown package:
    https://github.com/fastlane/fastlane/issues/18527 [closed] 4 💬
    19 Aug 2021
➡  snapshot works locally, but not on CircleCI
    https://github.com/fastlane/fastlane/issues/17193 [closed] 10 💬
    25 Dec 2020
➡  version.rb Malformed version number string undefined (ArgumentError)
    https://github.com/fastlane/fastlane/issues/15715 [closed] 4 💬
    28 Jan 2020
and 3 more at: [https://github.com/fastlane/fastlane/search?q=Malformed%20version%20number%20string%20undefined&type=Issues&utf8=✓](https://github.com/fastlane/fastlane/search?q=Malformed%20version%20number%20string%20undefined&type=Issues&utf8=%E2%9C%93)
/usr/lib/ruby/2.7.0/rubygems/version.rb:215:in `initialize': Malformed version number string undefined (ArgumentError)
	from /usr/lib/ruby/2.7.0/rubygems/version.rb:206:in `new'
	from /usr/lib/ruby/2.7.0/rubygems/version.rb:206:in `new'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/plugins/plugin_update_manager.rb:29:in `block in show_update_status'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/plugins/plugin_update_manager.rb:26:in `each'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/plugins/plugin_update_manager.rb:26:in `show_update_status'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/commands_generator.rb:45:in `ensure in start'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/commands_generator.rb:55:in `start'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/cli_tools_distributor.rb:123:in `take_off'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/bin/fastlane:23:in `<top (required)>'
	from /usr/local/bin/fastlane:23:in `load'
	from /usr/local/bin/fastlane:23:in `<main>'
/usr/lib/ruby/2.7.0/rubygems/version.rb:215:in `initialize': \e[31m[!] Malformed version number string undefined\e[0m (ArgumentError)
	from /usr/lib/ruby/2.7.0/rubygems/version.rb:206:in `new'
	from /usr/lib/ruby/2.7.0/rubygems/version.rb:206:in `new'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/plugins/plugin_update_manager.rb:29:in `block in show_update_status'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/plugins/plugin_update_manager.rb:26:in `each'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/plugins/plugin_update_manager.rb:26:in `show_update_status'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/lane_manager_base.rb:20:in `finish_fastlane'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/lane_manager.rb:67:in `cruise_lane'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/command_line_handler.rb:36:in `handle'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/commands_generator.rb:110:in `block (2 levels) in run'
	from /var/lib/gems/2.7.0/gems/commander-4.6.0/lib/commander/command.rb:187:in `call'
	from /var/lib/gems/2.7.0/gems/commander-4.6.0/lib/commander/command.rb:157:in `run'
	from /var/lib/gems/2.7.0/gems/commander-4.6.0/lib/commander/runner.rb:444:in `run_active_command'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:124:in `run!'
	from /var/lib/gems/2.7.0/gems/commander-4.6.0/lib/commander/delegates.rb:18:in `run!'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/commands_generator.rb:354:in `run'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/commands_generator.rb:43:in `start'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/cli_tools_distributor.rb:123:in `take_off'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/bin/fastlane:23:in `<top (required)>'
	from /usr/local/bin/fastlane:23:in `load'
	from /usr/local/bin/fastlane:23:in `<main>'
/usr/lib/ruby/2.7.0/rubygems/version.rb:215:in `initialize': Malformed version number string undefined (ArgumentError)
	from /usr/lib/ruby/2.7.0/rubygems/version.rb:206:in `new'
	from /usr/lib/ruby/2.7.0/rubygems/version.rb:206:in `new'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/plugins/plugin_update_manager.rb:29:in `block in show_update_status'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/plugins/plugin_update_manager.rb:26:in `each'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/plugins/plugin_update_manager.rb:26:in `show_update_status'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/lane_manager_base.rb:20:in `finish_fastlane'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/lane_manager.rb:67:in `cruise_lane'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/command_line_handler.rb:36:in `handle'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/commands_generator.rb:110:in `block (2 levels) in run'
	from /var/lib/gems/2.7.0/gems/commander-4.6.0/lib/commander/command.rb:187:in `call'
	from /var/lib/gems/2.7.0/gems/commander-4.6.0/lib/commander/command.rb:157:in `run'
	from /var/lib/gems/2.7.0/gems/commander-4.6.0/lib/commander/runner.rb:444:in `run_active_command'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:124:in `run!'
	from /var/lib/gems/2.7.0/gems/commander-4.6.0/lib/commander/delegates.rb:18:in `run!'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/commands_generator.rb:354:in `run'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/commands_generator.rb:43:in `start'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/fastlane/lib/fastlane/cli_tools_distributor.rb:123:in `take_off'
	from /var/lib/gems/2.7.0/gems/fastlane-2.211.0/bin/fastlane:23:in `<top (required)>'
	from /usr/local/bin/fastlane:23:in `load'
	from /usr/local/bin/fastlane:23:in `<main>'
Cleaning up project directory and file based variables

@iRYO400 hey, it seems like I may need some more info. Not sure what's going on, but maybe we can debug together. Can you join the Firebase community slack channel https://firebase.community/ and DM me?

@rebehe also hitting the same issue but with GH actions (opposed to gitlab here). Happy to send over more info if need be.
My gem/plugin file/ runner job are pretty much the same as @iRYO400.

Might have something to do with the version of Ruby installed. Try Ruby 3.1.2 @iRYO400 @rebehe

@iRYO400 hey, it seems like I may need some more info. Not sure what's going on, but maybe we can debug together. Can you join the Firebase community slack channel https://firebase.community/ and DM me?

It looks like we work in opposite time zones, so I'll try @anandsathe67’s assumption first

Might have something to do with the version of Ruby installed. Try Ruby 3.1.2 @iRYO400 @rebehe

I run ruby -v  command on CI and got next:
ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x86_64-linux-gnu]

Is it too old? @anandsathe67

I had a bunch of trouble with 2.7 @iRYO400 - things have worked smoothly since I went to 3.1 (currently on 3.1.2)
GIve it a shot if it isnt too much trouble

ruby -v
ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [arm64-darwin20]

@anandsathe67 thanks, I'll try it. But before I have to tell to my devops

@anandsathe67 / @iRYO400 I have been running this on ruby 3.0.2p107, noted by GH Actions. Looking at the blame for the file, it looks like it hasn't changed in 8 months so unsure if ruby would be the root cause here.
@rebehe there are a couple of old issues that allude to the same issue output but I don't understand enough the Fastlane plugin system to know if they would be related. e.g. #43 or #15

Thanks @mrtnrst, #15 looks particularly related, especially given the error message you're seeing is the one that was added because of that thread (PR).
Can't vouch for any personally, but just collecting a few of the recommendations in that thread here:

Upgrade your ruby version
Make sure you're using bundle exec fastlane instead of just fastlane for your command
Run bundle install --path vendor/bundle and then use bundle exec fastlane
Run sudo fastlane install_plugins
If on Mac, it could be a file permission issue. Edit the permission for the gem sudo chmod -R a+r /Library/Ruby/Gems/X.X.X./gems/fastlane-plugin-firebase_app_distribution-0.4.0. Alternatively, you could run sudo gem uninstall fastlane-plugin-firebase_app_distribution and then gem install fastlane-plugin-firebase_app_distribution --user-install to reinstall

We think we found the issue, and shipped a new version (0.4.2) to fix this. Can you all try again and let us know if it's fixed? @iRYO400 @mrtnrst

We think we found the issue, and shipped a new version (0.4.2) to fix this. Can you all try again and let us know if it's fixed? @iRYO400 @mrtnrst

🥳 seems like it fixes the issue without specifying the concrete version


Draft of dockermailserver API

NOTE: not intended to be merged, this PR is just to start discussions about tools and designs

Would be good it the setup.sh and the API could walk together so it would reflect in less code to be maintained.
Since setup.sh is already doing what is needed inside the container, maybe use something like shell2http to implement the API backend trought setup.sh would help.

That's indeed the goal as you can see here; It's basically a python envelope that handles the http part and goodies, but the actual work uses setup.sh where applicable.
Some hinted usage lead me to think that we'll also have some stuff outside from setup.sh at some point, but we'll see.
While shell2http looks nice, it seems to be more general purpose, and I'm not sure it would be a good fit :)


#111, #112 : Kinesis Channel issues fixed.

#111: Aws Kinesis must be set up on the local machine. Currently, 3 ways of Authentication are supported which are as follows :

Using Profile name: Using only a profile name is sufficient, but this will require the setup of configure and credentials files of  AWS on the local machine.
If the AWS account is user-based: Provide an access key, secret key, and session token in the CDD file for authentication. This does not require setting up AWS on the local machine.
If the AWS account is role-based: In addition to the access key, secret key, and session token, provide role arn and role session name for the authentication. This does not require setting up AWS on the local machine.

#112: AWS Kinesis, While starting the BE application, introduced a check whether the kinesis data stream exists or not, in case it does not exist, it will throw an error and terminate. This applies to producers and consumers.

Resolved issues #111 and #112.


Print URL when the server is started

Print URL to access(Like TensorBoard)
Moved line that was never executed

Now it looks
$ python py/server.py
It's Alive!
INFO:root:Application Started
(You can navigate to http://localhost:8097)

I don't think this is correct: in many use cases, the visdom server will run on a different machine than the machine on which your accessing the browser (and there will be no SSH tunnel). In such use cases, the right URL to access would be something like http://' + os.environ['HOSTNAME'] + ':' + FLAGS.port.

@lvdmaaten
You are right.
I made some changes.
Now it becomes
$ python py/server.py
It's Alive!
INFO:root:Application Started
(You can navigate to http://localhost:8097)

or
$ HOSTNAME=example.com python py/server.py
It's Alive!
INFO:root:Application Started
(You can navigate to http://example.com:8097)

I'm not confident whether it is enough or not...

I think this will work fine on OS X and Linux, but it may still give wrong results on Windows? I believe the corresponding environment variable is called %COMPUTERNAME% there, but I don't have a Windows machine so I cannot check it.
Merging this nevertheless, since it is better than nothing. Thanks!


[cmake] PopSift_DIR case sensitive

PopSift project name is CamelCase in develop and feature_repeatability branches. find_package(Popsift) failed, changed to find_package(PopSift).

Actually all the cmake part must be adjusted as we switched to modern cmake for popsift but the popsift branch is not up-to-date with it

Is the cmake update in progress? Otherwise, we could allow this patch anyway.

no I don't think so, this branch is not mergeable at the moment and needs to be cleaned a lot

I have started to squash the "popsift" and clean it... but it's not yet pushed.

OK, I've made equivalent changes on top of popart_squash.


Could not reproduce the results listed in your paper using a single 3090 card.

Details:
Here is your result :

I used the following commands to reproduce the results of using the LLaMA 7B model on the Guanaco (OASST1) dataset:
CUDA_VISIBLE_DEVICES=2     sh scripts/finetunanaco_7b.sh
and the best  reslut is  :

A 1% difference is not a big deal comrade. That could be noise.

For LLaMA 7B, I only reproduce the result of Alpaca (paper: 38.8), while gain a decrease to be 32.7 for chip2 (paper: 34.5), 30.9 for longform (paper: 32.1) and 33.7 for self-instruct (paper: 36.4).
Does anyone have ideas about this? Thanks!

For LLaMA 7B, I only reproduce the result of Alpaca (paper: 38.8), while gain a decrease to be 32.7 for chip2 (paper: 34.5), 30.9 for longform (paper: 32.1) and 33.7 for self-instruct (paper: 36.4).
Does anyone have ideas about this? Thanks!

about alpaca dataset, how to set the hyperparameters?

For LLaMA 7B, I only reproduce the result of Alpaca (paper: 38.8), while gain a decrease to be 32.7 for chip2 (paper: 34.5), 30.9 for longform (paper: 32.1) and 33.7 for self-instruct (paper: 36.4).
Does anyone have ideas about this? Thanks!

about alpaca dataset, how to set the hyperparameters?

simply follow the bash file in the ./scripts/
Actually, from another issue in this project, one said that they evaluate in the MMLU test set, but the qlora.py reports the performance in MMLU dev set. Hence, you need to modify the py file to add the evaluation on test set. Then the results on alpaca and longform should be reproduced, which I've done.

@Forence1999 could you share how you reproduced it? I only got 32.1 with the original hyperparameters.  It seems that so many params have been modified in your scripts.
Suggestions:

use the scripts provided by the author, with modifications as less as possible.
build a docker with the dockerfile provided by the author. Ensuring a consistent env is deadly important to reproduce the exact results, and it will also save u lots of time. If you don't wanna build from scratch, u can download my env (docker pull forence/open-instruct:v1) simply. Please note that I build this image with the author's dockerfile as reference, but not exactly follow it.

Hope this could help you a bit!


Feature: Settings

As discussed, we need to desgin a way to pass Monkey-Netty Settings.
There are multiple ways like:

Use JME AppSettings
Disadvantage with this solution is that someone would not know that he can configure it there, unless he will read Monkey-Netty wiki.

use Java Properties
Its String->String, instead String->Object so IMO bad solution for this one.

use own HashMap Settings class utility
Disadvantage that we need additional class instead of using ready2use solutions.

Im not sure what would be your decision, but for me options 1 and 3 are fine.
If you have more options for Settings, please tell

We could use apache configurations 2, I have used it on other projects and it works good. We could have a network.properties file.

We could use apache configurations 2, I have used it on other projects and it works good. We could have a network.properties file.

as a global settings, sounds good.
But what about "per Server/Client" settings?
For example some Client might want have high timeout value, while other low value.
Same about servers, since each might want different settings.
Do apache configurations 2 allow this?

as a global settings, sounds good.
But what about "per Server/Client" settings?
For example some Client might want have high timeout value, while other low value.
Same about servers, since each might want different settings.
Do apache configurations 2 allow this?

Hmm, apache configuration 2 would be fine for global settings, but not on a per server/client basis.

Hmm, apache configuration 2 would be fine for global settings, but not on a per server/client basis.

@oxplay2 I have created an initial implementation for settings.
I am still playing with it. I will probably make it a hashmap wrapper before I am done, but right now I am playing with how the architecture needs to change to accommodate this.
See #35 for my initial implementation. It does have some large changes to the architecture, which I have been wanting to do for some times anyways. But to implement these changes they were required. I will probably cut a v1.0.0 release for this, along with a couple other features I want to implement that will be breaking changes.

sounds good, no rush, this should be done very well, so take time to make it clever way.
thanks for notification :)


Fix value deprecation

Fix deprecations in typespec-next

Could this be merged proactively to main instead of typespec-next?

This cannot as this is a new syntax in the next version


Fix long manuscript urls

These urls look kind of long and are creating a horizontal scrollbar on the details. Add word-break for url to fix.

:+1:

linkStyle doesn't need to be a function here.  var linkStyle = {wordBreak: "break-word",}; and then style={linkStyle}

:+1:


Duplicate calendar picker when clone

if I clone a row that had a datepicker the next row has two calendar picker

Components in Metro4 have a complex structure that is created automatically when the component is initialized. so you can not just clone an element. you must create them explicitly.

Metro 4 not support jquery clone method


Update script.py

Description
This pull request updates the _auth() static method to work without presenting a credentials.json not found error. It simplifies the authentication process and removes unused modules like pickle and pathlib from the script.
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[x] Documentation Update

Checklist:

[x] My code follows the style guidelines(Clean Code) of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have created a helpful and easy-to-understand README.md
[ ] My documentation follows Template for README.md
[ ] I have added the project meta data in the PR template.
[ ] I have created the requirements.txt file if needed.

Project Metadata
Category:

[ ] Calculators
[ ] AI/ML
[ ] Scrappers
[ ] Social_Media
[ ] PDF
[ ] Image_Processing
[ ] Video_Processing
[ ] Games
[ ] Networking
[ ] OS_Utilities
[ ] Automation
[ ] Cryptography
[ ] Computer_Vision
[ ] Fun
[x] Others

Title: OAuth Update - Google Meet Scheduler
Folder: Google-Meet-Scheduler
Requirements: none
Script: script.py
Arguments: none
Contributor: vishnu-v-vardhan
Description: Helps you schedule a Google Meet and also invite people, all via a simple script.

Please create an issue for the same

@avinashkranjan
Created a new issue #3095


[5.2] levenshtein(): Argument string(s) too long

There is a problem when calling commands via $this->call() from another command with an extra long parameter list:
$this->call('ide-helper:models ' . $parameterList . ' --nowrite');

ErrorMessage:
[ErrorException]
  levenshtein(): Argument string(s) too long

levenshtein() allows a maximum length of 255 characters. Maybe similar_text() is an alternative. The PHP documentation does not include any information of limits for it.
The Error-Log shows that a file from symfony/console is calling the function. So i am not sure whether you want to care about this:
vendor/symfony/console/Application.php(993): levenshtein('models --dir="m...', 'make')

I do not tested it in 5.3.
Thanks and great job!

This error occurs for me too, but it is a little different on my case.
I simply created a Console Command signature with more than 255 chars.
Theses 255 chars include the options descriptions and the indentation (of 4 spaces), so it makes very difficult to document more than 4/5 options well indented on a single command.
My non working signature example :
protected $name = 'six:create user
                        {name : The name of the unix account}
                        {--password= : The password of the account}
                        {--home= : Create and assign a home directory to this user}
                        {--group= : Assign a specific group for this user}';

Laravel : 5.3.8
PHP : 7.0.8

Looks like we just need to document this limitation, or if anyone wants to send a good PR to fix this, that'd be fine too. :)

I will look for a PR on this, but for now i agree for the doc update limitation.
As 255 chars is a reasonable limit, more can be really useful with big signature indentation / documentation.

For large sigs, you can still use the other ways to define commands though. :)

I was digging the problem and just realized that I was using the old $name attribute instead of the $signature one for the signature ...
Works well with the $signature attribute as it parse sections separately. It means that we are only limited to 255 chars for a command name ... That is enough :)


Modernisation for AC_OUTPUT

I recommend to move the file name from the call of the macro "AC_OUTPUT" to the call of the macro "AC_CONFIG_FILES".
Would you like to clean-up the affected configuration script for the issue "Addition of a build system generator"?

Thanks for your small build script improvement.


Implement conditional sampling

Our current sampler is only from the prior.

Conditional sampling is implemented in https://github.com/Juice-jl/ProbabilisticCircuits.jl/commit/a2701593cc1b094248efd57d7bdc0ecf949df177


Merging Web vs Web.Shared and WindowsServer vs WindowsServer.Shared

Changes
Merging Web and Web.Shared projects
Merging WindowsServer and WindowsServer.Shared
Checklist

[X] I ran Unit Tests locally.
[ ] CHANGELOG.md updated with one line description of the fix, and a link to the original issue if available.

For significant contributions please make sure you have completed the following items:

[ ] Design discussion issue #
[ ] Changes in public surface reviewed

The PR will trigger build, unit tests, and functional tests automatically. Please follow these instructions to build and test locally.
Notes for authors:

FxCop and other analyzers will fail the build. To see these errors yourself, compile localy using the Release configuration.

Notes for reviewers:

We support comment build triggers

/AzurePipelines run will queue all builds
/AzurePipelines run <pipeline-name> will queue a specific build

looks like you made a bug in your previous Web PR.
I need you to change this line:
https://github.com/microsoft/ApplicationInsights-dotnet/blob/4915ed541f3b30cd36165dc14f27daf9fd15bac4/WEB/Src/Web/Web.Tests/Web.Tests.csproj#L9
These tests aren't being run because this value is incorrect.
remove the decimal. Value should be "net45"


Handle duplicate asset creation when the database was not purged after deleting

When an asset is deleted from Snipe-IT but the database is not purged, jamf2snipe will silently fail to recreate those assets.

jamf2snipe now throws an error when this condition occurs

jamf2snipe now throws an error when this condition occurs


Carbon doesn't print MongoDate from an array

Hallo,
my document is like this

birth
-- date // 1980-01-01
-- place
birthdate (created just for trying)

in User.php model I set
protected $dates = [         'birth.date',         'birthdate',     ];
in home.blade.php
{{ Carbon::parse(Auth::user()->birth['date'])->format('d.m.y') }}
{{ Carbon::parse(Auth::user()->birthdate)->format('d.m.y') }}
In the first parse I get another date:
30-11-19

Here it is the print_r of the "birth" and of the "birthdate"
Array
(
    [date] => MongoDB\BSON\UTCDateTime Object
        (
            [milliseconds] =><PHONE_NUMBER>00
        )

    [place] => "New York"
)

Carbon\Carbon Object
(
    [date] => 1980-01-01 00:00:00.000000
    [timezone_type] => 3
    [timezone] => UTC
)

First you need to UTC date object to carbon one: Auth::user()->birth['date']->toDateTime(), then probably you still need to change the timezone:
Auth::user()->birth['date']->toDateTime()->setTimezone(new DateTimeZone('YOUR_TIME_ZONE'))

The point is that the two dates are the same and Carbon should parse the equally, but it seems it cannot do thejob from inside an array (protected $dates)


Waiting for release instead of throwing error

I got the following error when second execution comes to the code wrapped in runWithLock:
 [Nest] 57  - 10/06/2024, 10:40:56 PM   ERROR [RpcExceptionsHandler] Failed to acquire lock for key updateLock: Failed to obtain lock for key updateLock after 3 attempts.

What does the error mean? Is this the correct behavior, to throw the error instead of waiting for the second execution to finish? I'd like to use the approach like in https://github.com/errorception/redis-lock where I wait for the previous code to be executed. Can I achieve this using murlock? Thanks

I have same issue like @asimaranov. I have also settings like this
MurLockModule.forRoot({
      redisOptions: { url: process.env.REDIS_URL },
      wait: 1000,
      maxAttempts: 3,
      logLevel: 'log',
      ignoreUnlockFail: true,
    }),

So it's should ignore failing right?


Fixed monitoring of node local dns.

How to categorize this PR?

/area networking
/area monitoring
/kind bug
What this PR does / why we need it:
Fixed monitoring of node local dns.
The migration from the helm charts to the go based deployment broke the monitoring
of node local dns as the credentials were not provided. Now, they are added again
analogously to the coredns monitoring.
Which issue(s) this PR fixes:
None.
Special notes for your reviewer:
Release note:

Monitoring dashboards of node local dns should work again.

This issue may have caused the following errors in the prometheus logs:
level=error ts=2022-05-17T14:18:00.790Z caller=klog.go:116 component=k8s_client_runtime func=ErrorDepth msg="pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Pod: failed to list *v1.Pod: Get \"https://kube-apiserver:443/api/v1/pods?limit=500&resourceVersion=0\": x509: certificate signed by unknown authority"

cc @shafeeqes, we probably could include this in v1.47 (minor change only)

Now includes the missing test change as well.

/milestone v1.47

/unhold


Faster build times (18x)

With the current version of middleman, a build of my website will take 76 seconds.
Through some experimentation I've come to a (hasty I might add) patch that does the exact same build in 4 seconds
The patch is here:
https://github.com/dtorres/middleman/commit/6d9b38216e7d225e6bf75fa49218d3f5cfe3be26
Some notes:

Currently middleman triggers parallel rendering but due to lock in Rack (I assume, I haven't thoroughly looked at how Rack works) all the benefits are rendered useless
Disabling concurrency in the current version shaves off 1 or 2 seconds
The machinery in Rack as is, accounts for the majority of build time.
Looking at how the builder and rack client works, it simply calls the render method in the resource, then the builder only uses the status code to know if the build succeeded.
With that in mind, the same can be achieved by knowing that if any errors occur during the build call, they will be caught by the catch in the builder class so it seems safe to directly call the render method.
The current process adds a seemingly unnecessary lookup. resource.path -> rack -> sitemap.find_by_path -> resource.

Also, this is an issue instead of a PR because..
a) the patch is unworthy
b) I am unsure if there are dragons laying ahead I haven't encountered in my testing. Maybe from this issue a better solution can be reached

Ran https://gitlab.com/gitlab-com/www-gitlab-com with and without the patch in my machine

With:
bundle exec middleman build --verbose  584.70s user 10.09s system 308% cpu 3:13.03 total
Without:
bundle exec middleman build --verbose  780.24s user 37.19s system 311% cpu 4:22.20 total

There is this old PR I made to replace all pieces of code relying on Rack to allow this kind of improvement: https://github.com/middleman/middleman/pull/1900
At the time, it was just after v4, so it seemed bad to break compatibility. Maybe it's time to bring that in, add something like this patch and get some great performance benefits.

This would be a major victory if we can speed up the build, its a big issue with many would be adopters, they are swayed to other platforms based on build speed.
I will run this patch on my own setups now and see if there are any issues. Im glad to see GitLab running the blog as I want to make sure it compatable with that.
Will report back on findings. @dtorres Can you create a pull request for this also based on some of the work from @tdreyno thread also ?

Let me know if you want me to run instruments / profile on the above. I use Webpack also which takes 7 seconds.

@iwarner Seeing your results prompt me to test some more and found that by removing Rack, as a side effect I stopped the pipeline that was actually taking time and preventing the multithreading from working.
Long story short, I had minify_javascript activated. deactivating it took me back to the ~4 seconds of build time with the current middleman release. (Along the journey I did some PRs for middleman-blog)
I will take a look at rewriting the minify_javascript extension to behave nicely but in the meantime... @tdreyno is it possible to put somewhere that using "minify_javascript" is strongly discouraged?

minify_javascript requires a JS runtime and the communication between ruby and that are just awfully slow :(
@dtorres I'm okay with updating the docs site to mention the poor performance and updating the default template (https://github.com/middleman/middleman-templates-default) with a callout.
If you know your platform and installed tools, it's probably fastest to add the cli for some of the popular minifiers and shell out to them directly after_build.
\uglifyjs -- build/site.js > build/site.min.js``

We could look at mini_racer now instead of ruby_racer which is a bugger to install, based on lots of previous comments on here.
@dtorres - I minify JS through webpack, but use the minify_html plugin of middleman.
So I assume the tests I did would still stand in this case.
I will deactivate minify_html and test again then.
Will check out the blog PRs


Return response if Chrome 51-66 is detected

Same-site = None will break for Chrome 51-66
This PR checks for those specific browser in HTTP_USER_AGENT and returns response on detection.

@vinyasmusic Hey, I see that you're busy. That's totally okay! I have some free time tomorrow and I'm planning to merge your PR (because I want to give you attribution that you deserve) and the other unsupported browsers. After that I'll release the next version :-)

Hey @jotes
Sorry for the delayed response, been a bit unwell.
Will make the regex change by today and update the PR.
Will add the unit-tests in a few days.
Thanks

@vinyasmusic No worries and no pressure. Feel free to ping me if you need help or don't have time.

Would need some help over here, I am bad with regex.
Ideally we should be looking at Chrome / Chromium 51 - 66 in regex, how do we achieve that ?

We internally pushed that old regex but it was breaking and have subsequently changed the regex to
(Chrome|Chromium)\/((5[1-9])|6[0-6])
Please verify this

@vinyasmusic in what scenarios that regex was broken? It would be useful to add tests for them.

Hey @vinyasmusic, I have a moment to take it over - are you okay with that? I'm going to merge this PR and add test suites for this behaviour. Thank you for your work!

Yes, please. I'm really sorry I couldn't complete the needed tests.
Additional info for the error we received : It was happening for Chrome 65 on Browserstack.


#12955: add support for closing tunnels independently

Ticket
Link to Github Issue
Problem description
Need support for closing individual tunnels in cluster.
What's changed
Move CloseDevices functionality to DevicePool.
Updated Device to allow for dev->close() and dev->initialize_and_launch_firmware() to work.
Cleaned up set_internal_routing_info_for_ethernet_cores, change it to work for subset of devices.
Checklist

[ ] Post commit CI passes
[ ] Blackhole Post commit (if applicable)
[ ] Model regression CI testing passes (if applicable)
[ ] Device performance regression CI testing passes (if applicable)
[ ] New/Existing tests provide coverage for changes

Closing PR, took out the cleanup portions of this PR and moved it here: https://github.com/tenstorrent/tt-metal/pull/13890
Will need to revisit the functional changes at a later date.


Lemur 1.2.0 : Missing file-saver.js , a piece of internet removed

Hi there,
I was able to install Lemur few weeks ago. But now, the make release command abort with following error message :
bower file-saver.js#~1.20150507.2       ECMDERR Failed to execute "git ls-remote --tags --heads https://github.com/Teleborder/FileSaver.js.git", exit code of #128 remote: Support for password authentication was removed on August 13, 2021. remote: Please see https://docs.github.com/en/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls for information on currently recommended modes of authentication. fatal: Authentication failed for 'https://github.com/Teleborder/FileSaver.js.git/'

Additional error details:
remote: Support for password authentication was removed on August 13, 2021.
remote: Please see https://docs.github.com/en/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls for information on currently recommended modes of authentication.
fatal: Authentication failed for 'https://github.com/Teleborder/FileSaver.js.git/'
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! Lemur@ postinstall: `bower install --allow-root --config.interactive=false`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the Lemur@ postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR!

As a temporary workaround to be able to install Lemur for my tests :

remove line about angular-file-saver in file bower.json
run command npm install angular-file-saver --save-dev
run command make release

The proposed workaround looks not working.
Even if make release pass the WebUI does not work. In the web browser, the console gives the following error message:
Error: [$injector:modulerr] Failed to instantiate module lemur due to:
[$injector:modulerr] Failed to instantiate module ngFileSaver due to:
[$injector:nomod] Module 'ngFileSaver' is not available! You either misspelled the module name or forgot to load it. If registering a module ensure that you specify the dependencies as the second argument.
http://errors.angularjs.org/1.4.9/$injector/nomod?p0=ngFileSaver
minErr/<@http://<IP_ADDRESS>:8080/scripts/vendor.js:17042:12
setupModuleLoader/</module/<@http://<IP_ADDRESS>:8080/scripts/vendor.js:18980:17
ensure@http://<IP_ADDRESS>:8080/scripts/vendor.js:18904:38
module@http://<IP_ADDRESS>:8080/scripts/vendor.js:18978:14
createInjector/loadModules/<@http://<IP_ADDRESS>:8080/scripts/vendor.js:21421:22
forEach@http://<IP_ADDRESS>:8080/scripts/vendor.js:17315:20
loadModules@http://<IP_ADDRESS>:8080/scripts/vendor.js:21405:12
createInjector/loadModules/<@http://<IP_ADDRESS>:8080/scripts/vendor.js:21422:40
forEach@http://<IP_ADDRESS>:8080/scripts/vendor.js:17315:20
loadModul…
vendor.js:29696:24

I tested release on current master https://github.com/Netflix/lemur/commit/1317a984225f18b13bb990c27d7d9df4c65eb8e0.
I was able to run make release without trouble. So probably a fix was done since Lemur release 1.2.
Is it planed to produce a new release for Lemur ?

we just did a release:
https://github.com/Netflix/lemur/releases/tag/v1.3.0


Diameter of Binary Tree

File(s) Modified: 543-Diameter-of-Binary-Tree.swift
Language(s) Used: swift
Submission URL: _https://leetcode.com/submissions/detail/791964964/_

Thanks, @Mahim1997!


Fix semantics labeling for paragraphs

This is cherrypicked out of https://github.com/flutter/flutter/pull/34368.
Fixes the labeling strategy for paragraph when dealing with inline widgets/placeholders
If https://github.com/flutter/flutter/pull/34368 lands, then this can be abandoned.

Setting records for fastest turnaround, other PR is landing soon, closing!


Reduce delay before flush when large timekey and small timekey_wait a…

When large timekey and relatively small timekey_wait are given, the delay before flush could be too long. The cause is running interval of enqueue thread which is set as timekey / 11.0. If timekey is 86400 (1 day), the interval is 7854 seconds. Even if timekey_wait is set to 600 seconds in this case, the actual delay before flush could be up to 600+7854 seconds. Taking timekey_wait into account for deciding enqueue thread interval would mitigate this.

Thanks. Will check soon.

Looks good. Thanks!

Thanks for reviewing and merging.


Enabled unsupported test for g++ alone #830

Moved unsupported test to supported and modified CMakeLists.txt so that the test is built and run only with g++

bors try

bors try


YARN container failures

Story
Handle master and worker container failures

-1
Reason:

TaskContext already contains most information of Processor.
Some field like taskclass is not useful for Task.
Some field will be changed when passing to runtime, like taskConf. It will merge with worker config.
It is risky to serialize user defined Processor and pass them to tasks. As we don't know what fields user may define.


[CARBONDATA-3089] Change task distribution for NO_SORT table loading

In case of NO_SORT, currently carbon follows task distribution the same way as
LOCAL_SORT, however, in this case it is better to follow the same task distribution as other file format does.
Thus, this PR change the task distribution as global sort, which depends on CSV input format to get the input split and send task based on compute framework scheduling.

[X] Any interfaces changed?
No
[X] Any backward compatibility impacted?
No
[X] Document update required?
No
[X] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
run all test
[X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
NA

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/1332/

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/1543/

Build Success with Spark 2.3.1, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/9591/

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/1334/

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/1545/

Build Failed  with Spark 2.3.1, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/9593/

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/1336/

Build Failed  with Spark 2.3.1, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/9595/

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/1547/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/1339/

Build Failed  with Spark 2.3.1, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/9598/

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/1550/

retest this please

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/1346/

Build Failed with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/1556/

Build Success with Spark 2.3.1, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/9604/

retest this please

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/1351/

Build Failed  with Spark 2.3.1, Please check CI http://<IP_ADDRESS>:8080/job/carbondataprbuilder2.3/9609/

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/1561/

retest this please

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/1384/

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/1594/

Since it is using CSV split in this PR, there are more tasks and each task is smaller, thus we can get better parallelism and require less resource to run in each task

Build Success with Spark 2.2.1, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/2157/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder2.1/1980/

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:12545/job/ApacheCarbonPRBuilder2.1/54/

Build Failed  with Spark 2.3.2, Please check CI http://<IP_ADDRESS>:12545/job/ApacheCarbonPRBuilder2.3/53/


[#106665814] Run smoke tests regularly on tsuru trial

What
Run smoketests regurarly on tsuru trial platform for the purpose of monitoring. The smoketests run every 2 minutes, so that the gap between issue happening and smoketests finding it is small. The smoketest jobs have 5 minutes timeout (they fail if they run longer), so that we can be alerted when the test gets stuck or platform gets a lot slower suddenly.
In this PR we also do a bit boyscouting:

disable email notifications in Vagrant - we don't want to send mails to the whole team when we do job development
explicitly wipe the job definitions dir of the DSL job. This way we don't keep re-creating old, removed jobs.

Testing
Use vagrant (make vagrant-up or simply vagrant up). Enable trial smoke tests job, wait and observe it running. You can test job timeouts by adding sleep 300 just before the smoketst step in the job itself.
Who?
Anyone from the team can review and merge this PR but @Jonty or @mtekel.
Deploying to Jenkins
This PR depends on https://github.com/alphagov/multicloud-deploy/pull/50 being deployed to Jenkins. Before you start applying, do check if this was done. If so, then:

Run ansible. This will create the new jobs. The jobs will run automatically.
Verify general health of the server and that nothing else is broken (e.g. some other jobs not running or failing).

:+1:  LGTM plus nice clean up!


Bump version to 0.7.1

What's Changed

Add basic flow examples and corresponding tests by @DelevoXDG in https://github.com/software-mansion/starknet-jvm/pull/300
Add support for getBlockWithTxHashes by @DelevoXDG in https://github.com/software-mansion/starknet-jvm/pull/305
Update simulateTransactions to match RPC 0.4 by @DelevoXDG in https://github.com/software-mansion/starknet-jvm/pull/308
Convert StarkNet to Starknet by @DelevoXDG in https://github.com/software-mansion/starknet-jvm/pull/306
Remove Testnet2 support by @DelevoXDG in https://github.com/software-mansion/starknet-jvm/pull/312
Support for Cairo 2 contracts by @DelevoXDG in https://github.com/software-mansion/starknet-jvm/pull/285

Codecov Report
Patch and project coverage have no change.

Comparison is base (9426ef3) 69.52% compared to head (fe73a34) 69.52%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #316   +/-   ##
=======================================
  Coverage   69.52%   69.52%
=======================================
  Files          67       67
  Lines        2487     2487
  Branches      216      216
=======================================
  Hits         1729     1729
  Misses        685      685
  Partials       73       73

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Document Plan Executions API

I would do #741 instead. We already have some docs in the CRD, with those two, it should be enough I think


how can we get multiple or all health data in a short way?

I am working on an app in which I want to show all the health kit data, so is there any way that I can get all the data using a single function. I can get different data using different functions but it gets lengthy and will take more time to load,
I was wondering in the documentation but I can't found such a function to get multiple data.

Hey @nHussain18 unfortunately there is no way to get all healthkit data with a single function, as this is currently not supported by Apple's Healthkit.


Add filesystem support to USB devices

Description: Add support to ntfs, maybe fat/vfat filesystems
Definition of Done (DoD): working fine with the filesystems above
Blockers: None.

Change buildroot: add SMB/CIFS support


Separate and structure components for each casino.

Now, there's only a single component handeling all the casinos. Try restructuring and separate these into multiple reusable components for App.js and content.js (Chrome Extention).

Hey Devilla. I notice you are still working on my extension. How can I help? If you want to continue where we left off without any middlemen controlling your money, hit me up on Telegram. I can fund this tool to completion.

Sounds good. Thanks.

Solved https://github.com/provably-fair/provably-fair-app/pull/24


[Feature] Enable Adapter Only Checkpointing for LoRA

LoRA recipes don’t need to save the merged checkpoint at each epoch as the much smaller adapter weights plus the original checkpoint is all you need. Checkpointer has an option on save_checkpoint called adapter_only that only saves the adapter weights. This PR would be to update all Lora based recipes to pass adapter_only=true for each epoch except the final one to get a final saved model.

Hi @pbontrager,
Could you assign Issue #1212 to me? Here are the changes I am considering:

Since the adapter_only option is already implemented in the save_checkpoint methods of each Checkpointer class in torchtune.utils._checkpointing._checkpointer.py, no modifications to this module are necessary.

#1210

Add the adapter_only option to the four modules in recipes/lora_**.py.
When the above option is set to True, skip the block of code around

https://github.com/pytorch/torchtune/blob/main/recipes/lora_dpo_distributed.py#L508-L527.

@spider-man-tm This sounds like a great plan - thank you for laying it out so clearly! Assigning this issue to you, but feel free to ping @pbontrager or me if there's anything we can do to help.


Port fix for URL helper redirect

Fixes #6910

Also, needs to be ported to https://github.com/aspnet/Mvc/tree/rel/2.0.1.

Yes there's a separate PR for 2.0.1.


HADOOP-18629. Hadoop DistCp supports specifying favoredNodes for data copying

Description of PR
Hadoop DistCp supports specifying favoredNodes for data copying.
Jira: https://issues.apache.org/jira/browse/HADOOP-18629
How was this patch tested?
Add new UT.
For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[x] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[x] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

-1 to anything exposing internal hdfs implementation methods. Sorry
People start using them and expect them to be stable and maintained. There is also the little detail that in cloud deployments do not always have hdfs jars on the class path; this PR would break those deployments.
What would make sense would be to use createFile() and for hdfs to add a .opt() option for those favoured nodes, createFile() is the public api, .opt() options can be ignorred by other filesystems, or reimplemented. There is a lot more in terms of design and wiring up but the benefit is that portability and maintainability.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 38s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 2 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
17m 32s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
31m 33s

trunk passed

+1 :green_heart:
compile
23m 39s

trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04

+1 :green_heart:
compile
20m 40s

trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08

+1 :green_heart:
checkstyle
3m 46s

trunk passed

+1 :green_heart:
mvnsite
2m  6s

trunk passed

+1 :green_heart:
javadoc
1m 49s

trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04

+1 :green_heart:
javadoc
1m 38s

trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08

+1 :green_heart:
spotbugs
3m 58s

trunk passed

+1 :green_heart:
shadedclient
23m 54s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 29s

the patch passed

+1 :green_heart:
compile
22m 41s

the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04

+1 :green_heart:
javac
22m 41s

the patch passed

+1 :green_heart:
compile
20m 43s

the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08

+1 :green_heart:
javac
20m 43s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
3m 37s
/results-checkstyle-root.txt
root: The patch generated 9 new + 54 unchanged - 1 fixed = 63 total (was 55)

+1 :green_heart:
mvnsite
2m  4s

the patch passed

+1 :green_heart:
javadoc
1m 41s

the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04

+1 :green_heart:
javadoc
1m 38s

the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08

+1 :green_heart:
spotbugs
4m 10s

the patch passed

+1 :green_heart:
shadedclient
24m  1s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 44s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
27m 55s
/patch-unit-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch passed.

+1 :green_heart:
asflicense
1m  1s

The patch does not generate ASF License warnings.

251m 27s

Reason
Tests

Failed junit tests
hadoop.tools.TestDistCpSync

hadoop.tools.TestDistCpWithRawXAttrs

hadoop.tools.TestDistCpSystem

hadoop.tools.TestDistCpWithAcls

hadoop.tools.TestDistCpWithXAttrs

hadoop.tools.contract.TestLocalContractDistCp

hadoop.tools.contract.TestHDFSContractDistCp

hadoop.tools.TestDistCpSyncReverseFromTarget

hadoop.tools.TestDistCpSyncReverseFromSource

Subsystem
Report/Notes

Docker
ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5391

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 060e891702b0 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / be67a5984771d510d7cfd96439215bccc3d5baca

Default Java
Private Build-1.8.0_352-8u352-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/1/testReport/

Max. process+thread count
566 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 39s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 2 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
7m 42s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
37m 13s

trunk passed

+1 :green_heart:
compile
23m 10s

trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04

+1 :green_heart:
compile
20m 29s

trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08

+1 :green_heart:
checkstyle
3m 47s

trunk passed

+1 :green_heart:
mvnsite
2m  6s

trunk passed

+1 :green_heart:
javadoc
1m 47s

trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04

+1 :green_heart:
javadoc
1m 38s

trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08

+1 :green_heart:
spotbugs
3m 56s

trunk passed

+1 :green_heart:
shadedclient
23m 53s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 30s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 27s

the patch passed

+1 :green_heart:
compile
22m 26s

the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04

+1 :green_heart:
javac
22m 26s

the patch passed

+1 :green_heart:
compile
20m 28s

the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08

+1 :green_heart:
javac
20m 28s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
3m 37s
/results-checkstyle-root.txt
root: The patch generated 8 new + 55 unchanged - 1 fixed = 63 total (was 56)

+1 :green_heart:
mvnsite
1m 59s

the patch passed

+1 :green_heart:
javadoc
1m 39s

the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04

+1 :green_heart:
javadoc
1m 38s

the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08

+1 :green_heart:
spotbugs
4m  7s

the patch passed

+1 :green_heart:
shadedclient
23m 53s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 44s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
29m 24s
/patch-unit-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch passed.

+1 :green_heart:
asflicense
1m  2s

The patch does not generate ASF License warnings.

247m 24s

Reason
Tests

Failed junit tests
hadoop.tools.TestDistCpSync

hadoop.tools.TestDistCpWithRawXAttrs

hadoop.tools.TestDistCpSystem

hadoop.tools.TestDistCpSyncReverseFromTarget

hadoop.tools.TestDistCpWithAcls

hadoop.tools.contract.TestHDFSContractDistCp

hadoop.tools.TestDistCpSyncReverseFromSource

hadoop.tools.contract.TestLocalContractDistCp

hadoop.tools.TestDistCpWithXAttrs

Subsystem
Report/Notes

Docker
ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5391

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux a25803cbcfde 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 08c0cafef93ba76b6b69613bdfcc2542467ca5d3

Default Java
Private Build-1.8.0_352-8u352-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/2/testReport/

Max. process+thread count
735 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@steveloughran hi, I have fixed some problems, could you please review it again? and please correct me if I'm wrong. Thank you :)

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 45s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 2 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
17m 17s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
31m 37s

trunk passed

+1 :green_heart:
compile
24m 53s

trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04

+1 :green_heart:
compile
22m 10s

trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08

+1 :green_heart:
checkstyle
3m 44s

trunk passed

+1 :green_heart:
mvnsite
2m  4s

trunk passed

+1 :green_heart:
javadoc
1m 49s

trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04

+1 :green_heart:
javadoc
1m 39s

trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08

+1 :green_heart:
spotbugs
3m 55s

trunk passed

+1 :green_heart:
shadedclient
23m 58s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 30s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 28s

the patch passed

+1 :green_heart:
compile
22m 25s

the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04

+1 :green_heart:
javac
22m 25s

the patch passed

+1 :green_heart:
compile
20m 25s

the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08

+1 :green_heart:
javac
20m 25s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
3m 36s

the patch passed

+1 :green_heart:
mvnsite
2m  3s

the patch passed

+1 :green_heart:
javadoc
1m 41s

the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04

+1 :green_heart:
javadoc
1m 41s

the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08

+1 :green_heart:
spotbugs
4m  8s

the patch passed

+1 :green_heart:
shadedclient
24m  2s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 45s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
14m 46s
/patch-unit-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch passed.

+1 :green_heart:
asflicense
0m 59s

The patch does not generate ASF License warnings.

240m 40s

Reason
Tests

Failed junit tests
hadoop.tools.contract.TestLocalContractDistCp

Subsystem
Report/Notes

Docker
ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5391

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux d770bec9be9c 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 1182895b78d4d92012d09f1fa11d88abd50ef6b7

Default Java
Private Build-1.8.0_352-8u352-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/3/testReport/

Max. process+thread count
667 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@steveloughran hi, thanks for your suggestion:) I know what you mean, but I find that it also uses  the hdfs public/stable API in source code.
https://github.com/apache/hadoop/blob/723535b788070f6b103be3bae621fefe3b753081/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java#L230
I just refer to its practice in the if branch, and if you think my code changes affect too much, can I just change the else branch code and add favoredNodes option in it? Please correct me if I'm wrong. Thank you :)

@steveloughran @toddlipcon @phunt
hi, could you give me some advice? Thank you!

@steveloughran @toddlipcon @phunt
hi, could you give me some advice? Thank you!

This is interesting. I know there are deployments without hdfs around (e.g. azure clusters), but do see that the import is there for snapshot updates (hdfs only) with explicit imports of SnapshotDiffReport. tracing it back, if you use "-diff" on the command line then hdfs must be on the classpath.
your link flags up that it is already in copymapper; looking at that I don't see it in branch-3.3; it came in with HADOOP-14254.
Given it is in there on a codepath hit with the option to copy erasure policy (and skipped if not), then again, provided the change goes in such that it is optional, your patch isn't going to force in a new run-time dependency, is it?
let me look at the new patch some more without worrying about that detail...the builder API is public.
Be aware that we are always very nervous about touching distcp because it is fairly old and brittle code that is used incredibly broadly -not just on the command line but actually at the Java API from applications like hive. I think this is fairly low risk but will highlight the JIRA on the HDFS mailing list so they can review it to.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 37s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 2 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 28s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m 50s

trunk passed

+1 :green_heart:
compile
23m 12s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
compile
20m 37s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
checkstyle
3m 49s

trunk passed

+1 :green_heart:
mvnsite
2m  5s

trunk passed

+1 :green_heart:
javadoc
1m 50s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
1m 39s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
spotbugs
3m 58s

trunk passed

+1 :green_heart:
shadedclient
20m 33s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 29s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 15s

the patch passed

+1 :green_heart:
compile
22m 29s

the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javac
22m 29s

the patch passed

+1 :green_heart:
compile
20m 27s

the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
javac
20m 27s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
3m 41s

the patch passed

+1 :green_heart:
mvnsite
2m  1s

the patch passed

+1 :green_heart:
javadoc
1m 41s

the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
1m 39s

the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
spotbugs
4m 10s

the patch passed

+1 :green_heart:
shadedclient
21m 10s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 44s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
14m  7s

hadoop-distcp in the patch passed.

+1 :green_heart:
asflicense
1m  3s

The patch does not generate ASF License warnings.

222m 28s

Subsystem
Report/Notes

Docker
ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5391

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux e50831db0918 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 662045ad8a6c5d2c5dff9907a0ac7e6842844153

Default Java
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/4/testReport/

Max. process+thread count
686 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
17m  9s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 3 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 36s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m 40s

trunk passed

+1 :green_heart:
compile
23m  0s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
compile
20m 26s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
checkstyle
3m 45s

trunk passed

+1 :green_heart:
mvnsite
2m  3s

trunk passed

+1 :green_heart:
javadoc
1m 49s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
1m 38s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
spotbugs
4m  2s

trunk passed

+1 :green_heart:
shadedclient
20m 46s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

-1 :x:
mvninstall
0m 15s
/patch-mvninstall-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
compile
12m 30s
/patch-compile-root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt
root in the patch failed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.

-1 :x:
javac
12m 30s
/patch-compile-root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt
root in the patch failed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.

-1 :x:
compile
11m 14s
/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt
root in the patch failed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.

-1 :x:
javac
11m 14s
/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt
root in the patch failed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.

+1 :green_heart:
blanks
0m  1s

The patch has no blanks issues.

-0 :warning:
checkstyle
3m 18s
/results-checkstyle-root.txt
root: The patch generated 1 new + 63 unchanged - 0 fixed = 64 total (was 63)

-1 :x:
mvnsite
0m 22s
/patch-mvnsite-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

-1 :x:
javadoc
0m 21s
/patch-javadoc-hadoop-tools_hadoop-distcp-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt
hadoop-distcp in the patch failed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.

-1 :x:
javadoc
0m 21s
/patch-javadoc-hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt
hadoop-distcp in the patch failed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.

-1 :x:
spotbugs
0m 23s
/patch-spotbugs-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

+1 :green_heart:
shadedclient
21m 52s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 33s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
0m 21s
/patch-unit-hadoop-tools_hadoop-distcp.txt
hadoop-distcp in the patch failed.

+1 :green_heart:
asflicense
0m 40s

The patch does not generate ASF License warnings.

198m 42s

Subsystem
Report/Notes

Docker
ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5391

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 6b1c4e22c346 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / b20d228c6939cded9fb2ca92ef667feb84b5397d

Default Java
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/5/testReport/

Max. process+thread count
648 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 42s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 3 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 13s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m 49s

trunk passed

+1 :green_heart:
compile
22m 59s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
compile
20m 26s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
checkstyle
3m 47s

trunk passed

+1 :green_heart:
mvnsite
2m  4s

trunk passed

+1 :green_heart:
javadoc
1m 48s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
1m 39s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
spotbugs
3m 59s

trunk passed

+1 :green_heart:
shadedclient
20m 50s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 15s

the patch passed

+1 :green_heart:
compile
22m 24s

the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javac
22m 24s

the patch passed

+1 :green_heart:
compile
20m 33s

the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
javac
20m 33s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
3m 38s
/results-checkstyle-root.txt
root: The patch generated 1 new + 62 unchanged - 0 fixed = 63 total (was 62)

+1 :green_heart:
mvnsite
2m  3s

the patch passed

+1 :green_heart:
javadoc
1m 41s

the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
1m 38s

the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
spotbugs
4m 10s

the patch passed

+1 :green_heart:
shadedclient
20m 51s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 42s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
13m 56s

hadoop-distcp in the patch passed.

+1 :green_heart:
asflicense
1m  2s

The patch does not generate ASF License warnings.

221m 26s

Subsystem
Report/Notes

Docker
ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5391

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux fecad139efb7 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / c906c78e3e7aecd12a818fcb76d281f713387323

Default Java
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/6/testReport/

Max. process+thread count
731 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5391/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.


Fix data lost on respawn

Fix the last part of https://github.com/SpongePowered/SpongeCommon/issues/2325

Do we want to run through the offer logic for this? We could take the old manipulator objects wholesale.

Do we want to run through the offer logic for this? We could take the old manipulator objects wholesale.

Yeah, this also affects vanilla manipulators. We need the offer logic to call the correct data processor.


System Check --> cần cấu hình sửa chữa đeer fix nhiều cảnh báo cho SEO Server

Required Private Directories:
⚠ Error: $MATOMO_URLconfig/config.ini.php⚠ Error: $MATOMO_URLtmp/cache/tracker/matomocache_general.php
We found that the above URLs are accessible via the browser, but they should NOT be. Allowing them to be accessed can pose a potential security risk since the contents can provide information about your server and potentially your users. Please restrict access to them.We also found that Matomo's config directory is publicly accessible. While attackers can't read the config now, if your webserver stops executing PHP files for some reason, your MySQL credentials and other information will be available to anyone. Please check your webserver config and deny access to this directory.Read this to learn more.
Recommended Private Directories:
⚠ Error: $MATOMO_URLtmp/⚠ Error: $MATOMO_URLtmp/empty⚠ Error: $MATOMO_URLlang/en.json We found that the above URLs are accessible via the browser, but we recommend they should not be. If possible, please restrict access to them. Read this to learn more.
Set up Cron (faster report-loading):
⚠ Warning: For optimal performance and a speedy Matomo, it is highly recommended to set up a crontab to automatically archive your reports, and to disable browser triggering in the Matomo settings. Learn more.
Max Packet Size:
⚠ Warning: It is recommended to configure a 'max_allowed_packet' size in your MySQL database of at least 64MB. Configured is currently 16MB.
Forced SSL Connection:
⚠ Warning: We recommend using Matomo over secure SSL connections only. To prevent insecure access over http, add force_ssl = 1 to the General section in your Matomo config/config.ini.php file.
Server Info:
⚠ Warning: nginx/1.24.0To ensure that sensitive files cannot be accessed directly it is recommended to configure your web server to restrict access to certain directories. For more information please see the  official nginx server configuration

Đã tham khảo cấu hình https://github.com/matomo-org/matomo-nginx/blob/master/sites-available/matomo.conf
Dựa trên cấu hình file matomo.conf sửa lại cấu hình code:
server {
listen [::]:80; # remove this if you don't want Matomo to be reachable from IPv6
listen 80;
server_name matomo.example.com;
# Redirect all HTTP requests to HTTPS with a 301 Moved Permanently response.
location / {
return 301 https://$host$request_uri;
}
}
server {
listen [::]:443 ssl http2; # remove this if you don't want Matomo to be reachable from IPv6
listen 443 ssl http2;
server_name matomo.example.com; # list all domains Matomo should be reachable from
access_log /var/log/nginx/matomo.access.log;
error_log /var/log/nginx/matomo.error.log;
## uncomment if you want to enable HSTS with 6 months cache
## ATTENTION: Be sure you know the implications of this change (you won't be able to disable HTTPS anymore)
#add_header Strict-Transport-Security max-age=15768000 always;

## replace with your SSL certificate
ssl_certificate /etc/letsencrypt/live/matomo.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/matomo.example.com/privkey.pem;

include ssl.conf; # if you want to support older browsers, please read through this file

add_header Referrer-Policy origin always; # make sure outgoing links don't show the URL to the Matomo instance
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;

root /var/www/matomo/; # replace with path to your matomo instance

index index.php;

## only allow accessing the following php files
location ~ ^/(index|matomo|piwik|js/index|plugins/HeatmapSessionRecording/configs)\.php$ {
    include snippets/fastcgi-php.conf; # if your Nginx setup doesn't come with a default fastcgi-php config, you can fetch it from https://github.com/nginx/nginx/blob/master/conf/fastcgi.conf
    try_files $fastcgi_script_name =404; # protects against CVE-2019-11043. If this line is already included in your snippets/fastcgi-php.conf you can comment it here.
    fastcgi_param HTTP_PROXY ""; # prohibit httpoxy: https://httpoxy.org/
    fastcgi_pass unix:/var/run/php/php7.2-fpm.sock; #replace with the path to your PHP socket file
    #fastcgi_pass <IP_ADDRESS>:9000; # uncomment if you are using PHP via TCP sockets (e.g. Docker container)
}

## deny access to all other .php files
location ~* ^.+\.php$ {
    deny all;
    return 403;
}

## serve all other files normally
location / {
    try_files $uri $uri/ =404;
}

## disable all access to the following directories
location ~ ^/(config|tmp|core|lang) {
    deny all;
    return 403; # replace with 404 to not show these directories exist
}

location ~ /\.ht {
    deny  all;
    return 403;
}

location ~ js/container_.*_preview\.js$ {
    expires off;
    add_header Cache-Control 'private, no-cache, no-store';
}

location ~ \.(gif|ico|jpg|png|svg|js|css|htm|html|mp3|mp4|wav|ogg|avi|ttf|eot|woff|woff2)$ {
    allow all;
    ## Cache images,CSS,JS and webfonts for an hour
    ## Increasing the duration may improve the load-time, but may cause old files to show after an Matomo upgrade
    expires 1h;
    add_header Pragma public;
    add_header Cache-Control "public";
}

location ~ ^/(libs|vendor|plugins|misc|node_modules) {
    deny all;
    return 403;
}

## properly display textfiles in root directory
location ~/(.*\.md|LEGALNOTICE|LICENSE) {
    default_type text/plain;
}

}
vim: filetype=nginx
Được sửa trong code .sh: https://github.com/PhDLeToanThang/seo/blob/master/s2_seo_deploy.sh
#Step 8. Configure NGINX
#Next, you will need to create an Nginx virtual host configuration file to host SEO:
#$ nano /etc/nginx/conf.d/$FQDN.conf
echo 'server {' >> /etc/nginx/conf.d/$FQDN.conf
echo '	listen [::]:80; # remove this if you don't want Matomo to be reachable from IPv6'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  listen 80;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  server_name '${FQDN}';'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  location / {'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  return 301 https://$host$request_uri;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  }'  >> /etc/nginx/conf.d/$FQDN.conf
echo '}'  >> /etc/nginx/conf.d/$FQDN.conf
echo 'server {'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  listen [::]:443 ssl http2;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  listen 443 ssl http2;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  server_name '${FQDN}';'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  access_log /var/log/nginx/${FQDN}.access.log;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  error_log /var/log/nginx/${FQDN}.error.log;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '	ssl_certificate /etc/letsencrypt/live/${FQDN}/fullchain.pem;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  ssl_certificate_key /etc/letsencrypt/live/${FQDN}/privkey.pem;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  include /etc/letsencrypt/options-ssl-nginx.conf;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  add_header Strict-Transport-Security "max-age=31536000" always;'  >> /etc/nginx/conf.d/$FQDN.conf
echo ' 	add_header Referrer-Policy origin always;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  add_header X-Content-Type-Options "nosniff" always;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  add_header X-XSS-Protection "1; mode=block" always;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  root /var/www/html/${FQDN};'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  index index.php;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  ocation ~ ^/(index|matomo|piwik|js/index|plugins/HeatmapSessionRecording/configs).php$ {'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  include snippets/fastcgi-php.conf;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  fastcgi_param HTTP_PROXY "";'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  fastcgi_pass unix:/var/run/php/php8.3-fpm.sock;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  }'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  location ~* ^.+.php$ {'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  deny all;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  return 403;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  }'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  location / {'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  try_files $uri $uri/ =404;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  }'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  location ~ ^/(config|tmp|core|lang) {'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  deny all;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  return 403;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  }'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  location ~ /.ht {'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  deny  all;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  return 403;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  }'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  location ~ js/container_._preview.js$ {'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  expires off;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  add_header Cache-Control 'private, no-cache, no-store';'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  }'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  location ~ .(gif|ico|jpg|png|svg|js|css|htm|html|mp3|mp4|wav|ogg|avi|ttf|eot|woff|woff2)$ {'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  allow all;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  expires 1h;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  add_header Pragma public;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  add_header Cache-Control "public";'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  }'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  location ~ ^/(libs|vendor|plugins|misc|node_modules) {'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  deny all;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  return 403;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  }'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  location ~/(..md|LEGALNOTICE|LICENSE) {'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  default_type text/plain;'  >> /etc/nginx/conf.d/$FQDN.conf
echo '  }'  >> /etc/nginx/conf.d/$FQDN.conf
echo '}'  >> /etc/nginx/conf.d/$FQDN.conf
#Save and close the file then verify the Nginx for any syntax error with the following command:
nginx -t


Reintroduce Autotext button

Autotext button can be reintroduced:
Once clicked, it auto creates a CVE description based on other fields as much as possible.
If the current description is still has the initial value:

update the current description

If it has been modified:

shows a popup modal with the suggested CVE description is copied to clipboard and manually pasted to description.

Button reintroduced. Generates a description based on other fields, except affected versions details.