feat: added GlobalFooter molecule

Overview

Adds a new GlobalFooter component to complement the existing GlobalHeader.

The footer in the monolith sources a lot of its data dynamically by querying external data sources, but rarely changes -- maybe 2-3 times a year to my recollection. I don't think implementing fetching that data in all consumers is worth the effort. Static data here instead, I say!
Monolith PR: https://github.com/codecademy-engineering/Codecademy/pull/23563
PR Checklist

[x] Related to designs: https://www.figma.com/file/as1W6qIWUlibYKiSUWB2S4/Global-Navigation-(Top-Nav-%2B-Footer)-Guidelines?node-id=0%3A1
[x] Related to JIRA ticket: REACH-836
[x] I have run this code to verify it works
[x] This PR includes unit tests for the code change

📬Published Alpha Packages:@codecademy/gamut-kit@0.2.47-alpha.48ae43.0@codecademy/gamut-labs@13.3.0-alpha.48ae43.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://606b4d84263526483943bd07--gamut-preview.netlify.appDeploy Logs

YES. 🙌
Haven't looked at the code yet but here are a few small visual things I noticed on Storybook:

Some spacing issues between the top headers and the top divider, and spacing above the "SUPPORT" heading

For mobile screens the Languages columns should be separated out to match the columns in the section above

The tabbing order is off for the links; it currently skips around a bit and should flow Company -> Resources -> Community -> Mobile -> Individual -> Enterprise -> Support -> Subjects -> Languages

I am pumped!!!

Thanks for the spotting @katiezutter! I think I fixed what you spotted out, and double-checked & did a few touchups mobile up. It should be clean now. 🤞

📬Published Alpha Packages:@codecademy/gamut-kit@0.2.47-alpha.a6578a.0@codecademy/gamut-labs@13.3.0-alpha.a6578a.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://606b601c81af7d007ac377dc--gamut-preview.netlify.appDeploy Logs

nit: inconsistent focus-visible on some of the links (ex. most are outlined purple but then the social media icons are navy (is that by design?)

📬Published Alpha Packages:@codecademy/gamut-kit@0.2.47-alpha.03351b.0@codecademy/gamut-labs@13.3.0-alpha.03351b.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://606cd01f0f1c7912bfc859d8--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.2.47-alpha.7a48ef.0@codecademy/gamut-labs@13.3.0-alpha.7a48ef.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://606cd2f68ee5e81c8cff73fb--gamut-preview.netlify.appDeploy Logs

@katiezutter @saghdaey I think I've gotten to everything -including onClick and tab order- but apologies if I missed a comment!

📬Published Alpha Packages:@codecademy/gamut-kit@0.2.47-alpha.cd0539.0@codecademy/gamut-labs@13.3.0-alpha.cd0539.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://606cd8477df3df1cf43514d2--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.2.47-alpha.4adb36.0@codecademy/gamut-labs@13.3.0-alpha.4adb36.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://606dc8b7c480bf2cee05eabe--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.2.47-alpha.0661f3.0@codecademy/gamut-labs@13.3.0-alpha.0661f3.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://606dcd1da980f731878e65b9--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.2.47-alpha.fe9f27.0@codecademy/gamut-labs@13.3.0-alpha.fe9f27.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://606df12449839700c604d08d--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.2.47-alpha.cfb8b1.0@codecademy/gamut-labs@13.3.0-alpha.cfb8b1.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://606e0b6150c50e253d52c06b--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.2.47-alpha.fb7593.0@codecademy/gamut-labs@13.3.0-alpha.fb7593.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://606e1f332144f23fceb7762b--gamut-preview.netlify.appDeploy Logs

This looks great other than 3 minor regressions in accessibility for the footer here. (You can refer to this PR to see how the code handled it before). It's also live in Prod.

The social icons are grouped as a single list item in the Company list. They should be their own list, so the company list should have 3 items, and then tabbing from "Shop" to the "Twitter icon" would label it as a "Social Media list 4 items".

The download on mobile images are no longer a list

In the Course Catalog "subjects" list, the list isn't labeled as "subjects" anymore, and there's a divider represented by "—" that is incorrectly counted as a list item (it's not focusable, so it shouldn't be one). This means the list reports 11 items instead of the 10 selectable ones.

I started a separate conversation in Slack with some of the (very good) points @ZackMFleischman brought up, since I'm not sure which way is best: https://codecademy.slack.com/archives/CGQPTPR27/p1617996737046900
Either way for now I suspect we can merge, since none of this blocks screenreader users from easily using the footer! Woop!

Very cool. I am taking a github pull request course right now! So cool to see collaboration in action!


nixos/statsd: refactor test

Motivation for this change
statsd is a daemon written in node to gather statistics over UDP.
The current test ensures that a port is open, but the basic
functionality isn't sufficiently tested.
This patch contains the following changes:

Simplified port scanning (waitForOpenPort rather than netcat magic).

Issue a TCP command to check the health of the statsd server.

Simple script to check if statsd receives data over UDP and confirms
the basic functionality of the TCP interface on 8126 for admin
purposes.

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

@xeji increased the timeouts to 120 minutes (-w 120), thanks for your feedback %)

@xeji yeah it doesn't wait as well on my side. Thanks for your help!

@GrahamcOfBorg test statsd

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: tests.statsd
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Attempted: tests.statsd
Partial log (click to expand)
machine: exit status 1
syncing
machine: running command: sync
machine: exit status 0
test script finished in 14.21s
cleaning up
killing machine (pid 593)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/z90dlkgm8mrp13qixhi36ym14lkkz9sy-vm-test-run-statsd

14 seconds - looks good :smiley:  - thanks for improving this test!


[Issue]: The "no-install" variant of this software does not work for several reasons.

What happened?

After unzipping the "no-install" variant, the software does not work, unless you find out that you need to copy and adapt the settings.txt from the examples directory (no mention about this in the README)
After adding the settings.txt, installation works, but nodejs is not installed in the path but in the subdirectory of root. (if that's intended, then what exactly is path for? Again: no documentation.)
nvm use does not work without administrator privileges (this defeats the whole purpose of a no-install distribution, because you usually want that, because you don't want to (or cannot) install something as admin.
It's entirely unclear what nvm use actually does, because it apparently does not add the directory of the node installation to PATH.

What did you expect to happen?

Unzipping the software should be enough to run it. If it's not, running the software should either create a default (working) settings, prompt for additional settings, or terminate with a reference to the manual where to read about settings that have to be created.
I expected the nodejs installation to be installed into the nodejs directory I have specified with the path setting. But that might be just a misinterpretation. In any case, settings shall be documented in the README.
nvm should not request admin privileges where it doesn't need to. A user only installation and use of the software should be possible.
The effect of nvm use should be, that I can run the node executable from anywhere in my system - in other words: the path of the nodejs installation that is going to be used, should be added to the PATH variable.

Version
1.1.11 or newer (Default)
Which version of Windows?
Windows 11+
Which locale?
None
Which shell are you running NVM4W in?
Command Prompt
User Permissions?
Standard Use, Non-Elevated
Is Developer Mode enabled?
None
Relevant log/console output
No response
Debug Output
is not using admin or elevated rights.
Error getting console handle: Der Vorgang wurde erfolgreich beendet.
Windows Version: 10.0 (Build 22621)

Windows Developer Mode: UNKNOWN (user cannot read registry)

NVM4W Version:      1.1.12
NVM4W Path:         C:\Development\Software\nvm\nvm.exe
NVM4W Settings:     C:\Development\Software\nvm\settings.txt
NVM_HOME:           C:\Development\Software\nvm
NVM_SYMLINK:        .
Node Installations: C:\Development\Software\nvm

Total Node.js Versions: 1
Active Node.js Version: none
(run "nvm use <version>" to activate a version)

PROBLEMS DETECTED
-----------------
Another Node.js installation is blocking NVM4W installations from running. Please uninstall the conflicting version or update the PATH environment variable to assure "." precedes "C:\Development\Software\nvm\v18.19.0".

The NVM4W symlink (.) was not found in the PATH environment variable.

NVM_SYMLINK (.) is not a valid symlink.

Connection type cannot be determined: dial tcp [::1]:80: connectex: Es konnte keine Verbindung hergestellt werden, da der Zielcomputer die Verbindung verweigerte.

Find help at https://github.com/coreybutler/nvm-windows/wiki/Common-Issues

Anything else?
The problem reported by nvm debug is maybe due to my manual addition of the nodejs installation to PATH. I don't see why this should be a problem. It is extremely interesting, that the software identifies the installation it has installed previously as a conflicting installation. That should not happen.

To be clear, the no-install option is not a variant of the application. It doesn't ship with an installer, but the app still needs to be installed in order to run. The no-install option is for people who need to manually create their own installers, such as a custom package to roll out via Active Directory.
Did you just skim the README, or did you read the no-install/manual guide? It is linked from the README (shown here):

The docs clearly state you need to create a settings.txt file and provide a link to the example:

It is expected that physical node installations are stored in the root, not in the PATH. The NVM_SYMLINK is supposed to be in the PATH. The NVM_SYMLINK changes when nvm use is run, relinking to the appropriate physical installation directory. This is spelled out in the README as the major distinction of this tool (https://github.com/coreybutler/nvm-windows#bulb-whats-the-big-difference):

Windows requires elevated privileges to symlink directories. Administrators have these privileges, as does anyone who has developer mode enabled. You can also explicitly grant symlink permissions to trusted users. Microsoft has more information about this here. There are also more tips in the common issues section of this project.
Please remember that installing multiple versions of Node.js is exactly that... installing software on the OS. Microsoft has safeguards in place to protect against unauthorized installations. This tool is designed to find a balance between these restrictions and developer freedom, but every user/organization's threat tolerance is different.
If you are seeking more enterprise-specific features, I recommend following the news for Runtime, the successor software we're currently building.

As per your last point about how the PATH should point to the Node installation: This was how one of the very first alpha editions handled it. It proved to be dangerous. If something fails or there is an odd path (custom character) added to the PATH, or even if the path is long, it can truncate the PATH environment variable leading to system-wide loss of other applications (or worse, system failure). Constantly changing the PATH whenever a different version of Node is required can cause race conditions between apps too. Throughout the issue history in this repo, I refer to this as PATH thrashing. This is why a symlink is used instead. We can "thrash" the symlink because it is limited to the NVM4W app, isolated from the rest of the system.
Additionally, the debugger appears to be thrown off by the invalid settings.txt. Use the example settings file as a guide.


fix: modifying offset of secondary nav for edu

Checklist

[x] Include a description of your pull request and instructions for the reviewer to verify your work.
[x] Link to the issue if this PR is issue-specific.
[ ] Create/update the corresponding story if this includes a UI component.
[ ] Create/update documentation. If not included, tell us why.
[ ] List the environments / browsers in which you tested your changes.
[ ] Tests, linting, or other required checks are passing.
[x] PR has an informative and human-readable title

PR titles are used to generate the change log in releases; good ones make that easier to scan.
PRs will be broadly categorized in the change log, but for even easier scanning, consider prefixing with a component name or other useful categorization, e.g., "BaseButton: fix layout bug", or "Storybook: Update dependencies".

[x] PR has been tagged with a SemVer label and a general category label, or skip-changelog.

These tags are used to do the aforementioned broad categorization in the change log and determine what the next release's version number should be.
Release Drafter will attempt to do the category labeling for you! Please double-check its work.

Description
Supports https://github.com/nasa-jpl/www/issues/244
Modifies secondary nav styles to have the correct offset when scrolling up (the EDU header is shorter than the main WWW header).
Instructions to test

Tested in the following environments/browsers:

Operating System

[ ] macOS
[ ] iOS
[ ] iPadOS
[ ] Windows

Browser

[ ] Chrome
[ ] Firefox ESR
[ ] Firefox
[ ] Safari
[ ] Edge

Hmm, this fix doesn't seem to be effective anymore. I've created a new issue to capture it: https://github.com/nasa-jpl/www/issues/254


No resource or method named `shell_out'

No resource or method named `shell_out' for `Chef::Provider::DockerService::Upstart ""'

With the latest versions (>v1.0.40) this is a problem again.
Old issue: #353

What version of chef are you using?
On Oct 5, 2015 7:54 AM, "Michael Schubert"<EMAIL_ADDRESS>wrote:

No resource or method named shell_out' for Chef::Provider::DockerService::Upstart ""'
With the latest versions (>v1.0.40) this is a problem again.
Old issue: #353 https://github.com/bflad/chef-docker/issues/353
—
Reply to this email directly or view it on GitHub
https://github.com/bflad/chef-docker/issues/479.

11.18.12 (with no_lazy_load set to true). So strictly speaking an unsupported version, but working until 68fe993bdd3fc61d3a2c654ea9af07231ede609f

This cookbook only supports Chef 12. Please upgrade your chef-client version.


[Snyk] Upgrade cli-table3 from 0.6.4 to 0.6.5

This PR was automatically created by Snyk using the credentials of a real user.![snyk-top-banner](https://github.com/andygongea/OWASP-Benchmark/assets/818805/c518c423-16fe-447e-b67f-ad5a49b5d123)
Snyk has created this PR to upgrade cli-table3 from 0.6.4 to 0.6.5.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 1 version ahead of your current version.

The recommended version was released on 2 months ago.

Release notes

Package name: cli-table3

0.6.5 - 2024-05-12What's Changed

add word wrap options to CellOptions ts interface by @ ItsNickBarry in #339
support bigint cell content type by @ ItsNickBarry in #340
Fix truncated href tags by @ ItsNickBarry in #342

New Contributors

@ ItsNickBarry made their first contribution in #339

Full Changelog: v0.6.4...v0.6.5

0.6.4 - 2024-03-23What's Changed

chore: dependabot schedule from daily to weekly by @ speedytwenty in #317
fix(types): add missed 'href' property by @ elaichenkov in #321

New Contributors

@ elaichenkov made their first contribution in #321

Full Changelog: v0.6.3...v0.6.4

    from cli-table3 GitHub release notes

[!IMPORTANT]

Check the changes in this PR to ensure they won't cause issues with your project.
This PR was automatically created by Snyk using the credentials of a real user.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:

🧐 View latest project report
📜 Customise PR templates
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

Already updated in previous PR


Sidebar does not work well when Resized

Currently, sidebar has the range width/height hardcoded.
While it's okay if the sidebar's size stays static, but I encountered issues when closing/opening sidebar after sidebar is resized using jquery's resizable.
https://jqueryui.com/resizable/
It would be great if the sidebar's width/height be dynamic instead of hardcoded.

Sounds like a bug. :bug:
Could you please create a reproduction page (JSFiddle or so), so I can see what's going on? Thanks!

I would be keen to call this a 'feature request', not a bug. =)
you can replicate it like this.
$('#lhs_toc').resizable()
$("#lhs_toc").sidebar();
now open/close the sidebar.. works great
now resize the sidebar
now open/close the sidebar... and the logic is busted.

Ah, I see what you mean. :+1:
However, #15 will fix the issue regarding browser window resizing, right?
It's more a bug. :smile:

I can't see browser window resizing causing issues with sidebar.
The issue lies when the sidebar's dimensions (i.e. width/height) is altered using resizable().

It does create issues when the sidebar has percentage size.

Fixed via #15. :sparkles:

I can't see browser window resizing causing issues with sidebar.

@teamrc The percentage size bug was fixed in #23 (:ship: 3.3.0). :tada:


dive bridge btp --chainA icon --chainB eth fails at step 128 with no output

❯ dive clean
INFO[2023-08-08 20:10:22] Successfully connected to kurtosis engine...
INFO[2023-08-08 20:10:22] Initializing cleaning process...
INFO[2023-08-08 20:10:24] Successfully destroyed and cleaned enclave dive
❯ dive bridge btp --chainA icon --chainB eth
_______\ "Execution in progress"  total_steps:190  current_step_number:128%
stack trace
DEBU[2023-08-09T00:17:54Z][extractor.go:extract] Running extractor against query '{"jsonrpc":"2.0","error":{"code":-31003,"message":"Executing: Executing"},"id":1}
' and input '.result.scoreAddress'
DEBU[2023-08-09T00:17:54Z][extractor.go:extract] Found match ''
DEBU[2023-08-09T00:17:56Z][startosis_runner.go:forwardKurtosisResponseLineChannelUntilSourceIsClosed] Received kurtosis execution line Kurtosis:
error:{execution_error:{error_message:"An error occurred executing instruction (number 128) at github.com/hugobyte/dive/services/jvm/icon/src/node-setup/contract_deploy.star[41:23]:\nwait(service_name="icon-node-0", recipe=PostHttpRequestRecipe(port_id="rpc", endpoint="/api/v3/icon_dex", body="{ \"jsonrpc\": \"2.0\", \"method\": \"icx_getTransactionResult\", \"id\": 1, \"params\": { \"txHash\": {{kurtosis:beea38dd67634f66aec6e8882dcb9b90:output.runtime_value}} } }", content_type="application/json", extract={"score_address": ".result.scoreAddress"}), field="code", assertion="==", target_value=200)\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/startosis_executor.go:130 (sendErrorAndFail) ---\nCaused by: An error occurred checking if service 'icon-node-0' is ready.\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/kurtosis_instruction/wait/wait.go:257 (WaitCapabilities.Execute) ---\nCaused by: Recipe execution timed-out waiting for the recipe execution to become valid on service 'icon-node-0'. Tried '10' times. Last recipe execution error was:\n '$An error occurred when running HTTP request recipe 'PostHttpRequestRecipe(port_id="rpc", endpoint="/api/v3/icon_dex", body="{ \"jsonrpc\": \"2.0\", \"method\": \"icx_getTransactionResult\", \"id\": 1, \"params\": { \"txHash\": {{kurtosis:beea38dd67634f66aec6e8882dcb9b90:output.runtime_value}} } }", content_type="application/json", extract={"score_address": ".result.scoreAddress"})'\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/recipe/post_http_request_recipe.go:175 (PostHttpRequestRecipe.Execute) ---\nCaused by: An error occurred while running extractors from HTTP recipe\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/recipe/http_request_recipe.go:95 (executeInternal) ---\nCaused by: An error occurred running extractor '.result.scoreAddress' on recipe\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/recipe/extractor.go:90 (runExtractors) ---\nCaused by: No field '.result.scoreAddress' was found on input '{"jsonrpc":"2.0","error":{"code":-31003,"message":"Executing: Executing"},"id":1}\n'\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/recipe/extractor.go:44 (extract) ---'\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/kurtosis_instruction/shared_helpers/service_helpers.go:111 (executeServiceAssertionWithRecipeWithTicker) ---"}}
DEBU[2023-08-09T00:17:56Z][api_container_service.go:runStarlark] Received response line from Starlark runner: 'error:{execution_error:{error_message:"An error occurred executing instruction (number 128) at github.com/hugobyte/dive/services/jvm/icon/src/node-setup/contract_deploy.star[41:23]:\nwait(service_name="icon-node-0", recipe=PostHttpRequestRecipe(port_id="rpc", endpoint="/api/v3/icon_dex", body="{ \"jsonrpc\": \"2.0\", \"method\": \"icx_getTransactionResult\", \"id\": 1, \"params\": { \"txHash\": {{kurtosis:beea38dd67634f66aec6e8882dcb9b90:output.runtime_value}} } }", content_type="application/json", extract={"score_address": ".result.scoreAddress"}), field="code", assertion="==", target_value=200)\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/startosis_executor.go:130 (sendErrorAndFail) ---\nCaused by: An error occurred checking if service 'icon-node-0' is ready.\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/kurtosis_instruction/wait/wait.go:257 (WaitCapabilities.Execute) ---\nCaused by: Recipe execution timed-out waiting for the recipe execution to become valid on service 'icon-node-0'. Tried '10' times. Last recipe execution error was:\n '$An error occurred when running HTTP request recipe 'PostHttpRequestRecipe(port_id="rpc", endpoint="/api/v3/icon_dex", body="{ \"jsonrpc\": \"2.0\", \"method\": \"icx_getTransactionResult\", \"id\": 1, \"params\": { \"txHash\": {{kurtosis:beea38dd67634f66aec6e8882dcb9b90:output.runtime_value}} } }", content_type="application/json", extract={"score_address": ".result.scoreAddress"})'\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/recipe/post_http_request_recipe.go:175 (PostHttpRequestRecipe.Execute) ---\nCaused by: An error occurred while running extractors from HTTP recipe\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/recipe/http_request_recipe.go:95 (executeInternal) ---\nCaused by: An error occurred running extractor '.result.scoreAddress' on recipe\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/recipe/extractor.go:90 (runExtractors) ---\nCaused by: No field '.result.scoreAddress' was found on input '{"jsonrpc":"2.0","error":{"code":-31003,"message":"Executing: Executing"},"id":1}\n'\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/recipe/extractor.go:44 (extract) ---'\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/kurtosis_instruction/shared_helpers/service_helpers.go:111 (executeServiceAssertionWithRecipeWithTicker) ---"}}'

+1 to this
I've hit this multiple times with similar errors stemming from github.com/hugobyte/dive/services/jvm/icon/src/node-setup/contract_deploy.star as well.

@CyrusVorwald  and @leeederek  is the issues still persist ?

Yes.
INFO   [2023-08-17 15:36:14] progress_info:{current_step_info:"Execution in progress" total_steps:191 current_step_number:129}
INFO   [2023-08-17 15:36:14] instruction:{position:{filename:"github.com/hugobyte/dive/services/jvm/icon/src/node-setup/contract_deploy.star" line:41 column:23} instruction_name:"wait" arguments:{serialized_arg_value:""icon-node-0xacbc4e"" arg_name:"service_name"} arguments:{serialized_arg_value:"PostHttpRequestRecipe(port_id="rpc", endpoint="/api/v3/icon_dex", body="{ \"jsonrpc\": \"2.0\", \"method\": \"icx_getTransactionResult\", \"id\": 1, \"params\": { \"txHash\": {{kurtosis:2a024810869242928178ea3dfc59dbe8:output.runtime_value}} } }", content_type="application/json", extract={"score_address": ".result.scoreAddress"})" arg_name:"recipe" is_representative:true} arguments:{serialized_arg_value:""code"" arg_name:"field" is_representative:true} arguments:{serialized_arg_value:""=="" arg_name:"assertion" is_representative:true} arguments:{serialized_arg_value:"200" arg_name:"target_value" is_representative:true} arguments:{} arguments:{} executable_instruction:"wait(service_name="icon-node-0xacbc4e", recipe=PostHttpRequestRecipe(port_id="rpc", endpoint="/api/v3/icon_dex", body="{ \"jsonrpc\": \"2.0\", \"method\": \"icx_getTransactionResult\", \"id\": 1, \"params\": { \"txHash\": {{kurtosis:2a024810869242928178ea3dfc59dbe8:output.runtime_value}} } }", content_type="application/json", extract={"score_address": ".result.scoreAddress"}), field="code", assertion="==", target_value=200)"}
INFO   [2023-08-17 15:36:25] error:{execution_error:{error_message:"An error occurred executing instruction (number 129) at github.com/hugobyte/dive/services/jvm/icon/src/node-setup/contract_deploy.star[41:23]:\nwait(service_name="icon-node-0xacbc4e", recipe=PostHttpRequestRecipe(port_id="rpc", endpoint="/api/v3/icon_dex", body="{ \"jsonrpc\": \"2.0\", \"method\": \"icx_getTransactionResult\", \"id\": 1, \"params\": { \"txHash\": {{kurtosis:2a024810869242928178ea3dfc59dbe8:output.runtime_value}} } }", content_type="application/json", extract={"score_address": ".result.scoreAddress"}), field="code", assertion="==", target_value=200)\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/startosis_executor.go:130 (sendErrorAndFail) ---\nCaused by: An error occurred checking if service 'icon-node-0xacbc4e' is ready.\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/kurtosis_instruction/wait/wait.go:257 (WaitCapabilities.Execute) ---\nCaused by: Recipe execution timed-out waiting for the recipe execution to become valid on service 'icon-node-0xacbc4e'. Tried '11' times. Last recipe execution error was:\n '$An error occurred when running HTTP request recipe 'PostHttpRequestRecipe(port_id="rpc", endpoint="/api/v3/icon_dex", body="{ \"jsonrpc\": \"2.0\", \"method\": \"icx_getTransactionResult\", \"id\": 1, \"params\": { \"txHash\": {{kurtosis:2a024810869242928178ea3dfc59dbe8:output.runtime_value}} } }", content_type="application/json", extract={"score_address": ".result.scoreAddress"})'\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/recipe/post_http_request_recipe.go:175 (PostHttpRequestRecipe.Execute) ---\nCaused by: An error occurred when running HTTP request recipe\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/recipe/http_request_recipe.go:76 (executeInternal) ---\nCaused by: An error occurred when getting service 'icon-node-0xacbc4e' for HTTP request\n --- at /home/circleci/project/core/server/api_container/server/service_network/default_service_network.go:604 (DefaultServiceNetwork.HttpRequestService) ---\nCaused by: An error occurred getting service 'b969dd0fbe2d470baa2484cdda4b85cf'\n --- at /home/circleci/project/core/server/api_container/server/service_network/default_service_network.go:645 (DefaultServiceNetwork.GetService) ---\nCaused by: An error occurred getting user services in enclave 'd92bf1eb449f4e97ba5d11f0e8f8c72b' using filters '&{Names:map[] UUIDs:map[b969dd0fbe2d470baa2484cdda4b85cf:true] Statuses:map[]}'\n --- at /home/circleci/project/container-engine-lib/lib/backend_impls/metrics_reporting/metrics_reporting_kurtosis_backend.go:278 (MetricsReportingKurtosisBackend.GetUserServices) ---\nCaused by: An error occurred getting user services matching filters '&{Names:map[] UUIDs:map[b969dd0fbe2d470baa2484cdda4b85cf:true] Statuses:map[]}'\n --- at /home/circleci/project/container-engine-lib/lib/backend_impls/docker/docker_kurtosis_backend/user_services_functions/get_user_services.go:23 (GetUserServices) ---\nCaused by: An error occurred getting matching user service resources\n --- at /home/circleci/project/container-engine-lib/lib/backend_impls/docker/docker_kurtosis_backend/shared_helpers/shared_helpers.go:313 (GetMatchingUserServiceObjsAndDockerResourcesNoMutex) ---\nCaused by: An error occurred getting user service containers in enclave 'd92bf1eb449f4e97ba5d11f0e8f8c72b' by labels: map[com.kurtosistech.app-id:kurtosis com.kurtosistech.container-type:user-service com.kurtosistech.enclave-id:d92bf1eb449f4e97ba5d11f0e8f8c72b]\n --- at /home/circleci/project/container-engine-lib/lib/backend_impls/docker/docker_kurtosis_backend/shared_helpers/shared_helpers.go:521 (getMatchingUserServiceDockerResources) ---\nCaused by: An error occurred getting containers with labels '{fields:map[label:map[com.kurtosistech.app-id=kurtosis:true com.kurtosistech.container-type=user-service:true com.kurtosistech.enclave-id=d92bf1eb449f4e97ba5d11f0e8f8c72b:true]]}'\n --- at /home/circleci/project/container-engine-lib/lib/backend_impls/docker/docker_manager/docker_manager.go:1111 (DockerManager.GetContainersByLabels) ---\nCaused by: An error occurred getting the docker containers with filter args '{fields:map[label:map[com.kurtosistech.app-id=kurtosis:true com.kurtosistech.container-type=user-service:true com.kurtosistech.enclave-id=d92bf1eb449f4e97ba5d11f0e8f8c72b:true]]}'\n --- at /home/circleci/project/container-engine-lib/lib/backend_impls/docker/docker_manager/docker_manager.go:1620 (DockerManager.getContainersByFilterArgs) ---\nCaused by: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.43/containers/json?all=1&filters=%7B%22label%22%3A%7B%22com.kurtosistech.app-id%3Dkurtosis%22%3Atrue%2C%22com.kurtosistech.container-type%3Duser-service%22%3Atrue%2C%22com.kurtosistech.enclave-id%3Dd92bf1eb449f4e97ba5d11f0e8f8c72b%22%3Atrue%7D%7D": context deadline exceeded'\n --- at /home/circleci/project/core/server/api_container/server/startosis_engine/kurtosis_instruction/shared_helpers/service_helpers.go:111 (executeServiceAssertionWithRecipeWithTicker) ---"}}
FATAL  [2023-08-17 15:36:37] Failed To Stop ServicesFound 1 validation errors: Error while validating instruction stop_service(name="prelaunch-data-generator-cl-validator-keystore"). The instruction can be found at DEFAULT_PACKAGE_ID_FOR_SCRIPT[3:19]
Caused by: There was an error validating 'stop_service' as service name 'prelaunch-data-generator-cl-validator-keystore' doesn't exist
--- at /root/go/pkg/mod/github.com/kurtosis-tech/kurtosis/api/golang@v0.81.6/core/lib/enclaves/enclave_context.go:448 (getErrFromStarlarkRunResult) ---

@CyrusVorwald please share the version of DIVE cli that you are using  and also the version of kurtosis CLI that you are using .

@CyrusVorwald
Please try once again with :-

kurtosis Version:              0.81.8
DIVE version                     0.0.7-beta

This is with those versions.
❯ kurtosis version
CLI Version:              0.81.8
Running Engine Version:   0.81.8
❯ dive version
CLI version - v0.0.7-beta

Hi @CyrusVorwald
this issue is fixed with the latest release v0.0.7 beta
closing this issue for now
if the issue still persists on your end we will re-open the issue


[Event Request] CodeUnit 7302 - WMS Management - OnBeforeCheckItemTrackingChange

Hi, would it be possible to get an event OnBeforeCheckItemTrackingChange codeunit 7302, procedure CheckItemTrackingChange, like this :

    procedure CheckItemTrackingChange(TrackingSpecification: Record "Tracking Specification"; xTrackingSpecification: Record "Tracking Specification")
    var
    IsHandled: Boolean;
    begin
        IsHandled:=false;
        OnBeforeCheckItemTrackingChange(TrackingSpecification, xTrackingSpecification, IsHandled);
        if not IsHandled then
        exit;
        with TrackingSpecification do
            if ("Source Type" = DATABASE::"Item Journal Line") and
               ("Item No." <> '') and
               ("Location Code" <> '')
            then begin
                if "Source Subtype" in [0, 1, 2, 3] then
                    if not HasSameTracking(xTrackingSpecification) or
                       ((xTrackingSpecification."Expiration Date" <> 0D) and
                        ("Expiration Date" <> xTrackingSpecification."Expiration Date")) or
                       ("Quantity (Base)" <> xTrackingSpecification."Quantity (Base)")
                    then begin
                        GetLocation("Location Code");
                        if Location."Directed Put-away and Pick" then begin
                            GetItem("Item No.");
                            if ItemTrackingCode.IsWarehouseTracking() then
                                Error(Text013,
                                  LowerCase(Item.TableCaption),
                                  LowerCase(Location.TableCaption), Location.Code, Location.FieldCaption("Directed Put-away and Pick"));
                        end;
                    end;

    [IntegrationEvent(false, false)]
    local procedure OnBeforeCheckItemTrackingChange(var TrackingSpecification: Record "Tracking Specification";var xTrackingSpecification: Record "Tracking Specification"; var IsHandled: Boolean)
    begin

Thanks

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


Input/Output format detection based on file extension

Currently setting an input/output SBOM format other than the default requires specifying a flag. However, most of the common specifications seem to have a preferred file extension that we could use to automatically determine the format -- e.g. SPDX JSON files often use spdx.json.
It should be possible to override the automatically detected format choice if needed, but this could make things a bit more user friendly.

See https://github.com/ossf/sbom-everywhere/blob/main/reference/sbom_naming.md for some naming convention guidelines.


RMarkdown file is not rendered correctly for flexdashboard output

Having as output flexdashboard the rmarkdown file is not rendered correctly (as rendered in RStudio).

title: "Review Dashboard"
output:
flexdashboard::flex_dashboard:
orientation: columns

used rmarkdown::run instead of rmarkdown::render and works ok


Internal siphash tests are not run.

Checklist

[ ] documentation is added or updated
[x] tests are added or updated

Description of change

Applied. Thanks.


Build on AppVeyor fails -- missing FSharp.Core

I use a project created from ProjectScaffold.
Building on Travis works OK, but building on AppVeyor fails with the following error:
1) System.IO.FileNotFoundException: Could not load file or assembly 'FSharp.Core, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

full log on AppVeyor
what am I doing wrong? builds of ProjectScaffold itself seem to be doing OK.

Does your appveyor config use VS2013 or VS2015?

According to build log FSharpLint could not find FSharp.Core.
It is not directly related to ProjectScaffold because we have no dependency to FSharpLint now.
//cc @duckmatt

@sergey-tihon is correct, that's coming from FSharpLint not anything in this project, try upgrading to at least 0.2.5 to fix

As I understand @sideeffffect uses FSharpLint (0.2.6) https://github.com/sideeffffect/FSharpHello/blob/master/paket.lock#L16

The linked build is 21 days old, fingers crossed it would work now on the current master
FSharpLint 0.2.3 unzipped to C:\projects\fsharphello\packages\FSharpLint

how can I try building on AppVeyor with FSharpLint from master?

Sorry I meant building the master branch of your project with FSharpLint >=0.2.5. From your latest builds on app veyor that seems to be the case as it's running as expected:
Successfully linted C:\projects\fsharphello\tests\FSharpHello.Tests\FSharpHello.Tests.fsproj. Linted 1 files and found 1 warnings.

But the build is now failed by a broken test FSharpHello.Tests.hello: https://ci.appveyor.com/project/sideeffffect/fsharphello/build/<IP_ADDRESS>

good, the 'missing FSharp.Core' error seems to be gone. now it fails with the following message.
I have no idea what's going wrong. should I create a new issue for this?
Errors and Failures:
1) Test Error : FSharpHello.Tests.hello returns 42
   System.MissingMethodException : Method not found: 'Void FsUnit.TopLevelOperators.should(Microsoft.FSharp.Core.FSharpFunc`2<!!0,!!1>, !!0, System.Object)'.
   at FSharpHello.Tests.hello returns 42()

Running build failed.
Error:
Fake.UnitTestCommon+FailedTestsException: NUnit test failed (1).
   at Fake.NUnitSequential.NUnit(FSharpFunc`2 setParams, IEnumerable`1 assemblies) in C:\code\fake\src\app\FakeLib\UnitTest\NUnit\Sequential.fs:line 45
   at FSI_0001.Build.clo@137-11.Invoke(Unit _arg7) in C:\projects\fsharphello\build.fsx:line 138
   at Fake.TargetHelper.runSingleTarget(TargetTemplate`1 target) in C:\code\fake\src\app\FakeLib\TargetHelper.fs:line 411

This is an issue with FsUnit: https://github.com/fsprojects/FsUnit/issues/23

this is fixed


Regularizing Makefile structure and adding GCP image generation

This is the final piece of the puzzle that gets rid of all the special casing for RPi in our Makefile (the only two left are CONF_FILES and PART_SPEC and even those are likely to go away real soon now).
This also introduces a new target called live-gcp that generates a Google Cloud compatible live EVE image. That, in turn, enables the following workflow:
make live-gcp
linuxkit push gcp -img-name live-eve -bucket lf-edge-eve -project zededa-sandbox -nested-virt dist/amd64/live.img.tar.gz
linuxkit run gcp -project zededa-sandbox -zone us-west1-a -machine n1-standard-2 live-eve

Not sure if it makes sense to wrap the two commands in a makefile target of some kind, but it seems to work as is (maybe we should teach linuxkit to take those values from some local configuration file instead -- what do you think @deitch ?)

Let me see if I get this. The targets are:

make live-% - will make for one of the supported images, right now: raw,gcp,qcow2 (based on this line)
make live - this is just a shortcut to make live-raw, is that right?

Additionally, you can modify the platform for which you are building based on the the HV and ZARCH parameters. So combinations are:

make live-raw ZARCH=amd64 HV=xen will build the original one
make live-raw ZARCH=arm64 HV=kvm will build a kvm based one for arm64
make live-gcp ZARCH=amd64 HV=kvm will build a kvm based one for amd64 to run on GCP

Basically:

ZARCH sets the architecture
HV sets the hypervisor
live-% sets the output file format

Is that correct? If so, I have a few questions:

can we document it that way? Make it clear from the beginning (definitely in the Makefile's help section, but also in the root README) how you set each of the above
should we also have the output file format be a parameter, rather than part of the target? I don't think it matters that much, but it is worth asking why the output file format is more worthy of a target than the hypervisor or architecture. As long as we have a good answer to it. Perhaps having it be a parameter, so I can do make live ZARCH=amd64 FORMAT=raw HV=kvm and then make live-kvm as a convenience might make it a little more even. That would make make live be truly neutral across all of the various parameters, and some conveniences to help.
do we have all of the variants to HV documented in the README (or elsewhere) and in the Makefile help section? We don't want to make people dig in one place for one parameter, another place for another, etc.

Not sure if it makes sense to wrap the two commands in a makefile target of some kind, but it seems to work as is (maybe we should teach linuxkit to take those values from some local configuration file instead -- what do you think @deitch ?)

These should just be variants on make run. I could do make run IMG=someimg and have options that allow me to run it via qemu or gcp or whatever. It isn't that different to how linuxkit run does it. There is a default option, based on your platform, and you can run the rest by passing an argument.

Hey @deitch

Let me see if I get this.

Yup. You've got it. One small tweak to your description is this:

make live - this is just a shortcut to make live-raw, is that right?

Not quite.  For all of these foo-bar targets, there's always a matching combination of just using the bar target with an appropriate setting of an env variable so make live-raw is equivalent to make IMG_FORMAT=raw live etc.
This is very much on purpose.

Basically:
ZARCH sets the architecture
HV sets the hypervisor

And IMG_FORMAT sets image format OR...

live-% sets the output file format

you can use foo-bar equivalents (e.g. live-raw)

can we document it that way?

sure. but the question is: document where? Because...

Make it clear from the beginning (definitely in the Makefile's help section, but also in the root README)

I actually don't want make help to grow much longer than it already is today. I'm slightly ambivalent about the top-level README.md so maybe the right place is docs/BUILDING.md and a short blurb in README.md?

should we also have the output file format be a parameter, rather than part of the target?

It is already the case.

These should just be variants on make run. I could do make run IMG=someimg and have options that allow me to run it via qemu or gcp or whatever. It isn't that different to how linuxkit run does it. There is a default option, based on your platform, and you can run the rest by passing an argument.

That's actually a great idea -- let me make it so.

Please see my comment in the comments section. Assuming I understood this correctly, requesting some minor target changes to make live and the FORMAT option to bring them up to par, and docs.

Tell you what -- let me do a small blurb in top level README.md, a longer one in docs/BUILDING.md and if you'd like to re-arrange it all -- PRs are welcome ;-)


Add option for JS aggregation/minifying

More than 30 js files are being loaded (using Theme One that is). There should be an option to aggregate and/or minify those assets for performance reasons.

+1, there is a lot of resources now and they can easily be concatenated

By the way - I've solved it in another way. Have look into Google's modPageSpeed library for Apache/nginx. Once you have your setup, the results are really impressive. Although some settings do not play well with the backend, but that can easily be solved by disabling the module on any admin/* location.
But in general it would be awesome if Pagekit had a setting for simply aggregating all the package's JS files.

modPageSpeed is amazing and I have all the proper optimizations installed on my server but an unminified source code + non concatenated resources are still not the best practice.
Appreciate your input though :)!

Just to have another example of the extend of this problem:
<script src="/app/assets/vue/dist/vue.min.js?v=cae3"></script>
<script src="/app/assets/jquery/dist/jquery.min.js?v=cae3"></script>
<script src="/app/assets/lodash/lodash.min.js?v=cae3"></script>
<script src="/system/intl/nl_NL?v=cae3"></script>
<script src="/app/system/app/bundle/vue.js?v=cae3"></script>
<script src="/packages/bixie/framework/app/bundle/bixie-framework.js?v=cae3"></script>
<script src="/app/assets/uikit/js/uikit.min.js?v=cae3"></script>
<script src="/app/assets/uikit/js/components/tooltip.min.js?v=cae3"></script>
<script src="/packages/bixie/framework/app/bundle/bixie-fieldtypes.js?v=cae3"></script>
<script src="/packages/bixie/framework/app/bundle/fieldtype-email.js?v=cae3"></script>
<script src="/app/system/modules/site/app/bundle/panel-link.js?v=cae3"></script>
<script src="/app/system/modules/finder/app/bundle/link-storage.js?v=cae3"></script>
<script src="/app/system/modules/site/app/bundle/link-page.js?v=cae3"></script>
<script src="/app/system/modules/user/app/bundle/link-user.js?v=cae3"></script>
<script src="/packages/bixie/formmaker/app/bundle/link-formmaker.js?v=cae3"></script>
<script src="/app/system/modules/site/app/bundle/input-link.js?v=cae3"></script>
<script src="/app/system/modules/editor/app/bundle/editor.js?v=cae3"></script>
<script src="/packages/bixie/framework/app/bundle/fieldtype-htmlcode.js?v=cae3"></script>
<script src="/app/assets/uikit/js/components/form-select.min.js?v=cae3"></script>
<script src="/app/assets/uikit/js/components/datepicker.min.js?v=cae3"></script>
<script src="/packages/bixie/framework/app/bundle/fieldtype-dob.js?v=cae3"></script>
<script src="/packages/bixie/framework/app/bundle/fieldtype-textbox.js?v=cae3"></script>
<script src="/packages/bixie/framework/app/bundle/fieldtype-pulldown.js?v=cae3"></script>
<script src="/packages/bixie/framework/app/bundle/fieldtype-checkbox.js?v=cae3"></script>
<script src="/app/assets/uikit/js/components/upload.min.js?v=cae3"></script>
<script src="/packages/bixie/framework/app/bundle/fieldtype-upload.js?v=cae3"></script>
<script src="/packages/bixie/framework/app/bundle/fieldtype-text.js?v=cae3"></script>
<script src="/packages/bixie/framework/app/bundle/fieldtype-radio.js?v=cae3"></script>
<script src="/packages/bixie/framework/app/bundle/fieldtype-sitelink.js?v=cae3"></script>
<script src="/packages/bixie/formmaker/app/bundle/formmaker.js?v=cae3"></script>
<script src="/app/assets/uikit/js/components/sticky.min.js?v=cae3"></script>
<script src="/app/assets/uikit/js/components/lightbox.min.js?v=cae3"></script>
<script src="/app/assets/uikit/js/components/parallax.min.js?v=cae3"></script>
<script src="/packages/pagekit/theme-avion/js/theme.js?v=cae3" defer></script>

This is just a ridiculous amount of requests to display a form.

Note: enabling CloudFlare Rocket Loader is a relatively simple workaround for now. By no means a "real" solution though.

In addition, there seems to be code combine assets but it is not being used? @MalteScharenberg

In the current Pagekit version we reduced the scripts which are loaded by default and we are also experimenting with the 'defer' attribute to mitigate the effects of multiple script files on https://pagekit.com. Modern browsers will download deferred scripts asynchronous, in parallel over multiple TCP connections (or are even capable of reusing a TCP connection for further files), so I think that the difference to one big file is negligible.
Problem with aggregating all script into one file is that you circumvent effective caching if the required scripts differ from page to page and it is hard to determine sets of scripts which make sense to combine automatically.
I think combining scripts in a meaningful manner should be the job of the extension (in your example Bixie Formmaker).

I agree with everything you said. Just a concat_all() would definitely not do the trick. I'll make an issue at bixie/pagekit-formmaker to have issue with formmaker addressed.
That being said, I noticed that adding defer to a script request with dependencies only adds it to the main script, not to all the dependencies. Thanks for your response.

The idea is that it should be possible to require a script with all its dependencies deferred. Currently this is not implemented in the public release (you can only see it at https://pagekit.com).

What happens when you have the following scenario:
Script 1 (defered / async)

jQuery
Lightbox

Script 2 (normal sync load)

jQuery
Something else

Assuming the assets manager doesn't include jQuery twice if it's called as a dependency twice. Is jQuery now loaded defered or not?

In this case jQuery would not be deferred because it would otherwise break script 2.

great :)

:guitar: Actually HTTP 2 is coming. My hosting provider is supporting this already. So minifying or concatenating makes no sense anymore.

Implemented in Pagekit.

+1

Sorry to say, but... HTTP/2 being enabled (and tested) on my sites does nothing in the way of bettering the corresponding parts of the scores at PageSpeed Insights, or GTmetrix, etc., when it comes to resources used. Anything not properly set as async or defer, will NOT magically disappear from the list of render-blocking resources just because HTTP/2 is in use.
A fresh install of PageKit 1.0.16 running under PHP 7.2.8, on a server using the latest version of Lightspeed, results in the following:

GTMetrix (YSlow tab) reports an "F" score (10% out of 100%) in Render-Blocking Resources, with 16 scripts reported in the list. Trying to use defer or async in theme code does nothing to change that - as it's already been noted.
PageSpeed Insights gives a site score of 76% out of 100% for mobile devices, with the majority of the non-content issues centered around those same resources, when not being loaded as async or deferred.

I have absolutely no issue with my current page speeds from PageKit sites running on my server setup (Home page: 0.9s, slowest page, 3.6s, average page, 1.9s) but the bigger concern for me, is the render-blocking issues (i.e., "11 of 27 resources not loaded") that I've been chasing for months that can cause indexing errors with Google search. Crawling times are exceeding whatever arbitrary limits they've set, and the crawler can't load all the resources it's expecting and will invalidate a previously indexed page, or will fail to validate a newly added page to be included in its index. There's much more at play here than simply getting faster site/page load times.
There is an online test to verify if any site is using HTTP/2 - available here: https://tools.keycdn.com/http2-test
My latest site tests:
GTmetrix: https://gtmetrix.com/reports/for-sale.lumiworx.com/1xCeQ0vr
PageSpeed (74% Mobile, 90% Desktop): https://developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Ffor-sale.lumiworx.com%2F


Tracker color issue fixed.

Thanks for the contribution. I noticed a minor issue in Chrome browser.

There is a small gap in the slider when lowering the value. It's just a minor issue.


Revert "Detect when isolate reload is barred and inform the user"

This reverts commit 09e68c33ccdb355fedd75a8716de0551c772f2de (PR #5582).
AOT snapshotter is broken at this engine rev.
https://flutter-dashboard.appspot.com/api/get-log?ownerKey=ahNzfmZsdXR0ZXItZGFzaGJvYXJkclgLEglDaGVja2xpc3QiOGZsdXR0ZXIvZmx1dHRlci8wOWU2OGMzM2NjZGIzNTVmZWRkNzVhODcxNmRlMDU1MWM3NzJmMmRlDAsSBFRhc2sYgICAgICAwAgM

lgtm. I dont know what broke though. Will investigate.

Thanks Yegor. The failure seems unrelated to my change but caused by the roll of the Dart VM itself.


AIRFLOW-51 && AIRFLOW-71 docker_operator improvements

AIRFLOW-51 Rename param Docker cpus -> cpu_shares
AIRFLOW-71 docker_operator - pulling from private repositories
https://github.com/apache/incubator-airflow/pull/1476
It can be merged at version 1.7

@mistercrunch

@asnir Thanks for the contribution! Before we can merge please provide a description in the PR, and squash your commits with proper messages as per guidelines. Thanks.


Performance of Reloaded Models are Much Worse than the Fine-Tuned Model

System Info
GPU: A100 (40G)
the peft package is directly pulled from this repo
Who can help?
No response
Information

[ ] The official example scripts
[X] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder
[X] My own task or dataset (give details below)

Reproduction
The model is initiated using the following codes.
When using a fine-tuned model, the "peft_weights" will be set to the output dir
    def model_init(
        self,
    ):
        model = LlamaForSequenceClassification.from_pretrained(
            "meta-llama/Llama-2-13b-hf",
            num_labels=1,
            load_in_8bit=True,
            torch_dtype=torch.float16,
            device_map="auto",
        )

        model = prepare_model_for_kbit_training(model)

        if self.peft_weights is not None: # self.peft_weights is the directory of the saved adapter_model.bin
            model = PeftModel.from_pretrained(
                model,
                self.peft_weights,
                torch_dtype=torch.float16,
                is_trainable=False,
            )
        else:
           config = LoraConfig(
                    r=8,
                    lora_alpha=16,
                    target_modules=["q_proj", "v_proj"],
                    lora_dropout=0.05,
                    bias="none",
                    task_type=TaskType.SEQ_CLS,
                )
            model = get_peft_model(model, config)
            if self.peft_weights:
                model.config.use_cache = True
            else:
                model.config.use_cache = False
            model.print_trainable_parameters()  # Be more transparent about the % of trainable params.

training:
    def train(self, parameter_search: bool = False):
        self.model.train()

        self.trainer = transformers.Trainer(
            model=self.model,
            train_dataset=self.train_data,
            eval_dataset=self.validate_data,
            args=transformers.TrainingArguments(
                per_device_train_batch_size=self.config.micro_batch_size,
                gradient_accumulation_steps=self.config.gradient_accumulation_steps,
                warmup_steps=self.config.warmup_steps,
                num_train_epochs=self.config.num_epochs,
                learning_rate=self.config.learning_rate,
                fp16=True,
                logging_steps=self.config.logging_steps,
                optim=self.config.optim,
                evaluation_strategy="steps",
                save_strategy="steps",
                eval_steps=self.config.eval_steps,
                save_steps=self.config.save_steps,
                output_dir=self.output_dir,
                save_total_limit=3,
                # remove_unused_columns=False,
                label_names=["labels"],
                load_best_model_at_end=True,
                metric_for_best_model=metric_for_best_model,
                ddp_find_unused_parameters=False if self.ddp else None,
                group_by_length=False,
                # report_to="wandb",
                # run_name=wandb_run_name if use_wandb else None,
                report_to=["tensorboard"],
                logging_dir=self.log_dir,
            ),
            data_collator=transformers.DataCollatorWithPadding(
                self.tokenizer, return_tensors="pt"
            )
        )

        self.trainer.train(resume_from_checkpoint=self.resume_from_checkpoint)
        self.model.save_pretrained(self.output_dir)

And the evaluation part:
    def sequence_eval(self):
        self.model.eval()
        predictor = transformers.Trainer(
            model=self.model,
            args=transformers.TrainingArguments(
                per_device_eval_batch_size=self.config.micro_batch_size,
                logging_steps=self.config.logging_steps,
                output_dir=self.output_dir,
                label_names=["labels"],
                # report_to="wandb",
                # run_name=wandb_run_name if use_wandb else None,
                logging_dir=self.log_dir,
                fp16=True,
                optim=self.config.optim,
                ddp_find_unused_parameters=False if self.ddp else None,
                group_by_length=False
                # remove_unused_columns=False,
            ),
            data_collator=transformers.DataCollatorWithPadding(
                self.tokenizer, return_tensors="pt"
            ),
        )
        pred = predictor.predict(test_dataset=self.test_data)
        print(pred.metrics)

However, when I evaluate the model directly after training, it outputs a loss about 1.0.
But when I reload the model by asserting the peft_weights param, it outputs a loss about 4.0.
Both use the same data and the same evaluation code.
I then printed the state_dict of the two models (trained one and reloaded one), it seems that the reloaded one had a different "weight_format".
And I also tried to save the model.state_dict directly using torch.save(), but the problem still occurs
Expected behavior
The reloaded model should perform similarly to the trained one.

Thank you for reporting this issue. Some question which might help discover the cause:

Is the loss of 4.0 the same that you see before you train the model?
Do you get the same outcome when you call prepare_model_for_kbit_training only for training, not for inference?
What do you mean by "different weight_format"?

Besides, I tried to directly save the state dict of the trained model using torch.save(), and load_state_dict() to reload. In this way, the trained model and reloaded model perform similar.

Okay, so it is very likely the case that some layers that are updated are not being loaded (correctly), which causes the issue. Something I don't see in the code you pasted is, if you use modules_to_save in the LoraConfig. When you use an LLM for sequence classification, a randomly initialized classification head will be added to the model (you should get a warning from transformers about it). That layer should be added to modules_to_save. PEFT tries to guess the name if not given ("classifier" or "score"), but that's not always the right name, so to be sure, I would try adding that layer specifically. Please try that and check if the error still occurs.
Some other observations:

num_labels=1 looks like a strange choice
the test loss of the trained model seems to be calculated on a different dataset, this probably doesn't explain the discrepancy, but still would be better to use the same
I still don't understand what I'm seeing with regard to the "weight_format"

I've checked the modules_to_save para and also the model structure. The model uses score as its sequence classification head, before which is a norm layer. I also tried to add norm to the modules_to_save, but that didn't help.
I got the warning as follows:
Some weights of LlamaForSequenceClassification were not initialized from the model checkpoint at meta-llama/Llama-2-13b-hf and are newly initialized: ['score.weight']

But it occured before I loaded the model llama-13b-hf with pretrained lora configs. So I just took it for granted.

To be clear, did you try adding "score" explicitly to modules_to_save and then run the experiment again?

No, but I'm sure that, during PeftModel.from_pretrained(), it calls PeftModel.load_adapters, and in the following lines:
            adapters_weights = torch.load(
                filename, map_location=torch.device("cuda" if torch.cuda.is_available() else "cpu")
            )

two keys are in the loaded adapters_weights:
base_model.model.model.norm.weight
base_model.model.score.weight

Let's just make 100% sure that it is so. Could you please load the adapter_model.bin and print all keys in the state dict?
Also, it seems like you're using an older PEFT version, if possible, could you please update to the latest one and rerun your experiments (preferentially with modules_to_save set explicitly)?

Hi, I've figured out this problem and solved it by modifying some codes in save_and_load.py
After PeftModel.from_pretrained(), we can get a PeftSequenceClassification model.
When calling PeftSequenceClassification.forward(), it's actually calling LoraModel.forward() according to line 789 in peft_model.py. (I'm using lora)
The forward() method of LoraModel inherits from BaseTuner.forward(), and in line 94 of tuner_utils.py, we can see return self.model.forward(*args, **kwargs), in which self.model refers to the original LlamaForSequenceClassification model.
So when we doing inference, the weight or score layer is actually base_model.model.score.original_module.weight, but we only load base_model.model.score.modules_to_save.default.weight.
I manually set the base_model.model.score.original_module.weight with the value of base_model.model.score.modules_to_save.default.weight, and the performances are then similar.
PS: I'm using peft==0.4.0. But I checked the newest code in this repo and the description above is based on the newest code

@BenjaminBossan Could you plz help me check if the problem I described above is correct? I'm not quite sure about it. And if so, shall I create a PR to fix it?

First of all, if it works for you, it works, so at least there is some progress there.
When it comes to what happens under the hood, I don't think your description is capturing it quite correctly. Yes, the base model's forward is called, but the base model is modified by adding the PEFT layers, so it should use the correct weight from modules_to_save.
I tried to replicate your issue but since I don't have the full code and data, that wasn't possible. So here is a minimal example:
import torch
from peft import PeftModel, LoraConfig, get_peft_model, TaskType
from transformers import AutoModelForSequenceClassification

torch.manual_seed(0)
X = torch.arange(50).reshape(10, 5).to(device)
y = torch.arange(10).to(device) % 2

model_name = "facebook/opt-125m"
device = 'cuda'
model_origin = AutoModelForSequenceClassification.from_pretrained(
    model_name,
    num_labels=2,
).to(device)
model_origin.eval()

print(model_origin(X).logits)
# prints:
# tensor([[-0.2883,  0.1495],
#         [-0.1383,  0.2464],
#         [ 0.4012,  1.1022],
#         [ 0.0190, -0.1290],
#         [ 0.4800,  0.6427],
#         [-0.1140,  0.5280],
#         [ 0.2909,  0.3563],
#         [ 0.7405, -0.0209],
#         [-0.8172, -0.2533],
#         [ 0.5035,  0.1919]], device='cuda:0', grad_fn=<IndexBackward0>)

config = LoraConfig(
    r=8,
    lora_alpha=16,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.SEQ_CLS,
)
peft_model = get_peft_model(model_origin, config)
print(peft_model.modules_to_save)
# prints:
# {'classifier', 'score'}

peft_model.train()
optimizer = torch.optim.Adam(peft_model.parameters(), 0.001)
for epoch in range(5):
    optimizer.zero_grad()
    pred = peft_model(X)
    loss = torch.nn.functional.cross_entropy(pred.logits, y)
    loss.backward()
    optimizer.step()

peft_model.eval()
print(peft_model(X).logits)
# prints:
# tensor([[ 2.0473, -2.3220],
#         [-1.7545,  1.2145],
#         [ 1.6624, -0.5274],
#         [-1.3836,  0.7652],
#         [ 2.0419, -0.9504],
#         [-1.7616,  1.2400],
#         [ 1.5823, -1.1156],
#         [-1.0041,  1.5738],
#         [ 0.9256, -1.9221],
#         [-1.1858,  1.4028]], device='cuda:0', grad_fn=<IndexBackward0>)

peft_model.save_pretrained("/tmp/issue-876.pt")
model_origin = AutoModelForSequenceClassification.from_pretrained(
    model_name,
    num_labels=2,
).to(device)
model_loaded = PeftModel.from_pretrained(
    model_origin,
    "/tmp/issue-876.pt/",
    is_trainable=False,
).to(device)
print(model_loaded(X).logits)
# prints same logits as after training:
# tensor([[ 2.0473, -2.3220],
#         [-1.7545,  1.2145],
#         [ 1.6624, -0.5274],
#         [-1.3836,  0.7652],
#         [ 2.0419, -0.9504],
#         [-1.7616,  1.2400],
#         [ 1.5823, -1.1156],
#         [-1.0041,  1.5738],
#         [ 0.9256, -1.9221],
#         [-1.1858,  1.4028]], device='cuda:0', grad_fn=<IndexBackward0>)

As you can see, in this example, loading the model works correctly, it creates the same prediction as it did after training.
Of course, there are a few differences between this and your code. I don't think that any the differences I can see would make a  differences, but sometimes there can be strange interactions. Maybe if you could use the minimal example and work it up to reflect your code, we could discover which step it is that causes the faulty behavior.

Thank you @BenjaminBossan
With your code, I can also output the same logits. However, after I reload the model with 8bit training, the logits became different.
code:
import torch
from peft import (
    PeftModel,
    LoraConfig,
    get_peft_model,
    TaskType,
    prepare_model_for_kbit_training,
)
from transformers import AutoModelForSequenceClassification

device = "cuda"
torch.manual_seed(0)
X = torch.arange(50).reshape(10, 5).to(device)
y = torch.arange(10).to(device) % 2

model_name = "meta-llama/Llama-2-13b-hf"
model_origin = AutoModelForSequenceClassification.from_pretrained(
    model_name, num_labels=2, load_in_8bit=True, device_map="auto"
)
model_origin.eval()

print(model_origin(X).logits)
# tensor([[-0.6875, -1.5029],
#         [-0.5869,  1.2402],
#         [-0.8667, -0.5913],
#         [ 0.1504, -1.1016],
#         [-0.1307, -0.3596],
#         [-1.1934,  0.2991],
#         [-0.0227,  1.5645],
#         [-0.7070, -0.7495],
#         [-0.7144, -0.5493],
#         [-0.5059, -0.7051]], device='cuda:0', dtype=torch.float16,
#        grad_fn=<IndexBackward0>)
model_origin = prepare_model_for_kbit_training(model_origin)
config = LoraConfig(
    r=8,
    lora_alpha=16,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.SEQ_CLS,
)

peft_model = get_peft_model(model_origin, config)
print(peft_model.modules_to_save)
# prints:
# {'classifier', 'score'}

peft_model.train()
optimizer = torch.optim.Adam(peft_model.parameters(), 0.001)
for epoch in range(5):
    optimizer.zero_grad()
    pred = peft_model(X)
    loss = torch.nn.functional.cross_entropy(pred.logits, y)
    loss.backward()
    optimizer.step()

peft_model.eval()
print(peft_model(X).logits)
# tensor([[ 2.7274, -4.9314],
#         [-6.0120,  6.4267],
#         [ 2.6807, -3.5113],
#         [-8.1143,  7.5306],
#         [ 2.8598, -3.8665],
#         [-5.2717,  5.0157],
#         [ 2.2844, -3.7415],
#         [-5.0672,  3.2364],
#         [ 1.9672, -2.9982],
#         [-5.2930,  3.6481]], device='cuda:0', grad_fn=<IndexBackward0>)

peft_model.save_pretrained("./tmp/issue-876.pt")
model_origin = AutoModelForSequenceClassification.from_pretrained(
    model_name, num_labels=2, load_in_8bit=True, device_map="auto"
)
model_loaded = PeftModel.from_pretrained(
    model_origin,
    "./tmp/issue-876.pt/",
    is_trainable=False,
).to(device)
print(model_loaded(X).logits)
# tensor([[ 3.0352, -5.0742],
#         [-5.7578,  5.8828],
#         [ 2.7852, -3.5547],
#         [-8.2734,  7.6836],
#         [ 2.9199, -3.9785],
#         [-5.3594,  5.0078],
#         [ 2.4180, -3.8730],
#         [-5.4023,  3.6133],
#         [ 1.9648, -3.0391],
#         [-5.1680,  3.6641]], device='cuda:0', dtype=torch.float16,
#        grad_fn=<IndexBackward0>)

Hi, thanks for checking this out. So the logits are not exactly the same, but they look similar, with the alternating positive/negative pattern per row, so we can at least exclude that random/no weights are being loaded. Still, there is a gap between before and after. One thing you could try is to apply prepare_model_for_kbit_training on the model_origin which is used for loading the PEFT model. Does that result in the same output?

Many thanks, I add prepare_model_for_kbit_training, and it then resulted in the same logits!
I checked the code with mine step by step, and it seems that "load_best_model_at_end=True," caused this problem. But I'm still checking it :)

I've figured out this problem.
The true reason is that I enabled fp16=True in the trainer arguments.
When I didn't set the fp16 para, the performances became normal

Hello, I am having similar problems when using QLoRA to fine-tune a sequence classification model. I tried to run the same test example and have different logits returned from the trained and loaded model.
import torch
from peft import LoraConfig, get_peft_model, TaskType, PeftModel, prepare_model_for_kbit_training, PeftConfig
from transformers import BitsAndBytesConfig, AutoModelForSequenceClassification
device='cuda'
torch.manual_seed(0)
X = torch.arange(50).reshape(10, 5).to(device)
y = torch.arange(10).to(device) % 2

model_name = "facebook/opt-125m"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16
)
model_origin = AutoModelForSequenceClassification.from_pretrained(
    model_name,
    return_dict=True,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
    num_labels=2
)
model_origin.eval()

print(model_origin(X).logits)
tensor([[ 0.4045, -0.1556],
        [-0.0499,  0.8711],
        [-0.5942,  0.5269],
        [ 0.3704,  0.5015],
        [-0.5283,  1.6162],
        [-0.4658,  0.0902],
        [-0.0798,  0.8389],
        [ 0.6646,  1.0020],
        [-0.6118, -0.3318],
        [ 0.3794,  0.5908]], device='cuda:0', dtype=torch.float16,
       grad_fn=<IndexBackward0>)

       model = prepare_model_for_kbit_training(model_origin)
config = LoraConfig(
    r=64,
    lora_alpha=16,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.1,
    bias="none",
    task_type=TaskType.SEQ_CLS
)

peft_model = get_peft_model(model, config)

peft_model.train()
optimizer = torch.optim.Adam(peft_model.parameters(), 0.001)
for epoch in range(5):
    optimizer.zero_grad()
    pred = peft_model(X)
    loss = torch.nn.functional.cross_entropy(pred.logits, y)
    loss.backward()
    optimizer.step()

peft_model.eval()
print(peft_model(X).logits)
peft_model.save_pretrained("issue-876")
# tensor([[ 1.5051, -1.1117],
#         [-1.4694,  2.0478],
#         [ 1.5312, -1.9390],
#         [-0.8304,  1.6731],
#         [ 2.4396, -0.7725],
#         [-1.0147,  1.6947],
#         [ 1.2751, -0.3976],
#         [-0.5274,  2.2715],
#         [ 0.8527, -1.4726],
#         [-0.7441,  1.8471]], device='cuda:0', grad_fn=<IndexBackward0>)

config = PeftConfig.from_pretrained('issue-876')

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16
)
reloaded_model = AutoModelForSequenceClassification.from_pretrained(
    config.base_model_name_or_path,
    return_dict=True,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
    num_labels=2
)
model_loaded = PeftModel.from_pretrained(
    reloaded_model,
    "issue-876",
    is_trainable=False,
).to(device)
model_loaded.eval()
print(model_loaded(X).logits)
# tensor([[-0.1921, -0.3909],
#         [-0.4866, -0.8794],
#         [-0.1503,  0.7964],
#         [-0.2961, -0.7729],
#         [-0.5757,  0.6689],
#         [-0.0484, -0.2642],
#         [-0.1884, -0.2212],
#         [ 0.3533, -0.4705],
#         [ 0.0359,  0.2169],
#         [ 0.2805, -0.2195]], device='cuda:0', dtype=torch.float16,
#        grad_fn=<IndexBackward0>)

# Change score weights
model_loaded.model.score.original_module.weight = model_loaded.model.score.modules_to_save.default.weight
print(model_loaded(X).logits)
# tensor([[ 1.1230, -0.7319],
#         [-0.7207,  1.2998],
#         [ 0.8989, -1.3037],
#         [-0.1592,  1.0020],
#         [ 1.6318,  0.0370],
#         [-0.6919,  1.3730],
#         [ 0.9175, -0.0410],
#         [ 0.0981,  1.6465],
#         [ 0.1375, -0.7598],
#         [-0.0732,  1.1768]], device='cuda:0', dtype=torch.float16,
#        grad_fn=<IndexBackward0>)

@ryan4martin I could not reproduce the issue using the code you provided. Possibly, this is an error that was fixed by now. What version of PEFT are you using? Could you try using the latest release? If that's not enough, could you try installing from the main branch from github?

I was using version 0.4.0, the problem no longer occurs when upgrading to version 0.5.0. Thanks!

I think this issue still exists
i'm seeing it on peft 0.6.1
when training using AutoModelForCausalLM
bellow is the revised script that reproduces the issue for a CausalLM model
import torch
from peft import LoraConfig, get_peft_model, TaskType, PeftModel, prepare_model_for_kbit_training, PeftConfig
from transformers import BitsAndBytesConfig, AutoModelForSequenceClassification, AutoModelForCausalLM

device = 'cuda'
torch.manual_seed(0)
X = torch.arange(50).reshape(10, 5).to(device)
y = torch.arange(10 * 50272).reshape((10, -1)).to(device) % 2

model_name = "facebook/opt-125m"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16
)
model_origin = AutoModelForCausalLM.from_pretrained(
    model_name,
    return_dict=True,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)
model_origin.eval()
print(model_origin(X).logits)
# tensor([[0.4045, -0.1556],
#         [-0.0499, 0.8711],
#         [-0.5942, 0.5269],
#         [0.3704, 0.5015],
#         [-0.5283, 1.6162],
#         [-0.4658, 0.0902],
#         [-0.0798, 0.8389],
#         [0.6646, 1.0020],
#         [-0.6118, -0.3318],
#         [0.3794, 0.5908]], device='cuda:0', dtype=torch.float16,
#        grad_fn= < IndexBackward0 >)

model = prepare_model_for_kbit_training(model_origin)
config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.CAUSAL_LM,
    target_modules=["q_proj", "v_proj"],
)

peft_model = get_peft_model(model, config)
model.config.use_cache = False
peft_model.train()
optimizer = torch.optim.Adam(peft_model.parameters(), 0.001)
for epoch in range(5):
    optimizer.zero_grad()
    pred = peft_model(X)
    loss = torch.nn.functional.cross_entropy(pred.logits, y)
    loss.backward()
    optimizer.step()

peft_model.eval()
print('after train peft model logits')
print(peft_model(X).logits)
temp_dir_name = "issue-877"
peft_model.save_pretrained(temp_dir_name)

# after train peft model logits
# tensor([[[-3.5573, -3.5542,  3.4039,  ..., -3.5838, -3.5625, -3.6611],
#          [-1.8647, -1.8649,  3.4243,  ..., -1.8605, -1.8324, -2.1610],
#          [-6.2448, -6.2437,  2.0923,  ..., -6.2480, -6.2455, -6.1857],
#          [-2.8594, -2.8577,  2.6637,  ..., -2.8307, -2.8595, -2.9530],
#          [-5.4817, -5.4840,  1.2584,  ..., -5.4769, -5.5483, -5.3396]],...

config = PeftConfig.from_pretrained(temp_dir_name)

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16
)
reloaded_model = AutoModelForCausalLM.from_pretrained(
    config.base_model_name_or_path,
    return_dict=True,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)
model_loaded = PeftModel.from_pretrained(
    reloaded_model,
    temp_dir_name,
    is_trainable=False,
).to(device)
model_loaded.eval()
print('after load peft model logits')
print(model_loaded(X).logits)
# after load peft model logits
# tensor([[[-3.5352, -3.5332,  3.4336,  ..., -3.5625, -3.5410, -3.6406],
#          [-1.8623, -1.8623,  3.4258,  ..., -1.8584, -1.8301, -2.1582],
#          [-6.2422, -6.2422,  2.0938,  ..., -6.2461, -6.2422, -6.1836],
#          [-2.8535, -2.8516,  2.6641,  ..., -2.8242, -2.8535, -2.9473],
#          [-5.4766, -5.4805,  1.2646,  ..., -5.4727, -5.5430, -5.3359]], ...

Hi @mdabbah the reason why you see the slight discrepancies is because in the first case, the model uses float32 for all non-quantized weights whereas in the second example, it uses float32 only for the LoRA parameters, whereas it uses float16 for all the other non-quantized parameters.
If you need to have 100% the same outputs after loading, you need to call model_loaded = prepare_model_for_kbit_training(model_loaded) after calling PeftModel.from_pretrained. Yes, it is not exactly intuitive, but in my testing, it worked. Otherwise, if small differences in logits are acceptable, you should see faster inference when using float16.

Could you please try this script:
import torch
from peft import LoraConfig, get_peft_model, TaskType, PeftModel, prepare_model_for_kbit_training, PeftConfig
from transformers import BitsAndBytesConfig, AutoModelForCausalLM

device = 'cuda'
torch.manual_seed(0)
X = torch.arange(50).reshape(10, 5).to(device)
y = torch.arange(10 * 50272).reshape((10, -1)).to(device) % 2

model_name = "facebook/opt-125m"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16
)
model_origin = AutoModelForCausalLM.from_pretrained(
    model_name,
    return_dict=True,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)
model_origin.eval()
# print(model_origin(X).logits[:3, 0, :7])  <== try uncommenting this line!

model = prepare_model_for_kbit_training(model_origin)
config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.CAUSAL_LM,
    target_modules=["q_proj", "v_proj"],
)

peft_model = get_peft_model(model, config)
model.config.use_cache = False
peft_model.train()
optimizer = torch.optim.Adam(peft_model.parameters(), 0.001)
for epoch in range(5):
    optimizer.zero_grad()
    pred = peft_model(X)
    loss = torch.nn.functional.cross_entropy(pred.logits, y)
    loss.backward()
    optimizer.step()

peft_model.eval()
print('after train peft model')
torch.manual_seed(0)
print(peft_model(X).logits[:3, 0, :7])
temp_dir_name = "/tmp/peft-issue-877"
peft_model.save_pretrained(temp_dir_name)

config = PeftConfig.from_pretrained(temp_dir_name)

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16
)
reloaded_model = AutoModelForCausalLM.from_pretrained(
    config.base_model_name_or_path,
    return_dict=True,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)
model_loaded = PeftModel.from_pretrained(
    reloaded_model,
    temp_dir_name,
    is_trainable=False,
).to(device)
model_loaded = prepare_model_for_kbit_training(model_loaded)
model_loaded.eval()
print('after loading peft model')
torch.manual_seed(0)
print(model_loaded(X).logits[:3, 0, :7])

For me, the exact same logits are produced when I call prepare_model_for_kbit_training on the loaded model.
Interestingly, however, I found that the logits change slightly if you first call inference on the base model. I have no idea why and if it's related to PEFT. Possibly, this is some caching issue, not sure. To see this effect in action, uncomment the line that I marked with <== try uncommenting this line! and all of a sudden, the results differ slightly, even though this should not influence the results. Probably, this is why you couldn't reproduce the logits exactly.

wow! very interesting
i see same effect as you described
i'm not sure how negligible the difference is,
i've started this whole investigation because i saw that the loss with the model before the save was 0 on a dataset i intentionally overfitted on but when i tried to load the model the loss was back to a starting point essentially random.
i've updated the simple script above to run for more steps and calculate accuracy of predicting y
i see that the loaded model has a different but high accuracy at first but when increasing the number of steps they converge.
import torch
from peft import LoraConfig, get_peft_model, TaskType, PeftModel, prepare_model_for_kbit_training, PeftConfig
from tqdm import tqdm
from transformers import BitsAndBytesConfig, AutoModelForCausalLM

device = 'cuda'
torch.manual_seed(0)
X = torch.arange(50).reshape(10, 5).to(device)
# y = torch.arange(10 * 50272).reshape((10, -1)).to(device) % 2
y = X + 1

model_name = "facebook/opt-125m"
model_name = "tiiuae/falcon-7b"
print(X.device)
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16
)
model_origin = AutoModelForCausalLM.from_pretrained(
    model_name,
    return_dict=True,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)
model_origin.eval()
print(model_origin(X).logits[:3, 0, :7])  # <== try uncommenting this line!

model = prepare_model_for_kbit_training(model_origin)
config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.CAUSAL_LM,
    # target_modules=["q_proj", "v_proj"],
    target_modules=[
        "query_key_value",
        "dense",
        "dense_h_to_4h",
        "dense_4h_to_h",
    ]
)

peft_model = get_peft_model(model, config)
model.config.use_cache = False
peft_model.train()
optimizer = torch.optim.Adam(peft_model.parameters(), 0.001)
for epoch in tqdm(range(10000)):
    optimizer.zero_grad()
    pred = peft_model(X)
    y_ = pred.logits.argmax(dim=-1)
    acc = (y_ == y).float().mean()
    if epoch % 100 == 0:
        print(f'epoch: {epoch} training acc: {acc.item():.4f}')
    loss = torch.nn.functional.cross_entropy(pred.logits.permute(0, 2, 1), y)
    loss.backward()
    optimizer.step()

peft_model.eval()
y_ = peft_model(X).logits.argmax(dim=-1)
acc = (y_ == y).float().mean()
print(f'peft_model final acc: {acc}')
print('after train peft model')
torch.manual_seed(0)
print(peft_model(X).logits[:3, 0, :7])
temp_dir_name = "/tmp/peft-issue-877"
peft_model.save_pretrained(temp_dir_name)

config = PeftConfig.from_pretrained(temp_dir_name)

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16
)
reloaded_model = AutoModelForCausalLM.from_pretrained(
    config.base_model_name_or_path,
    return_dict=True,
    quantization_config=bnb_config,
    device_map="auto",
    # trust_remote_code=True,
)
model_loaded = PeftModel.from_pretrained(
    reloaded_model,
    temp_dir_name,
    is_trainable=False,
)  # .to(device)
print(model_loaded.device)
# X=X.to(model_loaded.device)
model_loaded = prepare_model_for_kbit_training(model_loaded)
model_loaded.eval()
print('after loading peft model')
torch.manual_seed(0)
print(model_loaded(X).logits[:3, 0, :7])

y_ = model_loaded(X).logits.argmax(dim=-1)
acc = (y_ == y).float().mean()
print(f'model_loaded final acc: {acc}')

i've updated my main experiment code to load the model as above, but i still see random loss after loading
i'll my current guess is that it has something to do with the SFT trainer as i'm using the loaded model with the SFT trainer to calculate the loss.
i'll keep posting what i find.


copy local picture to ssh-remote md file

hello，i want to copy image to a ssh-remote md file， the extension show:
There is not an image in the clipboard.

You can't paste pictures directly, they don't share the clipboard, but you can use this button on the toolbar..