fix: explicitly support array of strings for AuthenticateHandler.options

Summary
In #267 some changes were made for compliance reasons where scopes should be treated as strings for our OAuth interfaces but as arrays of strings for internal purposes. As part of that change, we accidentally made the scope prop for the authenticate method only support strings.
This change remediates any potential backwards compatibility changes from v5.0 to v5.1
Linked issue(s)
#307 & #267
Involved parts of the project
authenticate handler
Added tests?
✅

We should make clear that array scopes need to be trimmed etc. We do that with strings in parseScope but we do not validate if it's an array. Could there be any issues involved with array scopes?

We can introduce a "validate scopes" function that validates an array of scopes - I did think about this, but decided that would lean this towards a "feature" rather than a "fix" (and I wanted to go for a fix here).
Because this array of scopes if provided by the developer (or should be!) then I don't think there's such a risk to have malformed arrays of scopes. Do we validated the arrays of scopes that are provided in any other part of the library?

Just to touch on the typings too - we could just put it as scope?: string[] if we feel use of a string is to be deprecated, because there's no need for the typings to include deprecated values and clearly the typings hadn't been causing problems for people up to now 😅

I've made the small change which essentially reverts the change from #305 so that the typings reflect our ideal implementation.
I think that keeps things straightforward for implementors in TS and also means it's less likely users will need to perform refactors when we do remove the string support if they've been following the types.

Any blockers here/on a release? At the moment I'm relying on ```ts
"foo" as unknown as string[]

Which is a bit of a code-smell, to say the least 😅

@Lordfirespeed I'm back at the office tomorrow and will then release an RC that reflects on all recent PRs.

@Lordfirespeed I'm back at the office tomorrow

Awesome! Thanks very much ♥ hope your holidays have been restful & enjoyable


Settings aren't gufe key stable with the AHFE protocol

Taking the Protocol, serializing it, and reading it back in, creates two Protocols with Settings equality but different gufe keys.

@IAlibay
I see a similar behavior for two classes in my PR: #346
Currently I locate the issue in classes, that inherit over multiple layers. There might be some modification missing from my side.
(e.g. GufeTokenizable -> NetworkPlan -> LigandNetwork)


Function to change local classifier

As of now, there is no functionality to update a classifier, one must recreate a new instance of the Hierarchical classifier. This PR adds a new method to update the local classifier model used.

Codecov Report
Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (a4c79e8) 98.40% compared to head (13feb3f) 98.25%.

Files
Patch %
Lines

hiclass/HierarchicalClassifier.py
83.33%
1 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #100      +/-   ##
==========================================
- Coverage   98.40%   98.25%   -0.16%
==========================================
  Files           8        8
  Lines         566      572       +6
==========================================
+ Hits          557      562       +5
- Misses          9       10       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


METRON-2136: Add parser aggregation sidebar

Contributor Comments
Link to the original JIRA: https://issues.apache.org/jira/browse/METRON-2136
Link to Parent Feature ticket: https://issues.apache.org/jira/browse/METRON-1856
This is the sidebar with which you can perform parser aggregation. In this sidebar, you can name of the aggregated sensors and add a description if you want. In case you want to aggregate a sensor with another sensor that is already aggregated with a 3rd sensor, it'll ask you whether you want to merge the three together or you would like to create a brand new group including the two.
Related commits
This PR is part of a larger sequence of PRs we are opening to make the reviewing process of the relatively large Parser Aggregation UI feature easier.
To see only the changes that apply to this PR, please review the last commit in the list of commits (the commit to review contains the title of the PR)
If you want to learn more about why we picked this method, please refer to the discussion thread here
Please find the full list of relating PRs below.
PR#1 METRON-2114: [UI] Moving components to sensor parser module
PR#2 METRON-2116: [UI] Removing redundant AppConfigService
PR#3 METRON-2117: [UI] Aligning models to grouping feature
PR#4 METRON-2115: [UI] Aligning UI to the parser aggregation AP
PR#5 METRON-2122: [UI] Fixing early app config access issue
PR#6 METRON-2124: [UI] Move status information and start/stop to the Aggregate level
PR#7 METRON-2125: [UI] Making changes visible in the parser list by marking changed items
PR#8 METRON-2131: Add NgRx and related dependencies
PR#9 METRON-2133: Add NgRx effects to communicate with the server
PR#10 METRON-2134: Add NgRx reducers to perform parser and group changes in the store
PR#11 METRON-2135: Add NgRx actions to trigger state changes
CURRENT PR#12 METRON-2136: Add parser aggregation sidebar
PR#13 METRON-2137: Implement drag and drop mechanism and wire NgRx
PR#14 METRON-2138: Code clean up
PR#15 METRON-2139: Refactoring sensor-parser-config.component and wire NgRx
Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our Development Guidelines for the complete guide to follow for contributions.
Please refer also to our Build Verification Guidelines for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
For all changes:

[X] Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
[X] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
[X] Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

[ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed?

[ ] Have you included steps or a guide to how the change may be verified and tested manually?

[ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh

[ ] Have you written or updated unit tests and or integration tests to verify your changes?

[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

[ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via site-book/target/site/index.html:
cd site-book
mvn site

[ ] Have you ensured that any documentation diagrams have been updated, along with their source files, using draw.io? See Metron Development Guidelines for instructions.

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.

+1


click settings will crash in Godot 4rc1

https://github.com/Ark2000/PankuConsole/commit/fce491a5a69cc2b325d04e5122af817b6712af33


Allow injection of persistence methods

I'm using the ability to inject custom persistence methods for save_cassette and load_cassette in order to save cassettes to a DB in a service I've built which uses vcrpy as a dependency.  Perhaps other folks will appreciate the ability to override the persistence methods?
You can see the custom methods I inject for my project here.
I've added a couple of new integration tests for save/load injection. Let me know if there's anything else you'd like for this PR!

Thanks!  I think this is a great idea.  I especially appreciate that you have tests in this PR. I would be happy to add this but I would like to change the API a bit.
If you go here, you can see that we have support for custom serializers and custom matchers:
https://github.com/kevin1024/vcrpy/blob/master/docs/advanced.rst#register-your-own-serializer
I would like to let you define custom persisters using a similar API, rather than a callback.  What do you think?

Great thanks for the feedback. Seems pretty straightforward.  I'll update the PR and see what you think!

Hey @kevin1024 - so it's refactored and almost all tests are passing, however there is some dark mock / lambda voodoo mixed in with one of the tests.  I've spent quite a lot of time attempting to make this test pass. Looks like we need to get the class instance into the lambda somehow, but it's beyond me.  Would you be willing to take a look?
TypeError: unbound method <lambda>() must be called with FilesystemPersister
instance as first argument (got str instance instead)

Dark mock and lambda voodoo?  Paging @IvanMalison 😛

I'll take a look later today

Its just because that lambda is not a class or static method. This means that when it is retrieved from the FilesystemClass it comes back as an unbound method, which causes an issue because the function is then called without a FileSystemClass instance as its first argument.
I made the appropriate change in https://github.com/j-funk/vcrpy/pull/1

Thanks @Ivanmalison :)

@IvanMalison thank you :)

@j-funk is this PR ready to merge?

Oh, actually, can you add a section to the docs showing how to use it? :cake:

@kevin1024 I've updated the docs and made a minor adjustment to the code.  Assuming tests pass, the only thing I am not sure about is the rst link I've made to the FilesystemPersister class.

Awesome, thanks!!!


Memory leak by broadcast() method

The broadcast method uses the getifaddrs function. The getifaddrs function returns a table with allocated buffers in sic memory, but they are not cleared anywhere.
On a virtual machine with 10+ interfaces, a couple of MB of memory may leak.

Memory allocated by ffi.new couldn't be freed manually. It automatically GCed as any LuaJIT cdata object.
However, since this buffer is used temporary without yields as optimization it could be allocated only once.


Fix the sample yaml in AKS virtual-kubelet

Fix the matchLabels in the nanoserver-iis sample yaml
Update the sample yaml to use common node labels in the nodeSelector

@robbiezhang : Thanks for your contribution! The author, @iainfoulds, has been notified to review your proposed change.

Fix #17860
Customer report issue:
https://github.com/virtual-kubelet/virtual-kubelet/issues/384

Thanks, @robbiezhang
#sign-off


fix: ensure SimpleFeatureStateViewSet uses correct permissions for segment overrides

Thanks for submitting a PR! Please check the boxes below:

[x] I have run pre-commit to check linting
[x] I have filled in the "Changes" section below?
[x] I have filled in the "How did you test this code" section below?
[x] I have used a Conventional Commit title for this Pull Request

Changes
Fixes an issue found when testing the new MANAGE_SEGMENTS permission functionality. The FE uses the SimpleFeatureStateViewSet for updating segment overrides, which the new permission didn't cover. This PR ensures that this endpoint is correctly covered by the new permission for creating (although this is not used by the FE) and updating segment overrides via this API route.
How did you test this code?
Added new unit tests.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (e4325a8) 95.56% compared to head (39d6ad5) 95.58%.
Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2990      +/-   ##
==========================================
+ Coverage   95.56%   95.58%   +0.01%
==========================================
  Files        1014     1016       +2
  Lines       29421    29525     +104
==========================================
+ Hits        28116    28221     +105
+ Misses       1305     1304       -1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


GuanWei_pytest

Checks

[ ] Your YAML file has required fields as listed in Specs
[ ] All checks have passed yaml-lint passed
[ ] You agree to contribute your work under MIT license

demonstrated pytest and refactoring, exceeding expectation. however the PR is dirty, closing


jsbundling files overwriting cssbundling generated files.

While generating files for jsbundling and cssbundling, some of the files are overwritten. Not sure if that will cause my app to behave abnormally in the future, so I would like to know beforehand how it impacts my app, because I am suspecting that this is probably breaking my Bootstrap JS.
I've tried rails assets:precompile, as well as running the app from bin/dev, but it just won't work.
Some of the errors/warnings I've received:
File unchanged! The supplied flag value not found!  app/assets/config/manifest.js

File unchanged! The supplied flag value not found!  .gitignore

File unchanged! The supplied flag value not found!  .gitignore

conflict  bin/dev
Overwrite /home/ashvith/Desktop/todo-app/bin/dev? (enter "h" for help) [Ynaqdhm] Y
    force  bin/dev

I've started my app with the default settings (importmap, hotwire, sqlite3). Removed the importmap-rails, and replaced them with jsbundling-rails and cssbundling-rails.

I think this issue has been resolved by using cssbundling-rails 1.1.0 and jsbundling-rails 1.0.2 via https://github.com/rails/jsbundling-rails/pull/95


Bug: Double right click on a file crashes app

Description
Double right click on a file crashes app
Steps To Reproduce

Right click a file and with the menu still open, right click the file again

Steps to fix

Fix the crash

Files Version
<IP_ADDRESS>
Windows Version
22623.1028
Relevant Assets/Logs
debug.log

Does this happen with any file or just a specific type?

oh,i'm whrong, it's double right click, it happened on any files, including folders

it seems that it will crash in a few seconds after i right click on anywhere

	System.Runtime.InteropServices.COMException (0x80010105): The server threw an exception. (0x80010105 (RPC_E_SERVERFAULT))```

Seems like a WinAppSdk issue but hard to know.

I don't think this is an issue in the latest version


Does Trivy Dream?

What steps did you take:
I didn't do it, actually. I was nowhere near there when it happened. It didn't even happen in a place! Several people saw me walking around outside near my residence and they're all willing to testify.  I'm innocent, I swear!
What happened:
https://github.com/vmware-tanzu/carvel-secretgen-controller/runs/7909711621?check_suite_focus=true
What did you expect:
Trivy not to fail
Anything else you would like to add:
Is the issue that Neil recently fixed all the vulnerabilities, so now we're learning that there's a bug in the codepath that happens if there was nothing to report?
Environment:
github action runner

Vote on this request
This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.
👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"
We are also happy to receive and review Pull Requests if you want to help working on this issue.

Take a look at this commit https://github.com/vmware-tanzu/carvel-release-scripts/commit/88586bd17419ef65b0aecbcf354f3cfd13b60fac it should provide the solution for this issue


FATAL: Could not get Location header for 3xx reply    Ret code: ded92

What model of system are you using?

[ ] Old 3DS
[ ] Old 3DS XL
[ ] 2DS (non-XL)
[ ] New 3DS
[ ] New 3DS XL
[·] New 2DS XL

What is your current Luma3DS version?
If you are running an hourly version of Luma3DS, please manually update to a stable version as I will no longer deal with people who are using hourlies.

[ ] Luma3DS v7.1 + boot9strap v1.0/v1.1
[ ] Luma3DS v8.0 + boot9strap v1.2+
[ ] Luma3DS v8.1 + boot9strap v1.2+
[ ] Luma3DS v8.1.1 + boot9strap v1.2+
[ ] Luma3DS v9.0 + boot9strap v1.2+
[ ] Luma3DS v9.1 + boot9strap v1.2+
[ ] Luma3DS v10.0+ + boot9strap v1.2+

What version of Luma Updater are you using?

[ ] v2.0
[ ] v2.1
[ ] v2.1.1
[ ] v2.1.2
[ ] v2.2
[ ] v2.3
[·] v2.4
[ ] v2.5

How are you running Luma Updater?

[·] .cia
[ ] .3dsx + HBL/Rosalina

What problems are you experiencing?
Please tell us what problems you are facing.
What is the exact error code you are receiving?    ded92
Please give the exact error code the app gives you.    ded92
Steps to reproduce

#53


Building on Arch Linux fails

Using gcc & g++ version 11. Note 12 is available.
Linux juno-console 5.15.47-1-lts #1 SMP Tue, 14 Jun 2022 17:59:27 +0000 x86_64 GNU/Linux
Unable to get crashpad to install or build, so disabled.
When building main and 1.80 the following tests fail:
The following tests FAILED:
        384 - ExecuteInProcessTest.ExecuteInProcess (Child aborted)
        385 - ExecuteInProcessTest.ExecuteInProcessWithMicrosoftCallingConvention (Child aborted)
        388 - InjectLibraryInTraceeTest.OpenUseAndCloseLibraryInUserCode (Child aborted)
        389 - InjectLibraryInTraceeTest.OpenUseAndCloseLibraryInSyscall (Child aborted)
        394 - InstrumentProcessTest.Instrument (Child aborted)
        396 - InstrumentProcessTest.ExitPayloadDoesNotUseX87Fpu (Child aborted)
        417 - InstrumentFunctionTest.DoSomething (Child aborted)
        418 - InstrumentFunctionTest.CheckStackAlignedTo16Bytes (Child aborted)
        420 - InstrumentFunctionTest.LongEnough (Child aborted)
        421 - InstrumentFunctionTest.RipRelativeAddressing (Child aborted)
        422 - InstrumentFunctionTest.UnconditionalJump8BitOffset (Child aborted)
        423 - InstrumentFunctionTest.UnconditionalJump32BitOffset (Child aborted)
        424 - InstrumentFunctionTest.ConditionalJump8BitOffset (Child aborted)
        425 - InstrumentFunctionTest.ConditionalJump32BitOffset (Child aborted)
        427 - InstrumentFunctionTest.CheckIntParameters (Child aborted)
        428 - InstrumentFunctionTest.CheckFloatParameters (Child aborted)
        429 - InstrumentFunctionTest.CheckM256iParameters (Child aborted)
        430 - InstrumentFunctionTest.CheckIntParametersMsAbi (Child aborted)
        431 - InstrumentFunctionTest.CheckFloatParametersMsAbi (Child aborted)
        432 - InstrumentFunctionTest.CheckNoX87UnderflowInReturnTrampoline (Child aborted)
        437 - InstrumentFunctionTest.LongConditionalJump32BitOffsetBackToBeginning (Child aborted)
        438 - InstrumentFunctionTest.UnableToDisassembleBadInstruction (Child aborted)

If I disable tests in the build the build finishes.
Data does appear to be collected though and looks reasonable.
I get around 23% DWARF unwinding errors.
run.sh and run_service.sh are run using sudo.

Some tests are actually flaky, or require to run as root. The build actually finishes even without the tests passing. But I see that you actually managed to run and take a capture.
23% DWARF unwinding errors sound alerting.  You can also right-click "Select these callstacks" to the unwinding errors in the Top-Down view, and inspect the "broken" callstacks in the "Sampling (Selection)" tab. Without further details, its pretty hard to tell much more.
In general, DWARF unwinding errors are related to missing or broken DWARF unwind information.

We've recently fixed a bug related with the tests, so might be worth another try.


Message is broken

Formatting is completely broken for a couple of days already.
I suppose the message is too big now so it gets split into two messages which, in turn, breaks the formatting.

fixed by https://github.com/DiSiqueira/coronabot/pull/6


Fix steps for running in Visual Studio

@schellap PTLA (These instructions got out of sync with the package layout changes.)

Ah. Thanks for updating these as well.

LGTM.


Unable to resolve module react-dom

bundling failed: Error: Unable to resolve module react-dom from .../node_modules/react-paypal-express-checkout/dist/index.js: Module react-dom does not exist in the Haste module map
When i add yarn add react-dom
After that give error for another dependency.
bundling failed: Error: Couldn't find preset "stage-0" relative to directory .../mdlink-reactnative/node_modules/react-async-script-loader
Thanks in Advance.

Hi, have you got the solution. I also have same issue

me too


BigCommerce API keys

Apologies if this is straightforward new to the project and to using bigcommerce. having trouble with getting api keys setup.
Background on my attempts so far:
Attempt # 1

used the Deployed with Vercel button.
tried to link to local project but didn't pull down env variables ( listed as secrete in the vercel manage project area )

Attempt # 2

tried to generate my own api keys via bigcommerce,
it asked me for 20 different permission questions, and this project doesn't have any documentation on what permissions to give it, plus looks like I might have to generate at least two api keys.

anyway hopefully I can get started on this project and maybe contribute back.

after much looking found #13 which mentions
vercel env pull .env.local
which has fixed my problem.
I'll close the issue, should be mentioned in this project docs however.


Fix #417 Expected Exceptions Message fails to match multi-line messages

Fix from #519

Don't forget to update CHANGES.txt :)

Yes. I did it. CHANGES should be up-to-date.


Move README.md into _shared

Replace the two separate pypackage and pyramid-app README.md templates with a single shared template.
This does result in a fairly long README.md template with a few pypackage-only and pyramid-app-only sections. But I think it's worth it to keep the README.md's consistent across different types of project and to make the template easier to maintain, especially since I'm planning to add a third cookiecutter that will share the same README.md template.
Testing
This shouldn't make any difference to the generated README.md for all the existing projects that're already using the pypackage cookiecutter. For example using cookiecutter-pypackage-test as a test:
$ git clone https://github.com/hypothesis/cookiecutter-pypackage-test.git /tmp/cookiecutter-pypackage-test
$ cd /tmp/cookiecutter-pypackage-test/
$ make template checkout=move-README.md-into-shared
$ git diff
$

This does make a small change to the generated README.md's for projects using the pyramid-app cookiecutter because there were some differences in how the pypackage and pyramid-app README.md's documented the same prerequisites. Fortunately we don't have any projects using that cookiecutter yet other than cookiecutter-pyramid-app-test. Also the new README.md template only puts make services and make devdata in the README.md if the project has services and devdata and not if it only has do-nothing make services and make devdata commands. cookiecutter-pyramid-app-test doesn't have services or devdata so they get removed from its README.md:
$ git clone https://github.com/hypothesis/cookiecutter-pyramid-app-test.git /tmp/cookiecutter-pyramid-app-test
$ cd /tmp/cookiecutter-pyramid-app-test/
$ make template checkout=move-README.md-into-shared
$ git diff
diff --git a/README.md b/README.md
index 2f00562..85524c9 100644
--- a/README.md
+++ b/README.md
@@ -13,23 +13,20 @@ First you'll need to install:

 * [Git](https://git-scm.com/).
   On Ubuntu: `sudo apt install git`, on macOS: `brew install git`.
+* [GNU Make](https://www.gnu.org/software/make/).
+  This is probably already installed, run `make --version` to check.
 * [pyenv](https://github.com/pyenv/pyenv).
-  See [pyenv's README](https://github.com/pyenv/pyenv#readme) for install instructions.
-  First you need to [install the Python build dependencies](https://github.com/pyenv/pyenv/wiki#suggested-build-environment)
-  then on macOS use the **Homebrew** installation method,
-  on Ubuntu use the **Basic GitHub Checkout** method.
+  Follow the instructions in pyenv's README to install it.
+  The **Homebrew** method works best on macOS.
+  The **Basic GitHub Checkout** method works best on Ubuntu.
   You _don't_ need to set up pyenv's shell integration ("shims"), you can
   [use pyenv without shims](https://github.com/pyenv/pyenv#using-pyenv-without-shims).
-* [GNU Make](https://www.gnu.org/software/make/).
-  This is probably already installed or will have been installed while installing pyenv, run `make --version` to check.

 Then to set up your development environment:

 ```terminal
 git clone https://github.com/hypothesis/pyramid-app-cookiecutter-test.git
 cd pyramid_app_cookiecutter_test
-make services
-make devdata
 make help

Try and indent where we can

I've added indentation to the one-liner {% if %} blocks in this PR because as you pointed out for one-liners Jinja2's whitespace control will remove the indentation. Unfortunately this can't be done for multi-line blocks: whitespace control will only dedent the first line not the rest. I think this is a limitation of Jinja2: IIRC there's no way to indent a block in a Jinja2 template and not have those lines be indented in the output file.
We can't use {% include %}, {% extends %}, {% block %}, etc

I think we can solve some of this with a combination of techniques:

Put long sections in separate files and include them, keeping blocks short
Think about keeping non-shared components out of the shared template and using extends

These unfortunately aren't possible. Due to the limitations of cookiecutter one template file within the cookiecutters repo can't really include or extend another. {% extends %}, {% include %} and {% import %} all depend on the searchpath argument to Jinja2's FileSystemLoader: you have to pass it the path or list of paths where it should look for the template files that're being extended or included. And unfortunately cookiecutter is hard-coded to pass no searchpath to FileSystemLoader and this isn't configurable.
So {% extends %}, {% include %} and {% import %} are disabled in cookiecutter. {% block %}, {% super %} and other related features can't be used either because they depend on {% extends %}.
I looked into this when I first created this cookiecutters repo, I couldn't see a way to get it working. I spoke to the cookiecutter and Jinja2 developers at the time and they confirmed that there isn't a way.

There's a horrible hack to get them working put I don't think we should do it

Actually there is a horrible hack to use {% include %} or {% extends %} but it's weird enough that I think we're better off not using it:

Cookiecutter actually does pass one search path to FileSystemLoader: "." which is the template directory of the particularly cookiecutter being used, pypackage/{{ cookiecutter.slug }}/, pyapp/{{ cookiecutter.slug }}/ or pyramid-app/{{ cookiecutter.slug }}/
Any files in those directories get rendered into the project. So if you add a pyapp/{{ cookiecutter.slug }}/includes/foo.md or a pyapp/{{ cookiecutter.slug }}/base/README.md those include files (includes/foo.md) and base files (base/README.md) would themselves get rendered directly into the project by cookiecutter.
We could use our paths_to_remove feature to remove those files from the projects.
It's weird that the include and base files would need to live in each of the individual cookiecutters, not in the _shared/ directory. If we want to split _shared/project/README.md into multiple files for example then we could put includes like {% include "includes/README/installing.md" %} but even though this installing.md file is included by the README.md template in the _shared directory the installing.md file itself would need to live in the individual cookiecutter directories (in triplicate for the three cookiecutters). We could avoid duplicating installing.md by putting in in _shared/includes/installing.md but then each of the three cookiecutters would need to contain a symlink to each shared include or base file. Cookiecutter would then render the include and base files into the project so we'd have to use post_gen_project.py to remove them.

I think the cure would be worse than the disease. We could move README.md's long {% if %}'s into separate files but the way we'd have to do that would be pretty twisted, with files and symlinks forced to live in odd places and post-generation code for removing files from the generated project. The fact that you can get this to work is not intended by cookiecutter, we'd be abusing an unintended implementation side-effect.
I understand that the large {% if %}'s aren't very readable. But it's a very simple approach. It's easy to understand how it works: there is one README.md template, that template contains some {% if %}'s, that's all there is to it. This is much better than crazy hackery involving intimate knowledge of unintended cookiecutter implementation side-effects.

There's an unreleased cookiecutter PR about this but I don't think it's useful

They actually merged a PR in June to add "../shared" to the search paths: https://github.com/cookiecutter/cookiecutter/pull/1485 But they haven't released a new version of cookiecutter since merging that. It's of limited use to us anyway. The include and base files would still need to live in the individual cookiecutters not in the shared directory. It's just that they can now live in pyapp/templates/, pypackage/templates/ and pyramid-app/templates/, meaning they're outside of the {{ cookiecutter.slug }} directory so cookiecutter won't render the include and base files into the projects and our post_gen_project.py won't need to remove them. But you still have the problem that README.md is in _shared but any files it wants to extend or include must live in each one of the individual cookiecutters. And this PR is unreleased anyway.

We could use macros
So I think we're limited to within-a-single-file techniques. I think macros may be all that's available to us. I think macros would maintain the simplicity that the current README.md has with {% if %}'s: it's just one README.md template and it calls some macros. All in one (admittedly long) file, simple to understand what's going on, no crazy cookiecutter hacks. Macros are limited in how much they help us however:

Macros would enable us to define the various parts that are specific to one cookiecutter or another in a sequence of macros at the top of the file, and then have a more concise "map" of the file (including all the {% if %} and {% else %} logic) at the bottom.
You'd still have long macros and they still can't be indented, so the "Am I in an {% if %}?" problem arguably just becomes "Am I in a macro?". The fact that all the macros are at the top of the file eases this somewhat
We can't use macros to break up the large file into smaller ones: all the macros would need to be defined within the file since we can't use {% import %}
It arguably makes readability worse in some ways: the output contents would now be out-of-order within the template file since some parts have been moved into macros at the top. We could mitigate this by moving every section into a macro, even the unconditional ones
It's arguably weird usage to define a bunch of macros that are each only used once and take no arguments. But it lets us split the file between contents (the macros) and map

Overall my feeling is that breaking up README.md into macros might be an improvement, worth a try...

I've spent a couple of hours playing with macros and indentation and I'm gonna say the the original {% if %}-based approach is by far the lesser of two evils here:

Macros by themselves (without indentation) don't make the template any more readable. You just have all the same long blocks but in macros and now with additional code at the end to actually call the macros.
Also it really plays hell with whitespace control: the section at the top of the file containing the actual macros introduces a load of blank lines into the output file (the blank lines between the macros and the newlines before and after the {% macro ... %} and {% endmacro %} lines). If you want any blank lines between all the {% if %}'s and {% else %}'s and macro calls at the bottom of the file to make that Jinja code more readable then those blank lines will get into the output file as well (and so will the new lines before and after the {% if %}, {% else %} and {% endif %} lines). It is possible to get the output right but it's very fiddly and the result is much harder to maintain than the {% if %}'s.

If you then indent the contents of the macros it's even worse. Jinja2 isn't meant to do this: it doesn't have a dedent filter or allow you to indent the contents of a block without that text getting indented in the output file. If you try to add it you get into a fight with Jinja2. Jinja2's whitespace control will dedent the first line of a macro which will then mean that Python's dedent() (called by your custom dedent filter that you've added) will not dedent the rest of the lines (since there's no longer a common indentation shared by all lines including the first one). But if you remove the Jinja2 whitespace control then that fixes the dedenting but also introduces unwanted blank lines into the file.

Basically the whitespace control and dedentation needed to get this working is extremely fiddly and difficult to understand, and that's a much worse maintainability problem than mere long {% if %} blocks.
My suggestion at this point: merge this PR as-is with the {% if %}'s and consider in the future splitting up README.md into a few separate docs/*.md files, probably something like.

Actually scratch all that, I found a couple of non-default settings that make Jinja2 behave much more sanely around whitespace: https://github.com/hypothesis/cookiecutters/pull/82. With that done it's now fairly easy to break README.md into indented macros without introducing a lot of unwanted whitespace changes. I pushed a couple of commits to this PR to do that. But this now depends on https://github.com/hypothesis/cookiecutters/pull/82

Rebased


How many sequences were used for pre-training?

Hi,
I'm trying to use this model for microbiome data.
As a practice, I trained a model using a small dataset (10K sequences extracted from viruses genomes).
Unfortunately, fine-tuned model using that pre-trained model shows extremely bad performance like a random model.
Can you inform me how many sequences were used for pre-training?

Hi,
I'm trying to use this model for microbiome data.
As a practice, I trained a model using a small dataset (10K sequences extracted from viruses genomes).
Unfortunately, fine-tuned model using that pre-trained model shows extremely bad performance like a random model.
Can you inform me how many sequences were used for pre-training?

What finetune task did you choose? Guess the length of finetune sequences and the preprocessing step matter a lot

Hi,
Can I get the exact number of sequences used for training?
For instance, given example_6_3k.txt includes 3,000 sequences.

Sorry, I forget the exact number. It should be around 5 million sequences.

@Zhihan1996
Thanks for your answer.
I have one last question.
Did you sample DNA sequences from one strand or both?
In other words, was the reverse complement sub-sequence involved in the training dataset?

Hi @hjgwak and @Zhihan1996,
Thanks for your interesting work on DNABERT.
Can the preprocessed pertaining data be available as the file format example_6_3k.txt?


Feature/travis config

Let's see if this fixes our build problems.

Looks good to me.  We did have an issue with Travis CI failing the 2.3.0 build, only to pass it when it was re-run. Does that cause us any concern?
Otherwise, it would be great to have the bundler version specification added from the other commit.

It seems like the intermittent failure is time dependent.
I'd suggest the way forward is to add the 2.3 build, and dig deeper if/when we see more failures. That the build reveals a potential problem seems like a bad reason to avoid it. :)

Sure, I agree. I was just thinking the addition of 2.3, which might be its own feature branch with its own testing phase, might be a separate concern. I could go either way on merging it now or waiting.

I force pushed a change removing the 2.3 build.


Support table of contents and relevant headings

This would be a very nice improvement. A large document with bullet points is kind of messy.


Mysql DB Sync + commands

Pointshop is not reacting the way it is supposed to. it's bugging out the buy menu when you try to holster and sell stuff, makes things worse when you try to change between player models and it bugs out. I am using a slightly tweaked playermodel.lua to incorporate hand model changing. It keeps bugging out like this unable to sell or holster or equip but buying still works. It just stays like that forever. Could you take a look at this please. I do not mind providing the changes that i have made and allowing access to my test server i am trying this on.

Go to scriptfodder

WHAT DOES SCRIPTFODDER HAVE TO DO WITH POINTSHOP MADE BY _UNDEFINED

Ok, for more detail. It's only happening to player models. I see no reason for this to be happening. all i added to the original script was places to put the model of the hands, the skin and body group of the hands, added fetching the hands entity and changing the model and skins. I see no reason for it to be acting up. Even for security measure i set the category to 1 player model equipped at a time. one odd thing is i can't take them either. with the admin part i cannot take player models, not like i would need it but just for debugging purposes

You changed the code according to what you said. Not Adams job to fix your code.

but im not saying it has to do with the code i have "Altered". if it does then this can be resolved kinda easy but there is nothing that leads me to figure out what the F*** is going on. also pointshop was made to be altered, you see you have to make scripts in order to use the bloody thing so please shut up and think about it. If someone "Altered" the script like i have, then doesn't that mean that nobody is allowed to post for help. What logic is there then, why don't you tell me if there is some logic here. I did not login to sit there and rant with some random person with no logic sense

Thanks HandsomeMatt


Bug with ARM Corretto + Maven

There appears to be a bug when building a project using Corretto, Maven, and ARM64.  Based on the error message it is likely a PATH issue in the Corretto installation.  If I switch back to AMD64 the problem goes away.  It also goes away if I switch off of Corretto.
Expected Behavior
The project successfully builds with pack
Current Behavior
The Maven layer fails because it cannot find javac.
Pack command
pack build --verbose --timestamps corretto-bug \
  --platform ${BUILDPACKS_PLATFORM} \
  --builder paketobuildpacks/builder-jammy-tiny \
  --buildpack paketo-buildpacks/amazon-corretto,paketo-buildpacks/java \
  --env BP_JVM_VERSION=21 \
  --env BP_MAVEN_BUILD_ARGUMENTS="-Dmaven.test.skip=true --no-transfer-progress package --batch-mode" \
  --env BP_MAVEN_BUILT_ARTIFACT=target/

Error message:
Paketo Buildpack for Maven 6.17.3
  unable to create application layer
  failed to generate expected metadata
  unable to determine java version
  error executing 'javac -version':
   Combined Output: :
  unable to start PTY
  fork/exec /layers/paketo-buildpacks_amazon-corretto/jdk/bin/javac: no such file or directory

Log output is below.  This error does not happen when using AMD64.
...
Running build for buildpack<EMAIL_ADDRESS>Looking up buildpack
2024/07/01 13:35:45.360959 Finding plan
2024/07/01 13:35:45.360992 Creating plan directory
2024/07/01 13:35:45.361239 Preparing paths
2024/07/01 13:35:45.361550 Running build command
2024/07/01 13:35:45.365705
Paketo Buildpack for Amazon Corretto 8.5.1
  https://github.com/paketo-buildpacks/amazon-corretto
  Build Configuration:
    $BP_JVM_JLINK_ARGS           --no-man-pages --no-header-files --strip-debug --compress=1  configure custom link arguments (--output must be omitted)
    $BP_JVM_JLINK_ENABLED        false                                                        enables running jlink tool to generate custom JRE
    $BP_JVM_TYPE                 JRE                                                          the JVM type - JDK or JRE
    $BP_JVM_VERSION              21                                                           the Java version
  Launch Configuration:
    $BPL_DEBUG_ENABLED           false                                                        enables Java remote debugging support
    $BPL_DEBUG_PORT              8000                                                         configure the remote debugging port
    $BPL_DEBUG_SUSPEND           false                                                        configure whether to suspend execution until a debugger has attached
    $BPL_HEAP_DUMP_PATH                                                                       write heap dumps on error to this path
    $BPL_JAVA_NMT_ENABLED        true                                                         enables Java Native Memory Tracking (NMT)
    $BPL_JAVA_NMT_LEVEL          summary                                                      configure level of NMT, summary or detail
    $BPL_JFR_ARGS                                                                             configure custom Java Flight Recording (JFR) arguments
    $BPL_JFR_ENABLED             false                                                        enables Java Flight Recording (JFR)
    $BPL_JMX_ENABLED             false                                                        enables Java Management Extensions (JMX)
    $BPL_JMX_PORT                5000                                                         configure the JMX port
    $BPL_JVM_HEAD_ROOM           0                                                            the headroom in memory calculation
    $BPL_JVM_LOADED_CLASS_COUNT  35% of classes                                               the number of loaded classes in memory calculation
    $BPL_JVM_THREAD_COUNT        250                                                          the number of threads in memory calculation
    $JAVA_TOOL_OPTIONS                                                                        the JVM launch flags
    Using Java version 21 from BP_JVM_VERSION
2024/07/01 13:35:45.365770   No valid JRE available, providing matching JDK instead. Using a JDK at runtime has security implications.
2024/07/01 13:35:45.365929   Corretto JDK 21.0.3: Contributing to layer
2024/07/01 13:35:45.365950     Downloading from https://corretto.aws/downloads/resources/<IP_ADDRESS>.1/amazon-corretto-<IP_ADDRESS>.1-linux-aarch64.tar.gz
2024/07/01 13:35:47.551877     Verifying checksum
2024/07/01 13:35:47.673413     Expanding to /layers/paketo-buildpacks_amazon-corretto/jdk
2024/07/01 13:35:49.844720     Adding 137 container CA certificates to JVM truststore
2024/07/01 13:35:50.108388     Writing env.build/JAVA_HOME.default
2024/07/01 13:35:50.108450     Writing env.build/JRE_HOME.default
    Writing env.launch/BPI_APPLICATION_PATH.default
2024/07/01 13:35:50.108466     Writing env.launch/BPI_JVM_CACERTS.default
    Writing env.launch/BPI_JVM_CLASS_COUNT.default
    Writing env.launch/BPI_JVM_SECURITY_PROVIDERS.default
    Writing env.launch/JAVA_HOME.default
2024/07/01 13:35:50.108479     Writing env.launch/JAVA_TOOL_OPTIONS.append
    Writing env.launch/JAVA_TOOL_OPTIONS.delim
    Writing env.launch/MALLOC_ARENA_MAX.default
2024/07/01 13:35:50.108812   Launch Helper: Contributing to layer
2024/07/01 13:35:50.110031     Creating /layers/paketo-buildpacks_amazon-corretto/helper/exec.d/java-opts
    Creating /layers/paketo-buildpacks_amazon-corretto/helper/exec.d/jvm-heap
    Creating /layers/paketo-buildpacks_amazon-corretto/helper/exec.d/link-local-dns
    Creating /layers/paketo-buildpacks_amazon-corretto/helper/exec.d/memory-calculator
    Creating /layers/paketo-buildpacks_amazon-corretto/helper/exec.d/security-providers-configurer
    Creating /layers/paketo-buildpacks_amazon-corretto/helper/exec.d/jmx
    Creating /layers/paketo-buildpacks_amazon-corretto/helper/exec.d/jfr
    Creating /layers/paketo-buildpacks_amazon-corretto/helper/exec.d/openssl-certificate-loader
    Creating /layers/paketo-buildpacks_amazon-corretto/helper/exec.d/security-providers-classpath-9
    Creating /layers/paketo-buildpacks_amazon-corretto/helper/exec.d/debug-9
    Creating /layers/paketo-buildpacks_amazon-corretto/helper/exec.d/nmt
2024/07/01 13:35:50.111528   Java Security Properties: Contributing to layer
    Writing env.launch/JAVA_SECURITY_PROPERTIES.default
    Writing env.launch/JAVA_TOOL_OPTIONS.append
2024/07/01 13:35:50.111985     Writing env.launch/JAVA_TOOL_OPTIONS.delim
2024/07/01 13:35:50.112003 Processing layers
2024/07/01 13:35:50.112608 Updating environment
2024/07/01 13:35:50.112922 Reading output files
2024/07/01 13:35:50.114692 Updating buildpack processes
2024/07/01 13:35:50.114720 Updating process list
2024/07/01 13:35:50.114888 Finished running build for buildpack<EMAIL_ADDRESS>Running build for buildpack<EMAIL_ADDRESS>Looking up buildpack
2024/07/01 13:35:50.114937 Finding plan
Creating plan directory
2024/07/01 13:35:50.115040 Preparing paths
2024/07/01 13:35:50.115296 Running build command
2024/07/01 13:35:50.150292
Paketo Buildpack for CA Certificates 3.8.2
2024/07/01 13:35:50.150350   https://github.com/paketo-buildpacks/ca-certificates
2024/07/01 13:35:50.150827   Build Configuration:
2024/07/01 13:35:50.150965     $BP_EMBED_CERTS                    false  Embed certificates into the image
    $BP_ENABLE_RUNTIME_CERT_BINDING    true   Deprecated: Enable/disable certificate helper layer to add certs at runtime
    $BP_RUNTIME_CERT_BINDING_DISABLED  false  Disable certificate helper layer to add certs at runtime
2024/07/01 13:35:50.154240   Launch Helper: Contributing to layer
2024/07/01 13:35:50.156644     Creating /layers/paketo-buildpacks_ca-certificates/helper/exec.d/ca-certificates-helper
2024/07/01 13:35:50.161171 Processing layers
2024/07/01 13:35:50.161414 Updating environment
Reading output files
2024/07/01 13:35:50.161445 Updating buildpack processes
Updating process list
Finished running build for buildpack<EMAIL_ADDRESS>Running build for buildpack<EMAIL_ADDRESS>Looking up buildpack
2024/07/01 13:35:50.162556 Finding plan
Creating plan directory
2024/07/01 13:35:50.162580 Preparing paths
2024/07/01 13:35:50.162664 Running build command
2024/07/01 13:35:50.201163 Processing layers
2024/07/01 13:35:50.201264 Updating environment
2024/07/01 13:35:50.201284 Reading output files
2024/07/01 13:35:50.201359 Updating buildpack processes
Updating process list
Finished running build for buildpack<EMAIL_ADDRESS>2024/07/01 13:35:50.201388 Running build for buildpack<EMAIL_ADDRESS>Looking up buildpack
2024/07/01 13:35:50.201694 Finding plan
Creating plan directory
Preparing paths
2024/07/01 13:35:50.201825 Running build command
2024/07/01 13:35:50.237719
Paketo Buildpack for Syft 1.47.1
  https://github.com/paketo-buildpacks/syft
2024/07/01 13:35:50.244544     Downloading from https://github.com/anchore/syft/releases/download/v0.105.1/syft_0.105.1_linux_amd64.tar.gz
2024/07/01 13:35:50.891596     Verifying checksum
2024/07/01 13:35:51.249119     Writing env.build/SYFT_CHECK_FOR_APP_UPDATE.default
2024/07/01 13:35:51.250905 Processing layers
2024/07/01 13:35:51.251140 Updating environment
2024/07/01 13:35:51.251233 Reading output files
2024/07/01 13:35:51.251618 Updating buildpack processes
Updating process list
Finished running build for buildpack<EMAIL_ADDRESS>Running build for buildpack<EMAIL_ADDRESS>Looking up buildpack
2024/07/01 13:35:51.253865 Finding plan
Creating plan directory
2024/07/01 13:35:51.254133 Preparing paths
2024/07/01 13:35:51.254237 Running build command
2024/07/01 13:35:51.292106
Paketo Buildpack for Maven 6.17.3
2024/07/01 13:35:51.292225   https://github.com/paketo-buildpacks/maven
2024/07/01 13:35:51.293045   Build Configuration:
2024/07/01 13:35:51.293169     $BP_EXCLUDE_FILES                                                                                colon separated list of glob patterns, matched source files are removed
    $BP_INCLUDE_FILES                                                                                colon separated list of glob patterns, matched source files are included
    $BP_JAVA_INSTALL_NODE                 false                                                               whether to install Yarn/Node binaries based on the presence of a package.json or yarn.lock file
    $BP_MAVEN_ACTIVE_PROFILES                                                                                the active profiles (comma separated: such as: p1,!p2,?p3) to pass to Maven
    $BP_MAVEN_ADDITIONAL_BUILD_ARGUMENTS                                                                      the additionnal arguments (appended to BP_MAVEN_BUILD_ARGUMENTS) to pass to Maven
    $BP_MAVEN_BUILD_ARGUMENTS             -Dmaven.test.skip=true --no-transfer-progress package --batch-mode  the arguments to pass to Maven
    $BP_MAVEN_BUILT_ARTIFACT              target/                                                             the built application artifact explicitly.  Supersedes $BP_MAVEN_BUILT_MODULE
    $BP_MAVEN_BUILT_MODULE                                                                                the module to find application artifact in
    $BP_MAVEN_DAEMON_ENABLED              false                                                               use maven daemon
    $BP_MAVEN_POM_FILE                    pom.xml                                                             the location of the main pom.xml file, relative to the application root
    $BP_MAVEN_SETTINGS_PATH                                                                                the path to a Maven settings file
    $BP_MAVEN_VERSION                     3                                                                   the Maven version
    $BP_NODE_PROJECT_PATH                                                                                configure a project subdirectory to look for `package.json` and `yarn.lock` files
2024/07/01 13:35:51.316994
Paketo Buildpack for Maven 6.17.3
  unable to create application layer
  failed to generate expected metadata
  unable to determine java version
  error executing 'javac -version':
   Combined Output: :
  unable to start PTY
  fork/exec /layers/paketo-buildpacks_amazon-corretto/jdk/bin/javac: no such file or directory
2024/07/01 13:35:51.317631 Timer: Builder ran for 5.958545336s and ended at 2024-07-01T18:35:51Z
2024/07/01 13:35:51.318236 ERROR: failed to build: exit status 1
2024/07/01 13:35:52.044104 ERROR: failed to build: executing lifecycle: failed with status code: 51

Possible Solution
I suspect that the aarch version of Corretto is installing the JDK in an unexpected location.  Given how abstracted the corretto install is with buildpacks, I was not able to confirm this. I also do not know how to direct the pack CLI to "stop" before the maven layer installation to inspect the docker image at that point.
Steps to Reproduce
I created an example repo with a very simple Java project. The repo has a Github action for the AMD build.  Github does not currently provide ARM runners for free repos.  There is a script in the project for the ARM build:  `
Motivations
We are exploring switching to using AWS Graviton services for our ECS services to reduce cost.  We are currently using Paketo buildpacks for building our docker images and would like to continue to do so.  We hit this bug while exploring how easy ARM builds would be.

Thanks for the detailed information and sample app. I tried to reproduce this but unfortunately, I haven't been able to. My builds just work.
I also took a look at the aarch64 and x86 downloads of the Corretto JVM and compared the two, thinking maybe there was a difference in the file structure that would cause the binaries to not be found. They both had the same structure.
Another possibility would be that the layer flags are not being set correctly. If the build flag isn't set on the JDK layer then those files won't be around for subsequent buildpacks at build time. That could possibly account for why it's failing with the error you reported, but I don't see how that would be architecture-dependent. That logic is the same across both architectures, so if that's the culprit, then I'd expect it to fail no matter where you build it. I'd also expect to be able to reproduce it when I'm testing.
Can you provide some more information about how you're encountering this error?

Where are you building this when you hit the issue? You mentioned Graviton, are you on a Graviton image? Are you on a different ARM64 machine?
What's your pack version?
Is there anything else that might help to reproduce the issue?

Thanks

I haven't seen an update on this so I'm going to close it out. If this is still an issue for you, please seem my previous note and include the requested information. Thanks!


COSBETA-366

Description
Update EU exit date
Checklist:

[x] Automated checks are passing locally.
[ ] CHANGELOG updated if change is worth telling users about.

General testing

[ ] Test without javascript
[ ] Test on small screen

Accessibility testing

[ ] Works keyboard only
[ ] Tested with one screen reader
[ ] Zoom page to 400% - content still visible
[ ] Disable css - does content make sense and appear in a logical order?
[ ] Passes automated checker (automated in build or manual)

Pull Request Test Coverage Report for Build 6303

0 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 94.389%

Totals

Change from base Build 6302:
0.0%

Covered Lines:
10513

Relevant Lines:
11138

💛  - Coveralls


ENHANCE BLOG PAGE DESIGN

Is there an existing feature request for this?

[X] I have searched the existing feature requests

Describe the feature you would like to request
I have improve uts designing as contributor page
What problem does this feature solve?
User experience
Alternative solutions
No response
Confirmation

[X] I have read the Bitbox Contributing Guidelines
[X] I am a GSSOC-EXT contributor
[X] I am a HACKTOBERFEST contributor
[X] I have starred the Bitbox repository
[X] I would like to help implement this feature

Hyy @Sawan-Kushwah, I'm assigning this issue to you and send me the video with PR.


F5 SSO provider

F5's implementation of SAML seems pretty straight forward.
With minor tweaks to my locally installed saml.py I was able to get it working.  I'll see about submitting a PR if I get a chance to set up a development environment.

Supporting providers outside of ADFS and Okta is out of scope for the project right now. We want to make sure that we can test them against a live service, which is a fairly significant undertaking for most IDPs. That said, I'll leave this feature request open. If we get enough love for the feature request we can see about taking it on.

Thanks.  Originally, I was using the scripts provided here: https://aws.amazon.com/blogs/security/how-to-implement-a-general-solution-for-federated-apicli-access-using-saml-2-0/ which work out of the box with our SAML provider, but I wanted to be able to tell my colleagues to use awsprocesscreds (since it had been mentioned in multiple talks at Re:Invent).
The ADFS3 code in that blog post was where I saw the comment "Some IdPs don't explicitly set a form action"  that lead to the fix I submitted to awsprocesscreds, so perhaps there are enough other Identity Providers out there (besides F5) that do not set a form action to justify putting this into the mainstream.


OpenSSL sockets

This might take a while before we get there, but I want to get the following code to work:
TCPSocket.open('natalie-lang.org', 443) do |socket|
  ssl_context = OpenSSL::SSL::SSLContext.new()
  ssl_socket = OpenSSL::SSL::SSLSocket.new(socket, ssl_context)
  ssl_socket.connect
  ssl_socket.write("GET / HTTP/1.1\r\nHost: natalie-lang.org\r\nConnection: close\r\n\r\n")

  p ssl_socket.read
  ssl_socket.close
end

Fun fact: I have never used the SSL sockets directly (normally you would use something like net-http), so I had to look up how to do this.
Some documentation links:

https://wiki.openssl.org/index.php/SSL/TLS_Client
https://wiki.openssl.org/index.php/Simple_TLS_Server (shows the socket more clearly)
https://www.rubydoc.info/stdlib/openssl/OpenSSL/SSL/SSLContext
https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_new.html

I think this is the minimal viable version to get this script to work. It probably has about a hundred security issues (I have no idea what TLS/SSL versions are allowed by default), so please don't use it for any serious applications.


graph: layer tar disassembly and reassembly

Overview
This pull request introduces a library (vbatts/tar-split) that is for inline disassembly of TAR archives. The disassembly does not do any extraction, but preserves the raw bytes of headers and padding from the archive, and defers the extraction. This way validation and extraction continue to be a lockstep process.
The reassembly use this preserved archive data and the relative path of the image layer, to produce a TAR archive stream.
Changes
This change is completely transparent to existing images and installations.
For an docker pull, docker load or docker commit after this feature, there will be a new state file stored for the image layer (e.g. /var/lib/docker/graph/<ID>/tar-data.json.gz).
For existing images that do not have this new state file, the tar archive produced by docker save or docker push will fallback to the traditional graphdriver.Diff as they have been.
Benefits
The benefit of this feature is that rather than hoping the graph.TarLayer produced is deterministic, the tar archive will be reassembled from the raw bytes of the original archive.
Presently the issue exists that an image pulled from a repo like the Docker hub, and then pushed to a local registry may likely have a new digests.
/cc @ncdc @stevvooe @dmcgowan @jlhawn @NathanMcCauley @crosbymichael

Cc @tiborvass

Thanks @vbatts, interesting PR!

Setting this to design review.
I am +1 on using tar split for this use case. This address part of the problem described in #14018.
I still would like to see an abstraction defined so that storing the entire tar contents could also be done at this same level.

@dmcgowan i agree, though i see this transparent integration as a first step, and could provide an early win for consistent checksums.
Seeing this ./graph/ I looked forward to overhauling the whole thing, though I decided that bike shed did not need to block this feature for now.

TESTS PASS. PTAL.

@jonboulle updated

rebased for test fix from master

updated. PTAL

@docker/maintainers can we get this roadmapped for 1.8 release? It will factor into consistent sums of layers for signed manifests by notary ...

I need to rebase since 52f4d09ffb376ffaa6677cb1e0413c6a97f53f24 by @jhowardmsft is merged :-)

rebased. PTAL.

oh, sorry, I didn't realize the PR is still in design review (I left a code-specific comment).
I'm +1 on the design. Keeping the original tar archive headers is the only way to ensure that we can recreate the original Tar archive given . Whether or not we can recreate the compressed archive is still an issue though isn't it? The consistency issue being digests of archives compressed with different algorithms even though they are identical uncompressed.

@jlhawn I really think we should decouple the digest of the uncompressed layer from that of the compressed layer. Downloading an image compressed with gzip and uploading it to a local registry with a different compression algorithm isn't such a bad idea. Perhaps we could also make this possible?

@jlhawn after several in-person discussions, we ruled that the having assumptions around compression is an acceptable position. The golang compress/gzip is consistent to itself, so for the forseeable future the compress will not matter. When we receive a tar.gz compressed by compress/gzip, stash the tar-split bytes and entries; later reassemble the original archive and compress with compress/gzip, the resulting checksum is the same.
Worst case is a) someone manually compresses with GNU gzip, and pushes to the hub b) the project later decides to trial a new compression algorithm that is non-consistent in the output stream (this should never happen).
Both of these cases are not worse than the current situation of hashes changing for the layer of the exact same image on push/save (from clean engines or multiple engines).

@unclejack I am not in disagreement. But let me make clear, this PR is transparent to whether the digest is actually decoupled from compressed/uncompressed layer.

@vbatts LGTM on Design.

@vbatts The compromise on the reproducibility of gzip seems reasonable to me. We can address the gzip stuff later with a similar approach to tar-split for the gzip protocol.

@vbatts

Both of these cases are not worse than the current situation of hashes changing for the layer of the exact same image on push/save (from clean engines or multiple engines).

Agreed. It would be better to (in the future) use the digest of the uncompressed archives so that compression only becomes a consideration during transport and at-rest storage.

@vbatts do we need to vendor those tar and text files that are added here?

@calavera the ./testdata/ stuff? I reckon that can be pruned, but i'm just using the ./hack/vendor.sh. @tianon care for more magic in that script? ;-)

@calavera the ./testdata/ stuff? I reckon that can be pruned, but i'm just using the ./hack/vendor.sh. @tianon care for more magic in that script? ;-)

we can remove it later too, it's cool.

@vbatts 1.8 should not be released without this PR, but the output of layer assembly needs to be verified against the digest. The crc checks do not provide the required level security.
Please do not merge this PR until the digest is properly verified.

@vbatts @stevvooe should we make this top-priority monday?

@vbatts https://github.com/vbatts @stevvooe
https://github.com/stevvooe should we make this top-priority monday?

I'm investigating the crc sum comparison, but as for full digest comparison
(like a sha256), that is not the intention here.
Today, you can modify the contents of ./etc/passwd and get a different
checksum when pushing to a v2 registry. That is still the case with
tar-split, but it as well accounts for the attributes in the initial tar
headers (xattrs, user-xattrs, ordering of marshalled data, atime, ctime,
etc). The crc sum is for basic checking. I will figure that out today.

all, i'm not sure the steps performed where you're seeing the crc sum fail. I've just tested on docker save and push to the registry, with having tampered with a file from the layer's root fs.
Using the docker.io/library/centos image (id 7322fbe74aa5).
I modified the file /var/lib/docker/overlay/c852f6d61e65cddf1e8af1f6cd7db78543bfb83cdcd36845541cf6d9dfef20a0/root/etc/centos-release with the following change:
--- centos-release.orig 2015-07-20 16:57:44.676267450 -0400
+++ centos-release      2015-07-20 16:57:50.998284291 -0400
@@ -1 +1 @@
-CentOS Linux release 7.1.1503 (Core)
+CentOS Finux release 7.1.1503 (Core)

and the output trying to push this mangled image is:
$ docker save centos > c1.tar
Error response from daemon: io: read/write on closed pipe
$ docker push <IP_ADDRESS>:5000/centos
The push refers to a repository [<IP_ADDRESS>:5000/centos] (len: 1)
7322fbe74aa5: Image successfully pushed
c852f6d61e65: Image push failed
Error pushing to registry: Server error: 400 trying to push centos blob -

if you want to put your hands more closely with the assembly/disassembly, I've pushed https://github.com/vbatts/tar-split/tree/master/cmd/tar-split (go get github.com/vbatts/tar-split/cmd/tar-split).
You can disassemble a tar, see the tar-data.json.gz file, and reassemble a new tar archive with the data and relative path.

rebased and isolated logic into functions back inside graph.go

@stevvooe your stance on digest being a part of this PR seems overloaded. I've already said that can be a follow-up PR, but does not need to collide in this particular PR of introducing new logic.
CRC is never intended for cryptographic validation. Let's not muddle these two items together...

a) someone manually compresses with GNU gzip, and pushes to the hub

It's very valid IMO to use systems other than docker build to produce layers and images.  For example, docker build currently runs as root, but it's quite possible to "cross build" completely as non-root and generate tarballs whose files unpack as root.  OpenEmbedded works this way, and GNOME Continuous also does builds as non-root.  (It also helps that OSTree was designed from the start to support non-root writers)
Another instance of course here is for Fedora we use Anaconda-in-ImageFactory, which indeed uses GNU gzip.
And further keep in mind the fact that gzip can have different compression levels.  And there's also other alternative gzip-compatible compressors like pigz that add explicit parallelism.

b) the project later decides to trial a new compression algorithm that is non-consistent in the output stream (this should never happen).

You say "The golang compress/gzip is consistent to itself, " - but will that really hold true over time?  Will the golang authors guarantee from now until the end of time to have the output of that code be reproducible?

We can address the gzip stuff later with a similar approach to tar-split for the gzip protocol.

I'm doubtful - tar-split works fine for handling the variations of tar because the vast majority of the data is the uncompressed regular file content.  But the nature of compression means you get cascading changes in the output any time the input changes a little bit.  So any such approach is going to end up duplicating on disk a lot of data.

@cgwalters I agree on the compression variations, but that much is true presently, but more so. Docker only pushes with the compression is can. Though it presently has the added side effect of also having differing content due to changes in atime, ctime and possibly mtimes. As well as varying ordering of attributes like xattrs in the tar headers.
This PR will give the common denominator of the base tar being consistent, regardless of the compression used on top. For docker push to registry, this will be the 80% use case. For any registry  ops, this will be a tremendous savings on the current model of pushing layers rebuilt from a diff every time.
Eventually I would love to have reference of the uncompressed checksum of the archive, then folks could store on disk how they see fit, and still determine they're using the same layer content.

We can address the gzip stuff later with a similar approach to tar-split for the gzip protocol.

I'm doubtful - tar-split works fine for handling the variations of tar because the vast majority of the data is the uncompressed regular file content. But the nature of compression means you get cascading changes in the output any time the input changes a little bit. So any such approach is going to end up duplicating on disk a lot of data.

indexing the compression is tricky and would be an isolated success, not a large and robust approach.

This PR will give the common denominator of the base tar being consistent, regardless of the compression used on top.

I definitely agree that's valuable.  And honestly, while bitwise layer reproducibility would surely solve some problems, in the end I think a higher priority is finishing Notary.  We've lived with traditional source code tarballs not being reproducible (see https://github.com/cgwalters/git-evtag for that), so we can likely live with Docker images not being easily bitwise reproducible too.
Good support for mirroring helps too, rather than having people trying to do pull -> unpack -> push workflows outside of quick dev feedback cases.

I definitely agree that's valuable. And honestly, while bitwise layer reproducibility would surely solve some problems, in the end I think a higher priority is finishing Notary.

Notary makes the need for this even more imminent. As it is attesting the hashes of tar layers pushed, those hashes are essentially one time. At best we can continue working towards a model where docker save, push, pull, load are all from mirrors of a notary signed registry. Completely out-of-band from the current graph model. This is fine, and being whiteboarded already, but will be an overhaul.
In the interim, as folks look forward to their docker daemon interacting with notary as-soon-as-possible, the need for the checksums signed by notary to be consistent is urgent.

LGTM on condition of #14825 being merged, as well.

So basically we should close that one in favor of #14825.

@icecrime I am still watching the tests over there. :-)

Closing in favor of #12825 (LGTM'ed already).

Reopening, because apparently that was a mistake.

LGTM

LGTM

LGTM


Plugin switch might not be working

I've globally installed imagemin and imagemin-mozjpeg. Then, curious how much difference the mozjpeg plugin makes, on a test image I executed:
imagemin test.jpg > test.min.jpeg
imagemin -p=mozjpeg test.jpg > test.moz.jpeg

Afterwards, test.min.jpg and test.moz.jpg are byte-for-byte identical. Same size, too.
So either the mozjpeg plugin isn't doing anything, or imagemin isn't using it.
I also tried:
imagemin -p=imagemin-mozjpeg test.jpg > test.min.jpeg
imagemin -p=yolo test.jpg > test.min.jpeg

Both produce the same result as in the first test. To me it appears the -p switch isn't working. I'm also not getting any errors, which is somewhat frusstrating - I have no way of knowing what I'm doing wrong, or if it can't find the plugin (and WHERE it can't find the plugin if that's the case). Maybe the plugin spews out some error that I can't see.
I'm on Windows 10, NodeJS 10.13, NPM 6.4.1.

Having the same problem, plugins are not being invoked, and output file is the same as the input file.
I've tried both of the following and neither is working:
                   const modifiedFileBuffer = await imagemin.buffer(originalFileBuffer, {
                        plugins: [
                            imageminWebp({
                                quality: 80
                            })
                        ]
                    });
                    await writeFile("uploads/testing.webp", modifiedFileBuffer);
                    imagemin([path], { destination: 'uploads/' + path.split(uploadDir)[1] + '.webp', plugins: [imageminWebp({ quality: 80 })] }).then((file) => {
                        console.log('Images optimized');
                        console.log(file);
                        cb(null, {
                            path: path,
                            filename: path.split(uploadDir)[1],
                            size: outStream.bytesWritten,
                            dimensions: dimensions
                        });
                    }).catch((err) =>{
                        console.log(err);
                    });

In my case it seems that the plugin was unable to support the file type. I'm not sure why no errors were thrown, but I've solved it using different methods and determining the mime type ahead of time....


1: Torch compiled without CUDA.   2: bad covariance_matrix

Trying to set this up on a windows 10 machine with 16 gig ram and an RTX 3070 8 gig card.
After following the instructions, my first attempt ended with an error saying torch was not compiled with CUDA:
[ERROR] Error during text-to-video generation: Torch not compiled with CUDA enabled
I installed torch and torchvision cuda versions to try and fix this, so the system was now like this:

import torch
import torchvision
print(torch.version)            # Check PyTorch version
2.4.1+cu118
print(torchvision.version)      # Check torchvision version
0.19.1+cu118
print(torch.cuda.is_available())    # Should return True if CUDA is enabled
True

Next attempt, tried both text to video and image to video, both times got an error to do with matrices.
[ERROR] Error during image-to-video generation: Expected parameter covariance_matrix (Tensor of shape (4, 4)) of distribution MultivariateNormal(loc: torch.Size([4]), covariance_matrix: torch.Size([4, 4])) to satisfy the constraint PositiveDefinite(), but found invalid values:
tensor([[ 1.0000, -0.3333, -0.3333, -0.3333],
[-0.3333,  1.0000, -0.3333, -0.3333],
[-0.3333, -0.3333,  1.0000, -0.3333],
[-0.3333, -0.3333, -0.3333,  1.0000]])

Installed older versions of cuda enabled torch and torchvision.  That got rid of the matrix error.
(pyramid) E:\PyraF\Pyramid-Flow>pip install torch==2.1.2+cu118 torchvision==0.16.2+cu118 -f https://download.pytorch.org/whl/torch_stable.html
Was running but then ran out of memory.
ERROR] Error during text-to-video generation: CUDA out of memory. Tried to allocate 80.00 MiB. GPU 0 has a total capacty of 8.00 GiB of which 0 bytes is free. Of the allocated memory 13.30 GiB is allocated by PyTorch, and 235.89 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
12%|█████████▋                                                                   | 2/16 [08:39<1:00:35, 259.71s/it]
[ERROR] Error during text-to-video generation: CUDA out of memory. Tried to allocate 76.00 MiB. GPU 0 has a total capacty of 8.00 GiB of which 0 bytes is free. Of the allocated memory 13.28 GiB is allocated by PyTorch, and 259.42 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
I'm not sure how I'm supposed to enable the low memory stuff that makes it run on 8 gig cards.

Glad to see that you resolved the initial issues using an older PyTorch version (see https://github.com/jy0205/Pyramid-Flow/issues/121 for more discussions).
For the CUDA out of memory error, please enable sequential CPU offloading according to https://github.com/jy0205/Pyramid-Flow/pull/75. It will be much slower, but might work with 8GB GPU memory.

I went into the code and changed pyramid_dit_for_video_gen_pipeline.py line 332
cpu_offloading: bool = True, # If true, reload device will be cuda
After that it works, and I successfully rendered a text to video. But I got a weird new issue.
Looking in the Anaconda console I can see it rendering all the seconds, I've tried 4 seconds and 16 seconds.
But then the video created doesn't match, the first was just 1 second and the other was just 5 seconds.
I don't know if I should mention that as a new issue.  Is it possible to save the individual frames?

I went into the code and changed pyramid_dit_for_video_gen_pipeline.py line 332
cpu_offloading: bool = True, # If true, reload device will be cuda
After that it works, and I successfully rendered a text to video. But I got a weird new issue. Looking in the Anaconda console I can see it rendering all the seconds, I've tried 4 seconds and 16 seconds. But then the video created doesn't match, the first was just 1 second and the other was just 5 seconds.
I don't know if I should mention that as a new issue. Is it possible to save the individual frames?

Nice observation. The temp parameter does not specify the generated video length in seconds, but the number of latent frames generated. In fact, 4 latent frames correspond to 1 second, while 16 latent frames correspond to 5 seconds. Please see https://github.com/jy0205/Pyramid-Flow/issues/135#issuecomment-2436723590 for details.

I see, thanks.

I guess I'd finish with the following suggestions:

Something to make sure the conda environment installs CUDA enabled torch and torchvision.
Either some way to enable cpu_offloading from the Gradio UI, or at least a note making it a bit clearer what you have to do to get it working.
Maybe a note on the Duration parameter explaining that it isn't seconds, but latent frames.

I'm still impressed that it managed to run on a 16 gig RAM and 8 gig GPU though, well done! :)
You can close this if you like.


support for wsl2 binding volume with distro in  windows

currently  using with  wsl2 but  alternativly slower than  linux
whenever  I started the docker  I get alert from notification from windows  to use  it  by distro mapping
when I click the alert then it  goes to
https://docs.docker.com/docker-for-windows/wsl/#prerequisites
Describe the solution you'd like
Hope  laradock support for  using  wsl distro below configuration.

Why don't you want to store the laradock files in the WSL filesystem? It works great.


'OmniSharp: Select Project' resulted in an error

We have written the needed data into your clipboard because it was too large to send. Please paste.

I was told in a previous issue that this is fixed in a prerelease:
https://github.com/OmniSharp/omnisharp-vscode/releases/tag/v1.23.18-beta2

how do i use the pre-release? it doesn't seem to be published in a way that i can easily select that version

https://github.com/OmniSharp/omnisharp-vscode/wiki/Installing-Beta-Releases

can someone on the omnisharp team please release the fix, or at least rollback breaking changes... ?, my omnisharp has been broken for days now :(

team please release the fix, or at least rollback breaking changes...

@jannickj Sorry, the changes in the new VS Code release conflicted with our existing behavior. Please try installing one of our prerelease builds (https://github.com/OmniSharp/omnisharp-vscode/releases/tag/v1.24.0-beta1) that include a fix. At this point we won't be able to do a proper release until after the winter holidays.

yeah my bad I forgot to update my comment I thought it was a broken release on omnisharp, but it was vscode that made the breaking change.

When I uninstall the extension C# Dev Kit, the problem is fixed. I think that has a conflict with that extension and Omnisharp.

Closing as original issue is resolved.


Highlight focused window

Request:

I set up highlighting of the focused window by changing its border properties similarly as described on the Krohnkite GitHub page. In order to do this, one has to modify various settings and even KDE config files directly. Moreover, the effect of these settings is hard to understand without fully understanding how KWin works. After an evening or two I managed to set it up as I like, but it was a frustrating experience, and I already forgot how I did it (I remember that at some point the borders were not visible even though I followed every step in the Krohnkite manual, that at some point the switching of focus between windows caused other windows to jump a bit, and that at some point I had to find out that for some KDE themes, including Breeze, the custom color configuration gets overwritten by a default one at a different place)

Solution:

Automatize setting up highlight of the focused window by changing properties of its borders.

Alternative:

Properly document how to do it for at least on KDE theme

If Khronkite already has a wiki page on this, it is difficult to understand for inexperenced users, a solution with border highlighting already exists when using a custom theme (a seemingly simpler solution), and the experience when using it is buggy, it doesn't really seem like something worth working on for me personally. PRs are open for wiki page edits if you feel this information is really necessary.

Kwin Oxygen theme applies a glow to active windows. It seems the most officially supported way to achieve what you want. Color and extension of the glow is configurable on the theme options.

I use this as a solution and works pretty well:
https://github.com/matinlotfali/KDE-Rounded-Corners


Switching multiple router instances at runtime

Regarding question #2596 it's possible to switch a router instance during runtime to react to route changes. Especially in a hot module replacement environment this is amazing! Thanks for that hint! It works great!
...but only if a don't work with virtual hosts. In our setup we create multiple router instances. One for every configured virtual host. Than we're using expressjs/vhost middleware to check onto which router we have to "redirect" the request. If routes will be changed during runtime we recreate every router instance like we do it with just one host (one router) but these changes won't be visible in the apps.
Does the router replacement on express side only work with one instance per runtime?
We're on express version 4.16.2.
var routing = function (vhost) {
                return function (req, res, next) {
                    _vhosts[vhost]["router"](req, res, next);
                }
            };
appHttp.use(routing(thisVhost));

_vhosts[vhost]["router"] contains a router instance for every virtual host. If there is no virtual host there will be a "*" entry and therefore only one router instance.

Hey @moecre.  Sorry there is a lot of activity in this repo, so sometimes issues slip through the cracks.  What you are doing is close to what I would do.  I don't know much about the vhost module, as I have never used it, but it all looks similar to an answer I gave the other day to a similar question.  Does this look like what you need?  https://github.com/expressjs/express/issues/1765#issuecomment-377987574

I dont think https://github.com/tommymessbauer/express-vhost is being maintained anymore. This one looks like it is. https://github.com/expressjs/vhost

@wesleytodd Thanks for the hint! I wasn't aware of the fact that I can have multiple Router instances with new express.Router(). I thought that express.Router() is kind of a factory.
Now every vhost gets his own router instance and with hot module replacement I switch instances if there are changes to controllers or routes configs. Works very well!! Many thanks again!


runtime/cgo: should not set -Werror

What version of Go are you using (go version)?

$ go version
go version go1.21.3 linux/amd64

Does this issue reproduce with the latest release?
This was reported by a user a while back for go 1.19, but looking at the source for 1.21.3 as indicated in the gentoo bug, the issue still exists.
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/william/.cache/go-build'
GOENV='/home/william/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/william/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/william/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/lib/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/usr/lib/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.3'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='x86_64-pc-linux-gnu-gcc'
CXX='x86_64-pc-linux-gnu-g++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1224117578=/tmp/go-build -gno-record-gcc-switches'

What did you do?

The only thing I can provide is the comment in the gentoo bug linked above.
The distro policy is that we shouldn't be setting -Werror on any compilations.
What did you expect to see?
cgo seems to force -Werror into all compilations of c code.
What did you see instead?
This should not happen according to our distro policy linked in the bug because it causes compilations to break much more often than necessary.
I don't know if you are willing to make this change upstream or not, but if you aren't, can you please advise me as the maintainer on Gentoo with regard to making this change?
Thanks for your time,
William

cgo seems to force -Werror into all compilations of c code.

The -Werror in the runtime/cgo package only affects the C code in the runtime/cgo package. It is not adding -Werror to user C code. Are you seeing -Werror causing failures to building the Go standard library itself?

@cherrymui I have not, but here is another bug that has been marked as a duplicate and seems to show that -Werror is added to user code.

The build.log.xz from that bug seems to indicate that all -Werror are applied to the runtime/cgo package itself.

Ok, that makes sense.
If I set CGO_CFLAGS to some value, would that replace the CFLAGS that are being applied to this code? I have a pending proposal to do that.

If I set CGO_CFLAGS to some value, would that replace the CFLAGS that are being applied to this code?

Yes and no. CGO_CFLAGS will be added to the C compilations, but not replacing the existing flags. I don't think we document/guarantee the order of flags that are passed. So if you set CGO_CFLAGS=-Wno-error, it may or may not cancel that -Werror.
Just to add, from the build.log.xz, the -Werror doesn't seem to cause any compilation failures.

I realize it isn't causing any compilation errors, but I'm getting flagged by our distro's qa just because it is there. Their documentation suggests dropping it for releases and keeping it for development.
Can you give me a way to disable the adding of -Werror, maybe an environment variable or a command line switch that the make scripts can use?

I don't think we currently have such a way. One way that I can think of is to have a wrapper script for the C compiler that scans the flags and drops -Werror before invoking the actual C compiler, and set CC to the wrapper script.
Are you proposing we introduce a way to do that?

I'm not sure that would be a good way to do it since we already set CC, so I'm going to bring one of our toolchain guys into this bug.
@thesamesam What do you think?

Policy-wise, I understand that some people may not agree with using -Werror. But I also don't really understand why it is a problem if one wants their own code to be compiled with -Werror (not other's code), or in general how their own code should be compiled. Thanks.

The issue is it makes building Go (or something using Go, given this came up when building e.g. Podman) a pain when testing new compilers which may introduce new warnings.

@cherrymui Do you have any thoughts on this?

Personally I don't have strong opinion about -Werror. But I think people frequently build Go with new versions of GCC and clang, so if a newer version of GCC or clang introduces new warnings, the failure is reported in a timely base. If you test a new version of GCC or clang and see such failures, you are also welcome to report. Or are you testing more customized C toolchains?

With CGO_ENABLED=1 and trying to build a darwin executable on Linux I get:
clang: error: argument unused during compilation: '-arch arm64' [-Werror,-Wunused-command-line-argument]

And setting CGO_CFLAGS="-Wno-error" does not override this, as the -Werror is added by go after that.

If you test a new version of GCC or clang and see such failures, you are also welcome to report. Or are you testing more customized C toolchains?

Sorry, I thought I'd answered this. Yeah, if it's simply a new default warning with GCC or Clang, we should report it and get it fixed in Go if appropriate, but if we're e.g. customising the toolchain to enable our own stuff by default, that becomes more problematic. It's also then a pain for keeping older Go versions working.

Not to miss the obvious, but I would expect that you could avoid the clang error with CGO_CFLAGS=-Wno-unused-command-line-argument or CGO_CFLAGS=-w.
That said, I think that we should always put CGO_CFLAGS last on the command line.


docs: update snap sync example conf

To use snap sync, you need install snapsync module first. This PR update the config in snap sync example.

Update: Just found there is auto install description in docs:

May I didn't use the latest mud version, I use 2.0.0-alpha.1.251+deda0235


Update Post “dva-c01-studying-s3-cloudfront”

Automatically generated by Netlify CMS

👷 Deploy Preview for dazzling-dieffenbachia-448a02 processing.

Name
Link

🔨 Latest commit
b9fc84accbd661fdf2add107753f2da85db7f6b8

🔍 Latest deploy log
https://app.netlify.com/sites/dazzling-dieffenbachia-448a02/deploys/62e19a1219a6240008becdb5


Fixing available resources for non-locked semaphores

As discussed in #30, available_count will return @resource_count if a semaphore is created and never locked. With a test.

Awesome, thanks!


For final model, do hyperparam optimization

Ideally with wandb sweep: https://github.com/ivadomed/ivadomed/issues/1124

I didn't do a wandb sweep per se but I played with the following important hyperparameters during my experiments:

Batch-size --> [2, 4, 8]

Ideally, the smaller the batch-size the better the generalization (see abstract of this paper). Batch-size of 2 could fit on romane's GPU so I fixed batch-size to 4.

Optimizer --> [Adam, SGD]

I tried SGD with a learning rate of 0.01 (same as nnUNet's) but I noticed that the performance was not close to Adam with learning rate of 1e-3. SGD was also taking a longer time to converge than Adam.

Model --> Monai's UNet, ivadomed's Modified 3D UNet

This is basically comparing different implementations of the UNet model. Monai's model was simply not powerful enough compared to ivadomed's (dice score difference of 0.1-0.2 between the two models). Hence, I fixed the model to the ivadomed's implementation

Patch-size --> [64x128x64, 80x192x160, 160x224x96]

This is the single most important hyperparameter. In general, smaller patch sizes tended towards more CSA bias compared to models with larger patch sizes. Internal progress reports show the CSA plots.
NOTE: These values are chosen by keeping the median patch size 192x228x106 as the reference (i.e. the tested patch size are close to some multiple of the median patch sizes).
In view of these results, the patch-size is fixed to 160x224x96.

i suggest adding the info above to the manuscript method section before we forget

We have a close-to-final version (just need to run on more seeds now) with all hyperparam tuning already done. Those hyperparms are described in the manuscript, hence closing!


Tuple with Optional models still expects fixed length

I am currently working with nested models which look like following:
class Model1(BaseModel):
    count: int

class Model2(BaseModel):
    method: str
    selector: dict
    count: int

class Final(BaseModel):
    slow: bool
    models: Tuple[
        Optional[Model1],
        Optional[Model2],
    ]]

I am expecting the Tuple to have a variable length since I'd like to pass in either Model1 or Model2 or both based on the requirements. However, the tuple seems to be expecting a fixed length of 2.
Please note that Union is not an option for me since I expect number of models to increase in future and the possible combinations will increase as well.
I'd appreciate any guidance.

Hi @shubhendumadhukar
When you write Tuple[Optional[Model1], Optional[Model2]]
you actually expect a tuple of length 2 with None or an instance of Model1 at index 0 and None or an instance of Model2 at index1.
I think I would simply use Tuple[Union[Model1, Model2],...] and then check in a validator the length and duplicates

It works perfectly. Thank you!


Custom Swipe Disable

can i disable custom swipe image .just slide by interval.

@PkXross No you can't do that in actual implementation


Update zio-config, zio-config-typesafe to 2.0.0

Updates

dev.zio:zio-config
dev.zio:zio-config-typesafe

from 1.0.10 to 2.0.0.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "dev.zio" } ]

labels: library-update, early-semver-major, semver-spec-major

Superseded by #85.


its possible run without sudo or root user ?

its possible run without sudo or root user ?
kubevpn connect --namespace=staging
INFO[0000] [sudo kubevpn connect --namespace=staging]
2022/01/11 11:30:53 connect.go:40: stat /root/.kube/config: no such file or directory
WARN[0000] exit status 1

for example i have user = user123 with kubeconfig in
/home/user123/.kube/config

running tools is not good as root

its possible run without sudo or root user ?
kubevpn connect --namespace=staging
INFO[0000] [sudo kubevpn connect --namespace=staging]
2022/01/11 11:30:53 connect.go:40: stat /root/.kube/config: no such file or directory
WARN[0000] exit status 1

for example i have user = user123 with kubeconfig in
/home/user123/.kube/config

running tools is not good as root

please use command kubevpn connect --namespace=staging --kubeconfig=/home/user123/.kube/config

yes, i know this issue before, but i did't found any good solution, you can use abstract file path to avoid this issue

okay thanks

i already fix this bug, now you can build a kubevpn manually，and use command kubevpn connect --namespace=staging, it works fine, thanks to for report this issue :)

its possible run without sudo or root user ?
kubevpn connect --namespace=staging
INFO[0000] [sudo kubevpn connect --namespace=staging]
2022/01/11 11:30:53 connect.go:40: stat /root/.kube/config: no such file or directory
WARN[0000] exit status 1

for example i have user = user123 with kubeconfig in
/home/user123/.kube/config

running tools is not good as root

please use command kubevpn connect --namespace=staging --kubeconfig=/home/user123/.kube/config

already fix it


how work the server?

Im building a package that use this server to get autocompletion for php, but until now i generate the index file (.padawan/project), after this point im a little lose cuz i start the padawan server but i don't know to what url make the query for the autocompletions.
I know that in the realm u have linked a vim plugin and until i see u call to a complete method passing 4 parameters, im trying to do that with a tester app to http request and restful api but i don't get any completion passing the 4 parameters.
Exists some doc about how the server use this 4 parameters or if need something more?

@Azakur4 hi!
First of all, have a look at this python client https://gist.github.com/mkusher/43bcff85d5e2f3ec3c55 and angular.js client http://mkusher.name/padawan/app.js
As of now there are 4 urls(You can see them here):

complete
update
generate
save

Talking about /complete you've just need to send in request uri(query string):
        params = {
            'filepath': filepath.replace(curPath, ""),
            'line': request_data['line_num'],
            'column': request_data['column_num'],
            'path': curPath
        }

and file contents in request body

Hi man, yeah i figure out, but until now was very hard understand the server. Do you have a place that we can chat need some advices if u have a time to help me.

I think skype chat is the right place, my nickname is mkusher

Added my nickname is Azakur4, please accept

Documentation can be found here https://github.com/mkusher/padawan.php/wiki/Editor-plugins


Use of HTTP Without TLS

Greetings,
I am a security researcher, who is looking for security smells in Chef scripts. I found instances where the HTTP protocol is used instead of HTTPS (HTTP with TLS). According to the Common Weakness Enumeration organization this is a security weakness (https://cwe.mitre.org/data/definitions/319.html). I was wondering why HTTP is used? Is it because of lack of tool support?
I am trying to find out if developers are forced to adopt bad practices due to lack of tool support when it comes to the HTTPS protocol. Maybe it is due to dependency on a resource that uses HTTP?
Any feedback is appreciated.
Source:

https://github.com/cookbooks/ic-cassandra/blob/master/cookbooks/hadoop_cluster/recipes/add_cloudera_repo.rb

Please report upstream.


This apparently fails with yarn

For some reason I can't really fathom (I mean, why npm does not fail) but check the result here

It might be because npm runs scripts as in node xxx while yarn runs the script by itself, using its shebang

Does 1.2.10 resolve this issue?

Let me check. Thanks anyway!
El sáb, 4 dic 2021 a las 14:39, Ben Vey @.***>)
escribió:

Does 1.2.10 resolve this issue?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Wiiseguy/aqa/issues/1#issuecomment-986028748, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAAD5F3MCG23WABFZPE2B3UPIKYTANCNFSM5JLQ43UQ
.

--
JJ

It works now with yarn. Thanks!