Allow hitting production web-secure from staging

Details

Fixed Issues
$ https://github.com/Expensify/App/issues/7371
Tests
There's no way to test this on dev really.
QA Steps

Ensure the user preference for the staging secure server toggle is off
Try to log in with the Chase credentials
Verify it doesn't work (because we are not on staging secure)
Toggle to staging secure server
Try to log in with the Chase credentials
Verify we can do so now since we are in the sandbox

[ ] Verify that no errors appear in the JS console

Tested On

[x] Web
[ ] Mobile Web
[ ] Desktop
[ ] iOS
[ ] Android

Screenshots

Web

Mobile Web

Desktop

iOS

Android

Triggered auto assignment to @danieldoglas (InternalQA), see https://stackoverflow.com/c/expensify/questions/5042 for more details.

@stitesExpensify looks like this was merged without passing tests. Please add a note explaining why this was done and remove the Emergency label if this is not an emergency.

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

This was merged with passing tests. Issue is here https://github.com/Expensify/Expensify/issues/189984

🚀 Deployed to staging by @stitesExpensify in version: 1.1.36-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

Tried testing this and I get a CSP error on staging:

I was testing just now and got the same as @roryabraham

Technically, this is QA pass because trying to log in didn't work :trollface:

I'm going to work on fixing the CSP

https://github.com/Expensify/Cloudflare-Workers/pull/29

CSP is fixed:

I did the following:

Ensured that the Use staging secure server toggle was off.
Tried to add a Chase bank account. Was taken to production Chase OAuth, and the sandbox credentials didn't work.
Toggled the Use staging secure server on.
Tried to add a Chase bank account. Ended up using Plaid sandbox mode (was not taken to actual production Chase OAuth login site), and sandbox credentials worked.

Which I think means this is a QA pass ✅ 🎉

🚀 Deployed to production by @roryabraham in version: 1.1.36-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Start migrating to doctest from gtest

Motivation

gtest makes it easy to do the wrong thing, see [https://github.com/sorbet/sorbet/pull/2968].
Test plan

See included automated tests.

Could we finish the migration before merging to not be in the in-between state?

Sure, we can do that.


Remove txt report

I remember that someone talked about this before. The txt report is kind of useless. The information that if give is not easy to read by a human and the information is not that interesting for a script to parse. I can imagine people to use it to count the number of issues based on the number of lines in that file. But I think that's not a good reason to keep a txt. If someone wants that they can build their own reporter that just write the count.
The only use case for it is to manually create a baseline. But that's a really odd use case. And if you really want to do that you can create it automatically and them hand-craft it from there.
So for those reasons I vote to remove it.

Warnings

:warning:

It looks like you're editing the un-versioned copy of our website. This affects only users on the 'next' version of detekt, and it's correct only if you intend to document a future change or feature. If you intended to make a change also for the current version of detekt, please make sure you edit the equivalent file inside website/versioned_docs/ as well.

Messages

:book:

Thank you very much for making our website better :heart:!

  Generated by :no_entry_sign: dangerJS against ed62e7e327b2a311e95d5cb41d69a70e55538a28

One helpful thing is that it's the easiest format to review from a CLI after a detekt run. And it's also easy to concatenate that output into a file instead.

The txt report is kind of useless. The information that if give is not easy to read by a human and the information is not that interesting for a script to parse.

Instead of removing it, we can improve it so it's more usable for human readers, I wouldn't expect scripts to parse it, that's what #6879 is for.

Instead of removing it, we can improve it so it's more usable for human readers, I wouldn't expect scripts to parse it, that's what #6879 is for.

Isn't the MD format already serving this same need? If so we could just remove txt or "alias" it to output the same output as the md reported

Isn't the MD format already serving this same need?

I guess it does - TXT output is more concise though.
Personally I rely on Gradle output and never look at the reports so I'm not personally affected by this, but it's not exactly a maintenance burden to keep it and others might use it. But we can revert if anyone complains after an RC :)

LGTM after a rebase


[Features Request] Various Improvements

Hi, I've "compiled" a list of IMHO useful improvements for the plugin. I'd also like to offer my support in coding some of them (namely #1, #4 and #5) for which I'll try to make PRs in the upcoming days.
Quality of life:

[x] #7
[x] #8
[x] Exclude from results items that have an empty "FoundIn" columns

Missing Rules:

[x]  #9

Windows Rules:

[ ] Add "dangerous" Windows API/Windows-related vulnerabilities

PS: feel free to split them into specific issues we can address or rework any of those items. :)

HI, thanks for the list. I will look at ways to make #2 and #3 happen (#3 should be quick as well as #1). I am not sure how much time I will have left until end of this year so please be patient :)

Having second thoughts on no.3, this will prevent marking items that are called from piece of code that is not marked as function. This would be undesirable behavior when reversing bare metal firmware (which we do a lot) as it often happens that all functions are not marked as functions even though they are.

I have addressed no. 1 and no. 3 in the dev branch. Please use that for your PR with no.4 and no.5. There is still need for some fine tuning and adjsutments, I will merge to main once verything is ready.
https://github.com/Accenture/VulFi/tree/v2_dev

Having second thoughts on no.3, this will prevent marking items that are called from piece of code that is not marked as function. This would be undesirable behavior when reversing bare metal firmware (which we do a lot) as it often happens that all functions are not marked as functions even though they are.

You're right it's better to have this type of information rather than to completely lose it. Worst case scenario one can "hide" them from the results after exporting them.

This (https://github.com/Accenture/VulFi/pull/9) fix the "Missing Rules: _stdio_common_vsprintf" task

For the Windows API/Windows-related vulnerabilities I'll need a bit more time as I'm quite busy atm

I will merger the branches to propagate changes so that those can be included in the new IDA plugin manager. Addition for the Win API calls will be included separately.

Sure, go ahead :D. I'm sorry but I didn't had the time to add that yet :(

I'm looking forward to your idea of Add "dangerous" Windows API/Windows-related vulnerabilities. At the same time, I want to know what progress it has currently. I think we can start with the high-risk functions of the Windwos api and some high-risk functions of the kernel.


Support sparse glyphs in .glyphs files

We see layer identifiers in Texturina that don't resolve. When this occurs it appears there is an associated master id:
associatedMasterId = "C4D96FDC-31F2-4CF4-BEA5-4CCE99FECD99";
layerId = "1FFA553E-9323-49A1-BA35-824C5E633887";

Not quite sure what this is; a "virtual layer"? Could also be associatedMasterId is unrelated, it just sounds related and sits right beside layer ids :)

See https://github.com/Omnibus-Type/Texturina, sources/Texturina.glyphs

https://github.com/schriftgestalt/GlyphsSDK/blob/Glyphs3/GlyphsFileFormat/GlyphsFileFormatv3.md

associatedMasterId string: ID of the master the layer is linked to. Not present if it equals layerID, i.e. if the layer is in use as master.

every layer in Glyphs.app is either a 'master' layer (one required master layer in each glyph, for each of the font masters) or it is a non-master layer, but it must still be associated with one of the master layers.
When associatedMasterId == layerId, then it is a master layer, and associatedMasterId is omitted on serializing.
Non master layers whose associatedMasterId != layerId can be either temporary drafts that are not taken into account when building the font, or they can be so-called "brace" or "bracket" layers...
It's very complicated, and I don't have the time right now to fully explain. You could start by searching for "associatedMasterId" inside glyphsLib to get an idea.
How about you skip these for the time being, and we'll get back to this later?

When associatedMasterId == layerId, then it is a master layer

to be more precise, a glyph layer corresponds to a font master when its layerId == master.id; it is also true that when this is the case, then its associatedMasterId == layerId (== master.id)

Per IM, to just compile the base default glyphs we can just ignore the ones whose layer id doesn't match any master. We'll use them for other things, perhaps alternates, but not the default glyph.

Non-master layers (associatedMasterId != layerId; layerId not in set(m.id for m in font.masters)) can be of several types. If you right click on such a layer in Glyphs.app you'll see from the drop-down menu the various kinds:

so-called "brace" layers, or additional intermediate masters for individual glyphs.
For Glyphs v2, there's this doc: https://glyphsapp.com/learn/additional-masters-for-individual-glyphs-the-brace-trick
For Glyphs v3, I haven't found an equivalent document yet. By looking at glyphsLib code, the difference seems to be that, instead of embedding the axis values directly in the layer.name like v2 does (e.g. "{500, 100}"), in v3 brace layers have a "coordinates" key in the layer.attributes containig the list of axis values, and the layer.name itself is irrelevant. I think
In this context, the associatedMasterId that links the given brace layer with some master, is used in order to fill in the omitted axis values with those from the associated master. E.g. if there are 3 axes but the brace layer only differ for the first one, its coordinates can contain only the first axis value and the rest are filled with the master's axis values.
For reference, here is where glyhphsLib converts from brace layers to designspace source layers (stored as additional font layers named after the brace layer within the UFO of the associated master). And here is how glyphsLib identifies a layer as "brace" in v2 and v3 and retrieves its coordinates.
I think we want to support these as they enable a degree of sparseness in the variable glyph sources. Basically, speaking in terms of our IR, some variable glyphs may define more sources than others. In ufo2fontir we already support this by loading glifs from the given DS source.layer optional attribute (corresponding to the name of the UFO font layer where glyph sources are contained).

so-called "bracket" layers, i.e. alternate layers substituted when axis conditions are met (GSUB feature-variations)
For Glyphs v3: https://glyphsapp.com/learn/switching-shapes
For Glyphs v2: https://glyphsapp.com/learn/alternating-glyph-shapes and https://glyphsapp.com/learn/alternating-glyph-shapes-with-multiple-axes
These are more complicated in that, even if Glyphs.app represent them as layers within a glyph, they actually end up becoming alternate glyphs to be substituted via GSUB in the final font. In fact when doing glyphs=>UFO that's what we do, we create new glyphs from them and add designspace <rules> that define when to substitute which. I suggest we exclude these for the time being.

I would also ignore for now the various color layers. I presume "Color" means COLRv1, "Color Palette" means COLRv0, "iColor" means sbix (I know, Georg is good at naming stuff) and "svg" well is self-explanatory.
By the way, the type of layers are not mutually exclusive and one can define at the same time an intermediate (brace) layer that is also, say, an alternate (bracket) layer, and in fact that drop-down menu allows you to select multiple ticks at the same time. Actually my minds blows away trying to imagine such a hydra-headed layer...

Forgot to add, if a non-master layer is neither of these special layer types (e.g. immediately after you click "+" to create one), then it is not going to be compiled in the final font, but is simply used as a sort of draft, or maybe kept in the source as an earlier revision for whatever reason. We can safely ignore those.

I just heard there's also a new kind of masters in town, called "Virtual Master", see https://github.com/googlefonts/glyphsLib/issues/859

Not exactly new, I think G2 had them.

That is probably what is called sparse masters elsewhere.

I've come across the "sparse masters" terminology when talking about "brace" or intermediate layers, which work like additional masters that are only defined for specific glyphs (as opposed to master layers which by definition are defined for all the glyphs). E.g. the crossbar of lowercase e may need an additional master along weight axis.
These "Virtual Masters" are somewhat similar and in fact use the same mechanism used for intermediate or brace layers, as described in the "Optional: add a virtual master" paragraph of the https://glyphsapp.com/learn/creating-a-variable-font doc page.
Below is the inline documentation provided when you add such a custom parameter from within Glyphs.app UI itself (emphasis mine):

Defines a font master for a variable font, thereby extending its design space accordingly. Rather than a ‘real’ master, which is defined in File > Font Info > Masters, a virtual master can only be drawn in the form of a Brace layer, i.e., a glyph layer with a name consisting of or ending in the comma-separated design space coordinates between curly braces, e.g., ‘{100, 300}’. A virtual master makes most sense for design axes that only apply to a limited number of glyphs, e.g., an axis that controls the middle crossbar heights of letters like A, E, F and H. The main advantages are that only affected glyphs need to be managed, and kerning does not need to be redone for an extra ‘real’ master.

Looks like it should be doable to translate these to fontmake-rs' glyph intermediate representation, where the sparseness is the default rather than the exception.

I think the only difference is that brace layers have an axis with existing full masters, while virtual masters don’t require a full master (other than the default).

https://github.com/googlefonts/fontc/pull/403 fixes this for .designspace; an equivalent .glyphs could be used to test this.

We now support intermediate (brace) layers but still don't support "Virtual Masters", so let's keep this open.

looks like glyphsLib (and thus fontmake) don't support these Virtual Masters yet, https://github.com/googlefonts/glyphsLib/issues/859

So many layer types. It would be quite helpful to have a wee writeup, akin to https://github.com/googlefonts/fontc/blob/main/resources/text/units.md, on all these different types of layers, what htey are for, etc.

hm yeah.. but Virtual Master is not a layer type strictly speaking, but a font-wide custom parameter that allows to define some additional "masters" that unlike real ones do not require all the glyphs to have a corresponding "master" layer associated with them (which also entails a new whole kerning dict). I'm still trying to wrap my mind around them.

Fixed by #569


Adding of new Algo PollardRhoAlgorithm

Pollard’s Rho Algorithm for Prime Factorization #1269
🛠 Issue (#1269)
Issue no #1269
👨💻 Changes proposed
Implement the algorithm in all 4 language that is java,c++,C and Python
✔ Check List (Check all the applicable boxes)

[x] My code follows the code style of this project.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

What is Algorithm means
Pollard’s Rho is a prime factorization algorithm, particularly fast for a large composite number with small prime factors. The Rho algorithm’s most remarkable success was the factorization of eighth Fermat number
Here is a Algorithm of Pollard's Rho algorithm:
1.Choose a random number 'x' and a function 'f(x)' that generates a sequence of values.
2.Initialize two variables, 'y' and 'd', to 1.
3.Repeat the following steps until a factor of the number is found:
a. Update 'x' by applying the function: x = f(x).
b. Update 'y' by applying the function twice: y = f(f(y)).
c. Calculate the absolute difference between 'x' and 'y': d = abs(x - y).
d. Find the greatest common divisor (GCD) of 'd' and the number being factorized.
e. If the GCD is a non-trivial factor (not 1 or the number itself), return it as a prime factor.
f. If the GCD is 1, continue to the next iteration.
4.If no factor is found after a certain number of iterations, choose a different random 'x' and repeat the process.
📷 Screenshots

@Aarsh30 Add comments explaining your approach

yaa i had added in comment once have a look at it

@Arun9739  i had made changes can you review it and merge the pr it is pending from manys days

@Aarsh30 You have not yet added any comments.
The changes requested is to add some comments


Fix bugs in Blockwise and ShuffleLayer

Fixes two HighLevelGraph bugs and adds test coverage in test_distributed.py::test_fused_blockwise_dataframe_merge:

ShuffleLayer._cull_dependencies bug (resulting in #7178)
Blockwise serialization bug (list arguments were being coverted to tuples by msgpack, and then accidentally stringified)

[x] Closes #7178
[x] Tests added / passed
[x] Passes black dask / flake8 dask

Thanks for the review @jrbourbeau !  Hopefully your comments/suggestions are now addressed :)


Fix typo

Fixed typo. Great meeting you at the poster session - really cool work!

Thank you for fixing the typo! Nice meeting you at the conference as well.


Move MoorhenWrapper to its own repository

This should make the current source tree cleaner.

Currently semi-working version of the wrapper can be found here


Major reworking of validation

And try to get chromosomes working.
This is a pretty big reworking of some of the code, so it would be good to go over it. There are a lot of tests that will need to be added, I suspect. It should, however, allow us to efficiently get the lengths of chromosomes out of the tables (see #27 )

Summary by CodeRabbit

Refactor

Enhanced data integrity and validation processes in the genetic inheritance graph library.

Tests

Added and updated tests for improved validation checks and data structure handling.

I have tested and profiled the simulations, and this doesn't add to the time (and the skip_validate flag does trim the time slightly). Probably more tests of the flags are needed, but I'm going to merge this as I think it's the right way forward, and is largely tested now (a few more tests wouldn't go amiss, though, as the validation logic is complicated)


Initial stack implementation

Summary
This PR adds the initial implementation for the Jammy Full stack.
It is based on the existing Bionic Full stack, with the following changes:

updated stack ID, descriptions, etc
updated packages that are not supported in Jammy but that have alternative packages providing the same functionality (e.g. libtirpc1:amd64 becomes libtirpc-common:amd64)
updated package versions that are not supported in Jammy, but have alternative versions providing the same functionality (e.g. libffi6 becomes libffi8)
removed packages that are not supported in Jammy and have no equivalent in bionic (e.g. python-bzrlib)

Checklist

[x] I have viewed, signed, and submitted the Contributor License Agreement.
[x] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)
[x] I have added an integration test, if necessary.
[x] I have reviewed the styleguide for guidance on my code quality.
[x] I'm happy with the commit history on this PR (I have rebased/squashed as needed).

Looks like the integration tests were killed after almost 6 minutes. Re-running them to see if the same issue happens again, and if so we can explore why this is occurring.

Can you enumerate the full list of packages you removed?

Looks like the integration tests were killed after almost 6 minutes. Re-running them to see if the same issue happens again, and if so we can explore why this is occurring.

Looks like they've failed again, are you able to investigate? It works locally for me!

Looks like they've failed again, are you able to investigate? It works locally for me!

Yeah, it passes locally for me too. I wonder if github actions are killing the process because there's no output in some amount of time? I haven't seen this anywhere else

I updated the description above to include a full list of removed / updated packages.

@robdimsdale the package diffs make sense to me - I couldn't find anything for the packages that have been removed either. Thanks for adding that!

@robdimsdale I wonder if something like a job timeout increase might fix the GHA issue.

@robdimsdale I wonder if something like a job timeout increase might fix the GHA issue.

We can try that, but I'm skeptical for two reasons:

the default timeout is 360 minutes (i.e. 6 hours). This job fails after about 5 minutes
The build logs suggest it is getting killed with exit code 137. That's a little hard to pin down, but it seems like it's being killed because it exceeds some limit (memory, disk space, etc).

I was able to get the tests passing in CI by manually increasing the swap size to 10GB (from the default of 4GB). I updated this PR to increase the swap size automatically.

@paketo-buildpacks/stacks-maintainers the acceptance tests now pass in CI with the increased swap size. Should be good for another review 👀

@robdimsdale I'd love to not have to increase the swap space if possible, especially since it seems like we're using someone's personal GHA with no support guarantees. I'm down to merge this in for now to get Jammy releases out, and then revisit this issue later to figure out the underlying issue in the jam create-stack command!


Investigate flakiness with JobRunnerTest's

I believe the flakiness has something to do with the fact that JobRunner executes jobs using a separate thread for timeouts. Not quite sure what else it could be other than that.

Fixed by 7b57d2960f607793e6bc0c58451fb946635fb9d1.


Does not work with current release/version.

Fix it. It should be clean and stackable and have at least a minimum resurrecting capability (defined by gravestone that allows you to activate to resurrect pet).

https://www.bountysource.com/issues/47984163-does-not-work-with-current-release-version


Oracle: error when editing trigger/package body/procedure/function and trying to persist it after creation

Operating system (distribution) and version: Windows 10 version 20H2 Build 19042.804

DBeaver version: <IP_ADDRESS>102141704

Database name and version: Oracle database 12c

Driver name: Oracle Client 12 - <IP_ADDRESS>.0 - oracle.jdbc.OracleDriver

Do you use tunnels or proxies (SSH, SOCKS, etc)?  No

Description/Steps to reproduce the problem
First, I created a trigger by executing the corresponding code in the SQL editor as a SQL statement. (with the ";" character at the end). After creation, I needed to adjust the trigger, so I open the trigger by:

Navigating to the corresponding table of the trigger using the navigator.
I click on the triggers tab
I right click on the the just created trigger and choose "View trigger"
I perform an edit on the trigger
I press ctrl-s to save the trigger
In the next window, I click on "Persist" after which the compilation fails with the error:
PLS-00103: Encountered the symbol "end-of-file" when expecting one of the following:
;
The symbol ";" was substituted for "end-of-file" to continue.
Compile error at line 69, column 0

After you then edit the trigger again and also place the ";" ending character manually, the persist works fine. The next time you edit the trigger using the GUI again, the problem is gone. It seems the problem only occurs the first time you edit the trigger using the GUI after creation using a sql editor pane.
Not a big issue, but it would be nice if it was not needed to place the ";" character manually at the end of the trigger after first creation. Thanks!

In the meanwhile I also noticed this problem when creating a package.
When using the SQL editor:

When creating a package using CREATE OR REPLACE PACKAGE ..., the delimiter is set correctly in the declaration when viewing the properties.
When creating a package body using CREATE OR REPLACE PACKAGE BODY, the delimiter is not set automatically (same problem as the described issue above) and you have to add it manually at the end before saving (and persisting).

I just noticed that the same problem also exists after creation of a procedure or a function. The delimiter is also not set automatically after creation of the procedure/function using the SQL editor.
The problem does not occur anymore after:

manually putting the delimiter at the end of the procedure/function using the GUI
Persisting the changes

The problem also only exists after first creation of the procedure/function.

Thank you for the report.
STR:

Create a procedure via SQL Editor e.g. CREATE OR REPLACE PROCEDURE HR.add_job_history2 (  p_emp_id          job_history2.employee_id%type , p_start_date      job_history2.start_date%type , p_end_date        job_history2.end_date%type , p_job_id          job_history2.job_id%type , p_department_id   job_history2.department_id%type ) IS BEGIN INSERT INTO job_history2 (employee_id, start_date, end_date,                         job_id, department_id)  VALUES(p_emp_id, p_start_date, p_end_date, p_job_id, p_department_id); END add_job_history2

Refresh the connection, open the procedure, make any valid changed and press Save

Expected: the changes are saved.
Actual result: the changes are not applied and the  error appears "PLS-00113: END identifier 'ADD_JOB_HISTORY' must match 'ADD_JOB_HISTORY2' at line 1, column 14
Compile error at line 12, column 5"

@kseniiaguzeeva in you STR, you are missing a ; in the end of the first statement already, which the OP instead claims is present!

@p91paul That is correct, the ; is present at the end of the statement when running it. I just noticed that it seems to work when you place the delimiter twice (so ;;) at the end of for example a package body. However, it would still be great if it would work with the one delimiter, because that is also the way it works for the other objects. It would also be a little bit of an annoyance if you have to put an extra delimiter every time you run these object creation statements.

Fixed again.

Verified


Fix doc formatting written in unescaped file path

This fixes an issue where file paths were not escaped and were displayed as follows:

Before submitting the PR make sure the following are checked:

[x] The PR relates to only one subject with a clear title and description in grammatically correct, complete sentences.
[x] Wrote good commit messages.

[-] Commit message starts with [Fix #issue-number] (if the related issue exists).

[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.

[-] Added tests.
[-] Ran bundle exec rake default. It executes all tests and runs RuboCop on its own code.
[-] Added an entry (file) to the changelog folder named {change_type}_{change_description}.md if the new code introduces user-observable changes. See changelog entry format for details.

[x] The PR relates to only one subject with a clear title
and description in grammatically correct, complete sentences.

[-] If this is a new cop, consider making a corresponding update to the Rails Style Guide.

Same as https://github.com/rubocop/rubocop/pull/10444#issuecomment-1060300335.

Thank you for your time, it is much appreciated.
This PR is closed as it will be resolved by https://github.com/rubocop/rubocop/pull/10444


LittleFS with Winbond qspi-nor-flash w25q128 not working

Describe the bug
I am trying to get the LittleFS sample to run with an external qspi flash W25Q128FV on my nucleo_h753zi board. But i keep running into the same error (Superblock 0x0 has become unwritable).
I already tried setting writeoc to PP_1_1_4 (Since it seems that Winbond devices don't always support PP_1_4_4)
Also i tried to use the zephyr flash shell commands, with mixed results/success.
Do you have any ideas what could cause this?
To Reproduce
target: nucleo_h753zi board
qspi-flash: W25Q128FV
using the Zephyr LittleFS sample project i adjusted the following:
My devicetree looks like this:
/delete-node/ &storage_partition;

/ {
  aliases {
  };
};

&quadspi {
  pinctrl-0 = <&quadspi_clk_pf10 &quadspi_bk2_ncs_pc11
  &quadspi_bk2_io0_pe7 &quadspi_bk2_io1_pe8
  &quadspi_bk2_io2_pe9 &quadspi_bk2_io3_pe10>;

  pinctrl-names = "default";

  flash-id = <2>;
  status = "okay";

  w25q128: qspi-nor-flash@90000000 {
    compatible = "st,stm32-qspi-nor";
    label = "W25Q128";
    reg = <0x90000000 DT_SIZE_M(16)>; /* 128 Mbits */
    qspi-max-frequency = <50000000>;
    spi-bus-width = <4>;
    writeoc = "PP_1_1_4";
    status = "okay";

    partitions {
      compatible = "fixed-partitions";
      #address-cells = <1>;
      #size-cells = <1>;

      storage_partition: partition@0 {
      label = "storage";
      reg = <0 DT_SIZE_M(16)>;
      };
    };
  };
};

To my proj.conf i added this:
#Filesystem
CONFIG_FILE_SYSTEM=y
CONFIG_FILE_SYSTEM_LITTLEFS=y
CONFIG_FS_LITTLEFS_CACHE_SIZE=4096
CONFIG_FS_LITTLEFS_PROG_SIZE=256
CONFIG_FS_LITTLEFS_READ_SIZE=256
CONFIG_FS_LITTLEFS_LOOKAHEAD_SIZE=256
CONFIG_FS_LITTLEFS_FC_HEAP_SIZE=8192
CONFIG_NOCACHE_MEMORY=y

# Flash
CONFIG_FLASH=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_FLASH_MAP=y
CONFIG_MPU_ALLOW_FLASH_WRITE=y
CONFIG_FLASH_STM32_QSPI=y
CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=8192
CONFIG_FLASH_SHELL=y
CONFIG_FLASH_SHELL_TEST_COMMANDS=y

CONFIG_FLASH_LOG_LEVEL_DBG=y

The Error:
[00:00:00.000,000] <dbg> flash_stm32h7: stm32h7_flash_init: Flash initialized. BS: 32
[00:00:00.000,000] <dbg> flash_stm32h7: stm32h7_flash_init: Block 0: bs: 131072 count: 16
[00:00:00.000,000] <dbg> flash_stm32h7: flash_stm32h7_write_protection: Disable write protection
[00:00:00.000,000] <dbg> flash_stm32_qspi: qspi_send_cmd: Instruction 0x66
[00:00:00.000,000] <dbg> flash_stm32_qspi: qspi_send_cmd: CCR 0x166
[00:00:00.000,000] <dbg> flash_stm32_qspi: qspi_send_cmd: Instruction 0x99
[00:00:00.000,000] <dbg> flash_stm32_qspi: qspi_send_cmd: CCR 0x199
[00:00:00.000,000] <inf> flash_stm32_qspi: Reading SFDP
[00:00:00.000,000] <inf> flash_stm32_qspi: W25Q128: SFDP v 1.5 AP ff with 1 PH
[00:00:00.000,000] <inf> flash_stm32_qspi: PH0: ff00 rev 1.5: 16 DW @ 80
[00:00:00.001,000] <inf> flash_stm32_qspi: Quad read mode 7 instr [0xeb] supported
[00:00:00.001,000] <inf> flash_stm32_qspi: Quad read mode 7 instr [0xeb] will be used
[00:00:00.001,000] <inf> flash_stm32_qspi: QE requirement mode: 4
[00:00:00.001,000] <inf> flash_stm32_qspi: Quad mode enabled
[00:00:00.001,000] <dbg> flash_stm32_qspi: setup_pages_layout: layout page 256 not compatible with erase size 4096
[00:00:00.001,000] <dbg> flash_stm32_qspi: setup_pages_layout: erase size will be used as layout page size
[00:00:00.001,000] <dbg> flash_stm32_qspi: setup_pages_layout: layout 4096 x 4096 By pages
[00:00:00.001,000] <inf> flash_stm32_qspi: NOR quad-flash at 0x90000000 (0x1000000 bytes)
*** Booting Zephyr OS build v3.6.0 ***
Sample program to r/w files on littlefs
Area 4 at 0x0 on W25Q128 for 16777216 bytes
[00:00:00.003,000] <inf> littlefs: LittleFS version 2.8, disk version 2.1
[00:00:00.007,000] <inf> littlefs: FS at W25Q128:0x0 is 4096 0x1000-byte blocks with 512 cycle
[00:00:00.007,000] <inf> littlefs: sizes: rd 256 ; pr 256 ; ca 256 ; la 256
[00:00:00.008,000] <err> littlefs: WEST_TOPDIR/modules/fs/littlefs/lfs.c:1351: Corrupted dir pair at {0x0, 0x1}
[00:00:00.008,000] <wrn> littlefs: can't mount (LFS -84); formatting
[00:00:00.008,000] <dbg> flash_stm32_qspi: qspi_send_cmd: Instruction 0x6
[00:00:00.008,000] <dbg> flash_stm32_qspi: qspi_send_cmd: CCR 0x106
[00:00:00.008,000] <dbg> flash_stm32_qspi: qspi_send_cmd: Instruction 0x20
[00:00:00.008,000] <dbg> flash_stm32_qspi: qspi_send_cmd: CCR 0x2520
[00:00:00.061,000] <dbg> flash_stm32_qspi: qspi_send_cmd: Instruction 0x6
[00:00:00.061,000] <dbg> flash_stm32_qspi: qspi_send_cmd: CCR 0x106
[00:00:00.061,000] <dbg> flash_stm32_qspi: qspi_write_access: Instruction 0x32
[00:00:00.061,000] <dbg> flash_stm32_qspi: qspi_write_access: CCR 0x3002532
[00:00:00.061,000] <wrn> littlefs: WEST_TOPDIR/modules/fs/littlefs/lfs.c:2059: Superblock 0x0 has become unwritable
[00:00:00.061,000] <err> littlefs: format failed (LFS -28)
[00:00:00.061,000] <err> fs: fs mount error (-28)
FAIL: mount id 4 at /lfs: -28

Flash Shell
I also tried the shell flash commands to manually write the qspi flash (following the Zephyr Flash Shell sample).
Writing here also seems not to work as expected, although on a second try with less data it worked.
Attempt 1
uart:~$ flash erase W25Q128 0x0 0x1000
Erase success.
[00:24:43.153,000] <dbg> flash_stm32_qspi: qspi_send_cmd: Instruction 0x6
[00:24:43.153,000] <dbg> flash_stm32_qspi: qspi_send_cmd: CCR 0x106
[00:24:43.154,000] <dbg> flash_stm32_qspi: qspi_send_cmd: Instruction 0x20
[00:24:43.154,000] <dbg> flash_stm32_qspi: qspi_send_cmd: CCR 0x2520
uart:~$ flash write W25Q128 0x0 0x12345678 0x9abcdef0
Write OK.

Impact
I would have to change to another hardware if i cannot get the Winbond flash up and running with the littleFS.
Environment:

OS: Ubuntu (Docker Container)
Zephyr 3.6 with SDK 0.16.5

@MSeesko Can you try erasing the Flash chip first before running the LittleFS sample?

Used to work back in the day. You have to use writeoc = "PP_1_1_4";.
Could you validate that QE (SR2 1st bit) is set?

@MSeesko Can you try erasing the Flash chip first before running the LittleFS sample? FYI https://docs.zephyrproject.org/latest/samples/drivers/jesd216/README.html

Yes i tried it manually via the flash shell command
uart:~$ flash erase W25Q128 0 1000000
Erase success.

and also via the KConfig Option from the LittleFS sample CONFIG_APP_WIPE_STORAGE.
Still the same error.

Used to work back in the day. You have to use writeoc = "PP_1_1_4"; (Instruction 0x32) and you do.
Could you validate that QE (SR2 1st bit) is set?

How exactly would i do that? Is there a zephyr API to call a register read for the QSPI?
Also specifically i have the W25Q128JVSSIQ which should have the QE Bit set by default by the factory.

If this fails in the Flash shell to then it is not LittleFS problem. There might be a problem in QSPI driver.

CC @erwango

How exactly would i do that? Is there a zephyr API to call a register read for the QSPI?

It is possible to set a breakpoint in drivers/flash/flash_stm32_qspi.c.
Unfortunately, I don't have HW with QSPI at the moment. The CCRs seem correct:

Write - 0x3002532 = 4 data lines, 0 dummy, no alt-bytes, 3-byte address, 1 address line, 1 instruction line, 0x32 (PP_1_1_4)
Erase - 0x2520 = 3-byte address, 1 address line, 1 instruction line, 0x20 (SPI_NOR_CMD_SE)
Write enable - 0x106 =  1 instruction line, 0x06 (SPI_NOR_CMD_WREN)

You can modify the driver to read out all status registers:
+++ b/drivers/flash/flash_stm32_qspi.c
@@ -1508,6 +1508,15 @@ static int flash_stm32_qspi_init(const struct device *dev)
                dev_cfg->flash_size);
 #endif

+       uint8_t regs[3];
+       for (unsigned i = 0; i < sizeof(regs); ++i) {
+               if (qspi_read_status_register(dev, i + 1, &regs[i]) < 0) {
+                       LOG_ERR("Read SR%u failed", i);
+               } else {
+                       LOG_INF("SR%u %x", i + 1, regs[i]);
+               }
+       }
+
        return 0;
 }

Thanks @GeorgeCGV for your help!
I added your suggestion for the driver and got the following output:
[00:00:00.001,000] <dbg> flash_stm32_qspi: qspi_send_cmd: Instruction 0x66
[00:00:00.001,000] <dbg> flash_stm32_qspi: qspi_send_cmd: CCR 0x166
[00:00:00.001,000] <dbg> flash_stm32_qspi: qspi_send_cmd: Instruction 0x99
[00:00:00.001,000] <dbg> flash_stm32_qspi: qspi_send_cmd: CCR 0x199
[00:00:00.001,000] <inf> flash_stm32_qspi: Reading SFDP
[00:00:00.001,000] <inf> flash_stm32_qspi: W25Q128: SFDP v 1.5 AP ff with 1 PH
[00:00:00.001,000] <inf> flash_stm32_qspi: PH0: ff00 rev 1.5: 16 DW @ 80
[00:00:00.001,000] <inf> flash_stm32_qspi: Reading SFDP
[00:00:00.001,000] <inf> flash_stm32_qspi: W25Q128: 16 MiBy flash
[00:00:00.001,000] <dbg> flash_stm32_qspi: spi_nor_process_bfp: Erase 4096 with 20
[00:00:00.001,000] <dbg> flash_stm32_qspi: spi_nor_process_bfp: Erase 32768 with 52
[00:00:00.002,000] <dbg> flash_stm32_qspi: spi_nor_process_bfp: Erase 65536 with d8
[00:00:00.002,000] <dbg> flash_stm32_qspi: spi_nor_process_bfp: Page size 256 bytes
[00:00:00.002,000] <dbg> flash_stm32_qspi: spi_nor_process_bfp: Flash size 16777216 bytes
[00:00:00.002,000] <inf> flash_stm32_qspi: Quad read mode 4 instr [0x6b] supported
[00:00:00.002,000] <inf> flash_stm32_qspi: Quad read mode 7 instr [0xeb] supported
[00:00:00.002,000] <inf> flash_stm32_qspi: Quad read mode 7 instr [0xeb] will be used
[00:00:00.002,000] <inf> flash_stm32_qspi: QE requirement mode: 4
[00:00:00.002,000] <inf> flash_stm32_qspi: Quad mode enabled
[00:00:00.002,000] <dbg> flash_stm32_qspi: setup_pages_layout: layout page 256 not compatible with erase s6
[00:00:00.002,000] <dbg> flash_stm32_qspi: setup_pages_layout: erase size will be used as layout page size
[00:00:00.002,000] <dbg> flash_stm32_qspi: setup_pages_layout: layout 4096 x 4096 By pages
[00:00:00.002,000] <inf> flash_stm32_qspi: NOR quad-flash at 0x90000000 (0x1000000 bytes)
[00:00:00.002,000] <inf> flash_stm32_qspi: SR1 0
[00:00:00.002,000] <inf> flash_stm32_qspi: SR2 2
[00:00:00.002,000] <inf> flash_stm32_qspi: SR3 60
[00:00:00.002,000] <dbg> flash_stm32h7: stm32h7_flash_init: Flash initialized. BS: 32
[00:00:00.002,000] <dbg> flash_stm32h7: stm32h7_flash_init: Block 0: bs: 131072 count: 16
[00:00:00.002,000] <dbg> flash_stm32h7: flash_stm32h7_write_protection: Disable write protection
[00:00:00.002,000] <dbg> fs: fs_register: fs register 0: 0
*** Booting Zephyr OS build v3.6.0 ***
[00:00:00.002,000] <dbg> disk: disk_access_register: disk interface(SD) registered
[00:00:00.002,000] <inf> main:  --- Driver Sensor Control ---
[00:00:00.002,000] <inf> flashdisk: Initialize device SD
[00:00:00.002,000] <inf> flashdisk: offset 0, sector size 4096, page size 4096, volume size 1048576
[00:00:00.003,000] <inf> fat_fs_test: Block count 256
Sector size 4096
Memory Size(MB) 1
[00:00:00.003,000] <inf> flashdisk: Initialize device SD
[00:00:00.003,000] <inf> flashdisk: offset 0, sector size 4096, page size 4096, volume size 1048576
[00:00:00.005,000] <inf> flashdisk: Initialize device SD
[00:00:00.005,000] <inf> flashdisk: offset 0, sector size 4096, page size 4096, volume size 1048576
[00:00:00.009,000] <dbg> flash_stm32_qspi: qspi_send_cmd: Instruction 0x6
[00:00:00.009,000] <dbg> flash_stm32_qspi: qspi_send_cmd: CCR 0x106
[00:00:00.009,000] <dbg> flash_stm32_qspi: qspi_send_cmd: Instruction 0x20
[00:00:00.009,000] <dbg> flash_stm32_qspi: qspi_send_cmd: CCR 0x2520
... the following 4 log entries repeat about 100 times
[00:00:00.060,000] <dbg> flash_stm32_qspi: qspi_send_cmd: Instruction 0x6
[00:00:00.060,000] <dbg> flash_stm32_qspi: qspi_send_cmd: CCR 0x106
[00:00:00.060,000] <dbg> flash_stm32_qspi: qspi_write_access: Instruction 0x32
[00:00:00.060,000] <dbg> flash_stm32_qspi: qspi_write_access: CCR 0x3002532
... until it ends with:
[00:00:00.385,000] <dbg> flash_stm32_qspi: qspi_send_cmd: Instruction 0x6
[00:00:00.385,000] <dbg> flash_stm32_qspi: qspi_send_cmd: CCR 0x106
[00:00:00.385,000] <dbg> flash_stm32_qspi: qspi_write_access: Instruction 0x32
[00:00:00.385,000] <dbg> flash_stm32_qspi: qspi_write_access: CCR 0x3002532
[00:00:00.386,000] <inf> flashdisk: Initialize device SD
[00:00:00.386,000] <inf> flashdisk: offset 0, sector size 4096, page size 4096, volume size 2097152
[00:00:00.389,000] <err> fs: fs mount error (-19)

As you can see i tried switching from littlefs to fatfs, but still it seems to fail at the same write access instruction
[00:00:00.060,000] <dbg> flash_stm32_qspi: qspi_send_cmd: Instruction 0x6
[00:00:00.060,000] <dbg> flash_stm32_qspi: qspi_send_cmd: CCR 0x106
[00:00:00.060,000] <dbg> flash_stm32_qspi: qspi_write_access: Instruction 0x32
[00:00:00.060,000] <dbg> flash_stm32_qspi: qspi_write_access: CCR 0x3002532

The QE Bit from Statusregister 2 (SR2 1st Bit) seems to be set.
[00:00:00.002,000] <inf> flash_stm32_qspi: SR2 2
Also i ordered another QSPI-Flash mx25l25645g, which is also used in the littlefs sample. Using the exact same devicetree-configuration from that sample and using that QSPI-Flash i still get the same behavior, regardless of using writeoc PP_1_1_4 or PP_1_4_4 (it's either Instruction 0x32 or 0x38, but still fails after trying qspi_write_access).

I also had issues with a Winbond Flash W25Q512 and found a solution for it. Maybe your problem is related to this: #76432

I also had issues with a Winbond Flash W25Q512 and found a solution for it. Maybe your problem is related to this: #76432

Not sure. here the  qspi flash driver is writing to the flash.
However this might be a issue with the qspi_program_addr_4b  command sent to the flash

I have removed the File System label as it more Flash/qspi issue.

@MSeesko can you please check if this: https://github.com/zephyrproject-rtos/zephyr/issues/76432   can fix the pb with that Winbond flash ?

The same problem here (W25q128 QSPI + STM32L475) and this https://github.com/zephyrproject-rtos/zephyr/issues/76432 couldn't solve the problem:
Board Schematic: https://github.com/RT-Thread/IoT_Board/blob/master/docs/board/Pandora_STM32L4_Board_V2.4_SCH.pdf

&quadspi {
	pinctrl-0 = <&quadspi_clk_pe10 &quadspi_ncs_pe11
			&quadspi_bk1_io0_pe12 &quadspi_bk1_io1_pe13
			&quadspi_bk1_io2_pe14 &quadspi_bk1_io3_pe15>;
	pinctrl-names = "default";

	status = "okay";
	w25q128jv: qspi-nor-flash@90000000 {
			compatible = "st,stm32-qspi-nor";
			reg = <0x90000000 DT_SIZE_M(16)>; /* 128 Mbits */
			qspi-max-frequency = <80000000>;
			jedec-id = [ef 40 18];
			spi-bus-width = <4>;
			status = "okay";
	};
};

&w25q128jv {
	partitions {
		compatible = "fixed-partitions";
		#address-cells = <1>;
		#size-cells = <1>;

		external_partition: partition@0 {
			label = "storage";
			reg = <0x00000000 DT_SIZE_M(16)>;
		};
	};
};

The official example  https://github.com/zephyrproject-rtos/zephyr/tree/main/samples/drivers/spi_flash could erase the flash but could not write to the flash:
qspi-nor-flash@90000000 SPI flash testing
==========================

Perform test on single sector
Test 1: Flash erase
Flash erase succeeded!

Test 2: Flash write
Attempting to write 4 bytes
Data read does not match data written!!
000ff000 wrote 55 read ff MISMATCH
000ff001 wrote aa read ff MISMATCH
000ff002 wrote 66 read ff MISMATCH
000ff003 wrote 99 read ff MISMATCH

Perform test on multiple consecutive sectors
Test 1: Flash erase
Flash erase succeeded!

Test 2: Flash write
Attempting to write 4 bytes at offset 0xff000
Data read does not match data written!!
000ff000 wrote 55 read ff MISMATCH
000ff001 wrote aa read ff MISMATCH
000ff002 wrote 66 read ff MISMATCH
000ff003 wrote 99 read ff MISMATCH
Attempting to write 4 bytes at offset 0x100000
Data read does not match data written!!
00100000 wrote 55 read ff MISMATCH
00100001 wrote aa read ff MISMATCH
00100002 wrote 66 read ff MISMATCH
00100003 wrote 99 read ff MISMATCH

[00:00:00.000,000] <inf> flash_stm32_qspi: Reading SFDP
[00:00:00.000,000] <inf> flash_stm32_qspi: qspi-nor-flash@90000000: SFDP v 1.5 AP ff with 1 PH
[00:00:00.000,000] <inf> flash_stm32_qspi: PH0: ff00 rev 1.5: 16 DW @ 80
[00:00:00.000,000] <inf> flash_stm32_qspi: Reading SFDP
[00:00:00.000,000] <inf> flash_stm32_qspi: qspi-nor-flash@90000000: 16 MiBy flash
[00:00:00.000,000] <inf> flash_stm32_qspi: Quad read mode 4 instr [0x6b] supported
[00:00:00.000,000] <inf> flash_stm32_qspi: Quad read mode 7 instr [0xeb] supported
[00:00:00.000,000] <inf> flash_stm32_qspi: Quad read mode 7 instr [0xeb] will be used
[00:00:00.000,000] <inf> flash_stm32_qspi: QE requirement mode: 4
[00:00:00.000,000] <inf> flash_stm32_qspi: Quad mode enabled
[00:00:00.000,000] <inf> flash_stm32_qspi: NOR quad-flash at 0x90000000 (0x1000000 bytes)
*** Booting Zephyr OS build v3.7.0-3953-g1bbeeadfbf43 ***

The PR https://github.com/zephyrproject-rtos/zephyr/pull/78663  is for the flash_stm32_ospi driver  to  write the flash in 4Bytes mode if the flash says it supports this SPI_NOR_CMD_4BA
Then it could be interesting to trace if the write operation to the flash is achieved in  SPI_NOR_CMD_4BA command
This command is also valid for the QSPI driver.


NO-TICKET: avoid injecting unique ID into each fresh round of address gossiping

It appears from examining debug log files that expired addresses are continually circulated around the remaining peers.  This has an avalanche effect in terms of network traffic, latency and the size of nodes' event queues.
The change in this PR should help avoid old gossiped addresses cycling around the network continually, but relies on keeping the config value gossip.finished_entry_duration_secs a reasonably low value (we're using 60 seconds as of now), and certainly it must be a significantly lower value than gossip.gossip_interval (we'll be using 5 mins as of now).  If not, then nodes will likely be unable to fully connect, as their attempts to gossip their address will be thwarted by peers' filters timing out at varying moments and partially or totally stalling the progress of the gossip message across the network.
This change is anticipated to fix #691 and to fix #692 both which appear to be related to the node becoming overwhelmed with address gossiped messages.  It's unclear to me if it's also related to #693.

bors r+


Update serializers.py

Recently, I opened an issue explaining a bug that I found while getting "dojo_groups" which haven't set "auth_group" through API. For more information read the https://github.com/DefectDojo/django-DefectDojo/issues/7445.
I have edited the code and tested the new possible operation and it works as I expected.

Hello, a workflow has failed and I don't know why. In this case, integration test with mysql-rabbitmq. I think that the change I made is not involved in this workflows. Can you help me @coheigea ?

@adrianasantex Sometimes the integration tests are a bit flaky, if you rebase the PR it will run them again

Thank's @coheigea, now it works!


Restrict to Python 3.8

@hal0x2328 anything else you think I should add before I pull the trigger?


Failed to load Google Fonts and many alerts

GET https://fonts.googleapis.com/icon?family=Material+Icons net::ERR_TIMED_OUT
GET https://fonts.googleapis.com/css?family=Open+Sans:400|Montserrat:400,700 net::ERR_TIMED_OUT
My chrome displays these in the latest version and I cannot change the index.php to Chinese because after did this my index will become totally garbled.
Please do some check. I will change to the older version. Thanks.

I think maybe it is Google's problem? Cause when I change back to the older version, although it can display Chinese charactors normally but from the browser console it still displays the error.

Can confirm that is Google's problem not yours.
This issue will be closed now.


Project resource quota deletion not propagating to namespaces

Rancher Server Setup

Rancher version: 2.6 RKE1
Installation option (Docker install/Helm Chart): Docker Install

Information about the Cluster

Kubernetes version: Kubernetes version: 1.20.6
Cluster Type (Local/Downstream):

If downstream, what type of cluster? (Custom/Imported or specify provider for Hosted/Infrastructure Provider):

Describe the bug

The removal of a resource quota in a project does not correctly propagate to namespaces.
To Reproduce

Create New Project with a resource quota
Create a new namespace in the project
Verify that a new resourcequota was created for the new namespace k get resourcequota -A
Remove the resourcequotas from the project

Result
The created resourcequota for the namespace is still present with the initial quota/limits set
Expected Result

The created resourcequota has no quotas/limits set
Screenshots

Additional context

Reproduced on v2.7-head. Created a project with a resource quota, then a namespace in it. When I go to edit the namespace, the UI shows the resource quota tab. When I delete the resource quota from the project, it disappears from the UI and also from the namespace config view. However, it remains in the database, and is enforced in the namespace.

Apologies, I merged this today by accident, hadn't looked at this issue in a while, didn't realize this has not been tested by QA yet. Thought it was fully done too soon. While it does have integration tests, it still needs to be tested by QA. If there are issues found, I will make a new PR with changes.

QA Testing
Root cause
When a quota limit is removed from a project, the corresponding limits are not removed in the project's
namespaces.
What was fixed, or what changes have occurred
When Rancher computes the new resource quota, it needs to include only those resources on the quota that are defined with some limit value on the project. If the project quota does not define a limit for some resource, then ignore the resource quota requested. Keep the current behavior where the requested amount trumps the default one, if the default exists in the project.
Areas or cases that should be tested
What areas could experience regressions?
Steps

Create a project with a resource quota for CPU 200m and Memory 256Mi for Project Limit and CPU 100m and Memory 128Mi for Namespace Default Limit.
Create a namespace called ns1 in the project.
Inspect a resource quota that got added to the ns1 namespace with kubectl get quota -n ns1.
Ensure that the quota has the folowing limits (the second number is important): limits.cpu: 0/100m, limits.memory: 0/128Mi.
Go to the Project configuration -> Resource Quotas.
Remove just the CPU limit from the quota and save.
Fetch the quota in the ns1 namespace again and ensure it has the folowing limit: limits.memory: 0/128Mi. Just the memory limit remained, the CPU limit got deleted from the project's namespace.
Go back to the Project configuration and remove the memory limit.
Fetch the quota in the ns1 namespace again and get No resources found in ns1 namespace. This means there is no quota anymore, since there are no limits.

I have validated this issue on Docker and HA installed rancher.
Rancher Version : v2.7-f7692f30cf47d00073cc41b879e0ff90fb0a68b0-head

Test   Scenario
Test Cases
Check point
Status

Resource Quota Create (HA)
1. Install rancher server via helm v2.7-head     2. As an admin, create 1 rke1 node driver downstream cluster      3. From the rke1 and EKS clusters, created a project     4. Create a NameSpace
a.   Check for the resource quota limits ( On cluster run the command : kubectl   get resourcequota -A )     Make sure the limits are showing properly     b. Delete the NameSpace and check for the resource quota limits     Make sure the limits are not showing ( On cluster run the command : kubectl   get resourcequota -A  OR kubectl get   quota -n ns1)
Pass

Resource Quota Create (HA)
1. Install rancher server via helm v2.7-head     2. As an admin, create 1 rke1 node driver downstream cluster      3. From the rke1 and EKS clusters, created a project     4. Create a NameSpace      5. Update the resource quota limits in Project and in NameSpace
a.   Check for the resource quota limits ( On cluster run the command : kubectl   get resourcequota -A )     Make sure the limits are showing properly     b. Delete the NameSpace and check for the resource quota limits     Make sure the limits are not showing ( On cluster run the command : kubectl   get resourcequota -A  OR. kubectl get   quota -n ns1)
Pass

Resource Quota Create multiple name spaces(HA)
1. Install rancher server via helm v2.7-head     2. As an admin, create 1 rke1 node driver downstream cluster      3. From the rke1 and EKS clusters, created a project     4. Create multiple NameSpaces
a.   Check for the resource quota limits ( On cluster run the command : kubectl   get resourcequota -A )     Make sure the limits are showing properly on all the clusters     b. Delete the NameSpaces one by one and check for the resource quota limits     Make sure the limits are not showing ( On cluster run the command : kubectl   get resourcequota -A OR. kubectl get quota -n ns1)
Pass

Resource Quota Create (Docker)
1. Install rancher server via helm v2.7-head     2. As an admin, create 1 rke1 node driver downstream cluster      3. From the rke1 and EKS clusters, created a project     4. Create a NameSpace
a.   Check for the resource quota limits ( On cluster run the command : kubectl   get resourcequota -A )     Make sure the limits are showing properly     b. Delete the NameSpace and check for the resource quota limits     Make sure the limits are not showing ( On cluster run the command : kubectl   get resourcequota -A  OR kubectl get   quota -n ns1)
Pass

Resource Quota Create (Docker)
1. Install rancher server via helm v2.7-head     2. As an admin, create 1 rke1 node driver downstream cluster      3. From the rke1 and EKS clusters, created a project     4. Create a NameSpace      5. Update the resource quota limits in Project and in NameSpace
a.   Check for the resource quota limits ( On cluster run the command : kubectl   get resourcequota -A )     Make sure the limits are showing properly     b. Delete the NameSpace and check for the resource quota limits     Make sure the limits are not showing ( On cluster run the command : kubectl   get resourcequota -A  OR. kubectl get   quota -n ns1)
Pass

Resource Quota Create multiple name spaces(Docker)
1. Install rancher server via helm v2.7-head     2. As an admin, create 1 rke1 node driver downstream cluster      3. From the rke1 and EKS clusters, created a project     4. Create multiple NameSpaces
a.   Check for the resource quota limits ( On cluster run the command : kubectl   get resourcequota -A )     Make sure the limits are showing properly on all the clusters     b. Delete the NameSpaces one by one and check for the resource quota limits     Make sure the limits are not showing ( On cluster run the command : kubectl   get resourcequota -A OR. kubectl get quota -n ns1)
Pass


Deformed Shape

Hi;
Is it possible to plot the deformed shape through out the command: "plot_principal_nodal_stress"?
It plots properly the nodal equivalent stresses but the model stays undeformed (see attachment).
I was also wondering how can I work with the plot screenshot, in order to add titles, modify the scale, modify the brightness...etc
Thank you in advance!

I'm glad you're getting good usage out of the library.
I'm currently in the process of refactoring the code due to name change of a dependency (vtki --> pyvista).  Once that's done I'll add some additional parameters to the plotting functions.

Thank you so much for your prompt response. If you let me suggest parameters to modify in the plots, those are the following:
1.-Background color (white, black, grey...)
2.-Deformed shape of the structure
3.-Title of the plot
4.-Brightness and light source position modification
5.-Option to delete the block of text which by default appears in the bottom left hand corner
(cumulative index, loadstep, substep, time...)
I hope it can give you some ideas. And thank you again for your amazing job!

Just released pyansys==0.37.0 and it has improved plotting functions.  See the gallery examples below:
https://akaszynski.github.io/pyansys/examples/index.html

@akaszynski dead link above

Fixed the link and reposting it here:
https://mapdldocs.pyansys.com/examples/index.html


Add "remove" as alias for app image rm

follow-up to https://github.com/docker/app/pull/648

@rumpl @silvin-lubecki


Run refactorization

Summary
Close #1240
This PR continues the work of #1267. The logic is pretty similar the biggest difference is that here we remove services references because we are moving all the logic to the serializers, finally.
Details and comments

I removed exceptions too because they were not working as expected. I will continue this effort as part of #1264

I'm going to wait for your merges @akihikokuroda and wait a bit for @psschwei or @IceKhan13 if they can take a look

Looks like docker/kubernetes tests are both failing with the same error in the running_programs_using_decorators notebook:
quantum_serverless.exception.QuantumServerlessException:
| Message: Http bad request.
| Code: 400
| Details: {"config":["This field is required."]}

Does the decorators code need an update?

I found a problem with #1285 I'm fixing it now and put here the problem, @akihikokuroda

Tests passed so here is what I found, @akihikokuroda . With run end-point if we set config the data that we generate in the client is the next one:
Configuration(workers=None, min_workers=None, max_workers=None, auto_scaling=False, python_version='py39')

{'title': 'my-first-pattern', 'entrypoint': 'basic-pattern.py', 'arguments': '{"test": "here it goes an argument"}', 'dependencies': '[]', 'config': {'workers': None, 'min_workers': None, 'max_workers': None, 'auto_scaling': False, 'python_version': 'py39'}}

Until here everything is fine with the changes from #1285
The problem comes in the gateway:
<QueryDict: {'title': ['my-first-pattern'], 'entrypoint': ['basic-pattern.py'], 'arguments': ['{"test": "here it goes an argument"}'], 'dependencies': ['[]'], 'config': ['workers', 'min_workers', 'max_workers', 'auto_scaling', 'python_version'], 'artifact': [<InMemoryUploadedFile: artifact.tar ()>]}>

I don't know why but Django transforms config into:
'config': ['workers', 'min_workers', 'max_workers', 'auto_scaling', 'python_version']

What it generates the error. We didn't find this problem basically because we didn't have a test for run end-point with config and old logic was not taking into account config format. So I needed to restore the changes for run method from #1285 .
My question now is, with Content-type: multipart/form-data we cannot send JSON (I don't know if the problem is the content-type or the QueryDict). Should we send config as string in run_existing too to just not have different logic between these end-points, @akihikokuroda ?

I saw Django handled the Content-type and the config field without converting to the string.  You may merge this PR as-is.  I can try the #1285 later.

Ok, I'm going to prepare a test for this anyway before merge and put on-hold #1290 until we clarify this.

With run end-point if we set config the data that we generate in the client is the next one:
Configuration(workers=None, min_workers=None, max_workers=None, auto_scaling=False, python_version='py39')

{'title': 'my-first-pattern', 'entrypoint': 'basic-pattern.py', 'arguments': '{"test": "here it goes an argument"}', 'dependencies': '[]', 'config': {'workers': None, 'min_workers': None, 'max_workers': None, 'auto_scaling': False, 'python_version': 'py39'}}

Is that configuration part of our code somewhere? (wasn't clear if it was or just an example you made up) If so, do we need to update the python version to py310?

Is that configuration part of our code somewhere? (wasn't clear if it was or just an example you made up) If so, do we need to update the python version to py310?

Oh no, no. It's just some examples that I'm doing to test things locally hehe.


Superman not going through workflow on m1 Machine

I'm trying to implement Superman in our update policies. When I deploy the policy it doesn't do anything but shows complete in the logs.
Tue May 21 14:30:42 = Script - (macOS) Superman Updates[36653]: Status: Deleting saved credentials for the --auth-jamf-client option.
Tue May 21 14:30:42 = Script - (macOS) Superman Updates[36653]: Warning: Automatic macOS update/upgrade enforcement on Apple Silicon computers requires authentication credentials.
Tue May 21 14:30:42 Script - (macOS) Superman Updates[36653]: Status: User authentication is required to perform a macOS update/upgrade.
Tue May 21 14:30:42 = Script - (macOS) Superman Updates[36653]: Status: Found that Jamf is installing or is the parent process, restarting via super LaunchDaemon...
Tue May 21 14:30:42 = Script - (macOS) Superman Updates[36653]: **** S.U.P.E.R.M.A.N. 4.0.3 - CLEAN EXIT ****

This is the expected workflow when deployed via Jamf Pro... please take some time to explore the super wiki before posting issues.
Specifically for your question: https://github.com/Macjutsu/super/wiki/Troubleshooting#understanding-the-superlog


docs: fix import path for ix-video bundle

TSIA

@sherwinski merged in since it's such a minor change. Tagging for viz.


Fix slow test

Follows-up #8109

/test slow

cupy_tests/random_tests/test_generator_api.py .                          [ 10%]

The failing test has fixed.


CTFE store variables on stack, second attempt.

This redoes reverted pull 506. I found one issue: CTFE variables created in global scope weren't being destroyed (this happens, for example, when a global constant is created using a struct with a constructor).

Still fails the same way.

It isn't a dmc issue, as the exact same error happens on the linux version.

There will end up being 3 different possibilities: global, stack, or heap (for closures), so I don't want to use the sign. But, I've added asserts to make sure that it global vars never get stack ids allocated to them, and vice versa.


opponentinfo: make health bar more active

Made the health bar continue to display while the opponent attacks the player.  Useful for keeping track of enemy hp while recoiling.

This looks like it would break whenever you are fighting something in a multi combat area and then had something else attack you.

This looks like it would break whenever you are fighting something in a multi combat area and then had something else attack you.

My bad, I had tested it in multi but clearly not well enough.  Should correctly prioritise the NPC you're attacking now.

The behavior this has now seems very odd.  Also, if a monster attacking you changed targets to another player--for instance, a hellhound attacking you became aggressive to a player running through your area--it would start tracking that no-longer-attacking-you hellhound as the active opponent?
This change very much would benefit from some tests being added, given its complexity.

The behavior this has now seems very odd.  Also, if a monster attacking you changed targets to another player--for instance, a hellhound attacking you became aggressive to a player running through your area--it would start tracking that no-longer-attacking-you hellhound as the active opponent?
This change very much would benefit from some tests being added, given its complexity.

The behavior this has now seems very odd.  Also, if a monster attacking you changed targets to another player--for instance, a hellhound attacking you became aggressive to a player running through your area--it would start tracking that no-longer-attacking-you hellhound as the active opponent?
This change very much would benefit from some tests being added, given its complexity.

I added some tests to clarify behaviour in some ambiguous scenarios.
To address the specific scenarios you brought up, you were partially right that I goofed the second scenario.  It wouldn't start tracking the hellhound if it wasn't already active opponent, but if it was, it would stay the active opponent until that NPC died or some other NPC became active opponent.  I've fixed that now.
As for the first scenario you mentioned, I don't think it's as weird as you say.  If the player is attacking an NPC, then that NPC will always be the active opponent regardless of what happens in multi.  The only time the most recent attacker will become the active opponent is if the player is not attacking any NPC at all.  That might sound odd, but if you're, say, cannoning while idle, that NPC is probably the most recent enemy your cannon hit, in which case it's behaving similarly to the current version of the plugin.
Regardless, I hope the tests are to your liking and can help illustrate my explanation.  Cheers.

The behavior this has now seems very odd.  Also, if a monster attacking you changed targets to another player--for instance, a hellhound attacking you became aggressive to a player running through your area--it would start tracking that no-longer-attacking-you hellhound as the active opponent?
This change very much would benefit from some tests being added, given its complexity.

I added some tests to clarify behaviour in some ambiguous scenarios.
To address the specific scenarios you brought up, you were partially right that I goofed the second scenario.  It wouldn't start tracking the hellhound if it wasn't already active opponent, but if it was, it would stay the active opponent until that NPC died or some other NPC became active opponent.  I've fixed that now.
As for the first scenario you mentioned, I don't think it's as weird as you say.  If the player is attacking an NPC, then that NPC will always be the active opponent regardless of what happens in multi.  The only time the most recent attacker will become the active opponent is if the player is not attacking any NPC at all.  That might sound odd, but if you're, say, cannoning while idle, that NPC is probably the most recent enemy your cannon hit, in which case it's behaving similarly to the current version of the plugin.
Regardless, I hope the tests are to your liking and can help illustrate my explanation.  Cheers.

While reviewing this, I've thought of another case which causes this code to fail, and I've written a brief test case which demonstrates it:
@Test
public void singleCombatIssue()
{
    Player localPlayer = mock(Player.class), two = mock(Player.class), three = mock(Player.class);
    when(client.getLocalPlayer()).thenReturn(localPlayer);
    // TODO: find some unique var/condition representing a player being in multi-combat and mock it to be false here

    opponentInfoPlugin.onInteractingChanged(new InteractingChanged(localPlayer, two));
    assertSame(two, opponentInfoPlugin.getLastOpponent());

    // Another player tries to attack or cast magic on the player which causes interaction target changed from null -> player -> null in succession
    opponentInfoPlugin.onInteractingChanged(new InteractingChanged(three, null));
    assertSame(two, opponentInfoPlugin.getLastOpponent());

    opponentInfoPlugin.onInteractingChanged(new InteractingChanged(three, localPlayer));
    assertSame(two, opponentInfoPlugin.getLastOpponent()); // this line fails

    opponentInfoPlugin.onInteractingChanged(new InteractingChanged(three, null));
    assertSame(two, opponentInfoPlugin.getLastOpponent());
}

Keep in mind this is, as far as I'm aware, the only case that would be affected by the player being in a multi-combat zone (or not, in this case)

If I'm understanding correctly, the reason this appears to change the value of lastOpponent is because you didn't also mock the localPlayer.getInteracting() method to return 'two'.  It uses that to check if the player is still attacking a different actor when an actor attacks them, to prevent lastOpponent from taking on the value of the actor they aren't attacking.  I did a test for this from lines 73-85.

While reviewing this, I've thought of another case which causes this code to fail, and I've written a brief test case which demonstrates it:
@Test
public void singleCombatIssue()
{
    Player localPlayer = mock(Player.class), two = mock(Player.class), three = mock(Player.class);
    when(client.getLocalPlayer()).thenReturn(localPlayer);
    // TODO: find some unique var/condition representing a player being in multi-combat and mock it to be false here

    opponentInfoPlugin.onInteractingChanged(new InteractingChanged(localPlayer, two));
    assertSame(two, opponentInfoPlugin.getLastOpponent());

    // Another player tries to attack or cast magic on the player which causes interaction target changed from null -> player -> null in succession
    opponentInfoPlugin.onInteractingChanged(new InteractingChanged(three, null));
    assertSame(two, opponentInfoPlugin.getLastOpponent());

    opponentInfoPlugin.onInteractingChanged(new InteractingChanged(three, localPlayer));
    assertSame(two, opponentInfoPlugin.getLastOpponent()); // this line fails

    opponentInfoPlugin.onInteractingChanged(new InteractingChanged(three, null));
    assertSame(two, opponentInfoPlugin.getLastOpponent());
}

Keep in mind this is, as far as I'm aware, the only case that would be affected by the player being in a multi-combat zone (or not, in this case)

If I'm understanding correctly, the reason this appears to change the value of lastOpponent is because you didn't also mock the localPlayer.getInteracting() method to return 'two'.  It uses that to check if the player is still attacking a different actor when an actor attacks them, to prevent lastOpponent from taking on the value of the actor they aren't attacking.  I did a test for this from lines 73-85.

The tests do add full coverage of the changed code, though I'm still not 100% sure I understand this code inside and out. I'll review this more in depth at a later time.

I've added some further tests and cleaned up the test class a bit. @chaticon, does this still look good to you? From some quick ingame testing (in addition to the full test coverage provided by this PR's tests) it seems to still work just fine.

I've added some further tests and cleaned up the test class a bit. @chaticon, does this still look good to you? From some quick ingame testing (in addition to the full test coverage provided by this PR's tests) it seems to still work just fine.

Sorry for the late response.  Looks good to me.  And I think I may have commented on something outdated that you'd already fixed, sorry about that.

This throws an exception on login
java.lang.NullPointerException: null
	at net.runelite.client.plugins.opponentinfo.OpponentInfoPlugin.onInteractingChanged(OpponentInfoPlugin.java:161)
	at net.runelite.client.eventbus.EventBus$Subscriber.invoke(EventBus.java:67)
	at net.runelite.client.eventbus.EventBus.post(EventBus.java:238)
	at net.runelite.client.callback.Hooks.post(Hooks.java:169)

Additionally we should probably add comments to all of this logic since it looks complicated and I don't follow all of it.

We may want to keep an eye on this since there is no despawn event hooks for the opponent actor, and it is still invoking getInteracting() on it. But I think it is probably okay.


Added last scrape labels metric

Added metrics that stores the amount of labels in the last scrape.
This could be a possible solution for #11372. Would also be a useful metric overall.
PS: Suggestions for improving the metric name are open :)
Signed-off-by: kaffarell<EMAIL_ADDRESS>

ping @roidelapluie or @LeviHarrison

@prymitive is right (#11372). This metrics only counts the labels of the last metric that was scraped. Not all the labels from the last scrape. BUT this means that the label_limit config is wrong. As you can see in the code it limits the labels in a single metric, but the documentation says this:
# Per-scrape limit on number of labels that will be accepted for a sample. If
# more than this number of labels are present post metric-relabeling, the
# entire scrape will be treated as failed. 0 means no limit.
[ label_limit: <int> | default = 0 ]

source

I don't think there's anything wrong with label_limit, it does say that the limit applies for a sample.

You are right. But we should rephrase the documentation... The "Per-scrape" part is really misleading. On a different note, now this limit doesn't make any sense to me. Why would you want to limit the labels per metric. And why would you discard a whole scrape if one metric has more labels than the limit?

Per scrape is clear to me, it is "in a scrape"
Also, we fail the complete scrape because it is really difficult to deal with incomplete data. Prometheus never scrapes partially

Ok, understood.


can streaming disabled by default and an option to turn on?

❗ please do not add sensitive information in issues, you can provide extra information via email using issue number as reference ❗
Describe the issue
Streaming makes the debugging with ladybug impossible, i do not see what is happening, in most cases I do not want to use streaming as I do not have anything to stream to.
Reporter
jjansenvr

Done!


Enhance Logging for Workflow

In what area(s)?

/area runtime

/area operator

/area placement

/area docs

/area test-and-release

Describe the feature

Currently, it is rather difficult to diagnose an issue that occurs when a user is using dapr workflow, and due to dapr workflow being a new feature, issues occur more frequently than hoped. By adding in additional logging, diagnosing the issue will be a much easier process. I am proposing the following logs to be added to the runtime:

Creation of a Workflow
c.logger.Infof("Created workflow %s with instanceID '%s'", <workflowName>, <instanceID>)

Include the Instance ID for the workflow
Include the name of the workflow

Scheduling of an Activity
c.logger.Infof("Scheduling activity '%s' for workflow %s with instanceID %s", <activityName>, <workflowName>, <instanceID>)

Include the name of the activity
Include the name of the workflow
Include the instanceID for the workflow

Execution of an Activity
c.logger.Infof("Executing activity '%s' for workflow %s with instanceID %s", <activityName>, <workflowName>, <instanceID>)

Include the name of the activity
Include the name of the workflow
Include the instanceID for the workflow

Completion of an Activity
c.logger.Infof("Activity '%s' completed with status: '%s' for workflow %s with instanceID %s", <activityName>, <activityStatus>, <workflowName>, <instanceID>)

Include the name of the activity
Include the name of the workflow
Include the instanceID for the workflow
Include the status of the activity (pass/fail)

Completion of a Workflow
c.logger.Infof("Workflow %s with instanceID %s completed with status: '%s'", <workflowName>, <instanceID>, <workflowStatus>)

Include the Instance ID for the workflow
Include the name of the workflow
Include the status of the workflow (pass/fail)

In addition to the proposed logs for the runtime, I am also proposing that we add logs to the various SDKs which will provide information that the runtime does not have access to. These logs are as follows:

Registration of a Workflow
log("Workflow <workflowName> has been successfully registered")

Include the name of the workflow

Registration of an Activity
log("Activity <activityName> has been successfully registered to workflow <workflowName>")

Include the name of the activity
Include the parent workflow of the activity

Additional Thoughts

Timestamps could also be added to the beginning of the various log statements to help create a timeline for the workflow execution. One downside would be that the logs may get too cluttered.
The current proposal is to have the log statements as a level of "info". Again, to avoid cluttering the output, the log level could (and quite possibly should) be toned down to "debug"
If/when more logs are added to the various SDKs, we should first determine if they can be added to the runtime, and if so, add them there. If not, we must ensure that the logs across all SKDKs are of a consistent format.

Release Note

RELEASE NOTE:

👍

Timestamps are already included in all logs by default, so we're good there
How about we add an option in the Configuration CRD to determine the log level for workflow? We already have a section for logging in there. This way we can determine if the wf logs are debug or info

Tiny nit: consider being consistent in use of quotes around names/IDs.

Have we considered whether some of this information could be added to the log schema? For example, instead of making the workflow ID part of the log message, could it just be a first-class property on the log record itself (i.e., structured logging)?

Can I make a super minor suggestion that you tweak the message format for 4 and 5 (and the registration messages) so that they follow the same pattern as 1, 2 3.
i.e. Activity '%s' completed with status: '%s' for workflow %s with instanceID %s"
becomes
Activity completed...
Workflow %s with instanceID %s completed with status: '%s'", <workflowName>, <instanceID>, <workflowStatus>
becomes
Workflow completed...
This format helps when doing searches across logs as I can easily search for the phrase Workflow completed rather than searching for workflow and then filtering that list of results further by searching for completed (which still may give too many unwanted matches)

For logging the Completion of a Workflow details, if the status is Fail, can we also capture the source of failure ex: DAPR Client/ Workflow Client/ DTF?

Something that came up in interviews with DF customers is that being able to see the input and output of activity functions would be helpful. Not sure if this is something to be exposed on the logging side, but exposing this info would be useful for debugging.

This two comments:

https://github.com/dapr/dapr/issues/7065#issuecomment-1775891875
https://github.com/dapr/dapr/issues/7065#issuecomment-1781846724

are really valid points and have not been addressed on the PR https://github.com/dapr/dapr/pull/7222 taking the most out of structured logging is crucial IMO to make the workflow logging useful for debugging, once structured logging is put in place it can more easily enable log processing and allows for more easily querying for the inputs and outputs of activities
I can create a PR if you are happy with it

@famarting I'm supportive of this, and would be happy to provide feedback and/or guidance.

This two comments:

Enhance Logging for Workflow #7065 (comment)
Enhance Logging for Workflow #7065 (comment)

are really valid points and have not been addressed on the PR #7222 . Taking the most out of structured logging is crucial IMO to make the workflow logging useful for debugging, once structured logging is put in place it can more easily enable log processing and allows for more easily querying for the inputs and outputs of activities
I can create a PR if you are happy with it

@famarting @cgillum As per comment https://github.com/dapr/dapr/issues/7065#issuecomment-1775891875, I was adding Workflow ID as part of the logger field, similar to other fields like time, level etc. But that would require a separate logger Object for every workflow instance, unlike currently where there is a single global wfLogger object. Also for consistency across logs, I added the prefix Workflow Actor: <WorkflowID> as part of Message which would help querying all logs from a specific workflow instance. Is there any advantage of adding this as part of logger field? Or do we have other fields as well that we want to add other than WorkflowID? (The current format does require all workflow instance logs to follow the specific format, but helps us in having a single object).
For querying activity inputs/outputs, do we want to add these as well in logs? Won't these be user information and can potentially be quite large in size?
For the other comment, yeah it makes sense to add Workflow Completed and Activity completed as prefix which would help in searching logs. Creating a quick PR to add this.


run_parler_tts_training.py gives datasets.table.CastError error and failure

I've run through the steps to train a single voice, and it goes well until it comes time to actually Fine-tuning Parler-TTS step, i'm hitting a wall.  It seems the previous Dataset Annotation instructions don't create all of the expected values?
File "/Users/durin/AI/Projects/.parler-env/lib/python3.12/site-packages/datasets/table.py", line 2249, in cast_table_to_schema     raise CastError( datasets.table.CastError: Couldn't cast text: string utterance_pitch_mean: float utterance_pitch_std: float snr: double c50: double speaking_rate: string phonemes: string noise: string reverberation: string speech_monotony: string -- schema metadata -- huggingface: '{"info": {"features": {"text": {"dtype": "string", "_type":' + 502 to {'text': Value(dtype='string', id=None), 'utterance_pitch_mean': Value(dtype='float32', id=None), 'utterance_pitch_std': Value(dtype='float32', id=None), 'snr': Value(dtype='float64', id=None), 'c50': Value(dtype='float64', id=None), 'speaking_rate': Value(dtype='string', id=None), 'phonemes': Value(dtype='string', id=None), 'noise': Value(dtype='string', id=None), 'reverberation': Value(dtype='string', id=None), 'speech_monotony': Value(dtype='string', id=None), 'audio': Audio(sampling_rate=44100, mono=True, decode=True, id=None)} because column names don't match
Just to confirm. when I run the previous step to view a sample from the dataset, here's the full contents:
{'text': " Tonight at 11 on Utah's Talk Radio.",    'utterance_pitch_mean':<PHONE_NUMBER>4716797,    'utterance_pitch_std': 30.472904205322266,    'snr': 60.48295974731445,    'c50': 59.44480895996094,    'speaking_rate': 'quite slowly',    'phonemes': " tʌnaɪt æt  ɑn jutɔ'ɛs tɔk ɹeɪdioʊ . .",    'noise': 'slightly clear',    'reverberation': 'very confined sounding',    'speech_monotony': 'quite monotone',    'text_description': "'Very clear recording, but the speech is very monotone and slightly muffled by the recording.'"}
Not sure what I might be doing wrong, and I won't pretend to be an expert at this, so any guidance would be appreciated.

Hey @duringleaves, could you share the datasets you want to fine-tune the model on?
Are you using one single dataset or multiple ones ?


workout.ssc on GPU after 1 pass gives this error

Hello,
I am trying to run the workout.ssc on a fairly powerful GPU box and I get this error with the mnist8m dataset. It doesn't run for 4 passes. Does anyone know what is happening ?
<<<<<<< Testing parallel KMeans with FilesDS on MNIST8M (4 passes) >>>>>>>
pass= 0
First pass random centroid initialization
First pass random centroid initialization
3.00%, ll=0.00000, gf=56.074, secs=2.3, GB=0.76, MB/s=327.85, GPUmem=0.89, 0.97
6.00%, ll=0.00000, gf=75.164, secs=3.5, GB=1.40, MB/s=402.84, GPUmem=0.89, 0.89
10.00%, ll=0.00000, gf=78.380, secs=5.0, GB=2.29, MB/s=458.26, GPUmem=0.89, 0.89
13.00%, ll=0.00000, gf=86.569, secs=6.0, GB=2.92, MB/s=485.05, GPUmem=0.89, 0.89
16.00%, ll=0.00000, gf=89.345, secs=7.3, GB=3.68, MB/s=504.96, GPUmem=0.89, 0.89
19.00%, ll=0.00000, gf=92.924, secs=8.4, GB=4.32, MB/s=513.11, GPUmem=0.89, 0.89
22.00%, ll=0.00000, gf=96.501, secs=9.5, GB=4.95, MB/s=523.90, GPUmem=0.89, 0.89
25.00%, ll=0.00000, gf=86.828, secs=12.0, GB=5.59, MB/s=465.35, GPUmem=0.89, 0.89
28.00%, ll=0.00000, gf=87.005, secs=13.5, GB=6.35, MB/s=471.01, GPUmem=0.89, 0.89
31.00%, ll=0.00000, gf=89.554, secs=14.6, GB=6.99, MB/s=479.96, GPUmem=0.89, 0.89
35.00%, ll=0.00000, gf=89.912, secs=15.9, GB=7.88, MB/s=493.82, GPUmem=0.89, 0.89
38.00%, ll=0.00000, gf=91.801, secs=17.0, GB=8.51, MB/s=499.45, GPUmem=0.89, 0.89
41.00%, ll=0.00000, gf=93.452, secs=18.1, GB=9.15, MB/s=504.35, GPUmem=0.89, 0.89
44.00%, ll=0.00000, gf=94.954, secs=19.2, GB=9.78, MB/s=508.90, GPUmem=0.89, 0.89
47.00%, ll=0.00000, gf=95.810, secs=20.4, GB=10.54, MB/s=516.60, GPUmem=0.89, 0.89
50.00%, ll=0.00000, gf=97.302, secs=21.4, GB=11.18, MB/s=521.48, GPUmem=0.89, 0.89
54.00%, ll=0.00000, gf=97.276, secs=22.8, GB=12.07, MB/s=529.71, GPUmem=0.89, 0.89
57.00%, ll=0.00000, gf=98.250, secs=23.9, GB=12.70, MB/s=531.88, GPUmem=0.89, 0.89
60.00%, ll=0.00000, gf=99.389, secs=24.9, GB=13.34, MB/s=535.22, GPUmem=0.89, 0.89
63.00%, ll=0.00000, gf=100.017, secs=26.1, GB=14.10, MB/s=540.91, GPUmem=0.89, 0.89
66.00%, ll=0.00000, gf=100.714, secs=27.2, GB=14.74, MB/s=542.11, GPUmem=0.89, 0.89
69.00%, ll=0.00000, gf=101.414, secs=28.3, GB=15.37, MB/s=543.52, GPUmem=0.89, 0.89
73.00%, ll=0.00000, gf=96.872, secs=31.0, GB=16.26, MB/s=525.33, GPUmem=0.89, 0.89
75.00%, ll=0.00000, gf=96.154, secs=32.5, GB=16.77, MB/s=515.33, GPUmem=0.89, 0.89
79.00%, ll=0.00000, gf=96.128, secs=33.9, GB=17.66, MB/s=520.81, GPUmem=0.89, 0.89
82.00%, ll=0.00000, gf=96.917, secs=35.0, GB=18.29, MB/s=523.05, GPUmem=0.89, 0.89
85.00%, ll=0.00000, gf=97.522, secs=36.1, GB=18.93, MB/s=524.42, GPUmem=0.89, 0.89
88.00%, ll=0.00000, gf=97.819, secs=37.3, GB=19.56, MB/s=524.26, GPUmem=0.89, 0.89
91.00%, ll=0.00000, gf=98.264, secs=38.5, GB=20.33, MB/s=528.29, GPUmem=0.89, 0.89
94.00%, ll=0.00000, gf=98.952, secs=39.5, GB=20.96, MB/s=530.32, GPUmem=0.89, 0.89
98.00%, ll=0.00000, gf=99.033, secs=40.8, GB=21.85, MB/s=535.43, GPUmem=0.89, 0.89
100.00%, ll=0.00000, gf=100.526, secs=41.5, GB=22.23, MB/s=535.70, GPUmem=0.89, 0.89
java.lang.RuntimeException: mismatched types for operator ~
at BIDMat.FMat.$tilde(FMat.scala:1135)
at BIDMach.models.KMeans$$anonfun$mergeModelPassFn$1.apply(KMeans.scala:95)
at BIDMach.models.KMeans$$anonfun$mergeModelPassFn$1.apply(KMeans.scala:92)
at scala.collection.immutable.Range.foreach(Range.scala:160)
at BIDMach.models.KMeans.mergeModelPassFn(KMeans.scala:92)
at BIDMach.ParLearner$.syncmodelsPass(Learner.scala:796)
at BIDMach.ParLearner.retrain(Learner.scala:353)
at BIDMach.ParLearnerF.retrain(Learner.scala:712)
at BIDMach.ParLearnerF.train(Learner.scala:709)
... 112 elided
:31: error: not found: value a
val (mm8,opts8) = NMF.learnPar(a, 256)
^
:32: error: value batchSize is not a member of Any
opts8.batchSize = 5000
^
:33: error: value useGPU is not a member of Any
opts8.useGPU=useGPU
^
:34: error: value npasses is not a member of Any
opts8.npasses=2
^
:35: error: value nthreads is not a member of Any
opts8.nthreads=Mat.hasCUDA
^
:36: error: value train is not a member of Any
mm8.train
^
:39: error: not found: value a
val (mm8,opts8) = NMF.learner(a, 128)
^
:40: error: value batchSize is not a member of Any
opts8.batchSize = 5000
^
:41: error: value useGPU is not a member of Any
opts8.useGPU=useGPU
^
:42: error: value npasses is not a member of Any
opts8.npasses=2
^
:43: error: value train is not a member of Any
mm8.train
^

There was a bug in the type inference for multi-GPU KMeans. It should be fixed now.

Thank you for fixing this. Do the binary 1.0.3 versions that are provided for download have these fixes ? Do I need to compile the binaries myself ? In which case are there any documentation as starting point you would recommend ?

Follow the current installation instructions. Basically  git clone and either run "getdevlibs.sh" or use maven.


live <= main

Docs Build status updates of commit 9c25fd5:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

advocates/index.html.yml
:white_check_mark:Succeeded
View

advocates/toc.yml
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


How to use webpack module

I saw the webpack loaders config in the file react-server/packages/react-server-cli/src/compileClient.js, how to use it?

Hi @xaviertung!  You can configure your webpack loaders with the cli.  You're looking for the section "Webpack options" in https://react-server.io/docs/guides/react-server-cli


Add a performance profile for a Istio crypto configuration

Current Behavior

Desired Behavior

Implementation

Acceptance Tests

Mockups

Contributor Guide

@gyohuangxin I have created an issue for this. Please do add any more information that might be needed here.

@asubedy Thank you! If I have more processes, I will add them.


Error: You are using Elm 0.17.1, but this project is saying it needs a version in this range: 0.16.0 <= v < 0.17.0

I'm running  1 / 2 in elm-repl, but got this error:

Already have the latest Elm installed on my macOS. Any idea?

So I just remove all these elm-* files under /usr/local/bin, and reinstall elm, it works then.


[Feature] Add Dark Theme

Description
@pooranjoyb
we can add dark theme in this website
Screenshots
No response
Checklist

[X] I have checked the existing issues
[X] I have read the Contributing Guidelines
[X] I am willing to work on this issue (optional)

Congratulations, @dhruv8433! 🎉 Thank you for creating your issue. Your contribution is greatly appreciated and we look forward to working with you to resolve the issue. Keep up the great work!We will promptly review your changes and offer feedback. Keep up the excellent work! Kindly remember to check our contributing guidelines

Not planned


Allow d3.forceManyBody().distanceMax() and .distanceMin() to accept functions? (feature request)

d3.forceManyBody().strength() can accept a function as its parameter, allowing differentiation of strength based on some other node property.  Why not implement the same for .distanceMax() and .distanceMin()?
One use case is setting up a ring fence of fixed nodes in an irregular polygon to contain a bunch of nodes that are free to move in the simulation.  You'd want the ring fence nodes to have strong repulsive strength but only at a local distance, while the free nodes could interact with each other over longer distances.

Interesting. It's easy enough to plug in two arrays to store the values (as we do with strengths) in manyBody.js, and use them when evaluating the force; I'm happy to help you create the relevant force function (or patched version of d3-force) if you want to give it a try.
However, I'm not sure it would work in the particular use case you've described. Since the fence nodes are fixed, they are never, at the end of the iteration, subject to movement, so it doesn't matter what this force has computed for them. In fact, the only impact of fixed nodes in the calculation is to inform a "density field" (*), computed in a first stage. In the second (and final) stage, we evaluate for each "receiving" node the impact of the field. DistanceMax is used in this final stage, so it seems that only the distanceMax pertaining to free nodes would be used in a meaningful way.
What we might do in the case you're describing is to layer two forceManyBody. One would include only the free nodes, and would be parametrized with a long distanceMax; the other would include all nodes with a short distanceMax.
Note that there are other ways of fencing nodes inside a polygon: for one example see forceWalls, and the whole discussion at #163.
(*) In the Barnes–Hut algorithm, nodes interact with a "presence field", materialized in D3 as a quadtree that registers, for each quad, how much matter is contained in that quad. In other words, nodes don't interact with individual nodes, but with quads.

Ahh, thanks for the information.  Perhaps I was misinterpreting the docs about .distanceMax():

If distance is specified, sets the maximum distance between nodes over which *this* force is considered. If distance is not specified, returns the current maximum distance, which defaults to infinity. Specifying a finite maximum distance improves performance and produces a more localized layout.```

(emphasis mine) - I interpreted that as the force belonging to the node itself, and therefore other nodes would only feel it if they were close by.  E.g., even though the fixed node doesn't move, free nodes would only feel its force if they were within `.distanceMax()` of the fixed node.  I guess if the feature request I submitted were implemented, it might be the case?

I will check out the forceWalls discussion as an alternative.  Thanks!

Hi,
I came across this issue because I wanted to pass a function to distanceMax and it did not work.
Motivated by your words, @Fil, I tried to naively implement the same array as used for strength also for the distanceMax2 - but it's not working.
The code seems to be failing because I am trying to access the index of the quad in quad.data.index but the data is undefined.
here the relevant code snippet:
function apply(quad, x1, _, x2) {
    if (!quad.value) return true;

    var x = quad.x - node.x,
        y = quad.y - node.y,
        w = x2 - x1,
        l = x * x + y * y;

    // Apply the Barnes-Hut approximation if possible.
    // Limit forces for very close nodes; randomize direction if coincident.
    if (w * w / theta2 < l) {
      if (l < distanceMax2s[quad.data.index]) {
        if (x === 0) x = jiggle(random), l += x * x;
        if (y === 0) y = jiggle(random), l += y * y;
        if (l < distanceMin2) l = Math.sqrt(distanceMin2 * l);
        node.vx += x * quad.value * alpha / l;
        node.vy += y * quad.value * alpha / l;
      }
      return true;
    }

what can I do if there is no data of the quad ?

quad.data is only present for leaf nodes, not for those which have 4 subquads. Test node.length to distinguish the two cases https://github.com/d3/d3-quadtree/#nodes

thanks for your explanation!
Without quad.data I'm puzzled how I should replace stuff like
    if (l < distanceMax2) {
if distanceMax2 is a function and I don't have an accessor (index) for the corresponding distanceMax2-Array.
Should I simply ignore distanceMax2 altogether if the node is not a leaf node?

Oh so in that case, you'd need to dig into the quad to see if any point contained has a distanceMax2 that says it must be counted…? Then it seems it would totally defeat the purpose of the quadtree, which is to eliminate points without looking at them. See my point about density field (*) in the first comment.
I'm going to close this issue, since it's more a topic for (fun) experimentations, than an issue with the current code. Feel free to continue the discussion here, or to open a thread in https://github.com/d3/d3/discussions instead.


Minor Bug: Typing a message ?

I notice that sometimes when typing a message the character count +1 when you press the space bar, but the cursor does not move one space to the right. This is random and may happen in one message but not in another.

@Nigrini The cursor does not seem to move but the system do receive your empty space. It's something related to the formatting of text when displayed.

Yes it's a weird thing. And it makes it difficult to trace because it is so random. But I consider this as minor, because you can complete your function as planned.

@Nigrini No it's not a bug, simply type a text in a word processor and depending of the formatting, you'll get the same effect. I don't remember the exact word for this in English (but will surely write a message when I find it). It's caused by the spacing management between word. For example if you write a space on the end of a line and you have a word following it, then even if you add another space on your end of line, you won't really see it. It's a typographic adaptation, somewhat like the widow and orphan rules used when writing but used for spaces.
If you change with the style of your message through the use of CSS then you can disable it. Use the developer tools in Firefox / Chrome and directly edit the style of the form you are typing in. You will see what I mean.


I'm having trouble importing gtk-layer-shell-rs.

This is the Cargo.toml for reference.

Issue is solved. I had to use the git version.


Backport /n modifier from PHP 8.2 to all T-Regx versions

It should be possible to run Pattern::of($pattern, 'n'); even for PHP 7.1.

This will be incompatible with #140.


[MM-12052] Add account setting option to hide channel switcher button in the sidebar

Summary
Add account setting option to hide channel switcher button in the sidebar
Ticket Link
Jira: https://mattermost.atlassian.net/browse/MM-12052
Checklist

[x] Ran make check-style to check for style errors (required for all pull requests)
[x] Ran make test to ensure unit and component tests passed
[x] Added or updated unit tests (required for all new features)
[x] Has UI changes
[x] Includes text changes and localization file (.../i18n/en.json) updates

Setup Test Server label detected. Spinmint test server created if build succeeds (checks pass and no conflicts with base branch).

Spinmint test server created at: https://i-0395464a1954242e4.test.spinmint.com
Test Admin Account: Username: sysadmin | Password: sysadmin
Test User Account: Username: user-1 | Password: user-1
Instance ID: i-0395464a1954242e4

@jasonblais aha! I thought that. my bad will fix

Spinmint test server destroyed

@jasonblais done. PTAL. thanks for the review

Setup Test Server label detected. Spinmint test server created if build succeeds (checks pass and no conflicts with base branch).

Spinmint test server created at: https://i-0aecd25505abeea3a.test.spinmint.com
Test Admin Account: Username: sysadmin | Password: sysadmin
Test User Account: Username: user-1 | Password: user-1
Instance ID: i-0aecd25505abeea3a

Spinmint test server destroyed


Feature: unnecessary config should be removed

In tauri.config.js, the build config is unnecessary, as well as the imported modules. These should be stripped from the config on init.

We actually do use the build config for pointing at an external devserver / folder source for the internal devserver as well as pointing at the folder where tauri can find the built assets for build.

The config is unnecessary on the vue-cli-plugin, that's what he meant.

Yeah, that's why I opened it there originally. I'd like to be able to pass a config object to the init command that will be merged with the default, so I can do things like disabling the embedded server by default.

My bad. I'll move it back.


Hey friend! Your theme is great! What font is used for this theme?

Hey friend! Your theme is great! What font is used for this theme?

Thank you! The font used in the screenshots is Cartograph.
If you need further help or have any other questions, feel free to ask!


(How/is it possible) to shadow cara.tsx without shadowing everyother .tsx that needs to be imported?

I have set up a site using LekoArts' gastby-theme cara
I understand that what I create in the theme folder (src/@lekoarts/gatsby-theme-cara) will shadow the files seen in the theme's src folder. Which can be viewed here on github:
What I want to do is shadow just the "templates/cara.tsx" file, there I want to increase the size of the projects section to make room for the extra project-cards I have on my site.

The issue is that that .tsx file has many other components that need to be imported that I don't want to shadow.
How do I import these files that are on the theme's src directory (without having to shadow them all locally)?

Hi!
You can import them from the npm package, e.g.
import Layout from "@lekoarts/gatsby-theme-cara/components/layout"

Thanks so much, I tried something so close to that but couldn't get it right!