Performance improvements for large filesystems

Is your feature request related to a problem? Please describe.
We have been having some performance issues with "very large" filesystems > 1 million files.
Describe the solution you'd like
A lot of these performance issues seem to stem from iterating through lists that contain each software or relationship entry. If we use Dicts or Sets where applicable we will be reducing the cost of those lookup operations from O(n) to amortized O(1).
Describe alternatives you've considered
None
Additional context
Please see https://github.com/mcutshaw/Surfactant/tree/perf_improvements for some possible adjustments. With one filesystem we've tested with it reduced a run that took several hours to less then 5 minutes.

With the changes in the perf_improvements branch, does (de)serializing the dataclasses to/from JSON still result in the same output? The one thing to be careful of is changing the data classes in a way that results in a (CyTRICS) SBOM being output that doesn't pass validation with the (CyTRICS) schema.
If the java file changes make a difference, I'd merge a PR with that change now.
I'm not sure the early return for the find functions is valid -- if I'm understanding the logic, it looks like it will make the search exit as soon as the first non-matching entry is encountered, without considering that any subsequent entries in the various lists could be the one that actually matches; I think the only time the correct result would be returned in this scenario would be if the matching entry is the first one checked.

I've been mulling over decoupling the internal BOM data structures used by Surfactant from the CyTRICS schema -- the current set up definitely makes reading/writing CyTRICS SBOMs easy (json.[load|dump]), but makes other tasks really difficult.
Establishing relationships between files based on the filesystem structure would lend itself well to internally storing things as a graph representation, and should make most of the relationship establishing algorithms O(nlogn) instead of O(n^2) -- from a recently run, establishing relationships for a large number of files took most of the time. It would also make so we could just use graph edges to represent relationships between files, without the kind of awkward relationship lists.
I think this structure would still let us have a lookup table by hash for software entries. Or at least a set to tracks which hashes have been seen (md5, sha1, sha256), since ideally the common case is an entry not already existing with a matching hash.
Thoughts or comments?

You are correct. On a second look I misunderstood that we were directly serializing the dataclass to JSON. I do believe that the lookup table provided the greatest performance improvements of the changes, so I would prefer to discuss ways to implement that and potentially just tack on the java changes as well.
For a quick fix we could just remove that field from the dataclass_fields attribute like so:
INTERNAL_FIELDS = {"software_lookup_by_sha256"}

@dataclass_json
@dataclass
class SBOM:
    systems: List[System] = field(default_factory=list)
    hardware: List[Hardware] = field(default_factory=list)
    software: List[Software] = field(default_factory=list)
    relationships: Set[Relationship] = field(default_factory=set)
    analysisData: List[AnalysisData] = we could override to_json on the sbom class and just parse out the lookup table before trying to serialize it. field(default_factory=list)
  observations: List[Observation] = field(default_factory=list)
  starRelationships: Set[StarRelationship] = field(default_factory=set)
  software_lookup_by_sha256: Dict = field(default_factory=dict)

  def __post_init__(self):
      self.__dataclass_fields__ = {
          k: v for k, v in self.__dataclass_fields__.items()
          if k not in INTERNAL_FIELDS
      }

It doesn't appear that the adjustments to change relationships to Sets or adding the hash method effects the serialization.
Does this seem like something you would accept a push request for, or would you want to wait until the internal structure is reworked?

Also after looking at the "fail fast" logic I wrote previously, I think we should be alright to implement it if we just replace all of the returns I had with "continue" as once all_match is set to False there is no operation in that loop that will set it to True again. We could also cleanup the function as well as currently we evalutate if all_match is True and then just return True if it is:
So from:
  def has_relationship(
      self, xUUID: str = None, yUUID: str = None, relationship: str = None
  ) -> bool:
      for rel in self.relationships:
          all_match = True
          if xUUID and rel.xUUID != xUUID:
              all_match = False
          if yUUID and rel.yUUID != yUUID:
              all_match = False
          if relationship and rel.relationship.upper() != relationship.upper():
              all_match = False
          if all_match:
              return True
      return False

To:
def has_relationship(
    self, xUUID: str = None, yUUID: str = None, relationship: str = None
) -> bool:
    for rel in self.relationships:
        if xUUID and rel.xUUID != xUUID:
            continue
        if yUUID and rel.yUUID != yUUID:
            continue
        if relationship and rel.relationship.upper() != relationship.upper():
            continue
        return True
    return False

Maybe a brief comment saying return True will only be reached if none of the continue statements are hit? While not uncommon, I suspect that there are some who might not be as familiar with its behavior compared to the more common break keyword.
For the dataclass changes, I'd mostly want to make sure that serializing sets to/from JSON works for all the versions of Python we're supporting (3.8+) -- and same for __dataclass_fields__.

As far as testing against all supported python versions to verify I don't break the schema. The current tests run by Github action don't currently cover running a full generation then validating the schema, correct? Should I proceed with testing manually, or is there a better way?

I think manually is the best way for now -- there should be some CI tests that try generating a complete SBOM, but there isn't any validation against the CyTRICS schema itself (which technically hasn't been released).
The main thing to look for is all of the fields that were changed to a "set" able to loaded and saved correctly, at least on Python 3.8 and 3.12 would probably be enough for reasonable confidence that things also work on the versions in between. A small script like this could be used to test outside of running Surfactant:
from surfactant.sbomtypes import SBOM

with open("test_sbom.json") as infile:
    SBOM.from_json(infile.read())
    # print some fields in SBOM, make sure the loaded info looks right

# create a test sbom = SBOM() object with some fake data added to the various changed field types
print(sbom.to_json(indent=2))

I used this version and tested across 3.8, 3.10, and 3.12. Output files attached. Current branch is https://github.com/mcutshaw/Surfactant/tree/perf2
from surfactant.sbomtypes import SBOM
import platform
vers = platform.python_version()

with open("test_sbom.json") as infile:
    sbom = SBOM.from_json(infile.read())
    print(f"{len(sbom.relationships)} Relationships as sets: ")
    print(sbom.relationships)
    print()

    print("Full SBOM ")
    print(sbom)
    print()

    # print some fields in SBOM, make sure the loaded info looks right

print("SBOM back to JSON")
# create a test sbom = SBOM() object with some fake data added to the various changed field types
print(sbom.to_json(indent=2))
with open(f"test_output_sbom_{vers}.json", 'w') as f:
    f.write(sbom.to_json(indent=2))

3.8_test.log
3.10_test.log
3.12_test.log
test_output_sbom_3.8.19.json
test_output_sbom_3.10.12.json
test_output_sbom_3.12.2.json

Ah okay, that looks like it will be fine then (I guess dataclasses-json handles support for serializing set).
It looks like the java change is just stylistic -- functions called to the right of the in keyword are only called once.

going to close this one as we merged perf2.


fix: long press app download links on mweb Safari to open the Copy to Clipboard menu

Details
Allow the app download links and other external links to display the Copy to Clipboard menu when a user long-presses the item. Refactor MenuItem to get rid of nested Pressables.
Fixed Issues
$ https://github.com/Expensify/App/issues/16946
PROPOSAL: https://github.com/Expensify/App/issues/16946#issuecomment-1498083798
Tests

Log in to any account
Open the Settings page.
Long-press the Help link. This should present the Copy to Clipboard overlay with the action and link displayed at the bottom.
Use the Copy to Clipboard action and verify the copied URL.
Open the About page.
Long-press the View the code link. This should present the Copy to Clipboard overlay.
Repeat step 4.
Long-press the View open jobs link. This should present the Copy to Clipboard overlay.
Repeat step 4.
Click on App download links.
Long-press on each of the links. The Copy to Clipboard overlay should be displayed after long-pressing each link.
Verify that copied URL matches for each of the links.

[x] Verify that no errors appear in the JS console

Offline tests
Same as tests.
QA Steps
Same as tests.

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is correct English and approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome
https://user-images.githubusercontent.com/4160319/231301584-7ce5ff6b-eba5-4574-a57f-e4896fea3372.webm

Mobile Web - Safari
https://user-images.githubusercontent.com/4160319/231301787-993f5ef0-5422-44d3-8601-614f02754317.mov

Desktop

iOS
https://user-images.githubusercontent.com/4160319/231301833-7703b11b-d72e-4bab-be35-50e262ad22fd.mov

Android
https://user-images.githubusercontent.com/4160319/231301587-f9bf1d8a-d517-4477-8d79-e1fe4a03da4e.webm

@mountiny @eVoloshchak One of you needs to copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

Great job, and great comments from @eVoloshchak! I would really make sure we cover testing of all the platforms for where the MenuItem components is used since its quite often used component

Thanks for the feedback. I manually tested the component on both (Android / iOS) native mobile and mobile web platforms, as well as desktop and regular web browsers.

@akinwale, besides one small change, the code looks good to me.
There are a couple of pages left this logic needs to be added to, namely GetAssistancePage and WorkspaceTravelVBAView (I think that's all the pages, but I'd appreciate it if you double-check once more if I missed something)

@eVoloshchak I am unable to determine one of the links for WorkspaceTravelVBAView as it makes use of the Link.openOldDotLink method instead of an actual constant. However, it looks like they defer to a component called Section which in turn renders a MenuItemList where the MenuItems are rendered. I will add the logic to MenuItemList and then the missing links that I cannot determine for the MenuItems can be updated in a separate PR. Is this ok?

I think it's possible to determine the link. If we move buildOldDotURL into the upper scope (so it can be called using Link.buildOldDotURL), we can then add the following here:
link: Link.buildOldDotURL('domain_companycards')

@eVoloshchak I am unable to determine one of the links for WorkspaceTravelVBAView as it makes use of the Link.openOldDotLink method instead of an actual constant. However, it looks like they defer to a component called Section which in turn renders a MenuItemList where the MenuItems are rendered. I will add the logic to MenuItemList and then the missing links that I cannot determine for the MenuItems can be updated in a separate PR. Is this ok?

I think it's possible to determine the link. If we move buildOldDotURL into the upper scope (so it can be called using Link.buildOldDotURL), we can then add the following here:
link: Link.buildOldDotURL('domain_companycards')

I don't think this will work. The buildldDotURL takes a shortLivedAuthToken as a parameter, which is different from the url parameter that is passed to openOldDotLink. This seems like it would be a more complex implementation.

The buildldDotURL takes a shortLivedAuthToken

It's optional: If shortLivedAuthToken is not accessible, fallback to opening the link without the token, so all we need to do is move it to the upper scope

The buildldDotURL takes a shortLivedAuthToken

It's optional: If shortLivedAuthToken is not accessible, fallback to opening the link without the token, so all we need to do is move it to the upper scope

Also, I feel that it's beyond the scope of this PR.

I'm not too sure about that, the original requirement was: any menu item which is a link should have this behaviour

The buildOldDotURL method returns a promise. Therefore assigning it to link property directly as you demonstrated still wouldn't work.
It is out of scope because it's a refactor of business logic for building old URLs. The logic for the handling of links to display the Copy to Clipboard menu has already been implemented in the MenuItemList. The only thing left that's left is to set the link property for this particular link. If I can get the actual URL that this points to, then I can define a new constant and use that instead.

@eVoloshchak
Looking at the method, it looks like the resulting URL will be https://www.expensify.com/domain_companycards. Can you confirm if this is correct? We can just define a string constant in WorkspaceTravelVBAView and set it to this.

@eVoloshchak Looking at the method, it looks like the resulting URL will be https://www.expensify.com/domain_companycards. Can you confirm if this is correct? We can just define a string constant in WorkspaceTravelVBAView and set it to this.

It also contains current email, so we can't define it as a constant
https://www.expensify.com.dev/domain_companycards?email=someemail%2B01%40gmail.com
I'm investigating how we can get the result of buildOldDotURL, hold on

@akinwale, we can add the following here:
link: () => Link.buildOldDotURL('domain_companycards')

And then in MenuItemList
onSecondaryInteraction={e => onSecondaryInteraction(menuItemProps.link, e)}, where
    const onSecondaryInteraction = (link, e) => {
        if (typeof link === 'function') {
            link().then(url => ReportActionContextMenu.showContextMenu(CONTEXT_MENU_TYPES.LINK, e, url, popoverAnchor));
        } else if (!_.isEmpty(link)) {
            ReportActionContextMenu.showContextMenu(CONTEXT_MENU_TYPES.LINK, e, link, popoverAnchor);
        }
    };

@eVoloshchak All done! Ready for final review.

@akinwale, awesome!
Thanks for the patience and quick response time
There are a lot of changes and a lot of quirks on different platforms, I'll go through the checklist tomorrow as a first priority

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is correct English and approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

@akinwale, could you please also merge the latest main into this branch?

Sure, will do.

Looks good
We're missing a couple of screens still, namely: WorkspaceBillsFirstSection, WorkspaceBillsVBAView, WorkspaceCardVBAWithECardView, WorkspaceInvoicesFirstSection, WorkspaceInvoicesVBAView, WorkspaceReimburseSection, WorkspaceReimburseView, WorkspaceTravelVBAView, Enable2FAPrompt

Looks good We're missing a couple of screens still, namely: WorkspaceBillsFirstSection, WorkspaceBillsVBAView, WorkspaceCardVBAWithECardView, WorkspaceInvoicesFirstSection, WorkspaceInvoicesVBAView, WorkspaceReimburseSection, WorkspaceReimburseView, WorkspaceTravelVBAView, Enable2FAPrompt

All done.

This is not a bug. Long-pressing on any UI element (even for text selection) in Safari on an iPhone will result in vibration feedback. You can test on ND and staging.new.expensify.com to verify that this behaviour was not introduced by this PR.

That is true for Safari, but not for Android (both web and native app) and iOS app

I have tested on the following platforms without this PR and I am getting vibration feedback on long press of MenuItem.

Here are results of my tests:
Chrome Android: no vibration before the PR, vibration after the PR (notice the difference on the video below, the item stays highlighted on new.expensify, but gets blurred (after vibration) with the PR
https://user-images.githubusercontent.com/9059945/231834396-a3de61ce-d693-4bcd-a896-d9743f76a758.mp4
Android app: no vibration before the PR, vibration after the PR (the video below has sound enabled, so you can hear the vibration)
https://user-images.githubusercontent.com/9059945/231834682-3f0d17b0-74dc-4c39-bd91-2e7662b37b8c.mp4
iOS app: no vibration before the PR, vibration after the PR

In addition to these tests, I haven't done anything to change the behaviour of Pressable which is a React Native component, and there is no code where vibration is even being triggered. Hence, if a vibration bug exists, it was not introduced in this PR.

This PR replaces Pressable with PressableWithSecondaryInteraction.
PressableWithSecondaryInteraction has onLongPress handler, which is called no matter if we have onSecondaryInteraction or not, which is causing this problem.
For web we need to check if we have onSecondaryInteraction and if not, return early here, and disable onLongPress handler here.
For native we just need to disable a single  onLongPress handler here

I have tested on the following platforms without this PR and I am getting vibration feedback on long press of MenuItem.

Here are results of my tests: Chrome Android: no vibration before the PR, vibration after the PR (notice the difference on the video below, the item stays highlighted on new.expensify, but gets blurred (after vibration) with the PR
Android app: no vibration before the PR, vibration after the PR (the video below has sound enabled, so you can hear the vibration)
iOS app: no vibration before the PR, vibration after the PR

In addition to these tests, I haven't done anything to change the behaviour of Pressable which is a React Native component, and there is no code where vibration is even being triggered. Hence, if a vibration bug exists, it was not introduced in this PR.

This PR replaces Pressable with PressableWithSecondaryInteraction. PressableWithSecondaryInteraction has onLongPress handler, which is called no matter if we have onSecondaryInteraction or not, which is causing this problem. For web we need to check if we have onSecondaryInteraction and if not, return early here, and disable onLongPress handler here. For native we just need to disable a single onLongPress handler here

Thanks for the explanation. All the MenuItems actually have the onSecondaryInteraction prop set, so I'll need to refactor all of them to set this conditionally as well based on whether or not the link prop is present.

@eVoloshchak Updated.
I had to make use of isEmpty for the conditional checks because lint was complaining about unnecessary Boolean conversions. Since link is expected to be a String, this follows the rules at https://github.com/Expensify/App/blob/main/contributingGuides/STYLE.md#inline-ternaries.
I think this should be it.

The bug is still reproducible.
I think the issue is caused by defaultProps here, it should be undefined
Please proactively re-test all the platforms before pushing the final commit

The bug is still reproducible. I think the issue is caused by defaultProps here, it should be undefined
Please proactively re-test all the platforms before pushing the final commit

I still have to push a commit to test for Android, as I'm using two development machines. I will let you know once I have the final commit ready.

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is correct English and approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/mountiny in version: 1.3.1-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

🚀 Deployed to staging by https://github.com/mountiny in version: 1.3.1-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/mountiny in version: 1.3.1-3 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/mountiny in version: 1.3.2-5 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


CuboCore.packages: added

Motivation for this change
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[x] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 139510 at 6639789e run on x86_64-linux 1

22 packages built successfully:

CuboCore.coreaction
CuboCore.corearchiver
CuboCore.corefm
CuboCore.coregarage
CuboCore.corehunt
CuboCore.coreimage
CuboCore.coreinfo
CuboCore.corekeyboard
CuboCore.corepad
CuboCore.corepaint
CuboCore.corepdf
CuboCore.corepins
CuboCore.corerenamer
CuboCore.coreshot
CuboCore.corestats
CuboCore.corestuff
CuboCore.coreterminal
CuboCore.coretime
CuboCore.coretoppings
CuboCore.coreuniverse
CuboCore.libcprime
CuboCore.libcsys


Allow registrations even without chat relay

If you turn faction chat relaying off, it will stop polling the chat completely. Since user registrations needs a faction chat confirmation, stopping to poll the chat means that no more confirmation messages can be sent.

No more use in implementing this


[Do Not Merge] Add organisations eligible for world tagging

Add FCO, UKVI, DIT, DFID, DExEU to the list of organisations allowed to
tag documents in Whitehall admin.
Trello
Paired with @davidbasalla

Looks good (but I'm biased as I looked over @bevanloon's shoulder). Tested this in Integration and everything still works!


What is the connection between vertices, triangles and colors.

Hello, thanks for sharing this work. I'm new in the 3d field. Can someone explain in a few lines, or share some tutorials about those concepts: vertices, triangles and colors? What is the relation between them. As far I can see is that the triangles are load from a file, so they are statically elements. What do they represent? Thank you

Thank you for your generously sharing!

@gsssrao
Is it from first to third column means (x, y, z) coordinates in the Triangles array?

@GewelsJI No, it isn't the x, y, z coordinates. triangles array just stores the face/triangle ordering in the mesh. Each line in it just says which of the vertices form that triangle. Say if the line in the obj reads f 1 2 3, it just means that vertices 1, 2 and 3 together form that triangle/face. The x, y, z coordinates of say vertex 1 would usually be in the first line of the obj file in the following format v x y z. The vertex 2 in the second line and so on...
I would suggest reading up the way obj is written to file and this document which explains obj file structure.
Note: Please note the reverse ordering of faces while saving the obj here. The ordering defines the normals for those faces.

Thanks @gsssrao


Select code editor component and finalize UX

We are currently using Codemirror for our editor.
In the past we have used Monaco, which was great and maybe the best option for us, but had some layout/resizing issues that need to be solved.
Another great option is Ace editor.
A good example experience we should use as inspiration is the Rust Playground. The layout and resizing is seamless and I really like the way the output window appears after a request is sent.

[x] Evaluate and select a text editor component
[x] Integrate and implement necessary features

Json Schema Validation
Syntax highlighting
Light/Dark mode switching
etc.

[x] Layout

Show output only if there is something to show
Make output resizeable

Let's stick with codemirror6. It's a close race between that and monaco, but codemirror, esp v6, is way lighter weight and plays better with a modern js ecosystem. I messed around with fixing a few bugs to test out the dev experience and it's solid enough.
Core features seem pretty solid already, but I'm putting up a final PR soon to handle localStorage better so that the request box gets populated with the default request whenever the verb schema changes.
Layout was fixed in the PR above (#2472)


set content length to avoid transfer-encoding header not supported by s3

What I Did
Set content length explicitly to avoid "Transfer-Encoding" header, which is not supported by S3. See here
How I Did it
How to verify it
Description for the Changelog
Picture of a Boat (not required but encouraged)

Did you verify this fixes your issue locally?


Add rand_range

Proposed changes
Related to #88

Add rand_range which Samples from ∼U(α,β)
Add rand_range and range_laplace test

Types of changes

[ ] Bugfix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation Update (if none of the other choices apply)

Further comments
This also make math functions: rand_laplace and rand_range available by importing quinn

@SemyonSinchenko What do you think about this?


e2e tests failing with cookie banner check

E2e tests often fail on cookie banner test since we added it. It seems to be flaky because it can help if we restart the github action.
Btw that assertion that fails, I would also rewrite it to be expect(gaCookiesArr.length).toBe(2) (made it up a bit but I am sure something like this is doable), so we get more info when the test fails.

fixed in https://github.com/wasp-lang/open-saas/pull/176


Improved docs for RetryPolicy.NonRetryableErrorTypes

What was changed
Extended XML docs for RetryPolicy.NonRetryableErrorTypes property.
Why?
Wanted to make it obvious that values from NonRetryableErrorTypes property are matched against ApplicationFailureException.ErrorType property.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks! Will merge if/when CI passes.


Bracketed paste highlight

zsh has grown a feature whereby pasted text can be highlighted. These patches are required for the new upstream feature to work when zsh-syntax-highlighting has been loaded.

zsh has grown a feature whereby pasted text can be highlighted. These patches are required for the new upstream feature to work when zsh-syntax-highlighting has been loaded.

That's not quite right: it causes highlighting not to be updated until the first keypress after the paste. This can happen in two cases:

When $zle_highlight does specify paste highlighting, and RBUFFER is not empty (i.e., the paste is not at the end of the command-line), and the paste changes the parsing of $RBUFFER; for example, if the command-line is echo and a " is pasted before the first letter, the letters echo should become yellow (part of a double-quoted string).

When $zle_highlight does not specify highlighting for pastes, the pasted text itself should be highlighted. For example, pasting echo foo into an empty command-line should highlight the echo in green underline (without an additional keypress).

This $zle_highlight/$region_highlight interaction might be related to the disussion on issue #165.

The proposed fix of #165 at https://github.com/zsh-users/zsh-syntax-highlighting/issues/165#issuecomment-138689186
The fix at (https://github.com/danielshahaf/zsh-syntax-highlighting/commit/0e31d6e1a298adfce9596b7048f207c9940d495d) also fixes this issue, but it requires  (which will be included in zsh-5.1.1 and newer).

The patch for issue #165 (linked at that issue) also fixes this issue, if REGION_ACTIVE,CURSOR,MARK are replaced by YANK_ACTIVE,YANK_START,YANK_END respectively. The last three are zle parameters added to zsh in https://github.com/zsh-users/zsh/commit/c088906423ba22eef8d57c84b396f8d5be83ca45, which is due to be released as part of zsh-5.1.1.

Proposed patch: https://github.com/danielshahaf/zsh-syntax-highlighting/commit/e5c2a88db023600d7aeacbb2b7d640e0c8671a83
(The branch containing that patch also contains the aforementioned proposed fix to #165.)

Proposed patch: danielshahaf@e5c2a88

Merged in https://github.com/zsh-users/zsh-syntax-highlighting/commit/51102bf83fdfdca34f5be82eb3cac5518be64305.


Interpret results

So sorry for opening another issue, but I'd really like this to work! Ok, making progress now, downloaded img-3-hops.txt, tested it out on feature n01498406 (deleted everythin else), getting some results ['0.0', '0.0', '0.0', '0.9', '3.8'], but how to interpret them? How to get the corresponding labels printed or how can I look them up?
How can I give it a picture and get a result in text form? (say cat or dog for example) Sorry if this questions sounds redundant, I'd really like to test this out. I've read the code pretty thoroughly, but no luck so far :-|

Yes. Sorry for replying late.
The number in the list corresponds to topk accuracy in top_retrv.


the requested PHP extension mbstring is missing from your system

You should add a require for the **mbstring ** extension in the composer.json file
"require": {
....
"ext-mbstring": "*"
},

Sorry, The composer contains this requirement


Automatically save protease as a file specific setting when files are calibrated

When files are calibrated, a fail specific toml is automatically created to save the search tolerances. When this is created we should just automatically save the protease that was used for the calibration as well.

if the file was calibrated with a file-specific protease, the file-specific protease should get carried over to the new file-specific toml. can you test this functionality on 0.0.303 and the current master? if it's missing, it's a bug

Also add separation mode to file specific settings


web_url is not passed back to the github pipeline

It looks like you did this a yesterday, but the @v1 of your hook doesn't include it even though the git tag does. Did you move an existing tag?

That seems to be what one is forced to do with GitHub actions if you also want to use semantic versioning. Eg, if you have v1.0.0, fix a bug, and retag v1.0.1, then you have to remove the v1 which was equal to v1.0.0 and create a new v1 tag to v1.0.1. Does that answer your question?

In any case the web_url function isn't being passed to the next step yet, so this issue is not fixed in main either.

It isn't being printed in the current use either. I think tagging the current head as v2 will get that part working.

I found this documentation which says you need to declare the output, but I haven't had luck making it work.

Closing since now implemented.


[SPARK-41486][SQL][TESTS] Upgrade MySQL docker image to 8.0.31 to support ARM64 test

What changes were proposed in this pull request?
This PR aims to upgrade MySQL JDBC Integration Test docker image from 5.7.36 to 8.0.31 to support ARM64 testing.
Why are the changes needed?
Since MySQL 8.0, MySQL provides multi-arch images. After this upgrade, we can run MySQL JDBC Integration Test on ARM64 environment.

Does this PR introduce any user-facing change?
No, this is a test code PR.
How was this patch tested?

Pass GitHub Action Ci for AMD64 architecture
Manually run MySQLIntegrationSuites on ARM64 architecture.

I manually run this PR on Apple Silicon.
$ ./build/sbt -Pdocker-integration-tests "testOnly *MySQLIntegrationSuite"
[info] MySQLIntegrationSuite:
[info] - SPARK-33034: ALTER TABLE ... add new columns (4 seconds, 766 milliseconds)
[info] - SPARK-33034: ALTER TABLE ... drop column (389 milliseconds)
[info] - SPARK-33034: ALTER TABLE ... update column type (285 milliseconds)
[info] - SPARK-33034: ALTER TABLE ... rename column (206 milliseconds)
[info] - SPARK-33034: ALTER TABLE ... update column nullability (167 milliseconds)
[info] - CREATE TABLE with table comment (98 milliseconds)
[info] - CREATE TABLE with table property (113 milliseconds)
[info] - SPARK-36895: Test INDEX Using SQL (559 milliseconds)
[info] - SPARK-37038: Test TABLESAMPLE (0 milliseconds)
[info] - scan with aggregate push-down: VAR_POP without DISTINCT (3 seconds, 718 milliseconds)
[info] - scan with aggregate push-down: VAR_SAMP without DISTINCT (602 milliseconds)
[info] - scan with aggregate push-down: STDDEV_POP without DISTINCT (624 milliseconds)
[info] - scan with aggregate push-down: STDDEV_SAMP without DISTINCT (390 milliseconds)
[info] MySQLIntegrationSuite:
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for mllib / Test / testOnly
[info] - Basic test (432 milliseconds)
[info] - Numeric types (324 milliseconds)
[info] - Date types (871 milliseconds)
[info] - String types (302 milliseconds)
[info] - Basic write test (1 second, 341 milliseconds)
[info] - query JDBC option (334 milliseconds)
[info] Run completed in 1 minute, 15 seconds.
[info] Total number of tests run: 19
[info] Suites: completed 2, aborted 0
[info] Tests: succeeded 19, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.

$ ./build/sbt -Pdocker-integration-tests "testOnly *MySQLNamespaceSuite"
[info] MySQLNamespaceSuite:
[info] - listNamespaces: basic behavior (1 second, 367 milliseconds)
[info] - Drop namespace (231 milliseconds)
[info] - Create or remove comment of namespace unsupported (186 milliseconds)
[info] Run completed in 33 seconds, 456 milliseconds.
[info] Total number of tests run: 3
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 3, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.

Also update MySQLNamespaceSuite？

Oh. I missed that. Thank you, @wangyum . Let me add that.

I updated the PR, @wangyum .

cc @viirya

Thank you for review and approval, @wangyum !

All tests passed in the previsou commit and Docker integration tests passed at the last commit again. Merged to master for Apache Spark 3.4.


Training config for llama3

Hello, can you help to add training config for llama 3 models (8B and 70B)?
Currently, I only see training configs for llama 2 in folder https://github.com/SafeAILab/EAGLE/tree/main/eagle/train
Thanks a lot!

Actually, I found it on hf https://huggingface.co/yuhuili/EAGLE-LLaMA3-Instruct-70B/blob/main/config.json
I think we just need to copy them to this git repo.

I have already copied it over.


delly: 1.1.8 -> 1.2.6

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/dellytools/delly/releases.
meta.description for delly is: Structural variant caller for mapped DNA sequenced data
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.2.6 with grep in /nix/store/jnw9zsf3d97kyqpg3rcc64gy8a787zda-delly-1.2.6
found 1.2.6 in filename of file in /nix/store/jnw9zsf3d97kyqpg3rcc64gy8a787zda-delly-1.2.6

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
delly

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/jnw9zsf3d97kyqpg3rcc64gy8a787zda-delly-1.2.6 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A delly https://github.com/r-ryantm/nixpkgs/archive/eb84f26482f7dd6054233cf48ff6beb215686748.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/eb84f26482f7dd6054233cf48ff6beb215686748#delly

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/jnw9zsf3d97kyqpg3rcc64gy8a787zda-delly-1.2.6
ls -la /nix/store/jnw9zsf3d97kyqpg3rcc64gy8a787zda-delly-1.2.6/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

delly

Maintainer pings
cc @scalavision for testing.

I was only able to test the binaries that they run, but that seems okay.


Rich text editor

Clear and concise description of the problem
It would be awesome to have a rich text editor so we can style input/text area components
Suggested solution
Create a new component, for example n-editor that will support basic functionality of styling the text inside the input fieldthe
Alternative
Currently, I have written something from scratch, customized to my needs, using editable div. There is a lot of work to handle the keyboard navigation, since it's not plain text. I consider using a 3rd party solution like this: https://v4.primevue.org/editor/ but I love naive-ui and I prefer not to install another 3rd library... so keeping my own implementation for now
Additional context
I'm happy to contribute to this feature
Validations

[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

https://quilljs.com/docs/quickstart/  可以了解下，个人不建议ui集成富文本 会造成太臃肿


Replaced uses of 'UNBRACKET' by typedefs to fix compiling on clang

The 'UNBRACKET' macro (defined in PyGLM/compiler_setup.h) doesn't seem to work everywhere with clang 10.0.1 (on FreeBSD 12.2-RELEASE), so replaced by types.
Also added explicit cast to silence warning.

Hey there :)
Thank you very much for your contribution!


Vulkan: Rendering glitch: random squares in image (Linux, AMDGPU)

Godot version
4.0.dev.20211004.official.2e8cba0bd
System information
NixOS 21.11.git.37cc619178fM, AMD RX 480, amdgpu, Vulkan
Issue description
https://youtu.be/NJ2FDqf3T4E?t=61
Downstream issue: https://git.gieszer.link/unfa/liblast/issues/88
Steps to reproduce

Download project
run ./Liblast\ Linux

Minimal reproduction project
File bigger than 25MB.
https://github.com/davidak/test/releases/download/0.1.0/Liblast.<IP_ADDRESS>-pre-alpha.zip

Hiding all the reflection probes in the Map scene makes the decal block problem go away for me, might be a spot to keep digging.

I'm getting this same issue on M1 MacOS, 4.0.alpha.14. Noticed the issue on earlier builds too though.
https://user-images.githubusercontent.com/819616/187313106-acabc0e2-27b8-4f02-a926-4db2e7f9857c.mp4
Minimal demo attached:
black-square-jitter-minimal-demo.zip

Tried to bisect but builds are broken on MacOS before I can find a "good" commit.

The rendering bug on macOS M1 is a separate issue: https://github.com/godotengine/godot/issues/61108

Thanks @Calinou — totally missed that one


Test more platforms

Test 3 architecture minus "go modules mode" on Go 1.10 - modules do not exist on Go 1.10

@tandr thanks for your contribution. The travis file is much more readable now.


button for Twitter Web - Night Mode not visible with Firefox 55.0.3

https://addons.mozilla.org/addon/twitter-web-night-mode/

… at the click of a button.

– and after installing the extension I do have night mode but I don't see (in the toolbar of Firefox) a button to disable the mode.
[grahamperrin@momh167-gjp4-hpelitebook8570p-freebsd] ~% uname -v
FreeBSD 12.0-CURRENT #0 r320869: Mon Jul 10 13:57:55 UTC 2017     root@releng3.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC
[grahamperrin@momh167-gjp4-hpelitebook8570p-freebsd] ~% pkg info firefox | grep -i version
Version        : 55.0.3_1,1
[grahamperrin@momh167-gjp4-hpelitebook8570p-freebsd] ~%

Reloading the page did not workaround.
Workaround

Quit Firefox

– the button appears.
(In the location bar.)

Ah, okay. Good that it worked. If you're not aware, twitter has launched the official version of night mode for web. https://twitter.com/TwitterSupport/status/905477696360112129


Integrate with Database

As of now the code uses a file called data.csv to keep track of previous bookings. This however is not practical in a real life scenario.
Instead move the complete system into a database - server system in which the main code integrates with a database to perform CRUD operations. Conditions to be met for resolving the issue:

Transfer the entire storage system into a database system.
Make sure to track each User created as well the different bookings that were created. You could use a table for each different movie if you like , or have one common table for all movies. You will need a separate table however for recording every new user.
Once the Database system is created, you will have to integrate it onto the main code in such a way that the Enter Details function, display details function, book tickets, and generate bills all rely on this database for the information.

Partial points may be given if you were able to achieve some of them, but not all.

can I get this assigned?

please assign this issue to me

please assign this issue to me

!assign @AdvaySanketi

please give me some time, I am going for dinner

Make a PR and show your current progress

!deassign

!assign @Ruchavw

Hey you have 45 mins from now. Please make a PR within that time and show your progress. Doesn't matter if you have completed it or not. I would like to track your progress.

Please make sure that you do make a PR before your timer ends. Once your timer has ended i will assign it to the next user

!deassign

If anyone else wants to work on it do message

@g-anupam Assign me this issue
Thank you

!assign @AdvaySanketi

Have you already been assigned an issue. If yes you cant work on another.

pls assign

!assign @Ayush-jk

!extend 120

!assign @Ayush-jk 120

!deassign

could you assign it to me!!

!extend 30

!deassign

!assign @Ayush-jk

@Ayush-jk Please make a PR and show Progress within 45 mins

Hey can you assign this issue to me

!deassign @Ayush-jk

!assign @Ayush-jk 120

@g-anupam can I get this assigned?

!deassign

!assign @aniruddhajoshi100

!extend 45

!assign @aniruddhajoshi100 120

!deassign @aniruddhajoshi100

Issue closed


No spotify connect after first reboot

Compatible Device

[X] I'm running Raspotify on a compatible Device

Are you sure?

[X] I'm not trying to run Raspotify on a ARMv6 device

Compatible OS

[X] I'm running Raspotify on a compatible OS

Compatible Configuration

[X] I'm running Raspotify on a compatible Configuration

Latest Version

[X] I'm running the latest version of Raspotify

Are you sure?

[X] I'm not running Raspotify <IP_ADDRESS> on a Pi v1 or Pi Zero

Due Diligence

[X] I have done my due diligence

What happened?
instant after installation on Zero 2 W 32bit lite OS raspotify is working well, but only until first reboot.
After reboot Spotify Connect doesn't show up any more on all other devices in same wifi.
I assume issue with librespot due respotify service log
Relevant log output and/or the contents of /etc/raspotify/crash_report if any ( sudo journalctl -u raspotify -b and sudo cat /etc/raspotify/crash_report )
○ raspotify.service - Raspotify (Spotify Connect Client)
     Loaded: loaded (/lib/systemd/system/raspotify.service; enabled; preset: enabled)
     Active: inactive (dead) since Mon 2024-12-09 18:55:08 CET; 10s ago
   Duration: 288ms
       Docs: https://github.com/dtcooper/raspotify
             https://github.com/librespot-org/librespot
             https://github.com/dtcooper/raspotify/wiki
             https://github.com/librespot-org/librespot/wiki/Options
    Process: 1308 ExecStart=/usr/bin/librespot (code=exited, status=1/FAILURE)
   Main PID: 1308 (code=exited, status=1/FAILURE)
        CPU: 289ms

Dec 09 18:55:08 AudiPi systemd[1]: Started raspotify.service - Raspotify (Spotify Connect Client).
Dec 09 18:55:08 AudiPi librespot[1308]: [2024-12-09T17:55:08Z ERROR librespot] Invalid `--autoplay` / `-A`: ""
Dec 09 18:55:08 AudiPi librespot[1308]: Valid `--autoplay` / `-A` values: on, off
Dec 09 18:55:08 AudiPi systemd[1]: raspotify.service: Main process exited, code=exited, status=1/FAILURE
Dec 09 18:55:08 AudiPi systemd[1]: raspotify.service: Failed with result 'exit-code'.
Dec 09 18:55:08 AudiPi systemd[1]: raspotify.service: Triggering OnFailure= dependencies.

Strange that this setting marker is left empty by default with installation?!

These values were previously allowed to be empty, but broke with the upgrade to librespot 0.5.0. See commit b927945884b92900ba7079d2784d7a1998395e1d.
Re-installing or upgrading will not replace your configuration file, but prompt you whether you want to keep your existing one or install the package maintainer's version. Unfortunately this leads to breakage. Happy to hear the fix worked for you.


Update Dockerfile

I suggest pinning the version to 20.04 which is latest Ubuntu version. This will prevent unwanted versions popping up in latest tag which can break the build

we don't version lock things round these parts.


@Bean method KotlinLambdaToFunctionAutoConfiguration.kotlinToFunctionTransformerOld is non-static and returns an object assignable to Spring's BeanFactoryPostProcessor interface

With spring-cloud-function-context 3.1.4 I get:
o.s.c.a.ConfigurationClassEnhancer       : @Bean method KotlinLambdaToFunctionAutoConfiguration.kotlinToFunctionTransformerOld is non-static and returns an object assignable to Spring's BeanFactoryPostProcessor interface. This will result in a failure to process annotations such as @Autowired, @Resource and @PostConstruct within the method's declaring @Configuration class. Add the 'static' modifier to this method to avoid these container lifecycle issues; see @Bean javadoc for complete details.

Which seems scary.

Same here :
2021-11-06 13:41:23.914  INFO 40157 --- [           main] o.s.c.a.ConfigurationClassEnhancer       : @Bean method KotlinLambdaToFunctionAutoConfiguration.kotlinToFunctionTransformerOld is non-static and returns an object assignable to Spring's BeanFactoryPostProcessor interface. This will result in a failure to process annotations such as @Autowired, @Resource and @PostConstruct within the method's declaring @Configuration class. Add the 'static' modifier to this method to avoid these container lifecycle issues; see @Bean javadoc for complete details.

Thanks for fix. An idea about 3.1.x backport release date ?
Or which version should we downgrade to prevent this ?


[Snyk] Security upgrade python from 3.7 to 3.14.0a1

Snyk has created this PR to fix 2 vulnerabilities in the dockerfile dependencies of this project.
Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.
Snyk changed the following file(s):

components/local/base/Dockerfile

We recommend upgrading to python:3.14.0a1, as this image has only 178 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.
Vulnerabilities that will be fixed with an upgrade:

Issue
Score

Out-of-bounds Write SNYK-DEBIAN12-GLIBC-5927132
  829

Out-of-bounds Write SNYK-DEBIAN12-GLIBC-5927132
  829

Out-of-bounds Write SNYK-DEBIAN12-GLIBC-5927132
  829

Out-of-bounds Write SNYK-DEBIAN12-GLIBC-5927132
  829

CVE-2023-44487 SNYK-DEBIAN12-NGHTTP2-5953379
  829

[!IMPORTANT]

Check the changes in this PR to ensure they won't cause issues with your project.
Max score is 1000. Note that the real score may have changed since the PR was raised.
This PR was automatically created by Snyk using the credentials of a real user.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
📜 Customise PR templates
🛠 Adjust project settings
📚 Read about Snyk's upgrade logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

Commit Checker results:
**NOTE**: These are the results of the commit checker scans.
If these are not commits from upstream kfp, then please ensure
you adhere to the commit checker formatting

commitchecker verson unknown
Validating 1 commits between e27b68731776c35efe5e363295c3cc4a782cec4f...93a445afcff1fdf1e49e612bff309831fe2ab797

UPSTREAM commit 93a445a has invalid summary fix: components/local/base/Dockerfile to reduce vulnerabilities.

The Data Science Pipelines team recommends testing this using the Data Science Pipelines Operator.  Check here for more information on using the DSPO.
To use and deploy a DSP stack with these images (assuming the DSPO is deployed), first save the following YAML to a file named dspa.pr-113.yaml:
apiVersion: datasciencepipelinesapplications.opendatahub.io/v1alpha1
kind: DataSciencePipelinesApplication
metadata:
  name: pr-113
spec:
  dspVersion: v2
  apiServer:
    image: "quay.io/opendatahub/ds-pipelines-api-server:pr-113"
    argoDriverImage: "quay.io/opendatahub/ds-pipelines-driver:pr-113"
    argoLauncherImage: "quay.io/opendatahub/ds-pipelines-launcher:pr-113"
  persistenceAgent:
    image: "quay.io/opendatahub/ds-pipelines-persistenceagent:pr-113"
  scheduledWorkflow:
    image: "quay.io/opendatahub/ds-pipelines-scheduledworkflow:pr-113"
  mlmd:
    deploy: true  # Optional component
    grpc:
      image: "quay.io/opendatahub/mlmd-grpc-server:latest"
    envoy:
      image: "registry.redhat.io/openshift-service-mesh/proxyv2-rhel8:2.3.9-2"
  mlpipelineUI:
    deploy: true  # Optional component
    image: "quay.io/opendatahub/ds-pipelines-frontend:pr-113"
  objectStorage:
    minio:
      deploy: true
      image: 'quay.io/opendatahub/minio:RELEASE.2019-08-14T20-37-41Z-license-compliance'

Then run the following:
cd $(mktemp -d)
git clone git@github.com:opendatahub-io/data-science-pipelines.git
cd data-science-pipelines/
git fetch origin pull/113/head
git checkout -b pullrequest 93a445afcff1fdf1e49e612bff309831fe2ab797
oc apply -f dspa.pr-113.yaml

More instructions here on how to deploy and test a Data Science Pipelines Application.


Remove 80 usages of Component.Owner, remove obsolete Dirty calls

About the PR
Media

[x] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

Updated some method signatures to use the new Entity struct.

merge after conflicts done

Conflict hell but I pulled some of the changes out to another PR.


[WIP] typechain-target-web3-v2

Description

[x] Added support for target web3-v2
[x] Cleaned up unused types within web3-v1 in the generated ones

Fixes: #170

@krzkaczor - I've added target for web3-v2 and the types seems to be generated are fine. But I'm unable to get the integration tests running locally. Can you please help me out here and let me know what I'm missing? Thanks!

@eswarasai looks good! I will check it out changes in details over the weekend for sure, thanks!

@krzkaczor - I've reverted the changes made to web3-v1 target like requested above. Can you please let me know on how can I get the test cases working for web3-v2 target? Thanks!
cc: @quezak

@eswarasai frankly... I don't know. I spent 0.5h today trying to figure it out but I made no progress. What I learned:

test/integration/targets/web3-v2/web3.ts deployContract is stuck
Providing transactionConfirmationBlocks: 0 for web3 options doesn't fix the issue.

Idea:

try running standalone ganache and see what happens, maybe web3v2 has some problems with ganche provider injected directly?

@eswarasai okay got it. This morning I saw very informative debug output in console from yesterday 😆
This fixes this (but tests still fail):
# test/integration/targets/web3-v2/web3.ts:13
const web3 = new Web3(ganache.provider(), undefined, {
    transactionConfirmationBlocks: 1,
  });

Could you fix it and continue fixing tests on your branch? @eswarasai

@krzkaczor - Thanks for taking a look into it 👍
I'll spend some time figuring out in fixing the tests. In case I need any further help, I'll reach out here. Thanks 🙂

@eswarasai how did it go with the tests? I think we could merge this PR once tests are fixed.

Coverage increased (+0.9%) to 92.432% when pulling 78fec1609c884d93c827eac3dc58efe932e58d4f on eswarasai:feature/web3-v2 into b77f091c16309736582887c6bc9531a2d7d8db60 on ethereum-ts:master.

@krzkaczor @quezak - I've resolved the conflicts as well as fixed the tests. Let me know if this good to be merged. Thanks 🙂

Thanks, looks amazing! If you don't mind I will merge it over the weekend as I need to conduct some more tests etc.

You've got a green light from me, merge whenever you're ready

@krzkaczor - Sure, feel free to. Although I'm already using this in one of my current project and so far haven't seen any issues. But yeah, I think we can keep adding more tests to ensure everything is working fine and we're on top of the breaking changes with latest web3 updates.
PS: I'm really missing a community chat like Slack to have such discussions much more frequently 😅

Looks good. Merging to tmp branch to apply some minor changes.

Thanks so much! 🎉  It's merged but I can't release new package for now because of some NPM problems (spam detection was triggered). I will be able to release in few h.

@eswarasai speaking about real-time communication. I just created a poll on our twitter: https://twitter.com/ethereum_ts/status/1204164341542588417

Is it going to be https://www.npmjs.com/package/typechain-target-web3-v2 ? I guess you need to write to<EMAIL_ADDRESS>.

@PeterTheOne yeah I already did.
Okay it's released 🎉<EMAIL_ADDRESS>


ipv6 server side config

Hi,
Since I noticed your commit for the ipv6 client side change earlier today. Can you have it handle the server side ipv6 config as well, so we can run it once on a server and it pulls in the local ipv4/ipv6 addresses and will connect on either... as well as setting up a private ipv6 network?
Might also be worth adding a bit of randomization for the ipv6 subnet vs hard coding one, as this site does: https://simpledns.com/private-ipv6
Thanks!

https://angristan.xyz/how-to-setup-vpn-server-wireguard-nat-ipv6/
@l-n-s
This will take maybe 2 mins to change the script, i don't want to open a pull request and wait for like a year for you to approve it...
See how its done...
[Interface]
Address = <IP_ADDRESS>/24,fd42:42:42::1/64
PostUp = iptables -t nat -A POSTROUTING -o ens3 -j MASQUERADE; ip6tables -t nat -A POSTROUTING -o ens3 -j MASQUERADE
PostDown = iptables -t nat -D POSTROUTING -o ens3 -j MASQUERADE; ip6tables -t nat -D POSTROUTING -o ens3 -j MASQUERADE
ListenPort = 53
PrivateKey = <server private key>

[Peer]
PublicKey = <client 1 public key>
AllowedIPs = <IP_ADDRESS>/32, fd42:42:42::2/128

[Peer]
PublicKey = <client 2 public key>
AllowedIPs = <IP_ADDRESS>/32, fd42:42:42::3/128
Peer 2 (client 1)
[Interface]
PrivateKey = <client 1 private key>
Address = <IP_ADDRESS>/24,fd42:42:42::2/64
DNS = <IP_ADDRESS>,<IP_ADDRESS>

[Peer]
PublicKey = <server public key>
Endpoint = <server public IP>:53
AllowedIPs = <IP_ADDRESS>/0,::/0
Peer 3 (client 2)
[Interface]
PrivateKey = <client 1 private key>
Address = <IP_ADDRESS>/24,fd42:42:42::3/64
DNS = <IP_ADDRESS>,<IP_ADDRESS>

[Peer]
PublicKey = <server public key>
Endpoint = <server public IP>:53
AllowedIPs = <IP_ADDRESS>/0,::/0

See how its done...
Server
etc/wireguard/wg0.conf
[Interface]
Address = <IP_ADDRESS>/24, fc00:8:0::1/64
ListenPort = 0000
PrivateKey = <PRIVATE_KEY>
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE; ip6tables -A FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE; ip6tables -D FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
SaveConfig = false
# client
[Peer]
PublicKey = <PUBLIC_KEY>
AllowedIPs = <IP_ADDRESS>/32, fc00:8:0::3/128

Client
[Interface]
PrivateKey = <PRIVATE_KEY>
Address = <IP_ADDRESS>/24, fc00:8:0::3/128
DNS = <IP_ADDRESS>,<IP_ADDRESS>
[Peer]
PublicKey = <PUBLIC_KEY>
AllowedIPs = <IP_ADDRESS>/0, ::/0
Endpoint = <IP_ADDRESS>:0000
PersistentKeepalive = 25

@l-n-s
This config will work we need to change this and add these to he script buddy...

See how its done...
Server
etc/wireguard/wg0.conf
[Interface]
Address = <IP_ADDRESS>/24, fc00:8:0::1/64
ListenPort = 0000
PrivateKey = <PRIVATE_KEY>
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE; ip6tables -A FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE; ip6tables -D FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
SaveConfig = false
# client
[Peer]
PublicKey = <PUBLIC_KEY>
AllowedIPs = <IP_ADDRESS>/32, fc00:8:0::3/64

Client
[Interface]
PrivateKey = <PRIVATE_KEY>
Address = <IP_ADDRESS>/24, fc00:8:0::3/64
DNS = <IP_ADDRESS>,<IP_ADDRESS>
[Peer]
PublicKey = <PUBLIC_KEY>
AllowedIPs = <IP_ADDRESS>/0, ::/0
Endpoint = <IP_ADDRESS>:0000
PersistentKeepalive = 25

@l-n-s
This config will work we need to change this and add these to the script buddy...

https://www.linode.com/stackscripts/view/348742

#!/bin/bash

# <UDF name="port" Label="Port" example="Wireguard Port" default="51820" />
# <UDF name="privateip" Label="Private IP and subnet for your Wireguard Server" example="<IP_ADDRESS>/24, <IP_ADDRESS>/24, <IP_ADDRESS>/24 etc" Default="<IP_ADDRESS>/24" />
# <UDF name="peerpubkey" Label="Peer Pubic Wireguard Key from your client" Default="" />
# <UDF name="pubkey" Label="Your SSH public key" default="" />
# <UDF name="privateip_client" Label="Private IP and subnet for your Wireguard Client" example="<IP_ADDRESS>/24, <IP_ADDRESS>/24, <IP_ADDRESS>/24 etc" Default="<IP_ADDRESS>/24" />

source <ssinclude StackScriptID="349139">

exec 1> >(tee -a "/var/log/stackscript.log") 2>&1

### Set hostname, Apt configuration and update/upgrade

set_hostname

echo "deb http://deb.debian.org/debian/ unstable main" > \
/etc/apt/sources.list.d/unstable-wireguard.list

printf 'Package: *\nPin: release a=unstable\nPin-Priority: 150\n' > \
/etc/apt/preferences.d/limit-unstable

apt_setup_update
if [[ "$PUBKEY" != "" ]]; then
  add_pubkey
fi

if [[ "$PORT" != "51820" ]]; then
  PORT="$PORT"
fi

apt-get install wireguard -y

# Wireguard

wg genkey | tee ~/wg-private.key | wg pubkey > ~/wg-public.key

PRIVATEKEY=`cat wg-private.key`

cat <<END >/etc/wireguard/wg0.conf
[Interface]
PrivateKey = $PRIVATEKEY
Address = $PRIVATEIP, fd86:ea04:1115::1/64
ListenPort = $PORT
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; \
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE; \
ip6tables -A FORWARD -i wg0 -j ACCEPT; \
ip6tables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; \
iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE; \
ip6tables -D FORWARD -i wg0 -j ACCEPT; \
ip6tables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
SaveConfig = true
[Peer]
PublicKey = $PEERPUBKEY
AllowedIPs = $PRIVATEIP_CLIENT, fd86:ea04:1115::0/64
END

### Services

wg-quick up wg0
systemctl enable wg-quick@wg0
wg show
ufw_install
ufw allow "$PORT"/udp
ufw enable

stackscript_cleanup

https://github.com/LiveChief/wireguard-install/tree/IPV6
@l-n-s


Menu Text Display Problem

Text in the menu is not displayed correctly in Win7 (bottom line of text is clipped). This problem has persisted since version 126. Before version 124, display is fine on both Win7 and XP. (Compare left: incorrect display in Win7 and right: correct display in XP, both running 126 R6 x86 version)

The hieroglyphs are difficult to deal with. In my opinion, there are differences in the fonts and perhaps in the font smoothing.

The hieroglyphs are difficult to deal with.

I don't think so, given that this is a rare problem to be seen in any software. Moreover the previous versions (<=124) of Supermium already did it correctly.


using statements are not supported

Hi!
Trying to optimise my program with the extension throws an error, because it doesn't appear to recognise the "using" keyword. I've attached a screenshot below.
In the prettier plugin, using statements do appear to be recognised (v1.0.10) but they don't seem to work with the optimiser extension. I suspect that the issue could also be because my using statement is inside the main class, not outside?
Cheers,
Sudarshan

In the prettier plugin, using statements do appear to be recognised (v1.0.10)

That's a very old version of the plugin. But as far as I can tell it doesn't accept using/import inside a class either - I get errors when I try to format such code using prettier.
But the fix to the plugin was trivial. I've just published a fix (v1.0.52). Hopefully the fixing the extension will be similarly straightforward.

Fixed by f7a3d278ecd0f60ce7b0d993788425d298f87c21, and released as v2.0.71

Legend, all working now! The Monkey C language documentation really is a funny thing.


Enable passing a list to visible argument in add_image

Description
as discussed in https://github.com/AllenCellModeling/napari-aicsimageio/issues/2, @JacksonMaxfield would benefit from being able to pass a list of booleans to the visibleparameter in add_image for multichannel.  This PR adds that functionality, along with a test.
Type of change

[x] New feature (non-breaking change which adds functionality)

How has this been tested?

[x] the test suite for my feature covers adding a 5 channel image with some visible and some not.
[x] all tests pass with my change

Final checklist:

[x] My PR is the minimum possible work for the desired functionality
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] I have added tests that prove my fix is effective or that my feature works

:eyes: Huge PR thank you so much :tada:

i suppose they could all conceivably accept lists, shall we do that here?  I also just noticed that a couple arguments (iso_threshold, attenuation) aren't being passed in multichannel...

I also just noticed that a couple arguments (iso_threshold, attenuation) aren't being passed in multichannel.

That seems like an oversight, would be good to fix.

i suppose they could all conceivably accept lists, shall we do that here?

I guess might as well, could have some nasty guessing then with lists of lists etc, but we already do some of that for contrast_limits.
Thoughts @jni?

Yeah the list of lists struggle is real. I'm almost thinking we should just break this into add_images or similar, but that would also be messy. I guess it all depends on the implementation. The Python gist on this is "in the face of ambiguity, refuse the temptation to guess", so the question is, can we avoid ambiguity? Can all arguments be unambiguously determined to be either a list to pass to the channels, or a single value to be broadcast across the channels? If so, then let's go for it, if not, then in the ones where broadcasting is not unambiguous, then we should not support a list.

ok, I made the whole thing accept single values or a sequence.  It got tremendously verbose, so I ended up refactoring quite a bit, and I do like how it reads now... but this is definitely a "proposal" more than anything, so please critique.
regarding lists of lists, the main challenges were clims, scale, and translate, which are handled by the new ensure_iterable_sequences.  room for improvement... but let me know if you like the general pattern.

looks like the failing test was in test_signature where we were checking that the add_image method literally had code that said layer = layers.Image(<all of the arguments typed out>).  Since that test is definitely incompatible with this approach, (and since it would make for a terribly ugly method otherwise), I modified the signature test for add_image. Would be great if @kne42 has a moment to take a look at that.

just updated the multichannel tests to use mark.parametrize to make sure we're hitting pretty much every case.  @jni, if you can take one more look, that'd be great.

Hi,
This seems to break my usage of colormap=("from_omero", cmap), when NOT using channel_axis. The docs say:
"If a tuple the first value must be a string to assign as a name to a colormap and the second item must be a Colormap".
Thanks,
Will.

ah indeed!  sorry we missed that one as a multi-type argument.  I'll add a PR and a test

just to check, are you getting TypeError("Received sequence ...

Yes:
File "/Users/wmoore/Desktop/NAPARI/napari/napari/components/add_layers_mixin.py", line 212, in add_image
    f"Received sequence for argument '{k}', "
TypeError: Received sequence for argument 'colormap', did you mean to specify a 'channel_axis'?


[CT-1095] Support Jobs Cluster for python models

Describe the feature
We are using the cluster provided in credentials to submit python jobs to a given cluster. But current setup means the Python and SQL models can only be submitted to the same cluster. This means user can only use all purpose cluster if they want to run both Python and SQL jobs now.
We want a way to specify a separate cluster for python models so that user can run python models on jobs cluster while keep the sql models running on SQL endpoint to reduce running cost.
Describe alternatives you've considered
Not really
Who will this benefit?
Users currently running SQL models on SQL endpoint
Are you interested in contributing this feature?
Yes

@jtcohen6  @Fleid This should be a relative easy lift after you all decide how should this separate cluster ID being specified by the user

@ChenyuLInx I think the way to square this circle would be by allowing users to configure endpoint in profiles.yml (still used for SQL models), and then adding a cluster config for a specific Python model. Just in the same way we're adding support for dataproc_cluster_name as a model-level config in dbt-bigquery.
def model(dbt, session):
    dbt.config(cluster = "1234-abcd-wxyz")

As far as I understand it, the names of interactive clusters are not secret, and there is no risk of checking these into version control. I'll ask the Databricks folks to make sure that's not a faulty assumption!
Then I think the way to support this in the plugin would be with logic like:
self.cluster = self.parsed_model["config"].get("timeout", self.credentials.cluster)

Eventually, we probably do want to allow per-model configuration of endpoint and cluster for SQL models, too. I think that would require a much bigger lift in our codebase — there isn't a simple use cluster / use warehouse mechanism, as in other adapters, and there's more distance between our standard connection logic and node configurations — but it feels technically possible.
One note here: I think this is different from actually supporting Jobs Clusters, which are technically different (and less $$) in Databricks than all-purpose interactive clusters. I imagine those would require a different submission method, with (I imagine) much higher latency — they are not, by definition, interactive.
(cc @Fleid @lostmygithubaccount)

@jtcohen6  yes, that similar logic has been used across places for different configs in python model(looks like you copied a piece of timeout code LOL).
I think I might have some extra cycle to play with Jobs cluster soon, I will take a look and report back

looks like you copied a piece of timeout code LOL

you caught me...

@jtcohen6 turns out my understanding of jobs cluster was wrong(I think???), looked up the docs(link1, link2) and it is just you create a new cluster to run a job each time, did some local test and running one job took 290s ish. Actual job 47s, rest was cluster provision stuff

@ChenyuLInx That sounds right to me. And actually faster than I expected! For long-running models performing complex transformations, that upfront cost (time) might be worth it, in exchange for the lower cost ($$) of per-minute compute.

It's pretty straightforward, Just need to figure out what options do we want to provide our user with, As in the example PR, I just copied some example config from Databricks tutorial, I am assuming users would want to configure that in project.yml, then define a submission method(job_cluster) to use it?
Since we now still run jobs using the notebook, we are still requiring a user for it. I feel like we should probably switch to upload to a dbfs file for job_cluster submission method so that users don't need to provide an extra user config.
Although this brings another things, maybe submission_method should be a combination? We got 2 types of clusters(interactive, job_cluster), 3 ways to upload file(notebook, dbfs, command), should we actually update the UX a little bit?

@jtcohen6
self.cluster = self.parsed_model["config"].get("cluster", self.credentials.cluster)

Is this already available if we call the above in adapters? If so, that would be great!

Eventually, we probably do want to allow per-model configuration of endpoint and cluster for SQL models, too.

I like the idea.
Actually we had a request to provide a way to select endpoints for each model. databricks/dbt-databricks#59.

self.cluster = self.parsed_model["config"].get("cluster", self.credentials.cluster)
# Is this already available if we call something like the above in adapters?

@ueshin I think this is not available now, but def happy to change things to that format. This is how currently that cluster is provided for using jobs API. (We also did some refactor of submission code recently, I am going to open an issue in dbt-databricks about it shortly)
What would do you expect user to provide in that cluster if they want to use a job cluster? Should user just write the whole dictionary for configurations there?

@jtcohen6
self.cluster = self.parsed_model["config"].get("cluster", self.credentials.cluster)
Is this already available if we call something like the above in adapters? If so, that would be great!

@ueshin I think this is not available now, but def happy to change things to that format.

@ChenyuLInx I'd think this should just work for the code in dbt-databricks, if the user has specified cluster in their config.
One note: It looks like dbt-databricks calls this cluster_id, instead of cluster. Could we shoot for consistency here? Perhaps cluster + cluster_id as supported aliases in both plugins?

@ChenyuLInx @lostmygithubaccount Bringing back the main topic in this thread:

Since we now still run jobs using the notebook, we are still requiring a user for it. I feel like we should probably switch to upload to a dbfs file for job_cluster submission method so that users don't need to provide an extra user config.
Although this brings another things, maybe submission_method should be a combination? We got 2 types of clusters(interactive, job_cluster), 3 ways to upload file(notebook, dbfs, command), should we actually update the UX a little bit?

We're imagining submission_method: job_cluster, configurable per-model, whereby dbt would ship up the Python code to run on a jobs cluster instead of an interactive cluster.
If this means shipping up the PySpark code to dbfs first, that feels fine & doable.
Makes sense that the user may wish to supply other / additional configuration! Do we need a whole dict of optional key-value pairs?

Makes sense that the user may wish to supply other / additional configuration! Do we need a whole dict of optional key-value pairs?

is the cluster already defined? could/would we allow specifying Spark details (executor/worker hardware specs) as part of the job submission?

If this means shipping up the PySpark code to dbfs first, that feels fine & doable.

on this, are we then managing some filesystem? would we have something like dbfs://dbt/runs/<invocation_id>/<model_file>?

is the cluster already defined?

I believe jobs clusters are transient — created, used, and then spun down.

could/would we allow specifying Spark details (executor/worker hardware specs) as part of the job submission?

This is definitely a possibility. It's something we've talked about for Dataproc, too. Do something sensible by default. Otherwise, allow user to specify a bundle of key-value config pairs. dbt passes them through, without any validation.

on this, are we then managing some filesystem? would we have something like dbfs://dbt/runs/<invocation_id>/<model_file>?

Maybe? For the notebook method, we're using the <schema> to disambiguate (plus user value, since notebooks are in user-level workspaces):
https://github.com/dbt-labs/dbt-spark/blob/e07b8a20fa409e24e4c9959739c3eaefaf2eeb5a/dbt/adapters/spark/python_submissions.py#L133
On subsequent runs within the same environment, dbt just updates (overwrites) that notebook.
That feels closer to the experience of dbt run overwriting a table in your schema (environment). We could add in a totally unique identifier like <invocation_id>. Although git + artifacts/logs have been the traditionally sufficient answer for preserving older versions of dbt code.


Support for smaller addressing modes of registers

Hi, I was wondering if you're ever planning to add support for different addressing modes for registers (RAX - EAX ...), and if so, how high/low it is on your to-do list.

Sorry, what's this in reference to? codegen or disassembler? because in terms of codegen currently all except the high regs are accessible (no AH, CH, DH, BH)

Ah, my bad 😄, I was talking about the code generator, but I must've missed that you're doing it. I was trying to do something like this:
i32 test() {
    i32 a = 100;
    i32 b = 200;

    return a + b;
}

And the generated assembly was using the entire RAX and RCX (iirc) registers, which got me a bit confused as it was using the entirety of the 64 bit registers to store 32 bit values, but on second thought I guess it doesn't really matter in this case.

Yea, the old assembler didn't display register labels differently for size, that's now fixed since it's doing a proper disassembler job and not just lazily logging what i do


HTTPS on OMV

Hi!
I try to add HTTPS on OMV (arm64_0.5.13) but if I try to install "letsencrypt" not work.
root@rock64-omv:~# sudo apt-get install letsencrypt python-letsencrypt-apache
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package letsencrypt
E: Unable to locate package python-letsencrypt-apache
N.B. The same problem arises when I try to openmediavault-letsencrypt 3.2 from plugin of OMV

armhf or arm64?

You probably need to configure jessie-backports.

arm64
I'll try to configure jessie-backports

"The file for arm64 is old and for this the install not work" say @ayufan after research. Thanks.
Possible solution:

Manual install of SSL in OMV using "Import" in SSL section
Use apache (if you have website) and install cert in separated folder and add it to apache virtual host configuration

The only solution today is to use OMV: armhf image as this is the only one properly supported by jessie.


add a lecture video for TensorFlow

@rickiepark saw your blog love it, can you email me? my email is<EMAIL_ADDRESS>


ShowText Node is no longer displaying text.

Updated ComfyUI, and now the ShowText Node is no longer displaying text.

It is however passing text though.

Its just not showing the text in the ShowText Node's Display like it use to anymore.

Hi,
I've been having the same problem. I investigated and debugged a bit, and here is a few pointers:
If you are using DeepFuze custom node, there seem to be a conflict as stated here
On the frontend side, when ShowText is done receiving input to display, it goes into nodeType.prototype.onExecuted from showText.js. This executes:
onExecuted?.apply(this, arguments);
populate.call(this, message.text);

The first line seems to wrongly trigger their function from DeepFuze's preview_audio.js
nodeType.prototype.onExecuted = function (data) {
    previewAudio(this, data.audio[0], data.audio[1]);
}

Which results in an error (as we aren't trying to preview audio or anything, the data value passed is null).
When such errors occur, I think comfyUI handles it by using a break, which stops the execution, thus not displaying the text through populate.call.
Possible solutions

Users having this problem can try uninstalling the DeepFuze custom node, assuming they aren't using it in the same workflow. This did it for me. I think there is a way to declare a conflict somehow, it could be useful to avoid this.
Switching the order of calls in the code seems to do it (still result in an error on DeepFuze's end, but after updating the text in ShowText node), these line 44 and 45 from showText.js would become :

	populate.call(this, message.text);
	onExecuted?.apply(this, arguments);

-> Would there be any side effect to switching the order to that? What is the goal of onExecuted?.apply(this, arguments);in ShowText's context?

I guess fixing DeepFuze to filter properly what is received on their end, like handling the case where data is null or not usable for their purposes would do it.

Hope it helps!

That has indeed helped.  you are a star Mr @Beatboxace
I hope DeepFuze gets a fix for it then.
DeepFuze has caused a number of issues with users I see, their requirement file uninstalls newer dependencies and replaces them with MUCH older ones.
Even me, personally, I had to re-install ONNX, Torch and XFormers after DeepFuze ballsed them up.
Or... We just stop using DeepFuze :(

I have same issue even I didn't installed the DeepFuze

I have this same issue and don't have DeepFuze installed. I hope the dev can fix it. Does anyone know of any other nodes that have the same functionality?

hello! Your letter I have received, thanks!

Please Pythongosssss, a lot of people, workflows and nodes depend on this one!


Ability to export a .tldr file

Description
tldr.com has the option to "Save a Copy" and "Open a file". Both of these operate on the .tldr extension.

Request
Implement the ability to export a .tldr from Obsidian.
Related
This is the inverse of #11

I can confirm this works as intended.


15 parse frontend bundler from browserify to parcel

parse from browserify to parcel.
for testing purposes, browserify still working on src/public

Not totally sure about the file and directory layout, and the inclusion of some files that look like they have been generated; looks good anyway, but it would be nice to have a few days (or until the next week) to check out and comment, if I may.

ok, no problem. I'll look at LitElements and cypress for testing in the next days

I create a new issue #22 concerning the organization of the repository.  The parsing and test is complete.