invalid lifting of structures fields usage

Binary Ninja Version: 2.2.2568-dev Personal, c88a4dc7
Platform: Windows
It seems that during lifting some information can be lost.
In attached archive there is two binaries:
First (test_no_cl_usage) doesn't include seta cl in x2 block and bn correctly shows that we will use struct1->field8 (last line):

also we can see correct output in HLIL:

but if we use seta cl in x2 block (test_cl_usage binary) we will get following picture:

in HLIL:

in this second case bn shows us that we are reading first field of structure and it seems a bit wrong..
bins.zip

Thanks for the report we will look into this.


fix "Cannot read property includeInto of undefined" error

This fixes an error in pool.js that causes an error message "Cannot read property includeInto of undefined" (and no table) to appear when opening csv/tsv files with the Table Editor.
It appears that vm.runInContext() does not return a copy of the class when called with an ES6 class declaration, although it may have returned a copy of the class when called with a coffescript class declaration in the past. This pull request refers to the class at the end of the source code, which causes vm.runInContext() to return the class instead of undefined. This prevents the error noted above.
This fixes issues #100 and #101.

I tested it and it worked for me on win 10 with latest atom.  Thanks!

Using branch in my atom and I can't open any CSV files at all once the package is installed. No error messages, it just shows the empty pane. Atom 1.28.2 on macOS 10.13.6

Confirm it is fixing the issue on Mac OS 10.14.1 (18B75)
Atom 1.34.0 x64

Feb 21 2019
Not working on Mac OS 10.13.6 High Sierra
Atom 1.34.0 x64

Worked for me.

Getting the same error :(


Autoscroll.Not Working

Steps to reproduce

Select autoscroll input any values
Autoscroll doesnt work, and page is stuck.

Expected behavior
I expect autoscroll well, automatically scroll.
Actual behavior
The page stays as it is, and no automatic scroll happens.
And before anyone asks, i have done evertyhing possible. That includes reinstalling, toggling each reader settings on and off and checking. Its still the same.
This is a new issje started on r451
Crash logs
No response
Tachiyomi version
r451
Android version
A11
Device
Galaxy Tab S7FE
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.
[X] I have written a short but informative title.
[X] If this is an issue with an extension, I should be opening an issue in the extensions repository.
[X] I have tried the troubleshooting guide.
[X] I have updated the app to version 1.8.3.
[X] I have updated all installed extensions.
[X] I will fill out all of the requested information in this form.

This is part of the scrolling bug main Tachi has, it will be fixed when main Tachi fixes the scrolling bug.


Terminal blockhash override

Adds a terminal block hash override for the merge fork as originally discussed here.
I'm creating this PR as a starting point for discussion while acknowledging that this might be somewhat controversial (more so than the total difficulty override) because coordinating on a specific blockhash might be perceived as more 'political'.

closing in favor of adding a constant as in https://github.com/ethereum/consensus-specs/pull/2605#issuecomment-924848499


src map works; filter ui works; schema: loc -> source; prev/next btns

must reprovision. we now serve from src so get full benefit
of source map.
filters are updated by url bar; update filters button updates
url bar.
changed 'loc' to more descriptive 'source'.

Reviewed, possibly could simplify filter functions


bugfix: event create date

closed #785

no, its not
because setTouched need to be true when user submit event create, and it done here through useField (you can`t do this only through useState (actually can, if you make useState be true when user click submit, but this is kostyl ebaniy))


not opening on osx

I'm still having this issue as well

Try troubleshooting using the wiki:

https://github.com/nwoolls/MultiMiner/wiki/Installation#mac-os-x


Page contract selfasserted 2.0.0 seems to break locatisation of built-in strings

This works (page contract 1.2.0):
<DataUri>urn:com:microsoft:aad:b2c:elements:contract:selfasserted:1.2.0</DataUri>

    <ContentDefinitions>
      <ContentDefinition Id="api.selfasserted">
        <LoadUri>https://example.com/unified?ui_locales={Culture:LanguageName}</LoadUri>
        <RecoveryUri>~/common/default_page_error.html</RecoveryUri>
        <DataUri>urn:com:microsoft:aad:b2c:elements:contract:selfasserted:1.2.0</DataUri>
        <Metadata>
          <Item Key="DisplayName">Collect information from user page</Item>
        </Metadata>
        <LocalizedResourcesReferences MergeBehavior="Prepend">
          <LocalizedResourcesReference Language="en" LocalizedResourcesReferenceId="api.selfasserted.en" />
          <LocalizedResourcesReference Language="nl" LocalizedResourcesReferenceId="api.selfasserted.nl" />
        </LocalizedResourcesReferences>
      </ContentDefinition>
    </ContentDefinitions>

    <Localization Enabled="true">
      <SupportedLanguages DefaultLanguage="en" MergeBehavior="ReplaceAll">
        <SupportedLanguage>en</SupportedLanguage>
        <SupportedLanguage>nl</SupportedLanguage>
      </SupportedLanguages>
      <LocalizedResources Id="api.selfasserted.en">
        <LocalizedStrings>
          <LocalizedString ElementType="ClaimType" ElementId="signInName" StringId="DisplayName">Email</LocalizedString>
        </LocalizedStrings>
      </LocalizedResources>
      <LocalizedResources Id="api.selfasserted.nl">
        <LocalizedStrings>
          <LocalizedString ElementType="ClaimType" ElementId="signInName" StringId="DisplayName">E-mailadres</LocalizedString>
        </LocalizedStrings>
      </LocalizedResources>
    </Localization>

This doesn't work (with page contract 2.0.0, built-in strings, e.g. the "Continue" button are not translated):
<DataUri>urn:com:microsoft:aad:b2c:elements:contract:selfasserted:2.0.0</DataUri>

    <ContentDefinitions>
      <ContentDefinition Id="api.selfasserted">
        <LoadUri>https://example.com/unified?ui_locales={Culture:LanguageName}</LoadUri>
        <RecoveryUri>~/common/default_page_error.html</RecoveryUri>
        <DataUri>urn:com:microsoft:aad:b2c:elements:contract:selfasserted:2.0.0</DataUri>
        <Metadata>
          <Item Key="DisplayName">Collect information from user page</Item>
        </Metadata>
        <LocalizedResourcesReferences MergeBehavior="Prepend">
          <LocalizedResourcesReference Language="en" LocalizedResourcesReferenceId="api.selfasserted.en" />
          <LocalizedResourcesReference Language="nl" LocalizedResourcesReferenceId="api.selfasserted.nl" />
        </LocalizedResourcesReferences>
      </ContentDefinition>
    </ContentDefinitions>

    <Localization Enabled="true">
      <SupportedLanguages DefaultLanguage="en" MergeBehavior="ReplaceAll">
        <SupportedLanguage>en</SupportedLanguage>
        <SupportedLanguage>nl</SupportedLanguage>
      </SupportedLanguages>
      <LocalizedResources Id="api.selfasserted.en">
        <LocalizedStrings>
          <LocalizedString ElementType="ClaimType" ElementId="signInName" StringId="DisplayName">Email</LocalizedString>
        </LocalizedStrings>
      </LocalizedResources>
      <LocalizedResources Id="api.selfasserted.nl">
        <LocalizedStrings>
          <LocalizedString ElementType="ClaimType" ElementId="signInName" StringId="DisplayName">E-mailadres</LocalizedString>
        </LocalizedStrings>
      </LocalizedResources>
    </Localization>

This workaround also works (specifying the built in strings, not only the custom ones), but would be annoying to do for all built-in strings:
<DataUri>urn:com:microsoft:aad:b2c:elements:contract:selfasserted:2.0.0</DataUri>

<LocalizedString ElementType="UxElement" StringId="button_continue">Doorgaan</LocalizedString>

    <ContentDefinitions>
      <ContentDefinition Id="api.selfasserted">
        <LoadUri>https://example.com/unified?ui_locales={Culture:LanguageName}</LoadUri>
        <RecoveryUri>~/common/default_page_error.html</RecoveryUri>
        <DataUri>urn:com:microsoft:aad:b2c:elements:contract:selfasserted:2.0.0</DataUri>
        <Metadata>
          <Item Key="DisplayName">Collect information from user page</Item>
        </Metadata>
        <LocalizedResourcesReferences MergeBehavior="Prepend">
          <LocalizedResourcesReference Language="en" LocalizedResourcesReferenceId="api.selfasserted.en" />
          <LocalizedResourcesReference Language="nl" LocalizedResourcesReferenceId="api.selfasserted.nl" />
        </LocalizedResourcesReferences>
      </ContentDefinition>
    </ContentDefinitions>

    <Localization Enabled="true">
      <SupportedLanguages DefaultLanguage="en" MergeBehavior="ReplaceAll">
        <SupportedLanguage>en</SupportedLanguage>
        <SupportedLanguage>nl</SupportedLanguage>
      </SupportedLanguages>
      <LocalizedResources Id="api.selfasserted.en">
        <LocalizedStrings>
          <LocalizedString ElementType="ClaimType" ElementId="signInName" StringId="DisplayName">Email</LocalizedString>
        </LocalizedStrings>
      </LocalizedResources>
      <LocalizedResources Id="api.selfasserted.nl">
        <LocalizedStrings>
          <LocalizedString ElementType="ClaimType" ElementId="signInName" StringId="DisplayName">E-mailadres</LocalizedString>
          <LocalizedString ElementType="UxElement" StringId="button_continue">Doorgaan</LocalizedString>
        </LocalizedStrings>
      </LocalizedResources>
    </Localization>

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c4272fbd-ad3d-6099-39bc-966320101b03
Version Independent ID: ac21035f-4145-b309-d45c-3c38e965301d
Content: Page layout versions - Azure AD B2C
Content Source: articles/active-directory-b2c/page-layout.md
Service: active-directory
Sub-service: b2c
GitHub Login: @msmimart
Microsoft Alias: mimart

@wouterdevinck, thank you for reaching out. We are looking into this and would get back to you soon on this thread.

@wouterdevinck Thank you for pointing this out. Based on the workaround you have mentioned, I understood that localization for continue button at signup page stops working with Page contract selfasserted 2.0.0.
Apart from that, have you noticed any other built-in strings with the same issue?

The issue occurs for all built-in strings (ux elements, error messages) when switching from selfasserted 1.2.0 to 2.0.0, not just the continue button.

@wouterdevinck I tested this and found that the behavior is different for different languages. When I used ui_locales=nl, localization for all built-in strings was not working and with ui_locales=hi only a few strings were not translated.
Even though there is different impact on different languages, selfasserted 2.0.0 is definitely breaking localization. I will be engaging our Product Team to address the issue.
Could you please send an email with the URL of the issue at azcommunity[at]microsoft[dot]com, so that I can open a free support case and a corresponding ticket with product team to address the issue?

Thank you, I didn't think to test in other languages. Will drop a mail.

Hello there, I can also confirm this issue happening for pt-br language. Thanks.

Open support case:
Support request ID |<PHONE_NUMBER>00504

@wouterdevinck Thanks for the support case.
I am closing this issue as we use this platform for Document enhancement but we'll keep posting comments to keep the thread up to date until the issue is fixed.

Hello @wouterdevinck, would you please share the support case link? I would like to keep posted about this issue as well. Thanks

I found the same issue, so I had to revert my selfAsserted back to 1.2.0
Thank you

This is fixed now. Kindly check and confirm.

@Feday and @henriquebelotto Have you had a chance to test this out?

Translation of the buttons looks good but the translation on errors not always.
for example this one get auto translated:

but these ones does not get translated:
one time password error 1:

one time password error 2:

Hello @amanmcse I still see strings that are not localized, for example, the email address placeholder and invalid email error message.


Nginx - Too Many Redirects

Steps required to reproduce the problem

Use following nginx config:

server {
    listen 443;
    server_name <domain>;
    root /var/www/opencfp/web;
    index index.php;

    ssl on;
    ssl_certificate ../fullchain.pem;
    ssl_certificate_key ../privkey.pem;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location ~ \.php$ {
        fastcgi_param CFP_ENV production;
        fastcgi_pass unix:/run/php/php7.0-fpm.sock;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param HTTPS off;
    }
}

Enable secure_ssl in the production.yml file.
Visit the site.

Expected Result
To see the OpenCFP dashboard.
Actual Result
Receive a ERR_TOO_MANY_REDIRECTS error in Chrome.
However, when disabling secure_ssl setting and visiting the site on HTTPS it loads the dashboard fine.

Nevermind...  Stupid copy paste mistakes.
Had to change this line:
fastcgi_param HTTPS off;

to:
fastcgi_param HTTPS on;

Original copy came from: http://silex.sensiolabs.org/doc/web_servers.html


docs: Update CONTRIBUTING.md

Add section to running Development Container outside of VSCode
What is it?

[ ] Feature / enhancement
[ ] Bug
[x] Docs / tests

Description
Update Contributing Guidelines to include instructions to running Development Container for non-VSCode users (like me).
Use cases and why

Allow non-VSCode users to easily contribute to Qwik.

Checklist:

[x] My code follows the developer guidelines of this project
[x] I have performed a self-review of my own code
[x] I have made corresponding changes to the documentation
[x] Added new tests to cover the fix / functionality

@zanettin you are definitely right I wrote wrong portion of the command (-v .:/home...) for docker which should've been -v $PWD:/home.... Fixed now.

Hi @mahmoudajawad
Thanks very much for the adaption 🙏  can you please run pnpm fmt to get prettier satisfied on the CI as well? 😄  thanks again 🙏

@zanettin, sorry for missing your message, and thanks for accepting my contribution.


The word "TO" lacks NLS

Hi,
In other locale, the word "TO" in the px-datetime-range-field lacks NLS. Could you please fix this.

Hi, what is NLS?

NLS (localization/internationalization) - browser language

Never heard of it referred to as 'NLS'... I understand now.
'TO' is localisable like the other strings in px-datetime-range-field: https://github.com/PredixDev/px-datetime-range-field/blob/master/px-datetime-range-field.html#L73
See the guide here:
https://www.predix-ui.com/#/develop/internationalization


README: There is no need to use promises

As you said in README there is no need to use promises for getting those values I think!

Can you elaborate on what you mean/what you think the issue is?

Can you elaborate on what you mean/what you think the issue is?
It is not an issue but it confuses.

The library recently underwent a transformation, which introduced promise based API's (see: https://github.com/react-native-community/react-native-device-info/pull/760).  This would improve startup time, etc.
That being said, if you don't want to use the promise based API's, then you are able to use synchronous variants i.e. xxxSync().  However, there is a known issue with using the synchronous API's (see: https://github.com/react-native-community/react-native-device-info/issues/776).


FRQ Question 1

Crossover Score: 2.7+
Score from Mr. M:
Adi Fastpages
Aadit Fastpages

[x] FRQ Question One
[x] 2D Iteration

Somewhat individual or pair focussed.  Not seeing help across team.  Budget of time is poor.
Adi - starts with Tech Talk on code.  Shared two ways of init on arrays.  Shared looping and direct access on arrays.  Shared way of outputting arrays in many different orders (front, back, horz, vert, etc).  2.8
Aadir - 2018, 1019, 2015.  Frog question, discussed algo on hops and simulations.  Leap years, discussed loop of range and including year2 also day of year.  Array Sum was final question which worked with 2D arrays.  Including isDiverse to determine if all sums were unique.  2.8
Soma - leap year discussed struggle use.  Very quick in presentation.  Also, share cars and random elimination. 2.6
Tianbin - did actual leap year calculation.    Discussed Monkey  code and highlighted verticle aspects.  Added to story with animal. 2.7

Nice presentation! Really liked all the code you guys submitted and also the way you had mini lessons about arrays at the start. FRQ questions also presented nice. Personally, it was hard to see teamwork amongst the team but otherwise the work was nice. I'd say all of you deserve 2.7+/3 or somewhere around that.

Overall Score: 2.9/3
The explanation of the array really helped me understand more about it. I also like how you implemented your own array not using the example of Mr. Mortensen and how you guys completed three FRQs. Everyone in the group did an amazing array and explained each FRQ very concisely. The only thing to improve on is maybe working more as a team by showing something like a scrum board to show group collaboration could do the job.

2.7+/3 Lots of frqs and arrays with good explanations- you can tell they put a lot of effort in. One thing they could add would be some more teamwork like with something like a scrum board

Nice presentation! Really liked how you displayed the code and taught the mini lesson at the start. Also liked how you presented the FRQs. Personally think you guys could have showed team work a little better but other than that have no negative views on your presentation. I feel that all of you should receive a 2.7+ or somewhere around that.


feat(test): extend karma.conf.js w/ kjhtml reporter

Add karma-jasmine-html-reporter (kjhtml) which displays Jasmine test runner in the test browser.
We use this in docs testing chapter so users can visualize their tests in the test browser rather than see a blank page.  Convenient for exploring broken tests and re-running certain specific tests.

Has no apparent ill-effects on CI. We run these tests with this config in docs CI.
Deliberately excluded from the test/code-coverage combo ... for no great reason other than I thought someone who was running with code-coverage wouldn't want it.

I think it would be a better reporter than just having a blank screen.

LGTM

will it be possible to view the report at a later point in time ? (like the coverage reports)
As part our CI, we have to run our unit tests using 'Phantom JS' & we would like to review the results later


List of missing aggregation stage operators

I took time to review our existing aggregation operators and compare them to mongodb aggregation stages. And here is the list of missing items:

[ ] bucket
[ ] changestream
[ ] changeStreamSplitLargeEvent
[ ] collStats
[ ] currentOp
[ ] densify
[ ] documents
[ ] fill
[ ] geoNear
[ ] indexStats
[ ] listLocalSessions
[ ] listSampledQueries
[ ] listSessions
[ ] planCacheStats
[ ] redact
[ ] search
[ ] searchMeta
[x] set alias for addFields
[ ] setWindowFields
[ ] shardedDataDistribution
[ ] unset
[ ] vectorSearch

I feel like most of these operators are quite niche, so they're definitely not high priority. We can add them when someone need them as well.
Also maybe it's a good idea to support a way to construct aggregation from org.boson.Bson or BsonValue as I mentioned in #40. So, We allow user to create their own aggregation in case of missing.


if a synclet doesn't finish, it hangs forever

Need a timer/timeout to kill it after some amount of time.

I don't think we ever really know how long a synclet would take to finish.  We can certainly speculate, but this could be disasterous (such as in the case of a new user with a massive amount of data in one service) and killing off the synclet would cause it to just do the same thing again the next time it runs.  Is there a synclet that's occasionally not finishing?  If so, we should fix that!

I've had a 4sq one get stuck for a day, it was likely a bum http request in some unknown state... Agree on the not-knowing-when being a real problem, I think the request module given a timeout might help too if that's the case.
I think for all the existing synclets that there's a reasonable upper bound, at most 30 minutes maybe as a default, but could be overridden in the manifest?

That sounds reasonable to me. +1.


tf.io.GFIle not working correctly with UTF-8 files and Python3

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu
TensorFlow installed from (source or binary): source internal Google
TensorFlow version (use command below): 1.5.0, internal Google
Python version: 3.6.7

Describe the current behavior
Calling 'read(X)' on the text files opened with GFile in python3 doesn't work properly (it fetches X bytes rather than X characters). This often results with the UnicodeDecodeError (as the read can happen in the middle of the unicode character).
Describe the expected behavior
It should behave like python3: reading the X characters.
Code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
my_text = 'Bären'
with open('/tmp/ex1', 'w') as f:
  f.write(my_text)

// Will read the whole string correctly.
with open('/tmp/ex1', 'r') as f:
  print(f.read())

// This will print 2 chars Ba
with open('/tmp/ex1', 'r') as f:
 print(f.read(2))

// This will print 3 chars: Bar
with open('/tmp/ex1', 'r') as f:
 print(f.read(3))

// This will print the whole thing.
with tf.io.gfile.GFile('/tmp/ex1', 'r') as f:
  print(f.read())

// This will crash.. :-(
with tf.io.gfile.GFile('/tmp/ex1', 'r') as f:
  print(f.read(2))

Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
The error will be:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 1: unexpected end of data

Confirmed in tf-nightly and python3.7

I think it is possible to check the utf8 length while reading, and compensate the remaining bytes. Created a PR #33590 for the fix.

Hi There,
We are checking to see if you still need help on this, as you are using an older version of tensorflow which is officially considered end of life . We recommend that you upgrade to the latest 2.x version and let us know if the issue still persists in newer versions. Please open a new issue for any help you need against 2.x, and we will get you the right help.
This issue will be closed automatically 7 days from now. If you still need help with this issue, please provide us with more information.


Update krew-index on release tag

Fixes https://github.com/submariner-io/releases/issues/904

I'm not sure if there's a way to actually run the new GHA but, once merged, we I think we should be able to manually trigger it from the Actions tab for the 0.16.3 release tag.

I'm not sure if there's a way to actually test the new GHA now but, once merged, we should be able to manually trigger it from the Actions tab for the 0.16.3 release tag.

See the GHA docs:
podman run -v .krew.yaml:/tmp/template-file.yaml ghcr.io/rajatjindal/krew-release-bot:v0.0.46 \
krew-release-bot template --tag v0.17.0-rc0 --template-file /tmp/template-file.yaml

in the releases project (for v0.17.0-rc0). This produces the index which would be pushed to update the subm.yaml file in the Krew index.

This would fix the job, but there’s a more general problem I think we need to address before we get things working on all releases: since we can only publish one version of the subm plugin, we need to ensure that (a) we only publish actual releases (not rcs etc.)

The krew-release-bot takes care of that: "If a release is marked as a 'prerelease' in github, it will not be released to the krew index.".

and (b) we only publish the latest release on the latest branch.

I was thinking about that too. I'm not sure how we can do that....

I'm not sure if there's a way to actually test the new GHA now but, once merged, we should be able to manually trigger it from the Actions tab for the 0.16.3 release tag.

See the GHA docs:
podman run -v .krew.yaml:/tmp/template-file.yaml ghcr.io/rajatjindal/krew-release-bot:v0.0.46 \
krew-release-bot template --tag v0.17.0-rc0 --template-file /tmp/template-file.yaml

in the releases project (for v0.17.0-rc0). This produces the index which would be pushed to update the subm.yaml file in the Krew index.

yeah that will test the .krew.yaml file but not the GHA.

I found a way to get the release tag marked as the latest.


Sort CodeGenCompleted listener tasks to run the inbuilt plugins before the package-provided ones

Purpose

Sort CodeGenCompleted listener tasks to run the inbuilt plugins before the package-provided ones
Fixes #

Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Can we simply run the C2C plugin as the last one? This will guarantees that the C2C plugin runs as the last one?
Otherwise we will have to introduce more hacks

Then we might need to iterate the keyset and find the c2c by the classname since at this point what we have is a map of <LifeCycleListenerInfo, List>
Is this ok to do?


Weird bug with one of the values not presented correctly

Not sure what is causing this, but all of the fields get shaded correctly except for one, which shows up as empty (however, hovering over it will show the correct value - it's just the colouring that is missing).

my settings look like this:
cal.init({
range: 5,
domain:"month",
subDomain:"x_day",
start: new Date(),
data: "/logs/returnDates.json",
highlight: "now",
domainDynamicDimension: false,
cellSize: 20,
cellPadding: 1,
domainGutter: 10,
legend: ,
legendCellSize: 15,
previousSelector: "#goLeft",
nextSelector: "#goRight"
});
the json returned by returnDates is:
{
"1375740000": 27,
"1389049200": 8,
"1389135600": 36,
"1389222000": 19,
"1389308400": 10,
"1389567600": 117,
"1389654000": 37,
"1389740400": 31,
"1389826800": 18,
"1389913200": 11,
"1390258800": 31,
"1390345200": 4,
"1390431600": 12,
"1390518000": 27,
"1390777200": 39,
"1390863600": 68,
"1390950000": 77,
"1391036400": 69,
"1391122800": 47,
"1391295600": 23,
"1391382000": 2,
"1391468400": 22
}
The value of $range is: [0, 23, 46, 70, 93]
Not sure if it's a bug or if I did something wrong :)

Closed since inactive

Sorry, completely forgot about this. I've just re-tested it on our current version and it works as intended. My apologies!


Doc adjustments for v1

With us heading into the marketplace real soon, I want to make sure the README will reflect the v1 status and latest discussions on naming 🤔

Oops 😂
I thought I had all three repos lined up with the same changes 🤯


Integration tests misleadingly use the word galley

Bug description
Some folks look at the code base and think galley is still present in the integration tests. Would be nice to close this out as a low priority issue.
Expected behavior
no change in behavior needed.
Steps to reproduce the bug
Version (include the output of istioctl version --remote and kubectl version and helm version if you used Helm)
master@ 64f6913a2e040bd23dba8574a107503c0e72d70e
How was Istio installed?
not installed.
Environment where bug was observed (cloud vendor, OS, etc)
in a testing environment.

Example: https://github.com/istio/istio/blob/master/tests/integration/pilot/main_test.go#L43

@nmittler I unassigned you and am closing this window. Hopefully an intern can pick this up as a day 30 project and hopefully you can coach.
Cheers
-steve

Agreed. This is a duplicate of https://github.com/istio/istio/issues/20299, lets consolidate there. We have made some progress on this


[Event Request] Table 5902 Service Line - procedure CalcVATAmountLines

Dear ALAppExtensions Team,
We would like to request the following change
procedure CalcVATAmountLines(QtyType: Option General,Invoicing,Shipping,Consuming; var ServHeader: Record "Service Header"; var ServiceLine: Record "Service Line"; var VATAmountLine: Record "VAT Amount Line"; isShip: Boolean)
var
    Currency: Record Currency;
    QtyFactor: Decimal;
    TotalVATAmount: Decimal;
    RoundingLineInserted: Boolean;
begin
    **//--> change request
    IsHandled := false;
    OnUpdateCalcVATAmountLinesOnBeforeProcessServiceLines(ServHeader, ServiceLine, VATAmountLine, QtyType, isShip);
    if IsHandled then
       exit
    //<-- change request**
    Currency.Initialize(ServHeader."Currency Code");

    VATAmountLine.DeleteAll();

    with ServiceLine do begin
        SetRange("Document Type", ServHeader."Document Type");
        SetRange("Document No.", ServHeader."No.");
        SetFilter(Type, '>0');
        SetFilter(Quantity, '<>0');
        if FindSet() then

  ........................

end;

[IntegrationEvent(false, false)]
local procedure OnUpdateCalcVATAmountLinesOnBeforeProcessServiceLines(var ServHeader: Record "Service Header"; var ServiceLine: Record "Service Line"; var VATAmountLine: Record "VAT Amount Line"; QtyType: Option General,Invoicing,Shipping; isShip: Boolean)
begin
end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 45720.


Swift 3.x compatibility

Is there a blocking issue with swift 3 compatibility? With a few minor changes, it seems to compile ...
(just casting a few Uint32 to Int for comparison)

I recently had to close a Pull Request (#16) that tried to make the project usable with Swift 3. The proposed changes broke the test suite and also violated some aspects of the ZIP specification.
I currently plan to reach 1.0 as soon as the final versions of Swift 4/Xcode 9 get released with Swift 4 as the minimum language target but you should be able to get a Swift 3 compatible fork by implementing the things I commented in closed PR.


Is this usable in android/iOS?

Hello, great project and i love using it for UI testing. I noticed, however, that when i run my tests on standalone android and iOS, that the following error shows up
Error Message: System.Exception : UnityMainThreadDispatcher could not find the UnityMainThreadDispatcher object. Please ensure you have added the MainThreadExecutor Prefab to your scene.
Was wondering if its something in my configuration or if this is not meant to be used in android/ios

Works perfect on Android, we have been using it without issues.

Works perfect on Android, we have been using it without issues.

Perfect, thank you! I'll try and fix it 👍 :D

Perfect, thank you! I'll try and fix it 👍 :D


Cannot edit ui-codemirror in a taskpane of an add-in in Excel for Windows

I just realise that we cannot modify texts in code-mirror in a taskpane of an add-in in Excel for Windows.
I have an Excel add-in built by mean-stack and angular-ui-router. In a page, I have
<textarea ng-model='testCM1' ui-codemirror='option'></textarea>
<ui-codemirror ng-model='testCM2' ui-codemirror-opts='option'></ui-codemirror>

<textarea ng-model='testTextarea'></textarea>

The 4 textareas can well be edited in Chrome, IE, Excel Online, Excel for Mac. However, all the 3 textareas by ui-codemirror can not be edited in Excel for Windows, whereas the last textarea is modifiable.
Is anyone aware of this odd behavior? How could we fix this? Is there any workaround?
PS: here is a JSBin for testing, we could link this file to a manifest and try it in Excel for Windows.

I don't have any suggestions (not sure anyone has tried to run the editor in Excel yet), and I'm sorry, but Excel is just not a supported platform, so you're on your own here.


[docs] feature compatibility matrix

Motivation
For newcomers, it can be hard to remember which features work together and which do not. Examples: The reader api cannot be used with partitioned topics, and cumulative acknowledgement does not work for key_shared subscriptions.
I have already discussed adding such a feature matrix on slack, and it has been endorsed by sijieg:

A feature matrix like this is great

https://apache-pulsar.slack.com/archives/C5Z4T36F7/p1580147801255400?thread_ts=1580126358.237900&cid=C5Z4T36F7
N.B. This is still WIP and would benefit from more experienced Pulsarians filling in the blanks. Also, my intention is to add more tables, but whether or not I'll do it in this PR will depend on how quickly I or other reviewers will fill in the blanks for the existing tables.
Modifications

add a "Feature Compatibility Matrix" page to the "Reference section"
add a link to that new page to the "Concepts / Overview" page

Verifying this change
I've tested this as described in pulsar/site2/README.md, i.e. launching yarn start and browsing to all affected pages

I was wondering whether it's good to add it in the reference page or on the WIKI page.
We have Client Features Matrix on WIKI page.
If we decide whether to add it in the reference section or on the WIKI page for different matrix, it would be easier to manage and maintain.


Several issues

Hi,
First I wanted to thank you for the great package. It is super useful! I've been using it for two projects, so I have a bunch of questions/issues and I didn't want to open a separate issue for each, but if you prefer that I can also do that.
Small issues:

On my personal computer, on windows, I have not been able to install pytwoway, even after a couple of hours of trying because of the dependency quadprog. This seems to be a known issue, and I tried the fixes, but havent been able to make it work. In particular, I think quadprog now made a version with a prebuilt wheel and is recommending to packages which use quadprog to put that as a dependency.
Wanted to mention 2 warnings which seem to have no effect on the commands running:

I get the warning: "future version 'df.iloc[:,i] = newvals' will attempt to set the values in place instead of always setting in a new array". Just wanted to give a heads-up in case that will change something.
When I run a model with fit(), it works, but I do get the following warning: "sys:1: ResourceWarning: unclosed socket" (and then a bit more detail)

New feature maybe?:

Is it currently possible to add a third fixed-effect (an origin firm fixed-effect) like in It Ain't Where You're From, It's Where You're At: Hiring Origins, Firm Heterogeneity, and Wages (Sabrina Di Addario, Patrick Kline, Raffaele Saggio, and Mikkel Sølvsten) Journal of Econometrics, 232 (April 2023), pp. 340-374.. In particular, does simply adding it as a control work (the connectedness might not be correct then?)?

(Apologies in what follows I don't have reproducible examples, because I'm using confidential admin data)
Bigger issues:

If I use summary on the cleaned data, and in the next line run FEEstimator, the variances of y are different (see examples below). This is also the case with the simulated datasets in the examples. This only happens with collapse_at_connectedness_measure: True. Is that because different weights are used?
Negative implied alphas or the terms don't add up to the variance. I am using a binary variable as a dependent variable (like in Employers and Unemployment Insurance Take-up (Lachowska, Sorkin and Woodbury)). Below are 3 different versions of the summary and estimation results:

with collapse_at_connectedness_measure: True

with collapse_at_connectedness_measure: False

with collapse_at_connectedness_measure: False and controls

Sorry about the very long post and I hope that I didn't do something stupid!
Thanks a lot for your help!

Hi Simon,
Thanks for using the package and for reaching out!
Small issues:

I will have to talk with Professor Lamadon about updating quadprog. The reason pytwoway requires a specific version is that he requires a Conda version of quadprog, and since they don't officially release that he needs to compile it himself (I can't since I don't have a Windows computer). If you are open to publishing an updated version of quadprog on Conda then I can update the dependency, but I will warn you that Professor Lamadon has told me it is a difficult process.
Thank you for letting me know about the warnings. I am going to open an issue for the first one, the second one I've been getting for a very long time. I've looked it up online, I think it's very difficult to diagnose and it hasn't caused any problems for me yet.

New Feature:

There is already an issue open for this, but thank you for the paper, I've been looking for a good resource on how to estimate this. So far I have written some sample code to estimate the connected set (not leave-one-out) that seems to work well in testing, but I've only tried it on simulated data. The code is based on the papers by Weeks & Williams (1964) and Godolphin & Godolphin (2001). This code is a simple implementation and depends on sparsity, so if you are including a categorical control with a small number of categories, the code will not work (since it constructs the XX' matrix (not X'X), so if there are only a small number of categories many of the matrix elements will become non-zero). When I have time I'll look at the paper you linked, and also try to optimize the code I already wrote. I think this is an important feature to have.
If you try out my code and have any feedback, I would greatly appreciate it!
While running the code I pasted can be helpful, in my experience the FE estimator will raise an error when estimating leverages if one of the leverages equals one (which happens when the set isn't leave-one-out connected). So if you are using a categorical control and the HE correction still works, it is very likely that the set is leave-one-out connected.

Bigger Issues:

Summary gives an unweighted variance whereas the FEEstimator gives the weighted variance. When you collapse, the spells receive a weight equal to their length so that's why you are getting different values.
Another user had an issue with negative estimates, but with the CRE estimator. The correspondence was over email, so I'll paste my response:

Professor Lamadon said you could bound the CRE variance below at 0, but the fact that it's negative points to other issues. I personally don't think bounding it below seems like the right decision, since it is unlikely the true variance is actually 0.
He said what you should do is to run attrition plots to investigate the estimators' behaviors. I just updated the attrition notebook (also available on the github under docs/source/notebooks), so you should be able to just fill it in with the data you are investigating to generate the attrition plots (I also updated the Monte Carlo, FE, and CRE notebooks, so you may want to check out the new versions of those on the documentation site as well).

The values don't sum to the variance since they are approximations, if you want the estimates to be better then you should increase the number of trace and leverage draws.

Please let me know if this helps address your questions, and please let me know whatever new questions you may have!
Best,
Adam

Here's the code to compute the connected components:
'''
Connected components for arbitrarily many categorical control variables.
'''
import numpy as np
import pandas as pd
import bipartitepandas as bpd
from bipartitepandas.util import to_list
from scipy.sparse import csc_matrix, hstack
from scipy.sparse.linalg import svds, eigsh
from igraph import Graph

def SPxD(sp, diag):
    '''
    Faster product of a sparse matrix and a diagonal, i.e. take sp @ diag.

    Arguments:
        sp (CSC Matrix): sparse matrix
        diag (NumPy Array or float): diagonal entries or multiplicative factor

    Returns:
        (CSC Matrix): product of sparse matrix and diagonal
    '''
    if isinstance(diag, (float, int)):
        # If multiplicative factor
        if diag == 1:
            return sp
        return diag * sp

    # If diagonal entries
    return sp.multiply(diag)

def n_connected_components(bdf, categorical_controls, method='approximate', n_categories=None, null_space_dim=1, warn=2, rng=None):
    '''
    Compute the n-way connected components of a dataframe.

    Arguments:
        bdf (BipartiteDataFrame): (collapsed) long format data
        categorical_controls (list): categorical controls
        method (str): if 'approximate', use the method of Weeks & Williams (1964) - this guarantees connectedness, but does not guarantee that different subsets are disconnected; if 'exact', use the method of Godolphin & Godolphin (2001) - this guarantees both connectedness and that different subsets are disconnected
        n_categories (dict of ints or None): link each categorical variable to the number of categories it has, for computational efficiency, using 'i' for worker ids and 'j' for firm ids; None is equivalent to {}
        null_space_dim (int): used when `method='exact'`; dimension of the null space, it is recommended to slowly increment this (by increasing the value) until the singular value warning disappears; then once the function outputs the correct number of 0 singular values, set `null_space_dim` to the correct value and set `warn=0`
        warn (int): used when `method='exact'`; if 2, raise an error if all the singular values are 0; if 1 or 2, print how many singular values are 0; if 0, do nothing
        rng (np.random.Generator or None): used when `method='exact'`; NumPy random number generator; None is equivalent to np.random.default_rng(None)

    Returns:
        (list of lists): each entry of the primary list gives a list of the observations included in a particular connected component
    '''
    if rng is None:
        rng = np.random.default_rng(None)

    categorical_controls = to_list(categorical_controls)

    ## Set n_categories ##
    if n_categories is None:
        n_categories = {}
    # Make sure not to overwrite original
    n_categories = n_categories.copy()
    if 'i' not in n_categories.keys():
        n_categories['i'] = bdf.n_workers()
    if 'j' not in n_categories.keys():
        n_categories['j'] = bdf.n_firms()
    for cat_control in categorical_controls:
        if cat_control not in n_categories.keys():
            n_categories[cat_control] = bdf.n_unique_ids(cat_control)

    ## Indices for each category ##
    cat_indices = {}
    cum_cols = 0

    ## General variables ##
    n_obs = len(bdf)
    ones = np.ones(n_obs, dtype=int)
    idx = np.arange(n_obs)

    ## Create sparse matrices ##
    X1 = csc_matrix((ones, (idx, bdf.loc[:, 'j'].to_numpy())), shape=(n_obs, n_categories['j']))
    X2 = csc_matrix((ones, (idx, bdf.loc[:, 'i'].to_numpy())), shape=(n_obs, n_categories['i']))

    # Update indices
    cat_indices['j'] = (cum_cols, cum_cols + n_categories['j'])
    cum_cols += n_categories['j']

    for cat_control in categorical_controls:
        # Control columns
        Xc = csc_matrix((ones, (idx, bdf.loc[:, cat_control].to_numpy())), shape=(n_obs, n_categories[cat_control]))
        X1 = hstack([X1, Xc])

        # Update indices
        cat_indices[cat_control] = (cum_cols, cum_cols + n_categories[cat_control])
        cum_cols += n_categories[cat_control]
    del Xc

    if method == 'approximate':
        ### Weeks & Williams (1964) ###
        X = hstack([X2, X1])
        del X1, X2

        # How many groups in common are required to consider observations adjacent
        adjacency_thres = len(n_categories.keys()) - 1

        ## Adjacency matrix ##
        adjacency_matrix = ((X @ X.T) == adjacency_thres).astype(int, copy=False)

        ## Construct graph ##
        G = Graph.Adjacency(adjacency_matrix)

        ## Find connected components ##
        cc_list = sorted(G.components(mode='weak'), reverse=True, key=len)

        return cc_list

    ### Godolphin & Godolphin (2001) ###
    bdf = bdf.copy(deep=True)
    ## Information Matrix ##
    N = X1.T @ X2
    C = X1.T @ X1 - SPxD(N, 1 / np.bincount(bdf.loc[:, 'i'].to_numpy())) @ N.T
    del X1, X2, N

    ## Singular Values ##
    # svecs, svals, _ = svds(C, k=null_space_dim, which='SM', return_singular_vectors='u', random_state=rng)
    evals, evecs = eigsh(C, k=null_space_dim, sigma=1e-10, which='LM')
    svals, svecs = evals, evecs

    n_zero_svals = np.sum(np.isclose(svals, 0))
    if n_zero_svals == len(n_categories.keys()) - 1:
        # Matrix is full rank
        return [np.arange(n_obs)]
    if (warn == 2) and n_zero_svals == null_space_dim:
        raise ValueError('All singular values are 0.')
    elif warn in [1, 2]:
        print(f'{n_zero_svals} singular values are 0.')

    ## Null Space ##
    kernel_basis = svecs[:, np.nonzero(np.isclose(svals, 0))[0]]

    # # Normalize kernel basis
    # kernel_basis -= kernel_basis.min(axis=0)
    # kernel_basis = MxD(kernel_basis, 1 / kernel_basis.max(axis=0))

    # Round (don't want to consider basis vectors different if they're extremely close)
    kernel_basis = np.round(kernel_basis, 6)

    ## Connectedness ##
    fully_identified = []
    for cat_control in ['j'] + categorical_controls:
        # Connected sets for each control variable are determined by unique rows of the kernel basis
        # Source for sparse unique rows: https://stackoverflow.com/a/52891452/17333120
        control_idx = cat_indices[cat_control]
        kernel_subset = kernel_basis[control_idx[0]: control_idx[1], :]
        unique_basis_vectors, control_concomps = np.unique(kernel_subset, axis=0, return_inverse=True)
        if (len(unique_basis_vectors) == 1) and (cat_control != 'j'):
            # If entire category is identified (ignore j)
            fully_identified.append(cat_control)
        else:
            # Create new category based on connected components
            bdf.loc[:, cat_control] = control_concomps[bdf.loc[:, cat_control].to_numpy()]

            # Update n_categories
            n_categories[cat_control] = len(unique_basis_vectors)
    if len(fully_identified) > 0:
        ## If some columns are fully identified, ignore them ##
        # Make sure not to overwrite original
        categorical_controls = categorical_controls.copy()
        for col in fully_identified:
            # Delete fully identified columns
            del categorical_controls[categorical_controls.index(col)]
            del n_categories[col]

    if len(categorical_controls) == 0:
        print('All components can be found by directly computing largest set of connected workers + firms.')
        return None

    return n_connected_components(bdf=bdf, categorical_controls=categorical_controls, method='approximate', n_categories=n_categories)

##############
## Simulate ##
##############
rng = np.random.default_rng(1234)
sim_params = bpd.sim_params(
    {
        'n_workers': 500,
        'firm_size': 10,
        'p_move': 0.03
    }
)
clean_params = bpd.clean_params(
    {
        'connectedness': 'connected',
        'copy': True
    }
)
sim_data = bpd.SimBipartite(sim_params).simulate(rng)

n_control1 = 15
n_control2 = 8
sim_data['worker_control'] = rng.integers(n_control1, size=len(sim_data['i'].unique()))[sim_data['i']]
sim_data['firm_control'] = rng.integers(n_control2, size=len(sim_data['j'].unique()))[sim_data['j']]

bdf = bpd.BipartiteDataFrame(
    sim_data,
    custom_categorical_dict={'worker_control': True, 'firm_control': True},
    custom_dtype_dict={'worker_control': 'categorical', 'firm_control': 'categorical'},
    custom_how_collapse_dict={'worker_control': 'first', 'firm_control': 'first'}
).clean()

categorical_controls = ['worker_control', 'firm_control']
n_categories = {'i': bdf.n_workers(), 'j': bdf.n_firms(), 'worker_control': n_control1, 'firm_control': n_control2}

cc_list1 = n_connected_components(bdf, categorical_controls, method='approximate', n_categories=n_categories, null_space_dim=1, warn=2, rng=None)

cc_list2 = n_connected_components(bdf, categorical_controls, method='exact', n_categories=n_categories, null_space_dim=30, warn=2, rng=None)

###################
## Connected set ##
###################
bdf2 = bdf.loc[cc_list2[0], :].clean()

categorical_controls2 = ['worker_control', 'firm_control']
n_categories2 = {'i': bdf2.n_workers(), 'j': bdf2.n_firms(), 'worker_control': bdf2.n_unique_ids('worker_control'), 'firm_control': bdf2.n_unique_ids('firm_control')}

cc_list2_1 = n_connected_components(bdf2, categorical_controls2, method='approximate', n_categories=n_categories2, null_space_dim=1, warn=2, rng=None)

cc_list2_2 = n_connected_components(bdf2, categorical_controls2, method='exact', n_categories=n_categories2, null_space_dim=7, warn=1, rng=None)

Hi Adam,
Thanks a lot for the fast answer! I will take a look at the things you suggested and get back to you.
Cheers,
Simon

Hi Simon,
In case you want to check out the papers I used for the connectedness, here are the links (I had to use another paper from Godolphin to find their algorithm, since the 2001 paper isn't available online):
Weeks & Williams (1964)
Godolphin (2004)
If you choose to read them and have any insights please let me know! I was a bit lost in the Godolphin paper since they use terminology I had never seen before, so I may be misinterpreting what their algorithm does. But like I said before, when I experimented with some sample data it does seem to find connected components that are larger than those found by the Weeks & Williams algorithm.
Best,
Adam

Hi Simon,
Just following up on this.
Best,
Adam


chore: Add kit

Copy the kit directory from v2 back to v1, to help backporting.

[ ] CHANGELOG.md updated with a link to the PR (not the Issue)
[x] Well-formatted commit messages
[x] Rebased/mergeable
[x] Tests pass)

no changelog for chores

I don't think the prometheus and cli bits will be very useful, they could potentially be taken out


Fix to incorrect merge (duplicated msg calls).

I merged this, but I'm curious why you don't also log details after the liquidation


⚠ Bofrost.de has degraded performance

In b6eb3ea, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1434 ms

Resolved: Bofrost.de performance has improved in b387dfd after 45 minutes.


Making the package platform specific just like shared_preferences package

Docs: https://docs.flutter.dev/packages-and-plugins/developing-packages

@mhbdev can u elaborate wht exactly needs to be done to solve this issue and i am ready to work on this u can assign me

@mhbdev can u elaborate wht exactly needs to be done to solve this issue and i am ready to work on this u can assign me

Hi there 😊
Thanks for participating.
We are going to make this plugin a Federated plugin means:

Federated plugins are a way of splitting support for different platforms into separate packages. So, a federated plugin can use one package for iOS, another for Android, another for web, and yet another for a car (as an example of an IoT device). Among other benefits, this approach allows a domain expert to extend an existing plugin to work for the platform they know best.

The complete guide on how to do so can be found in here.
I will assign this to you as soon as I get to my desktop cause I'm on the phone right now (github app).


all-in-one: add NO_INSTALL_SEDNA variable flag

If NO_INSTALL_SEDNA not 'false', don't install Sedna, it's useful to
develop, the developer will install Sedna later by manually.
Signed-off-by: llhuii<EMAIL_ADDRESS>

/lgtm

/approve


Fix : fullScreenByDefault does not work with autoPlay #710

The first attempt to go full screen just after checking if autoPlay is on fails and prevents the second want to be done.
See bug reported here for more information https://github.com/fluttercommunity/chewie/issues/710

@maxbritto LGTM.

Do you need anything else from me until you can merge the PR ?

@maxbritto Please re-sync your changes with master's latest changes. That way, I can evaluate them before I merge this PR.
Thanks in advance.


createMuiTheme override with Typescript

Hi, i have a problem:
When I use "createMuiTheme" with typescript, I get this error:

Thanks for the reply.

Hey @LuisEdMB , I also receive this TS error, and if I ignore it the styles are not propagated to the table as desired and as shown in the examples. Any luck with a solution?

Hi @aostrander-source, I had to pass my TS code to JS 😕.


Is this project still maintained?

I love this tool and use it frequently. But it seems it is no longer maintained. There are open PR's and issues.
I would be open to taking over maintenance to keep this great project going.

The original author @theSteveMitchell let me know he currently does not have time to give this project attention.
He has been kind of enough to add me as a contributor.


Investigate Path template performance

Recent investigation by @blowmage found that GAPIC utility methods that rely on the GAX path template code are very slow, to the point that they are the bottleneck for some google-cloud-ruby acceptance tests. This is likely due to the use of rly to implement template parsing.
Investigate these performance concerns. If rly is the issue, addressable may be a good substitute, although be cautious of differences between Google's path templates and RFC 6570. Another approach might be to drop parsing and use basic string interpolation, which @pongad has already done in the Go GAPICs.

Python also has just string interpolation (with a touch of validation). It's likely fairly trivial to port the Python implementation to Ruby: https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/api_core/google/api_core/path_template.py

Python also dropped the use of a full-blown parser and its path template module is extremely small, simple, and well tested: https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/api_core/google/api_core/path_template.py
I don't think we should deprecated them, I think we should provide them as helpers to construct strings in the correct format.

@jonparrott I'd bet you'd prefer that all things follow Python's approach though 😄
I think we'll have a complication either way here. These are currently implemented as class methods (IIRC this was why @frankyn didn't like them), which I do think looks weird and often verbose. If we made them available as instance methods it would look exactly like Python, but we'd probably have to keep both around for awhile and it does introduce the (remote) possibility for things like name collisions.
I just have a hard time believing that folks really needs these things (i.e. in the pub sub docs only 3/7 languages use them). @blowmage do you think we'll get complaints if we don't have these, or have we gotten issues/complaints related to this in the past?
Let's also check with our fellow Python friend @lukesneeringer - he had a long conversation about this the other day and I'd be curious what he'd suggest we do.

@jonparrott I'd bet you'd prefer that all things follow Python's approach though 😄

Bias aside, the Python implementation was well-thought out and intentionally intended to provide a basis for other languages and was reviewed by several others (including @lukesneeringer). This actually goes for all of api core.

I just have a hard time believing that folks really needs these things (i.e. in the pub sub docs only 3/7 languages use them)

I agree. I don't think anyone needs them, but they are a bit useful to have around.

These are currently implemented as class methods (IIRC this was why @frankyn didn't like them), which I do think looks weird and often verbose. If we made them available as instance methods it would look exactly like Python, but we'd probably have to keep both around for awhile and it does introduce the (remote) possibility for things like name collisions.

I have no strong feelings one way or the other here.

do you think we'll get complaints if we don't have these, or have we gotten issues/complaints related to this in the past?

It depends on what you mean by "these". Do you mean the _path methods on the GAPIC client classes? Or the Google::Gax::PathTemplate class?

Both.
I think we have to do something about PathTemplate if it's so slow. Given that, I'm suggesting we deprecate and stop using the _path helpers as others have done rather than re-implement them.
It seems like something that's better solved by documentation rather than code, which has to be documented and explained in addition. Just to use a concrete example, what do you think of this?
https://github.com/GoogleCloudPlatform/google-cloud-ruby/blob/master/google-cloud-logging/lib/google/cloud/logging/v2/logging_service_v2_client.rb#L280
To me, dropping the helpers here would be an improvement and a useful simplification. However, I don't want to drop them if you're really sensing that folks need the helpers. Or, maybe there are more complex cases where they are genuinely useful?

If we have any GA clients using Path templates then we can not deprecate or remove them.
I'm certainly in favor of just changing the implemention. The Python one took less than a day.

I'm sure we have GA clients that use them.
The time to change is not an issue. To recap, there are two potential problems here:

poor performance
currently implemented as class methods

I'm still hung up on the second item and I've pinged @frankyn to weigh in. This issue is causing us to intentionally not use the helper methods in favor of strings in sample code. If we only consider the first issue then it's pretty simple and we could just change the internal implementation, but I'd like to find a way to solve both problems if we can. Duplicating the helpers methods is an obvious solution, but I'm not sure it's a good one.

FWIW, my thinking on helper methods in Ruby is that they are there as a convenience to the programmer. If the method is not in the most convenient place, then add it. Convenience is more important than duplication.
DATABASE_PATH_TEMPLATE = Google::Gax::PathTemplate.new(
  "projects/{project}/instances/{instance}/databases/{database}"
)

private_constant :DATABASE_PATH_TEMPLATE

# Returns a fully-qualified database resource name string.
# @param project [String]
# @param instance [String]
# @param database [String]
# @return [String]
def self.database_path project, instance, database
  DATABASE_PATH_TEMPLATE.render(
    :"project" => project,
    :"instance" => instance,
    :"database" => database
  )
end

# Returns a fully-qualified database resource name string.
# @param project [String]
# @param instance [String]
# @param database [String]
# @return [String]
def database_path project, instance, database
  self.class.database_path project, instance, database
end

Thanks @blowmage that's great input.
Let's wait to confirm w/ Frank that this will unblock the sample code issue. If that works out and you're okay with the duplication then I'm happy to push this through as outlined above.

@jbolinger As you said, there are really two issues. I believe they are orthogonal and can be solved independently. Therefore I propose we move the helper method convenience discussion to #125.

This afternoon @quartzmo and I paired on a spike of implementing the python code, to see if it was a viable option. Sadly, the approach in the python code cannot 100% replace the functionality provided by Google::Gax::PathTemplate. With this in mind, we have come to the conclusion that there are 3 major options to take:

Replace Google::Gax::PathTemplate with a new class - this should deliver a boost in performance, but will also include introducing API incompatibility. Specifically, Google::Gax::PathTemplate#match doesn't seem to be easily provided by the regexp approach.
Leave Google::Gax::PathTemplate alone, and simply stop using it for the simple templates used by most of the path helper methods. All of the path templates we have checked so far in the generated GAPIC code have used named single segments. It should be possible to use the simple ruby string interpolation proposed in an earlier comment in these cases. Any future helper method that used a template with multi segment patterns could use the slower Google::Gax::PathTemplate.
Leave Google::Gax::PathTemplate alone, introduce a separate class based on the python regexp code. This would maintain backwards compatibility of the GA's google-gax library, while allowing helper methods to switch to using a different class with increased performance.

Responding to second item in #125.
As for performance concerns, removing the underlying implementation for the Python version sounds acceptable, but we may still want to verify performance.

The python code implements a simple validation of a path string to a template. The Google::Gax::PathTemplate#match method validates a path string to the template, and returns the parameters taken from the path that look like they could be sent to Google::Gax::PathTemplate#render to generate the path again. I'm not sure the regexp approach will enable this behavior.
I say "look like they could be sent to", since we also noticed some unexpected behavior in #match, which is detailed in #126.
I find the behavior of Google::Gax::PathTemplate#match to be surprising in other ways as well. If the path doesn't match the method raises an error. I expected it would behave similar to Regexp#match, which will return nil when it doesn't match instead of raising.

I might be thinking about this too naively, but if you have a/{b}/c/{d}, isn't it just regexp a/[^/]*/c/[^/]*? You could use capture groups to take out the params right? I also resonded to #126.

I suspect that one of the reasons this gem has a dependency on the rly gem is because it needs a full lexer to do lookahead. I mean, I don't know that for sure, but that is what I suspect.
Can you use capture groups for a template like v1/*/{name=parent/**}?

We found in python that while there was some utility in validating and
expanding path template, literally no one was parsing them.
I would offer up that we could drop the parsing feature.
On Fri, May 4, 2018, 4:50 PM Mike Moore<EMAIL_ADDRESS>wrote:

I suspect that one of the reasons this gem has a dependency on the rly
gem is because it needs a full lexer to do lookahead. I mean, I don't know
that for sure, but that is what I suspect.
Can you use capture groups for a template like v1/*/{name=parent/**}?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/googleapis/gax-ruby/issues/119#issuecomment-386762030,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAPUc4Q_BUBerZQuopA2radF-SHOMH7gks5tvOk9gaJpZM4SUGCd
.

I would be cool dropping features, ala option #1, but the gem already GA, so API compatibility would be a consideration IMO.

I can't comment on how disruptive things would be if we drop the feature, but the capture group for that template should be like v1/([^/]*)/(parent/.*) right?
I don't think any of these template actually needs full-featured LALR parsers. I think lex/yacc was used to parse the path template (v1/*/{name=parent/**}, etc) themselves, not the strings that our generated functions are supposed to match. I think it's possible to turn the path template into a regexp while generating the client, though I haven't thought much about it.

My understanding is this issue is still in progress.
(Replying to bring the issue back into compliance.)

@jbolinger Any updates?

I'm going to close this. The plan is to rewrite path template handling so there is no need to examine the performance any more than we already have.


update kubespray packer build, parametize docker tag

This packer build uses ansible and kubespray to build a base image (AMI) for use in a kubernetes cluster. The AMI improves provisioning new clusters and new nodes in existing clusters, and this improves the operator's deployment UX. As this is run on CoreOS, the packer build uses a docker image to provide ansible and kubespray. The current build uses (implicitly) the docker image's latest tag, and it's not possible for an operator to choose the release (of the docker image) they want Packer to use when running the build.
By making this parameter available to the operator, we can more easily test X or Y version/release of kubespray.
Code references:

https://github.com/fpco/terraform-aws-foundation/blob/77fb317260b30c721feca2c98f61cc12e22836b8/packer/kubespray-base-ami/packer.json#L45
https://github.com/fpco/terraform-aws-foundation/blob/77fb317260b30c721feca2c98f61cc12e22836b8/packer/kubespray-base-ami/run.sh#L14

This is resolved, great work!


Not working with React event handlers

Hi,
I'm using pjax along with some React components. I'm trying to disable some links programmatically, depending on the current state of the component, by using onClick/onClickCapture event handlers along with some stopPropagation and preventDefault methods. But this is not working as expected, as the links are still enabled.
Here is a small example to reproduce:

index.html

<!doctype html>

    <meta charset="utf-8">
    <title>Test pjax</title>

Go to /foo without React

foo.html

<!doctype html>

    <meta charset="utf-8">
    <title>Test pjax</title>

    This page should not have been reached

Tested with the latest versions of Firefox and Chrome.

From what I understand, this is because React's SyntheticEvents are always processed after native events, which are the one used by Pjax.
A quick test shows indeed that the isDefaultPrevented function returns false after the onClick handler is being called, but this is already too late.
The solution is... don't mix native events with React events, which means using a React-compatible version of pjax. Unfortunately for me, this won't be possible as my app is not a full-React app, but just isolated components here and there.
Anyway, if someone still manages to find a workaround for this particular case, I'd be happy to see it!


Import Django's slugify() from documented django.utils.text

https://docs.djangoproject.com/en/2.1/ref/utils/#django.utils.text.slugify
No need to involve template code for a simple text transformation.

Codecov Report

Merging #569 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master   #569   +/-   ##
=====================================
  Coverage      94%    94%
=====================================
  Files           4      4
  Lines         434    434
  Branches       77     77
=====================================
  Hits          408    408
  Misses         16     16
  Partials       10     10

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 20ab1c3...259d352. Read the comment docs.


[Question]: How to speed up index loading time

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question
In actual use, I found that the process of loading the index takes a long time, which greatly affects the real-time performance of the retrieval. Is there any way to speed it up? Or can it be converted into other more efficient storage methods (faiss, pgvector)?

@SCUT-ChenBD I would use a db integration to speed this up for sure. Qdrant and Chroma are easy to get started with

To load multiple indices from storage
indices = load_indices_from_storage(storage_context, index_ids=['index_id1', 'index_id2'])
how to use this indices for quary engin?
chat_engine = indices.as_chat_engine( )
this not work and get error


Migrate wasm-bindgen to using walrus

This commit moves wasm-bindgen the CLI tool from internally using
parity-wasm for wasm parsing/serialization to instead use walrus.
The walrus crate is something we've been working on recently with an
aim to replace the usage of parity-wasm in wasm-bindgen to make the
current CLI tool more maintainable as well as more future-proof.
The walrus crate provides a much nicer AST to work with as well as a
structured Module, whereas parity-wasm provides a very raw interface
to the wasm module which isn't really appropriate for our use case. The
many transformations and tweaks that wasm-bindgen does have a huge
amount of ad-hoc index management to carefully craft a final wasm
binary, but this is all entirely taken care for us with the walrus
crate.
Additionally, wasm-bindgen will ingest and rewrite the wasm file,
often changing the binary offsets of functions. Eventually with DWARF
debug information we'll need to be sure to preserve the debug
information throughout the transformations that wasm-bindgen does
today. This is practically impossible to do with the parity-wasm
architecture, but walrus was designed from the get-go to solve this
problem transparently in the walrus crate itself. (it doesn't today,
but this is planned work)
It is the intention that this does not end up regressing any
wasm-bindgen use cases, neither in functionality or in speed. As a
large change and refactoring, however, it's likely that at least
something will arise! We'll want to continue to remain vigilant to any
issues that come up with this commit.
Note that the gc crate has been deleted as part of this change, as the
gc crate is no longer necessary since walrus does it automatically.
Additionally the gc crate was one of the main problems with preserving
debug information as it often deletes wasm items!
Finally, this also starts moving crates to the 2018 edition where
necessary since walrus requires the 2018 edition, and in general it's
more pleasant to work within the 2018 edition!

+990 −3,531

Nice, looks like this is actually paying off :)


cc-wrapper: handle -nostdinc{++,}

Motivation for this change
Fixes #17769.
Things done

[x] Tested using sandboxing
(nix.useChroot on NixOS,
or option build-use-chroot in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] OS X
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

cc @copumpkin

I've tested that this works well on a slightly upgraded Darwin stdenv (and built vim and git with it), but not on the mainline one. It should otherwise be almost identical to the usual Darwin stdenv so I don't expect this to break Darwin.
It seems pretty safe to merge to me, since @abbradar has tested on NixOS.

I'll merge it in several days unless there are problems found.

Can we merge now? I see very little chance for anything going wrong, and we can revert if something does go wrong. It's not as if sitting quietly in this PR will cause a real major test to happen.

I planned to merge this along with Python patches, printing updates and font rendering updates. They seem to cover separate enough areas, so we can ease load on Hydra a bit. Among those only font updates are not fully ready and i plan to finish them today.
Nikolay.
Sent from a mobile device, please excuse my brevity.

Ah, okay. I just need this to continue my work on improving the darwin stdenv, but I can wait a bit.

On the second thought, Hydra will already have a staging rebuild soon because of master merge with a security update. Let's merge Python, printing and this one now and postpone font rendering until it's done. I can do it in an hour or you can press the buttons before me ~_^
Nikolay.
Sent from a mobile device, please excuse my brevity.

Very happy to press buttons. I have many years of practice!

Seems like I still have the problem in a nix-shell with llvm_4/clang
I am trying to compile some out of tree ebpf extension but even with -stdinc, glibc headers (and zlib are still picked up).  search starts here:
#include <...> search starts here:
 /home/teto/mptcp/build/include
 /home/teto/mptcp/build/arch/x86/include/generated
 /nix/store/bknhdy2d2919ni794mnr7hy0warkwydl-llvm-4.0.1/include
 /nix/store/i1lyv0v01z1ivndra433kvmaagfhlmym-ncurses-6.1-dev/include
 /nix/store/ggdq1si53c6qb0b67lgx145xkndfy4zy-zlib-1.2.11-dev/include
 /nix/store/pi0lr4cj88vig3s5bwl6dpnrvvpc9css-compiler-rt-5.0.2-dev/include
 /nix/store/z10kkr081zb02kyjw8ybf2s9zll49cvx-glibc-2.27-dev/include
End of search list.


Correct type hint

Hey @juliangums, just tested and I can see each line causes a syntax error:
protected Arrayable|Closure|string $descriptions = [];

Cannot use array as default value for property JaOcero\RadioDeck\Forms\Components\RadioDeck::$descriptions of type Illuminate\Contracts\Support\Arrayable|Closure|string

You should extend the union type back to how it was originally, by adding |array:
protected Arrayable|Closure|string|array $descriptions = [];

@JackWH can you check again?

Merge @juliangums @JackWH yeah sorry I didn't check the last PR I thought it only removes the facade but some data type are remove. Can you check it again @JackWH ?


[SPARK-33636][PYTHON][ML][3.0] Add labelsArray to PySpark StringIndexer

What changes were proposed in this pull request?

This is a followup to add missing labelsArray to PySpark StringIndexer.
Why are the changes needed?

labelsArray is for multi-column case for StringIndexer. We should provide this accessor at PySpark side too.
Does this PR introduce any user-facing change?

Yes, labelsArray was missing in PySpark StringIndexer in Spark 3.0.
How was this patch tested?

Unit test.

cc @srowen

cc @HyukjinKwon too.

Test build #132080 has started for PR 30580 at commit f5faca4.

LGTM but I will leave it to @srowen.

Test build #132080 has finished for PR 30580 at commit f5faca4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36680/

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132080/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36680/

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/36680/

Thanks. Then I will merge this to branch-3.0.


Develop an approach for settings navigation, changes, restoration, etc.

Once sync is in place some consideration needs to be given to how changes are made, saved and restored.

When a config option is changed from the UI, app->selected_menu_options will be updated. Maintain this behaviour.
Pressing enter on an option will save that option (back to esp32-gravity)
Add a new option to revert settings, restoring them from sync()
Display confirmation of saving using a modal dialogue

The above doesn't feel quite right. Get this right and then build it. Also create a new issue to save and load named configurations to and from Flipper

Item (2) in place, however also needs to update Flipper menu item(s)


layers: xattr handling is not well-specified

So, while implementing xattr support in umoci (cyphar/umoci#54) I discovered that it's not actually clear how to handle unpacking of attributes. In particular, it's not very clear what we should be doing when an extended attribute is defined for a file that has had its metadata modified.
Imagine you have a file foo which had the user.test xattr set. In the next layer, this xattr is missing but there is some other xattrs set. What should an extractor do:

Should we always ensure that the xattrs of a file match exactly the same xattrs of the layer (meaning that user.test will be removed).

Should we leave existing xattrs alone, so user.test will not be removed?

Should we only remove xattrs that were added explicitly by the extractor from a previous layer (so user.test will be removed but something like security.selinux that is not included in layers will not)?

Each of these solutions has a problem that we need to address:

This will cause issues with stuff like security.selinux which is automatically set on all created files based on your current context, and you cannot remove them.

This will mean that you cannot effectively delete an xattr, because there's no way of specifying an xattr whiteout.

This sounds like the most sane method. The problem is that it's a bit of a burden on implementors to have to keep track of what xattrs they've set on every file in a filesystem. In addition, how do you handle the case where a file already had some xattr set (which you didn't set) and then you're told to modify it. Should you remove it if it disappears in a future layer or not?

/cc @vbatts

Oh, in addition, we need to specify whether or not all xattrs should be included in a layer changeset. IMO we shouldn't be including security.selinux -- but some other xattrs might exist that cause us even more issues. My current solution (cyphar/umoci#54) is to just blacklist security.selinux pending further issues.

On Fri, Dec 16, 2016 at 10:52:04PM -0800, Aleksa Sarai wrote:

Should we always ensure that the xattrs of a file match exactly
the same xattrs of the layer (meaning that user.test will be
removed).

This is what the current spec calls for 1.  Here's GNU tar doing
just that:
$ tar --version
tar (GNU tar) 1.28
…
$ mkdir a
$ echo foo >a/test
$ setfattr -n user.foo -v foo a/test
$ mkdir b
$ echo bar >b/test
$ setfattr -n user.bar -v bar b/test
$ tar -cf test.tar -C b --xattrs .
$ tar -tf test.tar --xattrs -vv
drwxr-xr-x  wking/wking       0 2016-12-17 07:25 ./
-rw-r--r--* wking/wking       4 2016-12-17 07:25 ./test
x: 3 user.bar
$ strace -o /tmp/trace tar -xf test.tar -C a --xattrs
$ grep ./test /tmp/trace
mknodat(4, "./test", 0644)              = -1 EEXIST (File exists)
unlinkat(4, "./test", 0)                = 0
mknodat(4, "./test", 0644)              = 0
setxattr("/proc/self/fd/4/./test", "user.bar", "bar", 3, 0) = 0
openat(4, "./test", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK|O_CLOEXEC, 0644) = 5
$ getfattr -d a/test
file: a/test
user.bar="bar"
So the layer author has no way to pass through pre-existing xattrs,
but if you know what xattrs you want, you can always add a trailing
layer to restore anything that you don't want clobbered.

This will cause issues with stuff like security.selinux which
is automatically set on all created files based on your current
context, and you cannot remove them.

For what it's worth (probably not much with such a fuzzy connection to
our spec), GNU tar distinguishes between selinux, ACLs, and other
xattrs 2.  And there seem to be plans to eventually split out
security.capability as well 3.

@wking

Should we always ensure that the xattrs of a file match exactly
the same xattrs of the layer (meaning that user.test will be
removed).
This is what the current spec calls for [1].  Here's GNU tar doing
just that:

No it isn't. Because doing an unlink and then adding xattrs will still mean you have a security.selinux xattr set on the file -- that's how SELinux works. But "match exactly" would require removing security.selinux if it's not set in the tar archive.
And it still doesn't help with layer generation -- how do we know if we should include certain xattrs.

For what it's worth (probably not much with such a fuzzy connection to
our spec), GNU tar distinguishes between selinux, ACLs, and other
xattrs [2].  And there seem to be plans to eventually split out
security.capability as well [3].

You're right, it doesn't help to point to the implementation of GNU tar in order to answer a simple question of "what should we do with xattrs".

On Sat, Dec 17, 2016 at 07:52:48AM -0800, Aleksa Sarai wrote:

Should we always ensure that the xattrs of a file match exactly
the same xattrs of the layer (meaning that user.test will be
removed).
This is what the current spec calls for [1].  Here's GNU tar doing
just that:

No it isn't. Because doing an unlink and then adding xattrs will
still mean you have a security.selinux xattr set on the file --
that's how SELinux works. But "match exactly" would require
removing security.selinux if it's not set in the tar archive.

Fair.  The current spec calls for unlinking any pre-existing file,
that's what GNU tar does, and that's what we should do.  If you get
SELinux settings by doing that, you don't have to clear them.  I don't
think we need additional wording to cover the xattrs implications of
the already-specified unlink-and-replace wording.

And it still doesn't help with layer generation -- how do we know if
we should include certain xattrs.

Add a flag to your layer generator (like GNU tar does with --xattrs
and friends)?  I don't think there's a good way to make this call in
general.

For what it's worth (probably not much with such a fuzzy
connection to our spec), GNU tar distinguishes between selinux,
ACLs, and other xattrs [2].  And there seem to be plans to
eventually split out security.capability as well [3].

You're right, it doesn't help to point to the implementation of GNU
tar in order to answer a simple question of "what should we do with
xattrs".

It doesn't help clarify the current spec's stance, but it can inform
future spec changes.  They presumably had reasons for breaking down
xattrs into subcategories, and maybe those reasons apply to us too.

Add a flag to your layer generator (like GNU tar does with --xattrs
and friends)?  I don't think there's a good way to make this call in
general.

The spec should at least provide some guidance. For example:

user.*

Those are the three that come to mind immediately. 2 is because otherwise ping will be broken everywhere except in Ubuntu (where ping is setuid).

This sounds like the most sane method.

agreed

The problem is that it's a bit of a burden on implementors to have to keep track of what xattrs they've set on every file in a filesystem.

I don't understand why they need to track this so specifically - can't they just naively apply the xattrs at every layer? (e.g using the remove/recreate method). Tying into the next point - are there "implicit" sets that would happen on some layers but not on others?

In addition, how do you handle the case where a file already had some xattr set (which you didn't set)

Who else set it? Isn't this a file tree that we (the implementer) are rendering from scratch? Do you have any examples of cases other than the selinux one?

The problem is that it's a bit of a burden on implementors to have to keep track of what xattrs they've set on every file in a filesystem.
I don't understand why they need to track this so specifically - can't they just naively apply the xattrs at every layer? (e.g using the remove/recreate method). Tying into the next point - are there "implicit" sets that would happen on some layers but not on others?

So do the removal and re-creation for every file in an image, even if the content hasn't changed? I mean ... that could work fine (bit inefficient, but what can you do).0 The problem is not extraction but with generation -- how do I know which xattrs should I include in the next layer? Which ones did users set and which ones were implicitly set by the system? The way umoci solves this problem is by generating a manifest of the rootfs after extraction -- and then you can be sure that any changes were made by the user.

Who else set it? Isn't this a file tree that we (the implementer) are rendering from scratch? Do you have any examples of cases other than the selinux one?

Some of the trusted.* extended attributes in principle could be set by some super-root user, and if you run the tooling as root then you might see xattrs that were not part of the image. I'm not really sure.

So do the removal and re-creation for every file in an image, even if the content hasn't changed?

well, isn't the only way to represent an xattr change between layers by having a new copy of the file in the upper layer anyway?

The problem is not extraction but with generation -- how do I know which xattrs should I include in the next layer? Which ones did users set and which ones were implicitly set by the system? The way umoci solves this problem is by generating a manifest of the rootfs after extraction -- and then you can be sure that any changes were made by the user.

Ah, now I get you. Yeah, I don't have a great answer (or at least one better than umoci's).

well, isn't the only way to represent an xattr change between layers by having a new copy of the file in the upper layer anyway?

Sure, but implementations could optimise disk access by storing the checksums and then computing the checksum of tar as they read the data. If the checksum is the same, there's no need to update it. It's a small optimisation, but I'm just wondering if there's a way to define changes without cutting out the possibility for such optimisations.

omg... I think by having this discussion we're already wasting more than this optimisation would ever save :-)

@jonboulle Micro optimisations are where it's at. :smile_cat: In any case, ultimately I think this should up to the runtime since the unpacking document already has the unlink+create wording. So I'll close this.


Protein PTM output

I wrote some code yesterday to output the number of sites on each protein where a given PTM occurs. I think adding that output to MetaMorpheus would be a fast, easy way for users to visualize the PTM sites observed for each protein. I'll clean up the code and do a pull request sometime, if you think it would be worthwhile?

Zach, please coordinate this with Rob, since this feature would be closely intertwined with his parsimony code. It sounds useful, but I need more detail to understand the visualization part better!

It could also be useful to note where coverage wasn't observed, and thus where PTM sites could not be observed -- clearly annotating missing information.

How about something like this?
mgkgtpsfgkrhnkshtlcnrcgrrSFHVQKKTCSSCGYPAAKtrsynwgakakrrHTTGTGRmrylkhvsrrFKN[Deamidation of N]GFQTGSASKasa
lowercase = not observed residue
uppercase = observed residue
brackets = mod at that location, same notation as PSM output

I like Rob's suggestion, but that can still be cumbersome for a user to wade through to find differences in observed PTMs between different samples/conditions. I think we would still benefit by implementing the PTMs with the observed residues to provide an overview of what PTMs were found, but perhaps also have an additional column with an output like:
aa76v:[Deamidation of N] | aa101:[Acetylation] | etc.
Where:
"aa#" is position of PTM
"v:" signifies that the given residue was observed both with and without that PTM
"[*]" is the mod at that location, same notation as PSM output

You could put an estimated occupancy ratio (e.g. by PSM count) inside the brackets. xxxxxxxXXXX[mod1|info:occupancy=1]XXXX[mod2|info:occupancy=0.25][mod3|info:occupancy=0.5]xxXXXXXXXXX
In this example,
Mod1 is observed in all PSMs
Mod2 and mod3 are observed at the same site, which is unmodified 25% (1-0.25-0.5) of the time
Mod2 is observed 25% of the time
Mod3 is observed 50% of the time

I like the occupancy idea. It would still be nice to have a list of modifications with their indexes, though, so that you can easily compare what PTMs were observed where without having to scan through each protein.
I was thinking about occupancy last night, and if we're estimating occupancy by PSMs, then might it be useful to include the number of modified PSMs observed over the total? Rather than put "occupancy=0.5", have something like "occupancy=0.5 (2/4)", where two modified and two unmodified PSMs were detected. I'm worried that it might start to look cluttered, but I think that's valuable information for determining the accuracy of that ratio. For example, I would have greater confidence that all of the proteins possessed a given modification if it read "occupancy=1 (6/6)" rather than "occupancy=1 (1/1)". Likewise, the uncertainty in the occupancy for "occupancy=0.5 (1/2)" is much greater than for "occupancy=0.5 (5/10)". This would also provide a rough quantification when users are scanning the output.

Here's the CTDP document on proteoform nomenclature. You could use this format for annotating the PTM information. https://docs.google.com/document/d/1SpAQR8aPc2cCXXSjUobg_VXC85br1WukBKNZh5TPaQ8/edit#

What does Occupancy "occupancy=1 (6/6)" mean?  If there is an example show me the idea of how to calculate, that will be great.

it would be that out of 6 PSMs detected for the peptide base sequence, all 6 have that modification. so for example you observe these PSMs:
PEPTIDE[mod1]
PEPTIDE[mod1]
PEP[mod2]TIDE[mod1]
PEPTIDE[mod1]
you could say [mod1:occupancy=1 (4/4)] [mod2:occupancy=0.25(1/4)]

Done in https://github.com/smith-chem-wisc/MetaMorpheus/pull/359


Enforce FS for secret tree

Make sure encryption_secret is deleted as soon as the secret tree is created.

I think it would be a good idea to use secrets::Secret for keys/secrets then?
For reference: https://docs.rs/secrets/0.11.1/secrets/struct.Secret.html

Not sure if that crate is really what we need. Looks like this is doing a lot we don't need.


elm install freeze

elm install freeze and high CPU
Using following bash script I have tried to install all Elm packages in a project but after about 50 packages are installed, installation process stops and elm cpu usage is over 99% without progress.

#!/usr/bin/env bash

packages=`curl -s https://package.elm-lang.org/search.json | jq -r '.[].name'`

for item in ${packages[@]}; do
  echo Y | elm install "$item"
done

Elm: 0.19.0
Browser: none
Operating System: Ubuntu 18.04.2 LTS

Additional Details
Also I get the following error message:
-- CORRUPT JSON ----------------------------------------------------------------

The elm.json for tesk9/accessible-html-with-css 1.0.1 got corrupted somehow.

I removed it from my file cache, so if it was some transient error it should be
fixed if you try the same thing again. Please report this if it seems like an
Elm problem though!

I do not understand the goal of this script. I also suspect that it is not possible to install all the packages that exist at once because they are unlikely to all have constraints that match.
So I suspect the high CPU may be that it is expensive to exhaustively search all the different constraints that exist once you hit a package that can never fit with the others. That means downloading a ton of things from the servers (not ideal at all!) and then exploring an extremely large search space locally.
Given the information available, I think this is the best explanation I can give at this time. If you can reproduce this sort of thing in a way that is clearly reproducible (simple steps that cause an error every time) you are welcome to open a new issue about that!

It may also be worthwhile to ask around on https://discourse.elm-lang.org/ or http://elmlang.herokuapp.com/ to get help with your particular goals. At this time it is hard to tell if it is a bug in the compiler, and the folks there can help!

The goal of the script is to have all documentations in .cache folder and extract them for using in a vscode Elm extension. At the time there is no general tool for extension developers to pass a keyword and get information about that (elm-oracle is obsolete) and for code analysis, elm-analyse doesn't work in offline mode. There is a good extension for Elm in Atom IDE but the good one for vscode had no progress after 0.19.0 release (probably because of elm-oracle). So the original problem is: there is no extension for vscode to program Elm in offline mode as good as the Atom one, so I'm moving to Atom just because of this, while vscode is superior IMO.

Funfact currently the intelliJ implementation is king.
I'm working on something for vscode (or in fact all IDEs) - but it's done when it's done.
There is this repo that should have all docs
https://github.com/drathier/elm-offline
And https://github.com/dmy/elm-doc-preview to view it
Not sure if it is like you imagined, but I thought it might be nice to mention it.

It is possible to download the docs.json files directly from links like this, though I would ask that whatever tool you make does not have everyone download everything.
It'd be better to have it go through servers you run, so the server costs of the design are not passed along. You can also have folks download only as needed from your servers, to reduce the load even more.


[LiveComponent] Easily stopping propagation for LiveActions within LiveActions

Hi guys,
As always thanks for this great technology, I have been using it for a few years now and I really love it.
There is one thing I could not figure out how to do "natively": how to stop the event propagation of a live action.
Imagine you have a button with a live action (on click) within a bigger div with a live action on click as well:
<div
       data-action="click->live#action"
       data-live-action-param="doDivThing"
>
       Title
       <button
                data-action="live#action"
                data-live-action-param="doButtonThing"
        >Click me</button>
</div>

In this situation, clicking the button would trigger both "doButtonThing" and "doDivThing" actions, but in most cases we'd want the button to just trigger its own action. We then need to prevent the propagation of the click to the parent div.
We have the live#action:prevent to stop the default behavior of a form submission button for example, but it does not work for this purpose (or at least it did not work for me, let me know if it should).
Since I can see this use case being a very common occurrence, my proposition is to add a similar keyword to stop propagation, which would look something as such: live#action:stop-propagation.
What do you guys think? (Please let me know if something like this already exists and I missed it.)
For those looking to solve this in the current version, I came up with a work around by making a very simple "stop_propagation_controller.js" stimulus controller that I call whenever needed:
import { Controller } from '@hotwired/stimulus'

export default class extends Controller {
    stop(event) {
      event.stopPropagation();
    }
}

And the code simply looks like this:
<div
        data-action="click->live#action"
        data-live-action-param="doDivThing"
>
        Title
        <button
                data-action="stop-propagation#stop live#action"
                data-live-action-param="doButtonThing"
        >Click me</button>
</div>

Closing this as I found this solution already existed on the stimulus side:
data-action="live#action:stop"


[Audit_RV]Bug: Checking of usable account balance is insufficient

Usable balance of an account should be the free balance minus the frozen/locked balance.
Description
https://github.com/galacticcouncil/HydraDX-node/blob/20c8ac6079b8b3d6f16a9f15e9e43353ea6b9abc/pallets/omnipool/src/lib.rs#L1048
The quoted code above does not implement the correct logic of checking usable balance of Omnipool account.
To be consistent, it should be using ensure_can_withdraw() to check the usable balance.
Expected Behavior
It should have no effect to the function behaviour. Sufficient balance before transfer is guaranteed by MultiCurrency implementation of transfer()function.
Context
Audit by Runtime Verification

this is not going to be refactor and changed to use better balance check. IT would require significant effort to change to more robust balance check.
it has not impact on behavior as mentioned in the description and it is likely rare case when user submits a transaction with insufficient balance which would still pass this condition but fail on transfer.


Failed to install addon

im onHA
Core 2024.9.2
Supervisor 2024.09.1
Operating System13.1
Frontend 20240909.1
The command '/bin/bash -o pipefail -c apk add --no-cache --virtual .build-dependencies build-base=0.5-r3 cargo=1.71.1-r0 git=2.40.1-r0 protobuf-dev=3.21.12-r2 pulseaudio-dev=16.1-r10 && apk add --no-cache pulseaudio=16.1-r10 && apk add --no-cache librespot --repository=https://dl-cdn.alpinelinux.org/alpine/edge/testing && apk del --no-cache --purge .build-dependencies && rm -fr /tmp/* ~/.cargo /usr/.crates.toml /usr/.crates2.json' returned a non-zero code: 2

Got the same error a few minutes ago...

I did too

I edited the Dockerfile and now it works if you add my Fork as a repo in add-ons and install it from there:
https://github.com/Sciddie/addon-snapserver
It's dirty and I won't continue updating the fork, but for now it should work as expected.
Please write if you find any upcomming errors. I will fix them, as long as no official new Version comes.

yeah i did some updates of my own to get it to work - with the current versions that work with each other as well as I updated the base image