Add integration tests for the RAG pipeline covering OpenAI and Bedrock

Description
[Describe what this change achieves]
Issues Resolved
[List any issues this PR will resolve]
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Also fixes https://github.com/opensearch-project/ml-commons/issues/2169

@dhrubo-os @ylwu-amzn can you guys take a look?  Thanks!

> Task :opensearch-ml-search-processors:compileTestJava FAILED
/__w/ml-commons/ml-commons/search-processors/src/test/java/org/opensearch/searchpipelines/questionanswering/generative/GenerativeQAResponseProcessorTests.java:470: error: no suitable constructor found for GenerativeQAParameters(String,String,String,int,<null>,<null>)
        GenerativeQAParameters params = new GenerativeQAParameters("12345", "llm_model", "You are kind.", contextSize, null, null);

@ylwu-amzn Can you take a look again?

2 minor comments, approving the PR.

REPRODUCE WITH: gradlew ':opensearch-ml-plugin:test' --tests "org.opensearch.ml.action.prediction.PredictionITTests.testPredictionWithEmptyDataset_KMeans" -Dtests.seed=9E0AF175677EECFB -Dtests.security.manager=false -Dtests.locale=de-AT -Dtests.timezone=AGT -Druntime.java=21
org.opensearch.ml.action.prediction.PredictionITTests > testPredictionWithEmptyDataset_KMeans FAILED
    java.lang.AssertionError:
    Expected: (an instance of java.lang.IllegalArgumentException and exception with message a string containing "No document found")
         but: an instance of java.lang.IllegalArgumentException <org.junit.internal.runners.model.MultipleFailureException: There were 2 errors:
      java.lang.IllegalArgumentException(No document found)
      java.lang.AssertionError(Shard [.plugins-ml-config][0] is still locked after 5 sec waiting)> is a org.junit.internal.runners.model.MultipleFailureException
    Stacktrace was: java.lang.IllegalArgumentException: No document found
    	at org.opensearch.ml.engine.indices.MLInputDatasetHandler.lambda$parseSearchQueryInput$0(MLInputDatasetHandler.java:57)

I don't think that's related to my change.

@ylwu-amzn can you merge this?

backport PRs are failing. Could you please check? @austintlee

https://github.com/opensearch-project/ml-commons/pull/2235 @dhrubo-os does this look ok?  The diff doesn't look right to me.

git worktree add .worktrees/backport-2.13 2.13 is resulting in fatal: invalid reference: 2.13.

oh it seems 2.13 has not been created?!

For 2.x, I get this message:
There isn’t anything to compare.
opensearch-project:2.x is up to date with all commits from austintlee:backport/backport-2213-to-2.x. Try [switching the base](https://github.com/opensearch-project/ml-commons/compare/austintlee:backport/backport-2213-to-2.x...2.x) for your comparison.

From this workflow log: https://github.com/opensearch-project/ml-commons/actions/runs/8348785119/job/22851492548
Looks like backport is failing for :
https://github.com/opensearch-project/ml-commons/actions/runs/8348785119/job/22851492548

Did you take all the updates from the branch?


Temporarily excluding mllib from main image

Temporarily excluding MLlib from main image due to the following CVEs:

CVE-2022-25168
CVE-2022-42889

Note that MLlib will still be present in the seldonio/mlserver:X.X.X-mllib runtime-specific image.

Nice one @agm, shall we add to the description

Fixes https://github.com/SeldonIO/MLServer/issues/807

such that it will close https://github.com/SeldonIO/MLServer/issues/807


OpenSearch support

I am a huge fan of searching different sites straight out of Firefox (especially in combination with the customizable keywords).
Unfortunately it's currently unavailable. If I understood this blog entry correctly, this is handled by OpenSearch, so all that would be necessary to do is add the XML file.
If someone knows all the values to put in there it would be really fast for them to do, otherwise I can look into it myself and write a PR eventually.

Duplicate of #648


New version: Twinkstar.TwinkstarBrowser version 10.0.1000.2411

Created by 🥟 Dumplings in workflow run #9537.

Log
Updated: 9.3.1000.2409 → 10.0.1000.2411
No ReleaseNotes (zh-CN) for version 10.0.1000.2411
Submitting WinGet manifests
Creating manifests
Uploading manifests and making commits
Creating a pull request

Microsoft Reviewers: Open in CodeFlow

Automatic Validation ended with:

Installation failed with exit code -2145844845
2024-11-13 23:22:41.581 [CORE] DeliveryOptimization error: 0x80190193, extended error: 0x00000000
2024-11-13 23:22:41.638 [FAIL] D:\a_work\1\s\external\pkg\src\AppInstallerCommonCore\DODownloader.cpp(289)\WindowsPackageManager.dll!00007FFEDD5BA465: (caller: 00007FFEDD5BAD01) Exception(1) tid(1c4) 80190193 Forbidden (403).
2024-11-13 23:23:13.713 [CORE] Download request failed. Returned status: 403
2024-11-13 23:23:13.713 [FAIL] D:\a_work\1\s\external\pkg\src\AppInstallerCommonCore\Downloader.cpp(148)\WindowsPackageManager.dll!00007FFEDD5B43D6: (caller: 00007FFEDD5B5C49) Exception(2) tid(1c4) 80190193 Forbidden (403).
2024-11-13 23:23:13.714 [CLI ] Failed to download, waiting a bit and retry. Url: https://d1.twinkstar.com/win/Twinkstar_v10.0.1000.2411_Release2.exe
2024-11-13 23:23:15.401 [CORE] DeliveryOptimization error: 0x80190193, extended error: 0x00000000
2024-11-13 23:23:15.401 [FAIL] D:\a_work\1\s\external\pkg\src\AppInstallerCommonCore\DODownloader.cpp(289)\WindowsPackageManager.dll!00007FFEDD5BA465: (caller: 00007FFEDD5BAD01) Exception(3) tid(1c4) 80190193 Forbidden (403).
2024-11-13 23:23:15.708 [CORE] Download request failed. Returned status: 403
2024-11-13 23:23:15.708 [FAIL] D:\a_work\1\s\external\pkg\src\AppInstallerCommonCore\Downloader.cpp(148)\WindowsPackageManager.dll!00007FFEDD5B43D6: (caller: 00007FFEDD5B5C49) Exception(4) tid(1c4) 80190193 Forbidden (403).

Automated error analysis suggests -2145844845 may mean Forbidden (403).
Automated error analysis suggests 80190193 may mean Forbidden (403)
(Automated response - build 897.)


[FormRecognizer] re-enable Copy Model live tests after the change to make expirationDateTimeTicks is in Production

currently the test will fail in production because our core library cannot handle 64 bit integers in serialization/deserialization.

Related: need to create recorded test and make sure to clean it up #9214


bin function end, some statement can be a function && others.

bin function end, some statement can be a function.

https://github.com/statcompute/py_mob/blob/cda01475df173d8f8e003b49636fe818581c0b08/py_mob/py_mob.py#L293-L309

Nice.

https://github.com/statcompute/py_mob/blob/cda01475df173d8f8e003b49636fe818581c0b08/py_mob/py_mob.py#L189-L190

trying to run data.

BTW, the new bin function(kmn_bin、gbm_bin) it is great.

I agree. the rule parsing is not compact. let me think about it and see  if it can be wrapped into a function.

generating rules has been wrapped in a function in the  updated version. thanks for the suggestion.


🔖 Bump version

Codecov Report

Merging #54 (57ab21d) into master (a12f95d) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #54   +/-   ##
=======================================
  Coverage   96.09%   96.09%
=======================================
  Files           8        8
  Lines         563      563
=======================================
  Hits          541      541
  Misses         22       22

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a12f95d...57ab21d. Read the comment docs.


Add support for Role Tags

Pull Request Etiquette

[x] I have checked the PRs for upcoming features/bug fixes.
[x] I have read the contributing guidelines.

Changes

[x] Internal code
[x] Library interface (affecting end-user code)
[ ] Documentation
[ ] Other: _____

Description
Discord recently added "tags" object to Roles.
Tags appear for any of the following reasons:

To indicate a Server Booster role, example

"tags": {
    "premium_subscriber": null
}

"premium_subscriber" will always be null, we only care about the key.

To indicate a Bot integration, example

"tags": {
    "bot": "489034866206310410"
 }

To indicate a Twitch/YouTube member role, example

"tags": {
    "integration": "717460314068746300"
}

If not any of the above 3, tags will not appear.
I've only implemented the first one (partially) by adding Role#isBoosterRole(). I plan to move forward and possibly add Role#isBotRole() or Role#isIntegrationRole() or similar. It would also be nice to have a Guild#getBoosterRole() as well.
This has not been tested so as of now it's a draft, it's really just here to get feedback as I work on it and to make sure everything is good along the way :)

Iirc is there a Role#isManaged() function that is afaik for Bot and Twitch/YouTube roles.
Maybe we can implement those tags for those?

Iirc is there a Role#isManaged() function that is afaik for Bot and Twitch/YouTube roles.
Maybe we can implement those tags for those?

"Is Managed" applies to all 3 of these roles.
Managed =

Maybe we could add Role#getManagedType() that returns a Enum with the Type

Yeah, perhaps
RoleType
NORMAL - A normal everyday role
BOT - A role for bot integrations
BOOSTER - The booster role
INTEGRATION - Role from integration e.g. twitch or youtube

Woldn't it be better and simpler to have the RoleType as an enum inside the Role?
I don't see it as a separate entity.

Good call, considering how tiny it is. Forgot about classes in classes

It also might be better to rename NORMAL as NONE instead. Since there is no field present, it feels more grammatically correct and follows the patterns set by other enums.

I'm gonna implement this myself in #1343. I appreciate your efforts, but I have already planned how I would like to implement this a while ago.


SS-difference file misses required columns for AB

error when loading SS-diff-file into AB for scenario-based LCA
Superstructure: Missing required column(s) for superstructure: ['from database', 'from key', 'to database', 'to key']

Could it be, that the "flow type" column, which we have in the current export, is only required if the scenario data is read as a presamples-input file?
I think for the "normal" superstructure approach, this is not required if "from database" and "to database" or the keys are specified.

The AB does not calculate scenario results with "from database" and "to database" only, but without the keys. It seems that the keys are still required.

"to database" has no entry for biosphere flows


Fixed gh-date-time-picker not using site's timezone for min/max date

Summary

Expected behavior of the default 'now' option for both min and max date available was not met. 'now' option for min/max date used moment() function which did not consider the timezone of the site. Now moment.tz() is used instead of moment(), and blogTimezone attribute is provided as the timezone. With this change, default 'now' option will use the site's timezone.
Fixes: #15497

[x] There's a clear use-case for this code change, explained below
[x] Commit message has a short title & references relevant issues
[x] The build will pass (run yarn test:all and yarn lint)

Fantastic! 🙌 Thank you!

Hey @ozanuslan Not sure if you found this through hacktoberfest, but I've added the approved label to this PR to make sure it counts. We have plenty of open issues and ongoing projects for hacktoberfest and would love to see you around again 👋

Thank you for being so considerate @ErisDS :smiley: Really appreciate it. My hacktoberfest contributions have been fulfilled, so it's fine if this one doesn't count. I've contributed to Ghost at last year's hacktoberfest as well, so I though it would be a shame if I didn't contribute this year as well. I'll try to check out more issues as I find some free time. Thanks again :wave:

Ah yes! I thought I recognised your name from somewhere 🙂 We always have plenty of open issues - we now track more bugs and projects in the public repo - and would love to have more regular contributors to help us get more stuff out to our users ⚡ there's always so much to do!


is this page still valid - azure sql database v11

I am wondering if this page should be archived? Isn't everything v12? Also, do customers know what v11/v12 means?
Thanks.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3f8d9ca6-1795-8023-871e-9f5521440a7d
Version Independent ID: c8619519-e5a8-f6ff-89dd-3805476b0f19
Content: dbo.server_quotas (Azure SQL Database) - SQL Server
Content Source: docs/relational-databases/system-catalog-views/dbo-server-quotas-azure-sql-database.md
Service: sql-database
GitHub Login: @stevestein
Microsoft Alias: sstein

@amthomas46 Thank you for the feedback,
You are correct in that v11/v2 was deprecated since 2016,  see here.
Assigning this to the author for review.

@amthomas46 - Thanks for the feedback!
Hi Anna, thanks for pointing this out. I removed the old article.
Thanks!

#please-close


[4.2] Roundup - what to do with FormRequests & Validation

We have a quite few PRs that are interconnected/alternatives. We need to take a look at the big picture and have a clear unified position on how Inputs & FormRequests & Validation work inside Backpack.
Here's a round-up:

in https://github.com/Laravel-Backpack/CRUD/pull/3789 we're using adding the ability to edit the request inside FormRequest, and the resulting request will be processed by Backpack;
in https://github.com/Laravel-Backpack/CRUD/pull/3792 we're thinking about saving only what was validated by the request; I've just added an idea in the PR which would make that easy, but also a lot more;
in https://github.com/Laravel-Backpack/CRUD/pull/3533 we're thinking of making it easy to add validation logic inside CrudController (have validation on Create/Update, but without a FormRequest);

In addition:

in https://github.com/Laravel-Backpack/CRUD/pull/3928 we try to add another logic layer, before saving a field to the db; that way we can turn Some title to Some Title and maybe even include the file uploading logic right then-and-there, instead of having it inside a mutator on the Model;

The problem is... they ALL sound like good ideas to me! I see where each and every one of them could be useful 😅

So here's what I think:

We MUST do https://github.com/Laravel-Backpack/CRUD/pull/3789 . This will enable devs to do the same thing as https://github.com/Laravel-Backpack/CRUD/pull/3928, but in the FormRequest.

We SHOULD polish and merge https://github.com/Laravel-Backpack/CRUD/pull/3792 . This will enable devs to do the same thing as https://github.com/Laravel-Backpack/CRUD/pull/3928, but in the CrudController.

We COULD polish and merge https://github.com/Laravel-Backpack/CRUD/pull/3533 , to allow inline field validation and manual array validation;

We COULD do https://github.com/Laravel-Backpack/CRUD/pull/3928, but we could also NOT do it, since the same thing can already be done both in the FormRequest and in the Controller (see 1 and 2 above). Though... none of those would be as convenient as defining that logic on the field itself. And especially if we do (3)... it would make a lot of sense to define everything about a field in one place. So in total a field definition could be:

CRUD::field('title')->type('text')->validationRules('required|min:5|max:255')->beforeSaving(function($value) {
    return Str::of($value)->title();
});

Bottom line... I think we should do all of them 😅 This helped me structure my logic around these issues. Nobody has to read it, but if anybody does and has a different opinion, please let me know.

Note: 1 does not invalidate 4 as user might not be using FormRequests and 4 runs AFTER the validation, technically 4 is more a decorator than a validato, and I expect people would continue to write 'name' => 'required|min:2' in the FormRequest (if they use it) and strtoupper() after validation in 4.

Update: we're going to do all of them.
1&2 have already been merged.
For 3 take a look at https://github.com/Laravel-Backpack/CRUD/pull/4004
For 4 take a look at https://github.com/Laravel-Backpack/CRUD/pull/3928#issuecomment-990001381
This roundup has served its purpose, so I'm going to close it. Let's move the conversation to the issues above.


upstream connect error or disconnect/reset before headers since kubernetes 1.14

Summary
Since GKE's default kubernetes version became 1.14 and I merged the latest commit (c42ed87) on this repository, I started seeing a 503 response with a message upstream connect error or disconnect/reset before headers. reset reason: connection failure everytime I access the iap-connector frontend.
Context (Environment)
iap-connector is deployed using Google Deployment Manager on GCP, which is connected via Cloud VPN to an on-premises environment where backend services run.
Requests to iap-connector's frontend (Load Balancer) are proxied through iap-connector deployment to the on-premises backends.
Expected Behavior
A client's request to the frontend is proxied through iap-connector deployment to the backend, and the response from the backend is returned.
This has been working successfully on kubernetes version <= 1.13.
Current Behavior
A client's request does not reach the backend. A 503 response with a message upstream connect error or disconnect/reset before headers. reset reason: connection failure is returned to the client.
This has been occurring since kubernetes v1.14 became the default kubernetes version on GKE and I merged the latest master branch on this repository (c42ed87).

I've been trying to figure out where the issue is, but without luck.
I've found out that in the v1.14 deployment, ambassador is unable to reach the backend due to network unreachability (curl backend results in Operation timed out), whereas on v1.13 it is reachable.
Any advise on where to look for would also be nice :)

This issue was due to a change in kubernetes' IP masquerade behavior. I was able to resolve this with the following tips:

https://stackoverflow.com/questions/58309735/vpn-access-to-in-house-network-not-working-after-gke-cluster-upgrade-to-1-14-6
https://cloud.google.com/kubernetes-engine/docs/how-to/ip-masquerade-agent


Restore P>Q in RSA key generation (#558)

The PKCS#1 standard says nothing about the relation between P and Q
but many libraries guarantee P>Q and mbed TLS did so too in earlier
versions.
This commit restores this behaviour.

+1, but the changes need to be rebased.

I have rebased it.

Reviewed and approved.

Tested and merged.


Fix - i accept the expensify terms of service, checkbox is not checked when connected online with plaid

Details
Fixed Issues
$ https://github.com/Expensify/App/issues/17877
PROPOSAL: https://github.com/Expensify/App/issues/17877#issuecomment-1536376485
Tests
Verify if a checkbox is checked when the user returns to bank account step from company information step.

Open the app -> Workspaces -> Connect Bank Account
Click Connect Online with Plaid
Enter test credentials for bank account Fidelity, UN: user_good PW: pass_good(if you're in dev, you need to turn on 'use staging server' in preference setting first before connecting)
Upon returning to expensify, select a bank account from the dropdown options.
I accept the Expensify terms of service should appear before proceeding to Company Information step.
Click on the check box and continue to the next step.
Return back to the previous step.
The checkbox should remain checked.

[x] Verify that no errors appear in the JS console

Offline tests
Needs an internet connection to connect bank account online. Therefore, it is not applicable.
QA Steps
Same as "Tests" Section above.

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

https://github.com/Expensify/App/assets/39636266/e075c17c-dd7e-4107-9a90-7e86ce4992bc

Mobile Web - Chrome

Screencast-iAcceptTerms-mWebChrome.webm

Mobile Web - Safari

https://github.com/Expensify/App/assets/39636266/f1d70a87-bfe1-43b4-918a-d4da346a21b9

Desktop

https://github.com/Expensify/App/assets/39636266/89b138ea-6987-4981-8481-b19c391c4621

iOS

https://github.com/Expensify/App/assets/39636266/bfa7a187-3f56-450b-b251-2ccf3af9da30

Android

https://github.com/Expensify/App/assets/39636266/eda4aef4-1639-436f-926d-ce2953fb0a6a

@HezekielT can you fix the GH link, it's not supposed to be GHLink highlighted as mentioned in the PR template it should be the actual link of the PR.

Please replace GH_LINK with a URL link to the GitHub issue this Pull Request is fixing.

That is why we didn't get assigned for review

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

I'm getting this error on iOS
https://github.com/Expensify/App/assets/9059945/7cb92587-051b-4f65-9b94-fcc302ae4b99
But weirdly, it isn't present on android

@HezekielT, thanks for attaching those links!
I'll finish the remaining checklist items in a couple of hours

🚀 Deployed to staging by https://github.com/madmax330 in version: 1.3.14-2 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

🚀 Deployed to staging by https://github.com/madmax330 in version: 1.3.14-2 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/yuwenmemon in version: 1.3.14-14 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Willing to take over as a maintainer

Considering this package has essentially had no activity for over 3 years, I am willing to take over maintainership/ownership of it. We are using this library in the Keycloak project, specifically for our Keycloak JS adapter here at Red Hat.
I would specifically take the following actions:

Resolve existing bugs and pull requests, now and in the future.
Modernize the code to use modern JavaScript features, including modules.

cc @beatgammit @feross

Bumping this once again hoping to be able to reach out.

Closing this as we've forked the repo into https://github.com/feross/base64-js where I will help out with maintenance.


Bump version to 1.0

MNT: Re-rendered with conda-build 3.20.5, conda-smithy 3.8.5, and conda-forge-pinning 2<IP_ADDRESS>.36.34

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


Norwegian bokmål translation

100% done Norwegian bokmål translation

If other norwegians can support with looking over the translation that would be great.

Thanks!


Please demonstrate how to open an application from xbar?

I have a "Get Support" xbar sub-menu that provides typical IT URLs to my IT help desk site (ServiceNow), as well as user-facing KBs and documentation (Confluence). Very useful for end-users.
I'd like to add a clickable link to open a specific screen-sharing app located in /Applications (TeamViewer QuickSupport.app).
I have tried everything to get xbar to open an app when selected (via zsh plug-in), but I'm unable to do so. Can someone please provide an example of the appropriate API syntax, please?
Thanks!

This worked for me:
echo "Say hi | shell=open | param1=/Applications/Pages.app"

@dstranathan I hope this answers your question?


Add chain information

Is your feature request related to a problem? Please describe.
Looking to get certificate chain information displayed in the certificate info output
Describe the solution you'd like
A list of parent certificates in the chain

Fixed by #12


Request limit exceeded exponential backoff (Volume Tagging Monkey)

Have found that with lots of Volume Tagging; and Volume Tagging Monkey being always on Monkey time - that Exceeding API rate limits is common when tagging meta data on Volumes.
This solves this issue; and also fixes (at least for Volume Tagging Monkey) Issue #173

NetflixOSS » SimianArmy » SimianArmy-pull-requests #8 SUCCESS
This pull request looks good

SimianArmy-pull-requests #180 SUCCESS
This pull request looks good

Did you consider using the Amazon ClientConfiguration class instead of rolling custom retry code?
I did something similar on a fork to handle API limits with SimpleDB.  The code is simpler and it is globally applied instead of having to roll custom retry code at every client interaction.  In my case I just increased the retry count.
ClientConfiguration config = new ClientConfiguration();
configuration.setMaxErrorRetry(11);
....
client = new AmazonSimpleDBClient(configuration);

I experimented with custom retry policies but increasing the retry count worked just as well and was less code.

No; I did a quick and dirty hack for a limit I was seeing in production use :)
However it's a good idea. I'll look into it; and have a look at your latest commit. https://github.com/ebukoski/SimianArmy/commit/97c1a3a353fb99cc72142a523df5976bb27335b0

AWS explain that you should implement exponential back-off when you receive the server (5xx) or throttling errors.
http://docs.aws.amazon.com/general/latest/gr/api-retries.html
I'll close this and re-implement globally.


ImportError: cannot import name 'soft_unicode' from 'markupsafe'

Error when trying to install jinja :
   File "/usr/local/lib/python3.9/site-packages/jinja2/filters.py", line 13, in <module>      from markupsafe import soft_unicode  ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.9/site-packages/markupsafe/__init__.py)
With new version of markup safe (2.1.0), soft_unicode has been removed (it was previously deprecated). We need to "Use soft_str instead" : https://markupsafe.palletsprojects.com/en/2.1.x/changes/#version-2-1-0
Environment:

Python version: 3.9
Jinja version: ^2.11.3

https://github.com/pallets/jinja/issues/1585


Hide dropdown when button is clicked and 'dropdown-content' is active

@carlosperate what do you think about this pull request? This is evident when data-beloworigin is set to true.


⚠ ClamAV has degraded performance

In e289b10, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 458 ms

Resolved: ClamAV performance has improved in 92ba6a1.


Ensure Readme.md is correct name

Also pip does not seem to be synced with the latest repo:
$ sudo -H pip3 install phigaro
Collecting phigaro
  Downloading https://files.pythonhosted.org/packages/21/e9/863f9da4508ae1000f71224f5c391b4344de7c54c26ea719e83895cb5c52/phigaro-<IP_ADDRESS>.tar.gz (70kB)
     |████████████████████████████████| 71kB 2.0MB/s
    ERROR: Complete output from command python setup.py egg_info:
    ERROR: Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/tmp/pip-install-2bdw4v97/phigaro/setup.py", line 2, in <module>
        with open("README.md", "r") as fh:
    FileNotFoundError: [Errno 2] No such file or directory: 'README.md'
    ----------------------------------------
ERROR: Command "python setup.py egg_info" failed with error code 1 in /private/tmp/pip-install-2bdw4v97/phigaro/

Updated version on pypi


Ability to show current pin modes (INPUT, OUTPUT, ...)

This is a great project, thanks for the good work!
It would be nice to have additional pin state information like: INPUT, OUTPUT, INPUT_PULLUP and INPUT_PULLDOWN

This has been added ion the latest version.


KeyError: "Couldn't find field google.protobuf.DescriptorProto.ExtensionRange.options" when  import datastore

Just installed google-cloud (0.27.0) on my Mac OS X EI Capitan
Python 2.7
Anaconda
under a root env. when running: testLog.py
#Retrieve Datastore Entities
from google.cloud import datastore
client = datastore.Client()
product_key = client.key('Product', 123)
print(client.get(product_key))

Got following messages:
Traceback (most recent call last):
  File "testLog.py", line 3, in <module>
    from google.cloud import datastore
  File "/Users/David/anaconda/lib/python2.7/site-packages/google/cloud/datastore/__init__.py", line 60, in <module>
    from google.cloud.datastore.batch import Batch
  File "/Users/David/anaconda/lib/python2.7/site-packages/google/cloud/datastore/batch.py", line 24, in <module>
    from google.cloud.datastore import helpers
  File "/Users/David/anaconda/lib/python2.7/site-packages/google/cloud/datastore/helpers.py", line 23, in <module>
    from google.protobuf import struct_pb2
  File "/Users/David/anaconda/lib/python2.7/site-packages/google/protobuf/struct_pb2.py", line 11, in <module>
    from google.protobuf import descriptor_pb2
  File "/Users/David/anaconda/lib/python2.7/site-packages/google/protobuf/descriptor_pb2.py", line 409, in <module>
    options=None),
  File "/Users/David/anaconda/lib/python2.7/site-packages/google/protobuf/descriptor.py", line 501, in __new__
    return _message.default_pool.FindFieldByName(full_name)
KeyError: "Couldn't find field google.protobuf.DescriptorProto.ExtensionRange.options"

Please help, thanks

Hi @tianqig,
Thanks for reporting.
Could you try creating a virtualenv and installing only protobuf, and see if from google.protobuf import descriptor_pb2 passes?
Also, this seems like it is probably an issue with protobuf itself, so I am going to move this issue over there.

This issue was moved to google/protobuf#3631


Bot isnt starting up

The bot stopped working from one day to the other, I reinstalled it completely including git and python through all the links given and the bot doesnt go online on the server and I have no clue why.

http://prntscr.com/f2kl2f

Looks like an issue with your permissions.ini file, check around line 20 for any issues.

Line 20 is whitelisted commands , play queue and skip. What of those might be causing trouble


Robotium without any connection

From<EMAIL_ADDRESS>on June 28, 2015 17:44:40
I want to run automated app using webview Robotium without any connection.
For example, if a user touches the start button then, application start automated work.
Please advice How can i solve this problem.
Original issue: http://code.google.com/p/robotium/issues/detail?id=687

From<EMAIL_ADDRESS>on July 08, 2015 21:48:34
Please keep this tracker for issues.
Status: Invalid
Owner<EMAIL_ADDRESS>


GEODE-7774: Concourse BumpXYZ does not include {prerelease}

Update the bump jobs to increment and add the semver release token which
clears the wrong semver picking happening in past few days.
Signed-off-by: Robert Houghton<EMAIL_ADDRESS>Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

[x] Has your PR been rebased against the latest commit within the target branch (typically develop)?

[x] Is your initial contribution a single, squashed commit?

[ ] Does gradlew build run cleanly?

[ ] Have you written or updated unit tests to verify your changes?

[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

Note:
Please ensure that once the PR is submitted, check Concourse for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to<EMAIL_ADDRESS>

Just noticed, this might be using the wrong GEODE ticket number.  It's updating a starter/newb ticket about Lucene Serializer


fix emoji images

The emoji are not showing up correctly.
I had to add the following line to override the img block command.
img.emoji {
    display: inline-block !important;
}

I'm unable to reproduce. I think this was fixed indirectly by a previous commit that modified the img style.


Add first tx defition

1. What does this PR implement?
Coordination layer and transactions to use in the node. Processing will be added in a following PR.
This also refactor the organization of some important components of the nomos arch. In particular, cl crate has been moved inside nomos-core and nomos-core has been moved to chain-defs (although the package name does not change). This is done to reflect the fact that cl is about the data model while proofs and formats to be used on chain live in the chain-defs crate.
In addition, risc0 proofs and statements also are now under nomos-core and carnot leftovers have been removed.
Moreover, PoL definition will be moved from the ledger in a following PR. The goal is to remove any definition of chain data  format or proof system from the ledger crate, only dealing with the data model using whatever representation fits it better.
2. Does the code have enough context to be clearly understood?
Main reference is the Coordination Layer data model, for which you can find the spec in Notion
3. Who are the specification authors and who is accountable for this PR?
@davidrusu @alvatar and myself
4. Is the specification accurate and complete?
Yes, it has been reviewed after the implementation work and discrepancies have been resolved.
5. Does the implementation introduce changes in the specification?
Any changes have been already reflected in the spec.
The only notable difference is that generic hash function of the spec has been selected to be sha256 due to efficiency considerations inside risc0's zkvm.
Checklist

[X] 1. Description added.
[X] 2. Context and links to Specification document(s) added.
[x] 3. Main contact(s) (developers and specification authors) added
[x] 4. Implementation and Specification are 100% in sync including changes. This is critical.
[x] 5. Link PR to a specific milestone.

It looks good. Left a few comments with nitpicks and things I'm missing probably.


New command - Install-DbaMultiTool

Type of Change

[ ] Bug fix (non-breaking change, fixes # )
[x] New feature (non-breaking change, adds functionality, fixes #6954  )
[ ] Breaking change (effects multiple commands or functionality, fixes # )
[ ] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[ ] Adding code coverage to existing functionality
[x] Pester test is included
[ ] If new file reference added for test, has is been added to github.com/sqlcollaborative/appveyor-lab ?
[ ] Nunit test is included
[ ] Documentation
[ ] Build system

Purpose

Add an installer for dba-multitool T-SQL scripts as per #6954. There are currently four utility scripts: sp_doc, sp_estindex, sp_helpme, and sp_sizeoptimiser.
As discussed in the issue, I am happy to work on Invoke- functions for these later if the interest arises.
Approach

More or less a copy of Install-DbaFirstResponderKit with a few minor tweaks. Added an additional test scenario for 2008R2 to have better coverage for a "skipped" status for instances below the minimum version (2012).
While I do have an all-in-one installer script, installing them individually seemed like a better option for more granular results.
Commands to test

Install-DbaMultiTool

@wsmelton Last issue:
I switched all tests to use instance3 in commit "use one instances type for all tests", but the resulting appveyor build still tried to use 2008R2/2016:

The subsequent commit "switch to 2016 instance" changed them all to use instance2 and that built successfully. Is there something I am missing as to why it failed?

Instance failure seems like it is coming from Appveyor.

Defaulted back to using the same test instance as Install-DbaFirstResponderKit, which makes the tests file more or less the same now.

thanks so much, all! 🥳  will merge, then add to psd1 and psm1 and publish shortly


H2GIS driver

We'd like to add a H2GIS (www.h2gis.org) driver to the dbeaver connection wizard. As H2 database, the aim is to offer two types of connection : H2GIS embedded and H2GIS server
Would the community be interested?
If yes, here, how we plan to proceed :

Create a plugin bundle as org.jkiss.dbeaver.ext.h2 -> org.jkiss.dbeaver.ext.h2gis
Declare the required dependencies in dbeaver/plugins/org.jkiss.dbeaver.ext.generic/plugin.xml
Find a way to initialize the H2GIS spatial functions (see http://www.h2gis.org/docs/1.5.0/quickstart/)

Best regards

Based on H2 plugin, I added the H2GIS driver. Code is here : https://github.com/ebocher/dbeaver/tree/devel_h2gis/plugins/org.jkiss.dbeaver.ext.h2gis
The next step is to initialize the spatial functions with the following sql query :
CREATE ALIAS IF NOT EXISTS H2GIS_SPATIAL FOR "org.h2gis.functions.factory.H2GISFunctions.load";
CALL H2GIS_SPATIAL();

The initialization of the functions must also be done when the connection to the database is tested.
I have some doubts about the method.
Is it necessary to add a specific H2GIS dialect to load the spatial functions ?
Can you advise me or give me some hints ?
Regards

Looks good 👍
However, does it really make sense to create new extension? If GIS data is the only difference between h2gis and regular h2 then it probably can be done in org.jkiss.dbeaver.ext.h2 extension.
And add extra functionality there.
And of course new driver definition has to be added.
Do I understand correctly that showing GIS data (similarly to how it was done for PostgreSQL or MySQL)  is the main purpose of this new driver?

Dear @serge-rider
Thanks for the comment.
I am in favour of a specific extension.
H2GIS is not an "official" spatial extension as PostGIS for PG. Even if H2GIS is currently the most complete  spatial functions, some other extensions with a different architecture can be proposed by the community. So if you add it in the H2 extension you will have to add extra functionality for each of them.
Moreover (but I'm not sure about that), it seems that it will be interesting in the future to inject from the H2GISQLDialect some metadata in order to improve the user experience with DBbeaver UI.
Best

Another argument in favour, I will be interested to add an entry "import data" when the user right-click in the database browser. I think about a custom import to load spatial data in an H2GIS database.

@serge-rider
I'd like to display H2GIS geometry (stored natively as JTS object) with the GIS component. Could you point me the class who is doing the work to detect the geometry datatype ?
Best

@serge-rider
I found some information in the PG driver. An interface GisAttribute, constant to map Geometry type. Do I need to extend all classes as for example PostgreTableColumnManager ?

@ebocher Probably we need to discuss details more deeply. Please write me on<EMAIL_ADDRESS>

@serge-rider
+1

verified


Remove unstable uv lock from pip interface

Summary
I think we can start using uv lock and uv sync to test this instead.

The PR itself looks good. I think one remaining thing here (because I just tried it) is specifying the index URL. I can do that with uv pip compile, but I'm not sure how to do that with uv lock yet.


Unable to see nunuStudio page correctly

Unable to see nunuStudio page correctly
Description

The nunuStudio landing page is not showing correctly. The 3D demo is not showing.
The console shows u.viewport.fromArray is not a function
Tested with Firefox 65.0.2
Testes on Chrome 72.0.3626.121 (Official Build) (64-bit)

Version

Current webpage as for 2019-03-06.

Platform

Web Version [X]

Thanks a lot for reporting! Should be fixed now!


pizzly kallisto IOError: Unable to create file

Hi Rorky,
I am having trouble running the RNA-seq pipeline since the pizzly was added to the default run.
what look strange to me is that the IOError message report the filename all in lower case
5667_az/work/kallisto/sudhl10igo05667
do you have any idea?
Thanks!
my template:
Template for human RNA-seq using Illumina prepared samples

details:

analysis: RNA-seq
genome_build: hg19
algorithm:
aligner: star
disambiguate: mm10
strandedness: unstranded
#transcriptome_align: true
fusion_mode: true
upload:
dir: ../final
resources:
samtools:
memory: 6g

[2017-10-03T23:34Z] gpu-2-8.local: [Tue Oct 03 19:34:46 EDT 2017] ==Tissue specific
[2017-10-03T23:34Z] gpu-2-8.local: [Tue Oct 03 19:34:46 EDT 2017] Initializing classifier
[2017-10-03T23:34Z] gpu-2-8.local: [Tue Oct 03 19:34:46 EDT 2017] Classifying fusions
[2017-10-03T23:34Z] gpu-2-8.local: [Tue Oct 03 19:34:46 EDT 2017] Writing output
[2017-10-03T23:34Z] gpu-2-8.local: Unexpected error
Traceback (most recent call last):
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 50, in _setup_logging
yield config
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 313, in detect_fusions
return ipython.zip_args(apply(rnaseq.detect_fusions, *args))
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/rnaseq.py", line 124, in detect_fusions
pizzly_dir = pizzly.run_pizzly(data)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/rnaseq/pizzly.py", line 45, in run_pizzly
fraglength = get_fragment_length(data)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/rnaseq/pizzly.py", line 30, in get_fragment_length
with h5py.File(h5) as f:
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/h5py/_hl/files.py", line 271, in init
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/h5py/_hl/files.py", line 126, in make_fid
fid = h5f.create(name, h5f.ACC_EXCL, fapl=fapl, fcpl=fcpl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/feedstock_root/build_artefacts/h5py_1491364603221/work/h5py-2.7.0/h5py/_objects.c:2846)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/feedstock_root/build_artefacts/h5py_1491364603221/work/h5py-2.7.0/h5py/_objects.c:2804)
File "h5py/h5f.pyx", line 98, in h5py.h5f.create (/feedstock_root/build_artefacts/h5py_1491364603221/work/h5py-2.7.0/h5py/h5f.c:2290)
IOError: Unable to create file (Unable to open file: name = '/cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_az/5667_az/work/kallisto/ocily3igo05667/quant/abundance.h5', errno = 2, error message = 'no such file or directory', flags = 15, o_flags = c2)
[2017-10-03T23:34Z] gpu-1-6.local: Unexpected error
Traceback (most recent call last):
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 50, in _setup_logging
yield config
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 313, in detect_fusions
return ipython.zip_args(apply(rnaseq.detect_fusions, *args))
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/rnaseq.py", line 124, in detect_fusions
pizzly_dir = pizzly.run_pizzly(data)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/rnaseq/pizzly.py", line 45, in run_pizzly
fraglength = get_fragment_length(data)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/rnaseq/pizzly.py", line 30, in get_fragment_length
with h5py.File(h5) as f:
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/h5py/_hl/files.py", line 271, in init
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/h5py/_hl/files.py", line 126, in make_fid
fid = h5f.create(name, h5f.ACC_EXCL, fapl=fapl, fcpl=fcpl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/feedstock_root/build_artefacts/h5py_1491364603221/work/h5py-2.7.0/h5py/_objects.c:2846)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/feedstock_root/build_artefacts/h5py_1491364603221/work/h5py-2.7.0/h5py/_objects.c:2804)
File "h5py/h5f.pyx", line 98, in h5py.h5f.create (/feedstock_root/build_artefacts/h5py_1491364603221/work/h5py-2.7.0/h5py/h5f.c:2290)
IOError: Unable to create file (Unable to open file: name = '/cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_az/5667_az/work/kallisto/ocily3igo05667/quant/abundance.h5', errno = 2, error message = 'no such file or directory', flags = 15, o_flags = c2)
[2017-10-03T23:34Z] gpu-2-12.local: Unexpected error
Traceback (most recent call last):
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 50, in _setup_logging
yield config
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 313, in detect_fusions
return ipython.zip_args(apply(rnaseq.detect_fusions, *args))
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/rnaseq.py", line 124, in detect_fusions
pizzly_dir = pizzly.run_pizzly(data)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/rnaseq/pizzly.py", line 45, in run_pizzly
fraglength = get_fragment_length(data)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/rnaseq/pizzly.py", line 30, in get_fragment_length
with h5py.File(h5) as f:
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/h5py/_hl/files.py", line 271, in init
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/h5py/_hl/files.py", line 126, in make_fid
fid = h5f.create(name, h5f.ACC_EXCL, fapl=fapl, fcpl=fcpl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/feedstock_root/build_artefacts/h5py_1491364603221/work/h5py-2.7.0/h5py/_objects.c:2846)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/feedstock_root/build_artefacts/h5py_1491364603221/work/h5py-2.7.0/h5py/_objects.c:2804)
File "h5py/h5f.pyx", line 98, in h5py.h5f.create (/feedstock_root/build_artefacts/h5py_1491364603221/work/h5py-2.7.0/h5py/h5f.c:2290)
IOError: Unable to create file (Unable to open file: name = '/cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_az/5667_az/work/kallisto/ocily3igo05667/quant/abundance.h5', errno = 2, error message = 'no such file or directory', flags = 15, o_flags = c2)
[2017-10-03T23:34Z] gpu-2-12.local: Unexpected error
Traceback (most recent call last):
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 50, in _setup_logging
yield config
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 313, in detect_fusions
return ipython.zip_args(apply(rnaseq.detect_fusions, *args))
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/rnaseq.py", line 124, in detect_fusions
pizzly_dir = pizzly.run_pizzly(data)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/rnaseq/pizzly.py", line 45, in run_pizzly
fraglength = get_fragment_length(data)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/rnaseq/pizzly.py", line 30, in get_fragment_length
with h5py.File(h5) as f:
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/h5py/_hl/files.py", line 271, in init
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/h5py/_hl/files.py", line 126, in make_fid
fid = h5f.create(name, h5f.ACC_EXCL, fapl=fapl, fcpl=fcpl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/feedstock_root/build_artefacts/h5py_1491364603221/work/h5py-2.7.0/h5py/_objects.c:2846)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/feedstock_root/build_artefacts/h5py_1491364603221/work/h5py-2.7.0/h5py/_objects.c:2804)
File "h5py/h5f.pyx", line 98, in h5py.h5f.create (/feedstock_root/build_artefacts/h5py_1491364603221/work/h5py-2.7.0/h5py/h5f.c:2290)
IOError: Unable to create file (Unable to open file: name = '/cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_az/5667_az/work/kallisto/ocily3igo05667/quant/abundance.h5', errno = 2, error message = 'no such file or directory', flags = 15, o_flags = c2)
[2017-10-03T23:35Z] gpu-2-10.local: [Tue Oct 03 19:35:00 EDT 2017] [Post-processing RNASTAR input] 500000 reads analyzed
2017-10-03 19:35:01.107 [IPClusterStop] Using existing profile dir: u'/cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/log/ipython'
2017-10-03 19:35:01.145 [IPClusterStop] Stopping cluster [pid=1029] with [signal=2]
2017-10-03 19:35:02.616 [IPClusterStop] Using existing profile dir: u'/cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/log/ipython'
2017-10-03 19:35:02.623 [IPClusterStop] Stopping cluster [pid=1029] with [signal=2]
14 Engines running
Sending a shutdown signal to the controller and engines.
20 Engines running
Sending a shutdown signal to the controller and engines.
15 Engines running
Sending a shutdown signal to the controller and engines.
Traceback (most recent call last):
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/bin/bcbio_nextgen.py", line 234, in
main(**kwargs)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/bin/bcbio_nextgen.py", line 43, in main
run_main(**kwargs)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 50, in run_main
fc_dir, run_info_yaml)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 86, in _run_toplevel
for xs in pipeline(config, run_info_yaml, parallel, dirs, samples):
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 243, in rnaseqpipeline
samples = rnaseq.quantitate_expression_parallel(samples, run_parallel)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/rnaseq.py", line 116, in quantitate_expression_parallel
samples = run_parallel("detect_fusions", samples)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipython.py", line 136, in run
for data in view.map_sync(fn, items, track=False):
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 353, in map_sync
return self.map(f,*sequences,**kwargs)
File "", line 2, in map
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 54, in sync_results
ret = f(self, *args, **kwargs)
File "", line 2, in map
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 39, in save_ids
ret = f(self, *args, **kwargs)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 1119, in map
return pf.map(*sequences)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 268, in map
ret = self(*sequences)
File "", line 2, in call
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 75, in sync_view_results
return f(self, *args, **kwargs)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 251, in call
return r.get()
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/ipyparallel/client/asyncresult.py", line 98, in get
self.wait(timeout)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/ipyparallel/client/asyncresult.py", line 129, in wait
self._ready = self._client.wait(self.msg_ids, timeout)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/ipyparallel/client/client.py", line 1080, in wait
self.spin()
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/ipyparallel/client/client.py", line 1027, in spin
self._flush_iopub(self._iopub_socket)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/ipyparallel/client/client.py", line 893, in _flush_iopub
idents,msg = self.session.recv(sock, mode=zmq.NOBLOCK)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/jupyter_client/session.py", line 793, in recv
raise e
ValueError: Invalid Signature: '7a974c6e50a4953f0200aef255641dfc6df6c5211f61d35bd2f4da7e3278f01f'

I have the same issue as well - also with the lower case origin. Some code in the kallisto context does not seem to like path names with upper case characters it seems.

Thanks !
Is possible to turn off pizzly and kallisto? In a specific run?
Thanks!
Sent from my iPhone

On Oct 4, 2017, at 8:46 AM, Sven-Eric Schelhorn<EMAIL_ADDRESS>wrote:
I have the same issue as well - also with the lower case origin. Some code in the kallisto context does not seem to like path names with upper case characters it seems.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Thanks, sorry about that folks. Hm-- the lowercase returning of the path is due to this bug: https://github.com/h5py/h5py/issues/854, so it is a little bit of a red herring. Is the abundance.h5 not there for that sample if you use the correct path name?

Ok, I think I figured it out-- this is happening when fusion_mode is on and kallisto isn't listed in the expression_caller field. If you add expression_caller: [kallisto] to the entries, it will run kallisto and pizzly correctly. I'll push a fix that forces kallisto to run if fusion_mode is on and provide a way to turn it off for the next release.

Thanks!
Sent from my iPhone

On Oct 4, 2017, at 9:37 AM, Rory Kirchner<EMAIL_ADDRESS>wrote:
Closed #2089 via ab0748a.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Thanks all, I added the ability to turn individual fusion callers on via the fusion_caller field. fusion_mode will still work for the next several versions and will run oncofuse/pizzly but eventually we'll just switch over to fusion_caller only.

Thanks to you for implementing this so fast!
Sent from my iPhone

On Oct 4, 2017, at 12:30 PM, Rory Kirchner<EMAIL_ADDRESS>wrote:
Thanks all, I added the ability to turn individual fusion callers on via the fusion_caller field. fusion_mode will still work for the next several versions and will run oncofuse/pizzly but eventually we'll just switch over to fusion_caller only.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

HI Rorky,
i am still getting a error message from puzzle
[2017-10-09T19:17Z] gpu-2-14.local: Unexpected error
Traceback (most recent call last):
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 50, in _setup_logging
yield config
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 313, in detect_fusions
return ipython.zip_args(apply(rnaseq.detect_fusions, *args))
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/rnaseq.py", line 134, in detect_fusions
pizzly_dir = pizzly.run_pizzly(data)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/rnaseq/pizzly.py", line 50, in run_pizzly
fusions, samplename, data)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/rnaseq/pizzly.py", line 68, in pizzly
do.run(cmd.format(**locals()), message)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 23, in run
_do_run(cmd, checks, log_stdout, env=env)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 103, in _do_run
raise subprocess.CalledProcessError(exitcode, error_msg)
CalledProcessError: Command 'set -o pipefail; /cbio/ski/abdelwao/nobackup/pastore/bcbio/galaxy/../anaconda/bin/pizzly -k 31 --gtf /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/pizzly/pizzly.gtf --cache /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/pizzly/pizzly.cache --align-score 2 --insert-size 233 --fasta /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/inputs/transcriptome/hg19-mm10.fa --output /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/bcbiotx/tmpy_mmEh/CNS13IGO05667/CNS13IGO05667 /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/kallisto/CNS13IGO05667/quant/fusion.txt
Opening cached file ... loaded 0 genes and 0 transcripts
Error, could not find any transcript sequences check that the ids in the FASTA file and GTF file match
' returned non-zero exit status 1
[2017-10-09T19:17Z] gpu-1-4.local: Running pizzly on /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/kallisto/SUDHL5IGO05667/quant/fusion.txt.
[2017-10-09T19:17Z] gpu-1-4.local: Opening cached file ... loaded 0 genes and 0 transcripts
[2017-10-09T19:17Z] gpu-1-12.local: Error, could not find any transcript sequences check that the ids in the FASTA file and GTF file match
[2017-10-09T19:17Z] gpu-1-12.local: Uncaught exception occurred
Traceback (most recent call last):
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 23, in run
_do_run(cmd, checks, log_stdout, env=env)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 103, in _do_run
raise subprocess.CalledProcessError(exitcode, error_msg)
CalledProcessError: Command 'set -o pipefail; /cbio/ski/abdelwao/nobackup/pastore/bcbio/galaxy/../anaconda/bin/pizzly -k 31 --gtf /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/pizzly/pizzly.gtf --cache /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/pizzly/pizzly.cache --align-score 2 --insert-size 221 --fasta /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/inputs/transcriptome/hg19-mm10.fa --output /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/bcbiotx/tmpm_jT7X/SUDHL6IGO05667/SUDHL6IGO05667 /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/kallisto/SUDHL6IGO05667/quant/fusion.txt
Opening cached file ... loaded 0 genes and 0 transcripts
Error, could not find any transcript sequences check that the ids in the FASTA file and GTF file match
' returned non-zero exit status 1
[2017-10-09T19:17Z] gpu-1-12.local: Unexpected error
Traceback (most recent call last):
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 50, in _setup_logging
yield config
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 313, in detect_fusions
return ipython.zip_args(apply(rnaseq.detect_fusions, *args))
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/rnaseq.py", line 134, in detect_fusions
pizzly_dir = pizzly.run_pizzly(data)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/rnaseq/pizzly.py", line 50, in run_pizzly
fusions, samplename, data)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/rnaseq/pizzly.py", line 68, in pizzly
do.run(cmd.format(**locals()), message)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 23, in run
_do_run(cmd, checks, log_stdout, env=env)
File "/cbio/ski/abdelwao/home/pastore/nobackup/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 103, in _do_run
raise subprocess.CalledProcessError(exitcode, error_msg)
CalledProcessError: Command 'set -o pipefail; /cbio/ski/abdelwao/nobackup/pastore/bcbio/galaxy/../anaconda/bin/pizzly -k 31 --gtf /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/pizzly/pizzly.gtf --cache /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/pizzly/pizzly.cache --align-score 2 --insert-size 221 --fasta /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/inputs/transcriptome/hg19-mm10.fa --output /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/bcbiotx/tmpm_jT7X/SUDHL6IGO05667/SUDHL6IGO05667 /cbio/ski/abdelwao/nobackup/pastore/install/bcbio-vm/data/project/christian/5667_AZ/5667_AZ/work/kallisto/SUDHL6IGO05667/quant/fusion.txt
Opening cached file ... loaded 0 genes and 0 transcripts

Thanks, this is the same issue where we aren't handling disambiguated genomes correctly using pizzly. I'll push a workaround for this tomorrow; I think for now I'll disable pizzly calling on the paired transcriptome like this and just use the disambiguated calls from the target genome, else we'll end up with a bunch of false positive fusions.


error when load the lmdb data

RuntimeError                              Traceback (most recent call last)
 in
----> 1 data = pickle.loads(datapoint_pickled)
RuntimeError: Pre-condition Violation
Number of atom mismatch
Violation occurred on line 617 in file Code/GraphMol/ROMol.cpp
Failed Expression: conf->getNumAtoms() == this->getNumAtoms()
RDKIT: 2020.09.1
BOOST: 1_73

would you mind supply the fixed PDBBind data without RDKit error and duplicated protein, not just the pocket data

just update rdkit can fix this
pip install update rdkit


[BUG] Missing Release/Tag v1.4

Reproduction steps
I heard to release Matter v1.4https://csa-iot.org/newsroom/matter-1-4-enables-more-capable-smart-homes/
However, I could not find Tag v1.4 in SDK.
https://github.com/project-chip/connectedhomeip/tags
Bug prevalence
always
GitHub hash of the SDK that was being used
c3b35eb6c16437ad19d919faa69d269c95da10db
Platform
core
Platform Version(s)
No response
Anything else?
No response

See https://github.com/project-chip/connectedhomeip/tree/v1.4-branch

Is there going to be a tag for the v<IP_ADDRESS> release? What is the correct hash of the release? The v1.4 branch has recent commits.


Ubuntu app error code 80070008 (registering with WSL2?)

Version
Microsoft Windows [Version 10.0.19045.2364]
WSL Version

[X] WSL 2
[ ] WSL 1

Kernel Version
No response
Distro Version
Ubuntu 20.04
Other Software
No response
Repro Steps

Install WSL2 per https://learn.microsoft.com/en-us/windows/wsl/install-manual .
Because Microsoft Store is blocked (both for browser and for "wsl --install"), install "Ubuntu 20.04 (2021)" from SCCM.
(variant) add %UserProfile%.wslconfig with two lines "[wsl2]" and "memory=4GB # Limits VM memory in WSL 2 to 4 GB
" (also tried 8GB).
Run (a) Ubuntu from the Start menu (cannot run elevated, nor as administrator) or (b) ubuntu.exe in an elevated cmd.exe window with working directory C:\Temp .  This is with nearly 21GB free memory out of about 31GB.

NOTE: Windows+F doesn't work, and I don't have administrator permissions.
Expected Behavior
I was expecting ubuntu.exe to at least register with WSL2 (since wsl --list is empty), and perhaps to do more.
Actual Behavior
This is what was output in the Ubuntu window (and "wsl --list" still shows no distributions):
Installing, this may take a few minutes...
WslRegisterDistribution failed with error: 0x80070008
Error: 0x80070008 Not enough memory resources are available to process this command.
Press any key to continue...
Diagnostic Logs
No response

/logs

WslLogs-2023-01-24_08-43-30.zip
I was given admin permission for this, so I was able to run the script to get the logs.  Note that VMware is installed on the laptop, but it makes no difference whether its services are running or not, for this error.

Thanks @TDavidHudson. Looking at the logs, I see:
1013	True	Microsoft.Windows.Lxss.Manager	Error	0	01-24-2023 05:55:11.521	"	"	"code:
failurecount: 	1
file: 	onecore\vm\wsl\lxss\service\lib\utilityvmcontext.cpp
function:
hr: 	0x80070008
linenumber: 	1423
message: 	""HcnCreateNetwork failed {""""Success"""":false,""""Error"""":""""Not enough memory resources are available to process this command. """",""""ErrorCode"""":2147942408}""
threadid: 	15532
type: 	0"	onecore\vm\wsl\lxss\service\lib\utilityvmcontext.cpp			17424	15532	2		00000000-0000-0000-0000-000000000000

Which seems to indicate that there was not enough memory available on your machine.
How much memory was available on your machine when WSL started ? Can you share a screenshot of the 'Performance' tab in TaskManager ?

As I mentioned initially, there was nearly 21GB free memory, despite the error code indicating insufficient memory.  Now, after some reinstalls, there is still over 19GB free.  It's supposed to be possible to run WSL (actually, ubuntu.exe) in only 8GB, right?  And I still have the 8GB .wslconfig file in my home directory.

Interesting. This error was return from HNS, so this shouldn't be related to the memory settings in WSL. @keith-horton: Is there something we can do to diagnose why HNS returned that error ?

Around the time of the ubuntu.exe error, there are two closely spaced "Metadata staging failed, result=0x80070490 for container '{00000000-0000-0000-FFFF-FFFFFFFFFFFF}'" errors reported by Microsoft-Windows-DeviceSetupManager/Admin .  There is also a "HNS-Network-Create :-
Network id = '{b95d0c5e-57d4-412b-b571-18a81a16e005}'.
Network type = 'ICS'.
Result code = '0x80070008'. " error, which you've seen, reported by Microsoft-Windows-Host-Network-Service-Admin.  There are three almost simultaneous errors "'IpICSHlpStopDnsServer' : '0x80070490'.", "'IpICSHlpStartDnsServer' : '0x80070008'." and again "'IpICSHlpStopDnsServer' : '0x80070490'." also reported by HSN.
This is probably irrelevant, but about a half hour before the ubuntu.exe error, with a reboot intervening, there is a "The hns service depends on the VfpExt service which failed to start because of the following error:
A device attached to the system is not functioning." error reported by System.  The vfpext service is currently running.

I wouldn't be surprised if that was the root cause of the error you're seeing indeed. Sadly I don't have deep knowledge of that component so I don't have insights on how to diagnose it

@TDavidHudson , thanks for the debugging on your side! 80070008 == not enough resources. Just to double-check, can you verify that you don't have something listening on UDP port 53? NAT will use an internal DNS server to proxy DNS requests, and it will require binding to UDP <IP_ADDRESS>:53 and ::0:53 (UDP port 53 on the ephemeral IPv4 and IPv6 addresses).

Port 53 appears not to be in use:

netstat -na|select-string :53

TCP    <IP_ADDRESS>:53661    <IP_ADDRESS>:443       ESTABLISHED
UDP    <IP_ADDRESS>:5353           :
UDP    <IP_ADDRESS>:5353           :
UDP    <IP_ADDRESS>:5353           :
UDP    <IP_ADDRESS>:5353           :
UDP    <IP_ADDRESS>:5353           :
UDP    <IP_ADDRESS>:53866        :

FWIW, I'm able to listen on UDP port 53 using PowerShell, so there shouldn't be a problem with permissions.

@TDavidHudson I was having the same issue and finally managed to get past it... are using Cisco AnyConnect VPN?

@zadeluca Cisco AnyConnect is certainly installed, but no VPN is active when I'm in-office.  I don't recall trying to run ubuntu.exe outside the office.  What was your workaround?

@TDavidHudson I tried installing using WSL1 (wsl --set-default-version 1) and then converting to WSL2 (wsl --set-version Ubunti-22.04 2). This also failed, but showed me a more specific error message similar to what shows in your logs (failed to create network or something). That was my clue that AnyConnect may be the culprit, it seems to mess with a lot of different things.
I stopped the AnyConnect services - nam and namlm. There is another service called vpnagent, but even as admin I was not able to stop that one. Anyway, with those 2 services stopped, the conversion to WSL2 worked for me. Hope this helps.

@zadeluca Thanks!  Stopping the "Cisco AnyConnect Network Access Manager" and "Cisco AnyConnect Network Access Manager Logon Module" services allowed running C:\Users...\AppData\Local\Microsoft\WindowsApps\ubuntu.exe in my elevated PowerShell.  No VPN was in use....  I'll see what Cisco has to say about this.

@TDavidHudson Glad I could help! You might also need this to make networking inside WSL function, but I'm not sure if it is only when the VPN is activated or not: https://github.com/sakai135/wsl-vpnkit

It is important to note for the workaround that those two Cisco services probably need to be restarted before shutting down, if not sooner (but see below).  Starting the laptop without having done so made it impossible for me to use the in-office network to access this web page or anything else external; I had to do a network repair inside Cisco AnyConnect after starting those services, then reboot, to fully recover the ability to use the network.  After restarting the services but before doing the repair, I lost even more network connectivity.

I'm leaving this issue open for a while in order to give Cisco time to react.


rate limit: Limit should apply to all routes

Exceeding the rate limit should return 403 for all further requests; the route of the request should not matter.
The current behavior is such that exceeding the rate limit will only 403 that particular route.

@jwdeitch This is a good PR, Could you resolve the conflict and update the PR?

Wouldn't this depend on whether the api throttle is placed in a route group or a particular route? If particular route or even group within larger group - then it shouldn't I would think apply to all routes?

Actually, the middleware should throw a http status 429 (too many requests) as indicated here ( https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429 )


Doc - Add windows binary download/install doc

Signed-off-by: kcoriordan<EMAIL_ADDRESS>What this PR does / why we need it

Instructions for installing TCE on Windows using binary download/install method were removed before launch in PR:
https://github.com/vmware-tanzu/community-edition/pull/1983. This PR adds those instructions back in.

Removed repetition in prereqs - trying to avoid repeating prereqs in the actual install procedures. The prereq table above the install procedure should be the one source of truth. For example, the prereqs that were in the procedure had a different docker link to the prereq table. The prereq table has the correct one which links to the supported version 4.2.0

Added an internal link to our 'Docker-based Clusters on Windows' topic

Made the intro text consistent across the Getting Started guide and the Install Tanzu Community Edition topic in the Installation doc

Details for the Release Notes (PLEASE PROVIDE)

Which issue(s) this PR fixes

Fixes: #2974
Describe testing done for PR
Run hugo server as per instructions in readme
Special notes for your reviewer

I think this is good to go @thesteve0 any other comments?
thanks


Validation step is skipped when on_train_batch_start returns -1

🐛 Bug
It seems that when on_train_batch_start returns -1 the validation step is skipped.
Expected behavior
The trainer should skip the training epoch and start the evaluation.
 - pytorch-lightning version 1.0.4
 - PyTorch Version (e.g., 1.0): 1.7
 - OS (e.g., Linux): Linux
 - How you installed PyTorch (`conda`, `pip`, source): conda
 - Python version: 3.7

Yes, this is the correct behavior as described in the docs: If you return -1 here, you will skip training for the rest of the current epoch. one epoch includes training + validation, so it will skip validation step as well

Yes, this is the correct behavior as described in the docs: If you return -1 here, you will skip training for the rest of the current epoch. one epoch includes training + validation, so it will skip validation step as well

I understand .. you will skip training for the rest of the current epoch. as is written in documentation as only training is skipped. Yes, I want to only skip training. I think either the expected behavior of this -1 is wrong or the documentation is wrong. Is there any way to only skip training?

I see! yep, docs can be improved!
you can now return None in training_step to skip the training_step
See Returns Section in https://pytorch-lightning.readthedocs.io/en/latest/lightning_module.html#training-step

I see! yep, docs can be improved!
you can now return None in training_step to skip the training_step
See Returns Section in https://pytorch-lightning.readthedocs.io/en/latest/lightning_module.html#training-step

Right! But I have a stop criterion that is satisfied, I need to save it in a variable and the trainer will still loop over all batches that will be skipped. It seems more natural to do have a mechanism to skip the training entirely from a hook. Otherwise, the loop will be quite a waste of time/computation.

Thank you for your feedback @nicola-decao
noted the feature to implement when applicable
UserWarning: training_step returned None if it was on purpose, ignore this warning...

Most of us normally don't skip the training batch, so it's just notifying that user is skipping on purpose.

Hi developers! I wonder if there is a feature implemented to skip the training epoch only?


Expression statements can't begin with {, even in quasi

While trying to debug
macro swap(a, b) { return quasi { my temp = {{{a}}}; {{{a}}} = {{{b}}}; {{{b}}} = temp; None } }; my w = 5; my z = 10; swap(w, z); say([w, z])
(the swap example from #194) I found that the error message seems to be due to the statement:expr token's definition being:
token statement:expr {
    <![{]>       # }], you're welcome vim
    <EXPR>
}

This causes issues in expressions like
macro foo (a, b) {return quasi { {{{a}}} + {{{b}}} } };

Where the statement expression {{{a}}} + {{{b}}} starts with { and so is not accepted by statement:expr.
This issue also occurs with objects:
{a: 2}

fails to parse.
I don't feel like I understand perl 6, the parser well enough to know how to fix this issue. I assume that <![{]> is there since we want to accept blocks (which start with {) as blocks rather than expressions, but I'm not sure.

Thanks! :+1:
Discussed this one with @sergot tonight. I think I have a proper fix for it, namely this:
@@ -64,7 +64,7 @@ grammar _007::Parser::Syntax {
         ['=' <EXPR>]?
     }
     token statement:expr {
-        <![{]>       # }], you're welcome vim
+        <!before <!before '{{{'> '{'>   # } }}}, you're welcome vim
         <EXPR>
     }
     token statement:block { <pblock> }

It passes all the tests on master.
But I'd like to (a) explain/explore why that line is there in the first place, and (b) write a regression test involving unquotes first in an expression.

Fix uploaded as https://github.com/masak/007/pull/289 — @dan-simon and @sergot, feel free to have a look when you have time.

This fix looks good to me.
We still have the analogous problem for object literals, but I don't think that can be fixed (for example, the empty object literal looks the same as the empty block). This being a problem is very rare, but it comes up in macros like
macro moo () {
    return quasi {{{
        {a: 2}
    }
}

In this case one can just surround the object in parentheses. But if we ever want, for example, object destructuring assignment, that won't work.

Agreed about the problem for object literals. They way that problem is solved outside of quasis is with this disambiguation rule:

If it's at the start of a statement, it's a block.
Otherwise, it's an object literal.

As rules go, I'm pretty comfortable with this one, given that we want to avoid backtracking and given the unfortunate circumstance that object literals and blocks share a starter.
The one interesting complication is that quasis try to be "minimal" in the sense that they evaluate to the syntactically "simplest" form: a Q::Expr, a Q::Statement, or a Q::StatementList, whichever one's the first possible.
But, first, that shouldn't affect the parse at all. Having the parse be influenced by such a minimality criterion — such as, for example, choosing to parse something as an empty object literal instead of an empty block because that would make it able to return a simpler form — would not be a good idea.
In fact, I don't know if that minimality rule is necessary anymore, given the most recent happenings around Q::Expr::BlockAdapter.


Bash style fixes.

Use /bin/bash everywhere as we're using Bashisms.
Quote all exported variables
Fix incorrect use of 2>&1 > to instead be &>
Fix syntax error

Thanks to @Gnouc for submitting these in #21.

Gonna make this POSIXy (and add CI checks) for this instead.

:grinning: :neckbeard:


Info window on every youtube video I play?

Everytime I open a new YT-Video I see the "go to settings, donate..." windows. And I can't remember that I set that to happen somewhere in the settings. Is that a desperate call of the developer to gain money? If yes, it's very contraproductive... If no and there is a setting to turn this off, I apologize. But there are too many settings and I can't find it...

Hello?

Still happens with a fresh installation of the greasemonkey script...

Hello?!

No one willing to help me with this one? I am the only one with this issue?

@GarfieldDeepClone It's possible that YouTube Center is not able to store its settings and therefore it will revert to the welcome dialog every time you visit YouTube. The issue is probably caused by you having cookies disabled in your browser, which in some rare cases might cause the issue. I will recommend you to use the Greasemonkey version of YouTube Center as a solution to this fix as it will then use the GM API, which hopefully will not have your issue.

I'm already using the Greasemonkey script. And cookies are accepted. This problem appeared at the time I reported it but I was using the script also before.
I also tried to uninstall the script and reinstall it, but it didn't help.

@GarfieldDeepClone which browser are you using including its version?

Firefox 45.0.1

No idea?

?

Hmmmm, seems like I have to look for an alternative, because this is getting annoying.


does bsseg can usef for Semantic segmentation

hi, all the code are instance seg ,does bsseg can usef for Semantic segmentation?

@zhangyunming Code for semantic segmentation will be released futher. Please stay tune.

ok . wait in hope


Bundler could not find compatible versions for gem "railties"

I update the rails6.0 and then it always showed this error:

You forget to paste the error.
Also, please include steps to reproduce the issue you're having.
Thanks!

I update bundle and then all is okay, thank for your reply @deivid-rodriguez

Yeah, I'm sorry I got to this so late.Thanks for your feedback @MarcSteven!


Compilation error: cannot find cudnn v7 header

Hi everybody
I was seeing a compilation error because my cudnn was version 6.
Looks like the ngraph master branch now needs cudnn v7.
I see in CMakeList.txt :
find_package(CUDNN 5 QUIET REQUIRED)
Should nt it be something like :
find_package(CUDNN 7 REQUIRED)
?
Kind
W.

Thanks for reporting this.
The cudnn support is still incomplete, but we'll get this particular problem fixed soon.

https://github.com/NervanaSystems/ngraph/pull/755


Output requires module which isn't in the dependency chain

I have the uuid module as a dependency, and I have browser: true in the nodeResolve plugin config. I can't tell 100% where this issue originates but in looking at the rollup-node-resolve code, it doesn't deal with anything but resolving files.
The uuid module has a uuid.js main file, which has require('./lib/rng').
However, there are two different lib files, one which gets loaded in the browser:
"browser": {"./lib/rng.js": "./lib/rng-browser.js"}
With rollup, my umd output somehow has require('crypto'))despite having included the rng-browser.js code. Since I'm using requirejs and crypto wasn't a module, I get:
require.min.js:1 Uncaught Error: Script error for "crypto"
The only reference to require('crypto') in the source files is the original ./lib/rng.js which was replaced, so I'm confused where the name crypto is coming from.
I do see Treating 'crypto' as external dependency during the build process, but uuid is the only module I use that references crypto - which I confirmed by building a file with only import uuid from 'uuid';. I've tried setting up exclude rules for that file but it just winds up including both rng lib files.

Please see https://gist.github.com/Rich-Harris/88c5fc2ac6dc941b22e7996af05d70ff

https://github.com/viveleroi/rollup-uuid-require-bug

Thanks, I understand the problem now. The issue here is that – due to some spectacularly unfortunate design decisions taken in the early days of Node – the commonjs plugin has to resolve  module IDs by itself, to account for foo meaning foo/index.js and other such nonsense. But it's not equipped to handle things like pkg.browser, because that's the job of node-resolve.
The solution is actually very straightforward – you just need to ensure that the node-resolve plugin runs before the commonjs plugin. The real question I'm wondering about is how to make that more obvious, because it requires quite a deep understanding of the order in which various plugin operations take place.

Closing in favour of https://github.com/rollup/rollup/issues/1148

Thanks, that solved the error. Until there's a fix to rollup it would be helpful to document this, or make it more prominent if it is and I just missed it.


How to create Users in LoRa App server

I have installed LoRa server and LoRa App server , accessed App server through http , but there is only option of adding devices , how can i create users in it?

This is currently not yet implemented, but is definitely something I'll be working on soon. Could you please create a feature request at https://github.com/brocaar/lora-app-server so you can track the progress (as this feature should be part of LoRa App Server, not LoRa Server)?

yeah sure. Thanks it will be very helpful

How to create feature request?

I meant, could you create an issue describing the feature request for the above repo :-)

ok :-)


dfi to bnb switch failed.

TxID: 948bdf2595346bd2630a94f2dd90d58a9148ff456fafba39c6ac8d39a6f2c68e
Destination-Adress (BNB) : 0xE04E12cD82CA3601A03483DFe621e698F2b9697e
It was my mistake, no BNB on destination. BNB left, moved BNB to destination-ID, continue, but failed.
Also the restore process ( extimatited fee 0,57 BNB changed to 0,001 BNB ) failed.
Could you pls check? ( 149,7 DFI we are talking abount )

I see you have received the DFI by now.
https://bscscan.com/tx/0x7a782e2add0bad2780e2b7c03acd1c1b9e81be1efc0919327a02e57d910a8ca3


systemd: fix upstream regression "Failed to copy smack label"

systemd: fix upstream regression "Failed to copy smack label"
The code introduced in f8c1a81c51 (= systemd 227) failed with:
Failed to copy smack label from net_cls to /sys/fs/cgroup/net_cls: No such file or directory
There is no need for a symlink in this case because source and target
are identical. The symlink() call is allowed to fail when the target
already exists. When that happens, copying the Smack label must be
skipped.
But the code also failed when there is a symlink, like "cpu ->
cpu,cpuacct", because mac_smack_copy() got called with
src="cpu,cpuacct" which fails to find the entry because the current
directory is not inside /sys/fs/cgroup. The absolute path to the existing
entry must be used instead.
Resubmitting from 01org main repo, replacing PR #78. This is necessary because external pull requests do not update the sstate cache, so when compilation takes too long, it will always take too long.

@mythi @ipuustin this test has passed the TravisCI checks, albeit not quite cleanly (some issue with incomplete sstate cache, the "pr" test wasn't restarted). In addition, I also tested locally.
Please review and merge when ready.

@pohly +1, thanks for sending fixes in upstream too! I'll merge.


Add support for streamlit password st.text_input type

For the st.text_input widget, streamlit supports password masking using the type='password' param (https://docs.streamlit.io/library/api-reference/widgets/st.text_input).  Any way to get support for this?

Hi @zabrewer ,
Do you mean to ask if, for the blocks, the text input could be set to mask whatever that is typed in?

Yes, you understand correctly.  I may have a work around by adding a streamlit widget (text input) and access the data from that input rather then in the barfi execution flow.

Yeah, that could be a work around.

You can close this @krish-adi - I think my work around works well.

Cool! Thanks.