Difference between AddDefaultIdentity and AddIdentity??

In scaffolding Identity I get this in ASP.NET Core 2.1:
  services.AddDefaultIdentity<ApplicationUser>()
                    .AddEntityFrameworkStores<ApplicationDbContext>();

Whereas the previous templates produced this
 services.AddIdentity<ApplicationUser, IdentityRole>()
       .AddEntityFrameworkStores<ApplicationDbContext>()
       .AddDefaultTokenProviders();

Which was clearer.
Is is possible to understand the difference please? What does the former has that the latter does not? Can I just replace the former by the latter?
Thanks

AddDefault uses a different set of apis, but does more or less the same thing except it removes roles functionality by default.
Roles is the only real difference between the two, they both return IdentityBuilders which can further customize identity.

@HaoK , I am happy to get rid of Roles, except that they seem to be generated in the sql db anyway. 1) 1) Why is that?

Could you be somewhat more precise on what you mean by "more or less"?

Why is
      .AddDefaultTokenProviders();
not there anymore?

You can look at the differences here: https://github.com/aspnet/Identity/blob/c7276ce2f76312ddd7fccad6e399da96b9f6fae1/src/UI/IdentityServiceCollectionUIExtensions.cs#L49

It doesn't affect the migration, it just doesn't register any role services by default.  Its not meant to replace any existing identity code.

This sugar is mostly for templates to use to collapse DefaultUI/DefaultTokenProviders

Sorry @HaoK , I did not fully understand your last post. My scenario is that I am not using the defaultUI, so do I need to add AddDefaultTokenProviders to the pippleine if I use AddDefaultIdentity, or is AddDefaultIdentity already taking care of that? I stumbled upon the link you gave before writing this issue and it was not clear enough to me, sorry.

Oh @HaoK my deepest apologies , I misread the return value of AddDefaultIdentity. It calls AddDefaultTokenProviders.

All good

If you still need to use Roles you can do this:
            services.AddDefaultIdentity<ApplicationUser>()                 .AddRoles<ApplicationRole>()                                 .AddEntityFrameworkStores<MyApplicationContext>();


Add DPRINT support for more types

uint16_t, uint8_t, etc.

More types (u)int8/16/64 added here: https://github.com/tenstorrent-metal/tt-metal/pull/5463


commits

Description
Made an accordion-style FAQ with lorem ipsum filler.  Will have to add our actual FAQ info.
Checklist
Remove any items which are not applicable.

[ ] I have performed a self-review of my own code

Good work with the material UI!!


Echo Template doesn't support dot

Create a project like "Project.Bot" is not supported because Accessors generates errors

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1f1718fa-0745-3fee-6ef5-18e36b61fabf
Version Independent ID: fd9ef97f-4540-e624-d459-c076e94a50ad
Content: Create a bot with the Bot Builder SDK for .NET - Bot Service
Content Source: articles/dotnet/bot-builder-dotnet-sdk-quickstart.md
Service: bot-service
Product: bot-framework
GitHub Login: @Kamrani
Microsoft Alias: kamrani

You can report tooling issue here: https://github.com/Microsoft/botbuilder-tools/issues.


incorrect routeros version in assets

for CVE-2017-6297 there is version 6.83.3 (exactly as in CVE description) however there is no such version of RouterOS available. There should be 6.38.3

Hi, thanks for highlighting this issue, but we are creating the JSON file by querying the NVD CVEs database. This mistake comes from this DB.


[BUG] When hovers on the tag, notice the server name will move up a little bit

Describe the bug
When hovers on the tag, notice the server name will move up a little bit
https://github.com/user-attachments/assets/849bf7ff-60df-45bd-a926-c82ec93956ee
Expected behavior
When moues over on the "In use" tag, only the tag changes to "Check details" link button, the server itself should no change/move.

still existed


Request: Allow setting Upstream TLS 1.2 in DLBs

At this point in time and according to the docs:
https://registry.terraform.io/providers/mulesoft-anypoint/anypoint/latest/docs/resources/dlb
Upstream TLS 1.2 (upstream_tlsv12 (Boolean)) is a read-only value. Would it be possible to set it up via terraform sometime in the future?
Thanks
cc: @rubycube @RamziChoueri2903 @pmi-snmiguel

@soufi as discussed

Hi, This will be carried along with the issue #38


Create DOI

Permanent DOI link so users can always reference our project pages even if we change our urls.
Data Object Id

Need to discuss use cases

Superseded by humancellatlas/dcp#226.


Dini Mueter...

HANI MUETTER GHÖRT?

This one will need a rebase after fixing the typo

Fixed and rebased!

You could make it case insensitive.

@rnestler you just pushed a bad solution into my pull request ;) JavaScript has a RegEx modifier for that. I'll overwrite.

Yeah, and now deploy, I want to be insulted :laughing:

I have an instance running at https://telegram.me/rnestler_flame_bot

Done.

Thanks!


Is the source of the "base" seed available?

Hi,
I'd like to modify the meta attributes of the HSFM to fit with my own project, for example, adding a new Attribute to HSFM.State.
This is currently not possible because the HSFM meta is a Library, which prevents me from inheriting/modifying it. I know any changes are likely to break Simulation/Code generation, but that's ok.
Can you provide the original project for the HSFM library, please?

Hey fnzr,
This should be what you need (it's in the meta folder): https://github.com/finger563/webgme-hfsm/blob/master/src/meta/meta.webgmex

Hey fnzr,
This should be what you need (it's in the meta folder): https://github.com/finger563/webgme-hfsm/blob/master/src/meta/meta.webgmex

https://github.com/finger563/webgme-hfsm/wiki/How to integrate HFSM into your WebGME project

https://github.com/finger563/webgme-hfsm/wiki/How to integrate HFSM into your WebGME project

I had found the seed file, but trying to import it (Create new project -> Import -> Import from file) fails with the error:
error: [gme:server:standalone:middleware:BlobServer:BlobBackend] readStream on close
error: [gme:server:standalone:middleware:BlobServer] Error: ReadStream was closed while writing file!

This is the server log. The client fails with a Internal Server Error exception. I tried both in the version of this repository, and in a clean install of the latest webgme. Same error.
Does importing works for you?

I had found the seed file, but trying to import it (Create new project -> Import -> Import from file) fails with the error:
error: [gme:server:standalone:middleware:BlobServer:BlobBackend] readStream on close
error: [gme:server:standalone:middleware:BlobServer] Error: ReadStream was closed while writing file!

This is the server log. The client fails with a Internal Server Error exception. I tried both in the version of this repository, and in a clean install of the latest webgme. Same error.
Does importing works for you?

That is a known webgme issue. Known in the sense that there is a workaround and the upcoming release will solve it.
For the time being try to use this version of webgme https://github.com/webgme/webgme/releases/tag/v2.42.1-alpha.
(https://github.com/webgme/webgme-engine/issues/243)

That is a known webgme issue. Known in the sense that there is a workaround and the upcoming release will solve it.
For the time being try to use this version of webgme https://github.com/webgme/webgme/releases/tag/v2.42.1-alpha.
(https://github.com/webgme/webgme-engine/issues/243)

@kecso I see. Getting the release of<EMAIL_ADDRESS>wasnt enough though, the error persisted. I had to pull the master branch of webgme-engine and use it instead of v2.25.2-alpha. Now the upload works, thanks.
@finger563 That's exactly what I was looking for, thank you!

@kecso I see. Getting the release of<EMAIL_ADDRESS>wasnt enough though, the error persisted. I had to pull the master branch of webgme-engine and use it instead of v2.25.2-alpha. Now the upload works, thanks.
@finger563 That's exactly what I was looking for, thank you!


Wrong label for semicolon comments

Two small things in the Download options:

Apostrophies is misspelled. It should be Apostrophes.
The Semicolon comment type is mislabeled as Apostrophies.

Some other suggestions:
It might be more natural to use the singular name of the comment character if there is only one character.
Hash # -- not plural since there is just one character at the beginning of the line
Slashes // -- keep the plural since there are two
Dashes -- -- keep plural
Apostrophe ' -- singular since there is only one on the line
Backticks multi-line ``` ```
Four spaces ␣␣␣␣ -- make this wider so it looks like four spaces in a monospaced font - or use the open box (U+2423) character?
Semicolon ; -- singular

Fixed in https://github.com/lewish/asciiflow/pull/214


enforceHTTPS=true causes deis client to bail

Based on the documentation, this should work but doesn't:
[user@lavos ssl]$ deisctl config router set enforceHTTPS=true
true
[user@lavos django_boilerplate|master]$ deis apps:create django-boilerplate
Creating application... Couldn't connect to the Deis Controller:
('Connection aborted.', gaierror(8, 'nodename nor servname provided, or not known'))
Make sure that the Controller URI is correct and the server is running.
[user@lavos ssl]$ deisctl config router set enforceHTTPS=false
false
[user@lavos django_boilerplate|master]$ deis apps:create django-boilerplate
Creating application... done, created django-boilerplate
Git remote deis added

Don't know if it's related, but enforceHTTPS seems to be causing issues in 1.5.0 as the redirects are not happening properly. Did you use http or https when you logged in to the deis client?

After looking into this more, I suspect it's related to #3438.

enforceHTTPS seems to work better in v1.5.0, it used to be that in order to get it to work in v1.4.1 I had to set it to true, then false.
In v1.5.0 everything else seems to be working great, except the client.
If enforcing HTTPS was working properly for the controller, it wouldn't matter what URL I logged in as or whether I configured SSL for the deis client.
#3438 seems relevant but for normal application access, enforceHTTPS is working for me.

Actually, I stand corrected... if I restart my routers I get behavior resembling #3438.


Accessibility Checker should check more than the landing page

Currently it appears that the accessibility checker is crawling the landing page only. Please determine if either of the following are possible:

A list of URL's can be provided to check (might be more performant)
A recursive crawl can be done for the entire application

@mcritchlow I updated the seeds to populate data into development environment and added a list of URL to check.  The course report url works fine too.

@VivianChu - great! Can you compile a list of those core accessibility issues for your test course (accesslint-ci scan http://localhost:3000/courses/1/Spring/2015/test_course_1) and search results? I think a ticket for addressing those would be a good next step since you have the tool working well now. At a quick glance, it seemed that many of the issues might be able to be addressed in the global templates. Hopefully :)


[Upgrades] Create Upgrade RHN Panel, Confirmation Panel and connect to upgradeToCorporate action + apply to Report Fields

Part of https://github.com/Expensify/Expensify/issues/391954
Individual steps to complete:

Create generic workspace upgrade page with dynamic title / description & icon
Create upgrade confirmation page
Connect the workspace upgrade page with the upgradeToCorporate policy action
Create flow for Report Fields as a feature

Now that Sibtain is back we're back on it here. Currently testing the FE with the BE locally.

@allroundexperts assigned you a small bug in the flow here

Payment summary:
Contributor: @allroundexperts $250
Reviewer: @ishpaul777 $250
@ishpaul777 offer out to you here: https://www.upwork.com/ab/applicants/1813979251771429535/hires?filter=ish&page=1

@garrettmknight Can we hold my payment as well till we are done with this feature? The first iteration went to staging and we have a few minor bugs to squash. Once everything is good, we can handle the payment.

Works for me @allroundexperts

No prob, dropping to weekly. Lemme know when you're ready.

Still waiting for payment - @ishpaul777 any update?

Yep - paid!


making sure scrapli cleanup doesnt break things!

howdy folks!
@hellt is already in the loop on some of this, but for anyone else interested -- I've been doing a bit of housekeeping on scrapligo things and made some breaking-ish changes; this draft (until I finalize things ive been working on) is pinned to a dev branch and contains the little fixes that will be necessary if ya'll want to update the scrapligo pin. I think I should be fairly close to being done changing/breaking things and I will update this branch w/ pin to master branch (or perhaps even a real release finally) once I get to that point.
Carl

super dooper @carlmontanari
thanks


more specificity on Steps

make it clearer who should click on the link to the undergraduate content
prevent 'false pessimism' about the need for certain GCSEs, following BAT/GIT data sharing UR playback
tweak language so it's more geared to people who are studying for a degree
remove repetition of 'GCSE grade 4 (C) or above' through bullet lead-in sentence
move undergrad link to the bottom as we think this content doesn't apply to a large part of our audience
link from the "if" statement rather than "undergraduate teacher training" so it's more screen reader friendly and so there's less potential to click there when it doesn't apply to you

I like it but am concious that we're mentioning 'equivalents' in each of the first three paragraphs

we could probably remove it from the 3rd para? maybe the message comes across enough already by that point?

@peteryates - have removed the 3rd 'or equivalent'


Difference between concoct and viewbuilder

The libraries concoct and viewbuilder look much the same from the outside, and none of the two seem to depend on the other. What is the difference between them?

Also, from the perspective of someone who is looking to contribute: Which one of the two should I rather focus on? Is there any Roadmap available to somehow gauge what should be worked on (next)?

I think a roadmap would be a great idea, I'll try to get one out ASAP. Basically the plan is to have Viewbuilder be a generic renderer for native GUIs and concoct as a frontend. I'm really hoping this crate can be used with multiple GUI frameworks and concoct can be used with multiple backends (like this and the web).
Right now the roadmap for the next release of Viewbuilder is:

Finish the Semantics and Layout trees
Create an element tree that uses the above trees
Reimplement rendering with skia

Also I recently changed direction and my current plan is to integrate with dioxus first to see if their react-style of UI is better. Definitely keeping concoct around as an option though so I'd love help with either.

Interesting! I definitely liked the Elm-inspired architecture of concoct and have made contact with this style of UI development through the Iced library in the past. However, I didn't end up using Iced, because it lacks accessibility support, which viewbuilder seems to have (<3 !!!). The HTML-like API of viewbuilder is really nice too, though! It definitely feels very intuitive and easy to use.
So, if I understand correctly, viewbuilder is supposed to be a GUI-Library, whereas concoct is supposed to be a frontend library?
By the way, if you'd like to connect on Discord/Matrix/etc., I'd be happy to do so! :)

Glad ya like it! I had the same path, I feel like every rust UI framework is missing something so I just wanted to start from the ground up with what wasn't there.
I'd love to talk more on discord! My username is matthunz#2901


Adding /etc/modprobe.d folder for module configuration

This enables users to create specific module configurations in the /etc/modprobe.d folder, similarly to /etc/modules-load.d
The folder is mounted and hence retains changes. It can be populated using a config USB stick or the CONFIG folder on the boot partition.

That is tested?

Hi,
I have tested it locally, but did not find any testing guidelines in the repo, or any automated testing for that matter. If you have any, please let me know.
BR,


Create single file provider plugin binaries

Hello!

Vote on this issue by adding a 👍 reaction
If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details
Consider using https://github.com/indygreg/PyOxidizer or similar to produce single file native executables, similar to how we use https://github.com/vercel/pkg in https://github.com/pulumi/pulumi-component-provider-ts-boilerplate. This way, users won't have to have Python installed and configured correctly to use the provider.

One challenge with PyOxidizer is it doesn't appear to support cross-compilation: https://github.com/indygreg/PyOxidizer/issues/95
Which means, for example, currently the macOS binaries would have to be built on macOS.

@justinvp I bumped into this project today. Another option for you to review if it might help.
https://nuitka.net/


Python binding: No module named 'topi'

This script
import tvm
import tvm.relay as relay

fails:
  File "test-onnx.py", line 4, in <module>
    import tvm.relay as relay

  File "/usr/local/lib/python3.6/site-packages/tvm/relay/__init__.py", line 27, in <module>
    from . import expr_functor

  File "/usr/local/lib/python3.6/site-packages/tvm/relay/expr_functor.py", line 24, in <module>
    from .op import Op

  File "/usr/local/lib/python3.6/site-packages/tvm/relay/op/__init__.py", line 20, in <module>
    from .op import get, register, register_schedule, register_compute, register_gradient, \

  File "/usr/local/lib/python3.6/site-packages/tvm/relay/op/op.py", line 19, in <module>
    import topi

ModuleNotFoundError: No module named 'topi'

because the topi submodule isn't installed.

Thanks for reporting the problem, as per https://docs.tvm.ai/install/from_source.html#python-package-installation we will need to install both topi and tvm as part of the installation.
For future troubleshooting related threads, please open a new thread on https://discuss.tvm.ai/ instead :)

The tvm package did install /usr/local/lib/libtvm_topi.so.
So what's wrong with the python binding then that it complains? It obviously wants something like topi.so or topi.py in its python base dir and it isn't installed.
This means that the package is broken.


Column sizer tries to access non existing rows

There might be an issue with the computation of the tailGetResult in useColumnSizer (4.0.0-alpha2) here:
https://github.com/glideapps/glide-data-grid/blob/v4.0.0-alpha2/packages/core/src/data-editor/use-column-sizer.ts#L75
I have a table with 1000 rows, but this line of code triggers requests in my getCellContent for rows 1001-1008. I assume that tailRows needs to be used here instead of computeRows, or?: https://github.com/glideapps/glide-data-grid/blob/v4.0.0-alpha2/packages/core/src/data-editor/use-column-sizer.ts#L71

Yup also doesn't account for trailing row correctly.
On Tue, Mar 29, 2022 at 3:37 PM Lukas Masuch @.***>
wrote:

There might be an issue with the computation of the tailGetResult in
useColumnSizer (4.0.0-alpha2) here:
https://github.com/glideapps/glide-data-grid/blob/v4.0.0-alpha2/packages/core/src/data-editor/use-column-sizer.ts#L75
I have a table with 1000 rows, but this line of code triggers requests in
my getCellContent for rows 1001-1008. I assume that tailRows needs to be
used here instead of computeRows, or?:
https://github.com/glideapps/glide-data-grid/blob/v4.0.0-alpha2/packages/core/src/data-editor/use-column-sizer.ts#L71
—
Reply to this email directly, view it on GitHub
https://github.com/glideapps/glide-data-grid/issues/232, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAHN226MAQ7YLKUWARWQFLVCOAZLANCNFSM5R73YMUA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Fixed in 4.0


Not firing check on AJAX page soft-refresh load.

I've got midnight working great, but it's not firing .midnight() again after an AJAX page load, even when calling the function again on complete. The elements that are being applied midnight aren't reloading on page change.

Hi @wearelfb! Try running $('#yourHeader').midnight('refresh').
It's an undocumented feature for that specific use case.

Sorry, forgot to respond to this -- Worked great. Thanks!


Error: 'NSLayoutAttribute' has been renamed to 'NSLayoutConstraint.Attribute'

Info
Value

Platform
iOS

Platform Version
iOS 12 Developer Beta

SnapKit Version
4.0.0

Integration Method
cocoapods

Issue Description
When using SnapKit 4.0.0 with Xcode 10 Beta and swift 4, the project can not be compiled successfully. The error message is 'NSLayoutAttribute' has been renamed to 'NSLayoutConstraint.Attribute'. I have tried to clean build folder, but still error.

The error is caused by API changes in iOS 12. The commit #516 may has solved the problem.
Any idea about a new release for iOS 12 ? The recent version 4.0 is released on Sep 2017.

Until this is solved with an official iOS 12 / Swift 4.2 release, you can add something like this to your Podfile to ensure SnapKit continues to use Swift 4.1:
swift_4_1_pod_targets = ['SnapKit']

post_install do | installer |
    installer.pods_project.targets.each do |target|
        if swift_4_1_pod_targets.include?(target.name)
            target.build_configurations.each do |config|
                config.build_settings['SWIFT_VERSION'] = '4.1'
            end
        end
    end
end


feat(derive): enable more boxed types to be #[diagnostic_source]

Hello again! Nice and small change this time :)
The #[diagnostic_source] attribute already worked for Box<dyn Diagnostic>, but not for the highly-related Box<dyn Diagnostic + Send> and Box<dyn Diagnostic + Send + Sync> types!
It looks like someone already solved this issue for std::error:Error with a macro_rules! implementation, so I've done the same here to let those new cases compile! Without those Borrow<dyn Diagnostic> impls, you end up with compiler errors coming from inside of the derive macro:

Let me know if you have any questions! I'll separately try to sort out those failing tests (unrelated to this change) in another PR :)

With some luck, #339 should fix the tests failing for everyone — I don't think there was a behavior change, I think the reference strings just got mangled somehow in a commit from three days ago.


Obliteration compatibility list.

https://docs.google.com/spreadsheets/d/1FSr7vXfhMcIb5JfZ2R6O6sY4Ki7yKjLFOyogvwHguwU/edit
Made from my Kyty compatibility template, I now set up a sheet for Obliteration!
Anyone can contribute! (The examples below for the features are from the Kyty spreadsheet.)

Statuses are autocapitalized and color-coded.
Spreadsheet is sorted by alphabetical name.

PS4 and PS5 are color coded (PS4 is light blue, PS5 dark blue.)

Game tests with the latest emulator version have green text, while older emulator versions have red text.

Compatibility counter updates in real time! (Uses an COUNTIF statement on the status region)

Changes color depending on if a game was tested less than a week ago, a week or longer, a month or longer, and a year or longer.

Making this as an issue as idk where you wanna put the link :P

I think we should create a static website instead of Google Spreadsheet and let's people contribute by create a PR. With this way it is easier to use and we can prevent bad people from destroying it.

I think we should create a static website instead of Google Spreadsheet and let's people contribute by create a PR. With this way it is easier to use and we can prevent bad people from destroying it.

Most likely a Jekyll page using Github Pages?

Something like that. It will even better we can use a full SPA framework like React.js or Vue.js but I'm not sure if GitHub Page support it.

Something like that. It will even better we can use a full SPA framework like React.js or Vue.js but I'm not sure if GitHub Page support it.

It does, as long as its a static page.

The main problem is both React and Vue required build tool to make development easier. For Jekyll GitHub will run for us but what about React or Vue?

https://github.com/gitname/react-gh-pages

That one use a manual step to deploy to GitHub Page. What I mean is we push page source to the repository and GitHub will build for us automatically like Jekyll.

https://dev.to/achukka/deploy-react-app-using-github-actions-157d
This requires admin perms for keys, so have fun with this .w.

This one looks good except one thing, which is allow Actions to access SSH key that capable pushing to every branches on the repository. Let's say someone create a PR and successfully extract this SSH key now they can destroy the whole repository.

Huh, PS5 tag? Does this mean this emulator, like Kyty, should be able to boot both PS4 and PS5 stuff?

No any plan for PS5 yet.

Huh, PS5 tag? Does this mean this emulator, like Kyty, should be able to boot both PS4 and PS5 stuff?

Not in a long while. We don't even have PS4 done yet XD

You can just use actions for deploying vuejs to pages https://github.com/marketplace/actions/vue-to-github-pages

Thanks for the information.

Updated the sheet haha

I got (maybe) a bad news. I need to find a full-time job within this month. What the meaning of this is once I started to work on my full-time job it is likely that I'll have some spare time to work on Obliteration only weekends.

I got (maybe) a bad news. I need to find a full-time job within this month. What the meaning of this is once I started to work on my full-time job it is likely that I'll have some spare time to work on Obliteration only weekends.

Well, you know what you must do, right? S p e e d r u n

Superseded by #418.


binary releases of cleos and keosd

Please note this will generate a dmg with cleos and keosd on Mac OSX, and a full install for DEB and RPM packages.  Without changing all of the install cmake commands to disable during installation when using CPack generation, the work around is that it installs cleos and keosd to what CPack uses for it's default bin path (/usr/bin for both it seems) and everything else to /usr/local/eosio like normal.  This at least doesn't expose the other binaries unless someone adds /usr/local/eosio to their path.  sudo apt install  and sudo apt remove eos.io work cleanly.  And yum installs and uninstalls cleanly too.

Superseded by #6003


Remove min value count from properties when using sparse_max

~Set is_ragged=False if value_count min=max in loader backend.~
If https://github.com/NVIDIA-Merlin/core/pull/111 is merged, this will fail because a check has been introduced that if is_ragged=True, that value_count min != max
This PR removes the min from the value count specfied (which was previously being set to the same as the max value). In the case of this code. the intention is that if we're specifying sparse (ragged) columns, then we know they're ragged, and min value count != max value count.

@oliverholworthy Does this also need to go into the dataloader package?

rerun tests

@oliverholworthy Does this also need to go into the dataloader package?

This augment schema logic is only in NVTabular it seems. Or did you mean: does this functionality need to be moved into the dataloader?

yes it does, otherwise it will break. https://github.com/NVIDIA-Merlin/dataloader/blob/main/merlin/loader/loader_base.py#L105

Then that should be the only changes necessary in the dataloader repo. Sorry for posting earlier seems @oliverholworthy already had that done.

rerun tests

Looks like this has some linting issues, but LGTM once those are resolved


python312Packages.aiortm: 0.9.38 -> 0.9.42

Diff: https://github.com/MartinHjelmare/aiortm/compare/refs/tags/v0.9.38...v0.9.42
Changelog: https://github.com/MartinHjelmare/aiortm/blob/v0.9.42/CHANGELOG.md

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 362604

x86_64-linux

:white_check_mark: 2 packages built:

python312Packages.aiortm
python312Packages.aiortm.dist


Feature: fully add support for table_names paging

SDK
Rust
Description
The table_names method in the python remote client supported paging, but this was undocumented, and only supported on the remote client.  As a result, the parameters didn't get ported over when moving the python client to rust.  We should document it and add a simple in-memory implementation for local tables and then add support for the remote rust client.
Furthermore, until this is fixed, the remote client will only grab the first 10 items from LanceDb Cloud.  This means this feature is probably part of the new API MVP.

Closed by #1059


Added 3 more MC authTitles

I was very bored and went on a hunt for these to publish here.
All of them is live flow, tested, and works just fine for me.
If any issues or questions, let me know.

Can you add these to the index.d.ts file as well: https://github.com/PrismarineJS/prismarine-auth/blob/97f2053a0d14bfec45455873c3ecbd3bad609a36/index.d.ts#L106

Thanks


terrafor az in windows  not working  - Error: json: cannot unmarshal number into Go value of type string

I tried with linux machine and it works fine as expected. same code in windows (using az cli/terraform) its not working and gives the below error.
Az cli  version 2.56.0
Terraform CLI version 1.6.0
OS :  Windows 11

Planning failed. Terraform encountered an error while generating this plan.

╷

│ Error: Unexpected External Program Results

│

│   with data.external.get_access_token,

│   on providers.tf line 20, in data "external" "get_access_token":

│   20:   program = ["az", "account", "get-access-token", "--resource", "api://${local.ipam_apiId}"]

│

│ The data source received unexpected results after executing the program.

│

│ Program output must be a JSON encoded map of string keys and string values.

│

│ If the error is unclear, the output can be viewed by enabling Terraform's

│ logging at TRACE level. Terraform documentation on logging:

│ https://www.terraform.io/internals/debugging

│

│ Program: C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\az.cmd

│ Result Error: json: cannot unmarshal number into Go value of type string
Debugging | Terraform | HashiCorp Developer
Terraform has detailed logs which can be enabled by setting the TF_LOG environment variable to any value. This will cause detailed logs to appear on stderr

Hi @Veljen ,
We have already detected this problem in some of my implementations and it is due to the fact that the latest versions of az cli return additional information related to the access-token that is not directly serializable to JSON.
You can modify the az cli command to return only the required value, the access-token, as show below,

and then you must also modify slightly the terraform code that uses it

I hope it has been helpfu


ASCII only output during training

Big fan of your work here
I noticed that in the initial steps of training donut was outputting some Japanese characters and Chinese characters.
Which is ideal for multi lingual information extraction, But I was wondering if can we limit the output charecters of donut to only ASCII, so that the training will be much more smoother and inference time will also be less as there will be less charecters to classify on
(when dealing with english only scenarios)

Hi, thank you for your interest on our work :)
Let me get a quick/short answer first -> Yes, it would be possible by removing unnecessary tokens in the vocabulary of the tokenizer. (this will not require any additional training)

https://github.com/clovaai/donut/blob/e6623ad56c0e9f12a426dab2d8b2d65a39d64689/donut/model.py#L159-L161
Can I change the pretrained tokenizer from "hyunwoongko/asian-bart-ecjk" to "hyunwoongko/asian-bart-en". The later one is an english only decoder from the same repo, would that do the trick? Cause I am not sure on how to change the vocabulary

How can we remove these unnecessary tokens? Could you please send the part of the script we must change in order to do that?

https://github.com/clovaai/donut/blob/e6623ad56c0e9f12a426dab2d8b2d65a39d64689/donut/model.py#L159-L161
Can I change the pretrained tokenizer from "hyunwoongko/asian-bart-ecjk" to "hyunwoongko/asian-bart-en". The later one is an english only decoder from the same repo, would that do the trick? Cause I am not sure on how to change the vocabulary

Yes, you can do that by writing "hyunwoongko/asian-bart-en".
You can check this repo to see how to specify other models:
https://github.com/hyunwoongko/asian-bart

https://github.com/clovaai/donut/blob/e6623ad56c0e9f12a426dab2d8b2d65a39d64689/donut/model.py#L159-L161
Can I change the pretrained tokenizer from "hyunwoongko/asian-bart-ecjk" to "hyunwoongko/asian-bart-en". The later one is an english only decoder from the same repo, would that do the trick? Cause I am not sure on how to change the vocabulary

Yes, you can do that by writing "hyunwoongko/asian-bart-en". You can check this repo to see how to specify other models: https://github.com/hyunwoongko/asian-bart

Sure, will give that a shot and let you know

Tried the above mentioned change but still observed other lang charecters in prediction during intermediate epochs
Prediction: Examination Examination Generation: Generation: General Examination: GENERAL EXamination: GENERAL APPEANANACE normal, pleasant, pleasant, well nourished, well developed, in no acute distress. اساسي, no suspicious lesions, normal, no rashes。讓他們, in no suspicious lesions, in no rashes。讓他們, in no suspirum, in no europeiskal評估,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們,讓他們     Answer: <s_name>GAMBOA DELFINO</s_name><s_dob>12/24/1946</s_dob><s_account number>12727</s_account number><s_reason>1 MED REFILL</s_reason><s_assesments>Essential (primary) hypertension - I10 (Primary)<sep/>Unspecified atrial fibrillation - I48.91<sep/>Tachycardia, unspecified - R00.0<sep/>Impaired fasting glucose - R73.01</s_assesments>  Normed ED: 0.862521891418564

I think there are many options to implement this feature. First one is to remove unnecessary tokens in the vocabulary. For this, you should update vocabulary of the tokenizer and corresponding embedding weights in the model (a.k.a. token/word embeddings). Or, just simply, using bad_words_ids feature in generate might more simple. Listing all undesirable token ids in bad_words_ids will prevent the issue. I hope this comment is helpful to you :)

Will try it out, thanks for the tip!

@gwkrsrch Thank you for the great work. I have got a follow-up enquiry regarding the above issue.
Currently, the decoder of the model seemed to generate arbitrary repetitions of the cls_token (i.e. < s >) while training on DocVQA task.
As such, I have tried adding the cls_token to one of the bad_words_ids but the model continued to generate < s > in between its <s_answer> and </s_answer> tokens.
decoder_output = self.decoder.model.generate(decoder_input_ids=prompt_tensors,
encoder_outputs=encoder_outputs,
max_length=self.config.max_length,
early_stopping=True,
pad_token_id=self.decoder.tokenizer.pad_token_id,
eos_token_id=self.decoder.tokenizer.eos_token_id,
use_cache=True,
num_beams=1,
bad_words_ids=[[self.decoder.tokenizer.unk_token_id],[self.decoder.tokenizer.cls_token_id]],
return_dict_in_generate=True,output_attentions=return_attentions,)

Any advice / clarification on using bad_words_ids for the donut use-case would be much appreciated.


[Search] Update changelog for "11.5.0-beta.2" release

Contributing to the Azure SDK
Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.
For specific information about pull request etiquette and best practices, see this section.

API change check
API changes are not detected in this pull request.


improve search_embeddings()

Since this platform mainly consists of researchers involved in academic studies, we can discuss academic issues.
In the search_embeddings( ) function, only similarity sorting and selection of the top n=3 are done. Is there enough reason for this, or does it need to be improved?
I think it is possible to make a selection based on the distribution of similarity. We can select up to the highest point, and if it is a unimodal distribution, the top three should be the corpus near the peak. However, if the similarity distribution is not unimodal, like 2 peaks, selecting the top three directly may cause damage to the corpus, and it may be necessary to select the corpus near the peak and the surrounding area.
One possible method is to select the corpora based on the distribution of similarity. If the similarity distribution is unimodal, we can select the corpora near the peak of the similarity distribution, which should be the ones with the highest similarity. If the similarity distribution is not unimodal, we can select the corpus near the peak and its surrounding area to preserve the overall shape of similarity.
Selecting corpora based on similarity, the most complex (but maybe the best) approach may be to use a Transformer attention mechanism on sentence or paragraph level embedding-vectors to select appropriate reference corpora based on content.

Since this platform mainly consists of researchers involved in academic studies, we can discuss academic issues.
In the search_embeddings( ) function, only similarity sorting and selection of the top n=3 are done. Is there enough reason for this, or does it need to be improved?
I think it is possible to make a selection based on the distribution of similarity. We can select up to the highest point, and if it is a unimodal distribution, the top three should be the corpus near the peak. However, if the similarity distribution is not unimodal, like 2 peaks, selecting the top three directly may cause damage to the corpus, and it may be necessary to select the corpus near the peak and the surrounding area.
One possible method is to select the corpora based on the distribution of similarity. If the similarity distribution is unimodal, we can select the corpora near the peak of the similarity distribution, which should be the ones with the highest similarity. If the similarity distribution is not unimodal, we can select the corpus near the peak and its surrounding area to preserve the overall shape of similarity.
Selecting corpora based on similarity, the most complex (but maybe the best) approach may be to use a Transformer attention mechanism on sentence or paragraph level embedding-vectors to select appropriate reference corpora based on content.

I agree with you. However, I think the computational cost is another factor we need to focus. For this, the most simplistic approach I am considering is to add the GPTIndex for a better extraction in my fork.

Update: directly use GPTIndex has not outperformed the method of feeding embeddings with GPT-3/3.5.

To maximize the use of each conversation, the number of RESULTS can be dynamically selected.
As long as the total length does not exceed a threshold, they can be included as references.
ChatGPT will decide for itself whether these references are useful
length_max=3000
results = df.sort_values("similarity", ascending=False, ignore_index=True)
# make a dictionary of the the first N results with the page number as the key and the text as the value. The page number is a column in the dataframe.
# Calculate the cumulative sum of df["length"].
# When the cumulative sum > length_max, stop the accumulation and use the result of the accumulation as the value of n
results['length_cumsum'] = results['length'].cumsum()
results = results[results['length_cumsum'] <= length_max]
n=len(results)

To maximize the use of each conversation, the number of RESULTS can be dynamically selected. As long as the total length does not exceed a threshold, they can be included as references. ChatGPT will decide for itself whether these references are useful
length_max=3000
results = df.sort_values("similarity", ascending=False, ignore_index=True)
# make a dictionary of the the first N results with the page number as the key and the text as the value. The page number is a column in the dataframe.
# Calculate the cumulative sum of df["length"].
# When the cumulative sum > length_max, stop the accumulation and use the result of the accumulation as the value of n
results['length_cumsum'] = results['length'].cumsum()
results = results[results['length_cumsum'] <= length_max]
n=len(results)

Hi, I just implement this in my fork. Also. I split the main text and the reference to get a better search scope.

To maximize the use of each conversation, the number of RESULTS can be dynamically selected. As long as the total length does not exceed a threshold, they can be included as references. ChatGPT will decide for itself whether these references are useful
length_max=3000
results = df.sort_values("similarity", ascending=False, ignore_index=True)
# make a dictionary of the the first N results with the page number as the key and the text as the value. The page number is a column in the dataframe.
# Calculate the cumulative sum of df["length"].
# When the cumulative sum > length_max, stop the accumulation and use the result of the accumulation as the value of n
results['length_cumsum'] = results['length'].cumsum()
results = results[results['length_cumsum'] <= length_max]
n=len(results)

Hi, I just implement this in my fork. Also. I split the main text and the reference to get a better search scope (Although this splitting is a bit of naive).

@MrPeterJin
If the PDF file is large, it can easily cause errors. I'm don't know why.
Also, since your fork is mainly running locally, can you directly save df.to_pickle as a pickel file with the same name as the PDF? This way, next time I read the same PDF, I can skip the embedding process.
@mukulpatnaik used redis, but I don't know where it save the db

@MrPeterJin If the PDF file is large, it can easily cause errors. I'm don't know why. Also, since your fork is mainly running locally, can you directly save df.to_pickle as a pickel file with the same name as the PDF? This way, next time I read the same PDF, I can skip the embedding process. @mukulpatnaik used redis, but I don't know where it save the db

For errors caused by large documents, it does have a high probability to yield errors. I think a paper has this amount of data would have various layouts (For example, you may refer to pages 21-22 of this paper, its main part is nearly only images), which is not able for handling by my fork.

@MrPeterJin
In paper-qa
https://github.com/whitead/paper-qa/blob/main/paperqa/readers.py
He use "pypdf", will it be better than pdfplumber?

@MrPeterJin If the PDF file is large, it can easily cause errors. I'm don't know why. Also, since your fork is mainly running locally, can you directly save df.to_pickle as a pickel file with the same name as the PDF? This way, next time I read the same PDF, I can skip the embedding process. @mukulpatnaik used redis, but I don't know where it save the db

@goldengrape
I have implemented saving embeddings as pickle file. please check local.py for reference.
As for PyPDF, it cannot read pdfs whose language is other than English. That's why I switched to pdfplumber.

@MrPeterJin
or PyMuPDF?
https://github.com/pymupdf/PyMuPDF/issues/329

@MrPeterJin or PyMuPDF? pymupdf/PyMuPDF#329

Okay... Seems pdfplumber also supported reading images, so I consider to refactor my code to overcome this issue.

Maybe that's why ChatPDF.com totally give up to display pdf

@MrPeterJin If the PDF file is large, it can easily cause errors. I'm don't know why. Also, since your fork is mainly running locally, can you directly save df.to_pickle as a pickel file with the same name as the PDF? This way, next time I read the same PDF, I can skip the embedding process. @mukulpatnaik used redis, but I don't know where it save the db

For errors caused by large documents, it does have a high probability to yield errors. I think a paper has this amount of data would have various layouts (For example, you may refer to pages 21-22 of this paper, its main part is nearly only images), which is not able for handling by my fork.

@goldengrape
You may test my new fork. I fixed the errors in this long paper. Maybe it would work on your long papers?

@MrPeterJin
I understand the reason for my previous failure now. It's because I was testing with 'books' instead of 'research papers'. For books, the formatting may be more complex and errors can occur with book titles. Now that I've limited the PDFs to research papers, there are no issues.


(PUP-7098) Updated acceptance test for utf-8 characters in catalog

The original version of this test was modified to set the environment
variable LANG to "en_US.UTF-8" on each call to on() and apply_manifest_on().
There are some VM pooler templates that to not set this by default.
In addition, the utf-8 string is now defined by using the unicode values.

CLA signed by all contributors.

@seangriff can you post link to the output of a successful ad-hoc run before we merge this?

@MosesMendoza
successful output of ad-hoc run:
https://gist.github.com/seangriff/ff53e8b17ac82b8605b0b0bd18f327d3


chore: remove info key

Checklist:

[x] I have read and followed the contribution guidelines.
[x] I have read and followed the how to open a pull request guide.
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

https://github.com/freeCodeCamp/freeCodeCamp/pull/49286 removed the last place this was used.

Is this automagic for the other locales?

Is this automagic for the other locales?

Should be. Even if they don't, i18n will not try to get it.


Log pings similar to how receiving pongs are logged

I noticed an asymmetry in the debug logs, namely that "server - received solicited pong" is logged, but sending the ping isn't. I think it would be a little user-friendlier if the outgoing ping appeared similarly. That way people could match them more easily.
(The lower-level write_frame() log message does occur in the outgoing direction, but it's not obvious whether it corresponds to the ping. Also, in the incoming direction, you do have two messages: both the lower-level read_frame() message, as well as the pong message.)

When sending a ping, there's nothing else to say than "we're sending a ping", which is currently covered by the almost-network-level debug logs.
Where receiving a pong, there are two pieces of information: "we're receiving a pong" and "does this pong match a ping we sent earlier?"
My interpretation of your suggestion is the following: add more human-friendly versions of the debug logs, like "sending a text frame / ping / pong / etc." in addition to the low level codes which, indeed, don't make a ton of sense if you don't know the binary opcodes of the various types of frames.
An intermediary solution would be to display the name of the opcode i.e. OP_PING instead than 0x09 (which the current version does, I think).

My interpretation of your suggestion is the following: add more human-friendly versions of the debug logs, like "sending a text frame / ping / pong / etc."

Yes, that's what I was suggesting.

While I made this a goal of #960, it's somewhat independant, so I'll keep this issue open for now.


Benchmarks cli

Description
Introduces a command line utility to automatically run benchmarks using the data set found in https://github.com/openforcefield/protein-ligand-benchmark/
Motivation and context

The idea is having a way to automatically monitor and validate real-life free energy calculations results when we change something in perses.

Resolves #854
How has this been tested?

Manually tested (it is intended to be run manually but we can think about writing some automated tests for it if needed)
Change log

CLI tool for running standard protein-ligand benchmarks from https://github.com/openforcefield/protein-ligand-benchmark/

TODO

[x] automate executing benchmarks from https://github.com/openforcefield/protein-ligand-benchmark
[x] robust way for requesting files (http request)
~[ ] Specify forcefields xml files to use via cli~ (Pushed for a future API enhancement release with this change)
[x] Basic automated analysis of results
[x] Analysis plots/visualizations

A few comments after trying to run this!
Would be great to:

Have this organize the files it generates into directories
Neatly organize the distilled CSV data for plotting into a separate directory and separate plotting script
Provide a cleanup.sh script that cleans up all the files generated

@mikemhenry : Looks like we need to add arsenic to the dependencies upon release:
$ python benchmark_analysis.py
Warning: importing 'simtk.openmm' is deprecated.  Import 'openmm' instead.
hostname
Traceback (most recent call last):
  File "/lila/data/chodera/chodera/perses/perses/benchmarks/benchmark_analysis.py", line 22, in <module>
    from openff.arsenic import plotting, wrangle
ModuleNotFoundError: No module named 'openff.arsenic'

I'll add this to the dev environment, but let's be sure to add it to the conda-forge recipe feedstock.

Yes, I'll also add it to 0.9.2, then if we ever need to use that version to check a benchmark it will be easier to install

Some of these jobs appear to have crashed when they made it to the complex phase with this puzzling error:
022-01-29 01:39:31,361:(0.00s):openmmtools.multistate.multistatesampler:********************************************************************************
2022-01-29 01:39:31,362:(0.00s):openmmtools.multistate.multistatesampler:Iteration 1/5000
2022-01-29 01:39:31,362:(0.00s):openmmtools.multistate.multistatesampler:********************************************************************************
2022-01-29 01:39:31,362:(0.00s):mpiplus.mpiplus:Single node: executing <function ReplicaExchangeSampler._mix_replicas at 0x2ac7fe2043a0>
2022-01-29 01:39:31,362:(0.00s):openmmtools.multistate.replicaexchange:Mixing replicas...
2022-01-29 01:39:31,362:(0.00s):openmmtools.utils:Mixing of replicas took    0.001s
2022-01-29 01:39:31,363:(0.00s):openmmtools.multistate.replicaexchange:Accepted 8568/16000 attempted swaps (53.5%)
2022-01-29 01:39:31,363:(0.00s):openmmtools.multistate.multistatesampler:Propagating all replicas...
2022-01-29 01:39:31,363:(0.00s):mpiplus.mpiplus:Running _propagate_replica serially.
Traceback (most recent call last):
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/cache.py", line 445, in get_context
    context = self._lru[context_id]
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/cache.py", line 147, in __getitem__
    entry = self._data.pop(key)
KeyError: (-2109681550591862395,<PHONE_NUMBER>075896928)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/lila/data/chodera/chodera/perses/perses/benchmarks/run_benchmarks.py", line 176, in <module>
    run_relative_perturbation(lig_a_index, lig_b_index, reverse=is_reversed)
  File "/lila/data/chodera/chodera/perses/perses/benchmarks/run_benchmarks.py", line 92, in run_relative_perturbation
    run(new_yaml)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/perses/app/setup_relative_calculation.py", line 911, in run
    hss_run.extend(setup_options['n_cycles'])
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/multistate/multistatesampler.py", line 728, in extend
    self.run(n_iterations)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/multistate/multistatesampler.py", line 681, in run
    self._propagate_replicas()
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/utils.py", line 90, in _wrapper
    return func(*args, **kwargs)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/multistate/multistatesampler.py", line 1196, in _propagate_replicas
    propagated_states, replica_ids = mpiplus.distribute(self._propagate_replica, range(self.n_replicas),
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/mpiplus/mpiplus.py", line 512, in distribute
    all_results = [task(job_args, *other_args, **kwargs) for job_args in distributed_args]
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/mpiplus/mpiplus.py", line 512, in <listcomp>
    all_results = [task(job_args, *other_args, **kwargs) for job_args in distributed_args]
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/multistate/multistatesampler.py", line 1223, in _propagate_replica
    mcmc_move.apply(thermodynamic_state, sampler_state)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/mcmc.py", line 1114, in apply
    super(LangevinDynamicsMove, self).apply(thermodynamic_state, sampler_state)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/mcmc.py", line 655, in apply
    context, integrator = context_cache.get_context(thermodynamic_state, integrator)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/cache.py", line 447, in get_context
    context = thermodynamic_state.create_context(integrator, self._platform, self._platform_properties)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/states.py", line 1172, in create_context
    return openmm.Context(system, integrator)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmm/openmm.py", line 16305, in __init__
    _openmm.Context_swiginit(self, _openmm.new_Context(*args))
openmm.OpenMMException: Error initializing FFT: 5

I'll investigate further!

I've run into another stochastic failure:
2022-01-29 20:31:00,950:(0.00s):openmmtools.multistate.multistatesampler:Iteration 2036/5000
2022-01-29 20:31:00,950:(0.00s):openmmtools.multistate.multistatesampler:********************************************************************************
2022-01-29 20:31:00,951:(0.00s):mpiplus.mpiplus:Single node: executing <function ReplicaExchangeSampler._mix_replicas at 0x2b435e754430>
2022-01-29 20:31:00,951:(0.00s):openmmtools.multistate.replicaexchange:Mixing replicas...
2022-01-29 20:31:00,951:(0.00s):openmmtools.utils:Mixing of replicas took    0.000s
2022-01-29 20:31:00,951:(0.00s):openmmtools.multistate.replicaexchange:Accepted 458/3456 attempted swaps (13.3%)
2022-01-29 20:31:00,951:(0.00s):openmmtools.multistate.multistatesampler:Propagating all replicas...
2022-01-29 20:31:00,951:(0.00s):mpiplus.mpiplus:Running _propagate_replica serially.
2022-01-29 20:31:01,097:(0.15s):openmmtools.mcmc:Potential energy is NaN after 0 attempts of integration with move LangevinSplittingDynamicsMove Attempting a restart...
Traceback (most recent call last):
  File "/lila/data/chodera/chodera/perses/perses/benchmarks/run_benchmarks.py", line 176, in <module>
    run_relative_perturbation(lig_a_index, lig_b_index, reverse=is_reversed)
  File "/lila/data/chodera/chodera/perses/perses/benchmarks/run_benchmarks.py", line 92, in run_relative_perturbation
    run(new_yaml)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/perses/app/setup_relative_calculation.py", line 911, in run
    hss_run.extend(setup_options['n_cycles'])
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/multistate/multistatesampler.py", line 728, in extend
    self.run(n_iterations)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/multistate/multistatesampler.py", line 681, in run
    self._propagate_replicas()
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/utils.py", line 90, in _wrapper
    return func(*args, **kwargs)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/multistate/multistatesampler.py", line 1196, in _propagate_replicas
    propagated_states, replica_ids = mpiplus.distribute(self._propagate_replica, range(self.n_replicas),
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/mpiplus/mpiplus.py", line 512, in distribute
    all_results = [task(job_args, *other_args, **kwargs) for job_args in distributed_args]
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/mpiplus/mpiplus.py", line 512, in <listcomp>
    all_results = [task(job_args, *other_args, **kwargs) for job_args in distributed_args]
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/multistate/multistatesampler.py", line 1223, in _propagate_replica
    mcmc_move.apply(thermodynamic_state, sampler_state)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/mcmc.py", line 1114, in apply
    super(LangevinDynamicsMove, self).apply(thermodynamic_state, sampler_state)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/mcmc.py", line 664, in apply
    sampler_state.apply_to_context(context, ignore_velocities=self.reassign_velocities)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmmtools/states.py", line 2272, in apply_to_context
    context.setPositions(self._unitless_positions)
  File "/lila/home/chodera/miniconda/envs/perses-dev/lib/python3.9/site-packages/openmm/openmm.py", line 16035, in setPositions
    return _openmm.Context_setPositions(self, positions)
openmm.OpenMMException: Error downloading array posq: CUDA_ERROR_ILLEGAL_ADDRESS (700)
terminate called after throwing an instance of 'OpenMM::OpenMMException'
  what():  Error deleting array param1: CUDA_ERROR_ILLEGAL_ADDRESS (700)
/scratch/lsftmp/1643482489.4935758.16.shell: line 30: 60319 Aborted                 (core dumped) python run_benchmarks.py --target tyk2 --edge $(( $LSB_JOBINDEX - 1 ))

I think we need to protect against OpenMMExceptions with a try...except block. Ideally, we could figure out a way to put this within the MultiStateSampler machinery, or even within the ContextCache machinery.

@ijpulidos : Is there currently a way to restart calculations that die?

@jchodera I just re-run the command and it should automatically pick it up from the previous checkpoint.

I've also faced a similar issue with core being dumped in some of my runs. I'm not sure what's causing those.

There appear to be some issues with how the run_benchmarks.py script is written: If each script is supposed to handle a transformation, what happens when this code runs concurrently for all transformations?
target_dir = targets_dict[target]['dir']
pdb_url = f"{base_repo_url}/raw/master/data/{target_dir}/01_protein/crd/protein.pdb"
pdb_file = retrieve_file_url(pdb_url)

# Fetch cofactors crystalwater pdb file
# TODO: This part should be done using plbenchmarks API - once there is a conda pkg
cofactors_url = f"{base_repo_url}/raw/master/data/{target_dir}/01_protein/crd/cofactors_crystalwater.pdb"
cofactors_file = retrieve_file_url(cofactors_url)

# Concatenate protein with cofactors pdbs
concatenate_files((pdb_file, cofactors_file), 'target.pdb')

# Fetch ligands sdf files and concatenate them in one
# TODO: This part should be done using plbenchmarks API - once there is a conda pkg
ligands_url = f"{base_repo_url}/raw/master/data/{target_dir}/00_data/ligands.yml"
with fetch_url_contents(ligands_url) as response:
    ligands_dict = yaml.safe_load(response.read())
ligand_files = []
for ligand in ligands_dict.keys():
    ligand_url = f"{base_repo_url}/raw/master/data/{target_dir}/02_ligands/{ligand}/crd/{ligand}.sdf"
    ligand_file = retrieve_file_url(ligand_url)
    ligand_files.append(ligand_file)
# concatenate sdfs
concatenate_files(ligand_files, 'ligands.sdf')

Presumably, we want to break this into multiple stages

retrieve files needed for one or more benchmark system(s)
set up all transformations in parallel
run or resume all transformations in parallel
analyze data to generate plots

We'll want to find a more clever way to refactor this in the next release so we can support benchmarking multiple targets at the same time as well.

Thanks!

I've also faced a similar issue with core being dumped in some of my runs. I'm not sure what's causing those.

We should

Attempt to figure out the underlying cause with the HPC folks. In this case, this was ls15, CUDA device 0.
Implement some level of protection against these failures. One option is to wrap the whole simulation in a try...except block. That might be the most conservative, safest option, since we don't know how many CUDA contexts need to be invalidated if this occurs.

After correcting a few bugs in the conversion of experimental data, we're now seeing excellent agreement with experiment when using openff-2.0.0. Well done, everyone!

tyk2_arsenic - 5 ns - 12 states.csv

It looks like 1 ns/replica for 12 states is also doing very well. The RMSE is only slightly degraded to 0.75 [0.55, 1.01] kcal/mol.
We might want to have our analysis scripts plot the performance for various simulation lengths (e.g. every 1 ns/replica).

tyk2_arsenic - 1 ns - 12 states.csv

We’ll need to do more work remediating protein structures https://github.com/openforcefield/protein-ligand-benchmark/issues/20 before we can do more benchmarking for future releases, but I also managed to run mcl1.
Clearly more work to do here, but results are  concordant with the Gapsys and de Groot paper (Gapsys ΔΔG AUE is 1.26+-0.1 kcal/mol; ours is 1.35 [95% CI: 1.09, 1.68] kcal/mol).

mcl1_arsenic - 5 ns - 12 states.csv


Add --enable-build-libblsct-only configure option

Add --enable-build-libblsct-only confgure option.
The option disables all existing build targets and let make produce ./src/libblsct.a only which exposes blsct functions to outside.
libblsct.a now exposes two functions only to initialize bls/mcl libraries and confirm they can be used for a testing purpose.

LGTM approving
I wonder if we should add a ci task building just the blsct lib