Make the working directory explicit

Codecov Report

Merging #10 (deb816f) into main (960196b) will increase coverage by 0.92%.
The diff coverage is 96.15%.

@@            Coverage Diff             @@
##             main      #10      +/-   ##
==========================================
+ Coverage   85.12%   86.05%   +0.92%
==========================================
  Files           1        1
  Lines         195      208      +13
  Branches       19       19
==========================================
+ Hits          166      179      +13
  Misses         27       27
  Partials        2        2

Impacted Files
Coverage Δ

src/index.ts
86.05% <96.15%> (+0.92%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 960196b...deb816f. Read the comment docs.


Update links for Martin Zikmund

#WeAreDotnet Creator Submission Guidelines

[x] Check all points before submitting the PR.

[x] By submitting this pull request to add my creator card, I verify that I adhere to the #WeAreDotnet creator card submission guidelines.

[x] Make sure to follow our Code of Conduct

[x] We do not accept submissions with the sole intention of promoting a product or company.

[x]  We add you as a creator, not your company.

[x]  The moderator team decides whether we accept or reject a submission.

When Do We Accept Someone's Creator Card Submission?

[x] You must have published at least three content pieces (long form content) since 2022 on any of your channels.

.NET itself

.NET libraries and projects

.NET languages C#, F#, Visual Basic

Tools needed during development (for .NET)

CI/CD

Only in combination with .NET: SPA Frontend technologies like Angular, Vue.js, React etc.

Cloud services related to .NET applications (AWS, Azure...)

[x] There must be three content pieces in English.

[x] There needs to be more than tweets.

[x] Longer-form article on LinkedIn is good, but a self-hosted blog or one on a well-known platform like Hashnode, medium etc., is better.

[x] Alternatively, you contributed with another form on a platform like YouTube, Twitch, Udemy or Pluralsight. If you have some recorded talks of conferences or anywhere else, that counts too! So in the form of a recorded video or a book.

RSS or Atom Feed

[x] The feed was validated using https://validator.w3.org/feed/ and does not contain errors

[x] The content with feed.type = dotnet must be .NET related:

.NET itself

.NET libraries and projects

.NET languages C#, F#, Visual Basic

Tools needed during development (for .NET)

CI/CD

Only in combination with .NET: SPA Frontend technologies like Angular, Vue.js, React etc.

Cloud services related to .NET applications (AWS, Azure...)

[x] the content under feed.type = personal may contain off-topic posts.

Project management

leadership

Other

[x] Submit an Atom or RSS URL inside your creator card that belongs to your blog

[x] If you do not submit an RSS or Atom URL. We cannot consider your feed for the aggregation feature.

Content Guideline

[x] If you have a self-hosted blog, please ensure you have a valid HTTPS certificate.

[x] We host NO malicious or offensive content on the blog (including photos, swearing, etc.)

[x] You must add the full URL to your GitHub profile to your card, so we can assign it to the member account that used the same GitHub account for sign-up.

Hey @MartinZikmund could you use the new card format?

https://github.com/WeAreDotnet/dotnet-creator-cards#sample-card
https://github.com/WeAreDotnet/dotnet-creator-cards/blob/main/creator-cards/_creatorTemplate.json

Greetings and thanks in advance!
Matthias

@matthiasjost Updated, hope it is correct now 😊 (although I kept the bio, is there a replacement for it or not?)

@matthiasjost indeed! Updated, should be ok now 👍

Thanks @MartinZikmund!

@MartinZikmund Not sure why your cards show up two times. @DeeJayTC knows a way out for sure 😃
On the second issue with the YT videos not showing that has been resolved?
Thanks for your patience @MartinZikmund.

@matthiasjost not yet, but I hope https://github.com/WeAreDotnet/dotnet-creator-cards/pull/243 will fix it - can you approve that one please 👍  ?


Stripe Subscriptions error webhook

Issue Summary
We are getting a bit of trouble with the webhook called "invoice.payment_succeeded". All the subscriptions are created in the dashboard with the format "sub_DCXg.....", but the package make internal an id subscription with the format "sli_ABcDX..." (we dont know why).
In the function sync_invoice_items (pinax/stripe/actions/invoices.py), it is doing a comparation with invoice.subscription.stripe_id and item["id"] but they are different:

so in the else, it try to retrieve the subscription with the fields customer and the item id but it doesnt exist in my dashboard stripe panel raising an error:

Have you checked the event and EventProcessingException logs / entries to see where "sli_*" is coming from?

Hello blueyed, thank you for the answer.
i've been checked that error, and literally it show the same information.

We've already an account in Stripe for international payments using pinax-stripe(pinax-stripe==4.3.0), but it doesn't make that error in the webhook called invoice.payment_succeeded... (this Stripe account was created four years ago, and it's using webhook's version 2014, that's the unique diferente between this new Stripe account and the old Stripe account)
Greetings.

The issue is due to the new stripe api version. Edit pinax/stripe/actions/invoices.py line 167 from item["id"] to item["subscription"] .


Use redux-oidc with SSR

Hello,
I have a react application with Server Side Rendering.
I used the custom InMemoryWebStorage mentionned in a earlier closed GitHub Issues.
import { createUserManager } from 'redux-oidc';
import InMemoryStorageStore, { UserManager, WebStorageStateStore } from 'oidc-client';

// create a user manager instance
const config = {
  ...
  userStore: new WebStorageStateStore({ store: InMemoryStorageStore })
}

const userManager : UserManager = createUserManager(config);

export default userManager;

But I have this following exception:
Call to Node module failed with error: Prerendering failed because of error: ReferenceError: localStorage is not defined
at Function.get (D:\Dev\iNUI\Store\airxtouch\node_modules\oidc-client\lib\oidc-client.min.js:1:12917)

What do I missed? How can I solve this?
Thanks.

This is an issue I've been meaning to tackle, I just didn't have the time or someone using SSR to help me with it. But here you are :)
The issue is that this package relies on localStorage to set a key for checking whether or not a token callback has occurred. It is used in [the middlware9(https://github.com/maxmantz/redux-oidc/blob/master/src/oidcMiddleware.js#L34) and the callback component. We would have to remove this reliance and work with something else.
This "something else" shouldideally be a storage mechanism like the InMemoryStorageStore used by oidc-client so you can use one store for both. The problem here is that these stores use promises, so the code would have to be refactored in such a way as to make promises work.
I will look into this as soon as I find the time. Since you would be using it, I'd be grateful for feedback regarding my implementation. Feel free to submit a PR if you feel like you want to contribute. Looking forward to it!

I tried not to add the OIDC middleware on the server side (only in the configureStore on the client side). It seems to work properly. But the problem occurs with the userManager.signinRedirect() in my Login component/page. As soon as I uncomment this line, it crashes.
In my understanding, the problem seems coming from the oidc-client library. Even while specifying the other store "in memory" it doesn't work.

I tried it here : https://github.com/IdentityModel/oidc-client-js/issues/243

I have published v3.0.0-beta.1 to npm. It is now completely independent from window. The API has changed a little which is why I had to bump the version to 3.
The main changes for your app are that the middleware creator now only needs the userManager as an argument and secondly the CallbackComponent needs the userManager as a prop.
I'd be happy for any feedback!

Thanks for the update! I have this exception with the new v3.0.0-beta.1:
Exception: Call to Node module failed with error: Prerendering failed because of error: ReferenceError: regeneratorRuntime is not defined
at Object.defineProperty.value (...\node_modules\redux-oidc\dist\redux-oidc.js:1:9422)

You will need the babel polyfill for this to work. I'll add it as a peer dependency in the next update.

I was missing the import on the server side...
I had the other exception with the sessionStorage not defined.
I configure a custom userStore/stateStore for the UserManager (like the other linked GitHub Issue).
Exception: Call to Node module failed with error: Prerendering failed because of error: ReferenceError: sessionStorage is not defined
at Function.get (...\node_modules\oidc-client\lib\oidc-client.min.js:1:12982)

And then I have this error :
Exception: Call to Node module failed with error: Prerendering failed because of error: ReferenceError: XMLHttpRequest is not defined
at Function.get (...\node_modules\oidc-client\lib\oidc-client.min.js:1:13049)
at new t (...\node_modules\oidc-client\lib\oidc-client.min.js:1:24706)

But it's not directly related to the redux-oidc library. Did you have this last error? If so, how did you solve it?
Thanks.

Haven't tried SSR myself yet. Maybe this can help? https://www.npmjs.com/package/w3c-xmlhttprequest

I think at that point the only way to solved this is to replace all the xmlhttprequest inside the oidc-client-js by something else.
If you want to test quickly the SSR without a lot of work you can use the aspnet core generator here : https://www.npmjs.com/package/generator-aspnetcore-spa.

I modified the source code of oidc-client-js (I replaced the XMLHttpRequest by fecth).
Now I have an other error in this library probably linked to the custom store I used.
Uncaught (in promise) Error: Error handling redirect callback: No matching state found in storage
    at r.n.onRedirectError (redux-oidc.js:formatted:1)
    at redux-oidc.js:formatted:1

Have you tried this library with a custom store for user/state?

Sorry, can't help you there.

I have now a working solution (using a modified https://github.com/pellea/oidc-client-js/commit/0c35d47585c1d4ee947cd0d33304a9eb3c1181c2 and using the custom 3.0.0-beta.1 build).
What you have modified in this 3.0.0-beta.1 build seems to be enough in order to use Server Side Rendering.

Good to hear. I'll close this for now.

@pellea Would you consider a pull request of your changes to IdentityModel/oidc-client-js?  I'm running into the same problems you had.

I don't think it would be possible. I removed the IoC capabilities of the library (that the original author intended to have). I also renamed the npm package and published it on npm. I don't think that I could make a pull request as-is.
What do you think @brockallen ?

@pellea i tried ur way from here. But i get error again

ReferenceError: localStorage is not defined

import { InMemoryWebStorage } from 'oidc-client';
import MemoryStorage from 'memorystorage'

export default class CustomWebStorageStore extends InMemoryWebStorage
{
    _storage = new MemoryStorage;
    constructor(){
        super()
        this._storage = new MemoryStorage('my-app');
    }

    getItem(key) {
        return this._storage.getItem(key);
    }

    setItem(key, value){
        this._storage.setItem(key, value);
    }

    removeItem(key){
        this._storage.removeItem(key);
    }

    get length() {
        return this._storage.length();
    }

    key(index) {
        return this._storage.key(index);
    }
}

import { createUserManager } from 'redux-oidc';
import { UserManager, WebStorageStateStore, WebStorageStateStoreSettings } from 'oidc-client';
import CustomWebStorageStore  from './CustomWebStorageStore';

const myStore = new customStore();

const settings =
{
  prefix: "oidc.",
  store: myStore
};

const userManagerConfig = {
  client_id: 'mvc',
  redirect_uri: `localhost:3001/callback`,
  response_type: 'id_token token',
  scope: 'oopenid profile email api1 api2.read_only',
  authority: 'http://localhost:60388/',
  popup_redirect_uri:"localhost:3001/popup",
  silent_redirect_uri: "localhost:3001/silent",
  automaticSilentRenew: true,
  filterProtocolClaims: true,
  loadUserInfo: true,
  userStore: new WebStorageStateStore(settings)
};

const userManager = createUserManager(userManagerConfig);

export default userManager;

What am i missing?

So what is the best solution right now?
I am getting Prerendering failed because of error: ReferenceError: sessionStorage is not defined error.
I am using this starter template https://github.com/aspnet/JavaScriptServices (react-redux) and trying to setup redux-oidc there.
Thank you in advance.

It appears that your userManager still uses the default token storage mechanism, which is sessionStorage. Please refer to the oidc-client library on how to set up a custom storage mechanism.

@Shadowman4205 did you manage to create working solution?
When I try to use WebStorageStateStore, WebStorageStateStoreSettings I am getting compile error that redux-oidc has not imported these memebers.

No i did not, i started using this oidc-client-node


Selecting support and query sets randomly

Thank you for sharing your code.
If I understood correctly, you don't sample support and query sets randomly unlike the original paper of Protonet. Can this be the reason of overfitting on custom dataset?
data, _ = [_.cuda() for _ in batch]
p = args.shot * args.train_way
data_shot, data_query = data[:p], data[p:]

Sorry, actually now I just noticed that you do select them randomly.
pos = torch.randperm(len(l))[:self.n_per]


Add support for DETA 10W 940lm Grid Connect Smart LED Downlight

Link to product: https://www.bunnings.com.au/deta-10w-940lm-grid-connect-smart-led-downlight_p0102573
Product image: https://media.prod.bunnings.com.au/api/public/content/f2649bf228104a47950b04f21e76b8bd?v=0c1a02c1&t=w500dpr2
Note this product comes in 2 versions. They are differentiated by Series 2 and Series 3 markings, a different profile must be used to flash Series 2 vs Series 3, however they both use BK7231T, they do run different firmware versions though.
I've created my own custom profiles and have successfully flashed 50+ lights.
The profiles are linked below:
https://github.com/ryuzaki-ryuga/Tuya-Cloudcutter-Custom-Profiles/tree/main/Deta 10W Grid Smart Downlight Series 2 : Ver 1.3.2
https://github.com/ryuzaki-ryuga/Tuya-Cloudcutter-Custom-Profiles/tree/main/Deta 10W Grid Smart Downlight Series 3 : Ver 2.9.6

While the profiles will work, they are not in a form that can be added to CloudCutter as a new device because the device configuration and schema are not verified (they're from a different device with a compatible profile).  Because we use schemas for cut mode and device configuration for generating 3rd party firmware in some cases, these profiles would not work in those places.  They're fine for flashing, but so are the ones that are already present that you used as a base.
If you are able to provide full factory dumps of these devices, they can be added as specific devices.  Please reopen this issue if you are able to provide those dumps.  Otherwise, this will be closed and can be used for guiding to a compatible profile.
Summary of above:
DETA 10W Downlight Series 2 uses firmware 1.3.2 - oem_bk7231s_ceiling_light_ty
DETA 10W Downlight Series 3 uses firmware 2.9.6 - oem_bk7231s_light_ty


The ingress controller solution is not a valid solution.

The ingress controller solution does not work. Because the ingress resource has to be in the same namespace as the controller otherwise it cannot find the tls secret.
This contradicts the ingress/workload single namespace scenario as specified in the solution brief.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 99d11c89-19c6-f881-a5f7-31c17f092500
Version Independent ID: 340c35ed-ecea-ad25-efbe-e02bb8924fc0
Content: Set up Secrets Store CSI Driver to enable NGINX Ingress Controller with TLS on Azure Kubernetes Service - Azure Kubernetes Service
Content Source: articles/aks/csi-secrets-store-nginx-tls.md
Service: azure-kubernetes-service
GitHub Login: @Nickomang
Microsoft Alias: nickoman

@arsovski-ahnl
Thanks for your feedback! We will investigate and update as appropriate.

@arsovski-ahnl, Thanks for your feedback. Can you please clarify further exactly what in the article is contradicting the statement "In this case, ingress is specific to a single namespace/workload and is deployed in the same namespace as the application"?
As far as I can tell, the application, ingress resource, and ingress controller are all deployed to the ingress-basic namespace. While it's true that multiple resources are deployed to the same namespace, this doesn't violate the statement. Ingress is specific to a single namespace, but there's no guarantee that the namespace is specific to a single resource.
Apologies if I'm misunderstanding.

@arsovski-ahnl, Closing this due to inactivity. If this is still an issue, feel free to reopen and tag me.
#please-close

@Nickomang The issue i reported actually is that its not clear that all the resources need to be in the same namespace. As this is difficult to achieve in a real usage.
Usually we want the application to reside in its own namespace and controller in its own. But in this documentation in both scenarios that usable solution is not feasible.
I instead had to use the default ssl certificate annotation in the nginx controller instead, which worked:
--set controller.extraArgs.default-ssl-certificate="NAMESPACE/SECRET_"
So in my case this solution is not complete.
Maybe you should post it as a third solution.


Tip when command is copied

When I click copy command I recommend that the platform could show some tips that notify me the command is copied.
Something like this

The feature was added on PR #14 and it is already in production.


Add the openssf badge to the main README

Changes
Add the openssf badge to the main README so we may show the progress on the badge as we complete the base level and work through silver and gold levels.
Signed-off-by: Andrea Frittoli<EMAIL_ADDRESS>Submitter Checklist
As the author of this PR, please check off the items in this checklist:

[x] Has Docs included if any changes are user facing
[ ] Has Tests included if any functionality added or changed
[x] Follows the commit message standard
[x] Meets the Tekton contributor standards (including
functionality, content, code)
[x] Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
[ ] Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
[ ] Release notes contains the string "action required" if the change requires additional action from users switching to the new release

/kind documentation
Release Notes
NONE

/retest

/approve

/retest
Looks like we have a new flake on TestPipelineRunStatusSpec.

/lgtm


redundant code when working with yaml, compared to ruby

crystal
str = "---
- a: aaa
  b: bbb
- a: aaaaa
  b: bbbbb
"

require "yaml"
config = {} of String => Hash(String, String)

(Yaml.load(str) as Array).each do |el|
  h = el as Hash
  x = {} of String => String
  h.each do |k, v|
    x[k as String] = v as String
  end
  config[h["a"] as String] = x
end

p config
```

ruby
```ruby
str = "---
- a: aaa
  b: bbb
- a: aaaaa
  b: bbbbb
"

require "yaml"
config = {}
YAML.load(str).each do |h|
  config[h['a']] = h
end
p config
```

I'm closing this issue. Please open another issue if you want YAML mapping similar to JSON mapping (or better yet, open a pull request :wink:), but for now using type checks is the conventional way to do it.

yaml_mapping would be :heart:


"Pending" label is not centered

I was using web:

Looks like this has been fixed in the mean time:


Fixes issues with configuration in general and with fonts 0x10-0x1f.

See commit comments:

Fixed configuration: ONETIME vs PRIMARY.
Configuration was no longer working. The firmware was always reading the
ONETIME configuration section, while the config utility was writing the
PRIMARY section. And there was no way to wipe/invalidate the ONETIME
section, so the configuration utility had no effect.
Also, the "reboot" command was reading and rewriting the ONETIME config
section, so nothing changed.
There are several way to fix the configuration utility - depending on what
was actually intended with the ONETIME vs PRIMARY sections. This is one way
to fix the issue: a reboot now reads the PRIMARY section and writes it to
ONETIME.

Fixed bit mask for font configuration.
Fonts 0x10-0x1f could not be selected, since bit mask 0x2F always forced bit 4 to 0.
The firmware supports 40 fonts (0x00-0x2F). However,  0x2F is not a multiple of 2,
hence a simple bit mask cannot be used to mask the valid range.
=> Replaced with bit mask 0x3F, and added a separate range check for "FONT_COUNT".

I'm closing this merge request - for technical reasons (the request was from my "main" branch, which wasn't too clever, and I'm about to push other changes to "main" in my repo).
I'm going to add a new link with the relevant changes to this existing issue:
https://github.com/V2RetroComputing/analog/issues/30


[Request] Ability to set the 'userPrincipalName' obfuscated

It will be nice to have this, since we are doing fork and adding our configs on public repo. Don't wanna make the email public 🙂 for security purposes. Hope you take this under consideration.

I'm thinking of converting config files into .config.js, so that we can also use environment variables in them directly.
Putting your userPrincipalName in environment variables would definitely hide your email in production environments.
Thoughts?

Yeah it seems nice


Add VS KudosPlatform Solution and Azure Function Project and AppVeyor Pipeline

Fixes #3
Fixes #4
Fixes #5

This change is

Reviewed 8 of 8 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

Reviewed 1 of 8 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

Reviewed 8 of 8 files at r1.
Review status: :shipit: all files reviewed at latest revision, all discussions resolved, all commit checks successful.

Comments from Reviewable


custom-sort function in b-table is not getting isAsc status

Overview of the problem
Buefy version: 0.6.1, 0.6.2
Vuejs version: 2.5.4
OS/Browser: various
Description
I'm using custom-sort of b-table to sort a complex object list.
I combined custom-sort and sortable on b-table. When I click on the sortable column head, the table is sorted. But I click again on the column head, the table does not change, which will reverse on a sortable column without custom-sort function.
Finally, I found the custom-sort function is missing a parameter isAsc to determine whether the data should be sort ascend or descend.
Steps to reproduce

Expected behavior
The table should be reversed on the second click on the sortable head
Actual behavior
The table does not change on the second click on the sortable head

@frantic1048 You should listen sort event and save order parameter in your local data.

@jtommy Yes, It is a way.
But it makes the place of custom-sort being weird. We are using custom-sort to customize what to sort easily for each column. The only problem is custom-sort does not support isAsc status. It makes meanless of implementing custom-sort for the table component because it cannot behave like default sort to sort ascending/descending.
sort event can also handle this, by using a big switch/if else block, which I think is not as clear as defining custom-sort function for each sortable column.


Add note about DNS server IP

Proposed changes
Add a note about the IP of the DNS server being <IP_ADDRESS>.

Related issues (optional)
Fixes #1969

We'll take it, thanks @shubheksha :)


[WIP] Pep8 fixes of the current master

I have updated the Makefile to check pep8 with the newer script (kivy/tools/pep8checker/pep8.py) but these changes should also pass with the old script.
Help appreciated in fixing the whole repo :smile:

I have updated the Makefile to check pep8 with the newer script (kivy/tools/pep8checker/pep8.py)

I don't understand the Makefile change, pep8kivy imported pep8 and added other features, like being able to exclude paths.

@dessant  Ohh my bad! I thought pep8.py was a newer script. Now I get what you meant by

udiboy1209: the included pep8 module is old

I thought that referred to pep8kivy.py.
So do we want to update the pep8 module? It's missing a lot of newer errors. Anyway why are we using a custom pep8 and not the pip-installed version ?

Yes, updating pep8.pywould be good, the original code was taken from here: https://github.com/PyCQA/pycodestyle/blob/master/pycodestyle.py
There are two commits which should not be lost though:
https://github.com/kivy/kivy/commit/a2ea270f750d90673f63dbc7b6ed63b8c5c1d7ce#diff-43b700e60750dd13b002d04cc62dd894
https://github.com/kivy/kivy/commit/0c34d760570ec19d4ce0eca1317e441603113443

@udiboy1209, we could also switch to installing from pip, if we set up a way to add custom checks, like the commits linked above.
This update/switch can be done in a separate pr from the pep8 fixes of course.

If you want to provide extensibility / plugins, please see flake8 - pep8 doesn’t want or need a plugin architecture.

This is in the Developer's notes :stuck_out_tongue: .

A very old leftover indeed, i'll make a pr. 😁 https://github.com/PyCQA/pycodestyle/commit/4b48c39ca79f9bd6b6ef2490d867209ef5f0f336#diff-f0b41b6133ffe3efa4aed9b6033b32d2R12


gsoler2 Acrolinx quality push: auditpol-remove.md

#assign: @robinharwood, @Xelu86
@robinharwood @Xelu86
Can you review the proposed changes?
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

Looks good, merging.


Define web.Application.on_startup() signal handler

What do these changes do?

This change introduces new signal Application.on_startup()
that allows to register as many on_startup signal handlers as needed to run in the event loop along with the application's request handler just after the application's startup.
This is useful to run background tasks such as listening to different event sources (message queues, etc.) to process within an application (e.g. to forward to clients connected via WebSocket).
Are there changes in behavior for the user?

The change doesn't impact on existing behavior but extends the API and the functionality.
Related issue number

#1092
Checklist

[x] I think the code is well written
[x] Unit tests for the changes exist
[x] Documentation reflects the changes

Changes done

Define web.Application.on_startup() signal handler.
Run app.on_startup() along with the request handler within an event
loop in web.run_app().
Tests for Application.on_startup signal
Tests for multiple tasks to run on startup, including two long running
Extend tests on web.run_app(): ensure that Application.startup()
is called within web.run_app().
Add documentation for Application.on_startup signal handler:

Describe possible use cases
Minor: fix typo

Notes
In the tests/test_run_app.py increased delay for loop.call_later from
0.01s to 0.02s since with the old value loop used to stop
before coroutines gathered for on_startup finished, that caused
an exception.
As discussed earlier, the appropriate changes in the aiohttp.worker.GunicornWebWorker will be applied in a separate PR.

Current coverage is 97.63% (diff: 100%)

No coverage report found for master at 92b7b27.

Powered by Codecov. Last update 92b7b27...7da1d9f

Thanks you very much!

Thank you for your help! :)


Only connect filters with temporary files when necessary

A lot of act's internal process flow is based around 'filters', which are modules that take one input file and return a combination of output file and auxiliary output.  These filters can be chained together (conditionally or unconditionally), which currently results in a temporary file being created to hold the output.  For example, one pipeline looks like this:
                               C file      << filesystem
-> [C compiler (external)]  -> Assembly
-> [Litmusifier (internal)] -> Litmus test
-> [Herd]                   -> Herd output >> stdout

In some cases, both filters being chained are ones act implements itself.  In these cases, building a temporary file is fairly pointless—we should be able to send the output of one filter directly into the input of the other!
I've previously tried to implement this with Unix pipes, but ran into liveness problems (not sure what causes them, but it's probably due to a pipe not having an EOF until it's directly closed).  Since it's not a massive issue (yet), I've decided to leave this on the back-burner for now.
(Another possibility could be to pull the filter pipelines out of act entirely and use shell scripts to handle it.  This would be a bit more of a nightmare to maintain, and also quite difficult to get to work with the auxiliary output side of things, but might pay off in terms of composability/reusability later on.  Thoughts?)

Filters no longer exist!  Closing.


Issue #107 - Adds a text blurb component

Adds a simple text-blurb component. For now, only used on the measure listing page, but it's ready to be dropped in where ever we need it.
Resolves issue #107 .
Looks like this:

@KyleW this looks really good.  I have a couple small things for you to update, mostly things I have not communicated anywhere yet, so thank you for pointing these things out, and great job!

Can you change app/components/common/textBlurb/index.js to app/components/common/textBlurb/textBlurb.js ? I'm trying to refrain from using the filename index.* anywhere in the application, as much as possible.  It gets difficult to manage as the application grows :/
Please remove the parent class .text_blurb. You don't need it :)
There is a file app/components/common/core/core.js, that includes all the core Angular components and Angular UI Bootstrap.  If you require it in your index.js/textBlurb.js you won't need to reference ui bootstrap separately.  Also, you'll have access to the collapse directive if you want it.


Expose GCP PubSub PublishSettings as a configuration option

Google PubSub offers configuration options for batching multiple pubsub messages. (https://godoc.org/cloud.google.com/go/pubsub#PublishSettings) Gizmo's GCP publisher needs a way to set these options.

https://github.com/nytimes/gizmo/pull/239


mempool/dog: Design and implement tests for potential attacks

Design and implement the set of tests (together with acceptable KPIs) that the final algorithm should be passing.
Tests should include typical scenarios (perturbations) and known issues (same tx to multiple nodes) and possible attacks.
Timeboxed (risk of rabbit hole).

We tested:

[x]  Using number of transactions to trigger the redundancy controller. Experiments suggest it was too volatile and can be used as an attack (sending too many small txs just to trigger a node to send too many HaveTx transactions).
[x] Varying the adjustmentInterval .
[x] Varying the adjustmentInterval for different load and different .

We have two possible adversarial scenarios where the DOG protocol may not work as expected and even isolate nodes from transaction messages:

Node perturbations, where a node disconnecting from the network may leave the routing tables of other nodes in an inconsistent state. Also, peers of that node may heavily depend on the transactions sent through that connection. This scenario was already covered during the design of the protocol.
Two different nodes (that are not peers) receiving the same transaction. The transaction will be propagated in the network and it may happen that two other connected nodes receive it and then send it to each other at the same time. They could end up in a situation where they cut each other routes, cutting all transaction communication between them.

Both scenarios have been tested in #4606.


Add test cases to check for regression on 0.11.0

There might not be an error during deserialization then. Maybe the issue is more subtle - maybe the data being read in isn't the same as the data being read back out, although it looks like this should account for it. Were there any other messages uncovered in https://github.com/akkadotnet/akka.net/pull/5138 that could have been responsible for this other than this one?

This is pretty much all the messages that I had to add to the serializer on the Akka.NET side, so I'm at a bit of a loss here.

@Arkatufus use the smoke test from before and just log all of the messages that passed through Hyperion, rather than throwing an exception on the first one. We might have missed a case.

This is all of the message types that went through Hyperion in all of the MNTK tests:
https://docs.google.com/spreadsheets/d/1WBpcGiyKaVyV6xtEAR5xbPWFfPeroIOLY2n8FAtCgXc/edit?usp=sharing

The list looks clean, no other sharding message (except for Passivate) goes through it

serialized
Akka.Cluster.Sharding.Passivate
Akka.Cluster.Sharding

deserialized
Akka.Cluster.Sharding.Passivate
Akka.Cluster.Sharding

serialized
Akka.Cluster.Sharding.Tests.ClusterShardingFailureSpec+Add
Akka.Cluster.Sharding.Tests.MultiNode

deserialized
Akka.Cluster.Sharding.Tests.ClusterShardingFailureSpec+Add
Akka.Cluster.Sharding.Tests.MultiNode

serialized
Akka.Cluster.Sharding.Tests.ClusterShardingFailureSpec+Value
Akka.Cluster.Sharding.Tests.MultiNode

deserialized
Akka.Cluster.Sharding.Tests.ClusterShardingFailureSpec+Value
Akka.Cluster.Sharding.Tests.MultiNode

serialized
Akka.Cluster.Sharding.Tests.ClusterShardingGetStateSpec+Ping
Akka.Cluster.Sharding.Tests.MultiNode

deserialized
Akka.Cluster.Sharding.Tests.ClusterShardingGetStateSpec+Ping
Akka.Cluster.Sharding.Tests.MultiNode

serialized
Akka.Cluster.Sharding.Tests.ClusterShardingGetStateSpec+Pong
Akka.Cluster.Sharding.Tests.MultiNode

deserialized
Akka.Cluster.Sharding.Tests.ClusterShardingGetStateSpec+Pong
Akka.Cluster.Sharding.Tests.MultiNode

serialized
Akka.Cluster.Sharding.Tests.ClusterShardingLeavingSpec+GetLocations
Akka.Cluster.Sharding.Tests.MultiNode

deserialized
Akka.Cluster.Sharding.Tests.ClusterShardingLeavingSpec+GetLocations
Akka.Cluster.Sharding.Tests.MultiNode

serialized
Akka.Cluster.Sharding.Tests.ClusterShardingLeavingSpec+Locations
Akka.Cluster.Sharding.Tests.MultiNode

deserialized
Akka.Cluster.Sharding.Tests.ClusterShardingLeavingSpec+Locations
Akka.Cluster.Sharding.Tests.MultiNode

serialized
Akka.Cluster.Sharding.Tests.ClusterShardingLeavingSpec+Ping
Akka.Cluster.Sharding.Tests.MultiNode

deserialized
Akka.Cluster.Sharding.Tests.ClusterShardingLeavingSpec+Ping
Akka.Cluster.Sharding.Tests.MultiNode

serialized
Akka.Cluster.Sharding.Tests.Counter+EntityEnvelope
Akka.Cluster.Sharding.Tests.MultiNode

deserialized
Akka.Cluster.Sharding.Tests.Counter+EntityEnvelope
Akka.Cluster.Sharding.Tests.MultiNode

serialized
Akka.Cluster.Sharding.Tests.Counter+Get
Akka.Cluster.Sharding.Tests.MultiNode

deserialized
Akka.Cluster.Sharding.Tests.Counter+Get
Akka.Cluster.Sharding.Tests.MultiNode

serialized
Akka.Cluster.Sharding.Tests.Counter+Stop
Akka.Cluster.Sharding.Tests.MultiNode

deserialized
Akka.Cluster.Sharding.Tests.Counter+Stop
Akka.Cluster.Sharding.Tests.MultiNode

serialized
Akka.Cluster.Sharding.Tests.RollingUpdateShardAllocationSpec+GiveMeYourHome+Get
Akka.Cluster.Sharding.Tests.MultiNode

deserialized
Akka.Cluster.Sharding.Tests.RollingUpdateShardAllocationSpec+GiveMeYourHome+Get
Akka.Cluster.Sharding.Tests.MultiNode

serialized
Akka.Cluster.Sharding.Tests.RollingUpdateShardAllocationSpec+GiveMeYourHome+Home
Akka.Cluster.Sharding.Tests.MultiNode

deserialized
Akka.Cluster.Sharding.Tests.RollingUpdateShardAllocationSpec+GiveMeYourHome+Home
Akka.Cluster.Sharding.Tests.MultiNode

serialized
Akka.Persistence.RecoverySuccess
Akka.Persistence

deserialized
Akka.Persistence.RecoverySuccess
Akka.Persistence

serialized
Akka.Persistence.ReplayedMessage
Akka.Persistence

deserialized
Akka.Persistence.ReplayedMessage
Akka.Persistence

serialized
Akka.Persistence.ReplayMessages
Akka.Persistence

deserialized
Akka.Persistence.ReplayMessages
Akka.Persistence

serialized
Akka.Persistence.WriteMessages
Akka.Persistence

deserialized
Akka.Persistence.WriteMessages
Akka.Persistence

serialized
Akka.Persistence.WriteMessagesSuccessful
Akka.Persistence

deserialized
Akka.Persistence.WriteMessagesSuccessful
Akka.Persistence

serialized
Akka.Persistence.WriteMessageSuccess
Akka.Persistence

deserialized
Akka.Persistence.WriteMessageSuccess
Akka.Persistence

Closing this for now, we can't find any regression bug caused by Hyperion 0.11.0


Add script to add/update an organisations fixture for a worldwide location

I've found myself needing to do this quite a few times when adding regression tests for smart answer flows. I've updated the README - I think the old instructions were quite out-of-date.

This is going to be very helpful. LGTM :+1:


Allowing flexibility in integrator choice

Currently the SimulationFactory.generateSimFromStruct() method doesn't allow for specification of integrators and an NCMCVVAlchemicalIntegrator is automatically used as the integrator. I think allowing an option to either specify and/or pass in an integrator would be beneficial.

Any particular thoughts on this @nathanmlim?

@sgill2 - my perspective is that we should figure out what integrator(s) we should use in which situations internally and make this choice in a way which is generally hidden from the user.
That is to say, I don't think we want this to be an option, though I could be backed off from that position to the position that it should be an experts-only or hidden option to make your life easier.

The easiest way to test which integrators work best internally would be to have way to specify it somewhere in the run script, but I agree that it should be a mostly hidden option.

Let's wait for Nathan to weigh in. On one hand we don't want things to be unnecessarily difficult for you, but on the other we do want to set things up in a way where a user doesn't have to know much to get what we believe are best practices.

@nathanmlim - thoughts? (I believe you're about to start cross-comparing integrators in some tests so this becomes more relevant to you.)

From the set of tests I ran, it seems like the openmmtools set of splitting integrators performs better than the old VV based, so we've reverted to using that as our default for now.


Give it a new future or a hell

The storm is beginning. Stand shoreside only !
I'm waiting your offer.

We Are Your Parents.
You Are Our Hope.
We Are Securing Your Future.
We Don't Give Up.
Expect Us.

You offer option:
<EMAIL_ADDRESS><EMAIL_ADDRESS>3.No words in 30 days=default=hell.

神挡杀神，佛档屠佛，没时间反刍，只能生咽，我也会对不起你们，公司还这么小就让你们选边站队。
sorry


Doc edits

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/do-doc/build

/do-doc-build


[UI Enhancement] : Extend rule/text editing space horizontally

Hello.
Currently, the height of the rule/text editing space within uBlacklist can be adjusted vertically, but not horizontally. Please consider implementing the latter (see screenshot).
Thank you for your time and consideration. Kind Regards.

Declined.
I don't think it is useful at all.


Question about the input size of the model

Hello, first of all thank you very much @AnTao97 for this repository. I just have a couple of brief, simple questions about the input size of the segmentation model that, after reading carefully the paper, I don't understand.

The input size of the first nn.Conv2d layer of the part segmentation model is 6, is this because the input parameters to the model are one point (x, y, z) and its respective normals (nx, ny, nz)? If it's like this, then the input vector to the neural network is (x, y, z, nx, ny, nz) right?
And second, respect to the semantic segmentation model, why the input size in this case is 18? In the paper they specify that the input vector is 9D (XYZ, RGB and normalized spatial coordinates)

Thank you very much and sorry for the inconvenience.

In part segmentation, the point feature is a 3-dim vector (x, y, z). In function get_graph_feature the 3-dim vector is extended into a 6\times k matrix by graph feature extraction. The k represents the k neighbors around this point.
https://github.com/AnTao97/dgcnn.pytorch/blob/97785863ff7a82da8e2abe0945cf8d12c9cc6c18/model.py#L272
Please refer to more details of graph feature extraction in code: https://github.com/AnTao97/dgcnn.pytorch/blob/97785863ff7a82da8e2abe0945cf8d12c9cc6c18/model.py#L36-L62

Oh god, that's true. I didn't realise about it. thank you very much!


Update readme

Making this more clear for anyone else who got stuck on the whole 'variables need to be in list format' thing.

Thanks again @joshprewitt! I haven't updated some of these portions of the docs in a while so this is much needed.


Support to Update Custom URL plugins only

TGM Allow the use of plugins in custom URL or git repositories, but it doesn't detect new versions of those. Would be nice adding on CLI a way to update those plugins only (just reinstall them, even without updates). Maybe detecting the file size or hash can be an way to check if there is an update, but just forcing it would work fine 😄

I have tested and it shows Success: Installed 1 of 1 plugins. It Seems that worked.

@maxviewup v0.2.0 just released, details in #25 and you should be able to do what you need with wp tgmpa-plugin install --force $(wp tgmpa-plugin list --external --field=name)

Great! Will Work great updating private plugins.


Append rule help text/markdown in alert view for CodeQL SARIF files

In the SARIF files produced by CodeQL, the rule help text/markdown is ignored in favour of the corresponding .qhelp file content when displayed in code scanning alerts. Would it be possible to have the rule help text/markdown appended to the end of the markdown generated from the .qhelp file? This would allow any SARIF file pre-processors to enrich the displayed help text with additional material such as additional analysis results, contextual training resources, vulnerability risk ratings, etc.

Hey @cwong-scw. I think one way you could achieve this would be to publish an action to GitHub's Marketplace that would perform "post-processing" of a SARIF file, before the SARIF gets uploaded to GitHub. You could use this "post-processing" step to attach additional rule help.
E.g. imagine a workflow like this, note the cwong-scw/post-process-sarif@v1 step prior to the github/codeql-action/upload-sarif@v1 step :
name: "CodeQL"

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
  schedule:
    - cron: '0 0 * * 0'

jobs:
  analyze:
    name: Analyze
    runs-on: ubuntu-latest

    strategy:
      fail-fast: false
      matrix:
        language: [ 'cpp', 'csharp', 'java', 'javascript' ]

    steps:
    - name: Checkout repository
      uses: actions/checkout@v2

    - name: Initialize CodeQL
      uses: github/codeql-action/init@v1
      with:
        languages: ${{ matrix.language }}

    - name: Autobuild
      uses: github/codeql-action/autobuild@v1

    - name: Perform CodeQL Analysis
      uses: github/codeql-action/analyze@v1
      with:
        upload: false

    - name: Perform post-processing of CodeQL Analysis
      uses: cwong-scw/post-process-sarif@v1
      with:
        sarif_file: '../results'

    - name: Upload SARIF
      uses: github/codeql-action/upload-sarif@v1
      with:
        sarif_file: '../results'

Hey @cwong-scw. I think one way you could achieve this would be to publish an action to GitHub's Marketplace that would perform "post-processing" of a SARIF file, before the SARIF gets uploaded to GitHub. You could use this "post-processing" step to attach additional rule help.
E.g. imagine a workflow like this, note the cwong-scw/post-process-sarif@v1 step prior to the github/codeql-action/upload-sarif@v1 step :
name: "CodeQL"

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
  schedule:
    - cron: '0 0 * * 0'

jobs:
  analyze:
    name: Analyze
    runs-on: ubuntu-latest

    strategy:
      fail-fast: false
      matrix:
        language: [ 'cpp', 'csharp', 'java', 'javascript' ]

    steps:
    - name: Checkout repository
      uses: actions/checkout@v2

    - name: Initialize CodeQL
      uses: github/codeql-action/init@v1
      with:
        languages: ${{ matrix.language }}

    - name: Autobuild
      uses: github/codeql-action/autobuild@v1

    - name: Perform CodeQL Analysis
      uses: github/codeql-action/analyze@v1
      with:
        upload: false

    - name: Perform post-processing of CodeQL Analysis
      uses: cwong-scw/post-process-sarif@v1
      with:
        sarif_file: '../results'

    - name: Upload SARIF
      uses: github/codeql-action/upload-sarif@v1
      with:
        sarif_file: '../results'

Hi @swinton!  Thanks for the response! We have actually taken this exact approach to supplement the SARIF file prior to upload. However, the last time that I checked it, the problem we hit was that the help text in the processed SARIF file seemed to be ignored in the upload processing, and the CodeQL .qhelp file content was used instead. This meant that any additional help contents added to the SARIF file were not shown in the code scanning alerts once uploaded.
Our request here, which I can understand might be fairly low priority, is for a way to append any help text present in the SARIF rule to the main .qhelp file content generated by CodeQL during the upload processing. This would keep the valuable remediation advice from CodeQL while still allowing this to be supplemented by other tools. Do you think something like this would be possible?

Hi @swinton!  Thanks for the response! We have actually taken this exact approach to supplement the SARIF file prior to upload. However, the last time that I checked it, the problem we hit was that the help text in the processed SARIF file seemed to be ignored in the upload processing, and the CodeQL .qhelp file content was used instead. This meant that any additional help contents added to the SARIF file were not shown in the code scanning alerts once uploaded.
Our request here, which I can understand might be fairly low priority, is for a way to append any help text present in the SARIF rule to the main .qhelp file content generated by CodeQL during the upload processing. This would keep the valuable remediation advice from CodeQL while still allowing this to be supplemented by other tools. Do you think something like this would be possible?

Our request here, which I can understand might be fairly low priority, is for a way to append any help text present in the SARIF rule to the main .qhelp file content generated by CodeQL during the upload processing

Can you please help me understand what you mean by the main .qhelp file? I'm not sure what this is referring to, and I see no mention of qhelp in the SARIF specification. Thank you.

the last time that I checked it, the problem we hit was that the help text in the processed SARIF file seemed to be ignored in the upload processing

You should be able to provide additional content in help.text or help.markdown -- if this isn't working, could you link to an example repo so we can take a closer look?

Our request here, which I can understand might be fairly low priority, is for a way to append any help text present in the SARIF rule to the main .qhelp file content generated by CodeQL during the upload processing

Can you please help me understand what you mean by the main .qhelp file? I'm not sure what this is referring to, and I see no mention of qhelp in the SARIF specification. Thank you.

the last time that I checked it, the problem we hit was that the help text in the processed SARIF file seemed to be ignored in the upload processing

You should be able to provide additional content in help.text or help.markdown -- if this isn't working, could you link to an example repo so we can take a closer look?

Yep no problem! Just to clarify, the .qhelp files are used by CodeQL only and are not part of the SARIF spec. Sorry for the confusion.
If we use a workflow such as the example above that contains the analyze step with upload set to false, and then use the upload-sarif step separately, the following seems to happen:

CodeQL runs as normal and produces a SARIF file. The SARIF file will contain a number of rule elements corresponding to the .ql files that contain the queries being run by CodeQL to identify the vulnerabilities (e.g. https://github.com/github/codeql/blob/main/javascript/ql/src/Security/CWE-094/CodeInjection.ql)
These rule elements initially do not contain any help properties. The post-process step creates the help.text or help.markdown properties within the rule and adds content into these properties.
When the post-processed SARIF file is uploaded, CodeQL-produced SARIF files appear to be treated slightly differently (perhaps based on tool.driver.name?) than other SARIF files and any help text or markdown included in a rule is not used for display in the Code scanning alerts section of the Security tab.

For example, if the post-processed rule is something like this:
{
    "id": "js/code-injection",
    "name": "js/code-injection",
    "shortDescription": {
        "text": "Code injection"
    },
    "fullDescription": {
        "text": "Interpreting unsanitized user input as code allows a malicious user arbitrary code execution."
    },
    "defaultConfiguration": {
        "level": "error"
    },
    "properties": {
        "tags": [
            "security",
            "external/cwe/cwe-094",
            "external/cwe/cwe-079",
            "external/cwe/cwe-116"
        ],
        "kind": "path-problem",
        "precision": "high",
        "name": "Code injection",
        "description": "Interpreting unsanitized user input as code allows a malicious user arbitrary\n              code execution.",
        "id": "js/code-injection",
        "problem.severity": "error"
    },
    "help": {
        "text": "Some help text here",
        "markdown": "Some help text here"
    }
}

When uploaded, "Some help text here" is not shown in the code scanning alert. Instead, the help text section seems to be populated from the .qhelp file corresponding to the .ql file (e.g. https://github.com/github/codeql/blob/main/javascript/ql/src/Security/CWE-094/CodeInjection.qhelp). In the header of the help text section there is a Query field in addition to the regular Tool and Rule ID fields, with a View Source link to the corresponding .ql file (e.g. https://github.com/github/codeql/blob/main/javascript/ql/src/Security/CWE-094/CodeInjection.ql).
An example of this occurring: https://github.com/cwong-scw/action-playground/security/code-scanning/225
My guess is that CodeQL rules are processed in a special manner and make use of the more expressive .qhelp templating to generate the markdown that is displayed in the help text section of code scanning alerts. I am wondering if it would be possible to append the help.text or help.markdown from the SARIF file to the markdown generated from the .qhelp file for CodeQL rules to support the suggested post-processing approach.  Hopefully this helps explain it better!

Yep no problem! Just to clarify, the .qhelp files are used by CodeQL only and are not part of the SARIF spec. Sorry for the confusion.
If we use a workflow such as the example above that contains the analyze step with upload set to false, and then use the upload-sarif step separately, the following seems to happen:

CodeQL runs as normal and produces a SARIF file. The SARIF file will contain a number of rule elements corresponding to the .ql files that contain the queries being run by CodeQL to identify the vulnerabilities (e.g. https://github.com/github/codeql/blob/main/javascript/ql/src/Security/CWE-094/CodeInjection.ql)
These rule elements initially do not contain any help properties. The post-process step creates the help.text or help.markdown properties within the rule and adds content into these properties.
When the post-processed SARIF file is uploaded, CodeQL-produced SARIF files appear to be treated slightly differently (perhaps based on tool.driver.name?) than other SARIF files and any help text or markdown included in a rule is not used for display in the Code scanning alerts section of the Security tab.

For example, if the post-processed rule is something like this:
{
    "id": "js/code-injection",
    "name": "js/code-injection",
    "shortDescription": {
        "text": "Code injection"
    },
    "fullDescription": {
        "text": "Interpreting unsanitized user input as code allows a malicious user arbitrary code execution."
    },
    "defaultConfiguration": {
        "level": "error"
    },
    "properties": {
        "tags": [
            "security",
            "external/cwe/cwe-094",
            "external/cwe/cwe-079",
            "external/cwe/cwe-116"
        ],
        "kind": "path-problem",
        "precision": "high",
        "name": "Code injection",
        "description": "Interpreting unsanitized user input as code allows a malicious user arbitrary\n              code execution.",
        "id": "js/code-injection",
        "problem.severity": "error"
    },
    "help": {
        "text": "Some help text here",
        "markdown": "Some help text here"
    }
}

When uploaded, "Some help text here" is not shown in the code scanning alert. Instead, the help text section seems to be populated from the .qhelp file corresponding to the .ql file (e.g. https://github.com/github/codeql/blob/main/javascript/ql/src/Security/CWE-094/CodeInjection.qhelp). In the header of the help text section there is a Query field in addition to the regular Tool and Rule ID fields, with a View Source link to the corresponding .ql file (e.g. https://github.com/github/codeql/blob/main/javascript/ql/src/Security/CWE-094/CodeInjection.ql).
An example of this occurring: https://github.com/cwong-scw/action-playground/security/code-scanning/225
My guess is that CodeQL rules are processed in a special manner and make use of the more expressive .qhelp templating to generate the markdown that is displayed in the help text section of code scanning alerts. I am wondering if it would be possible to append the help.text or help.markdown from the SARIF file to the markdown generated from the .qhelp file for CodeQL rules to support the suggested post-processing approach.  Hopefully this helps explain it better!

@cwong-scw: unfortunately, security/code-scanning/ urls tend to be private to people w/ write access to the repository.
Sample rendering of a code injection based on the aforementioned repository

I'm very interested in this because I'd love to generate .markdown content, but at least some of the github sarif handling only supports .text, and at least some basic markdown elements appear to be stripped (esp. backticks...).


Why is pom.xml not build?

Reproduction steps
Hello, I'm opening an issue again because I can't understand where the problem comes from. I just changed the version number in pom.xml. but i am getting error. I had not changed the comitte version I threw for the previous pr, but I was still getting this error. Can someone explain in detail why I am getting these errors for a small update?
Expected Results
All checks have passed
Actual Results
https://github.com/jenkinsci/codethreat-scanner-plugin/runs/12168809294
https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fcodethreat-scanner-plugin/detail/bug%2Frelease/2/
Anything else?
No response

This repository is not for questions or support. Please consider using the community forums or chats if you want to ask a question.
The exception in question is part of your build log:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:3.1.0:install (default-install) on project codethreat-scanner: Failed to install artifact io.jenkins.plugins:codethreat-scanner:pom:1.12.v049ee5efe711 : Trailing char < > at index 121<EMAIL_ADDRESS>\codethreat-scanner-1.12.v049ee5efe711 .pom -> [Help 1]
11:26:05  org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-install-plugin:3.1.0:install (default-install) on project codethreat-scanner: Failed to install artifact io.jenkins.plugins:codethreat-scanner:pom:1.12.v049ee5efe711 : Trailing char < > at index 121<EMAIL_ADDRESS>\codethreat-scanner-1.12.v049ee5efe711 .pom

I opened a topic where you mentioned and asked the question. Don't get me wrong, but I don't get a bug with "mvn clean install" on my machine. It gives a "<" gata on line 121. but line 121 is missing.

I already explained what the problem is in https://github.com/jenkins-infra/repository-permissions-updater/issues/3202#issuecomment-1477832417.


Unless checking accesses, do not include unique symbols or numbers in index types

Fixes the issue of keyof now returning unique symbols that @kpdonn reported in #20721, and @ahejlsberg were you looking at another issue this fixed?
Regarding the implementation - I actually had to add a new relationship (ew) to handle comparing with constraints of generic index types. (To continue to allow, eg, T[typeof sym]).

FWIW I actually think it makes a lot of sense for keyof to include unique symbol. See #20721 for my reasoning, but I was only pointing it out because it seemed pretty clearly unintentional and not because I thought it was actually a problem.
If it made it through a release with nobody complaining maybe it's a sign it was a good change? 🤷♂

@kpdonn keyof T can produce symbols, but keyof T is assignable to string - that's a bit of a pickle.


ids: Preparation work for EndpointDataReference transfer

This PR introduces the concept of EndpointDataReference which basically describes how to call an existing API.
This API can be external of exposed directly by the data plane.
We also introduce in this PR the IDS sender/handler used to send/receive the EndpointDataReference on consumer/provider side.
An upcoming pull-request will introduce a dedicated Provisioner and a DataFlowController, that will leverage on the IDS sender created here in order to enable the provider connector to forward an EndpointDataReference to the consumer connector.

Codecov Report

Merging #676 (ea56d32) into main (206e60f) will increase coverage by 0.42%.
The diff coverage is 64.86%.

@@             Coverage Diff              @@
##               main     #676      +/-   ##
============================================
+ Coverage     55.72%   56.14%   +0.42%
- Complexity     2014     2053      +39
============================================
  Files           565      575      +10
  Lines         12558    12768     +210
  Branches        890      901      +11
============================================
+ Hits           6998     7169     +171
- Misses         5154     5183      +29
- Partials        406      416      +10

Impacted Files
Coverage Δ

...connector/transfer/core/CoreTransferExtension.java
0.00% <0.00%> (ø)

...e/edr/DefaultEndpointDataReferenceTransformer.java
0.00% <0.00%> (ø)

...edr/EndpointDataReferenceReceiverRegistryImpl.java
0.00% <0.00%> (ø)

...connector/iam/oauth2/core/Oauth2Configuration.java
85.00% <ø> (ø)

...th2/core/identity/IdentityProviderKeyResolver.java
40.81% <ø> (ø)

...nnector/token/TokenGenerationServiceExtension.java
0.00% <ø> (ø)

...or/spi/types/domain/edr/EndpointDataReference.java
0.00% <0.00%> (ø)

...types/domain/edr/EndpointDataReferenceRequest.java
0.00% <0.00%> (ø)

.../api/multipart/handler/ArtifactRequestHandler.java
61.11% <65.21%> (+43.36%)
:arrow_up:

...or/iam/oauth2/core/identity/Oauth2ServiceImpl.java
40.78% <87.50%> (ø)

... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 206e60f...ea56d32. Read the comment docs.


Tidying up! Add doctests

Love the reorganization in https://github.com/pyOpenSci/lessons/pull/48
Noticed that a bunch of the pages were not showing up in the toctree, so i went to add them, and i noticed that there were an avalanche of warnings that probably makes it hard to tell what is actually a meaningful warning and what is just noise, so i also fixed all those.
Not a contentful PR, but things to note:

I switched the docs build to docs-test so we can help ourselves out by alerting us to when links are broken/etc. and fix them as we go. we can disable that if people find it annoying, i personally find it helpful because i always forget to check before i push.
I renamed the python-conditionals link target to just conditionals because there was an ambiguity error (i assume since the file is named python-conditionals.md)
I added raises-exceptions tags to all the python notebook cells that were failing, most of them you had explicitly written that they should fail around them, but there were some where i wasn't sure, so if there are any i marked here that aren't supposed to fail then those probs need to be fixed :)

also feel free to ignore the errors if you're getting into crunch time prepping for the workshop but you already know that ;)


Crafting upgrade in a Constructor does not work

Issue description:
Crafting upgrade inside a constructor does not create a task visible on the crafting monitor. Crafting upgrade inside an exporter works fine.

Minecraft: 1.11
Forge:2188
Refined Storage:1.3.4

Does this issue occur on a server? yes

I can't reproduce this bug

No, this bug is only happening in "item dropping" mode as it's missing the crafting schedule call there. Easy fix.

Fixed


Centralize colors used by line number area and scroll flag area

Currently colors used by line area and scroll flag and icons is defined in different locations.
This should be centralized and reused.

Something minor, but I've noticed the colors from bug mark are not matching. Also, the X appears to be a bit darker, may using the color from bug-bars marks are better.

This is blocked by the icon manager class work. The icon manager will hold the general style /colors configuration for other plugins to use.

This is blocked by the icon manager class work

What work are you referring to?

Icon manager class work at https://github.com/spyder-ide/spyder/pull/9782
That would eventually be better named StyleManager or ThemeManager and hold icon related functionality and color, styles and themes in a centralized place.


A few questions about usage

So I've just installed ludusavi and ludusavi-playnite, and made a backup in Playnite using extensions > ludusavi > backup save data for all games.
Now, a few questions :

Where is the list of saved games and the date of the save ? Can we visualize it in Playnite or Ludusavi standalone ? Should we load the backup inside Ludusavi ?
In Ludusavi standalone, should we specify the same backup path as in Playnite ?
Is there a way to know to which game executable (path) corresponds each save ?
Can we check the status for each save (e.g. : never saved yet because new game, save is up to date, save needs backup, etc...) ?
To backup Ludusavi, is it sufficient to save only the backup files ? Or should we backup config files too %APPDATA%/ludusavi ?

Thanks for the clarification, and for this amazing software !

Hi! I'm glad you've found Ludusavi helpful :D

After doing the backup, you can click on the notification in Playnite for a list of all games and their backup sizes. You can also right click on any game in Playnite and use the Ludusavi context menu to see a list of backups for it. (If Ludusavi and Playnite have different names for the game, it may not show the backups in the context menu directly.)
That's up to you. For example, if you have auto-backups enabled during gameplay in Playnite, you might want to keep those separate from your manual backups from using Ludusavi on its own. You certainly can set them to the same path if you prefer to combine them.
Could you elaborate on this question? What are you trying to do/accomplish specifically?
Right now, the best way is to open Ludusavi separately and do a preview. It will show the + and Δ icons next to each game. The Playnite plugin will show these statuses after performing a backup (via the notification in Playnite), but it doesn't let you preview them beforehand.
If you've configured any custom games, then you'll probably want to back up %APPDATA%/ludusavi/config.yaml. Ludusavi  will automatically redownload manifest.yaml and recreate cache.yaml if necessary, so you don't have to back those up, but you certainly can back them up as well.

Thanks for taking time to answer !

Could you elaborate on this question? What are you trying to do/accomplish specifically?

Sometimes, I have several versions of a game, that maybe do not have their save folders in the same location, and I would like to check to which game executable corresponds a save in Ludusavi. Is this possible (something like right-click > show game folder) ?

Ah, I see. You can't automatically jump to the game folder, but you can do it with a few extra steps by checking the game's file list. For example, I have a few copies of Dead Cells, so I can do this:

https://user-images.githubusercontent.com/4934192/205176520-60d4b6eb-e0cf-4e40-92ec-94aeaca08582.mp4

Thank you, however this work only if the the save can be found in the game folder, which regularly isn't the case. Well, it's not a big deal, just a nice to have option maybe for a future update :)


variable importance option

Hello!
First of all, I highly appreciate your efforts for TFDF
Found that there are multiple options for variable importance such as NUM_AS_ROOT
variable_importance = model.make_inspector().variable_importances()['NUM_AS_ROOT']

Could you let me know which option I should use to get similar importance list as sklearn?
Where can I get the detailed descriptions on those options? (How to use, what they mean)

Thank you!

I ran the feature significant and compared the results to Sklearn output. not only the results are different, but also the results that I'm getting using this implementation doesn't make any sense(using the info that I have about my data).
for example a feature that is constant is one of the most significant features(it got the heights value).
maybe I don't know how to read the output properly?
("data:0.33" (1; #27), 235)
this means that feature number 27 got score of 235?
Tsachi

Thanks @Howard-ll , we are happy to hear the tools are useful!
There are various definitions of "feature importance" -- they are all metrics about the model/dataset, but there is not an absolute "truth" or best one.
Now we should have a clear documentation page with the list of all feature importances we support -- with pointers to the papers that define some of them -> making this as an "enhancement" for us to work on.

The list of features importances and their definition is given here in the Yggdrasil user manual.
model.make_inspector().variable_importances() returns a list of tuple ([py_tree.dataspec.SimpleColumnSpec](https://www.tensorflow.org/decision_forests/api_docs/python/tfdf/inspector/SimpleColumnSpec), float)  (see doc).
"data:0.33" (1; #27) is the representation of a SimpleColumnSpec object with the format `"{feature name}" (type idx, #column_idx)" (see here). The last displayed value is the variable importance value (235 in this case).
Note that different variable importances have different semantics. Unless specified, the greater the value, the most important the feature. NUM_AS_ROOT is an exemple of exception (see its definition).

Could you let me know which option I should use to get similar importance list as sklearn?

Sklean's "mean decrease in impurity" is likely close to the SUM_SCORE variable importance in.
Similarly, Sklean's "based on feature permutation" is likely close to the MEAN_DECREASE_IN_ACCURACY.

Where can I get the detailed descriptions on those options? (How to use, what they mean)

The Variable importance section of the user documentation and the model specific documentation (for example, Random Forest).
The variable_importances() method is used in both the beginner and advanced colabs. However, we have not yet published  any example of use of feature importance (e.g. for feature selection).


CMS: Expiry date shortcut buttons colour issue

Background colour should be dark when selected.
Look at text editor buttons for selected colour:

Also check if the white text colour for selected buttons is needed

The classes that we setted in the LButton component were in conflicts with what we had in EditContentBasic. I think now it is solved


一加6T-lineageos-18.1-kernelsu-0.9.4 刷了assistant疯狂重启，卡lineageos启动页面，黑屏

Describe the bug

Steps To Reproduce

Context

Device:
OS:
Version of Magisk, KSU or APatch:
Other Root Module(s): None
LSPosed Module(s): None

Logcat

Attach logs, will you? I don't see why it wouldn't work on LOS 18.1


Kinecosystem/master

whoops. this does not belong here. apologies.

No worries Ami
Sent from my iPhone

On Dec 11, 2018, at 7:51 AM, Ami Blonder<EMAIL_ADDRESS>wrote:
whoops. this does not belong here. apologies.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


social media links open in a new tab.

ty looks very good! will merge here in a second

Thanks @carlostojal !


Using Magic Form on October CMS 3

Hello,
I installed Magic Form on October CMS 3. When I want to send a test form I got this message:
`"Email "0" does not comply with addr-spec of RFC 2822." on line 54 of /Users/.../vendor/symfony/mime/Address.php
What does this error mean?

Plugin is not under development anymore.
You can read more here: https://github.com/skydiver/october-plugin-forms/discussions/267
Thanks.-

I'm getting the same error...

I found a solution:
Change function sendTo to send
classes/SendMail.php
line 59:
Mail::send($template, $data, function($message) use ($properties, $post, $files) {
$message->to($properties['mail_recipients']);


TravisCI Build continue to fail

TravisCI builds fail. On local computer everything ok...I'll try on a VM.

Issues only on linux builds, on osx everything works fine.

Still failing.... 🤕

Still failing, maybe it's a Travis bug.

Solved. Check .travis.yml latest changes.


language refresh issues

Settings > set a langauge > refresh = language goes back to previous, but stuff is in the language you selected.
settings > set a language > save. In a new tab, load an odysee page - language is not set to what you just configured
Maybe we need to store language in wallet/local data?

Also, not all areas are translated until you take an action:

Hi, I am a junior developer with 1.5 years of experience and a huge fan of this project. I decide to contribute. If I could join the team, it is a blessing for me.
I examined the codebase a little bit, starting with this issue. I am thinking about how language is implemented etc. I found out that USER_STATE_POPULATE action causes the problem, but I could not figure out the solution yet.
I have not analyzed @infinite-persistence 's solution yet, but I think the bug still exists. It behaves weirdly when changing language a second time. The previous language occurs in the select element.

Changing language a second time causes this. Maybe it should be another task.
https://www.loom.com/share/7f0a24ea397e4f99a3c1703babe4d0b3

Hey @gadabra sorry for not replying earlier! Would Appreciate any help with this issue

The tab-sync PR alleviated the remaining issue, but I do see a design problem in that the following all try to set each other in various places:

browser locale
local storage
redux

The slower API sync also causes conflicts between tabs.

[ ] Resolve the settings in a proper (uni-directional) way.  Going through redux rules should do the trick, I think.

More notes:

[ ] When localStorage is not available, the effective language doesn't match the drop-down selection (no-op)
[ ] We might need to always check localStorage's availability

localStorageAvailable is a one-time eval of Boolean(window.localStorage).
The screenshot suggests that window.localStorage could later change to null, perhaps through browser settings or from the GDPR thing

It seems like the error message is a lie. If the below is true, try-catch is the only proper way to access local storage -- can't just check if it's defined, since it could later be full.


Empty related table causes results to be empty

In
Line::search('fish', ['title','tags.tag'])->get();

if the related table (here "tags") is empty the search is also empty, even though the search term exists in the master table field.

Sorry - not correct. There is a problem but I am still working out what it is. I will post again once I know.


cargo run -p and cargo test -p uses different working directories

Problem

In a workspace setting, it seems that cargo test -p and  cargo run -p are using different working directory:

cargo test -p <package> uses the package directory as the default directory, while
cargo run -p <package> uses the workspace directory as the default directory.

Steps

create a new cargo workspace with one package foo inside.
create a text file bar.txt in src/ inside the package foo.
create a test inside the package foo that read the file bar.txt using path src/bar.txt.
In the main function of package foo, also read the file bar.txt using path src/bar.txt.
cargo test -p foo will run successfully (i.e. able to read bar.txt).
cargo run -p foo will fail (cannot find bar.txt).

Possible Solution(s)

cargo test -p and cargo run -p should use the same default working directory. IMO, both should use the package directory (not the workspace directory) as the default directory.
Notes
Output of cargo version:
cargo 1.43.0 (2cbe9048e 2020-05-03)

I think it is somewhat intentional that run uses the current directory.    It would also be a breaking change, so it would likely be too disruptive to change.
We have been talking about adding a -C argument that would change the current directory, though you would need to pass a path instead of a package name.

Presumably, that allows tests to reliably access paths from the package they are defined in.

I'm going to close this, as this is expected behavior and unlikely to change.  In the future, the -C argument may be an alternative approach to getting the desired behavior.


Use Observables instead of Promises to fetch cards

Hi,
I was trying to connect to the UI to the core using the core's REST API endpoint. That is when I stumpled over the CardService.getCards() method which returns a Promise<Card[]>. There seem to be quite some disadvantages when using a Promise compare to an Observable.
Consider these points (taken from here):

Don’t hit the search endpoint on every key stroke.
Treat the search endpoint as if you pay for it on a per-request basis. No matter if it’s your own hardware or not. We shouldn’t be hammering the search endpoint more often than needed. Basically we only want to hit it once the user has stopped typing instead of with every keystroke.
Don’t hit the search endpoint with the same query params for subsequent requests.
Consider you type foo, stop, type another o, followed by an immediate backspace and rest back at foo. That should be just one request with the term foo and not two even if we technically stopped twice after we had foo in the search box.
Deal with out-of-order responses.
When we have multiple requests in-flight at the same time we must account for cases where they come back in unexpected order. Consider we first typed computer, stop, a request goes out, we type car, stop, a request goes out. Now we have two requests in-flight. Unfortunately the request that carries the results for computer comes back after the request that carries the results for car. This may happen because they are served by different servers. If we don’t deal with such cases properly we may end up showing results for computer whereas the search box reads car.

This 7 minute video gives a nice overview of why wants to use Observables over Promises: https://egghead.io/lessons/rxjs-rxjs-observables-vs-promises
I understand that his requires some changes to how card objects get filtered and passed around.

@kwk
Observables work great for something like Search but not for a single Get call. Maybe if we are trying to keep the list up to date then we can use Observables. It is also worth noting that we are using the Promise wrapper around Observables. We are not using a true Promise here.
All that being said, I do want to use Observables as much as possible.  I will look into this more over the next several weeks.
As to the error you were working on, I fixed it. :-)

I think we can close this issue.

Agreed


Async keystore + Authority-Discovery async/await

This PR introduces:

A keystore implementation that forwards requests through "backends"
Currently, only a "local" backend is supported but a remote one can be introduced.
All keystore methods are async
SyncCryptoStore wraps the asynchronous keystore to hide block_on from sync crates.
Authority discovery was converted to async/await with the use of the async version of the keystore.
All other crates are temporarily using SyncCryptoStore until converted to use async/await

The changes in the Cargo.lock seem to have downgraded a lot of libraries, I suppose that this isn't intentional?
Could you copy over the Cargo.lock from the master branch? :pray:

The changes in the Cargo.lock seem to have downgraded a lot of libraries, I suppose that this isn't intentional?
Could you copy over the Cargo.lock from the master branch?

Ah thanks for looking at that... must have screwed things up when i merged latest master. Will fix

@rakanalh https://github.com/mxinden/substrate/commit/a1a0f8492f13e626ef85be14f31086ad2e0fd4dc should resolve your issue of the compiler not being able to infer the event stream to be Sync.

@rakanalh let me know once this is ready for another review.

@mxinden thanks for helping out in fixing that one :sweat_smile: I will ping once i resolve the rest of the feedback.

The failure i am seeing now is:
Thread 'tokio-runtime-worker' panicked at 'cannot execute `LocalPool` executor from within another executor: EnterError', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-executor-0.3.5/src/local_pool.rs:78

It's interesting that block_on would start a new executor compared to using the existing one if there is one already running.

@rakanalh https://github.com/mxinden/substrate/commit/8509a1b5998af0c2cdaaf7811bc7abcb99f7839d adjusts the authority discovery lookup_throttling test.

@rakanalh https://github.com/mxinden/substrate/commit/e4bb4b958b7fd75e6be70cbfda3a27789afb032d adjusts the authority discovery triggers_dht_get_query test.

Thank you so much @mxinden ... just cherry picked your commits and will monitor CI and see what remains to be done.

https://github.com/mxinden/substrate/commit/131240ed26e3d0399fbeee86ec35afc04b7987cc would remove the timing assumption in unit tests.
https://github.com/mxinden/substrate/commit/408d499050b187ac6242b0dfc6a12ed8c70e57ab reverts the changes to the termination unit test, pinning worker to the stack instead.
Let me know what you think @rakanalh.

PS: Force-with-lease pushed to remove the "unfinished" commit i had earlier

Following up on the discussion whether we still need SyncCryptoStorePtr started above (https://github.com/paritytech/substrate/pull/7000/#discussion_r492047124).
Static dispatching
My initial thought was to use static dispatching instead of dynamic dispatching. While I still think it would be worth the effort in the long run, I think dragging all the trait bounds through the code base is way to involved for this already large pull request.
Alternative suggestion
Instead of implementing SyncCryptoStore for some T: CryptoStore, implement it for Arc<dyn CryptoStore>:
impl SyncCryptoStore for Arc<dyn CryptoStore> {
  // ...
}

SyncCryptoStore is used in places where some T: CryptoStore is already wrapped in an Arc, thus there is no additional overhead. With the above there is no need for SyncCryptoPtr anymore thus (a) mental overhead is reduced and (b) new code can leverage the asynchronous methods on CryptoStore directly.
Commit https://github.com/mxinden/substrate/commit/46995341d1f0d1a668805f68d89e04ed284c57a6 updates client/finality-grandpa accordingly.

client/finality-grandpa: Use Arc<dyn CryptoStore> instead of SyncXXX
Instead of using SyncCryptoStorePtr within client/finality-grandpa,
which is a type alias for Arc<dyn SyncCryptoStore>, use Arc<dyn CryptoStore>. Benefits are:

No additional mental overhead of a SyncCryptoStorePtr.

Ability for new code to use the asynchronous methods of CryptoStore
instead of the synchronous SyncCryptoStore methods within
client/finality-granpa without the need for larger refactorings.

Note: This commit uses Arc<dyn CryptoStore> instead of
CryptoStorePtr, as I find the type signature more descriptive. This is
subjective and in no way required.

Let me know what you think.

@mxinden I ended up cherry-picking your commit and implementing the rest of the codebase on the same concept.
One note though: for KeystoreExt(Arc<dyn SyncCryptoStore>) i had to wrap Arc<dyn CryptoStore> with an additional Arc since we now implement SyncCryptoStore for Arc<dyn CryptoStore>. This is very nasty, i'll try to see if there are any alternatives. I think one alternative is to keep the implementation impl <T: CryptoStore> SyncCryptoStore for T but i am torn by the redundancy.

@mxinden I ended up cherry-picking your commit and implementing the rest of the codebase on the same concept.
One note though: for KeystoreExt(Arc<dyn SyncCryptoStore>) i had to wrap Arc<dyn CryptoStore> with an additional Arc since we now implement SyncCryptoStore for Arc<dyn CryptoStore>. This is very nasty, i'll try to see if there are any alternatives. I think one alternative is to keep the implementation impl <T: CryptoStore> SyncCryptoStore for T but i am torn by the redundancy.

How about defining SyncCryptostore as a pure extension trait, thus not requiring any duplicate code nor double Arcing:
pub trait SyncCryptoStore: CryptoStore + Send + Sync {
	/// Returns all sr25519 public keys for the given key type.
	fn sr25519_public_keys(&self, id: KeyTypeId) -> Vec<sr25519::Public> {
		block_on(CryptoStore::sr25519_public_keys(self, id))
	}

       // ...
}

https://github.com/mxinden/substrate/commit/34ae7d63b140c8cefbf5ed8a9a49b6c5886c4787 applies this pattern and updates all of its usages. It does introduce some rather not-so-pretty dereferencing when used with KeystoreExt, but in my mind that is well worth the reduced complexity.
let keystore = &***self.extension::<KeystoreExt>()
  .expect("No `keystore` associated for the current context!");
SyncCryptoStore::ecdsa_generate_new(keystore, id, seed)
  .expect("`ecdsa_generate` failed")

Let me know what you think.

@rakanalh https://github.com/mxinden/substrate/commit/fd8ff85c91448dec35149b20694279db21960db1 is a small clean up commit replacing all usages of Arc<dyn CryptoStore> with CryptoStorePtr.

* To solve this, I would propose we write a dead simple futures executor:

while future.poll().not_ready() {}

Reading this https://rust-lang.github.io/async-book/02_execution/04_executor.html and writing some minimal code we should have something working in like 100 lines of code.

I talked to @mxinden and we looked into async-std and they support recursive block_on calls, so it would be nice if you switch all the calls that make the keystore sync to async_std::block_on and revert all the changes you have done to workaround these nested block_on calls.

To solve this, I would propose we write a dead simple futures executor:

while future.poll().not_ready() {}

No opinion on whether it is a good idea, but you need to put this inside of the loop, other you might have an undefined behaviour.

To solve this, I would propose we write a dead simple futures executor:

while future.poll().not_ready() {}

No opinion on whether it is a good idea, but if we end up doing this we should put this inside of the loop, otherwise you might have an undefined behaviour.

I already said above that this was a stupid idea by me :D

I talked to @mxinden and we looked into async-std and they support recursive block_on calls, so it would be nice if you switch all the calls that make the keystore sync to async_std::block_on and revert all the changes you have done to workaround these nested block_on calls.

To extend a bit on this:
The async_std block_on allows recursive calls (see code). Using async_std's block_on in SyncCryptoStoreshould allow us to usefutures::executor::block_onin code calling any methods inSyncCryptoStore, thus unit tests can continue calling futures::executor::block_on` without panicing.
@rakanalh what do you think?

I like the idea of using async-std for block_on in the keystore. I will do that and revert my changes to other places where i have changed the runtime.

@bkchr Looked into using async-std in sp-keystore and apparently, when compiling wasm the version of block_on we get from async-std does not return a value: https://docs.rs/async-std/1.6.4/src/async_std/task/block_on.rs.html#37

OMG -.-
The documentation to this function are lying ;) If I understand it correctly, they just let this future run and return without waiting. WTF...
Maybe that is something we should ask for in upstream?
For the time being we could continue with async-std on native and use some self written hack executor for wasm? In the end it will just stay in some hot loop waiting for the result... And use spin_hint as proposed by @tomaka. (While I'm not sure if that will do anything when being called in a wasm VM)
@mxinden @tomaka or do you have better ideas?

I am pushing a revert to the change for using async-std inside the keystore and i would like to ask to merge this PR as it stands and we can later on follow up on using async-std again.

The documentation to this function are lying ;) If I understand it correctly, they just let this future run and return without waiting. WTF...

By design, block_on cannot work when target_os = "unknown". Either the target OS is really unknown and you can't block the thread, or you assume that target_os = "unknown" actually means "a JavaScript environment", in which case you still can't block the thread since JavaScript is fully asynchronous.

The documentation to this function are lying ;) If I understand it correctly, they just let this future run and return without waiting. WTF...

By design, block_on cannot work when target_os = "unknown". Either the target OS is really unknown and you can't block the thread because there's no way to, or you assume that target_os = "unknown" actually means "a JavaScript environment", in which case you still can't block the thread since JavaScript is fully asynchronous.
But yes, this function shouldn't be available in the first place when target_os = "unknown".

Why can it not block when the target_os is unknown? I would at least expect a loop that is doing busy waiting. Not a perfect solution, but as you already said, there is no better way.

@bkchr Keystore related code from sp-core has been moved into sp-keystore as requested.

Just for the record, I'm still not happy with the way the blocking is implemented now.

I agree with @bkchr . We should at least have ticket describing what needs to be done to move away from it/get rid of the SyncCryptoStore and link that into the docs there as a FIXME imho.

I agree with @bkchr . We should at least have ticket describing what needs to be done to move away from it/get rid of the SyncCryptoStore and link that into the docs there as a FIXME imho.

That is not the point, we will never (as long as the runtime doesnt support futures) get rid off SyncCryptoStore`. So, I really would like to have some block_on calls that support nesting (use async_std) and support wasm (we probably need our own implementation that is doing busy waiting)

the the issue should state that. also fine by me. just don't want to add more technical debt without having it clearly and transparently documented.

Continuing on the block-on and SyncCryptoStore discussion:
I would like to propose an additional alternative approach, relieving us of the
need for block-on for now, while still being future-proof.

As far as I know, please correct me if I am wrong, the crypto-related code
altered in this pull-request (CryptoStore, SyncCryptoStore +
client/keystore/*) needs to compile for the following platforms:

Native Linux / Mac/ Windows

In-browser WASM

The Substrate runtime is solely a user of this logic through one of the above
platforms, neither the CryptoStore, the SyncCryptoStore nor
client/keystore needs to compile for the runtime WASM environment.
The goal of this whole effort is to eventually support remote keystores, e.g.
HSMs. But this use-case is only needed on any of the native (Linux, Mac,
Windows) platforms, not on the in-browser WASM platform. Again, please correct
me if that is a wrong assumption.
Instead of SyncCryptoStore being an auto trait, automatically implemented for
everything implementing CryptoStore I am proposing to have each concrete
keystore implementation implement SyncCryptoStore specifically.
The CryptoStore trait would stay untouched:
/// Something that generates, stores and provides access to keys.
#[async_trait]
pub trait CryptoStore: Send + Sync {
	/// Returns all sr25519 public keys for the given key type.
	async fn sr25519_public_keys(&self, id: KeyTypeId) -> Vec<sr25519::Public>;
  // ...
}

The SyncCryptoStore would not offer any default methods, but would keep its
trait bound on CryptoStore as any synchonous keystore implementations can
always fulfill the asynchonous interface. (With the trait bound parts of old
components e.g. client/finality-grandpa can already start using the
asynchonous methods through CryptoStore in some places without having to
switch all at once).
pub trait SyncCryptoStore: CryptoStore + Send + Sync {
    /// Returns all sr25519 public keys for the given key type.
    fn sr25519_public_keys(&self, id: KeyTypeId) -> Vec<sr25519::Public>;
    // ...
}

Given that SyncCryptoStore is not an auto trait for all types implementing
CryptoStore anymore, our concrete client/keystore needs to implement it
manually:
impl SyncCryptoStore for LocalKeystore {
	fn keys(
		&self,
		id: KeyTypeId
	) -> std::result::Result<Vec<CryptoTypePublicPair>, TraitError> {
    // ...
	}
  // ...
}

Given that LocalKeystore now implements SyncCryptoStore, the function logic
does not need to be duplicated for the CryptoStore implementation of
LocalKeystore as it can just delegate to the SyncCryptoStore methods:
impl CryptoStore for LocalKeystore {
	async fn keys(
		&self,
		id: KeyTypeId
	) -> std::result::Result<Vec<CryptoTypePublicPair>, TraitError> {
    SyncCryptoStore::keys(self, id)
	}
  // ...
}

Currently, e.g. in client/finality-grandpa we are passing an Arc<dyn CryptoStore>.
That would need to change to an Arc<dyn SyncCryptoStore> given that
SyncCryptoStore is not implemented for all types that implement CryptoStore
anymore. Components like client/authority-discovery could still accept a
Arc<dyn CryptoStore> given that it is able to await crypto operations.
In the future we want to support remote keystores. But, as said above, only on
the native platforms. Thus we can restrict RemoteKeystore to
#[cfg(not(target_os = "unknown"))] only:
#[cfg(not(target_os = "unknown"))]
struct RemoteKeystore {
       // ...
}

RemoteKeystore could implement CryptoStore easily, as all of its methods are
async and SyncCryptoStore via async_std::task_block_on which supports
nested block_on calls.
#[cfg(not(target_os = "unknown"))]
impl CryptoStore for RemoteKeystore {
	async fn keys(
		&self,
		id: KeyTypeId
	) -> std::result::Result<Vec<CryptoTypePublicPair>, TraitError> {
    // ...
	}
  // ...
}

#[cfg(not(target_os = "unknown"))]
impl SyncCryptoStore for RemoteKeystore {
	fn keys(
		&self,
		id: KeyTypeId
	) -> std::result::Result<Vec<CryptoTypePublicPair>, TraitError> {
    async_std::task::block_on(CryptoStore::keys(self, id))
	}
  // ...
}

To summarize, the above allows old components like client/finality-grandpa to
still use the synchonous crypto operations via SyncCryptoStore, while allowing
new components to use the new asynchonous methods through CryptoStore. In the
case of a local keystore, e.g. on in-browser wasm, no block_on is needed. In
the case of a remote keystore one can still use synchronous methods via
async_std::task::block_on on the native platforms, while at the same time
allowing unit-tests to use futures::executor::block_on.

@mxinden sounds like a great idea to me!

@mxinden @bkchr I have implemented the suggestion above and reverted the change which imports async-std and replaces usage of futures::executor::block_on.
However, you guys didn't like introducing KeystoreContainer before, which was removed but then i had to restore that. This is because if i end up returning Arc<dyn SyncCryptoStore>, this type cannot be coerced to Arc<dyn CryptoStore>. Restoring KeystoreContainer allows me to coerce Arc<LocalKeystore> into both Arc<dyn SyncCryptoStore> and Arc<dyn CryptoStore>.

The PR should be good to merge if no further feedback is provided.

@bkchr i have had some review feedback from @ordian on the companion PR but i also think it's good to merge as well.

Some last nitpicks, after that it should be good to merge.

bot merge


Configure codecov to allow a 2% drop before complaining

Fixes #1476.

Will merge on green, then rebase #1447 to check that it has the desired effect.
(All of this may be in a few hours, when I land in Kiev...)

Given the length of the AppVeyor queue, I'm going to merge now, rebase and push #1447 so it's got a chance of actually showing up...


[Feature request] Lights off when video is playing option + transition

Keep up the good work with YouTubeCenter!
Would be nice to have an option to activate lights off when the video is playing, and back on if it is paused. Additionally an optional fade transition for aesthetics. +1 anyone?

Fairly simple to implement, just need to hook to the player state and listen for any event equal to 1 or 3 to turn lights off, and the opposite to turn on. The fade animation can be made via CSS's transition rules.
@YePpHa I'm using something like this:
function playerState(state) {
    if (state === 1 || state === 3) {
        document.documentElement.classList.add("lights_out");
    } else {
        document.documentElement.classList.remove("lights_out");
    }
}
playerApi.addEventListener("onStateChange", playerState);

Remember to add transition in the CSS rule for the fade in/out animation.


AWS::EC2::PlacementGroup validators

Add validator to AWS::EC2::PlacementGroup Strategy and SpreadLevel

Thanks! Merge manually via 3632835 due to conflicts from earlier commits.

Thanks