Need reference for PCMAX (default setting for PC in CL apparently)

After some sleuthing in the Python code and some performance testing with the help of Pablo Puig and Bryan Andrews, I've come to the conclusion that the default setting for PC in CL is maxP. That is an algorithm of mine called PCMAX. The only reference for for this algorithm currently in the literature that I know of is an arXiv tech report I put out in 2016, this one:
Ramsey, J. (2016). Improving accuracy and scalability of the pc algorithm by maximizing p-value. arXiv preprint arXiv:1610.00378.
Anyway, the performance of PC in CL is the same as the performance of PCMAX in Tetrad (and different from the performance of PC in Tetrad.)
If this is going to be made the default setting for PC for CL, perhaps this tech report should be referenced for the PC algorithm in the documentation? Otherwise no one will know what the algroithm is. I'll see if I can't sent that to a conference somewhere to get it published. I didn't realize that was being made the default here.

Sorry, I should say, if you can find a published reference for it, use that. Or at least describe the procedure in the documentation. I think.

Sure, thanks so much for your suggestions. I've added your paper as the reference for maxP and definiteMaxP (uc_rule). Please let me know if anything is wrong.
BTW, I'm not totally sure that our default setting is PCMAX, since the default value for 'uc_rule' is 0, as follows:
def pc(     data: ndarray,      alpha=0.05,      indep_test=fisherz,      stable: bool = True,      uc_rule: int = 0,      uc_priority: int = 2,     mvpc: bool = False,      correction_name: str = 'MV_Crtn_Fisher_Z',     background_knowledge: BackgroundKnowledge | None = None,      verbose: bool = False,      show_progress: bool = True,     node_names: List[str] | None = None,  ):
where

uc_rule: how unshielded colliders are oriented. Default: 0.
0: run uc_sepset.
1: run maxP. Orient an unshielded triple X-Y-Z as a collider with an additional CI test.
2: run definiteMaxP. Orient only the definite colliders in the skeleton and keep track of all the definite non-colliders as well.

It's very likely that I misunderstood something, as I'm not familiar with PCMAX. It might be the case that we are using PCMAX somewhere else?

Sorry, this is for a problem with 20 nodes, 40 edges, 1000 samples, averaged over 10 runs.
p = 20
a = 4
d = a / (p - 1)
n = 1000
runnum = 10

Sorry, linear, Gaussian, using Fisher Z for PC_CL and PCMAX_CC--but perhaps the tests are behaving differently in the two platforms--I'll look into it...

Anyway, I am very curious about this. If you are getting PCMAX performance from the PC sepsets step alone (without maxp) that is something I don't know how to do. Wayne Lam did a study of this for his MA thesis in Python and didn't see how to do it either, with several variants of the procedure. So I need to figure out exactly what the CL code is doing. I'll forward future comments offline and close this.

Yeah I get the same performance whether I set ucrule to 0 or 1. Setting it to 2 gives a better result, close to GRASP_CC but with lower recalls.
	Alg   AP      AR     AHP   AHPCE     AHR   AHRCE   McAdj   McArr   F1Adj   F1Arr     SHD    E 	1	1.00	0.71	0.95	0.95	0.52	0.73	0.81	0.68	0.83	0.67	12.40


Decorate process metrics with kubernetes metadata

Codecov Report
Attention: Patch coverage is 14.28571% with 6 lines in your changes missing coverage. Please review.

Project coverage is 35.67%. Comparing base (e94be3e) to head (6745b57).

Files
Patch %
Lines

pkg/internal/infraolly/process/status.go
0.00%
6 Missing :warning:

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #917       +/-   ##
===========================================
- Coverage   79.04%   35.67%   -43.38%
===========================================
  Files         131      130        -1
  Lines        9832     9830        -2
===========================================
- Hits         7772     3507     -4265
- Misses       1558     5929     +4371
+ Partials      502      394      -108

Flag
Coverage Δ

integration-test
?

k8s-integration-test
?

oats-test
35.67% <14.28%> (-0.02%)
:arrow_down:

unittests
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Improve callback types and IDE integration

Is your feature request related to a problem? Please describe.
Currently, all Callback types inside the package use unnamed parameters, e.g.
typedef OnInfiniteQuerySuccessCallback<T> = void Function(T);.
This is a neutral choice but uncommon in the dart ecosystem. The reason for this is that IDE autocompletion will use the name of the function type to insert code snippets.

This pattern is so common that VScode does not account for a scenario where a name is absent, and therefore inserts an awkward space where it would follow.
Describe the solution you'd like
Refactor all function types to also include parameter names.
A simple example would be
typedef OnInfiniteQuerySuccessCallback<T> = void Function(T data);.

Implemented in cached_query_flutter: ^2.4.1


fix: backport #8979, re-encode url to prevent fs.allow bypass (fixes #8498)

Description
backport #8979
fixes #8498
Additional context

What is the purpose of this pull request?

[x] Bug fix
[ ] New Feature
[ ] Documentation update
[ ] Other

Before submitting the PR, please make sure you do the following

[x] Read the Contributing Guidelines.
[x] Read the Pull Request Guidelines and follow the Commit Convention.
[x] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
[x] Ideally, include relevant tests that fail without this PR but pass with it.

Thanks for taking care of this @sapphi-red!


Unable to see the Service Tag in Azure Firewall

Even though this page says that it is applicable for NSG and Firewall, I am unable to see the service tags for Power Platform in Firewall.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 08217244-82bb-13b5-46f7-454801d8f865
Version Independent ID: 4a8b0a32-bf57-7544-c60d-6b7f250c4477
Content: Azure service tags overview
Content Source: articles/virtual-network/service-tags-overview.md
Service: virtual-network
GitHub Login: @allegradomel
Microsoft Alias: kumud

@rajyraman , thank you for your feedback. We'll investigate and update as appropriate!

@rajyraman , I checked with the Azure Firewall & Service tag Product group teams and they mentioned that the API seems to be not fetching all the service tags via Portal and they will be fixing it in the next release cycle. In the meantime, if portal is not displaying the required tag, customers should still be able to use PowerShell/Rest API to add it to their network rules.
I tried this in my lab and was able to add Power Platform service tag in Azure Firewall network rules via PowerShell:

So, I would request you to use Azure PowerShell/Rest API to add the service tags which are not available via Portal in Azure Firewall network rules till it is fixed by the PG team.
PowerShell commands for your reference:
$FirewallName = "AzureFirewall"
$ResourceGroup = "AzureFirewall-RG"
$azfirewall = Get-AzFirewall -Name $FirewallName -ResourceGroupName $ResourceGroup
$rule = New-AzFirewallNetworkRule -Name "AllowPowerPlatform" -Description "Allow access to Azure Power Platform" -SourceAddress "<IP_ADDRESS>/16" -DestinationAddress PowerPlatformInfra -DestinationPort * -Protocol TCP
$ruleCollection = New-AzFirewallNetworkRuleCollection -Name "PowerPlatform" -Priority 1000 -Rule $rule -ActionType Allow
$azFirewall.NetworkRuleCollections.add($ruleCollection)
Set-AzFirewall -AzureFirewall $azfirewall
Official Doc: https://docs.microsoft.com/en-us/azure/firewall/service-tags
We are closing this issue for now. If there are further questions regarding this matter, please reply and we will gladly continue the discussion.


Add signature verification to xdr viewer for transaction envelopes

This may need some work making it proper React code. It also might need some touchup on design. I kept it pretty simple.
Basically, whenever the user types or pastes in an xdr of type Transaction Envelope, this will asynchronously query for all signers on that transaction (base source account and all operation source accounts). It will then, for each signature on the transaction check to see if it is valid for a given signer -- the red means invalid, green means valid, and during loading, black means not yet loaded.
Here is what it looks like for a transaction has two valid signers and one invalid signer:

Closes #343

Fails for hash(x)

Now passes for pre-auth tx and hash x signers, should be good for review @bartekn

@bartekn thanks for the feedback. Utilizing Stellar-base and removing trivial hint optimization lead to much cleaner code.
Also squashed.

Fixed per your latest comments... you were right, hash(x) still failed under the previous commit. It should be fixed now with buffer/hex/bin sorted out a bit more.

Fixed for presentation level feedback. However, left a reply on a comment where I was a bit confused.

@bartekn ok updated for latest comments. Also squashed.


A11y issues

What version of daisyUI are you using?
4.4.12
Which browsers are you seeing the problem on?
Chrome
Reproduction URL
https://stackblitz.com/edit/daisyui-sveltekit-f7y6p2?file=src%2Froutes%2F%2Bpage.svelte
Describe your issue
There are some issues with how the library uses the label element.
Example from the docs:

     	This is a label

     <select class="select select-bordered w-full max-w-xs">
     	<option disabled selected>Who shot first?</option>
     	<option>Han Solo</option>
     	<option>Greedo</option>
     </select>

This produces a A11y error: "A11y: A form label must be associated with a control"
When putting the select element into the label we get the issue, that the label is displayed left to the input, not on top:

     	This is a label
     	<select class="select select-bordered w-full max-w-xs">
     		<option disabled selected>Who shot first?</option>
     		<option>Han Solo</option>
     		<option>Greedo</option>
     	</select>

As i am building a big platform and a lot of forms so i don't want any hacky solutions, would be good to know what the creators intended. Thanks!

This is not an issue from the library itself, but rather from the example on the docs.
Adding an ID for the <select> would fix it:
https://stackblitz.com/edit/daisyui-sveltekit-4puoch?file=src%2Froutes%2F%2Bpage.svelte

+   👈 Use the same ID here
    Card Number

+  <select class="select select-bordered w-full max-w-xs" id="my-id"> 👈 Use ID for select
    <option disabled selected>Who shot first?</option>
    <option>Han Solo</option>
    <option>Greedo</option>
  </select>

Read more: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/label
The ID is something developers must add themselves because the ID must be unique and it must be used based on the context of the page.
I'm going to add a sample ID in the docs example today.
Let me know if you have a question.

I understand.
It's better for accessibility to generate unique IDs for the fields because that way, screen readers would know exactly which label is associated with which input. (This is not about daisyUI. It's just how HTML must be written)
Another thing you can do is to use a div instead of label and use the label class for the div. It may look funny but it makes sense because if you can't use unique IDs for the inputs wrapping them inside a label is the only way. While the label class from daisyUI is not necessarily for  element. It's for the visual label on top of the input.

     	This is a label

     <select class="select select-bordered w-full max-w-xs">
     	<option disabled selected>Who shot first?</option>
     	<option>Han Solo</option>
     	<option>Greedo</option>
     </select>


[Question] How to not include "Managed" gateway by default when publishing APIs

Release version
v5.1.1
Question Details
Our APIM solely uses self-hosted gateways and thus does not require every API to be assigned to the "Managed" gateway.
However, even though the API is only assigned to a specific self-hosted gateway, the "Managed" gateway is still getting assigned to it.
I added a "Managed" gateway into the gateways directory and set the APIs to an empty array.
However, this only works when not using the commit id during publishing, as the Managed gateway file is not changed when I add a new API.
How is this supposed to be handled?
Expected behavior
"Managed" gateway is not assigned whenever I assign it to some custom gateways.
Actual behavior
"Managed" gateway is always assigned to new APIs.
Reproduction Steps
Create a fresh API and assign it to a custom gateway (not the "Managed" one)
Folder structure:

Content of gateways/self-hosted/apis.json
[
  {
    "name": "my-new-api"
  }
]

Content of gateways/Managed/apis.json
[]

Since the gateways/Managed/apis.json has not been touched/changed during the commit, the publisher does not update the Managed gateway assignment, so the new API is actually having both the "Managed" and "self-hosted" gateways assigned to it.

There is a similar issue with products in #245. This new implementation will fix it, and we're adding a similar one for gateways. Should be in the next major release.

I'm not sure what your baseline is, so I'll make some assumptions and walk through a few scenarios.
Let's say you start with a managed and self-hosted gateway in the portal. You have zero APIs. You run the extractor.  You run the extractor. Your folder structure will be:

artifacts

gateways

managed

gatewayInformation.json
apis

apiA

gatewayApiInformation.json

self-hosted

gatewayInformation.json
apis

apiB

gatewayApiInformation.json

apis

apiA

apiInformation.json

apiB

apiInformation.json

You then decide to do a code-first deployment: instead of making changes via the portal & extracting, you want to change the git artifacts and publish that commit. In this change, you remove API A from the managed gateway, add a new API C to the self-hosted gateway, and add a new API D that won't be associated with any gateways. Your folder should look like this:

artifacts

gateways

managed

gatewayInformation.json

self-hosted

gatewayInformation.json
apis

apiB

gatewayApiInformation.json

apiC

gatewayApiInformation.json

apis

apiA

apiInformation.json

apiB

apiInformation.json

apiC

apiInformation.json

apiD

apiInformation.json

The publisher will detect the following changes and make them accordingly when it runs:

artifacts/apis/apiC/apiInformation.json was created. API C will be created in APIM.
artifacts/apis/apiD/apiInformation.json was created. API D will be created in APIM.
artifacts/gateways/managed/apis/apiA/gatewayApiInformation.json was deleted. API A will be removed from the managed gateway.
artifacts/gateways/self-hosted/apis/apiC/gatewayApiInformation.json was created. API C will be added to the self-hosted gateway.

Does that make sense? Let me know if I'm missing something.

Something just clicked for me; I misread your initial question. When you talk about the "managed" gateway, you're not referring to a gateway you create called "managed". You're referring to the built-in managed gateway.
If that's the case, I doubt there's functionality to manage that "managed" gateway. You can't export it as an artifact and "manage" it. You can assign APIs to a self-hosted gateway using the methods described in the previous comment, but I'm not aware of a way to also disassociate them explicitly from the built-in gateway.
I would suggest creating a support ticket and asking whether that's possible (outside of our tools). If they show you how to do it, we can see if the steps are achievable through our tools.

Yes, I am talking about the built-in managed gateway. Sorry for the missing clarity.
I actually experimented and simply created a folder called "Managed" and add an apis.json that has an empty array in it.
When I use the publisher without the COMMIT_ID it actually disassociates the APIs from the built-in gateway.
But as I said, this does not work when only looking at changes within a git commit.

Can you elaborate on what you're seeing? How do you know they're no longer associated with the built-in gateway?

You can see the assigned built-in gateway when you have a look into the API in the portal.
The field "Gateways" either contains the "Managed" + your custom ones, or only the custom ones after I tested it.

Thanks for the information - makes sense now. I think our implementation will partially handle your scenario.
When the extractor runs, it queries APIM for a list of gateways. It then loops through all gateways and exports their associated APIs. The problem is that APIM's list of gateways does not include the managed gateway. We would have to add a step to explicitly ask for the managed gateway's APIs. The APIs would then show up in artifacts/gateways/managed/apis/....
Adding to our backlog for further investigation.

@daviian feel to submit a PR and we can merge accordingly.

@guythetechie - I've had a look into it and locally it looks good so far. However, there is one point I have concerns about.
The APIM API documentation Gateway Api - Create Or Update and Gateway Api - List By Service excludes "managed" from being an allowed value ("Must not have value 'managed'").
However, both APIs work as intended for our use case when passing "managed". Additionally this API is also used from the APIM Portal when assigning/removing APIs from the managed gateway, so I assume the API documentation requires an update.
The only API that returns a validation error is Gateway - Get, so I conditionally exclude fetching the gateway details for the "managed" gateway.
What are your thoughts about this? If you like I can create the PR so that you can have a preliminary look as well.


[sil] Mark all members of SILFunctionBuilder private and make composi…

…tion classes friend clases.
This is the final step towards making sure no one can use a SILFunctionBuilder
except through composition classes.
rdar://42301529

@swift-ci smoke test and merge

Do you have an interest in knowing all the composition classes? If not, this may actually be a case where inheritance is easier to use. (Then you can use protected.)

These are not meant to be substitutable with the original class. So it makes sense to make them composable.

Still, C++ private inheritance supports that.

Jordan and I chatted about this offline. I explained to him that I prefer to use composition instead of private inheritance since it is generally considered a code-smell.

To be clear, I consider it to be a code-smell. I wasn't trying to speak for Jordan.

FWIW: It's probably fine either way. If SILFunctionBuilder were a public API (as in, exported to compiler plugin or something) you would want to use private inheritance to limit access, or even force clients to inherit from a SILFunctionBuilderBuilder... For an internal utility there's really no problem with listing all the clients. I personally think private inheritance tends to obfuscate the code.


Should we avoid linking to the forum ?

As you may know, our access to the forum may be limited in the future. I'm wondering wouldn't it be better to copy-paste useful content from the forum to the repo (adding somewhere on the beginning name of the original author of each text) instead of linking to them?
Nonetheless, I would keep a general link to the forum.

The managers said that they are looking into fixing the problems with the forum.

I'm against to copy-pase, because the limitation of access in the forum has a reason.

Don't copy paste without asking for permission. I'd rather avoid links to the forum

I think a link to the forum is better than copying and pasting everything from the forum here.

For example: The Q&A links should link to the Google Drive File rather than to a separate README where the entire Q&A is pasted.

If we will have access to the forum even after the scholarship will end - then it's good to add links. If we won't have access, then we should find a different solution how to avoid losing most useful content.

Regarding Q&A example - Google Drive File is one solution (thank you @Mycroft1891). PDF stored on AWS?

Ohh right, I forgot forum access might me killed. Sorry. Yes for Q&A AWS of Google Drive with a public link would be enough. As for forums we will have to copy and paste but maybe we can copy and paste the forums content in a file and upload it to Google Drive or AWS.

My problem with Google Drive is this:

Great for editing documents but not for viewing the content

Its because you are using a google doc to save. Use PDF when saving the file

You don't have that problem when you save the files as PDF

For example I downloaded the AMA session and exported it as a PDF, what do you think:
https://drive.google.com/file/d/1dDWySCg16khjWeaJM9WMnPrFJOrrRL1e/view?usp=sharing

AWS is not necessary. We can host them here as assets.  But binary file formats like pdfs are not really useful for version control.

I think a public link to the PDF files does the trick. We don't have to always upload and download the AMA sessions from the repo and everyone can access it from google drive/mega/AWS (whatever we go with)

We can actually remove the AMA folder because all AMA sessions are stored inside a single file. Or do you want me to split them into seperate PDFs ?

I would remove the folder, keep a single file and split the content inside this file by adding headers for each session (to make it easier to find each of them).

Remove the folder, instead of linking to the README.md link to the Google Drive PDF I created. I will update it when the next AMA is out.


Imports in index.ts file are not getting correctly resolved.

"typescript": "3.2.2"

Imports in index.ts file are not getting correctly imported.
Inside engine folder, I have files as

engine

index.ts
engine.ts
test.ts

Inside index.ts,
I have

export { Engine } from './engine';
export { Test } from './test';

When I try to import, Engine class in another file using
import {Engine} from '../engine';
engine is not defined
I get it correct if I write this way.
import {Engine} from '../engine/engine';
This was working fine previously, but after resent ts version upgrade it is giving this error.

@ajafff It is already set

"compilerOptions": {
"allowJs": true,
"target": "es2017",
"module": "commonjs",
"moduleResolution": "node",
"isolatedModules": false,
"jsx": "react",
"experimentalDecorators": true,
"emitDecoratorMetadata": true,
//"declaration": false,
"noImplicitAny": false,
"removeComments": false,
"noLib": false,
"preserveConstEnums": true,
"skipLibCheck": true,
"skipDefaultLibCheck": true,
"suppressImplicitAnyIndexErrors": true,
"sourceMap": true,
"outDir": "release/",
"rootDir": "./",
"watch": true
},

@sheetalkamat It gives tons of logs in vscode terminal. Not sure how to find the useful one.

The full log would be more informative. It would tell you why ../engine was resolved to the file you are seeing.

Presuming that your folder structure looks something like this:
engine
    \index.ts
    \engine.ts
    \test.ts
other-file.ts

Then shouldn't other-file.ts be doing:
import { Engine } from './engine';

.. means "the parent directory of the one that contains this module".

engine
\index.ts
\engine.ts
\test.ts
routes
\tracker.ts
Inside tracker , I am doing,
import { Engine as BotEngine } from '../engine';
Which is shown correct path in vscode as well.

Hmm...  it seems a bit questionable from a structural point of view to import from the index.ts at your own level by way of the parent directory, but yeah, I don’t see any reason that shouldn’t work.

 - engine
    -  \index.ts
    -  \engine.ts
    -  \test.ts
 -  routes
    -  \tracker.ts

server.ts (base level)

In server.ts, I changed,
import {Engine} from './engine/engine';
to
import {Engine} from './engine';
And it started working.

Any recommendations available for writing exports in index.ts and using the exported modules internnally ? ..
this issue is really frustrating to debug : (..

You have to add a slash in the folder (module) name, this will treat index.ts as the folder index.
import {Engine} from '../engine/';

I'm having a similar issue when using baseUrl: "src"
Obs: the error is shown when the file is open, but the code runs smoothly. The import actually works. Maybe VSCode TS intellisense is the one to blame here.
Project structure:
constants
  foo.ts
  bar.ts
  index.ts // Re-exports from foo and bar
  xxx.ts // Re-exports from foo and bar (this was a test(

other-folder
  component.tsx  // Trying to import from 'constants'

Test results from component.tsx
import { FOO } from 'constants';  // ERROR
import { FOO } from 'constants/xxx';  // WORKS
import { FOO } from '../constants';  // WORKS
import { FOO } from 'constants/foo';  // WORKS

This is the error:

This is my tsconfig.json file:
TS version: "typescript": "^4.5.5"
{
  "compilerOptions": {
    "allowJs": true,
    "baseUrl": "src",
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "isolatedModules": true,
    "jsx": "react",
    "lib": ["dom", "dom.iterable", "esnext"],
    "noEmit": true,
    "module": "esnext",
    "moduleResolution": "node",
    "resolveJsonModule": true,
    "skipLibCheck": true,
    "strict": true,
  },
  "exclude": [
    "node_modules",
  ]
}

Obs: This is happening inside my packages/admin project. This is a monorepo using Yarn workspaces. Not sure if this is related or not. TypeScript is installed at the root of the workspace (but I think this error is coming from VSCode TS itself).

This problem just hit me as well. Code works fine, tests all pass, literally no issues with import / export within the project. But when I build / publish to npm, and import to another project from npm package, typescript can't figure out how index.ts works.

You have to add a slash in the folder (module) name, this will treat index.ts as the folder index.
import {Engine} from '../engine/';

@coderbuzz first of all, thank you. You've saved my sanity.
Second of all, are there docs/is this common knowledge/where is this learned if you had not been here? I certainly didn't see it in module resolution docs.

There is no docs, I resolved it by trying any possibilities  ;)

@cbdeveloper did you find any resolution for this ? apart from importing from current directory

For me putting the import in proper order fixes this issue.
(import those which are not depend on other's first and then dependent's)


Travis CI build status and Maven version of master

Fixes

Missing Travis CI build status of master

seems great, thanks!


How to use title,subtitle, button-ok and button-cancel atributes

I tried
div class="item" ion-datetime-picker title='my title' ng-model='mymodel'
{{myDate | date: 'yyy-MM-dd'}}
/div
and doesn't work for the title and the other atributes.

The solucion was add the single quotes
div class="item" ion-datetime-picker title="'my title'" ng-model="'mymodel'"
{{myDate | date: 'yyy-MM-dd'}}
/div


refactor(NestQldbModule): Fix the forRootAsync static method.

:tada: This PR is included in version 1.0.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Automated cherry pick of #4658: fix: 避免同步安全组失败后,没有任何日志记录

Cherry pick of #4658 on release/2.10.0.
#4658: fix: 避免同步安全组失败后,没有任何日志记录

Codecov Report

:exclamation: No coverage uploaded for pull request base (release/2.10.0@c8dd7d0). Click here to learn what that means.
The diff coverage is n/a.

@@               Coverage Diff                @@
##             release/2.10.0   #4664   +/-   ##
================================================
  Coverage                  ?   7.46%
================================================
  Files                     ?     418
  Lines                     ?   66721
  Branches                  ?       0
================================================
  Hits                      ?    4982
  Misses                    ?   61312
  Partials                  ?     427

Flag
Coverage Δ

#aFlag
7.46% <ø> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c8dd7d0...784e265. Read the comment docs.

/lgtm

/approve


feat: resolve knowledge requirement of BKM

Closes #15

@philippfromme is there anything blocking us from merging this?


Automated cherry pick of #2965: feature: support xsky object storage

Cherry pick of #2965 on release/2.12.
#2965: feature: support xsky object storage

Codecov Report

:exclamation: No coverage uploaded for pull request base (release/2.12@fa3eaf8). Click here to learn what that means.
The diff coverage is n/a.

@@              Coverage Diff               @@
##             release/2.12   #2979   +/-   ##
==============================================
  Coverage                ?   7.06%
==============================================
  Files                   ?     477
  Lines                   ?   76392
  Branches                ?       0
==============================================
  Hits                    ?    5394
  Misses                  ?   70545
  Partials                ?     453

Flag
Coverage Δ

#aFlag
7.06% <ø> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fa3eaf8...7773153. Read the comment docs.

/approve
/lgtm


can't fold code in split screen mode

This happens every so often.

Hey @hawsome, I think that what you’re experiencing is similar to #7337. I am sorry this is causing you headaches: a fix for that bug has already landed on master and it will ship with 0.211.0 (which hasn’t been released yet). :rocket:
I am pretty confident you won’t notice that pesky bug anymore as soon as the new version will be released, but in the meantime you may want to build Atom from master and give it a shot (please, have a look at https://github.com/atom/atom/tree/master/docs/build-instructions to know how the building process works on your platform).
By the way, the new version shouldn’t take long to be released and you may want to just wait for it. :smile_cat:
Thanks for your feedback! :bow:

Hi @as-cii, thank you for your reply!
I know I experienced the bug you referenced, but this one seems like a slightly differentiated one. It doesn't fold it at all and jumps up.
Hopefully those are the same issues though, crossing my fingers!
Thank you guys for being so quick on bug fixes and updates! I literally see one almost every day. Props to everyone!

I think it also jumps if you click on the gutter to select the line. This drives me crazy as well... :smile:

@hawsome 0.211 was just released, containing the fix that @as-cii mentioned above. Can you check if this bug is still happening?

I wish I could!

Looking good! Thank you!!!!! :smile:


:bug: Crash encountered if password has expired

Problem Statement
Issue #249 states:

Describe the bug
If a password has expired, the CLI crashes with a vague null pointer exception after the username, password, and MFA token have already been accepted.
To Reproduce
Steps to reproduce the behavior:
1. Ensure your password has expired

2. Run `awscli sts get-caller-identity`

3. Input your Username, Password, and Token

4. See error

Expected behavior
Some form of description along with the stack trace that indicates the password has expired, perhaps similar to how #101 includes the java.lang.RuntimeException: You do not have access to AWS through Okta. bit.
Additional context
[matt@matt-dev ~]$ awscli sts get-caller-identity
Username<EMAIL_ADDRESS>Password:

GOOGLE Token Factor Authentication
Enter 'change factor' to use a different factor
Token:
xxxxxx
Exception in thread "main" java.lang.NullPointerException
        at com.okta.tools.authentication.OktaMFA.handleTimeoutsAndChanges(OktaMFA.java:92)
        at com.okta.tools.authentication.OktaMFA.promptForFactor(OktaMFA.java:69)
        at com.okta.tools.authentication.OktaAuthentication.getOktaSessionToken(OktaAuthentication.java:90)
        at com.okta.tools.saml.OktaSaml.getSamlResponse(OktaSaml.java:58)
        at com.okta.tools.OktaAwsCliAssumeRole.doRequest(OktaAwsCliAssumeRole.java:120)
        at com.okta.tools.OktaAwsCliAssumeRole.run(OktaAwsCliAssumeRole.java:90)
        at com.okta.tools.awscli.main(awscli.java:41)

Ran against release okta-aws-cli-1.0.6.jar
I realize that this is somewhat of a rare issue, but I typically only use the CLI when using Okta as I usually don't have a need to sign in to the web panel often. If this isn't something that can necessarily be resolved, then I totally understand. As always, feel free to let me know if I can provide any additional information. Thanks! :)

Solution

Present clear error when password is expired

Present password expiration warning if configured in Okta

Present clear error on unsupported/undocumented states

Resolves #249

@staticfox want to give this a try and review/approve the change, please 🙂

@staticfox bump.

@staticfox no worries. Thank you for the review 😄


Official jqplay instance mangles raw output

I tried to come up with something that can convert JSON into HTML. It works (as far as I'm concerned), but when I turn on the "Raw Output" it mangles the output and injects a Cloudflare script into the body.
Example: https://jqplay.org/s/tyODQglm30J
Output I expect:
<!DOCTYPE html>
<html lang="en">
    <head>
        head
    </head>
    <body>
        body
    </body>
</html>

Output I get (some tokens I replaced with [ snip ], just in case)
<!DOCTYPE html>
<html lang="en">
<head>
head
</head>
<body>
body
<script defer src="https://static.cloudflareinsights.com/beacon.min.js/[ snip ]" integrity="sha512-euoFGowhlaLqXsPWQ48qSkBSCFs3DPRyiwVu3FjR96cMPx+Fr+gpWRhIafcHwqwCqWS42RZhIudOvEI+Ckf6MA==" data-cf-beacon='{"rayId":"[ snip ]","r":1,"version":"2023.10.0","token":"[ snip ]"}' crossorigin="anonymous"></script>
</body>
</html>

Is this something to be concerned? I can't reproduce this on self-hosted version.

This is fixed in https://github.com/owenthereal/jqplay/issues/149.


Don't allow duplicate Versions in the API (fixes #1025)

I don't see us overriding the default isolation level in mysql so that might have an affect on the behavior you're seeing while trying to test this. Some info: http://blog.ionelmc.ro/2014/12/28/terrible-choices-mysql/#the-transaction-isolation-level
Also, you may have some tables in your local db that were created with syncdb as opposed to created from migrations. Syncdb might be creating non-innodb tables but hopefully not (I think django might try to create innodb tables by default).

r+wc
I think it's worth adding a test for the new query you added: addon.versions.filter(version=file_.version).exists()
We should also make sure caching isn't involved in any of these queries, especially since we know that we still have cache invalidation problems in prod. I think you can wrap both functions in @skip_cache or something.

I changed my settings_base.py to have the init_command and this works as you would expect. I don't know if it is safe to make that change or not. It seems like what we'd want though.
DATABASES['default']['OPTIONS'] = {
    'sql_mode': 'STRICT_ALL_TABLES',  # This was already being set.
    'init_command': 'SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED',
}

I made the API reject uploads earlier also rather than accepting it and then waiting for the validation to complete to not create the Version. It's still quite possible that you get multiple FileUploads with the same version string. That should be expected as if the validation fails we should still allow more uploads.
It looks like you need to set the init_command for each database so I'm assuming this won't get applied to databases other than default. I'm not sure how the databases are configured but since I know we have more than one this is likely going to get them out of sync in how they handle transactions.
Even without the transaction working the way we want this should help.


Does not support Node.js 8.x

using node 8.2.1, and npm is 5.3.0. got errors:
Node Sass could not find a binding for your current environment: Windows 64-bit with Node.js 5.x
Found bindings for the following environments:
  - Windows 64-bit with Node.js 8.x

npm rebuild node-sass

Have a look at this solution


Fix disabling, only store size of enabled inspector.

/cc @FigBug

[x] Fix "disabling inspector resizes document window" broken in #39
[x] Document onClose, include an example of how to use with unique_ptr (thanks @FigBug for the example usage!)
[x] Only store width/height of enabled inspector, store position separately
[x] Disable resizing of disabled inspector

Notes:

resized still gets called a too often (setBounds, setResizable, setResizeLimits), but it doesn't seem harmful
Sort of unexpected, but a manual setBounds obeys both setResizable and setResizeLimits so all of those calls have to be in a precise order.
Instead of disabling, could choose to make a corner resize attempt enable the inspector

Merged, but feel free to shout if you see any issues with window positioning.

Just tried it, seems to be ok so far


Missing instructions or an example for handling authentication

I'm missing an example in the docs on how to handle authentication with api2html. For example, it would be nice to have an example on how get api2html to pass an Authorization: Bearer <token> header to the backing REST API.

hi, @flrnmrz
that feature is not supported.
api2html is designed to render static websites by consuming open apis. it uses a cached-enabled http client, so consuming private endpoints could end in a catastrophic security breach
cheers


Add information Type method for Factory

I need this for docker info don't want to use type assertions because actually LinuxFactory should be private.

LGTM

LGTM


parallel processing

leverage scale up and out section

Implement at the make step, thus this is a matter of proper documentation.

drake is superseded by targets for a number of reasons described here. In that chapter, there is a mapping table to map drake functions to targets functions which will be very helpful to us.
The motivating reason for this migration is improve parallel efficiency of this package.

Meta-analysis running quickly will depend on future and on a launch method being set on the package globally.

Upon further thought, switching to reticulate is not ideal since that introduces a python dependency users need to manage. I will continue to try to figure out implementation with synapser.


Cannot show battle detail of Aerial exchange node in 1-6

This one is probably because the battle data in Aerial exchange node is different from normal battle node. @Javran Do you mind taking a look at it?

Yeah I guess you are right, I also suspect the battle data differs. Do you have a sample for the response?

ok, I'll get one for you after finishing the current sortie :blush:

svdata={"api_result":1,"api_result_msg":"\u6210\u529f","api_data":{"api_dock_id":1,"api_ship_ke":[-1,528,523,522,501,501,501],"api_ship_lv":[-1,1,1,1,1,1,1],"api_nowhps":[-1,37,31,31,32,32,44,96,70,60,20,20,20],"api_maxhps":[-1,37,31,31,32,32,48,96,70,60,20,20,20],"api_midnight_flag":1,"api_eSlot":[[520,517,524,-1,-1],[520,523,516,-1,-1],[505,506,525,525,-1],[501,-1,-1,-1,-1],[501,-1,-1,-1,-1],[501,-1,-1,-1,-1]],"api_eKyouka":[[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0]],"api_fParam":[[57,54,116,53],[73,93,59,52],[57,84,72,52],[59,89,59,59],[70,87,52,54],[67,88,70,69]],"api_eParam":[[25,0,50,80],[0,0,15,35],[58,42,30,60],[5,15,6,5],[5,15,6,5],[5,15,6,5]],"api_search":[2,1],"api_formation":[3,3,2],"api_stage_flag":[1,1,1],"api_kouku":{"api_plane_from":[[-1],[7,8]],"api_stage1":{"api_f_count":0,"api_f_lostcount":0,"api_e_count":168,"api_e_lostcount":6,"api_disp_seiku":4,"api_touch_plane":[-1,-1]},"api_stage2":{"api_f_count":0,"api_f_lostcount":0,"api_e_count":109,"api_e_lostcount":48},"api_stage3":{"api_frai_flag":[-1,0,0,0,0,0,1],"api_erai_flag":[-1,0,0,0,0,0,0],"api_fbak_flag":[-1,0,1,0,0,1,0],"api_ebak_flag":[-1,0,0,0,0,0,0],"api_fcl_flag":[-1,0,0,0,0,0,0],"api_ecl_flag":[-1,0,0,0,0,0,0],"api_fdam":[-1,0,0.1,0,0,0,0],"api_edam":[-1,0,0,0,0,0,0]}},"api_stage_flag2":[1,1,1],"api_kouku2":{"api_plane_from":[[-1],[7,8]],"api_stage1":{"api_f_count":0,"api_f_lostcount":0,"api_e_count":114,"api_e_lostcount":2,"api_disp_seiku":4,"api_touch_plane":[-1,-1]},"api_stage2":{"api_f_count":0,"api_f_lostcount":0,"api_e_count":61,"api_e_lostcount":22},"api_stage3":{"api_frai_flag":[-1,0,0,0,0,1,0],"api_erai_flag":[-1,0,0,0,0,0,0],"api_fbak_flag":[-1,0,0,0,0,1,1],"api_ebak_flag":[-1,0,0,0,0,0,0],"api_fcl_flag":[-1,0,0,0,0,0,0],"api_ecl_flag":[-1,0,0,0,0,0,0],"api_fdam":[-1,0,0,0,0,0,6],"api_edam":[-1,0,0,0,0,0,0]}}}}

Thanks for the data, it does look a little different than what I've expected, will be working on the fix today.

I think the problem is fixed, although I don't have ship to actually try 1-6.
Would you mind if I add your data to my lib for testing?

Okay, I'm testing it. will tell you soon :+1:

Yes, it works perfectly :+1: Now we only need to test for combined fleet.

Do you mean support expedition? I have never tried that, would be able to analyze that if I can see some sample responses. For now I think it shouldn't crash my lib but the damage might be incorrect.
And also would you mind if I add your responses to my lib for testcases?

I mean the combined fleet used in events. That means the battle will involve 18 ships (12 from ours and 6 from Abyssal) instead of only 12 ships (6 for each side) and the phases probably will be different as well.
And yes, feel free to use my data posted here :+1:

I see, thanks :)


chore: update bug report template

Pull Request Test Coverage Report for Build 624560644

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 18.359%

Totals

Change from base Build 623887901:
0.0%

Covered Lines:
2022

Relevant Lines:
11294

💛  - Coveralls


Web: Error: No 'main' method found.

Hello! I'm testing your lib and it's working fine for desktop (macOS). Now I'm trying for the web but I got this error:

Any ideas?
Dart SDK version: 3.3.4 (stable) (Tue Apr 16 19:56:12 2024 +0000) on "macos_arm64"

I got it. I'm using squadron_builder and I called:
❯ dart compile js lib/workers/theme_service_worker.web.g.dart -o web/theme_service_worker.dart.js


EXIF to .csv extractor

Description of the new feature / enhancement
Users have requested this,   a tool like Exiftool included in powertoys that can batch extract all this data from images and put them in a csv file.
Scenario when this would be used?
For example when we are needing to gather specific data its eaiser to extract
"I am uploading ~4000 photos to NZ Flood Pics. Rather than manually sorting through each image to pull Name, Date/Time, GPS for metadata, this tool should generate a spreadsheet with all EXIF data for selected images."
Supporting information
https://exiftool.org/

/dup #4564


[BUG] taux0 and tauy0 not filled unless periodicExternalForcing=.TRUE.

Wind stress is stored differently in the model if it is constant versus if periodicExternalForcing=.TRUE.,
If False, the wind stress is read directly into fu, and fv:
https://github.com/MITgcm/MITgcm/blob/c26ad343d2c0195fac1c91f8adac25ba147e8b74/model/src/ini_forcing.F#L67-L78
If True, it gets stored in taux0 and tauy0, from external_fields_load.F.
This tripped me up because I needed the wind stress to calculate the piston velocity for air/sea gas exchange, and could not figure out why there was no air/sea bass exchange when using periodicExternalForcing=.FALSE.,
Proposed solution would be to populate taux/y0 from the ini file and then propagate to fu/v

If I need to access the wind-stress, I would use "fu/fv" which works:

if using constant forcing (periodicExternalForcing=.FALSE.,) without pkg/exf ;
if using periodic forcing (periodicExternalForcing=.TRUE.,) without pkg/exf ;
if using pkf/exf ('useEXF=.TRUE.`) with time-dependent or constant forcing.

Now this assumes I need to access the wind-stress after the call to S/R LOAD_FIELDS_DRIVER, which is (very) early in the time-stepping sequence (near the top of forward_step.F), so not generally an issue. Or may-be I am missing something ?

Ah, OK, so I didn't realize fu/fv were the wind stress - I assumed given the generic names that they were generic forcing in x and y, but looking at FFIELDS.h I see that they are indeed the wind stress.  I didn't think to look there, and I think perhaps I was copying code from the geochemistry module (trying to get O2 pumping).  Probably fine to close this, though I still find it confusing to have taux0 and fu mean the same thing (almost).  Thanks,

The variable names "fu/fv" are not the best, I agree.
And just to clarify: I did not list (in the 3 cases above) other ways to get the wind-stress (using pkg/bulk_force or when coupled to an atmosphere) but these would also behave like case (3).
And in case (2), taux0 is lagging behind the current model time since it's only updated every externForcingPeriod ; or an other way to phrase this is that it does not account for the (linear) time interpolation between taux0 and taux1.
I suggest we close this issue in the coming days.

As a mnemonic I always think of:
fu = surface Flux of U momentum
fv = surface Flux of V momentum
D.

On Feb 17, 2023, at 11:56 AM, Jean-Michel Campin @.***> wrote:
The variable names "fu/fv" are not the best, I agree.


b2: cleanup unfinished large files

What is the purpose of this change?

The cleanup command will delete unfinished large file uploads that
were started more than a day ago (to avoid deleting uploads that are
potentially still in progress).
Was the change discussed in an issue or in the forum before?

Fixes #2617
Checklist

[x] I have read the contribution guidelines.
[ ] I have added tests for all changes in this PR if appropriate.
[x] I have added documentation for the changes if appropriate.

[x] I'm done, this Pull Request is ready for review :-)

That looks very nice - thank you :-)
Looking for the start markers fits in very nicely with the rclone cleanup command.
Will merge in a moment.


Small improvements to _init_debugging_flags

What does this PR do?
Various small improvements:

Mark _configure_progress_bar and _determine_data_use_amount as protected
Add typehints to _init_debugging_flags
Simplify fast_dev_run logic

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

[x] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[ ] Did you make sure your PR does only one thing, instead of bundling different changes together?
[x] Did you make sure to update the documentation with your changes? (if necessary)
[x] Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
[x] Did you list all the breaking changes introduced by this pull request?
[x] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

[ ] Is this pull request ready for review? (if not, please submit in draft mode)
[ ] Check that all items from Before submitting are resolved
[ ] Make sure the title is self-explanatory and the description concisely explains the PR
[ ] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?
Make sure you had fun coding 🙃

Can anyone help me fix these mypy errors? I moved the data_connector and logger_connector on_trainer_init code into trainer.py, but the errors persist, and I am having trouble figure it out.

I tried to fix them by moving the data_connector and logger_connector on_trainer_init code into trainer.py

I'm not sure about the cause but that change should not be part of this PR
I suggest you re-do your original changes bit by bit and run mypy after every step to find out which causes it.

The solution was to remove type hints from _init_debugging_flags. Which was one of the main reasons I opened this PR :/ I don't understand what the problem is - anyone know a workaround or should I just merge without that?

You can merge, there is a lot of code to be typed, no point in getting blocked on this signature.


Update actors.py

Added Device HmIP-BROLL-2
Please point your pull request at the devel branch. Also provide some information about the changes.
This pull request:

fixes issue: <Link to issue>
adds support for HomeMatic device: <Device name goes here, e.g. HmIP-ABC-DEF>

New class: <e.g. YourSensor>
Home Assistant platform: <e.g. DISCOVER_LIGHTS>

does the following: <Description of what the change is intended to do>

This library is not being maintained anymore. Please switch to the new integration.


implement as transform

I took the original code and implemented it as browserify transform. This way we don't need to write temporary files to disk and we can leverage @cypress/browserify-preprocessor's handling of watchers and reloading. (ref #1)
I've also made the preprocessor accept options to be passed down to browserify.
Note: I've used the airbnb style simply because that's what I usually have set up. If you prefer a different style, let me know and I'll make the changes (maybe add an .eslintrc to the repo?)

I have a question about this. My understanding is that @cypress/browserify-preprocessor already takes care of watching. When trying to re-implement this, I tried omitting watching completely and I am observing identical behavior with my fork. Thus, why is chokidar necessary here?

Are your tests restarting when either .feature file or your step definitions change? Or only when your .feature file changes? I think that was the point here. Maybe it's started behaving differently comparing to 2018?

Are your tests restarting when either .feature file or your step definitions change?

Both!

Maybe it's started behaving differently comparing to 2018?

I thought that could be it, but they had watchify in v1.02 as well, seen here.


fix base item values - clay

this is to update  the values of clay and related crafting items

this requires adding flowers and dyes, both of which are mostly missing. #24

github is still having issues...
the above comment is due to terracotta requiring dyes

more number fudging here...
to make the dyed terracottas i am going to add 1.00 to the terracotta value, as most of the dyes cost 8.00 (2 x 4.00 items crafted) and the recipe for dyed terracotta is 8 terracotta and 1 dye, this works out to an even +1.  sometimes less complex is more.

that made the dyed terracotta 3.50 and the glazed 4.00 (after adding "coal / 8")


test

Note: Confirm the contribution guidelines before submitting a pull request.
Click the Preview tab and select a PR template:

Standard change
Small change

Do NOT send a PR with this description.

/cherry-pick PR_target

/cherry-pick Owen-Liuyuxuan/autoware:main

/cherry-pick Owen-Liuyuxuan/autoware:main

/cherry-pick main


fix: delete out_dir param

What type of PR is this?

fix
What this PR does / why we need it (en: English/zh: Chinese):

en: delete out_dir param
zh: 删除 cwgo server 和 client 的 out_dir 命令，使 hz 和 kitex 对齐
Which issue(s) this PR fixes:

解一下冲突 @chaoranz758

解一下冲突 @chaoranz758

done


Kestrel; "launchUrl" within launchSettings.json should be "applicationUrl"

By default, a freshly deployed 3.14.3 Serene project has "launchUrl" within the Kestrel section of launchSettings.json.
When changing the URL part within "launchUrl", only the browser changes its port but the Kestrel webserver does not.
Before:

Replacing "launchUrl" with "applicationUrl" solves this problem.
After:

With kind regards,
John

@VictorTomaili can you please check if that's the case?

launchUrl: 5000
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: https://localhost:5001
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://localhost:5000

Browser opens localhost:5001

launchUrl: 5080
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: https://localhost:5001
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://localhost:5000

Browser opens localhost:5080 - page cannot be loaded

applicationUrl: 5000
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://localhost:5000

Browser opens localhost:5000

applicationUrl: 5080
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://localhost:5080

Browser opens localhost:5080

applicationUrl: 5080
launchUrl: 5081
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://localhost:5080

Browser opens localhost:5081 - page cannot be loaded
@volkanceylan also start from commandline working as expected with applicationUrl

Ok then change it to applicationUrl


Enforce uniique SSNs and ITINs across seeds

Title: Use seed-specific chunks of artifact data when assigning IDs
Description

Category: implementation
JIRA issue: MIC-3855
Research reference: na

Changes and notes
Currently we have SSN and ITIN duplication when each simulation seed is
processed in parallel because they all pick off the top of the artifact data.
This PR assigns unique chunks of the artifact data to each seed to
then choose from
Verification and Testing
added pytest. Ran small make_results to make sure files are still output.

Oh, big note: currently this make_results returns very different processed data if you happen to run all seeds without jobmon, eg make_results (note no -j or -s arguments). In this case - which is what we've all gotten used to - the final processed results will be one large dataset per observer. In any other situation, you will get seed-specific datasets in their respective observer folder.

make_results ... --> one large dataset per observer
make_results -s <seed> --> one seed-specific dataset per observer, eg /_.csv.bz2
make_results -j --> all seed-specific datasets in their respective observer folders, same format as above
make_results -j -s <seed> --> one seed-specific dataset per observer for , same format as above


docker image create error :: AttributeError: module 'enum' has no attribute 'IntFlag'

Hi Team. I am using the Dockerfile listed to create a custom docker image but it fails with below error:
AttributeError: module 'enum' has no attribute 'IntFlag'
Docker image create output below
log:
(base) mymac:kubeflow-kale p.panda$ docker build -t kubeflow-kale .
Sending build context to Docker daemon  6.144kB
Step 1/12 : ARG IMAGE_TYPE="cpu"
Step 2/12 : FROM gcr.io/kubeflow-images-public/tensorflow-1.14.0-notebook-${IMAGE_TYPE}:v0.7.0
---> cd3b7b30dbd2
Step 3/12 : USER root
---> Using cache
---> 3f2df44cbf44
Step 4/12 : RUN apt-get update &&     apt-get install -y --no-install-recommends         ca-certificates bash-completion tar less         python-pip python-setuptools build-essential python-dev         python3-pip python3-wheel &&     rm -rf /var/lib/apt/lists/
---> Using cache
---> 01d3e144ee72
Step 5/12 : ENV SHELL /bin/bash
---> Using cache
---> e7b1cf5c115b
Step 6/12 : COPY bashrc /etc/bash.bashrc
---> Using cache
---> 9efe90da2770
Step 7/12 : RUN echo "set background=dark" >> /etc/vim/vimrc.local
---> Using cache
---> 5ffcc9ff10d2
Step 8/12 : RUN pip3 install --upgrade pip &&     pip3 install --upgrade "jupyterlab>=2.0.0,<3.0.0" &&     pip3 install https://storage.googleapis.com/ml-pipeline/release/latest/kfp.tar.gz --upgrade &&     pip3 install -U kubeflow-kale &&     jupyter labextension install kubeflow-kale-labextension
---> Running in 08d420206b81
Collecting pip
Downloading https://files.pythonhosted.org/packages/54/eb/4a3642e971f404d69d4f6fa3885559d67562801b99d7592487f1ecc4e017/pip-20.3.3-py2.py3-none-any.whl (1.5MB)
Installing collected packages: pip
Found existing installation: pip 19.1.1
Uninstalling pip-19.1.1:
Successfully uninstalled pip-19.1.1
Successfully installed pip-20.3.3
Collecting jupyterlab<3.0.0,>=2.0.0
Downloading jupyterlab-2.2.9-py3-none-any.whl (7.9 MB)
Requirement already satisfied: jinja2>=2.10 in /usr/local/lib/python3.6/dist-packages (from jupyterlab<3.0.0,>=2.0.0) (2.10.1)
Requirement already satisfied: tornado!=6.0.0,!=6.0.1,!=6.0.2 in /usr/local/lib/python3.6/dist-packages (from jupyterlab<3.0.0,>=2.0.0) (5.1.1)
Requirement already satisfied: notebook>=4.3.1 in /usr/local/lib/python3.6/dist-packages (from jupyterlab<3.0.0,>=2.0.0) (5.7.8)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from jinja2>=2.10->jupyterlab<3.0.0,>=2.0.0) (1.1.1)
Collecting jupyterlab-server<2.0,>=1.1.5
Downloading jupyterlab_server-1.2.0-py3-none-any.whl (29 kB)
Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from jupyterlab-server<2.0,>=1.1.5->jupyterlab<3.0.0,>=2.0.0) (2.22.0)
Requirement already satisfied: jsonschema>=3.0.1 in /usr/local/lib/python3.6/dist-packages (from jupyterlab-server<2.0,>=1.1.5->jupyterlab<3.0.0,>=2.0.0) (3.0.1)
Requirement already satisfied: json5 in /usr/local/lib/python3.6/dist-packages (from jupyterlab-server<2.0,>=1.1.5->jupyterlab<3.0.0,>=2.0.0) (0.8.5)
Requirement already satisfied: pyrsistent>=0.14.0 in /usr/local/lib/python3.6/dist-packages (from jsonschema>=3.0.1->jupyterlab-server<2.0,>=1.1.5->jupyterlab<3.0.0,>=2.0.0) (0.15.2)
Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from jsonschema>=3.0.1->jupyterlab-server<2.0,>=1.1.5->jupyterlab<3.0.0,>=2.0.0) (41.0.1)
Requirement already satisfied: six>=1.11.0 in /usr/lib/python3/dist-packages (from jsonschema>=3.0.1->jupyterlab-server<2.0,>=1.1.5->jupyterlab<3.0.0,>=2.0.0) (1.11.0)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.6/dist-packages (from jsonschema>=3.0.1->jupyterlab-server<2.0,>=1.1.5->jupyterlab<3.0.0,>=2.0.0) (19.1.0)
Requirement already satisfied: nbformat in /usr/local/lib/python3.6/dist-packages (from notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (4.4.0)
Requirement already satisfied: jupyter-client>=5.2.0 in /usr/local/lib/python3.6/dist-packages (from notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (5.2.4)
Requirement already satisfied: nbconvert in /usr/local/lib/python3.6/dist-packages (from notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (5.5.0)
Requirement already satisfied: terminado>=0.8.1 in /usr/local/lib/python3.6/dist-packages (from notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (0.8.2)
Requirement already satisfied: prometheus-client in /usr/local/lib/python3.6/dist-packages (from notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (0.7.1)
Requirement already satisfied: traitlets>=4.2.1 in /usr/local/lib/python3.6/dist-packages (from notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (4.3.2)
Requirement already satisfied: ipykernel in /usr/local/lib/python3.6/dist-packages (from notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (5.1.1)
Requirement already satisfied: ipython-genutils in /usr/local/lib/python3.6/dist-packages (from notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (0.2.0)
Requirement already satisfied: jupyter-core>=4.4.0 in /usr/local/lib/python3.6/dist-packages (from notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (4.5.0)
Requirement already satisfied: Send2Trash in /usr/local/lib/python3.6/dist-packages (from notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (1.5.0)
Requirement already satisfied: pyzmq>=17 in /usr/local/lib/python3.6/dist-packages (from notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (18.0.1)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from jupyter-client>=5.2.0->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (2.8.0)
Requirement already satisfied: ptyprocess in /usr/local/lib/python3.6/dist-packages (from terminado>=0.8.1->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (0.6.0)
Requirement already satisfied: decorator in /usr/local/lib/python3.6/dist-packages (from traitlets>=4.2.1->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (4.4.0)
Requirement already satisfied: ipython>=5.0.0 in /usr/local/lib/python3.6/dist-packages (from ipykernel->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (7.5.0)
Requirement already satisfied: pickleshare in /usr/local/lib/python3.6/dist-packages (from ipython>=5.0.0->ipykernel->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (0.7.5)
Requirement already satisfied: pygments in /usr/local/lib/python3.6/dist-packages (from ipython>=5.0.0->ipykernel->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (2.4.2)
Requirement already satisfied: pexpect in /usr/local/lib/python3.6/dist-packages (from ipython>=5.0.0->ipykernel->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (4.7.0)
Requirement already satisfied: backcall in /usr/local/lib/python3.6/dist-packages (from ipython>=5.0.0->ipykernel->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (0.1.0)
Requirement already satisfied: jedi>=0.10 in /usr/local/lib/python3.6/dist-packages (from ipython>=5.0.0->ipykernel->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (0.13.3)
Requirement already satisfied: prompt-toolkit<2.1.0,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from ipython>=5.0.0->ipykernel->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (2.0.9)
Requirement already satisfied: parso>=0.3.0 in /usr/local/lib/python3.6/dist-packages (from jedi>=0.10->ipython>=5.0.0->ipykernel->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (0.4.0)
Requirement already satisfied: wcwidth in /usr/local/lib/python3.6/dist-packages (from prompt-toolkit<2.1.0,>=2.0.0->ipython>=5.0.0->ipykernel->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (0.1.7)
Requirement already satisfied: entrypoints>=0.2.2 in /usr/local/lib/python3.6/dist-packages (from nbconvert->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (0.3)
Requirement already satisfied: mistune>=0.8.1 in /usr/local/lib/python3.6/dist-packages (from nbconvert->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (0.8.4)
Requirement already satisfied: testpath in /usr/local/lib/python3.6/dist-packages (from nbconvert->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (0.4.2)
Requirement already satisfied: defusedxml in /usr/local/lib/python3.6/dist-packages (from nbconvert->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (0.6.0)
Requirement already satisfied: pandocfilters>=1.4.1 in /usr/local/lib/python3.6/dist-packages (from nbconvert->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (1.4.2)
Requirement already satisfied: bleach in /usr/local/lib/python3.6/dist-packages (from nbconvert->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (3.1.0)
Requirement already satisfied: webencodings in /usr/local/lib/python3.6/dist-packages (from bleach->nbconvert->notebook>=4.3.1->jupyterlab<3.0.0,>=2.0.0) (0.5.1)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/lib/python3/dist-packages (from requests->jupyterlab-server<2.0,>=1.1.5->jupyterlab<3.0.0,>=2.0.0) (2.6)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->jupyterlab-server<2.0,>=1.1.5->jupyterlab<3.0.0,>=2.0.0) (1.24.3)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->jupyterlab-server<2.0,>=1.1.5->jupyterlab<3.0.0,>=2.0.0) (2019.9.11)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->jupyterlab-server<2.0,>=1.1.5->jupyterlab<3.0.0,>=2.0.0) (3.0.4)
Installing collected packages: jupyterlab-server, jupyterlab
Attempting uninstall: jupyterlab-server
Found existing installation: jupyterlab-server 1.0.6
Uninstalling jupyterlab-server-1.0.6:
Successfully uninstalled jupyterlab-server-1.0.6
Attempting uninstall: jupyterlab
Found existing installation: jupyterlab 1.1.4
Uninstalling jupyterlab-1.1.4:
Successfully uninstalled jupyterlab-1.1.4
Successfully installed jupyterlab-2.2.9 jupyterlab-server-1.2.0
Collecting https://storage.googleapis.com/ml-pipeline/release/latest/kfp.tar.gz
Downloading https://storage.googleapis.com/ml-pipeline/release/latest/kfp.tar.gz (165 kB)
Requirement already satisfied: PyYAML in /usr/local/lib/python3.6/dist-packages (from kfp==1.2.0) (5.1.2)
Requirement already satisfied: google-cloud-storage>=1.13.0 in /usr/local/lib/python3.6/dist-packages (from kfp==1.2.0) (1.20.0)
Requirement already satisfied: kubernetes<12.0.0,>=8.0.0 in /usr/local/lib/python3.6/dist-packages (from kfp==1.2.0) (10.0.1)
Requirement already satisfied: google-auth>=1.6.1 in /usr/local/lib/python3.6/dist-packages (from kfp==1.2.0) (1.6.3)
Requirement already satisfied: requests_toolbelt>=0.8.0 in /usr/local/lib/python3.6/dist-packages (from kfp==1.2.0) (0.9.1)
Requirement already satisfied: cloudpickle in /usr/local/lib/python3.6/dist-packages (from kfp==1.2.0) (1.2.2)
Requirement already satisfied: jsonschema>=3.0.1 in /usr/local/lib/python3.6/dist-packages (from kfp==1.2.0) (3.0.1)
Requirement already satisfied: tabulate in /usr/local/lib/python3.6/dist-packages (from kfp==1.2.0) (0.8.3)
Requirement already satisfied: click in /usr/local/lib/python3.6/dist-packages (from kfp==1.2.0) (7.0)
Collecting docstring-parser>=0.7.3
Downloading docstring_parser-0.7.3.tar.gz (13 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
_ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-_lzndjix/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.8.0' wheel
cwd: None
Complete output (14 lines):
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.6/dist-packages/pip/main.py", line 23, in
from pip._internal.cli.main import main as _main  # isort:skip # noqa
File "/usr/local/lib/python3.6/dist-packages/pip/internal/cli/main.py", line 5, in
import locale
File "/usr/lib/python3.6/locale.py", line 16, in
import re
File "/usr/lib/python3.6/re.py", line 142, in
class RegexFlag(enum.IntFlag):
AttributeError: module 'enum' has no attribute 'IntFlag'*
ERROR: Command errored out with exit status 1: /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-_lzndjix/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.8.0' wheel Check the logs for full command output.
The command '/bin/bash -c pip3 install --upgrade pip &&     pip3 install --upgrade "jupyterlab>=2.0.0,<3.0.0" &&     pip3 install https://storage.googleapis.com/ml-pipeline/release/latest/kfp.tar.gz --upgrade &&     pip3 install -U kubeflow-kale &&     jupyter labextension install kubeflow-kale-labextension' returned a non-zero code: 1

This issue seems to be avoided by adding below to the Dockerfile
pip3 install "enum34==1.1.8"

This issue seems to be avoided by adding below to the Dockerfile
pip3 install "enum34==1.1.8"

I can build the image using the 0.5.1 version of kubeflow-kale

I can build the image using the 0.5.1 version of kubeflow-kale

@nachogarciam yeah. please check issue #275. i have created a bug for that.

@nachogarciam yeah. please check issue #275. i have created a bug for that.

@pratyushpanda  This issue should be resolved by the referenced commit

@pratyushpanda  This issue should be resolved by the referenced commit


ResponseStrings.uk-UA.json

Translation from English to Ukrainian document ResponseStrings.en-US.json
Country: Ukraine
Language: Ukrainian

Do you have access to NadekoBot strings on POEditor?

no

I'm sorry I can not use POEditor

How to access NadekoBot strings on POEditor?

Join [this server](https://discord.gg/nadekobot] and send a DM to Kwoth#2560, he will send you instructions.

Thank you


Bump go dependencies

Changes

Submitter Checklist
As the author of this PR, please check off the items in this checklist:

[ ] Has Docs included if any changes are user facing
[ ] Has Tests included if any functionality added or changed
[ ] Follows the commit message standard
[ ] Meets the Tekton contributor standards (including
functionality, content, code)
[ ] Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
[ ] Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes
NONE

Let's merge this. The lint failure is due to the diff being so large. This is mostly a dependabot commit.


Add a "dummy" fedimint module to the fedimint repo

The purpose is to have:

template for other people to copy&paste&modify
comments to help new developers get a sense of what is what

By having it alongside real modules we will know it's always up to date.

I think it should be there but not wired up in the default server/client binary. There could be an example binary that attaches it to the federation for CI and example purposes. This of course all assumes finished modularization.

Yeap. We only want it to compile, and really use it.

Closed by https://github.com/fedimint/fedimint/pull/1195


CON-523: Few outstanding issues with Dokka API documentation

Do you know why these particular links are not working? Is this discussed somewhere already?

The docsite has the same problems right now, and since the fix is to manually link to the docsite URL, can you also cherry-pick these changes onto release/1.3 so that the docsite is also updated. Thanks.

The docsite has the same problems right now, and since the fix is to manually link to the docsite URL, can you also cherry-pick these changes onto release/1.3 so that the docsite is also updated. Thanks.

I will do that.

Do you know why these particular links are not working? Is this discussed somewhere already?

I haven't figured out the root cause for this. I will document it in another ticket.


Buttons: Missing a "hold down" action.

Should repeatedly fire events if the mouse is held down on the button.

Hey @davesmith00000, I can pick this up if you don't think it's too complex.
At a quick glance, it looks like we could implement this by updating Button.update to no longer require a mouse press event to fire mousedown events and instead keep firing them as long as the button state is "down". But I think this might be a simplistic take, and that what you want is a new onHoldDown member to differentiate between events to fire when a button is clicked and when the user keeps the button pressed. Am I right?
Should these events fire on every frame tick, and should there be a threshold before we start firing them? If so, should these things be configurable?

Hello, feel free!
The UI components need love in general, it's one of those areas I always plan to revisit when I have a concrete use case but rarely do.
I must warn you that the UI components can be deceptively fiddly! There should be some tests in place for them though. 🙂
I did mean the latter yes, a new onHoldDown action, and a configurable time delay might be nice but that means hold a time based bit of state which might be a pain. Also firing an event every frame means having access to the FrameTick event and I'm not sure it does currently?
Have a go if you like, see what you think. It sounds easy but might yet turn out to be difficult.
Dave

Also firing an event every frame means having access to the FrameTick event and I'm not sure it does currently?

This really isn't my area of expertise so I might have used the wrong terminology, sorry. I was asking whether we should fire the events on every call to Button.update or every N calls but it sounds like perhaps that should be left to whoever's developing the game. Same thing for the time delay before we start firing events.

I must warn you that the UI components can be deceptively fiddly!

Noted :-)

I think the expected behaviour would be an action that is fired every frame that the button is being held down on. Which I'm pretty sure is what you were saying too. The catch is that means getting one update trigger per frame, and that would normally come from the FrameTick event. I haven't looked - it might be that the button has access to that already - I'm just not sure.


Incorrect bounds check in fallback code

This bounds check is wrong:
https://github.com/RoDmitry/atoi_simd/blob/293e4e89021959c2f42e97d496e3b8c24b8c0611/src/fallback.rs#L145
It should be
 let mut val: u64 = unsafe { read_unaligned(s.get_safe_unchecked(8..16).as_ptr().cast()) };

Applying this change causes a number of tests to fail.
This issue can also be observed using Miri, if I run cargo +nightly miri test without the .cargo/config in this repo, I can get at least these two UB reports:
196 |             let val: u64 = unsafe { read_unaligned(s.get_safe_unchecked(8..).as_ptr().cast()) };
    |                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ using uninitialized data, but this operation requires initialized memory
    |

   --> src/fallback.rs:145:41
    |
145 |             let mut val: u64 = unsafe { read_unaligned(s.get_safe_unchecked(8..).as_ptr().cast()) };
    |                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ memory access failed: alloc29045 has size 9, so pointer to 8 bytes starting at offset 8 is out-of-bounds

Hi. It should not be 8..16. There is no problem with getting slice of s. You might not understand the concept of how this works.
For example we have code:
let s = "123";
let val: u64 = unsafe { read_unaligned(s.as_ptr().cast()) };

val at that point might be something like 0xAF3534_333231, that's why it warns us that it might be UB, because it has read extra memory, that we don't own, but on the next line we have:
let len = check_8(val).min(s.len());

which makes sure we have the correct len of our data.

let s = "123";
let val: u64 = unsafe { read_unaligned(s.as_ptr().cast()) };

This program is UB. It contains a buffer overflow.

To elaborate on what @saethlin said: it does not matter if data read from out of bounds is never used; the very act of reading from out of bounds is already a problem.
Languages like C or Rust define what is legal and illegal to do within them. Reading out of bounds is always illegal, and since it is illegal, the compiler assumes it will never happen. Such illegal operations are (somewhat confusingly) called "undefined behavior"; you can learn more about the concept here. In C and Rust any operation that reads out of bounds is undefined behavior, and should never be performed. Programs that read out of bounds are a typical cause of security vulnerabilities; they are #7 on the CWE list of most dangerous software vulnerabilities.
An out-of-bounds read would be legal in assembly, if and only if you also took precautions to never cross the page boundary, but using an unaligned read makes it illegal even in pure assembly.
That is to say, this is a genuine problem that needs to be fixed.

I have read about it, and I still don't understand why it is illegal. The only thing I have found is that it has access to that extra memory, but it does not write there, just reads and then drops it. @Shnatsel, you definitely has more experience than me about it. Can you give me examples of what bad can happen, and how dangerous it can be?

The best case for your current code is that it is lowered into assembly exactly as written, but when the buffer happens to be near the page boundary so that your read crosses it, your program is killed by the OS due to the illegal memory access.
The worst-case scenario for an out-of-bounds read is this: https://en.wikipedia.org/wiki/Heartbleed
Once you have performed an operation that the language explicitly specifies must never be performed, literally anything can happen. It may work for some inputs but not others; or it may work on some compiler version or not others, or with certain compiler flags or not others, or allow an attacker that provides a carefully crafted string for your code to parse execute arbitrary code on your computer. Absolutely anything can happen because you have broken the basic rules of reality.

Perhaps a more accessible explanation of undefined behavior can be found here: https://www.ralfj.de/blog/2019/07/14/uninit.html
atoi_simd also performs reads from uninitialized memory, so this is going to be relevant here as well.

I think about cloning data in an Array with bigger capacity, then the bounds will be right. But I think the speed will be worse. Have to benchmark it.

u64::from_ne_bytes lowers into a load. Here is proof on godbolt: https://godbolt.org/z/a5GPfs61s The function is just a bounds check and a load. (I have written a very detailed article on avoiding bounds checks that also covers inspecting the generated assembly, it might come in handy). However, u64::from_ne_bytes requires the input slice to be 8 bytes in length.
There is no way to read from a variable-length slice that may be under 8 bytes in length to a u64 efficiently and without UB. This is why when the compiler processes a slice in a loop with SIMD instructions, it has to insert special handling of the end that may be shorter than SIMD allows, where it reads byte-by-byte.

I think about cloning the data in an Array with bigger capacity, then the bounds will be right. But I think the speed will be worse. Have to benchmark it.

If you do this on a variable-length input, it will lower to a memcpy() call: https://godbolt.org/z/1P5hbr6jP
memcpy() is usually optimized for large inputs and will not be great for this use case. You could try writing a manual loop that copies byte-by-byte and see if that helps at all; I don't know if it will, but at least it's worth a shot.

Is this also UB for s = b"123"?

Yes. b"123" is 3 bytes (24 bits) long, while the SIMD load performs a 16-byte (128-bit) load and goes out of bounds. The original blog post gets away with this only because in their case the input is always 16 characters long.
You could still make use of SIMD if you pad the input with zeroes up to 16 characters, turning b"123" into b"000000000123". You can achieve this by making a [b'0'; 16] and copying the actual input to the end, but the copy still has to be done byte-by-byte. However, a 16 characters long buffer is not going to be enough for many numeric types. For example, the maximum value representable by a u64 is<PHONE_NUMBER>3709551615 which is 21 characters long. And u128 maximum value is 40 characters long. So you might have to use bigger buffers than 16 if you want to keep using SIMD.

For the record, at some point, I did use an array, and it was slower. Will test it again

I have made separate functions read_8 and read_16, and have tested this:
fn read_8(s: &[u8]) -> u64 {
    let mut buf = [0u8; 8];
    let len = s.len().min(8);
    buf[..len].copy_from_slice(&s[..len]);
    u64::from_ne_bytes(buf)
}

And also this:
fn read_8(s: &[u8]) -> u64 {
    if s.len() < 8 {
        let mut buf = [0u8; 8];
        buf[..s.len()].copy_from_slice(s);

        return u64::from_ne_bytes(buf);
    }

    unsafe { read_unaligned(s.as_ptr().cast()) }
}

It's more than 2x slower. (In case anybody wants, you can test it with cargo bench -- "parse u64/9", without the .cargo/config)
And Miri is happy now, but not me..

I agree that this performance degradation is frustrating.
You might want to check if a manual loop is faster than a copy_from_slice which is a memcpy() behind the scenes. memcpy() is usually optimized for big copies, not small ones, so a manual loop over a number of elements known to be small might be faster.

You are right. This:
fn read_8(s: &[u8]) -> u64 {
    if s.len() < 8 {
        let mut buf = [0u8; 8];
        for (i, ch) in s.into_iter().enumerate() {
            buf[i] = *ch;
        }

        return u64::from_ne_bytes(buf);
    }

    unsafe { read_unaligned(s.as_ptr().cast()) }
}

is faster than previous. But resulting assembly somehow looks like shit) And yet it's faster. I did not know that memcpy() is that slow for small datasets.

These two variants are also worth benchmarking:
fn read_8(s: &[u8]) -> u64 {
    if s.len() < 8 {
        let mut buf = [0u8; 8];
        for (b, ch) in buf.iter_mut().zip(s.into_iter()) {
            *b = *ch;
        }

        return u64::from_ne_bytes(buf);
    }

    unsafe { read_unaligned(s.as_ptr().cast()) }
}

(although you don't even need this unsafe because u64::from_ne_bytes(s[..8]) should produce the same assembly)
and

fn read_8(s: &[u8]) -> u64 {
    let mut buf = [0u8; 8];
    for (b, ch) in buf.iter_mut().zip(s.into_iter()) {
        *b = *ch;
    }

    return u64::from_ne_bytes(buf);
    }
}

First is as slow as copy_from_slice.
Without unsafe is even slower.

That makes me suspect something is wrong with the benchmarking setup, because the safe and unsafe variant literally produce the exact same assembly.
Unsafe: https://godbolt.org/z/M4h5bT476
Safe: https://godbolt.org/z/cK6zWfvK4

For a start, make sure you use std::hint::black_box on both the input and output in your benchmarking harness, and that the results are consistent between benchmarking runs.

Maybe I misunderstood you. Second time I ran this:
fn read_8(s: &[u8]) -> u64 {
    let mut buf = [0u8; 8];
    for (b, ch) in buf.iter_mut().zip(s.into_iter()) {
        *b = *ch;
    }
    u64::from_ne_bytes(buf)
}

Ah, I see. I thought you applied my suggestion to get rid of unsafe, and that slowed it down.
In that case the fastest option is this: https://godbolt.org/z/cK6zWfvK4
Although the recommendation to use black_box in benchmarks on both the input and output still stands. Otherwise the measurements may not be representative of the real world performance due to the compiler precomputing some of the results.

You can see my benchmarks here. As far as I understood, Criterion does not need extra black_box.
For the u64::from_ne_bytes(s[..8].try_into().unwrap()), yes it's the same speed, but crate edition is 2018, and this edition does not support try_into. If it has same assembly, then there is no difference, as I understand.

It supports it full well, you just have to import it. Just use std::convert::TryInto and it works.

Right, my bad

Criterion does not black-box inputs or outputs automatically (even though it arguably should, that would be much more intuitive).
Proof: https://docs.rs/criterion/latest/src/criterion/benchmark_group.rs.html#259-271 - no mention of black_box anywhere in the file.
In general you have to black_box both the input and output of the operation you are measuring for the measurement to be representative of real-world workloads. But maybe you got (un)lucky this time and the compiler did not constant-fold much.

It does: link

Ah, I stand corrected! That's great news.
Although it only does it for the outputs, not for inputs. Perhaps I should make a PR for that, but that's outside the scope of this issue.

Hmm, actually, I don't think the inputs can be automatically black-boxed with that API, so that still has to be done manually. But that is off-topic for this issue, we can make a new one if this needs to be discussed further.

The input is passed with .bench_with_input(). It must be doing something, if it's there)

@orlp came up with this, it should be quite fast at reading a u64 without undefined behavior:
pub fn load_padded_le_u64(bytes: &[u8]) -> u64 {
    let len = bytes.len();
    if len >= 8 {
        return u64::from_le_bytes(bytes[0..8].try_into().unwrap());
    }

    if len >= 4 {
        let lo = u32::from_le_bytes(bytes[0..4].try_into().unwrap());
        let hi = u32::from_le_bytes(bytes[len-4..len].try_into().unwrap());
        return (lo as u64) | ((hi as u64) << 8*(len - 4));
    }

    if len == 0 {
        return 0;
    }

    let lo = bytes[0] as u64;
    let mid = (bytes[len/2] as u64) << 8 * (len / 2);
    let hi = (bytes[len - 1] as u64) << 8 * (len - 1);
    lo | mid | hi
}

Did you test it? It does not work

@RoDmitry Can you give an example input for which it is incorrect? I did test it with random data but perhaps I overlooked something.

My mistake. It works good.
Meanwhile I did this:
fn read_8(s: &[u8]) -> u64 {
    if s.len() < 8 {
        let mut val = 0;
        for ch in s.into_iter().rev() {
            val = (val << 8) | *ch as u64;
        }

        return val;
    }
    u64::from_ne_bytes(s[..8].try_into().unwrap())
}

which is slower than yours)

Is / 2 (dividing by 2) optimized to the same speed as >> 1?

@RoDmitry please let me know once all the miri issues are fixed and you are satisfied with the performance of the implementations that do not trigger undefined behavior.
Once that's done, I'll perform additional verification of memory safety and correctness using a fuzzer.

Will do, thanks!)

Fixed out-of-bounds read for Fallback with b27bf23698b78f6d6efe01176695eaaf40cb463a

Nice!
miri currently cannot handle SIMD, but Address Sanitizer detects an out-of-bounds read in SIMD code still.

@orlp any ideas optimizing these two functions?
pub fn read_16(s: &[u8]) -> u128 {
    let len = s.len();
    if len >= 16 {
        return u128::from_le_bytes(s[0..16].try_into().unwrap());
    }

    if len >= 8 { // 12?
        let lo = u64::from_le_bytes(s[0..8].try_into().unwrap());
        let hi = u64::from_le_bytes(s[len - 8..len].try_into().unwrap());

        return ((hi as u128) << (8 * (len - 8))) | (lo as u128);
    }

    /* if len >= 8 {
        let lo = u64::from_le_bytes(s[0..8].try_into().unwrap());
        let hi = u32::from_le_bytes(s[len - 4..len].try_into().unwrap());

        return ((hi as u128) << 8 * (len - 4)) | (lo as u128);
    } */

    if len >= 4 {
        let lo = u32::from_le_bytes(s[0..4].try_into().unwrap());
        let hi = u32::from_le_bytes(s[len - 4..len].try_into().unwrap());

        return (((hi as u64) << (8 * (len - 4))) | (lo as u64)) as u128;
    }

    if len == 0 {
        return 0;
    }

    let lo = s[0] as u64;
    let mid = (s[len >> 1] as u64) << (8 * (len >> 1));
    let hi = (s[len - 1] as u64) << (8 * (len - 1));
    (lo | mid | hi) as u128
}

pub fn read_32(s: &[u8]) -> [u128; 2] {
    if s.len() > 16 {
        return [
            u128::from_le_bytes(s[0..16].try_into().unwrap()),
            read_16(&s[16..]),
        ];
    }

    [read_16(s), 0]
}

I am trying to use it in SIMD, but it's like 3-5x slower than core::mem::transmute_copy(&s)
I use it like that:
unsafe fn read(s: &[u8]) -> __m128i {
    // _mm_loadu_si128(core::mem::transmute_copy(&s))
    _mm_loadu_si128(&read_16(s) as *const u128 as *const __m128i)
}

#[inline(always)]
unsafe fn read_avx(s: &[u8]) -> __m256i {
    // _mm256_loadu_si256(core::mem::transmute_copy(&s))
    _mm256_loadu_si256(&read_32(s) as *const [u128; 2] as *const __m256i)
}

@RoDmitry u128 stuff can be pretty slow. I'm sorry I can't help much here right now, it would take too much of my time to look exactly into what you're doing.

Never mind, I will try to use SIMD there.

@Shnatsel fixed

Excellent! I'll give it a whirl with a fuzzer later today. If I don't find anything it'd be great to release the version with the fixes to crates.io.
Would you like me to also file a security advisory so that people using cargo audit or Github advisory service got notified about this issue if they use an older version?

I still don't consider it as security vulnerability, more like a bug, which can crash your program, but If you think it will be useful, then of course.

No more memory safety issues flagged by cargo fuzz or cargo miri test.
Let's ship it!


fix format/auth when only name & metaurl in secret

fix https://github.com/juicedata/juicefs-csi-driver/issues/384

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@75a29de). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #385   +/-   ##
=========================================
  Coverage          ?   46.74%
=========================================
  Files             ?       26
  Lines             ?     2982
  Branches          ?        0
=========================================
  Hits              ?     1394
  Misses            ?     1488
  Partials          ?      100

Help us with your feedback. Take ten seconds to tell us how you rate us.


Update imgix-php dependency

imgix-php 2.1.0 has been released now. Doesn't look like it has any big changes.

This was updated in Imager 2.1.0 (!).


adding exec to the java command in the docker entrypoint

this makes Java PID 1 which is recommended so that SIGINT works properly

Thanks for the contribution.


feat(tactics/ext): use applyc _ {new_goals := new_goals.all}

This causes goals to appear in the same order they appear as hypotheses of the
@[extensionality] lemma, rather than being reordered to put dependent
goals first.
This doesn't appear to affect any uses of ext in mathlib,
but is occasionally helpful in the development of category theory.
(Indeed, I have been running into tactic mode proofs that fail to
typecheck, when using ext, and this avoids the problem. My
actual dependent ext lemma is at here, and the place where I need
to swap the goals back into order by hand is here.)
TO CONTRIBUTORS:
Make sure you have:

[X] reviewed and applied the coding style: coding, naming
[ ] for tactics:

[ ] added or adapted documentation in tactics.md
[X] write an example of use of the new feature in tactics.lean

For reviewers: code review check list

I think you should add a configuration option rather than changing the default.

I added an optional apply_cfg argument, just to the non-interactive tactic. I also added a short wrapper in tidy, that decides whether to just report ext or also report passing a configuration. This logic could be further improved, perhaps.

Perhaps in Lean 4 we can discuss the best default value for apply_cfg.new_goals!


 ignores backface-visibility: hidden CSS property

Demo: https://codepen.io/anon/pen/mYgVRP
 continues to be visible on the other side of the element after it's been flipped, ignoring backface-visibilit: hidden property.

We'll need to explore this a bit, but it probably needs to fit into our theming capabilities, tracked in #526

Should be fixed in M3, please open a new issue if <md-checkbox> does not fix the problem!


Accessible action buttons

Currently action buttons do not raise the OnAction event when selected using the keyboard. This is due to the onFilter event not firing for keyboard events.

Release in 1.0.24


Where it installed?

Hi,green hand in linux and already met a dozen of problem,cuz it is based on the same PC, want to copy directly ,but i couldn't find where it installed,thx a lot !谢谢

I think i find it ,just in the installed package ,but how could i restart the centos?
It didn't work

OK after a minute closed the windows ,it update hhh

Ok. I'm relieved to hear that! よかった!


While popup appears to select the default apps to take photo or video, background shows a white screen.

Make sure you follow all of the setup directions before posting an issue: https://github.com/jamesmontemagno/MediaPlugin#important-permission-information
If you are creating an issue for a BUG please fill out this information. If you are asking a question or requesting a feature you can delete the sections below.
Failure to fill out this information will result in this issue being closed. If you post a full stack trace in a bug it will be closed, please post it to http://gist.github.com and then post the link here.
Bug Information
Version Number of Plugin:
Device Tested On: Samsung A50
Simulator Tested On:  Android 9.0 API 28
Version of VS:  VS 2019 version 4.7..03190
Version of Xamarin: <IP_ADDRESS>3080
Versions of other things you are using:
Steps to reproduce the Behavior:
Have multiple apps that support recording or taking photos without any defaults set. Click on take image or video button white screen appears behind the shown apps
Expected Behavior
While popup appears to select the apps the background should be transparent so that the user knows on which screen he was.
Actual Behavior
While popup appears to select the apps the background shows a white screen.
Code snippet
private async Task TakePhotoButtonClicked()
{
try
{
if (!CrossMedia.Current.IsCameraAvailable || !CrossMedia.Current.IsTakePhotoSupported)
{
await Application.Current.MainPage.DisplayAlert(Resources.ImaapMobile.AlertTitle,
Resources.ImaapMobile.NoCamera,
Resources.ImaapMobile.OkText);
return;
}
            var image = await CrossMedia.Current.TakePhotoAsync(new StoreCameraMediaOptions
            {
                Directory = Settings.ImageDirectory,
                SaveToAlbum = true,
                CompressionQuality = Settings.ImageCompressionQuality,
                CustomPhotoSize = Settings.CustomPhotoSize,
                PhotoSize = PhotoSize.Medium,
                MaxWidthHeight = Settings.ImageMaxWidthHeight,
                DefaultCamera = CameraDevice.Rear,
                RotateImage = true
            });
            if (image == null)
                return;

.
.
.
}
Screenshots

#857 @jamesmontemagno


Total refactor

This is the beginning of a complete rewrite of pangeo forge, learning from our initial steps.
The core of the new code is a Recipe class, which is used like this:
r = recipe.NetCDFtoZarrSequentialRecipe(
    input_urls=netcdf_local_paths,
    sequence_dim="time",
    inputs_per_chunk=1,
    nitems_per_input=daily_xarray_dataset.attrs["items_per_file"],
    target=tmp_target,
    input_cache=tmp_cache,
)

# manual execution of a recipe
r.prepare()
for input_key in r.iter_inputs():
    r.cache_input(input_key)
for chunk_key in r.iter_chunks():
    r.store_chunk(chunk_key)
r.finalize()

The idea is that we will then implement a RecipeExecutor class which runs these recipes. This will work because the methods prepare, cache_input, store_chunk and finalize are not regular methods. The are actually properties that return serializable functions.
There is significant work to do here, but I feel pretty good about where this is going...

Black and flake8 both pass on my local env but are failing in this action. Not what I want to be debugging right now.

Thanks a lot Joe! I'll first reply to some of your questions and then update my PR in response to your comments.

In retrospect, I feel like this approach of creating dozens of mixins and multiple inheritance is premature complexification / abstractifiction.  I recently read this blog post and felt like it was speaking directly to me! 😆
Now I think that what we should do is create a basic working recipe class that implements the methods executors expect. As we define new recipes based that don't fit this mold, we should slowly refactor this class to make it more generalizable (rather than trying to generalize everything from day 1.)
Working on this now.

In retrospect, I feel like this approach of creating dozens of mixins and multiple inheritance is premature complexification / abstractifiction.

Couldn't agree more. I think I went through this same realization a few months ago. I think for now, the executors may be enough of an abstraction to allow us to move forward. If we find that recipes are frequently sharing elements, we can pull those out one by one.

Just a note: in https://github.com/pangeo-data/rechunker/pull/77 I am working on an update to rechunker that intersects with this.

Just a note: in https://github.com/pangeo-data/rechunker/pull/77 I am working on an update to rechunker that intersects with this.

https://github.com/pangeo-forge/pangeo-forge/pull/27#pullrequestreview-548527179 my only concerns being how we leave space for extension to pipelines that may not use xarray or zarr.

This was my first impression as well. I've not yet been able to make multiscales representations openable by xarray.

https://github.com/pangeo-forge/pangeo-forge/pull/27#pullrequestreview-548527179 my only concerns being how we leave space for extension to pipelines that may not use xarray or zarr.

This was my first impression as well. I've not yet been able to make multiscales representations openable by xarray.

@joshmoore what other tool are you thinking about to generate multiscale representations?

@joshmoore what other tool are you thinking about to generate multiscale representations?

@davidbrochart : really anything producing (largish) images (or generally spatial arrays?) would benefit from a multiscale representation. For other datatypes, I'd defer to other domains whether it's useful or not.

@davidbrochart : really anything producing (largish) images (or generally spatial arrays?) would benefit from a multiscale representation. For other datatypes, I'd defer to other domains whether it's useful or not.

The tests are hanging intermittently. But I think everything is working.

The tests are hanging intermittently. But I think everything is working.

The tests have become so unreliable for this PR. I think it has to do with starting the HTTP server.

The tests have become so unreliable for this PR. I think it has to do with starting the HTTP server.