Remove python3.vm

This packages only install the dependency, then we can remove this package and use the dependency instead. In fact, we are already using the dependency directly in libraries.python3.vm. After merging, we should remove this package from MyGet.
Closes https://github.com/mandiant/VM-Packages/issues/301

Ah, I made this package because I was testing why the "python3" dependency was failing in the CI pipeline, see: https://github.com/mandiant/VM-Packages/pull/263
Since the GH VMs already have Python3 installed, the package was failing. I was trying to find a way around this... but didn't have enough time to get the issue ironed out. Sorry for the confusion.

@MalwareMechanic can we just add python 3.9 as dependency in the packages that require python? This is what we currently do in libraries.python3.vm and it works in the GH VMs.
We could document somewhere to use Python 3.9 when possible.


The column technology typecannot be before the assay name -- ERROR

The validator throws this error, but technology type is before assay name in the template. Which order is correct?

@levitsky can you tackle this issue?

I can take a look tomorrow :)


Content is missing

The page has a table of contents but no actual contents...

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 7f4e153e-0fe3-2f8f-58b9-0402e3d11951
Version Independent ID: 8f7af191-8ac1-ac6c-a733-0216075373ee
Content: Configure Azure Active Directory B2C as a SAML IdP to your applications
Content Source: articles/active-directory-b2c/saml-service-provider.md
Service: active-directory
Sub-service: b2c
GitHub Login: @kengaderdus
Microsoft Alias: kengaderdus

@ddBenny
Thanks for your feedback! We will investigate and update as appropriate.

@ddBenny
Kindly choose policy type as "custom policy" and you will be able to see the content.

@YashikaTyagii thank you for the help!
Do I understand it correctly, that SAML does not go together with User flow policies?

@ddBenny
Yes, You are correct. It is also mentioned  in the following table.https://learn.microsoft.com/en-us/azure/active-directory-b2c/user-flow-overview#comparing-user-flows-and-custom-policies


Roadmap - trl 0.2

A list of cool things that we can aim for trl 0.2! :
API:

[x] general decoder models support (OPT, BLOOM, GPT-J, etc): https://github.com/lvwerra/trl/pull/53
[x] accelerate integration for training in mixed precision, DP (multi-GPU), using DeepSpeed: https://github.com/lvwerra/trl/pull/58
[x] shared backbones for lighter training: https://github.com/lvwerra/trl/pull/61
[x] Once ^ is merged (or inside the PR) - make the reference model optional and create both models inside the trainer if no reference model is passed |  https://github.com/lvwerra/trl/pull/67
[x] soften requirements : https://github.com/lvwerra/trl/pull/66
[x] Make quality & make style: https://github.com/lvwerra/trl/pull/62
[x] use accelerator logger instead of wandb to remove hard dependency (for users that do not want to use wandb) | https://github.com/lvwerra/trl/pull/92
[X] Refactor Trainer (push to hub etc. cc @lewtun) https://github.com/lvwerra/trl/pull/68
[x] general encoder-decoder models support | https://github.com/lvwerra/trl/pull/93
[x] Support for Adam 8bit (for lighter training)? | https://github.com/lvwerra/trl/pull/78
[x] clean step (sanity checks) | https://github.com/lvwerra/trl/pull/76
[x] dataset attribute should be optional ? | https://github.com/lvwerra/trl/pull/85
[x] Add learning rate scheduler support | https://github.com/lvwerra/trl/pull/96
[x] Handle: https://github.com/lvwerra/trl/pull/68#discussion_r1062268630 | https://github.com/lvwerra/trl/pull/86

Documentation

[x] doc build: https://github.com/lvwerra/trl/pull/59
[x] document API (classes, functions) | https://github.com/lvwerra/trl/pull/91
[ ] document tips and good practices when doing RLHF (hyper-parameters etc.) / https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/
[x] clean up README (examples don't work anymore)  https://github.com/lvwerra/trl/pull/70

Improvements

[ ] add different example than fine-tuning gpt2 on sentiment analysis
[ ] add an example with a custom reward model (SetFit?)
[x] convert the notebook 05 https://github.com/lvwerra/trl/pull/80
[ ] What is the largest model we can train on?
[ ] What is the coolest dataset we can use to train a model (e.g. anthropic AI dataset: https://huggingface.co/datasets/Anthropic/hh-rlhf ?
[x] Make all internal imports relative (e.g. from trl.trainer -> from .trainer) https://github.com/lvwerra/trl/pull/70
[x] Clean up setup.py and remove settings.ini (legacy from nbdev) https://github.com/lvwerra/trl/pull/70
[x] Create & add a logo | https://github.com/lvwerra/trl/pull/95

Any suggestion very welcome!

BTW, I can confirm that SetFit does make for a really good zero shot RM. There are some issues with using contrastive models as RMs though. It often requires very careful data cleaning and identifying what kinds of clusters work as RMs is a dark art to the point where we decided that it wasn't worth seriously pursing further after CARP CoOp. Rerank models are much better.

I think that the "coolest" dataset we can use to train a model could be https://huggingface.co/datasets/openai/webgpt_comparisons, but it is hard to evaluate this sort of model after we train it. I might start by adding a summarization example, and then some decent ways by which it can be evaluated. Then the webgpt comparisons example

https://colab.research.google.com/drive/1hkPBFtMP5xBAjNYMjWH7NqYn118kRLOJ?usp=sharing
I am trying to implement own gpt + trl with QA retrival reward, but i think something is wrong with reward/or generation

@AlexWortega can you open a separate issue / PR for this? Looks interesting, but may get loss in this big 1.0 roadmap thread.

We ended up calling this release 0.2 (not 1.0). I am closing the issue and will move the open tasks to a new issue.

Hi @lvwerra  i opened PR https://github.com/lvwerra/trl/pull/149  with this feature(?) idea


"Process Not Running; uploader" on devices uploading rlogs

Describe the bug
With cafca4f89176855585835ff7c67dea5a9b08ca23, a "Process Not Running; uploader" alert appears when engaging after reboot around 1/3 of the time in the testing closet (which uploads rlogs as immediate_priority). Seems like it could be a race condition somewhere?
Issue does not occur with the previous commit 2cd9975e4fe31bf89cdbf41d3d5db02f519a2718.
cafca4f89176855585835ff7c67dea5a9b08ca23_b9b916de_00000136--9ec02ae39d--0_rlog.zst.zip
Provide a route where the issue occurs
attached
openpilot version
cafca4f89176855585835ff7c67dea5a9b08ca23
Additional info
No response

It looks like the error happens when accessing a subdirectory of the log root, causing an exception.
@adeebshihadeh, we need to revert https://github.com/commaai/openpilot/commit/cafca4f89176855585835ff7c67dea5a9b08ca23. However, I think we still need to address this issue to prevent cleanup failures due to inaccessible subdirectories

We have to catch this in CI.


Minor, add class TableDesc.TableProject to avoid the use of Pair

Pull Request Test Coverage Report for Build 5573

25 of 47 (53.19%)  changed or added relevant lines in 4 files are covered.
5 unchanged lines in 4 files lost coverage.
Overall coverage increased (+0.005%) to 27.516%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

tool/src/main/java/org/apache/kylin/tool/CubeMigrationCLI.java
0
2
0.0%

core-metadata/src/main/java/org/apache/kylin/metadata/TableMetadataManager.java
3
6
50.0%

core-metadata/src/main/java/org/apache/kylin/metadata/model/TableDesc.java
7
11
63.64%

core-metadata/src/main/java/org/apache/kylin/metadata/model/TableExtDesc.java
15
28
53.57%

Files with Coverage Reduction
New Missed Lines
%

core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java
1
78.12%

core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java
1
81.08%

stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/ColumnarStoreCache.java
1
56.82%

core-metadata/src/main/java/org/apache/kylin/metadata/model/TableExtDesc.java
2
62.39%

Totals

Change from base Build 5570:
0.005%

Covered Lines:
24167

Relevant Lines:
87830

💛  - Coveralls

Codecov Report

Merging #1087 into master will increase coverage by <.01%.
The diff coverage is 53.19%.

@@             Coverage Diff              @@
##             master    #1087      +/-   ##
============================================
+ Coverage     25.04%   25.05%   +<.01%
+ Complexity     6224     6222       -2
============================================
  Files          1440     1440
  Lines         87821    87830       +9
  Branches      12275    12275
============================================
+ Hits          21999    22004       +5
- Misses        63664    63666       +2
- Partials       2158     2160       +2

Impacted Files
Coverage Δ
Complexity Δ

...n/java/org/apache/kylin/tool/CubeMigrationCLI.java
0% <0%> (ø)
0 <0> (ø)
:arrow_down:

...rg/apache/kylin/metadata/TableMetadataManager.java
58.2% <50%> (ø)
26 <0> (ø)
:arrow_down:

.../org/apache/kylin/metadata/model/TableExtDesc.java
59.82% <53.57%> (+0.5%)
14 <3> (ø)
:arrow_down:

...ava/org/apache/kylin/metadata/model/TableDesc.java
47.64% <63.63%> (+0.77%)
27 <0> (ø)
:arrow_down:

...he/kylin/dict/lookup/cache/RocksDBLookupTable.java
72.97% <0%> (-5.41%)
6% <0%> (-1%)

...ream/core/storage/columnar/ColumnarStoreCache.java
50% <0%> (-2.28%)
7% <0%> (-1%)

...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java
69.6% <0%> (-0.61%)
7% <0%> (ø)

.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java
66.15% <0%> (+2.3%)
0% <0%> (ø)
:arrow_down:

...he/kylin/job/impl/threadpool/DefaultScheduler.java
76.74% <0%> (+2.32%)
12% <0%> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 26cf1f8...3969ce8. Read the comment docs.


Docker native tool version

For you information, I found way to create this using Docker native tools:
https://github.com/olljanat/multi-win-version-aspnet

Thanks @olljanat that looks pretty good!


Update CloudsModel.cpp

related to issue#11

issue #11

I probably commented that because I couldn't find where it's been used...
not in perlinworley.comp or in worley.comp
Can you confirm that, in case I missed something.


Change properties to optional / add missing properties

Please change the below attributes in code for the aquasec_group_creation resource:

logical_name, gateways ==>  make it optional instead of required values, should be able to pass as terraform vars.

container_activity_protection, network_protection ==>  make it as an optional value instead of mandatory value

runtme_type, enforcer_image, syscall_enabled, sync_host_images, risk_explorer_auto_discovery ==>  these values should be added and should be optional

Add an attribute for forcefully removing the enforcers before deletion of the aqua group.

For point 1, the logical_name and gateways are set to optional and they are working fine.
For point 2 the variables are not being mapped and whenever i try to run terraform plan/apply it shows that it will set the values to false/null even though i am setting them in my main.tf file.
Also, in point 3 the variables are set to optional and they dont throw an error if they are not provided. But, setting them doesnt reflect any change on the enforcer group.
Point 4, Naveen and I are working on this and will report soon.


Allow ability to submit form with 'Enter'

Doesn't look like there's anyway to use the keyboard to submit the form. Even tabbing to the button and pressing 'Enter' doesn't work.
Is this a limitation?

Hi @silvaDominic.
Thank you for taking the time to write this. Unfortunately, you are correct in that it is a limitation of the Firefox sidebar. We are only able to redirect it to open their webapp inside of it, and the behaviour of said webapp is not up to us from there on.

Got it. Curious if you know specifically why.
I did a cursory search and it seems it could have to do with the host site's handling of events.

Accidentally confirmed that issue has to do with how 'submit' is handled between mobile and desktop on the website. If you resize the side panel to trigger a tablet or greater view, the events works just like normal. Once you hit the mobile width threshold though, the events are handled differently.

Thank you for the follow-up. It satisfied my curiosity as well!


Support Azure AD joined device authentication

Describe the improvement
In order to provide support for Azure AD joined device authentication with multi-tenancy, the following product improvements are required.

Multi-tenant support for STS service
Support both tenant-aware & tenant usernames in the UT token of the STS request
Mex endpoint multitenancy support
Add missing endpoints in mex endpoint that are required for this flow
WS-Trust response issuer name to be configured in a multi-tenanted way

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Issue 3041

fixes #3041
I wasn't sure about my changes in import-data.md. Please double check.

This change is

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/b7376771b3870e3dcf4b485b43ca76acbda8cfb3/

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/b7376771b3870e3dcf4b485b43ca76acbda8cfb3/

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/1b11650505eb77f1778063cc3afd3befff45c561/

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/b52ff9a4e7e24f1fc9c2bc24fd4cd1b24539d07e/


Add Spatial :latest

If there’s a checkbox you can’t complete for any reason, that's okay, just explain in detail why you weren’t able to do so.
After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} reports no offenses.
[x] The commit message includes the cask’s name and version.

Additionally, if adding a new cask:

[x] Named the cask according to the token reference.
[x] brew cask install {{cask_file}} worked successfully.
[x] brew cask uninstall {{cask_file}} worked successfully.
[x] Checked there are no open pull requests for the same cask.
[x] Checked the cask was not already refused in closed issues.
[x] Checked the cask is submitted to the correct repo.

@miccal Thanks for taking a look! I think binary 'mac', target: 'spatial' is needed as the downloaded binary is called mac (I tested locally without and it fails with (Error: It seems the symlink source is not there)

@longdivision when I download the file it is called spatial?

@miccal I believe that's your browser honouring the Content-Disposition: attachment; filename=spatial header on the response.
If you run the following (without binary 'mac') you'll see cask fails to find the binary:
brew cask install https://raw.githubusercontent.com/longdivision/homebrew-cask/bb125ef7883b33ca8e7068c5450863eaf957751d/Casks/spatial.rb

If you use the previous version of the recipe (with binary 'mac') it manages to download and install:
brew cask install https://raw.githubusercontent.com/longdivision/homebrew-cask/6f675d2bad7a3bfbd6ccb7ce0e6e63ba6bd883fc/Casks/spatial.rb


frontend: User hub

Covering the page where the users can consult which books they have on loan, which they have requested, with just current data (no statistics for now).

Should be possible to do with pre-defined search queries.

Implemented in https://github.com/inveniosoftware/invenio-circulation/pull/59


WAFER-100W-4A part 2

How to find the PN of the wired connector for the connector WAFER-100W-4A ?

I don't know what this connector is.

It is on the supplied schematic!

i'm using this cable:
SH1.0 JST 4p.
https://www.aliexpress.com/item/1005001569197247.html

Well, I'm just dealing with software issues. There is no emphasis on the details of the hardware circuit.

I'm using these, although I did have to rearrange the wires so that the red and black were on the 3V3 and GND pins.
https://smile.amazon.com/dp/B08RHGT3W3?psc=1&ref=ppx_yo2ov_dt_b_product_details

It's the qwiic / stemmaQT / JST-SH connector. I'm using these at 80pence each with various breakout boards (adafruit / sparkfun / laskakit.cz ). Cheap enough to cut and solder to boards without qwiic sockets.
£0.80 per cable 10/15/20cm, £1.30 for 30cm and £1.50 for 40cm.
https://thepihut.com/search?q=qwiic&narrow_by=&sort_by=price_min_to_max&page=1

Hi, I have mostly M5Stack sensors with 2.5 pitch Grove connectors, so I made a converter!
Cheers !
Terry

On Dec 3, 2022, at 1:17 PM, Tyeth Gundry @.***> wrote:

It's the qwiic / stemmaQT / JST-SH connector. I'm using these at 80pence each with various breakout boards (adafruit / sparkfun / laskakit.cz ). Cheap enough to cut and solder to boards without qwiic sockets.
£0.80 per cable 10/15/20cm, £1.30 for 30cm and £1.50 for 40cm.
https://thepihut.com/search?q=qwiic&narrow_by=&sort_by=price_min_to_max&page=1
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.

Thanks a lot of help. I have bought the qwiic connector, but will get in after 2-3 weeks only.  But why this connector is not include in the box the same as the JST power connector? It would save a lot of time.

I don't think it's standard enough yet, as most daughter boards (sensor breakout boards etc) don't have the matching socket. It would be more common to include dupont jumper cables like an arduino starter kit.
Interestingly over the last year there has been a dramatic push by Adafruit to revamp all their boards with them, as the no-soldering route seems to enable access to a lot more users. Glad the connectors on there so thanks "LilyGo", but 75% of my sensors didn't come with the connector. I am now buying sensors with JST-SH/stemmaQT if possible, and adding cables to those that don't as the plug-and-play way is a winner.


Autocomplete : renderInput attribute

Hi okld and hi everyone, I hope this project still lives because it's very helpful to render nice Streamlit apps.
I'm working with Autocomplete component from MUI but I'm struggling making it works. The issue here is that mui.Autocomplete object needs a renderInput prop (it should be a lambda function returning another component such as TextField). Here is how I proceed to try making it works :
with elements("some-name"):
  options=[{"label": labels[i], "id": i} for i in range(len(labels))] # labels is defined as a list of strings
  autocomplete = mui.Autocomplete(options=options, renderInput=lambda params: mui.TextField(params, label="Some label"))

The problem here is that it throws an ElementsFrameError cause mui.TextField is out of an with elements("id") statement.
I tried some variations such as :

wrapping the mui.TextField in a function defined before the mui.Autocomplete declaration : but it renders the TextField without Autocomplete before other declared elements...
passing a mui.TextField in a with mui.Autocomplete(options=options) statement. As expected, it throws an Exception because the renderInput prop isn't specified...

My question is : How to make mui.Autocomplete work with streamlit-elements module ?
NB : Moreover, I was wondering if we could have some tutorials on how to make Date Pickers work for example. Thank you very much for your help and consideration
Victor

@vcourintracks did you figured out how to make autocomplete work?

Anyone figured out how the autocomplete work?


Containerized builds through GitHub Actions.

Scope
Containerized builds for Linux distros not supported through GitHub's own runners.
Fixes #4.
Fixes #8 (makes it superfluous).
Changes
Build on the work in https://github.com/chevah/server/pull/1744 to add containerized builds for:

[x] Alpine 3.12
[x] CentOS 5.11
[x] CentOS 8.2 (which has OpenSSL 1.1.1c for backward compatibility with 8.0-8.2).

Drive-by changes:

Forward ported the latest improvements from python-package.
Specific support for Amazon Linux 2 and RHEL 7 was removed.
Trimmed down Alpine deps, to have it work on minimal containers.
Updated embedded OpenSSL libs to 1.1.1k.
Updated cryptography to 3.4.7.

Have experimented with building cryptography with Rust.  On pypi.chevah.com, the following new modules were needed: semantic_version and setuptools_rust.
In the end, have built cryptography without Rust, because it would have added one more dep: libgcc (which is more or less an essential package). So I've used CRYPTOGRAPHY_DONT_BUILD_RUST=1.
Testing
Please review changes.
Check the automated tests and the resulting uploads to https://bin.chevah.com:20443/testing/3.8.6.ee5d7c0/.

great work. I think that we can have the job defined in the main workflow .

great work. I think that we can have the job defined in the main workflow .

great work. I think that we can have the job defined in the main workflow .

Yeah, I don't know, let's see how verbose it turns out in the end. I hope to use a matrix for everything but Centos 5 and 6.

great work. I think that we can have the job defined in the main workflow .

Yeah, I don't know, let's see how verbose it turns out in the end. I hope to use a matrix for everything but Centos 5 and 6.

uses: chevah/auto-cancel-redundant-job@v1
can be removed as it no longer works.

You need to use the new "concurency" feature from GitHub like we have at
chevah/server#1792

Thanks, I'll look into it.

Also fetch-depth: 2 can be removed... I think that it was only used for our skip-ci hack... but now GitHub has native support.

I have removed it once and then put it back because I realize it is actually needed. Can't recall the details though, so perhaps I'll remove it again to document this better if this is still the case.

Also fetch-depth: 2 can be removed

This was a very minor comment... I think that we can also leave it as it is as it does no harm.


Unabe to stopPropagation for Overflowmenu

Detailed description
I have a scenario where overflowmenu is inside a div (which has click event).
FYI: onOverflowMenuClick is a callback for overflow-menu openChange and it is emitting a boolean value instead of actual event.
Current behaviour:
When I trigger overflow-menu both onOverflowMenuClick and onCardClick callbacks are calling.

    <ng-content></ng-content>
    <overflow-menu [menuOptions]="menuOptions" (overflowmenuClick)="onOverflowMenuClick($event)"></overflow-menu>

Expected behaviour:
I want to do stopPropagation onOverflowMenuClick callback so that onCardClick will not get trigger.
Pls suggest any other possible way if you can think of.

This works 💯

    <ng-content></ng-content>

        <buc-overflow-menu [menuOptions]="menuOptions" (overflowmenuClick)="onOverflowMenuClick()"></buc-overflow-menu>


XML Language Improvements

Issue Type: Improvement/Bug Report (?)

As a frequent user of XML for planning layouts and storing/retrieving data, I can say that some parts of XML aren't that well covered by VSCode. Specifically, this is XML DTD (Document Type Definitions) and the highlighting (and when inspecting using TM Scopes, the internal structure) isn't as fleshed out. Within this issue, I'd like to submit my own ideas for improving this. Below is a screenshot of some XML DTD Code which I will reference frequently to make my points:

I am using the theme "One Dark Pro" but testing the default themes also gives this same issue.
The example I am using is taken from

This issue will cover the following areas which I've created:

Keywords
Special Characters
Highlighting and Structure - This is more interweaved throughout, but thought I'd mention it for clarity

DISCLAIMER: For those who say use XSD, I do use that but sometimes I revert to DTD just for historical reasons or (at least for me), it's easier to define document-only related elements/attributes without having to do too much XSD Namespacing and that. Now onto the bulk!

Keywords
In XML DTD, Keywords can be split into multiple sections (at least this is what I can split them into):

Declarations: These include <!ELEMENT> and <!ATTLIST> and the name says it all
Modifiers: Such as EMPTY, ANY, #REQUIRED, #IMPLIED
Data Keywords: Including ones such as #PCDATA, CDATA
Values: Usually something that the user would type in. According to w3schools, these can be examples such as attribute-name`` and attribute-value```

As per my screenshot above, it is evident that for Declarations, there is some syntax there, as <!DOCTYPE> and <!ENTITY> is highlighted, these both fit in the scope keyword.other.doctype.xml and keyword.other.entity.xml respectively. However, it seems other keywords, especially <!ELEMENT> and <!ATTLIST> don't seem to have anything. The scope only shows them as part of meta.internalsubset.xml. Here, the obvious resolution would be to stick these with their own scope such as keyword.other.element.xml and keyword.other.attlist.xml and give them that purply highlighting (or blue in default dark+).
Secondly, for Modifier Keywords, I believe they should have a colour similar to that of JavaScript type objects such as Boolean (green default), and have a scope such as keyword.other.modifier.xml.
Again, these keywords present meaning and therefore, should be highlighted to give this meaning. Now this is where it gets more ambiguous as Data Keywords also share similar syntax, they can have #s too but not have them at the same time, in my examples, I've given #PCDATA and CDATA. To me, these should have the scope of charData or something of the sorts, since these keywords represent what can be 'parsed' inside an XML element, so possibly along the lines of keyword.other.char-data.xml, and also highlight them in the colour which is used in JavaScript variable number values
Now, this part gets really complicated. Welcome to Values! This section gets crazy because there are many different scopes that could be implemented and some can only appear on certain Declarations etc., and it's just a huge XMessL (see what I did there? I'm sorry). If I were to take the <!ELEMENT> declaration and give a quick diagnose:

Accepts element-name
Accepts category (This is essentially Modifiers but I renamed it to cover more values)/Modifiers
Accepts element-content including Data Keywords and Other Elements, along with Special Characters (more on that later)

Some of this is easy, some not. element-name, easy, just highlight it like any HTML/XML element that a theme would do? Default is blue, but One Dark Pro recognises it as a variable so it's red, doesn't matter to me that much though. Modifiers, explained pretty much. element-content is slightly more challenging, since it uses () and within the brackets, Data Keywords can be put inside, or special characters and other elements, separated by commas. In my screenshot, a good example is Line 10: <!ELEMENT PRODUCT (SPECIFICATIONS+,OPTIONS?,PRICE+,NOTES?)> and Line 23: <!ELEMENT OPTIONS (#PCDATA)>. My solution to highlight those elements is simply the element colour. I don't think the brackets need to be highlighted though, since I think there is enough mishmash XML colour here already right?
The next step is to diagnose the <!ATTLIST> declaration, as there is whole lot more that can be put into here:

They take element-name and modifiers as stated above, but also:
attribute-name
attribute-type
attribute-value
Can repeat the attribute stuff again so you can have multiple

Good examples from the screenshot are at Lines 19 - 21:
<!ATTLIST SPECIFICATIONS
WEIGHT CDATA #IMPLIED
POWER CDATA #IMPLIED>

and Lines 24 - 26:
<!ATTLIST OPTIONS FINISH (Metal|Polished|Matte) "Matte"
ADAPTER (Included|Optional|NotApplicable) "Included"
CASE (HardShell|Soft|NotApplicable) "HardShell">

Just to explain briefly, this issue is already goddamn long enough.... The first example best represents the basic syntax. It has element SPECIFICATIONS, attribute WEIGHT, type CDATA (characters basically) and a modifier of #IMPLIED. The Second one is monstrous at first sight, but really, the only differences are that it contains a Default Value (in the double quotes) and Options for the attribute-value, separated by the special character |. To resolve this structure, obviously provide scopes such as keyword.other.attlist.attribute-name.xml, and replace attribute-name with the relevant one such as type and value. For highlighting the ones in brackets in example 2, the words would probably have to be highlighted in the colour used for attribute values in HTML/XML, same as the default-value since, well they are and can be values after all. Possibly add a .attribute-value-range scope?
That's basically it for this section, next up is Special Characters! Luckily, this is a much shorter section, less to develop so don't worry, you're almost done (if you haven't stopped reading already)
Special Characters
So far, we've already touched up on some special characters, these include the + ? | * operators, their meanings aren't relevant right now, but they usually appear inside parentheses, and denote when and how an element to appear (basically frequency). These should have a scope such as operators.xml and be highlighted in the colour of normal operators in say JavaScript (yes I've been referencing JS the most but it's one language I'm very close to)
One thing that I anticipate could be thrown up, is controlling it appearing in actual XML documents or in DTD. In actual XML, any text between elements is just text, so it shouldn't be highlighted. Plus, It's scope and hierarchal based, so it wouldn't be incorrect anyways. There are examples of this everywhere.
Finally (yes, I know!), i know I've probably made some errors, I've grouped things unnecessarily and jumped a few barriers. Below I'll provide a few links for reading into XML DTD more so you can understand where I;m coming from, and if any questions pop up, feel free to ask:

Thanks for the write-up. Yes it would be nice if the XML syntax highlighting would support the full DTD syntax.
For XML we use the grammar for https://github.com/atom/language-xml/
Would you mind filing the issue there?


removing warning since xcode 4.2 and ios 5.0

removing warning since xcode 4.2 and ios 5.0 by casting the variable "branch" in an "NSArray*" when it is one.

too old, closing


missing ".with_cigar()" in Aligner?

Hello! Thank you for creating chopper for us. However, I noticed when I was trying to remove DCS reads from my fastq files that a good portion of contaminating reads still remain. This is an example of one read blasted against the DCS sequence.

(query) bad read: 3,800 bp (90% =3,420bp)
(target) DCS: 3,560 bp
Chopper left these reads, so I decided to manually run minimap2 -ax map-ont DCS.fasta read.fq to see the PAF results. My "match_len" was 3,510bp. Please correct me if I'm misinterpreting the filter function, but I assume because 3,510bp > 3,420bp it should be classified as a contaminate.
Alternatively if i run minimap2 -x map-ont DCS.fasta read.fq  my "match_len" was 3,268bp. Because it is not greater than 3,420bp the read would be retained. Could chopper be inaccurately reporting the lengths because the Aligner setup in lines: 178-184 is missing ".with_cigar()"? https://github.com/lh3/minimap2/issues/158
fn setup_contamination_filter(contam_fasta: &str) -> Aligner {
    Aligner::builder()
        .with_threads(8)
        .map_ont()
        .with_index(contam_fasta, None)
        .expect("Unable to build index")
}

.with_cigar() fixed the problem! Normally I don't have to worry about DCS reads because I filter 5kb+ but the throughout of my current dataset required me to lower my threshold. Host read removal also looks better.
Thank you for the update 😃

Thanks for confirming, I will push out a new release now.

Hello, I'm still experiencing this issue with version 0.7.0 (bioconda install on M1 Mac). minimap2 -aLyx map-ont aligns ~200 reads to the DNA CS sequence; chopper -c DCS.fa does not remove any of them. Other filtering flags (quality, length) are working as expected. Thanks for taking a look!

Is this data you can share with me?

Here is the the FASTQ and reference FASTA for DCS:
barcode12.all.fastq.gz
DCS.fasta.txt

Hi,
The requirement for the contamination filter is that at least 90% of the fragment aligns with the contaminant to avoid removing short aspecific matches that are in the read by chance. It appears that in your dataset, about 40-60% of the fragment aligns with the DNA-CS. Can you think of a reason for this? Are there long adapters or something like that?


Solve contrast issues on light theme

Contrast for some logos and UI elements is not compliant and even leads to almost-invisible elements.
Example:

Review light theme and deliver proper contrast.

Light theme is going to be removed as it increases the complexity and maintenance that I cannot afford.


Configurable state cache size and enforce exact state cache size

Closes https://github.com/paritytech/substrate/issues/1360 and replaces the inaccurate guesstimate which was used when setting  up the state cache with exact tracking/enforcement.
The difference in the function signature of storage_insert(…) and storage_remove(…) is because when storage_remove is called cache is already borrowed mutably.
Side note: https://crates.io/crates/lru-cache contains

WARNING: THIS PROJECT IS IN MAINTENANCE MODE, DUE TO INSUFFICIENT MAINTAINER RESOURCES
It works fine, but will generally no longer be improved.

@DemiMarie-parity Regarding the bookkeeping overhead, it is hard to measure it exactly without getting into LruCache implementation details. I'm fine with specified cache size being the size of actual state data being kept in cache, not the maximum memory that it allocates. Otherwise we'd also have to account for Vec::capacity vs Vec::len and probably require use of heapsize or similar crate.


Image zoom for mobile browser apps

Details
Image zoom for mobile browser apps
Fixed Issues

$ https://github.com/Expensify/App/issues/36597
PROPOSAL: https://github.com/Expensify/App/issues/36597#issuecomment-2119960518
Tests

Open reports with attachments on mobile web.
Open image attachment.
Verify image can be dragged and zoomed using pan and pinch gestures.

[x] Verify that no errors appear in the JS console

Offline tests

Open reports with attachments on mobile web.
Open image attachment.
Verify image can be dragged and zoomed using pan and pinch gestures.

QA Steps

Open reports with attachments on mobile web.
Open image attachment.
Verify image can be dragged and zoomed using pan and pinch gestures.

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native
https://github.com/Expensify/App/assets/68128028/72ca4318-358f-42b8-bb44-b925385ae4c4

Android: mWeb Chrome
https://github.com/Expensify/App/assets/68128028/6c53fca3-cbeb-4459-bcc3-686f526a9997

iOS: Native
https://github.com/Expensify/App/assets/68128028/b89cd16b-c745-4e0b-9cfc-c6b6b97f48ee

iOS: mWeb Safari
https://github.com/Expensify/App/assets/68128028/b905c9e2-572f-47d1-9dc6-f9e740559018

MacOS: Chrome / Safari
https://github.com/Expensify/App/assets/68128028/a0519356-ec0f-47aa-93b5-4ee6a522074f

MacOS: Desktop
https://github.com/Expensify/App/assets/68128028/c3e584b2-c286-4dc0-9a00-9733e580a91d

Going to build some adhocs for this for easier testing.

One thing I noticed while testing this is that when I use a single finger to drag the image around, the image doesn't move "live" in sync with my finger. It only moves after my touch gesture has finished. Is that expected? That's not how it works on native at least.

Ah try zooming in first, and then dragging with one finger

Starting the builds now!

Tested and this feels pretty good to me!

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/flodnv in version: 9.0.4-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Cherry-picked to staging by https://github.com/tgolen in version: 9.0.4-5 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

@Expensify/applauseleads please QA this PR and check it off on the deploy checklist if it passes.

🚀 Deployed to production by https://github.com/Julesssss in version: 9.0.5-13 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/thienlnam in version: 9.0.6-8 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


能否支持多用户登录

Qinglong version
其他用户登录，就会被挤下来，很麻烦
Steps to reproduce

What is expected?
允许多用户登录
What is actually happening?
无
System Info
No response
Any additional comments?
No response

repeat https://github.com/whyour/qinglong/issues/1947


[Docs]: remove runtimeCaching for a specific endpoint

Provide environment information
Using latest version on next js 14
Describe the problem
Hi! I want to prevent the runtime caching on all the endpoints that starts with /api/auth/[auth0].ts, is there a way to do it updating the current defaultCache in next/worker?
Expected behavior
Exclude some specific path from runtime caching
Screenshots (if relevant)
No response
Additional information (if relevant)
I've been able to do it but I had to remove all the references to "api" inside the defaultCache and with exclude true here
const withSerwist = (await import("@serwist/next")).default({
			swSrc: "src/worker/index.ts",
			swDest: "public/sw.js",
			exclude: [() => true],
		});

I've been able to do it but I had to remove all the references to "api" inside the defaultCache and with exclude true here

😅
You don't need to do that. exclude is meant for precaching rather than runtime caching. To prevent runtime caching for all such routes, prepend a network only entry that specifically target those routes:
const routesToFilter = ["/api/foo", "/api/bar"];

const serwist = new Serwist({
  runtimeCaching: [
    {
      matcher({ sameOrigin, url }) {
        return sameOrigin && routesToFilter.includes(url.pathname);
      },
      handler: new NetworkOnly(),
    },
    ...defaultCache,
  ],
});

serwist.addEventListeners();

wow that was super useful! thanks a lot now it's working as expected :)

Hi @DuCanhGH, hope you are good
Related to this, is it possible to do it also for a server action?
Thank you

@Sapessii server actions are fundamentally just HTTP POST requests, so you can just filter out routes that have server actions. Remember to add method: "POST" to each runtimeCaching entry you want to match those actions though, otherwise you'll risk filtering out GET requests instead.

@Sapessii server actions are fundamentally just HTTP POST requests, so you can just filter out routes that have server actions. Remember to add method: "POST" to each runtimeCaching entry you want to match those actions though, otherwise you'll risk filtering out GET requests instead.

Thank you @DuCanhGH for the fast reply, I have tried with this but is not working
Basically I have a problem with react query where I am prefetching a server action inside the path /messages/{roomId}
Am i doing something wrong? Because using the standard api under /api/chat/messages the bug that I have is not happening
Thank you!
const routesToFilter = [
	"/api/auth/login",
	"/api/auth/callback",
	"/api/auth/logout",
	"/api/auth/me",
	"/api/chat/messages",
];

const actionsToFilter = ["/messages/"];

const serwist = new Serwist({
	precacheEntries: self.__SW_MANIFEST,
	skipWaiting: true,
	clientsClaim: true,
	navigationPreload: true,
	runtimeCaching: [
		{
			matcher({ sameOrigin, url }) {
				return sameOrigin && routesToFilter.includes(url.pathname);
			},
			handler: new NetworkOnly(),
		},
		{
			matcher({ sameOrigin, url, request }) {
				return (
					sameOrigin &&
					actionsToFilter.includes(url.pathname) &&
					request.method === "POST"
				);
			},
			handler: new NetworkOnly(),
		},
		...defaultCache,
	],
});

@Sapessii add method: "POST" alongside handler: ..., matcher: ..., etc., not as a check in your matcher function.


cherry-pick backup related fixes to release-0.20

For the db restore drill.
#5797
#5824
#5826
#5821
#5838

Looks like there is a linting failure.

/land

😁


make generic POST/PUT/GET/ delete blocks with just text

started / See chuck norris joke from the demo. also see blocklyXHR.js


Jfrog client doesn't properly pass maven java parameters (-D) containing spaces -ERROR org.apache.maven.cli.MavenCli - Unknown lifecycle phase

From Azure Devops "Artifactory Maven " Task execute the maven with following parameters:
Maven POM file: pom.xml
Goal:clean
Options:  compile exec:java -X -Dsun.jnu.encoding=UTF-8 -Dexec.mainClass=net.lidl.taf.jira.corefunction.JiraLauncher  -Djira.issue.custom.field.values=$(jira.issue.custom.field.values)

Variable $(jira.issue.custom.field.values) has the following value : "Test automation"

Artifactory maven task run the following command:
c:\AzureAgent_tool\jfrog\1.39.5\x64\jfrog.exe rt mvn clean -f c:\AzureAgent\16\s\pom.xml compile exec:java -X -Dsun.jnu.encoding=UTF-8 -Dexec.mainClass=net.lidl.taf.jira.corefunction.JiraLauncher  -Djira.issue.custom.field.values="Test automation"

Result; Space braks the mvn command and then consider the remaining part as invalid lifecycle .
I tries to eascape the space using '""' or \ or double escape but without success
JFrog CLI version: 1.39.5 (it is downloaded by Maven Artifactory and we cannot change it)
2020-11-06T16:46:08.4071608Z [main] ERROR org.apache.maven.cli.MavenCli - Unknown lifecycle phase "automation". You must specify a valid lifecycle phase or a goal in the format : or :[:]:. Available lifecycle phases are: validate, initialize, generate-sources, process-sources, generate-resources, process-resources, compile, process-classes, generate-test-sources, process-test-sources, generate-test-resources, process-test-resources, test-compile, process-test-classes, test, prepare-package, package, pre-integration-test, integration-test, post-integration-test, verify, install, deploy, pre-clean, clean, post-clean, pre-site, site, post-site, site-deploy. -> [Help 1]
2020-11-06T16:46:08.4099394Z org.apache.maven.lifecycle.LifecyclePhaseNotFoundException: Unknown lifecycle phase "automation". You must specify a valid lifecycle phase or a goal in the format : or :[:]:. Available lifecycle phases are: validate, initialize, generate-sources, process-sources, generate-resources, process-resources, compile, process-classes, generate-test-sources, process-test-sources, generate-test-resources, process-test-resources, test-compile, process-test-classes, test, prepare-package, package, pre-integration-test, integration-test, post-integration-test, verify, install, deploy, pre-clean, clean, post-clean, pre-site, site, post-site, site-deploy.
For more info thee is also ticket #146533

Thanks for reporting this @victortanase.
The issue seems to caused, or at least related to the following ParseArgs function in the jfrog-cli-core module -
// Iterate over each argument, if env variable is found (e.g $HOME) replace it with env value.
func ParseArgs(args []string) ([]string, error) {
	// Escape backslash & space
	for i := 0; i < len(args); i++ {
		args[i] = strings.ReplaceAll(args[i], `\`, `\\`)
		if strings.Index(args[i], ` `) != -1 && !isQuote(args[i]) {
			args[i] = strings.ReplaceAll(args[i], `"`, ``)
			args[i] = strings.ReplaceAll(args[i], `'`, ``)
			args[i] = `"` + args[i] + `"`
		}
	}
	parser := shellwords.NewParser()
	parser.ParseEnv = true
	return parser.Parse(strings.Join(args, " "))
}

We're are trying to figure out if this function is still needed. This function removes quotes from the command args, which seems to lead to this issue.


Social Media Colors

FEATURE REQUEST: Any way to add a palette of popular social media colors? Would be very helpful!

brandcolors has been added.


Shared execution context draft

A shared execution context is used to reduce initialization and reduction costs. The feature is in a draft state, works only for a keyless perfect hash with COUNT and SUM aggregates. Works with sub-fragments only.

@ienkovich how is this branch looking in terms of moving out of draft? Also saw the asan failure in GeospatialTest, any ideas on that?

@ienkovich how is this branch looking in terms of moving out of draft? Also saw the asan failure in GeospatialTest, any ideas on that?

The functionality of this patch is quite limited but it passes tests and can be used for experiments and further extension.
Asan failure description is not very helpful. Right now can't see anything in the patch which could cause such an error, especially with the feature disabled during testing.


Support custom generated source extensions

Since URI_HAS_NOT_BEEN_GENERATED has been added 2 years ago, there is still a TODO to generalize the mechanism for finding generated sources. Now that everything is being migrated over to the build package, where everything has to have a new extensions, it would be nice to have this generalized somehow, even if it's just a generated-source-extensions option in analysis_options.yaml.

@natebosch Is this still an issue for package:build generated files?

yes, it's still possible to have files with any extension that can be produced during the build and show up before a build as an error.


added selected google fonts

I added the Google Fonts discussed on Trello.
Currently Headers are Noto Serif, Paragraphs are Andika, and Bingo Words are Mulish
We may want to adjust the fonts sizes

I can make both of these changes tomorrow

I increased the size of the themed text as well as the themed headers. The headers now scale slightly with screen width. Also "Let's Play Bingo!" is now header sized


feat: add option to add additional custom pod labels

What this PR does / why we need it:
This PR adds the option to configure additional custom pod labels for all pods created by the controller deployment.
This makes it possible to use the controller in environments that require the use of certain labels on all pods, e.g. for governance reasons. One concrete example would be the requirement to label each pod with the cost center that pays for it.
The PR adds an additionalPodLabels parameter, which is a pattern that is used by many helm charts to solve this issue.
To configure the additional labels, you need to do the following:
(A) For the helm chart deployment, add to your values.yaml:
deploy:
  additionalPodLabels:
    foo: bar
runtimeConfig:
  manager:
    additionalPodLabels:
      foo: bar

(B) For the manifest deployment, edit your manifest:

Add the label to the eraser-controller-manager pod template
Add to the contents of the eraser-manager-config config map:

manager:
  additionalPodLabels:
    foo: bar

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
None. This PR proposes this feature.
Special notes for your reviewer:
Thank you for looking at this. I'm looking forward to your feedback and hope we can add this feature to the controller.

@patrick-vonsteht the api changes lgtm, thanks! we will also need to change the controller code to set the labels if they are present in the config (https://github.com/eraser-dev/eraser/blob/main/controllers/imagecollector/imagecollector_controller.go#L326) before creating the ImageJob.

@ashnamehrotra thank you for the hint, I'll take a look at that.

@ashnamehrotra I have extended the code to add the additional labels also to the ImageJob PodTemplateSpec.

Looks like all the tests are successful, except the vulnerability scan. But the vulnerabilities are unrelated to my changes, because I didn't change any dependencies. How should I proceed with this?

Looks like all the tests are successful, except the vulnerability scan. But the vulnerabilities are unrelated to my changes, because I didn't change any dependencies. How should I proceed with this?

Since that is a vulnerability in the trivy binary we can ignore that.

Looks like all the tests are successful, except the vulnerability scan. But the vulnerabilities are unrelated to my changes, because I didn't change any dependencies. How should I proceed with this?

I can put up a separate PR to update that vulnerability.


6785 Wrong error message from pragma(msg) of failed instantiation

Fixes issues 6785.
More _error suppression.

The commit message failed to be recognized by github/bugzilla because it did not follow the form:
fix issue 6785


build: bump fuel-vm to 0.4 and fuel-tx to 0.5

Use the newer versions of fuel-tx and fuel-vm in dependencies, and adapt the code accordingly
This PR closes #84.

Currently struggling with using fuel-tx::{Transaction,Receipts} properly,I keep getting this error:

The version bumps now makes 5 tests fail (others run well):
test call_with_empty_return ... FAILED
test abigen_different_structs_same_arg_name ... FAILED
test example_workflow ... FAILED
test call_with_structs ... FAILED
test type_safe_output_values ... FAILED

They fail with Custom { kind: Other, error: "Response errors; The provided identifier is invalid!" }, which occurs on contract calls (in script.rs).


Opaque type does not unify with underlying type in declaring module when used as type parameter

Opaque type does not unify with underlying type in declaring module when used as type parameter
I am running into what I believe is a bug relating to instantation of
polymorphic types at an opaque type. I have a workaround, but I want to
make sure that I am understanding the situation correctly, and in
particular that this workaround is not expected to be made inviable in a
future release.
The example requires two files, and does not reproduce in the
playground (but does on a fresh install of Flow 0.73.0, etc.).
In id.js:
// @flow

export type Id<T> = (T) => T;

export function makeNumberId(): Id<number> {
  return (n: number): number => n;
}

In nonneg.js:
// @flow

import {type Id, makeNumberId} from "./id";

export opaque type NonNeg: number = number;

export const nonNegId_broken: Id<NonNeg> = makeNumberId();  // error

export const numberId: Id<number> = makeNumberId();
export const nonNegId_working: Id<NonNeg> = numberId;

The error, on the line thusly marked, is:

Cannot assign makeNumberId() to nonNegId_broken because number
is incompatible with NonNeg in type argument T.

Full error text
$ yarn -s flow --message-width 80
Error ┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈ nonneg.js:7:44

Cannot assign makeNumberId() to nonNegId_broken because number [1] is
incompatible with NonNeg [2] in type argument T [3].

     nonneg.js
      4│
      5│ export opaque type NonNeg: number = number;
      6│
 [2]  7│ export const nonNegId_broken: Id<NonNeg> = makeNumberId();  // error
      8│
      9│ export const numberId: Id<number> = makeNumberId();
     10│ export const nonNegId_working: Id<NonNeg> = numberId;

     id.js
 [3]  3│ export type Id<T> = (T) => T;
      4│
 [1]  5│ export function makeNumberId(): Id<number> {

(Note that the embedding of this code into the playground does not
reproduce the error, which is not that surprising given the context.)
First, I posit that the two blocks of declarations should be equivalent.
Second, I posit that the error is incorrect; that is, the second block
of declarations is completely valid. The result of makeNumberId is an
Id<number>. Within the nonneg.js module, number is known to be
exactly equal to NonNeg, not just a subtype or supertype of it.
Therefore, Id<number> and Id<NonNeg> should be completely
equivalent, regardless of the fact that Id is neither covariant nor
contravariant in its argument.
It is of course not true that for every type constructor F it actually
makes sense that values of F<number> and F<NonNeg> are equivalent:
for instance, subtract: (number) => number violates NonNeg
invariants. But, critically, within the module that declares NonNeg,
this substitution should be valid, because it is up to this module to
maintain the invariants.
It would be great if someone could confirm my understanding of the
situation. I’d like to write code that depends on this behavior, so I
don’t want to upgrade Flow only to find that my code no longer
typechecks for nontrivial reasons. I would really like to not have
casts through any here.
Thanks!

Perhaps a simpler workaround is to introduce an identity cast:
export const nonNegId_working: Id<NonNeg> = (makeNumberId(): Id<number>);


Update sql-server-connector-maintenance-troubleshooting.md

Adding note not to disable original key upon rotation - will stop SQL Server recovering the DB until that version is enabled once again.  @MashaMSFT  - FYI.

Learn Build status updates of commit 47cdef5:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/relational-databases/security/encryption/sql-server-connector-maintenance-troubleshooting.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

@rupp29
Can you review the proposed changes? IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

Learn Build status updates of commit c111489:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/relational-databases/security/encryption/sql-server-connector-maintenance-troubleshooting.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

#sign-off thanks @PhilB-MSFT


answer queries with no matching handler with RcodeRefused

... instead of RcodeServerFailure.

Codecov Report

Merging #1151 into master will decrease coverage by 0.00%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #1151      +/-   ##
==========================================
- Coverage   55.90%   55.89%   -0.01%
==========================================
  Files          41       41
  Lines       10134    10136       +2
==========================================
+ Hits         5665     5666       +1
- Misses       3464     3465       +1
  Partials     1005     1005

Impacted Files
Coverage Δ

serve_mux.go
72.34% <0.00%> (-3.22%)
:arrow_down:

server.go
70.71% <0.00%> (+0.27%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update da812ee...08aacde. Read the comment docs.

Agreeing to what Tom says here
On Tue, 18 Aug 2020, 14:24 Tom Thorogood<EMAIL_ADDRESS>wrote:

@tmthrgd commented on this pull request.
In server.go
https://github.com/miekg/dns/pull/1151#discussion_r472191543:

func HandleFailed(w ResponseWriter, r *Msg) {

println("dns: HandleFailed: this function is deprecated")

Informing developers that they’re using a deprecated symbol is the job of
external tooling like staticcheck, that is precisely why the Go Team
standardised on a way to mark symbols and packages deprecated. Being loud
here would just be us being obnoxious, especially when this function is
liable to be called so frequently. Hell it could conceivably lead to a DOS
depending on where stdout was going.
The reason it’s exported is that it’s exported, you’re right in that it
doesn’t really make much sense to add a new similar function, but
comparability guarantees apply to exported symbols.
 I don’t care much about it bring
deprecated, though I don’t think we gain much from that, but it shouldn’t
be noisey.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/miekg/dns/pull/1151#discussion_r472191543, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AACWIWZGACHR3FCPTOQEOL3SBJ6HLANCNFSM4QBUX42Q
.

I wanted to make sure this is the right RCODE to return and I think it probably is, but I just wanted to write down my thoughts in case it gets queried later on. I don’t think this needs to be documentation though.
RFC 1035 defines all the potentially applicable RCODES:
                2               Server failure - The name server was
                                unable to process this query due to a
                                problem with the name server.

                3               Name Error - Meaningful only for
                                responses from an authoritative name
                                server, this code signifies that the
                                domain name referenced in the query does
                                not exist.

                5               Refused - The name server refuses to
                                perform the specified operation for
                                policy reasons.  For example, a name
                                server may not wish to provide the
                                information to the particular requester,
                                or a name server may not wish to perform
                                a particular operation (e.g., zone
                                transfer) for particular data.

SRVFAIL is what we’re returning currently and it doesn’t seem like the right code. Think of it as a web server returning a 500 Internal Server Error for every unknown or unconfigured page.
NXDomain (Name Error) seems like the right choice at first glance, but the RFC seems to suggest it’s only appropriate for authoritative servers which we can’t know if we are or not from ServeMux. It doesn’t actually say you can’t though, just that it’s not meaningful. Also (from memory) NXDomain does have interactions with DNSSEC that we can’t really account for here.
Refused is what’s proposed here and it’s not a perfect fit, but the abstract definition matches well enough. You could definitely consider not configuring a handler for "." as a ‘policy reason’ and we are ‘refusing’ to answer the request.
RFC 1035 is dated and some of these might have been redefined, but it is what the IANA registry links to so :man_shrugging:.
Ultimately I think this is a good change and probably the right code to return. Thanks for the pull request and it has my LGTM.

[ Quoting<EMAIL_ADDRESS>in "Re: [miekg/dns] answer queries with..." ]

SRVFAIL is what we’re returning currently and it doesn’t seem like the right
code. Think of it as a web server returning a 500 Internal Server Error for
every unknown or unconfigured page.

yep, correct.

NXDomain (Name Error) seems like the right choice at first glance, but the RFC
seems to suggest it’s only appropriate for authoritative servers which we can’t
know if we are or not from ServeMux. It doesn’t actually say you can’t though,
just that it’s not meaningful. Also (from memory) NXDomain does have
interactions with DNSSEC that we can’t really account for here.

these rfc are all so poorly written. But yes, something like nxdomain would be nice here,
or nxzone (but this doesn't exist).
Note nxdomain is returned when you have entered a zone and a particular name doesn't
exist.
And as you mentioned this requires DNSSEC records as well, which you need to be able to
access.

Refused is what’s proposed here and it’s not a perfect fit, but the abstract
definition matches well enough. You could definitely consider not configuring a
handler for "." as a ‘policy reason’ and we are ‘refusing’ to answer the
request.
RFC 1035 is dated and some of these might have been redefined, but it is what
the IANA registry links to so 🤷♂.
Ultimately I think this is a good change and probably the right code to return.
Thanks for the pull request and it has my LGTM.

yes refused also mimics what coredns/nsd/bind already do IIRC


question can options be extended

the project works great, what would be the best way to add options such as cpushares, memory, mounting volumes from host to container ?

Volumes should currently work. Feel free to add the options as well

I'm thinking it would make sense to modify index.js so that it takes separate create and start options (opts.create and opts.start), that implies that the bulk of parsing moves from index.js to bin.js, logic like opts.fork changes Attach options would be in bin.js. I have a working fork of index.js for that, but I did not go around patching bin.js (mainly because I don't use it).
Thoughts ?

i think that sounds good. you should just open a pr with your changes and we'll discuss it there :)


how to convert prediction result tensor back to relation text

Thanks for this awesome NRE repo ! I see some relevant code in Config.py :
r = np.argmax(predict_re[i, j])
if (h_idx, t_idx, r) in label:
  top1_acc += 1

Does h_idx, t_idx  mean the start and end index of the word in document and r means the tensor for relation type ? How to convert them back to text document ? Thanks !

Hi, h_idx, t_idx are the index of the entity in the VertexSet. Each entity in theVertexSet has the corresponding textual names, which can be used for the conversion.


Update isort version for pre-commit

Merging in with CI fail, the one failing test will be fixed in #91


Request addition of more octane levels to Fuel Types

OSM Tag(s)
fuel:octane_XX where XX = 85 through 91
How would you like this tag to see supported?
As a Field
Label
Fuel Types
Aliases
No response
Terms
No response
Link to OSM Wiki page
https://wiki.openstreetmap.org/wiki/Template:Fuel_types
Status of the Tag
Unspecified / Unknown
Usage of the tag
15000
Replaces other Tag?
No
Regional Tag?
No response
Further Information
I have added new Octane levels commonly found in the western US to the list of fuel types maintained in https://wiki.openstreetmap.org/wiki/Template:Fuel_types
In my travels I have encountered octane levels ranging from 85 through 91.  Some of them are already listed in the file https://github.com/openstreetmap/id-tagging-schema/blob/7a9a9d5e391a2f08b1a74588eeb0e6edd56eec31/data/fields/fuel/fuel_multi.json
I request to add the missing octane levels 85, 88 and 90 to the list.  This will then enable a project to correctly map all gas stations in some areas with the list of octane levels that they offer to travelers.

For context, the main reason for these much lower values is that some countries (including all of North America and also Brazil) measure octane using the AKI formula rather than the RON formula. This was a point of confusion on the wiki and elsewhere until 2019.

some current usage stats

tag value
usage
listed on wiki table
currently included in field

octane_76
⣧⠀⠀⠀ 52
:x:
:x:

octane_80
⣿⣿⣇⠀ 2.9k
:heavy_check_mark:
:heavy_check_mark:

octane_83
⠀⠀⠀⠀ 12
:x:
:x:

octane_84
⣆⠀⠀⠀ 20
:x:
:x:

octane_85
⣿⣄⠀⠀ 150
:heavy_check_mark:
:x:

octane_86
⣄⠀⠀⠀ 14
:heavy_check_mark:
:x:

octane_87
⣿⣿⣧⠀ 4k
:heavy_check_mark:
:heavy_check_mark:

octane_88
⣿⠀⠀⠀ 93
:heavy_check_mark:
:x:

octane_89
⣿⣿⠀⠀ 1.1k
:heavy_check_mark:
:heavy_check_mark:

octane_90
⣿⣄⠀⠀ 170
:heavy_check_mark:
:x:

octane_91
⣿⣿⣿⠀ 9.5k
:heavy_check_mark:
:heavy_check_mark:

octane_92
⣿⣿⣿⣄ 16k
:heavy_check_mark:
:heavy_check_mark:

octane_93
⣿⣷⠀⠀ 620
:heavy_check_mark:
:x:

octane_94
⣧⠀⠀⠀ 60
:x:
:x:

octane_95
⣿⣿⣿⣿ 74k
:heavy_check_mark:
:heavy_check_mark:

octane_96
⣧⠀⠀⠀ 47
:x:
:x:

octane_97
⣿⣷⠀⠀ 630
:x:
:x:

octane_98
⣿⣿⣿⣧ 41k
:heavy_check_mark:
:heavy_check_mark:

octane_99
⣇⠀⠀⠀ 35
:x:
:x:

octane_100
⣿⣿⣷⠀ 6.7k
:heavy_check_mark:
:heavy_check_mark:

octane_101
⠀⠀⠀⠀ 9
:x:
:x:

octane_102
⣿⣆⠀⠀ 200
:x:
:x:


RFC: Twister v2 – a general rework of the testing framework

Why?
Twister was developed and evolved in a more “reactive” manner, adding new features when they were needed. In its current state Twister lacks a coherent overall architecture. A lot of parts are done in a procedural way, where a single function is responsible for too many operations (e.g. handle() method which: locks a device where a test will be run, creates a command for a runner, flashes, counts execution time, sets test statuses, unlocks the device). This makes the framework hard to maintain and expand.
Several places in the code have a similar responsibility (e.g. different report formats have their own decision trees handling test statuses) which makes maintenance very error-prone and requires high vigilance on all other parts that have to be synced when a change is made in one of them.
In addition, transparency, reliability, and traceability of test specifications/results are hard to obtain due to the current design . This is a major concern of test managers.
The current state of Twister makes many developers reluctant towards contributing to the source code, which is harmful to an open-source project. Our goal is to make the software more transparent and modularized, flattening the learning curve and making the code more approachable for potential contributors.
Having Quality Assurance at heart it seems we are at a point where a general rework of Twister is a necessary path to follow.
How?
Twister will get a grand rework. We will follow ISTQB (International Software Testing Qualifications Board) standards for test architecture design. The idea is to start with a generic Test Automation Architecture (from ISTQB test automation engineer syllabus, section 3.1) and adapt it to our needs. The core principles to follow when designing Twister v2 are:

Single responsibility of components,
Extension not modification: components open for extensions, but closed for modifications,
Replicability: components replaceable without affecting the overall behavior,
Component segregation: many specific components > several multi-purpose components
Dependency inversion: components depend on abstractions rather than on low-level details.

The new design will also enforce higher QA standards at its core. New components will be designed according to the requirements and will be developed alongside corresponding tests verifying the correctness of components’ performance.
Overall requirements:

Full backward compatibility is not strictly required but some parts must be adapted to use the existing resources:

Compatible with the existing tests (however individual tests might need to be improved to fulfill new standards)
Maybe we will be able to leave most of the existing CLI args?

Twister operation will be divided into 3 major and separate stages: test plan (specification), test session (execution), and test results (reporting).

One must be able to run each block individually and be able to start the workflow from any point between them (e.g. start with an already existing test plan, or run just the results stage to create a different format of a report for the already existing results.
The above will be achieved by enforcing test data (configurations) serialization.

Reliability will be enforced by the process where results (after test execution) are matched against the specification and 1:1 relation is required (no configurations in the specification without results, no results without matching entry in the specification)
Traceability of tests (being able to easily match a certain test with a corresponding software component) has to be improved (TBD, e.g. improved hierarchy of tests and some verification process)
Functional tests have to be delivered and fulfilled for each component before it is added to the framework
Be able to conduct complex test scenarios, involving usage of multiple devices and multiple images (e.g. bootloader tests where a bootloader, an image, and its upgrade are consecutively flashed on a single device and the output is verified after each step; Bluetooth tests with multiple devices or tests requiring more than just flashing an image, like resetting a device or sending some data to the device)
Coding standards will be enforced (to define which ones and how)

The above requirements were gathered during testing wg meetings and discussions with colleagues involved in testing (at various levels). Just to stress this out, nothing is set in stone. I would really appreciate all the suggestions and/or additional requirements.
The process for developing Twister v2 will be transparent and contributors will be encourage to take part in it. I volunteer to create and maintain a GitHub "project" for this process. I will split the framework into major modules (e.g. test configuration, reporting, execution etc.) and gather/provide a separate high-level description and requirements for each of them. The idea is to be able to implement them in parallel. I also hope that having modules, components and corresponding tasks divided into smaller chunks and easily visible on a kanban-style table will attract contributors to fulfilling them. I would also like to try using good first issue and help wanted labels on easier and smaller tasks.
PoC (pytest)
Goal
Show the concept behind pytest and its major benefits. Show that the framework can be compatible with zephyr and its tests.
Requirements:
Show how plugins/fixtures/hooks works and how a framework can be built from blocks -> benefits from standardized apis and specs:

Show that we can reproduce already existing tests based on twister v1 workflow, but where the current methods are divided into smaller blocks according to pytest schema. Can start with the existing code test runner but „wrapped” in pytest
Show that test specification, test execution, test results blocks can be called individually
Show how test identification can be handled reusing the existing schemas or some other simple yaml based parser -> test configuration (suite) level. Weak req: show test case level (parsing C code).
Show that both qemu and hardware tests are working and providing results
Show simplified example of report generation at test configuration (suite) level. Weak: also at test case level.
Show example of test coverage for a framework component (e.g. some simplistic new test and how the code coverage is calculated for a specific component)

ETA:
Hard to estimate yet.
I will keep editing this entry, by adding more graphs, requirements, and details.

Random thoughts: we could consider re-using pytest ecosystem, see for example https://github.com/pytest-dev/pytest-cpp (not tested, though)

After having several discussions I started to be convinced that pytest might be a good framework to wrap twister around. I've heard several times: "Why do you want to reinvent the wheel and not to take an existing framework, where your requirements are already supported, and which is in a common usage? Why not pytest?". From my initial prestudy, using pytest looks really tempting. We will collect and present more information about it and its benefits. We will also present a Proof Of Concept for the pytest-based twister. Below is my idea for the requirements for the PoC. I also added them to the RFC text. Of course, they are a subject to be discussed:
PoC
Goal
Show the concept behind pytest and its major benefits. Show that the framework can be compatible with zephyr and its tests.
Requirements:
Show how plugins/fixtures/hooks works and how a framework can be built from blocks -> benefits from standardized apis and specs:

Show that we can reproduce already existing tests based on twister v1 workflow, but where the current methods are divided into smaller blocks according to pytest schema. Can start with the existing code test runner but „wrapped” in pytest
Show that test specification, test execution, test results blocks can be called individually
Show how test identification can be handled reusing the existing schemas or some other simple yaml based parser -> test configuration (suite) level. Weak req: show test case level (parsing C code).
Show that both qemu and hardware tests are working and providing results
Show simplified example of report generation at test configuration (suite) level. Weak: also at test case level.
Show example of test coverage for a framework component (e.g. some simplistic new test and how the code coverage is calculated for a specific component)

I just wanted to chime in and mention IDE support. VS Code introduced a test runner API a couple of months ago that allows listing and running tests through a UI in the sidebar. Microsoft's Python extension already supports this API, and will populate the test list with tests discovered using pytest: https://code.visualstudio.com/docs/python/testing. They're also working on a similar API for code coverage, but that's still not in the public release.
If pytest is a viable solution, Twister will essentially get a VS Code UI for free. I'm not sure how debugging would work for Twister tests, but it's likely possible to implement this through Zephyr side python code.
I haven't tried it myself, but CLion also appears to support this.
Regardless, we'd like to create a Twister user experience with Nordic's VS Code extensions - either as a new standalone Twister extension, or as part of the main extension. For this to work out, we'll need to be able to:

List test scenarios in the workspace in a machine readable manner
Run test scenarios through an API-locked command line interface ("porcelain")
Read out test outcomes through some result files, preferably in a common, existing format, like junit
Have a clear set of permutations of board types and toolchains for each test case, so we can condense the possible test cases to a single button, then be able to discover what permutations are available for each test suite. This could potentially be some sort of runner profile the user sets up (e.g. "all tests available on QEMU_x86"). Listing, running and discovering test cases matching these profiles would need a clear mechanism. Some mechanism for listing potential mutations, or the limitations of each test case would also be very helpful. Currently, it can be hard to discover how you can even run a test case, with all the toolchain, board and tag parameters.

I'd like to add a minor point about junit.
Currently,  the generated xml is displayed like this in Jenkins:

I suggest that for Twister v2, the output xml should be adjusted such that tests/samples in Jenkins ends up being browsable in a similar layout to the actual filesystem layout.
So the above image (which shows samples in sdk-nrf/samples/crypto/*) would be something more close to
sdk-nrf/
    samples/
        crypto/
            aes_cbc/
                nrf52840dk_nrf52840/
                <other boards>/
            aes_ccm/
                nrf52840dk_nrf52840/
                <other boards>/
    tests/
        ...
zephyr/
    samples/
        ...
    tests/
        ...

This will make it less noisy in the UI and more obvious where to locate tests.
I believe this should be as simple as rearranging the components of the test names.

Be able to conduct complex test scenarios, involving usage of multiple devices and multiple images (e.g. bootloader tests where a bootloader, an image, and its upgrade are consecutively flashed on a single device and the output is verified after each step; Bluetooth tests with multiple devices or tests requiring more than just flashing an image, like resetting a device or sending some data to the device)

For this, at least from a mcumgr standpoint, it's vital that there is also support for being able to use and interact with an external utility (likely python based, but any type of external script/application would be preferred) so that it can send/receive commands to/from real devices (qemu devices could also be considered) and to also use the return code of the process when exiting to influence if a test is successful or not.

@lairdjm Could you rephrase this a bit? Not sure if I follow. What would be a use case? I understand it now as to be able to use twister as a middleman between external unity and hardware/emulator. Is it the case?

So for testing mcumgr, it will have the code on the module and it will need to communicate with an external utility either via serial, bluetooth, UDP or another transport method and as part of that the test will need to set the tool up and run it. Example would be for getting a file from the filesystem of a device, zephyr application gets loaded to the target board and runs, an application/script runs on the test PC which then communicates with the device to download the file and checks the commands work and that the received file is what was expected.

Similar as to what you can do using the pytest twister harness today?

For this, at least from a mcumgr standpoint, it's vital that there is also support for being able to use and interact with an external utility (likely python based, but any type of external script/application would be preferred)

for example this: https://github.com/zephyrproject-rtos/zephyr/issues/10112 and this https://github.com/zephyrproject-rtos/zephyr/issues/5325

@PerMac  is there an update on pytest PoC? We need to decide very soon if this is going to be the way to go or if we have to implement things differently.

The status is that me and my colleague will work on a prototype for the pytest-based runner. Some initial work was already done, but it is too early to show anything meaningful yet. We had to get internally a green light for the commitment and its scope and it was granted recently.
I started calling it a prototype after seeing that there are some differences between PoC and prototype, as PoC tends to focus more on a theoretical side. I think now we believe that pytest can be used for embedded, based on e.g. espressif approach https://github.com/espressif/pytest-embedded. So now it is more about showing how we can achieve what we want and need.
I am working on a more detailed plan and schedule for the prototype. It will be divided into major components (e.g. test identification, execution, etc) and into phases. I will define what functionality is needed for all components at any stage. We will implement features incrementally. E.g. During phase one, test execution will be handled by reading a static txt file with a mock output. Reading hw/qemu output will be added in phase 2 and phase 3 will add parallelization.
I plan to finish such proposal before our next wg meeting, so we can discuss the approach during the meeting.

can we close this please?

This RFC is closed since twister v2 won't be introduced. The strategy to improve the existing framework instead was adapted. Working on v2 brought an understanding on how to integrate pytest with twister allowing introduction of more complex tests, which were postulated in this PR. Such tests are now possibile to be executed within the current framework.  https://docs.zephyrproject.org/latest/develop/test/pytest.html.
More context:
We managed to get quite far with v2, i.e. twister rewritten to work as a plugin to pytest. We were able to execute the majority of existing tests on qemu, native posix, and hardware. However, we realized that to get the same performance and user experience pytest will require a lot of bending, furtherly complicating the whole framework. As an example, we found a major issue with memory consumption when working with parallel test execution. The issue is that xdist i.e. pytest library responsible for test parallelization must do the process of creating test instances for every spawned worker. Zephyr test base is huge and there are ~600 platforms. Executing "all on all" creates ~1000000 test instances, resulting in ~1GB ram consumption. In pytest, this number has to be multiplied by the number of parallel workers, ending in crushing the operation.  Current framework collects all the test instances in advance and only then parallelize the execution with custom scheduler. Some workarounds could be adapted to mitigate this issue in pytest, however it would continue to make the framework more and more complex and moving further from the standard pytest behavior.
We decided to descope the project. We abandon the idea of reimplementing what is already working into pytest. We focused on exporting code we developed for new type of tests  and integrating it with the existing twister. This resulted in pytest plugin which allows to execute tests written in pytest, where interactions with devices are possible. In the current workflow, whenever it is detected a test requires pytest harness, twister calls pytest as a subprocess, passing all the required information, and then parses results from pytest.


Swagger folder missing

Not sure if this is an issue or not, but the swagger folder is missing from the API project.
When you try to navigate to the URL http://localhost:51864/swagger/
You get the following error:
HTTP Error 404.0 - Not Found

There's no swagger folder; swagger output is generated by Swashbuckle.  For a more current sample, see the tutorial https://azure.microsoft.com/en-us/documentation/articles/app-service-api-dotnet-get-started/

Thanks for the reply. The line that needs to be uncommented is actually commented on SwaggerConfig.cs. Shouldn't it be uncommented by default on this sample project?

It's commented by default when you create an app using the API App template in Visual Studio.  It was left that way here because the sample app was used with a tutorial and the tutorial showed how to enable the Swagger UI.  In general the thinking is that it's better to have it off by default so people who aren't familiar with it don't accidentally leave the Swagger UI open on their API.

Understood. Thanks a lot for the quick reply.


Incorrect primitive_type in generated code causes "Encountered an unknown type during deserialization str"

What are you generating using Kiota, clients or plugins?
API Client/SDK
In what context or format are you using Kiota?
Linux executable
Client library/SDK language
Python
Describe the bug
When I use the versions below of Kiota the following error occurs:
Exception: Encountered an unknown type during deserialization str
I think that the bug is due to send_collection_of_primitive_async passing argument primitive_type as str instead of <class type[str]> and located here: https://github.com/microsoft/kiota/blob/main/src/Kiota.Builder/Writers/Python/CodeMethodWriter.cs#L808
"microsoft-kiota-abstractions==1.3.3",
"microsoft-kiota-http==1.3.3",
"microsoft-kiota-serialization-json==1.3.2",
"microsoft-kiota-serialization-text==1.0.0",
"microsoft-kiota-serialization-form==0.1.1",
"microsoft-kiota-serialization-multipart==0.1.0",
Expected behavior
The get method should return a list of strings (names).
How to reproduce
Have Kiota generate a python code for an endpoint that returns a JSON collection of string (python equivalent: List[str]).
Open API description file
        "/names": {
            "get": {
                "summary": "Get names",
                "operationId": "getNames",
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "uniqueItems": true,
                                    "type": "array",
                                    "items": {
                                        "type": "string"
                                    }
                                }
                            }
                        }
                    },

Kiota Version
1.18.0+5c6b5d0ef23865ba2f9d9f0b9fe4b944cf26b1ec
Latest Kiota version known to work for scenario above?(Not required)
No response
Known Workarounds
Manually edited the generated code and replace "str" with str.
Configuration

OS: Ubuntu

Can't see that it is specific to this configuration.
Debug output
Other information
The generated get method for the endpoint seems to be incorrect ("str" instead of str):
return await self.request_adapter.send_collection_of_primitive_async(request_info, "str", None)
Which will cause the  if generic_type in primitive_types: in JsonParseNode.get_collection_of_primitive_values  to be False
since  "str"  is NOT in primitive_types due it being an actual string (and not str, i.e. a ).
Example:
>>> types = {bool, str, int, float, bytes}
>>> type(types)
<class 'set'>
>>> str in types
True
>>> 'str' in types
False

    def get_collection_of_primitive_values(self, primitive_type: Any) -> Optional[List[T]]:
        """Gets the collection of primitive values of the node
        Returns:
            List[T]: The collection of primitive values
        """

        primitive_types = {bool, str, int, float, UUID, datetime, timedelta, date, time, bytes}

        def func(item):
            generic_type = primitive_type if primitive_type else type(item)
            current_parse_node = self._create_new_node(item)
            if generic_type in primitive_types:
                method = getattr(current_parse_node, f'get_{generic_type.__name__.lower()}_value')
                return method()
            raise Exception(f"Encountered an unknown type during deserialization {generic_type}")

        if isinstance(self._json_node, str):
            return list(map(func, json.loads(self._json_node)))
        return list(map(func, list(self._json_node)))

Hi @jimisola
Thank you for using kiota and for reaching out.
Looking at other examples your analysis seems to be correct.
Is this something you'd like to submit a pull request for provided some guidance?


Move string interning to util

Code that handles string interning is rather an util functionality than
a core functionality.

r? @ehuss
(rust_highfive has picked a reviewer for you, use r? to override)

I have been seeing advice lately not to name a mod "misc" because "in a year it will be completely unclear what belongs in it". Thank you @est31 for starting to talk about this long standing teck det, What belongs in "util"? The status quo is not really ok. I know Eric had opinions, so I am glad he is the reviewer.
My prior belief is that we should be reorganizing to not have a "util" but to have things is mods that describe what they do. On the other hand if this helps untangle our mods, "don't let the perfect be the enemy of the good", we can move interning if it is helpful.

IMO interning fits to be next to mods like canonical_url or queue... basically self contained generic helper code not containing any logic that's too specific to cargo, but still too tailored to cargo's needs/too small/too undocumented to become its own indepedent crates.io crate.
Of course there's plenty of stuff in util that contains very cargo-specific stuff, like the config and toml modules. IMO those two should be put into a separate newly created module called defs which would collect most pure datatypes that just need serde and nothing much else. The command prelude can be put into the bin directory. And suddenly most dependencies of util on the core module are gone, and util can be put into its own crate. The defs module then can be crate-ified as well, and depend on util. The main cargo crate would depend on all of them and be a bit leaner. I need to experiment how this affects compile time. Hopefully serde-using stuff being moved out has a large effect.

One thing I would eventually like to reach is to remove the cyclical dependency with cargo-test-support.  I sorta envisioned something like a cargo-util crate that would contain whatever is necessary to make that happen. This helps move towards that direction, since InternedString is used all over.
I'm :+1: for doing this.  @alexcrichton / @Eh2406 are you OK with this?

I am convinced by @est31's vition for how to get untangled. So I am 👍.

I don't have too many preferences about Cargo's internal organization, we've never really put too much effort into organizing it any particular way. I don't really think it's worth it to try to micro-optimize one way or another because we don't really have an overall vision for the crate.
I'm not really against nor in favor of this, it's just moving code around which is fine by me so long as we don't have a preexisting style saying the code shouldn't be moved (and we don't have a such a preexisting style in this case)

OK, I think this is fine for now. I think it may help with creating separate crates in the future.
@bors r+

:pushpin: Commit 836e91ce4c1b38b2f7bff8ebcc38b119edea3022 has been approved by ehuss

:hourglass: Testing commit 836e91ce4c1b38b2f7bff8ebcc38b119edea3022 with merge f12d72e11b8643b4851a50325d77af962931fe98...

:sunny: Test successful - checks-azure
Approved by: ehuss
Pushing f12d72e11b8643b4851a50325d77af962931fe98 to master...


Jshadbolt basic interop

This will add a fairly basic react page with a file upload section. The server will receive that file and pritn out the parsed contents to the CLI

Milestone 1 main branch code commit


'Interesting' item selection is wrong

Current approach is just to take the lowest indexed BS weapon.
New approach:

Have list of important secondary weapons (panzer, A-Kanone, chain colt, etc).
for lowest indexed option, take the 1st BS weapon and anything on the secondary list.
for all other options, if they have the same weapon load out as the lowest option, then format as per the lowest, otherwise take the 1st BS weapon the lowest option lacks and any important secondaries.

Is this functionally different from the simpler:
take the 1st BS weapon and anything from the secondary list?


Use Python 3.8 in notebook image

This PR ensures coiled/notebook uses Python 3.8 by default as Dask doesn't yet support Python 3.9. We've also observed some conda environment solve conflicts in our notebook CI builds which this may help resolve.

I tried this out locally and still ran into some conda solve issues, so we may also need to revert JLab 3 for the time being.
Longer term, I think that updating one conda spec with another fully solved conda spec (as we are doing in the backend) is very brittle, and we may need to change it.

Okay, thanks for trying that out locally @ian-r-rose! The changes here still seem valid given the current versions of Python that Dask supports. Let's go ahead and merge this and then switch back to JLab < 3 in a a separate PR. Are you up for submitting that one?

Yes, I agree that this is still correct.
One thing I saw in my poking at this: the initial solve can also now spit out python 3.9 (perhaps some default changed in the conda repodata?), which can be a real problem when resolving differences, even after this is merged.

the initial solve can also now spit out python 3.9

Hmm my first guess is something like the python requirement in the coiled conda-forge recipe could lead to that sort of thing

the initial solve can also now spit out python 3.9

Hmm my first guess is something like the python requirement in the coiled conda-forge recipe could lead to that sort of thing

That may be helpful, but I don't think will solve the underlying issue. e.g., if I get a spec from the backend right now:
from software_environments.core import solve_conda_spec
spec = solve_conda_spec({"channels": ["conda-forge"], "dependencies": ["botocore"]}, "linux-64")
spec["dependencies"]

produces
['_libgcc_mutex=0.1',
 '_openmp_mutex=4.5',
 'botocore=1.20.11',
 'brotlipy=0.7.0',
 'ca-certificates=2020.12.5',
 'certifi=2020.12.5',
 'cffi=1.14.5',
 'cryptography=3.4.4',
 'idna=3.1',
 'jmespath=0.10.0',
 'ld_impl_linux-64=2.35.1',
 'libffi=3.3',
 'libgcc-ng=9.3.0',
 'libgomp=9.3.0',
 'libstdcxx-ng=9.3.0',
 'ncurses=6.2',
 'openssl=1.1.1j',
 'pip=21.0.1',
 'pycparser=2.20',
 'pyopenssl=20.0.1',
 'pysocks=1.7.1',
 'python-dateutil=2.8.1',
 'python=3.9.1',
 'python_abi=3.9',
 'readline=8.0',
 'setuptools=49.6.0',
 'six=1.15.0',
 'sqlite=3.34.0',
 'tk=8.6.10',
 'tzdata=2021a',
 'urllib3=1.26.3',
 'wheel=0.36.2',
 'xz=5.2.5',
 'zlib=1.2.11']

Beyond getting python 3.9 (naturally, nothing is declaring that incompatible), this is a fully solved spec which will be much harder to merge with the underlying environment.

(to be clear, this is a bigger conversation, and I think we should just do the pins/reversions for the moment)

Yep, totally agree with you 👍


fix stylesheet types

'fontStylestylesheetUrl' is deprecated.ts(6385) client.d.ts(42, 6): The declaration was marked as deprecated here.
-  import fontStylestylesheetUrl from './styles/font.css'
-  import tailwindStylesheetUrl from './styles/tailwind.css'
+  import fontStylestylesheetUrl from './styles/font.css?inline'
+  import tailwindStylesheetUrl from './styles/tailwind.css?inline'

why we need this duplication in the same function?
https://github.com/epicweb-dev/epic-stack/blob/f21bbe93ce26b3ff391874a102e8a7c83319be11/app/root.tsx#L40-L43
https://github.com/epicweb-dev/epic-stack/blob/f21bbe93ce26b3ff391874a102e8a7c83319be11/app/root.tsx#L63-L65

It's a bit verbose, but that line solves this particular warning in Lighthouse:

Details on preloading CSS: https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/rel/preload
This lets the main thread to continue rendering the HTML while the css streams in. Otherwise, html rendering is on hold until all CSS comes in.