InferenceService Status restructuring

Currently, ModelMesh has its own set of status fields that it populates in the InferenceService status since we allow unknown fields there. Similarly, KServe has its own status fields that are predominantly based on knative fields.
The goal here is to converge on a common set of status fields that would work with the various different deployment modes.
As an overview, here are the current status fields:
InferenceServiceStatus
InferenceServiceStatus
  ├──── duckv1.Status `json:",inline"`
  │        │    // The 'Generation' of the Service that was last processed by the controller.
  │        ├─── ObservedGeneration int64 `json:"observedGeneration,omitempty"`
  │        │
  │        │    // Conditions the latest available observations of a resource's current state.
  │        |    // Condition defines a readiness condition for a Knative resource.
  │        ├─── Conditions Conditions `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"`
  │        |
  │        |     // Additional Status fields for the Resource to save some additional State as well as convey more information to the user.
  │        ├─── Annotations map[string]string `json:"annotations,omitempty"`
  │
  ├──── Address *duckv1.Addressable `json:"address,omitempty"`
  ├──── URL *apis.URL `json:"url,omitempty"`
  ├──── Components map[ComponentType]ComponentStatusSpec `json:"components,omitempty"`

ComponentTypes are predictor, explainer, transformer. Which map to a ComponentStatusSpec.
An example of an actual status can be found here: https://pastebin.com/wkCrZyxk
ModelMesh Predictor Status
PredictorStatus
  ├──── Available bool `json:"available"`
  │
  │     // One of 'UpToDate', 'InProgress', 'BlockedByFailedLoad', or 'InvalidSpec'
  ├──── TransitionStatus TransitionStatus `json:"transitionStatus"
  │
  │     // High level state string: Pending, Standby, Loading, Loaded, FailedToLoad
  ├──── ActiveModelState ModelState `json:"activeModelState"
  ├──── TargetModelState ModelState `json:"targetModelState"`
  │
  │     // Details of last failure, when load of target model is failed or blocked
  ├──── LastFailureInfo *FailureInfo `json:"lastFailureInfo,omitempty"`
  │
  │     // Addressable endpoint for the deployed trained model. This will be "static" and will not change when the model is mutated
  ├──── HTTPEndpoint string `json:"httpEndpoint"`
  ├──── GrpcEndpoint string `json:"grpcEndpoint"`
  │
  │     // How many copies of this predictor's models failed to load recently
  ├──── FailedCopies int `json:"failedCopies"`

An example of an actual predictor status can be found here:  https://pastebin.com/wBM3WgFW

Action item: Decide how we can restructure the fields in a way that make sense for all the deployment modes. Do we still rely on the Knative types?

FYI @yuzisun , @njhill , @chinhuang007

My proposal is to collapse the model related fields from ModelMesh's PredictorStatus into a new struct ModelStatus on InferenceService status. On InferenceService there isn't any status field about the model, when storage initializer fails to download or model server fails to load the model, we just say that service is not ready which could be due to many other reasons, so I think it is good to start to populate the model status fields for single model serving case.
HTTPEndpoint and gRPCEndpoint can be merged onto the Predictor's ComponentStatusSpec url fields, available I think it is basically the predictor ready condition, WDYT?
ModelStatus
  │     // One of 'UpToDate', 'InProgress', 'BlockedByFailedLoad', or 'InvalidSpec'
  ├──── TransitionStatus TransitionStatus `json:"transitionStatus"
  │
  │     // High level state string: Pending, Standby, Loading, Loaded, FailedToLoad
  ├──── ActiveModelState ModelState `json:"activeModelState"
  ├──── TargetModelState ModelState `json:"targetModelState"`
  │
  │     // Details of last failure, when load of target model is failed or blocked
  ├──── LastFailureInfo *FailureInfo `json:"lastFailureInfo,omitempty"`
  │
  │     // How many copies of this predictor's models failed to load recently
  ├──── FailedCopies int `json:"failedCopies"`

Components map[ComponentType]ComponentStatusSpec `json:"components,omitempty"`
duckv1.Status `json:",inline"`
URL *apis.URL `json:"url,omitempty"`

I think that's seems pretty tenable. So it looks like we maintain the use of Conditions where ModelMesh will populate the necessary Predictor types (i.e. PredictorRouteReady, etc.) ?
Also, do we omit the Address field that was in the original Status?

Feels like ModelStatus is additional to, not a replacement of, ComponentStatusSpec. So that seems to suggest the predictor component and its status should be populated and maintained regardless it is single model or ModelMesh. Please help clarify if my thinking is incorrect.

Feels like ModelStatus is additional to, not a replacement of, ComponentStatusSpec. So that seems to suggest the predictor component and its status should be populated and maintained regardless it is single model or ModelMesh. Please help clarify if my thinking is incorrect.

ye we can add the ModelStatus in addition to the component status, we don't need to populate every single field on component status and most of the fields are optional, ModelMesh can set the PredictorReady condition and the url related fields.

/cc @Suresh-Nakkeran

Thinking more, does it make sense to have ModelStatus contents be a part of the predictor ComponentSpec instead of being top level since these are fields related to the Predictor?
I know currently a lot of the fields in the ComponentSpec are Knative specific with things like Revisions and TrafficTarget, but we could potentially mesh the two sets of status fields. ModelMesh does kinda have the concept of revisions with internal modelIds that change if a user does an update. So these could potentially be mapped to some of the revision fields.
In any case, we could just not populate some of the fields that might not be useful for ModelMesh-deployed predictors and the omitempty in the struct tags would ensure these aren't shown to the user.
Example Modelmesh-deployed InferenceService status:
Components:
    Predictor:
      Latest Created Revision:    sample-mnist__isvc-b52e1c8a05
      Latest Ready Revision:      sample-mnist__isvc-b52e1c8a05
      Latest Rolledout Revision:  sample-mnist__isvc-b52e1c8a05

      Transition Status:  UpToDate
      Active Model State: Loaded
      Failed Copies: 0

      HTTP URL: http://modelmesh-serving.modelmesh-serving:8008
      Grpc URL: grpc://modelmesh-serving.modelmesh-serving:8033
    Transformer:
      Address:
        URL:                      http://sample-mnist-transformer-default.modelmesh-serving.svc.cluster.local
    ...
Conditions:
  Predictor:
    Type:                  PredictorReady
    Last Transition Time:  2022-02-22T04:46:34Z
    Severity:              Info
    Status:                True
    Type:                  Ready
    Last Transition Time:  2022-02-22T04:47:08Z
    Severity:              Info
    Status:                True
    ...

LastFailureInfo for ModelMesh can also be put in Conditions similar to how single-model serving errors are propagated to the user through something like the PredictorReady condition type.
Throwing this out there as an option.


[WIP] Fix #782 - Fix overlapping errors

WIP for #782

Here's what the updated UI looks like:

Basically integrated into the QuickInfo pane

Looks great!


Move model check to splash message

Pull Request Checklist

[x] Documentation has been updated, or this change doesn't require that

no bot review?

no bot review?

No, I had mentat installed on the server with pip install -e . so as I developed this pr the server crashed.


Cite this document panel

@ctschroeder will want to add more static/universal content to the "Cite this document" panel.  Prose content.  Exact content ctschroeder will add will depend on whether the document level metadata is exposed or hidden (see https://github.com/CopticScriptorium/cts/issues/18).

So @lukehollis at the end of the process will need to point me where to go.  This is NOT urgent; for the very end.

Okay, that sounds great!  The URNs/Cite this data starts here: https://github.com/CopticScriptorium/cts/blob/master/coptic/templates/index.html#L86  Let me know what else I can help with here.

Also panel needs to say that the urns resolve to the latest edition of the data.  Citations by scholars should include date accessed in their fns; if you're looking for earlier data, then go to GitHub for archives

The Cite this Document panel has been updated to reflect the text from your ticket here.  If you'd like something verbatim, send it my way, and I'll apply it to the view.  Otherwise, as the earlier ticket mentions, the text is here: https://github.com/CopticScriptorium/cts/blob/master/coptic/templates/index.html#L86

leaving open for now; @ctschroeder needs to make revisions to this section

leaving open for now; @ctschroeder needs to make revisions to this section

done.


Git diff view does not show untracked nested repositories, but will commit them

Issue Type: Bug
VS Code does not show untracked nested Git repositories it in the "Source Control" panel's diff view, but will add them to a commit using the "Commit" button, or stage them using the "Stage All Changes" button.
In my case I wanted to add an external repository into my yarn workspace for development, but added it to my .gitignore incorrectly, which made me accidentally commit it because it did not show up in VS Code's diff view.
Steps to reproduce:

Clone a Git repository and open it in VS Code
Optional: Make changes to a file
Clone a nested repository into a subfolder

Expected result:

The "Source Control" view shows both the edited file and the untracked nested repository

Actual result:

Only the changes to the edited file are displayed in the "Source Control" view

When committing all changes by pressing the "Commit" button at the top of the "Source Control" view, the nested repository is committed as well

> git diff HEAD^1

...

diff --git a/packages/test b/packages/test
new file mode 160000
index 000000000..edcb146c9
--- /dev/null
+++ b/packages/test
@@ -0,0 +1 @@
+Subproject commit edcb146c9d9904038f83a7020a802cd6b8212d3b

Only after the nested repository is committed, The "Source Control" view starts displaying changes made to it

Side Notes:

The nested repository is displayed in the "Staged Changes" diff view when the "Stage All Changes" button is clicked or it is added via git add through the shell. This bug only seems to affect nested repositories that are untracked.
The Git command line warns about adding nested repositories that are not proper submodules. Maybe VS Code should add a warning in these cases as well?

> git add packages/test

warning: adding embedded git repository: packages/test
hint: You've added another git repository inside your current repository.
hint: Clones of the outer repository will not contain the contents of
hint: the embedded repository and will not know how to obtain it.
hint: If you meant to add a submodule, use:
hint:
hint:   git submodule add <url> packages/test
hint:
hint: If you added this path by mistake, you can remove it from the
hint: index with:
hint:
hint:   git rm --cached packages/test
hint:
hint: See "git help submodule" for more information.

Related: https://github.com/Microsoft/vscode/issues/37947
VS Code version: Code 1.32.3 (a3db5be9b5c6ba46bb7555ec5d60178ecc2eaae4, 2019-03-14T22:46:48.547Z)
OS version: Darwin x64 18.5.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz (8 x 2300)

GPU Status
2d_canvas: enabledchecker_imaging: disabled_offflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: enabledrasterization: enabledsurface_synchronization: enabled_onvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 2

Memory (System)
16.00GB (4.18GB free)

Process Argv
-psn_0_77843

Screen Reader
no

VM
0%

[x] Tried with extensions disabled

(not a duplicate as this issue is not about showing the nested repository as a SCM provider, but rather the diff it causes to the parent repository)

#37947


CI: Introduce commands workflow to run on Linux, Mac and Windows

What this PR does / why we need it:
Introduces a new "commands" workflow to validate the commands in the bundled packages work on all flavours of OS. This should help us catch issues with cross operating system compatibility without the need for firing up virtual machines locally.
It also bumps the versions of the actions used as per #114

I'm closing this for now. Until we find a way around bypassing conditional prompts I can't see a way forward for this.


Can't export to CONLL using valueType="url" settings (NER Annotation)

Describe the bug
I'm not sure if this is really a bug, or it is the expected behavior of the label-studio. I imported several text files in a row using the time series option with  -valueType="url"- set. The task I'm performing is named entity recognition (NER) in word level with custom tags. When I need to export the annotations, the CONLL file generated has the URLs of the imported files instead of words with each tag annotated.
PS: I tried also the saveTextResult="yes" option, but it didn't provide the expected result.
To Reproduce
Steps to reproduce the behavior:

Import some text files using the valueType="url" option as time series.
Annotate some data.
Click on export selecting the CONLL2003 option.

Hi @KonstantinKorotaev,
Thanks for your answer. I deleted the option valueType="url" but the old behavior persists. I already imported and annotated the data as time series. Does delete this tag, after all, impact in something?

Hi @guilhermenoronha
You will need to reimport all tasks with texts.
For example you can do it this way:

Get back valueType="url" to your config
Export your annotation as json
Download texts with script and save them to new json:

    with open(export_filename, mode='r') as f:
        data = json.load(f)

    for each in data:
        url = each['data'][list(each['data'].keys())[0]]
        r = requests.get(url)
        each['data'][list(each['data'].keys())[0]] = r.text

    with open(new_filename, mode='w') as f:
        json.dump(data, f)

Import new file to the project without valueType="url" (you can create a copy of your project)

If you have complicated config - replace cycle intructions

Hi @KonstantinKorotaev  . Thanks again. I took some time to reply because the code you provided needed some adjustments to my project. It worked smoothly, thanks! I'm going to post the fully functional code here because I believe it may be helpful to someone else.
import json
import requests

with open('your_json_file.json', mode='r', encoding='UTF-8') as f:
        data = json.load(f)

for each in data:
    # You may have to change your url depending on where is hosted.
    url = f"http://localhost:8080{each['data'][list(each['data'].keys())[0]]}"
    # You can Find your token clicking at upper right panel button in Label-Studio and Account & Settings.
    r = requests.get(url, headers={'Authorization':'Token put_your_token_here', })
    # My text is in Portuguese, so I had to change the encoding to UTF-8.
    r.encoding = 'UTF-8'
    each['data'][list(each['data'].keys())[0]] = r.text

with open('conll.json', mode='w', encoding='UTF-8') as f:
    json.dump(data, f)

Although your solution works well and fits to my problem, I found it a bit clumsy to execute. Expecting a common user who needs to export several tasks in CONLL format, it doesn't seem very natural step to be executed in a production environment, right? Said that, don't you believe it would be best to add as feature request to export  TimeSeries texts as CONLL format?
Sincerely.


tablekeyparameter: fix short name reference resolution

unfortunately, the Table and TableRow classes are only available during type checking (i.e., not during normal execution) because this would lead to circular imports.
this should fix https://github.com/mercedes-benz/odxtools/issues/324.
Andreas Lauser<EMAIL_ADDRESS>on behalf of MBition GmbH.
Provider Information

Not possible to just remove if TYPE_CHECKING: in line 17 ?

unfortunately not. this leads to cylic imports...

@lz4v4: have you tested this?

Do the import inside _resolve_snrefs, that way it's lazy and does not cause circular reference

Do the import inside _resolve_snrefs, that way it's lazy and does not cause circular reference

I'd prefer not to since I'm unsure about the performance implications. (and IMO it is very inelegant as well...)
anyway, since @lz4v4 confirmed that the patch works, let's merge it for now and maybe switch to another solution later.


feat: Delete old pending FileSkeletons

Over time, pending FileSkeletons (datastore entities) can accumulate when upload urls have been requested but nothing has been uploaded. Therefore this adds the PeriodicTask start_delete_pending_files which deletes pending files older than 90 days.
To do the deletion safely on skeleton level a customization of the DeleteEntitiesIter was necessary. I see no reason not to use this for skeletons as long as you call the appropriate .delete() method.

Why 90 day a Uploadurl is expired after 1 hour (https://github.com/viur-framework/viur-core/blob/a54556277b9e719b50863a2d20771ed8375058de/core/modules/file.py#L467)  .
Can we say 1 day or so ?

Why 90 day a Uploadurl is expired after 1 hour

But only for signed URLs from this method (signUploadURL), which isn't used anywhere in the core.
URLs created by getUploadURL seems to be unlimited?!
Tbh, the 90 doesn't have a specific reason. It should be a time that is enough to process an upload, but not unlimited.
I can't imagine a case where you need a validity of several days. For user uploads, you would actually always pick up the URL immediately before the upload.
So for me, 1 day would also be okay. What do the others (/ping @viur-framework/developers and especially @phorward) think?

In the viur meeting, we decided to set this value to 7 days.


.

Can someone please suggest what am i doing wrong and how to resolve this? @zezha-msft
I have this output :
Out[10]: b'\xef\xbb\xbfInvalidHeaderValueThe value for one of the HTTP headers is not in the correct format.\nRequestId:8fe1135e-f01e-002f-2ff4-e82696000000\nTime:2021-01-12T15:06:52.6849689ZContent-Length0'
Python code for the request :
from msrestazure.azure_active_directory import AADTokenCredentials
Other required imports
import adal, uuid, time
authority_host_uri = 'https://login.microsoftonline.com'
tenant = 'XXX'
authority_uri = authority_host_uri + '/' + tenant
resource_uri = 'https://storage.azure.com/'
client_id = 'XXX'
client_secret = 'XXX'
context = adal.AuthenticationContext(authority_uri, api_version=None)
mgmt_token = context.acquire_token_with_client_credentials(resource_uri, client_id, client_secret)
credentials = AADTokenCredentials(mgmt_token, client_id)
token = mgmt_token.get('accessToken')
r = requests.post('https://abc.blob.core.windows.net/?restype=service&comp=userdelegationkey', headers={'Authorization' : 'Bearer ' + token, 'x-ms-version': '2020-04-08'})
content=r.content
data=open("smart_flows.txt","wb")
data.write(content)
data.close()
My Service Principal have the Storage Blob Data Contributor  and storage blob delegator role on the storage.

@tasherif-msft any suggestions?

@tasherif-msft any suggestions?

@xiafu-msft can you suggest please ?

@xiafu-msft can you suggest please ?

Can someone please help with this issue ?

Can someone please help with this issue ?

@ujwalkasth
Hi,
Apologies for the delay, I checked the service settings for your account and everything seems to be setup fine. Would you please try this again and send me the request id (like in your first message). We hold on to service logs only for so long, so I would need another repro.
Tag me directly and i will look up the logs for your rightaway.

@ujwalkasth
Hi,
Apologies for the delay, I checked the service settings for your account and everything seems to be setup fine. Would you please try this again and send me the request id (like in your first message). We hold on to service logs only for so long, so I would need another repro.
Tag me directly and i will look up the logs for your rightaway.

As the error message: "RequestId:8fe1135e-f01e-002f-2ff4-e82696000000\nTime:2021-01-12T15:06:52.6849689ZContent-Length0'", it means you POST with a request body, but you didn't specify the content-length, which triggers the error.
Could you please add the content-length header and try again?

As the error message: "RequestId:8fe1135e-f01e-002f-2ff4-e82696000000\nTime:2021-01-12T15:06:52.6849689ZContent-Length0'", it means you POST with a request body, but you didn't specify the content-length, which triggers the error.
Could you please add the content-length header and try again?

@ujwalkasth
Hi,
Apologies for the delay, I checked the service settings for your account and everything seems to be setup fine. Would you please try this again and send me the request id (like in your first message). We hold on to service logs only for so long, so I would need another repro.
Tag me directly and i will look up the logs for your rightaway.

@amishra-dev Here is the new request ID: 4314d4ee-a01e-0022-557e-04ee42000000 . Can you please verify this request ID?

Hi @ujwalkasth are you still getting this error thrown? could you please send an updated request ID I will make sure to look into this the day you provide it


seed persistence

@ehanoc  After our discussion I went the other direction with this PR. Rather than simplifying the calls to deriveKey by not requiring the rootKey, I removed seed from the constructor so that rootKey must be passed to any function that needs it. The tests were adjusted accordingly.
The reasoning is that if the seed was only there for testing purposes, as the comment indicated, and the value should be zeroed as you described, then the functions should not rely on it as they did.


[az monitor metrics alert][Unable to update|disable availability alert]

Describe the bug
I've created availability checks in app insights and it automatically creates alert. When I try to disable it, I get the next error message
Command Name
az monitor metrics alert update
Errors:
validation error: Parameter 'MetricAlertCriteria.odatatype' can not be None.

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Create some availability check.
Try to update alerts using command below.
az monitor metrics alert update --enabled false --name {} --resource-group {} --debug

Expected Behavior
Disable that alert
Environment Summary
Linux-4.4.0-18362-Microsoft-x86_64-with-debian-buster-sid
Python 3.6.5
Shell: bash

azure-cli 2.0.68 *

Additional Context

UPD
Also tried with
azure-cli 2.0.70
The same issue.

I have the same issue.

I have similar issue when updating log profile locations

Same issue here.
Tried with azure-cli 2.0.63 on windows and 2.0.72 on cloud shell.
The error is thrown with either value for enabled parameter and for both name + resource-group parameter combination or ids parameter

Seems that you can also do this by modifying the resource, i.e.
az resource update --set properties.enabled=false --name {} --resource-group {} --resource-type Microsoft.Insights/webtests
does the same thing, except with resource update the command actually works. I added the resource-type parameter just in case

@sMiik
Yes, it works great for me.
Thanks a lot!

@Sviatik Hello. I'm looking into this issue recently. I used the az monitor metric alert create to create an alert rule for storage account. Then I tried to disable it and it worked. I couldn't reproduce this error. Could you please give me more details about your process? What's your process to create a resource and then create a metric alert?

@MyronFanQiu
Hello,
Here are steps to reproduce:

Create manually availability check (web test) in application insight. It can be ping to google.com (It automatically create alert)

.

Try to disable this alert using az monitor metrics alert update --enabled false --name {} --resource-group {} --debug

@Sviatik Thanks! I have reproduced the error. I will take a look.

@Sviatik Hello. Looks like the server will return some odata.types which are not supported by python SDK. There is not subtype  for Microsoft.Azure.Monitor.WebtestLocationAvailabilityCriteria. I will invite service team to take a look at this issue.

@Dalubash Hello. Could you please take a look at this issue?

Disabling the alert rule of an Application Insights web test using the az monitor metrics alert update command isn't supported currently.
As a workaround, you can achieve this by directly modifying the 'enabled' property of the alert rule's underlying resource, using the following command:
az resource update --set properties.enabled=false --name {} --resource-group {} --resource-type Microsoft.Insights/metricAlerts
Please note that the command above will only disable the web test's alert rule, and the web test itself will continue running. If you’d like to also disable the web test, use the following command:
az resource update --set properties.enabled=false --name {} --resource-group {} --resource-type Microsoft.Insights/webtests

@Sviatik Hello. Since service team doesn't provide SDK for CLI currently. I will left this issue open and move it to backlog.

This item is on our backlog to address (backlog item #5365351).


MemoolKit fails to progress through processing steps

This on dev, package cleared ingestion on prod (probably because it only started failing after it had successfully ingested on prod but hadn't yet reached its turn on dev).
Locally reproducible with spi_dev-2023-08-28.dump:
[ INFO ] Ingesting (id: 2810693E-8D7F-4BA8-968A-BC7C168B8324) ... [component: ingest]
[ WARNING ] fetchResource https://api.github.com/repos/FlorianHubl/MempoolKit/license request failed with status 404 Not Found [component: server]
[ WARNING ] storeS3Readme failed [component: ingest]
[ WARNING ] Environment variable not set: METRICS_PUSHGATEWAY_URL [component: ingest]

select * from packages where status != 'ok';

e6f418d6-6004-41d4-80df-1ee7162b74db | 2020-05-30 13:07:00.713297+00 | 2023-08-25 10:33:31.367182+00 | ingestion | 100 | ingestion_failed | https://github.com/amosavian/FileProvider.git | {ios,macos,macos,tvos}
2810693e-8d7f-4ba8-968a-bc7c168b8324 | 2023-08-27 20:09:55.681513+00 | 2023-08-28 07:05:56.448764+00 | ingestion | 0 | new | https://github.com/FlorianHubl/MempoolKit.git | {}

(https://github.com/amosavian/FileProvider.git is a package that disappeared completely, separate "issue", we should remove it.)
The license call to https://api.github.com/repos/FlorianHubl/MempoolKit/license is indeed a 404 but we shouldn't prevent forward progress due to it.

NB: this is only happening on dev, where we're previewing work in progress that's addressing the issues cause by the ShellOut change.

This should be addressed by the latest ShellOut version.


As revelry, I want to make updates to certain components in order to follow WCAG Standards.

In order to follow WCAG standards, we need to make a few updates to our components to provide a more accessible experience.
See education ticket that covers the standards.
Heres a list of where these updates need to happen:
Language:

[x] Add language to Possums html tag

Inputs:

[x] Inputs should have a name="" attribute
[x] File Upload inputs should have a name and/or role

Nav:

[x] Nav links should have role=" " attribute

Menu:

[x] Menu links should have role="menu" attribute

Links styled as buttons:

[ ] Should have a role="" attribute so you can declare the role as role="button"

Flash message, Tooltips, Modals, dropdowns, CloseButton:

[ ] dismissible: (content that is shown on :hover or on :focus must be dismissible via keyboard actions without requiring the user to change focus)
[ ] hoverable: (content that shows on :hover must remain showing if user mouses over content)
[ ] Also need a way to close out of them if they overlap text. Possible solutions:

Esc Key for dropdowns
Add close button to tool tips. Make tool tips only show on click (?) and then exit when the user clicks the close button.
All flash messages should include a close button. And then can also be closed by hitting the ESC key. (Would there ever be an instance where theres an flash message and a modal open at the same time? Might want to consider how the esc key would work if there were ever both open)

[x] Dropdown Selects are dismissible by hitting the esc key

Images:

[ ] Need to have alt text

Buttons:

[ ] Role if its a submit button?
[ ] Name for everything else?
Look into what WCAG requires for toggle buttons.

Expanding columns:

[ ] Add the ability to collapse using the esc key

Lintomat checks to add:

All Images, videos, ect (Non-text content) have text alternatives
(Idk if this is something that lintomat can check for or not) if theres moving content used such as image carousel or background video, make sure there's a way to pause or stop the movement.
Page Title: Each time a new page is created, check that it has a title.
During project set up, check to make sure that the language of the page is set <html lang="en">
All inputs that require a certain format must have error messages set up.
If its possible: check for contrast ratio: Contrast ratio between text and background is at least 4.5:1
Check that all Inputs (stacked and/or unstacked) have a name="" assigned to them
Check that all FileInputs have a name="" assigned to them
Check that all InputGroups have a name="" assigned to the Input inside <InputGroup.Field>
Check that every link inside a <Menu.Item> has a role="menu" assigned to them (This goes for the menu component as well as the topbar component)
All links styled as a button have a role="button"
When <CloseButton> component is used, check to make sure whatever it is closing is also dismissible with the esc key

Background: the close button component is pretty much just a style thing which is why it doesn't function on the possum site

NEW Rules to add to lintomat after March 30th:

Line height: at least 1.5 times the font size;
Spacing following paragraphs to at least 2 times the font size;
Letter spacing (tracking) to at least 0.12 times the font size;
Word spacing to at least 0.16 times the font size.
Target Size: 44px target areas

Other Notes:

programmatically set states, properties, and values

Untrapping Keyboard users:

All parts of the site must be reachable by keyboard
User must also be able to get away from all parts of the site
Check that all third-party adverts or widgets are accessible (usually theyre not)

Links:

If a link is an image, the alt text of the image makes the link purpose clear
Links going to the same destination should have the same description. Links going to different destinations should have different descriptions.

On Focus:

No automatic pop-ups without the using taking action (such a clicking)
Focus never jumps to another element automatically
Elements can change on focus if context doesn't change (i.e: when nav options drop down when you hover over an item--> thats not a change of context)

On Input:

Don't let forms automatically submit

@blazebarsamian @CheetoMao I'm assigning myself to help out with some of the checklist since I have some spare time this week!

yes please @brittanygay!! Im supposed to start working on these updates this week. I feel like I'm missing more of the name, value, role examples that we need to include but I haven't found examples.
I was about to start looking into how to add props to our input components to add name, role, ect.

I've been messing with some form elements and using voice over to read off the page and test.
When I inspect the elements, the name and role attributes are there but are not being read. Looking into this more now.

From what I understandl the purpose of name, role and value is just for the software to understand what the custom controls are so the screen reader is not confused with a div or span.
https://accessibility.huit.harvard.edu/provide-name-role-and-value-information
I checked our checkboxes using the wave accessibility tester and they pass even though the voice over doesnt verbally read out the names of the checkboxes.

Update: I'm going to move this back to icebox until I can get back to work on it (It looks like ill have some time this week).
I need to get someone to pair with me for the esc-to-dismiss functionality for modals. I've tried to figure it out myself but I think I'm missing something.

Inputs should have a name="" attribute

In most cases this is fine. There will be cases when this is not a good idea to do. Having a name attribute for an input in a form means the value will be sent to the server when submitted. For things like credit cards, crypto secrets, and personal id numbers, we may not want to send them to the server.
That's the only caveat. Just wanted to make sure to say it out loud.


Embedding support

Refs:

#185

Here's an example plugin that works with https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2
import llm
from sentence_transformers import SentenceTransformer

@llm.hookimpl
def register_embedding_models(register):
    model_id = "sentence-transformers/all-MiniLM-L6-v2"
    register(SentenceTransformerModel(
        model_id, model_id, 384)
    )

class SentenceTransformerModel(llm.EmbeddingModel):
    def __init__(self, model_id, model_name, embedding_size):
        self.model_id = model_id
        self.model_name = model_name
        self.embedding_size = embedding_size
        self._model = None

    def embed(self, text):
        if self._model is None:
            self._model = SentenceTransformer(self.model_name)
        return list(self._model.encode([text])[0])

I introduced https://github.com/simonw/sqlite-migrate here, because I need a mechanism for ensuring any database passed to llm embed -d/--database has the necessary tables - independent of the migrations that create the responses etc tables in the main logs.db database.
I should absolutely port the other LLM migrations to this system too, having two migrations systems is gnarly.

In order to write tests for this I'm going to build a plugin that only activates during tests, which has a VERY simple embedding mechanism: it returns 16 floats, each float is the length of a word in the input text, truncates to 16 words and fills in with 0 for missing words.
I'll use this pattern: https://til.simonwillison.net/pytest/registering-plugins-in-tests

I'm going to document this, then land it and then work on llm similar separately.


get() method for synth options?

For the creation of presets it would be very helpful to have a get() method for all parameters of a synth. Could this be added to the base instrument class? I’m currently implementing tone.js in the visual programming editor cables, where users can change parameters by dragging sliders.

very cool! i'm excited to see Tone in cables.
get and set currently exist in all classes since it's implemented in the Tone base class.
So you can do synth.get() which will return all of the parameter.


Bastrikov quickfix

Quickfix for 5b0358adb5397775b4f33b3b990312db2cccb4b0 in case scalachess/whatever else Lichess uses won't accept having 3 commas (family+4 variations) in the name. If not an issue, please close.
Also, it would be helpful to know some more guidelines:

Max no. of variations allowed in a name (currently Family:+3variations)
Are move numbers allowed in the name(?) - (Assuming no because I could find none)
Whether certain words like "with" instead of a comma are allowed in the name. The current blacklist only contains "refused". (In which case I will also change 4a81157d0afb940d3ded8712caee13ec50c2b49a)

Answers could/should be implemented in the lint. Thanks!

Thanks!

Within reason, more than 3 variations seem fine to me, but in this case they don't add much value, so merging.
Until recently, even mentioning moves was discouraged. But https://github.com/lichess-org/chess-openings/pull/60 showed that it is required in some cases to get unique shortest names. We still should restrict it only to those cases. I am unsure about move numbers. But since transpositions are possible, maybe best to avoid them?
If I recall correctly there was a mix of refused/declined, sometimes even in the same opening. So that lint helped keep things consistent. Using with seems fine to me. Maybe we should use <COMMA> with a4, so that splitting by , gives the hierarchy?


ui design of nav,footer,search box completed

merging latest changes of dev to main branch.

Done


Add resources as piped's default value

What this PR does / why we need it:
This aims to make it clear that the resource can be specified.
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?:

NONE

/approve


BLE400 / nRF51822 (nRF51) PWM clock too low (servo-motor example)

I'm trying to use the "servo-motor" example to control a servo with my BLE400 using a Core51822 module and the nRF5 software PWM implementation.
Unfortunately it does not work.
I measured with a signal analyzer on the GPIO/PWM pin and I'm seeing pulsewidths of only 8.1 ms maximum, despite the code being set to about 20 ms (and I tried higher values as well).
It seems a 125 kHz clock is used (the prescaler is set to 16 MHz by default), which would explain the permanent 8.1 ms. It's possible to go lower, but no higher.
Can anyone confirm or explain this, or am I missing something?

@cvinayak could you please take a look?

@tg-- I have add hw pwm for nrf52 chips. #14853 Do you have time to review it?

I think this issue can be closed now, @tg-- ?


File Names with 80% make broken links

This appears to be a problem with "%", it's not getting URL-encoded or something.

An example would help. Assuming the issue is "%" appearing in a file name, one way to work with this is to replace the "%" with "_" in the html_path. This replacement is now done with the following characters: " ","?","#".
Is this acceptable?

here is an confusing observation:
I made a wiki page in Obsidian named "# the 80% good enough claim".
I built the website locally and the URL: "http://localhost:8000/Link_workbench/the_80%_good_enough_claim.html" is accessible in local browsers.
However, that same web page fails once the website is built using Netlify.
So, what is going on here?
I think PK is right; the URL needs to be encoded.
But the difference between localhost and netlify hosting is puzzling.

@band, you've got the right solution:replace the "%" with "_" in the html_path (and collapsing repeated underscore characters to one), since we're already doing that for other characters.
For the local/web difference, offhand I'd guess the Python mini HTTP server isn't urldecoding the URL, while the Netlify HTTP server is. Or something like that. Interesting, but not surprising.


Expose the ion endpoint as a parameter

Follow up to https://github.com/CesiumGS/cesium-native/pull/444, this PR exposes the cesium-native ion endpoint parameter in Cesium for Unreal.

Thanks for the pull request @xuelongmu!

:heavy_check_mark: Signed CLA found.
:grey_question: CHANGES.md was not updated.

If this change updates the public API in any way, please add a bullet point to CHANGES.md.

Reviewers, don't forget to make sure that:

[ ] Cesium for Unreal Samples works.

Ready for another review, thanks!

Thanks @xuelongmu, I will merge this as soon as CI passes.

This is only failing on IOS with a known (and strange) problem we are looking at. I'm going to merge this, since we'll need to fix that issue before the next release anyways.
cc: @kring


Move utility programs out of the main coreruleset repository

Since ages ago, probably when there was only one repo assigned in the OWASP organization, we pushed all the utilities related to CRS in the same repository as the rules.
This has lead to having lots of different tools and scripts in the same place, making it more difficult to test properly, and to perform updates on tools independent from the rules themselves.
After considering it in our October 2024 monthly chat, we decided to move away from this pattern.
The idea is then to split the tools and scripts in different repos. This will be the epic ticket to start the move.

My initial proposal is:

[ ] move av-scanning to its own repository.
[ ] sunset change-version. crs-toolchain has the feature embedded.
[ ] move crs-rules-check to its own repository.
[ ] remove crs2-renumbering. legacy, and it will be in the 3.x branch anyways.
[ ] find-max-datalen-in-tests ?
[ ] find-rules-without-test ?
[ ] fp-finder? move to its own repo? add feature to crs-toolchain?
[ ] geo-location 👉 remove.
[ ] honeypot-sensor update for v4, create plugin?
[ ] join-multiline-rules add feature to crs-toolchain?
[ ] php-dictionary-gen add feature to crs-toolchain? move to own repo?
[ ] regexp-tricks add as crs-toolchain feature?
[ ] virtual-patching move to its own repo. I think we can start adding more products2secrules here, so there is space for improvement.
[ ] move the rule_ctl directory to its own repository.

Do we want to keep the names? Eg. is the crs-rules-check is a good name or do we want to continue with rules-check?

Question 1: How many of the utilities in /util are no longer useful and can simply be sunset at this time?
Question 2: How many of the utilities that we want to keep will break if they no longer live in /util? E.g. hard coded paths and similar?

Question 2: How many of the utilities that we want to keep will break if they no longer live in /util? E.g. hard coded paths and similar? (E.g. it might be easier to simply sunset rather than try and rewrite/fix some of them if moving them will cause them to break?)

In our CI workflow we use crs-check-rules script. I don't know about other tools which is used.

Do we want to keep the names? Eg. is the crs-rules-check is a good name or do we want to continue with rules-check?

I think we use is as our main rules-linter for now.

Question 1: How many of the utilities in /util are no longer useful and can simply be sunset at this time? (I couldn't see all of them listed, hence asking this question.)
Question 2: How many of the utilities that we want to keep will break if they no longer live in /util? E.g. hard coded paths and similar? (E.g. it might be easier to simply sunset rather than try and rewrite/fix some of them if moving them will cause them to break?)

Answer 1: Updated the list to cover all files and directories. Proposed removal, move and we need to know what to do with the unknowns.
Answer 2: probably not much, as we call them from our ci/cd or we provide it for people to use as standalone scripts. So we can remove without looking back.

My take where I do not agree with original proposal.

move av-scanning to its own repository. Why? We have a plugin and the plugin is better than this cruft.

move the rule_ctl directory to its own repository. This is strongly tied to our rules. If we move this, we run the risk to lose the functionality. I did not use it myself like ever, but it has its uses.

virtual-patching move to its own repo. I think we can start adding more products2secrules here, so there is space for improvement. Is this maintained? Is it useful? Can't we simply remove it?

honeypot-sensor update for v4, create plugin? Remove and point to honeypot project in a message

find-max-datalen-in-tests ? Not sure this works with recent YAML format changes. Remove or store far away from users (but so we can find it again if use it in like 27 years out of the blue)

find-rules-without-test ? Is this still relevant?

fp-finder? move to its own repo? add feature to crs-toolchain? We are going with the go implementation of the quantiatative testing, don't we? The functioanlity is somewhat overlapping if I get this one correctly. But even if it's not, I think we want to integrate everything into the toolchain.

join-multiline-rules add feature to crs-toolchain? No longer necessary. Remove.

php-dictionary-gen add feature to crs-toolchain? move to own repo? I'd rather have it as part of the toolchain.

regexp-tricks add as crs-toolchain feature? This is a folder containing one such trick. That one is marked as experimental and lacks a description of what it really does. Hard to say what the course of action should be.

file [send-payload-pls.sh](https://github.com/coreruleset/coreruleset/blob/main/util/send-payload-pls.sh). Don't know its utility. Remove? This is fairly useful. It's an adoption of Andrea of one of my own scripts. I use it when assessing a payload after somebody claims he / she has found a bypass. Could be done via a script wrapping a sandbox call. As it stands maybe remove it.

[verify.rb](https://github.com/coreruleset/coreruleset/blob/main/util/verify.rb) and [d-range](https://github.com/coreruleset/coreruleset/blob/main/util/id-range) looks like they work together. Don't know what they do, I'll just remove those. The work together indeed. Contributed by the FlameEyes project, a former ModSec rule project. It checks rule IDs for duplicates and if they fit into the assigned range of the project. Let's remove this.

I'll add this to the above decision then.

About the rule_ctl: all tools are strongly tied to our rules. go-ftw, crs-toolchain, etc. This is one similar tool, that adds linting to the rules, with our specific requirements. So it is not different from others.
Moving it to a new repository would allow us to write proper python tests, and eliminate the undesired behavior that we have now that is sending a patch when something changes, and "just reading the code" to see if it works.

You wrote above:

APPROVED_TAGS file is used by crs-rules-check. Must be moved with the script

I think this is not a good idea.
Consider someone wants to add new tag(s) into rule(s). Until now it was enough to add the new tag(s) to utils/APPROVED_TAGS file. But from now the contributor needs to add a PR to the another repository too.
I vote to keep this file inside of coreruleset repository.

Makes sense.

the script might not need to know which tags are approved.

That's why there is the -t option. We can pass the file with that which contains the approved tags.


Recipe-app

Project title
Recipe-app
Project Description
The idea behind this project
Getting a recipe note for any dish.
How does it work
We have a container with a search field you just need to enter any dish name and hit the search button you will get the ingredient for the recipe and steps for cooking as well.
Stack:
And in order to tick the check box just put x inside them for example - [x] like this. Please delete options that are not relevant.

[x] Html
[x] CSS
[x] JavaScript

for more details please refer to the attached images.

https://user-images.githubusercontent.com/73521123/170865762-d879b5cf-4da6-4ae2-b04a-f5d79c9e0cb4.mp4

@AyushSingh22 sir plz assing me


Upgrade for better chatgpt.js framework

Hook library, has been integrated into:https://greasyfork.org/scripts/465483-hookrequestandfetch/code/hookRequestAndFetch.js


Violation about Preventing invalid impressions

I used the same code demo from Native Ads in my app and the banner displays correctly but when I published it on AppGallery I got the following message:
An ad cannot be displayed repeatedly for one request. After being displayed, the ad needs to be loaded again in real time for the next request.
[Recommended] The ad object on each page is independent. You need to use the createXX method to create ads and register a callback.
Please, has anyone had the same problem with this code being used?

Yes, it is recommended to create a new one at a time, as in the demo.

I updated my code today and the ads are showing correctly on the phone. but i'm not sur if it's the correct way to avoid the message : An ad cannot be displayed repeatedly for one request
Please have a look https://github.com/AmiriOS2019/myfiles2/blob/main/MyActivity.java

Look at the code and I think your problem can be solved.
But, I don't think it's handled right here, why did you destory ad just loaded?

Thank you ttljtw for your remark. I put it first like on the demo code link, but after seeing the documentation I noticed that I should destroy the native ad after displaying it, so I move it after adScrollView.addView(nativeView);


8 test launching job workflow

Added just simple test that launches the job and exist succes

I have added the direct path to the test workflow using -W flag


Ambiguous "If any changes are found"

In the " Where your file is saved makes a difference" under "OneDrive - Business", it is ambiguous what the change is.
Is the change a change in the raw data point in the dataset, a change of a visual, calculated column? What happens when a new entry is added to an SQL database, will OneDrive know about that change?
Note: first time using GitHub please mention me somehow to acknowledge this comment.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 394d90a7-d6ad-c666-9099-a85e7ab4d817
Version Independent ID: 5077293e-a2c0-16b6-08bb-b65b11b1e9e5
Content: Get data from files - Power BI
Content Source: powerbi-docs/service-get-data-from-files.md
Service: powerbi
GitHub Login: @davidiseminger
Microsoft Alias: davidi

Hi @coreyindahouse - sorry for any ambiguity. The change is for anything in a file that's stored in OneDrive for Business in that location. So changes to Excel or PBIX files that are then saved in that location would show up. If you have a connection to a database in that Excel file and the database is changed, but the Excel file doesn't automatically pick it up, the report would only be updated once the source Excel file is actually updated. I hope that helps.

Answered - thanks for posting!


Preview is broken for custom fields on pages

You have to save the page to preview your changes to the custom fields.
Here is a possible solution: https://wordpress.org/support/topic/preview-and-post-meta-custom-fields-solution

Looks like when you click the update button on the custom field value it will propagate to the next time you click preview. So this isn't really an issue just a confusing bit of the WordPress UI.


Update composer.lock

please allow higher json-schema version than 1.6~

@cbka Thank you for looking into this, but the composer.lock is auto-generated and should not be changed. It looks like you may want to change the composer.json instead?
I believe this has been answered, so I'm closing this for now. Please come back with more details if this problem persists and we can always reopen this :+1:


Add ability to set splash screen length via web-config

Add the ability to set the length of time that the splash screen is active for via the web-ui.
Options would include:
Default
5 seconds
10 seconds
30 seconds
Always on

It's in with manual timer


Support linalg.depthwise_conv in linalg on tensors path

At the moment mhlo.convolution to linalg.depthwise_conv_2d conversion still happens in one step in HLOToLinalgOnBuffers. This should be changed to first convert mhlo.convolution to linalg.depthwise_conv_2d on tensors in HLOToLinalgOnTensors like what Hanhan has done for those normal convolution cases.

I have something for this, will send out for review soon.


Removing erroneous comment

@stevector, thanks for your PR! By analyzing the blame information on this pull request, we identified @danielbachhuber to be a potential reviewer


Latest az bot extension is not available via az extension add -n botservice

Tool
Name: Az bot
Version: 0.0.3
OS: any
Describe the bug
C:\repos\BotBuilder-Samples\samples\javascript_nodejs\51.cafe-bot>az extension list
[
  {
    "extensionType": "whl",
    "name": "botservice",
    "version": "0.0.3"
  }
]

C:\repos\BotBuilder-Samples\samples\javascript_nodejs\51.cafe-bot>az extension remove -n botservice

C:\repos\BotBuilder-Samples\samples\javascript_nodejs\51.cafe-bot>az extension list
[]

C:\repos\BotBuilder-Samples\samples\javascript_nodejs\51.cafe-bot>az extension add -n botservice
The installed extension 'botservice' is in preview.

C:\repos\BotBuilder-Samples\samples\javascript_nodejs\51.cafe-bot>az extension list
[
  {
    "extensionType": "whl",
    "name": "botservice",
    "version": "0.0.3"
  }
]

Expected behavior
Expect az extension command to install the latest 0.4.1 az bot extension
[bug]

@swagatmishra2007; @carlosscastro  - this appears to be an issue because I was on the older AZ CLI version. We should update the README to call out that the users need to have AZ CLI version >= 2.0.45 installed.

I am still getting the same error as per the first entry!

@sorgina13  what is your core cli version? Can you run az --version?
See the topmost line of the output. It should be azure-cli (2.0.45) or greater.

@swagatmishra2007 , thanks for your reply. It is indeed azure-cli (2.0.35). How do I upgrade it?

sorted! just downloaded the latest msi and reinstalled!

Hi, i have cli 2.0.53 and i am expecting bot service 0.4.3. But i only have botservice 0.4.1.
How can i get botservice 0.4.3?

You can run
az extension remove -n botservice
az extension add -n botservice

It ll still install bot service 0.4.1 but i am looking for 0.4.3 to run my virtual assistant

I have the same problem (az version 2.0.53).  Debug output shows this:
Skipping 'botservice-0.4.3-py2.py3-none-any.whl' as not compatible with this version of the CLI. Extension compatibility result: is_compatible=False cli_core_version=2.0.53 min_required=2.0.52 max_required=2.0.52

@ashoksamal6363 how did you get the msi file? The google drive link has the source code only. MSI file is missing.

Please check this.
https://drive.google.com/open?id=1OmGtfMkd5bEuz3hwqiLstJDGsshK6-AN

az --version ? shows azure-cli as 2.0.54, but botservice as 0.1.3, when i try to remove botservice thru this command, az extension remove -n botservice, it says The extension botservice is not installed. How to update botservice extension.
When i run my msbot clone services...., i keep getting You need to upgrade your az bot service version to >= 0.4.3... how to fix it?

@armstrong-charles, we’re planning on releasing a new version of msbot soon that will fix this problem. With the upcoming release you won’t need to add the extension in addition to having azure-cli installed to use msbot

Is there any workaround now? it is a blocking issue, any help will be greatly appreciated.

@stevengum, how about uninstalling and installing older version of Azure CLI? any suggestion?

@stevengum, how to get binary release of azure CLI v2.0.52 without building it from https://github.com/Azure/azure-cli/releases source.

We haven’t done a release on NPM yet, but our daily feed does have the bits needed for msbot to work with azure-cli 2.0.53 and greater. It can be found here:
https://botbuilder.myget.org/feed/botbuilder-tools-daily/package/npm/msbot/4.3.2-125

@armstrong-charles, I've found a link that should work; this should get you azure-cli 2.0.52: https://azurecliprod.azureedge.net/msi/azure-cli-2.0.52.msi

@armstrong-charles Have you solved your problem? I am facing the same issue as you did (need to upgrade my botservice).
Tried hints from this thread but doesnt work with me (I am totally new in programming so not sure if its my mistake or other issues)

I solved it by downgrading to the older version of azure CLI. Two options:
Option1: use chocolatey
choco install azure-cli --version 2.0.52
Option 2: Get it from here:
https://azurecliprod.azureedge.net/msi/azure-cli-2.0.52.msi
A few hints:
When MsBot  fails, it shows you the az bot command failed.  you can run the command at the command line and get more detail by az group deployment operation list --resource-group
I spent a few hours until I found that I have to change the bot name everytime I re-try the MsBot  command.
I think MsBot  is just a patch solution untill this product becomes more mature. You would not see such bad product design when Google engineers to design and deliver something.

Hi @AllanXu49
I tried following your solution, however, I get the following error:
With msbot 4.3.2 and later, the only AZ CLI prerequisite is having an AZ CLI version >= 2.0.53. If you have the botservice extension installed as well, please remove this via "az extension remove --name botservice".
Checking az botservice version
az -v
You need to upgrade your AZ CLI version to >= 2.0.53.
You can install the latest AZ CLI from https://aka.ms/az-cli-download
what version of msbot are you running?

@AllanXu49 I tried the first option (not 100% sure how to use the other one) and proceeded like this:

install choco (0.10.11)
install azure - cli (2.0.52) using the command you sent
used command: 'az extension add -n botservice'
with error No matching extensions for 'botservice'.
my msbot is 4.3.1
Did I miss some step? Thanks.

Any update here? I am getting the same error 'No extension found with name 'botservice''

Hi Folks,
The setup I mentioned worked for me. Mind that I used  msbot clone services command only.
Please take a look at #738 as well. The product group supposed to fix this last Friday. You might want to follow up with them too. @sgellock explained that they are updating MSBOT tool.

Here the same; I do have the same version but getting error No matching extensions for 'botservice'. while trying to add extention botservice using command az extension add -n botservice. @stevengum any idea when the release is ready. Is support for azure-cli 2.0.54 included?

Az CLI 2.0.54 has shipped.  It installs botservice 0.1.3.
Running: az extension add -n botservice errors out with:
No matching extensions for 'botservice'. Use --debug for more information.
From: Steven Gum<EMAIL_ADDRESS>Reply-To: Microsoft/botbuilder-tools<EMAIL_ADDRESS>Date: Monday, December 31, 2018 at 9:36 AM
To: Microsoft/botbuilder-tools<EMAIL_ADDRESS>Cc: Scott Gellock<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [Microsoft/botbuilder-tools] Latest az bot extension is not available via az extension add -n botservice (#615)
Reopened #615https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fbotbuilder-tools%2Fissues%2F615&data=02|01|scott.gellock%40microsoft.com|6ab8dfdf0f4f49cdc2f708d66f46739f|72f988bf86f141af91ab2d7cd011db47|1|0|636818745716697966&sdata=80GP7hpjWKpUHVqjwWQprnE0d7gd1wNsXbRe6nG9GuA%3D&reserved=0.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fbotbuilder-tools%2Fissues%2F615%23event-2049109546&data=02|01|scott.gellock%40microsoft.com|6ab8dfdf0f4f49cdc2f708d66f46739f|72f988bf86f141af91ab2d7cd011db47|1|0|636818745716707976&sdata=LcjJcBbgwgK9d8BipiCGRfrsUF1wHfXmlNB4DMjKITo%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAsxdy-tCjOE-gpLZGAK6MWZfG00sfH_ks5u-ksJgaJpZM4XBCz3&data=02|01|scott.gellock%40microsoft.com|6ab8dfdf0f4f49cdc2f708d66f46739f|72f988bf86f141af91ab2d7cd011db47|1|0|636818745716707976&sdata=%2F9excJJJzBzhHlaadeBuoz8p8JKD0vrSU2HS5ls7p3s%3D&reserved=0.

@stevengum , Thanks, that's working now!

Closing as this has been addressed with the latest public release of msbot, version 4.3.2. This version of msbot does not use the botservice extension, so you are not required to call az extension add -n botservice.
If you still run into this issue, please @mention me in this issue or open up a new issue.

@stevengum, i have version 4.3.2 of msbot and come up with the same error
C:\Users\olgeorg\Documents\Visual Studio 2017\Projects\SovereignBot\SovereignBot>msbot clone services --name "SovereignBot" --luisAuthoringKey "2c92b0812f6b47cfa5d4da293f2bbaa3" --folder "DeploymentScripts\en" --location "westus" With msbot 4.3.2 and later, the only AZ CLI prerequisite is having an AZ CLI version >= 2.0.53. If you have the botservice extension installed as well, please remove this via "az extension remove --name botservice". Checking az botservice version You need to upgrade your AZ CLI version to >= 2.0.53.         You can install the latest AZ CLI from https://aka.ms/az-cli-download

@olufemig what version of Azure CLI to you have, also, if you've installed a version of Azure CLI prior to 2.0.53, can you check if the bot extension is installed?  The error message you've supplied would imply that an old extension is installed and needs to be removed:
If you have the botservice extension installed as well, please remove this via "az extension remove --name botservice".

@stevengum  @sgellock I'm having the same issue.
msbot version 4.3.2
Azure CLI version: 2.0.58
Bot extension has not been previously installed.

@LuNest and @olufemig please use Azure CLI 2.0.57 while we investigate this issue. This specific version of Azure CLI can be installed via this link: https://azurecliprod.azureedge.net/msi/azure-cli-2.0.57.msi

I am having the same issue on macOS High Sierra 10.13.6.
msbot: 4.3.2
Azure CLI: 2.0.58
Command az extension add -n botservice --debug results in :
...
Skipping 'botservice-0.4.3-py2.py3-none-any.whl' as not compatible with this version of the CLI. Extension compatibility result: is_compatible=False cli_core_version=2.0.58 min_required=2.0.52 max_required=2.0.52
..
At the moment most of the official sample bots can not be configured as described in their readme files on github.

We're working to get 4.3.3 of msbot out that contains a fix for this.


feat(plugin): Watch for changes to dark mode

👋 Hi, right now when people visit a website using the prefers-dark.js script, it'll only set the theme at the beginning when they load the page, so if they change their system theme afterward, it won't be updated on the site.
This pull request, which is modeled after https://github.com/ChanceArthur/tailwindcss-dark-mode/pull/34, fixes that by using the listener / observer feature of matchMedia.

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.
🔍 Inspect: https://vercel.com/ejmaito/tailwindcss-multi-theme/r9pjaqgkc
✅ Preview: https://tailwindcss-multi-theme-git-fork-jakenavith-master.ejmaito.vercel.app

By the way, you won't be able to see a difference on this automatically generated preview (very cool, by the way!) because you use Alpine code there instead of the prefers-dark script.


Goreleaser enhancements

make now generally leans on goreleaser instead of a separate build process
version string has much more interesting data in it.
better testing
.gitignore ignores  dist
CI for goreleaser config

hmm. Ci for goreleaser config isn't running.

@philandstuff My biggest concern here is the use of goreleaser as part of generalized "make" if you don't see that as an issue I'm content to move forward and get the build processes consistent.


Add Windows build

Hey, I saw that you merged a PR about adding Windows support.
Is there a way I can help to release a Windows build so that anyone can download it?
Thanks.

Closed due to inactivity


Simplify the operation handling code by replacing fragmented control flow with regular async/await

Proposed changes
This PR should make the operation handle simpler to reason about and change, by allowing operation handlers to use .await without blocking. The scope of the PR is to show the idea working on the config_snapshot and config_update operations.
Below i describe the current state, which is subject to change.

In 6b90d51e4420dbfe5cd7906bcf57e48494e9ae4e, the main C8y mapper actor is prepared for its messages to be handled concurrently in separate tasks. For now, we use Mutexes to guard the MqttPublisher and CumulocityConverter:

using wrapper struct MqttPublisher(Arc<Mutex<LoggingSender<MqttMessage>>>) with fn send(&self) is ok, as we lock the mutex to use the sender, and unlock it immediately after we finished sending, so it shouldn't block. The alternative would be to clone the sender for each worker, and I will explore it later.
with Mutex<CumulocityConverter>, it ensures that only a single worker can use CumulocityConverter at any given time, which can lead to deadlocks if we're not careful, and currently results in the same behaviour as before, i.e. blocking in the operation handling code will block processing of other messages, because the lock will be open across await points.
This is temporary, and will have to be fixed, but is used right now so I could show how the full operation handling function would look like, without the current fragmentation. Also, the test demonstrating that the converter doesn't block should be made.

In 72dc9d09bb644758c4684253ee8f276d53168666, I export Identity type from the download crate, so I can use it in the next commit.
In 0e8656851b2fb1d9be90021ef0feab33d9d7d818, Downloader is used directly in the operation handler, to show how we can use it directly without going through the actor: if we only want to download a file via HTTP, and this download shouldn't influence anything else, we can use it directly.
In 892ecbec71a21a9e9bc93252f27abdac4057390d we use UploaderActor via ClientMessageBox, where we receive the response directly in the same place where we send(), instead of going to the top-level receiver in the top-level actor, which then needs to decide to call an operation handling function again. This way we still use an uploader actor, and have messages sent between them, but don't need to fragment our control flow. This was done to show that we can still send and receive messages from other actors from within the proposed new operation handlers.

Next steps

[x] rebase newest changes
[x] replace lock on mqtt sender with cloned senders (can be done, but will require &mut self)
[ ] add more tests to see which operations are blocked/executed concurrently
[ ] make necessary changes to CumulocityConverter so operation handling doesn't block
[ ] clean up operation handling code

Types of changes

[ ] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
[ ] Documentation Update (if none of the other choices apply)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

#2880

Checklist

[x] I have read the CONTRIBUTING doc
[x] I have signed the CLA (in all commits with git commit -s)
[x] I ran cargo fmt as mentioned in CODING_GUIDELINES
[x] I used cargo clippy as mentioned in CODING_GUIDELINES
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)

Further comments

Closing in favour of #2904, which is a more promising and feasible approach.


Document how files are determined to be CommonJS

it would be great if there was a note in the documentation how rollup-plugin-commonjs determines if a file is CommonJS. For example, the fact that the existence of a reference to globals causes the file to be interpreted as CommonJS is not super obvious, even if the package.json uses jsnext:main or module.

Yeah, this should be spelled out a bit more clearly. I'd actually be in favour of implementing something along the unambiguous grammar proposal – i.e. if there's an import or export anywhere, it's not CommonJS (and only once we've failed to find import/export do we look for CommonJS features)

Hey folks (this is a canned reply, but we mean it!). Thanks to everyone who participated in this issue. We're getting ready to move this plugin to a new home at https://github.com/rollup/plugins, and we have to do some spring cleaning of the issues to make that happen. We're going to close this one, but it doesn't mean that it's not still valid. We've got some time yet before the move while we resolve pending Pull Requests, so if this issue is still relevant, please @ me and I'll make sure it gets transferred to the new repo. :beer:


Implement native ORM

It would be an excellent decision native implementation of an ORM own, or take an existing one and integrate, to give more tools to developers.

Thanks for your interest! At this time, we don't plan on including an ORM, but are looking into ways to make it simpler to integrate with third-party ORMs.
However, most third-party ORMs can be used within any  CodeIgniter application already. Use Composer to load it. Ensure the use_composer flag in the main config file is setup. Then make a new instance of the ORM and go to town! No special integration needed.

excellent thanks for your reply

I found simple ORM for CodeIgniter 3.
https://github.com/yidas/codeigniter-model/blob/master/README.md
I think this can be used as CI4 default Model or added as optional model library.
Thanks.

You can implement the Eloquent ORM easily, I think it is currently the simplest and best option.

You can implement the Eloquent ORM easily, I think it is currently the simplest and best option.

I first read that as it would be a simple thing for us to build in the framework and starting laughing. I know that Taylor has said the most difficult thing he did with the framework was the ORM.

I found simple ORM for CodeIgniter 3.
https://github.com/yidas/codeigniter-model/blob/master/README.md
I think this can be used as CI4 default Model or added as optional model library.

Well... our model already handles 80%+ of that. Some of that belongs at the framework level and not the model level, and it doesn't support any form of relationship management, which is what most people want from an ORM.
But if you like it - help them modify it to work in CI4 and use it your projects.

A quick question @lonnieezell about how load a new service like ORM on CI4, for example Eloquent:
composer.json
"illuminate/database": "^5.7"

application/Services/DatabaseService.php
<?php
namespace App\Services;

use Illuminate\Database\Capsule\Manager;

class DatabaseService {
    public function __construct() {
        $manager = new Manager();
        $manager->addConnection([
            "driver"    => 'mysql',
            "host"      => getenv('database.default.hostname'),
            "database"  => getenv('database.default.database'),
            "username"  => getenv('database.default.username'),
            "password"  => getenv('database.default.password'),
        ]);
        $manager->bootEloquent();
    }
}

application/Models/User.php
<?php
namespace App\Models;

use Illuminate\Database\Eloquent\Model;

class User extends Model {
    protected $fillable = ['name', 'email', 'password'];

    public function scopeRandom () {
        return $this->orderByRaw("RAND()")->first();
    }

    public function getNameAndEmailAttribute () {
        return sprintf('Nombre: %s, Email: %s', $this->name, $this->email);
    }
}

This is a full implementation, but, which is the best way for load a service like it?
Thanks for your help!

@uno-de-piera I would just create a new method in application/Config/Services.php if you're wanting to make it simple to reuse that connection.

Thanks @lonnieezell, great job.


collection.with_options with default write_concern is choking

Using mongomock and mongoengine (0.16.1) results in mongomock raising a NotImplementedError.
See https://github.com/MongoEngine/mongoengine/issues/1953.
This is currently making it impossible to use mongomock in unit tests.
Is there any chance this can be changed in mongomock or can anyone suggest a simple work-around?

Hi @mcsloe sorry that mongomock is blocking your unit tests, and thanks for reporting.
Note that we should accept a default WriteConcern (i.e. with no specific options). If you choke on the NotImplementedError, either we have a bug or you are setting a specific WriteConcern somewhere. Can you check in your code if you are setting some write concerns at one point? I know that at one point there was a bug because mongoengine was recreating a WriteConcern and that we didn't manage to compare it with the default option, but it should be fixed in v3.13.
If you can find which option you are setting, I might implement it/or accept the current implementation as valid for this write concern.

After a little digging, it seems that https://github.com/MongoEngine/mongoengine/blob/34e3e45843ef0475824b79b3405285a2d75d7648/mongoengine/document.py#L367 is setting it to {'w': 1} which according to the doc is the default.
So I'll make sure to fix accept this value as a default. Sorry about that.

Would it be possible to cut a release with this change? We're looking to upgrade to 0.16.1 of mongoengine and would prefer to get mongomock to a tagged version as well.
thanks for the project!

@svanburen I'd be happy to. Can you please first check that my fix actually solves your issue?

:+1: with 18e5ad6 all my mongo tests work again.
$ ./manage.py test tests.test_mongo
...
FAILED (errors=6)

$ curl 'https://github.com/mongomock/mongomock/commit/18e5ad6f9287a8587a56c799e098f36ac836510d.patch' | patch -p1

$ ./manage.py test tests.test_mongo
...
Ran 6 tests in 0.028s

OK

Thanks for checking. Released as v 3.14.0.

Fixes my tests as well. Thanks for the quick turnaround!

@pcorpet I seem to be getting a similar error after upgrading to v 3.14.0:
colc.with_options(write_concern = 1).update_one(..)
NotImplementedError: write_concern is a valid parameter for with_options but it is currently not implemented in Mongomock
Should it be supported or is this a different usecase (should I open a new issue?)

I think the issue here is that write_concern param should be a WriteConcern object, not an int. It would still be nicer if the message error would tell you that :-)

Sorry, it's my mistake earlier on (i wrote the 1 value incorrectly).
The actual code looks like this:
write_concern = WriteConcern(w=2, wtimeout = 60000)
colc.with_options(write_concern=write_concern).update_one(...)

OK, that makes more sense.
This is a different write concern and for now mongomock does not support it (as stated in the error). You can open another issue for that feature request. Please give a hint on how you think mongomock should behave with a different write_concern.

Thanks, opened #472


update docs/conversion.md

docs/conversion.md is slightly out of the for example tmpfs is as unsupported

we can close this now

done


shellcheck: minimize exclusions

Only ignore unsuported languages (SC1071)
Hide false positives on a case by case basis

:+1:


Docs: pnpm install of eslint not correct on the website

What is the improvement or update you wish to see?
First I just want to thank everyone at Vercel for the tools you make. You all make the web / js / ts development significantly better and it's appreciated.
The documentation for the eslint install for pnpm is pnpm install jest --save-dev --recursive --filter=ui --filter=docs --filter=web which is incorrect.
Is there any context that might help us understand?

Does the docs page already exist? Please link to it.
https://turbo.build/repo/docs/guides/tools/eslint

Can you say how it's incorrect? In what way?

Can you say how it's incorrect? In what way?
Happy to fix, just not sure why it's wrong because it works well when I try it.

It suggests installing jest instead of just installing eslint was this by design? It seems like an unnecessary added dependency. In my case we are using vitest why do I have to install jest also?

🤦 I was completely looking past that at the flag combination. Fixing now.

🤦 I was completely looking past that, focusing on the flag combination. You're totally right. Fixing now.

Okay good 😅 I was starting to second guess myself. I believe yarn says jest as well.

This is definitely a 13 pointer though you'll probably have to take the rest of the day off after fixing and testing this one...😉

T-Shirt Size: Hulk in Green Mode

Fixed now. Thank you for reporting.


Use alias assignment in Filter

Code is greatly simplified, memory consumed is reduced.

generated/dscanner-9364d6f15f4a610fda49a693dbc18608bfc701bb/dsc --config .dscanner.ini --styleCheck etc std -I.
---
  | std/meta.d(885:20)[error]: no identifier for declarator
  | std/meta.d(885:44)[warn]: Empty declaration
  | posix.mak:577: recipe for target 'dscanner' failed

The style checker seems not to like this idiom.

Oh I see that you explained using experimental features in #8033. I assume you have discussed with Walter and/or Atila about the issue?

Oh I see that you explained using experimental features in #8033. I assume you have discussed with Walter and/or Atila about the issue?

Yes, @WalterBright and @atilaneves are in the loop. Essentially we can't get an estimate on how well the feature could fare outside the stdlib because the stdlib would be its primary beneficiary (which would then improve the life of client code).
The PRs are intentionally bite-sized so we can easily dustmite and pinpoint performance problems back to them.

Worst case you could always revert the specific commits that use this if the feature were removed too. Probably best to keep a list of them just in case.

It took me a sec to realize what the recursive AliasSeq assignment was with the eponymous alias but I get it now and indeed this is really quite decent. Cool.

Someone would need to upgrade/improve/fix dscanner before this can be used.

That has been fixed. :)


Add python-mongo template terraform

this PR adds the terraform modules code for todo-python-mongo template by adding :

[ ]  Python-mongo infra terraform code.
[ ]  application-insights, a common infra module, terraform code.
[ ] repo.yaml, and azure.yaml config files for the template repo generation.

➡Note : Target template remote to be created Azure-Samples/todo-python-mongo-terraform
➡Note : The generated template has a dependency on this issue being addressed rewrite module path normalization .
fixes #232

@HadwaAbdelhalem are you planning to add launch.json and tasks.json files for local development?
We might want to reuse them between this template and the Bicep-based one. This would require moving them to a shared folder.

@HadwaAbdelhalem are you planning to add launch.json and tasks.json files for local development?
We might want to reuse them between this template and the Bicep-based one. This would require moving them to a shared folder.

@karolz-ms , I had them in a previous PR. and received feedback to exclude them as they are specific tothe todo-apps https://github.com/Azure/azure-dev/pull/437/commits/60a463bae9339847ef0d5b3daf4e5417cc21d34d

Repoman Generation Results
Repoman pushed changes to remotes for the following projects:
Project: todo-python-mongo-terraform
Remote: azure-samples-staging
Branch: pr/532
You can initialize this project with:
azd init -t Azure-Samples/todo-python-mongo-terraform -b pr/532

View Changes | Compare Changes

Repoman Generation Results
Repoman pushed changes to remotes for the following projects:
Project: todo-python-mongo-terraform
Remote: azure-samples-staging
Branch: pr/532
You can initialize this project with:
azd init -t Azure-Samples/todo-python-mongo-terraform -b pr/532

View Changes | Compare Changes

@HadwaAbdelhalem are you planning to add launch.json and tasks.json files for local development?
We might want to reuse them between this template and the Bicep-based one. This would require moving them to a shared folder.

@karolz-ms , I had them in a previous PR. and received feedback to exclude them as they are specific tothe todo-apps 60a463b

@HadwaAbdelhalem and I synced offline. The VS Code files will be included automatically as part of template generation; and the files that we have for Python-Mongo flavor of the ToDo app will work unchanged.

Repoman Generation Results
Repoman pushed changes to remotes for the following projects:
Project: todo-python-mongo-terraform
Remote: azure-samples-staging
Branch: pr/532
You can initialize this project with:
azd init -t Azure-Samples/todo-python-mongo-terraform -b pr/532

View Changes | Compare Changes

Repoman Generation Results
Repoman pushed changes to remotes for the following projects:
Project: todo-python-mongo-terraform
Remote: azure-samples-staging
Branch: pr/532
You can initialize this project with:
azd init -t Azure-Samples/todo-python-mongo-terraform -b pr/532

View Changes | Compare Changes

Repoman Generation Results
Repoman pushed changes to remotes for the following projects:
Project: todo-python-mongo-terraform
Remote: azure-samples-staging
Branch: pr/532
You can initialize this project with:
azd init -t Azure-Samples/todo-python-mongo-terraform -b pr/532

View Changes | Compare Changes


Pleeeeease, don't remove prop-to-attribute mapping.

This is a good feature that I'm relying on, and the removal of it (see the comment there) means we have to side-step SkateJS API to achieve it. EDIT: oops, I may have commented in the wrong place there, but I see the latest version of with-update in the v6 branch does not have the prop-to-attribute feature any more. I believe that is a really good feature, because it is needed sometimes, and the previous API makes it really easy to automate.

Yeah, I removed it to see what it actually felt like and I think it's going to have to go back as an opt-in to sate those that need that behaviour.

What the API will probably end up looking like is:
type PropConfig = {
  attribute: string => string,
  deserialize: string => any,
  serialize: any => string
}

IMO property to attribute reflection should be kept
Some actual use case: https://github.com/morbidick/bootstrap-webcomponents/issues/1
Also LitElement is getting this feature as part of its redesign

Reflecting properties to attributes will be kept. That said I’m unsure if
name mapping is necessary at this point. Keep in mind v6 is still very much
a playground for me but keep giving feedback for sure.
On Mon, 13 Aug 2018 at 11:06 am, Luiz Américo<EMAIL_ADDRESS>wrote:

IMO property to attribute reflection should be kept
Some actual use case: morbidick/bootstrap-webcomponents#1
https://github.com/morbidick/bootstrap-webcomponents/issues/1
Also LitElement is getting this feature as part of its redesign
https://github.com/Polymer/lit-element/pull/132
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/skatejs/skatejs/issues/1381#issuecomment-412384994,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIVbGQP7l26-c8P3auxBe3bbndRedpAks5uQNECgaJpZM4SJltZ
.

Reflecting properties to attributes will be kept

I noticed. Yaaaay. 👍
I think this can be closed now.

so don't be too alarmed if you see this in experimental branches in the future, but totes keep feeding back.

👍
Yep, I saw it gone in v6 branch, but back in v6-trey.
Just like PropTypes in React is an optional feature, it'd be great to also have runtime type stuff in Skate as an optional feature.


Start voting after 11 minutes despite unit file.

Closing this: It's not urgent and needs some discussion first.


No module named 'test'

Seems like module 'test' is missing in current version, is it changed to evaluation? Also, benchmarks for IJB-A, LFW, CFP-FF, CFP-FP, AgeDB-30, calfw, cplfw are nowhere to be found.

Hi, module 'test' is not needed. I removed it now. Benchmarks for those except for IJB-A and MegaFace are located in evaluation/verify.py. The testing data are included in the data folder, e.g., data/ms1m/lfw.bin, once you correctly preprocess the data following the README. The testing code for IJB-A is not ready for now.

Thanks! But there is still some code related to 'test' in datasets.py.

Thank you for pointing it out. Updated now.

It works now, thank you.