Dev reservations microsoft.capacity 2022 11 01

ARM API Information (Control Plane)
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Azure 1st Party Service can try out the Shift Left experience to initiate API design review from ADO code repo.  If you are interested, may request engineering support by filling in with the form https://aka.ms/ShiftLeftSupportForm.
Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[x] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
By default, Azure SDKs of all languages (.NET/Python/Java/JavaScript for both management-plane SDK and data-plane SDK, Go for management-plane SDK only ) MUST be refreshed with/after swagger of new version is published. If you prefer NOT to refresh any specific SDK language upon swagger updates in the current PR, please leave details with justification here.

Contribution checklist (MS Employees Only):

[ ] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[x] Check this box if any of the following apply to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review.  Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki.

[x] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[x] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If you have any breaking changes as defined in the Breaking Change Policy, request approval from the Breaking Change Review Board.
Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Additional details on the process and office hours are on the Breaking Change Wiki.
NOTE: To update API(s) in public preview for over 1 year (refer to Retirement of Previews)
Please follow the link to find more details on PR review process.

    "Cancelled",

Problem from old api version: I am under the impression that provisioningState 'canceled' needs to be spelled with just one 'l' (L) in order for ARM to understand it.
@rpmanda @JeffRichter should they fix this up now even though its a breaking change?

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:1182 in 4aa9531. [](commit_id = 4aa9531fcbe4fc54edf99718f5837ec9e218e90b, deletion_comment = False)

    "summary": "Get the regions and skus that are available for RI purchase for the specified Azure subscription.",

Gets the list of region and .......

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:76 in 7f20066. [](commit_id = 7f20066ecdb8dd12aeecaf1452fd97d4ce2ff20f, deletion_comment = False)

    "operationId": "GetCatalog",

This is a List call at the Subscription scope to bring all the catalogs available for RI purchase
ListCatalogs_BySubscriptionId

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:77 in 7f20066. [](commit_id = 7f20066ecdb8dd12aeecaf1452fd97d4ce2ff20f, deletion_comment = False)

        "$ref": "./examples/GetCatalog.json"

Create an Example for List Catalogs. Response will be array of Catalogs

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:80 in 7f20066. [](commit_id = 7f20066ecdb8dd12aeecaf1452fd97d4ce2ff20f, deletion_comment = False)

"AppliedReservationList": {

Add "x-ms-idenifiers" for array

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:3057 in 7f20066. [](commit_id = 7f20066ecdb8dd12aeecaf1452fd97d4ce2ff20f, deletion_comment = False)

      "$ref": "#/definitions/SubscriptionScopeProperties"

Replace boolean/switch properties with better enum
https://armwiki.azurewebsites.net/rp_onboarding/process/api_review_best_practices.html

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:1950 in 7f20066. [](commit_id = 7f20066ecdb8dd12aeecaf1452fd97d4ce2ff20f, deletion_comment = False)

"CatalogsResult": {

x-ms-identifiers need to be added

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:1477 in 7f20066. [](commit_id = 7f20066ecdb8dd12aeecaf1452fd97d4ce2ff20f, deletion_comment = False)

      "x-ms-client-flatten": true,

x-ms-client-flatten should not be added here. Can be added to a collection within properties.

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:3044 in 7f20066. [](commit_id = 7f20066ecdb8dd12aeecaf1452fd97d4ce2ff20f, deletion_comment = False)

      "x-ms-identifiers": []

they don't have any property thats a unique identifier or key?

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:1365 in 7f20066. [](commit_id = 7f20066ecdb8dd12aeecaf1452fd97d4ce2ff20f, deletion_comment = False)

      "description": "Fully qualified id of the Savings Plan order being purchased"

resource id?? Or its some other full qualifier like an URI?

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:2444 in 7f20066. [](commit_id = 7f20066ecdb8dd12aeecaf1452fd97d4ce2ff20f, deletion_comment = False)

  "description": "Subscription that will be charged for purchasing Reservation or Savings Plan"

Try to keep docs in plain english style without a lot of capitalizing?

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:3250 in 7f20066. [](commit_id = 7f20066ecdb8dd12aeecaf1452fd97d4ce2ff20f, deletion_comment = False)

      "description": "Fully qualified id of the Savings Plan order being purchased"

resource id?? Or its some other full qualifier like an URI?
Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:2444 in 7f20066. [](commit_id = 7f20066, deletion_comment = False)

It is something like "/providers/Microsoft.BillingBenefits/savingsPlanOrders/20000000-0000-0000-0000-000000000000"

"/subscriptions/{subscriptionId}/providers/Microsoft.Capacity/appliedReservations": {

This is a list call. Please add "x-ms-pageable": { "nextLinkName": "nextLink" },
Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:158 in 7f20066. [](commit_id = 7f20066, deletion_comment = False)

The nextLink is only for the deep nested value inside, not for the response object. When I tried to add, I got error that response object doesn't contain nextLink

      "x-ms-identifiers": []

Is it not also
"x-ms-identifiers": [
"reservationId"
]
?

In reply to:<PHONE_NUMBER>

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:1365 in 7f20066. [](commit_id = 7f20066ecdb8dd12aeecaf1452fd97d4ce2ff20f, deletion_comment = False)

      "x-ms-client-flatten": true,

x-ms-flatten is not supposed to be done on the properties to flatten up to top level properties. Applicable across the PR.
Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:1651 in 7f20066. [](commit_id = 7f20066, deletion_comment = False)

This change would cause change of SDK's property setup for users. So decided to keep as it is

      "x-ms-identifiers": []

I'm surprised that its not symmetrical with reservationsToExchange, which has the reservationId unique identifier?

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:2236 in 22235cb. [](commit_id = 22235cbe50b6ad860b2ed7c7f383ce1fe57c0c1b, deletion_comment = False)

    },

Why does this savingsPlansToPurchase array not have same x-ms-identifiers as the other?
    },
    "savingsPlansToPurchase": {
      "type": "array",
      "description": "Details of the savings plans being purchased",
      "items": {
        "$ref": "#/definitions/SavingsPlanToPurchaseExchange"
      },
      "x-ms-identifiers": [
        "savingsPlanId"
      ]

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:2245 in 22235cb. [](commit_id = 22235cbe50b6ad860b2ed7c7f383ce1fe57c0c1b, deletion_comment = False)

      "$ref": "#/definitions/SubscriptionScopeProperties"

Replace boolean/switch properties with better enum https://armwiki.azurewebsites.net/rp_onboarding/process/api_review_best_practices.html
Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:1950 in 7f20066. [](commit_id = 7f20066, deletion_comment = False)

This will introduce a type change which is not discussed in the breaking change meeting. keep it as it is for now.

    },

Basically just wondering about the API design here...

In reply to:<PHONE_NUMBER>

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:2245 in 75bd6d3. [](commit_id = 75bd6d3106060937fcfb917f9569d5b9ada12ad2, deletion_comment = False)

      "x-ms-identifiers": []

Basically just wondering about the API design here...

In reply to:<PHONE_NUMBER>

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:2236 in 75bd6d3. [](commit_id = 75bd6d3106060937fcfb917f9569d5b9ada12ad2, deletion_comment = False)

"CatalogsResult": {

x-ms-identifiers need to be added
Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:1477 in 7f20066. [](commit_id = 7f20066, deletion_comment = False)

already added for value list

      "x-ms-identifiers": []

I'm surprised that its not symmetrical with reservationsToExchange, which has the reservationId unique identifier?
Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:2236 in 22235cb. [](commit_id = 22235cb, deletion_comment = False)

The ReservationToPurchase/SavingsPlanToPurchase part contains purchase request, which only contains what type of RI user wants to purchase, so no unique identifier will be presented. It only presents for ReservationsToExchange because that is returning a list of existing reservations. Then in the response of exchange, the purchase request has been received and started in the backend, so a unique id will be generated by then

          "$ref": "#/definitions/AppliedReservations"

SInce this is a collection Get, need pagination on it.

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:165 in 6bae505. [](commit_id = 6bae505c71e21f14a4f97e59a8f3e660062a6bad, deletion_comment = False)

"/subscriptions/{subscriptionId}/providers/Microsoft.Capacity/appliedReservations": {

This is a list call. Please add "x-ms-pageable": { "nextLinkName": "nextLink" },
Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:158 in 7f20066. [](commit_id = 7f20066, deletion_comment = False)

The nextLink is only for the deep nested value inside, not for the response object. When I tried to add, I got error that response object doesn't contain nextLink

Here is the guidance on this from Roopesh "If a lot of elements are expected in an array, model it as a nested child type because you can't provide pagination on a property irrespective of how deep it is. The nested type can then have a LIST call with pagination."

"/subscriptions/{subscriptionId}/providers/Microsoft.Capacity/appliedReservations": {

This is a list call. Please add "x-ms-pageable": { "nextLinkName": "nextLink" },
Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:158 in 7f20066. [](commit_id = 7f20066, deletion_comment = False)

The nextLink is only for the deep nested value inside, not for the response object. When I tried to add, I got error that response object doesn't contain nextLink

Here is the guidance on this from Roopesh "If a lot of elements are expected in an array, model it as a nested child type because you can't provide pagination on a property irrespective of how deep it is. The nested type can then have a LIST call with pagination."

Currently the pagination is happening in body->properties->reservationsOrderIds due to initial design. The change of design to the API would introduce breaking change, that will be none compatible with previous versions, so will keep it as it is for this PR.

        }

In general for Azure apis that return lists, you're recommended not to return object wrappers. Since you're already fixing that for one GET api maybe you can fix it for the POST api also.
https://github.com/microsoft/api-guidelines/blob/vNext/azure/Guidelines.md#collections

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:403 in 2921c73. [](commit_id = 2921c73fed787e1335302caaf9bea1220a867f8e, deletion_comment = False)

      },

In general for Azure apis that return lists, you're recommended not to return object wrappers, not arrays. Since you're already fixing that for one GET api maybe you can fix it for the POST api also.
https://github.com/microsoft/api-guidelines/blob/vNext/azure/Guidelines.md#collections

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:465 in 2921c73. [](commit_id = 2921c73fed787e1335302caaf9bea1220a867f8e, deletion_comment = False)

      },

I'm also unclear about why this returns a list. If merging creates a single reservation, why doesn't it just return one ReservationResponse?

In reply to:<PHONE_NUMBER>

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:465 in 75bd6d3. [](commit_id = 75bd6d3106060937fcfb917f9569d5b9ada12ad2, deletion_comment = False)

    "description": "Moves a `Reservation` to the state it was before archiving.\n",

How bout 'Restores' ?

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:673 in 2921c73. [](commit_id = 2921c73fed787e1335302caaf9bea1220a867f8e, deletion_comment = False)

      "description": "The provisioning sub state of the reservation, e.g. Succeeded",

suggest: sub-state ?

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:2586 in 2921c73. [](commit_id = 2921c73fed787e1335302caaf9bea1220a867f8e, deletion_comment = False)

      "description": "This is the date-time when the Reservation was purchased."

lowercase reservation in docs

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:2598 in 2921c73. [](commit_id = 2921c73fed787e1335302caaf9bea1220a867f8e, deletion_comment = False)

      "description": "Represent the term of Reservation.",

lowercase reservation in docs

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:2631 in 2921c73. [](commit_id = 2921c73fed787e1335302caaf9bea1220a867f8e, deletion_comment = False)

  "put": {

Why is there no "delete" operation?

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:273 in 2921c73. [](commit_id = 2921c73fed787e1335302caaf9bea1220a867f8e, deletion_comment = False)

        "description": "The resource has been created",

resources should return 201 for created

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:302 in 2921c73. [](commit_id = 2921c73fed787e1335302caaf9bea1220a867f8e, deletion_comment = False)

"ReservationResponse": {

Is this a TrackedResurce or a ProxyResource? Looks like its a tracked resource since it has a 'location'. But it doesn't have a "put" verb. Can you refactor this to 'allOf' inherit one of TrackedResource or ProxyResource from common types, to clarify your intention? (and match your RP's manifest)

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:1893 in 2921c73. [](commit_id = 2921c73fed787e1335302caaf9bea1220a867f8e, deletion_comment = False)

  "patch": {

Why support "patch" but not "put"?

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:571 in 2921c73. [](commit_id = 2921c73fed787e1335302caaf9bea1220a867f8e, deletion_comment = False)

    "name": {

Why is this inside 'properties' property-bag and not at the top-level scope, same as a ReservationResponse?
Is it not supposed to be same as the reservation's name/key from the resource path?

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:2880 in 2921c73. [](commit_id = 2921c73fed787e1335302caaf9bea1220a867f8e, deletion_comment = False)

    "x-ms-long-running-operation": true,

specify
    "x-ms-long-running-operation-options":

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:580 in 2921c73. [](commit_id = 2921c73fed787e1335302caaf9bea1220a867f8e, deletion_comment = False)

"/providers/Microsoft.Capacity/calculateExchange": {

specify
"x-ms-long-running-operation": true,
and
"x-ms-long-running-operation-options":

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:872 in 2921c73. [](commit_id = 2921c73fed787e1335302caaf9bea1220a867f8e, deletion_comment = False)

      "description": "Fully qualified id of the Savings Plan order being purchased"

I wouldn't capitalize 'Savings Plan' in the description text sentences - its a documentation context.

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:2492 in d26a64c. [](commit_id = d26a64ccfeb457455ab32d3eaf7020d2149a83af, deletion_comment = False)

  "description": "Represent SavingsPlan term in ISO 8601 format.",

savings plan

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:2511 in d26a64c. [](commit_id = d26a64ccfeb457455ab32d3eaf7020d2149a83af, deletion_comment = False)

      "description": "The Azure Region where the reserved resource lives.",

nit don't capitalize region

Refers to: specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json:2996 in d26a64c. [](commit_id = d26a64ccfeb457455ab32d3eaf7020d2149a83af, deletion_comment = False)

Signing off or ARM, please follow up with the PR assignee about how to resolve any remaining validation errors.

Hi @gaoyp830 in 2022-03-01, Reservation_AvailableScopes is LRO operation,
https://github.com/Azure/azure-rest-api-specs/blob/fce0b25dda01303f2c70de34031169b5d326998b/specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-03-01/reservations.json#L34-L40
But in 2022-11-01, it is not:
https://github.com/Azure/azure-rest-api-specs/blob/f1ef98d76c8d46c2fde8a922b329764f65f3c8a1/specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json#L31-L40
From LRO to non-LRO, it is SDK breaking change. Could you tell us the context? Thanks!
cc @tadelesh @qiaozha

Hi @gaoyp830 in 2022-03-01, Reservation_AvailableScopes is LRO operation,
https://github.com/Azure/azure-rest-api-specs/blob/fce0b25dda01303f2c70de34031169b5d326998b/specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-03-01/reservations.json#L34-L40
But in 2022-11-01, it is not:
https://github.com/Azure/azure-rest-api-specs/blob/f1ef98d76c8d46c2fde8a922b329764f65f3c8a1/specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json#L31-L40
From LRO to non-LRO, it is SDK breaking change. Could you tell us the context? Thanks! cc @tadelesh @qiaozha

Hi @msyyc, this is a get API that always return 200. It was marked as LRO in 2019-04-01 version when it was introduced and that was not correct. So we are removing the LRO flag now. The API is used by our UX, so we wouldn't expect a big impact on SDK users

Hi @qiaozha @tadelesh Please help confirm SDK breaking.

Hi @gaoyp830 in 2022-03-01, Reservation_AvailableScopes is LRO operation,
https://github.com/Azure/azure-rest-api-specs/blob/fce0b25dda01303f2c70de34031169b5d326998b/specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-03-01/reservations.json#L34-L40
But in 2022-11-01, it is not:
https://github.com/Azure/azure-rest-api-specs/blob/f1ef98d76c8d46c2fde8a922b329764f65f3c8a1/specification/reservations/resource-manager/Microsoft.Capacity/stable/2022-11-01/reservations.json#L31-L40
From LRO to non-LRO, it is SDK breaking change. Could you tell us the context? Thanks! cc @tadelesh @qiaozha

@msyyc This API have a slow 95 percentile response time, so we decided to keep it as LRO for now for us to add 201/202 response in future versions if needed.

@chchen3 In addtion to LRO change. The operation "GetCatalog" has been changed to pageable operation. Has this breaking change also been approved? Could you please provide some context? Thanks.

@tadelesh this pr already had breaking change approval

@chchen3 In addtion to LRO change. The operation "GetCatalog" has been changed to pageable operation. Has this breaking change also been approved? Could you please provide some context? Thanks.

@tadelesh Yes, this breaking change is approved by breaking change reviewers. The context for the change is that we are expecting a lot more skus to be returned in the near future. So the current implementation of returning all the skus in one list is limited. We added the paging ability so that when the number of skus is large, we can returned the list in pages. This api also mainly used be by our UX team to show available products for users in portal.


What changes to do to compile on Github

Hi,
I really like the design of the website. Thanks for taking your time to port Tufte CSS to jekyll.
However, one serious problem is that we will not able to use GitHub pages as the repo uses plugins.
Is there a way to work around this?
May be that would be doable without AMP?
If you think doing this yourself is too much work, can you please tell me what all changes that need to be done to get it working, so that I can work on it myself.
Thanks,
Sasank.

You can use it with Github Pages, as long as you compile the files yourself (using serve or build) and upload the compiled HTML/CSS in (the entire _site directory) to Github.

Thanks for your reply.
I am doing that already, but it'd be great to have GitHub compile automatically.

I guess it's possible if you remove the shortcodes file.

Thanks, that should work.
But I still want to use your Sidenote and Image tags.
Is there a way to use them as default footnote and image of markdown?

That's not possible without hacking into kramdown, which is what Jekyll/Immaculate is using as Markdown parser. Alternatively, if you don't mind, you can just write raw HTML in your posts.

Thanks for your time, I will do that !

No problem, glad I was able to help!

Just hit this same problem. I fixed it another way - by using the (I think new?) ability to have the "docs" directory as the site root. Simply set "destination: docs" in your config file and enable this in the repo. Hope this helps someone else! Obv this still requires compiling locally.


chore(workflows): clean up imports

Changes

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#998  👈
#996
#995
#988
#986
#974
#987
#977
#965
#964
#963
#962
#957
#956
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @MasterPtato and the rest of your teammates on  Graphite


What is end date in visit?

Assuming subjid = visit_id always?

Yes, need to establish if subjid = visit_id is the same in other datasets.

In this case, do we need both end_date and date_outcome? If so, what's the difference between the two?

I think date_outcome was one of the options/mapping fields for end_date ... there might be other options/fields for end_date ...

It looks like in the schema at the moment both are attribute options. Maybe it makes sense to have end_date as the only attribute, with date_outcome (dsstdtc) as the preferred field for it? I guess if the visit is referring to admission/followup there won't a outcome date to record, the end date will come from somewhere else...

Sounds sensible ..
On Mon, Feb 20, 2023 at 10:46 AM Pip Liggins @.***>
wrote:

It looks like in the schema at the moment both are attribute options.
Maybe it makes sense to have end_date as the only attribute, with
date_outcome (dsstdtc) as the preferred field for it? I guess if the visit
is referring to admission/followup there won't a outcome date to record,
the end date will come from somewhere else...
—
Reply to this email directly, view it on GitHub
https://github.com/globaldothealth/isaric/issues/45#issuecomment-1436732275,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABW7SJABFQUN5QVZWQNQSE3WYNDPNANCNFSM6AAAAAAUFEVOOA
.
You are receiving this because you commented.Message ID:
@.***>

Fixed by removing end_date in 4736edaa327241856630a25fc289b1270349a6ee


Getting domain during SSR

I am writing an app building on the React + Redux SPA which has a client and an api at different domains, something like this:
[local]
localhost:5000 (client)
localhost:5001 (api)
[web]
app.domain.com (client)
api.domain.com (api)
Now, when writing my fetch requests, I'd like to be able to send requests to the localhost api when I make requests from the localhost client, and the web api when requested from the web client. To do this, I have to be able to set my base url, so I am trying to write a getApiDomain method which I can use to pick the right domain.
let fetchTask = fetch(`${Domain.getApiDomain()}/api/Item/${id}`)...

My method looks like this:
export abstract class Domain {
    public static isLocalhost = Boolean(window.location.hostname === 'localhost' );

    public static getApiDomain(): string {

        return this.isLocalhost
            ? "http://localhost:5001"
            : "http://api.domain.com";
    }
}

This works well for the client, but I can't get it to work on the server during SSR due to the use of window. How can I get the hostname while executing on the server?

Glad you found a solution!


Unable to run introduction tutorial

Running this section of the tutorial
data = {
    "Column1": ["Apple", "Banana", "Cherry", "Date"])
    "Column2": [10, 20, 15, 5],
    "Column3": [True, False, True, False],
    "Column4": [
        pd.Timestamp("2023-08-01"),
        pd.Timestamp("2023-08-15"),
        pd.Timestamp("2023-08-10"),
        pd.Timestamp("2023-08-05"),
    ],
    "Column5": pd.Categorical(["dead", "alive", "dead", "dead"]),
}

df = pd.DataFrame(data)
df

adata = ep.ad.df_to_anndata(df)
adata

produces following error
 ValueError: could not convert string to float: 'Apple'
using versions:

ehrapy '0.6.0'
anndata '0.8.0'

Huh, I'll look into it!

I just tested it again.
Would it be possible for you to create a new environment, install ehrapy in it and try again? I fear that your anndata version isn't the latest and your Pandas version might not be either.

We're also running this notebook as part of our CI: https://github.com/theislab/ehrapy/actions/runs/7533074715/job/20504918714 and it's always working

Upgradin anndata to version '0.10.4' made the trick. Thank you so much!

Great! I'd encourage you to open issues if you have feature requests or run into bugs. This is very early stage and we're happy to take feedback.


Structural Glass

Rename Dynamic Glass to Structural Glass, doesn't store cache, block-based "IStructuralMultiblock" with "isStructure(world, x, y, z, manager)"

Never mind, I'm going to stick to the current multi block dynamic


ReferenceError: Unknown plugin "babel-runtime-es2015-instance-methods" specified in "base" at 1

I installed through npm and added 'babel-runtime-es2015-instance-methods' to my plugin array but I am still getting this error.  What am I doing wrong?

@UX-Unicorn I have not been using and maintaining this plugin for while. However I just tried using it with the latest version of Babel and I works fine. That being said you have to make sure that babel-runtime-es2015-instance-methods/call-instance-method.js is picked up by Babel because otherwise the methods will be missing.
How you ensure that this is the case totally depends on your build setup. If you provide me your configuration (.babelrc, webpack.conf.js, etc.) I will have a look at it.


Nuget.org search results are not based on 'popularity'

From VSFeedback - 554126
Search for 'pinvoke' on nuget.org
it will give you packages in random order where packages with as low as 120 downloads are in top list.
Now, search for 'pinvoke author:andrew',
it will give andrew arnott's pinvoke libraries which has hundreds of thousands downloads.
Should first search query not have listed these packages in top list?

This is a known issue caused by this bug: https://github.com/NuGet/NuGetGallery/issues/3778
Closing as duplicate.


Almost All Image Metrics Save All Images In a Buffer

🐛 Bug
Currently the SSIM module seems to cache all of the predicted and reference images for the entire dataset, computing the metric only at the end of validation.
That's probably not a good idea for a couple of reasons:

Images tend to be quite large, video frames make this even larger. This is a great way to run out of memory.
The code crashes if the images are of different sizes. In other libraries this is solvable with a batch size of 1, but not in torchmetrics

I think most people would compute SSIM over each image in the batch and sum the results over all batches and divide by the total number of images. As far as I can tell there is no advantage to keeping the entire prediction and reference sets in working memory.
This is also true for the following metrics

SpectralDistortionIndex
ErrorRelativeGlobalDimensionlessSynthesis
InceptionScore
KernelInceptionDistance
SpectralAngleMapper
SSIM
MS-SSIM
UniversalImageQualityIndex

Additionally a number of these modules have incorrect help that mentions the wrong metric, almost like someone was just blindly copying and pasting code when the implementing them (far be it from me to make that accusation though)
This module needs a lot of love
To Reproduce

Take two images of different sizes
Call update on them separately
Call compute

Code sample
import torch
from torchmetrics import StructuralSimilarityIndexMeasure

ssim = StructuralSimilarityIndexMeasure()

a = torch.rand(5, 3, 10, 10)
b = torch.rand(5, 3, 20, 20)

ssim.update(a, a)
ssim.update(b, b)

print(ssim.compute())

output:
Traceback (most recent call last):
  File "repro.py", line 12, in <module>
    print(ssim.compute())
  File "/mnt/fsx-outputs-chipdesign/mehrlich/poetry_cache/metabit-wl5Ts0fA-py3.8/lib/python3.8/site-packages/torchmetrics/metric.py", line 531, in wrapped_func
    value = compute(*args, **kwargs)
  File "/mnt/fsx-outputs-chipdesign/mehrlich/poetry_cache/metabit-wl5Ts0fA-py3.8/lib/python3.8/site-packages/torchmetrics/image/ssim.py", line 117, in compute
    preds = dim_zero_cat(self.preds)
  File "/mnt/fsx-outputs-chipdesign/mehrlich/poetry_cache/metabit-wl5Ts0fA-py3.8/lib/python3.8/site-packages/torchmetrics/utilities/data.py", line 42, in dim_zero_cat
    return torch.cat(x, dim=0)
RuntimeError: Sizes of tensors must match except in dimension 0. Expected size 10 but got size 20 for tensor number 1 in the list.

Expected behavior

Environment

TorchMetrics version (and how you installed TM, e.g. conda, pip, build from source): git master, installed with pip
Python & PyTorch Version (e.g., 1.0): 3.8/1.12
Any other relevant information such as OS (e.g., Linux):

I can't even find any discussion about this using git blame because the ssim files have been moved around so many times
Anyway I have a fixed implementation of this that I will open a PR with. It looks like many of your image metrics suffer from this problem, so if you want I can probably fix them all for you.

@Queuecumber thanks for raising this issue.
I am aware that many of our metrics are probably not optimal in some cases. We always welcome contributions to making our metrics faster or more memory efficient.

I have a prototype for SSIM, probably will have something complete for a draft PR early next week

I'm curious if you have any testimonials from people who are using the image metrics in production code
With the current implementation I don't see how it's possible outside of some very specific circumstances

That's definitively possible, I'm still scoping out what exactly is required for the other metrics

Cool, lets take one metric at the time. If we can make due with only saving some intermediate state that would be wonderful as long as the tests are still passing for correctness.

BTW it looks like the "none" reduction is broken for MS-SSIM. If you pass anything larger than a batch size of 1 it crashes because the shapes of things aren't correct.
not sure what you want to do about that

Is there any particular objection to depending on pytorch-mssim instead of reimplementing this?
The current implementation seems to be very broken, pytorch-mssim supports everything that it needs and it is known to be correct, and reinventing this code seems like it's just inviting bugs for no reason.

Is there any particular objection to depending on pytorch-mssim instead of reimplementing this?

I would say that having our own implementation would be perfect; we just need to be careful of not copy their code for eventual licencing issues

Yes I was able to get a correct implementation of both without needing the extra dependency

@Queuecumber, can we look forward to some more contributions? :rabbit:

Yes, CVPR deadline tomorrow after that I should have more time to work on the rest of these


Add a literal fragment specifier to macro_rules!.

Add a literal fragment specifier for macro_rules! patterns that matches literal constants.

I've been missing this, for example in a macro that uses consecutive integer literals like shuffle![v, 1 2 3 4]. You can work around it using tt.

Aye; there's an important alternative not mentioned in the RFC: use tt instead.
The problem I can see with this proposal is that it doesn't really simplify existing macros, nor open up new ones.  The lack of backtracking means that even if this matcher is more specific than tt, it won't permit macros that wouldn't have been expressible before.
This will probably give better error messages on matching failures, but nothing else I can think of.  No, wait, it does have one advantage over tt: you won't need the reparse trick, since you should be able to assume a literal is an expression.  ... still not sure that's enough to pull its weight, though.

@DanielKeep The weight should be that big. And backtracking is coming with rust-lang/rust#33840. The better error messages are a big gain. :/

(Edited to mention the possibility of using tt.)

cc @jseyfried in case you have an opinion on this

also cc @cswords

I'm in favour of this, though slightly (and vaguely) worried about the implementation details

I'm also in favour of this.
I don't think implementation will be a problem, but I don't know the details of our fragment implementation that well.
n.b. the reparse trick is no longer needed, so that's no longer an advantage over tt.

Hear ye, hear ye! This RFC is now entering final comment period. We discussed this at the most recent @rust-lang/lang meeting and decided that we are currently inclined to accept (of course no final decision has been reached).
To summarize the discussion so far:

one can currently use tt for this purpose, but the error messages are less good

I like this RFC.
Would it make sense to also have more specific versions for specific kinds of literals?  Several times, I've wanted to have a macro accept a string literal specifically.

I don't think implementation will be a problem, but I'm not too familiar with the details of how fragments are implemented.

It's actually very simple, it just calls the parse_whatever() functions from libsyntax::parse::parser.

one can currently use tt for this purpose, but the error messages are less good

And the future-proofing is less good too. Using a more specific matcher restricts the FIRST sets, which is good.

Would it make sense to also have more specific versions for specific kinds of literals? Several times, I've wanted to have a macro accept a string literal specifically.

I guess it would make sense, though the gain in terms of error messages is small. I'm not sure if it's possible though, since I think the libsyntax parser has a single parsing function for all literal types. It can surely be refactored and split into several functions, but I'm not sure it's worth it.

Would it make sense to also have more specific versions for specific kinds of literals? Several times, I've wanted to have a macro accept a string literal specifically.

Yeah, I wondered the same. Could be useful, but I guess there is value in "any literal" as well, and it's something we could easily add later.

@nrc Absolutely!  To clarify, I'd love to see the proposed RFC accepted, and I don't want to bikeshed it.  I'd also love to see specifiers for specific types of literals, which could either go in this RFC or a future RFC.

Huzzah! The @rust-lang/lang team has decided to accept this RFC.

Tracking issue: https://github.com/rust-lang/rust/issues/35625
If you'd like to keep following the development of this feature, please subscribe to that issue, thanks! :)

Dear all, the implementation is now in master :)


#[help] attribute is inconsistent in struct and field positions

I would expect these two error types to be equivalent:
#[derive(Debug, Error, Diagnostic)]
#[error("error!")]
struct MyError {
    #[help]
    help_message: String,
}

#[derive(Debug, Error, Diagnostic)]
#[error("error!")]
#[help("{help_message}")]
struct MyError {
    help: String,
}

however in the second one, the #[help] attribute is silently ignored. Instead, we have to use #[diagnostic(help("{help_message}"))], which seems inconsistent (especially since that form doesn't work in field position)
Ideally, a consistent attribute would work for both, and the derive macro would fail if an attribute is being ignored.

My understanding is that this is the convention used by other derive macros, like thiserror: prefix the struct-level arguments, but don't necessarily prefix the field-level ones, no?

Ah, I didn't realize there was a convention like that.
In any case, I think the main issue is just that attributes can get silently ignored. It would be nice to be able to tell the user that they've used the wrong one.


Keyboard shortcuts for selecting the primary sidebar items

Like Mail and NetNewsWire, it would be great to be able to navigate to the primary sidebar items / app sections (All Bookmarks, Today, Untagged, etc) using keyboard shortcuts. The natural choice for this would be Cmd + Number (Cmd+1, Cmd+2, Cmd+3, etc). These actions should also be available in the menu bar.
Here's an example of how NetNewsWire does this:

Done.


slow highlight performance

I am very satisfied with the speed of editing.
But, Highlighting is slow when code is large.
When entering new code after several thousand lines of code, highlighting takes more than a few seconds.
Is there any solution?

@ellie-k-nam Hello, thanks for your feedback. In fact, this is a known issue. Syntax highlighting requires a large number of regular matches, but the editor does not cache the results, resulting in many meaningless repeated executions. It is feasible to consume some storage space in exchange for efficiency, so I need to modify code of re-highlight. However, the code of re-highlight is completely synchronized with highlightjs, I don't plan to make additional changes.

I just started using this library, and this was the first thing I noticed too. (After the fact that it has built-in support for horizontal scrolling :partying_face:)
It's not a dealbreaker for my project, but it would be nice if it could be improved somehow...


v0.1.12 stopped rendering in WebStorm 2022.2.3

The plugin initially worked in WebStorm, then all of a sudden it stopped rendering diagrams and displays just XML

Plugin version: 0.1.12
WebStorm 2022.2.3
MacBook Pro M1, macOS 12.6.1 Monterey

Thanks for reporting this. Could you please look at the bottom of the editor window, There should be two tabs for the two editors ... can you switch to diagrams.net editor there?

I actually just figured it out and was about to delete this issue. Thanks!


Error creating scheduled periodic actions

Description
The creation of a periodic scheduled VM action without end is failing with the following error:
[one.vm.update] Cannot update template. Error END_VALUE defined but not valid END_TYPE found

To Reproduce
Try to schedule a repeatable action, snapshot-create for example:

Select snapshot-create from the actions list
Tick the Periodic checkbox
Set (for example) Weekly, Monday
Leave "Ends" radio button to Never
Click Add

The above error is logged
Expected behavior
The VM action should be scheduled
Details

Affected Component: [Sunstone]
Hypervisor: [KVM]
Version: [5.10.2]

Additional context
The following details are logged in the oned log:
Wed Mar  4 15:27:06 2020 [Z0][ReM][D]: Req:1360 UID:0 IP:<IP_ADDRESS> one.vm.update invoked , 221, "INPUTS_ORDER = ""
LOGO = "images/logos/centos.png"
MEMORY_UNIT_COST = "MB"
SCHED_DS_REQUIREMENTS = "ID=\"100\" | ID=\"104\""
SCHED_REQUIREMENTS = "CLUSTER_ID=\"102\" | CLUSTER_ID=\"101\""
SCHED_ACTION = [
  END_TYPE = "",
  END_VALUE = "1",
  TIME = "1583317800",
  DAYS = "0",
  REPEAT = "",
  ACTION = "snapshot-create",
  ID = "0" ]
", 0
Wed Mar  4 15:27:06 2020 [Z0][ReM][E]: Req:1360 UID:0 one.vm.update result FAILURE [one.vm.update] Cannot update template. Error END_VALUE defined but not valid END_TYPE found

Progress Status

[ ] Branch created
[ ] Code committed to development branch
[ ] Testing - QA
[ ] Documentation
[ ] Release notes - resolved issues, compatibility, known issues
[ ] Code committed to upstream release/hotfix branches
[ ] Documentation committed to upstream release/hotfix branches

Hello @atodorov-storpool,
I think this is solved here: https://github.com/OpenNebula/one/issues/4380. If not, please reopen.


Can we clean multiples files using globs ?

If I have css files than I would like to clean like so:
css/
  style.css
  contact/
    contact.css

Can I use clean-css-cli like:
cleancss --level 1 --output css/**/*.min.css css/**/*.css

I do not want to list all my css in the command line or neither to edit the command line every time I add a new css file.
What I am aiming for is this result:
css/
  style.css
  style.min.css
  contact/
    contact.css
    contact.min.css

I could also accept having the source css file overwritten...
Thank you for your help.

I was able to do it thanks to a comment inside the issue #18 :
https://github.com/jakubpawlowicz/clean-css-cli/issues/18#issuecomment-372310146
This npm script will replace the files...
"css-minify-all": "foreach -g \"css/**/*.css\" -x \"cleancss --level 1 #{path} -o #{path}\" --no-c"

Please note it can be now done via --batch option in clean-css-cli v5: see https://github.com/jakubpawlowicz/clean-css-cli#how-to-process-multiple-files-without-concatenating-them-into-one-output-file


Errors in the log when creating a site using the Fitness bp

Describe the issue
There are errors in the log when creating a site using the Fitness bp.
Steps to reproduce

Create a site in Studio using the Fitness bp from the marketplace
Watch the catalina log and notice the errors

[ERROR] 2019-10-16T14:21:20,460 [http-nio-8080-exec-1] [dependency.RegexDependencyResolver] | Unexpected error resolving dependencies for site: fitness path: /site/components/footer/footer1.xml
[ERROR] 2019-10-16T14:21:20,471 [http-nio-8080-exec-1] [dependency.RegexDependencyResolver] | Unexpected error resolving dependencies for site: fitness path: /templates/web/components/footer.ftl
[ERROR] 2019-10-16T14:21:20,494 [http-nio-8080-exec-1] [dependency.RegexDependencyResolver] | Unexpected error resolving dependencies for site: fitness path: /config/studio/content-types/component/component-footer/form-definition.xml

Expected Behavior
There should be no errors when creating a site using the Fitness bp
Logs
Here's a complete log for when the site was created using the Fitness bp from the marketplace
https://gist.github.com/alhambrav/21fec2b22d65abeaa6b7b12888b224cd
Specs
Version
Studio Version Number: 3.1.4-SNAPSHOT-31df36
Build Number: 31df362f307966926633b458806695069f844f77
Build Date/Time: 10-16-2019 10:01:14 -0400
OS
OS X
Browser
Chrome browser

Closing the ticket.  Errors are gone after fix to create site as an orphan in Studio.


ApplicationInsights:Invalid JSON config file when using APPLICATIONINSIGHTS_CONNECTION_STRING

Environment:
OS: NixOS 23.05 (Linux)
Node: 18.17.1
applicationinsights version: 2.9.0
After upgrading the applicationinsights package in my project from 2.7.3 to 2.8.0 and then to 2.9.0 when running my NodeJS application using the APPLICATIONINSIGHTS_CONNECTION_STRING variable as done prior to 2.8.0 I get the warning:
[1] ApplicationInsights:Invalid JSON config file:  [
[1]   SyntaxError: Unexpected end of JSON input
[1]       at JSON.parse (<anonymous>)
[1]       at r._loadJsonFile (/home/keanu/Documents/Development/redacted/dist/redacted.js:2:5417)
[1]       at new r (/home/keanu/Documents/Development/redacted/dist/redacted.js:2:4957)
[1]       at r.getInstance (/home/keanu/Documents/Development/redacted/dist/redacted.js:2:5038)
[1]       at /home/keanu/Documents/Development/redacted/dist/redacted.js:7:72232
[1]       at /home/keanu/Documents/Development/redacted/dist/redacted.js:1:263
[1]       at /home/keanu/Documents/Development/redacted/dist/redacted.js:12:32841
[1]       at /home/keanu/Documents/Development/redacted/dist/redacted.js:1:263
[1]       at /home/keanu/Documents/Development/redacted/dist/redacted.js:43:26129
[1]       at /home/keanu/Documents/Development/redacted/dist/redacted.js:1:263
[1] ]

the warning is not present during build and goes away if i provide the applicationinsights.json however this functionality looks to be opt-in via the APPLICATIONINSIGHTS_CONFIGURATION_FILE variable which I've not set.
When the application is running after the warning, the live metrics page doesnt load and reports no telemetry is being received, this works again as expected when downgrading to 2.7.3.
My application insights initialisation looks like the following:
applicationInsights.setup()
  .setSendLiveMetrics(true)
  .setAutoCollectConsole(true, true)
  .setAutoCollectDependencies(true)
  .setAutoCollectExceptions(true)
  .setAutoCollectHeartbeat(true)
  .setAutoCollectPerformance(true)
  .setAutoCollectPreAggregatedMetrics(true)
  .setAutoDependencyCorrelation(true)
  .start();

@kashw2 are you bundling your code with the applicationinsights SDK code?, the code always try to read the file, and by default it will read the applicationinsighs.json that is part of the actual npm package, I suppose this could be a problem if there is some bundling involved

Hi, yes I'm currently using esbuild to bundle. Something odd that I only just noticed when reproducing this issue again was that the location that it was requiring the applicationinsights.json file to be was actually not part of the application directory itself and was in the parent directory.
The project I work on is apart of a monorepo with the project in question having a layout like:
myProject
|
------ V1
|
------ V2

When reproducing this issue now, it actually asks for the applicationinsights.json file to be present in the myProject directory instead of the V2 directory that I am working on and building out of.

I'm actually experiencing the same issue. It's asking for the config file located outside my project directory. I have downgraded to version 2.7.3 and the error has disappeared.
I'm trying to isolate the changes that triggered the error and will update once I have found it.

Hi, yes I'm currently using esbuild to bundle. Something odd that I only just noticed when reproducing this issue again was that the location that it was requiring the applicationinsights.json file to be was actually not part of the application directory itself and was in the parent directory.
The project I work on is apart of a monorepo with the project in question having a layout like:
myProject
|
------ V1
|
------ V2

When reproducing this issue now, it actually asks for the applicationinsights.json file to be present in the myProject directory instead of the V2 directory that I am working on and building out of.

Experiencing something similar - did you find a workaround? @kashw2

We have been able to overwrite using: process.env.APPLICATIONINSIGHTS_CONFIGURATION_CONTENT = '{}'
per docs:

Alternatively, instead of using a configuration file, you can specify the entire content of the JSON configuration via the APPLICATIONINSIGHTS_CONFIGURATION_CONTENT environment variable .

Just got a chance to check the solution from @jakereifer.
While this does remove the warning during build, the issue of telemetry data not being sent to Application Insights and viewable in the Live Metrics pane still persists. Is this also the case when you attempted this?

@kashw2 this still sends telem to app insights for me - we're also using @vscode/extension-telemetry which depends on applicationinsights. not sure if that changes anything

@kashw2 are you bundling your code with the applicationinsights SDK code?, the code always try to read the file, and by default it will read the applicationinsighs.json that is part of the actual npm package, I suppose this could be a problem if there is some bundling involved

@hectorhdzg: This is definitely a problem for us. We are also bundling the code (with esbuild) and unless we explicitly exclude applicationinsights from bundling we get this error message.
And for us it also looks in a folder which is one level above from where our application is in. Maybe this could even be used as an attack vector, if a file is placed there, to divert the telemetry data to a different place.
How a file from the source code is dynamically loaded during runtime in https://github.com/microsoft/ApplicationInsights-node.js/blob/0217324c477a96b5dd659510bbccad27934084a3/Library/JsonConfig.ts#L129-L131 really looks like a bug to me. This could be fixed e.g. by instead requiring that file statically and using that instead. With this in place bundlers could also handle this correctly.


Loki Query is limtied to 100 log lines only

Expected Behavior
Logs greater than 100 should be visible.
Actual Behavior
Only last 100 logs lines are present..
v0.12.0 of Results + LokiStack

Resolved by https://github.com/tektoncd/results/pull/823


"cytotrophoblast": use CL "trophoblast cell" or "mononuclear cytotrophoblast cell"

CL for variations of cytotrophoblast such as "cd49f_positive_cytotrophoblast", "cytotrophoblast cell" and "hla-g_positive_extravillous_cytotrophoblast" is not available, but "trophoblast cell" and "mononuclear cytotrophoblast cell" are.
example ids: "IHECRE00003716.2", "IHECRE00004591.1", "IHECRE00004721.1"

It is okay to change all to “trophoblast cell.”
(Naoto Kubota)

done.


[WIP] feature: add pkg/plugins for dragonfly

Signed-off-by: lowzj<EMAIL_ADDRESS>
Ⅰ. Describe what this PR did
Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)? (你真的觉得不需要加测试吗？)
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

Codecov Report

Merging #1376 into master will increase coverage by 0.43%.
The diff coverage is 93.54%.

@@            Coverage Diff             @@
##           master    #1376      +/-   ##
==========================================
+ Coverage   51.88%   52.31%   +0.43%
==========================================
  Files         135      137       +2
  Lines        8803     8896      +93
==========================================
+ Hits         4567     4654      +87
- Misses       3859     3863       +4
- Partials      377      379       +2

Impacted Files
Coverage Δ

pkg/plugins/plugin_manager.go
91.42% <91.42%> (ø)

pkg/plugins/plugins.go
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5981693...8f75870. Read the comment docs.


MCPWM MVP implementation

Major things missing:

[x] Need to resolve TODOs
[x] Documentation
[x] Test on ESP32 Hardware
[x] Test on ESP32-S3 Hardware
[ ] something else?

Request for feedback:
Are we allowed to copy text passages from the reference manuals for documentation? What about pictures?
The MCPWM peripheral is quite complex with lots of moving parts so adding some graphics might help.
What are people thinking about the API design?
I chose an implementation without looking much at the existing LEDC code or the ESP IDF API.
There might be some disconnect here.
Things I tried to consider were (in no particular order):

compile errors over runtime errors
"zero-cost"-ness
use of const-generics with the option of using adt_const_params at some later point
ability to add more features later without major architectural changes
discoverability

I'd love to have some feedback here whether these goal were accomplished and if other considerations were missed.
What's the pwm period of a Count-Up-Down Mode timer?
The docs  mention:
> The PWM timer starts increasing from zero until the period value is reached. Then, the timer decreases back to zero. This pattern is then repeated. The PWM period is the result of (the value of the period field × 2 + 1)
Looking at the timing diagrams is seems to me that it would rather be period field × 2. Is that a mistake in the docs?
Doing some measurements confirmed that the docs indeed are wrong here the period is period field × 2

Are we allowed to copy text passages from the reference manuals for documentation? What about pictures?

I guess it's okay since the docs are Apache 2.0 license but I can ask internally.
Not looking into the LEDC driver is definitely a good thing since the API is not what we really want. It's just a port of the former attempt on creating an esp32-hal 👍

compile errors over runtime error
is definitely something we want

In general, looks good to me! Especially if the initial architecture fits things can be changes and extended later.
One thing I spotted: Currently some functions take clock dividers - would be more aligned with other APIs if they would take fugit::Rate

Didn't get an answer to the license question, yet

If you'd like I can add Rate taking alternatives, but I'd like to keep the divider taking ones, if that's ok :)

I see your concerns. I think a Rate taking alternative should be added since in general all the other drivers take Rate
I think it's in general not decided yet if we want to follow this "new" pattern (have the choice of using Rate and naked dividers) - at least there is a comment that should have started this discussion: https://github.com/esp-rs/esp-hal/issues/174#issuecomment-1232661991

I think this is ready for review.
Let me know if and how I should squash the commits.
For the most part this should be a good starting point for the HAL.
Things like timer sync, capturing, dead-time or carrier modulation should be easily addable in the future.
One thing I'm not sure about is how to make it possible to release the peripheral. I think you'd either have to stick a lifetime into everything or provide methods that collect all the parts. Both seem unergonomic...

Sorry for the delay, thanks for the updates. I will review the code and verify this with a logic analyzer today or tomorrow.
If you would like to squash the commits down a bit you're more than welcome to; we don't really have a policy regarding this, less than 10 would be nice I guess, but just whatever makes sense is fine.

Code looks fine to me - well documented! Thanks for the contribution

Guess I don't squash it then, haha.
Thanks a lot for reviewing and the kind words!

arghhh .... 🤦♂


Last update broke some brushes

It looks like the one of the recent updates broke the pencil, brush and claw brushes. There's no output on those strokes.

Have you updated MaLiang to the latest version? I'm not sure witch version are you currently using.

I meet the issue with the latest version.

@Joey-Lee Have you tried to clean the build folder in Xcode, there are always some building cache causing strange problems


Hashes missing in output file

Hi,
If I have a text file which has X number of hashes and Im looking them up with nsrllookup, it produces two files, one with the known hashes eg k.txt and another unknown u.txt. However the sum of hashes in both files is always less than X. It happens when i lookup 50 hashes or 5000. I think the RDS hits would be in the known files and the misses will be in the Unknown. It happens both when I use my own instance of nsrlsvr and nsrllookup.com Can you please help me with this?

I'm sorry, for some reason I didn't see this issue before.
There is a good reason for the difference in results.  If there are multiple copies of a hash in the input set, redundant hashes will be deduplicated.
If you have a set of X unique hashes, and you're getting sets k and u which do not add up to X, then we've got a problem.  Can you look into this for me and tell me what's going on?

@rjhansen Hi Rob,
The fault was on my part, there were non unique hashes in the file. Thank you for the guidance.


During initialization, invalid video links or network disconnections can cause crashes

Provide an invalid video address (empty string or a non-existent URL address) or disconnect from the network, which will crash during initialization
Launching lib\main.dart on Windows in debug mode...
Building Windows application...
√  Built build\windows\x64\runner\Debug\video_player_win_example.exe.
Debug service listening on ws://<IP_ADDRESS>:3230/cuDVVJ-LYFw=/ws
Syncing files to device Windows...
HandleMethodCall: openVideo
[video_player_win] isOpenVideo: 1
[video_player_win] path: https://files.acadsoc.com.cn/iesfiles/TempVideo/20230625/0f106ae4-301f-4027-a70f-edcc132d1001q.mp4
[video_player_win] convResult: 99
[video_player_win]: textureId=2129078886064
[video_player_win]: OpenURL end:0
[video_player_win]: OpenURL isSuccess =0
[native] ~MyPlayer()
Lost connection to device.

Hi @jakky1 , i think that the player should be able to open video files that have no common extension.
For example i receive the error "controller intialize (open video) failed" when passing an mp4 file with .phv extension

About file extension, I have no idea how to solve it. So I won't fix it.
If crash no longer occurs, I will close this issue.

@Andrekarma
About file extension, I have no idea how to solve it. So I won't fix it.
And please open a NEW issue if your question is not related to current issue.


Remove unused function

The child function it calls also no longer exists.

https://ocd.appscale.com:8080/job/Daily Build/1169/


Script editor search: F3 hotkey for "next" conflicts with 3D viewport

Operating system or device - Godot version:
Linux x64, Godot master
Issue description:
When doing a text search in the script editor, F3 (supposed to highlight the next match) goes to the 3D viewport. The script editor shortcut should take precedence over the editor node one.
In 2.1 it used to work by chance as F3 was the shortcut for the script editor itself.
Steps to reproduce:

Ctrl+F, search something with several matches
Press F3

"Fixed" by 5e361ba.


Logged In Tabbing tests fail depending on test order

E2E tests should work independently (ie. not rely on data set by other tests) so that proceeding tests don't fail if a preceeding test changes the user data.

Fixed with a new .pressTab() custom method.


Added requirements

I added the requirements.txt file, which is needed to quickly install dependencies. I also updated the README.md file with clearer instructions for installation.

I did things in a more userfriendly way by adding the --output argument, so you can do more tests without deleting old output

Thank you for this!


Update README dependency example to 0.12

Fixes #43

Coverage decreased (-0.02%) to 93.689% when pulling 71b898e2c91ce5895b590a0499b0262a00f9a890 on parrotmac:patch-1 into faa4397ba7c5f441cb9a2b436c1e84a0d52ae942 on kennytm:master.


[FEAT]: CCM 15811

Harness Developer Pull Request
What Type of PR is This?

[  ] Issue
[ ] Feature
[x ] Maintenance/Chore

If tied to an Issue, list the Issue(s) here:

Issue(s)

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[ ] Tested Locally
[ ] Optional Screen Shoot.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65a8b14dec9a243418b0412b--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65a8e6ba976b0958134adde8--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65a8e7df48140f5b53e60523--harness-developer.netlify.app


Guys please realize your money is scammed

You guys took in hundreds of BTC during the ico. These excuses are so lame!!! Just exit already.

If you dont have patient then sell your coins mate, why create FUD for nothing, did you see the number of tickets and number of instances where devs are helping ?
I care for my investment and so do others..so keep the FUD to yourself.

just sell if you don't like monyx, don't worry real investors will buy your coins
Best for you m8, @debow22

@debow22  Mate, can i please ask you to close the ticket. If you want to Vent your frustration please do it on Telegram non official group..atleast leave Github Issues separate.

PLEASE CLOSE THIS TICKET
cry babies shouldnt be in crypto full stop lol sell up an move on champ

I, the developer, am pretty sure the platform won't be closed and money are not taken by the team, since we still have some communication. There is a delay with announcing new information, because it took more time for them to determine a new direction, start preparing things, etc.
Unfortunately it seems we'd need to wait like another week, but again, we have pretty big amount of premined MYX as founders and it's really in our interest to make the price higher (who would want to lose like 6 figures potential earnings?).
So I'm sorry it takes so long, there is nothing I myself can change, only keeping the exchange working and fixing bugs.


fix macOs cpu_count

fix macOs lscpu  ps aux
🔍 What type of PR is this?
macos bug fix

I have read the CLA Document and I hereby sign the CLA


Server is not serving as a listener

I've created a plugin for my custom purpose. And this plugin reads some data from external service and pushes to RSI server. But whenever a new data is comes, server rebuild and restart is required. How can i change this?

Can you share what you built? It is hard to help otherwise
On Mon 12. Mar 2018 at 11:46 arjunkoroth<EMAIL_ADDRESS>wrote:

I've created a plugin for my custom purpose. And this plugin reads some
data from external service and pushes to RSI server. But whenever a new
data is comes, server rebuild is required. How can i change this?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/wzr1337/rsi.server/issues/75, or mute the thread
https://github.com/notifications/unsubscribe-auth/ADiF7eXeqdLm_dl6pDOC4Tc3BBoLxPK_ks5tdlIGgaJpZM4Sma11
.

--

Patrick (Mobile)

` getObdData() {
    this.client.on('data', (data: any) => {
       // console.log(data.toString());
        this.callback(data.toString());
        // client.end();
    });

}
callback(data: any) {
    if (data != null && data != undefined) {
        let obdRenderer = new BehaviorSubject<ObdElement>({
            lastUpdate: Date.now(),
            propertiesChanged: [],
            data: {
                uri: "/" + this.service.name.toLowerCase() + "/" + this.name.toLowerCase() + "/" + ObdData.obdDataId,
                id: ObdData.obdDataId,
                name: "OBD",
                speed: JSON.parse(data).speed,
                rpm: JSON.parse(data).rpm,
                engineLoad: JSON.parse(data).engineLoad,
                ect: JSON.parse(data).ect
            }
        });
        this.obdRenderer.push(obdRenderer);
    }
}`

I've built a plugin to listen an external server through TCP. Plugin reads the data from external server and pushes it to RSI server.
see,
callback(data: any) {         if (data != null && data != undefined) {             let obdRenderer = new BehaviorSubject<ObdElement>({                 lastUpdate: Date.now(),                 propertiesChanged: [],                 data: {                     uri: "/" + this.service.name.toLowerCase() + "/" + this.name.toLowerCase() + "/" + ObdData.obdDataId,                     id: ObdData.obdDataId,                     name: "OBD",                     speed: JSON.parse(data).speed,                     rpm: JSON.parse(data).rpm,                     engineLoad: JSON.parse(data).engineLoad,                     ect: JSON.parse(data).ect                 }             });             this.obdRenderer.push(obdRenderer);         }     }
I can subscribe it from client side through http://localhost:300/<some_url>. But, whenever a new data pushes by the plugin to RSI server, it won't reflect in the client side. For that , i need to rebuild and restart the server.

it seems you are not telling the framework which properties changed becaus you hand it an empty array. We chose to explicitly tell the framework because their are situations where you want to postpone a notification until you collected all data.
Let me know if that helps
      if (data != null && data != undefined) {
          let obdRenderer = new BehaviorSubject<ObdElement>({
              lastUpdate: Date.now(),
              propertiesChanged: ["speed", "rpm", "engineLoad", "ect"],
              data: {
                  uri: "/" + this.service.name.toLowerCase() + "/" + this.name.toLowerCase() + "/" + ObdData.obdDataId,
                  id: ObdData.obdDataId,
                  name: "OBD",
                  speed: JSON.parse(data).speed,
                  rpm: JSON.parse(data).rpm,
                  engineLoad: JSON.parse(data).engineLoad,
                  ect: JSON.parse(data).ect
              }
          });
          this.obdRenderer.push(obdRenderer);
      }
  }

No it won't fixed,

See the below attached image. There is a TCP application running on port 4000 and i've created a plugin named animalTest for RSI server which listens on the TCP application. I need to push the data immediately to the server when it changes in TCP application.
Please see the second image,

contents of index.ts
`
import { BehaviorSubject } from '@reactivex/rxjs';
import { AnimalDataObject } from "./schema";
import * as uuid from "uuid";
import { Service, Resource, Element, ResourceUpdate, StatusCode, ElementResponse, CollectionResponse } from "../../plugins/rsiPlugin";
import { Socket } from 'net';
import { setTimeout } from 'timers';
class AnimalService extends Service {
constructor() {
super();
this.id = "test-plugin-2000";
this.resources.push(new AnimalData(this));
}
}
interface AnimalElement extends Element {
data: AnimalDataObject;
}
class AnimalData implements Resource {
private static animalDataId = "d6ebfd90-d2c1-11e6-9376-df943f51f0d8"; //fixed id for webbsocket subscription. don't knw why :-(
private animalRenderer: BehaviorSubject[] = [];
private _change: BehaviorSubject;
//for obd listening
private net: any = require('net');
private client: any = null;
////////////////////////////

constructor(private service: Service) {
    this.client = this.net.createConnection({ port: 4000 }, () => {
        console.log('connected to server on port : 4000');
    });
    this.getObdData();
}

get name(): string {
    return this.constructor.name;
};

get elementSubscribable(): Boolean {
    return true;
};

get change(): BehaviorSubject<ResourceUpdate> {
    return this._change;
}

getElement(elementId: string): ElementResponse {
    var data = this.animalRenderer.find((element: BehaviorSubject<AnimalElement>) => {
        if (element) return (<{ id: string }>element.getValue().data).id === elementId;
        return undefined;
    });
    return data ? {
        status: "ok",
        data: data
    } : undefined;
};

getResource(offset?: string | number, limit?: string | number): CollectionResponse {
    let resp: BehaviorSubject<AnimalElement>[];
    if ((typeof offset === "number" && typeof limit === "number") ||
        (typeof limit === "number" && !offset) || (typeof offset === "number" && !limit) || (!offset && !limit)) {
        resp = this.animalRenderer.slice(<number>offset, <number>limit);
    }

    return { status: "ok", data: resp };
};

getObdData() {

    this.client.on('data', (data: any) => {
        this.callback(data.toString());
    });

}
callback(data: any) {
    if (data != null && data != undefined) {
        let anim = new BehaviorSubject<AnimalElement>({
            lastUpdate: Date.now(),
            propertiesChanged: ["animalname", "genus", "genus"],
            data: {
                uri: "/" + this.service.name.toLowerCase() + "/" + this.name.toLowerCase() + "/" + AnimalData.animalDataId,
                id: AnimalData.animalDataId,
                name: "OBD",
                animalname: JSON.parse(data).animalName,
                genus: JSON.parse(data).genus,
                species: JSON.parse(data).species
            }
        });
        this.animalRenderer.push(anim);
    }
}

}
export { AnimalService as Service, AnimalData }
schema.d.ts import {xObject} from "../../types";
export interface AnimalDataObject{
id: string;
name: string;
uri: string;
animalname: string;
genus: string;
species: string;
}
`

Reading your code, I understand that:

You want to offer a resource animaldata
in this rescource, you create an element
you want to update the element once you receive TCP data

By definging the elementSubscribable property as true you tell the framework that elements may change over lifetime => all good
In your callback(), you create new elements everytime you receive data. This is not what you want to do, because you want to update the existing element.
Solution: keep a handle to your element (this.anim = new BehaviorSubject<AnimalElement>...) then you want to call .next() on this.anim to publish updates.
Thinking it trough brought me to the conclusion that the frame work should report an error if you push an element with an aready known id (https://github.com/wzr1337/rsi.server/issues/77).

So how about this.animalRenderer.push(anim); ? Is it required to change ?

no, not for your usecase
this would publish a NEW element on the resouces, thus a resource event would be fired, not an update on elment level

this.anim = new BehaviorSubject<AnimalElement>... does it need to make a class member? how is the use of next() ?

Hi
Can you just explain the use of next() by keeping a handle on this.anim = new BehaviorSubject<AnimalElement>... ?

Finally found the solution. Thanks for your support.

Thanks around,
As this an open source effort.. May I kindly ask you to write a short
howto.md add it to throat level and send a pull request?
This would help a lot of people I think
On Wed 25. Apr 2018 at 11:47 Arjun K<EMAIL_ADDRESS>wrote:

Finally found the solution. Thanks for your support.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/wzr1337/rsi.server/issues/75#issuecomment-384228879,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADiF7Z8P5NxhwA8eCGeL012QyOubNXdqks5tsEYkgaJpZM4Sma11
.

--

Patrick (Mobile)


Add kernelCTF CVE-2024-1085_cos

Hey!
I took a look at the exploit and I had to request a few changes (36 comments in total...) as I think some parts of the exploit could be explained better and there are some other code quality issues.
Please take a look and make the necessary changes.
Thanks!

Hi,
I've updated the exploit.c to add all the changes. Please check it.
Thanks

Hey! There are still several unresolved comments (23 out of the original 36 comments). Could you please fix those too? Thanks!

Hey! There are still several unresolved comments (23 out of the original 36 comments). Could you please fix those too? Thanks!

Hi, I've fixed all the comments, please check it! Thanks!


fill with all UTF-8 chars and longer selectors

This request addresses two issues:
1.
Currently only ASCII chars are working for method "fill".
The reason has been using size instead of bytes.length which lead to playwright driver error if msg had longer byte size than advised before.
This change will allow also multiple byte chars in UTF-8 for fill.
2.
Larger values for fill or selectors lead to "Encoding::UndefinedConversionError: "\x.." from ASCII-8BIT to UTF-8" if the message length became longer than 128.
If @stdin file ist set to bin mode than also larger integer values may be transferred without conversion errors.
This change fixed the problems I had with UTF-8 and larger element IDs, but may be I haven't yet recognised the whole picture.

Thank you very much for your contribution.
  it 'should type all kinds of characters', sinatra: true do
    with_page do |page|
      page.goto("#{server_prefix}/input/textarea.html")
      page.focus('textarea')
      text = 'This text goes onto two lines.\nThis character is 嗨.'
      page.keyboard.type(text)
      expect(page.eval_on_selector('textarea', 't => t.value')).to eq(text)
    end
  end

  it 'should type emoji', sinatra: true do
    with_page do |page|
      page.goto("#{server_prefix}/input/textarea.html")
      text = '👹 Tokyo street Japan 🇯🇵'
      page.type('textarea', text)
      expect(page.eval_on_selector('textarea', 'textarea => textarea.value')).to eq(text)
    end
  end

I confirmed the fix with the RSpec above. (copied from https://github.com/microsoft/playwright/blob/master/tests/page-keyboard.spec.ts )


Add TargetFramework .net6.0

We have Maui app, where we use ZXing.Net.Maui
But with this project we cannot use targetframeowrk .net6.0 and do unit testing...
Can you add this target framework please?

you will have downgrade to the latest 0.2.0-preview

@g0dpain There is not .net6 in target framework also

Same problem here.
Are there any workarounds for unit testing scenarios?
Thank you.

I believe this is a duplicate of https://github.com/Redth/ZXing.Net.Maui/issues/50

@Redth  any update on this adding target framework net7.0 for unitest ?


Youtube: authentication failure

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like this: [x])
Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.08.04. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x ] I've verified and I assure that I'm running youtube-dl 2018.08.04

Before submitting an issue make sure you have:

[x ] At least skimmed through the README, most notably the FAQ and BUGS sections
[x ] Searched the bugtracker for similar issues including closed ones
[x ] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[x ] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u':ytwatchlater', u'-u', u'PRIVATE', u'-p', u'PRIVATE', u'-s', u'-j', u'--flat-playlist', u'--write-pages']
WARNING: Assuming --restrict-filenames since file system encoding cannot encode all characters. Set the LC_ALL environment variable to fix this.
[debug] Encodings: locale ANSI_X3.4-1968, fs ANSI_X3.4-1968, out None, pref ANSI_X3.4-1968
[debug] youtube-dl version 2018.08.04
[debug] Python version 2.7.9 (CPython) - Linux-3.14.0-kirkwood-tld-1-armv5tel-with-debian-8.10
[debug] exe versions: avconv 11.11-6, avprobe 11.11-6, ffmpeg 0.8.21-6
[debug] Proxy map: {}
WARNING: Unable to login: Invalid password
ERROR: This playlist does not exist.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 792, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 502, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 2801, in _real_extract
    _, playlist = self._extract_playlist('WL')
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 2316, in _extract_playlist
    raise ExtractorError(message, expected=True)
ExtractorError: This playlist does not exist.
...
<end of log>

Description of your issue, suggested solution and other information
Explanation of your issue in arbitrary form goes here.
Youtube fails to authenticate. Looks like the sign-in page has been redesigned again. Previous reference: https://github.com/rg3/youtube-dl/issues/12973

Got presented with verification on another desktop machine logging into gmail. Now the login on the linux box running the app above is able to login again.

Having much the same issue here. Been happening for the last couple of days.

I changed my youtube account to 2FA and back to normal password. That seems to solve the problem for 4 days. Since today have the "invalid password" error again.

Yeah, happening again today for me as well.

Seems like it depends on IP,
2 days ago i was in other country and had

Unable to login: Invalid password

today, from home, it "works": password is OK but

'Unable to submit TFA code: HTTP Error 500: Internal Server Error'

:D

Switched to cookie authentication instead of user/pass and it appears to be working.

How do you do this? is cookie authentication the same as .netrc?

https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-pass-cookies-to-youtube-dl

This report is still open, right? I'm getting the same "Invalid password" warning and videos are not being marked as watched. Version 2019.11.28, tested today.

I am experiencing this for about ~4 days. (maybe more, i run youtube-dl weekly)

I tried on different machines and OS's (Win10, Win7)
Tried with different WAN ip adresses.

Here is the verbose:
`
f:\vid>youtube-dl.exe -v -i -u *** -p *** -f "[height<=?800]" https://www.youtube.com/playlist?list=WL
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-i', '-u', 'PRIVATE', '-p', 'PRIVATE', '-f',
'[height<=?800]', 'https://www.youtube.com/playlist?list=WL']
[debug] Encodings: locale cp1254, fs mbcs, out cp857, pref cp1254
[debug] youtube-dl version 2019.11.28
[debug] Python version 3.4.4 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: none
[debug] Proxy map: {}
[youtube:watchlater] Downloading login page
[youtube:watchlater] Looking up account info
[youtube:watchlater] Logging in
WARNING: Unable to login: Invalid password
[youtube:watchlater] WL: Downloading webpage
ERROR: This playlist does not exist.
Traceback (most recent call last):
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpwy0zjf
mc\build\youtube_dl\YoutubeDL.py", line 796, in extract_info
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpwy0zjf
mc\build\youtube_dl\extractor\common.py", line 530, in extract
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpwy0zjf
mc\build\youtube_dl\extractor\youtube.py", line 3227, in _real_extract
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpwy0zjf
mc\build\youtube_dl\extractor\youtube.py", line 2730, in _extract_playlist
youtube_dl.utils.ExtractorError: This playlist does not exist.
f:\vid>
`

Checklist

[ x ] I'm reporting a broken site support
[ x ] I've verified that I'm running youtube-dl version 2019.11.28
[ x ] I've checked that all provided URLs are alive and playable in a browser
[ x ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[ x ] I've searched the bugtracker for similar issues including closed ones

Verbose log
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-f', 'best', '-u', 'PRIVATE', '-p', 'PRIVATE', '-v', 'https://youtu.be/J7msxRmoA5o']
[debug] Encodings: locale cp1251, fs mbcs, out cp1251, pref cp1251
[debug] youtube-dl version 2019.11.28
[debug] Python version 3.4.4 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-65616-g7cdef77
[debug] Proxy map: {}
[youtube] Downloading login page
[youtube] Looking up account info
[youtube] Logging in
WARNING: Unable to login: Invalid password
[youtube] J7msxRmoA5o: Downloading webpage
[youtube] J7msxRmoA5o: Downloading embed webpage
[youtube] J7msxRmoA5o: Refetching age-gated info webpage
ERROR: Content Warning
This video may be inappropriate for some users.
       Sign in to confirm your age
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpwy0zjfmc\build\youtube_dl\YoutubeDL.py", line 796, in extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpwy0zjfmc\build\youtube_dl\extractor\common.py", line 530, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpwy0zjfmc\build\youtube_dl\extractor\youtube.py", line 2148, in _real_extract
youtube_dl.utils.ExtractorError: Content Warning
This video may be inappropriate for some users.
       Sign in to confirm your age

Description
I can't login on YouTube and can't download video when authorization is required.
In google profile settings "Less secure app access" is "On".

From README.md:

Passing cookies to youtube-dl is a good way to workaround login when a particular extractor does not implement it explicitly. Another use case is working around CAPTCHA some websites require you to solve in particular cases in order to get access (e.g. YouTube, CloudFlare).

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://youtu.be/J7msxRmoA5o', '--cookies', 'cookies.txt', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2019.11.28
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: ffmpeg git-2019-12-17-bd83191, ffprobe git-2019-12-17-bd83191, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] J7msxRmoA5o: Downloading webpage
[youtube] J7msxRmoA5o: Downloading video info webpage
[debug] Default format spec: bestvideo+bestaudio/best
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[debug] Invoking downloader on 'https://r3---sn-vuxbavcx-5ui6.googlevideo.com/videoplayback?expire=1577207510&ei=dvIBXvbQKMOrgAf-vLOIBg&ip=<IP_ADDRESS>&id=o-A
KACdRm3OPKkNKz0bbHQVaGA7knBxEEbwaPbXQ0-T1Q4&itag=136&aitags=133%2C134%2C135%2C136%2C160%2C242%2C243%2C244%2C247%2C278&source=youtube&requiressl=yes&mm=31%2C29&m
n=sn-vuxbavcx-5ui6%2Csn-4g5ednle&ms=au%2Crdu&mv=m&mvi=2&pl=16&initcwndbps=785000&mime=video%2Fmp4&gir=yes&clen=13314429&dur=249.560&lmt=1541860492337570&mt=1577
185844&fvip=3&keepalive=yes&fexp=23842630&c=WEB&txp=5432432&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=ALg
xI2wwRAIgAeshPJZmKwY8zGY91KT2TXtQkJnLYPwbqTEioB2Zs0ACIB4hIUqXnzdC_i1Y5QyQPVnHWdkjWRq5sE3pCreKKVib&lsparams=mm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AHylm
l4wRAIgGWUsKC2SDvOJ44IvUohcli19qKzZGK8NA2_P1UsdzaECIEPOsbok8yY0cd1THEIcTVMjsXIyNtHcgPmAtB3Cl8qp&ratebypass=yes'
[download] Destination: MINE by Álvaro de la Herrán-J7msxRmoA5o.f136.mp4
[download] 100% of 12.70MiB in 00:19
[debug] Invoking downloader on 'https://r3---sn-vuxbavcx-5ui6.googlevideo.com/videoplayback?expire=1577207510&ei=dvIBXvbQKMOrgAf-vLOIBg&ip=<IP_ADDRESS>&id=o-A
KACdRm3OPKkNKz0bbHQVaGA7knBxEEbwaPbXQ0-T1Q4&itag=251&source=youtube&requiressl=yes&mm=31%2C29&mn=sn-vuxbavcx-5ui6%2Csn-4g5ednle&ms=au%2Crdu&mv=m&mvi=2&pl=16&ini
tcwndbps=785000&mime=audio%2Fwebm&gir=yes&clen=4203740&dur=249.601&lmt=1541861657916130&mt=1577185844&fvip=3&keepalive=yes&fexp=23842630&c=WEB&txp=5411222&spara
ms=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=ALgxI2wwRQIgcaaNeNQMgj8QRkqbO1vdyw6f8omDgeoxYlLLnOarXKsCIQD1mtq5t-wk5Z
f5llD5OgvGCN_TH6nA6J5WcKime0JZRA%3D%3D&lsparams=mm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AHylml4wRAIgGWUsKC2SDvOJ44IvUohcli19qKzZGK8NA2_P1UsdzaECIEPOsbok
8yY0cd1THEIcTVMjsXIyNtHcgPmAtB3Cl8qp&ratebypass=yes'
[download] Destination: MINE by Álvaro de la Herrán-J7msxRmoA5o.f251.webm
[download] 100% of 4.01MiB in 00:06
[ffmpeg] Merging formats into "MINE by Álvaro de la Herrán-J7msxRmoA5o.mkv"
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:MINE by Álvaro de la Herrán-J7msxRmoA5o.f136.mp4" -i "file:MINE by Álvaro de la Herrán-J
7msxRmoA5o.f251.webm" -c copy -map "0:v:0" -map "1:a:0" "file:MINE by Álvaro de la Herrán-J7msxRmoA5o.temp.mkv"
Deleting original file MINE by Álvaro de la Herrán-J7msxRmoA5o.f136.mp4 (pass -k to keep)
Deleting original file MINE by Álvaro de la Herrán-J7msxRmoA5o.f251.webm (pass -k to keep)
``

@bitraid Thank you, it works.
But previously youtube-dl worked perfectly without this trick.

SUGGESTION
Change youtube-dl so that it's crystal-clear that login is failing.
It's very easy to skim past WARNING: Unable to extract result entry and think that the ERROR: This playlist does not exist. is the real problem.


Remove delegates

Closes #188
It's finally here! This is also breaking so we should release whatever we already have if we want them available under 1.x.

This was pretty simple and straightforward, I think most of the work will be improving a bit how we detect whether the packages are hosted on github and where. This still uses homepage, I believe to mimic what delegates were doing. We don't need to do this anymore!

We need to fix Cmdliner's warning first!

I'll rebase on #429 so we can see the CI!

#429 has been merged and we released 1.6.1 with the fix since some people needed these fixes.


ARROW-13249: [Java][CI] Consistent timeout in the Java JNI build

The JNI build gets stopped due to build timeout. Seems like the docker cache isn't valid anymore so it must build the docker image as well, but doesn't have the opportunity to push at the and of the build.

As you can see, after bumping the timeout a lot, it fails with another error. I would suggest sending an e-mail to the dev mailing-list about JNI CI, and if nobody answers, disable the build.

After using maven 3.5.4 the build work again. I'm going to reduce the build timeout to 90 minutes so it can rebuild the image on a cache miss.

@github-actions crossbow submit test-r-linux-valgrind


feat(password): add password confirmation

Description

Add a new password confirmation field.
Make the account page's password field, clearly meant to set a new password.

Motivation and context
Why is this change required? What problem does it solve?
If it fixes an open issue, please link to the issue here (if you write fixes #num
or closes #num, the issue will be automatically closed when the pull is accepted.)
How has this been tested?
Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran to
see how your change affects other areas of the code, etc.
Screenshots (if appropriate)
Types of changes
What types of changes does your code introduce? Put an x in all the boxes that apply:

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:
Go over all the following points, and put an x in all the boxes that apply.
Please, please, please, don't send your pull request until all of the boxes are ticked. Once your pull request is created, it will trigger a build on our continuous integration server to make sure your tests and code style pass.

[ ] I have read the CONTRIBUTING document.
[ ] My pull request addresses exactly one patch/feature.
[ ] I have created a branch for this patch/feature.
[ ] Each individual commit in the pull request is meaningful.
[ ] If my change requires a change to the documentation, I have updated it accordingly.

If you're unsure about any of these, don't hesitate to ask. We're here to help!

:tada: This PR is included in version 3.8.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Shared sub-directory permissions

Hey,
Thanks for sharing your envoy script, it's greatly helped me out.
I may have misunderstood but I need the storage and bootstrap/cache directories to have the www-data user.  I've set the service_owner_default to www-data but it doesn't change.
The 'syncshareddata_remotesrc' task looks like it should be changing the group, but it doesn't work.

what's the login user in your config file?
the 'syncshareddata_remotesrc' task will use {$tmpdir}/service_owner file content (which is your server connection login user,scp to remote server on 'rcp_env_to_remote' task )
the 'service_owner_default' settings will work only when the script can't find a login user on your server connection string config settings.

I see.  The login user is a deploy user which is fine for most of the files and directories.  I need to 'chown' the shared directories somewhere in the deployment to get them owned by www-data.

ok,if you need to set shared directories owner different from login user you can custom the script section 'syncshareddata_remotesrc' code:
if [ -e {{ $tmp_dir }}/service_owner ]; then
        service_owner=`cat {{ $tmp_dir }}/service_owner`;
    else
        service_owner="{{ $settings['service_owner_default'] }}";
    fi

into

service_owner="{{ $settings['service_owner_default'] }}";

and set
$settings['service_owner_default'] = 'www-data';

in your config file.

Thank you for your help.

I have the same issue and I made the change mentioned above and now I get a error:
$ rsync: chgrp "/var/www/{app_name}/shared/storage/." failed: Operation not permitted (1)
$ rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1183) [sender=3.1.1]


Error in Topology Check tool in ArcPro

I attempted to run the Topology Check tool in ArcPro and it fails to run due to these errors.

Is this an error on my end or within the script? How may I fix it?
Thank you

I am guessing you have some nulls in HierarchyKey? Is that true?

Oops. Yes I just discovered one. It's working now - sorry about that!


Update production keys on Ramp Network

I would merge it and then will update keys to production ones on nuxt
I believe we will migrate sooner to nuxt than Ramp will get back to me.

Originally posted by @yangwao in https://github.com/kodadot/nft-gallery/issues/6727#issuecomment-1748910720

Localhost:

done, still waiting to approve prod key from them


extend rule features and rename

closes #966

@ezbeat @jtothej please take a look

Thank you for taking my feedback into consideration.

Regarding the ProcessSignaturePolicy configuration, wouldn't it be clearer and more consistent to use policy.MicrosoftSignedOnly instead of policy.flags? This would enhance readability and align with the explicit naming convention used in other mitigation policies.

For the detection of blockdlls, rather than relying on the InitializeProcThreadAttributeList function, how about using the UpdateProcThreadAttribute function? This approach could directly focus on when the Attribute is actually set, potentially improving accuracy and reducing false positives.

nice, great feedback here, thanks! does the rule now look good to you?

Looks good. Thank you for considering the suggestions!

Oh :facepalm: @jtothej already did all the work... Thanks, for noticing. I'll update accordingly.

Thanks, @Ezbeat (and @jtothej)!!


fix: duplicate proposals in stfc instrument sci proposal page

Closes: https://github.com/UserOfficeProject/issue-tracker/issues/1058
Description
This PR addresses an issue of duplicate proposals appearing in the STFC Instrument Scientist Proposal page.
Motivation and Context
This change is required to ensure that each proposal appears only once in the STFC Instrument Scientist Proposal page, thus eliminating confusion and improving user experience.
Changes
update stfc instrument sci data source
How Has This Been Tested?

Depends On

Tests included/Docs Updated?

[ ] I have added tests to cover my changes.
[ ] All relevant doc has been updated

This issue might be covered with the changes in the following PR https://github.com/UserOfficeProject/user-office-core/pull/575. It reduces a lot of redundant code in the proposal view.


fix eval run circular import

Current dependencies on/for this PR:

main

PR #45   👈

This stack of pull requests is managed by Graphite.

Merge activity

Feb 3, 2:55 PM EST: @willseltzer merged this pull request with Graphite.


Update REStFul api and GraphQL to handle list of feature values

Update REStFul api and GraphQL to handle list of feature values for bulk prediction

Merge to develop


Warning IsNull is deprecated

When I build after cloning the repository, I get the following warning in JetBrains Rider 2022.3.2 :
ALSXTAnimationInstance.cpp(14): [C4996] 'TObjectPtr<AALSXTCharacter>::IsNull': IsNull is deprecated, please use operator bool instead.  if (!MyObjectPtr) { ... } Please update your code to the new API before upgrading to the next release, otherwise your project will no longer compile.

Thanks for that, it doesn't appear in Visual Studio Pro 2019. Fixed in 3e845e4edf879431a4a9d89ef1afe4aa009217a9