Update sbt to 1.3.13

Updates org.scala-sbt:sbt from 1.3.10 to 1.3.13.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-sbt", artifactId = "sbt" } ]

labels: library-update, semver-patch

Codecov Report

Merging #585 into master will decrease coverage by 0.23%.
The diff coverage is 69.23%.

@@            Coverage Diff             @@
##           master     #585      +/-   ##
==========================================
- Coverage   78.88%   78.65%   -0.24%
==========================================
  Files         122      122
  Lines        4012     4053      +41
  Branches       90       95       +5
==========================================
+ Hits         3165     3188      +23
- Misses        847      865      +18

Impacted Files
Coverage Δ

...in/scala/com/ing/baker/baas/interaction/Main.scala
0.00% <0.00%> (ø)

...src/main/scala/com/ing/baker/baas/state/Main.scala
0.00% <0.00%> (ø)

.../scala/com/ing/bakery/clustercontroller/Main.scala
0.00% <0.00%> (ø)

...lustercontroller/controllers/BakerController.scala
88.67% <91.66%> (-0.46%)
:arrow_down:

.../clustercontroller/controllers/BakerResource.scala
100.00% <100.00%> (ø)

...rcontroller/controllers/ControllerOperations.scala
89.79% <100.00%> (+3.52%)
:arrow_up:

...controller/controllers/InteractionController.scala
92.78% <100.00%> (+0.73%)
:arrow_up:

...rotomappings/SensoryEventStatusMappingHelper.scala
76.92% <0.00%> (-15.39%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b931be7...142c084. Read the comment docs.

Done in #587


"About" says no API calls, but rates makes an API call

About text: "currency converter library that doesn't require subscribing to any API calls."
index.js line 225:
return got(`https://www.google.co.in/search?q=${this.currencyAmount}+${this.currencyFrom}+to+${this.currencyTo}`)

The whole point of using a library like this for many consumers would be that it doesn't make external network calls, and your "About" misleads about that, requiring developers to find out the hard way.
😕

this api not subscribing payment required

About text: "currency converter library that doesn't require subscribing to any API calls."
index.js line 225:
return got(`https://www.google.co.in/search?q=${this.currencyAmount}+${this.currencyFrom}+to+${this.currencyTo}`)

The whole point of using a library like this for many consumers would be that it doesn't make external network calls, and your "About" misleads about that, requiring developers to find out the hard way.
😕

this api not subscribing payment required

this api not subscribing payment required

That's inconsequential. When you say "subscribing to API calls" I read "it doesn't make API calls", not that it doesn't cost money: "subscribing" has a connotation in programming that a consumer just opens an eternal listening connection. Many people will make the same assumption I did.

not subscribing payment required

okay bro i will change my description.
thanks for your suggestion

this api not subscribing payment required

That's inconsequential. When you say "subscribing to API calls" I read "it doesn't make API calls", not that it doesn't cost money: "subscribing" has a connotation in programming that a consumer just opens an eternal listening connection. Many people will make the same assumption I did.

okay bro i will change my description.
thanks for your suggestion

Thank you! 🙌


[Adaptive Card Templating][Adaptive card rendering issue if link has a space character]

Platform
What platform is your issue or question related to? (Delete other platforms).

JavaScript

Author or host
Host - Microsoft Search
What version are you using? Ex: NuGet 1.0.2, or latest master, etc...
Latest master
Explanation:

If a textblock is linked to a URL value that has space, the URL isn't formed correctly.
For example, if the text of textblock is as follows: [{Title}](http://{Projectname}.com)
If ProjectName is bound to a value with space (test url),

The link is not rendered properly and the adaptive card renders as above

@dclaux Could you please take a look at this issue?

Great find here @RamyaRamanathan28 ! @paulcam206 can you see if you can investigate to see if this is one of the known inconsistency issues you've been tracking at all?
Btw, @RamyaRamanathan28 curious about your consumption model of our renderer on Search. Is there a reason you take a dependency on our source in master vs npm released packages?

#5853 is tracking markdown issues -- will track progress there.


Working Draft missing descriptions of acoustic-emission properties and potential mismatch in property names in schema

Bug report
Following input from Alex James I drilled down into the latest working draft and found some omissions/possible mismatches.  These are described below:
In the report Clause <IP_ADDRESS>.2, Table 16 we tabulate “Properties of each Operating mode” and list “acoustic_emissions” there.  BUT in the schema under “operating_modes”  in the list of “required” items “acoustic_emisions” does not appear.  As I write I recall we made this optional.
In Table 16 we reference Table 19 of <IP_ADDRESS>.5 where we have Properties of acoustic emissions – but I cannot find description of
“margin”
“weighting”
“frequency” or
“sound_pressure_level” properties.
It looks like the schema I am looking at on Github has some disconnects with the working draft on rows 1376 potentially through 1560.
What is the expected behavior?
We need to have the labeling harmonized between the schema and the working draft, plus ensure the working draft has all the properties/sub properties used in the schema.

We're getting slightly tripped over here by something I've done to reduce duplication in the schema.
As acoustic emissions can be specified as 1/3, full, or total, we'd have to repeat the definitions of margin, weighting, wind_speed and frequency over and again, duplicating those definitions.
So, what we do in the schema is we $define a term for each of those, which is inlined into the oneOf where needed.
So our oneOf looks like this:

                "oneOf": [
                  {
                    "title": "Emissions at one-third-octave intervals",
                    "properties": {
                      "margin": {
                        "$ref": "#/$defs/acoustic_emissions_margin"
                      },
                      // ... and the other properties for one-third intervals
                  },
                  {
                    "title": "Emissions at full-octave intervals",
                    "properties": {
                      "margin": {
                        "$ref": "#/$defs/acoustic_emissions_margin"
                      },
                      // ... and the other properties for full intervals
                  },
                  {
                    "title": "Total emissions",
                    "properties": {
                      "margin": {
                        "$ref": "#/$defs/acoustic_emissions_margin"
                      },
                      // ... and the other propoerties for total intervals
                  }
                ]

Where you see things like:
                        "$ref": "#/$defs/acoustic_emissions_margin"

that's an instruction to in-line the common definition for margin. It means "take the definition at $defs/acoustic_emissions_margin (which is here in the scheme and insert it. This definition looks like this:

    "acoustic_emissions_margin": {
      "title": "Margin",
      "description": "A recommended margin of error [dB] which may be added to the given sound pressure levels for practical use.",
      "type": "number",
      "examples": [0, 2],
      "minimum": 0
    },

So the schema is functionally equivalent to the following, except if we need to update the definition we can do so in one place rather than several, and we can keep the schema compact too:

                "oneOf": [
                  {
                    "title": "Emissions at one-third-octave intervals",
                    "properties": {
                      "margin": {
                         "title": "Margin",
                         "description": "A recommended margin of error [dB] which may be added to the given sound pressure levels for practical use.",
                         "type": "number",
                         "examples": [0, 2],
                         "minimum": 0
                      },
                      // ... and the other properties for one-third intervals
                  },
                  {
                    "title": "Emissions at full-octave intervals",
                    "properties": {
                      "margin": {
                         "title": "Margin",
                         "description": "A recommended margin of error [dB] which may be added to the given sound pressure levels for practical use.",
                         "type": "number",
                         "examples": [0, 2],
                         "minimum": 0
                      },
                      // ... and the other properties for full intervals
                  },
                  {
                    "title": "Total emissions",
                    "properties": {
                      "margin": {
                         "title": "Margin",
                         "description": "A recommended margin of error [dB] which may be added to the given sound pressure levels for practical use.",
                        "type": "number",
                        "examples": [0, 2],
                        "minimum": 0
                      },
                      // ... and the other propoerties for total intervals
                  }
                ]

The path of the reference is purely referential and not part of the schema. What's happened at some point though is that these references to acoustic_emissions_margin have got placed incorrectly into the document. So I've tidied up the property names in the standard draft. No changes to teh schema itself are required.

Tom,
Thanks for the explanation. I have accepted all your changes.
I did find you left the definition of SPL locked - can you unlock it please?
Excited to have finished this phase.  It will be interesting to see the feedback we get.
Probably speak again in a few months
Thanks for all the hard work
Emil
From: Tom Clark @.>
Sent: Thursday, September 19, 2024 8:05 AM
To: octue/power-curve-schema @.>
Cc: Moroz, Emil @.>; Author @.>
Subject: Re: [octue/power-curve-schema] Working Draft missing descriptions of acoustic-emission properties and potential mismatch in property names in schema (Issue #74)
We're getting slightly tripped over here by something I've done to reduce duplication in the schema.
As acoustic emissions can be specified as 1/3, full, or total, we'd have to repeat the definitions of margin, weighting, wind_speed and frequency over and again, duplicating those definitions.
So, what we do in the schema is we $define a term for each of those, which is inlined into the oneOf where needed.
So our oneOf looks like this:
            "oneOf": [

              {

                "title": "Emissions at one-third-octave intervals",

                "properties": {

                  "margin": {

                    "$ref": "#/$defs/acoustic_emissions_margin"

                  },

                  // ... and the other properties for one-third intervals

              },

              {

                "title": "Emissions at full-octave intervals",

                "properties": {

                  "margin": {

                    "$ref": "#/$defs/acoustic_emissions_margin"

                  },

                  // ... and the other properties for full intervals

              },

              {

                "title": "Total emissions",

                "properties": {

                  "margin": {

                    "$ref": "#/$defs/acoustic_emissions_margin"

                  },

                  // ... and the other propoerties for total intervals

              }

            ]

Where you see things like:
                    "$ref": "#/$defs/acoustic_emissions_margin"

that's an instruction to in-line the common definition for margin. It means "take the definition at $defs/acoustic_emissions_margin (which is here in the schemehttps://github.com/octue/power-curve-schema/blob/2b95e3b853d5f994ad06686e530a156caf298471/power-curve-schema/schema.json#L342 and insert it. This definition looks like this:
"acoustic_emissions_margin": {

  "title": "Margin",

  "description": "A recommended margin of error [dB] which may be added to the given sound pressure levels for practical use.",

  "type": "number",

  "examples": [0, 2],

  "minimum": 0

},

So the schema is functionally equivalent to the following, except if we need to update the definition we can do so in one place rather than several, and we can keep the schema compact too:
            "oneOf": [

              {

                "title": "Emissions at one-third-octave intervals",

                "properties": {

                  "margin": {

                     "title": "Margin",

                     "description": "A recommended margin of error [dB] which may be added to the given sound pressure levels for practical use.",

                     "type": "number",

                     "examples": [0, 2],

                     "minimum": 0

                  },

                  // ... and the other properties for one-third intervals

              },

              {

                "title": "Emissions at full-octave intervals",

                "properties": {

                  "margin": {

                     "title": "Margin",

                     "description": "A recommended margin of error [dB] which may be added to the given sound pressure levels for practical use.",

                     "type": "number",

                     "examples": [0, 2],

                     "minimum": 0

                  },

                  // ... and the other properties for full intervals

              },

              {

                "title": "Total emissions",

                "properties": {

                  "margin": {

                     "title": "Margin",

                     "description": "A recommended margin of error [dB] which may be added to the given sound pressure levels for practical use.",

                    "type": "number",

                    "examples": [0, 2],

                    "minimum": 0

                  },

                  // ... and the other propoerties for total intervals

              }

            ]

The path of the reference is purely referential and not part of the schema. What's happened at some point though is that these references to acoustic_emissions_margin have got placed incorrectly into the document. So I've tidied up the property names in the standard draft. No changes to teh schema itself are required.

Reply to this email directly, view it on GitHubhttps://github.com/octue/power-curve-schema/issues/74#issuecomment-2361276713, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BBDWUHMZR6XS5JDKIPVBQXTZXLR2DAVCNFSM6AAAAABNM2GPACVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRRGI3TMNZRGM.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>This e-mail may contain privileged or confidential information. If you are not the intended recipient: (1) you may not disclose, use, distribute, copy or rely upon this message or attachment(s); and (2) please notify the sender by reply e-mail, and then delete this message and its attachment(s). UL Solutions Inc. and its affiliates disclaim all liability for any errors, omissions, corruption or virus in this message or any attachments.


DxActionSheet - The dxi-item option doesn't work correctly

The DevExtreme Angular package version is 16.2.1-alpha.3
See an example

This behavior is by design. Since you are using the dxi-item configuration components to specify the action sheet items, their content is considered as a template. That's why only text is rendered. If you'd like buttons to be rendered instead of text, you have two options:

Since your items are static, you can specify them via the 'items' attribute.
You can use the 'dx-button' components inside the 'dxi-item' component as a template.


Airdrop hunter address, one person and one address created 11 multi-signatures,Here are 11 of them I reported

Related Safe Addresses
0x84ae80c522c2a0fdb26e63f2ae527755d0000b81
0x006b1baa7469adf48fa632e9efc00635d77ea1f5
0x006d1194256a0754b489dc9b8596a4c4a721cb9c
0x01c439abc0bb34d3cc10279e150cde4afda19642
0x066419eaef5de53cc5da0d8702b990c5bc7d1ab3
0x06874c6f62209508115593d90c9b03110baec64a
0x06cb966d1de30caa6f8845dd94e4bde9ece56d4d
0x06ea07c21c7e85608267466fc03b368f752ca102
0x07944147cd1d166179edc226ee0201ef0bdb32ca
0x081d84829b1b8f8a46cce0392c524604ee785508
0x09c065b2895a7e9614289684d66bb05d41494e72

Reasoning
Starting from February 2021, 0x6860f6d1F8829FD8dBE9Bf20FC5B5922a46Bd647 will provide similar amounts of funds to all addresses (1-2). The first transaction at all addresses is the same and completed within the same hour. Other polygon activities are very similar, and each multi signed address has been airdropped. I have reason to prove that it was done by the same person
Methodology
For example, as I said in the previous article,I use Google bigquery to build a database, which contains all transactions from qualified airdrop addresses to metamask router on eth. Then, I began to box the data to try to find irregular behaviors until the group appeared. I'm pretty sure this is the witch.
Safe Address
0x6D524D88aAa1377cd16141C175650d1c5a67F75c

Starting from February 2021, 0x6860f6d1F8829FD8dBE9Bf20FC5B5922a46Bd647 will provide similar amounts of funds to all addresses (1-2). The first transaction at all addresses is the same and completed within the same hour.

I'm unable to follow, pls elaborate.

Other polygon activities are very similar,

Checked first 2 addresses, they made no tx on polygon.
0x84ae80c522c2a0fdb26e63f2ae527755d0000b81
0x006b1baa7469adf48fa632e9efc00635d77ea1f5

We haven't received any further information on this ticket and therefore deemed this submission invalid.
Unfortunately just the pure fact that Safes...

were created by the same address
have the same owner
conducted the same transactions
received assets from the same address

... is not enough reasoning for a valid submission, as there are legit user behaviour leading to such as well (relayer, scripts, power users, exchanges)
This submission didn't provide sufficient indicators to convince the team that there is some targeted airdrop farming behind the provided addresses. As per the rules, when in doubt we will default in favor of the airdrop recipients in order to not risk any false positives when sorting out airdrop hunters.


Change how buttons are injected in the folder page.

This makes the process more robust for some other layout changes.

Is there going to be an additional PR for the itemList.js and adding the convert to DIVE button for Video files in an itemList?

Yes, in a short bit.


Toast: missing properties in documentation

I'm submitting a...

[x] Bug report
[ ] Feature request
[ ] Documentation issue or request

Current behavior
Documentation of the toast properties seems to be missing.
https://ix.siemens.io/docs/controls/toast

In this case we need a better documentation for the toast-utils (https://github.com/siemens/ix/blob/main/packages/core/src/components/toast/toast-utils.ts)
This implementation is different for each framework
angular: https://github.com/siemens/ix/blob/main/packages/angular/src/toast/toast.service.ts
react: https://github.com/siemens/ix/blob/main/packages/react/src/toast/toast.ts

As discussed with ux/ui we use tabs under an "how to use" section:

/cc @goncalosard @nuke-ellington


Changing map to vector in Robot.h in storing links and joints

Use vector to store links and joints.
Do a linear search in requesting a joint/link by name.

@yetongumich?

Ping @yetongumich

Closing due to no response. As detailed in my above comment, a map's constant time access is preferred instead of searching through a vector.


problem with datepicker in complex form demo

hello,
i am using:
https://github.com/g00fy-/angular-datepicker
when clicking on the date field the complete modal window is removed. no exceptions are thrown.
to reproduce change the complex demo by setting for one of the form fields:
<input type="text" class="form-control" id="name"    ng-model="dt.date" 		    date-time date-time format="DD.MM.YY" view="date" min-view="date" 											auto-close="true"  											name="start" required="true" ng-pattern="/^(\d{2}).(\d{2}).(\d{2})$/" 											>

I think this might be an issue with the datepicker library, as it is seemingly only present when this library is included.


Question: Regarding autoscaling logic

Can someone explain the criteria and logic regarding autoscaling.
The documentation isn't very precise regarding that.

How and when exactly scale up is happening
How and when exactly scale down is happening
How and when does scale to zero happen
How and where to change/modify the criteria regarding scaling?

I have set autoscaling to 0-5 for a rio-demo application. However, nothing happens  if I hammer it with load or there are no inbound request going in.
If someone can explain that to me, I will update the docs accordingly with the criteria.

@Vad1mo Autoscaling is based calculating in-flight requests of your application. It scales according to the number of in-flight requests and concurrency of your pods(concurrency means how much in-flight request your pod can handle each single time).
If your pod has concurrency of 10 and total in-flight requests are 60, the pod will be scaled to 6. Since the metric is changing and based on real stats, the math won't be that exact but It will scale the pod around that number. Also as the in-flight requests come down the pod will also be scaled down. Once autoscaler observes that within a certain amount of time there is no request, it will scale down the pod to zero.
For your issue:
Can you describe the steps to reproduce it? Also you can check autoscaler logs by doing rio -s logs autoscaler

Speaking about in-flight does it refer to inbound requests only or are outbound requests calculated as well?
Suppose I have a batch job like service that receives its order via the API and does some long-running operations but does call internal URL and external API. Will the service be shutdown or will it live as long as it is active and processes some in/out requests?

@Vad1mo It supposes to take only In-bound requests at this moment. But it should be easy to configure both in-bound and out-bound requests too.

Could you point me to the code that does the calculation.
I found something but it is outside of rio https://github.com/rancher/rio/blob/cdb631d12eb49dda1efefb2440a866c99231e1f9/modules/autoscale/controller/service/controller.go#L100

Once autoscaler observes that within a certain amount of time there is no request, it will scale down the pod to zero.
Can you tell me what the timeframe is after which scaledowns should happen?

And, perhaps a bit OT, how do people determine how many in-flight requests a container could handle? for example a container running nginx serving static files can handle an enormous amount of in-flight requests, another container may get into resource struggle pretty fast.


v1.1.0 ndi.rotate documentation – reused parameters not filled in

In the v1.1.0 docs for ndimage.rotate, the "reused"/predefined parameters have not been replaced with their actual definitions; they still read as follows:
%(input)s
angle : float
    The rotation angle in degrees.
axes : tuple of 2 ints, optional
    The two axes that define the plane of rotation. Default is the first
    two axes.
reshape : bool, optional
    If `reshape` is true, the output shape is adapted so that the input
    array is contained completely in the output. Default is True.
%(output)s
order : int, optional
    The order of the spline interpolation, default is 3.
    The order has to be in the range 0-5.
%(mode)s
%(cval)s
%(prefilter)s

Thanks @kne42. Looks like a clear oversight, a @_ni_docstrings.docfiller got misplaced somewhere.
I've marked this as a good first issue for a new contributor - example of how it should be is ndimage.shift.

fixed in gh-8938


[To rel/0.12] change sync version check

change version checking rule for sync process
before:

Check full version number between sender and receiver.
Stop sync process if full version number does not match

Now:

Check major version number between sender and receiver
Stop sync process if major version number does not match

Cherry pick to master branch.

got it


[ja] fix term "repository" variations

fixes #45039

Hi  @tamilselvan1102, first, I deeply apologize that we cannot merge this PR. Our localization rules have been updated by #45037, and we have decided to change the notation to "リポジトリ"
Therefore, let me close this PR as "not planned." Regards
/close not-planned

/close


Python 'inout' parameter support - #44

This should solve the issue.

Hi @nchokoev,
thank you for pull request. Once i will be again focused on eRPC i will review your commit.

Hi @nchokoev , it looks like you fixed issue only in encoder. Do you think decoder is working correctly?

Hi @Hadatko,
I think it's ok, bit it's been a long time. I have to double check.

Thank you for your PR @nchokoev but i did new implementation in #237.


linear memory via mmap

linear memory is located in virtual memory o process.
growing the linear memory may require to re-alloc complete array and copying from old to new, larger one.
Instead mmap should be used to reserve the virtual linear memory,  mapping it directly onto physical memory pages; this way linear could grow without re-alloc, but just mapping additional physical memory pages into the consecutive mmap-memory address space.

Hi, sure, this is the issue we have in wasmi interpreter!
https://github.com/paritytech/wasmi/issues/45


Hardware donation

I'm having trouble maintaining support for all cams because I have very few models available. Especially older ones that are no longer available for purchase.
If you have cameras that you don't use anymore (maybe broken) and you want to get rid of them, please send to me.

Can you provide a list of cams you need to continue to support such models?
Or vice versa, a list of the cams that you have.
The destination country would be Spain?

About allwinner I have just a y20ga.

Good idea.
I will do it.

Please do so, even though I already donated a few euros last year, I would like to support you in the further development of this great mod and ideally extend the device support.


Add new commands: closestWithAttribute and closestWithAttributeAndValue

Proposed changes
Add two new commands for SelenideElement - closestWithAttribute and closestWithAttributeAndValue.
There's an existing command closest that will locate an ancestor with a given tag or class. That's not enough in some cases. With modern UI frameworks (React, for example), classes are often autogenerated and the same class appears on many elements. For example:

First div element

Second div element

If I want to find a container element of the #first-div element, then closest() won't help. The container element doesn't have a unique and reliable class name to search for. The tag name is also not unique, all elements are divs, so the closest() will find the first div.
Thus, two new commands will search for an ancestor by the attribute and attribute's value. In comparison to classes, elements could have already some custom attributes or the one can add a unique attribute that is needed for his/her test automation purposes. This way, these commands will help to find a target ancestor and users won't need to build up complex and erroneous XPath expressions in their projects.
Checklist

[x] Checkstyle and unit tests are passed locally with my changes by running gradlew check chrome_headless firefox_headless command
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have added necessary documentation (if appropriate)

I understand people do need to find elements that are hard to find :)
But it seems to me that there might be a plenty of such methods. In the end, it will be hard for people to find the right method among all these "find*", "closest*", "nearest*", "parent*", "grandparent*", "sibling*" etc. In the end, is it really simpler than just using xpath?
Compare:

$("#first-div"). closestWithAttributeAndValue("test-attribute", "test-value")
$x("//*[@id='first-div']//anccesstor::*[@test-atribute='test-value']")

The xpath implementation is even shorter. :)
And if you write a wrong xpath, IDEA will highlight it.
I don't dislike this feature, just not sure. Open for discussions.
P.S. Probably we should just introduce some nice API for building XPath in Java code?

@asolntsev I understand your concern. New commands and conditions indeed may get lost when there are many of them
My motivation for this was to simplify working with XPath. If the one doesn't construct XPaths that often in his project, then it will take him some time to make up the one that will find the target element. XPath's syntax is not that obvious, so it might be confusing whether it should have a // or / or ./ when I call a find(By.xpath) on the element and so on...

Thank you for PR!
What about this. Could you instead of adding two new methods to API, just extend the functionality the closest(String) method in this way:
closest("[test-attribute]") - locates the first up-tree element containing the attribute
closest("[test-attribute=test-value]") - locates the first up-tree element containing the attribute with value
I thinks @asolntsev will approve this solution.

@vinogradoff @asolntsev yes, I can extend the existing closest() as you propose.
The very first idea of how to do it is to check if the string starts with [ and doesn't contain =, then it is just the attribute. If it does contain the =, then everything after that is a value. But I see two concerns here. What if the attribute name will contain the =? Yes, it might be weird to have an attribute containing a = character, but who knows? :)
And second, if the user will forget to surround the attribute with square brackets? Selenide will think it's a regular tag.
Also, not sure about the value wrapped in single or double quotes...
What do you think?

I don't think "="
allowed in attribute names. If user forgets to close - nothing will be found, right? So no problem too. If you throws an exception
In case attribute is malformed - its okay too

@vinogradoff ok, I will update this PR and will submit the updated implementation.

@asolntsev @vinogradoff ready for review. Changed as per Alexei's comments to enhance the existing closest() method to handle attribute and attribute value. I decided to implement this using the rule engine pattern. Because there are 4 conditions (tag, class, attr, attr w/ value), then the straightforward approach with if-else wouldn't look good. With the rule engine it looks clean and would be very easy to enhance the method further should we add another way to find the closest element.

@plagov Ooops, it seems we have conflicts with master branch. Would it be hard to resolve those?

@asolntsev I see that #1567 PR has updated the former GetClosest class and the current implementation is not compatible with my implementation. I will have to refactor my PR in order to incorporate new changes. But before doing this, could you please review the current implementation and give some feedback on whether it's good or not and if it is suitable to go forward with it? If all good, then I will continue and will resolve the merge conflict.
Just would like to get some feedback and wouldn't want to waste time on future work if something is not OK already at this point :)

@plagov sorry for the inconvenience.

@asolntsev I've resolved all merge conflicts. Ready for review.


Where can I get the @types for @carbon/ibmdotcom-react?

Weighing in; Wondering if this is about TypeScript types or something else.

@hadnet Did you find the information you need?  Can we close this issue?

Closing this issue.


Observable._deferUnregister generates synchronous behavior in BabylonNative causing event callbacks to get dropped when Observable.addOnce is used

Repro
It's unclear whether this repros when using Babylon.js in a browser. It does repro in BabylonNative builds using babylon.max.js. To recreate:

Make multiple subscriptions to WebXRSessionManager.onXRSessionInit using addOnce function
Call WebXRSessionManager.onXRSessionInit.notifyObservers

Expected result:
All subscriptions see event callbacks fired. addOnce subscribers are correctly removed after seeing a single event callback.

Current result:
addOnce subscribers are synchronously removed from the Observable._observers array when still iterating over the Observable._observers array. Observable._deferUnregister effectively reduces the Observable._observers array length while iterating over the array (It's unclear whether this is expected behavior or if there is an assumption that deferred unregister should remove observers after finishing iterating over the Observable._observers array).

In babylon.max.js this causes the array length to change, prematurely terminating the for loop that iterates over the Observable._observers array. This then causes some observer callbacks to not fire.
Desktop (please complete the following information):

OS: Windows
Browser: BabylonNative
Version: BabylonNative master branch

This behavior has been observed for both x64 and ARM64 UWP flavors for windows mixed reality headsets and hololens 2.

@RaananW this is related to what we were seeing here:
https://github.com/BabylonJS/Babylon.js/pull/9307#discussion_r514570845

Happy to help drive a fix for this issue. It would be great if Babylon.js maintainers could call out what the expected behavior is for Observable._deferUnregister

Do you repro outside of native?

addOnce subscribers are synchronously removed from the Observable._observers array when still iterating over the Observable._observers array. Observable._deferUnregister effectively reduces the Observable._observers array length while iterating over the array

This is the expected behavior

So I'm answering myself here: This is an issue related to babylonNative only I guess :)

I'm not super familiar with how setTimeout(0) is supposed to work. It seems like this has synchronous or parallel execution in BabylonNative for UWP/Chakra. I can also try testing this in Babylon.js. Would we expect different browsers to have different implementations/behaviors for setTimeout(0)?

So I've just checked on browsers and it works as expected (ie. it executes the code AFTER the loop)
@bghgary will know more but I feel like setTimeout was an issue at some point with BJN

Adding @syntheticmagus as well

Yes I feel it will be the case but just waiting for Gary or Justin to confirm ;)


Cannot select any text in any table cell

I'm submitting a ...  (check one with "x")
[x] bug report => search github for a similar issue or PR before submitting
[ ] feature request
[ ] support request => Please do not submit support request here

Current behavior
If i try to select the text content of a table cell by press and hold the right mouse button. It doest work. Double clicking the text content doesnt work either.
Expected behavior
See the text contents highlighted like this http://imgur.com/a/2Btlx
Debug
I realized if i do it programatically it doesnt work either.
var range = document.createRange();
range.selectNodeContents(node);
window.getSelection().removeAllRanges();
window.getSelection().addRange(range);

When i disabled css i could select it. So this is a css issue. To be more specific it is because of these lines: https://github.com/swimlane/ngx-datatable/blob/master/src/components/datatable.scss#L101-L105

This is a specific reason for this, though I can't remember now haha. Just override this in your CSS for your particular element.

@amcdnl i did that. Altho i specifically opened this issue to find out what feature will this break :D

Cool, no worries. I can't remember but I know I did it for a specific reason.

I am using ngx data table for my angular 2 development.
My table has select box
[selected]="selected"
[selectionType]="'cell'"
and I have removed the below code from datatable.scss.
-webkit-user-select: none;
-moz-user-select: none;
-ms-user-select: none;
-o-user-select: none;
user-select: none;
But I am still facing the same issue.

fixed.
refered this link https://github.com/swimlane/ngx-datatable/issues/491
thanks


Test Application Stacks Operator with OCP 4.3

We need to do some testing against OCP 4.3 for our next Application Stacks Operator release.

@edavidj @JusteenR Hopefully, we'll have the driver to test soon. In the meantime, please gather scenarios to test - especially the ones not covered by the automated tests for Application Stacks Operator. Thanks.


Toolchain tests

Codecov Report

Merging #26 (e33372b) into development (d493651) will increase coverage by 0.25%.
The diff coverage is n/a.

@@               Coverage Diff               @@
##           development      #26      +/-   ##
===============================================
+ Coverage        49.51%   49.76%   +0.25%
===============================================
  Files               76       76
  Lines             2791     2791
===============================================
+ Hits              1382     1389       +7
+ Misses            1409     1402       -7

Impacted Files
Coverage Δ

src/SMCE/Toolchain.cpp
71.76% <0.00%> (+3.52%)
:arrow_up:

include/SMCE/Toolchain.hpp
100.00% <0.00%> (+33.33%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d493651...e33372b. Read the comment docs.


Cannot use with pod

I add pod "ImageViewer" to pod file, make pod install and it installs. After that how to use ImageViewer class if there is no module to import and it does not see ImageViewer class?

Hey @NuVeS, sometimes that happens. Just import wherever you import UIKit. I'm taking this from iOS Readme Template
To get the full benefits import ImageViewer wherever you import UIKit
import UIKit
import ImageViewer

Sometimes Xcode won't recognize, so just press shift + option+command + K to clean the build.
If you prefer, you can also download and drop /ImageViewer folder in your project


ImmutableDB: use TempResourceRegistry in modifyOpenState

Fixes https://github.com/input-output-hk/cardano-node/issues/651
Consider the following scenario:

We open the ChainDB
Which opens the ImmutableDB
The background thread in the ChainDB that copies blocks from the VolatileDB
to the ImmutableDB is started
...
We have copied enough blocks such that we have to start a new epoch in the
ImmutableDB, calling:mkOpenState registry HasFS{..} index epoch tip existing = do
    eHnd <- allocateHandle $ hOpen (renderFile "epoch" epoch)     appendMode
    pHnd <- allocateHandle $ Index.openPrimaryIndex index epoch existing
    sHnd <- allocateHandle $ hOpen (renderFile "secondary" epoch) appendMode
    ...

...
Now we CTRL-C the node
The ResourceRegistry starts to release resources in LIFO order: It
releases the three handles of the current epoch. Note that it does this
before it releases the background thread that copies blocks to the
ImmutableDB. So there's a short time frame in which the handles are closed
and the background thread copying blocks hasn't been killed yet. In this
time frame, we get a FsIllegalOperation because we're trying to write
using a closed handle.

The solution is to not use the global registry for the per-epoch handles,
because they will be released before a thread using them is terminated. The
main reason these handles are stored in a ResourceRegistry is to avoid
leaking them before they have made it into the new ImmutableDB state. After
that point, they won't leak, as closing the ImmutableDB (a release action
registered in the global registry) will close them.
Instead, we use modifyWithTempRegistry to release them in case we encounter
an exception before we store them in the state.

LGTM. Is there any specific motive for untrackResource and the additional checks? Or is it more of a sanity checklist that all resources have made it to the final destination?

Exactly!

bors r+

bors r-

I should rebase on #1792 to be sure.

bors r+

bors r+

bors r+


chore: group build scripts inside scripts/build

This makes it easier to have re-usable functionality that we can
call/source from all scripts, like logic to determine the current
operating system, or to check for a dependency in the $PATH.
Signed-off-by: Juan Cruz Viotti<EMAIL_ADDRESS>

Should the .bat scripts from the windows directory also be moved into this common directory?

Should the .bat scripts from the windows directory also be moved into this common directory?

The plan is to get rid of all the .bat scripts and require users that want to build Etcher to install MinGW. The advantages are:

MinGW (as opposed to something like Cygwin), makes all the UNIX commands available from cmd.exe, so we can call it from the "Visual Studio" console.

Allows us to re-use the bash scripts, which is a big deal in terms of re-usability.

Allows us to ditch Batch, which has several limitations. The one that annoy me the most is that there is nothing similar to set -e, to make the script exit if a command fails, so the "best practice" seems to be to spam the whole thing with error checks.

Its available out of the box in Appveyor.

I'm ignoring .bat files in this PR, since I'll delete them completely once I get the Windows build up and running on bash.

The plan is to get rid of all the .bat scripts and require users that want to build Etcher to install MinGW.

Sounds awesome, as long as installing MinGW isn't too much of a big download & install burden?
OffTopic: Would it be possible to extend the MinGW approach, and use that instead of (or as an alternative to) Visual Studio? RUNNING-LOCALLY.md says that "Visual Studio Community 2013" is required, but I can't actually find the download link for that anywhere on the Microsoft site? Only "Visual Studio Community 2015"...

P.S. This branch now has conflicts.

Sounds awesome, as long as installing MinGW isn't too much of a big download & install burden?

I think its fine. A lot of developers running Windows have it installed already, and even if you have to, it takes 1/10 the time it takes to install Visual Studio :)

OffTopic: Would it be possible to extend the MinGW approach, and use that instead of (or as an alternative to) Visual Studio?

I don't think we'll be able to get rid of Visual Studio any time soon. Its a dependency of gyp, the build system shipped by npm to install native NodeJS modules.

RUNNING-LOCALLY.md says that "Visual Studio Community 2013" is required, but I can't actually find the download link for that anywhere on the Microsoft site? Only "Visual Studio Community 2015"...

Oh, weird. It seems that they are suddenly redirecting that link to VS2015. I'll find a direct one and update the guide.

I resolved the conflicts!

I resolved the conflicts!

Looks good, but I suspect this will need rebasing again after #939 gets merged ;-)

Yeah, I was expecting this when a PRed so many things at once, but git resolves renames very gracefully, so its not a big deal (luckily)!

For those scripts that are still platform-specific (e.g. scripts/build/electron-sign-app.sh) I wonder if it might make sense to add the platform into the filename (e.g. scripts/build/electron-sign-app-darwin.sh or scripts/build/electron-sign-darwin-app.sh) ?

Yeah, makes sense. What do you prefer: prefixing the os in the filename, adding it as a suffix, or something else?

Thinking about it, I think my preference would be to add the OS as a suffix. So we'd have:
scripts/build/electron-sign-app-darwin.sh
scripts/build/electron-sign-exe-windows.sh
scripts/build/electron-installer-debian-linux.sh

etc. (which also fits in well with the electron-configure-package-linux.sh filenames you already have!)

Done!

Can you try restarting the AppVeyor builds?

@lurch What's your resin email? I'll make sure you have access to the CI servers.

BTW, the Appveyor errors are legitimate ones after the rebase. I'll fix in a second...


It seems as though this doesnt work anymore?

Windows 10 Professional   20H2
Does it work with?  I am getting ampersand operator not allowed errors, and '<' operator is reserved for future use.
Thanks!!

Use PowerShell, and not PowerShell ISE. Also make sure to run it as admin.

Thanks for the suggestion.  I am definitely running it as administrator in and not in ISE.  I forgot to add, however, that I am in "audit mode" trying to amke these changes.
I did try several times and exactly the same result.
Thanks your your help!

So... Is this issue fixed or not? Please close it if it's been fixed.

No...its not.  See the response?
Thank you

Well, I don't believe this script is fully built to support Audit Mode. It's meant to be run after a Windows installation has been made.
What are you using Audit Mode for?

Okay then.


validate pipeline input when input equal null

HKube micro-service
Api server
Describe the bug
execute pipeline  when flowinput equal to "null "  getting message that flowinput was not found
Expected behavior
flowinput exist there for the pipeline should be executed
To Reproduce
execute the following pipeline
{
    "name": "bool",
    "nodes": [
        {
            "nodeName": "trueFalse",
            "algorithmName": "eval-alg",
            "input": [
<EMAIL_ADDRESS>            ],
            "extraData": {
                "code": [
                    "(input,require)=> {",
                    "const txt = input[0];",
                    "return txt;}"
                ]
            }
        }
    ],
    "flowInput": {
        "inputs": null
    },
    "experimentName": "main",
    "options": {
        "ttl": 3600,
        "batchTolerance": 80,
        "progressVerbosityLevel": "info"
    },
    "priority": 3
}

Known issues#477

tested  "systemVersion": "v1.3.27",


fix: fix systemd configs

Changes

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#1708
#1707  👈 (View in Graphite)
#1706
#1695
main

How to use the Graphite Merge Queue
Add the label merge-queue to this PR to add it to the merge queue.
You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has required the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.


How to import ed25519 keys?

I am trying to import PEM encoded ed25519 keys (these are derived from test vectors, not used in anything real)
-----BEGIN PRIVATE KEY-----
MFECAQEEIOpPW/6GlNi7dLe1lARjL9WWi3dO1UXoEN6cMqT7QZL0oAUGAytlcKEj
AyEAW6O5rG6Q6D7/zSWsTlihNlqeNaPTrl6we55NkLz3UG0=
-----END PRIVATE KEY-----
-----BEGIN PUBLIC KEY-----
MDQwDgYHKoZIzj0CAQYDK2VwAyIAAFujuaxukOg+/80lrE5YoTZanjWj065esHue
TZC891Bt
-----END PUBLIC KEY-----

But when I do importPrivateKey I get:
Error: Module: virgil/crypto. Error code: 7. Invalid format of the Public Key.
Module: virgil/crypto/mbedtls. Error code: -14848. PK - Elliptic curve is unsupported (only NIST curves are supported)

Is there some other way to create VirgilPrivateKey and VirgilPublicKey objects?

Hi @maxogden. Thanks for your time diving to our crypto products.
There are some issues with the private key.
-----BEGIN PRIVATE KEY-----
MFECAQEEIOpPW/6GlNi7dLe1lARjL9WWi3dO1UXoEN6cMqT7QZL0oAUGAytlcKEj
AyEAW6O5rG6Q6D7/zSWsTlihNlqeNaPTrl6we55NkLz3UG0=
-----END PRIVATE KEY-----

Used PEM title BEGIN PRIVATE KEY specifies that private key is encoded as PKCS#8 ASN.1 type PrivateKeyInfo:

PrivateKeyInfo ::= SEQUENCE {
    version                   Version,
    privateKeyAlgorithm       PrivateKeyAlgorithmIdentifier,
    privateKey                PrivateKey,
    attributes                [0]  IMPLICIT Attributes OPTIONAL }

But actually it is encoded as SEC 1 ASN.1 type ECPrivateKey:

ECPrivateKey ::= SEQUENCE {
     version        INTEGER { ecPrivkeyVer1(1) } (ecPrivkeyVer1),
     privateKey     OCTET STRING,
     parameters [0] ECParameters {{ NamedCurve }} OPTIONAL,
     publicKey  [1] BIT STRING OPTIONAL }

And as defined in the RFC 5915 PEM header and footer should be :
   -----BEGIN EC PRIVATE KEY-----
   -----END EC PRIVATE KEY-----

Nevertheless, Header and Footer replacement can not completely fix the issue, as curves ed25519 and curve25519 are not included to the SEC 1 standard and RFC 5915 and can not be represented as a NamedCurve defined in the RFC 5912.

Solution:

Use ASN.1 type PrivateKeyInfo or it's newer version OneAsymmetricKey defined in the RFC 5958 and referenced in the RFC 8410 for keys for Ed25519, Ed448, X25519, and X448.

Generate new ed25519 private keys via this library:

import { VirgilCrypto } from 'virgil-crypto';

const virgilCrypto = new VirgilCrypto();
const keyPair = virgilCrypto.generateKeys();

or by Virgil CLI.
Note, for inspecting ASN.1 DER and PEM encoded types, useful online tool can be used.

Ah that makes sense, my mistake. Thank you :)

Got it working
-----BEGIN PRIVATE KEY-----
MC4CAQAwBQYDK2VwBCIEIOpPW/6GlNi7dLe1lARjL9WWi3dO1UXoEN6cMqT7QZL0
-----END PRIVATE KEY-----

-----BEGIN PUBLIC KEY-----
MCowBQYDK2VwAyEAW6O5rG6Q6D7/zSWsTlihNlqeNaPTrl6we55NkLz3UG0=
-----END PUBLIC KEY-----

Ended up using pkcs8-pem for the private key and spki-pem for the public key.
import { composePrivateKey, composePublicKey } from 'crypto-key-composer'

function privateToPem(privateKey) {
  var decomposed = {
    format: 'pkcs8-pem',
    keyAlgorithm: { id: 'ed25519' },
    keyData: {
      seed: privateKey,
    },
  }
  return composePrivateKey(decomposed)
}

function publicToPem(publicKey) {
  var decomposed = {
    format: 'spki-pem',
    keyAlgorithm: { id: 'ed25519' },
    keyData: {
      bytes: publicKey,
    },
  }
  return composePublicKey(decomposed)
}


Property Decorators last argument is undefined

TypeScript Version:  4.0.2

Search Terms:  property decorator undefined the last argument
Code
function propertyDecorator(...args) {
  console.log(args);
}

class Test {
  @propertyDecorator
  name: string;
}

Expected behavior:
[ Test {}, 'name' ]
Actual behavior:
[LOG]: [ {}, "name", null ]
Playground Link: https://www.typescriptlang.org/play?ts=4.0.2#code/GYVwdgxgLglg9mABABwE52QU1VAngEUwjlQEMoSAKAOltNQHMBnALkVLFwG0BdASkQBvAFCJExMEzgAbTNWlwGlesz4BuYQF9hwiNNJMmiACqYmUIaMQABNBmx5CxMhVRWwpALaY251DDAGDU0gA
Related Issues: Not found

Just in case [LOG]: [ {}, "name", null ] looks like website bug. In browser console it correctly logs [{…}, "name", undefined].
/cc @orta

Thanks
Seems pretty legit, I've given the whole logging system an overhaul to handle more specific primitives in https://github.com/microsoft/TypeScript-Website/pull/1108
It'll either go live with the next prod website deploy (scheduled every Monday) or sometime before if I need to make a deploy.

@orta Sorry, but my reported issue is not about the website logging, but about the wrong number of decorator parameters.  PropertyDecorator should receive only two arguments, because of its core definition lib.es5.d.ts#L1415. But somehow we receive 3 parameters and the last one is undefined.

OK, so, I think I'll need @rbuckton to give this a look over.
The JS version of your code looks like this:
"use strict";
var __decorate = (this && this.__decorate) || function (decorators, target, key, desc) {
    var c = arguments.length, r = c < 3 ? target : desc === null ? desc = Object.getOwnPropertyDescriptor(target, key) : desc, d;
    if (typeof Reflect === "object" && typeof Reflect.decorate === "function")
        r = Reflect.decorate(decorators, target, key, desc);
    else
        for (var i = decorators.length - 1; i >= 0; i--)
            if (d = decorators[i])
                r = (c < 3 ? d(r) : c > 3 ? d(target, key, r) : d(target, key)) || r;
    return c > 3 && r && Object.defineProperty(target, key, r), r;
};
var __metadata = (this && this.__metadata) || function (k, v) {
    if (typeof Reflect === "object" && typeof Reflect.metadata === "function")
        return Reflect.metadata(k, v);
};
function propertyDecorator(...args) {
    debugger;
    console.log(args);
}
class Test {
}
__decorate([
    propertyDecorator,
    __metadata("design:type", String)
], Test.prototype, "name", void 0);

With d(target, key, r) being the call which passes the undefined through as the third arg. This can be switched out with an extra check for the undefined as the r:
-                r = (c < 3 ? d(r) : c > 3 ? d(target, key, r) : d(target, key)) || r;
+                r = (c < 3 ? d(r) : c > 3  && r !== undefined ? d(target, key, r) : d(target, key)) || r;

Then it will give a console log of [ {}, 'name' ].
I'm just not sure of the network effects of such a change.

This is intended behavior:

We call __decorate(decorators, F) for class decorators
We used call __decorate(decorators, target, key, Object.getOwnPropertyDescriptor(target, key)) for methods/accessors.
We now call __decorate(decorators, target, key, null) for methods/accessors (null indicates we should call
Object.getOwnPropertyDescriptor in the helper). This was a backwards compatibility feature in a release where we reduced the footprint of calls to __decorate.
We call __decorate(decorators, target, key) for methods/accessors/fields when targeting ES3, because ES3 has no Object.defineProperty.
We call __decorate(decorators, target, key, void 0) for fields when targeting ES5 or higher, because you can convert a field into an accessor by returning a PropertyDescriptor.

Run the following code in the playground and you'll see what I mean:
https://www.typescriptlang.org/play?ts=4.0.2#code/GYVwdgxgLglg9mABABwE52QU1VAngEUwjlQEMoSAKAOltNQHMBnALkVLFwG0BdASkQBvAFCJEqTFBCokgxADdSAGxCY2ARkQBfYTuGhIsBCnRYcBIiXIkATDTqNW7TrwEixxMEzhLM1JXAMlPTMfADcusLCEEqkTEyIACqYTFBCoogAAmgY2HiExGQUqDYZ2aZ5FoXWqBlgpAC2aoipqDBgDBFaQA
As esoteric as it is, if you check arguments.length in your decorator and only get 2, that essentially means your decorator is being evaluated by a TypeScript compiler targeting ES3, and returning a value from it will have no effect.

Here's a playground link for the same code targeting ES3:
https://www.typescriptlang.org/play?target=0&ts=4.0.2#code/GYVwdgxgLglg9mABABwE52QU1VAngEUwjlQEMoSAKAOltNQHMBnALkVLFwG0BdASkQBvAFCJEqTFBCokgxADdSAGxCY2AIgAWMdYgC+wg8NCRYCFOiw4CREuRIAmGnUat2nXgJFjiYJnCVMaiU4Bkp6Zj4AbkNhYQglUiYmRAAVTCYoIVFEAAE0DGw8QmIyClQHHPzLIptS+1QcsFIAWzVETNQYMAYYo19-QODQynTM6gKKPCxqZrbooA
And the output:
[LOG]: [ {}, "name" ]
[LOG]: undefined

Thanks, @rbuckton, now it's clear about the actual behavior. Lately, for ES3, to make a general decorator for any decorator type, we could define which type it has, with just arguments.length, but for the ES5+ we need to clean up the argument array first. It's a bit confusing but okay for me.
Nice job, glad to be helped!

Is there any way for the development environment to make the last parameter not undefined?

Is there any way for the development environment to make the last parameter not undefined?

No, not with the legacy decorator support provided by --experimentalDecorators. Fields do not have descriptors as they do not exist until after they are installed, such as when an instance is created. This is a capability that changed somewhat with native decorators (currently Stage 3), as a field decorator will receive a context object like any other decoration target, and are able to inject an initializer function. We are not likely to make further changes to legacy decorators at this time.


BUG: Minor bug with OAuth Connection buttons

This is not a major issue, but when you click on an Oauth Connection button on the API Integrations page and cancel the request and click on another OAuth button (or a nav button), it seems to retain the previous connection state resulting in the previous integration popup firing.
I haven't had a chance to dive into it, but seems to a be a issue with ConnectOAuthButton when it catches an error from a cancelled request -https://github.com/triggerdotdev/trigger.dev/blob/bf57430b6663fa62fc4a395d3aeb05104eae0d1b/apps/webapp/app/components/integrations/ConnectOAuthButton.tsx#L99-L107
See:
Screencast from 2023-02-03 13-40-18.webm

Thanks for the report, will definitely look into this ✨


IS_SDL_CONTEXT_ALIVE set to false when calling sdl::init twice

I don't think this is the correct behaviour as it'll panic when dropping the already created sdl context, asserting on the was_alive check?

Yep, you're right; this isn't the intended behavior.
Do you mind creating a pull request?


Latex command symbols

I'm trying to figure out how to include symbols in key entries on graphs.
For some reason the < > text symbols produce inverted ! ? symbols.
Normally I'd use /textgreater as a workaround but this gives an unrecognised control sequence.
Attempting to a math environment also seems to not work as it simply prints the $$s.

Hi, it's best to ask this question on the gle mailing list and post some example code.  The list address cab be found on the gle webpage.

Apologies, I've asked here because the GLE mailing lists show very little activity and require one to subscribe to post.
The issue I've mentioned is due to the Latex encoder GLE is using being fairly out of date:
https://tex.stackexchange.com/questions/2369/why-do-the-less-than-symbol-and-the-greater-than-symbol-appear-wrong-as
Does GLE do the Latex compile internally or is it looking somewhere else on the machine for this?

I believe it uses the external latex, but you might need to enclose it in \tex{}

use write "\tex{\textgreater}" greater than > and write "\tex{\textless}" for less than <


Update readme

@blink1073 Looks great. One small suggestion.

Thanks for the nudge @willingc, updated.

FYI for those reading this issue, the utils.loadObject call is the one that needs requirejs: https://github.com/jupyter/jupyter-js-services/blob/41de1089edf0bb7f75e2a70a5a53bf93771525d4/src/kernel.ts#L976

Looks like the examples work. Thanks!

Thanks @blink1073.


Fix recording removed links in edit history

Fixes #3546

Coverage increased (+0.04%) to 68.657% when pulling 15958cd4a31644e86f1967ef8c3c6c7c7598581b on fix-wire-history into 3a26c5cd651224f5c0a0366d184db050d7dc8ebd on master.


Issues with Natural Earth masks

I recently used the Natural Earth masks introduced in #488 for the first time and had very strange results with the following recipe:
datasets:
  - {dataset: MTE, project: OBS, tier: 2, type: reanaly, version: 1, start_year: 2000,  end_year: 2002}

preprocessors:
  mask:
    mask_landsea:
      mask_out: sea

diagnostics:

  diagnostic_1: &diag
    description: Preprocessor test diagnostic.
    variables:
      gpp:
        mip: Lmon
        preprocessor: mask
        field: T2Ms
    scripts: null

The original data looks like this:

However, the preprocessed cube is totally messed up, it looks like somehow half of the mask was applied twice:

I know that the file is already masked in the first place, but I need it in combination with other (unmasked) datasets and I think applying a mask on an already masked dataset should have (nearly) no effect. The original dataset has a 0.5° x 0.5° resolution.

@schlunma sorry for the delay in getting back - I am well baffled by the result of applying the NE mask - I have tested that with a whole bunch of files (masked and not) and it works fine for both CMIP and OBS models; it looks to me that data has a messed up lat-lon grid but I can't say for sure. Any way you can share the MTE data with me so I can replicate the issue and see what can be done? Cheers, man

@schlunma man, I downloaded the MTE file and replicated the weird behaviour you have seen yourself, checked the NE masks and their application - they are correct and the files have not been corrupted in the meantime - what baffles me is that in ncview when viewing the MTE data it appears that no rotation would be needed (rotation needed to overlap with the NE mask, since the NE mask is from -180 to 180) but when actually checking out the coordinates in iris they are 0-360 which would imply the need for rotation that is currently done in _mask.py as per standard; I even rigridded this file down to coarser grids and still the same behaviour; so I went on and played with the rotation in _mask.py and got it right by reverting the x-axis rotation in the opposite way (the general case for all the 'good' files: where x>180, return x-180, or not - whereas in this case where x<180, return x+180, or not) -- this means that the file's data is seen in a mirror even though the x-points are monotonic. I have absolutely no clue how to fix the data on this thing, but at least the NE mask is doing the job it's supposed to be doing. You or @jvegasbsc may figure out how to fix this cube :grin:

Found the same issues with two other observational datasets:

HWSD (cSoil data)
NDP (cVeg data)

So apparently this problem affects multiple datasets.

@schlunma sorry man, this has slipped through the cracks - the problem is that if data is rotated, Natural earth masks will always be applied in a wrong way - this in fact not an issue about the NE mask on its own but rather an issue with the way the data is placed in the lat-lon matrix which is wrong and needs to be fixed regardless of mask application or not

Hi @valeriupredoi, sorry to bother you again with this masking stuff, but I have some questions/problems :sweat_smile:
The MTE file looks totally fine to me, ncview, iris and ncdump all show longitudes of [0, 360] and latitudes of [-90, 90]. This is CMOR standard, right?
You said that the mask "lives" on a [-180, 180] grid. Then why do you only need to return lon - 180 if lon > 180 and not for all lons? Where in the code of that shift? In this line, I can only find a -360 shift:
https://github.com/ESMValGroup/ESMValTool/blob/2081007682800566d133af97649312a084886fc8/esmvaltool/preprocessor/_mask.py#L163

@schlunma sorry, bud, I was away at the Met Office yesterday and had only my phone with me :stuck_out_tongue: So the bit of code that alligns the cube's x-axis to the Natural Earth x-axis is indeed the one you linked:
x_p_180 = np.where(x_p >= 180., x_p - 360., x_p)   (1)

this transforms a [0, 360] axis to [-180, 180] one since the NE mask is on a [-180, 180] axis, then after this, it looks for the x.points from the cube in the NE's x.points.
The normal way of rearranging the cube's x-points (1) gives you a mesh row of x=[0, 1,...180, -180, -179,..., -1] which does not overlap with the MTE mask. What works for MTE is
x_p_180 = x_p - 180.   (2)

so this will give you a mesh row of x=[-180, -179,..., 179, 180], because the MTE data is rotated the other way around as all the other CMIP and normal OBS data. Even though you see longitude points are monotonic and from 0-360 you have to be very careful how the data is anchored on the grid, the 1D array of longitude points does not hold that information. So in this case the MTE data needs to be rotated horizontally by 180 degrees

this is visible in ncview when you view the data - see ERA and MTE - MTE you see the full projection without any wraparound, with ERA the continents get wrapped around

Thanks V. for the great explanation! The fix was actually way easier than I thought :smile::
class gpp(Fix):
    """Class to fix gpp."""

    def fix_data(self, cube):
        """
        Fix data for gpp.

        Data lives on lon = [0, 360] grid, but lon = 0 is East Asia, not
        Europe. To fix this, the data needs to be rotated.

        """
        lon_coord = cube.coord('longitude')
        lon_coord.points = lon_coord.points - 180.0
        cube = cube.intersection(longitude=(0.0, 360.0))
        return cube

Thanks for your help!

@schlunma that's awesome, sorry for being ambiguous on my last post - kickass, man :grin:


🎨 Support for vertex colours

Expected vs. Actual Behavior
We currently only support render materials. Vertex colours have also been requested!
https://speckle.community/t/mesh-colours-gh-to-blender/2931
https://speckle.xyz/streams/5e54e5bafb/commits/549b268b63

I've taken a look at implementing vertex colors ToSpeckle in Blender.
And I've run into another case where Blender aligns mesh attributes differently than our Mesh format expects.
Previously it was texture coordinates, this time it's vertex colors.
I fixed this previously by adding a AlignVerticesWithTexCoordsByIndex function to Mesh that restructures faces to map between blenders vertex instance indexing, and vertex indexing.
In order for me to add support for vertex colors in blender,
I may have to do the same thing now for colors. But I'm interested in looking at better ways we can handle this that doesn't require every connector to call this Align method

Just ran into this sending a mesh with vertex colors from Rhino > Blender.

So in the very beginning of the SpeckleBlender add-on, there was a provision for pulling in vertex colors into Blender. If a mesh object had a property colors the same length as its vertex list, then these would be translated into vertex colors in Blender. The list of colors would be encoded as a single int32 for every color, interpreted as 4 bytes (ARGB).
The function is still there: https://github.com/specklesystems/speckle-blender/blob/fbda0110cd53201bd4e57e605c055c53bf6c6cca/bpy_speckle/convert/util.py#L154
I unfortunately have been away from the party for a while, so I'm not sure if that has changed, or what the receiving code now looks like, but maybe that helps whoever oversees this now.

Closing this ticket as we are moving to Jira to reduce noise. You can access it there.


"libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown" when creating the new container

For some reason I need to rebuild the container and when I run the command docker run -it -d -p 2222:22 --gpus all --name csc4005 <IMAGE ID>, I meet the error shown as below:

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown.

My computer is win10 operating system and has an NVIDIA graphics card. The detailed information of nvidia-smi shown as below
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 537.42                 Driver Version: 537.42       CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                     TCC/WDDM  | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 2060      WDDM  | 00000000:01:00.0  On |                  N/A |
| N/A   58C    P8               6W /  80W |    524MiB /  6144MiB |     32%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

After I tried to reinstall the docker and update the NVIDIA driver the problem remains.
Thank you very much if anyone can help.

The image is from the command docker pull nvcr.io/nvidia/nvhpc:21.7-devel-cuda11.4-centos7

Hi torit0,
I have searched on the internet, but did not find a satisfactory solution for your problem.
From the error message, it seems that the shared object cannot be found on your computer, so I suppose it is due to the Windows system path. The following two are the system paths that I set on my Windows computer, just for your reference. Please have a check where the NVIDIA driver is installed in your computer, and have a try by setting the system path to see if it is works.
C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common
C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR

I checked the PATH in the system variable and found these two paths in the PATH list. Then I also tried to add them into the PATH from the user variable. After I reboot my computer the problem remains.
But still thanks for your help

The problem is solved after I reinstall the NVIDIA driver and the docker desktop. Thanks!


Home.jsx: Implement the home/landing page design as per the figma files

Design link
Hex Codes
Using the above design

Implement the design for the home page. For the features section, feel free to reach out to @sikehish for the text content.
Stick to the hex codes used in the design and ensure that it is responsive. If Navbar.jsx isn't implemented yet, then we would recommend you to take up that issue first. Navbar issue

Hi @sikehish @ghpranav @chandansgowda @ShivamMenda please assign me this issue under hacktoberfest2023.

I've assigned it to you. Good luck Sumit!

@sikehish as a part of hacktober fest 2023 , I wish to contribute to your website .Please assign this issue to me

@sikehish as a part of hacktober fest 2023 , I wish to contribute to your website .Please assign this issue to me

Hi Jinesh. Appreciate your enthusiasm, but for now it assigned to @sumit-kumar-jha1709 .

@sikehish can you provide the text content, under the Feature heading

@sumit-kumar-jha1709 Hi Sumit, you can stick to some dummy content for now.


Uncompiled Internal Functions

Overview
We use the contract sizer to analyze our contract size a lot. An issue we find is if internal functions are not been called in the contract, they will not be compiled into the contract, so it's hard to analyze the size of internal functions. This case does happen if I have a contract A with some internal functions, and then contract B inherits A and calls the internal functions.
Proposed Solution
Show how many internal functions that are igonre when compiling. I'm not sure whether it's doable. It's just an idea. Hope someone can propose better ideas or guidance.

I have a contract A with some internal functions, and then contract B inherits A and calls the internal functions.
In this case, the size of those internal functions will be reflected in the size of contract B.  That's really the best that one can hope for, because the concept of "size" of an individual function is not very meaningful.

I do like your suggestion to display rough data about skipped functions.  I would suggest generalizing the idea and also displaying counts of external functions and internal functions that are not skipped.
I'm not sure how to do this, however, as I've never worked with the compiler directly.  It's possible that there's a flag that will trigger output of this information.  If not, the source code might need to be analyzed directly.  The latter is possibly too slow and/or complex to be within the scope of this plugin, but could possibly be released as a separate plugin.
I would consider a PR if anyone knows a good way to do this.

Closing because I don't think this is possible.


Jupyter notebook support

When I try to view pandas dataframe using gtabview with jupyter notebook, it prints out like
<gtabview.ViewController at 0x1c3427715c8>
and output pops up, but then it freezes.
Is there a possible solution to use gtabview within jupyter notebook?

See the README about correct initialization for notebooks (you have to set DETACH before calling view).
Sadly I just tried on a current version of jupyter/pyqt5 and I cannot it to work (not sure yet what exactly changed in either jupyter or pyqt5).

I confirm this behaviour but with relatively large datasets only: 300x300 works fine, but 4200x12 doesn't.
Both in normal mode.
In DETACH=True mode I get the following error:
Exception in thread Thread-6:
Traceback (most recent call last):
  File "c:\python37-64\lib\threading.py", line 926, in _bootstrap_inner
    self.run()
  File "c:\python37-64\lib\site-packages\gtabview\__init__.py", line 64, in run
    APP = QtGui.QApplication.instance()
AttributeError: module 'PyQt5.QtGui' has no attribute 'QApplication'

The above error should be fixed in master, along with a fix for python 3.9. I think it's time for a new release soon...

I've been retesting this with jupyter, but it looks like something is preventing Qt to work at all inside a notebook.
When running in detached mode, the windows shows up, but doesn't seem to receive any event. A stack trace seems to reveal that python is stuck inside a selector loop and doesn't relinquish control until some asyncio event occurs.
In normal mode Qt is not even receiving events despite running a blocking event loop.
Incredibly annoying to debug due to the sheer size of both environments. I'm pushing this to the next release..

Looks like PyQt5 cannot be started from a separated thread like in earlier versions, which kinds of defeat the purpose of the "DETACH" flag. So we need assistance from the main program's loop to get the same.
I managed to get it working on jupyter by hooking up on the main asyncio's event loop, with polling... At least on X11 this can be made optimal through the QtX11Extras module, fetching the xcb connection descriptor, and waiting on the descriptor itself instead of waiting.
When using Matplotlib with the Qt backend it already handles the event loop for us, so we just rely on matplotlib itself for event dispatching.
On windows/mac I assume the polling approach will work, but I have no idea how this could be turned optimal

After further testing, it seems that Qt5 is also reading from the X socket with a separate thread, so the option of monitoring the socket is also busted, since events get skipped if the thread reads them before we finish...

I managed to test the fix in ipython and jupyter with both inline and separate plot windows on linux.
If somebody could test this under windows it would be great!


update recipe to include abs, prod endpoint, app insights, and blob

This needs to go to generators VSIX and Yeoman

we've done a sweep on samples and have integrated this with each appropriate sample


Please provide a way to list variants of dependencies

For example to know the variants for a Maven dependency you currently have to look into the docs, search for the right chapter and hope it is up-to-date, while some plugin could also have added more variants.
For projects that are built, there is outgoingVariants, but for some published library with or without Gradle Module Metadata there is currently (7.3.1) no way to list the variants that are available.

The workaround is to add the dependency to a dummy configuration, and Gradle would complain it can't tell which variant matches the best 😂


Move PR data to its own config

Moving the current states of any open PRs to the job config , or keeping it in the plugin config, isn't a very good option.   Several issues have been created based on the trigger saving those configs many times, and those same files are sometimes kept historically causing a lot of noise.
This PR moves that status into the build directory, which can then be filtered by change tracking systems.  This also lays the groundwork for multiple GitHub repositories to be handles by one job trigger.

@mmitche @daspilker @Eriten and @fstehle I believe this is the right solution, correct me if I am wrong.

Thank you for this pull request! Please check this document for how the Jenkins project handles pull requests.


NMN-149 Overhaul the UI & update branding

It's about time we introduce a massive rework to the UI to improve the visual & user experience of the site. We didn't have a lot of UI elements in previous iterations so we kept things simple to focus on the functionality. However, as we shift to the stage where we will need to implement more UI elements for features such as perks selection and other game events, we need to make future UI implementations more agile and maintainable. Other than the visible new UI and animations, this PR does the following underneath:

Adds re-usable UI components based on unstyled Radix UI primitives, which are also styled to match the theme of our site using Tailwind CSS
Drops anime.js in favor of Tailwind's animate plugin and Framer Motion

While anime.js is incredibly light, it was also very complicated to write animation sequences and handle exit animations. Framer Motion makes it much easier to handle animation flows while also adds GPU acceleration. While Framer handles our heavier animation with JS, we use Tailwind's animate plugin to write simpler CSS animations in our reusable components so we don't have to load Framer in every small components.
Framer does make our bundle size visibly bigger. However, this can be mitigated by lazy loading the features if needed.

Check out the preview!


Add Grammars

Fixes https://github.com/vllm-project/vllm/issues/1229
Implement incremental LALR / Regex parser to determine legal-next-token set.
Rendered documentation
TODO

InteractivePredictiveLALRParser

[x] Load Lark EBNF grammars
[x] Track terminals and state transitions
[x] Determine token validity whether it partially or wholly completes the terminals pattern

TokenTrie

[x] Efficiently retrieve candidate tokens

NextTokenValidator

[x] Update parser state
[x] Get valid token IDs given a tokenizer vocabulary and the completion-in-process' text

GrammarLogitProcessor

[x] Implement def __call__(self, token_ids, logits) , which updates the parsers state with new tokens and filters based on NextTokenValidator's valid token IDs

Testing

[x] Performance tests: see docs

[/] Documentation
[x] Clean up tests, right now they're all lurking in grammar.py
[x] Handle EOS token
[x] finish test cases

Ramblings from previous implementation:

Grammar Token Filter Algorithm
The GTF algorithm involves calculating the set of valid next-tokens given an incomplete sequence and a grammar.
Its core components are

A set of tokens, for which every token is either a single character or can be generated with a combination of other tokens
An interactive parser which can determine the validity of incomplete sequencess.

Lazy Approach
The simplest algorithm is
def get_valid_tokens(base_sequence, token_vocabulary):
    valid_tokens = set()
    for token in token_vocabulary:
	    if parser.is_valid_sequence(base_sequence + token)
	        valid_tokens.add(token)
    return valid_tokens

This approach has two core inefficiencies:

If the token "foobar" is valid, then we already know "foo" is valid, thus redundant work is performed
Parser applies state transitions for base_sequence once for every token

This PR's Approach
The current GTF algorithm improves on the lazy approach in two aspects:

token_vocabulary is a trie, allowing us to check "foo", and if invalid, we know "foobar", "foobaz", and "foobarbaz" are also invalid.
The parser is interactive, meaning it doesn't need to recalculate the base_sequence each time.

Current approach algorithm is a depth first search of the token trie with a base_sequence-warmed parser.
def get_valid_tokens(parser, token_trie, trie_root=""):
    valid_tokens = set()
    for token in token_trie.children(trie_root):
        if parser.is_valid_next_token(token):
            child_parser = parser.step_seq(token)
                valid_tokens.add(token)
                valid_tokens |= get_valid_tokens(child_parser, token_trie, token)
    return valid_tokens

The main weakness of this implementation involves regular expressions. If a terminal rule is a regular expression, an incomplete match must be searched for redundantly.
Optimal (Future) Approach
The optimal GTF approach involves all terminal rules being a single character. All terminals, including regular expressions must be decomposed.
For example the regular expression
\d{5}(-\d{4})?
Must be decomposed into
digit = "\d"
five_digits = digit, digit, digit, digit, digit;
four_digits = digit, digit, digit, digit;
optional_suffix = "-", four_digits;
zipcode = five_digits, [optional_suffix];

Additionally we can use a helper function legal_chars(character_expr) which retrieves all characters legal within a character regexp, e.g.

legal_chars("\d") = set(["0", "1", "2", ...])
legal_chars("[ae") = se["a", "e"]

With this optimization the GTF algorithm would be as follows:
def get_valid_tokens(parser, token_trie, token_trie_roots=None):
    valid_tokens = set()
    for next_terminal_expr in parser.get_next_terminals_exprs():
        parser_next_chars = legal_chars(next_terminal_expr)
        legal_token_prefixes = token_trie.children(token_trie_roots) | parser_next_chars
        if legal_token_prefixes:
            child_parser = parser.transition(next_terminal_expr)
            legal_token_suffixes = get_valid_tokens(child_parser, token_trie, legal_token_prefixes)
            valid_tokens |= trie.combine(legal_token_prefixes, legal_token_suffixes)
    return valid_tokens

This function requires only applying a state transition once for every transition which is legal within the token set. As opposed to the current implementation which applies a state transition once for each unique token trie node.
Breaking down into single character terminals provides another advantage: we don't have to recompute a regular expression partial redundantly, if foo matches (foo|bar)(bazbif), we don't need to recalculate the entire regex for foobaz again. In fact, we don't compute regular expresisons at all, we simply generate the valid character set for a given atomic character expression and intersect it with the tries valid token prefix set.
Example
I use a simple Thompson's-style regex to generate the eNFA dict via automata_toolkit.
Sample code which assigns random values to logits and generates a grammer-constrained completion:
    regexp = r"(large )?(language )((models )+(inference engines ))(are )((useful)+((very )*complex))."

    sample_from_logits = lambda lgts: np.random.choice(len(lgts), p=np.exp(lgts)/np.sum(np.exp(lgts)))

    for i in range(4):

        logit_processor = TokenConstraintLogitProcessor(
            tokenizer=tokenizer,
            nfa=EpsilonNFA(nfa=regex_to_nfa.regex_to_nfa(regexp)),
        )

        token_ids = []
        while True:
            logits = logit_processor(
                token_ids=token_ids,
                logits=np.random.uniform(-10, 10, len(tokenizer.vocab))
            )
            new_token_id = sample_from_logits(logits)
            token_ids.append(new_token_id)
            if new_token_id == tokenizer.eos_token_id:
                break
        print(f"run #{i}")
        print("\ttokenid", token_ids)
        print("\ttokens:", [tokenizer.decode(tok_id, ) for tok_id in token_ids])
        print("\tresult:", tokenizer.decode(token_ids, skip_special_tokens=False))

Output:
regexp: r"(large )?(language )((models )+(inference engines ))(are )((useful)+((very )*complex))."

Thanks for putting this together @lapp0 .
I managed to integrate it with the rest of the vllm and got legit outputs!
A minor change I made was that the return from  GrammarLogitProcessor.__call__ should be tensor, not a list.
I made a minor change to make it work, hope it helps.
N = len(self.tokenizer.vocab)
mask = torch.zeros(N, dtype=torch.bool)
valid = torch.tensor(valid_token_ids, dtype=torch.long)
mask[valid] = True
logits[~mask] = float('-inf')
return logits

Appreciate your review, fix, and interest @xuy. Will integrate that after I'm done with some bug fixes!

What needs to happen to get this grammar pull request approved and merged? I'd love to start using grammars with vLLM.

The code had a major bug - only single character tokens were being selected.
I just pushed a fix which makes the functionality stateless, caches the parser based on the state stack, and is a bit cleaner.

Currently validating 25 tokens per second on a single core of my laptop, indicating this would bottleneck for vLLM. Will try to optimize further.

Does this only work with the OpenAI API at the moment? If so, could it be added to the vllm api as well?

Works nicely so far. I noticed the preprocessing for batching being done on only one core and hence significantly stalling the process. Is that due to grammar implementation? And is there a way to fix that, to either use GPU or more than a single core?

@lapp0 Could you post your multiprocessing branch, even if its incomplete? I've been trying to implement it myself, but it seems I can't get it quite right.

@brucethemoose It's pretty poorly implemented, but here you go: https://github.com/lapp0/vllm/tree/grammar-multiprocessing
I've been working on integrating some of my caching techniques into https://github.com/outlines-dev/outlines which already has regex-based guidance for vLLM.

Tested the grammar support from your branch.
Added receiving the grammar parameter to ChatCompletionRequest as well, since I use an instruct fine-tuned model.
Model: TheBloke/deepseek-coder-33B-instruct-AWQ
System: "You are a helpful AI assistant. You give concise answers. If you do not know something, then say so."
User: "Write down the first 10 prime numbers as a comma separated list on a single line. Do not write anything else."
Without the grammar the model gives this reponse:
2, 3, 5, 7, 11, 13, 17, 19, 23, 29
So in the grammar I intentionally denied it to use white-space, so the expected output must be:
2,3,5,7,11,13,17,19,23,29
Grammar:
?start: SIGNED_NUMBER ( "," SIGNED_NUMBER)*
%import common.SIGNED_NUMBER

While it conforms to the grammar it fails the task:
2,3,5,7,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,

It may happen that the grammar code somehow denies it from writing 11 there, e.g. it cannot write a number of multiple digits.
Grammar support would be really awesome to have for my use case. I actually started implementing Lark support and already figured out the PR's algorithm in my tests outside vLLM when I found your PR. I'm really glad you had this much progress already, so there is a chance to have grammar support soon.
In llama.cpp there is a grammar named GBNF, which is an EBNF variant. That already works and its integration with the sampling code can give us some ideas on how to optimize this in vLLM.

When I change the grammar to allow whitespace, then it can generate the primes properly:
'2, 3, 5, 7, 11, 13, 17, 19, 23, 29'

Grammar:
?start: _WS? DIGIT+ ( _WS? "," _WS? DIGIT+ )* _WS?
%import common.DIGIT
%import common.WS -> _WS

It worked without code changes and could stop. I do not know why it cannot stop if no white-space is allowed. The model writes spaces only after the commas and no newline is generated at the end of completion.

@viktor-ferenczi
The parser doesn't handle ambiguous terminals well. Could you try converting them to a rule? Something along the lines of
    signed_number: ["+"|"-"] number
    number: float | int
    float: int exp | decimal exp?
    decimal: int "." int? | "." int
    exp: ("e"|"E")
    signed_int signed_int: ["+"|"-"] int
    int: DIGIT+ DIGIT: "0".."9"

And yes, the speed is bad. Outlines addresses this by precompiling the regex FSM and using Numba. I'm leaning heavily towards thinking vLLM should be a strong, simple inference engine and outlines should be a wrapper on top for grammars.
Outlines vLLM CFG implementation merged yesterday https://github.com/outlines-dev/outlines/pull/517

The grammar you suggested crashes vLLM with this exception:
TypeError: UnexpectedToken.__init__() missing 2 required positional arguments: 'token' and 'expected'

The traceback is useless because of the use of Ray (2 GPUs).
Performance: I was running vLLM with cProfile and executed the completion some 50 times in about 2 minutes. Found the grammar responsible for only ~550ms of CPU runtime, so I don't see from the profiling data where the experienced slowdown is. Grammar's CPU load is 60-70% of a core, so it does not seem to be CPU bound there. I guess the load does not show up on the CPU or the Python profiler, but introduced by the use of Tensor (GPU RAM access?) or similar. I don't know enough Torch and CUDA yet to tell exactly.

I'm leaning heavily towards thinking vLLM should be a strong, simple inference engine and outlines should be a wrapper on top for grammars.

Where would you put the grammar support? If we keep it inside vLLM, then it can be used via the REST APIs. That's what I prefer, at least for my use case. It allows for hosting the LLM separately from the application and better scalability, all without having to write a custom server for each application which require a grammar or forcing the application to run the LLM directly in-process.

The exception due to the grammar:
...
  File "/home/viktor/dep/vllm-contrib/vllm/model_executor/layers/sampler.py", line 155, in _apply_logits_processors
    logits_row = logits_processor(token_ids, logits_row)
  File "/home/viktor/dep/vllm-contrib/vllm/grammar.py", line 472, in __call__
    return ray.get(result_id)
... ray ...
ray.exceptions.RaySystemError: System error: Failed to unpickle serialized exception
... ray ...
TypeError: UnexpectedToken.__init__() missing 2 required positional arguments: 'token' and 'expected'

So it cannot relay Lark's UnexpectedToken error. It is also not handled properly and turned into a Bad Request error by the API Server, apparently.

Performance: I was running vLLM with cProfile and executed the completion some 50 times in about 2 minutes. Found the grammar responsible for only ~550ms of CPU runtime, so I don't see from the profiling data where the experienced slowdown is. Grammar's CPU load is 60-70% of a core, so it does not seem to be CPU bound there. I guess the load does not show up on the CPU or the Python profiler, but introduced by the use of Tensor (GPU RAM access?) or similar. I don't know enough Torch and CUDA yet to tell exactly.

Are you using multiple GPUs? I'm seeing a substantial slowdown when passing the tensors to the logits processor ray actor.

Where would you put the grammar support? If we keep it inside vLLM, then it can be used via the REST APIs. That's what I prefer, at least for my use case. It allows for hosting the LLM separately from the application and better scalability, all without having to write a custom server for each application or forcing the application to run the LLM directly in-process.

https://outlines-dev.github.io/outlines/reference/vllm/

@lapp0 Tried the outlines.serve.serve way. The JSON schema and Regex work, but the grammar (cfg) does not. See the outlines bug report on this. Also, that solution does not work with tensor parallel at all (see bug ticket). It looks like everything is implemented, just not reliable yet.

There are already libraries actively maintained for guided generation that can integrate with vLLM, like Outlines. I would be wary of introducing code that is tangentially related to this library and will require a substantial amount of maintenance when this can be solved by an import. Why not contribute this code to these libraries and import them here instead?
http://outlines-dev.github.io/outlines/reference/vllm/

@jqueguiner The custom logits processors need some more information to be passed to avoid having to patch vLLM the hard way. Primary example is a way to identify the sequence (seq_id) and maybe more. Please look into the implementation of outlines.serve.serve, specifically _patched_apply_logits_processors.

The seq_id can probably be replaced with a hash of the token ids if that’s really the blocker. But that’s beside the point, even if we needed to pass seq_id, I agree with @jqueguiner that it’s an easier change for the vLLM team that requires substantially less maintenance over time.

@jqueguiner The custom logits processors need some more information to be passed to avoid having to patch vLLM the hard way. Primary example is a way to identify the sequence (seq_id) and maybe more. Please look into the implementation of outlines.serve.serve, specifically _patched_apply_logits_processors.

This experimental change where the state is managed by cached by the hash of the prior token ids is working for me so far:
https://github.com/outlines-dev/outlines/commit/8b1ff9a7863915f5fdee421d4cae8f0840d58b33#diff-f65ffb5f52b2e358c713ccb8f32a700769426c6c8b655f689e3cdccae07d22ac

A hash on preceding tokens is even better than seq_id, because it would allow for further optimization should prompts be repeated.

Hi everyone, thank you so much for the very active discussion here. As vLLM maintainer, I want to express my sincere thanks for your enthusiasm. vLLM as a project is focused on optimizing LLM inference and provide a fully compatible OpenAI API; constrained decoding is not our strong suit, and we don't have the expertise to maintain it.
@lapp0 would you be able to consider closing this PR and merge into outlines instead? I think you mentioned it here. I would very much like to use outlines directly in vLLM after #2488 is merged. (or sooner, adding it to completion API is another option).
@lapp0 and @viktor-ferenczi, please let us know what interface and scheduling change on the vLLM side is needed to better support this functionality.

Sure @simon-mo will follow up with you for any changes to vLLM which are necessary. Thanks for your enthusiastic support!
Closing in favor of outlines. A few changes necessary in outlines to consider grammars complete for vLLM:

https://github.com/outlines-dev/outlines/issues/524
https://github.com/outlines-dev/outlines/pull/539


Implement training with keras.js?

Given that we know the derivatives for most activation functions, are you planning to implement backpropagation for training the models and not just do inference?
I know Javascript doesn't have the speed of C++, but depending of the size of datasets, Javascript performance can be quite good compared to Python.

Seconded. I've got a project that was implemented with convnetjs because it needed to be run in a browser. It was trained on generated images for training data because no real-world data existed yet. The other benefit to in-browser training is that browsers are great at generating and rendering SVGs programmatically, which makes for a very convenient way to generate training data.
Unfortunately convnetjs is showing its age (no longer maintained) and doesn't have support for more recent developments in neural networks like ELU activation or BatchNormalization. Would be great to be able to switch to kera.js if the training capability was there.

I think web browser DO NOT have the ability to training the large datasets and model in one web session by the limiting of memory.
convjs is great but often dump in demos when the task is small (just a few layers).
Python is good because the backend (matrix computation and etc..) is based on CUDA.


Vite 4's bundle will run out of memory in browser

Describe the bug
I tried to upgrade to Vite v4 today, and the production build will crash on our site's specific page in BROWSER (unlike #2433 which is crash in the Node.js process). Because it's a private repo, I have no idea how to share a minimal reproducible example. So I'm also asking for advice about what could provide to help solve this issue.
I tried upgrading only vite to v4 and upgrading the following packages:
vite 3.2.5 -> 4.0.0
@vitejs/plugin-vue 3.2.0 -> 4.0.0
@vitejs/plugin-vue-jsx 2.1.1 -> 3.0.0

And both will crash in Chrome and Firefox.
Reproduction
N/A
Steps to reproduce
No response
System Info
System:
    OS: macOS 13.0.1
    CPU: (8) arm64 Apple M2
    Memory: 4.25 GB / 24.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 16.18.1 - ~/.asdf/installs/nodejs/16.18.1/bin/node
    Yarn: 3.3.0 - ~/.asdf/installs/nodejs/16.18.1/bin/yarn
    npm: 9.2.0 - ~/.asdf/plugins/nodejs/shims/npm
  Browsers:
    Chrome: 108.0.5359.98
    Firefox: 103.0.2
    Firefox Developer Edition: 108.0
    Safari: 16.1
  npmPackages:
    @vitejs/plugin-vue: 3.2.0 => 3.2.0
    @vitejs/plugin-vue-jsx: 2.1.1 => 2.1.1
    vite: 4.0.0 => 4.0.0

Used Package Manager
yarn
Logs
No response
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to vuejs/core instead.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

We cannot help without a reproduction.
Maybe it's same with #11330?

Thanks a lot. I tried it with vite 4.0.1 + rollup 3.7.4 and the issue disappear


⬆  Bump<EMAIL_ADDRESS>

⬆  Bump<EMAIL_ADDRESS>

arrow_up Bump<EMAIL_ADDRESS>
Affidavit